2024-08-19T01:17:04.011 INFO:root:teuthology version: 1.2.0 2024-08-19T01:17:04.036 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-08-19T01:17:04.130 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/teuthology-2024-08-18_21:24:02-fs-squid-distro-default-smithi/7860117 branch: squid description: fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate 3-modules} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/kclient/{base/{mount-syntax/{v2} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} ms_mode/legacy wsync/no} objectstore-ec/bluestore-bitmap omap_limit/10 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/random export-check n/3 replication/default} standby-replay tasks/{0-subvolume/{with-quota} 1-check-counter 2-scrub/yes 3-snaps/no 4-flush/yes 5-quiesce/no 6-workunit/suites/blogbench}} email: ceph-qa@ceph.com first_in_suite: false job_id: '7860117' kernel: client: branch: testing kdb: 1 sha1: distro ktype: upstream last_in_suite: false log-rotate: ceph-mds: 10G ceph-osd: 10G machine_type: smithi mgrmodules: sequential: - print: Enabling mgr modules name: teuthology-2024-08-18_21:24:02-fs-squid-distro-default-smithi no_nested_subset: false openstack: - volumes: count: 4 size: 20 - machine: disk: 200 os_type: centos os_version: 9.stream overrides: admin_socket: branch: squid ceph: cephfs: max_mds: 3 session_timeout: 300 standby_replay: true conf: client: client mount timeout: 600 debug client: 20 debug ms: 1 rados mon op timeout: 900 rados osd op timeout: 900 global: ms die on skipped message: false mds: debug mds: 20 debug mds balancer: 20 debug ms: 1 mds bal fragment size max: 10000 mds bal merge size: 5 mds bal split bits: 3 mds bal split size: 100 mds debug frag: true mds debug scatterstat: true mds op complaint time: 180 mds verify scatter: true mds_export_ephemeral_random_max: 0.1 osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mgr: cephsqlite lock renewal timeout: 900000 debug client: 20 debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon down mkfs grace: 300 mon op complaint time: 120 mon warn on pool no app: false osd: bdev async discard: true bdev enable discard: true bluestore allocator: bitmap bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug ms: 1 debug osd: 20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore osd op complaint time: 180 osd shutdown pgref assert: true osd_max_omap_entries_per_request: 10 flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - FS_DEGRADED - FS_INLINE_DATA_DEPRECATED - FS_WITH_FAILED_MDS - MDS_ALL_DOWN - MDS_DAMAGE - MDS_DEGRADED - MDS_FAILED - MDS_INSUFFICIENT_STANDBY - MDS_UP_LESS_THAN_MAX - filesystem is online with fewer MDS than max_mds - POOL_APP_NOT_ENABLED - do not have an application enabled - overall HEALTH_ - Replacing daemon - deprecated feature inline_data - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding - MON_DOWN - PG_AVAILABILITY - PG_DEGRADED - Reduced data availability - Degraded data redundancy - slow metadata IO - SLOW_OPS - slow request sha1: de09dd018de4baddfa6ca4d43bc8fb784cf75833 subvols: create: 2 subvol_options: --size 25000000000 ceph-deploy: bluestore: true conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} osd: bdev async discard: true bdev enable discard: true bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore fs: xfs ceph-fuse: client.0: mount_subvol_num: 0 check-counter: counters: mds: - mds.exported - mds.imported - mds.dir_split install: ceph: flavor: default sha1: de09dd018de4baddfa6ca4d43bc8fb784cf75833 kclient: client.0: mount_subvol_num: 1 mntopts: - ms_mode=legacy - nowsync syntax: v2 selinux: allowlist: - scontext=system_u:system_r:logrotate_t:s0 - scontext=system_u:system_r:getty_t:s0 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: squid sha1: 78215f5a1cf5eaf321d506a66539942e12851f34 owner: scheduled_teuthology@teuthology priority: 100 repo: https://git.ceph.com/ceph.git roles: - - mon.a - mgr.x - mds.a - mds.d - mds.g - mds.j - osd.0 - osd.3 - osd.6 - osd.9 - client.0 - - mon.b - mgr.y - mds.b - mds.e - mds.h - mds.k - osd.1 - osd.4 - osd.7 - osd.10 - - mon.c - mgr.z - mds.c - mds.f - mds.i - mds.l - osd.2 - osd.5 - osd.8 - osd.11 seed: 1664 sha1: de09dd018de4baddfa6ca4d43bc8fb784cf75833 sleep_before_teardown: 0 subset: 36/512 suite: fs suite_branch: squid suite_path: /home/teuthworker/src/git.ceph.com_ceph_78215f5a1cf5eaf321d506a66539942e12851f34/qa suite_relpath: qa suite_repo: https://git.ceph.com/ceph.git suite_sha1: 78215f5a1cf5eaf321d506a66539942e12851f34 syslog: ignorelist: - WARNING*.*check_session_state - WARNING*.*__ceph_remove_cap targets: smithi083.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBNxNuxkRUTXNGUfNiIHTF84bc+9Toq6X8bc5eMEc6JD2c0ULO7pmeJtbxg5RVSFKESywa1J2HLM7StZwIK05IZk= smithi117.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBDBk715dHkSulVVSMgPfIdbeXFNl/Qe7xdtB6lB54JXweM7aX6VP9nbbVCwP/1iS8sRtbosB1IEJyoVJu4LR95A= smithi167.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBCeS3olJtXBnWzRmpizLFFMMGydQJH0pX1AGnmSNEKLEQdGmoXnvILGdkdM6umxdpDT0jnDRF6E79wMzbMh5Sv0= tasks: - install: extra_packages: deb: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror rpm: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror extra_system_packages: deb: - bison - flex - libelf-dev - libssl-dev - network-manager - iproute2 - util-linux - dump - indent - libaio-dev - libtool-bin - uuid-dev - xfslibs-dev - postgresql - postgresql-client - postgresql-common - postgresql-contrib rpm: - bison - flex - elfutils-libelf-devel - openssl-devel - NetworkManager - iproute - util-linux - libacl-devel - libaio-devel - libattr-devel - libtool - libuuid-devel - xfsdump - xfsprogs - xfsprogs-devel - libaio-devel - libtool - libuuid-devel - xfsprogs-devel - postgresql - postgresql-server - postgresql-contrib - cephadm: roleless: false - cephadm.shell: mon.a: - ceph orch status - ceph orch ps - ceph orch ls - ceph orch host ls - ceph orch device ls - cephadm.shell: mon.a: - ceph fs dump - ceph osd dump - fs.ready: timeout: 300 - sequential: - mgrmodules - kclient: null - exec: mon.a: - ceph fs set cephfs balance_automate false - ceph fs subvolumegroup pin cephfs qa random 0.10 - check-counter: {} - fwd_scrub: scrub_timeout: 900 sleep_between_iterations: 1 - background_exec: mon.a: - while sleep 13; do ceph tell mds.cephfs:0 flush journal; done - workunit: clients: all: - suites/blogbench.sh teuthology: fragments_dropped: [] meta: {} postmerge: - local function is_kupstream() return yaml.ktype == 'upstream' end - local function is_kdistro() return yaml.ktype == 'distro' end - local function is_fuse() return false end - local function syntax_version() return yaml.overrides.kclient.syntax end teuthology_branch: main teuthology_sha1: 67db276914d29505c9707d04f3a3b8fb9d8aee4b timestamp: 2024-08-18_21:24:02 tube: smithi user: teuthology verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.3401856 2024-08-19T01:17:04.130 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/git.ceph.com_ceph_78215f5a1cf5eaf321d506a66539942e12851f34/qa; will attempt to use it 2024-08-19T01:17:04.131 INFO:teuthology.run:Found tasks at /home/teuthworker/src/git.ceph.com_ceph_78215f5a1cf5eaf321d506a66539942e12851f34/qa/tasks 2024-08-19T01:17:04.131 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-08-19T01:17:04.132 INFO:teuthology.task.internal:Checking packages... 2024-08-19T01:17:04.154 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'default' and ceph hash 'de09dd018de4baddfa6ca4d43bc8fb784cf75833' 2024-08-19T01:17:04.154 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-08-19T01:17:04.154 INFO:teuthology.packaging:ref: None 2024-08-19T01:17:04.154 INFO:teuthology.packaging:tag: None 2024-08-19T01:17:04.154 INFO:teuthology.packaging:branch: squid 2024-08-19T01:17:04.154 INFO:teuthology.packaging:sha1: de09dd018de4baddfa6ca4d43bc8fb784cf75833 2024-08-19T01:17:04.155 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=squid 2024-08-19T01:17:04.383 INFO:teuthology.task.internal:Found packages for ceph version 19.1.0-1386.g78215f5a 2024-08-19T01:17:04.384 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-08-19T01:17:04.391 INFO:teuthology.task.internal:no buildpackages task found 2024-08-19T01:17:04.391 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-08-19T01:17:04.397 INFO:teuthology.task.internal:Saving configuration 2024-08-19T01:17:04.416 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-08-19T01:17:04.423 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-08-19T01:17:04.448 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi083.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/teuthology-2024-08-18_21:24:02-fs-squid-distro-default-smithi/7860117', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9.stream', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-08-19 01:10:07.911348', 'locked_by': 'scheduled_teuthology@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBNxNuxkRUTXNGUfNiIHTF84bc+9Toq6X8bc5eMEc6JD2c0ULO7pmeJtbxg5RVSFKESywa1J2HLM7StZwIK05IZk='} 2024-08-19T01:17:04.471 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi117.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/teuthology-2024-08-18_21:24:02-fs-squid-distro-default-smithi/7860117', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9.stream', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-08-19 01:10:07.909065', 'locked_by': 'scheduled_teuthology@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBDBk715dHkSulVVSMgPfIdbeXFNl/Qe7xdtB6lB54JXweM7aX6VP9nbbVCwP/1iS8sRtbosB1IEJyoVJu4LR95A='} 2024-08-19T01:17:04.494 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi167.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/teuthology-2024-08-18_21:24:02-fs-squid-distro-default-smithi/7860117', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9.stream', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-08-19 01:10:07.912458', 'locked_by': 'scheduled_teuthology@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBCeS3olJtXBnWzRmpizLFFMMGydQJH0pX1AGnmSNEKLEQdGmoXnvILGdkdM6umxdpDT0jnDRF6E79wMzbMh5Sv0='} 2024-08-19T01:17:04.494 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-08-19T01:17:04.501 INFO:teuthology.task.internal:roles: ubuntu@smithi083.front.sepia.ceph.com - ['mon.a', 'mgr.x', 'mds.a', 'mds.d', 'mds.g', 'mds.j', 'osd.0', 'osd.3', 'osd.6', 'osd.9', 'client.0'] 2024-08-19T01:17:04.501 INFO:teuthology.task.internal:roles: ubuntu@smithi117.front.sepia.ceph.com - ['mon.b', 'mgr.y', 'mds.b', 'mds.e', 'mds.h', 'mds.k', 'osd.1', 'osd.4', 'osd.7', 'osd.10'] 2024-08-19T01:17:04.501 INFO:teuthology.task.internal:roles: ubuntu@smithi167.front.sepia.ceph.com - ['mon.c', 'mgr.z', 'mds.c', 'mds.f', 'mds.i', 'mds.l', 'osd.2', 'osd.5', 'osd.8', 'osd.11'] 2024-08-19T01:17:04.501 INFO:teuthology.run_tasks:Running task console_log... 2024-08-19T01:17:04.589 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f68e8d10b80>, signals=[15]) 2024-08-19T01:17:04.590 INFO:teuthology.run_tasks:Running task internal.connect... 2024-08-19T01:17:04.596 INFO:teuthology.task.internal:Opening connections... 2024-08-19T01:17:04.596 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi083.front.sepia.ceph.com 2024-08-19T01:17:04.597 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi083.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-08-19T01:17:04.679 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi117.front.sepia.ceph.com 2024-08-19T01:17:04.680 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi117.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-08-19T01:17:04.755 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi167.front.sepia.ceph.com 2024-08-19T01:17:04.755 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi167.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-08-19T01:17:04.829 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-08-19T01:17:04.836 DEBUG:teuthology.orchestra.run.smithi083:> uname -m 2024-08-19T01:17:04.854 INFO:teuthology.orchestra.run.smithi083.stdout:x86_64 2024-08-19T01:17:04.855 DEBUG:teuthology.orchestra.run.smithi083:> cat /etc/os-release 2024-08-19T01:17:04.913 INFO:teuthology.orchestra.run.smithi083.stdout:NAME="CentOS Stream" 2024-08-19T01:17:04.913 INFO:teuthology.orchestra.run.smithi083.stdout:VERSION="9" 2024-08-19T01:17:04.913 INFO:teuthology.orchestra.run.smithi083.stdout:ID="centos" 2024-08-19T01:17:04.913 INFO:teuthology.orchestra.run.smithi083.stdout:ID_LIKE="rhel fedora" 2024-08-19T01:17:04.913 INFO:teuthology.orchestra.run.smithi083.stdout:VERSION_ID="9" 2024-08-19T01:17:04.913 INFO:teuthology.orchestra.run.smithi083.stdout:PLATFORM_ID="platform:el9" 2024-08-19T01:17:04.914 INFO:teuthology.orchestra.run.smithi083.stdout:PRETTY_NAME="CentOS Stream 9" 2024-08-19T01:17:04.914 INFO:teuthology.orchestra.run.smithi083.stdout:ANSI_COLOR="0;31" 2024-08-19T01:17:04.914 INFO:teuthology.orchestra.run.smithi083.stdout:LOGO="fedora-logo-icon" 2024-08-19T01:17:04.914 INFO:teuthology.orchestra.run.smithi083.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-08-19T01:17:04.914 INFO:teuthology.orchestra.run.smithi083.stdout:HOME_URL="https://centos.org/" 2024-08-19T01:17:04.914 INFO:teuthology.orchestra.run.smithi083.stdout:BUG_REPORT_URL="https://issues.redhat.com/" 2024-08-19T01:17:04.914 INFO:teuthology.orchestra.run.smithi083.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-08-19T01:17:04.914 INFO:teuthology.orchestra.run.smithi083.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-08-19T01:17:04.915 INFO:teuthology.lock.ops:Updating smithi083.front.sepia.ceph.com on lock server 2024-08-19T01:17:04.941 DEBUG:teuthology.orchestra.run.smithi117:> uname -m 2024-08-19T01:17:04.961 INFO:teuthology.orchestra.run.smithi117.stdout:x86_64 2024-08-19T01:17:04.961 DEBUG:teuthology.orchestra.run.smithi117:> cat /etc/os-release 2024-08-19T01:17:05.019 INFO:teuthology.orchestra.run.smithi117.stdout:NAME="CentOS Stream" 2024-08-19T01:17:05.019 INFO:teuthology.orchestra.run.smithi117.stdout:VERSION="9" 2024-08-19T01:17:05.019 INFO:teuthology.orchestra.run.smithi117.stdout:ID="centos" 2024-08-19T01:17:05.019 INFO:teuthology.orchestra.run.smithi117.stdout:ID_LIKE="rhel fedora" 2024-08-19T01:17:05.019 INFO:teuthology.orchestra.run.smithi117.stdout:VERSION_ID="9" 2024-08-19T01:17:05.020 INFO:teuthology.orchestra.run.smithi117.stdout:PLATFORM_ID="platform:el9" 2024-08-19T01:17:05.020 INFO:teuthology.orchestra.run.smithi117.stdout:PRETTY_NAME="CentOS Stream 9" 2024-08-19T01:17:05.020 INFO:teuthology.orchestra.run.smithi117.stdout:ANSI_COLOR="0;31" 2024-08-19T01:17:05.020 INFO:teuthology.orchestra.run.smithi117.stdout:LOGO="fedora-logo-icon" 2024-08-19T01:17:05.020 INFO:teuthology.orchestra.run.smithi117.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-08-19T01:17:05.020 INFO:teuthology.orchestra.run.smithi117.stdout:HOME_URL="https://centos.org/" 2024-08-19T01:17:05.020 INFO:teuthology.orchestra.run.smithi117.stdout:BUG_REPORT_URL="https://issues.redhat.com/" 2024-08-19T01:17:05.020 INFO:teuthology.orchestra.run.smithi117.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-08-19T01:17:05.020 INFO:teuthology.orchestra.run.smithi117.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-08-19T01:17:05.021 INFO:teuthology.lock.ops:Updating smithi117.front.sepia.ceph.com on lock server 2024-08-19T01:17:05.041 DEBUG:teuthology.orchestra.run.smithi167:> uname -m 2024-08-19T01:17:05.060 INFO:teuthology.orchestra.run.smithi167.stdout:x86_64 2024-08-19T01:17:05.061 DEBUG:teuthology.orchestra.run.smithi167:> cat /etc/os-release 2024-08-19T01:17:05.119 INFO:teuthology.orchestra.run.smithi167.stdout:NAME="CentOS Stream" 2024-08-19T01:17:05.120 INFO:teuthology.orchestra.run.smithi167.stdout:VERSION="9" 2024-08-19T01:17:05.120 INFO:teuthology.orchestra.run.smithi167.stdout:ID="centos" 2024-08-19T01:17:05.120 INFO:teuthology.orchestra.run.smithi167.stdout:ID_LIKE="rhel fedora" 2024-08-19T01:17:05.120 INFO:teuthology.orchestra.run.smithi167.stdout:VERSION_ID="9" 2024-08-19T01:17:05.120 INFO:teuthology.orchestra.run.smithi167.stdout:PLATFORM_ID="platform:el9" 2024-08-19T01:17:05.120 INFO:teuthology.orchestra.run.smithi167.stdout:PRETTY_NAME="CentOS Stream 9" 2024-08-19T01:17:05.120 INFO:teuthology.orchestra.run.smithi167.stdout:ANSI_COLOR="0;31" 2024-08-19T01:17:05.120 INFO:teuthology.orchestra.run.smithi167.stdout:LOGO="fedora-logo-icon" 2024-08-19T01:17:05.120 INFO:teuthology.orchestra.run.smithi167.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-08-19T01:17:05.120 INFO:teuthology.orchestra.run.smithi167.stdout:HOME_URL="https://centos.org/" 2024-08-19T01:17:05.120 INFO:teuthology.orchestra.run.smithi167.stdout:BUG_REPORT_URL="https://issues.redhat.com/" 2024-08-19T01:17:05.121 INFO:teuthology.orchestra.run.smithi167.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-08-19T01:17:05.121 INFO:teuthology.orchestra.run.smithi167.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-08-19T01:17:05.121 INFO:teuthology.lock.ops:Updating smithi167.front.sepia.ceph.com on lock server 2024-08-19T01:17:05.141 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-08-19T01:17:05.152 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-08-19T01:17:05.158 INFO:teuthology.task.internal:Checking for old test directory... 2024-08-19T01:17:05.159 DEBUG:teuthology.orchestra.run.smithi083:> test '!' -e /home/ubuntu/cephtest 2024-08-19T01:17:05.161 DEBUG:teuthology.orchestra.run.smithi117:> test '!' -e /home/ubuntu/cephtest 2024-08-19T01:17:05.164 DEBUG:teuthology.orchestra.run.smithi167:> test '!' -e /home/ubuntu/cephtest 2024-08-19T01:17:05.181 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-08-19T01:17:05.188 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-08-19T01:17:05.188 DEBUG:teuthology.orchestra.run.smithi083:> test -z $(ls -A /var/lib/ceph) 2024-08-19T01:17:05.219 DEBUG:teuthology.orchestra.run.smithi117:> test -z $(ls -A /var/lib/ceph) 2024-08-19T01:17:05.222 DEBUG:teuthology.orchestra.run.smithi167:> test -z $(ls -A /var/lib/ceph) 2024-08-19T01:17:05.318 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-08-19T01:17:05.397 INFO:teuthology.run_tasks:Running task kernel... 2024-08-19T01:17:05.411 INFO:teuthology.task.kernel:normalize config orig: {'client': {'branch': 'testing'}, 'kdb': 1, 'sha1': 'distro'} 2024-08-19T01:17:05.411 INFO:teuthology.task.kernel:normalize config final: {'client.0': {'branch': 'testing'}} 2024-08-19T01:17:05.412 INFO:teuthology.task.kernel:config {'client.0': {'branch': 'testing'}}, timeout 300 2024-08-19T01:17:05.412 DEBUG:teuthology.orchestra.run.smithi083:> test -f /run/.containerenv -o -f /.dockerenv 2024-08-19T01:17:05.429 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-08-19T01:17:05.430 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F9%2Fx86_64&ref=testing 2024-08-19T01:17:05.592 DEBUG:teuthology.task.kernel:sha1 for client.0 is 7503b862ff5bc4e556b3daee1c7881aa3e90ed62 2024-08-19T01:17:05.592 INFO:teuthology.task.kernel:Checking kernel version of client.0, want "7503b862ff5bc4e556b3daee1c7881aa3e90ed62"... 2024-08-19T01:17:05.592 DEBUG:teuthology.orchestra.run.smithi083:> uname -r 2024-08-19T01:17:05.610 INFO:teuthology.orchestra.run.smithi083.stdout:5.14.0-480.el9.x86_64 2024-08-19T01:17:05.611 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-480.el9.x86_64 vs 7503b862ff5bc4e556b3daee1c7881aa3e90ed62 2024-08-19T01:17:05.611 DEBUG:teuthology.task.kernel:failed to parse current kernel version 2024-08-19T01:17:05.611 DEBUG:teuthology.orchestra.run.smithi083:> sudo yum upgrade -y linux-firmware 2024-08-19T01:17:09.627 INFO:teuthology.orchestra.run.smithi083.stdout:CentOS Stream 9 - BaseOS 3.8 MB/s | 8.2 MB 00:02 2024-08-19T01:17:11.761 INFO:teuthology.orchestra.run.smithi083.stdout:CentOS Stream 9 - AppStream 19 MB/s | 20 MB 00:01 2024-08-19T01:17:16.012 INFO:teuthology.orchestra.run.smithi083.stdout:CentOS Stream 9 - CRB 12 MB/s | 6.6 MB 00:00 2024-08-19T01:17:17.887 INFO:teuthology.orchestra.run.smithi083.stdout:CentOS Stream 9 - Extras packages 28 kB/s | 18 kB 00:00 2024-08-19T01:17:19.927 INFO:teuthology.orchestra.run.smithi083.stdout:Extra Packages for Enterprise Linux 12 MB/s | 22 MB 00:01 2024-08-19T01:17:25.364 INFO:teuthology.orchestra.run.smithi083.stdout:lab-extras 32 kB/s | 1.7 kB 00:00 2024-08-19T01:17:27.377 INFO:teuthology.orchestra.run.smithi083.stdout:Dependencies resolved. 2024-08-19T01:17:27.378 INFO:teuthology.orchestra.run.smithi083.stdout:Nothing to do. 2024-08-19T01:17:27.378 INFO:teuthology.orchestra.run.smithi083.stdout:Complete! 2024-08-19T01:17:27.484 INFO:teuthology.task.kernel:Downloading kernel 7503b862ff5bc4e556b3daee1c7881aa3e90ed62 on client.0... 2024-08-19T01:17:27.484 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F9%2Fx86_64&sha1=7503b862ff5bc4e556b3daee1c7881aa3e90ed62 2024-08-19T01:17:27.626 INFO:teuthology.task.kernel:fetching, builder baseurl is https://2.chacra.ceph.com/r/kernel/testing/7503b862ff5bc4e556b3daee1c7881aa3e90ed62/centos/9/flavors/default/x86_64/ 2024-08-19T01:17:27.626 DEBUG:teuthology.orchestra.run.smithi083:> rm -f /tmp/kernel.x86_64.rpm && echo kernel-6.11.0_rc3_g7503b862ff5b-1.x86_64.rpm | wget -nv -O /tmp/kernel.x86_64.rpm --base=https://2.chacra.ceph.com/r/kernel/testing/7503b862ff5bc4e556b3daee1c7881aa3e90ed62/centos/9/flavors/default/x86_64/ --input-file=- 2024-08-19T01:17:27.629 DEBUG:teuthology.task.kernel:Waiting for download/copy to ubuntu@smithi083.front.sepia.ceph.com to complete... 2024-08-19T01:17:34.160 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19 01:17:34 URL:https://2.chacra.ceph.com/r/kernel/testing/7503b862ff5bc4e556b3daee1c7881aa3e90ed62/centos/9/flavors/default/x86_64/kernel-6.11.0_rc3_g7503b862ff5b-1.x86_64.rpm [360536130/360536130] -> "/tmp/kernel.x86_64.rpm" [1] 2024-08-19T01:17:34.160 INFO:teuthology.orchestra.run.smithi083.stderr:FINISHED --2024-08-19 01:17:34-- 2024-08-19T01:17:34.160 INFO:teuthology.orchestra.run.smithi083.stderr:Total wall clock time: 6.5s 2024-08-19T01:17:34.160 INFO:teuthology.orchestra.run.smithi083.stderr:Downloaded: 1 files, 344M in 6.2s (55.3 MB/s) 2024-08-19T01:17:34.163 INFO:teuthology.task.kernel:Installing kernel 7503b862ff5bc4e556b3daee1c7881aa3e90ed62 on client.0... 2024-08-19T01:17:34.163 DEBUG:teuthology.orchestra.run.smithi083:> sudo rpm -ivh --oldpackage --replacefiles --replacepkgs /tmp/kernel.x86_64.rpm 2024-08-19T01:17:36.283 INFO:teuthology.orchestra.run.smithi083.stdout:Verifying... ######################################## 2024-08-19T01:17:36.443 INFO:teuthology.orchestra.run.smithi083.stdout:Preparing... ######################################## 2024-08-19T01:17:36.558 INFO:teuthology.orchestra.run.smithi083.stdout:Updating / installing... 2024-08-19T01:17:44.549 INFO:teuthology.orchestra.run.smithi083.stdout:kernel-6.11.0_rc3_g7503b862ff5b-1 ######################################## 2024-08-19T01:18:24.820 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi083.front.sepia.ceph.com, path=/tmp/kernel.x86_64.rpm, version=None) 2024-08-19T01:18:24.821 DEBUG:teuthology.orchestra.run.smithi083:> rpm -qlp /tmp/kernel.x86_64.rpm 2024-08-19T01:18:24.932 INFO:teuthology.orchestra.run.smithi083.stdout:/boot/System.map-6.11.0-rc3-g7503b862ff5b 2024-08-19T01:18:24.932 INFO:teuthology.orchestra.run.smithi083.stdout:/boot/config-6.11.0-rc3-g7503b862ff5b 2024-08-19T01:18:24.932 INFO:teuthology.orchestra.run.smithi083.stdout:/boot/vmlinuz-6.11.0-rc3-g7503b862ff5b 2024-08-19T01:18:24.932 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b 2024-08-19T01:18:24.932 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/System.map 2024-08-19T01:18:24.932 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/config 2024-08-19T01:18:24.932 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel 2024-08-19T01:18:24.932 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/arch 2024-08-19T01:18:24.933 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/arch/x86 2024-08-19T01:18:24.933 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/arch/x86/crypto 2024-08-19T01:18:24.933 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/arch/x86/crypto/blowfish-x86_64.ko 2024-08-19T01:18:24.933 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/arch/x86/crypto/camellia-aesni-avx-x86_64.ko 2024-08-19T01:18:24.933 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/arch/x86/crypto/camellia-aesni-avx2.ko 2024-08-19T01:18:24.933 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/arch/x86/crypto/camellia-x86_64.ko 2024-08-19T01:18:24.933 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/arch/x86/crypto/cast5-avx-x86_64.ko 2024-08-19T01:18:24.933 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/arch/x86/crypto/cast6-avx-x86_64.ko 2024-08-19T01:18:24.933 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/arch/x86/crypto/crc32-pclmul.ko 2024-08-19T01:18:24.933 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/arch/x86/crypto/crc32c-intel.ko 2024-08-19T01:18:24.933 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/arch/x86/crypto/crct10dif-pclmul.ko 2024-08-19T01:18:24.933 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/arch/x86/crypto/des3_ede-x86_64.ko 2024-08-19T01:18:24.933 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/arch/x86/crypto/ghash-clmulni-intel.ko 2024-08-19T01:18:24.934 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/arch/x86/crypto/serpent-avx-x86_64.ko 2024-08-19T01:18:24.934 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/arch/x86/crypto/serpent-avx2.ko 2024-08-19T01:18:24.934 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/arch/x86/crypto/serpent-sse2-x86_64.ko 2024-08-19T01:18:24.934 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/arch/x86/crypto/sha1-ssse3.ko 2024-08-19T01:18:24.934 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/arch/x86/crypto/sha256-ssse3.ko 2024-08-19T01:18:24.934 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/arch/x86/crypto/sha512-ssse3.ko 2024-08-19T01:18:24.934 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/arch/x86/crypto/twofish-avx-x86_64.ko 2024-08-19T01:18:24.934 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/arch/x86/crypto/twofish-x86_64-3way.ko 2024-08-19T01:18:24.934 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/arch/x86/crypto/twofish-x86_64.ko 2024-08-19T01:18:24.934 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/arch/x86/kernel 2024-08-19T01:18:24.934 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/arch/x86/kernel/cpu 2024-08-19T01:18:24.934 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/arch/x86/kernel/cpu/mce 2024-08-19T01:18:24.934 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/arch/x86/kernel/cpu/mce/mce-inject.ko 2024-08-19T01:18:24.935 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/arch/x86/kvm 2024-08-19T01:18:24.935 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/arch/x86/kvm/kvm-amd.ko 2024-08-19T01:18:24.935 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/arch/x86/kvm/kvm-intel.ko 2024-08-19T01:18:24.935 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/arch/x86/kvm/kvm.ko 2024-08-19T01:18:24.935 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/crypto 2024-08-19T01:18:24.935 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/crypto/ansi_cprng.ko 2024-08-19T01:18:24.935 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/crypto/anubis.ko 2024-08-19T01:18:24.935 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/crypto/arc4.ko 2024-08-19T01:18:24.935 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/crypto/async_tx 2024-08-19T01:18:24.935 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/crypto/async_tx/async_memcpy.ko 2024-08-19T01:18:24.935 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/crypto/async_tx/async_pq.ko 2024-08-19T01:18:24.935 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/crypto/async_tx/async_raid6_recov.ko 2024-08-19T01:18:24.935 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/crypto/async_tx/async_tx.ko 2024-08-19T01:18:24.936 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/crypto/async_tx/async_xor.ko 2024-08-19T01:18:24.936 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/crypto/async_tx/raid6test.ko 2024-08-19T01:18:24.936 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/crypto/authenc.ko 2024-08-19T01:18:24.936 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/crypto/authencesn.ko 2024-08-19T01:18:24.936 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/crypto/blake2b_generic.ko 2024-08-19T01:18:24.936 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/crypto/blowfish_common.ko 2024-08-19T01:18:24.936 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/crypto/blowfish_generic.ko 2024-08-19T01:18:24.936 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/crypto/camellia_generic.ko 2024-08-19T01:18:24.936 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/crypto/cast5_generic.ko 2024-08-19T01:18:24.936 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/crypto/cast6_generic.ko 2024-08-19T01:18:24.936 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/crypto/cast_common.ko 2024-08-19T01:18:24.936 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/crypto/ccm.ko 2024-08-19T01:18:24.936 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/crypto/cmac.ko 2024-08-19T01:18:24.936 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/crypto/crc32_generic.ko 2024-08-19T01:18:24.937 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/crypto/crypto_engine.ko 2024-08-19T01:18:24.937 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/crypto/crypto_user.ko 2024-08-19T01:18:24.937 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/crypto/cts.ko 2024-08-19T01:18:24.937 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/crypto/des_generic.ko 2024-08-19T01:18:24.937 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/crypto/dh_generic.ko 2024-08-19T01:18:24.937 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/crypto/ecc.ko 2024-08-19T01:18:24.937 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/crypto/ecdh_generic.ko 2024-08-19T01:18:24.937 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/crypto/echainiv.ko 2024-08-19T01:18:24.937 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/crypto/essiv.ko 2024-08-19T01:18:24.937 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/crypto/fcrypt.ko 2024-08-19T01:18:24.937 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/crypto/khazad.ko 2024-08-19T01:18:24.937 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/crypto/lz4.ko 2024-08-19T01:18:24.937 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/crypto/lz4hc.ko 2024-08-19T01:18:24.938 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/crypto/md4.ko 2024-08-19T01:18:24.938 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/crypto/michael_mic.ko 2024-08-19T01:18:24.938 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/crypto/pcbc.ko 2024-08-19T01:18:24.938 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/crypto/pcrypt.ko 2024-08-19T01:18:24.938 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/crypto/rmd160.ko 2024-08-19T01:18:24.938 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/crypto/seed.ko 2024-08-19T01:18:24.938 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/crypto/serpent_generic.ko 2024-08-19T01:18:24.938 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/crypto/tcrypt.ko 2024-08-19T01:18:24.938 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/crypto/tea.ko 2024-08-19T01:18:24.938 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/crypto/twofish_common.ko 2024-08-19T01:18:24.938 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/crypto/twofish_generic.ko 2024-08-19T01:18:24.938 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/crypto/vmac.ko 2024-08-19T01:18:24.938 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/crypto/wp512.ko 2024-08-19T01:18:24.938 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/crypto/xcbc.ko 2024-08-19T01:18:24.939 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/crypto/xor.ko 2024-08-19T01:18:24.939 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/crypto/xxhash_generic.ko 2024-08-19T01:18:24.939 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers 2024-08-19T01:18:24.939 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/acpi 2024-08-19T01:18:24.939 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/acpi/acpi_ipmi.ko 2024-08-19T01:18:24.939 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/acpi/acpi_pad.ko 2024-08-19T01:18:24.939 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/acpi/ec_sys.ko 2024-08-19T01:18:24.939 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/acpi/platform_profile.ko 2024-08-19T01:18:24.939 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/acpi/sbs.ko 2024-08-19T01:18:24.939 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/acpi/sbshc.ko 2024-08-19T01:18:24.939 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/acpi/video.ko 2024-08-19T01:18:24.939 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/ata 2024-08-19T01:18:24.939 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/ata/acard-ahci.ko 2024-08-19T01:18:24.940 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/ata/ahci_platform.ko 2024-08-19T01:18:24.940 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/ata/ata_generic.ko 2024-08-19T01:18:24.940 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/ata/libahci_platform.ko 2024-08-19T01:18:24.940 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/ata/pata_acpi.ko 2024-08-19T01:18:24.940 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/ata/pata_ali.ko 2024-08-19T01:18:24.940 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/ata/pata_amd.ko 2024-08-19T01:18:24.940 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/ata/pata_artop.ko 2024-08-19T01:18:24.940 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/ata/pata_atiixp.ko 2024-08-19T01:18:24.940 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/ata/pata_atp867x.ko 2024-08-19T01:18:24.940 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/ata/pata_cmd640.ko 2024-08-19T01:18:24.940 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/ata/pata_cmd64x.ko 2024-08-19T01:18:24.940 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/ata/pata_cypress.ko 2024-08-19T01:18:24.940 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/ata/pata_efar.ko 2024-08-19T01:18:24.940 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/ata/pata_hpt366.ko 2024-08-19T01:18:24.941 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/ata/pata_hpt37x.ko 2024-08-19T01:18:24.941 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/ata/pata_hpt3x2n.ko 2024-08-19T01:18:24.941 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/ata/pata_hpt3x3.ko 2024-08-19T01:18:24.941 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/ata/pata_it8213.ko 2024-08-19T01:18:24.941 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/ata/pata_it821x.ko 2024-08-19T01:18:24.941 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/ata/pata_jmicron.ko 2024-08-19T01:18:24.941 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/ata/pata_marvell.ko 2024-08-19T01:18:24.941 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/ata/pata_mpiix.ko 2024-08-19T01:18:24.941 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/ata/pata_netcell.ko 2024-08-19T01:18:24.941 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/ata/pata_ninja32.ko 2024-08-19T01:18:24.941 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/ata/pata_ns87410.ko 2024-08-19T01:18:24.941 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/ata/pata_ns87415.ko 2024-08-19T01:18:24.941 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/ata/pata_oldpiix.ko 2024-08-19T01:18:24.941 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/ata/pata_opti.ko 2024-08-19T01:18:24.942 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/ata/pata_optidma.ko 2024-08-19T01:18:24.942 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/ata/pata_pcmcia.ko 2024-08-19T01:18:24.942 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/ata/pata_pdc2027x.ko 2024-08-19T01:18:24.942 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/ata/pata_pdc202xx_old.ko 2024-08-19T01:18:24.942 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/ata/pata_piccolo.ko 2024-08-19T01:18:24.942 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/ata/pata_rdc.ko 2024-08-19T01:18:24.942 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/ata/pata_sch.ko 2024-08-19T01:18:24.942 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/ata/pata_serverworks.ko 2024-08-19T01:18:24.942 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/ata/pata_sil680.ko 2024-08-19T01:18:24.942 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/ata/pata_sis.ko 2024-08-19T01:18:24.942 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/ata/pata_sl82c105.ko 2024-08-19T01:18:24.942 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/ata/pata_triflex.ko 2024-08-19T01:18:24.942 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/ata/pata_via.ko 2024-08-19T01:18:24.943 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/ata/pdc_adma.ko 2024-08-19T01:18:24.943 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/ata/sata_inic162x.ko 2024-08-19T01:18:24.943 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/ata/sata_mv.ko 2024-08-19T01:18:24.943 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/ata/sata_nv.ko 2024-08-19T01:18:24.943 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/ata/sata_promise.ko 2024-08-19T01:18:24.943 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/ata/sata_qstor.ko 2024-08-19T01:18:24.943 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/ata/sata_sil.ko 2024-08-19T01:18:24.943 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/ata/sata_sil24.ko 2024-08-19T01:18:24.943 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/ata/sata_sis.ko 2024-08-19T01:18:24.943 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/ata/sata_svw.ko 2024-08-19T01:18:24.943 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/ata/sata_sx4.ko 2024-08-19T01:18:24.943 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/ata/sata_uli.ko 2024-08-19T01:18:24.943 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/ata/sata_via.ko 2024-08-19T01:18:24.943 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/ata/sata_vsc.ko 2024-08-19T01:18:24.944 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/auxdisplay 2024-08-19T01:18:24.944 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/auxdisplay/ks0108.ko 2024-08-19T01:18:24.944 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/base 2024-08-19T01:18:24.944 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/base/regmap 2024-08-19T01:18:24.944 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/base/regmap/regmap-i2c.ko 2024-08-19T01:18:24.944 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/base/regmap/regmap-mmio.ko 2024-08-19T01:18:24.944 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/bcma 2024-08-19T01:18:24.944 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/bcma/bcma.ko 2024-08-19T01:18:24.944 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/block 2024-08-19T01:18:24.944 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/block/aoe 2024-08-19T01:18:24.944 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/block/aoe/aoe.ko 2024-08-19T01:18:24.944 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/block/brd.ko 2024-08-19T01:18:24.944 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/block/drbd 2024-08-19T01:18:24.944 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/block/drbd/drbd.ko 2024-08-19T01:18:24.945 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/block/floppy.ko 2024-08-19T01:18:24.945 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/block/loop.ko 2024-08-19T01:18:24.945 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/block/mtip32xx 2024-08-19T01:18:24.945 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/block/mtip32xx/mtip32xx.ko 2024-08-19T01:18:24.945 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/block/nbd.ko 2024-08-19T01:18:24.945 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/block/null_blk 2024-08-19T01:18:24.945 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/block/null_blk/null_blk.ko 2024-08-19T01:18:24.945 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/block/pktcdvd.ko 2024-08-19T01:18:24.945 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/block/rbd.ko 2024-08-19T01:18:24.945 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/block/virtio_blk.ko 2024-08-19T01:18:24.945 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/block/xen-blkback 2024-08-19T01:18:24.945 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/block/xen-blkback/xen-blkback.ko 2024-08-19T01:18:24.945 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/block/xen-blkfront.ko 2024-08-19T01:18:24.946 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/block/zram 2024-08-19T01:18:24.946 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/block/zram/zram.ko 2024-08-19T01:18:24.946 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/char 2024-08-19T01:18:24.946 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/char/hangcheck-timer.ko 2024-08-19T01:18:24.946 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/char/hw_random 2024-08-19T01:18:24.946 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/char/hw_random/amd-rng.ko 2024-08-19T01:18:24.946 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/char/hw_random/intel-rng.ko 2024-08-19T01:18:24.946 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/char/hw_random/timeriomem-rng.ko 2024-08-19T01:18:24.946 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/char/hw_random/via-rng.ko 2024-08-19T01:18:24.946 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/char/hw_random/virtio-rng.ko 2024-08-19T01:18:24.946 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/char/ipmi 2024-08-19T01:18:24.946 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/char/ipmi/ipmi_devintf.ko 2024-08-19T01:18:24.946 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/char/ipmi/ipmi_msghandler.ko 2024-08-19T01:18:24.947 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/char/ipmi/ipmi_poweroff.ko 2024-08-19T01:18:24.947 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/char/ipmi/ipmi_si.ko 2024-08-19T01:18:24.947 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/char/ipmi/ipmi_ssif.ko 2024-08-19T01:18:24.947 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/char/ipmi/ipmi_watchdog.ko 2024-08-19T01:18:24.947 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/char/lp.ko 2024-08-19T01:18:24.947 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/char/mwave 2024-08-19T01:18:24.947 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/char/mwave/mwave.ko 2024-08-19T01:18:24.947 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/char/ppdev.ko 2024-08-19T01:18:24.947 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/char/tlclk.ko 2024-08-19T01:18:24.947 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/char/tpm 2024-08-19T01:18:24.947 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/char/tpm/tpm.ko 2024-08-19T01:18:24.947 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/char/tpm/tpm_atmel.ko 2024-08-19T01:18:24.947 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/char/tpm/tpm_infineon.ko 2024-08-19T01:18:24.947 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/char/tpm/tpm_nsc.ko 2024-08-19T01:18:24.948 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/char/tpm/tpm_tis.ko 2024-08-19T01:18:24.948 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/char/tpm/tpm_tis_core.ko 2024-08-19T01:18:24.948 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/char/uv_mmtimer.ko 2024-08-19T01:18:24.948 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/char/virtio_console.ko 2024-08-19T01:18:24.948 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/cpufreq 2024-08-19T01:18:24.948 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/cpufreq/acpi-cpufreq.ko 2024-08-19T01:18:24.948 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/cpufreq/amd_freq_sensitivity.ko 2024-08-19T01:18:24.948 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/cpufreq/p4-clockmod.ko 2024-08-19T01:18:24.948 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/cpufreq/pcc-cpufreq.ko 2024-08-19T01:18:24.948 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/cpufreq/powernow-k8.ko 2024-08-19T01:18:24.948 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/cpufreq/speedstep-lib.ko 2024-08-19T01:18:24.948 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/crypto 2024-08-19T01:18:24.948 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/crypto/ccp 2024-08-19T01:18:24.949 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/crypto/ccp/ccp-crypto.ko 2024-08-19T01:18:24.949 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/crypto/ccp/ccp.ko 2024-08-19T01:18:24.949 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/crypto/intel 2024-08-19T01:18:24.949 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/crypto/intel/qat 2024-08-19T01:18:24.949 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/crypto/intel/qat/qat_common 2024-08-19T01:18:24.949 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/crypto/intel/qat/qat_common/intel_qat.ko 2024-08-19T01:18:24.949 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/crypto/intel/qat/qat_dh895xcc 2024-08-19T01:18:24.949 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/crypto/intel/qat/qat_dh895xcc/qat_dh895xcc.ko 2024-08-19T01:18:24.949 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/crypto/padlock-aes.ko 2024-08-19T01:18:24.949 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/crypto/padlock-sha.ko 2024-08-19T01:18:24.949 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/crypto/virtio 2024-08-19T01:18:24.949 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/crypto/virtio/virtio_crypto.ko 2024-08-19T01:18:24.949 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/dca 2024-08-19T01:18:24.949 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/dca/dca.ko 2024-08-19T01:18:24.950 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/devfreq 2024-08-19T01:18:24.950 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/devfreq/governor_simpleondemand.ko 2024-08-19T01:18:24.950 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/dma 2024-08-19T01:18:24.950 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/dma/dw 2024-08-19T01:18:24.950 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/dma/dw/dw_dmac.ko 2024-08-19T01:18:24.950 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/dma/ioat 2024-08-19T01:18:24.950 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/dma/ioat/ioatdma.ko 2024-08-19T01:18:24.950 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/edac 2024-08-19T01:18:24.950 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/edac/amd64_edac.ko 2024-08-19T01:18:24.950 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/edac/e752x_edac.ko 2024-08-19T01:18:24.950 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/edac/edac_mce_amd.ko 2024-08-19T01:18:24.950 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/edac/i3000_edac.ko 2024-08-19T01:18:24.950 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/edac/i3200_edac.ko 2024-08-19T01:18:24.951 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/edac/i5100_edac.ko 2024-08-19T01:18:24.951 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/edac/i5400_edac.ko 2024-08-19T01:18:24.951 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/edac/i7300_edac.ko 2024-08-19T01:18:24.951 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/edac/i7core_edac.ko 2024-08-19T01:18:24.951 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/edac/i82975x_edac.ko 2024-08-19T01:18:24.951 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/edac/ie31200_edac.ko 2024-08-19T01:18:24.951 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/edac/sb_edac.ko 2024-08-19T01:18:24.951 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/edac/x38_edac.ko 2024-08-19T01:18:24.951 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/firmware 2024-08-19T01:18:24.951 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/firmware/edd.ko 2024-08-19T01:18:24.951 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/firmware/iscsi_ibft.ko 2024-08-19T01:18:24.951 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/gpio 2024-08-19T01:18:24.951 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/gpio/gpio-ich.ko 2024-08-19T01:18:24.951 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/gpio/gpio-viperboard.ko 2024-08-19T01:18:24.952 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/hid 2024-08-19T01:18:24.952 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/hid/hid-appleir.ko 2024-08-19T01:18:24.952 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/hid/hid-aureal.ko 2024-08-19T01:18:24.952 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/hid/hid-axff.ko 2024-08-19T01:18:24.952 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/hid/hid-dr.ko 2024-08-19T01:18:24.952 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/hid/hid-elecom.ko 2024-08-19T01:18:24.952 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/hid/hid-elo.ko 2024-08-19T01:18:24.952 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/hid/hid-emsff.ko 2024-08-19T01:18:24.952 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/hid/hid-gaff.ko 2024-08-19T01:18:24.952 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/hid/hid-gt683r.ko 2024-08-19T01:18:24.952 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/hid/hid-gyration.ko 2024-08-19T01:18:24.952 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/hid/hid-holtek-kbd.ko 2024-08-19T01:18:24.952 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/hid/hid-holtek-mouse.ko 2024-08-19T01:18:24.952 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/hid/hid-holtekff.ko 2024-08-19T01:18:24.953 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/hid/hid-hyperv.ko 2024-08-19T01:18:24.953 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/hid/hid-icade.ko 2024-08-19T01:18:24.953 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/hid/hid-keytouch.ko 2024-08-19T01:18:24.953 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/hid/hid-kye.ko 2024-08-19T01:18:24.953 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/hid/hid-lcpower.ko 2024-08-19T01:18:24.953 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/hid/hid-led.ko 2024-08-19T01:18:24.953 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/hid/hid-lenovo.ko 2024-08-19T01:18:24.953 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/hid/hid-logitech-dj.ko 2024-08-19T01:18:24.953 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/hid/hid-logitech-hidpp.ko 2024-08-19T01:18:24.953 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/hid/hid-multitouch.ko 2024-08-19T01:18:24.953 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/hid/hid-ortek.ko 2024-08-19T01:18:24.953 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/hid/hid-penmount.ko 2024-08-19T01:18:24.953 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/hid/hid-petalynx.ko 2024-08-19T01:18:24.953 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/hid/hid-picolcd.ko 2024-08-19T01:18:24.954 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/hid/hid-pl.ko 2024-08-19T01:18:24.954 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/hid/hid-plantronics.ko 2024-08-19T01:18:24.954 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/hid/hid-primax.ko 2024-08-19T01:18:24.954 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/hid/hid-rmi.ko 2024-08-19T01:18:24.954 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/hid/hid-roccat-arvo.ko 2024-08-19T01:18:24.954 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/hid/hid-roccat-common.ko 2024-08-19T01:18:24.954 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/hid/hid-roccat-isku.ko 2024-08-19T01:18:24.954 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/hid/hid-roccat-kone.ko 2024-08-19T01:18:24.954 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/hid/hid-roccat-koneplus.ko 2024-08-19T01:18:24.954 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/hid/hid-roccat-konepure.ko 2024-08-19T01:18:24.954 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/hid/hid-roccat-kovaplus.ko 2024-08-19T01:18:24.954 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/hid/hid-roccat-lua.ko 2024-08-19T01:18:24.954 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/hid/hid-roccat-pyra.ko 2024-08-19T01:18:24.955 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/hid/hid-roccat-ryos.ko 2024-08-19T01:18:24.955 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/hid/hid-roccat-savu.ko 2024-08-19T01:18:24.955 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/hid/hid-roccat.ko 2024-08-19T01:18:24.955 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/hid/hid-saitek.ko 2024-08-19T01:18:24.955 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/hid/hid-samsung.ko 2024-08-19T01:18:24.955 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/hid/hid-sensor-hub.ko 2024-08-19T01:18:24.955 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/hid/hid-sjoy.ko 2024-08-19T01:18:24.955 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/hid/hid-sony.ko 2024-08-19T01:18:24.955 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/hid/hid-speedlink.ko 2024-08-19T01:18:24.955 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/hid/hid-steelseries.ko 2024-08-19T01:18:24.955 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/hid/hid-sunplus.ko 2024-08-19T01:18:24.955 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/hid/hid-thrustmaster.ko 2024-08-19T01:18:24.955 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/hid/hid-tivo.ko 2024-08-19T01:18:24.955 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/hid/hid-tmff.ko 2024-08-19T01:18:24.956 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/hid/hid-topseed.ko 2024-08-19T01:18:24.956 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/hid/hid-twinhan.ko 2024-08-19T01:18:24.956 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/hid/hid-uclogic.ko 2024-08-19T01:18:24.956 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/hid/hid-waltop.ko 2024-08-19T01:18:24.956 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/hid/hid-wiimote.ko 2024-08-19T01:18:24.956 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/hid/hid-xinmo.ko 2024-08-19T01:18:24.956 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/hid/hid-zpff.ko 2024-08-19T01:18:24.956 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/hid/hid-zydacron.ko 2024-08-19T01:18:24.956 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/hid/uhid.ko 2024-08-19T01:18:24.956 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/hid/wacom.ko 2024-08-19T01:18:24.956 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/hv 2024-08-19T01:18:24.956 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/hv/hv_balloon.ko 2024-08-19T01:18:24.956 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/hv/hv_utils.ko 2024-08-19T01:18:24.956 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/hv/hv_vmbus.ko 2024-08-19T01:18:24.956 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/hwmon 2024-08-19T01:18:24.957 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/hwmon/abituguru.ko 2024-08-19T01:18:24.957 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/hwmon/abituguru3.ko 2024-08-19T01:18:24.957 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/hwmon/acpi_power_meter.ko 2024-08-19T01:18:24.957 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/hwmon/ad7414.ko 2024-08-19T01:18:24.957 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/hwmon/ad7418.ko 2024-08-19T01:18:24.957 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/hwmon/adc128d818.ko 2024-08-19T01:18:24.957 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/hwmon/adm1025.ko 2024-08-19T01:18:24.957 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/hwmon/adm1026.ko 2024-08-19T01:18:24.957 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/hwmon/adm1029.ko 2024-08-19T01:18:24.957 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/hwmon/adm1031.ko 2024-08-19T01:18:24.957 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/hwmon/adm9240.ko 2024-08-19T01:18:24.957 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/hwmon/ads7828.ko 2024-08-19T01:18:24.957 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/hwmon/adt7410.ko 2024-08-19T01:18:24.957 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/hwmon/adt7411.ko 2024-08-19T01:18:24.958 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/hwmon/adt7462.ko 2024-08-19T01:18:24.958 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/hwmon/adt7470.ko 2024-08-19T01:18:24.958 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/hwmon/adt7475.ko 2024-08-19T01:18:24.958 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/hwmon/adt7x10.ko 2024-08-19T01:18:24.958 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/hwmon/amc6821.ko 2024-08-19T01:18:24.958 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/hwmon/applesmc.ko 2024-08-19T01:18:24.958 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/hwmon/asb100.ko 2024-08-19T01:18:24.958 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/hwmon/asc7621.ko 2024-08-19T01:18:24.958 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/hwmon/asus_atk0110.ko 2024-08-19T01:18:24.958 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/hwmon/atxp1.ko 2024-08-19T01:18:24.958 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/hwmon/coretemp.ko 2024-08-19T01:18:24.958 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/hwmon/dell-smm-hwmon.ko 2024-08-19T01:18:24.958 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/hwmon/dme1737.ko 2024-08-19T01:18:24.958 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/hwmon/ds1621.ko 2024-08-19T01:18:24.959 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/hwmon/ds620.ko 2024-08-19T01:18:24.959 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/hwmon/emc1403.ko 2024-08-19T01:18:24.959 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/hwmon/emc6w201.ko 2024-08-19T01:18:24.959 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/hwmon/f71805f.ko 2024-08-19T01:18:24.959 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/hwmon/f71882fg.ko 2024-08-19T01:18:24.959 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/hwmon/f75375s.ko 2024-08-19T01:18:24.959 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/hwmon/fam15h_power.ko 2024-08-19T01:18:24.959 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/hwmon/fschmd.ko 2024-08-19T01:18:24.959 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/hwmon/g760a.ko 2024-08-19T01:18:24.959 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/hwmon/g762.ko 2024-08-19T01:18:24.959 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/hwmon/gl518sm.ko 2024-08-19T01:18:24.959 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/hwmon/gl520sm.ko 2024-08-19T01:18:24.959 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/hwmon/hwmon-vid.ko 2024-08-19T01:18:24.960 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/hwmon/i5500_temp.ko 2024-08-19T01:18:24.960 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/hwmon/i5k_amb.ko 2024-08-19T01:18:24.960 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/hwmon/ibmaem.ko 2024-08-19T01:18:24.960 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/hwmon/ibmpex.ko 2024-08-19T01:18:24.960 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/hwmon/ina209.ko 2024-08-19T01:18:24.960 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/hwmon/ina2xx.ko 2024-08-19T01:18:24.960 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/hwmon/it87.ko 2024-08-19T01:18:24.960 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/hwmon/k10temp.ko 2024-08-19T01:18:24.960 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/hwmon/k8temp.ko 2024-08-19T01:18:24.960 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/hwmon/lineage-pem.ko 2024-08-19T01:18:24.960 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/hwmon/lm63.ko 2024-08-19T01:18:24.960 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/hwmon/lm73.ko 2024-08-19T01:18:24.960 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/hwmon/lm75.ko 2024-08-19T01:18:24.960 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/hwmon/lm77.ko 2024-08-19T01:18:24.960 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/hwmon/lm78.ko 2024-08-19T01:18:24.961 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/hwmon/lm80.ko 2024-08-19T01:18:24.961 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/hwmon/lm83.ko 2024-08-19T01:18:24.961 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/hwmon/lm85.ko 2024-08-19T01:18:24.961 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/hwmon/lm87.ko 2024-08-19T01:18:24.961 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/hwmon/lm90.ko 2024-08-19T01:18:24.961 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/hwmon/lm92.ko 2024-08-19T01:18:24.961 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/hwmon/lm93.ko 2024-08-19T01:18:24.961 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/hwmon/lm95234.ko 2024-08-19T01:18:24.961 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/hwmon/lm95241.ko 2024-08-19T01:18:24.961 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/hwmon/lm95245.ko 2024-08-19T01:18:24.961 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/hwmon/ltc2945.ko 2024-08-19T01:18:24.961 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/hwmon/ltc4151.ko 2024-08-19T01:18:24.961 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/hwmon/ltc4215.ko 2024-08-19T01:18:24.961 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/hwmon/ltc4222.ko 2024-08-19T01:18:24.962 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/hwmon/ltc4245.ko 2024-08-19T01:18:24.962 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/hwmon/ltc4260.ko 2024-08-19T01:18:24.962 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/hwmon/ltc4261.ko 2024-08-19T01:18:24.962 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/hwmon/max16065.ko 2024-08-19T01:18:24.962 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/hwmon/max1619.ko 2024-08-19T01:18:24.962 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/hwmon/max1668.ko 2024-08-19T01:18:24.962 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/hwmon/max197.ko 2024-08-19T01:18:24.962 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/hwmon/max6639.ko 2024-08-19T01:18:24.962 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/hwmon/max6650.ko 2024-08-19T01:18:24.962 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/hwmon/max6697.ko 2024-08-19T01:18:24.962 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/hwmon/mcp3021.ko 2024-08-19T01:18:24.962 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/hwmon/nct6683.ko 2024-08-19T01:18:24.963 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/hwmon/nct6775-core.ko 2024-08-19T01:18:24.963 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/hwmon/nct6775.ko 2024-08-19T01:18:24.963 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/hwmon/nct7802.ko 2024-08-19T01:18:24.963 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/hwmon/pc87360.ko 2024-08-19T01:18:24.963 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/hwmon/pc87427.ko 2024-08-19T01:18:24.963 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/hwmon/pcf8591.ko 2024-08-19T01:18:24.963 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/hwmon/pmbus 2024-08-19T01:18:24.963 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/hwmon/pmbus/adm1275.ko 2024-08-19T01:18:24.963 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/hwmon/pmbus/lm25066.ko 2024-08-19T01:18:24.963 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/hwmon/pmbus/ltc2978.ko 2024-08-19T01:18:24.963 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/hwmon/pmbus/max16064.ko 2024-08-19T01:18:24.963 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/hwmon/pmbus/max34440.ko 2024-08-19T01:18:24.963 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/hwmon/pmbus/max8688.ko 2024-08-19T01:18:24.963 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/hwmon/pmbus/pmbus.ko 2024-08-19T01:18:24.964 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/hwmon/pmbus/pmbus_core.ko 2024-08-19T01:18:24.964 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/hwmon/pmbus/tps40422.ko 2024-08-19T01:18:24.964 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/hwmon/pmbus/ucd9000.ko 2024-08-19T01:18:24.964 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/hwmon/pmbus/ucd9200.ko 2024-08-19T01:18:24.964 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/hwmon/pmbus/zl6100.ko 2024-08-19T01:18:24.964 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/hwmon/powr1220.ko 2024-08-19T01:18:24.964 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/hwmon/sch5627.ko 2024-08-19T01:18:24.964 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/hwmon/sch5636.ko 2024-08-19T01:18:24.964 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/hwmon/sch56xx-common.ko 2024-08-19T01:18:24.964 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/hwmon/sht15.ko 2024-08-19T01:18:24.964 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/hwmon/sht21.ko 2024-08-19T01:18:24.964 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/hwmon/shtc1.ko 2024-08-19T01:18:24.964 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/hwmon/sis5595.ko 2024-08-19T01:18:24.964 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/hwmon/smsc47b397.ko 2024-08-19T01:18:24.965 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/hwmon/smsc47m1.ko 2024-08-19T01:18:24.965 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/hwmon/smsc47m192.ko 2024-08-19T01:18:24.965 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/hwmon/thmc50.ko 2024-08-19T01:18:24.965 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/hwmon/tmp102.ko 2024-08-19T01:18:24.965 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/hwmon/tmp103.ko 2024-08-19T01:18:24.965 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/hwmon/tmp401.ko 2024-08-19T01:18:24.965 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/hwmon/tmp421.ko 2024-08-19T01:18:24.965 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/hwmon/via-cputemp.ko 2024-08-19T01:18:24.965 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/hwmon/via686a.ko 2024-08-19T01:18:24.965 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/hwmon/vt1211.ko 2024-08-19T01:18:24.965 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/hwmon/vt8231.ko 2024-08-19T01:18:24.965 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/hwmon/w83627ehf.ko 2024-08-19T01:18:24.965 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/hwmon/w83627hf.ko 2024-08-19T01:18:24.965 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/hwmon/w83781d.ko 2024-08-19T01:18:24.966 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/hwmon/w83791d.ko 2024-08-19T01:18:24.966 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/hwmon/w83792d.ko 2024-08-19T01:18:24.966 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/hwmon/w83793.ko 2024-08-19T01:18:24.966 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/hwmon/w83795.ko 2024-08-19T01:18:24.966 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/hwmon/w83l785ts.ko 2024-08-19T01:18:24.966 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/hwmon/w83l786ng.ko 2024-08-19T01:18:24.966 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/i2c 2024-08-19T01:18:24.966 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/i2c/algos 2024-08-19T01:18:24.966 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/i2c/algos/i2c-algo-bit.ko 2024-08-19T01:18:24.966 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/i2c/algos/i2c-algo-pca.ko 2024-08-19T01:18:24.966 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/i2c/busses 2024-08-19T01:18:24.966 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/i2c/busses/i2c-amd756-s4882.ko 2024-08-19T01:18:24.966 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/i2c/busses/i2c-amd756.ko 2024-08-19T01:18:24.967 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/i2c/busses/i2c-amd8111.ko 2024-08-19T01:18:24.967 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/i2c/busses/i2c-ccgx-ucsi.ko 2024-08-19T01:18:24.967 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/i2c/busses/i2c-designware-core.ko 2024-08-19T01:18:24.967 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/i2c/busses/i2c-designware-pci.ko 2024-08-19T01:18:24.967 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/i2c/busses/i2c-designware-platform.ko 2024-08-19T01:18:24.967 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/i2c/busses/i2c-diolan-u2c.ko 2024-08-19T01:18:24.967 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/i2c/busses/i2c-i801.ko 2024-08-19T01:18:24.967 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/i2c/busses/i2c-isch.ko 2024-08-19T01:18:24.967 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/i2c/busses/i2c-ismt.ko 2024-08-19T01:18:24.967 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/i2c/busses/i2c-nforce2-s4985.ko 2024-08-19T01:18:24.967 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/i2c/busses/i2c-nforce2.ko 2024-08-19T01:18:24.967 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/i2c/busses/i2c-parport.ko 2024-08-19T01:18:24.967 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/i2c/busses/i2c-pca-platform.ko 2024-08-19T01:18:24.967 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/i2c/busses/i2c-piix4.ko 2024-08-19T01:18:24.968 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/i2c/busses/i2c-scmi.ko 2024-08-19T01:18:24.968 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/i2c/busses/i2c-simtec.ko 2024-08-19T01:18:24.968 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/i2c/busses/i2c-sis96x.ko 2024-08-19T01:18:24.968 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/i2c/busses/i2c-tiny-usb.ko 2024-08-19T01:18:24.968 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/i2c/busses/i2c-via.ko 2024-08-19T01:18:24.968 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/i2c/busses/i2c-viapro.ko 2024-08-19T01:18:24.968 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/i2c/busses/i2c-viperboard.ko 2024-08-19T01:18:24.968 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/i2c/i2c-dev.ko 2024-08-19T01:18:24.968 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/i2c/i2c-mux.ko 2024-08-19T01:18:24.968 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/i2c/i2c-slave-eeprom.ko 2024-08-19T01:18:24.968 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/i2c/i2c-smbus.ko 2024-08-19T01:18:24.968 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/i2c/i2c-stub.ko 2024-08-19T01:18:24.968 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/infiniband 2024-08-19T01:18:24.968 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/infiniband/core 2024-08-19T01:18:24.969 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/infiniband/core/ib_cm.ko 2024-08-19T01:18:24.969 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/infiniband/core/ib_core.ko 2024-08-19T01:18:24.969 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/infiniband/core/ib_umad.ko 2024-08-19T01:18:24.969 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/infiniband/core/ib_uverbs.ko 2024-08-19T01:18:24.969 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/infiniband/core/iw_cm.ko 2024-08-19T01:18:24.969 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/infiniband/core/rdma_cm.ko 2024-08-19T01:18:24.969 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/infiniband/core/rdma_ucm.ko 2024-08-19T01:18:24.969 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/infiniband/hw 2024-08-19T01:18:24.969 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/infiniband/hw/cxgb4 2024-08-19T01:18:24.969 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/infiniband/hw/cxgb4/iw_cxgb4.ko 2024-08-19T01:18:24.969 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/infiniband/hw/mlx4 2024-08-19T01:18:24.969 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/infiniband/hw/mlx4/mlx4_ib.ko 2024-08-19T01:18:24.969 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/infiniband/hw/mlx5 2024-08-19T01:18:24.970 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/infiniband/hw/mlx5/mlx5_ib.ko 2024-08-19T01:18:24.970 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/infiniband/hw/mthca 2024-08-19T01:18:24.970 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/infiniband/hw/mthca/ib_mthca.ko 2024-08-19T01:18:24.970 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/infiniband/ulp 2024-08-19T01:18:24.970 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/infiniband/ulp/ipoib 2024-08-19T01:18:24.970 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/infiniband/ulp/ipoib/ib_ipoib.ko 2024-08-19T01:18:24.970 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/infiniband/ulp/iser 2024-08-19T01:18:24.970 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/infiniband/ulp/iser/ib_iser.ko 2024-08-19T01:18:24.970 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/infiniband/ulp/isert 2024-08-19T01:18:24.970 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/infiniband/ulp/isert/ib_isert.ko 2024-08-19T01:18:24.970 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/infiniband/ulp/srp 2024-08-19T01:18:24.970 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/infiniband/ulp/srp/ib_srp.ko 2024-08-19T01:18:24.970 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/infiniband/ulp/srpt 2024-08-19T01:18:24.970 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/infiniband/ulp/srpt/ib_srpt.ko 2024-08-19T01:18:24.971 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/input 2024-08-19T01:18:24.971 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/input/gameport 2024-08-19T01:18:24.971 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/input/gameport/emu10k1-gp.ko 2024-08-19T01:18:24.971 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/input/gameport/fm801-gp.ko 2024-08-19T01:18:24.971 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/input/gameport/gameport.ko 2024-08-19T01:18:24.971 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/input/joydev.ko 2024-08-19T01:18:24.971 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/input/keyboard 2024-08-19T01:18:24.971 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/input/keyboard/gpio_keys.ko 2024-08-19T01:18:24.971 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/input/mouse 2024-08-19T01:18:24.971 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/input/mouse/appletouch.ko 2024-08-19T01:18:24.971 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/input/mouse/bcm5974.ko 2024-08-19T01:18:24.971 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/input/mouse/cyapatp.ko 2024-08-19T01:18:24.971 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/input/mouse/elan_i2c.ko 2024-08-19T01:18:24.971 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/input/mouse/sermouse.ko 2024-08-19T01:18:24.972 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/input/mouse/synaptics_i2c.ko 2024-08-19T01:18:24.972 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/input/mouse/synaptics_usb.ko 2024-08-19T01:18:24.972 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/input/mouse/vsxxxaa.ko 2024-08-19T01:18:24.972 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/input/rmi4 2024-08-19T01:18:24.972 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/input/rmi4/rmi_core.ko 2024-08-19T01:18:24.972 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/input/serio 2024-08-19T01:18:24.972 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/input/serio/altera_ps2.ko 2024-08-19T01:18:24.972 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/input/serio/arc_ps2.ko 2024-08-19T01:18:24.972 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/input/serio/hyperv-keyboard.ko 2024-08-19T01:18:24.972 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/input/serio/serio_raw.ko 2024-08-19T01:18:24.972 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/input/sparse-keymap.ko 2024-08-19T01:18:24.972 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/leds 2024-08-19T01:18:24.972 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/leds/leds-blinkm.ko 2024-08-19T01:18:24.973 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/leds/leds-lm3530.ko 2024-08-19T01:18:24.973 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/leds/leds-lp3944.ko 2024-08-19T01:18:24.973 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/leds/leds-ss4200.ko 2024-08-19T01:18:24.973 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/leds/trigger 2024-08-19T01:18:24.973 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/leds/trigger/ledtrig-backlight.ko 2024-08-19T01:18:24.973 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/leds/trigger/ledtrig-camera.ko 2024-08-19T01:18:24.973 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/leds/trigger/ledtrig-default-on.ko 2024-08-19T01:18:24.973 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/leds/trigger/ledtrig-gpio.ko 2024-08-19T01:18:24.973 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/leds/trigger/ledtrig-heartbeat.ko 2024-08-19T01:18:24.973 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/leds/trigger/ledtrig-oneshot.ko 2024-08-19T01:18:24.973 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/leds/trigger/ledtrig-timer.ko 2024-08-19T01:18:24.973 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/leds/trigger/ledtrig-transient.ko 2024-08-19T01:18:24.973 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/md 2024-08-19T01:18:24.973 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/md/bcache 2024-08-19T01:18:24.974 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/md/bcache/bcache.ko 2024-08-19T01:18:24.974 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/md/dm-bio-prison.ko 2024-08-19T01:18:24.974 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/md/dm-cache-smq.ko 2024-08-19T01:18:24.974 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/md/dm-cache.ko 2024-08-19T01:18:24.974 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/md/dm-crypt.ko 2024-08-19T01:18:24.974 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/md/dm-delay.ko 2024-08-19T01:18:24.974 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/md/dm-flakey.ko 2024-08-19T01:18:24.974 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/md/dm-log-userspace.ko 2024-08-19T01:18:24.974 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/md/dm-multipath.ko 2024-08-19T01:18:24.974 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/md/dm-queue-length.ko 2024-08-19T01:18:24.974 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/md/dm-raid.ko 2024-08-19T01:18:24.974 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/md/dm-round-robin.ko 2024-08-19T01:18:24.974 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/md/dm-service-time.ko 2024-08-19T01:18:24.975 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/md/dm-switch.ko 2024-08-19T01:18:24.975 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/md/dm-thin-pool.ko 2024-08-19T01:18:24.975 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/md/dm-verity.ko 2024-08-19T01:18:24.975 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/md/persistent-data 2024-08-19T01:18:24.975 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/md/persistent-data/dm-persistent-data.ko 2024-08-19T01:18:24.975 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/md/raid0.ko 2024-08-19T01:18:24.975 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/md/raid1.ko 2024-08-19T01:18:24.975 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/md/raid10.ko 2024-08-19T01:18:24.975 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/md/raid456.ko 2024-08-19T01:18:24.975 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/message 2024-08-19T01:18:24.975 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/message/fusion 2024-08-19T01:18:24.975 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/message/fusion/mptbase.ko 2024-08-19T01:18:24.975 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/message/fusion/mptctl.ko 2024-08-19T01:18:24.975 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/message/fusion/mptfc.ko 2024-08-19T01:18:24.975 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/message/fusion/mptlan.ko 2024-08-19T01:18:24.976 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/message/fusion/mptsas.ko 2024-08-19T01:18:24.976 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/message/fusion/mptscsih.ko 2024-08-19T01:18:24.976 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/message/fusion/mptspi.ko 2024-08-19T01:18:24.976 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/mfd 2024-08-19T01:18:24.976 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/mfd/lpc_ich.ko 2024-08-19T01:18:24.976 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/mfd/lpc_sch.ko 2024-08-19T01:18:24.976 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/mfd/mfd-core.ko 2024-08-19T01:18:24.976 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/mfd/sm501.ko 2024-08-19T01:18:24.976 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/mfd/viperboard.ko 2024-08-19T01:18:24.976 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/mfd/vx855.ko 2024-08-19T01:18:24.976 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/mfd/wl1273-core.ko 2024-08-19T01:18:24.976 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/misc 2024-08-19T01:18:24.976 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/misc/altera-stapl 2024-08-19T01:18:24.976 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/misc/altera-stapl/altera-stapl.ko 2024-08-19T01:18:24.976 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/misc/apds9802als.ko 2024-08-19T01:18:24.977 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/misc/apds990x.ko 2024-08-19T01:18:24.977 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/misc/bh1770glc.ko 2024-08-19T01:18:24.977 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/misc/cb710 2024-08-19T01:18:24.977 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/misc/cb710/cb710.ko 2024-08-19T01:18:24.977 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/misc/eeprom 2024-08-19T01:18:24.977 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/misc/eeprom/at24.ko 2024-08-19T01:18:24.977 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/misc/eeprom/eeprom_93cx6.ko 2024-08-19T01:18:24.977 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/misc/eeprom/max6875.ko 2024-08-19T01:18:24.977 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/misc/enclosure.ko 2024-08-19T01:18:24.977 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/misc/hpilo.ko 2024-08-19T01:18:24.977 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/misc/isl29003.ko 2024-08-19T01:18:24.977 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/misc/isl29020.ko 2024-08-19T01:18:24.977 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/misc/lis3lv02d 2024-08-19T01:18:24.977 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/misc/lis3lv02d/lis3lv02d.ko 2024-08-19T01:18:24.977 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/misc/lis3lv02d/lis3lv02d_i2c.ko 2024-08-19T01:18:24.978 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/misc/mei 2024-08-19T01:18:24.978 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/misc/mei/mei-me.ko 2024-08-19T01:18:24.978 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/misc/mei/mei-txe.ko 2024-08-19T01:18:24.978 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/misc/mei/mei.ko 2024-08-19T01:18:24.978 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/misc/sgi-gru 2024-08-19T01:18:24.978 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/misc/sgi-gru/gru.ko 2024-08-19T01:18:24.978 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/misc/sgi-xp 2024-08-19T01:18:24.978 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/misc/sgi-xp/xp.ko 2024-08-19T01:18:24.978 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/misc/sgi-xp/xpc.ko 2024-08-19T01:18:24.978 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/misc/sgi-xp/xpnet.ko 2024-08-19T01:18:24.978 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/misc/tifm_7xx1.ko 2024-08-19T01:18:24.978 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/misc/tifm_core.ko 2024-08-19T01:18:24.978 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/misc/tsl2550.ko 2024-08-19T01:18:24.978 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/misc/vmw_balloon.ko 2024-08-19T01:18:24.979 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/misc/vmw_vmci 2024-08-19T01:18:24.979 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/misc/vmw_vmci/vmw_vmci.ko 2024-08-19T01:18:24.979 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/net 2024-08-19T01:18:24.979 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/net/bonding 2024-08-19T01:18:24.979 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/net/bonding/bonding.ko 2024-08-19T01:18:24.979 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/net/dsa 2024-08-19T01:18:24.979 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/net/dsa/b53 2024-08-19T01:18:24.979 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/net/dsa/b53/b53_common.ko 2024-08-19T01:18:24.979 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/net/dsa/bcm-sf2.ko 2024-08-19T01:18:24.979 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/net/dsa/mv88e6060.ko 2024-08-19T01:18:24.979 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/net/dsa/mv88e6xxx 2024-08-19T01:18:24.979 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/net/dsa/mv88e6xxx/mv88e6xxx.ko 2024-08-19T01:18:24.979 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/net/dummy.ko 2024-08-19T01:18:24.979 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/net/eql.ko 2024-08-19T01:18:24.980 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/net/ethernet 2024-08-19T01:18:24.980 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/net/ethernet/3com 2024-08-19T01:18:24.980 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/net/ethernet/3com/3c574_cs.ko 2024-08-19T01:18:24.980 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/net/ethernet/3com/3c589_cs.ko 2024-08-19T01:18:24.980 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/net/ethernet/3com/3c59x.ko 2024-08-19T01:18:24.980 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/net/ethernet/3com/typhoon.ko 2024-08-19T01:18:24.980 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/net/ethernet/8390 2024-08-19T01:18:24.980 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/net/ethernet/8390/8390.ko 2024-08-19T01:18:24.980 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/net/ethernet/8390/axnet_cs.ko 2024-08-19T01:18:24.980 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/net/ethernet/8390/ne2k-pci.ko 2024-08-19T01:18:24.980 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/net/ethernet/8390/pcnet_cs.ko 2024-08-19T01:18:24.980 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/net/ethernet/adaptec 2024-08-19T01:18:24.980 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/net/ethernet/adaptec/starfire.ko 2024-08-19T01:18:24.981 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/net/ethernet/agere 2024-08-19T01:18:24.981 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/net/ethernet/agere/et131x.ko 2024-08-19T01:18:24.981 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/net/ethernet/alteon 2024-08-19T01:18:24.981 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/net/ethernet/alteon/acenic.ko 2024-08-19T01:18:24.981 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/net/ethernet/altera 2024-08-19T01:18:24.981 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/net/ethernet/altera/altera_tse.ko 2024-08-19T01:18:24.981 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/net/ethernet/amd 2024-08-19T01:18:24.981 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/net/ethernet/amd/amd8111e.ko 2024-08-19T01:18:24.981 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/net/ethernet/amd/nmclan_cs.ko 2024-08-19T01:18:24.981 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/net/ethernet/amd/pcnet32.ko 2024-08-19T01:18:24.981 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/net/ethernet/atheros 2024-08-19T01:18:24.981 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/net/ethernet/atheros/alx 2024-08-19T01:18:24.981 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/net/ethernet/atheros/alx/alx.ko 2024-08-19T01:18:24.981 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/net/ethernet/atheros/atl1c 2024-08-19T01:18:24.981 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/net/ethernet/atheros/atl1c/atl1c.ko 2024-08-19T01:18:24.982 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/net/ethernet/atheros/atl1e 2024-08-19T01:18:24.982 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/net/ethernet/atheros/atl1e/atl1e.ko 2024-08-19T01:18:24.982 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/net/ethernet/atheros/atlx 2024-08-19T01:18:24.982 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/net/ethernet/atheros/atlx/atl1.ko 2024-08-19T01:18:24.982 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/net/ethernet/atheros/atlx/atl2.ko 2024-08-19T01:18:24.982 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/net/ethernet/broadcom 2024-08-19T01:18:24.982 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/net/ethernet/broadcom/b44.ko 2024-08-19T01:18:24.982 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/net/ethernet/broadcom/bnx2.ko 2024-08-19T01:18:24.982 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/net/ethernet/broadcom/bnx2x 2024-08-19T01:18:24.982 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/net/ethernet/broadcom/bnx2x/bnx2x.ko 2024-08-19T01:18:24.982 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/net/ethernet/broadcom/cnic.ko 2024-08-19T01:18:24.982 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/net/ethernet/broadcom/genet 2024-08-19T01:18:24.983 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/net/ethernet/broadcom/genet/genet.ko 2024-08-19T01:18:24.983 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/net/ethernet/broadcom/tg3.ko 2024-08-19T01:18:24.983 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/net/ethernet/brocade 2024-08-19T01:18:24.983 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/net/ethernet/brocade/bna 2024-08-19T01:18:24.983 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/net/ethernet/brocade/bna/bna.ko 2024-08-19T01:18:24.983 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/net/ethernet/chelsio 2024-08-19T01:18:24.983 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/net/ethernet/chelsio/cxgb 2024-08-19T01:18:24.983 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/net/ethernet/chelsio/cxgb/cxgb.ko 2024-08-19T01:18:24.983 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/net/ethernet/chelsio/cxgb3 2024-08-19T01:18:24.983 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/net/ethernet/chelsio/cxgb3/cxgb3.ko 2024-08-19T01:18:24.983 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/net/ethernet/chelsio/cxgb4 2024-08-19T01:18:24.983 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/net/ethernet/chelsio/cxgb4/cxgb4.ko 2024-08-19T01:18:24.983 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/net/ethernet/chelsio/cxgb4vf 2024-08-19T01:18:24.983 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/net/ethernet/chelsio/cxgb4vf/cxgb4vf.ko 2024-08-19T01:18:24.984 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/net/ethernet/chelsio/libcxgb 2024-08-19T01:18:24.984 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/net/ethernet/chelsio/libcxgb/libcxgb.ko 2024-08-19T01:18:24.984 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/net/ethernet/cisco 2024-08-19T01:18:24.984 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/net/ethernet/cisco/enic 2024-08-19T01:18:24.984 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/net/ethernet/cisco/enic/enic.ko 2024-08-19T01:18:24.984 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/net/ethernet/dec 2024-08-19T01:18:24.984 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/net/ethernet/dec/tulip 2024-08-19T01:18:24.984 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/net/ethernet/dec/tulip/de2104x.ko 2024-08-19T01:18:24.984 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/net/ethernet/dec/tulip/dmfe.ko 2024-08-19T01:18:24.984 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/net/ethernet/dec/tulip/tulip.ko 2024-08-19T01:18:24.984 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/net/ethernet/dec/tulip/uli526x.ko 2024-08-19T01:18:24.984 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/net/ethernet/dec/tulip/winbond-840.ko 2024-08-19T01:18:24.984 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/net/ethernet/dec/tulip/xircom_cb.ko 2024-08-19T01:18:24.984 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/net/ethernet/dlink 2024-08-19T01:18:24.985 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/net/ethernet/dlink/dl2k.ko 2024-08-19T01:18:24.985 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/net/ethernet/dlink/sundance.ko 2024-08-19T01:18:24.985 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/net/ethernet/dnet.ko 2024-08-19T01:18:24.985 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/net/ethernet/emulex 2024-08-19T01:18:24.985 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/net/ethernet/emulex/benet 2024-08-19T01:18:24.985 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/net/ethernet/emulex/benet/be2net.ko 2024-08-19T01:18:24.985 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/net/ethernet/ethoc.ko 2024-08-19T01:18:24.985 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/net/ethernet/fealnx.ko 2024-08-19T01:18:24.985 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/net/ethernet/intel 2024-08-19T01:18:24.985 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/net/ethernet/intel/e100.ko 2024-08-19T01:18:24.985 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/net/ethernet/intel/e1000 2024-08-19T01:18:24.985 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/net/ethernet/intel/e1000/e1000.ko 2024-08-19T01:18:24.985 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/net/ethernet/intel/e1000e 2024-08-19T01:18:24.985 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/net/ethernet/intel/e1000e/e1000e.ko 2024-08-19T01:18:24.986 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/net/ethernet/intel/fm10k 2024-08-19T01:18:24.986 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/net/ethernet/intel/fm10k/fm10k.ko 2024-08-19T01:18:24.986 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/net/ethernet/intel/i40e 2024-08-19T01:18:24.986 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/net/ethernet/intel/i40e/i40e.ko 2024-08-19T01:18:24.986 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/net/ethernet/intel/iavf 2024-08-19T01:18:24.986 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/net/ethernet/intel/iavf/iavf.ko 2024-08-19T01:18:24.986 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/net/ethernet/intel/igb 2024-08-19T01:18:24.986 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/net/ethernet/intel/igb/igb.ko 2024-08-19T01:18:24.986 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/net/ethernet/intel/igbvf 2024-08-19T01:18:24.986 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/net/ethernet/intel/igbvf/igbvf.ko 2024-08-19T01:18:24.986 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/net/ethernet/intel/ixgbe 2024-08-19T01:18:24.986 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/net/ethernet/intel/ixgbe/ixgbe.ko 2024-08-19T01:18:24.986 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/net/ethernet/intel/ixgbevf 2024-08-19T01:18:24.987 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/net/ethernet/intel/ixgbevf/ixgbevf.ko 2024-08-19T01:18:24.987 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/net/ethernet/intel/libeth 2024-08-19T01:18:24.987 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/net/ethernet/intel/libeth/libeth.ko 2024-08-19T01:18:24.987 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/net/ethernet/intel/libie 2024-08-19T01:18:24.987 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/net/ethernet/intel/libie/libie.ko 2024-08-19T01:18:24.987 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/net/ethernet/jme.ko 2024-08-19T01:18:24.987 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/net/ethernet/marvell 2024-08-19T01:18:24.987 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/net/ethernet/marvell/mvmdio.ko 2024-08-19T01:18:24.987 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/net/ethernet/marvell/skge.ko 2024-08-19T01:18:24.987 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/net/ethernet/marvell/sky2.ko 2024-08-19T01:18:24.987 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/net/ethernet/mellanox 2024-08-19T01:18:24.987 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/net/ethernet/mellanox/mlx4 2024-08-19T01:18:24.987 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_core.ko 2024-08-19T01:18:24.987 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_en.ko 2024-08-19T01:18:24.988 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/net/ethernet/mellanox/mlx5 2024-08-19T01:18:24.988 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/net/ethernet/mellanox/mlx5/core 2024-08-19T01:18:24.988 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/net/ethernet/mellanox/mlx5/core/mlx5_core.ko 2024-08-19T01:18:24.988 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/net/ethernet/micrel 2024-08-19T01:18:24.988 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/net/ethernet/micrel/ksz884x.ko 2024-08-19T01:18:24.988 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/net/ethernet/myricom 2024-08-19T01:18:24.988 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/net/ethernet/myricom/myri10ge 2024-08-19T01:18:24.988 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/net/ethernet/myricom/myri10ge/myri10ge.ko 2024-08-19T01:18:24.988 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/net/ethernet/natsemi 2024-08-19T01:18:24.988 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/net/ethernet/natsemi/natsemi.ko 2024-08-19T01:18:24.988 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/net/ethernet/natsemi/ns83820.ko 2024-08-19T01:18:24.988 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/net/ethernet/neterion 2024-08-19T01:18:24.988 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/net/ethernet/neterion/s2io.ko 2024-08-19T01:18:24.988 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/net/ethernet/nvidia 2024-08-19T01:18:24.988 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/net/ethernet/nvidia/forcedeth.ko 2024-08-19T01:18:24.989 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/net/ethernet/packetengines 2024-08-19T01:18:24.989 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/net/ethernet/packetengines/hamachi.ko 2024-08-19T01:18:24.989 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/net/ethernet/packetengines/yellowfin.ko 2024-08-19T01:18:24.989 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/net/ethernet/qlogic 2024-08-19T01:18:24.989 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/net/ethernet/qlogic/netxen 2024-08-19T01:18:24.989 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/net/ethernet/qlogic/netxen/netxen_nic.ko 2024-08-19T01:18:24.989 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/net/ethernet/qlogic/qla3xxx.ko 2024-08-19T01:18:24.989 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/net/ethernet/qlogic/qlcnic 2024-08-19T01:18:24.989 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/net/ethernet/qlogic/qlcnic/qlcnic.ko 2024-08-19T01:18:24.989 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/net/ethernet/rdc 2024-08-19T01:18:24.989 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/net/ethernet/rdc/r6040.ko 2024-08-19T01:18:24.989 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/net/ethernet/realtek 2024-08-19T01:18:24.989 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/net/ethernet/realtek/8139cp.ko 2024-08-19T01:18:24.990 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/net/ethernet/realtek/8139too.ko 2024-08-19T01:18:24.990 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/net/ethernet/realtek/atp.ko 2024-08-19T01:18:24.990 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/net/ethernet/realtek/r8169.ko 2024-08-19T01:18:24.990 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/net/ethernet/rocker 2024-08-19T01:18:24.990 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/net/ethernet/rocker/rocker.ko 2024-08-19T01:18:24.990 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/net/ethernet/sfc 2024-08-19T01:18:24.990 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/net/ethernet/sfc/sfc.ko 2024-08-19T01:18:24.990 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/net/ethernet/silan 2024-08-19T01:18:24.990 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/net/ethernet/silan/sc92031.ko 2024-08-19T01:18:24.990 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/net/ethernet/sis 2024-08-19T01:18:24.990 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/net/ethernet/sis/sis190.ko 2024-08-19T01:18:24.990 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/net/ethernet/sis/sis900.ko 2024-08-19T01:18:24.990 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/net/ethernet/smsc 2024-08-19T01:18:24.991 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/net/ethernet/smsc/epic100.ko 2024-08-19T01:18:24.991 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/net/ethernet/smsc/smc91c92_cs.ko 2024-08-19T01:18:24.991 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/net/ethernet/smsc/smsc911x.ko 2024-08-19T01:18:24.991 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/net/ethernet/smsc/smsc9420.ko 2024-08-19T01:18:24.991 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/net/ethernet/stmicro 2024-08-19T01:18:24.991 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/net/ethernet/stmicro/stmmac 2024-08-19T01:18:24.991 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/net/ethernet/stmicro/stmmac/dwmac-intel.ko 2024-08-19T01:18:24.991 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/net/ethernet/stmicro/stmmac/stmmac.ko 2024-08-19T01:18:24.991 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/net/ethernet/sun 2024-08-19T01:18:24.991 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/net/ethernet/sun/cassini.ko 2024-08-19T01:18:24.991 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/net/ethernet/sun/niu.ko 2024-08-19T01:18:24.991 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/net/ethernet/sun/sungem.ko 2024-08-19T01:18:24.991 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/net/ethernet/sun/sunhme.ko 2024-08-19T01:18:24.992 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/net/ethernet/tehuti 2024-08-19T01:18:24.992 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/net/ethernet/tehuti/tehuti.ko 2024-08-19T01:18:24.992 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/net/ethernet/ti 2024-08-19T01:18:24.992 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/net/ethernet/ti/tlan.ko 2024-08-19T01:18:24.992 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/net/ethernet/via 2024-08-19T01:18:24.992 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/net/ethernet/via/via-rhine.ko 2024-08-19T01:18:24.992 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/net/ethernet/via/via-velocity.ko 2024-08-19T01:18:24.992 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/net/ethernet/wiznet 2024-08-19T01:18:24.992 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/net/ethernet/wiznet/w5100.ko 2024-08-19T01:18:24.992 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/net/ethernet/wiznet/w5300.ko 2024-08-19T01:18:24.992 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/net/ethernet/xircom 2024-08-19T01:18:24.992 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/net/ethernet/xircom/xirc2ps_cs.ko 2024-08-19T01:18:24.992 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/net/geneve.ko 2024-08-19T01:18:24.993 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/net/hyperv 2024-08-19T01:18:24.993 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/net/hyperv/hv_netvsc.ko 2024-08-19T01:18:24.993 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/net/ieee802154 2024-08-19T01:18:24.993 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/net/ieee802154/fakelb.ko 2024-08-19T01:18:24.993 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/net/ifb.ko 2024-08-19T01:18:24.993 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/net/ipvlan 2024-08-19T01:18:24.993 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/net/ipvlan/ipvlan.ko 2024-08-19T01:18:24.993 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/net/macvlan.ko 2024-08-19T01:18:24.993 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/net/macvtap.ko 2024-08-19T01:18:24.993 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/net/mdio 2024-08-19T01:18:24.993 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/net/mdio.ko 2024-08-19T01:18:24.993 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/net/mdio/mdio-bcm-unimac.ko 2024-08-19T01:18:24.993 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/net/mdio/mdio-bitbang.ko 2024-08-19T01:18:24.993 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/net/mdio/mdio-regmap.ko 2024-08-19T01:18:24.993 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/net/mii.ko 2024-08-19T01:18:24.994 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/net/net_failover.ko 2024-08-19T01:18:24.994 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/net/netconsole.ko 2024-08-19T01:18:24.994 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/net/nlmon.ko 2024-08-19T01:18:24.994 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/net/pcs 2024-08-19T01:18:24.994 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/net/pcs/pcs-lynx.ko 2024-08-19T01:18:24.994 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/net/pcs/pcs_xpcs.ko 2024-08-19T01:18:24.994 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/net/phy 2024-08-19T01:18:24.994 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/net/phy/amd.ko 2024-08-19T01:18:24.994 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/net/phy/bcm-phy-lib.ko 2024-08-19T01:18:24.994 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/net/phy/bcm-phy-ptp.ko 2024-08-19T01:18:24.994 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/net/phy/bcm7xxx.ko 2024-08-19T01:18:24.994 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/net/phy/bcm87xx.ko 2024-08-19T01:18:24.994 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/net/phy/broadcom.ko 2024-08-19T01:18:24.995 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/net/phy/cicada.ko 2024-08-19T01:18:24.995 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/net/phy/davicom.ko 2024-08-19T01:18:24.995 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/net/phy/dp83640.ko 2024-08-19T01:18:24.995 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/net/phy/et1011c.ko 2024-08-19T01:18:24.995 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/net/phy/icplus.ko 2024-08-19T01:18:24.995 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/net/phy/lxt.ko 2024-08-19T01:18:24.995 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/net/phy/marvell.ko 2024-08-19T01:18:24.995 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/net/phy/micrel.ko 2024-08-19T01:18:24.995 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/net/phy/national.ko 2024-08-19T01:18:24.995 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/net/phy/phylink.ko 2024-08-19T01:18:24.995 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/net/phy/qsemi.ko 2024-08-19T01:18:24.995 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/net/phy/realtek.ko 2024-08-19T01:18:24.995 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/net/phy/smsc.ko 2024-08-19T01:18:24.996 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/net/phy/ste10Xp.ko 2024-08-19T01:18:24.996 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/net/phy/vitesse.ko 2024-08-19T01:18:24.996 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/net/ppp 2024-08-19T01:18:24.996 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/net/ppp/bsd_comp.ko 2024-08-19T01:18:24.996 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/net/ppp/ppp_async.ko 2024-08-19T01:18:24.996 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/net/ppp/ppp_deflate.ko 2024-08-19T01:18:24.996 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/net/ppp/ppp_generic.ko 2024-08-19T01:18:24.996 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/net/ppp/ppp_mppe.ko 2024-08-19T01:18:24.996 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/net/ppp/ppp_synctty.ko 2024-08-19T01:18:24.996 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/net/ppp/pppoe.ko 2024-08-19T01:18:24.996 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/net/ppp/pppox.ko 2024-08-19T01:18:24.996 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/net/ppp/pptp.ko 2024-08-19T01:18:24.996 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/net/slip 2024-08-19T01:18:24.996 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/net/slip/slhc.ko 2024-08-19T01:18:24.997 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/net/slip/slip.ko 2024-08-19T01:18:24.997 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/net/sungem_phy.ko 2024-08-19T01:18:24.997 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/net/tap.ko 2024-08-19T01:18:24.997 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/net/team 2024-08-19T01:18:24.997 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/net/team/team.ko 2024-08-19T01:18:24.997 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/net/team/team_mode_activebackup.ko 2024-08-19T01:18:24.997 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/net/team/team_mode_broadcast.ko 2024-08-19T01:18:24.997 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/net/team/team_mode_loadbalance.ko 2024-08-19T01:18:24.997 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/net/team/team_mode_random.ko 2024-08-19T01:18:24.997 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/net/team/team_mode_roundrobin.ko 2024-08-19T01:18:24.997 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/net/tun.ko 2024-08-19T01:18:24.997 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/net/veth.ko 2024-08-19T01:18:24.997 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/net/virtio_net.ko 2024-08-19T01:18:24.997 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/net/vmxnet3 2024-08-19T01:18:24.998 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/net/vmxnet3/vmxnet3.ko 2024-08-19T01:18:24.998 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/net/vxlan 2024-08-19T01:18:24.998 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/net/vxlan/vxlan.ko 2024-08-19T01:18:24.998 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/net/xen-netback 2024-08-19T01:18:24.998 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/net/xen-netback/xen-netback.ko 2024-08-19T01:18:24.998 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/net/xen-netfront.ko 2024-08-19T01:18:24.998 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/ntb 2024-08-19T01:18:24.998 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/ntb/ntb.ko 2024-08-19T01:18:24.998 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/nvme 2024-08-19T01:18:24.998 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/nvme/host 2024-08-19T01:18:24.998 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/nvme/host/nvme-core.ko 2024-08-19T01:18:24.998 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/nvme/host/nvme.ko 2024-08-19T01:18:24.998 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/parport 2024-08-19T01:18:24.999 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/parport/parport.ko 2024-08-19T01:18:24.999 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/parport/parport_cs.ko 2024-08-19T01:18:24.999 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/parport/parport_pc.ko 2024-08-19T01:18:24.999 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/parport/parport_serial.ko 2024-08-19T01:18:24.999 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/pci 2024-08-19T01:18:24.999 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/pci/hotplug 2024-08-19T01:18:24.999 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/pci/hotplug/acpiphp_ibm.ko 2024-08-19T01:18:24.999 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/pci/pcie 2024-08-19T01:18:24.999 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/pci/pcie/aer_inject.ko 2024-08-19T01:18:24.999 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/pci/xen-pcifront.ko 2024-08-19T01:18:24.999 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/pcmcia 2024-08-19T01:18:24.999 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/pcmcia/i82092.ko 2024-08-19T01:18:24.999 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/pcmcia/pd6729.ko 2024-08-19T01:18:24.999 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/pcmcia/yenta_socket.ko 2024-08-19T01:18:25.000 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/pinctrl 2024-08-19T01:18:25.000 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/pinctrl/intel 2024-08-19T01:18:25.000 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/pinctrl/intel/pinctrl-cherryview.ko 2024-08-19T01:18:25.000 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/platform 2024-08-19T01:18:25.000 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/platform/x86 2024-08-19T01:18:25.000 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/platform/x86/acer-wmi.ko 2024-08-19T01:18:25.000 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/platform/x86/acerhdf.ko 2024-08-19T01:18:25.000 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/platform/x86/apple-gmux.ko 2024-08-19T01:18:25.000 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/platform/x86/asus-laptop.ko 2024-08-19T01:18:25.000 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/platform/x86/asus-nb-wmi.ko 2024-08-19T01:18:25.000 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/platform/x86/asus-wmi.ko 2024-08-19T01:18:25.000 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/platform/x86/classmate-laptop.ko 2024-08-19T01:18:25.000 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/platform/x86/eeepc-laptop.ko 2024-08-19T01:18:25.000 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/platform/x86/eeepc-wmi.ko 2024-08-19T01:18:25.001 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/platform/x86/fujitsu-laptop.ko 2024-08-19T01:18:25.001 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/platform/x86/fujitsu-tablet.ko 2024-08-19T01:18:25.001 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/platform/x86/hdaps.ko 2024-08-19T01:18:25.001 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/platform/x86/intel 2024-08-19T01:18:25.001 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/platform/x86/intel/intel-rst.ko 2024-08-19T01:18:25.001 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/platform/x86/intel_ips.ko 2024-08-19T01:18:25.001 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/platform/x86/msi-wmi.ko 2024-08-19T01:18:25.001 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/platform/x86/mxm-wmi.ko 2024-08-19T01:18:25.001 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/platform/x86/panasonic-laptop.ko 2024-08-19T01:18:25.001 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/platform/x86/samsung-laptop.ko 2024-08-19T01:18:25.001 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/platform/x86/samsung-q10.ko 2024-08-19T01:18:25.001 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/platform/x86/topstar-laptop.ko 2024-08-19T01:18:25.001 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/platform/x86/toshiba_bluetooth.ko 2024-08-19T01:18:25.002 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/platform/x86/toshiba_haps.ko 2024-08-19T01:18:25.002 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/platform/x86/wmi-bmof.ko 2024-08-19T01:18:25.002 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/platform/x86/wmi.ko 2024-08-19T01:18:25.002 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/powercap 2024-08-19T01:18:25.002 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/powercap/intel_rapl_common.ko 2024-08-19T01:18:25.002 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/powercap/intel_rapl_msr.ko 2024-08-19T01:18:25.002 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/pps 2024-08-19T01:18:25.002 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/pps/clients 2024-08-19T01:18:25.002 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/pps/clients/pps-gpio.ko 2024-08-19T01:18:25.002 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/pps/clients/pps-ldisc.ko 2024-08-19T01:18:25.002 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/pps/clients/pps_parport.ko 2024-08-19T01:18:25.002 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/pps/pps_core.ko 2024-08-19T01:18:25.002 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/ptp 2024-08-19T01:18:25.002 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/ptp/ptp.ko 2024-08-19T01:18:25.003 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/ptp/ptp_kvm.ko 2024-08-19T01:18:25.003 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/ras 2024-08-19T01:18:25.003 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/ras/amd 2024-08-19T01:18:25.003 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/ras/amd/atl 2024-08-19T01:18:25.003 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/ras/amd/atl/amd_atl.ko 2024-08-19T01:18:25.003 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/ras/amd/fmpm.ko 2024-08-19T01:18:25.003 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/rtc 2024-08-19T01:18:25.003 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/rtc/rtc-bq32k.ko 2024-08-19T01:18:25.003 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/rtc/rtc-ds1286.ko 2024-08-19T01:18:25.003 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/rtc/rtc-ds1307.ko 2024-08-19T01:18:25.003 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/rtc/rtc-ds1374.ko 2024-08-19T01:18:25.003 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/rtc/rtc-ds1511.ko 2024-08-19T01:18:25.003 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/rtc/rtc-ds1553.ko 2024-08-19T01:18:25.004 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/rtc/rtc-ds1672.ko 2024-08-19T01:18:25.004 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/rtc/rtc-ds1742.ko 2024-08-19T01:18:25.004 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/rtc/rtc-ds2404.ko 2024-08-19T01:18:25.004 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/rtc/rtc-ds3232.ko 2024-08-19T01:18:25.004 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/rtc/rtc-em3027.ko 2024-08-19T01:18:25.004 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/rtc/rtc-fm3130.ko 2024-08-19T01:18:25.004 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/rtc/rtc-isl12022.ko 2024-08-19T01:18:25.004 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/rtc/rtc-isl1208.ko 2024-08-19T01:18:25.004 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/rtc/rtc-m41t80.ko 2024-08-19T01:18:25.004 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/rtc/rtc-m48t35.ko 2024-08-19T01:18:25.004 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/rtc/rtc-m48t59.ko 2024-08-19T01:18:25.004 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/rtc/rtc-max6900.ko 2024-08-19T01:18:25.004 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/rtc/rtc-msm6242.ko 2024-08-19T01:18:25.004 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/rtc/rtc-pcf2127.ko 2024-08-19T01:18:25.005 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/rtc/rtc-pcf85063.ko 2024-08-19T01:18:25.005 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/rtc/rtc-pcf8523.ko 2024-08-19T01:18:25.005 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/rtc/rtc-pcf8563.ko 2024-08-19T01:18:25.005 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/rtc/rtc-pcf8583.ko 2024-08-19T01:18:25.005 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/rtc/rtc-rp5c01.ko 2024-08-19T01:18:25.005 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/rtc/rtc-rs5c372.ko 2024-08-19T01:18:25.005 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/rtc/rtc-rv3029c2.ko 2024-08-19T01:18:25.005 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/rtc/rtc-rx8025.ko 2024-08-19T01:18:25.005 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/rtc/rtc-rx8581.ko 2024-08-19T01:18:25.005 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/rtc/rtc-stk17ta8.ko 2024-08-19T01:18:25.005 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/rtc/rtc-x1205.ko 2024-08-19T01:18:25.005 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/scsi 2024-08-19T01:18:25.005 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/scsi/3w-9xxx.ko 2024-08-19T01:18:25.005 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/scsi/3w-sas.ko 2024-08-19T01:18:25.006 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/scsi/3w-xxxx.ko 2024-08-19T01:18:25.006 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/scsi/BusLogic.ko 2024-08-19T01:18:25.006 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/scsi/a100u2w.ko 2024-08-19T01:18:25.006 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/scsi/aacraid 2024-08-19T01:18:25.006 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/scsi/aacraid/aacraid.ko 2024-08-19T01:18:25.006 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/scsi/advansys.ko 2024-08-19T01:18:25.006 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/scsi/aic7xxx 2024-08-19T01:18:25.006 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/scsi/aic7xxx/aic79xx.ko 2024-08-19T01:18:25.006 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/scsi/aic7xxx/aic7xxx.ko 2024-08-19T01:18:25.006 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/scsi/am53c974.ko 2024-08-19T01:18:25.006 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/scsi/arcmsr 2024-08-19T01:18:25.006 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/scsi/arcmsr/arcmsr.ko 2024-08-19T01:18:25.006 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/scsi/atp870u.ko 2024-08-19T01:18:25.007 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/scsi/be2iscsi 2024-08-19T01:18:25.007 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/scsi/be2iscsi/be2iscsi.ko 2024-08-19T01:18:25.007 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/scsi/bfa 2024-08-19T01:18:25.007 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/scsi/bfa/bfa.ko 2024-08-19T01:18:25.007 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/scsi/bnx2fc 2024-08-19T01:18:25.007 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/scsi/bnx2fc/bnx2fc.ko 2024-08-19T01:18:25.007 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/scsi/bnx2i 2024-08-19T01:18:25.007 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/scsi/bnx2i/bnx2i.ko 2024-08-19T01:18:25.007 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/scsi/ch.ko 2024-08-19T01:18:25.007 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/scsi/csiostor 2024-08-19T01:18:25.007 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/scsi/csiostor/csiostor.ko 2024-08-19T01:18:25.007 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/scsi/cxgbi 2024-08-19T01:18:25.007 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/scsi/cxgbi/cxgb3i 2024-08-19T01:18:25.008 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/scsi/cxgbi/cxgb3i/cxgb3i.ko 2024-08-19T01:18:25.008 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/scsi/cxgbi/cxgb4i 2024-08-19T01:18:25.008 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/scsi/cxgbi/cxgb4i/cxgb4i.ko 2024-08-19T01:18:25.008 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/scsi/cxgbi/libcxgbi.ko 2024-08-19T01:18:25.008 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/scsi/dc395x.ko 2024-08-19T01:18:25.008 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/scsi/device_handler 2024-08-19T01:18:25.008 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/scsi/device_handler/scsi_dh_alua.ko 2024-08-19T01:18:25.008 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/scsi/device_handler/scsi_dh_emc.ko 2024-08-19T01:18:25.008 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/scsi/device_handler/scsi_dh_hp_sw.ko 2024-08-19T01:18:25.008 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/scsi/device_handler/scsi_dh_rdac.ko 2024-08-19T01:18:25.008 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/scsi/dmx3191d.ko 2024-08-19T01:18:25.008 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/scsi/esas2r 2024-08-19T01:18:25.008 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/scsi/esas2r/esas2r.ko 2024-08-19T01:18:25.008 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/scsi/esp_scsi.ko 2024-08-19T01:18:25.009 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/scsi/fcoe 2024-08-19T01:18:25.009 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/scsi/fcoe/fcoe.ko 2024-08-19T01:18:25.009 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/scsi/fcoe/libfcoe.ko 2024-08-19T01:18:25.009 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/scsi/fnic 2024-08-19T01:18:25.009 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/scsi/fnic/fnic.ko 2024-08-19T01:18:25.009 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/scsi/hpsa.ko 2024-08-19T01:18:25.009 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/scsi/hptiop.ko 2024-08-19T01:18:25.009 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/scsi/hv_storvsc.ko 2024-08-19T01:18:25.009 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/scsi/initio.ko 2024-08-19T01:18:25.009 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/scsi/ipr.ko 2024-08-19T01:18:25.009 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/scsi/ips.ko 2024-08-19T01:18:25.009 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/scsi/isci 2024-08-19T01:18:25.009 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/scsi/isci/isci.ko 2024-08-19T01:18:25.009 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/scsi/iscsi_boot_sysfs.ko 2024-08-19T01:18:25.010 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/scsi/iscsi_tcp.ko 2024-08-19T01:18:25.010 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/scsi/libfc 2024-08-19T01:18:25.010 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/scsi/libfc/libfc.ko 2024-08-19T01:18:25.010 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/scsi/libiscsi.ko 2024-08-19T01:18:25.010 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/scsi/libiscsi_tcp.ko 2024-08-19T01:18:25.010 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/scsi/libsas 2024-08-19T01:18:25.010 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/scsi/libsas/libsas.ko 2024-08-19T01:18:25.010 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/scsi/lpfc 2024-08-19T01:18:25.010 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/scsi/lpfc/lpfc.ko 2024-08-19T01:18:25.010 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/scsi/megaraid 2024-08-19T01:18:25.010 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/scsi/megaraid.ko 2024-08-19T01:18:25.010 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/scsi/megaraid/megaraid_mbox.ko 2024-08-19T01:18:25.010 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/scsi/megaraid/megaraid_mm.ko 2024-08-19T01:18:25.010 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/scsi/megaraid/megaraid_sas.ko 2024-08-19T01:18:25.011 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/scsi/mpt3sas 2024-08-19T01:18:25.011 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/scsi/mpt3sas/mpt3sas.ko 2024-08-19T01:18:25.011 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/scsi/mvsas 2024-08-19T01:18:25.011 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/scsi/mvsas/mvsas.ko 2024-08-19T01:18:25.011 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/scsi/mvumi.ko 2024-08-19T01:18:25.011 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/scsi/pm8001 2024-08-19T01:18:25.011 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/scsi/pm8001/pm80xx.ko 2024-08-19T01:18:25.011 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/scsi/pmcraid.ko 2024-08-19T01:18:25.011 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/scsi/qla1280.ko 2024-08-19T01:18:25.011 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/scsi/qla2xxx 2024-08-19T01:18:25.011 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/scsi/qla2xxx/qla2xxx.ko 2024-08-19T01:18:25.011 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/scsi/qla2xxx/tcm_qla2xxx.ko 2024-08-19T01:18:25.011 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/scsi/qla4xxx 2024-08-19T01:18:25.011 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/scsi/qla4xxx/qla4xxx.ko 2024-08-19T01:18:25.012 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/scsi/raid_class.ko 2024-08-19T01:18:25.012 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/scsi/scsi_debug.ko 2024-08-19T01:18:25.012 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/scsi/scsi_transport_fc.ko 2024-08-19T01:18:25.012 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/scsi/scsi_transport_iscsi.ko 2024-08-19T01:18:25.012 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/scsi/scsi_transport_sas.ko 2024-08-19T01:18:25.012 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/scsi/scsi_transport_spi.ko 2024-08-19T01:18:25.012 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/scsi/scsi_transport_srp.ko 2024-08-19T01:18:25.012 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/scsi/ses.ko 2024-08-19T01:18:25.012 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/scsi/st.ko 2024-08-19T01:18:25.012 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/scsi/stex.ko 2024-08-19T01:18:25.012 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/scsi/sym53c8xx_2 2024-08-19T01:18:25.012 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/scsi/sym53c8xx_2/sym53c8xx.ko 2024-08-19T01:18:25.012 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/scsi/virtio_scsi.ko 2024-08-19T01:18:25.013 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/scsi/vmw_pvscsi.ko 2024-08-19T01:18:25.013 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/scsi/wd719x.ko 2024-08-19T01:18:25.013 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/ssb 2024-08-19T01:18:25.013 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/ssb/ssb.ko 2024-08-19T01:18:25.013 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/target 2024-08-19T01:18:25.013 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/target/iscsi 2024-08-19T01:18:25.013 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/target/iscsi/iscsi_target_mod.ko 2024-08-19T01:18:25.013 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/target/loopback 2024-08-19T01:18:25.013 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/target/loopback/tcm_loop.ko 2024-08-19T01:18:25.013 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/target/target_core_file.ko 2024-08-19T01:18:25.013 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/target/target_core_iblock.ko 2024-08-19T01:18:25.013 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/target/target_core_mod.ko 2024-08-19T01:18:25.013 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/target/target_core_pscsi.ko 2024-08-19T01:18:25.013 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/target/target_core_user.ko 2024-08-19T01:18:25.014 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/target/tcm_fc 2024-08-19T01:18:25.014 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/target/tcm_fc/tcm_fc.ko 2024-08-19T01:18:25.014 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/thermal 2024-08-19T01:18:25.014 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/thermal/intel 2024-08-19T01:18:25.014 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/thermal/intel/int340x_thermal 2024-08-19T01:18:25.014 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/thermal/intel/int340x_thermal/acpi_thermal_rel.ko 2024-08-19T01:18:25.014 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/thermal/intel/int340x_thermal/int3400_thermal.ko 2024-08-19T01:18:25.014 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/thermal/intel/int340x_thermal/int3401_thermal.ko 2024-08-19T01:18:25.014 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/thermal/intel/int340x_thermal/int3402_thermal.ko 2024-08-19T01:18:25.014 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/thermal/intel/int340x_thermal/int3403_thermal.ko 2024-08-19T01:18:25.014 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/thermal/intel/int340x_thermal/int340x_thermal_zone.ko 2024-08-19T01:18:25.014 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device.ko 2024-08-19T01:18:25.014 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci.ko 2024-08-19T01:18:25.014 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci_legacy.ko 2024-08-19T01:18:25.015 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_mbox.ko 2024-08-19T01:18:25.015 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_power_floor.ko 2024-08-19T01:18:25.015 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rapl.ko 2024-08-19T01:18:25.015 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rfim.ko 2024-08-19T01:18:25.015 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_wt_hint.ko 2024-08-19T01:18:25.015 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_wt_req.ko 2024-08-19T01:18:25.015 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/thermal/intel/intel_soc_dts_iosf.ko 2024-08-19T01:18:25.015 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/thermal/intel/intel_soc_dts_thermal.ko 2024-08-19T01:18:25.015 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/thermal/intel/x86_pkg_temp_thermal.ko 2024-08-19T01:18:25.015 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/tty 2024-08-19T01:18:25.015 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/tty/ipwireless 2024-08-19T01:18:25.015 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/tty/ipwireless/ipwireless.ko 2024-08-19T01:18:25.015 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/tty/n_gsm.ko 2024-08-19T01:18:25.015 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/tty/n_hdlc.ko 2024-08-19T01:18:25.016 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/tty/nozomi.ko 2024-08-19T01:18:25.016 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/tty/serial 2024-08-19T01:18:25.016 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/tty/serial/8250 2024-08-19T01:18:25.016 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/tty/serial/8250/serial_cs.ko 2024-08-19T01:18:25.016 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/tty/serial/arc_uart.ko 2024-08-19T01:18:25.016 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/tty/serial/jsm 2024-08-19T01:18:25.016 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/tty/serial/jsm/jsm.ko 2024-08-19T01:18:25.016 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/ufs 2024-08-19T01:18:25.016 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/ufs/core 2024-08-19T01:18:25.016 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/ufs/core/ufshcd-core.ko 2024-08-19T01:18:25.016 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/ufs/host 2024-08-19T01:18:25.016 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/ufs/host/ufshcd-pci.ko 2024-08-19T01:18:25.016 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/uio 2024-08-19T01:18:25.016 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/uio/uio.ko 2024-08-19T01:18:25.016 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/uio/uio_aec.ko 2024-08-19T01:18:25.017 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/uio/uio_cif.ko 2024-08-19T01:18:25.017 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/uio/uio_pci_generic.ko 2024-08-19T01:18:25.017 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/uio/uio_sercos3.ko 2024-08-19T01:18:25.017 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/usb 2024-08-19T01:18:25.017 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/usb/class 2024-08-19T01:18:25.017 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/usb/class/cdc-acm.ko 2024-08-19T01:18:25.017 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/usb/class/cdc-wdm.ko 2024-08-19T01:18:25.017 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/usb/class/usblp.ko 2024-08-19T01:18:25.017 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/usb/class/usbtmc.ko 2024-08-19T01:18:25.017 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/usb/host 2024-08-19T01:18:25.017 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/usb/host/sl811-hcd.ko 2024-08-19T01:18:25.017 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/usb/image 2024-08-19T01:18:25.017 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/usb/image/mdc800.ko 2024-08-19T01:18:25.018 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/usb/image/microtek.ko 2024-08-19T01:18:25.018 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/usb/misc 2024-08-19T01:18:25.018 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/usb/misc/adutux.ko 2024-08-19T01:18:25.018 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/usb/misc/appledisplay.ko 2024-08-19T01:18:25.018 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/usb/misc/emi26.ko 2024-08-19T01:18:25.018 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/usb/misc/emi62.ko 2024-08-19T01:18:25.018 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/usb/misc/ezusb.ko 2024-08-19T01:18:25.018 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/usb/misc/idmouse.ko 2024-08-19T01:18:25.018 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/usb/misc/iowarrior.ko 2024-08-19T01:18:25.018 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/usb/misc/isight_firmware.ko 2024-08-19T01:18:25.018 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/usb/misc/ldusb.ko 2024-08-19T01:18:25.018 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/usb/misc/legousbtower.ko 2024-08-19T01:18:25.019 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/usb/misc/sisusbvga 2024-08-19T01:18:25.019 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/usb/misc/sisusbvga/sisusbvga.ko 2024-08-19T01:18:25.019 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/usb/misc/trancevibrator.ko 2024-08-19T01:18:25.019 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/usb/misc/usb3503.ko 2024-08-19T01:18:25.019 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/usb/misc/usblcd.ko 2024-08-19T01:18:25.019 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/usb/misc/usbsevseg.ko 2024-08-19T01:18:25.019 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/usb/misc/uss720.ko 2024-08-19T01:18:25.019 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/usb/misc/yurex.ko 2024-08-19T01:18:25.019 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/usb/phy 2024-08-19T01:18:25.019 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/usb/phy/phy-generic.ko 2024-08-19T01:18:25.019 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/usb/serial 2024-08-19T01:18:25.020 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/usb/serial/aircable.ko 2024-08-19T01:18:25.020 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/usb/serial/ark3116.ko 2024-08-19T01:18:25.020 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/usb/serial/belkin_sa.ko 2024-08-19T01:18:25.020 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/usb/serial/ch341.ko 2024-08-19T01:18:25.020 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/usb/serial/cp210x.ko 2024-08-19T01:18:25.020 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/usb/serial/cyberjack.ko 2024-08-19T01:18:25.020 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/usb/serial/cypress_m8.ko 2024-08-19T01:18:25.020 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/usb/serial/digi_acceleport.ko 2024-08-19T01:18:25.020 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/usb/serial/empeg.ko 2024-08-19T01:18:25.020 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/usb/serial/ftdi_sio.ko 2024-08-19T01:18:25.021 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/usb/serial/garmin_gps.ko 2024-08-19T01:18:25.021 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/usb/serial/io_edgeport.ko 2024-08-19T01:18:25.021 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/usb/serial/io_ti.ko 2024-08-19T01:18:25.021 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/usb/serial/ipaq.ko 2024-08-19T01:18:25.021 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/usb/serial/ipw.ko 2024-08-19T01:18:25.021 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/usb/serial/ir-usb.ko 2024-08-19T01:18:25.021 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/usb/serial/iuu_phoenix.ko 2024-08-19T01:18:25.021 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/usb/serial/keyspan.ko 2024-08-19T01:18:25.021 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/usb/serial/keyspan_pda.ko 2024-08-19T01:18:25.021 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/usb/serial/kl5kusb105.ko 2024-08-19T01:18:25.021 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/usb/serial/kobil_sct.ko 2024-08-19T01:18:25.021 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/usb/serial/mct_u232.ko 2024-08-19T01:18:25.022 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/usb/serial/mos7720.ko 2024-08-19T01:18:25.022 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/usb/serial/mos7840.ko 2024-08-19T01:18:25.022 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/usb/serial/navman.ko 2024-08-19T01:18:25.022 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/usb/serial/omninet.ko 2024-08-19T01:18:25.022 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/usb/serial/opticon.ko 2024-08-19T01:18:25.022 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/usb/serial/option.ko 2024-08-19T01:18:25.022 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/usb/serial/oti6858.ko 2024-08-19T01:18:25.022 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/usb/serial/pl2303.ko 2024-08-19T01:18:25.022 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/usb/serial/qcaux.ko 2024-08-19T01:18:25.022 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/usb/serial/qcserial.ko 2024-08-19T01:18:25.023 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/usb/serial/quatech2.ko 2024-08-19T01:18:25.023 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/usb/serial/safe_serial.ko 2024-08-19T01:18:25.023 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/usb/serial/sierra.ko 2024-08-19T01:18:25.023 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/usb/serial/spcp8x5.ko 2024-08-19T01:18:25.023 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/usb/serial/ssu100.ko 2024-08-19T01:18:25.023 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/usb/serial/symbolserial.ko 2024-08-19T01:18:25.023 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/usb/serial/ti_usb_3410_5052.ko 2024-08-19T01:18:25.023 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/usb/serial/usb-serial-simple.ko 2024-08-19T01:18:25.023 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/usb/serial/usb_debug.ko 2024-08-19T01:18:25.023 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/usb/serial/usb_wwan.ko 2024-08-19T01:18:25.023 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/usb/serial/visor.ko 2024-08-19T01:18:25.024 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/usb/serial/whiteheat.ko 2024-08-19T01:18:25.024 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/usb/serial/xsens_mt.ko 2024-08-19T01:18:25.024 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/usb/storage 2024-08-19T01:18:25.024 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/usb/storage/uas.ko 2024-08-19T01:18:25.024 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/usb/storage/ums-alauda.ko 2024-08-19T01:18:25.024 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/usb/storage/ums-cypress.ko 2024-08-19T01:18:25.024 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/usb/storage/ums-datafab.ko 2024-08-19T01:18:25.024 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/usb/storage/ums-eneub6250.ko 2024-08-19T01:18:25.024 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/usb/storage/ums-freecom.ko 2024-08-19T01:18:25.024 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/usb/storage/ums-isd200.ko 2024-08-19T01:18:25.024 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/usb/storage/ums-jumpshot.ko 2024-08-19T01:18:25.025 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/usb/storage/ums-karma.ko 2024-08-19T01:18:25.025 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/usb/storage/ums-onetouch.ko 2024-08-19T01:18:25.025 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/usb/storage/ums-realtek.ko 2024-08-19T01:18:25.025 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/usb/storage/ums-sddr09.ko 2024-08-19T01:18:25.025 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/usb/storage/ums-sddr55.ko 2024-08-19T01:18:25.025 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/usb/storage/ums-usbat.ko 2024-08-19T01:18:25.025 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/usb/storage/usb-storage.ko 2024-08-19T01:18:25.025 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/usb/usbip 2024-08-19T01:18:25.025 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/usb/usbip/usbip-core.ko 2024-08-19T01:18:25.025 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/usb/usbip/usbip-host.ko 2024-08-19T01:18:25.025 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/usb/usbip/vhci-hcd.ko 2024-08-19T01:18:25.025 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/vfio 2024-08-19T01:18:25.025 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/vfio/pci 2024-08-19T01:18:25.026 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/vfio/pci/vfio-pci-core.ko 2024-08-19T01:18:25.026 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/vfio/pci/vfio-pci.ko 2024-08-19T01:18:25.026 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/vfio/vfio.ko 2024-08-19T01:18:25.026 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/vfio/vfio_iommu_type1.ko 2024-08-19T01:18:25.026 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/vhost 2024-08-19T01:18:25.026 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/vhost/vhost.ko 2024-08-19T01:18:25.026 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/vhost/vhost_iotlb.ko 2024-08-19T01:18:25.026 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/vhost/vhost_net.ko 2024-08-19T01:18:25.026 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/vhost/vhost_scsi.ko 2024-08-19T01:18:25.026 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/video 2024-08-19T01:18:25.026 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/video/backlight 2024-08-19T01:18:25.026 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/video/backlight/apple_bl.ko 2024-08-19T01:18:25.026 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/video/backlight/lcd.ko 2024-08-19T01:18:25.026 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/video/backlight/platform_lcd.ko 2024-08-19T01:18:25.027 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/virtio 2024-08-19T01:18:25.027 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/virtio/virtio.ko 2024-08-19T01:18:25.027 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/virtio/virtio_balloon.ko 2024-08-19T01:18:25.027 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/virtio/virtio_mmio.ko 2024-08-19T01:18:25.027 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/virtio/virtio_pci.ko 2024-08-19T01:18:25.027 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/virtio/virtio_pci_legacy_dev.ko 2024-08-19T01:18:25.027 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/virtio/virtio_pci_modern_dev.ko 2024-08-19T01:18:25.027 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/virtio/virtio_ring.ko 2024-08-19T01:18:25.027 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/w1 2024-08-19T01:18:25.027 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/w1/masters 2024-08-19T01:18:25.028 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/w1/masters/ds2482.ko 2024-08-19T01:18:25.028 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/w1/masters/ds2490.ko 2024-08-19T01:18:25.028 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/w1/slaves 2024-08-19T01:18:25.028 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/w1/slaves/w1_ds2406.ko 2024-08-19T01:18:25.028 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/w1/slaves/w1_ds2408.ko 2024-08-19T01:18:25.028 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/w1/slaves/w1_ds2413.ko 2024-08-19T01:18:25.028 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/w1/slaves/w1_ds2423.ko 2024-08-19T01:18:25.028 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/w1/slaves/w1_ds2431.ko 2024-08-19T01:18:25.028 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/w1/slaves/w1_ds2433.ko 2024-08-19T01:18:25.028 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/w1/slaves/w1_ds2780.ko 2024-08-19T01:18:25.029 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/w1/slaves/w1_ds2781.ko 2024-08-19T01:18:25.029 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/w1/slaves/w1_ds28e04.ko 2024-08-19T01:18:25.029 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/w1/slaves/w1_smem.ko 2024-08-19T01:18:25.029 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/w1/slaves/w1_therm.ko 2024-08-19T01:18:25.029 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/w1/wire.ko 2024-08-19T01:18:25.029 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/watchdog 2024-08-19T01:18:25.029 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/watchdog/alim1535_wdt.ko 2024-08-19T01:18:25.029 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/watchdog/alim7101_wdt.ko 2024-08-19T01:18:25.029 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/watchdog/f71808e_wdt.ko 2024-08-19T01:18:25.029 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/watchdog/hpwdt.ko 2024-08-19T01:18:25.029 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/watchdog/i6300esb.ko 2024-08-19T01:18:25.030 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/watchdog/iTCO_vendor_support.ko 2024-08-19T01:18:25.030 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/watchdog/iTCO_wdt.ko 2024-08-19T01:18:25.030 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/watchdog/ib700wdt.ko 2024-08-19T01:18:25.030 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/watchdog/ibmasr.ko 2024-08-19T01:18:25.030 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/watchdog/ie6xx_wdt.ko 2024-08-19T01:18:25.030 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/watchdog/it8712f_wdt.ko 2024-08-19T01:18:25.030 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/watchdog/it87_wdt.ko 2024-08-19T01:18:25.030 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/watchdog/machzwd.ko 2024-08-19T01:18:25.030 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/watchdog/nv_tco.ko 2024-08-19T01:18:25.030 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/watchdog/pcwd_pci.ko 2024-08-19T01:18:25.030 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/watchdog/pcwd_usb.ko 2024-08-19T01:18:25.030 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/watchdog/sbc_fitpc2_wdt.ko 2024-08-19T01:18:25.031 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/watchdog/sch311x_wdt.ko 2024-08-19T01:18:25.031 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/watchdog/softdog.ko 2024-08-19T01:18:25.031 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/watchdog/sp5100_tco.ko 2024-08-19T01:18:25.031 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/watchdog/via_wdt.ko 2024-08-19T01:18:25.031 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/watchdog/w83627hf_wdt.ko 2024-08-19T01:18:25.031 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/watchdog/w83877f_wdt.ko 2024-08-19T01:18:25.031 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/watchdog/w83977f_wdt.ko 2024-08-19T01:18:25.031 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/watchdog/wdt_pci.ko 2024-08-19T01:18:25.031 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/watchdog/xen_wdt.ko 2024-08-19T01:18:25.031 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/xen 2024-08-19T01:18:25.031 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/xen/xen-acpi-processor.ko 2024-08-19T01:18:25.031 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/xen/xen-evtchn.ko 2024-08-19T01:18:25.031 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/xen/xen-gntalloc.ko 2024-08-19T01:18:25.031 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/xen/xen-gntdev.ko 2024-08-19T01:18:25.031 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/xen/xen-pciback 2024-08-19T01:18:25.032 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/xen/xen-pciback/xen-pciback.ko 2024-08-19T01:18:25.032 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/xen/xen-privcmd.ko 2024-08-19T01:18:25.032 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/xen/xenfs 2024-08-19T01:18:25.032 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/xen/xenfs/xenfs.ko 2024-08-19T01:18:25.032 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/fs 2024-08-19T01:18:25.032 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/fs/9p 2024-08-19T01:18:25.032 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/fs/9p/9p.ko 2024-08-19T01:18:25.032 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/fs/affs 2024-08-19T01:18:25.032 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/fs/affs/affs.ko 2024-08-19T01:18:25.032 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/fs/befs 2024-08-19T01:18:25.032 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/fs/befs/befs.ko 2024-08-19T01:18:25.032 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/fs/binfmt_misc.ko 2024-08-19T01:18:25.032 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/fs/btrfs 2024-08-19T01:18:25.032 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/fs/btrfs/btrfs.ko 2024-08-19T01:18:25.033 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/fs/cachefiles 2024-08-19T01:18:25.033 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/fs/cachefiles/cachefiles.ko 2024-08-19T01:18:25.033 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/fs/ceph 2024-08-19T01:18:25.033 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/fs/ceph/ceph.ko 2024-08-19T01:18:25.033 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/fs/coda 2024-08-19T01:18:25.033 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/fs/coda/coda.ko 2024-08-19T01:18:25.033 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/fs/cramfs 2024-08-19T01:18:25.033 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/fs/cramfs/cramfs.ko 2024-08-19T01:18:25.033 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/fs/dlm 2024-08-19T01:18:25.033 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/fs/dlm/dlm.ko 2024-08-19T01:18:25.033 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/fs/ecryptfs 2024-08-19T01:18:25.033 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/fs/ecryptfs/ecryptfs.ko 2024-08-19T01:18:25.033 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/fs/f2fs 2024-08-19T01:18:25.034 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/fs/f2fs/f2fs.ko 2024-08-19T01:18:25.034 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/fs/fat 2024-08-19T01:18:25.034 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/fs/fat/fat.ko 2024-08-19T01:18:25.034 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/fs/fat/msdos.ko 2024-08-19T01:18:25.034 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/fs/fat/vfat.ko 2024-08-19T01:18:25.034 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/fs/fuse 2024-08-19T01:18:25.034 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/fs/fuse/cuse.ko 2024-08-19T01:18:25.034 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/fs/fuse/fuse.ko 2024-08-19T01:18:25.034 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/fs/gfs2 2024-08-19T01:18:25.034 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/fs/gfs2/gfs2.ko 2024-08-19T01:18:25.034 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/fs/hfs 2024-08-19T01:18:25.034 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/fs/hfs/hfs.ko 2024-08-19T01:18:25.034 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/fs/hfsplus 2024-08-19T01:18:25.034 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/fs/hfsplus/hfsplus.ko 2024-08-19T01:18:25.034 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/fs/isofs 2024-08-19T01:18:25.035 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/fs/isofs/isofs.ko 2024-08-19T01:18:25.035 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/fs/jfs 2024-08-19T01:18:25.035 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/fs/jfs/jfs.ko 2024-08-19T01:18:25.035 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/fs/lockd 2024-08-19T01:18:25.035 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/fs/lockd/lockd.ko 2024-08-19T01:18:25.035 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/fs/minix 2024-08-19T01:18:25.035 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/fs/minix/minix.ko 2024-08-19T01:18:25.035 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/fs/netfs 2024-08-19T01:18:25.035 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/fs/netfs/netfs.ko 2024-08-19T01:18:25.035 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/fs/nfs 2024-08-19T01:18:25.035 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/fs/nfs/blocklayout 2024-08-19T01:18:25.035 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/fs/nfs/blocklayout/blocklayoutdriver.ko 2024-08-19T01:18:25.035 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/fs/nfs/filelayout 2024-08-19T01:18:25.035 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/fs/nfs/filelayout/nfs_layout_nfsv41_files.ko 2024-08-19T01:18:25.036 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/fs/nfs/flexfilelayout 2024-08-19T01:18:25.036 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/fs/nfs/flexfilelayout/nfs_layout_flexfiles.ko 2024-08-19T01:18:25.036 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/fs/nfs/nfs.ko 2024-08-19T01:18:25.036 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/fs/nfs/nfsv3.ko 2024-08-19T01:18:25.036 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/fs/nfs/nfsv4.ko 2024-08-19T01:18:25.036 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/fs/nfs_common 2024-08-19T01:18:25.036 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/fs/nfs_common/grace.ko 2024-08-19T01:18:25.036 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/fs/nfs_common/nfs_acl.ko 2024-08-19T01:18:25.036 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/fs/nfsd 2024-08-19T01:18:25.036 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/fs/nfsd/nfsd.ko 2024-08-19T01:18:25.036 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/fs/nilfs2 2024-08-19T01:18:25.036 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/fs/nilfs2/nilfs2.ko 2024-08-19T01:18:25.036 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/fs/nls 2024-08-19T01:18:25.036 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/fs/nls/mac-celtic.ko 2024-08-19T01:18:25.037 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/fs/nls/mac-centeuro.ko 2024-08-19T01:18:25.037 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/fs/nls/mac-croatian.ko 2024-08-19T01:18:25.037 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/fs/nls/mac-cyrillic.ko 2024-08-19T01:18:25.037 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/fs/nls/mac-gaelic.ko 2024-08-19T01:18:25.037 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/fs/nls/mac-greek.ko 2024-08-19T01:18:25.037 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/fs/nls/mac-iceland.ko 2024-08-19T01:18:25.037 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/fs/nls/mac-inuit.ko 2024-08-19T01:18:25.037 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/fs/nls/mac-roman.ko 2024-08-19T01:18:25.037 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/fs/nls/mac-romanian.ko 2024-08-19T01:18:25.037 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/fs/nls/mac-turkish.ko 2024-08-19T01:18:25.037 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/fs/nls/nls_cp1250.ko 2024-08-19T01:18:25.037 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/fs/nls/nls_cp1251.ko 2024-08-19T01:18:25.037 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/fs/nls/nls_cp1255.ko 2024-08-19T01:18:25.037 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/fs/nls/nls_cp737.ko 2024-08-19T01:18:25.038 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/fs/nls/nls_cp775.ko 2024-08-19T01:18:25.038 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/fs/nls/nls_cp850.ko 2024-08-19T01:18:25.038 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/fs/nls/nls_cp852.ko 2024-08-19T01:18:25.038 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/fs/nls/nls_cp855.ko 2024-08-19T01:18:25.038 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/fs/nls/nls_cp857.ko 2024-08-19T01:18:25.038 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/fs/nls/nls_cp860.ko 2024-08-19T01:18:25.038 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/fs/nls/nls_cp861.ko 2024-08-19T01:18:25.038 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/fs/nls/nls_cp862.ko 2024-08-19T01:18:25.038 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/fs/nls/nls_cp863.ko 2024-08-19T01:18:25.038 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/fs/nls/nls_cp864.ko 2024-08-19T01:18:25.038 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/fs/nls/nls_cp865.ko 2024-08-19T01:18:25.038 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/fs/nls/nls_cp866.ko 2024-08-19T01:18:25.038 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/fs/nls/nls_cp869.ko 2024-08-19T01:18:25.038 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/fs/nls/nls_cp874.ko 2024-08-19T01:18:25.039 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/fs/nls/nls_cp932.ko 2024-08-19T01:18:25.039 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/fs/nls/nls_cp936.ko 2024-08-19T01:18:25.039 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/fs/nls/nls_cp949.ko 2024-08-19T01:18:25.039 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/fs/nls/nls_cp950.ko 2024-08-19T01:18:25.039 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/fs/nls/nls_euc-jp.ko 2024-08-19T01:18:25.039 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/fs/nls/nls_iso8859-1.ko 2024-08-19T01:18:25.039 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/fs/nls/nls_iso8859-13.ko 2024-08-19T01:18:25.039 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/fs/nls/nls_iso8859-14.ko 2024-08-19T01:18:25.039 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/fs/nls/nls_iso8859-15.ko 2024-08-19T01:18:25.039 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/fs/nls/nls_iso8859-2.ko 2024-08-19T01:18:25.039 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/fs/nls/nls_iso8859-3.ko 2024-08-19T01:18:25.039 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/fs/nls/nls_iso8859-4.ko 2024-08-19T01:18:25.039 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/fs/nls/nls_iso8859-5.ko 2024-08-19T01:18:25.040 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/fs/nls/nls_iso8859-6.ko 2024-08-19T01:18:25.040 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/fs/nls/nls_iso8859-7.ko 2024-08-19T01:18:25.040 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/fs/nls/nls_iso8859-9.ko 2024-08-19T01:18:25.040 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/fs/nls/nls_koi8-r.ko 2024-08-19T01:18:25.040 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/fs/nls/nls_koi8-ru.ko 2024-08-19T01:18:25.040 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/fs/nls/nls_koi8-u.ko 2024-08-19T01:18:25.040 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/fs/nls/nls_ucs2_utils.ko 2024-08-19T01:18:25.040 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/fs/nls/nls_utf8.ko 2024-08-19T01:18:25.040 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/fs/ocfs2 2024-08-19T01:18:25.040 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/fs/ocfs2/cluster 2024-08-19T01:18:25.040 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/fs/ocfs2/cluster/ocfs2_nodemanager.ko 2024-08-19T01:18:25.040 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/fs/ocfs2/dlm 2024-08-19T01:18:25.040 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/fs/ocfs2/dlm/ocfs2_dlm.ko 2024-08-19T01:18:25.040 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/fs/ocfs2/dlmfs 2024-08-19T01:18:25.040 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/fs/ocfs2/dlmfs/ocfs2_dlmfs.ko 2024-08-19T01:18:25.041 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/fs/ocfs2/ocfs2.ko 2024-08-19T01:18:25.041 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/fs/ocfs2/ocfs2_stack_o2cb.ko 2024-08-19T01:18:25.041 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/fs/ocfs2/ocfs2_stack_user.ko 2024-08-19T01:18:25.041 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/fs/ocfs2/ocfs2_stackglue.ko 2024-08-19T01:18:25.041 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/fs/overlayfs 2024-08-19T01:18:25.041 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/fs/overlayfs/overlay.ko 2024-08-19T01:18:25.041 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/fs/pstore 2024-08-19T01:18:25.041 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/fs/pstore/ramoops.ko 2024-08-19T01:18:25.041 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/fs/reiserfs 2024-08-19T01:18:25.041 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/fs/reiserfs/reiserfs.ko 2024-08-19T01:18:25.041 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/fs/romfs 2024-08-19T01:18:25.041 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/fs/romfs/romfs.ko 2024-08-19T01:18:25.041 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/fs/smb 2024-08-19T01:18:25.042 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/fs/smb/client 2024-08-19T01:18:25.042 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/fs/smb/client/cifs.ko 2024-08-19T01:18:25.042 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/fs/smb/common 2024-08-19T01:18:25.042 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/fs/smb/common/cifs_arc4.ko 2024-08-19T01:18:25.042 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/fs/smb/common/cifs_md4.ko 2024-08-19T01:18:25.042 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/fs/squashfs 2024-08-19T01:18:25.042 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/fs/squashfs/squashfs.ko 2024-08-19T01:18:25.042 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/fs/sysv 2024-08-19T01:18:25.042 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/fs/sysv/sysv.ko 2024-08-19T01:18:25.042 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/fs/udf 2024-08-19T01:18:25.042 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/fs/udf/udf.ko 2024-08-19T01:18:25.042 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/fs/ufs 2024-08-19T01:18:25.042 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/fs/ufs/ufs.ko 2024-08-19T01:18:25.043 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/fs/xfs 2024-08-19T01:18:25.043 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/fs/xfs/xfs.ko 2024-08-19T01:18:25.043 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/kernel 2024-08-19T01:18:25.043 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/kernel/rcu 2024-08-19T01:18:25.043 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/kernel/rcu/rcutorture.ko 2024-08-19T01:18:25.043 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/kernel/torture.ko 2024-08-19T01:18:25.043 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/kernel/trace 2024-08-19T01:18:25.043 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/kernel/trace/ring_buffer_benchmark.ko 2024-08-19T01:18:25.043 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/lib 2024-08-19T01:18:25.043 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/lib/asn1_encoder.ko 2024-08-19T01:18:25.043 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/lib/crc-itu-t.ko 2024-08-19T01:18:25.043 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/lib/crc8.ko 2024-08-19T01:18:25.043 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/lib/crypto 2024-08-19T01:18:25.043 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/lib/crypto/libaescfb.ko 2024-08-19T01:18:25.044 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/lib/crypto/libarc4.ko 2024-08-19T01:18:25.044 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/lib/crypto/libdes.ko 2024-08-19T01:18:25.044 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/lib/libcrc32c.ko 2024-08-19T01:18:25.044 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/lib/lru_cache.ko 2024-08-19T01:18:25.044 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/lib/lz4 2024-08-19T01:18:25.044 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/lib/lz4/lz4_compress.ko 2024-08-19T01:18:25.044 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/lib/lz4/lz4hc_compress.ko 2024-08-19T01:18:25.044 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/lib/math 2024-08-19T01:18:25.044 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/lib/math/cordic.ko 2024-08-19T01:18:25.044 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/lib/raid6 2024-08-19T01:18:25.044 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/lib/raid6/raid6_pq.ko 2024-08-19T01:18:25.044 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/lib/reed_solomon 2024-08-19T01:18:25.044 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/lib/reed_solomon/reed_solomon.ko 2024-08-19T01:18:25.044 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/lib/ts_bm.ko 2024-08-19T01:18:25.045 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/lib/ts_fsm.ko 2024-08-19T01:18:25.045 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/lib/ts_kmp.ko 2024-08-19T01:18:25.045 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/lib/zstd 2024-08-19T01:18:25.045 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/lib/zstd/zstd_compress.ko 2024-08-19T01:18:25.045 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/mm 2024-08-19T01:18:25.045 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/mm/hwpoison-inject.ko 2024-08-19T01:18:25.045 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net 2024-08-19T01:18:25.045 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/6lowpan 2024-08-19T01:18:25.045 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/6lowpan/6lowpan.ko 2024-08-19T01:18:25.045 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/6lowpan/nhc_dest.ko 2024-08-19T01:18:25.045 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/6lowpan/nhc_fragment.ko 2024-08-19T01:18:25.045 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/6lowpan/nhc_hop.ko 2024-08-19T01:18:25.045 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/6lowpan/nhc_ipv6.ko 2024-08-19T01:18:25.045 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/6lowpan/nhc_mobility.ko 2024-08-19T01:18:25.046 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/6lowpan/nhc_routing.ko 2024-08-19T01:18:25.046 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/6lowpan/nhc_udp.ko 2024-08-19T01:18:25.046 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/802 2024-08-19T01:18:25.046 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/802/garp.ko 2024-08-19T01:18:25.046 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/802/mrp.ko 2024-08-19T01:18:25.046 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/802/p8022.ko 2024-08-19T01:18:25.046 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/802/psnap.ko 2024-08-19T01:18:25.046 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/802/stp.ko 2024-08-19T01:18:25.046 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/8021q 2024-08-19T01:18:25.046 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/8021q/8021q.ko 2024-08-19T01:18:25.046 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/9p 2024-08-19T01:18:25.046 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/9p/9pnet.ko 2024-08-19T01:18:25.046 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/9p/9pnet_fd.ko 2024-08-19T01:18:25.047 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/9p/9pnet_rdma.ko 2024-08-19T01:18:25.047 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/9p/9pnet_virtio.ko 2024-08-19T01:18:25.047 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/batman-adv 2024-08-19T01:18:25.047 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/batman-adv/batman-adv.ko 2024-08-19T01:18:25.047 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/bridge 2024-08-19T01:18:25.047 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/bridge/br_netfilter.ko 2024-08-19T01:18:25.047 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/bridge/bridge.ko 2024-08-19T01:18:25.047 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/bridge/netfilter 2024-08-19T01:18:25.047 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/bridge/netfilter/ebt_802_3.ko 2024-08-19T01:18:25.047 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/bridge/netfilter/ebt_among.ko 2024-08-19T01:18:25.047 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/bridge/netfilter/ebt_arp.ko 2024-08-19T01:18:25.047 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/bridge/netfilter/ebt_arpreply.ko 2024-08-19T01:18:25.047 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/bridge/netfilter/ebt_dnat.ko 2024-08-19T01:18:25.047 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/bridge/netfilter/ebt_ip.ko 2024-08-19T01:18:25.047 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/bridge/netfilter/ebt_ip6.ko 2024-08-19T01:18:25.048 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/bridge/netfilter/ebt_limit.ko 2024-08-19T01:18:25.048 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/bridge/netfilter/ebt_log.ko 2024-08-19T01:18:25.048 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/bridge/netfilter/ebt_mark.ko 2024-08-19T01:18:25.048 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/bridge/netfilter/ebt_mark_m.ko 2024-08-19T01:18:25.048 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/bridge/netfilter/ebt_nflog.ko 2024-08-19T01:18:25.048 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/bridge/netfilter/ebt_pkttype.ko 2024-08-19T01:18:25.048 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/bridge/netfilter/ebt_redirect.ko 2024-08-19T01:18:25.048 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/bridge/netfilter/ebt_snat.ko 2024-08-19T01:18:25.048 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/bridge/netfilter/ebt_stp.ko 2024-08-19T01:18:25.048 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/bridge/netfilter/ebt_vlan.ko 2024-08-19T01:18:25.048 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/bridge/netfilter/ebtable_broute.ko 2024-08-19T01:18:25.048 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/bridge/netfilter/ebtable_filter.ko 2024-08-19T01:18:25.048 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/bridge/netfilter/ebtable_nat.ko 2024-08-19T01:18:25.049 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/bridge/netfilter/ebtables.ko 2024-08-19T01:18:25.049 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/bridge/netfilter/nf_conntrack_bridge.ko 2024-08-19T01:18:25.049 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/bridge/netfilter/nft_meta_bridge.ko 2024-08-19T01:18:25.049 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/bridge/netfilter/nft_reject_bridge.ko 2024-08-19T01:18:25.049 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/ceph 2024-08-19T01:18:25.049 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/ceph/libceph.ko 2024-08-19T01:18:25.049 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/core 2024-08-19T01:18:25.049 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/core/failover.ko 2024-08-19T01:18:25.049 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/core/pktgen.ko 2024-08-19T01:18:25.049 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/dccp 2024-08-19T01:18:25.049 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/dccp/dccp.ko 2024-08-19T01:18:25.049 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/dccp/dccp_diag.ko 2024-08-19T01:18:25.049 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/dccp/dccp_ipv4.ko 2024-08-19T01:18:25.049 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/dccp/dccp_ipv6.ko 2024-08-19T01:18:25.049 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/dns_resolver 2024-08-19T01:18:25.050 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/dns_resolver/dns_resolver.ko 2024-08-19T01:18:25.050 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/dsa 2024-08-19T01:18:25.050 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/dsa/dsa_core.ko 2024-08-19T01:18:25.050 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/dsa/tag_brcm.ko 2024-08-19T01:18:25.050 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/dsa/tag_dsa.ko 2024-08-19T01:18:25.050 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/dsa/tag_none.ko 2024-08-19T01:18:25.050 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/dsa/tag_trailer.ko 2024-08-19T01:18:25.050 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/ieee802154 2024-08-19T01:18:25.050 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/ieee802154/6lowpan 2024-08-19T01:18:25.050 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/ieee802154/6lowpan/ieee802154_6lowpan.ko 2024-08-19T01:18:25.050 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/ieee802154/ieee802154.ko 2024-08-19T01:18:25.050 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/ieee802154/ieee802154_socket.ko 2024-08-19T01:18:25.050 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/ipv4 2024-08-19T01:18:25.050 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/ipv4/ah4.ko 2024-08-19T01:18:25.051 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/ipv4/esp4.ko 2024-08-19T01:18:25.051 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/ipv4/fou.ko 2024-08-19T01:18:25.051 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/ipv4/gre.ko 2024-08-19T01:18:25.051 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/ipv4/inet_diag.ko 2024-08-19T01:18:25.051 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/ipv4/ip_gre.ko 2024-08-19T01:18:25.051 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/ipv4/ip_tunnel.ko 2024-08-19T01:18:25.051 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/ipv4/ip_vti.ko 2024-08-19T01:18:25.051 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/ipv4/ipcomp.ko 2024-08-19T01:18:25.051 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/ipv4/ipip.ko 2024-08-19T01:18:25.051 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/ipv4/netfilter 2024-08-19T01:18:25.051 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/ipv4/netfilter/arp_tables.ko 2024-08-19T01:18:25.051 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/ipv4/netfilter/arpt_mangle.ko 2024-08-19T01:18:25.051 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/ipv4/netfilter/arptable_filter.ko 2024-08-19T01:18:25.052 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/ipv4/netfilter/ipt_ECN.ko 2024-08-19T01:18:25.052 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/ipv4/netfilter/ipt_SYNPROXY.ko 2024-08-19T01:18:25.052 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/ipv4/netfilter/ipt_ah.ko 2024-08-19T01:18:25.052 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/ipv4/netfilter/ipt_rpfilter.ko 2024-08-19T01:18:25.052 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/ipv4/netfilter/iptable_mangle.ko 2024-08-19T01:18:25.052 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/ipv4/netfilter/iptable_nat.ko 2024-08-19T01:18:25.052 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/ipv4/netfilter/iptable_raw.ko 2024-08-19T01:18:25.052 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/ipv4/netfilter/iptable_security.ko 2024-08-19T01:18:25.052 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/ipv4/netfilter/nf_defrag_ipv4.ko 2024-08-19T01:18:25.052 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/ipv4/netfilter/nf_dup_ipv4.ko 2024-08-19T01:18:25.052 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/ipv4/netfilter/nf_nat_h323.ko 2024-08-19T01:18:25.052 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/ipv4/netfilter/nf_nat_pptp.ko 2024-08-19T01:18:25.052 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/ipv4/netfilter/nf_nat_snmp_basic.ko 2024-08-19T01:18:25.052 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/ipv4/netfilter/nf_socket_ipv4.ko 2024-08-19T01:18:25.053 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/ipv4/netfilter/nf_tproxy_ipv4.ko 2024-08-19T01:18:25.053 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/ipv4/netfilter/nft_dup_ipv4.ko 2024-08-19T01:18:25.053 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/ipv4/netfilter/nft_fib_ipv4.ko 2024-08-19T01:18:25.053 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/ipv4/netfilter/nft_reject_ipv4.ko 2024-08-19T01:18:25.053 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/ipv4/tcp_bic.ko 2024-08-19T01:18:25.053 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/ipv4/tcp_dctcp.ko 2024-08-19T01:18:25.053 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/ipv4/tcp_diag.ko 2024-08-19T01:18:25.053 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/ipv4/tcp_highspeed.ko 2024-08-19T01:18:25.053 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/ipv4/tcp_htcp.ko 2024-08-19T01:18:25.053 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/ipv4/tcp_hybla.ko 2024-08-19T01:18:25.053 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/ipv4/tcp_illinois.ko 2024-08-19T01:18:25.053 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/ipv4/tcp_lp.ko 2024-08-19T01:18:25.053 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/ipv4/tcp_scalable.ko 2024-08-19T01:18:25.053 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/ipv4/tcp_vegas.ko 2024-08-19T01:18:25.054 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/ipv4/tcp_veno.ko 2024-08-19T01:18:25.054 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/ipv4/tcp_westwood.ko 2024-08-19T01:18:25.054 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/ipv4/tcp_yeah.ko 2024-08-19T01:18:25.054 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/ipv4/tunnel4.ko 2024-08-19T01:18:25.054 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/ipv4/udp_diag.ko 2024-08-19T01:18:25.054 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/ipv4/udp_tunnel.ko 2024-08-19T01:18:25.054 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/ipv4/xfrm4_tunnel.ko 2024-08-19T01:18:25.054 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/ipv6 2024-08-19T01:18:25.054 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/ipv6/ah6.ko 2024-08-19T01:18:25.054 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/ipv6/esp6.ko 2024-08-19T01:18:25.054 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/ipv6/fou6.ko 2024-08-19T01:18:25.054 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/ipv6/ip6_tunnel.ko 2024-08-19T01:18:25.054 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/ipv6/ip6_udp_tunnel.ko 2024-08-19T01:18:25.055 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/ipv6/ip6_vti.ko 2024-08-19T01:18:25.055 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/ipv6/ipcomp6.ko 2024-08-19T01:18:25.055 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/ipv6/netfilter 2024-08-19T01:18:25.055 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/ipv6/netfilter/ip6_tables.ko 2024-08-19T01:18:25.055 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/ipv6/netfilter/ip6t_REJECT.ko 2024-08-19T01:18:25.055 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/ipv6/netfilter/ip6t_SYNPROXY.ko 2024-08-19T01:18:25.055 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/ipv6/netfilter/ip6t_ah.ko 2024-08-19T01:18:25.055 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/ipv6/netfilter/ip6t_eui64.ko 2024-08-19T01:18:25.055 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/ipv6/netfilter/ip6t_frag.ko 2024-08-19T01:18:25.055 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/ipv6/netfilter/ip6t_hbh.ko 2024-08-19T01:18:25.055 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/ipv6/netfilter/ip6t_ipv6header.ko 2024-08-19T01:18:25.055 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/ipv6/netfilter/ip6t_mh.ko 2024-08-19T01:18:25.055 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/ipv6/netfilter/ip6t_rpfilter.ko 2024-08-19T01:18:25.055 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/ipv6/netfilter/ip6t_rt.ko 2024-08-19T01:18:25.055 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/ipv6/netfilter/ip6table_filter.ko 2024-08-19T01:18:25.056 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/ipv6/netfilter/ip6table_mangle.ko 2024-08-19T01:18:25.056 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/ipv6/netfilter/ip6table_nat.ko 2024-08-19T01:18:25.056 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/ipv6/netfilter/ip6table_raw.ko 2024-08-19T01:18:25.056 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/ipv6/netfilter/ip6table_security.ko 2024-08-19T01:18:25.056 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/ipv6/netfilter/nf_defrag_ipv6.ko 2024-08-19T01:18:25.056 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/ipv6/netfilter/nf_dup_ipv6.ko 2024-08-19T01:18:25.056 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/ipv6/netfilter/nf_reject_ipv6.ko 2024-08-19T01:18:25.056 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/ipv6/netfilter/nf_socket_ipv6.ko 2024-08-19T01:18:25.056 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/ipv6/netfilter/nf_tproxy_ipv6.ko 2024-08-19T01:18:25.056 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/ipv6/netfilter/nft_dup_ipv6.ko 2024-08-19T01:18:25.056 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/ipv6/netfilter/nft_fib_ipv6.ko 2024-08-19T01:18:25.056 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/ipv6/netfilter/nft_reject_ipv6.ko 2024-08-19T01:18:25.056 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/ipv6/sit.ko 2024-08-19T01:18:25.056 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/ipv6/tunnel6.ko 2024-08-19T01:18:25.057 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/ipv6/xfrm6_tunnel.ko 2024-08-19T01:18:25.057 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/key 2024-08-19T01:18:25.057 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/key/af_key.ko 2024-08-19T01:18:25.057 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/l2tp 2024-08-19T01:18:25.057 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/l2tp/l2tp_core.ko 2024-08-19T01:18:25.057 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/l2tp/l2tp_debugfs.ko 2024-08-19T01:18:25.057 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/l2tp/l2tp_eth.ko 2024-08-19T01:18:25.057 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/l2tp/l2tp_ip.ko 2024-08-19T01:18:25.057 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/l2tp/l2tp_ip6.ko 2024-08-19T01:18:25.057 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/l2tp/l2tp_netlink.ko 2024-08-19T01:18:25.057 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/l2tp/l2tp_ppp.ko 2024-08-19T01:18:25.057 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/llc 2024-08-19T01:18:25.057 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/llc/llc.ko 2024-08-19T01:18:25.057 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/mac802154 2024-08-19T01:18:25.058 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/mac802154/mac802154.ko 2024-08-19T01:18:25.058 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/mpls 2024-08-19T01:18:25.058 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/mpls/mpls_gso.ko 2024-08-19T01:18:25.058 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/netfilter 2024-08-19T01:18:25.058 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/netfilter/ipset 2024-08-19T01:18:25.058 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/netfilter/ipset/ip_set.ko 2024-08-19T01:18:25.058 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/netfilter/ipset/ip_set_bitmap_ip.ko 2024-08-19T01:18:25.058 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/netfilter/ipset/ip_set_bitmap_ipmac.ko 2024-08-19T01:18:25.058 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/netfilter/ipset/ip_set_bitmap_port.ko 2024-08-19T01:18:25.058 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/netfilter/ipset/ip_set_hash_ip.ko 2024-08-19T01:18:25.058 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/netfilter/ipset/ip_set_hash_ipmark.ko 2024-08-19T01:18:25.058 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/netfilter/ipset/ip_set_hash_ipport.ko 2024-08-19T01:18:25.058 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/netfilter/ipset/ip_set_hash_ipportip.ko 2024-08-19T01:18:25.059 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/netfilter/ipset/ip_set_hash_ipportnet.ko 2024-08-19T01:18:25.059 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/netfilter/ipset/ip_set_hash_mac.ko 2024-08-19T01:18:25.059 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/netfilter/ipset/ip_set_hash_net.ko 2024-08-19T01:18:25.059 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/netfilter/ipset/ip_set_hash_netiface.ko 2024-08-19T01:18:25.059 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/netfilter/ipset/ip_set_hash_netnet.ko 2024-08-19T01:18:25.059 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/netfilter/ipset/ip_set_hash_netport.ko 2024-08-19T01:18:25.059 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/netfilter/ipset/ip_set_hash_netportnet.ko 2024-08-19T01:18:25.059 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/netfilter/ipset/ip_set_list_set.ko 2024-08-19T01:18:25.059 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/netfilter/ipvs 2024-08-19T01:18:25.059 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/netfilter/ipvs/ip_vs.ko 2024-08-19T01:18:25.059 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/netfilter/ipvs/ip_vs_dh.ko 2024-08-19T01:18:25.059 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/netfilter/ipvs/ip_vs_fo.ko 2024-08-19T01:18:25.059 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/netfilter/ipvs/ip_vs_ftp.ko 2024-08-19T01:18:25.059 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/netfilter/ipvs/ip_vs_lblc.ko 2024-08-19T01:18:25.060 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/netfilter/ipvs/ip_vs_lblcr.ko 2024-08-19T01:18:25.060 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/netfilter/ipvs/ip_vs_lc.ko 2024-08-19T01:18:25.060 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/netfilter/ipvs/ip_vs_nq.ko 2024-08-19T01:18:25.060 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/netfilter/ipvs/ip_vs_pe_sip.ko 2024-08-19T01:18:25.060 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/netfilter/ipvs/ip_vs_rr.ko 2024-08-19T01:18:25.060 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/netfilter/ipvs/ip_vs_sed.ko 2024-08-19T01:18:25.060 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/netfilter/ipvs/ip_vs_sh.ko 2024-08-19T01:18:25.060 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/netfilter/ipvs/ip_vs_wlc.ko 2024-08-19T01:18:25.060 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/netfilter/ipvs/ip_vs_wrr.ko 2024-08-19T01:18:25.060 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/netfilter/nf_conncount.ko 2024-08-19T01:18:25.060 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/netfilter/nf_conntrack.ko 2024-08-19T01:18:25.060 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/netfilter/nf_conntrack_amanda.ko 2024-08-19T01:18:25.060 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/netfilter/nf_conntrack_broadcast.ko 2024-08-19T01:18:25.060 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/netfilter/nf_conntrack_ftp.ko 2024-08-19T01:18:25.061 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/netfilter/nf_conntrack_h323.ko 2024-08-19T01:18:25.061 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/netfilter/nf_conntrack_irc.ko 2024-08-19T01:18:25.061 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/netfilter/nf_conntrack_netbios_ns.ko 2024-08-19T01:18:25.061 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/netfilter/nf_conntrack_netlink.ko 2024-08-19T01:18:25.061 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/netfilter/nf_conntrack_pptp.ko 2024-08-19T01:18:25.061 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/netfilter/nf_conntrack_sane.ko 2024-08-19T01:18:25.061 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/netfilter/nf_conntrack_sip.ko 2024-08-19T01:18:25.061 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/netfilter/nf_conntrack_snmp.ko 2024-08-19T01:18:25.061 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/netfilter/nf_conntrack_tftp.ko 2024-08-19T01:18:25.061 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/netfilter/nf_dup_netdev.ko 2024-08-19T01:18:25.061 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/netfilter/nf_flow_table.ko 2024-08-19T01:18:25.061 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/netfilter/nf_flow_table_inet.ko 2024-08-19T01:18:25.061 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/netfilter/nf_log_syslog.ko 2024-08-19T01:18:25.061 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/netfilter/nf_nat.ko 2024-08-19T01:18:25.062 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/netfilter/nf_nat_amanda.ko 2024-08-19T01:18:25.062 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/netfilter/nf_nat_ftp.ko 2024-08-19T01:18:25.062 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/netfilter/nf_nat_irc.ko 2024-08-19T01:18:25.062 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/netfilter/nf_nat_sip.ko 2024-08-19T01:18:25.062 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/netfilter/nf_nat_tftp.ko 2024-08-19T01:18:25.062 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/netfilter/nf_synproxy_core.ko 2024-08-19T01:18:25.062 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/netfilter/nf_tables.ko 2024-08-19T01:18:25.062 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/netfilter/nfnetlink.ko 2024-08-19T01:18:25.062 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/netfilter/nfnetlink_acct.ko 2024-08-19T01:18:25.062 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/netfilter/nfnetlink_log.ko 2024-08-19T01:18:25.062 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/netfilter/nfnetlink_osf.ko 2024-08-19T01:18:25.062 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/netfilter/nfnetlink_queue.ko 2024-08-19T01:18:25.062 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/netfilter/nft_chain_nat.ko 2024-08-19T01:18:25.062 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/netfilter/nft_compat.ko 2024-08-19T01:18:25.063 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/netfilter/nft_ct.ko 2024-08-19T01:18:25.063 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/netfilter/nft_dup_netdev.ko 2024-08-19T01:18:25.063 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/netfilter/nft_fib.ko 2024-08-19T01:18:25.063 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/netfilter/nft_fwd_netdev.ko 2024-08-19T01:18:25.063 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/netfilter/nft_hash.ko 2024-08-19T01:18:25.063 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/netfilter/nft_limit.ko 2024-08-19T01:18:25.063 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/netfilter/nft_log.ko 2024-08-19T01:18:25.063 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/netfilter/nft_masq.ko 2024-08-19T01:18:25.063 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/netfilter/nft_nat.ko 2024-08-19T01:18:25.063 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/netfilter/nft_numgen.ko 2024-08-19T01:18:25.063 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/netfilter/nft_queue.ko 2024-08-19T01:18:25.063 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/netfilter/nft_quota.ko 2024-08-19T01:18:25.063 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/netfilter/nft_redir.ko 2024-08-19T01:18:25.063 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/netfilter/nft_reject.ko 2024-08-19T01:18:25.064 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/netfilter/nft_reject_inet.ko 2024-08-19T01:18:25.064 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/netfilter/nft_socket.ko 2024-08-19T01:18:25.064 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/netfilter/nft_tunnel.ko 2024-08-19T01:18:25.064 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/netfilter/xt_AUDIT.ko 2024-08-19T01:18:25.064 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/netfilter/xt_CHECKSUM.ko 2024-08-19T01:18:25.064 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/netfilter/xt_CLASSIFY.ko 2024-08-19T01:18:25.064 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/netfilter/xt_CONNSECMARK.ko 2024-08-19T01:18:25.064 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/netfilter/xt_CT.ko 2024-08-19T01:18:25.064 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/netfilter/xt_DSCP.ko 2024-08-19T01:18:25.064 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/netfilter/xt_HL.ko 2024-08-19T01:18:25.064 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/netfilter/xt_HMARK.ko 2024-08-19T01:18:25.064 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/netfilter/xt_IDLETIMER.ko 2024-08-19T01:18:25.064 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/netfilter/xt_LED.ko 2024-08-19T01:18:25.064 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/netfilter/xt_LOG.ko 2024-08-19T01:18:25.064 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/netfilter/xt_MASQUERADE.ko 2024-08-19T01:18:25.065 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/netfilter/xt_NETMAP.ko 2024-08-19T01:18:25.065 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/netfilter/xt_NFLOG.ko 2024-08-19T01:18:25.065 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/netfilter/xt_NFQUEUE.ko 2024-08-19T01:18:25.065 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/netfilter/xt_RATEEST.ko 2024-08-19T01:18:25.065 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/netfilter/xt_REDIRECT.ko 2024-08-19T01:18:25.065 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/netfilter/xt_SECMARK.ko 2024-08-19T01:18:25.065 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/netfilter/xt_TCPMSS.ko 2024-08-19T01:18:25.065 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/netfilter/xt_TCPOPTSTRIP.ko 2024-08-19T01:18:25.065 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/netfilter/xt_TEE.ko 2024-08-19T01:18:25.065 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/netfilter/xt_TPROXY.ko 2024-08-19T01:18:25.065 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/netfilter/xt_TRACE.ko 2024-08-19T01:18:25.065 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/netfilter/xt_addrtype.ko 2024-08-19T01:18:25.065 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/netfilter/xt_bpf.ko 2024-08-19T01:18:25.065 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/netfilter/xt_cgroup.ko 2024-08-19T01:18:25.065 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/netfilter/xt_cluster.ko 2024-08-19T01:18:25.066 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/netfilter/xt_comment.ko 2024-08-19T01:18:25.066 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/netfilter/xt_connbytes.ko 2024-08-19T01:18:25.066 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/netfilter/xt_connlabel.ko 2024-08-19T01:18:25.066 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/netfilter/xt_connlimit.ko 2024-08-19T01:18:25.066 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/netfilter/xt_connmark.ko 2024-08-19T01:18:25.066 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/netfilter/xt_conntrack.ko 2024-08-19T01:18:25.066 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/netfilter/xt_cpu.ko 2024-08-19T01:18:25.066 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/netfilter/xt_dccp.ko 2024-08-19T01:18:25.066 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/netfilter/xt_devgroup.ko 2024-08-19T01:18:25.066 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/netfilter/xt_dscp.ko 2024-08-19T01:18:25.066 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/netfilter/xt_ecn.ko 2024-08-19T01:18:25.066 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/netfilter/xt_esp.ko 2024-08-19T01:18:25.066 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/netfilter/xt_hashlimit.ko 2024-08-19T01:18:25.066 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/netfilter/xt_helper.ko 2024-08-19T01:18:25.067 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/netfilter/xt_hl.ko 2024-08-19T01:18:25.067 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/netfilter/xt_ipcomp.ko 2024-08-19T01:18:25.067 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/netfilter/xt_iprange.ko 2024-08-19T01:18:25.067 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/netfilter/xt_ipvs.ko 2024-08-19T01:18:25.067 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/netfilter/xt_l2tp.ko 2024-08-19T01:18:25.067 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/netfilter/xt_length.ko 2024-08-19T01:18:25.067 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/netfilter/xt_limit.ko 2024-08-19T01:18:25.067 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/netfilter/xt_mac.ko 2024-08-19T01:18:25.067 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/netfilter/xt_mark.ko 2024-08-19T01:18:25.067 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/netfilter/xt_multiport.ko 2024-08-19T01:18:25.067 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/netfilter/xt_nat.ko 2024-08-19T01:18:25.067 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/netfilter/xt_nfacct.ko 2024-08-19T01:18:25.067 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/netfilter/xt_osf.ko 2024-08-19T01:18:25.067 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/netfilter/xt_owner.ko 2024-08-19T01:18:25.067 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/netfilter/xt_physdev.ko 2024-08-19T01:18:25.068 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/netfilter/xt_pkttype.ko 2024-08-19T01:18:25.068 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/netfilter/xt_policy.ko 2024-08-19T01:18:25.068 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/netfilter/xt_quota.ko 2024-08-19T01:18:25.068 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/netfilter/xt_rateest.ko 2024-08-19T01:18:25.068 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/netfilter/xt_realm.ko 2024-08-19T01:18:25.068 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/netfilter/xt_recent.ko 2024-08-19T01:18:25.068 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/netfilter/xt_sctp.ko 2024-08-19T01:18:25.068 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/netfilter/xt_set.ko 2024-08-19T01:18:25.068 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/netfilter/xt_socket.ko 2024-08-19T01:18:25.068 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/netfilter/xt_state.ko 2024-08-19T01:18:25.068 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/netfilter/xt_statistic.ko 2024-08-19T01:18:25.068 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/netfilter/xt_string.ko 2024-08-19T01:18:25.068 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/netfilter/xt_tcpmss.ko 2024-08-19T01:18:25.068 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/netfilter/xt_time.ko 2024-08-19T01:18:25.069 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/netfilter/xt_u32.ko 2024-08-19T01:18:25.069 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/netlink 2024-08-19T01:18:25.069 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/netlink/netlink_diag.ko 2024-08-19T01:18:25.069 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/nsh 2024-08-19T01:18:25.069 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/nsh/nsh.ko 2024-08-19T01:18:25.069 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/openvswitch 2024-08-19T01:18:25.069 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/openvswitch/openvswitch.ko 2024-08-19T01:18:25.069 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/openvswitch/vport-geneve.ko 2024-08-19T01:18:25.069 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/openvswitch/vport-gre.ko 2024-08-19T01:18:25.069 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/openvswitch/vport-vxlan.ko 2024-08-19T01:18:25.069 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/packet 2024-08-19T01:18:25.069 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/packet/af_packet_diag.ko 2024-08-19T01:18:25.069 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/rds 2024-08-19T01:18:25.069 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/rds/rds.ko 2024-08-19T01:18:25.069 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/rds/rds_rdma.ko 2024-08-19T01:18:25.070 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/rds/rds_tcp.ko 2024-08-19T01:18:25.070 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/sched 2024-08-19T01:18:25.070 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/sched/act_csum.ko 2024-08-19T01:18:25.070 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/sched/act_gact.ko 2024-08-19T01:18:25.070 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/sched/act_mirred.ko 2024-08-19T01:18:25.070 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/sched/act_nat.ko 2024-08-19T01:18:25.070 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/sched/act_pedit.ko 2024-08-19T01:18:25.070 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/sched/act_police.ko 2024-08-19T01:18:25.070 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/sched/act_simple.ko 2024-08-19T01:18:25.070 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/sched/act_skbedit.ko 2024-08-19T01:18:25.070 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/sched/act_vlan.ko 2024-08-19T01:18:25.070 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/sched/cls_basic.ko 2024-08-19T01:18:25.070 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/sched/cls_bpf.ko 2024-08-19T01:18:25.070 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/sched/cls_flow.ko 2024-08-19T01:18:25.070 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/sched/cls_fw.ko 2024-08-19T01:18:25.071 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/sched/cls_route.ko 2024-08-19T01:18:25.071 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/sched/cls_u32.ko 2024-08-19T01:18:25.071 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/sched/em_cmp.ko 2024-08-19T01:18:25.071 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/sched/em_ipset.ko 2024-08-19T01:18:25.071 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/sched/em_meta.ko 2024-08-19T01:18:25.071 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/sched/em_nbyte.ko 2024-08-19T01:18:25.071 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/sched/em_text.ko 2024-08-19T01:18:25.071 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/sched/em_u32.ko 2024-08-19T01:18:25.071 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/sched/sch_choke.ko 2024-08-19T01:18:25.071 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/sched/sch_codel.ko 2024-08-19T01:18:25.071 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/sched/sch_drr.ko 2024-08-19T01:18:25.071 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/sched/sch_fq.ko 2024-08-19T01:18:25.071 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/sched/sch_gred.ko 2024-08-19T01:18:25.071 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/sched/sch_hfsc.ko 2024-08-19T01:18:25.071 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/sched/sch_hhf.ko 2024-08-19T01:18:25.072 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/sched/sch_htb.ko 2024-08-19T01:18:25.072 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/sched/sch_ingress.ko 2024-08-19T01:18:25.072 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/sched/sch_mqprio.ko 2024-08-19T01:18:25.072 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/sched/sch_mqprio_lib.ko 2024-08-19T01:18:25.072 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/sched/sch_multiq.ko 2024-08-19T01:18:25.072 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/sched/sch_netem.ko 2024-08-19T01:18:25.072 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/sched/sch_pie.ko 2024-08-19T01:18:25.072 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/sched/sch_plug.ko 2024-08-19T01:18:25.072 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/sched/sch_prio.ko 2024-08-19T01:18:25.072 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/sched/sch_qfq.ko 2024-08-19T01:18:25.072 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/sched/sch_red.ko 2024-08-19T01:18:25.072 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/sched/sch_sfb.ko 2024-08-19T01:18:25.073 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/sched/sch_sfq.ko 2024-08-19T01:18:25.073 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/sched/sch_tbf.ko 2024-08-19T01:18:25.073 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/sched/sch_teql.ko 2024-08-19T01:18:25.073 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/sctp 2024-08-19T01:18:25.073 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/sctp/sctp.ko 2024-08-19T01:18:25.073 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/sctp/sctp_diag.ko 2024-08-19T01:18:25.073 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/sunrpc 2024-08-19T01:18:25.073 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/sunrpc/auth_gss 2024-08-19T01:18:25.073 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/sunrpc/auth_gss/auth_rpcgss.ko 2024-08-19T01:18:25.073 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/sunrpc/auth_gss/rpcsec_gss_krb5.ko 2024-08-19T01:18:25.073 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/sunrpc/sunrpc.ko 2024-08-19T01:18:25.073 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/sunrpc/xprtrdma 2024-08-19T01:18:25.073 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/sunrpc/xprtrdma/rpcrdma.ko 2024-08-19T01:18:25.073 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/tipc 2024-08-19T01:18:25.074 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/tipc/tipc.ko 2024-08-19T01:18:25.074 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/tipc/tipc_diag.ko 2024-08-19T01:18:25.074 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/unix 2024-08-19T01:18:25.074 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/unix/unix_diag.ko 2024-08-19T01:18:25.074 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/vmw_vsock 2024-08-19T01:18:25.074 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/vmw_vsock/vmw_vsock_virtio_transport_common.ko 2024-08-19T01:18:25.074 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/vmw_vsock/vmw_vsock_vmci_transport.ko 2024-08-19T01:18:25.074 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/vmw_vsock/vsock.ko 2024-08-19T01:18:25.074 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/vmw_vsock/vsock_diag.ko 2024-08-19T01:18:25.074 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/vmw_vsock/vsock_loopback.ko 2024-08-19T01:18:25.074 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/xfrm 2024-08-19T01:18:25.074 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/xfrm/xfrm_ipcomp.ko 2024-08-19T01:18:25.074 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/security 2024-08-19T01:18:25.074 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/security/keys 2024-08-19T01:18:25.075 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/security/keys/encrypted-keys 2024-08-19T01:18:25.075 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/security/keys/encrypted-keys/encrypted-keys.ko 2024-08-19T01:18:25.075 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/security/keys/trusted-keys 2024-08-19T01:18:25.075 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/security/keys/trusted-keys/trusted.ko 2024-08-19T01:18:25.075 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/modules.alias 2024-08-19T01:18:25.075 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/modules.alias.bin 2024-08-19T01:18:25.075 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/modules.builtin 2024-08-19T01:18:25.075 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/modules.builtin.alias.bin 2024-08-19T01:18:25.075 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/modules.builtin.bin 2024-08-19T01:18:25.075 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/modules.builtin.modinfo 2024-08-19T01:18:25.075 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/modules.dep 2024-08-19T01:18:25.075 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/modules.dep.bin 2024-08-19T01:18:25.075 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/modules.devname 2024-08-19T01:18:25.075 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/modules.order 2024-08-19T01:18:25.076 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/modules.softdep 2024-08-19T01:18:25.076 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/modules.symbols 2024-08-19T01:18:25.076 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/modules.symbols.bin 2024-08-19T01:18:25.076 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/modules.weakdep 2024-08-19T01:18:25.076 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/vmlinuz 2024-08-19T01:18:25.080 DEBUG:teuthology.task.kernel:get_image_version: 6.11.0-rc3-g7503b862ff5b 2024-08-19T01:18:25.080 DEBUG:teuthology.orchestra.run.smithi083:> rpm --scripts -qp /tmp/kernel.x86_64.rpm 2024-08-19T01:18:25.103 INFO:teuthology.orchestra.run.smithi083.stdout:postinstall scriptlet (using /bin/sh): 2024-08-19T01:18:25.103 INFO:teuthology.orchestra.run.smithi083.stdout:if [ -x /usr/bin/kernel-install ]; then 2024-08-19T01:18:25.104 INFO:teuthology.orchestra.run.smithi083.stdout: /usr/bin/kernel-install add 6.11.0-rc3-g7503b862ff5b /lib/modules/6.11.0-rc3-g7503b862ff5b/vmlinuz 2024-08-19T01:18:25.104 INFO:teuthology.orchestra.run.smithi083.stdout:fi 2024-08-19T01:18:25.104 INFO:teuthology.orchestra.run.smithi083.stdout:for file in vmlinuz System.map config; do 2024-08-19T01:18:25.104 INFO:teuthology.orchestra.run.smithi083.stdout: if ! cmp --silent "/lib/modules/6.11.0-rc3-g7503b862ff5b/${file}" "/boot/${file}-6.11.0-rc3-g7503b862ff5b"; then 2024-08-19T01:18:25.104 INFO:teuthology.orchestra.run.smithi083.stdout: cp "/lib/modules/6.11.0-rc3-g7503b862ff5b/${file}" "/boot/${file}-6.11.0-rc3-g7503b862ff5b" 2024-08-19T01:18:25.104 INFO:teuthology.orchestra.run.smithi083.stdout: fi 2024-08-19T01:18:25.104 INFO:teuthology.orchestra.run.smithi083.stdout:done 2024-08-19T01:18:25.104 INFO:teuthology.orchestra.run.smithi083.stdout:if [ -d "/lib/modules/6.11.0-rc3-g7503b862ff5b/dtb" ] && \ 2024-08-19T01:18:25.104 INFO:teuthology.orchestra.run.smithi083.stdout: ! diff -rq "/lib/modules/6.11.0-rc3-g7503b862ff5b/dtb" "/boot/dtb-6.11.0-rc3-g7503b862ff5b" >/dev/null 2>&1; then 2024-08-19T01:18:25.104 INFO:teuthology.orchestra.run.smithi083.stdout: rm -rf "/boot/dtb-6.11.0-rc3-g7503b862ff5b" 2024-08-19T01:18:25.104 INFO:teuthology.orchestra.run.smithi083.stdout: cp -r "/lib/modules/6.11.0-rc3-g7503b862ff5b/dtb" "/boot/dtb-6.11.0-rc3-g7503b862ff5b" 2024-08-19T01:18:25.104 INFO:teuthology.orchestra.run.smithi083.stdout:fi 2024-08-19T01:18:25.104 INFO:teuthology.orchestra.run.smithi083.stdout:if [ ! -e "/lib/modules/6.11.0-rc3-g7503b862ff5b/modules.dep" ]; then 2024-08-19T01:18:25.104 INFO:teuthology.orchestra.run.smithi083.stdout: /usr/sbin/depmod 6.11.0-rc3-g7503b862ff5b 2024-08-19T01:18:25.105 INFO:teuthology.orchestra.run.smithi083.stdout:fi 2024-08-19T01:18:25.105 INFO:teuthology.orchestra.run.smithi083.stdout:preuninstall scriptlet (using /bin/sh): 2024-08-19T01:18:25.105 INFO:teuthology.orchestra.run.smithi083.stdout:if [ -x /usr/bin/kernel-install ]; then 2024-08-19T01:18:25.105 INFO:teuthology.orchestra.run.smithi083.stdout:kernel-install remove 6.11.0-rc3-g7503b862ff5b 2024-08-19T01:18:25.105 INFO:teuthology.orchestra.run.smithi083.stdout:fi 2024-08-19T01:18:25.106 DEBUG:teuthology.orchestra.run.smithi083:> sudo rpm -qi grub2-tools 2024-08-19T01:18:25.179 INFO:teuthology.orchestra.run.smithi083.stdout:Name : grub2-tools 2024-08-19T01:18:25.179 INFO:teuthology.orchestra.run.smithi083.stdout:Epoch : 1 2024-08-19T01:18:25.179 INFO:teuthology.orchestra.run.smithi083.stdout:Version : 2.06 2024-08-19T01:18:25.179 INFO:teuthology.orchestra.run.smithi083.stdout:Release : 82.el9 2024-08-19T01:18:25.179 INFO:teuthology.orchestra.run.smithi083.stdout:Architecture: x86_64 2024-08-19T01:18:25.179 INFO:teuthology.orchestra.run.smithi083.stdout:Install Date: Sun 18 Aug 2024 08:28:07 PM UTC 2024-08-19T01:18:25.179 INFO:teuthology.orchestra.run.smithi083.stdout:Group : Unspecified 2024-08-19T01:18:25.179 INFO:teuthology.orchestra.run.smithi083.stdout:Size : 8274670 2024-08-19T01:18:25.179 INFO:teuthology.orchestra.run.smithi083.stdout:License : GPLv3+ 2024-08-19T01:18:25.179 INFO:teuthology.orchestra.run.smithi083.stdout:Signature : RSA/SHA256, Mon 01 Jul 2024 07:46:42 AM UTC, Key ID 05b555b38483c65d 2024-08-19T01:18:25.179 INFO:teuthology.orchestra.run.smithi083.stdout:Source RPM : grub2-2.06-82.el9.src.rpm 2024-08-19T01:18:25.180 INFO:teuthology.orchestra.run.smithi083.stdout:Build Date : Fri 28 Jun 2024 09:24:44 AM UTC 2024-08-19T01:18:25.180 INFO:teuthology.orchestra.run.smithi083.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-08-19T01:18:25.180 INFO:teuthology.orchestra.run.smithi083.stdout:Packager : builder@centos.org 2024-08-19T01:18:25.180 INFO:teuthology.orchestra.run.smithi083.stdout:Vendor : CentOS 2024-08-19T01:18:25.180 INFO:teuthology.orchestra.run.smithi083.stdout:URL : http://www.gnu.org/software/grub/ 2024-08-19T01:18:25.180 INFO:teuthology.orchestra.run.smithi083.stdout:Summary : Support tools for GRUB. 2024-08-19T01:18:25.180 INFO:teuthology.orchestra.run.smithi083.stdout:Description : 2024-08-19T01:18:25.180 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-08-19T01:18:25.180 INFO:teuthology.orchestra.run.smithi083.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-08-19T01:18:25.180 INFO:teuthology.orchestra.run.smithi083.stdout:customizable bootloader with modular architecture. It supports a rich 2024-08-19T01:18:25.180 INFO:teuthology.orchestra.run.smithi083.stdout:variety of kernel formats, file systems, computer architectures and 2024-08-19T01:18:25.180 INFO:teuthology.orchestra.run.smithi083.stdout:hardware devices. 2024-08-19T01:18:25.180 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-08-19T01:18:25.180 INFO:teuthology.orchestra.run.smithi083.stdout:This subpackage provides tools for support of all platforms. 2024-08-19T01:18:25.181 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-08-19T01:18:25.181 INFO:teuthology.task.kernel:Updating grub on smithi083 to boot 6.11.0-rc3-g7503b862ff5b 2024-08-19T01:18:25.182 DEBUG:teuthology.orchestra.run.smithi083:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-08-19T01:18:25.772 INFO:teuthology.orchestra.run.smithi083.stderr:Generating grub configuration file ... 2024-08-19T01:18:27.036 INFO:teuthology.orchestra.run.smithi083.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-08-19T01:18:27.047 INFO:teuthology.orchestra.run.smithi083.stderr:done 2024-08-19T01:18:27.049 DEBUG:teuthology.orchestra.run.smithi083:> mktemp 2024-08-19T01:18:27.064 INFO:teuthology.orchestra.run.smithi083.stdout:/tmp/tmp.rEySyNAy5I 2024-08-19T01:18:27.065 DEBUG:teuthology.orchestra.run.smithi083:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.rEySyNAy5I 2024-08-19T01:18:27.131 DEBUG:teuthology.orchestra.run.smithi083:> sudo chmod 0666 /tmp/tmp.rEySyNAy5I 2024-08-19T01:18:27.277 DEBUG:teuthology.orchestra.remote:smithi083:/tmp/tmp.rEySyNAy5I is 6KB 2024-08-19T01:18:27.325 DEBUG:teuthology.orchestra.run.smithi083:> rm -fr /tmp/tmp.rEySyNAy5I 2024-08-19T01:18:27.340 DEBUG:teuthology.orchestra.run.smithi083:> sudo /bin/ls /boot/loader/entries || true 2024-08-19T01:18:27.403 INFO:teuthology.orchestra.run.smithi083.stdout:fde2fbf59b46438eaea4e94a04206310-0-rescue.conf 2024-08-19T01:18:27.403 INFO:teuthology.orchestra.run.smithi083.stdout:fde2fbf59b46438eaea4e94a04206310-5.14.0-239.el9.x86_64.conf 2024-08-19T01:18:27.403 INFO:teuthology.orchestra.run.smithi083.stdout:fde2fbf59b46438eaea4e94a04206310-5.14.0-480.el9.x86_64.conf 2024-08-19T01:18:27.403 INFO:teuthology.orchestra.run.smithi083.stdout:fde2fbf59b46438eaea4e94a04206310-6.11.0-rc3-g7503b862ff5b.conf 2024-08-19T01:18:27.405 DEBUG:teuthology.orchestra.run.smithi083:> sudo grub2-set-default fde2fbf59b46438eaea4e94a04206310-6.11.0-rc3-g7503b862ff5b 2024-08-19T01:18:27.546 DEBUG:teuthology.orchestra.run.smithi083:> sudo shutdown -r now 2024-08-19T01:18:57.549 INFO:teuthology.task.kernel:Checking client client.0 for new kernel version... 2024-08-19T01:18:57.549 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi083.front.sepia.ceph.com' 2024-08-19T01:18:57.550 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi083.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-08-19T01:19:15.934 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.83 2024-08-19T01:19:24.935 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi083.front.sepia.ceph.com' 2024-08-19T01:19:24.936 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi083.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-08-19T01:19:27.998 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.83 2024-08-19T01:19:39.999 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi083.front.sepia.ceph.com' 2024-08-19T01:19:40.000 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi083.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-08-19T01:20:11.924 DEBUG:teuthology.orchestra.run.smithi083:> true 2024-08-19T01:20:12.506 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi083.front.sepia.ceph.com' 2024-08-19T01:20:12.507 INFO:teuthology.task.kernel:Checking kernel version of client.0, want "6.11.0-rc3-g7503b862ff5b"... 2024-08-19T01:20:12.507 DEBUG:teuthology.orchestra.run.smithi083:> uname -r 2024-08-19T01:20:12.541 INFO:teuthology.orchestra.run.smithi083.stdout:6.11.0-rc3-g7503b862ff5b 2024-08-19T01:20:12.542 DEBUG:teuthology.task.kernel:current kernel version is 6.11.0-rc3-g7503b862ff5b vs 6.11.0-rc3-g7503b862ff5b 2024-08-19T01:20:12.542 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-08-19T01:20:12.542 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-08-19T01:20:13.543 DEBUG:teuthology.parallel:result is None 2024-08-19T01:20:13.566 INFO:teuthology.run_tasks:Running task internal.base... 2024-08-19T01:20:13.578 INFO:teuthology.task.internal:Creating test directory... 2024-08-19T01:20:13.579 DEBUG:teuthology.orchestra.run.smithi083:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-08-19T01:20:13.583 DEBUG:teuthology.orchestra.run.smithi117:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-08-19T01:20:13.585 DEBUG:teuthology.orchestra.run.smithi167:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-08-19T01:20:13.614 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-08-19T01:20:13.620 INFO:teuthology.run_tasks:Running task internal.archive... 2024-08-19T01:20:13.633 INFO:teuthology.task.internal:Creating archive directory... 2024-08-19T01:20:13.633 DEBUG:teuthology.orchestra.run.smithi083:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-08-19T01:20:13.661 DEBUG:teuthology.orchestra.run.smithi117:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-08-19T01:20:13.663 DEBUG:teuthology.orchestra.run.smithi167:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-08-19T01:20:13.838 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-08-19T01:20:13.849 INFO:teuthology.task.internal:Enabling coredump saving... 2024-08-19T01:20:13.849 DEBUG:teuthology.orchestra.run.smithi117:> test -f /run/.containerenv -o -f /.dockerenv 2024-08-19T01:20:13.866 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-08-19T01:20:13.866 DEBUG:teuthology.orchestra.run.smithi167:> test -f /run/.containerenv -o -f /.dockerenv 2024-08-19T01:20:13.883 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-08-19T01:20:13.884 DEBUG:teuthology.orchestra.run.smithi083:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-08-19T01:20:13.887 DEBUG:teuthology.orchestra.run.smithi117:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-08-19T01:20:13.909 DEBUG:teuthology.orchestra.run.smithi167:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-08-19T01:20:13.952 INFO:teuthology.orchestra.run.smithi117.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-08-19T01:20:13.961 INFO:teuthology.orchestra.run.smithi167.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-08-19T01:20:13.968 INFO:teuthology.orchestra.run.smithi117.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-08-19T01:20:13.983 INFO:teuthology.orchestra.run.smithi167.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-08-19T01:20:14.018 INFO:teuthology.orchestra.run.smithi083.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-08-19T01:20:14.173 INFO:teuthology.orchestra.run.smithi083.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-08-19T01:20:14.177 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-08-19T01:20:14.185 INFO:teuthology.task.internal:Configuring sudo... 2024-08-19T01:20:14.185 DEBUG:teuthology.orchestra.run.smithi083:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-08-19T01:20:14.189 DEBUG:teuthology.orchestra.run.smithi117:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-08-19T01:20:14.191 DEBUG:teuthology.orchestra.run.smithi167:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-08-19T01:20:14.243 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-08-19T01:20:14.256 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-08-19T01:20:14.256 DEBUG:teuthology.orchestra.run.smithi083:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-08-19T01:20:14.260 DEBUG:teuthology.orchestra.run.smithi117:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-08-19T01:20:14.262 DEBUG:teuthology.orchestra.run.smithi167:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-08-19T01:20:14.291 DEBUG:teuthology.orchestra.run.smithi083:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-08-19T01:20:14.399 DEBUG:teuthology.orchestra.run.smithi083:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-08-19T01:20:14.487 DEBUG:teuthology.orchestra.run.smithi083:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-08-19T01:20:14.550 DEBUG:teuthology.orchestra.run.smithi083:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-08-19T01:20:14.621 DEBUG:teuthology.orchestra.run.smithi083:> set -ex 2024-08-19T01:20:14.621 DEBUG:teuthology.orchestra.run.smithi083:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-08-19T01:20:14.678 DEBUG:teuthology.orchestra.run.smithi117:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-08-19T01:20:14.724 DEBUG:teuthology.orchestra.run.smithi117:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-08-19T01:20:14.788 DEBUG:teuthology.orchestra.run.smithi117:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-08-19T01:20:14.833 DEBUG:teuthology.orchestra.run.smithi117:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-08-19T01:20:14.873 DEBUG:teuthology.orchestra.run.smithi117:> set -ex 2024-08-19T01:20:14.874 DEBUG:teuthology.orchestra.run.smithi117:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-08-19T01:20:14.940 DEBUG:teuthology.orchestra.run.smithi167:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-08-19T01:20:14.986 DEBUG:teuthology.orchestra.run.smithi167:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-08-19T01:20:15.050 DEBUG:teuthology.orchestra.run.smithi167:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-08-19T01:20:15.095 DEBUG:teuthology.orchestra.run.smithi167:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-08-19T01:20:15.138 DEBUG:teuthology.orchestra.run.smithi167:> set -ex 2024-08-19T01:20:15.138 DEBUG:teuthology.orchestra.run.smithi167:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-08-19T01:20:15.205 DEBUG:teuthology.orchestra.run.smithi083:> sudo service rsyslog restart 2024-08-19T01:20:15.209 DEBUG:teuthology.orchestra.run.smithi117:> sudo service rsyslog restart 2024-08-19T01:20:15.212 DEBUG:teuthology.orchestra.run.smithi167:> sudo service rsyslog restart 2024-08-19T01:20:15.245 INFO:teuthology.orchestra.run.smithi117.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-08-19T01:20:15.282 INFO:teuthology.orchestra.run.smithi167.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-08-19T01:20:15.320 INFO:teuthology.orchestra.run.smithi083.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-08-19T01:20:15.740 INFO:teuthology.run_tasks:Running task internal.timer... 2024-08-19T01:20:15.746 INFO:teuthology.task.internal:Starting timer... 2024-08-19T01:20:15.747 INFO:teuthology.run_tasks:Running task pcp... 2024-08-19T01:20:15.761 INFO:teuthology.run_tasks:Running task selinux... 2024-08-19T01:20:15.770 DEBUG:teuthology.task:Applying overrides for task selinux: {'allowlist': ['scontext=system_u:system_r:logrotate_t:s0', 'scontext=system_u:system_r:getty_t:s0']} 2024-08-19T01:20:15.770 DEBUG:teuthology.orchestra.run.smithi083:> sudo service auditd rotate 2024-08-19T01:20:15.854 INFO:teuthology.orchestra.run.smithi083.stdout:Rotating logs: 2024-08-19T01:20:15.857 DEBUG:teuthology.orchestra.run.smithi117:> sudo service auditd rotate 2024-08-19T01:20:15.910 INFO:teuthology.orchestra.run.smithi117.stdout:Rotating logs: 2024-08-19T01:20:15.912 DEBUG:teuthology.orchestra.run.smithi167:> sudo service auditd rotate 2024-08-19T01:20:15.990 INFO:teuthology.orchestra.run.smithi167.stdout:Rotating logs: 2024-08-19T01:20:15.992 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-08-19T01:20:15.992 DEBUG:teuthology.orchestra.run.smithi083:> /usr/sbin/getenforce 2024-08-19T01:20:16.034 INFO:teuthology.orchestra.run.smithi083.stdout:Permissive 2024-08-19T01:20:16.035 DEBUG:teuthology.orchestra.run.smithi117:> /usr/sbin/getenforce 2024-08-19T01:20:16.061 INFO:teuthology.orchestra.run.smithi117.stdout:Permissive 2024-08-19T01:20:16.061 DEBUG:teuthology.orchestra.run.smithi167:> /usr/sbin/getenforce 2024-08-19T01:20:16.085 INFO:teuthology.orchestra.run.smithi167.stdout:Permissive 2024-08-19T01:20:16.085 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi083.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi117.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi167.front.sepia.ceph.com': 'permissive'} 2024-08-19T01:20:16.085 DEBUG:teuthology.orchestra.run.smithi083:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-08-19T01:20:16.144 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-08-19T01:20:16.145 DEBUG:teuthology.orchestra.run.smithi117:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-08-19T01:20:16.174 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-08-19T01:20:16.175 DEBUG:teuthology.orchestra.run.smithi167:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-08-19T01:20:16.200 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-08-19T01:20:16.200 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-08-19T01:20:16.201 DEBUG:teuthology.orchestra.run.smithi083:> sudo /usr/sbin/setenforce permissive 2024-08-19T01:20:16.264 DEBUG:teuthology.orchestra.run.smithi117:> sudo /usr/sbin/setenforce permissive 2024-08-19T01:20:16.292 DEBUG:teuthology.orchestra.run.smithi167:> sudo /usr/sbin/setenforce permissive 2024-08-19T01:20:16.321 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-08-19T01:20:16.334 DEBUG:teuthology.repo_utils:Setting repo remote to https://git.ceph.com/ceph-cm-ansible.git 2024-08-19T01:20:16.336 INFO:teuthology.repo_utils:Fetching git.ceph.com_ceph-cm-ansible_main from origin 2024-08-19T01:20:16.446 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-08-19T01:20:16.457 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-08-19T01:20:16.458 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi083.front.sepia.ceph.com,smithi117.front.sepia.ceph.com,smithi167.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-08-19T01:22:21.875 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi083.front.sepia.ceph.com'), Remote(name='ubuntu@smithi117.front.sepia.ceph.com'), Remote(name='ubuntu@smithi167.front.sepia.ceph.com')] 2024-08-19T01:22:21.877 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi083.front.sepia.ceph.com' 2024-08-19T01:22:21.878 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi083.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-08-19T01:22:21.966 DEBUG:teuthology.orchestra.run.smithi083:> true 2024-08-19T01:22:22.085 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi083.front.sepia.ceph.com' 2024-08-19T01:22:22.085 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi117.front.sepia.ceph.com' 2024-08-19T01:22:22.085 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi117.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-08-19T01:22:22.160 DEBUG:teuthology.orchestra.run.smithi117:> true 2024-08-19T01:22:22.198 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi117.front.sepia.ceph.com' 2024-08-19T01:22:22.199 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi167.front.sepia.ceph.com' 2024-08-19T01:22:22.199 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi167.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-08-19T01:22:22.273 DEBUG:teuthology.orchestra.run.smithi167:> true 2024-08-19T01:22:22.367 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi167.front.sepia.ceph.com' 2024-08-19T01:22:22.367 INFO:teuthology.run_tasks:Running task clock... 2024-08-19T01:22:22.377 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-08-19T01:22:22.378 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-08-19T01:22:22.378 DEBUG:teuthology.orchestra.run.smithi083:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-08-19T01:22:22.382 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-08-19T01:22:22.382 DEBUG:teuthology.orchestra.run.smithi117:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-08-19T01:22:22.385 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-08-19T01:22:22.385 DEBUG:teuthology.orchestra.run.smithi167:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-08-19T01:22:22.416 INFO:teuthology.orchestra.run.smithi117.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-08-19T01:22:22.432 INFO:teuthology.orchestra.run.smithi117.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-08-19T01:22:22.438 INFO:teuthology.orchestra.run.smithi167.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-08-19T01:22:22.442 INFO:teuthology.orchestra.run.smithi083.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-08-19T01:22:22.453 INFO:teuthology.orchestra.run.smithi167.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-08-19T01:22:22.459 INFO:teuthology.orchestra.run.smithi117.stderr:sudo: ntpd: command not found 2024-08-19T01:22:22.469 INFO:teuthology.orchestra.run.smithi083.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-08-19T01:22:22.471 INFO:teuthology.orchestra.run.smithi117.stdout:506 Cannot talk to daemon 2024-08-19T01:22:22.478 INFO:teuthology.orchestra.run.smithi167.stderr:sudo: ntpd: command not found 2024-08-19T01:22:22.486 INFO:teuthology.orchestra.run.smithi117.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-08-19T01:22:22.490 INFO:teuthology.orchestra.run.smithi167.stdout:506 Cannot talk to daemon 2024-08-19T01:22:22.501 INFO:teuthology.orchestra.run.smithi117.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-08-19T01:22:22.504 INFO:teuthology.orchestra.run.smithi167.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-08-19T01:22:22.518 INFO:teuthology.orchestra.run.smithi167.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-08-19T01:22:22.520 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: ntpd: command not found 2024-08-19T01:22:22.544 INFO:teuthology.orchestra.run.smithi083.stdout:506 Cannot talk to daemon 2024-08-19T01:22:22.556 INFO:teuthology.orchestra.run.smithi117.stderr:bash: line 1: ntpq: command not found 2024-08-19T01:22:22.560 INFO:teuthology.orchestra.run.smithi117.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-08-19T01:22:22.560 INFO:teuthology.orchestra.run.smithi117.stdout:=============================================================================== 2024-08-19T01:22:22.560 INFO:teuthology.orchestra.run.smithi117.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-08-19T01:22:22.560 INFO:teuthology.orchestra.run.smithi117.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-08-19T01:22:22.560 INFO:teuthology.orchestra.run.smithi117.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-08-19T01:22:22.560 INFO:teuthology.orchestra.run.smithi117.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-08-19T01:22:22.573 INFO:teuthology.orchestra.run.smithi167.stderr:bash: line 1: ntpq: command not found 2024-08-19T01:22:22.573 INFO:teuthology.orchestra.run.smithi083.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-08-19T01:22:22.576 INFO:teuthology.orchestra.run.smithi167.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-08-19T01:22:22.576 INFO:teuthology.orchestra.run.smithi167.stdout:=============================================================================== 2024-08-19T01:22:22.576 INFO:teuthology.orchestra.run.smithi167.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-08-19T01:22:22.576 INFO:teuthology.orchestra.run.smithi167.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-08-19T01:22:22.576 INFO:teuthology.orchestra.run.smithi167.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-08-19T01:22:22.576 INFO:teuthology.orchestra.run.smithi167.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-08-19T01:22:22.602 INFO:teuthology.orchestra.run.smithi083.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-08-19T01:22:22.703 INFO:teuthology.orchestra.run.smithi083.stderr:bash: line 1: ntpq: command not found 2024-08-19T01:22:22.709 INFO:teuthology.orchestra.run.smithi083.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-08-19T01:22:22.709 INFO:teuthology.orchestra.run.smithi083.stdout:=============================================================================== 2024-08-19T01:22:22.710 INFO:teuthology.orchestra.run.smithi083.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-08-19T01:22:22.710 INFO:teuthology.orchestra.run.smithi083.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-08-19T01:22:22.710 INFO:teuthology.orchestra.run.smithi083.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-08-19T01:22:22.710 INFO:teuthology.orchestra.run.smithi083.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-08-19T01:22:22.711 INFO:teuthology.run_tasks:Running task install... 2024-08-19T01:22:22.719 DEBUG:teuthology.task.install:project ceph 2024-08-19T01:22:22.719 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': 'de09dd018de4baddfa6ca4d43bc8fb784cf75833'}} 2024-08-19T01:22:22.719 DEBUG:teuthology.task.install:config {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': 'de09dd018de4baddfa6ca4d43bc8fb784cf75833'} 2024-08-19T01:22:22.719 INFO:teuthology.task.install:Using flavor: default 2024-08-19T01:22:22.725 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-08-19T01:22:22.725 INFO:teuthology.task.install:extra packages: {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']} 2024-08-19T01:22:22.725 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'de09dd018de4baddfa6ca4d43bc8fb784cf75833', 'tag': None, 'wait_for_package': False} 2024-08-19T01:22:22.726 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=de09dd018de4baddfa6ca4d43bc8fb784cf75833 2024-08-19T01:22:22.727 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'de09dd018de4baddfa6ca4d43bc8fb784cf75833', 'tag': None, 'wait_for_package': False} 2024-08-19T01:22:22.728 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=de09dd018de4baddfa6ca4d43bc8fb784cf75833 2024-08-19T01:22:22.728 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'de09dd018de4baddfa6ca4d43bc8fb784cf75833', 'tag': None, 'wait_for_package': False} 2024-08-19T01:22:22.729 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=de09dd018de4baddfa6ca4d43bc8fb784cf75833 2024-08-19T01:22:22.870 INFO:teuthology.task.install.rpm:Pulling from https://1.chacra.ceph.com/r/ceph/squid/de09dd018de4baddfa6ca4d43bc8fb784cf75833/centos/9/flavors/default/ 2024-08-19T01:22:22.870 INFO:teuthology.task.install.rpm:Package version is 19.1.0-1382.gde09dd01 2024-08-19T01:22:22.882 INFO:teuthology.task.install.rpm:Pulling from https://1.chacra.ceph.com/r/ceph/squid/de09dd018de4baddfa6ca4d43bc8fb784cf75833/centos/9/flavors/default/ 2024-08-19T01:22:22.883 INFO:teuthology.task.install.rpm:Package version is 19.1.0-1382.gde09dd01 2024-08-19T01:22:22.888 INFO:teuthology.task.install.rpm:Pulling from https://1.chacra.ceph.com/r/ceph/squid/de09dd018de4baddfa6ca4d43bc8fb784cf75833/centos/9/flavors/default/ 2024-08-19T01:22:22.889 INFO:teuthology.task.install.rpm:Package version is 19.1.0-1382.gde09dd01 2024-08-19T01:22:22.979 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://1.chacra.ceph.com/r/ceph/squid/de09dd018de4baddfa6ca4d43bc8fb784cf75833/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://1.chacra.ceph.com/r/ceph/squid/de09dd018de4baddfa6ca4d43bc8fb784cf75833/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://1.chacra.ceph.com/r/ceph/squid/de09dd018de4baddfa6ca4d43bc8fb784cf75833/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-08-19T01:22:22.980 DEBUG:teuthology.orchestra.run.smithi083:> set -ex 2024-08-19T01:22:22.980 DEBUG:teuthology.orchestra.run.smithi083:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-08-19T01:22:22.994 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://1.chacra.ceph.com/r/ceph/squid/de09dd018de4baddfa6ca4d43bc8fb784cf75833/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://1.chacra.ceph.com/r/ceph/squid/de09dd018de4baddfa6ca4d43bc8fb784cf75833/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://1.chacra.ceph.com/r/ceph/squid/de09dd018de4baddfa6ca4d43bc8fb784cf75833/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-08-19T01:22:22.994 DEBUG:teuthology.orchestra.run.smithi167:> set -ex 2024-08-19T01:22:22.994 DEBUG:teuthology.orchestra.run.smithi167:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-08-19T01:22:22.997 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://1.chacra.ceph.com/r/ceph/squid/de09dd018de4baddfa6ca4d43bc8fb784cf75833/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://1.chacra.ceph.com/r/ceph/squid/de09dd018de4baddfa6ca4d43bc8fb784cf75833/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://1.chacra.ceph.com/r/ceph/squid/de09dd018de4baddfa6ca4d43bc8fb784cf75833/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-08-19T01:22:22.997 DEBUG:teuthology.orchestra.run.smithi117:> set -ex 2024-08-19T01:22:22.997 DEBUG:teuthology.orchestra.run.smithi117:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-08-19T01:22:23.023 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-08-19T01:22:23.023 DEBUG:teuthology.orchestra.run.smithi167:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/de09dd018de4baddfa6ca4d43bc8fb784cf75833/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-08-19T01:22:23.025 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-08-19T01:22:23.025 DEBUG:teuthology.orchestra.run.smithi117:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/de09dd018de4baddfa6ca4d43bc8fb784cf75833/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-08-19T01:22:23.036 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-08-19T01:22:23.036 DEBUG:teuthology.orchestra.run.smithi083:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/de09dd018de4baddfa6ca4d43bc8fb784cf75833/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-08-19T01:22:23.091 DEBUG:teuthology.orchestra.run.smithi167:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-08-19T01:22:23.094 DEBUG:teuthology.orchestra.run.smithi117:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-08-19T01:22:23.132 DEBUG:teuthology.orchestra.run.smithi083:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-08-19T01:22:23.166 DEBUG:teuthology.orchestra.run.smithi167:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-08-19T01:22:23.170 DEBUG:teuthology.orchestra.run.smithi117:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-08-19T01:22:23.211 DEBUG:teuthology.orchestra.run.smithi083:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-08-19T01:22:23.231 INFO:teuthology.orchestra.run.smithi167.stdout:check_obsoletes = 1 2024-08-19T01:22:23.232 DEBUG:teuthology.orchestra.run.smithi167:> sudo yum clean all 2024-08-19T01:22:23.236 INFO:teuthology.orchestra.run.smithi117.stdout:check_obsoletes = 1 2024-08-19T01:22:23.238 DEBUG:teuthology.orchestra.run.smithi117:> sudo yum clean all 2024-08-19T01:22:23.266 INFO:teuthology.orchestra.run.smithi083.stdout:check_obsoletes = 1 2024-08-19T01:22:23.270 DEBUG:teuthology.orchestra.run.smithi083:> sudo yum clean all 2024-08-19T01:22:23.457 INFO:teuthology.orchestra.run.smithi167.stdout:45 files removed 2024-08-19T01:22:23.463 INFO:teuthology.orchestra.run.smithi117.stdout:45 files removed 2024-08-19T01:22:23.484 DEBUG:teuthology.orchestra.run.smithi167:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-08-19T01:22:23.490 DEBUG:teuthology.orchestra.run.smithi117:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-08-19T01:22:23.537 INFO:teuthology.orchestra.run.smithi083.stdout:45 files removed 2024-08-19T01:22:23.570 DEBUG:teuthology.orchestra.run.smithi083:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-08-19T01:22:24.011 INFO:teuthology.orchestra.run.smithi167.stdout:ceph packages for x86_64 328 kB/s | 84 kB 00:00 2024-08-19T01:22:24.041 INFO:teuthology.orchestra.run.smithi117.stdout:ceph packages for x86_64 302 kB/s | 84 kB 00:00 2024-08-19T01:22:24.277 INFO:teuthology.orchestra.run.smithi083.stdout:ceph packages for x86_64 263 kB/s | 84 kB 00:00 2024-08-19T01:22:24.303 INFO:teuthology.orchestra.run.smithi167.stdout:ceph noarch packages 47 kB/s | 12 kB 00:00 2024-08-19T01:22:24.326 INFO:teuthology.orchestra.run.smithi117.stdout:ceph noarch packages 48 kB/s | 12 kB 00:00 2024-08-19T01:22:24.571 INFO:teuthology.orchestra.run.smithi167.stdout:ceph source packages 8.1 kB/s | 1.9 kB 00:00 2024-08-19T01:22:24.577 INFO:teuthology.orchestra.run.smithi083.stdout:ceph noarch packages 46 kB/s | 12 kB 00:00 2024-08-19T01:22:24.593 INFO:teuthology.orchestra.run.smithi117.stdout:ceph source packages 8.2 kB/s | 1.9 kB 00:00 2024-08-19T01:22:24.861 INFO:teuthology.orchestra.run.smithi083.stdout:ceph source packages 7.7 kB/s | 1.9 kB 00:00 2024-08-19T01:22:25.296 INFO:teuthology.orchestra.run.smithi167.stdout:CentOS Stream 9 - BaseOS 12 MB/s | 8.2 MB 00:00 2024-08-19T01:22:25.418 INFO:teuthology.orchestra.run.smithi117.stdout:CentOS Stream 9 - BaseOS 10 MB/s | 8.2 MB 00:00 2024-08-19T01:22:27.414 INFO:teuthology.orchestra.run.smithi167.stdout:CentOS Stream 9 - AppStream 20 MB/s | 20 MB 00:01 2024-08-19T01:22:28.002 INFO:teuthology.orchestra.run.smithi117.stdout:CentOS Stream 9 - AppStream 13 MB/s | 20 MB 00:01 2024-08-19T01:22:28.854 INFO:teuthology.orchestra.run.smithi083.stdout:CentOS Stream 9 - BaseOS 2.1 MB/s | 8.2 MB 00:03 2024-08-19T01:22:31.790 INFO:teuthology.orchestra.run.smithi167.stdout:CentOS Stream 9 - CRB 11 MB/s | 6.6 MB 00:00 2024-08-19T01:22:32.228 INFO:teuthology.orchestra.run.smithi117.stdout:CentOS Stream 9 - CRB 14 MB/s | 6.6 MB 00:00 2024-08-19T01:22:33.035 INFO:teuthology.orchestra.run.smithi083.stdout:CentOS Stream 9 - AppStream 6.7 MB/s | 20 MB 00:03 2024-08-19T01:22:33.182 INFO:teuthology.orchestra.run.smithi167.stdout:CentOS Stream 9 - Extras packages 103 kB/s | 18 kB 00:00 2024-08-19T01:22:33.595 INFO:teuthology.orchestra.run.smithi117.stdout:CentOS Stream 9 - Extras packages 120 kB/s | 18 kB 00:00 2024-08-19T01:22:34.345 INFO:teuthology.orchestra.run.smithi167.stdout:Extra Packages for Enterprise Linux 21 MB/s | 22 MB 00:01 2024-08-19T01:22:34.449 INFO:teuthology.orchestra.run.smithi117.stdout:Extra Packages for Enterprise Linux 30 MB/s | 22 MB 00:00 2024-08-19T01:22:37.847 INFO:teuthology.orchestra.run.smithi083.stdout:CentOS Stream 9 - CRB 12 MB/s | 6.6 MB 00:00 2024-08-19T01:22:39.681 INFO:teuthology.orchestra.run.smithi083.stdout:CentOS Stream 9 - Extras packages 36 kB/s | 18 kB 00:00 2024-08-19T01:22:39.776 INFO:teuthology.orchestra.run.smithi167.stdout:lab-extras 31 kB/s | 1.7 kB 00:00 2024-08-19T01:22:39.854 INFO:teuthology.orchestra.run.smithi117.stdout:lab-extras 33 kB/s | 1.7 kB 00:00 2024-08-19T01:22:40.727 INFO:teuthology.orchestra.run.smithi083.stdout:Extra Packages for Enterprise Linux 24 MB/s | 22 MB 00:00 2024-08-19T01:22:42.039 INFO:teuthology.orchestra.run.smithi117.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-08-19T01:22:42.040 INFO:teuthology.orchestra.run.smithi117.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-08-19T01:22:42.051 INFO:teuthology.orchestra.run.smithi117.stdout:Package NetworkManager-1:1.48.2-2.el9.x86_64 is already installed. 2024-08-19T01:22:42.051 INFO:teuthology.orchestra.run.smithi117.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-08-19T01:22:42.052 INFO:teuthology.orchestra.run.smithi117.stdout:Package util-linux-2.37.4-18.el9.x86_64 is already installed. 2024-08-19T01:22:42.053 INFO:teuthology.orchestra.run.smithi117.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-08-19T01:22:42.055 INFO:teuthology.orchestra.run.smithi117.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-08-19T01:22:42.057 INFO:teuthology.orchestra.run.smithi117.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-08-19T01:22:42.058 INFO:teuthology.orchestra.run.smithi117.stdout:Package xfsprogs-6.4.0-3.el9.x86_64 is already installed. 2024-08-19T01:22:42.113 INFO:teuthology.orchestra.run.smithi167.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-08-19T01:22:42.114 INFO:teuthology.orchestra.run.smithi167.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-08-19T01:22:42.117 INFO:teuthology.orchestra.run.smithi117.stdout:Dependencies resolved. 2024-08-19T01:22:42.127 INFO:teuthology.orchestra.run.smithi167.stdout:Package NetworkManager-1:1.48.2-2.el9.x86_64 is already installed. 2024-08-19T01:22:42.128 INFO:teuthology.orchestra.run.smithi167.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-08-19T01:22:42.129 INFO:teuthology.orchestra.run.smithi167.stdout:Package util-linux-2.37.4-18.el9.x86_64 is already installed. 2024-08-19T01:22:42.130 INFO:teuthology.orchestra.run.smithi167.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-08-19T01:22:42.132 INFO:teuthology.orchestra.run.smithi167.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-08-19T01:22:42.135 INFO:teuthology.orchestra.run.smithi167.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-08-19T01:22:42.136 INFO:teuthology.orchestra.run.smithi167.stdout:Package xfsprogs-6.4.0-3.el9.x86_64 is already installed. 2024-08-19T01:22:42.141 INFO:teuthology.orchestra.run.smithi117.stdout:======================================================================================= 2024-08-19T01:22:42.142 INFO:teuthology.orchestra.run.smithi117.stdout: Package Arch Version Repository Size 2024-08-19T01:22:42.142 INFO:teuthology.orchestra.run.smithi117.stdout:======================================================================================= 2024-08-19T01:22:42.142 INFO:teuthology.orchestra.run.smithi117.stdout:Installing: 2024-08-19T01:22:42.142 INFO:teuthology.orchestra.run.smithi117.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-08-19T01:22:42.142 INFO:teuthology.orchestra.run.smithi117.stdout: ceph x86_64 2:19.1.0-1382.gde09dd01.el9 ceph 6.6 k 2024-08-19T01:22:42.142 INFO:teuthology.orchestra.run.smithi117.stdout: ceph-base x86_64 2:19.1.0-1382.gde09dd01.el9 ceph 5.4 M 2024-08-19T01:22:42.142 INFO:teuthology.orchestra.run.smithi117.stdout: ceph-fuse x86_64 2:19.1.0-1382.gde09dd01.el9 ceph 896 k 2024-08-19T01:22:42.142 INFO:teuthology.orchestra.run.smithi117.stdout: ceph-immutable-object-cache x86_64 2:19.1.0-1382.gde09dd01.el9 ceph 145 k 2024-08-19T01:22:42.142 INFO:teuthology.orchestra.run.smithi117.stdout: ceph-mgr x86_64 2:19.1.0-1382.gde09dd01.el9 ceph 1.5 M 2024-08-19T01:22:42.142 INFO:teuthology.orchestra.run.smithi117.stdout: ceph-mgr-cephadm noarch 2:19.1.0-1382.gde09dd01.el9 ceph-noarch 146 k 2024-08-19T01:22:42.142 INFO:teuthology.orchestra.run.smithi117.stdout: ceph-mgr-dashboard noarch 2:19.1.0-1382.gde09dd01.el9 ceph-noarch 3.6 M 2024-08-19T01:22:42.143 INFO:teuthology.orchestra.run.smithi117.stdout: ceph-mgr-diskprediction-local noarch 2:19.1.0-1382.gde09dd01.el9 ceph-noarch 7.4 M 2024-08-19T01:22:42.143 INFO:teuthology.orchestra.run.smithi117.stdout: ceph-mgr-rook noarch 2:19.1.0-1382.gde09dd01.el9 ceph-noarch 49 k 2024-08-19T01:22:42.143 INFO:teuthology.orchestra.run.smithi117.stdout: ceph-radosgw x86_64 2:19.1.0-1382.gde09dd01.el9 ceph 11 M 2024-08-19T01:22:42.143 INFO:teuthology.orchestra.run.smithi117.stdout: ceph-test x86_64 2:19.1.0-1382.gde09dd01.el9 ceph 49 M 2024-08-19T01:22:42.143 INFO:teuthology.orchestra.run.smithi117.stdout: ceph-volume noarch 2:19.1.0-1382.gde09dd01.el9 ceph-noarch 285 k 2024-08-19T01:22:42.143 INFO:teuthology.orchestra.run.smithi117.stdout: cephadm noarch 2:19.1.0-1382.gde09dd01.el9 ceph-noarch 766 k 2024-08-19T01:22:42.143 INFO:teuthology.orchestra.run.smithi117.stdout: cephfs-mirror x86_64 2:19.1.0-1382.gde09dd01.el9 ceph 225 k 2024-08-19T01:22:42.143 INFO:teuthology.orchestra.run.smithi117.stdout: cephfs-shell x86_64 2:19.1.0-1382.gde09dd01.el9 ceph 27 k 2024-08-19T01:22:42.143 INFO:teuthology.orchestra.run.smithi117.stdout: cephfs-top noarch 2:19.1.0-1382.gde09dd01.el9 ceph-noarch 18 k 2024-08-19T01:22:42.143 INFO:teuthology.orchestra.run.smithi117.stdout: elfutils-libelf-devel x86_64 0.191-4.el9 appstream 22 k 2024-08-19T01:22:42.143 INFO:teuthology.orchestra.run.smithi117.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-08-19T01:22:42.143 INFO:teuthology.orchestra.run.smithi117.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-08-19T01:22:42.144 INFO:teuthology.orchestra.run.smithi117.stdout: libcephfs-devel x86_64 2:19.1.0-1382.gde09dd01.el9 ceph 32 k 2024-08-19T01:22:42.144 INFO:teuthology.orchestra.run.smithi117.stdout: libcephfs2 x86_64 2:19.1.0-1382.gde09dd01.el9 ceph 753 k 2024-08-19T01:22:42.144 INFO:teuthology.orchestra.run.smithi117.stdout: librados-devel x86_64 2:19.1.0-1382.gde09dd01.el9 ceph 127 k 2024-08-19T01:22:42.144 INFO:teuthology.orchestra.run.smithi117.stdout: libtool x86_64 2.4.6-46.el9 appstream 581 k 2024-08-19T01:22:42.144 INFO:teuthology.orchestra.run.smithi117.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-08-19T01:22:42.144 INFO:teuthology.orchestra.run.smithi117.stdout: openssl-devel x86_64 1:3.2.2-2.el9 appstream 4.4 M 2024-08-19T01:22:42.144 INFO:teuthology.orchestra.run.smithi117.stdout: postgresql x86_64 13.14-1.el9 appstream 1.6 M 2024-08-19T01:22:42.144 INFO:teuthology.orchestra.run.smithi117.stdout: postgresql-contrib x86_64 13.14-1.el9 appstream 911 k 2024-08-19T01:22:42.144 INFO:teuthology.orchestra.run.smithi117.stdout: postgresql-server x86_64 13.14-1.el9 appstream 5.8 M 2024-08-19T01:22:42.144 INFO:teuthology.orchestra.run.smithi117.stdout: python3-cephfs x86_64 2:19.1.0-1382.gde09dd01.el9 ceph 188 k 2024-08-19T01:22:42.144 INFO:teuthology.orchestra.run.smithi117.stdout: python3-rados x86_64 2:19.1.0-1382.gde09dd01.el9 ceph 369 k 2024-08-19T01:22:42.144 INFO:teuthology.orchestra.run.smithi117.stdout: python3-rbd x86_64 2:19.1.0-1382.gde09dd01.el9 ceph 350 k 2024-08-19T01:22:42.144 INFO:teuthology.orchestra.run.smithi117.stdout: python3-rgw x86_64 2:19.1.0-1382.gde09dd01.el9 ceph 117 k 2024-08-19T01:22:42.145 INFO:teuthology.orchestra.run.smithi117.stdout: rbd-fuse x86_64 2:19.1.0-1382.gde09dd01.el9 ceph 85 k 2024-08-19T01:22:42.145 INFO:teuthology.orchestra.run.smithi117.stdout: rbd-mirror x86_64 2:19.1.0-1382.gde09dd01.el9 ceph 3.1 M 2024-08-19T01:22:42.145 INFO:teuthology.orchestra.run.smithi117.stdout: rbd-nbd x86_64 2:19.1.0-1382.gde09dd01.el9 ceph 170 k 2024-08-19T01:22:42.145 INFO:teuthology.orchestra.run.smithi117.stdout: xfsprogs-devel x86_64 6.4.0-3.el9 appstream 96 k 2024-08-19T01:22:42.145 INFO:teuthology.orchestra.run.smithi117.stdout:Upgrading: 2024-08-19T01:22:42.145 INFO:teuthology.orchestra.run.smithi117.stdout: librados2 x86_64 2:19.1.0-1382.gde09dd01.el9 ceph 3.4 M 2024-08-19T01:22:42.145 INFO:teuthology.orchestra.run.smithi117.stdout: librbd1 x86_64 2:19.1.0-1382.gde09dd01.el9 ceph 3.2 M 2024-08-19T01:22:42.145 INFO:teuthology.orchestra.run.smithi117.stdout:Installing dependencies: 2024-08-19T01:22:42.145 INFO:teuthology.orchestra.run.smithi117.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-08-19T01:22:42.145 INFO:teuthology.orchestra.run.smithi117.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-08-19T01:22:42.145 INFO:teuthology.orchestra.run.smithi117.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-08-19T01:22:42.146 INFO:teuthology.orchestra.run.smithi117.stdout: ceph-common x86_64 2:19.1.0-1382.gde09dd01.el9 ceph 21 M 2024-08-19T01:22:42.146 INFO:teuthology.orchestra.run.smithi117.stdout: ceph-grafana-dashboards noarch 2:19.1.0-1382.gde09dd01.el9 ceph-noarch 32 k 2024-08-19T01:22:42.146 INFO:teuthology.orchestra.run.smithi117.stdout: ceph-mds x86_64 2:19.1.0-1382.gde09dd01.el9 ceph 2.4 M 2024-08-19T01:22:42.146 INFO:teuthology.orchestra.run.smithi117.stdout: ceph-mgr-modules-core noarch 2:19.1.0-1382.gde09dd01.el9 ceph-noarch 249 k 2024-08-19T01:22:42.146 INFO:teuthology.orchestra.run.smithi117.stdout: ceph-mon x86_64 2:19.1.0-1382.gde09dd01.el9 ceph 4.7 M 2024-08-19T01:22:42.146 INFO:teuthology.orchestra.run.smithi117.stdout: ceph-osd x86_64 2:19.1.0-1382.gde09dd01.el9 ceph 17 M 2024-08-19T01:22:42.146 INFO:teuthology.orchestra.run.smithi117.stdout: ceph-prometheus-alerts noarch 2:19.1.0-1382.gde09dd01.el9 ceph-noarch 16 k 2024-08-19T01:22:42.146 INFO:teuthology.orchestra.run.smithi117.stdout: ceph-selinux x86_64 2:19.1.0-1382.gde09dd01.el9 ceph 25 k 2024-08-19T01:22:42.146 INFO:teuthology.orchestra.run.smithi117.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-08-19T01:22:42.146 INFO:teuthology.orchestra.run.smithi117.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-08-19T01:22:42.146 INFO:teuthology.orchestra.run.smithi117.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-08-19T01:22:42.146 INFO:teuthology.orchestra.run.smithi117.stdout: fuse x86_64 2.9.9-16.el9 baseos 80 k 2024-08-19T01:22:42.146 INFO:teuthology.orchestra.run.smithi117.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-08-19T01:22:42.147 INFO:teuthology.orchestra.run.smithi117.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-08-19T01:22:42.147 INFO:teuthology.orchestra.run.smithi117.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-08-19T01:22:42.147 INFO:teuthology.orchestra.run.smithi117.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-08-19T01:22:42.147 INFO:teuthology.orchestra.run.smithi117.stdout: libcephsqlite x86_64 2:19.1.0-1382.gde09dd01.el9 ceph 163 k 2024-08-19T01:22:42.147 INFO:teuthology.orchestra.run.smithi117.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-08-19T01:22:42.147 INFO:teuthology.orchestra.run.smithi117.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-08-19T01:22:42.147 INFO:teuthology.orchestra.run.smithi117.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-08-19T01:22:42.147 INFO:teuthology.orchestra.run.smithi117.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-08-19T01:22:42.147 INFO:teuthology.orchestra.run.smithi117.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-08-19T01:22:42.147 INFO:teuthology.orchestra.run.smithi117.stdout: libradosstriper1 x86_64 2:19.1.0-1382.gde09dd01.el9 ceph 507 k 2024-08-19T01:22:42.147 INFO:teuthology.orchestra.run.smithi117.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-08-19T01:22:42.147 INFO:teuthology.orchestra.run.smithi117.stdout: librgw2 x86_64 2:19.1.0-1382.gde09dd01.el9 ceph 5.4 M 2024-08-19T01:22:42.147 INFO:teuthology.orchestra.run.smithi117.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-08-19T01:22:42.147 INFO:teuthology.orchestra.run.smithi117.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-08-19T01:22:42.148 INFO:teuthology.orchestra.run.smithi117.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-08-19T01:22:42.148 INFO:teuthology.orchestra.run.smithi117.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-08-19T01:22:42.148 INFO:teuthology.orchestra.run.smithi117.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-08-19T01:22:42.148 INFO:teuthology.orchestra.run.smithi117.stdout: luarocks noarch 3.9.2-5.el9 epel 151 k 2024-08-19T01:22:42.148 INFO:teuthology.orchestra.run.smithi117.stdout: openblas x86_64 0.3.21-2.el9 appstream 35 k 2024-08-19T01:22:42.148 INFO:teuthology.orchestra.run.smithi117.stdout: openblas-openmp x86_64 0.3.21-2.el9 appstream 4.7 M 2024-08-19T01:22:42.148 INFO:teuthology.orchestra.run.smithi117.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-08-19T01:22:42.148 INFO:teuthology.orchestra.run.smithi117.stdout: postgresql-private-libs x86_64 13.14-1.el9 appstream 133 k 2024-08-19T01:22:42.148 INFO:teuthology.orchestra.run.smithi117.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-08-19T01:22:42.148 INFO:teuthology.orchestra.run.smithi117.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-08-19T01:22:42.148 INFO:teuthology.orchestra.run.smithi117.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-08-19T01:22:42.148 INFO:teuthology.orchestra.run.smithi117.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-08-19T01:22:42.148 INFO:teuthology.orchestra.run.smithi117.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-08-19T01:22:42.148 INFO:teuthology.orchestra.run.smithi117.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-08-19T01:22:42.149 INFO:teuthology.orchestra.run.smithi117.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-08-19T01:22:42.149 INFO:teuthology.orchestra.run.smithi117.stdout: python3-ceph-argparse x86_64 2:19.1.0-1382.gde09dd01.el9 ceph 45 k 2024-08-19T01:22:42.149 INFO:teuthology.orchestra.run.smithi117.stdout: python3-ceph-common x86_64 2:19.1.0-1382.gde09dd01.el9 ceph 136 k 2024-08-19T01:22:42.149 INFO:teuthology.orchestra.run.smithi117.stdout: python3-certifi noarch 2023.05.07-4.el9 epel 14 k 2024-08-19T01:22:42.149 INFO:teuthology.orchestra.run.smithi117.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-08-19T01:22:42.149 INFO:teuthology.orchestra.run.smithi117.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-08-19T01:22:42.149 INFO:teuthology.orchestra.run.smithi117.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-08-19T01:22:42.149 INFO:teuthology.orchestra.run.smithi117.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-08-19T01:22:42.149 INFO:teuthology.orchestra.run.smithi117.stdout: python3-colorama noarch 0.4.6-3.el9 appstream 61 k 2024-08-19T01:22:42.149 INFO:teuthology.orchestra.run.smithi117.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-08-19T01:22:42.149 INFO:teuthology.orchestra.run.smithi117.stdout: python3-devel x86_64 3.9.19-4.el9 appstream 244 k 2024-08-19T01:22:42.149 INFO:teuthology.orchestra.run.smithi117.stdout: python3-google-auth noarch 1:2.33.0-1.el9 epel 220 k 2024-08-19T01:22:42.150 INFO:teuthology.orchestra.run.smithi117.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-08-19T01:22:42.150 INFO:teuthology.orchestra.run.smithi117.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-08-19T01:22:42.150 INFO:teuthology.orchestra.run.smithi117.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-08-19T01:22:42.150 INFO:teuthology.orchestra.run.smithi117.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-08-19T01:22:42.150 INFO:teuthology.orchestra.run.smithi117.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-08-19T01:22:42.150 INFO:teuthology.orchestra.run.smithi117.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-08-19T01:22:42.150 INFO:teuthology.orchestra.run.smithi117.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-08-19T01:22:42.150 INFO:teuthology.orchestra.run.smithi117.stdout: python3-jinja2 noarch 2.11.3-6.el9 appstream 248 k 2024-08-19T01:22:42.150 INFO:teuthology.orchestra.run.smithi117.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-08-19T01:22:42.150 INFO:teuthology.orchestra.run.smithi117.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-08-19T01:22:42.150 INFO:teuthology.orchestra.run.smithi117.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-08-19T01:22:42.150 INFO:teuthology.orchestra.run.smithi117.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-08-19T01:22:42.151 INFO:teuthology.orchestra.run.smithi117.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-08-19T01:22:42.151 INFO:teuthology.orchestra.run.smithi117.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-08-19T01:22:42.151 INFO:teuthology.orchestra.run.smithi117.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-08-19T01:22:42.151 INFO:teuthology.orchestra.run.smithi117.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-08-19T01:22:42.151 INFO:teuthology.orchestra.run.smithi117.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-08-19T01:22:42.151 INFO:teuthology.orchestra.run.smithi117.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-08-19T01:22:42.151 INFO:teuthology.orchestra.run.smithi117.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-08-19T01:22:42.151 INFO:teuthology.orchestra.run.smithi117.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-08-19T01:22:42.151 INFO:teuthology.orchestra.run.smithi117.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-08-19T01:22:42.151 INFO:teuthology.orchestra.run.smithi117.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-08-19T01:22:42.151 INFO:teuthology.orchestra.run.smithi117.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-08-19T01:22:42.151 INFO:teuthology.orchestra.run.smithi117.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-08-19T01:22:42.151 INFO:teuthology.orchestra.run.smithi117.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-08-19T01:22:42.151 INFO:teuthology.orchestra.run.smithi117.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-08-19T01:22:42.152 INFO:teuthology.orchestra.run.smithi117.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-08-19T01:22:42.152 INFO:teuthology.orchestra.run.smithi117.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-08-19T01:22:42.152 INFO:teuthology.orchestra.run.smithi117.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-08-19T01:22:42.152 INFO:teuthology.orchestra.run.smithi117.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-08-19T01:22:42.152 INFO:teuthology.orchestra.run.smithi117.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-08-19T01:22:42.152 INFO:teuthology.orchestra.run.smithi117.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-08-19T01:22:42.152 INFO:teuthology.orchestra.run.smithi117.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-08-19T01:22:42.152 INFO:teuthology.orchestra.run.smithi117.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-08-19T01:22:42.152 INFO:teuthology.orchestra.run.smithi117.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-08-19T01:22:42.152 INFO:teuthology.orchestra.run.smithi117.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-08-19T01:22:42.152 INFO:teuthology.orchestra.run.smithi117.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-08-19T01:22:42.152 INFO:teuthology.orchestra.run.smithi117.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-08-19T01:22:42.152 INFO:teuthology.orchestra.run.smithi117.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-08-19T01:22:42.152 INFO:teuthology.orchestra.run.smithi117.stdout: python3-typing-extensions noarch 4.12.2-2.el9 epel 74 k 2024-08-19T01:22:42.153 INFO:teuthology.orchestra.run.smithi117.stdout: python3-urllib3 noarch 1.26.5-6.el9 baseos 215 k 2024-08-19T01:22:42.153 INFO:teuthology.orchestra.run.smithi117.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-08-19T01:22:42.153 INFO:teuthology.orchestra.run.smithi117.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-08-19T01:22:42.153 INFO:teuthology.orchestra.run.smithi117.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-08-19T01:22:42.153 INFO:teuthology.orchestra.run.smithi117.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-08-19T01:22:42.153 INFO:teuthology.orchestra.run.smithi117.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-08-19T01:22:42.153 INFO:teuthology.orchestra.run.smithi117.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-08-19T01:22:42.153 INFO:teuthology.orchestra.run.smithi117.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-08-19T01:22:42.153 INFO:teuthology.orchestra.run.smithi117.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-08-19T01:22:42.153 INFO:teuthology.orchestra.run.smithi117.stdout: socat x86_64 1.7.4.1-6.el9 appstream 304 k 2024-08-19T01:22:42.153 INFO:teuthology.orchestra.run.smithi117.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-08-19T01:22:42.153 INFO:teuthology.orchestra.run.smithi117.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-08-19T01:22:42.153 INFO:teuthology.orchestra.run.smithi117.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-08-19T01:22:42.153 INFO:teuthology.orchestra.run.smithi117.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-08-19T01:22:42.154 INFO:teuthology.orchestra.run.smithi117.stdout:Installing weak dependencies: 2024-08-19T01:22:42.154 INFO:teuthology.orchestra.run.smithi117.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-08-19T01:22:42.154 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-08-19T01:22:42.154 INFO:teuthology.orchestra.run.smithi117.stdout:Transaction Summary 2024-08-19T01:22:42.154 INFO:teuthology.orchestra.run.smithi117.stdout:======================================================================================= 2024-08-19T01:22:42.154 INFO:teuthology.orchestra.run.smithi117.stdout:Install 148 Packages 2024-08-19T01:22:42.154 INFO:teuthology.orchestra.run.smithi117.stdout:Upgrade 2 Packages 2024-08-19T01:22:42.154 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-08-19T01:22:42.154 INFO:teuthology.orchestra.run.smithi117.stdout:Total download size: 218 M 2024-08-19T01:22:42.155 INFO:teuthology.orchestra.run.smithi117.stdout:Downloading Packages: 2024-08-19T01:22:42.195 INFO:teuthology.orchestra.run.smithi167.stdout:Dependencies resolved. 2024-08-19T01:22:42.210 INFO:teuthology.orchestra.run.smithi167.stdout:======================================================================================= 2024-08-19T01:22:42.210 INFO:teuthology.orchestra.run.smithi167.stdout: Package Arch Version Repository Size 2024-08-19T01:22:42.210 INFO:teuthology.orchestra.run.smithi167.stdout:======================================================================================= 2024-08-19T01:22:42.210 INFO:teuthology.orchestra.run.smithi167.stdout:Installing: 2024-08-19T01:22:42.210 INFO:teuthology.orchestra.run.smithi167.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-08-19T01:22:42.210 INFO:teuthology.orchestra.run.smithi167.stdout: ceph x86_64 2:19.1.0-1382.gde09dd01.el9 ceph 6.6 k 2024-08-19T01:22:42.211 INFO:teuthology.orchestra.run.smithi167.stdout: ceph-base x86_64 2:19.1.0-1382.gde09dd01.el9 ceph 5.4 M 2024-08-19T01:22:42.211 INFO:teuthology.orchestra.run.smithi167.stdout: ceph-fuse x86_64 2:19.1.0-1382.gde09dd01.el9 ceph 896 k 2024-08-19T01:22:42.211 INFO:teuthology.orchestra.run.smithi167.stdout: ceph-immutable-object-cache x86_64 2:19.1.0-1382.gde09dd01.el9 ceph 145 k 2024-08-19T01:22:42.211 INFO:teuthology.orchestra.run.smithi167.stdout: ceph-mgr x86_64 2:19.1.0-1382.gde09dd01.el9 ceph 1.5 M 2024-08-19T01:22:42.211 INFO:teuthology.orchestra.run.smithi167.stdout: ceph-mgr-cephadm noarch 2:19.1.0-1382.gde09dd01.el9 ceph-noarch 146 k 2024-08-19T01:22:42.211 INFO:teuthology.orchestra.run.smithi167.stdout: ceph-mgr-dashboard noarch 2:19.1.0-1382.gde09dd01.el9 ceph-noarch 3.6 M 2024-08-19T01:22:42.211 INFO:teuthology.orchestra.run.smithi167.stdout: ceph-mgr-diskprediction-local noarch 2:19.1.0-1382.gde09dd01.el9 ceph-noarch 7.4 M 2024-08-19T01:22:42.211 INFO:teuthology.orchestra.run.smithi167.stdout: ceph-mgr-rook noarch 2:19.1.0-1382.gde09dd01.el9 ceph-noarch 49 k 2024-08-19T01:22:42.211 INFO:teuthology.orchestra.run.smithi167.stdout: ceph-radosgw x86_64 2:19.1.0-1382.gde09dd01.el9 ceph 11 M 2024-08-19T01:22:42.211 INFO:teuthology.orchestra.run.smithi167.stdout: ceph-test x86_64 2:19.1.0-1382.gde09dd01.el9 ceph 49 M 2024-08-19T01:22:42.211 INFO:teuthology.orchestra.run.smithi167.stdout: ceph-volume noarch 2:19.1.0-1382.gde09dd01.el9 ceph-noarch 285 k 2024-08-19T01:22:42.211 INFO:teuthology.orchestra.run.smithi167.stdout: cephadm noarch 2:19.1.0-1382.gde09dd01.el9 ceph-noarch 766 k 2024-08-19T01:22:42.211 INFO:teuthology.orchestra.run.smithi167.stdout: cephfs-mirror x86_64 2:19.1.0-1382.gde09dd01.el9 ceph 225 k 2024-08-19T01:22:42.212 INFO:teuthology.orchestra.run.smithi167.stdout: cephfs-shell x86_64 2:19.1.0-1382.gde09dd01.el9 ceph 27 k 2024-08-19T01:22:42.212 INFO:teuthology.orchestra.run.smithi167.stdout: cephfs-top noarch 2:19.1.0-1382.gde09dd01.el9 ceph-noarch 18 k 2024-08-19T01:22:42.212 INFO:teuthology.orchestra.run.smithi167.stdout: elfutils-libelf-devel x86_64 0.191-4.el9 appstream 22 k 2024-08-19T01:22:42.212 INFO:teuthology.orchestra.run.smithi167.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-08-19T01:22:42.212 INFO:teuthology.orchestra.run.smithi167.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-08-19T01:22:42.212 INFO:teuthology.orchestra.run.smithi167.stdout: libcephfs-devel x86_64 2:19.1.0-1382.gde09dd01.el9 ceph 32 k 2024-08-19T01:22:42.212 INFO:teuthology.orchestra.run.smithi167.stdout: libcephfs2 x86_64 2:19.1.0-1382.gde09dd01.el9 ceph 753 k 2024-08-19T01:22:42.212 INFO:teuthology.orchestra.run.smithi167.stdout: librados-devel x86_64 2:19.1.0-1382.gde09dd01.el9 ceph 127 k 2024-08-19T01:22:42.212 INFO:teuthology.orchestra.run.smithi167.stdout: libtool x86_64 2.4.6-46.el9 appstream 581 k 2024-08-19T01:22:42.212 INFO:teuthology.orchestra.run.smithi167.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-08-19T01:22:42.212 INFO:teuthology.orchestra.run.smithi167.stdout: openssl-devel x86_64 1:3.2.2-2.el9 appstream 4.4 M 2024-08-19T01:22:42.212 INFO:teuthology.orchestra.run.smithi167.stdout: postgresql x86_64 13.14-1.el9 appstream 1.6 M 2024-08-19T01:22:42.212 INFO:teuthology.orchestra.run.smithi167.stdout: postgresql-contrib x86_64 13.14-1.el9 appstream 911 k 2024-08-19T01:22:42.213 INFO:teuthology.orchestra.run.smithi167.stdout: postgresql-server x86_64 13.14-1.el9 appstream 5.8 M 2024-08-19T01:22:42.213 INFO:teuthology.orchestra.run.smithi167.stdout: python3-cephfs x86_64 2:19.1.0-1382.gde09dd01.el9 ceph 188 k 2024-08-19T01:22:42.213 INFO:teuthology.orchestra.run.smithi167.stdout: python3-rados x86_64 2:19.1.0-1382.gde09dd01.el9 ceph 369 k 2024-08-19T01:22:42.213 INFO:teuthology.orchestra.run.smithi167.stdout: python3-rbd x86_64 2:19.1.0-1382.gde09dd01.el9 ceph 350 k 2024-08-19T01:22:42.213 INFO:teuthology.orchestra.run.smithi167.stdout: python3-rgw x86_64 2:19.1.0-1382.gde09dd01.el9 ceph 117 k 2024-08-19T01:22:42.213 INFO:teuthology.orchestra.run.smithi167.stdout: rbd-fuse x86_64 2:19.1.0-1382.gde09dd01.el9 ceph 85 k 2024-08-19T01:22:42.213 INFO:teuthology.orchestra.run.smithi167.stdout: rbd-mirror x86_64 2:19.1.0-1382.gde09dd01.el9 ceph 3.1 M 2024-08-19T01:22:42.213 INFO:teuthology.orchestra.run.smithi167.stdout: rbd-nbd x86_64 2:19.1.0-1382.gde09dd01.el9 ceph 170 k 2024-08-19T01:22:42.213 INFO:teuthology.orchestra.run.smithi167.stdout: xfsprogs-devel x86_64 6.4.0-3.el9 appstream 96 k 2024-08-19T01:22:42.213 INFO:teuthology.orchestra.run.smithi167.stdout:Upgrading: 2024-08-19T01:22:42.213 INFO:teuthology.orchestra.run.smithi167.stdout: librados2 x86_64 2:19.1.0-1382.gde09dd01.el9 ceph 3.4 M 2024-08-19T01:22:42.213 INFO:teuthology.orchestra.run.smithi167.stdout: librbd1 x86_64 2:19.1.0-1382.gde09dd01.el9 ceph 3.2 M 2024-08-19T01:22:42.213 INFO:teuthology.orchestra.run.smithi167.stdout:Installing dependencies: 2024-08-19T01:22:42.214 INFO:teuthology.orchestra.run.smithi167.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-08-19T01:22:42.214 INFO:teuthology.orchestra.run.smithi167.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-08-19T01:22:42.214 INFO:teuthology.orchestra.run.smithi167.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-08-19T01:22:42.214 INFO:teuthology.orchestra.run.smithi167.stdout: ceph-common x86_64 2:19.1.0-1382.gde09dd01.el9 ceph 21 M 2024-08-19T01:22:42.214 INFO:teuthology.orchestra.run.smithi167.stdout: ceph-grafana-dashboards noarch 2:19.1.0-1382.gde09dd01.el9 ceph-noarch 32 k 2024-08-19T01:22:42.214 INFO:teuthology.orchestra.run.smithi167.stdout: ceph-mds x86_64 2:19.1.0-1382.gde09dd01.el9 ceph 2.4 M 2024-08-19T01:22:42.214 INFO:teuthology.orchestra.run.smithi167.stdout: ceph-mgr-modules-core noarch 2:19.1.0-1382.gde09dd01.el9 ceph-noarch 249 k 2024-08-19T01:22:42.214 INFO:teuthology.orchestra.run.smithi167.stdout: ceph-mon x86_64 2:19.1.0-1382.gde09dd01.el9 ceph 4.7 M 2024-08-19T01:22:42.214 INFO:teuthology.orchestra.run.smithi167.stdout: ceph-osd x86_64 2:19.1.0-1382.gde09dd01.el9 ceph 17 M 2024-08-19T01:22:42.214 INFO:teuthology.orchestra.run.smithi167.stdout: ceph-prometheus-alerts noarch 2:19.1.0-1382.gde09dd01.el9 ceph-noarch 16 k 2024-08-19T01:22:42.214 INFO:teuthology.orchestra.run.smithi167.stdout: ceph-selinux x86_64 2:19.1.0-1382.gde09dd01.el9 ceph 25 k 2024-08-19T01:22:42.214 INFO:teuthology.orchestra.run.smithi167.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-08-19T01:22:42.214 INFO:teuthology.orchestra.run.smithi167.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-08-19T01:22:42.215 INFO:teuthology.orchestra.run.smithi167.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-08-19T01:22:42.215 INFO:teuthology.orchestra.run.smithi167.stdout: fuse x86_64 2.9.9-16.el9 baseos 80 k 2024-08-19T01:22:42.215 INFO:teuthology.orchestra.run.smithi167.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-08-19T01:22:42.215 INFO:teuthology.orchestra.run.smithi167.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-08-19T01:22:42.215 INFO:teuthology.orchestra.run.smithi167.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-08-19T01:22:42.215 INFO:teuthology.orchestra.run.smithi167.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-08-19T01:22:42.215 INFO:teuthology.orchestra.run.smithi167.stdout: libcephsqlite x86_64 2:19.1.0-1382.gde09dd01.el9 ceph 163 k 2024-08-19T01:22:42.215 INFO:teuthology.orchestra.run.smithi167.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-08-19T01:22:42.215 INFO:teuthology.orchestra.run.smithi167.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-08-19T01:22:42.215 INFO:teuthology.orchestra.run.smithi167.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-08-19T01:22:42.215 INFO:teuthology.orchestra.run.smithi167.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-08-19T01:22:42.215 INFO:teuthology.orchestra.run.smithi167.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-08-19T01:22:42.215 INFO:teuthology.orchestra.run.smithi167.stdout: libradosstriper1 x86_64 2:19.1.0-1382.gde09dd01.el9 ceph 507 k 2024-08-19T01:22:42.215 INFO:teuthology.orchestra.run.smithi167.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-08-19T01:22:42.216 INFO:teuthology.orchestra.run.smithi167.stdout: librgw2 x86_64 2:19.1.0-1382.gde09dd01.el9 ceph 5.4 M 2024-08-19T01:22:42.216 INFO:teuthology.orchestra.run.smithi167.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-08-19T01:22:42.216 INFO:teuthology.orchestra.run.smithi167.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-08-19T01:22:42.216 INFO:teuthology.orchestra.run.smithi167.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-08-19T01:22:42.216 INFO:teuthology.orchestra.run.smithi167.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-08-19T01:22:42.216 INFO:teuthology.orchestra.run.smithi167.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-08-19T01:22:42.216 INFO:teuthology.orchestra.run.smithi167.stdout: luarocks noarch 3.9.2-5.el9 epel 151 k 2024-08-19T01:22:42.216 INFO:teuthology.orchestra.run.smithi167.stdout: openblas x86_64 0.3.21-2.el9 appstream 35 k 2024-08-19T01:22:42.216 INFO:teuthology.orchestra.run.smithi167.stdout: openblas-openmp x86_64 0.3.21-2.el9 appstream 4.7 M 2024-08-19T01:22:42.216 INFO:teuthology.orchestra.run.smithi167.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-08-19T01:22:42.216 INFO:teuthology.orchestra.run.smithi167.stdout: postgresql-private-libs x86_64 13.14-1.el9 appstream 133 k 2024-08-19T01:22:42.216 INFO:teuthology.orchestra.run.smithi167.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-08-19T01:22:42.216 INFO:teuthology.orchestra.run.smithi167.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-08-19T01:22:42.217 INFO:teuthology.orchestra.run.smithi167.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-08-19T01:22:42.217 INFO:teuthology.orchestra.run.smithi167.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-08-19T01:22:42.217 INFO:teuthology.orchestra.run.smithi167.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-08-19T01:22:42.217 INFO:teuthology.orchestra.run.smithi167.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-08-19T01:22:42.217 INFO:teuthology.orchestra.run.smithi167.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-08-19T01:22:42.217 INFO:teuthology.orchestra.run.smithi167.stdout: python3-ceph-argparse x86_64 2:19.1.0-1382.gde09dd01.el9 ceph 45 k 2024-08-19T01:22:42.217 INFO:teuthology.orchestra.run.smithi167.stdout: python3-ceph-common x86_64 2:19.1.0-1382.gde09dd01.el9 ceph 136 k 2024-08-19T01:22:42.217 INFO:teuthology.orchestra.run.smithi167.stdout: python3-certifi noarch 2023.05.07-4.el9 epel 14 k 2024-08-19T01:22:42.217 INFO:teuthology.orchestra.run.smithi167.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-08-19T01:22:42.217 INFO:teuthology.orchestra.run.smithi167.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-08-19T01:22:42.217 INFO:teuthology.orchestra.run.smithi167.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-08-19T01:22:42.217 INFO:teuthology.orchestra.run.smithi167.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-08-19T01:22:42.217 INFO:teuthology.orchestra.run.smithi167.stdout: python3-colorama noarch 0.4.6-3.el9 appstream 61 k 2024-08-19T01:22:42.218 INFO:teuthology.orchestra.run.smithi167.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-08-19T01:22:42.218 INFO:teuthology.orchestra.run.smithi167.stdout: python3-devel x86_64 3.9.19-4.el9 appstream 244 k 2024-08-19T01:22:42.218 INFO:teuthology.orchestra.run.smithi167.stdout: python3-google-auth noarch 1:2.33.0-1.el9 epel 220 k 2024-08-19T01:22:42.218 INFO:teuthology.orchestra.run.smithi167.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-08-19T01:22:42.218 INFO:teuthology.orchestra.run.smithi167.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-08-19T01:22:42.218 INFO:teuthology.orchestra.run.smithi167.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-08-19T01:22:42.218 INFO:teuthology.orchestra.run.smithi167.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-08-19T01:22:42.218 INFO:teuthology.orchestra.run.smithi167.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-08-19T01:22:42.218 INFO:teuthology.orchestra.run.smithi167.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-08-19T01:22:42.218 INFO:teuthology.orchestra.run.smithi167.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-08-19T01:22:42.218 INFO:teuthology.orchestra.run.smithi167.stdout: python3-jinja2 noarch 2.11.3-6.el9 appstream 248 k 2024-08-19T01:22:42.218 INFO:teuthology.orchestra.run.smithi167.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-08-19T01:22:42.218 INFO:teuthology.orchestra.run.smithi167.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-08-19T01:22:42.218 INFO:teuthology.orchestra.run.smithi167.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-08-19T01:22:42.219 INFO:teuthology.orchestra.run.smithi167.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-08-19T01:22:42.219 INFO:teuthology.orchestra.run.smithi167.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-08-19T01:22:42.219 INFO:teuthology.orchestra.run.smithi167.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-08-19T01:22:42.219 INFO:teuthology.orchestra.run.smithi167.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-08-19T01:22:42.219 INFO:teuthology.orchestra.run.smithi167.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-08-19T01:22:42.219 INFO:teuthology.orchestra.run.smithi167.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-08-19T01:22:42.219 INFO:teuthology.orchestra.run.smithi167.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-08-19T01:22:42.219 INFO:teuthology.orchestra.run.smithi167.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-08-19T01:22:42.219 INFO:teuthology.orchestra.run.smithi167.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-08-19T01:22:42.219 INFO:teuthology.orchestra.run.smithi167.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-08-19T01:22:42.219 INFO:teuthology.orchestra.run.smithi167.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-08-19T01:22:42.219 INFO:teuthology.orchestra.run.smithi167.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-08-19T01:22:42.219 INFO:teuthology.orchestra.run.smithi167.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-08-19T01:22:42.219 INFO:teuthology.orchestra.run.smithi167.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-08-19T01:22:42.220 INFO:teuthology.orchestra.run.smithi167.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-08-19T01:22:42.220 INFO:teuthology.orchestra.run.smithi167.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-08-19T01:22:42.220 INFO:teuthology.orchestra.run.smithi167.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-08-19T01:22:42.220 INFO:teuthology.orchestra.run.smithi167.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-08-19T01:22:42.220 INFO:teuthology.orchestra.run.smithi167.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-08-19T01:22:42.220 INFO:teuthology.orchestra.run.smithi167.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-08-19T01:22:42.220 INFO:teuthology.orchestra.run.smithi167.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-08-19T01:22:42.220 INFO:teuthology.orchestra.run.smithi167.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-08-19T01:22:42.220 INFO:teuthology.orchestra.run.smithi167.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-08-19T01:22:42.220 INFO:teuthology.orchestra.run.smithi167.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-08-19T01:22:42.220 INFO:teuthology.orchestra.run.smithi167.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-08-19T01:22:42.220 INFO:teuthology.orchestra.run.smithi167.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-08-19T01:22:42.220 INFO:teuthology.orchestra.run.smithi167.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-08-19T01:22:42.220 INFO:teuthology.orchestra.run.smithi167.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-08-19T01:22:42.221 INFO:teuthology.orchestra.run.smithi167.stdout: python3-typing-extensions noarch 4.12.2-2.el9 epel 74 k 2024-08-19T01:22:42.221 INFO:teuthology.orchestra.run.smithi167.stdout: python3-urllib3 noarch 1.26.5-6.el9 baseos 215 k 2024-08-19T01:22:42.221 INFO:teuthology.orchestra.run.smithi167.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-08-19T01:22:42.221 INFO:teuthology.orchestra.run.smithi167.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-08-19T01:22:42.221 INFO:teuthology.orchestra.run.smithi167.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-08-19T01:22:42.221 INFO:teuthology.orchestra.run.smithi167.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-08-19T01:22:42.221 INFO:teuthology.orchestra.run.smithi167.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-08-19T01:22:42.221 INFO:teuthology.orchestra.run.smithi167.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-08-19T01:22:42.221 INFO:teuthology.orchestra.run.smithi167.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-08-19T01:22:42.221 INFO:teuthology.orchestra.run.smithi167.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-08-19T01:22:42.221 INFO:teuthology.orchestra.run.smithi167.stdout: socat x86_64 1.7.4.1-6.el9 appstream 304 k 2024-08-19T01:22:42.221 INFO:teuthology.orchestra.run.smithi167.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-08-19T01:22:42.221 INFO:teuthology.orchestra.run.smithi167.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-08-19T01:22:42.221 INFO:teuthology.orchestra.run.smithi167.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-08-19T01:22:42.222 INFO:teuthology.orchestra.run.smithi167.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-08-19T01:22:42.222 INFO:teuthology.orchestra.run.smithi167.stdout:Installing weak dependencies: 2024-08-19T01:22:42.222 INFO:teuthology.orchestra.run.smithi167.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-08-19T01:22:42.222 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-08-19T01:22:42.222 INFO:teuthology.orchestra.run.smithi167.stdout:Transaction Summary 2024-08-19T01:22:42.222 INFO:teuthology.orchestra.run.smithi167.stdout:======================================================================================= 2024-08-19T01:22:42.222 INFO:teuthology.orchestra.run.smithi167.stdout:Install 148 Packages 2024-08-19T01:22:42.222 INFO:teuthology.orchestra.run.smithi167.stdout:Upgrade 2 Packages 2024-08-19T01:22:42.222 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-08-19T01:22:42.223 INFO:teuthology.orchestra.run.smithi167.stdout:Total download size: 218 M 2024-08-19T01:22:42.223 INFO:teuthology.orchestra.run.smithi167.stdout:Downloading Packages: 2024-08-19T01:22:42.636 INFO:teuthology.orchestra.run.smithi167.stdout:(1/150): ceph-19.1.0-1382.gde09dd01.el9.x86_64. 53 kB/s | 6.6 kB 00:00 2024-08-19T01:22:42.853 INFO:teuthology.orchestra.run.smithi167.stdout:(2/150): ceph-fuse-19.1.0-1382.gde09dd01.el9.x8 4.0 MB/s | 896 kB 00:00 2024-08-19T01:22:42.937 INFO:teuthology.orchestra.run.smithi167.stdout:(3/150): ceph-immutable-object-cache-19.1.0-138 1.7 MB/s | 145 kB 00:00 2024-08-19T01:22:43.037 INFO:teuthology.orchestra.run.smithi167.stdout:(4/150): ceph-base-19.1.0-1382.gde09dd01.el9.x8 10 MB/s | 5.4 MB 00:00 2024-08-19T01:22:43.072 INFO:teuthology.orchestra.run.smithi117.stdout:(1/150): ceph-19.1.0-1382.gde09dd01.el9.x86_64. 31 kB/s | 6.6 kB 00:00 2024-08-19T01:22:43.112 INFO:teuthology.orchestra.run.smithi167.stdout:(5/150): ceph-mds-19.1.0-1382.gde09dd01.el9.x86 14 MB/s | 2.4 MB 00:00 2024-08-19T01:22:43.171 INFO:teuthology.orchestra.run.smithi167.stdout:(6/150): ceph-mgr-19.1.0-1382.gde09dd01.el9.x86 11 MB/s | 1.5 MB 00:00 2024-08-19T01:22:43.479 INFO:teuthology.orchestra.run.smithi167.stdout:(7/150): ceph-mon-19.1.0-1382.gde09dd01.el9.x86 13 MB/s | 4.7 MB 00:00 2024-08-19T01:22:43.688 INFO:teuthology.orchestra.run.smithi167.stdout:(8/150): ceph-common-19.1.0-1382.gde09dd01.el9. 18 MB/s | 21 MB 00:01 2024-08-19T01:22:43.730 INFO:teuthology.orchestra.run.smithi167.stdout:(9/150): ceph-selinux-19.1.0-1382.gde09dd01.el9 608 kB/s | 25 kB 00:00 2024-08-19T01:22:43.906 INFO:teuthology.orchestra.run.smithi117.stdout:(2/150): ceph-fuse-19.1.0-1382.gde09dd01.el9.x8 1.1 MB/s | 896 kB 00:00 2024-08-19T01:22:44.064 INFO:teuthology.orchestra.run.smithi117.stdout:(3/150): ceph-immutable-object-cache-19.1.0-138 917 kB/s | 145 kB 00:00 2024-08-19T01:22:44.105 INFO:teuthology.orchestra.run.smithi167.stdout:(10/150): ceph-radosgw-19.1.0-1382.gde09dd01.el 17 MB/s | 11 MB 00:00 2024-08-19T01:22:44.305 INFO:teuthology.orchestra.run.smithi167.stdout:(11/150): ceph-osd-19.1.0-1382.gde09dd01.el9.x8 15 MB/s | 17 MB 00:01 2024-08-19T01:22:44.330 INFO:teuthology.orchestra.run.smithi167.stdout:(12/150): cephfs-mirror-19.1.0-1382.gde09dd01.e 999 kB/s | 225 kB 00:00 2024-08-19T01:22:44.356 INFO:teuthology.orchestra.run.smithi167.stdout:(13/150): cephfs-shell-19.1.0-1382.gde09dd01.el 541 kB/s | 27 kB 00:00 2024-08-19T01:22:44.381 INFO:teuthology.orchestra.run.smithi167.stdout:(14/150): libcephfs-devel-19.1.0-1382.gde09dd01 633 kB/s | 32 kB 00:00 2024-08-19T01:22:44.414 INFO:teuthology.orchestra.run.smithi167.stdout:(15/150): libcephfs2-19.1.0-1382.gde09dd01.el9. 13 MB/s | 753 kB 00:00 2024-08-19T01:22:44.440 INFO:teuthology.orchestra.run.smithi167.stdout:(16/150): libcephsqlite-19.1.0-1382.gde09dd01.e 2.7 MB/s | 163 kB 00:00 2024-08-19T01:22:44.465 INFO:teuthology.orchestra.run.smithi167.stdout:(17/150): librados-devel-19.1.0-1382.gde09dd01. 2.5 MB/s | 127 kB 00:00 2024-08-19T01:22:44.490 INFO:teuthology.orchestra.run.smithi167.stdout:(18/150): libradosstriper1-19.1.0-1382.gde09dd0 9.8 MB/s | 507 kB 00:00 2024-08-19T01:22:44.549 INFO:teuthology.orchestra.run.smithi167.stdout:(19/150): python3-ceph-argparse-19.1.0-1382.gde 772 kB/s | 45 kB 00:00 2024-08-19T01:22:44.625 INFO:teuthology.orchestra.run.smithi167.stdout:(20/150): python3-ceph-common-19.1.0-1382.gde09 1.8 MB/s | 136 kB 00:00 2024-08-19T01:22:44.750 INFO:teuthology.orchestra.run.smithi167.stdout:(21/150): librgw2-19.1.0-1382.gde09dd01.el9.x86 19 MB/s | 5.4 MB 00:00 2024-08-19T01:22:44.775 INFO:teuthology.orchestra.run.smithi167.stdout:(22/150): python3-cephfs-19.1.0-1382.gde09dd01. 1.2 MB/s | 188 kB 00:00 2024-08-19T01:22:44.817 INFO:teuthology.orchestra.run.smithi167.stdout:(23/150): python3-rados-19.1.0-1382.gde09dd01.e 5.4 MB/s | 369 kB 00:00 2024-08-19T01:22:44.842 INFO:teuthology.orchestra.run.smithi167.stdout:(24/150): python3-rbd-19.1.0-1382.gde09dd01.el9 5.1 MB/s | 350 kB 00:00 2024-08-19T01:22:44.876 INFO:teuthology.orchestra.run.smithi167.stdout:(25/150): python3-rgw-19.1.0-1382.gde09dd01.el9 1.9 MB/s | 117 kB 00:00 2024-08-19T01:22:44.901 INFO:teuthology.orchestra.run.smithi167.stdout:(26/150): rbd-fuse-19.1.0-1382.gde09dd01.el9.x8 1.4 MB/s | 85 kB 00:00 2024-08-19T01:22:44.985 INFO:teuthology.orchestra.run.smithi167.stdout:(27/150): rbd-nbd-19.1.0-1382.gde09dd01.el9.x86 2.0 MB/s | 170 kB 00:00 2024-08-19T01:22:45.043 INFO:teuthology.orchestra.run.smithi167.stdout:(28/150): rbd-mirror-19.1.0-1382.gde09dd01.el9. 19 MB/s | 3.1 MB 00:00 2024-08-19T01:22:45.047 INFO:teuthology.orchestra.run.smithi117.stdout:(4/150): ceph-mds-19.1.0-1382.gde09dd01.el9.x86 2.4 MB/s | 2.4 MB 00:00 2024-08-19T01:22:45.069 INFO:teuthology.orchestra.run.smithi167.stdout:(29/150): ceph-grafana-dashboards-19.1.0-1382.g 380 kB/s | 32 kB 00:00 2024-08-19T01:22:45.094 INFO:teuthology.orchestra.run.smithi167.stdout:(30/150): ceph-mgr-cephadm-19.1.0-1382.gde09dd0 2.8 MB/s | 146 kB 00:00 2024-08-19T01:22:45.395 INFO:teuthology.orchestra.run.smithi167.stdout:(31/150): ceph-mgr-dashboard-19.1.0-1382.gde09d 11 MB/s | 3.6 MB 00:00 2024-08-19T01:22:45.486 INFO:teuthology.orchestra.run.smithi167.stdout:(32/150): ceph-mgr-diskprediction-local-19.1.0- 19 MB/s | 7.4 MB 00:00 2024-08-19T01:22:45.512 INFO:teuthology.orchestra.run.smithi167.stdout:(33/150): ceph-mgr-modules-core-19.1.0-1382.gde 2.1 MB/s | 249 kB 00:00 2024-08-19T01:22:45.537 INFO:teuthology.orchestra.run.smithi167.stdout:(34/150): ceph-mgr-rook-19.1.0-1382.gde09dd01.e 982 kB/s | 49 kB 00:00 2024-08-19T01:22:45.562 INFO:teuthology.orchestra.run.smithi167.stdout:(35/150): ceph-prometheus-alerts-19.1.0-1382.gd 326 kB/s | 16 kB 00:00 2024-08-19T01:22:45.596 INFO:teuthology.orchestra.run.smithi167.stdout:(36/150): ceph-volume-19.1.0-1382.gde09dd01.el9 4.8 MB/s | 285 kB 00:00 2024-08-19T01:22:45.630 INFO:teuthology.orchestra.run.smithi167.stdout:(37/150): cephadm-19.1.0-1382.gde09dd01.el9.noa 11 MB/s | 766 kB 00:00 2024-08-19T01:22:45.631 INFO:teuthology.orchestra.run.smithi117.stdout:(5/150): ceph-base-19.1.0-1382.gde09dd01.el9.x8 1.9 MB/s | 5.4 MB 00:02 2024-08-19T01:22:45.655 INFO:teuthology.orchestra.run.smithi167.stdout:(38/150): cephfs-top-19.1.0-1382.gde09dd01.el9. 312 kB/s | 18 kB 00:00 2024-08-19T01:22:45.731 INFO:teuthology.orchestra.run.smithi117.stdout:(6/150): ceph-mgr-19.1.0-1382.gde09dd01.el9.x86 2.2 MB/s | 1.5 MB 00:00 2024-08-19T01:22:45.815 INFO:teuthology.orchestra.run.smithi167.stdout:(39/150): fuse-2.9.9-16.el9.x86_64.rpm 430 kB/s | 80 kB 00:00 2024-08-19T01:22:45.897 INFO:teuthology.orchestra.run.smithi167.stdout:(40/150): libquadmath-11.4.1-3.el9.x86_64.rpm 2.3 MB/s | 195 kB 00:00 2024-08-19T01:22:45.956 INFO:teuthology.orchestra.run.smithi167.stdout:(41/150): libgfortran-11.4.1-3.el9.x86_64.rpm 2.6 MB/s | 803 kB 00:00 2024-08-19T01:22:46.381 INFO:teuthology.orchestra.run.smithi167.stdout:(42/150): ceph-test-19.1.0-1382.gde09dd01.el9.x 18 MB/s | 49 MB 00:02 2024-08-19T01:22:46.406 INFO:teuthology.orchestra.run.smithi167.stdout:(43/150): python3-cffi-1.14.5-5.el9.x86_64.rpm 497 kB/s | 253 kB 00:00 2024-08-19T01:22:46.506 INFO:teuthology.orchestra.run.smithi167.stdout:(44/150): python3-pycparser-2.20-6.el9.noarch.r 1.3 MB/s | 135 kB 00:00 2024-08-19T01:22:46.540 INFO:teuthology.orchestra.run.smithi167.stdout:(45/150): python3-ply-3.11-14.el9.noarch.rpm 672 kB/s | 106 kB 00:00 2024-08-19T01:22:46.640 INFO:teuthology.orchestra.run.smithi117.stdout:(7/150): ceph-mon-19.1.0-1382.gde09dd01.el9.x86 4.7 MB/s | 4.7 MB 00:01 2024-08-19T01:22:46.641 INFO:teuthology.orchestra.run.smithi167.stdout:(46/150): python3-cryptography-36.0.1-4.el9.x86 1.8 MB/s | 1.2 MB 00:00 2024-08-19T01:22:46.666 INFO:teuthology.orchestra.run.smithi167.stdout:(47/150): python3-urllib3-1.26.5-6.el9.noarch.r 1.7 MB/s | 215 kB 00:00 2024-08-19T01:22:46.691 INFO:teuthology.orchestra.run.smithi167.stdout:(48/150): python3-requests-2.25.1-8.el9.noarch. 677 kB/s | 125 kB 00:00 2024-08-19T01:22:46.867 INFO:teuthology.orchestra.run.smithi167.stdout:(49/150): boost-program-options-1.75.0-8.el9.x8 611 kB/s | 107 kB 00:00 2024-08-19T01:22:46.917 INFO:teuthology.orchestra.run.smithi167.stdout:(50/150): elfutils-libelf-devel-0.191-4.el9.x86 434 kB/s | 22 kB 00:00 2024-08-19T01:22:46.942 INFO:teuthology.orchestra.run.smithi167.stdout:(51/150): automake-1.16.2-8.el9.noarch.rpm 2.2 MB/s | 689 kB 00:00 2024-08-19T01:22:47.009 INFO:teuthology.orchestra.run.smithi167.stdout:(52/150): bison-3.7.4-5.el9.x86_64.rpm 2.7 MB/s | 943 kB 00:00 2024-08-19T01:22:47.034 INFO:teuthology.orchestra.run.smithi167.stdout:(53/150): flexiblas-3.0.4-8.el9.x86_64.rpm 355 kB/s | 33 kB 00:00 2024-08-19T01:22:47.068 INFO:teuthology.orchestra.run.smithi167.stdout:(54/150): flex-2.6.4-9.el9.x86_64.rpm 2.0 MB/s | 313 kB 00:00 2024-08-19T01:22:47.085 INFO:teuthology.orchestra.run.smithi167.stdout:(55/150): flexiblas-openblas-openmp-3.0.4-8.el9 363 kB/s | 18 kB 00:00 2024-08-19T01:22:47.091 INFO:teuthology.orchestra.run.smithi083.stdout:lab-extras 32 kB/s | 1.7 kB 00:00 2024-08-19T01:22:47.106 INFO:teuthology.orchestra.run.smithi117.stdout:(8/150): ceph-common-19.1.0-1382.gde09dd01.el9. 5.0 MB/s | 21 MB 00:04 2024-08-19T01:22:47.110 INFO:teuthology.orchestra.run.smithi167.stdout:(56/150): libaio-devel-0.3.111-13.el9.x86_64.rp 267 kB/s | 11 kB 00:00 2024-08-19T01:22:47.144 INFO:teuthology.orchestra.run.smithi167.stdout:(57/150): libpmemobj-1.12.1-1.el9.x86_64.rpm 2.7 MB/s | 160 kB 00:00 2024-08-19T01:22:47.148 INFO:teuthology.orchestra.run.smithi117.stdout:(9/150): ceph-selinux-19.1.0-1382.gde09dd01.el9 607 kB/s | 25 kB 00:00 2024-08-19T01:22:47.160 INFO:teuthology.orchestra.run.smithi167.stdout:(58/150): librabbitmq-0.11.0-7.el9.x86_64.rpm 903 kB/s | 45 kB 00:00 2024-08-19T01:22:47.253 INFO:teuthology.orchestra.run.smithi167.stdout:(59/150): libtool-2.4.6-46.el9.x86_64.rpm 6.2 MB/s | 581 kB 00:00 2024-08-19T01:22:47.286 INFO:teuthology.orchestra.run.smithi167.stdout:(60/150): librdkafka-1.6.1-102.el9.x86_64.rpm 4.6 MB/s | 662 kB 00:00 2024-08-19T01:22:47.312 INFO:teuthology.orchestra.run.smithi167.stdout:(61/150): libuuid-devel-2.37.4-18.el9.x86_64.rp 434 kB/s | 25 kB 00:00 2024-08-19T01:22:47.337 INFO:teuthology.orchestra.run.smithi167.stdout:(62/150): libzstd-devel-1.5.1-2.el9.x86_64.rpm 944 kB/s | 47 kB 00:00 2024-08-19T01:22:47.370 INFO:teuthology.orchestra.run.smithi167.stdout:(63/150): lttng-ust-2.12.0-6.el9.x86_64.rpm 4.9 MB/s | 292 kB 00:00 2024-08-19T01:22:47.421 INFO:teuthology.orchestra.run.smithi167.stdout:(64/150): openblas-0.3.21-2.el9.x86_64.rpm 707 kB/s | 35 kB 00:00 2024-08-19T01:22:47.446 INFO:teuthology.orchestra.run.smithi167.stdout:(65/150): lua-5.4.4-4.el9.x86_64.rpm 1.7 MB/s | 188 kB 00:00 2024-08-19T01:22:48.255 INFO:teuthology.orchestra.run.smithi167.stdout:(66/150): flexiblas-netlib-3.0.4-8.el9.x86_64.r 2.4 MB/s | 3.0 MB 00:01 2024-08-19T01:22:48.349 INFO:teuthology.orchestra.run.smithi117.stdout:(10/150): ceph-radosgw-19.1.0-1382.gde09dd01.el 6.2 MB/s | 11 MB 00:01 2024-08-19T01:22:48.415 INFO:teuthology.orchestra.run.smithi117.stdout:(11/150): cephfs-mirror-19.1.0-1382.gde09dd01.e 3.3 MB/s | 225 kB 00:00 2024-08-19T01:22:48.457 INFO:teuthology.orchestra.run.smithi117.stdout:(12/150): cephfs-shell-19.1.0-1382.gde09dd01.el 650 kB/s | 27 kB 00:00 2024-08-19T01:22:48.499 INFO:teuthology.orchestra.run.smithi117.stdout:(13/150): libcephfs-devel-19.1.0-1382.gde09dd01 766 kB/s | 32 kB 00:00 2024-08-19T01:22:48.663 INFO:teuthology.orchestra.run.smithi167.stdout:(67/150): openssl-devel-3.2.2-2.el9.x86_64.rpm 3.6 MB/s | 4.4 MB 00:01 2024-08-19T01:22:48.705 INFO:teuthology.orchestra.run.smithi167.stdout:(68/150): postgresql-13.14-1.el9.x86_64.rpm 3.5 MB/s | 1.6 MB 00:00 2024-08-19T01:22:48.755 INFO:teuthology.orchestra.run.smithi167.stdout:(69/150): postgresql-contrib-13.14-1.el9.x86_64 9.7 MB/s | 911 kB 00:00 2024-08-19T01:22:48.775 INFO:teuthology.orchestra.run.smithi117.stdout:(14/150): ceph-osd-19.1.0-1382.gde09dd01.el9.x8 5.5 MB/s | 17 MB 00:03 2024-08-19T01:22:48.800 INFO:teuthology.orchestra.run.smithi117.stdout:(15/150): libcephfs2-19.1.0-1382.gde09dd01.el9. 2.4 MB/s | 753 kB 00:00 2024-08-19T01:22:48.822 INFO:teuthology.orchestra.run.smithi167.stdout:(70/150): openblas-openmp-0.3.21-2.el9.x86_64.r 3.4 MB/s | 4.7 MB 00:01 2024-08-19T01:22:48.833 INFO:teuthology.orchestra.run.smithi117.stdout:(16/150): libcephsqlite-19.1.0-1382.gde09dd01.e 2.7 MB/s | 163 kB 00:00 2024-08-19T01:22:48.863 INFO:teuthology.orchestra.run.smithi117.stdout:(17/150): librados-devel-19.1.0-1382.gde09dd01. 2.1 MB/s | 127 kB 00:00 2024-08-19T01:22:48.881 INFO:teuthology.orchestra.run.smithi167.stdout:(71/150): postgresql-private-libs-13.14-1.el9.x 759 kB/s | 133 kB 00:00 2024-08-19T01:22:48.923 INFO:teuthology.orchestra.run.smithi167.stdout:(72/150): protobuf-3.14.0-13.el9.x86_64.rpm 10 MB/s | 1.0 MB 00:00 2024-08-19T01:22:48.925 INFO:teuthology.orchestra.run.smithi117.stdout:(18/150): libradosstriper1-19.1.0-1382.gde09dd0 5.4 MB/s | 507 kB 00:00 2024-08-19T01:22:48.976 INFO:teuthology.orchestra.run.smithi117.stdout:(19/150): python3-ceph-argparse-19.1.0-1382.gde 906 kB/s | 45 kB 00:00 2024-08-19T01:22:49.007 INFO:teuthology.orchestra.run.smithi167.stdout:(73/150): postgresql-server-13.14-1.el9.x86_64. 23 MB/s | 5.8 MB 00:00 2024-08-19T01:22:49.018 INFO:teuthology.orchestra.run.smithi117.stdout:(20/150): python3-ceph-common-19.1.0-1382.gde09 3.2 MB/s | 136 kB 00:00 2024-08-19T01:22:49.032 INFO:teuthology.orchestra.run.smithi167.stdout:(74/150): python3-attrs-20.3.0-7.el9.noarch.rpm 581 kB/s | 87 kB 00:00 2024-08-19T01:22:49.057 INFO:teuthology.orchestra.run.smithi167.stdout:(75/150): python3-colorama-0.4.6-3.el9.noarch.r 1.2 MB/s | 61 kB 00:00 2024-08-19T01:22:49.085 INFO:teuthology.orchestra.run.smithi117.stdout:(21/150): python3-cephfs-19.1.0-1382.gde09dd01. 2.8 MB/s | 188 kB 00:00 2024-08-19T01:22:49.116 INFO:teuthology.orchestra.run.smithi167.stdout:(76/150): python3-jinja2-2.11.3-6.el9.noarch.rp 4.2 MB/s | 248 kB 00:00 2024-08-19T01:22:49.168 INFO:teuthology.orchestra.run.smithi117.stdout:(22/150): python3-rados-19.1.0-1382.gde09dd01.e 4.3 MB/s | 369 kB 00:00 2024-08-19T01:22:49.191 INFO:teuthology.orchestra.run.smithi167.stdout:(77/150): python3-babel-2.9.1-2.el9.noarch.rpm 22 MB/s | 6.0 MB 00:00 2024-08-19T01:22:49.219 INFO:teuthology.orchestra.run.smithi167.stdout:(78/150): python3-jsonpatch-1.21-16.el9.noarch. 256 kB/s | 26 kB 00:00 2024-08-19T01:22:49.235 INFO:teuthology.orchestra.run.smithi117.stdout:(23/150): python3-rbd-19.1.0-1382.gde09dd01.el9 5.1 MB/s | 350 kB 00:00 2024-08-19T01:22:49.242 INFO:teuthology.orchestra.run.smithi167.stdout:(79/150): python3-devel-3.9.19-4.el9.x86_64.rpm 1.1 MB/s | 244 kB 00:00 2024-08-19T01:22:49.277 INFO:teuthology.orchestra.run.smithi117.stdout:(24/150): python3-rgw-19.1.0-1382.gde09dd01.el9 2.7 MB/s | 117 kB 00:00 2024-08-19T01:22:49.284 INFO:teuthology.orchestra.run.smithi167.stdout:(80/150): python3-jsonpointer-2.0-4.el9.noarch. 208 kB/s | 19 kB 00:00 2024-08-19T01:22:49.309 INFO:teuthology.orchestra.run.smithi167.stdout:(81/150): python3-markupsafe-1.1.1-12.el9.x86_6 518 kB/s | 35 kB 00:00 2024-08-19T01:22:49.327 INFO:teuthology.orchestra.run.smithi117.stdout:(25/150): rbd-fuse-19.1.0-1382.gde09dd01.el9.x8 1.7 MB/s | 85 kB 00:00 2024-08-19T01:22:49.334 INFO:teuthology.orchestra.run.smithi167.stdout:(82/150): python3-mako-1.1.4-6.el9.noarch.rpm 1.4 MB/s | 172 kB 00:00 2024-08-19T01:22:49.393 INFO:teuthology.orchestra.run.smithi167.stdout:(83/150): python3-oauthlib-3.1.1-5.el9.noarch.r 3.7 MB/s | 222 kB 00:00 2024-08-19T01:22:49.460 INFO:teuthology.orchestra.run.smithi167.stdout:(84/150): python3-numpy-1.20.1-5.el9.x86_64.rpm 29 MB/s | 5.1 MB 00:00 2024-08-19T01:22:49.485 INFO:teuthology.orchestra.run.smithi167.stdout:(85/150): python3-packaging-20.9-5.el9.noarch.r 840 kB/s | 77 kB 00:00 2024-08-19T01:22:49.510 INFO:teuthology.orchestra.run.smithi167.stdout:(86/150): python3-numpy-f2py-1.20.1-5.el9.x86_6 1.2 MB/s | 250 kB 00:00 2024-08-19T01:22:49.512 INFO:teuthology.orchestra.run.smithi083.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-08-19T01:22:49.513 INFO:teuthology.orchestra.run.smithi083.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-08-19T01:22:49.525 INFO:teuthology.orchestra.run.smithi083.stdout:Package NetworkManager-1:1.48.2-2.el9.x86_64 is already installed. 2024-08-19T01:22:49.525 INFO:teuthology.orchestra.run.smithi083.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-08-19T01:22:49.526 INFO:teuthology.orchestra.run.smithi083.stdout:Package util-linux-2.37.4-18.el9.x86_64 is already installed. 2024-08-19T01:22:49.527 INFO:teuthology.orchestra.run.smithi083.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-08-19T01:22:49.529 INFO:teuthology.orchestra.run.smithi083.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-08-19T01:22:49.531 INFO:teuthology.orchestra.run.smithi083.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-08-19T01:22:49.532 INFO:teuthology.orchestra.run.smithi083.stdout:Package xfsprogs-6.4.0-3.el9.x86_64 is already installed. 2024-08-19T01:22:49.535 INFO:teuthology.orchestra.run.smithi167.stdout:(87/150): python3-prettytable-0.7.2-27.el9.noar 561 kB/s | 42 kB 00:00 2024-08-19T01:22:49.561 INFO:teuthology.orchestra.run.smithi167.stdout:(88/150): python3-protobuf-3.14.0-13.el9.noarch 3.5 MB/s | 269 kB 00:00 2024-08-19T01:22:49.594 INFO:teuthology.orchestra.run.smithi167.stdout:(89/150): python3-pyasn1-modules-0.4.8-6.el9.no 4.6 MB/s | 279 kB 00:00 2024-08-19T01:22:49.595 INFO:teuthology.orchestra.run.smithi083.stdout:Dependencies resolved. 2024-08-19T01:22:49.595 INFO:teuthology.orchestra.run.smithi117.stdout:(26/150): librgw2-19.1.0-1382.gde09dd01.el9.x86 7.3 MB/s | 5.4 MB 00:00 2024-08-19T01:22:49.619 INFO:teuthology.orchestra.run.smithi083.stdout:======================================================================================= 2024-08-19T01:22:49.619 INFO:teuthology.orchestra.run.smithi083.stdout: Package Arch Version Repository Size 2024-08-19T01:22:49.619 INFO:teuthology.orchestra.run.smithi083.stdout:======================================================================================= 2024-08-19T01:22:49.619 INFO:teuthology.orchestra.run.smithi083.stdout:Installing: 2024-08-19T01:22:49.619 INFO:teuthology.orchestra.run.smithi083.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-08-19T01:22:49.619 INFO:teuthology.orchestra.run.smithi083.stdout: ceph x86_64 2:19.1.0-1382.gde09dd01.el9 ceph 6.6 k 2024-08-19T01:22:49.619 INFO:teuthology.orchestra.run.smithi083.stdout: ceph-base x86_64 2:19.1.0-1382.gde09dd01.el9 ceph 5.4 M 2024-08-19T01:22:49.619 INFO:teuthology.orchestra.run.smithi083.stdout: ceph-fuse x86_64 2:19.1.0-1382.gde09dd01.el9 ceph 896 k 2024-08-19T01:22:49.619 INFO:teuthology.orchestra.run.smithi083.stdout: ceph-immutable-object-cache x86_64 2:19.1.0-1382.gde09dd01.el9 ceph 145 k 2024-08-19T01:22:49.619 INFO:teuthology.orchestra.run.smithi083.stdout: ceph-mgr x86_64 2:19.1.0-1382.gde09dd01.el9 ceph 1.5 M 2024-08-19T01:22:49.619 INFO:teuthology.orchestra.run.smithi083.stdout: ceph-mgr-cephadm noarch 2:19.1.0-1382.gde09dd01.el9 ceph-noarch 146 k 2024-08-19T01:22:49.620 INFO:teuthology.orchestra.run.smithi083.stdout: ceph-mgr-dashboard noarch 2:19.1.0-1382.gde09dd01.el9 ceph-noarch 3.6 M 2024-08-19T01:22:49.620 INFO:teuthology.orchestra.run.smithi083.stdout: ceph-mgr-diskprediction-local noarch 2:19.1.0-1382.gde09dd01.el9 ceph-noarch 7.4 M 2024-08-19T01:22:49.620 INFO:teuthology.orchestra.run.smithi083.stdout: ceph-mgr-rook noarch 2:19.1.0-1382.gde09dd01.el9 ceph-noarch 49 k 2024-08-19T01:22:49.620 INFO:teuthology.orchestra.run.smithi083.stdout: ceph-radosgw x86_64 2:19.1.0-1382.gde09dd01.el9 ceph 11 M 2024-08-19T01:22:49.620 INFO:teuthology.orchestra.run.smithi083.stdout: ceph-test x86_64 2:19.1.0-1382.gde09dd01.el9 ceph 49 M 2024-08-19T01:22:49.620 INFO:teuthology.orchestra.run.smithi083.stdout: ceph-volume noarch 2:19.1.0-1382.gde09dd01.el9 ceph-noarch 285 k 2024-08-19T01:22:49.620 INFO:teuthology.orchestra.run.smithi083.stdout: cephadm noarch 2:19.1.0-1382.gde09dd01.el9 ceph-noarch 766 k 2024-08-19T01:22:49.620 INFO:teuthology.orchestra.run.smithi083.stdout: cephfs-mirror x86_64 2:19.1.0-1382.gde09dd01.el9 ceph 225 k 2024-08-19T01:22:49.620 INFO:teuthology.orchestra.run.smithi083.stdout: cephfs-shell x86_64 2:19.1.0-1382.gde09dd01.el9 ceph 27 k 2024-08-19T01:22:49.620 INFO:teuthology.orchestra.run.smithi083.stdout: cephfs-top noarch 2:19.1.0-1382.gde09dd01.el9 ceph-noarch 18 k 2024-08-19T01:22:49.620 INFO:teuthology.orchestra.run.smithi083.stdout: elfutils-libelf-devel x86_64 0.191-4.el9 appstream 22 k 2024-08-19T01:22:49.620 INFO:teuthology.orchestra.run.smithi083.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-08-19T01:22:49.620 INFO:teuthology.orchestra.run.smithi083.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-08-19T01:22:49.621 INFO:teuthology.orchestra.run.smithi083.stdout: libcephfs-devel x86_64 2:19.1.0-1382.gde09dd01.el9 ceph 32 k 2024-08-19T01:22:49.621 INFO:teuthology.orchestra.run.smithi083.stdout: libcephfs2 x86_64 2:19.1.0-1382.gde09dd01.el9 ceph 753 k 2024-08-19T01:22:49.621 INFO:teuthology.orchestra.run.smithi083.stdout: librados-devel x86_64 2:19.1.0-1382.gde09dd01.el9 ceph 127 k 2024-08-19T01:22:49.621 INFO:teuthology.orchestra.run.smithi083.stdout: libtool x86_64 2.4.6-46.el9 appstream 581 k 2024-08-19T01:22:49.621 INFO:teuthology.orchestra.run.smithi083.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-08-19T01:22:49.621 INFO:teuthology.orchestra.run.smithi083.stdout: openssl-devel x86_64 1:3.2.2-2.el9 appstream 4.4 M 2024-08-19T01:22:49.621 INFO:teuthology.orchestra.run.smithi083.stdout: postgresql x86_64 13.14-1.el9 appstream 1.6 M 2024-08-19T01:22:49.621 INFO:teuthology.orchestra.run.smithi083.stdout: postgresql-contrib x86_64 13.14-1.el9 appstream 911 k 2024-08-19T01:22:49.621 INFO:teuthology.orchestra.run.smithi083.stdout: postgresql-server x86_64 13.14-1.el9 appstream 5.8 M 2024-08-19T01:22:49.621 INFO:teuthology.orchestra.run.smithi083.stdout: python3-cephfs x86_64 2:19.1.0-1382.gde09dd01.el9 ceph 188 k 2024-08-19T01:22:49.621 INFO:teuthology.orchestra.run.smithi083.stdout: python3-rados x86_64 2:19.1.0-1382.gde09dd01.el9 ceph 369 k 2024-08-19T01:22:49.621 INFO:teuthology.orchestra.run.smithi083.stdout: python3-rbd x86_64 2:19.1.0-1382.gde09dd01.el9 ceph 350 k 2024-08-19T01:22:49.621 INFO:teuthology.orchestra.run.smithi083.stdout: python3-rgw x86_64 2:19.1.0-1382.gde09dd01.el9 ceph 117 k 2024-08-19T01:22:49.622 INFO:teuthology.orchestra.run.smithi083.stdout: rbd-fuse x86_64 2:19.1.0-1382.gde09dd01.el9 ceph 85 k 2024-08-19T01:22:49.622 INFO:teuthology.orchestra.run.smithi083.stdout: rbd-mirror x86_64 2:19.1.0-1382.gde09dd01.el9 ceph 3.1 M 2024-08-19T01:22:49.622 INFO:teuthology.orchestra.run.smithi083.stdout: rbd-nbd x86_64 2:19.1.0-1382.gde09dd01.el9 ceph 170 k 2024-08-19T01:22:49.622 INFO:teuthology.orchestra.run.smithi083.stdout: xfsprogs-devel x86_64 6.4.0-3.el9 appstream 96 k 2024-08-19T01:22:49.622 INFO:teuthology.orchestra.run.smithi083.stdout:Upgrading: 2024-08-19T01:22:49.622 INFO:teuthology.orchestra.run.smithi083.stdout: librados2 x86_64 2:19.1.0-1382.gde09dd01.el9 ceph 3.4 M 2024-08-19T01:22:49.622 INFO:teuthology.orchestra.run.smithi083.stdout: librbd1 x86_64 2:19.1.0-1382.gde09dd01.el9 ceph 3.2 M 2024-08-19T01:22:49.622 INFO:teuthology.orchestra.run.smithi083.stdout:Installing dependencies: 2024-08-19T01:22:49.622 INFO:teuthology.orchestra.run.smithi083.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-08-19T01:22:49.622 INFO:teuthology.orchestra.run.smithi083.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-08-19T01:22:49.622 INFO:teuthology.orchestra.run.smithi083.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-08-19T01:22:49.622 INFO:teuthology.orchestra.run.smithi083.stdout: ceph-common x86_64 2:19.1.0-1382.gde09dd01.el9 ceph 21 M 2024-08-19T01:22:49.623 INFO:teuthology.orchestra.run.smithi083.stdout: ceph-grafana-dashboards noarch 2:19.1.0-1382.gde09dd01.el9 ceph-noarch 32 k 2024-08-19T01:22:49.625 INFO:teuthology.orchestra.run.smithi083.stdout: ceph-mds x86_64 2:19.1.0-1382.gde09dd01.el9 ceph 2.4 M 2024-08-19T01:22:49.625 INFO:teuthology.orchestra.run.smithi083.stdout: ceph-mgr-modules-core noarch 2:19.1.0-1382.gde09dd01.el9 ceph-noarch 249 k 2024-08-19T01:22:49.625 INFO:teuthology.orchestra.run.smithi083.stdout: ceph-mon x86_64 2:19.1.0-1382.gde09dd01.el9 ceph 4.7 M 2024-08-19T01:22:49.625 INFO:teuthology.orchestra.run.smithi083.stdout: ceph-osd x86_64 2:19.1.0-1382.gde09dd01.el9 ceph 17 M 2024-08-19T01:22:49.626 INFO:teuthology.orchestra.run.smithi083.stdout: ceph-prometheus-alerts noarch 2:19.1.0-1382.gde09dd01.el9 ceph-noarch 16 k 2024-08-19T01:22:49.626 INFO:teuthology.orchestra.run.smithi083.stdout: ceph-selinux x86_64 2:19.1.0-1382.gde09dd01.el9 ceph 25 k 2024-08-19T01:22:49.626 INFO:teuthology.orchestra.run.smithi083.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-08-19T01:22:49.626 INFO:teuthology.orchestra.run.smithi083.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-08-19T01:22:49.626 INFO:teuthology.orchestra.run.smithi083.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-08-19T01:22:49.626 INFO:teuthology.orchestra.run.smithi083.stdout: fuse x86_64 2.9.9-16.el9 baseos 80 k 2024-08-19T01:22:49.626 INFO:teuthology.orchestra.run.smithi083.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-08-19T01:22:49.626 INFO:teuthology.orchestra.run.smithi083.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-08-19T01:22:49.627 INFO:teuthology.orchestra.run.smithi083.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-08-19T01:22:49.627 INFO:teuthology.orchestra.run.smithi083.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-08-19T01:22:49.627 INFO:teuthology.orchestra.run.smithi083.stdout: libcephsqlite x86_64 2:19.1.0-1382.gde09dd01.el9 ceph 163 k 2024-08-19T01:22:49.627 INFO:teuthology.orchestra.run.smithi083.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-08-19T01:22:49.627 INFO:teuthology.orchestra.run.smithi083.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-08-19T01:22:49.627 INFO:teuthology.orchestra.run.smithi083.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-08-19T01:22:49.627 INFO:teuthology.orchestra.run.smithi083.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-08-19T01:22:49.627 INFO:teuthology.orchestra.run.smithi083.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-08-19T01:22:49.627 INFO:teuthology.orchestra.run.smithi083.stdout: libradosstriper1 x86_64 2:19.1.0-1382.gde09dd01.el9 ceph 507 k 2024-08-19T01:22:49.627 INFO:teuthology.orchestra.run.smithi083.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-08-19T01:22:49.627 INFO:teuthology.orchestra.run.smithi083.stdout: librgw2 x86_64 2:19.1.0-1382.gde09dd01.el9 ceph 5.4 M 2024-08-19T01:22:49.627 INFO:teuthology.orchestra.run.smithi083.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-08-19T01:22:49.627 INFO:teuthology.orchestra.run.smithi083.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-08-19T01:22:49.627 INFO:teuthology.orchestra.run.smithi083.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-08-19T01:22:49.628 INFO:teuthology.orchestra.run.smithi083.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-08-19T01:22:49.628 INFO:teuthology.orchestra.run.smithi083.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-08-19T01:22:49.628 INFO:teuthology.orchestra.run.smithi083.stdout: luarocks noarch 3.9.2-5.el9 epel 151 k 2024-08-19T01:22:49.628 INFO:teuthology.orchestra.run.smithi083.stdout: openblas x86_64 0.3.21-2.el9 appstream 35 k 2024-08-19T01:22:49.628 INFO:teuthology.orchestra.run.smithi083.stdout: openblas-openmp x86_64 0.3.21-2.el9 appstream 4.7 M 2024-08-19T01:22:49.628 INFO:teuthology.orchestra.run.smithi083.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-08-19T01:22:49.628 INFO:teuthology.orchestra.run.smithi083.stdout: postgresql-private-libs x86_64 13.14-1.el9 appstream 133 k 2024-08-19T01:22:49.628 INFO:teuthology.orchestra.run.smithi083.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-08-19T01:22:49.628 INFO:teuthology.orchestra.run.smithi083.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-08-19T01:22:49.628 INFO:teuthology.orchestra.run.smithi083.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-08-19T01:22:49.628 INFO:teuthology.orchestra.run.smithi083.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-08-19T01:22:49.628 INFO:teuthology.orchestra.run.smithi083.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-08-19T01:22:49.628 INFO:teuthology.orchestra.run.smithi083.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-08-19T01:22:49.628 INFO:teuthology.orchestra.run.smithi083.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-08-19T01:22:49.629 INFO:teuthology.orchestra.run.smithi083.stdout: python3-ceph-argparse x86_64 2:19.1.0-1382.gde09dd01.el9 ceph 45 k 2024-08-19T01:22:49.629 INFO:teuthology.orchestra.run.smithi083.stdout: python3-ceph-common x86_64 2:19.1.0-1382.gde09dd01.el9 ceph 136 k 2024-08-19T01:22:49.629 INFO:teuthology.orchestra.run.smithi083.stdout: python3-certifi noarch 2023.05.07-4.el9 epel 14 k 2024-08-19T01:22:49.629 INFO:teuthology.orchestra.run.smithi083.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-08-19T01:22:49.629 INFO:teuthology.orchestra.run.smithi083.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-08-19T01:22:49.629 INFO:teuthology.orchestra.run.smithi083.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-08-19T01:22:49.629 INFO:teuthology.orchestra.run.smithi083.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-08-19T01:22:49.629 INFO:teuthology.orchestra.run.smithi083.stdout: python3-colorama noarch 0.4.6-3.el9 appstream 61 k 2024-08-19T01:22:49.629 INFO:teuthology.orchestra.run.smithi083.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-08-19T01:22:49.629 INFO:teuthology.orchestra.run.smithi083.stdout: python3-devel x86_64 3.9.19-4.el9 appstream 244 k 2024-08-19T01:22:49.629 INFO:teuthology.orchestra.run.smithi083.stdout: python3-google-auth noarch 1:2.33.0-1.el9 epel 220 k 2024-08-19T01:22:49.629 INFO:teuthology.orchestra.run.smithi083.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-08-19T01:22:49.630 INFO:teuthology.orchestra.run.smithi083.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-08-19T01:22:49.630 INFO:teuthology.orchestra.run.smithi083.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-08-19T01:22:49.630 INFO:teuthology.orchestra.run.smithi083.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-08-19T01:22:49.630 INFO:teuthology.orchestra.run.smithi083.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-08-19T01:22:49.630 INFO:teuthology.orchestra.run.smithi083.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-08-19T01:22:49.630 INFO:teuthology.orchestra.run.smithi083.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-08-19T01:22:49.630 INFO:teuthology.orchestra.run.smithi083.stdout: python3-jinja2 noarch 2.11.3-6.el9 appstream 248 k 2024-08-19T01:22:49.630 INFO:teuthology.orchestra.run.smithi083.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-08-19T01:22:49.630 INFO:teuthology.orchestra.run.smithi083.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-08-19T01:22:49.630 INFO:teuthology.orchestra.run.smithi083.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-08-19T01:22:49.630 INFO:teuthology.orchestra.run.smithi083.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-08-19T01:22:49.630 INFO:teuthology.orchestra.run.smithi083.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-08-19T01:22:49.630 INFO:teuthology.orchestra.run.smithi083.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-08-19T01:22:49.631 INFO:teuthology.orchestra.run.smithi083.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-08-19T01:22:49.631 INFO:teuthology.orchestra.run.smithi083.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-08-19T01:22:49.631 INFO:teuthology.orchestra.run.smithi083.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-08-19T01:22:49.631 INFO:teuthology.orchestra.run.smithi083.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-08-19T01:22:49.631 INFO:teuthology.orchestra.run.smithi083.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-08-19T01:22:49.631 INFO:teuthology.orchestra.run.smithi083.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-08-19T01:22:49.631 INFO:teuthology.orchestra.run.smithi083.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-08-19T01:22:49.631 INFO:teuthology.orchestra.run.smithi083.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-08-19T01:22:49.631 INFO:teuthology.orchestra.run.smithi083.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-08-19T01:22:49.631 INFO:teuthology.orchestra.run.smithi083.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-08-19T01:22:49.631 INFO:teuthology.orchestra.run.smithi083.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-08-19T01:22:49.631 INFO:teuthology.orchestra.run.smithi083.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-08-19T01:22:49.631 INFO:teuthology.orchestra.run.smithi083.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-08-19T01:22:49.632 INFO:teuthology.orchestra.run.smithi083.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-08-19T01:22:49.632 INFO:teuthology.orchestra.run.smithi083.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-08-19T01:22:49.632 INFO:teuthology.orchestra.run.smithi083.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-08-19T01:22:49.632 INFO:teuthology.orchestra.run.smithi083.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-08-19T01:22:49.632 INFO:teuthology.orchestra.run.smithi083.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-08-19T01:22:49.632 INFO:teuthology.orchestra.run.smithi083.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-08-19T01:22:49.632 INFO:teuthology.orchestra.run.smithi083.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-08-19T01:22:49.632 INFO:teuthology.orchestra.run.smithi083.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-08-19T01:22:49.632 INFO:teuthology.orchestra.run.smithi083.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-08-19T01:22:49.632 INFO:teuthology.orchestra.run.smithi083.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-08-19T01:22:49.632 INFO:teuthology.orchestra.run.smithi083.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-08-19T01:22:49.632 INFO:teuthology.orchestra.run.smithi083.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-08-19T01:22:49.632 INFO:teuthology.orchestra.run.smithi083.stdout: python3-typing-extensions noarch 4.12.2-2.el9 epel 74 k 2024-08-19T01:22:49.632 INFO:teuthology.orchestra.run.smithi083.stdout: python3-urllib3 noarch 1.26.5-6.el9 baseos 215 k 2024-08-19T01:22:49.633 INFO:teuthology.orchestra.run.smithi083.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-08-19T01:22:49.633 INFO:teuthology.orchestra.run.smithi083.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-08-19T01:22:49.633 INFO:teuthology.orchestra.run.smithi083.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-08-19T01:22:49.633 INFO:teuthology.orchestra.run.smithi083.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-08-19T01:22:49.633 INFO:teuthology.orchestra.run.smithi083.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-08-19T01:22:49.633 INFO:teuthology.orchestra.run.smithi083.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-08-19T01:22:49.633 INFO:teuthology.orchestra.run.smithi083.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-08-19T01:22:49.633 INFO:teuthology.orchestra.run.smithi083.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-08-19T01:22:49.633 INFO:teuthology.orchestra.run.smithi083.stdout: socat x86_64 1.7.4.1-6.el9 appstream 304 k 2024-08-19T01:22:49.633 INFO:teuthology.orchestra.run.smithi083.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-08-19T01:22:49.633 INFO:teuthology.orchestra.run.smithi083.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-08-19T01:22:49.633 INFO:teuthology.orchestra.run.smithi083.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-08-19T01:22:49.633 INFO:teuthology.orchestra.run.smithi083.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-08-19T01:22:49.634 INFO:teuthology.orchestra.run.smithi083.stdout:Installing weak dependencies: 2024-08-19T01:22:49.634 INFO:teuthology.orchestra.run.smithi083.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-08-19T01:22:49.634 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-08-19T01:22:49.634 INFO:teuthology.orchestra.run.smithi083.stdout:Transaction Summary 2024-08-19T01:22:49.634 INFO:teuthology.orchestra.run.smithi083.stdout:======================================================================================= 2024-08-19T01:22:49.634 INFO:teuthology.orchestra.run.smithi083.stdout:Install 148 Packages 2024-08-19T01:22:49.634 INFO:teuthology.orchestra.run.smithi083.stdout:Upgrade 2 Packages 2024-08-19T01:22:49.634 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-08-19T01:22:49.635 INFO:teuthology.orchestra.run.smithi167.stdout:(90/150): python3-pytz-2021.1-5.el9.noarch.rpm 872 kB/s | 51 kB 00:00 2024-08-19T01:22:49.635 INFO:teuthology.orchestra.run.smithi083.stdout:Total download size: 218 M 2024-08-19T01:22:49.635 INFO:teuthology.orchestra.run.smithi083.stdout:Downloading Packages: 2024-08-19T01:22:49.645 INFO:teuthology.orchestra.run.smithi117.stdout:(27/150): rbd-nbd-19.1.0-1382.gde09dd01.el9.x86 3.4 MB/s | 170 kB 00:00 2024-08-19T01:22:49.645 INFO:teuthology.orchestra.run.smithi167.stdout:(91/150): python3-pyasn1-0.4.8-6.el9.noarch.rpm 1.2 MB/s | 159 kB 00:00 2024-08-19T01:22:49.670 INFO:teuthology.orchestra.run.smithi167.stdout:(92/150): python3-requests-oauthlib-1.3.0-12.el 711 kB/s | 54 kB 00:00 2024-08-19T01:22:49.686 INFO:teuthology.orchestra.run.smithi117.stdout:(28/150): ceph-grafana-dashboards-19.1.0-1382.g 766 kB/s | 32 kB 00:00 2024-08-19T01:22:49.729 INFO:teuthology.orchestra.run.smithi167.stdout:(93/150): python3-toml-0.10.2-6.el9.noarch.rpm 499 kB/s | 42 kB 00:00 2024-08-19T01:22:49.737 INFO:teuthology.orchestra.run.smithi117.stdout:(29/150): ceph-mgr-cephadm-19.1.0-1382.gde09dd0 2.9 MB/s | 146 kB 00:00 2024-08-19T01:22:49.754 INFO:teuthology.orchestra.run.smithi167.stdout:(94/150): python3-wcwidth-0.2.5-8.el9.noarch.rp 529 kB/s | 44 kB 00:00 2024-08-19T01:22:49.787 INFO:teuthology.orchestra.run.smithi117.stdout:(30/150): rbd-mirror-19.1.0-1382.gde09dd01.el9. 6.7 MB/s | 3.1 MB 00:00 2024-08-19T01:22:49.813 INFO:teuthology.orchestra.run.smithi167.stdout:(95/150): qatlib-service-24.02.0-1.el9.x86_64.r 615 kB/s | 36 kB 00:00 2024-08-19T01:22:49.871 INFO:teuthology.orchestra.run.smithi167.stdout:(96/150): qatzip-libs-1.2.0-1.el9.x86_64.rpm 814 kB/s | 47 kB 00:00 2024-08-19T01:22:49.922 INFO:teuthology.orchestra.run.smithi167.stdout:(97/150): qatlib-24.02.0-1.el9.x86_64.rpm 1.1 MB/s | 221 kB 00:00 2024-08-19T01:22:49.947 INFO:teuthology.orchestra.run.smithi167.stdout:(98/150): socat-1.7.4.1-6.el9.x86_64.rpm 4.0 MB/s | 304 kB 00:00 2024-08-19T01:22:50.106 INFO:teuthology.orchestra.run.smithi167.stdout:(99/150): python3-scipy-1.6.2-8.el9.x86_64.rpm 33 MB/s | 16 MB 00:00 2024-08-19T01:22:50.131 INFO:teuthology.orchestra.run.smithi167.stdout:(100/150): uuid-1.6.2-55.el9.x86_64.rpm 277 kB/s | 58 kB 00:00 2024-08-19T01:22:50.156 INFO:teuthology.orchestra.run.smithi167.stdout:(101/150): xfsprogs-devel-6.4.0-3.el9.x86_64.rp 458 kB/s | 96 kB 00:00 2024-08-19T01:22:50.182 INFO:teuthology.orchestra.run.smithi167.stdout:(102/150): xmlstarlet-1.6.1-20.el9.x86_64.rpm 847 kB/s | 64 kB 00:00 2024-08-19T01:22:50.196 INFO:teuthology.orchestra.run.smithi117.stdout:(31/150): ceph-mgr-dashboard-19.1.0-1382.gde09d 7.8 MB/s | 3.6 MB 00:00 2024-08-19T01:22:50.207 INFO:teuthology.orchestra.run.smithi167.stdout:(103/150): zlib-devel-1.2.11-41.el9.x86_64.rpm 594 kB/s | 45 kB 00:00 2024-08-19T01:22:50.257 INFO:teuthology.orchestra.run.smithi167.stdout:(104/150): abseil-cpp-20211102.0-3.el9.x86_64.r 11 MB/s | 548 kB 00:00 2024-08-19T01:22:50.263 INFO:teuthology.orchestra.run.smithi117.stdout:(32/150): ceph-mgr-modules-core-19.1.0-1382.gde 3.7 MB/s | 249 kB 00:00 2024-08-19T01:22:50.282 INFO:teuthology.orchestra.run.smithi167.stdout:(105/150): lua-devel-5.4.4-4.el9.x86_64.rpm 177 kB/s | 22 kB 00:00 2024-08-19T01:22:50.313 INFO:teuthology.orchestra.run.smithi117.stdout:(33/150): ceph-mgr-rook-19.1.0-1382.gde09dd01.e 988 kB/s | 49 kB 00:00 2024-08-19T01:22:50.324 INFO:teuthology.orchestra.run.smithi167.stdout:(106/150): gperftools-libs-2.9.1-2.el9.x86_64.r 4.5 MB/s | 309 kB 00:00 2024-08-19T01:22:50.350 INFO:teuthology.orchestra.run.smithi167.stdout:(107/150): grpc-data-1.46.7-10.el9.noarch.rpm 291 kB/s | 19 kB 00:00 2024-08-19T01:22:50.363 INFO:teuthology.orchestra.run.smithi117.stdout:(34/150): ceph-prometheus-alerts-19.1.0-1382.gd 329 kB/s | 16 kB 00:00 2024-08-19T01:22:50.375 INFO:teuthology.orchestra.run.smithi167.stdout:(108/150): libarrow-doc-9.0.0-11.el9.noarch.rpm 1.0 MB/s | 26 kB 00:00 2024-08-19T01:22:50.401 INFO:teuthology.orchestra.run.smithi167.stdout:(109/150): liboath-2.6.7-2.el9.x86_64.rpm 1.9 MB/s | 49 kB 00:00 2024-08-19T01:22:50.426 INFO:teuthology.orchestra.run.smithi167.stdout:(110/150): protobuf-compiler-3.14.0-13.el9.x86_ 3.5 MB/s | 863 kB 00:00 2024-08-19T01:22:50.430 INFO:teuthology.orchestra.run.smithi117.stdout:(35/150): ceph-volume-19.1.0-1382.gde09dd01.el9 4.2 MB/s | 285 kB 00:00 2024-08-19T01:22:50.435 INFO:teuthology.orchestra.run.smithi083.stdout:(1/150): ceph-19.1.0-1382.gde09dd01.el9.x86_64. 44 kB/s | 6.6 kB 00:00 2024-08-19T01:22:50.459 INFO:teuthology.orchestra.run.smithi167.stdout:(111/150): libunwind-1.6.2-1.el9.x86_64.rpm 1.1 MB/s | 67 kB 00:00 2024-08-19T01:22:50.493 INFO:teuthology.orchestra.run.smithi167.stdout:(112/150): luarocks-3.9.2-5.el9.noarch.rpm 2.2 MB/s | 151 kB 00:00 2024-08-19T01:22:50.543 INFO:teuthology.orchestra.run.smithi167.stdout:(113/150): parquet-libs-9.0.0-11.el9.x86_64.rpm 10 MB/s | 839 kB 00:00 2024-08-19T01:22:50.569 INFO:teuthology.orchestra.run.smithi167.stdout:(114/150): python3-bcrypt-3.2.2-1.el9.x86_64.rp 1.7 MB/s | 43 kB 00:00 2024-08-19T01:22:50.572 INFO:teuthology.orchestra.run.smithi117.stdout:(36/150): cephadm-19.1.0-1382.gde09dd01.el9.noa 5.3 MB/s | 766 kB 00:00 2024-08-19T01:22:50.594 INFO:teuthology.orchestra.run.smithi167.stdout:(115/150): python3-cachetools-4.2.4-1.el9.noarc 1.3 MB/s | 32 kB 00:00 2024-08-19T01:22:50.614 INFO:teuthology.orchestra.run.smithi117.stdout:(37/150): cephfs-top-19.1.0-1382.gde09dd01.el9. 440 kB/s | 18 kB 00:00 2024-08-19T01:22:50.619 INFO:teuthology.orchestra.run.smithi167.stdout:(116/150): python3-asyncssh-2.13.2-5.el9.noarch 4.3 MB/s | 548 kB 00:00 2024-08-19T01:22:50.644 INFO:teuthology.orchestra.run.smithi167.stdout:(117/150): python3-certifi-2023.05.07-4.el9.noa 281 kB/s | 14 kB 00:00 2024-08-19T01:22:50.670 INFO:teuthology.orchestra.run.smithi167.stdout:(118/150): python3-cheroot-8.6.0-4.el9.noarch.r 3.3 MB/s | 172 kB 00:00 2024-08-19T01:22:50.715 INFO:teuthology.orchestra.run.smithi117.stdout:(38/150): ceph-mgr-diskprediction-local-19.1.0- 8.0 MB/s | 7.4 MB 00:00 2024-08-19T01:22:50.737 INFO:teuthology.orchestra.run.smithi167.stdout:(119/150): libarrow-9.0.0-11.el9.x86_64.rpm 11 MB/s | 4.4 MB 00:00 2024-08-19T01:22:50.743 INFO:teuthology.orchestra.run.smithi083.stdout:(2/150): ceph-fuse-19.1.0-1382.gde09dd01.el9.x8 2.8 MB/s | 896 kB 00:00 2024-08-19T01:22:50.762 INFO:teuthology.orchestra.run.smithi167.stdout:(120/150): python3-cherrypy-18.6.1-2.el9.noarch 3.0 MB/s | 358 kB 00:00 2024-08-19T01:22:50.787 INFO:teuthology.orchestra.run.smithi167.stdout:(121/150): python3-cmd2-2.4.2-2.el9.noarch.rpm 2.5 MB/s | 303 kB 00:00 2024-08-19T01:22:50.818 INFO:teuthology.orchestra.run.smithi083.stdout:(3/150): ceph-immutable-object-cache-19.1.0-138 1.9 MB/s | 145 kB 00:00 2024-08-19T01:22:50.821 INFO:teuthology.orchestra.run.smithi167.stdout:(122/150): python3-google-auth-2.33.0-1.el9.noa 2.6 MB/s | 220 kB 00:00 2024-08-19T01:22:50.837 INFO:teuthology.orchestra.run.smithi167.stdout:(123/150): python3-jaraco-8.2.1-3.el9.noarch.rp 647 kB/s | 11 kB 00:00 2024-08-19T01:22:50.863 INFO:teuthology.orchestra.run.smithi167.stdout:(124/150): python3-grpcio-tools-1.46.7-10.el9.x 1.9 MB/s | 144 kB 00:00 2024-08-19T01:22:50.880 INFO:teuthology.orchestra.run.smithi167.stdout:(125/150): python3-jaraco-classes-3.2.1-5.el9.n 421 kB/s | 18 kB 00:00 2024-08-19T01:22:50.897 INFO:teuthology.orchestra.run.smithi167.stdout:(126/150): python3-jaraco-collections-3.0.0-8.e 693 kB/s | 23 kB 00:00 2024-08-19T01:22:50.914 INFO:teuthology.orchestra.run.smithi167.stdout:(127/150): python3-jaraco-functools-3.5.0-2.el9 582 kB/s | 19 kB 00:00 2024-08-19T01:22:50.931 INFO:teuthology.orchestra.run.smithi167.stdout:(128/150): python3-jaraco-text-3.2.0-6.el9.noar 586 kB/s | 20 kB 00:00 2024-08-19T01:22:50.956 INFO:teuthology.orchestra.run.smithi167.stdout:(129/150): python3-logutils-0.3.5-21.el9.noarch 1.8 MB/s | 46 kB 00:00 2024-08-19T01:22:50.981 INFO:teuthology.orchestra.run.smithi167.stdout:(130/150): python3-more-itertools-8.12.0-2.el9. 3.1 MB/s | 79 kB 00:00 2024-08-19T01:22:51.007 INFO:teuthology.orchestra.run.smithi167.stdout:(131/150): python3-natsort-7.1.1-5.el9.noarch.r 2.3 MB/s | 58 kB 00:00 2024-08-19T01:22:51.056 INFO:teuthology.orchestra.run.smithi117.stdout:(39/150): fuse-2.9.9-16.el9.x86_64.rpm 181 kB/s | 80 kB 00:00 2024-08-19T01:22:51.057 INFO:teuthology.orchestra.run.smithi167.stdout:(132/150): python3-kubernetes-26.1.0-2.el9.noar 7.2 MB/s | 1.0 MB 00:00 2024-08-19T01:22:51.082 INFO:teuthology.orchestra.run.smithi167.stdout:(133/150): python3-portend-3.1.0-2.el9.noarch.r 651 kB/s | 16 kB 00:00 2024-08-19T01:22:51.116 INFO:teuthology.orchestra.run.smithi167.stdout:(134/150): python3-pecan-1.4.2-3.el9.noarch.rpm 2.4 MB/s | 272 kB 00:00 2024-08-19T01:22:51.141 INFO:teuthology.orchestra.run.smithi167.stdout:(135/150): python3-pyOpenSSL-21.0.0-1.el9.noarc 1.5 MB/s | 90 kB 00:00 2024-08-19T01:22:51.167 INFO:teuthology.orchestra.run.smithi167.stdout:(136/150): python3-pyperclip-1.8.2-2.el9.noarch 562 kB/s | 28 kB 00:00 2024-08-19T01:22:51.192 INFO:teuthology.orchestra.run.smithi167.stdout:(137/150): python3-repoze-lru-0.7-16.el9.noarch 613 kB/s | 31 kB 00:00 2024-08-19T01:22:51.217 INFO:teuthology.orchestra.run.smithi167.stdout:(138/150): python3-rsa-4.9-2.el9.noarch.rpm 2.3 MB/s | 59 kB 00:00 2024-08-19T01:22:51.219 INFO:teuthology.orchestra.run.smithi083.stdout:(4/150): ceph-mds-19.1.0-1382.gde09dd01.el9.x86 6.0 MB/s | 2.4 MB 00:00 2024-08-19T01:22:51.240 INFO:teuthology.orchestra.run.smithi117.stdout:(40/150): libquadmath-11.4.1-3.el9.x86_64.rpm 1.0 MB/s | 195 kB 00:00 2024-08-19T01:22:51.242 INFO:teuthology.orchestra.run.smithi167.stdout:(139/150): python3-tempora-5.0.0-2.el9.noarch.r 1.4 MB/s | 36 kB 00:00 2024-08-19T01:22:51.276 INFO:teuthology.orchestra.run.smithi167.stdout:(140/150): python3-routes-2.5.1-5.el9.noarch.rp 1.7 MB/s | 188 kB 00:00 2024-08-19T01:22:51.301 INFO:teuthology.orchestra.run.smithi167.stdout:(141/150): python3-typing-extensions-4.12.2-2.e 1.2 MB/s | 74 kB 00:00 2024-08-19T01:22:51.327 INFO:teuthology.orchestra.run.smithi083.stdout:(5/150): ceph-base-19.1.0-1382.gde09dd01.el9.x8 5.2 MB/s | 5.4 MB 00:01 2024-08-19T01:22:51.348 INFO:teuthology.orchestra.run.smithi117.stdout:(41/150): python3-cffi-1.14.5-5.el9.x86_64.rpm 2.3 MB/s | 253 kB 00:00 2024-08-19T01:22:51.352 INFO:teuthology.orchestra.run.smithi167.stdout:(142/150): python3-grpcio-1.46.7-10.el9.x86_64. 3.5 MB/s | 2.0 MB 00:00 2024-08-19T01:22:51.377 INFO:teuthology.orchestra.run.smithi167.stdout:(143/150): python3-websocket-client-1.2.3-2.el9 1.2 MB/s | 90 kB 00:00 2024-08-19T01:22:51.382 INFO:teuthology.orchestra.run.smithi117.stdout:(42/150): libgfortran-11.4.1-3.el9.x86_64.rpm 1.2 MB/s | 803 kB 00:00 2024-08-19T01:22:51.402 INFO:teuthology.orchestra.run.smithi167.stdout:(144/150): python3-webob-1.8.7-6.el9.noarch.rpm 1.8 MB/s | 230 kB 00:00 2024-08-19T01:22:51.428 INFO:teuthology.orchestra.run.smithi167.stdout:(145/150): python3-zc-lockfile-2.0-10.el9.noarc 398 kB/s | 20 kB 00:00 2024-08-19T01:22:51.444 INFO:teuthology.orchestra.run.smithi083.stdout:(6/150): ceph-mgr-19.1.0-1382.gde09dd01.el9.x86 6.8 MB/s | 1.5 MB 00:00 2024-08-19T01:22:51.453 INFO:teuthology.orchestra.run.smithi167.stdout:(146/150): re2-20211101-3.el9.x86_64.rpm 3.7 MB/s | 192 kB 00:00 2024-08-19T01:22:51.478 INFO:teuthology.orchestra.run.smithi167.stdout:(147/150): python3-werkzeug-2.0.3-3.el9.1.noarc 3.3 MB/s | 427 kB 00:00 2024-08-19T01:22:51.499 INFO:teuthology.orchestra.run.smithi117.stdout:(43/150): python3-ply-3.11-14.el9.noarch.rpm 915 kB/s | 106 kB 00:00 2024-08-19T01:22:51.529 INFO:teuthology.orchestra.run.smithi167.stdout:(148/150): thrift-0.15.0-2.el9.x86_64.rpm 16 MB/s | 1.6 MB 00:00 2024-08-19T01:22:51.549 INFO:teuthology.orchestra.run.smithi117.stdout:(44/150): python3-cryptography-36.0.1-4.el9.x86 6.1 MB/s | 1.2 MB 00:00 2024-08-19T01:22:51.753 INFO:teuthology.orchestra.run.smithi083.stdout:(7/150): ceph-mon-19.1.0-1382.gde09dd01.el9.x86 11 MB/s | 4.7 MB 00:00 2024-08-19T01:22:51.779 INFO:teuthology.orchestra.run.smithi167.stdout:(149/150): librados2-19.1.0-1382.gde09dd01.el9. 11 MB/s | 3.4 MB 00:00 2024-08-19T01:22:52.025 INFO:teuthology.orchestra.run.smithi117.stdout:(45/150): ceph-test-19.1.0-1382.gde09dd01.el9.x 10 MB/s | 49 MB 00:04 2024-08-19T01:22:52.050 INFO:teuthology.orchestra.run.smithi117.stdout:(46/150): python3-requests-2.25.1-8.el9.noarch. 249 kB/s | 125 kB 00:00 2024-08-19T01:22:52.075 INFO:teuthology.orchestra.run.smithi117.stdout:(47/150): python3-pycparser-2.20-6.el9.noarch.r 235 kB/s | 135 kB 00:00 2024-08-19T01:22:52.311 INFO:teuthology.orchestra.run.smithi083.stdout:(8/150): ceph-common-19.1.0-1382.gde09dd01.el9. 10 MB/s | 21 MB 00:02 2024-08-19T01:22:52.361 INFO:teuthology.orchestra.run.smithi083.stdout:(9/150): ceph-selinux-19.1.0-1382.gde09dd01.el9 507 kB/s | 25 kB 00:00 2024-08-19T01:22:52.479 INFO:teuthology.orchestra.run.smithi167.stdout:(150/150): librbd1-19.1.0-1382.gde09dd01.el9.x8 3.2 MB/s | 3.2 MB 00:01 2024-08-19T01:22:52.486 INFO:teuthology.orchestra.run.smithi167.stdout:-------------------------------------------------------------------------------- 2024-08-19T01:22:52.486 INFO:teuthology.orchestra.run.smithi167.stdout:Total 21 MB/s | 218 MB 00:10 2024-08-19T01:22:52.503 INFO:teuthology.orchestra.run.smithi083.stdout:(10/150): ceph-radosgw-19.1.0-1382.gde09dd01.el 14 MB/s | 11 MB 00:00 2024-08-19T01:22:52.525 INFO:teuthology.orchestra.run.smithi117.stdout:(48/150): python3-urllib3-1.26.5-6.el9.noarch.r 429 kB/s | 215 kB 00:00 2024-08-19T01:22:52.562 INFO:teuthology.orchestra.run.smithi083.stdout:(11/150): cephfs-mirror-19.1.0-1382.gde09dd01.e 3.8 MB/s | 225 kB 00:00 2024-08-19T01:22:52.746 INFO:teuthology.orchestra.run.smithi083.stdout:(12/150): ceph-osd-19.1.0-1382.gde09dd01.el9.x8 13 MB/s | 17 MB 00:01 2024-08-19T01:22:52.763 INFO:teuthology.orchestra.run.smithi083.stdout:(13/150): cephfs-shell-19.1.0-1382.gde09dd01.el 135 kB/s | 27 kB 00:00 2024-08-19T01:22:52.805 INFO:teuthology.orchestra.run.smithi083.stdout:(14/150): libcephfs-devel-19.1.0-1382.gde09dd01 545 kB/s | 32 kB 00:00 2024-08-19T01:22:52.838 INFO:teuthology.orchestra.run.smithi083.stdout:(15/150): libcephfs2-19.1.0-1382.gde09dd01.el9. 9.7 MB/s | 753 kB 00:00 2024-08-19T01:22:52.864 INFO:teuthology.orchestra.run.smithi083.stdout:(16/150): libcephsqlite-19.1.0-1382.gde09dd01.e 2.7 MB/s | 163 kB 00:00 2024-08-19T01:22:52.889 INFO:teuthology.orchestra.run.smithi083.stdout:(17/150): librados-devel-19.1.0-1382.gde09dd01. 2.5 MB/s | 127 kB 00:00 2024-08-19T01:22:52.923 INFO:teuthology.orchestra.run.smithi083.stdout:(18/150): libradosstriper1-19.1.0-1382.gde09dd0 8.5 MB/s | 507 kB 00:00 2024-08-19T01:22:52.950 INFO:teuthology.orchestra.run.smithi117.stdout:(49/150): automake-1.16.2-8.el9.noarch.rpm 765 kB/s | 689 kB 00:00 2024-08-19T01:22:52.981 INFO:teuthology.orchestra.run.smithi083.stdout:(19/150): python3-ceph-argparse-19.1.0-1382.gde 776 kB/s | 45 kB 00:00 2024-08-19T01:22:52.984 INFO:teuthology.orchestra.run.smithi117.stdout:(50/150): bison-3.7.4-5.el9.x86_64.rpm 1.0 MB/s | 943 kB 00:00 2024-08-19T01:22:53.048 INFO:teuthology.orchestra.run.smithi083.stdout:(20/150): python3-ceph-common-19.1.0-1382.gde09 2.0 MB/s | 136 kB 00:00 2024-08-19T01:22:53.140 INFO:teuthology.orchestra.run.smithi083.stdout:(21/150): librgw2-19.1.0-1382.gde09dd01.el9.x86 21 MB/s | 5.4 MB 00:00 2024-08-19T01:22:53.190 INFO:teuthology.orchestra.run.smithi083.stdout:(22/150): python3-cephfs-19.1.0-1382.gde09dd01. 1.3 MB/s | 188 kB 00:00 2024-08-19T01:22:53.192 INFO:teuthology.orchestra.run.smithi117.stdout:(51/150): boost-program-options-1.75.0-8.el9.x8 160 kB/s | 107 kB 00:00 2024-08-19T01:22:53.224 INFO:teuthology.orchestra.run.smithi083.stdout:(23/150): python3-rados-19.1.0-1382.gde09dd01.e 4.3 MB/s | 369 kB 00:00 2024-08-19T01:22:53.258 INFO:teuthology.orchestra.run.smithi083.stdout:(24/150): python3-rbd-19.1.0-1382.gde09dd01.el9 5.0 MB/s | 350 kB 00:00 2024-08-19T01:22:53.283 INFO:teuthology.orchestra.run.smithi083.stdout:(25/150): python3-rgw-19.1.0-1382.gde09dd01.el9 2.0 MB/s | 117 kB 00:00 2024-08-19T01:22:53.309 INFO:teuthology.orchestra.run.smithi083.stdout:(26/150): rbd-fuse-19.1.0-1382.gde09dd01.el9.x8 1.7 MB/s | 85 kB 00:00 2024-08-19T01:22:53.367 INFO:teuthology.orchestra.run.smithi083.stdout:(27/150): rbd-nbd-19.1.0-1382.gde09dd01.el9.x86 2.9 MB/s | 170 kB 00:00 2024-08-19T01:22:53.426 INFO:teuthology.orchestra.run.smithi083.stdout:(28/150): rbd-mirror-19.1.0-1382.gde09dd01.el9. 22 MB/s | 3.1 MB 00:00 2024-08-19T01:22:53.451 INFO:teuthology.orchestra.run.smithi083.stdout:(29/150): ceph-grafana-dashboards-19.1.0-1382.g 381 kB/s | 32 kB 00:00 2024-08-19T01:22:53.476 INFO:teuthology.orchestra.run.smithi117.stdout:(52/150): elfutils-libelf-devel-0.191-4.el9.x86 41 kB/s | 22 kB 00:00 2024-08-19T01:22:53.485 INFO:teuthology.orchestra.run.smithi083.stdout:(30/150): ceph-mgr-cephadm-19.1.0-1382.gde09dd0 2.5 MB/s | 146 kB 00:00 2024-08-19T01:22:53.597 INFO:teuthology.orchestra.run.smithi167.stdout:Running transaction check 2024-08-19T01:22:53.723 INFO:teuthology.orchestra.run.smithi167.stdout:Transaction check succeeded. 2024-08-19T01:22:53.723 INFO:teuthology.orchestra.run.smithi167.stdout:Running transaction test 2024-08-19T01:22:53.735 INFO:teuthology.orchestra.run.smithi083.stdout:(31/150): ceph-mgr-dashboard-19.1.0-1382.gde09d 13 MB/s | 3.6 MB 00:00 2024-08-19T01:22:53.743 INFO:teuthology.orchestra.run.smithi117.stdout:(53/150): flexiblas-3.0.4-8.el9.x86_64.rpm 59 kB/s | 33 kB 00:00 2024-08-19T01:22:53.819 INFO:teuthology.orchestra.run.smithi083.stdout:(32/150): ceph-mgr-modules-core-19.1.0-1382.gde 2.9 MB/s | 249 kB 00:00 2024-08-19T01:22:53.927 INFO:teuthology.orchestra.run.smithi083.stdout:(33/150): ceph-mgr-diskprediction-local-19.1.0- 17 MB/s | 7.4 MB 00:00 2024-08-19T01:22:53.951 INFO:teuthology.orchestra.run.smithi117.stdout:(54/150): flex-2.6.4-9.el9.x86_64.rpm 324 kB/s | 313 kB 00:00 2024-08-19T01:22:53.952 INFO:teuthology.orchestra.run.smithi083.stdout:(34/150): ceph-mgr-rook-19.1.0-1382.gde09dd01.e 369 kB/s | 49 kB 00:00 2024-08-19T01:22:53.995 INFO:teuthology.orchestra.run.smithi083.stdout:(35/150): ceph-prometheus-alerts-19.1.0-1382.gd 245 kB/s | 16 kB 00:00 2024-08-19T01:22:54.028 INFO:teuthology.orchestra.run.smithi083.stdout:(36/150): ceph-volume-19.1.0-1382.gde09dd01.el9 3.7 MB/s | 285 kB 00:00 2024-08-19T01:22:54.070 INFO:teuthology.orchestra.run.smithi083.stdout:(37/150): cephadm-19.1.0-1382.gde09dd01.el9.noa 10 MB/s | 766 kB 00:00 2024-08-19T01:22:54.096 INFO:teuthology.orchestra.run.smithi083.stdout:(38/150): cephfs-top-19.1.0-1382.gde09dd01.el9. 273 kB/s | 18 kB 00:00 2024-08-19T01:22:54.212 INFO:teuthology.orchestra.run.smithi083.stdout:(39/150): fuse-2.9.9-16.el9.x86_64.rpm 563 kB/s | 80 kB 00:00 2024-08-19T01:22:54.220 INFO:teuthology.orchestra.run.smithi117.stdout:(55/150): flexiblas-openblas-openmp-3.0.4-8.el9 38 kB/s | 18 kB 00:00 2024-08-19T01:22:54.246 INFO:teuthology.orchestra.run.smithi083.stdout:(40/150): libgfortran-11.4.1-3.el9.x86_64.rpm 5.2 MB/s | 803 kB 00:00 2024-08-19T01:22:54.296 INFO:teuthology.orchestra.run.smithi083.stdout:(41/150): libquadmath-11.4.1-3.el9.x86_64.rpm 2.3 MB/s | 195 kB 00:00 2024-08-19T01:22:54.321 INFO:teuthology.orchestra.run.smithi083.stdout:(42/150): python3-cffi-1.14.5-5.el9.x86_64.rpm 3.3 MB/s | 253 kB 00:00 2024-08-19T01:22:54.413 INFO:teuthology.orchestra.run.smithi117.stdout:(56/150): libaio-devel-0.3.111-13.el9.x86_64.rp 24 kB/s | 11 kB 00:00 2024-08-19T01:22:54.772 INFO:teuthology.orchestra.run.smithi083.stdout:(43/150): ceph-test-19.1.0-1382.gde09dd01.el9.x 20 MB/s | 49 MB 00:02 2024-08-19T01:22:54.822 INFO:teuthology.orchestra.run.smithi083.stdout:(44/150): python3-ply-3.11-14.el9.noarch.rpm 213 kB/s | 106 kB 00:00 2024-08-19T01:22:54.864 INFO:teuthology.orchestra.run.smithi083.stdout:(45/150): python3-cryptography-36.0.1-4.el9.x86 2.2 MB/s | 1.2 MB 00:00 2024-08-19T01:22:54.871 INFO:teuthology.orchestra.run.smithi117.stdout:(57/150): flexiblas-netlib-3.0.4-8.el9.x86_64.r 2.1 MB/s | 3.0 MB 00:01 2024-08-19T01:22:54.889 INFO:teuthology.orchestra.run.smithi083.stdout:(46/150): python3-requests-2.25.1-8.el9.noarch. 1.8 MB/s | 125 kB 00:00 2024-08-19T01:22:54.921 INFO:teuthology.orchestra.run.smithi117.stdout:(58/150): libpmemobj-1.12.1-1.el9.x86_64.rpm 229 kB/s | 160 kB 00:00 2024-08-19T01:22:54.931 INFO:teuthology.orchestra.run.smithi083.stdout:(47/150): python3-pycparser-2.20-6.el9.noarch.r 851 kB/s | 135 kB 00:00 2024-08-19T01:22:54.957 INFO:teuthology.orchestra.run.smithi083.stdout:(48/150): python3-urllib3-1.26.5-6.el9.noarch.r 2.3 MB/s | 215 kB 00:00 2024-08-19T01:22:54.980 INFO:teuthology.orchestra.run.smithi117.stdout:(59/150): librabbitmq-0.11.0-7.el9.x86_64.rpm 80 kB/s | 45 kB 00:00 2024-08-19T01:22:55.090 INFO:teuthology.orchestra.run.smithi083.stdout:(49/150): boost-program-options-1.75.0-8.el9.x8 805 kB/s | 107 kB 00:00 2024-08-19T01:22:55.132 INFO:teuthology.orchestra.run.smithi083.stdout:(50/150): automake-1.16.2-8.el9.noarch.rpm 2.8 MB/s | 689 kB 00:00 2024-08-19T01:22:55.158 INFO:teuthology.orchestra.run.smithi083.stdout:(51/150): elfutils-libelf-devel-0.191-4.el9.x86 323 kB/s | 22 kB 00:00 2024-08-19T01:22:55.211 INFO:teuthology.orchestra.run.smithi083.stdout:(52/150): flex-2.6.4-9.el9.x86_64.rpm 3.9 MB/s | 313 kB 00:00 2024-08-19T01:22:55.242 INFO:teuthology.orchestra.run.smithi083.stdout:(53/150): flexiblas-3.0.4-8.el9.x86_64.rpm 392 kB/s | 33 kB 00:00 2024-08-19T01:22:55.284 INFO:teuthology.orchestra.run.smithi083.stdout:(54/150): bison-3.7.4-5.el9.x86_64.rpm 2.6 MB/s | 943 kB 00:00 2024-08-19T01:22:55.309 INFO:teuthology.orchestra.run.smithi083.stdout:(55/150): flexiblas-openblas-openmp-3.0.4-8.el9 273 kB/s | 18 kB 00:00 2024-08-19T01:22:55.334 INFO:teuthology.orchestra.run.smithi083.stdout:(56/150): libaio-devel-0.3.111-13.el9.x86_64.rp 224 kB/s | 11 kB 00:00 2024-08-19T01:22:55.368 INFO:teuthology.orchestra.run.smithi083.stdout:(57/150): libpmemobj-1.12.1-1.el9.x86_64.rpm 2.7 MB/s | 160 kB 00:00 2024-08-19T01:22:55.393 INFO:teuthology.orchestra.run.smithi083.stdout:(58/150): librabbitmq-0.11.0-7.el9.x86_64.rpm 773 kB/s | 45 kB 00:00 2024-08-19T01:22:55.435 INFO:teuthology.orchestra.run.smithi083.stdout:(59/150): librdkafka-1.6.1-102.el9.x86_64.rpm 9.7 MB/s | 662 kB 00:00 2024-08-19T01:22:55.443 INFO:teuthology.orchestra.run.smithi167.stdout:Transaction test succeeded. 2024-08-19T01:22:55.443 INFO:teuthology.orchestra.run.smithi167.stdout:Running transaction 2024-08-19T01:22:55.477 INFO:teuthology.orchestra.run.smithi083.stdout:(60/150): libuuid-devel-2.37.4-18.el9.x86_64.rp 613 kB/s | 25 kB 00:00 2024-08-19T01:22:55.530 INFO:teuthology.orchestra.run.smithi117.stdout:(60/150): libuuid-devel-2.37.4-18.el9.x86_64.rp 46 kB/s | 25 kB 00:00 2024-08-19T01:22:55.544 INFO:teuthology.orchestra.run.smithi083.stdout:(61/150): flexiblas-netlib-3.0.4-8.el9.x86_64.r 9.0 MB/s | 3.0 MB 00:00 2024-08-19T01:22:55.569 INFO:teuthology.orchestra.run.smithi083.stdout:(62/150): libzstd-devel-1.5.1-2.el9.x86_64.rpm 517 kB/s | 47 kB 00:00 2024-08-19T01:22:55.631 INFO:teuthology.orchestra.run.smithi083.stdout:(63/150): libtool-2.4.6-46.el9.x86_64.rpm 2.4 MB/s | 581 kB 00:00 2024-08-19T01:22:55.670 INFO:teuthology.orchestra.run.smithi083.stdout:(64/150): lttng-ust-2.12.0-6.el9.x86_64.rpm 2.3 MB/s | 292 kB 00:00 2024-08-19T01:22:55.695 INFO:teuthology.orchestra.run.smithi083.stdout:(65/150): openblas-0.3.21-2.el9.x86_64.rpm 550 kB/s | 35 kB 00:00 2024-08-19T01:22:55.720 INFO:teuthology.orchestra.run.smithi083.stdout:(66/150): lua-5.4.4-4.el9.x86_64.rpm 1.2 MB/s | 188 kB 00:00 2024-08-19T01:22:55.764 INFO:teuthology.orchestra.run.smithi117.stdout:(61/150): librdkafka-1.6.1-102.el9.x86_64.rpm 743 kB/s | 662 kB 00:00 2024-08-19T01:22:55.797 INFO:teuthology.orchestra.run.smithi117.stdout:(62/150): libtool-2.4.6-46.el9.x86_64.rpm 663 kB/s | 581 kB 00:00 2024-08-19T01:22:55.812 INFO:teuthology.orchestra.run.smithi083.stdout:(67/150): postgresql-13.14-1.el9.x86_64.rpm 17 MB/s | 1.6 MB 00:00 2024-08-19T01:22:55.888 INFO:teuthology.orchestra.run.smithi083.stdout:(68/150): openblas-openmp-0.3.21-2.el9.x86_64.r 22 MB/s | 4.7 MB 00:00 2024-08-19T01:22:55.963 INFO:teuthology.orchestra.run.smithi083.stdout:(69/150): openssl-devel-3.2.2-2.el9.x86_64.rpm 17 MB/s | 4.4 MB 00:00 2024-08-19T01:22:55.988 INFO:teuthology.orchestra.run.smithi083.stdout:(70/150): postgresql-private-libs-13.14-1.el9.x 1.3 MB/s | 133 kB 00:00 2024-08-19T01:22:56.042 INFO:teuthology.orchestra.run.smithi083.stdout:(71/150): postgresql-contrib-13.14-1.el9.x86_64 3.9 MB/s | 911 kB 00:00 2024-08-19T01:22:56.089 INFO:teuthology.orchestra.run.smithi083.stdout:(72/150): protobuf-3.14.0-13.el9.x86_64.rpm 10 MB/s | 1.0 MB 00:00 2024-08-19T01:22:56.114 INFO:teuthology.orchestra.run.smithi083.stdout:(73/150): python3-attrs-20.3.0-7.el9.noarch.rpm 1.2 MB/s | 87 kB 00:00 2024-08-19T01:22:56.131 INFO:teuthology.orchestra.run.smithi117.stdout:(63/150): libzstd-devel-1.5.1-2.el9.x86_64.rpm 79 kB/s | 47 kB 00:00 2024-08-19T01:22:56.156 INFO:teuthology.orchestra.run.smithi083.stdout:(74/150): python3-colorama-0.4.6-3.el9.noarch.r 1.4 MB/s | 61 kB 00:00 2024-08-19T01:22:56.240 INFO:teuthology.orchestra.run.smithi083.stdout:(75/150): postgresql-server-13.14-1.el9.x86_64. 21 MB/s | 5.8 MB 00:00 2024-08-19T01:22:56.265 INFO:teuthology.orchestra.run.smithi083.stdout:(76/150): python3-devel-3.9.19-4.el9.x86_64.rpm 2.2 MB/s | 244 kB 00:00 2024-08-19T01:22:56.290 INFO:teuthology.orchestra.run.smithi083.stdout:(77/150): python3-jinja2-2.11.3-6.el9.noarch.rp 4.8 MB/s | 248 kB 00:00 2024-08-19T01:22:56.307 INFO:teuthology.orchestra.run.smithi083.stdout:(78/150): python3-jsonpatch-1.21-16.el9.noarch. 613 kB/s | 26 kB 00:00 2024-08-19T01:22:56.324 INFO:teuthology.orchestra.run.smithi083.stdout:(79/150): python3-jsonpointer-2.0-4.el9.noarch. 574 kB/s | 19 kB 00:00 2024-08-19T01:22:56.358 INFO:teuthology.orchestra.run.smithi083.stdout:(80/150): python3-mako-1.1.4-6.el9.noarch.rpm 3.3 MB/s | 172 kB 00:00 2024-08-19T01:22:56.383 INFO:teuthology.orchestra.run.smithi083.stdout:(81/150): python3-markupsafe-1.1.1-12.el9.x86_6 590 kB/s | 35 kB 00:00 2024-08-19T01:22:56.492 INFO:teuthology.orchestra.run.smithi083.stdout:(82/150): python3-babel-2.9.1-2.el9.noarch.rpm 15 MB/s | 6.0 MB 00:00 2024-08-19T01:22:56.517 INFO:teuthology.orchestra.run.smithi083.stdout:(83/150): python3-numpy-f2py-1.20.1-5.el9.x86_6 1.8 MB/s | 250 kB 00:00 2024-08-19T01:22:56.522 INFO:teuthology.orchestra.run.smithi117.stdout:(64/150): lua-5.4.4-4.el9.x86_64.rpm 260 kB/s | 188 kB 00:00 2024-08-19T01:22:56.543 INFO:teuthology.orchestra.run.smithi083.stdout:(84/150): python3-oauthlib-3.1.1-5.el9.noarch.r 4.3 MB/s | 222 kB 00:00 2024-08-19T01:22:56.548 INFO:teuthology.orchestra.run.smithi117.stdout:(65/150): lttng-ust-2.12.0-6.el9.x86_64.rpm 377 kB/s | 292 kB 00:00 2024-08-19T01:22:56.568 INFO:teuthology.orchestra.run.smithi083.stdout:(85/150): python3-packaging-20.9-5.el9.noarch.r 1.5 MB/s | 77 kB 00:00 2024-08-19T01:22:56.593 INFO:teuthology.orchestra.run.smithi083.stdout:(86/150): python3-prettytable-0.7.2-27.el9.noar 847 kB/s | 42 kB 00:00 2024-08-19T01:22:56.618 INFO:teuthology.orchestra.run.smithi083.stdout:(87/150): python3-protobuf-3.14.0-13.el9.noarch 5.3 MB/s | 269 kB 00:00 2024-08-19T01:22:56.644 INFO:teuthology.orchestra.run.smithi083.stdout:(88/150): python3-pyasn1-0.4.8-6.el9.noarch.rpm 3.1 MB/s | 159 kB 00:00 2024-08-19T01:22:56.669 INFO:teuthology.orchestra.run.smithi083.stdout:(89/150): python3-pyasn1-modules-0.4.8-6.el9.no 5.4 MB/s | 279 kB 00:00 2024-08-19T01:22:56.694 INFO:teuthology.orchestra.run.smithi083.stdout:(90/150): python3-pytz-2021.1-5.el9.noarch.rpm 1.0 MB/s | 51 kB 00:00 2024-08-19T01:22:56.719 INFO:teuthology.orchestra.run.smithi083.stdout:(91/150): python3-requests-oauthlib-1.3.0-12.el 1.0 MB/s | 54 kB 00:00 2024-08-19T01:22:56.781 INFO:teuthology.orchestra.run.smithi117.stdout:(66/150): openblas-0.3.21-2.el9.x86_64.rpm 54 kB/s | 35 kB 00:00 2024-08-19T01:22:56.803 INFO:teuthology.orchestra.run.smithi083.stdout:(92/150): python3-numpy-1.20.1-5.el9.x86_64.rpm 11 MB/s | 5.1 MB 00:00 2024-08-19T01:22:56.853 INFO:teuthology.orchestra.run.smithi083.stdout:(93/150): python3-toml-0.10.2-6.el9.noarch.rpm 312 kB/s | 42 kB 00:00 2024-08-19T01:22:56.879 INFO:teuthology.orchestra.run.smithi083.stdout:(94/150): python3-wcwidth-0.2.5-8.el9.noarch.rp 589 kB/s | 44 kB 00:00 2024-08-19T01:22:56.904 INFO:teuthology.orchestra.run.smithi083.stdout:(95/150): qatlib-24.02.0-1.el9.x86_64.rpm 4.3 MB/s | 221 kB 00:00 2024-08-19T01:22:56.929 INFO:teuthology.orchestra.run.smithi083.stdout:(96/150): qatlib-service-24.02.0-1.el9.x86_64.r 716 kB/s | 36 kB 00:00 2024-08-19T01:22:56.955 INFO:teuthology.orchestra.run.smithi083.stdout:(97/150): qatzip-libs-1.2.0-1.el9.x86_64.rpm 944 kB/s | 47 kB 00:00 2024-08-19T01:22:56.988 INFO:teuthology.orchestra.run.smithi083.stdout:(98/150): socat-1.7.4.1-6.el9.x86_64.rpm 5.1 MB/s | 304 kB 00:00 2024-08-19T01:22:57.013 INFO:teuthology.orchestra.run.smithi083.stdout:(99/150): uuid-1.6.2-55.el9.x86_64.rpm 991 kB/s | 58 kB 00:00 2024-08-19T01:22:57.039 INFO:teuthology.orchestra.run.smithi083.stdout:(100/150): xfsprogs-devel-6.4.0-3.el9.x86_64.rp 1.9 MB/s | 96 kB 00:00 2024-08-19T01:22:57.064 INFO:teuthology.orchestra.run.smithi083.stdout:(101/150): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.2 MB/s | 64 kB 00:00 2024-08-19T01:22:57.089 INFO:teuthology.orchestra.run.smithi083.stdout:(102/150): zlib-devel-1.2.11-41.el9.x86_64.rpm 897 kB/s | 45 kB 00:00 2024-08-19T01:22:57.114 INFO:teuthology.orchestra.run.smithi083.stdout:(103/150): lua-devel-5.4.4-4.el9.x86_64.rpm 444 kB/s | 22 kB 00:00 2024-08-19T01:22:57.156 INFO:teuthology.orchestra.run.smithi083.stdout:(104/150): protobuf-compiler-3.14.0-13.el9.x86_ 13 MB/s | 863 kB 00:00 2024-08-19T01:22:57.240 INFO:teuthology.orchestra.run.smithi083.stdout:(105/150): abseil-cpp-20211102.0-3.el9.x86_64.r 4.3 MB/s | 548 kB 00:00 2024-08-19T01:22:57.282 INFO:teuthology.orchestra.run.smithi083.stdout:(106/150): grpc-data-1.46.7-10.el9.noarch.rpm 467 kB/s | 19 kB 00:00 2024-08-19T01:22:57.307 INFO:teuthology.orchestra.run.smithi083.stdout:(107/150): gperftools-libs-2.9.1-2.el9.x86_64.r 2.0 MB/s | 309 kB 00:00 2024-08-19T01:22:57.333 INFO:teuthology.orchestra.run.smithi083.stdout:(108/150): libarrow-doc-9.0.0-11.el9.noarch.rpm 1.0 MB/s | 26 kB 00:00 2024-08-19T01:22:57.352 INFO:teuthology.orchestra.run.smithi083.stdout:(109/150): liboath-2.6.7-2.el9.x86_64.rpm 2.5 MB/s | 49 kB 00:00 2024-08-19T01:22:57.385 INFO:teuthology.orchestra.run.smithi083.stdout:(110/150): libunwind-1.6.2-1.el9.x86_64.rpm 2.0 MB/s | 67 kB 00:00 2024-08-19T01:22:57.419 INFO:teuthology.orchestra.run.smithi083.stdout:(111/150): luarocks-3.9.2-5.el9.noarch.rpm 4.4 MB/s | 151 kB 00:00 2024-08-19T01:22:57.494 INFO:teuthology.orchestra.run.smithi083.stdout:(112/150): parquet-libs-9.0.0-11.el9.x86_64.rpm 11 MB/s | 839 kB 00:00 2024-08-19T01:22:57.537 INFO:teuthology.orchestra.run.smithi083.stdout:(113/150): python3-asyncssh-2.13.2-5.el9.noarch 13 MB/s | 548 kB 00:00 2024-08-19T01:22:57.561 INFO:teuthology.orchestra.run.smithi083.stdout:(114/150): python3-bcrypt-3.2.2-1.el9.x86_64.rp 1.7 MB/s | 43 kB 00:00 2024-08-19T01:22:57.587 INFO:teuthology.orchestra.run.smithi083.stdout:(115/150): python3-cachetools-4.2.4-1.el9.noarc 1.3 MB/s | 32 kB 00:00 2024-08-19T01:22:57.665 INFO:teuthology.orchestra.run.smithi117.stdout:(67/150): openssl-devel-3.2.2-2.el9.x86_64.rpm 4.0 MB/s | 4.4 MB 00:01 2024-08-19T01:22:57.677 INFO:teuthology.orchestra.run.smithi083.stdout:(116/150): libarrow-9.0.0-11.el9.x86_64.rpm 11 MB/s | 4.4 MB 00:00 2024-08-19T01:22:57.704 INFO:teuthology.orchestra.run.smithi083.stdout:(117/150): python3-certifi-2023.05.07-4.el9.noa 121 kB/s | 14 kB 00:00 2024-08-19T01:22:57.740 INFO:teuthology.orchestra.run.smithi117.stdout:(68/150): openblas-openmp-0.3.21-2.el9.x86_64.r 3.9 MB/s | 4.7 MB 00:01 2024-08-19T01:22:57.782 INFO:teuthology.orchestra.run.smithi117.stdout:(69/150): postgresql-13.14-1.el9.x86_64.rpm 1.6 MB/s | 1.6 MB 00:00 2024-08-19T01:22:57.871 INFO:teuthology.orchestra.run.smithi083.stdout:(118/150): python3-scipy-1.6.2-8.el9.x86_64.rpm 14 MB/s | 16 MB 00:01 2024-08-19T01:22:57.896 INFO:teuthology.orchestra.run.smithi083.stdout:(119/150): python3-cheroot-8.6.0-4.el9.noarch.r 786 kB/s | 172 kB 00:00 2024-08-19T01:22:57.938 INFO:teuthology.orchestra.run.smithi083.stdout:(120/150): python3-cherrypy-18.6.1-2.el9.noarch 1.5 MB/s | 358 kB 00:00 2024-08-19T01:22:57.964 INFO:teuthology.orchestra.run.smithi083.stdout:(121/150): python3-google-auth-2.33.0-1.el9.noa 3.7 MB/s | 220 kB 00:00 2024-08-19T01:22:57.989 INFO:teuthology.orchestra.run.smithi083.stdout:(122/150): python3-grpcio-tools-1.46.7-10.el9.x 5.6 MB/s | 144 kB 00:00 2024-08-19T01:22:58.014 INFO:teuthology.orchestra.run.smithi083.stdout:(123/150): python3-jaraco-8.2.1-3.el9.noarch.rp 428 kB/s | 11 kB 00:00 2024-08-19T01:22:58.040 INFO:teuthology.orchestra.run.smithi083.stdout:(124/150): python3-cmd2-2.4.2-2.el9.noarch.rpm 1.8 MB/s | 303 kB 00:00 2024-08-19T01:22:58.098 INFO:teuthology.orchestra.run.smithi083.stdout:(125/150): python3-jaraco-classes-3.2.1-5.el9.n 212 kB/s | 18 kB 00:00 2024-08-19T01:22:58.124 INFO:teuthology.orchestra.run.smithi083.stdout:(126/150): python3-jaraco-collections-3.0.0-8.e 276 kB/s | 23 kB 00:00 2024-08-19T01:22:58.149 INFO:teuthology.orchestra.run.smithi083.stdout:(127/150): python3-jaraco-functools-3.5.0-2.el9 387 kB/s | 19 kB 00:00 2024-08-19T01:22:58.174 INFO:teuthology.orchestra.run.smithi083.stdout:(128/150): python3-jaraco-text-3.2.0-6.el9.noar 394 kB/s | 20 kB 00:00 2024-08-19T01:22:58.208 INFO:teuthology.orchestra.run.smithi083.stdout:(129/150): python3-logutils-0.3.5-21.el9.noarch 1.4 MB/s | 46 kB 00:00 2024-08-19T01:22:58.242 INFO:teuthology.orchestra.run.smithi083.stdout:(130/150): python3-more-itertools-8.12.0-2.el9. 2.3 MB/s | 79 kB 00:00 2024-08-19T01:22:58.267 INFO:teuthology.orchestra.run.smithi083.stdout:(131/150): python3-natsort-7.1.1-5.el9.noarch.r 2.3 MB/s | 58 kB 00:00 2024-08-19T01:22:58.309 INFO:teuthology.orchestra.run.smithi083.stdout:(132/150): python3-kubernetes-26.1.0-2.el9.noar 6.4 MB/s | 1.0 MB 00:00 2024-08-19T01:22:58.334 INFO:teuthology.orchestra.run.smithi083.stdout:(133/150): python3-portend-3.1.0-2.el9.noarch.r 665 kB/s | 16 kB 00:00 2024-08-19T01:22:58.368 INFO:teuthology.orchestra.run.smithi083.stdout:(134/150): python3-pecan-1.4.2-3.el9.noarch.rpm 2.6 MB/s | 272 kB 00:00 2024-08-19T01:22:58.393 INFO:teuthology.orchestra.run.smithi083.stdout:(135/150): python3-pyperclip-1.8.2-2.el9.noarch 1.1 MB/s | 28 kB 00:00 2024-08-19T01:22:58.418 INFO:teuthology.orchestra.run.smithi083.stdout:(136/150): python3-pyOpenSSL-21.0.0-1.el9.noarc 1.0 MB/s | 90 kB 00:00 2024-08-19T01:22:58.440 INFO:teuthology.orchestra.run.smithi117.stdout:(70/150): postgresql-private-libs-13.14-1.el9.x 190 kB/s | 133 kB 00:00 2024-08-19T01:22:58.444 INFO:teuthology.orchestra.run.smithi083.stdout:(137/150): python3-repoze-lru-0.7-16.el9.noarch 608 kB/s | 31 kB 00:00 2024-08-19T01:22:58.486 INFO:teuthology.orchestra.run.smithi083.stdout:(138/150): python3-rsa-4.9-2.el9.noarch.rpm 1.4 MB/s | 59 kB 00:00 2024-08-19T01:22:58.511 INFO:teuthology.orchestra.run.smithi083.stdout:(139/150): python3-tempora-5.0.0-2.el9.noarch.r 1.4 MB/s | 36 kB 00:00 2024-08-19T01:22:58.536 INFO:teuthology.orchestra.run.smithi083.stdout:(140/150): python3-routes-2.5.1-5.el9.noarch.rp 1.6 MB/s | 188 kB 00:00 2024-08-19T01:22:58.570 INFO:teuthology.orchestra.run.smithi083.stdout:(141/150): python3-typing-extensions-4.12.2-2.e 1.2 MB/s | 74 kB 00:00 2024-08-19T01:22:58.582 INFO:teuthology.orchestra.run.smithi117.stdout:(71/150): postgresql-contrib-13.14-1.el9.x86_64 994 kB/s | 911 kB 00:00 2024-08-19T01:22:58.720 INFO:teuthology.orchestra.run.smithi083.stdout:(142/150): python3-grpcio-1.46.7-10.el9.x86_64. 2.6 MB/s | 2.0 MB 00:00 2024-08-19T01:22:58.746 INFO:teuthology.orchestra.run.smithi083.stdout:(143/150): python3-websocket-client-1.2.3-2.el9 511 kB/s | 90 kB 00:00 2024-08-19T01:22:58.771 INFO:teuthology.orchestra.run.smithi083.stdout:(144/150): python3-webob-1.8.7-6.el9.noarch.rpm 980 kB/s | 230 kB 00:00 2024-08-19T01:22:58.788 INFO:teuthology.orchestra.run.smithi083.stdout:(145/150): python3-zc-lockfile-2.0-10.el9.noarc 479 kB/s | 20 kB 00:00 2024-08-19T01:22:58.813 INFO:teuthology.orchestra.run.smithi083.stdout:(146/150): re2-20211101-3.el9.x86_64.rpm 4.5 MB/s | 192 kB 00:00 2024-08-19T01:22:58.847 INFO:teuthology.orchestra.run.smithi083.stdout:(147/150): python3-werkzeug-2.0.3-3.el9.1.noarc 3.3 MB/s | 427 kB 00:00 2024-08-19T01:22:58.906 INFO:teuthology.orchestra.run.smithi083.stdout:(148/150): thrift-0.15.0-2.el9.x86_64.rpm 13 MB/s | 1.6 MB 00:00 2024-08-19T01:22:58.999 INFO:teuthology.orchestra.run.smithi117.stdout:(72/150): postgresql-server-13.14-1.el9.x86_64. 4.7 MB/s | 5.8 MB 00:01 2024-08-19T01:22:59.081 INFO:teuthology.orchestra.run.smithi083.stdout:(149/150): librados2-19.1.0-1382.gde09dd01.el9. 13 MB/s | 3.4 MB 00:00 2024-08-19T01:22:59.139 INFO:teuthology.orchestra.run.smithi083.stdout:(150/150): librbd1-19.1.0-1382.gde09dd01.el9.x8 11 MB/s | 3.2 MB 00:00 2024-08-19T01:22:59.144 INFO:teuthology.orchestra.run.smithi083.stdout:-------------------------------------------------------------------------------- 2024-08-19T01:22:59.144 INFO:teuthology.orchestra.run.smithi083.stdout:Total 23 MB/s | 218 MB 00:09 2024-08-19T01:22:59.308 INFO:teuthology.orchestra.run.smithi117.stdout:(73/150): python3-attrs-20.3.0-7.el9.noarch.rpm 121 kB/s | 87 kB 00:00 2024-08-19T01:22:59.433 INFO:teuthology.orchestra.run.smithi117.stdout:(74/150): protobuf-3.14.0-13.el9.x86_64.rpm 1.0 MB/s | 1.0 MB 00:00 2024-08-19T01:22:59.925 INFO:teuthology.orchestra.run.smithi117.stdout:(75/150): python3-colorama-0.4.6-3.el9.noarch.r 99 kB/s | 61 kB 00:00 2024-08-19T01:23:00.225 INFO:teuthology.orchestra.run.smithi117.stdout:(76/150): python3-devel-3.9.19-4.el9.x86_64.rpm 309 kB/s | 244 kB 00:00 2024-08-19T01:23:00.525 INFO:teuthology.orchestra.run.smithi117.stdout:(77/150): python3-babel-2.9.1-2.el9.noarch.rpm 3.9 MB/s | 6.0 MB 00:01 2024-08-19T01:23:00.709 INFO:teuthology.orchestra.run.smithi117.stdout:(78/150): python3-jinja2-2.11.3-6.el9.noarch.rp 317 kB/s | 248 kB 00:00 2024-08-19T01:23:00.759 INFO:teuthology.orchestra.run.smithi117.stdout:(79/150): python3-jsonpatch-1.21-16.el9.noarch. 48 kB/s | 26 kB 00:00 2024-08-19T01:23:00.987 INFO:teuthology.orchestra.run.smithi083.stdout:Running transaction check 2024-08-19T01:23:01.001 INFO:teuthology.orchestra.run.smithi117.stdout:(80/150): python3-jsonpointer-2.0-4.el9.noarch. 40 kB/s | 19 kB 00:00 2024-08-19T01:23:01.149 INFO:teuthology.orchestra.run.smithi083.stdout:Transaction check succeeded. 2024-08-19T01:23:01.150 INFO:teuthology.orchestra.run.smithi083.stdout:Running transaction test 2024-08-19T01:23:01.301 INFO:teuthology.orchestra.run.smithi117.stdout:(81/150): python3-markupsafe-1.1.1-12.el9.x86_6 64 kB/s | 35 kB 00:00 2024-08-19T01:23:01.333 INFO:teuthology.orchestra.run.smithi167.stdout: Preparing : 1/1 2024-08-19T01:23:01.415 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : thrift-0.15.0-2.el9.x86_64 1/152 2024-08-19T01:23:01.440 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 2/152 2024-08-19T01:23:01.443 INFO:teuthology.orchestra.run.smithi117.stdout:(82/150): python3-mako-1.1.4-6.el9.noarch.rpm 235 kB/s | 172 kB 00:00 2024-08-19T01:23:01.482 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 3/152 2024-08-19T01:23:01.895 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 4/152 2024-08-19T01:23:02.016 INFO:teuthology.orchestra.run.smithi167.stdout: Upgrading : librados2-2:19.1.0-1382.gde09dd01.el9.x86_64 5/152 2024-08-19T01:23:02.076 INFO:teuthology.orchestra.run.smithi117.stdout:(83/150): python3-numpy-f2py-1.20.1-5.el9.x86_6 324 kB/s | 250 kB 00:00 2024-08-19T01:23:02.140 INFO:teuthology.orchestra.run.smithi167.stdout: Running scriptlet: librados2-2:19.1.0-1382.gde09dd01.el9.x86_64 5/152 2024-08-19T01:23:02.167 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : libcephfs2-2:19.1.0-1382.gde09dd01.el9.x86_64 6/152 2024-08-19T01:23:02.227 INFO:teuthology.orchestra.run.smithi117.stdout:(84/150): python3-numpy-1.20.1-5.el9.x86_64.rpm 4.2 MB/s | 5.1 MB 00:01 2024-08-19T01:23:02.263 INFO:teuthology.orchestra.run.smithi167.stdout: Running scriptlet: libcephfs2-2:19.1.0-1382.gde09dd01.el9.x86_64 6/152 2024-08-19T01:23:02.302 INFO:teuthology.orchestra.run.smithi117.stdout:(85/150): python3-oauthlib-3.1.1-5.el9.noarch.r 259 kB/s | 222 kB 00:00 2024-08-19T01:23:02.310 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-rados-2:19.1.0-1382.gde09dd01.el9.x86_64 7/152 2024-08-19T01:23:02.338 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 8/152 2024-08-19T01:23:02.364 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 9/152 2024-08-19T01:23:02.518 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : postgresql-private-libs-13.14-1.el9.x86_64 10/152 2024-08-19T01:23:02.578 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : postgresql-13.14-1.el9.x86_64 11/152 2024-08-19T01:23:02.606 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : libcephsqlite-2:19.1.0-1382.gde09dd01.el9.x86_64 12/152 2024-08-19T01:23:02.711 INFO:teuthology.orchestra.run.smithi167.stdout: Running scriptlet: libcephsqlite-2:19.1.0-1382.gde09dd01.el9.x86_64 12/152 2024-08-19T01:23:02.727 INFO:teuthology.orchestra.run.smithi117.stdout:(86/150): python3-packaging-20.9-5.el9.noarch.r 119 kB/s | 77 kB 00:00 2024-08-19T01:23:02.732 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : libradosstriper1-2:19.1.0-1382.gde09dd01.el9.x86 13/152 2024-08-19T01:23:02.795 INFO:teuthology.orchestra.run.smithi167.stdout: Running scriptlet: libradosstriper1-2:19.1.0-1382.gde09dd01.el9.x86 13/152 2024-08-19T01:23:02.802 INFO:teuthology.orchestra.run.smithi117.stdout:(87/150): python3-prettytable-0.7.2-27.el9.noar 73 kB/s | 42 kB 00:00 2024-08-19T01:23:02.817 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 14/152 2024-08-19T01:23:02.840 INFO:teuthology.orchestra.run.smithi167.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 14/152 2024-08-19T01:23:03.108 INFO:teuthology.orchestra.run.smithi167.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 15/152 2024-08-19T01:23:03.119 INFO:teuthology.orchestra.run.smithi117.stdout:(88/150): python3-protobuf-3.14.0-13.el9.noarch 332 kB/s | 269 kB 00:00 2024-08-19T01:23:03.142 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 15/152 2024-08-19T01:23:03.255 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : re2-1:20211101-3.el9.x86_64 16/152 2024-08-19T01:23:03.358 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 17/152 2024-08-19T01:23:03.391 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 18/152 2024-08-19T01:23:03.426 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : liboath-2.6.7-2.el9.x86_64 19/152 2024-08-19T01:23:03.448 INFO:teuthology.orchestra.run.smithi083.stdout:Transaction test succeeded. 2024-08-19T01:23:03.448 INFO:teuthology.orchestra.run.smithi083.stdout:Running transaction 2024-08-19T01:23:03.478 INFO:teuthology.orchestra.run.smithi117.stdout:(89/150): python3-pyasn1-0.4.8-6.el9.noarch.rpm 212 kB/s | 159 kB 00:00 2024-08-19T01:23:03.502 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 20/152 2024-08-19T01:23:03.562 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 21/152 2024-08-19T01:23:03.602 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-packaging-20.9-5.el9.noarch 22/152 2024-08-19T01:23:03.603 INFO:teuthology.orchestra.run.smithi117.stdout:(90/150): python3-pyasn1-modules-0.4.8-6.el9.no 348 kB/s | 279 kB 00:00 2024-08-19T01:23:03.645 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 23/152 2024-08-19T01:23:03.677 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 24/152 2024-08-19T01:23:03.705 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : lua-5.4.4-4.el9.x86_64 25/152 2024-08-19T01:23:03.720 INFO:teuthology.orchestra.run.smithi117.stdout:(91/150): python3-pytz-2021.1-5.el9.noarch.rpm 85 kB/s | 51 kB 00:00 2024-08-19T01:23:03.777 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 26/152 2024-08-19T01:23:03.829 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-urllib3-1.26.5-6.el9.noarch 27/152 2024-08-19T01:23:03.861 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 28/152 2024-08-19T01:23:03.911 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 29/152 2024-08-19T01:23:04.005 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 30/152 2024-08-19T01:23:04.037 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-ceph-common-2:19.1.0-1382.gde09dd01.el9. 31/152 2024-08-19T01:23:04.082 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-ceph-argparse-2:19.1.0-1382.gde09dd01.el 32/152 2024-08-19T01:23:04.109 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-cephfs-2:19.1.0-1382.gde09dd01.el9.x86_6 33/152 2024-08-19T01:23:04.137 INFO:teuthology.orchestra.run.smithi117.stdout:(92/150): python3-requests-oauthlib-1.3.0-12.el 81 kB/s | 54 kB 00:00 2024-08-19T01:23:04.173 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 34/152 2024-08-19T01:23:04.225 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : luarocks-3.9.2-5.el9.noarch 35/152 2024-08-19T01:23:04.287 INFO:teuthology.orchestra.run.smithi117.stdout:(93/150): python3-toml-0.10.2-6.el9.noarch.rpm 74 kB/s | 42 kB 00:00 2024-08-19T01:23:04.301 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 36/152 2024-08-19T01:23:04.449 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 37/152 2024-08-19T01:23:04.502 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 38/152 2024-08-19T01:23:04.729 INFO:teuthology.orchestra.run.smithi117.stdout:(94/150): python3-wcwidth-0.2.5-8.el9.noarch.rp 75 kB/s | 44 kB 00:00 2024-08-19T01:23:05.045 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-rsa-4.9-2.el9.noarch 39/152 2024-08-19T01:23:05.133 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 40/152 2024-08-19T01:23:05.146 INFO:teuthology.orchestra.run.smithi117.stdout:(95/150): qatlib-24.02.0-1.el9.x86_64.rpm 257 kB/s | 221 kB 00:00 2024-08-19T01:23:05.168 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-jinja2-2.11.3-6.el9.noarch 41/152 2024-08-19T01:23:05.211 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 42/152 2024-08-19T01:23:05.250 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : librados-devel-2:19.1.0-1382.gde09dd01.el9.x86_6 43/152 2024-08-19T01:23:05.279 INFO:teuthology.orchestra.run.smithi117.stdout:(96/150): qatlib-service-24.02.0-1.el9.x86_64.r 65 kB/s | 36 kB 00:00 2024-08-19T01:23:05.282 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 44/152 2024-08-19T01:23:05.339 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 45/152 2024-08-19T01:23:05.413 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 46/152 2024-08-19T01:23:05.459 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 47/152 2024-08-19T01:23:05.495 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-typing-extensions-4.12.2-2.el9.noarch 48/152 2024-08-19T01:23:05.543 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 49/152 2024-08-19T01:23:05.578 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 50/152 2024-08-19T01:23:05.621 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 51/152 2024-08-19T01:23:05.663 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 52/152 2024-08-19T01:23:05.730 INFO:teuthology.orchestra.run.smithi117.stdout:(97/150): qatzip-libs-1.2.0-1.el9.x86_64.rpm 81 kB/s | 47 kB 00:00 2024-08-19T01:23:05.802 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 53/152 2024-08-19T01:23:05.848 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 54/152 2024-08-19T01:23:05.890 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-certifi-2023.05.07-4.el9.noarch 55/152 2024-08-19T01:23:06.002 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 56/152 2024-08-19T01:23:06.040 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-google-auth-1:2.33.0-1.el9.noarch 57/152 2024-08-19T01:23:06.072 INFO:teuthology.orchestra.run.smithi117.stdout:(98/150): socat-1.7.4.1-6.el9.x86_64.rpm 384 kB/s | 304 kB 00:00 2024-08-19T01:23:06.079 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 58/152 2024-08-19T01:23:06.105 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 59/152 2024-08-19T01:23:06.130 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 60/152 2024-08-19T01:23:06.205 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 61/152 2024-08-19T01:23:06.334 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 62/152 2024-08-19T01:23:06.355 INFO:teuthology.orchestra.run.smithi117.stdout:(99/150): uuid-1.6.2-55.el9.x86_64.rpm 92 kB/s | 58 kB 00:00 2024-08-19T01:23:06.362 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 63/152 2024-08-19T01:23:06.390 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 64/152 2024-08-19T01:23:06.425 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 65/152 2024-08-19T01:23:06.469 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : uuid-1.6.2-55.el9.x86_64 66/152 2024-08-19T01:23:06.507 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : socat-1.7.4.1-6.el9.x86_64 67/152 2024-08-19T01:23:06.514 INFO:teuthology.orchestra.run.smithi117.stdout:(100/150): python3-scipy-1.6.2-8.el9.x86_64.rpm 5.6 MB/s | 16 MB 00:02 2024-08-19T01:23:06.538 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 68/152 2024-08-19T01:23:06.568 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 69/152 2024-08-19T01:23:06.602 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 70/152 2024-08-19T01:23:06.636 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 71/152 2024-08-19T01:23:06.677 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 72/152 2024-08-19T01:23:06.712 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 73/152 2024-08-19T01:23:06.885 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 74/152 2024-08-19T01:23:06.897 INFO:teuthology.orchestra.run.smithi117.stdout:(101/150): xfsprogs-devel-6.4.0-3.el9.x86_64.rp 116 kB/s | 96 kB 00:00 2024-08-19T01:23:06.935 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 75/152 2024-08-19T01:23:06.964 INFO:teuthology.orchestra.run.smithi117.stdout:(102/150): lua-devel-5.4.4-4.el9.x86_64.rpm 336 kB/s | 22 kB 00:00 2024-08-19T01:23:06.965 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 76/152 2024-08-19T01:23:07.070 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 77/152 2024-08-19T01:23:07.098 INFO:teuthology.orchestra.run.smithi117.stdout:(103/150): protobuf-compiler-3.14.0-13.el9.x86_ 6.3 MB/s | 863 kB 00:00 2024-08-19T01:23:07.123 INFO:teuthology.orchestra.run.smithi117.stdout:(104/150): xmlstarlet-1.6.1-20.el9.x86_64.rpm 83 kB/s | 64 kB 00:00 2024-08-19T01:23:07.126 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 78/152 2024-08-19T01:23:07.148 INFO:teuthology.orchestra.run.smithi117.stdout:(105/150): zlib-devel-1.2.11-41.el9.x86_64.rpm 71 kB/s | 45 kB 00:00 2024-08-19T01:23:07.215 INFO:teuthology.orchestra.run.smithi117.stdout:(106/150): abseil-cpp-20211102.0-3.el9.x86_64.r 4.6 MB/s | 548 kB 00:00 2024-08-19T01:23:07.240 INFO:teuthology.orchestra.run.smithi117.stdout:(107/150): grpc-data-1.46.7-10.el9.noarch.rpm 250 kB/s | 19 kB 00:00 2024-08-19T01:23:07.265 INFO:teuthology.orchestra.run.smithi117.stdout:(108/150): gperftools-libs-2.9.1-2.el9.x86_64.r 2.1 MB/s | 309 kB 00:00 2024-08-19T01:23:07.291 INFO:teuthology.orchestra.run.smithi117.stdout:(109/150): libarrow-doc-9.0.0-11.el9.noarch.rpm 519 kB/s | 26 kB 00:00 2024-08-19T01:23:07.316 INFO:teuthology.orchestra.run.smithi117.stdout:(110/150): liboath-2.6.7-2.el9.x86_64.rpm 967 kB/s | 49 kB 00:00 2024-08-19T01:23:07.350 INFO:teuthology.orchestra.run.smithi117.stdout:(111/150): libunwind-1.6.2-1.el9.x86_64.rpm 1.1 MB/s | 67 kB 00:00 2024-08-19T01:23:07.375 INFO:teuthology.orchestra.run.smithi117.stdout:(112/150): luarocks-3.9.2-5.el9.noarch.rpm 2.5 MB/s | 151 kB 00:00 2024-08-19T01:23:07.409 INFO:teuthology.orchestra.run.smithi117.stdout:(113/150): parquet-libs-9.0.0-11.el9.x86_64.rpm 14 MB/s | 839 kB 00:00 2024-08-19T01:23:07.434 INFO:teuthology.orchestra.run.smithi117.stdout:(114/150): python3-bcrypt-3.2.2-1.el9.x86_64.rp 1.7 MB/s | 43 kB 00:00 2024-08-19T01:23:07.459 INFO:teuthology.orchestra.run.smithi117.stdout:(115/150): python3-asyncssh-2.13.2-5.el9.noarch 6.4 MB/s | 548 kB 00:00 2024-08-19T01:23:07.485 INFO:teuthology.orchestra.run.smithi117.stdout:(116/150): python3-cachetools-4.2.4-1.el9.noarc 637 kB/s | 32 kB 00:00 2024-08-19T01:23:07.510 INFO:teuthology.orchestra.run.smithi117.stdout:(117/150): python3-certifi-2023.05.07-4.el9.noa 280 kB/s | 14 kB 00:00 2024-08-19T01:23:07.535 INFO:teuthology.orchestra.run.smithi117.stdout:(118/150): python3-cheroot-8.6.0-4.el9.noarch.r 3.3 MB/s | 172 kB 00:00 2024-08-19T01:23:07.561 INFO:teuthology.orchestra.run.smithi117.stdout:(119/150): python3-cherrypy-18.6.1-2.el9.noarch 7.0 MB/s | 358 kB 00:00 2024-08-19T01:23:07.586 INFO:teuthology.orchestra.run.smithi117.stdout:(120/150): python3-cmd2-2.4.2-2.el9.noarch.rpm 5.9 MB/s | 303 kB 00:00 2024-08-19T01:23:07.611 INFO:teuthology.orchestra.run.smithi117.stdout:(121/150): python3-google-auth-2.33.0-1.el9.noa 4.3 MB/s | 220 kB 00:00 2024-08-19T01:23:07.645 INFO:teuthology.orchestra.run.smithi117.stdout:(122/150): python3-grpcio-tools-1.46.7-10.el9.x 4.2 MB/s | 144 kB 00:00 2024-08-19T01:23:07.670 INFO:teuthology.orchestra.run.smithi117.stdout:(123/150): python3-jaraco-8.2.1-3.el9.noarch.rp 430 kB/s | 11 kB 00:00 2024-08-19T01:23:07.695 INFO:teuthology.orchestra.run.smithi117.stdout:(124/150): python3-jaraco-classes-3.2.1-5.el9.n 714 kB/s | 18 kB 00:00 2024-08-19T01:23:07.720 INFO:teuthology.orchestra.run.smithi117.stdout:(125/150): python3-jaraco-collections-3.0.0-8.e 931 kB/s | 23 kB 00:00 2024-08-19T01:23:07.746 INFO:teuthology.orchestra.run.smithi117.stdout:(126/150): python3-jaraco-functools-3.5.0-2.el9 776 kB/s | 19 kB 00:00 2024-08-19T01:23:07.771 INFO:teuthology.orchestra.run.smithi117.stdout:(127/150): python3-jaraco-text-3.2.0-6.el9.noar 788 kB/s | 20 kB 00:00 2024-08-19T01:23:07.830 INFO:teuthology.orchestra.run.smithi117.stdout:(128/150): python3-grpcio-1.46.7-10.el9.x86_64. 8.4 MB/s | 2.0 MB 00:00 2024-08-19T01:23:07.855 INFO:teuthology.orchestra.run.smithi117.stdout:(129/150): python3-logutils-0.3.5-21.el9.noarch 1.8 MB/s | 46 kB 00:00 2024-08-19T01:23:07.862 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 79/152 2024-08-19T01:23:07.889 INFO:teuthology.orchestra.run.smithi117.stdout:(130/150): python3-kubernetes-26.1.0-2.el9.noar 8.7 MB/s | 1.0 MB 00:00 2024-08-19T01:23:07.906 INFO:teuthology.orchestra.run.smithi117.stdout:(131/150): python3-more-itertools-8.12.0-2.el9. 1.5 MB/s | 79 kB 00:00 2024-08-19T01:23:07.907 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 80/152 2024-08-19T01:23:07.931 INFO:teuthology.orchestra.run.smithi117.stdout:(132/150): python3-natsort-7.1.1-5.el9.noarch.r 1.3 MB/s | 58 kB 00:00 2024-08-19T01:23:07.940 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 81/152 2024-08-19T01:23:07.989 INFO:teuthology.orchestra.run.smithi117.stdout:(133/150): libarrow-9.0.0-11.el9.x86_64.rpm 5.7 MB/s | 4.4 MB 00:00 2024-08-19T01:23:08.015 INFO:teuthology.orchestra.run.smithi117.stdout:(134/150): python3-portend-3.1.0-2.el9.noarch.r 197 kB/s | 16 kB 00:00 2024-08-19T01:23:08.041 INFO:teuthology.orchestra.run.smithi117.stdout:(135/150): python3-pyperclip-1.8.2-2.el9.noarch 1.1 MB/s | 28 kB 00:00 2024-08-19T01:23:08.042 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 82/152 2024-08-19T01:23:08.065 INFO:teuthology.orchestra.run.smithi117.stdout:(136/150): python3-pyOpenSSL-21.0.0-1.el9.noarc 1.2 MB/s | 90 kB 00:00 2024-08-19T01:23:08.082 INFO:teuthology.orchestra.run.smithi117.stdout:(137/150): python3-repoze-lru-0.7-16.el9.noarch 734 kB/s | 31 kB 00:00 2024-08-19T01:23:08.096 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-devel-3.9.19-4.el9.x86_64 83/152 2024-08-19T01:23:08.107 INFO:teuthology.orchestra.run.smithi117.stdout:(138/150): python3-pecan-1.4.2-3.el9.noarch.rpm 1.3 MB/s | 272 kB 00:00 2024-08-19T01:23:08.124 INFO:teuthology.orchestra.run.smithi117.stdout:(139/150): python3-rsa-4.9-2.el9.noarch.rpm 1.4 MB/s | 59 kB 00:00 2024-08-19T01:23:08.140 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-colorama-0.4.6-3.el9.noarch 84/152 2024-08-19T01:23:08.141 INFO:teuthology.orchestra.run.smithi117.stdout:(140/150): python3-tempora-5.0.0-2.el9.noarch.r 1.0 MB/s | 36 kB 00:00 2024-08-19T01:23:08.166 INFO:teuthology.orchestra.run.smithi117.stdout:(141/150): python3-routes-2.5.1-5.el9.noarch.rp 1.8 MB/s | 188 kB 00:00 2024-08-19T01:23:08.183 INFO:teuthology.orchestra.run.smithi117.stdout:(142/150): python3-typing-extensions-4.12.2-2.e 1.2 MB/s | 74 kB 00:00 2024-08-19T01:23:08.209 INFO:teuthology.orchestra.run.smithi117.stdout:(143/150): python3-websocket-client-1.2.3-2.el9 2.1 MB/s | 90 kB 00:00 2024-08-19T01:23:08.232 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 85/152 2024-08-19T01:23:08.242 INFO:teuthology.orchestra.run.smithi117.stdout:(144/150): python3-webob-1.8.7-6.el9.noarch.rpm 2.2 MB/s | 230 kB 00:00 2024-08-19T01:23:08.268 INFO:teuthology.orchestra.run.smithi117.stdout:(145/150): python3-zc-lockfile-2.0-10.el9.noarc 340 kB/s | 20 kB 00:00 2024-08-19T01:23:08.284 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 86/152 2024-08-19T01:23:08.293 INFO:teuthology.orchestra.run.smithi117.stdout:(146/150): re2-20211101-3.el9.x86_64.rpm 3.7 MB/s | 192 kB 00:00 2024-08-19T01:23:08.318 INFO:teuthology.orchestra.run.smithi117.stdout:(147/150): python3-werkzeug-2.0.3-3.el9.1.noarc 3.1 MB/s | 427 kB 00:00 2024-08-19T01:23:08.369 INFO:teuthology.orchestra.run.smithi117.stdout:(148/150): thrift-0.15.0-2.el9.x86_64.rpm 16 MB/s | 1.6 MB 00:00 2024-08-19T01:23:08.460 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : openblas-0.3.21-2.el9.x86_64 87/152 2024-08-19T01:23:08.499 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 88/152 2024-08-19T01:23:08.576 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 89/152 2024-08-19T01:23:08.744 INFO:teuthology.orchestra.run.smithi117.stdout:(149/150): librados2-19.1.0-1382.gde09dd01.el9. 7.6 MB/s | 3.4 MB 00:00 2024-08-19T01:23:08.794 INFO:teuthology.orchestra.run.smithi117.stdout:(150/150): librbd1-19.1.0-1382.gde09dd01.el9.x8 6.6 MB/s | 3.2 MB 00:00 2024-08-19T01:23:08.800 INFO:teuthology.orchestra.run.smithi117.stdout:-------------------------------------------------------------------------------- 2024-08-19T01:23:08.800 INFO:teuthology.orchestra.run.smithi117.stdout:Total 8.2 MB/s | 218 MB 00:26 2024-08-19T01:23:09.236 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 90/152 2024-08-19T01:23:09.350 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 91/152 2024-08-19T01:23:09.850 INFO:teuthology.orchestra.run.smithi083.stdout: Preparing : 1/1 2024-08-19T01:23:09.897 INFO:teuthology.orchestra.run.smithi117.stdout:Running transaction check 2024-08-19T01:23:09.952 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : thrift-0.15.0-2.el9.x86_64 1/152 2024-08-19T01:23:09.993 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 2/152 2024-08-19T01:23:10.023 INFO:teuthology.orchestra.run.smithi117.stdout:Transaction check succeeded. 2024-08-19T01:23:10.023 INFO:teuthology.orchestra.run.smithi117.stdout:Running transaction test 2024-08-19T01:23:10.073 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 3/152 2024-08-19T01:23:10.464 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 4/152 2024-08-19T01:23:10.585 INFO:teuthology.orchestra.run.smithi083.stdout: Upgrading : librados2-2:19.1.0-1382.gde09dd01.el9.x86_64 5/152 2024-08-19T01:23:10.600 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 92/152 2024-08-19T01:23:10.687 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 93/152 2024-08-19T01:23:10.715 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 94/152 2024-08-19T01:23:10.748 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 95/152 2024-08-19T01:23:10.779 INFO:teuthology.orchestra.run.smithi083.stdout: Running scriptlet: librados2-2:19.1.0-1382.gde09dd01.el9.x86_64 5/152 2024-08-19T01:23:10.811 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : libcephfs2-2:19.1.0-1382.gde09dd01.el9.x86_64 6/152 2024-08-19T01:23:11.007 INFO:teuthology.orchestra.run.smithi083.stdout: Running scriptlet: libcephfs2-2:19.1.0-1382.gde09dd01.el9.x86_64 6/152 2024-08-19T01:23:11.057 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 96/152 2024-08-19T01:23:11.066 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-rados-2:19.1.0-1382.gde09dd01.el9.x86_64 7/152 2024-08-19T01:23:11.095 INFO:teuthology.orchestra.run.smithi167.stdout: Upgrading : librbd1-2:19.1.0-1382.gde09dd01.el9.x86_64 97/152 2024-08-19T01:23:11.100 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 8/152 2024-08-19T01:23:11.134 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 9/152 2024-08-19T01:23:11.190 INFO:teuthology.orchestra.run.smithi167.stdout: Running scriptlet: librbd1-2:19.1.0-1382.gde09dd01.el9.x86_64 97/152 2024-08-19T01:23:11.224 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-rbd-2:19.1.0-1382.gde09dd01.el9.x86_64 98/152 2024-08-19T01:23:11.261 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 99/152 2024-08-19T01:23:11.377 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : postgresql-private-libs-13.14-1.el9.x86_64 10/152 2024-08-19T01:23:11.444 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : postgresql-13.14-1.el9.x86_64 11/152 2024-08-19T01:23:11.475 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : libcephsqlite-2:19.1.0-1382.gde09dd01.el9.x86_64 12/152 2024-08-19T01:23:11.627 INFO:teuthology.orchestra.run.smithi083.stdout: Running scriptlet: libcephsqlite-2:19.1.0-1382.gde09dd01.el9.x86_64 12/152 2024-08-19T01:23:11.651 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : libradosstriper1-2:19.1.0-1382.gde09dd01.el9.x86 13/152 2024-08-19T01:23:11.733 INFO:teuthology.orchestra.run.smithi117.stdout:Transaction test succeeded. 2024-08-19T01:23:11.733 INFO:teuthology.orchestra.run.smithi117.stdout:Running transaction 2024-08-19T01:23:11.734 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 100/152 2024-08-19T01:23:11.773 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : librgw2-2:19.1.0-1382.gde09dd01.el9.x86_64 101/152 2024-08-19T01:23:11.784 INFO:teuthology.orchestra.run.smithi083.stdout: Running scriptlet: libradosstriper1-2:19.1.0-1382.gde09dd01.el9.x86 13/152 2024-08-19T01:23:11.811 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 14/152 2024-08-19T01:23:11.845 INFO:teuthology.orchestra.run.smithi167.stdout: Running scriptlet: librgw2-2:19.1.0-1382.gde09dd01.el9.x86_64 101/152 2024-08-19T01:23:11.849 INFO:teuthology.orchestra.run.smithi083.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 14/152 2024-08-19T01:23:12.083 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-rgw-2:19.1.0-1382.gde09dd01.el9.x86_64 102/152 2024-08-19T01:23:12.385 INFO:teuthology.orchestra.run.smithi083.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 15/152 2024-08-19T01:23:12.427 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 15/152 2024-08-19T01:23:12.582 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : re2-1:20211101-3.el9.x86_64 16/152 2024-08-19T01:23:12.701 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 17/152 2024-08-19T01:23:12.745 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 18/152 2024-08-19T01:23:12.790 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : liboath-2.6.7-2.el9.x86_64 19/152 2024-08-19T01:23:12.873 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 20/152 2024-08-19T01:23:12.927 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 21/152 2024-08-19T01:23:12.990 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-packaging-20.9-5.el9.noarch 22/152 2024-08-19T01:23:13.044 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 23/152 2024-08-19T01:23:13.083 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 24/152 2024-08-19T01:23:13.125 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : lua-5.4.4-4.el9.x86_64 25/152 2024-08-19T01:23:13.216 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 26/152 2024-08-19T01:23:13.276 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-urllib3-1.26.5-6.el9.noarch 27/152 2024-08-19T01:23:13.312 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 28/152 2024-08-19T01:23:13.391 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 29/152 2024-08-19T01:23:13.473 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 30/152 2024-08-19T01:23:13.507 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-ceph-common-2:19.1.0-1382.gde09dd01.el9. 31/152 2024-08-19T01:23:13.553 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-ceph-argparse-2:19.1.0-1382.gde09dd01.el 32/152 2024-08-19T01:23:13.588 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-cephfs-2:19.1.0-1382.gde09dd01.el9.x86_6 33/152 2024-08-19T01:23:13.670 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 34/152 2024-08-19T01:23:13.732 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : luarocks-3.9.2-5.el9.noarch 35/152 2024-08-19T01:23:13.840 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 36/152 2024-08-19T01:23:13.999 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 37/152 2024-08-19T01:23:14.061 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 38/152 2024-08-19T01:23:14.362 INFO:teuthology.orchestra.run.smithi167.stdout: Running scriptlet: ceph-common-2:19.1.0-1382.gde09dd01.el9.x86_64 103/152 2024-08-19T01:23:14.404 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : ceph-common-2:19.1.0-1382.gde09dd01.el9.x86_64 103/152 2024-08-19T01:23:14.697 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-rsa-4.9-2.el9.noarch 39/152 2024-08-19T01:23:14.811 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 40/152 2024-08-19T01:23:14.847 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-jinja2-2.11.3-6.el9.noarch 41/152 2024-08-19T01:23:14.890 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 42/152 2024-08-19T01:23:14.929 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : librados-devel-2:19.1.0-1382.gde09dd01.el9.x86_6 43/152 2024-08-19T01:23:14.953 INFO:teuthology.orchestra.run.smithi167.stdout: Running scriptlet: ceph-common-2:19.1.0-1382.gde09dd01.el9.x86_64 103/152 2024-08-19T01:23:14.992 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : ceph-base-2:19.1.0-1382.gde09dd01.el9.x86_64 104/152 2024-08-19T01:23:14.994 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 44/152 2024-08-19T01:23:15.055 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 45/152 2024-08-19T01:23:15.077 INFO:teuthology.orchestra.run.smithi167.stdout: Running scriptlet: ceph-base-2:19.1.0-1382.gde09dd01.el9.x86_64 104/152 2024-08-19T01:23:15.077 INFO:teuthology.orchestra.run.smithi167.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-08-19T01:23:15.077 INFO:teuthology.orchestra.run.smithi167.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-08-19T01:23:15.077 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-08-19T01:23:15.126 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : ceph-selinux-2:19.1.0-1382.gde09dd01.el9.x86_64 105/152 2024-08-19T01:23:15.141 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 46/152 2024-08-19T01:23:15.196 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 47/152 2024-08-19T01:23:15.233 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-typing-extensions-4.12.2-2.el9.noarch 48/152 2024-08-19T01:23:15.292 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 49/152 2024-08-19T01:23:15.331 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 50/152 2024-08-19T01:23:15.405 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 51/152 2024-08-19T01:23:15.452 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 52/152 2024-08-19T01:23:15.802 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 53/152 2024-08-19T01:23:15.853 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 54/152 2024-08-19T01:23:15.895 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-certifi-2023.05.07-4.el9.noarch 55/152 2024-08-19T01:23:16.012 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 56/152 2024-08-19T01:23:16.061 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-google-auth-1:2.33.0-1.el9.noarch 57/152 2024-08-19T01:23:16.104 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 58/152 2024-08-19T01:23:16.133 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 59/152 2024-08-19T01:23:16.183 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 60/152 2024-08-19T01:23:16.315 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 61/152 2024-08-19T01:23:16.469 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 62/152 2024-08-19T01:23:16.517 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 63/152 2024-08-19T01:23:16.586 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 64/152 2024-08-19T01:23:16.625 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 65/152 2024-08-19T01:23:16.676 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : uuid-1.6.2-55.el9.x86_64 66/152 2024-08-19T01:23:16.721 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : socat-1.7.4.1-6.el9.x86_64 67/152 2024-08-19T01:23:16.760 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 68/152 2024-08-19T01:23:16.797 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 69/152 2024-08-19T01:23:16.831 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 70/152 2024-08-19T01:23:16.865 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 71/152 2024-08-19T01:23:16.916 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 72/152 2024-08-19T01:23:16.950 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 73/152 2024-08-19T01:23:17.075 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 74/152 2024-08-19T01:23:17.125 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 75/152 2024-08-19T01:23:17.161 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 76/152 2024-08-19T01:23:17.286 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 77/152 2024-08-19T01:23:17.349 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 78/152 2024-08-19T01:23:17.610 INFO:teuthology.orchestra.run.smithi117.stdout: Preparing : 1/1 2024-08-19T01:23:17.679 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : thrift-0.15.0-2.el9.x86_64 1/152 2024-08-19T01:23:17.704 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 2/152 2024-08-19T01:23:17.746 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 3/152 2024-08-19T01:23:18.111 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 4/152 2024-08-19T01:23:18.209 INFO:teuthology.orchestra.run.smithi117.stdout: Upgrading : librados2-2:19.1.0-1382.gde09dd01.el9.x86_64 5/152 2024-08-19T01:23:18.249 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 79/152 2024-08-19T01:23:18.301 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 80/152 2024-08-19T01:23:18.335 INFO:teuthology.orchestra.run.smithi117.stdout: Running scriptlet: librados2-2:19.1.0-1382.gde09dd01.el9.x86_64 5/152 2024-08-19T01:23:18.335 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 81/152 2024-08-19T01:23:18.364 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : libcephfs2-2:19.1.0-1382.gde09dd01.el9.x86_64 6/152 2024-08-19T01:23:18.451 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 82/152 2024-08-19T01:23:18.459 INFO:teuthology.orchestra.run.smithi117.stdout: Running scriptlet: libcephfs2-2:19.1.0-1382.gde09dd01.el9.x86_64 6/152 2024-08-19T01:23:18.499 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-rados-2:19.1.0-1382.gde09dd01.el9.x86_64 7/152 2024-08-19T01:23:18.518 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-devel-3.9.19-4.el9.x86_64 83/152 2024-08-19T01:23:18.527 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 8/152 2024-08-19T01:23:18.553 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 9/152 2024-08-19T01:23:18.572 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-colorama-0.4.6-3.el9.noarch 84/152 2024-08-19T01:23:18.676 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 85/152 2024-08-19T01:23:18.707 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : postgresql-private-libs-13.14-1.el9.x86_64 10/152 2024-08-19T01:23:18.720 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 86/152 2024-08-19T01:23:18.758 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : postgresql-13.14-1.el9.x86_64 11/152 2024-08-19T01:23:18.786 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : libcephsqlite-2:19.1.0-1382.gde09dd01.el9.x86_64 12/152 2024-08-19T01:23:18.890 INFO:teuthology.orchestra.run.smithi117.stdout: Running scriptlet: libcephsqlite-2:19.1.0-1382.gde09dd01.el9.x86_64 12/152 2024-08-19T01:23:18.912 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : libradosstriper1-2:19.1.0-1382.gde09dd01.el9.x86 13/152 2024-08-19T01:23:18.943 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : openblas-0.3.21-2.el9.x86_64 87/152 2024-08-19T01:23:18.966 INFO:teuthology.orchestra.run.smithi117.stdout: Running scriptlet: libradosstriper1-2:19.1.0-1382.gde09dd01.el9.x86 13/152 2024-08-19T01:23:18.988 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 14/152 2024-08-19T01:23:19.000 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 88/152 2024-08-19T01:23:19.008 INFO:teuthology.orchestra.run.smithi117.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 14/152 2024-08-19T01:23:19.094 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 89/152 2024-08-19T01:23:19.321 INFO:teuthology.orchestra.run.smithi117.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 15/152 2024-08-19T01:23:19.354 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 15/152 2024-08-19T01:23:19.461 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : re2-1:20211101-3.el9.x86_64 16/152 2024-08-19T01:23:19.561 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 17/152 2024-08-19T01:23:19.595 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 18/152 2024-08-19T01:23:19.631 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : liboath-2.6.7-2.el9.x86_64 19/152 2024-08-19T01:23:19.714 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 20/152 2024-08-19T01:23:19.773 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 21/152 2024-08-19T01:23:19.813 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-packaging-20.9-5.el9.noarch 22/152 2024-08-19T01:23:19.857 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 23/152 2024-08-19T01:23:19.898 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 24/152 2024-08-19T01:23:19.906 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 90/152 2024-08-19T01:23:19.925 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : lua-5.4.4-4.el9.x86_64 25/152 2024-08-19T01:23:19.994 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 26/152 2024-08-19T01:23:20.048 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-urllib3-1.26.5-6.el9.noarch 27/152 2024-08-19T01:23:20.049 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 91/152 2024-08-19T01:23:20.078 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 28/152 2024-08-19T01:23:20.122 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 29/152 2024-08-19T01:23:20.198 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 30/152 2024-08-19T01:23:20.233 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-ceph-common-2:19.1.0-1382.gde09dd01.el9. 31/152 2024-08-19T01:23:20.278 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-ceph-argparse-2:19.1.0-1382.gde09dd01.el 32/152 2024-08-19T01:23:20.304 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-cephfs-2:19.1.0-1382.gde09dd01.el9.x86_6 33/152 2024-08-19T01:23:20.375 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 34/152 2024-08-19T01:23:20.419 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : luarocks-3.9.2-5.el9.noarch 35/152 2024-08-19T01:23:20.493 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 36/152 2024-08-19T01:23:20.629 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 37/152 2024-08-19T01:23:20.689 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 38/152 2024-08-19T01:23:21.213 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-rsa-4.9-2.el9.noarch 39/152 2024-08-19T01:23:21.292 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 40/152 2024-08-19T01:23:21.321 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-jinja2-2.11.3-6.el9.noarch 41/152 2024-08-19T01:23:21.364 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 42/152 2024-08-19T01:23:21.402 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : librados-devel-2:19.1.0-1382.gde09dd01.el9.x86_6 43/152 2024-08-19T01:23:21.435 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 44/152 2024-08-19T01:23:21.491 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 45/152 2024-08-19T01:23:21.563 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 46/152 2024-08-19T01:23:21.594 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 47/152 2024-08-19T01:23:21.623 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 92/152 2024-08-19T01:23:21.630 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-typing-extensions-4.12.2-2.el9.noarch 48/152 2024-08-19T01:23:21.677 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 49/152 2024-08-19T01:23:21.713 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 50/152 2024-08-19T01:23:21.742 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 93/152 2024-08-19T01:23:21.756 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 51/152 2024-08-19T01:23:21.798 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 52/152 2024-08-19T01:23:21.806 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 94/152 2024-08-19T01:23:21.836 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 95/152 2024-08-19T01:23:21.935 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 53/152 2024-08-19T01:23:21.975 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 54/152 2024-08-19T01:23:22.017 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-certifi-2023.05.07-4.el9.noarch 55/152 2024-08-19T01:23:22.119 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 56/152 2024-08-19T01:23:22.166 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-google-auth-1:2.33.0-1.el9.noarch 57/152 2024-08-19T01:23:22.173 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 96/152 2024-08-19T01:23:22.207 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 58/152 2024-08-19T01:23:22.232 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 59/152 2024-08-19T01:23:22.239 INFO:teuthology.orchestra.run.smithi083.stdout: Upgrading : librbd1-2:19.1.0-1382.gde09dd01.el9.x86_64 97/152 2024-08-19T01:23:22.257 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 60/152 2024-08-19T01:23:22.331 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 61/152 2024-08-19T01:23:22.356 INFO:teuthology.orchestra.run.smithi083.stdout: Running scriptlet: librbd1-2:19.1.0-1382.gde09dd01.el9.x86_64 97/152 2024-08-19T01:23:22.395 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-rbd-2:19.1.0-1382.gde09dd01.el9.x86_64 98/152 2024-08-19T01:23:22.452 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 99/152 2024-08-19T01:23:22.459 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 62/152 2024-08-19T01:23:22.497 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 63/152 2024-08-19T01:23:22.525 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 64/152 2024-08-19T01:23:22.560 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 65/152 2024-08-19T01:23:22.604 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : uuid-1.6.2-55.el9.x86_64 66/152 2024-08-19T01:23:22.641 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : socat-1.7.4.1-6.el9.x86_64 67/152 2024-08-19T01:23:22.672 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 68/152 2024-08-19T01:23:22.702 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 69/152 2024-08-19T01:23:22.729 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 70/152 2024-08-19T01:23:22.762 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 71/152 2024-08-19T01:23:22.812 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 72/152 2024-08-19T01:23:22.846 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 73/152 2024-08-19T01:23:22.934 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 74/152 2024-08-19T01:23:22.963 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 100/152 2024-08-19T01:23:22.985 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 75/152 2024-08-19T01:23:23.016 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 76/152 2024-08-19T01:23:23.018 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : librgw2-2:19.1.0-1382.gde09dd01.el9.x86_64 101/152 2024-08-19T01:23:23.110 INFO:teuthology.orchestra.run.smithi083.stdout: Running scriptlet: librgw2-2:19.1.0-1382.gde09dd01.el9.x86_64 101/152 2024-08-19T01:23:23.120 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 77/152 2024-08-19T01:23:23.136 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-rgw-2:19.1.0-1382.gde09dd01.el9.x86_64 102/152 2024-08-19T01:23:23.184 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 78/152 2024-08-19T01:23:23.916 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 79/152 2024-08-19T01:23:23.965 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 80/152 2024-08-19T01:23:23.999 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 81/152 2024-08-19T01:23:24.099 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 82/152 2024-08-19T01:23:24.154 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-devel-3.9.19-4.el9.x86_64 83/152 2024-08-19T01:23:24.199 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-colorama-0.4.6-3.el9.noarch 84/152 2024-08-19T01:23:24.295 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 85/152 2024-08-19T01:23:24.335 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 86/152 2024-08-19T01:23:24.509 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : openblas-0.3.21-2.el9.x86_64 87/152 2024-08-19T01:23:24.540 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 88/152 2024-08-19T01:23:24.618 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 89/152 2024-08-19T01:23:25.268 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 90/152 2024-08-19T01:23:25.372 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 91/152 2024-08-19T01:23:25.689 INFO:teuthology.orchestra.run.smithi083.stdout: Running scriptlet: ceph-common-2:19.1.0-1382.gde09dd01.el9.x86_64 103/152 2024-08-19T01:23:25.741 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : ceph-common-2:19.1.0-1382.gde09dd01.el9.x86_64 103/152 2024-08-19T01:23:26.376 INFO:teuthology.orchestra.run.smithi083.stdout: Running scriptlet: ceph-common-2:19.1.0-1382.gde09dd01.el9.x86_64 103/152 2024-08-19T01:23:26.428 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : ceph-base-2:19.1.0-1382.gde09dd01.el9.x86_64 104/152 2024-08-19T01:23:26.578 INFO:teuthology.orchestra.run.smithi083.stdout: Running scriptlet: ceph-base-2:19.1.0-1382.gde09dd01.el9.x86_64 104/152 2024-08-19T01:23:26.578 INFO:teuthology.orchestra.run.smithi083.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-08-19T01:23:26.578 INFO:teuthology.orchestra.run.smithi083.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-08-19T01:23:26.578 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-08-19T01:23:26.618 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 92/152 2024-08-19T01:23:26.638 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : ceph-selinux-2:19.1.0-1382.gde09dd01.el9.x86_64 105/152 2024-08-19T01:23:26.708 INFO:teuthology.orchestra.run.smithi167.stdout: Running scriptlet: ceph-selinux-2:19.1.0-1382.gde09dd01.el9.x86_64 105/152 2024-08-19T01:23:26.709 INFO:teuthology.orchestra.run.smithi167.stdout:skipping the directory /sys 2024-08-19T01:23:26.709 INFO:teuthology.orchestra.run.smithi167.stdout:skipping the directory /proc 2024-08-19T01:23:26.709 INFO:teuthology.orchestra.run.smithi167.stdout:skipping the directory /mnt 2024-08-19T01:23:26.709 INFO:teuthology.orchestra.run.smithi167.stdout:skipping the directory /var/tmp 2024-08-19T01:23:26.709 INFO:teuthology.orchestra.run.smithi167.stdout:skipping the directory /home 2024-08-19T01:23:26.709 INFO:teuthology.orchestra.run.smithi167.stdout:skipping the directory /root 2024-08-19T01:23:26.709 INFO:teuthology.orchestra.run.smithi167.stdout:skipping the directory /tmp 2024-08-19T01:23:26.709 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-08-19T01:23:26.711 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 93/152 2024-08-19T01:23:26.739 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 94/152 2024-08-19T01:23:26.771 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 95/152 2024-08-19T01:23:26.890 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : ceph-volume-2:19.1.0-1382.gde09dd01.el9.noarch 106/152 2024-08-19T01:23:26.906 INFO:teuthology.orchestra.run.smithi167.stdout: Running scriptlet: ceph-volume-2:19.1.0-1382.gde09dd01.el9.noarch 106/152 2024-08-19T01:23:26.906 INFO:teuthology.orchestra.run.smithi167.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-08-19T01:23:26.906 INFO:teuthology.orchestra.run.smithi167.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-08-19T01:23:26.907 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-08-19T01:23:27.080 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 96/152 2024-08-19T01:23:27.126 INFO:teuthology.orchestra.run.smithi117.stdout: Upgrading : librbd1-2:19.1.0-1382.gde09dd01.el9.x86_64 97/152 2024-08-19T01:23:27.221 INFO:teuthology.orchestra.run.smithi117.stdout: Running scriptlet: librbd1-2:19.1.0-1382.gde09dd01.el9.x86_64 97/152 2024-08-19T01:23:27.248 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-rbd-2:19.1.0-1382.gde09dd01.el9.x86_64 98/152 2024-08-19T01:23:27.284 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 99/152 2024-08-19T01:23:27.756 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 100/152 2024-08-19T01:23:27.787 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : librgw2-2:19.1.0-1382.gde09dd01.el9.x86_64 101/152 2024-08-19T01:23:27.852 INFO:teuthology.orchestra.run.smithi117.stdout: Running scriptlet: librgw2-2:19.1.0-1382.gde09dd01.el9.x86_64 101/152 2024-08-19T01:23:27.881 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-rgw-2:19.1.0-1382.gde09dd01.el9.x86_64 102/152 2024-08-19T01:23:28.386 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : ceph-osd-2:19.1.0-1382.gde09dd01.el9.x86_64 107/152 2024-08-19T01:23:28.414 INFO:teuthology.orchestra.run.smithi167.stdout: Running scriptlet: ceph-osd-2:19.1.0-1382.gde09dd01.el9.x86_64 107/152 2024-08-19T01:23:28.414 INFO:teuthology.orchestra.run.smithi167.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-08-19T01:23:28.414 INFO:teuthology.orchestra.run.smithi167.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-08-19T01:23:28.414 INFO:teuthology.orchestra.run.smithi167.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-08-19T01:23:28.414 INFO:teuthology.orchestra.run.smithi167.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-08-19T01:23:28.414 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-08-19T01:23:28.637 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : ceph-mds-2:19.1.0-1382.gde09dd01.el9.x86_64 108/152 2024-08-19T01:23:28.661 INFO:teuthology.orchestra.run.smithi167.stdout: Running scriptlet: ceph-mds-2:19.1.0-1382.gde09dd01.el9.x86_64 108/152 2024-08-19T01:23:28.661 INFO:teuthology.orchestra.run.smithi167.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-08-19T01:23:28.662 INFO:teuthology.orchestra.run.smithi167.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-08-19T01:23:28.662 INFO:teuthology.orchestra.run.smithi167.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-08-19T01:23:28.662 INFO:teuthology.orchestra.run.smithi167.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-08-19T01:23:28.662 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-08-19T01:23:29.104 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : ceph-mon-2:19.1.0-1382.gde09dd01.el9.x86_64 109/152 2024-08-19T01:23:29.128 INFO:teuthology.orchestra.run.smithi167.stdout: Running scriptlet: ceph-mon-2:19.1.0-1382.gde09dd01.el9.x86_64 109/152 2024-08-19T01:23:29.128 INFO:teuthology.orchestra.run.smithi167.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-08-19T01:23:29.128 INFO:teuthology.orchestra.run.smithi167.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-08-19T01:23:29.128 INFO:teuthology.orchestra.run.smithi167.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-08-19T01:23:29.128 INFO:teuthology.orchestra.run.smithi167.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-08-19T01:23:29.128 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-08-19T01:23:29.234 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : automake-1.16.2-8.el9.noarch 110/152 2024-08-19T01:23:29.291 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-ply-3.11-14.el9.noarch 111/152 2024-08-19T01:23:29.346 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 112/152 2024-08-19T01:23:29.531 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 113/152 2024-08-19T01:23:29.588 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 114/152 2024-08-19T01:23:29.665 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 115/152 2024-08-19T01:23:29.754 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 116/152 2024-08-19T01:23:29.884 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 117/152 2024-08-19T01:23:29.938 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 118/152 2024-08-19T01:23:29.970 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 119/152 2024-08-19T01:23:29.998 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : fuse-2.9.9-16.el9.x86_64 120/152 2024-08-19T01:23:30.086 INFO:teuthology.orchestra.run.smithi117.stdout: Running scriptlet: ceph-common-2:19.1.0-1382.gde09dd01.el9.x86_64 103/152 2024-08-19T01:23:30.127 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : ceph-common-2:19.1.0-1382.gde09dd01.el9.x86_64 103/152 2024-08-19T01:23:30.470 INFO:teuthology.orchestra.run.smithi167.stdout: Running scriptlet: cephadm-2:19.1.0-1382.gde09dd01.el9.noarch 121/152 2024-08-19T01:23:30.497 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : cephadm-2:19.1.0-1382.gde09dd01.el9.noarch 121/152 2024-08-19T01:23:30.529 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : ceph-prometheus-alerts-2:19.1.0-1382.gde09dd01.e 122/152 2024-08-19T01:23:30.591 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : ceph-grafana-dashboards-2:19.1.0-1382.gde09dd01. 123/152 2024-08-19T01:23:30.628 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : ceph-mgr-cephadm-2:19.1.0-1382.gde09dd01.el9.noa 124/152 2024-08-19T01:23:30.676 INFO:teuthology.orchestra.run.smithi117.stdout: Running scriptlet: ceph-common-2:19.1.0-1382.gde09dd01.el9.x86_64 103/152 2024-08-19T01:23:30.722 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : ceph-base-2:19.1.0-1382.gde09dd01.el9.x86_64 104/152 2024-08-19T01:23:30.807 INFO:teuthology.orchestra.run.smithi117.stdout: Running scriptlet: ceph-base-2:19.1.0-1382.gde09dd01.el9.x86_64 104/152 2024-08-19T01:23:30.807 INFO:teuthology.orchestra.run.smithi117.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-08-19T01:23:30.807 INFO:teuthology.orchestra.run.smithi117.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-08-19T01:23:30.807 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-08-19T01:23:30.848 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : ceph-selinux-2:19.1.0-1382.gde09dd01.el9.x86_64 105/152 2024-08-19T01:23:31.673 INFO:teuthology.orchestra.run.smithi167.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.1.0-1382.gde09dd01.el9.noa 124/152 2024-08-19T01:23:31.730 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : ceph-mgr-dashboard-2:19.1.0-1382.gde09dd01.el9.n 125/152 2024-08-19T01:23:32.693 INFO:teuthology.orchestra.run.smithi167.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.1.0-1382.gde09dd01.el9.n 125/152 2024-08-19T01:23:32.722 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : ceph-mgr-diskprediction-local-2:19.1.0-1382.gde0 126/152 2024-08-19T01:23:32.826 INFO:teuthology.orchestra.run.smithi167.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.1.0-1382.gde0 126/152 2024-08-19T01:23:33.011 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : ceph-mgr-modules-core-2:19.1.0-1382.gde09dd01.el 127/152 2024-08-19T01:23:33.041 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : ceph-mgr-2:19.1.0-1382.gde09dd01.el9.x86_64 128/152 2024-08-19T01:23:33.066 INFO:teuthology.orchestra.run.smithi167.stdout: Running scriptlet: ceph-mgr-2:19.1.0-1382.gde09dd01.el9.x86_64 128/152 2024-08-19T01:23:33.066 INFO:teuthology.orchestra.run.smithi167.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-08-19T01:23:33.066 INFO:teuthology.orchestra.run.smithi167.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-08-19T01:23:33.066 INFO:teuthology.orchestra.run.smithi167.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-08-19T01:23:33.066 INFO:teuthology.orchestra.run.smithi167.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-08-19T01:23:33.067 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-08-19T01:23:33.117 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : ceph-mgr-rook-2:19.1.0-1382.gde09dd01.el9.noarch 129/152 2024-08-19T01:23:33.129 INFO:teuthology.orchestra.run.smithi167.stdout: Running scriptlet: ceph-mgr-rook-2:19.1.0-1382.gde09dd01.el9.noarch 129/152 2024-08-19T01:23:33.227 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : ceph-2:19.1.0-1382.gde09dd01.el9.x86_64 130/152 2024-08-19T01:23:33.274 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : ceph-fuse-2:19.1.0-1382.gde09dd01.el9.x86_64 131/152 2024-08-19T01:23:34.184 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : libtool-2.4.6-46.el9.x86_64 132/152 2024-08-19T01:23:34.211 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : ceph-radosgw-2:19.1.0-1382.gde09dd01.el9.x86_64 133/152 2024-08-19T01:23:34.236 INFO:teuthology.orchestra.run.smithi167.stdout: Running scriptlet: ceph-radosgw-2:19.1.0-1382.gde09dd01.el9.x86_64 133/152 2024-08-19T01:23:34.236 INFO:teuthology.orchestra.run.smithi167.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-08-19T01:23:34.236 INFO:teuthology.orchestra.run.smithi167.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-08-19T01:23:34.236 INFO:teuthology.orchestra.run.smithi167.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-08-19T01:23:34.236 INFO:teuthology.orchestra.run.smithi167.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-08-19T01:23:34.236 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-08-19T01:23:34.279 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : ceph-immutable-object-cache-2:19.1.0-1382.gde09d 134/152 2024-08-19T01:23:34.303 INFO:teuthology.orchestra.run.smithi167.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.1.0-1382.gde09d 134/152 2024-08-19T01:23:34.303 INFO:teuthology.orchestra.run.smithi167.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-08-19T01:23:34.303 INFO:teuthology.orchestra.run.smithi167.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-08-19T01:23:34.303 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-08-19T01:23:34.354 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : cephfs-mirror-2:19.1.0-1382.gde09dd01.el9.x86_64 135/152 2024-08-19T01:23:34.378 INFO:teuthology.orchestra.run.smithi167.stdout: Running scriptlet: cephfs-mirror-2:19.1.0-1382.gde09dd01.el9.x86_64 135/152 2024-08-19T01:23:34.378 INFO:teuthology.orchestra.run.smithi167.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-08-19T01:23:34.378 INFO:teuthology.orchestra.run.smithi167.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-08-19T01:23:34.378 INFO:teuthology.orchestra.run.smithi167.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-08-19T01:23:34.378 INFO:teuthology.orchestra.run.smithi167.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-08-19T01:23:34.378 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-08-19T01:23:34.680 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : rbd-mirror-2:19.1.0-1382.gde09dd01.el9.x86_64 136/152 2024-08-19T01:23:34.705 INFO:teuthology.orchestra.run.smithi167.stdout: Running scriptlet: rbd-mirror-2:19.1.0-1382.gde09dd01.el9.x86_64 136/152 2024-08-19T01:23:34.706 INFO:teuthology.orchestra.run.smithi167.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-08-19T01:23:34.706 INFO:teuthology.orchestra.run.smithi167.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-08-19T01:23:34.706 INFO:teuthology.orchestra.run.smithi167.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-08-19T01:23:34.706 INFO:teuthology.orchestra.run.smithi167.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-08-19T01:23:34.706 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-08-19T01:23:39.063 INFO:teuthology.orchestra.run.smithi083.stdout: Running scriptlet: ceph-selinux-2:19.1.0-1382.gde09dd01.el9.x86_64 105/152 2024-08-19T01:23:39.063 INFO:teuthology.orchestra.run.smithi083.stdout:skipping the directory /sys 2024-08-19T01:23:39.063 INFO:teuthology.orchestra.run.smithi083.stdout:skipping the directory /proc 2024-08-19T01:23:39.063 INFO:teuthology.orchestra.run.smithi083.stdout:skipping the directory /mnt 2024-08-19T01:23:39.063 INFO:teuthology.orchestra.run.smithi083.stdout:skipping the directory /var/tmp 2024-08-19T01:23:39.063 INFO:teuthology.orchestra.run.smithi083.stdout:skipping the directory /home 2024-08-19T01:23:39.063 INFO:teuthology.orchestra.run.smithi083.stdout:skipping the directory /root 2024-08-19T01:23:39.063 INFO:teuthology.orchestra.run.smithi083.stdout:skipping the directory /tmp 2024-08-19T01:23:39.064 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-08-19T01:23:39.242 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : ceph-test-2:19.1.0-1382.gde09dd01.el9.x86_64 137/152 2024-08-19T01:23:39.284 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : rbd-fuse-2:19.1.0-1382.gde09dd01.el9.x86_64 138/152 2024-08-19T01:23:39.286 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : ceph-volume-2:19.1.0-1382.gde09dd01.el9.noarch 106/152 2024-08-19T01:23:39.324 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : rbd-nbd-2:19.1.0-1382.gde09dd01.el9.x86_64 139/152 2024-08-19T01:23:39.327 INFO:teuthology.orchestra.run.smithi083.stdout: Running scriptlet: ceph-volume-2:19.1.0-1382.gde09dd01.el9.noarch 106/152 2024-08-19T01:23:39.327 INFO:teuthology.orchestra.run.smithi083.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-08-19T01:23:39.328 INFO:teuthology.orchestra.run.smithi083.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-08-19T01:23:39.328 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-08-19T01:23:39.356 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : xfsprogs-devel-6.4.0-3.el9.x86_64 140/152 2024-08-19T01:23:39.383 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : elfutils-libelf-devel-0.191-4.el9.x86_64 141/152 2024-08-19T01:23:39.567 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : cephfs-shell-2:19.1.0-1382.gde09dd01.el9.x86_64 142/152 2024-08-19T01:23:39.615 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : postgresql-contrib-13.14-1.el9.x86_64 143/152 2024-08-19T01:23:39.636 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : libcephfs-devel-2:19.1.0-1382.gde09dd01.el9.x86_ 144/152 2024-08-19T01:23:40.235 INFO:teuthology.orchestra.run.smithi167.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 145/152 2024-08-19T01:23:40.285 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : postgresql-server-13.14-1.el9.x86_64 145/152 2024-08-19T01:23:40.306 INFO:teuthology.orchestra.run.smithi167.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 145/152 2024-08-19T01:23:40.922 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : ceph-osd-2:19.1.0-1382.gde09dd01.el9.x86_64 107/152 2024-08-19T01:23:40.979 INFO:teuthology.orchestra.run.smithi083.stdout: Running scriptlet: ceph-osd-2:19.1.0-1382.gde09dd01.el9.x86_64 107/152 2024-08-19T01:23:40.979 INFO:teuthology.orchestra.run.smithi083.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-08-19T01:23:40.979 INFO:teuthology.orchestra.run.smithi083.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-08-19T01:23:40.980 INFO:teuthology.orchestra.run.smithi083.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-08-19T01:23:40.980 INFO:teuthology.orchestra.run.smithi083.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-08-19T01:23:40.980 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-08-19T01:23:41.157 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : cephfs-top-2:19.1.0-1382.gde09dd01.el9.noarch 146/152 2024-08-19T01:23:41.231 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : ceph-mds-2:19.1.0-1382.gde09dd01.el9.x86_64 108/152 2024-08-19T01:23:41.257 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : openssl-devel-1:3.2.2-2.el9.x86_64 147/152 2024-08-19T01:23:41.281 INFO:teuthology.orchestra.run.smithi083.stdout: Running scriptlet: ceph-mds-2:19.1.0-1382.gde09dd01.el9.x86_64 108/152 2024-08-19T01:23:41.281 INFO:teuthology.orchestra.run.smithi083.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-08-19T01:23:41.281 INFO:teuthology.orchestra.run.smithi083.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-08-19T01:23:41.281 INFO:teuthology.orchestra.run.smithi083.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-08-19T01:23:41.281 INFO:teuthology.orchestra.run.smithi083.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-08-19T01:23:41.281 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-08-19T01:23:41.297 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 148/152 2024-08-19T01:23:41.379 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : flex-2.6.4-9.el9.x86_64 149/152 2024-08-19T01:23:41.411 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : bison-3.7.4-5.el9.x86_64 150/152 2024-08-19T01:23:41.412 INFO:teuthology.orchestra.run.smithi167.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 151/152 2024-08-19T01:23:41.520 INFO:teuthology.orchestra.run.smithi167.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 151/152 2024-08-19T01:23:41.520 INFO:teuthology.orchestra.run.smithi167.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 152/152 2024-08-19T01:23:41.773 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : ceph-mon-2:19.1.0-1382.gde09dd01.el9.x86_64 109/152 2024-08-19T01:23:41.824 INFO:teuthology.orchestra.run.smithi083.stdout: Running scriptlet: ceph-mon-2:19.1.0-1382.gde09dd01.el9.x86_64 109/152 2024-08-19T01:23:41.824 INFO:teuthology.orchestra.run.smithi083.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-08-19T01:23:41.824 INFO:teuthology.orchestra.run.smithi083.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-08-19T01:23:41.824 INFO:teuthology.orchestra.run.smithi083.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-08-19T01:23:41.824 INFO:teuthology.orchestra.run.smithi083.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-08-19T01:23:41.824 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-08-19T01:23:41.956 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : automake-1.16.2-8.el9.noarch 110/152 2024-08-19T01:23:42.016 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-ply-3.11-14.el9.noarch 111/152 2024-08-19T01:23:42.099 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 112/152 2024-08-19T01:23:42.165 INFO:teuthology.orchestra.run.smithi117.stdout: Running scriptlet: ceph-selinux-2:19.1.0-1382.gde09dd01.el9.x86_64 105/152 2024-08-19T01:23:42.165 INFO:teuthology.orchestra.run.smithi117.stdout:skipping the directory /sys 2024-08-19T01:23:42.165 INFO:teuthology.orchestra.run.smithi117.stdout:skipping the directory /proc 2024-08-19T01:23:42.165 INFO:teuthology.orchestra.run.smithi117.stdout:skipping the directory /mnt 2024-08-19T01:23:42.165 INFO:teuthology.orchestra.run.smithi117.stdout:skipping the directory /var/tmp 2024-08-19T01:23:42.165 INFO:teuthology.orchestra.run.smithi117.stdout:skipping the directory /home 2024-08-19T01:23:42.165 INFO:teuthology.orchestra.run.smithi117.stdout:skipping the directory /root 2024-08-19T01:23:42.166 INFO:teuthology.orchestra.run.smithi117.stdout:skipping the directory /tmp 2024-08-19T01:23:42.166 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-08-19T01:23:42.321 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 113/152 2024-08-19T01:23:42.333 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : ceph-volume-2:19.1.0-1382.gde09dd01.el9.noarch 106/152 2024-08-19T01:23:42.348 INFO:teuthology.orchestra.run.smithi117.stdout: Running scriptlet: ceph-volume-2:19.1.0-1382.gde09dd01.el9.noarch 106/152 2024-08-19T01:23:42.348 INFO:teuthology.orchestra.run.smithi117.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-08-19T01:23:42.349 INFO:teuthology.orchestra.run.smithi117.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-08-19T01:23:42.349 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-08-19T01:23:42.377 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 114/152 2024-08-19T01:23:42.488 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 115/152 2024-08-19T01:23:42.588 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 116/152 2024-08-19T01:23:42.755 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 117/152 2024-08-19T01:23:42.802 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 118/152 2024-08-19T01:23:42.858 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 119/152 2024-08-19T01:23:42.884 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : fuse-2.9.9-16.el9.x86_64 120/152 2024-08-19T01:23:43.399 INFO:teuthology.orchestra.run.smithi083.stdout: Running scriptlet: cephadm-2:19.1.0-1382.gde09dd01.el9.noarch 121/152 2024-08-19T01:23:43.434 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : cephadm-2:19.1.0-1382.gde09dd01.el9.noarch 121/152 2024-08-19T01:23:43.475 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : ceph-prometheus-alerts-2:19.1.0-1382.gde09dd01.e 122/152 2024-08-19T01:23:43.542 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : ceph-grafana-dashboards-2:19.1.0-1382.gde09dd01. 123/152 2024-08-19T01:23:43.573 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : ceph-mgr-cephadm-2:19.1.0-1382.gde09dd01.el9.noa 124/152 2024-08-19T01:23:43.827 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : ceph-osd-2:19.1.0-1382.gde09dd01.el9.x86_64 107/152 2024-08-19T01:23:43.853 INFO:teuthology.orchestra.run.smithi117.stdout: Running scriptlet: ceph-osd-2:19.1.0-1382.gde09dd01.el9.x86_64 107/152 2024-08-19T01:23:43.854 INFO:teuthology.orchestra.run.smithi117.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-08-19T01:23:43.854 INFO:teuthology.orchestra.run.smithi117.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-08-19T01:23:43.854 INFO:teuthology.orchestra.run.smithi117.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-08-19T01:23:43.854 INFO:teuthology.orchestra.run.smithi117.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-08-19T01:23:43.854 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-08-19T01:23:44.087 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : ceph-mds-2:19.1.0-1382.gde09dd01.el9.x86_64 108/152 2024-08-19T01:23:44.110 INFO:teuthology.orchestra.run.smithi117.stdout: Running scriptlet: ceph-mds-2:19.1.0-1382.gde09dd01.el9.x86_64 108/152 2024-08-19T01:23:44.111 INFO:teuthology.orchestra.run.smithi117.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-08-19T01:23:44.111 INFO:teuthology.orchestra.run.smithi117.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-08-19T01:23:44.111 INFO:teuthology.orchestra.run.smithi117.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-08-19T01:23:44.111 INFO:teuthology.orchestra.run.smithi117.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-08-19T01:23:44.111 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-08-19T01:23:44.547 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : ceph-mon-2:19.1.0-1382.gde09dd01.el9.x86_64 109/152 2024-08-19T01:23:44.570 INFO:teuthology.orchestra.run.smithi117.stdout: Running scriptlet: ceph-mon-2:19.1.0-1382.gde09dd01.el9.x86_64 109/152 2024-08-19T01:23:44.570 INFO:teuthology.orchestra.run.smithi117.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-08-19T01:23:44.570 INFO:teuthology.orchestra.run.smithi117.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-08-19T01:23:44.571 INFO:teuthology.orchestra.run.smithi117.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-08-19T01:23:44.571 INFO:teuthology.orchestra.run.smithi117.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-08-19T01:23:44.571 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-08-19T01:23:44.668 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : automake-1.16.2-8.el9.noarch 110/152 2024-08-19T01:23:44.725 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-ply-3.11-14.el9.noarch 111/152 2024-08-19T01:23:44.781 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 112/152 2024-08-19T01:23:44.783 INFO:teuthology.orchestra.run.smithi083.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.1.0-1382.gde09dd01.el9.noa 124/152 2024-08-19T01:23:44.845 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : ceph-mgr-dashboard-2:19.1.0-1382.gde09dd01.el9.n 125/152 2024-08-19T01:23:44.965 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 113/152 2024-08-19T01:23:45.014 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 114/152 2024-08-19T01:23:45.091 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 115/152 2024-08-19T01:23:45.188 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 116/152 2024-08-19T01:23:45.326 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 117/152 2024-08-19T01:23:45.364 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 118/152 2024-08-19T01:23:45.396 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 119/152 2024-08-19T01:23:45.424 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : fuse-2.9.9-16.el9.x86_64 120/152 2024-08-19T01:23:45.860 INFO:teuthology.orchestra.run.smithi117.stdout: Running scriptlet: cephadm-2:19.1.0-1382.gde09dd01.el9.noarch 121/152 2024-08-19T01:23:45.897 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : cephadm-2:19.1.0-1382.gde09dd01.el9.noarch 121/152 2024-08-19T01:23:45.920 INFO:teuthology.orchestra.run.smithi083.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.1.0-1382.gde09dd01.el9.n 125/152 2024-08-19T01:23:45.928 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : ceph-prometheus-alerts-2:19.1.0-1382.gde09dd01.e 122/152 2024-08-19T01:23:45.942 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : ceph-mgr-diskprediction-local-2:19.1.0-1382.gde0 126/152 2024-08-19T01:23:45.991 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : ceph-grafana-dashboards-2:19.1.0-1382.gde09dd01. 123/152 2024-08-19T01:23:46.020 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : ceph-mgr-cephadm-2:19.1.0-1382.gde09dd01.el9.noa 124/152 2024-08-19T01:23:46.080 INFO:teuthology.orchestra.run.smithi083.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.1.0-1382.gde0 126/152 2024-08-19T01:23:46.258 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : ceph-mgr-modules-core-2:19.1.0-1382.gde09dd01.el 127/152 2024-08-19T01:23:46.303 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : ceph-mgr-2:19.1.0-1382.gde09dd01.el9.x86_64 128/152 2024-08-19T01:23:46.353 INFO:teuthology.orchestra.run.smithi083.stdout: Running scriptlet: ceph-mgr-2:19.1.0-1382.gde09dd01.el9.x86_64 128/152 2024-08-19T01:23:46.354 INFO:teuthology.orchestra.run.smithi083.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-08-19T01:23:46.354 INFO:teuthology.orchestra.run.smithi083.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-08-19T01:23:46.354 INFO:teuthology.orchestra.run.smithi083.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-08-19T01:23:46.354 INFO:teuthology.orchestra.run.smithi083.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-08-19T01:23:46.354 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-08-19T01:23:46.412 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : ceph-mgr-rook-2:19.1.0-1382.gde09dd01.el9.noarch 129/152 2024-08-19T01:23:46.439 INFO:teuthology.orchestra.run.smithi083.stdout: Running scriptlet: ceph-mgr-rook-2:19.1.0-1382.gde09dd01.el9.noarch 129/152 2024-08-19T01:23:46.552 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : ceph-2:19.1.0-1382.gde09dd01.el9.x86_64 130/152 2024-08-19T01:23:46.615 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : ceph-fuse-2:19.1.0-1382.gde09dd01.el9.x86_64 131/152 2024-08-19T01:23:47.053 INFO:teuthology.orchestra.run.smithi117.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.1.0-1382.gde09dd01.el9.noa 124/152 2024-08-19T01:23:47.089 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : ceph-mgr-dashboard-2:19.1.0-1382.gde09dd01.el9.n 125/152 2024-08-19T01:23:47.448 INFO:teuthology.orchestra.run.smithi167.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 152/152 2024-08-19T01:23:47.448 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : ceph-2:19.1.0-1382.gde09dd01.el9.x86_64 1/152 2024-08-19T01:23:47.448 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : ceph-base-2:19.1.0-1382.gde09dd01.el9.x86_64 2/152 2024-08-19T01:23:47.448 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : ceph-common-2:19.1.0-1382.gde09dd01.el9.x86_64 3/152 2024-08-19T01:23:47.448 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : ceph-fuse-2:19.1.0-1382.gde09dd01.el9.x86_64 4/152 2024-08-19T01:23:47.448 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : ceph-immutable-object-cache-2:19.1.0-1382.gde09d 5/152 2024-08-19T01:23:47.448 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : ceph-mds-2:19.1.0-1382.gde09dd01.el9.x86_64 6/152 2024-08-19T01:23:47.448 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : ceph-mgr-2:19.1.0-1382.gde09dd01.el9.x86_64 7/152 2024-08-19T01:23:47.449 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : ceph-mon-2:19.1.0-1382.gde09dd01.el9.x86_64 8/152 2024-08-19T01:23:47.449 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : ceph-osd-2:19.1.0-1382.gde09dd01.el9.x86_64 9/152 2024-08-19T01:23:47.449 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : ceph-radosgw-2:19.1.0-1382.gde09dd01.el9.x86_64 10/152 2024-08-19T01:23:47.449 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : ceph-selinux-2:19.1.0-1382.gde09dd01.el9.x86_64 11/152 2024-08-19T01:23:47.449 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : ceph-test-2:19.1.0-1382.gde09dd01.el9.x86_64 12/152 2024-08-19T01:23:47.449 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : cephfs-mirror-2:19.1.0-1382.gde09dd01.el9.x86_64 13/152 2024-08-19T01:23:47.449 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : cephfs-shell-2:19.1.0-1382.gde09dd01.el9.x86_64 14/152 2024-08-19T01:23:47.449 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : libcephfs-devel-2:19.1.0-1382.gde09dd01.el9.x86_ 15/152 2024-08-19T01:23:47.449 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : libcephfs2-2:19.1.0-1382.gde09dd01.el9.x86_64 16/152 2024-08-19T01:23:47.449 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : libcephsqlite-2:19.1.0-1382.gde09dd01.el9.x86_64 17/152 2024-08-19T01:23:47.449 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : librados-devel-2:19.1.0-1382.gde09dd01.el9.x86_6 18/152 2024-08-19T01:23:47.449 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : libradosstriper1-2:19.1.0-1382.gde09dd01.el9.x86 19/152 2024-08-19T01:23:47.449 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : librgw2-2:19.1.0-1382.gde09dd01.el9.x86_64 20/152 2024-08-19T01:23:47.450 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-ceph-argparse-2:19.1.0-1382.gde09dd01.el 21/152 2024-08-19T01:23:47.450 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-ceph-common-2:19.1.0-1382.gde09dd01.el9. 22/152 2024-08-19T01:23:47.450 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-cephfs-2:19.1.0-1382.gde09dd01.el9.x86_6 23/152 2024-08-19T01:23:47.450 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-rados-2:19.1.0-1382.gde09dd01.el9.x86_64 24/152 2024-08-19T01:23:47.450 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-rbd-2:19.1.0-1382.gde09dd01.el9.x86_64 25/152 2024-08-19T01:23:47.450 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-rgw-2:19.1.0-1382.gde09dd01.el9.x86_64 26/152 2024-08-19T01:23:47.450 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : rbd-fuse-2:19.1.0-1382.gde09dd01.el9.x86_64 27/152 2024-08-19T01:23:47.450 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : rbd-mirror-2:19.1.0-1382.gde09dd01.el9.x86_64 28/152 2024-08-19T01:23:47.450 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : rbd-nbd-2:19.1.0-1382.gde09dd01.el9.x86_64 29/152 2024-08-19T01:23:47.450 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : ceph-grafana-dashboards-2:19.1.0-1382.gde09dd01. 30/152 2024-08-19T01:23:47.460 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : ceph-mgr-cephadm-2:19.1.0-1382.gde09dd01.el9.noa 31/152 2024-08-19T01:23:47.460 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : ceph-mgr-dashboard-2:19.1.0-1382.gde09dd01.el9.n 32/152 2024-08-19T01:23:47.461 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.1.0-1382.gde0 33/152 2024-08-19T01:23:47.461 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : ceph-mgr-modules-core-2:19.1.0-1382.gde09dd01.el 34/152 2024-08-19T01:23:47.461 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : ceph-mgr-rook-2:19.1.0-1382.gde09dd01.el9.noarch 35/152 2024-08-19T01:23:47.461 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : ceph-prometheus-alerts-2:19.1.0-1382.gde09dd01.e 36/152 2024-08-19T01:23:47.461 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : ceph-volume-2:19.1.0-1382.gde09dd01.el9.noarch 37/152 2024-08-19T01:23:47.461 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : cephadm-2:19.1.0-1382.gde09dd01.el9.noarch 38/152 2024-08-19T01:23:47.461 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : cephfs-top-2:19.1.0-1382.gde09dd01.el9.noarch 39/152 2024-08-19T01:23:47.461 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : fuse-2.9.9-16.el9.x86_64 40/152 2024-08-19T01:23:47.461 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 41/152 2024-08-19T01:23:47.461 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 42/152 2024-08-19T01:23:47.461 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/152 2024-08-19T01:23:47.461 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/152 2024-08-19T01:23:47.462 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/152 2024-08-19T01:23:47.462 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/152 2024-08-19T01:23:47.462 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/152 2024-08-19T01:23:47.462 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-urllib3-1.26.5-6.el9.noarch 48/152 2024-08-19T01:23:47.462 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/152 2024-08-19T01:23:47.462 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/152 2024-08-19T01:23:47.462 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/152 2024-08-19T01:23:47.462 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : elfutils-libelf-devel-0.191-4.el9.x86_64 52/152 2024-08-19T01:23:47.462 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/152 2024-08-19T01:23:47.462 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/152 2024-08-19T01:23:47.462 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/152 2024-08-19T01:23:47.462 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/152 2024-08-19T01:23:47.462 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/152 2024-08-19T01:23:47.462 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/152 2024-08-19T01:23:47.463 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/152 2024-08-19T01:23:47.463 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/152 2024-08-19T01:23:47.463 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : libtool-2.4.6-46.el9.x86_64 61/152 2024-08-19T01:23:47.463 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 62/152 2024-08-19T01:23:47.463 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/152 2024-08-19T01:23:47.463 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/152 2024-08-19T01:23:47.463 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/152 2024-08-19T01:23:47.463 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 66/152 2024-08-19T01:23:47.463 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 67/152 2024-08-19T01:23:47.463 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : openssl-devel-1:3.2.2-2.el9.x86_64 68/152 2024-08-19T01:23:47.463 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : postgresql-13.14-1.el9.x86_64 69/152 2024-08-19T01:23:47.463 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : postgresql-contrib-13.14-1.el9.x86_64 70/152 2024-08-19T01:23:47.463 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : postgresql-private-libs-13.14-1.el9.x86_64 71/152 2024-08-19T01:23:47.464 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : postgresql-server-13.14-1.el9.x86_64 72/152 2024-08-19T01:23:47.464 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/152 2024-08-19T01:23:47.464 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/152 2024-08-19T01:23:47.464 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/152 2024-08-19T01:23:47.464 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-colorama-0.4.6-3.el9.noarch 76/152 2024-08-19T01:23:47.464 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-devel-3.9.19-4.el9.x86_64 77/152 2024-08-19T01:23:47.464 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-jinja2-2.11.3-6.el9.noarch 78/152 2024-08-19T01:23:47.464 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 79/152 2024-08-19T01:23:47.464 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 80/152 2024-08-19T01:23:47.464 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 81/152 2024-08-19T01:23:47.464 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 82/152 2024-08-19T01:23:47.464 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 83/152 2024-08-19T01:23:47.464 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 84/152 2024-08-19T01:23:47.464 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 85/152 2024-08-19T01:23:47.465 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 86/152 2024-08-19T01:23:47.465 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 87/152 2024-08-19T01:23:47.465 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 88/152 2024-08-19T01:23:47.465 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 89/152 2024-08-19T01:23:47.465 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 90/152 2024-08-19T01:23:47.465 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 91/152 2024-08-19T01:23:47.465 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 92/152 2024-08-19T01:23:47.465 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 93/152 2024-08-19T01:23:47.465 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 94/152 2024-08-19T01:23:47.465 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 95/152 2024-08-19T01:23:47.465 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 96/152 2024-08-19T01:23:47.465 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 97/152 2024-08-19T01:23:47.465 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 98/152 2024-08-19T01:23:47.466 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : socat-1.7.4.1-6.el9.x86_64 99/152 2024-08-19T01:23:47.466 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 100/152 2024-08-19T01:23:47.466 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : xfsprogs-devel-6.4.0-3.el9.x86_64 101/152 2024-08-19T01:23:47.466 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 102/152 2024-08-19T01:23:47.466 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 103/152 2024-08-19T01:23:47.466 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 104/152 2024-08-19T01:23:47.466 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 105/152 2024-08-19T01:23:47.466 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 106/152 2024-08-19T01:23:47.466 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 107/152 2024-08-19T01:23:47.466 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 108/152 2024-08-19T01:23:47.466 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 109/152 2024-08-19T01:23:47.466 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 110/152 2024-08-19T01:23:47.466 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 111/152 2024-08-19T01:23:47.466 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 112/152 2024-08-19T01:23:47.467 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : luarocks-3.9.2-5.el9.noarch 113/152 2024-08-19T01:23:47.467 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 114/152 2024-08-19T01:23:47.467 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 115/152 2024-08-19T01:23:47.467 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 116/152 2024-08-19T01:23:47.467 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 117/152 2024-08-19T01:23:47.467 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-certifi-2023.05.07-4.el9.noarch 118/152 2024-08-19T01:23:47.467 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 119/152 2024-08-19T01:23:47.467 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 120/152 2024-08-19T01:23:47.467 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 121/152 2024-08-19T01:23:47.467 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-google-auth-1:2.33.0-1.el9.noarch 122/152 2024-08-19T01:23:47.467 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/152 2024-08-19T01:23:47.467 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/152 2024-08-19T01:23:47.467 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/152 2024-08-19T01:23:47.467 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/152 2024-08-19T01:23:47.468 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/152 2024-08-19T01:23:47.468 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/152 2024-08-19T01:23:47.468 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/152 2024-08-19T01:23:47.468 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/152 2024-08-19T01:23:47.468 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/152 2024-08-19T01:23:47.468 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/152 2024-08-19T01:23:47.468 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/152 2024-08-19T01:23:47.468 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/152 2024-08-19T01:23:47.468 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/152 2024-08-19T01:23:47.468 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/152 2024-08-19T01:23:47.468 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/152 2024-08-19T01:23:47.468 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/152 2024-08-19T01:23:47.468 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/152 2024-08-19T01:23:47.468 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/152 2024-08-19T01:23:47.469 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/152 2024-08-19T01:23:47.469 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-typing-extensions-4.12.2-2.el9.noarch 142/152 2024-08-19T01:23:47.469 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 143/152 2024-08-19T01:23:47.469 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/152 2024-08-19T01:23:47.469 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/152 2024-08-19T01:23:47.469 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/152 2024-08-19T01:23:47.469 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : re2-1:20211101-3.el9.x86_64 147/152 2024-08-19T01:23:47.469 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/152 2024-08-19T01:23:47.469 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : librados2-2:19.1.0-1382.gde09dd01.el9.x86_64 149/152 2024-08-19T01:23:47.469 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/152 2024-08-19T01:23:47.469 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : librbd1-2:19.1.0-1382.gde09dd01.el9.x86_64 151/152 2024-08-19T01:23:47.599 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : libtool-2.4.6-46.el9.x86_64 132/152 2024-08-19T01:23:47.672 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : ceph-radosgw-2:19.1.0-1382.gde09dd01.el9.x86_64 133/152 2024-08-19T01:23:47.722 INFO:teuthology.orchestra.run.smithi083.stdout: Running scriptlet: ceph-radosgw-2:19.1.0-1382.gde09dd01.el9.x86_64 133/152 2024-08-19T01:23:47.722 INFO:teuthology.orchestra.run.smithi083.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-08-19T01:23:47.722 INFO:teuthology.orchestra.run.smithi083.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-08-19T01:23:47.722 INFO:teuthology.orchestra.run.smithi083.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-08-19T01:23:47.722 INFO:teuthology.orchestra.run.smithi083.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-08-19T01:23:47.722 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-08-19T01:23:47.798 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : ceph-immutable-object-cache-2:19.1.0-1382.gde09d 134/152 2024-08-19T01:23:47.847 INFO:teuthology.orchestra.run.smithi083.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.1.0-1382.gde09d 134/152 2024-08-19T01:23:47.848 INFO:teuthology.orchestra.run.smithi083.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-08-19T01:23:47.848 INFO:teuthology.orchestra.run.smithi083.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-08-19T01:23:47.848 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-08-19T01:23:47.899 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : cephfs-mirror-2:19.1.0-1382.gde09dd01.el9.x86_64 135/152 2024-08-19T01:23:47.949 INFO:teuthology.orchestra.run.smithi083.stdout: Running scriptlet: cephfs-mirror-2:19.1.0-1382.gde09dd01.el9.x86_64 135/152 2024-08-19T01:23:47.949 INFO:teuthology.orchestra.run.smithi083.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-08-19T01:23:47.950 INFO:teuthology.orchestra.run.smithi083.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-08-19T01:23:47.950 INFO:teuthology.orchestra.run.smithi083.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-08-19T01:23:47.950 INFO:teuthology.orchestra.run.smithi083.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-08-19T01:23:47.950 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-08-19T01:23:48.044 INFO:teuthology.orchestra.run.smithi117.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.1.0-1382.gde09dd01.el9.n 125/152 2024-08-19T01:23:48.072 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : ceph-mgr-diskprediction-local-2:19.1.0-1382.gde0 126/152 2024-08-19T01:23:48.173 INFO:teuthology.orchestra.run.smithi117.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.1.0-1382.gde0 126/152 2024-08-19T01:23:48.275 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : rbd-mirror-2:19.1.0-1382.gde09dd01.el9.x86_64 136/152 2024-08-19T01:23:48.325 INFO:teuthology.orchestra.run.smithi083.stdout: Running scriptlet: rbd-mirror-2:19.1.0-1382.gde09dd01.el9.x86_64 136/152 2024-08-19T01:23:48.326 INFO:teuthology.orchestra.run.smithi083.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-08-19T01:23:48.326 INFO:teuthology.orchestra.run.smithi083.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-08-19T01:23:48.326 INFO:teuthology.orchestra.run.smithi083.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-08-19T01:23:48.326 INFO:teuthology.orchestra.run.smithi083.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-08-19T01:23:48.326 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-08-19T01:23:48.342 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : ceph-mgr-modules-core-2:19.1.0-1382.gde09dd01.el 127/152 2024-08-19T01:23:48.374 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : ceph-mgr-2:19.1.0-1382.gde09dd01.el9.x86_64 128/152 2024-08-19T01:23:48.398 INFO:teuthology.orchestra.run.smithi117.stdout: Running scriptlet: ceph-mgr-2:19.1.0-1382.gde09dd01.el9.x86_64 128/152 2024-08-19T01:23:48.398 INFO:teuthology.orchestra.run.smithi117.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-08-19T01:23:48.398 INFO:teuthology.orchestra.run.smithi117.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-08-19T01:23:48.399 INFO:teuthology.orchestra.run.smithi117.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-08-19T01:23:48.399 INFO:teuthology.orchestra.run.smithi117.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-08-19T01:23:48.399 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-08-19T01:23:48.442 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : ceph-mgr-rook-2:19.1.0-1382.gde09dd01.el9.noarch 129/152 2024-08-19T01:23:48.453 INFO:teuthology.orchestra.run.smithi117.stdout: Running scriptlet: ceph-mgr-rook-2:19.1.0-1382.gde09dd01.el9.noarch 129/152 2024-08-19T01:23:48.550 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : ceph-2:19.1.0-1382.gde09dd01.el9.x86_64 130/152 2024-08-19T01:23:48.598 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : ceph-fuse-2:19.1.0-1382.gde09dd01.el9.x86_64 131/152 2024-08-19T01:23:49.505 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : libtool-2.4.6-46.el9.x86_64 132/152 2024-08-19T01:23:49.544 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : ceph-radosgw-2:19.1.0-1382.gde09dd01.el9.x86_64 133/152 2024-08-19T01:23:49.567 INFO:teuthology.orchestra.run.smithi117.stdout: Running scriptlet: ceph-radosgw-2:19.1.0-1382.gde09dd01.el9.x86_64 133/152 2024-08-19T01:23:49.567 INFO:teuthology.orchestra.run.smithi117.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-08-19T01:23:49.568 INFO:teuthology.orchestra.run.smithi117.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-08-19T01:23:49.568 INFO:teuthology.orchestra.run.smithi117.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-08-19T01:23:49.568 INFO:teuthology.orchestra.run.smithi117.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-08-19T01:23:49.568 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-08-19T01:23:49.611 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : ceph-immutable-object-cache-2:19.1.0-1382.gde09d 134/152 2024-08-19T01:23:49.634 INFO:teuthology.orchestra.run.smithi117.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.1.0-1382.gde09d 134/152 2024-08-19T01:23:49.634 INFO:teuthology.orchestra.run.smithi117.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-08-19T01:23:49.634 INFO:teuthology.orchestra.run.smithi117.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-08-19T01:23:49.634 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-08-19T01:23:49.678 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : cephfs-mirror-2:19.1.0-1382.gde09dd01.el9.x86_64 135/152 2024-08-19T01:23:49.683 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/152 2024-08-19T01:23:49.683 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-08-19T01:23:49.683 INFO:teuthology.orchestra.run.smithi167.stdout:Upgraded: 2024-08-19T01:23:49.683 INFO:teuthology.orchestra.run.smithi167.stdout: librados2-2:19.1.0-1382.gde09dd01.el9.x86_64 2024-08-19T01:23:49.683 INFO:teuthology.orchestra.run.smithi167.stdout: librbd1-2:19.1.0-1382.gde09dd01.el9.x86_64 2024-08-19T01:23:49.683 INFO:teuthology.orchestra.run.smithi167.stdout:Installed: 2024-08-19T01:23:49.683 INFO:teuthology.orchestra.run.smithi167.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-08-19T01:23:49.683 INFO:teuthology.orchestra.run.smithi167.stdout: automake-1.16.2-8.el9.noarch 2024-08-19T01:23:49.683 INFO:teuthology.orchestra.run.smithi167.stdout: bison-3.7.4-5.el9.x86_64 2024-08-19T01:23:49.683 INFO:teuthology.orchestra.run.smithi167.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-08-19T01:23:49.683 INFO:teuthology.orchestra.run.smithi167.stdout: ceph-2:19.1.0-1382.gde09dd01.el9.x86_64 2024-08-19T01:23:49.683 INFO:teuthology.orchestra.run.smithi167.stdout: ceph-base-2:19.1.0-1382.gde09dd01.el9.x86_64 2024-08-19T01:23:49.683 INFO:teuthology.orchestra.run.smithi167.stdout: ceph-common-2:19.1.0-1382.gde09dd01.el9.x86_64 2024-08-19T01:23:49.684 INFO:teuthology.orchestra.run.smithi167.stdout: ceph-fuse-2:19.1.0-1382.gde09dd01.el9.x86_64 2024-08-19T01:23:49.684 INFO:teuthology.orchestra.run.smithi167.stdout: ceph-grafana-dashboards-2:19.1.0-1382.gde09dd01.el9.noarch 2024-08-19T01:23:49.684 INFO:teuthology.orchestra.run.smithi167.stdout: ceph-immutable-object-cache-2:19.1.0-1382.gde09dd01.el9.x86_64 2024-08-19T01:23:49.684 INFO:teuthology.orchestra.run.smithi167.stdout: ceph-mds-2:19.1.0-1382.gde09dd01.el9.x86_64 2024-08-19T01:23:49.684 INFO:teuthology.orchestra.run.smithi167.stdout: ceph-mgr-2:19.1.0-1382.gde09dd01.el9.x86_64 2024-08-19T01:23:49.684 INFO:teuthology.orchestra.run.smithi167.stdout: ceph-mgr-cephadm-2:19.1.0-1382.gde09dd01.el9.noarch 2024-08-19T01:23:49.684 INFO:teuthology.orchestra.run.smithi167.stdout: ceph-mgr-dashboard-2:19.1.0-1382.gde09dd01.el9.noarch 2024-08-19T01:23:49.684 INFO:teuthology.orchestra.run.smithi167.stdout: ceph-mgr-diskprediction-local-2:19.1.0-1382.gde09dd01.el9.noarch 2024-08-19T01:23:49.684 INFO:teuthology.orchestra.run.smithi167.stdout: ceph-mgr-modules-core-2:19.1.0-1382.gde09dd01.el9.noarch 2024-08-19T01:23:49.684 INFO:teuthology.orchestra.run.smithi167.stdout: ceph-mgr-rook-2:19.1.0-1382.gde09dd01.el9.noarch 2024-08-19T01:23:49.684 INFO:teuthology.orchestra.run.smithi167.stdout: ceph-mon-2:19.1.0-1382.gde09dd01.el9.x86_64 2024-08-19T01:23:49.684 INFO:teuthology.orchestra.run.smithi167.stdout: ceph-osd-2:19.1.0-1382.gde09dd01.el9.x86_64 2024-08-19T01:23:49.684 INFO:teuthology.orchestra.run.smithi167.stdout: ceph-prometheus-alerts-2:19.1.0-1382.gde09dd01.el9.noarch 2024-08-19T01:23:49.684 INFO:teuthology.orchestra.run.smithi167.stdout: ceph-radosgw-2:19.1.0-1382.gde09dd01.el9.x86_64 2024-08-19T01:23:49.685 INFO:teuthology.orchestra.run.smithi167.stdout: ceph-selinux-2:19.1.0-1382.gde09dd01.el9.x86_64 2024-08-19T01:23:49.685 INFO:teuthology.orchestra.run.smithi167.stdout: ceph-test-2:19.1.0-1382.gde09dd01.el9.x86_64 2024-08-19T01:23:49.685 INFO:teuthology.orchestra.run.smithi167.stdout: ceph-volume-2:19.1.0-1382.gde09dd01.el9.noarch 2024-08-19T01:23:49.685 INFO:teuthology.orchestra.run.smithi167.stdout: cephadm-2:19.1.0-1382.gde09dd01.el9.noarch 2024-08-19T01:23:49.685 INFO:teuthology.orchestra.run.smithi167.stdout: cephfs-mirror-2:19.1.0-1382.gde09dd01.el9.x86_64 2024-08-19T01:23:49.685 INFO:teuthology.orchestra.run.smithi167.stdout: cephfs-shell-2:19.1.0-1382.gde09dd01.el9.x86_64 2024-08-19T01:23:49.685 INFO:teuthology.orchestra.run.smithi167.stdout: cephfs-top-2:19.1.0-1382.gde09dd01.el9.noarch 2024-08-19T01:23:49.685 INFO:teuthology.orchestra.run.smithi167.stdout: elfutils-libelf-devel-0.191-4.el9.x86_64 2024-08-19T01:23:49.685 INFO:teuthology.orchestra.run.smithi167.stdout: flex-2.6.4-9.el9.x86_64 2024-08-19T01:23:49.685 INFO:teuthology.orchestra.run.smithi167.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-08-19T01:23:49.685 INFO:teuthology.orchestra.run.smithi167.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-08-19T01:23:49.685 INFO:teuthology.orchestra.run.smithi167.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-08-19T01:23:49.686 INFO:teuthology.orchestra.run.smithi167.stdout: fuse-2.9.9-16.el9.x86_64 2024-08-19T01:23:49.686 INFO:teuthology.orchestra.run.smithi167.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-08-19T01:23:49.686 INFO:teuthology.orchestra.run.smithi167.stdout: grpc-data-1.46.7-10.el9.noarch 2024-08-19T01:23:49.686 INFO:teuthology.orchestra.run.smithi167.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-08-19T01:23:49.686 INFO:teuthology.orchestra.run.smithi167.stdout: libarrow-9.0.0-11.el9.x86_64 2024-08-19T01:23:49.686 INFO:teuthology.orchestra.run.smithi167.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-08-19T01:23:49.686 INFO:teuthology.orchestra.run.smithi167.stdout: libcephfs-devel-2:19.1.0-1382.gde09dd01.el9.x86_64 2024-08-19T01:23:49.686 INFO:teuthology.orchestra.run.smithi167.stdout: libcephfs2-2:19.1.0-1382.gde09dd01.el9.x86_64 2024-08-19T01:23:49.686 INFO:teuthology.orchestra.run.smithi167.stdout: libcephsqlite-2:19.1.0-1382.gde09dd01.el9.x86_64 2024-08-19T01:23:49.686 INFO:teuthology.orchestra.run.smithi167.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-08-19T01:23:49.686 INFO:teuthology.orchestra.run.smithi167.stdout: liboath-2.6.7-2.el9.x86_64 2024-08-19T01:23:49.686 INFO:teuthology.orchestra.run.smithi167.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-08-19T01:23:49.686 INFO:teuthology.orchestra.run.smithi167.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-08-19T01:23:49.686 INFO:teuthology.orchestra.run.smithi167.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-08-19T01:23:49.687 INFO:teuthology.orchestra.run.smithi167.stdout: librados-devel-2:19.1.0-1382.gde09dd01.el9.x86_64 2024-08-19T01:23:49.687 INFO:teuthology.orchestra.run.smithi167.stdout: libradosstriper1-2:19.1.0-1382.gde09dd01.el9.x86_64 2024-08-19T01:23:49.687 INFO:teuthology.orchestra.run.smithi167.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-08-19T01:23:49.687 INFO:teuthology.orchestra.run.smithi167.stdout: librgw2-2:19.1.0-1382.gde09dd01.el9.x86_64 2024-08-19T01:23:49.687 INFO:teuthology.orchestra.run.smithi167.stdout: libtool-2.4.6-46.el9.x86_64 2024-08-19T01:23:49.687 INFO:teuthology.orchestra.run.smithi167.stdout: libunwind-1.6.2-1.el9.x86_64 2024-08-19T01:23:49.687 INFO:teuthology.orchestra.run.smithi167.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-08-19T01:23:49.687 INFO:teuthology.orchestra.run.smithi167.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-08-19T01:23:49.687 INFO:teuthology.orchestra.run.smithi167.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-08-19T01:23:49.687 INFO:teuthology.orchestra.run.smithi167.stdout: lua-5.4.4-4.el9.x86_64 2024-08-19T01:23:49.687 INFO:teuthology.orchestra.run.smithi167.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-08-19T01:23:49.687 INFO:teuthology.orchestra.run.smithi167.stdout: luarocks-3.9.2-5.el9.noarch 2024-08-19T01:23:49.687 INFO:teuthology.orchestra.run.smithi167.stdout: openblas-0.3.21-2.el9.x86_64 2024-08-19T01:23:49.687 INFO:teuthology.orchestra.run.smithi167.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-08-19T01:23:49.687 INFO:teuthology.orchestra.run.smithi167.stdout: openssl-devel-1:3.2.2-2.el9.x86_64 2024-08-19T01:23:49.688 INFO:teuthology.orchestra.run.smithi167.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-08-19T01:23:49.688 INFO:teuthology.orchestra.run.smithi167.stdout: postgresql-13.14-1.el9.x86_64 2024-08-19T01:23:49.688 INFO:teuthology.orchestra.run.smithi167.stdout: postgresql-contrib-13.14-1.el9.x86_64 2024-08-19T01:23:49.688 INFO:teuthology.orchestra.run.smithi167.stdout: postgresql-private-libs-13.14-1.el9.x86_64 2024-08-19T01:23:49.688 INFO:teuthology.orchestra.run.smithi167.stdout: postgresql-server-13.14-1.el9.x86_64 2024-08-19T01:23:49.688 INFO:teuthology.orchestra.run.smithi167.stdout: protobuf-3.14.0-13.el9.x86_64 2024-08-19T01:23:49.688 INFO:teuthology.orchestra.run.smithi167.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-08-19T01:23:49.688 INFO:teuthology.orchestra.run.smithi167.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-08-19T01:23:49.688 INFO:teuthology.orchestra.run.smithi167.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-08-19T01:23:49.688 INFO:teuthology.orchestra.run.smithi167.stdout: python3-babel-2.9.1-2.el9.noarch 2024-08-19T01:23:49.688 INFO:teuthology.orchestra.run.smithi167.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-08-19T01:23:49.688 INFO:teuthology.orchestra.run.smithi167.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-08-19T01:23:49.688 INFO:teuthology.orchestra.run.smithi167.stdout: python3-ceph-argparse-2:19.1.0-1382.gde09dd01.el9.x86_64 2024-08-19T01:23:49.688 INFO:teuthology.orchestra.run.smithi167.stdout: python3-ceph-common-2:19.1.0-1382.gde09dd01.el9.x86_64 2024-08-19T01:23:49.688 INFO:teuthology.orchestra.run.smithi167.stdout: python3-cephfs-2:19.1.0-1382.gde09dd01.el9.x86_64 2024-08-19T01:23:49.689 INFO:teuthology.orchestra.run.smithi167.stdout: python3-certifi-2023.05.07-4.el9.noarch 2024-08-19T01:23:49.689 INFO:teuthology.orchestra.run.smithi167.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-08-19T01:23:49.689 INFO:teuthology.orchestra.run.smithi167.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-08-19T01:23:49.689 INFO:teuthology.orchestra.run.smithi167.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-08-19T01:23:49.689 INFO:teuthology.orchestra.run.smithi167.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-08-19T01:23:49.689 INFO:teuthology.orchestra.run.smithi167.stdout: python3-colorama-0.4.6-3.el9.noarch 2024-08-19T01:23:49.689 INFO:teuthology.orchestra.run.smithi167.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-08-19T01:23:49.689 INFO:teuthology.orchestra.run.smithi167.stdout: python3-devel-3.9.19-4.el9.x86_64 2024-08-19T01:23:49.689 INFO:teuthology.orchestra.run.smithi167.stdout: python3-google-auth-1:2.33.0-1.el9.noarch 2024-08-19T01:23:49.689 INFO:teuthology.orchestra.run.smithi167.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-08-19T01:23:49.689 INFO:teuthology.orchestra.run.smithi167.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-08-19T01:23:49.689 INFO:teuthology.orchestra.run.smithi167.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-08-19T01:23:49.689 INFO:teuthology.orchestra.run.smithi167.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-08-19T01:23:49.689 INFO:teuthology.orchestra.run.smithi167.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-08-19T01:23:49.690 INFO:teuthology.orchestra.run.smithi167.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-08-19T01:23:49.690 INFO:teuthology.orchestra.run.smithi167.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-08-19T01:23:49.690 INFO:teuthology.orchestra.run.smithi167.stdout: python3-jinja2-2.11.3-6.el9.noarch 2024-08-19T01:23:49.690 INFO:teuthology.orchestra.run.smithi167.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-08-19T01:23:49.690 INFO:teuthology.orchestra.run.smithi167.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-08-19T01:23:49.690 INFO:teuthology.orchestra.run.smithi167.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-08-19T01:23:49.690 INFO:teuthology.orchestra.run.smithi167.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-08-19T01:23:49.690 INFO:teuthology.orchestra.run.smithi167.stdout: python3-mako-1.1.4-6.el9.noarch 2024-08-19T01:23:49.690 INFO:teuthology.orchestra.run.smithi167.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-08-19T01:23:49.690 INFO:teuthology.orchestra.run.smithi167.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-08-19T01:23:49.690 INFO:teuthology.orchestra.run.smithi167.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-08-19T01:23:49.690 INFO:teuthology.orchestra.run.smithi167.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-08-19T01:23:49.690 INFO:teuthology.orchestra.run.smithi167.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-08-19T01:23:49.691 INFO:teuthology.orchestra.run.smithi167.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-08-19T01:23:49.691 INFO:teuthology.orchestra.run.smithi167.stdout: python3-packaging-20.9-5.el9.noarch 2024-08-19T01:23:49.691 INFO:teuthology.orchestra.run.smithi167.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-08-19T01:23:49.691 INFO:teuthology.orchestra.run.smithi167.stdout: python3-ply-3.11-14.el9.noarch 2024-08-19T01:23:49.691 INFO:teuthology.orchestra.run.smithi167.stdout: python3-portend-3.1.0-2.el9.noarch 2024-08-19T01:23:49.691 INFO:teuthology.orchestra.run.smithi167.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-08-19T01:23:49.691 INFO:teuthology.orchestra.run.smithi167.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-08-19T01:23:49.691 INFO:teuthology.orchestra.run.smithi167.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-08-19T01:23:49.691 INFO:teuthology.orchestra.run.smithi167.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-08-19T01:23:49.691 INFO:teuthology.orchestra.run.smithi167.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-08-19T01:23:49.691 INFO:teuthology.orchestra.run.smithi167.stdout: python3-pycparser-2.20-6.el9.noarch 2024-08-19T01:23:49.691 INFO:teuthology.orchestra.run.smithi167.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-08-19T01:23:49.691 INFO:teuthology.orchestra.run.smithi167.stdout: python3-pytz-2021.1-5.el9.noarch 2024-08-19T01:23:49.691 INFO:teuthology.orchestra.run.smithi167.stdout: python3-rados-2:19.1.0-1382.gde09dd01.el9.x86_64 2024-08-19T01:23:49.691 INFO:teuthology.orchestra.run.smithi167.stdout: python3-rbd-2:19.1.0-1382.gde09dd01.el9.x86_64 2024-08-19T01:23:49.692 INFO:teuthology.orchestra.run.smithi167.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-08-19T01:23:49.692 INFO:teuthology.orchestra.run.smithi167.stdout: python3-requests-2.25.1-8.el9.noarch 2024-08-19T01:23:49.692 INFO:teuthology.orchestra.run.smithi167.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-08-19T01:23:49.692 INFO:teuthology.orchestra.run.smithi167.stdout: python3-rgw-2:19.1.0-1382.gde09dd01.el9.x86_64 2024-08-19T01:23:49.692 INFO:teuthology.orchestra.run.smithi167.stdout: python3-routes-2.5.1-5.el9.noarch 2024-08-19T01:23:49.692 INFO:teuthology.orchestra.run.smithi167.stdout: python3-rsa-4.9-2.el9.noarch 2024-08-19T01:23:49.692 INFO:teuthology.orchestra.run.smithi167.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-08-19T01:23:49.692 INFO:teuthology.orchestra.run.smithi167.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-08-19T01:23:49.692 INFO:teuthology.orchestra.run.smithi167.stdout: python3-toml-0.10.2-6.el9.noarch 2024-08-19T01:23:49.692 INFO:teuthology.orchestra.run.smithi167.stdout: python3-typing-extensions-4.12.2-2.el9.noarch 2024-08-19T01:23:49.693 INFO:teuthology.orchestra.run.smithi167.stdout: python3-urllib3-1.26.5-6.el9.noarch 2024-08-19T01:23:49.693 INFO:teuthology.orchestra.run.smithi167.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-08-19T01:23:49.693 INFO:teuthology.orchestra.run.smithi167.stdout: python3-webob-1.8.7-6.el9.noarch 2024-08-19T01:23:49.693 INFO:teuthology.orchestra.run.smithi167.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-08-19T01:23:49.694 INFO:teuthology.orchestra.run.smithi167.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-08-19T01:23:49.694 INFO:teuthology.orchestra.run.smithi167.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-08-19T01:23:49.694 INFO:teuthology.orchestra.run.smithi167.stdout: qatlib-24.02.0-1.el9.x86_64 2024-08-19T01:23:49.694 INFO:teuthology.orchestra.run.smithi167.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-08-19T01:23:49.694 INFO:teuthology.orchestra.run.smithi167.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-08-19T01:23:49.694 INFO:teuthology.orchestra.run.smithi167.stdout: rbd-fuse-2:19.1.0-1382.gde09dd01.el9.x86_64 2024-08-19T01:23:49.694 INFO:teuthology.orchestra.run.smithi167.stdout: rbd-mirror-2:19.1.0-1382.gde09dd01.el9.x86_64 2024-08-19T01:23:49.694 INFO:teuthology.orchestra.run.smithi167.stdout: rbd-nbd-2:19.1.0-1382.gde09dd01.el9.x86_64 2024-08-19T01:23:49.694 INFO:teuthology.orchestra.run.smithi167.stdout: re2-1:20211101-3.el9.x86_64 2024-08-19T01:23:49.694 INFO:teuthology.orchestra.run.smithi167.stdout: socat-1.7.4.1-6.el9.x86_64 2024-08-19T01:23:49.694 INFO:teuthology.orchestra.run.smithi167.stdout: thrift-0.15.0-2.el9.x86_64 2024-08-19T01:23:49.695 INFO:teuthology.orchestra.run.smithi167.stdout: uuid-1.6.2-55.el9.x86_64 2024-08-19T01:23:49.695 INFO:teuthology.orchestra.run.smithi167.stdout: xfsprogs-devel-6.4.0-3.el9.x86_64 2024-08-19T01:23:49.695 INFO:teuthology.orchestra.run.smithi167.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-08-19T01:23:49.695 INFO:teuthology.orchestra.run.smithi167.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-08-19T01:23:49.695 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-08-19T01:23:49.695 INFO:teuthology.orchestra.run.smithi167.stdout:Complete! 2024-08-19T01:23:49.701 INFO:teuthology.orchestra.run.smithi117.stdout: Running scriptlet: cephfs-mirror-2:19.1.0-1382.gde09dd01.el9.x86_64 135/152 2024-08-19T01:23:49.701 INFO:teuthology.orchestra.run.smithi117.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-08-19T01:23:49.701 INFO:teuthology.orchestra.run.smithi117.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-08-19T01:23:49.702 INFO:teuthology.orchestra.run.smithi117.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-08-19T01:23:49.702 INFO:teuthology.orchestra.run.smithi117.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-08-19T01:23:49.702 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-08-19T01:23:50.004 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : rbd-mirror-2:19.1.0-1382.gde09dd01.el9.x86_64 136/152 2024-08-19T01:23:50.027 INFO:teuthology.orchestra.run.smithi117.stdout: Running scriptlet: rbd-mirror-2:19.1.0-1382.gde09dd01.el9.x86_64 136/152 2024-08-19T01:23:50.028 INFO:teuthology.orchestra.run.smithi117.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-08-19T01:23:50.028 INFO:teuthology.orchestra.run.smithi117.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-08-19T01:23:50.028 INFO:teuthology.orchestra.run.smithi117.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-08-19T01:23:50.028 INFO:teuthology.orchestra.run.smithi117.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-08-19T01:23:50.028 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-08-19T01:23:50.041 DEBUG:teuthology.parallel:result is None 2024-08-19T01:23:53.238 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : ceph-test-2:19.1.0-1382.gde09dd01.el9.x86_64 137/152 2024-08-19T01:23:53.313 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : rbd-fuse-2:19.1.0-1382.gde09dd01.el9.x86_64 138/152 2024-08-19T01:23:53.374 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : rbd-nbd-2:19.1.0-1382.gde09dd01.el9.x86_64 139/152 2024-08-19T01:23:53.410 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : xfsprogs-devel-6.4.0-3.el9.x86_64 140/152 2024-08-19T01:23:53.446 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : elfutils-libelf-devel-0.191-4.el9.x86_64 141/152 2024-08-19T01:23:53.685 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : cephfs-shell-2:19.1.0-1382.gde09dd01.el9.x86_64 142/152 2024-08-19T01:23:53.743 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : postgresql-contrib-13.14-1.el9.x86_64 143/152 2024-08-19T01:23:53.771 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : libcephfs-devel-2:19.1.0-1382.gde09dd01.el9.x86_ 144/152 2024-08-19T01:23:54.549 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : ceph-test-2:19.1.0-1382.gde09dd01.el9.x86_64 137/152 2024-08-19T01:23:54.590 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : rbd-fuse-2:19.1.0-1382.gde09dd01.el9.x86_64 138/152 2024-08-19T01:23:54.607 INFO:teuthology.orchestra.run.smithi083.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 145/152 2024-08-19T01:23:54.631 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : rbd-nbd-2:19.1.0-1382.gde09dd01.el9.x86_64 139/152 2024-08-19T01:23:54.653 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : postgresql-server-13.14-1.el9.x86_64 145/152 2024-08-19T01:23:54.663 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : xfsprogs-devel-6.4.0-3.el9.x86_64 140/152 2024-08-19T01:23:54.682 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : elfutils-libelf-devel-0.191-4.el9.x86_64 141/152 2024-08-19T01:23:54.694 INFO:teuthology.orchestra.run.smithi083.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 145/152 2024-08-19T01:23:54.862 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : cephfs-shell-2:19.1.0-1382.gde09dd01.el9.x86_64 142/152 2024-08-19T01:23:54.903 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : postgresql-contrib-13.14-1.el9.x86_64 143/152 2024-08-19T01:23:54.933 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : libcephfs-devel-2:19.1.0-1382.gde09dd01.el9.x86_ 144/152 2024-08-19T01:23:55.447 INFO:teuthology.orchestra.run.smithi117.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 145/152 2024-08-19T01:23:55.489 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : postgresql-server-13.14-1.el9.x86_64 145/152 2024-08-19T01:23:55.509 INFO:teuthology.orchestra.run.smithi117.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 145/152 2024-08-19T01:23:55.951 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : cephfs-top-2:19.1.0-1382.gde09dd01.el9.noarch 146/152 2024-08-19T01:23:56.067 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : openssl-devel-1:3.2.2-2.el9.x86_64 147/152 2024-08-19T01:23:56.150 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 148/152 2024-08-19T01:23:56.266 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : flex-2.6.4-9.el9.x86_64 149/152 2024-08-19T01:23:56.307 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : bison-3.7.4-5.el9.x86_64 150/152 2024-08-19T01:23:56.307 INFO:teuthology.orchestra.run.smithi083.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 151/152 2024-08-19T01:23:56.323 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : cephfs-top-2:19.1.0-1382.gde09dd01.el9.noarch 146/152 2024-08-19T01:23:56.418 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : openssl-devel-1:3.2.2-2.el9.x86_64 147/152 2024-08-19T01:23:56.422 INFO:teuthology.orchestra.run.smithi083.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 151/152 2024-08-19T01:23:56.422 INFO:teuthology.orchestra.run.smithi083.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 152/152 2024-08-19T01:23:56.459 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 148/152 2024-08-19T01:23:56.540 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : flex-2.6.4-9.el9.x86_64 149/152 2024-08-19T01:23:56.572 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : bison-3.7.4-5.el9.x86_64 150/152 2024-08-19T01:23:56.573 INFO:teuthology.orchestra.run.smithi117.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 151/152 2024-08-19T01:23:56.656 INFO:teuthology.orchestra.run.smithi117.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 151/152 2024-08-19T01:23:56.656 INFO:teuthology.orchestra.run.smithi117.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 152/152 2024-08-19T01:23:56.854 INFO:teuthology.orchestra.run.smithi083.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 152/152 2024-08-19T01:23:56.854 INFO:teuthology.orchestra.run.smithi083.stdout:Couldn't write '0' to 'kernel/yama/ptrace_scope', ignoring: No such file or directory 2024-08-19T01:23:56.854 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-08-19T01:24:02.342 INFO:teuthology.orchestra.run.smithi117.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 152/152 2024-08-19T01:24:02.342 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : ceph-2:19.1.0-1382.gde09dd01.el9.x86_64 1/152 2024-08-19T01:24:02.342 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : ceph-base-2:19.1.0-1382.gde09dd01.el9.x86_64 2/152 2024-08-19T01:24:02.342 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : ceph-common-2:19.1.0-1382.gde09dd01.el9.x86_64 3/152 2024-08-19T01:24:02.342 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : ceph-fuse-2:19.1.0-1382.gde09dd01.el9.x86_64 4/152 2024-08-19T01:24:02.342 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : ceph-immutable-object-cache-2:19.1.0-1382.gde09d 5/152 2024-08-19T01:24:02.342 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : ceph-mds-2:19.1.0-1382.gde09dd01.el9.x86_64 6/152 2024-08-19T01:24:02.342 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : ceph-mgr-2:19.1.0-1382.gde09dd01.el9.x86_64 7/152 2024-08-19T01:24:02.342 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : ceph-mon-2:19.1.0-1382.gde09dd01.el9.x86_64 8/152 2024-08-19T01:24:02.343 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : ceph-osd-2:19.1.0-1382.gde09dd01.el9.x86_64 9/152 2024-08-19T01:24:02.343 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : ceph-radosgw-2:19.1.0-1382.gde09dd01.el9.x86_64 10/152 2024-08-19T01:24:02.343 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : ceph-selinux-2:19.1.0-1382.gde09dd01.el9.x86_64 11/152 2024-08-19T01:24:02.343 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : ceph-test-2:19.1.0-1382.gde09dd01.el9.x86_64 12/152 2024-08-19T01:24:02.343 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : cephfs-mirror-2:19.1.0-1382.gde09dd01.el9.x86_64 13/152 2024-08-19T01:24:02.343 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : cephfs-shell-2:19.1.0-1382.gde09dd01.el9.x86_64 14/152 2024-08-19T01:24:02.343 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : libcephfs-devel-2:19.1.0-1382.gde09dd01.el9.x86_ 15/152 2024-08-19T01:24:02.343 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : libcephfs2-2:19.1.0-1382.gde09dd01.el9.x86_64 16/152 2024-08-19T01:24:02.343 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : libcephsqlite-2:19.1.0-1382.gde09dd01.el9.x86_64 17/152 2024-08-19T01:24:02.343 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : librados-devel-2:19.1.0-1382.gde09dd01.el9.x86_6 18/152 2024-08-19T01:24:02.343 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : libradosstriper1-2:19.1.0-1382.gde09dd01.el9.x86 19/152 2024-08-19T01:24:02.343 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : librgw2-2:19.1.0-1382.gde09dd01.el9.x86_64 20/152 2024-08-19T01:24:02.343 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-ceph-argparse-2:19.1.0-1382.gde09dd01.el 21/152 2024-08-19T01:24:02.344 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-ceph-common-2:19.1.0-1382.gde09dd01.el9. 22/152 2024-08-19T01:24:02.344 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-cephfs-2:19.1.0-1382.gde09dd01.el9.x86_6 23/152 2024-08-19T01:24:02.344 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-rados-2:19.1.0-1382.gde09dd01.el9.x86_64 24/152 2024-08-19T01:24:02.344 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-rbd-2:19.1.0-1382.gde09dd01.el9.x86_64 25/152 2024-08-19T01:24:02.344 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-rgw-2:19.1.0-1382.gde09dd01.el9.x86_64 26/152 2024-08-19T01:24:02.344 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : rbd-fuse-2:19.1.0-1382.gde09dd01.el9.x86_64 27/152 2024-08-19T01:24:02.344 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : rbd-mirror-2:19.1.0-1382.gde09dd01.el9.x86_64 28/152 2024-08-19T01:24:02.344 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : rbd-nbd-2:19.1.0-1382.gde09dd01.el9.x86_64 29/152 2024-08-19T01:24:02.344 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : ceph-grafana-dashboards-2:19.1.0-1382.gde09dd01. 30/152 2024-08-19T01:24:02.344 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : ceph-mgr-cephadm-2:19.1.0-1382.gde09dd01.el9.noa 31/152 2024-08-19T01:24:02.344 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : ceph-mgr-dashboard-2:19.1.0-1382.gde09dd01.el9.n 32/152 2024-08-19T01:24:02.344 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.1.0-1382.gde0 33/152 2024-08-19T01:24:02.344 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : ceph-mgr-modules-core-2:19.1.0-1382.gde09dd01.el 34/152 2024-08-19T01:24:02.345 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : ceph-mgr-rook-2:19.1.0-1382.gde09dd01.el9.noarch 35/152 2024-08-19T01:24:02.345 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : ceph-prometheus-alerts-2:19.1.0-1382.gde09dd01.e 36/152 2024-08-19T01:24:02.345 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : ceph-volume-2:19.1.0-1382.gde09dd01.el9.noarch 37/152 2024-08-19T01:24:02.345 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : cephadm-2:19.1.0-1382.gde09dd01.el9.noarch 38/152 2024-08-19T01:24:02.345 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : cephfs-top-2:19.1.0-1382.gde09dd01.el9.noarch 39/152 2024-08-19T01:24:02.345 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : fuse-2.9.9-16.el9.x86_64 40/152 2024-08-19T01:24:02.345 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 41/152 2024-08-19T01:24:02.345 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 42/152 2024-08-19T01:24:02.345 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/152 2024-08-19T01:24:02.345 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/152 2024-08-19T01:24:02.345 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/152 2024-08-19T01:24:02.345 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/152 2024-08-19T01:24:02.345 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/152 2024-08-19T01:24:02.346 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-urllib3-1.26.5-6.el9.noarch 48/152 2024-08-19T01:24:02.346 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/152 2024-08-19T01:24:02.346 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/152 2024-08-19T01:24:02.346 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/152 2024-08-19T01:24:02.346 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : elfutils-libelf-devel-0.191-4.el9.x86_64 52/152 2024-08-19T01:24:02.346 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/152 2024-08-19T01:24:02.346 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/152 2024-08-19T01:24:02.346 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/152 2024-08-19T01:24:02.346 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/152 2024-08-19T01:24:02.346 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/152 2024-08-19T01:24:02.346 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/152 2024-08-19T01:24:02.346 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/152 2024-08-19T01:24:02.347 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/152 2024-08-19T01:24:02.347 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : libtool-2.4.6-46.el9.x86_64 61/152 2024-08-19T01:24:02.347 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 62/152 2024-08-19T01:24:02.347 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/152 2024-08-19T01:24:02.347 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/152 2024-08-19T01:24:02.347 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/152 2024-08-19T01:24:02.347 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 66/152 2024-08-19T01:24:02.347 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 67/152 2024-08-19T01:24:02.347 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : openssl-devel-1:3.2.2-2.el9.x86_64 68/152 2024-08-19T01:24:02.347 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : postgresql-13.14-1.el9.x86_64 69/152 2024-08-19T01:24:02.347 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : postgresql-contrib-13.14-1.el9.x86_64 70/152 2024-08-19T01:24:02.347 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : postgresql-private-libs-13.14-1.el9.x86_64 71/152 2024-08-19T01:24:02.347 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : postgresql-server-13.14-1.el9.x86_64 72/152 2024-08-19T01:24:02.348 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/152 2024-08-19T01:24:02.348 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/152 2024-08-19T01:24:02.348 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/152 2024-08-19T01:24:02.348 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-colorama-0.4.6-3.el9.noarch 76/152 2024-08-19T01:24:02.348 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-devel-3.9.19-4.el9.x86_64 77/152 2024-08-19T01:24:02.348 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-jinja2-2.11.3-6.el9.noarch 78/152 2024-08-19T01:24:02.348 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 79/152 2024-08-19T01:24:02.348 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 80/152 2024-08-19T01:24:02.348 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 81/152 2024-08-19T01:24:02.348 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 82/152 2024-08-19T01:24:02.348 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 83/152 2024-08-19T01:24:02.348 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 84/152 2024-08-19T01:24:02.348 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 85/152 2024-08-19T01:24:02.348 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 86/152 2024-08-19T01:24:02.349 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 87/152 2024-08-19T01:24:02.349 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 88/152 2024-08-19T01:24:02.349 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 89/152 2024-08-19T01:24:02.349 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 90/152 2024-08-19T01:24:02.349 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 91/152 2024-08-19T01:24:02.349 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 92/152 2024-08-19T01:24:02.349 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 93/152 2024-08-19T01:24:02.349 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 94/152 2024-08-19T01:24:02.349 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 95/152 2024-08-19T01:24:02.349 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 96/152 2024-08-19T01:24:02.349 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 97/152 2024-08-19T01:24:02.349 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 98/152 2024-08-19T01:24:02.349 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : socat-1.7.4.1-6.el9.x86_64 99/152 2024-08-19T01:24:02.350 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 100/152 2024-08-19T01:24:02.350 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : xfsprogs-devel-6.4.0-3.el9.x86_64 101/152 2024-08-19T01:24:02.350 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 102/152 2024-08-19T01:24:02.350 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 103/152 2024-08-19T01:24:02.350 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 104/152 2024-08-19T01:24:02.350 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 105/152 2024-08-19T01:24:02.350 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 106/152 2024-08-19T01:24:02.350 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 107/152 2024-08-19T01:24:02.350 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 108/152 2024-08-19T01:24:02.350 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 109/152 2024-08-19T01:24:02.350 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 110/152 2024-08-19T01:24:02.350 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 111/152 2024-08-19T01:24:02.350 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 112/152 2024-08-19T01:24:02.351 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : luarocks-3.9.2-5.el9.noarch 113/152 2024-08-19T01:24:02.351 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 114/152 2024-08-19T01:24:02.351 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 115/152 2024-08-19T01:24:02.351 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 116/152 2024-08-19T01:24:02.351 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 117/152 2024-08-19T01:24:02.351 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-certifi-2023.05.07-4.el9.noarch 118/152 2024-08-19T01:24:02.351 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 119/152 2024-08-19T01:24:02.351 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 120/152 2024-08-19T01:24:02.351 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 121/152 2024-08-19T01:24:02.351 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-google-auth-1:2.33.0-1.el9.noarch 122/152 2024-08-19T01:24:02.351 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/152 2024-08-19T01:24:02.351 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/152 2024-08-19T01:24:02.351 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/152 2024-08-19T01:24:02.352 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/152 2024-08-19T01:24:02.352 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/152 2024-08-19T01:24:02.352 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/152 2024-08-19T01:24:02.352 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/152 2024-08-19T01:24:02.352 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/152 2024-08-19T01:24:02.352 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/152 2024-08-19T01:24:02.352 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/152 2024-08-19T01:24:02.352 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/152 2024-08-19T01:24:02.352 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/152 2024-08-19T01:24:02.352 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/152 2024-08-19T01:24:02.352 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/152 2024-08-19T01:24:02.352 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/152 2024-08-19T01:24:02.352 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/152 2024-08-19T01:24:02.352 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/152 2024-08-19T01:24:02.353 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/152 2024-08-19T01:24:02.353 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/152 2024-08-19T01:24:02.353 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-typing-extensions-4.12.2-2.el9.noarch 142/152 2024-08-19T01:24:02.353 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 143/152 2024-08-19T01:24:02.353 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/152 2024-08-19T01:24:02.353 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/152 2024-08-19T01:24:02.353 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/152 2024-08-19T01:24:02.353 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : re2-1:20211101-3.el9.x86_64 147/152 2024-08-19T01:24:02.353 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/152 2024-08-19T01:24:02.353 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : librados2-2:19.1.0-1382.gde09dd01.el9.x86_64 149/152 2024-08-19T01:24:02.353 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/152 2024-08-19T01:24:02.353 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : librbd1-2:19.1.0-1382.gde09dd01.el9.x86_64 151/152 2024-08-19T01:24:02.827 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : ceph-2:19.1.0-1382.gde09dd01.el9.x86_64 1/152 2024-08-19T01:24:02.827 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : ceph-base-2:19.1.0-1382.gde09dd01.el9.x86_64 2/152 2024-08-19T01:24:02.827 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : ceph-common-2:19.1.0-1382.gde09dd01.el9.x86_64 3/152 2024-08-19T01:24:02.827 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : ceph-fuse-2:19.1.0-1382.gde09dd01.el9.x86_64 4/152 2024-08-19T01:24:02.827 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : ceph-immutable-object-cache-2:19.1.0-1382.gde09d 5/152 2024-08-19T01:24:02.827 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : ceph-mds-2:19.1.0-1382.gde09dd01.el9.x86_64 6/152 2024-08-19T01:24:02.827 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : ceph-mgr-2:19.1.0-1382.gde09dd01.el9.x86_64 7/152 2024-08-19T01:24:02.827 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : ceph-mon-2:19.1.0-1382.gde09dd01.el9.x86_64 8/152 2024-08-19T01:24:02.827 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : ceph-osd-2:19.1.0-1382.gde09dd01.el9.x86_64 9/152 2024-08-19T01:24:02.827 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : ceph-radosgw-2:19.1.0-1382.gde09dd01.el9.x86_64 10/152 2024-08-19T01:24:02.827 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : ceph-selinux-2:19.1.0-1382.gde09dd01.el9.x86_64 11/152 2024-08-19T01:24:02.828 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : ceph-test-2:19.1.0-1382.gde09dd01.el9.x86_64 12/152 2024-08-19T01:24:02.828 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : cephfs-mirror-2:19.1.0-1382.gde09dd01.el9.x86_64 13/152 2024-08-19T01:24:02.828 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : cephfs-shell-2:19.1.0-1382.gde09dd01.el9.x86_64 14/152 2024-08-19T01:24:02.828 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : libcephfs-devel-2:19.1.0-1382.gde09dd01.el9.x86_ 15/152 2024-08-19T01:24:02.828 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : libcephfs2-2:19.1.0-1382.gde09dd01.el9.x86_64 16/152 2024-08-19T01:24:02.828 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : libcephsqlite-2:19.1.0-1382.gde09dd01.el9.x86_64 17/152 2024-08-19T01:24:02.828 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : librados-devel-2:19.1.0-1382.gde09dd01.el9.x86_6 18/152 2024-08-19T01:24:02.828 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : libradosstriper1-2:19.1.0-1382.gde09dd01.el9.x86 19/152 2024-08-19T01:24:02.828 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : librgw2-2:19.1.0-1382.gde09dd01.el9.x86_64 20/152 2024-08-19T01:24:02.828 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-ceph-argparse-2:19.1.0-1382.gde09dd01.el 21/152 2024-08-19T01:24:02.828 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-ceph-common-2:19.1.0-1382.gde09dd01.el9. 22/152 2024-08-19T01:24:02.828 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-cephfs-2:19.1.0-1382.gde09dd01.el9.x86_6 23/152 2024-08-19T01:24:02.828 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-rados-2:19.1.0-1382.gde09dd01.el9.x86_64 24/152 2024-08-19T01:24:02.828 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-rbd-2:19.1.0-1382.gde09dd01.el9.x86_64 25/152 2024-08-19T01:24:02.829 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-rgw-2:19.1.0-1382.gde09dd01.el9.x86_64 26/152 2024-08-19T01:24:02.829 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : rbd-fuse-2:19.1.0-1382.gde09dd01.el9.x86_64 27/152 2024-08-19T01:24:02.839 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : rbd-mirror-2:19.1.0-1382.gde09dd01.el9.x86_64 28/152 2024-08-19T01:24:02.840 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : rbd-nbd-2:19.1.0-1382.gde09dd01.el9.x86_64 29/152 2024-08-19T01:24:02.840 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : ceph-grafana-dashboards-2:19.1.0-1382.gde09dd01. 30/152 2024-08-19T01:24:02.840 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : ceph-mgr-cephadm-2:19.1.0-1382.gde09dd01.el9.noa 31/152 2024-08-19T01:24:02.840 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : ceph-mgr-dashboard-2:19.1.0-1382.gde09dd01.el9.n 32/152 2024-08-19T01:24:02.840 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.1.0-1382.gde0 33/152 2024-08-19T01:24:02.840 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : ceph-mgr-modules-core-2:19.1.0-1382.gde09dd01.el 34/152 2024-08-19T01:24:02.840 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : ceph-mgr-rook-2:19.1.0-1382.gde09dd01.el9.noarch 35/152 2024-08-19T01:24:02.840 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : ceph-prometheus-alerts-2:19.1.0-1382.gde09dd01.e 36/152 2024-08-19T01:24:02.840 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : ceph-volume-2:19.1.0-1382.gde09dd01.el9.noarch 37/152 2024-08-19T01:24:02.840 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : cephadm-2:19.1.0-1382.gde09dd01.el9.noarch 38/152 2024-08-19T01:24:02.840 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : cephfs-top-2:19.1.0-1382.gde09dd01.el9.noarch 39/152 2024-08-19T01:24:02.840 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : fuse-2.9.9-16.el9.x86_64 40/152 2024-08-19T01:24:02.840 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 41/152 2024-08-19T01:24:02.841 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 42/152 2024-08-19T01:24:02.841 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/152 2024-08-19T01:24:02.841 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/152 2024-08-19T01:24:02.841 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/152 2024-08-19T01:24:02.841 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/152 2024-08-19T01:24:02.841 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/152 2024-08-19T01:24:02.841 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-urllib3-1.26.5-6.el9.noarch 48/152 2024-08-19T01:24:02.841 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/152 2024-08-19T01:24:02.842 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/152 2024-08-19T01:24:02.842 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/152 2024-08-19T01:24:02.842 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : elfutils-libelf-devel-0.191-4.el9.x86_64 52/152 2024-08-19T01:24:02.842 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/152 2024-08-19T01:24:02.842 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/152 2024-08-19T01:24:02.842 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/152 2024-08-19T01:24:02.842 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/152 2024-08-19T01:24:02.842 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/152 2024-08-19T01:24:02.842 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/152 2024-08-19T01:24:02.842 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/152 2024-08-19T01:24:02.842 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/152 2024-08-19T01:24:02.842 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : libtool-2.4.6-46.el9.x86_64 61/152 2024-08-19T01:24:02.843 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 62/152 2024-08-19T01:24:02.843 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/152 2024-08-19T01:24:02.843 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/152 2024-08-19T01:24:02.843 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/152 2024-08-19T01:24:02.843 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 66/152 2024-08-19T01:24:02.843 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 67/152 2024-08-19T01:24:02.843 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : openssl-devel-1:3.2.2-2.el9.x86_64 68/152 2024-08-19T01:24:02.843 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : postgresql-13.14-1.el9.x86_64 69/152 2024-08-19T01:24:02.843 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : postgresql-contrib-13.14-1.el9.x86_64 70/152 2024-08-19T01:24:02.843 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : postgresql-private-libs-13.14-1.el9.x86_64 71/152 2024-08-19T01:24:02.843 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : postgresql-server-13.14-1.el9.x86_64 72/152 2024-08-19T01:24:02.843 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/152 2024-08-19T01:24:02.843 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/152 2024-08-19T01:24:02.844 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/152 2024-08-19T01:24:02.844 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-colorama-0.4.6-3.el9.noarch 76/152 2024-08-19T01:24:02.844 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-devel-3.9.19-4.el9.x86_64 77/152 2024-08-19T01:24:02.844 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-jinja2-2.11.3-6.el9.noarch 78/152 2024-08-19T01:24:02.844 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 79/152 2024-08-19T01:24:02.844 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 80/152 2024-08-19T01:24:02.844 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 81/152 2024-08-19T01:24:02.844 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 82/152 2024-08-19T01:24:02.844 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 83/152 2024-08-19T01:24:02.844 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 84/152 2024-08-19T01:24:02.844 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 85/152 2024-08-19T01:24:02.844 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 86/152 2024-08-19T01:24:02.844 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 87/152 2024-08-19T01:24:02.845 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 88/152 2024-08-19T01:24:02.845 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 89/152 2024-08-19T01:24:02.845 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 90/152 2024-08-19T01:24:02.845 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 91/152 2024-08-19T01:24:02.845 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 92/152 2024-08-19T01:24:02.845 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 93/152 2024-08-19T01:24:02.845 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 94/152 2024-08-19T01:24:02.845 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 95/152 2024-08-19T01:24:02.845 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 96/152 2024-08-19T01:24:02.845 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 97/152 2024-08-19T01:24:02.845 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 98/152 2024-08-19T01:24:02.845 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : socat-1.7.4.1-6.el9.x86_64 99/152 2024-08-19T01:24:02.845 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 100/152 2024-08-19T01:24:02.846 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : xfsprogs-devel-6.4.0-3.el9.x86_64 101/152 2024-08-19T01:24:02.846 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 102/152 2024-08-19T01:24:02.846 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 103/152 2024-08-19T01:24:02.846 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 104/152 2024-08-19T01:24:02.846 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 105/152 2024-08-19T01:24:02.846 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 106/152 2024-08-19T01:24:02.846 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 107/152 2024-08-19T01:24:02.846 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 108/152 2024-08-19T01:24:02.846 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 109/152 2024-08-19T01:24:02.846 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 110/152 2024-08-19T01:24:02.846 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 111/152 2024-08-19T01:24:02.846 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 112/152 2024-08-19T01:24:02.847 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : luarocks-3.9.2-5.el9.noarch 113/152 2024-08-19T01:24:02.847 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 114/152 2024-08-19T01:24:02.847 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 115/152 2024-08-19T01:24:02.847 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 116/152 2024-08-19T01:24:02.847 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 117/152 2024-08-19T01:24:02.847 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-certifi-2023.05.07-4.el9.noarch 118/152 2024-08-19T01:24:02.847 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 119/152 2024-08-19T01:24:02.847 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 120/152 2024-08-19T01:24:02.847 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 121/152 2024-08-19T01:24:02.847 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-google-auth-1:2.33.0-1.el9.noarch 122/152 2024-08-19T01:24:02.848 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/152 2024-08-19T01:24:02.848 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/152 2024-08-19T01:24:02.848 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/152 2024-08-19T01:24:02.848 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/152 2024-08-19T01:24:02.848 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/152 2024-08-19T01:24:02.848 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/152 2024-08-19T01:24:02.848 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/152 2024-08-19T01:24:02.848 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/152 2024-08-19T01:24:02.848 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/152 2024-08-19T01:24:02.848 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/152 2024-08-19T01:24:02.848 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/152 2024-08-19T01:24:02.848 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/152 2024-08-19T01:24:02.849 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/152 2024-08-19T01:24:02.849 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/152 2024-08-19T01:24:02.849 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/152 2024-08-19T01:24:02.849 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/152 2024-08-19T01:24:02.849 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/152 2024-08-19T01:24:02.849 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/152 2024-08-19T01:24:02.849 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/152 2024-08-19T01:24:02.849 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-typing-extensions-4.12.2-2.el9.noarch 142/152 2024-08-19T01:24:02.849 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 143/152 2024-08-19T01:24:02.849 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/152 2024-08-19T01:24:02.849 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/152 2024-08-19T01:24:02.849 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/152 2024-08-19T01:24:02.849 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : re2-1:20211101-3.el9.x86_64 147/152 2024-08-19T01:24:02.849 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/152 2024-08-19T01:24:02.850 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : librados2-2:19.1.0-1382.gde09dd01.el9.x86_64 149/152 2024-08-19T01:24:02.850 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/152 2024-08-19T01:24:02.850 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : librbd1-2:19.1.0-1382.gde09dd01.el9.x86_64 151/152 2024-08-19T01:24:04.499 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/152 2024-08-19T01:24:04.499 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-08-19T01:24:04.499 INFO:teuthology.orchestra.run.smithi117.stdout:Upgraded: 2024-08-19T01:24:04.499 INFO:teuthology.orchestra.run.smithi117.stdout: librados2-2:19.1.0-1382.gde09dd01.el9.x86_64 2024-08-19T01:24:04.499 INFO:teuthology.orchestra.run.smithi117.stdout: librbd1-2:19.1.0-1382.gde09dd01.el9.x86_64 2024-08-19T01:24:04.499 INFO:teuthology.orchestra.run.smithi117.stdout:Installed: 2024-08-19T01:24:04.499 INFO:teuthology.orchestra.run.smithi117.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-08-19T01:24:04.499 INFO:teuthology.orchestra.run.smithi117.stdout: automake-1.16.2-8.el9.noarch 2024-08-19T01:24:04.500 INFO:teuthology.orchestra.run.smithi117.stdout: bison-3.7.4-5.el9.x86_64 2024-08-19T01:24:04.500 INFO:teuthology.orchestra.run.smithi117.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-08-19T01:24:04.500 INFO:teuthology.orchestra.run.smithi117.stdout: ceph-2:19.1.0-1382.gde09dd01.el9.x86_64 2024-08-19T01:24:04.500 INFO:teuthology.orchestra.run.smithi117.stdout: ceph-base-2:19.1.0-1382.gde09dd01.el9.x86_64 2024-08-19T01:24:04.500 INFO:teuthology.orchestra.run.smithi117.stdout: ceph-common-2:19.1.0-1382.gde09dd01.el9.x86_64 2024-08-19T01:24:04.500 INFO:teuthology.orchestra.run.smithi117.stdout: ceph-fuse-2:19.1.0-1382.gde09dd01.el9.x86_64 2024-08-19T01:24:04.500 INFO:teuthology.orchestra.run.smithi117.stdout: ceph-grafana-dashboards-2:19.1.0-1382.gde09dd01.el9.noarch 2024-08-19T01:24:04.500 INFO:teuthology.orchestra.run.smithi117.stdout: ceph-immutable-object-cache-2:19.1.0-1382.gde09dd01.el9.x86_64 2024-08-19T01:24:04.500 INFO:teuthology.orchestra.run.smithi117.stdout: ceph-mds-2:19.1.0-1382.gde09dd01.el9.x86_64 2024-08-19T01:24:04.500 INFO:teuthology.orchestra.run.smithi117.stdout: ceph-mgr-2:19.1.0-1382.gde09dd01.el9.x86_64 2024-08-19T01:24:04.500 INFO:teuthology.orchestra.run.smithi117.stdout: ceph-mgr-cephadm-2:19.1.0-1382.gde09dd01.el9.noarch 2024-08-19T01:24:04.500 INFO:teuthology.orchestra.run.smithi117.stdout: ceph-mgr-dashboard-2:19.1.0-1382.gde09dd01.el9.noarch 2024-08-19T01:24:04.500 INFO:teuthology.orchestra.run.smithi117.stdout: ceph-mgr-diskprediction-local-2:19.1.0-1382.gde09dd01.el9.noarch 2024-08-19T01:24:04.501 INFO:teuthology.orchestra.run.smithi117.stdout: ceph-mgr-modules-core-2:19.1.0-1382.gde09dd01.el9.noarch 2024-08-19T01:24:04.501 INFO:teuthology.orchestra.run.smithi117.stdout: ceph-mgr-rook-2:19.1.0-1382.gde09dd01.el9.noarch 2024-08-19T01:24:04.501 INFO:teuthology.orchestra.run.smithi117.stdout: ceph-mon-2:19.1.0-1382.gde09dd01.el9.x86_64 2024-08-19T01:24:04.501 INFO:teuthology.orchestra.run.smithi117.stdout: ceph-osd-2:19.1.0-1382.gde09dd01.el9.x86_64 2024-08-19T01:24:04.501 INFO:teuthology.orchestra.run.smithi117.stdout: ceph-prometheus-alerts-2:19.1.0-1382.gde09dd01.el9.noarch 2024-08-19T01:24:04.501 INFO:teuthology.orchestra.run.smithi117.stdout: ceph-radosgw-2:19.1.0-1382.gde09dd01.el9.x86_64 2024-08-19T01:24:04.501 INFO:teuthology.orchestra.run.smithi117.stdout: ceph-selinux-2:19.1.0-1382.gde09dd01.el9.x86_64 2024-08-19T01:24:04.501 INFO:teuthology.orchestra.run.smithi117.stdout: ceph-test-2:19.1.0-1382.gde09dd01.el9.x86_64 2024-08-19T01:24:04.501 INFO:teuthology.orchestra.run.smithi117.stdout: ceph-volume-2:19.1.0-1382.gde09dd01.el9.noarch 2024-08-19T01:24:04.501 INFO:teuthology.orchestra.run.smithi117.stdout: cephadm-2:19.1.0-1382.gde09dd01.el9.noarch 2024-08-19T01:24:04.501 INFO:teuthology.orchestra.run.smithi117.stdout: cephfs-mirror-2:19.1.0-1382.gde09dd01.el9.x86_64 2024-08-19T01:24:04.501 INFO:teuthology.orchestra.run.smithi117.stdout: cephfs-shell-2:19.1.0-1382.gde09dd01.el9.x86_64 2024-08-19T01:24:04.501 INFO:teuthology.orchestra.run.smithi117.stdout: cephfs-top-2:19.1.0-1382.gde09dd01.el9.noarch 2024-08-19T01:24:04.502 INFO:teuthology.orchestra.run.smithi117.stdout: elfutils-libelf-devel-0.191-4.el9.x86_64 2024-08-19T01:24:04.502 INFO:teuthology.orchestra.run.smithi117.stdout: flex-2.6.4-9.el9.x86_64 2024-08-19T01:24:04.502 INFO:teuthology.orchestra.run.smithi117.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-08-19T01:24:04.502 INFO:teuthology.orchestra.run.smithi117.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-08-19T01:24:04.502 INFO:teuthology.orchestra.run.smithi117.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-08-19T01:24:04.502 INFO:teuthology.orchestra.run.smithi117.stdout: fuse-2.9.9-16.el9.x86_64 2024-08-19T01:24:04.502 INFO:teuthology.orchestra.run.smithi117.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-08-19T01:24:04.502 INFO:teuthology.orchestra.run.smithi117.stdout: grpc-data-1.46.7-10.el9.noarch 2024-08-19T01:24:04.502 INFO:teuthology.orchestra.run.smithi117.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-08-19T01:24:04.502 INFO:teuthology.orchestra.run.smithi117.stdout: libarrow-9.0.0-11.el9.x86_64 2024-08-19T01:24:04.502 INFO:teuthology.orchestra.run.smithi117.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-08-19T01:24:04.502 INFO:teuthology.orchestra.run.smithi117.stdout: libcephfs-devel-2:19.1.0-1382.gde09dd01.el9.x86_64 2024-08-19T01:24:04.502 INFO:teuthology.orchestra.run.smithi117.stdout: libcephfs2-2:19.1.0-1382.gde09dd01.el9.x86_64 2024-08-19T01:24:04.503 INFO:teuthology.orchestra.run.smithi117.stdout: libcephsqlite-2:19.1.0-1382.gde09dd01.el9.x86_64 2024-08-19T01:24:04.503 INFO:teuthology.orchestra.run.smithi117.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-08-19T01:24:04.503 INFO:teuthology.orchestra.run.smithi117.stdout: liboath-2.6.7-2.el9.x86_64 2024-08-19T01:24:04.503 INFO:teuthology.orchestra.run.smithi117.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-08-19T01:24:04.503 INFO:teuthology.orchestra.run.smithi117.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-08-19T01:24:04.503 INFO:teuthology.orchestra.run.smithi117.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-08-19T01:24:04.503 INFO:teuthology.orchestra.run.smithi117.stdout: librados-devel-2:19.1.0-1382.gde09dd01.el9.x86_64 2024-08-19T01:24:04.503 INFO:teuthology.orchestra.run.smithi117.stdout: libradosstriper1-2:19.1.0-1382.gde09dd01.el9.x86_64 2024-08-19T01:24:04.503 INFO:teuthology.orchestra.run.smithi117.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-08-19T01:24:04.503 INFO:teuthology.orchestra.run.smithi117.stdout: librgw2-2:19.1.0-1382.gde09dd01.el9.x86_64 2024-08-19T01:24:04.503 INFO:teuthology.orchestra.run.smithi117.stdout: libtool-2.4.6-46.el9.x86_64 2024-08-19T01:24:04.503 INFO:teuthology.orchestra.run.smithi117.stdout: libunwind-1.6.2-1.el9.x86_64 2024-08-19T01:24:04.503 INFO:teuthology.orchestra.run.smithi117.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-08-19T01:24:04.503 INFO:teuthology.orchestra.run.smithi117.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-08-19T01:24:04.504 INFO:teuthology.orchestra.run.smithi117.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-08-19T01:24:04.504 INFO:teuthology.orchestra.run.smithi117.stdout: lua-5.4.4-4.el9.x86_64 2024-08-19T01:24:04.504 INFO:teuthology.orchestra.run.smithi117.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-08-19T01:24:04.504 INFO:teuthology.orchestra.run.smithi117.stdout: luarocks-3.9.2-5.el9.noarch 2024-08-19T01:24:04.504 INFO:teuthology.orchestra.run.smithi117.stdout: openblas-0.3.21-2.el9.x86_64 2024-08-19T01:24:04.504 INFO:teuthology.orchestra.run.smithi117.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-08-19T01:24:04.504 INFO:teuthology.orchestra.run.smithi117.stdout: openssl-devel-1:3.2.2-2.el9.x86_64 2024-08-19T01:24:04.504 INFO:teuthology.orchestra.run.smithi117.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-08-19T01:24:04.504 INFO:teuthology.orchestra.run.smithi117.stdout: postgresql-13.14-1.el9.x86_64 2024-08-19T01:24:04.504 INFO:teuthology.orchestra.run.smithi117.stdout: postgresql-contrib-13.14-1.el9.x86_64 2024-08-19T01:24:04.504 INFO:teuthology.orchestra.run.smithi117.stdout: postgresql-private-libs-13.14-1.el9.x86_64 2024-08-19T01:24:04.504 INFO:teuthology.orchestra.run.smithi117.stdout: postgresql-server-13.14-1.el9.x86_64 2024-08-19T01:24:04.504 INFO:teuthology.orchestra.run.smithi117.stdout: protobuf-3.14.0-13.el9.x86_64 2024-08-19T01:24:04.504 INFO:teuthology.orchestra.run.smithi117.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-08-19T01:24:04.505 INFO:teuthology.orchestra.run.smithi117.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-08-19T01:24:04.505 INFO:teuthology.orchestra.run.smithi117.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-08-19T01:24:04.505 INFO:teuthology.orchestra.run.smithi117.stdout: python3-babel-2.9.1-2.el9.noarch 2024-08-19T01:24:04.505 INFO:teuthology.orchestra.run.smithi117.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-08-19T01:24:04.505 INFO:teuthology.orchestra.run.smithi117.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-08-19T01:24:04.505 INFO:teuthology.orchestra.run.smithi117.stdout: python3-ceph-argparse-2:19.1.0-1382.gde09dd01.el9.x86_64 2024-08-19T01:24:04.505 INFO:teuthology.orchestra.run.smithi117.stdout: python3-ceph-common-2:19.1.0-1382.gde09dd01.el9.x86_64 2024-08-19T01:24:04.505 INFO:teuthology.orchestra.run.smithi117.stdout: python3-cephfs-2:19.1.0-1382.gde09dd01.el9.x86_64 2024-08-19T01:24:04.505 INFO:teuthology.orchestra.run.smithi117.stdout: python3-certifi-2023.05.07-4.el9.noarch 2024-08-19T01:24:04.505 INFO:teuthology.orchestra.run.smithi117.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-08-19T01:24:04.505 INFO:teuthology.orchestra.run.smithi117.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-08-19T01:24:04.505 INFO:teuthology.orchestra.run.smithi117.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-08-19T01:24:04.505 INFO:teuthology.orchestra.run.smithi117.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-08-19T01:24:04.505 INFO:teuthology.orchestra.run.smithi117.stdout: python3-colorama-0.4.6-3.el9.noarch 2024-08-19T01:24:04.506 INFO:teuthology.orchestra.run.smithi117.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-08-19T01:24:04.506 INFO:teuthology.orchestra.run.smithi117.stdout: python3-devel-3.9.19-4.el9.x86_64 2024-08-19T01:24:04.506 INFO:teuthology.orchestra.run.smithi117.stdout: python3-google-auth-1:2.33.0-1.el9.noarch 2024-08-19T01:24:04.506 INFO:teuthology.orchestra.run.smithi117.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-08-19T01:24:04.506 INFO:teuthology.orchestra.run.smithi117.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-08-19T01:24:04.506 INFO:teuthology.orchestra.run.smithi117.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-08-19T01:24:04.506 INFO:teuthology.orchestra.run.smithi117.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-08-19T01:24:04.506 INFO:teuthology.orchestra.run.smithi117.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-08-19T01:24:04.506 INFO:teuthology.orchestra.run.smithi117.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-08-19T01:24:04.506 INFO:teuthology.orchestra.run.smithi117.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-08-19T01:24:04.506 INFO:teuthology.orchestra.run.smithi117.stdout: python3-jinja2-2.11.3-6.el9.noarch 2024-08-19T01:24:04.506 INFO:teuthology.orchestra.run.smithi117.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-08-19T01:24:04.506 INFO:teuthology.orchestra.run.smithi117.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-08-19T01:24:04.506 INFO:teuthology.orchestra.run.smithi117.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-08-19T01:24:04.507 INFO:teuthology.orchestra.run.smithi117.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-08-19T01:24:04.507 INFO:teuthology.orchestra.run.smithi117.stdout: python3-mako-1.1.4-6.el9.noarch 2024-08-19T01:24:04.507 INFO:teuthology.orchestra.run.smithi117.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-08-19T01:24:04.507 INFO:teuthology.orchestra.run.smithi117.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-08-19T01:24:04.507 INFO:teuthology.orchestra.run.smithi117.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-08-19T01:24:04.507 INFO:teuthology.orchestra.run.smithi117.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-08-19T01:24:04.507 INFO:teuthology.orchestra.run.smithi117.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-08-19T01:24:04.507 INFO:teuthology.orchestra.run.smithi117.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-08-19T01:24:04.507 INFO:teuthology.orchestra.run.smithi117.stdout: python3-packaging-20.9-5.el9.noarch 2024-08-19T01:24:04.507 INFO:teuthology.orchestra.run.smithi117.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-08-19T01:24:04.507 INFO:teuthology.orchestra.run.smithi117.stdout: python3-ply-3.11-14.el9.noarch 2024-08-19T01:24:04.507 INFO:teuthology.orchestra.run.smithi117.stdout: python3-portend-3.1.0-2.el9.noarch 2024-08-19T01:24:04.507 INFO:teuthology.orchestra.run.smithi117.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-08-19T01:24:04.508 INFO:teuthology.orchestra.run.smithi117.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-08-19T01:24:04.508 INFO:teuthology.orchestra.run.smithi117.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-08-19T01:24:04.508 INFO:teuthology.orchestra.run.smithi117.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-08-19T01:24:04.508 INFO:teuthology.orchestra.run.smithi117.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-08-19T01:24:04.508 INFO:teuthology.orchestra.run.smithi117.stdout: python3-pycparser-2.20-6.el9.noarch 2024-08-19T01:24:04.508 INFO:teuthology.orchestra.run.smithi117.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-08-19T01:24:04.508 INFO:teuthology.orchestra.run.smithi117.stdout: python3-pytz-2021.1-5.el9.noarch 2024-08-19T01:24:04.508 INFO:teuthology.orchestra.run.smithi117.stdout: python3-rados-2:19.1.0-1382.gde09dd01.el9.x86_64 2024-08-19T01:24:04.508 INFO:teuthology.orchestra.run.smithi117.stdout: python3-rbd-2:19.1.0-1382.gde09dd01.el9.x86_64 2024-08-19T01:24:04.508 INFO:teuthology.orchestra.run.smithi117.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-08-19T01:24:04.508 INFO:teuthology.orchestra.run.smithi117.stdout: python3-requests-2.25.1-8.el9.noarch 2024-08-19T01:24:04.509 INFO:teuthology.orchestra.run.smithi117.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-08-19T01:24:04.509 INFO:teuthology.orchestra.run.smithi117.stdout: python3-rgw-2:19.1.0-1382.gde09dd01.el9.x86_64 2024-08-19T01:24:04.509 INFO:teuthology.orchestra.run.smithi117.stdout: python3-routes-2.5.1-5.el9.noarch 2024-08-19T01:24:04.509 INFO:teuthology.orchestra.run.smithi117.stdout: python3-rsa-4.9-2.el9.noarch 2024-08-19T01:24:04.509 INFO:teuthology.orchestra.run.smithi117.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-08-19T01:24:04.509 INFO:teuthology.orchestra.run.smithi117.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-08-19T01:24:04.509 INFO:teuthology.orchestra.run.smithi117.stdout: python3-toml-0.10.2-6.el9.noarch 2024-08-19T01:24:04.509 INFO:teuthology.orchestra.run.smithi117.stdout: python3-typing-extensions-4.12.2-2.el9.noarch 2024-08-19T01:24:04.509 INFO:teuthology.orchestra.run.smithi117.stdout: python3-urllib3-1.26.5-6.el9.noarch 2024-08-19T01:24:04.509 INFO:teuthology.orchestra.run.smithi117.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-08-19T01:24:04.509 INFO:teuthology.orchestra.run.smithi117.stdout: python3-webob-1.8.7-6.el9.noarch 2024-08-19T01:24:04.510 INFO:teuthology.orchestra.run.smithi117.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-08-19T01:24:04.510 INFO:teuthology.orchestra.run.smithi117.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-08-19T01:24:04.510 INFO:teuthology.orchestra.run.smithi117.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-08-19T01:24:04.510 INFO:teuthology.orchestra.run.smithi117.stdout: qatlib-24.02.0-1.el9.x86_64 2024-08-19T01:24:04.510 INFO:teuthology.orchestra.run.smithi117.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-08-19T01:24:04.510 INFO:teuthology.orchestra.run.smithi117.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-08-19T01:24:04.510 INFO:teuthology.orchestra.run.smithi117.stdout: rbd-fuse-2:19.1.0-1382.gde09dd01.el9.x86_64 2024-08-19T01:24:04.510 INFO:teuthology.orchestra.run.smithi117.stdout: rbd-mirror-2:19.1.0-1382.gde09dd01.el9.x86_64 2024-08-19T01:24:04.510 INFO:teuthology.orchestra.run.smithi117.stdout: rbd-nbd-2:19.1.0-1382.gde09dd01.el9.x86_64 2024-08-19T01:24:04.510 INFO:teuthology.orchestra.run.smithi117.stdout: re2-1:20211101-3.el9.x86_64 2024-08-19T01:24:04.510 INFO:teuthology.orchestra.run.smithi117.stdout: socat-1.7.4.1-6.el9.x86_64 2024-08-19T01:24:04.510 INFO:teuthology.orchestra.run.smithi117.stdout: thrift-0.15.0-2.el9.x86_64 2024-08-19T01:24:04.510 INFO:teuthology.orchestra.run.smithi117.stdout: uuid-1.6.2-55.el9.x86_64 2024-08-19T01:24:04.510 INFO:teuthology.orchestra.run.smithi117.stdout: xfsprogs-devel-6.4.0-3.el9.x86_64 2024-08-19T01:24:04.511 INFO:teuthology.orchestra.run.smithi117.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-08-19T01:24:04.513 INFO:teuthology.orchestra.run.smithi117.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-08-19T01:24:04.513 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-08-19T01:24:04.513 INFO:teuthology.orchestra.run.smithi117.stdout:Complete! 2024-08-19T01:24:04.849 DEBUG:teuthology.parallel:result is None 2024-08-19T01:24:05.344 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/152 2024-08-19T01:24:05.344 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-08-19T01:24:05.344 INFO:teuthology.orchestra.run.smithi083.stdout:Upgraded: 2024-08-19T01:24:05.344 INFO:teuthology.orchestra.run.smithi083.stdout: librados2-2:19.1.0-1382.gde09dd01.el9.x86_64 2024-08-19T01:24:05.344 INFO:teuthology.orchestra.run.smithi083.stdout: librbd1-2:19.1.0-1382.gde09dd01.el9.x86_64 2024-08-19T01:24:05.344 INFO:teuthology.orchestra.run.smithi083.stdout:Installed: 2024-08-19T01:24:05.344 INFO:teuthology.orchestra.run.smithi083.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-08-19T01:24:05.345 INFO:teuthology.orchestra.run.smithi083.stdout: automake-1.16.2-8.el9.noarch 2024-08-19T01:24:05.345 INFO:teuthology.orchestra.run.smithi083.stdout: bison-3.7.4-5.el9.x86_64 2024-08-19T01:24:05.345 INFO:teuthology.orchestra.run.smithi083.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-08-19T01:24:05.345 INFO:teuthology.orchestra.run.smithi083.stdout: ceph-2:19.1.0-1382.gde09dd01.el9.x86_64 2024-08-19T01:24:05.345 INFO:teuthology.orchestra.run.smithi083.stdout: ceph-base-2:19.1.0-1382.gde09dd01.el9.x86_64 2024-08-19T01:24:05.345 INFO:teuthology.orchestra.run.smithi083.stdout: ceph-common-2:19.1.0-1382.gde09dd01.el9.x86_64 2024-08-19T01:24:05.345 INFO:teuthology.orchestra.run.smithi083.stdout: ceph-fuse-2:19.1.0-1382.gde09dd01.el9.x86_64 2024-08-19T01:24:05.345 INFO:teuthology.orchestra.run.smithi083.stdout: ceph-grafana-dashboards-2:19.1.0-1382.gde09dd01.el9.noarch 2024-08-19T01:24:05.345 INFO:teuthology.orchestra.run.smithi083.stdout: ceph-immutable-object-cache-2:19.1.0-1382.gde09dd01.el9.x86_64 2024-08-19T01:24:05.345 INFO:teuthology.orchestra.run.smithi083.stdout: ceph-mds-2:19.1.0-1382.gde09dd01.el9.x86_64 2024-08-19T01:24:05.345 INFO:teuthology.orchestra.run.smithi083.stdout: ceph-mgr-2:19.1.0-1382.gde09dd01.el9.x86_64 2024-08-19T01:24:05.345 INFO:teuthology.orchestra.run.smithi083.stdout: ceph-mgr-cephadm-2:19.1.0-1382.gde09dd01.el9.noarch 2024-08-19T01:24:05.346 INFO:teuthology.orchestra.run.smithi083.stdout: ceph-mgr-dashboard-2:19.1.0-1382.gde09dd01.el9.noarch 2024-08-19T01:24:05.346 INFO:teuthology.orchestra.run.smithi083.stdout: ceph-mgr-diskprediction-local-2:19.1.0-1382.gde09dd01.el9.noarch 2024-08-19T01:24:05.346 INFO:teuthology.orchestra.run.smithi083.stdout: ceph-mgr-modules-core-2:19.1.0-1382.gde09dd01.el9.noarch 2024-08-19T01:24:05.346 INFO:teuthology.orchestra.run.smithi083.stdout: ceph-mgr-rook-2:19.1.0-1382.gde09dd01.el9.noarch 2024-08-19T01:24:05.346 INFO:teuthology.orchestra.run.smithi083.stdout: ceph-mon-2:19.1.0-1382.gde09dd01.el9.x86_64 2024-08-19T01:24:05.346 INFO:teuthology.orchestra.run.smithi083.stdout: ceph-osd-2:19.1.0-1382.gde09dd01.el9.x86_64 2024-08-19T01:24:05.346 INFO:teuthology.orchestra.run.smithi083.stdout: ceph-prometheus-alerts-2:19.1.0-1382.gde09dd01.el9.noarch 2024-08-19T01:24:05.346 INFO:teuthology.orchestra.run.smithi083.stdout: ceph-radosgw-2:19.1.0-1382.gde09dd01.el9.x86_64 2024-08-19T01:24:05.346 INFO:teuthology.orchestra.run.smithi083.stdout: ceph-selinux-2:19.1.0-1382.gde09dd01.el9.x86_64 2024-08-19T01:24:05.346 INFO:teuthology.orchestra.run.smithi083.stdout: ceph-test-2:19.1.0-1382.gde09dd01.el9.x86_64 2024-08-19T01:24:05.346 INFO:teuthology.orchestra.run.smithi083.stdout: ceph-volume-2:19.1.0-1382.gde09dd01.el9.noarch 2024-08-19T01:24:05.346 INFO:teuthology.orchestra.run.smithi083.stdout: cephadm-2:19.1.0-1382.gde09dd01.el9.noarch 2024-08-19T01:24:05.346 INFO:teuthology.orchestra.run.smithi083.stdout: cephfs-mirror-2:19.1.0-1382.gde09dd01.el9.x86_64 2024-08-19T01:24:05.347 INFO:teuthology.orchestra.run.smithi083.stdout: cephfs-shell-2:19.1.0-1382.gde09dd01.el9.x86_64 2024-08-19T01:24:05.347 INFO:teuthology.orchestra.run.smithi083.stdout: cephfs-top-2:19.1.0-1382.gde09dd01.el9.noarch 2024-08-19T01:24:05.347 INFO:teuthology.orchestra.run.smithi083.stdout: elfutils-libelf-devel-0.191-4.el9.x86_64 2024-08-19T01:24:05.347 INFO:teuthology.orchestra.run.smithi083.stdout: flex-2.6.4-9.el9.x86_64 2024-08-19T01:24:05.347 INFO:teuthology.orchestra.run.smithi083.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-08-19T01:24:05.347 INFO:teuthology.orchestra.run.smithi083.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-08-19T01:24:05.347 INFO:teuthology.orchestra.run.smithi083.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-08-19T01:24:05.347 INFO:teuthology.orchestra.run.smithi083.stdout: fuse-2.9.9-16.el9.x86_64 2024-08-19T01:24:05.347 INFO:teuthology.orchestra.run.smithi083.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-08-19T01:24:05.347 INFO:teuthology.orchestra.run.smithi083.stdout: grpc-data-1.46.7-10.el9.noarch 2024-08-19T01:24:05.347 INFO:teuthology.orchestra.run.smithi083.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-08-19T01:24:05.347 INFO:teuthology.orchestra.run.smithi083.stdout: libarrow-9.0.0-11.el9.x86_64 2024-08-19T01:24:05.347 INFO:teuthology.orchestra.run.smithi083.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-08-19T01:24:05.348 INFO:teuthology.orchestra.run.smithi083.stdout: libcephfs-devel-2:19.1.0-1382.gde09dd01.el9.x86_64 2024-08-19T01:24:05.348 INFO:teuthology.orchestra.run.smithi083.stdout: libcephfs2-2:19.1.0-1382.gde09dd01.el9.x86_64 2024-08-19T01:24:05.348 INFO:teuthology.orchestra.run.smithi083.stdout: libcephsqlite-2:19.1.0-1382.gde09dd01.el9.x86_64 2024-08-19T01:24:05.348 INFO:teuthology.orchestra.run.smithi083.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-08-19T01:24:05.348 INFO:teuthology.orchestra.run.smithi083.stdout: liboath-2.6.7-2.el9.x86_64 2024-08-19T01:24:05.348 INFO:teuthology.orchestra.run.smithi083.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-08-19T01:24:05.348 INFO:teuthology.orchestra.run.smithi083.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-08-19T01:24:05.348 INFO:teuthology.orchestra.run.smithi083.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-08-19T01:24:05.348 INFO:teuthology.orchestra.run.smithi083.stdout: librados-devel-2:19.1.0-1382.gde09dd01.el9.x86_64 2024-08-19T01:24:05.348 INFO:teuthology.orchestra.run.smithi083.stdout: libradosstriper1-2:19.1.0-1382.gde09dd01.el9.x86_64 2024-08-19T01:24:05.348 INFO:teuthology.orchestra.run.smithi083.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-08-19T01:24:05.348 INFO:teuthology.orchestra.run.smithi083.stdout: librgw2-2:19.1.0-1382.gde09dd01.el9.x86_64 2024-08-19T01:24:05.348 INFO:teuthology.orchestra.run.smithi083.stdout: libtool-2.4.6-46.el9.x86_64 2024-08-19T01:24:05.349 INFO:teuthology.orchestra.run.smithi083.stdout: libunwind-1.6.2-1.el9.x86_64 2024-08-19T01:24:05.349 INFO:teuthology.orchestra.run.smithi083.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-08-19T01:24:05.349 INFO:teuthology.orchestra.run.smithi083.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-08-19T01:24:05.349 INFO:teuthology.orchestra.run.smithi083.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-08-19T01:24:05.349 INFO:teuthology.orchestra.run.smithi083.stdout: lua-5.4.4-4.el9.x86_64 2024-08-19T01:24:05.349 INFO:teuthology.orchestra.run.smithi083.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-08-19T01:24:05.349 INFO:teuthology.orchestra.run.smithi083.stdout: luarocks-3.9.2-5.el9.noarch 2024-08-19T01:24:05.349 INFO:teuthology.orchestra.run.smithi083.stdout: openblas-0.3.21-2.el9.x86_64 2024-08-19T01:24:05.349 INFO:teuthology.orchestra.run.smithi083.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-08-19T01:24:05.349 INFO:teuthology.orchestra.run.smithi083.stdout: openssl-devel-1:3.2.2-2.el9.x86_64 2024-08-19T01:24:05.349 INFO:teuthology.orchestra.run.smithi083.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-08-19T01:24:05.349 INFO:teuthology.orchestra.run.smithi083.stdout: postgresql-13.14-1.el9.x86_64 2024-08-19T01:24:05.350 INFO:teuthology.orchestra.run.smithi083.stdout: postgresql-contrib-13.14-1.el9.x86_64 2024-08-19T01:24:05.350 INFO:teuthology.orchestra.run.smithi083.stdout: postgresql-private-libs-13.14-1.el9.x86_64 2024-08-19T01:24:05.350 INFO:teuthology.orchestra.run.smithi083.stdout: postgresql-server-13.14-1.el9.x86_64 2024-08-19T01:24:05.350 INFO:teuthology.orchestra.run.smithi083.stdout: protobuf-3.14.0-13.el9.x86_64 2024-08-19T01:24:05.350 INFO:teuthology.orchestra.run.smithi083.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-08-19T01:24:05.350 INFO:teuthology.orchestra.run.smithi083.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-08-19T01:24:05.350 INFO:teuthology.orchestra.run.smithi083.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-08-19T01:24:05.350 INFO:teuthology.orchestra.run.smithi083.stdout: python3-babel-2.9.1-2.el9.noarch 2024-08-19T01:24:05.350 INFO:teuthology.orchestra.run.smithi083.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-08-19T01:24:05.350 INFO:teuthology.orchestra.run.smithi083.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-08-19T01:24:05.350 INFO:teuthology.orchestra.run.smithi083.stdout: python3-ceph-argparse-2:19.1.0-1382.gde09dd01.el9.x86_64 2024-08-19T01:24:05.350 INFO:teuthology.orchestra.run.smithi083.stdout: python3-ceph-common-2:19.1.0-1382.gde09dd01.el9.x86_64 2024-08-19T01:24:05.350 INFO:teuthology.orchestra.run.smithi083.stdout: python3-cephfs-2:19.1.0-1382.gde09dd01.el9.x86_64 2024-08-19T01:24:05.351 INFO:teuthology.orchestra.run.smithi083.stdout: python3-certifi-2023.05.07-4.el9.noarch 2024-08-19T01:24:05.351 INFO:teuthology.orchestra.run.smithi083.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-08-19T01:24:05.351 INFO:teuthology.orchestra.run.smithi083.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-08-19T01:24:05.351 INFO:teuthology.orchestra.run.smithi083.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-08-19T01:24:05.351 INFO:teuthology.orchestra.run.smithi083.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-08-19T01:24:05.351 INFO:teuthology.orchestra.run.smithi083.stdout: python3-colorama-0.4.6-3.el9.noarch 2024-08-19T01:24:05.351 INFO:teuthology.orchestra.run.smithi083.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-08-19T01:24:05.351 INFO:teuthology.orchestra.run.smithi083.stdout: python3-devel-3.9.19-4.el9.x86_64 2024-08-19T01:24:05.351 INFO:teuthology.orchestra.run.smithi083.stdout: python3-google-auth-1:2.33.0-1.el9.noarch 2024-08-19T01:24:05.351 INFO:teuthology.orchestra.run.smithi083.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-08-19T01:24:05.351 INFO:teuthology.orchestra.run.smithi083.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-08-19T01:24:05.351 INFO:teuthology.orchestra.run.smithi083.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-08-19T01:24:05.351 INFO:teuthology.orchestra.run.smithi083.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-08-19T01:24:05.352 INFO:teuthology.orchestra.run.smithi083.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-08-19T01:24:05.352 INFO:teuthology.orchestra.run.smithi083.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-08-19T01:24:05.352 INFO:teuthology.orchestra.run.smithi083.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-08-19T01:24:05.352 INFO:teuthology.orchestra.run.smithi083.stdout: python3-jinja2-2.11.3-6.el9.noarch 2024-08-19T01:24:05.352 INFO:teuthology.orchestra.run.smithi083.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-08-19T01:24:05.352 INFO:teuthology.orchestra.run.smithi083.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-08-19T01:24:05.352 INFO:teuthology.orchestra.run.smithi083.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-08-19T01:24:05.352 INFO:teuthology.orchestra.run.smithi083.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-08-19T01:24:05.352 INFO:teuthology.orchestra.run.smithi083.stdout: python3-mako-1.1.4-6.el9.noarch 2024-08-19T01:24:05.352 INFO:teuthology.orchestra.run.smithi083.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-08-19T01:24:05.352 INFO:teuthology.orchestra.run.smithi083.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-08-19T01:24:05.352 INFO:teuthology.orchestra.run.smithi083.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-08-19T01:24:05.353 INFO:teuthology.orchestra.run.smithi083.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-08-19T01:24:05.353 INFO:teuthology.orchestra.run.smithi083.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-08-19T01:24:05.353 INFO:teuthology.orchestra.run.smithi083.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-08-19T01:24:05.353 INFO:teuthology.orchestra.run.smithi083.stdout: python3-packaging-20.9-5.el9.noarch 2024-08-19T01:24:05.353 INFO:teuthology.orchestra.run.smithi083.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-08-19T01:24:05.353 INFO:teuthology.orchestra.run.smithi083.stdout: python3-ply-3.11-14.el9.noarch 2024-08-19T01:24:05.353 INFO:teuthology.orchestra.run.smithi083.stdout: python3-portend-3.1.0-2.el9.noarch 2024-08-19T01:24:05.353 INFO:teuthology.orchestra.run.smithi083.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-08-19T01:24:05.353 INFO:teuthology.orchestra.run.smithi083.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-08-19T01:24:05.353 INFO:teuthology.orchestra.run.smithi083.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-08-19T01:24:05.353 INFO:teuthology.orchestra.run.smithi083.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-08-19T01:24:05.353 INFO:teuthology.orchestra.run.smithi083.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-08-19T01:24:05.354 INFO:teuthology.orchestra.run.smithi083.stdout: python3-pycparser-2.20-6.el9.noarch 2024-08-19T01:24:05.354 INFO:teuthology.orchestra.run.smithi083.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-08-19T01:24:05.354 INFO:teuthology.orchestra.run.smithi083.stdout: python3-pytz-2021.1-5.el9.noarch 2024-08-19T01:24:05.354 INFO:teuthology.orchestra.run.smithi083.stdout: python3-rados-2:19.1.0-1382.gde09dd01.el9.x86_64 2024-08-19T01:24:05.354 INFO:teuthology.orchestra.run.smithi083.stdout: python3-rbd-2:19.1.0-1382.gde09dd01.el9.x86_64 2024-08-19T01:24:05.354 INFO:teuthology.orchestra.run.smithi083.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-08-19T01:24:05.354 INFO:teuthology.orchestra.run.smithi083.stdout: python3-requests-2.25.1-8.el9.noarch 2024-08-19T01:24:05.354 INFO:teuthology.orchestra.run.smithi083.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-08-19T01:24:05.354 INFO:teuthology.orchestra.run.smithi083.stdout: python3-rgw-2:19.1.0-1382.gde09dd01.el9.x86_64 2024-08-19T01:24:05.354 INFO:teuthology.orchestra.run.smithi083.stdout: python3-routes-2.5.1-5.el9.noarch 2024-08-19T01:24:05.354 INFO:teuthology.orchestra.run.smithi083.stdout: python3-rsa-4.9-2.el9.noarch 2024-08-19T01:24:05.354 INFO:teuthology.orchestra.run.smithi083.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-08-19T01:24:05.355 INFO:teuthology.orchestra.run.smithi083.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-08-19T01:24:05.355 INFO:teuthology.orchestra.run.smithi083.stdout: python3-toml-0.10.2-6.el9.noarch 2024-08-19T01:24:05.355 INFO:teuthology.orchestra.run.smithi083.stdout: python3-typing-extensions-4.12.2-2.el9.noarch 2024-08-19T01:24:05.355 INFO:teuthology.orchestra.run.smithi083.stdout: python3-urllib3-1.26.5-6.el9.noarch 2024-08-19T01:24:05.355 INFO:teuthology.orchestra.run.smithi083.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-08-19T01:24:05.355 INFO:teuthology.orchestra.run.smithi083.stdout: python3-webob-1.8.7-6.el9.noarch 2024-08-19T01:24:05.355 INFO:teuthology.orchestra.run.smithi083.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-08-19T01:24:05.355 INFO:teuthology.orchestra.run.smithi083.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-08-19T01:24:05.355 INFO:teuthology.orchestra.run.smithi083.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-08-19T01:24:05.355 INFO:teuthology.orchestra.run.smithi083.stdout: qatlib-24.02.0-1.el9.x86_64 2024-08-19T01:24:05.355 INFO:teuthology.orchestra.run.smithi083.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-08-19T01:24:05.355 INFO:teuthology.orchestra.run.smithi083.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-08-19T01:24:05.355 INFO:teuthology.orchestra.run.smithi083.stdout: rbd-fuse-2:19.1.0-1382.gde09dd01.el9.x86_64 2024-08-19T01:24:05.356 INFO:teuthology.orchestra.run.smithi083.stdout: rbd-mirror-2:19.1.0-1382.gde09dd01.el9.x86_64 2024-08-19T01:24:05.356 INFO:teuthology.orchestra.run.smithi083.stdout: rbd-nbd-2:19.1.0-1382.gde09dd01.el9.x86_64 2024-08-19T01:24:05.356 INFO:teuthology.orchestra.run.smithi083.stdout: re2-1:20211101-3.el9.x86_64 2024-08-19T01:24:05.356 INFO:teuthology.orchestra.run.smithi083.stdout: socat-1.7.4.1-6.el9.x86_64 2024-08-19T01:24:05.356 INFO:teuthology.orchestra.run.smithi083.stdout: thrift-0.15.0-2.el9.x86_64 2024-08-19T01:24:05.356 INFO:teuthology.orchestra.run.smithi083.stdout: uuid-1.6.2-55.el9.x86_64 2024-08-19T01:24:05.356 INFO:teuthology.orchestra.run.smithi083.stdout: xfsprogs-devel-6.4.0-3.el9.x86_64 2024-08-19T01:24:05.356 INFO:teuthology.orchestra.run.smithi083.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-08-19T01:24:05.357 INFO:teuthology.orchestra.run.smithi083.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-08-19T01:24:05.357 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-08-19T01:24:05.357 INFO:teuthology.orchestra.run.smithi083.stdout:Complete! 2024-08-19T01:24:05.857 DEBUG:teuthology.parallel:result is None 2024-08-19T01:24:05.858 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=de09dd018de4baddfa6ca4d43bc8fb784cf75833 2024-08-19T01:24:06.011 DEBUG:teuthology.orchestra.run.smithi083:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-08-19T01:24:06.057 INFO:teuthology.orchestra.run.smithi083.stdout:19.1.0-1382.gde09dd01.el9 2024-08-19T01:24:06.057 INFO:teuthology.packaging:The installed version of ceph is 19.1.0-1382.gde09dd01.el9 2024-08-19T01:24:06.057 INFO:teuthology.task.install:The correct ceph version 19.1.0-1382.gde09dd01 is installed. 2024-08-19T01:24:06.060 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=de09dd018de4baddfa6ca4d43bc8fb784cf75833 2024-08-19T01:24:06.211 DEBUG:teuthology.orchestra.run.smithi117:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-08-19T01:24:06.235 INFO:teuthology.orchestra.run.smithi117.stdout:19.1.0-1382.gde09dd01.el9 2024-08-19T01:24:06.235 INFO:teuthology.packaging:The installed version of ceph is 19.1.0-1382.gde09dd01.el9 2024-08-19T01:24:06.235 INFO:teuthology.task.install:The correct ceph version 19.1.0-1382.gde09dd01 is installed. 2024-08-19T01:24:06.238 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=de09dd018de4baddfa6ca4d43bc8fb784cf75833 2024-08-19T01:24:06.380 DEBUG:teuthology.orchestra.run.smithi167:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-08-19T01:24:06.407 INFO:teuthology.orchestra.run.smithi167.stdout:19.1.0-1382.gde09dd01.el9 2024-08-19T01:24:06.407 INFO:teuthology.packaging:The installed version of ceph is 19.1.0-1382.gde09dd01.el9 2024-08-19T01:24:06.407 INFO:teuthology.task.install:The correct ceph version 19.1.0-1382.gde09dd01 is installed. 2024-08-19T01:24:06.409 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-08-19T01:24:06.409 DEBUG:teuthology.orchestra.run.smithi083:> set -ex 2024-08-19T01:24:06.409 DEBUG:teuthology.orchestra.run.smithi083:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-08-19T01:24:06.469 DEBUG:teuthology.orchestra.run.smithi117:> set -ex 2024-08-19T01:24:06.469 DEBUG:teuthology.orchestra.run.smithi117:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-08-19T01:24:06.497 DEBUG:teuthology.orchestra.run.smithi167:> set -ex 2024-08-19T01:24:06.497 DEBUG:teuthology.orchestra.run.smithi167:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-08-19T01:24:06.529 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-08-19T01:24:06.529 DEBUG:teuthology.orchestra.run.smithi083:> set -ex 2024-08-19T01:24:06.529 DEBUG:teuthology.orchestra.run.smithi083:> sudo dd of=/usr/bin/daemon-helper 2024-08-19T01:24:06.584 DEBUG:teuthology.orchestra.run.smithi083:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-08-19T01:24:06.680 DEBUG:teuthology.orchestra.run.smithi117:> set -ex 2024-08-19T01:24:06.680 DEBUG:teuthology.orchestra.run.smithi117:> sudo dd of=/usr/bin/daemon-helper 2024-08-19T01:24:06.705 DEBUG:teuthology.orchestra.run.smithi117:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-08-19T01:24:06.771 DEBUG:teuthology.orchestra.run.smithi167:> set -ex 2024-08-19T01:24:06.771 DEBUG:teuthology.orchestra.run.smithi167:> sudo dd of=/usr/bin/daemon-helper 2024-08-19T01:24:06.802 DEBUG:teuthology.orchestra.run.smithi167:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-08-19T01:24:06.869 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-08-19T01:24:06.869 DEBUG:teuthology.orchestra.run.smithi083:> set -ex 2024-08-19T01:24:06.870 DEBUG:teuthology.orchestra.run.smithi083:> sudo dd of=/usr/bin/adjust-ulimits 2024-08-19T01:24:06.927 DEBUG:teuthology.orchestra.run.smithi083:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-08-19T01:24:07.025 DEBUG:teuthology.orchestra.run.smithi117:> set -ex 2024-08-19T01:24:07.025 DEBUG:teuthology.orchestra.run.smithi117:> sudo dd of=/usr/bin/adjust-ulimits 2024-08-19T01:24:07.050 DEBUG:teuthology.orchestra.run.smithi117:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-08-19T01:24:07.114 DEBUG:teuthology.orchestra.run.smithi167:> set -ex 2024-08-19T01:24:07.114 DEBUG:teuthology.orchestra.run.smithi167:> sudo dd of=/usr/bin/adjust-ulimits 2024-08-19T01:24:07.140 DEBUG:teuthology.orchestra.run.smithi167:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-08-19T01:24:07.205 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-08-19T01:24:07.205 DEBUG:teuthology.orchestra.run.smithi083:> set -ex 2024-08-19T01:24:07.205 DEBUG:teuthology.orchestra.run.smithi083:> sudo dd of=/usr/bin/stdin-killer 2024-08-19T01:24:07.261 DEBUG:teuthology.orchestra.run.smithi083:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-08-19T01:24:07.357 DEBUG:teuthology.orchestra.run.smithi117:> set -ex 2024-08-19T01:24:07.357 DEBUG:teuthology.orchestra.run.smithi117:> sudo dd of=/usr/bin/stdin-killer 2024-08-19T01:24:07.381 DEBUG:teuthology.orchestra.run.smithi117:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-08-19T01:24:07.447 DEBUG:teuthology.orchestra.run.smithi167:> set -ex 2024-08-19T01:24:07.447 DEBUG:teuthology.orchestra.run.smithi167:> sudo dd of=/usr/bin/stdin-killer 2024-08-19T01:24:07.473 DEBUG:teuthology.orchestra.run.smithi167:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-08-19T01:24:07.538 INFO:teuthology.run_tasks:Running task cephadm... 2024-08-19T01:24:07.632 INFO:tasks.cephadm:Config: {'roleless': False, 'cephfs': {'max_mds': 3, 'session_timeout': 300, 'standby_replay': True}, 'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'ms die on skipped message': False}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds bal fragment size max': 10000, 'mds bal merge size': 5, 'mds bal split bits': 3, 'mds bal split size': 100, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'mds_export_ephemeral_random_max': 0.1, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'cephsqlite lock renewal timeout': 900000, 'debug client': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon down mkfs grace': 300, 'mon op complaint time': 120, 'mon warn on pool no app': False}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180, 'osd shutdown pgref assert': True, 'osd_max_omap_entries_per_request': 10}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'filesystem is online with fewer MDS than max_mds', 'POOL_APP_NOT_ENABLED', 'do not have an application enabled', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'MON_DOWN', 'PG_AVAILABILITY', 'PG_DEGRADED', 'Reduced data availability', 'Degraded data redundancy', 'slow metadata IO', 'SLOW_OPS', 'slow request'], 'sha1': 'de09dd018de4baddfa6ca4d43bc8fb784cf75833', 'subvols': {'create': 2, 'subvol_options': '--size 25000000000'}} 2024-08-19T01:24:07.632 INFO:tasks.cephadm:Cluster image is quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833 2024-08-19T01:24:07.633 INFO:tasks.cephadm:Cluster fsid is bad8780e-5dc9-11ef-bccf-c7b262605968 2024-08-19T01:24:07.633 INFO:tasks.cephadm:Choosing monitor IPs and ports... 2024-08-19T01:24:07.633 INFO:tasks.cephadm:Monitor IPs: {'mon.a': '172.21.15.83', 'mon.b': '172.21.15.117', 'mon.c': '172.21.15.167'} 2024-08-19T01:24:07.633 INFO:tasks.cephadm:First mon is mon.a on smithi083 2024-08-19T01:24:07.633 INFO:tasks.cephadm:First mgr is x 2024-08-19T01:24:07.634 INFO:tasks.cephadm:Normalizing hostnames... 2024-08-19T01:24:07.634 DEBUG:teuthology.orchestra.run.smithi083:> sudo hostname $(hostname -s) 2024-08-19T01:24:07.691 DEBUG:teuthology.orchestra.run.smithi117:> sudo hostname $(hostname -s) 2024-08-19T01:24:07.717 DEBUG:teuthology.orchestra.run.smithi167:> sudo hostname $(hostname -s) 2024-08-19T01:24:07.742 INFO:tasks.cephadm:Downloading "compiled" cephadm from cachra 2024-08-19T01:24:07.743 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=de09dd018de4baddfa6ca4d43bc8fb784cf75833 2024-08-19T01:24:07.885 INFO:tasks.cephadm:builder_project result: [{'status': 'ready', 'sha1': 'de09dd018de4baddfa6ca4d43bc8fb784cf75833', 'extra': {'build_url': 'https://jenkins.ceph.com/job/ceph-dev-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic/55869/', 'root_build_cause': 'SCMTRIGGER', 'version': '19.1.0-1382-gde09dd01', 'node_name': '172.21.5.40+adami10', 'job_name': 'ceph-dev-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic', 'package_manager_version': '19.1.0-1382.gde09dd01'}, 'url': 'https://1.chacra.ceph.com/r/ceph/squid/de09dd018de4baddfa6ca4d43bc8fb784cf75833/centos/9/flavors/default/', 'distro_codename': None, 'modified': '2024-08-18 16:22:46.893995', 'distro_version': '9', 'project': 'ceph', 'flavor': 'default', 'ref': 'squid', 'chacra_url': 'https://1.chacra.ceph.com/repos/ceph/squid/de09dd018de4baddfa6ca4d43bc8fb784cf75833/centos/9/flavors/default/', 'archs': ['x86_64', 'source'], 'distro': 'centos'}] 2024-08-19T01:24:08.022 INFO:tasks.util.chacra:got chacra host 1.chacra.ceph.com, ref squid, sha1 de09dd018de4baddfa6ca4d43bc8fb784cf75833 from https://shaman.ceph.com/api/search/?project=ceph&distros=centos%2F9%2Fx86_64&flavor=default&sha1=de09dd018de4baddfa6ca4d43bc8fb784cf75833 2024-08-19T01:24:08.023 INFO:tasks.cephadm:Discovered cachra url: https://1.chacra.ceph.com/binaries/ceph/squid/de09dd018de4baddfa6ca4d43bc8fb784cf75833/centos/9/x86_64/flavors/default/cephadm 2024-08-19T01:24:08.023 INFO:tasks.cephadm:Downloading cephadm from url: https://1.chacra.ceph.com/binaries/ceph/squid/de09dd018de4baddfa6ca4d43bc8fb784cf75833/centos/9/x86_64/flavors/default/cephadm 2024-08-19T01:24:08.024 DEBUG:teuthology.orchestra.run.smithi083:> curl --silent -L https://1.chacra.ceph.com/binaries/ceph/squid/de09dd018de4baddfa6ca4d43bc8fb784cf75833/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-08-19T01:24:09.089 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r--. 1 ubuntu ubuntu 784608 Aug 19 01:24 /home/ubuntu/cephtest/cephadm 2024-08-19T01:24:09.090 DEBUG:teuthology.orchestra.run.smithi117:> curl --silent -L https://1.chacra.ceph.com/binaries/ceph/squid/de09dd018de4baddfa6ca4d43bc8fb784cf75833/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-08-19T01:24:10.000 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r--. 1 ubuntu ubuntu 784608 Aug 19 01:24 /home/ubuntu/cephtest/cephadm 2024-08-19T01:24:10.001 DEBUG:teuthology.orchestra.run.smithi167:> curl --silent -L https://1.chacra.ceph.com/binaries/ceph/squid/de09dd018de4baddfa6ca4d43bc8fb784cf75833/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-08-19T01:24:11.384 INFO:teuthology.orchestra.run.smithi167.stdout:-rw-r--r--. 1 ubuntu ubuntu 784608 Aug 19 01:24 /home/ubuntu/cephtest/cephadm 2024-08-19T01:24:11.384 DEBUG:teuthology.orchestra.run.smithi083:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-08-19T01:24:11.422 DEBUG:teuthology.orchestra.run.smithi117:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-08-19T01:24:11.443 DEBUG:teuthology.orchestra.run.smithi167:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-08-19T01:24:11.477 INFO:tasks.cephadm:Pulling image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833 on all hosts... 2024-08-19T01:24:11.478 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833 pull 2024-08-19T01:24:11.481 DEBUG:teuthology.orchestra.run.smithi117:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833 pull 2024-08-19T01:24:11.485 DEBUG:teuthology.orchestra.run.smithi167:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833 pull 2024-08-19T01:24:13.190 INFO:teuthology.orchestra.run.smithi167.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833... 2024-08-19T01:24:13.191 INFO:teuthology.orchestra.run.smithi117.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833... 2024-08-19T01:24:14.555 INFO:teuthology.orchestra.run.smithi083.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833... 2024-08-19T01:24:44.684 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-08-19T01:24:44.685 INFO:teuthology.orchestra.run.smithi083.stdout: "ceph_version": "ceph version 19.1.0-1382-gde09dd01 (de09dd018de4baddfa6ca4d43bc8fb784cf75833) squid (rc)", 2024-08-19T01:24:44.685 INFO:teuthology.orchestra.run.smithi083.stdout: "image_id": "ea7a5189f30bad3aca9524788e2965d2368d7ce786df09d4edcd8c3edf9cefba", 2024-08-19T01:24:44.685 INFO:teuthology.orchestra.run.smithi083.stdout: "repo_digests": [ 2024-08-19T01:24:44.685 INFO:teuthology.orchestra.run.smithi083.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c9dd28080a631be25c77c36c63b17a3ef340ffb2e21c87d4d7b56b6398436d9" 2024-08-19T01:24:44.685 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-08-19T01:24:44.685 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-08-19T01:25:00.056 INFO:teuthology.orchestra.run.smithi117.stdout:{ 2024-08-19T01:25:00.057 INFO:teuthology.orchestra.run.smithi117.stdout: "ceph_version": "ceph version 19.1.0-1382-gde09dd01 (de09dd018de4baddfa6ca4d43bc8fb784cf75833) squid (rc)", 2024-08-19T01:25:00.057 INFO:teuthology.orchestra.run.smithi117.stdout: "image_id": "ea7a5189f30bad3aca9524788e2965d2368d7ce786df09d4edcd8c3edf9cefba", 2024-08-19T01:25:00.057 INFO:teuthology.orchestra.run.smithi117.stdout: "repo_digests": [ 2024-08-19T01:25:00.057 INFO:teuthology.orchestra.run.smithi117.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c9dd28080a631be25c77c36c63b17a3ef340ffb2e21c87d4d7b56b6398436d9" 2024-08-19T01:25:00.057 INFO:teuthology.orchestra.run.smithi117.stdout: ] 2024-08-19T01:25:00.057 INFO:teuthology.orchestra.run.smithi117.stdout:} 2024-08-19T01:25:01.731 INFO:teuthology.orchestra.run.smithi167.stdout:{ 2024-08-19T01:25:01.731 INFO:teuthology.orchestra.run.smithi167.stdout: "ceph_version": "ceph version 19.1.0-1382-gde09dd01 (de09dd018de4baddfa6ca4d43bc8fb784cf75833) squid (rc)", 2024-08-19T01:25:01.731 INFO:teuthology.orchestra.run.smithi167.stdout: "image_id": "ea7a5189f30bad3aca9524788e2965d2368d7ce786df09d4edcd8c3edf9cefba", 2024-08-19T01:25:01.731 INFO:teuthology.orchestra.run.smithi167.stdout: "repo_digests": [ 2024-08-19T01:25:01.731 INFO:teuthology.orchestra.run.smithi167.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c9dd28080a631be25c77c36c63b17a3ef340ffb2e21c87d4d7b56b6398436d9" 2024-08-19T01:25:01.731 INFO:teuthology.orchestra.run.smithi167.stdout: ] 2024-08-19T01:25:01.731 INFO:teuthology.orchestra.run.smithi167.stdout:} 2024-08-19T01:25:01.755 DEBUG:teuthology.orchestra.run.smithi083:> sudo mkdir -p /etc/ceph 2024-08-19T01:25:01.821 DEBUG:teuthology.orchestra.run.smithi117:> sudo mkdir -p /etc/ceph 2024-08-19T01:25:01.855 DEBUG:teuthology.orchestra.run.smithi167:> sudo mkdir -p /etc/ceph 2024-08-19T01:25:01.883 DEBUG:teuthology.orchestra.run.smithi083:> sudo chmod 777 /etc/ceph 2024-08-19T01:25:01.942 DEBUG:teuthology.orchestra.run.smithi117:> sudo chmod 777 /etc/ceph 2024-08-19T01:25:01.972 DEBUG:teuthology.orchestra.run.smithi167:> sudo chmod 777 /etc/ceph 2024-08-19T01:25:01.997 INFO:tasks.cephadm:Writing seed config... 2024-08-19T01:25:01.998 INFO:tasks.cephadm: override: [client] client mount timeout = 600 2024-08-19T01:25:01.998 INFO:tasks.cephadm: override: [client] debug client = 20 2024-08-19T01:25:01.998 INFO:tasks.cephadm: override: [client] debug ms = 1 2024-08-19T01:25:01.998 INFO:tasks.cephadm: override: [client] rados mon op timeout = 900 2024-08-19T01:25:01.998 INFO:tasks.cephadm: override: [client] rados osd op timeout = 900 2024-08-19T01:25:01.998 INFO:tasks.cephadm: override: [global] ms die on skipped message = False 2024-08-19T01:25:01.998 INFO:tasks.cephadm: override: [mds] debug mds = 20 2024-08-19T01:25:01.999 INFO:tasks.cephadm: override: [mds] debug mds balancer = 20 2024-08-19T01:25:01.999 INFO:tasks.cephadm: override: [mds] debug ms = 1 2024-08-19T01:25:01.999 INFO:tasks.cephadm: override: [mds] mds bal fragment size max = 10000 2024-08-19T01:25:01.999 INFO:tasks.cephadm: override: [mds] mds bal merge size = 5 2024-08-19T01:25:01.999 INFO:tasks.cephadm: override: [mds] mds bal split bits = 3 2024-08-19T01:25:01.999 INFO:tasks.cephadm: override: [mds] mds bal split size = 100 2024-08-19T01:25:01.999 INFO:tasks.cephadm: override: [mds] mds debug frag = True 2024-08-19T01:25:01.999 INFO:tasks.cephadm: override: [mds] mds debug scatterstat = True 2024-08-19T01:25:01.999 INFO:tasks.cephadm: override: [mds] mds op complaint time = 180 2024-08-19T01:25:01.999 INFO:tasks.cephadm: override: [mds] mds verify scatter = True 2024-08-19T01:25:01.999 INFO:tasks.cephadm: override: [mds] mds_export_ephemeral_random_max = 0.1 2024-08-19T01:25:01.999 INFO:tasks.cephadm: override: [mds] osd op complaint time = 180 2024-08-19T01:25:01.999 INFO:tasks.cephadm: override: [mds] rados mon op timeout = 900 2024-08-19T01:25:01.999 INFO:tasks.cephadm: override: [mds] rados osd op timeout = 900 2024-08-19T01:25:01.999 INFO:tasks.cephadm: override: [mgr] cephsqlite lock renewal timeout = 900000 2024-08-19T01:25:01.999 INFO:tasks.cephadm: override: [mgr] debug client = 20 2024-08-19T01:25:02.000 INFO:tasks.cephadm: override: [mgr] debug mgr = 20 2024-08-19T01:25:02.000 INFO:tasks.cephadm: override: [mgr] debug ms = 1 2024-08-19T01:25:02.000 INFO:tasks.cephadm: override: [mon] debug mon = 20 2024-08-19T01:25:02.000 INFO:tasks.cephadm: override: [mon] debug ms = 1 2024-08-19T01:25:02.000 INFO:tasks.cephadm: override: [mon] debug paxos = 20 2024-08-19T01:25:02.000 INFO:tasks.cephadm: override: [mon] mon down mkfs grace = 300 2024-08-19T01:25:02.000 INFO:tasks.cephadm: override: [mon] mon op complaint time = 120 2024-08-19T01:25:02.000 INFO:tasks.cephadm: override: [mon] mon warn on pool no app = False 2024-08-19T01:25:02.000 INFO:tasks.cephadm: override: [osd] bdev async discard = True 2024-08-19T01:25:02.000 INFO:tasks.cephadm: override: [osd] bdev enable discard = True 2024-08-19T01:25:02.000 INFO:tasks.cephadm: override: [osd] bluestore allocator = bitmap 2024-08-19T01:25:02.000 INFO:tasks.cephadm: override: [osd] bluestore block size = 96636764160 2024-08-19T01:25:02.000 INFO:tasks.cephadm: override: [osd] bluestore fsck on mount = True 2024-08-19T01:25:02.000 INFO:tasks.cephadm: override: [osd] debug bluefs = 1/20 2024-08-19T01:25:02.001 INFO:tasks.cephadm: override: [osd] debug bluestore = 1/20 2024-08-19T01:25:02.001 INFO:tasks.cephadm: override: [osd] debug ms = 1 2024-08-19T01:25:02.001 INFO:tasks.cephadm: override: [osd] debug osd = 20 2024-08-19T01:25:02.001 INFO:tasks.cephadm: override: [osd] debug rocksdb = 4/10 2024-08-19T01:25:02.001 INFO:tasks.cephadm: override: [osd] mon osd backfillfull_ratio = 0.85 2024-08-19T01:25:02.001 INFO:tasks.cephadm: override: [osd] mon osd full ratio = 0.9 2024-08-19T01:25:02.001 INFO:tasks.cephadm: override: [osd] mon osd nearfull ratio = 0.8 2024-08-19T01:25:02.001 INFO:tasks.cephadm: override: [osd] osd failsafe full ratio = 0.95 2024-08-19T01:25:02.001 INFO:tasks.cephadm: override: [osd] osd objectstore = bluestore 2024-08-19T01:25:02.001 INFO:tasks.cephadm: override: [osd] osd op complaint time = 180 2024-08-19T01:25:02.001 INFO:tasks.cephadm: override: [osd] osd shutdown pgref assert = True 2024-08-19T01:25:02.001 INFO:tasks.cephadm: override: [osd] osd_max_omap_entries_per_request = 10 2024-08-19T01:25:02.002 DEBUG:teuthology.orchestra.run.smithi083:> set -ex 2024-08-19T01:25:02.002 DEBUG:teuthology.orchestra.run.smithi083:> dd of=/home/ubuntu/cephtest/seed.ceph.conf 2024-08-19T01:25:02.037 DEBUG:tasks.cephadm:Final config: [global] # make logging friendly to teuthology log_to_file = true log_to_stderr = false log to journald = false mon cluster log to file = true mon cluster log file level = debug mon clock drift allowed = 1.000 # replicate across OSDs, not hosts osd crush chooseleaf type = 0 #osd pool default size = 2 osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd # enable some debugging auth debug = true ms die on old message = true ms die on bug = true debug asserts on shutdown = true # adjust warnings mon max pg per osd = 10000# >= luminous mon pg warn max object skew = 0 mon osd allow primary affinity = true mon osd allow pg remap = true mon warn on legacy crush tunables = false mon warn on crush straw calc version zero = false mon warn on no sortbitwise = false mon warn on osd down out interval zero = false mon warn on too few osds = false mon_warn_on_pool_pg_num_not_power_of_two = false # disable pg_autoscaler by default for new pools osd_pool_default_pg_autoscale_mode = off # tests delete pools mon allow pool delete = true fsid = bad8780e-5dc9-11ef-bccf-c7b262605968 ms die on skipped message = False [osd] osd scrub load threshold = 5.0 osd scrub max interval = 600 osd mclock profile = high_recovery_ops osd recover clone overlap = true osd recovery max chunk = 1048576 osd deep scrub update digest min age = 30 osd map max advance = 10 osd memory target autotune = true # debugging osd debug shutdown = true osd debug op order = true osd debug verify stray on activate = true osd debug pg log writeout = true osd debug verify cached snaps = true osd debug verify missing on start = true osd debug misdirected ops = true osd op queue = debug_random osd op queue cut off = debug_random osd shutdown pgref assert = True bdev debug aio = true osd sloppy crc = true bdev async discard = True bdev enable discard = True bluestore allocator = bitmap bluestore block size = 96636764160 bluestore fsck on mount = True debug bluefs = 1/20 debug bluestore = 1/20 debug ms = 1 debug osd = 20 debug rocksdb = 4/10 mon osd backfillfull_ratio = 0.85 mon osd full ratio = 0.9 mon osd nearfull ratio = 0.8 osd failsafe full ratio = 0.95 osd objectstore = bluestore osd op complaint time = 180 osd_max_omap_entries_per_request = 10 [mgr] mon reweight min pgs per osd = 4 mon reweight min bytes per osd = 10 mgr/telemetry/nag = false cephsqlite lock renewal timeout = 900000 debug client = 20 debug mgr = 20 debug ms = 1 [mon] mon data avail warn = 5 mon mgr mkfs grace = 240 mon reweight min pgs per osd = 4 mon osd reporter subtree level = osd mon osd prime pg temp = true mon reweight min bytes per osd = 10 # rotate auth tickets quickly to exercise renewal paths auth mon ticket ttl = 660# 11m auth service ticket ttl = 240# 4m # don't complain about global id reclaim mon_warn_on_insecure_global_id_reclaim = false mon_warn_on_insecure_global_id_reclaim_allowed = false debug mon = 20 debug ms = 1 debug paxos = 20 mon down mkfs grace = 300 mon op complaint time = 120 mon warn on pool no app = False [client.rgw] rgw cache enabled = true rgw enable ops log = true rgw enable usage log = true [client] client mount timeout = 600 debug client = 20 debug ms = 1 rados mon op timeout = 900 rados osd op timeout = 900 [mds] debug mds = 20 debug mds balancer = 20 debug ms = 1 mds bal fragment size max = 10000 mds bal merge size = 5 mds bal split bits = 3 mds bal split size = 100 mds debug frag = True mds debug scatterstat = True mds op complaint time = 180 mds verify scatter = True mds_export_ephemeral_random_max = 0.1 osd op complaint time = 180 rados mon op timeout = 900 rados osd op timeout = 900 2024-08-19T01:25:02.037 DEBUG:teuthology.orchestra.run.smithi083:mon.a> sudo journalctl -f -n 0 -u ceph-bad8780e-5dc9-11ef-bccf-c7b262605968@mon.a.service 2024-08-19T01:25:02.081 DEBUG:teuthology.orchestra.run.smithi083:mgr.x> sudo journalctl -f -n 0 -u ceph-bad8780e-5dc9-11ef-bccf-c7b262605968@mgr.x.service 2024-08-19T01:25:02.125 INFO:tasks.cephadm:Bootstrapping... 2024-08-19T01:25:02.125 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833 -v bootstrap --fsid bad8780e-5dc9-11ef-bccf-c7b262605968 --config /home/ubuntu/cephtest/seed.ceph.conf --output-config /etc/ceph/ceph.conf --output-keyring /etc/ceph/ceph.client.admin.keyring --output-pub-ssh-key /home/ubuntu/cephtest/ceph.pub --mon-id a --mgr-id x --orphan-initial-daemons --skip-monitoring-stack --mon-ip 172.21.15.83 --skip-admin-label && sudo chmod +r /etc/ceph/ceph.client.admin.keyring 2024-08-19T01:25:02.386 INFO:teuthology.orchestra.run.smithi083.stdout:-------------------------------------------------------------------------------- 2024-08-19T01:25:02.386 INFO:teuthology.orchestra.run.smithi083.stdout:cephadm ['--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833', '-v', 'bootstrap', '--fsid', 'bad8780e-5dc9-11ef-bccf-c7b262605968', '--config', '/home/ubuntu/cephtest/seed.ceph.conf', '--output-config', '/etc/ceph/ceph.conf', '--output-keyring', '/etc/ceph/ceph.client.admin.keyring', '--output-pub-ssh-key', '/home/ubuntu/cephtest/ceph.pub', '--mon-id', 'a', '--mgr-id', 'x', '--orphan-initial-daemons', '--skip-monitoring-stack', '--mon-ip', '172.21.15.83', '--skip-admin-label'] 2024-08-19T01:25:02.386 INFO:teuthology.orchestra.run.smithi083.stderr:Specifying an fsid for your cluster offers no advantages and may increase the likelihood of fsid conflicts. 2024-08-19T01:25:02.387 INFO:teuthology.orchestra.run.smithi083.stdout:Verifying podman|docker is present... 2024-08-19T01:25:02.434 INFO:teuthology.orchestra.run.smithi083.stdout:/bin/podman: stdout 5.1.2 2024-08-19T01:25:02.434 INFO:teuthology.orchestra.run.smithi083.stdout:Verifying lvm2 is present... 2024-08-19T01:25:02.434 INFO:teuthology.orchestra.run.smithi083.stdout:Verifying time synchronization is in place... 2024-08-19T01:25:02.448 INFO:teuthology.orchestra.run.smithi083.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-08-19T01:25:02.448 INFO:teuthology.orchestra.run.smithi083.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-08-19T01:25:02.460 INFO:teuthology.orchestra.run.smithi083.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-08-19T01:25:02.460 INFO:teuthology.orchestra.run.smithi083.stdout:systemctl: stdout inactive 2024-08-19T01:25:02.474 INFO:teuthology.orchestra.run.smithi083.stdout:systemctl: stdout enabled 2024-08-19T01:25:02.486 INFO:teuthology.orchestra.run.smithi083.stdout:systemctl: stdout active 2024-08-19T01:25:02.486 INFO:teuthology.orchestra.run.smithi083.stdout:Unit chronyd.service is enabled and running 2024-08-19T01:25:02.486 INFO:teuthology.orchestra.run.smithi083.stdout:Repeating the final host check... 2024-08-19T01:25:02.576 INFO:teuthology.orchestra.run.smithi083.stdout:/bin/podman: stdout 5.1.2 2024-08-19T01:25:02.577 INFO:teuthology.orchestra.run.smithi083.stdout:podman (/bin/podman) version 5.1.2 is present 2024-08-19T01:25:02.577 INFO:teuthology.orchestra.run.smithi083.stdout:systemctl is present 2024-08-19T01:25:02.577 INFO:teuthology.orchestra.run.smithi083.stdout:lvcreate is present 2024-08-19T01:25:02.589 INFO:teuthology.orchestra.run.smithi083.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-08-19T01:25:02.589 INFO:teuthology.orchestra.run.smithi083.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-08-19T01:25:02.601 INFO:teuthology.orchestra.run.smithi083.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-08-19T01:25:02.601 INFO:teuthology.orchestra.run.smithi083.stdout:systemctl: stdout inactive 2024-08-19T01:25:02.615 INFO:teuthology.orchestra.run.smithi083.stdout:systemctl: stdout enabled 2024-08-19T01:25:02.627 INFO:teuthology.orchestra.run.smithi083.stdout:systemctl: stdout active 2024-08-19T01:25:02.627 INFO:teuthology.orchestra.run.smithi083.stdout:Unit chronyd.service is enabled and running 2024-08-19T01:25:02.627 INFO:teuthology.orchestra.run.smithi083.stdout:Host looks OK 2024-08-19T01:25:02.628 INFO:teuthology.orchestra.run.smithi083.stdout:Cluster fsid: bad8780e-5dc9-11ef-bccf-c7b262605968 2024-08-19T01:25:02.628 INFO:teuthology.orchestra.run.smithi083.stdout:Acquiring lock 139746772113920 on /run/cephadm/bad8780e-5dc9-11ef-bccf-c7b262605968.lock 2024-08-19T01:25:02.628 INFO:teuthology.orchestra.run.smithi083.stdout:Lock 139746772113920 acquired on /run/cephadm/bad8780e-5dc9-11ef-bccf-c7b262605968.lock 2024-08-19T01:25:02.628 INFO:teuthology.orchestra.run.smithi083.stdout:Verifying IP 172.21.15.83 port 3300 ... 2024-08-19T01:25:02.629 INFO:teuthology.orchestra.run.smithi083.stdout:Verifying IP 172.21.15.83 port 6789 ... 2024-08-19T01:25:02.629 INFO:teuthology.orchestra.run.smithi083.stdout:Base mon IP(s) is [172.21.15.83:3300, 172.21.15.83:6789], mon addrv is [v2:172.21.15.83:3300,v1:172.21.15.83:6789] 2024-08-19T01:25:02.636 INFO:teuthology.orchestra.run.smithi083.stdout:/sbin/ip: stdout default via 172.21.15.254 dev enp3s0f1 proto dhcp src 172.21.15.83 metric 100 2024-08-19T01:25:02.636 INFO:teuthology.orchestra.run.smithi083.stdout:/sbin/ip: stdout 172.21.0.0/20 dev enp3s0f1 proto kernel scope link src 172.21.15.83 metric 100 2024-08-19T01:25:02.642 INFO:teuthology.orchestra.run.smithi083.stdout:/sbin/ip: stdout ::1 dev lo proto kernel metric 256 pref medium 2024-08-19T01:25:02.642 INFO:teuthology.orchestra.run.smithi083.stdout:/sbin/ip: stdout fe80::/64 dev enp3s0f1 proto kernel metric 1024 pref medium 2024-08-19T01:25:02.642 INFO:teuthology.orchestra.run.smithi083.stdout:/sbin/ip: stdout default via fe80::327c:5e00:6487:71e0 dev enp3s0f1 proto ra metric 100 pref medium 2024-08-19T01:25:02.648 INFO:teuthology.orchestra.run.smithi083.stdout:/sbin/ip: stdout 1: lo: mtu 65536 state UNKNOWN qlen 1000 2024-08-19T01:25:02.648 INFO:teuthology.orchestra.run.smithi083.stdout:/sbin/ip: stdout inet6 ::1/128 scope host 2024-08-19T01:25:02.648 INFO:teuthology.orchestra.run.smithi083.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-08-19T01:25:02.648 INFO:teuthology.orchestra.run.smithi083.stdout:/sbin/ip: stdout 5: enp3s0f1: mtu 1500 state UP qlen 1000 2024-08-19T01:25:02.648 INFO:teuthology.orchestra.run.smithi083.stdout:/sbin/ip: stdout inet6 fe80::ec4:7aff:fe8f:ce91/64 scope link noprefixroute 2024-08-19T01:25:02.648 INFO:teuthology.orchestra.run.smithi083.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-08-19T01:25:02.649 INFO:teuthology.orchestra.run.smithi083.stdout:Mon IP `172.21.15.83` is in CIDR network `172.21.0.0/20` 2024-08-19T01:25:02.649 INFO:teuthology.orchestra.run.smithi083.stdout:Mon IP `172.21.15.83` is in CIDR network `172.21.0.0/20` 2024-08-19T01:25:02.649 INFO:teuthology.orchestra.run.smithi083.stdout:Inferred mon public CIDR from local network configuration ['172.21.0.0/20', '172.21.0.0/20'] 2024-08-19T01:25:02.650 INFO:teuthology.orchestra.run.smithi083.stdout:Internal network (--cluster-network) has not been provided, OSD replication will default to the public_network 2024-08-19T01:25:02.651 INFO:teuthology.orchestra.run.smithi083.stdout:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833... 2024-08-19T01:25:03.189 INFO:teuthology.orchestra.run.smithi083.stdout:/bin/podman: stdout ea7a5189f30bad3aca9524788e2965d2368d7ce786df09d4edcd8c3edf9cefba 2024-08-19T01:25:03.189 INFO:teuthology.orchestra.run.smithi083.stdout:/bin/podman: stderr Trying to pull quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833... 2024-08-19T01:25:03.189 INFO:teuthology.orchestra.run.smithi083.stdout:/bin/podman: stderr Getting image source signatures 2024-08-19T01:25:03.189 INFO:teuthology.orchestra.run.smithi083.stdout:/bin/podman: stderr Copying blob sha256:7556061847b3cc957dc118ff2516cc1cd809c4272dcb16074c5c7fe28afc4774 2024-08-19T01:25:03.190 INFO:teuthology.orchestra.run.smithi083.stdout:/bin/podman: stderr Copying blob sha256:e8b54c863393b7a8216a71737771b73b16a8e43ec7acf927c7faa175c255e551 2024-08-19T01:25:03.190 INFO:teuthology.orchestra.run.smithi083.stdout:/bin/podman: stderr Copying config sha256:ea7a5189f30bad3aca9524788e2965d2368d7ce786df09d4edcd8c3edf9cefba 2024-08-19T01:25:03.190 INFO:teuthology.orchestra.run.smithi083.stdout:/bin/podman: stderr Writing manifest to image destination 2024-08-19T01:25:04.284 INFO:teuthology.orchestra.run.smithi083.stdout:ceph: stdout ceph version 19.1.0-1382-gde09dd01 (de09dd018de4baddfa6ca4d43bc8fb784cf75833) squid (rc) 2024-08-19T01:25:04.284 INFO:teuthology.orchestra.run.smithi083.stdout:Ceph version: ceph version 19.1.0-1382-gde09dd01 (de09dd018de4baddfa6ca4d43bc8fb784cf75833) squid (rc) 2024-08-19T01:25:04.284 INFO:teuthology.orchestra.run.smithi083.stdout:Extracting ceph user uid/gid from container image... 2024-08-19T01:25:05.407 INFO:teuthology.orchestra.run.smithi083.stdout:stat: stdout 167 167 2024-08-19T01:25:05.407 INFO:teuthology.orchestra.run.smithi083.stdout:Creating initial keys... 2024-08-19T01:25:06.463 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph-authtool: stdout AQDxnsJm2jR7LxAAWhOaAKakw0PgYtvg1uCgDg== 2024-08-19T01:25:07.543 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph-authtool: stdout AQDynsJmHzZDNBAAE1iUoCL7K26y0rKlAU6PRQ== 2024-08-19T01:25:08.624 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph-authtool: stdout AQDznsJm0X4TOBAA6uMjc0yGuji+R5TzSYkCVg== 2024-08-19T01:25:08.625 INFO:teuthology.orchestra.run.smithi083.stdout:Creating initial monmap... 2024-08-19T01:25:09.629 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: monmap file /tmp/monmap 2024-08-19T01:25:09.630 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/monmaptool: stdout setting min_mon_release = quincy 2024-08-19T01:25:09.630 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: set fsid to bad8780e-5dc9-11ef-bccf-c7b262605968 2024-08-19T01:25:09.630 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-08-19T01:25:09.630 INFO:teuthology.orchestra.run.smithi083.stdout:monmaptool for a [v2:172.21.15.83:3300,v1:172.21.15.83:6789] on /usr/bin/monmaptool: monmap file /tmp/monmap 2024-08-19T01:25:09.630 INFO:teuthology.orchestra.run.smithi083.stdout:setting min_mon_release = quincy 2024-08-19T01:25:09.630 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/monmaptool: set fsid to bad8780e-5dc9-11ef-bccf-c7b262605968 2024-08-19T01:25:09.630 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-08-19T01:25:09.630 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-08-19T01:25:09.630 INFO:teuthology.orchestra.run.smithi083.stdout:Creating mon... 2024-08-19T01:25:10.611 INFO:teuthology.orchestra.run.smithi083.stdout:create mon.a on 2024-08-19T01:25:10.903 INFO:teuthology.orchestra.run.smithi083.stdout:systemctl: stderr Removed "/etc/systemd/system/multi-user.target.wants/ceph.target". 2024-08-19T01:25:11.126 INFO:teuthology.orchestra.run.smithi083.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /etc/systemd/system/ceph.target. 2024-08-19T01:25:11.393 INFO:teuthology.orchestra.run.smithi083.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph-bad8780e-5dc9-11ef-bccf-c7b262605968.target → /etc/systemd/system/ceph-bad8780e-5dc9-11ef-bccf-c7b262605968.target. 2024-08-19T01:25:11.393 INFO:teuthology.orchestra.run.smithi083.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph.target.wants/ceph-bad8780e-5dc9-11ef-bccf-c7b262605968.target → /etc/systemd/system/ceph-bad8780e-5dc9-11ef-bccf-c7b262605968.target. 2024-08-19T01:25:11.719 INFO:teuthology.orchestra.run.smithi083.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-bad8780e-5dc9-11ef-bccf-c7b262605968@mon.a 2024-08-19T01:25:11.720 INFO:teuthology.orchestra.run.smithi083.stdout:systemctl: stderr Failed to reset failed state of unit ceph-bad8780e-5dc9-11ef-bccf-c7b262605968@mon.a.service: Unit ceph-bad8780e-5dc9-11ef-bccf-c7b262605968@mon.a.service not loaded. 2024-08-19T01:25:11.980 INFO:teuthology.orchestra.run.smithi083.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-bad8780e-5dc9-11ef-bccf-c7b262605968.target.wants/ceph-bad8780e-5dc9-11ef-bccf-c7b262605968@mon.a.service → /etc/systemd/system/ceph-bad8780e-5dc9-11ef-bccf-c7b262605968@.service. 2024-08-19T01:25:12.631 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:25:12 smithi083 podman[16725]: 2024-08-19 01:25:12.565718657 +0000 UTC m=+0.350709121 container start 65e09ec4c7ae5eddde361e3147866283e071589bf995639312568f50d39e58f5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833, name=ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-mon-a, org.label-schema.schema-version=1.0, GIT_CLEAN=True, ceph=True, io.buildah.version=1.36.0, org.label-schema.vendor=CentOS, RELEASE=squid-de09dd0, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD) 2024-08-19T01:25:12.678 INFO:teuthology.orchestra.run.smithi083.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-08-19T01:25:12.678 INFO:teuthology.orchestra.run.smithi083.stdout:systemctl: stdout disabled 2024-08-19T01:25:12.691 INFO:teuthology.orchestra.run.smithi083.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-08-19T01:25:12.691 INFO:teuthology.orchestra.run.smithi083.stdout:systemctl: stdout inactive 2024-08-19T01:25:12.691 INFO:teuthology.orchestra.run.smithi083.stdout:firewalld.service is not enabled 2024-08-19T01:25:12.691 INFO:teuthology.orchestra.run.smithi083.stdout:Not possible to enable service . firewalld.service is not available 2024-08-19T01:25:12.692 INFO:teuthology.orchestra.run.smithi083.stdout:Waiting for mon to start... 2024-08-19T01:25:12.692 INFO:teuthology.orchestra.run.smithi083.stdout:Waiting for mon... 2024-08-19T01:25:12.886 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:25:12 smithi083 ceph-mon[16745]: mkfs bad8780e-5dc9-11ef-bccf-c7b262605968 2024-08-19T01:25:12.886 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:25:12 smithi083 ceph-mon[16745]: mon.a is new leader, mons a in quorum (ranks 0) 2024-08-19T01:25:12.886 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:25:12 smithi083 bash[16725]: 65e09ec4c7ae5eddde361e3147866283e071589bf995639312568f50d39e58f5 2024-08-19T01:25:12.886 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:25:12 smithi083 systemd[1]: Started Ceph mon.a for bad8780e-5dc9-11ef-bccf-c7b262605968. 2024-08-19T01:25:13.774 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:25:13 smithi083 ceph-mon[16745]: mon.a is new leader, mons a in quorum (ranks 0) 2024-08-19T01:25:13.774 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:25:13 smithi083 ceph-mon[16745]: monmap epoch 1 2024-08-19T01:25:13.774 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:25:13 smithi083 ceph-mon[16745]: fsid bad8780e-5dc9-11ef-bccf-c7b262605968 2024-08-19T01:25:13.774 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:25:13 smithi083 ceph-mon[16745]: last_changed 2024-08-19T01:25:09.034517+0000 2024-08-19T01:25:13.774 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:25:13 smithi083 ceph-mon[16745]: created 2024-08-19T01:25:09.034517+0000 2024-08-19T01:25:13.774 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:25:13 smithi083 ceph-mon[16745]: min_mon_release 19 (squid) 2024-08-19T01:25:13.774 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:25:13 smithi083 ceph-mon[16745]: election_strategy: 1 2024-08-19T01:25:13.774 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:25:13 smithi083 ceph-mon[16745]: 0: [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] mon.a 2024-08-19T01:25:13.774 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:25:13 smithi083 ceph-mon[16745]: fsmap 2024-08-19T01:25:13.774 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:25:13 smithi083 ceph-mon[16745]: osdmap e1: 0 total, 0 up, 0 in 2024-08-19T01:25:13.775 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:25:13 smithi083 ceph-mon[16745]: mgrmap e1: no daemons active 2024-08-19T01:25:13.775 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:25:13 smithi083 ceph-mon[16745]: from='client.? 172.21.15.83:0/1190905893' entity='client.admin' cmd=[{"prefix": "status"}]: dispatch 2024-08-19T01:25:14.145 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout cluster: 2024-08-19T01:25:14.145 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout id: bad8780e-5dc9-11ef-bccf-c7b262605968 2024-08-19T01:25:14.145 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout health: HEALTH_OK 2024-08-19T01:25:14.145 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout 2024-08-19T01:25:14.145 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout services: 2024-08-19T01:25:14.145 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout mon: 1 daemons, quorum a (age 0.682853s) 2024-08-19T01:25:14.145 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout mgr: no daemons active 2024-08-19T01:25:14.145 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout osd: 0 osds: 0 up, 0 in 2024-08-19T01:25:14.145 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout 2024-08-19T01:25:14.146 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout data: 2024-08-19T01:25:14.146 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout pools: 0 pools, 0 pgs 2024-08-19T01:25:14.146 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout objects: 0 objects, 0 B 2024-08-19T01:25:14.146 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout usage: 0 B used, 0 B / 0 B avail 2024-08-19T01:25:14.146 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout pgs: 2024-08-19T01:25:14.146 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout 2024-08-19T01:25:14.146 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:13.228+0000 7f86b8200640 1 Processor -- start 2024-08-19T01:25:14.146 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:13.229+0000 7f86b8200640 1 -- start start 2024-08-19T01:25:14.146 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:13.229+0000 7f86b8200640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f86b010aac0 0x7f86b010aec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:25:14.146 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:13.229+0000 7f86b8200640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f86b010b490 con 0x7f86b010aac0 2024-08-19T01:25:14.146 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:13.230+0000 7f86b5800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f86b010aac0 0x7f86b010aec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:25:14.146 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:13.230+0000 7f86b5800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f86b010aac0 0x7f86b010aec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:41190/0 (socket says 172.21.15.83:41190) 2024-08-19T01:25:14.147 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:13.230+0000 7f86b5800640 1 -- 172.21.15.83:0/2463466399 learned_addr learned my addr 172.21.15.83:0/2463466399 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-08-19T01:25:14.147 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:13.230+0000 7f86b5800640 1 -- 172.21.15.83:0/2463466399 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f86b010b610 con 0x7f86b010aac0 2024-08-19T01:25:14.147 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:13.235+0000 7f86b5800640 1 --2- 172.21.15.83:0/2463466399 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f86b010aac0 0x7f86b010aec0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f86a4009820 tx=0x7f86a402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=854778c9997a5b00 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:25:14.147 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:13.235+0000 7f86afe00640 1 -- 172.21.15.83:0/2463466399 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f86a4037070 con 0x7f86b010aac0 2024-08-19T01:25:14.147 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:13.235+0000 7f86afe00640 1 -- 172.21.15.83:0/2463466399 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f86a4032440 con 0x7f86b010aac0 2024-08-19T01:25:14.147 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:13.235+0000 7f86afe00640 1 -- 172.21.15.83:0/2463466399 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f86a4032740 con 0x7f86b010aac0 2024-08-19T01:25:14.147 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:13.236+0000 7f86b8200640 1 -- 172.21.15.83:0/2463466399 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f86b010aac0 msgr2=0x7f86b010aec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:25:14.147 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:13.236+0000 7f86b8200640 1 --2- 172.21.15.83:0/2463466399 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f86b010aac0 0x7f86b010aec0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f86a4009820 tx=0x7f86a402a270 comp rx=0 tx=0).stop 2024-08-19T01:25:14.147 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:13.236+0000 7f86b8200640 1 -- 172.21.15.83:0/2463466399 shutdown_connections 2024-08-19T01:25:14.147 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:13.236+0000 7f86b8200640 1 --2- 172.21.15.83:0/2463466399 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f86b010aac0 0x7f86b010aec0 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:25:14.147 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:13.236+0000 7f86b8200640 1 -- 172.21.15.83:0/2463466399 >> 172.21.15.83:0/2463466399 conn(0x7f86b007cc40 msgr2=0x7f86b007d050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:25:14.148 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:13.236+0000 7f86b8200640 1 -- 172.21.15.83:0/2463466399 shutdown_connections 2024-08-19T01:25:14.148 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:13.236+0000 7f86b8200640 1 -- 172.21.15.83:0/2463466399 wait complete. 2024-08-19T01:25:14.148 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:13.237+0000 7f86b8200640 1 Processor -- start 2024-08-19T01:25:14.148 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:13.237+0000 7f86b8200640 1 -- start start 2024-08-19T01:25:14.148 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:13.238+0000 7f86b8200640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f86b010aac0 0x7f86b019ce10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:25:14.148 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:13.238+0000 7f86b8200640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f86b0109900 con 0x7f86b010aac0 2024-08-19T01:25:14.148 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:13.238+0000 7f86b5800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f86b010aac0 0x7f86b019ce10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:25:14.148 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:13.238+0000 7f86b5800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f86b010aac0 0x7f86b019ce10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:41200/0 (socket says 172.21.15.83:41200) 2024-08-19T01:25:14.148 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:13.238+0000 7f86b5800640 1 -- 172.21.15.83:0/1190905893 learned_addr learned my addr 172.21.15.83:0/1190905893 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-08-19T01:25:14.148 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:13.238+0000 7f86b5800640 1 -- 172.21.15.83:0/1190905893 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f86b019d350 con 0x7f86b010aac0 2024-08-19T01:25:14.148 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:13.239+0000 7f86b5800640 1 --2- 172.21.15.83:0/1190905893 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f86b010aac0 0x7f86b019ce10 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f86a40090b0 tx=0x7f86a40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:25:14.148 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:13.239+0000 7f86ae000640 1 -- 172.21.15.83:0/1190905893 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f86a4041070 con 0x7f86b010aac0 2024-08-19T01:25:14.149 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:13.239+0000 7f86ae000640 1 -- 172.21.15.83:0/1190905893 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f86a403c7a0 con 0x7f86b010aac0 2024-08-19T01:25:14.149 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:13.239+0000 7f86ae000640 1 -- 172.21.15.83:0/1190905893 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f86a4037040 con 0x7f86b010aac0 2024-08-19T01:25:14.149 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:13.239+0000 7f86b8200640 1 -- 172.21.15.83:0/1190905893 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f86b019d5e0 con 0x7f86b010aac0 2024-08-19T01:25:14.149 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:13.239+0000 7f86b8200640 1 -- 172.21.15.83:0/1190905893 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f86b01a02d0 con 0x7f86b010aac0 2024-08-19T01:25:14.149 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:13.239+0000 7f86ae000640 1 -- 172.21.15.83:0/1190905893 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f86a4044070 con 0x7f86b010aac0 2024-08-19T01:25:14.149 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:13.240+0000 7f86ae000640 1 -- 172.21.15.83:0/1190905893 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f86a403d730 con 0x7f86b010aac0 2024-08-19T01:25:14.149 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:13.240+0000 7f86b8200640 1 -- 172.21.15.83:0/1190905893 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f86b010af40 con 0x7f86b010aac0 2024-08-19T01:25:14.149 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:13.243+0000 7f86ae000640 1 -- 172.21.15.83:0/1190905893 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73482 (secure 0 0 0) 0x7f86a403caa0 con 0x7f86b010aac0 2024-08-19T01:25:14.149 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:13.306+0000 7f86b8200640 1 -- 172.21.15.83:0/1190905893 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "status"} v 0) -- 0x7f86b01a04a0 con 0x7f86b010aac0 2024-08-19T01:25:14.149 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:13.307+0000 7f86ae000640 1 -- 172.21.15.83:0/1190905893 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "status"}]=0 v0) ==== 54+0+317 (secure 0 0 0) 0x7f86b01a04a0 con 0x7f86b010aac0 2024-08-19T01:25:14.149 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:13.308+0000 7f86b8200640 1 -- 172.21.15.83:0/1190905893 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f86b010aac0 msgr2=0x7f86b019ce10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:25:14.149 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:13.308+0000 7f86b8200640 1 --2- 172.21.15.83:0/1190905893 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f86b010aac0 0x7f86b019ce10 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f86a40090b0 tx=0x7f86a40046c0 comp rx=0 tx=0).stop 2024-08-19T01:25:14.149 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:13.308+0000 7f86b8200640 1 -- 172.21.15.83:0/1190905893 shutdown_connections 2024-08-19T01:25:14.150 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:13.308+0000 7f86b8200640 1 --2- 172.21.15.83:0/1190905893 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f86b010aac0 0x7f86b019ce10 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:25:14.150 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:13.308+0000 7f86b8200640 1 -- 172.21.15.83:0/1190905893 >> 172.21.15.83:0/1190905893 conn(0x7f86b007cc40 msgr2=0x7f86b0191170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:25:14.150 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:13.308+0000 7f86b8200640 1 -- 172.21.15.83:0/1190905893 shutdown_connections 2024-08-19T01:25:14.150 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:13.308+0000 7f86b8200640 1 -- 172.21.15.83:0/1190905893 wait complete. 2024-08-19T01:25:14.150 INFO:teuthology.orchestra.run.smithi083.stdout:mon is available 2024-08-19T01:25:14.150 INFO:teuthology.orchestra.run.smithi083.stdout:Assimilating anything we can from ceph.conf... 2024-08-19T01:25:15.081 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:25:14 smithi083 ceph-mon[16745]: from='client.? 172.21.15.83:0/3491764523' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2024-08-19T01:25:15.081 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:25:14 smithi083 ceph-mon[16745]: from='client.? 172.21.15.83:0/3491764523' entity='client.admin' cmd='[{"prefix": "config assimilate-conf"}]': finished 2024-08-19T01:25:15.397 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout 2024-08-19T01:25:15.397 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout [global] 2024-08-19T01:25:15.397 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout fsid = bad8780e-5dc9-11ef-bccf-c7b262605968 2024-08-19T01:25:15.397 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout mon_host = [v2:172.21.15.83:3300,v1:172.21.15.83:6789] 2024-08-19T01:25:15.397 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-08-19T01:25:15.397 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-08-19T01:25:15.397 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-08-19T01:25:15.397 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-08-19T01:25:15.397 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout 2024-08-19T01:25:15.397 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout [mgr] 2024-08-19T01:25:15.397 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-08-19T01:25:15.398 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout 2024-08-19T01:25:15.398 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout [osd] 2024-08-19T01:25:15.398 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2024-08-19T01:25:15.398 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2024-08-19T01:25:15.398 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2024-08-19T01:25:15.398 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-08-19T01:25:15.398 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-08-19T01:25:15.398 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:14.620+0000 7f60d2e00640 1 Processor -- start 2024-08-19T01:25:15.398 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:14.620+0000 7f60d2e00640 1 -- start start 2024-08-19T01:25:15.398 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:14.620+0000 7f60d2e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f60cc107240 0x7f60cc107640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:25:15.398 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:14.620+0000 7f60d2e00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f60cc107c10 con 0x7f60cc107240 2024-08-19T01:25:15.398 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:14.621+0000 7f60d1a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f60cc107240 0x7f60cc107640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:25:15.399 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:14.621+0000 7f60d1a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f60cc107240 0x7f60cc107640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:41216/0 (socket says 172.21.15.83:41216) 2024-08-19T01:25:15.399 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:14.621+0000 7f60d1a00640 1 -- 172.21.15.83:0/3905203386 learned_addr learned my addr 172.21.15.83:0/3905203386 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-08-19T01:25:15.399 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:14.621+0000 7f60d1a00640 1 -- 172.21.15.83:0/3905203386 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f60cc108440 con 0x7f60cc107240 2024-08-19T01:25:15.399 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:14.623+0000 7f60d1a00640 1 --2- 172.21.15.83:0/3905203386 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f60cc107240 0x7f60cc107640 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7f60c0009820 tx=0x7f60c002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eb66152dc85c2b3f server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:25:15.399 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:14.623+0000 7f60cbe00640 1 -- 172.21.15.83:0/3905203386 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f60c0037070 con 0x7f60cc107240 2024-08-19T01:25:15.399 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:14.623+0000 7f60cbe00640 1 -- 172.21.15.83:0/3905203386 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f60c0032440 con 0x7f60cc107240 2024-08-19T01:25:15.399 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:14.623+0000 7f60cbe00640 1 -- 172.21.15.83:0/3905203386 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f60c0032740 con 0x7f60cc107240 2024-08-19T01:25:15.399 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:14.624+0000 7f60d2e00640 1 -- 172.21.15.83:0/3905203386 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f60cc107240 msgr2=0x7f60cc107640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:25:15.399 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:14.624+0000 7f60d2e00640 1 --2- 172.21.15.83:0/3905203386 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f60cc107240 0x7f60cc107640 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7f60c0009820 tx=0x7f60c002a270 comp rx=0 tx=0).stop 2024-08-19T01:25:15.399 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:14.624+0000 7f60d2e00640 1 -- 172.21.15.83:0/3905203386 shutdown_connections 2024-08-19T01:25:15.399 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:14.624+0000 7f60d2e00640 1 --2- 172.21.15.83:0/3905203386 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f60cc107240 0x7f60cc107640 unknown :-1 s=CLOSED pgs=3 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:25:15.400 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:14.624+0000 7f60d2e00640 1 -- 172.21.15.83:0/3905203386 >> 172.21.15.83:0/3905203386 conn(0x7f60cc1029b0 msgr2=0x7f60cc104e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:25:15.400 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:14.624+0000 7f60d2e00640 1 -- 172.21.15.83:0/3905203386 shutdown_connections 2024-08-19T01:25:15.400 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:14.624+0000 7f60d2e00640 1 -- 172.21.15.83:0/3905203386 wait complete. 2024-08-19T01:25:15.400 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:14.626+0000 7f60d2e00640 1 Processor -- start 2024-08-19T01:25:15.400 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:14.626+0000 7f60d2e00640 1 -- start start 2024-08-19T01:25:15.400 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:14.626+0000 7f60d2e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f60cc107240 0x7f60cc19cc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:25:15.400 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:14.626+0000 7f60d2e00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f60cc1089e0 con 0x7f60cc107240 2024-08-19T01:25:15.400 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:14.626+0000 7f60d1a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f60cc107240 0x7f60cc19cc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:25:15.400 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:14.626+0000 7f60d1a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f60cc107240 0x7f60cc19cc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:41220/0 (socket says 172.21.15.83:41220) 2024-08-19T01:25:15.400 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:14.626+0000 7f60d1a00640 1 -- 172.21.15.83:0/3491764523 learned_addr learned my addr 172.21.15.83:0/3491764523 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-08-19T01:25:15.400 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:14.627+0000 7f60d1a00640 1 -- 172.21.15.83:0/3491764523 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f60cc19d1b0 con 0x7f60cc107240 2024-08-19T01:25:15.400 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:14.627+0000 7f60d1a00640 1 --2- 172.21.15.83:0/3491764523 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f60cc107240 0x7f60cc19cc70 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7f60c00090b0 tx=0x7f60c00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:25:15.401 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:14.627+0000 7f60ca000640 1 -- 172.21.15.83:0/3491764523 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f60c0041070 con 0x7f60cc107240 2024-08-19T01:25:15.401 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:14.627+0000 7f60ca000640 1 -- 172.21.15.83:0/3491764523 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f60c003c7a0 con 0x7f60cc107240 2024-08-19T01:25:15.401 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:14.628+0000 7f60d2e00640 1 -- 172.21.15.83:0/3491764523 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f60cc19d440 con 0x7f60cc107240 2024-08-19T01:25:15.401 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:14.628+0000 7f60ca000640 1 -- 172.21.15.83:0/3491764523 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f60c0037040 con 0x7f60cc107240 2024-08-19T01:25:15.401 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:14.628+0000 7f60d2e00640 1 -- 172.21.15.83:0/3491764523 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f60cc1a0130 con 0x7f60cc107240 2024-08-19T01:25:15.401 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:14.628+0000 7f60ca000640 1 -- 172.21.15.83:0/3491764523 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f60c0044070 con 0x7f60cc107240 2024-08-19T01:25:15.401 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:14.628+0000 7f60ca000640 1 -- 172.21.15.83:0/3491764523 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f60c0046dc0 con 0x7f60cc107240 2024-08-19T01:25:15.401 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:14.629+0000 7f60d2e00640 1 -- 172.21.15.83:0/3491764523 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f60cc109030 con 0x7f60cc107240 2024-08-19T01:25:15.401 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:14.631+0000 7f60ca000640 1 -- 172.21.15.83:0/3491764523 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73482 (secure 0 0 0) 0x7f60c0046450 con 0x7f60cc107240 2024-08-19T01:25:15.401 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:14.691+0000 7f60d2e00640 1 -- 172.21.15.83:0/3491764523 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) -- 0x7f60cc1a1800 con 0x7f60cc107240 2024-08-19T01:25:15.401 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:14.696+0000 7f60ca000640 1 -- 172.21.15.83:0/3491764523 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v2) ==== 70+0+429 (secure 0 0 0) 0x7f60c003caa0 con 0x7f60cc107240 2024-08-19T01:25:15.401 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:14.697+0000 7f60ca000640 1 -- 172.21.15.83:0/3491764523 <== mon.0 v2:172.21.15.83:3300/0 8 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f60c0032440 con 0x7f60cc107240 2024-08-19T01:25:15.401 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:14.698+0000 7f60d2e00640 1 -- 172.21.15.83:0/3491764523 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f60cc107240 msgr2=0x7f60cc19cc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:25:15.402 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:14.698+0000 7f60d2e00640 1 --2- 172.21.15.83:0/3491764523 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f60cc107240 0x7f60cc19cc70 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7f60c00090b0 tx=0x7f60c00046c0 comp rx=0 tx=0).stop 2024-08-19T01:25:15.402 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:14.698+0000 7f60d2e00640 1 -- 172.21.15.83:0/3491764523 shutdown_connections 2024-08-19T01:25:15.402 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:14.698+0000 7f60d2e00640 1 --2- 172.21.15.83:0/3491764523 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f60cc107240 0x7f60cc19cc70 unknown :-1 s=CLOSED pgs=4 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:25:15.402 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:14.698+0000 7f60d2e00640 1 -- 172.21.15.83:0/3491764523 >> 172.21.15.83:0/3491764523 conn(0x7f60cc1029b0 msgr2=0x7f60cc07af50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:25:15.402 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:14.698+0000 7f60d2e00640 1 -- 172.21.15.83:0/3491764523 shutdown_connections 2024-08-19T01:25:15.402 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:14.698+0000 7f60d2e00640 1 -- 172.21.15.83:0/3491764523 wait complete. 2024-08-19T01:25:15.402 INFO:teuthology.orchestra.run.smithi083.stdout:Generating new minimal ceph.conf... 2024-08-19T01:25:16.254 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:25:15 smithi083 ceph-mon[16745]: from='client.? 172.21.15.83:0/1878188486' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-19T01:25:16.529 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:15.864+0000 7f053ca00640 1 Processor -- start 2024-08-19T01:25:16.534 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:15.864+0000 7f053ca00640 1 -- start start 2024-08-19T01:25:16.534 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:15.864+0000 7f053ca00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0538107450 0x7f0538107850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:25:16.534 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:15.864+0000 7f053ca00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f0538107e20 con 0x7f0538107450 2024-08-19T01:25:16.534 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:15.865+0000 7f0535e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0538107450 0x7f0538107850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:25:16.534 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:15.865+0000 7f0535e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0538107450 0x7f0538107850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:41234/0 (socket says 172.21.15.83:41234) 2024-08-19T01:25:16.534 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:15.865+0000 7f0535e00640 1 -- 172.21.15.83:0/4246259240 learned_addr learned my addr 172.21.15.83:0/4246259240 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-08-19T01:25:16.534 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:15.865+0000 7f0535e00640 1 -- 172.21.15.83:0/4246259240 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0538108650 con 0x7f0538107450 2024-08-19T01:25:16.535 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:15.866+0000 7f0535e00640 1 --2- 172.21.15.83:0/4246259240 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0538107450 0x7f0538107850 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f0528009820 tx=0x7f052802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3df4b18f9f98c67a server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:25:16.535 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:15.866+0000 7f0534a00640 1 -- 172.21.15.83:0/4246259240 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0528037070 con 0x7f0538107450 2024-08-19T01:25:16.535 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:15.866+0000 7f0534a00640 1 -- 172.21.15.83:0/4246259240 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f0528032900 con 0x7f0538107450 2024-08-19T01:25:16.535 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:15.866+0000 7f0534a00640 1 -- 172.21.15.83:0/4246259240 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0528030850 con 0x7f0538107450 2024-08-19T01:25:16.535 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:15.867+0000 7f053ca00640 1 -- 172.21.15.83:0/4246259240 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0538107450 msgr2=0x7f0538107850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:25:16.535 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:15.867+0000 7f053ca00640 1 --2- 172.21.15.83:0/4246259240 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0538107450 0x7f0538107850 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f0528009820 tx=0x7f052802a270 comp rx=0 tx=0).stop 2024-08-19T01:25:16.535 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:15.867+0000 7f053ca00640 1 -- 172.21.15.83:0/4246259240 shutdown_connections 2024-08-19T01:25:16.535 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:15.867+0000 7f053ca00640 1 --2- 172.21.15.83:0/4246259240 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0538107450 0x7f0538107850 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:25:16.535 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:15.867+0000 7f053ca00640 1 -- 172.21.15.83:0/4246259240 >> 172.21.15.83:0/4246259240 conn(0x7f0538102c00 msgr2=0x7f0538105020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:25:16.535 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:15.867+0000 7f053ca00640 1 -- 172.21.15.83:0/4246259240 shutdown_connections 2024-08-19T01:25:16.535 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:15.867+0000 7f053ca00640 1 -- 172.21.15.83:0/4246259240 wait complete. 2024-08-19T01:25:16.535 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:15.868+0000 7f053ca00640 1 Processor -- start 2024-08-19T01:25:16.535 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:15.868+0000 7f053ca00640 1 -- start start 2024-08-19T01:25:16.535 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:15.868+0000 7f053ca00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0538107450 0x7f053819b440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:25:16.536 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:15.868+0000 7f053ca00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f053810ba00 con 0x7f0538107450 2024-08-19T01:25:16.536 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:15.869+0000 7f0535e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0538107450 0x7f053819b440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:25:16.536 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:15.869+0000 7f0535e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0538107450 0x7f053819b440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:41250/0 (socket says 172.21.15.83:41250) 2024-08-19T01:25:16.536 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:15.869+0000 7f0535e00640 1 -- 172.21.15.83:0/1878188486 learned_addr learned my addr 172.21.15.83:0/1878188486 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-08-19T01:25:16.536 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:15.869+0000 7f0535e00640 1 -- 172.21.15.83:0/1878188486 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f053819b980 con 0x7f0538107450 2024-08-19T01:25:16.536 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:15.869+0000 7f0535e00640 1 --2- 172.21.15.83:0/1878188486 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0538107450 0x7f053819b440 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f05280097f0 tx=0x7f0528030d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:25:16.536 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:15.869+0000 7f052ea00640 1 -- 172.21.15.83:0/1878188486 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0528037040 con 0x7f0538107450 2024-08-19T01:25:16.536 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:15.869+0000 7f052ea00640 1 -- 172.21.15.83:0/1878188486 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f0528040070 con 0x7f0538107450 2024-08-19T01:25:16.536 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:15.869+0000 7f052ea00640 1 -- 172.21.15.83:0/1878188486 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f052803a390 con 0x7f0538107450 2024-08-19T01:25:16.536 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:15.869+0000 7f053ca00640 1 -- 172.21.15.83:0/1878188486 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f053819bc10 con 0x7f0538107450 2024-08-19T01:25:16.536 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:15.870+0000 7f053ca00640 1 -- 172.21.15.83:0/1878188486 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f053819e900 con 0x7f0538107450 2024-08-19T01:25:16.536 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:15.870+0000 7f052ea00640 1 -- 172.21.15.83:0/1878188486 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f052803a5e0 con 0x7f0538107450 2024-08-19T01:25:16.536 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:15.870+0000 7f052ea00640 1 -- 172.21.15.83:0/1878188486 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f0528045950 con 0x7f0538107450 2024-08-19T01:25:16.536 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:15.871+0000 7f053ca00640 1 -- 172.21.15.83:0/1878188486 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f05381078d0 con 0x7f0538107450 2024-08-19T01:25:16.537 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:15.873+0000 7f052ea00640 1 -- 172.21.15.83:0/1878188486 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73482 (secure 0 0 0) 0x7f0528045460 con 0x7f0538107450 2024-08-19T01:25:16.537 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:15.936+0000 7f053ca00640 1 -- 172.21.15.83:0/1878188486 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) -- 0x7f053819ead0 con 0x7f0538107450 2024-08-19T01:25:16.537 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:15.937+0000 7f052ea00640 1 -- 172.21.15.83:0/1878188486 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v2) ==== 76+0+175 (secure 0 0 0) 0x7f052803b430 con 0x7f0538107450 2024-08-19T01:25:16.537 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:15.938+0000 7f053ca00640 1 -- 172.21.15.83:0/1878188486 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0538107450 msgr2=0x7f053819b440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:25:16.537 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:15.938+0000 7f053ca00640 1 --2- 172.21.15.83:0/1878188486 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0538107450 0x7f053819b440 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f05280097f0 tx=0x7f0528030d80 comp rx=0 tx=0).stop 2024-08-19T01:25:16.537 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:15.938+0000 7f053ca00640 1 -- 172.21.15.83:0/1878188486 shutdown_connections 2024-08-19T01:25:16.537 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:15.938+0000 7f053ca00640 1 --2- 172.21.15.83:0/1878188486 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0538107450 0x7f053819b440 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:25:16.537 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:15.938+0000 7f053ca00640 1 -- 172.21.15.83:0/1878188486 >> 172.21.15.83:0/1878188486 conn(0x7f0538102c00 msgr2=0x7f0538103760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:25:16.537 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:15.938+0000 7f053ca00640 1 -- 172.21.15.83:0/1878188486 shutdown_connections 2024-08-19T01:25:16.537 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:15.938+0000 7f053ca00640 1 -- 172.21.15.83:0/1878188486 wait complete. 2024-08-19T01:25:16.537 INFO:teuthology.orchestra.run.smithi083.stdout:Restarting the monitor... 2024-08-19T01:25:16.905 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:25:16 smithi083 systemd[1]: Stopping Ceph mon.a for bad8780e-5dc9-11ef-bccf-c7b262605968... 2024-08-19T01:25:17.405 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:25:16 smithi083 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-mon-a[16741]: 2024-08-19T01:25:16.919+0000 7f30f5800640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-08-19T01:25:17.405 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:25:16 smithi083 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-mon-a[16741]: 2024-08-19T01:25:16.919+0000 7f30f5800640 -1 mon.a@0(leader) e1 *** Got Signal Terminated *** 2024-08-19T01:25:17.405 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:25:17 smithi083 podman[16997]: 2024-08-19 01:25:17.114784772 +0000 UTC m=+0.341568565 container died 65e09ec4c7ae5eddde361e3147866283e071589bf995639312568f50d39e58f5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833, name=ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-mon-a, RELEASE=squid-de09dd0, org.label-schema.vendor=CentOS, ceph=True, io.buildah.version=1.36.0, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD) 2024-08-19T01:25:17.786 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:25:17 smithi083 podman[16997]: 2024-08-19 01:25:17.550131619 +0000 UTC m=+0.776915412 container cleanup 65e09ec4c7ae5eddde361e3147866283e071589bf995639312568f50d39e58f5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833, name=ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-mon-a, io.buildah.version=1.36.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_CLEAN=True, RELEASE=squid-de09dd0, ceph=True, GIT_BRANCH=HEAD) 2024-08-19T01:25:17.786 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:25:17 smithi083 bash[16997]: ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-mon-a 2024-08-19T01:25:18.046 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:25:17 smithi083 podman[17010]: 2024-08-19 01:25:17.784244544 +0000 UTC m=+0.658157753 container remove 65e09ec4c7ae5eddde361e3147866283e071589bf995639312568f50d39e58f5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833, name=ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-mon-a, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, GIT_CLEAN=True, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, RELEASE=squid-de09dd0, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, io.buildah.version=1.36.0) 2024-08-19T01:25:18.369 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:25:18 smithi083 systemd[1]: ceph-bad8780e-5dc9-11ef-bccf-c7b262605968@mon.a.service: Deactivated successfully. 2024-08-19T01:25:18.370 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:25:18 smithi083 systemd[1]: Stopped Ceph mon.a for bad8780e-5dc9-11ef-bccf-c7b262605968. 2024-08-19T01:25:18.370 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:25:18 smithi083 systemd[1]: ceph-bad8780e-5dc9-11ef-bccf-c7b262605968@mon.a.service: Consumed 1.264s CPU time. 2024-08-19T01:25:18.370 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:25:18 smithi083 systemd[1]: Starting Ceph mon.a for bad8780e-5dc9-11ef-bccf-c7b262605968... 2024-08-19T01:25:18.646 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:25:18 smithi083 podman[17101]: 2024-08-19 01:25:18.368562879 +0000 UTC m=+0.120521662 container create dbe1fc960772ed81dc7b03ac39c0b11bfb4e77ec275e14044b305ec1335132dd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833, name=ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-mon-a, ceph=True, RELEASE=squid-de09dd0, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, io.buildah.version=1.36.0) 2024-08-19T01:25:18.646 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:25:18 smithi083 podman[17101]: 2024-08-19 01:25:18.277090652 +0000 UTC m=+0.029049443 image pull ea7a5189f30bad3aca9524788e2965d2368d7ce786df09d4edcd8c3edf9cefba quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833 2024-08-19T01:25:18.647 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:25:18 smithi083 podman[17101]: 2024-08-19 01:25:18.604862525 +0000 UTC m=+0.356821309 container init dbe1fc960772ed81dc7b03ac39c0b11bfb4e77ec275e14044b305ec1335132dd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833, name=ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-mon-a, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , RELEASE=squid-de09dd0, ceph=True, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, io.buildah.version=1.36.0, org.label-schema.build-date=20240731, GIT_CLEAN=True) 2024-08-19T01:25:18.647 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:25:18 smithi083 podman[17101]: 2024-08-19 01:25:18.611300235 +0000 UTC m=+0.363259019 container start dbe1fc960772ed81dc7b03ac39c0b11bfb4e77ec275e14044b305ec1335132dd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833, name=ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-mon-a, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, RELEASE=squid-de09dd0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.36.0, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2) 2024-08-19T01:25:18.698 INFO:teuthology.orchestra.run.smithi083.stdout:Setting public_network to 172.21.0.0/20 in mon config section 2024-08-19T01:25:18.906 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:25:18 smithi083 ceph-mon[17120]: set uid:gid to 167:167 (ceph:ceph) 2024-08-19T01:25:18.906 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:25:18 smithi083 ceph-mon[17120]: ceph version 19.1.0-1382-gde09dd01 (de09dd018de4baddfa6ca4d43bc8fb784cf75833) squid (rc), process ceph-mon, pid 2 2024-08-19T01:25:18.906 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:25:18 smithi083 ceph-mon[17120]: pidfile_write: ignore empty --pid-file 2024-08-19T01:25:18.906 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:25:18 smithi083 ceph-mon[17120]: load: jerasure load: lrc 2024-08-19T01:25:18.906 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:25:18 smithi083 ceph-mon[17120]: rocksdb: RocksDB version: 7.9.2 2024-08-19T01:25:18.906 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:25:18 smithi083 ceph-mon[17120]: rocksdb: Git sha 0 2024-08-19T01:25:18.906 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:25:18 smithi083 ceph-mon[17120]: rocksdb: Compile date 2024-08-18 15:19:56 2024-08-19T01:25:18.907 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:25:18 smithi083 ceph-mon[17120]: rocksdb: DB SUMMARY 2024-08-19T01:25:18.907 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:25:18 smithi083 ceph-mon[17120]: rocksdb: DB Session ID: 8XMZOFCDKUSF6IHT3EHP 2024-08-19T01:25:18.907 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:25:18 smithi083 ceph-mon[17120]: rocksdb: CURRENT file: CURRENT 2024-08-19T01:25:18.907 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:25:18 smithi083 ceph-mon[17120]: rocksdb: IDENTITY file: IDENTITY 2024-08-19T01:25:18.907 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:25:18 smithi083 ceph-mon[17120]: rocksdb: MANIFEST file: MANIFEST-000010 size: 179 Bytes 2024-08-19T01:25:18.907 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:25:18 smithi083 ceph-mon[17120]: rocksdb: SST files in /var/lib/ceph/mon/ceph-a/store.db dir, Total Num: 1, files: 000008.sst 2024-08-19T01:25:18.907 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:25:18 smithi083 ceph-mon[17120]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-a/store.db: 000009.log size: 106198 ; 2024-08-19T01:25:18.907 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:25:18 smithi083 ceph-mon[17120]: rocksdb: Options.error_if_exists: 0 2024-08-19T01:25:18.907 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:25:18 smithi083 ceph-mon[17120]: rocksdb: Options.create_if_missing: 0 2024-08-19T01:25:18.907 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:25:18 smithi083 ceph-mon[17120]: rocksdb: Options.paranoid_checks: 1 2024-08-19T01:25:18.907 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:25:18 smithi083 ceph-mon[17120]: rocksdb: Options.flush_verify_memtable_count: 1 2024-08-19T01:25:18.907 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:25:18 smithi083 ceph-mon[17120]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-08-19T01:25:18.908 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:25:18 smithi083 ceph-mon[17120]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-08-19T01:25:18.908 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:25:18 smithi083 ceph-mon[17120]: rocksdb: Options.env: 0x558a136b47a0 2024-08-19T01:25:18.908 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:25:18 smithi083 ceph-mon[17120]: rocksdb: Options.fs: PosixFileSystem 2024-08-19T01:25:18.908 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:25:18 smithi083 ceph-mon[17120]: rocksdb: Options.info_log: 0x558a1c0654a0 2024-08-19T01:25:18.908 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:25:18 smithi083 ceph-mon[17120]: rocksdb: Options.max_file_opening_threads: 16 2024-08-19T01:25:18.908 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:25:18 smithi083 ceph-mon[17120]: rocksdb: Options.statistics: (nil) 2024-08-19T01:25:18.908 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:25:18 smithi083 ceph-mon[17120]: rocksdb: Options.use_fsync: 0 2024-08-19T01:25:18.908 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:25:18 smithi083 ceph-mon[17120]: rocksdb: Options.max_log_file_size: 0 2024-08-19T01:25:18.908 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:25:18 smithi083 ceph-mon[17120]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-08-19T01:25:18.908 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:25:18 smithi083 ceph-mon[17120]: rocksdb: Options.log_file_time_to_roll: 0 2024-08-19T01:25:18.908 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:25:18 smithi083 ceph-mon[17120]: rocksdb: Options.keep_log_file_num: 1000 2024-08-19T01:25:18.908 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:25:18 smithi083 ceph-mon[17120]: rocksdb: Options.recycle_log_file_num: 0 2024-08-19T01:25:18.908 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:25:18 smithi083 ceph-mon[17120]: rocksdb: Options.allow_fallocate: 1 2024-08-19T01:25:18.909 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:25:18 smithi083 ceph-mon[17120]: rocksdb: Options.allow_mmap_reads: 0 2024-08-19T01:25:18.909 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:25:18 smithi083 ceph-mon[17120]: rocksdb: Options.allow_mmap_writes: 0 2024-08-19T01:25:18.909 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:25:18 smithi083 ceph-mon[17120]: rocksdb: Options.use_direct_reads: 0 2024-08-19T01:25:18.909 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:25:18 smithi083 ceph-mon[17120]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-08-19T01:25:18.909 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:25:18 smithi083 ceph-mon[17120]: rocksdb: Options.create_missing_column_families: 0 2024-08-19T01:25:18.909 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:25:18 smithi083 ceph-mon[17120]: rocksdb: Options.db_log_dir: 2024-08-19T01:25:18.909 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:25:18 smithi083 ceph-mon[17120]: rocksdb: Options.wal_dir: 2024-08-19T01:25:18.909 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:25:18 smithi083 ceph-mon[17120]: rocksdb: Options.table_cache_numshardbits: 6 2024-08-19T01:25:18.910 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:25:18 smithi083 ceph-mon[17120]: rocksdb: Options.WAL_ttl_seconds: 0 2024-08-19T01:25:18.910 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:25:18 smithi083 ceph-mon[17120]: rocksdb: Options.WAL_size_limit_MB: 0 2024-08-19T01:25:18.911 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:25:18 smithi083 ceph-mon[17120]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-08-19T01:25:18.911 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:25:18 smithi083 ceph-mon[17120]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-08-19T01:25:18.911 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:25:18 smithi083 ceph-mon[17120]: rocksdb: Options.is_fd_close_on_exec: 1 2024-08-19T01:25:18.911 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:25:18 smithi083 ceph-mon[17120]: rocksdb: Options.advise_random_on_open: 1 2024-08-19T01:25:18.911 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:25:18 smithi083 ceph-mon[17120]: rocksdb: Options.db_write_buffer_size: 0 2024-08-19T01:25:18.911 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:25:18 smithi083 ceph-mon[17120]: rocksdb: Options.write_buffer_manager: 0x558a1c074dc0 2024-08-19T01:25:18.911 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:25:18 smithi083 ceph-mon[17120]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-08-19T01:25:18.911 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:25:18 smithi083 ceph-mon[17120]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-08-19T01:25:18.911 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:25:18 smithi083 ceph-mon[17120]: rocksdb: Options.use_adaptive_mutex: 0 2024-08-19T01:25:18.911 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:25:18 smithi083 ceph-mon[17120]: rocksdb: Options.rate_limiter: (nil) 2024-08-19T01:25:18.911 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:25:18 smithi083 ceph-mon[17120]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-08-19T01:25:18.911 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:25:18 smithi083 ceph-mon[17120]: rocksdb: Options.wal_recovery_mode: 2 2024-08-19T01:25:18.911 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:25:18 smithi083 ceph-mon[17120]: rocksdb: Options.enable_thread_tracking: 0 2024-08-19T01:25:18.911 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:25:18 smithi083 ceph-mon[17120]: rocksdb: Options.enable_pipelined_write: 0 2024-08-19T01:25:18.912 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:25:18 smithi083 ceph-mon[17120]: rocksdb: Options.unordered_write: 0 2024-08-19T01:25:18.912 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:25:18 smithi083 ceph-mon[17120]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-08-19T01:25:18.912 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:25:18 smithi083 ceph-mon[17120]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-08-19T01:25:18.912 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:25:18 smithi083 ceph-mon[17120]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-08-19T01:25:18.912 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:25:18 smithi083 ceph-mon[17120]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-08-19T01:25:18.912 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:25:18 smithi083 ceph-mon[17120]: rocksdb: Options.row_cache: None 2024-08-19T01:25:18.912 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:25:18 smithi083 ceph-mon[17120]: rocksdb: Options.wal_filter: None 2024-08-19T01:25:18.912 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:25:18 smithi083 ceph-mon[17120]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-08-19T01:25:18.912 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:25:18 smithi083 ceph-mon[17120]: rocksdb: Options.allow_ingest_behind: 0 2024-08-19T01:25:18.912 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:25:18 smithi083 ceph-mon[17120]: rocksdb: Options.two_write_queues: 0 2024-08-19T01:25:18.912 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:25:18 smithi083 ceph-mon[17120]: rocksdb: Options.manual_wal_flush: 0 2024-08-19T01:25:18.912 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:25:18 smithi083 ceph-mon[17120]: rocksdb: Options.wal_compression: 0 2024-08-19T01:25:18.912 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:25:18 smithi083 ceph-mon[17120]: rocksdb: Options.atomic_flush: 0 2024-08-19T01:25:18.912 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:25:18 smithi083 ceph-mon[17120]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-08-19T01:25:18.913 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:25:18 smithi083 ceph-mon[17120]: rocksdb: Options.persist_stats_to_disk: 0 2024-08-19T01:25:18.913 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:25:18 smithi083 ceph-mon[17120]: rocksdb: Options.write_dbid_to_manifest: 0 2024-08-19T01:25:18.913 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:25:18 smithi083 ceph-mon[17120]: rocksdb: Options.log_readahead_size: 0 2024-08-19T01:25:18.913 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:25:18 smithi083 ceph-mon[17120]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-08-19T01:25:18.913 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:25:18 smithi083 ceph-mon[17120]: rocksdb: Options.best_efforts_recovery: 0 2024-08-19T01:25:18.913 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:25:18 smithi083 ceph-mon[17120]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-08-19T01:25:18.913 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:25:18 smithi083 ceph-mon[17120]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-08-19T01:25:18.913 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:25:18 smithi083 ceph-mon[17120]: rocksdb: Options.allow_data_in_errors: 0 2024-08-19T01:25:18.913 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:25:18 smithi083 ceph-mon[17120]: rocksdb: Options.db_host_id: __hostname__ 2024-08-19T01:25:18.913 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:25:18 smithi083 ceph-mon[17120]: rocksdb: Options.enforce_single_del_contracts: true 2024-08-19T01:25:18.913 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:25:18 smithi083 ceph-mon[17120]: rocksdb: Options.max_background_jobs: 2 2024-08-19T01:25:18.913 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:25:18 smithi083 ceph-mon[17120]: rocksdb: Options.max_background_compactions: -1 2024-08-19T01:25:18.913 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:25:18 smithi083 ceph-mon[17120]: rocksdb: Options.max_subcompactions: 1 2024-08-19T01:25:18.913 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:25:18 smithi083 ceph-mon[17120]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-08-19T01:25:18.914 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:25:18 smithi083 ceph-mon[17120]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-08-19T01:25:18.914 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:25:18 smithi083 ceph-mon[17120]: rocksdb: Options.delayed_write_rate : 16777216 2024-08-19T01:25:18.914 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:25:18 smithi083 ceph-mon[17120]: rocksdb: Options.max_total_wal_size: 0 2024-08-19T01:25:18.914 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:25:18 smithi083 ceph-mon[17120]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-08-19T01:25:18.914 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:25:18 smithi083 ceph-mon[17120]: rocksdb: Options.stats_dump_period_sec: 600 2024-08-19T01:25:18.914 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:25:18 smithi083 ceph-mon[17120]: rocksdb: Options.stats_persist_period_sec: 600 2024-08-19T01:25:18.914 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:25:18 smithi083 ceph-mon[17120]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-08-19T01:25:18.914 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:25:18 smithi083 ceph-mon[17120]: rocksdb: Options.max_open_files: -1 2024-08-19T01:25:18.914 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:25:18 smithi083 ceph-mon[17120]: rocksdb: Options.bytes_per_sync: 0 2024-08-19T01:25:18.914 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:25:18 smithi083 ceph-mon[17120]: rocksdb: Options.wal_bytes_per_sync: 0 2024-08-19T01:25:18.914 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:25:18 smithi083 ceph-mon[17120]: rocksdb: Options.strict_bytes_per_sync: 0 2024-08-19T01:25:18.914 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:25:18 smithi083 ceph-mon[17120]: rocksdb: Options.compaction_readahead_size: 0 2024-08-19T01:25:18.914 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:25:18 smithi083 ceph-mon[17120]: rocksdb: Options.max_background_flushes: -1 2024-08-19T01:25:18.914 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:25:18 smithi083 ceph-mon[17120]: rocksdb: Compression algorithms supported: 2024-08-19T01:25:18.915 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:25:18 smithi083 ceph-mon[17120]: rocksdb: kZSTD supported: 0 2024-08-19T01:25:18.915 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:25:18 smithi083 ceph-mon[17120]: rocksdb: kXpressCompression supported: 0 2024-08-19T01:25:18.915 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:25:18 smithi083 ceph-mon[17120]: rocksdb: kBZip2Compression supported: 0 2024-08-19T01:25:18.915 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:25:18 smithi083 ceph-mon[17120]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-08-19T01:25:18.915 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:25:18 smithi083 ceph-mon[17120]: rocksdb: kLZ4Compression supported: 1 2024-08-19T01:25:18.915 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:25:18 smithi083 ceph-mon[17120]: rocksdb: kZlibCompression supported: 1 2024-08-19T01:25:18.915 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:25:18 smithi083 ceph-mon[17120]: rocksdb: kLZ4HCCompression supported: 1 2024-08-19T01:25:18.915 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:25:18 smithi083 ceph-mon[17120]: rocksdb: kSnappyCompression supported: 1 2024-08-19T01:25:18.915 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:25:18 smithi083 ceph-mon[17120]: rocksdb: Fast CRC32 supported: Supported on x86 2024-08-19T01:25:18.915 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:25:18 smithi083 ceph-mon[17120]: rocksdb: DMutex implementation: pthread_mutex_t 2024-08-19T01:25:18.915 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:25:18 smithi083 ceph-mon[17120]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 2024-08-19T01:25:18.915 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:25:18 smithi083 ceph-mon[17120]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-08-19T01:25:18.916 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:25:18 smithi083 ceph-mon[17120]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-08-19T01:25:18.916 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:25:18 smithi083 ceph-mon[17120]: rocksdb: Options.merge_operator: 2024-08-19T01:25:18.916 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:25:18 smithi083 ceph-mon[17120]: rocksdb: Options.compaction_filter: None 2024-08-19T01:25:18.916 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:25:18 smithi083 ceph-mon[17120]: rocksdb: Options.compaction_filter_factory: None 2024-08-19T01:25:18.916 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:25:18 smithi083 ceph-mon[17120]: rocksdb: Options.sst_partitioner_factory: None 2024-08-19T01:25:18.916 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:25:18 smithi083 ceph-mon[17120]: rocksdb: Options.memtable_factory: SkipListFactory 2024-08-19T01:25:18.916 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:25:18 smithi083 ceph-mon[17120]: rocksdb: Options.table_factory: BlockBasedTable 2024-08-19T01:25:18.916 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:25:18 smithi083 ceph-mon[17120]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x558a1c0650e0) 2024-08-19T01:25:18.916 INFO:journalctl@ceph.mon.a.smithi083.stdout: cache_index_and_filter_blocks: 1 2024-08-19T01:25:18.916 INFO:journalctl@ceph.mon.a.smithi083.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-08-19T01:25:18.916 INFO:journalctl@ceph.mon.a.smithi083.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-08-19T01:25:18.917 INFO:journalctl@ceph.mon.a.smithi083.stdout: pin_top_level_index_and_filter: 1 2024-08-19T01:25:18.917 INFO:journalctl@ceph.mon.a.smithi083.stdout: index_type: 0 2024-08-19T01:25:18.917 INFO:journalctl@ceph.mon.a.smithi083.stdout: data_block_index_type: 0 2024-08-19T01:25:18.917 INFO:journalctl@ceph.mon.a.smithi083.stdout: index_shortening: 1 2024-08-19T01:25:18.917 INFO:journalctl@ceph.mon.a.smithi083.stdout: data_block_hash_table_util_ratio: 0.750000 2024-08-19T01:25:18.917 INFO:journalctl@ceph.mon.a.smithi083.stdout: checksum: 4 2024-08-19T01:25:18.917 INFO:journalctl@ceph.mon.a.smithi083.stdout: no_block_cache: 0 2024-08-19T01:25:18.917 INFO:journalctl@ceph.mon.a.smithi083.stdout: block_cache: 0x558a1c059090 2024-08-19T01:25:18.917 INFO:journalctl@ceph.mon.a.smithi083.stdout: block_cache_name: BinnedLRUCache 2024-08-19T01:25:18.917 INFO:journalctl@ceph.mon.a.smithi083.stdout: block_cache_options: 2024-08-19T01:25:18.917 INFO:journalctl@ceph.mon.a.smithi083.stdout: capacity : 536870912 2024-08-19T01:25:18.918 INFO:journalctl@ceph.mon.a.smithi083.stdout: num_shard_bits : 4 2024-08-19T01:25:18.918 INFO:journalctl@ceph.mon.a.smithi083.stdout: strict_capacity_limit : 0 2024-08-19T01:25:18.918 INFO:journalctl@ceph.mon.a.smithi083.stdout: high_pri_pool_ratio: 0.000 2024-08-19T01:25:18.918 INFO:journalctl@ceph.mon.a.smithi083.stdout: block_cache_compressed: (nil) 2024-08-19T01:25:18.918 INFO:journalctl@ceph.mon.a.smithi083.stdout: persistent_cache: (nil) 2024-08-19T01:25:18.918 INFO:journalctl@ceph.mon.a.smithi083.stdout: block_size: 4096 2024-08-19T01:25:18.918 INFO:journalctl@ceph.mon.a.smithi083.stdout: block_size_deviation: 10 2024-08-19T01:25:18.918 INFO:journalctl@ceph.mon.a.smithi083.stdout: block_restart_interval: 16 2024-08-19T01:25:18.918 INFO:journalctl@ceph.mon.a.smithi083.stdout: index_block_restart_interval: 1 2024-08-19T01:25:18.918 INFO:journalctl@ceph.mon.a.smithi083.stdout: metadata_block_size: 4096 2024-08-19T01:25:18.918 INFO:journalctl@ceph.mon.a.smithi083.stdout: partition_filters: 0 2024-08-19T01:25:18.918 INFO:journalctl@ceph.mon.a.smithi083.stdout: use_delta_encoding: 1 2024-08-19T01:25:18.919 INFO:journalctl@ceph.mon.a.smithi083.stdout: filter_policy: bloomfilter 2024-08-19T01:25:18.919 INFO:journalctl@ceph.mon.a.smithi083.stdout: whole_key_filtering: 1 2024-08-19T01:25:18.919 INFO:journalctl@ceph.mon.a.smithi083.stdout: verify_compression: 0 2024-08-19T01:25:18.919 INFO:journalctl@ceph.mon.a.smithi083.stdout: read_amp_bytes_per_bit: 0 2024-08-19T01:25:18.919 INFO:journalctl@ceph.mon.a.smithi083.stdout: format_version: 5 2024-08-19T01:25:18.919 INFO:journalctl@ceph.mon.a.smithi083.stdout: enable_index_compression: 1 2024-08-19T01:25:18.919 INFO:journalctl@ceph.mon.a.smithi083.stdout: block_align: 0 2024-08-19T01:25:18.919 INFO:journalctl@ceph.mon.a.smithi083.stdout: max_auto_readahead_size: 262144 2024-08-19T01:25:18.919 INFO:journalctl@ceph.mon.a.smithi083.stdout: prepopulate_block_cache: 0 2024-08-19T01:25:18.919 INFO:journalctl@ceph.mon.a.smithi083.stdout: initial_auto_readahead_size: 8192 2024-08-19T01:25:18.919 INFO:journalctl@ceph.mon.a.smithi083.stdout: num_file_reads_for_auto_readahead: 2 2024-08-19T01:25:18.919 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:25:18 smithi083 ceph-mon[17120]: rocksdb: Options.write_buffer_size: 33554432 2024-08-19T01:25:18.919 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:25:18 smithi083 ceph-mon[17120]: rocksdb: Options.max_write_buffer_number: 2 2024-08-19T01:25:18.920 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:25:18 smithi083 ceph-mon[17120]: rocksdb: Options.compression: NoCompression 2024-08-19T01:25:18.920 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:25:18 smithi083 ceph-mon[17120]: rocksdb: Options.bottommost_compression: Disabled 2024-08-19T01:25:18.920 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:25:18 smithi083 ceph-mon[17120]: rocksdb: Options.prefix_extractor: nullptr 2024-08-19T01:25:18.920 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:25:18 smithi083 ceph-mon[17120]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-08-19T01:25:18.920 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:25:18 smithi083 ceph-mon[17120]: rocksdb: Options.num_levels: 7 2024-08-19T01:25:18.920 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:25:18 smithi083 ceph-mon[17120]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-08-19T01:25:18.920 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:25:18 smithi083 ceph-mon[17120]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-08-19T01:25:18.920 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:25:18 smithi083 ceph-mon[17120]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-08-19T01:25:18.920 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:25:18 smithi083 ceph-mon[17120]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-08-19T01:25:18.920 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:25:18 smithi083 ceph-mon[17120]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-08-19T01:25:18.920 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:25:18 smithi083 ceph-mon[17120]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-08-19T01:25:18.920 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:25:18 smithi083 ceph-mon[17120]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-08-19T01:25:18.920 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:25:18 smithi083 ceph-mon[17120]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-08-19T01:25:18.921 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:25:18 smithi083 ceph-mon[17120]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-08-19T01:25:18.921 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:25:18 smithi083 ceph-mon[17120]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-08-19T01:25:18.921 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:25:18 smithi083 ceph-mon[17120]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-08-19T01:25:18.921 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:25:18 smithi083 ceph-mon[17120]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-08-19T01:25:18.921 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:25:18 smithi083 ceph-mon[17120]: rocksdb: Options.compression_opts.window_bits: -14 2024-08-19T01:25:18.921 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:25:18 smithi083 ceph-mon[17120]: rocksdb: Options.compression_opts.level: 32767 2024-08-19T01:25:18.921 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:25:18 smithi083 ceph-mon[17120]: rocksdb: Options.compression_opts.strategy: 0 2024-08-19T01:25:18.921 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:25:18 smithi083 ceph-mon[17120]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-08-19T01:25:18.921 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:25:18 smithi083 ceph-mon[17120]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-08-19T01:25:18.921 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:25:18 smithi083 ceph-mon[17120]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-08-19T01:25:18.921 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:25:18 smithi083 ceph-mon[17120]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-08-19T01:25:18.921 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:25:18 smithi083 ceph-mon[17120]: rocksdb: Options.compression_opts.enabled: false 2024-08-19T01:25:18.921 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:25:18 smithi083 ceph-mon[17120]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-08-19T01:25:18.921 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:25:18 smithi083 ceph-mon[17120]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-08-19T01:25:18.922 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:25:18 smithi083 ceph-mon[17120]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-08-19T01:25:18.922 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:25:18 smithi083 ceph-mon[17120]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-08-19T01:25:18.922 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:25:18 smithi083 ceph-mon[17120]: rocksdb: Options.target_file_size_base: 67108864 2024-08-19T01:25:18.922 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:25:18 smithi083 ceph-mon[17120]: rocksdb: Options.target_file_size_multiplier: 1 2024-08-19T01:25:18.922 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:25:18 smithi083 ceph-mon[17120]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-08-19T01:25:18.922 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:25:18 smithi083 ceph-mon[17120]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-08-19T01:25:18.922 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:25:18 smithi083 ceph-mon[17120]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-08-19T01:25:18.922 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:25:18 smithi083 ceph-mon[17120]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-08-19T01:25:18.922 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:25:18 smithi083 ceph-mon[17120]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-08-19T01:25:18.922 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:25:18 smithi083 ceph-mon[17120]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-08-19T01:25:18.922 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:25:18 smithi083 ceph-mon[17120]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-08-19T01:25:18.922 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:25:18 smithi083 ceph-mon[17120]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-08-19T01:25:18.922 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:25:18 smithi083 ceph-mon[17120]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-08-19T01:25:18.922 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:25:18 smithi083 ceph-mon[17120]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-08-19T01:25:18.923 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:25:18 smithi083 ceph-mon[17120]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-08-19T01:25:18.923 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:25:18 smithi083 ceph-mon[17120]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-08-19T01:25:18.923 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:25:18 smithi083 ceph-mon[17120]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-08-19T01:25:18.923 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:25:18 smithi083 ceph-mon[17120]: rocksdb: Options.arena_block_size: 1048576 2024-08-19T01:25:18.923 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:25:18 smithi083 ceph-mon[17120]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-08-19T01:25:18.923 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:25:18 smithi083 ceph-mon[17120]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-08-19T01:25:18.923 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:25:18 smithi083 ceph-mon[17120]: rocksdb: Options.disable_auto_compactions: 0 2024-08-19T01:25:18.923 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:25:18 smithi083 ceph-mon[17120]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-08-19T01:25:18.923 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:25:18 smithi083 ceph-mon[17120]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-08-19T01:25:18.923 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:25:18 smithi083 ceph-mon[17120]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-08-19T01:25:18.923 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:25:18 smithi083 ceph-mon[17120]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-08-19T01:25:18.923 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:25:18 smithi083 ceph-mon[17120]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-08-19T01:25:18.923 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:25:18 smithi083 ceph-mon[17120]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-08-19T01:25:18.923 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:25:18 smithi083 ceph-mon[17120]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-08-19T01:25:18.924 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:25:18 smithi083 ceph-mon[17120]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-08-19T01:25:18.924 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:25:18 smithi083 ceph-mon[17120]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-08-19T01:25:18.924 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:25:18 smithi083 ceph-mon[17120]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-08-19T01:25:18.924 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:25:18 smithi083 ceph-mon[17120]: rocksdb: Options.table_properties_collectors: CompactOnDeletionCollector (Sliding window size = 32768 Deletion trigger = 16384 Deletion ratio = 0); 2024-08-19T01:25:18.924 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:25:18 smithi083 ceph-mon[17120]: rocksdb: Options.inplace_update_support: 0 2024-08-19T01:25:18.924 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:25:18 smithi083 ceph-mon[17120]: rocksdb: Options.inplace_update_num_locks: 10000 2024-08-19T01:25:18.924 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:25:18 smithi083 ceph-mon[17120]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-08-19T01:25:18.924 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:25:18 smithi083 ceph-mon[17120]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-08-19T01:25:18.924 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:25:18 smithi083 ceph-mon[17120]: rocksdb: Options.memtable_huge_page_size: 0 2024-08-19T01:25:18.924 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:25:18 smithi083 ceph-mon[17120]: rocksdb: Options.bloom_locality: 0 2024-08-19T01:25:18.924 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:25:18 smithi083 ceph-mon[17120]: rocksdb: Options.max_successive_merges: 0 2024-08-19T01:25:18.924 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:25:18 smithi083 ceph-mon[17120]: rocksdb: Options.optimize_filters_for_hits: 0 2024-08-19T01:25:18.924 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:25:18 smithi083 ceph-mon[17120]: rocksdb: Options.paranoid_file_checks: 0 2024-08-19T01:25:18.924 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:25:18 smithi083 ceph-mon[17120]: rocksdb: Options.force_consistency_checks: 1 2024-08-19T01:25:18.925 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:25:18 smithi083 ceph-mon[17120]: rocksdb: Options.report_bg_io_stats: 0 2024-08-19T01:25:18.925 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:25:18 smithi083 ceph-mon[17120]: rocksdb: Options.ttl: 2592000 2024-08-19T01:25:18.925 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:25:18 smithi083 ceph-mon[17120]: rocksdb: Options.periodic_compaction_seconds: 0 2024-08-19T01:25:18.925 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:25:18 smithi083 ceph-mon[17120]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-08-19T01:25:18.925 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:25:18 smithi083 ceph-mon[17120]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-08-19T01:25:18.925 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:25:18 smithi083 ceph-mon[17120]: rocksdb: Options.enable_blob_files: false 2024-08-19T01:25:18.925 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:25:18 smithi083 ceph-mon[17120]: rocksdb: Options.min_blob_size: 0 2024-08-19T01:25:18.925 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:25:18 smithi083 ceph-mon[17120]: rocksdb: Options.blob_file_size: 268435456 2024-08-19T01:25:18.925 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:25:18 smithi083 ceph-mon[17120]: rocksdb: Options.blob_compression_type: NoCompression 2024-08-19T01:25:18.925 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:25:18 smithi083 ceph-mon[17120]: rocksdb: Options.enable_blob_garbage_collection: false 2024-08-19T01:25:18.925 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:25:18 smithi083 ceph-mon[17120]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-08-19T01:25:18.925 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:25:18 smithi083 ceph-mon[17120]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-08-19T01:25:18.925 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:25:18 smithi083 ceph-mon[17120]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-08-19T01:25:18.925 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:25:18 smithi083 ceph-mon[17120]: rocksdb: Options.blob_file_starting_level: 0 2024-08-19T01:25:18.926 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:25:18 smithi083 ceph-mon[17120]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-08-19T01:25:18.926 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:25:18 smithi083 ceph-mon[17120]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 succeeded,manifest_file_number is 10, next_file_number is 12, last_sequence is 5, log_number is 5,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 5 2024-08-19T01:25:18.926 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:25:18 smithi083 ceph-mon[17120]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 5 2024-08-19T01:25:18.926 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:25:18 smithi083 ceph-mon[17120]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: f27259aa-2a4f-477b-a0db-f3ff794e85d1 2024-08-19T01:25:18.926 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:25:18 smithi083 ceph-mon[17120]: rocksdb: EVENT_LOG_v1 {"time_micros": 1724030718655068, "job": 1, "event": "recovery_started", "wal_files": [9]} 2024-08-19T01:25:18.926 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:25:18 smithi083 ceph-mon[17120]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #9 mode 2 2024-08-19T01:25:18.926 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:25:18 smithi083 ceph-mon[17120]: rocksdb: EVENT_LOG_v1 {"time_micros": 1724030718658006, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 13, "file_size": 101154, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 8, "largest_seqno": 332, "table_properties": {"data_size": 99039, "index_size": 307, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 773, "raw_key_size": 14538, "raw_average_key_size": 49, "raw_value_size": 91195, "raw_average_value_size": 313, "num_data_blocks": 12, "num_entries": 291, "num_filter_entries": 291, "num_deletions": 3, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[CompactOnDeletionCollector]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1724030718, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "f27259aa-2a4f-477b-a0db-f3ff794e85d1", "db_session_id": "8XMZOFCDKUSF6IHT3EHP", "orig_file_number": 13, "seqno_to_time_mapping": "N/A"}} 2024-08-19T01:25:18.926 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:25:18 smithi083 ceph-mon[17120]: rocksdb: EVENT_LOG_v1 {"time_micros": 1724030718658317, "job": 1, "event": "recovery_finished"} 2024-08-19T01:25:18.926 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:25:18 smithi083 ceph-mon[17120]: rocksdb: [db/version_set.cc:5047] Creating manifest 15 2024-08-19T01:25:18.926 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:25:18 smithi083 ceph-mon[17120]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-a/store.db/000009.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-08-19T01:25:18.926 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:25:18 smithi083 ceph-mon[17120]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x558a1c088e00 2024-08-19T01:25:18.926 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:25:18 smithi083 ceph-mon[17120]: rocksdb: DB pointer 0x558a1c182000 2024-08-19T01:25:18.926 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:25:18 smithi083 ceph-mon[17120]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-08-19T01:25:18.927 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:25:18 smithi083 ceph-mon[17120]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-08-19T01:25:18.927 INFO:journalctl@ceph.mon.a.smithi083.stdout: ** DB Stats ** 2024-08-19T01:25:18.927 INFO:journalctl@ceph.mon.a.smithi083.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-08-19T01:25:18.927 INFO:journalctl@ceph.mon.a.smithi083.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-08-19T01:25:18.927 INFO:journalctl@ceph.mon.a.smithi083.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-08-19T01:25:18.927 INFO:journalctl@ceph.mon.a.smithi083.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-08-19T01:25:18.927 INFO:journalctl@ceph.mon.a.smithi083.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-08-19T01:25:18.927 INFO:journalctl@ceph.mon.a.smithi083.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-08-19T01:25:18.927 INFO:journalctl@ceph.mon.a.smithi083.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-08-19T01:25:18.927 INFO:journalctl@ceph.mon.a.smithi083.stdout: 2024-08-19T01:25:18.927 INFO:journalctl@ceph.mon.a.smithi083.stdout: ** Compaction Stats [default] ** 2024-08-19T01:25:18.927 INFO:journalctl@ceph.mon.a.smithi083.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-08-19T01:25:18.927 INFO:journalctl@ceph.mon.a.smithi083.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-08-19T01:25:18.927 INFO:journalctl@ceph.mon.a.smithi083.stdout: L0 2/0 100.64 KB 0.5 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 53.6 0.00 0.00 1 0.002 0 0 0.0 0.0 2024-08-19T01:25:18.928 INFO:journalctl@ceph.mon.a.smithi083.stdout: Sum 2/0 100.64 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 53.6 0.00 0.00 1 0.002 0 0 0.0 0.0 2024-08-19T01:25:18.928 INFO:journalctl@ceph.mon.a.smithi083.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 53.6 0.00 0.00 1 0.002 0 0 0.0 0.0 2024-08-19T01:25:18.928 INFO:journalctl@ceph.mon.a.smithi083.stdout: 2024-08-19T01:25:18.928 INFO:journalctl@ceph.mon.a.smithi083.stdout: ** Compaction Stats [default] ** 2024-08-19T01:25:18.928 INFO:journalctl@ceph.mon.a.smithi083.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-08-19T01:25:18.928 INFO:journalctl@ceph.mon.a.smithi083.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-08-19T01:25:18.928 INFO:journalctl@ceph.mon.a.smithi083.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 53.6 0.00 0.00 1 0.002 0 0 0.0 0.0 2024-08-19T01:25:18.928 INFO:journalctl@ceph.mon.a.smithi083.stdout: 2024-08-19T01:25:18.928 INFO:journalctl@ceph.mon.a.smithi083.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-08-19T01:25:18.928 INFO:journalctl@ceph.mon.a.smithi083.stdout: 2024-08-19T01:25:18.928 INFO:journalctl@ceph.mon.a.smithi083.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-08-19T01:25:18.928 INFO:journalctl@ceph.mon.a.smithi083.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-08-19T01:25:18.928 INFO:journalctl@ceph.mon.a.smithi083.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-08-19T01:25:18.928 INFO:journalctl@ceph.mon.a.smithi083.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-08-19T01:25:18.929 INFO:journalctl@ceph.mon.a.smithi083.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-08-19T01:25:18.929 INFO:journalctl@ceph.mon.a.smithi083.stdout: AddFile(Keys): cumulative 0, interval 0 2024-08-19T01:25:18.929 INFO:journalctl@ceph.mon.a.smithi083.stdout: Cumulative compaction: 0.00 GB write, 10.49 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-08-19T01:25:18.929 INFO:journalctl@ceph.mon.a.smithi083.stdout: Interval compaction: 0.00 GB write, 10.49 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-08-19T01:25:18.929 INFO:journalctl@ceph.mon.a.smithi083.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-08-19T01:25:18.929 INFO:journalctl@ceph.mon.a.smithi083.stdout: Block cache BinnedLRUCache@0x558a1c059090#2 capacity: 512.00 MB usage: 1.52 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 1e-05 secs_since: 0 2024-08-19T01:25:18.929 INFO:journalctl@ceph.mon.a.smithi083.stdout: Block cache entry stats(count,size,portion): FilterBlock(2,1.02 KB,0.000193715%) IndexBlock(2,0.50 KB,9.53674e-05%) Misc(1,0.00 KB,0%) 2024-08-19T01:25:18.929 INFO:journalctl@ceph.mon.a.smithi083.stdout: 2024-08-19T01:25:18.929 INFO:journalctl@ceph.mon.a.smithi083.stdout: ** File Read Latency Histogram By Level [default] ** 2024-08-19T01:25:18.929 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:25:18 smithi083 ceph-mon[17120]: starting mon.a rank 0 at public addrs [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] at bind addrs [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] mon_data /var/lib/ceph/mon/ceph-a fsid bad8780e-5dc9-11ef-bccf-c7b262605968 2024-08-19T01:25:18.930 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:25:18 smithi083 ceph-mon[17120]: mon.a@-1(???) e1 preinit fsid bad8780e-5dc9-11ef-bccf-c7b262605968 2024-08-19T01:25:18.930 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:25:18 smithi083 ceph-mon[17120]: mon.a@-1(???).mds e1 new map 2024-08-19T01:25:18.930 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:25:18 smithi083 ceph-mon[17120]: mon.a@-1(???).mds e1 print_map 2024-08-19T01:25:18.930 INFO:journalctl@ceph.mon.a.smithi083.stdout: e1 2024-08-19T01:25:18.930 INFO:journalctl@ceph.mon.a.smithi083.stdout: btime 2024-08-19T01:25:12:626585+0000 2024-08-19T01:25:18.930 INFO:journalctl@ceph.mon.a.smithi083.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-08-19T01:25:18.930 INFO:journalctl@ceph.mon.a.smithi083.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-08-19T01:25:18.930 INFO:journalctl@ceph.mon.a.smithi083.stdout: legacy client fscid: -1 2024-08-19T01:25:18.930 INFO:journalctl@ceph.mon.a.smithi083.stdout: 2024-08-19T01:25:18.930 INFO:journalctl@ceph.mon.a.smithi083.stdout: No filesystems configured 2024-08-19T01:25:18.930 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:25:18 smithi083 ceph-mon[17120]: mon.a@-1(???).osd e1 crush map has features 3314932999778484224, adjusting msgr requires 2024-08-19T01:25:18.930 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:25:18 smithi083 ceph-mon[17120]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-08-19T01:25:18.930 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:25:18 smithi083 ceph-mon[17120]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-08-19T01:25:18.930 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:25:18 smithi083 ceph-mon[17120]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-08-19T01:25:18.931 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:25:18 smithi083 ceph-mon[17120]: mon.a@-1(???).paxosservice(auth 1..2) refresh upgraded, format 0 -> 3 2024-08-19T01:25:18.931 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:25:18 smithi083 bash[17101]: dbe1fc960772ed81dc7b03ac39c0b11bfb4e77ec275e14044b305ec1335132dd 2024-08-19T01:25:18.931 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:25:18 smithi083 systemd[1]: Started Ceph mon.a for bad8780e-5dc9-11ef-bccf-c7b262605968. 2024-08-19T01:25:18.931 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:25:18 smithi083 ceph-mon[17120]: mon.a is new leader, mons a in quorum (ranks 0) 2024-08-19T01:25:18.931 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:25:18 smithi083 ceph-mon[17120]: monmap epoch 1 2024-08-19T01:25:18.931 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:25:18 smithi083 ceph-mon[17120]: fsid bad8780e-5dc9-11ef-bccf-c7b262605968 2024-08-19T01:25:18.931 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:25:18 smithi083 ceph-mon[17120]: last_changed 2024-08-19T01:25:09.034517+0000 2024-08-19T01:25:18.931 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:25:18 smithi083 ceph-mon[17120]: created 2024-08-19T01:25:09.034517+0000 2024-08-19T01:25:18.931 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:25:18 smithi083 ceph-mon[17120]: min_mon_release 19 (squid) 2024-08-19T01:25:18.931 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:25:18 smithi083 ceph-mon[17120]: election_strategy: 1 2024-08-19T01:25:18.931 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:25:18 smithi083 ceph-mon[17120]: 0: [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] mon.a 2024-08-19T01:25:18.931 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:25:18 smithi083 ceph-mon[17120]: fsmap 2024-08-19T01:25:18.931 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:25:18 smithi083 ceph-mon[17120]: osdmap e1: 0 total, 0 up, 0 in 2024-08-19T01:25:18.931 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:25:18 smithi083 ceph-mon[17120]: mgrmap e1: no daemons active 2024-08-19T01:25:19.922 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:19.225+0000 7fd111200640 1 Processor -- start 2024-08-19T01:25:19.922 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:19.225+0000 7fd111200640 1 -- start start 2024-08-19T01:25:19.922 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:19.226+0000 7fd111200640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd10c07de60 0x7fd10c07c2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:25:19.922 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:19.226+0000 7fd111200640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fd10c07c890 con 0x7fd10c07de60 2024-08-19T01:25:19.922 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:19.226+0000 7fd109e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd10c07de60 0x7fd10c07c2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:25:19.922 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:19.226+0000 7fd109e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd10c07de60 0x7fd10c07c2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:41266/0 (socket says 172.21.15.83:41266) 2024-08-19T01:25:19.922 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:19.227+0000 7fd109e00640 1 -- 172.21.15.83:0/232416886 learned_addr learned my addr 172.21.15.83:0/232416886 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-08-19T01:25:19.922 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:19.227+0000 7fd109e00640 1 -- 172.21.15.83:0/232416886 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd10c07ca10 con 0x7fd10c07de60 2024-08-19T01:25:19.922 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:19.229+0000 7fd109e00640 1 --2- 172.21.15.83:0/232416886 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd10c07de60 0x7fd10c07c2c0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7fd0fc009820 tx=0x7fd0fc02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=30b6a6fe0c40f4b0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:25:19.922 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:19.230+0000 7fd109400640 1 -- 172.21.15.83:0/232416886 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd0fc037070 con 0x7fd10c07de60 2024-08-19T01:25:19.923 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:19.230+0000 7fd109400640 1 -- 172.21.15.83:0/232416886 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fd0fc032900 con 0x7fd10c07de60 2024-08-19T01:25:19.923 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:19.230+0000 7fd109400640 1 -- 172.21.15.83:0/232416886 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd0fc030850 con 0x7fd10c07de60 2024-08-19T01:25:19.923 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:19.230+0000 7fd111200640 1 -- 172.21.15.83:0/232416886 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd10c07de60 msgr2=0x7fd10c07c2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:25:19.923 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:19.230+0000 7fd111200640 1 --2- 172.21.15.83:0/232416886 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd10c07de60 0x7fd10c07c2c0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7fd0fc009820 tx=0x7fd0fc02a270 comp rx=0 tx=0).stop 2024-08-19T01:25:19.923 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:19.230+0000 7fd111200640 1 -- 172.21.15.83:0/232416886 shutdown_connections 2024-08-19T01:25:19.923 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:19.230+0000 7fd111200640 1 --2- 172.21.15.83:0/232416886 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd10c07de60 0x7fd10c07c2c0 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:25:19.923 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:19.230+0000 7fd111200640 1 -- 172.21.15.83:0/232416886 >> 172.21.15.83:0/232416886 conn(0x7fd10c102c00 msgr2=0x7fd10c105020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:25:19.923 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:19.231+0000 7fd111200640 1 -- 172.21.15.83:0/232416886 shutdown_connections 2024-08-19T01:25:19.923 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:19.231+0000 7fd111200640 1 -- 172.21.15.83:0/232416886 wait complete. 2024-08-19T01:25:19.923 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:19.231+0000 7fd111200640 1 Processor -- start 2024-08-19T01:25:19.923 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:19.232+0000 7fd111200640 1 -- start start 2024-08-19T01:25:19.923 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:19.232+0000 7fd111200640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd10c07de60 0x7fd10c19d670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:25:19.923 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:19.232+0000 7fd111200640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fd10c10da80 con 0x7fd10c07de60 2024-08-19T01:25:19.924 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:19.232+0000 7fd109e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd10c07de60 0x7fd10c19d670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:25:19.924 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:19.232+0000 7fd109e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd10c07de60 0x7fd10c19d670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:41278/0 (socket says 172.21.15.83:41278) 2024-08-19T01:25:19.924 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:19.232+0000 7fd109e00640 1 -- 172.21.15.83:0/1113224748 learned_addr learned my addr 172.21.15.83:0/1113224748 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-08-19T01:25:19.924 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:19.232+0000 7fd109e00640 1 -- 172.21.15.83:0/1113224748 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd10c19dbb0 con 0x7fd10c07de60 2024-08-19T01:25:19.924 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:19.233+0000 7fd109e00640 1 --2- 172.21.15.83:0/1113224748 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd10c07de60 0x7fd10c19d670 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7fd0fc0097f0 tx=0x7fd0fc030d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:25:19.924 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:19.233+0000 7fd103400640 1 -- 172.21.15.83:0/1113224748 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd0fc037040 con 0x7fd10c07de60 2024-08-19T01:25:19.924 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:19.233+0000 7fd103400640 1 -- 172.21.15.83:0/1113224748 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fd0fc040070 con 0x7fd10c07de60 2024-08-19T01:25:19.924 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:19.233+0000 7fd103400640 1 -- 172.21.15.83:0/1113224748 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd0fc03a390 con 0x7fd10c07de60 2024-08-19T01:25:19.924 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:19.233+0000 7fd111200640 1 -- 172.21.15.83:0/1113224748 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd10c19de40 con 0x7fd10c07de60 2024-08-19T01:25:19.924 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:19.233+0000 7fd111200640 1 -- 172.21.15.83:0/1113224748 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd10c1a0b30 con 0x7fd10c07de60 2024-08-19T01:25:19.924 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:19.234+0000 7fd103400640 1 -- 172.21.15.83:0/1113224748 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fd0fc03a630 con 0x7fd10c07de60 2024-08-19T01:25:19.924 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:19.234+0000 7fd103400640 1 -- 172.21.15.83:0/1113224748 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fd0fc0459a0 con 0x7fd10c07de60 2024-08-19T01:25:19.924 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:19.234+0000 7fd111200640 1 -- 172.21.15.83:0/1113224748 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd10c07c340 con 0x7fd10c07de60 2024-08-19T01:25:19.925 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:19.237+0000 7fd103400640 1 -- 172.21.15.83:0/1113224748 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73482 (secure 0 0 0) 0x7fd0fc045460 con 0x7fd10c07de60 2024-08-19T01:25:19.925 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:19.295+0000 7fd111200640 1 -- 172.21.15.83:0/1113224748 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command([{prefix=config set, name=public_network}] v 0) -- 0x7fd10c19e2c0 con 0x7fd10c07de60 2024-08-19T01:25:19.925 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:19.297+0000 7fd103400640 1 -- 172.21.15.83:0/1113224748 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{prefix=config set, name=public_network}]=0 v3)=0 v3) ==== 124+0+0 (secure 0 0 0) 0x7fd0fc03b430 con 0x7fd10c07de60 2024-08-19T01:25:19.925 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:19.298+0000 7fd111200640 1 -- 172.21.15.83:0/1113224748 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd10c07de60 msgr2=0x7fd10c19d670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:25:19.925 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:19.298+0000 7fd111200640 1 --2- 172.21.15.83:0/1113224748 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd10c07de60 0x7fd10c19d670 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7fd0fc0097f0 tx=0x7fd0fc030d80 comp rx=0 tx=0).stop 2024-08-19T01:25:19.925 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:19.298+0000 7fd111200640 1 -- 172.21.15.83:0/1113224748 shutdown_connections 2024-08-19T01:25:19.925 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:19.298+0000 7fd111200640 1 --2- 172.21.15.83:0/1113224748 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd10c07de60 0x7fd10c19d670 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:25:19.925 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:19.298+0000 7fd111200640 1 -- 172.21.15.83:0/1113224748 >> 172.21.15.83:0/1113224748 conn(0x7fd10c102c00 msgr2=0x7fd10c103790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:25:19.925 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:19.299+0000 7fd111200640 1 -- 172.21.15.83:0/1113224748 shutdown_connections 2024-08-19T01:25:19.925 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:19.299+0000 7fd111200640 1 -- 172.21.15.83:0/1113224748 wait complete. 2024-08-19T01:25:19.925 INFO:teuthology.orchestra.run.smithi083.stdout:Wrote config to /etc/ceph/ceph.conf 2024-08-19T01:25:19.953 INFO:teuthology.orchestra.run.smithi083.stdout:Wrote keyring to /etc/ceph/ceph.client.admin.keyring 2024-08-19T01:25:19.953 INFO:teuthology.orchestra.run.smithi083.stdout:Creating mgr... 2024-08-19T01:25:19.953 INFO:teuthology.orchestra.run.smithi083.stdout:Verifying port 0.0.0.0:9283 ... 2024-08-19T01:25:19.954 INFO:teuthology.orchestra.run.smithi083.stdout:Verifying port 0.0.0.0:8765 ... 2024-08-19T01:25:20.303 INFO:teuthology.orchestra.run.smithi083.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-bad8780e-5dc9-11ef-bccf-c7b262605968@mgr.x 2024-08-19T01:25:20.303 INFO:teuthology.orchestra.run.smithi083.stdout:systemctl: stderr Failed to reset failed state of unit ceph-bad8780e-5dc9-11ef-bccf-c7b262605968@mgr.x.service: Unit ceph-bad8780e-5dc9-11ef-bccf-c7b262605968@mgr.x.service not loaded. 2024-08-19T01:25:20.542 INFO:teuthology.orchestra.run.smithi083.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-bad8780e-5dc9-11ef-bccf-c7b262605968.target.wants/ceph-bad8780e-5dc9-11ef-bccf-c7b262605968@mgr.x.service → /etc/systemd/system/ceph-bad8780e-5dc9-11ef-bccf-c7b262605968@.service. 2024-08-19T01:25:20.576 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:25:20 smithi083 ceph-mon[17120]: from='client.? 172.21.15.83:0/1113224748' entity='client.admin' 2024-08-19T01:25:20.900 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Aug 19 01:25:20 smithi083 systemd[1]: Starting Ceph mgr.x for bad8780e-5dc9-11ef-bccf-c7b262605968... 2024-08-19T01:25:21.156 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Aug 19 01:25:20 smithi083 podman[17329]: 2024-08-19 01:25:20.800971084 +0000 UTC m=+0.028647452 image pull ea7a5189f30bad3aca9524788e2965d2368d7ce786df09d4edcd8c3edf9cefba quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833 2024-08-19T01:25:21.156 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Aug 19 01:25:20 smithi083 podman[17329]: 2024-08-19 01:25:20.904617291 +0000 UTC m=+0.132293663 container create c168c67385d5dd2105e374e7a8d10bbed67b5ffd3f14a963f3deaea98ea71195 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833, name=ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-mgr-x, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, io.buildah.version=1.36.0, RELEASE=squid-de09dd0) 2024-08-19T01:25:21.156 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Aug 19 01:25:21 smithi083 podman[17329]: 2024-08-19 01:25:21.132093703 +0000 UTC m=+0.359770083 container init c168c67385d5dd2105e374e7a8d10bbed67b5ffd3f14a963f3deaea98ea71195 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833, name=ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-mgr-x, io.buildah.version=1.36.0, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True, RELEASE=squid-de09dd0, org.label-schema.license=GPLv2, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731) 2024-08-19T01:25:21.156 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Aug 19 01:25:21 smithi083 podman[17329]: 2024-08-19 01:25:21.138383821 +0000 UTC m=+0.366060194 container start c168c67385d5dd2105e374e7a8d10bbed67b5ffd3f14a963f3deaea98ea71195 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833, name=ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-mgr-x, maintainer=Guillaume Abrioux , io.buildah.version=1.36.0, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, RELEASE=squid-de09dd0, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True) 2024-08-19T01:25:21.260 INFO:teuthology.orchestra.run.smithi083.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-08-19T01:25:21.260 INFO:teuthology.orchestra.run.smithi083.stdout:systemctl: stdout disabled 2024-08-19T01:25:21.274 INFO:teuthology.orchestra.run.smithi083.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-08-19T01:25:21.275 INFO:teuthology.orchestra.run.smithi083.stdout:systemctl: stdout inactive 2024-08-19T01:25:21.275 INFO:teuthology.orchestra.run.smithi083.stdout:firewalld.service is not enabled 2024-08-19T01:25:21.275 INFO:teuthology.orchestra.run.smithi083.stdout:Not possible to enable service . firewalld.service is not available 2024-08-19T01:25:21.292 INFO:teuthology.orchestra.run.smithi083.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-08-19T01:25:21.292 INFO:teuthology.orchestra.run.smithi083.stdout:systemctl: stdout disabled 2024-08-19T01:25:21.306 INFO:teuthology.orchestra.run.smithi083.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-08-19T01:25:21.306 INFO:teuthology.orchestra.run.smithi083.stdout:systemctl: stdout inactive 2024-08-19T01:25:21.306 INFO:teuthology.orchestra.run.smithi083.stdout:firewalld.service is not enabled 2024-08-19T01:25:21.306 INFO:teuthology.orchestra.run.smithi083.stdout:Not possible to open ports <[9283, 8765]>. firewalld.service is not available 2024-08-19T01:25:21.306 INFO:teuthology.orchestra.run.smithi083.stdout:Waiting for mgr to start... 2024-08-19T01:25:21.306 INFO:teuthology.orchestra.run.smithi083.stdout:Waiting for mgr... 2024-08-19T01:25:21.523 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Aug 19 01:25:21 smithi083 bash[17329]: c168c67385d5dd2105e374e7a8d10bbed67b5ffd3f14a963f3deaea98ea71195 2024-08-19T01:25:21.523 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Aug 19 01:25:21 smithi083 systemd[1]: Started Ceph mgr.x for bad8780e-5dc9-11ef-bccf-c7b262605968. 2024-08-19T01:25:21.523 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Aug 19 01:25:21 smithi083 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-mgr-x[17344]: 2024-08-19T01:25:21.443+0000 7f55a4123040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-08-19T01:25:21.524 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Aug 19 01:25:21 smithi083 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-mgr-x[17344]: 2024-08-19T01:25:21.520+0000 7f55a4123040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-08-19T01:25:21.795 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Aug 19 01:25:21 smithi083 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-mgr-x[17344]: 2024-08-19T01:25:21.690+0000 7f55a4123040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-08-19T01:25:22.055 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:25:21 smithi083 ceph-mon[17120]: from='client.? 172.21.15.83:0/3349291744' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-08-19T01:25:22.055 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Aug 19 01:25:21 smithi083 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-mgr-x[17344]: 2024-08-19T01:25:21.880+0000 7f55a4123040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-08-19T01:25:22.056 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Aug 19 01:25:22 smithi083 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-mgr-x[17344]: 2024-08-19T01:25:22.053+0000 7f55a4123040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-08-19T01:25:22.359 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Aug 19 01:25:22 smithi083 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-mgr-x[17344]: 2024-08-19T01:25:22.202+0000 7f55a4123040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-08-19T01:25:22.359 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Aug 19 01:25:22 smithi083 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-mgr-x[17344]: 2024-08-19T01:25:22.356+0000 7f55a4123040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-08-19T01:25:22.558 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout 2024-08-19T01:25:22.558 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout { 2024-08-19T01:25:22.559 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "fsid": "bad8780e-5dc9-11ef-bccf-c7b262605968", 2024-08-19T01:25:22.559 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "health": { 2024-08-19T01:25:22.559 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-08-19T01:25:22.559 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-08-19T01:25:22.559 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-08-19T01:25:22.559 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout }, 2024-08-19T01:25:22.559 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-08-19T01:25:22.559 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-08-19T01:25:22.559 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout 0 2024-08-19T01:25:22.559 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout ], 2024-08-19T01:25:22.559 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-08-19T01:25:22.559 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "a" 2024-08-19T01:25:22.560 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout ], 2024-08-19T01:25:22.560 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "quorum_age": 3, 2024-08-19T01:25:22.560 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "monmap": { 2024-08-19T01:25:22.560 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-08-19T01:25:22.560 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-08-19T01:25:22.560 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-08-19T01:25:22.560 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout }, 2024-08-19T01:25:22.560 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-08-19T01:25:22.560 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-08-19T01:25:22.560 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-08-19T01:25:22.560 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-08-19T01:25:22.561 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-08-19T01:25:22.561 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-08-19T01:25:22.561 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-08-19T01:25:22.561 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-08-19T01:25:22.561 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout }, 2024-08-19T01:25:22.561 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-08-19T01:25:22.561 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-08-19T01:25:22.561 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-08-19T01:25:22.561 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-08-19T01:25:22.561 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-08-19T01:25:22.562 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-08-19T01:25:22.562 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-08-19T01:25:22.562 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-08-19T01:25:22.562 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-08-19T01:25:22.562 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout }, 2024-08-19T01:25:22.562 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-08-19T01:25:22.562 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-08-19T01:25:22.562 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "btime": "2024-08-19T01:25:12:626585+0000", 2024-08-19T01:25:22.562 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-08-19T01:25:22.562 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-08-19T01:25:22.563 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout }, 2024-08-19T01:25:22.563 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-08-19T01:25:22.563 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "available": false, 2024-08-19T01:25:22.563 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-08-19T01:25:22.563 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "modules": [ 2024-08-19T01:25:22.563 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "iostat", 2024-08-19T01:25:22.563 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "nfs", 2024-08-19T01:25:22.563 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "restful" 2024-08-19T01:25:22.563 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout ], 2024-08-19T01:25:22.564 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "services": {} 2024-08-19T01:25:22.564 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout }, 2024-08-19T01:25:22.564 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-08-19T01:25:22.564 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-08-19T01:25:22.564 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "modified": "2024-08-19T01:25:12.627369+0000", 2024-08-19T01:25:22.564 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "services": {} 2024-08-19T01:25:22.564 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout }, 2024-08-19T01:25:22.564 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-08-19T01:25:22.564 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout } 2024-08-19T01:25:22.564 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:21.792+0000 7f78aba00640 1 Processor -- start 2024-08-19T01:25:22.564 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:21.792+0000 7f78aba00640 1 -- start start 2024-08-19T01:25:22.564 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:21.793+0000 7f78aba00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f78a410aac0 0x7f78a410aec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:25:22.564 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:21.793+0000 7f78aba00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f78a410b490 con 0x7f78a410aac0 2024-08-19T01:25:22.565 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:21.793+0000 7f78a9000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f78a410aac0 0x7f78a410aec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:25:22.565 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:21.793+0000 7f78a9000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f78a410aac0 0x7f78a410aec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:56978/0 (socket says 172.21.15.83:56978) 2024-08-19T01:25:22.565 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:21.793+0000 7f78a9000640 1 -- 172.21.15.83:0/1132519122 learned_addr learned my addr 172.21.15.83:0/1132519122 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-08-19T01:25:22.565 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:21.794+0000 7f78a9000640 1 -- 172.21.15.83:0/1132519122 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f78a410b610 con 0x7f78a410aac0 2024-08-19T01:25:22.565 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:21.795+0000 7f78a9000640 1 --2- 172.21.15.83:0/1132519122 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f78a410aac0 0x7f78a410aec0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f7894009820 tx=0x7f789402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6dd9ba962b6c28b server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:25:22.565 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:21.795+0000 7f78a3e00640 1 -- 172.21.15.83:0/1132519122 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7894037070 con 0x7f78a410aac0 2024-08-19T01:25:22.565 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:21.795+0000 7f78a3e00640 1 -- 172.21.15.83:0/1132519122 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f7894032900 con 0x7f78a410aac0 2024-08-19T01:25:22.565 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:21.795+0000 7f78a3e00640 1 -- 172.21.15.83:0/1132519122 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7894030850 con 0x7f78a410aac0 2024-08-19T01:25:22.565 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:21.796+0000 7f78aba00640 1 -- 172.21.15.83:0/1132519122 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f78a410aac0 msgr2=0x7f78a410aec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:25:22.565 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:21.796+0000 7f78aba00640 1 --2- 172.21.15.83:0/1132519122 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f78a410aac0 0x7f78a410aec0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f7894009820 tx=0x7f789402a270 comp rx=0 tx=0).stop 2024-08-19T01:25:22.565 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:21.796+0000 7f78aba00640 1 -- 172.21.15.83:0/1132519122 shutdown_connections 2024-08-19T01:25:22.565 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:21.796+0000 7f78aba00640 1 --2- 172.21.15.83:0/1132519122 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f78a410aac0 0x7f78a410aec0 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:25:22.565 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:21.796+0000 7f78aba00640 1 -- 172.21.15.83:0/1132519122 >> 172.21.15.83:0/1132519122 conn(0x7f78a407cc40 msgr2=0x7f78a407d050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:25:22.566 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:21.796+0000 7f78aba00640 1 -- 172.21.15.83:0/1132519122 shutdown_connections 2024-08-19T01:25:22.566 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:21.796+0000 7f78aba00640 1 -- 172.21.15.83:0/1132519122 wait complete. 2024-08-19T01:25:22.566 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:21.797+0000 7f78aba00640 1 Processor -- start 2024-08-19T01:25:22.566 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:21.797+0000 7f78aba00640 1 -- start start 2024-08-19T01:25:22.566 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:21.798+0000 7f78aba00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f78a410aac0 0x7f78a419d690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:25:22.566 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:21.798+0000 7f78aba00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f78a410db20 con 0x7f78a410aac0 2024-08-19T01:25:22.566 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:21.798+0000 7f78a9000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f78a410aac0 0x7f78a419d690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:25:22.566 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:21.798+0000 7f78a9000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f78a410aac0 0x7f78a419d690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:56992/0 (socket says 172.21.15.83:56992) 2024-08-19T01:25:22.566 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:21.798+0000 7f78a9000640 1 -- 172.21.15.83:0/3349291744 learned_addr learned my addr 172.21.15.83:0/3349291744 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-08-19T01:25:22.566 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:21.798+0000 7f78a9000640 1 -- 172.21.15.83:0/3349291744 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f78a419dbd0 con 0x7f78a410aac0 2024-08-19T01:25:22.566 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:21.799+0000 7f78a9000640 1 --2- 172.21.15.83:0/3349291744 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f78a410aac0 0x7f78a419d690 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f78940097f0 tx=0x7f7894030ec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:25:22.566 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:21.799+0000 7f78a2000640 1 -- 172.21.15.83:0/3349291744 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7894042070 con 0x7f78a410aac0 2024-08-19T01:25:22.566 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:21.799+0000 7f78a2000640 1 -- 172.21.15.83:0/3349291744 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f7894032ca0 con 0x7f78a410aac0 2024-08-19T01:25:22.567 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:21.799+0000 7f78aba00640 1 -- 172.21.15.83:0/3349291744 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f78a419de60 con 0x7f78a410aac0 2024-08-19T01:25:22.567 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:21.799+0000 7f78a2000640 1 -- 172.21.15.83:0/3349291744 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7894037040 con 0x7f78a410aac0 2024-08-19T01:25:22.567 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:21.799+0000 7f78aba00640 1 -- 172.21.15.83:0/3349291744 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f78a41a0b50 con 0x7f78a410aac0 2024-08-19T01:25:22.567 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:21.799+0000 7f78a2000640 1 -- 172.21.15.83:0/3349291744 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f7894045070 con 0x7f78a410aac0 2024-08-19T01:25:22.567 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:21.799+0000 7f78a2000640 1 -- 172.21.15.83:0/3349291744 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f7894047a30 con 0x7f78a410aac0 2024-08-19T01:25:22.567 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:21.800+0000 7f78aba00640 1 -- 172.21.15.83:0/3349291744 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f78a410dca0 con 0x7f78a410aac0 2024-08-19T01:25:22.567 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:21.803+0000 7f78a2000640 1 -- 172.21.15.83:0/3349291744 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73482 (secure 0 0 0) 0x7f789403da60 con 0x7f78a410aac0 2024-08-19T01:25:22.567 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:21.866+0000 7f78aba00640 1 -- 172.21.15.83:0/3349291744 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7f78a41a1030 con 0x7f78a410aac0 2024-08-19T01:25:22.567 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:21.867+0000 7f78a2000640 1 -- 172.21.15.83:0/3349291744 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1291 (secure 0 0 0) 0x7f789403d430 con 0x7f78a410aac0 2024-08-19T01:25:22.567 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:21.868+0000 7f78aba00640 1 -- 172.21.15.83:0/3349291744 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f78a410aac0 msgr2=0x7f78a419d690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:25:22.567 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:21.868+0000 7f78aba00640 1 --2- 172.21.15.83:0/3349291744 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f78a410aac0 0x7f78a419d690 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f78940097f0 tx=0x7f7894030ec0 comp rx=0 tx=0).stop 2024-08-19T01:25:22.567 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:21.868+0000 7f78aba00640 1 -- 172.21.15.83:0/3349291744 shutdown_connections 2024-08-19T01:25:22.568 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:21.868+0000 7f78aba00640 1 --2- 172.21.15.83:0/3349291744 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f78a410aac0 0x7f78a419d690 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:25:22.568 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:21.868+0000 7f78aba00640 1 -- 172.21.15.83:0/3349291744 >> 172.21.15.83:0/3349291744 conn(0x7f78a407cc40 msgr2=0x7f78a4107550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:25:22.568 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:21.868+0000 7f78aba00640 1 -- 172.21.15.83:0/3349291744 shutdown_connections 2024-08-19T01:25:22.568 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:21.868+0000 7f78aba00640 1 -- 172.21.15.83:0/3349291744 wait complete. 2024-08-19T01:25:22.568 INFO:teuthology.orchestra.run.smithi083.stdout:mgr not available, waiting (1/15)... 2024-08-19T01:25:22.655 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Aug 19 01:25:22 smithi083 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-mgr-x[17344]: 2024-08-19T01:25:22.424+0000 7f55a4123040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-08-19T01:25:23.125 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Aug 19 01:25:22 smithi083 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-mgr-x[17344]: 2024-08-19T01:25:22.791+0000 7f55a4123040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-08-19T01:25:23.125 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Aug 19 01:25:22 smithi083 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-mgr-x[17344]: 2024-08-19T01:25:22.862+0000 7f55a4123040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-08-19T01:25:23.125 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Aug 19 01:25:22 smithi083 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-mgr-x[17344]: 2024-08-19T01:25:22.939+0000 7f55a4123040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-08-19T01:25:23.125 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Aug 19 01:25:23 smithi083 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-mgr-x[17344]: 2024-08-19T01:25:23.122+0000 7f55a4123040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-08-19T01:25:23.405 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Aug 19 01:25:23 smithi083 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-mgr-x[17344]: 2024-08-19T01:25:23.192+0000 7f55a4123040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-08-19T01:25:23.405 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Aug 19 01:25:23 smithi083 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-mgr-x[17344]: 2024-08-19T01:25:23.269+0000 7f55a4123040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-08-19T01:25:23.905 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Aug 19 01:25:23 smithi083 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-mgr-x[17344]: 2024-08-19T01:25:23.475+0000 7f55a4123040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-08-19T01:25:23.905 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Aug 19 01:25:23 smithi083 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-mgr-x[17344]: 2024-08-19T01:25:23.546+0000 7f55a4123040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-08-19T01:25:23.905 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Aug 19 01:25:23 smithi083 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-mgr-x[17344]: 2024-08-19T01:25:23.615+0000 7f55a4123040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-08-19T01:25:23.905 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Aug 19 01:25:23 smithi083 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-mgr-x[17344]: 2024-08-19T01:25:23.694+0000 7f55a4123040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-08-19T01:25:24.405 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Aug 19 01:25:23 smithi083 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-mgr-x[17344]: 2024-08-19T01:25:23.932+0000 7f55a4123040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-08-19T01:25:24.405 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Aug 19 01:25:24 smithi083 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-mgr-x[17344]: 2024-08-19T01:25:24.002+0000 7f55a4123040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-08-19T01:25:24.934 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Aug 19 01:25:24 smithi083 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-mgr-x[17344]: 2024-08-19T01:25:24.673+0000 7f55a4123040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-08-19T01:25:25.405 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:25:25 smithi083 ceph-mon[17120]: from='client.? 172.21.15.83:0/1534620399' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-08-19T01:25:25.405 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Aug 19 01:25:25 smithi083 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-mgr-x[17344]: 2024-08-19T01:25:25.056+0000 7f55a4123040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-08-19T01:25:25.779 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Aug 19 01:25:25 smithi083 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-mgr-x[17344]: 2024-08-19T01:25:25.595+0000 7f55a4123040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-08-19T01:25:25.779 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Aug 19 01:25:25 smithi083 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-mgr-x[17344]: 2024-08-19T01:25:25.660+0000 7f55a4123040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-08-19T01:25:25.780 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Aug 19 01:25:25 smithi083 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-mgr-x[17344]: 2024-08-19T01:25:25.733+0000 7f55a4123040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-08-19T01:25:25.787 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout 2024-08-19T01:25:25.787 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout { 2024-08-19T01:25:25.788 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "fsid": "bad8780e-5dc9-11ef-bccf-c7b262605968", 2024-08-19T01:25:25.788 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "health": { 2024-08-19T01:25:25.788 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-08-19T01:25:25.788 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-08-19T01:25:25.788 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-08-19T01:25:25.788 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout }, 2024-08-19T01:25:25.788 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-08-19T01:25:25.788 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-08-19T01:25:25.788 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout 0 2024-08-19T01:25:25.788 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout ], 2024-08-19T01:25:25.788 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-08-19T01:25:25.788 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "a" 2024-08-19T01:25:25.788 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout ], 2024-08-19T01:25:25.789 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "quorum_age": 6, 2024-08-19T01:25:25.789 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "monmap": { 2024-08-19T01:25:25.789 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-08-19T01:25:25.789 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-08-19T01:25:25.789 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-08-19T01:25:25.789 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout }, 2024-08-19T01:25:25.789 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-08-19T01:25:25.789 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-08-19T01:25:25.789 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-08-19T01:25:25.789 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-08-19T01:25:25.789 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-08-19T01:25:25.797 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-08-19T01:25:25.797 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-08-19T01:25:25.797 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-08-19T01:25:25.797 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout }, 2024-08-19T01:25:25.797 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-08-19T01:25:25.797 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-08-19T01:25:25.797 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-08-19T01:25:25.797 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-08-19T01:25:25.797 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-08-19T01:25:25.797 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-08-19T01:25:25.797 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-08-19T01:25:25.798 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-08-19T01:25:25.798 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-08-19T01:25:25.798 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout }, 2024-08-19T01:25:25.798 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-08-19T01:25:25.798 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-08-19T01:25:25.798 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "btime": "2024-08-19T01:25:12:626585+0000", 2024-08-19T01:25:25.798 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-08-19T01:25:25.798 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-08-19T01:25:25.798 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout }, 2024-08-19T01:25:25.798 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-08-19T01:25:25.798 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "available": false, 2024-08-19T01:25:25.798 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-08-19T01:25:25.799 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "modules": [ 2024-08-19T01:25:25.799 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "iostat", 2024-08-19T01:25:25.799 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "nfs", 2024-08-19T01:25:25.799 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "restful" 2024-08-19T01:25:25.799 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout ], 2024-08-19T01:25:25.799 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "services": {} 2024-08-19T01:25:25.799 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout }, 2024-08-19T01:25:25.799 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-08-19T01:25:25.799 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-08-19T01:25:25.799 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "modified": "2024-08-19T01:25:12.627369+0000", 2024-08-19T01:25:25.799 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "services": {} 2024-08-19T01:25:25.799 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout }, 2024-08-19T01:25:25.799 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-08-19T01:25:25.800 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout } 2024-08-19T01:25:25.800 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:25.037+0000 7f0d8c200640 1 Processor -- start 2024-08-19T01:25:25.800 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:25.038+0000 7f0d8c200640 1 -- start start 2024-08-19T01:25:25.800 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:25.038+0000 7f0d8c200640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0d84105ca0 0x7f0d841060a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:25:25.800 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:25.038+0000 7f0d8c200640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f0d8407be30 con 0x7f0d84105ca0 2024-08-19T01:25:25.800 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:25.038+0000 7f0d89800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0d84105ca0 0x7f0d841060a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:25:25.800 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:25.039+0000 7f0d89800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0d84105ca0 0x7f0d841060a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:56996/0 (socket says 172.21.15.83:56996) 2024-08-19T01:25:25.800 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:25.039+0000 7f0d89800640 1 -- 172.21.15.83:0/4181522340 learned_addr learned my addr 172.21.15.83:0/4181522340 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-08-19T01:25:25.800 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:25.039+0000 7f0d89800640 1 -- 172.21.15.83:0/4181522340 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0d841065e0 con 0x7f0d84105ca0 2024-08-19T01:25:25.800 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:25.040+0000 7f0d89800640 1 --2- 172.21.15.83:0/4181522340 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0d84105ca0 0x7f0d841060a0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f0d78009820 tx=0x7f0d7802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=169b1c396847edf6 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:25:25.800 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:25.040+0000 7f0d83e00640 1 -- 172.21.15.83:0/4181522340 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0d78037070 con 0x7f0d84105ca0 2024-08-19T01:25:25.800 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:25.040+0000 7f0d83e00640 1 -- 172.21.15.83:0/4181522340 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f0d78032900 con 0x7f0d84105ca0 2024-08-19T01:25:25.800 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:25.040+0000 7f0d83e00640 1 -- 172.21.15.83:0/4181522340 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0d78030850 con 0x7f0d84105ca0 2024-08-19T01:25:25.801 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:25.041+0000 7f0d8c200640 1 -- 172.21.15.83:0/4181522340 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0d84105ca0 msgr2=0x7f0d841060a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:25:25.801 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:25.041+0000 7f0d8c200640 1 --2- 172.21.15.83:0/4181522340 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0d84105ca0 0x7f0d841060a0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f0d78009820 tx=0x7f0d7802a270 comp rx=0 tx=0).stop 2024-08-19T01:25:25.801 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:25.041+0000 7f0d8c200640 1 -- 172.21.15.83:0/4181522340 shutdown_connections 2024-08-19T01:25:25.801 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:25.041+0000 7f0d8c200640 1 --2- 172.21.15.83:0/4181522340 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0d84105ca0 0x7f0d841060a0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:25:25.801 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:25.041+0000 7f0d8c200640 1 -- 172.21.15.83:0/4181522340 >> 172.21.15.83:0/4181522340 conn(0x7f0d84101b50 msgr2=0x7f0d84103f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:25:25.801 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:25.041+0000 7f0d8c200640 1 -- 172.21.15.83:0/4181522340 shutdown_connections 2024-08-19T01:25:25.801 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:25.041+0000 7f0d8c200640 1 -- 172.21.15.83:0/4181522340 wait complete. 2024-08-19T01:25:25.801 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:25.043+0000 7f0d8c200640 1 Processor -- start 2024-08-19T01:25:25.801 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:25.043+0000 7f0d8c200640 1 -- start start 2024-08-19T01:25:25.801 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:25.043+0000 7f0d8c200640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0d84105ca0 0x7f0d8419d660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:25:25.801 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:25.043+0000 7f0d8c200640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f0d841076e0 con 0x7f0d84105ca0 2024-08-19T01:25:25.801 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:25.043+0000 7f0d89800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0d84105ca0 0x7f0d8419d660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:25:25.801 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:25.043+0000 7f0d89800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0d84105ca0 0x7f0d8419d660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:57010/0 (socket says 172.21.15.83:57010) 2024-08-19T01:25:25.801 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:25.043+0000 7f0d89800640 1 -- 172.21.15.83:0/1534620399 learned_addr learned my addr 172.21.15.83:0/1534620399 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-08-19T01:25:25.802 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:25.044+0000 7f0d89800640 1 -- 172.21.15.83:0/1534620399 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0d8419dba0 con 0x7f0d84105ca0 2024-08-19T01:25:25.802 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:25.044+0000 7f0d89800640 1 --2- 172.21.15.83:0/1534620399 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0d84105ca0 0x7f0d8419d660 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f0d780097f0 tx=0x7f0d78030ec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:25:25.802 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:25.044+0000 7f0d82000640 1 -- 172.21.15.83:0/1534620399 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0d78042070 con 0x7f0d84105ca0 2024-08-19T01:25:25.802 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:25.044+0000 7f0d82000640 1 -- 172.21.15.83:0/1534620399 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f0d78032ca0 con 0x7f0d84105ca0 2024-08-19T01:25:25.802 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:25.044+0000 7f0d8c200640 1 -- 172.21.15.83:0/1534620399 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0d8419de30 con 0x7f0d84105ca0 2024-08-19T01:25:25.802 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:25.045+0000 7f0d82000640 1 -- 172.21.15.83:0/1534620399 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0d78037040 con 0x7f0d84105ca0 2024-08-19T01:25:25.802 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:25.045+0000 7f0d8c200640 1 -- 172.21.15.83:0/1534620399 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0d841a0b20 con 0x7f0d84105ca0 2024-08-19T01:25:25.802 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:25.045+0000 7f0d82000640 1 -- 172.21.15.83:0/1534620399 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f0d78045070 con 0x7f0d84105ca0 2024-08-19T01:25:25.802 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:25.045+0000 7f0d82000640 1 -- 172.21.15.83:0/1534620399 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f0d78047aa0 con 0x7f0d84105ca0 2024-08-19T01:25:25.802 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:25.046+0000 7f0d8c200640 1 -- 172.21.15.83:0/1534620399 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0d841060a0 con 0x7f0d84105ca0 2024-08-19T01:25:25.802 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:25.048+0000 7f0d82000640 1 -- 172.21.15.83:0/1534620399 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73482 (secure 0 0 0) 0x7f0d7803db10 con 0x7f0d84105ca0 2024-08-19T01:25:25.802 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:25.112+0000 7f0d8c200640 1 -- 172.21.15.83:0/1534620399 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7f0d841a0cf0 con 0x7f0d84105ca0 2024-08-19T01:25:25.803 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:25.112+0000 7f0d82000640 1 -- 172.21.15.83:0/1534620399 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1291 (secure 0 0 0) 0x7f0d7803d430 con 0x7f0d84105ca0 2024-08-19T01:25:25.803 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:25.113+0000 7f0d8c200640 1 -- 172.21.15.83:0/1534620399 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0d84105ca0 msgr2=0x7f0d8419d660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:25:25.803 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:25.113+0000 7f0d8c200640 1 --2- 172.21.15.83:0/1534620399 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0d84105ca0 0x7f0d8419d660 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f0d780097f0 tx=0x7f0d78030ec0 comp rx=0 tx=0).stop 2024-08-19T01:25:25.803 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:25.113+0000 7f0d8c200640 1 -- 172.21.15.83:0/1534620399 shutdown_connections 2024-08-19T01:25:25.803 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:25.113+0000 7f0d8c200640 1 --2- 172.21.15.83:0/1534620399 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0d84105ca0 0x7f0d8419d660 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:25:25.803 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:25.113+0000 7f0d8c200640 1 -- 172.21.15.83:0/1534620399 >> 172.21.15.83:0/1534620399 conn(0x7f0d84101b50 msgr2=0x7f0d84191950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:25:25.803 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:25.113+0000 7f0d8c200640 1 -- 172.21.15.83:0/1534620399 shutdown_connections 2024-08-19T01:25:25.803 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:25.113+0000 7f0d8c200640 1 -- 172.21.15.83:0/1534620399 wait complete. 2024-08-19T01:25:25.803 INFO:teuthology.orchestra.run.smithi083.stdout:mgr not available, waiting (2/15)... 2024-08-19T01:25:27.406 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:25:27 smithi083 ceph-mon[17120]: Activating manager daemon x 2024-08-19T01:25:27.406 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:25:27 smithi083 ceph-mon[17120]: mgrmap e2: x(active, starting, since 0.00538717s) 2024-08-19T01:25:27.406 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:25:27 smithi083 ceph-mon[17120]: from='mgr.14100 172.21.15.83:0/4188407881' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-08-19T01:25:27.406 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:25:27 smithi083 ceph-mon[17120]: from='mgr.14100 172.21.15.83:0/4188407881' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-08-19T01:25:27.406 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:25:27 smithi083 ceph-mon[17120]: from='mgr.14100 172.21.15.83:0/4188407881' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-08-19T01:25:27.406 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:25:27 smithi083 ceph-mon[17120]: from='mgr.14100 172.21.15.83:0/4188407881' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-08-19T01:25:27.406 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:25:27 smithi083 ceph-mon[17120]: from='mgr.14100 172.21.15.83:0/4188407881' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-08-19T01:25:27.406 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:25:27 smithi083 ceph-mon[17120]: Manager daemon x is now available 2024-08-19T01:25:27.407 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:25:27 smithi083 ceph-mon[17120]: from='mgr.14100 172.21.15.83:0/4188407881' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-08-19T01:25:27.407 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:25:27 smithi083 ceph-mon[17120]: from='mgr.14100 172.21.15.83:0/4188407881' entity='mgr.x' 2024-08-19T01:25:27.407 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:25:27 smithi083 ceph-mon[17120]: from='mgr.14100 172.21.15.83:0/4188407881' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-08-19T01:25:27.407 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:25:27 smithi083 ceph-mon[17120]: from='mgr.14100 172.21.15.83:0/4188407881' entity='mgr.x' 2024-08-19T01:25:27.407 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:25:27 smithi083 ceph-mon[17120]: from='mgr.14100 172.21.15.83:0/4188407881' entity='mgr.x' 2024-08-19T01:25:29.169 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:25:29 smithi083 ceph-mon[17120]: mgrmap e3: x(active, since 1.0087s) 2024-08-19T01:25:29.169 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:25:29 smithi083 ceph-mon[17120]: from='client.? 172.21.15.83:0/3326888951' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-08-19T01:25:29.176 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout 2024-08-19T01:25:29.176 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout { 2024-08-19T01:25:29.177 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "fsid": "bad8780e-5dc9-11ef-bccf-c7b262605968", 2024-08-19T01:25:29.177 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "health": { 2024-08-19T01:25:29.177 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-08-19T01:25:29.177 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-08-19T01:25:29.177 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-08-19T01:25:29.177 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout }, 2024-08-19T01:25:29.177 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-08-19T01:25:29.177 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-08-19T01:25:29.177 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout 0 2024-08-19T01:25:29.177 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout ], 2024-08-19T01:25:29.177 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-08-19T01:25:29.177 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "a" 2024-08-19T01:25:29.177 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout ], 2024-08-19T01:25:29.177 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "quorum_age": 9, 2024-08-19T01:25:29.177 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "monmap": { 2024-08-19T01:25:29.178 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-08-19T01:25:29.178 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-08-19T01:25:29.178 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-08-19T01:25:29.178 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout }, 2024-08-19T01:25:29.178 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-08-19T01:25:29.178 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-08-19T01:25:29.178 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-08-19T01:25:29.178 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-08-19T01:25:29.178 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-08-19T01:25:29.190 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-08-19T01:25:29.190 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-08-19T01:25:29.190 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-08-19T01:25:29.190 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout }, 2024-08-19T01:25:29.191 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-08-19T01:25:29.191 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-08-19T01:25:29.191 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-08-19T01:25:29.191 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-08-19T01:25:29.191 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-08-19T01:25:29.191 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-08-19T01:25:29.191 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-08-19T01:25:29.191 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-08-19T01:25:29.191 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-08-19T01:25:29.191 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout }, 2024-08-19T01:25:29.191 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-08-19T01:25:29.191 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-08-19T01:25:29.191 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "btime": "2024-08-19T01:25:12:626585+0000", 2024-08-19T01:25:29.192 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-08-19T01:25:29.192 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-08-19T01:25:29.192 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout }, 2024-08-19T01:25:29.192 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-08-19T01:25:29.192 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "available": true, 2024-08-19T01:25:29.192 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-08-19T01:25:29.192 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "modules": [ 2024-08-19T01:25:29.192 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "iostat", 2024-08-19T01:25:29.192 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "nfs", 2024-08-19T01:25:29.192 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "restful" 2024-08-19T01:25:29.192 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout ], 2024-08-19T01:25:29.192 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "services": {} 2024-08-19T01:25:29.192 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout }, 2024-08-19T01:25:29.192 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-08-19T01:25:29.192 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-08-19T01:25:29.192 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "modified": "2024-08-19T01:25:12.627369+0000", 2024-08-19T01:25:29.193 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "services": {} 2024-08-19T01:25:29.193 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout }, 2024-08-19T01:25:29.193 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-08-19T01:25:29.193 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout } 2024-08-19T01:25:29.193 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:28.251+0000 7fc2a2000640 1 Processor -- start 2024-08-19T01:25:29.193 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:28.251+0000 7fc2a2000640 1 -- start start 2024-08-19T01:25:29.193 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:28.251+0000 7fc2a2000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc29c107450 0x7fc29c107850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:25:29.193 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:28.251+0000 7fc2a2000640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fc29c107e20 con 0x7fc29c107450 2024-08-19T01:25:29.193 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:28.252+0000 7fc29b400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc29c107450 0x7fc29c107850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:25:29.193 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:28.252+0000 7fc29b400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc29c107450 0x7fc29c107850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:57092/0 (socket says 172.21.15.83:57092) 2024-08-19T01:25:29.193 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:28.252+0000 7fc29b400640 1 -- 172.21.15.83:0/3789665718 learned_addr learned my addr 172.21.15.83:0/3789665718 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-08-19T01:25:29.193 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:28.252+0000 7fc29b400640 1 -- 172.21.15.83:0/3789665718 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc29c108650 con 0x7fc29c107450 2024-08-19T01:25:29.193 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:28.253+0000 7fc29b400640 1 --2- 172.21.15.83:0/3789665718 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc29c107450 0x7fc29c107850 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fc28c009820 tx=0x7fc28c02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=24796c1cad6c7e73 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:25:29.194 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:28.254+0000 7fc29a000640 1 -- 172.21.15.83:0/3789665718 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc28c037070 con 0x7fc29c107450 2024-08-19T01:25:29.194 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:28.254+0000 7fc29a000640 1 -- 172.21.15.83:0/3789665718 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fc28c032900 con 0x7fc29c107450 2024-08-19T01:25:29.194 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:28.254+0000 7fc29a000640 1 -- 172.21.15.83:0/3789665718 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc28c030850 con 0x7fc29c107450 2024-08-19T01:25:29.194 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:28.254+0000 7fc2a2000640 1 -- 172.21.15.83:0/3789665718 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc29c107450 msgr2=0x7fc29c107850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:25:29.194 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:28.254+0000 7fc2a2000640 1 --2- 172.21.15.83:0/3789665718 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc29c107450 0x7fc29c107850 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fc28c009820 tx=0x7fc28c02a270 comp rx=0 tx=0).stop 2024-08-19T01:25:29.194 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:28.254+0000 7fc2a2000640 1 -- 172.21.15.83:0/3789665718 shutdown_connections 2024-08-19T01:25:29.194 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:28.254+0000 7fc2a2000640 1 --2- 172.21.15.83:0/3789665718 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc29c107450 0x7fc29c107850 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:25:29.194 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:28.254+0000 7fc2a2000640 1 -- 172.21.15.83:0/3789665718 >> 172.21.15.83:0/3789665718 conn(0x7fc29c102c00 msgr2=0x7fc29c105020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:25:29.194 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:28.254+0000 7fc2a2000640 1 -- 172.21.15.83:0/3789665718 shutdown_connections 2024-08-19T01:25:29.194 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:28.254+0000 7fc2a2000640 1 -- 172.21.15.83:0/3789665718 wait complete. 2024-08-19T01:25:29.194 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:28.255+0000 7fc2a2000640 1 Processor -- start 2024-08-19T01:25:29.194 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:28.256+0000 7fc2a2000640 1 -- start start 2024-08-19T01:25:29.194 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:28.256+0000 7fc2a2000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc29c107450 0x7fc29c19d6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:25:29.194 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:28.256+0000 7fc2a2000640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fc29c10ba00 con 0x7fc29c107450 2024-08-19T01:25:29.195 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:28.256+0000 7fc29b400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc29c107450 0x7fc29c19d6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:25:29.195 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:28.256+0000 7fc29b400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc29c107450 0x7fc29c19d6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:57108/0 (socket says 172.21.15.83:57108) 2024-08-19T01:25:29.195 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:28.256+0000 7fc29b400640 1 -- 172.21.15.83:0/3326888951 learned_addr learned my addr 172.21.15.83:0/3326888951 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-08-19T01:25:29.195 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:28.256+0000 7fc29b400640 1 -- 172.21.15.83:0/3326888951 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc29c19dc10 con 0x7fc29c107450 2024-08-19T01:25:29.195 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:28.257+0000 7fc29b400640 1 --2- 172.21.15.83:0/3326888951 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc29c107450 0x7fc29c19d6d0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7fc28c0097f0 tx=0x7fc28c030d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:25:29.195 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:28.257+0000 7fc293e00640 1 -- 172.21.15.83:0/3326888951 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc28c037040 con 0x7fc29c107450 2024-08-19T01:25:29.195 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:28.257+0000 7fc2a2000640 1 -- 172.21.15.83:0/3326888951 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc29c19dea0 con 0x7fc29c107450 2024-08-19T01:25:29.195 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:28.257+0000 7fc293e00640 1 -- 172.21.15.83:0/3326888951 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fc28c040070 con 0x7fc29c107450 2024-08-19T01:25:29.195 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:28.257+0000 7fc293e00640 1 -- 172.21.15.83:0/3326888951 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc28c03a390 con 0x7fc29c107450 2024-08-19T01:25:29.195 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:28.257+0000 7fc2a2000640 1 -- 172.21.15.83:0/3326888951 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc29c1a0b90 con 0x7fc29c107450 2024-08-19T01:25:29.195 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:28.258+0000 7fc293e00640 1 -- 172.21.15.83:0/3326888951 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 3) ==== 46170+0+0 (secure 0 0 0) 0x7fc28c032900 con 0x7fc29c107450 2024-08-19T01:25:29.195 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:28.258+0000 7fc2a2000640 1 -- 172.21.15.83:0/3326888951 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc29c1078d0 con 0x7fc29c107450 2024-08-19T01:25:29.195 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:28.259+0000 7fc293e00640 1 --2- 172.21.15.83:0/3326888951 >> [v2:172.21.15.83:6800/3261152818,v1:172.21.15.83:6801/3261152818] conn(0x7fc280038e10 0x7fc28003b2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:25:29.196 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:28.259+0000 7fc293e00640 1 -- 172.21.15.83:0/3326888951 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fc28c06bee0 con 0x7fc29c107450 2024-08-19T01:25:29.196 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:28.259+0000 7fc29aa00640 1 --2- 172.21.15.83:0/3326888951 >> [v2:172.21.15.83:6800/3261152818,v1:172.21.15.83:6801/3261152818] conn(0x7fc280038e10 0x7fc28003b2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:25:29.196 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:28.260+0000 7fc29aa00640 1 --2- 172.21.15.83:0/3326888951 >> [v2:172.21.15.83:6800/3261152818,v1:172.21.15.83:6801/3261152818] conn(0x7fc280038e10 0x7fc28003b2d0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fc2880098c0 tx=0x7fc288006db0 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:25:29.196 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:28.265+0000 7fc293e00640 1 -- 172.21.15.83:0/3326888951 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7fc28c032bf0 con 0x7fc29c107450 2024-08-19T01:25:29.196 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:28.506+0000 7fc2a2000640 1 -- 172.21.15.83:0/3326888951 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7fc29c19e320 con 0x7fc29c107450 2024-08-19T01:25:29.196 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:28.506+0000 7fc293e00640 1 -- 172.21.15.83:0/3326888951 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1290 (secure 0 0 0) 0x7fc28c040ca0 con 0x7fc29c107450 2024-08-19T01:25:29.196 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:28.510+0000 7fc2a2000640 1 -- 172.21.15.83:0/3326888951 >> [v2:172.21.15.83:6800/3261152818,v1:172.21.15.83:6801/3261152818] conn(0x7fc280038e10 msgr2=0x7fc28003b2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:25:29.196 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:28.510+0000 7fc2a2000640 1 --2- 172.21.15.83:0/3326888951 >> [v2:172.21.15.83:6800/3261152818,v1:172.21.15.83:6801/3261152818] conn(0x7fc280038e10 0x7fc28003b2d0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fc2880098c0 tx=0x7fc288006db0 comp rx=0 tx=0).stop 2024-08-19T01:25:29.196 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:28.510+0000 7fc2a2000640 1 -- 172.21.15.83:0/3326888951 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc29c107450 msgr2=0x7fc29c19d6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:25:29.196 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:28.510+0000 7fc2a2000640 1 --2- 172.21.15.83:0/3326888951 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc29c107450 0x7fc29c19d6d0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7fc28c0097f0 tx=0x7fc28c030d80 comp rx=0 tx=0).stop 2024-08-19T01:25:29.196 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:28.510+0000 7fc2a2000640 1 -- 172.21.15.83:0/3326888951 shutdown_connections 2024-08-19T01:25:29.196 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:28.510+0000 7fc2a2000640 1 --2- 172.21.15.83:0/3326888951 >> [v2:172.21.15.83:6800/3261152818,v1:172.21.15.83:6801/3261152818] conn(0x7fc280038e10 0x7fc28003b2d0 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:25:29.196 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:28.510+0000 7fc2a2000640 1 --2- 172.21.15.83:0/3326888951 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc29c107450 0x7fc29c19d6d0 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:25:29.196 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:28.510+0000 7fc2a2000640 1 -- 172.21.15.83:0/3326888951 >> 172.21.15.83:0/3326888951 conn(0x7fc29c102c00 msgr2=0x7fc29c103700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:25:29.196 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:28.510+0000 7fc2a2000640 1 -- 172.21.15.83:0/3326888951 shutdown_connections 2024-08-19T01:25:29.197 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:28.510+0000 7fc2a2000640 1 -- 172.21.15.83:0/3326888951 wait complete. 2024-08-19T01:25:29.197 INFO:teuthology.orchestra.run.smithi083.stdout:mgr is available 2024-08-19T01:25:30.155 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:25:30 smithi083 ceph-mon[17120]: mgrmap e4: x(active, since 2s) 2024-08-19T01:25:30.155 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:25:30 smithi083 ceph-mon[17120]: from='client.? 172.21.15.83:0/1110480226' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2024-08-19T01:25:30.512 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout 2024-08-19T01:25:30.512 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout [global] 2024-08-19T01:25:30.512 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout fsid = bad8780e-5dc9-11ef-bccf-c7b262605968 2024-08-19T01:25:30.512 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout mon_host = [v2:172.21.15.83:3300,v1:172.21.15.83:6789] 2024-08-19T01:25:30.512 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-08-19T01:25:30.512 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-08-19T01:25:30.512 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-08-19T01:25:30.512 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-08-19T01:25:30.513 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout 2024-08-19T01:25:30.513 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout [mgr] 2024-08-19T01:25:30.513 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-08-19T01:25:30.513 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout 2024-08-19T01:25:30.513 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout [osd] 2024-08-19T01:25:30.513 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2024-08-19T01:25:30.513 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2024-08-19T01:25:30.513 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2024-08-19T01:25:30.513 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-08-19T01:25:30.513 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-08-19T01:25:30.513 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:29.649+0000 7f0643a00640 1 Processor -- start 2024-08-19T01:25:30.513 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:29.649+0000 7f0643a00640 1 -- start start 2024-08-19T01:25:30.513 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:29.649+0000 7f0643a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f063c07de60 0x7f063c07c2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:25:30.513 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:29.650+0000 7f0643a00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f063c07c890 con 0x7f063c07de60 2024-08-19T01:25:30.514 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:29.650+0000 7f0641000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f063c07de60 0x7f063c07c2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:25:30.514 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:29.650+0000 7f0641000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f063c07de60 0x7f063c07c2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:57110/0 (socket says 172.21.15.83:57110) 2024-08-19T01:25:30.514 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:29.650+0000 7f0641000640 1 -- 172.21.15.83:0/2479776956 learned_addr learned my addr 172.21.15.83:0/2479776956 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-08-19T01:25:30.514 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:29.650+0000 7f0641000640 1 -- 172.21.15.83:0/2479776956 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f063c07ca10 con 0x7f063c07de60 2024-08-19T01:25:30.514 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:29.651+0000 7f0641000640 1 --2- 172.21.15.83:0/2479776956 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f063c07de60 0x7f063c07c2c0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f062c009820 tx=0x7f062c02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b0258a4ca0a1e17c server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:25:30.514 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:29.651+0000 7f063b400640 1 -- 172.21.15.83:0/2479776956 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f062c037070 con 0x7f063c07de60 2024-08-19T01:25:30.514 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:29.652+0000 7f063b400640 1 -- 172.21.15.83:0/2479776956 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f062c032900 con 0x7f063c07de60 2024-08-19T01:25:30.514 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:29.652+0000 7f063b400640 1 -- 172.21.15.83:0/2479776956 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f062c030850 con 0x7f063c07de60 2024-08-19T01:25:30.514 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:29.652+0000 7f0643a00640 1 -- 172.21.15.83:0/2479776956 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f063c07de60 msgr2=0x7f063c07c2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:25:30.514 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:29.652+0000 7f0643a00640 1 --2- 172.21.15.83:0/2479776956 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f063c07de60 0x7f063c07c2c0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f062c009820 tx=0x7f062c02a270 comp rx=0 tx=0).stop 2024-08-19T01:25:30.514 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:29.652+0000 7f0643a00640 1 -- 172.21.15.83:0/2479776956 shutdown_connections 2024-08-19T01:25:30.514 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:29.652+0000 7f0643a00640 1 --2- 172.21.15.83:0/2479776956 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f063c07de60 0x7f063c07c2c0 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:25:30.514 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:29.652+0000 7f0643a00640 1 -- 172.21.15.83:0/2479776956 >> 172.21.15.83:0/2479776956 conn(0x7f063c102c00 msgr2=0x7f063c105020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:25:30.515 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:29.652+0000 7f0643a00640 1 -- 172.21.15.83:0/2479776956 shutdown_connections 2024-08-19T01:25:30.515 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:29.652+0000 7f0643a00640 1 -- 172.21.15.83:0/2479776956 wait complete. 2024-08-19T01:25:30.515 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:29.653+0000 7f0643a00640 1 Processor -- start 2024-08-19T01:25:30.515 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:29.653+0000 7f0643a00640 1 -- start start 2024-08-19T01:25:30.515 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:29.654+0000 7f0643a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f063c07de60 0x7f063c1a1a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:25:30.515 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:29.654+0000 7f0643a00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f063c109670 con 0x7f063c07de60 2024-08-19T01:25:30.515 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:29.654+0000 7f0641000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f063c07de60 0x7f063c1a1a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:25:30.515 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:29.654+0000 7f0641000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f063c07de60 0x7f063c1a1a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:57116/0 (socket says 172.21.15.83:57116) 2024-08-19T01:25:30.515 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:29.654+0000 7f0641000640 1 -- 172.21.15.83:0/1110480226 learned_addr learned my addr 172.21.15.83:0/1110480226 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-08-19T01:25:30.515 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:29.654+0000 7f0641000640 1 -- 172.21.15.83:0/1110480226 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f063c1a1fc0 con 0x7f063c07de60 2024-08-19T01:25:30.515 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:29.655+0000 7f0641000640 1 --2- 172.21.15.83:0/1110480226 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f063c07de60 0x7f063c1a1a80 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f062c0097f0 tx=0x7f062c030d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:25:30.515 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:29.655+0000 7f0639600640 1 -- 172.21.15.83:0/1110480226 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f062c037040 con 0x7f063c07de60 2024-08-19T01:25:30.515 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:29.655+0000 7f0639600640 1 -- 172.21.15.83:0/1110480226 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f062c040070 con 0x7f063c07de60 2024-08-19T01:25:30.515 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:29.655+0000 7f0639600640 1 -- 172.21.15.83:0/1110480226 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f062c03a390 con 0x7f063c07de60 2024-08-19T01:25:30.516 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:29.655+0000 7f0643a00640 1 -- 172.21.15.83:0/1110480226 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f063c1a2250 con 0x7f063c07de60 2024-08-19T01:25:30.516 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:29.655+0000 7f0643a00640 1 -- 172.21.15.83:0/1110480226 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f063c1a4f40 con 0x7f063c07de60 2024-08-19T01:25:30.516 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:29.656+0000 7f0639600640 1 -- 172.21.15.83:0/1110480226 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 4) ==== 46276+0+0 (secure 0 0 0) 0x7f062c03b430 con 0x7f063c07de60 2024-08-19T01:25:30.516 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:29.656+0000 7f0643a00640 1 -- 172.21.15.83:0/1110480226 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f063c07c340 con 0x7f063c07de60 2024-08-19T01:25:30.516 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:29.657+0000 7f0639600640 1 --2- 172.21.15.83:0/1110480226 >> [v2:172.21.15.83:6800/3261152818,v1:172.21.15.83:6801/3261152818] conn(0x7f0620038f70 0x7f062003b430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:25:30.516 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:29.657+0000 7f0639600640 1 -- 172.21.15.83:0/1110480226 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f062c06c080 con 0x7f063c07de60 2024-08-19T01:25:30.516 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:29.657+0000 7f063be00640 1 --2- 172.21.15.83:0/1110480226 >> [v2:172.21.15.83:6800/3261152818,v1:172.21.15.83:6801/3261152818] conn(0x7f0620038f70 0x7f062003b430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:25:30.516 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:29.657+0000 7f063be00640 1 --2- 172.21.15.83:0/1110480226 >> [v2:172.21.15.83:6800/3261152818,v1:172.21.15.83:6801/3261152818] conn(0x7f0620038f70 0x7f062003b430 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f06240098c0 tx=0x7f0624006db0 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:25:30.516 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:29.662+0000 7f0639600640 1 -- 172.21.15.83:0/1110480226 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7f062c06bde0 con 0x7f063c07de60 2024-08-19T01:25:30.516 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:29.835+0000 7f0643a00640 1 -- 172.21.15.83:0/1110480226 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) -- 0x7f063c02d810 con 0x7f063c07de60 2024-08-19T01:25:30.516 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:29.836+0000 7f0639600640 1 -- 172.21.15.83:0/1110480226 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v3) ==== 70+0+429 (secure 0 0 0) 0x7f062c032900 con 0x7f063c07de60 2024-08-19T01:25:30.516 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:29.838+0000 7f0643a00640 1 -- 172.21.15.83:0/1110480226 >> [v2:172.21.15.83:6800/3261152818,v1:172.21.15.83:6801/3261152818] conn(0x7f0620038f70 msgr2=0x7f062003b430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:25:30.516 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:29.838+0000 7f0643a00640 1 --2- 172.21.15.83:0/1110480226 >> [v2:172.21.15.83:6800/3261152818,v1:172.21.15.83:6801/3261152818] conn(0x7f0620038f70 0x7f062003b430 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f06240098c0 tx=0x7f0624006db0 comp rx=0 tx=0).stop 2024-08-19T01:25:30.516 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:29.838+0000 7f0643a00640 1 -- 172.21.15.83:0/1110480226 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f063c07de60 msgr2=0x7f063c1a1a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:25:30.517 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:29.838+0000 7f0643a00640 1 --2- 172.21.15.83:0/1110480226 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f063c07de60 0x7f063c1a1a80 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f062c0097f0 tx=0x7f062c030d80 comp rx=0 tx=0).stop 2024-08-19T01:25:30.517 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:29.838+0000 7f0643a00640 1 -- 172.21.15.83:0/1110480226 shutdown_connections 2024-08-19T01:25:30.517 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:29.838+0000 7f0643a00640 1 --2- 172.21.15.83:0/1110480226 >> [v2:172.21.15.83:6800/3261152818,v1:172.21.15.83:6801/3261152818] conn(0x7f0620038f70 0x7f062003b430 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:25:30.517 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:29.838+0000 7f0643a00640 1 --2- 172.21.15.83:0/1110480226 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f063c07de60 0x7f063c1a1a80 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:25:30.517 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:29.838+0000 7f0643a00640 1 -- 172.21.15.83:0/1110480226 >> 172.21.15.83:0/1110480226 conn(0x7f063c102c00 msgr2=0x7f063c198740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:25:30.517 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:29.839+0000 7f0643a00640 1 -- 172.21.15.83:0/1110480226 shutdown_connections 2024-08-19T01:25:30.517 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:29.839+0000 7f0643a00640 1 -- 172.21.15.83:0/1110480226 wait complete. 2024-08-19T01:25:30.517 INFO:teuthology.orchestra.run.smithi083.stdout:Enabling cephadm module... 2024-08-19T01:25:31.618 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:25:31 smithi083 ceph-mon[17120]: from='client.? 172.21.15.83:0/4123695668' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "cephadm"}]: dispatch 2024-08-19T01:25:31.618 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Aug 19 01:25:31 smithi083 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-mgr-x[17344]: ignoring --setuser ceph since I am not root 2024-08-19T01:25:31.618 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Aug 19 01:25:31 smithi083 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-mgr-x[17344]: ignoring --setgroup ceph since I am not root 2024-08-19T01:25:31.618 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Aug 19 01:25:31 smithi083 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-mgr-x[17344]: 2024-08-19T01:25:31.615+0000 7ffae9723040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-08-19T01:25:31.905 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Aug 19 01:25:31 smithi083 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-mgr-x[17344]: 2024-08-19T01:25:31.695+0000 7ffae9723040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-08-19T01:25:31.905 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Aug 19 01:25:31 smithi083 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-mgr-x[17344]: 2024-08-19T01:25:31.860+0000 7ffae9723040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-08-19T01:25:32.025 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:31.006+0000 7f65d6200640 1 Processor -- start 2024-08-19T01:25:32.026 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:31.007+0000 7f65d6200640 1 -- start start 2024-08-19T01:25:32.026 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:31.007+0000 7f65d6200640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f65d010aac0 0x7f65d010aec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:25:32.026 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:31.007+0000 7f65d6200640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f65d010b490 con 0x7f65d010aac0 2024-08-19T01:25:32.026 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:31.007+0000 7f65cf400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f65d010aac0 0x7f65d010aec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:25:32.026 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:31.007+0000 7f65cf400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f65d010aac0 0x7f65d010aec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:57066/0 (socket says 172.21.15.83:57066) 2024-08-19T01:25:32.026 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:31.007+0000 7f65cf400640 1 -- 172.21.15.83:0/1922005591 learned_addr learned my addr 172.21.15.83:0/1922005591 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-08-19T01:25:32.026 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:31.008+0000 7f65cf400640 1 -- 172.21.15.83:0/1922005591 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f65d010b610 con 0x7f65d010aac0 2024-08-19T01:25:32.026 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:31.009+0000 7f65cf400640 1 --2- 172.21.15.83:0/1922005591 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f65d010aac0 0x7f65d010aec0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f65c0009820 tx=0x7f65c002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=58318b14d87f5422 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:25:32.026 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:31.009+0000 7f65ce000640 1 -- 172.21.15.83:0/1922005591 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f65c0037070 con 0x7f65d010aac0 2024-08-19T01:25:32.026 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:31.009+0000 7f65ce000640 1 -- 172.21.15.83:0/1922005591 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f65c0032900 con 0x7f65d010aac0 2024-08-19T01:25:32.026 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:31.009+0000 7f65ce000640 1 -- 172.21.15.83:0/1922005591 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f65c0030850 con 0x7f65d010aac0 2024-08-19T01:25:32.027 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:31.009+0000 7f65d6200640 1 -- 172.21.15.83:0/1922005591 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f65d010aac0 msgr2=0x7f65d010aec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:25:32.027 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:31.009+0000 7f65d6200640 1 --2- 172.21.15.83:0/1922005591 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f65d010aac0 0x7f65d010aec0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f65c0009820 tx=0x7f65c002a270 comp rx=0 tx=0).stop 2024-08-19T01:25:32.027 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:31.010+0000 7f65d6200640 1 -- 172.21.15.83:0/1922005591 shutdown_connections 2024-08-19T01:25:32.027 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:31.010+0000 7f65d6200640 1 --2- 172.21.15.83:0/1922005591 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f65d010aac0 0x7f65d010aec0 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:25:32.027 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:31.010+0000 7f65d6200640 1 -- 172.21.15.83:0/1922005591 >> 172.21.15.83:0/1922005591 conn(0x7f65d007cc40 msgr2=0x7f65d007d050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:25:32.027 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:31.010+0000 7f65d6200640 1 -- 172.21.15.83:0/1922005591 shutdown_connections 2024-08-19T01:25:32.027 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:31.010+0000 7f65d6200640 1 -- 172.21.15.83:0/1922005591 wait complete. 2024-08-19T01:25:32.027 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:31.011+0000 7f65d6200640 1 Processor -- start 2024-08-19T01:25:32.027 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:31.011+0000 7f65d6200640 1 -- start start 2024-08-19T01:25:32.027 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:31.011+0000 7f65d6200640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f65d010aac0 0x7f65d019d6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:25:32.027 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:31.011+0000 7f65d6200640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f65d010db20 con 0x7f65d010aac0 2024-08-19T01:25:32.027 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:31.011+0000 7f65cf400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f65d010aac0 0x7f65d019d6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:25:32.027 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:31.011+0000 7f65cf400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f65d010aac0 0x7f65d019d6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:57076/0 (socket says 172.21.15.83:57076) 2024-08-19T01:25:32.028 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:31.011+0000 7f65cf400640 1 -- 172.21.15.83:0/4123695668 learned_addr learned my addr 172.21.15.83:0/4123695668 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-08-19T01:25:32.028 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:31.012+0000 7f65cf400640 1 -- 172.21.15.83:0/4123695668 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f65d019dbe0 con 0x7f65d010aac0 2024-08-19T01:25:32.028 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:31.012+0000 7f65cf400640 1 --2- 172.21.15.83:0/4123695668 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f65d010aac0 0x7f65d019d6a0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f65c00097f0 tx=0x7f65c0030ec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:25:32.028 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:31.012+0000 7f65c7e00640 1 -- 172.21.15.83:0/4123695668 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f65c0042070 con 0x7f65d010aac0 2024-08-19T01:25:32.028 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:31.012+0000 7f65c7e00640 1 -- 172.21.15.83:0/4123695668 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f65c0032ca0 con 0x7f65d010aac0 2024-08-19T01:25:32.028 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:31.012+0000 7f65d6200640 1 -- 172.21.15.83:0/4123695668 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f65d019de70 con 0x7f65d010aac0 2024-08-19T01:25:32.028 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:31.012+0000 7f65c7e00640 1 -- 172.21.15.83:0/4123695668 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f65c0037040 con 0x7f65d010aac0 2024-08-19T01:25:32.028 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:31.012+0000 7f65d6200640 1 -- 172.21.15.83:0/4123695668 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f65d01a0b60 con 0x7f65d010aac0 2024-08-19T01:25:32.028 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:31.013+0000 7f65c7e00640 1 -- 172.21.15.83:0/4123695668 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 4) ==== 46276+0+0 (secure 0 0 0) 0x7f65c0045070 con 0x7f65d010aac0 2024-08-19T01:25:32.028 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:31.013+0000 7f65d6200640 1 -- 172.21.15.83:0/4123695668 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f65d010dca0 con 0x7f65d010aac0 2024-08-19T01:25:32.028 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:31.014+0000 7f65c7e00640 1 --2- 172.21.15.83:0/4123695668 >> [v2:172.21.15.83:6800/3261152818,v1:172.21.15.83:6801/3261152818] conn(0x7f65b4038f70 0x7f65b403b430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:25:32.028 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:31.014+0000 7f65c7e00640 1 -- 172.21.15.83:0/4123695668 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f65c006be50 con 0x7f65d010aac0 2024-08-19T01:25:32.028 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:31.014+0000 7f65cea00640 1 --2- 172.21.15.83:0/4123695668 >> [v2:172.21.15.83:6800/3261152818,v1:172.21.15.83:6801/3261152818] conn(0x7f65b4038f70 0x7f65b403b430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:25:32.028 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:31.015+0000 7f65cea00640 1 --2- 172.21.15.83:0/4123695668 >> [v2:172.21.15.83:6800/3261152818,v1:172.21.15.83:6801/3261152818] conn(0x7f65b4038f70 0x7f65b403b430 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f65bc0098c0 tx=0x7f65bc006db0 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:25:32.029 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:31.019+0000 7f65c7e00640 1 -- 172.21.15.83:0/4123695668 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7f65c004b170 con 0x7f65d010aac0 2024-08-19T01:25:32.029 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:31.241+0000 7f65d6200640 1 -- 172.21.15.83:0/4123695668 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "cephadm"} v 0) -- 0x7f65d01a11d0 con 0x7f65d010aac0 2024-08-19T01:25:32.029 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:31.293+0000 7f65c7e00640 1 -- 172.21.15.83:0/4123695668 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "cephadm"}]=0 v5) ==== 86+0+0 (secure 0 0 0) 0x7f65c006b870 con 0x7f65d010aac0 2024-08-19T01:25:32.029 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:31.295+0000 7f65c7e00640 1 -- 172.21.15.83:0/4123695668 <== mon.0 v2:172.21.15.83:3300/0 8 ==== mgrmap(e 5) ==== 46287+0+0 (secure 0 0 0) 0x7f65c0072080 con 0x7f65d010aac0 2024-08-19T01:25:32.029 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:31.297+0000 7f65d6200640 1 -- 172.21.15.83:0/4123695668 >> [v2:172.21.15.83:6800/3261152818,v1:172.21.15.83:6801/3261152818] conn(0x7f65b4038f70 msgr2=0x7f65b403b430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:25:32.029 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:31.297+0000 7f65d6200640 1 --2- 172.21.15.83:0/4123695668 >> [v2:172.21.15.83:6800/3261152818,v1:172.21.15.83:6801/3261152818] conn(0x7f65b4038f70 0x7f65b403b430 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f65bc0098c0 tx=0x7f65bc006db0 comp rx=0 tx=0).stop 2024-08-19T01:25:32.029 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:31.297+0000 7f65d6200640 1 -- 172.21.15.83:0/4123695668 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f65d010aac0 msgr2=0x7f65d019d6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:25:32.029 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:31.297+0000 7f65d6200640 1 --2- 172.21.15.83:0/4123695668 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f65d010aac0 0x7f65d019d6a0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f65c00097f0 tx=0x7f65c0030ec0 comp rx=0 tx=0).stop 2024-08-19T01:25:32.029 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:31.297+0000 7f65d6200640 1 -- 172.21.15.83:0/4123695668 shutdown_connections 2024-08-19T01:25:32.029 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:31.297+0000 7f65d6200640 1 --2- 172.21.15.83:0/4123695668 >> [v2:172.21.15.83:6800/3261152818,v1:172.21.15.83:6801/3261152818] conn(0x7f65b4038f70 0x7f65b403b430 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:25:32.029 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:31.297+0000 7f65d6200640 1 --2- 172.21.15.83:0/4123695668 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f65d010aac0 0x7f65d019d6a0 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:25:32.029 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:31.297+0000 7f65d6200640 1 -- 172.21.15.83:0/4123695668 >> 172.21.15.83:0/4123695668 conn(0x7f65d007cc40 msgr2=0x7f65d0107570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:25:32.029 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:31.297+0000 7f65d6200640 1 -- 172.21.15.83:0/4123695668 shutdown_connections 2024-08-19T01:25:32.030 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:31.297+0000 7f65d6200640 1 -- 172.21.15.83:0/4123695668 wait complete. 2024-08-19T01:25:32.301 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Aug 19 01:25:32 smithi083 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-mgr-x[17344]: 2024-08-19T01:25:32.045+0000 7ffae9723040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-08-19T01:25:32.301 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Aug 19 01:25:32 smithi083 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-mgr-x[17344]: 2024-08-19T01:25:32.221+0000 7ffae9723040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-08-19T01:25:32.609 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:25:32 smithi083 ceph-mon[17120]: from='client.? 172.21.15.83:0/4123695668' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "cephadm"}]': finished 2024-08-19T01:25:32.609 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:25:32 smithi083 ceph-mon[17120]: mgrmap e5: x(active, since 4s) 2024-08-19T01:25:32.609 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Aug 19 01:25:32 smithi083 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-mgr-x[17344]: 2024-08-19T01:25:32.379+0000 7ffae9723040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-08-19T01:25:32.610 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Aug 19 01:25:32 smithi083 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-mgr-x[17344]: 2024-08-19T01:25:32.536+0000 7ffae9723040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-08-19T01:25:32.610 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Aug 19 01:25:32 smithi083 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-mgr-x[17344]: 2024-08-19T01:25:32.607+0000 7ffae9723040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-08-19T01:25:33.302 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Aug 19 01:25:32 smithi083 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-mgr-x[17344]: 2024-08-19T01:25:32.992+0000 7ffae9723040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-08-19T01:25:33.302 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Aug 19 01:25:33 smithi083 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-mgr-x[17344]: 2024-08-19T01:25:33.069+0000 7ffae9723040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-08-19T01:25:33.302 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Aug 19 01:25:33 smithi083 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-mgr-x[17344]: 2024-08-19T01:25:33.151+0000 7ffae9723040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-08-19T01:25:33.488 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout { 2024-08-19T01:25:33.489 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "epoch": 5, 2024-08-19T01:25:33.489 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "available": true, 2024-08-19T01:25:33.489 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "active_name": "x", 2024-08-19T01:25:33.489 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-08-19T01:25:33.489 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout } 2024-08-19T01:25:33.489 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:32.532+0000 7f382f600640 1 Processor -- start 2024-08-19T01:25:33.489 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:32.532+0000 7f382f600640 1 -- start start 2024-08-19T01:25:33.489 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:32.532+0000 7f382f600640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f382810aac0 0x7f382810aec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:25:33.489 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:32.532+0000 7f382f600640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f382810b490 con 0x7f382810aac0 2024-08-19T01:25:33.490 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:32.533+0000 7f382cc00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f382810aac0 0x7f382810aec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:25:33.490 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:32.533+0000 7f382cc00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f382810aac0 0x7f382810aec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:57112/0 (socket says 172.21.15.83:57112) 2024-08-19T01:25:33.490 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:32.533+0000 7f382cc00640 1 -- 172.21.15.83:0/3588987518 learned_addr learned my addr 172.21.15.83:0/3588987518 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-08-19T01:25:33.490 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:32.533+0000 7f382cc00640 1 -- 172.21.15.83:0/3588987518 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f382810b610 con 0x7f382810aac0 2024-08-19T01:25:33.490 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:32.534+0000 7f382cc00640 1 --2- 172.21.15.83:0/3588987518 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f382810aac0 0x7f382810aec0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f381c009820 tx=0x7f381c02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=10366e728a40d6e1 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:25:33.490 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:32.534+0000 7f3827400640 1 -- 172.21.15.83:0/3588987518 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f381c037070 con 0x7f382810aac0 2024-08-19T01:25:33.490 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:32.534+0000 7f3827400640 1 -- 172.21.15.83:0/3588987518 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f381c032900 con 0x7f382810aac0 2024-08-19T01:25:33.490 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:32.535+0000 7f3827400640 1 -- 172.21.15.83:0/3588987518 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f381c030850 con 0x7f382810aac0 2024-08-19T01:25:33.490 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:32.535+0000 7f382f600640 1 -- 172.21.15.83:0/3588987518 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f382810aac0 msgr2=0x7f382810aec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:25:33.490 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:32.535+0000 7f382f600640 1 --2- 172.21.15.83:0/3588987518 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f382810aac0 0x7f382810aec0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f381c009820 tx=0x7f381c02a270 comp rx=0 tx=0).stop 2024-08-19T01:25:33.490 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:32.535+0000 7f382f600640 1 -- 172.21.15.83:0/3588987518 shutdown_connections 2024-08-19T01:25:33.490 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:32.535+0000 7f382f600640 1 --2- 172.21.15.83:0/3588987518 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f382810aac0 0x7f382810aec0 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:25:33.491 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:32.535+0000 7f382f600640 1 -- 172.21.15.83:0/3588987518 >> 172.21.15.83:0/3588987518 conn(0x7f382807cc40 msgr2=0x7f382807d050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:25:33.491 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:32.535+0000 7f382f600640 1 -- 172.21.15.83:0/3588987518 shutdown_connections 2024-08-19T01:25:33.491 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:32.536+0000 7f382f600640 1 -- 172.21.15.83:0/3588987518 wait complete. 2024-08-19T01:25:33.491 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:32.536+0000 7f382f600640 1 Processor -- start 2024-08-19T01:25:33.491 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:32.537+0000 7f382f600640 1 -- start start 2024-08-19T01:25:33.491 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:32.537+0000 7f382f600640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f382810aac0 0x7f382819d690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:25:33.491 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:32.537+0000 7f382f600640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f382810db20 con 0x7f382810aac0 2024-08-19T01:25:33.491 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:32.537+0000 7f382cc00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f382810aac0 0x7f382819d690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:25:33.491 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:32.537+0000 7f382cc00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f382810aac0 0x7f382819d690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:57120/0 (socket says 172.21.15.83:57120) 2024-08-19T01:25:33.491 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:32.537+0000 7f382cc00640 1 -- 172.21.15.83:0/1198860204 learned_addr learned my addr 172.21.15.83:0/1198860204 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-08-19T01:25:33.491 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:32.538+0000 7f382cc00640 1 -- 172.21.15.83:0/1198860204 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f382819dbd0 con 0x7f382810aac0 2024-08-19T01:25:33.491 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:32.538+0000 7f382cc00640 1 --2- 172.21.15.83:0/1198860204 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f382810aac0 0x7f382819d690 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f381c0097f0 tx=0x7f381c030ec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:25:33.491 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:32.538+0000 7f3825600640 1 -- 172.21.15.83:0/1198860204 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f381c042070 con 0x7f382810aac0 2024-08-19T01:25:33.491 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:32.538+0000 7f3825600640 1 -- 172.21.15.83:0/1198860204 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f381c032ca0 con 0x7f382810aac0 2024-08-19T01:25:33.492 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:32.538+0000 7f3825600640 1 -- 172.21.15.83:0/1198860204 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f381c037040 con 0x7f382810aac0 2024-08-19T01:25:33.492 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:32.538+0000 7f382f600640 1 -- 172.21.15.83:0/1198860204 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f382819de60 con 0x7f382810aac0 2024-08-19T01:25:33.492 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:32.538+0000 7f382f600640 1 -- 172.21.15.83:0/1198860204 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f38281a0b50 con 0x7f382810aac0 2024-08-19T01:25:33.492 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:32.539+0000 7f3825600640 1 -- 172.21.15.83:0/1198860204 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 46287+0+0 (secure 0 0 0) 0x7f381c045070 con 0x7f382810aac0 2024-08-19T01:25:33.492 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:32.539+0000 7f382f600640 1 -- 172.21.15.83:0/1198860204 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f382810dca0 con 0x7f382810aac0 2024-08-19T01:25:33.492 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:32.540+0000 7f3825600640 1 --2- 172.21.15.83:0/1198860204 >> [v2:172.21.15.83:6800/3261152818,v1:172.21.15.83:6801/3261152818] conn(0x7f3810038fc0 0x7f381003b480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:25:33.492 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:32.540+0000 7f3825600640 1 -- 172.21.15.83:0/1198860204 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f381c06be70 con 0x7f382810aac0 2024-08-19T01:25:33.492 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:32.540+0000 7f3827e00640 1 -- 172.21.15.83:0/1198860204 >> [v2:172.21.15.83:6800/3261152818,v1:172.21.15.83:6801/3261152818] conn(0x7f3810038fc0 msgr2=0x7f381003b480 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.83:6800/3261152818 2024-08-19T01:25:33.492 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:32.540+0000 7f3827e00640 1 --2- 172.21.15.83:0/1198860204 >> [v2:172.21.15.83:6800/3261152818,v1:172.21.15.83:6801/3261152818] conn(0x7f3810038fc0 0x7f381003b480 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-08-19T01:25:33.492 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:32.545+0000 7f3825600640 1 -- 172.21.15.83:0/1198860204 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7f381c04b170 con 0x7f382810aac0 2024-08-19T01:25:33.492 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:32.741+0000 7f3827e00640 1 -- 172.21.15.83:0/1198860204 >> [v2:172.21.15.83:6800/3261152818,v1:172.21.15.83:6801/3261152818] conn(0x7f3810038fc0 msgr2=0x7f381003b480 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.83:6800/3261152818 2024-08-19T01:25:33.492 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:32.741+0000 7f3827e00640 1 --2- 172.21.15.83:0/1198860204 >> [v2:172.21.15.83:6800/3261152818,v1:172.21.15.83:6801/3261152818] conn(0x7f3810038fc0 0x7f381003b480 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-08-19T01:25:33.492 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:32.771+0000 7f382f600640 1 -- 172.21.15.83:0/1198860204 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) -- 0x7f38281a11c0 con 0x7f382810aac0 2024-08-19T01:25:33.492 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:32.771+0000 7f3825600640 1 -- 172.21.15.83:0/1198860204 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v5) ==== 56+0+88 (secure 0 0 0) 0x7f381c06b890 con 0x7f382810aac0 2024-08-19T01:25:33.493 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:32.773+0000 7f382f600640 1 -- 172.21.15.83:0/1198860204 >> [v2:172.21.15.83:6800/3261152818,v1:172.21.15.83:6801/3261152818] conn(0x7f3810038fc0 msgr2=0x7f381003b480 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-08-19T01:25:33.493 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:32.773+0000 7f382f600640 1 --2- 172.21.15.83:0/1198860204 >> [v2:172.21.15.83:6800/3261152818,v1:172.21.15.83:6801/3261152818] conn(0x7f3810038fc0 0x7f381003b480 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:25:33.493 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:32.773+0000 7f382f600640 1 -- 172.21.15.83:0/1198860204 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f382810aac0 msgr2=0x7f382819d690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:25:33.493 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:32.773+0000 7f382f600640 1 --2- 172.21.15.83:0/1198860204 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f382810aac0 0x7f382819d690 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f381c0097f0 tx=0x7f381c030ec0 comp rx=0 tx=0).stop 2024-08-19T01:25:33.493 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:32.773+0000 7f382f600640 1 -- 172.21.15.83:0/1198860204 shutdown_connections 2024-08-19T01:25:33.493 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:32.773+0000 7f382f600640 1 --2- 172.21.15.83:0/1198860204 >> [v2:172.21.15.83:6800/3261152818,v1:172.21.15.83:6801/3261152818] conn(0x7f3810038fc0 0x7f381003b480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:25:33.493 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:32.773+0000 7f382f600640 1 --2- 172.21.15.83:0/1198860204 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f382810aac0 0x7f382819d690 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:25:33.493 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:32.773+0000 7f382f600640 1 -- 172.21.15.83:0/1198860204 >> 172.21.15.83:0/1198860204 conn(0x7f382807cc40 msgr2=0x7f3828107550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:25:33.493 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:32.774+0000 7f382f600640 1 -- 172.21.15.83:0/1198860204 shutdown_connections 2024-08-19T01:25:33.493 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:32.774+0000 7f382f600640 1 -- 172.21.15.83:0/1198860204 wait complete. 2024-08-19T01:25:33.493 INFO:teuthology.orchestra.run.smithi083.stdout:Waiting for the mgr to restart... 2024-08-19T01:25:33.493 INFO:teuthology.orchestra.run.smithi083.stdout:Waiting for mgr epoch 5... 2024-08-19T01:25:33.642 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:25:33 smithi083 ceph-mon[17120]: from='client.? 172.21.15.83:0/1198860204' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-08-19T01:25:33.643 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Aug 19 01:25:33 smithi083 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-mgr-x[17344]: 2024-08-19T01:25:33.338+0000 7ffae9723040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-08-19T01:25:33.643 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Aug 19 01:25:33 smithi083 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-mgr-x[17344]: 2024-08-19T01:25:33.411+0000 7ffae9723040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-08-19T01:25:33.643 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Aug 19 01:25:33 smithi083 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-mgr-x[17344]: 2024-08-19T01:25:33.493+0000 7ffae9723040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-08-19T01:25:33.905 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Aug 19 01:25:33 smithi083 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-mgr-x[17344]: 2024-08-19T01:25:33.721+0000 7ffae9723040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-08-19T01:25:33.905 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Aug 19 01:25:33 smithi083 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-mgr-x[17344]: 2024-08-19T01:25:33.800+0000 7ffae9723040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-08-19T01:25:33.905 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Aug 19 01:25:33 smithi083 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-mgr-x[17344]: 2024-08-19T01:25:33.877+0000 7ffae9723040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-08-19T01:25:34.211 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Aug 19 01:25:33 smithi083 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-mgr-x[17344]: 2024-08-19T01:25:33.959+0000 7ffae9723040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-08-19T01:25:34.211 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Aug 19 01:25:34 smithi083 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-mgr-x[17344]: 2024-08-19T01:25:34.209+0000 7ffae9723040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-08-19T01:25:34.655 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Aug 19 01:25:34 smithi083 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-mgr-x[17344]: 2024-08-19T01:25:34.281+0000 7ffae9723040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-08-19T01:25:35.272 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Aug 19 01:25:34 smithi083 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-mgr-x[17344]: 2024-08-19T01:25:34.917+0000 7ffae9723040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-08-19T01:25:35.273 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Aug 19 01:25:35 smithi083 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-mgr-x[17344]: 2024-08-19T01:25:35.271+0000 7ffae9723040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-08-19T01:25:36.155 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Aug 19 01:25:35 smithi083 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-mgr-x[17344]: 2024-08-19T01:25:35.786+0000 7ffae9723040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-08-19T01:25:36.155 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Aug 19 01:25:35 smithi083 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-mgr-x[17344]: 2024-08-19T01:25:35.851+0000 7ffae9723040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-08-19T01:25:36.155 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Aug 19 01:25:35 smithi083 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-mgr-x[17344]: 2024-08-19T01:25:35.921+0000 7ffae9723040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-08-19T01:25:37.405 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:25:37 smithi083 ceph-mon[17120]: Active manager daemon x restarted 2024-08-19T01:25:37.405 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:25:37 smithi083 ceph-mon[17120]: Activating manager daemon x 2024-08-19T01:25:37.405 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:25:37 smithi083 ceph-mon[17120]: osdmap e2: 0 total, 0 up, 0 in 2024-08-19T01:25:37.405 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:25:37 smithi083 ceph-mon[17120]: mgrmap e6: x(active, starting, since 0.00405774s) 2024-08-19T01:25:37.406 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:25:37 smithi083 ceph-mon[17120]: from='mgr.14118 172.21.15.83:0/3307054916' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-08-19T01:25:37.406 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:25:37 smithi083 ceph-mon[17120]: from='mgr.14118 172.21.15.83:0/3307054916' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-08-19T01:25:37.406 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:25:37 smithi083 ceph-mon[17120]: from='mgr.14118 172.21.15.83:0/3307054916' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-08-19T01:25:37.406 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:25:37 smithi083 ceph-mon[17120]: from='mgr.14118 172.21.15.83:0/3307054916' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-08-19T01:25:37.406 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:25:37 smithi083 ceph-mon[17120]: from='mgr.14118 172.21.15.83:0/3307054916' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-08-19T01:25:37.406 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:25:37 smithi083 ceph-mon[17120]: Manager daemon x is now available 2024-08-19T01:25:37.406 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:25:37 smithi083 ceph-mon[17120]: from='mgr.14118 172.21.15.83:0/3307054916' entity='mgr.x' 2024-08-19T01:25:37.406 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:25:37 smithi083 ceph-mon[17120]: from='mgr.14118 172.21.15.83:0/3307054916' entity='mgr.x' 2024-08-19T01:25:37.406 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:25:37 smithi083 ceph-mon[17120]: from='mgr.14118 172.21.15.83:0/3307054916' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-19T01:25:37.406 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:25:37 smithi083 ceph-mon[17120]: from='mgr.14118 172.21.15.83:0/3307054916' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-19T01:25:38.492 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:25:38 smithi083 ceph-mon[17120]: Found migration_current of "None". Setting to last migration. 2024-08-19T01:25:38.492 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:25:38 smithi083 ceph-mon[17120]: from='mgr.14118 172.21.15.83:0/3307054916' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-08-19T01:25:38.492 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:25:38 smithi083 ceph-mon[17120]: from='mgr.14118 172.21.15.83:0/3307054916' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-08-19T01:25:38.493 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:25:38 smithi083 ceph-mon[17120]: from='mgr.14118 172.21.15.83:0/3307054916' entity='mgr.x' 2024-08-19T01:25:38.493 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:25:38 smithi083 ceph-mon[17120]: from='mgr.14118 172.21.15.83:0/3307054916' entity='mgr.x' 2024-08-19T01:25:38.493 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:25:38 smithi083 ceph-mon[17120]: mgrmap e7: x(active, since 1.00742s) 2024-08-19T01:25:38.820 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Aug 19 01:25:38 smithi083 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-mgr-x[17344]: 2024-08-19T01:25:38.491+0000 7ffa49200640 -1 log_channel(cephadm) log [ERR] : [19/Aug/2024:01:25:38] ENGINE Error in HTTPServer.serve 2024-08-19T01:25:38.820 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Aug 19 01:25:38 smithi083 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-mgr-x[17344]: Traceback (most recent call last): 2024-08-19T01:25:38.820 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Aug 19 01:25:38 smithi083 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-mgr-x[17344]: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-08-19T01:25:38.820 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Aug 19 01:25:38 smithi083 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-mgr-x[17344]: self._connections.run(self.expiration_interval) 2024-08-19T01:25:38.820 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Aug 19 01:25:38 smithi083 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-mgr-x[17344]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-08-19T01:25:38.820 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Aug 19 01:25:38 smithi083 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-mgr-x[17344]: self._run(expiration_interval) 2024-08-19T01:25:38.821 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Aug 19 01:25:38 smithi083 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-mgr-x[17344]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-08-19T01:25:38.821 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Aug 19 01:25:38 smithi083 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-mgr-x[17344]: new_conn = self._from_server_socket(self.server.socket) 2024-08-19T01:25:38.821 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Aug 19 01:25:38 smithi083 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-mgr-x[17344]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-08-19T01:25:38.821 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Aug 19 01:25:38 smithi083 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-mgr-x[17344]: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-08-19T01:25:38.821 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Aug 19 01:25:38 smithi083 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-mgr-x[17344]: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-08-19T01:25:38.821 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Aug 19 01:25:38 smithi083 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-mgr-x[17344]: s = self.context.wrap_socket( 2024-08-19T01:25:38.821 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Aug 19 01:25:38 smithi083 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-mgr-x[17344]: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-08-19T01:25:38.821 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Aug 19 01:25:38 smithi083 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-mgr-x[17344]: return self.sslsocket_class._create( 2024-08-19T01:25:38.821 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Aug 19 01:25:38 smithi083 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-mgr-x[17344]: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-08-19T01:25:38.821 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Aug 19 01:25:38 smithi083 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-mgr-x[17344]: self.do_handshake() 2024-08-19T01:25:38.821 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Aug 19 01:25:38 smithi083 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-mgr-x[17344]: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-08-19T01:25:38.821 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Aug 19 01:25:38 smithi083 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-mgr-x[17344]: self._sslobj.do_handshake() 2024-08-19T01:25:38.822 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Aug 19 01:25:38 smithi083 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-mgr-x[17344]: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-08-19T01:25:38.822 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Aug 19 01:25:38 smithi083 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-mgr-x[17344]: 2024-08-19T01:25:39.050 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout { 2024-08-19T01:25:39.051 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 7, 2024-08-19T01:25:39.051 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "initialized": true 2024-08-19T01:25:39.051 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout } 2024-08-19T01:25:39.051 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:34.049+0000 7fa459c00640 1 Processor -- start 2024-08-19T01:25:39.052 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:34.049+0000 7fa459c00640 1 -- start start 2024-08-19T01:25:39.052 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:34.049+0000 7fa459c00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa454109840 0x7fa454109c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:25:39.052 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:34.049+0000 7fa459c00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fa45410a210 con 0x7fa454109840 2024-08-19T01:25:39.052 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:34.049+0000 7fa452a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa454109840 0x7fa454109c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:25:39.052 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:34.049+0000 7fa452a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa454109840 0x7fa454109c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:57122/0 (socket says 172.21.15.83:57122) 2024-08-19T01:25:39.052 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:34.049+0000 7fa452a00640 1 -- 172.21.15.83:0/3294030850 learned_addr learned my addr 172.21.15.83:0/3294030850 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-08-19T01:25:39.052 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:34.050+0000 7fa452a00640 1 -- 172.21.15.83:0/3294030850 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa45410a390 con 0x7fa454109840 2024-08-19T01:25:39.052 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:34.051+0000 7fa452a00640 1 --2- 172.21.15.83:0/3294030850 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa454109840 0x7fa454109c40 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fa444009820 tx=0x7fa44402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=53f073a54eb07b0a server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:25:39.052 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:34.051+0000 7fa451600640 1 -- 172.21.15.83:0/3294030850 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa444037070 con 0x7fa454109840 2024-08-19T01:25:39.053 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:34.051+0000 7fa451600640 1 -- 172.21.15.83:0/3294030850 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fa444032900 con 0x7fa454109840 2024-08-19T01:25:39.053 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:34.051+0000 7fa451600640 1 -- 172.21.15.83:0/3294030850 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa444030850 con 0x7fa454109840 2024-08-19T01:25:39.053 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:34.052+0000 7fa459c00640 1 -- 172.21.15.83:0/3294030850 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa454109840 msgr2=0x7fa454109c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:25:39.053 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:34.052+0000 7fa459c00640 1 --2- 172.21.15.83:0/3294030850 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa454109840 0x7fa454109c40 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fa444009820 tx=0x7fa44402a270 comp rx=0 tx=0).stop 2024-08-19T01:25:39.053 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:34.052+0000 7fa459c00640 1 -- 172.21.15.83:0/3294030850 shutdown_connections 2024-08-19T01:25:39.053 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:34.052+0000 7fa459c00640 1 --2- 172.21.15.83:0/3294030850 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa454109840 0x7fa454109c40 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:25:39.053 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:34.052+0000 7fa459c00640 1 -- 172.21.15.83:0/3294030850 >> 172.21.15.83:0/3294030850 conn(0x7fa45407cc50 msgr2=0x7fa45407d060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:25:39.053 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:34.052+0000 7fa459c00640 1 -- 172.21.15.83:0/3294030850 shutdown_connections 2024-08-19T01:25:39.053 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:34.052+0000 7fa459c00640 1 -- 172.21.15.83:0/3294030850 wait complete. 2024-08-19T01:25:39.053 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:34.053+0000 7fa459c00640 1 Processor -- start 2024-08-19T01:25:39.054 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:34.053+0000 7fa459c00640 1 -- start start 2024-08-19T01:25:39.054 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:34.053+0000 7fa459c00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa454109840 0x7fa45419d5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:25:39.054 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:34.053+0000 7fa459c00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fa45410dac0 con 0x7fa454109840 2024-08-19T01:25:39.054 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:34.054+0000 7fa452a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa454109840 0x7fa45419d5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:25:39.054 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:34.054+0000 7fa452a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa454109840 0x7fa45419d5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:57136/0 (socket says 172.21.15.83:57136) 2024-08-19T01:25:39.054 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:34.054+0000 7fa452a00640 1 -- 172.21.15.83:0/3532490381 learned_addr learned my addr 172.21.15.83:0/3532490381 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-08-19T01:25:39.054 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:34.054+0000 7fa452a00640 1 -- 172.21.15.83:0/3532490381 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa45419db30 con 0x7fa454109840 2024-08-19T01:25:39.054 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:34.054+0000 7fa452a00640 1 --2- 172.21.15.83:0/3532490381 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa454109840 0x7fa45419d5f0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fa4440097f0 tx=0x7fa444030d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:25:39.054 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:34.055+0000 7fa44b400640 1 -- 172.21.15.83:0/3532490381 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa444037040 con 0x7fa454109840 2024-08-19T01:25:39.055 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:34.055+0000 7fa44b400640 1 -- 172.21.15.83:0/3532490381 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fa444040070 con 0x7fa454109840 2024-08-19T01:25:39.055 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:34.055+0000 7fa44b400640 1 -- 172.21.15.83:0/3532490381 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa44403a390 con 0x7fa454109840 2024-08-19T01:25:39.055 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:34.055+0000 7fa459c00640 1 -- 172.21.15.83:0/3532490381 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa45419ddc0 con 0x7fa454109840 2024-08-19T01:25:39.055 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:34.055+0000 7fa459c00640 1 -- 172.21.15.83:0/3532490381 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa4541a0ab0 con 0x7fa454109840 2024-08-19T01:25:39.055 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:34.056+0000 7fa44b400640 1 -- 172.21.15.83:0/3532490381 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 46287+0+0 (secure 0 0 0) 0x7fa44403b430 con 0x7fa454109840 2024-08-19T01:25:39.055 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:34.056+0000 7fa44b400640 1 --2- 172.21.15.83:0/3532490381 >> [v2:172.21.15.83:6800/3261152818,v1:172.21.15.83:6801/3261152818] conn(0x7fa438038fc0 0x7fa43803b480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:25:39.055 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:34.056+0000 7fa44b400640 1 -- 172.21.15.83:0/3532490381 --> [v2:172.21.15.83:6800/3261152818,v1:172.21.15.83:6801/3261152818] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7fa43803bb90 con 0x7fa438038fc0 2024-08-19T01:25:39.056 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:34.056+0000 7fa44b400640 1 -- 172.21.15.83:0/3532490381 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fa44406c0f0 con 0x7fa454109840 2024-08-19T01:25:39.056 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:34.056+0000 7fa452000640 1 -- 172.21.15.83:0/3532490381 >> [v2:172.21.15.83:6800/3261152818,v1:172.21.15.83:6801/3261152818] conn(0x7fa438038fc0 msgr2=0x7fa43803b480 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.83:6800/3261152818 2024-08-19T01:25:39.056 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:34.056+0000 7fa452000640 1 --2- 172.21.15.83:0/3532490381 >> [v2:172.21.15.83:6800/3261152818,v1:172.21.15.83:6801/3261152818] conn(0x7fa438038fc0 0x7fa43803b480 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-08-19T01:25:39.056 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:34.257+0000 7fa452000640 1 -- 172.21.15.83:0/3532490381 >> [v2:172.21.15.83:6800/3261152818,v1:172.21.15.83:6801/3261152818] conn(0x7fa438038fc0 msgr2=0x7fa43803b480 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.83:6800/3261152818 2024-08-19T01:25:39.056 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:34.257+0000 7fa452000640 1 --2- 172.21.15.83:0/3532490381 >> [v2:172.21.15.83:6800/3261152818,v1:172.21.15.83:6801/3261152818] conn(0x7fa438038fc0 0x7fa43803b480 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-08-19T01:25:39.056 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:34.658+0000 7fa452000640 1 -- 172.21.15.83:0/3532490381 >> [v2:172.21.15.83:6800/3261152818,v1:172.21.15.83:6801/3261152818] conn(0x7fa438038fc0 msgr2=0x7fa43803b480 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.83:6800/3261152818 2024-08-19T01:25:39.056 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:34.658+0000 7fa452000640 1 --2- 172.21.15.83:0/3532490381 >> [v2:172.21.15.83:6800/3261152818,v1:172.21.15.83:6801/3261152818] conn(0x7fa438038fc0 0x7fa43803b480 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-08-19T01:25:39.056 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:35.459+0000 7fa452000640 1 -- 172.21.15.83:0/3532490381 >> [v2:172.21.15.83:6800/3261152818,v1:172.21.15.83:6801/3261152818] conn(0x7fa438038fc0 msgr2=0x7fa43803b480 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.83:6800/3261152818 2024-08-19T01:25:39.056 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:35.459+0000 7fa452000640 1 --2- 172.21.15.83:0/3532490381 >> [v2:172.21.15.83:6800/3261152818,v1:172.21.15.83:6801/3261152818] conn(0x7fa438038fc0 0x7fa43803b480 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-08-19T01:25:39.056 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:37.060+0000 7fa452000640 1 -- 172.21.15.83:0/3532490381 >> [v2:172.21.15.83:6800/3261152818,v1:172.21.15.83:6801/3261152818] conn(0x7fa438038fc0 msgr2=0x7fa43803b480 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.83:6800/3261152818 2024-08-19T01:25:39.056 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:37.060+0000 7fa452000640 1 --2- 172.21.15.83:0/3532490381 >> [v2:172.21.15.83:6800/3261152818,v1:172.21.15.83:6801/3261152818] conn(0x7fa438038fc0 0x7fa43803b480 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2024-08-19T01:25:39.057 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:37.231+0000 7fa44b400640 1 -- 172.21.15.83:0/3532490381 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mgrmap(e 6) ==== 46054+0+0 (secure 0 0 0) 0x7fa44404acf0 con 0x7fa454109840 2024-08-19T01:25:39.057 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:37.231+0000 7fa44b400640 1 -- 172.21.15.83:0/3532490381 >> [v2:172.21.15.83:6800/3261152818,v1:172.21.15.83:6801/3261152818] conn(0x7fa438038fc0 msgr2=0x7fa43803b480 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-08-19T01:25:39.057 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:37.231+0000 7fa44b400640 1 --2- 172.21.15.83:0/3532490381 >> [v2:172.21.15.83:6800/3261152818,v1:172.21.15.83:6801/3261152818] conn(0x7fa438038fc0 0x7fa43803b480 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:25:39.057 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:38.233+0000 7fa44b400640 1 -- 172.21.15.83:0/3532490381 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mgrmap(e 7) ==== 46181+0+0 (secure 0 0 0) 0x7fa444044720 con 0x7fa454109840 2024-08-19T01:25:39.057 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:38.233+0000 7fa44b400640 1 --2- 172.21.15.83:0/3532490381 >> [v2:172.21.15.83:6800/573907319,v1:172.21.15.83:6801/573907319] conn(0x7fa43803c8f0 0x7fa43803ece0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:25:39.057 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:38.233+0000 7fa44b400640 1 -- 172.21.15.83:0/3532490381 --> [v2:172.21.15.83:6800/573907319,v1:172.21.15.83:6801/573907319] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7fa44405ac70 con 0x7fa43803c8f0 2024-08-19T01:25:39.057 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:38.234+0000 7fa452000640 1 --2- 172.21.15.83:0/3532490381 >> [v2:172.21.15.83:6800/573907319,v1:172.21.15.83:6801/573907319] conn(0x7fa43803c8f0 0x7fa43803ece0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:25:39.057 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:38.235+0000 7fa452000640 1 --2- 172.21.15.83:0/3532490381 >> [v2:172.21.15.83:6800/573907319,v1:172.21.15.83:6801/573907319] conn(0x7fa43803c8f0 0x7fa43803ece0 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7fa440003980 tx=0x7fa4400092b0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:25:39.057 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:38.236+0000 7fa44b400640 1 -- 172.21.15.83:0/3532490381 <== mgr.14118 v2:172.21.15.83:6800/573907319 1 ==== command_reply(tid 0: 0 ) ==== 8+0+9564 (secure 0 0 0) 0x7fa44405ac70 con 0x7fa43803c8f0 2024-08-19T01:25:39.057 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:38.243+0000 7fa459c00640 1 -- 172.21.15.83:0/3532490381 --> [v2:172.21.15.83:6800/573907319,v1:172.21.15.83:6801/573907319] -- command(tid 1: {"prefix": "mgr_status"}) -- 0x7fa454109cc0 con 0x7fa43803c8f0 2024-08-19T01:25:39.057 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:38.244+0000 7fa44b400640 1 -- 172.21.15.83:0/3532490381 <== mgr.14118 v2:172.21.15.83:6800/573907319 2 ==== command_reply(tid 1: 0 ) ==== 8+0+51 (secure 0 0 0) 0x7fa454109cc0 con 0x7fa43803c8f0 2024-08-19T01:25:39.057 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:38.244+0000 7fa459c00640 1 -- 172.21.15.83:0/3532490381 >> [v2:172.21.15.83:6800/573907319,v1:172.21.15.83:6801/573907319] conn(0x7fa43803c8f0 msgr2=0x7fa43803ece0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:25:39.057 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:38.244+0000 7fa459c00640 1 --2- 172.21.15.83:0/3532490381 >> [v2:172.21.15.83:6800/573907319,v1:172.21.15.83:6801/573907319] conn(0x7fa43803c8f0 0x7fa43803ece0 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7fa440003980 tx=0x7fa4400092b0 comp rx=0 tx=0).stop 2024-08-19T01:25:39.057 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:38.244+0000 7fa459c00640 1 -- 172.21.15.83:0/3532490381 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa454109840 msgr2=0x7fa45419d5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:25:39.058 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:38.244+0000 7fa459c00640 1 --2- 172.21.15.83:0/3532490381 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa454109840 0x7fa45419d5f0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fa4440097f0 tx=0x7fa444030d80 comp rx=0 tx=0).stop 2024-08-19T01:25:39.058 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:38.244+0000 7fa459c00640 1 -- 172.21.15.83:0/3532490381 shutdown_connections 2024-08-19T01:25:39.058 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:38.244+0000 7fa459c00640 1 --2- 172.21.15.83:0/3532490381 >> [v2:172.21.15.83:6800/573907319,v1:172.21.15.83:6801/573907319] conn(0x7fa43803c8f0 0x7fa43803ece0 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:25:39.058 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:38.244+0000 7fa459c00640 1 --2- 172.21.15.83:0/3532490381 >> [v2:172.21.15.83:6800/3261152818,v1:172.21.15.83:6801/3261152818] conn(0x7fa438038fc0 0x7fa43803b480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:25:39.058 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:38.244+0000 7fa459c00640 1 --2- 172.21.15.83:0/3532490381 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa454109840 0x7fa45419d5f0 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:25:39.058 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:38.244+0000 7fa459c00640 1 -- 172.21.15.83:0/3532490381 >> 172.21.15.83:0/3532490381 conn(0x7fa45407cc50 msgr2=0x7fa454107500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:25:39.058 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:38.244+0000 7fa459c00640 1 -- 172.21.15.83:0/3532490381 shutdown_connections 2024-08-19T01:25:39.058 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:38.245+0000 7fa459c00640 1 -- 172.21.15.83:0/3532490381 wait complete. 2024-08-19T01:25:39.058 INFO:teuthology.orchestra.run.smithi083.stdout:mgr epoch 5 is available 2024-08-19T01:25:39.058 INFO:teuthology.orchestra.run.smithi083.stdout:Setting orchestrator backend to cephadm... 2024-08-19T01:25:39.473 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:25:39 smithi083 ceph-mon[17120]: from='client.14122 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-08-19T01:25:39.473 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:25:39 smithi083 ceph-mon[17120]: from='client.14122 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-08-19T01:25:39.473 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:25:39 smithi083 ceph-mon[17120]: [19/Aug/2024:01:25:38] ENGINE Bus STARTING 2024-08-19T01:25:39.473 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:25:39 smithi083 ceph-mon[17120]: from='mgr.14118 172.21.15.83:0/3307054916' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-19T01:25:40.405 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:25:40 smithi083 ceph-mon[17120]: [19/Aug/2024:01:25:38] ENGINE Serving on https://172.21.15.83:7150 2024-08-19T01:25:40.405 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:25:40 smithi083 ceph-mon[17120]: [19/Aug/2024:01:25:38] ENGINE Error in HTTPServer.serve 2024-08-19T01:25:40.405 INFO:journalctl@ceph.mon.a.smithi083.stdout: Traceback (most recent call last): 2024-08-19T01:25:40.405 INFO:journalctl@ceph.mon.a.smithi083.stdout: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-08-19T01:25:40.406 INFO:journalctl@ceph.mon.a.smithi083.stdout: self._connections.run(self.expiration_interval) 2024-08-19T01:25:40.406 INFO:journalctl@ceph.mon.a.smithi083.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-08-19T01:25:40.406 INFO:journalctl@ceph.mon.a.smithi083.stdout: self._run(expiration_interval) 2024-08-19T01:25:40.406 INFO:journalctl@ceph.mon.a.smithi083.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-08-19T01:25:40.406 INFO:journalctl@ceph.mon.a.smithi083.stdout: new_conn = self._from_server_socket(self.server.socket) 2024-08-19T01:25:40.406 INFO:journalctl@ceph.mon.a.smithi083.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-08-19T01:25:40.406 INFO:journalctl@ceph.mon.a.smithi083.stdout: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-08-19T01:25:40.406 INFO:journalctl@ceph.mon.a.smithi083.stdout: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-08-19T01:25:40.406 INFO:journalctl@ceph.mon.a.smithi083.stdout: s = self.context.wrap_socket( 2024-08-19T01:25:40.406 INFO:journalctl@ceph.mon.a.smithi083.stdout: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-08-19T01:25:40.406 INFO:journalctl@ceph.mon.a.smithi083.stdout: return self.sslsocket_class._create( 2024-08-19T01:25:40.406 INFO:journalctl@ceph.mon.a.smithi083.stdout: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-08-19T01:25:40.407 INFO:journalctl@ceph.mon.a.smithi083.stdout: self.do_handshake() 2024-08-19T01:25:40.407 INFO:journalctl@ceph.mon.a.smithi083.stdout: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-08-19T01:25:40.407 INFO:journalctl@ceph.mon.a.smithi083.stdout: self._sslobj.do_handshake() 2024-08-19T01:25:40.407 INFO:journalctl@ceph.mon.a.smithi083.stdout: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-08-19T01:25:40.407 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:25:40 smithi083 ceph-mon[17120]: [19/Aug/2024:01:25:38] ENGINE Serving on http://172.21.15.83:8765 2024-08-19T01:25:40.407 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:25:40 smithi083 ceph-mon[17120]: [19/Aug/2024:01:25:38] ENGINE Bus STARTED 2024-08-19T01:25:40.407 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:25:40 smithi083 ceph-mon[17120]: mgrmap e8: x(active, since 2s) 2024-08-19T01:25:40.407 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:25:40 smithi083 ceph-mon[17120]: from='mgr.14118 172.21.15.83:0/3307054916' entity='mgr.x' 2024-08-19T01:25:40.407 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:25:40 smithi083 ceph-mon[17120]: from='mgr.14118 172.21.15.83:0/3307054916' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-19T01:25:40.414 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:39.577+0000 7f72a9400640 1 Processor -- start 2024-08-19T01:25:40.421 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:39.577+0000 7f72a9400640 1 -- start start 2024-08-19T01:25:40.421 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:39.578+0000 7f72a9400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f72a410aac0 0x7f72a410aec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:25:40.421 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:39.578+0000 7f72a9400640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f72a410b490 con 0x7f72a410aac0 2024-08-19T01:25:40.421 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:39.578+0000 7f72a2800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f72a410aac0 0x7f72a410aec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:25:40.421 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:39.578+0000 7f72a2800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f72a410aac0 0x7f72a410aec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:57212/0 (socket says 172.21.15.83:57212) 2024-08-19T01:25:40.421 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:39.578+0000 7f72a2800640 1 -- 172.21.15.83:0/3580947932 learned_addr learned my addr 172.21.15.83:0/3580947932 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-08-19T01:25:40.421 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:39.578+0000 7f72a2800640 1 -- 172.21.15.83:0/3580947932 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f72a410b610 con 0x7f72a410aac0 2024-08-19T01:25:40.421 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:39.579+0000 7f72a2800640 1 --2- 172.21.15.83:0/3580947932 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f72a410aac0 0x7f72a410aec0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f7294009820 tx=0x7f729402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8521bdaabfd166e3 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:25:40.421 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:39.579+0000 7f72a1400640 1 -- 172.21.15.83:0/3580947932 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7294037070 con 0x7f72a410aac0 2024-08-19T01:25:40.422 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:39.580+0000 7f72a1400640 1 -- 172.21.15.83:0/3580947932 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f7294032900 con 0x7f72a410aac0 2024-08-19T01:25:40.422 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:39.580+0000 7f72a1400640 1 -- 172.21.15.83:0/3580947932 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7294030850 con 0x7f72a410aac0 2024-08-19T01:25:40.422 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:39.580+0000 7f72a9400640 1 -- 172.21.15.83:0/3580947932 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f72a410aac0 msgr2=0x7f72a410aec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:25:40.422 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:39.580+0000 7f72a9400640 1 --2- 172.21.15.83:0/3580947932 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f72a410aac0 0x7f72a410aec0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f7294009820 tx=0x7f729402a270 comp rx=0 tx=0).stop 2024-08-19T01:25:40.422 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:39.580+0000 7f72a9400640 1 -- 172.21.15.83:0/3580947932 shutdown_connections 2024-08-19T01:25:40.422 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:39.580+0000 7f72a9400640 1 --2- 172.21.15.83:0/3580947932 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f72a410aac0 0x7f72a410aec0 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:25:40.422 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:39.580+0000 7f72a9400640 1 -- 172.21.15.83:0/3580947932 >> 172.21.15.83:0/3580947932 conn(0x7f72a407cc40 msgr2=0x7f72a407d050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:25:40.422 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:39.580+0000 7f72a9400640 1 -- 172.21.15.83:0/3580947932 shutdown_connections 2024-08-19T01:25:40.422 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:39.580+0000 7f72a9400640 1 -- 172.21.15.83:0/3580947932 wait complete. 2024-08-19T01:25:40.422 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:39.581+0000 7f72a9400640 1 Processor -- start 2024-08-19T01:25:40.422 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:39.581+0000 7f72a9400640 1 -- start start 2024-08-19T01:25:40.422 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:39.581+0000 7f72a9400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f72a410aac0 0x7f72a419d6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:25:40.422 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:39.581+0000 7f72a9400640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f72a410db20 con 0x7f72a410aac0 2024-08-19T01:25:40.423 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:39.582+0000 7f72a2800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f72a410aac0 0x7f72a419d6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:25:40.423 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:39.582+0000 7f72a2800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f72a410aac0 0x7f72a419d6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:57218/0 (socket says 172.21.15.83:57218) 2024-08-19T01:25:40.423 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:39.582+0000 7f72a2800640 1 -- 172.21.15.83:0/1836601179 learned_addr learned my addr 172.21.15.83:0/1836601179 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-08-19T01:25:40.423 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:39.582+0000 7f72a2800640 1 -- 172.21.15.83:0/1836601179 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f72a419dbe0 con 0x7f72a410aac0 2024-08-19T01:25:40.423 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:39.582+0000 7f72a2800640 1 --2- 172.21.15.83:0/1836601179 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f72a410aac0 0x7f72a419d6a0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f72940097f0 tx=0x7f7294030ec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:25:40.423 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:39.582+0000 7f729b400640 1 -- 172.21.15.83:0/1836601179 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7294042070 con 0x7f72a410aac0 2024-08-19T01:25:40.423 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:39.582+0000 7f729b400640 1 -- 172.21.15.83:0/1836601179 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f7294032ca0 con 0x7f72a410aac0 2024-08-19T01:25:40.423 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:39.582+0000 7f729b400640 1 -- 172.21.15.83:0/1836601179 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7294037040 con 0x7f72a410aac0 2024-08-19T01:25:40.423 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:39.583+0000 7f72a9400640 1 -- 172.21.15.83:0/1836601179 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f72a419de70 con 0x7f72a410aac0 2024-08-19T01:25:40.423 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:39.583+0000 7f72a9400640 1 -- 172.21.15.83:0/1836601179 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f72a41a0b60 con 0x7f72a410aac0 2024-08-19T01:25:40.423 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:39.584+0000 7f729b400640 1 -- 172.21.15.83:0/1836601179 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 8) ==== 46287+0+0 (secure 0 0 0) 0x7f7294045070 con 0x7f72a410aac0 2024-08-19T01:25:40.423 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:39.584+0000 7f72a9400640 1 -- 172.21.15.83:0/1836601179 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f72a410dca0 con 0x7f72a410aac0 2024-08-19T01:25:40.423 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:39.584+0000 7f729b400640 1 --2- 172.21.15.83:0/1836601179 >> [v2:172.21.15.83:6800/573907319,v1:172.21.15.83:6801/573907319] conn(0x7f7288038fc0 0x7f728803b480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:25:40.424 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:39.584+0000 7f729b400640 1 -- 172.21.15.83:0/1836601179 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f729406bf40 con 0x7f72a410aac0 2024-08-19T01:25:40.424 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:39.584+0000 7f72a1e00640 1 --2- 172.21.15.83:0/1836601179 >> [v2:172.21.15.83:6800/573907319,v1:172.21.15.83:6801/573907319] conn(0x7f7288038fc0 0x7f728803b480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:25:40.424 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:39.585+0000 7f72a1e00640 1 --2- 172.21.15.83:0/1836601179 >> [v2:172.21.15.83:6800/573907319,v1:172.21.15.83:6801/573907319] conn(0x7f7288038fc0 0x7f728803b480 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f72900098c0 tx=0x7f7290006db0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:25:40.424 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:39.589+0000 7f729b400640 1 -- 172.21.15.83:0/1836601179 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7f729404b170 con 0x7f72a410aac0 2024-08-19T01:25:40.424 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:39.776+0000 7f72a9400640 1 -- 172.21.15.83:0/1836601179 --> [v2:172.21.15.83:6800/573907319,v1:172.21.15.83:6801/573907319] -- mgr_command(tid 0: {"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}) -- 0x7f72a4107c80 con 0x7f7288038fc0 2024-08-19T01:25:40.424 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:39.784+0000 7f729b400640 1 -- 172.21.15.83:0/1836601179 <== mgr.14118 v2:172.21.15.83:6800/573907319 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f72a4107c80 con 0x7f7288038fc0 2024-08-19T01:25:40.424 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:39.786+0000 7f72a9400640 1 -- 172.21.15.83:0/1836601179 >> [v2:172.21.15.83:6800/573907319,v1:172.21.15.83:6801/573907319] conn(0x7f7288038fc0 msgr2=0x7f728803b480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:25:40.424 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:39.786+0000 7f72a9400640 1 --2- 172.21.15.83:0/1836601179 >> [v2:172.21.15.83:6800/573907319,v1:172.21.15.83:6801/573907319] conn(0x7f7288038fc0 0x7f728803b480 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f72900098c0 tx=0x7f7290006db0 comp rx=0 tx=0).stop 2024-08-19T01:25:40.424 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:39.786+0000 7f72a9400640 1 -- 172.21.15.83:0/1836601179 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f72a410aac0 msgr2=0x7f72a419d6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:25:40.424 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:39.786+0000 7f72a9400640 1 --2- 172.21.15.83:0/1836601179 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f72a410aac0 0x7f72a419d6a0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f72940097f0 tx=0x7f7294030ec0 comp rx=0 tx=0).stop 2024-08-19T01:25:40.424 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:39.787+0000 7f72a9400640 1 -- 172.21.15.83:0/1836601179 shutdown_connections 2024-08-19T01:25:40.424 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:39.787+0000 7f72a9400640 1 --2- 172.21.15.83:0/1836601179 >> [v2:172.21.15.83:6800/573907319,v1:172.21.15.83:6801/573907319] conn(0x7f7288038fc0 0x7f728803b480 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:25:40.424 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:39.787+0000 7f72a9400640 1 --2- 172.21.15.83:0/1836601179 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f72a410aac0 0x7f72a419d6a0 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:25:40.424 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:39.787+0000 7f72a9400640 1 -- 172.21.15.83:0/1836601179 >> 172.21.15.83:0/1836601179 conn(0x7f72a407cc40 msgr2=0x7f72a4107570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:25:40.425 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:39.787+0000 7f72a9400640 1 -- 172.21.15.83:0/1836601179 shutdown_connections 2024-08-19T01:25:40.425 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:39.787+0000 7f72a9400640 1 -- 172.21.15.83:0/1836601179 wait complete. 2024-08-19T01:25:41.557 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:25:41 smithi083 ceph-mon[17120]: from='client.14130 -' entity='client.admin' cmd=[{"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}]: dispatch 2024-08-19T01:25:41.771 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout value unchanged 2024-08-19T01:25:41.772 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:40.915+0000 7efc5b200640 1 Processor -- start 2024-08-19T01:25:41.772 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:40.916+0000 7efc5b200640 1 -- start start 2024-08-19T01:25:41.772 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:40.916+0000 7efc5b200640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7efc5407de20 0x7efc5407c280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:25:41.772 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:40.916+0000 7efc5b200640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7efc5407c850 con 0x7efc5407de20 2024-08-19T01:25:41.772 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:40.916+0000 7efc53e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7efc5407de20 0x7efc5407c280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:25:41.772 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:40.916+0000 7efc53e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7efc5407de20 0x7efc5407c280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:41210/0 (socket says 172.21.15.83:41210) 2024-08-19T01:25:41.772 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:40.916+0000 7efc53e00640 1 -- 172.21.15.83:0/2693852980 learned_addr learned my addr 172.21.15.83:0/2693852980 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-08-19T01:25:41.772 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:40.917+0000 7efc53e00640 1 -- 172.21.15.83:0/2693852980 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efc5407c9d0 con 0x7efc5407de20 2024-08-19T01:25:41.772 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:40.918+0000 7efc53e00640 1 --2- 172.21.15.83:0/2693852980 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7efc5407de20 0x7efc5407c280 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7efc44009820 tx=0x7efc4402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f019818e74af271c server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:25:41.773 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:40.918+0000 7efc53400640 1 -- 172.21.15.83:0/2693852980 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7efc44037070 con 0x7efc5407de20 2024-08-19T01:25:41.773 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:40.918+0000 7efc53400640 1 -- 172.21.15.83:0/2693852980 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7efc44032900 con 0x7efc5407de20 2024-08-19T01:25:41.773 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:40.918+0000 7efc53400640 1 -- 172.21.15.83:0/2693852980 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7efc44030850 con 0x7efc5407de20 2024-08-19T01:25:41.773 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:40.919+0000 7efc5b200640 1 -- 172.21.15.83:0/2693852980 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7efc5407de20 msgr2=0x7efc5407c280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:25:41.773 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:40.919+0000 7efc5b200640 1 --2- 172.21.15.83:0/2693852980 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7efc5407de20 0x7efc5407c280 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7efc44009820 tx=0x7efc4402a270 comp rx=0 tx=0).stop 2024-08-19T01:25:41.773 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:40.919+0000 7efc5b200640 1 -- 172.21.15.83:0/2693852980 shutdown_connections 2024-08-19T01:25:41.773 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:40.919+0000 7efc5b200640 1 --2- 172.21.15.83:0/2693852980 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7efc5407de20 0x7efc5407c280 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:25:41.773 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:40.919+0000 7efc5b200640 1 -- 172.21.15.83:0/2693852980 >> 172.21.15.83:0/2693852980 conn(0x7efc54102bd0 msgr2=0x7efc54105010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:25:41.773 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:40.919+0000 7efc5b200640 1 -- 172.21.15.83:0/2693852980 shutdown_connections 2024-08-19T01:25:41.774 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:40.919+0000 7efc5b200640 1 -- 172.21.15.83:0/2693852980 wait complete. 2024-08-19T01:25:41.774 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:40.920+0000 7efc5b200640 1 Processor -- start 2024-08-19T01:25:41.774 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:40.920+0000 7efc5b200640 1 -- start start 2024-08-19T01:25:41.774 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:40.920+0000 7efc5b200640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7efc5407de20 0x7efc5419d620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:25:41.774 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:40.920+0000 7efc5b200640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7efc5410dab0 con 0x7efc5407de20 2024-08-19T01:25:41.774 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:40.920+0000 7efc53e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7efc5407de20 0x7efc5419d620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:25:41.774 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:40.921+0000 7efc53e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7efc5407de20 0x7efc5419d620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:41212/0 (socket says 172.21.15.83:41212) 2024-08-19T01:25:41.774 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:40.921+0000 7efc53e00640 1 -- 172.21.15.83:0/736366097 learned_addr learned my addr 172.21.15.83:0/736366097 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-08-19T01:25:41.774 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:40.921+0000 7efc53e00640 1 -- 172.21.15.83:0/736366097 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efc5419db60 con 0x7efc5407de20 2024-08-19T01:25:41.774 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:40.921+0000 7efc53e00640 1 --2- 172.21.15.83:0/736366097 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7efc5407de20 0x7efc5419d620 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7efc440097f0 tx=0x7efc44030d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:25:41.774 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:40.922+0000 7efc51600640 1 -- 172.21.15.83:0/736366097 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7efc44037040 con 0x7efc5407de20 2024-08-19T01:25:41.775 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:40.922+0000 7efc51600640 1 -- 172.21.15.83:0/736366097 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7efc44040070 con 0x7efc5407de20 2024-08-19T01:25:41.775 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:40.922+0000 7efc51600640 1 -- 172.21.15.83:0/736366097 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7efc4403a390 con 0x7efc5407de20 2024-08-19T01:25:41.775 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:40.922+0000 7efc5b200640 1 -- 172.21.15.83:0/736366097 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efc5419ddf0 con 0x7efc5407de20 2024-08-19T01:25:41.775 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:40.922+0000 7efc5b200640 1 -- 172.21.15.83:0/736366097 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efc541a0ae0 con 0x7efc5407de20 2024-08-19T01:25:41.776 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:40.923+0000 7efc5b200640 1 -- 172.21.15.83:0/736366097 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efc5407c300 con 0x7efc5407de20 2024-08-19T01:25:41.776 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:40.923+0000 7efc51600640 1 -- 172.21.15.83:0/736366097 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 8) ==== 46287+0+0 (secure 0 0 0) 0x7efc4403b430 con 0x7efc5407de20 2024-08-19T01:25:41.776 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:40.924+0000 7efc51600640 1 --2- 172.21.15.83:0/736366097 >> [v2:172.21.15.83:6800/573907319,v1:172.21.15.83:6801/573907319] conn(0x7efc38038fc0 0x7efc3803b480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:25:41.776 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:40.924+0000 7efc51600640 1 -- 172.21.15.83:0/736366097 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7efc4406c550 con 0x7efc5407de20 2024-08-19T01:25:41.776 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:40.924+0000 7efc4b400640 1 --2- 172.21.15.83:0/736366097 >> [v2:172.21.15.83:6800/573907319,v1:172.21.15.83:6801/573907319] conn(0x7efc38038fc0 0x7efc3803b480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:25:41.776 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:40.925+0000 7efc4b400640 1 --2- 172.21.15.83:0/736366097 >> [v2:172.21.15.83:6800/573907319,v1:172.21.15.83:6801/573907319] conn(0x7efc38038fc0 0x7efc3803b480 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7efc3c00ad30 tx=0x7efc3c0093f0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:25:41.776 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:40.929+0000 7efc51600640 1 -- 172.21.15.83:0/736366097 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7efc4406bea0 con 0x7efc5407de20 2024-08-19T01:25:41.776 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:41.106+0000 7efc5b200640 1 -- 172.21.15.83:0/736366097 --> [v2:172.21.15.83:6800/573907319,v1:172.21.15.83:6801/573907319] -- mgr_command(tid 0: {"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}) -- 0x7efc54103ea0 con 0x7efc38038fc0 2024-08-19T01:25:41.777 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:41.106+0000 7efc51600640 1 -- 172.21.15.83:0/736366097 <== mgr.14118 v2:172.21.15.83:6800/573907319 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+16 (secure 0 0 0) 0x7efc54103ea0 con 0x7efc38038fc0 2024-08-19T01:25:41.777 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:41.109+0000 7efc5b200640 1 -- 172.21.15.83:0/736366097 >> [v2:172.21.15.83:6800/573907319,v1:172.21.15.83:6801/573907319] conn(0x7efc38038fc0 msgr2=0x7efc3803b480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:25:41.777 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:41.109+0000 7efc5b200640 1 --2- 172.21.15.83:0/736366097 >> [v2:172.21.15.83:6800/573907319,v1:172.21.15.83:6801/573907319] conn(0x7efc38038fc0 0x7efc3803b480 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7efc3c00ad30 tx=0x7efc3c0093f0 comp rx=0 tx=0).stop 2024-08-19T01:25:41.777 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:41.109+0000 7efc5b200640 1 -- 172.21.15.83:0/736366097 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7efc5407de20 msgr2=0x7efc5419d620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:25:41.777 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:41.109+0000 7efc5b200640 1 --2- 172.21.15.83:0/736366097 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7efc5407de20 0x7efc5419d620 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7efc440097f0 tx=0x7efc44030d80 comp rx=0 tx=0).stop 2024-08-19T01:25:41.777 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:41.109+0000 7efc5b200640 1 -- 172.21.15.83:0/736366097 shutdown_connections 2024-08-19T01:25:41.777 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:41.109+0000 7efc5b200640 1 --2- 172.21.15.83:0/736366097 >> [v2:172.21.15.83:6800/573907319,v1:172.21.15.83:6801/573907319] conn(0x7efc38038fc0 0x7efc3803b480 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:25:41.777 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:41.109+0000 7efc5b200640 1 --2- 172.21.15.83:0/736366097 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7efc5407de20 0x7efc5419d620 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:25:41.777 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:41.109+0000 7efc5b200640 1 -- 172.21.15.83:0/736366097 >> 172.21.15.83:0/736366097 conn(0x7efc54102bd0 msgr2=0x7efc54103790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:25:41.777 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:41.109+0000 7efc5b200640 1 -- 172.21.15.83:0/736366097 shutdown_connections 2024-08-19T01:25:41.777 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:41.109+0000 7efc5b200640 1 -- 172.21.15.83:0/736366097 wait complete. 2024-08-19T01:25:41.777 INFO:teuthology.orchestra.run.smithi083.stdout:Generating ssh key... 2024-08-19T01:25:42.466 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:25:42 smithi083 ceph-mon[17120]: from='client.14132 -' entity='client.admin' cmd=[{"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}]: dispatch 2024-08-19T01:25:42.857 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Aug 19 01:25:42 smithi083 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-mgr-x[17344]: Generating public/private rsa key pair. 2024-08-19T01:25:42.857 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Aug 19 01:25:42 smithi083 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-mgr-x[17344]: Your identification has been saved in /tmp/tmpixn064ry/key 2024-08-19T01:25:42.857 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Aug 19 01:25:42 smithi083 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-mgr-x[17344]: Your public key has been saved in /tmp/tmpixn064ry/key.pub 2024-08-19T01:25:42.857 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Aug 19 01:25:42 smithi083 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-mgr-x[17344]: The key fingerprint is: 2024-08-19T01:25:42.857 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Aug 19 01:25:42 smithi083 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-mgr-x[17344]: SHA256:APgGHqCzysFHaz+rbrLSwfYqghVFoRGyayaaUwDUZGI ceph-bad8780e-5dc9-11ef-bccf-c7b262605968 2024-08-19T01:25:42.858 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Aug 19 01:25:42 smithi083 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-mgr-x[17344]: The key's randomart image is: 2024-08-19T01:25:42.858 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Aug 19 01:25:42 smithi083 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-mgr-x[17344]: +---[RSA 3072]----+ 2024-08-19T01:25:42.858 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Aug 19 01:25:42 smithi083 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-mgr-x[17344]: |=E=*+. | 2024-08-19T01:25:42.858 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Aug 19 01:25:42 smithi083 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-mgr-x[17344]: |+o*+.. | 2024-08-19T01:25:42.858 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Aug 19 01:25:42 smithi083 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-mgr-x[17344]: |=..* . | 2024-08-19T01:25:42.858 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Aug 19 01:25:42 smithi083 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-mgr-x[17344]: |.=+ + . | 2024-08-19T01:25:42.858 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Aug 19 01:25:42 smithi083 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-mgr-x[17344]: |+*.* S | 2024-08-19T01:25:42.858 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Aug 19 01:25:42 smithi083 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-mgr-x[17344]: |BoO . | 2024-08-19T01:25:42.858 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Aug 19 01:25:42 smithi083 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-mgr-x[17344]: |*= o o | 2024-08-19T01:25:42.858 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Aug 19 01:25:42 smithi083 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-mgr-x[17344]: |=o... o | 2024-08-19T01:25:42.858 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Aug 19 01:25:42 smithi083 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-mgr-x[17344]: |oo*+.. | 2024-08-19T01:25:42.858 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Aug 19 01:25:42 smithi083 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-mgr-x[17344]: +----[SHA256]-----+ 2024-08-19T01:25:43.173 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:42.271+0000 7f3e06600640 1 Processor -- start 2024-08-19T01:25:43.174 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:42.271+0000 7f3e06600640 1 -- start start 2024-08-19T01:25:43.174 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:42.271+0000 7f3e06600640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3e0010aac0 0x7f3e0010aec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:25:43.174 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:42.271+0000 7f3e06600640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f3e0010b490 con 0x7f3e0010aac0 2024-08-19T01:25:43.174 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:42.272+0000 7f3dff400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3e0010aac0 0x7f3e0010aec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:25:43.174 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:42.272+0000 7f3dff400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3e0010aac0 0x7f3e0010aec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:41218/0 (socket says 172.21.15.83:41218) 2024-08-19T01:25:43.174 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:42.272+0000 7f3dff400640 1 -- 172.21.15.83:0/398490321 learned_addr learned my addr 172.21.15.83:0/398490321 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-08-19T01:25:43.174 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:42.272+0000 7f3dff400640 1 -- 172.21.15.83:0/398490321 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3e0010b610 con 0x7f3e0010aac0 2024-08-19T01:25:43.174 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:42.273+0000 7f3dff400640 1 --2- 172.21.15.83:0/398490321 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3e0010aac0 0x7f3e0010aec0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f3df0009820 tx=0x7f3df002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f25e1504b3f72b8b server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:25:43.174 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:42.273+0000 7f3dfe000640 1 -- 172.21.15.83:0/398490321 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3df0037070 con 0x7f3e0010aac0 2024-08-19T01:25:43.174 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:42.274+0000 7f3dfe000640 1 -- 172.21.15.83:0/398490321 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f3df0032900 con 0x7f3e0010aac0 2024-08-19T01:25:43.174 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:42.274+0000 7f3dfe000640 1 -- 172.21.15.83:0/398490321 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3df0030850 con 0x7f3e0010aac0 2024-08-19T01:25:43.174 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:42.274+0000 7f3e06600640 1 -- 172.21.15.83:0/398490321 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3e0010aac0 msgr2=0x7f3e0010aec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:25:43.174 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:42.274+0000 7f3e06600640 1 --2- 172.21.15.83:0/398490321 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3e0010aac0 0x7f3e0010aec0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f3df0009820 tx=0x7f3df002a270 comp rx=0 tx=0).stop 2024-08-19T01:25:43.175 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:42.274+0000 7f3e06600640 1 -- 172.21.15.83:0/398490321 shutdown_connections 2024-08-19T01:25:43.175 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:42.274+0000 7f3e06600640 1 --2- 172.21.15.83:0/398490321 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3e0010aac0 0x7f3e0010aec0 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:25:43.175 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:42.274+0000 7f3e06600640 1 -- 172.21.15.83:0/398490321 >> 172.21.15.83:0/398490321 conn(0x7f3e0007cc40 msgr2=0x7f3e0007d050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:25:43.175 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:42.274+0000 7f3e06600640 1 -- 172.21.15.83:0/398490321 shutdown_connections 2024-08-19T01:25:43.175 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:42.274+0000 7f3e06600640 1 -- 172.21.15.83:0/398490321 wait complete. 2024-08-19T01:25:43.175 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:42.275+0000 7f3e06600640 1 Processor -- start 2024-08-19T01:25:43.175 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:42.275+0000 7f3e06600640 1 -- start start 2024-08-19T01:25:43.175 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:42.276+0000 7f3e06600640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3e0010aac0 0x7f3e0019d690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:25:43.175 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:42.276+0000 7f3e06600640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f3e0010db20 con 0x7f3e0010aac0 2024-08-19T01:25:43.175 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:42.276+0000 7f3dff400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3e0010aac0 0x7f3e0019d690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:25:43.175 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:42.276+0000 7f3dff400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3e0010aac0 0x7f3e0019d690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:41222/0 (socket says 172.21.15.83:41222) 2024-08-19T01:25:43.175 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:42.276+0000 7f3dff400640 1 -- 172.21.15.83:0/1888919261 learned_addr learned my addr 172.21.15.83:0/1888919261 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-08-19T01:25:43.175 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:42.276+0000 7f3dff400640 1 -- 172.21.15.83:0/1888919261 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3e0019dbd0 con 0x7f3e0010aac0 2024-08-19T01:25:43.176 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:42.277+0000 7f3dff400640 1 --2- 172.21.15.83:0/1888919261 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3e0010aac0 0x7f3e0019d690 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f3df0009940 tx=0x7f3df00326e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:25:43.176 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:42.277+0000 7f3df7e00640 1 -- 172.21.15.83:0/1888919261 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3df0042070 con 0x7f3e0010aac0 2024-08-19T01:25:43.176 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:42.277+0000 7f3df7e00640 1 -- 172.21.15.83:0/1888919261 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f3df0032be0 con 0x7f3e0010aac0 2024-08-19T01:25:43.176 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:42.277+0000 7f3e06600640 1 -- 172.21.15.83:0/1888919261 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3e0019de60 con 0x7f3e0010aac0 2024-08-19T01:25:43.176 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:42.277+0000 7f3df7e00640 1 -- 172.21.15.83:0/1888919261 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3df0037040 con 0x7f3e0010aac0 2024-08-19T01:25:43.176 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:42.277+0000 7f3e06600640 1 -- 172.21.15.83:0/1888919261 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3e001a0b50 con 0x7f3e0010aac0 2024-08-19T01:25:43.176 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:42.278+0000 7f3df7e00640 1 -- 172.21.15.83:0/1888919261 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 8) ==== 46287+0+0 (secure 0 0 0) 0x7f3df0045070 con 0x7f3e0010aac0 2024-08-19T01:25:43.176 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:42.278+0000 7f3e06600640 1 -- 172.21.15.83:0/1888919261 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3e0010dca0 con 0x7f3e0010aac0 2024-08-19T01:25:43.176 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:42.279+0000 7f3df7e00640 1 --2- 172.21.15.83:0/1888919261 >> [v2:172.21.15.83:6800/573907319,v1:172.21.15.83:6801/573907319] conn(0x7f3de4038fc0 0x7f3de403b480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:25:43.176 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:42.279+0000 7f3df7e00640 1 -- 172.21.15.83:0/1888919261 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f3df006bf90 con 0x7f3e0010aac0 2024-08-19T01:25:43.176 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:42.279+0000 7f3dfea00640 1 --2- 172.21.15.83:0/1888919261 >> [v2:172.21.15.83:6800/573907319,v1:172.21.15.83:6801/573907319] conn(0x7f3de4038fc0 0x7f3de403b480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:25:43.176 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:42.279+0000 7f3dfea00640 1 --2- 172.21.15.83:0/1888919261 >> [v2:172.21.15.83:6800/573907319,v1:172.21.15.83:6801/573907319] conn(0x7f3de4038fc0 0x7f3de403b480 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f3dec0098c0 tx=0x7f3dec006db0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:25:43.176 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:42.283+0000 7f3df7e00640 1 -- 172.21.15.83:0/1888919261 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7f3df004b170 con 0x7f3e0010aac0 2024-08-19T01:25:43.176 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:42.464+0000 7f3e06600640 1 -- 172.21.15.83:0/1888919261 --> [v2:172.21.15.83:6800/573907319,v1:172.21.15.83:6801/573907319] -- mgr_command(tid 0: {"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}) -- 0x7f3e00107c60 con 0x7f3de4038fc0 2024-08-19T01:25:43.177 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:42.534+0000 7f3df7e00640 1 -- 172.21.15.83:0/1888919261 <== mgr.14118 v2:172.21.15.83:6800/573907319 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f3e00107c60 con 0x7f3de4038fc0 2024-08-19T01:25:43.177 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:42.537+0000 7f3e06600640 1 -- 172.21.15.83:0/1888919261 >> [v2:172.21.15.83:6800/573907319,v1:172.21.15.83:6801/573907319] conn(0x7f3de4038fc0 msgr2=0x7f3de403b480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:25:43.177 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:42.537+0000 7f3e06600640 1 --2- 172.21.15.83:0/1888919261 >> [v2:172.21.15.83:6800/573907319,v1:172.21.15.83:6801/573907319] conn(0x7f3de4038fc0 0x7f3de403b480 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f3dec0098c0 tx=0x7f3dec006db0 comp rx=0 tx=0).stop 2024-08-19T01:25:43.177 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:42.537+0000 7f3e06600640 1 -- 172.21.15.83:0/1888919261 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3e0010aac0 msgr2=0x7f3e0019d690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:25:43.177 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:42.537+0000 7f3e06600640 1 --2- 172.21.15.83:0/1888919261 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3e0010aac0 0x7f3e0019d690 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f3df0009940 tx=0x7f3df00326e0 comp rx=0 tx=0).stop 2024-08-19T01:25:43.177 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:42.537+0000 7f3e06600640 1 -- 172.21.15.83:0/1888919261 shutdown_connections 2024-08-19T01:25:43.177 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:42.537+0000 7f3e06600640 1 --2- 172.21.15.83:0/1888919261 >> [v2:172.21.15.83:6800/573907319,v1:172.21.15.83:6801/573907319] conn(0x7f3de4038fc0 0x7f3de403b480 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:25:43.177 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:42.537+0000 7f3e06600640 1 --2- 172.21.15.83:0/1888919261 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3e0010aac0 0x7f3e0019d690 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:25:43.177 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:42.537+0000 7f3e06600640 1 -- 172.21.15.83:0/1888919261 >> 172.21.15.83:0/1888919261 conn(0x7f3e0007cc40 msgr2=0x7f3e00107550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:25:43.177 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:42.537+0000 7f3e06600640 1 -- 172.21.15.83:0/1888919261 shutdown_connections 2024-08-19T01:25:43.177 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:42.537+0000 7f3e06600640 1 -- 172.21.15.83:0/1888919261 wait complete. 2024-08-19T01:25:43.690 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:25:43 smithi083 ceph-mon[17120]: from='client.14134 -' entity='client.admin' cmd=[{"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}]: dispatch 2024-08-19T01:25:43.690 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:25:43 smithi083 ceph-mon[17120]: Generating ssh key... 2024-08-19T01:25:43.690 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:25:43 smithi083 ceph-mon[17120]: from='mgr.14118 172.21.15.83:0/3307054916' entity='mgr.x' 2024-08-19T01:25:43.690 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:25:43 smithi083 ceph-mon[17120]: from='mgr.14118 172.21.15.83:0/3307054916' entity='mgr.x' 2024-08-19T01:25:44.513 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCrvIGdPB8Hdno6mG7Qs8/pMv7eXZxIWWZCRBi5vWdNl6Dczcbp+t0qGD2pLrph8IZrtS9VH2kdbTCamX8oBGbFDwwpx1iArwlf/OZumZLnEsj0sLMUVD9VkfY/xLfr4IY0lQWgjkmmzrcE8LNoyKFlGRElC9M/K04zoQQyRz2UTjkotDtpVGCsogWJlqnQScxO+XNTSQSb7bR27N7OIclfvId0qMUAAECpMIVgTSDlRnF5Os4WwewnOuQ0WEs5vX2Nx3if4kAeIAYGm5+Bdj1XpGiG+BkwueH9QNZ1gngixBp2fY+gQn7X2GAYP60DB6mdez9f7B/gxaBjCzjtogMbcoYDCaK3PSFMP5IfSSLqySu66CCh2NlppeR77RMnpVlrmlemHSJLvuDKpF98BbB9rIjYDNcVpsjj2LgeBN7QLOTpbSS2FE56eBf9FFUvtexmPnabWCUOFM/sAIn/IT1Xob3lcAQNHPSW44r0drd31eokC8VmoYSGHiZ67AC/KBE= ceph-bad8780e-5dc9-11ef-bccf-c7b262605968 2024-08-19T01:25:44.514 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:43.674+0000 7fd636400640 1 Processor -- start 2024-08-19T01:25:44.514 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:43.674+0000 7fd636400640 1 -- start start 2024-08-19T01:25:44.514 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:43.674+0000 7fd636400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd630109470 0x7fd630109870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:25:44.514 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:43.674+0000 7fd636400640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fd630109e40 con 0x7fd630109470 2024-08-19T01:25:44.514 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:43.674+0000 7fd635000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd630109470 0x7fd630109870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:25:44.514 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:43.674+0000 7fd635000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd630109470 0x7fd630109870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:41226/0 (socket says 172.21.15.83:41226) 2024-08-19T01:25:44.514 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:43.674+0000 7fd635000640 1 -- 172.21.15.83:0/1423315229 learned_addr learned my addr 172.21.15.83:0/1423315229 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-08-19T01:25:44.514 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:43.675+0000 7fd635000640 1 -- 172.21.15.83:0/1423315229 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd63010a670 con 0x7fd630109470 2024-08-19T01:25:44.514 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:43.676+0000 7fd635000640 1 --2- 172.21.15.83:0/1423315229 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd630109470 0x7fd630109870 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fd624009820 tx=0x7fd62402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5cc581df628e4edd server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:25:44.514 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:43.676+0000 7fd62f400640 1 -- 172.21.15.83:0/1423315229 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd624037070 con 0x7fd630109470 2024-08-19T01:25:44.514 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:43.676+0000 7fd62f400640 1 -- 172.21.15.83:0/1423315229 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fd624032900 con 0x7fd630109470 2024-08-19T01:25:44.515 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:43.676+0000 7fd62f400640 1 -- 172.21.15.83:0/1423315229 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd624030850 con 0x7fd630109470 2024-08-19T01:25:44.515 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:43.677+0000 7fd636400640 1 -- 172.21.15.83:0/1423315229 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd630109470 msgr2=0x7fd630109870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:25:44.515 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:43.677+0000 7fd636400640 1 --2- 172.21.15.83:0/1423315229 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd630109470 0x7fd630109870 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fd624009820 tx=0x7fd62402a270 comp rx=0 tx=0).stop 2024-08-19T01:25:44.515 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:43.677+0000 7fd636400640 1 -- 172.21.15.83:0/1423315229 shutdown_connections 2024-08-19T01:25:44.515 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:43.677+0000 7fd636400640 1 --2- 172.21.15.83:0/1423315229 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd630109470 0x7fd630109870 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:25:44.515 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:43.677+0000 7fd636400640 1 -- 172.21.15.83:0/1423315229 >> 172.21.15.83:0/1423315229 conn(0x7fd63007ca50 msgr2=0x7fd630107340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:25:44.515 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:43.677+0000 7fd636400640 1 -- 172.21.15.83:0/1423315229 shutdown_connections 2024-08-19T01:25:44.515 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:43.677+0000 7fd636400640 1 -- 172.21.15.83:0/1423315229 wait complete. 2024-08-19T01:25:44.515 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:43.678+0000 7fd636400640 1 Processor -- start 2024-08-19T01:25:44.515 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:43.678+0000 7fd636400640 1 -- start start 2024-08-19T01:25:44.515 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:43.678+0000 7fd636400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd630109470 0x7fd63019d470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:25:44.515 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:43.678+0000 7fd636400640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fd63010da20 con 0x7fd630109470 2024-08-19T01:25:44.515 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:43.678+0000 7fd635000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd630109470 0x7fd63019d470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:25:44.515 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:43.679+0000 7fd635000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd630109470 0x7fd63019d470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:41228/0 (socket says 172.21.15.83:41228) 2024-08-19T01:25:44.516 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:43.679+0000 7fd635000640 1 -- 172.21.15.83:0/1321063200 learned_addr learned my addr 172.21.15.83:0/1321063200 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-08-19T01:25:44.516 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:43.679+0000 7fd635000640 1 -- 172.21.15.83:0/1321063200 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd63019d9b0 con 0x7fd630109470 2024-08-19T01:25:44.516 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:43.679+0000 7fd635000640 1 --2- 172.21.15.83:0/1321063200 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd630109470 0x7fd63019d470 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fd6240097f0 tx=0x7fd624030d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:25:44.516 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:43.679+0000 7fd62d600640 1 -- 172.21.15.83:0/1321063200 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd624037040 con 0x7fd630109470 2024-08-19T01:25:44.516 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:43.679+0000 7fd62d600640 1 -- 172.21.15.83:0/1321063200 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fd624040070 con 0x7fd630109470 2024-08-19T01:25:44.516 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:43.679+0000 7fd636400640 1 -- 172.21.15.83:0/1321063200 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd63019dc40 con 0x7fd630109470 2024-08-19T01:25:44.516 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:43.679+0000 7fd62d600640 1 -- 172.21.15.83:0/1321063200 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd62403a390 con 0x7fd630109470 2024-08-19T01:25:44.516 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:43.680+0000 7fd636400640 1 -- 172.21.15.83:0/1321063200 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd6301a0930 con 0x7fd630109470 2024-08-19T01:25:44.516 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:43.680+0000 7fd636400640 1 -- 172.21.15.83:0/1321063200 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd6100052c0 con 0x7fd630109470 2024-08-19T01:25:44.516 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:43.681+0000 7fd62d600640 1 -- 172.21.15.83:0/1321063200 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 8) ==== 46287+0+0 (secure 0 0 0) 0x7fd62403b430 con 0x7fd630109470 2024-08-19T01:25:44.516 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:43.682+0000 7fd62d600640 1 --2- 172.21.15.83:0/1321063200 >> [v2:172.21.15.83:6800/573907319,v1:172.21.15.83:6801/573907319] conn(0x7fd618038fc0 0x7fd61803b480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:25:44.516 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:43.682+0000 7fd62d600640 1 -- 172.21.15.83:0/1321063200 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fd62406c490 con 0x7fd630109470 2024-08-19T01:25:44.516 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:43.682+0000 7fd62fe00640 1 --2- 172.21.15.83:0/1321063200 >> [v2:172.21.15.83:6800/573907319,v1:172.21.15.83:6801/573907319] conn(0x7fd618038fc0 0x7fd61803b480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:25:44.516 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:43.683+0000 7fd62fe00640 1 --2- 172.21.15.83:0/1321063200 >> [v2:172.21.15.83:6800/573907319,v1:172.21.15.83:6801/573907319] conn(0x7fd618038fc0 0x7fd61803b480 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fd61c0098c0 tx=0x7fd61c006db0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:25:44.517 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:43.687+0000 7fd62d600640 1 -- 172.21.15.83:0/1321063200 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7fd6240371e0 con 0x7fd630109470 2024-08-19T01:25:44.517 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:43.861+0000 7fd636400640 1 -- 172.21.15.83:0/1321063200 --> [v2:172.21.15.83:6800/573907319,v1:172.21.15.83:6801/573907319] -- mgr_command(tid 0: {"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}) -- 0x7fd610000bf0 con 0x7fd618038fc0 2024-08-19T01:25:44.517 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:43.862+0000 7fd62d600640 1 -- 172.21.15.83:0/1321063200 <== mgr.14118 v2:172.21.15.83:6800/573907319 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+595 (secure 0 0 0) 0x7fd610000bf0 con 0x7fd618038fc0 2024-08-19T01:25:44.517 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:43.864+0000 7fd636400640 1 -- 172.21.15.83:0/1321063200 >> [v2:172.21.15.83:6800/573907319,v1:172.21.15.83:6801/573907319] conn(0x7fd618038fc0 msgr2=0x7fd61803b480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:25:44.517 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:43.864+0000 7fd636400640 1 --2- 172.21.15.83:0/1321063200 >> [v2:172.21.15.83:6800/573907319,v1:172.21.15.83:6801/573907319] conn(0x7fd618038fc0 0x7fd61803b480 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fd61c0098c0 tx=0x7fd61c006db0 comp rx=0 tx=0).stop 2024-08-19T01:25:44.517 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:43.864+0000 7fd636400640 1 -- 172.21.15.83:0/1321063200 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd630109470 msgr2=0x7fd63019d470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:25:44.517 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:43.864+0000 7fd636400640 1 --2- 172.21.15.83:0/1321063200 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd630109470 0x7fd63019d470 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fd6240097f0 tx=0x7fd624030d80 comp rx=0 tx=0).stop 2024-08-19T01:25:44.517 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:43.864+0000 7fd636400640 1 -- 172.21.15.83:0/1321063200 shutdown_connections 2024-08-19T01:25:44.517 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:43.864+0000 7fd636400640 1 --2- 172.21.15.83:0/1321063200 >> [v2:172.21.15.83:6800/573907319,v1:172.21.15.83:6801/573907319] conn(0x7fd618038fc0 0x7fd61803b480 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:25:44.517 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:43.864+0000 7fd636400640 1 --2- 172.21.15.83:0/1321063200 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd630109470 0x7fd63019d470 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:25:44.517 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:43.864+0000 7fd636400640 1 -- 172.21.15.83:0/1321063200 >> 172.21.15.83:0/1321063200 conn(0x7fd63007ca50 msgr2=0x7fd630106ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:25:44.517 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:43.864+0000 7fd636400640 1 -- 172.21.15.83:0/1321063200 shutdown_connections 2024-08-19T01:25:44.517 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:43.864+0000 7fd636400640 1 -- 172.21.15.83:0/1321063200 wait complete. 2024-08-19T01:25:44.517 INFO:teuthology.orchestra.run.smithi083.stdout:Wrote public SSH key to /home/ubuntu/cephtest/ceph.pub 2024-08-19T01:25:44.518 INFO:teuthology.orchestra.run.smithi083.stdout:Adding key to root@localhost authorized_keys... 2024-08-19T01:25:44.528 INFO:teuthology.orchestra.run.smithi083.stdout:Adding host smithi083... 2024-08-19T01:25:44.784 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:25:44 smithi083 ceph-mon[17120]: from='client.14136 -' entity='client.admin' cmd=[{"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}]: dispatch 2024-08-19T01:25:45.655 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:25:45 smithi083 ceph-mon[17120]: from='client.14138 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi083", "addr": "172.21.15.83", "target": ["mon-mgr", ""]}]: dispatch 2024-08-19T01:25:46.787 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:25:46 smithi083 ceph-mon[17120]: Deploying cephadm binary to smithi083 2024-08-19T01:25:49.788 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout Added host 'smithi083' with addr '172.21.15.83' 2024-08-19T01:25:49.789 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:44.997+0000 7f6691600640 1 Processor -- start 2024-08-19T01:25:49.789 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:44.997+0000 7f6691600640 1 -- start start 2024-08-19T01:25:49.789 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:44.997+0000 7f6691600640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f668c107270 0x7f668c107670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:25:49.789 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:44.997+0000 7f6691600640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f668c107c40 con 0x7f668c107270 2024-08-19T01:25:49.789 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:44.998+0000 7f668be00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f668c107270 0x7f668c107670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:25:49.789 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:44.998+0000 7f668be00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f668c107270 0x7f668c107670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:41236/0 (socket says 172.21.15.83:41236) 2024-08-19T01:25:49.789 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:44.998+0000 7f668be00640 1 -- 172.21.15.83:0/3349341543 learned_addr learned my addr 172.21.15.83:0/3349341543 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-08-19T01:25:49.789 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:44.998+0000 7f668be00640 1 -- 172.21.15.83:0/3349341543 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f668c108470 con 0x7f668c107270 2024-08-19T01:25:49.789 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:44.999+0000 7f668be00640 1 --2- 172.21.15.83:0/3349341543 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f668c107270 0x7f668c107670 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f667c009820 tx=0x7f667c02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=605e4ab91b7fef92 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:25:49.789 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:44.999+0000 7f668aa00640 1 -- 172.21.15.83:0/3349341543 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f667c037070 con 0x7f668c107270 2024-08-19T01:25:49.789 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:44.999+0000 7f668aa00640 1 -- 172.21.15.83:0/3349341543 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f667c032900 con 0x7f668c107270 2024-08-19T01:25:49.790 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:44.999+0000 7f668aa00640 1 -- 172.21.15.83:0/3349341543 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f667c030850 con 0x7f668c107270 2024-08-19T01:25:49.790 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:45.000+0000 7f6691600640 1 -- 172.21.15.83:0/3349341543 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f668c107270 msgr2=0x7f668c107670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:25:49.790 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:45.000+0000 7f6691600640 1 --2- 172.21.15.83:0/3349341543 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f668c107270 0x7f668c107670 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f667c009820 tx=0x7f667c02a270 comp rx=0 tx=0).stop 2024-08-19T01:25:49.790 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:45.000+0000 7f6691600640 1 -- 172.21.15.83:0/3349341543 shutdown_connections 2024-08-19T01:25:49.790 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:45.000+0000 7f6691600640 1 --2- 172.21.15.83:0/3349341543 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f668c107270 0x7f668c107670 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:25:49.790 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:45.000+0000 7f6691600640 1 -- 172.21.15.83:0/3349341543 >> 172.21.15.83:0/3349341543 conn(0x7f668c102a20 msgr2=0x7f668c104e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:25:49.790 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:45.000+0000 7f6691600640 1 -- 172.21.15.83:0/3349341543 shutdown_connections 2024-08-19T01:25:49.790 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:45.000+0000 7f6691600640 1 -- 172.21.15.83:0/3349341543 wait complete. 2024-08-19T01:25:49.790 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:45.001+0000 7f6691600640 1 Processor -- start 2024-08-19T01:25:49.790 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:45.001+0000 7f6691600640 1 -- start start 2024-08-19T01:25:49.790 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:45.002+0000 7f6691600640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f668c107270 0x7f668c19d4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:25:49.790 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:45.002+0000 7f6691600640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f668c10b820 con 0x7f668c107270 2024-08-19T01:25:49.791 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:45.002+0000 7f668be00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f668c107270 0x7f668c19d4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:25:49.791 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:45.002+0000 7f668be00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f668c107270 0x7f668c19d4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:41240/0 (socket says 172.21.15.83:41240) 2024-08-19T01:25:49.791 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:45.002+0000 7f668be00640 1 -- 172.21.15.83:0/2638213925 learned_addr learned my addr 172.21.15.83:0/2638213925 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-08-19T01:25:49.791 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:45.002+0000 7f668be00640 1 -- 172.21.15.83:0/2638213925 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f668c19d9e0 con 0x7f668c107270 2024-08-19T01:25:49.791 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:45.003+0000 7f668be00640 1 --2- 172.21.15.83:0/2638213925 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f668c107270 0x7f668c19d4a0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f667c0097f0 tx=0x7f667c030d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:25:49.791 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:45.003+0000 7f6688c00640 1 -- 172.21.15.83:0/2638213925 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f667c037040 con 0x7f668c107270 2024-08-19T01:25:49.791 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:45.003+0000 7f6688c00640 1 -- 172.21.15.83:0/2638213925 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f667c040070 con 0x7f668c107270 2024-08-19T01:25:49.791 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:45.003+0000 7f6688c00640 1 -- 172.21.15.83:0/2638213925 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f667c03a390 con 0x7f668c107270 2024-08-19T01:25:49.791 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:45.003+0000 7f6691600640 1 -- 172.21.15.83:0/2638213925 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f668c19dc70 con 0x7f668c107270 2024-08-19T01:25:49.791 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:45.003+0000 7f6691600640 1 -- 172.21.15.83:0/2638213925 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f668c1a0960 con 0x7f668c107270 2024-08-19T01:25:49.791 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:45.004+0000 7f6688c00640 1 -- 172.21.15.83:0/2638213925 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 8) ==== 46287+0+0 (secure 0 0 0) 0x7f667c03b430 con 0x7f668c107270 2024-08-19T01:25:49.791 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:45.004+0000 7f6691600640 1 -- 172.21.15.83:0/2638213925 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f668c1076f0 con 0x7f668c107270 2024-08-19T01:25:49.791 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:45.004+0000 7f6688c00640 1 --2- 172.21.15.83:0/2638213925 >> [v2:172.21.15.83:6800/573907319,v1:172.21.15.83:6801/573907319] conn(0x7f6670038fc0 0x7f667003b480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:25:49.792 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:45.005+0000 7f6688c00640 1 -- 172.21.15.83:0/2638213925 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f667c06c4e0 con 0x7f668c107270 2024-08-19T01:25:49.792 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:45.005+0000 7f668b400640 1 --2- 172.21.15.83:0/2638213925 >> [v2:172.21.15.83:6800/573907319,v1:172.21.15.83:6801/573907319] conn(0x7f6670038fc0 0x7f667003b480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:25:49.792 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:45.005+0000 7f668b400640 1 --2- 172.21.15.83:0/2638213925 >> [v2:172.21.15.83:6800/573907319,v1:172.21.15.83:6801/573907319] conn(0x7f6670038fc0 0x7f667003b480 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f66780098c0 tx=0x7f6678006db0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:25:49.792 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:45.010+0000 7f6688c00640 1 -- 172.21.15.83:0/2638213925 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7f667c06be30 con 0x7f668c107270 2024-08-19T01:25:49.792 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:45.196+0000 7f6691600640 1 -- 172.21.15.83:0/2638213925 --> [v2:172.21.15.83:6800/573907319,v1:172.21.15.83:6801/573907319] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi083", "addr": "172.21.15.83", "target": ["mon-mgr", ""]}) -- 0x7f668c103c20 con 0x7f6670038fc0 2024-08-19T01:25:49.792 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:49.102+0000 7f6688c00640 1 -- 172.21.15.83:0/2638213925 <== mgr.14118 v2:172.21.15.83:6800/573907319 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+48 (secure 0 0 0) 0x7f668c103c20 con 0x7f6670038fc0 2024-08-19T01:25:49.792 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:49.105+0000 7f6691600640 1 -- 172.21.15.83:0/2638213925 >> [v2:172.21.15.83:6800/573907319,v1:172.21.15.83:6801/573907319] conn(0x7f6670038fc0 msgr2=0x7f667003b480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:25:49.792 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:49.105+0000 7f6691600640 1 --2- 172.21.15.83:0/2638213925 >> [v2:172.21.15.83:6800/573907319,v1:172.21.15.83:6801/573907319] conn(0x7f6670038fc0 0x7f667003b480 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f66780098c0 tx=0x7f6678006db0 comp rx=0 tx=0).stop 2024-08-19T01:25:49.792 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:49.105+0000 7f6691600640 1 -- 172.21.15.83:0/2638213925 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f668c107270 msgr2=0x7f668c19d4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:25:49.792 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:49.105+0000 7f6691600640 1 --2- 172.21.15.83:0/2638213925 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f668c107270 0x7f668c19d4a0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f667c0097f0 tx=0x7f667c030d80 comp rx=0 tx=0).stop 2024-08-19T01:25:49.792 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:49.105+0000 7f6691600640 1 -- 172.21.15.83:0/2638213925 shutdown_connections 2024-08-19T01:25:49.792 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:49.105+0000 7f6691600640 1 --2- 172.21.15.83:0/2638213925 >> [v2:172.21.15.83:6800/573907319,v1:172.21.15.83:6801/573907319] conn(0x7f6670038fc0 0x7f667003b480 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:25:49.792 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:49.105+0000 7f6691600640 1 --2- 172.21.15.83:0/2638213925 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f668c107270 0x7f668c19d4a0 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:25:49.793 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:49.105+0000 7f6691600640 1 -- 172.21.15.83:0/2638213925 >> 172.21.15.83:0/2638213925 conn(0x7f668c102a20 msgr2=0x7f668c103510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:25:49.793 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:49.105+0000 7f6691600640 1 -- 172.21.15.83:0/2638213925 shutdown_connections 2024-08-19T01:25:49.793 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:49.105+0000 7f6691600640 1 -- 172.21.15.83:0/2638213925 wait complete. 2024-08-19T01:25:49.793 INFO:teuthology.orchestra.run.smithi083.stdout:Deploying unmanaged mon service... 2024-08-19T01:25:50.300 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:25:50 smithi083 ceph-mon[17120]: from='mgr.14118 172.21.15.83:0/3307054916' entity='mgr.x' 2024-08-19T01:25:50.300 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:25:50 smithi083 ceph-mon[17120]: Added host smithi083 2024-08-19T01:25:50.301 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:25:50 smithi083 ceph-mon[17120]: from='mgr.14118 172.21.15.83:0/3307054916' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-19T01:25:51.163 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout Scheduled mon update... 2024-08-19T01:25:51.163 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:50.297+0000 7f54a3200640 1 Processor -- start 2024-08-19T01:25:51.163 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:50.297+0000 7f54a3200640 1 -- start start 2024-08-19T01:25:51.163 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:50.298+0000 7f54a3200640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f549c10aac0 0x7f549c10aec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:25:51.163 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:50.298+0000 7f54a3200640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f549c10b490 con 0x7f549c10aac0 2024-08-19T01:25:51.163 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:50.298+0000 7f549be00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f549c10aac0 0x7f549c10aec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:25:51.163 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:50.298+0000 7f549be00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f549c10aac0 0x7f549c10aec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:41244/0 (socket says 172.21.15.83:41244) 2024-08-19T01:25:51.163 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:50.298+0000 7f549be00640 1 -- 172.21.15.83:0/4218754341 learned_addr learned my addr 172.21.15.83:0/4218754341 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-08-19T01:25:51.163 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:50.299+0000 7f549be00640 1 -- 172.21.15.83:0/4218754341 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f549c10b610 con 0x7f549c10aac0 2024-08-19T01:25:51.163 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:50.300+0000 7f549be00640 1 --2- 172.21.15.83:0/4218754341 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f549c10aac0 0x7f549c10aec0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f548c009820 tx=0x7f548c02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fc782e7e7a8c012c server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:25:51.163 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:50.300+0000 7f549b400640 1 -- 172.21.15.83:0/4218754341 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f548c037070 con 0x7f549c10aac0 2024-08-19T01:25:51.164 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:50.300+0000 7f549b400640 1 -- 172.21.15.83:0/4218754341 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f548c032900 con 0x7f549c10aac0 2024-08-19T01:25:51.164 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:50.300+0000 7f549b400640 1 -- 172.21.15.83:0/4218754341 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f548c030850 con 0x7f549c10aac0 2024-08-19T01:25:51.164 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:50.300+0000 7f54a3200640 1 -- 172.21.15.83:0/4218754341 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f549c10aac0 msgr2=0x7f549c10aec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:25:51.164 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:50.300+0000 7f54a3200640 1 --2- 172.21.15.83:0/4218754341 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f549c10aac0 0x7f549c10aec0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f548c009820 tx=0x7f548c02a270 comp rx=0 tx=0).stop 2024-08-19T01:25:51.164 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:50.301+0000 7f54a3200640 1 -- 172.21.15.83:0/4218754341 shutdown_connections 2024-08-19T01:25:51.164 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:50.301+0000 7f54a3200640 1 --2- 172.21.15.83:0/4218754341 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f549c10aac0 0x7f549c10aec0 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:25:51.164 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:50.301+0000 7f54a3200640 1 -- 172.21.15.83:0/4218754341 >> 172.21.15.83:0/4218754341 conn(0x7f549c07cc40 msgr2=0x7f549c07d050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:25:51.164 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:50.301+0000 7f54a3200640 1 -- 172.21.15.83:0/4218754341 shutdown_connections 2024-08-19T01:25:51.164 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:50.301+0000 7f54a3200640 1 -- 172.21.15.83:0/4218754341 wait complete. 2024-08-19T01:25:51.164 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:50.301+0000 7f54a3200640 1 Processor -- start 2024-08-19T01:25:51.164 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:50.302+0000 7f54a3200640 1 -- start start 2024-08-19T01:25:51.164 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:50.302+0000 7f54a3200640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f549c10aac0 0x7f549c19d6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:25:51.164 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:50.302+0000 7f54a3200640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f549c10db20 con 0x7f549c10aac0 2024-08-19T01:25:51.165 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:50.302+0000 7f549be00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f549c10aac0 0x7f549c19d6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:25:51.165 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:50.302+0000 7f549be00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f549c10aac0 0x7f549c19d6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:41252/0 (socket says 172.21.15.83:41252) 2024-08-19T01:25:51.165 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:50.302+0000 7f549be00640 1 -- 172.21.15.83:0/1249383300 learned_addr learned my addr 172.21.15.83:0/1249383300 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-08-19T01:25:51.165 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:50.302+0000 7f549be00640 1 -- 172.21.15.83:0/1249383300 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f549c19dbe0 con 0x7f549c10aac0 2024-08-19T01:25:51.165 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:50.303+0000 7f549be00640 1 --2- 172.21.15.83:0/1249383300 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f549c10aac0 0x7f549c19d6a0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f548c0097f0 tx=0x7f548c030ec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:25:51.165 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:50.303+0000 7f5499600640 1 -- 172.21.15.83:0/1249383300 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f548c042070 con 0x7f549c10aac0 2024-08-19T01:25:51.165 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:50.303+0000 7f5499600640 1 -- 172.21.15.83:0/1249383300 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f548c032ca0 con 0x7f549c10aac0 2024-08-19T01:25:51.165 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:50.303+0000 7f5499600640 1 -- 172.21.15.83:0/1249383300 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f548c037040 con 0x7f549c10aac0 2024-08-19T01:25:51.165 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:50.303+0000 7f54a3200640 1 -- 172.21.15.83:0/1249383300 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f549c19de70 con 0x7f549c10aac0 2024-08-19T01:25:51.165 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:50.303+0000 7f54a3200640 1 -- 172.21.15.83:0/1249383300 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f549c1a0b60 con 0x7f549c10aac0 2024-08-19T01:25:51.165 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:50.304+0000 7f5499600640 1 -- 172.21.15.83:0/1249383300 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 8) ==== 46287+0+0 (secure 0 0 0) 0x7f548c045070 con 0x7f549c10aac0 2024-08-19T01:25:51.165 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:50.304+0000 7f54a3200640 1 -- 172.21.15.83:0/1249383300 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f549c10dca0 con 0x7f549c10aac0 2024-08-19T01:25:51.165 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:50.305+0000 7f5499600640 1 --2- 172.21.15.83:0/1249383300 >> [v2:172.21.15.83:6800/573907319,v1:172.21.15.83:6801/573907319] conn(0x7f5480038fc0 0x7f548003b480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:25:51.165 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:50.305+0000 7f5499600640 1 -- 172.21.15.83:0/1249383300 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f548c06bf50 con 0x7f549c10aac0 2024-08-19T01:25:51.166 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:50.305+0000 7f5493400640 1 --2- 172.21.15.83:0/1249383300 >> [v2:172.21.15.83:6800/573907319,v1:172.21.15.83:6801/573907319] conn(0x7f5480038fc0 0x7f548003b480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:25:51.166 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:50.306+0000 7f5493400640 1 --2- 172.21.15.83:0/1249383300 >> [v2:172.21.15.83:6800/573907319,v1:172.21.15.83:6801/573907319] conn(0x7f5480038fc0 0x7f548003b480 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f54840098c0 tx=0x7f5484006db0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:25:51.166 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:50.310+0000 7f5499600640 1 -- 172.21.15.83:0/1249383300 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7f548c045c60 con 0x7f549c10aac0 2024-08-19T01:25:51.166 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:50.498+0000 7f54a3200640 1 -- 172.21.15.83:0/1249383300 --> [v2:172.21.15.83:6800/573907319,v1:172.21.15.83:6801/573907319] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}) -- 0x7f549c107c80 con 0x7f5480038fc0 2024-08-19T01:25:51.166 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:50.504+0000 7f5499600640 1 -- 172.21.15.83:0/1249383300 <== mgr.14118 v2:172.21.15.83:6800/573907319 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7f549c107c80 con 0x7f5480038fc0 2024-08-19T01:25:51.166 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:50.506+0000 7f54a3200640 1 -- 172.21.15.83:0/1249383300 >> [v2:172.21.15.83:6800/573907319,v1:172.21.15.83:6801/573907319] conn(0x7f5480038fc0 msgr2=0x7f548003b480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:25:51.166 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:50.506+0000 7f54a3200640 1 --2- 172.21.15.83:0/1249383300 >> [v2:172.21.15.83:6800/573907319,v1:172.21.15.83:6801/573907319] conn(0x7f5480038fc0 0x7f548003b480 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f54840098c0 tx=0x7f5484006db0 comp rx=0 tx=0).stop 2024-08-19T01:25:51.166 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:50.506+0000 7f54a3200640 1 -- 172.21.15.83:0/1249383300 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f549c10aac0 msgr2=0x7f549c19d6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:25:51.166 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:50.506+0000 7f54a3200640 1 --2- 172.21.15.83:0/1249383300 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f549c10aac0 0x7f549c19d6a0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f548c0097f0 tx=0x7f548c030ec0 comp rx=0 tx=0).stop 2024-08-19T01:25:51.166 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:50.506+0000 7f54a3200640 1 -- 172.21.15.83:0/1249383300 shutdown_connections 2024-08-19T01:25:51.166 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:50.506+0000 7f54a3200640 1 --2- 172.21.15.83:0/1249383300 >> [v2:172.21.15.83:6800/573907319,v1:172.21.15.83:6801/573907319] conn(0x7f5480038fc0 0x7f548003b480 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:25:51.166 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:50.506+0000 7f54a3200640 1 --2- 172.21.15.83:0/1249383300 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f549c10aac0 0x7f549c19d6a0 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:25:51.166 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:50.506+0000 7f54a3200640 1 -- 172.21.15.83:0/1249383300 >> 172.21.15.83:0/1249383300 conn(0x7f549c07cc40 msgr2=0x7f549c107570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:25:51.167 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:50.506+0000 7f54a3200640 1 -- 172.21.15.83:0/1249383300 shutdown_connections 2024-08-19T01:25:51.167 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:50.506+0000 7f54a3200640 1 -- 172.21.15.83:0/1249383300 wait complete. 2024-08-19T01:25:51.167 INFO:teuthology.orchestra.run.smithi083.stdout:Deploying unmanaged mgr service... 2024-08-19T01:25:51.604 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:25:51 smithi083 ceph-mon[17120]: from='client.14140 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2024-08-19T01:25:51.604 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:25:51 smithi083 ceph-mon[17120]: Saving service mon spec with placement count:5 2024-08-19T01:25:51.604 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:25:51 smithi083 ceph-mon[17120]: from='mgr.14118 172.21.15.83:0/3307054916' entity='mgr.x' 2024-08-19T01:25:53.639 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:25:53 smithi083 ceph-mon[17120]: from='mgr.14118 172.21.15.83:0/3307054916' entity='mgr.x' 2024-08-19T01:25:53.639 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:25:53 smithi083 ceph-mon[17120]: from='mgr.14118 172.21.15.83:0/3307054916' entity='mgr.x' 2024-08-19T01:25:53.639 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:25:53 smithi083 ceph-mon[17120]: from='mgr.14118 172.21.15.83:0/3307054916' entity='mgr.x' 2024-08-19T01:25:53.649 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout Scheduled mgr update... 2024-08-19T01:25:53.649 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:52.750+0000 7f040f200640 1 Processor -- start 2024-08-19T01:25:53.649 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:52.750+0000 7f040f200640 1 -- start start 2024-08-19T01:25:53.649 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:52.751+0000 7f040f200640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0408107240 0x7f0408107640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:25:53.649 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:52.751+0000 7f040f200640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f0408107c10 con 0x7f0408107240 2024-08-19T01:25:53.649 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:52.751+0000 7f040de00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0408107240 0x7f0408107640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:25:53.649 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:52.751+0000 7f040de00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0408107240 0x7f0408107640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:42856/0 (socket says 172.21.15.83:42856) 2024-08-19T01:25:53.649 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:52.751+0000 7f040de00640 1 -- 172.21.15.83:0/1924676260 learned_addr learned my addr 172.21.15.83:0/1924676260 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-08-19T01:25:53.650 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:52.752+0000 7f040de00640 1 -- 172.21.15.83:0/1924676260 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0408108440 con 0x7f0408107240 2024-08-19T01:25:53.650 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:52.752+0000 7f040de00640 1 --2- 172.21.15.83:0/1924676260 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0408107240 0x7f0408107640 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f03fc009820 tx=0x7f03fc02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=af680c44f25d18d server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:25:53.650 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:52.753+0000 7f040ca00640 1 -- 172.21.15.83:0/1924676260 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f03fc037070 con 0x7f0408107240 2024-08-19T01:25:53.650 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:52.753+0000 7f040ca00640 1 -- 172.21.15.83:0/1924676260 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f03fc032920 con 0x7f0408107240 2024-08-19T01:25:53.650 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:52.753+0000 7f040ca00640 1 -- 172.21.15.83:0/1924676260 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f03fc030810 con 0x7f0408107240 2024-08-19T01:25:53.650 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:52.753+0000 7f040f200640 1 -- 172.21.15.83:0/1924676260 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0408107240 msgr2=0x7f0408107640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:25:53.650 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:52.753+0000 7f040f200640 1 --2- 172.21.15.83:0/1924676260 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0408107240 0x7f0408107640 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f03fc009820 tx=0x7f03fc02a270 comp rx=0 tx=0).stop 2024-08-19T01:25:53.650 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:52.754+0000 7f040f200640 1 -- 172.21.15.83:0/1924676260 shutdown_connections 2024-08-19T01:25:53.650 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:52.754+0000 7f040f200640 1 --2- 172.21.15.83:0/1924676260 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0408107240 0x7f0408107640 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:25:53.650 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:52.754+0000 7f040f200640 1 -- 172.21.15.83:0/1924676260 >> 172.21.15.83:0/1924676260 conn(0x7f04081029b0 msgr2=0x7f0408104e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:25:53.650 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:52.754+0000 7f040f200640 1 -- 172.21.15.83:0/1924676260 shutdown_connections 2024-08-19T01:25:53.650 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:52.754+0000 7f040f200640 1 -- 172.21.15.83:0/1924676260 wait complete. 2024-08-19T01:25:53.650 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:52.755+0000 7f040f200640 1 Processor -- start 2024-08-19T01:25:53.651 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:52.755+0000 7f040f200640 1 -- start start 2024-08-19T01:25:53.651 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:52.755+0000 7f040f200640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0408107240 0x7f040819d470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:25:53.651 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:52.756+0000 7f040f200640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f040810b7f0 con 0x7f0408107240 2024-08-19T01:25:53.651 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:52.756+0000 7f040de00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0408107240 0x7f040819d470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:25:53.651 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:52.756+0000 7f040de00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0408107240 0x7f040819d470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:42860/0 (socket says 172.21.15.83:42860) 2024-08-19T01:25:53.651 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:52.756+0000 7f040de00640 1 -- 172.21.15.83:0/101406116 learned_addr learned my addr 172.21.15.83:0/101406116 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-08-19T01:25:53.651 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:52.756+0000 7f040de00640 1 -- 172.21.15.83:0/101406116 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f040819d9b0 con 0x7f0408107240 2024-08-19T01:25:53.651 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:52.756+0000 7f040de00640 1 --2- 172.21.15.83:0/101406116 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0408107240 0x7f040819d470 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f03fc0097f0 tx=0x7f03fc0327b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:25:53.651 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:52.757+0000 7f0402a00640 1 -- 172.21.15.83:0/101406116 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f03fc037040 con 0x7f0408107240 2024-08-19T01:25:53.651 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:52.757+0000 7f0402a00640 1 -- 172.21.15.83:0/101406116 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f03fc030cc0 con 0x7f0408107240 2024-08-19T01:25:53.651 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:52.757+0000 7f0402a00640 1 -- 172.21.15.83:0/101406116 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f03fc03a640 con 0x7f0408107240 2024-08-19T01:25:53.651 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:52.757+0000 7f040f200640 1 -- 172.21.15.83:0/101406116 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f040819dc40 con 0x7f0408107240 2024-08-19T01:25:53.651 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:52.757+0000 7f040f200640 1 -- 172.21.15.83:0/101406116 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f04081a0930 con 0x7f0408107240 2024-08-19T01:25:53.651 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:52.758+0000 7f040f200640 1 -- 172.21.15.83:0/101406116 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f040810b970 con 0x7f0408107240 2024-08-19T01:25:53.652 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:52.758+0000 7f0402a00640 1 -- 172.21.15.83:0/101406116 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 8) ==== 46287+0+0 (secure 0 0 0) 0x7f03fc03b3f0 con 0x7f0408107240 2024-08-19T01:25:53.652 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:52.759+0000 7f0402a00640 1 --2- 172.21.15.83:0/101406116 >> [v2:172.21.15.83:6800/573907319,v1:172.21.15.83:6801/573907319] conn(0x7f03f0038fc0 0x7f03f003b480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:25:53.652 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:52.759+0000 7f0402a00640 1 -- 172.21.15.83:0/101406116 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f03fc06c560 con 0x7f0408107240 2024-08-19T01:25:53.652 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:52.759+0000 7f040d400640 1 --2- 172.21.15.83:0/101406116 >> [v2:172.21.15.83:6800/573907319,v1:172.21.15.83:6801/573907319] conn(0x7f03f0038fc0 0x7f03f003b480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:25:53.652 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:52.759+0000 7f040d400640 1 --2- 172.21.15.83:0/101406116 >> [v2:172.21.15.83:6800/573907319,v1:172.21.15.83:6801/573907319] conn(0x7f03f0038fc0 0x7f03f003b480 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f03f80098c0 tx=0x7f03f8006db0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:25:53.652 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:52.763+0000 7f0402a00640 1 -- 172.21.15.83:0/101406116 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7f03fc0328c0 con 0x7f0408107240 2024-08-19T01:25:53.652 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:52.963+0000 7f040f200640 1 -- 172.21.15.83:0/101406116 --> [v2:172.21.15.83:6800/573907319,v1:172.21.15.83:6801/573907319] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}) -- 0x7f0408103be0 con 0x7f03f0038fc0 2024-08-19T01:25:53.652 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:52.968+0000 7f0402a00640 1 -- 172.21.15.83:0/101406116 <== mgr.14118 v2:172.21.15.83:6800/573907319 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7f0408103be0 con 0x7f03f0038fc0 2024-08-19T01:25:53.652 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:52.971+0000 7f040f200640 1 -- 172.21.15.83:0/101406116 >> [v2:172.21.15.83:6800/573907319,v1:172.21.15.83:6801/573907319] conn(0x7f03f0038fc0 msgr2=0x7f03f003b480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:25:53.652 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:52.971+0000 7f040f200640 1 --2- 172.21.15.83:0/101406116 >> [v2:172.21.15.83:6800/573907319,v1:172.21.15.83:6801/573907319] conn(0x7f03f0038fc0 0x7f03f003b480 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f03f80098c0 tx=0x7f03f8006db0 comp rx=0 tx=0).stop 2024-08-19T01:25:53.652 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:52.971+0000 7f040f200640 1 -- 172.21.15.83:0/101406116 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0408107240 msgr2=0x7f040819d470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:25:53.652 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:52.971+0000 7f040f200640 1 --2- 172.21.15.83:0/101406116 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0408107240 0x7f040819d470 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f03fc0097f0 tx=0x7f03fc0327b0 comp rx=0 tx=0).stop 2024-08-19T01:25:53.652 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:52.971+0000 7f040f200640 1 -- 172.21.15.83:0/101406116 shutdown_connections 2024-08-19T01:25:53.652 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:52.971+0000 7f040f200640 1 --2- 172.21.15.83:0/101406116 >> [v2:172.21.15.83:6800/573907319,v1:172.21.15.83:6801/573907319] conn(0x7f03f0038fc0 0x7f03f003b480 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:25:53.653 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:52.971+0000 7f040f200640 1 --2- 172.21.15.83:0/101406116 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0408107240 0x7f040819d470 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:25:53.653 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:52.971+0000 7f040f200640 1 -- 172.21.15.83:0/101406116 >> 172.21.15.83:0/101406116 conn(0x7f04081029b0 msgr2=0x7f04081034d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:25:53.653 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:52.971+0000 7f040f200640 1 -- 172.21.15.83:0/101406116 shutdown_connections 2024-08-19T01:25:53.653 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:52.971+0000 7f040f200640 1 -- 172.21.15.83:0/101406116 wait complete. 2024-08-19T01:25:54.405 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:25:54 smithi083 ceph-mon[17120]: from='client.14142 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2024-08-19T01:25:54.405 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:25:54 smithi083 ceph-mon[17120]: Saving service mgr spec with placement count:2 2024-08-19T01:25:55.624 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:25:55 smithi083 ceph-mon[17120]: from='client.? 172.21.15.83:0/2457718176' entity='client.admin' 2024-08-19T01:25:55.624 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:25:55 smithi083 ceph-mon[17120]: from='mgr.14118 172.21.15.83:0/3307054916' entity='mgr.x' 2024-08-19T01:25:55.635 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:54.150+0000 7f3d2b000640 1 Processor -- start 2024-08-19T01:25:55.635 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:54.150+0000 7f3d2b000640 1 -- start start 2024-08-19T01:25:55.636 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:54.150+0000 7f3d2b000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3d24107d80 0x7f3d2410a170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:25:55.636 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:54.150+0000 7f3d2b000640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f3d24063180 con 0x7f3d24107d80 2024-08-19T01:25:55.636 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:54.150+0000 7f3d29c00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3d24107d80 0x7f3d2410a170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:25:55.636 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:54.151+0000 7f3d29c00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3d24107d80 0x7f3d2410a170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:42866/0 (socket says 172.21.15.83:42866) 2024-08-19T01:25:55.636 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:54.151+0000 7f3d29c00640 1 -- 172.21.15.83:0/3547907509 learned_addr learned my addr 172.21.15.83:0/3547907509 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-08-19T01:25:55.636 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:54.151+0000 7f3d29c00640 1 -- 172.21.15.83:0/3547907509 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3d2410a6b0 con 0x7f3d24107d80 2024-08-19T01:25:55.636 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:54.152+0000 7f3d29c00640 1 --2- 172.21.15.83:0/3547907509 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3d24107d80 0x7f3d2410a170 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f3d18009820 tx=0x7f3d1802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c1c5b5592ef64dfa server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:25:55.636 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:54.152+0000 7f3d23e00640 1 -- 172.21.15.83:0/3547907509 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3d18037070 con 0x7f3d24107d80 2024-08-19T01:25:55.636 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:54.152+0000 7f3d23e00640 1 -- 172.21.15.83:0/3547907509 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3d18032920 con 0x7f3d24107d80 2024-08-19T01:25:55.636 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:54.152+0000 7f3d23e00640 1 -- 172.21.15.83:0/3547907509 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3d18030810 con 0x7f3d24107d80 2024-08-19T01:25:55.636 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:54.153+0000 7f3d2b000640 1 -- 172.21.15.83:0/3547907509 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3d24107d80 msgr2=0x7f3d2410a170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:25:55.636 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:54.153+0000 7f3d2b000640 1 --2- 172.21.15.83:0/3547907509 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3d24107d80 0x7f3d2410a170 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f3d18009820 tx=0x7f3d1802a270 comp rx=0 tx=0).stop 2024-08-19T01:25:55.636 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:54.153+0000 7f3d2b000640 1 -- 172.21.15.83:0/3547907509 shutdown_connections 2024-08-19T01:25:55.637 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:54.153+0000 7f3d2b000640 1 --2- 172.21.15.83:0/3547907509 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3d24107d80 0x7f3d2410a170 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:25:55.637 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:54.153+0000 7f3d2b000640 1 -- 172.21.15.83:0/3547907509 >> 172.21.15.83:0/3547907509 conn(0x7f3d241018e0 msgr2=0x7f3d24103d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:25:55.637 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:54.153+0000 7f3d2b000640 1 -- 172.21.15.83:0/3547907509 shutdown_connections 2024-08-19T01:25:55.637 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:54.153+0000 7f3d2b000640 1 -- 172.21.15.83:0/3547907509 wait complete. 2024-08-19T01:25:55.637 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:54.154+0000 7f3d2b000640 1 Processor -- start 2024-08-19T01:25:55.637 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:54.154+0000 7f3d2b000640 1 -- start start 2024-08-19T01:25:55.637 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:54.154+0000 7f3d2b000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3d24107d80 0x7f3d2419d470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:25:55.637 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:54.154+0000 7f3d2b000640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f3d2410b6d0 con 0x7f3d24107d80 2024-08-19T01:25:55.637 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:54.155+0000 7f3d29c00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3d24107d80 0x7f3d2419d470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:25:55.637 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:54.155+0000 7f3d29c00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3d24107d80 0x7f3d2419d470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:42880/0 (socket says 172.21.15.83:42880) 2024-08-19T01:25:55.637 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:54.155+0000 7f3d29c00640 1 -- 172.21.15.83:0/2457718176 learned_addr learned my addr 172.21.15.83:0/2457718176 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-08-19T01:25:55.637 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:54.155+0000 7f3d29c00640 1 -- 172.21.15.83:0/2457718176 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3d2419d9b0 con 0x7f3d24107d80 2024-08-19T01:25:55.637 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:54.155+0000 7f3d29c00640 1 --2- 172.21.15.83:0/2457718176 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3d24107d80 0x7f3d2419d470 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f3d180097f0 tx=0x7f3d180327b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:25:55.637 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:54.155+0000 7f3d22000640 1 -- 172.21.15.83:0/2457718176 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3d18037040 con 0x7f3d24107d80 2024-08-19T01:25:55.637 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:54.155+0000 7f3d22000640 1 -- 172.21.15.83:0/2457718176 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3d18030cc0 con 0x7f3d24107d80 2024-08-19T01:25:55.638 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:54.155+0000 7f3d22000640 1 -- 172.21.15.83:0/2457718176 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3d1803a6b0 con 0x7f3d24107d80 2024-08-19T01:25:55.638 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:54.156+0000 7f3d2b000640 1 -- 172.21.15.83:0/2457718176 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3d2419dc40 con 0x7f3d24107d80 2024-08-19T01:25:55.638 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:54.156+0000 7f3d2b000640 1 -- 172.21.15.83:0/2457718176 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3d241a0930 con 0x7f3d24107d80 2024-08-19T01:25:55.638 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:54.157+0000 7f3d2b000640 1 -- 172.21.15.83:0/2457718176 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3d241081e0 con 0x7f3d24107d80 2024-08-19T01:25:55.638 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:54.157+0000 7f3d22000640 1 -- 172.21.15.83:0/2457718176 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 8) ==== 46287+0+0 (secure 0 0 0) 0x7f3d1803baf0 con 0x7f3d24107d80 2024-08-19T01:25:55.638 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:54.157+0000 7f3d22000640 1 --2- 172.21.15.83:0/2457718176 >> [v2:172.21.15.83:6800/573907319,v1:172.21.15.83:6801/573907319] conn(0x7f3d0c038f70 0x7f3d0c03b430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:25:55.638 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:54.157+0000 7f3d22000640 1 -- 172.21.15.83:0/2457718176 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f3d1806c5c0 con 0x7f3d24107d80 2024-08-19T01:25:55.638 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:54.157+0000 7f3d29200640 1 --2- 172.21.15.83:0/2457718176 >> [v2:172.21.15.83:6800/573907319,v1:172.21.15.83:6801/573907319] conn(0x7f3d0c038f70 0x7f3d0c03b430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:25:55.638 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:54.158+0000 7f3d29200640 1 --2- 172.21.15.83:0/2457718176 >> [v2:172.21.15.83:6800/573907319,v1:172.21.15.83:6801/573907319] conn(0x7f3d0c038f70 0x7f3d0c03b430 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f3d100098c0 tx=0x7f3d10006db0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:25:55.638 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:54.162+0000 7f3d22000640 1 -- 172.21.15.83:0/2457718176 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7f3d180371e0 con 0x7f3d24107d80 2024-08-19T01:25:55.638 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:54.347+0000 7f3d2b000640 1 -- 172.21.15.83:0/2457718176 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/container_init}] v 0) -- 0x7f3d241a0f30 con 0x7f3d24107d80 2024-08-19T01:25:55.638 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:54.351+0000 7f3d22000640 1 -- 172.21.15.83:0/2457718176 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/container_init}]=0 v7)=0 v7) ==== 142+0+0 (secure 0 0 0) 0x7f3d1806bf10 con 0x7f3d24107d80 2024-08-19T01:25:55.638 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:54.353+0000 7f3d2b000640 1 -- 172.21.15.83:0/2457718176 >> [v2:172.21.15.83:6800/573907319,v1:172.21.15.83:6801/573907319] conn(0x7f3d0c038f70 msgr2=0x7f3d0c03b430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:25:55.638 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:54.353+0000 7f3d2b000640 1 --2- 172.21.15.83:0/2457718176 >> [v2:172.21.15.83:6800/573907319,v1:172.21.15.83:6801/573907319] conn(0x7f3d0c038f70 0x7f3d0c03b430 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f3d100098c0 tx=0x7f3d10006db0 comp rx=0 tx=0).stop 2024-08-19T01:25:55.639 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:54.353+0000 7f3d2b000640 1 -- 172.21.15.83:0/2457718176 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3d24107d80 msgr2=0x7f3d2419d470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:25:55.639 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:54.353+0000 7f3d2b000640 1 --2- 172.21.15.83:0/2457718176 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3d24107d80 0x7f3d2419d470 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f3d180097f0 tx=0x7f3d180327b0 comp rx=0 tx=0).stop 2024-08-19T01:25:55.639 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:54.353+0000 7f3d2b000640 1 -- 172.21.15.83:0/2457718176 shutdown_connections 2024-08-19T01:25:55.639 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:54.353+0000 7f3d2b000640 1 --2- 172.21.15.83:0/2457718176 >> [v2:172.21.15.83:6800/573907319,v1:172.21.15.83:6801/573907319] conn(0x7f3d0c038f70 0x7f3d0c03b430 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:25:55.639 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:54.353+0000 7f3d2b000640 1 --2- 172.21.15.83:0/2457718176 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3d24107d80 0x7f3d2419d470 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:25:55.639 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:54.353+0000 7f3d2b000640 1 -- 172.21.15.83:0/2457718176 >> 172.21.15.83:0/2457718176 conn(0x7f3d241018e0 msgr2=0x7f3d24102440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:25:55.639 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:54.353+0000 7f3d2b000640 1 -- 172.21.15.83:0/2457718176 shutdown_connections 2024-08-19T01:25:55.639 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:54.353+0000 7f3d2b000640 1 -- 172.21.15.83:0/2457718176 wait complete. 2024-08-19T01:25:57.027 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:56.137+0000 7f5930200640 1 Processor -- start 2024-08-19T01:25:57.027 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:56.138+0000 7f5930200640 1 -- start start 2024-08-19T01:25:57.027 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:56.138+0000 7f5930200640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f592810aac0 0x7f592810aec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:25:57.027 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:56.138+0000 7f5930200640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f592810b490 con 0x7f592810aac0 2024-08-19T01:25:57.027 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:56.139+0000 7f592d800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f592810aac0 0x7f592810aec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:25:57.027 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:56.139+0000 7f592d800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f592810aac0 0x7f592810aec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:42894/0 (socket says 172.21.15.83:42894) 2024-08-19T01:25:57.027 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:56.139+0000 7f592d800640 1 -- 172.21.15.83:0/2209895630 learned_addr learned my addr 172.21.15.83:0/2209895630 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-08-19T01:25:57.028 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:56.139+0000 7f592d800640 1 -- 172.21.15.83:0/2209895630 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f592810b610 con 0x7f592810aac0 2024-08-19T01:25:57.028 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:56.140+0000 7f592d800640 1 --2- 172.21.15.83:0/2209895630 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f592810aac0 0x7f592810aec0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f591c009e80 tx=0x7f591c02a7a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e69747c111763f74 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:25:57.028 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:56.140+0000 7f5927e00640 1 -- 172.21.15.83:0/2209895630 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f591c036070 con 0x7f592810aac0 2024-08-19T01:25:57.028 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:56.140+0000 7f5927e00640 1 -- 172.21.15.83:0/2209895630 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f591c0318e0 con 0x7f592810aac0 2024-08-19T01:25:57.028 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:56.140+0000 7f5927e00640 1 -- 172.21.15.83:0/2209895630 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f591c007a00 con 0x7f592810aac0 2024-08-19T01:25:57.028 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:56.141+0000 7f5930200640 1 -- 172.21.15.83:0/2209895630 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f592810aac0 msgr2=0x7f592810aec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:25:57.028 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:56.141+0000 7f5930200640 1 --2- 172.21.15.83:0/2209895630 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f592810aac0 0x7f592810aec0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f591c009e80 tx=0x7f591c02a7a0 comp rx=0 tx=0).stop 2024-08-19T01:25:57.028 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:56.141+0000 7f5930200640 1 -- 172.21.15.83:0/2209895630 shutdown_connections 2024-08-19T01:25:57.028 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:56.141+0000 7f5930200640 1 --2- 172.21.15.83:0/2209895630 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f592810aac0 0x7f592810aec0 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:25:57.028 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:56.141+0000 7f5930200640 1 -- 172.21.15.83:0/2209895630 >> 172.21.15.83:0/2209895630 conn(0x7f592807cc40 msgr2=0x7f592807d050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:25:57.028 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:56.141+0000 7f5930200640 1 -- 172.21.15.83:0/2209895630 shutdown_connections 2024-08-19T01:25:57.028 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:56.141+0000 7f5930200640 1 -- 172.21.15.83:0/2209895630 wait complete. 2024-08-19T01:25:57.028 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:56.142+0000 7f5930200640 1 Processor -- start 2024-08-19T01:25:57.029 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:56.142+0000 7f5930200640 1 -- start start 2024-08-19T01:25:57.029 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:56.143+0000 7f5930200640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f592810aac0 0x7f592819d690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:25:57.029 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:56.143+0000 7f5930200640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f592810db20 con 0x7f592810aac0 2024-08-19T01:25:57.029 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:56.143+0000 7f592d800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f592810aac0 0x7f592819d690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:25:57.029 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:56.143+0000 7f592d800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f592810aac0 0x7f592819d690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:42898/0 (socket says 172.21.15.83:42898) 2024-08-19T01:25:57.029 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:56.143+0000 7f592d800640 1 -- 172.21.15.83:0/2698807062 learned_addr learned my addr 172.21.15.83:0/2698807062 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-08-19T01:25:57.029 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:56.143+0000 7f592d800640 1 -- 172.21.15.83:0/2698807062 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f592819dbd0 con 0x7f592810aac0 2024-08-19T01:25:57.029 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:56.143+0000 7f592d800640 1 --2- 172.21.15.83:0/2698807062 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f592810aac0 0x7f592819d690 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f591c02acc0 tx=0x7f591c031cc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:25:57.029 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:56.144+0000 7f5926000640 1 -- 172.21.15.83:0/2698807062 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f591c036050 con 0x7f592810aac0 2024-08-19T01:25:57.029 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:56.144+0000 7f5930200640 1 -- 172.21.15.83:0/2698807062 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f592819de60 con 0x7f592810aac0 2024-08-19T01:25:57.029 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:56.144+0000 7f5926000640 1 -- 172.21.15.83:0/2698807062 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f591c037040 con 0x7f592810aac0 2024-08-19T01:25:57.029 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:56.144+0000 7f5926000640 1 -- 172.21.15.83:0/2698807062 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f591c03b920 con 0x7f592810aac0 2024-08-19T01:25:57.029 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:56.144+0000 7f5930200640 1 -- 172.21.15.83:0/2698807062 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f59281a0b50 con 0x7f592810aac0 2024-08-19T01:25:57.029 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:56.145+0000 7f5926000640 1 -- 172.21.15.83:0/2698807062 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 8) ==== 46287+0+0 (secure 0 0 0) 0x7f591c044020 con 0x7f592810aac0 2024-08-19T01:25:57.030 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:56.145+0000 7f5930200640 1 -- 172.21.15.83:0/2698807062 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f592810aec0 con 0x7f592810aac0 2024-08-19T01:25:57.030 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:56.145+0000 7f5926000640 1 --2- 172.21.15.83:0/2698807062 >> [v2:172.21.15.83:6800/573907319,v1:172.21.15.83:6801/573907319] conn(0x7f5910038f70 0x7f591003b430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:25:57.030 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:56.145+0000 7f5926000640 1 -- 172.21.15.83:0/2698807062 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f591c042020 con 0x7f592810aac0 2024-08-19T01:25:57.030 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:56.146+0000 7f592ce00640 1 --2- 172.21.15.83:0/2698807062 >> [v2:172.21.15.83:6800/573907319,v1:172.21.15.83:6801/573907319] conn(0x7f5910038f70 0x7f591003b430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:25:57.030 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:56.146+0000 7f592ce00640 1 --2- 172.21.15.83:0/2698807062 >> [v2:172.21.15.83:6800/573907319,v1:172.21.15.83:6801/573907319] conn(0x7f5910038f70 0x7f591003b430 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f59140098c0 tx=0x7f5914006db0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:25:57.030 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:56.150+0000 7f5926000640 1 -- 172.21.15.83:0/2698807062 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7f591c04b020 con 0x7f592810aac0 2024-08-19T01:25:57.030 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:56.331+0000 7f5930200640 1 -- 172.21.15.83:0/2698807062 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command([{prefix=config set, name=mgr/dashboard/ssl_server_port}] v 0) -- 0x7f59281a11b0 con 0x7f592810aac0 2024-08-19T01:25:57.030 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:56.334+0000 7f5926000640 1 -- 172.21.15.83:0/2698807062 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/dashboard/ssl_server_port}]=0 v8)=0 v8) ==== 130+0+0 (secure 0 0 0) 0x7f591c03a360 con 0x7f592810aac0 2024-08-19T01:25:57.030 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:56.337+0000 7f5930200640 1 -- 172.21.15.83:0/2698807062 >> [v2:172.21.15.83:6800/573907319,v1:172.21.15.83:6801/573907319] conn(0x7f5910038f70 msgr2=0x7f591003b430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:25:57.030 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:56.337+0000 7f5930200640 1 --2- 172.21.15.83:0/2698807062 >> [v2:172.21.15.83:6800/573907319,v1:172.21.15.83:6801/573907319] conn(0x7f5910038f70 0x7f591003b430 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f59140098c0 tx=0x7f5914006db0 comp rx=0 tx=0).stop 2024-08-19T01:25:57.030 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:56.337+0000 7f5930200640 1 -- 172.21.15.83:0/2698807062 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f592810aac0 msgr2=0x7f592819d690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:25:57.030 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:56.337+0000 7f5930200640 1 --2- 172.21.15.83:0/2698807062 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f592810aac0 0x7f592819d690 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f591c02acc0 tx=0x7f591c031cc0 comp rx=0 tx=0).stop 2024-08-19T01:25:57.030 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:56.337+0000 7f5930200640 1 -- 172.21.15.83:0/2698807062 shutdown_connections 2024-08-19T01:25:57.031 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:56.337+0000 7f5930200640 1 --2- 172.21.15.83:0/2698807062 >> [v2:172.21.15.83:6800/573907319,v1:172.21.15.83:6801/573907319] conn(0x7f5910038f70 0x7f591003b430 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:25:57.031 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:56.337+0000 7f5930200640 1 --2- 172.21.15.83:0/2698807062 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f592810aac0 0x7f592819d690 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:25:57.031 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:56.337+0000 7f5930200640 1 -- 172.21.15.83:0/2698807062 >> 172.21.15.83:0/2698807062 conn(0x7f592807cc40 msgr2=0x7f59281074c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:25:57.031 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:56.337+0000 7f5930200640 1 -- 172.21.15.83:0/2698807062 shutdown_connections 2024-08-19T01:25:57.031 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:56.337+0000 7f5930200640 1 -- 172.21.15.83:0/2698807062 wait complete. 2024-08-19T01:25:57.031 INFO:teuthology.orchestra.run.smithi083.stdout:Enabling the dashboard module... 2024-08-19T01:25:57.553 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:25:57 smithi083 ceph-mon[17120]: from='client.? 172.21.15.83:0/2698807062' entity='client.admin' 2024-08-19T01:25:57.553 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:25:57 smithi083 ceph-mon[17120]: from='mgr.14118 172.21.15.83:0/3307054916' entity='mgr.x' 2024-08-19T01:25:58.655 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:25:58 smithi083 ceph-mon[17120]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-19T01:25:58.655 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:25:58 smithi083 ceph-mon[17120]: from='client.? 172.21.15.83:0/3718670571' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "dashboard"}]: dispatch 2024-08-19T01:25:58.655 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Aug 19 01:25:58 smithi083 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-mgr-x[17344]: ignoring --setuser ceph since I am not root 2024-08-19T01:25:58.655 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Aug 19 01:25:58 smithi083 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-mgr-x[17344]: ignoring --setgroup ceph since I am not root 2024-08-19T01:25:59.013 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Aug 19 01:25:58 smithi083 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-mgr-x[17344]: 2024-08-19T01:25:58.685+0000 7f03153dc040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-08-19T01:25:59.013 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Aug 19 01:25:58 smithi083 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-mgr-x[17344]: 2024-08-19T01:25:58.763+0000 7f03153dc040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-08-19T01:25:59.013 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Aug 19 01:25:58 smithi083 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-mgr-x[17344]: 2024-08-19T01:25:58.929+0000 7f03153dc040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-08-19T01:25:59.297 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Aug 19 01:25:59 smithi083 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-mgr-x[17344]: 2024-08-19T01:25:59.113+0000 7f03153dc040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-08-19T01:25:59.297 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Aug 19 01:25:59 smithi083 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-mgr-x[17344]: 2024-08-19T01:25:59.295+0000 7f03153dc040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-08-19T01:25:59.461 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:57.569+0000 7efdcba00640 1 Processor -- start 2024-08-19T01:25:59.461 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:57.569+0000 7efdcba00640 1 -- start start 2024-08-19T01:25:59.462 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:57.569+0000 7efdcba00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7efdc4107440 0x7efdc4107840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:25:59.462 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:57.569+0000 7efdcba00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7efdc4107e10 con 0x7efdc4107440 2024-08-19T01:25:59.462 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:57.570+0000 7efdc9000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7efdc4107440 0x7efdc4107840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:25:59.462 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:57.570+0000 7efdc9000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7efdc4107440 0x7efdc4107840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:42900/0 (socket says 172.21.15.83:42900) 2024-08-19T01:25:59.462 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:57.570+0000 7efdc9000640 1 -- 172.21.15.83:0/1924748019 learned_addr learned my addr 172.21.15.83:0/1924748019 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-08-19T01:25:59.462 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:57.570+0000 7efdc9000640 1 -- 172.21.15.83:0/1924748019 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efdc4108640 con 0x7efdc4107440 2024-08-19T01:25:59.462 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:57.571+0000 7efdc9000640 1 --2- 172.21.15.83:0/1924748019 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7efdc4107440 0x7efdc4107840 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7efdb4009820 tx=0x7efdb402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=925298f810298152 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:25:59.462 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:57.571+0000 7efdc3400640 1 -- 172.21.15.83:0/1924748019 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7efdb4037070 con 0x7efdc4107440 2024-08-19T01:25:59.462 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:57.572+0000 7efdc3400640 1 -- 172.21.15.83:0/1924748019 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efdb4032920 con 0x7efdc4107440 2024-08-19T01:25:59.462 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:57.572+0000 7efdc3400640 1 -- 172.21.15.83:0/1924748019 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7efdb4030810 con 0x7efdc4107440 2024-08-19T01:25:59.462 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:57.572+0000 7efdcba00640 1 -- 172.21.15.83:0/1924748019 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7efdc4107440 msgr2=0x7efdc4107840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:25:59.462 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:57.572+0000 7efdcba00640 1 --2- 172.21.15.83:0/1924748019 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7efdc4107440 0x7efdc4107840 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7efdb4009820 tx=0x7efdb402a270 comp rx=0 tx=0).stop 2024-08-19T01:25:59.463 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:57.573+0000 7efdcba00640 1 -- 172.21.15.83:0/1924748019 shutdown_connections 2024-08-19T01:25:59.463 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:57.573+0000 7efdcba00640 1 --2- 172.21.15.83:0/1924748019 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7efdc4107440 0x7efdc4107840 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:25:59.463 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:57.573+0000 7efdcba00640 1 -- 172.21.15.83:0/1924748019 >> 172.21.15.83:0/1924748019 conn(0x7efdc4102bf0 msgr2=0x7efdc4105010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:25:59.463 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:57.573+0000 7efdcba00640 1 -- 172.21.15.83:0/1924748019 shutdown_connections 2024-08-19T01:25:59.463 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:57.573+0000 7efdcba00640 1 -- 172.21.15.83:0/1924748019 wait complete. 2024-08-19T01:25:59.463 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:57.574+0000 7efdcba00640 1 Processor -- start 2024-08-19T01:25:59.463 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:57.574+0000 7efdcba00640 1 -- start start 2024-08-19T01:25:59.463 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:57.574+0000 7efdcba00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7efdc4107440 0x7efdc407c9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:25:59.463 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:57.574+0000 7efdcba00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7efdc410b9f0 con 0x7efdc4107440 2024-08-19T01:25:59.464 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:57.574+0000 7efdc9000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7efdc4107440 0x7efdc407c9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:25:59.464 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:57.574+0000 7efdc9000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7efdc4107440 0x7efdc407c9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:42912/0 (socket says 172.21.15.83:42912) 2024-08-19T01:25:59.464 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:57.574+0000 7efdc9000640 1 -- 172.21.15.83:0/3718670571 learned_addr learned my addr 172.21.15.83:0/3718670571 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-08-19T01:25:59.464 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:57.575+0000 7efdc9000640 1 -- 172.21.15.83:0/3718670571 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efdc407cf20 con 0x7efdc4107440 2024-08-19T01:25:59.464 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:57.575+0000 7efdc9000640 1 --2- 172.21.15.83:0/3718670571 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7efdc4107440 0x7efdc407c9e0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7efdb4030560 tx=0x7efdb4032670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:25:59.464 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:57.575+0000 7efdc1600640 1 -- 172.21.15.83:0/3718670571 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7efdb4037040 con 0x7efdc4107440 2024-08-19T01:25:59.464 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:57.575+0000 7efdc1600640 1 -- 172.21.15.83:0/3718670571 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efdb4040070 con 0x7efdc4107440 2024-08-19T01:25:59.464 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:57.575+0000 7efdcba00640 1 -- 172.21.15.83:0/3718670571 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efdc407b110 con 0x7efdc4107440 2024-08-19T01:25:59.464 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:57.576+0000 7efdc1600640 1 -- 172.21.15.83:0/3718670571 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7efdb403a740 con 0x7efdc4107440 2024-08-19T01:25:59.464 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:57.576+0000 7efdcba00640 1 -- 172.21.15.83:0/3718670571 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efdc407b650 con 0x7efdc4107440 2024-08-19T01:25:59.464 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:57.577+0000 7efdc1600640 1 -- 172.21.15.83:0/3718670571 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 8) ==== 46287+0+0 (secure 0 0 0) 0x7efdb403b420 con 0x7efdc4107440 2024-08-19T01:25:59.464 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:57.577+0000 7efdbaa00640 1 -- 172.21.15.83:0/3718670571 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efdc410bb70 con 0x7efdc4107440 2024-08-19T01:25:59.464 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:57.577+0000 7efdc1600640 1 --2- 172.21.15.83:0/3718670571 >> [v2:172.21.15.83:6800/573907319,v1:172.21.15.83:6801/573907319] conn(0x7efda8038f70 0x7efda803b430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:25:59.465 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:57.577+0000 7efdc1600640 1 -- 172.21.15.83:0/3718670571 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7efdb406c690 con 0x7efdc4107440 2024-08-19T01:25:59.465 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:57.579+0000 7efdc3e00640 1 --2- 172.21.15.83:0/3718670571 >> [v2:172.21.15.83:6800/573907319,v1:172.21.15.83:6801/573907319] conn(0x7efda8038f70 0x7efda803b430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:25:59.465 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:57.579+0000 7efdc3e00640 1 --2- 172.21.15.83:0/3718670571 >> [v2:172.21.15.83:6800/573907319,v1:172.21.15.83:6801/573907319] conn(0x7efda8038f70 0x7efda803b430 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7efdb00098c0 tx=0x7efdb0006db0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:25:59.465 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:57.582+0000 7efdc1600640 1 -- 172.21.15.83:0/3718670571 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7efdb4038280 con 0x7efdc4107440 2024-08-19T01:25:59.465 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:57.817+0000 7efdbaa00640 1 -- 172.21.15.83:0/3718670571 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "dashboard"} v 0) -- 0x7efdc41a2ba0 con 0x7efdc4107440 2024-08-19T01:25:59.465 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:58.341+0000 7efdc1600640 1 -- 172.21.15.83:0/3718670571 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "dashboard"}]=0 v9) ==== 88+0+0 (secure 0 0 0) 0x7efdb403b710 con 0x7efdc4107440 2024-08-19T01:25:59.465 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:58.342+0000 7efdc1600640 1 -- 172.21.15.83:0/3718670571 <== mon.0 v2:172.21.15.83:3300/0 8 ==== mgrmap(e 9) ==== 46300+0+0 (secure 0 0 0) 0x7efdb403bb90 con 0x7efdc4107440 2024-08-19T01:25:59.465 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:58.344+0000 7efdbaa00640 1 -- 172.21.15.83:0/3718670571 >> [v2:172.21.15.83:6800/573907319,v1:172.21.15.83:6801/573907319] conn(0x7efda8038f70 msgr2=0x7efda803b430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:25:59.465 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:58.344+0000 7efdbaa00640 1 --2- 172.21.15.83:0/3718670571 >> [v2:172.21.15.83:6800/573907319,v1:172.21.15.83:6801/573907319] conn(0x7efda8038f70 0x7efda803b430 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7efdb00098c0 tx=0x7efdb0006db0 comp rx=0 tx=0).stop 2024-08-19T01:25:59.465 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:58.345+0000 7efdbaa00640 1 -- 172.21.15.83:0/3718670571 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7efdc4107440 msgr2=0x7efdc407c9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:25:59.466 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:58.345+0000 7efdbaa00640 1 --2- 172.21.15.83:0/3718670571 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7efdc4107440 0x7efdc407c9e0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7efdb4030560 tx=0x7efdb4032670 comp rx=0 tx=0).stop 2024-08-19T01:25:59.466 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:58.345+0000 7efdbaa00640 1 -- 172.21.15.83:0/3718670571 shutdown_connections 2024-08-19T01:25:59.466 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:58.345+0000 7efdbaa00640 1 --2- 172.21.15.83:0/3718670571 >> [v2:172.21.15.83:6800/573907319,v1:172.21.15.83:6801/573907319] conn(0x7efda8038f70 0x7efda803b430 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:25:59.466 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:58.345+0000 7efdbaa00640 1 --2- 172.21.15.83:0/3718670571 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7efdc4107440 0x7efdc407c9e0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:25:59.466 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:58.345+0000 7efdbaa00640 1 -- 172.21.15.83:0/3718670571 >> 172.21.15.83:0/3718670571 conn(0x7efdc4102bf0 msgr2=0x7efdc41037a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:25:59.466 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:58.345+0000 7efdbaa00640 1 -- 172.21.15.83:0/3718670571 shutdown_connections 2024-08-19T01:25:59.466 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:58.346+0000 7efdbaa00640 1 -- 172.21.15.83:0/3718670571 wait complete. 2024-08-19T01:25:59.602 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:25:59 smithi083 ceph-mon[17120]: from='client.? 172.21.15.83:0/3718670571' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "dashboard"}]': finished 2024-08-19T01:25:59.602 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:25:59 smithi083 ceph-mon[17120]: mgrmap e9: x(active, since 21s) 2024-08-19T01:25:59.602 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Aug 19 01:25:59 smithi083 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-mgr-x[17344]: 2024-08-19T01:25:59.449+0000 7f03153dc040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-08-19T01:25:59.868 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Aug 19 01:25:59 smithi083 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-mgr-x[17344]: 2024-08-19T01:25:59.603+0000 7f03153dc040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-08-19T01:25:59.868 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Aug 19 01:25:59 smithi083 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-mgr-x[17344]: 2024-08-19T01:25:59.673+0000 7f03153dc040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-08-19T01:26:00.129 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Aug 19 01:26:00 smithi083 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-mgr-x[17344]: 2024-08-19T01:26:00.051+0000 7f03153dc040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-08-19T01:26:00.129 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Aug 19 01:26:00 smithi083 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-mgr-x[17344]: 2024-08-19T01:26:00.126+0000 7f03153dc040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-08-19T01:26:00.399 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:26:00 smithi083 ceph-mon[17120]: from='client.? 172.21.15.83:0/2272625585' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-08-19T01:26:00.399 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Aug 19 01:26:00 smithi083 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-mgr-x[17344]: 2024-08-19T01:26:00.208+0000 7f03153dc040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-08-19T01:26:00.399 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Aug 19 01:26:00 smithi083 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-mgr-x[17344]: 2024-08-19T01:26:00.396+0000 7f03153dc040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-08-19T01:26:00.655 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Aug 19 01:26:00 smithi083 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-mgr-x[17344]: 2024-08-19T01:26:00.470+0000 7f03153dc040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-08-19T01:26:00.655 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Aug 19 01:26:00 smithi083 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-mgr-x[17344]: 2024-08-19T01:26:00.548+0000 7f03153dc040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-08-19T01:26:00.857 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout { 2024-08-19T01:26:00.857 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "epoch": 9, 2024-08-19T01:26:00.857 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "available": true, 2024-08-19T01:26:00.857 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "active_name": "x", 2024-08-19T01:26:00.857 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-08-19T01:26:00.858 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout } 2024-08-19T01:26:00.858 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:59.974+0000 7f395ac00640 1 Processor -- start 2024-08-19T01:26:00.858 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:59.975+0000 7f395ac00640 1 -- start start 2024-08-19T01:26:00.858 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:59.975+0000 7f395ac00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3954104f30 0x7f3954105ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:26:00.858 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:59.975+0000 7f395ac00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f39541053c0 con 0x7f3954104f30 2024-08-19T01:26:00.858 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:59.975+0000 7f3953e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3954104f30 0x7f3954105ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:26:00.858 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:59.975+0000 7f3953e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3954104f30 0x7f3954105ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:42934/0 (socket says 172.21.15.83:42934) 2024-08-19T01:26:00.858 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:59.975+0000 7f3953e00640 1 -- 172.21.15.83:0/641990818 learned_addr learned my addr 172.21.15.83:0/641990818 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-08-19T01:26:00.858 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:59.976+0000 7f3953e00640 1 -- 172.21.15.83:0/641990818 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3954105540 con 0x7f3954104f30 2024-08-19T01:26:00.858 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:59.977+0000 7f3953e00640 1 --2- 172.21.15.83:0/641990818 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3954104f30 0x7f3954105ba0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f3944009820 tx=0x7f394402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f6e331a0201507a1 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:26:00.858 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:59.977+0000 7f3952a00640 1 -- 172.21.15.83:0/641990818 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3944037070 con 0x7f3954104f30 2024-08-19T01:26:00.858 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:59.977+0000 7f3952a00640 1 -- 172.21.15.83:0/641990818 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3944032920 con 0x7f3954104f30 2024-08-19T01:26:00.858 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:59.977+0000 7f3952a00640 1 -- 172.21.15.83:0/641990818 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3944030810 con 0x7f3954104f30 2024-08-19T01:26:00.859 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:59.978+0000 7f395ac00640 1 -- 172.21.15.83:0/641990818 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3954104f30 msgr2=0x7f3954105ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:26:00.859 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:59.978+0000 7f395ac00640 1 --2- 172.21.15.83:0/641990818 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3954104f30 0x7f3954105ba0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f3944009820 tx=0x7f394402a270 comp rx=0 tx=0).stop 2024-08-19T01:26:00.859 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:59.978+0000 7f395ac00640 1 -- 172.21.15.83:0/641990818 shutdown_connections 2024-08-19T01:26:00.859 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:59.978+0000 7f395ac00640 1 --2- 172.21.15.83:0/641990818 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3954104f30 0x7f3954105ba0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:26:00.859 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:59.978+0000 7f395ac00640 1 -- 172.21.15.83:0/641990818 >> 172.21.15.83:0/641990818 conn(0x7f3954100e40 msgr2=0x7f3954103260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:26:00.859 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:59.978+0000 7f395ac00640 1 -- 172.21.15.83:0/641990818 shutdown_connections 2024-08-19T01:26:00.859 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:59.978+0000 7f395ac00640 1 -- 172.21.15.83:0/641990818 wait complete. 2024-08-19T01:26:00.859 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:59.979+0000 7f395ac00640 1 Processor -- start 2024-08-19T01:26:00.859 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:59.979+0000 7f395ac00640 1 -- start start 2024-08-19T01:26:00.859 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:59.979+0000 7f395ac00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3954104f30 0x7f395419b4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:26:00.859 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:59.979+0000 7f395ac00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f3954107500 con 0x7f3954104f30 2024-08-19T01:26:00.859 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:59.980+0000 7f3953e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3954104f30 0x7f395419b4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:26:00.859 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:59.980+0000 7f3953e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3954104f30 0x7f395419b4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:42946/0 (socket says 172.21.15.83:42946) 2024-08-19T01:26:00.859 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:59.980+0000 7f3953e00640 1 -- 172.21.15.83:0/2272625585 learned_addr learned my addr 172.21.15.83:0/2272625585 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-08-19T01:26:00.860 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:59.980+0000 7f3953e00640 1 -- 172.21.15.83:0/2272625585 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f395419b9e0 con 0x7f3954104f30 2024-08-19T01:26:00.860 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:59.980+0000 7f3953e00640 1 --2- 172.21.15.83:0/2272625585 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3954104f30 0x7f395419b4a0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f3944030560 tx=0x7f3944032670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:26:00.860 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:59.981+0000 7f3950c00640 1 -- 172.21.15.83:0/2272625585 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3944037040 con 0x7f3954104f30 2024-08-19T01:26:00.860 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:59.981+0000 7f3950c00640 1 -- 172.21.15.83:0/2272625585 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3944040070 con 0x7f3954104f30 2024-08-19T01:26:00.860 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:59.981+0000 7f395ac00640 1 -- 172.21.15.83:0/2272625585 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f395419bc70 con 0x7f3954104f30 2024-08-19T01:26:00.860 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:59.981+0000 7f3950c00640 1 -- 172.21.15.83:0/2272625585 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f394403a6d0 con 0x7f3954104f30 2024-08-19T01:26:00.860 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:59.981+0000 7f395ac00640 1 -- 172.21.15.83:0/2272625585 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f395419e960 con 0x7f3954104f30 2024-08-19T01:26:00.860 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:59.982+0000 7f395ac00640 1 -- 172.21.15.83:0/2272625585 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3954105ba0 con 0x7f3954104f30 2024-08-19T01:26:00.860 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:59.982+0000 7f3950c00640 1 -- 172.21.15.83:0/2272625585 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 9) ==== 46300+0+0 (secure 0 0 0) 0x7f394403a940 con 0x7f3954104f30 2024-08-19T01:26:00.860 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:59.983+0000 7f3950c00640 1 --2- 172.21.15.83:0/2272625585 >> [v2:172.21.15.83:6800/573907319,v1:172.21.15.83:6801/573907319] conn(0x7f3938038fc0 0x7f393803b480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:26:00.860 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:59.983+0000 7f3950c00640 1 -- 172.21.15.83:0/2272625585 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f394406c8c0 con 0x7f3954104f30 2024-08-19T01:26:00.860 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:59.983+0000 7f3953400640 1 -- 172.21.15.83:0/2272625585 >> [v2:172.21.15.83:6800/573907319,v1:172.21.15.83:6801/573907319] conn(0x7f3938038fc0 msgr2=0x7f393803b480 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.83:6800/573907319 2024-08-19T01:26:00.861 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:59.983+0000 7f3953400640 1 --2- 172.21.15.83:0/2272625585 >> [v2:172.21.15.83:6800/573907319,v1:172.21.15.83:6801/573907319] conn(0x7f3938038fc0 0x7f393803b480 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-08-19T01:26:00.861 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:25:59.988+0000 7f3950c00640 1 -- 172.21.15.83:0/2272625585 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7f3944044c70 con 0x7f3954104f30 2024-08-19T01:26:00.861 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:00.184+0000 7f3953400640 1 -- 172.21.15.83:0/2272625585 >> [v2:172.21.15.83:6800/573907319,v1:172.21.15.83:6801/573907319] conn(0x7f3938038fc0 msgr2=0x7f393803b480 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.83:6800/573907319 2024-08-19T01:26:00.861 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:00.184+0000 7f3953400640 1 --2- 172.21.15.83:0/2272625585 >> [v2:172.21.15.83:6800/573907319,v1:172.21.15.83:6801/573907319] conn(0x7f3938038fc0 0x7f393803b480 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-08-19T01:26:00.861 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:00.214+0000 7f395ac00640 1 -- 172.21.15.83:0/2272625585 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) -- 0x7f395402d7c0 con 0x7f3954104f30 2024-08-19T01:26:00.861 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:00.214+0000 7f3950c00640 1 -- 172.21.15.83:0/2272625585 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v9) ==== 56+0+88 (secure 0 0 0) 0x7f39440328f0 con 0x7f3954104f30 2024-08-19T01:26:00.861 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:00.217+0000 7f395ac00640 1 -- 172.21.15.83:0/2272625585 >> [v2:172.21.15.83:6800/573907319,v1:172.21.15.83:6801/573907319] conn(0x7f3938038fc0 msgr2=0x7f393803b480 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-08-19T01:26:00.861 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:00.217+0000 7f395ac00640 1 --2- 172.21.15.83:0/2272625585 >> [v2:172.21.15.83:6800/573907319,v1:172.21.15.83:6801/573907319] conn(0x7f3938038fc0 0x7f393803b480 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:26:00.861 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:00.217+0000 7f395ac00640 1 -- 172.21.15.83:0/2272625585 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3954104f30 msgr2=0x7f395419b4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:26:00.861 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:00.217+0000 7f395ac00640 1 --2- 172.21.15.83:0/2272625585 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3954104f30 0x7f395419b4a0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f3944030560 tx=0x7f3944032670 comp rx=0 tx=0).stop 2024-08-19T01:26:00.861 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:00.217+0000 7f395ac00640 1 -- 172.21.15.83:0/2272625585 shutdown_connections 2024-08-19T01:26:00.862 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:00.217+0000 7f395ac00640 1 --2- 172.21.15.83:0/2272625585 >> [v2:172.21.15.83:6800/573907319,v1:172.21.15.83:6801/573907319] conn(0x7f3938038fc0 0x7f393803b480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:26:00.862 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:00.217+0000 7f395ac00640 1 --2- 172.21.15.83:0/2272625585 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3954104f30 0x7f395419b4a0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:26:00.862 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:00.217+0000 7f395ac00640 1 -- 172.21.15.83:0/2272625585 >> 172.21.15.83:0/2272625585 conn(0x7f3954100e40 msgr2=0x7f3954192120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:26:00.862 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:00.217+0000 7f395ac00640 1 -- 172.21.15.83:0/2272625585 shutdown_connections 2024-08-19T01:26:00.862 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:00.217+0000 7f395ac00640 1 -- 172.21.15.83:0/2272625585 wait complete. 2024-08-19T01:26:00.862 INFO:teuthology.orchestra.run.smithi083.stdout:Waiting for the mgr to restart... 2024-08-19T01:26:00.862 INFO:teuthology.orchestra.run.smithi083.stdout:Waiting for mgr epoch 9... 2024-08-19T01:26:01.039 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Aug 19 01:26:00 smithi083 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-mgr-x[17344]: 2024-08-19T01:26:00.759+0000 7f03153dc040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-08-19T01:26:01.039 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Aug 19 01:26:00 smithi083 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-mgr-x[17344]: 2024-08-19T01:26:00.834+0000 7f03153dc040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-08-19T01:26:01.039 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Aug 19 01:26:00 smithi083 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-mgr-x[17344]: 2024-08-19T01:26:00.912+0000 7f03153dc040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-08-19T01:26:01.039 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Aug 19 01:26:00 smithi083 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-mgr-x[17344]: 2024-08-19T01:26:00.994+0000 7f03153dc040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-08-19T01:26:01.292 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Aug 19 01:26:01 smithi083 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-mgr-x[17344]: 2024-08-19T01:26:01.248+0000 7f03153dc040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-08-19T01:26:01.609 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Aug 19 01:26:01 smithi083 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-mgr-x[17344]: 2024-08-19T01:26:01.321+0000 7f03153dc040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-08-19T01:26:02.333 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Aug 19 01:26:01 smithi083 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-mgr-x[17344]: 2024-08-19T01:26:01.978+0000 7f03153dc040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-08-19T01:26:02.333 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Aug 19 01:26:02 smithi083 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-mgr-x[17344]: 2024-08-19T01:26:02.330+0000 7f03153dc040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-08-19T01:26:03.155 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Aug 19 01:26:02 smithi083 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-mgr-x[17344]: 2024-08-19T01:26:02.839+0000 7f03153dc040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-08-19T01:26:03.155 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Aug 19 01:26:02 smithi083 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-mgr-x[17344]: 2024-08-19T01:26:02.905+0000 7f03153dc040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-08-19T01:26:03.155 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Aug 19 01:26:02 smithi083 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-mgr-x[17344]: 2024-08-19T01:26:02.975+0000 7f03153dc040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-08-19T01:26:04.655 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:26:04 smithi083 ceph-mon[17120]: Active manager daemon x restarted 2024-08-19T01:26:04.655 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:26:04 smithi083 ceph-mon[17120]: Activating manager daemon x 2024-08-19T01:26:04.655 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:26:04 smithi083 ceph-mon[17120]: osdmap e3: 0 total, 0 up, 0 in 2024-08-19T01:26:04.655 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:26:04 smithi083 ceph-mon[17120]: mgrmap e10: x(active, starting, since 0.0037177s) 2024-08-19T01:26:04.656 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:26:04 smithi083 ceph-mon[17120]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-08-19T01:26:04.656 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:26:04 smithi083 ceph-mon[17120]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-08-19T01:26:04.656 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:26:04 smithi083 ceph-mon[17120]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-08-19T01:26:04.656 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:26:04 smithi083 ceph-mon[17120]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-08-19T01:26:04.656 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:26:04 smithi083 ceph-mon[17120]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-08-19T01:26:04.656 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:26:04 smithi083 ceph-mon[17120]: Manager daemon x is now available 2024-08-19T01:26:04.656 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:26:04 smithi083 ceph-mon[17120]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-19T01:26:05.583 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:26:05 smithi083 ceph-mon[17120]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-08-19T01:26:05.583 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:26:05 smithi083 ceph-mon[17120]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-08-19T01:26:05.583 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:26:05 smithi083 ceph-mon[17120]: mgrmap e11: x(active, since 1.0078s) 2024-08-19T01:26:05.855 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Aug 19 01:26:05 smithi083 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-mgr-x[17344]: 2024-08-19T01:26:05.580+0000 7f026b000640 -1 log_channel(cephadm) log [ERR] : [19/Aug/2024:01:26:05] ENGINE Error in HTTPServer.serve 2024-08-19T01:26:05.855 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Aug 19 01:26:05 smithi083 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-mgr-x[17344]: Traceback (most recent call last): 2024-08-19T01:26:05.855 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Aug 19 01:26:05 smithi083 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-mgr-x[17344]: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-08-19T01:26:05.855 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Aug 19 01:26:05 smithi083 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-mgr-x[17344]: self._connections.run(self.expiration_interval) 2024-08-19T01:26:05.855 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Aug 19 01:26:05 smithi083 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-mgr-x[17344]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-08-19T01:26:05.855 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Aug 19 01:26:05 smithi083 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-mgr-x[17344]: self._run(expiration_interval) 2024-08-19T01:26:05.855 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Aug 19 01:26:05 smithi083 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-mgr-x[17344]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-08-19T01:26:05.855 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Aug 19 01:26:05 smithi083 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-mgr-x[17344]: new_conn = self._from_server_socket(self.server.socket) 2024-08-19T01:26:05.855 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Aug 19 01:26:05 smithi083 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-mgr-x[17344]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-08-19T01:26:05.855 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Aug 19 01:26:05 smithi083 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-mgr-x[17344]: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-08-19T01:26:05.856 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Aug 19 01:26:05 smithi083 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-mgr-x[17344]: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-08-19T01:26:05.856 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Aug 19 01:26:05 smithi083 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-mgr-x[17344]: s = self.context.wrap_socket( 2024-08-19T01:26:05.856 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Aug 19 01:26:05 smithi083 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-mgr-x[17344]: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-08-19T01:26:05.856 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Aug 19 01:26:05 smithi083 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-mgr-x[17344]: return self.sslsocket_class._create( 2024-08-19T01:26:05.856 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Aug 19 01:26:05 smithi083 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-mgr-x[17344]: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-08-19T01:26:05.856 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Aug 19 01:26:05 smithi083 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-mgr-x[17344]: self.do_handshake() 2024-08-19T01:26:05.856 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Aug 19 01:26:05 smithi083 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-mgr-x[17344]: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-08-19T01:26:05.856 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Aug 19 01:26:05 smithi083 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-mgr-x[17344]: self._sslobj.do_handshake() 2024-08-19T01:26:05.856 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Aug 19 01:26:05 smithi083 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-mgr-x[17344]: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-08-19T01:26:05.856 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Aug 19 01:26:05 smithi083 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-mgr-x[17344]: 2024-08-19T01:26:06.491 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:26:06 smithi083 ceph-mon[17120]: [19/Aug/2024:01:26:05] ENGINE Bus STARTING 2024-08-19T01:26:06.491 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:26:06 smithi083 ceph-mon[17120]: [19/Aug/2024:01:26:05] ENGINE Serving on http://172.21.15.83:8765 2024-08-19T01:26:06.491 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:26:06 smithi083 ceph-mon[17120]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' 2024-08-19T01:26:06.506 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout { 2024-08-19T01:26:06.506 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 11, 2024-08-19T01:26:06.506 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "initialized": true 2024-08-19T01:26:06.506 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout } 2024-08-19T01:26:06.506 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:01.398+0000 7fecf6400640 1 Processor -- start 2024-08-19T01:26:06.506 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:01.398+0000 7fecf6400640 1 -- start start 2024-08-19T01:26:06.506 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:01.398+0000 7fecf6400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fecf0109840 0x7fecf0109c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:26:06.507 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:01.399+0000 7fecf6400640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fecf010a210 con 0x7fecf0109840 2024-08-19T01:26:06.507 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:01.399+0000 7fecef400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fecf0109840 0x7fecf0109c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:26:06.507 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:01.399+0000 7fecef400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fecf0109840 0x7fecf0109c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:55468/0 (socket says 172.21.15.83:55468) 2024-08-19T01:26:06.507 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:01.399+0000 7fecef400640 1 -- 172.21.15.83:0/3181772119 learned_addr learned my addr 172.21.15.83:0/3181772119 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-08-19T01:26:06.507 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:01.399+0000 7fecef400640 1 -- 172.21.15.83:0/3181772119 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fecf010a390 con 0x7fecf0109840 2024-08-19T01:26:06.507 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:01.400+0000 7fecef400640 1 --2- 172.21.15.83:0/3181772119 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fecf0109840 0x7fecf0109c40 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fece0009820 tx=0x7fece002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e65fa8f08b1da576 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:26:06.507 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:01.401+0000 7fecee000640 1 -- 172.21.15.83:0/3181772119 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fece0037070 con 0x7fecf0109840 2024-08-19T01:26:06.507 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:01.401+0000 7fecee000640 1 -- 172.21.15.83:0/3181772119 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fece0032920 con 0x7fecf0109840 2024-08-19T01:26:06.507 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:01.401+0000 7fecee000640 1 -- 172.21.15.83:0/3181772119 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fece0030810 con 0x7fecf0109840 2024-08-19T01:26:06.507 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:01.401+0000 7fecf6400640 1 -- 172.21.15.83:0/3181772119 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fecf0109840 msgr2=0x7fecf0109c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:26:06.507 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:01.401+0000 7fecf6400640 1 --2- 172.21.15.83:0/3181772119 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fecf0109840 0x7fecf0109c40 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fece0009820 tx=0x7fece002a270 comp rx=0 tx=0).stop 2024-08-19T01:26:06.507 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:01.401+0000 7fecf6400640 1 -- 172.21.15.83:0/3181772119 shutdown_connections 2024-08-19T01:26:06.507 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:01.401+0000 7fecf6400640 1 --2- 172.21.15.83:0/3181772119 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fecf0109840 0x7fecf0109c40 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:26:06.508 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:01.401+0000 7fecf6400640 1 -- 172.21.15.83:0/3181772119 >> 172.21.15.83:0/3181772119 conn(0x7fecf007cc50 msgr2=0x7fecf007d060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:26:06.508 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:01.402+0000 7fecf6400640 1 -- 172.21.15.83:0/3181772119 shutdown_connections 2024-08-19T01:26:06.508 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:01.402+0000 7fecf6400640 1 -- 172.21.15.83:0/3181772119 wait complete. 2024-08-19T01:26:06.508 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:01.402+0000 7fecf6400640 1 Processor -- start 2024-08-19T01:26:06.508 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:01.402+0000 7fecf6400640 1 -- start start 2024-08-19T01:26:06.508 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:01.403+0000 7fecf6400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fecf0109840 0x7fecf019d640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:26:06.508 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:01.403+0000 7fecf6400640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fecf010dac0 con 0x7fecf0109840 2024-08-19T01:26:06.508 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:01.403+0000 7fecef400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fecf0109840 0x7fecf019d640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:26:06.508 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:01.403+0000 7fecef400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fecf0109840 0x7fecf019d640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:55472/0 (socket says 172.21.15.83:55472) 2024-08-19T01:26:06.508 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:01.403+0000 7fecef400640 1 -- 172.21.15.83:0/1849740715 learned_addr learned my addr 172.21.15.83:0/1849740715 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-08-19T01:26:06.508 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:01.403+0000 7fecef400640 1 -- 172.21.15.83:0/1849740715 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fecf019db80 con 0x7fecf0109840 2024-08-19T01:26:06.508 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:01.404+0000 7fecef400640 1 --2- 172.21.15.83:0/1849740715 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fecf0109840 0x7fecf019d640 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fece00097f0 tx=0x7fece0030d40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:26:06.508 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:01.404+0000 7fece7e00640 1 -- 172.21.15.83:0/1849740715 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fece0037040 con 0x7fecf0109840 2024-08-19T01:26:06.508 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:01.404+0000 7fece7e00640 1 -- 172.21.15.83:0/1849740715 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fece0040070 con 0x7fecf0109840 2024-08-19T01:26:06.508 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:01.404+0000 7fece7e00640 1 -- 172.21.15.83:0/1849740715 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fece003a610 con 0x7fecf0109840 2024-08-19T01:26:06.509 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:01.404+0000 7fecf6400640 1 -- 172.21.15.83:0/1849740715 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fecf019de10 con 0x7fecf0109840 2024-08-19T01:26:06.509 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:01.404+0000 7fecf6400640 1 -- 172.21.15.83:0/1849740715 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fecf01a0b00 con 0x7fecf0109840 2024-08-19T01:26:06.509 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:01.405+0000 7fece7e00640 1 -- 172.21.15.83:0/1849740715 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 9) ==== 46300+0+0 (secure 0 0 0) 0x7fece00327b0 con 0x7fecf0109840 2024-08-19T01:26:06.509 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:01.405+0000 7fece7e00640 1 --2- 172.21.15.83:0/1849740715 >> [v2:172.21.15.83:6800/573907319,v1:172.21.15.83:6801/573907319] conn(0x7fecd4038fc0 0x7fecd403b480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:26:06.509 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:01.405+0000 7fece7e00640 1 -- 172.21.15.83:0/1849740715 --> [v2:172.21.15.83:6800/573907319,v1:172.21.15.83:6801/573907319] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7fecd403bb90 con 0x7fecd4038fc0 2024-08-19T01:26:06.509 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:01.406+0000 7fece7e00640 1 -- 172.21.15.83:0/1849740715 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fece006c680 con 0x7fecf0109840 2024-08-19T01:26:06.509 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:01.406+0000 7feceea00640 1 -- 172.21.15.83:0/1849740715 >> [v2:172.21.15.83:6800/573907319,v1:172.21.15.83:6801/573907319] conn(0x7fecd4038fc0 msgr2=0x7fecd403b480 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.83:6800/573907319 2024-08-19T01:26:06.509 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:01.406+0000 7feceea00640 1 --2- 172.21.15.83:0/1849740715 >> [v2:172.21.15.83:6800/573907319,v1:172.21.15.83:6801/573907319] conn(0x7fecd4038fc0 0x7fecd403b480 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-08-19T01:26:06.509 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:01.607+0000 7feceea00640 1 -- 172.21.15.83:0/1849740715 >> [v2:172.21.15.83:6800/573907319,v1:172.21.15.83:6801/573907319] conn(0x7fecd4038fc0 msgr2=0x7fecd403b480 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.83:6800/573907319 2024-08-19T01:26:06.509 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:01.607+0000 7feceea00640 1 --2- 172.21.15.83:0/1849740715 >> [v2:172.21.15.83:6800/573907319,v1:172.21.15.83:6801/573907319] conn(0x7fecd4038fc0 0x7fecd403b480 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-08-19T01:26:06.509 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:02.008+0000 7feceea00640 1 -- 172.21.15.83:0/1849740715 >> [v2:172.21.15.83:6800/573907319,v1:172.21.15.83:6801/573907319] conn(0x7fecd4038fc0 msgr2=0x7fecd403b480 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.83:6800/573907319 2024-08-19T01:26:06.509 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:02.008+0000 7feceea00640 1 --2- 172.21.15.83:0/1849740715 >> [v2:172.21.15.83:6800/573907319,v1:172.21.15.83:6801/573907319] conn(0x7fecd4038fc0 0x7fecd403b480 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-08-19T01:26:06.509 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:02.809+0000 7feceea00640 1 -- 172.21.15.83:0/1849740715 >> [v2:172.21.15.83:6800/573907319,v1:172.21.15.83:6801/573907319] conn(0x7fecd4038fc0 msgr2=0x7fecd403b480 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.83:6800/573907319 2024-08-19T01:26:06.509 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:02.809+0000 7feceea00640 1 --2- 172.21.15.83:0/1849740715 >> [v2:172.21.15.83:6800/573907319,v1:172.21.15.83:6801/573907319] conn(0x7fecd4038fc0 0x7fecd403b480 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-08-19T01:26:06.510 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:04.279+0000 7fece7e00640 1 -- 172.21.15.83:0/1849740715 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mgrmap(e 10) ==== 46067+0+0 (secure 0 0 0) 0x7fece006b130 con 0x7fecf0109840 2024-08-19T01:26:06.510 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:04.279+0000 7fece7e00640 1 -- 172.21.15.83:0/1849740715 >> [v2:172.21.15.83:6800/573907319,v1:172.21.15.83:6801/573907319] conn(0x7fecd4038fc0 msgr2=0x7fecd403b480 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-08-19T01:26:06.510 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:04.279+0000 7fece7e00640 1 --2- 172.21.15.83:0/1849740715 >> [v2:172.21.15.83:6800/573907319,v1:172.21.15.83:6801/573907319] conn(0x7fecd4038fc0 0x7fecd403b480 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:26:06.510 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:05.281+0000 7fece7e00640 1 -- 172.21.15.83:0/1849740715 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mgrmap(e 11) ==== 46194+0+0 (secure 0 0 0) 0x7fece003b420 con 0x7fecf0109840 2024-08-19T01:26:06.510 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:05.281+0000 7fece7e00640 1 --2- 172.21.15.83:0/1849740715 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7fecd403ca80 0x7fecd403ee70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:26:06.510 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:05.281+0000 7fece7e00640 1 -- 172.21.15.83:0/1849740715 --> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7fece00657a0 con 0x7fecd403ca80 2024-08-19T01:26:06.510 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:05.282+0000 7feceea00640 1 --2- 172.21.15.83:0/1849740715 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7fecd403ca80 0x7fecd403ee70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:26:06.510 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:05.284+0000 7feceea00640 1 --2- 172.21.15.83:0/1849740715 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7fecd403ca80 0x7fecd403ee70 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7fecdc003980 tx=0x7fecdc006e10 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:26:06.510 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:05.285+0000 7fece7e00640 1 -- 172.21.15.83:0/1849740715 <== mgr.14150 v2:172.21.15.83:6800/683740916 1 ==== command_reply(tid 0: 0 ) ==== 8+0+9564 (secure 0 0 0) 0x7fece00657a0 con 0x7fecd403ca80 2024-08-19T01:26:06.510 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:05.292+0000 7fecf6400640 1 -- 172.21.15.83:0/1849740715 --> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] -- command(tid 1: {"prefix": "mgr_status"}) -- 0x7fecf0109cc0 con 0x7fecd403ca80 2024-08-19T01:26:06.510 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:05.293+0000 7fece7e00640 1 -- 172.21.15.83:0/1849740715 <== mgr.14150 v2:172.21.15.83:6800/683740916 2 ==== command_reply(tid 1: 0 ) ==== 8+0+52 (secure 0 0 0) 0x7fecf0109cc0 con 0x7fecd403ca80 2024-08-19T01:26:06.510 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:05.293+0000 7fecf6400640 1 -- 172.21.15.83:0/1849740715 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7fecd403ca80 msgr2=0x7fecd403ee70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:26:06.510 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:05.293+0000 7fecf6400640 1 --2- 172.21.15.83:0/1849740715 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7fecd403ca80 0x7fecd403ee70 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7fecdc003980 tx=0x7fecdc006e10 comp rx=0 tx=0).stop 2024-08-19T01:26:06.510 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:05.293+0000 7fecf6400640 1 -- 172.21.15.83:0/1849740715 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fecf0109840 msgr2=0x7fecf019d640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:26:06.510 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:05.293+0000 7fecf6400640 1 --2- 172.21.15.83:0/1849740715 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fecf0109840 0x7fecf019d640 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fece00097f0 tx=0x7fece0030d40 comp rx=0 tx=0).stop 2024-08-19T01:26:06.511 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:05.293+0000 7fecf6400640 1 -- 172.21.15.83:0/1849740715 shutdown_connections 2024-08-19T01:26:06.511 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:05.293+0000 7fecf6400640 1 --2- 172.21.15.83:0/1849740715 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7fecd403ca80 0x7fecd403ee70 unknown :-1 s=CLOSED pgs=3 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:26:06.511 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:05.293+0000 7fecf6400640 1 --2- 172.21.15.83:0/1849740715 >> [v2:172.21.15.83:6800/573907319,v1:172.21.15.83:6801/573907319] conn(0x7fecd4038fc0 0x7fecd403b480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:26:06.511 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:05.293+0000 7fecf6400640 1 --2- 172.21.15.83:0/1849740715 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fecf0109840 0x7fecf019d640 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:26:06.511 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:05.293+0000 7fecf6400640 1 -- 172.21.15.83:0/1849740715 >> 172.21.15.83:0/1849740715 conn(0x7fecf007cc50 msgr2=0x7fecf0107500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:26:06.511 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:05.293+0000 7fecf6400640 1 -- 172.21.15.83:0/1849740715 shutdown_connections 2024-08-19T01:26:06.511 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:05.293+0000 7fecf6400640 1 -- 172.21.15.83:0/1849740715 wait complete. 2024-08-19T01:26:06.511 INFO:teuthology.orchestra.run.smithi083.stdout:mgr epoch 9 is available 2024-08-19T01:26:06.511 INFO:teuthology.orchestra.run.smithi083.stdout:Generating a dashboard self-signed certificate... 2024-08-19T01:26:07.655 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:26:07 smithi083 ceph-mon[17120]: [19/Aug/2024:01:26:05] ENGINE Serving on https://172.21.15.83:7150 2024-08-19T01:26:07.655 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:26:07 smithi083 ceph-mon[17120]: [19/Aug/2024:01:26:05] ENGINE Bus STARTED 2024-08-19T01:26:07.655 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:26:07 smithi083 ceph-mon[17120]: [19/Aug/2024:01:26:05] ENGINE Error in HTTPServer.serve 2024-08-19T01:26:07.655 INFO:journalctl@ceph.mon.a.smithi083.stdout: Traceback (most recent call last): 2024-08-19T01:26:07.655 INFO:journalctl@ceph.mon.a.smithi083.stdout: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-08-19T01:26:07.655 INFO:journalctl@ceph.mon.a.smithi083.stdout: self._connections.run(self.expiration_interval) 2024-08-19T01:26:07.656 INFO:journalctl@ceph.mon.a.smithi083.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-08-19T01:26:07.656 INFO:journalctl@ceph.mon.a.smithi083.stdout: self._run(expiration_interval) 2024-08-19T01:26:07.656 INFO:journalctl@ceph.mon.a.smithi083.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-08-19T01:26:07.656 INFO:journalctl@ceph.mon.a.smithi083.stdout: new_conn = self._from_server_socket(self.server.socket) 2024-08-19T01:26:07.656 INFO:journalctl@ceph.mon.a.smithi083.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-08-19T01:26:07.656 INFO:journalctl@ceph.mon.a.smithi083.stdout: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-08-19T01:26:07.656 INFO:journalctl@ceph.mon.a.smithi083.stdout: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-08-19T01:26:07.656 INFO:journalctl@ceph.mon.a.smithi083.stdout: s = self.context.wrap_socket( 2024-08-19T01:26:07.656 INFO:journalctl@ceph.mon.a.smithi083.stdout: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-08-19T01:26:07.656 INFO:journalctl@ceph.mon.a.smithi083.stdout: return self.sslsocket_class._create( 2024-08-19T01:26:07.656 INFO:journalctl@ceph.mon.a.smithi083.stdout: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-08-19T01:26:07.656 INFO:journalctl@ceph.mon.a.smithi083.stdout: self.do_handshake() 2024-08-19T01:26:07.657 INFO:journalctl@ceph.mon.a.smithi083.stdout: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-08-19T01:26:07.657 INFO:journalctl@ceph.mon.a.smithi083.stdout: self._sslobj.do_handshake() 2024-08-19T01:26:07.657 INFO:journalctl@ceph.mon.a.smithi083.stdout: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-08-19T01:26:07.657 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:26:07 smithi083 ceph-mon[17120]: mgrmap e12: x(active, since 2s) 2024-08-19T01:26:07.657 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:26:07 smithi083 ceph-mon[17120]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' 2024-08-19T01:26:07.657 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:26:07 smithi083 ceph-mon[17120]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' 2024-08-19T01:26:07.931 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout Self-signed certificate created 2024-08-19T01:26:07.932 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:07.026+0000 7fe77be00640 1 Processor -- start 2024-08-19T01:26:07.932 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:07.026+0000 7fe77be00640 1 -- start start 2024-08-19T01:26:07.932 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:07.026+0000 7fe77be00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe77c107240 0x7fe77c107640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:26:07.932 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:07.026+0000 7fe77be00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fe77c107c10 con 0x7fe77c107240 2024-08-19T01:26:07.932 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:07.027+0000 7fe77aa00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe77c107240 0x7fe77c107640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:26:07.932 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:07.027+0000 7fe77aa00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe77c107240 0x7fe77c107640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:55552/0 (socket says 172.21.15.83:55552) 2024-08-19T01:26:07.932 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:07.027+0000 7fe77aa00640 1 -- 172.21.15.83:0/3114237743 learned_addr learned my addr 172.21.15.83:0/3114237743 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-08-19T01:26:07.932 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:07.027+0000 7fe77aa00640 1 -- 172.21.15.83:0/3114237743 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe77c108440 con 0x7fe77c107240 2024-08-19T01:26:07.932 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:07.028+0000 7fe77aa00640 1 --2- 172.21.15.83:0/3114237743 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe77c107240 0x7fe77c107640 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fe76c009820 tx=0x7fe76c02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d76a0f120a61fea7 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:26:07.937 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:07.029+0000 7fe779600640 1 -- 172.21.15.83:0/3114237743 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe76c037070 con 0x7fe77c107240 2024-08-19T01:26:07.937 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:07.029+0000 7fe779600640 1 -- 172.21.15.83:0/3114237743 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe76c032920 con 0x7fe77c107240 2024-08-19T01:26:07.938 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:07.029+0000 7fe779600640 1 -- 172.21.15.83:0/3114237743 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe76c030810 con 0x7fe77c107240 2024-08-19T01:26:07.938 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:07.029+0000 7fe77be00640 1 -- 172.21.15.83:0/3114237743 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe77c107240 msgr2=0x7fe77c107640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:26:07.938 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:07.029+0000 7fe77be00640 1 --2- 172.21.15.83:0/3114237743 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe77c107240 0x7fe77c107640 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fe76c009820 tx=0x7fe76c02a270 comp rx=0 tx=0).stop 2024-08-19T01:26:07.938 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:07.029+0000 7fe77be00640 1 -- 172.21.15.83:0/3114237743 shutdown_connections 2024-08-19T01:26:07.938 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:07.029+0000 7fe77be00640 1 --2- 172.21.15.83:0/3114237743 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe77c107240 0x7fe77c107640 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:26:07.938 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:07.029+0000 7fe77be00640 1 -- 172.21.15.83:0/3114237743 >> 172.21.15.83:0/3114237743 conn(0x7fe77c1029b0 msgr2=0x7fe77c104e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:26:07.938 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:07.029+0000 7fe77be00640 1 -- 172.21.15.83:0/3114237743 shutdown_connections 2024-08-19T01:26:07.938 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:07.029+0000 7fe77be00640 1 -- 172.21.15.83:0/3114237743 wait complete. 2024-08-19T01:26:07.939 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:07.030+0000 7fe77be00640 1 Processor -- start 2024-08-19T01:26:07.939 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:07.030+0000 7fe77be00640 1 -- start start 2024-08-19T01:26:07.939 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:07.031+0000 7fe77be00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe77c107240 0x7fe77c19b230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:26:07.939 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:07.031+0000 7fe77be00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fe77c10b7f0 con 0x7fe77c107240 2024-08-19T01:26:07.939 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:07.031+0000 7fe77aa00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe77c107240 0x7fe77c19b230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:26:07.939 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:07.031+0000 7fe77aa00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe77c107240 0x7fe77c19b230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:55560/0 (socket says 172.21.15.83:55560) 2024-08-19T01:26:07.939 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:07.031+0000 7fe77aa00640 1 -- 172.21.15.83:0/1688385172 learned_addr learned my addr 172.21.15.83:0/1688385172 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-08-19T01:26:07.939 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:07.031+0000 7fe77aa00640 1 -- 172.21.15.83:0/1688385172 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe77c19b770 con 0x7fe77c107240 2024-08-19T01:26:07.940 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:07.032+0000 7fe77aa00640 1 --2- 172.21.15.83:0/1688385172 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe77c107240 0x7fe77c19b230 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fe76c0097f0 tx=0x7fe76c0327b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:26:07.940 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:07.032+0000 7fe773400640 1 -- 172.21.15.83:0/1688385172 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe76c037040 con 0x7fe77c107240 2024-08-19T01:26:07.940 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:07.032+0000 7fe773400640 1 -- 172.21.15.83:0/1688385172 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe76c030cc0 con 0x7fe77c107240 2024-08-19T01:26:07.940 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:07.032+0000 7fe77be00640 1 -- 172.21.15.83:0/1688385172 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe77c19ba00 con 0x7fe77c107240 2024-08-19T01:26:07.940 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:07.032+0000 7fe773400640 1 -- 172.21.15.83:0/1688385172 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe76c03a640 con 0x7fe77c107240 2024-08-19T01:26:07.940 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:07.032+0000 7fe77be00640 1 -- 172.21.15.83:0/1688385172 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe77c19e6f0 con 0x7fe77c107240 2024-08-19T01:26:07.940 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:07.033+0000 7fe770c00640 1 -- 172.21.15.83:0/1688385172 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe754005290 con 0x7fe77c107240 2024-08-19T01:26:07.940 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:07.034+0000 7fe773400640 1 -- 172.21.15.83:0/1688385172 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 12) ==== 46300+0+0 (secure 0 0 0) 0x7fe76c03b3f0 con 0x7fe77c107240 2024-08-19T01:26:07.940 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:07.034+0000 7fe773400640 1 --2- 172.21.15.83:0/1688385172 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7fe760038fc0 0x7fe76003b480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:26:07.940 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:07.034+0000 7fe773400640 1 -- 172.21.15.83:0/1688385172 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fe76c06c710 con 0x7fe77c107240 2024-08-19T01:26:07.940 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:07.034+0000 7fe77a000640 1 --2- 172.21.15.83:0/1688385172 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7fe760038fc0 0x7fe76003b480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:26:07.940 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:07.035+0000 7fe77a000640 1 --2- 172.21.15.83:0/1688385172 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7fe760038fc0 0x7fe76003b480 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fe7680098c0 tx=0x7fe768006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:26:07.940 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:07.040+0000 7fe773400640 1 -- 172.21.15.83:0/1688385172 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7fe76c0313a0 con 0x7fe77c107240 2024-08-19T01:26:07.940 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:07.231+0000 7fe770c00640 1 -- 172.21.15.83:0/1688385172 --> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] -- mgr_command(tid 0: {"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}) -- 0x7fe754000bc0 con 0x7fe760038fc0 2024-08-19T01:26:07.941 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:07.306+0000 7fe773400640 1 -- 172.21.15.83:0/1688385172 <== mgr.14150 v2:172.21.15.83:6800/683740916 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+32 (secure 0 0 0) 0x7fe754000bc0 con 0x7fe760038fc0 2024-08-19T01:26:07.941 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:07.308+0000 7fe770c00640 1 -- 172.21.15.83:0/1688385172 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7fe760038fc0 msgr2=0x7fe76003b480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:26:07.941 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:07.308+0000 7fe770c00640 1 --2- 172.21.15.83:0/1688385172 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7fe760038fc0 0x7fe76003b480 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fe7680098c0 tx=0x7fe768006db0 comp rx=0 tx=0).stop 2024-08-19T01:26:07.941 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:07.308+0000 7fe770c00640 1 -- 172.21.15.83:0/1688385172 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe77c107240 msgr2=0x7fe77c19b230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:26:07.941 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:07.308+0000 7fe770c00640 1 --2- 172.21.15.83:0/1688385172 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe77c107240 0x7fe77c19b230 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fe76c0097f0 tx=0x7fe76c0327b0 comp rx=0 tx=0).stop 2024-08-19T01:26:07.941 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:07.308+0000 7fe770c00640 1 -- 172.21.15.83:0/1688385172 shutdown_connections 2024-08-19T01:26:07.941 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:07.308+0000 7fe770c00640 1 --2- 172.21.15.83:0/1688385172 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7fe760038fc0 0x7fe76003b480 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:26:07.941 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:07.308+0000 7fe770c00640 1 --2- 172.21.15.83:0/1688385172 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe77c107240 0x7fe77c19b230 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:26:07.941 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:07.308+0000 7fe770c00640 1 -- 172.21.15.83:0/1688385172 >> 172.21.15.83:0/1688385172 conn(0x7fe77c1029b0 msgr2=0x7fe77c103510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:26:07.941 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:07.309+0000 7fe770c00640 1 -- 172.21.15.83:0/1688385172 shutdown_connections 2024-08-19T01:26:07.941 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:07.309+0000 7fe770c00640 1 -- 172.21.15.83:0/1688385172 wait complete. 2024-08-19T01:26:07.941 INFO:teuthology.orchestra.run.smithi083.stdout:Creating initial admin user... 2024-08-19T01:26:08.426 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:26:08 smithi083 ceph-mon[17120]: from='client.14162 -' entity='client.admin' cmd=[{"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}]: dispatch 2024-08-19T01:26:08.426 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:26:08 smithi083 ceph-mon[17120]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' 2024-08-19T01:26:10.155 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:26:09 smithi083 ceph-mon[17120]: from='client.14164 -' entity='client.admin' cmd=[{"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}]: dispatch 2024-08-19T01:26:10.155 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:26:09 smithi083 ceph-mon[17120]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' 2024-08-19T01:26:10.337 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout {"username": "admin", "password": "$2b$12$SLiji3wCWUxswdXp2qYpeeN6oDpSejNPolsoBv0HInvTtPsCELc8G", "roles": ["administrator"], "name": null, "email": null, "lastUpdate": 1724030768, "enabled": true, "pwdExpirationDate": null, "pwdUpdateRequired": true} 2024-08-19T01:26:10.337 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:08.419+0000 7f2094c00640 1 Processor -- start 2024-08-19T01:26:10.337 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:08.419+0000 7f2094c00640 1 -- start start 2024-08-19T01:26:10.337 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:08.419+0000 7f2094c00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2090106c00 0x7f2090108ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:26:10.338 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:08.419+0000 7f2094c00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f2090109530 con 0x7f2090106c00 2024-08-19T01:26:10.338 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:08.420+0000 7f208de00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2090106c00 0x7f2090108ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:26:10.338 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:08.420+0000 7f208de00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2090106c00 0x7f2090108ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:55570/0 (socket says 172.21.15.83:55570) 2024-08-19T01:26:10.338 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:08.420+0000 7f208de00640 1 -- 172.21.15.83:0/1165499949 learned_addr learned my addr 172.21.15.83:0/1165499949 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-08-19T01:26:10.338 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:08.420+0000 7f208de00640 1 -- 172.21.15.83:0/1165499949 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f20901096b0 con 0x7f2090106c00 2024-08-19T01:26:10.339 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:08.421+0000 7f208de00640 1 --2- 172.21.15.83:0/1165499949 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2090106c00 0x7f2090108ff0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f2080009820 tx=0x7f208002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=67b6545b47e0a5ee server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:26:10.339 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:08.421+0000 7f208ca00640 1 -- 172.21.15.83:0/1165499949 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2080037070 con 0x7f2090106c00 2024-08-19T01:26:10.339 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:08.421+0000 7f208ca00640 1 -- 172.21.15.83:0/1165499949 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2080032920 con 0x7f2090106c00 2024-08-19T01:26:10.339 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:08.421+0000 7f208ca00640 1 -- 172.21.15.83:0/1165499949 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2080030810 con 0x7f2090106c00 2024-08-19T01:26:10.339 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:08.422+0000 7f2094c00640 1 -- 172.21.15.83:0/1165499949 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2090106c00 msgr2=0x7f2090108ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:26:10.339 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:08.422+0000 7f2094c00640 1 --2- 172.21.15.83:0/1165499949 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2090106c00 0x7f2090108ff0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f2080009820 tx=0x7f208002a270 comp rx=0 tx=0).stop 2024-08-19T01:26:10.340 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:08.422+0000 7f2094c00640 1 -- 172.21.15.83:0/1165499949 shutdown_connections 2024-08-19T01:26:10.340 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:08.422+0000 7f2094c00640 1 --2- 172.21.15.83:0/1165499949 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2090106c00 0x7f2090108ff0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:26:10.340 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:08.422+0000 7f2094c00640 1 -- 172.21.15.83:0/1165499949 >> 172.21.15.83:0/1165499949 conn(0x7f2090100e40 msgr2=0x7f2090103260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:26:10.340 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:08.423+0000 7f2094c00640 1 -- 172.21.15.83:0/1165499949 shutdown_connections 2024-08-19T01:26:10.340 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:08.423+0000 7f2094c00640 1 -- 172.21.15.83:0/1165499949 wait complete. 2024-08-19T01:26:10.341 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:08.424+0000 7f2094c00640 1 Processor -- start 2024-08-19T01:26:10.341 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:08.424+0000 7f2094c00640 1 -- start start 2024-08-19T01:26:10.341 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:08.424+0000 7f2094c00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2090106c00 0x7f209019d6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:26:10.341 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:08.424+0000 7f2094c00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f209010a720 con 0x7f2090106c00 2024-08-19T01:26:10.341 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:08.424+0000 7f208de00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2090106c00 0x7f209019d6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:26:10.342 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:08.424+0000 7f208de00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2090106c00 0x7f209019d6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:55572/0 (socket says 172.21.15.83:55572) 2024-08-19T01:26:10.342 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:08.424+0000 7f208de00640 1 -- 172.21.15.83:0/2514233591 learned_addr learned my addr 172.21.15.83:0/2514233591 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-08-19T01:26:10.342 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:08.425+0000 7f208de00640 1 -- 172.21.15.83:0/2514233591 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f209019dbe0 con 0x7f2090106c00 2024-08-19T01:26:10.342 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:08.425+0000 7f208de00640 1 --2- 172.21.15.83:0/2514233591 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2090106c00 0x7f209019d6a0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f20800097f0 tx=0x7f20800327b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:26:10.342 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:08.426+0000 7f2086a00640 1 -- 172.21.15.83:0/2514233591 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2080037040 con 0x7f2090106c00 2024-08-19T01:26:10.343 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:08.426+0000 7f2086a00640 1 -- 172.21.15.83:0/2514233591 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2080030c40 con 0x7f2090106c00 2024-08-19T01:26:10.343 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:08.426+0000 7f2094c00640 1 -- 172.21.15.83:0/2514233591 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f209019de70 con 0x7f2090106c00 2024-08-19T01:26:10.343 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:08.426+0000 7f2086a00640 1 -- 172.21.15.83:0/2514233591 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f208003a5b0 con 0x7f2090106c00 2024-08-19T01:26:10.343 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:08.426+0000 7f2094c00640 1 -- 172.21.15.83:0/2514233591 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f20901a0b60 con 0x7f2090106c00 2024-08-19T01:26:10.343 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:08.427+0000 7f2094c00640 1 -- 172.21.15.83:0/2514233591 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2090107060 con 0x7f2090106c00 2024-08-19T01:26:10.343 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:08.427+0000 7f2086a00640 1 -- 172.21.15.83:0/2514233591 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 12) ==== 46300+0+0 (secure 0 0 0) 0x7f208003a7a0 con 0x7f2090106c00 2024-08-19T01:26:10.344 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:08.428+0000 7f2086a00640 1 --2- 172.21.15.83:0/2514233591 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f2074039020 0x7f207403b4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:26:10.344 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:08.428+0000 7f2086a00640 1 -- 172.21.15.83:0/2514233591 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f208006c6b0 con 0x7f2090106c00 2024-08-19T01:26:10.344 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:08.428+0000 7f208d400640 1 --2- 172.21.15.83:0/2514233591 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f2074039020 0x7f207403b4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:26:10.344 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:08.429+0000 7f208d400640 1 --2- 172.21.15.83:0/2514233591 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f2074039020 0x7f207403b4e0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f207c0098c0 tx=0x7f207c006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:26:10.344 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:08.436+0000 7f2086a00640 1 -- 172.21.15.83:0/2514233591 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7f2080031c00 con 0x7f2090106c00 2024-08-19T01:26:10.345 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:08.633+0000 7f2094c00640 1 -- 172.21.15.83:0/2514233591 --> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] -- mgr_command(tid 0: {"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}) -- 0x7f209002d850 con 0x7f2074039020 2024-08-19T01:26:10.345 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:08.861+0000 7f2086a00640 1 -- 172.21.15.83:0/2514233591 <== mgr.14150 v2:172.21.15.83:6800/683740916 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+252 (secure 0 0 0) 0x7f209002d850 con 0x7f2074039020 2024-08-19T01:26:10.345 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:08.864+0000 7f2094c00640 1 -- 172.21.15.83:0/2514233591 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f2074039020 msgr2=0x7f207403b4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:26:10.345 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:08.864+0000 7f2094c00640 1 --2- 172.21.15.83:0/2514233591 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f2074039020 0x7f207403b4e0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f207c0098c0 tx=0x7f207c006db0 comp rx=0 tx=0).stop 2024-08-19T01:26:10.345 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:08.864+0000 7f2094c00640 1 -- 172.21.15.83:0/2514233591 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2090106c00 msgr2=0x7f209019d6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:26:10.345 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:08.864+0000 7f2094c00640 1 --2- 172.21.15.83:0/2514233591 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2090106c00 0x7f209019d6a0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f20800097f0 tx=0x7f20800327b0 comp rx=0 tx=0).stop 2024-08-19T01:26:10.346 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:08.864+0000 7f2094c00640 1 -- 172.21.15.83:0/2514233591 shutdown_connections 2024-08-19T01:26:10.346 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:08.864+0000 7f2094c00640 1 --2- 172.21.15.83:0/2514233591 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f2074039020 0x7f207403b4e0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:26:10.346 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:08.864+0000 7f2094c00640 1 --2- 172.21.15.83:0/2514233591 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2090106c00 0x7f209019d6a0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:26:10.346 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:08.864+0000 7f2094c00640 1 -- 172.21.15.83:0/2514233591 >> 172.21.15.83:0/2514233591 conn(0x7f2090100e40 msgr2=0x7f2090101d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:26:10.346 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:08.865+0000 7f2094c00640 1 -- 172.21.15.83:0/2514233591 shutdown_connections 2024-08-19T01:26:10.347 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:08.865+0000 7f2094c00640 1 -- 172.21.15.83:0/2514233591 wait complete. 2024-08-19T01:26:10.347 INFO:teuthology.orchestra.run.smithi083.stdout:Fetching dashboard port number... 2024-08-19T01:26:11.586 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout 8443 2024-08-19T01:26:11.586 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:10.793+0000 7f7a72000640 1 Processor -- start 2024-08-19T01:26:11.586 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:10.793+0000 7f7a72000640 1 -- start start 2024-08-19T01:26:11.587 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:10.793+0000 7f7a72000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7a6c10aac0 0x7f7a6c10aec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:26:11.587 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:10.793+0000 7f7a72000640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f7a6c10b490 con 0x7f7a6c10aac0 2024-08-19T01:26:11.587 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:10.793+0000 7f7a6b400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7a6c10aac0 0x7f7a6c10aec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:26:11.587 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:10.793+0000 7f7a6b400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7a6c10aac0 0x7f7a6c10aec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:55584/0 (socket says 172.21.15.83:55584) 2024-08-19T01:26:11.587 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:10.793+0000 7f7a6b400640 1 -- 172.21.15.83:0/1220638774 learned_addr learned my addr 172.21.15.83:0/1220638774 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-08-19T01:26:11.587 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:10.794+0000 7f7a6b400640 1 -- 172.21.15.83:0/1220638774 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7a6c10b610 con 0x7f7a6c10aac0 2024-08-19T01:26:11.587 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:10.795+0000 7f7a6b400640 1 --2- 172.21.15.83:0/1220638774 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7a6c10aac0 0x7f7a6c10aec0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f7a5c009e80 tx=0x7f7a5c02a7a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=93b3f3c747c5f138 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:26:11.587 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:10.795+0000 7f7a6aa00640 1 -- 172.21.15.83:0/1220638774 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7a5c036070 con 0x7f7a6c10aac0 2024-08-19T01:26:11.587 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:10.795+0000 7f7a6aa00640 1 -- 172.21.15.83:0/1220638774 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7a5c0318e0 con 0x7f7a6c10aac0 2024-08-19T01:26:11.587 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:10.795+0000 7f7a6aa00640 1 -- 172.21.15.83:0/1220638774 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7a5c007a00 con 0x7f7a6c10aac0 2024-08-19T01:26:11.587 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:10.796+0000 7f7a72000640 1 -- 172.21.15.83:0/1220638774 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7a6c10aac0 msgr2=0x7f7a6c10aec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:26:11.587 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:10.796+0000 7f7a72000640 1 --2- 172.21.15.83:0/1220638774 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7a6c10aac0 0x7f7a6c10aec0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f7a5c009e80 tx=0x7f7a5c02a7a0 comp rx=0 tx=0).stop 2024-08-19T01:26:11.587 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:10.796+0000 7f7a72000640 1 -- 172.21.15.83:0/1220638774 shutdown_connections 2024-08-19T01:26:11.588 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:10.796+0000 7f7a72000640 1 --2- 172.21.15.83:0/1220638774 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7a6c10aac0 0x7f7a6c10aec0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:26:11.588 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:10.796+0000 7f7a72000640 1 -- 172.21.15.83:0/1220638774 >> 172.21.15.83:0/1220638774 conn(0x7f7a6c07cc40 msgr2=0x7f7a6c07d050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:26:11.588 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:10.796+0000 7f7a72000640 1 -- 172.21.15.83:0/1220638774 shutdown_connections 2024-08-19T01:26:11.588 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:10.796+0000 7f7a72000640 1 -- 172.21.15.83:0/1220638774 wait complete. 2024-08-19T01:26:11.588 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:10.797+0000 7f7a72000640 1 Processor -- start 2024-08-19T01:26:11.588 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:10.797+0000 7f7a72000640 1 -- start start 2024-08-19T01:26:11.588 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:10.797+0000 7f7a72000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7a6c10aac0 0x7f7a6c19d6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:26:11.588 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:10.797+0000 7f7a72000640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f7a6c10db20 con 0x7f7a6c10aac0 2024-08-19T01:26:11.588 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:10.798+0000 7f7a6b400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7a6c10aac0 0x7f7a6c19d6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:26:11.588 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:10.798+0000 7f7a6b400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7a6c10aac0 0x7f7a6c19d6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:55598/0 (socket says 172.21.15.83:55598) 2024-08-19T01:26:11.588 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:10.798+0000 7f7a6b400640 1 -- 172.21.15.83:0/3085123757 learned_addr learned my addr 172.21.15.83:0/3085123757 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-08-19T01:26:11.588 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:10.798+0000 7f7a6b400640 1 -- 172.21.15.83:0/3085123757 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7a6c19dc10 con 0x7f7a6c10aac0 2024-08-19T01:26:11.588 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:10.798+0000 7f7a6b400640 1 --2- 172.21.15.83:0/3085123757 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7a6c10aac0 0x7f7a6c19d6d0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f7a5c02acc0 tx=0x7f7a5c031cc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:26:11.589 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:10.799+0000 7f7a68c00640 1 -- 172.21.15.83:0/3085123757 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7a5c036050 con 0x7f7a6c10aac0 2024-08-19T01:26:11.589 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:10.799+0000 7f7a68c00640 1 -- 172.21.15.83:0/3085123757 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7a5c037040 con 0x7f7a6c10aac0 2024-08-19T01:26:11.589 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:10.799+0000 7f7a72000640 1 -- 172.21.15.83:0/3085123757 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7a6c19dea0 con 0x7f7a6c10aac0 2024-08-19T01:26:11.589 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:10.799+0000 7f7a68c00640 1 -- 172.21.15.83:0/3085123757 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7a5c03b920 con 0x7f7a6c10aac0 2024-08-19T01:26:11.589 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:10.799+0000 7f7a72000640 1 -- 172.21.15.83:0/3085123757 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7a6c1a0b90 con 0x7f7a6c10aac0 2024-08-19T01:26:11.589 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:10.800+0000 7f7a68c00640 1 -- 172.21.15.83:0/3085123757 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 12) ==== 46300+0+0 (secure 0 0 0) 0x7f7a5c044020 con 0x7f7a6c10aac0 2024-08-19T01:26:11.589 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:10.800+0000 7f7a72000640 1 -- 172.21.15.83:0/3085123757 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7a6c10aec0 con 0x7f7a6c10aac0 2024-08-19T01:26:11.589 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:10.800+0000 7f7a68c00640 1 --2- 172.21.15.83:0/3085123757 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f7a50039020 0x7f7a5003b4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:26:11.589 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:10.800+0000 7f7a68c00640 1 -- 172.21.15.83:0/3085123757 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f7a5c042020 con 0x7f7a6c10aac0 2024-08-19T01:26:11.589 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:10.801+0000 7f7a62a00640 1 --2- 172.21.15.83:0/3085123757 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f7a50039020 0x7f7a5003b4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:26:11.589 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:10.801+0000 7f7a62a00640 1 --2- 172.21.15.83:0/3085123757 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f7a50039020 0x7f7a5003b4e0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f7a540098c0 tx=0x7f7a54006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:26:11.589 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:10.806+0000 7f7a68c00640 1 -- 172.21.15.83:0/3085123757 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7f7a5c04b020 con 0x7f7a6c10aac0 2024-08-19T01:26:11.589 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:10.862+0000 7f7a68c00640 1 -- 172.21.15.83:0/3085123757 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mgrmap(e 13) ==== 46343+0+0 (secure 0 0 0) 0x7f7a5c03d450 con 0x7f7a6c10aac0 2024-08-19T01:26:11.590 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:10.981+0000 7f7a72000640 1 -- 172.21.15.83:0/3085123757 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"} v 0) -- 0x7f7a6c1a1180 con 0x7f7a6c10aac0 2024-08-19T01:26:11.590 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:10.981+0000 7f7a68c00640 1 -- 172.21.15.83:0/3085123757 <== mon.0 v2:172.21.15.83:3300/0 8 ==== mon_command_ack([{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]=0 v8) ==== 112+0+5 (secure 0 0 0) 0x7f7a5c038260 con 0x7f7a6c10aac0 2024-08-19T01:26:11.590 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:10.984+0000 7f7a72000640 1 -- 172.21.15.83:0/3085123757 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f7a50039020 msgr2=0x7f7a5003b4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:26:11.590 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:10.984+0000 7f7a72000640 1 --2- 172.21.15.83:0/3085123757 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f7a50039020 0x7f7a5003b4e0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f7a540098c0 tx=0x7f7a54006db0 comp rx=0 tx=0).stop 2024-08-19T01:26:11.590 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:10.984+0000 7f7a72000640 1 -- 172.21.15.83:0/3085123757 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7a6c10aac0 msgr2=0x7f7a6c19d6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:26:11.590 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:10.984+0000 7f7a72000640 1 --2- 172.21.15.83:0/3085123757 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7a6c10aac0 0x7f7a6c19d6d0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f7a5c02acc0 tx=0x7f7a5c031cc0 comp rx=0 tx=0).stop 2024-08-19T01:26:11.590 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:10.984+0000 7f7a72000640 1 -- 172.21.15.83:0/3085123757 shutdown_connections 2024-08-19T01:26:11.590 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:10.984+0000 7f7a72000640 1 --2- 172.21.15.83:0/3085123757 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f7a50039020 0x7f7a5003b4e0 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:26:11.590 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:10.984+0000 7f7a72000640 1 --2- 172.21.15.83:0/3085123757 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7a6c10aac0 0x7f7a6c19d6d0 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:26:11.590 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:10.984+0000 7f7a72000640 1 -- 172.21.15.83:0/3085123757 >> 172.21.15.83:0/3085123757 conn(0x7f7a6c07cc40 msgr2=0x7f7a6c107530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:26:11.590 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:10.984+0000 7f7a72000640 1 -- 172.21.15.83:0/3085123757 shutdown_connections 2024-08-19T01:26:11.591 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:10.984+0000 7f7a72000640 1 -- 172.21.15.83:0/3085123757 wait complete. 2024-08-19T01:26:11.602 INFO:teuthology.orchestra.run.smithi083.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-08-19T01:26:11.602 INFO:teuthology.orchestra.run.smithi083.stdout:systemctl: stdout disabled 2024-08-19T01:26:11.616 INFO:teuthology.orchestra.run.smithi083.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-08-19T01:26:11.616 INFO:teuthology.orchestra.run.smithi083.stdout:systemctl: stdout inactive 2024-08-19T01:26:11.616 INFO:teuthology.orchestra.run.smithi083.stdout:firewalld.service is not enabled 2024-08-19T01:26:11.616 INFO:teuthology.orchestra.run.smithi083.stdout:Not possible to open ports <[8443]>. firewalld.service is not available 2024-08-19T01:26:11.618 INFO:teuthology.orchestra.run.smithi083.stdout:Ceph Dashboard is now available at: 2024-08-19T01:26:11.618 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-08-19T01:26:11.618 INFO:teuthology.orchestra.run.smithi083.stdout: URL: https://smithi083.front.sepia.ceph.com:8443/ 2024-08-19T01:26:11.619 INFO:teuthology.orchestra.run.smithi083.stdout: User: admin 2024-08-19T01:26:11.619 INFO:teuthology.orchestra.run.smithi083.stdout: Password: trcx5yerli 2024-08-19T01:26:11.619 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-08-19T01:26:11.619 INFO:teuthology.orchestra.run.smithi083.stdout:Saving cluster configuration to /var/lib/ceph/bad8780e-5dc9-11ef-bccf-c7b262605968/config directory 2024-08-19T01:26:12.155 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:26:11 smithi083 ceph-mon[17120]: mgrmap e13: x(active, since 6s) 2024-08-19T01:26:12.155 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:26:11 smithi083 ceph-mon[17120]: from='client.? 172.21.15.83:0/3085123757' entity='client.admin' cmd=[{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]: dispatch 2024-08-19T01:26:12.970 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:12.082+0000 7fdf67400640 1 Processor -- start 2024-08-19T01:26:12.970 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:12.083+0000 7fdf67400640 1 -- start start 2024-08-19T01:26:12.970 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:12.083+0000 7fdf67400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdf6007dde0 0x7fdf6007c240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:26:12.970 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:12.083+0000 7fdf67400640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fdf6007c780 con 0x7fdf6007dde0 2024-08-19T01:26:12.970 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:12.083+0000 7fdf64a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdf6007dde0 0x7fdf6007c240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:26:12.970 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:12.083+0000 7fdf64a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdf6007dde0 0x7fdf6007c240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:46996/0 (socket says 172.21.15.83:46996) 2024-08-19T01:26:12.971 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:12.083+0000 7fdf64a00640 1 -- 172.21.15.83:0/2737473448 learned_addr learned my addr 172.21.15.83:0/2737473448 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-08-19T01:26:12.971 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:12.084+0000 7fdf64a00640 1 -- 172.21.15.83:0/2737473448 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdf6007c900 con 0x7fdf6007dde0 2024-08-19T01:26:12.971 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:12.085+0000 7fdf64a00640 1 --2- 172.21.15.83:0/2737473448 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdf6007dde0 0x7fdf6007c240 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fdf50009820 tx=0x7fdf5002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2eb280eb579279c6 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:26:12.971 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:12.085+0000 7fdf5fe00640 1 -- 172.21.15.83:0/2737473448 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fdf50037070 con 0x7fdf6007dde0 2024-08-19T01:26:12.971 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:12.085+0000 7fdf5fe00640 1 -- 172.21.15.83:0/2737473448 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdf50032920 con 0x7fdf6007dde0 2024-08-19T01:26:12.971 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:12.085+0000 7fdf5fe00640 1 -- 172.21.15.83:0/2737473448 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fdf50030810 con 0x7fdf6007dde0 2024-08-19T01:26:12.971 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:12.085+0000 7fdf67400640 1 -- 172.21.15.83:0/2737473448 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdf6007dde0 msgr2=0x7fdf6007c240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:26:12.971 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:12.085+0000 7fdf67400640 1 --2- 172.21.15.83:0/2737473448 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdf6007dde0 0x7fdf6007c240 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fdf50009820 tx=0x7fdf5002a270 comp rx=0 tx=0).stop 2024-08-19T01:26:12.971 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:12.086+0000 7fdf67400640 1 -- 172.21.15.83:0/2737473448 shutdown_connections 2024-08-19T01:26:12.971 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:12.086+0000 7fdf67400640 1 --2- 172.21.15.83:0/2737473448 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdf6007dde0 0x7fdf6007c240 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:26:12.971 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:12.086+0000 7fdf67400640 1 -- 172.21.15.83:0/2737473448 >> 172.21.15.83:0/2737473448 conn(0x7fdf60102c00 msgr2=0x7fdf60105020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:26:12.971 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:12.086+0000 7fdf67400640 1 -- 172.21.15.83:0/2737473448 shutdown_connections 2024-08-19T01:26:12.972 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:12.086+0000 7fdf67400640 1 -- 172.21.15.83:0/2737473448 wait complete. 2024-08-19T01:26:12.972 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:12.086+0000 7fdf67400640 1 Processor -- start 2024-08-19T01:26:12.972 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:12.087+0000 7fdf67400640 1 -- start start 2024-08-19T01:26:12.972 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:12.087+0000 7fdf67400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdf6007dde0 0x7fdf6019d630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:26:12.972 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:12.087+0000 7fdf67400640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fdf6010daa0 con 0x7fdf6007dde0 2024-08-19T01:26:12.972 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:12.087+0000 7fdf64a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdf6007dde0 0x7fdf6019d630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:26:12.972 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:12.087+0000 7fdf64a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdf6007dde0 0x7fdf6019d630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:47004/0 (socket says 172.21.15.83:47004) 2024-08-19T01:26:12.972 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:12.087+0000 7fdf64a00640 1 -- 172.21.15.83:0/2930735354 learned_addr learned my addr 172.21.15.83:0/2930735354 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-08-19T01:26:12.972 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:12.087+0000 7fdf64a00640 1 -- 172.21.15.83:0/2930735354 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdf6019db70 con 0x7fdf6007dde0 2024-08-19T01:26:12.972 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:12.088+0000 7fdf64a00640 1 --2- 172.21.15.83:0/2930735354 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdf6007dde0 0x7fdf6019d630 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fdf500097f0 tx=0x7fdf500327b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:26:12.972 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:12.088+0000 7fdf5e000640 1 -- 172.21.15.83:0/2930735354 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fdf50037040 con 0x7fdf6007dde0 2024-08-19T01:26:12.972 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:12.088+0000 7fdf5e000640 1 -- 172.21.15.83:0/2930735354 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdf50030cc0 con 0x7fdf6007dde0 2024-08-19T01:26:12.973 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:12.088+0000 7fdf5e000640 1 -- 172.21.15.83:0/2930735354 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fdf5003a6b0 con 0x7fdf6007dde0 2024-08-19T01:26:12.973 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:12.088+0000 7fdf67400640 1 -- 172.21.15.83:0/2930735354 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdf6019de00 con 0x7fdf6007dde0 2024-08-19T01:26:12.973 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:12.088+0000 7fdf67400640 1 -- 172.21.15.83:0/2930735354 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdf601a0af0 con 0x7fdf6007dde0 2024-08-19T01:26:12.973 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:12.089+0000 7fdf5e000640 1 -- 172.21.15.83:0/2930735354 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 13) ==== 46343+0+0 (secure 0 0 0) 0x7fdf5003baf0 con 0x7fdf6007dde0 2024-08-19T01:26:12.973 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:12.089+0000 7fdf67400640 1 -- 172.21.15.83:0/2930735354 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdf6007c2c0 con 0x7fdf6007dde0 2024-08-19T01:26:12.973 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:12.090+0000 7fdf5e000640 1 --2- 172.21.15.83:0/2930735354 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7fdf440390c0 0x7fdf4403b580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:26:12.973 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:12.090+0000 7fdf5e000640 1 -- 172.21.15.83:0/2930735354 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fdf5006c780 con 0x7fdf6007dde0 2024-08-19T01:26:12.973 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:12.090+0000 7fdf57e00640 1 --2- 172.21.15.83:0/2930735354 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7fdf440390c0 0x7fdf4403b580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:26:12.973 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:12.091+0000 7fdf57e00640 1 --2- 172.21.15.83:0/2930735354 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7fdf440390c0 0x7fdf4403b580 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fdf480098c0 tx=0x7fdf48006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:26:12.973 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:12.095+0000 7fdf5e000640 1 -- 172.21.15.83:0/2930735354 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7fdf50043e10 con 0x7fdf6007dde0 2024-08-19T01:26:12.973 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:12.359+0000 7fdf67400640 1 -- 172.21.15.83:0/2930735354 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command([{prefix=config-key set, key=mgr/dashboard/cluster/status}] v 0) -- 0x7fdf601a10e0 con 0x7fdf6007dde0 2024-08-19T01:26:12.974 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:12.362+0000 7fdf5e000640 1 -- 172.21.15.83:0/2930735354 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{prefix=config-key set, key=mgr/dashboard/cluster/status}]=0 set mgr/dashboard/cluster/status v28)=0 set mgr/dashboard/cluster/status v28) ==== 153+0+0 (secure 0 0 0) 0x7fdf5003a850 con 0x7fdf6007dde0 2024-08-19T01:26:12.974 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr set mgr/dashboard/cluster/status 2024-08-19T01:26:12.974 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:12.364+0000 7fdf67400640 1 -- 172.21.15.83:0/2930735354 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7fdf440390c0 msgr2=0x7fdf4403b580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:26:12.974 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:12.364+0000 7fdf67400640 1 --2- 172.21.15.83:0/2930735354 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7fdf440390c0 0x7fdf4403b580 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fdf480098c0 tx=0x7fdf48006db0 comp rx=0 tx=0).stop 2024-08-19T01:26:12.974 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:12.364+0000 7fdf67400640 1 -- 172.21.15.83:0/2930735354 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdf6007dde0 msgr2=0x7fdf6019d630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:26:12.974 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:12.364+0000 7fdf67400640 1 --2- 172.21.15.83:0/2930735354 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdf6007dde0 0x7fdf6019d630 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fdf500097f0 tx=0x7fdf500327b0 comp rx=0 tx=0).stop 2024-08-19T01:26:12.974 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:12.364+0000 7fdf67400640 1 -- 172.21.15.83:0/2930735354 shutdown_connections 2024-08-19T01:26:12.974 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:12.364+0000 7fdf67400640 1 --2- 172.21.15.83:0/2930735354 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7fdf440390c0 0x7fdf4403b580 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:26:12.974 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:12.364+0000 7fdf67400640 1 --2- 172.21.15.83:0/2930735354 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdf6007dde0 0x7fdf6019d630 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:26:12.974 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:12.364+0000 7fdf67400640 1 -- 172.21.15.83:0/2930735354 >> 172.21.15.83:0/2930735354 conn(0x7fdf60102c00 msgr2=0x7fdf60103770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:26:12.974 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:12.365+0000 7fdf67400640 1 -- 172.21.15.83:0/2930735354 shutdown_connections 2024-08-19T01:26:12.975 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:12.365+0000 7fdf67400640 1 -- 172.21.15.83:0/2930735354 wait complete. 2024-08-19T01:26:12.975 INFO:teuthology.orchestra.run.smithi083.stdout:You can access the Ceph CLI as following in case of multi-cluster or non-default config: 2024-08-19T01:26:12.975 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-08-19T01:26:12.975 INFO:teuthology.orchestra.run.smithi083.stdout: sudo /home/ubuntu/cephtest/cephadm shell --fsid bad8780e-5dc9-11ef-bccf-c7b262605968 -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring 2024-08-19T01:26:12.975 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-08-19T01:26:12.975 INFO:teuthology.orchestra.run.smithi083.stdout:Or, if you are only running a single cluster on this host: 2024-08-19T01:26:12.975 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-08-19T01:26:12.975 INFO:teuthology.orchestra.run.smithi083.stdout: sudo /home/ubuntu/cephtest/cephadm shell 2024-08-19T01:26:12.975 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-08-19T01:26:12.975 INFO:teuthology.orchestra.run.smithi083.stdout:Please consider enabling telemetry to help improve Ceph: 2024-08-19T01:26:12.975 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-08-19T01:26:12.976 INFO:teuthology.orchestra.run.smithi083.stdout: ceph telemetry on 2024-08-19T01:26:12.976 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-08-19T01:26:12.976 INFO:teuthology.orchestra.run.smithi083.stdout:For more information see: 2024-08-19T01:26:12.976 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-08-19T01:26:12.976 INFO:teuthology.orchestra.run.smithi083.stdout: https://docs.ceph.com/en/latest/mgr/telemetry/ 2024-08-19T01:26:12.976 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-08-19T01:26:12.976 INFO:teuthology.orchestra.run.smithi083.stdout:Bootstrap complete. 2024-08-19T01:26:13.023 INFO:tasks.cephadm:Fetching config... 2024-08-19T01:26:13.024 DEBUG:teuthology.orchestra.run.smithi083:> set -ex 2024-08-19T01:26:13.024 DEBUG:teuthology.orchestra.run.smithi083:> dd if=/etc/ceph/ceph.conf of=/dev/stdout 2024-08-19T01:26:13.060 INFO:tasks.cephadm:Fetching client.admin keyring... 2024-08-19T01:26:13.060 DEBUG:teuthology.orchestra.run.smithi083:> set -ex 2024-08-19T01:26:13.060 DEBUG:teuthology.orchestra.run.smithi083:> dd if=/etc/ceph/ceph.client.admin.keyring of=/dev/stdout 2024-08-19T01:26:13.138 INFO:tasks.cephadm:Fetching mon keyring... 2024-08-19T01:26:13.138 DEBUG:teuthology.orchestra.run.smithi083:> set -ex 2024-08-19T01:26:13.138 DEBUG:teuthology.orchestra.run.smithi083:> sudo dd if=/var/lib/ceph/bad8780e-5dc9-11ef-bccf-c7b262605968/mon.a/keyring of=/dev/stdout 2024-08-19T01:26:13.234 INFO:tasks.cephadm:Fetching pub ssh key... 2024-08-19T01:26:13.234 DEBUG:teuthology.orchestra.run.smithi083:> set -ex 2024-08-19T01:26:13.234 DEBUG:teuthology.orchestra.run.smithi083:> dd if=/home/ubuntu/cephtest/ceph.pub of=/dev/stdout 2024-08-19T01:26:13.309 INFO:tasks.cephadm:Installing pub ssh key for root users... 2024-08-19T01:26:13.309 DEBUG:teuthology.orchestra.run.smithi083:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCrvIGdPB8Hdno6mG7Qs8/pMv7eXZxIWWZCRBi5vWdNl6Dczcbp+t0qGD2pLrph8IZrtS9VH2kdbTCamX8oBGbFDwwpx1iArwlf/OZumZLnEsj0sLMUVD9VkfY/xLfr4IY0lQWgjkmmzrcE8LNoyKFlGRElC9M/K04zoQQyRz2UTjkotDtpVGCsogWJlqnQScxO+XNTSQSb7bR27N7OIclfvId0qMUAAECpMIVgTSDlRnF5Os4WwewnOuQ0WEs5vX2Nx3if4kAeIAYGm5+Bdj1XpGiG+BkwueH9QNZ1gngixBp2fY+gQn7X2GAYP60DB6mdez9f7B/gxaBjCzjtogMbcoYDCaK3PSFMP5IfSSLqySu66CCh2NlppeR77RMnpVlrmlemHSJLvuDKpF98BbB9rIjYDNcVpsjj2LgeBN7QLOTpbSS2FE56eBf9FFUvtexmPnabWCUOFM/sAIn/IT1Xob3lcAQNHPSW44r0drd31eokC8VmoYSGHiZ67AC/KBE= ceph-bad8780e-5dc9-11ef-bccf-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-08-19T01:26:13.433 INFO:teuthology.orchestra.run.smithi083.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCrvIGdPB8Hdno6mG7Qs8/pMv7eXZxIWWZCRBi5vWdNl6Dczcbp+t0qGD2pLrph8IZrtS9VH2kdbTCamX8oBGbFDwwpx1iArwlf/OZumZLnEsj0sLMUVD9VkfY/xLfr4IY0lQWgjkmmzrcE8LNoyKFlGRElC9M/K04zoQQyRz2UTjkotDtpVGCsogWJlqnQScxO+XNTSQSb7bR27N7OIclfvId0qMUAAECpMIVgTSDlRnF5Os4WwewnOuQ0WEs5vX2Nx3if4kAeIAYGm5+Bdj1XpGiG+BkwueH9QNZ1gngixBp2fY+gQn7X2GAYP60DB6mdez9f7B/gxaBjCzjtogMbcoYDCaK3PSFMP5IfSSLqySu66CCh2NlppeR77RMnpVlrmlemHSJLvuDKpF98BbB9rIjYDNcVpsjj2LgeBN7QLOTpbSS2FE56eBf9FFUvtexmPnabWCUOFM/sAIn/IT1Xob3lcAQNHPSW44r0drd31eokC8VmoYSGHiZ67AC/KBE= ceph-bad8780e-5dc9-11ef-bccf-c7b262605968 2024-08-19T01:26:13.458 DEBUG:teuthology.orchestra.run.smithi117:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCrvIGdPB8Hdno6mG7Qs8/pMv7eXZxIWWZCRBi5vWdNl6Dczcbp+t0qGD2pLrph8IZrtS9VH2kdbTCamX8oBGbFDwwpx1iArwlf/OZumZLnEsj0sLMUVD9VkfY/xLfr4IY0lQWgjkmmzrcE8LNoyKFlGRElC9M/K04zoQQyRz2UTjkotDtpVGCsogWJlqnQScxO+XNTSQSb7bR27N7OIclfvId0qMUAAECpMIVgTSDlRnF5Os4WwewnOuQ0WEs5vX2Nx3if4kAeIAYGm5+Bdj1XpGiG+BkwueH9QNZ1gngixBp2fY+gQn7X2GAYP60DB6mdez9f7B/gxaBjCzjtogMbcoYDCaK3PSFMP5IfSSLqySu66CCh2NlppeR77RMnpVlrmlemHSJLvuDKpF98BbB9rIjYDNcVpsjj2LgeBN7QLOTpbSS2FE56eBf9FFUvtexmPnabWCUOFM/sAIn/IT1Xob3lcAQNHPSW44r0drd31eokC8VmoYSGHiZ67AC/KBE= ceph-bad8780e-5dc9-11ef-bccf-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-08-19T01:26:13.499 INFO:teuthology.orchestra.run.smithi117.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCrvIGdPB8Hdno6mG7Qs8/pMv7eXZxIWWZCRBi5vWdNl6Dczcbp+t0qGD2pLrph8IZrtS9VH2kdbTCamX8oBGbFDwwpx1iArwlf/OZumZLnEsj0sLMUVD9VkfY/xLfr4IY0lQWgjkmmzrcE8LNoyKFlGRElC9M/K04zoQQyRz2UTjkotDtpVGCsogWJlqnQScxO+XNTSQSb7bR27N7OIclfvId0qMUAAECpMIVgTSDlRnF5Os4WwewnOuQ0WEs5vX2Nx3if4kAeIAYGm5+Bdj1XpGiG+BkwueH9QNZ1gngixBp2fY+gQn7X2GAYP60DB6mdez9f7B/gxaBjCzjtogMbcoYDCaK3PSFMP5IfSSLqySu66CCh2NlppeR77RMnpVlrmlemHSJLvuDKpF98BbB9rIjYDNcVpsjj2LgeBN7QLOTpbSS2FE56eBf9FFUvtexmPnabWCUOFM/sAIn/IT1Xob3lcAQNHPSW44r0drd31eokC8VmoYSGHiZ67AC/KBE= ceph-bad8780e-5dc9-11ef-bccf-c7b262605968 2024-08-19T01:26:13.511 DEBUG:teuthology.orchestra.run.smithi167:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCrvIGdPB8Hdno6mG7Qs8/pMv7eXZxIWWZCRBi5vWdNl6Dczcbp+t0qGD2pLrph8IZrtS9VH2kdbTCamX8oBGbFDwwpx1iArwlf/OZumZLnEsj0sLMUVD9VkfY/xLfr4IY0lQWgjkmmzrcE8LNoyKFlGRElC9M/K04zoQQyRz2UTjkotDtpVGCsogWJlqnQScxO+XNTSQSb7bR27N7OIclfvId0qMUAAECpMIVgTSDlRnF5Os4WwewnOuQ0WEs5vX2Nx3if4kAeIAYGm5+Bdj1XpGiG+BkwueH9QNZ1gngixBp2fY+gQn7X2GAYP60DB6mdez9f7B/gxaBjCzjtogMbcoYDCaK3PSFMP5IfSSLqySu66CCh2NlppeR77RMnpVlrmlemHSJLvuDKpF98BbB9rIjYDNcVpsjj2LgeBN7QLOTpbSS2FE56eBf9FFUvtexmPnabWCUOFM/sAIn/IT1Xob3lcAQNHPSW44r0drd31eokC8VmoYSGHiZ67AC/KBE= ceph-bad8780e-5dc9-11ef-bccf-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-08-19T01:26:13.536 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:26:13 smithi083 ceph-mon[17120]: from='client.? 172.21.15.83:0/2930735354' entity='client.admin' 2024-08-19T01:26:13.552 INFO:teuthology.orchestra.run.smithi167.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCrvIGdPB8Hdno6mG7Qs8/pMv7eXZxIWWZCRBi5vWdNl6Dczcbp+t0qGD2pLrph8IZrtS9VH2kdbTCamX8oBGbFDwwpx1iArwlf/OZumZLnEsj0sLMUVD9VkfY/xLfr4IY0lQWgjkmmzrcE8LNoyKFlGRElC9M/K04zoQQyRz2UTjkotDtpVGCsogWJlqnQScxO+XNTSQSb7bR27N7OIclfvId0qMUAAECpMIVgTSDlRnF5Os4WwewnOuQ0WEs5vX2Nx3if4kAeIAYGm5+Bdj1XpGiG+BkwueH9QNZ1gngixBp2fY+gQn7X2GAYP60DB6mdez9f7B/gxaBjCzjtogMbcoYDCaK3PSFMP5IfSSLqySu66CCh2NlppeR77RMnpVlrmlemHSJLvuDKpF98BbB9rIjYDNcVpsjj2LgeBN7QLOTpbSS2FE56eBf9FFUvtexmPnabWCUOFM/sAIn/IT1Xob3lcAQNHPSW44r0drd31eokC8VmoYSGHiZ67AC/KBE= ceph-bad8780e-5dc9-11ef-bccf-c7b262605968 2024-08-19T01:26:13.565 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid bad8780e-5dc9-11ef-bccf-c7b262605968 -- ceph config set mgr mgr/cephadm/allow_ptrace true 2024-08-19T01:26:13.932 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/bad8780e-5dc9-11ef-bccf-c7b262605968/mon.a/config 2024-08-19T01:26:14.477 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:26:14.475+0000 7f0761400640 1 -- 172.21.15.83:0/1233848503 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f075c101bd0 msgr2=0x7f075c103fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:26:14.477 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:26:14.475+0000 7f0761400640 1 --2- 172.21.15.83:0/1233848503 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f075c101bd0 0x7f075c103fc0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f074c009880 tx=0x7f074c02a5d0 comp rx=0 tx=0).stop 2024-08-19T01:26:14.477 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:26:14.475+0000 7f0761400640 1 -- 172.21.15.83:0/1233848503 shutdown_connections 2024-08-19T01:26:14.478 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:26:14.475+0000 7f0761400640 1 --2- 172.21.15.83:0/1233848503 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f075c101bd0 0x7f075c103fc0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:26:14.478 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:26:14.475+0000 7f0761400640 1 -- 172.21.15.83:0/1233848503 >> 172.21.15.83:0/1233848503 conn(0x7f075c0fb7d0 msgr2=0x7f075c0fdc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:26:14.478 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:26:14.475+0000 7f0761400640 1 -- 172.21.15.83:0/1233848503 shutdown_connections 2024-08-19T01:26:14.478 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:26:14.475+0000 7f0761400640 1 -- 172.21.15.83:0/1233848503 wait complete. 2024-08-19T01:26:14.478 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:26:14.476+0000 7f0761400640 1 Processor -- start 2024-08-19T01:26:14.478 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:26:14.476+0000 7f0761400640 1 -- start start 2024-08-19T01:26:14.479 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:26:14.477+0000 7f0761400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f075c101bd0 0x7f075c198ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:26:14.479 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:26:14.477+0000 7f0761400640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f075c1071e0 con 0x7f075c101bd0 2024-08-19T01:26:14.479 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:26:14.477+0000 7f075be00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f075c101bd0 0x7f075c198ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:26:14.479 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:26:14.477+0000 7f075be00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f075c101bd0 0x7f075c198ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:47022/0 (socket says 172.21.15.83:47022) 2024-08-19T01:26:14.479 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:26:14.477+0000 7f075be00640 1 -- 172.21.15.83:0/2149154848 learned_addr learned my addr 172.21.15.83:0/2149154848 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-08-19T01:26:14.479 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:26:14.477+0000 7f075be00640 1 -- 172.21.15.83:0/2149154848 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f075c199000 con 0x7f075c101bd0 2024-08-19T01:26:14.479 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:26:14.478+0000 7f075be00640 1 --2- 172.21.15.83:0/2149154848 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f075c101bd0 0x7f075c198ac0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f074c004f60 tx=0x7f074c0048f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:26:14.480 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:26:14.478+0000 7f0758c00640 1 -- 172.21.15.83:0/2149154848 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f074c03f070 con 0x7f075c101bd0 2024-08-19T01:26:14.480 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:26:14.478+0000 7f0758c00640 1 -- 172.21.15.83:0/2149154848 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f074c003ad0 con 0x7f075c101bd0 2024-08-19T01:26:14.480 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:26:14.478+0000 7f0761400640 1 -- 172.21.15.83:0/2149154848 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f075c1992f0 con 0x7f075c101bd0 2024-08-19T01:26:14.480 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:26:14.478+0000 7f0758c00640 1 -- 172.21.15.83:0/2149154848 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f074c035030 con 0x7f075c101bd0 2024-08-19T01:26:14.480 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:26:14.478+0000 7f0761400640 1 -- 172.21.15.83:0/2149154848 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f075c19bf80 con 0x7f075c101bd0 2024-08-19T01:26:14.481 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:26:14.480+0000 7f0761400640 1 -- 172.21.15.83:0/2149154848 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f075c107960 con 0x7f075c101bd0 2024-08-19T01:26:14.481 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:26:14.480+0000 7f0758c00640 1 -- 172.21.15.83:0/2149154848 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 13) ==== 46343+0+0 (secure 0 0 0) 0x7f074c042070 con 0x7f075c101bd0 2024-08-19T01:26:14.482 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:26:14.481+0000 7f0758c00640 1 --2- 172.21.15.83:0/2149154848 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f07400390c0 0x7f074003b580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:26:14.482 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:26:14.481+0000 7f0758c00640 1 -- 172.21.15.83:0/2149154848 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f074c06cda0 con 0x7f075c101bd0 2024-08-19T01:26:14.482 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:26:14.481+0000 7f075b400640 1 --2- 172.21.15.83:0/2149154848 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f07400390c0 0x7f074003b580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:26:14.482 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:26:14.482+0000 7f075b400640 1 --2- 172.21.15.83:0/2149154848 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f07400390c0 0x7f074003b580 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f07480098c0 tx=0x7f0748006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:26:14.490 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:26:14.489+0000 7f0758c00640 1 -- 172.21.15.83:0/2149154848 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7f074c03a3c0 con 0x7f075c101bd0 2024-08-19T01:26:14.689 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:26:14.688+0000 7f0761400640 1 -- 172.21.15.83:0/2149154848 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/allow_ptrace}] v 0) -- 0x7f075c02d7c0 con 0x7f075c101bd0 2024-08-19T01:26:14.692 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:26:14.690+0000 7f0758c00640 1 -- 172.21.15.83:0/2149154848 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/allow_ptrace}]=0 v9)=0 v9) ==== 125+0+0 (secure 0 0 0) 0x7f074c0412c0 con 0x7f075c101bd0 2024-08-19T01:26:14.695 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:26:14.693+0000 7f0761400640 1 -- 172.21.15.83:0/2149154848 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f07400390c0 msgr2=0x7f074003b580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:26:14.695 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:26:14.693+0000 7f0761400640 1 --2- 172.21.15.83:0/2149154848 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f07400390c0 0x7f074003b580 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f07480098c0 tx=0x7f0748006db0 comp rx=0 tx=0).stop 2024-08-19T01:26:14.695 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:26:14.694+0000 7f0761400640 1 -- 172.21.15.83:0/2149154848 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f075c101bd0 msgr2=0x7f075c198ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:26:14.695 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:26:14.694+0000 7f0761400640 1 --2- 172.21.15.83:0/2149154848 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f075c101bd0 0x7f075c198ac0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f074c004f60 tx=0x7f074c0048f0 comp rx=0 tx=0).stop 2024-08-19T01:26:14.695 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:26:14.694+0000 7f0761400640 1 -- 172.21.15.83:0/2149154848 shutdown_connections 2024-08-19T01:26:14.695 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:26:14.694+0000 7f0761400640 1 --2- 172.21.15.83:0/2149154848 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f07400390c0 0x7f074003b580 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:26:14.695 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:26:14.694+0000 7f0761400640 1 --2- 172.21.15.83:0/2149154848 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f075c101bd0 0x7f075c198ac0 secure :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f074c004f60 tx=0x7f074c0048f0 comp rx=0 tx=0).stop 2024-08-19T01:26:14.695 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:26:14.694+0000 7f0761400640 1 -- 172.21.15.83:0/2149154848 >> 172.21.15.83:0/2149154848 conn(0x7f075c0fb7d0 msgr2=0x7f075c0fdc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:26:14.696 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:26:14.694+0000 7f0761400640 1 -- 172.21.15.83:0/2149154848 shutdown_connections 2024-08-19T01:26:14.696 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:26:14.694+0000 7f0761400640 1 -- 172.21.15.83:0/2149154848 wait complete. 2024-08-19T01:26:15.427 INFO:tasks.cephadm:Distributing conf and client.admin keyring to all hosts + 0755 2024-08-19T01:26:15.427 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid bad8780e-5dc9-11ef-bccf-c7b262605968 -- ceph orch client-keyring set client.admin '*' --mode 0755 2024-08-19T01:26:15.981 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/bad8780e-5dc9-11ef-bccf-c7b262605968/mon.a/config 2024-08-19T01:26:16.124 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:26:15 smithi083 ceph-mon[17120]: from='client.? 172.21.15.83:0/2149154848' entity='client.admin' 2024-08-19T01:26:16.711 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:26:16.710+0000 7f96fc000640 1 -- 172.21.15.83:0/863822058 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f96f4103540 msgr2=0x7f96f4103940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:26:16.711 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:26:16.710+0000 7f96fc000640 1 --2- 172.21.15.83:0/863822058 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f96f4103540 0x7f96f4103940 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f96e8009880 tx=0x7f96e802a5d0 comp rx=0 tx=0).stop 2024-08-19T01:26:16.711 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:26:16.710+0000 7f96fc000640 1 -- 172.21.15.83:0/863822058 shutdown_connections 2024-08-19T01:26:16.712 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:26:16.710+0000 7f96fc000640 1 --2- 172.21.15.83:0/863822058 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f96f4103540 0x7f96f4103940 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:26:16.712 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:26:16.710+0000 7f96fc000640 1 -- 172.21.15.83:0/863822058 >> 172.21.15.83:0/863822058 conn(0x7f96f40fecf0 msgr2=0x7f96f4101110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:26:16.712 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:26:16.710+0000 7f96fc000640 1 -- 172.21.15.83:0/863822058 shutdown_connections 2024-08-19T01:26:16.712 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:26:16.710+0000 7f96fc000640 1 -- 172.21.15.83:0/863822058 wait complete. 2024-08-19T01:26:16.712 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:26:16.711+0000 7f96fc000640 1 Processor -- start 2024-08-19T01:26:16.712 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:26:16.711+0000 7f96fc000640 1 -- start start 2024-08-19T01:26:16.713 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:26:16.712+0000 7f96fc000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f96f4103540 0x7f96f4198c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:26:16.713 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:26:16.712+0000 7f96fc000640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f96f4109650 con 0x7f96f4103540 2024-08-19T01:26:16.713 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:26:16.712+0000 7f96f9600640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f96f4103540 0x7f96f4198c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:26:16.713 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:26:16.712+0000 7f96f9600640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f96f4103540 0x7f96f4198c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:47034/0 (socket says 172.21.15.83:47034) 2024-08-19T01:26:16.713 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:26:16.712+0000 7f96f9600640 1 -- 172.21.15.83:0/2501881644 learned_addr learned my addr 172.21.15.83:0/2501881644 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-08-19T01:26:16.714 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:26:16.712+0000 7f96f9600640 1 -- 172.21.15.83:0/2501881644 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f96f41991c0 con 0x7f96f4103540 2024-08-19T01:26:16.714 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:26:16.713+0000 7f96f9600640 1 --2- 172.21.15.83:0/2501881644 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f96f4103540 0x7f96f4198c80 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f96e8004f60 tx=0x7f96e80048f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:26:16.714 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:26:16.713+0000 7f96f2000640 1 -- 172.21.15.83:0/2501881644 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f96e803f070 con 0x7f96f4103540 2024-08-19T01:26:16.714 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:26:16.713+0000 7f96f2000640 1 -- 172.21.15.83:0/2501881644 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f96e8004d40 con 0x7f96f4103540 2024-08-19T01:26:16.715 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:26:16.713+0000 7f96f2000640 1 -- 172.21.15.83:0/2501881644 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f96e8035030 con 0x7f96f4103540 2024-08-19T01:26:16.715 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:26:16.713+0000 7f96fc000640 1 -- 172.21.15.83:0/2501881644 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f96f4199450 con 0x7f96f4103540 2024-08-19T01:26:16.715 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:26:16.713+0000 7f96fc000640 1 -- 172.21.15.83:0/2501881644 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f96f419c140 con 0x7f96f4103540 2024-08-19T01:26:16.715 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:26:16.714+0000 7f96f2000640 1 -- 172.21.15.83:0/2501881644 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 13) ==== 46343+0+0 (secure 0 0 0) 0x7f96e8042070 con 0x7f96f4103540 2024-08-19T01:26:16.715 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:26:16.714+0000 7f96fc000640 1 -- 172.21.15.83:0/2501881644 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f96f4109dd0 con 0x7f96f4103540 2024-08-19T01:26:16.716 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:26:16.714+0000 7f96f2000640 1 --2- 172.21.15.83:0/2501881644 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f96dc039060 0x7f96dc03b520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:26:16.716 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:26:16.715+0000 7f96f2000640 1 -- 172.21.15.83:0/2501881644 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f96e806cd90 con 0x7f96f4103540 2024-08-19T01:26:16.716 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:26:16.715+0000 7f96f8c00640 1 --2- 172.21.15.83:0/2501881644 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f96dc039060 0x7f96dc03b520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:26:16.716 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:26:16.715+0000 7f96f8c00640 1 --2- 172.21.15.83:0/2501881644 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f96dc039060 0x7f96dc03b520 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f96e00098c0 tx=0x7f96e0006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:26:16.720 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:26:16.719+0000 7f96f2000640 1 -- 172.21.15.83:0/2501881644 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7f96e804a260 con 0x7f96f4103540 2024-08-19T01:26:16.904 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:26:16.903+0000 7f96fc000640 1 -- 172.21.15.83:0/2501881644 --> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] -- mgr_command(tid 0: {"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}) -- 0x7f96f402d650 con 0x7f96dc039060 2024-08-19T01:26:16.908 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:26:16.907+0000 7f96f2000640 1 -- 172.21.15.83:0/2501881644 <== mgr.14150 v2:172.21.15.83:6800/683740916 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f96f402d650 con 0x7f96dc039060 2024-08-19T01:26:16.910 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:26:16.909+0000 7f96fc000640 1 -- 172.21.15.83:0/2501881644 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f96dc039060 msgr2=0x7f96dc03b520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:26:16.910 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:26:16.909+0000 7f96fc000640 1 --2- 172.21.15.83:0/2501881644 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f96dc039060 0x7f96dc03b520 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f96e00098c0 tx=0x7f96e0006db0 comp rx=0 tx=0).stop 2024-08-19T01:26:16.910 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:26:16.909+0000 7f96fc000640 1 -- 172.21.15.83:0/2501881644 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f96f4103540 msgr2=0x7f96f4198c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:26:16.910 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:26:16.909+0000 7f96fc000640 1 --2- 172.21.15.83:0/2501881644 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f96f4103540 0x7f96f4198c80 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f96e8004f60 tx=0x7f96e80048f0 comp rx=0 tx=0).stop 2024-08-19T01:26:16.910 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:26:16.909+0000 7f96fc000640 1 -- 172.21.15.83:0/2501881644 shutdown_connections 2024-08-19T01:26:16.911 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:26:16.909+0000 7f96fc000640 1 --2- 172.21.15.83:0/2501881644 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f96dc039060 0x7f96dc03b520 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:26:16.911 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:26:16.909+0000 7f96fc000640 1 --2- 172.21.15.83:0/2501881644 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f96f4103540 0x7f96f4198c80 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:26:16.911 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:26:16.909+0000 7f96fc000640 1 -- 172.21.15.83:0/2501881644 >> 172.21.15.83:0/2501881644 conn(0x7f96f40fecf0 msgr2=0x7f96f40ff650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:26:16.911 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:26:16.909+0000 7f96fc000640 1 -- 172.21.15.83:0/2501881644 shutdown_connections 2024-08-19T01:26:16.911 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:26:16.909+0000 7f96fc000640 1 -- 172.21.15.83:0/2501881644 wait complete. 2024-08-19T01:26:17.660 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi117 2024-08-19T01:26:17.660 DEBUG:teuthology.orchestra.run.smithi117:> set -ex 2024-08-19T01:26:17.660 DEBUG:teuthology.orchestra.run.smithi117:> dd of=/etc/ceph/ceph.conf 2024-08-19T01:26:17.680 DEBUG:teuthology.orchestra.run.smithi117:> set -ex 2024-08-19T01:26:17.680 DEBUG:teuthology.orchestra.run.smithi117:> dd of=/etc/ceph/ceph.client.admin.keyring 2024-08-19T01:26:17.739 INFO:tasks.cephadm:Adding host smithi117 to orchestrator... 2024-08-19T01:26:17.740 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid bad8780e-5dc9-11ef-bccf-c7b262605968 -- ceph orch host add smithi117 2024-08-19T01:26:18.155 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:26:17 smithi083 ceph-mon[17120]: from='client.14172 -' entity='client.admin' cmd=[{"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}]: dispatch 2024-08-19T01:26:18.155 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:26:17 smithi083 ceph-mon[17120]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' 2024-08-19T01:26:18.669 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/bad8780e-5dc9-11ef-bccf-c7b262605968/mon.a/config 2024-08-19T01:26:19.255 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:26:19.254+0000 7f0645400640 1 -- 172.21.15.83:0/2821049381 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0640101350 msgr2=0x7f0640101750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:26:19.255 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:26:19.254+0000 7f0645400640 1 --2- 172.21.15.83:0/2821049381 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0640101350 0x7f0640101750 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f0630009880 tx=0x7f063002a5d0 comp rx=0 tx=0).stop 2024-08-19T01:26:19.256 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:26:19.254+0000 7f0645400640 1 -- 172.21.15.83:0/2821049381 shutdown_connections 2024-08-19T01:26:19.256 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:26:19.254+0000 7f0645400640 1 --2- 172.21.15.83:0/2821049381 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0640101350 0x7f0640101750 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:26:19.256 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:26:19.254+0000 7f0645400640 1 -- 172.21.15.83:0/2821049381 >> 172.21.15.83:0/2821049381 conn(0x7f06400fcae0 msgr2=0x7f06400fef20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:26:19.256 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:26:19.254+0000 7f0645400640 1 -- 172.21.15.83:0/2821049381 shutdown_connections 2024-08-19T01:26:19.256 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:26:19.254+0000 7f0645400640 1 -- 172.21.15.83:0/2821049381 wait complete. 2024-08-19T01:26:19.257 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:26:19.255+0000 7f0645400640 1 Processor -- start 2024-08-19T01:26:19.257 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:26:19.255+0000 7f0645400640 1 -- start start 2024-08-19T01:26:19.257 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:26:19.255+0000 7f0645400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0640101350 0x7f0640198cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:26:19.257 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:26:19.255+0000 7f0645400640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f0640107460 con 0x7f0640101350 2024-08-19T01:26:19.257 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:26:19.256+0000 7f063e800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0640101350 0x7f0640198cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:26:19.257 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:26:19.256+0000 7f063e800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0640101350 0x7f0640198cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:47064/0 (socket says 172.21.15.83:47064) 2024-08-19T01:26:19.257 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:26:19.256+0000 7f063e800640 1 -- 172.21.15.83:0/1986362490 learned_addr learned my addr 172.21.15.83:0/1986362490 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-08-19T01:26:19.258 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:26:19.256+0000 7f063e800640 1 -- 172.21.15.83:0/1986362490 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f06401991f0 con 0x7f0640101350 2024-08-19T01:26:19.258 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:26:19.256+0000 7f063e800640 1 --2- 172.21.15.83:0/1986362490 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0640101350 0x7f0640198cb0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f0630004f60 tx=0x7f06300048f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:26:19.258 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:26:19.257+0000 7f0637400640 1 -- 172.21.15.83:0/1986362490 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f063003f070 con 0x7f0640101350 2024-08-19T01:26:19.258 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:26:19.257+0000 7f0637400640 1 -- 172.21.15.83:0/1986362490 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0630004cc0 con 0x7f0640101350 2024-08-19T01:26:19.258 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:26:19.257+0000 7f0645400640 1 -- 172.21.15.83:0/1986362490 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0640199480 con 0x7f0640101350 2024-08-19T01:26:19.258 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:26:19.257+0000 7f0637400640 1 -- 172.21.15.83:0/1986362490 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0630035030 con 0x7f0640101350 2024-08-19T01:26:19.259 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:26:19.257+0000 7f0645400640 1 -- 172.21.15.83:0/1986362490 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f064019c170 con 0x7f0640101350 2024-08-19T01:26:19.260 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:26:19.258+0000 7f0637400640 1 -- 172.21.15.83:0/1986362490 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 13) ==== 46343+0+0 (secure 0 0 0) 0x7f0630042070 con 0x7f0640101350 2024-08-19T01:26:19.260 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:26:19.258+0000 7f0645400640 1 -- 172.21.15.83:0/1986362490 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0640107be0 con 0x7f0640101350 2024-08-19T01:26:19.260 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:26:19.259+0000 7f0637400640 1 --2- 172.21.15.83:0/1986362490 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f0624039060 0x7f062403b520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:26:19.260 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:26:19.259+0000 7f0637400640 1 -- 172.21.15.83:0/1986362490 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f063006ce40 con 0x7f0640101350 2024-08-19T01:26:19.260 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:26:19.259+0000 7f063de00640 1 --2- 172.21.15.83:0/1986362490 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f0624039060 0x7f062403b520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:26:19.261 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:26:19.260+0000 7f063de00640 1 --2- 172.21.15.83:0/1986362490 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f0624039060 0x7f062403b520 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f062c0098c0 tx=0x7f062c006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:26:19.265 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:26:19.264+0000 7f0637400640 1 -- 172.21.15.83:0/1986362490 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7f0630036da0 con 0x7f0640101350 2024-08-19T01:26:19.456 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:26:19.454+0000 7f0645400640 1 -- 172.21.15.83:0/1986362490 --> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi117", "target": ["mon-mgr", ""]}) -- 0x7f064002d650 con 0x7f0624039060 2024-08-19T01:26:19.905 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:26:19 smithi083 ceph-mon[17120]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' 2024-08-19T01:26:19.905 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:26:19 smithi083 ceph-mon[17120]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' 2024-08-19T01:26:19.905 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:26:19 smithi083 ceph-mon[17120]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi083", "name": "osd_memory_target"}]: dispatch 2024-08-19T01:26:19.905 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:26:19 smithi083 ceph-mon[17120]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-19T01:26:19.905 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:26:19 smithi083 ceph-mon[17120]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-19T01:26:19.905 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:26:19 smithi083 ceph-mon[17120]: Updating smithi083:/etc/ceph/ceph.conf 2024-08-19T01:26:19.905 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:26:19 smithi083 ceph-mon[17120]: Updating smithi083:/var/lib/ceph/bad8780e-5dc9-11ef-bccf-c7b262605968/config/ceph.conf 2024-08-19T01:26:19.905 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:26:19 smithi083 ceph-mon[17120]: from='client.14174 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi117", "target": ["mon-mgr", ""]}]: dispatch 2024-08-19T01:26:20.905 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:26:20 smithi083 ceph-mon[17120]: Updating smithi083:/etc/ceph/ceph.client.admin.keyring 2024-08-19T01:26:20.905 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:26:20 smithi083 ceph-mon[17120]: Updating smithi083:/var/lib/ceph/bad8780e-5dc9-11ef-bccf-c7b262605968/config/ceph.client.admin.keyring 2024-08-19T01:26:20.905 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:26:20 smithi083 ceph-mon[17120]: Deploying cephadm binary to smithi117 2024-08-19T01:26:20.905 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:26:20 smithi083 ceph-mon[17120]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' 2024-08-19T01:26:20.905 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:26:20 smithi083 ceph-mon[17120]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' 2024-08-19T01:26:20.905 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:26:20 smithi083 ceph-mon[17120]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' 2024-08-19T01:26:21.905 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:26:21 smithi083 ceph-mon[17120]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-19T01:26:21.905 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:26:21 smithi083 ceph-mon[17120]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-19T01:26:21.905 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:26:21 smithi083 ceph-mon[17120]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-19T01:26:21.905 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:26:21 smithi083 ceph-mon[17120]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' 2024-08-19T01:26:22.649 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:26:22.646+0000 7f0637400640 1 -- 172.21.15.83:0/1986362490 <== mgr.14150 v2:172.21.15.83:6800/683740916 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+49 (secure 0 0 0) 0x7f064002d650 con 0x7f0624039060 2024-08-19T01:26:22.649 INFO:teuthology.orchestra.run.smithi083.stdout:Added host 'smithi117' with addr '172.21.15.117' 2024-08-19T01:26:22.651 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:26:22.649+0000 7f0645400640 1 -- 172.21.15.83:0/1986362490 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f0624039060 msgr2=0x7f062403b520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:26:22.651 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:26:22.649+0000 7f0645400640 1 --2- 172.21.15.83:0/1986362490 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f0624039060 0x7f062403b520 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f062c0098c0 tx=0x7f062c006db0 comp rx=0 tx=0).stop 2024-08-19T01:26:22.651 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:26:22.649+0000 7f0645400640 1 -- 172.21.15.83:0/1986362490 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0640101350 msgr2=0x7f0640198cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:26:22.652 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:26:22.649+0000 7f0645400640 1 --2- 172.21.15.83:0/1986362490 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0640101350 0x7f0640198cb0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f0630004f60 tx=0x7f06300048f0 comp rx=0 tx=0).stop 2024-08-19T01:26:22.652 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:26:22.649+0000 7f0645400640 1 -- 172.21.15.83:0/1986362490 shutdown_connections 2024-08-19T01:26:22.652 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:26:22.649+0000 7f0645400640 1 --2- 172.21.15.83:0/1986362490 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f0624039060 0x7f062403b520 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:26:22.652 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:26:22.649+0000 7f0645400640 1 --2- 172.21.15.83:0/1986362490 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0640101350 0x7f0640198cb0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:26:22.652 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:26:22.649+0000 7f0645400640 1 -- 172.21.15.83:0/1986362490 >> 172.21.15.83:0/1986362490 conn(0x7f06400fcae0 msgr2=0x7f06400fd550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:26:22.652 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:26:22.650+0000 7f0645400640 1 -- 172.21.15.83:0/1986362490 shutdown_connections 2024-08-19T01:26:22.653 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:26:22.650+0000 7f0645400640 1 -- 172.21.15.83:0/1986362490 wait complete. 2024-08-19T01:26:23.516 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid bad8780e-5dc9-11ef-bccf-c7b262605968 -- ceph orch host ls --format=json 2024-08-19T01:26:23.902 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/bad8780e-5dc9-11ef-bccf-c7b262605968/mon.a/config 2024-08-19T01:26:23.905 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:26:23 smithi083 ceph-mon[17120]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' 2024-08-19T01:26:23.905 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:26:23 smithi083 ceph-mon[17120]: Added host smithi117 2024-08-19T01:26:23.905 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:26:23 smithi083 ceph-mon[17120]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-19T01:26:23.905 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:26:23 smithi083 ceph-mon[17120]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' 2024-08-19T01:26:23.905 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:26:23 smithi083 ceph-mon[17120]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' 2024-08-19T01:26:24.424 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:26:24.389+0000 7fbd25800640 1 -- 172.21.15.83:0/4046470240 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbd20102290 msgr2=0x7fbd20102670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:26:24.424 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:26:24.389+0000 7fbd25800640 1 --2- 172.21.15.83:0/4046470240 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbd20102290 0x7fbd20102670 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fbd10009880 tx=0x7fbd1002a670 comp rx=0 tx=0).stop 2024-08-19T01:26:24.424 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:26:24.389+0000 7fbd25800640 1 -- 172.21.15.83:0/4046470240 shutdown_connections 2024-08-19T01:26:24.424 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:26:24.389+0000 7fbd25800640 1 --2- 172.21.15.83:0/4046470240 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbd20102290 0x7fbd20102670 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:26:24.424 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:26:24.389+0000 7fbd25800640 1 -- 172.21.15.83:0/4046470240 >> 172.21.15.83:0/4046470240 conn(0x7fbd20076ea0 msgr2=0x7fbd200fe1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:26:24.424 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:26:24.389+0000 7fbd25800640 1 -- 172.21.15.83:0/4046470240 shutdown_connections 2024-08-19T01:26:24.424 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:26:24.389+0000 7fbd25800640 1 -- 172.21.15.83:0/4046470240 wait complete. 2024-08-19T01:26:24.425 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:26:24.390+0000 7fbd25800640 1 Processor -- start 2024-08-19T01:26:24.425 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:26:24.390+0000 7fbd25800640 1 -- start start 2024-08-19T01:26:24.425 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:26:24.391+0000 7fbd25800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbd20102290 0x7fbd20196d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:26:24.425 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:26:24.391+0000 7fbd25800640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fbd2010b800 con 0x7fbd20102290 2024-08-19T01:26:24.425 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:26:24.391+0000 7fbd1fe00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbd20102290 0x7fbd20196d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:26:24.425 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:26:24.391+0000 7fbd1fe00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbd20102290 0x7fbd20196d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:39206/0 (socket says 172.21.15.83:39206) 2024-08-19T01:26:24.425 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:26:24.391+0000 7fbd1fe00640 1 -- 172.21.15.83:0/388689105 learned_addr learned my addr 172.21.15.83:0/388689105 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-08-19T01:26:24.425 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:26:24.391+0000 7fbd1fe00640 1 -- 172.21.15.83:0/388689105 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbd201972b0 con 0x7fbd20102290 2024-08-19T01:26:24.425 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:26:24.391+0000 7fbd1fe00640 1 --2- 172.21.15.83:0/388689105 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbd20102290 0x7fbd20196d70 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fbd10009230 tx=0x7fbd10003d40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:26:24.425 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:26:24.392+0000 7fbd1d600640 1 -- 172.21.15.83:0/388689105 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbd1003f070 con 0x7fbd20102290 2024-08-19T01:26:24.425 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:26:24.392+0000 7fbd1d600640 1 -- 172.21.15.83:0/388689105 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbd1003bc80 con 0x7fbd20102290 2024-08-19T01:26:24.425 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:26:24.392+0000 7fbd1d600640 1 -- 172.21.15.83:0/388689105 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbd10035030 con 0x7fbd20102290 2024-08-19T01:26:24.425 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:26:24.392+0000 7fbd25800640 1 -- 172.21.15.83:0/388689105 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbd20197540 con 0x7fbd20102290 2024-08-19T01:26:24.425 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:26:24.392+0000 7fbd25800640 1 -- 172.21.15.83:0/388689105 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbd201979a0 con 0x7fbd20102290 2024-08-19T01:26:24.425 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:26:24.393+0000 7fbd1d600640 1 -- 172.21.15.83:0/388689105 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 13) ==== 46343+0+0 (secure 0 0 0) 0x7fbd10042070 con 0x7fbd20102290 2024-08-19T01:26:24.426 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:26:24.393+0000 7fbd25800640 1 -- 172.21.15.83:0/388689105 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbd20069cc0 con 0x7fbd20102290 2024-08-19T01:26:24.426 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:26:24.393+0000 7fbd1d600640 1 --2- 172.21.15.83:0/388689105 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7fbd040390c0 0x7fbd0403b580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:26:24.426 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:26:24.393+0000 7fbd1d600640 1 -- 172.21.15.83:0/388689105 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fbd1006cd80 con 0x7fbd20102290 2024-08-19T01:26:24.426 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:26:24.394+0000 7fbd17e00640 1 --2- 172.21.15.83:0/388689105 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7fbd040390c0 0x7fbd0403b580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:26:24.426 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:26:24.394+0000 7fbd17e00640 1 --2- 172.21.15.83:0/388689105 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7fbd040390c0 0x7fbd0403b580 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fbd080098c0 tx=0x7fbd08006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:26:24.426 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:26:24.399+0000 7fbd1d600640 1 -- 172.21.15.83:0/388689105 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7fbd100372d0 con 0x7fbd20102290 2024-08-19T01:26:24.580 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:26:24.577+0000 7fbd25800640 1 -- 172.21.15.83:0/388689105 --> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fbd20198130 con 0x7fbd040390c0 2024-08-19T01:26:24.581 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:26:24.579+0000 7fbd1d600640 1 -- 172.21.15.83:0/388689105 <== mgr.14150 v2:172.21.15.83:6800/683740916 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+160 (secure 0 0 0) 0x7fbd20198130 con 0x7fbd040390c0 2024-08-19T01:26:24.581 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-08-19T01:26:24.581 INFO:teuthology.orchestra.run.smithi083.stdout:[{"addr": "172.21.15.83", "hostname": "smithi083", "labels": [], "status": ""}, {"addr": "172.21.15.117", "hostname": "smithi117", "labels": [], "status": ""}] 2024-08-19T01:26:24.583 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:26:24.581+0000 7fbd25800640 1 -- 172.21.15.83:0/388689105 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7fbd040390c0 msgr2=0x7fbd0403b580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:26:24.583 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:26:24.581+0000 7fbd25800640 1 --2- 172.21.15.83:0/388689105 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7fbd040390c0 0x7fbd0403b580 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fbd080098c0 tx=0x7fbd08006db0 comp rx=0 tx=0).stop 2024-08-19T01:26:24.583 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:26:24.581+0000 7fbd25800640 1 -- 172.21.15.83:0/388689105 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbd20102290 msgr2=0x7fbd20196d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:26:24.583 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:26:24.581+0000 7fbd25800640 1 --2- 172.21.15.83:0/388689105 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbd20102290 0x7fbd20196d70 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fbd10009230 tx=0x7fbd10003d40 comp rx=0 tx=0).stop 2024-08-19T01:26:24.584 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:26:24.581+0000 7fbd25800640 1 -- 172.21.15.83:0/388689105 shutdown_connections 2024-08-19T01:26:24.584 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:26:24.581+0000 7fbd25800640 1 --2- 172.21.15.83:0/388689105 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7fbd040390c0 0x7fbd0403b580 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:26:24.584 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:26:24.581+0000 7fbd25800640 1 --2- 172.21.15.83:0/388689105 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbd20102290 0x7fbd20196d70 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:26:24.584 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:26:24.581+0000 7fbd25800640 1 -- 172.21.15.83:0/388689105 >> 172.21.15.83:0/388689105 conn(0x7fbd20076ea0 msgr2=0x7fbd20105050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:26:24.584 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:26:24.581+0000 7fbd25800640 1 -- 172.21.15.83:0/388689105 shutdown_connections 2024-08-19T01:26:24.584 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:26:24.581+0000 7fbd25800640 1 -- 172.21.15.83:0/388689105 wait complete. 2024-08-19T01:26:25.301 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:26:25 smithi083 ceph-mon[17120]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' 2024-08-19T01:26:25.301 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:26:25 smithi083 ceph-mon[17120]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-19T01:26:25.343 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi167 2024-08-19T01:26:25.343 DEBUG:teuthology.orchestra.run.smithi167:> set -ex 2024-08-19T01:26:25.343 DEBUG:teuthology.orchestra.run.smithi167:> dd of=/etc/ceph/ceph.conf 2024-08-19T01:26:25.361 DEBUG:teuthology.orchestra.run.smithi167:> set -ex 2024-08-19T01:26:25.361 DEBUG:teuthology.orchestra.run.smithi167:> dd of=/etc/ceph/ceph.client.admin.keyring 2024-08-19T01:26:25.417 INFO:tasks.cephadm:Adding host smithi167 to orchestrator... 2024-08-19T01:26:25.417 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid bad8780e-5dc9-11ef-bccf-c7b262605968 -- ceph orch host add smithi167 2024-08-19T01:26:25.781 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/bad8780e-5dc9-11ef-bccf-c7b262605968/mon.a/config 2024-08-19T01:26:26.195 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:26:26 smithi083 ceph-mon[17120]: from='client.14176 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-08-19T01:26:26.306 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:26:26.303+0000 7fedcd000640 1 -- 172.21.15.83:0/2594416684 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fedc8109570 msgr2=0x7fedc8109950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:26:26.306 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:26:26.303+0000 7fedcd000640 1 --2- 172.21.15.83:0/2594416684 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fedc8109570 0x7fedc8109950 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fedb8009880 tx=0x7fedb802a5d0 comp rx=0 tx=0).stop 2024-08-19T01:26:26.306 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:26:26.303+0000 7fedcd000640 1 -- 172.21.15.83:0/2594416684 shutdown_connections 2024-08-19T01:26:26.306 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:26:26.304+0000 7fedcd000640 1 --2- 172.21.15.83:0/2594416684 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fedc8109570 0x7fedc8109950 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:26:26.306 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:26:26.304+0000 7fedcd000640 1 -- 172.21.15.83:0/2594416684 >> 172.21.15.83:0/2594416684 conn(0x7fedc80ff180 msgr2=0x7fedc81015a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:26:26.306 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:26:26.304+0000 7fedcd000640 1 -- 172.21.15.83:0/2594416684 shutdown_connections 2024-08-19T01:26:26.307 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:26:26.304+0000 7fedcd000640 1 -- 172.21.15.83:0/2594416684 wait complete. 2024-08-19T01:26:26.307 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:26:26.304+0000 7fedcd000640 1 Processor -- start 2024-08-19T01:26:26.307 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:26:26.305+0000 7fedcd000640 1 -- start start 2024-08-19T01:26:26.308 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:26:26.305+0000 7fedcd000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fedc8109570 0x7fedc8199040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:26:26.308 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:26:26.305+0000 7fedcd000640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fedc810dba0 con 0x7fedc8109570 2024-08-19T01:26:26.308 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:26:26.305+0000 7fedc5e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fedc8109570 0x7fedc8199040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:26:26.308 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:26:26.305+0000 7fedc5e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fedc8109570 0x7fedc8199040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:39224/0 (socket says 172.21.15.83:39224) 2024-08-19T01:26:26.308 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:26:26.305+0000 7fedc5e00640 1 -- 172.21.15.83:0/1859584447 learned_addr learned my addr 172.21.15.83:0/1859584447 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-08-19T01:26:26.308 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:26:26.306+0000 7fedc5e00640 1 -- 172.21.15.83:0/1859584447 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fedc8199580 con 0x7fedc8109570 2024-08-19T01:26:26.309 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:26:26.306+0000 7fedc5e00640 1 --2- 172.21.15.83:0/1859584447 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fedc8109570 0x7fedc8199040 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fedb8009230 tx=0x7fedb8003d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:26:26.309 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:26:26.306+0000 7fedbea00640 1 -- 172.21.15.83:0/1859584447 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fedb803f070 con 0x7fedc8109570 2024-08-19T01:26:26.309 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:26:26.306+0000 7fedbea00640 1 -- 172.21.15.83:0/1859584447 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fedb803bd40 con 0x7fedc8109570 2024-08-19T01:26:26.309 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:26:26.306+0000 7fedcd000640 1 -- 172.21.15.83:0/1859584447 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fedc8199810 con 0x7fedc8109570 2024-08-19T01:26:26.309 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:26:26.306+0000 7fedbea00640 1 -- 172.21.15.83:0/1859584447 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fedb8035030 con 0x7fedc8109570 2024-08-19T01:26:26.309 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:26:26.306+0000 7fedcd000640 1 -- 172.21.15.83:0/1859584447 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fedc819d910 con 0x7fedc8109570 2024-08-19T01:26:26.310 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:26:26.307+0000 7fedbea00640 1 -- 172.21.15.83:0/1859584447 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 13) ==== 46343+0+0 (secure 0 0 0) 0x7fedb8042070 con 0x7fedc8109570 2024-08-19T01:26:26.310 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:26:26.307+0000 7fedcd000640 1 -- 172.21.15.83:0/1859584447 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fedc8104be0 con 0x7fedc8109570 2024-08-19T01:26:26.310 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:26:26.308+0000 7fedbea00640 1 --2- 172.21.15.83:0/1859584447 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7fedac0390c0 0x7fedac03b580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:26:26.310 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:26:26.308+0000 7fedbea00640 1 -- 172.21.15.83:0/1859584447 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fedb806ccd0 con 0x7fedc8109570 2024-08-19T01:26:26.310 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:26:26.308+0000 7fedc5400640 1 --2- 172.21.15.83:0/1859584447 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7fedac0390c0 0x7fedac03b580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:26:26.311 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:26:26.309+0000 7fedc5400640 1 --2- 172.21.15.83:0/1859584447 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7fedac0390c0 0x7fedac03b580 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fedb40098c0 tx=0x7fedb4006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:26:26.314 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:26:26.312+0000 7fedbea00640 1 -- 172.21.15.83:0/1859584447 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7fedb80412c0 con 0x7fedc8109570 2024-08-19T01:26:26.491 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:26:26.488+0000 7fedcd000640 1 -- 172.21.15.83:0/1859584447 --> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi167", "target": ["mon-mgr", ""]}) -- 0x7fedc819a3a0 con 0x7fedac0390c0 2024-08-19T01:26:27.405 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:26:27 smithi083 ceph-mon[17120]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-19T01:26:27.405 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:26:27 smithi083 ceph-mon[17120]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' 2024-08-19T01:26:27.405 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:26:27 smithi083 ceph-mon[17120]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' 2024-08-19T01:26:27.405 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:26:27 smithi083 ceph-mon[17120]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' 2024-08-19T01:26:27.405 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:26:27 smithi083 ceph-mon[17120]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' 2024-08-19T01:26:27.405 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:26:27 smithi083 ceph-mon[17120]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi117", "name": "osd_memory_target"}]: dispatch 2024-08-19T01:26:27.405 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:26:27 smithi083 ceph-mon[17120]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-19T01:26:27.405 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:26:27 smithi083 ceph-mon[17120]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-19T01:26:28.406 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:26:28 smithi083 ceph-mon[17120]: from='client.14178 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi167", "target": ["mon-mgr", ""]}]: dispatch 2024-08-19T01:26:28.406 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:26:28 smithi083 ceph-mon[17120]: Updating smithi117:/etc/ceph/ceph.conf 2024-08-19T01:26:28.406 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:26:28 smithi083 ceph-mon[17120]: Updating smithi117:/var/lib/ceph/bad8780e-5dc9-11ef-bccf-c7b262605968/config/ceph.conf 2024-08-19T01:26:28.406 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:26:28 smithi083 ceph-mon[17120]: Updating smithi117:/etc/ceph/ceph.client.admin.keyring 2024-08-19T01:26:28.406 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:26:28 smithi083 ceph-mon[17120]: Updating smithi117:/var/lib/ceph/bad8780e-5dc9-11ef-bccf-c7b262605968/config/ceph.client.admin.keyring 2024-08-19T01:26:28.406 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:26:28 smithi083 ceph-mon[17120]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' 2024-08-19T01:26:28.406 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:26:28 smithi083 ceph-mon[17120]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' 2024-08-19T01:26:28.406 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:26:28 smithi083 ceph-mon[17120]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' 2024-08-19T01:26:29.405 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:26:29 smithi083 ceph-mon[17120]: Deploying cephadm binary to smithi167 2024-08-19T01:26:29.405 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:26:29 smithi083 ceph-mon[17120]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-19T01:26:29.809 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:26:29.807+0000 7fedbea00640 1 -- 172.21.15.83:0/1859584447 <== mgr.14150 v2:172.21.15.83:6800/683740916 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+49 (secure 0 0 0) 0x7fedc819a3a0 con 0x7fedac0390c0 2024-08-19T01:26:29.810 INFO:teuthology.orchestra.run.smithi083.stdout:Added host 'smithi167' with addr '172.21.15.167' 2024-08-19T01:26:29.812 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:26:29.810+0000 7fedcd000640 1 -- 172.21.15.83:0/1859584447 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7fedac0390c0 msgr2=0x7fedac03b580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:26:29.812 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:26:29.810+0000 7fedcd000640 1 --2- 172.21.15.83:0/1859584447 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7fedac0390c0 0x7fedac03b580 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fedb40098c0 tx=0x7fedb4006db0 comp rx=0 tx=0).stop 2024-08-19T01:26:29.812 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:26:29.810+0000 7fedcd000640 1 -- 172.21.15.83:0/1859584447 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fedc8109570 msgr2=0x7fedc8199040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:26:29.813 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:26:29.810+0000 7fedcd000640 1 --2- 172.21.15.83:0/1859584447 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fedc8109570 0x7fedc8199040 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fedb8009230 tx=0x7fedb8003d80 comp rx=0 tx=0).stop 2024-08-19T01:26:29.813 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:26:29.810+0000 7fedcd000640 1 -- 172.21.15.83:0/1859584447 shutdown_connections 2024-08-19T01:26:29.813 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:26:29.810+0000 7fedcd000640 1 --2- 172.21.15.83:0/1859584447 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7fedac0390c0 0x7fedac03b580 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:26:29.813 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:26:29.810+0000 7fedcd000640 1 --2- 172.21.15.83:0/1859584447 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fedc8109570 0x7fedc8199040 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:26:29.813 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:26:29.810+0000 7fedcd000640 1 -- 172.21.15.83:0/1859584447 >> 172.21.15.83:0/1859584447 conn(0x7fedc80ff180 msgr2=0x7fedc8100a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:26:29.813 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:26:29.810+0000 7fedcd000640 1 -- 172.21.15.83:0/1859584447 shutdown_connections 2024-08-19T01:26:29.813 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:26:29.810+0000 7fedcd000640 1 -- 172.21.15.83:0/1859584447 wait complete. 2024-08-19T01:26:30.487 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid bad8780e-5dc9-11ef-bccf-c7b262605968 -- ceph orch host ls --format=json 2024-08-19T01:26:30.850 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/bad8780e-5dc9-11ef-bccf-c7b262605968/mon.a/config 2024-08-19T01:26:31.101 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:26:30 smithi083 ceph-mon[17120]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' 2024-08-19T01:26:31.101 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:26:30 smithi083 ceph-mon[17120]: Added host smithi167 2024-08-19T01:26:31.102 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:26:30 smithi083 ceph-mon[17120]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-19T01:26:31.102 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:26:30 smithi083 ceph-mon[17120]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' 2024-08-19T01:26:31.102 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:26:30 smithi083 ceph-mon[17120]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-19T01:26:31.102 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:26:30 smithi083 ceph-mon[17120]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' 2024-08-19T01:26:31.371 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:26:31.368+0000 7f50f9600640 1 -- 172.21.15.83:0/1097488233 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f50f41095c0 msgr2=0x7f50f41099a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:26:31.371 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:26:31.368+0000 7f50f9600640 1 --2- 172.21.15.83:0/1097488233 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f50f41095c0 0x7f50f41099a0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f50e4009880 tx=0x7f50e402a5d0 comp rx=0 tx=0).stop 2024-08-19T01:26:31.371 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:26:31.369+0000 7f50f9600640 1 -- 172.21.15.83:0/1097488233 shutdown_connections 2024-08-19T01:26:31.371 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:26:31.369+0000 7f50f9600640 1 --2- 172.21.15.83:0/1097488233 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f50f41095c0 0x7f50f41099a0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:26:31.371 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:26:31.369+0000 7f50f9600640 1 -- 172.21.15.83:0/1097488233 >> 172.21.15.83:0/1097488233 conn(0x7f50f40ff1b0 msgr2=0x7f50f41015d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:26:31.371 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:26:31.369+0000 7f50f9600640 1 -- 172.21.15.83:0/1097488233 shutdown_connections 2024-08-19T01:26:31.371 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:26:31.369+0000 7f50f9600640 1 -- 172.21.15.83:0/1097488233 wait complete. 2024-08-19T01:26:31.372 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:26:31.370+0000 7f50f9600640 1 Processor -- start 2024-08-19T01:26:31.372 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:26:31.370+0000 7f50f9600640 1 -- start start 2024-08-19T01:26:31.372 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:26:31.370+0000 7f50f9600640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f50f41095c0 0x7f50f4199060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:26:31.372 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:26:31.370+0000 7f50f9600640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f50f410dbf0 con 0x7f50f41095c0 2024-08-19T01:26:31.372 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:26:31.370+0000 7f50f2a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f50f41095c0 0x7f50f4199060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:26:31.373 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:26:31.370+0000 7f50f2a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f50f41095c0 0x7f50f4199060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:53808/0 (socket says 172.21.15.83:53808) 2024-08-19T01:26:31.373 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:26:31.370+0000 7f50f2a00640 1 -- 172.21.15.83:0/2412045241 learned_addr learned my addr 172.21.15.83:0/2412045241 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-08-19T01:26:31.373 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:26:31.371+0000 7f50f2a00640 1 -- 172.21.15.83:0/2412045241 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f50f41995a0 con 0x7f50f41095c0 2024-08-19T01:26:31.373 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:26:31.371+0000 7f50f2a00640 1 --2- 172.21.15.83:0/2412045241 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f50f41095c0 0x7f50f4199060 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f50e4009230 tx=0x7f50e4003d40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:26:31.374 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:26:31.371+0000 7f50ebe00640 1 -- 172.21.15.83:0/2412045241 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f50e403f070 con 0x7f50f41095c0 2024-08-19T01:26:31.374 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:26:31.371+0000 7f50ebe00640 1 -- 172.21.15.83:0/2412045241 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f50e403bc80 con 0x7f50f41095c0 2024-08-19T01:26:31.374 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:26:31.371+0000 7f50f9600640 1 -- 172.21.15.83:0/2412045241 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f50f4199830 con 0x7f50f41095c0 2024-08-19T01:26:31.374 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:26:31.371+0000 7f50ebe00640 1 -- 172.21.15.83:0/2412045241 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f50e4035030 con 0x7f50f41095c0 2024-08-19T01:26:31.374 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:26:31.371+0000 7f50f9600640 1 -- 172.21.15.83:0/2412045241 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f50f419d930 con 0x7f50f41095c0 2024-08-19T01:26:31.374 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:26:31.372+0000 7f50ebe00640 1 -- 172.21.15.83:0/2412045241 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 13) ==== 46343+0+0 (secure 0 0 0) 0x7f50e4042070 con 0x7f50f41095c0 2024-08-19T01:26:31.374 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:26:31.372+0000 7f50f9600640 1 -- 172.21.15.83:0/2412045241 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f50f4104c30 con 0x7f50f41095c0 2024-08-19T01:26:31.375 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:26:31.373+0000 7f50ebe00640 1 --2- 172.21.15.83:0/2412045241 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f50d8039060 0x7f50d803b520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:26:31.375 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:26:31.373+0000 7f50ebe00640 1 -- 172.21.15.83:0/2412045241 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f50e406cdc0 con 0x7f50f41095c0 2024-08-19T01:26:31.375 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:26:31.373+0000 7f50ea000640 1 --2- 172.21.15.83:0/2412045241 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f50d8039060 0x7f50d803b520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:26:31.375 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:26:31.374+0000 7f50ea000640 1 --2- 172.21.15.83:0/2412045241 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f50d8039060 0x7f50d803b520 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f50dc0098c0 tx=0x7f50dc006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:26:31.380 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:26:31.378+0000 7f50ebe00640 1 -- 172.21.15.83:0/2412045241 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7f50e4031e20 con 0x7f50f41095c0 2024-08-19T01:26:31.558 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:26:31.556+0000 7f50f9600640 1 -- 172.21.15.83:0/2412045241 --> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f50f419a3c0 con 0x7f50d8039060 2024-08-19T01:26:31.559 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:26:31.557+0000 7f50ebe00640 1 -- 172.21.15.83:0/2412045241 <== mgr.14150 v2:172.21.15.83:6800/683740916 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+240 (secure 0 0 0) 0x7f50f419a3c0 con 0x7f50d8039060 2024-08-19T01:26:31.560 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-08-19T01:26:31.560 INFO:teuthology.orchestra.run.smithi083.stdout:[{"addr": "172.21.15.83", "hostname": "smithi083", "labels": [], "status": ""}, {"addr": "172.21.15.117", "hostname": "smithi117", "labels": [], "status": ""}, {"addr": "172.21.15.167", "hostname": "smithi167", "labels": [], "status": ""}] 2024-08-19T01:26:31.562 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:26:31.560+0000 7f50f9600640 1 -- 172.21.15.83:0/2412045241 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f50d8039060 msgr2=0x7f50d803b520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:26:31.562 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:26:31.560+0000 7f50f9600640 1 --2- 172.21.15.83:0/2412045241 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f50d8039060 0x7f50d803b520 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f50dc0098c0 tx=0x7f50dc006db0 comp rx=0 tx=0).stop 2024-08-19T01:26:31.562 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:26:31.560+0000 7f50f9600640 1 -- 172.21.15.83:0/2412045241 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f50f41095c0 msgr2=0x7f50f4199060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:26:31.562 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:26:31.560+0000 7f50f9600640 1 --2- 172.21.15.83:0/2412045241 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f50f41095c0 0x7f50f4199060 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f50e4009230 tx=0x7f50e4003d40 comp rx=0 tx=0).stop 2024-08-19T01:26:31.562 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:26:31.560+0000 7f50f9600640 1 -- 172.21.15.83:0/2412045241 shutdown_connections 2024-08-19T01:26:31.562 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:26:31.560+0000 7f50f9600640 1 --2- 172.21.15.83:0/2412045241 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f50d8039060 0x7f50d803b520 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:26:31.562 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:26:31.560+0000 7f50f9600640 1 --2- 172.21.15.83:0/2412045241 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f50f41095c0 0x7f50f4199060 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:26:31.562 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:26:31.560+0000 7f50f9600640 1 -- 172.21.15.83:0/2412045241 >> 172.21.15.83:0/2412045241 conn(0x7f50f40ff1b0 msgr2=0x7f50f40ff590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:26:31.563 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:26:31.560+0000 7f50f9600640 1 -- 172.21.15.83:0/2412045241 shutdown_connections 2024-08-19T01:26:31.563 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:26:31.560+0000 7f50f9600640 1 -- 172.21.15.83:0/2412045241 wait complete. 2024-08-19T01:26:32.187 INFO:tasks.cephadm:Setting crush tunables to default 2024-08-19T01:26:32.187 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid bad8780e-5dc9-11ef-bccf-c7b262605968 -- ceph osd crush tunables default 2024-08-19T01:26:32.405 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:26:32 smithi083 ceph-mon[17120]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' 2024-08-19T01:26:32.551 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/bad8780e-5dc9-11ef-bccf-c7b262605968/mon.a/config 2024-08-19T01:26:33.043 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:26:33.041+0000 7fc910200640 1 -- 172.21.15.83:0/303348944 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc9081095c0 msgr2=0x7fc9081099a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:26:33.044 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:26:33.041+0000 7fc910200640 1 --2- 172.21.15.83:0/303348944 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc9081095c0 0x7fc9081099a0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fc8fc009880 tx=0x7fc8fc02a5d0 comp rx=0 tx=0).stop 2024-08-19T01:26:33.044 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:26:33.041+0000 7fc910200640 1 -- 172.21.15.83:0/303348944 shutdown_connections 2024-08-19T01:26:33.044 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:26:33.041+0000 7fc910200640 1 --2- 172.21.15.83:0/303348944 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc9081095c0 0x7fc9081099a0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:26:33.044 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:26:33.041+0000 7fc910200640 1 -- 172.21.15.83:0/303348944 >> 172.21.15.83:0/303348944 conn(0x7fc9080ff1b0 msgr2=0x7fc9081015d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:26:33.044 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:26:33.041+0000 7fc910200640 1 -- 172.21.15.83:0/303348944 shutdown_connections 2024-08-19T01:26:33.044 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:26:33.041+0000 7fc910200640 1 -- 172.21.15.83:0/303348944 wait complete. 2024-08-19T01:26:33.045 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:26:33.042+0000 7fc910200640 1 Processor -- start 2024-08-19T01:26:33.045 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:26:33.042+0000 7fc910200640 1 -- start start 2024-08-19T01:26:33.045 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:26:33.043+0000 7fc910200640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc9081095c0 0x7fc908199060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:26:33.045 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:26:33.043+0000 7fc910200640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fc90810dbf0 con 0x7fc9081095c0 2024-08-19T01:26:33.045 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:26:33.043+0000 7fc90d800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc9081095c0 0x7fc908199060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:26:33.046 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:26:33.043+0000 7fc90d800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc9081095c0 0x7fc908199060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:53834/0 (socket says 172.21.15.83:53834) 2024-08-19T01:26:33.046 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:26:33.043+0000 7fc90d800640 1 -- 172.21.15.83:0/4167846443 learned_addr learned my addr 172.21.15.83:0/4167846443 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-08-19T01:26:33.046 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:26:33.043+0000 7fc90d800640 1 -- 172.21.15.83:0/4167846443 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc9081995a0 con 0x7fc9081095c0 2024-08-19T01:26:33.047 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:26:33.043+0000 7fc90d800640 1 --2- 172.21.15.83:0/4167846443 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc9081095c0 0x7fc908199060 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fc8fc009230 tx=0x7fc8fc003d40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:26:33.047 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:26:33.044+0000 7fc906000640 1 -- 172.21.15.83:0/4167846443 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc8fc03f070 con 0x7fc9081095c0 2024-08-19T01:26:33.047 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:26:33.044+0000 7fc906000640 1 -- 172.21.15.83:0/4167846443 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc8fc03bc80 con 0x7fc9081095c0 2024-08-19T01:26:33.047 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:26:33.044+0000 7fc910200640 1 -- 172.21.15.83:0/4167846443 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc908199830 con 0x7fc9081095c0 2024-08-19T01:26:33.047 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:26:33.044+0000 7fc906000640 1 -- 172.21.15.83:0/4167846443 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc8fc035030 con 0x7fc9081095c0 2024-08-19T01:26:33.047 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:26:33.044+0000 7fc910200640 1 -- 172.21.15.83:0/4167846443 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc90819d930 con 0x7fc9081095c0 2024-08-19T01:26:33.047 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:26:33.045+0000 7fc906000640 1 -- 172.21.15.83:0/4167846443 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 13) ==== 46343+0+0 (secure 0 0 0) 0x7fc8fc042070 con 0x7fc9081095c0 2024-08-19T01:26:33.047 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:26:33.045+0000 7fc910200640 1 -- 172.21.15.83:0/4167846443 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc908104c30 con 0x7fc9081095c0 2024-08-19T01:26:33.048 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:26:33.045+0000 7fc906000640 1 --2- 172.21.15.83:0/4167846443 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7fc8f00390c0 0x7fc8f003b580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:26:33.048 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:26:33.045+0000 7fc906000640 1 -- 172.21.15.83:0/4167846443 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fc8fc06cd80 con 0x7fc9081095c0 2024-08-19T01:26:33.049 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:26:33.046+0000 7fc90ce00640 1 --2- 172.21.15.83:0/4167846443 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7fc8f00390c0 0x7fc8f003b580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:26:33.049 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:26:33.046+0000 7fc90ce00640 1 --2- 172.21.15.83:0/4167846443 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7fc8f00390c0 0x7fc8f003b580 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7fc8f40098c0 tx=0x7fc8f4006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:26:33.052 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:26:33.050+0000 7fc906000640 1 -- 172.21.15.83:0/4167846443 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7fc8fc0372d0 con 0x7fc9081095c0 2024-08-19T01:26:33.234 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:26:33.231+0000 7fc910200640 1 -- 172.21.15.83:0/4167846443 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) -- 0x7fc90819a3c0 con 0x7fc9081095c0 2024-08-19T01:26:33.234 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:26:33 smithi083 ceph-mon[17120]: from='client.14180 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-08-19T01:26:33.234 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:26:33 smithi083 ceph-mon[17120]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-19T01:26:34.169 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:26:34.167+0000 7fc906000640 1 -- 172.21.15.83:0/4167846443 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v4) ==== 124+0+0 (secure 0 0 0) 0x7fc8fc031dd0 con 0x7fc9081095c0 2024-08-19T01:26:34.169 INFO:teuthology.orchestra.run.smithi083.stderr:adjusted tunables profile to default 2024-08-19T01:26:34.172 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:26:34.169+0000 7fc910200640 1 -- 172.21.15.83:0/4167846443 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7fc8f00390c0 msgr2=0x7fc8f003b580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:26:34.172 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:26:34.169+0000 7fc910200640 1 --2- 172.21.15.83:0/4167846443 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7fc8f00390c0 0x7fc8f003b580 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7fc8f40098c0 tx=0x7fc8f4006db0 comp rx=0 tx=0).stop 2024-08-19T01:26:34.172 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:26:34.169+0000 7fc910200640 1 -- 172.21.15.83:0/4167846443 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc9081095c0 msgr2=0x7fc908199060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:26:34.172 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:26:34.169+0000 7fc910200640 1 --2- 172.21.15.83:0/4167846443 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc9081095c0 0x7fc908199060 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fc8fc009230 tx=0x7fc8fc003d40 comp rx=0 tx=0).stop 2024-08-19T01:26:34.172 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:26:34.169+0000 7fc910200640 1 -- 172.21.15.83:0/4167846443 shutdown_connections 2024-08-19T01:26:34.172 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:26:34.169+0000 7fc910200640 1 --2- 172.21.15.83:0/4167846443 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7fc8f00390c0 0x7fc8f003b580 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:26:34.172 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:26:34.170+0000 7fc910200640 1 --2- 172.21.15.83:0/4167846443 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc9081095c0 0x7fc908199060 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:26:34.172 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:26:34.170+0000 7fc910200640 1 -- 172.21.15.83:0/4167846443 >> 172.21.15.83:0/4167846443 conn(0x7fc9080ff1b0 msgr2=0x7fc9080ff590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:26:34.172 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:26:34.170+0000 7fc910200640 1 -- 172.21.15.83:0/4167846443 shutdown_connections 2024-08-19T01:26:34.173 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:26:34.170+0000 7fc910200640 1 -- 172.21.15.83:0/4167846443 wait complete. 2024-08-19T01:26:34.562 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:26:34 smithi083 ceph-mon[17120]: from='client.? 172.21.15.83:0/4167846443' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-08-19T01:26:34.563 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:26:34 smithi083 ceph-mon[17120]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' 2024-08-19T01:26:34.563 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:26:34 smithi083 ceph-mon[17120]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' 2024-08-19T01:26:34.563 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:26:34 smithi083 ceph-mon[17120]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' 2024-08-19T01:26:34.563 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:26:34 smithi083 ceph-mon[17120]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' 2024-08-19T01:26:34.563 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:26:34 smithi083 ceph-mon[17120]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi167", "name": "osd_memory_target"}]: dispatch 2024-08-19T01:26:34.563 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:26:34 smithi083 ceph-mon[17120]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-19T01:26:34.563 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:26:34 smithi083 ceph-mon[17120]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-19T01:26:34.943 INFO:tasks.cephadm:Adding mon.a on smithi083 2024-08-19T01:26:34.943 INFO:tasks.cephadm:Adding mon.b on smithi117 2024-08-19T01:26:34.944 INFO:tasks.cephadm:Adding mon.c on smithi167 2024-08-19T01:26:34.944 DEBUG:teuthology.orchestra.run.smithi167:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid bad8780e-5dc9-11ef-bccf-c7b262605968 -- ceph orch apply mon '3;smithi083:172.21.15.83=a;smithi117:172.21.15.117=b;smithi167:172.21.15.167=c' 2024-08-19T01:26:35.183 INFO:teuthology.orchestra.run.smithi167.stderr:Inferring config /var/lib/ceph/bad8780e-5dc9-11ef-bccf-c7b262605968/config/ceph.conf 2024-08-19T01:26:35.573 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-19T01:26:35.570+0000 7f12696c5640 1 -- 172.21.15.167:0/3667978736 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1264101520 msgr2=0x7f1264101900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:26:35.573 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-19T01:26:35.570+0000 7f12696c5640 1 --2- 172.21.15.167:0/3667978736 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1264101520 0x7f1264101900 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f1254009880 tx=0x7f125402a5d0 comp rx=0 tx=0).stop 2024-08-19T01:26:35.573 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-19T01:26:35.570+0000 7f12696c5640 1 -- 172.21.15.167:0/3667978736 shutdown_connections 2024-08-19T01:26:35.573 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-19T01:26:35.570+0000 7f12696c5640 1 --2- 172.21.15.167:0/3667978736 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1264101520 0x7f1264101900 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:26:35.573 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-19T01:26:35.570+0000 7f12696c5640 1 -- 172.21.15.167:0/3667978736 >> 172.21.15.167:0/3667978736 conn(0x7f12640fcfa0 msgr2=0x7f12640ff3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:26:35.573 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-19T01:26:35.570+0000 7f12696c5640 1 -- 172.21.15.167:0/3667978736 shutdown_connections 2024-08-19T01:26:35.573 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-19T01:26:35.570+0000 7f12696c5640 1 -- 172.21.15.167:0/3667978736 wait complete. 2024-08-19T01:26:35.573 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-19T01:26:35.571+0000 7f12696c5640 1 Processor -- start 2024-08-19T01:26:35.573 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-19T01:26:35.571+0000 7f12696c5640 1 -- start start 2024-08-19T01:26:35.574 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-19T01:26:35.572+0000 7f12696c5640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1264101520 0x7f1264196ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:26:35.575 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-19T01:26:35.572+0000 7f12696c5640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f126410b970 con 0x7f1264101520 2024-08-19T01:26:35.575 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-19T01:26:35.572+0000 7f1262ffd640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1264101520 0x7f1264196ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:26:35.575 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-19T01:26:35.572+0000 7f1262ffd640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1264101520 0x7f1264196ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.167:35536/0 (socket says 172.21.15.167:35536) 2024-08-19T01:26:35.575 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-19T01:26:35.572+0000 7f1262ffd640 1 -- 172.21.15.167:0/721403565 learned_addr learned my addr 172.21.15.167:0/721403565 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-08-19T01:26:35.575 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-19T01:26:35.572+0000 7f1262ffd640 1 -- 172.21.15.167:0/721403565 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1264197400 con 0x7f1264101520 2024-08-19T01:26:35.575 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-19T01:26:35.572+0000 7f1262ffd640 1 --2- 172.21.15.167:0/721403565 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1264101520 0x7f1264196ec0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f1254009230 tx=0x7f1254003d40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:26:35.575 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-19T01:26:35.573+0000 7f124bfff640 1 -- 172.21.15.167:0/721403565 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f125403f070 con 0x7f1264101520 2024-08-19T01:26:35.575 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-19T01:26:35.573+0000 7f12696c5640 1 -- 172.21.15.167:0/721403565 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f12641976f0 con 0x7f1264101520 2024-08-19T01:26:35.575 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-19T01:26:35.573+0000 7f124bfff640 1 -- 172.21.15.167:0/721403565 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f125403bc80 con 0x7f1264101520 2024-08-19T01:26:35.575 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-19T01:26:35.573+0000 7f124bfff640 1 -- 172.21.15.167:0/721403565 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1254049060 con 0x7f1264101520 2024-08-19T01:26:35.575 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-19T01:26:35.573+0000 7f12696c5640 1 -- 172.21.15.167:0/721403565 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f126419b790 con 0x7f1264101520 2024-08-19T01:26:35.576 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-19T01:26:35.574+0000 7f12696c5640 1 -- 172.21.15.167:0/721403565 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1264102a00 con 0x7f1264101520 2024-08-19T01:26:35.576 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-19T01:26:35.574+0000 7f124bfff640 1 -- 172.21.15.167:0/721403565 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 13) ==== 46343+0+0 (secure 0 0 0) 0x7f1254042070 con 0x7f1264101520 2024-08-19T01:26:35.576 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-19T01:26:35.574+0000 7f124bfff640 1 --2- 172.21.15.167:0/721403565 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f124c039060 0x7f124c03b520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:26:35.576 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-19T01:26:35.574+0000 7f124bfff640 1 -- 172.21.15.167:0/721403565 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f125406cc70 con 0x7f1264101520 2024-08-19T01:26:35.576 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-19T01:26:35.575+0000 7f12627fc640 1 --2- 172.21.15.167:0/721403565 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f124c039060 0x7f124c03b520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:26:35.577 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-19T01:26:35.575+0000 7f12627fc640 1 --2- 172.21.15.167:0/721403565 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f124c039060 0x7f124c03b520 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f12580098c0 tx=0x7f1258006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:26:35.582 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-19T01:26:35.579+0000 7f124bfff640 1 -- 172.21.15.167:0/721403565 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7f1254031d40 con 0x7f1264101520 2024-08-19T01:26:35.655 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:26:35 smithi083 ceph-mon[17120]: Updating smithi167:/etc/ceph/ceph.conf 2024-08-19T01:26:35.655 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:26:35 smithi083 ceph-mon[17120]: from='client.? 172.21.15.83:0/4167846443' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-08-19T01:26:35.655 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:26:35 smithi083 ceph-mon[17120]: osdmap e4: 0 total, 0 up, 0 in 2024-08-19T01:26:35.656 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:26:35 smithi083 ceph-mon[17120]: Updating smithi167:/var/lib/ceph/bad8780e-5dc9-11ef-bccf-c7b262605968/config/ceph.conf 2024-08-19T01:26:35.656 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:26:35 smithi083 ceph-mon[17120]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-19T01:26:35.656 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:26:35 smithi083 ceph-mon[17120]: Updating smithi167:/etc/ceph/ceph.client.admin.keyring 2024-08-19T01:26:35.656 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:26:35 smithi083 ceph-mon[17120]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' 2024-08-19T01:26:35.656 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:26:35 smithi083 ceph-mon[17120]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' 2024-08-19T01:26:35.656 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:26:35 smithi083 ceph-mon[17120]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' 2024-08-19T01:26:35.767 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-19T01:26:35.765+0000 7f12696c5640 1 -- 172.21.15.167:0/721403565 --> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi083:172.21.15.83=a;smithi117:172.21.15.117=b;smithi167:172.21.15.167=c", "target": ["mon-mgr", ""]}) -- 0x7f1264198220 con 0x7f124c039060 2024-08-19T01:26:35.774 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-19T01:26:35.772+0000 7f124bfff640 1 -- 172.21.15.167:0/721403565 <== mgr.14150 v2:172.21.15.83:6800/683740916 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7f1264198220 con 0x7f124c039060 2024-08-19T01:26:35.774 INFO:teuthology.orchestra.run.smithi167.stdout:Scheduled mon update... 2024-08-19T01:26:35.776 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-19T01:26:35.774+0000 7f12696c5640 1 -- 172.21.15.167:0/721403565 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f124c039060 msgr2=0x7f124c03b520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:26:35.776 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-19T01:26:35.774+0000 7f12696c5640 1 --2- 172.21.15.167:0/721403565 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f124c039060 0x7f124c03b520 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f12580098c0 tx=0x7f1258006db0 comp rx=0 tx=0).stop 2024-08-19T01:26:35.776 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-19T01:26:35.774+0000 7f12696c5640 1 -- 172.21.15.167:0/721403565 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1264101520 msgr2=0x7f1264196ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:26:35.776 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-19T01:26:35.774+0000 7f12696c5640 1 --2- 172.21.15.167:0/721403565 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1264101520 0x7f1264196ec0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f1254009230 tx=0x7f1254003d40 comp rx=0 tx=0).stop 2024-08-19T01:26:35.776 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-19T01:26:35.774+0000 7f12696c5640 1 -- 172.21.15.167:0/721403565 shutdown_connections 2024-08-19T01:26:35.776 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-19T01:26:35.774+0000 7f12696c5640 1 --2- 172.21.15.167:0/721403565 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f124c039060 0x7f124c03b520 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:26:35.776 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-19T01:26:35.774+0000 7f12696c5640 1 --2- 172.21.15.167:0/721403565 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1264101520 0x7f1264196ec0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:26:35.776 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-19T01:26:35.774+0000 7f12696c5640 1 -- 172.21.15.167:0/721403565 >> 172.21.15.167:0/721403565 conn(0x7f12640fcfa0 msgr2=0x7f1264108ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:26:35.777 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-19T01:26:35.774+0000 7f12696c5640 1 -- 172.21.15.167:0/721403565 shutdown_connections 2024-08-19T01:26:35.777 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-19T01:26:35.774+0000 7f12696c5640 1 -- 172.21.15.167:0/721403565 wait complete. 2024-08-19T01:26:36.364 DEBUG:teuthology.orchestra.run.smithi117:mon.b> sudo journalctl -f -n 0 -u ceph-bad8780e-5dc9-11ef-bccf-c7b262605968@mon.b.service 2024-08-19T01:26:36.367 DEBUG:teuthology.orchestra.run.smithi167:mon.c> sudo journalctl -f -n 0 -u ceph-bad8780e-5dc9-11ef-bccf-c7b262605968@mon.c.service 2024-08-19T01:26:36.369 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-08-19T01:26:36.370 DEBUG:teuthology.orchestra.run.smithi167:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid bad8780e-5dc9-11ef-bccf-c7b262605968 -- ceph mon dump -f json 2024-08-19T01:26:36.655 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:26:36 smithi083 ceph-mon[17120]: Updating smithi167:/var/lib/ceph/bad8780e-5dc9-11ef-bccf-c7b262605968/config/ceph.client.admin.keyring 2024-08-19T01:26:36.655 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:26:36 smithi083 ceph-mon[17120]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' 2024-08-19T01:26:36.655 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:26:36 smithi083 ceph-mon[17120]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-19T01:26:36.655 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:26:36 smithi083 ceph-mon[17120]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-19T01:26:36.655 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:26:36 smithi083 ceph-mon[17120]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-19T01:26:36.655 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:26:36 smithi083 ceph-mon[17120]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' 2024-08-19T01:26:36.656 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:26:36 smithi083 ceph-mon[17120]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-08-19T01:26:36.656 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:26:36 smithi083 ceph-mon[17120]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-19T01:26:37.488 INFO:teuthology.orchestra.run.smithi167.stderr:Inferring config /var/lib/ceph/bad8780e-5dc9-11ef-bccf-c7b262605968/config/ceph.conf 2024-08-19T01:26:37.655 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:26:37 smithi083 ceph-mon[17120]: from='client.14184 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi083:172.21.15.83=a;smithi117:172.21.15.117=b;smithi167:172.21.15.167=c", "target": ["mon-mgr", ""]}]: dispatch 2024-08-19T01:26:37.655 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:26:37 smithi083 ceph-mon[17120]: Saving service mon spec with placement smithi083:172.21.15.83=a;smithi117:172.21.15.117=b;smithi167:172.21.15.167=c;count:3 2024-08-19T01:26:37.655 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:26:37 smithi083 ceph-mon[17120]: Deploying daemon mon.c on smithi167 2024-08-19T01:26:37.656 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:26:37 smithi083 ceph-mon[17120]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-19T01:26:38.936 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-19T01:26:38.915+0000 7f054a888640 1 -- 172.21.15.167:0/2014811046 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0544100400 msgr2=0x7f05441007e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:26:38.936 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-19T01:26:38.915+0000 7f054a888640 1 --2- 172.21.15.167:0/2014811046 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0544100400 0x7f05441007e0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f0534009810 tx=0x7f053402a540 comp rx=0 tx=0).stop 2024-08-19T01:26:38.937 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-19T01:26:38.915+0000 7f054a888640 1 -- 172.21.15.167:0/2014811046 shutdown_connections 2024-08-19T01:26:38.937 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-19T01:26:38.915+0000 7f054a888640 1 --2- 172.21.15.167:0/2014811046 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0544100400 0x7f05441007e0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:26:38.937 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-19T01:26:38.915+0000 7f054a888640 1 -- 172.21.15.167:0/2014811046 >> 172.21.15.167:0/2014811046 conn(0x7f0544077040 msgr2=0x7f05440fe370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:26:38.937 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-19T01:26:38.915+0000 7f054a888640 1 -- 172.21.15.167:0/2014811046 shutdown_connections 2024-08-19T01:26:38.937 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-19T01:26:38.915+0000 7f054a888640 1 -- 172.21.15.167:0/2014811046 wait complete. 2024-08-19T01:26:38.938 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-19T01:26:38.915+0000 7f054a888640 1 Processor -- start 2024-08-19T01:26:38.938 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-19T01:26:38.916+0000 7f054a888640 1 -- start start 2024-08-19T01:26:38.938 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-19T01:26:38.916+0000 7f054a888640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0544100400 0x7f054410b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:26:38.938 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-19T01:26:38.916+0000 7f054a888640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f054410a850 con 0x7f0544100400 2024-08-19T01:26:38.938 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-19T01:26:38.916+0000 7f0543fff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0544100400 0x7f054410b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:26:38.938 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-19T01:26:38.916+0000 7f0543fff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0544100400 0x7f054410b0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.167:37646/0 (socket says 172.21.15.167:37646) 2024-08-19T01:26:38.938 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-19T01:26:38.916+0000 7f0543fff640 1 -- 172.21.15.167:0/3907150059 learned_addr learned my addr 172.21.15.167:0/3907150059 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-08-19T01:26:38.938 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-19T01:26:38.917+0000 7f0543fff640 1 -- 172.21.15.167:0/3907150059 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f054410cf90 con 0x7f0544100400 2024-08-19T01:26:38.938 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-19T01:26:38.917+0000 7f0543fff640 1 --2- 172.21.15.167:0/3907150059 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0544100400 0x7f054410b0a0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f05340045e0 tx=0x7f0534004610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:26:38.939 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-19T01:26:38.917+0000 7f05417fa640 1 -- 172.21.15.167:0/3907150059 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0534030e70 con 0x7f0544100400 2024-08-19T01:26:38.939 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-19T01:26:38.917+0000 7f054a888640 1 -- 172.21.15.167:0/3907150059 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f054410b640 con 0x7f0544100400 2024-08-19T01:26:38.939 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-19T01:26:38.917+0000 7f05417fa640 1 -- 172.21.15.167:0/3907150059 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f05340318e0 con 0x7f0544100400 2024-08-19T01:26:38.939 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-19T01:26:38.917+0000 7f05417fa640 1 -- 172.21.15.167:0/3907150059 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f05340049c0 con 0x7f0544100400 2024-08-19T01:26:38.939 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-19T01:26:38.917+0000 7f054a888640 1 -- 172.21.15.167:0/3907150059 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f054410bb00 con 0x7f0544100400 2024-08-19T01:26:38.939 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-19T01:26:38.918+0000 7f054a888640 1 -- 172.21.15.167:0/3907150059 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f05441018e0 con 0x7f0544100400 2024-08-19T01:26:38.939 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-19T01:26:38.918+0000 7f05417fa640 1 -- 172.21.15.167:0/3907150059 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 13) ==== 46343+0+0 (secure 0 0 0) 0x7f0534039040 con 0x7f0544100400 2024-08-19T01:26:38.939 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-19T01:26:38.918+0000 7f05417fa640 1 --2- 172.21.15.167:0/3907150059 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f052c039060 0x7f052c03b520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:26:38.940 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-19T01:26:38.918+0000 7f05417fa640 1 -- 172.21.15.167:0/3907150059 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f053406d260 con 0x7f0544100400 2024-08-19T01:26:38.940 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-19T01:26:38.919+0000 7f05437fe640 1 --2- 172.21.15.167:0/3907150059 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f052c039060 0x7f052c03b520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:26:38.940 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-19T01:26:38.919+0000 7f05437fe640 1 --2- 172.21.15.167:0/3907150059 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f052c039060 0x7f052c03b520 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f05380098c0 tx=0x7f0538006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:26:38.940 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-19T01:26:38.924+0000 7f05417fa640 1 -- 172.21.15.167:0/3907150059 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7f0534040030 con 0x7f0544100400 2024-08-19T01:26:39.184 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-19T01:26:39.182+0000 7f054a888640 1 -- 172.21.15.167:0/3907150059 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7f054410c5d0 con 0x7f0544100400 2024-08-19T01:26:39.184 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-19T01:26:39.183+0000 7f05417fa640 1 -- 172.21.15.167:0/3907150059 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) ==== 95+0+745 (secure 0 0 0) 0x7f053403ae40 con 0x7f0544100400 2024-08-19T01:26:39.185 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-08-19T01:26:39.185 INFO:teuthology.orchestra.run.smithi167.stdout:{"epoch":1,"fsid":"bad8780e-5dc9-11ef-bccf-c7b262605968","modified":"2024-08-19T01:25:09.034517Z","created":"2024-08-19T01:25:09.034517Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:3300","nonce":0},{"type":"v1","addr":"172.21.15.83:6789","nonce":0}]},"addr":"172.21.15.83:6789/0","public_addr":"172.21.15.83:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-08-19T01:26:39.185 INFO:teuthology.orchestra.run.smithi167.stderr:dumped monmap epoch 1 2024-08-19T01:26:39.187 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-19T01:26:39.186+0000 7f054a888640 1 -- 172.21.15.167:0/3907150059 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f052c039060 msgr2=0x7f052c03b520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:26:39.187 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-19T01:26:39.186+0000 7f054a888640 1 --2- 172.21.15.167:0/3907150059 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f052c039060 0x7f052c03b520 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f05380098c0 tx=0x7f0538006db0 comp rx=0 tx=0).stop 2024-08-19T01:26:39.187 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-19T01:26:39.186+0000 7f054a888640 1 -- 172.21.15.167:0/3907150059 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0544100400 msgr2=0x7f054410b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:26:39.187 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-19T01:26:39.186+0000 7f054a888640 1 --2- 172.21.15.167:0/3907150059 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0544100400 0x7f054410b0a0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f05340045e0 tx=0x7f0534004610 comp rx=0 tx=0).stop 2024-08-19T01:26:39.188 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-19T01:26:39.186+0000 7f054a888640 1 -- 172.21.15.167:0/3907150059 shutdown_connections 2024-08-19T01:26:39.188 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-19T01:26:39.186+0000 7f054a888640 1 --2- 172.21.15.167:0/3907150059 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f052c039060 0x7f052c03b520 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:26:39.188 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-19T01:26:39.186+0000 7f054a888640 1 --2- 172.21.15.167:0/3907150059 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0544100400 0x7f054410b0a0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:26:39.188 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-19T01:26:39.186+0000 7f054a888640 1 -- 172.21.15.167:0/3907150059 >> 172.21.15.167:0/3907150059 conn(0x7f0544077040 msgr2=0x7f0544107cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:26:39.188 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-19T01:26:39.186+0000 7f054a888640 1 -- 172.21.15.167:0/3907150059 shutdown_connections 2024-08-19T01:26:39.188 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-19T01:26:39.186+0000 7f054a888640 1 -- 172.21.15.167:0/3907150059 wait complete. 2024-08-19T01:26:39.655 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:26:39 smithi083 ceph-mon[17120]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-19T01:26:39.655 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:26:39 smithi083 ceph-mon[17120]: from='client.? 172.21.15.167:0/3907150059' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-08-19T01:26:40.716 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:26:40 smithi167 systemd[1]: Started Ceph mon.c for bad8780e-5dc9-11ef-bccf-c7b262605968. 2024-08-19T01:26:40.954 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-08-19T01:26:40.954 DEBUG:teuthology.orchestra.run.smithi167:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid bad8780e-5dc9-11ef-bccf-c7b262605968 -- ceph mon dump -f json 2024-08-19T01:26:41.202 INFO:teuthology.orchestra.run.smithi167.stderr:Inferring config /var/lib/ceph/bad8780e-5dc9-11ef-bccf-c7b262605968/mon.c/config 2024-08-19T01:26:45.905 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:26:45 smithi083 ceph-mon[17120]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-08-19T01:26:45.906 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:26:45 smithi083 ceph-mon[17120]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-08-19T01:26:45.906 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:26:45 smithi083 ceph-mon[17120]: mon.a calling monitor election 2024-08-19T01:26:45.906 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:26:45 smithi083 ceph-mon[17120]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-08-19T01:26:45.906 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:26:45 smithi083 ceph-mon[17120]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-19T01:26:45.906 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:26:45 smithi083 ceph-mon[17120]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-08-19T01:26:45.906 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:26:45 smithi083 ceph-mon[17120]: mon.c calling monitor election 2024-08-19T01:26:45.906 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:26:45 smithi083 ceph-mon[17120]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-08-19T01:26:45.906 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:26:45 smithi083 ceph-mon[17120]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-19T01:26:45.906 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:26:45 smithi083 ceph-mon[17120]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-08-19T01:26:45.907 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:26:45 smithi083 ceph-mon[17120]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-08-19T01:26:45.907 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:26:45 smithi083 ceph-mon[17120]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-08-19T01:26:45.907 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:26:45 smithi083 ceph-mon[17120]: monmap epoch 2 2024-08-19T01:26:45.907 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:26:45 smithi083 ceph-mon[17120]: fsid bad8780e-5dc9-11ef-bccf-c7b262605968 2024-08-19T01:26:45.907 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:26:45 smithi083 ceph-mon[17120]: last_changed 2024-08-19T01:26:40.479750+0000 2024-08-19T01:26:45.907 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:26:45 smithi083 ceph-mon[17120]: created 2024-08-19T01:25:09.034517+0000 2024-08-19T01:26:45.907 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:26:45 smithi083 ceph-mon[17120]: min_mon_release 19 (squid) 2024-08-19T01:26:45.907 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:26:45 smithi083 ceph-mon[17120]: election_strategy: 1 2024-08-19T01:26:45.907 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:26:45 smithi083 ceph-mon[17120]: 0: [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] mon.a 2024-08-19T01:26:45.908 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:26:45 smithi083 ceph-mon[17120]: 1: [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] mon.c 2024-08-19T01:26:45.908 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:26:45 smithi083 ceph-mon[17120]: fsmap 2024-08-19T01:26:45.908 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:26:45 smithi083 ceph-mon[17120]: osdmap e4: 0 total, 0 up, 0 in 2024-08-19T01:26:45.908 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:26:45 smithi083 ceph-mon[17120]: mgrmap e13: x(active, since 41s) 2024-08-19T01:26:45.908 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:26:45 smithi083 ceph-mon[17120]: overall HEALTH_OK 2024-08-19T01:26:45.908 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:26:45 smithi083 ceph-mon[17120]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' 2024-08-19T01:26:45.908 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:26:45 smithi083 ceph-mon[17120]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' 2024-08-19T01:26:45.908 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:26:45 smithi083 ceph-mon[17120]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' 2024-08-19T01:26:45.908 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:26:45 smithi083 ceph-mon[17120]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-08-19T01:26:45.908 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:26:45 smithi083 ceph-mon[17120]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-19T01:26:45.967 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:26:45 smithi167 ceph-mon[21875]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-08-19T01:26:45.967 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:26:45 smithi167 ceph-mon[21875]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-08-19T01:26:45.967 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:26:45 smithi167 ceph-mon[21875]: mon.a calling monitor election 2024-08-19T01:26:45.968 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:26:45 smithi167 ceph-mon[21875]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-08-19T01:26:45.968 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:26:45 smithi167 ceph-mon[21875]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-19T01:26:45.968 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:26:45 smithi167 ceph-mon[21875]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-08-19T01:26:45.968 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:26:45 smithi167 ceph-mon[21875]: mon.c calling monitor election 2024-08-19T01:26:45.968 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:26:45 smithi167 ceph-mon[21875]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-08-19T01:26:45.968 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:26:45 smithi167 ceph-mon[21875]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-19T01:26:45.968 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:26:45 smithi167 ceph-mon[21875]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-08-19T01:26:45.968 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:26:45 smithi167 ceph-mon[21875]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-08-19T01:26:45.968 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:26:45 smithi167 ceph-mon[21875]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-08-19T01:26:45.968 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:26:45 smithi167 ceph-mon[21875]: monmap epoch 2 2024-08-19T01:26:45.968 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:26:45 smithi167 ceph-mon[21875]: fsid bad8780e-5dc9-11ef-bccf-c7b262605968 2024-08-19T01:26:45.968 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:26:45 smithi167 ceph-mon[21875]: last_changed 2024-08-19T01:26:40.479750+0000 2024-08-19T01:26:45.968 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:26:45 smithi167 ceph-mon[21875]: created 2024-08-19T01:25:09.034517+0000 2024-08-19T01:26:45.969 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:26:45 smithi167 ceph-mon[21875]: min_mon_release 19 (squid) 2024-08-19T01:26:45.969 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:26:45 smithi167 ceph-mon[21875]: election_strategy: 1 2024-08-19T01:26:45.969 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:26:45 smithi167 ceph-mon[21875]: 0: [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] mon.a 2024-08-19T01:26:45.969 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:26:45 smithi167 ceph-mon[21875]: 1: [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] mon.c 2024-08-19T01:26:45.969 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:26:45 smithi167 ceph-mon[21875]: fsmap 2024-08-19T01:26:45.969 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:26:45 smithi167 ceph-mon[21875]: osdmap e4: 0 total, 0 up, 0 in 2024-08-19T01:26:45.969 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:26:45 smithi167 ceph-mon[21875]: mgrmap e13: x(active, since 41s) 2024-08-19T01:26:45.969 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:26:45 smithi167 ceph-mon[21875]: overall HEALTH_OK 2024-08-19T01:26:45.969 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:26:45 smithi167 ceph-mon[21875]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' 2024-08-19T01:26:45.969 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:26:45 smithi167 ceph-mon[21875]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' 2024-08-19T01:26:45.969 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:26:45 smithi167 ceph-mon[21875]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' 2024-08-19T01:26:45.969 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:26:45 smithi167 ceph-mon[21875]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-08-19T01:26:45.969 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:26:45 smithi167 ceph-mon[21875]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-19T01:26:46.068 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-19T01:26:46.062+0000 7f03ee877640 1 -- 172.21.15.167:0/456781489 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f03d0003530 msgr2=0x7f03d00059c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:26:46.068 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-19T01:26:46.062+0000 7f03ee877640 1 --2- 172.21.15.167:0/456781489 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f03d0003530 0x7f03d00059c0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f03e806a710 tx=0x7f03d802c490 comp rx=0 tx=0).stop 2024-08-19T01:26:46.068 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-19T01:26:46.062+0000 7f03ee877640 1 -- 172.21.15.167:0/456781489 shutdown_connections 2024-08-19T01:26:46.068 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-19T01:26:46.062+0000 7f03ee877640 1 --2- 172.21.15.167:0/456781489 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f03d0003530 0x7f03d00059c0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:26:46.068 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-19T01:26:46.062+0000 7f03ee877640 1 -- 172.21.15.167:0/456781489 >> 172.21.15.167:0/456781489 conn(0x7f03e80fbf50 msgr2=0x7f03e80fe370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:26:46.068 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-19T01:26:46.062+0000 7f03ee877640 1 -- 172.21.15.167:0/456781489 shutdown_connections 2024-08-19T01:26:46.068 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-19T01:26:46.062+0000 7f03ee877640 1 -- 172.21.15.167:0/456781489 wait complete. 2024-08-19T01:26:46.069 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-19T01:26:46.063+0000 7f03ee877640 1 Processor -- start 2024-08-19T01:26:46.069 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-19T01:26:46.063+0000 7f03ee877640 1 -- start start 2024-08-19T01:26:46.069 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-19T01:26:46.064+0000 7f03ee877640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f03d0003530 0x7f03e8100230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:26:46.069 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-19T01:26:46.064+0000 7f03ee877640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f03e8100770 0x7f03e8100bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:26:46.069 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-19T01:26:46.064+0000 7f03ee877640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f03e81094d0 con 0x7f03d0003530 2024-08-19T01:26:46.069 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-19T01:26:46.064+0000 7f03ee877640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f03e8109350 con 0x7f03e8100770 2024-08-19T01:26:46.069 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-19T01:26:46.064+0000 7f03e77fe640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f03e8100770 0x7f03e8100bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:26:46.069 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-19T01:26:46.064+0000 7f03e77fe640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f03e8100770 0x7f03e8100bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:42212/0 (socket says 172.21.15.167:42212) 2024-08-19T01:26:46.069 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-19T01:26:46.064+0000 7f03e77fe640 1 -- 172.21.15.167:0/149407054 learned_addr learned my addr 172.21.15.167:0/149407054 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-08-19T01:26:46.069 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-19T01:26:46.064+0000 7f03e7fff640 1 --2- 172.21.15.167:0/149407054 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f03d0003530 0x7f03e8100230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:26:46.069 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-19T01:26:46.064+0000 7f03e77fe640 1 -- 172.21.15.167:0/149407054 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f03e8100770 msgr2=0x7f03e8100bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 13 2024-08-19T01:26:46.070 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-19T01:26:46.064+0000 7f03e77fe640 1 -- 172.21.15.167:0/149407054 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f03e8100770 msgr2=0x7f03e8100bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-08-19T01:26:46.070 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-19T01:26:46.064+0000 7f03e77fe640 1 --2- 172.21.15.167:0/149407054 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f03e8100770 0x7f03e8100bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-08-19T01:26:46.070 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-19T01:26:46.064+0000 7f03e77fe640 1 --2- 172.21.15.167:0/149407054 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f03e8100770 0x7f03e8100bd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-08-19T01:26:46.070 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-19T01:26:46.064+0000 7f03e7fff640 1 -- 172.21.15.167:0/149407054 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f03e8100770 msgr2=0x7f03e8100bd0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-08-19T01:26:46.070 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-19T01:26:46.064+0000 7f03e7fff640 1 --2- 172.21.15.167:0/149407054 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f03e8100770 0x7f03e8100bd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:26:46.070 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-19T01:26:46.064+0000 7f03e7fff640 1 -- 172.21.15.167:0/149407054 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f03e8102120 con 0x7f03d0003530 2024-08-19T01:26:46.070 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-19T01:26:46.065+0000 7f03e7fff640 1 --2- 172.21.15.167:0/149407054 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f03d0003530 0x7f03e8100230 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f03d8005f30 tx=0x7f03d8032b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:26:46.070 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-19T01:26:46.065+0000 7f03e57fa640 1 -- 172.21.15.167:0/149407054 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 309+0+0 (secure 0 0 0) 0x7f03d8038040 con 0x7f03d0003530 2024-08-19T01:26:46.070 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-19T01:26:46.065+0000 7f03e57fa640 1 -- 172.21.15.167:0/149407054 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f03d803c410 con 0x7f03d0003530 2024-08-19T01:26:46.070 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-19T01:26:46.065+0000 7f03ee877640 1 -- 172.21.15.167:0/149407054 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f03e81a32b0 con 0x7f03d0003530 2024-08-19T01:26:46.070 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-19T01:26:46.065+0000 7f03e57fa640 1 -- 172.21.15.167:0/149407054 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 309+0+0 (secure 0 0 0) 0x7f03d803b370 con 0x7f03d0003530 2024-08-19T01:26:46.070 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-19T01:26:46.065+0000 7f03ee877640 1 -- 172.21.15.167:0/149407054 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f03e81a3840 con 0x7f03d0003530 2024-08-19T01:26:46.071 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-19T01:26:46.066+0000 7f03e57fa640 1 -- 172.21.15.167:0/149407054 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 13) ==== 46343+0+0 (secure 0 0 0) 0x7f03d8043020 con 0x7f03d0003530 2024-08-19T01:26:46.071 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-19T01:26:46.066+0000 7f03e57fa640 1 --2- 172.21.15.167:0/149407054 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f03d4038e50 0x7f03d403b310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:26:46.071 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-19T01:26:46.066+0000 7f03e57fa640 1 -- 172.21.15.167:0/149407054 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f03d806f630 con 0x7f03d0003530 2024-08-19T01:26:46.071 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-19T01:26:46.067+0000 7f03ee877640 1 -- 172.21.15.167:0/149407054 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f03e806a9c0 con 0x7f03d0003530 2024-08-19T01:26:46.072 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-19T01:26:46.067+0000 7f03e77fe640 1 --2- 172.21.15.167:0/149407054 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f03d4038e50 0x7f03d403b310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:26:46.072 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-19T01:26:46.067+0000 7f03e77fe640 1 --2- 172.21.15.167:0/149407054 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f03d4038e50 0x7f03d403b310 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f03dc009100 tx=0x7f03dc008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:26:46.074 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-19T01:26:46.072+0000 7f03e57fa640 1 -- 172.21.15.167:0/149407054 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7f03d804a020 con 0x7f03d0003530 2024-08-19T01:26:46.309 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-19T01:26:46.306+0000 7f03ee877640 1 -- 172.21.15.167:0/149407054 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7f03e8101760 con 0x7f03d0003530 2024-08-19T01:26:46.309 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-19T01:26:46.307+0000 7f03e57fa640 1 -- 172.21.15.167:0/149407054 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 2 v2) ==== 95+0+1015 (secure 0 0 0) 0x7f03d80422c0 con 0x7f03d0003530 2024-08-19T01:26:46.310 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-08-19T01:26:46.310 INFO:teuthology.orchestra.run.smithi167.stdout:{"epoch":2,"fsid":"bad8780e-5dc9-11ef-bccf-c7b262605968","modified":"2024-08-19T01:26:40.479750Z","created":"2024-08-19T01:25:09.034517Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:3300","nonce":0},{"type":"v1","addr":"172.21.15.83:6789","nonce":0}]},"addr":"172.21.15.83:6789/0","public_addr":"172.21.15.83:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:3300","nonce":0},{"type":"v1","addr":"172.21.15.167:6789","nonce":0}]},"addr":"172.21.15.167:6789/0","public_addr":"172.21.15.167:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-08-19T01:26:46.310 INFO:teuthology.orchestra.run.smithi167.stderr:dumped monmap epoch 2 2024-08-19T01:26:46.311 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-19T01:26:46.309+0000 7f03ee877640 1 -- 172.21.15.167:0/149407054 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f03d4038e50 msgr2=0x7f03d403b310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:26:46.311 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-19T01:26:46.309+0000 7f03ee877640 1 --2- 172.21.15.167:0/149407054 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f03d4038e50 0x7f03d403b310 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f03dc009100 tx=0x7f03dc008040 comp rx=0 tx=0).stop 2024-08-19T01:26:46.311 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-19T01:26:46.309+0000 7f03ee877640 1 -- 172.21.15.167:0/149407054 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f03d0003530 msgr2=0x7f03e8100230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:26:46.311 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-19T01:26:46.309+0000 7f03ee877640 1 --2- 172.21.15.167:0/149407054 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f03d0003530 0x7f03e8100230 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f03d8005f30 tx=0x7f03d8032b50 comp rx=0 tx=0).stop 2024-08-19T01:26:46.312 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-19T01:26:46.309+0000 7f03ee877640 1 -- 172.21.15.167:0/149407054 shutdown_connections 2024-08-19T01:26:46.312 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-19T01:26:46.309+0000 7f03ee877640 1 --2- 172.21.15.167:0/149407054 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f03d4038e50 0x7f03d403b310 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:26:46.312 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-19T01:26:46.309+0000 7f03ee877640 1 --2- 172.21.15.167:0/149407054 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f03e8100770 0x7f03e8100bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:26:46.312 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-19T01:26:46.309+0000 7f03ee877640 1 --2- 172.21.15.167:0/149407054 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f03d0003530 0x7f03e8100230 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:26:46.312 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-19T01:26:46.309+0000 7f03ee877640 1 -- 172.21.15.167:0/149407054 >> 172.21.15.167:0/149407054 conn(0x7f03e80fbf50 msgr2=0x7f03e80fde30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:26:46.312 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-19T01:26:46.309+0000 7f03ee877640 1 -- 172.21.15.167:0/149407054 shutdown_connections 2024-08-19T01:26:46.312 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-19T01:26:46.309+0000 7f03ee877640 1 -- 172.21.15.167:0/149407054 wait complete. 2024-08-19T01:26:46.607 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:26:46 smithi167 ceph-mon[21875]: Deploying daemon mon.b on smithi117 2024-08-19T01:26:46.607 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:26:46 smithi167 ceph-mon[21875]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-19T01:26:46.607 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:26:46 smithi167 ceph-mon[21875]: from='client.? 172.21.15.167:0/149407054' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-08-19T01:26:46.607 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:26:46 smithi167 ceph-mon[21875]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-08-19T01:26:46.905 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:26:46 smithi083 ceph-mon[17120]: Deploying daemon mon.b on smithi117 2024-08-19T01:26:46.905 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:26:46 smithi083 ceph-mon[17120]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-19T01:26:46.905 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:26:46 smithi083 ceph-mon[17120]: from='client.? 172.21.15.167:0/149407054' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-08-19T01:26:46.905 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:26:46 smithi083 ceph-mon[17120]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-08-19T01:26:47.903 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-08-19T01:26:47.903 DEBUG:teuthology.orchestra.run.smithi167:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid bad8780e-5dc9-11ef-bccf-c7b262605968 -- ceph mon dump -f json 2024-08-19T01:26:48.162 INFO:teuthology.orchestra.run.smithi167.stderr:Inferring config /var/lib/ceph/bad8780e-5dc9-11ef-bccf-c7b262605968/mon.c/config 2024-08-19T01:26:48.593 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-19T01:26:48.581+0000 7efc5b428640 1 -- 172.21.15.167:0/2344842354 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7efc541004f0 msgr2=0x7efc541008d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:26:48.593 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-19T01:26:48.581+0000 7efc5b428640 1 --2- 172.21.15.167:0/2344842354 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7efc541004f0 0x7efc541008d0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7efc50009880 tx=0x7efc5002a5d0 comp rx=0 tx=0).stop 2024-08-19T01:26:48.593 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-19T01:26:48.581+0000 7efc5b428640 1 -- 172.21.15.167:0/2344842354 shutdown_connections 2024-08-19T01:26:48.594 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-19T01:26:48.581+0000 7efc5b428640 1 --2- 172.21.15.167:0/2344842354 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7efc541004f0 0x7efc541008d0 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:26:48.594 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-19T01:26:48.581+0000 7efc5b428640 1 -- 172.21.15.167:0/2344842354 >> 172.21.15.167:0/2344842354 conn(0x7efc540fbf50 msgr2=0x7efc540fe370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:26:48.594 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-19T01:26:48.581+0000 7efc5b428640 1 -- 172.21.15.167:0/2344842354 shutdown_connections 2024-08-19T01:26:48.594 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-19T01:26:48.581+0000 7efc5b428640 1 -- 172.21.15.167:0/2344842354 wait complete. 2024-08-19T01:26:48.594 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-19T01:26:48.582+0000 7efc5b428640 1 Processor -- start 2024-08-19T01:26:48.594 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-19T01:26:48.582+0000 7efc5b428640 1 -- start start 2024-08-19T01:26:48.594 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-19T01:26:48.582+0000 7efc5b428640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7efc541004f0 0x7efc541970e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:26:48.594 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-19T01:26:48.582+0000 7efc5b428640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7efc54197620 0x7efc5419b9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:26:48.594 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-19T01:26:48.582+0000 7efc5b428640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7efc5410a940 con 0x7efc541004f0 2024-08-19T01:26:48.594 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-19T01:26:48.582+0000 7efc5b428640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7efc5410a7c0 con 0x7efc54197620 2024-08-19T01:26:48.594 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-19T01:26:48.582+0000 7efc589c3640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7efc54197620 0x7efc5419b9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:26:48.594 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-19T01:26:48.582+0000 7efc589c3640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7efc54197620 0x7efc5419b9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:47980/0 (socket says 172.21.15.167:47980) 2024-08-19T01:26:48.595 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-19T01:26:48.582+0000 7efc589c3640 1 -- 172.21.15.167:0/640143198 learned_addr learned my addr 172.21.15.167:0/640143198 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-08-19T01:26:48.595 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-19T01:26:48.582+0000 7efc589c3640 1 -- 172.21.15.167:0/640143198 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7efc54197620 msgr2=0x7efc5419b9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 13 2024-08-19T01:26:48.595 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-19T01:26:48.582+0000 7efc589c3640 1 -- 172.21.15.167:0/640143198 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7efc54197620 msgr2=0x7efc5419b9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-08-19T01:26:48.595 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-19T01:26:48.582+0000 7efc589c3640 1 --2- 172.21.15.167:0/640143198 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7efc54197620 0x7efc5419b9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-08-19T01:26:48.595 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-19T01:26:48.582+0000 7efc589c3640 1 --2- 172.21.15.167:0/640143198 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7efc54197620 0x7efc5419b9b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-08-19T01:26:48.595 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-19T01:26:48.582+0000 7efc591c4640 1 --2- 172.21.15.167:0/640143198 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7efc541004f0 0x7efc541970e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:26:48.595 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-19T01:26:48.583+0000 7efc591c4640 1 -- 172.21.15.167:0/640143198 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7efc54197620 msgr2=0x7efc5419b9b0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-08-19T01:26:48.595 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-19T01:26:48.583+0000 7efc591c4640 1 --2- 172.21.15.167:0/640143198 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7efc54197620 0x7efc5419b9b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:26:48.595 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-19T01:26:48.583+0000 7efc591c4640 1 -- 172.21.15.167:0/640143198 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efc54197c40 con 0x7efc541004f0 2024-08-19T01:26:48.595 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-19T01:26:48.583+0000 7efc591c4640 1 --2- 172.21.15.167:0/640143198 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7efc541004f0 0x7efc541970e0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7efc50005ab0 tx=0x7efc50032c50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:26:48.595 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-19T01:26:48.583+0000 7efc4a7fc640 1 -- 172.21.15.167:0/640143198 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 309+0+0 (secure 0 0 0) 0x7efc50035070 con 0x7efc541004f0 2024-08-19T01:26:48.595 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-19T01:26:48.583+0000 7efc5b428640 1 -- 172.21.15.167:0/640143198 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efc5419bef0 con 0x7efc541004f0 2024-08-19T01:26:48.596 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-19T01:26:48.583+0000 7efc4a7fc640 1 -- 172.21.15.167:0/640143198 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efc50032e40 con 0x7efc541004f0 2024-08-19T01:26:48.596 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-19T01:26:48.583+0000 7efc5b428640 1 -- 172.21.15.167:0/640143198 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efc5419c430 con 0x7efc541004f0 2024-08-19T01:26:48.596 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-19T01:26:48.583+0000 7efc4a7fc640 1 -- 172.21.15.167:0/640143198 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 309+0+0 (secure 0 0 0) 0x7efc5003bc10 con 0x7efc541004f0 2024-08-19T01:26:48.596 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-19T01:26:48.584+0000 7efc5b428640 1 -- 172.21.15.167:0/640143198 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efc541019d0 con 0x7efc541004f0 2024-08-19T01:26:48.596 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-19T01:26:48.584+0000 7efc4a7fc640 1 -- 172.21.15.167:0/640143198 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 13) ==== 46343+0+0 (secure 0 0 0) 0x7efc50042030 con 0x7efc541004f0 2024-08-19T01:26:48.596 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-19T01:26:48.585+0000 7efc4a7fc640 1 --2- 172.21.15.167:0/640143198 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7efc40038e00 0x7efc4003b2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:26:48.596 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-19T01:26:48.585+0000 7efc4a7fc640 1 -- 172.21.15.167:0/640143198 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7efc50040070 con 0x7efc541004f0 2024-08-19T01:26:48.596 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-19T01:26:48.585+0000 7efc589c3640 1 --2- 172.21.15.167:0/640143198 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7efc40038e00 0x7efc4003b2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:26:48.596 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-19T01:26:48.585+0000 7efc589c3640 1 --2- 172.21.15.167:0/640143198 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7efc40038e00 0x7efc4003b2c0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7efc44009100 tx=0x7efc44008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:26:48.596 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-19T01:26:48.590+0000 7efc4a7fc640 1 -- 172.21.15.167:0/640143198 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7efc5003b460 con 0x7efc541004f0 2024-08-19T01:26:48.823 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-19T01:26:48.821+0000 7efc5b428640 1 -- 172.21.15.167:0/640143198 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7efc5419c7a0 con 0x7efc541004f0 2024-08-19T01:26:48.824 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-19T01:26:48.822+0000 7efc4a7fc640 1 -- 172.21.15.167:0/640143198 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 2 v2) ==== 95+0+1015 (secure 0 0 0) 0x7efc50003800 con 0x7efc541004f0 2024-08-19T01:26:48.824 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-08-19T01:26:48.825 INFO:teuthology.orchestra.run.smithi167.stdout:{"epoch":2,"fsid":"bad8780e-5dc9-11ef-bccf-c7b262605968","modified":"2024-08-19T01:26:40.479750Z","created":"2024-08-19T01:25:09.034517Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:3300","nonce":0},{"type":"v1","addr":"172.21.15.83:6789","nonce":0}]},"addr":"172.21.15.83:6789/0","public_addr":"172.21.15.83:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:3300","nonce":0},{"type":"v1","addr":"172.21.15.167:6789","nonce":0}]},"addr":"172.21.15.167:6789/0","public_addr":"172.21.15.167:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-08-19T01:26:48.825 INFO:teuthology.orchestra.run.smithi167.stderr:dumped monmap epoch 2 2024-08-19T01:26:48.827 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-19T01:26:48.824+0000 7efc5b428640 1 -- 172.21.15.167:0/640143198 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7efc40038e00 msgr2=0x7efc4003b2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:26:48.827 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-19T01:26:48.824+0000 7efc5b428640 1 --2- 172.21.15.167:0/640143198 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7efc40038e00 0x7efc4003b2c0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7efc44009100 tx=0x7efc44008040 comp rx=0 tx=0).stop 2024-08-19T01:26:48.827 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-19T01:26:48.824+0000 7efc5b428640 1 -- 172.21.15.167:0/640143198 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7efc541004f0 msgr2=0x7efc541970e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:26:48.827 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-19T01:26:48.824+0000 7efc5b428640 1 --2- 172.21.15.167:0/640143198 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7efc541004f0 0x7efc541970e0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7efc50005ab0 tx=0x7efc50032c50 comp rx=0 tx=0).stop 2024-08-19T01:26:48.827 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-19T01:26:48.824+0000 7efc5b428640 1 -- 172.21.15.167:0/640143198 shutdown_connections 2024-08-19T01:26:48.827 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-19T01:26:48.824+0000 7efc5b428640 1 --2- 172.21.15.167:0/640143198 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7efc40038e00 0x7efc4003b2c0 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:26:48.827 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-19T01:26:48.824+0000 7efc5b428640 1 --2- 172.21.15.167:0/640143198 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7efc54197620 0x7efc5419b9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:26:48.827 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-19T01:26:48.824+0000 7efc5b428640 1 --2- 172.21.15.167:0/640143198 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7efc541004f0 0x7efc541970e0 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:26:48.827 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-19T01:26:48.824+0000 7efc5b428640 1 -- 172.21.15.167:0/640143198 >> 172.21.15.167:0/640143198 conn(0x7efc540fbf50 msgr2=0x7efc54108160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:26:48.827 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-19T01:26:48.824+0000 7efc5b428640 1 -- 172.21.15.167:0/640143198 shutdown_connections 2024-08-19T01:26:48.827 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-19T01:26:48.824+0000 7efc5b428640 1 -- 172.21.15.167:0/640143198 wait complete. 2024-08-19T01:26:49.703 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:26:49 smithi117 systemd[1]: Started Ceph mon.b for bad8780e-5dc9-11ef-bccf-c7b262605968. 2024-08-19T01:26:50.417 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-08-19T01:26:50.417 DEBUG:teuthology.orchestra.run.smithi167:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid bad8780e-5dc9-11ef-bccf-c7b262605968 -- ceph mon dump -f json 2024-08-19T01:26:50.674 INFO:teuthology.orchestra.run.smithi167.stderr:Inferring config /var/lib/ceph/bad8780e-5dc9-11ef-bccf-c7b262605968/mon.c/config 2024-08-19T01:26:54.655 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:26:54 smithi083 ceph-mon[17120]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-08-19T01:26:54.656 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:26:54 smithi083 ceph-mon[17120]: mon.c calling monitor election 2024-08-19T01:26:54.656 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:26:54 smithi083 ceph-mon[17120]: mon.a calling monitor election 2024-08-19T01:26:54.656 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:26:54 smithi083 ceph-mon[17120]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-08-19T01:26:54.656 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:26:54 smithi083 ceph-mon[17120]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-08-19T01:26:54.656 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:26:54 smithi083 ceph-mon[17120]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-19T01:26:54.656 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:26:54 smithi083 ceph-mon[17120]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-08-19T01:26:54.656 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:26:54 smithi083 ceph-mon[17120]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-08-19T01:26:54.656 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:26:54 smithi083 ceph-mon[17120]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-19T01:26:54.656 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:26:54 smithi083 ceph-mon[17120]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-08-19T01:26:54.656 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:26:54 smithi083 ceph-mon[17120]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-08-19T01:26:54.656 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:26:54 smithi083 ceph-mon[17120]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-08-19T01:26:54.656 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:26:54 smithi083 ceph-mon[17120]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-08-19T01:26:54.657 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:26:54 smithi083 ceph-mon[17120]: monmap epoch 3 2024-08-19T01:26:54.657 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:26:54 smithi083 ceph-mon[17120]: fsid bad8780e-5dc9-11ef-bccf-c7b262605968 2024-08-19T01:26:54.657 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:26:54 smithi083 ceph-mon[17120]: last_changed 2024-08-19T01:26:49.333005+0000 2024-08-19T01:26:54.657 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:26:54 smithi083 ceph-mon[17120]: created 2024-08-19T01:25:09.034517+0000 2024-08-19T01:26:54.657 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:26:54 smithi083 ceph-mon[17120]: min_mon_release 19 (squid) 2024-08-19T01:26:54.657 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:26:54 smithi083 ceph-mon[17120]: election_strategy: 1 2024-08-19T01:26:54.657 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:26:54 smithi083 ceph-mon[17120]: 0: [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] mon.a 2024-08-19T01:26:54.657 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:26:54 smithi083 ceph-mon[17120]: 1: [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] mon.c 2024-08-19T01:26:54.657 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:26:54 smithi083 ceph-mon[17120]: 2: [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] mon.b 2024-08-19T01:26:54.657 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:26:54 smithi083 ceph-mon[17120]: fsmap 2024-08-19T01:26:54.657 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:26:54 smithi083 ceph-mon[17120]: osdmap e4: 0 total, 0 up, 0 in 2024-08-19T01:26:54.657 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:26:54 smithi083 ceph-mon[17120]: mgrmap e13: x(active, since 50s) 2024-08-19T01:26:54.657 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:26:54 smithi083 ceph-mon[17120]: overall HEALTH_OK 2024-08-19T01:26:54.657 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:26:54 smithi083 ceph-mon[17120]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' 2024-08-19T01:26:54.657 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:26:54 smithi083 ceph-mon[17120]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' 2024-08-19T01:26:54.658 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:26:54 smithi083 ceph-mon[17120]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' 2024-08-19T01:26:54.658 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:26:54 smithi083 ceph-mon[17120]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' 2024-08-19T01:26:54.717 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:26:54 smithi167 ceph-mon[21875]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-08-19T01:26:54.717 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:26:54 smithi167 ceph-mon[21875]: mon.c calling monitor election 2024-08-19T01:26:54.717 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:26:54 smithi167 ceph-mon[21875]: mon.a calling monitor election 2024-08-19T01:26:54.717 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:26:54 smithi167 ceph-mon[21875]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-08-19T01:26:54.717 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:26:54 smithi167 ceph-mon[21875]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-08-19T01:26:54.717 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:26:54 smithi167 ceph-mon[21875]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-19T01:26:54.718 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:26:54 smithi167 ceph-mon[21875]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-08-19T01:26:54.718 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:26:54 smithi167 ceph-mon[21875]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-08-19T01:26:54.718 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:26:54 smithi167 ceph-mon[21875]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-19T01:26:54.718 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:26:54 smithi167 ceph-mon[21875]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-08-19T01:26:54.718 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:26:54 smithi167 ceph-mon[21875]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-08-19T01:26:54.718 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:26:54 smithi167 ceph-mon[21875]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-08-19T01:26:54.718 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:26:54 smithi167 ceph-mon[21875]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-08-19T01:26:54.718 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:26:54 smithi167 ceph-mon[21875]: monmap epoch 3 2024-08-19T01:26:54.718 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:26:54 smithi167 ceph-mon[21875]: fsid bad8780e-5dc9-11ef-bccf-c7b262605968 2024-08-19T01:26:54.718 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:26:54 smithi167 ceph-mon[21875]: last_changed 2024-08-19T01:26:49.333005+0000 2024-08-19T01:26:54.718 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:26:54 smithi167 ceph-mon[21875]: created 2024-08-19T01:25:09.034517+0000 2024-08-19T01:26:54.719 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:26:54 smithi167 ceph-mon[21875]: min_mon_release 19 (squid) 2024-08-19T01:26:54.719 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:26:54 smithi167 ceph-mon[21875]: election_strategy: 1 2024-08-19T01:26:54.719 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:26:54 smithi167 ceph-mon[21875]: 0: [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] mon.a 2024-08-19T01:26:54.719 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:26:54 smithi167 ceph-mon[21875]: 1: [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] mon.c 2024-08-19T01:26:54.719 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:26:54 smithi167 ceph-mon[21875]: 2: [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] mon.b 2024-08-19T01:26:54.719 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:26:54 smithi167 ceph-mon[21875]: fsmap 2024-08-19T01:26:54.719 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:26:54 smithi167 ceph-mon[21875]: osdmap e4: 0 total, 0 up, 0 in 2024-08-19T01:26:54.719 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:26:54 smithi167 ceph-mon[21875]: mgrmap e13: x(active, since 50s) 2024-08-19T01:26:54.719 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:26:54 smithi167 ceph-mon[21875]: overall HEALTH_OK 2024-08-19T01:26:54.719 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:26:54 smithi167 ceph-mon[21875]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' 2024-08-19T01:26:54.719 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:26:54 smithi167 ceph-mon[21875]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' 2024-08-19T01:26:54.719 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:26:54 smithi167 ceph-mon[21875]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' 2024-08-19T01:26:54.720 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:26:54 smithi167 ceph-mon[21875]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' 2024-08-19T01:26:54.722 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-19T01:26:54.714+0000 7f97e241e640 1 -- 172.21.15.167:0/1263684075 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f97c8005740 msgr2=0x7f97c8007bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:26:54.722 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-19T01:26:54.714+0000 7f97e241e640 1 --2- 172.21.15.167:0/1263684075 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f97c8005740 0x7f97c8007bd0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f97dc101720 tx=0x7f97cc02c3f0 comp rx=0 tx=0).stop 2024-08-19T01:26:54.723 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-19T01:26:54.714+0000 7f97e241e640 1 -- 172.21.15.167:0/1263684075 shutdown_connections 2024-08-19T01:26:54.723 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-19T01:26:54.714+0000 7f97e241e640 1 --2- 172.21.15.167:0/1263684075 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f97c8005740 0x7f97c8007bd0 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:26:54.723 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-19T01:26:54.714+0000 7f97e241e640 1 -- 172.21.15.167:0/1263684075 >> 172.21.15.167:0/1263684075 conn(0x7f97dc0fbf50 msgr2=0x7f97dc0fe370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:26:54.723 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-19T01:26:54.715+0000 7f97e241e640 1 -- 172.21.15.167:0/1263684075 shutdown_connections 2024-08-19T01:26:54.723 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-19T01:26:54.715+0000 7f97e241e640 1 -- 172.21.15.167:0/1263684075 wait complete. 2024-08-19T01:26:54.723 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-19T01:26:54.715+0000 7f97e241e640 1 Processor -- start 2024-08-19T01:26:54.723 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-19T01:26:54.715+0000 7f97e241e640 1 -- start start 2024-08-19T01:26:54.723 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-19T01:26:54.716+0000 7f97e241e640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f97c8005740 0x7f97dc194e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:26:54.723 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-19T01:26:54.716+0000 7f97e241e640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f97dc1953d0 0x7f97dc199760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:26:54.723 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-19T01:26:54.716+0000 7f97e241e640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f97dc199ca0 0x7f97dc19a150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:26:54.723 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-19T01:26:54.716+0000 7f97e241e640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f97dc1072f0 con 0x7f97dc199ca0 2024-08-19T01:26:54.723 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-19T01:26:54.716+0000 7f97e241e640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f97dc107170 con 0x7f97dc1953d0 2024-08-19T01:26:54.724 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-19T01:26:54.716+0000 7f97e241e640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f97dc107470 con 0x7f97c8005740 2024-08-19T01:26:54.724 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-19T01:26:54.716+0000 7f97dbfff640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f97c8005740 0x7f97dc194e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:26:54.724 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-19T01:26:54.716+0000 7f97dbfff640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f97c8005740 0x7f97dc194e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:47986/0 (socket says 172.21.15.167:47986) 2024-08-19T01:26:54.724 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-19T01:26:54.716+0000 7f97dbfff640 1 -- 172.21.15.167:0/2382580719 learned_addr learned my addr 172.21.15.167:0/2382580719 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-08-19T01:26:54.724 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-19T01:26:54.716+0000 7f97dbfff640 1 -- 172.21.15.167:0/2382580719 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f97c8005740 msgr2=0x7f97dc194e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 12 2024-08-19T01:26:54.724 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-19T01:26:54.716+0000 7f97dbfff640 1 -- 172.21.15.167:0/2382580719 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f97c8005740 msgr2=0x7f97dc194e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-08-19T01:26:54.724 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-19T01:26:54.716+0000 7f97dbfff640 1 --2- 172.21.15.167:0/2382580719 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f97c8005740 0x7f97dc194e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-08-19T01:26:54.724 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-19T01:26:54.716+0000 7f97db7fe640 1 --2- 172.21.15.167:0/2382580719 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f97dc1953d0 0x7f97dc199760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:26:54.724 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-19T01:26:54.716+0000 7f97dbfff640 1 --2- 172.21.15.167:0/2382580719 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f97c8005740 0x7f97dc194e90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-08-19T01:26:54.724 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-19T01:26:54.716+0000 7f97db7fe640 1 -- 172.21.15.167:0/2382580719 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f97dc1953d0 msgr2=0x7f97dc199760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 13 2024-08-19T01:26:54.724 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-19T01:26:54.716+0000 7f97db7fe640 1 -- 172.21.15.167:0/2382580719 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f97dc1953d0 msgr2=0x7f97dc199760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-08-19T01:26:54.724 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-19T01:26:54.716+0000 7f97db7fe640 1 --2- 172.21.15.167:0/2382580719 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f97dc1953d0 0x7f97dc199760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-08-19T01:26:54.724 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-19T01:26:54.716+0000 7f97db7fe640 1 --2- 172.21.15.167:0/2382580719 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f97dc1953d0 0x7f97dc199760 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-08-19T01:26:54.725 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-19T01:26:54.716+0000 7f97e09bb640 1 --2- 172.21.15.167:0/2382580719 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f97dc199ca0 0x7f97dc19a150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:26:54.725 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-19T01:26:54.717+0000 7f97e09bb640 1 -- 172.21.15.167:0/2382580719 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f97c8005740 msgr2=0x7f97dc194e90 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-08-19T01:26:54.725 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-19T01:26:54.717+0000 7f97e09bb640 1 --2- 172.21.15.167:0/2382580719 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f97c8005740 0x7f97dc194e90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:26:54.725 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-19T01:26:54.717+0000 7f97e09bb640 1 -- 172.21.15.167:0/2382580719 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f97dc1953d0 msgr2=0x7f97dc199760 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-08-19T01:26:54.725 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-19T01:26:54.717+0000 7f97e09bb640 1 --2- 172.21.15.167:0/2382580719 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f97dc1953d0 0x7f97dc199760 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:26:54.725 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-19T01:26:54.717+0000 7f97e09bb640 1 -- 172.21.15.167:0/2382580719 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f97dc1a1a10 con 0x7f97dc199ca0 2024-08-19T01:26:54.725 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-19T01:26:54.717+0000 7f97e09bb640 1 --2- 172.21.15.167:0/2382580719 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f97dc199ca0 0x7f97dc19a150 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f97d400bd50 tx=0x7f97d400be40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:26:54.725 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-19T01:26:54.717+0000 7f97d97fa640 1 -- 172.21.15.167:0/2382580719 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97d4019070 con 0x7f97dc199ca0 2024-08-19T01:26:54.725 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-19T01:26:54.717+0000 7f97d97fa640 1 -- 172.21.15.167:0/2382580719 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f97d4014920 con 0x7f97dc199ca0 2024-08-19T01:26:54.725 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-19T01:26:54.717+0000 7f97d97fa640 1 -- 172.21.15.167:0/2382580719 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97d4012990 con 0x7f97dc199ca0 2024-08-19T01:26:54.725 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-19T01:26:54.717+0000 7f97e241e640 1 -- 172.21.15.167:0/2382580719 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f97dc1a1c40 con 0x7f97dc199ca0 2024-08-19T01:26:54.725 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-19T01:26:54.717+0000 7f97e241e640 1 -- 172.21.15.167:0/2382580719 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f97dc1a2100 con 0x7f97dc199ca0 2024-08-19T01:26:54.725 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-19T01:26:54.718+0000 7f97d97fa640 1 -- 172.21.15.167:0/2382580719 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 13) ==== 46343+0+0 (secure 0 0 0) 0x7f97d401c070 con 0x7f97dc199ca0 2024-08-19T01:26:54.725 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-19T01:26:54.718+0000 7f97e241e640 1 -- 172.21.15.167:0/2382580719 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f97dc1083c0 con 0x7f97dc199ca0 2024-08-19T01:26:54.726 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-19T01:26:54.719+0000 7f97d97fa640 1 --2- 172.21.15.167:0/2382580719 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f97bc0394f0 0x7f97bc03b9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:26:54.726 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-19T01:26:54.719+0000 7f97d97fa640 1 -- 172.21.15.167:0/2382580719 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f97d404be50 con 0x7f97dc199ca0 2024-08-19T01:26:54.726 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-19T01:26:54.719+0000 7f97dbfff640 1 --2- 172.21.15.167:0/2382580719 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f97bc0394f0 0x7f97bc03b9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:26:54.726 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-19T01:26:54.719+0000 7f97dbfff640 1 --2- 172.21.15.167:0/2382580719 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f97bc0394f0 0x7f97bc03b9b0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f97cc000c00 tx=0x7f97cc0149c0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:26:54.726 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-19T01:26:54.724+0000 7f97d97fa640 1 -- 172.21.15.167:0/2382580719 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7f97d4013ba0 con 0x7f97dc199ca0 2024-08-19T01:26:54.976 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-19T01:26:54.973+0000 7f97e241e640 1 -- 172.21.15.167:0/2382580719 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7f97dc02d5e0 con 0x7f97dc199ca0 2024-08-19T01:26:54.977 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-19T01:26:54.974+0000 7f97d97fa640 1 -- 172.21.15.167:0/2382580719 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 3 v3) ==== 95+0+1283 (secure 0 0 0) 0x7f97d400e090 con 0x7f97dc199ca0 2024-08-19T01:26:54.977 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-08-19T01:26:54.977 INFO:teuthology.orchestra.run.smithi167.stdout:{"epoch":3,"fsid":"bad8780e-5dc9-11ef-bccf-c7b262605968","modified":"2024-08-19T01:26:49.333005Z","created":"2024-08-19T01:25:09.034517Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:3300","nonce":0},{"type":"v1","addr":"172.21.15.83:6789","nonce":0}]},"addr":"172.21.15.83:6789/0","public_addr":"172.21.15.83:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:3300","nonce":0},{"type":"v1","addr":"172.21.15.167:6789","nonce":0}]},"addr":"172.21.15.167:6789/0","public_addr":"172.21.15.167:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:3300","nonce":0},{"type":"v1","addr":"172.21.15.117:6789","nonce":0}]},"addr":"172.21.15.117:6789/0","public_addr":"172.21.15.117:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-08-19T01:26:54.977 INFO:teuthology.orchestra.run.smithi167.stderr:dumped monmap epoch 3 2024-08-19T01:26:54.978 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-19T01:26:54.976+0000 7f97e241e640 1 -- 172.21.15.167:0/2382580719 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f97bc0394f0 msgr2=0x7f97bc03b9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:26:54.978 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-19T01:26:54.976+0000 7f97e241e640 1 --2- 172.21.15.167:0/2382580719 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f97bc0394f0 0x7f97bc03b9b0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f97cc000c00 tx=0x7f97cc0149c0 comp rx=0 tx=0).stop 2024-08-19T01:26:54.978 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-19T01:26:54.976+0000 7f97e241e640 1 -- 172.21.15.167:0/2382580719 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f97dc199ca0 msgr2=0x7f97dc19a150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:26:54.979 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-19T01:26:54.976+0000 7f97e241e640 1 --2- 172.21.15.167:0/2382580719 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f97dc199ca0 0x7f97dc19a150 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f97d400bd50 tx=0x7f97d400be40 comp rx=0 tx=0).stop 2024-08-19T01:26:54.979 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-19T01:26:54.976+0000 7f97e241e640 1 -- 172.21.15.167:0/2382580719 shutdown_connections 2024-08-19T01:26:54.979 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-19T01:26:54.976+0000 7f97e241e640 1 --2- 172.21.15.167:0/2382580719 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f97bc0394f0 0x7f97bc03b9b0 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:26:54.979 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-19T01:26:54.976+0000 7f97e241e640 1 --2- 172.21.15.167:0/2382580719 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f97dc199ca0 0x7f97dc19a150 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:26:54.979 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-19T01:26:54.976+0000 7f97e241e640 1 --2- 172.21.15.167:0/2382580719 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f97dc1953d0 0x7f97dc199760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:26:54.979 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-19T01:26:54.976+0000 7f97e241e640 1 --2- 172.21.15.167:0/2382580719 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f97c8005740 0x7f97dc194e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:26:54.979 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-19T01:26:54.976+0000 7f97e241e640 1 -- 172.21.15.167:0/2382580719 >> 172.21.15.167:0/2382580719 conn(0x7f97dc0fbf50 msgr2=0x7f97dc0fd0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:26:54.979 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-19T01:26:54.976+0000 7f97e241e640 1 -- 172.21.15.167:0/2382580719 shutdown_connections 2024-08-19T01:26:54.979 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-19T01:26:54.976+0000 7f97e241e640 1 -- 172.21.15.167:0/2382580719 wait complete. 2024-08-19T01:26:55.655 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:26:55 smithi083 ceph-mon[17120]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-19T01:26:55.655 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:26:55 smithi083 ceph-mon[17120]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-19T01:26:55.655 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:26:55 smithi083 ceph-mon[17120]: from='client.? 172.21.15.167:0/2382580719' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-08-19T01:26:55.655 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:26:55 smithi083 ceph-mon[17120]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-08-19T01:26:55.716 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:26:55 smithi167 ceph-mon[21875]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-19T01:26:55.717 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:26:55 smithi167 ceph-mon[21875]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-19T01:26:55.717 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:26:55 smithi167 ceph-mon[21875]: from='client.? 172.21.15.167:0/2382580719' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-08-19T01:26:55.717 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:26:55 smithi167 ceph-mon[21875]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-08-19T01:26:55.866 INFO:tasks.cephadm:Generating final ceph.conf file... 2024-08-19T01:26:55.866 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid bad8780e-5dc9-11ef-bccf-c7b262605968 -- ceph config generate-minimal-conf 2024-08-19T01:26:56.256 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/bad8780e-5dc9-11ef-bccf-c7b262605968/mon.a/config 2024-08-19T01:26:56.703 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:26:56 smithi117 ceph-mon[22016]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-08-19T01:26:56.703 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:26:56 smithi117 ceph-mon[22016]: mon.c calling monitor election 2024-08-19T01:26:56.703 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:26:56 smithi117 ceph-mon[22016]: mon.a calling monitor election 2024-08-19T01:26:56.703 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:26:56 smithi117 ceph-mon[22016]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-08-19T01:26:56.704 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:26:56 smithi117 ceph-mon[22016]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-08-19T01:26:56.704 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:26:56 smithi117 ceph-mon[22016]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-19T01:26:56.704 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:26:56 smithi117 ceph-mon[22016]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-08-19T01:26:56.704 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:26:56 smithi117 ceph-mon[22016]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-08-19T01:26:56.704 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:26:56 smithi117 ceph-mon[22016]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-19T01:26:56.704 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:26:56 smithi117 ceph-mon[22016]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-08-19T01:26:56.704 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:26:56 smithi117 ceph-mon[22016]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-08-19T01:26:56.704 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:26:56 smithi117 ceph-mon[22016]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-08-19T01:26:56.704 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:26:56 smithi117 ceph-mon[22016]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-08-19T01:26:56.704 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:26:56 smithi117 ceph-mon[22016]: monmap epoch 3 2024-08-19T01:26:56.704 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:26:56 smithi117 ceph-mon[22016]: fsid bad8780e-5dc9-11ef-bccf-c7b262605968 2024-08-19T01:26:56.704 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:26:56 smithi117 ceph-mon[22016]: last_changed 2024-08-19T01:26:49.333005+0000 2024-08-19T01:26:56.704 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:26:56 smithi117 ceph-mon[22016]: created 2024-08-19T01:25:09.034517+0000 2024-08-19T01:26:56.705 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:26:56 smithi117 ceph-mon[22016]: min_mon_release 19 (squid) 2024-08-19T01:26:56.705 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:26:56 smithi117 ceph-mon[22016]: election_strategy: 1 2024-08-19T01:26:56.705 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:26:56 smithi117 ceph-mon[22016]: 0: [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] mon.a 2024-08-19T01:26:56.705 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:26:56 smithi117 ceph-mon[22016]: 1: [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] mon.c 2024-08-19T01:26:56.705 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:26:56 smithi117 ceph-mon[22016]: 2: [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] mon.b 2024-08-19T01:26:56.705 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:26:56 smithi117 ceph-mon[22016]: fsmap 2024-08-19T01:26:56.705 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:26:56 smithi117 ceph-mon[22016]: osdmap e4: 0 total, 0 up, 0 in 2024-08-19T01:26:56.705 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:26:56 smithi117 ceph-mon[22016]: mgrmap e13: x(active, since 50s) 2024-08-19T01:26:56.705 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:26:56 smithi117 ceph-mon[22016]: overall HEALTH_OK 2024-08-19T01:26:56.705 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:26:56 smithi117 ceph-mon[22016]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' 2024-08-19T01:26:56.705 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:26:56 smithi117 ceph-mon[22016]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' 2024-08-19T01:26:56.705 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:26:56 smithi117 ceph-mon[22016]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' 2024-08-19T01:26:56.705 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:26:56 smithi117 ceph-mon[22016]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' 2024-08-19T01:26:56.705 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:26:56 smithi117 ceph-mon[22016]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-19T01:26:56.706 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:26:56 smithi117 ceph-mon[22016]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-19T01:26:56.706 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:26:56 smithi117 ceph-mon[22016]: from='client.? 172.21.15.167:0/2382580719' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-08-19T01:26:56.706 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:26:56 smithi117 ceph-mon[22016]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-08-19T01:26:56.943 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:26:56.915+0000 7f47a1000640 1 -- 172.21.15.83:0/4013924883 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f479c1015c0 msgr2=0x7f479c1019a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:26:56.943 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:26:56.915+0000 7f47a1000640 1 --2- 172.21.15.83:0/4013924883 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f479c1015c0 0x7f479c1019a0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f478c009880 tx=0x7f478c02a5d0 comp rx=0 tx=0).stop 2024-08-19T01:26:56.943 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:26:56.915+0000 7f47a1000640 1 -- 172.21.15.83:0/4013924883 shutdown_connections 2024-08-19T01:26:56.943 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:26:56.915+0000 7f47a1000640 1 --2- 172.21.15.83:0/4013924883 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f479c1015c0 0x7f479c1019a0 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:26:56.943 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:26:56.915+0000 7f47a1000640 1 -- 172.21.15.83:0/4013924883 >> 172.21.15.83:0/4013924883 conn(0x7f479c076e50 msgr2=0x7f479c077260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:26:56.943 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:26:56.915+0000 7f47a1000640 1 -- 172.21.15.83:0/4013924883 shutdown_connections 2024-08-19T01:26:56.943 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:26:56.915+0000 7f47a1000640 1 -- 172.21.15.83:0/4013924883 wait complete. 2024-08-19T01:26:56.943 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:26:56.916+0000 7f47a1000640 1 Processor -- start 2024-08-19T01:26:56.943 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:26:56.916+0000 7f47a1000640 1 -- start start 2024-08-19T01:26:56.943 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:26:56.917+0000 7f47a1000640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f479c1015c0 0x7f479c199130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:26:56.944 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:26:56.917+0000 7f47a1000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f479c199670 0x7f479c19da00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:26:56.944 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:26:56.917+0000 7f47a1000640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f479c19df40 0x7f479c19e3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:26:56.944 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:26:56.917+0000 7f47a1000640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f479c10b770 con 0x7f479c199670 2024-08-19T01:26:56.944 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:26:56.917+0000 7f47a1000640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f479c10b5f0 con 0x7f479c19df40 2024-08-19T01:26:56.944 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:26:56.917+0000 7f47a1000640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f479c10b8f0 con 0x7f479c1015c0 2024-08-19T01:26:56.944 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:26:56.917+0000 7f479aa00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f479c199670 0x7f479c19da00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:26:56.944 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:26:56.917+0000 7f479b400640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f479c1015c0 0x7f479c199130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:26:56.944 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:26:56.917+0000 7f479aa00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f479c199670 0x7f479c19da00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:49032/0 (socket says 172.21.15.83:49032) 2024-08-19T01:26:56.944 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:26:56.917+0000 7f479aa00640 1 -- 172.21.15.83:0/374129258 learned_addr learned my addr 172.21.15.83:0/374129258 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-08-19T01:26:56.944 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:26:56.917+0000 7f479be00640 1 --2- 172.21.15.83:0/374129258 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f479c19df40 0x7f479c19e3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:26:56.944 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:26:56.917+0000 7f479aa00640 1 -- 172.21.15.83:0/374129258 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f479c1015c0 msgr2=0x7f479c199130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:26:56.944 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:26:56.917+0000 7f479aa00640 1 --2- 172.21.15.83:0/374129258 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f479c1015c0 0x7f479c199130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:26:56.945 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:26:56.917+0000 7f479aa00640 1 -- 172.21.15.83:0/374129258 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f479c19df40 msgr2=0x7f479c19e3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:26:56.945 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:26:56.917+0000 7f479aa00640 1 --2- 172.21.15.83:0/374129258 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f479c19df40 0x7f479c19e3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:26:56.945 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:26:56.917+0000 7f479aa00640 1 -- 172.21.15.83:0/374129258 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f479c1a5cb0 con 0x7f479c199670 2024-08-19T01:26:56.945 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:26:56.917+0000 7f479b400640 1 --2- 172.21.15.83:0/374129258 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f479c1015c0 0x7f479c199130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-19T01:26:56.945 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:26:56.917+0000 7f479be00640 1 --2- 172.21.15.83:0/374129258 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f479c19df40 0x7f479c19e3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-19T01:26:56.945 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:26:56.918+0000 7f479aa00640 1 --2- 172.21.15.83:0/374129258 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f479c199670 0x7f479c19da00 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f478800b520 tx=0x7f478800b9e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:26:56.945 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:26:56.918+0000 7f4793e00640 1 -- 172.21.15.83:0/374129258 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4788004230 con 0x7f479c199670 2024-08-19T01:26:56.945 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:26:56.918+0000 7f4793e00640 1 -- 172.21.15.83:0/374129258 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f47880043d0 con 0x7f479c199670 2024-08-19T01:26:56.945 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:26:56.918+0000 7f47a1000640 1 -- 172.21.15.83:0/374129258 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f479c1a5ee0 con 0x7f479c199670 2024-08-19T01:26:56.945 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:26:56.918+0000 7f47a1000640 1 -- 172.21.15.83:0/374129258 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f479c1a63e0 con 0x7f479c199670 2024-08-19T01:26:56.945 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:26:56.918+0000 7f4793e00640 1 -- 172.21.15.83:0/374129258 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4788003d60 con 0x7f479c199670 2024-08-19T01:26:56.945 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:26:56.919+0000 7f47a1000640 1 -- 172.21.15.83:0/374129258 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f479c101a20 con 0x7f479c199670 2024-08-19T01:26:56.945 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:26:56.919+0000 7f4793e00640 1 -- 172.21.15.83:0/374129258 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 13) ==== 46343+0+0 (secure 0 0 0) 0x7f478800e070 con 0x7f479c199670 2024-08-19T01:26:56.946 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:26:56.920+0000 7f4793e00640 1 --2- 172.21.15.83:0/374129258 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f47800394f0 0x7f478003b9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:26:56.946 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:26:56.920+0000 7f4793e00640 1 -- 172.21.15.83:0/374129258 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f478804b920 con 0x7f479c199670 2024-08-19T01:26:56.946 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:26:56.920+0000 7f479b400640 1 --2- 172.21.15.83:0/374129258 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f47800394f0 0x7f478003b9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:26:56.946 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:26:56.921+0000 7f479b400640 1 --2- 172.21.15.83:0/374129258 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f47800394f0 0x7f478003b9b0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f478c000c00 tx=0x7f478c005a20 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:26:56.946 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:26:56.926+0000 7f4793e00640 1 -- 172.21.15.83:0/374129258 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7f4788014020 con 0x7f479c199670 2024-08-19T01:26:57.116 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:26:57.114+0000 7f47a1000640 1 -- 172.21.15.83:0/374129258 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) -- 0x7f479c1a6820 con 0x7f479c199670 2024-08-19T01:26:57.116 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:26:57.115+0000 7f4793e00640 1 -- 172.21.15.83:0/374129258 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v9) ==== 76+0+275 (secure 0 0 0) 0x7f4788010d70 con 0x7f479c199670 2024-08-19T01:26:57.116 INFO:teuthology.orchestra.run.smithi083.stdout:# minimal ceph.conf for bad8780e-5dc9-11ef-bccf-c7b262605968 2024-08-19T01:26:57.116 INFO:teuthology.orchestra.run.smithi083.stdout:[global] 2024-08-19T01:26:57.116 INFO:teuthology.orchestra.run.smithi083.stdout: fsid = bad8780e-5dc9-11ef-bccf-c7b262605968 2024-08-19T01:26:57.116 INFO:teuthology.orchestra.run.smithi083.stdout: mon_host = [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] 2024-08-19T01:26:57.118 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:26:57.117+0000 7f47a1000640 1 -- 172.21.15.83:0/374129258 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f47800394f0 msgr2=0x7f478003b9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:26:57.119 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:26:57.117+0000 7f47a1000640 1 --2- 172.21.15.83:0/374129258 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f47800394f0 0x7f478003b9b0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f478c000c00 tx=0x7f478c005a20 comp rx=0 tx=0).stop 2024-08-19T01:26:57.119 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:26:57.117+0000 7f47a1000640 1 -- 172.21.15.83:0/374129258 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f479c199670 msgr2=0x7f479c19da00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:26:57.119 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:26:57.117+0000 7f47a1000640 1 --2- 172.21.15.83:0/374129258 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f479c199670 0x7f479c19da00 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f478800b520 tx=0x7f478800b9e0 comp rx=0 tx=0).stop 2024-08-19T01:26:57.119 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:26:57.117+0000 7f47a1000640 1 -- 172.21.15.83:0/374129258 shutdown_connections 2024-08-19T01:26:57.119 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:26:57.117+0000 7f47a1000640 1 --2- 172.21.15.83:0/374129258 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f47800394f0 0x7f478003b9b0 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:26:57.119 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:26:57.117+0000 7f47a1000640 1 --2- 172.21.15.83:0/374129258 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f479c19df40 0x7f479c19e3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:26:57.119 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:26:57.117+0000 7f47a1000640 1 --2- 172.21.15.83:0/374129258 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f479c199670 0x7f479c19da00 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:26:57.119 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:26:57.117+0000 7f47a1000640 1 --2- 172.21.15.83:0/374129258 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f479c1015c0 0x7f479c199130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:26:57.119 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:26:57.117+0000 7f47a1000640 1 -- 172.21.15.83:0/374129258 >> 172.21.15.83:0/374129258 conn(0x7f479c076e50 msgr2=0x7f479c074150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:26:57.119 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:26:57.117+0000 7f47a1000640 1 -- 172.21.15.83:0/374129258 shutdown_connections 2024-08-19T01:26:57.119 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:26:57.117+0000 7f47a1000640 1 -- 172.21.15.83:0/374129258 wait complete. 2024-08-19T01:26:57.401 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:26:57 smithi083 ceph-mon[17120]: mon.b calling monitor election 2024-08-19T01:26:57.401 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:26:57 smithi083 ceph-mon[17120]: Updating smithi083:/etc/ceph/ceph.conf 2024-08-19T01:26:57.401 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:26:57 smithi083 ceph-mon[17120]: Updating smithi117:/etc/ceph/ceph.conf 2024-08-19T01:26:57.401 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:26:57 smithi083 ceph-mon[17120]: Updating smithi167:/etc/ceph/ceph.conf 2024-08-19T01:26:57.402 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:26:57 smithi083 ceph-mon[17120]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-19T01:26:57.402 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:26:57 smithi083 ceph-mon[17120]: mon.b calling monitor election 2024-08-19T01:26:57.402 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:26:57 smithi083 ceph-mon[17120]: mon.c calling monitor election 2024-08-19T01:26:57.402 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:26:57 smithi083 ceph-mon[17120]: mon.a calling monitor election 2024-08-19T01:26:57.402 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:26:57 smithi083 ceph-mon[17120]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-08-19T01:26:57.402 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:26:57 smithi083 ceph-mon[17120]: monmap epoch 3 2024-08-19T01:26:57.402 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:26:57 smithi083 ceph-mon[17120]: fsid bad8780e-5dc9-11ef-bccf-c7b262605968 2024-08-19T01:26:57.402 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:26:57 smithi083 ceph-mon[17120]: last_changed 2024-08-19T01:26:49.333005+0000 2024-08-19T01:26:57.402 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:26:57 smithi083 ceph-mon[17120]: created 2024-08-19T01:25:09.034517+0000 2024-08-19T01:26:57.402 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:26:57 smithi083 ceph-mon[17120]: min_mon_release 19 (squid) 2024-08-19T01:26:57.402 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:26:57 smithi083 ceph-mon[17120]: election_strategy: 1 2024-08-19T01:26:57.402 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:26:57 smithi083 ceph-mon[17120]: 0: [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] mon.a 2024-08-19T01:26:57.402 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:26:57 smithi083 ceph-mon[17120]: 1: [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] mon.c 2024-08-19T01:26:57.403 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:26:57 smithi083 ceph-mon[17120]: 2: [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] mon.b 2024-08-19T01:26:57.403 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:26:57 smithi083 ceph-mon[17120]: fsmap 2024-08-19T01:26:57.403 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:26:57 smithi083 ceph-mon[17120]: osdmap e4: 0 total, 0 up, 0 in 2024-08-19T01:26:57.403 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:26:57 smithi083 ceph-mon[17120]: mgrmap e13: x(active, since 52s) 2024-08-19T01:26:57.403 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:26:57 smithi083 ceph-mon[17120]: overall HEALTH_OK 2024-08-19T01:26:57.403 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:26:57 smithi083 ceph-mon[17120]: Updating smithi167:/var/lib/ceph/bad8780e-5dc9-11ef-bccf-c7b262605968/config/ceph.conf 2024-08-19T01:26:57.403 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:26:57 smithi083 ceph-mon[17120]: Updating smithi117:/var/lib/ceph/bad8780e-5dc9-11ef-bccf-c7b262605968/config/ceph.conf 2024-08-19T01:26:57.403 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:26:57 smithi083 ceph-mon[17120]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' 2024-08-19T01:26:57.403 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:26:57 smithi083 ceph-mon[17120]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' 2024-08-19T01:26:57.403 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:26:57 smithi083 ceph-mon[17120]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' 2024-08-19T01:26:57.403 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:26:57 smithi083 ceph-mon[17120]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' 2024-08-19T01:26:57.453 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:26:57 smithi117 ceph-mon[22016]: mon.b calling monitor election 2024-08-19T01:26:57.453 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:26:57 smithi117 ceph-mon[22016]: Updating smithi083:/etc/ceph/ceph.conf 2024-08-19T01:26:57.453 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:26:57 smithi117 ceph-mon[22016]: Updating smithi117:/etc/ceph/ceph.conf 2024-08-19T01:26:57.453 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:26:57 smithi117 ceph-mon[22016]: Updating smithi167:/etc/ceph/ceph.conf 2024-08-19T01:26:57.453 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:26:57 smithi117 ceph-mon[22016]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-19T01:26:57.454 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:26:57 smithi117 ceph-mon[22016]: mon.b calling monitor election 2024-08-19T01:26:57.454 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:26:57 smithi117 ceph-mon[22016]: mon.c calling monitor election 2024-08-19T01:26:57.454 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:26:57 smithi117 ceph-mon[22016]: mon.a calling monitor election 2024-08-19T01:26:57.454 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:26:57 smithi117 ceph-mon[22016]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-08-19T01:26:57.454 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:26:57 smithi117 ceph-mon[22016]: monmap epoch 3 2024-08-19T01:26:57.454 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:26:57 smithi117 ceph-mon[22016]: fsid bad8780e-5dc9-11ef-bccf-c7b262605968 2024-08-19T01:26:57.454 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:26:57 smithi117 ceph-mon[22016]: last_changed 2024-08-19T01:26:49.333005+0000 2024-08-19T01:26:57.454 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:26:57 smithi117 ceph-mon[22016]: created 2024-08-19T01:25:09.034517+0000 2024-08-19T01:26:57.454 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:26:57 smithi117 ceph-mon[22016]: min_mon_release 19 (squid) 2024-08-19T01:26:57.454 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:26:57 smithi117 ceph-mon[22016]: election_strategy: 1 2024-08-19T01:26:57.454 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:26:57 smithi117 ceph-mon[22016]: 0: [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] mon.a 2024-08-19T01:26:57.454 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:26:57 smithi117 ceph-mon[22016]: 1: [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] mon.c 2024-08-19T01:26:57.454 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:26:57 smithi117 ceph-mon[22016]: 2: [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] mon.b 2024-08-19T01:26:57.455 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:26:57 smithi117 ceph-mon[22016]: fsmap 2024-08-19T01:26:57.455 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:26:57 smithi117 ceph-mon[22016]: osdmap e4: 0 total, 0 up, 0 in 2024-08-19T01:26:57.455 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:26:57 smithi117 ceph-mon[22016]: mgrmap e13: x(active, since 52s) 2024-08-19T01:26:57.455 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:26:57 smithi117 ceph-mon[22016]: overall HEALTH_OK 2024-08-19T01:26:57.455 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:26:57 smithi117 ceph-mon[22016]: Updating smithi167:/var/lib/ceph/bad8780e-5dc9-11ef-bccf-c7b262605968/config/ceph.conf 2024-08-19T01:26:57.455 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:26:57 smithi117 ceph-mon[22016]: Updating smithi117:/var/lib/ceph/bad8780e-5dc9-11ef-bccf-c7b262605968/config/ceph.conf 2024-08-19T01:26:57.455 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:26:57 smithi117 ceph-mon[22016]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' 2024-08-19T01:26:57.455 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:26:57 smithi117 ceph-mon[22016]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' 2024-08-19T01:26:57.455 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:26:57 smithi117 ceph-mon[22016]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' 2024-08-19T01:26:57.455 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:26:57 smithi117 ceph-mon[22016]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' 2024-08-19T01:26:57.467 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:26:57 smithi167 ceph-mon[21875]: mon.b calling monitor election 2024-08-19T01:26:57.467 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:26:57 smithi167 ceph-mon[21875]: Updating smithi083:/etc/ceph/ceph.conf 2024-08-19T01:26:57.467 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:26:57 smithi167 ceph-mon[21875]: Updating smithi117:/etc/ceph/ceph.conf 2024-08-19T01:26:57.467 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:26:57 smithi167 ceph-mon[21875]: Updating smithi167:/etc/ceph/ceph.conf 2024-08-19T01:26:57.467 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:26:57 smithi167 ceph-mon[21875]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-19T01:26:57.467 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:26:57 smithi167 ceph-mon[21875]: mon.b calling monitor election 2024-08-19T01:26:57.467 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:26:57 smithi167 ceph-mon[21875]: mon.c calling monitor election 2024-08-19T01:26:57.467 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:26:57 smithi167 ceph-mon[21875]: mon.a calling monitor election 2024-08-19T01:26:57.467 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:26:57 smithi167 ceph-mon[21875]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-08-19T01:26:57.467 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:26:57 smithi167 ceph-mon[21875]: monmap epoch 3 2024-08-19T01:26:57.467 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:26:57 smithi167 ceph-mon[21875]: fsid bad8780e-5dc9-11ef-bccf-c7b262605968 2024-08-19T01:26:57.467 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:26:57 smithi167 ceph-mon[21875]: last_changed 2024-08-19T01:26:49.333005+0000 2024-08-19T01:26:57.468 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:26:57 smithi167 ceph-mon[21875]: created 2024-08-19T01:25:09.034517+0000 2024-08-19T01:26:57.468 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:26:57 smithi167 ceph-mon[21875]: min_mon_release 19 (squid) 2024-08-19T01:26:57.468 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:26:57 smithi167 ceph-mon[21875]: election_strategy: 1 2024-08-19T01:26:57.468 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:26:57 smithi167 ceph-mon[21875]: 0: [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] mon.a 2024-08-19T01:26:57.468 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:26:57 smithi167 ceph-mon[21875]: 1: [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] mon.c 2024-08-19T01:26:57.468 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:26:57 smithi167 ceph-mon[21875]: 2: [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] mon.b 2024-08-19T01:26:57.468 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:26:57 smithi167 ceph-mon[21875]: fsmap 2024-08-19T01:26:57.468 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:26:57 smithi167 ceph-mon[21875]: osdmap e4: 0 total, 0 up, 0 in 2024-08-19T01:26:57.468 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:26:57 smithi167 ceph-mon[21875]: mgrmap e13: x(active, since 52s) 2024-08-19T01:26:57.468 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:26:57 smithi167 ceph-mon[21875]: overall HEALTH_OK 2024-08-19T01:26:57.468 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:26:57 smithi167 ceph-mon[21875]: Updating smithi167:/var/lib/ceph/bad8780e-5dc9-11ef-bccf-c7b262605968/config/ceph.conf 2024-08-19T01:26:57.468 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:26:57 smithi167 ceph-mon[21875]: Updating smithi117:/var/lib/ceph/bad8780e-5dc9-11ef-bccf-c7b262605968/config/ceph.conf 2024-08-19T01:26:57.468 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:26:57 smithi167 ceph-mon[21875]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' 2024-08-19T01:26:57.468 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:26:57 smithi167 ceph-mon[21875]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' 2024-08-19T01:26:57.469 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:26:57 smithi167 ceph-mon[21875]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' 2024-08-19T01:26:57.469 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:26:57 smithi167 ceph-mon[21875]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' 2024-08-19T01:26:57.876 INFO:tasks.cephadm:Distributing (final) config and client.admin keyring... 2024-08-19T01:26:57.877 DEBUG:teuthology.orchestra.run.smithi083:> set -ex 2024-08-19T01:26:57.877 DEBUG:teuthology.orchestra.run.smithi083:> sudo dd of=/etc/ceph/ceph.conf 2024-08-19T01:26:57.934 DEBUG:teuthology.orchestra.run.smithi083:> set -ex 2024-08-19T01:26:57.934 DEBUG:teuthology.orchestra.run.smithi083:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-08-19T01:26:58.029 DEBUG:teuthology.orchestra.run.smithi117:> set -ex 2024-08-19T01:26:58.029 DEBUG:teuthology.orchestra.run.smithi117:> sudo dd of=/etc/ceph/ceph.conf 2024-08-19T01:26:58.062 DEBUG:teuthology.orchestra.run.smithi117:> set -ex 2024-08-19T01:26:58.063 DEBUG:teuthology.orchestra.run.smithi117:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-08-19T01:26:58.132 DEBUG:teuthology.orchestra.run.smithi167:> set -ex 2024-08-19T01:26:58.132 DEBUG:teuthology.orchestra.run.smithi167:> sudo dd of=/etc/ceph/ceph.conf 2024-08-19T01:26:58.164 DEBUG:teuthology.orchestra.run.smithi167:> set -ex 2024-08-19T01:26:58.164 DEBUG:teuthology.orchestra.run.smithi167:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-08-19T01:26:58.232 INFO:tasks.cephadm:Adding mgr.x on smithi083 2024-08-19T01:26:58.232 INFO:tasks.cephadm:Adding mgr.y on smithi117 2024-08-19T01:26:58.232 INFO:tasks.cephadm:Adding mgr.z on smithi167 2024-08-19T01:26:58.232 DEBUG:teuthology.orchestra.run.smithi167:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid bad8780e-5dc9-11ef-bccf-c7b262605968 -- ceph orch apply mgr '3;smithi083=x;smithi117=y;smithi167=z' 2024-08-19T01:26:58.241 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:26:58 smithi083 ceph-mon[17120]: Updating smithi083:/var/lib/ceph/bad8780e-5dc9-11ef-bccf-c7b262605968/config/ceph.conf 2024-08-19T01:26:58.241 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:26:58 smithi083 ceph-mon[17120]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' 2024-08-19T01:26:58.242 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:26:58 smithi083 ceph-mon[17120]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' 2024-08-19T01:26:58.242 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:26:58 smithi083 ceph-mon[17120]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' 2024-08-19T01:26:58.242 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:26:58 smithi083 ceph-mon[17120]: from='client.? 172.21.15.83:0/374129258' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-19T01:26:58.242 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:26:58 smithi083 ceph-mon[17120]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' 2024-08-19T01:26:58.242 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:26:58 smithi083 ceph-mon[17120]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' 2024-08-19T01:26:58.242 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:26:58 smithi083 ceph-mon[17120]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' 2024-08-19T01:26:58.242 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:26:58 smithi083 ceph-mon[17120]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' 2024-08-19T01:26:58.242 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:26:58 smithi083 ceph-mon[17120]: Reconfiguring mon.a (unknown last config time)... 2024-08-19T01:26:58.242 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:26:58 smithi083 ceph-mon[17120]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-08-19T01:26:58.242 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:26:58 smithi083 ceph-mon[17120]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-08-19T01:26:58.242 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:26:58 smithi083 ceph-mon[17120]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-19T01:26:58.243 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:26:58 smithi083 ceph-mon[17120]: Reconfiguring daemon mon.a on smithi083 2024-08-19T01:26:58.243 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:26:58 smithi083 ceph-mon[17120]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-08-19T01:26:58.453 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:26:58 smithi117 ceph-mon[22016]: Updating smithi083:/var/lib/ceph/bad8780e-5dc9-11ef-bccf-c7b262605968/config/ceph.conf 2024-08-19T01:26:58.453 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:26:58 smithi117 ceph-mon[22016]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' 2024-08-19T01:26:58.453 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:26:58 smithi117 ceph-mon[22016]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' 2024-08-19T01:26:58.453 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:26:58 smithi117 ceph-mon[22016]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' 2024-08-19T01:26:58.454 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:26:58 smithi117 ceph-mon[22016]: from='client.? 172.21.15.83:0/374129258' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-19T01:26:58.454 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:26:58 smithi117 ceph-mon[22016]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' 2024-08-19T01:26:58.454 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:26:58 smithi117 ceph-mon[22016]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' 2024-08-19T01:26:58.454 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:26:58 smithi117 ceph-mon[22016]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' 2024-08-19T01:26:58.454 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:26:58 smithi117 ceph-mon[22016]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' 2024-08-19T01:26:58.454 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:26:58 smithi117 ceph-mon[22016]: Reconfiguring mon.a (unknown last config time)... 2024-08-19T01:26:58.454 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:26:58 smithi117 ceph-mon[22016]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-08-19T01:26:58.454 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:26:58 smithi117 ceph-mon[22016]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-08-19T01:26:58.454 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:26:58 smithi117 ceph-mon[22016]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-19T01:26:58.454 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:26:58 smithi117 ceph-mon[22016]: Reconfiguring daemon mon.a on smithi083 2024-08-19T01:26:58.454 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:26:58 smithi117 ceph-mon[22016]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-08-19T01:26:58.466 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:26:58 smithi167 ceph-mon[21875]: Updating smithi083:/var/lib/ceph/bad8780e-5dc9-11ef-bccf-c7b262605968/config/ceph.conf 2024-08-19T01:26:58.466 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:26:58 smithi167 ceph-mon[21875]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' 2024-08-19T01:26:58.466 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:26:58 smithi167 ceph-mon[21875]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' 2024-08-19T01:26:58.467 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:26:58 smithi167 ceph-mon[21875]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' 2024-08-19T01:26:58.467 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:26:58 smithi167 ceph-mon[21875]: from='client.? 172.21.15.83:0/374129258' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-19T01:26:58.467 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:26:58 smithi167 ceph-mon[21875]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' 2024-08-19T01:26:58.467 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:26:58 smithi167 ceph-mon[21875]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' 2024-08-19T01:26:58.467 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:26:58 smithi167 ceph-mon[21875]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' 2024-08-19T01:26:58.467 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:26:58 smithi167 ceph-mon[21875]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' 2024-08-19T01:26:58.467 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:26:58 smithi167 ceph-mon[21875]: Reconfiguring mon.a (unknown last config time)... 2024-08-19T01:26:58.467 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:26:58 smithi167 ceph-mon[21875]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-08-19T01:26:58.467 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:26:58 smithi167 ceph-mon[21875]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-08-19T01:26:58.467 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:26:58 smithi167 ceph-mon[21875]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-19T01:26:58.467 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:26:58 smithi167 ceph-mon[21875]: Reconfiguring daemon mon.a on smithi083 2024-08-19T01:26:58.467 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:26:58 smithi167 ceph-mon[21875]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-08-19T01:26:58.518 INFO:teuthology.orchestra.run.smithi167.stderr:Inferring config /var/lib/ceph/bad8780e-5dc9-11ef-bccf-c7b262605968/mon.c/config 2024-08-19T01:26:58.655 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Aug 19 01:26:58 smithi083 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-mgr-x[17344]: 2024-08-19T01:26:58.333+0000 7f02b2400640 -1 mgr.server handle_report got status from non-daemon mon.b 2024-08-19T01:26:58.929 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-19T01:26:58.924+0000 7f800f76d640 1 -- 172.21.15.167:0/3235013651 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8008101410 msgr2=0x7f8008101810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:26:58.929 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-19T01:26:58.924+0000 7f800f76d640 1 --2- 172.21.15.167:0/3235013651 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8008101410 0x7f8008101810 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f8004009880 tx=0x7f800402a5d0 comp rx=0 tx=0).stop 2024-08-19T01:26:58.929 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-19T01:26:58.924+0000 7f800f76d640 1 -- 172.21.15.167:0/3235013651 shutdown_connections 2024-08-19T01:26:58.929 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-19T01:26:58.924+0000 7f800f76d640 1 --2- 172.21.15.167:0/3235013651 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8008101410 0x7f8008101810 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:26:58.929 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-19T01:26:58.924+0000 7f800f76d640 1 -- 172.21.15.167:0/3235013651 >> 172.21.15.167:0/3235013651 conn(0x7f80080fcbc0 msgr2=0x7f80080fefe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:26:58.929 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-19T01:26:58.924+0000 7f800f76d640 1 -- 172.21.15.167:0/3235013651 shutdown_connections 2024-08-19T01:26:58.929 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-19T01:26:58.924+0000 7f800f76d640 1 -- 172.21.15.167:0/3235013651 wait complete. 2024-08-19T01:26:58.929 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-19T01:26:58.924+0000 7f800f76d640 1 Processor -- start 2024-08-19T01:26:58.929 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-19T01:26:58.925+0000 7f800f76d640 1 -- start start 2024-08-19T01:26:58.929 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-19T01:26:58.925+0000 7f800f76d640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8008101410 0x7f8008075cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:26:58.929 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-19T01:26:58.925+0000 7f800f76d640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8008074310 0x7f8008074790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:26:58.930 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-19T01:26:58.925+0000 7f800f76d640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8008074cd0 0x7f80081a0d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:26:58.930 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-19T01:26:58.925+0000 7f800f76d640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f8008107500 con 0x7f8008074cd0 2024-08-19T01:26:58.930 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-19T01:26:58.925+0000 7f800f76d640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f8008107380 con 0x7f8008101410 2024-08-19T01:26:58.930 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-19T01:26:58.925+0000 7f800f76d640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f8008107680 con 0x7f8008074310 2024-08-19T01:26:58.930 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-19T01:26:58.925+0000 7f800cd08640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8008074310 0x7f8008074790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:26:58.930 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-19T01:26:58.925+0000 7f800cd08640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8008074310 0x7f8008074790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:59196/0 (socket says 172.21.15.167:59196) 2024-08-19T01:26:58.930 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-19T01:26:58.925+0000 7f800cd08640 1 -- 172.21.15.167:0/1053794424 learned_addr learned my addr 172.21.15.167:0/1053794424 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-08-19T01:26:58.930 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-19T01:26:58.925+0000 7f800dd0a640 1 --2- 172.21.15.167:0/1053794424 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8008074cd0 0x7f80081a0d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:26:58.930 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-19T01:26:58.925+0000 7f800d509640 1 --2- 172.21.15.167:0/1053794424 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8008101410 0x7f8008075cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:26:58.930 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-19T01:26:58.925+0000 7f800cd08640 1 -- 172.21.15.167:0/1053794424 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8008101410 msgr2=0x7f8008075cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:26:58.930 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-19T01:26:58.925+0000 7f800cd08640 1 --2- 172.21.15.167:0/1053794424 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8008101410 0x7f8008075cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:26:58.930 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-19T01:26:58.925+0000 7f800cd08640 1 -- 172.21.15.167:0/1053794424 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8008074cd0 msgr2=0x7f80081a0d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:26:58.931 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-19T01:26:58.925+0000 7f800cd08640 1 --2- 172.21.15.167:0/1053794424 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8008074cd0 0x7f80081a0d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:26:58.931 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-19T01:26:58.926+0000 7f800cd08640 1 -- 172.21.15.167:0/1053794424 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f80081a1300 con 0x7f8008074310 2024-08-19T01:26:58.931 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-19T01:26:58.926+0000 7f800dd0a640 1 --2- 172.21.15.167:0/1053794424 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8008074cd0 0x7f80081a0d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-19T01:26:58.931 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-19T01:26:58.926+0000 7f800cd08640 1 --2- 172.21.15.167:0/1053794424 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8008074310 0x7f8008074790 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f7ff800b520 tx=0x7f7ff800b9e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:26:58.931 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-19T01:26:58.926+0000 7f7ffe7fc640 1 -- 172.21.15.167:0/1053794424 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ff8004230 con 0x7f8008074310 2024-08-19T01:26:58.931 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-19T01:26:58.926+0000 7f7ffe7fc640 1 -- 172.21.15.167:0/1053794424 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7ff80043d0 con 0x7f8008074310 2024-08-19T01:26:58.931 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-19T01:26:58.926+0000 7f7ffe7fc640 1 -- 172.21.15.167:0/1053794424 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ff8003d60 con 0x7f8008074310 2024-08-19T01:26:58.931 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-19T01:26:58.926+0000 7f800f76d640 1 -- 172.21.15.167:0/1053794424 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f80081a15f0 con 0x7f8008074310 2024-08-19T01:26:58.931 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-19T01:26:58.926+0000 7f800f76d640 1 -- 172.21.15.167:0/1053794424 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f80081a1b60 con 0x7f8008074310 2024-08-19T01:26:58.932 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-19T01:26:58.927+0000 7f7ffe7fc640 1 -- 172.21.15.167:0/1053794424 <== mon.1 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 13) ==== 46343+0+0 (secure 0 0 0) 0x7f7ff800e070 con 0x7f8008074310 2024-08-19T01:26:58.932 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-19T01:26:58.927+0000 7f800f76d640 1 -- 172.21.15.167:0/1053794424 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f80081902c0 con 0x7f8008074310 2024-08-19T01:26:58.932 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-19T01:26:58.927+0000 7f7ffe7fc640 1 --2- 172.21.15.167:0/1053794424 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f7ff40394f0 0x7f7ff403b9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:26:58.932 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-19T01:26:58.928+0000 7f7ffe7fc640 1 -- 172.21.15.167:0/1053794424 <== mon.1 v2:172.21.15.167:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f7ff804b910 con 0x7f8008074310 2024-08-19T01:26:58.932 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-19T01:26:58.928+0000 7f800d509640 1 --2- 172.21.15.167:0/1053794424 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f7ff40394f0 0x7f7ff403b9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:26:58.932 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-19T01:26:58.929+0000 7f800d509640 1 --2- 172.21.15.167:0/1053794424 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f7ff40394f0 0x7f7ff403b9b0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f8004009230 tx=0x7f80040315d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:26:58.935 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-19T01:26:58.932+0000 7f7ffe7fc640 1 -- 172.21.15.167:0/1053794424 <== mon.1 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7f7ff8014020 con 0x7f8008074310 2024-08-19T01:26:59.113 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-19T01:26:59.110+0000 7f800f76d640 1 -- 172.21.15.167:0/1053794424 --> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi083=x;smithi117=y;smithi167=z", "target": ["mon-mgr", ""]}) -- 0x7f80080623f0 con 0x7f7ff40394f0 2024-08-19T01:26:59.122 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-19T01:26:59.119+0000 7f7ffe7fc640 1 -- 172.21.15.167:0/1053794424 <== mgr.14150 v2:172.21.15.83:6800/683740916 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7f80080623f0 con 0x7f7ff40394f0 2024-08-19T01:26:59.122 INFO:teuthology.orchestra.run.smithi167.stdout:Scheduled mgr update... 2024-08-19T01:26:59.124 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-19T01:26:59.121+0000 7f800f76d640 1 -- 172.21.15.167:0/1053794424 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f7ff40394f0 msgr2=0x7f7ff403b9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:26:59.124 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-19T01:26:59.121+0000 7f800f76d640 1 --2- 172.21.15.167:0/1053794424 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f7ff40394f0 0x7f7ff403b9b0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f8004009230 tx=0x7f80040315d0 comp rx=0 tx=0).stop 2024-08-19T01:26:59.124 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-19T01:26:59.121+0000 7f800f76d640 1 -- 172.21.15.167:0/1053794424 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8008074310 msgr2=0x7f8008074790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:26:59.124 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-19T01:26:59.121+0000 7f800f76d640 1 --2- 172.21.15.167:0/1053794424 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8008074310 0x7f8008074790 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f7ff800b520 tx=0x7f7ff800b9e0 comp rx=0 tx=0).stop 2024-08-19T01:26:59.124 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-19T01:26:59.121+0000 7f800f76d640 1 -- 172.21.15.167:0/1053794424 shutdown_connections 2024-08-19T01:26:59.124 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-19T01:26:59.121+0000 7f800f76d640 1 --2- 172.21.15.167:0/1053794424 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f7ff40394f0 0x7f7ff403b9b0 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:26:59.124 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-19T01:26:59.121+0000 7f800f76d640 1 --2- 172.21.15.167:0/1053794424 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8008074cd0 0x7f80081a0d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:26:59.125 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-19T01:26:59.121+0000 7f800f76d640 1 --2- 172.21.15.167:0/1053794424 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8008074310 0x7f8008074790 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:26:59.125 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-19T01:26:59.121+0000 7f800f76d640 1 --2- 172.21.15.167:0/1053794424 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8008101410 0x7f8008075cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:26:59.125 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-19T01:26:59.121+0000 7f800f76d640 1 -- 172.21.15.167:0/1053794424 >> 172.21.15.167:0/1053794424 conn(0x7f80080fcbc0 msgr2=0x7f80080fd6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:26:59.125 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-19T01:26:59.122+0000 7f800f76d640 1 -- 172.21.15.167:0/1053794424 shutdown_connections 2024-08-19T01:26:59.125 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-19T01:26:59.122+0000 7f800f76d640 1 -- 172.21.15.167:0/1053794424 wait complete. 2024-08-19T01:26:59.216 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:26:59 smithi167 ceph-mon[21875]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-19T01:26:59.405 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:26:59 smithi083 ceph-mon[17120]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-19T01:26:59.453 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:26:59 smithi117 ceph-mon[22016]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-19T01:26:59.649 DEBUG:teuthology.orchestra.run.smithi117:mgr.y> sudo journalctl -f -n 0 -u ceph-bad8780e-5dc9-11ef-bccf-c7b262605968@mgr.y.service 2024-08-19T01:26:59.652 DEBUG:teuthology.orchestra.run.smithi167:mgr.z> sudo journalctl -f -n 0 -u ceph-bad8780e-5dc9-11ef-bccf-c7b262605968@mgr.z.service 2024-08-19T01:26:59.655 INFO:tasks.cephadm:Deploying OSDs... 2024-08-19T01:26:59.655 DEBUG:teuthology.orchestra.run.smithi083:> set -ex 2024-08-19T01:26:59.655 DEBUG:teuthology.orchestra.run.smithi083:> dd if=/scratch_devs of=/dev/stdout 2024-08-19T01:26:59.691 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-08-19T01:26:59.691 DEBUG:teuthology.orchestra.run.smithi083:> stat /dev/vg_nvme/lv_1 2024-08-19T01:26:59.768 INFO:teuthology.orchestra.run.smithi083.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-08-19T01:26:59.768 INFO:teuthology.orchestra.run.smithi083.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-08-19T01:26:59.768 INFO:teuthology.orchestra.run.smithi083.stdout:Device: 6h/6d Inode: 634 Links: 1 2024-08-19T01:26:59.768 INFO:teuthology.orchestra.run.smithi083.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-08-19T01:26:59.768 INFO:teuthology.orchestra.run.smithi083.stdout:Context: system_u:object_r:device_t:s0 2024-08-19T01:26:59.768 INFO:teuthology.orchestra.run.smithi083.stdout:Access: 2024-08-19 01:26:17.541095345 +0000 2024-08-19T01:26:59.768 INFO:teuthology.orchestra.run.smithi083.stdout:Modify: 2024-08-19 01:21:58.045428315 +0000 2024-08-19T01:26:59.768 INFO:teuthology.orchestra.run.smithi083.stdout:Change: 2024-08-19 01:21:58.045428315 +0000 2024-08-19T01:26:59.768 INFO:teuthology.orchestra.run.smithi083.stdout: Birth: 2024-08-19 01:21:58.045428315 +0000 2024-08-19T01:26:59.769 DEBUG:teuthology.orchestra.run.smithi083:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-08-19T01:26:59.882 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-08-19T01:26:59.882 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-08-19T01:26:59.883 INFO:teuthology.orchestra.run.smithi083.stderr:512 bytes copied, 0.000285551 s, 1.8 MB/s 2024-08-19T01:26:59.885 DEBUG:teuthology.orchestra.run.smithi083:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-08-19T01:26:59.922 DEBUG:teuthology.orchestra.run.smithi083:> stat /dev/vg_nvme/lv_2 2024-08-19T01:27:00.001 INFO:teuthology.orchestra.run.smithi083.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-08-19T01:27:00.001 INFO:teuthology.orchestra.run.smithi083.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-08-19T01:27:00.001 INFO:teuthology.orchestra.run.smithi083.stdout:Device: 6h/6d Inode: 653 Links: 1 2024-08-19T01:27:00.001 INFO:teuthology.orchestra.run.smithi083.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-08-19T01:27:00.001 INFO:teuthology.orchestra.run.smithi083.stdout:Context: system_u:object_r:device_t:s0 2024-08-19T01:27:00.001 INFO:teuthology.orchestra.run.smithi083.stdout:Access: 2024-08-19 01:26:17.591094161 +0000 2024-08-19T01:27:00.001 INFO:teuthology.orchestra.run.smithi083.stdout:Modify: 2024-08-19 01:21:58.534417261 +0000 2024-08-19T01:27:00.001 INFO:teuthology.orchestra.run.smithi083.stdout:Change: 2024-08-19 01:21:58.534417261 +0000 2024-08-19T01:27:00.001 INFO:teuthology.orchestra.run.smithi083.stdout: Birth: 2024-08-19 01:21:58.534417261 +0000 2024-08-19T01:27:00.002 DEBUG:teuthology.orchestra.run.smithi083:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-08-19T01:27:00.095 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-08-19T01:27:00.095 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-08-19T01:27:00.096 INFO:teuthology.orchestra.run.smithi083.stderr:512 bytes copied, 0.000276638 s, 1.9 MB/s 2024-08-19T01:27:00.098 DEBUG:teuthology.orchestra.run.smithi083:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-08-19T01:27:00.135 DEBUG:teuthology.orchestra.run.smithi083:> stat /dev/vg_nvme/lv_3 2024-08-19T01:27:00.211 INFO:teuthology.orchestra.run.smithi083.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-08-19T01:27:00.211 INFO:teuthology.orchestra.run.smithi083.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-08-19T01:27:00.211 INFO:teuthology.orchestra.run.smithi083.stdout:Device: 6h/6d Inode: 672 Links: 1 2024-08-19T01:27:00.211 INFO:teuthology.orchestra.run.smithi083.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-08-19T01:27:00.211 INFO:teuthology.orchestra.run.smithi083.stdout:Context: system_u:object_r:device_t:s0 2024-08-19T01:27:00.211 INFO:teuthology.orchestra.run.smithi083.stdout:Access: 2024-08-19 01:26:17.651092740 +0000 2024-08-19T01:27:00.211 INFO:teuthology.orchestra.run.smithi083.stdout:Modify: 2024-08-19 01:21:59.027406116 +0000 2024-08-19T01:27:00.212 INFO:teuthology.orchestra.run.smithi083.stdout:Change: 2024-08-19 01:21:59.027406116 +0000 2024-08-19T01:27:00.212 INFO:teuthology.orchestra.run.smithi083.stdout: Birth: 2024-08-19 01:21:59.027406116 +0000 2024-08-19T01:27:00.212 DEBUG:teuthology.orchestra.run.smithi083:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-08-19T01:27:00.299 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:27:00 smithi083 ceph-mon[17120]: from='client.24103 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi083=x;smithi117=y;smithi167=z", "target": ["mon-mgr", ""]}]: dispatch 2024-08-19T01:27:00.299 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:27:00 smithi083 ceph-mon[17120]: Saving service mgr spec with placement smithi083=x;smithi117=y;smithi167=z;count:3 2024-08-19T01:27:00.299 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:27:00 smithi083 ceph-mon[17120]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' 2024-08-19T01:27:00.300 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:27:00 smithi083 ceph-mon[17120]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' 2024-08-19T01:27:00.300 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:27:00 smithi083 ceph-mon[17120]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' 2024-08-19T01:27:00.300 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:27:00 smithi083 ceph-mon[17120]: Reconfiguring mgr.x (unknown last config time)... 2024-08-19T01:27:00.300 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:27:00 smithi083 ceph-mon[17120]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-08-19T01:27:00.300 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:27:00 smithi083 ceph-mon[17120]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-08-19T01:27:00.300 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:27:00 smithi083 ceph-mon[17120]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-19T01:27:00.300 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:27:00 smithi083 ceph-mon[17120]: Reconfiguring daemon mgr.x on smithi083 2024-08-19T01:27:00.300 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:27:00 smithi083 ceph-mon[17120]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' 2024-08-19T01:27:00.305 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-08-19T01:27:00.305 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-08-19T01:27:00.305 INFO:teuthology.orchestra.run.smithi083.stderr:512 bytes copied, 0.000287073 s, 1.8 MB/s 2024-08-19T01:27:00.308 DEBUG:teuthology.orchestra.run.smithi083:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-08-19T01:27:00.345 DEBUG:teuthology.orchestra.run.smithi083:> stat /dev/vg_nvme/lv_4 2024-08-19T01:27:00.420 INFO:teuthology.orchestra.run.smithi083.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-08-19T01:27:00.420 INFO:teuthology.orchestra.run.smithi083.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-08-19T01:27:00.420 INFO:teuthology.orchestra.run.smithi083.stdout:Device: 6h/6d Inode: 688 Links: 1 2024-08-19T01:27:00.420 INFO:teuthology.orchestra.run.smithi083.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-08-19T01:27:00.421 INFO:teuthology.orchestra.run.smithi083.stdout:Context: system_u:object_r:device_t:s0 2024-08-19T01:27:00.421 INFO:teuthology.orchestra.run.smithi083.stdout:Access: 2024-08-19 01:26:17.702091532 +0000 2024-08-19T01:27:00.421 INFO:teuthology.orchestra.run.smithi083.stdout:Modify: 2024-08-19 01:21:59.464396238 +0000 2024-08-19T01:27:00.421 INFO:teuthology.orchestra.run.smithi083.stdout:Change: 2024-08-19 01:21:59.464396238 +0000 2024-08-19T01:27:00.421 INFO:teuthology.orchestra.run.smithi083.stdout: Birth: 2024-08-19 01:21:59.464396238 +0000 2024-08-19T01:27:00.421 DEBUG:teuthology.orchestra.run.smithi083:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-08-19T01:27:00.453 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:27:00 smithi117 ceph-mon[22016]: from='client.24103 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi083=x;smithi117=y;smithi167=z", "target": ["mon-mgr", ""]}]: dispatch 2024-08-19T01:27:00.453 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:27:00 smithi117 ceph-mon[22016]: Saving service mgr spec with placement smithi083=x;smithi117=y;smithi167=z;count:3 2024-08-19T01:27:00.453 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:27:00 smithi117 ceph-mon[22016]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' 2024-08-19T01:27:00.453 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:27:00 smithi117 ceph-mon[22016]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' 2024-08-19T01:27:00.453 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:27:00 smithi117 ceph-mon[22016]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' 2024-08-19T01:27:00.453 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:27:00 smithi117 ceph-mon[22016]: Reconfiguring mgr.x (unknown last config time)... 2024-08-19T01:27:00.453 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:27:00 smithi117 ceph-mon[22016]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-08-19T01:27:00.453 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:27:00 smithi117 ceph-mon[22016]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-08-19T01:27:00.454 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:27:00 smithi117 ceph-mon[22016]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-19T01:27:00.454 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:27:00 smithi117 ceph-mon[22016]: Reconfiguring daemon mgr.x on smithi083 2024-08-19T01:27:00.454 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:27:00 smithi117 ceph-mon[22016]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' 2024-08-19T01:27:00.467 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:27:00 smithi167 ceph-mon[21875]: from='client.24103 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi083=x;smithi117=y;smithi167=z", "target": ["mon-mgr", ""]}]: dispatch 2024-08-19T01:27:00.467 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:27:00 smithi167 ceph-mon[21875]: Saving service mgr spec with placement smithi083=x;smithi117=y;smithi167=z;count:3 2024-08-19T01:27:00.467 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:27:00 smithi167 ceph-mon[21875]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' 2024-08-19T01:27:00.467 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:27:00 smithi167 ceph-mon[21875]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' 2024-08-19T01:27:00.467 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:27:00 smithi167 ceph-mon[21875]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' 2024-08-19T01:27:00.467 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:27:00 smithi167 ceph-mon[21875]: Reconfiguring mgr.x (unknown last config time)... 2024-08-19T01:27:00.467 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:27:00 smithi167 ceph-mon[21875]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-08-19T01:27:00.467 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:27:00 smithi167 ceph-mon[21875]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-08-19T01:27:00.467 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:27:00 smithi167 ceph-mon[21875]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-19T01:27:00.467 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:27:00 smithi167 ceph-mon[21875]: Reconfiguring daemon mgr.x on smithi083 2024-08-19T01:27:00.467 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:27:00 smithi167 ceph-mon[21875]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' 2024-08-19T01:27:00.518 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-08-19T01:27:00.518 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-08-19T01:27:00.519 INFO:teuthology.orchestra.run.smithi083.stderr:512 bytes copied, 0.00483054 s, 106 kB/s 2024-08-19T01:27:00.521 DEBUG:teuthology.orchestra.run.smithi083:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-08-19T01:27:00.557 DEBUG:teuthology.orchestra.run.smithi117:> set -ex 2024-08-19T01:27:00.558 DEBUG:teuthology.orchestra.run.smithi117:> dd if=/scratch_devs of=/dev/stdout 2024-08-19T01:27:00.577 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-08-19T01:27:00.577 DEBUG:teuthology.orchestra.run.smithi117:> stat /dev/vg_nvme/lv_1 2024-08-19T01:27:00.636 INFO:teuthology.orchestra.run.smithi117.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-08-19T01:27:00.636 INFO:teuthology.orchestra.run.smithi117.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-08-19T01:27:00.637 INFO:teuthology.orchestra.run.smithi117.stdout:Device: 5h/5d Inode: 663 Links: 1 2024-08-19T01:27:00.637 INFO:teuthology.orchestra.run.smithi117.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-08-19T01:27:00.637 INFO:teuthology.orchestra.run.smithi117.stdout:Context: system_u:object_r:device_t:s0 2024-08-19T01:27:00.637 INFO:teuthology.orchestra.run.smithi117.stdout:Access: 2024-08-19 01:26:48.760769974 +0000 2024-08-19T01:27:00.637 INFO:teuthology.orchestra.run.smithi117.stdout:Modify: 2024-08-19 01:21:58.742708475 +0000 2024-08-19T01:27:00.637 INFO:teuthology.orchestra.run.smithi117.stdout:Change: 2024-08-19 01:21:58.743708455 +0000 2024-08-19T01:27:00.637 INFO:teuthology.orchestra.run.smithi117.stdout: Birth: 2024-08-19 01:21:58.742708475 +0000 2024-08-19T01:27:00.637 DEBUG:teuthology.orchestra.run.smithi117:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-08-19T01:27:00.704 INFO:teuthology.orchestra.run.smithi117.stderr:1+0 records in 2024-08-19T01:27:00.704 INFO:teuthology.orchestra.run.smithi117.stderr:1+0 records out 2024-08-19T01:27:00.705 INFO:teuthology.orchestra.run.smithi117.stderr:512 bytes copied, 0.000269539 s, 1.9 MB/s 2024-08-19T01:27:00.706 DEBUG:teuthology.orchestra.run.smithi117:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-08-19T01:27:00.765 DEBUG:teuthology.orchestra.run.smithi117:> stat /dev/vg_nvme/lv_2 2024-08-19T01:27:00.822 INFO:teuthology.orchestra.run.smithi117.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-08-19T01:27:00.823 INFO:teuthology.orchestra.run.smithi117.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-08-19T01:27:00.823 INFO:teuthology.orchestra.run.smithi117.stdout:Device: 5h/5d Inode: 683 Links: 1 2024-08-19T01:27:00.823 INFO:teuthology.orchestra.run.smithi117.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-08-19T01:27:00.823 INFO:teuthology.orchestra.run.smithi117.stdout:Context: system_u:object_r:device_t:s0 2024-08-19T01:27:00.823 INFO:teuthology.orchestra.run.smithi117.stdout:Access: 2024-08-19 01:26:48.760769974 +0000 2024-08-19T01:27:00.823 INFO:teuthology.orchestra.run.smithi117.stdout:Modify: 2024-08-19 01:21:59.079701797 +0000 2024-08-19T01:27:00.823 INFO:teuthology.orchestra.run.smithi117.stdout:Change: 2024-08-19 01:21:59.079701797 +0000 2024-08-19T01:27:00.823 INFO:teuthology.orchestra.run.smithi117.stdout: Birth: 2024-08-19 01:21:59.079701797 +0000 2024-08-19T01:27:00.823 DEBUG:teuthology.orchestra.run.smithi117:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-08-19T01:27:00.888 INFO:teuthology.orchestra.run.smithi117.stderr:1+0 records in 2024-08-19T01:27:00.888 INFO:teuthology.orchestra.run.smithi117.stderr:1+0 records out 2024-08-19T01:27:00.888 INFO:teuthology.orchestra.run.smithi117.stderr:512 bytes copied, 0.000204752 s, 2.5 MB/s 2024-08-19T01:27:00.889 DEBUG:teuthology.orchestra.run.smithi117:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-08-19T01:27:00.947 DEBUG:teuthology.orchestra.run.smithi117:> stat /dev/vg_nvme/lv_3 2024-08-19T01:27:01.003 INFO:teuthology.orchestra.run.smithi117.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-08-19T01:27:01.003 INFO:teuthology.orchestra.run.smithi117.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-08-19T01:27:01.004 INFO:teuthology.orchestra.run.smithi117.stdout:Device: 5h/5d Inode: 701 Links: 1 2024-08-19T01:27:01.004 INFO:teuthology.orchestra.run.smithi117.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-08-19T01:27:01.004 INFO:teuthology.orchestra.run.smithi117.stdout:Context: system_u:object_r:device_t:s0 2024-08-19T01:27:01.004 INFO:teuthology.orchestra.run.smithi117.stdout:Access: 2024-08-19 01:26:48.761769955 +0000 2024-08-19T01:27:01.004 INFO:teuthology.orchestra.run.smithi117.stdout:Modify: 2024-08-19 01:21:59.397695495 +0000 2024-08-19T01:27:01.004 INFO:teuthology.orchestra.run.smithi117.stdout:Change: 2024-08-19 01:21:59.397695495 +0000 2024-08-19T01:27:01.004 INFO:teuthology.orchestra.run.smithi117.stdout: Birth: 2024-08-19 01:21:59.397695495 +0000 2024-08-19T01:27:01.004 DEBUG:teuthology.orchestra.run.smithi117:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-08-19T01:27:01.068 INFO:teuthology.orchestra.run.smithi117.stderr:1+0 records in 2024-08-19T01:27:01.068 INFO:teuthology.orchestra.run.smithi117.stderr:1+0 records out 2024-08-19T01:27:01.068 INFO:teuthology.orchestra.run.smithi117.stderr:512 bytes copied, 0.000171705 s, 3.0 MB/s 2024-08-19T01:27:01.069 DEBUG:teuthology.orchestra.run.smithi117:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-08-19T01:27:01.126 DEBUG:teuthology.orchestra.run.smithi117:> stat /dev/vg_nvme/lv_4 2024-08-19T01:27:01.182 INFO:teuthology.orchestra.run.smithi117.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-08-19T01:27:01.182 INFO:teuthology.orchestra.run.smithi117.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-08-19T01:27:01.182 INFO:teuthology.orchestra.run.smithi117.stdout:Device: 5h/5d Inode: 720 Links: 1 2024-08-19T01:27:01.182 INFO:teuthology.orchestra.run.smithi117.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-08-19T01:27:01.182 INFO:teuthology.orchestra.run.smithi117.stdout:Context: system_u:object_r:device_t:s0 2024-08-19T01:27:01.182 INFO:teuthology.orchestra.run.smithi117.stdout:Access: 2024-08-19 01:26:48.761769955 +0000 2024-08-19T01:27:01.182 INFO:teuthology.orchestra.run.smithi117.stdout:Modify: 2024-08-19 01:21:59.714689214 +0000 2024-08-19T01:27:01.182 INFO:teuthology.orchestra.run.smithi117.stdout:Change: 2024-08-19 01:21:59.714689214 +0000 2024-08-19T01:27:01.182 INFO:teuthology.orchestra.run.smithi117.stdout: Birth: 2024-08-19 01:21:59.714689214 +0000 2024-08-19T01:27:01.183 DEBUG:teuthology.orchestra.run.smithi117:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-08-19T01:27:01.245 INFO:teuthology.orchestra.run.smithi117.stderr:1+0 records in 2024-08-19T01:27:01.245 INFO:teuthology.orchestra.run.smithi117.stderr:1+0 records out 2024-08-19T01:27:01.245 INFO:teuthology.orchestra.run.smithi117.stderr:512 bytes copied, 0.000190721 s, 2.7 MB/s 2024-08-19T01:27:01.246 DEBUG:teuthology.orchestra.run.smithi117:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-08-19T01:27:01.302 DEBUG:teuthology.orchestra.run.smithi167:> set -ex 2024-08-19T01:27:01.303 DEBUG:teuthology.orchestra.run.smithi167:> dd if=/scratch_devs of=/dev/stdout 2024-08-19T01:27:01.322 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-08-19T01:27:01.323 DEBUG:teuthology.orchestra.run.smithi167:> stat /dev/vg_nvme/lv_1 2024-08-19T01:27:01.381 INFO:teuthology.orchestra.run.smithi167.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-08-19T01:27:01.381 INFO:teuthology.orchestra.run.smithi167.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-08-19T01:27:01.381 INFO:teuthology.orchestra.run.smithi167.stdout:Device: 5h/5d Inode: 666 Links: 1 2024-08-19T01:27:01.381 INFO:teuthology.orchestra.run.smithi167.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-08-19T01:27:01.381 INFO:teuthology.orchestra.run.smithi167.stdout:Context: system_u:object_r:device_t:s0 2024-08-19T01:27:01.381 INFO:teuthology.orchestra.run.smithi167.stdout:Access: 2024-08-19 01:26:39.705029306 +0000 2024-08-19T01:27:01.381 INFO:teuthology.orchestra.run.smithi167.stdout:Modify: 2024-08-19 01:21:58.611099869 +0000 2024-08-19T01:27:01.381 INFO:teuthology.orchestra.run.smithi167.stdout:Change: 2024-08-19 01:21:58.611099869 +0000 2024-08-19T01:27:01.382 INFO:teuthology.orchestra.run.smithi167.stdout: Birth: 2024-08-19 01:21:58.611099869 +0000 2024-08-19T01:27:01.382 DEBUG:teuthology.orchestra.run.smithi167:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-08-19T01:27:01.405 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:27:01 smithi083 ceph-mon[17120]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-19T01:27:01.405 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:27:01 smithi083 ceph-mon[17120]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' 2024-08-19T01:27:01.405 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:27:01 smithi083 ceph-mon[17120]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' 2024-08-19T01:27:01.405 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:27:01 smithi083 ceph-mon[17120]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-08-19T01:27:01.405 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:27:01 smithi083 ceph-mon[17120]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-08-19T01:27:01.405 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:27:01 smithi083 ceph-mon[17120]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-19T01:27:01.420 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:27:01 smithi117 ceph-mon[22016]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-19T01:27:01.421 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:27:01 smithi117 ceph-mon[22016]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' 2024-08-19T01:27:01.421 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:27:01 smithi117 ceph-mon[22016]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' 2024-08-19T01:27:01.421 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:27:01 smithi117 ceph-mon[22016]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-08-19T01:27:01.421 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:27:01 smithi117 ceph-mon[22016]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-08-19T01:27:01.421 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:27:01 smithi117 ceph-mon[22016]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-19T01:27:01.445 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:27:01 smithi167 ceph-mon[21875]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-19T01:27:01.445 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:27:01 smithi167 ceph-mon[21875]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' 2024-08-19T01:27:01.445 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:27:01 smithi167 ceph-mon[21875]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' 2024-08-19T01:27:01.445 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:27:01 smithi167 ceph-mon[21875]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-08-19T01:27:01.446 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:27:01 smithi167 ceph-mon[21875]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-08-19T01:27:01.446 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:27:01 smithi167 ceph-mon[21875]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-19T01:27:01.448 INFO:teuthology.orchestra.run.smithi167.stderr:1+0 records in 2024-08-19T01:27:01.448 INFO:teuthology.orchestra.run.smithi167.stderr:1+0 records out 2024-08-19T01:27:01.448 INFO:teuthology.orchestra.run.smithi167.stderr:512 bytes copied, 0.000268972 s, 1.9 MB/s 2024-08-19T01:27:01.450 DEBUG:teuthology.orchestra.run.smithi167:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-08-19T01:27:01.508 DEBUG:teuthology.orchestra.run.smithi167:> stat /dev/vg_nvme/lv_2 2024-08-19T01:27:01.564 INFO:teuthology.orchestra.run.smithi167.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-08-19T01:27:01.565 INFO:teuthology.orchestra.run.smithi167.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-08-19T01:27:01.565 INFO:teuthology.orchestra.run.smithi167.stdout:Device: 5h/5d Inode: 682 Links: 1 2024-08-19T01:27:01.565 INFO:teuthology.orchestra.run.smithi167.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-08-19T01:27:01.565 INFO:teuthology.orchestra.run.smithi167.stdout:Context: system_u:object_r:device_t:s0 2024-08-19T01:27:01.565 INFO:teuthology.orchestra.run.smithi167.stdout:Access: 2024-08-19 01:26:39.706029289 +0000 2024-08-19T01:27:01.565 INFO:teuthology.orchestra.run.smithi167.stdout:Modify: 2024-08-19 01:21:58.932094222 +0000 2024-08-19T01:27:01.565 INFO:teuthology.orchestra.run.smithi167.stdout:Change: 2024-08-19 01:21:58.932094222 +0000 2024-08-19T01:27:01.565 INFO:teuthology.orchestra.run.smithi167.stdout: Birth: 2024-08-19 01:21:58.932094222 +0000 2024-08-19T01:27:01.566 DEBUG:teuthology.orchestra.run.smithi167:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-08-19T01:27:01.630 INFO:teuthology.orchestra.run.smithi167.stderr:1+0 records in 2024-08-19T01:27:01.631 INFO:teuthology.orchestra.run.smithi167.stderr:1+0 records out 2024-08-19T01:27:01.631 INFO:teuthology.orchestra.run.smithi167.stderr:512 bytes copied, 0.000238126 s, 2.2 MB/s 2024-08-19T01:27:01.632 DEBUG:teuthology.orchestra.run.smithi167:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-08-19T01:27:01.689 DEBUG:teuthology.orchestra.run.smithi167:> stat /dev/vg_nvme/lv_3 2024-08-19T01:27:01.744 INFO:teuthology.orchestra.run.smithi167.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-08-19T01:27:01.745 INFO:teuthology.orchestra.run.smithi167.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-08-19T01:27:01.745 INFO:teuthology.orchestra.run.smithi167.stdout:Device: 5h/5d Inode: 702 Links: 1 2024-08-19T01:27:01.745 INFO:teuthology.orchestra.run.smithi167.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-08-19T01:27:01.745 INFO:teuthology.orchestra.run.smithi167.stdout:Context: system_u:object_r:device_t:s0 2024-08-19T01:27:01.745 INFO:teuthology.orchestra.run.smithi167.stdout:Access: 2024-08-19 01:26:39.706029289 +0000 2024-08-19T01:27:01.745 INFO:teuthology.orchestra.run.smithi167.stdout:Modify: 2024-08-19 01:21:59.266088346 +0000 2024-08-19T01:27:01.745 INFO:teuthology.orchestra.run.smithi167.stdout:Change: 2024-08-19 01:21:59.266088346 +0000 2024-08-19T01:27:01.745 INFO:teuthology.orchestra.run.smithi167.stdout: Birth: 2024-08-19 01:21:59.266088346 +0000 2024-08-19T01:27:01.745 DEBUG:teuthology.orchestra.run.smithi167:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-08-19T01:27:01.809 INFO:teuthology.orchestra.run.smithi167.stderr:1+0 records in 2024-08-19T01:27:01.809 INFO:teuthology.orchestra.run.smithi167.stderr:1+0 records out 2024-08-19T01:27:01.809 INFO:teuthology.orchestra.run.smithi167.stderr:512 bytes copied, 0.000188599 s, 2.7 MB/s 2024-08-19T01:27:01.811 DEBUG:teuthology.orchestra.run.smithi167:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-08-19T01:27:01.867 DEBUG:teuthology.orchestra.run.smithi167:> stat /dev/vg_nvme/lv_4 2024-08-19T01:27:01.923 INFO:teuthology.orchestra.run.smithi167.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-08-19T01:27:01.923 INFO:teuthology.orchestra.run.smithi167.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-08-19T01:27:01.923 INFO:teuthology.orchestra.run.smithi167.stdout:Device: 5h/5d Inode: 723 Links: 1 2024-08-19T01:27:01.923 INFO:teuthology.orchestra.run.smithi167.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-08-19T01:27:01.923 INFO:teuthology.orchestra.run.smithi167.stdout:Context: system_u:object_r:device_t:s0 2024-08-19T01:27:01.923 INFO:teuthology.orchestra.run.smithi167.stdout:Access: 2024-08-19 01:26:39.706029289 +0000 2024-08-19T01:27:01.923 INFO:teuthology.orchestra.run.smithi167.stdout:Modify: 2024-08-19 01:21:59.595082558 +0000 2024-08-19T01:27:01.923 INFO:teuthology.orchestra.run.smithi167.stdout:Change: 2024-08-19 01:21:59.595082558 +0000 2024-08-19T01:27:01.923 INFO:teuthology.orchestra.run.smithi167.stdout: Birth: 2024-08-19 01:21:59.595082558 +0000 2024-08-19T01:27:01.924 DEBUG:teuthology.orchestra.run.smithi167:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-08-19T01:27:01.986 INFO:teuthology.orchestra.run.smithi167.stderr:1+0 records in 2024-08-19T01:27:01.987 INFO:teuthology.orchestra.run.smithi167.stderr:1+0 records out 2024-08-19T01:27:01.987 INFO:teuthology.orchestra.run.smithi167.stderr:512 bytes copied, 0.000201604 s, 2.5 MB/s 2024-08-19T01:27:01.988 DEBUG:teuthology.orchestra.run.smithi167:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-08-19T01:27:02.045 INFO:tasks.cephadm:Deploying osd.0 on smithi083 with /dev/vg_nvme/lv_4... 2024-08-19T01:27:02.045 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833 ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid bad8780e-5dc9-11ef-bccf-c7b262605968 -- lvm zap /dev/vg_nvme/lv_4 2024-08-19T01:27:02.405 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:27:02 smithi083 ceph-mon[17120]: Reconfiguring mon.b (monmap changed)... 2024-08-19T01:27:02.405 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:27:02 smithi083 ceph-mon[17120]: Reconfiguring daemon mon.b on smithi117 2024-08-19T01:27:02.423 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/bad8780e-5dc9-11ef-bccf-c7b262605968/mon.a/config 2024-08-19T01:27:02.424 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:27:02 smithi117 ceph-mon[22016]: Reconfiguring mon.b (monmap changed)... 2024-08-19T01:27:02.424 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:27:02 smithi117 ceph-mon[22016]: Reconfiguring daemon mon.b on smithi117 2024-08-19T01:27:02.466 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:27:02 smithi167 ceph-mon[21875]: Reconfiguring mon.b (monmap changed)... 2024-08-19T01:27:02.466 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:27:02 smithi167 ceph-mon[21875]: Reconfiguring daemon mon.b on smithi117 2024-08-19T01:27:03.567 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:27:03 smithi083 ceph-mon[17120]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' 2024-08-19T01:27:03.567 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:27:03 smithi083 ceph-mon[17120]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' 2024-08-19T01:27:03.567 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:27:03 smithi083 ceph-mon[17120]: Reconfiguring mon.c (monmap changed)... 2024-08-19T01:27:03.567 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:27:03 smithi083 ceph-mon[17120]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-08-19T01:27:03.567 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:27:03 smithi083 ceph-mon[17120]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-08-19T01:27:03.567 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:27:03 smithi083 ceph-mon[17120]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-19T01:27:03.567 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:27:03 smithi083 ceph-mon[17120]: Reconfiguring daemon mon.c on smithi167 2024-08-19T01:27:03.567 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:27:03 smithi083 ceph-mon[17120]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-19T01:27:03.587 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:27:03 smithi167 ceph-mon[21875]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' 2024-08-19T01:27:03.587 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:27:03 smithi167 ceph-mon[21875]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' 2024-08-19T01:27:03.587 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:27:03 smithi167 ceph-mon[21875]: Reconfiguring mon.c (monmap changed)... 2024-08-19T01:27:03.587 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:27:03 smithi167 ceph-mon[21875]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-08-19T01:27:03.588 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:27:03 smithi167 ceph-mon[21875]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-08-19T01:27:03.588 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:27:03 smithi167 ceph-mon[21875]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-19T01:27:03.588 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:27:03 smithi167 ceph-mon[21875]: Reconfiguring daemon mon.c on smithi167 2024-08-19T01:27:03.588 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:27:03 smithi167 ceph-mon[21875]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-19T01:27:03.703 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:27:03 smithi117 ceph-mon[22016]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' 2024-08-19T01:27:03.703 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:27:03 smithi117 ceph-mon[22016]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' 2024-08-19T01:27:03.703 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:27:03 smithi117 ceph-mon[22016]: Reconfiguring mon.c (monmap changed)... 2024-08-19T01:27:03.703 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:27:03 smithi117 ceph-mon[22016]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-08-19T01:27:03.703 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:27:03 smithi117 ceph-mon[22016]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-08-19T01:27:03.703 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:27:03 smithi117 ceph-mon[22016]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-19T01:27:03.703 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:27:03 smithi117 ceph-mon[22016]: Reconfiguring daemon mon.c on smithi167 2024-08-19T01:27:03.703 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:27:03 smithi117 ceph-mon[22016]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-19T01:27:04.905 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:27:04 smithi083 ceph-mon[17120]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' 2024-08-19T01:27:04.905 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:27:04 smithi083 ceph-mon[17120]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' 2024-08-19T01:27:04.905 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:27:04 smithi083 ceph-mon[17120]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-19T01:27:04.905 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:27:04 smithi083 ceph-mon[17120]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-19T01:27:04.906 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:27:04 smithi083 ceph-mon[17120]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-19T01:27:04.906 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:27:04 smithi083 ceph-mon[17120]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' 2024-08-19T01:27:04.906 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:27:04 smithi083 ceph-mon[17120]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-08-19T01:27:04.906 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:27:04 smithi083 ceph-mon[17120]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-08-19T01:27:04.906 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:27:04 smithi083 ceph-mon[17120]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-08-19T01:27:04.906 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:27:04 smithi083 ceph-mon[17120]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-19T01:27:04.906 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:27:04 smithi083 ceph-mon[17120]: Deploying daemon mgr.z on smithi167 2024-08-19T01:27:04.906 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:27:04 smithi083 ceph-mon[17120]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-19T01:27:04.935 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:27:04 smithi167 ceph-mon[21875]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' 2024-08-19T01:27:04.935 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:27:04 smithi167 ceph-mon[21875]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' 2024-08-19T01:27:04.935 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:27:04 smithi167 ceph-mon[21875]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-19T01:27:04.935 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:27:04 smithi167 ceph-mon[21875]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-19T01:27:04.935 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:27:04 smithi167 ceph-mon[21875]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-19T01:27:04.935 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:27:04 smithi167 ceph-mon[21875]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' 2024-08-19T01:27:04.935 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:27:04 smithi167 ceph-mon[21875]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-08-19T01:27:04.935 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:27:04 smithi167 ceph-mon[21875]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-08-19T01:27:04.935 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:27:04 smithi167 ceph-mon[21875]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-08-19T01:27:04.936 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:27:04 smithi167 ceph-mon[21875]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-19T01:27:04.936 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:27:04 smithi167 ceph-mon[21875]: Deploying daemon mgr.z on smithi167 2024-08-19T01:27:04.936 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:27:04 smithi167 ceph-mon[21875]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-19T01:27:04.953 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:27:04 smithi117 ceph-mon[22016]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' 2024-08-19T01:27:04.953 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:27:04 smithi117 ceph-mon[22016]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' 2024-08-19T01:27:04.953 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:27:04 smithi117 ceph-mon[22016]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-19T01:27:04.953 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:27:04 smithi117 ceph-mon[22016]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-19T01:27:04.953 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:27:04 smithi117 ceph-mon[22016]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-19T01:27:04.953 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:27:04 smithi117 ceph-mon[22016]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' 2024-08-19T01:27:04.953 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:27:04 smithi117 ceph-mon[22016]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-08-19T01:27:04.953 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:27:04 smithi117 ceph-mon[22016]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-08-19T01:27:04.953 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:27:04 smithi117 ceph-mon[22016]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-08-19T01:27:04.954 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:27:04 smithi117 ceph-mon[22016]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-19T01:27:04.954 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:27:04 smithi117 ceph-mon[22016]: Deploying daemon mgr.z on smithi167 2024-08-19T01:27:04.954 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:27:04 smithi117 ceph-mon[22016]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-19T01:27:05.321 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-08-19T01:27:05.353 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid bad8780e-5dc9-11ef-bccf-c7b262605968 -- ceph orch daemon add osd smithi083:vg_nvme/lv_4 2024-08-19T01:27:05.554 INFO:journalctl@ceph.mgr.z.smithi167.stdout:Aug 19 01:27:05 smithi167 podman[23678]: 2024-08-19 01:27:05.553057673 +0000 UTC m=+0.151555653 container create e7079f51b08daa4e8ee256732662fb1a4d1b90173a6a67f33ef8a6315c1b26a7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c9dd28080a631be25c77c36c63b17a3ef340ffb2e21c87d4d7b56b6398436d9, name=ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-mgr-z, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.build-date=20240731, GIT_CLEAN=True, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.36.0, RELEASE=squid-de09dd0, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0) 2024-08-19T01:27:05.718 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/bad8780e-5dc9-11ef-bccf-c7b262605968/mon.a/config 2024-08-19T01:27:05.809 INFO:journalctl@ceph.mgr.z.smithi167.stdout:Aug 19 01:27:05 smithi167 podman[23678]: 2024-08-19 01:27:05.780364591 +0000 UTC m=+0.378862540 container init e7079f51b08daa4e8ee256732662fb1a4d1b90173a6a67f33ef8a6315c1b26a7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c9dd28080a631be25c77c36c63b17a3ef340ffb2e21c87d4d7b56b6398436d9, name=ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-mgr-z, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, io.buildah.version=1.36.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-de09dd0, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=) 2024-08-19T01:27:05.810 INFO:journalctl@ceph.mgr.z.smithi167.stdout:Aug 19 01:27:05 smithi167 podman[23678]: 2024-08-19 01:27:05.783409352 +0000 UTC m=+0.381907290 container start e7079f51b08daa4e8ee256732662fb1a4d1b90173a6a67f33ef8a6315c1b26a7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c9dd28080a631be25c77c36c63b17a3ef340ffb2e21c87d4d7b56b6398436d9, name=ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-mgr-z, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, RELEASE=squid-de09dd0, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, io.buildah.version=1.36.0, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, ceph=True, GIT_CLEAN=True, org.label-schema.license=GPLv2) 2024-08-19T01:27:06.080 INFO:journalctl@ceph.mgr.z.smithi167.stdout:Aug 19 01:27:05 smithi167 ceph-mgr[23697]: -- 172.21.15.167:0/317804669 <== mon.1 v2:172.21.15.167:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x557dd6e830e0 con 0x557dd6e61400 2024-08-19T01:27:06.080 INFO:journalctl@ceph.mgr.z.smithi167.stdout:Aug 19 01:27:05 smithi167 bash[23678]: e7079f51b08daa4e8ee256732662fb1a4d1b90173a6a67f33ef8a6315c1b26a7 2024-08-19T01:27:06.080 INFO:journalctl@ceph.mgr.z.smithi167.stdout:Aug 19 01:27:05 smithi167 systemd[1]: Started Ceph mgr.z for bad8780e-5dc9-11ef-bccf-c7b262605968. 2024-08-19T01:27:06.080 INFO:journalctl@ceph.mgr.z.smithi167.stdout:Aug 19 01:27:06 smithi167 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-mgr-z[23693]: 2024-08-19T01:27:06.012+0000 7f9803dca040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-08-19T01:27:06.080 INFO:journalctl@ceph.mgr.z.smithi167.stdout:Aug 19 01:27:06 smithi167 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-mgr-z[23693]: 2024-08-19T01:27:06.078+0000 7f9803dca040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-08-19T01:27:06.313 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:27:06.310+0000 7f4fafa00640 1 -- 172.21.15.83:0/2446790290 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4fa8104d30 msgr2=0x7f4fa81051b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:27:06.313 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:27:06.310+0000 7f4fafa00640 1 --2- 172.21.15.83:0/2446790290 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4fa8104d30 0x7f4fa81051b0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f4f90009900 tx=0x7f4f9002a410 comp rx=0 tx=0).stop 2024-08-19T01:27:06.313 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:27:06.310+0000 7f4fafa00640 1 -- 172.21.15.83:0/2446790290 shutdown_connections 2024-08-19T01:27:06.313 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:27:06.310+0000 7f4fafa00640 1 --2- 172.21.15.83:0/2446790290 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4fa81056f0 0x7f4fa810bf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:27:06.313 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:27:06.310+0000 7f4fafa00640 1 --2- 172.21.15.83:0/2446790290 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4fa8104d30 0x7f4fa81051b0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:27:06.314 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:27:06.310+0000 7f4fafa00640 1 --2- 172.21.15.83:0/2446790290 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4fa8103b30 0x7f4fa8103f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:27:06.314 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:27:06.310+0000 7f4fafa00640 1 -- 172.21.15.83:0/2446790290 >> 172.21.15.83:0/2446790290 conn(0x7f4fa80ff2e0 msgr2=0x7f4fa8101700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:27:06.314 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:27:06.310+0000 7f4fafa00640 1 -- 172.21.15.83:0/2446790290 shutdown_connections 2024-08-19T01:27:06.314 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:27:06.310+0000 7f4fafa00640 1 -- 172.21.15.83:0/2446790290 wait complete. 2024-08-19T01:27:06.315 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:27:06.312+0000 7f4fafa00640 1 Processor -- start 2024-08-19T01:27:06.315 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:27:06.312+0000 7f4fafa00640 1 -- start start 2024-08-19T01:27:06.316 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:27:06.312+0000 7f4fafa00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4fa8103b30 0x7f4fa8199e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:27:06.316 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:27:06.312+0000 7f4fafa00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4fa8104d30 0x7f4fa819a360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:27:06.316 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:27:06.312+0000 7f4fafa00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4fa81056f0 0x7f4fa81a13e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:27:06.316 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:27:06.312+0000 7f4fafa00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f4fa810f7b0 con 0x7f4fa8103b30 2024-08-19T01:27:06.316 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:27:06.312+0000 7f4fafa00640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f4fa810f630 con 0x7f4fa8104d30 2024-08-19T01:27:06.316 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:27:06.312+0000 7f4fafa00640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f4fa810f930 con 0x7f4fa81056f0 2024-08-19T01:27:06.316 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:27:06.312+0000 7f4fad000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4fa8103b30 0x7f4fa8199e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:27:06.316 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:27:06.312+0000 7f4fad000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4fa8103b30 0x7f4fa8199e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:53582/0 (socket says 172.21.15.83:53582) 2024-08-19T01:27:06.316 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:27:06.312+0000 7f4f9fe00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4fa8104d30 0x7f4fa819a360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:27:06.316 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:27:06.312+0000 7f4fad000640 1 -- 172.21.15.83:0/1949739374 learned_addr learned my addr 172.21.15.83:0/1949739374 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-08-19T01:27:06.316 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:27:06.313+0000 7f4fada00640 1 --2- 172.21.15.83:0/1949739374 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4fa81056f0 0x7f4fa81a13e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:27:06.316 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:27:06.313+0000 7f4f9fe00640 1 -- 172.21.15.83:0/1949739374 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4fa81056f0 msgr2=0x7f4fa81a13e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:27:06.316 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:27:06.313+0000 7f4f9fe00640 1 --2- 172.21.15.83:0/1949739374 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4fa81056f0 0x7f4fa81a13e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:27:06.317 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:27:06.313+0000 7f4f9fe00640 1 -- 172.21.15.83:0/1949739374 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4fa8103b30 msgr2=0x7f4fa8199e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:27:06.317 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:27:06.313+0000 7f4f9fe00640 1 --2- 172.21.15.83:0/1949739374 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4fa8103b30 0x7f4fa8199e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:27:06.317 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:27:06.313+0000 7f4f9fe00640 1 -- 172.21.15.83:0/1949739374 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4fa81a1a50 con 0x7f4fa8104d30 2024-08-19T01:27:06.317 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:27:06.313+0000 7f4f9fe00640 1 --2- 172.21.15.83:0/1949739374 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4fa8104d30 0x7f4fa819a360 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f4f90031440 tx=0x7f4f90004270 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:27:06.317 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:27:06.313+0000 7f4fa6000640 1 -- 172.21.15.83:0/1949739374 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f9003f070 con 0x7f4fa8104d30 2024-08-19T01:27:06.317 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:27:06.313+0000 7f4fafa00640 1 -- 172.21.15.83:0/1949739374 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4fa81a1ce0 con 0x7f4fa8104d30 2024-08-19T01:27:06.317 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:27:06.313+0000 7f4fa6000640 1 -- 172.21.15.83:0/1949739374 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4f90003cd0 con 0x7f4fa8104d30 2024-08-19T01:27:06.317 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:27:06.313+0000 7f4fa6000640 1 -- 172.21.15.83:0/1949739374 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f90035070 con 0x7f4fa8104d30 2024-08-19T01:27:06.317 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:27:06.314+0000 7f4fafa00640 1 -- 172.21.15.83:0/1949739374 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4fa81a2190 con 0x7f4fa8104d30 2024-08-19T01:27:06.318 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:27:06.315+0000 7f4fa6000640 1 -- 172.21.15.83:0/1949739374 <== mon.2 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 13) ==== 46343+0+0 (secure 0 0 0) 0x7f4f900037f0 con 0x7f4fa8104d30 2024-08-19T01:27:06.318 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:27:06.315+0000 7f4fafa00640 1 -- 172.21.15.83:0/1949739374 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4fa810af30 con 0x7f4fa8104d30 2024-08-19T01:27:06.318 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:27:06.315+0000 7f4fa6000640 1 --2- 172.21.15.83:0/1949739374 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f4f8c0392c0 0x7f4f8c03b780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:27:06.318 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:27:06.315+0000 7f4fa6000640 1 -- 172.21.15.83:0/1949739374 <== mon.2 v2:172.21.15.117:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f4f9006cc10 con 0x7f4fa8104d30 2024-08-19T01:27:06.318 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:27:06.315+0000 7f4fad000640 1 --2- 172.21.15.83:0/1949739374 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f4f8c0392c0 0x7f4f8c03b780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:27:06.318 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:27:06.316+0000 7f4fad000640 1 --2- 172.21.15.83:0/1949739374 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f4f8c0392c0 0x7f4f8c03b780 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f4f980058c0 tx=0x7f4f98005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:27:06.322 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:27:06.320+0000 7f4fa6000640 1 -- 172.21.15.83:0/1949739374 <== mon.2 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7f4f90036260 con 0x7f4fa8104d30 2024-08-19T01:27:06.382 INFO:journalctl@ceph.mgr.z.smithi167.stdout:Aug 19 01:27:06 smithi167 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-mgr-z[23693]: 2024-08-19T01:27:06.220+0000 7f9803dca040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-08-19T01:27:06.383 INFO:journalctl@ceph.mgr.z.smithi167.stdout:Aug 19 01:27:06 smithi167 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-mgr-z[23693]: 2024-08-19T01:27:06.381+0000 7f9803dca040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-08-19T01:27:06.505 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:27:06.503+0000 7f4fafa00640 1 -- 172.21.15.83:0/1949739374 --> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi083:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7f4fa8062360 con 0x7f4f8c0392c0 2024-08-19T01:27:06.646 INFO:journalctl@ceph.mgr.z.smithi167.stdout:Aug 19 01:27:06 smithi167 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-mgr-z[23693]: 2024-08-19T01:27:06.522+0000 7f9803dca040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-08-19T01:27:06.646 INFO:journalctl@ceph.mgr.z.smithi167.stdout:Aug 19 01:27:06 smithi167 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-mgr-z[23693]: 2024-08-19T01:27:06.644+0000 7f9803dca040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-08-19T01:27:06.933 INFO:journalctl@ceph.mgr.z.smithi167.stdout:Aug 19 01:27:06 smithi167 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-mgr-z[23693]: 2024-08-19T01:27:06.767+0000 7f9803dca040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-08-19T01:27:06.933 INFO:journalctl@ceph.mgr.z.smithi167.stdout:Aug 19 01:27:06 smithi167 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-mgr-z[23693]: 2024-08-19T01:27:06.823+0000 7f9803dca040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-08-19T01:27:07.188 INFO:journalctl@ceph.mgr.z.smithi167.stdout:Aug 19 01:27:07 smithi167 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-mgr-z[23693]: 2024-08-19T01:27:07.125+0000 7f9803dca040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-08-19T01:27:07.188 INFO:journalctl@ceph.mgr.z.smithi167.stdout:Aug 19 01:27:07 smithi167 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-mgr-z[23693]: 2024-08-19T01:27:07.187+0000 7f9803dca040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-08-19T01:27:07.189 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:27:06 smithi167 ceph-mon[21875]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' 2024-08-19T01:27:07.189 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:27:06 smithi167 ceph-mon[21875]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' 2024-08-19T01:27:07.189 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:27:06 smithi167 ceph-mon[21875]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' 2024-08-19T01:27:07.189 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:27:06 smithi167 ceph-mon[21875]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-08-19T01:27:07.189 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:27:06 smithi167 ceph-mon[21875]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-08-19T01:27:07.189 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:27:06 smithi167 ceph-mon[21875]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-08-19T01:27:07.189 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:27:06 smithi167 ceph-mon[21875]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-19T01:27:07.189 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:27:06 smithi167 ceph-mon[21875]: Deploying daemon mgr.y on smithi117 2024-08-19T01:27:07.189 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:27:06 smithi167 ceph-mon[21875]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-19T01:27:07.190 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:27:06 smithi167 ceph-mon[21875]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-08-19T01:27:07.190 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:27:06 smithi167 ceph-mon[21875]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-08-19T01:27:07.190 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:27:06 smithi167 ceph-mon[21875]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-19T01:27:07.196 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:27:06 smithi117 ceph-mon[22016]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' 2024-08-19T01:27:07.196 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:27:06 smithi117 ceph-mon[22016]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' 2024-08-19T01:27:07.196 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:27:06 smithi117 ceph-mon[22016]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' 2024-08-19T01:27:07.196 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:27:06 smithi117 ceph-mon[22016]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-08-19T01:27:07.196 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:27:06 smithi117 ceph-mon[22016]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-08-19T01:27:07.196 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:27:06 smithi117 ceph-mon[22016]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-08-19T01:27:07.196 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:27:06 smithi117 ceph-mon[22016]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-19T01:27:07.196 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:27:06 smithi117 ceph-mon[22016]: Deploying daemon mgr.y on smithi117 2024-08-19T01:27:07.196 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:27:06 smithi117 ceph-mon[22016]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-19T01:27:07.196 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:27:06 smithi117 ceph-mon[22016]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-08-19T01:27:07.196 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:27:06 smithi117 ceph-mon[22016]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-08-19T01:27:07.197 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:27:06 smithi117 ceph-mon[22016]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-19T01:27:07.211 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:27:06 smithi083 ceph-mon[17120]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' 2024-08-19T01:27:07.211 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:27:06 smithi083 ceph-mon[17120]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' 2024-08-19T01:27:07.211 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:27:06 smithi083 ceph-mon[17120]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' 2024-08-19T01:27:07.211 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:27:06 smithi083 ceph-mon[17120]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-08-19T01:27:07.212 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:27:06 smithi083 ceph-mon[17120]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-08-19T01:27:07.212 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:27:06 smithi083 ceph-mon[17120]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-08-19T01:27:07.212 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:27:06 smithi083 ceph-mon[17120]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-19T01:27:07.212 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:27:06 smithi083 ceph-mon[17120]: Deploying daemon mgr.y on smithi117 2024-08-19T01:27:07.212 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:27:06 smithi083 ceph-mon[17120]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-19T01:27:07.212 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:27:06 smithi083 ceph-mon[17120]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-08-19T01:27:07.212 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:27:06 smithi083 ceph-mon[17120]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-08-19T01:27:07.212 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:27:06 smithi083 ceph-mon[17120]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-19T01:27:07.457 INFO:journalctl@ceph.mgr.z.smithi167.stdout:Aug 19 01:27:07 smithi167 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-mgr-z[23693]: 2024-08-19T01:27:07.253+0000 7f9803dca040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-08-19T01:27:07.458 INFO:journalctl@ceph.mgr.z.smithi167.stdout:Aug 19 01:27:07 smithi167 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-mgr-z[23693]: 2024-08-19T01:27:07.398+0000 7f9803dca040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-08-19T01:27:07.458 INFO:journalctl@ceph.mgr.z.smithi167.stdout:Aug 19 01:27:07 smithi167 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-mgr-z[23693]: 2024-08-19T01:27:07.456+0000 7f9803dca040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-08-19T01:27:07.716 INFO:journalctl@ceph.mgr.z.smithi167.stdout:Aug 19 01:27:07 smithi167 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-mgr-z[23693]: 2024-08-19T01:27:07.521+0000 7f9803dca040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-08-19T01:27:07.716 INFO:journalctl@ceph.mgr.z.smithi167.stdout:Aug 19 01:27:07 smithi167 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-mgr-z[23693]: 2024-08-19T01:27:07.702+0000 7f9803dca040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-08-19T01:27:08.105 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:27:07 smithi167 ceph-mon[21875]: from='client.24116 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi083:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-08-19T01:27:08.105 INFO:journalctl@ceph.mgr.z.smithi167.stdout:Aug 19 01:27:07 smithi167 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-mgr-z[23693]: 2024-08-19T01:27:07.762+0000 7f9803dca040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-08-19T01:27:08.105 INFO:journalctl@ceph.mgr.z.smithi167.stdout:Aug 19 01:27:07 smithi167 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-mgr-z[23693]: 2024-08-19T01:27:07.820+0000 7f9803dca040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-08-19T01:27:08.105 INFO:journalctl@ceph.mgr.z.smithi167.stdout:Aug 19 01:27:07 smithi167 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-mgr-z[23693]: 2024-08-19T01:27:07.886+0000 7f9803dca040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-08-19T01:27:08.105 INFO:journalctl@ceph.mgr.z.smithi167.stdout:Aug 19 01:27:08 smithi167 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-mgr-z[23693]: 2024-08-19T01:27:08.103+0000 7f9803dca040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-08-19T01:27:08.109 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:27:07 smithi083 ceph-mon[17120]: from='client.24116 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi083:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-08-19T01:27:08.146 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:27:07 smithi117 ceph-mon[22016]: from='client.24116 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi083:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-08-19T01:27:08.146 INFO:journalctl@ceph.mgr.y.smithi117.stdout:Aug 19 01:27:08 smithi117 systemd[1]: Started Ceph mgr.y for bad8780e-5dc9-11ef-bccf-c7b262605968. 2024-08-19T01:27:08.375 INFO:journalctl@ceph.mgr.z.smithi167.stdout:Aug 19 01:27:08 smithi167 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-mgr-z[23693]: 2024-08-19T01:27:08.164+0000 7f9803dca040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-08-19T01:27:08.453 INFO:journalctl@ceph.mgr.y.smithi117.stdout:Aug 19 01:27:08 smithi117 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-mgr-y[23381]: 2024-08-19T01:27:08.144+0000 7f548ee53040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-08-19T01:27:08.453 INFO:journalctl@ceph.mgr.y.smithi117.stdout:Aug 19 01:27:08 smithi117 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-mgr-y[23381]: 2024-08-19T01:27:08.211+0000 7f548ee53040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-08-19T01:27:08.453 INFO:journalctl@ceph.mgr.y.smithi117.stdout:Aug 19 01:27:08 smithi117 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-mgr-y[23381]: 2024-08-19T01:27:08.361+0000 7f548ee53040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-08-19T01:27:08.810 INFO:journalctl@ceph.mgr.y.smithi117.stdout:Aug 19 01:27:08 smithi117 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-mgr-y[23381]: 2024-08-19T01:27:08.528+0000 7f548ee53040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-08-19T01:27:08.810 INFO:journalctl@ceph.mgr.y.smithi117.stdout:Aug 19 01:27:08 smithi117 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-mgr-y[23381]: 2024-08-19T01:27:08.685+0000 7f548ee53040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-08-19T01:27:08.811 INFO:journalctl@ceph.mgr.y.smithi117.stdout:Aug 19 01:27:08 smithi117 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-mgr-y[23381]: 2024-08-19T01:27:08.808+0000 7f548ee53040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-08-19T01:27:09.037 INFO:journalctl@ceph.mgr.z.smithi167.stdout:Aug 19 01:27:08 smithi167 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-mgr-z[23693]: 2024-08-19T01:27:08.748+0000 7f9803dca040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-08-19T01:27:09.095 INFO:journalctl@ceph.mgr.y.smithi117.stdout:Aug 19 01:27:08 smithi117 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-mgr-y[23381]: 2024-08-19T01:27:08.939+0000 7f548ee53040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-08-19T01:27:09.096 INFO:journalctl@ceph.mgr.y.smithi117.stdout:Aug 19 01:27:09 smithi117 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-mgr-y[23381]: 2024-08-19T01:27:09.000+0000 7f548ee53040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-08-19T01:27:09.376 INFO:journalctl@ceph.mgr.y.smithi117.stdout:Aug 19 01:27:09 smithi117 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-mgr-y[23381]: 2024-08-19T01:27:09.306+0000 7f548ee53040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-08-19T01:27:09.377 INFO:journalctl@ceph.mgr.y.smithi117.stdout:Aug 19 01:27:09 smithi117 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-mgr-y[23381]: 2024-08-19T01:27:09.374+0000 7f548ee53040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-08-19T01:27:09.377 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:27:09 smithi117 ceph-mon[22016]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' 2024-08-19T01:27:09.377 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:27:09 smithi117 ceph-mon[22016]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' 2024-08-19T01:27:09.377 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:27:09 smithi117 ceph-mon[22016]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' 2024-08-19T01:27:09.377 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:27:09 smithi117 ceph-mon[22016]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' 2024-08-19T01:27:09.377 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:27:09 smithi117 ceph-mon[22016]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-19T01:27:09.377 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:27:09 smithi117 ceph-mon[22016]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-19T01:27:09.405 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:27:09 smithi083 ceph-mon[17120]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' 2024-08-19T01:27:09.405 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:27:09 smithi083 ceph-mon[17120]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' 2024-08-19T01:27:09.405 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:27:09 smithi083 ceph-mon[17120]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' 2024-08-19T01:27:09.405 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:27:09 smithi083 ceph-mon[17120]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' 2024-08-19T01:27:09.405 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:27:09 smithi083 ceph-mon[17120]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-19T01:27:09.405 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:27:09 smithi083 ceph-mon[17120]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-19T01:27:09.466 INFO:journalctl@ceph.mgr.z.smithi167.stdout:Aug 19 01:27:09 smithi167 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-mgr-z[23693]: 2024-08-19T01:27:09.078+0000 7f9803dca040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-08-19T01:27:09.467 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:27:09 smithi167 ceph-mon[21875]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' 2024-08-19T01:27:09.467 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:27:09 smithi167 ceph-mon[21875]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' 2024-08-19T01:27:09.467 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:27:09 smithi167 ceph-mon[21875]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' 2024-08-19T01:27:09.467 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:27:09 smithi167 ceph-mon[21875]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' 2024-08-19T01:27:09.467 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:27:09 smithi167 ceph-mon[21875]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-19T01:27:09.467 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:27:09 smithi167 ceph-mon[21875]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-19T01:27:09.656 INFO:journalctl@ceph.mgr.y.smithi117.stdout:Aug 19 01:27:09 smithi117 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-mgr-y[23381]: 2024-08-19T01:27:09.444+0000 7f548ee53040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-08-19T01:27:09.657 INFO:journalctl@ceph.mgr.y.smithi117.stdout:Aug 19 01:27:09 smithi117 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-mgr-y[23381]: 2024-08-19T01:27:09.594+0000 7f548ee53040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-08-19T01:27:09.657 INFO:journalctl@ceph.mgr.y.smithi117.stdout:Aug 19 01:27:09 smithi117 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-mgr-y[23381]: 2024-08-19T01:27:09.654+0000 7f548ee53040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-08-19T01:27:09.908 INFO:journalctl@ceph.mgr.y.smithi117.stdout:Aug 19 01:27:09 smithi117 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-mgr-y[23381]: 2024-08-19T01:27:09.721+0000 7f548ee53040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-08-19T01:27:09.908 INFO:journalctl@ceph.mgr.y.smithi117.stdout:Aug 19 01:27:09 smithi117 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-mgr-y[23381]: 2024-08-19T01:27:09.905+0000 7f548ee53040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-08-19T01:27:09.967 INFO:journalctl@ceph.mgr.z.smithi167.stdout:Aug 19 01:27:09 smithi167 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-mgr-z[23693]: 2024-08-19T01:27:09.510+0000 7f9803dca040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-08-19T01:27:09.967 INFO:journalctl@ceph.mgr.z.smithi167.stdout:Aug 19 01:27:09 smithi167 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-mgr-z[23693]: 2024-08-19T01:27:09.570+0000 7f9803dca040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-08-19T01:27:09.967 INFO:journalctl@ceph.mgr.z.smithi167.stdout:Aug 19 01:27:09 smithi167 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-mgr-z[23693]: 2024-08-19T01:27:09.630+0000 7f9803dca040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-08-19T01:27:10.203 INFO:journalctl@ceph.mgr.y.smithi117.stdout:Aug 19 01:27:09 smithi117 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-mgr-y[23381]: 2024-08-19T01:27:09.966+0000 7f548ee53040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-08-19T01:27:10.203 INFO:journalctl@ceph.mgr.y.smithi117.stdout:Aug 19 01:27:10 smithi117 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-mgr-y[23381]: 2024-08-19T01:27:10.025+0000 7f548ee53040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-08-19T01:27:10.203 INFO:journalctl@ceph.mgr.y.smithi117.stdout:Aug 19 01:27:10 smithi117 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-mgr-y[23381]: 2024-08-19T01:27:10.092+0000 7f548ee53040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-08-19T01:27:10.655 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:27:10 smithi083 ceph-mon[17120]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' 2024-08-19T01:27:10.655 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:27:10 smithi083 ceph-mon[17120]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' 2024-08-19T01:27:10.655 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:27:10 smithi083 ceph-mon[17120]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' 2024-08-19T01:27:10.656 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:27:10 smithi083 ceph-mon[17120]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' 2024-08-19T01:27:10.656 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:27:10 smithi083 ceph-mon[17120]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' 2024-08-19T01:27:10.656 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:27:10 smithi083 ceph-mon[17120]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-19T01:27:10.656 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:27:10 smithi083 ceph-mon[17120]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-19T01:27:10.656 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:27:10 smithi083 ceph-mon[17120]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' 2024-08-19T01:27:10.656 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:27:10 smithi083 ceph-mon[17120]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-19T01:27:10.656 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:27:10 smithi083 ceph-mon[17120]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-19T01:27:10.656 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:27:10 smithi083 ceph-mon[17120]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-19T01:27:10.656 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:27:10 smithi083 ceph-mon[17120]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' 2024-08-19T01:27:10.656 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:27:10 smithi083 ceph-mon[17120]: from='client.? 172.21.15.83:0/3870579895' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ffa74041-f9d2-4555-bb43-508b8c581487"}]: dispatch 2024-08-19T01:27:10.656 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:27:10 smithi083 ceph-mon[17120]: from='client.? 172.21.15.83:0/3870579895' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "ffa74041-f9d2-4555-bb43-508b8c581487"}]': finished 2024-08-19T01:27:10.657 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:27:10 smithi083 ceph-mon[17120]: osdmap e5: 1 total, 0 up, 1 in 2024-08-19T01:27:10.657 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:27:10 smithi083 ceph-mon[17120]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-08-19T01:27:10.657 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:27:10 smithi083 ceph-mon[17120]: from='client.? 172.21.15.83:0/84461792' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-08-19T01:27:10.703 INFO:journalctl@ceph.mgr.y.smithi117.stdout:Aug 19 01:27:10 smithi117 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-mgr-y[23381]: 2024-08-19T01:27:10.300+0000 7f548ee53040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-08-19T01:27:10.703 INFO:journalctl@ceph.mgr.y.smithi117.stdout:Aug 19 01:27:10 smithi117 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-mgr-y[23381]: 2024-08-19T01:27:10.359+0000 7f548ee53040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-08-19T01:27:10.703 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:27:10 smithi117 ceph-mon[22016]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' 2024-08-19T01:27:10.703 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:27:10 smithi117 ceph-mon[22016]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' 2024-08-19T01:27:10.703 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:27:10 smithi117 ceph-mon[22016]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' 2024-08-19T01:27:10.704 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:27:10 smithi117 ceph-mon[22016]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' 2024-08-19T01:27:10.704 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:27:10 smithi117 ceph-mon[22016]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' 2024-08-19T01:27:10.704 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:27:10 smithi117 ceph-mon[22016]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-19T01:27:10.704 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:27:10 smithi117 ceph-mon[22016]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-19T01:27:10.704 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:27:10 smithi117 ceph-mon[22016]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' 2024-08-19T01:27:10.704 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:27:10 smithi117 ceph-mon[22016]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-19T01:27:10.704 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:27:10 smithi117 ceph-mon[22016]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-19T01:27:10.704 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:27:10 smithi117 ceph-mon[22016]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-19T01:27:10.704 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:27:10 smithi117 ceph-mon[22016]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' 2024-08-19T01:27:10.704 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:27:10 smithi117 ceph-mon[22016]: from='client.? 172.21.15.83:0/3870579895' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ffa74041-f9d2-4555-bb43-508b8c581487"}]: dispatch 2024-08-19T01:27:10.704 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:27:10 smithi117 ceph-mon[22016]: from='client.? 172.21.15.83:0/3870579895' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "ffa74041-f9d2-4555-bb43-508b8c581487"}]': finished 2024-08-19T01:27:10.705 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:27:10 smithi117 ceph-mon[22016]: osdmap e5: 1 total, 0 up, 1 in 2024-08-19T01:27:10.705 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:27:10 smithi117 ceph-mon[22016]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-08-19T01:27:10.705 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:27:10 smithi117 ceph-mon[22016]: from='client.? 172.21.15.83:0/84461792' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-08-19T01:27:10.717 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:27:10 smithi167 ceph-mon[21875]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' 2024-08-19T01:27:10.717 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:27:10 smithi167 ceph-mon[21875]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' 2024-08-19T01:27:10.717 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:27:10 smithi167 ceph-mon[21875]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' 2024-08-19T01:27:10.717 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:27:10 smithi167 ceph-mon[21875]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' 2024-08-19T01:27:10.717 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:27:10 smithi167 ceph-mon[21875]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' 2024-08-19T01:27:10.717 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:27:10 smithi167 ceph-mon[21875]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-19T01:27:10.717 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:27:10 smithi167 ceph-mon[21875]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-19T01:27:10.718 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:27:10 smithi167 ceph-mon[21875]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' 2024-08-19T01:27:10.718 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:27:10 smithi167 ceph-mon[21875]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-19T01:27:10.718 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:27:10 smithi167 ceph-mon[21875]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-19T01:27:10.718 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:27:10 smithi167 ceph-mon[21875]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-19T01:27:10.718 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:27:10 smithi167 ceph-mon[21875]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' 2024-08-19T01:27:10.718 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:27:10 smithi167 ceph-mon[21875]: from='client.? 172.21.15.83:0/3870579895' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ffa74041-f9d2-4555-bb43-508b8c581487"}]: dispatch 2024-08-19T01:27:10.718 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:27:10 smithi167 ceph-mon[21875]: from='client.? 172.21.15.83:0/3870579895' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "ffa74041-f9d2-4555-bb43-508b8c581487"}]': finished 2024-08-19T01:27:10.718 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:27:10 smithi167 ceph-mon[21875]: osdmap e5: 1 total, 0 up, 1 in 2024-08-19T01:27:10.718 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:27:10 smithi167 ceph-mon[21875]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-08-19T01:27:10.718 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:27:10 smithi167 ceph-mon[21875]: from='client.? 172.21.15.83:0/84461792' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-08-19T01:27:11.182 INFO:journalctl@ceph.mgr.y.smithi117.stdout:Aug 19 01:27:10 smithi117 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-mgr-y[23381]: 2024-08-19T01:27:10.875+0000 7f548ee53040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-08-19T01:27:11.453 INFO:journalctl@ceph.mgr.y.smithi117.stdout:Aug 19 01:27:11 smithi117 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-mgr-y[23381]: 2024-08-19T01:27:11.180+0000 7f548ee53040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-08-19T01:27:11.453 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:27:11 smithi117 ceph-mon[22016]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-19T01:27:11.453 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:27:11 smithi117 ceph-mon[22016]: Standby manager daemon z started 2024-08-19T01:27:11.453 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:27:11 smithi117 ceph-mon[22016]: from='mgr.? 172.21.15.167:0/2884533184' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-08-19T01:27:11.453 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:27:11 smithi117 ceph-mon[22016]: from='mgr.? 172.21.15.167:0/2884533184' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-08-19T01:27:11.453 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:27:11 smithi117 ceph-mon[22016]: from='mgr.? 172.21.15.167:0/2884533184' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-08-19T01:27:11.453 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:27:11 smithi117 ceph-mon[22016]: from='mgr.? 172.21.15.167:0/2884533184' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-08-19T01:27:11.626 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:27:11.623+0000 7f4fa6000640 1 -- 172.21.15.83:0/1949739374 <== mon.2 v2:172.21.15.117:3300/0 7 ==== mgrmap(e 14) ==== 92060+0+0 (secure 0 0 0) 0x7f4f900303a0 con 0x7f4fa8104d30 2024-08-19T01:27:11.655 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:27:11 smithi083 ceph-mon[17120]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-19T01:27:11.655 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:27:11 smithi083 ceph-mon[17120]: Standby manager daemon z started 2024-08-19T01:27:11.655 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:27:11 smithi083 ceph-mon[17120]: from='mgr.? 172.21.15.167:0/2884533184' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-08-19T01:27:11.655 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:27:11 smithi083 ceph-mon[17120]: from='mgr.? 172.21.15.167:0/2884533184' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-08-19T01:27:11.655 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:27:11 smithi083 ceph-mon[17120]: from='mgr.? 172.21.15.167:0/2884533184' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-08-19T01:27:11.655 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:27:11 smithi083 ceph-mon[17120]: from='mgr.? 172.21.15.167:0/2884533184' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-08-19T01:27:11.717 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:27:11 smithi167 ceph-mon[21875]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-19T01:27:11.717 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:27:11 smithi167 ceph-mon[21875]: Standby manager daemon z started 2024-08-19T01:27:11.717 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:27:11 smithi167 ceph-mon[21875]: from='mgr.? 172.21.15.167:0/2884533184' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-08-19T01:27:11.717 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:27:11 smithi167 ceph-mon[21875]: from='mgr.? 172.21.15.167:0/2884533184' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-08-19T01:27:11.717 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:27:11 smithi167 ceph-mon[21875]: from='mgr.? 172.21.15.167:0/2884533184' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-08-19T01:27:11.717 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:27:11 smithi167 ceph-mon[21875]: from='mgr.? 172.21.15.167:0/2884533184' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-08-19T01:27:11.953 INFO:journalctl@ceph.mgr.y.smithi117.stdout:Aug 19 01:27:11 smithi117 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-mgr-y[23381]: 2024-08-19T01:27:11.587+0000 7f548ee53040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-08-19T01:27:11.953 INFO:journalctl@ceph.mgr.y.smithi117.stdout:Aug 19 01:27:11 smithi117 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-mgr-y[23381]: 2024-08-19T01:27:11.643+0000 7f548ee53040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-08-19T01:27:11.953 INFO:journalctl@ceph.mgr.y.smithi117.stdout:Aug 19 01:27:11 smithi117 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-mgr-y[23381]: 2024-08-19T01:27:11.701+0000 7f548ee53040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-08-19T01:27:12.905 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:27:12 smithi083 ceph-mon[17120]: mgrmap e14: x(active, since 67s), standbys: z 2024-08-19T01:27:12.905 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:27:12 smithi083 ceph-mon[17120]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-08-19T01:27:12.905 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:27:12 smithi083 ceph-mon[17120]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-19T01:27:12.952 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:27:12 smithi117 ceph-mon[22016]: mgrmap e14: x(active, since 67s), standbys: z 2024-08-19T01:27:12.953 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:27:12 smithi117 ceph-mon[22016]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-08-19T01:27:12.953 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:27:12 smithi117 ceph-mon[22016]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-19T01:27:12.966 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:27:12 smithi167 ceph-mon[21875]: mgrmap e14: x(active, since 67s), standbys: z 2024-08-19T01:27:12.966 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:27:12 smithi167 ceph-mon[21875]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-08-19T01:27:12.966 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:27:12 smithi167 ceph-mon[21875]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-19T01:27:13.641 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:27:13.638+0000 7f4fa6000640 1 -- 172.21.15.83:0/1949739374 <== mon.2 v2:172.21.15.117:3300/0 8 ==== mgrmap(e 15) ==== 137777+0+0 (secure 0 0 0) 0x7f4f9003b4f0 con 0x7f4fa8104d30 2024-08-19T01:27:13.905 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:27:13 smithi083 ceph-mon[17120]: Standby manager daemon y started 2024-08-19T01:27:13.905 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:27:13 smithi083 ceph-mon[17120]: from='mgr.? 172.21.15.117:0/1949399549' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-08-19T01:27:13.905 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:27:13 smithi083 ceph-mon[17120]: from='mgr.? 172.21.15.117:0/1949399549' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-08-19T01:27:13.905 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:27:13 smithi083 ceph-mon[17120]: from='mgr.? 172.21.15.117:0/1949399549' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-08-19T01:27:13.905 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:27:13 smithi083 ceph-mon[17120]: from='mgr.? 172.21.15.117:0/1949399549' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-08-19T01:27:13.953 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:27:13 smithi117 ceph-mon[22016]: Standby manager daemon y started 2024-08-19T01:27:13.953 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:27:13 smithi117 ceph-mon[22016]: from='mgr.? 172.21.15.117:0/1949399549' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-08-19T01:27:13.953 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:27:13 smithi117 ceph-mon[22016]: from='mgr.? 172.21.15.117:0/1949399549' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-08-19T01:27:13.953 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:27:13 smithi117 ceph-mon[22016]: from='mgr.? 172.21.15.117:0/1949399549' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-08-19T01:27:13.953 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:27:13 smithi117 ceph-mon[22016]: from='mgr.? 172.21.15.117:0/1949399549' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-08-19T01:27:13.966 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:27:13 smithi167 ceph-mon[21875]: Standby manager daemon y started 2024-08-19T01:27:13.967 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:27:13 smithi167 ceph-mon[21875]: from='mgr.? 172.21.15.117:0/1949399549' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-08-19T01:27:13.967 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:27:13 smithi167 ceph-mon[21875]: from='mgr.? 172.21.15.117:0/1949399549' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-08-19T01:27:13.967 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:27:13 smithi167 ceph-mon[21875]: from='mgr.? 172.21.15.117:0/1949399549' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-08-19T01:27:13.967 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:27:13 smithi167 ceph-mon[21875]: from='mgr.? 172.21.15.117:0/1949399549' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-08-19T01:27:14.905 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:27:14 smithi083 ceph-mon[17120]: mgrmap e15: x(active, since 69s), standbys: z, y 2024-08-19T01:27:14.905 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:27:14 smithi083 ceph-mon[17120]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-08-19T01:27:14.905 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:27:14 smithi083 ceph-mon[17120]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-19T01:27:14.953 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:27:14 smithi117 ceph-mon[22016]: mgrmap e15: x(active, since 69s), standbys: z, y 2024-08-19T01:27:14.953 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:27:14 smithi117 ceph-mon[22016]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-08-19T01:27:14.953 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:27:14 smithi117 ceph-mon[22016]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-19T01:27:14.966 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:27:14 smithi167 ceph-mon[21875]: mgrmap e15: x(active, since 69s), standbys: z, y 2024-08-19T01:27:14.967 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:27:14 smithi167 ceph-mon[21875]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-08-19T01:27:14.967 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:27:14 smithi167 ceph-mon[21875]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-19T01:27:17.508 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:27:17 smithi083 ceph-mon[17120]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-19T01:27:17.656 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:27:17 smithi117 ceph-mon[22016]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-19T01:27:17.669 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:27:17 smithi167 ceph-mon[21875]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-19T01:27:18.405 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:27:18 smithi083 ceph-mon[17120]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-08-19T01:27:18.405 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:27:18 smithi083 ceph-mon[17120]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-19T01:27:18.703 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:27:18 smithi117 ceph-mon[22016]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-08-19T01:27:18.703 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:27:18 smithi117 ceph-mon[22016]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-19T01:27:18.718 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:27:18 smithi167 ceph-mon[21875]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-08-19T01:27:18.718 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:27:18 smithi167 ceph-mon[21875]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-19T01:27:19.486 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:27:19 smithi083 ceph-mon[17120]: Deploying daemon osd.0 on smithi083 2024-08-19T01:27:19.487 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:27:19 smithi083 ceph-mon[17120]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-19T01:27:19.703 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:27:19 smithi117 ceph-mon[22016]: Deploying daemon osd.0 on smithi083 2024-08-19T01:27:19.703 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:27:19 smithi117 ceph-mon[22016]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-19T01:27:19.716 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:27:19 smithi167 ceph-mon[21875]: Deploying daemon osd.0 on smithi083 2024-08-19T01:27:19.717 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:27:19 smithi167 ceph-mon[21875]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-19T01:27:21.390 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:27:21 smithi083 ceph-mon[17120]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-19T01:27:21.703 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:27:21 smithi117 ceph-mon[22016]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-19T01:27:21.716 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:27:21 smithi167 ceph-mon[21875]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-19T01:27:23.599 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:27:23 smithi083 ceph-mon[17120]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-19T01:27:23.703 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:27:23 smithi117 ceph-mon[22016]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-19T01:27:23.716 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:27:23 smithi167 ceph-mon[21875]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-19T01:27:24.520 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:27:24 smithi083 ceph-mon[17120]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-19T01:27:24.520 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:27:24 smithi083 ceph-mon[17120]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' 2024-08-19T01:27:24.520 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:27:24 smithi083 ceph-mon[17120]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' 2024-08-19T01:27:24.703 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:27:24 smithi117 ceph-mon[22016]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-19T01:27:24.703 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:27:24 smithi117 ceph-mon[22016]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' 2024-08-19T01:27:24.703 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:27:24 smithi117 ceph-mon[22016]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' 2024-08-19T01:27:24.716 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:27:24 smithi167 ceph-mon[21875]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-19T01:27:24.716 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:27:24 smithi167 ceph-mon[21875]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' 2024-08-19T01:27:24.716 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:27:24 smithi167 ceph-mon[21875]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' 2024-08-19T01:27:25.421 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:27:25 smithi083 ceph-mon[17120]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-19T01:27:25.703 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:27:25 smithi117 ceph-mon[22016]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-19T01:27:25.716 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:27:25 smithi167 ceph-mon[21875]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-19T01:27:27.405 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:27:27 smithi083 ceph-mon[17120]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-19T01:27:27.405 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:27:27 smithi083 ceph-mon[17120]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' 2024-08-19T01:27:27.405 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:27:27 smithi083 ceph-mon[17120]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' 2024-08-19T01:27:27.405 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:27:27 smithi083 ceph-mon[17120]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-19T01:27:27.405 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:27:27 smithi083 ceph-mon[17120]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-19T01:27:27.405 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:27:27 smithi083 ceph-mon[17120]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' 2024-08-19T01:27:27.703 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:27:27 smithi117 ceph-mon[22016]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-19T01:27:27.703 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:27:27 smithi117 ceph-mon[22016]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' 2024-08-19T01:27:27.703 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:27:27 smithi117 ceph-mon[22016]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' 2024-08-19T01:27:27.703 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:27:27 smithi117 ceph-mon[22016]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-19T01:27:27.703 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:27:27 smithi117 ceph-mon[22016]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-19T01:27:27.703 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:27:27 smithi117 ceph-mon[22016]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' 2024-08-19T01:27:27.716 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:27:27 smithi167 ceph-mon[21875]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-19T01:27:27.716 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:27:27 smithi167 ceph-mon[21875]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' 2024-08-19T01:27:27.716 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:27:27 smithi167 ceph-mon[21875]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' 2024-08-19T01:27:27.717 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:27:27 smithi167 ceph-mon[21875]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-19T01:27:27.717 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:27:27 smithi167 ceph-mon[21875]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-19T01:27:27.717 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:27:27 smithi167 ceph-mon[21875]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' 2024-08-19T01:27:27.890 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:27:27.888+0000 7f4fa6000640 1 -- 172.21.15.83:0/1949739374 <== mgr.14150 v2:172.21.15.83:6800/683740916 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f4fa8062360 con 0x7f4f8c0392c0 2024-08-19T01:27:27.890 INFO:teuthology.orchestra.run.smithi083.stdout:Created osd(s) 0 on host 'smithi083' 2024-08-19T01:27:27.892 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:27:27.890+0000 7f4fafa00640 1 -- 172.21.15.83:0/1949739374 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f4f8c0392c0 msgr2=0x7f4f8c03b780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:27:27.893 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:27:27.890+0000 7f4fafa00640 1 --2- 172.21.15.83:0/1949739374 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f4f8c0392c0 0x7f4f8c03b780 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f4f980058c0 tx=0x7f4f98005f90 comp rx=0 tx=0).stop 2024-08-19T01:27:27.893 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:27:27.890+0000 7f4fafa00640 1 -- 172.21.15.83:0/1949739374 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4fa8104d30 msgr2=0x7f4fa819a360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:27:27.893 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:27:27.890+0000 7f4fafa00640 1 --2- 172.21.15.83:0/1949739374 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4fa8104d30 0x7f4fa819a360 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f4f90031440 tx=0x7f4f90004270 comp rx=0 tx=0).stop 2024-08-19T01:27:27.893 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:27:27.890+0000 7f4fafa00640 1 -- 172.21.15.83:0/1949739374 shutdown_connections 2024-08-19T01:27:27.893 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:27:27.890+0000 7f4fafa00640 1 --2- 172.21.15.83:0/1949739374 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f4f8c0392c0 0x7f4f8c03b780 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:27:27.893 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:27:27.890+0000 7f4fafa00640 1 --2- 172.21.15.83:0/1949739374 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4fa81056f0 0x7f4fa81a13e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:27:27.893 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:27:27.890+0000 7f4fafa00640 1 --2- 172.21.15.83:0/1949739374 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4fa8104d30 0x7f4fa819a360 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:27:27.893 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:27:27.890+0000 7f4fafa00640 1 --2- 172.21.15.83:0/1949739374 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4fa8103b30 0x7f4fa8199e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:27:27.893 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:27:27.890+0000 7f4fafa00640 1 -- 172.21.15.83:0/1949739374 >> 172.21.15.83:0/1949739374 conn(0x7f4fa80ff2e0 msgr2=0x7f4fa8108100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:27:27.893 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:27:27.890+0000 7f4fafa00640 1 -- 172.21.15.83:0/1949739374 shutdown_connections 2024-08-19T01:27:27.893 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:27:27.890+0000 7f4fafa00640 1 -- 172.21.15.83:0/1949739374 wait complete. 2024-08-19T01:27:28.588 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:27:28 smithi083 ceph-mon[17120]: from='osd.0 [v2:172.21.15.83:6802/1082526824,v1:172.21.15.83:6803/1082526824]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-08-19T01:27:28.588 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:27:28 smithi083 ceph-mon[17120]: from='osd.0 ' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-08-19T01:27:28.588 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:27:28 smithi083 ceph-mon[17120]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-19T01:27:28.588 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:27:28 smithi083 ceph-mon[17120]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' 2024-08-19T01:27:28.588 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:27:28 smithi083 ceph-mon[17120]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' 2024-08-19T01:27:28.624 DEBUG:teuthology.orchestra.run.smithi083:osd.0> sudo journalctl -f -n 0 -u ceph-bad8780e-5dc9-11ef-bccf-c7b262605968@osd.0.service 2024-08-19T01:27:28.628 INFO:tasks.cephadm:Deploying osd.1 on smithi117 with /dev/vg_nvme/lv_4... 2024-08-19T01:27:28.629 DEBUG:teuthology.orchestra.run.smithi117:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833 ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid bad8780e-5dc9-11ef-bccf-c7b262605968 -- lvm zap /dev/vg_nvme/lv_4 2024-08-19T01:27:28.653 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:27:28 smithi117 ceph-mon[22016]: from='osd.0 [v2:172.21.15.83:6802/1082526824,v1:172.21.15.83:6803/1082526824]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-08-19T01:27:28.653 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:27:28 smithi117 ceph-mon[22016]: from='osd.0 ' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-08-19T01:27:28.653 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:27:28 smithi117 ceph-mon[22016]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-19T01:27:28.653 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:27:28 smithi117 ceph-mon[22016]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' 2024-08-19T01:27:28.653 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:27:28 smithi117 ceph-mon[22016]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' 2024-08-19T01:27:28.716 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:27:28 smithi167 ceph-mon[21875]: from='osd.0 [v2:172.21.15.83:6802/1082526824,v1:172.21.15.83:6803/1082526824]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-08-19T01:27:28.717 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:27:28 smithi167 ceph-mon[21875]: from='osd.0 ' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-08-19T01:27:28.717 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:27:28 smithi167 ceph-mon[21875]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-19T01:27:28.717 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:27:28 smithi167 ceph-mon[21875]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' 2024-08-19T01:27:28.717 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:27:28 smithi167 ceph-mon[21875]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' 2024-08-19T01:27:28.874 INFO:teuthology.orchestra.run.smithi117.stderr:Inferring config /var/lib/ceph/bad8780e-5dc9-11ef-bccf-c7b262605968/mon.b/config 2024-08-19T01:27:29.625 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:27:29 smithi083 ceph-mon[17120]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-19T01:27:29.625 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:27:29 smithi083 ceph-mon[17120]: from='osd.0 ' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-08-19T01:27:29.625 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:27:29 smithi083 ceph-mon[17120]: from='osd.0 [v2:172.21.15.83:6802/1082526824,v1:172.21.15.83:6803/1082526824]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi083", "root=default"]}]: dispatch 2024-08-19T01:27:29.626 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:27:29 smithi083 ceph-mon[17120]: osdmap e6: 1 total, 0 up, 1 in 2024-08-19T01:27:29.626 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:27:29 smithi083 ceph-mon[17120]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-08-19T01:27:29.626 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:27:29 smithi083 ceph-mon[17120]: from='osd.0 ' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi083", "root=default"]}]: dispatch 2024-08-19T01:27:29.703 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:27:29 smithi117 ceph-mon[22016]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-19T01:27:29.703 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:27:29 smithi117 ceph-mon[22016]: from='osd.0 ' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-08-19T01:27:29.703 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:27:29 smithi117 ceph-mon[22016]: from='osd.0 [v2:172.21.15.83:6802/1082526824,v1:172.21.15.83:6803/1082526824]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi083", "root=default"]}]: dispatch 2024-08-19T01:27:29.703 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:27:29 smithi117 ceph-mon[22016]: osdmap e6: 1 total, 0 up, 1 in 2024-08-19T01:27:29.703 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:27:29 smithi117 ceph-mon[22016]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-08-19T01:27:29.704 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:27:29 smithi117 ceph-mon[22016]: from='osd.0 ' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi083", "root=default"]}]: dispatch 2024-08-19T01:27:29.716 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:27:29 smithi167 ceph-mon[21875]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-19T01:27:29.717 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:27:29 smithi167 ceph-mon[21875]: from='osd.0 ' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-08-19T01:27:29.717 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:27:29 smithi167 ceph-mon[21875]: from='osd.0 [v2:172.21.15.83:6802/1082526824,v1:172.21.15.83:6803/1082526824]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi083", "root=default"]}]: dispatch 2024-08-19T01:27:29.717 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:27:29 smithi167 ceph-mon[21875]: osdmap e6: 1 total, 0 up, 1 in 2024-08-19T01:27:29.717 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:27:29 smithi167 ceph-mon[21875]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-08-19T01:27:29.717 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:27:29 smithi167 ceph-mon[21875]: from='osd.0 ' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi083", "root=default"]}]: dispatch 2024-08-19T01:27:30.489 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:27:30 smithi083 ceph-mon[17120]: from='osd.0 ' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi083", "root=default"]}]': finished 2024-08-19T01:27:30.489 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:27:30 smithi083 ceph-mon[17120]: osdmap e7: 1 total, 0 up, 1 in 2024-08-19T01:27:30.490 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:27:30 smithi083 ceph-mon[17120]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-08-19T01:27:30.490 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:27:30 smithi083 ceph-mon[17120]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-08-19T01:27:30.490 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:27:30 smithi083 ceph-mon[17120]: from='osd.0 ' entity='osd.0' 2024-08-19T01:27:30.490 INFO:journalctl@ceph.osd.0.smithi083.stdout:Aug 19 01:27:30 smithi083 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-0[31265]: 2024-08-19T01:27:30.366+0000 7fb3bba00640 -1 osd.0 0 waiting for initial osdmap 2024-08-19T01:27:30.490 INFO:journalctl@ceph.osd.0.smithi083.stdout:Aug 19 01:27:30 smithi083 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-0[31265]: 2024-08-19T01:27:30.385+0000 7fb3b2400640 -1 osd.0 7 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-08-19T01:27:30.703 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:27:30 smithi117 ceph-mon[22016]: from='osd.0 ' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi083", "root=default"]}]': finished 2024-08-19T01:27:30.703 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:27:30 smithi117 ceph-mon[22016]: osdmap e7: 1 total, 0 up, 1 in 2024-08-19T01:27:30.703 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:27:30 smithi117 ceph-mon[22016]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-08-19T01:27:30.703 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:27:30 smithi117 ceph-mon[22016]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-08-19T01:27:30.703 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:27:30 smithi117 ceph-mon[22016]: from='osd.0 ' entity='osd.0' 2024-08-19T01:27:30.716 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:27:30 smithi167 ceph-mon[21875]: from='osd.0 ' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi083", "root=default"]}]': finished 2024-08-19T01:27:30.716 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:27:30 smithi167 ceph-mon[21875]: osdmap e7: 1 total, 0 up, 1 in 2024-08-19T01:27:30.716 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:27:30 smithi167 ceph-mon[21875]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-08-19T01:27:30.716 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:27:30 smithi167 ceph-mon[21875]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-08-19T01:27:30.716 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:27:30 smithi167 ceph-mon[21875]: from='osd.0 ' entity='osd.0' 2024-08-19T01:27:31.042 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-08-19T01:27:31.067 DEBUG:teuthology.orchestra.run.smithi117:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid bad8780e-5dc9-11ef-bccf-c7b262605968 -- ceph orch daemon add osd smithi117:vg_nvme/lv_4 2024-08-19T01:27:31.360 INFO:teuthology.orchestra.run.smithi117.stderr:Inferring config /var/lib/ceph/bad8780e-5dc9-11ef-bccf-c7b262605968/mon.b/config 2024-08-19T01:27:31.597 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:27:31 smithi117 ceph-mon[22016]: purged_snaps scrub starts 2024-08-19T01:27:31.598 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:27:31 smithi117 ceph-mon[22016]: purged_snaps scrub ok 2024-08-19T01:27:31.598 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:27:31 smithi117 ceph-mon[22016]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-19T01:27:31.598 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:27:31 smithi117 ceph-mon[22016]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-08-19T01:27:31.598 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:27:31 smithi117 ceph-mon[22016]: osd.0 [v2:172.21.15.83:6802/1082526824,v1:172.21.15.83:6803/1082526824] boot 2024-08-19T01:27:31.598 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:27:31 smithi117 ceph-mon[22016]: osdmap e8: 1 total, 1 up, 1 in 2024-08-19T01:27:31.598 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:27:31 smithi117 ceph-mon[22016]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-08-19T01:27:31.655 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:27:31 smithi083 ceph-mon[17120]: purged_snaps scrub starts 2024-08-19T01:27:31.655 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:27:31 smithi083 ceph-mon[17120]: purged_snaps scrub ok 2024-08-19T01:27:31.655 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:27:31 smithi083 ceph-mon[17120]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-19T01:27:31.655 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:27:31 smithi083 ceph-mon[17120]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-08-19T01:27:31.656 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:27:31 smithi083 ceph-mon[17120]: osd.0 [v2:172.21.15.83:6802/1082526824,v1:172.21.15.83:6803/1082526824] boot 2024-08-19T01:27:31.656 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:27:31 smithi083 ceph-mon[17120]: osdmap e8: 1 total, 1 up, 1 in 2024-08-19T01:27:31.656 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:27:31 smithi083 ceph-mon[17120]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-08-19T01:27:31.716 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:27:31 smithi167 ceph-mon[21875]: purged_snaps scrub starts 2024-08-19T01:27:31.716 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:27:31 smithi167 ceph-mon[21875]: purged_snaps scrub ok 2024-08-19T01:27:31.717 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:27:31 smithi167 ceph-mon[21875]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-19T01:27:31.717 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:27:31 smithi167 ceph-mon[21875]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-08-19T01:27:31.717 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:27:31 smithi167 ceph-mon[21875]: osd.0 [v2:172.21.15.83:6802/1082526824,v1:172.21.15.83:6803/1082526824] boot 2024-08-19T01:27:31.717 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:27:31 smithi167 ceph-mon[21875]: osdmap e8: 1 total, 1 up, 1 in 2024-08-19T01:27:31.717 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:27:31 smithi167 ceph-mon[21875]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-08-19T01:27:31.797 INFO:teuthology.orchestra.run.smithi117.stderr:2024-08-19T01:27:31.792+0000 7f7fa0dee640 1 -- 172.21.15.117:0/452502558 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7f9c101a00 msgr2=0x7f7f9c101e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:27:31.797 INFO:teuthology.orchestra.run.smithi117.stderr:2024-08-19T01:27:31.792+0000 7f7fa0dee640 1 --2- 172.21.15.117:0/452502558 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7f9c101a00 0x7f7f9c101e80 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f7f90009ca0 tx=0x7f7f9002a8c0 comp rx=0 tx=0).stop 2024-08-19T01:27:31.797 INFO:teuthology.orchestra.run.smithi117.stderr:2024-08-19T01:27:31.792+0000 7f7fa0dee640 1 -- 172.21.15.117:0/452502558 shutdown_connections 2024-08-19T01:27:31.797 INFO:teuthology.orchestra.run.smithi117.stderr:2024-08-19T01:27:31.792+0000 7f7fa0dee640 1 --2- 172.21.15.117:0/452502558 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7f9c1023c0 0x7f7f9c108c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:27:31.797 INFO:teuthology.orchestra.run.smithi117.stderr:2024-08-19T01:27:31.792+0000 7f7fa0dee640 1 --2- 172.21.15.117:0/452502558 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7f9c101a00 0x7f7f9c101e80 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:27:31.798 INFO:teuthology.orchestra.run.smithi117.stderr:2024-08-19T01:27:31.792+0000 7f7fa0dee640 1 --2- 172.21.15.117:0/452502558 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7f9c100800 0x7f7f9c100c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:27:31.798 INFO:teuthology.orchestra.run.smithi117.stderr:2024-08-19T01:27:31.792+0000 7f7fa0dee640 1 -- 172.21.15.117:0/452502558 >> 172.21.15.117:0/452502558 conn(0x7f7f9c0fbfb0 msgr2=0x7f7f9c0fe3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:27:31.798 INFO:teuthology.orchestra.run.smithi117.stderr:2024-08-19T01:27:31.792+0000 7f7fa0dee640 1 -- 172.21.15.117:0/452502558 shutdown_connections 2024-08-19T01:27:31.798 INFO:teuthology.orchestra.run.smithi117.stderr:2024-08-19T01:27:31.792+0000 7f7fa0dee640 1 -- 172.21.15.117:0/452502558 wait complete. 2024-08-19T01:27:31.798 INFO:teuthology.orchestra.run.smithi117.stderr:2024-08-19T01:27:31.793+0000 7f7fa0dee640 1 Processor -- start 2024-08-19T01:27:31.798 INFO:teuthology.orchestra.run.smithi117.stderr:2024-08-19T01:27:31.793+0000 7f7fa0dee640 1 -- start start 2024-08-19T01:27:31.798 INFO:teuthology.orchestra.run.smithi117.stderr:2024-08-19T01:27:31.793+0000 7f7fa0dee640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7f9c100800 0x7f7f9c104930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:27:31.798 INFO:teuthology.orchestra.run.smithi117.stderr:2024-08-19T01:27:31.793+0000 7f7fa0dee640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7f9c101a00 0x7f7f9c102fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:27:31.798 INFO:teuthology.orchestra.run.smithi117.stderr:2024-08-19T01:27:31.793+0000 7f7fa0dee640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7f9c1023c0 0x7f7f9c103510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:27:31.798 INFO:teuthology.orchestra.run.smithi117.stderr:2024-08-19T01:27:31.793+0000 7f7fa0dee640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f7f9c10c480 con 0x7f7f9c1023c0 2024-08-19T01:27:31.798 INFO:teuthology.orchestra.run.smithi117.stderr:2024-08-19T01:27:31.793+0000 7f7fa0dee640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f7f9c10c300 con 0x7f7f9c100800 2024-08-19T01:27:31.798 INFO:teuthology.orchestra.run.smithi117.stderr:2024-08-19T01:27:31.793+0000 7f7fa0dee640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f7f9c10c600 con 0x7f7f9c101a00 2024-08-19T01:27:31.798 INFO:teuthology.orchestra.run.smithi117.stderr:2024-08-19T01:27:31.793+0000 7f7f9a59c640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7f9c100800 0x7f7f9c104930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:27:31.799 INFO:teuthology.orchestra.run.smithi117.stderr:2024-08-19T01:27:31.793+0000 7f7f9a59c640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7f9c100800 0x7f7f9c104930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:36054/0 (socket says 172.21.15.117:36054) 2024-08-19T01:27:31.799 INFO:teuthology.orchestra.run.smithi117.stderr:2024-08-19T01:27:31.793+0000 7f7f9a59c640 1 -- 172.21.15.117:0/2810621282 learned_addr learned my addr 172.21.15.117:0/2810621282 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-08-19T01:27:31.799 INFO:teuthology.orchestra.run.smithi117.stderr:2024-08-19T01:27:31.793+0000 7f7f99d9b640 1 --2- 172.21.15.117:0/2810621282 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7f9c101a00 0x7f7f9c102fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:27:31.799 INFO:teuthology.orchestra.run.smithi117.stderr:2024-08-19T01:27:31.794+0000 7f7f9ad9d640 1 --2- 172.21.15.117:0/2810621282 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7f9c1023c0 0x7f7f9c103510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:27:31.799 INFO:teuthology.orchestra.run.smithi117.stderr:2024-08-19T01:27:31.794+0000 7f7f9a59c640 1 -- 172.21.15.117:0/2810621282 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7f9c101a00 msgr2=0x7f7f9c102fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:27:31.799 INFO:teuthology.orchestra.run.smithi117.stderr:2024-08-19T01:27:31.794+0000 7f7f9a59c640 1 --2- 172.21.15.117:0/2810621282 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7f9c101a00 0x7f7f9c102fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:27:31.799 INFO:teuthology.orchestra.run.smithi117.stderr:2024-08-19T01:27:31.794+0000 7f7f9a59c640 1 -- 172.21.15.117:0/2810621282 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7f9c1023c0 msgr2=0x7f7f9c103510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:27:31.799 INFO:teuthology.orchestra.run.smithi117.stderr:2024-08-19T01:27:31.794+0000 7f7f9a59c640 1 --2- 172.21.15.117:0/2810621282 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7f9c1023c0 0x7f7f9c103510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:27:31.799 INFO:teuthology.orchestra.run.smithi117.stderr:2024-08-19T01:27:31.794+0000 7f7f9a59c640 1 -- 172.21.15.117:0/2810621282 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7f9c103d40 con 0x7f7f9c100800 2024-08-19T01:27:31.799 INFO:teuthology.orchestra.run.smithi117.stderr:2024-08-19T01:27:31.794+0000 7f7f9ad9d640 1 --2- 172.21.15.117:0/2810621282 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7f9c1023c0 0x7f7f9c103510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-19T01:27:31.799 INFO:teuthology.orchestra.run.smithi117.stderr:2024-08-19T01:27:31.794+0000 7f7f9a59c640 1 --2- 172.21.15.117:0/2810621282 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7f9c100800 0x7f7f9c104930 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f7f9400d950 tx=0x7f7f9400de10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:27:31.799 INFO:teuthology.orchestra.run.smithi117.stderr:2024-08-19T01:27:31.794+0000 7f7f8b7fe640 1 -- 172.21.15.117:0/2810621282 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f94017020 con 0x7f7f9c100800 2024-08-19T01:27:31.799 INFO:teuthology.orchestra.run.smithi117.stderr:2024-08-19T01:27:31.794+0000 7f7fa0dee640 1 -- 172.21.15.117:0/2810621282 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7f9c1a1d10 con 0x7f7f9c100800 2024-08-19T01:27:31.800 INFO:teuthology.orchestra.run.smithi117.stderr:2024-08-19T01:27:31.794+0000 7f7f8b7fe640 1 -- 172.21.15.117:0/2810621282 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7f94009c30 con 0x7f7f9c100800 2024-08-19T01:27:31.800 INFO:teuthology.orchestra.run.smithi117.stderr:2024-08-19T01:27:31.794+0000 7f7f8b7fe640 1 -- 172.21.15.117:0/2810621282 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f9400f6b0 con 0x7f7f9c100800 2024-08-19T01:27:31.800 INFO:teuthology.orchestra.run.smithi117.stderr:2024-08-19T01:27:31.794+0000 7f7fa0dee640 1 -- 172.21.15.117:0/2810621282 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7f9c1a2190 con 0x7f7f9c100800 2024-08-19T01:27:31.800 INFO:teuthology.orchestra.run.smithi117.stderr:2024-08-19T01:27:31.795+0000 7f7fa0dee640 1 -- 172.21.15.117:0/2810621282 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7f9c1a2420 con 0x7f7f9c100800 2024-08-19T01:27:31.801 INFO:teuthology.orchestra.run.smithi117.stderr:2024-08-19T01:27:31.797+0000 7f7f8b7fe640 1 -- 172.21.15.117:0/2810621282 <== mon.2 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 15) ==== 137777+0+0 (secure 0 0 0) 0x7f7f9401a020 con 0x7f7f9c100800 2024-08-19T01:27:31.801 INFO:teuthology.orchestra.run.smithi117.stderr:2024-08-19T01:27:31.797+0000 7f7f8b7fe640 1 --2- 172.21.15.117:0/2810621282 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f7f840a2af0 0x7f7f840a4fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:27:31.801 INFO:teuthology.orchestra.run.smithi117.stderr:2024-08-19T01:27:31.798+0000 7f7f8b7fe640 1 -- 172.21.15.117:0/2810621282 <== mon.2 v2:172.21.15.117:3300/0 5 ==== osd_map(8..8 src has 1..8) ==== 1767+0+0 (secure 0 0 0) 0x7f7f940cd980 con 0x7f7f9c100800 2024-08-19T01:27:31.801 INFO:teuthology.orchestra.run.smithi117.stderr:2024-08-19T01:27:31.798+0000 7f7f99d9b640 1 --2- 172.21.15.117:0/2810621282 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f7f840a2af0 0x7f7f840a4fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:27:31.801 INFO:teuthology.orchestra.run.smithi117.stderr:2024-08-19T01:27:31.798+0000 7f7f99d9b640 1 --2- 172.21.15.117:0/2810621282 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f7f840a2af0 0x7f7f840a4fb0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f7f9c1046b0 tx=0x7f7f90000f40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:27:31.804 INFO:teuthology.orchestra.run.smithi117.stderr:2024-08-19T01:27:31.801+0000 7f7f8b7fe640 1 -- 172.21.15.117:0/2810621282 <== mon.2 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7f7f94021020 con 0x7f7f9c100800 2024-08-19T01:27:31.985 INFO:teuthology.orchestra.run.smithi117.stderr:2024-08-19T01:27:31.982+0000 7f7fa0dee640 1 -- 172.21.15.117:0/2810621282 --> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi117:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7f7f9c1a25f0 con 0x7f7f840a2af0 2024-08-19T01:27:32.603 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:27:32 smithi117 ceph-mon[22016]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-08-19T01:27:32.604 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:27:32 smithi117 ceph-mon[22016]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-08-19T01:27:32.604 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:27:32 smithi117 ceph-mon[22016]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-19T01:27:32.604 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:27:32 smithi117 ceph-mon[22016]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' 2024-08-19T01:27:32.604 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:27:32 smithi117 ceph-mon[22016]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' 2024-08-19T01:27:32.604 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:27:32 smithi117 ceph-mon[22016]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-08-19T01:27:32.604 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:27:32 smithi117 ceph-mon[22016]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' 2024-08-19T01:27:32.604 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:27:32 smithi117 ceph-mon[22016]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-19T01:27:32.604 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:27:32 smithi117 ceph-mon[22016]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-19T01:27:32.604 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:27:32 smithi117 ceph-mon[22016]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' 2024-08-19T01:27:32.658 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:27:32 smithi083 ceph-mon[17120]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-08-19T01:27:32.659 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:27:32 smithi083 ceph-mon[17120]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-08-19T01:27:32.659 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:27:32 smithi083 ceph-mon[17120]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-19T01:27:32.659 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:27:32 smithi083 ceph-mon[17120]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' 2024-08-19T01:27:32.659 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:27:32 smithi083 ceph-mon[17120]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' 2024-08-19T01:27:32.659 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:27:32 smithi083 ceph-mon[17120]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-08-19T01:27:32.660 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:27:32 smithi083 ceph-mon[17120]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' 2024-08-19T01:27:32.660 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:27:32 smithi083 ceph-mon[17120]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-19T01:27:32.660 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:27:32 smithi083 ceph-mon[17120]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-19T01:27:32.660 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:27:32 smithi083 ceph-mon[17120]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' 2024-08-19T01:27:32.716 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:27:32 smithi167 ceph-mon[21875]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-08-19T01:27:32.717 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:27:32 smithi167 ceph-mon[21875]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-08-19T01:27:32.717 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:27:32 smithi167 ceph-mon[21875]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-19T01:27:32.717 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:27:32 smithi167 ceph-mon[21875]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' 2024-08-19T01:27:32.717 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:27:32 smithi167 ceph-mon[21875]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' 2024-08-19T01:27:32.717 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:27:32 smithi167 ceph-mon[21875]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-08-19T01:27:32.718 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:27:32 smithi167 ceph-mon[21875]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' 2024-08-19T01:27:32.718 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:27:32 smithi167 ceph-mon[21875]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-19T01:27:32.718 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:27:32 smithi167 ceph-mon[21875]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-19T01:27:32.718 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:27:32 smithi167 ceph-mon[21875]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' 2024-08-19T01:27:33.655 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:27:33 smithi083 ceph-mon[17120]: from='client.24149 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi117:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-08-19T01:27:33.655 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:27:33 smithi083 ceph-mon[17120]: pgmap v42: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-19T01:27:33.655 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:27:33 smithi083 ceph-mon[17120]: Detected new or changed devices on smithi083 2024-08-19T01:27:33.655 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:27:33 smithi083 ceph-mon[17120]: Adjusting osd_memory_target on smithi083 to 17089M 2024-08-19T01:27:33.655 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:27:33 smithi083 ceph-mon[17120]: osdmap e9: 1 total, 1 up, 1 in 2024-08-19T01:27:33.668 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:27:33 smithi117 ceph-mon[22016]: from='client.24149 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi117:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-08-19T01:27:33.668 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:27:33 smithi117 ceph-mon[22016]: pgmap v42: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-19T01:27:33.668 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:27:33 smithi117 ceph-mon[22016]: Detected new or changed devices on smithi083 2024-08-19T01:27:33.668 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:27:33 smithi117 ceph-mon[22016]: Adjusting osd_memory_target on smithi083 to 17089M 2024-08-19T01:27:33.668 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:27:33 smithi117 ceph-mon[22016]: osdmap e9: 1 total, 1 up, 1 in 2024-08-19T01:27:33.716 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:27:33 smithi167 ceph-mon[21875]: from='client.24149 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi117:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-08-19T01:27:33.716 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:27:33 smithi167 ceph-mon[21875]: pgmap v42: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-19T01:27:33.716 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:27:33 smithi167 ceph-mon[21875]: Detected new or changed devices on smithi083 2024-08-19T01:27:33.717 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:27:33 smithi167 ceph-mon[21875]: Adjusting osd_memory_target on smithi083 to 17089M 2024-08-19T01:27:33.717 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:27:33 smithi167 ceph-mon[21875]: osdmap e9: 1 total, 1 up, 1 in 2024-08-19T01:27:34.703 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:27:34 smithi117 ceph-mon[22016]: from='client.? 172.21.15.117:0/1825086326' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1977570b-b4f5-44fd-bb1e-3a31318be649"}]: dispatch 2024-08-19T01:27:34.703 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:27:34 smithi117 ceph-mon[22016]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1977570b-b4f5-44fd-bb1e-3a31318be649"}]: dispatch 2024-08-19T01:27:34.704 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:27:34 smithi117 ceph-mon[22016]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "1977570b-b4f5-44fd-bb1e-3a31318be649"}]': finished 2024-08-19T01:27:34.704 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:27:34 smithi117 ceph-mon[22016]: osdmap e10: 2 total, 1 up, 2 in 2024-08-19T01:27:34.704 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:27:34 smithi117 ceph-mon[22016]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-08-19T01:27:34.718 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:27:34 smithi167 ceph-mon[21875]: from='client.? 172.21.15.117:0/1825086326' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1977570b-b4f5-44fd-bb1e-3a31318be649"}]: dispatch 2024-08-19T01:27:34.718 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:27:34 smithi167 ceph-mon[21875]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1977570b-b4f5-44fd-bb1e-3a31318be649"}]: dispatch 2024-08-19T01:27:34.718 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:27:34 smithi167 ceph-mon[21875]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "1977570b-b4f5-44fd-bb1e-3a31318be649"}]': finished 2024-08-19T01:27:34.718 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:27:34 smithi167 ceph-mon[21875]: osdmap e10: 2 total, 1 up, 2 in 2024-08-19T01:27:34.718 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:27:34 smithi167 ceph-mon[21875]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-08-19T01:27:34.905 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:27:34 smithi083 ceph-mon[17120]: from='client.? 172.21.15.117:0/1825086326' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1977570b-b4f5-44fd-bb1e-3a31318be649"}]: dispatch 2024-08-19T01:27:34.905 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:27:34 smithi083 ceph-mon[17120]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1977570b-b4f5-44fd-bb1e-3a31318be649"}]: dispatch 2024-08-19T01:27:34.905 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:27:34 smithi083 ceph-mon[17120]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "1977570b-b4f5-44fd-bb1e-3a31318be649"}]': finished 2024-08-19T01:27:34.905 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:27:34 smithi083 ceph-mon[17120]: osdmap e10: 2 total, 1 up, 2 in 2024-08-19T01:27:34.905 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:27:34 smithi083 ceph-mon[17120]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-08-19T01:27:35.703 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:27:35 smithi117 ceph-mon[22016]: pgmap v45: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-08-19T01:27:35.703 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:27:35 smithi117 ceph-mon[22016]: from='client.? 172.21.15.117:0/4117140434' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-08-19T01:27:35.716 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:27:35 smithi167 ceph-mon[21875]: pgmap v45: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-08-19T01:27:35.716 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:27:35 smithi167 ceph-mon[21875]: from='client.? 172.21.15.117:0/4117140434' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-08-19T01:27:35.905 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:27:35 smithi083 ceph-mon[17120]: pgmap v45: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-08-19T01:27:35.905 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:27:35 smithi083 ceph-mon[17120]: from='client.? 172.21.15.117:0/4117140434' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-08-19T01:27:37.703 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:27:37 smithi117 ceph-mon[22016]: pgmap v46: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-08-19T01:27:37.716 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:27:37 smithi167 ceph-mon[21875]: pgmap v46: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-08-19T01:27:37.905 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:27:37 smithi083 ceph-mon[17120]: pgmap v46: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-08-19T01:27:39.662 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:27:39 smithi117 ceph-mon[22016]: pgmap v47: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-08-19T01:27:39.716 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:27:39 smithi167 ceph-mon[21875]: pgmap v47: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-08-19T01:27:39.905 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:27:39 smithi083 ceph-mon[17120]: pgmap v47: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-08-19T01:27:41.703 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:27:41 smithi117 ceph-mon[22016]: pgmap v48: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-08-19T01:27:41.703 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:27:41 smithi117 ceph-mon[22016]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-08-19T01:27:41.703 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:27:41 smithi117 ceph-mon[22016]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-19T01:27:41.716 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:27:41 smithi167 ceph-mon[21875]: pgmap v48: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-08-19T01:27:41.716 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:27:41 smithi167 ceph-mon[21875]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-08-19T01:27:41.717 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:27:41 smithi167 ceph-mon[21875]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-19T01:27:41.905 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:27:41 smithi083 ceph-mon[17120]: pgmap v48: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-08-19T01:27:41.905 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:27:41 smithi083 ceph-mon[17120]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-08-19T01:27:41.905 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:27:41 smithi083 ceph-mon[17120]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-19T01:27:42.453 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:27:42 smithi117 ceph-mon[22016]: Deploying daemon osd.1 on smithi117 2024-08-19T01:27:42.716 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:27:42 smithi167 ceph-mon[21875]: Deploying daemon osd.1 on smithi117 2024-08-19T01:27:42.905 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:27:42 smithi083 ceph-mon[17120]: Deploying daemon osd.1 on smithi117 2024-08-19T01:27:43.594 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:27:43 smithi117 ceph-mon[22016]: pgmap v49: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-08-19T01:27:43.716 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:27:43 smithi167 ceph-mon[21875]: pgmap v49: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-08-19T01:27:43.905 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:27:43 smithi083 ceph-mon[17120]: pgmap v49: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-08-19T01:27:45.549 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:27:45 smithi117 ceph-mon[22016]: pgmap v50: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-08-19T01:27:45.708 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:27:45 smithi083 ceph-mon[17120]: pgmap v50: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-08-19T01:27:45.716 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:27:45 smithi167 ceph-mon[21875]: pgmap v50: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-08-19T01:27:46.703 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:27:46 smithi117 ceph-mon[22016]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-19T01:27:46.703 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:27:46 smithi117 ceph-mon[22016]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' 2024-08-19T01:27:46.703 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:27:46 smithi117 ceph-mon[22016]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' 2024-08-19T01:27:46.716 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:27:46 smithi167 ceph-mon[21875]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-19T01:27:46.716 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:27:46 smithi167 ceph-mon[21875]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' 2024-08-19T01:27:46.716 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:27:46 smithi167 ceph-mon[21875]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' 2024-08-19T01:27:46.736 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:27:46 smithi083 ceph-mon[17120]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-19T01:27:46.736 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:27:46 smithi083 ceph-mon[17120]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' 2024-08-19T01:27:46.736 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:27:46 smithi083 ceph-mon[17120]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' 2024-08-19T01:27:47.453 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:27:47 smithi117 ceph-mon[22016]: pgmap v51: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-08-19T01:27:47.655 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:27:47 smithi083 ceph-mon[17120]: pgmap v51: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-08-19T01:27:47.716 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:27:47 smithi167 ceph-mon[21875]: pgmap v51: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-08-19T01:27:48.796 INFO:teuthology.orchestra.run.smithi117.stderr:2024-08-19T01:27:48.793+0000 7f7f8b7fe640 1 -- 172.21.15.117:0/2810621282 <== mgr.14150 v2:172.21.15.83:6800/683740916 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f7f9c1a25f0 con 0x7f7f840a2af0 2024-08-19T01:27:48.796 INFO:teuthology.orchestra.run.smithi117.stdout:Created osd(s) 1 on host 'smithi117' 2024-08-19T01:27:48.797 INFO:teuthology.orchestra.run.smithi117.stderr:2024-08-19T01:27:48.796+0000 7f7fa0dee640 1 -- 172.21.15.117:0/2810621282 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f7f840a2af0 msgr2=0x7f7f840a4fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:27:48.798 INFO:teuthology.orchestra.run.smithi117.stderr:2024-08-19T01:27:48.796+0000 7f7fa0dee640 1 --2- 172.21.15.117:0/2810621282 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f7f840a2af0 0x7f7f840a4fb0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f7f9c1046b0 tx=0x7f7f90000f40 comp rx=0 tx=0).stop 2024-08-19T01:27:48.798 INFO:teuthology.orchestra.run.smithi117.stderr:2024-08-19T01:27:48.796+0000 7f7fa0dee640 1 -- 172.21.15.117:0/2810621282 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7f9c100800 msgr2=0x7f7f9c104930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:27:48.798 INFO:teuthology.orchestra.run.smithi117.stderr:2024-08-19T01:27:48.796+0000 7f7fa0dee640 1 --2- 172.21.15.117:0/2810621282 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7f9c100800 0x7f7f9c104930 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f7f9400d950 tx=0x7f7f9400de10 comp rx=0 tx=0).stop 2024-08-19T01:27:48.798 INFO:teuthology.orchestra.run.smithi117.stderr:2024-08-19T01:27:48.796+0000 7f7fa0dee640 1 -- 172.21.15.117:0/2810621282 shutdown_connections 2024-08-19T01:27:48.798 INFO:teuthology.orchestra.run.smithi117.stderr:2024-08-19T01:27:48.796+0000 7f7fa0dee640 1 --2- 172.21.15.117:0/2810621282 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f7f840a2af0 0x7f7f840a4fb0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:27:48.798 INFO:teuthology.orchestra.run.smithi117.stderr:2024-08-19T01:27:48.796+0000 7f7fa0dee640 1 --2- 172.21.15.117:0/2810621282 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7f9c1023c0 0x7f7f9c103510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:27:48.798 INFO:teuthology.orchestra.run.smithi117.stderr:2024-08-19T01:27:48.796+0000 7f7fa0dee640 1 --2- 172.21.15.117:0/2810621282 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7f9c101a00 0x7f7f9c102fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:27:48.798 INFO:teuthology.orchestra.run.smithi117.stderr:2024-08-19T01:27:48.796+0000 7f7fa0dee640 1 --2- 172.21.15.117:0/2810621282 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7f9c100800 0x7f7f9c104930 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:27:48.798 INFO:teuthology.orchestra.run.smithi117.stderr:2024-08-19T01:27:48.796+0000 7f7fa0dee640 1 -- 172.21.15.117:0/2810621282 >> 172.21.15.117:0/2810621282 conn(0x7f7f9c0fbfb0 msgr2=0x7f7f9c0fe070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:27:48.798 INFO:teuthology.orchestra.run.smithi117.stderr:2024-08-19T01:27:48.796+0000 7f7fa0dee640 1 -- 172.21.15.117:0/2810621282 shutdown_connections 2024-08-19T01:27:48.798 INFO:teuthology.orchestra.run.smithi117.stderr:2024-08-19T01:27:48.796+0000 7f7fa0dee640 1 -- 172.21.15.117:0/2810621282 wait complete. 2024-08-19T01:27:49.100 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:27:48 smithi117 ceph-mon[22016]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' 2024-08-19T01:27:49.100 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:27:48 smithi117 ceph-mon[22016]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' 2024-08-19T01:27:49.100 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:27:48 smithi117 ceph-mon[22016]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-19T01:27:49.100 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:27:48 smithi117 ceph-mon[22016]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-19T01:27:49.100 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:27:48 smithi117 ceph-mon[22016]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' 2024-08-19T01:27:49.100 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:27:48 smithi117 ceph-mon[22016]: pgmap v52: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-08-19T01:27:49.155 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:27:48 smithi083 ceph-mon[17120]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' 2024-08-19T01:27:49.155 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:27:48 smithi083 ceph-mon[17120]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' 2024-08-19T01:27:49.155 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:27:48 smithi083 ceph-mon[17120]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-19T01:27:49.155 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:27:48 smithi083 ceph-mon[17120]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-19T01:27:49.155 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:27:48 smithi083 ceph-mon[17120]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' 2024-08-19T01:27:49.155 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:27:48 smithi083 ceph-mon[17120]: pgmap v52: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-08-19T01:27:49.216 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:27:48 smithi167 ceph-mon[21875]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' 2024-08-19T01:27:49.217 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:27:48 smithi167 ceph-mon[21875]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' 2024-08-19T01:27:49.217 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:27:48 smithi167 ceph-mon[21875]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-19T01:27:49.217 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:27:48 smithi167 ceph-mon[21875]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-19T01:27:49.217 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:27:48 smithi167 ceph-mon[21875]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' 2024-08-19T01:27:49.217 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:27:48 smithi167 ceph-mon[21875]: pgmap v52: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-08-19T01:27:49.408 DEBUG:teuthology.orchestra.run.smithi117:osd.1> sudo journalctl -f -n 0 -u ceph-bad8780e-5dc9-11ef-bccf-c7b262605968@osd.1.service 2024-08-19T01:27:49.411 INFO:tasks.cephadm:Deploying osd.2 on smithi167 with /dev/vg_nvme/lv_4... 2024-08-19T01:27:49.411 DEBUG:teuthology.orchestra.run.smithi167:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833 ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid bad8780e-5dc9-11ef-bccf-c7b262605968 -- lvm zap /dev/vg_nvme/lv_4 2024-08-19T01:27:49.641 INFO:journalctl@ceph.osd.1.smithi117.stdout:Aug 19 01:27:49 smithi117 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-1[26443]: 2024-08-19T01:27:49.492+0000 7f6d7c918640 -1 osd.1 0 log_to_monitors true 2024-08-19T01:27:49.666 INFO:teuthology.orchestra.run.smithi167.stderr:Inferring config /var/lib/ceph/bad8780e-5dc9-11ef-bccf-c7b262605968/mon.c/config 2024-08-19T01:27:49.953 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:27:49 smithi117 ceph-mon[22016]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-19T01:27:49.953 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:27:49 smithi117 ceph-mon[22016]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' 2024-08-19T01:27:49.953 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:27:49 smithi117 ceph-mon[22016]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' 2024-08-19T01:27:49.953 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:27:49 smithi117 ceph-mon[22016]: from='osd.1 [v2:172.21.15.117:6800/1925419802,v1:172.21.15.117:6801/1925419802]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-08-19T01:27:49.953 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:27:49 smithi117 ceph-mon[22016]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-08-19T01:27:50.085 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:27:49 smithi167 ceph-mon[21875]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-19T01:27:50.085 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:27:49 smithi167 ceph-mon[21875]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' 2024-08-19T01:27:50.085 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:27:49 smithi167 ceph-mon[21875]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' 2024-08-19T01:27:50.085 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:27:49 smithi167 ceph-mon[21875]: from='osd.1 [v2:172.21.15.117:6800/1925419802,v1:172.21.15.117:6801/1925419802]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-08-19T01:27:50.085 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:27:49 smithi167 ceph-mon[21875]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-08-19T01:27:50.155 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:27:49 smithi083 ceph-mon[17120]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-19T01:27:50.155 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:27:49 smithi083 ceph-mon[17120]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' 2024-08-19T01:27:50.155 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:27:49 smithi083 ceph-mon[17120]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' 2024-08-19T01:27:50.155 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:27:49 smithi083 ceph-mon[17120]: from='osd.1 [v2:172.21.15.117:6800/1925419802,v1:172.21.15.117:6801/1925419802]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-08-19T01:27:50.155 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:27:49 smithi083 ceph-mon[17120]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-08-19T01:27:50.974 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:27:50 smithi167 ceph-mon[21875]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-08-19T01:27:50.975 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:27:50 smithi167 ceph-mon[21875]: from='osd.1 [v2:172.21.15.117:6800/1925419802,v1:172.21.15.117:6801/1925419802]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi117", "root=default"]}]: dispatch 2024-08-19T01:27:50.975 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:27:50 smithi167 ceph-mon[21875]: osdmap e11: 2 total, 1 up, 2 in 2024-08-19T01:27:50.975 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:27:50 smithi167 ceph-mon[21875]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-08-19T01:27:50.975 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:27:50 smithi167 ceph-mon[21875]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi117", "root=default"]}]: dispatch 2024-08-19T01:27:50.975 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:27:50 smithi167 ceph-mon[21875]: pgmap v54: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-08-19T01:27:51.155 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:27:50 smithi083 ceph-mon[17120]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-08-19T01:27:51.155 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:27:50 smithi083 ceph-mon[17120]: from='osd.1 [v2:172.21.15.117:6800/1925419802,v1:172.21.15.117:6801/1925419802]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi117", "root=default"]}]: dispatch 2024-08-19T01:27:51.155 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:27:50 smithi083 ceph-mon[17120]: osdmap e11: 2 total, 1 up, 2 in 2024-08-19T01:27:51.155 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:27:50 smithi083 ceph-mon[17120]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-08-19T01:27:51.155 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:27:50 smithi083 ceph-mon[17120]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi117", "root=default"]}]: dispatch 2024-08-19T01:27:51.155 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:27:50 smithi083 ceph-mon[17120]: pgmap v54: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-08-19T01:27:51.203 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:27:50 smithi117 ceph-mon[22016]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-08-19T01:27:51.203 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:27:50 smithi117 ceph-mon[22016]: from='osd.1 [v2:172.21.15.117:6800/1925419802,v1:172.21.15.117:6801/1925419802]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi117", "root=default"]}]: dispatch 2024-08-19T01:27:51.203 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:27:50 smithi117 ceph-mon[22016]: osdmap e11: 2 total, 1 up, 2 in 2024-08-19T01:27:51.203 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:27:50 smithi117 ceph-mon[22016]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-08-19T01:27:51.203 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:27:50 smithi117 ceph-mon[22016]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi117", "root=default"]}]: dispatch 2024-08-19T01:27:51.203 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:27:50 smithi117 ceph-mon[22016]: pgmap v54: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-08-19T01:27:51.204 INFO:journalctl@ceph.osd.1.smithi117.stdout:Aug 19 01:27:50 smithi117 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-1[26443]: 2024-08-19T01:27:50.799+0000 7f6d78078640 -1 osd.1 0 waiting for initial osdmap 2024-08-19T01:27:51.204 INFO:journalctl@ceph.osd.1.smithi117.stdout:Aug 19 01:27:50 smithi117 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-1[26443]: 2024-08-19T01:27:50.803+0000 7f6d746b2640 -1 osd.1 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-08-19T01:27:51.915 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-08-19T01:27:51.943 DEBUG:teuthology.orchestra.run.smithi167:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid bad8780e-5dc9-11ef-bccf-c7b262605968 -- ceph orch daemon add osd smithi167:vg_nvme/lv_4 2024-08-19T01:27:51.957 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:27:51 smithi117 ceph-mon[22016]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi117", "root=default"]}]': finished 2024-08-19T01:27:51.957 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:27:51 smithi117 ceph-mon[22016]: osdmap e12: 2 total, 1 up, 2 in 2024-08-19T01:27:51.957 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:27:51 smithi117 ceph-mon[22016]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-08-19T01:27:51.958 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:27:51 smithi117 ceph-mon[22016]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-08-19T01:27:52.134 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:27:51 smithi167 ceph-mon[21875]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi117", "root=default"]}]': finished 2024-08-19T01:27:52.134 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:27:51 smithi167 ceph-mon[21875]: osdmap e12: 2 total, 1 up, 2 in 2024-08-19T01:27:52.134 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:27:51 smithi167 ceph-mon[21875]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-08-19T01:27:52.134 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:27:51 smithi167 ceph-mon[21875]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-08-19T01:27:52.155 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:27:51 smithi083 ceph-mon[17120]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi117", "root=default"]}]': finished 2024-08-19T01:27:52.155 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:27:51 smithi083 ceph-mon[17120]: osdmap e12: 2 total, 1 up, 2 in 2024-08-19T01:27:52.155 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:27:51 smithi083 ceph-mon[17120]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-08-19T01:27:52.155 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:27:51 smithi083 ceph-mon[17120]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-08-19T01:27:52.195 INFO:teuthology.orchestra.run.smithi167.stderr:Inferring config /var/lib/ceph/bad8780e-5dc9-11ef-bccf-c7b262605968/mon.c/config 2024-08-19T01:27:52.661 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-19T01:27:52.656+0000 7f3c27b7f640 1 -- 172.21.15.167:0/1331135477 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3c20105480 msgr2=0x7f3c20105930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:27:52.661 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-19T01:27:52.656+0000 7f3c27b7f640 1 --2- 172.21.15.167:0/1331135477 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3c20105480 0x7f3c20105930 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f3c18009ca0 tx=0x7f3c1802a8c0 comp rx=0 tx=0).stop 2024-08-19T01:27:52.661 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-19T01:27:52.656+0000 7f3c27b7f640 1 -- 172.21.15.167:0/1331135477 shutdown_connections 2024-08-19T01:27:52.662 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-19T01:27:52.656+0000 7f3c27b7f640 1 --2- 172.21.15.167:0/1331135477 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3c20105480 0x7f3c20105930 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:27:52.662 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-19T01:27:52.656+0000 7f3c27b7f640 1 --2- 172.21.15.167:0/1331135477 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3c20102ad0 0x7f3c20104f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:27:52.662 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-19T01:27:52.656+0000 7f3c27b7f640 1 --2- 172.21.15.167:0/1331135477 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3c20100180 0x7f3c20102590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:27:52.662 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-19T01:27:52.656+0000 7f3c27b7f640 1 -- 172.21.15.167:0/1331135477 >> 172.21.15.167:0/1331135477 conn(0x7f3c200fbfb0 msgr2=0x7f3c200fe3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:27:52.662 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-19T01:27:52.656+0000 7f3c27b7f640 1 -- 172.21.15.167:0/1331135477 shutdown_connections 2024-08-19T01:27:52.662 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-19T01:27:52.656+0000 7f3c27b7f640 1 -- 172.21.15.167:0/1331135477 wait complete. 2024-08-19T01:27:52.662 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-19T01:27:52.657+0000 7f3c27b7f640 1 Processor -- start 2024-08-19T01:27:52.662 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-19T01:27:52.657+0000 7f3c27b7f640 1 -- start start 2024-08-19T01:27:52.662 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-19T01:27:52.657+0000 7f3c27b7f640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3c20100180 0x7f3c2019e1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:27:52.662 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-19T01:27:52.657+0000 7f3c27b7f640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3c20102ad0 0x7f3c2019e6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:27:52.662 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-19T01:27:52.657+0000 7f3c27b7f640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3c20105480 0x7f3c201a5770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:27:52.662 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-19T01:27:52.657+0000 7f3c27b7f640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f3c20113b60 con 0x7f3c20102ad0 2024-08-19T01:27:52.663 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-19T01:27:52.657+0000 7f3c27b7f640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f3c201139e0 con 0x7f3c20105480 2024-08-19T01:27:52.663 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-19T01:27:52.657+0000 7f3c27b7f640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f3c20113ce0 con 0x7f3c20100180 2024-08-19T01:27:52.663 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-19T01:27:52.657+0000 7f3c2591b640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3c20100180 0x7f3c2019e1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:27:52.663 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-19T01:27:52.657+0000 7f3c2591b640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3c20100180 0x7f3c2019e1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:48856/0 (socket says 172.21.15.167:48856) 2024-08-19T01:27:52.663 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-19T01:27:52.657+0000 7f3c2591b640 1 -- 172.21.15.167:0/918631276 learned_addr learned my addr 172.21.15.167:0/918631276 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-08-19T01:27:52.663 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-19T01:27:52.657+0000 7f3c2611c640 1 --2- 172.21.15.167:0/918631276 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3c20105480 0x7f3c201a5770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:27:52.663 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-19T01:27:52.657+0000 7f3c2511a640 1 --2- 172.21.15.167:0/918631276 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3c20102ad0 0x7f3c2019e6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:27:52.663 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-19T01:27:52.657+0000 7f3c2591b640 1 -- 172.21.15.167:0/918631276 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3c20105480 msgr2=0x7f3c201a5770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:27:52.663 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-19T01:27:52.657+0000 7f3c2591b640 1 --2- 172.21.15.167:0/918631276 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3c20105480 0x7f3c201a5770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:27:52.663 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-19T01:27:52.657+0000 7f3c2591b640 1 -- 172.21.15.167:0/918631276 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3c20102ad0 msgr2=0x7f3c2019e6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:27:52.663 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-19T01:27:52.657+0000 7f3c2591b640 1 --2- 172.21.15.167:0/918631276 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3c20102ad0 0x7f3c2019e6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:27:52.663 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-19T01:27:52.657+0000 7f3c2591b640 1 -- 172.21.15.167:0/918631276 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3c201a5de0 con 0x7f3c20100180 2024-08-19T01:27:52.664 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-19T01:27:52.657+0000 7f3c2611c640 1 --2- 172.21.15.167:0/918631276 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3c20105480 0x7f3c201a5770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-19T01:27:52.664 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-19T01:27:52.658+0000 7f3c2591b640 1 --2- 172.21.15.167:0/918631276 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3c20100180 0x7f3c2019e1b0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f3c1c00c890 tx=0x7f3c1c00cd50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:27:52.664 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-19T01:27:52.658+0000 7f3c16ffd640 1 -- 172.21.15.167:0/918631276 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c1c004220 con 0x7f3c20100180 2024-08-19T01:27:52.664 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-19T01:27:52.658+0000 7f3c27b7f640 1 -- 172.21.15.167:0/918631276 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3c201a60d0 con 0x7f3c20100180 2024-08-19T01:27:52.664 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-19T01:27:52.658+0000 7f3c16ffd640 1 -- 172.21.15.167:0/918631276 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3c1c0043c0 con 0x7f3c20100180 2024-08-19T01:27:52.664 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-19T01:27:52.658+0000 7f3c16ffd640 1 -- 172.21.15.167:0/918631276 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c1c004a00 con 0x7f3c20100180 2024-08-19T01:27:52.664 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-19T01:27:52.658+0000 7f3c27b7f640 1 -- 172.21.15.167:0/918631276 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3c201a6610 con 0x7f3c20100180 2024-08-19T01:27:52.664 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-19T01:27:52.659+0000 7f3c27b7f640 1 -- 172.21.15.167:0/918631276 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3c200fdaa0 con 0x7f3c20100180 2024-08-19T01:27:52.665 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-19T01:27:52.661+0000 7f3c16ffd640 1 -- 172.21.15.167:0/918631276 <== mon.1 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 15) ==== 137777+0+0 (secure 0 0 0) 0x7f3c1c0079b0 con 0x7f3c20100180 2024-08-19T01:27:52.665 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-19T01:27:52.661+0000 7f3c16ffd640 1 --2- 172.21.15.167:0/918631276 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f3c0c0a2b20 0x7f3c0c0a4fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:27:52.665 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-19T01:27:52.662+0000 7f3c16ffd640 1 -- 172.21.15.167:0/918631276 <== mon.1 v2:172.21.15.167:3300/0 5 ==== osd_map(13..13 src has 1..13) ==== 2309+0+0 (secure 0 0 0) 0x7f3c1c0cda70 con 0x7f3c20100180 2024-08-19T01:27:52.665 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-19T01:27:52.662+0000 7f3c2511a640 1 --2- 172.21.15.167:0/918631276 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f3c0c0a2b20 0x7f3c0c0a4fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:27:52.665 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-19T01:27:52.663+0000 7f3c2511a640 1 --2- 172.21.15.167:0/918631276 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f3c0c0a2b20 0x7f3c0c0a4fe0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f3c2019f6d0 tx=0x7f3c1000a4c0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:27:52.667 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-19T01:27:52.664+0000 7f3c16ffd640 1 -- 172.21.15.167:0/918631276 <== mon.1 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7f3c1c014030 con 0x7f3c20100180 2024-08-19T01:27:52.845 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-19T01:27:52.843+0000 7f3c27b7f640 1 -- 172.21.15.167:0/918631276 --> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi167:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7f3c20062360 con 0x7f3c0c0a2b20 2024-08-19T01:27:52.966 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:27:52 smithi167 ceph-mon[21875]: purged_snaps scrub starts 2024-08-19T01:27:52.966 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:27:52 smithi167 ceph-mon[21875]: purged_snaps scrub ok 2024-08-19T01:27:52.966 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:27:52 smithi167 ceph-mon[21875]: osd.1 [v2:172.21.15.117:6800/1925419802,v1:172.21.15.117:6801/1925419802] boot 2024-08-19T01:27:52.967 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:27:52 smithi167 ceph-mon[21875]: osdmap e13: 2 total, 2 up, 2 in 2024-08-19T01:27:52.967 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:27:52 smithi167 ceph-mon[21875]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-08-19T01:27:52.967 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:27:52 smithi167 ceph-mon[21875]: Detected new or changed devices on smithi117 2024-08-19T01:27:52.967 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:27:52 smithi167 ceph-mon[21875]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' 2024-08-19T01:27:52.967 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:27:52 smithi167 ceph-mon[21875]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' 2024-08-19T01:27:52.967 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:27:52 smithi167 ceph-mon[21875]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-08-19T01:27:52.967 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:27:52 smithi167 ceph-mon[21875]: Adjusting osd_memory_target on smithi117 to 17088M 2024-08-19T01:27:52.967 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:27:52 smithi167 ceph-mon[21875]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' 2024-08-19T01:27:52.967 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:27:52 smithi167 ceph-mon[21875]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-19T01:27:52.967 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:27:52 smithi167 ceph-mon[21875]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-19T01:27:52.967 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:27:52 smithi167 ceph-mon[21875]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' 2024-08-19T01:27:52.967 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:27:52 smithi167 ceph-mon[21875]: pgmap v57: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-08-19T01:27:53.155 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:27:52 smithi083 ceph-mon[17120]: purged_snaps scrub starts 2024-08-19T01:27:53.155 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:27:52 smithi083 ceph-mon[17120]: purged_snaps scrub ok 2024-08-19T01:27:53.155 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:27:52 smithi083 ceph-mon[17120]: osd.1 [v2:172.21.15.117:6800/1925419802,v1:172.21.15.117:6801/1925419802] boot 2024-08-19T01:27:53.155 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:27:52 smithi083 ceph-mon[17120]: osdmap e13: 2 total, 2 up, 2 in 2024-08-19T01:27:53.155 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:27:52 smithi083 ceph-mon[17120]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-08-19T01:27:53.155 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:27:52 smithi083 ceph-mon[17120]: Detected new or changed devices on smithi117 2024-08-19T01:27:53.156 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:27:52 smithi083 ceph-mon[17120]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' 2024-08-19T01:27:53.156 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:27:52 smithi083 ceph-mon[17120]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' 2024-08-19T01:27:53.156 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:27:52 smithi083 ceph-mon[17120]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-08-19T01:27:53.156 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:27:52 smithi083 ceph-mon[17120]: Adjusting osd_memory_target on smithi117 to 17088M 2024-08-19T01:27:53.156 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:27:52 smithi083 ceph-mon[17120]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' 2024-08-19T01:27:53.156 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:27:52 smithi083 ceph-mon[17120]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-19T01:27:53.156 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:27:52 smithi083 ceph-mon[17120]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-19T01:27:53.156 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:27:52 smithi083 ceph-mon[17120]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' 2024-08-19T01:27:53.156 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:27:52 smithi083 ceph-mon[17120]: pgmap v57: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-08-19T01:27:53.203 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:27:52 smithi117 ceph-mon[22016]: purged_snaps scrub starts 2024-08-19T01:27:53.203 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:27:52 smithi117 ceph-mon[22016]: purged_snaps scrub ok 2024-08-19T01:27:53.203 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:27:52 smithi117 ceph-mon[22016]: osd.1 [v2:172.21.15.117:6800/1925419802,v1:172.21.15.117:6801/1925419802] boot 2024-08-19T01:27:53.203 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:27:52 smithi117 ceph-mon[22016]: osdmap e13: 2 total, 2 up, 2 in 2024-08-19T01:27:53.203 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:27:52 smithi117 ceph-mon[22016]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-08-19T01:27:53.203 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:27:52 smithi117 ceph-mon[22016]: Detected new or changed devices on smithi117 2024-08-19T01:27:53.203 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:27:52 smithi117 ceph-mon[22016]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' 2024-08-19T01:27:53.203 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:27:52 smithi117 ceph-mon[22016]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' 2024-08-19T01:27:53.203 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:27:52 smithi117 ceph-mon[22016]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-08-19T01:27:53.203 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:27:52 smithi117 ceph-mon[22016]: Adjusting osd_memory_target on smithi117 to 17088M 2024-08-19T01:27:53.204 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:27:52 smithi117 ceph-mon[22016]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' 2024-08-19T01:27:53.204 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:27:52 smithi117 ceph-mon[22016]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-19T01:27:53.204 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:27:52 smithi117 ceph-mon[22016]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-19T01:27:53.204 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:27:52 smithi117 ceph-mon[22016]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' 2024-08-19T01:27:53.204 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:27:52 smithi117 ceph-mon[22016]: pgmap v57: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-08-19T01:27:53.966 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:27:53 smithi167 ceph-mon[21875]: from='client.24160 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi167:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-08-19T01:27:53.967 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:27:53 smithi167 ceph-mon[21875]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-08-19T01:27:53.967 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:27:53 smithi167 ceph-mon[21875]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-08-19T01:27:53.967 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:27:53 smithi167 ceph-mon[21875]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-19T01:27:53.967 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:27:53 smithi167 ceph-mon[21875]: osdmap e14: 2 total, 2 up, 2 in 2024-08-19T01:27:54.155 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:27:53 smithi083 ceph-mon[17120]: from='client.24160 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi167:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-08-19T01:27:54.155 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:27:53 smithi083 ceph-mon[17120]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-08-19T01:27:54.155 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:27:53 smithi083 ceph-mon[17120]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-08-19T01:27:54.155 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:27:53 smithi083 ceph-mon[17120]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-19T01:27:54.155 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:27:53 smithi083 ceph-mon[17120]: osdmap e14: 2 total, 2 up, 2 in 2024-08-19T01:27:54.203 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:27:53 smithi117 ceph-mon[22016]: from='client.24160 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi167:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-08-19T01:27:54.203 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:27:53 smithi117 ceph-mon[22016]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-08-19T01:27:54.203 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:27:53 smithi117 ceph-mon[22016]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-08-19T01:27:54.203 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:27:53 smithi117 ceph-mon[22016]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-19T01:27:54.203 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:27:53 smithi117 ceph-mon[22016]: osdmap e14: 2 total, 2 up, 2 in 2024-08-19T01:27:55.155 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:27:54 smithi083 ceph-mon[17120]: pgmap v59: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-08-19T01:27:55.203 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:27:54 smithi117 ceph-mon[22016]: pgmap v59: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-08-19T01:27:55.216 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:27:54 smithi167 ceph-mon[21875]: pgmap v59: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-08-19T01:27:55.967 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:27:55 smithi167 ceph-mon[21875]: from='client.? 172.21.15.167:0/59341641' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a75278d1-161f-41d5-a92f-058d1077bf2a"}]: dispatch 2024-08-19T01:27:55.967 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:27:55 smithi167 ceph-mon[21875]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a75278d1-161f-41d5-a92f-058d1077bf2a"}]: dispatch 2024-08-19T01:27:55.967 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:27:55 smithi167 ceph-mon[21875]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "a75278d1-161f-41d5-a92f-058d1077bf2a"}]': finished 2024-08-19T01:27:55.967 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:27:55 smithi167 ceph-mon[21875]: osdmap e15: 3 total, 2 up, 3 in 2024-08-19T01:27:55.967 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:27:55 smithi167 ceph-mon[21875]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-08-19T01:27:55.967 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:27:55 smithi167 ceph-mon[21875]: from='client.? 172.21.15.167:0/1465449369' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-08-19T01:27:56.155 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:27:55 smithi083 ceph-mon[17120]: from='client.? 172.21.15.167:0/59341641' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a75278d1-161f-41d5-a92f-058d1077bf2a"}]: dispatch 2024-08-19T01:27:56.155 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:27:55 smithi083 ceph-mon[17120]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a75278d1-161f-41d5-a92f-058d1077bf2a"}]: dispatch 2024-08-19T01:27:56.156 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:27:55 smithi083 ceph-mon[17120]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "a75278d1-161f-41d5-a92f-058d1077bf2a"}]': finished 2024-08-19T01:27:56.156 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:27:55 smithi083 ceph-mon[17120]: osdmap e15: 3 total, 2 up, 3 in 2024-08-19T01:27:56.156 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:27:55 smithi083 ceph-mon[17120]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-08-19T01:27:56.156 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:27:55 smithi083 ceph-mon[17120]: from='client.? 172.21.15.167:0/1465449369' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-08-19T01:27:56.203 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:27:55 smithi117 ceph-mon[22016]: from='client.? 172.21.15.167:0/59341641' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a75278d1-161f-41d5-a92f-058d1077bf2a"}]: dispatch 2024-08-19T01:27:56.203 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:27:55 smithi117 ceph-mon[22016]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a75278d1-161f-41d5-a92f-058d1077bf2a"}]: dispatch 2024-08-19T01:27:56.203 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:27:55 smithi117 ceph-mon[22016]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "a75278d1-161f-41d5-a92f-058d1077bf2a"}]': finished 2024-08-19T01:27:56.204 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:27:55 smithi117 ceph-mon[22016]: osdmap e15: 3 total, 2 up, 3 in 2024-08-19T01:27:56.204 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:27:55 smithi117 ceph-mon[22016]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-08-19T01:27:56.204 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:27:55 smithi117 ceph-mon[22016]: from='client.? 172.21.15.167:0/1465449369' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-08-19T01:27:57.155 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:27:56 smithi083 ceph-mon[17120]: pgmap v61: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-08-19T01:27:57.203 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:27:56 smithi117 ceph-mon[22016]: pgmap v61: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-08-19T01:27:57.216 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:27:56 smithi167 ceph-mon[21875]: pgmap v61: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-08-19T01:27:59.505 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:27:59 smithi167 ceph-mon[21875]: pgmap v62: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-08-19T01:27:59.655 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:27:59 smithi083 ceph-mon[17120]: pgmap v62: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-08-19T01:27:59.703 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:27:59 smithi117 ceph-mon[22016]: pgmap v62: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-08-19T01:28:01.619 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:28:01 smithi167 ceph-mon[21875]: pgmap v63: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-08-19T01:28:01.655 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:28:01 smithi083 ceph-mon[17120]: pgmap v63: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-08-19T01:28:01.703 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:28:01 smithi117 ceph-mon[22016]: pgmap v63: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-08-19T01:28:02.655 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:28:02 smithi083 ceph-mon[17120]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-08-19T01:28:02.655 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:28:02 smithi083 ceph-mon[17120]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-19T01:28:02.703 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:28:02 smithi117 ceph-mon[22016]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-08-19T01:28:02.703 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:28:02 smithi117 ceph-mon[22016]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-19T01:28:02.713 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:28:02 smithi167 ceph-mon[21875]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-08-19T01:28:02.713 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:28:02 smithi167 ceph-mon[21875]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-19T01:28:03.655 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:28:03 smithi083 ceph-mon[17120]: Deploying daemon osd.2 on smithi167 2024-08-19T01:28:03.655 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:28:03 smithi083 ceph-mon[17120]: pgmap v64: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-08-19T01:28:03.703 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:28:03 smithi117 ceph-mon[22016]: Deploying daemon osd.2 on smithi167 2024-08-19T01:28:03.703 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:28:03 smithi117 ceph-mon[22016]: pgmap v64: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-08-19T01:28:03.708 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:28:03 smithi167 ceph-mon[21875]: Deploying daemon osd.2 on smithi167 2024-08-19T01:28:03.708 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:28:03 smithi167 ceph-mon[21875]: pgmap v64: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-08-19T01:28:05.655 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:28:05 smithi083 ceph-mon[17120]: pgmap v65: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-08-19T01:28:05.669 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:28:05 smithi167 ceph-mon[21875]: pgmap v65: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-08-19T01:28:05.703 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:28:05 smithi117 ceph-mon[22016]: pgmap v65: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-08-19T01:28:07.655 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:28:07 smithi083 ceph-mon[17120]: pgmap v66: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-08-19T01:28:07.655 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:28:07 smithi083 ceph-mon[17120]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-19T01:28:07.655 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:28:07 smithi083 ceph-mon[17120]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' 2024-08-19T01:28:07.655 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:28:07 smithi083 ceph-mon[17120]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' 2024-08-19T01:28:07.699 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:28:07 smithi167 ceph-mon[21875]: pgmap v66: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-08-19T01:28:07.699 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:28:07 smithi167 ceph-mon[21875]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-19T01:28:07.699 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:28:07 smithi167 ceph-mon[21875]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' 2024-08-19T01:28:07.699 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:28:07 smithi167 ceph-mon[21875]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' 2024-08-19T01:28:07.702 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:28:07 smithi117 ceph-mon[22016]: pgmap v66: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-08-19T01:28:07.703 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:28:07 smithi117 ceph-mon[22016]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-19T01:28:07.703 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:28:07 smithi117 ceph-mon[22016]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' 2024-08-19T01:28:07.703 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:28:07 smithi117 ceph-mon[22016]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' 2024-08-19T01:28:09.613 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:28:09 smithi167 ceph-mon[21875]: pgmap v67: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-08-19T01:28:09.655 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:28:09 smithi083 ceph-mon[17120]: pgmap v67: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-08-19T01:28:09.702 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:28:09 smithi117 ceph-mon[22016]: pgmap v67: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-08-19T01:28:10.264 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-19T01:28:10.261+0000 7f3c16ffd640 1 -- 172.21.15.167:0/918631276 <== mgr.14150 v2:172.21.15.83:6800/683740916 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f3c20062360 con 0x7f3c0c0a2b20 2024-08-19T01:28:10.264 INFO:teuthology.orchestra.run.smithi167.stdout:Created osd(s) 2 on host 'smithi167' 2024-08-19T01:28:10.266 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-19T01:28:10.264+0000 7f3c27b7f640 1 -- 172.21.15.167:0/918631276 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f3c0c0a2b20 msgr2=0x7f3c0c0a4fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:28:10.266 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-19T01:28:10.264+0000 7f3c27b7f640 1 --2- 172.21.15.167:0/918631276 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f3c0c0a2b20 0x7f3c0c0a4fe0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f3c2019f6d0 tx=0x7f3c1000a4c0 comp rx=0 tx=0).stop 2024-08-19T01:28:10.266 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-19T01:28:10.264+0000 7f3c27b7f640 1 -- 172.21.15.167:0/918631276 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3c20100180 msgr2=0x7f3c2019e1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:28:10.266 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-19T01:28:10.264+0000 7f3c27b7f640 1 --2- 172.21.15.167:0/918631276 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3c20100180 0x7f3c2019e1b0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f3c1c00c890 tx=0x7f3c1c00cd50 comp rx=0 tx=0).stop 2024-08-19T01:28:10.266 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-19T01:28:10.264+0000 7f3c27b7f640 1 -- 172.21.15.167:0/918631276 shutdown_connections 2024-08-19T01:28:10.267 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-19T01:28:10.264+0000 7f3c27b7f640 1 --2- 172.21.15.167:0/918631276 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f3c0c0a2b20 0x7f3c0c0a4fe0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:28:10.267 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-19T01:28:10.264+0000 7f3c27b7f640 1 --2- 172.21.15.167:0/918631276 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3c20105480 0x7f3c201a5770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:28:10.267 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-19T01:28:10.264+0000 7f3c27b7f640 1 --2- 172.21.15.167:0/918631276 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3c20102ad0 0x7f3c2019e6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:28:10.267 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-19T01:28:10.264+0000 7f3c27b7f640 1 --2- 172.21.15.167:0/918631276 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3c20100180 0x7f3c2019e1b0 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:28:10.267 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-19T01:28:10.265+0000 7f3c27b7f640 1 -- 172.21.15.167:0/918631276 >> 172.21.15.167:0/918631276 conn(0x7f3c200fbfb0 msgr2=0x7f3c20100a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:28:10.267 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-19T01:28:10.265+0000 7f3c27b7f640 1 -- 172.21.15.167:0/918631276 shutdown_connections 2024-08-19T01:28:10.267 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-19T01:28:10.265+0000 7f3c27b7f640 1 -- 172.21.15.167:0/918631276 wait complete. 2024-08-19T01:28:10.852 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:28:10 smithi167 ceph-mon[21875]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' 2024-08-19T01:28:10.852 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:28:10 smithi167 ceph-mon[21875]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' 2024-08-19T01:28:10.852 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:28:10 smithi167 ceph-mon[21875]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-19T01:28:10.852 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:28:10 smithi167 ceph-mon[21875]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-19T01:28:10.852 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:28:10 smithi167 ceph-mon[21875]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' 2024-08-19T01:28:10.852 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:28:10 smithi167 ceph-mon[21875]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-19T01:28:10.852 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:28:10 smithi167 ceph-mon[21875]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' 2024-08-19T01:28:10.852 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:28:10 smithi167 ceph-mon[21875]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' 2024-08-19T01:28:10.852 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:28:10 smithi167 ceph-mon[21875]: pgmap v68: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-08-19T01:28:10.878 DEBUG:teuthology.orchestra.run.smithi167:osd.2> sudo journalctl -f -n 0 -u ceph-bad8780e-5dc9-11ef-bccf-c7b262605968@osd.2.service 2024-08-19T01:28:10.881 INFO:tasks.cephadm:Deploying osd.3 on smithi083 with /dev/vg_nvme/lv_3... 2024-08-19T01:28:10.881 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833 ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid bad8780e-5dc9-11ef-bccf-c7b262605968 -- lvm zap /dev/vg_nvme/lv_3 2024-08-19T01:28:10.905 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:28:10 smithi083 ceph-mon[17120]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' 2024-08-19T01:28:10.905 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:28:10 smithi083 ceph-mon[17120]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' 2024-08-19T01:28:10.905 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:28:10 smithi083 ceph-mon[17120]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-19T01:28:10.905 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:28:10 smithi083 ceph-mon[17120]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-19T01:28:10.905 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:28:10 smithi083 ceph-mon[17120]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' 2024-08-19T01:28:10.905 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:28:10 smithi083 ceph-mon[17120]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-19T01:28:10.905 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:28:10 smithi083 ceph-mon[17120]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' 2024-08-19T01:28:10.905 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:28:10 smithi083 ceph-mon[17120]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' 2024-08-19T01:28:10.906 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:28:10 smithi083 ceph-mon[17120]: pgmap v68: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-08-19T01:28:10.953 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:28:10 smithi117 ceph-mon[22016]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' 2024-08-19T01:28:10.953 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:28:10 smithi117 ceph-mon[22016]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' 2024-08-19T01:28:10.953 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:28:10 smithi117 ceph-mon[22016]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-19T01:28:10.953 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:28:10 smithi117 ceph-mon[22016]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-19T01:28:10.953 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:28:10 smithi117 ceph-mon[22016]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' 2024-08-19T01:28:10.953 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:28:10 smithi117 ceph-mon[22016]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-19T01:28:10.953 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:28:10 smithi117 ceph-mon[22016]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' 2024-08-19T01:28:10.954 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:28:10 smithi117 ceph-mon[22016]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' 2024-08-19T01:28:10.954 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:28:10 smithi117 ceph-mon[22016]: pgmap v68: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-08-19T01:28:11.102 INFO:journalctl@ceph.osd.2.smithi167.stdout:Aug 19 01:28:10 smithi167 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-2[26831]: 2024-08-19T01:28:10.997+0000 7fa6ede85640 -1 osd.2 0 log_to_monitors true 2024-08-19T01:28:11.250 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/bad8780e-5dc9-11ef-bccf-c7b262605968/mon.a/config 2024-08-19T01:28:11.655 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:28:11 smithi083 ceph-mon[17120]: from='osd.2 [v2:172.21.15.167:6800/551625328,v1:172.21.15.167:6801/551625328]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-08-19T01:28:11.655 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:28:11 smithi083 ceph-mon[17120]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-08-19T01:28:11.716 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:28:11 smithi167 ceph-mon[21875]: from='osd.2 [v2:172.21.15.167:6800/551625328,v1:172.21.15.167:6801/551625328]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-08-19T01:28:11.716 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:28:11 smithi167 ceph-mon[21875]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-08-19T01:28:11.953 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:28:11 smithi117 ceph-mon[22016]: from='osd.2 [v2:172.21.15.167:6800/551625328,v1:172.21.15.167:6801/551625328]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-08-19T01:28:11.953 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:28:11 smithi117 ceph-mon[22016]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-08-19T01:28:12.823 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:28:12 smithi083 ceph-mon[17120]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-08-19T01:28:12.823 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:28:12 smithi083 ceph-mon[17120]: osdmap e16: 3 total, 2 up, 3 in 2024-08-19T01:28:12.823 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:28:12 smithi083 ceph-mon[17120]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-08-19T01:28:12.823 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:28:12 smithi083 ceph-mon[17120]: from='osd.2 [v2:172.21.15.167:6800/551625328,v1:172.21.15.167:6801/551625328]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi167", "root=default"]}]: dispatch 2024-08-19T01:28:12.823 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:28:12 smithi083 ceph-mon[17120]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi167", "root=default"]}]: dispatch 2024-08-19T01:28:12.824 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:28:12 smithi083 ceph-mon[17120]: pgmap v70: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-08-19T01:28:12.953 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:28:12 smithi117 ceph-mon[22016]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-08-19T01:28:12.953 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:28:12 smithi117 ceph-mon[22016]: osdmap e16: 3 total, 2 up, 3 in 2024-08-19T01:28:12.953 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:28:12 smithi117 ceph-mon[22016]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-08-19T01:28:12.953 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:28:12 smithi117 ceph-mon[22016]: from='osd.2 [v2:172.21.15.167:6800/551625328,v1:172.21.15.167:6801/551625328]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi167", "root=default"]}]: dispatch 2024-08-19T01:28:12.953 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:28:12 smithi117 ceph-mon[22016]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi167", "root=default"]}]: dispatch 2024-08-19T01:28:12.953 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:28:12 smithi117 ceph-mon[22016]: pgmap v70: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-08-19T01:28:12.966 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:28:12 smithi167 ceph-mon[21875]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-08-19T01:28:12.967 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:28:12 smithi167 ceph-mon[21875]: osdmap e16: 3 total, 2 up, 3 in 2024-08-19T01:28:12.967 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:28:12 smithi167 ceph-mon[21875]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-08-19T01:28:12.967 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:28:12 smithi167 ceph-mon[21875]: from='osd.2 [v2:172.21.15.167:6800/551625328,v1:172.21.15.167:6801/551625328]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi167", "root=default"]}]: dispatch 2024-08-19T01:28:12.967 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:28:12 smithi167 ceph-mon[21875]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi167", "root=default"]}]: dispatch 2024-08-19T01:28:12.967 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:28:12 smithi167 ceph-mon[21875]: pgmap v70: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-08-19T01:28:13.716 INFO:journalctl@ceph.osd.2.smithi167.stdout:Aug 19 01:28:13 smithi167 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-2[26831]: 2024-08-19T01:28:13.444+0000 7fa6e95e5640 -1 osd.2 0 waiting for initial osdmap 2024-08-19T01:28:13.716 INFO:journalctl@ceph.osd.2.smithi167.stdout:Aug 19 01:28:13 smithi167 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-2[26831]: 2024-08-19T01:28:13.453+0000 7fa6e1c17640 -1 osd.2 17 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-08-19T01:28:13.717 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:28:13 smithi167 ceph-mon[21875]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi167", "root=default"]}]': finished 2024-08-19T01:28:13.717 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:28:13 smithi167 ceph-mon[21875]: osdmap e17: 3 total, 2 up, 3 in 2024-08-19T01:28:13.717 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:28:13 smithi167 ceph-mon[21875]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-08-19T01:28:13.717 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:28:13 smithi167 ceph-mon[21875]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-08-19T01:28:13.717 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:28:13 smithi167 ceph-mon[21875]: from='osd.2 ' entity='osd.2' 2024-08-19T01:28:13.905 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:28:13 smithi083 ceph-mon[17120]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi167", "root=default"]}]': finished 2024-08-19T01:28:13.905 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:28:13 smithi083 ceph-mon[17120]: osdmap e17: 3 total, 2 up, 3 in 2024-08-19T01:28:13.905 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:28:13 smithi083 ceph-mon[17120]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-08-19T01:28:13.905 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:28:13 smithi083 ceph-mon[17120]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-08-19T01:28:13.905 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:28:13 smithi083 ceph-mon[17120]: from='osd.2 ' entity='osd.2' 2024-08-19T01:28:13.952 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:28:13 smithi117 ceph-mon[22016]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi167", "root=default"]}]': finished 2024-08-19T01:28:13.953 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:28:13 smithi117 ceph-mon[22016]: osdmap e17: 3 total, 2 up, 3 in 2024-08-19T01:28:13.953 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:28:13 smithi117 ceph-mon[22016]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-08-19T01:28:13.953 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:28:13 smithi117 ceph-mon[22016]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-08-19T01:28:13.953 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:28:13 smithi117 ceph-mon[22016]: from='osd.2 ' entity='osd.2' 2024-08-19T01:28:14.438 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-08-19T01:28:14.474 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid bad8780e-5dc9-11ef-bccf-c7b262605968 -- ceph orch daemon add osd smithi083:vg_nvme/lv_3 2024-08-19T01:28:14.884 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/bad8780e-5dc9-11ef-bccf-c7b262605968/mon.a/config 2024-08-19T01:28:14.905 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:28:14 smithi083 ceph-mon[17120]: purged_snaps scrub starts 2024-08-19T01:28:14.905 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:28:14 smithi083 ceph-mon[17120]: purged_snaps scrub ok 2024-08-19T01:28:14.905 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:28:14 smithi083 ceph-mon[17120]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-08-19T01:28:14.905 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:28:14 smithi083 ceph-mon[17120]: Detected new or changed devices on smithi167 2024-08-19T01:28:14.905 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:28:14 smithi083 ceph-mon[17120]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' 2024-08-19T01:28:14.905 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:28:14 smithi083 ceph-mon[17120]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' 2024-08-19T01:28:14.905 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:28:14 smithi083 ceph-mon[17120]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-08-19T01:28:14.905 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:28:14 smithi083 ceph-mon[17120]: Adjusting osd_memory_target on smithi167 to 17088M 2024-08-19T01:28:14.905 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:28:14 smithi083 ceph-mon[17120]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' 2024-08-19T01:28:14.906 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:28:14 smithi083 ceph-mon[17120]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-19T01:28:14.906 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:28:14 smithi083 ceph-mon[17120]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-19T01:28:14.906 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:28:14 smithi083 ceph-mon[17120]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' 2024-08-19T01:28:14.906 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:28:14 smithi083 ceph-mon[17120]: pgmap v72: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-08-19T01:28:14.906 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:28:14 smithi083 ceph-mon[17120]: osd.2 [v2:172.21.15.167:6800/551625328,v1:172.21.15.167:6801/551625328] boot 2024-08-19T01:28:14.906 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:28:14 smithi083 ceph-mon[17120]: osdmap e18: 3 total, 3 up, 3 in 2024-08-19T01:28:14.906 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:28:14 smithi083 ceph-mon[17120]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-08-19T01:28:14.953 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:28:14 smithi117 ceph-mon[22016]: purged_snaps scrub starts 2024-08-19T01:28:14.953 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:28:14 smithi117 ceph-mon[22016]: purged_snaps scrub ok 2024-08-19T01:28:14.953 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:28:14 smithi117 ceph-mon[22016]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-08-19T01:28:14.953 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:28:14 smithi117 ceph-mon[22016]: Detected new or changed devices on smithi167 2024-08-19T01:28:14.953 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:28:14 smithi117 ceph-mon[22016]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' 2024-08-19T01:28:14.953 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:28:14 smithi117 ceph-mon[22016]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' 2024-08-19T01:28:14.953 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:28:14 smithi117 ceph-mon[22016]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-08-19T01:28:14.953 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:28:14 smithi117 ceph-mon[22016]: Adjusting osd_memory_target on smithi167 to 17088M 2024-08-19T01:28:14.953 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:28:14 smithi117 ceph-mon[22016]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' 2024-08-19T01:28:14.953 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:28:14 smithi117 ceph-mon[22016]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-19T01:28:14.954 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:28:14 smithi117 ceph-mon[22016]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-19T01:28:14.954 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:28:14 smithi117 ceph-mon[22016]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' 2024-08-19T01:28:14.954 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:28:14 smithi117 ceph-mon[22016]: pgmap v72: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-08-19T01:28:14.954 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:28:14 smithi117 ceph-mon[22016]: osd.2 [v2:172.21.15.167:6800/551625328,v1:172.21.15.167:6801/551625328] boot 2024-08-19T01:28:14.954 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:28:14 smithi117 ceph-mon[22016]: osdmap e18: 3 total, 3 up, 3 in 2024-08-19T01:28:14.954 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:28:14 smithi117 ceph-mon[22016]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-08-19T01:28:14.966 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:28:14 smithi167 ceph-mon[21875]: purged_snaps scrub starts 2024-08-19T01:28:14.966 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:28:14 smithi167 ceph-mon[21875]: purged_snaps scrub ok 2024-08-19T01:28:14.967 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:28:14 smithi167 ceph-mon[21875]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-08-19T01:28:14.967 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:28:14 smithi167 ceph-mon[21875]: Detected new or changed devices on smithi167 2024-08-19T01:28:14.967 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:28:14 smithi167 ceph-mon[21875]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' 2024-08-19T01:28:14.967 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:28:14 smithi167 ceph-mon[21875]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' 2024-08-19T01:28:14.967 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:28:14 smithi167 ceph-mon[21875]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-08-19T01:28:14.967 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:28:14 smithi167 ceph-mon[21875]: Adjusting osd_memory_target on smithi167 to 17088M 2024-08-19T01:28:14.967 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:28:14 smithi167 ceph-mon[21875]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' 2024-08-19T01:28:14.967 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:28:14 smithi167 ceph-mon[21875]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-19T01:28:14.967 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:28:14 smithi167 ceph-mon[21875]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-19T01:28:14.967 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:28:14 smithi167 ceph-mon[21875]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' 2024-08-19T01:28:14.967 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:28:14 smithi167 ceph-mon[21875]: pgmap v72: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-08-19T01:28:14.967 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:28:14 smithi167 ceph-mon[21875]: osd.2 [v2:172.21.15.167:6800/551625328,v1:172.21.15.167:6801/551625328] boot 2024-08-19T01:28:14.967 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:28:14 smithi167 ceph-mon[21875]: osdmap e18: 3 total, 3 up, 3 in 2024-08-19T01:28:14.967 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:28:14 smithi167 ceph-mon[21875]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-08-19T01:28:15.520 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:28:15.517+0000 7f31fa600640 1 -- 172.21.15.83:0/3457383646 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f31f4104d30 msgr2=0x7f31f41051b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:28:15.520 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:28:15.517+0000 7f31fa600640 1 --2- 172.21.15.83:0/3457383646 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f31f4104d30 0x7f31f41051b0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f31e0009900 tx=0x7f31e002a3f0 comp rx=0 tx=0).stop 2024-08-19T01:28:15.520 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:28:15.518+0000 7f31fa600640 1 -- 172.21.15.83:0/3457383646 shutdown_connections 2024-08-19T01:28:15.520 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:28:15.518+0000 7f31fa600640 1 --2- 172.21.15.83:0/3457383646 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f31f41056f0 0x7f31f410bf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:28:15.521 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:28:15.518+0000 7f31fa600640 1 --2- 172.21.15.83:0/3457383646 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f31f4104d30 0x7f31f41051b0 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:28:15.521 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:28:15.518+0000 7f31fa600640 1 --2- 172.21.15.83:0/3457383646 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f31f4103b30 0x7f31f4103f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:28:15.521 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:28:15.518+0000 7f31fa600640 1 -- 172.21.15.83:0/3457383646 >> 172.21.15.83:0/3457383646 conn(0x7f31f40ff2e0 msgr2=0x7f31f4101700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:28:15.521 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:28:15.518+0000 7f31fa600640 1 -- 172.21.15.83:0/3457383646 shutdown_connections 2024-08-19T01:28:15.521 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:28:15.518+0000 7f31fa600640 1 -- 172.21.15.83:0/3457383646 wait complete. 2024-08-19T01:28:15.523 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:28:15.519+0000 7f31fa600640 1 Processor -- start 2024-08-19T01:28:15.523 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:28:15.519+0000 7f31fa600640 1 -- start start 2024-08-19T01:28:15.523 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:28:15.519+0000 7f31fa600640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f31f4103b30 0x7f31f4079cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:28:15.523 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:28:15.519+0000 7f31fa600640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f31f4104d30 0x7f31f407a210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:28:15.523 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:28:15.519+0000 7f31fa600640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f31f41056f0 0x7f31f4076830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:28:15.523 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:28:15.519+0000 7f31fa600640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f31f410f800 con 0x7f31f41056f0 2024-08-19T01:28:15.523 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:28:15.519+0000 7f31fa600640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f31f410f680 con 0x7f31f4103b30 2024-08-19T01:28:15.524 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:28:15.519+0000 7f31fa600640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f31f410f980 con 0x7f31f4104d30 2024-08-19T01:28:15.524 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:28:15.519+0000 7f31f3e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f31f41056f0 0x7f31f4076830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:28:15.524 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:28:15.519+0000 7f31f3400640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f31f4103b30 0x7f31f4079cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:28:15.524 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:28:15.519+0000 7f31f3400640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f31f4103b30 0x7f31f4079cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.83:40130/0 (socket says 172.21.15.83:40130) 2024-08-19T01:28:15.524 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:28:15.519+0000 7f31f3400640 1 -- 172.21.15.83:0/2693023465 learned_addr learned my addr 172.21.15.83:0/2693023465 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-08-19T01:28:15.524 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:28:15.519+0000 7f31f3e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f31f41056f0 0x7f31f4076830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:39200/0 (socket says 172.21.15.83:39200) 2024-08-19T01:28:15.524 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:28:15.520+0000 7f31f2a00640 1 --2- 172.21.15.83:0/2693023465 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f31f4104d30 0x7f31f407a210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:28:15.524 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:28:15.520+0000 7f31f3400640 1 -- 172.21.15.83:0/2693023465 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f31f4104d30 msgr2=0x7f31f407a210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:28:15.524 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:28:15.520+0000 7f31f3400640 1 --2- 172.21.15.83:0/2693023465 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f31f4104d30 0x7f31f407a210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:28:15.524 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:28:15.520+0000 7f31f3400640 1 -- 172.21.15.83:0/2693023465 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f31f41056f0 msgr2=0x7f31f4076830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:28:15.524 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:28:15.520+0000 7f31f3400640 1 --2- 172.21.15.83:0/2693023465 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f31f41056f0 0x7f31f4076830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:28:15.524 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:28:15.520+0000 7f31f3400640 1 -- 172.21.15.83:0/2693023465 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f31f40770f0 con 0x7f31f4103b30 2024-08-19T01:28:15.524 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:28:15.520+0000 7f31f3400640 1 --2- 172.21.15.83:0/2693023465 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f31f4103b30 0x7f31f4079cd0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f31e400e9c0 tx=0x7f31e400edc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:28:15.525 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:28:15.520+0000 7f31ebe00640 1 -- 172.21.15.83:0/2693023465 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31e400ccc0 con 0x7f31f4103b30 2024-08-19T01:28:15.525 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:28:15.521+0000 7f31fa600640 1 -- 172.21.15.83:0/2693023465 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f31f41a62b0 con 0x7f31f4103b30 2024-08-19T01:28:15.525 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:28:15.521+0000 7f31ebe00640 1 -- 172.21.15.83:0/2693023465 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f31e40035f0 con 0x7f31f4103b30 2024-08-19T01:28:15.525 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:28:15.521+0000 7f31ebe00640 1 -- 172.21.15.83:0/2693023465 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31e400ce60 con 0x7f31f4103b30 2024-08-19T01:28:15.525 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:28:15.521+0000 7f31fa600640 1 -- 172.21.15.83:0/2693023465 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f31f41a66a0 con 0x7f31f4103b30 2024-08-19T01:28:15.525 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:28:15.522+0000 7f31fa600640 1 -- 172.21.15.83:0/2693023465 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f31f410aee0 con 0x7f31f4103b30 2024-08-19T01:28:15.525 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:28:15.524+0000 7f31ebe00640 1 -- 172.21.15.83:0/2693023465 <== mon.2 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 15) ==== 137777+0+0 (secure 0 0 0) 0x7f31e40037a0 con 0x7f31f4103b30 2024-08-19T01:28:15.527 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:28:15.525+0000 7f31ebe00640 1 --2- 172.21.15.83:0/2693023465 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f31d80a2b70 0x7f31d80a5030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:28:15.527 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:28:15.525+0000 7f31ebe00640 1 -- 172.21.15.83:0/2693023465 <== mon.2 v2:172.21.15.117:3300/0 5 ==== osd_map(18..18 src has 1..18) ==== 2851+0+0 (secure 0 0 0) 0x7f31e40cde40 con 0x7f31f4103b30 2024-08-19T01:28:15.527 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:28:15.525+0000 7f31f2a00640 1 --2- 172.21.15.83:0/2693023465 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f31d80a2b70 0x7f31d80a5030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:28:15.527 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:28:15.526+0000 7f31f2a00640 1 --2- 172.21.15.83:0/2693023465 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f31d80a2b70 0x7f31d80a5030 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f31e0031440 tx=0x7f31e00315b0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:28:15.529 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:28:15.527+0000 7f31ebe00640 1 -- 172.21.15.83:0/2693023465 <== mon.2 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7f31e409c330 con 0x7f31f4103b30 2024-08-19T01:28:15.712 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:28:15.710+0000 7f31fa600640 1 -- 172.21.15.83:0/2693023465 --> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi083:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7f31f4062360 con 0x7f31d80a2b70 2024-08-19T01:28:16.155 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:28:15 smithi083 ceph-mon[17120]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-08-19T01:28:16.155 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:28:15 smithi083 ceph-mon[17120]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-08-19T01:28:16.155 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:28:15 smithi083 ceph-mon[17120]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-19T01:28:16.203 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:28:15 smithi117 ceph-mon[22016]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-08-19T01:28:16.203 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:28:15 smithi117 ceph-mon[22016]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-08-19T01:28:16.203 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:28:15 smithi117 ceph-mon[22016]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-19T01:28:16.216 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:28:15 smithi167 ceph-mon[21875]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-08-19T01:28:16.216 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:28:15 smithi167 ceph-mon[21875]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-08-19T01:28:16.216 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:28:15 smithi167 ceph-mon[21875]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-19T01:28:17.070 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:28:16 smithi083 ceph-mon[17120]: osdmap e19: 3 total, 3 up, 3 in 2024-08-19T01:28:17.071 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:28:16 smithi083 ceph-mon[17120]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-08-19T01:28:17.203 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:28:16 smithi117 ceph-mon[22016]: osdmap e19: 3 total, 3 up, 3 in 2024-08-19T01:28:17.203 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:28:16 smithi117 ceph-mon[22016]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-08-19T01:28:17.216 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:28:16 smithi167 ceph-mon[21875]: osdmap e19: 3 total, 3 up, 3 in 2024-08-19T01:28:17.216 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:28:16 smithi167 ceph-mon[21875]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-08-19T01:28:17.944 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:28:17 smithi083 ceph-mon[17120]: from='client.24194 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi083:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-08-19T01:28:17.944 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:28:17 smithi083 ceph-mon[17120]: pgmap v75: 0 pgs: ; 0 B data, 88 MiB used, 268 GiB / 268 GiB avail 2024-08-19T01:28:17.944 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:28:17 smithi083 ceph-mon[17120]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-08-19T01:28:17.944 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:28:17 smithi083 ceph-mon[17120]: osdmap e20: 3 total, 3 up, 3 in 2024-08-19T01:28:17.944 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:28:17 smithi083 ceph-mon[17120]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-08-19T01:28:18.120 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:28:17 smithi117 ceph-mon[22016]: from='client.24194 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi083:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-08-19T01:28:18.120 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:28:17 smithi117 ceph-mon[22016]: pgmap v75: 0 pgs: ; 0 B data, 88 MiB used, 268 GiB / 268 GiB avail 2024-08-19T01:28:18.121 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:28:17 smithi117 ceph-mon[22016]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-08-19T01:28:18.121 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:28:17 smithi117 ceph-mon[22016]: osdmap e20: 3 total, 3 up, 3 in 2024-08-19T01:28:18.121 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:28:17 smithi117 ceph-mon[22016]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-08-19T01:28:18.133 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:28:17 smithi167 ceph-mon[21875]: from='client.24194 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi083:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-08-19T01:28:18.134 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:28:17 smithi167 ceph-mon[21875]: pgmap v75: 0 pgs: ; 0 B data, 88 MiB used, 268 GiB / 268 GiB avail 2024-08-19T01:28:18.134 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:28:17 smithi167 ceph-mon[21875]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-08-19T01:28:18.134 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:28:17 smithi167 ceph-mon[21875]: osdmap e20: 3 total, 3 up, 3 in 2024-08-19T01:28:18.134 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:28:17 smithi167 ceph-mon[21875]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-08-19T01:28:19.203 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:28:18 smithi117 ceph-mon[22016]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-08-19T01:28:19.203 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:28:18 smithi117 ceph-mon[22016]: osdmap e21: 3 total, 3 up, 3 in 2024-08-19T01:28:19.203 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:28:18 smithi117 ceph-mon[22016]: pgmap v78: 1 pgs: 1 creating+peering; 0 B data, 88 MiB used, 268 GiB / 268 GiB avail 2024-08-19T01:28:19.203 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:28:18 smithi117 ceph-mon[22016]: from='client.? 172.21.15.83:0/2737285076' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d1fb16f9-910e-46fa-b1e3-8e5bef4d346d"}]: dispatch 2024-08-19T01:28:19.203 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:28:18 smithi117 ceph-mon[22016]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d1fb16f9-910e-46fa-b1e3-8e5bef4d346d"}]: dispatch 2024-08-19T01:28:19.203 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:28:18 smithi117 ceph-mon[22016]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "d1fb16f9-910e-46fa-b1e3-8e5bef4d346d"}]': finished 2024-08-19T01:28:19.203 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:28:18 smithi117 ceph-mon[22016]: osdmap e22: 4 total, 3 up, 4 in 2024-08-19T01:28:19.203 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:28:18 smithi117 ceph-mon[22016]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-08-19T01:28:19.217 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:28:18 smithi167 ceph-mon[21875]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-08-19T01:28:19.217 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:28:18 smithi167 ceph-mon[21875]: osdmap e21: 3 total, 3 up, 3 in 2024-08-19T01:28:19.217 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:28:18 smithi167 ceph-mon[21875]: pgmap v78: 1 pgs: 1 creating+peering; 0 B data, 88 MiB used, 268 GiB / 268 GiB avail 2024-08-19T01:28:19.217 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:28:18 smithi167 ceph-mon[21875]: from='client.? 172.21.15.83:0/2737285076' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d1fb16f9-910e-46fa-b1e3-8e5bef4d346d"}]: dispatch 2024-08-19T01:28:19.217 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:28:18 smithi167 ceph-mon[21875]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d1fb16f9-910e-46fa-b1e3-8e5bef4d346d"}]: dispatch 2024-08-19T01:28:19.217 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:28:18 smithi167 ceph-mon[21875]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "d1fb16f9-910e-46fa-b1e3-8e5bef4d346d"}]': finished 2024-08-19T01:28:19.218 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:28:18 smithi167 ceph-mon[21875]: osdmap e22: 4 total, 3 up, 4 in 2024-08-19T01:28:19.218 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:28:18 smithi167 ceph-mon[21875]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-08-19T01:28:19.347 INFO:journalctl@ceph.osd.0.smithi083.stdout:Aug 19 01:28:19 smithi083 sudo[36011]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-08-19T01:28:19.348 INFO:journalctl@ceph.osd.0.smithi083.stdout:Aug 19 01:28:19 smithi083 sudo[36011]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-08-19T01:28:19.348 INFO:journalctl@ceph.osd.0.smithi083.stdout:Aug 19 01:28:19 smithi083 sudo[36011]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-08-19T01:28:19.348 INFO:journalctl@ceph.osd.0.smithi083.stdout:Aug 19 01:28:19 smithi083 sudo[36011]: pam_unix(sudo:session): session closed for user root 2024-08-19T01:28:19.348 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:28:18 smithi083 ceph-mon[17120]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-08-19T01:28:19.348 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:28:18 smithi083 ceph-mon[17120]: osdmap e21: 3 total, 3 up, 3 in 2024-08-19T01:28:19.348 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:28:18 smithi083 ceph-mon[17120]: pgmap v78: 1 pgs: 1 creating+peering; 0 B data, 88 MiB used, 268 GiB / 268 GiB avail 2024-08-19T01:28:19.348 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:28:18 smithi083 ceph-mon[17120]: from='client.? 172.21.15.83:0/2737285076' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d1fb16f9-910e-46fa-b1e3-8e5bef4d346d"}]: dispatch 2024-08-19T01:28:19.348 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:28:18 smithi083 ceph-mon[17120]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d1fb16f9-910e-46fa-b1e3-8e5bef4d346d"}]: dispatch 2024-08-19T01:28:19.348 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:28:18 smithi083 ceph-mon[17120]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "d1fb16f9-910e-46fa-b1e3-8e5bef4d346d"}]': finished 2024-08-19T01:28:19.349 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:28:18 smithi083 ceph-mon[17120]: osdmap e22: 4 total, 3 up, 4 in 2024-08-19T01:28:19.349 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:28:18 smithi083 ceph-mon[17120]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-08-19T01:28:19.905 INFO:journalctl@ceph.osd.0.smithi083.stdout:Aug 19 01:28:19 smithi083 sudo[36057]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-08-19T01:28:19.906 INFO:journalctl@ceph.osd.0.smithi083.stdout:Aug 19 01:28:19 smithi083 sudo[36057]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-08-19T01:28:19.906 INFO:journalctl@ceph.osd.0.smithi083.stdout:Aug 19 01:28:19 smithi083 sudo[36057]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-08-19T01:28:19.906 INFO:journalctl@ceph.osd.0.smithi083.stdout:Aug 19 01:28:19 smithi083 sudo[36057]: pam_unix(sudo:session): session closed for user root 2024-08-19T01:28:20.203 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:28:19 smithi117 ceph-mon[22016]: from='client.? 172.21.15.83:0/3450922194' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-08-19T01:28:20.203 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:28:19 smithi117 ceph-mon[22016]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' 2024-08-19T01:28:20.203 INFO:journalctl@ceph.osd.1.smithi117.stdout:Aug 19 01:28:20 smithi117 sudo[30141]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-08-19T01:28:20.203 INFO:journalctl@ceph.osd.1.smithi117.stdout:Aug 19 01:28:20 smithi117 sudo[30141]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-08-19T01:28:20.203 INFO:journalctl@ceph.osd.1.smithi117.stdout:Aug 19 01:28:20 smithi117 sudo[30141]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-08-19T01:28:20.203 INFO:journalctl@ceph.osd.1.smithi117.stdout:Aug 19 01:28:20 smithi117 sudo[30141]: pam_unix(sudo:session): session closed for user root 2024-08-19T01:28:20.216 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:28:19 smithi167 ceph-mon[21875]: from='client.? 172.21.15.83:0/3450922194' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-08-19T01:28:20.216 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:28:19 smithi167 ceph-mon[21875]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' 2024-08-19T01:28:20.405 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:28:19 smithi083 ceph-mon[17120]: from='client.? 172.21.15.83:0/3450922194' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-08-19T01:28:20.405 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:28:19 smithi083 ceph-mon[17120]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' 2024-08-19T01:28:20.703 INFO:journalctl@ceph.osd.1.smithi117.stdout:Aug 19 01:28:20 smithi117 sudo[30144]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-08-19T01:28:20.703 INFO:journalctl@ceph.osd.1.smithi117.stdout:Aug 19 01:28:20 smithi117 sudo[30144]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-08-19T01:28:20.703 INFO:journalctl@ceph.osd.1.smithi117.stdout:Aug 19 01:28:20 smithi117 sudo[30144]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-08-19T01:28:20.703 INFO:journalctl@ceph.osd.1.smithi117.stdout:Aug 19 01:28:20 smithi117 sudo[30144]: pam_unix(sudo:session): session closed for user root 2024-08-19T01:28:20.878 INFO:journalctl@ceph.osd.2.smithi167.stdout:Aug 19 01:28:20 smithi167 sudo[30472]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-08-19T01:28:20.878 INFO:journalctl@ceph.osd.2.smithi167.stdout:Aug 19 01:28:20 smithi167 sudo[30472]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-08-19T01:28:20.878 INFO:journalctl@ceph.osd.2.smithi167.stdout:Aug 19 01:28:20 smithi167 sudo[30472]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-08-19T01:28:20.878 INFO:journalctl@ceph.osd.2.smithi167.stdout:Aug 19 01:28:20 smithi167 sudo[30472]: pam_unix(sudo:session): session closed for user root 2024-08-19T01:28:21.216 INFO:journalctl@ceph.osd.2.smithi167.stdout:Aug 19 01:28:20 smithi167 sudo[30475]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-08-19T01:28:21.216 INFO:journalctl@ceph.osd.2.smithi167.stdout:Aug 19 01:28:20 smithi167 sudo[30475]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-08-19T01:28:21.216 INFO:journalctl@ceph.osd.2.smithi167.stdout:Aug 19 01:28:20 smithi167 sudo[30475]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-08-19T01:28:21.216 INFO:journalctl@ceph.osd.2.smithi167.stdout:Aug 19 01:28:20 smithi167 sudo[30475]: pam_unix(sudo:session): session closed for user root 2024-08-19T01:28:21.905 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:28:21 smithi083 sudo[36834]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-08-19T01:28:21.905 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:28:21 smithi083 sudo[36834]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-08-19T01:28:21.905 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:28:21 smithi083 sudo[36834]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-08-19T01:28:21.905 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:28:21 smithi083 sudo[36834]: pam_unix(sudo:session): session closed for user root 2024-08-19T01:28:22.006 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:28:22.004+0000 7f31ebe00640 1 -- 172.21.15.83:0/2693023465 <== mon.2 v2:172.21.15.117:3300/0 7 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f31e40982a0 con 0x7f31f4103b30 2024-08-19T01:28:22.405 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:28:21 smithi083 sudo[37026]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-08-19T01:28:22.405 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:28:21 smithi083 sudo[37026]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-08-19T01:28:22.405 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:28:21 smithi083 sudo[37026]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-08-19T01:28:22.405 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:28:21 smithi083 sudo[37026]: pam_unix(sudo:session): session closed for user root 2024-08-19T01:28:22.405 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:28:21 smithi083 ceph-mon[17120]: pgmap v80: 1 pgs: 1 creating+peering; 0 B data, 89 MiB used, 268 GiB / 268 GiB avail 2024-08-19T01:28:22.405 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:28:21 smithi083 ceph-mon[17120]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' 2024-08-19T01:28:22.405 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:28:21 smithi083 ceph-mon[17120]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' 2024-08-19T01:28:22.406 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:28:21 smithi083 ceph-mon[17120]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-08-19T01:28:22.453 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:28:22 smithi117 ceph-mon[22016]: pgmap v80: 1 pgs: 1 creating+peering; 0 B data, 89 MiB used, 268 GiB / 268 GiB avail 2024-08-19T01:28:22.453 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:28:22 smithi117 ceph-mon[22016]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' 2024-08-19T01:28:22.453 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:28:22 smithi117 ceph-mon[22016]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' 2024-08-19T01:28:22.453 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:28:22 smithi117 ceph-mon[22016]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-08-19T01:28:22.493 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:28:22 smithi167 sudo[30478]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-08-19T01:28:22.494 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:28:22 smithi167 sudo[30478]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-08-19T01:28:22.494 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:28:22 smithi167 sudo[30478]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-08-19T01:28:22.494 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:28:22 smithi167 sudo[30478]: pam_unix(sudo:session): session closed for user root 2024-08-19T01:28:22.966 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:28:22 smithi167 sudo[30481]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-08-19T01:28:22.966 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:28:22 smithi167 sudo[30481]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-08-19T01:28:22.967 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:28:22 smithi167 sudo[30481]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-08-19T01:28:22.967 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:28:22 smithi167 sudo[30481]: pam_unix(sudo:session): session closed for user root 2024-08-19T01:28:22.967 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:28:22 smithi167 ceph-mon[21875]: pgmap v80: 1 pgs: 1 creating+peering; 0 B data, 89 MiB used, 268 GiB / 268 GiB avail 2024-08-19T01:28:22.967 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:28:22 smithi167 ceph-mon[21875]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' 2024-08-19T01:28:22.967 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:28:22 smithi167 ceph-mon[21875]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' 2024-08-19T01:28:22.967 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:28:22 smithi167 ceph-mon[21875]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-08-19T01:28:22.996 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:28:22 smithi117 sudo[30147]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-08-19T01:28:22.996 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:28:22 smithi117 sudo[30147]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-08-19T01:28:22.996 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:28:22 smithi117 sudo[30147]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-08-19T01:28:22.996 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:28:22 smithi117 sudo[30147]: pam_unix(sudo:session): session closed for user root 2024-08-19T01:28:23.405 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:28:23 smithi083 ceph-mon[17120]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-08-19T01:28:23.405 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:28:23 smithi083 ceph-mon[17120]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-08-19T01:28:23.405 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:28:23 smithi083 ceph-mon[17120]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-08-19T01:28:23.405 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:28:23 smithi083 ceph-mon[17120]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-08-19T01:28:23.405 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:28:23 smithi083 ceph-mon[17120]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-08-19T01:28:23.405 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:28:23 smithi083 ceph-mon[17120]: mgrmap e16: x(active, since 2m), standbys: z, y 2024-08-19T01:28:23.406 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:28:23 smithi083 ceph-mon[17120]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-08-19T01:28:23.406 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:28:23 smithi083 ceph-mon[17120]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-08-19T01:28:23.406 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:28:23 smithi083 ceph-mon[17120]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-08-19T01:28:23.406 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:28:23 smithi083 ceph-mon[17120]: pgmap v81: 1 pgs: 1 creating+peering; 0 B data, 89 MiB used, 268 GiB / 268 GiB avail 2024-08-19T01:28:23.406 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:28:23 smithi083 ceph-mon[17120]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-08-19T01:28:23.406 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:28:23 smithi083 ceph-mon[17120]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-08-19T01:28:23.406 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:28:23 smithi083 ceph-mon[17120]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-08-19T01:28:23.406 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:28:23 smithi083 ceph-mon[17120]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-08-19T01:28:23.453 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:28:22 smithi117 sudo[30150]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-08-19T01:28:23.453 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:28:22 smithi117 sudo[30150]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-08-19T01:28:23.453 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:28:22 smithi117 sudo[30150]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-08-19T01:28:23.453 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:28:22 smithi117 sudo[30150]: pam_unix(sudo:session): session closed for user root 2024-08-19T01:28:23.453 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:28:23 smithi117 ceph-mon[22016]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-08-19T01:28:23.453 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:28:23 smithi117 ceph-mon[22016]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-08-19T01:28:23.453 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:28:23 smithi117 ceph-mon[22016]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-08-19T01:28:23.453 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:28:23 smithi117 ceph-mon[22016]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-08-19T01:28:23.453 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:28:23 smithi117 ceph-mon[22016]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-08-19T01:28:23.453 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:28:23 smithi117 ceph-mon[22016]: mgrmap e16: x(active, since 2m), standbys: z, y 2024-08-19T01:28:23.454 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:28:23 smithi117 ceph-mon[22016]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-08-19T01:28:23.454 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:28:23 smithi117 ceph-mon[22016]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-08-19T01:28:23.454 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:28:23 smithi117 ceph-mon[22016]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-08-19T01:28:23.454 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:28:23 smithi117 ceph-mon[22016]: pgmap v81: 1 pgs: 1 creating+peering; 0 B data, 89 MiB used, 268 GiB / 268 GiB avail 2024-08-19T01:28:23.454 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:28:23 smithi117 ceph-mon[22016]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-08-19T01:28:23.454 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:28:23 smithi117 ceph-mon[22016]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-08-19T01:28:23.454 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:28:23 smithi117 ceph-mon[22016]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-08-19T01:28:23.454 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:28:23 smithi117 ceph-mon[22016]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-08-19T01:28:23.466 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:28:23 smithi167 ceph-mon[21875]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-08-19T01:28:23.467 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:28:23 smithi167 ceph-mon[21875]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-08-19T01:28:23.467 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:28:23 smithi167 ceph-mon[21875]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-08-19T01:28:23.467 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:28:23 smithi167 ceph-mon[21875]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-08-19T01:28:23.467 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:28:23 smithi167 ceph-mon[21875]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-08-19T01:28:23.467 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:28:23 smithi167 ceph-mon[21875]: mgrmap e16: x(active, since 2m), standbys: z, y 2024-08-19T01:28:23.467 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:28:23 smithi167 ceph-mon[21875]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-08-19T01:28:23.467 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:28:23 smithi167 ceph-mon[21875]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-08-19T01:28:23.467 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:28:23 smithi167 ceph-mon[21875]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-08-19T01:28:23.467 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:28:23 smithi167 ceph-mon[21875]: pgmap v81: 1 pgs: 1 creating+peering; 0 B data, 89 MiB used, 268 GiB / 268 GiB avail 2024-08-19T01:28:23.467 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:28:23 smithi167 ceph-mon[21875]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-08-19T01:28:23.467 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:28:23 smithi167 ceph-mon[21875]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-08-19T01:28:23.467 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:28:23 smithi167 ceph-mon[21875]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-08-19T01:28:23.468 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:28:23 smithi167 ceph-mon[21875]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-08-19T01:28:24.301 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:28:24 smithi083 ceph-mon[17120]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-08-19T01:28:24.301 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:28:24 smithi083 ceph-mon[17120]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-08-19T01:28:24.453 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:28:24 smithi117 ceph-mon[22016]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-08-19T01:28:24.453 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:28:24 smithi117 ceph-mon[22016]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-08-19T01:28:24.466 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:28:24 smithi167 ceph-mon[21875]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-08-19T01:28:24.466 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:28:24 smithi167 ceph-mon[21875]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-08-19T01:28:25.164 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:28:25 smithi083 ceph-mon[17120]: pgmap v82: 1 pgs: 1 active+clean; 577 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-08-19T01:28:25.452 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:28:25 smithi117 ceph-mon[22016]: pgmap v82: 1 pgs: 1 active+clean; 577 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-08-19T01:28:25.466 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:28:25 smithi167 ceph-mon[21875]: pgmap v82: 1 pgs: 1 active+clean; 577 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-08-19T01:28:27.650 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:28:27 smithi083 ceph-mon[17120]: pgmap v83: 1 pgs: 1 active+clean; 577 KiB data, 95 MiB used, 268 GiB / 268 GiB avail 2024-08-19T01:28:27.650 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:28:27 smithi083 ceph-mon[17120]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-08-19T01:28:27.650 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:28:27 smithi083 ceph-mon[17120]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-19T01:28:27.703 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:28:27 smithi117 ceph-mon[22016]: pgmap v83: 1 pgs: 1 active+clean; 577 KiB data, 95 MiB used, 268 GiB / 268 GiB avail 2024-08-19T01:28:27.703 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:28:27 smithi117 ceph-mon[22016]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-08-19T01:28:27.703 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:28:27 smithi117 ceph-mon[22016]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-19T01:28:27.716 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:28:27 smithi167 ceph-mon[21875]: pgmap v83: 1 pgs: 1 active+clean; 577 KiB data, 95 MiB used, 268 GiB / 268 GiB avail 2024-08-19T01:28:27.716 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:28:27 smithi167 ceph-mon[21875]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-08-19T01:28:27.716 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:28:27 smithi167 ceph-mon[21875]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-19T01:28:28.402 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:28:28 smithi083 ceph-mon[17120]: Deploying daemon osd.3 on smithi083 2024-08-19T01:28:28.702 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:28:28 smithi117 ceph-mon[22016]: Deploying daemon osd.3 on smithi083 2024-08-19T01:28:28.716 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:28:28 smithi167 ceph-mon[21875]: Deploying daemon osd.3 on smithi083 2024-08-19T01:28:29.653 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:28:29 smithi083 ceph-mon[17120]: pgmap v84: 1 pgs: 1 active+clean; 577 KiB data, 95 MiB used, 268 GiB / 268 GiB avail; 2.2 KiB/s rd, 65 KiB/s wr, 5 op/s 2024-08-19T01:28:29.703 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:28:29 smithi117 ceph-mon[22016]: pgmap v84: 1 pgs: 1 active+clean; 577 KiB data, 95 MiB used, 268 GiB / 268 GiB avail; 2.2 KiB/s rd, 65 KiB/s wr, 5 op/s 2024-08-19T01:28:29.716 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:28:29 smithi167 ceph-mon[21875]: pgmap v84: 1 pgs: 1 active+clean; 577 KiB data, 95 MiB used, 268 GiB / 268 GiB avail; 2.2 KiB/s rd, 65 KiB/s wr, 5 op/s 2024-08-19T01:28:31.658 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:28:31 smithi083 ceph-mon[17120]: pgmap v85: 1 pgs: 1 active+clean; 577 KiB data, 95 MiB used, 268 GiB / 268 GiB avail; 1.5 KiB/s rd, 44 KiB/s wr, 3 op/s 2024-08-19T01:28:31.703 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:28:31 smithi117 ceph-mon[22016]: pgmap v85: 1 pgs: 1 active+clean; 577 KiB data, 95 MiB used, 268 GiB / 268 GiB avail; 1.5 KiB/s rd, 44 KiB/s wr, 3 op/s 2024-08-19T01:28:31.716 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:28:31 smithi167 ceph-mon[21875]: pgmap v85: 1 pgs: 1 active+clean; 577 KiB data, 95 MiB used, 268 GiB / 268 GiB avail; 1.5 KiB/s rd, 44 KiB/s wr, 3 op/s 2024-08-19T01:28:33.655 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:28:33 smithi083 ceph-mon[17120]: pgmap v86: 1 pgs: 1 active+clean; 577 KiB data, 95 MiB used, 268 GiB / 268 GiB avail; 1.1 KiB/s rd, 33 KiB/s wr, 2 op/s 2024-08-19T01:28:33.655 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:28:33 smithi083 ceph-mon[17120]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-19T01:28:33.655 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:28:33 smithi083 ceph-mon[17120]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' 2024-08-19T01:28:33.655 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:28:33 smithi083 ceph-mon[17120]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' 2024-08-19T01:28:33.702 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:28:33 smithi117 ceph-mon[22016]: pgmap v86: 1 pgs: 1 active+clean; 577 KiB data, 95 MiB used, 268 GiB / 268 GiB avail; 1.1 KiB/s rd, 33 KiB/s wr, 2 op/s 2024-08-19T01:28:33.703 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:28:33 smithi117 ceph-mon[22016]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-19T01:28:33.703 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:28:33 smithi117 ceph-mon[22016]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' 2024-08-19T01:28:33.703 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:28:33 smithi117 ceph-mon[22016]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' 2024-08-19T01:28:33.716 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:28:33 smithi167 ceph-mon[21875]: pgmap v86: 1 pgs: 1 active+clean; 577 KiB data, 95 MiB used, 268 GiB / 268 GiB avail; 1.1 KiB/s rd, 33 KiB/s wr, 2 op/s 2024-08-19T01:28:33.716 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:28:33 smithi167 ceph-mon[21875]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-19T01:28:33.716 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:28:33 smithi167 ceph-mon[21875]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' 2024-08-19T01:28:33.716 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:28:33 smithi167 ceph-mon[21875]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' 2024-08-19T01:28:35.636 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:28:35 smithi083 ceph-mon[17120]: pgmap v87: 1 pgs: 1 active+clean; 577 KiB data, 95 MiB used, 268 GiB / 268 GiB avail; 921 B/s rd, 26 KiB/s wr, 2 op/s 2024-08-19T01:28:35.703 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:28:35 smithi117 ceph-mon[22016]: pgmap v87: 1 pgs: 1 active+clean; 577 KiB data, 95 MiB used, 268 GiB / 268 GiB avail; 921 B/s rd, 26 KiB/s wr, 2 op/s 2024-08-19T01:28:35.716 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:28:35 smithi167 ceph-mon[21875]: pgmap v87: 1 pgs: 1 active+clean; 577 KiB data, 95 MiB used, 268 GiB / 268 GiB avail; 921 B/s rd, 26 KiB/s wr, 2 op/s 2024-08-19T01:28:37.156 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:28:37.154+0000 7f31ebe00640 1 -- 172.21.15.83:0/2693023465 <== mgr.14150 v2:172.21.15.83:6800/683740916 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f31f4062360 con 0x7f31d80a2b70 2024-08-19T01:28:37.157 INFO:teuthology.orchestra.run.smithi083.stdout:Created osd(s) 3 on host 'smithi083' 2024-08-19T01:28:37.159 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:28:37.156+0000 7f31fa600640 1 -- 172.21.15.83:0/2693023465 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f31d80a2b70 msgr2=0x7f31d80a5030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:28:37.159 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:28:37.156+0000 7f31fa600640 1 --2- 172.21.15.83:0/2693023465 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f31d80a2b70 0x7f31d80a5030 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f31e0031440 tx=0x7f31e00315b0 comp rx=0 tx=0).stop 2024-08-19T01:28:37.159 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:28:37.156+0000 7f31fa600640 1 -- 172.21.15.83:0/2693023465 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f31f4103b30 msgr2=0x7f31f4079cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:28:37.159 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:28:37.156+0000 7f31fa600640 1 --2- 172.21.15.83:0/2693023465 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f31f4103b30 0x7f31f4079cd0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f31e400e9c0 tx=0x7f31e400edc0 comp rx=0 tx=0).stop 2024-08-19T01:28:37.159 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:28:37.157+0000 7f31fa600640 1 -- 172.21.15.83:0/2693023465 shutdown_connections 2024-08-19T01:28:37.159 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:28:37.157+0000 7f31fa600640 1 --2- 172.21.15.83:0/2693023465 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f31d80a2b70 0x7f31d80a5030 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:28:37.159 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:28:37.157+0000 7f31fa600640 1 --2- 172.21.15.83:0/2693023465 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f31f41056f0 0x7f31f4076830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:28:37.159 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:28:37.157+0000 7f31fa600640 1 --2- 172.21.15.83:0/2693023465 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f31f4104d30 0x7f31f407a210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:28:37.159 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:28:37.157+0000 7f31fa600640 1 --2- 172.21.15.83:0/2693023465 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f31f4103b30 0x7f31f4079cd0 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:28:37.159 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:28:37.157+0000 7f31fa600640 1 -- 172.21.15.83:0/2693023465 >> 172.21.15.83:0/2693023465 conn(0x7f31f40ff2e0 msgr2=0x7f31f41080b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:28:37.159 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:28:37.157+0000 7f31fa600640 1 -- 172.21.15.83:0/2693023465 shutdown_connections 2024-08-19T01:28:37.160 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:28:37.157+0000 7f31fa600640 1 -- 172.21.15.83:0/2693023465 wait complete. 2024-08-19T01:28:37.405 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:28:37 smithi083 ceph-mon[17120]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' 2024-08-19T01:28:37.405 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:28:37 smithi083 ceph-mon[17120]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' 2024-08-19T01:28:37.405 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:28:37 smithi083 ceph-mon[17120]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-19T01:28:37.405 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:28:37 smithi083 ceph-mon[17120]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-19T01:28:37.405 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:28:37 smithi083 ceph-mon[17120]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' 2024-08-19T01:28:37.405 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:28:37 smithi083 ceph-mon[17120]: pgmap v88: 1 pgs: 1 active+clean; 577 KiB data, 95 MiB used, 268 GiB / 268 GiB avail; 767 B/s rd, 22 KiB/s wr, 1 op/s 2024-08-19T01:28:37.405 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:28:37 smithi083 ceph-mon[17120]: from='osd.3 [v2:172.21.15.83:6810/3138942062,v1:172.21.15.83:6811/3138942062]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-08-19T01:28:37.406 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:28:37 smithi083 ceph-mon[17120]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-19T01:28:37.406 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:28:37 smithi083 ceph-mon[17120]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' 2024-08-19T01:28:37.406 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:28:37 smithi083 ceph-mon[17120]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' 2024-08-19T01:28:37.453 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:28:37 smithi117 ceph-mon[22016]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' 2024-08-19T01:28:37.453 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:28:37 smithi117 ceph-mon[22016]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' 2024-08-19T01:28:37.453 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:28:37 smithi117 ceph-mon[22016]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-19T01:28:37.453 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:28:37 smithi117 ceph-mon[22016]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-19T01:28:37.453 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:28:37 smithi117 ceph-mon[22016]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' 2024-08-19T01:28:37.453 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:28:37 smithi117 ceph-mon[22016]: pgmap v88: 1 pgs: 1 active+clean; 577 KiB data, 95 MiB used, 268 GiB / 268 GiB avail; 767 B/s rd, 22 KiB/s wr, 1 op/s 2024-08-19T01:28:37.454 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:28:37 smithi117 ceph-mon[22016]: from='osd.3 [v2:172.21.15.83:6810/3138942062,v1:172.21.15.83:6811/3138942062]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-08-19T01:28:37.454 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:28:37 smithi117 ceph-mon[22016]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-19T01:28:37.454 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:28:37 smithi117 ceph-mon[22016]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' 2024-08-19T01:28:37.454 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:28:37 smithi117 ceph-mon[22016]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' 2024-08-19T01:28:37.467 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:28:37 smithi167 ceph-mon[21875]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' 2024-08-19T01:28:37.467 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:28:37 smithi167 ceph-mon[21875]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' 2024-08-19T01:28:37.467 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:28:37 smithi167 ceph-mon[21875]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-19T01:28:37.467 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:28:37 smithi167 ceph-mon[21875]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-19T01:28:37.467 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:28:37 smithi167 ceph-mon[21875]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' 2024-08-19T01:28:37.467 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:28:37 smithi167 ceph-mon[21875]: pgmap v88: 1 pgs: 1 active+clean; 577 KiB data, 95 MiB used, 268 GiB / 268 GiB avail; 767 B/s rd, 22 KiB/s wr, 1 op/s 2024-08-19T01:28:37.467 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:28:37 smithi167 ceph-mon[21875]: from='osd.3 [v2:172.21.15.83:6810/3138942062,v1:172.21.15.83:6811/3138942062]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-08-19T01:28:37.467 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:28:37 smithi167 ceph-mon[21875]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-19T01:28:37.467 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:28:37 smithi167 ceph-mon[21875]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' 2024-08-19T01:28:37.467 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:28:37 smithi167 ceph-mon[21875]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' 2024-08-19T01:28:37.852 DEBUG:teuthology.orchestra.run.smithi083:osd.3> sudo journalctl -f -n 0 -u ceph-bad8780e-5dc9-11ef-bccf-c7b262605968@osd.3.service 2024-08-19T01:28:37.856 INFO:tasks.cephadm:Deploying osd.4 on smithi117 with /dev/vg_nvme/lv_3... 2024-08-19T01:28:37.856 DEBUG:teuthology.orchestra.run.smithi117:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833 ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid bad8780e-5dc9-11ef-bccf-c7b262605968 -- lvm zap /dev/vg_nvme/lv_3 2024-08-19T01:28:38.114 INFO:teuthology.orchestra.run.smithi117.stderr:Inferring config /var/lib/ceph/bad8780e-5dc9-11ef-bccf-c7b262605968/mon.b/config 2024-08-19T01:28:38.464 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:28:38 smithi083 ceph-mon[17120]: from='osd.3 [v2:172.21.15.83:6810/3138942062,v1:172.21.15.83:6811/3138942062]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-08-19T01:28:38.464 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:28:38 smithi083 ceph-mon[17120]: osdmap e23: 4 total, 3 up, 4 in 2024-08-19T01:28:38.464 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:28:38 smithi083 ceph-mon[17120]: from='osd.3 [v2:172.21.15.83:6810/3138942062,v1:172.21.15.83:6811/3138942062]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi083", "root=default"]}]: dispatch 2024-08-19T01:28:38.464 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:28:38 smithi083 ceph-mon[17120]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-08-19T01:28:38.464 INFO:journalctl@ceph.osd.3.smithi083.stdout:Aug 19 01:28:38 smithi083 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-3[38049]: 2024-08-19T01:28:38.232+0000 7f499ee00640 -1 osd.3 0 waiting for initial osdmap 2024-08-19T01:28:38.464 INFO:journalctl@ceph.osd.3.smithi083.stdout:Aug 19 01:28:38 smithi083 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-3[38049]: 2024-08-19T01:28:38.238+0000 7f499a800640 -1 osd.3 24 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-08-19T01:28:38.535 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:28:38 smithi117 ceph-mon[22016]: from='osd.3 [v2:172.21.15.83:6810/3138942062,v1:172.21.15.83:6811/3138942062]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-08-19T01:28:38.535 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:28:38 smithi117 ceph-mon[22016]: osdmap e23: 4 total, 3 up, 4 in 2024-08-19T01:28:38.535 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:28:38 smithi117 ceph-mon[22016]: from='osd.3 [v2:172.21.15.83:6810/3138942062,v1:172.21.15.83:6811/3138942062]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi083", "root=default"]}]: dispatch 2024-08-19T01:28:38.535 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:28:38 smithi117 ceph-mon[22016]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-08-19T01:28:38.716 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:28:38 smithi167 ceph-mon[21875]: from='osd.3 [v2:172.21.15.83:6810/3138942062,v1:172.21.15.83:6811/3138942062]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-08-19T01:28:38.716 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:28:38 smithi167 ceph-mon[21875]: osdmap e23: 4 total, 3 up, 4 in 2024-08-19T01:28:38.716 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:28:38 smithi167 ceph-mon[21875]: from='osd.3 [v2:172.21.15.83:6810/3138942062,v1:172.21.15.83:6811/3138942062]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi083", "root=default"]}]: dispatch 2024-08-19T01:28:38.717 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:28:38 smithi167 ceph-mon[21875]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-08-19T01:28:39.356 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:28:39 smithi117 ceph-mon[22016]: from='osd.3 [v2:172.21.15.83:6810/3138942062,v1:172.21.15.83:6811/3138942062]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi083", "root=default"]}]': finished 2024-08-19T01:28:39.357 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:28:39 smithi117 ceph-mon[22016]: osdmap e24: 4 total, 3 up, 4 in 2024-08-19T01:28:39.357 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:28:39 smithi117 ceph-mon[22016]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-08-19T01:28:39.357 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:28:39 smithi117 ceph-mon[22016]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-08-19T01:28:39.357 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:28:39 smithi117 ceph-mon[22016]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-08-19T01:28:39.357 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:28:39 smithi117 ceph-mon[22016]: osd.3 [v2:172.21.15.83:6810/3138942062,v1:172.21.15.83:6811/3138942062] boot 2024-08-19T01:28:39.357 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:28:39 smithi117 ceph-mon[22016]: osdmap e25: 4 total, 4 up, 4 in 2024-08-19T01:28:39.357 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:28:39 smithi117 ceph-mon[22016]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-08-19T01:28:39.405 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:28:39 smithi083 ceph-mon[17120]: from='osd.3 [v2:172.21.15.83:6810/3138942062,v1:172.21.15.83:6811/3138942062]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi083", "root=default"]}]': finished 2024-08-19T01:28:39.405 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:28:39 smithi083 ceph-mon[17120]: osdmap e24: 4 total, 3 up, 4 in 2024-08-19T01:28:39.405 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:28:39 smithi083 ceph-mon[17120]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-08-19T01:28:39.405 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:28:39 smithi083 ceph-mon[17120]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-08-19T01:28:39.405 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:28:39 smithi083 ceph-mon[17120]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-08-19T01:28:39.405 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:28:39 smithi083 ceph-mon[17120]: osd.3 [v2:172.21.15.83:6810/3138942062,v1:172.21.15.83:6811/3138942062] boot 2024-08-19T01:28:39.405 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:28:39 smithi083 ceph-mon[17120]: osdmap e25: 4 total, 4 up, 4 in 2024-08-19T01:28:39.405 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:28:39 smithi083 ceph-mon[17120]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-08-19T01:28:39.716 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:28:39 smithi167 ceph-mon[21875]: from='osd.3 [v2:172.21.15.83:6810/3138942062,v1:172.21.15.83:6811/3138942062]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi083", "root=default"]}]': finished 2024-08-19T01:28:39.716 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:28:39 smithi167 ceph-mon[21875]: osdmap e24: 4 total, 3 up, 4 in 2024-08-19T01:28:39.716 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:28:39 smithi167 ceph-mon[21875]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-08-19T01:28:39.717 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:28:39 smithi167 ceph-mon[21875]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-08-19T01:28:39.717 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:28:39 smithi167 ceph-mon[21875]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-08-19T01:28:39.717 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:28:39 smithi167 ceph-mon[21875]: osd.3 [v2:172.21.15.83:6810/3138942062,v1:172.21.15.83:6811/3138942062] boot 2024-08-19T01:28:39.717 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:28:39 smithi167 ceph-mon[21875]: osdmap e25: 4 total, 4 up, 4 in 2024-08-19T01:28:39.717 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:28:39 smithi167 ceph-mon[21875]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-08-19T01:28:40.314 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-08-19T01:28:40.341 DEBUG:teuthology.orchestra.run.smithi117:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid bad8780e-5dc9-11ef-bccf-c7b262605968 -- ceph orch daemon add osd smithi117:vg_nvme/lv_3 2024-08-19T01:28:40.593 INFO:teuthology.orchestra.run.smithi117.stderr:Inferring config /var/lib/ceph/bad8780e-5dc9-11ef-bccf-c7b262605968/mon.b/config 2024-08-19T01:28:40.861 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:28:40 smithi117 ceph-mon[22016]: purged_snaps scrub starts 2024-08-19T01:28:40.862 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:28:40 smithi117 ceph-mon[22016]: purged_snaps scrub ok 2024-08-19T01:28:40.862 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:28:40 smithi117 ceph-mon[22016]: osdmap e26: 4 total, 4 up, 4 in 2024-08-19T01:28:40.862 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:28:40 smithi117 ceph-mon[22016]: pgmap v94: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-08-19T01:28:40.905 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:28:40 smithi083 ceph-mon[17120]: purged_snaps scrub starts 2024-08-19T01:28:40.905 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:28:40 smithi083 ceph-mon[17120]: purged_snaps scrub ok 2024-08-19T01:28:40.905 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:28:40 smithi083 ceph-mon[17120]: osdmap e26: 4 total, 4 up, 4 in 2024-08-19T01:28:40.905 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:28:40 smithi083 ceph-mon[17120]: pgmap v94: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-08-19T01:28:40.966 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:28:40 smithi167 ceph-mon[21875]: purged_snaps scrub starts 2024-08-19T01:28:40.966 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:28:40 smithi167 ceph-mon[21875]: purged_snaps scrub ok 2024-08-19T01:28:40.966 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:28:40 smithi167 ceph-mon[21875]: osdmap e26: 4 total, 4 up, 4 in 2024-08-19T01:28:40.967 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:28:40 smithi167 ceph-mon[21875]: pgmap v94: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-08-19T01:28:41.066 INFO:teuthology.orchestra.run.smithi117.stderr:2024-08-19T01:28:41.049+0000 7f7011a3f640 1 -- 172.21.15.117:0/1385727853 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f700c1023c0 msgr2=0x7f700c108c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:28:41.066 INFO:teuthology.orchestra.run.smithi117.stderr:2024-08-19T01:28:41.049+0000 7f7011a3f640 1 --2- 172.21.15.117:0/1385727853 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f700c1023c0 0x7f700c108c50 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f6ffc009900 tx=0x7f6ffc02a410 comp rx=0 tx=0).stop 2024-08-19T01:28:41.066 INFO:teuthology.orchestra.run.smithi117.stderr:2024-08-19T01:28:41.049+0000 7f7011a3f640 1 -- 172.21.15.117:0/1385727853 shutdown_connections 2024-08-19T01:28:41.066 INFO:teuthology.orchestra.run.smithi117.stderr:2024-08-19T01:28:41.049+0000 7f7011a3f640 1 --2- 172.21.15.117:0/1385727853 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f700c1023c0 0x7f700c108c50 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:28:41.066 INFO:teuthology.orchestra.run.smithi117.stderr:2024-08-19T01:28:41.049+0000 7f7011a3f640 1 --2- 172.21.15.117:0/1385727853 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f700c101a00 0x7f700c101e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:28:41.066 INFO:teuthology.orchestra.run.smithi117.stderr:2024-08-19T01:28:41.049+0000 7f7011a3f640 1 --2- 172.21.15.117:0/1385727853 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f700c100800 0x7f700c100c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:28:41.066 INFO:teuthology.orchestra.run.smithi117.stderr:2024-08-19T01:28:41.049+0000 7f7011a3f640 1 -- 172.21.15.117:0/1385727853 >> 172.21.15.117:0/1385727853 conn(0x7f700c0fbfb0 msgr2=0x7f700c0fe3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:28:41.066 INFO:teuthology.orchestra.run.smithi117.stderr:2024-08-19T01:28:41.049+0000 7f7011a3f640 1 -- 172.21.15.117:0/1385727853 shutdown_connections 2024-08-19T01:28:41.067 INFO:teuthology.orchestra.run.smithi117.stderr:2024-08-19T01:28:41.049+0000 7f7011a3f640 1 -- 172.21.15.117:0/1385727853 wait complete. 2024-08-19T01:28:41.067 INFO:teuthology.orchestra.run.smithi117.stderr:2024-08-19T01:28:41.049+0000 7f7011a3f640 1 Processor -- start 2024-08-19T01:28:41.067 INFO:teuthology.orchestra.run.smithi117.stderr:2024-08-19T01:28:41.050+0000 7f7011a3f640 1 -- start start 2024-08-19T01:28:41.067 INFO:teuthology.orchestra.run.smithi117.stderr:2024-08-19T01:28:41.050+0000 7f7011a3f640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f700c100800 0x7f700c197c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:28:41.067 INFO:teuthology.orchestra.run.smithi117.stderr:2024-08-19T01:28:41.050+0000 7f7011a3f640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f700c101a00 0x7f700c1981a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:28:41.067 INFO:teuthology.orchestra.run.smithi117.stderr:2024-08-19T01:28:41.050+0000 7f7011a3f640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f700c1023c0 0x7f700c19f1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:28:41.067 INFO:teuthology.orchestra.run.smithi117.stderr:2024-08-19T01:28:41.050+0000 7f7011a3f640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f700c10c500 con 0x7f700c100800 2024-08-19T01:28:41.067 INFO:teuthology.orchestra.run.smithi117.stderr:2024-08-19T01:28:41.050+0000 7f7011a3f640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f700c10c380 con 0x7f700c1023c0 2024-08-19T01:28:41.067 INFO:teuthology.orchestra.run.smithi117.stderr:2024-08-19T01:28:41.050+0000 7f7011a3f640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f700c10c680 con 0x7f700c101a00 2024-08-19T01:28:41.067 INFO:teuthology.orchestra.run.smithi117.stderr:2024-08-19T01:28:41.050+0000 7f700b7fe640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f700c1023c0 0x7f700c19f1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:28:41.067 INFO:teuthology.orchestra.run.smithi117.stderr:2024-08-19T01:28:41.050+0000 7f700b7fe640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f700c1023c0 0x7f700c19f1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:56706/0 (socket says 172.21.15.117:56706) 2024-08-19T01:28:41.068 INFO:teuthology.orchestra.run.smithi117.stderr:2024-08-19T01:28:41.050+0000 7f700b7fe640 1 -- 172.21.15.117:0/2855049453 learned_addr learned my addr 172.21.15.117:0/2855049453 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-08-19T01:28:41.068 INFO:teuthology.orchestra.run.smithi117.stderr:2024-08-19T01:28:41.050+0000 7f700affd640 1 --2- 172.21.15.117:0/2855049453 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f700c100800 0x7f700c197c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:28:41.068 INFO:teuthology.orchestra.run.smithi117.stderr:2024-08-19T01:28:41.050+0000 7f700b7fe640 1 -- 172.21.15.117:0/2855049453 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f700c101a00 msgr2=0x7f700c1981a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:28:41.068 INFO:teuthology.orchestra.run.smithi117.stderr:2024-08-19T01:28:41.050+0000 7f700a7fc640 1 --2- 172.21.15.117:0/2855049453 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f700c101a00 0x7f700c1981a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:28:41.068 INFO:teuthology.orchestra.run.smithi117.stderr:2024-08-19T01:28:41.050+0000 7f700b7fe640 1 --2- 172.21.15.117:0/2855049453 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f700c101a00 0x7f700c1981a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:28:41.068 INFO:teuthology.orchestra.run.smithi117.stderr:2024-08-19T01:28:41.050+0000 7f700b7fe640 1 -- 172.21.15.117:0/2855049453 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f700c100800 msgr2=0x7f700c197c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:28:41.068 INFO:teuthology.orchestra.run.smithi117.stderr:2024-08-19T01:28:41.050+0000 7f700b7fe640 1 --2- 172.21.15.117:0/2855049453 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f700c100800 0x7f700c197c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:28:41.068 INFO:teuthology.orchestra.run.smithi117.stderr:2024-08-19T01:28:41.050+0000 7f700b7fe640 1 -- 172.21.15.117:0/2855049453 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f700c19f880 con 0x7f700c1023c0 2024-08-19T01:28:41.068 INFO:teuthology.orchestra.run.smithi117.stderr:2024-08-19T01:28:41.051+0000 7f700b7fe640 1 --2- 172.21.15.117:0/2855049453 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f700c1023c0 0x7f700c19f1e0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f6ffc0314e0 tx=0x7f6ffc004360 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:28:41.068 INFO:teuthology.orchestra.run.smithi117.stderr:2024-08-19T01:28:41.051+0000 7f7010a3d640 1 -- 172.21.15.117:0/2855049453 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ffc03f030 con 0x7f700c1023c0 2024-08-19T01:28:41.068 INFO:teuthology.orchestra.run.smithi117.stderr:2024-08-19T01:28:41.051+0000 7f7010a3d640 1 -- 172.21.15.117:0/2855049453 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6ffc003ce0 con 0x7f700c1023c0 2024-08-19T01:28:41.068 INFO:teuthology.orchestra.run.smithi117.stderr:2024-08-19T01:28:41.051+0000 7f7011a3f640 1 -- 172.21.15.117:0/2855049453 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f700c19fb10 con 0x7f700c1023c0 2024-08-19T01:28:41.069 INFO:teuthology.orchestra.run.smithi117.stderr:2024-08-19T01:28:41.051+0000 7f7010a3d640 1 -- 172.21.15.117:0/2855049453 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ffc035070 con 0x7f700c1023c0 2024-08-19T01:28:41.069 INFO:teuthology.orchestra.run.smithi117.stderr:2024-08-19T01:28:41.051+0000 7f7011a3f640 1 -- 172.21.15.117:0/2855049453 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f700c19fff0 con 0x7f700c1023c0 2024-08-19T01:28:41.069 INFO:teuthology.orchestra.run.smithi117.stderr:2024-08-19T01:28:41.052+0000 7f7011a3f640 1 -- 172.21.15.117:0/2855049453 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f700c1a0280 con 0x7f700c1023c0 2024-08-19T01:28:41.069 INFO:teuthology.orchestra.run.smithi117.stderr:2024-08-19T01:28:41.053+0000 7f7010a3d640 1 -- 172.21.15.117:0/2855049453 <== mon.2 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f6ffc042030 con 0x7f700c1023c0 2024-08-19T01:28:41.069 INFO:teuthology.orchestra.run.smithi117.stderr:2024-08-19T01:28:41.054+0000 7f7010a3d640 1 --2- 172.21.15.117:0/2855049453 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f6ff40a2b80 0x7f6ff40a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:28:41.069 INFO:teuthology.orchestra.run.smithi117.stderr:2024-08-19T01:28:41.054+0000 7f7010a3d640 1 -- 172.21.15.117:0/2855049453 <== mon.2 v2:172.21.15.117:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 3698+0+0 (secure 0 0 0) 0x7f6ffc0ee7f0 con 0x7f700c1023c0 2024-08-19T01:28:41.069 INFO:teuthology.orchestra.run.smithi117.stderr:2024-08-19T01:28:41.054+0000 7f700affd640 1 --2- 172.21.15.117:0/2855049453 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f6ff40a2b80 0x7f6ff40a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:28:41.069 INFO:teuthology.orchestra.run.smithi117.stderr:2024-08-19T01:28:41.055+0000 7f700affd640 1 --2- 172.21.15.117:0/2855049453 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f6ff40a2b80 0x7f6ff40a5040 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f700c199140 tx=0x7f700400b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:28:41.069 INFO:teuthology.orchestra.run.smithi117.stderr:2024-08-19T01:28:41.058+0000 7f7010a3d640 1 -- 172.21.15.117:0/2855049453 <== mon.2 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7f6ffc0bc950 con 0x7f700c1023c0 2024-08-19T01:28:41.239 INFO:teuthology.orchestra.run.smithi117.stderr:2024-08-19T01:28:41.238+0000 7f7011a3f640 1 -- 172.21.15.117:0/2855049453 --> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi117:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7f700c06a3e0 con 0x7f6ff40a2b80 2024-08-19T01:28:41.869 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:28:41 smithi117 ceph-mon[22016]: osdmap e27: 4 total, 4 up, 4 in 2024-08-19T01:28:41.869 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:28:41 smithi117 ceph-mon[22016]: from='client.24230 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi117:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-08-19T01:28:41.869 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:28:41 smithi117 ceph-mon[22016]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-08-19T01:28:41.869 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:28:41 smithi117 ceph-mon[22016]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-08-19T01:28:41.869 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:28:41 smithi117 ceph-mon[22016]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-19T01:28:41.869 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:28:41 smithi117 ceph-mon[22016]: Detected new or changed devices on smithi083 2024-08-19T01:28:41.869 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:28:41 smithi117 ceph-mon[22016]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' 2024-08-19T01:28:41.870 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:28:41 smithi117 ceph-mon[22016]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' 2024-08-19T01:28:41.870 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:28:41 smithi117 ceph-mon[22016]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-08-19T01:28:41.870 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:28:41 smithi117 ceph-mon[22016]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-08-19T01:28:41.870 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:28:41 smithi117 ceph-mon[22016]: Adjusting osd_memory_target on smithi083 to 8544M 2024-08-19T01:28:41.870 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:28:41 smithi117 ceph-mon[22016]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' 2024-08-19T01:28:41.870 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:28:41 smithi117 ceph-mon[22016]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-19T01:28:41.870 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:28:41 smithi117 ceph-mon[22016]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-19T01:28:41.870 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:28:41 smithi117 ceph-mon[22016]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' 2024-08-19T01:28:41.966 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:28:41 smithi167 ceph-mon[21875]: osdmap e27: 4 total, 4 up, 4 in 2024-08-19T01:28:41.967 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:28:41 smithi167 ceph-mon[21875]: from='client.24230 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi117:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-08-19T01:28:41.967 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:28:41 smithi167 ceph-mon[21875]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-08-19T01:28:41.967 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:28:41 smithi167 ceph-mon[21875]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-08-19T01:28:41.967 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:28:41 smithi167 ceph-mon[21875]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-19T01:28:41.967 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:28:41 smithi167 ceph-mon[21875]: Detected new or changed devices on smithi083 2024-08-19T01:28:41.967 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:28:41 smithi167 ceph-mon[21875]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' 2024-08-19T01:28:41.967 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:28:41 smithi167 ceph-mon[21875]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' 2024-08-19T01:28:41.967 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:28:41 smithi167 ceph-mon[21875]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-08-19T01:28:41.967 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:28:41 smithi167 ceph-mon[21875]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-08-19T01:28:41.967 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:28:41 smithi167 ceph-mon[21875]: Adjusting osd_memory_target on smithi083 to 8544M 2024-08-19T01:28:41.967 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:28:41 smithi167 ceph-mon[21875]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' 2024-08-19T01:28:41.967 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:28:41 smithi167 ceph-mon[21875]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-19T01:28:41.967 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:28:41 smithi167 ceph-mon[21875]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-19T01:28:41.968 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:28:41 smithi167 ceph-mon[21875]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' 2024-08-19T01:28:42.155 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:28:41 smithi083 ceph-mon[17120]: osdmap e27: 4 total, 4 up, 4 in 2024-08-19T01:28:42.155 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:28:41 smithi083 ceph-mon[17120]: from='client.24230 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi117:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-08-19T01:28:42.155 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:28:41 smithi083 ceph-mon[17120]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-08-19T01:28:42.155 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:28:41 smithi083 ceph-mon[17120]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-08-19T01:28:42.155 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:28:41 smithi083 ceph-mon[17120]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-19T01:28:42.155 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:28:41 smithi083 ceph-mon[17120]: Detected new or changed devices on smithi083 2024-08-19T01:28:42.155 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:28:41 smithi083 ceph-mon[17120]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' 2024-08-19T01:28:42.155 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:28:41 smithi083 ceph-mon[17120]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' 2024-08-19T01:28:42.156 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:28:41 smithi083 ceph-mon[17120]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-08-19T01:28:42.156 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:28:41 smithi083 ceph-mon[17120]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-08-19T01:28:42.156 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:28:41 smithi083 ceph-mon[17120]: Adjusting osd_memory_target on smithi083 to 8544M 2024-08-19T01:28:42.156 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:28:41 smithi083 ceph-mon[17120]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' 2024-08-19T01:28:42.156 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:28:41 smithi083 ceph-mon[17120]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-19T01:28:42.156 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:28:41 smithi083 ceph-mon[17120]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-19T01:28:42.156 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:28:41 smithi083 ceph-mon[17120]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' 2024-08-19T01:28:42.851 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:28:42 smithi117 ceph-mon[22016]: osdmap e28: 4 total, 4 up, 4 in 2024-08-19T01:28:42.852 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:28:42 smithi117 ceph-mon[22016]: pgmap v97: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-08-19T01:28:43.155 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:28:42 smithi083 ceph-mon[17120]: osdmap e28: 4 total, 4 up, 4 in 2024-08-19T01:28:43.155 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:28:42 smithi083 ceph-mon[17120]: pgmap v97: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-08-19T01:28:43.216 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:28:42 smithi167 ceph-mon[21875]: osdmap e28: 4 total, 4 up, 4 in 2024-08-19T01:28:43.216 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:28:42 smithi167 ceph-mon[21875]: pgmap v97: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-08-19T01:28:44.110 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:28:43 smithi117 ceph-mon[22016]: osdmap e29: 4 total, 4 up, 4 in 2024-08-19T01:28:44.110 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:28:43 smithi117 ceph-mon[22016]: from='client.? 172.21.15.117:0/710684744' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "8176ed18-1fe2-446d-8994-8c60f6e08f38"}]: dispatch 2024-08-19T01:28:44.110 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:28:43 smithi117 ceph-mon[22016]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "8176ed18-1fe2-446d-8994-8c60f6e08f38"}]: dispatch 2024-08-19T01:28:44.110 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:28:43 smithi117 ceph-mon[22016]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "8176ed18-1fe2-446d-8994-8c60f6e08f38"}]': finished 2024-08-19T01:28:44.110 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:28:43 smithi117 ceph-mon[22016]: osdmap e30: 5 total, 4 up, 5 in 2024-08-19T01:28:44.110 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:28:43 smithi117 ceph-mon[22016]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-08-19T01:28:44.155 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:28:43 smithi083 ceph-mon[17120]: osdmap e29: 4 total, 4 up, 4 in 2024-08-19T01:28:44.155 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:28:43 smithi083 ceph-mon[17120]: from='client.? 172.21.15.117:0/710684744' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "8176ed18-1fe2-446d-8994-8c60f6e08f38"}]: dispatch 2024-08-19T01:28:44.155 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:28:43 smithi083 ceph-mon[17120]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "8176ed18-1fe2-446d-8994-8c60f6e08f38"}]: dispatch 2024-08-19T01:28:44.155 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:28:43 smithi083 ceph-mon[17120]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "8176ed18-1fe2-446d-8994-8c60f6e08f38"}]': finished 2024-08-19T01:28:44.155 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:28:43 smithi083 ceph-mon[17120]: osdmap e30: 5 total, 4 up, 5 in 2024-08-19T01:28:44.155 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:28:43 smithi083 ceph-mon[17120]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-08-19T01:28:44.216 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:28:43 smithi167 ceph-mon[21875]: osdmap e29: 4 total, 4 up, 4 in 2024-08-19T01:28:44.216 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:28:43 smithi167 ceph-mon[21875]: from='client.? 172.21.15.117:0/710684744' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "8176ed18-1fe2-446d-8994-8c60f6e08f38"}]: dispatch 2024-08-19T01:28:44.216 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:28:43 smithi167 ceph-mon[21875]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "8176ed18-1fe2-446d-8994-8c60f6e08f38"}]: dispatch 2024-08-19T01:28:44.216 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:28:43 smithi167 ceph-mon[21875]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "8176ed18-1fe2-446d-8994-8c60f6e08f38"}]': finished 2024-08-19T01:28:44.217 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:28:43 smithi167 ceph-mon[21875]: osdmap e30: 5 total, 4 up, 5 in 2024-08-19T01:28:44.217 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:28:43 smithi167 ceph-mon[21875]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-08-19T01:28:45.056 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:28:44 smithi083 ceph-mon[17120]: from='client.? 172.21.15.117:0/2652129118' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-08-19T01:28:45.057 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:28:44 smithi083 ceph-mon[17120]: pgmap v100: 1 pgs: 1 active+clean; 577 KiB data, 121 MiB used, 357 GiB / 358 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-08-19T01:28:45.203 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:28:44 smithi117 ceph-mon[22016]: from='client.? 172.21.15.117:0/2652129118' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-08-19T01:28:45.203 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:28:44 smithi117 ceph-mon[22016]: pgmap v100: 1 pgs: 1 active+clean; 577 KiB data, 121 MiB used, 357 GiB / 358 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-08-19T01:28:45.216 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:28:44 smithi167 ceph-mon[21875]: from='client.? 172.21.15.117:0/2652129118' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-08-19T01:28:45.216 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:28:44 smithi167 ceph-mon[21875]: pgmap v100: 1 pgs: 1 active+clean; 577 KiB data, 121 MiB used, 357 GiB / 358 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-08-19T01:28:47.655 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:28:47 smithi083 ceph-mon[17120]: pgmap v101: 1 pgs: 1 active+clean; 577 KiB data, 121 MiB used, 357 GiB / 358 GiB avail; 103 KiB/s, 0 objects/s recovering 2024-08-19T01:28:47.682 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:28:47 smithi117 ceph-mon[22016]: pgmap v101: 1 pgs: 1 active+clean; 577 KiB data, 121 MiB used, 357 GiB / 358 GiB avail; 103 KiB/s, 0 objects/s recovering 2024-08-19T01:28:47.716 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:28:47 smithi167 ceph-mon[21875]: pgmap v101: 1 pgs: 1 active+clean; 577 KiB data, 121 MiB used, 357 GiB / 358 GiB avail; 103 KiB/s, 0 objects/s recovering 2024-08-19T01:28:49.655 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:28:49 smithi083 ceph-mon[17120]: pgmap v102: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail; 88 KiB/s, 0 objects/s recovering 2024-08-19T01:28:49.683 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:28:49 smithi117 ceph-mon[22016]: pgmap v102: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail; 88 KiB/s, 0 objects/s recovering 2024-08-19T01:28:49.716 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:28:49 smithi167 ceph-mon[21875]: pgmap v102: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail; 88 KiB/s, 0 objects/s recovering 2024-08-19T01:28:51.452 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:28:51 smithi117 ceph-mon[22016]: pgmap v103: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-08-19T01:28:51.453 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:28:51 smithi117 ceph-mon[22016]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-08-19T01:28:51.453 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:28:51 smithi117 ceph-mon[22016]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-19T01:28:51.655 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:28:51 smithi083 ceph-mon[17120]: pgmap v103: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-08-19T01:28:51.655 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:28:51 smithi083 ceph-mon[17120]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-08-19T01:28:51.655 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:28:51 smithi083 ceph-mon[17120]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-19T01:28:51.716 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:28:51 smithi167 ceph-mon[21875]: pgmap v103: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-08-19T01:28:51.716 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:28:51 smithi167 ceph-mon[21875]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-08-19T01:28:51.717 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:28:51 smithi167 ceph-mon[21875]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-19T01:28:52.655 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:28:52 smithi083 ceph-mon[17120]: Deploying daemon osd.4 on smithi117 2024-08-19T01:28:52.702 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:28:52 smithi117 ceph-mon[22016]: Deploying daemon osd.4 on smithi117 2024-08-19T01:28:52.716 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:28:52 smithi167 ceph-mon[21875]: Deploying daemon osd.4 on smithi117 2024-08-19T01:28:53.642 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:28:53 smithi117 ceph-mon[22016]: pgmap v104: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail; 60 KiB/s, 0 objects/s recovering 2024-08-19T01:28:53.654 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:28:53 smithi083 ceph-mon[17120]: pgmap v104: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail; 60 KiB/s, 0 objects/s recovering 2024-08-19T01:28:53.716 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:28:53 smithi167 ceph-mon[21875]: pgmap v104: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail; 60 KiB/s, 0 objects/s recovering 2024-08-19T01:28:55.655 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:28:55 smithi083 ceph-mon[17120]: pgmap v105: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-08-19T01:28:55.655 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:28:55 smithi083 ceph-mon[17120]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-19T01:28:55.655 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:28:55 smithi083 ceph-mon[17120]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' 2024-08-19T01:28:55.655 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:28:55 smithi083 ceph-mon[17120]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' 2024-08-19T01:28:55.660 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:28:55 smithi117 ceph-mon[22016]: pgmap v105: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-08-19T01:28:55.660 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:28:55 smithi117 ceph-mon[22016]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-19T01:28:55.661 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:28:55 smithi117 ceph-mon[22016]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' 2024-08-19T01:28:55.661 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:28:55 smithi117 ceph-mon[22016]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' 2024-08-19T01:28:55.716 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:28:55 smithi167 ceph-mon[21875]: pgmap v105: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-08-19T01:28:55.716 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:28:55 smithi167 ceph-mon[21875]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-19T01:28:55.716 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:28:55 smithi167 ceph-mon[21875]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' 2024-08-19T01:28:55.716 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:28:55 smithi167 ceph-mon[21875]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' 2024-08-19T01:28:57.639 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:28:57 smithi117 ceph-mon[22016]: pgmap v106: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-08-19T01:28:57.654 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:28:57 smithi083 ceph-mon[17120]: pgmap v106: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-08-19T01:28:57.716 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:28:57 smithi167 ceph-mon[21875]: pgmap v106: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-08-19T01:28:58.404 INFO:teuthology.orchestra.run.smithi117.stderr:2024-08-19T01:28:58.402+0000 7f7010a3d640 1 -- 172.21.15.117:0/2855049453 <== mgr.14150 v2:172.21.15.83:6800/683740916 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f700c06a3e0 con 0x7f6ff40a2b80 2024-08-19T01:28:58.404 INFO:teuthology.orchestra.run.smithi117.stdout:Created osd(s) 4 on host 'smithi117' 2024-08-19T01:28:58.406 INFO:teuthology.orchestra.run.smithi117.stderr:2024-08-19T01:28:58.404+0000 7f7011a3f640 1 -- 172.21.15.117:0/2855049453 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f6ff40a2b80 msgr2=0x7f6ff40a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:28:58.406 INFO:teuthology.orchestra.run.smithi117.stderr:2024-08-19T01:28:58.404+0000 7f7011a3f640 1 --2- 172.21.15.117:0/2855049453 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f6ff40a2b80 0x7f6ff40a5040 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f700c199140 tx=0x7f700400b390 comp rx=0 tx=0).stop 2024-08-19T01:28:58.406 INFO:teuthology.orchestra.run.smithi117.stderr:2024-08-19T01:28:58.404+0000 7f7011a3f640 1 -- 172.21.15.117:0/2855049453 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f700c1023c0 msgr2=0x7f700c19f1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:28:58.406 INFO:teuthology.orchestra.run.smithi117.stderr:2024-08-19T01:28:58.404+0000 7f7011a3f640 1 --2- 172.21.15.117:0/2855049453 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f700c1023c0 0x7f700c19f1e0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f6ffc0314e0 tx=0x7f6ffc004360 comp rx=0 tx=0).stop 2024-08-19T01:28:58.407 INFO:teuthology.orchestra.run.smithi117.stderr:2024-08-19T01:28:58.404+0000 7f7011a3f640 1 -- 172.21.15.117:0/2855049453 shutdown_connections 2024-08-19T01:28:58.407 INFO:teuthology.orchestra.run.smithi117.stderr:2024-08-19T01:28:58.405+0000 7f7011a3f640 1 --2- 172.21.15.117:0/2855049453 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f6ff40a2b80 0x7f6ff40a5040 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:28:58.407 INFO:teuthology.orchestra.run.smithi117.stderr:2024-08-19T01:28:58.405+0000 7f7011a3f640 1 --2- 172.21.15.117:0/2855049453 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f700c1023c0 0x7f700c19f1e0 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:28:58.407 INFO:teuthology.orchestra.run.smithi117.stderr:2024-08-19T01:28:58.405+0000 7f7011a3f640 1 --2- 172.21.15.117:0/2855049453 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f700c101a00 0x7f700c1981a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:28:58.407 INFO:teuthology.orchestra.run.smithi117.stderr:2024-08-19T01:28:58.405+0000 7f7011a3f640 1 --2- 172.21.15.117:0/2855049453 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f700c100800 0x7f700c197c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:28:58.407 INFO:teuthology.orchestra.run.smithi117.stderr:2024-08-19T01:28:58.405+0000 7f7011a3f640 1 -- 172.21.15.117:0/2855049453 >> 172.21.15.117:0/2855049453 conn(0x7f700c0fbfb0 msgr2=0x7f700c069cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:28:58.407 INFO:teuthology.orchestra.run.smithi117.stderr:2024-08-19T01:28:58.405+0000 7f7011a3f640 1 -- 172.21.15.117:0/2855049453 shutdown_connections 2024-08-19T01:28:58.407 INFO:teuthology.orchestra.run.smithi117.stderr:2024-08-19T01:28:58.405+0000 7f7011a3f640 1 -- 172.21.15.117:0/2855049453 wait complete. 2024-08-19T01:28:58.703 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:28:58 smithi117 ceph-mon[22016]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' 2024-08-19T01:28:58.703 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:28:58 smithi117 ceph-mon[22016]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' 2024-08-19T01:28:58.703 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:28:58 smithi117 ceph-mon[22016]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-19T01:28:58.704 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:28:58 smithi117 ceph-mon[22016]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-19T01:28:58.704 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:28:58 smithi117 ceph-mon[22016]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' 2024-08-19T01:28:58.704 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:28:58 smithi117 ceph-mon[22016]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-08-19T01:28:58.704 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:28:58 smithi117 ceph-mon[22016]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-19T01:28:58.704 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:28:58 smithi117 ceph-mon[22016]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' 2024-08-19T01:28:58.704 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:28:58 smithi117 ceph-mon[22016]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' 2024-08-19T01:28:58.905 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:28:58 smithi083 ceph-mon[17120]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' 2024-08-19T01:28:58.905 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:28:58 smithi083 ceph-mon[17120]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' 2024-08-19T01:28:58.905 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:28:58 smithi083 ceph-mon[17120]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-19T01:28:58.905 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:28:58 smithi083 ceph-mon[17120]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-19T01:28:58.905 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:28:58 smithi083 ceph-mon[17120]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' 2024-08-19T01:28:58.905 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:28:58 smithi083 ceph-mon[17120]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-08-19T01:28:58.905 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:28:58 smithi083 ceph-mon[17120]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-19T01:28:58.905 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:28:58 smithi083 ceph-mon[17120]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' 2024-08-19T01:28:58.906 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:28:58 smithi083 ceph-mon[17120]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' 2024-08-19T01:28:58.966 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:28:58 smithi167 ceph-mon[21875]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' 2024-08-19T01:28:58.967 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:28:58 smithi167 ceph-mon[21875]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' 2024-08-19T01:28:58.967 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:28:58 smithi167 ceph-mon[21875]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-19T01:28:58.967 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:28:58 smithi167 ceph-mon[21875]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-19T01:28:58.967 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:28:58 smithi167 ceph-mon[21875]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' 2024-08-19T01:28:58.967 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:28:58 smithi167 ceph-mon[21875]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-08-19T01:28:58.967 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:28:58 smithi167 ceph-mon[21875]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-19T01:28:58.967 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:28:58 smithi167 ceph-mon[21875]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' 2024-08-19T01:28:58.967 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:28:58 smithi167 ceph-mon[21875]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' 2024-08-19T01:28:59.030 DEBUG:teuthology.orchestra.run.smithi117:osd.4> sudo journalctl -f -n 0 -u ceph-bad8780e-5dc9-11ef-bccf-c7b262605968@osd.4.service 2024-08-19T01:28:59.032 INFO:tasks.cephadm:Deploying osd.5 on smithi167 with /dev/vg_nvme/lv_3... 2024-08-19T01:28:59.032 DEBUG:teuthology.orchestra.run.smithi167:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833 ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid bad8780e-5dc9-11ef-bccf-c7b262605968 -- lvm zap /dev/vg_nvme/lv_3 2024-08-19T01:28:59.291 INFO:teuthology.orchestra.run.smithi167.stderr:Inferring config /var/lib/ceph/bad8780e-5dc9-11ef-bccf-c7b262605968/mon.c/config 2024-08-19T01:28:59.335 INFO:journalctl@ceph.osd.4.smithi117.stdout:Aug 19 01:28:59 smithi117 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-4[32888]: 2024-08-19T01:28:59.159+0000 7fa6ffe26640 -1 osd.4 0 log_to_monitors true 2024-08-19T01:28:59.592 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:28:59 smithi117 ceph-mon[22016]: from='osd.4 [v2:172.21.15.117:6808/3978863827,v1:172.21.15.117:6809/3978863827]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-08-19T01:28:59.592 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:28:59 smithi117 ceph-mon[22016]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-08-19T01:28:59.666 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:28:59 smithi167 ceph-mon[21875]: from='osd.4 [v2:172.21.15.117:6808/3978863827,v1:172.21.15.117:6809/3978863827]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-08-19T01:28:59.666 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:28:59 smithi167 ceph-mon[21875]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-08-19T01:28:59.905 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:28:59 smithi083 ceph-mon[17120]: from='osd.4 [v2:172.21.15.117:6808/3978863827,v1:172.21.15.117:6809/3978863827]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-08-19T01:28:59.905 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:28:59 smithi083 ceph-mon[17120]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-08-19T01:29:00.905 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:29:00 smithi083 ceph-mon[17120]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-08-19T01:29:00.905 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:29:00 smithi083 ceph-mon[17120]: from='osd.4 [v2:172.21.15.117:6808/3978863827,v1:172.21.15.117:6809/3978863827]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi117", "root=default"]}]: dispatch 2024-08-19T01:29:00.905 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:29:00 smithi083 ceph-mon[17120]: osdmap e31: 5 total, 4 up, 5 in 2024-08-19T01:29:00.905 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:29:00 smithi083 ceph-mon[17120]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-08-19T01:29:00.905 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:29:00 smithi083 ceph-mon[17120]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi117", "root=default"]}]: dispatch 2024-08-19T01:29:00.905 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:29:00 smithi083 ceph-mon[17120]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-08-19T01:29:00.931 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:29:00 smithi167 ceph-mon[21875]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-08-19T01:29:00.931 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:29:00 smithi167 ceph-mon[21875]: from='osd.4 [v2:172.21.15.117:6808/3978863827,v1:172.21.15.117:6809/3978863827]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi117", "root=default"]}]: dispatch 2024-08-19T01:29:00.931 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:29:00 smithi167 ceph-mon[21875]: osdmap e31: 5 total, 4 up, 5 in 2024-08-19T01:29:00.931 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:29:00 smithi167 ceph-mon[21875]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-08-19T01:29:00.931 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:29:00 smithi167 ceph-mon[21875]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi117", "root=default"]}]: dispatch 2024-08-19T01:29:00.931 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:29:00 smithi167 ceph-mon[21875]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-08-19T01:29:00.946 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:29:00 smithi117 ceph-mon[22016]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-08-19T01:29:00.946 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:29:00 smithi117 ceph-mon[22016]: from='osd.4 [v2:172.21.15.117:6808/3978863827,v1:172.21.15.117:6809/3978863827]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi117", "root=default"]}]: dispatch 2024-08-19T01:29:00.946 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:29:00 smithi117 ceph-mon[22016]: osdmap e31: 5 total, 4 up, 5 in 2024-08-19T01:29:00.946 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:29:00 smithi117 ceph-mon[22016]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-08-19T01:29:00.946 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:29:00 smithi117 ceph-mon[22016]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi117", "root=default"]}]: dispatch 2024-08-19T01:29:00.946 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:29:00 smithi117 ceph-mon[22016]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-08-19T01:29:01.581 INFO:journalctl@ceph.osd.4.smithi117.stdout:Aug 19 01:29:01 smithi117 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-4[32888]: 2024-08-19T01:29:01.364+0000 7fa6fb586640 -1 osd.4 0 waiting for initial osdmap 2024-08-19T01:29:01.581 INFO:journalctl@ceph.osd.4.smithi117.stdout:Aug 19 01:29:01 smithi117 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-4[32888]: 2024-08-19T01:29:01.372+0000 7fa6f3bb8640 -1 osd.4 32 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-08-19T01:29:01.598 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-08-19T01:29:01.624 DEBUG:teuthology.orchestra.run.smithi167:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid bad8780e-5dc9-11ef-bccf-c7b262605968 -- ceph orch daemon add osd smithi167:vg_nvme/lv_3 2024-08-19T01:29:01.886 INFO:teuthology.orchestra.run.smithi167.stderr:Inferring config /var/lib/ceph/bad8780e-5dc9-11ef-bccf-c7b262605968/mon.c/config 2024-08-19T01:29:01.904 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:29:01 smithi083 ceph-mon[17120]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi117", "root=default"]}]': finished 2024-08-19T01:29:01.905 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:29:01 smithi083 ceph-mon[17120]: osdmap e32: 5 total, 4 up, 5 in 2024-08-19T01:29:01.905 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:29:01 smithi083 ceph-mon[17120]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-08-19T01:29:01.905 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:29:01 smithi083 ceph-mon[17120]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-08-19T01:29:01.905 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:29:01 smithi083 ceph-mon[17120]: from='osd.4 ' entity='osd.4' 2024-08-19T01:29:01.905 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:29:01 smithi083 ceph-mon[17120]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' 2024-08-19T01:29:01.905 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:29:01 smithi083 ceph-mon[17120]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' 2024-08-19T01:29:01.905 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:29:01 smithi083 ceph-mon[17120]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-08-19T01:29:01.905 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:29:01 smithi083 ceph-mon[17120]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-08-19T01:29:01.905 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:29:01 smithi083 ceph-mon[17120]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' 2024-08-19T01:29:01.905 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:29:01 smithi083 ceph-mon[17120]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-19T01:29:01.905 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:29:01 smithi083 ceph-mon[17120]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-19T01:29:01.952 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:29:01 smithi117 ceph-mon[22016]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi117", "root=default"]}]': finished 2024-08-19T01:29:01.953 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:29:01 smithi117 ceph-mon[22016]: osdmap e32: 5 total, 4 up, 5 in 2024-08-19T01:29:01.953 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:29:01 smithi117 ceph-mon[22016]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-08-19T01:29:01.953 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:29:01 smithi117 ceph-mon[22016]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-08-19T01:29:01.953 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:29:01 smithi117 ceph-mon[22016]: from='osd.4 ' entity='osd.4' 2024-08-19T01:29:01.953 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:29:01 smithi117 ceph-mon[22016]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' 2024-08-19T01:29:01.953 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:29:01 smithi117 ceph-mon[22016]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' 2024-08-19T01:29:01.953 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:29:01 smithi117 ceph-mon[22016]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-08-19T01:29:01.953 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:29:01 smithi117 ceph-mon[22016]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-08-19T01:29:01.953 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:29:01 smithi117 ceph-mon[22016]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' 2024-08-19T01:29:01.953 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:29:01 smithi117 ceph-mon[22016]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-19T01:29:01.953 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:29:01 smithi117 ceph-mon[22016]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-19T01:29:01.966 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:29:01 smithi167 ceph-mon[21875]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi117", "root=default"]}]': finished 2024-08-19T01:29:01.966 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:29:01 smithi167 ceph-mon[21875]: osdmap e32: 5 total, 4 up, 5 in 2024-08-19T01:29:01.966 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:29:01 smithi167 ceph-mon[21875]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-08-19T01:29:01.966 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:29:01 smithi167 ceph-mon[21875]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-08-19T01:29:01.966 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:29:01 smithi167 ceph-mon[21875]: from='osd.4 ' entity='osd.4' 2024-08-19T01:29:01.966 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:29:01 smithi167 ceph-mon[21875]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' 2024-08-19T01:29:01.967 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:29:01 smithi167 ceph-mon[21875]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' 2024-08-19T01:29:01.967 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:29:01 smithi167 ceph-mon[21875]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-08-19T01:29:01.967 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:29:01 smithi167 ceph-mon[21875]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-08-19T01:29:01.967 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:29:01 smithi167 ceph-mon[21875]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' 2024-08-19T01:29:01.967 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:29:01 smithi167 ceph-mon[21875]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-19T01:29:01.967 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:29:01 smithi167 ceph-mon[21875]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-19T01:29:02.315 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-19T01:29:02.307+0000 7fa934848640 1 -- 172.21.15.167:0/1409892670 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa930101a00 msgr2=0x7fa930101e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:29:02.315 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-19T01:29:02.307+0000 7fa934848640 1 --2- 172.21.15.167:0/1409892670 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa930101a00 0x7fa930101e80 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fa924009900 tx=0x7fa92402a410 comp rx=0 tx=0).stop 2024-08-19T01:29:02.315 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-19T01:29:02.307+0000 7fa934848640 1 -- 172.21.15.167:0/1409892670 shutdown_connections 2024-08-19T01:29:02.315 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-19T01:29:02.307+0000 7fa934848640 1 --2- 172.21.15.167:0/1409892670 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa9301023c0 0x7fa930108c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:29:02.315 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-19T01:29:02.307+0000 7fa934848640 1 --2- 172.21.15.167:0/1409892670 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa930101a00 0x7fa930101e80 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:29:02.315 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-19T01:29:02.307+0000 7fa934848640 1 --2- 172.21.15.167:0/1409892670 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa930100800 0x7fa930100c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:29:02.315 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-19T01:29:02.307+0000 7fa934848640 1 -- 172.21.15.167:0/1409892670 >> 172.21.15.167:0/1409892670 conn(0x7fa9300fbfb0 msgr2=0x7fa9300fe3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:29:02.315 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-19T01:29:02.307+0000 7fa934848640 1 -- 172.21.15.167:0/1409892670 shutdown_connections 2024-08-19T01:29:02.316 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-19T01:29:02.307+0000 7fa934848640 1 -- 172.21.15.167:0/1409892670 wait complete. 2024-08-19T01:29:02.316 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-19T01:29:02.308+0000 7fa934848640 1 Processor -- start 2024-08-19T01:29:02.316 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-19T01:29:02.308+0000 7fa934848640 1 -- start start 2024-08-19T01:29:02.316 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-19T01:29:02.308+0000 7fa934848640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa930100800 0x7fa930197b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:29:02.316 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-19T01:29:02.308+0000 7fa934848640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa930101a00 0x7fa9301980a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:29:02.316 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-19T01:29:02.308+0000 7fa934848640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa9301023c0 0x7fa93019f120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:29:02.316 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-19T01:29:02.308+0000 7fa934848640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fa93010d5b0 con 0x7fa930101a00 2024-08-19T01:29:02.316 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-19T01:29:02.308+0000 7fa934848640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fa93010d430 con 0x7fa930100800 2024-08-19T01:29:02.316 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-19T01:29:02.308+0000 7fa934848640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fa93010d730 con 0x7fa9301023c0 2024-08-19T01:29:02.316 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-19T01:29:02.308+0000 7fa92ed9d640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa9301023c0 0x7fa93019f120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:29:02.317 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-19T01:29:02.308+0000 7fa92ed9d640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa9301023c0 0x7fa93019f120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:32884/0 (socket says 172.21.15.167:32884) 2024-08-19T01:29:02.317 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-19T01:29:02.308+0000 7fa92ed9d640 1 -- 172.21.15.167:0/1282672980 learned_addr learned my addr 172.21.15.167:0/1282672980 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-08-19T01:29:02.317 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-19T01:29:02.308+0000 7fa92e59c640 1 --2- 172.21.15.167:0/1282672980 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa930100800 0x7fa930197b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:29:02.317 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-19T01:29:02.308+0000 7fa92dd9b640 1 --2- 172.21.15.167:0/1282672980 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa930101a00 0x7fa9301980a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:29:02.317 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-19T01:29:02.308+0000 7fa92ed9d640 1 -- 172.21.15.167:0/1282672980 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa930100800 msgr2=0x7fa930197b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:29:02.317 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-19T01:29:02.308+0000 7fa92ed9d640 1 --2- 172.21.15.167:0/1282672980 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa930100800 0x7fa930197b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:29:02.317 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-19T01:29:02.308+0000 7fa92ed9d640 1 -- 172.21.15.167:0/1282672980 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa930101a00 msgr2=0x7fa9301980a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:29:02.317 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-19T01:29:02.308+0000 7fa92ed9d640 1 --2- 172.21.15.167:0/1282672980 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa930101a00 0x7fa9301980a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:29:02.317 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-19T01:29:02.308+0000 7fa92ed9d640 1 -- 172.21.15.167:0/1282672980 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa93019f790 con 0x7fa9301023c0 2024-08-19T01:29:02.317 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-19T01:29:02.309+0000 7fa92ed9d640 1 --2- 172.21.15.167:0/1282672980 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa9301023c0 0x7fa93019f120 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fa92800eaa0 tx=0x7fa92800ef60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:29:02.317 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-19T01:29:02.309+0000 7fa91f7fe640 1 -- 172.21.15.167:0/1282672980 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa92800cdb0 con 0x7fa9301023c0 2024-08-19T01:29:02.317 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-19T01:29:02.309+0000 7fa91f7fe640 1 -- 172.21.15.167:0/1282672980 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa928005020 con 0x7fa9301023c0 2024-08-19T01:29:02.317 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-19T01:29:02.309+0000 7fa934848640 1 -- 172.21.15.167:0/1282672980 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa93019fa80 con 0x7fa9301023c0 2024-08-19T01:29:02.318 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-19T01:29:02.309+0000 7fa91f7fe640 1 -- 172.21.15.167:0/1282672980 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9280116b0 con 0x7fa9301023c0 2024-08-19T01:29:02.318 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-19T01:29:02.309+0000 7fa934848640 1 -- 172.21.15.167:0/1282672980 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa93019ffe0 con 0x7fa9301023c0 2024-08-19T01:29:02.318 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-19T01:29:02.310+0000 7fa934848640 1 -- 172.21.15.167:0/1282672980 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa930107b80 con 0x7fa9301023c0 2024-08-19T01:29:02.318 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-19T01:29:02.311+0000 7fa91f7fe640 1 -- 172.21.15.167:0/1282672980 <== mon.1 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7fa928005560 con 0x7fa9301023c0 2024-08-19T01:29:02.318 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-19T01:29:02.311+0000 7fa91f7fe640 1 --2- 172.21.15.167:0/1282672980 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7fa9180a2c30 0x7fa9180a50f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:29:02.318 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-19T01:29:02.311+0000 7fa91f7fe640 1 -- 172.21.15.167:0/1282672980 <== mon.1 v2:172.21.15.167:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 3821+0+0 (secure 0 0 0) 0x7fa9280ce050 con 0x7fa9301023c0 2024-08-19T01:29:02.318 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-19T01:29:02.312+0000 7fa92e59c640 1 --2- 172.21.15.167:0/1282672980 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7fa9180a2c30 0x7fa9180a50f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:29:02.318 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-19T01:29:02.312+0000 7fa92e59c640 1 --2- 172.21.15.167:0/1282672980 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7fa9180a2c30 0x7fa9180a50f0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fa930101860 tx=0x7fa920005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:29:02.318 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-19T01:29:02.316+0000 7fa91f7fe640 1 -- 172.21.15.167:0/1282672980 <== mon.1 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7fa928016030 con 0x7fa9301023c0 2024-08-19T01:29:02.495 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-19T01:29:02.493+0000 7fa934848640 1 -- 172.21.15.167:0/1282672980 --> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi167:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7fa930062360 con 0x7fa9180a2c30 2024-08-19T01:29:02.890 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:29:02 smithi167 ceph-mon[21875]: purged_snaps scrub starts 2024-08-19T01:29:02.891 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:29:02 smithi167 ceph-mon[21875]: purged_snaps scrub ok 2024-08-19T01:29:02.891 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:29:02 smithi167 ceph-mon[21875]: Detected new or changed devices on smithi117 2024-08-19T01:29:02.891 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:29:02 smithi167 ceph-mon[21875]: Adjusting osd_memory_target on smithi117 to 8544M 2024-08-19T01:29:02.891 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:29:02 smithi167 ceph-mon[21875]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-08-19T01:29:02.891 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:29:02 smithi167 ceph-mon[21875]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' 2024-08-19T01:29:02.891 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:29:02 smithi167 ceph-mon[21875]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-08-19T01:29:02.891 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:29:02 smithi167 ceph-mon[21875]: osd.4 [v2:172.21.15.117:6808/3978863827,v1:172.21.15.117:6809/3978863827] boot 2024-08-19T01:29:02.891 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:29:02 smithi167 ceph-mon[21875]: osdmap e33: 5 total, 5 up, 5 in 2024-08-19T01:29:02.891 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:29:02 smithi167 ceph-mon[21875]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-08-19T01:29:02.891 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:29:02 smithi167 ceph-mon[21875]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-08-19T01:29:02.891 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:29:02 smithi167 ceph-mon[21875]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-08-19T01:29:02.891 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:29:02 smithi167 ceph-mon[21875]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-19T01:29:02.905 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:29:02 smithi083 ceph-mon[17120]: purged_snaps scrub starts 2024-08-19T01:29:02.905 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:29:02 smithi083 ceph-mon[17120]: purged_snaps scrub ok 2024-08-19T01:29:02.905 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:29:02 smithi083 ceph-mon[17120]: Detected new or changed devices on smithi117 2024-08-19T01:29:02.905 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:29:02 smithi083 ceph-mon[17120]: Adjusting osd_memory_target on smithi117 to 8544M 2024-08-19T01:29:02.905 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:29:02 smithi083 ceph-mon[17120]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-08-19T01:29:02.905 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:29:02 smithi083 ceph-mon[17120]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' 2024-08-19T01:29:02.905 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:29:02 smithi083 ceph-mon[17120]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-08-19T01:29:02.905 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:29:02 smithi083 ceph-mon[17120]: osd.4 [v2:172.21.15.117:6808/3978863827,v1:172.21.15.117:6809/3978863827] boot 2024-08-19T01:29:02.906 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:29:02 smithi083 ceph-mon[17120]: osdmap e33: 5 total, 5 up, 5 in 2024-08-19T01:29:02.906 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:29:02 smithi083 ceph-mon[17120]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-08-19T01:29:02.906 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:29:02 smithi083 ceph-mon[17120]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-08-19T01:29:02.906 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:29:02 smithi083 ceph-mon[17120]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-08-19T01:29:02.906 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:29:02 smithi083 ceph-mon[17120]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-19T01:29:02.953 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:29:02 smithi117 ceph-mon[22016]: purged_snaps scrub starts 2024-08-19T01:29:02.953 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:29:02 smithi117 ceph-mon[22016]: purged_snaps scrub ok 2024-08-19T01:29:02.953 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:29:02 smithi117 ceph-mon[22016]: Detected new or changed devices on smithi117 2024-08-19T01:29:02.953 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:29:02 smithi117 ceph-mon[22016]: Adjusting osd_memory_target on smithi117 to 8544M 2024-08-19T01:29:02.953 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:29:02 smithi117 ceph-mon[22016]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-08-19T01:29:02.953 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:29:02 smithi117 ceph-mon[22016]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' 2024-08-19T01:29:02.953 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:29:02 smithi117 ceph-mon[22016]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-08-19T01:29:02.953 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:29:02 smithi117 ceph-mon[22016]: osd.4 [v2:172.21.15.117:6808/3978863827,v1:172.21.15.117:6809/3978863827] boot 2024-08-19T01:29:02.953 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:29:02 smithi117 ceph-mon[22016]: osdmap e33: 5 total, 5 up, 5 in 2024-08-19T01:29:02.953 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:29:02 smithi117 ceph-mon[22016]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-08-19T01:29:02.953 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:29:02 smithi117 ceph-mon[22016]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-08-19T01:29:02.953 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:29:02 smithi117 ceph-mon[22016]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-08-19T01:29:02.954 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:29:02 smithi117 ceph-mon[22016]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-19T01:29:03.905 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:29:03 smithi083 ceph-mon[17120]: from='client.24241 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi167:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-08-19T01:29:03.930 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:29:03 smithi167 ceph-mon[21875]: from='client.24241 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi167:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-08-19T01:29:03.953 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:29:03 smithi117 ceph-mon[22016]: from='client.24241 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi167:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-08-19T01:29:04.905 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:29:04 smithi083 ceph-mon[17120]: osdmap e34: 5 total, 5 up, 5 in 2024-08-19T01:29:04.905 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:29:04 smithi083 ceph-mon[17120]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-08-19T01:29:04.952 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:29:04 smithi117 ceph-mon[22016]: osdmap e34: 5 total, 5 up, 5 in 2024-08-19T01:29:04.953 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:29:04 smithi117 ceph-mon[22016]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-08-19T01:29:04.967 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:29:04 smithi167 ceph-mon[21875]: osdmap e34: 5 total, 5 up, 5 in 2024-08-19T01:29:04.967 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:29:04 smithi167 ceph-mon[21875]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-08-19T01:29:05.905 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:29:05 smithi083 ceph-mon[17120]: from='client.? 172.21.15.167:0/1700563622' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e01e3f15-25c8-4d2c-be6f-a170f036064c"}]: dispatch 2024-08-19T01:29:05.905 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:29:05 smithi083 ceph-mon[17120]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e01e3f15-25c8-4d2c-be6f-a170f036064c"}]: dispatch 2024-08-19T01:29:05.905 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:29:05 smithi083 ceph-mon[17120]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "e01e3f15-25c8-4d2c-be6f-a170f036064c"}]': finished 2024-08-19T01:29:05.905 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:29:05 smithi083 ceph-mon[17120]: osdmap e35: 6 total, 5 up, 6 in 2024-08-19T01:29:05.905 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:29:05 smithi083 ceph-mon[17120]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-08-19T01:29:05.905 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:29:05 smithi083 ceph-mon[17120]: from='client.? 172.21.15.167:0/3281800503' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-08-19T01:29:05.953 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:29:05 smithi117 ceph-mon[22016]: from='client.? 172.21.15.167:0/1700563622' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e01e3f15-25c8-4d2c-be6f-a170f036064c"}]: dispatch 2024-08-19T01:29:05.953 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:29:05 smithi117 ceph-mon[22016]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e01e3f15-25c8-4d2c-be6f-a170f036064c"}]: dispatch 2024-08-19T01:29:05.953 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:29:05 smithi117 ceph-mon[22016]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "e01e3f15-25c8-4d2c-be6f-a170f036064c"}]': finished 2024-08-19T01:29:05.953 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:29:05 smithi117 ceph-mon[22016]: osdmap e35: 6 total, 5 up, 6 in 2024-08-19T01:29:05.953 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:29:05 smithi117 ceph-mon[22016]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-08-19T01:29:05.953 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:29:05 smithi117 ceph-mon[22016]: from='client.? 172.21.15.167:0/3281800503' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-08-19T01:29:05.966 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:29:05 smithi167 ceph-mon[21875]: from='client.? 172.21.15.167:0/1700563622' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e01e3f15-25c8-4d2c-be6f-a170f036064c"}]: dispatch 2024-08-19T01:29:05.966 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:29:05 smithi167 ceph-mon[21875]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e01e3f15-25c8-4d2c-be6f-a170f036064c"}]: dispatch 2024-08-19T01:29:05.966 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:29:05 smithi167 ceph-mon[21875]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "e01e3f15-25c8-4d2c-be6f-a170f036064c"}]': finished 2024-08-19T01:29:05.966 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:29:05 smithi167 ceph-mon[21875]: osdmap e35: 6 total, 5 up, 6 in 2024-08-19T01:29:05.966 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:29:05 smithi167 ceph-mon[21875]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-08-19T01:29:05.966 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:29:05 smithi167 ceph-mon[21875]: from='client.? 172.21.15.167:0/3281800503' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-08-19T01:29:06.905 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:29:06 smithi083 ceph-mon[17120]: pgmap v116: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-08-19T01:29:06.953 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:29:06 smithi117 ceph-mon[22016]: pgmap v116: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-08-19T01:29:06.966 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:29:06 smithi167 ceph-mon[21875]: pgmap v116: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-08-19T01:29:09.466 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:29:09 smithi167 ceph-mon[21875]: pgmap v117: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-08-19T01:29:09.655 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:29:09 smithi083 ceph-mon[17120]: pgmap v117: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-08-19T01:29:09.702 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:29:09 smithi117 ceph-mon[22016]: pgmap v117: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-08-19T01:29:11.655 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:29:11 smithi083 ceph-mon[17120]: pgmap v118: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-08-19T01:29:11.661 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:29:11 smithi167 ceph-mon[21875]: pgmap v118: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-08-19T01:29:11.702 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:29:11 smithi117 ceph-mon[22016]: pgmap v118: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-08-19T01:29:12.654 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:29:12 smithi083 ceph-mon[17120]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-08-19T01:29:12.655 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:29:12 smithi083 ceph-mon[17120]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-19T01:29:12.702 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:29:12 smithi117 ceph-mon[22016]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-08-19T01:29:12.703 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:29:12 smithi117 ceph-mon[22016]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-19T01:29:12.716 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:29:12 smithi167 ceph-mon[21875]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-08-19T01:29:12.716 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:29:12 smithi167 ceph-mon[21875]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-19T01:29:13.560 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:29:13 smithi167 ceph-mon[21875]: Deploying daemon osd.5 on smithi167 2024-08-19T01:29:13.560 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:29:13 smithi167 ceph-mon[21875]: pgmap v119: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-08-19T01:29:13.655 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:29:13 smithi083 ceph-mon[17120]: Deploying daemon osd.5 on smithi167 2024-08-19T01:29:13.655 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:29:13 smithi083 ceph-mon[17120]: pgmap v119: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-08-19T01:29:13.702 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:29:13 smithi117 ceph-mon[22016]: Deploying daemon osd.5 on smithi167 2024-08-19T01:29:13.703 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:29:13 smithi117 ceph-mon[22016]: pgmap v119: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-08-19T01:29:15.617 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:29:15 smithi167 ceph-mon[21875]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-08-19T01:29:15.654 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:29:15 smithi083 ceph-mon[17120]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-08-19T01:29:15.702 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:29:15 smithi117 ceph-mon[22016]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-08-19T01:29:17.466 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:29:17 smithi167 ceph-mon[21875]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-08-19T01:29:17.467 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:29:17 smithi167 ceph-mon[21875]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-19T01:29:17.467 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:29:17 smithi167 ceph-mon[21875]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' 2024-08-19T01:29:17.467 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:29:17 smithi167 ceph-mon[21875]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' 2024-08-19T01:29:17.655 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:29:17 smithi083 ceph-mon[17120]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-08-19T01:29:17.655 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:29:17 smithi083 ceph-mon[17120]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-19T01:29:17.655 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:29:17 smithi083 ceph-mon[17120]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' 2024-08-19T01:29:17.655 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:29:17 smithi083 ceph-mon[17120]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' 2024-08-19T01:29:17.703 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:29:17 smithi117 ceph-mon[22016]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-08-19T01:29:17.703 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:29:17 smithi117 ceph-mon[22016]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-19T01:29:17.703 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:29:17 smithi117 ceph-mon[22016]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' 2024-08-19T01:29:17.703 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:29:17 smithi117 ceph-mon[22016]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' 2024-08-19T01:29:19.655 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:29:19 smithi083 ceph-mon[17120]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-08-19T01:29:19.655 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:29:19 smithi083 ceph-mon[17120]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' 2024-08-19T01:29:19.655 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:29:19 smithi083 ceph-mon[17120]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' 2024-08-19T01:29:19.655 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:29:19 smithi083 ceph-mon[17120]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-19T01:29:19.655 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:29:19 smithi083 ceph-mon[17120]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-19T01:29:19.655 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:29:19 smithi083 ceph-mon[17120]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' 2024-08-19T01:29:19.691 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:29:19 smithi167 ceph-mon[21875]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-08-19T01:29:19.691 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:29:19 smithi167 ceph-mon[21875]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' 2024-08-19T01:29:19.692 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:29:19 smithi167 ceph-mon[21875]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' 2024-08-19T01:29:19.692 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:29:19 smithi167 ceph-mon[21875]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-19T01:29:19.692 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:29:19 smithi167 ceph-mon[21875]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-19T01:29:19.692 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:29:19 smithi167 ceph-mon[21875]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' 2024-08-19T01:29:19.702 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:29:19 smithi117 ceph-mon[22016]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-08-19T01:29:19.703 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:29:19 smithi117 ceph-mon[22016]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' 2024-08-19T01:29:19.703 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:29:19 smithi117 ceph-mon[22016]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' 2024-08-19T01:29:19.703 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:29:19 smithi117 ceph-mon[22016]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-19T01:29:19.703 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:29:19 smithi117 ceph-mon[22016]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-19T01:29:19.703 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:29:19 smithi117 ceph-mon[22016]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' 2024-08-19T01:29:19.735 INFO:teuthology.orchestra.run.smithi167.stdout:Created osd(s) 5 on host 'smithi167' 2024-08-19T01:29:19.735 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-19T01:29:19.733+0000 7fa91f7fe640 1 -- 172.21.15.167:0/1282672980 <== mgr.14150 v2:172.21.15.83:6800/683740916 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fa930062360 con 0x7fa9180a2c30 2024-08-19T01:29:19.737 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-19T01:29:19.735+0000 7fa934848640 1 -- 172.21.15.167:0/1282672980 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7fa9180a2c30 msgr2=0x7fa9180a50f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:29:19.737 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-19T01:29:19.735+0000 7fa934848640 1 --2- 172.21.15.167:0/1282672980 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7fa9180a2c30 0x7fa9180a50f0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fa930101860 tx=0x7fa920005f90 comp rx=0 tx=0).stop 2024-08-19T01:29:19.737 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-19T01:29:19.735+0000 7fa934848640 1 -- 172.21.15.167:0/1282672980 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa9301023c0 msgr2=0x7fa93019f120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:29:19.737 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-19T01:29:19.735+0000 7fa934848640 1 --2- 172.21.15.167:0/1282672980 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa9301023c0 0x7fa93019f120 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fa92800eaa0 tx=0x7fa92800ef60 comp rx=0 tx=0).stop 2024-08-19T01:29:19.737 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-19T01:29:19.735+0000 7fa934848640 1 -- 172.21.15.167:0/1282672980 shutdown_connections 2024-08-19T01:29:19.737 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-19T01:29:19.735+0000 7fa934848640 1 --2- 172.21.15.167:0/1282672980 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7fa9180a2c30 0x7fa9180a50f0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:29:19.737 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-19T01:29:19.735+0000 7fa934848640 1 --2- 172.21.15.167:0/1282672980 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa9301023c0 0x7fa93019f120 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:29:19.737 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-19T01:29:19.735+0000 7fa934848640 1 --2- 172.21.15.167:0/1282672980 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa930101a00 0x7fa9301980a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:29:19.737 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-19T01:29:19.735+0000 7fa934848640 1 --2- 172.21.15.167:0/1282672980 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa930100800 0x7fa930197b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:29:19.737 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-19T01:29:19.735+0000 7fa934848640 1 -- 172.21.15.167:0/1282672980 >> 172.21.15.167:0/1282672980 conn(0x7fa9300fbfb0 msgr2=0x7fa930104d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:29:19.737 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-19T01:29:19.736+0000 7fa934848640 1 -- 172.21.15.167:0/1282672980 shutdown_connections 2024-08-19T01:29:19.738 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-19T01:29:19.736+0000 7fa934848640 1 -- 172.21.15.167:0/1282672980 wait complete. 2024-08-19T01:29:20.397 DEBUG:teuthology.orchestra.run.smithi167:osd.5> sudo journalctl -f -n 0 -u ceph-bad8780e-5dc9-11ef-bccf-c7b262605968@osd.5.service 2024-08-19T01:29:20.399 INFO:tasks.cephadm:Deploying osd.6 on smithi083 with /dev/vg_nvme/lv_2... 2024-08-19T01:29:20.399 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833 ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid bad8780e-5dc9-11ef-bccf-c7b262605968 -- lvm zap /dev/vg_nvme/lv_2 2024-08-19T01:29:20.655 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:29:20 smithi083 ceph-mon[17120]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-19T01:29:20.655 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:29:20 smithi083 ceph-mon[17120]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' 2024-08-19T01:29:20.655 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:29:20 smithi083 ceph-mon[17120]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' 2024-08-19T01:29:20.655 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:29:20 smithi083 ceph-mon[17120]: from='osd.5 [v2:172.21.15.167:6808/2761858035,v1:172.21.15.167:6809/2761858035]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-08-19T01:29:20.655 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:29:20 smithi083 ceph-mon[17120]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-08-19T01:29:20.698 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:29:20 smithi167 ceph-mon[21875]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-19T01:29:20.698 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:29:20 smithi167 ceph-mon[21875]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' 2024-08-19T01:29:20.699 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:29:20 smithi167 ceph-mon[21875]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' 2024-08-19T01:29:20.699 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:29:20 smithi167 ceph-mon[21875]: from='osd.5 [v2:172.21.15.167:6808/2761858035,v1:172.21.15.167:6809/2761858035]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-08-19T01:29:20.699 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:29:20 smithi167 ceph-mon[21875]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-08-19T01:29:20.702 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:29:20 smithi117 ceph-mon[22016]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-19T01:29:20.702 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:29:20 smithi117 ceph-mon[22016]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' 2024-08-19T01:29:20.703 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:29:20 smithi117 ceph-mon[22016]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' 2024-08-19T01:29:20.703 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:29:20 smithi117 ceph-mon[22016]: from='osd.5 [v2:172.21.15.167:6808/2761858035,v1:172.21.15.167:6809/2761858035]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-08-19T01:29:20.703 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:29:20 smithi117 ceph-mon[22016]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-08-19T01:29:20.780 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/bad8780e-5dc9-11ef-bccf-c7b262605968/mon.a/config 2024-08-19T01:29:21.527 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:29:21 smithi167 ceph-mon[21875]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-08-19T01:29:21.527 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:29:21 smithi167 ceph-mon[21875]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-08-19T01:29:21.527 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:29:21 smithi167 ceph-mon[21875]: from='osd.5 [v2:172.21.15.167:6808/2761858035,v1:172.21.15.167:6809/2761858035]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi167", "root=default"]}]: dispatch 2024-08-19T01:29:21.527 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:29:21 smithi167 ceph-mon[21875]: osdmap e36: 6 total, 5 up, 6 in 2024-08-19T01:29:21.527 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:29:21 smithi167 ceph-mon[21875]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-08-19T01:29:21.527 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:29:21 smithi167 ceph-mon[21875]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi167", "root=default"]}]: dispatch 2024-08-19T01:29:21.655 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:29:21 smithi083 ceph-mon[17120]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-08-19T01:29:21.655 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:29:21 smithi083 ceph-mon[17120]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-08-19T01:29:21.655 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:29:21 smithi083 ceph-mon[17120]: from='osd.5 [v2:172.21.15.167:6808/2761858035,v1:172.21.15.167:6809/2761858035]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi167", "root=default"]}]: dispatch 2024-08-19T01:29:21.655 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:29:21 smithi083 ceph-mon[17120]: osdmap e36: 6 total, 5 up, 6 in 2024-08-19T01:29:21.655 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:29:21 smithi083 ceph-mon[17120]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-08-19T01:29:21.655 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:29:21 smithi083 ceph-mon[17120]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi167", "root=default"]}]: dispatch 2024-08-19T01:29:21.702 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:29:21 smithi117 ceph-mon[22016]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-08-19T01:29:21.703 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:29:21 smithi117 ceph-mon[22016]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-08-19T01:29:21.703 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:29:21 smithi117 ceph-mon[22016]: from='osd.5 [v2:172.21.15.167:6808/2761858035,v1:172.21.15.167:6809/2761858035]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi167", "root=default"]}]: dispatch 2024-08-19T01:29:21.703 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:29:21 smithi117 ceph-mon[22016]: osdmap e36: 6 total, 5 up, 6 in 2024-08-19T01:29:21.703 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:29:21 smithi117 ceph-mon[22016]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-08-19T01:29:21.703 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:29:21 smithi117 ceph-mon[22016]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi167", "root=default"]}]: dispatch 2024-08-19T01:29:21.794 INFO:journalctl@ceph.osd.5.smithi167.stdout:Aug 19 01:29:21 smithi167 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-5[33276]: 2024-08-19T01:29:21.748+0000 7f8193ef4640 -1 osd.5 0 waiting for initial osdmap 2024-08-19T01:29:21.794 INFO:journalctl@ceph.osd.5.smithi167.stdout:Aug 19 01:29:21 smithi167 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-5[33276]: 2024-08-19T01:29:21.755+0000 7f819052e640 -1 osd.5 37 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-08-19T01:29:22.835 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:29:22 smithi167 ceph-mon[21875]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi167", "root=default"]}]': finished 2024-08-19T01:29:22.835 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:29:22 smithi167 ceph-mon[21875]: osdmap e37: 6 total, 5 up, 6 in 2024-08-19T01:29:22.835 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:29:22 smithi167 ceph-mon[21875]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-08-19T01:29:22.835 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:29:22 smithi167 ceph-mon[21875]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-08-19T01:29:22.835 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:29:22 smithi167 ceph-mon[21875]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-08-19T01:29:22.905 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:29:22 smithi083 ceph-mon[17120]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi167", "root=default"]}]': finished 2024-08-19T01:29:22.905 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:29:22 smithi083 ceph-mon[17120]: osdmap e37: 6 total, 5 up, 6 in 2024-08-19T01:29:22.905 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:29:22 smithi083 ceph-mon[17120]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-08-19T01:29:22.905 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:29:22 smithi083 ceph-mon[17120]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-08-19T01:29:22.905 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:29:22 smithi083 ceph-mon[17120]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-08-19T01:29:23.203 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:29:22 smithi117 ceph-mon[22016]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi167", "root=default"]}]': finished 2024-08-19T01:29:23.203 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:29:22 smithi117 ceph-mon[22016]: osdmap e37: 6 total, 5 up, 6 in 2024-08-19T01:29:23.203 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:29:22 smithi117 ceph-mon[22016]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-08-19T01:29:23.203 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:29:22 smithi117 ceph-mon[22016]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-08-19T01:29:23.203 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:29:22 smithi117 ceph-mon[22016]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-08-19T01:29:23.869 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-08-19T01:29:23.902 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid bad8780e-5dc9-11ef-bccf-c7b262605968 -- ceph orch daemon add osd smithi083:vg_nvme/lv_2 2024-08-19T01:29:24.155 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:29:23 smithi083 ceph-mon[17120]: purged_snaps scrub starts 2024-08-19T01:29:24.155 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:29:23 smithi083 ceph-mon[17120]: purged_snaps scrub ok 2024-08-19T01:29:24.155 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:29:23 smithi083 ceph-mon[17120]: osd.5 [v2:172.21.15.167:6808/2761858035,v1:172.21.15.167:6809/2761858035] boot 2024-08-19T01:29:24.155 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:29:23 smithi083 ceph-mon[17120]: osdmap e38: 6 total, 6 up, 6 in 2024-08-19T01:29:24.155 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:29:23 smithi083 ceph-mon[17120]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-08-19T01:29:24.155 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:29:23 smithi083 ceph-mon[17120]: Detected new or changed devices on smithi167 2024-08-19T01:29:24.155 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:29:23 smithi083 ceph-mon[17120]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' 2024-08-19T01:29:24.155 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:29:23 smithi083 ceph-mon[17120]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' 2024-08-19T01:29:24.155 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:29:23 smithi083 ceph-mon[17120]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-08-19T01:29:24.155 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:29:23 smithi083 ceph-mon[17120]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-08-19T01:29:24.156 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:29:23 smithi083 ceph-mon[17120]: Adjusting osd_memory_target on smithi167 to 8544M 2024-08-19T01:29:24.156 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:29:23 smithi083 ceph-mon[17120]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' 2024-08-19T01:29:24.156 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:29:23 smithi083 ceph-mon[17120]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-19T01:29:24.156 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:29:23 smithi083 ceph-mon[17120]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-19T01:29:24.156 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:29:23 smithi083 ceph-mon[17120]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' 2024-08-19T01:29:24.156 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:29:23 smithi083 ceph-mon[17120]: osdmap e39: 6 total, 6 up, 6 in 2024-08-19T01:29:24.203 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:29:23 smithi117 ceph-mon[22016]: purged_snaps scrub starts 2024-08-19T01:29:24.203 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:29:23 smithi117 ceph-mon[22016]: purged_snaps scrub ok 2024-08-19T01:29:24.203 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:29:23 smithi117 ceph-mon[22016]: osd.5 [v2:172.21.15.167:6808/2761858035,v1:172.21.15.167:6809/2761858035] boot 2024-08-19T01:29:24.203 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:29:23 smithi117 ceph-mon[22016]: osdmap e38: 6 total, 6 up, 6 in 2024-08-19T01:29:24.203 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:29:23 smithi117 ceph-mon[22016]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-08-19T01:29:24.203 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:29:23 smithi117 ceph-mon[22016]: Detected new or changed devices on smithi167 2024-08-19T01:29:24.203 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:29:23 smithi117 ceph-mon[22016]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' 2024-08-19T01:29:24.203 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:29:23 smithi117 ceph-mon[22016]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' 2024-08-19T01:29:24.204 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:29:23 smithi117 ceph-mon[22016]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-08-19T01:29:24.204 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:29:23 smithi117 ceph-mon[22016]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-08-19T01:29:24.204 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:29:23 smithi117 ceph-mon[22016]: Adjusting osd_memory_target on smithi167 to 8544M 2024-08-19T01:29:24.204 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:29:23 smithi117 ceph-mon[22016]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' 2024-08-19T01:29:24.204 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:29:23 smithi117 ceph-mon[22016]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-19T01:29:24.204 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:29:23 smithi117 ceph-mon[22016]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-19T01:29:24.204 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:29:23 smithi117 ceph-mon[22016]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' 2024-08-19T01:29:24.204 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:29:23 smithi117 ceph-mon[22016]: osdmap e39: 6 total, 6 up, 6 in 2024-08-19T01:29:24.216 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:29:23 smithi167 ceph-mon[21875]: purged_snaps scrub starts 2024-08-19T01:29:24.216 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:29:23 smithi167 ceph-mon[21875]: purged_snaps scrub ok 2024-08-19T01:29:24.217 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:29:23 smithi167 ceph-mon[21875]: osd.5 [v2:172.21.15.167:6808/2761858035,v1:172.21.15.167:6809/2761858035] boot 2024-08-19T01:29:24.217 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:29:23 smithi167 ceph-mon[21875]: osdmap e38: 6 total, 6 up, 6 in 2024-08-19T01:29:24.217 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:29:23 smithi167 ceph-mon[21875]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-08-19T01:29:24.217 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:29:23 smithi167 ceph-mon[21875]: Detected new or changed devices on smithi167 2024-08-19T01:29:24.217 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:29:23 smithi167 ceph-mon[21875]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' 2024-08-19T01:29:24.217 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:29:23 smithi167 ceph-mon[21875]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' 2024-08-19T01:29:24.217 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:29:23 smithi167 ceph-mon[21875]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-08-19T01:29:24.217 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:29:23 smithi167 ceph-mon[21875]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-08-19T01:29:24.217 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:29:23 smithi167 ceph-mon[21875]: Adjusting osd_memory_target on smithi167 to 8544M 2024-08-19T01:29:24.217 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:29:23 smithi167 ceph-mon[21875]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' 2024-08-19T01:29:24.217 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:29:23 smithi167 ceph-mon[21875]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-19T01:29:24.217 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:29:23 smithi167 ceph-mon[21875]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-19T01:29:24.217 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:29:23 smithi167 ceph-mon[21875]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' 2024-08-19T01:29:24.217 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:29:23 smithi167 ceph-mon[21875]: osdmap e39: 6 total, 6 up, 6 in 2024-08-19T01:29:24.283 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/bad8780e-5dc9-11ef-bccf-c7b262605968/mon.a/config 2024-08-19T01:29:24.865 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:29:24.862+0000 7f535b600640 1 -- 172.21.15.83:0/1512399715 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f535410df10 msgr2=0x7f5354110300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:29:24.865 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:29:24.862+0000 7f535b600640 1 --2- 172.21.15.83:0/1512399715 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f535410df10 0x7f5354110300 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f534c009900 tx=0x7f534c02a410 comp rx=0 tx=0).stop 2024-08-19T01:29:24.865 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:29:24.862+0000 7f535b600640 1 -- 172.21.15.83:0/1512399715 shutdown_connections 2024-08-19T01:29:24.865 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:29:24.862+0000 7f535b600640 1 --2- 172.21.15.83:0/1512399715 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f535410df10 0x7f5354110300 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:29:24.865 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:29:24.862+0000 7f535b600640 1 --2- 172.21.15.83:0/1512399715 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5354074980 0x7f5354074de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:29:24.865 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:29:24.862+0000 7f535b600640 1 --2- 172.21.15.83:0/1512399715 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5354075f50 0x7f53540743b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:29:24.865 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:29:24.862+0000 7f535b600640 1 -- 172.21.15.83:0/1512399715 >> 172.21.15.83:0/1512399715 conn(0x7f53540fd0c0 msgr2=0x7f53540ff500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:29:24.865 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:29:24.862+0000 7f535b600640 1 -- 172.21.15.83:0/1512399715 shutdown_connections 2024-08-19T01:29:24.865 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:29:24.862+0000 7f535b600640 1 -- 172.21.15.83:0/1512399715 wait complete. 2024-08-19T01:29:24.865 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:29:24.863+0000 7f535b600640 1 Processor -- start 2024-08-19T01:29:24.871 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:29:24.863+0000 7f535b600640 1 -- start start 2024-08-19T01:29:24.871 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:29:24.864+0000 7f535b600640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5354074980 0x7f535419e1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:29:24.871 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:29:24.864+0000 7f535b600640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5354075f50 0x7f535419e710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:29:24.871 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:29:24.864+0000 7f535b600640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f535410df10 0x7f53541a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:29:24.871 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:29:24.864+0000 7f535b600640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f5354113bc0 con 0x7f535410df10 2024-08-19T01:29:24.871 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:29:24.864+0000 7f535b600640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f5354113a40 con 0x7f5354074980 2024-08-19T01:29:24.871 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:29:24.864+0000 7f535b600640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f5354113d40 con 0x7f5354075f50 2024-08-19T01:29:24.871 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:29:24.864+0000 7f5353e00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5354075f50 0x7f535419e710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:29:24.871 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:29:24.864+0000 7f5358c00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5354074980 0x7f535419e1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:29:24.871 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:29:24.864+0000 7f5359600640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f535410df10 0x7f53541a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:29:24.871 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:29:24.864+0000 7f5353e00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5354075f50 0x7f535419e710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.83:44292/0 (socket says 172.21.15.83:44292) 2024-08-19T01:29:24.872 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:29:24.864+0000 7f5353e00640 1 -- 172.21.15.83:0/2972543017 learned_addr learned my addr 172.21.15.83:0/2972543017 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-08-19T01:29:24.872 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:29:24.864+0000 7f5358c00640 1 -- 172.21.15.83:0/2972543017 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5354075f50 msgr2=0x7f535419e710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:29:24.872 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:29:24.864+0000 7f5358c00640 1 --2- 172.21.15.83:0/2972543017 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5354075f50 0x7f535419e710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:29:24.872 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:29:24.864+0000 7f5358c00640 1 -- 172.21.15.83:0/2972543017 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f535410df10 msgr2=0x7f53541a5790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:29:24.872 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:29:24.864+0000 7f5358c00640 1 --2- 172.21.15.83:0/2972543017 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f535410df10 0x7f53541a5790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:29:24.872 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:29:24.864+0000 7f5358c00640 1 -- 172.21.15.83:0/2972543017 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f53541a5e00 con 0x7f5354074980 2024-08-19T01:29:24.872 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:29:24.864+0000 7f5359600640 1 --2- 172.21.15.83:0/2972543017 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f535410df10 0x7f53541a5790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-19T01:29:24.872 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:29:24.865+0000 7f5358c00640 1 --2- 172.21.15.83:0/2972543017 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5354074980 0x7f535419e1d0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f5344005fd0 tx=0x7f534400df40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:29:24.872 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:29:24.865+0000 7f5351600640 1 -- 172.21.15.83:0/2972543017 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5344017030 con 0x7f5354074980 2024-08-19T01:29:24.872 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:29:24.865+0000 7f535b600640 1 -- 172.21.15.83:0/2972543017 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f53541a60f0 con 0x7f5354074980 2024-08-19T01:29:24.872 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:29:24.865+0000 7f5351600640 1 -- 172.21.15.83:0/2972543017 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5344003e50 con 0x7f5354074980 2024-08-19T01:29:24.872 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:29:24.865+0000 7f535b600640 1 -- 172.21.15.83:0/2972543017 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f53541a6680 con 0x7f5354074980 2024-08-19T01:29:24.872 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:29:24.865+0000 7f5351600640 1 -- 172.21.15.83:0/2972543017 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5344010460 con 0x7f5354074980 2024-08-19T01:29:24.873 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:29:24.866+0000 7f535b600640 1 -- 172.21.15.83:0/2972543017 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f53541040e0 con 0x7f5354074980 2024-08-19T01:29:24.873 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:29:24.868+0000 7f5351600640 1 -- 172.21.15.83:0/2972543017 <== mon.2 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f534401a070 con 0x7f5354074980 2024-08-19T01:29:24.873 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:29:24.869+0000 7f5351600640 1 --2- 172.21.15.83:0/2972543017 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f53380a2bb0 0x7f53380a5070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:29:24.873 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:29:24.870+0000 7f5351600640 1 -- 172.21.15.83:0/2972543017 <== mon.2 v2:172.21.15.117:3300/0 5 ==== osd_map(40..40 src has 1..40) ==== 4562+0+0 (secure 0 0 0) 0x7f53440cdc40 con 0x7f5354074980 2024-08-19T01:29:24.873 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:29:24.870+0000 7f5353e00640 1 --2- 172.21.15.83:0/2972543017 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f53380a2bb0 0x7f53380a5070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:29:24.874 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:29:24.870+0000 7f5353e00640 1 --2- 172.21.15.83:0/2972543017 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f53380a2bb0 0x7f53380a5070 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f535419f6f0 tx=0x7f534000b480 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:29:24.875 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:29:24.874+0000 7f5351600640 1 -- 172.21.15.83:0/2972543017 <== mon.2 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7f5344021070 con 0x7f5354074980 2024-08-19T01:29:25.060 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:29:25.058+0000 7f535b600640 1 -- 172.21.15.83:0/2972543017 --> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi083:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7f5354062360 con 0x7f53380a2bb0 2024-08-19T01:29:25.061 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:29:24 smithi083 ceph-mon[17120]: pgmap v129: 1 pgs: 1 remapped+peering; 577 KiB data, 178 MiB used, 536 GiB / 536 GiB avail 2024-08-19T01:29:25.061 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:29:24 smithi083 ceph-mon[17120]: osdmap e40: 6 total, 6 up, 6 in 2024-08-19T01:29:25.203 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:29:24 smithi117 ceph-mon[22016]: pgmap v129: 1 pgs: 1 remapped+peering; 577 KiB data, 178 MiB used, 536 GiB / 536 GiB avail 2024-08-19T01:29:25.203 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:29:24 smithi117 ceph-mon[22016]: osdmap e40: 6 total, 6 up, 6 in 2024-08-19T01:29:25.216 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:29:24 smithi167 ceph-mon[21875]: pgmap v129: 1 pgs: 1 remapped+peering; 577 KiB data, 178 MiB used, 536 GiB / 536 GiB avail 2024-08-19T01:29:25.216 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:29:24 smithi167 ceph-mon[21875]: osdmap e40: 6 total, 6 up, 6 in 2024-08-19T01:29:25.896 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:29:25 smithi083 ceph-mon[17120]: from='client.24272 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi083:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-08-19T01:29:25.896 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:29:25 smithi083 ceph-mon[17120]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-08-19T01:29:25.896 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:29:25 smithi083 ceph-mon[17120]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-08-19T01:29:25.896 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:29:25 smithi083 ceph-mon[17120]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-19T01:29:26.202 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:29:25 smithi117 ceph-mon[22016]: from='client.24272 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi083:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-08-19T01:29:26.203 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:29:25 smithi117 ceph-mon[22016]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-08-19T01:29:26.203 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:29:25 smithi117 ceph-mon[22016]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-08-19T01:29:26.203 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:29:25 smithi117 ceph-mon[22016]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-19T01:29:26.216 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:29:25 smithi167 ceph-mon[21875]: from='client.24272 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi083:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-08-19T01:29:26.216 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:29:25 smithi167 ceph-mon[21875]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-08-19T01:29:26.216 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:29:25 smithi167 ceph-mon[21875]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-08-19T01:29:26.216 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:29:25 smithi167 ceph-mon[21875]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-19T01:29:26.904 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:29:26 smithi083 ceph-mon[17120]: osdmap e41: 6 total, 6 up, 6 in 2024-08-19T01:29:26.905 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:29:26 smithi083 ceph-mon[17120]: pgmap v132: 1 pgs: 1 remapped+peering; 577 KiB data, 178 MiB used, 536 GiB / 536 GiB avail 2024-08-19T01:29:27.203 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:29:26 smithi117 ceph-mon[22016]: osdmap e41: 6 total, 6 up, 6 in 2024-08-19T01:29:27.203 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:29:26 smithi117 ceph-mon[22016]: pgmap v132: 1 pgs: 1 remapped+peering; 577 KiB data, 178 MiB used, 536 GiB / 536 GiB avail 2024-08-19T01:29:27.216 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:29:26 smithi167 ceph-mon[21875]: osdmap e41: 6 total, 6 up, 6 in 2024-08-19T01:29:27.216 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:29:26 smithi167 ceph-mon[21875]: pgmap v132: 1 pgs: 1 remapped+peering; 577 KiB data, 178 MiB used, 536 GiB / 536 GiB avail 2024-08-19T01:29:27.904 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:29:27 smithi083 ceph-mon[17120]: osdmap e42: 6 total, 6 up, 6 in 2024-08-19T01:29:28.190 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:29:27 smithi117 ceph-mon[22016]: osdmap e42: 6 total, 6 up, 6 in 2024-08-19T01:29:28.216 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:29:27 smithi167 ceph-mon[21875]: osdmap e42: 6 total, 6 up, 6 in 2024-08-19T01:29:29.155 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:29:28 smithi083 ceph-mon[17120]: from='client.? 172.21.15.83:0/165245731' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "78cf0a9b-9165-444f-b073-ebe13016de9d"}]: dispatch 2024-08-19T01:29:29.155 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:29:28 smithi083 ceph-mon[17120]: from='client.? 172.21.15.83:0/165245731' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "78cf0a9b-9165-444f-b073-ebe13016de9d"}]': finished 2024-08-19T01:29:29.155 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:29:28 smithi083 ceph-mon[17120]: osdmap e43: 7 total, 6 up, 7 in 2024-08-19T01:29:29.155 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:29:28 smithi083 ceph-mon[17120]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-08-19T01:29:29.155 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:29:28 smithi083 ceph-mon[17120]: pgmap v135: 1 pgs: 1 active+clean; 577 KiB data, 179 MiB used, 536 GiB / 536 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-08-19T01:29:29.155 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:29:28 smithi083 ceph-mon[17120]: from='client.? 172.21.15.83:0/112801183' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-08-19T01:29:29.203 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:29:28 smithi117 ceph-mon[22016]: from='client.? 172.21.15.83:0/165245731' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "78cf0a9b-9165-444f-b073-ebe13016de9d"}]: dispatch 2024-08-19T01:29:29.203 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:29:28 smithi117 ceph-mon[22016]: from='client.? 172.21.15.83:0/165245731' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "78cf0a9b-9165-444f-b073-ebe13016de9d"}]': finished 2024-08-19T01:29:29.203 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:29:28 smithi117 ceph-mon[22016]: osdmap e43: 7 total, 6 up, 7 in 2024-08-19T01:29:29.203 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:29:28 smithi117 ceph-mon[22016]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-08-19T01:29:29.203 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:29:28 smithi117 ceph-mon[22016]: pgmap v135: 1 pgs: 1 active+clean; 577 KiB data, 179 MiB used, 536 GiB / 536 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-08-19T01:29:29.203 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:29:28 smithi117 ceph-mon[22016]: from='client.? 172.21.15.83:0/112801183' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-08-19T01:29:29.216 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:29:28 smithi167 ceph-mon[21875]: from='client.? 172.21.15.83:0/165245731' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "78cf0a9b-9165-444f-b073-ebe13016de9d"}]: dispatch 2024-08-19T01:29:29.216 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:29:28 smithi167 ceph-mon[21875]: from='client.? 172.21.15.83:0/165245731' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "78cf0a9b-9165-444f-b073-ebe13016de9d"}]': finished 2024-08-19T01:29:29.216 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:29:28 smithi167 ceph-mon[21875]: osdmap e43: 7 total, 6 up, 7 in 2024-08-19T01:29:29.216 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:29:28 smithi167 ceph-mon[21875]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-08-19T01:29:29.216 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:29:28 smithi167 ceph-mon[21875]: pgmap v135: 1 pgs: 1 active+clean; 577 KiB data, 179 MiB used, 536 GiB / 536 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-08-19T01:29:29.217 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:29:28 smithi167 ceph-mon[21875]: from='client.? 172.21.15.83:0/112801183' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-08-19T01:29:31.655 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:29:31 smithi083 ceph-mon[17120]: pgmap v136: 1 pgs: 1 active+clean; 577 KiB data, 179 MiB used, 536 GiB / 536 GiB avail; 103 KiB/s, 0 objects/s recovering 2024-08-19T01:29:31.702 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:29:31 smithi117 ceph-mon[22016]: pgmap v136: 1 pgs: 1 active+clean; 577 KiB data, 179 MiB used, 536 GiB / 536 GiB avail; 103 KiB/s, 0 objects/s recovering 2024-08-19T01:29:31.716 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:29:31 smithi167 ceph-mon[21875]: pgmap v136: 1 pgs: 1 active+clean; 577 KiB data, 179 MiB used, 536 GiB / 536 GiB avail; 103 KiB/s, 0 objects/s recovering 2024-08-19T01:29:33.655 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:29:33 smithi083 ceph-mon[17120]: pgmap v137: 1 pgs: 1 active+clean; 577 KiB data, 179 MiB used, 536 GiB / 536 GiB avail; 88 KiB/s, 0 objects/s recovering 2024-08-19T01:29:33.703 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:29:33 smithi117 ceph-mon[22016]: pgmap v137: 1 pgs: 1 active+clean; 577 KiB data, 179 MiB used, 536 GiB / 536 GiB avail; 88 KiB/s, 0 objects/s recovering 2024-08-19T01:29:33.716 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:29:33 smithi167 ceph-mon[21875]: pgmap v137: 1 pgs: 1 active+clean; 577 KiB data, 179 MiB used, 536 GiB / 536 GiB avail; 88 KiB/s, 0 objects/s recovering 2024-08-19T01:29:35.494 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:29:35 smithi083 ceph-mon[17120]: pgmap v138: 1 pgs: 1 active+clean; 577 KiB data, 179 MiB used, 536 GiB / 536 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-08-19T01:29:35.702 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:29:35 smithi117 ceph-mon[22016]: pgmap v138: 1 pgs: 1 active+clean; 577 KiB data, 179 MiB used, 536 GiB / 536 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-08-19T01:29:35.718 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:29:35 smithi167 ceph-mon[21875]: pgmap v138: 1 pgs: 1 active+clean; 577 KiB data, 179 MiB used, 536 GiB / 536 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-08-19T01:29:37.655 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:29:37 smithi083 ceph-mon[17120]: pgmap v139: 1 pgs: 1 active+clean; 577 KiB data, 179 MiB used, 536 GiB / 536 GiB avail; 60 KiB/s, 0 objects/s recovering 2024-08-19T01:29:37.655 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:29:37 smithi083 ceph-mon[17120]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-08-19T01:29:37.655 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:29:37 smithi083 ceph-mon[17120]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-19T01:29:37.655 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:29:37 smithi083 ceph-mon[17120]: Deploying daemon osd.6 on smithi083 2024-08-19T01:29:37.703 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:29:37 smithi117 ceph-mon[22016]: pgmap v139: 1 pgs: 1 active+clean; 577 KiB data, 179 MiB used, 536 GiB / 536 GiB avail; 60 KiB/s, 0 objects/s recovering 2024-08-19T01:29:37.703 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:29:37 smithi117 ceph-mon[22016]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-08-19T01:29:37.703 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:29:37 smithi117 ceph-mon[22016]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-19T01:29:37.703 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:29:37 smithi117 ceph-mon[22016]: Deploying daemon osd.6 on smithi083 2024-08-19T01:29:37.716 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:29:37 smithi167 ceph-mon[21875]: pgmap v139: 1 pgs: 1 active+clean; 577 KiB data, 179 MiB used, 536 GiB / 536 GiB avail; 60 KiB/s, 0 objects/s recovering 2024-08-19T01:29:37.716 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:29:37 smithi167 ceph-mon[21875]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-08-19T01:29:37.716 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:29:37 smithi167 ceph-mon[21875]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-19T01:29:37.716 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:29:37 smithi167 ceph-mon[21875]: Deploying daemon osd.6 on smithi083 2024-08-19T01:29:39.545 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:29:39 smithi083 ceph-mon[17120]: pgmap v140: 1 pgs: 1 active+clean; 577 KiB data, 179 MiB used, 536 GiB / 536 GiB avail 2024-08-19T01:29:39.702 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:29:39 smithi117 ceph-mon[22016]: pgmap v140: 1 pgs: 1 active+clean; 577 KiB data, 179 MiB used, 536 GiB / 536 GiB avail 2024-08-19T01:29:39.716 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:29:39 smithi167 ceph-mon[21875]: pgmap v140: 1 pgs: 1 active+clean; 577 KiB data, 179 MiB used, 536 GiB / 536 GiB avail 2024-08-19T01:29:41.658 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:29:41 smithi083 ceph-mon[17120]: pgmap v141: 1 pgs: 1 active+clean; 577 KiB data, 179 MiB used, 536 GiB / 536 GiB avail 2024-08-19T01:29:41.702 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:29:41 smithi117 ceph-mon[22016]: pgmap v141: 1 pgs: 1 active+clean; 577 KiB data, 179 MiB used, 536 GiB / 536 GiB avail 2024-08-19T01:29:41.716 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:29:41 smithi167 ceph-mon[21875]: pgmap v141: 1 pgs: 1 active+clean; 577 KiB data, 179 MiB used, 536 GiB / 536 GiB avail 2024-08-19T01:29:43.492 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:29:43 smithi083 ceph-mon[17120]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 179 MiB used, 536 GiB / 536 GiB avail 2024-08-19T01:29:43.492 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:29:43 smithi083 ceph-mon[17120]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-19T01:29:43.492 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:29:43 smithi083 ceph-mon[17120]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' 2024-08-19T01:29:43.492 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:29:43 smithi083 ceph-mon[17120]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' 2024-08-19T01:29:43.702 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:29:43 smithi117 ceph-mon[22016]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 179 MiB used, 536 GiB / 536 GiB avail 2024-08-19T01:29:43.703 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:29:43 smithi117 ceph-mon[22016]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-19T01:29:43.703 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:29:43 smithi117 ceph-mon[22016]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' 2024-08-19T01:29:43.703 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:29:43 smithi117 ceph-mon[22016]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' 2024-08-19T01:29:43.716 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:29:43 smithi167 ceph-mon[21875]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 179 MiB used, 536 GiB / 536 GiB avail 2024-08-19T01:29:43.716 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:29:43 smithi167 ceph-mon[21875]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-19T01:29:43.716 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:29:43 smithi167 ceph-mon[21875]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' 2024-08-19T01:29:43.716 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:29:43 smithi167 ceph-mon[21875]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' 2024-08-19T01:29:45.650 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:29:45 smithi083 ceph-mon[17120]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 179 MiB used, 536 GiB / 536 GiB avail 2024-08-19T01:29:45.702 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:29:45 smithi117 ceph-mon[22016]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 179 MiB used, 536 GiB / 536 GiB avail 2024-08-19T01:29:45.716 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:29:45 smithi167 ceph-mon[21875]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 179 MiB used, 536 GiB / 536 GiB avail 2024-08-19T01:29:47.155 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:29:46 smithi083 ceph-mon[17120]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' 2024-08-19T01:29:47.155 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:29:46 smithi083 ceph-mon[17120]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' 2024-08-19T01:29:47.155 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:29:46 smithi083 ceph-mon[17120]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-19T01:29:47.156 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:29:46 smithi083 ceph-mon[17120]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-19T01:29:47.156 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:29:46 smithi083 ceph-mon[17120]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' 2024-08-19T01:29:47.156 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:29:46 smithi083 ceph-mon[17120]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 179 MiB used, 536 GiB / 536 GiB avail 2024-08-19T01:29:47.203 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:29:46 smithi117 ceph-mon[22016]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' 2024-08-19T01:29:47.203 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:29:46 smithi117 ceph-mon[22016]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' 2024-08-19T01:29:47.203 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:29:46 smithi117 ceph-mon[22016]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-19T01:29:47.203 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:29:46 smithi117 ceph-mon[22016]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-19T01:29:47.204 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:29:46 smithi117 ceph-mon[22016]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' 2024-08-19T01:29:47.204 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:29:46 smithi117 ceph-mon[22016]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 179 MiB used, 536 GiB / 536 GiB avail 2024-08-19T01:29:47.216 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:29:46 smithi167 ceph-mon[21875]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' 2024-08-19T01:29:47.216 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:29:46 smithi167 ceph-mon[21875]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' 2024-08-19T01:29:47.216 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:29:46 smithi167 ceph-mon[21875]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-19T01:29:47.217 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:29:46 smithi167 ceph-mon[21875]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-19T01:29:47.217 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:29:46 smithi167 ceph-mon[21875]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' 2024-08-19T01:29:47.217 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:29:46 smithi167 ceph-mon[21875]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 179 MiB used, 536 GiB / 536 GiB avail 2024-08-19T01:29:47.287 INFO:teuthology.orchestra.run.smithi083.stdout:Created osd(s) 6 on host 'smithi083' 2024-08-19T01:29:47.287 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:29:47.285+0000 7f5351600640 1 -- 172.21.15.83:0/2972543017 <== mgr.14150 v2:172.21.15.83:6800/683740916 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f5354062360 con 0x7f53380a2bb0 2024-08-19T01:29:47.290 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:29:47.288+0000 7f535b600640 1 -- 172.21.15.83:0/2972543017 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f53380a2bb0 msgr2=0x7f53380a5070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:29:47.290 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:29:47.288+0000 7f535b600640 1 --2- 172.21.15.83:0/2972543017 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f53380a2bb0 0x7f53380a5070 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f535419f6f0 tx=0x7f534000b480 comp rx=0 tx=0).stop 2024-08-19T01:29:47.290 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:29:47.288+0000 7f535b600640 1 -- 172.21.15.83:0/2972543017 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5354074980 msgr2=0x7f535419e1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:29:47.290 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:29:47.288+0000 7f535b600640 1 --2- 172.21.15.83:0/2972543017 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5354074980 0x7f535419e1d0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f5344005fd0 tx=0x7f534400df40 comp rx=0 tx=0).stop 2024-08-19T01:29:47.291 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:29:47.288+0000 7f535b600640 1 -- 172.21.15.83:0/2972543017 shutdown_connections 2024-08-19T01:29:47.291 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:29:47.288+0000 7f535b600640 1 --2- 172.21.15.83:0/2972543017 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f53380a2bb0 0x7f53380a5070 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:29:47.291 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:29:47.288+0000 7f535b600640 1 --2- 172.21.15.83:0/2972543017 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f535410df10 0x7f53541a5790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:29:47.291 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:29:47.288+0000 7f535b600640 1 --2- 172.21.15.83:0/2972543017 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5354075f50 0x7f535419e710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:29:47.291 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:29:47.288+0000 7f535b600640 1 --2- 172.21.15.83:0/2972543017 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5354074980 0x7f535419e1d0 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:29:47.292 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:29:47.288+0000 7f535b600640 1 -- 172.21.15.83:0/2972543017 >> 172.21.15.83:0/2972543017 conn(0x7f53540fd0c0 msgr2=0x7f53541012b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:29:47.292 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:29:47.289+0000 7f535b600640 1 -- 172.21.15.83:0/2972543017 shutdown_connections 2024-08-19T01:29:47.292 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:29:47.289+0000 7f535b600640 1 -- 172.21.15.83:0/2972543017 wait complete. 2024-08-19T01:29:48.039 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:29:47 smithi083 ceph-mon[17120]: from='osd.6 [v2:172.21.15.83:6818/260644552,v1:172.21.15.83:6819/260644552]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-08-19T01:29:48.039 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:29:47 smithi083 ceph-mon[17120]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-19T01:29:48.039 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:29:47 smithi083 ceph-mon[17120]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' 2024-08-19T01:29:48.039 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:29:47 smithi083 ceph-mon[17120]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' 2024-08-19T01:29:48.075 DEBUG:teuthology.orchestra.run.smithi083:osd.6> sudo journalctl -f -n 0 -u ceph-bad8780e-5dc9-11ef-bccf-c7b262605968@osd.6.service 2024-08-19T01:29:48.079 INFO:tasks.cephadm:Deploying osd.7 on smithi117 with /dev/vg_nvme/lv_2... 2024-08-19T01:29:48.079 DEBUG:teuthology.orchestra.run.smithi117:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833 ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid bad8780e-5dc9-11ef-bccf-c7b262605968 -- lvm zap /dev/vg_nvme/lv_2 2024-08-19T01:29:48.102 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:29:47 smithi117 ceph-mon[22016]: from='osd.6 [v2:172.21.15.83:6818/260644552,v1:172.21.15.83:6819/260644552]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-08-19T01:29:48.102 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:29:47 smithi117 ceph-mon[22016]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-19T01:29:48.102 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:29:47 smithi117 ceph-mon[22016]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' 2024-08-19T01:29:48.102 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:29:47 smithi117 ceph-mon[22016]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' 2024-08-19T01:29:48.216 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:29:47 smithi167 ceph-mon[21875]: from='osd.6 [v2:172.21.15.83:6818/260644552,v1:172.21.15.83:6819/260644552]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-08-19T01:29:48.217 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:29:47 smithi167 ceph-mon[21875]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-19T01:29:48.217 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:29:47 smithi167 ceph-mon[21875]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' 2024-08-19T01:29:48.217 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:29:47 smithi167 ceph-mon[21875]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' 2024-08-19T01:29:48.333 INFO:teuthology.orchestra.run.smithi117.stderr:Inferring config /var/lib/ceph/bad8780e-5dc9-11ef-bccf-c7b262605968/mon.b/config 2024-08-19T01:29:48.952 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:29:48 smithi117 ceph-mon[22016]: from='osd.6 [v2:172.21.15.83:6818/260644552,v1:172.21.15.83:6819/260644552]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-08-19T01:29:48.953 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:29:48 smithi117 ceph-mon[22016]: osdmap e44: 7 total, 6 up, 7 in 2024-08-19T01:29:48.953 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:29:48 smithi117 ceph-mon[22016]: from='osd.6 [v2:172.21.15.83:6818/260644552,v1:172.21.15.83:6819/260644552]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi083", "root=default"]}]: dispatch 2024-08-19T01:29:48.953 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:29:48 smithi117 ceph-mon[22016]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-08-19T01:29:48.953 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:29:48 smithi117 ceph-mon[22016]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 179 MiB used, 536 GiB / 536 GiB avail 2024-08-19T01:29:49.155 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:29:48 smithi083 ceph-mon[17120]: from='osd.6 [v2:172.21.15.83:6818/260644552,v1:172.21.15.83:6819/260644552]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-08-19T01:29:49.155 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:29:48 smithi083 ceph-mon[17120]: osdmap e44: 7 total, 6 up, 7 in 2024-08-19T01:29:49.156 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:29:48 smithi083 ceph-mon[17120]: from='osd.6 [v2:172.21.15.83:6818/260644552,v1:172.21.15.83:6819/260644552]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi083", "root=default"]}]: dispatch 2024-08-19T01:29:49.156 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:29:48 smithi083 ceph-mon[17120]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-08-19T01:29:49.156 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:29:48 smithi083 ceph-mon[17120]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 179 MiB used, 536 GiB / 536 GiB avail 2024-08-19T01:29:49.156 INFO:journalctl@ceph.osd.6.smithi083.stdout:Aug 19 01:29:48 smithi083 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-6[44809]: 2024-08-19T01:29:48.870+0000 7f3cc9600640 -1 osd.6 0 waiting for initial osdmap 2024-08-19T01:29:49.156 INFO:journalctl@ceph.osd.6.smithi083.stdout:Aug 19 01:29:48 smithi083 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-6[44809]: 2024-08-19T01:29:48.881+0000 7f3cc5000640 -1 osd.6 45 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-08-19T01:29:49.216 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:29:48 smithi167 ceph-mon[21875]: from='osd.6 [v2:172.21.15.83:6818/260644552,v1:172.21.15.83:6819/260644552]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-08-19T01:29:49.216 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:29:48 smithi167 ceph-mon[21875]: osdmap e44: 7 total, 6 up, 7 in 2024-08-19T01:29:49.217 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:29:48 smithi167 ceph-mon[21875]: from='osd.6 [v2:172.21.15.83:6818/260644552,v1:172.21.15.83:6819/260644552]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi083", "root=default"]}]: dispatch 2024-08-19T01:29:49.217 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:29:48 smithi167 ceph-mon[21875]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-08-19T01:29:49.217 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:29:48 smithi167 ceph-mon[21875]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 179 MiB used, 536 GiB / 536 GiB avail 2024-08-19T01:29:49.905 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:29:49 smithi083 ceph-mon[17120]: purged_snaps scrub starts 2024-08-19T01:29:49.905 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:29:49 smithi083 ceph-mon[17120]: purged_snaps scrub ok 2024-08-19T01:29:49.905 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:29:49 smithi083 ceph-mon[17120]: from='osd.6 [v2:172.21.15.83:6818/260644552,v1:172.21.15.83:6819/260644552]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi083", "root=default"]}]': finished 2024-08-19T01:29:49.905 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:29:49 smithi083 ceph-mon[17120]: osdmap e45: 7 total, 6 up, 7 in 2024-08-19T01:29:49.905 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:29:49 smithi083 ceph-mon[17120]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-08-19T01:29:49.905 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:29:49 smithi083 ceph-mon[17120]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-08-19T01:29:49.953 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:29:49 smithi117 ceph-mon[22016]: purged_snaps scrub starts 2024-08-19T01:29:49.953 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:29:49 smithi117 ceph-mon[22016]: purged_snaps scrub ok 2024-08-19T01:29:49.953 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:29:49 smithi117 ceph-mon[22016]: from='osd.6 [v2:172.21.15.83:6818/260644552,v1:172.21.15.83:6819/260644552]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi083", "root=default"]}]': finished 2024-08-19T01:29:49.953 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:29:49 smithi117 ceph-mon[22016]: osdmap e45: 7 total, 6 up, 7 in 2024-08-19T01:29:49.954 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:29:49 smithi117 ceph-mon[22016]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-08-19T01:29:49.954 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:29:49 smithi117 ceph-mon[22016]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-08-19T01:29:50.216 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:29:49 smithi167 ceph-mon[21875]: purged_snaps scrub starts 2024-08-19T01:29:50.216 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:29:49 smithi167 ceph-mon[21875]: purged_snaps scrub ok 2024-08-19T01:29:50.216 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:29:49 smithi167 ceph-mon[21875]: from='osd.6 [v2:172.21.15.83:6818/260644552,v1:172.21.15.83:6819/260644552]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi083", "root=default"]}]': finished 2024-08-19T01:29:50.216 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:29:49 smithi167 ceph-mon[21875]: osdmap e45: 7 total, 6 up, 7 in 2024-08-19T01:29:50.216 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:29:49 smithi167 ceph-mon[21875]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-08-19T01:29:50.217 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:29:49 smithi167 ceph-mon[21875]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-08-19T01:29:50.696 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-08-19T01:29:50.721 DEBUG:teuthology.orchestra.run.smithi117:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid bad8780e-5dc9-11ef-bccf-c7b262605968 -- ceph orch daemon add osd smithi117:vg_nvme/lv_2 2024-08-19T01:29:51.014 INFO:teuthology.orchestra.run.smithi117.stderr:Inferring config /var/lib/ceph/bad8780e-5dc9-11ef-bccf-c7b262605968/mon.b/config 2024-08-19T01:29:51.136 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:29:50 smithi117 ceph-mon[22016]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-08-19T01:29:51.136 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:29:50 smithi117 ceph-mon[22016]: osd.6 [v2:172.21.15.83:6818/260644552,v1:172.21.15.83:6819/260644552] boot 2024-08-19T01:29:51.136 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:29:50 smithi117 ceph-mon[22016]: osdmap e46: 7 total, 7 up, 7 in 2024-08-19T01:29:51.136 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:29:50 smithi117 ceph-mon[22016]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-08-19T01:29:51.136 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:29:50 smithi117 ceph-mon[22016]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 179 MiB used, 536 GiB / 536 GiB avail 2024-08-19T01:29:51.154 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:29:50 smithi083 ceph-mon[17120]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-08-19T01:29:51.155 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:29:50 smithi083 ceph-mon[17120]: osd.6 [v2:172.21.15.83:6818/260644552,v1:172.21.15.83:6819/260644552] boot 2024-08-19T01:29:51.155 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:29:50 smithi083 ceph-mon[17120]: osdmap e46: 7 total, 7 up, 7 in 2024-08-19T01:29:51.155 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:29:50 smithi083 ceph-mon[17120]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-08-19T01:29:51.155 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:29:50 smithi083 ceph-mon[17120]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 179 MiB used, 536 GiB / 536 GiB avail 2024-08-19T01:29:51.216 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:29:50 smithi167 ceph-mon[21875]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-08-19T01:29:51.216 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:29:50 smithi167 ceph-mon[21875]: osd.6 [v2:172.21.15.83:6818/260644552,v1:172.21.15.83:6819/260644552] boot 2024-08-19T01:29:51.216 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:29:50 smithi167 ceph-mon[21875]: osdmap e46: 7 total, 7 up, 7 in 2024-08-19T01:29:51.217 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:29:50 smithi167 ceph-mon[21875]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-08-19T01:29:51.217 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:29:50 smithi167 ceph-mon[21875]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 179 MiB used, 536 GiB / 536 GiB avail 2024-08-19T01:29:51.421 INFO:teuthology.orchestra.run.smithi117.stderr:2024-08-19T01:29:51.419+0000 7fcdf4a9c640 1 -- 172.21.15.117:0/1448250415 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcdf0100840 msgr2=0x7fcdf0100c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:29:51.421 INFO:teuthology.orchestra.run.smithi117.stderr:2024-08-19T01:29:51.419+0000 7fcdf4a9c640 1 --2- 172.21.15.117:0/1448250415 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcdf0100840 0x7fcdf0100c40 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7fcde0009900 tx=0x7fcde002a410 comp rx=0 tx=0).stop 2024-08-19T01:29:51.421 INFO:teuthology.orchestra.run.smithi117.stderr:2024-08-19T01:29:51.419+0000 7fcdf4a9c640 1 -- 172.21.15.117:0/1448250415 shutdown_connections 2024-08-19T01:29:51.421 INFO:teuthology.orchestra.run.smithi117.stderr:2024-08-19T01:29:51.419+0000 7fcdf4a9c640 1 --2- 172.21.15.117:0/1448250415 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcdf0102400 0x7fcdf0108c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:29:51.421 INFO:teuthology.orchestra.run.smithi117.stderr:2024-08-19T01:29:51.419+0000 7fcdf4a9c640 1 --2- 172.21.15.117:0/1448250415 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fcdf0101a40 0x7fcdf0101ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:29:51.422 INFO:teuthology.orchestra.run.smithi117.stderr:2024-08-19T01:29:51.419+0000 7fcdf4a9c640 1 --2- 172.21.15.117:0/1448250415 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcdf0100840 0x7fcdf0100c40 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:29:51.422 INFO:teuthology.orchestra.run.smithi117.stderr:2024-08-19T01:29:51.419+0000 7fcdf4a9c640 1 -- 172.21.15.117:0/1448250415 >> 172.21.15.117:0/1448250415 conn(0x7fcdf00fbfd0 msgr2=0x7fcdf00fe410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:29:51.422 INFO:teuthology.orchestra.run.smithi117.stderr:2024-08-19T01:29:51.419+0000 7fcdf4a9c640 1 -- 172.21.15.117:0/1448250415 shutdown_connections 2024-08-19T01:29:51.422 INFO:teuthology.orchestra.run.smithi117.stderr:2024-08-19T01:29:51.419+0000 7fcdf4a9c640 1 -- 172.21.15.117:0/1448250415 wait complete. 2024-08-19T01:29:51.422 INFO:teuthology.orchestra.run.smithi117.stderr:2024-08-19T01:29:51.420+0000 7fcdf4a9c640 1 Processor -- start 2024-08-19T01:29:51.422 INFO:teuthology.orchestra.run.smithi117.stderr:2024-08-19T01:29:51.420+0000 7fcdf4a9c640 1 -- start start 2024-08-19T01:29:51.424 INFO:teuthology.orchestra.run.smithi117.stderr:2024-08-19T01:29:51.420+0000 7fcdf4a9c640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcdf0100840 0x7fcdf0199f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:29:51.424 INFO:teuthology.orchestra.run.smithi117.stderr:2024-08-19T01:29:51.420+0000 7fcdf4a9c640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fcdf0101a40 0x7fcdf019a440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:29:51.424 INFO:teuthology.orchestra.run.smithi117.stderr:2024-08-19T01:29:51.420+0000 7fcdf4a9c640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcdf0102400 0x7fcdf01a14c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:29:51.424 INFO:teuthology.orchestra.run.smithi117.stderr:2024-08-19T01:29:51.420+0000 7fcdf4a9c640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fcdf010d630 con 0x7fcdf0100840 2024-08-19T01:29:51.424 INFO:teuthology.orchestra.run.smithi117.stderr:2024-08-19T01:29:51.420+0000 7fcdf4a9c640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fcdf010d4b0 con 0x7fcdf0102400 2024-08-19T01:29:51.424 INFO:teuthology.orchestra.run.smithi117.stderr:2024-08-19T01:29:51.420+0000 7fcdf4a9c640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fcdf010d7b0 con 0x7fcdf0101a40 2024-08-19T01:29:51.424 INFO:teuthology.orchestra.run.smithi117.stderr:2024-08-19T01:29:51.420+0000 7fcdeed9d640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcdf0102400 0x7fcdf01a14c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:29:51.424 INFO:teuthology.orchestra.run.smithi117.stderr:2024-08-19T01:29:51.420+0000 7fcdeed9d640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcdf0102400 0x7fcdf01a14c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:58412/0 (socket says 172.21.15.117:58412) 2024-08-19T01:29:51.424 INFO:teuthology.orchestra.run.smithi117.stderr:2024-08-19T01:29:51.420+0000 7fcdee59c640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcdf0100840 0x7fcdf0199f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:29:51.424 INFO:teuthology.orchestra.run.smithi117.stderr:2024-08-19T01:29:51.420+0000 7fcdeed9d640 1 -- 172.21.15.117:0/4056837916 learned_addr learned my addr 172.21.15.117:0/4056837916 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-08-19T01:29:51.424 INFO:teuthology.orchestra.run.smithi117.stderr:2024-08-19T01:29:51.420+0000 7fcdedd9b640 1 --2- 172.21.15.117:0/4056837916 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fcdf0101a40 0x7fcdf019a440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:29:51.424 INFO:teuthology.orchestra.run.smithi117.stderr:2024-08-19T01:29:51.421+0000 7fcdeed9d640 1 -- 172.21.15.117:0/4056837916 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fcdf0101a40 msgr2=0x7fcdf019a440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:29:51.425 INFO:teuthology.orchestra.run.smithi117.stderr:2024-08-19T01:29:51.421+0000 7fcdeed9d640 1 --2- 172.21.15.117:0/4056837916 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fcdf0101a40 0x7fcdf019a440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:29:51.425 INFO:teuthology.orchestra.run.smithi117.stderr:2024-08-19T01:29:51.421+0000 7fcdeed9d640 1 -- 172.21.15.117:0/4056837916 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcdf0100840 msgr2=0x7fcdf0199f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:29:51.425 INFO:teuthology.orchestra.run.smithi117.stderr:2024-08-19T01:29:51.421+0000 7fcdeed9d640 1 --2- 172.21.15.117:0/4056837916 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcdf0100840 0x7fcdf0199f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:29:51.425 INFO:teuthology.orchestra.run.smithi117.stderr:2024-08-19T01:29:51.421+0000 7fcdeed9d640 1 -- 172.21.15.117:0/4056837916 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcdf01a1b30 con 0x7fcdf0102400 2024-08-19T01:29:51.425 INFO:teuthology.orchestra.run.smithi117.stderr:2024-08-19T01:29:51.421+0000 7fcdeed9d640 1 --2- 172.21.15.117:0/4056837916 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcdf0102400 0x7fcdf01a14c0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fcde800cad0 tx=0x7fcde800cf90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:29:51.425 INFO:teuthology.orchestra.run.smithi117.stderr:2024-08-19T01:29:51.421+0000 7fcde77fe640 1 -- 172.21.15.117:0/4056837916 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcde800ae20 con 0x7fcdf0102400 2024-08-19T01:29:51.425 INFO:teuthology.orchestra.run.smithi117.stderr:2024-08-19T01:29:51.421+0000 7fcde77fe640 1 -- 172.21.15.117:0/4056837916 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcde8005020 con 0x7fcdf0102400 2024-08-19T01:29:51.425 INFO:teuthology.orchestra.run.smithi117.stderr:2024-08-19T01:29:51.421+0000 7fcde77fe640 1 -- 172.21.15.117:0/4056837916 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcde800f6f0 con 0x7fcdf0102400 2024-08-19T01:29:51.425 INFO:teuthology.orchestra.run.smithi117.stderr:2024-08-19T01:29:51.421+0000 7fcdf4a9c640 1 -- 172.21.15.117:0/4056837916 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcdf01a1e20 con 0x7fcdf0102400 2024-08-19T01:29:51.425 INFO:teuthology.orchestra.run.smithi117.stderr:2024-08-19T01:29:51.421+0000 7fcdf4a9c640 1 -- 172.21.15.117:0/4056837916 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcdf01a2300 con 0x7fcdf0102400 2024-08-19T01:29:51.425 INFO:teuthology.orchestra.run.smithi117.stderr:2024-08-19T01:29:51.422+0000 7fcdf4a9c640 1 -- 172.21.15.117:0/4056837916 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcdf01a25c0 con 0x7fcdf0102400 2024-08-19T01:29:51.426 INFO:teuthology.orchestra.run.smithi117.stderr:2024-08-19T01:29:51.423+0000 7fcde77fe640 1 -- 172.21.15.117:0/4056837916 <== mon.2 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7fcde80051d0 con 0x7fcdf0102400 2024-08-19T01:29:51.426 INFO:teuthology.orchestra.run.smithi117.stderr:2024-08-19T01:29:51.424+0000 7fcde77fe640 1 --2- 172.21.15.117:0/4056837916 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7fcddc0a2bb0 0x7fcddc0a5070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:29:51.426 INFO:teuthology.orchestra.run.smithi117.stderr:2024-08-19T01:29:51.424+0000 7fcde77fe640 1 -- 172.21.15.117:0/4056837916 <== mon.2 v2:172.21.15.117:3300/0 5 ==== osd_map(47..47 src has 1..47) ==== 4965+0+0 (secure 0 0 0) 0x7fcde80ce720 con 0x7fcdf0102400 2024-08-19T01:29:51.426 INFO:teuthology.orchestra.run.smithi117.stderr:2024-08-19T01:29:51.424+0000 7fcdee59c640 1 --2- 172.21.15.117:0/4056837916 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7fcddc0a2bb0 0x7fcddc0a5070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:29:51.426 INFO:teuthology.orchestra.run.smithi117.stderr:2024-08-19T01:29:51.425+0000 7fcdee59c640 1 --2- 172.21.15.117:0/4056837916 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7fcddc0a2bb0 0x7fcddc0a5070 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fcde0005ab0 tx=0x7fcde0000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:29:51.429 INFO:teuthology.orchestra.run.smithi117.stderr:2024-08-19T01:29:51.427+0000 7fcde77fe640 1 -- 172.21.15.117:0/4056837916 <== mon.2 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7fcde8014030 con 0x7fcdf0102400 2024-08-19T01:29:51.604 INFO:teuthology.orchestra.run.smithi117.stderr:2024-08-19T01:29:51.603+0000 7fcdf4a9c640 1 -- 172.21.15.117:0/4056837916 --> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi117:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7fcdf01a2790 con 0x7fcddc0a2bb0 2024-08-19T01:29:51.906 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:29:51 smithi083 ceph-mon[17120]: osdmap e47: 7 total, 7 up, 7 in 2024-08-19T01:29:51.906 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:29:51 smithi083 ceph-mon[17120]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-08-19T01:29:51.906 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:29:51 smithi083 ceph-mon[17120]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-08-19T01:29:51.906 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:29:51 smithi083 ceph-mon[17120]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-19T01:29:51.906 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:29:51 smithi083 ceph-mon[17120]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' 2024-08-19T01:29:51.906 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:29:51 smithi083 ceph-mon[17120]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' 2024-08-19T01:29:51.906 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:29:51 smithi083 ceph-mon[17120]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-08-19T01:29:51.907 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:29:51 smithi083 ceph-mon[17120]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-08-19T01:29:51.907 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:29:51 smithi083 ceph-mon[17120]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-08-19T01:29:51.907 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:29:51 smithi083 ceph-mon[17120]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' 2024-08-19T01:29:51.907 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:29:51 smithi083 ceph-mon[17120]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-19T01:29:51.907 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:29:51 smithi083 ceph-mon[17120]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-19T01:29:51.907 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:29:51 smithi083 ceph-mon[17120]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' 2024-08-19T01:29:52.198 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:29:51 smithi117 ceph-mon[22016]: osdmap e47: 7 total, 7 up, 7 in 2024-08-19T01:29:52.199 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:29:51 smithi117 ceph-mon[22016]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-08-19T01:29:52.199 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:29:51 smithi117 ceph-mon[22016]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-08-19T01:29:52.199 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:29:51 smithi117 ceph-mon[22016]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-19T01:29:52.199 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:29:51 smithi117 ceph-mon[22016]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' 2024-08-19T01:29:52.199 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:29:51 smithi117 ceph-mon[22016]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' 2024-08-19T01:29:52.199 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:29:51 smithi117 ceph-mon[22016]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-08-19T01:29:52.199 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:29:51 smithi117 ceph-mon[22016]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-08-19T01:29:52.199 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:29:51 smithi117 ceph-mon[22016]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-08-19T01:29:52.199 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:29:51 smithi117 ceph-mon[22016]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' 2024-08-19T01:29:52.199 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:29:51 smithi117 ceph-mon[22016]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-19T01:29:52.199 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:29:51 smithi117 ceph-mon[22016]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-19T01:29:52.199 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:29:51 smithi117 ceph-mon[22016]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' 2024-08-19T01:29:52.216 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:29:51 smithi167 ceph-mon[21875]: osdmap e47: 7 total, 7 up, 7 in 2024-08-19T01:29:52.216 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:29:51 smithi167 ceph-mon[21875]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-08-19T01:29:52.216 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:29:51 smithi167 ceph-mon[21875]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-08-19T01:29:52.216 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:29:51 smithi167 ceph-mon[21875]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-19T01:29:52.217 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:29:51 smithi167 ceph-mon[21875]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' 2024-08-19T01:29:52.217 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:29:51 smithi167 ceph-mon[21875]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' 2024-08-19T01:29:52.217 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:29:51 smithi167 ceph-mon[21875]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-08-19T01:29:52.217 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:29:51 smithi167 ceph-mon[21875]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-08-19T01:29:52.217 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:29:51 smithi167 ceph-mon[21875]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-08-19T01:29:52.217 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:29:51 smithi167 ceph-mon[21875]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' 2024-08-19T01:29:52.217 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:29:51 smithi167 ceph-mon[21875]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-19T01:29:52.217 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:29:51 smithi167 ceph-mon[21875]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-19T01:29:52.217 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:29:51 smithi167 ceph-mon[21875]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' 2024-08-19T01:29:53.136 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:29:52 smithi117 ceph-mon[22016]: from='client.24299 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi117:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-08-19T01:29:53.136 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:29:52 smithi117 ceph-mon[22016]: Detected new or changed devices on smithi083 2024-08-19T01:29:53.136 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:29:52 smithi117 ceph-mon[22016]: Adjusting osd_memory_target on smithi083 to 5696M 2024-08-19T01:29:53.137 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:29:52 smithi117 ceph-mon[22016]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 205 MiB used, 626 GiB / 626 GiB avail 2024-08-19T01:29:53.154 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:29:52 smithi083 ceph-mon[17120]: from='client.24299 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi117:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-08-19T01:29:53.155 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:29:52 smithi083 ceph-mon[17120]: Detected new or changed devices on smithi083 2024-08-19T01:29:53.155 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:29:52 smithi083 ceph-mon[17120]: Adjusting osd_memory_target on smithi083 to 5696M 2024-08-19T01:29:53.155 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:29:52 smithi083 ceph-mon[17120]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 205 MiB used, 626 GiB / 626 GiB avail 2024-08-19T01:29:53.216 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:29:52 smithi167 ceph-mon[21875]: from='client.24299 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi117:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-08-19T01:29:53.216 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:29:52 smithi167 ceph-mon[21875]: Detected new or changed devices on smithi083 2024-08-19T01:29:53.217 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:29:52 smithi167 ceph-mon[21875]: Adjusting osd_memory_target on smithi083 to 5696M 2024-08-19T01:29:53.217 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:29:52 smithi167 ceph-mon[21875]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 205 MiB used, 626 GiB / 626 GiB avail 2024-08-19T01:29:54.264 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:29:53 smithi117 ceph-mon[22016]: from='client.? 172.21.15.117:0/925435873' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ee524f7f-672e-430e-9f58-1a9117513466"}]: dispatch 2024-08-19T01:29:54.265 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:29:53 smithi117 ceph-mon[22016]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ee524f7f-672e-430e-9f58-1a9117513466"}]: dispatch 2024-08-19T01:29:54.265 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:29:53 smithi117 ceph-mon[22016]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "ee524f7f-672e-430e-9f58-1a9117513466"}]': finished 2024-08-19T01:29:54.265 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:29:53 smithi117 ceph-mon[22016]: osdmap e48: 8 total, 7 up, 8 in 2024-08-19T01:29:54.265 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:29:53 smithi117 ceph-mon[22016]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-08-19T01:29:54.405 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:29:53 smithi083 ceph-mon[17120]: from='client.? 172.21.15.117:0/925435873' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ee524f7f-672e-430e-9f58-1a9117513466"}]: dispatch 2024-08-19T01:29:54.405 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:29:53 smithi083 ceph-mon[17120]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ee524f7f-672e-430e-9f58-1a9117513466"}]: dispatch 2024-08-19T01:29:54.405 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:29:53 smithi083 ceph-mon[17120]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "ee524f7f-672e-430e-9f58-1a9117513466"}]': finished 2024-08-19T01:29:54.405 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:29:53 smithi083 ceph-mon[17120]: osdmap e48: 8 total, 7 up, 8 in 2024-08-19T01:29:54.405 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:29:53 smithi083 ceph-mon[17120]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-08-19T01:29:54.466 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:29:53 smithi167 ceph-mon[21875]: from='client.? 172.21.15.117:0/925435873' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ee524f7f-672e-430e-9f58-1a9117513466"}]: dispatch 2024-08-19T01:29:54.466 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:29:53 smithi167 ceph-mon[21875]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ee524f7f-672e-430e-9f58-1a9117513466"}]: dispatch 2024-08-19T01:29:54.466 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:29:53 smithi167 ceph-mon[21875]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "ee524f7f-672e-430e-9f58-1a9117513466"}]': finished 2024-08-19T01:29:54.466 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:29:53 smithi167 ceph-mon[21875]: osdmap e48: 8 total, 7 up, 8 in 2024-08-19T01:29:54.466 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:29:53 smithi167 ceph-mon[21875]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-08-19T01:29:55.405 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:29:54 smithi083 ceph-mon[17120]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-08-19T01:29:55.405 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:29:54 smithi083 ceph-mon[17120]: from='client.? 172.21.15.117:0/1609414393' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-08-19T01:29:55.452 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:29:54 smithi117 ceph-mon[22016]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-08-19T01:29:55.453 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:29:54 smithi117 ceph-mon[22016]: from='client.? 172.21.15.117:0/1609414393' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-08-19T01:29:55.466 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:29:54 smithi167 ceph-mon[21875]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-08-19T01:29:55.466 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:29:54 smithi167 ceph-mon[21875]: from='client.? 172.21.15.117:0/1609414393' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-08-19T01:29:57.655 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:29:57 smithi083 ceph-mon[17120]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-08-19T01:29:57.673 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:29:57 smithi117 ceph-mon[22016]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-08-19T01:29:57.716 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:29:57 smithi167 ceph-mon[21875]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-08-19T01:29:59.654 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:29:59 smithi083 ceph-mon[17120]: pgmap v155: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-08-19T01:29:59.702 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:29:59 smithi117 ceph-mon[22016]: pgmap v155: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-08-19T01:29:59.716 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:29:59 smithi167 ceph-mon[21875]: pgmap v155: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-08-19T01:30:00.654 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:30:00 smithi083 ceph-mon[17120]: overall HEALTH_OK 2024-08-19T01:30:00.702 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:30:00 smithi117 ceph-mon[22016]: overall HEALTH_OK 2024-08-19T01:30:00.716 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:30:00 smithi167 ceph-mon[21875]: overall HEALTH_OK 2024-08-19T01:30:01.655 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:30:01 smithi083 ceph-mon[17120]: pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-08-19T01:30:01.655 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:30:01 smithi083 ceph-mon[17120]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-08-19T01:30:01.655 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:30:01 smithi083 ceph-mon[17120]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-19T01:30:01.702 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:30:01 smithi117 ceph-mon[22016]: pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-08-19T01:30:01.702 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:30:01 smithi117 ceph-mon[22016]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-08-19T01:30:01.702 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:30:01 smithi117 ceph-mon[22016]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-19T01:30:01.716 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:30:01 smithi167 ceph-mon[21875]: pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-08-19T01:30:01.716 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:30:01 smithi167 ceph-mon[21875]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-08-19T01:30:01.716 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:30:01 smithi167 ceph-mon[21875]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-19T01:30:02.452 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:30:02 smithi117 ceph-mon[22016]: Deploying daemon osd.7 on smithi117 2024-08-19T01:30:02.654 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:30:02 smithi083 ceph-mon[17120]: Deploying daemon osd.7 on smithi117 2024-08-19T01:30:02.716 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:30:02 smithi167 ceph-mon[21875]: Deploying daemon osd.7 on smithi117 2024-08-19T01:30:03.593 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:30:03 smithi117 ceph-mon[22016]: pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-08-19T01:30:03.655 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:30:03 smithi083 ceph-mon[17120]: pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-08-19T01:30:03.716 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:30:03 smithi167 ceph-mon[21875]: pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-08-19T01:30:05.532 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:30:05 smithi117 ceph-mon[22016]: pgmap v158: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-08-19T01:30:05.716 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:30:05 smithi167 ceph-mon[21875]: pgmap v158: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-08-19T01:30:05.905 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:30:05 smithi083 ceph-mon[17120]: pgmap v158: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-08-19T01:30:06.426 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:30:06 smithi117 ceph-mon[22016]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-19T01:30:06.426 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:30:06 smithi117 ceph-mon[22016]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' 2024-08-19T01:30:06.426 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:30:06 smithi117 ceph-mon[22016]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' 2024-08-19T01:30:06.716 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:30:06 smithi167 ceph-mon[21875]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-19T01:30:06.716 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:30:06 smithi167 ceph-mon[21875]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' 2024-08-19T01:30:06.716 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:30:06 smithi167 ceph-mon[21875]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' 2024-08-19T01:30:06.904 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:30:06 smithi083 ceph-mon[17120]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-19T01:30:06.905 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:30:06 smithi083 ceph-mon[17120]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' 2024-08-19T01:30:06.905 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:30:06 smithi083 ceph-mon[17120]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' 2024-08-19T01:30:07.674 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:30:07 smithi117 ceph-mon[22016]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-08-19T01:30:07.716 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:30:07 smithi167 ceph-mon[21875]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-08-19T01:30:07.905 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:30:07 smithi083 ceph-mon[17120]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-08-19T01:30:08.997 INFO:teuthology.orchestra.run.smithi117.stderr:2024-08-19T01:30:08.994+0000 7fcde77fe640 1 -- 172.21.15.117:0/4056837916 <== mgr.14150 v2:172.21.15.83:6800/683740916 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fcdf01a2790 con 0x7fcddc0a2bb0 2024-08-19T01:30:08.997 INFO:teuthology.orchestra.run.smithi117.stdout:Created osd(s) 7 on host 'smithi117' 2024-08-19T01:30:08.999 INFO:teuthology.orchestra.run.smithi117.stderr:2024-08-19T01:30:08.996+0000 7fcdf4a9c640 1 -- 172.21.15.117:0/4056837916 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7fcddc0a2bb0 msgr2=0x7fcddc0a5070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:30:08.999 INFO:teuthology.orchestra.run.smithi117.stderr:2024-08-19T01:30:08.996+0000 7fcdf4a9c640 1 --2- 172.21.15.117:0/4056837916 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7fcddc0a2bb0 0x7fcddc0a5070 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fcde0005ab0 tx=0x7fcde0000bc0 comp rx=0 tx=0).stop 2024-08-19T01:30:08.999 INFO:teuthology.orchestra.run.smithi117.stderr:2024-08-19T01:30:08.996+0000 7fcdf4a9c640 1 -- 172.21.15.117:0/4056837916 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcdf0102400 msgr2=0x7fcdf01a14c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:30:08.999 INFO:teuthology.orchestra.run.smithi117.stderr:2024-08-19T01:30:08.996+0000 7fcdf4a9c640 1 --2- 172.21.15.117:0/4056837916 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcdf0102400 0x7fcdf01a14c0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fcde800cad0 tx=0x7fcde800cf90 comp rx=0 tx=0).stop 2024-08-19T01:30:08.999 INFO:teuthology.orchestra.run.smithi117.stderr:2024-08-19T01:30:08.996+0000 7fcdf4a9c640 1 -- 172.21.15.117:0/4056837916 shutdown_connections 2024-08-19T01:30:08.999 INFO:teuthology.orchestra.run.smithi117.stderr:2024-08-19T01:30:08.996+0000 7fcdf4a9c640 1 --2- 172.21.15.117:0/4056837916 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7fcddc0a2bb0 0x7fcddc0a5070 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:30:08.999 INFO:teuthology.orchestra.run.smithi117.stderr:2024-08-19T01:30:08.996+0000 7fcdf4a9c640 1 --2- 172.21.15.117:0/4056837916 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcdf0102400 0x7fcdf01a14c0 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:30:09.000 INFO:teuthology.orchestra.run.smithi117.stderr:2024-08-19T01:30:08.996+0000 7fcdf4a9c640 1 --2- 172.21.15.117:0/4056837916 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fcdf0101a40 0x7fcdf019a440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:30:09.000 INFO:teuthology.orchestra.run.smithi117.stderr:2024-08-19T01:30:08.996+0000 7fcdf4a9c640 1 --2- 172.21.15.117:0/4056837916 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcdf0100840 0x7fcdf0199f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:30:09.000 INFO:teuthology.orchestra.run.smithi117.stderr:2024-08-19T01:30:08.996+0000 7fcdf4a9c640 1 -- 172.21.15.117:0/4056837916 >> 172.21.15.117:0/4056837916 conn(0x7fcdf00fbfd0 msgr2=0x7fcdf0107200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:30:09.000 INFO:teuthology.orchestra.run.smithi117.stderr:2024-08-19T01:30:08.997+0000 7fcdf4a9c640 1 -- 172.21.15.117:0/4056837916 shutdown_connections 2024-08-19T01:30:09.000 INFO:teuthology.orchestra.run.smithi117.stderr:2024-08-19T01:30:08.997+0000 7fcdf4a9c640 1 -- 172.21.15.117:0/4056837916 wait complete. 2024-08-19T01:30:09.601 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:30:09 smithi117 ceph-mon[22016]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-08-19T01:30:09.601 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:30:09 smithi117 ceph-mon[22016]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' 2024-08-19T01:30:09.601 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:30:09 smithi117 ceph-mon[22016]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' 2024-08-19T01:30:09.601 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:30:09 smithi117 ceph-mon[22016]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-19T01:30:09.601 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:30:09 smithi117 ceph-mon[22016]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-19T01:30:09.601 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:30:09 smithi117 ceph-mon[22016]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' 2024-08-19T01:30:09.602 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:30:09 smithi117 ceph-mon[22016]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-19T01:30:09.602 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:30:09 smithi117 ceph-mon[22016]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' 2024-08-19T01:30:09.602 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:30:09 smithi117 ceph-mon[22016]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' 2024-08-19T01:30:09.629 DEBUG:teuthology.orchestra.run.smithi117:osd.7> sudo journalctl -f -n 0 -u ceph-bad8780e-5dc9-11ef-bccf-c7b262605968@osd.7.service 2024-08-19T01:30:09.631 INFO:tasks.cephadm:Deploying osd.8 on smithi167 with /dev/vg_nvme/lv_2... 2024-08-19T01:30:09.631 DEBUG:teuthology.orchestra.run.smithi167:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833 ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid bad8780e-5dc9-11ef-bccf-c7b262605968 -- lvm zap /dev/vg_nvme/lv_2 2024-08-19T01:30:09.655 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:30:09 smithi083 ceph-mon[17120]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-08-19T01:30:09.655 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:30:09 smithi083 ceph-mon[17120]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' 2024-08-19T01:30:09.655 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:30:09 smithi083 ceph-mon[17120]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' 2024-08-19T01:30:09.655 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:30:09 smithi083 ceph-mon[17120]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-19T01:30:09.655 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:30:09 smithi083 ceph-mon[17120]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-19T01:30:09.655 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:30:09 smithi083 ceph-mon[17120]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' 2024-08-19T01:30:09.655 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:30:09 smithi083 ceph-mon[17120]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-19T01:30:09.655 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:30:09 smithi083 ceph-mon[17120]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' 2024-08-19T01:30:09.655 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:30:09 smithi083 ceph-mon[17120]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' 2024-08-19T01:30:09.660 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:30:09 smithi167 ceph-mon[21875]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-08-19T01:30:09.660 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:30:09 smithi167 ceph-mon[21875]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' 2024-08-19T01:30:09.660 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:30:09 smithi167 ceph-mon[21875]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' 2024-08-19T01:30:09.660 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:30:09 smithi167 ceph-mon[21875]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-19T01:30:09.660 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:30:09 smithi167 ceph-mon[21875]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-19T01:30:09.660 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:30:09 smithi167 ceph-mon[21875]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' 2024-08-19T01:30:09.660 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:30:09 smithi167 ceph-mon[21875]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-19T01:30:09.660 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:30:09 smithi167 ceph-mon[21875]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' 2024-08-19T01:30:09.660 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:30:09 smithi167 ceph-mon[21875]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' 2024-08-19T01:30:09.893 INFO:teuthology.orchestra.run.smithi167.stderr:Inferring config /var/lib/ceph/bad8780e-5dc9-11ef-bccf-c7b262605968/mon.c/config 2024-08-19T01:30:09.939 INFO:journalctl@ceph.osd.7.smithi117.stdout:Aug 19 01:30:09 smithi117 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-7[39667]: 2024-08-19T01:30:09.676+0000 7f66f5919640 -1 osd.7 0 log_to_monitors true 2024-08-19T01:30:10.654 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:30:10 smithi083 ceph-mon[17120]: from='osd.7 [v2:172.21.15.117:6816/104113252,v1:172.21.15.117:6817/104113252]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-08-19T01:30:10.655 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:30:10 smithi083 ceph-mon[17120]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-08-19T01:30:10.702 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:30:10 smithi117 ceph-mon[22016]: from='osd.7 [v2:172.21.15.117:6816/104113252,v1:172.21.15.117:6817/104113252]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-08-19T01:30:10.702 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:30:10 smithi117 ceph-mon[22016]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-08-19T01:30:10.716 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:30:10 smithi167 ceph-mon[21875]: from='osd.7 [v2:172.21.15.117:6816/104113252,v1:172.21.15.117:6817/104113252]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-08-19T01:30:10.716 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:30:10 smithi167 ceph-mon[21875]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-08-19T01:30:11.655 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:30:11 smithi083 ceph-mon[17120]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-08-19T01:30:11.655 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:30:11 smithi083 ceph-mon[17120]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-08-19T01:30:11.655 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:30:11 smithi083 ceph-mon[17120]: osdmap e49: 8 total, 7 up, 8 in 2024-08-19T01:30:11.655 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:30:11 smithi083 ceph-mon[17120]: from='osd.7 [v2:172.21.15.117:6816/104113252,v1:172.21.15.117:6817/104113252]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi117", "root=default"]}]: dispatch 2024-08-19T01:30:11.655 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:30:11 smithi083 ceph-mon[17120]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-08-19T01:30:11.655 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:30:11 smithi083 ceph-mon[17120]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi117", "root=default"]}]: dispatch 2024-08-19T01:30:11.703 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:30:11 smithi117 ceph-mon[22016]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-08-19T01:30:11.703 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:30:11 smithi117 ceph-mon[22016]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-08-19T01:30:11.703 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:30:11 smithi117 ceph-mon[22016]: osdmap e49: 8 total, 7 up, 8 in 2024-08-19T01:30:11.703 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:30:11 smithi117 ceph-mon[22016]: from='osd.7 [v2:172.21.15.117:6816/104113252,v1:172.21.15.117:6817/104113252]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi117", "root=default"]}]: dispatch 2024-08-19T01:30:11.704 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:30:11 smithi117 ceph-mon[22016]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-08-19T01:30:11.704 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:30:11 smithi117 ceph-mon[22016]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi117", "root=default"]}]: dispatch 2024-08-19T01:30:11.716 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:30:11 smithi167 ceph-mon[21875]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-08-19T01:30:11.717 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:30:11 smithi167 ceph-mon[21875]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-08-19T01:30:11.717 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:30:11 smithi167 ceph-mon[21875]: osdmap e49: 8 total, 7 up, 8 in 2024-08-19T01:30:11.717 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:30:11 smithi167 ceph-mon[21875]: from='osd.7 [v2:172.21.15.117:6816/104113252,v1:172.21.15.117:6817/104113252]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi117", "root=default"]}]: dispatch 2024-08-19T01:30:11.717 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:30:11 smithi167 ceph-mon[21875]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-08-19T01:30:11.717 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:30:11 smithi167 ceph-mon[21875]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi117", "root=default"]}]: dispatch 2024-08-19T01:30:12.307 INFO:journalctl@ceph.osd.7.smithi117.stdout:Aug 19 01:30:12 smithi117 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-7[39667]: 2024-08-19T01:30:12.180+0000 7f66f1079640 -1 osd.7 0 waiting for initial osdmap 2024-08-19T01:30:12.308 INFO:journalctl@ceph.osd.7.smithi117.stdout:Aug 19 01:30:12 smithi117 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-7[39667]: 2024-08-19T01:30:12.191+0000 7f66e96ab640 -1 osd.7 50 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-08-19T01:30:12.443 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:30:12 smithi167 ceph-mon[21875]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi117", "root=default"]}]': finished 2024-08-19T01:30:12.443 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:30:12 smithi167 ceph-mon[21875]: osdmap e50: 8 total, 7 up, 8 in 2024-08-19T01:30:12.443 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:30:12 smithi167 ceph-mon[21875]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-08-19T01:30:12.444 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:30:12 smithi167 ceph-mon[21875]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-08-19T01:30:12.444 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:30:12 smithi167 ceph-mon[21875]: from='osd.7 ' entity='osd.7' 2024-08-19T01:30:12.444 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:30:12 smithi167 ceph-mon[21875]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' 2024-08-19T01:30:12.444 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:30:12 smithi167 ceph-mon[21875]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' 2024-08-19T01:30:12.444 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:30:12 smithi167 ceph-mon[21875]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-08-19T01:30:12.444 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:30:12 smithi167 ceph-mon[21875]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-08-19T01:30:12.444 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:30:12 smithi167 ceph-mon[21875]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-08-19T01:30:12.444 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:30:12 smithi167 ceph-mon[21875]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' 2024-08-19T01:30:12.444 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:30:12 smithi167 ceph-mon[21875]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-19T01:30:12.444 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:30:12 smithi167 ceph-mon[21875]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-19T01:30:12.444 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:30:12 smithi167 ceph-mon[21875]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' 2024-08-19T01:30:12.446 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-08-19T01:30:12.473 DEBUG:teuthology.orchestra.run.smithi167:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid bad8780e-5dc9-11ef-bccf-c7b262605968 -- ceph orch daemon add osd smithi167:vg_nvme/lv_2 2024-08-19T01:30:12.655 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:30:12 smithi083 ceph-mon[17120]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi117", "root=default"]}]': finished 2024-08-19T01:30:12.655 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:30:12 smithi083 ceph-mon[17120]: osdmap e50: 8 total, 7 up, 8 in 2024-08-19T01:30:12.655 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:30:12 smithi083 ceph-mon[17120]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-08-19T01:30:12.655 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:30:12 smithi083 ceph-mon[17120]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-08-19T01:30:12.655 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:30:12 smithi083 ceph-mon[17120]: from='osd.7 ' entity='osd.7' 2024-08-19T01:30:12.655 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:30:12 smithi083 ceph-mon[17120]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' 2024-08-19T01:30:12.655 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:30:12 smithi083 ceph-mon[17120]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' 2024-08-19T01:30:12.655 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:30:12 smithi083 ceph-mon[17120]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-08-19T01:30:12.655 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:30:12 smithi083 ceph-mon[17120]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-08-19T01:30:12.656 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:30:12 smithi083 ceph-mon[17120]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-08-19T01:30:12.656 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:30:12 smithi083 ceph-mon[17120]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' 2024-08-19T01:30:12.656 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:30:12 smithi083 ceph-mon[17120]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-19T01:30:12.656 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:30:12 smithi083 ceph-mon[17120]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-19T01:30:12.656 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:30:12 smithi083 ceph-mon[17120]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' 2024-08-19T01:30:12.702 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:30:12 smithi117 ceph-mon[22016]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi117", "root=default"]}]': finished 2024-08-19T01:30:12.703 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:30:12 smithi117 ceph-mon[22016]: osdmap e50: 8 total, 7 up, 8 in 2024-08-19T01:30:12.703 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:30:12 smithi117 ceph-mon[22016]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-08-19T01:30:12.703 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:30:12 smithi117 ceph-mon[22016]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-08-19T01:30:12.703 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:30:12 smithi117 ceph-mon[22016]: from='osd.7 ' entity='osd.7' 2024-08-19T01:30:12.703 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:30:12 smithi117 ceph-mon[22016]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' 2024-08-19T01:30:12.703 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:30:12 smithi117 ceph-mon[22016]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' 2024-08-19T01:30:12.703 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:30:12 smithi117 ceph-mon[22016]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-08-19T01:30:12.703 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:30:12 smithi117 ceph-mon[22016]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-08-19T01:30:12.703 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:30:12 smithi117 ceph-mon[22016]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-08-19T01:30:12.703 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:30:12 smithi117 ceph-mon[22016]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' 2024-08-19T01:30:12.703 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:30:12 smithi117 ceph-mon[22016]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-19T01:30:12.703 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:30:12 smithi117 ceph-mon[22016]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-19T01:30:12.703 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:30:12 smithi117 ceph-mon[22016]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' 2024-08-19T01:30:12.770 INFO:teuthology.orchestra.run.smithi167.stderr:Inferring config /var/lib/ceph/bad8780e-5dc9-11ef-bccf-c7b262605968/mon.c/config 2024-08-19T01:30:13.272 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-19T01:30:13.230+0000 7f7d0e374640 1 -- 172.21.15.167:0/2742400252 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7d08107650 msgr2=0x7f7d08109a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:30:13.272 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-19T01:30:13.230+0000 7f7d0e374640 1 --2- 172.21.15.167:0/2742400252 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7d08107650 0x7f7d08109a40 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f7d00009ca0 tx=0x7f7d0002a8c0 comp rx=0 tx=0).stop 2024-08-19T01:30:13.272 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-19T01:30:13.230+0000 7f7d0e374640 1 -- 172.21.15.167:0/2742400252 shutdown_connections 2024-08-19T01:30:13.272 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-19T01:30:13.230+0000 7f7d0e374640 1 --2- 172.21.15.167:0/2742400252 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7d08107650 0x7f7d08109a40 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:30:13.272 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-19T01:30:13.230+0000 7f7d0e374640 1 --2- 172.21.15.167:0/2742400252 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7d08104d20 0x7f7d08107110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:30:13.272 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-19T01:30:13.230+0000 7f7d0e374640 1 --2- 172.21.15.167:0/2742400252 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7d081023f0 0x7f7d081047e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:30:13.272 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-19T01:30:13.230+0000 7f7d0e374640 1 -- 172.21.15.167:0/2742400252 >> 172.21.15.167:0/2742400252 conn(0x7f7d080fbfb0 msgr2=0x7f7d080fe3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:30:13.272 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-19T01:30:13.230+0000 7f7d0e374640 1 -- 172.21.15.167:0/2742400252 shutdown_connections 2024-08-19T01:30:13.273 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-19T01:30:13.230+0000 7f7d0e374640 1 -- 172.21.15.167:0/2742400252 wait complete. 2024-08-19T01:30:13.273 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-19T01:30:13.231+0000 7f7d0e374640 1 Processor -- start 2024-08-19T01:30:13.273 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-19T01:30:13.231+0000 7f7d0e374640 1 -- start start 2024-08-19T01:30:13.273 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-19T01:30:13.231+0000 7f7d0e374640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7d081023f0 0x7f7d08197c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:30:13.273 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-19T01:30:13.231+0000 7f7d0e374640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7d08104d20 0x7f7d08198170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:30:13.273 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-19T01:30:13.231+0000 7f7d0e374640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7d08107650 0x7f7d0819f1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:30:13.273 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-19T01:30:13.231+0000 7f7d0e374640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f7d0810d5b0 con 0x7f7d08107650 2024-08-19T01:30:13.273 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-19T01:30:13.232+0000 7f7d0e374640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f7d0810d430 con 0x7f7d08104d20 2024-08-19T01:30:13.273 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-19T01:30:13.232+0000 7f7d0e374640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f7d0810d730 con 0x7f7d081023f0 2024-08-19T01:30:13.273 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-19T01:30:13.232+0000 7f7d07fff640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7d081023f0 0x7f7d08197c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:30:13.273 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-19T01:30:13.232+0000 7f7d07fff640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7d081023f0 0x7f7d08197c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:55552/0 (socket says 172.21.15.167:55552) 2024-08-19T01:30:13.273 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-19T01:30:13.232+0000 7f7d07fff640 1 -- 172.21.15.167:0/4165846140 learned_addr learned my addr 172.21.15.167:0/4165846140 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-08-19T01:30:13.273 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-19T01:30:13.232+0000 7f7d077fe640 1 --2- 172.21.15.167:0/4165846140 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7d08104d20 0x7f7d08198170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:30:13.274 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-19T01:30:13.232+0000 7f7d07fff640 1 -- 172.21.15.167:0/4165846140 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7d08104d20 msgr2=0x7f7d08198170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:30:13.274 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-19T01:30:13.232+0000 7f7d07fff640 1 --2- 172.21.15.167:0/4165846140 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7d08104d20 0x7f7d08198170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:30:13.274 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-19T01:30:13.232+0000 7f7d07fff640 1 -- 172.21.15.167:0/4165846140 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7d08107650 msgr2=0x7f7d0819f1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:30:13.274 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-19T01:30:13.232+0000 7f7d07fff640 1 --2- 172.21.15.167:0/4165846140 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7d08107650 0x7f7d0819f1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:30:13.274 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-19T01:30:13.232+0000 7f7d07fff640 1 -- 172.21.15.167:0/4165846140 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7d0819f850 con 0x7f7d081023f0 2024-08-19T01:30:13.274 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-19T01:30:13.232+0000 7f7d07fff640 1 --2- 172.21.15.167:0/4165846140 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7d081023f0 0x7f7d08197c30 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f7cf800d770 tx=0x7f7cf800dc30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:30:13.274 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-19T01:30:13.232+0000 7f7d057fa640 1 -- 172.21.15.167:0/4165846140 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7cf8017070 con 0x7f7d081023f0 2024-08-19T01:30:13.274 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-19T01:30:13.232+0000 7f7d057fa640 1 -- 172.21.15.167:0/4165846140 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7cf8005500 con 0x7f7d081023f0 2024-08-19T01:30:13.274 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-19T01:30:13.232+0000 7f7d0e374640 1 -- 172.21.15.167:0/4165846140 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7d0819fb40 con 0x7f7d081023f0 2024-08-19T01:30:13.274 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-19T01:30:13.232+0000 7f7d057fa640 1 -- 172.21.15.167:0/4165846140 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7cf8004e00 con 0x7f7d081023f0 2024-08-19T01:30:13.274 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-19T01:30:13.233+0000 7f7d0e374640 1 -- 172.21.15.167:0/4165846140 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7d081a0080 con 0x7f7d081023f0 2024-08-19T01:30:13.274 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-19T01:30:13.233+0000 7f7d0e374640 1 -- 172.21.15.167:0/4165846140 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7d080fdaa0 con 0x7f7d081023f0 2024-08-19T01:30:13.274 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-19T01:30:13.235+0000 7f7d057fa640 1 -- 172.21.15.167:0/4165846140 <== mon.1 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f7cf8004fa0 con 0x7f7d081023f0 2024-08-19T01:30:13.275 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-19T01:30:13.235+0000 7f7d057fa640 1 --2- 172.21.15.167:0/4165846140 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f7cf00a2c00 0x7f7cf00a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:30:13.275 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-19T01:30:13.235+0000 7f7d057fa640 1 -- 172.21.15.167:0/4165846140 <== mon.1 v2:172.21.15.167:3300/0 5 ==== osd_map(51..51 src has 1..51) ==== 5430+0+0 (secure 0 0 0) 0x7f7cf80cded0 con 0x7f7d081023f0 2024-08-19T01:30:13.275 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-19T01:30:13.236+0000 7f7d077fe640 1 --2- 172.21.15.167:0/4165846140 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f7cf00a2c00 0x7f7cf00a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:30:13.275 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-19T01:30:13.236+0000 7f7d077fe640 1 --2- 172.21.15.167:0/4165846140 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f7cf00a2c00 0x7f7cf00a50c0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f7cfc0058c0 tx=0x7f7cfc00a2d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:30:13.275 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-19T01:30:13.240+0000 7f7d057fa640 1 -- 172.21.15.167:0/4165846140 <== mon.1 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7f7cf809b970 con 0x7f7d081023f0 2024-08-19T01:30:13.423 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-19T01:30:13.422+0000 7f7d0e374640 1 -- 172.21.15.167:0/4165846140 --> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi167:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7f7d08062360 con 0x7f7cf00a2c00 2024-08-19T01:30:13.655 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:30:13 smithi083 ceph-mon[17120]: purged_snaps scrub starts 2024-08-19T01:30:13.655 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:30:13 smithi083 ceph-mon[17120]: purged_snaps scrub ok 2024-08-19T01:30:13.655 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:30:13 smithi083 ceph-mon[17120]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-08-19T01:30:13.655 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:30:13 smithi083 ceph-mon[17120]: Detected new or changed devices on smithi117 2024-08-19T01:30:13.655 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:30:13 smithi083 ceph-mon[17120]: Adjusting osd_memory_target on smithi117 to 5696M 2024-08-19T01:30:13.655 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:30:13 smithi083 ceph-mon[17120]: osd.7 [v2:172.21.15.117:6816/104113252,v1:172.21.15.117:6817/104113252] boot 2024-08-19T01:30:13.655 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:30:13 smithi083 ceph-mon[17120]: osdmap e51: 8 total, 8 up, 8 in 2024-08-19T01:30:13.655 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:30:13 smithi083 ceph-mon[17120]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-08-19T01:30:13.701 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:30:13 smithi167 ceph-mon[21875]: purged_snaps scrub starts 2024-08-19T01:30:13.702 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:30:13 smithi167 ceph-mon[21875]: purged_snaps scrub ok 2024-08-19T01:30:13.702 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:30:13 smithi167 ceph-mon[21875]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-08-19T01:30:13.702 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:30:13 smithi167 ceph-mon[21875]: Detected new or changed devices on smithi117 2024-08-19T01:30:13.702 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:30:13 smithi167 ceph-mon[21875]: Adjusting osd_memory_target on smithi117 to 5696M 2024-08-19T01:30:13.702 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:30:13 smithi167 ceph-mon[21875]: osd.7 [v2:172.21.15.117:6816/104113252,v1:172.21.15.117:6817/104113252] boot 2024-08-19T01:30:13.702 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:30:13 smithi167 ceph-mon[21875]: osdmap e51: 8 total, 8 up, 8 in 2024-08-19T01:30:13.702 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:30:13 smithi167 ceph-mon[21875]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-08-19T01:30:13.703 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:30:13 smithi117 ceph-mon[22016]: purged_snaps scrub starts 2024-08-19T01:30:13.703 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:30:13 smithi117 ceph-mon[22016]: purged_snaps scrub ok 2024-08-19T01:30:13.703 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:30:13 smithi117 ceph-mon[22016]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-08-19T01:30:13.703 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:30:13 smithi117 ceph-mon[22016]: Detected new or changed devices on smithi117 2024-08-19T01:30:13.703 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:30:13 smithi117 ceph-mon[22016]: Adjusting osd_memory_target on smithi117 to 5696M 2024-08-19T01:30:13.703 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:30:13 smithi117 ceph-mon[22016]: osd.7 [v2:172.21.15.117:6816/104113252,v1:172.21.15.117:6817/104113252] boot 2024-08-19T01:30:13.703 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:30:13 smithi117 ceph-mon[22016]: osdmap e51: 8 total, 8 up, 8 in 2024-08-19T01:30:13.703 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:30:13 smithi117 ceph-mon[22016]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-08-19T01:30:14.680 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:30:14 smithi167 ceph-mon[21875]: osdmap e52: 8 total, 8 up, 8 in 2024-08-19T01:30:14.681 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:30:14 smithi167 ceph-mon[21875]: from='client.24307 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi167:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-08-19T01:30:14.681 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:30:14 smithi167 ceph-mon[21875]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-08-19T01:30:14.681 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:30:14 smithi167 ceph-mon[21875]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-08-19T01:30:14.681 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:30:14 smithi167 ceph-mon[21875]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-19T01:30:14.702 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:30:14 smithi117 ceph-mon[22016]: osdmap e52: 8 total, 8 up, 8 in 2024-08-19T01:30:14.702 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:30:14 smithi117 ceph-mon[22016]: from='client.24307 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi167:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-08-19T01:30:14.703 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:30:14 smithi117 ceph-mon[22016]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-08-19T01:30:14.703 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:30:14 smithi117 ceph-mon[22016]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-08-19T01:30:14.703 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:30:14 smithi117 ceph-mon[22016]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-19T01:30:14.904 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:30:14 smithi083 ceph-mon[17120]: osdmap e52: 8 total, 8 up, 8 in 2024-08-19T01:30:14.905 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:30:14 smithi083 ceph-mon[17120]: from='client.24307 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi167:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-08-19T01:30:14.905 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:30:14 smithi083 ceph-mon[17120]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-08-19T01:30:14.905 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:30:14 smithi083 ceph-mon[17120]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-08-19T01:30:14.905 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:30:14 smithi083 ceph-mon[17120]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-19T01:30:15.702 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:30:15 smithi117 ceph-mon[22016]: pgmap v167: 1 pgs: 1 remapped; 577 KiB data, 568 MiB used, 715 GiB / 715 GiB avail 2024-08-19T01:30:15.703 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:30:15 smithi117 ceph-mon[22016]: osdmap e53: 8 total, 8 up, 8 in 2024-08-19T01:30:15.716 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:30:15 smithi167 ceph-mon[21875]: pgmap v167: 1 pgs: 1 remapped; 577 KiB data, 568 MiB used, 715 GiB / 715 GiB avail 2024-08-19T01:30:15.716 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:30:15 smithi167 ceph-mon[21875]: osdmap e53: 8 total, 8 up, 8 in 2024-08-19T01:30:15.806 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:30:15 smithi083 ceph-mon[17120]: pgmap v167: 1 pgs: 1 remapped; 577 KiB data, 568 MiB used, 715 GiB / 715 GiB avail 2024-08-19T01:30:15.806 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:30:15 smithi083 ceph-mon[17120]: osdmap e53: 8 total, 8 up, 8 in 2024-08-19T01:30:16.703 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:30:16 smithi117 ceph-mon[22016]: osdmap e54: 8 total, 8 up, 8 in 2024-08-19T01:30:16.703 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:30:16 smithi117 ceph-mon[22016]: from='client.? 172.21.15.167:0/1508943618' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "76d5317d-4845-4138-b210-e7bd2d80472a"}]: dispatch 2024-08-19T01:30:16.703 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:30:16 smithi117 ceph-mon[22016]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "76d5317d-4845-4138-b210-e7bd2d80472a"}]: dispatch 2024-08-19T01:30:16.703 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:30:16 smithi117 ceph-mon[22016]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "76d5317d-4845-4138-b210-e7bd2d80472a"}]': finished 2024-08-19T01:30:16.703 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:30:16 smithi117 ceph-mon[22016]: osdmap e55: 9 total, 8 up, 9 in 2024-08-19T01:30:16.703 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:30:16 smithi117 ceph-mon[22016]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-08-19T01:30:16.703 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:30:16 smithi117 ceph-mon[22016]: from='client.? 172.21.15.167:0/1511926946' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-08-19T01:30:16.716 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:30:16 smithi167 ceph-mon[21875]: osdmap e54: 8 total, 8 up, 8 in 2024-08-19T01:30:16.716 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:30:16 smithi167 ceph-mon[21875]: from='client.? 172.21.15.167:0/1508943618' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "76d5317d-4845-4138-b210-e7bd2d80472a"}]: dispatch 2024-08-19T01:30:16.716 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:30:16 smithi167 ceph-mon[21875]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "76d5317d-4845-4138-b210-e7bd2d80472a"}]: dispatch 2024-08-19T01:30:16.716 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:30:16 smithi167 ceph-mon[21875]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "76d5317d-4845-4138-b210-e7bd2d80472a"}]': finished 2024-08-19T01:30:16.716 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:30:16 smithi167 ceph-mon[21875]: osdmap e55: 9 total, 8 up, 9 in 2024-08-19T01:30:16.716 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:30:16 smithi167 ceph-mon[21875]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-08-19T01:30:16.716 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:30:16 smithi167 ceph-mon[21875]: from='client.? 172.21.15.167:0/1511926946' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-08-19T01:30:16.832 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:30:16 smithi083 ceph-mon[17120]: osdmap e54: 8 total, 8 up, 8 in 2024-08-19T01:30:16.832 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:30:16 smithi083 ceph-mon[17120]: from='client.? 172.21.15.167:0/1508943618' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "76d5317d-4845-4138-b210-e7bd2d80472a"}]: dispatch 2024-08-19T01:30:16.832 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:30:16 smithi083 ceph-mon[17120]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "76d5317d-4845-4138-b210-e7bd2d80472a"}]: dispatch 2024-08-19T01:30:16.832 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:30:16 smithi083 ceph-mon[17120]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "76d5317d-4845-4138-b210-e7bd2d80472a"}]': finished 2024-08-19T01:30:16.832 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:30:16 smithi083 ceph-mon[17120]: osdmap e55: 9 total, 8 up, 9 in 2024-08-19T01:30:16.832 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:30:16 smithi083 ceph-mon[17120]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-08-19T01:30:16.832 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:30:16 smithi083 ceph-mon[17120]: from='client.? 172.21.15.167:0/1511926946' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-08-19T01:30:17.702 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:30:17 smithi117 ceph-mon[22016]: pgmap v171: 1 pgs: 1 remapped; 577 KiB data, 568 MiB used, 715 GiB / 715 GiB avail 2024-08-19T01:30:17.716 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:30:17 smithi167 ceph-mon[21875]: pgmap v171: 1 pgs: 1 remapped; 577 KiB data, 568 MiB used, 715 GiB / 715 GiB avail 2024-08-19T01:30:17.904 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:30:17 smithi083 ceph-mon[17120]: pgmap v171: 1 pgs: 1 remapped; 577 KiB data, 568 MiB used, 715 GiB / 715 GiB avail 2024-08-19T01:30:19.702 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:30:19 smithi117 ceph-mon[22016]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 237 MiB used, 715 GiB / 715 GiB avail 2024-08-19T01:30:19.716 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:30:19 smithi167 ceph-mon[21875]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 237 MiB used, 715 GiB / 715 GiB avail 2024-08-19T01:30:19.905 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:30:19 smithi083 ceph-mon[17120]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 237 MiB used, 715 GiB / 715 GiB avail 2024-08-19T01:30:21.466 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:30:21 smithi167 ceph-mon[21875]: pgmap v173: 1 pgs: 1 active+clean; 577 KiB data, 237 MiB used, 715 GiB / 715 GiB avail 2024-08-19T01:30:21.702 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:30:21 smithi117 ceph-mon[22016]: pgmap v173: 1 pgs: 1 active+clean; 577 KiB data, 237 MiB used, 715 GiB / 715 GiB avail 2024-08-19T01:30:21.905 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:30:21 smithi083 ceph-mon[17120]: pgmap v173: 1 pgs: 1 active+clean; 577 KiB data, 237 MiB used, 715 GiB / 715 GiB avail 2024-08-19T01:30:23.702 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:30:23 smithi117 ceph-mon[22016]: pgmap v174: 1 pgs: 1 active+clean; 577 KiB data, 237 MiB used, 715 GiB / 715 GiB avail 2024-08-19T01:30:23.703 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:30:23 smithi117 ceph-mon[22016]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-08-19T01:30:23.703 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:30:23 smithi117 ceph-mon[22016]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-19T01:30:23.716 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:30:23 smithi167 ceph-mon[21875]: pgmap v174: 1 pgs: 1 active+clean; 577 KiB data, 237 MiB used, 715 GiB / 715 GiB avail 2024-08-19T01:30:23.716 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:30:23 smithi167 ceph-mon[21875]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-08-19T01:30:23.716 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:30:23 smithi167 ceph-mon[21875]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-19T01:30:23.905 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:30:23 smithi083 ceph-mon[17120]: pgmap v174: 1 pgs: 1 active+clean; 577 KiB data, 237 MiB used, 715 GiB / 715 GiB avail 2024-08-19T01:30:23.905 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:30:23 smithi083 ceph-mon[17120]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-08-19T01:30:23.905 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:30:23 smithi083 ceph-mon[17120]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-19T01:30:24.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:30:24 smithi167 ceph-mon[21875]: Deploying daemon osd.8 on smithi167 2024-08-19T01:30:24.702 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:30:24 smithi117 ceph-mon[22016]: Deploying daemon osd.8 on smithi167 2024-08-19T01:30:24.904 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:30:24 smithi083 ceph-mon[17120]: Deploying daemon osd.8 on smithi167 2024-08-19T01:30:25.702 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:30:25 smithi117 ceph-mon[22016]: pgmap v175: 1 pgs: 1 active+clean; 577 KiB data, 237 MiB used, 715 GiB / 715 GiB avail 2024-08-19T01:30:25.716 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:30:25 smithi167 ceph-mon[21875]: pgmap v175: 1 pgs: 1 active+clean; 577 KiB data, 237 MiB used, 715 GiB / 715 GiB avail 2024-08-19T01:30:25.904 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:30:25 smithi083 ceph-mon[17120]: pgmap v175: 1 pgs: 1 active+clean; 577 KiB data, 237 MiB used, 715 GiB / 715 GiB avail 2024-08-19T01:30:27.701 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:30:27 smithi167 ceph-mon[21875]: pgmap v176: 1 pgs: 1 active+clean; 577 KiB data, 237 MiB used, 715 GiB / 715 GiB avail 2024-08-19T01:30:27.702 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:30:27 smithi117 ceph-mon[22016]: pgmap v176: 1 pgs: 1 active+clean; 577 KiB data, 237 MiB used, 715 GiB / 715 GiB avail 2024-08-19T01:30:27.726 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:30:27 smithi083 ceph-mon[17120]: pgmap v176: 1 pgs: 1 active+clean; 577 KiB data, 237 MiB used, 715 GiB / 715 GiB avail 2024-08-19T01:30:28.654 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:30:28 smithi083 ceph-mon[17120]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-19T01:30:28.655 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:30:28 smithi083 ceph-mon[17120]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' 2024-08-19T01:30:28.655 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:30:28 smithi083 ceph-mon[17120]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' 2024-08-19T01:30:28.702 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:30:28 smithi117 ceph-mon[22016]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-19T01:30:28.703 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:30:28 smithi117 ceph-mon[22016]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' 2024-08-19T01:30:28.703 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:30:28 smithi117 ceph-mon[22016]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' 2024-08-19T01:30:28.716 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:30:28 smithi167 ceph-mon[21875]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-19T01:30:28.716 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:30:28 smithi167 ceph-mon[21875]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' 2024-08-19T01:30:28.717 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:30:28 smithi167 ceph-mon[21875]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' 2024-08-19T01:30:29.702 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:30:29 smithi117 ceph-mon[22016]: pgmap v177: 1 pgs: 1 active+clean; 577 KiB data, 237 MiB used, 715 GiB / 715 GiB avail 2024-08-19T01:30:29.716 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:30:29 smithi167 ceph-mon[21875]: pgmap v177: 1 pgs: 1 active+clean; 577 KiB data, 237 MiB used, 715 GiB / 715 GiB avail 2024-08-19T01:30:29.905 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:30:29 smithi083 ceph-mon[17120]: pgmap v177: 1 pgs: 1 active+clean; 577 KiB data, 237 MiB used, 715 GiB / 715 GiB avail 2024-08-19T01:30:30.859 INFO:teuthology.orchestra.run.smithi167.stdout:Created osd(s) 8 on host 'smithi167' 2024-08-19T01:30:30.859 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-19T01:30:30.857+0000 7f7d057fa640 1 -- 172.21.15.167:0/4165846140 <== mgr.14150 v2:172.21.15.83:6800/683740916 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f7d08062360 con 0x7f7cf00a2c00 2024-08-19T01:30:30.861 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-19T01:30:30.859+0000 7f7d0e374640 1 -- 172.21.15.167:0/4165846140 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f7cf00a2c00 msgr2=0x7f7cf00a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:30:30.861 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-19T01:30:30.859+0000 7f7d0e374640 1 --2- 172.21.15.167:0/4165846140 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f7cf00a2c00 0x7f7cf00a50c0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f7cfc0058c0 tx=0x7f7cfc00a2d0 comp rx=0 tx=0).stop 2024-08-19T01:30:30.861 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-19T01:30:30.859+0000 7f7d0e374640 1 -- 172.21.15.167:0/4165846140 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7d081023f0 msgr2=0x7f7d08197c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:30:30.861 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-19T01:30:30.859+0000 7f7d0e374640 1 --2- 172.21.15.167:0/4165846140 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7d081023f0 0x7f7d08197c30 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f7cf800d770 tx=0x7f7cf800dc30 comp rx=0 tx=0).stop 2024-08-19T01:30:30.861 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-19T01:30:30.859+0000 7f7d0e374640 1 -- 172.21.15.167:0/4165846140 shutdown_connections 2024-08-19T01:30:30.861 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-19T01:30:30.859+0000 7f7d0e374640 1 --2- 172.21.15.167:0/4165846140 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f7cf00a2c00 0x7f7cf00a50c0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:30:30.861 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-19T01:30:30.859+0000 7f7d0e374640 1 --2- 172.21.15.167:0/4165846140 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7d08107650 0x7f7d0819f1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:30:30.861 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-19T01:30:30.859+0000 7f7d0e374640 1 --2- 172.21.15.167:0/4165846140 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7d08104d20 0x7f7d08198170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:30:30.861 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-19T01:30:30.859+0000 7f7d0e374640 1 --2- 172.21.15.167:0/4165846140 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7d081023f0 0x7f7d08197c30 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:30:30.861 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-19T01:30:30.859+0000 7f7d0e374640 1 -- 172.21.15.167:0/4165846140 >> 172.21.15.167:0/4165846140 conn(0x7f7d080fbfb0 msgr2=0x7f7d08102c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:30:30.861 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-19T01:30:30.859+0000 7f7d0e374640 1 -- 172.21.15.167:0/4165846140 shutdown_connections 2024-08-19T01:30:30.862 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-19T01:30:30.859+0000 7f7d0e374640 1 -- 172.21.15.167:0/4165846140 wait complete. 2024-08-19T01:30:31.510 DEBUG:teuthology.orchestra.run.smithi167:osd.8> sudo journalctl -f -n 0 -u ceph-bad8780e-5dc9-11ef-bccf-c7b262605968@osd.8.service 2024-08-19T01:30:31.512 INFO:tasks.cephadm:Deploying osd.9 on smithi083 with /dev/vg_nvme/lv_1... 2024-08-19T01:30:31.513 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833 ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid bad8780e-5dc9-11ef-bccf-c7b262605968 -- lvm zap /dev/vg_nvme/lv_1 2024-08-19T01:30:31.703 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:30:31 smithi117 ceph-mon[22016]: pgmap v178: 1 pgs: 1 active+clean; 577 KiB data, 237 MiB used, 715 GiB / 715 GiB avail 2024-08-19T01:30:31.703 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:30:31 smithi117 ceph-mon[22016]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' 2024-08-19T01:30:31.703 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:30:31 smithi117 ceph-mon[22016]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' 2024-08-19T01:30:31.703 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:30:31 smithi117 ceph-mon[22016]: from='osd.8 [v2:172.21.15.167:6816/2196898857,v1:172.21.15.167:6817/2196898857]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-08-19T01:30:31.703 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:30:31 smithi117 ceph-mon[22016]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-08-19T01:30:31.716 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:30:31 smithi167 ceph-mon[21875]: pgmap v178: 1 pgs: 1 active+clean; 577 KiB data, 237 MiB used, 715 GiB / 715 GiB avail 2024-08-19T01:30:31.716 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:30:31 smithi167 ceph-mon[21875]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' 2024-08-19T01:30:31.716 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:30:31 smithi167 ceph-mon[21875]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' 2024-08-19T01:30:31.716 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:30:31 smithi167 ceph-mon[21875]: from='osd.8 [v2:172.21.15.167:6816/2196898857,v1:172.21.15.167:6817/2196898857]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-08-19T01:30:31.716 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:30:31 smithi167 ceph-mon[21875]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-08-19T01:30:31.905 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:30:31 smithi083 ceph-mon[17120]: pgmap v178: 1 pgs: 1 active+clean; 577 KiB data, 237 MiB used, 715 GiB / 715 GiB avail 2024-08-19T01:30:31.905 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:30:31 smithi083 ceph-mon[17120]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' 2024-08-19T01:30:31.905 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:30:31 smithi083 ceph-mon[17120]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' 2024-08-19T01:30:31.905 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:30:31 smithi083 ceph-mon[17120]: from='osd.8 [v2:172.21.15.167:6816/2196898857,v1:172.21.15.167:6817/2196898857]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-08-19T01:30:31.905 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:30:31 smithi083 ceph-mon[17120]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-08-19T01:30:31.906 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/bad8780e-5dc9-11ef-bccf-c7b262605968/mon.a/config 2024-08-19T01:30:32.882 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:30:32 smithi083 ceph-mon[17120]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' 2024-08-19T01:30:32.882 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:30:32 smithi083 ceph-mon[17120]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' 2024-08-19T01:30:32.882 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:30:32 smithi083 ceph-mon[17120]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-08-19T01:30:32.882 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:30:32 smithi083 ceph-mon[17120]: osdmap e56: 9 total, 8 up, 9 in 2024-08-19T01:30:32.882 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:30:32 smithi083 ceph-mon[17120]: from='osd.8 [v2:172.21.15.167:6816/2196898857,v1:172.21.15.167:6817/2196898857]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi167", "root=default"]}]: dispatch 2024-08-19T01:30:32.883 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:30:32 smithi083 ceph-mon[17120]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-08-19T01:30:32.883 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:30:32 smithi083 ceph-mon[17120]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi167", "root=default"]}]: dispatch 2024-08-19T01:30:32.883 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:30:32 smithi083 ceph-mon[17120]: pgmap v180: 1 pgs: 1 active+clean; 577 KiB data, 237 MiB used, 715 GiB / 715 GiB avail 2024-08-19T01:30:32.952 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:30:32 smithi117 ceph-mon[22016]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' 2024-08-19T01:30:32.953 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:30:32 smithi117 ceph-mon[22016]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' 2024-08-19T01:30:32.953 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:30:32 smithi117 ceph-mon[22016]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-08-19T01:30:32.953 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:30:32 smithi117 ceph-mon[22016]: osdmap e56: 9 total, 8 up, 9 in 2024-08-19T01:30:32.953 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:30:32 smithi117 ceph-mon[22016]: from='osd.8 [v2:172.21.15.167:6816/2196898857,v1:172.21.15.167:6817/2196898857]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi167", "root=default"]}]: dispatch 2024-08-19T01:30:32.953 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:30:32 smithi117 ceph-mon[22016]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-08-19T01:30:32.953 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:30:32 smithi117 ceph-mon[22016]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi167", "root=default"]}]: dispatch 2024-08-19T01:30:32.953 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:30:32 smithi117 ceph-mon[22016]: pgmap v180: 1 pgs: 1 active+clean; 577 KiB data, 237 MiB used, 715 GiB / 715 GiB avail 2024-08-19T01:30:32.966 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:30:32 smithi167 ceph-mon[21875]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' 2024-08-19T01:30:32.966 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:30:32 smithi167 ceph-mon[21875]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' 2024-08-19T01:30:32.966 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:30:32 smithi167 ceph-mon[21875]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-08-19T01:30:32.966 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:30:32 smithi167 ceph-mon[21875]: osdmap e56: 9 total, 8 up, 9 in 2024-08-19T01:30:32.966 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:30:32 smithi167 ceph-mon[21875]: from='osd.8 [v2:172.21.15.167:6816/2196898857,v1:172.21.15.167:6817/2196898857]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi167", "root=default"]}]: dispatch 2024-08-19T01:30:32.966 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:30:32 smithi167 ceph-mon[21875]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-08-19T01:30:32.967 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:30:32 smithi167 ceph-mon[21875]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi167", "root=default"]}]: dispatch 2024-08-19T01:30:32.967 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:30:32 smithi167 ceph-mon[21875]: pgmap v180: 1 pgs: 1 active+clean; 577 KiB data, 237 MiB used, 715 GiB / 715 GiB avail 2024-08-19T01:30:33.967 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:30:33 smithi167 ceph-mon[21875]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi167", "root=default"]}]': finished 2024-08-19T01:30:33.967 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:30:33 smithi167 ceph-mon[21875]: osdmap e57: 9 total, 8 up, 9 in 2024-08-19T01:30:33.967 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:30:33 smithi167 ceph-mon[21875]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-08-19T01:30:33.967 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:30:33 smithi167 ceph-mon[21875]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-08-19T01:30:33.967 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:30:33 smithi167 ceph-mon[21875]: from='osd.8 ' entity='osd.8' 2024-08-19T01:30:33.967 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:30:33 smithi167 ceph-mon[21875]: osd.8 [v2:172.21.15.167:6816/2196898857,v1:172.21.15.167:6817/2196898857] boot 2024-08-19T01:30:33.967 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:30:33 smithi167 ceph-mon[21875]: osdmap e58: 9 total, 9 up, 9 in 2024-08-19T01:30:33.967 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:30:33 smithi167 ceph-mon[21875]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-08-19T01:30:33.967 INFO:journalctl@ceph.osd.8.smithi167.stdout:Aug 19 01:30:33 smithi167 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-8[39928]: 2024-08-19T01:30:33.674+0000 7f1806223640 -1 osd.8 0 waiting for initial osdmap 2024-08-19T01:30:33.968 INFO:journalctl@ceph.osd.8.smithi167.stdout:Aug 19 01:30:33 smithi167 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-8[39928]: 2024-08-19T01:30:33.692+0000 7f17fd01d640 -1 osd.8 57 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-08-19T01:30:34.155 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:30:33 smithi083 ceph-mon[17120]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi167", "root=default"]}]': finished 2024-08-19T01:30:34.155 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:30:33 smithi083 ceph-mon[17120]: osdmap e57: 9 total, 8 up, 9 in 2024-08-19T01:30:34.155 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:30:33 smithi083 ceph-mon[17120]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-08-19T01:30:34.155 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:30:33 smithi083 ceph-mon[17120]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-08-19T01:30:34.156 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:30:33 smithi083 ceph-mon[17120]: from='osd.8 ' entity='osd.8' 2024-08-19T01:30:34.156 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:30:33 smithi083 ceph-mon[17120]: osd.8 [v2:172.21.15.167:6816/2196898857,v1:172.21.15.167:6817/2196898857] boot 2024-08-19T01:30:34.156 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:30:33 smithi083 ceph-mon[17120]: osdmap e58: 9 total, 9 up, 9 in 2024-08-19T01:30:34.156 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:30:33 smithi083 ceph-mon[17120]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-08-19T01:30:34.203 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:30:33 smithi117 ceph-mon[22016]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi167", "root=default"]}]': finished 2024-08-19T01:30:34.203 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:30:33 smithi117 ceph-mon[22016]: osdmap e57: 9 total, 8 up, 9 in 2024-08-19T01:30:34.203 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:30:33 smithi117 ceph-mon[22016]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-08-19T01:30:34.203 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:30:33 smithi117 ceph-mon[22016]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-08-19T01:30:34.203 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:30:33 smithi117 ceph-mon[22016]: from='osd.8 ' entity='osd.8' 2024-08-19T01:30:34.203 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:30:33 smithi117 ceph-mon[22016]: osd.8 [v2:172.21.15.167:6816/2196898857,v1:172.21.15.167:6817/2196898857] boot 2024-08-19T01:30:34.203 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:30:33 smithi117 ceph-mon[22016]: osdmap e58: 9 total, 9 up, 9 in 2024-08-19T01:30:34.203 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:30:33 smithi117 ceph-mon[22016]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-08-19T01:30:34.966 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:30:34 smithi167 ceph-mon[21875]: purged_snaps scrub starts 2024-08-19T01:30:34.967 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:30:34 smithi167 ceph-mon[21875]: purged_snaps scrub ok 2024-08-19T01:30:34.967 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:30:34 smithi167 ceph-mon[21875]: pgmap v183: 1 pgs: 1 active+clean; 577 KiB data, 664 MiB used, 804 GiB / 805 GiB avail 2024-08-19T01:30:34.967 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:30:34 smithi167 ceph-mon[21875]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' 2024-08-19T01:30:34.967 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:30:34 smithi167 ceph-mon[21875]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' 2024-08-19T01:30:34.967 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:30:34 smithi167 ceph-mon[21875]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-08-19T01:30:34.968 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:30:34 smithi167 ceph-mon[21875]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-08-19T01:30:34.968 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:30:34 smithi167 ceph-mon[21875]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-08-19T01:30:34.968 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:30:34 smithi167 ceph-mon[21875]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' 2024-08-19T01:30:34.968 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:30:34 smithi167 ceph-mon[21875]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-19T01:30:34.968 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:30:34 smithi167 ceph-mon[21875]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-19T01:30:34.968 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:30:34 smithi167 ceph-mon[21875]: osdmap e59: 9 total, 9 up, 9 in 2024-08-19T01:30:34.969 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:30:34 smithi167 ceph-mon[21875]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' 2024-08-19T01:30:35.152 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:30:34 smithi083 ceph-mon[17120]: purged_snaps scrub starts 2024-08-19T01:30:35.152 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:30:34 smithi083 ceph-mon[17120]: purged_snaps scrub ok 2024-08-19T01:30:35.152 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:30:34 smithi083 ceph-mon[17120]: pgmap v183: 1 pgs: 1 active+clean; 577 KiB data, 664 MiB used, 804 GiB / 805 GiB avail 2024-08-19T01:30:35.152 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:30:34 smithi083 ceph-mon[17120]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' 2024-08-19T01:30:35.152 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:30:34 smithi083 ceph-mon[17120]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' 2024-08-19T01:30:35.153 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:30:34 smithi083 ceph-mon[17120]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-08-19T01:30:35.153 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:30:34 smithi083 ceph-mon[17120]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-08-19T01:30:35.153 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:30:34 smithi083 ceph-mon[17120]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-08-19T01:30:35.153 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:30:34 smithi083 ceph-mon[17120]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' 2024-08-19T01:30:35.153 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:30:34 smithi083 ceph-mon[17120]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-19T01:30:35.153 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:30:34 smithi083 ceph-mon[17120]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-19T01:30:35.153 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:30:34 smithi083 ceph-mon[17120]: osdmap e59: 9 total, 9 up, 9 in 2024-08-19T01:30:35.153 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:30:34 smithi083 ceph-mon[17120]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' 2024-08-19T01:30:35.158 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-08-19T01:30:35.189 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid bad8780e-5dc9-11ef-bccf-c7b262605968 -- ceph orch daemon add osd smithi083:vg_nvme/lv_1 2024-08-19T01:30:35.202 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:30:34 smithi117 ceph-mon[22016]: purged_snaps scrub starts 2024-08-19T01:30:35.203 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:30:34 smithi117 ceph-mon[22016]: purged_snaps scrub ok 2024-08-19T01:30:35.203 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:30:34 smithi117 ceph-mon[22016]: pgmap v183: 1 pgs: 1 active+clean; 577 KiB data, 664 MiB used, 804 GiB / 805 GiB avail 2024-08-19T01:30:35.203 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:30:34 smithi117 ceph-mon[22016]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' 2024-08-19T01:30:35.203 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:30:34 smithi117 ceph-mon[22016]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' 2024-08-19T01:30:35.203 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:30:34 smithi117 ceph-mon[22016]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-08-19T01:30:35.203 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:30:34 smithi117 ceph-mon[22016]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-08-19T01:30:35.203 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:30:34 smithi117 ceph-mon[22016]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-08-19T01:30:35.203 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:30:34 smithi117 ceph-mon[22016]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' 2024-08-19T01:30:35.203 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:30:34 smithi117 ceph-mon[22016]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-19T01:30:35.203 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:30:34 smithi117 ceph-mon[22016]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-19T01:30:35.203 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:30:34 smithi117 ceph-mon[22016]: osdmap e59: 9 total, 9 up, 9 in 2024-08-19T01:30:35.203 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:30:34 smithi117 ceph-mon[22016]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' 2024-08-19T01:30:35.561 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/bad8780e-5dc9-11ef-bccf-c7b262605968/mon.a/config 2024-08-19T01:30:36.007 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:30:35 smithi083 ceph-mon[17120]: Detected new or changed devices on smithi167 2024-08-19T01:30:36.008 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:30:35 smithi083 ceph-mon[17120]: Adjusting osd_memory_target on smithi167 to 5696M 2024-08-19T01:30:36.008 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:30:35 smithi083 ceph-mon[17120]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-19T01:30:36.008 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:30:35 smithi083 ceph-mon[17120]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-19T01:30:36.008 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:30:35 smithi083 ceph-mon[17120]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-19T01:30:36.008 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:30:35 smithi083 ceph-mon[17120]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' 2024-08-19T01:30:36.008 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:30:35 smithi083 ceph-mon[17120]: osdmap e60: 9 total, 9 up, 9 in 2024-08-19T01:30:36.131 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:30:36.117+0000 7fa959800640 1 -- 172.21.15.83:0/1095266835 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa954103b30 msgr2=0x7fa954103f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:30:36.131 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:30:36.117+0000 7fa959800640 1 --2- 172.21.15.83:0/1095266835 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa954103b30 0x7fa954103f30 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fa944009900 tx=0x7fa94402a410 comp rx=0 tx=0).stop 2024-08-19T01:30:36.131 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:30:36.117+0000 7fa959800640 1 -- 172.21.15.83:0/1095266835 shutdown_connections 2024-08-19T01:30:36.131 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:30:36.117+0000 7fa959800640 1 --2- 172.21.15.83:0/1095266835 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa9541056f0 0x7fa95410bf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:30:36.131 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:30:36.117+0000 7fa959800640 1 --2- 172.21.15.83:0/1095266835 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa954104d30 0x7fa9541051b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:30:36.131 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:30:36.117+0000 7fa959800640 1 --2- 172.21.15.83:0/1095266835 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa954103b30 0x7fa954103f30 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:30:36.131 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:30:36.117+0000 7fa959800640 1 -- 172.21.15.83:0/1095266835 >> 172.21.15.83:0/1095266835 conn(0x7fa9540ff2e0 msgr2=0x7fa954101700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:30:36.131 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:30:36.118+0000 7fa959800640 1 -- 172.21.15.83:0/1095266835 shutdown_connections 2024-08-19T01:30:36.131 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:30:36.118+0000 7fa959800640 1 -- 172.21.15.83:0/1095266835 wait complete. 2024-08-19T01:30:36.132 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:30:36.118+0000 7fa959800640 1 Processor -- start 2024-08-19T01:30:36.132 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:30:36.119+0000 7fa959800640 1 -- start start 2024-08-19T01:30:36.132 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:30:36.119+0000 7fa959800640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa954103b30 0x7fa954079cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:30:36.132 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:30:36.119+0000 7fa959800640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa954104d30 0x7fa95407a210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:30:36.132 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:30:36.119+0000 7fa959800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa9541056f0 0x7fa954076830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:30:36.132 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:30:36.119+0000 7fa959800640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fa95410f800 con 0x7fa9541056f0 2024-08-19T01:30:36.132 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:30:36.119+0000 7fa959800640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fa95410f680 con 0x7fa954103b30 2024-08-19T01:30:36.132 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:30:36.119+0000 7fa959800640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fa95410f980 con 0x7fa954104d30 2024-08-19T01:30:36.132 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:30:36.119+0000 7fa952000640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa954104d30 0x7fa95407a210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:30:36.132 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:30:36.119+0000 7fa953400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa9541056f0 0x7fa954076830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:30:36.132 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:30:36.119+0000 7fa952000640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa954104d30 0x7fa95407a210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.83:43450/0 (socket says 172.21.15.83:43450) 2024-08-19T01:30:36.133 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:30:36.119+0000 7fa952000640 1 -- 172.21.15.83:0/1037005532 learned_addr learned my addr 172.21.15.83:0/1037005532 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-08-19T01:30:36.133 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:30:36.119+0000 7fa952a00640 1 --2- 172.21.15.83:0/1037005532 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa954103b30 0x7fa954079cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:30:36.133 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:30:36.119+0000 7fa953400640 1 -- 172.21.15.83:0/1037005532 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa954104d30 msgr2=0x7fa95407a210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:30:36.133 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:30:36.119+0000 7fa953400640 1 --2- 172.21.15.83:0/1037005532 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa954104d30 0x7fa95407a210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:30:36.133 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:30:36.119+0000 7fa953400640 1 -- 172.21.15.83:0/1037005532 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa954103b30 msgr2=0x7fa954079cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:30:36.133 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:30:36.119+0000 7fa953400640 1 --2- 172.21.15.83:0/1037005532 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa954103b30 0x7fa954079cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:30:36.133 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:30:36.119+0000 7fa953400640 1 -- 172.21.15.83:0/1037005532 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa9540770f0 con 0x7fa9541056f0 2024-08-19T01:30:36.133 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:30:36.119+0000 7fa952000640 1 --2- 172.21.15.83:0/1037005532 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa954104d30 0x7fa95407a210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-19T01:30:36.133 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:30:36.119+0000 7fa952a00640 1 --2- 172.21.15.83:0/1037005532 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa954103b30 0x7fa954079cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-19T01:30:36.133 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:30:36.120+0000 7fa953400640 1 --2- 172.21.15.83:0/1037005532 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa9541056f0 0x7fa954076830 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fa94c00ec50 tx=0x7fa94c00c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:30:36.133 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:30:36.120+0000 7fa94b400640 1 -- 172.21.15.83:0/1037005532 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa94c003b70 con 0x7fa9541056f0 2024-08-19T01:30:36.133 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:30:36.120+0000 7fa959800640 1 -- 172.21.15.83:0/1037005532 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa9541a62b0 con 0x7fa9541056f0 2024-08-19T01:30:36.134 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:30:36.120+0000 7fa94b400640 1 -- 172.21.15.83:0/1037005532 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa94c003d10 con 0x7fa9541056f0 2024-08-19T01:30:36.134 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:30:36.120+0000 7fa94b400640 1 -- 172.21.15.83:0/1037005532 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa94c010890 con 0x7fa9541056f0 2024-08-19T01:30:36.134 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:30:36.120+0000 7fa959800640 1 -- 172.21.15.83:0/1037005532 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa9541a66a0 con 0x7fa9541056f0 2024-08-19T01:30:36.134 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:30:36.121+0000 7fa959800640 1 -- 172.21.15.83:0/1037005532 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa954103f30 con 0x7fa9541056f0 2024-08-19T01:30:36.134 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:30:36.123+0000 7fa94b400640 1 -- 172.21.15.83:0/1037005532 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7fa94c003fc0 con 0x7fa9541056f0 2024-08-19T01:30:36.134 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:30:36.124+0000 7fa94b400640 1 --2- 172.21.15.83:0/1037005532 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7fa9380a2b80 0x7fa9380a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:30:36.134 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:30:36.124+0000 7fa94b400640 1 -- 172.21.15.83:0/1037005532 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(60..60 src has 1..60) ==== 5858+0+0 (secure 0 0 0) 0x7fa94c0ceda0 con 0x7fa9541056f0 2024-08-19T01:30:36.134 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:30:36.125+0000 7fa952a00640 1 --2- 172.21.15.83:0/1037005532 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7fa9380a2b80 0x7fa9380a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:30:36.134 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:30:36.125+0000 7fa952a00640 1 --2- 172.21.15.83:0/1037005532 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7fa9380a2b80 0x7fa9380a5040 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fa94402aa00 tx=0x7fa944000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:30:36.134 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:30:36.129+0000 7fa94b400640 1 -- 172.21.15.83:0/1037005532 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7fa94c09c5f0 con 0x7fa9541056f0 2024-08-19T01:30:36.202 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:30:35 smithi117 ceph-mon[22016]: Detected new or changed devices on smithi167 2024-08-19T01:30:36.203 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:30:35 smithi117 ceph-mon[22016]: Adjusting osd_memory_target on smithi167 to 5696M 2024-08-19T01:30:36.203 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:30:35 smithi117 ceph-mon[22016]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-19T01:30:36.203 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:30:35 smithi117 ceph-mon[22016]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-19T01:30:36.203 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:30:35 smithi117 ceph-mon[22016]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-19T01:30:36.203 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:30:35 smithi117 ceph-mon[22016]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' 2024-08-19T01:30:36.203 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:30:35 smithi117 ceph-mon[22016]: osdmap e60: 9 total, 9 up, 9 in 2024-08-19T01:30:36.216 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:30:35 smithi167 ceph-mon[21875]: Detected new or changed devices on smithi167 2024-08-19T01:30:36.216 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:30:35 smithi167 ceph-mon[21875]: Adjusting osd_memory_target on smithi167 to 5696M 2024-08-19T01:30:36.216 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:30:35 smithi167 ceph-mon[21875]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-19T01:30:36.216 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:30:35 smithi167 ceph-mon[21875]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-19T01:30:36.216 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:30:35 smithi167 ceph-mon[21875]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-19T01:30:36.216 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:30:35 smithi167 ceph-mon[21875]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' 2024-08-19T01:30:36.216 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:30:35 smithi167 ceph-mon[21875]: osdmap e60: 9 total, 9 up, 9 in 2024-08-19T01:30:36.314 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:30:36.312+0000 7fa959800640 1 -- 172.21.15.83:0/1037005532 --> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi083:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7fa954062360 con 0x7fa9380a2b80 2024-08-19T01:30:37.154 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:30:36 smithi083 ceph-mon[17120]: from='client.14343 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi083:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-08-19T01:30:37.155 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:30:36 smithi083 ceph-mon[17120]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-08-19T01:30:37.155 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:30:36 smithi083 ceph-mon[17120]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-08-19T01:30:37.155 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:30:36 smithi083 ceph-mon[17120]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-19T01:30:37.155 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:30:36 smithi083 ceph-mon[17120]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 664 MiB used, 804 GiB / 805 GiB avail 2024-08-19T01:30:37.155 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:30:36 smithi083 ceph-mon[17120]: osdmap e61: 9 total, 9 up, 9 in 2024-08-19T01:30:37.202 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:30:36 smithi117 ceph-mon[22016]: from='client.14343 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi083:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-08-19T01:30:37.203 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:30:36 smithi117 ceph-mon[22016]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-08-19T01:30:37.203 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:30:36 smithi117 ceph-mon[22016]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-08-19T01:30:37.203 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:30:36 smithi117 ceph-mon[22016]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-19T01:30:37.203 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:30:36 smithi117 ceph-mon[22016]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 664 MiB used, 804 GiB / 805 GiB avail 2024-08-19T01:30:37.203 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:30:36 smithi117 ceph-mon[22016]: osdmap e61: 9 total, 9 up, 9 in 2024-08-19T01:30:37.216 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:30:36 smithi167 ceph-mon[21875]: from='client.14343 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi083:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-08-19T01:30:37.216 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:30:36 smithi167 ceph-mon[21875]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-08-19T01:30:37.216 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:30:36 smithi167 ceph-mon[21875]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-08-19T01:30:37.216 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:30:36 smithi167 ceph-mon[21875]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-19T01:30:37.216 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:30:36 smithi167 ceph-mon[21875]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 664 MiB used, 804 GiB / 805 GiB avail 2024-08-19T01:30:37.216 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:30:36 smithi167 ceph-mon[21875]: osdmap e61: 9 total, 9 up, 9 in 2024-08-19T01:30:39.106 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:30:38 smithi083 ceph-mon[17120]: osdmap e62: 9 total, 9 up, 9 in 2024-08-19T01:30:39.106 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:30:38 smithi083 ceph-mon[17120]: pgmap v189: 1 pgs: 1 peering; 577 KiB data, 665 MiB used, 804 GiB / 805 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-08-19T01:30:39.203 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:30:38 smithi117 ceph-mon[22016]: osdmap e62: 9 total, 9 up, 9 in 2024-08-19T01:30:39.203 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:30:38 smithi117 ceph-mon[22016]: pgmap v189: 1 pgs: 1 peering; 577 KiB data, 665 MiB used, 804 GiB / 805 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-08-19T01:30:39.216 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:30:38 smithi167 ceph-mon[21875]: osdmap e62: 9 total, 9 up, 9 in 2024-08-19T01:30:39.216 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:30:38 smithi167 ceph-mon[21875]: pgmap v189: 1 pgs: 1 peering; 577 KiB data, 665 MiB used, 804 GiB / 805 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-08-19T01:30:40.155 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:30:39 smithi083 ceph-mon[17120]: from='client.? 172.21.15.83:0/2107850758' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0151bc2a-3ad2-4db3-8505-5855792cc9bb"}]: dispatch 2024-08-19T01:30:40.155 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:30:39 smithi083 ceph-mon[17120]: from='client.? 172.21.15.83:0/2107850758' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "0151bc2a-3ad2-4db3-8505-5855792cc9bb"}]': finished 2024-08-19T01:30:40.155 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:30:39 smithi083 ceph-mon[17120]: osdmap e63: 10 total, 9 up, 10 in 2024-08-19T01:30:40.155 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:30:39 smithi083 ceph-mon[17120]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-08-19T01:30:40.202 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:30:39 smithi117 ceph-mon[22016]: from='client.? 172.21.15.83:0/2107850758' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0151bc2a-3ad2-4db3-8505-5855792cc9bb"}]: dispatch 2024-08-19T01:30:40.203 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:30:39 smithi117 ceph-mon[22016]: from='client.? 172.21.15.83:0/2107850758' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "0151bc2a-3ad2-4db3-8505-5855792cc9bb"}]': finished 2024-08-19T01:30:40.203 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:30:39 smithi117 ceph-mon[22016]: osdmap e63: 10 total, 9 up, 10 in 2024-08-19T01:30:40.203 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:30:39 smithi117 ceph-mon[22016]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-08-19T01:30:40.216 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:30:39 smithi167 ceph-mon[21875]: from='client.? 172.21.15.83:0/2107850758' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0151bc2a-3ad2-4db3-8505-5855792cc9bb"}]: dispatch 2024-08-19T01:30:40.216 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:30:39 smithi167 ceph-mon[21875]: from='client.? 172.21.15.83:0/2107850758' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "0151bc2a-3ad2-4db3-8505-5855792cc9bb"}]': finished 2024-08-19T01:30:40.216 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:30:39 smithi167 ceph-mon[21875]: osdmap e63: 10 total, 9 up, 10 in 2024-08-19T01:30:40.216 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:30:39 smithi167 ceph-mon[21875]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-08-19T01:30:41.154 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:30:40 smithi083 ceph-mon[17120]: from='client.? 172.21.15.83:0/1206445176' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-08-19T01:30:41.155 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:30:40 smithi083 ceph-mon[17120]: pgmap v191: 1 pgs: 1 peering; 577 KiB data, 269 MiB used, 804 GiB / 805 GiB avail; 125 KiB/s, 0 objects/s recovering 2024-08-19T01:30:41.202 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:30:40 smithi117 ceph-mon[22016]: from='client.? 172.21.15.83:0/1206445176' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-08-19T01:30:41.202 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:30:40 smithi117 ceph-mon[22016]: pgmap v191: 1 pgs: 1 peering; 577 KiB data, 269 MiB used, 804 GiB / 805 GiB avail; 125 KiB/s, 0 objects/s recovering 2024-08-19T01:30:41.216 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:30:40 smithi167 ceph-mon[21875]: from='client.? 172.21.15.83:0/1206445176' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-08-19T01:30:41.216 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:30:40 smithi167 ceph-mon[21875]: pgmap v191: 1 pgs: 1 peering; 577 KiB data, 269 MiB used, 804 GiB / 805 GiB avail; 125 KiB/s, 0 objects/s recovering 2024-08-19T01:30:43.654 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:30:43 smithi083 ceph-mon[17120]: pgmap v192: 1 pgs: 1 peering; 577 KiB data, 269 MiB used, 804 GiB / 805 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-08-19T01:30:43.702 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:30:43 smithi117 ceph-mon[22016]: pgmap v192: 1 pgs: 1 peering; 577 KiB data, 269 MiB used, 804 GiB / 805 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-08-19T01:30:43.716 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:30:43 smithi167 ceph-mon[21875]: pgmap v192: 1 pgs: 1 peering; 577 KiB data, 269 MiB used, 804 GiB / 805 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-08-19T01:30:45.654 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:30:45 smithi083 ceph-mon[17120]: pgmap v193: 1 pgs: 1 active+clean; 577 KiB data, 269 MiB used, 804 GiB / 805 GiB avail; 76 KiB/s, 0 objects/s recovering 2024-08-19T01:30:45.702 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:30:45 smithi117 ceph-mon[22016]: pgmap v193: 1 pgs: 1 active+clean; 577 KiB data, 269 MiB used, 804 GiB / 805 GiB avail; 76 KiB/s, 0 objects/s recovering 2024-08-19T01:30:45.716 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:30:45 smithi167 ceph-mon[21875]: pgmap v193: 1 pgs: 1 active+clean; 577 KiB data, 269 MiB used, 804 GiB / 805 GiB avail; 76 KiB/s, 0 objects/s recovering 2024-08-19T01:30:47.654 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:30:47 smithi083 ceph-mon[17120]: pgmap v194: 1 pgs: 1 active+clean; 577 KiB data, 269 MiB used, 804 GiB / 805 GiB avail 2024-08-19T01:30:47.702 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:30:47 smithi117 ceph-mon[22016]: pgmap v194: 1 pgs: 1 active+clean; 577 KiB data, 269 MiB used, 804 GiB / 805 GiB avail 2024-08-19T01:30:47.716 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:30:47 smithi167 ceph-mon[21875]: pgmap v194: 1 pgs: 1 active+clean; 577 KiB data, 269 MiB used, 804 GiB / 805 GiB avail 2024-08-19T01:30:48.639 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:30:48 smithi083 ceph-mon[17120]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-08-19T01:30:48.639 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:30:48 smithi083 ceph-mon[17120]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-19T01:30:48.702 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:30:48 smithi117 ceph-mon[22016]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-08-19T01:30:48.702 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:30:48 smithi117 ceph-mon[22016]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-19T01:30:48.716 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:30:48 smithi167 ceph-mon[21875]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-08-19T01:30:48.716 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:30:48 smithi167 ceph-mon[21875]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-19T01:30:49.635 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:30:49 smithi083 ceph-mon[17120]: Deploying daemon osd.9 on smithi083 2024-08-19T01:30:49.635 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:30:49 smithi083 ceph-mon[17120]: pgmap v195: 1 pgs: 1 active+clean; 577 KiB data, 269 MiB used, 804 GiB / 805 GiB avail 2024-08-19T01:30:49.676 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:30:49 smithi167 ceph-mon[21875]: Deploying daemon osd.9 on smithi083 2024-08-19T01:30:49.676 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:30:49 smithi167 ceph-mon[21875]: pgmap v195: 1 pgs: 1 active+clean; 577 KiB data, 269 MiB used, 804 GiB / 805 GiB avail 2024-08-19T01:30:49.702 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:30:49 smithi117 ceph-mon[22016]: Deploying daemon osd.9 on smithi083 2024-08-19T01:30:49.703 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:30:49 smithi117 ceph-mon[22016]: pgmap v195: 1 pgs: 1 active+clean; 577 KiB data, 269 MiB used, 804 GiB / 805 GiB avail 2024-08-19T01:30:51.489 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:30:51 smithi083 ceph-mon[17120]: pgmap v196: 1 pgs: 1 active+clean; 577 KiB data, 269 MiB used, 804 GiB / 805 GiB avail 2024-08-19T01:30:51.702 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:30:51 smithi117 ceph-mon[22016]: pgmap v196: 1 pgs: 1 active+clean; 577 KiB data, 269 MiB used, 804 GiB / 805 GiB avail 2024-08-19T01:30:51.716 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:30:51 smithi167 ceph-mon[21875]: pgmap v196: 1 pgs: 1 active+clean; 577 KiB data, 269 MiB used, 804 GiB / 805 GiB avail 2024-08-19T01:30:53.655 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:30:53 smithi083 ceph-mon[17120]: pgmap v197: 1 pgs: 1 active+clean; 577 KiB data, 269 MiB used, 804 GiB / 805 GiB avail 2024-08-19T01:30:53.702 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:30:53 smithi117 ceph-mon[22016]: pgmap v197: 1 pgs: 1 active+clean; 577 KiB data, 269 MiB used, 804 GiB / 805 GiB avail 2024-08-19T01:30:53.716 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:30:53 smithi167 ceph-mon[21875]: pgmap v197: 1 pgs: 1 active+clean; 577 KiB data, 269 MiB used, 804 GiB / 805 GiB avail 2024-08-19T01:30:55.643 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:30:55 smithi083 ceph-mon[17120]: pgmap v198: 1 pgs: 1 active+clean; 577 KiB data, 269 MiB used, 804 GiB / 805 GiB avail 2024-08-19T01:30:55.643 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:30:55 smithi083 ceph-mon[17120]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-19T01:30:55.643 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:30:55 smithi083 ceph-mon[17120]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' 2024-08-19T01:30:55.643 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:30:55 smithi083 ceph-mon[17120]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' 2024-08-19T01:30:55.702 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:30:55 smithi117 ceph-mon[22016]: pgmap v198: 1 pgs: 1 active+clean; 577 KiB data, 269 MiB used, 804 GiB / 805 GiB avail 2024-08-19T01:30:55.702 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:30:55 smithi117 ceph-mon[22016]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-19T01:30:55.702 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:30:55 smithi117 ceph-mon[22016]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' 2024-08-19T01:30:55.703 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:30:55 smithi117 ceph-mon[22016]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' 2024-08-19T01:30:55.716 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:30:55 smithi167 ceph-mon[21875]: pgmap v198: 1 pgs: 1 active+clean; 577 KiB data, 269 MiB used, 804 GiB / 805 GiB avail 2024-08-19T01:30:55.716 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:30:55 smithi167 ceph-mon[21875]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-19T01:30:55.716 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:30:55 smithi167 ceph-mon[21875]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' 2024-08-19T01:30:55.716 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:30:55 smithi167 ceph-mon[21875]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' 2024-08-19T01:30:57.655 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:30:57 smithi083 ceph-mon[17120]: pgmap v199: 1 pgs: 1 active+clean; 577 KiB data, 269 MiB used, 804 GiB / 805 GiB avail 2024-08-19T01:30:57.702 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:30:57 smithi117 ceph-mon[22016]: pgmap v199: 1 pgs: 1 active+clean; 577 KiB data, 269 MiB used, 804 GiB / 805 GiB avail 2024-08-19T01:30:57.716 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:30:57 smithi167 ceph-mon[21875]: pgmap v199: 1 pgs: 1 active+clean; 577 KiB data, 269 MiB used, 804 GiB / 805 GiB avail 2024-08-19T01:30:59.206 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:30:59.204+0000 7fa94b400640 1 -- 172.21.15.83:0/1037005532 <== mgr.14150 v2:172.21.15.83:6800/683740916 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fa954062360 con 0x7fa9380a2b80 2024-08-19T01:30:59.207 INFO:teuthology.orchestra.run.smithi083.stdout:Created osd(s) 9 on host 'smithi083' 2024-08-19T01:30:59.210 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:30:59.207+0000 7fa959800640 1 -- 172.21.15.83:0/1037005532 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7fa9380a2b80 msgr2=0x7fa9380a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:30:59.210 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:30:59.207+0000 7fa959800640 1 --2- 172.21.15.83:0/1037005532 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7fa9380a2b80 0x7fa9380a5040 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fa94402aa00 tx=0x7fa944000bc0 comp rx=0 tx=0).stop 2024-08-19T01:30:59.210 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:30:59.207+0000 7fa959800640 1 -- 172.21.15.83:0/1037005532 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa9541056f0 msgr2=0x7fa954076830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:30:59.210 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:30:59.207+0000 7fa959800640 1 --2- 172.21.15.83:0/1037005532 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa9541056f0 0x7fa954076830 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fa94c00ec50 tx=0x7fa94c00c6a0 comp rx=0 tx=0).stop 2024-08-19T01:30:59.210 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:30:59.207+0000 7fa959800640 1 -- 172.21.15.83:0/1037005532 shutdown_connections 2024-08-19T01:30:59.210 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:30:59.207+0000 7fa959800640 1 --2- 172.21.15.83:0/1037005532 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7fa9380a2b80 0x7fa9380a5040 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:30:59.210 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:30:59.207+0000 7fa959800640 1 --2- 172.21.15.83:0/1037005532 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa9541056f0 0x7fa954076830 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:30:59.210 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:30:59.207+0000 7fa959800640 1 --2- 172.21.15.83:0/1037005532 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa954104d30 0x7fa95407a210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:30:59.210 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:30:59.207+0000 7fa959800640 1 --2- 172.21.15.83:0/1037005532 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa954103b30 0x7fa954079cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:30:59.210 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:30:59.207+0000 7fa959800640 1 -- 172.21.15.83:0/1037005532 >> 172.21.15.83:0/1037005532 conn(0x7fa9540ff2e0 msgr2=0x7fa954108150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:30:59.210 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:30:59.208+0000 7fa959800640 1 -- 172.21.15.83:0/1037005532 shutdown_connections 2024-08-19T01:30:59.210 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:30:59.208+0000 7fa959800640 1 -- 172.21.15.83:0/1037005532 wait complete. 2024-08-19T01:30:59.220 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:30:58 smithi083 ceph-mon[17120]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' 2024-08-19T01:30:59.220 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:30:58 smithi083 ceph-mon[17120]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' 2024-08-19T01:30:59.220 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:30:58 smithi083 ceph-mon[17120]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-19T01:30:59.220 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:30:58 smithi083 ceph-mon[17120]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-19T01:30:59.220 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:30:58 smithi083 ceph-mon[17120]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' 2024-08-19T01:30:59.220 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:30:58 smithi083 ceph-mon[17120]: pgmap v200: 1 pgs: 1 active+clean; 577 KiB data, 269 MiB used, 804 GiB / 805 GiB avail 2024-08-19T01:30:59.220 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:30:58 smithi083 ceph-mon[17120]: from='osd.9 [v2:172.21.15.83:6826/903463450,v1:172.21.15.83:6827/903463450]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-08-19T01:30:59.453 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:30:58 smithi117 ceph-mon[22016]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' 2024-08-19T01:30:59.453 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:30:58 smithi117 ceph-mon[22016]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' 2024-08-19T01:30:59.453 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:30:58 smithi117 ceph-mon[22016]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-19T01:30:59.453 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:30:58 smithi117 ceph-mon[22016]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-19T01:30:59.453 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:30:58 smithi117 ceph-mon[22016]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' 2024-08-19T01:30:59.453 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:30:58 smithi117 ceph-mon[22016]: pgmap v200: 1 pgs: 1 active+clean; 577 KiB data, 269 MiB used, 804 GiB / 805 GiB avail 2024-08-19T01:30:59.453 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:30:58 smithi117 ceph-mon[22016]: from='osd.9 [v2:172.21.15.83:6826/903463450,v1:172.21.15.83:6827/903463450]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-08-19T01:30:59.466 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:30:58 smithi167 ceph-mon[21875]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' 2024-08-19T01:30:59.466 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:30:58 smithi167 ceph-mon[21875]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' 2024-08-19T01:30:59.466 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:30:58 smithi167 ceph-mon[21875]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-19T01:30:59.467 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:30:58 smithi167 ceph-mon[21875]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-19T01:30:59.467 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:30:58 smithi167 ceph-mon[21875]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' 2024-08-19T01:30:59.467 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:30:58 smithi167 ceph-mon[21875]: pgmap v200: 1 pgs: 1 active+clean; 577 KiB data, 269 MiB used, 804 GiB / 805 GiB avail 2024-08-19T01:30:59.467 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:30:58 smithi167 ceph-mon[21875]: from='osd.9 [v2:172.21.15.83:6826/903463450,v1:172.21.15.83:6827/903463450]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-08-19T01:31:00.069 DEBUG:teuthology.orchestra.run.smithi083:osd.9> sudo journalctl -f -n 0 -u ceph-bad8780e-5dc9-11ef-bccf-c7b262605968@osd.9.service 2024-08-19T01:31:00.072 INFO:tasks.cephadm:Deploying osd.10 on smithi117 with /dev/vg_nvme/lv_1... 2024-08-19T01:31:00.073 DEBUG:teuthology.orchestra.run.smithi117:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833 ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid bad8780e-5dc9-11ef-bccf-c7b262605968 -- lvm zap /dev/vg_nvme/lv_1 2024-08-19T01:31:00.332 INFO:teuthology.orchestra.run.smithi117.stderr:Inferring config /var/lib/ceph/bad8780e-5dc9-11ef-bccf-c7b262605968/mon.b/config 2024-08-19T01:31:00.380 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:31:00 smithi083 ceph-mon[17120]: from='osd.9 [v2:172.21.15.83:6826/903463450,v1:172.21.15.83:6827/903463450]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-08-19T01:31:00.380 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:31:00 smithi083 ceph-mon[17120]: osdmap e64: 10 total, 9 up, 10 in 2024-08-19T01:31:00.380 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:31:00 smithi083 ceph-mon[17120]: from='osd.9 [v2:172.21.15.83:6826/903463450,v1:172.21.15.83:6827/903463450]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi083", "root=default"]}]: dispatch 2024-08-19T01:31:00.380 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:31:00 smithi083 ceph-mon[17120]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-08-19T01:31:00.380 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:31:00 smithi083 ceph-mon[17120]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-19T01:31:00.380 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:31:00 smithi083 ceph-mon[17120]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' 2024-08-19T01:31:00.381 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:31:00 smithi083 ceph-mon[17120]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' 2024-08-19T01:31:00.453 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:31:00 smithi117 ceph-mon[22016]: from='osd.9 [v2:172.21.15.83:6826/903463450,v1:172.21.15.83:6827/903463450]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-08-19T01:31:00.453 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:31:00 smithi117 ceph-mon[22016]: osdmap e64: 10 total, 9 up, 10 in 2024-08-19T01:31:00.453 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:31:00 smithi117 ceph-mon[22016]: from='osd.9 [v2:172.21.15.83:6826/903463450,v1:172.21.15.83:6827/903463450]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi083", "root=default"]}]: dispatch 2024-08-19T01:31:00.453 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:31:00 smithi117 ceph-mon[22016]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-08-19T01:31:00.453 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:31:00 smithi117 ceph-mon[22016]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-19T01:31:00.453 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:31:00 smithi117 ceph-mon[22016]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' 2024-08-19T01:31:00.453 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:31:00 smithi117 ceph-mon[22016]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' 2024-08-19T01:31:00.466 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:31:00 smithi167 ceph-mon[21875]: from='osd.9 [v2:172.21.15.83:6826/903463450,v1:172.21.15.83:6827/903463450]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-08-19T01:31:00.466 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:31:00 smithi167 ceph-mon[21875]: osdmap e64: 10 total, 9 up, 10 in 2024-08-19T01:31:00.466 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:31:00 smithi167 ceph-mon[21875]: from='osd.9 [v2:172.21.15.83:6826/903463450,v1:172.21.15.83:6827/903463450]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi083", "root=default"]}]: dispatch 2024-08-19T01:31:00.466 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:31:00 smithi167 ceph-mon[21875]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-08-19T01:31:00.466 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:31:00 smithi167 ceph-mon[21875]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-19T01:31:00.466 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:31:00 smithi167 ceph-mon[21875]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' 2024-08-19T01:31:00.466 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:31:00 smithi167 ceph-mon[21875]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' 2024-08-19T01:31:01.350 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:31:01 smithi117 ceph-mon[22016]: from='osd.9 [v2:172.21.15.83:6826/903463450,v1:172.21.15.83:6827/903463450]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi083", "root=default"]}]': finished 2024-08-19T01:31:01.350 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:31:01 smithi117 ceph-mon[22016]: osdmap e65: 10 total, 9 up, 10 in 2024-08-19T01:31:01.350 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:31:01 smithi117 ceph-mon[22016]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-08-19T01:31:01.350 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:31:01 smithi117 ceph-mon[22016]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-08-19T01:31:01.350 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:31:01 smithi117 ceph-mon[22016]: pgmap v203: 1 pgs: 1 active+clean; 577 KiB data, 269 MiB used, 804 GiB / 805 GiB avail 2024-08-19T01:31:01.350 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:31:01 smithi117 ceph-mon[22016]: from='osd.9 [v2:172.21.15.83:6826/903463450,v1:172.21.15.83:6827/903463450]' entity='osd.9' 2024-08-19T01:31:01.405 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:31:01 smithi083 ceph-mon[17120]: from='osd.9 [v2:172.21.15.83:6826/903463450,v1:172.21.15.83:6827/903463450]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi083", "root=default"]}]': finished 2024-08-19T01:31:01.405 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:31:01 smithi083 ceph-mon[17120]: osdmap e65: 10 total, 9 up, 10 in 2024-08-19T01:31:01.405 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:31:01 smithi083 ceph-mon[17120]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-08-19T01:31:01.405 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:31:01 smithi083 ceph-mon[17120]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-08-19T01:31:01.405 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:31:01 smithi083 ceph-mon[17120]: pgmap v203: 1 pgs: 1 active+clean; 577 KiB data, 269 MiB used, 804 GiB / 805 GiB avail 2024-08-19T01:31:01.405 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:31:01 smithi083 ceph-mon[17120]: from='osd.9 [v2:172.21.15.83:6826/903463450,v1:172.21.15.83:6827/903463450]' entity='osd.9' 2024-08-19T01:31:01.405 INFO:journalctl@ceph.osd.9.smithi083.stdout:Aug 19 01:31:00 smithi083 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-9[51472]: 2024-08-19T01:31:00.955+0000 7f54f2200640 -1 osd.9 0 waiting for initial osdmap 2024-08-19T01:31:01.405 INFO:journalctl@ceph.osd.9.smithi083.stdout:Aug 19 01:31:00 smithi083 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-9[51472]: 2024-08-19T01:31:00.986+0000 7f54e6e00640 -1 osd.9 65 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-08-19T01:31:01.466 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:31:01 smithi167 ceph-mon[21875]: from='osd.9 [v2:172.21.15.83:6826/903463450,v1:172.21.15.83:6827/903463450]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi083", "root=default"]}]': finished 2024-08-19T01:31:01.466 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:31:01 smithi167 ceph-mon[21875]: osdmap e65: 10 total, 9 up, 10 in 2024-08-19T01:31:01.466 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:31:01 smithi167 ceph-mon[21875]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-08-19T01:31:01.466 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:31:01 smithi167 ceph-mon[21875]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-08-19T01:31:01.466 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:31:01 smithi167 ceph-mon[21875]: pgmap v203: 1 pgs: 1 active+clean; 577 KiB data, 269 MiB used, 804 GiB / 805 GiB avail 2024-08-19T01:31:01.466 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:31:01 smithi167 ceph-mon[21875]: from='osd.9 [v2:172.21.15.83:6826/903463450,v1:172.21.15.83:6827/903463450]' entity='osd.9' 2024-08-19T01:31:02.399 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:31:02 smithi117 ceph-mon[22016]: purged_snaps scrub starts 2024-08-19T01:31:02.399 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:31:02 smithi117 ceph-mon[22016]: purged_snaps scrub ok 2024-08-19T01:31:02.399 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:31:02 smithi117 ceph-mon[22016]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-08-19T01:31:02.400 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:31:02 smithi117 ceph-mon[22016]: osd.9 [v2:172.21.15.83:6826/903463450,v1:172.21.15.83:6827/903463450] boot 2024-08-19T01:31:02.400 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:31:02 smithi117 ceph-mon[22016]: osdmap e66: 10 total, 10 up, 10 in 2024-08-19T01:31:02.400 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:31:02 smithi117 ceph-mon[22016]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-08-19T01:31:02.404 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:31:02 smithi083 ceph-mon[17120]: purged_snaps scrub starts 2024-08-19T01:31:02.405 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:31:02 smithi083 ceph-mon[17120]: purged_snaps scrub ok 2024-08-19T01:31:02.405 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:31:02 smithi083 ceph-mon[17120]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-08-19T01:31:02.405 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:31:02 smithi083 ceph-mon[17120]: osd.9 [v2:172.21.15.83:6826/903463450,v1:172.21.15.83:6827/903463450] boot 2024-08-19T01:31:02.405 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:31:02 smithi083 ceph-mon[17120]: osdmap e66: 10 total, 10 up, 10 in 2024-08-19T01:31:02.405 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:31:02 smithi083 ceph-mon[17120]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-08-19T01:31:02.466 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:31:02 smithi167 ceph-mon[21875]: purged_snaps scrub starts 2024-08-19T01:31:02.466 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:31:02 smithi167 ceph-mon[21875]: purged_snaps scrub ok 2024-08-19T01:31:02.466 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:31:02 smithi167 ceph-mon[21875]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-08-19T01:31:02.467 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:31:02 smithi167 ceph-mon[21875]: osd.9 [v2:172.21.15.83:6826/903463450,v1:172.21.15.83:6827/903463450] boot 2024-08-19T01:31:02.467 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:31:02 smithi167 ceph-mon[21875]: osdmap e66: 10 total, 10 up, 10 in 2024-08-19T01:31:02.467 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:31:02 smithi167 ceph-mon[21875]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-08-19T01:31:02.719 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-08-19T01:31:02.746 DEBUG:teuthology.orchestra.run.smithi117:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid bad8780e-5dc9-11ef-bccf-c7b262605968 -- ceph orch daemon add osd smithi117:vg_nvme/lv_1 2024-08-19T01:31:02.999 INFO:teuthology.orchestra.run.smithi117.stderr:Inferring config /var/lib/ceph/bad8780e-5dc9-11ef-bccf-c7b262605968/mon.b/config 2024-08-19T01:31:03.036 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:31:03 smithi117 ceph-mon[22016]: pgmap v205: 1 pgs: 1 active+clean; 577 KiB data, 300 MiB used, 894 GiB / 894 GiB avail 2024-08-19T01:31:03.279 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:31:03 smithi083 ceph-mon[17120]: pgmap v205: 1 pgs: 1 active+clean; 577 KiB data, 300 MiB used, 894 GiB / 894 GiB avail 2024-08-19T01:31:03.466 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:31:03 smithi167 ceph-mon[21875]: pgmap v205: 1 pgs: 1 active+clean; 577 KiB data, 300 MiB used, 894 GiB / 894 GiB avail 2024-08-19T01:31:03.477 INFO:teuthology.orchestra.run.smithi117.stderr:2024-08-19T01:31:03.468+0000 7f49c9d90640 1 -- 172.21.15.117:0/2022285074 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f49c4101a00 msgr2=0x7f49c4101e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:31:03.478 INFO:teuthology.orchestra.run.smithi117.stderr:2024-08-19T01:31:03.468+0000 7f49c9d90640 1 --2- 172.21.15.117:0/2022285074 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f49c4101a00 0x7f49c4101e80 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f49ac009900 tx=0x7f49ac02a410 comp rx=0 tx=0).stop 2024-08-19T01:31:03.478 INFO:teuthology.orchestra.run.smithi117.stderr:2024-08-19T01:31:03.468+0000 7f49c9d90640 1 -- 172.21.15.117:0/2022285074 shutdown_connections 2024-08-19T01:31:03.478 INFO:teuthology.orchestra.run.smithi117.stderr:2024-08-19T01:31:03.468+0000 7f49c9d90640 1 --2- 172.21.15.117:0/2022285074 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f49c41023c0 0x7f49c4108c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:31:03.478 INFO:teuthology.orchestra.run.smithi117.stderr:2024-08-19T01:31:03.468+0000 7f49c9d90640 1 --2- 172.21.15.117:0/2022285074 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f49c4101a00 0x7f49c4101e80 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:31:03.478 INFO:teuthology.orchestra.run.smithi117.stderr:2024-08-19T01:31:03.468+0000 7f49c9d90640 1 --2- 172.21.15.117:0/2022285074 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f49c4100800 0x7f49c4100c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:31:03.478 INFO:teuthology.orchestra.run.smithi117.stderr:2024-08-19T01:31:03.468+0000 7f49c9d90640 1 -- 172.21.15.117:0/2022285074 >> 172.21.15.117:0/2022285074 conn(0x7f49c40fbfb0 msgr2=0x7f49c40fe3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:31:03.478 INFO:teuthology.orchestra.run.smithi117.stderr:2024-08-19T01:31:03.468+0000 7f49c9d90640 1 -- 172.21.15.117:0/2022285074 shutdown_connections 2024-08-19T01:31:03.478 INFO:teuthology.orchestra.run.smithi117.stderr:2024-08-19T01:31:03.468+0000 7f49c9d90640 1 -- 172.21.15.117:0/2022285074 wait complete. 2024-08-19T01:31:03.478 INFO:teuthology.orchestra.run.smithi117.stderr:2024-08-19T01:31:03.468+0000 7f49c9d90640 1 Processor -- start 2024-08-19T01:31:03.478 INFO:teuthology.orchestra.run.smithi117.stderr:2024-08-19T01:31:03.468+0000 7f49c9d90640 1 -- start start 2024-08-19T01:31:03.478 INFO:teuthology.orchestra.run.smithi117.stderr:2024-08-19T01:31:03.469+0000 7f49c9d90640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f49c4100800 0x7f49c4197c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:31:03.479 INFO:teuthology.orchestra.run.smithi117.stderr:2024-08-19T01:31:03.469+0000 7f49c9d90640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f49c4101a00 0x7f49c41981c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:31:03.479 INFO:teuthology.orchestra.run.smithi117.stderr:2024-08-19T01:31:03.469+0000 7f49c9d90640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f49c41023c0 0x7f49c419f230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:31:03.479 INFO:teuthology.orchestra.run.smithi117.stderr:2024-08-19T01:31:03.469+0000 7f49c9d90640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f49c410d610 con 0x7f49c41023c0 2024-08-19T01:31:03.479 INFO:teuthology.orchestra.run.smithi117.stderr:2024-08-19T01:31:03.469+0000 7f49c9d90640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f49c410d490 con 0x7f49c4100800 2024-08-19T01:31:03.479 INFO:teuthology.orchestra.run.smithi117.stderr:2024-08-19T01:31:03.469+0000 7f49c9d90640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f49c410d790 con 0x7f49c4101a00 2024-08-19T01:31:03.479 INFO:teuthology.orchestra.run.smithi117.stderr:2024-08-19T01:31:03.469+0000 7f49c37fe640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f49c4100800 0x7f49c4197c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:31:03.479 INFO:teuthology.orchestra.run.smithi117.stderr:2024-08-19T01:31:03.469+0000 7f49c37fe640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f49c4100800 0x7f49c4197c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:36284/0 (socket says 172.21.15.117:36284) 2024-08-19T01:31:03.479 INFO:teuthology.orchestra.run.smithi117.stderr:2024-08-19T01:31:03.469+0000 7f49c37fe640 1 -- 172.21.15.117:0/1284878281 learned_addr learned my addr 172.21.15.117:0/1284878281 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-08-19T01:31:03.479 INFO:teuthology.orchestra.run.smithi117.stderr:2024-08-19T01:31:03.469+0000 7f49c3fff640 1 --2- 172.21.15.117:0/1284878281 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f49c41023c0 0x7f49c419f230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:31:03.479 INFO:teuthology.orchestra.run.smithi117.stderr:2024-08-19T01:31:03.469+0000 7f49c37fe640 1 -- 172.21.15.117:0/1284878281 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f49c4101a00 msgr2=0x7f49c41981c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:31:03.479 INFO:teuthology.orchestra.run.smithi117.stderr:2024-08-19T01:31:03.469+0000 7f49baffd640 1 --2- 172.21.15.117:0/1284878281 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f49c4101a00 0x7f49c41981c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:31:03.479 INFO:teuthology.orchestra.run.smithi117.stderr:2024-08-19T01:31:03.469+0000 7f49c37fe640 1 --2- 172.21.15.117:0/1284878281 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f49c4101a00 0x7f49c41981c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:31:03.479 INFO:teuthology.orchestra.run.smithi117.stderr:2024-08-19T01:31:03.469+0000 7f49c37fe640 1 -- 172.21.15.117:0/1284878281 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f49c41023c0 msgr2=0x7f49c419f230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:31:03.480 INFO:teuthology.orchestra.run.smithi117.stderr:2024-08-19T01:31:03.469+0000 7f49c37fe640 1 --2- 172.21.15.117:0/1284878281 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f49c41023c0 0x7f49c419f230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:31:03.480 INFO:teuthology.orchestra.run.smithi117.stderr:2024-08-19T01:31:03.469+0000 7f49c37fe640 1 -- 172.21.15.117:0/1284878281 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f49c419f8a0 con 0x7f49c4100800 2024-08-19T01:31:03.480 INFO:teuthology.orchestra.run.smithi117.stderr:2024-08-19T01:31:03.469+0000 7f49c3fff640 1 --2- 172.21.15.117:0/1284878281 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f49c41023c0 0x7f49c419f230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-19T01:31:03.480 INFO:teuthology.orchestra.run.smithi117.stderr:2024-08-19T01:31:03.469+0000 7f49baffd640 1 --2- 172.21.15.117:0/1284878281 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f49c4101a00 0x7f49c41981c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-19T01:31:03.480 INFO:teuthology.orchestra.run.smithi117.stderr:2024-08-19T01:31:03.469+0000 7f49c37fe640 1 --2- 172.21.15.117:0/1284878281 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f49c4100800 0x7f49c4197c80 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f49b400b450 tx=0x7f49b400b910 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:31:03.480 INFO:teuthology.orchestra.run.smithi117.stderr:2024-08-19T01:31:03.469+0000 7f49c17fa640 1 -- 172.21.15.117:0/1284878281 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49b4015020 con 0x7f49c4100800 2024-08-19T01:31:03.480 INFO:teuthology.orchestra.run.smithi117.stderr:2024-08-19T01:31:03.469+0000 7f49c17fa640 1 -- 172.21.15.117:0/1284878281 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f49b4011d70 con 0x7f49c4100800 2024-08-19T01:31:03.480 INFO:teuthology.orchestra.run.smithi117.stderr:2024-08-19T01:31:03.469+0000 7f49c9d90640 1 -- 172.21.15.117:0/1284878281 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f49c419fb90 con 0x7f49c4100800 2024-08-19T01:31:03.480 INFO:teuthology.orchestra.run.smithi117.stderr:2024-08-19T01:31:03.469+0000 7f49c17fa640 1 -- 172.21.15.117:0/1284878281 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49b400f340 con 0x7f49c4100800 2024-08-19T01:31:03.480 INFO:teuthology.orchestra.run.smithi117.stderr:2024-08-19T01:31:03.469+0000 7f49c9d90640 1 -- 172.21.15.117:0/1284878281 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f49c41a00d0 con 0x7f49c4100800 2024-08-19T01:31:03.480 INFO:teuthology.orchestra.run.smithi117.stderr:2024-08-19T01:31:03.471+0000 7f49c9d90640 1 -- 172.21.15.117:0/1284878281 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f49c4107ca0 con 0x7f49c4100800 2024-08-19T01:31:03.480 INFO:teuthology.orchestra.run.smithi117.stderr:2024-08-19T01:31:03.472+0000 7f49c17fa640 1 -- 172.21.15.117:0/1284878281 <== mon.2 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f49b4018020 con 0x7f49c4100800 2024-08-19T01:31:03.480 INFO:teuthology.orchestra.run.smithi117.stderr:2024-08-19T01:31:03.472+0000 7f49c17fa640 1 --2- 172.21.15.117:0/1284878281 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f49b00a2b80 0x7f49b00a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:31:03.481 INFO:teuthology.orchestra.run.smithi117.stderr:2024-08-19T01:31:03.472+0000 7f49c17fa640 1 -- 172.21.15.117:0/1284878281 <== mon.2 v2:172.21.15.117:3300/0 5 ==== osd_map(67..67 src has 1..67) ==== 6261+0+0 (secure 0 0 0) 0x7f49b40ce3a0 con 0x7f49c4100800 2024-08-19T01:31:03.481 INFO:teuthology.orchestra.run.smithi117.stderr:2024-08-19T01:31:03.473+0000 7f49baffd640 1 --2- 172.21.15.117:0/1284878281 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f49b00a2b80 0x7f49b00a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:31:03.481 INFO:teuthology.orchestra.run.smithi117.stderr:2024-08-19T01:31:03.473+0000 7f49baffd640 1 --2- 172.21.15.117:0/1284878281 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f49b00a2b80 0x7f49b00a5040 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f49ac000c00 tx=0x7f49ac031400 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:31:03.481 INFO:teuthology.orchestra.run.smithi117.stderr:2024-08-19T01:31:03.477+0000 7f49c17fa640 1 -- 172.21.15.117:0/1284878281 <== mon.2 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7f49b401f020 con 0x7f49c4100800 2024-08-19T01:31:03.653 INFO:teuthology.orchestra.run.smithi117.stderr:2024-08-19T01:31:03.652+0000 7f49c9d90640 1 -- 172.21.15.117:0/1284878281 --> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi117:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7f49c4062360 con 0x7f49b00a2b80 2024-08-19T01:31:04.314 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:31:04 smithi117 ceph-mon[22016]: osdmap e67: 10 total, 10 up, 10 in 2024-08-19T01:31:04.314 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:31:04 smithi117 ceph-mon[22016]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-08-19T01:31:04.314 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:31:04 smithi117 ceph-mon[22016]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-08-19T01:31:04.314 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:31:04 smithi117 ceph-mon[22016]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-19T01:31:04.314 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:31:04 smithi117 ceph-mon[22016]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' 2024-08-19T01:31:04.314 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:31:04 smithi117 ceph-mon[22016]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' 2024-08-19T01:31:04.314 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:31:04 smithi117 ceph-mon[22016]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-08-19T01:31:04.315 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:31:04 smithi117 ceph-mon[22016]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-08-19T01:31:04.315 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:31:04 smithi117 ceph-mon[22016]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-08-19T01:31:04.315 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:31:04 smithi117 ceph-mon[22016]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-08-19T01:31:04.315 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:31:04 smithi117 ceph-mon[22016]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' 2024-08-19T01:31:04.315 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:31:04 smithi117 ceph-mon[22016]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-19T01:31:04.315 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:31:04 smithi117 ceph-mon[22016]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-19T01:31:04.315 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:31:04 smithi117 ceph-mon[22016]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' 2024-08-19T01:31:04.404 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:31:04 smithi083 ceph-mon[17120]: osdmap e67: 10 total, 10 up, 10 in 2024-08-19T01:31:04.405 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:31:04 smithi083 ceph-mon[17120]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-08-19T01:31:04.405 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:31:04 smithi083 ceph-mon[17120]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-08-19T01:31:04.405 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:31:04 smithi083 ceph-mon[17120]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-19T01:31:04.405 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:31:04 smithi083 ceph-mon[17120]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' 2024-08-19T01:31:04.405 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:31:04 smithi083 ceph-mon[17120]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' 2024-08-19T01:31:04.405 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:31:04 smithi083 ceph-mon[17120]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-08-19T01:31:04.405 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:31:04 smithi083 ceph-mon[17120]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-08-19T01:31:04.405 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:31:04 smithi083 ceph-mon[17120]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-08-19T01:31:04.405 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:31:04 smithi083 ceph-mon[17120]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-08-19T01:31:04.405 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:31:04 smithi083 ceph-mon[17120]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' 2024-08-19T01:31:04.405 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:31:04 smithi083 ceph-mon[17120]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-19T01:31:04.405 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:31:04 smithi083 ceph-mon[17120]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-19T01:31:04.405 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:31:04 smithi083 ceph-mon[17120]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' 2024-08-19T01:31:04.466 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:31:04 smithi167 ceph-mon[21875]: osdmap e67: 10 total, 10 up, 10 in 2024-08-19T01:31:04.466 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:31:04 smithi167 ceph-mon[21875]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-08-19T01:31:04.466 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:31:04 smithi167 ceph-mon[21875]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-08-19T01:31:04.466 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:31:04 smithi167 ceph-mon[21875]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-19T01:31:04.466 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:31:04 smithi167 ceph-mon[21875]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' 2024-08-19T01:31:04.467 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:31:04 smithi167 ceph-mon[21875]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' 2024-08-19T01:31:04.467 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:31:04 smithi167 ceph-mon[21875]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-08-19T01:31:04.467 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:31:04 smithi167 ceph-mon[21875]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-08-19T01:31:04.467 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:31:04 smithi167 ceph-mon[21875]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-08-19T01:31:04.467 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:31:04 smithi167 ceph-mon[21875]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-08-19T01:31:04.467 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:31:04 smithi167 ceph-mon[21875]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' 2024-08-19T01:31:04.467 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:31:04 smithi167 ceph-mon[21875]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-19T01:31:04.467 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:31:04 smithi167 ceph-mon[21875]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-19T01:31:04.467 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:31:04 smithi167 ceph-mon[21875]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' 2024-08-19T01:31:05.098 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:31:05 smithi083 ceph-mon[17120]: from='client.24371 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi117:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-08-19T01:31:05.098 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:31:05 smithi083 ceph-mon[17120]: Detected new or changed devices on smithi083 2024-08-19T01:31:05.098 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:31:05 smithi083 ceph-mon[17120]: Adjusting osd_memory_target on smithi083 to 4272M 2024-08-19T01:31:05.098 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:31:05 smithi083 ceph-mon[17120]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 300 MiB used, 894 GiB / 894 GiB avail 2024-08-19T01:31:05.271 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:31:05 smithi117 ceph-mon[22016]: from='client.24371 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi117:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-08-19T01:31:05.271 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:31:05 smithi117 ceph-mon[22016]: Detected new or changed devices on smithi083 2024-08-19T01:31:05.271 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:31:05 smithi117 ceph-mon[22016]: Adjusting osd_memory_target on smithi083 to 4272M 2024-08-19T01:31:05.271 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:31:05 smithi117 ceph-mon[22016]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 300 MiB used, 894 GiB / 894 GiB avail 2024-08-19T01:31:05.466 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:31:05 smithi167 ceph-mon[21875]: from='client.24371 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi117:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-08-19T01:31:05.466 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:31:05 smithi167 ceph-mon[21875]: Detected new or changed devices on smithi083 2024-08-19T01:31:05.466 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:31:05 smithi167 ceph-mon[21875]: Adjusting osd_memory_target on smithi083 to 4272M 2024-08-19T01:31:05.466 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:31:05 smithi167 ceph-mon[21875]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 300 MiB used, 894 GiB / 894 GiB avail 2024-08-19T01:31:06.301 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:31:06 smithi117 ceph-mon[22016]: from='client.? 172.21.15.117:0/4072892852' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "437ed848-0da4-4cd4-8716-0c55bb99b676"}]: dispatch 2024-08-19T01:31:06.301 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:31:06 smithi117 ceph-mon[22016]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "437ed848-0da4-4cd4-8716-0c55bb99b676"}]: dispatch 2024-08-19T01:31:06.301 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:31:06 smithi117 ceph-mon[22016]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "437ed848-0da4-4cd4-8716-0c55bb99b676"}]': finished 2024-08-19T01:31:06.301 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:31:06 smithi117 ceph-mon[22016]: osdmap e68: 11 total, 10 up, 11 in 2024-08-19T01:31:06.301 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:31:06 smithi117 ceph-mon[22016]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-08-19T01:31:06.405 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:31:06 smithi083 ceph-mon[17120]: from='client.? 172.21.15.117:0/4072892852' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "437ed848-0da4-4cd4-8716-0c55bb99b676"}]: dispatch 2024-08-19T01:31:06.405 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:31:06 smithi083 ceph-mon[17120]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "437ed848-0da4-4cd4-8716-0c55bb99b676"}]: dispatch 2024-08-19T01:31:06.405 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:31:06 smithi083 ceph-mon[17120]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "437ed848-0da4-4cd4-8716-0c55bb99b676"}]': finished 2024-08-19T01:31:06.405 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:31:06 smithi083 ceph-mon[17120]: osdmap e68: 11 total, 10 up, 11 in 2024-08-19T01:31:06.405 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:31:06 smithi083 ceph-mon[17120]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-08-19T01:31:06.466 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:31:06 smithi167 ceph-mon[21875]: from='client.? 172.21.15.117:0/4072892852' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "437ed848-0da4-4cd4-8716-0c55bb99b676"}]: dispatch 2024-08-19T01:31:06.466 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:31:06 smithi167 ceph-mon[21875]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "437ed848-0da4-4cd4-8716-0c55bb99b676"}]: dispatch 2024-08-19T01:31:06.466 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:31:06 smithi167 ceph-mon[21875]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "437ed848-0da4-4cd4-8716-0c55bb99b676"}]': finished 2024-08-19T01:31:06.466 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:31:06 smithi167 ceph-mon[21875]: osdmap e68: 11 total, 10 up, 11 in 2024-08-19T01:31:06.466 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:31:06 smithi167 ceph-mon[21875]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-08-19T01:31:07.404 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:31:07 smithi083 ceph-mon[17120]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 894 GiB / 894 GiB avail 2024-08-19T01:31:07.405 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:31:07 smithi083 ceph-mon[17120]: from='client.? 172.21.15.117:0/2878927613' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-08-19T01:31:07.452 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:31:07 smithi117 ceph-mon[22016]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 894 GiB / 894 GiB avail 2024-08-19T01:31:07.452 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:31:07 smithi117 ceph-mon[22016]: from='client.? 172.21.15.117:0/2878927613' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-08-19T01:31:07.466 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:31:07 smithi167 ceph-mon[21875]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 894 GiB / 894 GiB avail 2024-08-19T01:31:07.466 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:31:07 smithi167 ceph-mon[21875]: from='client.? 172.21.15.117:0/2878927613' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-08-19T01:31:09.655 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:31:09 smithi083 ceph-mon[17120]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 894 GiB / 894 GiB avail 2024-08-19T01:31:09.702 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:31:09 smithi117 ceph-mon[22016]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 894 GiB / 894 GiB avail 2024-08-19T01:31:09.716 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:31:09 smithi167 ceph-mon[21875]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 894 GiB / 894 GiB avail 2024-08-19T01:31:11.655 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:31:11 smithi083 ceph-mon[17120]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 894 GiB / 894 GiB avail 2024-08-19T01:31:11.702 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:31:11 smithi117 ceph-mon[22016]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 894 GiB / 894 GiB avail 2024-08-19T01:31:11.716 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:31:11 smithi167 ceph-mon[21875]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 894 GiB / 894 GiB avail 2024-08-19T01:31:13.654 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:31:13 smithi083 ceph-mon[17120]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 894 GiB / 894 GiB avail 2024-08-19T01:31:13.655 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:31:13 smithi083 ceph-mon[17120]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-08-19T01:31:13.655 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:31:13 smithi083 ceph-mon[17120]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-19T01:31:13.702 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:31:13 smithi117 ceph-mon[22016]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 894 GiB / 894 GiB avail 2024-08-19T01:31:13.703 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:31:13 smithi117 ceph-mon[22016]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-08-19T01:31:13.703 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:31:13 smithi117 ceph-mon[22016]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-19T01:31:13.716 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:31:13 smithi167 ceph-mon[21875]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 894 GiB / 894 GiB avail 2024-08-19T01:31:13.716 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:31:13 smithi167 ceph-mon[21875]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-08-19T01:31:13.716 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:31:13 smithi167 ceph-mon[21875]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-19T01:31:14.634 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:31:14 smithi117 ceph-mon[22016]: Deploying daemon osd.10 on smithi117 2024-08-19T01:31:14.716 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:31:14 smithi167 ceph-mon[21875]: Deploying daemon osd.10 on smithi117 2024-08-19T01:31:14.905 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:31:14 smithi083 ceph-mon[17120]: Deploying daemon osd.10 on smithi117 2024-08-19T01:31:15.692 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:31:15 smithi083 ceph-mon[17120]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 894 GiB / 894 GiB avail 2024-08-19T01:31:15.702 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:31:15 smithi117 ceph-mon[22016]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 894 GiB / 894 GiB avail 2024-08-19T01:31:15.716 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:31:15 smithi167 ceph-mon[21875]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 894 GiB / 894 GiB avail 2024-08-19T01:31:17.703 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:31:17 smithi117 ceph-mon[22016]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 894 GiB / 894 GiB avail 2024-08-19T01:31:17.716 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:31:17 smithi167 ceph-mon[21875]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 894 GiB / 894 GiB avail 2024-08-19T01:31:17.904 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:31:17 smithi083 ceph-mon[17120]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 894 GiB / 894 GiB avail 2024-08-19T01:31:18.531 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:31:18 smithi117 ceph-mon[22016]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-19T01:31:18.531 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:31:18 smithi117 ceph-mon[22016]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' 2024-08-19T01:31:18.531 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:31:18 smithi117 ceph-mon[22016]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' 2024-08-19T01:31:18.716 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:31:18 smithi167 ceph-mon[21875]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-19T01:31:18.716 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:31:18 smithi167 ceph-mon[21875]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' 2024-08-19T01:31:18.716 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:31:18 smithi167 ceph-mon[21875]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' 2024-08-19T01:31:18.905 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:31:18 smithi083 ceph-mon[17120]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-19T01:31:18.905 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:31:18 smithi083 ceph-mon[17120]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' 2024-08-19T01:31:18.905 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:31:18 smithi083 ceph-mon[17120]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' 2024-08-19T01:31:19.556 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:31:19 smithi117 ceph-mon[22016]: pgmap v215: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 894 GiB / 894 GiB avail 2024-08-19T01:31:19.716 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:31:19 smithi167 ceph-mon[21875]: pgmap v215: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 894 GiB / 894 GiB avail 2024-08-19T01:31:19.904 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:31:19 smithi083 ceph-mon[17120]: pgmap v215: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 894 GiB / 894 GiB avail 2024-08-19T01:31:21.172 INFO:teuthology.orchestra.run.smithi117.stderr:2024-08-19T01:31:21.170+0000 7f49c17fa640 1 -- 172.21.15.117:0/1284878281 <== mgr.14150 v2:172.21.15.83:6800/683740916 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+38 (secure 0 0 0) 0x7f49c4062360 con 0x7f49b00a2b80 2024-08-19T01:31:21.172 INFO:teuthology.orchestra.run.smithi117.stdout:Created osd(s) 10 on host 'smithi117' 2024-08-19T01:31:21.174 INFO:teuthology.orchestra.run.smithi117.stderr:2024-08-19T01:31:21.172+0000 7f49c9d90640 1 -- 172.21.15.117:0/1284878281 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f49b00a2b80 msgr2=0x7f49b00a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:31:21.174 INFO:teuthology.orchestra.run.smithi117.stderr:2024-08-19T01:31:21.172+0000 7f49c9d90640 1 --2- 172.21.15.117:0/1284878281 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f49b00a2b80 0x7f49b00a5040 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f49ac000c00 tx=0x7f49ac031400 comp rx=0 tx=0).stop 2024-08-19T01:31:21.174 INFO:teuthology.orchestra.run.smithi117.stderr:2024-08-19T01:31:21.172+0000 7f49c9d90640 1 -- 172.21.15.117:0/1284878281 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f49c4100800 msgr2=0x7f49c4197c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:31:21.174 INFO:teuthology.orchestra.run.smithi117.stderr:2024-08-19T01:31:21.172+0000 7f49c9d90640 1 --2- 172.21.15.117:0/1284878281 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f49c4100800 0x7f49c4197c80 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f49b400b450 tx=0x7f49b400b910 comp rx=0 tx=0).stop 2024-08-19T01:31:21.174 INFO:teuthology.orchestra.run.smithi117.stderr:2024-08-19T01:31:21.172+0000 7f49c9d90640 1 -- 172.21.15.117:0/1284878281 shutdown_connections 2024-08-19T01:31:21.174 INFO:teuthology.orchestra.run.smithi117.stderr:2024-08-19T01:31:21.172+0000 7f49c9d90640 1 --2- 172.21.15.117:0/1284878281 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f49b00a2b80 0x7f49b00a5040 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:31:21.174 INFO:teuthology.orchestra.run.smithi117.stderr:2024-08-19T01:31:21.172+0000 7f49c9d90640 1 --2- 172.21.15.117:0/1284878281 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f49c41023c0 0x7f49c419f230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:31:21.175 INFO:teuthology.orchestra.run.smithi117.stderr:2024-08-19T01:31:21.172+0000 7f49c9d90640 1 --2- 172.21.15.117:0/1284878281 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f49c4101a00 0x7f49c41981c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:31:21.175 INFO:teuthology.orchestra.run.smithi117.stderr:2024-08-19T01:31:21.172+0000 7f49c9d90640 1 --2- 172.21.15.117:0/1284878281 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f49c4100800 0x7f49c4197c80 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:31:21.175 INFO:teuthology.orchestra.run.smithi117.stderr:2024-08-19T01:31:21.172+0000 7f49c9d90640 1 -- 172.21.15.117:0/1284878281 >> 172.21.15.117:0/1284878281 conn(0x7f49c40fbfb0 msgr2=0x7f49c4104e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:31:21.175 INFO:teuthology.orchestra.run.smithi117.stderr:2024-08-19T01:31:21.173+0000 7f49c9d90640 1 -- 172.21.15.117:0/1284878281 shutdown_connections 2024-08-19T01:31:21.175 INFO:teuthology.orchestra.run.smithi117.stderr:2024-08-19T01:31:21.173+0000 7f49c9d90640 1 -- 172.21.15.117:0/1284878281 wait complete. 2024-08-19T01:31:21.734 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:31:21 smithi117 ceph-mon[22016]: pgmap v216: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 894 GiB / 894 GiB avail 2024-08-19T01:31:21.734 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:31:21 smithi117 ceph-mon[22016]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' 2024-08-19T01:31:21.734 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:31:21 smithi117 ceph-mon[22016]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' 2024-08-19T01:31:21.734 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:31:21 smithi117 ceph-mon[22016]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-19T01:31:21.734 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:31:21 smithi117 ceph-mon[22016]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-19T01:31:21.734 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:31:21 smithi117 ceph-mon[22016]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' 2024-08-19T01:31:21.734 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:31:21 smithi117 ceph-mon[22016]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-19T01:31:21.734 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:31:21 smithi117 ceph-mon[22016]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' 2024-08-19T01:31:21.734 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:31:21 smithi117 ceph-mon[22016]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' 2024-08-19T01:31:21.770 DEBUG:teuthology.orchestra.run.smithi117:osd.10> sudo journalctl -f -n 0 -u ceph-bad8780e-5dc9-11ef-bccf-c7b262605968@osd.10.service 2024-08-19T01:31:21.772 INFO:tasks.cephadm:Deploying osd.11 on smithi167 with /dev/vg_nvme/lv_1... 2024-08-19T01:31:21.772 DEBUG:teuthology.orchestra.run.smithi167:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833 ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid bad8780e-5dc9-11ef-bccf-c7b262605968 -- lvm zap /dev/vg_nvme/lv_1 2024-08-19T01:31:21.800 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:31:21 smithi167 ceph-mon[21875]: pgmap v216: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 894 GiB / 894 GiB avail 2024-08-19T01:31:21.800 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:31:21 smithi167 ceph-mon[21875]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' 2024-08-19T01:31:21.800 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:31:21 smithi167 ceph-mon[21875]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' 2024-08-19T01:31:21.800 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:31:21 smithi167 ceph-mon[21875]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-19T01:31:21.800 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:31:21 smithi167 ceph-mon[21875]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-19T01:31:21.801 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:31:21 smithi167 ceph-mon[21875]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' 2024-08-19T01:31:21.801 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:31:21 smithi167 ceph-mon[21875]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-19T01:31:21.801 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:31:21 smithi167 ceph-mon[21875]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' 2024-08-19T01:31:21.801 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:31:21 smithi167 ceph-mon[21875]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' 2024-08-19T01:31:21.905 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:31:21 smithi083 ceph-mon[17120]: pgmap v216: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 894 GiB / 894 GiB avail 2024-08-19T01:31:21.905 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:31:21 smithi083 ceph-mon[17120]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' 2024-08-19T01:31:21.905 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:31:21 smithi083 ceph-mon[17120]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' 2024-08-19T01:31:21.905 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:31:21 smithi083 ceph-mon[17120]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-19T01:31:21.905 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:31:21 smithi083 ceph-mon[17120]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-19T01:31:21.905 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:31:21 smithi083 ceph-mon[17120]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' 2024-08-19T01:31:21.905 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:31:21 smithi083 ceph-mon[17120]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-19T01:31:21.905 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:31:21 smithi083 ceph-mon[17120]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' 2024-08-19T01:31:21.905 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:31:21 smithi083 ceph-mon[17120]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' 2024-08-19T01:31:22.032 INFO:teuthology.orchestra.run.smithi167.stderr:Inferring config /var/lib/ceph/bad8780e-5dc9-11ef-bccf-c7b262605968/mon.c/config 2024-08-19T01:31:22.034 INFO:journalctl@ceph.osd.10.smithi117.stdout:Aug 19 01:31:21 smithi117 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-10[46616]: 2024-08-19T01:31:21.827+0000 7f34ba65a640 -1 osd.10 0 log_to_monitors true 2024-08-19T01:31:22.716 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:31:22 smithi167 ceph-mon[21875]: from='osd.10 [v2:172.21.15.117:6824/1183570837,v1:172.21.15.117:6825/1183570837]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-08-19T01:31:22.716 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:31:22 smithi167 ceph-mon[21875]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-08-19T01:31:22.815 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:31:22 smithi117 ceph-mon[22016]: from='osd.10 [v2:172.21.15.117:6824/1183570837,v1:172.21.15.117:6825/1183570837]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-08-19T01:31:22.815 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:31:22 smithi117 ceph-mon[22016]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-08-19T01:31:22.905 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:31:22 smithi083 ceph-mon[17120]: from='osd.10 [v2:172.21.15.117:6824/1183570837,v1:172.21.15.117:6825/1183570837]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-08-19T01:31:22.905 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:31:22 smithi083 ceph-mon[17120]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-08-19T01:31:23.685 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:31:23 smithi167 ceph-mon[21875]: pgmap v217: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 894 GiB / 894 GiB avail 2024-08-19T01:31:23.686 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:31:23 smithi167 ceph-mon[21875]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-08-19T01:31:23.686 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:31:23 smithi167 ceph-mon[21875]: from='osd.10 [v2:172.21.15.117:6824/1183570837,v1:172.21.15.117:6825/1183570837]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi117", "root=default"]}]: dispatch 2024-08-19T01:31:23.686 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:31:23 smithi167 ceph-mon[21875]: osdmap e69: 11 total, 10 up, 11 in 2024-08-19T01:31:23.686 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:31:23 smithi167 ceph-mon[21875]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-08-19T01:31:23.686 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:31:23 smithi167 ceph-mon[21875]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi117", "root=default"]}]: dispatch 2024-08-19T01:31:23.855 INFO:journalctl@ceph.osd.10.smithi117.stdout:Aug 19 01:31:23 smithi117 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-10[46616]: 2024-08-19T01:31:23.495+0000 7f34b5dba640 -1 osd.10 0 waiting for initial osdmap 2024-08-19T01:31:23.856 INFO:journalctl@ceph.osd.10.smithi117.stdout:Aug 19 01:31:23 smithi117 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-10[46616]: 2024-08-19T01:31:23.507+0000 7f34b23f4640 -1 osd.10 70 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-08-19T01:31:23.856 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:31:23 smithi117 ceph-mon[22016]: pgmap v217: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 894 GiB / 894 GiB avail 2024-08-19T01:31:23.856 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:31:23 smithi117 ceph-mon[22016]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-08-19T01:31:23.856 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:31:23 smithi117 ceph-mon[22016]: from='osd.10 [v2:172.21.15.117:6824/1183570837,v1:172.21.15.117:6825/1183570837]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi117", "root=default"]}]: dispatch 2024-08-19T01:31:23.856 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:31:23 smithi117 ceph-mon[22016]: osdmap e69: 11 total, 10 up, 11 in 2024-08-19T01:31:23.856 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:31:23 smithi117 ceph-mon[22016]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-08-19T01:31:23.856 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:31:23 smithi117 ceph-mon[22016]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi117", "root=default"]}]: dispatch 2024-08-19T01:31:23.905 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:31:23 smithi083 ceph-mon[17120]: pgmap v217: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 894 GiB / 894 GiB avail 2024-08-19T01:31:23.905 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:31:23 smithi083 ceph-mon[17120]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-08-19T01:31:23.905 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:31:23 smithi083 ceph-mon[17120]: from='osd.10 [v2:172.21.15.117:6824/1183570837,v1:172.21.15.117:6825/1183570837]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi117", "root=default"]}]: dispatch 2024-08-19T01:31:23.905 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:31:23 smithi083 ceph-mon[17120]: osdmap e69: 11 total, 10 up, 11 in 2024-08-19T01:31:23.905 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:31:23 smithi083 ceph-mon[17120]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-08-19T01:31:23.905 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:31:23 smithi083 ceph-mon[17120]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi117", "root=default"]}]: dispatch 2024-08-19T01:31:24.346 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-08-19T01:31:24.371 DEBUG:teuthology.orchestra.run.smithi167:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid bad8780e-5dc9-11ef-bccf-c7b262605968 -- ceph orch daemon add osd smithi167:vg_nvme/lv_1 2024-08-19T01:31:24.667 INFO:teuthology.orchestra.run.smithi167.stderr:Inferring config /var/lib/ceph/bad8780e-5dc9-11ef-bccf-c7b262605968/mon.c/config 2024-08-19T01:31:24.716 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:31:24 smithi167 ceph-mon[21875]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi117", "root=default"]}]': finished 2024-08-19T01:31:24.716 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:31:24 smithi167 ceph-mon[21875]: osdmap e70: 11 total, 10 up, 11 in 2024-08-19T01:31:24.716 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:31:24 smithi167 ceph-mon[21875]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-08-19T01:31:24.717 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:31:24 smithi167 ceph-mon[21875]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-08-19T01:31:24.717 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:31:24 smithi167 ceph-mon[21875]: osd.10 [v2:172.21.15.117:6824/1183570837,v1:172.21.15.117:6825/1183570837] boot 2024-08-19T01:31:24.717 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:31:24 smithi167 ceph-mon[21875]: osdmap e71: 11 total, 11 up, 11 in 2024-08-19T01:31:24.717 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:31:24 smithi167 ceph-mon[21875]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-08-19T01:31:24.905 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:31:24 smithi083 ceph-mon[17120]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi117", "root=default"]}]': finished 2024-08-19T01:31:24.905 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:31:24 smithi083 ceph-mon[17120]: osdmap e70: 11 total, 10 up, 11 in 2024-08-19T01:31:24.905 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:31:24 smithi083 ceph-mon[17120]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-08-19T01:31:24.905 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:31:24 smithi083 ceph-mon[17120]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-08-19T01:31:24.905 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:31:24 smithi083 ceph-mon[17120]: osd.10 [v2:172.21.15.117:6824/1183570837,v1:172.21.15.117:6825/1183570837] boot 2024-08-19T01:31:24.905 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:31:24 smithi083 ceph-mon[17120]: osdmap e71: 11 total, 11 up, 11 in 2024-08-19T01:31:24.905 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:31:24 smithi083 ceph-mon[17120]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-08-19T01:31:24.952 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:31:24 smithi117 ceph-mon[22016]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi117", "root=default"]}]': finished 2024-08-19T01:31:24.953 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:31:24 smithi117 ceph-mon[22016]: osdmap e70: 11 total, 10 up, 11 in 2024-08-19T01:31:24.953 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:31:24 smithi117 ceph-mon[22016]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-08-19T01:31:24.953 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:31:24 smithi117 ceph-mon[22016]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-08-19T01:31:24.953 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:31:24 smithi117 ceph-mon[22016]: osd.10 [v2:172.21.15.117:6824/1183570837,v1:172.21.15.117:6825/1183570837] boot 2024-08-19T01:31:24.953 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:31:24 smithi117 ceph-mon[22016]: osdmap e71: 11 total, 11 up, 11 in 2024-08-19T01:31:24.953 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:31:24 smithi117 ceph-mon[22016]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-08-19T01:31:25.140 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-19T01:31:25.123+0000 7fafc94c2640 1 -- 172.21.15.167:0/295173833 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fafc4104d20 msgr2=0x7fafc4107110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:31:25.140 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-19T01:31:25.123+0000 7fafc94c2640 1 --2- 172.21.15.167:0/295173833 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fafc4104d20 0x7fafc4107110 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fafac009900 tx=0x7fafac02a410 comp rx=0 tx=0).stop 2024-08-19T01:31:25.140 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-19T01:31:25.123+0000 7fafc94c2640 1 -- 172.21.15.167:0/295173833 shutdown_connections 2024-08-19T01:31:25.140 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-19T01:31:25.123+0000 7fafc94c2640 1 --2- 172.21.15.167:0/295173833 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fafc4107650 0x7fafc4109a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:31:25.140 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-19T01:31:25.123+0000 7fafc94c2640 1 --2- 172.21.15.167:0/295173833 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fafc4104d20 0x7fafc4107110 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:31:25.141 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-19T01:31:25.123+0000 7fafc94c2640 1 --2- 172.21.15.167:0/295173833 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fafc41023f0 0x7fafc41047e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:31:25.141 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-19T01:31:25.123+0000 7fafc94c2640 1 -- 172.21.15.167:0/295173833 >> 172.21.15.167:0/295173833 conn(0x7fafc40fbfb0 msgr2=0x7fafc40fe3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:31:25.141 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-19T01:31:25.123+0000 7fafc94c2640 1 -- 172.21.15.167:0/295173833 shutdown_connections 2024-08-19T01:31:25.141 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-19T01:31:25.123+0000 7fafc94c2640 1 -- 172.21.15.167:0/295173833 wait complete. 2024-08-19T01:31:25.141 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-19T01:31:25.124+0000 7fafc94c2640 1 Processor -- start 2024-08-19T01:31:25.141 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-19T01:31:25.124+0000 7fafc94c2640 1 -- start start 2024-08-19T01:31:25.142 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-19T01:31:25.124+0000 7fafc94c2640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fafc41023f0 0x7fafc4197be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:31:25.142 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-19T01:31:25.124+0000 7fafc94c2640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fafc4104d20 0x7fafc4198120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:31:25.142 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-19T01:31:25.124+0000 7fafc94c2640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fafc4107650 0x7fafc419f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:31:25.142 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-19T01:31:25.124+0000 7fafc94c2640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fafc410d560 con 0x7fafc4104d20 2024-08-19T01:31:25.142 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-19T01:31:25.124+0000 7fafc94c2640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fafc410d3e0 con 0x7fafc4107650 2024-08-19T01:31:25.143 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-19T01:31:25.124+0000 7fafc94c2640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fafc410d6e0 con 0x7fafc41023f0 2024-08-19T01:31:25.143 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-19T01:31:25.124+0000 7fafc2ffd640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fafc41023f0 0x7fafc4197be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:31:25.143 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-19T01:31:25.124+0000 7fafc2ffd640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fafc41023f0 0x7fafc4197be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:35480/0 (socket says 172.21.15.167:35480) 2024-08-19T01:31:25.143 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-19T01:31:25.124+0000 7fafc2ffd640 1 -- 172.21.15.167:0/2214063705 learned_addr learned my addr 172.21.15.167:0/2214063705 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-08-19T01:31:25.143 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-19T01:31:25.124+0000 7fafc37fe640 1 --2- 172.21.15.167:0/2214063705 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fafc4107650 0x7fafc419f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:31:25.143 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-19T01:31:25.124+0000 7fafc27fc640 1 --2- 172.21.15.167:0/2214063705 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fafc4104d20 0x7fafc4198120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:31:25.144 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-19T01:31:25.125+0000 7fafc2ffd640 1 -- 172.21.15.167:0/2214063705 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fafc4107650 msgr2=0x7fafc419f1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:31:25.144 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-19T01:31:25.125+0000 7fafc2ffd640 1 --2- 172.21.15.167:0/2214063705 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fafc4107650 0x7fafc419f1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:31:25.144 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-19T01:31:25.125+0000 7fafc2ffd640 1 -- 172.21.15.167:0/2214063705 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fafc4104d20 msgr2=0x7fafc4198120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:31:25.144 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-19T01:31:25.125+0000 7fafc2ffd640 1 --2- 172.21.15.167:0/2214063705 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fafc4104d20 0x7fafc4198120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:31:25.145 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-19T01:31:25.125+0000 7fafc2ffd640 1 -- 172.21.15.167:0/2214063705 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fafc419f810 con 0x7fafc41023f0 2024-08-19T01:31:25.145 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-19T01:31:25.125+0000 7fafc2ffd640 1 --2- 172.21.15.167:0/2214063705 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fafc41023f0 0x7fafc4197be0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fafb400c890 tx=0x7fafb400cd50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:31:25.145 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-19T01:31:25.125+0000 7fafbbfff640 1 -- 172.21.15.167:0/2214063705 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafb4004220 con 0x7fafc41023f0 2024-08-19T01:31:25.145 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-19T01:31:25.125+0000 7fafbbfff640 1 -- 172.21.15.167:0/2214063705 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fafb40043c0 con 0x7fafc41023f0 2024-08-19T01:31:25.145 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-19T01:31:25.125+0000 7fafbbfff640 1 -- 172.21.15.167:0/2214063705 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafb40049e0 con 0x7fafc41023f0 2024-08-19T01:31:25.146 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-19T01:31:25.125+0000 7fafc94c2640 1 -- 172.21.15.167:0/2214063705 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fafc419fb00 con 0x7fafc41023f0 2024-08-19T01:31:25.146 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-19T01:31:25.125+0000 7fafc94c2640 1 -- 172.21.15.167:0/2214063705 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fafc41a0060 con 0x7fafc41023f0 2024-08-19T01:31:25.146 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-19T01:31:25.126+0000 7fafc94c2640 1 -- 172.21.15.167:0/2214063705 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fafc40fdaa0 con 0x7fafc41023f0 2024-08-19T01:31:25.147 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-19T01:31:25.128+0000 7fafbbfff640 1 -- 172.21.15.167:0/2214063705 <== mon.1 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7fafb40079b0 con 0x7fafc41023f0 2024-08-19T01:31:25.147 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-19T01:31:25.129+0000 7fafbbfff640 1 --2- 172.21.15.167:0/2214063705 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7fafb00a2c30 0x7fafb00a50f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:31:25.147 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-19T01:31:25.129+0000 7fafbbfff640 1 -- 172.21.15.167:0/2214063705 <== mon.1 v2:172.21.15.167:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 6723+0+0 (secure 0 0 0) 0x7fafb40ce920 con 0x7fafc41023f0 2024-08-19T01:31:25.148 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-19T01:31:25.129+0000 7fafc27fc640 1 --2- 172.21.15.167:0/2214063705 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7fafb00a2c30 0x7fafb00a50f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:31:25.148 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-19T01:31:25.130+0000 7fafc27fc640 1 --2- 172.21.15.167:0/2214063705 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7fafb00a2c30 0x7fafb00a50f0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fafac005ab0 tx=0x7fafac000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:31:25.148 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-19T01:31:25.133+0000 7fafbbfff640 1 -- 172.21.15.167:0/2214063705 <== mon.1 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7fafb4014030 con 0x7fafc41023f0 2024-08-19T01:31:25.318 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-19T01:31:25.316+0000 7fafc94c2640 1 -- 172.21.15.167:0/2214063705 --> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi167:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7fafc4062360 con 0x7fafb00a2c30 2024-08-19T01:31:25.607 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:31:25 smithi167 ceph-mon[21875]: purged_snaps scrub starts 2024-08-19T01:31:25.607 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:31:25 smithi167 ceph-mon[21875]: purged_snaps scrub ok 2024-08-19T01:31:25.607 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:31:25 smithi167 ceph-mon[21875]: pgmap v221: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 894 GiB / 894 GiB avail 2024-08-19T01:31:25.607 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:31:25 smithi167 ceph-mon[21875]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' 2024-08-19T01:31:25.607 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:31:25 smithi167 ceph-mon[21875]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' 2024-08-19T01:31:25.607 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:31:25 smithi167 ceph-mon[21875]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-08-19T01:31:25.608 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:31:25 smithi167 ceph-mon[21875]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-08-19T01:31:25.608 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:31:25 smithi167 ceph-mon[21875]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-08-19T01:31:25.608 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:31:25 smithi167 ceph-mon[21875]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-08-19T01:31:25.608 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:31:25 smithi167 ceph-mon[21875]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' 2024-08-19T01:31:25.608 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:31:25 smithi167 ceph-mon[21875]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-19T01:31:25.608 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:31:25 smithi167 ceph-mon[21875]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-19T01:31:25.608 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:31:25 smithi167 ceph-mon[21875]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' 2024-08-19T01:31:25.608 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:31:25 smithi167 ceph-mon[21875]: osdmap e72: 11 total, 11 up, 11 in 2024-08-19T01:31:25.608 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:31:25 smithi167 ceph-mon[21875]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-08-19T01:31:25.608 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:31:25 smithi167 ceph-mon[21875]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-08-19T01:31:25.608 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:31:25 smithi167 ceph-mon[21875]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-19T01:31:25.905 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:31:25 smithi083 ceph-mon[17120]: purged_snaps scrub starts 2024-08-19T01:31:25.905 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:31:25 smithi083 ceph-mon[17120]: purged_snaps scrub ok 2024-08-19T01:31:25.905 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:31:25 smithi083 ceph-mon[17120]: pgmap v221: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 894 GiB / 894 GiB avail 2024-08-19T01:31:25.905 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:31:25 smithi083 ceph-mon[17120]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' 2024-08-19T01:31:25.905 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:31:25 smithi083 ceph-mon[17120]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' 2024-08-19T01:31:25.905 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:31:25 smithi083 ceph-mon[17120]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-08-19T01:31:25.905 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:31:25 smithi083 ceph-mon[17120]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-08-19T01:31:25.905 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:31:25 smithi083 ceph-mon[17120]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-08-19T01:31:25.905 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:31:25 smithi083 ceph-mon[17120]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-08-19T01:31:25.905 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:31:25 smithi083 ceph-mon[17120]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' 2024-08-19T01:31:25.905 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:31:25 smithi083 ceph-mon[17120]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-19T01:31:25.906 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:31:25 smithi083 ceph-mon[17120]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-19T01:31:25.906 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:31:25 smithi083 ceph-mon[17120]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' 2024-08-19T01:31:25.906 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:31:25 smithi083 ceph-mon[17120]: osdmap e72: 11 total, 11 up, 11 in 2024-08-19T01:31:25.906 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:31:25 smithi083 ceph-mon[17120]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-08-19T01:31:25.906 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:31:25 smithi083 ceph-mon[17120]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-08-19T01:31:25.906 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:31:25 smithi083 ceph-mon[17120]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-19T01:31:25.952 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:31:25 smithi117 ceph-mon[22016]: purged_snaps scrub starts 2024-08-19T01:31:25.953 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:31:25 smithi117 ceph-mon[22016]: purged_snaps scrub ok 2024-08-19T01:31:25.953 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:31:25 smithi117 ceph-mon[22016]: pgmap v221: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 894 GiB / 894 GiB avail 2024-08-19T01:31:25.953 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:31:25 smithi117 ceph-mon[22016]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' 2024-08-19T01:31:25.953 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:31:25 smithi117 ceph-mon[22016]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' 2024-08-19T01:31:25.953 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:31:25 smithi117 ceph-mon[22016]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-08-19T01:31:25.953 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:31:25 smithi117 ceph-mon[22016]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-08-19T01:31:25.953 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:31:25 smithi117 ceph-mon[22016]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-08-19T01:31:25.953 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:31:25 smithi117 ceph-mon[22016]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-08-19T01:31:25.953 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:31:25 smithi117 ceph-mon[22016]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' 2024-08-19T01:31:25.953 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:31:25 smithi117 ceph-mon[22016]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-19T01:31:25.953 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:31:25 smithi117 ceph-mon[22016]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-19T01:31:25.954 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:31:25 smithi117 ceph-mon[22016]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' 2024-08-19T01:31:25.954 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:31:25 smithi117 ceph-mon[22016]: osdmap e72: 11 total, 11 up, 11 in 2024-08-19T01:31:25.954 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:31:25 smithi117 ceph-mon[22016]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-08-19T01:31:25.954 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:31:25 smithi117 ceph-mon[22016]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-08-19T01:31:25.954 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:31:25 smithi117 ceph-mon[22016]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-19T01:31:26.716 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:31:26 smithi167 ceph-mon[21875]: Detected new or changed devices on smithi117 2024-08-19T01:31:26.716 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:31:26 smithi167 ceph-mon[21875]: Adjusting osd_memory_target on smithi117 to 4272M 2024-08-19T01:31:26.716 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:31:26 smithi167 ceph-mon[21875]: from='client.24367 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi167:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-08-19T01:31:26.716 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:31:26 smithi167 ceph-mon[21875]: osdmap e73: 11 total, 11 up, 11 in 2024-08-19T01:31:26.905 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:31:26 smithi083 ceph-mon[17120]: Detected new or changed devices on smithi117 2024-08-19T01:31:26.905 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:31:26 smithi083 ceph-mon[17120]: Adjusting osd_memory_target on smithi117 to 4272M 2024-08-19T01:31:26.905 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:31:26 smithi083 ceph-mon[17120]: from='client.24367 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi167:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-08-19T01:31:26.905 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:31:26 smithi083 ceph-mon[17120]: osdmap e73: 11 total, 11 up, 11 in 2024-08-19T01:31:26.952 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:31:26 smithi117 ceph-mon[22016]: Detected new or changed devices on smithi117 2024-08-19T01:31:26.953 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:31:26 smithi117 ceph-mon[22016]: Adjusting osd_memory_target on smithi117 to 4272M 2024-08-19T01:31:26.953 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:31:26 smithi117 ceph-mon[22016]: from='client.24367 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi167:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-08-19T01:31:26.953 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:31:26 smithi117 ceph-mon[22016]: osdmap e73: 11 total, 11 up, 11 in 2024-08-19T01:31:27.904 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:31:27 smithi083 ceph-mon[17120]: pgmap v224: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 983 GiB / 983 GiB avail 2024-08-19T01:31:27.905 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:31:27 smithi083 ceph-mon[17120]: osdmap e74: 11 total, 11 up, 11 in 2024-08-19T01:31:27.938 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:31:27 smithi167 ceph-mon[21875]: pgmap v224: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 983 GiB / 983 GiB avail 2024-08-19T01:31:27.938 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:31:27 smithi167 ceph-mon[21875]: osdmap e74: 11 total, 11 up, 11 in 2024-08-19T01:31:27.952 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:31:27 smithi117 ceph-mon[22016]: pgmap v224: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 983 GiB / 983 GiB avail 2024-08-19T01:31:27.952 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:31:27 smithi117 ceph-mon[22016]: osdmap e74: 11 total, 11 up, 11 in 2024-08-19T01:31:28.904 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:31:28 smithi083 ceph-mon[17120]: from='client.? 172.21.15.167:0/1981997754' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "13345f05-cdc0-4dae-a58b-551abbec1915"}]: dispatch 2024-08-19T01:31:28.905 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:31:28 smithi083 ceph-mon[17120]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "13345f05-cdc0-4dae-a58b-551abbec1915"}]: dispatch 2024-08-19T01:31:28.905 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:31:28 smithi083 ceph-mon[17120]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "13345f05-cdc0-4dae-a58b-551abbec1915"}]': finished 2024-08-19T01:31:28.905 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:31:28 smithi083 ceph-mon[17120]: osdmap e75: 12 total, 11 up, 12 in 2024-08-19T01:31:28.905 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:31:28 smithi083 ceph-mon[17120]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-08-19T01:31:28.905 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:31:28 smithi083 ceph-mon[17120]: from='client.? 172.21.15.167:0/1990933119' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-08-19T01:31:28.952 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:31:28 smithi117 ceph-mon[22016]: from='client.? 172.21.15.167:0/1981997754' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "13345f05-cdc0-4dae-a58b-551abbec1915"}]: dispatch 2024-08-19T01:31:28.952 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:31:28 smithi117 ceph-mon[22016]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "13345f05-cdc0-4dae-a58b-551abbec1915"}]: dispatch 2024-08-19T01:31:28.953 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:31:28 smithi117 ceph-mon[22016]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "13345f05-cdc0-4dae-a58b-551abbec1915"}]': finished 2024-08-19T01:31:28.953 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:31:28 smithi117 ceph-mon[22016]: osdmap e75: 12 total, 11 up, 12 in 2024-08-19T01:31:28.953 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:31:28 smithi117 ceph-mon[22016]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-08-19T01:31:28.953 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:31:28 smithi117 ceph-mon[22016]: from='client.? 172.21.15.167:0/1990933119' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-08-19T01:31:28.966 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:31:28 smithi167 ceph-mon[21875]: from='client.? 172.21.15.167:0/1981997754' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "13345f05-cdc0-4dae-a58b-551abbec1915"}]: dispatch 2024-08-19T01:31:28.966 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:31:28 smithi167 ceph-mon[21875]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "13345f05-cdc0-4dae-a58b-551abbec1915"}]: dispatch 2024-08-19T01:31:28.966 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:31:28 smithi167 ceph-mon[21875]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "13345f05-cdc0-4dae-a58b-551abbec1915"}]': finished 2024-08-19T01:31:28.966 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:31:28 smithi167 ceph-mon[21875]: osdmap e75: 12 total, 11 up, 12 in 2024-08-19T01:31:28.966 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:31:28 smithi167 ceph-mon[21875]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-08-19T01:31:28.966 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:31:28 smithi167 ceph-mon[21875]: from='client.? 172.21.15.167:0/1990933119' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-08-19T01:31:29.904 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:31:29 smithi083 ceph-mon[17120]: pgmap v227: 1 pgs: 1 peering; 577 KiB data, 328 MiB used, 983 GiB / 983 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-08-19T01:31:29.952 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:31:29 smithi117 ceph-mon[22016]: pgmap v227: 1 pgs: 1 peering; 577 KiB data, 328 MiB used, 983 GiB / 983 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-08-19T01:31:29.966 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:31:29 smithi167 ceph-mon[21875]: pgmap v227: 1 pgs: 1 peering; 577 KiB data, 328 MiB used, 983 GiB / 983 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-08-19T01:31:30.904 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:31:30 smithi083 ceph-mon[17120]: pgmap v228: 1 pgs: 1 peering; 577 KiB data, 329 MiB used, 983 GiB / 983 GiB avail; 103 KiB/s, 0 objects/s recovering 2024-08-19T01:31:30.952 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:31:30 smithi117 ceph-mon[22016]: pgmap v228: 1 pgs: 1 peering; 577 KiB data, 329 MiB used, 983 GiB / 983 GiB avail; 103 KiB/s, 0 objects/s recovering 2024-08-19T01:31:30.966 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:31:30 smithi167 ceph-mon[21875]: pgmap v228: 1 pgs: 1 peering; 577 KiB data, 329 MiB used, 983 GiB / 983 GiB avail; 103 KiB/s, 0 objects/s recovering 2024-08-19T01:31:33.649 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:31:33 smithi167 ceph-mon[21875]: pgmap v229: 1 pgs: 1 peering; 577 KiB data, 329 MiB used, 983 GiB / 983 GiB avail; 87 KiB/s, 0 objects/s recovering 2024-08-19T01:31:33.654 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:31:33 smithi083 ceph-mon[17120]: pgmap v229: 1 pgs: 1 peering; 577 KiB data, 329 MiB used, 983 GiB / 983 GiB avail; 87 KiB/s, 0 objects/s recovering 2024-08-19T01:31:33.702 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:31:33 smithi117 ceph-mon[22016]: pgmap v229: 1 pgs: 1 peering; 577 KiB data, 329 MiB used, 983 GiB / 983 GiB avail; 87 KiB/s, 0 objects/s recovering 2024-08-19T01:31:35.655 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:31:35 smithi083 ceph-mon[17120]: pgmap v230: 1 pgs: 1 active+clean; 577 KiB data, 329 MiB used, 983 GiB / 983 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-08-19T01:31:35.655 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:31:35 smithi083 ceph-mon[17120]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-08-19T01:31:35.655 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:31:35 smithi083 ceph-mon[17120]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-19T01:31:35.671 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:31:35 smithi167 ceph-mon[21875]: pgmap v230: 1 pgs: 1 active+clean; 577 KiB data, 329 MiB used, 983 GiB / 983 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-08-19T01:31:35.672 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:31:35 smithi167 ceph-mon[21875]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-08-19T01:31:35.672 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:31:35 smithi167 ceph-mon[21875]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-19T01:31:35.702 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:31:35 smithi117 ceph-mon[22016]: pgmap v230: 1 pgs: 1 active+clean; 577 KiB data, 329 MiB used, 983 GiB / 983 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-08-19T01:31:35.702 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:31:35 smithi117 ceph-mon[22016]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-08-19T01:31:35.702 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:31:35 smithi117 ceph-mon[22016]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-19T01:31:36.702 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:31:36 smithi117 ceph-mon[22016]: Deploying daemon osd.11 on smithi167 2024-08-19T01:31:36.716 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:31:36 smithi167 ceph-mon[21875]: Deploying daemon osd.11 on smithi167 2024-08-19T01:31:36.904 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:31:36 smithi083 ceph-mon[17120]: Deploying daemon osd.11 on smithi167 2024-08-19T01:31:37.647 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:31:37 smithi167 ceph-mon[21875]: pgmap v231: 1 pgs: 1 active+clean; 577 KiB data, 329 MiB used, 983 GiB / 983 GiB avail; 60 KiB/s, 0 objects/s recovering 2024-08-19T01:31:37.702 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:31:37 smithi117 ceph-mon[22016]: pgmap v231: 1 pgs: 1 active+clean; 577 KiB data, 329 MiB used, 983 GiB / 983 GiB avail; 60 KiB/s, 0 objects/s recovering 2024-08-19T01:31:37.905 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:31:37 smithi083 ceph-mon[17120]: pgmap v231: 1 pgs: 1 active+clean; 577 KiB data, 329 MiB used, 983 GiB / 983 GiB avail; 60 KiB/s, 0 objects/s recovering 2024-08-19T01:31:39.466 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:31:39 smithi167 ceph-mon[21875]: pgmap v232: 1 pgs: 1 active+clean; 577 KiB data, 329 MiB used, 983 GiB / 983 GiB avail 2024-08-19T01:31:39.702 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:31:39 smithi117 ceph-mon[22016]: pgmap v232: 1 pgs: 1 active+clean; 577 KiB data, 329 MiB used, 983 GiB / 983 GiB avail 2024-08-19T01:31:39.905 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:31:39 smithi083 ceph-mon[17120]: pgmap v232: 1 pgs: 1 active+clean; 577 KiB data, 329 MiB used, 983 GiB / 983 GiB avail 2024-08-19T01:31:40.702 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:31:40 smithi117 ceph-mon[22016]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-19T01:31:40.703 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:31:40 smithi117 ceph-mon[22016]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' 2024-08-19T01:31:40.703 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:31:40 smithi117 ceph-mon[22016]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' 2024-08-19T01:31:40.716 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:31:40 smithi167 ceph-mon[21875]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-19T01:31:40.716 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:31:40 smithi167 ceph-mon[21875]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' 2024-08-19T01:31:40.716 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:31:40 smithi167 ceph-mon[21875]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' 2024-08-19T01:31:40.904 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:31:40 smithi083 ceph-mon[17120]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-19T01:31:40.905 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:31:40 smithi083 ceph-mon[17120]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' 2024-08-19T01:31:40.905 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:31:40 smithi083 ceph-mon[17120]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' 2024-08-19T01:31:41.492 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:31:41 smithi167 ceph-mon[21875]: pgmap v233: 1 pgs: 1 active+clean; 577 KiB data, 329 MiB used, 983 GiB / 983 GiB avail 2024-08-19T01:31:41.702 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:31:41 smithi117 ceph-mon[22016]: pgmap v233: 1 pgs: 1 active+clean; 577 KiB data, 329 MiB used, 983 GiB / 983 GiB avail 2024-08-19T01:31:41.905 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:31:41 smithi083 ceph-mon[17120]: pgmap v233: 1 pgs: 1 active+clean; 577 KiB data, 329 MiB used, 983 GiB / 983 GiB avail 2024-08-19T01:31:43.040 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-19T01:31:43.037+0000 7fafbbfff640 1 -- 172.21.15.167:0/2214063705 <== mgr.14150 v2:172.21.15.83:6800/683740916 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+38 (secure 0 0 0) 0x7fafc4062360 con 0x7fafb00a2c30 2024-08-19T01:31:43.040 INFO:teuthology.orchestra.run.smithi167.stdout:Created osd(s) 11 on host 'smithi167' 2024-08-19T01:31:43.042 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-19T01:31:43.039+0000 7fafc94c2640 1 -- 172.21.15.167:0/2214063705 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7fafb00a2c30 msgr2=0x7fafb00a50f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:31:43.042 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-19T01:31:43.039+0000 7fafc94c2640 1 --2- 172.21.15.167:0/2214063705 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7fafb00a2c30 0x7fafb00a50f0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fafac005ab0 tx=0x7fafac000bc0 comp rx=0 tx=0).stop 2024-08-19T01:31:43.042 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-19T01:31:43.040+0000 7fafc94c2640 1 -- 172.21.15.167:0/2214063705 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fafc41023f0 msgr2=0x7fafc4197be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:31:43.042 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-19T01:31:43.040+0000 7fafc94c2640 1 --2- 172.21.15.167:0/2214063705 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fafc41023f0 0x7fafc4197be0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fafb400c890 tx=0x7fafb400cd50 comp rx=0 tx=0).stop 2024-08-19T01:31:43.043 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-19T01:31:43.040+0000 7fafc94c2640 1 -- 172.21.15.167:0/2214063705 shutdown_connections 2024-08-19T01:31:43.043 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-19T01:31:43.040+0000 7fafc94c2640 1 --2- 172.21.15.167:0/2214063705 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7fafb00a2c30 0x7fafb00a50f0 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:31:43.043 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-19T01:31:43.040+0000 7fafc94c2640 1 --2- 172.21.15.167:0/2214063705 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fafc4107650 0x7fafc419f1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:31:43.043 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-19T01:31:43.040+0000 7fafc94c2640 1 --2- 172.21.15.167:0/2214063705 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fafc4104d20 0x7fafc4198120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:31:43.043 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-19T01:31:43.040+0000 7fafc94c2640 1 --2- 172.21.15.167:0/2214063705 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fafc41023f0 0x7fafc4197be0 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:31:43.043 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-19T01:31:43.040+0000 7fafc94c2640 1 -- 172.21.15.167:0/2214063705 >> 172.21.15.167:0/2214063705 conn(0x7fafc40fbfb0 msgr2=0x7fafc4102c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:31:43.043 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-19T01:31:43.040+0000 7fafc94c2640 1 -- 172.21.15.167:0/2214063705 shutdown_connections 2024-08-19T01:31:43.044 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-19T01:31:43.040+0000 7fafc94c2640 1 -- 172.21.15.167:0/2214063705 wait complete. 2024-08-19T01:31:43.607 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:31:43 smithi167 ceph-mon[21875]: pgmap v234: 1 pgs: 1 active+clean; 577 KiB data, 329 MiB used, 983 GiB / 983 GiB avail 2024-08-19T01:31:43.607 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:31:43 smithi167 ceph-mon[21875]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' 2024-08-19T01:31:43.608 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:31:43 smithi167 ceph-mon[21875]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' 2024-08-19T01:31:43.702 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:31:43 smithi117 ceph-mon[22016]: pgmap v234: 1 pgs: 1 active+clean; 577 KiB data, 329 MiB used, 983 GiB / 983 GiB avail 2024-08-19T01:31:43.703 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:31:43 smithi117 ceph-mon[22016]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' 2024-08-19T01:31:43.703 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:31:43 smithi117 ceph-mon[22016]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' 2024-08-19T01:31:43.903 DEBUG:teuthology.orchestra.run.smithi167:osd.11> sudo journalctl -f -n 0 -u ceph-bad8780e-5dc9-11ef-bccf-c7b262605968@osd.11.service 2024-08-19T01:31:43.904 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:31:43 smithi083 ceph-mon[17120]: pgmap v234: 1 pgs: 1 active+clean; 577 KiB data, 329 MiB used, 983 GiB / 983 GiB avail 2024-08-19T01:31:43.904 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:31:43 smithi083 ceph-mon[17120]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' 2024-08-19T01:31:43.905 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:31:43 smithi083 ceph-mon[17120]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' 2024-08-19T01:31:43.906 INFO:tasks.cephadm:Waiting for 12 OSDs to come up... 2024-08-19T01:31:43.906 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid bad8780e-5dc9-11ef-bccf-c7b262605968 -- ceph osd stat -f json 2024-08-19T01:31:44.216 INFO:journalctl@ceph.osd.11.smithi167.stdout:Aug 19 01:31:44 smithi167 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-11[46643]: 2024-08-19T01:31:44.055+0000 7f448a08f640 -1 osd.11 0 log_to_monitors true 2024-08-19T01:31:44.291 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/bad8780e-5dc9-11ef-bccf-c7b262605968/mon.a/config 2024-08-19T01:31:44.683 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:31:44 smithi083 ceph-mon[17120]: from='osd.11 [v2:172.21.15.167:6824/187027350,v1:172.21.15.167:6825/187027350]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-08-19T01:31:44.683 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:31:44 smithi083 ceph-mon[17120]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-08-19T01:31:44.683 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:31:44 smithi083 ceph-mon[17120]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' 2024-08-19T01:31:44.684 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:31:44 smithi083 ceph-mon[17120]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' 2024-08-19T01:31:44.695 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:31:44 smithi167 ceph-mon[21875]: from='osd.11 [v2:172.21.15.167:6824/187027350,v1:172.21.15.167:6825/187027350]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-08-19T01:31:44.695 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:31:44 smithi167 ceph-mon[21875]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-08-19T01:31:44.695 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:31:44 smithi167 ceph-mon[21875]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' 2024-08-19T01:31:44.695 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:31:44 smithi167 ceph-mon[21875]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' 2024-08-19T01:31:44.702 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:31:44 smithi117 ceph-mon[22016]: from='osd.11 [v2:172.21.15.167:6824/187027350,v1:172.21.15.167:6825/187027350]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-08-19T01:31:44.702 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:31:44 smithi117 ceph-mon[22016]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-08-19T01:31:44.703 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:31:44 smithi117 ceph-mon[22016]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' 2024-08-19T01:31:44.703 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:31:44 smithi117 ceph-mon[22016]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' 2024-08-19T01:31:44.914 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:31:44.889+0000 7fed32000640 1 -- 172.21.15.83:0/1488729981 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fed2c102b30 msgr2=0x7fed2c102fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:31:44.915 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:31:44.889+0000 7fed32000640 1 --2- 172.21.15.83:0/1488729981 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fed2c102b30 0x7fed2c102fb0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fed14009900 tx=0x7fed1402a410 comp rx=0 tx=0).stop 2024-08-19T01:31:44.915 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:31:44.889+0000 7fed32000640 1 -- 172.21.15.83:0/1488729981 shutdown_connections 2024-08-19T01:31:44.915 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:31:44.889+0000 7fed32000640 1 --2- 172.21.15.83:0/1488729981 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fed2c1034f0 0x7fed2c109d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:31:44.915 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:31:44.889+0000 7fed32000640 1 --2- 172.21.15.83:0/1488729981 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fed2c102b30 0x7fed2c102fb0 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:31:44.915 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:31:44.889+0000 7fed32000640 1 --2- 172.21.15.83:0/1488729981 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fed2c101930 0x7fed2c101d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:31:44.915 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:31:44.889+0000 7fed32000640 1 -- 172.21.15.83:0/1488729981 >> 172.21.15.83:0/1488729981 conn(0x7fed2c0fd0c0 msgr2=0x7fed2c0ff500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:31:44.915 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:31:44.889+0000 7fed32000640 1 -- 172.21.15.83:0/1488729981 shutdown_connections 2024-08-19T01:31:44.915 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:31:44.889+0000 7fed32000640 1 -- 172.21.15.83:0/1488729981 wait complete. 2024-08-19T01:31:44.915 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:31:44.891+0000 7fed32000640 1 Processor -- start 2024-08-19T01:31:44.915 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:31:44.891+0000 7fed32000640 1 -- start start 2024-08-19T01:31:44.916 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:31:44.891+0000 7fed32000640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fed2c101930 0x7fed2c197c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:31:44.916 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:31:44.891+0000 7fed32000640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fed2c102b30 0x7fed2c198170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:31:44.916 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:31:44.891+0000 7fed32000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fed2c1034f0 0x7fed2c19f1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:31:44.916 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:31:44.891+0000 7fed32000640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fed2c10d590 con 0x7fed2c1034f0 2024-08-19T01:31:44.916 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:31:44.891+0000 7fed32000640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fed2c10d410 con 0x7fed2c101930 2024-08-19T01:31:44.916 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:31:44.891+0000 7fed32000640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fed2c10d710 con 0x7fed2c102b30 2024-08-19T01:31:44.916 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:31:44.891+0000 7fed23e00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fed2c102b30 0x7fed2c198170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:31:44.916 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:31:44.891+0000 7fed23e00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fed2c102b30 0x7fed2c198170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.83:50122/0 (socket says 172.21.15.83:50122) 2024-08-19T01:31:44.917 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:31:44.891+0000 7fed23e00640 1 -- 172.21.15.83:0/952838612 learned_addr learned my addr 172.21.15.83:0/952838612 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-08-19T01:31:44.917 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:31:44.891+0000 7fed2b400640 1 --2- 172.21.15.83:0/952838612 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fed2c101930 0x7fed2c197c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:31:44.917 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:31:44.891+0000 7fed2be00640 1 --2- 172.21.15.83:0/952838612 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fed2c1034f0 0x7fed2c19f1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:31:44.917 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:31:44.892+0000 7fed2be00640 1 -- 172.21.15.83:0/952838612 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fed2c102b30 msgr2=0x7fed2c198170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:31:44.917 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:31:44.892+0000 7fed2be00640 1 --2- 172.21.15.83:0/952838612 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fed2c102b30 0x7fed2c198170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:31:44.917 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:31:44.892+0000 7fed2be00640 1 -- 172.21.15.83:0/952838612 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fed2c101930 msgr2=0x7fed2c197c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:31:44.917 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:31:44.892+0000 7fed2be00640 1 --2- 172.21.15.83:0/952838612 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fed2c101930 0x7fed2c197c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:31:44.917 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:31:44.892+0000 7fed2be00640 1 -- 172.21.15.83:0/952838612 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fed2c19f850 con 0x7fed2c1034f0 2024-08-19T01:31:44.918 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:31:44.892+0000 7fed2b400640 1 --2- 172.21.15.83:0/952838612 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fed2c101930 0x7fed2c197c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-19T01:31:44.918 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:31:44.892+0000 7fed2be00640 1 --2- 172.21.15.83:0/952838612 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fed2c1034f0 0x7fed2c19f1b0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fed2400ec50 tx=0x7fed2400c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:31:44.918 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:31:44.892+0000 7fed28c00640 1 -- 172.21.15.83:0/952838612 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed24003b70 con 0x7fed2c1034f0 2024-08-19T01:31:44.918 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:31:44.892+0000 7fed28c00640 1 -- 172.21.15.83:0/952838612 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fed24003d10 con 0x7fed2c1034f0 2024-08-19T01:31:44.918 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:31:44.892+0000 7fed32000640 1 -- 172.21.15.83:0/952838612 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fed2c19fb40 con 0x7fed2c1034f0 2024-08-19T01:31:44.918 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:31:44.892+0000 7fed28c00640 1 -- 172.21.15.83:0/952838612 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed24010890 con 0x7fed2c1034f0 2024-08-19T01:31:44.918 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:31:44.892+0000 7fed32000640 1 -- 172.21.15.83:0/952838612 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fed2c1a0080 con 0x7fed2c1034f0 2024-08-19T01:31:44.918 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:31:44.893+0000 7fed32000640 1 -- 172.21.15.83:0/952838612 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fed2c108dd0 con 0x7fed2c1034f0 2024-08-19T01:31:44.918 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:31:44.896+0000 7fed28c00640 1 -- 172.21.15.83:0/952838612 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7fed24004b40 con 0x7fed2c1034f0 2024-08-19T01:31:44.918 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:31:44.897+0000 7fed28c00640 1 --2- 172.21.15.83:0/952838612 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7fed100a2c00 0x7fed100a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:31:44.918 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:31:44.897+0000 7fed28c00640 1 -- 172.21.15.83:0/952838612 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 6831+0+0 (secure 0 0 0) 0x7fed240cf090 con 0x7fed2c1034f0 2024-08-19T01:31:44.918 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:31:44.897+0000 7fed2b400640 1 --2- 172.21.15.83:0/952838612 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7fed100a2c00 0x7fed100a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:31:44.919 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:31:44.898+0000 7fed2b400640 1 --2- 172.21.15.83:0/952838612 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7fed100a2c00 0x7fed100a50c0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fed2c199110 tx=0x7fed1c005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:31:44.919 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:31:44.900+0000 7fed28c00640 1 -- 172.21.15.83:0/952838612 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7fed2409c5b0 con 0x7fed2c1034f0 2024-08-19T01:31:45.074 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:31:45.073+0000 7fed32000640 1 -- 172.21.15.83:0/952838612 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) -- 0x7fed2c02d800 con 0x7fed2c1034f0 2024-08-19T01:31:45.075 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:31:45.073+0000 7fed28c00640 1 -- 172.21.15.83:0/952838612 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v76) ==== 74+0+133 (secure 0 0 0) 0x7fed2409beb0 con 0x7fed2c1034f0 2024-08-19T01:31:45.075 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-08-19T01:31:45.078 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:31:45.076+0000 7fed32000640 1 -- 172.21.15.83:0/952838612 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7fed100a2c00 msgr2=0x7fed100a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:31:45.078 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:31:45.076+0000 7fed32000640 1 --2- 172.21.15.83:0/952838612 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7fed100a2c00 0x7fed100a50c0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fed2c199110 tx=0x7fed1c005f90 comp rx=0 tx=0).stop 2024-08-19T01:31:45.078 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:31:45.076+0000 7fed32000640 1 -- 172.21.15.83:0/952838612 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fed2c1034f0 msgr2=0x7fed2c19f1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:31:45.078 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:31:45.076+0000 7fed32000640 1 --2- 172.21.15.83:0/952838612 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fed2c1034f0 0x7fed2c19f1b0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fed2400ec50 tx=0x7fed2400c6a0 comp rx=0 tx=0).stop 2024-08-19T01:31:45.078 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:31:45.076+0000 7fed32000640 1 -- 172.21.15.83:0/952838612 shutdown_connections 2024-08-19T01:31:45.078 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:31:45.076+0000 7fed32000640 1 --2- 172.21.15.83:0/952838612 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7fed100a2c00 0x7fed100a50c0 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:31:45.079 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:31:45.076+0000 7fed32000640 1 --2- 172.21.15.83:0/952838612 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fed2c1034f0 0x7fed2c19f1b0 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:31:45.079 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:31:45.076+0000 7fed32000640 1 --2- 172.21.15.83:0/952838612 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fed2c102b30 0x7fed2c198170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:31:45.079 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:31:45.076+0000 7fed32000640 1 --2- 172.21.15.83:0/952838612 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fed2c101930 0x7fed2c197c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:31:45.079 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:31:45.076+0000 7fed32000640 1 -- 172.21.15.83:0/952838612 >> 172.21.15.83:0/952838612 conn(0x7fed2c0fd0c0 msgr2=0x7fed2c105fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:31:45.079 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:31:45.076+0000 7fed32000640 1 -- 172.21.15.83:0/952838612 shutdown_connections 2024-08-19T01:31:45.079 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:31:45.076+0000 7fed32000640 1 -- 172.21.15.83:0/952838612 wait complete. 2024-08-19T01:31:45.702 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:31:45 smithi117 ceph-mon[22016]: pgmap v235: 1 pgs: 1 active+clean; 577 KiB data, 329 MiB used, 983 GiB / 983 GiB avail 2024-08-19T01:31:45.703 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:31:45 smithi117 ceph-mon[22016]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-08-19T01:31:45.703 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:31:45 smithi117 ceph-mon[22016]: from='osd.11 [v2:172.21.15.167:6824/187027350,v1:172.21.15.167:6825/187027350]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi167", "root=default"]}]: dispatch 2024-08-19T01:31:45.703 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:31:45 smithi117 ceph-mon[22016]: osdmap e76: 12 total, 11 up, 12 in 2024-08-19T01:31:45.703 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:31:45 smithi117 ceph-mon[22016]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-08-19T01:31:45.703 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:31:45 smithi117 ceph-mon[22016]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi167", "root=default"]}]: dispatch 2024-08-19T01:31:45.703 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:31:45 smithi117 ceph-mon[22016]: from='client.? 172.21.15.83:0/952838612' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-08-19T01:31:45.703 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:31:45 smithi117 ceph-mon[22016]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi167", "root=default"]}]': finished 2024-08-19T01:31:45.703 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:31:45 smithi117 ceph-mon[22016]: osdmap e77: 12 total, 11 up, 12 in 2024-08-19T01:31:45.716 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:31:45 smithi167 ceph-mon[21875]: pgmap v235: 1 pgs: 1 active+clean; 577 KiB data, 329 MiB used, 983 GiB / 983 GiB avail 2024-08-19T01:31:45.716 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:31:45 smithi167 ceph-mon[21875]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-08-19T01:31:45.716 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:31:45 smithi167 ceph-mon[21875]: from='osd.11 [v2:172.21.15.167:6824/187027350,v1:172.21.15.167:6825/187027350]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi167", "root=default"]}]: dispatch 2024-08-19T01:31:45.716 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:31:45 smithi167 ceph-mon[21875]: osdmap e76: 12 total, 11 up, 12 in 2024-08-19T01:31:45.716 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:31:45 smithi167 ceph-mon[21875]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-08-19T01:31:45.716 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:31:45 smithi167 ceph-mon[21875]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi167", "root=default"]}]: dispatch 2024-08-19T01:31:45.716 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:31:45 smithi167 ceph-mon[21875]: from='client.? 172.21.15.83:0/952838612' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-08-19T01:31:45.717 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:31:45 smithi167 ceph-mon[21875]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi167", "root=default"]}]': finished 2024-08-19T01:31:45.717 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:31:45 smithi167 ceph-mon[21875]: osdmap e77: 12 total, 11 up, 12 in 2024-08-19T01:31:45.860 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:31:45 smithi083 ceph-mon[17120]: pgmap v235: 1 pgs: 1 active+clean; 577 KiB data, 329 MiB used, 983 GiB / 983 GiB avail 2024-08-19T01:31:45.860 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:31:45 smithi083 ceph-mon[17120]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-08-19T01:31:45.860 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:31:45 smithi083 ceph-mon[17120]: from='osd.11 [v2:172.21.15.167:6824/187027350,v1:172.21.15.167:6825/187027350]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi167", "root=default"]}]: dispatch 2024-08-19T01:31:45.860 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:31:45 smithi083 ceph-mon[17120]: osdmap e76: 12 total, 11 up, 12 in 2024-08-19T01:31:45.860 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:31:45 smithi083 ceph-mon[17120]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-08-19T01:31:45.860 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:31:45 smithi083 ceph-mon[17120]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi167", "root=default"]}]: dispatch 2024-08-19T01:31:45.861 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:31:45 smithi083 ceph-mon[17120]: from='client.? 172.21.15.83:0/952838612' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-08-19T01:31:45.861 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:31:45 smithi083 ceph-mon[17120]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi167", "root=default"]}]': finished 2024-08-19T01:31:45.861 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:31:45 smithi083 ceph-mon[17120]: osdmap e77: 12 total, 11 up, 12 in 2024-08-19T01:31:45.899 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":76,"num_osds":12,"num_up_osds":11,"osd_up_since":1724031083,"num_in_osds":12,"osd_in_since":1724031087,"num_remapped_pgs":0} 2024-08-19T01:31:46.672 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:31:46 smithi167 ceph-mon[21875]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-08-19T01:31:46.673 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:31:46 smithi167 ceph-mon[21875]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-08-19T01:31:46.673 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:31:46 smithi167 ceph-mon[21875]: from='osd.11 ' entity='osd.11' 2024-08-19T01:31:46.673 INFO:journalctl@ceph.osd.11.smithi167.stdout:Aug 19 01:31:46 smithi167 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-11[46643]: 2024-08-19T01:31:46.250+0000 7f44857ef640 -1 osd.11 0 waiting for initial osdmap 2024-08-19T01:31:46.673 INFO:journalctl@ceph.osd.11.smithi167.stdout:Aug 19 01:31:46 smithi167 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-11[46643]: 2024-08-19T01:31:46.274+0000 7f447de21640 -1 osd.11 77 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-08-19T01:31:46.702 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:31:46 smithi117 ceph-mon[22016]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-08-19T01:31:46.702 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:31:46 smithi117 ceph-mon[22016]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-08-19T01:31:46.702 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:31:46 smithi117 ceph-mon[22016]: from='osd.11 ' entity='osd.11' 2024-08-19T01:31:46.900 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid bad8780e-5dc9-11ef-bccf-c7b262605968 -- ceph osd stat -f json 2024-08-19T01:31:46.905 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:31:46 smithi083 ceph-mon[17120]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-08-19T01:31:46.905 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:31:46 smithi083 ceph-mon[17120]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-08-19T01:31:46.905 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:31:46 smithi083 ceph-mon[17120]: from='osd.11 ' entity='osd.11' 2024-08-19T01:31:47.278 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/bad8780e-5dc9-11ef-bccf-c7b262605968/mon.a/config 2024-08-19T01:31:47.689 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:31:47 smithi083 ceph-mon[17120]: purged_snaps scrub starts 2024-08-19T01:31:47.689 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:31:47 smithi083 ceph-mon[17120]: purged_snaps scrub ok 2024-08-19T01:31:47.689 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:31:47 smithi083 ceph-mon[17120]: pgmap v238: 1 pgs: 1 active+clean; 577 KiB data, 329 MiB used, 983 GiB / 983 GiB avail 2024-08-19T01:31:47.689 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:31:47 smithi083 ceph-mon[17120]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-08-19T01:31:47.689 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:31:47 smithi083 ceph-mon[17120]: osd.11 [v2:172.21.15.167:6824/187027350,v1:172.21.15.167:6825/187027350] boot 2024-08-19T01:31:47.689 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:31:47 smithi083 ceph-mon[17120]: osdmap e78: 12 total, 12 up, 12 in 2024-08-19T01:31:47.689 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:31:47 smithi083 ceph-mon[17120]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-08-19T01:31:47.702 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:31:47 smithi117 ceph-mon[22016]: purged_snaps scrub starts 2024-08-19T01:31:47.703 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:31:47 smithi117 ceph-mon[22016]: purged_snaps scrub ok 2024-08-19T01:31:47.703 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:31:47 smithi117 ceph-mon[22016]: pgmap v238: 1 pgs: 1 active+clean; 577 KiB data, 329 MiB used, 983 GiB / 983 GiB avail 2024-08-19T01:31:47.703 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:31:47 smithi117 ceph-mon[22016]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-08-19T01:31:47.703 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:31:47 smithi117 ceph-mon[22016]: osd.11 [v2:172.21.15.167:6824/187027350,v1:172.21.15.167:6825/187027350] boot 2024-08-19T01:31:47.703 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:31:47 smithi117 ceph-mon[22016]: osdmap e78: 12 total, 12 up, 12 in 2024-08-19T01:31:47.703 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:31:47 smithi117 ceph-mon[22016]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-08-19T01:31:47.716 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:31:47 smithi167 ceph-mon[21875]: purged_snaps scrub starts 2024-08-19T01:31:47.716 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:31:47 smithi167 ceph-mon[21875]: purged_snaps scrub ok 2024-08-19T01:31:47.716 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:31:47 smithi167 ceph-mon[21875]: pgmap v238: 1 pgs: 1 active+clean; 577 KiB data, 329 MiB used, 983 GiB / 983 GiB avail 2024-08-19T01:31:47.716 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:31:47 smithi167 ceph-mon[21875]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-08-19T01:31:47.716 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:31:47 smithi167 ceph-mon[21875]: osd.11 [v2:172.21.15.167:6824/187027350,v1:172.21.15.167:6825/187027350] boot 2024-08-19T01:31:47.716 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:31:47 smithi167 ceph-mon[21875]: osdmap e78: 12 total, 12 up, 12 in 2024-08-19T01:31:47.716 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:31:47 smithi167 ceph-mon[21875]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-08-19T01:31:47.802 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:31:47.800+0000 7f2751600640 1 -- 172.21.15.83:0/3817995126 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f274c1056f0 msgr2=0x7f274c10bf80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:31:47.803 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:31:47.800+0000 7f2751600640 1 --2- 172.21.15.83:0/3817995126 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f274c1056f0 0x7f274c10bf80 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f2744009ca0 tx=0x7f274402a8c0 comp rx=0 tx=0).stop 2024-08-19T01:31:47.803 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:31:47.801+0000 7f2751600640 1 -- 172.21.15.83:0/3817995126 shutdown_connections 2024-08-19T01:31:47.803 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:31:47.801+0000 7f2751600640 1 --2- 172.21.15.83:0/3817995126 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f274c1056f0 0x7f274c10bf80 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:31:47.803 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:31:47.801+0000 7f2751600640 1 --2- 172.21.15.83:0/3817995126 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f274c104d30 0x7f274c1051b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:31:47.803 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:31:47.801+0000 7f2751600640 1 --2- 172.21.15.83:0/3817995126 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f274c103b30 0x7f274c103f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:31:47.803 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:31:47.801+0000 7f2751600640 1 -- 172.21.15.83:0/3817995126 >> 172.21.15.83:0/3817995126 conn(0x7f274c0ff2e0 msgr2=0x7f274c101700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:31:47.803 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:31:47.801+0000 7f2751600640 1 -- 172.21.15.83:0/3817995126 shutdown_connections 2024-08-19T01:31:47.803 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:31:47.801+0000 7f2751600640 1 -- 172.21.15.83:0/3817995126 wait complete. 2024-08-19T01:31:47.805 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:31:47.802+0000 7f2751600640 1 Processor -- start 2024-08-19T01:31:47.806 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:31:47.802+0000 7f2751600640 1 -- start start 2024-08-19T01:31:47.806 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:31:47.802+0000 7f2751600640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f274c103b30 0x7f274c199dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:31:47.806 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:31:47.802+0000 7f2751600640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f274c104d30 0x7f274c19a310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:31:47.806 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:31:47.803+0000 7f2751600640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f274c1056f0 0x7f274c1a1390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:31:47.806 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:31:47.803+0000 7f2751600640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f274c10f800 con 0x7f274c1056f0 2024-08-19T01:31:47.806 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:31:47.803+0000 7f2751600640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f274c10f680 con 0x7f274c103b30 2024-08-19T01:31:47.806 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:31:47.803+0000 7f2751600640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f274c10f980 con 0x7f274c104d30 2024-08-19T01:31:47.806 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:31:47.803+0000 7f274aa00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f274c103b30 0x7f274c199dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:31:47.806 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:31:47.803+0000 7f274b400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f274c1056f0 0x7f274c1a1390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:31:47.806 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:31:47.803+0000 7f274aa00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f274c103b30 0x7f274c199dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.83:50362/0 (socket says 172.21.15.83:50362) 2024-08-19T01:31:47.807 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:31:47.803+0000 7f274aa00640 1 -- 172.21.15.83:0/2214182832 learned_addr learned my addr 172.21.15.83:0/2214182832 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-08-19T01:31:47.807 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:31:47.803+0000 7f274a000640 1 --2- 172.21.15.83:0/2214182832 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f274c104d30 0x7f274c19a310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:31:47.807 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:31:47.803+0000 7f274aa00640 1 -- 172.21.15.83:0/2214182832 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f274c104d30 msgr2=0x7f274c19a310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:31:47.807 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:31:47.803+0000 7f274aa00640 1 --2- 172.21.15.83:0/2214182832 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f274c104d30 0x7f274c19a310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:31:47.807 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:31:47.803+0000 7f274aa00640 1 -- 172.21.15.83:0/2214182832 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f274c1056f0 msgr2=0x7f274c1a1390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:31:47.807 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:31:47.803+0000 7f274aa00640 1 --2- 172.21.15.83:0/2214182832 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f274c1056f0 0x7f274c1a1390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:31:47.807 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:31:47.803+0000 7f274aa00640 1 -- 172.21.15.83:0/2214182832 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f274c1a1a00 con 0x7f274c103b30 2024-08-19T01:31:47.807 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:31:47.803+0000 7f274aa00640 1 --2- 172.21.15.83:0/2214182832 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f274c103b30 0x7f274c199dd0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f273c00e9c0 tx=0x7f273c00ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:31:47.807 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:31:47.804+0000 7f2743400640 1 -- 172.21.15.83:0/2214182832 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f273c00cd80 con 0x7f274c103b30 2024-08-19T01:31:47.807 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:31:47.804+0000 7f2743400640 1 -- 172.21.15.83:0/2214182832 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f273c0035f0 con 0x7f274c103b30 2024-08-19T01:31:47.807 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:31:47.804+0000 7f2743400640 1 -- 172.21.15.83:0/2214182832 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f273c010690 con 0x7f274c103b30 2024-08-19T01:31:47.807 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:31:47.804+0000 7f2751600640 1 -- 172.21.15.83:0/2214182832 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f274c1a1cf0 con 0x7f274c103b30 2024-08-19T01:31:47.807 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:31:47.804+0000 7f2751600640 1 -- 172.21.15.83:0/2214182832 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f274c1a2180 con 0x7f274c103b30 2024-08-19T01:31:47.808 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:31:47.805+0000 7f2751600640 1 -- 172.21.15.83:0/2214182832 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f274c10aeb0 con 0x7f274c103b30 2024-08-19T01:31:47.808 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:31:47.807+0000 7f2743400640 1 -- 172.21.15.83:0/2214182832 <== mon.2 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f273c0037a0 con 0x7f274c103b30 2024-08-19T01:31:47.810 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:31:47.808+0000 7f2743400640 1 --2- 172.21.15.83:0/2214182832 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f27300a2bb0 0x7f27300a5070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:31:47.810 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:31:47.808+0000 7f2743400640 1 -- 172.21.15.83:0/2214182832 <== mon.2 v2:172.21.15.117:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 7127+0+0 (secure 0 0 0) 0x7f273c0cef90 con 0x7f274c103b30 2024-08-19T01:31:47.810 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:31:47.808+0000 7f274a000640 1 --2- 172.21.15.83:0/2214182832 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f27300a2bb0 0x7f27300a5070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:31:47.810 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:31:47.809+0000 7f274a000640 1 --2- 172.21.15.83:0/2214182832 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f27300a2bb0 0x7f27300a5070 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f274c19b2f0 tx=0x7f2738005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:31:47.818 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:31:47.814+0000 7f2743400640 1 -- 172.21.15.83:0/2214182832 <== mon.2 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7f273c09c2f0 con 0x7f274c103b30 2024-08-19T01:31:47.992 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:31:47.990+0000 7f2751600640 1 -- 172.21.15.83:0/2214182832 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) -- 0x7f274c062c20 con 0x7f274c103b30 2024-08-19T01:31:47.993 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:31:47.991+0000 7f2743400640 1 -- 172.21.15.83:0/2214182832 <== mon.2 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v78) ==== 74+0+133 (secure 0 0 0) 0x7f273c09bbf0 con 0x7f274c103b30 2024-08-19T01:31:47.993 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-08-19T01:31:47.996 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:31:47.994+0000 7f2751600640 1 -- 172.21.15.83:0/2214182832 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f27300a2bb0 msgr2=0x7f27300a5070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:31:47.997 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:31:47.994+0000 7f2751600640 1 --2- 172.21.15.83:0/2214182832 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f27300a2bb0 0x7f27300a5070 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f274c19b2f0 tx=0x7f2738005f90 comp rx=0 tx=0).stop 2024-08-19T01:31:47.997 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:31:47.994+0000 7f2751600640 1 -- 172.21.15.83:0/2214182832 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f274c103b30 msgr2=0x7f274c199dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:31:47.997 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:31:47.994+0000 7f2751600640 1 --2- 172.21.15.83:0/2214182832 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f274c103b30 0x7f274c199dd0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f273c00e9c0 tx=0x7f273c00ee80 comp rx=0 tx=0).stop 2024-08-19T01:31:47.997 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:31:47.994+0000 7f2751600640 1 -- 172.21.15.83:0/2214182832 shutdown_connections 2024-08-19T01:31:47.997 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:31:47.994+0000 7f2751600640 1 --2- 172.21.15.83:0/2214182832 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f27300a2bb0 0x7f27300a5070 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:31:47.997 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:31:47.994+0000 7f2751600640 1 --2- 172.21.15.83:0/2214182832 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f274c1056f0 0x7f274c1a1390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:31:47.997 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:31:47.994+0000 7f2751600640 1 --2- 172.21.15.83:0/2214182832 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f274c104d30 0x7f274c19a310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:31:47.997 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:31:47.994+0000 7f2751600640 1 --2- 172.21.15.83:0/2214182832 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f274c103b30 0x7f274c199dd0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:31:47.997 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:31:47.994+0000 7f2751600640 1 -- 172.21.15.83:0/2214182832 >> 172.21.15.83:0/2214182832 conn(0x7f274c0ff2e0 msgr2=0x7f274c108080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:31:47.997 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:31:47.995+0000 7f2751600640 1 -- 172.21.15.83:0/2214182832 shutdown_connections 2024-08-19T01:31:47.997 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:31:47.995+0000 7f2751600640 1 -- 172.21.15.83:0/2214182832 wait complete. 2024-08-19T01:31:48.627 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:31:48 smithi083 ceph-mon[17120]: Detected new or changed devices on smithi167 2024-08-19T01:31:48.627 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:31:48 smithi083 ceph-mon[17120]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' 2024-08-19T01:31:48.627 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:31:48 smithi083 ceph-mon[17120]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' 2024-08-19T01:31:48.628 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:31:48 smithi083 ceph-mon[17120]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-08-19T01:31:48.628 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:31:48 smithi083 ceph-mon[17120]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-08-19T01:31:48.628 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:31:48 smithi083 ceph-mon[17120]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-08-19T01:31:48.628 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:31:48 smithi083 ceph-mon[17120]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-08-19T01:31:48.628 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:31:48 smithi083 ceph-mon[17120]: Adjusting osd_memory_target on smithi167 to 4272M 2024-08-19T01:31:48.628 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:31:48 smithi083 ceph-mon[17120]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' 2024-08-19T01:31:48.628 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:31:48 smithi083 ceph-mon[17120]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-19T01:31:48.628 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:31:48 smithi083 ceph-mon[17120]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-19T01:31:48.628 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:31:48 smithi083 ceph-mon[17120]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' 2024-08-19T01:31:48.628 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:31:48 smithi083 ceph-mon[17120]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-19T01:31:48.628 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:31:48 smithi083 ceph-mon[17120]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-19T01:31:48.628 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:31:48 smithi083 ceph-mon[17120]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-19T01:31:48.628 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:31:48 smithi083 ceph-mon[17120]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' 2024-08-19T01:31:48.629 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:31:48 smithi083 ceph-mon[17120]: from='client.? 172.21.15.83:0/2214182832' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-08-19T01:31:48.667 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":78,"num_osds":12,"num_up_osds":12,"osd_up_since":1724031107,"num_in_osds":12,"osd_in_since":1724031087,"num_remapped_pgs":0} 2024-08-19T01:31:48.667 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833 shell --fsid bad8780e-5dc9-11ef-bccf-c7b262605968 -- ceph osd dump --format=json 2024-08-19T01:31:48.797 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:31:48 smithi117 ceph-mon[22016]: Detected new or changed devices on smithi167 2024-08-19T01:31:48.797 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:31:48 smithi117 ceph-mon[22016]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' 2024-08-19T01:31:48.797 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:31:48 smithi117 ceph-mon[22016]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' 2024-08-19T01:31:48.797 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:31:48 smithi117 ceph-mon[22016]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-08-19T01:31:48.797 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:31:48 smithi117 ceph-mon[22016]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-08-19T01:31:48.797 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:31:48 smithi117 ceph-mon[22016]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-08-19T01:31:48.797 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:31:48 smithi117 ceph-mon[22016]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-08-19T01:31:48.798 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:31:48 smithi117 ceph-mon[22016]: Adjusting osd_memory_target on smithi167 to 4272M 2024-08-19T01:31:48.798 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:31:48 smithi117 ceph-mon[22016]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' 2024-08-19T01:31:48.798 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:31:48 smithi117 ceph-mon[22016]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-19T01:31:48.798 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:31:48 smithi117 ceph-mon[22016]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-19T01:31:48.798 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:31:48 smithi117 ceph-mon[22016]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' 2024-08-19T01:31:48.798 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:31:48 smithi117 ceph-mon[22016]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-19T01:31:48.798 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:31:48 smithi117 ceph-mon[22016]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-19T01:31:48.798 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:31:48 smithi117 ceph-mon[22016]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-19T01:31:48.798 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:31:48 smithi117 ceph-mon[22016]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' 2024-08-19T01:31:48.798 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:31:48 smithi117 ceph-mon[22016]: from='client.? 172.21.15.83:0/2214182832' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-08-19T01:31:48.815 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:31:48 smithi167 ceph-mon[21875]: Detected new or changed devices on smithi167 2024-08-19T01:31:48.815 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:31:48 smithi167 ceph-mon[21875]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' 2024-08-19T01:31:48.815 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:31:48 smithi167 ceph-mon[21875]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' 2024-08-19T01:31:48.815 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:31:48 smithi167 ceph-mon[21875]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-08-19T01:31:48.815 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:31:48 smithi167 ceph-mon[21875]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-08-19T01:31:48.815 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:31:48 smithi167 ceph-mon[21875]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-08-19T01:31:48.815 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:31:48 smithi167 ceph-mon[21875]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-08-19T01:31:48.816 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:31:48 smithi167 ceph-mon[21875]: Adjusting osd_memory_target on smithi167 to 4272M 2024-08-19T01:31:48.816 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:31:48 smithi167 ceph-mon[21875]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' 2024-08-19T01:31:48.816 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:31:48 smithi167 ceph-mon[21875]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-19T01:31:48.816 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:31:48 smithi167 ceph-mon[21875]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-19T01:31:48.816 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:31:48 smithi167 ceph-mon[21875]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' 2024-08-19T01:31:48.816 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:31:48 smithi167 ceph-mon[21875]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-19T01:31:48.816 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:31:48 smithi167 ceph-mon[21875]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-19T01:31:48.816 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:31:48 smithi167 ceph-mon[21875]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-19T01:31:48.816 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:31:48 smithi167 ceph-mon[21875]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' 2024-08-19T01:31:48.816 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:31:48 smithi167 ceph-mon[21875]: from='client.? 172.21.15.83:0/2214182832' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-08-19T01:31:49.038 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/bad8780e-5dc9-11ef-bccf-c7b262605968/mon.a/config 2024-08-19T01:31:49.617 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:31:49.615+0000 7f504fe00640 1 -- 172.21.15.83:0/4120253044 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5050109900 msgr2=0x7f5050109ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:31:49.617 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:31:49.615+0000 7f504fe00640 1 --2- 172.21.15.83:0/4120253044 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5050109900 0x7f5050109ce0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f5040009930 tx=0x7f504002a420 comp rx=0 tx=0).stop 2024-08-19T01:31:49.617 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:31:49.615+0000 7f504fe00640 1 -- 172.21.15.83:0/4120253044 shutdown_connections 2024-08-19T01:31:49.617 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:31:49.615+0000 7f504fe00640 1 --2- 172.21.15.83:0/4120253044 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f50501042a0 0x7f50501107d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:31:49.617 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:31:49.615+0000 7f504fe00640 1 --2- 172.21.15.83:0/4120253044 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5050103900 0x7f5050103d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:31:49.618 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:31:49.615+0000 7f504fe00640 1 --2- 172.21.15.83:0/4120253044 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5050109900 0x7f5050109ce0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:31:49.618 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:31:49.615+0000 7f504fe00640 1 -- 172.21.15.83:0/4120253044 >> 172.21.15.83:0/4120253044 conn(0x7f50500ff640 msgr2=0x7f5050101a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:31:49.618 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:31:49.615+0000 7f504fe00640 1 -- 172.21.15.83:0/4120253044 shutdown_connections 2024-08-19T01:31:49.618 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:31:49.615+0000 7f504fe00640 1 -- 172.21.15.83:0/4120253044 wait complete. 2024-08-19T01:31:49.619 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:31:49.616+0000 7f504fe00640 1 Processor -- start 2024-08-19T01:31:49.619 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:31:49.617+0000 7f504fe00640 1 -- start start 2024-08-19T01:31:49.619 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:31:49.617+0000 7f504fe00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5050103900 0x7f505019a200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:31:49.619 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:31:49.617+0000 7f504fe00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f50501042a0 0x7f505019a740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:31:49.619 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:31:49.617+0000 7f504fe00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5050109900 0x7f505019ead0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:31:49.619 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:31:49.617+0000 7f504fe00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f5050113d50 con 0x7f5050103900 2024-08-19T01:31:49.619 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:31:49.617+0000 7f504fe00640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f5050113bd0 con 0x7f5050109900 2024-08-19T01:31:49.619 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:31:49.617+0000 7f504fe00640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f5050113ed0 con 0x7f50501042a0 2024-08-19T01:31:49.619 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:31:49.617+0000 7f5047e00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f50501042a0 0x7f505019a740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:31:49.619 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:31:49.617+0000 7f504ea00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5050103900 0x7f505019a200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:31:49.619 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:31:49.617+0000 7f5047e00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f50501042a0 0x7f505019a740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.83:50166/0 (socket says 172.21.15.83:50166) 2024-08-19T01:31:49.619 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:31:49.617+0000 7f5047e00640 1 -- 172.21.15.83:0/1079339429 learned_addr learned my addr 172.21.15.83:0/1079339429 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-08-19T01:31:49.621 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:31:49.617+0000 7f504f400640 1 --2- 172.21.15.83:0/1079339429 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5050109900 0x7f505019ead0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:31:49.621 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:31:49.618+0000 7f5047e00640 1 -- 172.21.15.83:0/1079339429 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5050109900 msgr2=0x7f505019ead0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:31:49.621 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:31:49.618+0000 7f5047e00640 1 --2- 172.21.15.83:0/1079339429 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5050109900 0x7f505019ead0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:31:49.621 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:31:49.618+0000 7f5047e00640 1 -- 172.21.15.83:0/1079339429 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5050103900 msgr2=0x7f505019a200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:31:49.621 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:31:49.618+0000 7f5047e00640 1 --2- 172.21.15.83:0/1079339429 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5050103900 0x7f505019a200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:31:49.621 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:31:49.618+0000 7f5047e00640 1 -- 172.21.15.83:0/1079339429 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f505019f1b0 con 0x7f50501042a0 2024-08-19T01:31:49.621 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:31:49.618+0000 7f5047e00640 1 --2- 172.21.15.83:0/1079339429 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f50501042a0 0x7f505019a740 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f503800ea60 tx=0x7f503800ef20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:31:49.621 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:31:49.618+0000 7f5047400640 1 -- 172.21.15.83:0/1079339429 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f503800cde0 con 0x7f50501042a0 2024-08-19T01:31:49.621 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:31:49.618+0000 7f5047400640 1 -- 172.21.15.83:0/1079339429 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f50380035f0 con 0x7f50501042a0 2024-08-19T01:31:49.622 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:31:49.618+0000 7f5047400640 1 -- 172.21.15.83:0/1079339429 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5038010670 con 0x7f50501042a0 2024-08-19T01:31:49.622 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:31:49.618+0000 7f504fe00640 1 -- 172.21.15.83:0/1079339429 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f505019f4a0 con 0x7f50501042a0 2024-08-19T01:31:49.622 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:31:49.618+0000 7f504fe00640 1 -- 172.21.15.83:0/1079339429 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f50501a6d80 con 0x7f50501042a0 2024-08-19T01:31:49.622 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:31:49.619+0000 7f504fe00640 1 -- 172.21.15.83:0/1079339429 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5050105040 con 0x7f50501042a0 2024-08-19T01:31:49.623 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:31:49.621+0000 7f5047400640 1 -- 172.21.15.83:0/1079339429 <== mon.1 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f50380037a0 con 0x7f50501042a0 2024-08-19T01:31:49.625 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:31:49.623+0000 7f5047400640 1 --2- 172.21.15.83:0/1079339429 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f50340a2c00 0x7f50340a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:31:49.625 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:31:49.623+0000 7f5047400640 1 -- 172.21.15.83:0/1079339429 <== mon.1 v2:172.21.15.167:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 7127+0+0 (secure 0 0 0) 0x7f50380ceff0 con 0x7f50501042a0 2024-08-19T01:31:49.625 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:31:49.623+0000 7f504ea00640 1 --2- 172.21.15.83:0/1079339429 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f50340a2c00 0x7f50340a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:31:49.625 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:31:49.624+0000 7f504ea00640 1 --2- 172.21.15.83:0/1079339429 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f50340a2c00 0x7f50340a50c0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f5040006010 tx=0x7f5040000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:31:49.626 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:31:49.624+0000 7f5047400640 1 -- 172.21.15.83:0/1079339429 <== mon.1 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7f503809c3e0 con 0x7f50501042a0 2024-08-19T01:31:49.802 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:31:49.800+0000 7f504fe00640 1 -- 172.21.15.83:0/1079339429 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f5050103d60 con 0x7f50501042a0 2024-08-19T01:31:49.802 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:31:49 smithi083 ceph-mon[17120]: pgmap v240: 1 pgs: 1 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-19T01:31:49.803 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:31:49 smithi083 ceph-mon[17120]: osdmap e79: 12 total, 12 up, 12 in 2024-08-19T01:31:49.803 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:31:49.801+0000 7f5047400640 1 -- 172.21.15.83:0/1079339429 <== mon.1 v2:172.21.15.167:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v79) ==== 74+0+18431 (secure 0 0 0) 0x7f503809bce0 con 0x7f50501042a0 2024-08-19T01:31:49.804 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-08-19T01:31:49.804 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":79,"fsid":"bad8780e-5dc9-11ef-bccf-c7b262605968","created":"2024-08-19T01:25:12.626841+0000","modified":"2024-08-19T01:31:48.655776+0000","last_up_change":"2024-08-19T01:31:47.254559+0000","last_in_change":"2024-08-19T01:31:27.652847+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-08-19T01:28:16.862007+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ffa74041-f9d2-4555-bb43-508b8c581487","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6802","nonce":1082526824},{"type":"v1","addr":"172.21.15.83:6803","nonce":1082526824}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6804","nonce":1082526824},{"type":"v1","addr":"172.21.15.83:6805","nonce":1082526824}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6808","nonce":1082526824},{"type":"v1","addr":"172.21.15.83:6809","nonce":1082526824}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6806","nonce":1082526824},{"type":"v1","addr":"172.21.15.83:6807","nonce":1082526824}]},"public_addr":"172.21.15.83:6803/1082526824","cluster_addr":"172.21.15.83:6805/1082526824","heartbeat_back_addr":"172.21.15.83:6809/1082526824","heartbeat_front_addr":"172.21.15.83:6807/1082526824","state":["exists","up"]},{"osd":1,"uuid":"1977570b-b4f5-44fd-bb1e-3a31318be649","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6800","nonce":1925419802},{"type":"v1","addr":"172.21.15.117:6801","nonce":1925419802}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6802","nonce":1925419802},{"type":"v1","addr":"172.21.15.117:6803","nonce":1925419802}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6806","nonce":1925419802},{"type":"v1","addr":"172.21.15.117:6807","nonce":1925419802}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6804","nonce":1925419802},{"type":"v1","addr":"172.21.15.117:6805","nonce":1925419802}]},"public_addr":"172.21.15.117:6801/1925419802","cluster_addr":"172.21.15.117:6803/1925419802","heartbeat_back_addr":"172.21.15.117:6807/1925419802","heartbeat_front_addr":"172.21.15.117:6805/1925419802","state":["exists","up"]},{"osd":2,"uuid":"a75278d1-161f-41d5-a92f-058d1077bf2a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6800","nonce":551625328},{"type":"v1","addr":"172.21.15.167:6801","nonce":551625328}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6802","nonce":551625328},{"type":"v1","addr":"172.21.15.167:6803","nonce":551625328}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6806","nonce":551625328},{"type":"v1","addr":"172.21.15.167:6807","nonce":551625328}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6804","nonce":551625328},{"type":"v1","addr":"172.21.15.167:6805","nonce":551625328}]},"public_addr":"172.21.15.167:6801/551625328","cluster_addr":"172.21.15.167:6803/551625328","heartbeat_back_addr":"172.21.15.167:6807/551625328","heartbeat_front_addr":"172.21.15.167:6805/551625328","state":["exists","up"]},{"osd":3,"uuid":"d1fb16f9-910e-46fa-b1e3-8e5bef4d346d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6810","nonce":3138942062},{"type":"v1","addr":"172.21.15.83:6811","nonce":3138942062}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6812","nonce":3138942062},{"type":"v1","addr":"172.21.15.83:6813","nonce":3138942062}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6816","nonce":3138942062},{"type":"v1","addr":"172.21.15.83:6817","nonce":3138942062}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6814","nonce":3138942062},{"type":"v1","addr":"172.21.15.83:6815","nonce":3138942062}]},"public_addr":"172.21.15.83:6811/3138942062","cluster_addr":"172.21.15.83:6813/3138942062","heartbeat_back_addr":"172.21.15.83:6817/3138942062","heartbeat_front_addr":"172.21.15.83:6815/3138942062","state":["exists","up"]},{"osd":4,"uuid":"8176ed18-1fe2-446d-8994-8c60f6e08f38","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6808","nonce":3978863827},{"type":"v1","addr":"172.21.15.117:6809","nonce":3978863827}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6810","nonce":3978863827},{"type":"v1","addr":"172.21.15.117:6811","nonce":3978863827}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6814","nonce":3978863827},{"type":"v1","addr":"172.21.15.117:6815","nonce":3978863827}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6812","nonce":3978863827},{"type":"v1","addr":"172.21.15.117:6813","nonce":3978863827}]},"public_addr":"172.21.15.117:6809/3978863827","cluster_addr":"172.21.15.117:6811/3978863827","heartbeat_back_addr":"172.21.15.117:6815/3978863827","heartbeat_front_addr":"172.21.15.117:6813/3978863827","state":["exists","up"]},{"osd":5,"uuid":"e01e3f15-25c8-4d2c-be6f-a170f036064c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6808","nonce":2761858035},{"type":"v1","addr":"172.21.15.167:6809","nonce":2761858035}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6810","nonce":2761858035},{"type":"v1","addr":"172.21.15.167:6811","nonce":2761858035}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6814","nonce":2761858035},{"type":"v1","addr":"172.21.15.167:6815","nonce":2761858035}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6812","nonce":2761858035},{"type":"v1","addr":"172.21.15.167:6813","nonce":2761858035}]},"public_addr":"172.21.15.167:6809/2761858035","cluster_addr":"172.21.15.167:6811/2761858035","heartbeat_back_addr":"172.21.15.167:6815/2761858035","heartbeat_front_addr":"172.21.15.167:6813/2761858035","state":["exists","up"]},{"osd":6,"uuid":"78cf0a9b-9165-444f-b073-ebe13016de9d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6818","nonce":260644552},{"type":"v1","addr":"172.21.15.83:6819","nonce":260644552}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6820","nonce":260644552},{"type":"v1","addr":"172.21.15.83:6821","nonce":260644552}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6824","nonce":260644552},{"type":"v1","addr":"172.21.15.83:6825","nonce":260644552}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6822","nonce":260644552},{"type":"v1","addr":"172.21.15.83:6823","nonce":260644552}]},"public_addr":"172.21.15.83:6819/260644552","cluster_addr":"172.21.15.83:6821/260644552","heartbeat_back_addr":"172.21.15.83:6825/260644552","heartbeat_front_addr":"172.21.15.83:6823/260644552","state":["exists","up"]},{"osd":7,"uuid":"ee524f7f-672e-430e-9f58-1a9117513466","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6816","nonce":104113252},{"type":"v1","addr":"172.21.15.117:6817","nonce":104113252}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6818","nonce":104113252},{"type":"v1","addr":"172.21.15.117:6819","nonce":104113252}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6822","nonce":104113252},{"type":"v1","addr":"172.21.15.117:6823","nonce":104113252}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6820","nonce":104113252},{"type":"v1","addr":"172.21.15.117:6821","nonce":104113252}]},"public_addr":"172.21.15.117:6817/104113252","cluster_addr":"172.21.15.117:6819/104113252","heartbeat_back_addr":"172.21.15.117:6823/104113252","heartbeat_front_addr":"172.21.15.117:6821/104113252","state":["exists","up"]},{"osd":8,"uuid":"76d5317d-4845-4138-b210-e7bd2d80472a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":58,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6816","nonce":2196898857},{"type":"v1","addr":"172.21.15.167:6817","nonce":2196898857}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6818","nonce":2196898857},{"type":"v1","addr":"172.21.15.167:6819","nonce":2196898857}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6822","nonce":2196898857},{"type":"v1","addr":"172.21.15.167:6823","nonce":2196898857}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6820","nonce":2196898857},{"type":"v1","addr":"172.21.15.167:6821","nonce":2196898857}]},"public_addr":"172.21.15.167:6817/2196898857","cluster_addr":"172.21.15.167:6819/2196898857","heartbeat_back_addr":"172.21.15.167:6823/2196898857","heartbeat_front_addr":"172.21.15.167:6821/2196898857","state":["exists","up"]},{"osd":9,"uuid":"0151bc2a-3ad2-4db3-8505-5855792cc9bb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6826","nonce":903463450},{"type":"v1","addr":"172.21.15.83:6827","nonce":903463450}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6828","nonce":903463450},{"type":"v1","addr":"172.21.15.83:6829","nonce":903463450}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6832","nonce":903463450},{"type":"v1","addr":"172.21.15.83:6833","nonce":903463450}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6830","nonce":903463450},{"type":"v1","addr":"172.21.15.83:6831","nonce":903463450}]},"public_addr":"172.21.15.83:6827/903463450","cluster_addr":"172.21.15.83:6829/903463450","heartbeat_back_addr":"172.21.15.83:6833/903463450","heartbeat_front_addr":"172.21.15.83:6831/903463450","state":["exists","up"]},{"osd":10,"uuid":"437ed848-0da4-4cd4-8716-0c55bb99b676","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6824","nonce":1183570837},{"type":"v1","addr":"172.21.15.117:6825","nonce":1183570837}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6826","nonce":1183570837},{"type":"v1","addr":"172.21.15.117:6827","nonce":1183570837}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6830","nonce":1183570837},{"type":"v1","addr":"172.21.15.117:6831","nonce":1183570837}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6828","nonce":1183570837},{"type":"v1","addr":"172.21.15.117:6829","nonce":1183570837}]},"public_addr":"172.21.15.117:6825/1183570837","cluster_addr":"172.21.15.117:6827/1183570837","heartbeat_back_addr":"172.21.15.117:6831/1183570837","heartbeat_front_addr":"172.21.15.117:6829/1183570837","state":["exists","up"]},{"osd":11,"uuid":"13345f05-cdc0-4dae-a58b-551abbec1915","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":78,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6824","nonce":187027350},{"type":"v1","addr":"172.21.15.167:6825","nonce":187027350}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6826","nonce":187027350},{"type":"v1","addr":"172.21.15.167:6827","nonce":187027350}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6830","nonce":187027350},{"type":"v1","addr":"172.21.15.167:6831","nonce":187027350}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6828","nonce":187027350},{"type":"v1","addr":"172.21.15.167:6829","nonce":187027350}]},"public_addr":"172.21.15.167:6825/187027350","cluster_addr":"172.21.15.167:6827/187027350","heartbeat_back_addr":"172.21.15.167:6831/187027350","heartbeat_front_addr":"172.21.15.167:6829/187027350","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-19T01:27:28.760517+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-19T01:27:50.493028+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-19T01:28:11.997674+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-19T01:28:38.109934+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-19T01:29:00.173065+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-19T01:29:21.336085+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-19T01:29:47.871498+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-19T01:30:10.720498+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-19T01:30:32.425438+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-19T01:30:59.524989+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-19T01:31:22.806162+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-19T01:31:45.043246+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.83:0/3138095466":"2024-08-20T01:26:04.277716+0000","172.21.15.83:0/3835005198":"2024-08-20T01:26:04.277716+0000","172.21.15.83:0/1790374088":"2024-08-20T01:25:37.228583+0000","172.21.15.83:6801/573907319":"2024-08-20T01:26:04.277716+0000","172.21.15.83:6800/573907319":"2024-08-20T01:26:04.277716+0000","172.21.15.83:0/664676002":"2024-08-20T01:25:37.228583+0000","172.21.15.83:0/2144611449":"2024-08-20T01:25:37.228583+0000","172.21.15.83:6801/3261152818":"2024-08-20T01:25:37.228583+0000","172.21.15.83:0/1609552666":"2024-08-20T01:26:04.277716+0000","172.21.15.83:6800/3261152818":"2024-08-20T01:25:37.228583+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-08-19T01:31:49.806 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:31:49.804+0000 7f504fe00640 1 -- 172.21.15.83:0/1079339429 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f50340a2c00 msgr2=0x7f50340a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:31:49.806 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:31:49.804+0000 7f504fe00640 1 --2- 172.21.15.83:0/1079339429 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f50340a2c00 0x7f50340a50c0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f5040006010 tx=0x7f5040000bc0 comp rx=0 tx=0).stop 2024-08-19T01:31:49.806 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:31:49.804+0000 7f504fe00640 1 -- 172.21.15.83:0/1079339429 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f50501042a0 msgr2=0x7f505019a740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:31:49.806 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:31:49.804+0000 7f504fe00640 1 --2- 172.21.15.83:0/1079339429 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f50501042a0 0x7f505019a740 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f503800ea60 tx=0x7f503800ef20 comp rx=0 tx=0).stop 2024-08-19T01:31:49.806 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:31:49.804+0000 7f504fe00640 1 -- 172.21.15.83:0/1079339429 shutdown_connections 2024-08-19T01:31:49.806 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:31:49.804+0000 7f504fe00640 1 --2- 172.21.15.83:0/1079339429 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f50340a2c00 0x7f50340a50c0 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:31:49.806 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:31:49.804+0000 7f504fe00640 1 --2- 172.21.15.83:0/1079339429 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5050109900 0x7f505019ead0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:31:49.806 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:31:49.804+0000 7f504fe00640 1 --2- 172.21.15.83:0/1079339429 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f50501042a0 0x7f505019a740 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:31:49.806 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:31:49.804+0000 7f504fe00640 1 --2- 172.21.15.83:0/1079339429 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5050103900 0x7f505019a200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:31:49.806 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:31:49.804+0000 7f504fe00640 1 -- 172.21.15.83:0/1079339429 >> 172.21.15.83:0/1079339429 conn(0x7f50500ff640 msgr2=0x7f50500ffeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:31:49.806 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:31:49.804+0000 7f504fe00640 1 -- 172.21.15.83:0/1079339429 shutdown_connections 2024-08-19T01:31:49.807 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:31:49.804+0000 7f504fe00640 1 -- 172.21.15.83:0/1079339429 wait complete. 2024-08-19T01:31:49.952 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:31:49 smithi117 ceph-mon[22016]: pgmap v240: 1 pgs: 1 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-19T01:31:49.952 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:31:49 smithi117 ceph-mon[22016]: osdmap e79: 12 total, 12 up, 12 in 2024-08-19T01:31:49.966 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:31:49 smithi167 ceph-mon[21875]: pgmap v240: 1 pgs: 1 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-19T01:31:49.966 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:31:49 smithi167 ceph-mon[21875]: osdmap e79: 12 total, 12 up, 12 in 2024-08-19T01:31:50.566 INFO:tasks.cephadm.ceph_manager.ceph:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-08-19T01:28:16.862007+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-08-19T01:31:50.566 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833 shell --fsid bad8780e-5dc9-11ef-bccf-c7b262605968 -- ceph osd pool get .mgr pg_num 2024-08-19T01:31:50.904 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:31:50 smithi083 ceph-mon[17120]: from='client.? 172.21.15.83:0/1079339429' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-08-19T01:31:50.934 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/bad8780e-5dc9-11ef-bccf-c7b262605968/mon.a/config 2024-08-19T01:31:50.952 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:31:50 smithi117 ceph-mon[22016]: from='client.? 172.21.15.83:0/1079339429' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-08-19T01:31:50.966 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:31:50 smithi167 ceph-mon[21875]: from='client.? 172.21.15.83:0/1079339429' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-08-19T01:31:51.458 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:31:51.456+0000 7f0d93e00640 1 -- 172.21.15.83:0/1232504751 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0d94109900 msgr2=0x7f0d94109ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:31:51.458 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:31:51.456+0000 7f0d93e00640 1 --2- 172.21.15.83:0/1232504751 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0d94109900 0x7f0d94109ce0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f0d84009930 tx=0x7f0d8402a420 comp rx=0 tx=0).stop 2024-08-19T01:31:51.458 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:31:51.456+0000 7f0d93e00640 1 -- 172.21.15.83:0/1232504751 shutdown_connections 2024-08-19T01:31:51.458 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:31:51.456+0000 7f0d93e00640 1 --2- 172.21.15.83:0/1232504751 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0d941042a0 0x7f0d941107d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:31:51.458 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:31:51.456+0000 7f0d93e00640 1 --2- 172.21.15.83:0/1232504751 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0d94103900 0x7f0d94103d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:31:51.458 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:31:51.456+0000 7f0d93e00640 1 --2- 172.21.15.83:0/1232504751 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0d94109900 0x7f0d94109ce0 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:31:51.458 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:31:51.456+0000 7f0d93e00640 1 -- 172.21.15.83:0/1232504751 >> 172.21.15.83:0/1232504751 conn(0x7f0d940ff600 msgr2=0x7f0d94101a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:31:51.458 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:31:51.456+0000 7f0d93e00640 1 -- 172.21.15.83:0/1232504751 shutdown_connections 2024-08-19T01:31:51.459 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:31:51.456+0000 7f0d93e00640 1 -- 172.21.15.83:0/1232504751 wait complete. 2024-08-19T01:31:51.460 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:31:51.457+0000 7f0d93e00640 1 Processor -- start 2024-08-19T01:31:51.461 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:31:51.457+0000 7f0d93e00640 1 -- start start 2024-08-19T01:31:51.461 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:31:51.457+0000 7f0d93e00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0d94103900 0x7f0d9419a250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:31:51.461 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:31:51.457+0000 7f0d93e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0d941042a0 0x7f0d9419a790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:31:51.461 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:31:51.457+0000 7f0d93e00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0d94109900 0x7f0d9419eb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:31:51.461 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:31:51.457+0000 7f0d93e00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f0d94113d50 con 0x7f0d941042a0 2024-08-19T01:31:51.461 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:31:51.457+0000 7f0d93e00640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f0d94113bd0 con 0x7f0d94103900 2024-08-19T01:31:51.461 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:31:51.457+0000 7f0d93e00640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f0d94113ed0 con 0x7f0d94109900 2024-08-19T01:31:51.461 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:31:51.458+0000 7f0d8a000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0d941042a0 0x7f0d9419a790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:31:51.461 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:31:51.458+0000 7f0d93400640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0d94109900 0x7f0d9419eb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:31:51.461 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:31:51.458+0000 7f0d8a000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0d941042a0 0x7f0d9419a790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:45914/0 (socket says 172.21.15.83:45914) 2024-08-19T01:31:51.461 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:31:51.458+0000 7f0d8a000640 1 -- 172.21.15.83:0/1569078009 learned_addr learned my addr 172.21.15.83:0/1569078009 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-08-19T01:31:51.461 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:31:51.458+0000 7f0d92a00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0d94103900 0x7f0d9419a250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:31:51.462 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:31:51.458+0000 7f0d8a000640 1 -- 172.21.15.83:0/1569078009 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0d94109900 msgr2=0x7f0d9419eb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:31:51.462 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:31:51.458+0000 7f0d8a000640 1 --2- 172.21.15.83:0/1569078009 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0d94109900 0x7f0d9419eb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:31:51.462 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:31:51.458+0000 7f0d8a000640 1 -- 172.21.15.83:0/1569078009 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0d94103900 msgr2=0x7f0d9419a250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:31:51.462 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:31:51.458+0000 7f0d8a000640 1 --2- 172.21.15.83:0/1569078009 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0d94103900 0x7f0d9419a250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:31:51.462 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:31:51.458+0000 7f0d8a000640 1 -- 172.21.15.83:0/1569078009 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0d9419f200 con 0x7f0d941042a0 2024-08-19T01:31:51.462 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:31:51.458+0000 7f0d92a00640 1 --2- 172.21.15.83:0/1569078009 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0d94103900 0x7f0d9419a250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-19T01:31:51.462 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:31:51.458+0000 7f0d8a000640 1 --2- 172.21.15.83:0/1569078009 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0d941042a0 0x7f0d9419a790 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f0d7c00d770 tx=0x7f0d7c00dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:31:51.462 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:31:51.459+0000 7f0d8be00640 1 -- 172.21.15.83:0/1569078009 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d7c017070 con 0x7f0d941042a0 2024-08-19T01:31:51.462 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:31:51.459+0000 7f0d8be00640 1 -- 172.21.15.83:0/1569078009 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0d7c005500 con 0x7f0d941042a0 2024-08-19T01:31:51.462 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:31:51.459+0000 7f0d8be00640 1 -- 172.21.15.83:0/1569078009 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d7c004e00 con 0x7f0d941042a0 2024-08-19T01:31:51.462 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:31:51.459+0000 7f0d93e00640 1 -- 172.21.15.83:0/1569078009 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0d9419f4f0 con 0x7f0d941042a0 2024-08-19T01:31:51.462 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:31:51.459+0000 7f0d93e00640 1 -- 172.21.15.83:0/1569078009 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0d941a6dd0 con 0x7f0d941042a0 2024-08-19T01:31:51.463 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:31:51.460+0000 7f0d93e00640 1 -- 172.21.15.83:0/1569078009 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0d94103d60 con 0x7f0d941042a0 2024-08-19T01:31:51.463 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:31:51.462+0000 7f0d8be00640 1 -- 172.21.15.83:0/1569078009 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f0d7c004fa0 con 0x7f0d941042a0 2024-08-19T01:31:51.465 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:31:51.464+0000 7f0d8be00640 1 --2- 172.21.15.83:0/1569078009 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f0d780a2c00 0x7f0d780a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:31:51.466 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:31:51.464+0000 7f0d8be00640 1 -- 172.21.15.83:0/1569078009 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 7127+0+0 (secure 0 0 0) 0x7f0d7c0ce9d0 con 0x7f0d941042a0 2024-08-19T01:31:51.466 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:31:51.464+0000 7f0d92a00640 1 --2- 172.21.15.83:0/1569078009 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f0d780a2c00 0x7f0d780a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:31:51.466 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:31:51.465+0000 7f0d92a00640 1 --2- 172.21.15.83:0/1569078009 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f0d780a2c00 0x7f0d780a50c0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f0d84006010 tx=0x7f0d84031510 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:31:51.467 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:31:51.466+0000 7f0d8be00640 1 -- 172.21.15.83:0/1569078009 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7f0d7c09b760 con 0x7f0d941042a0 2024-08-19T01:31:51.645 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:31:51.643+0000 7f0d93e00640 1 -- 172.21.15.83:0/1569078009 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f0d9402d5e0 con 0x7f0d941042a0 2024-08-19T01:31:51.645 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:31:51 smithi083 ceph-mon[17120]: pgmap v242: 1 pgs: 1 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-19T01:31:51.646 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:31:51.644+0000 7f0d8be00640 1 -- 172.21.15.83:0/1569078009 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v79) ==== 93+0+10 (secure 0 0 0) 0x7f0d7c09fa60 con 0x7f0d941042a0 2024-08-19T01:31:51.646 INFO:teuthology.orchestra.run.smithi083.stdout:pg_num: 1 2024-08-19T01:31:51.648 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:31:51.646+0000 7f0d93e00640 1 -- 172.21.15.83:0/1569078009 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f0d780a2c00 msgr2=0x7f0d780a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:31:51.648 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:31:51.646+0000 7f0d93e00640 1 --2- 172.21.15.83:0/1569078009 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f0d780a2c00 0x7f0d780a50c0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f0d84006010 tx=0x7f0d84031510 comp rx=0 tx=0).stop 2024-08-19T01:31:51.648 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:31:51.646+0000 7f0d93e00640 1 -- 172.21.15.83:0/1569078009 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0d941042a0 msgr2=0x7f0d9419a790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:31:51.648 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:31:51.646+0000 7f0d93e00640 1 --2- 172.21.15.83:0/1569078009 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0d941042a0 0x7f0d9419a790 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f0d7c00d770 tx=0x7f0d7c00dc30 comp rx=0 tx=0).stop 2024-08-19T01:31:51.649 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:31:51.646+0000 7f0d93e00640 1 -- 172.21.15.83:0/1569078009 shutdown_connections 2024-08-19T01:31:51.649 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:31:51.646+0000 7f0d93e00640 1 --2- 172.21.15.83:0/1569078009 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f0d780a2c00 0x7f0d780a50c0 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:31:51.649 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:31:51.646+0000 7f0d93e00640 1 --2- 172.21.15.83:0/1569078009 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0d94109900 0x7f0d9419eb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:31:51.649 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:31:51.646+0000 7f0d93e00640 1 --2- 172.21.15.83:0/1569078009 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0d941042a0 0x7f0d9419a790 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:31:51.649 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:31:51.646+0000 7f0d93e00640 1 --2- 172.21.15.83:0/1569078009 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0d94103900 0x7f0d9419a250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:31:51.649 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:31:51.646+0000 7f0d93e00640 1 -- 172.21.15.83:0/1569078009 >> 172.21.15.83:0/1569078009 conn(0x7f0d940ff600 msgr2=0x7f0d940ffe80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:31:51.649 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:31:51.647+0000 7f0d93e00640 1 -- 172.21.15.83:0/1569078009 shutdown_connections 2024-08-19T01:31:51.649 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:31:51.647+0000 7f0d93e00640 1 -- 172.21.15.83:0/1569078009 wait complete. 2024-08-19T01:31:51.952 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:31:51 smithi117 ceph-mon[22016]: pgmap v242: 1 pgs: 1 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-19T01:31:51.966 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:31:51 smithi167 ceph-mon[21875]: pgmap v242: 1 pgs: 1 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-19T01:31:52.348 INFO:tasks.cephadm:Adding mds.a on smithi083 2024-08-19T01:31:52.348 INFO:tasks.cephadm:Adding mds.d on smithi083 2024-08-19T01:31:52.348 INFO:tasks.cephadm:Adding mds.g on smithi083 2024-08-19T01:31:52.348 INFO:tasks.cephadm:Adding mds.j on smithi083 2024-08-19T01:31:52.349 INFO:tasks.cephadm:Adding mds.b on smithi117 2024-08-19T01:31:52.349 INFO:tasks.cephadm:Adding mds.e on smithi117 2024-08-19T01:31:52.349 INFO:tasks.cephadm:Adding mds.h on smithi117 2024-08-19T01:31:52.349 INFO:tasks.cephadm:Adding mds.k on smithi117 2024-08-19T01:31:52.349 INFO:tasks.cephadm:Adding mds.c on smithi167 2024-08-19T01:31:52.349 INFO:tasks.cephadm:Adding mds.f on smithi167 2024-08-19T01:31:52.349 INFO:tasks.cephadm:Adding mds.i on smithi167 2024-08-19T01:31:52.349 INFO:tasks.cephadm:Adding mds.l on smithi167 2024-08-19T01:31:52.349 DEBUG:teuthology.orchestra.run.smithi167:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid bad8780e-5dc9-11ef-bccf-c7b262605968 -- ceph orch apply mds all '12;smithi083=a;smithi083=d;smithi083=g;smithi083=j;smithi117=b;smithi117=e;smithi117=h;smithi117=k;smithi167=c;smithi167=f;smithi167=i;smithi167=l' 2024-08-19T01:31:52.616 INFO:teuthology.orchestra.run.smithi167.stderr:Inferring config /var/lib/ceph/bad8780e-5dc9-11ef-bccf-c7b262605968/mon.c/config 2024-08-19T01:31:52.654 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:31:52 smithi083 ceph-mon[17120]: from='client.? 172.21.15.83:0/1569078009' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-08-19T01:31:52.716 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:31:52 smithi167 ceph-mon[21875]: from='client.? 172.21.15.83:0/1569078009' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-08-19T01:31:52.952 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:31:52 smithi117 ceph-mon[22016]: from='client.? 172.21.15.83:0/1569078009' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-08-19T01:31:53.121 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-19T01:31:53.080+0000 7f26e5518640 1 -- 172.21.15.167:0/3163193469 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f26e0100830 msgr2=0x7f26e0100c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:31:53.122 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-19T01:31:53.080+0000 7f26e5518640 1 --2- 172.21.15.167:0/3163193469 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f26e0100830 0x7f26e0100c30 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f26d0009900 tx=0x7f26d002a410 comp rx=0 tx=0).stop 2024-08-19T01:31:53.122 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-19T01:31:53.080+0000 7f26e5518640 1 -- 172.21.15.167:0/3163193469 shutdown_connections 2024-08-19T01:31:53.122 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-19T01:31:53.080+0000 7f26e5518640 1 --2- 172.21.15.167:0/3163193469 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f26e01023f0 0x7f26e0108c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:31:53.122 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-19T01:31:53.080+0000 7f26e5518640 1 --2- 172.21.15.167:0/3163193469 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f26e0101a30 0x7f26e0101eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:31:53.122 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-19T01:31:53.080+0000 7f26e5518640 1 --2- 172.21.15.167:0/3163193469 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f26e0100830 0x7f26e0100c30 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:31:53.122 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-19T01:31:53.080+0000 7f26e5518640 1 -- 172.21.15.167:0/3163193469 >> 172.21.15.167:0/3163193469 conn(0x7f26e00fbfa0 msgr2=0x7f26e00fe400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:31:53.122 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-19T01:31:53.080+0000 7f26e5518640 1 -- 172.21.15.167:0/3163193469 shutdown_connections 2024-08-19T01:31:53.122 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-19T01:31:53.080+0000 7f26e5518640 1 -- 172.21.15.167:0/3163193469 wait complete. 2024-08-19T01:31:53.122 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-19T01:31:53.080+0000 7f26e5518640 1 Processor -- start 2024-08-19T01:31:53.122 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-19T01:31:53.080+0000 7f26e5518640 1 -- start start 2024-08-19T01:31:53.122 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-19T01:31:53.081+0000 7f26e5518640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f26e0100830 0x7f26e0197d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:31:53.123 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-19T01:31:53.081+0000 7f26e5518640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f26e0101a30 0x7f26e0198250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:31:53.123 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-19T01:31:53.081+0000 7f26e5518640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f26e01023f0 0x7f26e019f290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:31:53.123 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-19T01:31:53.081+0000 7f26e5518640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f26e010c530 con 0x7f26e01023f0 2024-08-19T01:31:53.123 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-19T01:31:53.081+0000 7f26e5518640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f26e010c3b0 con 0x7f26e0101a30 2024-08-19T01:31:53.123 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-19T01:31:53.081+0000 7f26e5518640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f26e010c6b0 con 0x7f26e0100830 2024-08-19T01:31:53.123 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-19T01:31:53.081+0000 7f26deffd640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f26e0100830 0x7f26e0197d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:31:53.123 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-19T01:31:53.081+0000 7f26deffd640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f26e0100830 0x7f26e0197d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:53134/0 (socket says 172.21.15.167:53134) 2024-08-19T01:31:53.123 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-19T01:31:53.081+0000 7f26deffd640 1 -- 172.21.15.167:0/2189859071 learned_addr learned my addr 172.21.15.167:0/2189859071 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-08-19T01:31:53.123 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-19T01:31:53.081+0000 7f26d7fff640 1 --2- 172.21.15.167:0/2189859071 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f26e0101a30 0x7f26e0198250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:31:53.123 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-19T01:31:53.081+0000 7f26df7fe640 1 --2- 172.21.15.167:0/2189859071 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f26e01023f0 0x7f26e019f290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:31:53.123 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-19T01:31:53.081+0000 7f26deffd640 1 -- 172.21.15.167:0/2189859071 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f26e0101a30 msgr2=0x7f26e0198250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:31:53.123 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-19T01:31:53.081+0000 7f26deffd640 1 --2- 172.21.15.167:0/2189859071 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f26e0101a30 0x7f26e0198250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:31:53.123 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-19T01:31:53.081+0000 7f26deffd640 1 -- 172.21.15.167:0/2189859071 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f26e01023f0 msgr2=0x7f26e019f290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:31:53.123 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-19T01:31:53.081+0000 7f26deffd640 1 --2- 172.21.15.167:0/2189859071 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f26e01023f0 0x7f26e019f290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:31:53.124 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-19T01:31:53.081+0000 7f26deffd640 1 -- 172.21.15.167:0/2189859071 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f26e019f930 con 0x7f26e0100830 2024-08-19T01:31:53.124 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-19T01:31:53.081+0000 7f26deffd640 1 --2- 172.21.15.167:0/2189859071 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f26e0100830 0x7f26e0197d10 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f26d0006010 tx=0x7f26d0031b60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:31:53.124 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-19T01:31:53.081+0000 7f26dcff9640 1 -- 172.21.15.167:0/2189859071 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26d003f030 con 0x7f26e0100830 2024-08-19T01:31:53.124 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-19T01:31:53.081+0000 7f26dcff9640 1 -- 172.21.15.167:0/2189859071 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f26d00044a0 con 0x7f26e0100830 2024-08-19T01:31:53.124 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-19T01:31:53.081+0000 7f26e5518640 1 -- 172.21.15.167:0/2189859071 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f26e019fbc0 con 0x7f26e0100830 2024-08-19T01:31:53.124 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-19T01:31:53.081+0000 7f26dcff9640 1 -- 172.21.15.167:0/2189859071 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26d0035070 con 0x7f26e0100830 2024-08-19T01:31:53.124 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-19T01:31:53.082+0000 7f26e5518640 1 -- 172.21.15.167:0/2189859071 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f26e01a00a0 con 0x7f26e0100830 2024-08-19T01:31:53.124 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-19T01:31:53.082+0000 7f26e5518640 1 -- 172.21.15.167:0/2189859071 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f26e01a0270 con 0x7f26e0100830 2024-08-19T01:31:53.124 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-19T01:31:53.084+0000 7f26dcff9640 1 -- 172.21.15.167:0/2189859071 <== mon.1 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f26d0042030 con 0x7f26e0100830 2024-08-19T01:31:53.124 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-19T01:31:53.084+0000 7f26dcff9640 1 --2- 172.21.15.167:0/2189859071 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f26cc0a2b80 0x7f26cc0a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:31:53.124 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-19T01:31:53.084+0000 7f26dcff9640 1 -- 172.21.15.167:0/2189859071 <== mon.1 v2:172.21.15.167:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 7127+0+0 (secure 0 0 0) 0x7f26d00ef120 con 0x7f26e0100830 2024-08-19T01:31:53.124 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-19T01:31:53.084+0000 7f26d7fff640 1 --2- 172.21.15.167:0/2189859071 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f26cc0a2b80 0x7f26cc0a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:31:53.124 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-19T01:31:53.085+0000 7f26d7fff640 1 --2- 172.21.15.167:0/2189859071 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f26cc0a2b80 0x7f26cc0a5040 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f26c8009990 tx=0x7f26c8009110 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:31:53.124 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-19T01:31:53.088+0000 7f26dcff9640 1 -- 172.21.15.167:0/2189859071 <== mon.1 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7f26d00bc520 con 0x7f26e0100830 2024-08-19T01:31:53.266 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-19T01:31:53.265+0000 7f26e5518640 1 -- 172.21.15.167:0/2189859071 --> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] -- mgr_command(tid 0: {"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi083=a;smithi083=d;smithi083=g;smithi083=j;smithi117=b;smithi117=e;smithi117=h;smithi117=k;smithi167=c;smithi167=f;smithi167=i;smithi167=l", "target": ["mon-mgr", ""]}) -- 0x7f26e0062460 con 0x7f26cc0a2b80 2024-08-19T01:31:53.274 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-19T01:31:53.273+0000 7f26dcff9640 1 -- 172.21.15.167:0/2189859071 <== mgr.14150 v2:172.21.15.83:6800/683740916 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+28 (secure 0 0 0) 0x7f26e0062460 con 0x7f26cc0a2b80 2024-08-19T01:31:53.275 INFO:teuthology.orchestra.run.smithi167.stdout:Scheduled mds.all update... 2024-08-19T01:31:53.277 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-19T01:31:53.275+0000 7f26e5518640 1 -- 172.21.15.167:0/2189859071 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f26cc0a2b80 msgr2=0x7f26cc0a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:31:53.277 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-19T01:31:53.275+0000 7f26e5518640 1 --2- 172.21.15.167:0/2189859071 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f26cc0a2b80 0x7f26cc0a5040 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f26c8009990 tx=0x7f26c8009110 comp rx=0 tx=0).stop 2024-08-19T01:31:53.277 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-19T01:31:53.275+0000 7f26e5518640 1 -- 172.21.15.167:0/2189859071 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f26e0100830 msgr2=0x7f26e0197d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:31:53.277 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-19T01:31:53.275+0000 7f26e5518640 1 --2- 172.21.15.167:0/2189859071 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f26e0100830 0x7f26e0197d10 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f26d0006010 tx=0x7f26d0031b60 comp rx=0 tx=0).stop 2024-08-19T01:31:53.277 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-19T01:31:53.275+0000 7f26e5518640 1 -- 172.21.15.167:0/2189859071 shutdown_connections 2024-08-19T01:31:53.277 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-19T01:31:53.275+0000 7f26e5518640 1 --2- 172.21.15.167:0/2189859071 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f26cc0a2b80 0x7f26cc0a5040 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:31:53.277 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-19T01:31:53.275+0000 7f26e5518640 1 --2- 172.21.15.167:0/2189859071 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f26e01023f0 0x7f26e019f290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:31:53.278 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-19T01:31:53.275+0000 7f26e5518640 1 --2- 172.21.15.167:0/2189859071 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f26e0101a30 0x7f26e0198250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:31:53.278 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-19T01:31:53.275+0000 7f26e5518640 1 --2- 172.21.15.167:0/2189859071 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f26e0100830 0x7f26e0197d10 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:31:53.278 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-19T01:31:53.276+0000 7f26e5518640 1 -- 172.21.15.167:0/2189859071 >> 172.21.15.167:0/2189859071 conn(0x7f26e00fbfa0 msgr2=0x7f26e00fe110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:31:53.278 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-19T01:31:53.276+0000 7f26e5518640 1 -- 172.21.15.167:0/2189859071 shutdown_connections 2024-08-19T01:31:53.278 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-19T01:31:53.276+0000 7f26e5518640 1 -- 172.21.15.167:0/2189859071 wait complete. 2024-08-19T01:31:53.716 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:31:53 smithi167 ceph-mon[21875]: pgmap v243: 1 pgs: 1 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-19T01:31:53.716 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:31:53 smithi167 ceph-mon[21875]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' 2024-08-19T01:31:53.716 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:31:53 smithi167 ceph-mon[21875]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-19T01:31:53.716 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:31:53 smithi167 ceph-mon[21875]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-19T01:31:53.716 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:31:53 smithi167 ceph-mon[21875]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-19T01:31:53.716 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:31:53 smithi167 ceph-mon[21875]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' 2024-08-19T01:31:53.717 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:31:53 smithi167 ceph-mon[21875]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-08-19T01:31:53.717 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:31:53 smithi167 ceph-mon[21875]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-08-19T01:31:53.717 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:31:53 smithi167 ceph-mon[21875]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-19T01:31:53.905 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:31:53 smithi083 ceph-mon[17120]: pgmap v243: 1 pgs: 1 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-19T01:31:53.905 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:31:53 smithi083 ceph-mon[17120]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' 2024-08-19T01:31:53.905 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:31:53 smithi083 ceph-mon[17120]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-19T01:31:53.905 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:31:53 smithi083 ceph-mon[17120]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-19T01:31:53.905 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:31:53 smithi083 ceph-mon[17120]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-19T01:31:53.905 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:31:53 smithi083 ceph-mon[17120]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' 2024-08-19T01:31:53.905 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:31:53 smithi083 ceph-mon[17120]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-08-19T01:31:53.905 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:31:53 smithi083 ceph-mon[17120]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-08-19T01:31:53.905 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:31:53 smithi083 ceph-mon[17120]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-19T01:31:53.937 DEBUG:teuthology.orchestra.run.smithi083:mds.a> sudo journalctl -f -n 0 -u ceph-bad8780e-5dc9-11ef-bccf-c7b262605968@mds.a.service 2024-08-19T01:31:53.941 DEBUG:teuthology.orchestra.run.smithi083:mds.d> sudo journalctl -f -n 0 -u ceph-bad8780e-5dc9-11ef-bccf-c7b262605968@mds.d.service 2024-08-19T01:31:53.952 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:31:53 smithi117 ceph-mon[22016]: pgmap v243: 1 pgs: 1 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-19T01:31:53.952 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:31:53 smithi117 ceph-mon[22016]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' 2024-08-19T01:31:53.952 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:31:53 smithi117 ceph-mon[22016]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-19T01:31:53.953 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:31:53 smithi117 ceph-mon[22016]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-19T01:31:53.953 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:31:53 smithi117 ceph-mon[22016]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-19T01:31:53.953 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:31:53 smithi117 ceph-mon[22016]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' 2024-08-19T01:31:53.953 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:31:53 smithi117 ceph-mon[22016]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-08-19T01:31:53.953 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:31:53 smithi117 ceph-mon[22016]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-08-19T01:31:53.953 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:31:53 smithi117 ceph-mon[22016]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-19T01:31:53.987 DEBUG:teuthology.orchestra.run.smithi083:mds.g> sudo journalctl -f -n 0 -u ceph-bad8780e-5dc9-11ef-bccf-c7b262605968@mds.g.service 2024-08-19T01:31:54.031 DEBUG:teuthology.orchestra.run.smithi083:mds.j> sudo journalctl -f -n 0 -u ceph-bad8780e-5dc9-11ef-bccf-c7b262605968@mds.j.service 2024-08-19T01:31:54.076 DEBUG:teuthology.orchestra.run.smithi117:mds.b> sudo journalctl -f -n 0 -u ceph-bad8780e-5dc9-11ef-bccf-c7b262605968@mds.b.service 2024-08-19T01:31:54.079 DEBUG:teuthology.orchestra.run.smithi117:mds.e> sudo journalctl -f -n 0 -u ceph-bad8780e-5dc9-11ef-bccf-c7b262605968@mds.e.service 2024-08-19T01:31:54.122 DEBUG:teuthology.orchestra.run.smithi117:mds.h> sudo journalctl -f -n 0 -u ceph-bad8780e-5dc9-11ef-bccf-c7b262605968@mds.h.service 2024-08-19T01:31:54.166 DEBUG:teuthology.orchestra.run.smithi117:mds.k> sudo journalctl -f -n 0 -u ceph-bad8780e-5dc9-11ef-bccf-c7b262605968@mds.k.service 2024-08-19T01:31:54.209 DEBUG:teuthology.orchestra.run.smithi167:mds.c> sudo journalctl -f -n 0 -u ceph-bad8780e-5dc9-11ef-bccf-c7b262605968@mds.c.service 2024-08-19T01:31:54.212 DEBUG:teuthology.orchestra.run.smithi167:mds.f> sudo journalctl -f -n 0 -u ceph-bad8780e-5dc9-11ef-bccf-c7b262605968@mds.f.service 2024-08-19T01:31:54.255 DEBUG:teuthology.orchestra.run.smithi167:mds.i> sudo journalctl -f -n 0 -u ceph-bad8780e-5dc9-11ef-bccf-c7b262605968@mds.i.service 2024-08-19T01:31:54.298 DEBUG:teuthology.orchestra.run.smithi167:mds.l> sudo journalctl -f -n 0 -u ceph-bad8780e-5dc9-11ef-bccf-c7b262605968@mds.l.service 2024-08-19T01:31:54.342 INFO:tasks.cephadm:Setting up CephFS filesystem(s)... 2024-08-19T01:31:54.342 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833 shell --fsid bad8780e-5dc9-11ef-bccf-c7b262605968 -- ceph fs dump --format=json 2024-08-19T01:31:54.655 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:31:54 smithi083 ceph-mon[17120]: from='client.24418 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi083=a;smithi083=d;smithi083=g;smithi083=j;smithi117=b;smithi117=e;smithi117=h;smithi117=k;smithi167=c;smithi167=f;smithi167=i;smithi167=l", "target": ["mon-mgr", ""]}]: dispatch 2024-08-19T01:31:54.655 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:31:54 smithi083 ceph-mon[17120]: Saving service mds.all spec with placement smithi083=a;smithi083=d;smithi083=g;smithi083=j;smithi117=b;smithi117=e;smithi117=h;smithi117=k;smithi167=c;smithi167=f;smithi167=i;smithi167=l;count:12 2024-08-19T01:31:54.655 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:31:54 smithi083 ceph-mon[17120]: Deploying daemon mds.l on smithi167 2024-08-19T01:31:54.712 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/bad8780e-5dc9-11ef-bccf-c7b262605968/mon.a/config 2024-08-19T01:31:54.716 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:31:54 smithi167 ceph-mon[21875]: from='client.24418 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi083=a;smithi083=d;smithi083=g;smithi083=j;smithi117=b;smithi117=e;smithi117=h;smithi117=k;smithi167=c;smithi167=f;smithi167=i;smithi167=l", "target": ["mon-mgr", ""]}]: dispatch 2024-08-19T01:31:54.716 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:31:54 smithi167 ceph-mon[21875]: Saving service mds.all spec with placement smithi083=a;smithi083=d;smithi083=g;smithi083=j;smithi117=b;smithi117=e;smithi117=h;smithi117=k;smithi167=c;smithi167=f;smithi167=i;smithi167=l;count:12 2024-08-19T01:31:54.716 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:31:54 smithi167 ceph-mon[21875]: Deploying daemon mds.l on smithi167 2024-08-19T01:31:54.952 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:31:54 smithi117 ceph-mon[22016]: from='client.24418 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi083=a;smithi083=d;smithi083=g;smithi083=j;smithi117=b;smithi117=e;smithi117=h;smithi117=k;smithi167=c;smithi167=f;smithi167=i;smithi167=l", "target": ["mon-mgr", ""]}]: dispatch 2024-08-19T01:31:54.953 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:31:54 smithi117 ceph-mon[22016]: Saving service mds.all spec with placement smithi083=a;smithi083=d;smithi083=g;smithi083=j;smithi117=b;smithi117=e;smithi117=h;smithi117=k;smithi167=c;smithi167=f;smithi167=i;smithi167=l;count:12 2024-08-19T01:31:54.953 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:31:54 smithi117 ceph-mon[22016]: Deploying daemon mds.l on smithi167 2024-08-19T01:31:55.312 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:31:55.277+0000 7ff2f9800640 1 -- 172.21.15.83:0/2375185367 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff2f4102250 msgr2=0x7ff2f410e780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:31:55.312 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:31:55.277+0000 7ff2f9800640 1 --2- 172.21.15.83:0/2375185367 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff2f4102250 0x7ff2f410e780 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7ff2ec009cd0 tx=0x7ff2ec02a8e0 comp rx=0 tx=0).stop 2024-08-19T01:31:55.313 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:31:55.278+0000 7ff2f9800640 1 -- 172.21.15.83:0/2375185367 shutdown_connections 2024-08-19T01:31:55.313 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:31:55.278+0000 7ff2f9800640 1 --2- 172.21.15.83:0/2375185367 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff2f4102250 0x7ff2f410e780 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:31:55.313 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:31:55.278+0000 7ff2f9800640 1 --2- 172.21.15.83:0/2375185367 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff2f41018b0 0x7ff2f4101d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:31:55.313 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:31:55.278+0000 7ff2f9800640 1 --2- 172.21.15.83:0/2375185367 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff2f41078b0 0x7ff2f4107c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:31:55.313 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:31:55.278+0000 7ff2f9800640 1 -- 172.21.15.83:0/2375185367 >> 172.21.15.83:0/2375185367 conn(0x7ff2f40fd5f0 msgr2=0x7ff2f40ffa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:31:55.313 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:31:55.278+0000 7ff2f9800640 1 -- 172.21.15.83:0/2375185367 shutdown_connections 2024-08-19T01:31:55.313 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:31:55.278+0000 7ff2f9800640 1 -- 172.21.15.83:0/2375185367 wait complete. 2024-08-19T01:31:55.313 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:31:55.279+0000 7ff2f9800640 1 Processor -- start 2024-08-19T01:31:55.313 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:31:55.279+0000 7ff2f9800640 1 -- start start 2024-08-19T01:31:55.313 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:31:55.279+0000 7ff2f9800640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff2f41018b0 0x7ff2f419a310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:31:55.313 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:31:55.279+0000 7ff2f9800640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff2f4102250 0x7ff2f419a850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:31:55.313 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:31:55.279+0000 7ff2f9800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff2f41078b0 0x7ff2f419ebe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:31:55.314 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:31:55.279+0000 7ff2f9800640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7ff2f4111d00 con 0x7ff2f41078b0 2024-08-19T01:31:55.314 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:31:55.279+0000 7ff2f9800640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7ff2f4111b80 con 0x7ff2f41018b0 2024-08-19T01:31:55.314 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:31:55.279+0000 7ff2f9800640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7ff2f4111e80 con 0x7ff2f4102250 2024-08-19T01:31:55.314 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:31:55.279+0000 7ff2f2000640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff2f4102250 0x7ff2f419a850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:31:55.314 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:31:55.279+0000 7ff2f3400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff2f41078b0 0x7ff2f419ebe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:31:55.314 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:31:55.279+0000 7ff2f2a00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff2f41018b0 0x7ff2f419a310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:31:55.314 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:31:55.279+0000 7ff2f2000640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff2f4102250 0x7ff2f419a850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.83:33738/0 (socket says 172.21.15.83:33738) 2024-08-19T01:31:55.314 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:31:55.279+0000 7ff2f2000640 1 -- 172.21.15.83:0/4233401755 learned_addr learned my addr 172.21.15.83:0/4233401755 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-08-19T01:31:55.314 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:31:55.280+0000 7ff2f2000640 1 -- 172.21.15.83:0/4233401755 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff2f41018b0 msgr2=0x7ff2f419a310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:31:55.314 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:31:55.280+0000 7ff2f2000640 1 --2- 172.21.15.83:0/4233401755 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff2f41018b0 0x7ff2f419a310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:31:55.314 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:31:55.280+0000 7ff2f2000640 1 -- 172.21.15.83:0/4233401755 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff2f41078b0 msgr2=0x7ff2f419ebe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:31:55.314 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:31:55.280+0000 7ff2f2000640 1 --2- 172.21.15.83:0/4233401755 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff2f41078b0 0x7ff2f419ebe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:31:55.314 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:31:55.280+0000 7ff2f2000640 1 -- 172.21.15.83:0/4233401755 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff2f419f2c0 con 0x7ff2f4102250 2024-08-19T01:31:55.315 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:31:55.280+0000 7ff2f2000640 1 --2- 172.21.15.83:0/4233401755 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff2f4102250 0x7ff2f419a850 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7ff2e000d950 tx=0x7ff2e000de10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:31:55.315 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:31:55.280+0000 7ff2eb400640 1 -- 172.21.15.83:0/4233401755 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2e0017030 con 0x7ff2f4102250 2024-08-19T01:31:55.315 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:31:55.280+0000 7ff2f9800640 1 -- 172.21.15.83:0/4233401755 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff2f419f5b0 con 0x7ff2f4102250 2024-08-19T01:31:55.315 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:31:55.280+0000 7ff2eb400640 1 -- 172.21.15.83:0/4233401755 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff2e0004020 con 0x7ff2f4102250 2024-08-19T01:31:55.315 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:31:55.280+0000 7ff2eb400640 1 -- 172.21.15.83:0/4233401755 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2e00047a0 con 0x7ff2f4102250 2024-08-19T01:31:55.315 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:31:55.281+0000 7ff2f9800640 1 -- 172.21.15.83:0/4233401755 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff2f41a6e90 con 0x7ff2f4102250 2024-08-19T01:31:55.315 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:31:55.282+0000 7ff2f9800640 1 -- 172.21.15.83:0/4233401755 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff2f4101d10 con 0x7ff2f4102250 2024-08-19T01:31:55.315 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:31:55.284+0000 7ff2eb400640 1 -- 172.21.15.83:0/4233401755 <== mon.1 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7ff2e00041d0 con 0x7ff2f4102250 2024-08-19T01:31:55.315 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:31:55.285+0000 7ff2eb400640 1 --2- 172.21.15.83:0/4233401755 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7ff2d80a2b80 0x7ff2d80a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:31:55.316 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:31:55.286+0000 7ff2eb400640 1 -- 172.21.15.83:0/4233401755 <== mon.1 v2:172.21.15.167:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 7127+0+0 (secure 0 0 0) 0x7ff2e00ce840 con 0x7ff2f4102250 2024-08-19T01:31:55.316 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:31:55.286+0000 7ff2f2a00640 1 --2- 172.21.15.83:0/4233401755 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7ff2d80a2b80 0x7ff2d80a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:31:55.316 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:31:55.286+0000 7ff2f2a00640 1 --2- 172.21.15.83:0/4233401755 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7ff2d80a2b80 0x7ff2d80a5040 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7ff2e4009b50 tx=0x7ff2e4005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:31:55.316 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:31:55.287+0000 7ff2eb400640 1 -- 172.21.15.83:0/4233401755 <== mon.1 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7ff2e009bba0 con 0x7ff2f4102250 2024-08-19T01:31:55.498 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:31:55.497+0000 7ff2f9800640 1 -- 172.21.15.83:0/4233401755 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff2f419b670 con 0x7ff2f4102250 2024-08-19T01:31:55.498 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:31:55.497+0000 7ff2eb400640 1 -- 172.21.15.83:0/4233401755 <== mon.1 v2:172.21.15.167:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 1 v1) ==== 93+0+611 (secure 0 0 0) 0x7ff2f419b670 con 0x7ff2f4102250 2024-08-19T01:31:55.498 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-08-19T01:31:55.498 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":1,"btime":"2024-08-19T01:25:12:626585+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[]} 2024-08-19T01:31:55.498 INFO:teuthology.orchestra.run.smithi083.stderr:dumped fsmap epoch 1 2024-08-19T01:31:55.501 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:31:55.500+0000 7ff2f9800640 1 -- 172.21.15.83:0/4233401755 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7ff2d80a2b80 msgr2=0x7ff2d80a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:31:55.501 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:31:55.500+0000 7ff2f9800640 1 --2- 172.21.15.83:0/4233401755 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7ff2d80a2b80 0x7ff2d80a5040 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7ff2e4009b50 tx=0x7ff2e4005f90 comp rx=0 tx=0).stop 2024-08-19T01:31:55.501 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:31:55.500+0000 7ff2f9800640 1 -- 172.21.15.83:0/4233401755 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff2f4102250 msgr2=0x7ff2f419a850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:31:55.501 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:31:55.500+0000 7ff2f9800640 1 --2- 172.21.15.83:0/4233401755 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff2f4102250 0x7ff2f419a850 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7ff2e000d950 tx=0x7ff2e000de10 comp rx=0 tx=0).stop 2024-08-19T01:31:55.502 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:31:55.500+0000 7ff2f9800640 1 -- 172.21.15.83:0/4233401755 shutdown_connections 2024-08-19T01:31:55.502 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:31:55.500+0000 7ff2f9800640 1 --2- 172.21.15.83:0/4233401755 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7ff2d80a2b80 0x7ff2d80a5040 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:31:55.502 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:31:55.500+0000 7ff2f9800640 1 --2- 172.21.15.83:0/4233401755 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff2f41078b0 0x7ff2f419ebe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:31:55.502 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:31:55.500+0000 7ff2f9800640 1 --2- 172.21.15.83:0/4233401755 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff2f4102250 0x7ff2f419a850 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:31:55.502 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:31:55.500+0000 7ff2f9800640 1 --2- 172.21.15.83:0/4233401755 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff2f41018b0 0x7ff2f419a310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:31:55.502 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:31:55.500+0000 7ff2f9800640 1 -- 172.21.15.83:0/4233401755 >> 172.21.15.83:0/4233401755 conn(0x7ff2f40fd5f0 msgr2=0x7ff2f40febd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:31:55.502 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:31:55.501+0000 7ff2f9800640 1 -- 172.21.15.83:0/4233401755 shutdown_connections 2024-08-19T01:31:55.502 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:31:55.501+0000 7ff2f9800640 1 -- 172.21.15.83:0/4233401755 wait complete. 2024-08-19T01:31:55.716 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:31:55 smithi167 ceph-mon[21875]: pgmap v244: 1 pgs: 1 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-19T01:31:55.716 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:31:55 smithi167 ceph-mon[21875]: from='client.? 172.21.15.83:0/4233401755' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-19T01:31:55.717 INFO:journalctl@ceph.mds.l.smithi167.stdout:Aug 19 01:31:55 smithi167 systemd[1]: Starting Ceph mds.l for bad8780e-5dc9-11ef-bccf-c7b262605968... 2024-08-19T01:31:55.863 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:31:55 smithi083 ceph-mon[17120]: pgmap v244: 1 pgs: 1 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-19T01:31:55.863 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:31:55 smithi083 ceph-mon[17120]: from='client.? 172.21.15.83:0/4233401755' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-19T01:31:55.952 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:31:55 smithi117 ceph-mon[22016]: pgmap v244: 1 pgs: 1 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-19T01:31:55.953 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:31:55 smithi117 ceph-mon[22016]: from='client.? 172.21.15.83:0/4233401755' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-19T01:31:56.040 INFO:journalctl@ceph.mds.l.smithi167.stdout:Aug 19 01:31:55 smithi167 podman[51284]: 2024-08-19 01:31:55.671015089 +0000 UTC m=+0.016716956 image pull ea7a5189f30bad3aca9524788e2965d2368d7ce786df09d4edcd8c3edf9cefba quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c9dd28080a631be25c77c36c63b17a3ef340ffb2e21c87d4d7b56b6398436d9 2024-08-19T01:31:56.040 INFO:journalctl@ceph.mds.l.smithi167.stdout:Aug 19 01:31:55 smithi167 podman[51284]: 2024-08-19 01:31:55.802543254 +0000 UTC m=+0.148245104 container create 3d38eebe4a75426e5a17ca0f3c599c4447342abd046d26b3f0ee8c9410644187 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c9dd28080a631be25c77c36c63b17a3ef340ffb2e21c87d4d7b56b6398436d9, name=ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-mds-l, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, RELEASE=squid-de09dd0, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , io.buildah.version=1.36.0, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_CLEAN=True) 2024-08-19T01:31:56.040 INFO:journalctl@ceph.mds.l.smithi167.stdout:Aug 19 01:31:56 smithi167 podman[51284]: 2024-08-19 01:31:56.013407519 +0000 UTC m=+0.359109383 container init 3d38eebe4a75426e5a17ca0f3c599c4447342abd046d26b3f0ee8c9410644187 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c9dd28080a631be25c77c36c63b17a3ef340ffb2e21c87d4d7b56b6398436d9, name=ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-mds-l, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, RELEASE=squid-de09dd0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , io.buildah.version=1.36.0, GIT_BRANCH=HEAD) 2024-08-19T01:31:56.040 INFO:journalctl@ceph.mds.l.smithi167.stdout:Aug 19 01:31:56 smithi167 podman[51284]: 2024-08-19 01:31:56.016062561 +0000 UTC m=+0.361764419 container start 3d38eebe4a75426e5a17ca0f3c599c4447342abd046d26b3f0ee8c9410644187 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c9dd28080a631be25c77c36c63b17a3ef340ffb2e21c87d4d7b56b6398436d9, name=ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-mds-l, io.buildah.version=1.36.0, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, RELEASE=squid-de09dd0, ceph=True) 2024-08-19T01:31:56.466 INFO:journalctl@ceph.mds.l.smithi167.stdout:Aug 19 01:31:56 smithi167 ceph-mds[51303]: -- 172.21.15.167:0/1212566649 <== mon.1 v2:172.21.15.167:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x562ddc1b5860 con 0x562ddc192800 2024-08-19T01:31:56.466 INFO:journalctl@ceph.mds.l.smithi167.stdout:Aug 19 01:31:56 smithi167 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-mds-l[51299]: starting mds.l at 2024-08-19T01:31:56.466 INFO:journalctl@ceph.mds.l.smithi167.stdout:Aug 19 01:31:56 smithi167 bash[51284]: 3d38eebe4a75426e5a17ca0f3c599c4447342abd046d26b3f0ee8c9410644187 2024-08-19T01:31:56.466 INFO:journalctl@ceph.mds.l.smithi167.stdout:Aug 19 01:31:56 smithi167 systemd[1]: Started Ceph mds.l for bad8780e-5dc9-11ef-bccf-c7b262605968. 2024-08-19T01:31:57.202 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:31:57 smithi117 ceph-mon[22016]: mds.? [v2:172.21.15.167:6832/2505076046,v1:172.21.15.167:6833/2505076046] up:boot 2024-08-19T01:31:57.202 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:31:57 smithi117 ceph-mon[22016]: fsmap 1 up:standby 2024-08-19T01:31:57.203 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:31:57 smithi117 ceph-mon[22016]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-08-19T01:31:57.203 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:31:57 smithi117 ceph-mon[22016]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' 2024-08-19T01:31:57.203 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:31:57 smithi117 ceph-mon[22016]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' 2024-08-19T01:31:57.203 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:31:57 smithi117 ceph-mon[22016]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' 2024-08-19T01:31:57.203 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:31:57 smithi117 ceph-mon[22016]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-08-19T01:31:57.203 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:31:57 smithi117 ceph-mon[22016]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-08-19T01:31:57.203 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:31:57 smithi117 ceph-mon[22016]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-19T01:31:57.203 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:31:57 smithi117 ceph-mon[22016]: Deploying daemon mds.k on smithi117 2024-08-19T01:31:57.203 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:31:57 smithi117 ceph-mon[22016]: pgmap v245: 1 pgs: 1 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-19T01:31:57.405 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:31:57 smithi083 ceph-mon[17120]: mds.? [v2:172.21.15.167:6832/2505076046,v1:172.21.15.167:6833/2505076046] up:boot 2024-08-19T01:31:57.405 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:31:57 smithi083 ceph-mon[17120]: fsmap 1 up:standby 2024-08-19T01:31:57.405 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:31:57 smithi083 ceph-mon[17120]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-08-19T01:31:57.405 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:31:57 smithi083 ceph-mon[17120]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' 2024-08-19T01:31:57.405 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:31:57 smithi083 ceph-mon[17120]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' 2024-08-19T01:31:57.405 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:31:57 smithi083 ceph-mon[17120]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' 2024-08-19T01:31:57.405 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:31:57 smithi083 ceph-mon[17120]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-08-19T01:31:57.405 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:31:57 smithi083 ceph-mon[17120]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-08-19T01:31:57.405 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:31:57 smithi083 ceph-mon[17120]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-19T01:31:57.406 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:31:57 smithi083 ceph-mon[17120]: Deploying daemon mds.k on smithi117 2024-08-19T01:31:57.406 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:31:57 smithi083 ceph-mon[17120]: pgmap v245: 1 pgs: 1 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-19T01:31:57.466 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:31:57 smithi167 ceph-mon[21875]: mds.? [v2:172.21.15.167:6832/2505076046,v1:172.21.15.167:6833/2505076046] up:boot 2024-08-19T01:31:57.467 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:31:57 smithi167 ceph-mon[21875]: fsmap 1 up:standby 2024-08-19T01:31:57.467 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:31:57 smithi167 ceph-mon[21875]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-08-19T01:31:57.467 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:31:57 smithi167 ceph-mon[21875]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' 2024-08-19T01:31:57.467 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:31:57 smithi167 ceph-mon[21875]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' 2024-08-19T01:31:57.467 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:31:57 smithi167 ceph-mon[21875]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' 2024-08-19T01:31:57.467 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:31:57 smithi167 ceph-mon[21875]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-08-19T01:31:57.467 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:31:57 smithi167 ceph-mon[21875]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-08-19T01:31:57.467 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:31:57 smithi167 ceph-mon[21875]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-19T01:31:57.467 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:31:57 smithi167 ceph-mon[21875]: Deploying daemon mds.k on smithi117 2024-08-19T01:31:57.468 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:31:57 smithi167 ceph-mon[21875]: pgmap v245: 1 pgs: 1 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-19T01:31:58.201 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833 shell --fsid bad8780e-5dc9-11ef-bccf-c7b262605968 -- ceph fs dump --format=json 2024-08-19T01:31:58.202 INFO:journalctl@ceph.mds.k.smithi117.stdout:Aug 19 01:31:57 smithi117 systemd[1]: Starting Ceph mds.k for bad8780e-5dc9-11ef-bccf-c7b262605968... 2024-08-19T01:31:58.522 INFO:journalctl@ceph.mds.k.smithi117.stdout:Aug 19 01:31:58 smithi117 podman[50960]: 2024-08-19 01:31:58.112386254 +0000 UTC m=+0.017229102 image pull ea7a5189f30bad3aca9524788e2965d2368d7ce786df09d4edcd8c3edf9cefba quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c9dd28080a631be25c77c36c63b17a3ef340ffb2e21c87d4d7b56b6398436d9 2024-08-19T01:31:58.522 INFO:journalctl@ceph.mds.k.smithi117.stdout:Aug 19 01:31:58 smithi117 podman[50960]: 2024-08-19 01:31:58.218606744 +0000 UTC m=+0.123449577 container create 13892578aec1ff716a77dc2b13bf84642671ed95c4a3995b43855be57447b37f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c9dd28080a631be25c77c36c63b17a3ef340ffb2e21c87d4d7b56b6398436d9, name=ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-mds-k, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , io.buildah.version=1.36.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, ceph=True, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-de09dd0) 2024-08-19T01:31:58.522 INFO:journalctl@ceph.mds.k.smithi117.stdout:Aug 19 01:31:58 smithi117 podman[50960]: 2024-08-19 01:31:58.429313627 +0000 UTC m=+0.334156460 container init 13892578aec1ff716a77dc2b13bf84642671ed95c4a3995b43855be57447b37f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c9dd28080a631be25c77c36c63b17a3ef340ffb2e21c87d4d7b56b6398436d9, name=ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-mds-k, org.label-schema.license=GPLv2, io.buildah.version=1.36.0, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , ceph=True, RELEASE=squid-de09dd0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS) 2024-08-19T01:31:58.522 INFO:journalctl@ceph.mds.k.smithi117.stdout:Aug 19 01:31:58 smithi117 podman[50960]: 2024-08-19 01:31:58.432193666 +0000 UTC m=+0.337036500 container start 13892578aec1ff716a77dc2b13bf84642671ed95c4a3995b43855be57447b37f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c9dd28080a631be25c77c36c63b17a3ef340ffb2e21c87d4d7b56b6398436d9, name=ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-mds-k, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , RELEASE=squid-de09dd0, GIT_CLEAN=True, GIT_BRANCH=HEAD, io.buildah.version=1.36.0, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, ceph=True) 2024-08-19T01:31:58.522 INFO:journalctl@ceph.mds.k.smithi117.stdout:Aug 19 01:31:58 smithi117 ceph-mds[50979]: -- 172.21.15.117:0/3518539354 <== mon.2 v2:172.21.15.117:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55a936c45860 con 0x55a936c23000 2024-08-19T01:31:58.522 INFO:journalctl@ceph.mds.k.smithi117.stdout:Aug 19 01:31:58 smithi117 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-mds-k[50975]: starting mds.k at 2024-08-19T01:31:58.522 INFO:journalctl@ceph.mds.k.smithi117.stdout:Aug 19 01:31:58 smithi117 bash[50960]: 13892578aec1ff716a77dc2b13bf84642671ed95c4a3995b43855be57447b37f 2024-08-19T01:31:58.582 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/bad8780e-5dc9-11ef-bccf-c7b262605968/mon.a/config 2024-08-19T01:31:58.858 INFO:journalctl@ceph.mds.k.smithi117.stdout:Aug 19 01:31:58 smithi117 systemd[1]: Started Ceph mds.k for bad8780e-5dc9-11ef-bccf-c7b262605968. 2024-08-19T01:31:59.111 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:31:59.109+0000 7fafaf400640 1 -- 172.21.15.83:0/2288910754 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fafa80742c0 msgr2=0x7fafa80746a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:31:59.111 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:31:59.109+0000 7fafaf400640 1 --2- 172.21.15.83:0/2288910754 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fafa80742c0 0x7fafa80746a0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7faf98009900 tx=0x7faf9802a490 comp rx=0 tx=0).stop 2024-08-19T01:31:59.112 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:31:59.110+0000 7fafaf400640 1 -- 172.21.15.83:0/2288910754 shutdown_connections 2024-08-19T01:31:59.112 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:31:59.110+0000 7fafaf400640 1 --2- 172.21.15.83:0/2288910754 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fafa81128c0 0x7fafa8114cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:31:59.112 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:31:59.110+0000 7fafaf400640 1 --2- 172.21.15.83:0/2288910754 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fafa8074be0 0x7fafa8112380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:31:59.112 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:31:59.110+0000 7fafaf400640 1 --2- 172.21.15.83:0/2288910754 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fafa80742c0 0x7fafa80746a0 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:31:59.112 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:31:59.110+0000 7fafaf400640 1 -- 172.21.15.83:0/2288910754 >> 172.21.15.83:0/2288910754 conn(0x7fafa80fd5f0 msgr2=0x7fafa80ffa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:31:59.113 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:31:59.110+0000 7fafaf400640 1 -- 172.21.15.83:0/2288910754 shutdown_connections 2024-08-19T01:31:59.113 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:31:59.110+0000 7fafaf400640 1 -- 172.21.15.83:0/2288910754 wait complete. 2024-08-19T01:31:59.113 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:31:59.111+0000 7fafaf400640 1 Processor -- start 2024-08-19T01:31:59.115 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:31:59.111+0000 7fafaf400640 1 -- start start 2024-08-19T01:31:59.115 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:31:59.111+0000 7fafaf400640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fafa80742c0 0x7fafa81a4400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:31:59.115 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:31:59.111+0000 7fafaf400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fafa8074be0 0x7fafa81a4940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:31:59.115 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:31:59.111+0000 7fafaf400640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fafa81128c0 0x7fafa819e5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:31:59.115 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:31:59.111+0000 7fafaf400640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fafa81182d0 con 0x7fafa8074be0 2024-08-19T01:31:59.115 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:31:59.111+0000 7fafaf400640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fafa8118150 con 0x7fafa80742c0 2024-08-19T01:31:59.115 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:31:59.111+0000 7fafaf400640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fafa8118450 con 0x7fafa81128c0 2024-08-19T01:31:59.115 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:31:59.112+0000 7fafa7e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fafa8074be0 0x7fafa81a4940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:31:59.115 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:31:59.112+0000 7fafaca00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fafa80742c0 0x7fafa81a4400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:31:59.115 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:31:59.112+0000 7fafaca00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fafa80742c0 0x7fafa81a4400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.83:39326/0 (socket says 172.21.15.83:39326) 2024-08-19T01:31:59.116 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:31:59.112+0000 7fafa7e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fafa8074be0 0x7fafa81a4940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:45952/0 (socket says 172.21.15.83:45952) 2024-08-19T01:31:59.116 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:31:59.112+0000 7fafaca00640 1 -- 172.21.15.83:0/3189078659 learned_addr learned my addr 172.21.15.83:0/3189078659 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-08-19T01:31:59.116 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:31:59.112+0000 7fafad400640 1 --2- 172.21.15.83:0/3189078659 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fafa81128c0 0x7fafa819e5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:31:59.116 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:31:59.112+0000 7fafaca00640 1 -- 172.21.15.83:0/3189078659 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fafa81128c0 msgr2=0x7fafa819e5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:31:59.116 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:31:59.112+0000 7fafaca00640 1 --2- 172.21.15.83:0/3189078659 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fafa81128c0 0x7fafa819e5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:31:59.116 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:31:59.112+0000 7fafaca00640 1 -- 172.21.15.83:0/3189078659 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fafa8074be0 msgr2=0x7fafa81a4940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:31:59.116 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:31:59.112+0000 7fafaca00640 1 --2- 172.21.15.83:0/3189078659 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fafa8074be0 0x7fafa81a4940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:31:59.116 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:31:59.112+0000 7fafaca00640 1 -- 172.21.15.83:0/3189078659 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fafa819ecf0 con 0x7fafa80742c0 2024-08-19T01:31:59.116 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:31:59.112+0000 7fafaca00640 1 --2- 172.21.15.83:0/3189078659 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fafa80742c0 0x7fafa81a4400 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7faf98004770 tx=0x7faf98003c10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:31:59.116 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:31:59.113+0000 7fafa5600640 1 -- 172.21.15.83:0/3189078659 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf9803f070 con 0x7fafa80742c0 2024-08-19T01:31:59.116 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:31:59.113+0000 7fafa5600640 1 -- 172.21.15.83:0/3189078659 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7faf98031ca0 con 0x7fafa80742c0 2024-08-19T01:31:59.116 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:31:59.113+0000 7fafa5600640 1 -- 172.21.15.83:0/3189078659 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf98035030 con 0x7fafa80742c0 2024-08-19T01:31:59.116 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:31:59.113+0000 7fafaf400640 1 -- 172.21.15.83:0/3189078659 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fafa819ef80 con 0x7fafa80742c0 2024-08-19T01:31:59.116 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:31:59.113+0000 7fafaf400640 1 -- 172.21.15.83:0/3189078659 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fafa81ab2d0 con 0x7fafa80742c0 2024-08-19T01:31:59.117 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:31:59.114+0000 7fafaf400640 1 -- 172.21.15.83:0/3189078659 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fafa80746a0 con 0x7fafa80742c0 2024-08-19T01:31:59.117 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:31:59.116+0000 7fafa5600640 1 -- 172.21.15.83:0/3189078659 <== mon.2 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7faf98042070 con 0x7fafa80742c0 2024-08-19T01:31:59.118 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:31:59.117+0000 7fafa5600640 1 --2- 172.21.15.83:0/3189078659 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7faf8c0a2b80 0x7faf8c0a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:31:59.118 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:31:59.117+0000 7fafa5600640 1 -- 172.21.15.83:0/3189078659 <== mon.2 v2:172.21.15.117:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 7127+0+0 (secure 0 0 0) 0x7faf980ef880 con 0x7fafa80742c0 2024-08-19T01:31:59.118 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:31:59.117+0000 7fafa7e00640 1 --2- 172.21.15.83:0/3189078659 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7faf8c0a2b80 0x7faf8c0a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:31:59.118 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:31:59.118+0000 7fafa7e00640 1 --2- 172.21.15.83:0/3189078659 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7faf8c0a2b80 0x7faf8c0a5040 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fafa80ffc80 tx=0x7faf94005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:31:59.119 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:31:59.119+0000 7fafa5600640 1 -- 172.21.15.83:0/3189078659 <== mon.2 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7faf980bc590 con 0x7fafa80742c0 2024-08-19T01:31:59.368 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:31:59.368+0000 7fafaf400640 1 -- 172.21.15.83:0/3189078659 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fafa81ab4a0 con 0x7fafa80742c0 2024-08-19T01:31:59.369 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:31:59.368+0000 7fafa5600640 1 -- 172.21.15.83:0/3189078659 <== mon.2 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 2 v2) ==== 93+0+1438 (secure 0 0 0) 0x7faf980c0890 con 0x7fafa80742c0 2024-08-19T01:31:59.369 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-08-19T01:31:59.369 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":2,"btime":"2024-08-19T01:31:56:044306+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24430,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6833/2505076046","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6832","nonce":2505076046},{"type":"v1","addr":"172.21.15.167:6833","nonce":2505076046}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[]} 2024-08-19T01:31:59.370 INFO:teuthology.orchestra.run.smithi083.stderr:dumped fsmap epoch 2 2024-08-19T01:31:59.372 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:31:59.371+0000 7fafaf400640 1 -- 172.21.15.83:0/3189078659 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7faf8c0a2b80 msgr2=0x7faf8c0a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:31:59.372 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:31:59.371+0000 7fafaf400640 1 --2- 172.21.15.83:0/3189078659 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7faf8c0a2b80 0x7faf8c0a5040 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fafa80ffc80 tx=0x7faf94005f90 comp rx=0 tx=0).stop 2024-08-19T01:31:59.372 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:31:59.371+0000 7fafaf400640 1 -- 172.21.15.83:0/3189078659 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fafa80742c0 msgr2=0x7fafa81a4400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:31:59.372 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:31:59.371+0000 7fafaf400640 1 --2- 172.21.15.83:0/3189078659 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fafa80742c0 0x7fafa81a4400 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7faf98004770 tx=0x7faf98003c10 comp rx=0 tx=0).stop 2024-08-19T01:31:59.372 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:31:59.371+0000 7fafaf400640 1 -- 172.21.15.83:0/3189078659 shutdown_connections 2024-08-19T01:31:59.373 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:31:59.371+0000 7fafaf400640 1 --2- 172.21.15.83:0/3189078659 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7faf8c0a2b80 0x7faf8c0a5040 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:31:59.373 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:31:59.371+0000 7fafaf400640 1 --2- 172.21.15.83:0/3189078659 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fafa81128c0 0x7fafa819e5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:31:59.373 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:31:59.371+0000 7fafaf400640 1 --2- 172.21.15.83:0/3189078659 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fafa8074be0 0x7fafa81a4940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:31:59.373 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:31:59.371+0000 7fafaf400640 1 --2- 172.21.15.83:0/3189078659 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fafa80742c0 0x7fafa81a4400 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:31:59.373 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:31:59.371+0000 7fafaf400640 1 -- 172.21.15.83:0/3189078659 >> 172.21.15.83:0/3189078659 conn(0x7fafa80fd5f0 msgr2=0x7fafa80fece0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:31:59.373 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:31:59.371+0000 7fafaf400640 1 -- 172.21.15.83:0/3189078659 shutdown_connections 2024-08-19T01:31:59.373 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:31:59.372+0000 7fafaf400640 1 -- 172.21.15.83:0/3189078659 wait complete. 2024-08-19T01:31:59.655 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:31:59 smithi083 ceph-mon[17120]: pgmap v246: 1 pgs: 1 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-19T01:31:59.655 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:31:59 smithi083 ceph-mon[17120]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' 2024-08-19T01:31:59.655 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:31:59 smithi083 ceph-mon[17120]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' 2024-08-19T01:31:59.655 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:31:59 smithi083 ceph-mon[17120]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' 2024-08-19T01:31:59.655 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:31:59 smithi083 ceph-mon[17120]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-08-19T01:31:59.655 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:31:59 smithi083 ceph-mon[17120]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-08-19T01:31:59.656 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:31:59 smithi083 ceph-mon[17120]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-19T01:31:59.656 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:31:59 smithi083 ceph-mon[17120]: from='client.? 172.21.15.83:0/3189078659' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-19T01:31:59.702 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:31:59 smithi117 ceph-mon[22016]: pgmap v246: 1 pgs: 1 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-19T01:31:59.703 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:31:59 smithi117 ceph-mon[22016]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' 2024-08-19T01:31:59.703 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:31:59 smithi117 ceph-mon[22016]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' 2024-08-19T01:31:59.703 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:31:59 smithi117 ceph-mon[22016]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' 2024-08-19T01:31:59.703 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:31:59 smithi117 ceph-mon[22016]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-08-19T01:31:59.703 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:31:59 smithi117 ceph-mon[22016]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-08-19T01:31:59.703 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:31:59 smithi117 ceph-mon[22016]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-19T01:31:59.703 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:31:59 smithi117 ceph-mon[22016]: from='client.? 172.21.15.83:0/3189078659' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-19T01:31:59.716 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:31:59 smithi167 ceph-mon[21875]: pgmap v246: 1 pgs: 1 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-19T01:31:59.716 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:31:59 smithi167 ceph-mon[21875]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' 2024-08-19T01:31:59.716 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:31:59 smithi167 ceph-mon[21875]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' 2024-08-19T01:31:59.716 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:31:59 smithi167 ceph-mon[21875]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' 2024-08-19T01:31:59.717 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:31:59 smithi167 ceph-mon[21875]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-08-19T01:31:59.717 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:31:59 smithi167 ceph-mon[21875]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-08-19T01:31:59.717 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:31:59 smithi167 ceph-mon[21875]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-19T01:31:59.717 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:31:59 smithi167 ceph-mon[21875]: from='client.? 172.21.15.83:0/3189078659' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-19T01:32:00.654 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:32:00 smithi083 ceph-mon[17120]: Deploying daemon mds.j on smithi083 2024-08-19T01:32:00.655 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:32:00 smithi083 ceph-mon[17120]: mds.? [v2:172.21.15.117:6832/3557520478,v1:172.21.15.117:6833/3557520478] up:boot 2024-08-19T01:32:00.655 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:32:00 smithi083 ceph-mon[17120]: fsmap 2 up:standby 2024-08-19T01:32:00.655 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:32:00 smithi083 ceph-mon[17120]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-08-19T01:32:00.702 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:32:00 smithi117 ceph-mon[22016]: Deploying daemon mds.j on smithi083 2024-08-19T01:32:00.702 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:32:00 smithi117 ceph-mon[22016]: mds.? [v2:172.21.15.117:6832/3557520478,v1:172.21.15.117:6833/3557520478] up:boot 2024-08-19T01:32:00.703 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:32:00 smithi117 ceph-mon[22016]: fsmap 2 up:standby 2024-08-19T01:32:00.703 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:32:00 smithi117 ceph-mon[22016]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-08-19T01:32:00.716 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:32:00 smithi167 ceph-mon[21875]: Deploying daemon mds.j on smithi083 2024-08-19T01:32:00.716 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:32:00 smithi167 ceph-mon[21875]: mds.? [v2:172.21.15.117:6832/3557520478,v1:172.21.15.117:6833/3557520478] up:boot 2024-08-19T01:32:00.716 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:32:00 smithi167 ceph-mon[21875]: fsmap 2 up:standby 2024-08-19T01:32:00.716 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:32:00 smithi167 ceph-mon[21875]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-08-19T01:32:01.654 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:32:01 smithi083 ceph-mon[17120]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-19T01:32:01.702 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:32:01 smithi117 ceph-mon[22016]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-19T01:32:01.716 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:32:01 smithi167 ceph-mon[21875]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-19T01:32:02.334 INFO:journalctl@ceph.mds.j.smithi083.stdout:Aug 19 01:32:02 smithi083 systemd[1]: Starting Ceph mds.j for bad8780e-5dc9-11ef-bccf-c7b262605968... 2024-08-19T01:32:02.636 INFO:journalctl@ceph.mds.j.smithi083.stdout:Aug 19 01:32:02 smithi083 podman[56457]: 2024-08-19 01:32:02.332672321 +0000 UTC m=+0.109606842 container create 86e4712742a36404d69bb63aebdf070527cdb6d6442eda3ddb27ae5ee651a42e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c9dd28080a631be25c77c36c63b17a3ef340ffb2e21c87d4d7b56b6398436d9, name=ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-mds-j, RELEASE=squid-de09dd0, io.buildah.version=1.36.0, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, ceph=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=) 2024-08-19T01:32:02.637 INFO:journalctl@ceph.mds.j.smithi083.stdout:Aug 19 01:32:02 smithi083 podman[56457]: 2024-08-19 01:32:02.252504478 +0000 UTC m=+0.029439016 image pull ea7a5189f30bad3aca9524788e2965d2368d7ce786df09d4edcd8c3edf9cefba quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c9dd28080a631be25c77c36c63b17a3ef340ffb2e21c87d4d7b56b6398436d9 2024-08-19T01:32:02.637 INFO:journalctl@ceph.mds.j.smithi083.stdout:Aug 19 01:32:02 smithi083 podman[56457]: 2024-08-19 01:32:02.63548026 +0000 UTC m=+0.412414781 container init 86e4712742a36404d69bb63aebdf070527cdb6d6442eda3ddb27ae5ee651a42e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c9dd28080a631be25c77c36c63b17a3ef340ffb2e21c87d4d7b56b6398436d9, name=ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-mds-j, GIT_CLEAN=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , RELEASE=squid-de09dd0, io.buildah.version=1.36.0, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, ceph=True) 2024-08-19T01:32:02.905 INFO:journalctl@ceph.mds.j.smithi083.stdout:Aug 19 01:32:02 smithi083 podman[56457]: 2024-08-19 01:32:02.641843155 +0000 UTC m=+0.418777676 container start 86e4712742a36404d69bb63aebdf070527cdb6d6442eda3ddb27ae5ee651a42e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c9dd28080a631be25c77c36c63b17a3ef340ffb2e21c87d4d7b56b6398436d9, name=ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-mds-j, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-de09dd0, org.label-schema.build-date=20240731, io.buildah.version=1.36.0, CEPH_POINT_RELEASE=, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-08-19T01:32:02.905 INFO:journalctl@ceph.mds.j.smithi083.stdout:Aug 19 01:32:02 smithi083 ceph-mds[56476]: -- 172.21.15.83:0/4176861290 <== mon.0 v2:172.21.15.83:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x557bda591860 con 0x557bda56e800 2024-08-19T01:32:02.905 INFO:journalctl@ceph.mds.j.smithi083.stdout:Aug 19 01:32:02 smithi083 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-mds-j[56472]: starting mds.j at 2024-08-19T01:32:02.905 INFO:journalctl@ceph.mds.j.smithi083.stdout:Aug 19 01:32:02 smithi083 bash[56457]: 86e4712742a36404d69bb63aebdf070527cdb6d6442eda3ddb27ae5ee651a42e 2024-08-19T01:32:02.905 INFO:journalctl@ceph.mds.j.smithi083.stdout:Aug 19 01:32:02 smithi083 systemd[1]: Started Ceph mds.j for bad8780e-5dc9-11ef-bccf-c7b262605968. 2024-08-19T01:32:02.971 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833 shell --fsid bad8780e-5dc9-11ef-bccf-c7b262605968 -- ceph fs dump --format=json 2024-08-19T01:32:03.368 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/bad8780e-5dc9-11ef-bccf-c7b262605968/mon.a/config 2024-08-19T01:32:03.689 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:32:03 smithi083 ceph-mon[17120]: pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-19T01:32:03.689 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:32:03 smithi083 ceph-mon[17120]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' 2024-08-19T01:32:03.689 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:32:03 smithi083 ceph-mon[17120]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' 2024-08-19T01:32:03.689 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:32:03 smithi083 ceph-mon[17120]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' 2024-08-19T01:32:03.689 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:32:03 smithi083 ceph-mon[17120]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-08-19T01:32:03.689 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:32:03 smithi083 ceph-mon[17120]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-08-19T01:32:03.689 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:32:03 smithi083 ceph-mon[17120]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-19T01:32:03.703 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:32:03 smithi117 ceph-mon[22016]: pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-19T01:32:03.703 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:32:03 smithi117 ceph-mon[22016]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' 2024-08-19T01:32:03.703 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:32:03 smithi117 ceph-mon[22016]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' 2024-08-19T01:32:03.703 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:32:03 smithi117 ceph-mon[22016]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' 2024-08-19T01:32:03.703 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:32:03 smithi117 ceph-mon[22016]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-08-19T01:32:03.703 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:32:03 smithi117 ceph-mon[22016]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-08-19T01:32:03.703 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:32:03 smithi117 ceph-mon[22016]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-19T01:32:03.716 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:32:03 smithi167 ceph-mon[21875]: pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-19T01:32:03.716 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:32:03 smithi167 ceph-mon[21875]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' 2024-08-19T01:32:03.716 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:32:03 smithi167 ceph-mon[21875]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' 2024-08-19T01:32:03.716 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:32:03 smithi167 ceph-mon[21875]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' 2024-08-19T01:32:03.716 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:32:03 smithi167 ceph-mon[21875]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-08-19T01:32:03.716 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:32:03 smithi167 ceph-mon[21875]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-08-19T01:32:03.716 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:32:03 smithi167 ceph-mon[21875]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-19T01:32:04.160 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:04.158+0000 7f1baf200640 1 -- 172.21.15.83:0/2640345393 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1bb01031e0 msgr2=0x7f1bb010b6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:32:04.160 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:04.158+0000 7f1baf200640 1 --2- 172.21.15.83:0/2640345393 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1bb01031e0 0x7f1bb010b6d0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f1b9c009900 tx=0x7f1b9c02a480 comp rx=0 tx=0).stop 2024-08-19T01:32:04.160 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:04.158+0000 7f1baf200640 1 -- 172.21.15.83:0/2640345393 shutdown_connections 2024-08-19T01:32:04.161 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:04.158+0000 7f1baf200640 1 --2- 172.21.15.83:0/2640345393 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1bb010bcf0 0x7f1bb010e190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:32:04.161 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:04.158+0000 7f1baf200640 1 --2- 172.21.15.83:0/2640345393 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1bb01031e0 0x7f1bb010b6d0 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:32:04.161 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:04.158+0000 7f1baf200640 1 --2- 172.21.15.83:0/2640345393 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1bb01028c0 0x7f1bb0102ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:32:04.161 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:04.158+0000 7f1baf200640 1 -- 172.21.15.83:0/2640345393 >> 172.21.15.83:0/2640345393 conn(0x7f1bb00fc340 msgr2=0x7f1bb00fe760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:32:04.161 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:04.158+0000 7f1baf200640 1 -- 172.21.15.83:0/2640345393 shutdown_connections 2024-08-19T01:32:04.161 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:04.158+0000 7f1baf200640 1 -- 172.21.15.83:0/2640345393 wait complete. 2024-08-19T01:32:04.163 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:04.159+0000 7f1baf200640 1 Processor -- start 2024-08-19T01:32:04.164 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:04.159+0000 7f1baf200640 1 -- start start 2024-08-19T01:32:04.164 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:04.160+0000 7f1baf200640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1bb01028c0 0x7f1bb019a200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:32:04.164 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:04.160+0000 7f1baf200640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1bb01031e0 0x7f1bb019a740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:32:04.164 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:04.160+0000 7f1baf200640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1bb010bcf0 0x7f1bb019ead0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:32:04.164 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:04.160+0000 7f1baf200640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f1bb0111b40 con 0x7f1bb01028c0 2024-08-19T01:32:04.164 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:04.160+0000 7f1baf200640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f1bb01119c0 con 0x7f1bb01031e0 2024-08-19T01:32:04.164 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:04.160+0000 7f1baf200640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f1bb0111cc0 con 0x7f1bb010bcf0 2024-08-19T01:32:04.164 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:04.160+0000 7f1bade00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1bb01028c0 0x7f1bb019a200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:32:04.164 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:04.160+0000 7f1bade00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1bb01028c0 0x7f1bb019a200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:45972/0 (socket says 172.21.15.83:45972) 2024-08-19T01:32:04.164 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:04.160+0000 7f1bade00640 1 -- 172.21.15.83:0/155216019 learned_addr learned my addr 172.21.15.83:0/155216019 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-08-19T01:32:04.164 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:04.160+0000 7f1bad400640 1 --2- 172.21.15.83:0/155216019 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1bb01031e0 0x7f1bb019a740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:32:04.165 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:04.160+0000 7f1bae800640 1 --2- 172.21.15.83:0/155216019 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1bb010bcf0 0x7f1bb019ead0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:32:04.165 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:04.160+0000 7f1bade00640 1 -- 172.21.15.83:0/155216019 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1bb010bcf0 msgr2=0x7f1bb019ead0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:32:04.165 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:04.160+0000 7f1bade00640 1 --2- 172.21.15.83:0/155216019 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1bb010bcf0 0x7f1bb019ead0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:32:04.165 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:04.160+0000 7f1bade00640 1 -- 172.21.15.83:0/155216019 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1bb01031e0 msgr2=0x7f1bb019a740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:32:04.165 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:04.160+0000 7f1bade00640 1 --2- 172.21.15.83:0/155216019 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1bb01031e0 0x7f1bb019a740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:32:04.165 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:04.160+0000 7f1bade00640 1 -- 172.21.15.83:0/155216019 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1bb019f1b0 con 0x7f1bb01028c0 2024-08-19T01:32:04.165 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:04.160+0000 7f1bae800640 1 --2- 172.21.15.83:0/155216019 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1bb010bcf0 0x7f1bb019ead0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-19T01:32:04.165 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:04.161+0000 7f1bade00640 1 --2- 172.21.15.83:0/155216019 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1bb01028c0 0x7f1bb019a200 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f1ba000ea60 tx=0x7f1ba000ef20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:32:04.165 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:04.161+0000 7f1ba6a00640 1 -- 172.21.15.83:0/155216019 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ba000cde0 con 0x7f1bb01028c0 2024-08-19T01:32:04.165 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:04.161+0000 7f1ba6a00640 1 -- 172.21.15.83:0/155216019 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1ba00035f0 con 0x7f1bb01028c0 2024-08-19T01:32:04.165 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:04.161+0000 7f1baf200640 1 -- 172.21.15.83:0/155216019 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1bb019f4a0 con 0x7f1bb01028c0 2024-08-19T01:32:04.165 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:04.161+0000 7f1ba6a00640 1 -- 172.21.15.83:0/155216019 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ba0010670 con 0x7f1bb01028c0 2024-08-19T01:32:04.165 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:04.161+0000 7f1baf200640 1 -- 172.21.15.83:0/155216019 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1bb01a6d80 con 0x7f1bb01028c0 2024-08-19T01:32:04.166 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:04.162+0000 7f1baf200640 1 -- 172.21.15.83:0/155216019 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1bb0102ca0 con 0x7f1bb01028c0 2024-08-19T01:32:04.166 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:04.164+0000 7f1ba6a00640 1 -- 172.21.15.83:0/155216019 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f1ba00037a0 con 0x7f1bb01028c0 2024-08-19T01:32:04.167 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:04.165+0000 7f1ba6a00640 1 --2- 172.21.15.83:0/155216019 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f1b940a2c00 0x7f1b940a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:32:04.167 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:04.166+0000 7f1ba6a00640 1 -- 172.21.15.83:0/155216019 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 7127+0+0 (secure 0 0 0) 0x7f1ba00cef30 con 0x7f1bb01028c0 2024-08-19T01:32:04.167 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:04.166+0000 7f1bad400640 1 --2- 172.21.15.83:0/155216019 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f1b940a2c00 0x7f1b940a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:32:04.168 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:04.166+0000 7f1bad400640 1 --2- 172.21.15.83:0/155216019 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f1b940a2c00 0x7f1b940a50c0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f1b9c000c00 tx=0x7f1b9c005b50 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:32:04.171 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:04.170+0000 7f1ba6a00640 1 -- 172.21.15.83:0/155216019 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7f1ba009c290 con 0x7f1bb01028c0 2024-08-19T01:32:04.390 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:04.388+0000 7f1baf200640 1 -- 172.21.15.83:0/155216019 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f1bb01a7010 con 0x7f1bb01028c0 2024-08-19T01:32:04.390 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:04.388+0000 7f1ba6a00640 1 -- 172.21.15.83:0/155216019 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 4 v4) ==== 93+0+3088 (secure 0 0 0) 0x7f1ba009bb90 con 0x7f1bb01028c0 2024-08-19T01:32:04.391 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-08-19T01:32:04.391 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":4,"btime":"2024-08-19T01:32:03:421307+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14421,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6835/988556067","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":988556067},{"type":"v1","addr":"172.21.15.83:6835","nonce":988556067}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24430,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6833/2505076046","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6832","nonce":2505076046},{"type":"v1","addr":"172.21.15.167:6833","nonce":2505076046}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24449,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6833/3557520478","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":3557520478},{"type":"v1","addr":"172.21.15.117:6833","nonce":3557520478}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[]} 2024-08-19T01:32:04.391 INFO:teuthology.orchestra.run.smithi083.stderr:dumped fsmap epoch 4 2024-08-19T01:32:04.393 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:04.391+0000 7f1baf200640 1 -- 172.21.15.83:0/155216019 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f1b940a2c00 msgr2=0x7f1b940a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:32:04.393 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:04.391+0000 7f1baf200640 1 --2- 172.21.15.83:0/155216019 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f1b940a2c00 0x7f1b940a50c0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f1b9c000c00 tx=0x7f1b9c005b50 comp rx=0 tx=0).stop 2024-08-19T01:32:04.393 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:04.391+0000 7f1baf200640 1 -- 172.21.15.83:0/155216019 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1bb01028c0 msgr2=0x7f1bb019a200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:32:04.393 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:04.391+0000 7f1baf200640 1 --2- 172.21.15.83:0/155216019 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1bb01028c0 0x7f1bb019a200 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f1ba000ea60 tx=0x7f1ba000ef20 comp rx=0 tx=0).stop 2024-08-19T01:32:04.393 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:04.391+0000 7f1baf200640 1 -- 172.21.15.83:0/155216019 shutdown_connections 2024-08-19T01:32:04.393 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:04.391+0000 7f1baf200640 1 --2- 172.21.15.83:0/155216019 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f1b940a2c00 0x7f1b940a50c0 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:32:04.393 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:04.391+0000 7f1baf200640 1 --2- 172.21.15.83:0/155216019 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1bb010bcf0 0x7f1bb019ead0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:32:04.393 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:04.391+0000 7f1baf200640 1 --2- 172.21.15.83:0/155216019 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1bb01031e0 0x7f1bb019a740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:32:04.393 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:04.391+0000 7f1baf200640 1 --2- 172.21.15.83:0/155216019 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1bb01028c0 0x7f1bb019a200 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:32:04.393 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:04.391+0000 7f1baf200640 1 -- 172.21.15.83:0/155216019 >> 172.21.15.83:0/155216019 conn(0x7f1bb00fc340 msgr2=0x7f1bb00fe730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:32:04.393 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:04.391+0000 7f1baf200640 1 -- 172.21.15.83:0/155216019 shutdown_connections 2024-08-19T01:32:04.393 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:04.391+0000 7f1baf200640 1 -- 172.21.15.83:0/155216019 wait complete. 2024-08-19T01:32:04.632 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:32:04 smithi083 ceph-mon[17120]: Deploying daemon mds.d on smithi083 2024-08-19T01:32:04.632 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:32:04 smithi083 ceph-mon[17120]: mds.? [v2:172.21.15.83:6834/988556067,v1:172.21.15.83:6835/988556067] up:boot 2024-08-19T01:32:04.632 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:32:04 smithi083 ceph-mon[17120]: fsmap 3 up:standby 2024-08-19T01:32:04.632 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:32:04 smithi083 ceph-mon[17120]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-08-19T01:32:04.632 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:32:04 smithi083 ceph-mon[17120]: from='client.? 172.21.15.83:0/155216019' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-19T01:32:04.702 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:32:04 smithi117 ceph-mon[22016]: Deploying daemon mds.d on smithi083 2024-08-19T01:32:04.703 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:32:04 smithi117 ceph-mon[22016]: mds.? [v2:172.21.15.83:6834/988556067,v1:172.21.15.83:6835/988556067] up:boot 2024-08-19T01:32:04.703 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:32:04 smithi117 ceph-mon[22016]: fsmap 3 up:standby 2024-08-19T01:32:04.703 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:32:04 smithi117 ceph-mon[22016]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-08-19T01:32:04.703 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:32:04 smithi117 ceph-mon[22016]: from='client.? 172.21.15.83:0/155216019' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-19T01:32:04.716 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:32:04 smithi167 ceph-mon[21875]: Deploying daemon mds.d on smithi083 2024-08-19T01:32:04.716 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:32:04 smithi167 ceph-mon[21875]: mds.? [v2:172.21.15.83:6834/988556067,v1:172.21.15.83:6835/988556067] up:boot 2024-08-19T01:32:04.716 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:32:04 smithi167 ceph-mon[21875]: fsmap 3 up:standby 2024-08-19T01:32:04.716 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:32:04 smithi167 ceph-mon[21875]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-08-19T01:32:04.716 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:32:04 smithi167 ceph-mon[21875]: from='client.? 172.21.15.83:0/155216019' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-19T01:32:05.655 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:32:05 smithi083 ceph-mon[17120]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-19T01:32:05.702 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:32:05 smithi117 ceph-mon[22016]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-19T01:32:05.716 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:32:05 smithi167 ceph-mon[21875]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-19T01:32:06.350 INFO:journalctl@ceph.mds.d.smithi083.stdout:Aug 19 01:32:06 smithi083 systemd[1]: Starting Ceph mds.d for bad8780e-5dc9-11ef-bccf-c7b262605968... 2024-08-19T01:32:06.616 INFO:journalctl@ceph.mds.d.smithi083.stdout:Aug 19 01:32:06 smithi083 podman[56868]: 2024-08-19 01:32:06.252826006 +0000 UTC m=+0.029525527 image pull ea7a5189f30bad3aca9524788e2965d2368d7ce786df09d4edcd8c3edf9cefba quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c9dd28080a631be25c77c36c63b17a3ef340ffb2e21c87d4d7b56b6398436d9 2024-08-19T01:32:06.616 INFO:journalctl@ceph.mds.d.smithi083.stdout:Aug 19 01:32:06 smithi083 podman[56868]: 2024-08-19 01:32:06.362491404 +0000 UTC m=+0.139190918 container create c3e18ff80221dee8b0363077499a81eda105420ec5817270279d2f57e89f0f5c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c9dd28080a631be25c77c36c63b17a3ef340ffb2e21c87d4d7b56b6398436d9, name=ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-mds-d, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, RELEASE=squid-de09dd0, GIT_CLEAN=True, io.buildah.version=1.36.0, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=) 2024-08-19T01:32:06.616 INFO:journalctl@ceph.mds.d.smithi083.stdout:Aug 19 01:32:06 smithi083 podman[56868]: 2024-08-19 01:32:06.615371701 +0000 UTC m=+0.392071216 container init c3e18ff80221dee8b0363077499a81eda105420ec5817270279d2f57e89f0f5c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c9dd28080a631be25c77c36c63b17a3ef340ffb2e21c87d4d7b56b6398436d9, name=ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-mds-d, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, io.buildah.version=1.36.0, RELEASE=squid-de09dd0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, maintainer=Guillaume Abrioux ) 2024-08-19T01:32:06.905 INFO:journalctl@ceph.mds.d.smithi083.stdout:Aug 19 01:32:06 smithi083 podman[56868]: 2024-08-19 01:32:06.621961732 +0000 UTC m=+0.398661246 container start c3e18ff80221dee8b0363077499a81eda105420ec5817270279d2f57e89f0f5c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c9dd28080a631be25c77c36c63b17a3ef340ffb2e21c87d4d7b56b6398436d9, name=ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-mds-d, org.label-schema.vendor=CentOS, RELEASE=squid-de09dd0, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, io.buildah.version=1.36.0, ceph=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-08-19T01:32:06.905 INFO:journalctl@ceph.mds.d.smithi083.stdout:Aug 19 01:32:06 smithi083 ceph-mds[56887]: -- 172.21.15.83:0/3008643416 <== mon.0 v2:172.21.15.83:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5563255ed860 con 0x5563255cb400 2024-08-19T01:32:06.905 INFO:journalctl@ceph.mds.d.smithi083.stdout:Aug 19 01:32:06 smithi083 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-mds-d[56883]: starting mds.d at 2024-08-19T01:32:06.905 INFO:journalctl@ceph.mds.d.smithi083.stdout:Aug 19 01:32:06 smithi083 bash[56868]: c3e18ff80221dee8b0363077499a81eda105420ec5817270279d2f57e89f0f5c 2024-08-19T01:32:06.905 INFO:journalctl@ceph.mds.d.smithi083.stdout:Aug 19 01:32:06 smithi083 systemd[1]: Started Ceph mds.d for bad8780e-5dc9-11ef-bccf-c7b262605968. 2024-08-19T01:32:07.466 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:32:07 smithi167 ceph-mon[21875]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-19T01:32:07.466 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:32:07 smithi167 ceph-mon[21875]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' 2024-08-19T01:32:07.466 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:32:07 smithi167 ceph-mon[21875]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' 2024-08-19T01:32:07.466 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:32:07 smithi167 ceph-mon[21875]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' 2024-08-19T01:32:07.466 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:32:07 smithi167 ceph-mon[21875]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-08-19T01:32:07.466 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:32:07 smithi167 ceph-mon[21875]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-08-19T01:32:07.466 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:32:07 smithi167 ceph-mon[21875]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-19T01:32:07.494 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:32:07 smithi117 ceph-mon[22016]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-19T01:32:07.494 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:32:07 smithi117 ceph-mon[22016]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' 2024-08-19T01:32:07.494 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:32:07 smithi117 ceph-mon[22016]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' 2024-08-19T01:32:07.495 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:32:07 smithi117 ceph-mon[22016]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' 2024-08-19T01:32:07.495 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:32:07 smithi117 ceph-mon[22016]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-08-19T01:32:07.495 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:32:07 smithi117 ceph-mon[22016]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-08-19T01:32:07.495 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:32:07 smithi117 ceph-mon[22016]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-19T01:32:07.827 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833 shell --fsid bad8780e-5dc9-11ef-bccf-c7b262605968 -- ceph fs dump --format=json 2024-08-19T01:32:07.883 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:32:07 smithi083 ceph-mon[17120]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-19T01:32:07.883 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:32:07 smithi083 ceph-mon[17120]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' 2024-08-19T01:32:07.883 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:32:07 smithi083 ceph-mon[17120]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' 2024-08-19T01:32:07.883 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:32:07 smithi083 ceph-mon[17120]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' 2024-08-19T01:32:07.883 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:32:07 smithi083 ceph-mon[17120]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-08-19T01:32:07.883 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:32:07 smithi083 ceph-mon[17120]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-08-19T01:32:07.883 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:32:07 smithi083 ceph-mon[17120]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-19T01:32:08.212 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/bad8780e-5dc9-11ef-bccf-c7b262605968/mon.a/config 2024-08-19T01:32:08.560 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:32:08 smithi117 ceph-mon[22016]: Deploying daemon mds.e on smithi117 2024-08-19T01:32:08.560 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:32:08 smithi117 ceph-mon[22016]: mds.? [v2:172.21.15.83:6836/869702632,v1:172.21.15.83:6837/869702632] up:boot 2024-08-19T01:32:08.560 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:32:08 smithi117 ceph-mon[22016]: fsmap 4 up:standby 2024-08-19T01:32:08.560 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:32:08 smithi117 ceph-mon[22016]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-08-19T01:32:08.636 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:32:08 smithi083 ceph-mon[17120]: Deploying daemon mds.e on smithi117 2024-08-19T01:32:08.637 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:32:08 smithi083 ceph-mon[17120]: mds.? [v2:172.21.15.83:6836/869702632,v1:172.21.15.83:6837/869702632] up:boot 2024-08-19T01:32:08.637 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:32:08 smithi083 ceph-mon[17120]: fsmap 4 up:standby 2024-08-19T01:32:08.637 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:32:08 smithi083 ceph-mon[17120]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-08-19T01:32:08.716 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:32:08 smithi167 ceph-mon[21875]: Deploying daemon mds.e on smithi117 2024-08-19T01:32:08.716 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:32:08 smithi167 ceph-mon[21875]: mds.? [v2:172.21.15.83:6836/869702632,v1:172.21.15.83:6837/869702632] up:boot 2024-08-19T01:32:08.716 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:32:08 smithi167 ceph-mon[21875]: fsmap 4 up:standby 2024-08-19T01:32:08.716 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:32:08 smithi167 ceph-mon[21875]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-08-19T01:32:08.746 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:08.744+0000 7f3942e00640 1 -- 172.21.15.83:0/768089081 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f393c1128c0 msgr2=0x7f393c114cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:32:08.747 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:08.744+0000 7f3942e00640 1 --2- 172.21.15.83:0/768089081 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f393c1128c0 0x7f393c114cb0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f3934009ca0 tx=0x7f393402a950 comp rx=0 tx=0).stop 2024-08-19T01:32:08.747 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:08.744+0000 7f3942e00640 1 -- 172.21.15.83:0/768089081 shutdown_connections 2024-08-19T01:32:08.747 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:08.744+0000 7f3942e00640 1 --2- 172.21.15.83:0/768089081 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f393c1128c0 0x7f393c114cb0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:32:08.747 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:08.744+0000 7f3942e00640 1 --2- 172.21.15.83:0/768089081 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f393c074be0 0x7f393c112380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:32:08.747 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:08.744+0000 7f3942e00640 1 --2- 172.21.15.83:0/768089081 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f393c0742c0 0x7f393c0746a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:32:08.747 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:08.744+0000 7f3942e00640 1 -- 172.21.15.83:0/768089081 >> 172.21.15.83:0/768089081 conn(0x7f393c0fd5f0 msgr2=0x7f393c0ffa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:32:08.747 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:08.744+0000 7f3942e00640 1 -- 172.21.15.83:0/768089081 shutdown_connections 2024-08-19T01:32:08.747 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:08.745+0000 7f3942e00640 1 -- 172.21.15.83:0/768089081 wait complete. 2024-08-19T01:32:08.748 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:08.745+0000 7f3942e00640 1 Processor -- start 2024-08-19T01:32:08.748 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:08.746+0000 7f3942e00640 1 -- start start 2024-08-19T01:32:08.748 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:08.746+0000 7f3942e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f393c0742c0 0x7f393c1a43e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:32:08.748 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:08.746+0000 7f3942e00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f393c074be0 0x7f393c1a4920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:32:08.748 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:08.746+0000 7f3942e00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f393c1128c0 0x7f393c19e5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:32:08.748 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:08.746+0000 7f3942e00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f393c1182f0 con 0x7f393c0742c0 2024-08-19T01:32:08.748 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:08.746+0000 7f3942e00640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f393c118170 con 0x7f393c074be0 2024-08-19T01:32:08.748 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:08.746+0000 7f3942e00640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f393c118470 con 0x7f393c1128c0 2024-08-19T01:32:08.749 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:08.746+0000 7f393b400640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f393c074be0 0x7f393c1a4920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:32:08.749 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:08.746+0000 7f393be00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f393c0742c0 0x7f393c1a43e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:32:08.749 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:08.746+0000 7f393b400640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f393c074be0 0x7f393c1a4920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.83:51548/0 (socket says 172.21.15.83:51548) 2024-08-19T01:32:08.749 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:08.746+0000 7f393b400640 1 -- 172.21.15.83:0/1552725329 learned_addr learned my addr 172.21.15.83:0/1552725329 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-08-19T01:32:08.749 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:08.747+0000 7f3940e00640 1 --2- 172.21.15.83:0/1552725329 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f393c1128c0 0x7f393c19e5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:32:08.750 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:08.747+0000 7f393be00640 1 -- 172.21.15.83:0/1552725329 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f393c1128c0 msgr2=0x7f393c19e5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:32:08.750 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:08.747+0000 7f393be00640 1 --2- 172.21.15.83:0/1552725329 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f393c1128c0 0x7f393c19e5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:32:08.750 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:08.747+0000 7f393be00640 1 -- 172.21.15.83:0/1552725329 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f393c074be0 msgr2=0x7f393c1a4920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:32:08.750 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:08.747+0000 7f393be00640 1 --2- 172.21.15.83:0/1552725329 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f393c074be0 0x7f393c1a4920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:32:08.750 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:08.747+0000 7f393be00640 1 -- 172.21.15.83:0/1552725329 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f393c19ecd0 con 0x7f393c0742c0 2024-08-19T01:32:08.750 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:08.747+0000 7f393b400640 1 --2- 172.21.15.83:0/1552725329 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f393c074be0 0x7f393c1a4920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-19T01:32:08.750 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:08.747+0000 7f393be00640 1 --2- 172.21.15.83:0/1552725329 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f393c0742c0 0x7f393c1a43e0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f392c00e9f0 tx=0x7f392c00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:32:08.750 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:08.747+0000 7f3938c00640 1 -- 172.21.15.83:0/1552725329 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f392c00cd80 con 0x7f393c0742c0 2024-08-19T01:32:08.750 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:08.747+0000 7f3938c00640 1 -- 172.21.15.83:0/1552725329 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f392c0035f0 con 0x7f393c0742c0 2024-08-19T01:32:08.750 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:08.747+0000 7f3938c00640 1 -- 172.21.15.83:0/1552725329 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f392c010690 con 0x7f393c0742c0 2024-08-19T01:32:08.750 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:08.747+0000 7f3942e00640 1 -- 172.21.15.83:0/1552725329 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f393c19efc0 con 0x7f393c0742c0 2024-08-19T01:32:08.750 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:08.748+0000 7f3942e00640 1 -- 172.21.15.83:0/1552725329 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f393c1ab2b0 con 0x7f393c0742c0 2024-08-19T01:32:08.751 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:08.749+0000 7f3942e00640 1 -- 172.21.15.83:0/1552725329 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f393c0746a0 con 0x7f393c0742c0 2024-08-19T01:32:08.752 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:08.750+0000 7f3938c00640 1 -- 172.21.15.83:0/1552725329 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f392c003b30 con 0x7f393c0742c0 2024-08-19T01:32:08.753 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:08.752+0000 7f3938c00640 1 --2- 172.21.15.83:0/1552725329 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f39200a2c00 0x7f39200a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:32:08.753 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:08.752+0000 7f3938c00640 1 -- 172.21.15.83:0/1552725329 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 7127+0+0 (secure 0 0 0) 0x7f392c0ceed0 con 0x7f393c0742c0 2024-08-19T01:32:08.754 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:08.752+0000 7f393b400640 1 --2- 172.21.15.83:0/1552725329 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f39200a2c00 0x7f39200a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:32:08.754 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:08.753+0000 7f393b400640 1 --2- 172.21.15.83:0/1552725329 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f39200a2c00 0x7f39200a50c0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f393c0ffd30 tx=0x7f392800b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:32:08.757 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:08.756+0000 7f3938c00640 1 -- 172.21.15.83:0/1552725329 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7f392c09c230 con 0x7f393c0742c0 2024-08-19T01:32:08.953 INFO:journalctl@ceph.mds.e.smithi117.stdout:Aug 19 01:32:08 smithi117 systemd[1]: Starting Ceph mds.e for bad8780e-5dc9-11ef-bccf-c7b262605968... 2024-08-19T01:32:08.968 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:08.966+0000 7f3942e00640 1 -- 172.21.15.83:0/1552725329 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f393c19fad0 con 0x7f393c0742c0 2024-08-19T01:32:08.968 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:08.967+0000 7f3938c00640 1 -- 172.21.15.83:0/1552725329 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 5 v5) ==== 93+0+3910 (secure 0 0 0) 0x7f392c09bb30 con 0x7f393c0742c0 2024-08-19T01:32:08.969 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-08-19T01:32:08.969 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":5,"btime":"2024-08-19T01:32:07:435332+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14421,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6835/988556067","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":988556067},{"type":"v1","addr":"172.21.15.83:6835","nonce":988556067}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14433,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6837/869702632","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6836","nonce":869702632},{"type":"v1","addr":"172.21.15.83:6837","nonce":869702632}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24430,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6833/2505076046","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6832","nonce":2505076046},{"type":"v1","addr":"172.21.15.167:6833","nonce":2505076046}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24449,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6833/3557520478","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":3557520478},{"type":"v1","addr":"172.21.15.117:6833","nonce":3557520478}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[]} 2024-08-19T01:32:08.969 INFO:teuthology.orchestra.run.smithi083.stderr:dumped fsmap epoch 5 2024-08-19T01:32:08.971 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:08.969+0000 7f3942e00640 1 -- 172.21.15.83:0/1552725329 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f39200a2c00 msgr2=0x7f39200a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:32:08.971 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:08.969+0000 7f3942e00640 1 --2- 172.21.15.83:0/1552725329 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f39200a2c00 0x7f39200a50c0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f393c0ffd30 tx=0x7f392800b410 comp rx=0 tx=0).stop 2024-08-19T01:32:08.971 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:08.969+0000 7f3942e00640 1 -- 172.21.15.83:0/1552725329 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f393c0742c0 msgr2=0x7f393c1a43e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:32:08.971 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:08.969+0000 7f3942e00640 1 --2- 172.21.15.83:0/1552725329 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f393c0742c0 0x7f393c1a43e0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f392c00e9f0 tx=0x7f392c00eeb0 comp rx=0 tx=0).stop 2024-08-19T01:32:08.972 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:08.969+0000 7f3942e00640 1 -- 172.21.15.83:0/1552725329 shutdown_connections 2024-08-19T01:32:08.972 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:08.969+0000 7f3942e00640 1 --2- 172.21.15.83:0/1552725329 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f39200a2c00 0x7f39200a50c0 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:32:08.972 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:08.969+0000 7f3942e00640 1 --2- 172.21.15.83:0/1552725329 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f393c1128c0 0x7f393c19e5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:32:08.972 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:08.969+0000 7f3942e00640 1 --2- 172.21.15.83:0/1552725329 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f393c074be0 0x7f393c1a4920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:32:08.972 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:08.969+0000 7f3942e00640 1 --2- 172.21.15.83:0/1552725329 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f393c0742c0 0x7f393c1a43e0 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:32:08.972 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:08.969+0000 7f3942e00640 1 -- 172.21.15.83:0/1552725329 >> 172.21.15.83:0/1552725329 conn(0x7f393c0fd5f0 msgr2=0x7f393c0fee20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:32:08.972 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:08.970+0000 7f3942e00640 1 -- 172.21.15.83:0/1552725329 shutdown_connections 2024-08-19T01:32:08.972 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:08.970+0000 7f3942e00640 1 -- 172.21.15.83:0/1552725329 wait complete. 2024-08-19T01:32:09.224 INFO:journalctl@ceph.mds.e.smithi117.stdout:Aug 19 01:32:08 smithi117 podman[51275]: 2024-08-19 01:32:08.863614367 +0000 UTC m=+0.017641690 image pull ea7a5189f30bad3aca9524788e2965d2368d7ce786df09d4edcd8c3edf9cefba quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c9dd28080a631be25c77c36c63b17a3ef340ffb2e21c87d4d7b56b6398436d9 2024-08-19T01:32:09.224 INFO:journalctl@ceph.mds.e.smithi117.stdout:Aug 19 01:32:08 smithi117 podman[51275]: 2024-08-19 01:32:08.97060396 +0000 UTC m=+0.124631306 container create 8f8b5c7000c43c452fb20951b381f665f4ee4c9c6eead8dd38648981cbc1bac0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c9dd28080a631be25c77c36c63b17a3ef340ffb2e21c87d4d7b56b6398436d9, name=ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-mds-e, io.buildah.version=1.36.0, GIT_CLEAN=True, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, RELEASE=squid-de09dd0) 2024-08-19T01:32:09.224 INFO:journalctl@ceph.mds.e.smithi117.stdout:Aug 19 01:32:09 smithi117 podman[51275]: 2024-08-19 01:32:09.22318142 +0000 UTC m=+0.377208740 container init 8f8b5c7000c43c452fb20951b381f665f4ee4c9c6eead8dd38648981cbc1bac0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c9dd28080a631be25c77c36c63b17a3ef340ffb2e21c87d4d7b56b6398436d9, name=ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-mds-e, io.buildah.version=1.36.0, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, GIT_CLEAN=True, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, RELEASE=squid-de09dd0, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731) 2024-08-19T01:32:09.655 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:32:09 smithi083 ceph-mon[17120]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-19T01:32:09.655 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:32:09 smithi083 ceph-mon[17120]: from='client.? 172.21.15.83:0/1552725329' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-19T01:32:09.655 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:32:09 smithi083 ceph-mon[17120]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' 2024-08-19T01:32:09.655 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:32:09 smithi083 ceph-mon[17120]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' 2024-08-19T01:32:09.655 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:32:09 smithi083 ceph-mon[17120]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' 2024-08-19T01:32:09.655 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:32:09 smithi083 ceph-mon[17120]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-08-19T01:32:09.655 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:32:09 smithi083 ceph-mon[17120]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-08-19T01:32:09.655 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:32:09 smithi083 ceph-mon[17120]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-19T01:32:09.702 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:32:09 smithi117 ceph-mon[22016]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-19T01:32:09.703 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:32:09 smithi117 ceph-mon[22016]: from='client.? 172.21.15.83:0/1552725329' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-19T01:32:09.703 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:32:09 smithi117 ceph-mon[22016]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' 2024-08-19T01:32:09.703 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:32:09 smithi117 ceph-mon[22016]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' 2024-08-19T01:32:09.703 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:32:09 smithi117 ceph-mon[22016]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' 2024-08-19T01:32:09.703 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:32:09 smithi117 ceph-mon[22016]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-08-19T01:32:09.703 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:32:09 smithi117 ceph-mon[22016]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-08-19T01:32:09.703 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:32:09 smithi117 ceph-mon[22016]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-19T01:32:09.703 INFO:journalctl@ceph.mds.e.smithi117.stdout:Aug 19 01:32:09 smithi117 podman[51275]: 2024-08-19 01:32:09.22606935 +0000 UTC m=+0.380096663 container start 8f8b5c7000c43c452fb20951b381f665f4ee4c9c6eead8dd38648981cbc1bac0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c9dd28080a631be25c77c36c63b17a3ef340ffb2e21c87d4d7b56b6398436d9, name=ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-mds-e, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-de09dd0, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , io.buildah.version=1.36.0, ceph=True, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_CLEAN=True) 2024-08-19T01:32:09.703 INFO:journalctl@ceph.mds.e.smithi117.stdout:Aug 19 01:32:09 smithi117 ceph-mds[51294]: -- 172.21.15.117:0/1696143135 <== mon.2 v2:172.21.15.117:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x562f060ab860 con 0x562f06089000 2024-08-19T01:32:09.703 INFO:journalctl@ceph.mds.e.smithi117.stdout:Aug 19 01:32:09 smithi117 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-mds-e[51290]: starting mds.e at 2024-08-19T01:32:09.703 INFO:journalctl@ceph.mds.e.smithi117.stdout:Aug 19 01:32:09 smithi117 bash[51275]: 8f8b5c7000c43c452fb20951b381f665f4ee4c9c6eead8dd38648981cbc1bac0 2024-08-19T01:32:09.703 INFO:journalctl@ceph.mds.e.smithi117.stdout:Aug 19 01:32:09 smithi117 systemd[1]: Started Ceph mds.e for bad8780e-5dc9-11ef-bccf-c7b262605968. 2024-08-19T01:32:09.716 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:32:09 smithi167 ceph-mon[21875]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-19T01:32:09.716 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:32:09 smithi167 ceph-mon[21875]: from='client.? 172.21.15.83:0/1552725329' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-19T01:32:09.716 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:32:09 smithi167 ceph-mon[21875]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' 2024-08-19T01:32:09.716 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:32:09 smithi167 ceph-mon[21875]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' 2024-08-19T01:32:09.716 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:32:09 smithi167 ceph-mon[21875]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' 2024-08-19T01:32:09.716 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:32:09 smithi167 ceph-mon[21875]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-08-19T01:32:09.716 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:32:09 smithi167 ceph-mon[21875]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-08-19T01:32:09.716 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:32:09 smithi167 ceph-mon[21875]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-19T01:32:10.716 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:32:10 smithi167 ceph-mon[21875]: Deploying daemon mds.f on smithi167 2024-08-19T01:32:10.716 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:32:10 smithi167 ceph-mon[21875]: mds.? [v2:172.21.15.117:6834/1024213994,v1:172.21.15.117:6835/1024213994] up:boot 2024-08-19T01:32:10.716 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:32:10 smithi167 ceph-mon[21875]: fsmap 5 up:standby 2024-08-19T01:32:10.716 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:32:10 smithi167 ceph-mon[21875]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-08-19T01:32:10.905 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:32:10 smithi083 ceph-mon[17120]: Deploying daemon mds.f on smithi167 2024-08-19T01:32:10.905 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:32:10 smithi083 ceph-mon[17120]: mds.? [v2:172.21.15.117:6834/1024213994,v1:172.21.15.117:6835/1024213994] up:boot 2024-08-19T01:32:10.905 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:32:10 smithi083 ceph-mon[17120]: fsmap 5 up:standby 2024-08-19T01:32:10.905 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:32:10 smithi083 ceph-mon[17120]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-08-19T01:32:10.952 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:32:10 smithi117 ceph-mon[22016]: Deploying daemon mds.f on smithi167 2024-08-19T01:32:10.952 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:32:10 smithi117 ceph-mon[22016]: mds.? [v2:172.21.15.117:6834/1024213994,v1:172.21.15.117:6835/1024213994] up:boot 2024-08-19T01:32:10.952 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:32:10 smithi117 ceph-mon[22016]: fsmap 5 up:standby 2024-08-19T01:32:10.953 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:32:10 smithi117 ceph-mon[22016]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-08-19T01:32:11.464 INFO:journalctl@ceph.mds.f.smithi167.stdout:Aug 19 01:32:11 smithi167 systemd[1]: Starting Ceph mds.f for bad8780e-5dc9-11ef-bccf-c7b262605968... 2024-08-19T01:32:11.717 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:32:11 smithi167 ceph-mon[21875]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-19T01:32:11.717 INFO:journalctl@ceph.mds.f.smithi167.stdout:Aug 19 01:32:11 smithi167 podman[51590]: 2024-08-19 01:32:11.440708009 +0000 UTC m=+0.017087724 image pull ea7a5189f30bad3aca9524788e2965d2368d7ce786df09d4edcd8c3edf9cefba quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c9dd28080a631be25c77c36c63b17a3ef340ffb2e21c87d4d7b56b6398436d9 2024-08-19T01:32:11.717 INFO:journalctl@ceph.mds.f.smithi167.stdout:Aug 19 01:32:11 smithi167 podman[51590]: 2024-08-19 01:32:11.56395344 +0000 UTC m=+0.140333154 container create 15e840bcee59960895a4c4909923b8621aac75d8677afa2a96ef1ce9e2da0653 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c9dd28080a631be25c77c36c63b17a3ef340ffb2e21c87d4d7b56b6398436d9, name=ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-mds-f, io.buildah.version=1.36.0, RELEASE=squid-de09dd0, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, ceph=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0) 2024-08-19T01:32:11.822 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833 shell --fsid bad8780e-5dc9-11ef-bccf-c7b262605968 -- ceph fs dump --format=json 2024-08-19T01:32:11.877 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:32:11 smithi083 ceph-mon[17120]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-19T01:32:11.952 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:32:11 smithi117 ceph-mon[22016]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-19T01:32:12.216 INFO:journalctl@ceph.mds.f.smithi167.stdout:Aug 19 01:32:11 smithi167 podman[51590]: 2024-08-19 01:32:11.799728529 +0000 UTC m=+0.376108256 container init 15e840bcee59960895a4c4909923b8621aac75d8677afa2a96ef1ce9e2da0653 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c9dd28080a631be25c77c36c63b17a3ef340ffb2e21c87d4d7b56b6398436d9, name=ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-mds-f, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.36.0, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_CLEAN=True, RELEASE=squid-de09dd0, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , ceph=True, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD) 2024-08-19T01:32:12.216 INFO:journalctl@ceph.mds.f.smithi167.stdout:Aug 19 01:32:11 smithi167 podman[51590]: 2024-08-19 01:32:11.80260693 +0000 UTC m=+0.378986644 container start 15e840bcee59960895a4c4909923b8621aac75d8677afa2a96ef1ce9e2da0653 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c9dd28080a631be25c77c36c63b17a3ef340ffb2e21c87d4d7b56b6398436d9, name=ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-mds-f, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.36.0, GIT_CLEAN=True, RELEASE=squid-de09dd0, ceph=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux ) 2024-08-19T01:32:12.216 INFO:journalctl@ceph.mds.f.smithi167.stdout:Aug 19 01:32:11 smithi167 ceph-mds[51609]: -- 172.21.15.167:0/3558078482 <== mon.1 v2:172.21.15.167:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55e4fff31860 con 0x55e4fff0e800 2024-08-19T01:32:12.216 INFO:journalctl@ceph.mds.f.smithi167.stdout:Aug 19 01:32:11 smithi167 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-mds-f[51605]: starting mds.f at 2024-08-19T01:32:12.216 INFO:journalctl@ceph.mds.f.smithi167.stdout:Aug 19 01:32:11 smithi167 bash[51590]: 15e840bcee59960895a4c4909923b8621aac75d8677afa2a96ef1ce9e2da0653 2024-08-19T01:32:12.216 INFO:journalctl@ceph.mds.f.smithi167.stdout:Aug 19 01:32:11 smithi167 systemd[1]: Started Ceph mds.f for bad8780e-5dc9-11ef-bccf-c7b262605968. 2024-08-19T01:32:12.221 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/bad8780e-5dc9-11ef-bccf-c7b262605968/mon.a/config 2024-08-19T01:32:12.789 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:12.787+0000 7f8989c00640 1 -- 172.21.15.83:0/2895810456 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8984103ae0 msgr2=0x7f8984103f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:32:12.789 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:12.787+0000 7f8989c00640 1 --2- 172.21.15.83:0/2895810456 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8984103ae0 0x7f8984103f40 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f896c009930 tx=0x7f896c02a420 comp rx=0 tx=0).stop 2024-08-19T01:32:12.789 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:12.787+0000 7f8989c00640 1 -- 172.21.15.83:0/2895810456 shutdown_connections 2024-08-19T01:32:12.789 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:12.787+0000 7f8989c00640 1 --2- 172.21.15.83:0/2895810456 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8984104480 0x7f89841109b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:32:12.789 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:12.787+0000 7f8989c00640 1 --2- 172.21.15.83:0/2895810456 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8984103ae0 0x7f8984103f40 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:32:12.789 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:12.787+0000 7f8989c00640 1 --2- 172.21.15.83:0/2895810456 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8984109b70 0x7f8984109f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:32:12.790 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:12.787+0000 7f8989c00640 1 -- 172.21.15.83:0/2895810456 >> 172.21.15.83:0/2895810456 conn(0x7f89840ff780 msgr2=0x7f8984101ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:32:12.790 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:12.787+0000 7f8989c00640 1 -- 172.21.15.83:0/2895810456 shutdown_connections 2024-08-19T01:32:12.790 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:12.787+0000 7f8989c00640 1 -- 172.21.15.83:0/2895810456 wait complete. 2024-08-19T01:32:12.791 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:12.788+0000 7f8989c00640 1 Processor -- start 2024-08-19T01:32:12.792 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:12.788+0000 7f8989c00640 1 -- start start 2024-08-19T01:32:12.792 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:12.788+0000 7f8989c00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8984103ae0 0x7f898419a3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:32:12.792 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:12.788+0000 7f8989c00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8984104480 0x7f898419a920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:32:12.792 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:12.788+0000 7f8989c00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8984109b70 0x7f898419ecb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:32:12.792 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:12.788+0000 7f8989c00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f8984113ee0 con 0x7f8984103ae0 2024-08-19T01:32:12.792 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:12.788+0000 7f8989c00640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f8984113d60 con 0x7f8984109b70 2024-08-19T01:32:12.792 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:12.788+0000 7f8989c00640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f8984114060 con 0x7f8984104480 2024-08-19T01:32:12.792 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:12.789+0000 7f8982a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8984103ae0 0x7f898419a3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:32:12.792 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:12.789+0000 7f8982a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8984103ae0 0x7f898419a3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:50164/0 (socket says 172.21.15.83:50164) 2024-08-19T01:32:12.792 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:12.789+0000 7f8982a00640 1 -- 172.21.15.83:0/3434266162 learned_addr learned my addr 172.21.15.83:0/3434266162 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-08-19T01:32:12.792 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:12.789+0000 7f8983400640 1 --2- 172.21.15.83:0/3434266162 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8984109b70 0x7f898419ecb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:32:12.792 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:12.789+0000 7f897a000640 1 --2- 172.21.15.83:0/3434266162 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8984104480 0x7f898419a920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:32:12.793 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:12.789+0000 7f8982a00640 1 -- 172.21.15.83:0/3434266162 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8984104480 msgr2=0x7f898419a920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:32:12.793 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:12.789+0000 7f8982a00640 1 --2- 172.21.15.83:0/3434266162 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8984104480 0x7f898419a920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:32:12.793 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:12.789+0000 7f8982a00640 1 -- 172.21.15.83:0/3434266162 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8984109b70 msgr2=0x7f898419ecb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:32:12.793 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:12.789+0000 7f8982a00640 1 --2- 172.21.15.83:0/3434266162 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8984109b70 0x7f898419ecb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:32:12.793 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:12.789+0000 7f8982a00640 1 -- 172.21.15.83:0/3434266162 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f898419f390 con 0x7f8984103ae0 2024-08-19T01:32:12.793 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:12.789+0000 7f897a000640 1 --2- 172.21.15.83:0/3434266162 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8984104480 0x7f898419a920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-19T01:32:12.793 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:12.789+0000 7f8982a00640 1 --2- 172.21.15.83:0/3434266162 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8984103ae0 0x7f898419a3e0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f897400e9f0 tx=0x7f897400edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:32:12.793 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:12.790+0000 7f897be00640 1 -- 172.21.15.83:0/3434266162 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f897400ccc0 con 0x7f8984103ae0 2024-08-19T01:32:12.793 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:12.790+0000 7f897be00640 1 -- 172.21.15.83:0/3434266162 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8974010910 con 0x7f8984103ae0 2024-08-19T01:32:12.793 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:12.790+0000 7f8989c00640 1 -- 172.21.15.83:0/3434266162 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f898419f680 con 0x7f8984103ae0 2024-08-19T01:32:12.793 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:12.790+0000 7f897be00640 1 -- 172.21.15.83:0/3434266162 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89740188a0 con 0x7f8984103ae0 2024-08-19T01:32:12.793 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:12.790+0000 7f8989c00640 1 -- 172.21.15.83:0/3434266162 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f89841a6f60 con 0x7f8984103ae0 2024-08-19T01:32:12.794 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:12.791+0000 7f8989c00640 1 -- 172.21.15.83:0/3434266162 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8984103f40 con 0x7f8984103ae0 2024-08-19T01:32:12.794 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:12.793+0000 7f897be00640 1 -- 172.21.15.83:0/3434266162 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f8974010ac0 con 0x7f8984103ae0 2024-08-19T01:32:12.796 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:12.794+0000 7f897be00640 1 --2- 172.21.15.83:0/3434266162 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f89680a2c00 0x7f89680a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:32:12.796 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:12.794+0000 7f897be00640 1 -- 172.21.15.83:0/3434266162 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 7127+0+0 (secure 0 0 0) 0x7f89740cefe0 con 0x7f8984103ae0 2024-08-19T01:32:12.796 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:12.794+0000 7f897a000640 1 --2- 172.21.15.83:0/3434266162 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f89680a2c00 0x7f89680a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:32:12.796 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:12.795+0000 7f897a000640 1 --2- 172.21.15.83:0/3434266162 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f89680a2c00 0x7f89680a50c0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f896c000c00 tx=0x7f896c005a20 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:32:12.799 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:12.798+0000 7f897be00640 1 -- 172.21.15.83:0/3434266162 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7f897409c380 con 0x7f8984103ae0 2024-08-19T01:32:13.012 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:13.011+0000 7f8989c00640 1 -- 172.21.15.83:0/3434266162 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f898410c050 con 0x7f8984103ae0 2024-08-19T01:32:13.013 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:13.011+0000 7f897be00640 1 -- 172.21.15.83:0/3434266162 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) ==== 93+0+5566 (secure 0 0 0) 0x7f897409bc80 con 0x7f8984103ae0 2024-08-19T01:32:13.013 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-08-19T01:32:13.013 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":7,"btime":"2024-08-19T01:32:12:462931+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14421,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6835/988556067","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":988556067},{"type":"v1","addr":"172.21.15.83:6835","nonce":988556067}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14433,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6837/869702632","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6836","nonce":869702632},{"type":"v1","addr":"172.21.15.83:6837","nonce":869702632}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24430,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6833/2505076046","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6832","nonce":2505076046},{"type":"v1","addr":"172.21.15.167:6833","nonce":2505076046}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24449,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6833/3557520478","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":3557520478},{"type":"v1","addr":"172.21.15.117:6833","nonce":3557520478}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24466,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6835/3482045966","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":3482045966},{"type":"v1","addr":"172.21.15.167:6835","nonce":3482045966}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24485,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6835/1024213994","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":1024213994},{"type":"v1","addr":"172.21.15.117:6835","nonce":1024213994}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[]} 2024-08-19T01:32:13.014 INFO:teuthology.orchestra.run.smithi083.stderr:dumped fsmap epoch 7 2024-08-19T01:32:13.015 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:13.013+0000 7f8989c00640 1 -- 172.21.15.83:0/3434266162 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f89680a2c00 msgr2=0x7f89680a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:32:13.016 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:13.013+0000 7f8989c00640 1 --2- 172.21.15.83:0/3434266162 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f89680a2c00 0x7f89680a50c0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f896c000c00 tx=0x7f896c005a20 comp rx=0 tx=0).stop 2024-08-19T01:32:13.016 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:13.013+0000 7f8989c00640 1 -- 172.21.15.83:0/3434266162 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8984103ae0 msgr2=0x7f898419a3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:32:13.016 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:13.013+0000 7f8989c00640 1 --2- 172.21.15.83:0/3434266162 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8984103ae0 0x7f898419a3e0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f897400e9f0 tx=0x7f897400edf0 comp rx=0 tx=0).stop 2024-08-19T01:32:13.016 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:13.014+0000 7f8989c00640 1 -- 172.21.15.83:0/3434266162 shutdown_connections 2024-08-19T01:32:13.016 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:13.014+0000 7f8989c00640 1 --2- 172.21.15.83:0/3434266162 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f89680a2c00 0x7f89680a50c0 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:32:13.016 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:13.014+0000 7f8989c00640 1 --2- 172.21.15.83:0/3434266162 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8984109b70 0x7f898419ecb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:32:13.016 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:13.014+0000 7f8989c00640 1 --2- 172.21.15.83:0/3434266162 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8984104480 0x7f898419a920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:32:13.016 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:13.014+0000 7f8989c00640 1 --2- 172.21.15.83:0/3434266162 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8984103ae0 0x7f898419a3e0 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:32:13.016 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:13.014+0000 7f8989c00640 1 -- 172.21.15.83:0/3434266162 >> 172.21.15.83:0/3434266162 conn(0x7f89840ff780 msgr2=0x7f8984100070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:32:13.016 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:13.014+0000 7f8989c00640 1 -- 172.21.15.83:0/3434266162 shutdown_connections 2024-08-19T01:32:13.016 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:13.014+0000 7f8989c00640 1 -- 172.21.15.83:0/3434266162 wait complete. 2024-08-19T01:32:13.099 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:32:12 smithi167 ceph-mon[21875]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' 2024-08-19T01:32:13.099 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:32:12 smithi167 ceph-mon[21875]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' 2024-08-19T01:32:13.100 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:32:12 smithi167 ceph-mon[21875]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' 2024-08-19T01:32:13.100 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:32:12 smithi167 ceph-mon[21875]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-08-19T01:32:13.100 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:32:12 smithi167 ceph-mon[21875]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-08-19T01:32:13.100 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:32:12 smithi167 ceph-mon[21875]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-19T01:32:13.100 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:32:12 smithi167 ceph-mon[21875]: Deploying daemon mds.c on smithi167 2024-08-19T01:32:13.100 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:32:12 smithi167 ceph-mon[21875]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-19T01:32:13.100 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:32:12 smithi167 ceph-mon[21875]: mds.? [v2:172.21.15.167:6834/3482045966,v1:172.21.15.167:6835/3482045966] up:boot 2024-08-19T01:32:13.100 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:32:12 smithi167 ceph-mon[21875]: fsmap 6 up:standby 2024-08-19T01:32:13.100 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:32:12 smithi167 ceph-mon[21875]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-08-19T01:32:13.349 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:32:12 smithi083 ceph-mon[17120]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' 2024-08-19T01:32:13.349 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:32:12 smithi083 ceph-mon[17120]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' 2024-08-19T01:32:13.350 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:32:12 smithi083 ceph-mon[17120]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' 2024-08-19T01:32:13.350 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:32:12 smithi083 ceph-mon[17120]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-08-19T01:32:13.350 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:32:12 smithi083 ceph-mon[17120]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-08-19T01:32:13.350 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:32:12 smithi083 ceph-mon[17120]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-19T01:32:13.350 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:32:12 smithi083 ceph-mon[17120]: Deploying daemon mds.c on smithi167 2024-08-19T01:32:13.350 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:32:12 smithi083 ceph-mon[17120]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-19T01:32:13.350 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:32:12 smithi083 ceph-mon[17120]: mds.? [v2:172.21.15.167:6834/3482045966,v1:172.21.15.167:6835/3482045966] up:boot 2024-08-19T01:32:13.350 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:32:12 smithi083 ceph-mon[17120]: fsmap 6 up:standby 2024-08-19T01:32:13.350 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:32:12 smithi083 ceph-mon[17120]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-08-19T01:32:13.453 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:32:12 smithi117 ceph-mon[22016]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' 2024-08-19T01:32:13.453 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:32:12 smithi117 ceph-mon[22016]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' 2024-08-19T01:32:13.453 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:32:12 smithi117 ceph-mon[22016]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' 2024-08-19T01:32:13.453 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:32:12 smithi117 ceph-mon[22016]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-08-19T01:32:13.453 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:32:12 smithi117 ceph-mon[22016]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-08-19T01:32:13.453 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:32:12 smithi117 ceph-mon[22016]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-19T01:32:13.453 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:32:12 smithi117 ceph-mon[22016]: Deploying daemon mds.c on smithi167 2024-08-19T01:32:13.453 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:32:12 smithi117 ceph-mon[22016]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-19T01:32:13.453 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:32:12 smithi117 ceph-mon[22016]: mds.? [v2:172.21.15.167:6834/3482045966,v1:172.21.15.167:6835/3482045966] up:boot 2024-08-19T01:32:13.453 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:32:12 smithi117 ceph-mon[22016]: fsmap 6 up:standby 2024-08-19T01:32:13.453 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:32:12 smithi117 ceph-mon[22016]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-08-19T01:32:13.966 INFO:journalctl@ceph.mds.c.smithi167.stdout:Aug 19 01:32:13 smithi167 systemd[1]: Starting Ceph mds.c for bad8780e-5dc9-11ef-bccf-c7b262605968... 2024-08-19T01:32:14.154 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:32:13 smithi083 ceph-mon[17120]: from='client.? 172.21.15.83:0/3434266162' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-19T01:32:14.230 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:32:13 smithi167 ceph-mon[21875]: from='client.? 172.21.15.83:0/3434266162' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-19T01:32:14.230 INFO:journalctl@ceph.mds.c.smithi167.stdout:Aug 19 01:32:14 smithi167 podman[51889]: 2024-08-19 01:32:13.961051123 +0000 UTC m=+0.016723727 image pull ea7a5189f30bad3aca9524788e2965d2368d7ce786df09d4edcd8c3edf9cefba quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c9dd28080a631be25c77c36c63b17a3ef340ffb2e21c87d4d7b56b6398436d9 2024-08-19T01:32:14.230 INFO:journalctl@ceph.mds.c.smithi167.stdout:Aug 19 01:32:14 smithi167 podman[51889]: 2024-08-19 01:32:14.084981059 +0000 UTC m=+0.140653650 container create 2ce4263dad56a40f241a6eee6cdf3e9d02f92e141a847bc3ad074e3b5da74cd7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c9dd28080a631be25c77c36c63b17a3ef340ffb2e21c87d4d7b56b6398436d9, name=ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-mds-c, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, GIT_CLEAN=True, ceph=True, CEPH_POINT_RELEASE=, io.buildah.version=1.36.0, RELEASE=squid-de09dd0, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-08-19T01:32:14.452 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:32:13 smithi117 ceph-mon[22016]: from='client.? 172.21.15.83:0/3434266162' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-19T01:32:14.716 INFO:journalctl@ceph.mds.c.smithi167.stdout:Aug 19 01:32:14 smithi167 podman[51889]: 2024-08-19 01:32:14.362646853 +0000 UTC m=+0.418319459 container init 2ce4263dad56a40f241a6eee6cdf3e9d02f92e141a847bc3ad074e3b5da74cd7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c9dd28080a631be25c77c36c63b17a3ef340ffb2e21c87d4d7b56b6398436d9, name=ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-mds-c, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.36.0, RELEASE=squid-de09dd0, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=) 2024-08-19T01:32:14.716 INFO:journalctl@ceph.mds.c.smithi167.stdout:Aug 19 01:32:14 smithi167 podman[51889]: 2024-08-19 01:32:14.365749527 +0000 UTC m=+0.421422117 container start 2ce4263dad56a40f241a6eee6cdf3e9d02f92e141a847bc3ad074e3b5da74cd7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c9dd28080a631be25c77c36c63b17a3ef340ffb2e21c87d4d7b56b6398436d9, name=ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-mds-c, CEPH_POINT_RELEASE=, RELEASE=squid-de09dd0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, io.buildah.version=1.36.0, ceph=True) 2024-08-19T01:32:14.716 INFO:journalctl@ceph.mds.c.smithi167.stdout:Aug 19 01:32:14 smithi167 ceph-mds[51908]: -- 172.21.15.167:0/3413481642 <== mon.1 v2:172.21.15.167:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55e078fa7860 con 0x55e078f85000 2024-08-19T01:32:14.716 INFO:journalctl@ceph.mds.c.smithi167.stdout:Aug 19 01:32:14 smithi167 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-mds-c[51904]: starting mds.c at 2024-08-19T01:32:14.716 INFO:journalctl@ceph.mds.c.smithi167.stdout:Aug 19 01:32:14 smithi167 bash[51889]: 2ce4263dad56a40f241a6eee6cdf3e9d02f92e141a847bc3ad074e3b5da74cd7 2024-08-19T01:32:14.716 INFO:journalctl@ceph.mds.c.smithi167.stdout:Aug 19 01:32:14 smithi167 systemd[1]: Started Ceph mds.c for bad8780e-5dc9-11ef-bccf-c7b262605968. 2024-08-19T01:32:15.655 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:32:15 smithi117 ceph-mon[22016]: pgmap v254: 1 pgs: 1 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-19T01:32:15.655 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:32:15 smithi117 ceph-mon[22016]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' 2024-08-19T01:32:15.655 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:32:15 smithi117 ceph-mon[22016]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' 2024-08-19T01:32:15.655 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:32:15 smithi117 ceph-mon[22016]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' 2024-08-19T01:32:15.655 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:32:15 smithi117 ceph-mon[22016]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-08-19T01:32:15.655 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:32:15 smithi117 ceph-mon[22016]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-08-19T01:32:15.655 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:32:15 smithi117 ceph-mon[22016]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-19T01:32:15.655 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:32:15 smithi117 ceph-mon[22016]: mds.? [v2:172.21.15.167:6836/1762130408,v1:172.21.15.167:6837/1762130408] up:boot 2024-08-19T01:32:15.655 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:32:15 smithi117 ceph-mon[22016]: fsmap 7 up:standby 2024-08-19T01:32:15.655 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:32:15 smithi117 ceph-mon[22016]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-08-19T01:32:15.852 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833 shell --fsid bad8780e-5dc9-11ef-bccf-c7b262605968 -- ceph fs dump --format=json 2024-08-19T01:32:15.904 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:32:15 smithi083 ceph-mon[17120]: pgmap v254: 1 pgs: 1 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-19T01:32:15.905 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:32:15 smithi083 ceph-mon[17120]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' 2024-08-19T01:32:15.905 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:32:15 smithi083 ceph-mon[17120]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' 2024-08-19T01:32:15.905 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:32:15 smithi083 ceph-mon[17120]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' 2024-08-19T01:32:15.905 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:32:15 smithi083 ceph-mon[17120]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-08-19T01:32:15.905 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:32:15 smithi083 ceph-mon[17120]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-08-19T01:32:15.905 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:32:15 smithi083 ceph-mon[17120]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-19T01:32:15.905 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:32:15 smithi083 ceph-mon[17120]: mds.? [v2:172.21.15.167:6836/1762130408,v1:172.21.15.167:6837/1762130408] up:boot 2024-08-19T01:32:15.905 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:32:15 smithi083 ceph-mon[17120]: fsmap 7 up:standby 2024-08-19T01:32:15.905 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:32:15 smithi083 ceph-mon[17120]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-08-19T01:32:15.966 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:32:15 smithi167 ceph-mon[21875]: pgmap v254: 1 pgs: 1 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-19T01:32:15.966 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:32:15 smithi167 ceph-mon[21875]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' 2024-08-19T01:32:15.967 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:32:15 smithi167 ceph-mon[21875]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' 2024-08-19T01:32:15.967 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:32:15 smithi167 ceph-mon[21875]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' 2024-08-19T01:32:15.967 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:32:15 smithi167 ceph-mon[21875]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-08-19T01:32:15.967 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:32:15 smithi167 ceph-mon[21875]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-08-19T01:32:15.967 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:32:15 smithi167 ceph-mon[21875]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-19T01:32:15.967 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:32:15 smithi167 ceph-mon[21875]: mds.? [v2:172.21.15.167:6836/1762130408,v1:172.21.15.167:6837/1762130408] up:boot 2024-08-19T01:32:15.967 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:32:15 smithi167 ceph-mon[21875]: fsmap 7 up:standby 2024-08-19T01:32:15.967 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:32:15 smithi167 ceph-mon[21875]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-08-19T01:32:16.266 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/bad8780e-5dc9-11ef-bccf-c7b262605968/mon.a/config 2024-08-19T01:32:16.523 INFO:journalctl@ceph.mds.h.smithi117.stdout:Aug 19 01:32:16 smithi117 systemd[1]: Starting Ceph mds.h for bad8780e-5dc9-11ef-bccf-c7b262605968... 2024-08-19T01:32:16.831 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:32:16 smithi083 ceph-mon[17120]: Deploying daemon mds.h on smithi117 2024-08-19T01:32:16.834 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:32:16 smithi117 ceph-mon[22016]: Deploying daemon mds.h on smithi117 2024-08-19T01:32:16.835 INFO:journalctl@ceph.mds.h.smithi117.stdout:Aug 19 01:32:16 smithi117 podman[51576]: 2024-08-19 01:32:16.525951444 +0000 UTC m=+0.017963507 image pull ea7a5189f30bad3aca9524788e2965d2368d7ce786df09d4edcd8c3edf9cefba quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c9dd28080a631be25c77c36c63b17a3ef340ffb2e21c87d4d7b56b6398436d9 2024-08-19T01:32:16.835 INFO:journalctl@ceph.mds.h.smithi117.stdout:Aug 19 01:32:16 smithi117 podman[51576]: 2024-08-19 01:32:16.640042195 +0000 UTC m=+0.132054244 container create 682ad742e1a1298db016c9feaa2629176284daf32257eed1eef5c95a55c3b4ef (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c9dd28080a631be25c77c36c63b17a3ef340ffb2e21c87d4d7b56b6398436d9, name=ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-mds-h, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-de09dd0, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, io.buildah.version=1.36.0, ceph=True, maintainer=Guillaume Abrioux ) 2024-08-19T01:32:16.835 INFO:journalctl@ceph.mds.h.smithi117.stdout:Aug 19 01:32:16 smithi117 podman[51576]: 2024-08-19 01:32:16.834027816 +0000 UTC m=+0.326039863 container init 682ad742e1a1298db016c9feaa2629176284daf32257eed1eef5c95a55c3b4ef (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c9dd28080a631be25c77c36c63b17a3ef340ffb2e21c87d4d7b56b6398436d9, name=ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-mds-h, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.36.0, RELEASE=squid-de09dd0, GIT_CLEAN=True) 2024-08-19T01:32:16.862 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:16.860+0000 7f7556400640 1 -- 172.21.15.83:0/44637000 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f75501042a0 msgr2=0x7f75501107d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:32:16.862 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:16.860+0000 7f7556400640 1 --2- 172.21.15.83:0/44637000 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f75501042a0 0x7f75501107d0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f7548009cd0 tx=0x7f754802a8f0 comp rx=0 tx=0).stop 2024-08-19T01:32:16.862 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:16.860+0000 7f7556400640 1 -- 172.21.15.83:0/44637000 shutdown_connections 2024-08-19T01:32:16.862 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:16.860+0000 7f7556400640 1 --2- 172.21.15.83:0/44637000 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f75501042a0 0x7f75501107d0 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:32:16.862 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:16.860+0000 7f7556400640 1 --2- 172.21.15.83:0/44637000 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7550103900 0x7f7550103d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:32:16.862 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:16.860+0000 7f7556400640 1 --2- 172.21.15.83:0/44637000 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7550109900 0x7f7550109ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:32:16.862 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:16.860+0000 7f7556400640 1 -- 172.21.15.83:0/44637000 >> 172.21.15.83:0/44637000 conn(0x7f75500ff600 msgr2=0x7f7550101a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:32:16.862 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:16.860+0000 7f7556400640 1 -- 172.21.15.83:0/44637000 shutdown_connections 2024-08-19T01:32:16.862 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:16.860+0000 7f7556400640 1 -- 172.21.15.83:0/44637000 wait complete. 2024-08-19T01:32:16.863 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:16.861+0000 7f7556400640 1 Processor -- start 2024-08-19T01:32:16.863 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:16.861+0000 7f7556400640 1 -- start start 2024-08-19T01:32:16.863 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:16.861+0000 7f7556400640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7550103900 0x7f755019a1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:32:16.864 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:16.861+0000 7f7556400640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f75501042a0 0x7f755019a720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:32:16.864 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:16.861+0000 7f7556400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7550109900 0x7f755019eab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:32:16.864 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:16.861+0000 7f7556400640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f7550113d00 con 0x7f7550109900 2024-08-19T01:32:16.864 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:16.861+0000 7f7556400640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f7550113b80 con 0x7f7550103900 2024-08-19T01:32:16.864 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:16.861+0000 7f7556400640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f7550113e80 con 0x7f75501042a0 2024-08-19T01:32:16.864 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:16.861+0000 7f7555000640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7550103900 0x7f755019a1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:32:16.864 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:16.862+0000 7f7555000640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7550103900 0x7f755019a1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.83:50776/0 (socket says 172.21.15.83:50776) 2024-08-19T01:32:16.864 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:16.862+0000 7f7555000640 1 -- 172.21.15.83:0/4251509901 learned_addr learned my addr 172.21.15.83:0/4251509901 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-08-19T01:32:16.864 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:16.862+0000 7f7555a00640 1 --2- 172.21.15.83:0/4251509901 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7550109900 0x7f755019eab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:32:16.864 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:16.862+0000 7f754fe00640 1 --2- 172.21.15.83:0/4251509901 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f75501042a0 0x7f755019a720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:32:16.865 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:16.862+0000 7f7555a00640 1 -- 172.21.15.83:0/4251509901 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f75501042a0 msgr2=0x7f755019a720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:32:16.865 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:16.862+0000 7f7555a00640 1 --2- 172.21.15.83:0/4251509901 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f75501042a0 0x7f755019a720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:32:16.865 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:16.862+0000 7f7555a00640 1 -- 172.21.15.83:0/4251509901 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7550103900 msgr2=0x7f755019a1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:32:16.865 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:16.862+0000 7f7555a00640 1 --2- 172.21.15.83:0/4251509901 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7550103900 0x7f755019a1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:32:16.865 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:16.862+0000 7f7555a00640 1 -- 172.21.15.83:0/4251509901 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f755019f190 con 0x7f7550109900 2024-08-19T01:32:16.865 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:16.862+0000 7f7555000640 1 --2- 172.21.15.83:0/4251509901 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7550103900 0x7f755019a1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-19T01:32:16.865 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:16.863+0000 7f7555a00640 1 --2- 172.21.15.83:0/4251509901 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7550109900 0x7f755019eab0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f754802ae10 tx=0x7f7548030ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:32:16.866 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:16.863+0000 7f754d600640 1 -- 172.21.15.83:0/4251509901 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7548003d80 con 0x7f7550109900 2024-08-19T01:32:16.866 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:16.863+0000 7f754d600640 1 -- 172.21.15.83:0/4251509901 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7548040070 con 0x7f7550109900 2024-08-19T01:32:16.866 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:16.863+0000 7f7556400640 1 -- 172.21.15.83:0/4251509901 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f755019f420 con 0x7f7550109900 2024-08-19T01:32:16.866 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:16.863+0000 7f754d600640 1 -- 172.21.15.83:0/4251509901 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7548045ad0 con 0x7f7550109900 2024-08-19T01:32:16.867 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:16.863+0000 7f7556400640 1 -- 172.21.15.83:0/4251509901 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f75501a6d60 con 0x7f7550109900 2024-08-19T01:32:16.867 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:16.864+0000 7f7556400640 1 -- 172.21.15.83:0/4251509901 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7550103d60 con 0x7f7550109900 2024-08-19T01:32:16.869 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:16.867+0000 7f754d600640 1 -- 172.21.15.83:0/4251509901 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f7548004040 con 0x7f7550109900 2024-08-19T01:32:16.870 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:16.868+0000 7f754d600640 1 --2- 172.21.15.83:0/4251509901 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f75380a2b80 0x7f75380a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:32:16.870 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:16.869+0000 7f754d600640 1 -- 172.21.15.83:0/4251509901 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 7127+0+0 (secure 0 0 0) 0x7f75480efc90 con 0x7f7550109900 2024-08-19T01:32:16.870 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:16.869+0000 7f7555000640 1 --2- 172.21.15.83:0/4251509901 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f75380a2b80 0x7f75380a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:32:16.871 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:16.870+0000 7f7555000640 1 --2- 172.21.15.83:0/4251509901 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f75380a2b80 0x7f75380a5040 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f75440095f0 tx=0x7f7544005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:32:16.874 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:16.873+0000 7f754d600640 1 -- 172.21.15.83:0/4251509901 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7f75480bcff0 con 0x7f7550109900 2024-08-19T01:32:16.966 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:32:16 smithi167 ceph-mon[21875]: Deploying daemon mds.h on smithi117 2024-08-19T01:32:17.094 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:17.092+0000 7f7556400640 1 -- 172.21.15.83:0/4251509901 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f755010be90 con 0x7f7550109900 2024-08-19T01:32:17.095 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:17.093+0000 7f754d600640 1 -- 172.21.15.83:0/4251509901 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) ==== 93+0+6394 (secure 0 0 0) 0x7f75480bc8f0 con 0x7f7550109900 2024-08-19T01:32:17.095 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-08-19T01:32:17.095 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":8,"btime":"2024-08-19T01:32:14:969250+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14421,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6835/988556067","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":988556067},{"type":"v1","addr":"172.21.15.83:6835","nonce":988556067}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14433,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6837/869702632","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6836","nonce":869702632},{"type":"v1","addr":"172.21.15.83:6837","nonce":869702632}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24430,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6833/2505076046","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6832","nonce":2505076046},{"type":"v1","addr":"172.21.15.167:6833","nonce":2505076046}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24449,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6833/3557520478","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":3557520478},{"type":"v1","addr":"172.21.15.117:6833","nonce":3557520478}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24466,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6835/3482045966","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":3482045966},{"type":"v1","addr":"172.21.15.167:6835","nonce":3482045966}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24478,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6837/1762130408","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6836","nonce":1762130408},{"type":"v1","addr":"172.21.15.167:6837","nonce":1762130408}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24485,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6835/1024213994","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":1024213994},{"type":"v1","addr":"172.21.15.117:6835","nonce":1024213994}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[]} 2024-08-19T01:32:17.095 INFO:teuthology.orchestra.run.smithi083.stderr:dumped fsmap epoch 8 2024-08-19T01:32:17.097 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:17.095+0000 7f7556400640 1 -- 172.21.15.83:0/4251509901 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f75380a2b80 msgr2=0x7f75380a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:32:17.097 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:17.095+0000 7f7556400640 1 --2- 172.21.15.83:0/4251509901 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f75380a2b80 0x7f75380a5040 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f75440095f0 tx=0x7f7544005f90 comp rx=0 tx=0).stop 2024-08-19T01:32:17.097 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:17.095+0000 7f7556400640 1 -- 172.21.15.83:0/4251509901 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7550109900 msgr2=0x7f755019eab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:32:17.097 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:17.095+0000 7f7556400640 1 --2- 172.21.15.83:0/4251509901 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7550109900 0x7f755019eab0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f754802ae10 tx=0x7f7548030ed0 comp rx=0 tx=0).stop 2024-08-19T01:32:17.097 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:17.095+0000 7f7556400640 1 -- 172.21.15.83:0/4251509901 shutdown_connections 2024-08-19T01:32:17.098 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:17.095+0000 7f7556400640 1 --2- 172.21.15.83:0/4251509901 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f75380a2b80 0x7f75380a5040 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:32:17.098 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:17.095+0000 7f7556400640 1 --2- 172.21.15.83:0/4251509901 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7550109900 0x7f755019eab0 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:32:17.098 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:17.095+0000 7f7556400640 1 --2- 172.21.15.83:0/4251509901 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f75501042a0 0x7f755019a720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:32:17.098 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:17.095+0000 7f7556400640 1 --2- 172.21.15.83:0/4251509901 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7550103900 0x7f755019a1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:32:17.098 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:17.095+0000 7f7556400640 1 -- 172.21.15.83:0/4251509901 >> 172.21.15.83:0/4251509901 conn(0x7f75500ff600 msgr2=0x7f7550100da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:32:17.098 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:17.096+0000 7f7556400640 1 -- 172.21.15.83:0/4251509901 shutdown_connections 2024-08-19T01:32:17.098 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:17.096+0000 7f7556400640 1 -- 172.21.15.83:0/4251509901 wait complete. 2024-08-19T01:32:17.202 INFO:journalctl@ceph.mds.h.smithi117.stdout:Aug 19 01:32:16 smithi117 podman[51576]: 2024-08-19 01:32:16.837110497 +0000 UTC m=+0.329122555 container start 682ad742e1a1298db016c9feaa2629176284daf32257eed1eef5c95a55c3b4ef (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c9dd28080a631be25c77c36c63b17a3ef340ffb2e21c87d4d7b56b6398436d9, name=ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-mds-h, GIT_CLEAN=True, org.label-schema.license=GPLv2, io.buildah.version=1.36.0, RELEASE=squid-de09dd0, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, ceph=True) 2024-08-19T01:32:17.203 INFO:journalctl@ceph.mds.h.smithi117.stdout:Aug 19 01:32:16 smithi117 ceph-mds[51595]: -- 172.21.15.117:0/4227801709 <== mon.2 v2:172.21.15.117:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x564ffdc2d860 con 0x564ffdc0a800 2024-08-19T01:32:17.203 INFO:journalctl@ceph.mds.h.smithi117.stdout:Aug 19 01:32:16 smithi117 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-mds-h[51591]: starting mds.h at 2024-08-19T01:32:17.203 INFO:journalctl@ceph.mds.h.smithi117.stdout:Aug 19 01:32:16 smithi117 bash[51576]: 682ad742e1a1298db016c9feaa2629176284daf32257eed1eef5c95a55c3b4ef 2024-08-19T01:32:17.203 INFO:journalctl@ceph.mds.h.smithi117.stdout:Aug 19 01:32:16 smithi117 systemd[1]: Started Ceph mds.h for bad8780e-5dc9-11ef-bccf-c7b262605968. 2024-08-19T01:32:17.696 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:32:17 smithi083 ceph-mon[17120]: pgmap v255: 1 pgs: 1 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-19T01:32:17.696 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:32:17 smithi083 ceph-mon[17120]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' 2024-08-19T01:32:17.696 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:32:17 smithi083 ceph-mon[17120]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' 2024-08-19T01:32:17.697 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:32:17 smithi083 ceph-mon[17120]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' 2024-08-19T01:32:17.697 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:32:17 smithi083 ceph-mon[17120]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-08-19T01:32:17.697 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:32:17 smithi083 ceph-mon[17120]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-08-19T01:32:17.697 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:32:17 smithi083 ceph-mon[17120]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-19T01:32:17.697 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:32:17 smithi083 ceph-mon[17120]: from='client.? 172.21.15.83:0/4251509901' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-19T01:32:17.716 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:32:17 smithi167 ceph-mon[21875]: pgmap v255: 1 pgs: 1 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-19T01:32:17.716 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:32:17 smithi167 ceph-mon[21875]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' 2024-08-19T01:32:17.717 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:32:17 smithi167 ceph-mon[21875]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' 2024-08-19T01:32:17.717 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:32:17 smithi167 ceph-mon[21875]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' 2024-08-19T01:32:17.717 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:32:17 smithi167 ceph-mon[21875]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-08-19T01:32:17.717 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:32:17 smithi167 ceph-mon[21875]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-08-19T01:32:17.717 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:32:17 smithi167 ceph-mon[21875]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-19T01:32:17.717 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:32:17 smithi167 ceph-mon[21875]: from='client.? 172.21.15.83:0/4251509901' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-19T01:32:17.952 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:32:17 smithi117 ceph-mon[22016]: pgmap v255: 1 pgs: 1 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-19T01:32:17.953 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:32:17 smithi117 ceph-mon[22016]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' 2024-08-19T01:32:17.953 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:32:17 smithi117 ceph-mon[22016]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' 2024-08-19T01:32:17.953 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:32:17 smithi117 ceph-mon[22016]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' 2024-08-19T01:32:17.953 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:32:17 smithi117 ceph-mon[22016]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-08-19T01:32:17.953 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:32:17 smithi117 ceph-mon[22016]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-08-19T01:32:17.953 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:32:17 smithi117 ceph-mon[22016]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-19T01:32:17.953 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:32:17 smithi117 ceph-mon[22016]: from='client.? 172.21.15.83:0/4251509901' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-19T01:32:18.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:32:18 smithi167 ceph-mon[21875]: Deploying daemon mds.i on smithi167 2024-08-19T01:32:18.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:32:18 smithi167 ceph-mon[21875]: mds.? [v2:172.21.15.117:6836/3185224684,v1:172.21.15.117:6837/3185224684] up:boot 2024-08-19T01:32:18.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:32:18 smithi167 ceph-mon[21875]: fsmap 8 up:standby 2024-08-19T01:32:18.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:32:18 smithi167 ceph-mon[21875]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-08-19T01:32:18.905 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:32:18 smithi083 ceph-mon[17120]: Deploying daemon mds.i on smithi167 2024-08-19T01:32:18.905 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:32:18 smithi083 ceph-mon[17120]: mds.? [v2:172.21.15.117:6836/3185224684,v1:172.21.15.117:6837/3185224684] up:boot 2024-08-19T01:32:18.905 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:32:18 smithi083 ceph-mon[17120]: fsmap 8 up:standby 2024-08-19T01:32:18.905 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:32:18 smithi083 ceph-mon[17120]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-08-19T01:32:18.952 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:32:18 smithi117 ceph-mon[22016]: Deploying daemon mds.i on smithi167 2024-08-19T01:32:18.952 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:32:18 smithi117 ceph-mon[22016]: mds.? [v2:172.21.15.117:6836/3185224684,v1:172.21.15.117:6837/3185224684] up:boot 2024-08-19T01:32:18.953 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:32:18 smithi117 ceph-mon[22016]: fsmap 8 up:standby 2024-08-19T01:32:18.953 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:32:18 smithi117 ceph-mon[22016]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-08-19T01:32:19.187 INFO:journalctl@ceph.mds.i.smithi167.stdout:Aug 19 01:32:18 smithi167 systemd[1]: Starting Ceph mds.i for bad8780e-5dc9-11ef-bccf-c7b262605968... 2024-08-19T01:32:19.459 INFO:journalctl@ceph.mds.i.smithi167.stdout:Aug 19 01:32:19 smithi167 podman[52203]: 2024-08-19 01:32:19.088092258 +0000 UTC m=+0.017449978 image pull ea7a5189f30bad3aca9524788e2965d2368d7ce786df09d4edcd8c3edf9cefba quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c9dd28080a631be25c77c36c63b17a3ef340ffb2e21c87d4d7b56b6398436d9 2024-08-19T01:32:19.459 INFO:journalctl@ceph.mds.i.smithi167.stdout:Aug 19 01:32:19 smithi167 podman[52203]: 2024-08-19 01:32:19.221784817 +0000 UTC m=+0.151142529 container create 7b30511a109093df3b1d6a32480cc08f09b5186d048c958e19c4c35f9cab2947 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c9dd28080a631be25c77c36c63b17a3ef340ffb2e21c87d4d7b56b6398436d9, name=ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-mds-i, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.license=GPLv2, RELEASE=squid-de09dd0, org.label-schema.build-date=20240731, io.buildah.version=1.36.0, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=) 2024-08-19T01:32:19.459 INFO:journalctl@ceph.mds.i.smithi167.stdout:Aug 19 01:32:19 smithi167 podman[52203]: 2024-08-19 01:32:19.432545994 +0000 UTC m=+0.361903703 container init 7b30511a109093df3b1d6a32480cc08f09b5186d048c958e19c4c35f9cab2947 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c9dd28080a631be25c77c36c63b17a3ef340ffb2e21c87d4d7b56b6398436d9, name=ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-mds-i, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , ceph=True, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.36.0, org.label-schema.vendor=CentOS, RELEASE=squid-de09dd0, GIT_BRANCH=HEAD) 2024-08-19T01:32:19.459 INFO:journalctl@ceph.mds.i.smithi167.stdout:Aug 19 01:32:19 smithi167 podman[52203]: 2024-08-19 01:32:19.435526179 +0000 UTC m=+0.364883889 container start 7b30511a109093df3b1d6a32480cc08f09b5186d048c958e19c4c35f9cab2947 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c9dd28080a631be25c77c36c63b17a3ef340ffb2e21c87d4d7b56b6398436d9, name=ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-mds-i, GIT_CLEAN=True, RELEASE=squid-de09dd0, maintainer=Guillaume Abrioux , ceph=True, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, io.buildah.version=1.36.0, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-08-19T01:32:19.702 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:32:19 smithi117 ceph-mon[22016]: pgmap v256: 1 pgs: 1 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-19T01:32:19.716 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:32:19 smithi167 ceph-mon[21875]: pgmap v256: 1 pgs: 1 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-19T01:32:19.716 INFO:journalctl@ceph.mds.i.smithi167.stdout:Aug 19 01:32:19 smithi167 ceph-mds[52223]: -- 172.21.15.167:0/2158664852 <== mon.1 v2:172.21.15.167:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55bb06997860 con 0x55bb06974800 2024-08-19T01:32:19.716 INFO:journalctl@ceph.mds.i.smithi167.stdout:Aug 19 01:32:19 smithi167 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-mds-i[52219]: starting mds.i at 2024-08-19T01:32:19.716 INFO:journalctl@ceph.mds.i.smithi167.stdout:Aug 19 01:32:19 smithi167 bash[52203]: 7b30511a109093df3b1d6a32480cc08f09b5186d048c958e19c4c35f9cab2947 2024-08-19T01:32:19.716 INFO:journalctl@ceph.mds.i.smithi167.stdout:Aug 19 01:32:19 smithi167 systemd[1]: Started Ceph mds.i for bad8780e-5dc9-11ef-bccf-c7b262605968. 2024-08-19T01:32:19.904 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:32:19 smithi083 ceph-mon[17120]: pgmap v256: 1 pgs: 1 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-19T01:32:20.114 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833 shell --fsid bad8780e-5dc9-11ef-bccf-c7b262605968 -- ceph fs dump --format=json 2024-08-19T01:32:20.905 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:32:20 smithi083 ceph-mon[17120]: mds.? [v2:172.21.15.167:6838/3507387682,v1:172.21.15.167:6839/3507387682] up:boot 2024-08-19T01:32:20.905 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:32:20 smithi083 ceph-mon[17120]: fsmap 9 up:standby 2024-08-19T01:32:20.905 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:32:20 smithi083 ceph-mon[17120]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-08-19T01:32:20.905 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:32:20 smithi083 ceph-mon[17120]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' 2024-08-19T01:32:20.905 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:32:20 smithi083 ceph-mon[17120]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' 2024-08-19T01:32:20.905 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:32:20 smithi083 ceph-mon[17120]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' 2024-08-19T01:32:20.905 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:32:20 smithi083 ceph-mon[17120]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-08-19T01:32:20.905 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:32:20 smithi083 ceph-mon[17120]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-08-19T01:32:20.905 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:32:20 smithi083 ceph-mon[17120]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-19T01:32:20.906 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:32:20 smithi083 ceph-mon[17120]: Deploying daemon mds.g on smithi083 2024-08-19T01:32:20.906 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:32:20 smithi083 ceph-mon[17120]: pgmap v257: 1 pgs: 1 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-19T01:32:20.952 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:32:20 smithi117 ceph-mon[22016]: mds.? [v2:172.21.15.167:6838/3507387682,v1:172.21.15.167:6839/3507387682] up:boot 2024-08-19T01:32:20.953 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:32:20 smithi117 ceph-mon[22016]: fsmap 9 up:standby 2024-08-19T01:32:20.953 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:32:20 smithi117 ceph-mon[22016]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-08-19T01:32:20.953 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:32:20 smithi117 ceph-mon[22016]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' 2024-08-19T01:32:20.953 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:32:20 smithi117 ceph-mon[22016]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' 2024-08-19T01:32:20.953 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:32:20 smithi117 ceph-mon[22016]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' 2024-08-19T01:32:20.953 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:32:20 smithi117 ceph-mon[22016]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-08-19T01:32:20.953 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:32:20 smithi117 ceph-mon[22016]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-08-19T01:32:20.953 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:32:20 smithi117 ceph-mon[22016]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-19T01:32:20.953 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:32:20 smithi117 ceph-mon[22016]: Deploying daemon mds.g on smithi083 2024-08-19T01:32:20.953 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:32:20 smithi117 ceph-mon[22016]: pgmap v257: 1 pgs: 1 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-19T01:32:20.966 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:32:20 smithi167 ceph-mon[21875]: mds.? [v2:172.21.15.167:6838/3507387682,v1:172.21.15.167:6839/3507387682] up:boot 2024-08-19T01:32:20.966 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:32:20 smithi167 ceph-mon[21875]: fsmap 9 up:standby 2024-08-19T01:32:20.966 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:32:20 smithi167 ceph-mon[21875]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-08-19T01:32:20.966 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:32:20 smithi167 ceph-mon[21875]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' 2024-08-19T01:32:20.966 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:32:20 smithi167 ceph-mon[21875]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' 2024-08-19T01:32:20.966 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:32:20 smithi167 ceph-mon[21875]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' 2024-08-19T01:32:20.966 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:32:20 smithi167 ceph-mon[21875]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-08-19T01:32:20.966 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:32:20 smithi167 ceph-mon[21875]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-08-19T01:32:20.966 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:32:20 smithi167 ceph-mon[21875]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-19T01:32:20.967 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:32:20 smithi167 ceph-mon[21875]: Deploying daemon mds.g on smithi083 2024-08-19T01:32:20.967 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:32:20 smithi167 ceph-mon[21875]: pgmap v257: 1 pgs: 1 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-19T01:32:21.626 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/bad8780e-5dc9-11ef-bccf-c7b262605968/mon.a/config 2024-08-19T01:32:22.382 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:22.376+0000 7f0252600640 1 -- 172.21.15.83:0/1654499216 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f024c1031e0 msgr2=0x7f024c10b6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:32:22.382 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:22.376+0000 7f0252600640 1 --2- 172.21.15.83:0/1654499216 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f024c1031e0 0x7f024c10b6d0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f0238009900 tx=0x7f023802a480 comp rx=0 tx=0).stop 2024-08-19T01:32:22.382 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:22.376+0000 7f0252600640 1 -- 172.21.15.83:0/1654499216 shutdown_connections 2024-08-19T01:32:22.382 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:22.376+0000 7f0252600640 1 --2- 172.21.15.83:0/1654499216 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f024c10bcf0 0x7f024c10e190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:32:22.382 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:22.376+0000 7f0252600640 1 --2- 172.21.15.83:0/1654499216 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f024c1031e0 0x7f024c10b6d0 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:32:22.383 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:22.376+0000 7f0252600640 1 --2- 172.21.15.83:0/1654499216 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f024c1028c0 0x7f024c102ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:32:22.383 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:22.376+0000 7f0252600640 1 -- 172.21.15.83:0/1654499216 >> 172.21.15.83:0/1654499216 conn(0x7f024c0fc340 msgr2=0x7f024c0fe760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:32:22.383 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:22.376+0000 7f0252600640 1 -- 172.21.15.83:0/1654499216 shutdown_connections 2024-08-19T01:32:22.383 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:22.376+0000 7f0252600640 1 -- 172.21.15.83:0/1654499216 wait complete. 2024-08-19T01:32:22.383 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:22.377+0000 7f0252600640 1 Processor -- start 2024-08-19T01:32:22.383 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:22.377+0000 7f0252600640 1 -- start start 2024-08-19T01:32:22.383 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:22.378+0000 7f0252600640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f024c1028c0 0x7f024c198030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:32:22.383 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:22.378+0000 7f0252600640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f024c1031e0 0x7f024c198570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:32:22.383 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:22.378+0000 7f0252600640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f024c10bcf0 0x7f024c19c900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:32:22.383 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:22.378+0000 7f0252600640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f024c111b40 con 0x7f024c1028c0 2024-08-19T01:32:22.383 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:22.378+0000 7f0252600640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f024c1119c0 con 0x7f024c1031e0 2024-08-19T01:32:22.383 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:22.378+0000 7f0252600640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f024c111cc0 con 0x7f024c10bcf0 2024-08-19T01:32:22.383 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:22.378+0000 7f0251200640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f024c1028c0 0x7f024c198030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:32:22.384 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:22.378+0000 7f0251200640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f024c1028c0 0x7f024c198030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:34162/0 (socket says 172.21.15.83:34162) 2024-08-19T01:32:22.384 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:22.378+0000 7f0251200640 1 -- 172.21.15.83:0/3957876884 learned_addr learned my addr 172.21.15.83:0/3957876884 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-08-19T01:32:22.384 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:22.378+0000 7f024be00640 1 --2- 172.21.15.83:0/3957876884 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f024c1031e0 0x7f024c198570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:32:22.384 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:22.378+0000 7f0251c00640 1 --2- 172.21.15.83:0/3957876884 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f024c10bcf0 0x7f024c19c900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:32:22.384 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:22.378+0000 7f0251200640 1 -- 172.21.15.83:0/3957876884 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f024c10bcf0 msgr2=0x7f024c19c900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:32:22.384 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:22.378+0000 7f0251200640 1 --2- 172.21.15.83:0/3957876884 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f024c10bcf0 0x7f024c19c900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:32:22.384 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:22.378+0000 7f0251200640 1 -- 172.21.15.83:0/3957876884 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f024c1031e0 msgr2=0x7f024c198570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:32:22.384 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:22.378+0000 7f0251200640 1 --2- 172.21.15.83:0/3957876884 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f024c1031e0 0x7f024c198570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:32:22.384 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:22.378+0000 7f0251200640 1 -- 172.21.15.83:0/3957876884 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f024c19cfe0 con 0x7f024c1028c0 2024-08-19T01:32:22.384 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:22.379+0000 7f0251200640 1 --2- 172.21.15.83:0/3957876884 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f024c1028c0 0x7f024c198030 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f024000d950 tx=0x7f024000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:32:22.384 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:22.379+0000 7f0249600640 1 -- 172.21.15.83:0/3957876884 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0240017070 con 0x7f024c1028c0 2024-08-19T01:32:22.384 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:22.379+0000 7f0252600640 1 -- 172.21.15.83:0/3957876884 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f024c19d2d0 con 0x7f024c1028c0 2024-08-19T01:32:22.384 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:22.379+0000 7f0249600640 1 -- 172.21.15.83:0/3957876884 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0240009c30 con 0x7f024c1028c0 2024-08-19T01:32:22.384 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:22.379+0000 7f0249600640 1 -- 172.21.15.83:0/3957876884 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f024000f6b0 con 0x7f024c1028c0 2024-08-19T01:32:22.385 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:22.379+0000 7f0252600640 1 -- 172.21.15.83:0/3957876884 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f024c1a4bb0 con 0x7f024c1028c0 2024-08-19T01:32:22.385 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:22.380+0000 7f0252600640 1 -- 172.21.15.83:0/3957876884 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f024c102ca0 con 0x7f024c1028c0 2024-08-19T01:32:22.385 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:22.382+0000 7f0249600640 1 -- 172.21.15.83:0/3957876884 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f02400109d0 con 0x7f024c1028c0 2024-08-19T01:32:22.385 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:22.383+0000 7f0249600640 1 --2- 172.21.15.83:0/3957876884 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f02340a2b80 0x7f02340a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:32:22.386 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:22.384+0000 7f0249600640 1 -- 172.21.15.83:0/3957876884 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 7127+0+0 (secure 0 0 0) 0x7f02400ce980 con 0x7f024c1028c0 2024-08-19T01:32:22.386 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:22.384+0000 7f024be00640 1 --2- 172.21.15.83:0/3957876884 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f02340a2b80 0x7f02340a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:32:22.386 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:22.384+0000 7f024be00640 1 --2- 172.21.15.83:0/3957876884 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f02340a2b80 0x7f02340a5040 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f0238000c00 tx=0x7f0238005a20 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:32:22.389 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:22.388+0000 7f0249600640 1 -- 172.21.15.83:0/3957876884 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7f0240009de0 con 0x7f024c1028c0 2024-08-19T01:32:22.609 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:22.607+0000 7f0252600640 1 -- 172.21.15.83:0/3957876884 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f024c106dc0 con 0x7f024c1028c0 2024-08-19T01:32:22.610 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:22.608+0000 7f0249600640 1 -- 172.21.15.83:0/3957876884 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 10 v10) ==== 94+0+8052 (secure 0 0 0) 0x7f024009bce0 con 0x7f024c1028c0 2024-08-19T01:32:22.610 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-08-19T01:32:22.610 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":10,"btime":"2024-08-19T01:32:19:536150+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14421,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6835/988556067","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":988556067},{"type":"v1","addr":"172.21.15.83:6835","nonce":988556067}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14433,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6837/869702632","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6836","nonce":869702632},{"type":"v1","addr":"172.21.15.83:6837","nonce":869702632}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24430,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6833/2505076046","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6832","nonce":2505076046},{"type":"v1","addr":"172.21.15.167:6833","nonce":2505076046}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24449,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6833/3557520478","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":3557520478},{"type":"v1","addr":"172.21.15.117:6833","nonce":3557520478}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24466,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6835/3482045966","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":3482045966},{"type":"v1","addr":"172.21.15.167:6835","nonce":3482045966}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24478,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6837/1762130408","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6836","nonce":1762130408},{"type":"v1","addr":"172.21.15.167:6837","nonce":1762130408}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24485,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6835/1024213994","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":1024213994},{"type":"v1","addr":"172.21.15.117:6835","nonce":1024213994}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24496,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6839/3507387682","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6838","nonce":3507387682},{"type":"v1","addr":"172.21.15.167:6839","nonce":3507387682}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":24509,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6837/3185224684","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":3185224684},{"type":"v1","addr":"172.21.15.117:6837","nonce":3185224684}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9}],"filesystems":[]} 2024-08-19T01:32:22.610 INFO:teuthology.orchestra.run.smithi083.stderr:dumped fsmap epoch 10 2024-08-19T01:32:22.612 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:22.610+0000 7f0252600640 1 -- 172.21.15.83:0/3957876884 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f02340a2b80 msgr2=0x7f02340a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:32:22.612 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:22.610+0000 7f0252600640 1 --2- 172.21.15.83:0/3957876884 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f02340a2b80 0x7f02340a5040 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f0238000c00 tx=0x7f0238005a20 comp rx=0 tx=0).stop 2024-08-19T01:32:22.612 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:22.610+0000 7f0252600640 1 -- 172.21.15.83:0/3957876884 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f024c1028c0 msgr2=0x7f024c198030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:32:22.612 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:22.610+0000 7f0252600640 1 --2- 172.21.15.83:0/3957876884 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f024c1028c0 0x7f024c198030 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f024000d950 tx=0x7f024000de10 comp rx=0 tx=0).stop 2024-08-19T01:32:22.612 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:22.610+0000 7f0252600640 1 -- 172.21.15.83:0/3957876884 shutdown_connections 2024-08-19T01:32:22.612 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:22.610+0000 7f0252600640 1 --2- 172.21.15.83:0/3957876884 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f02340a2b80 0x7f02340a5040 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:32:22.612 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:22.610+0000 7f0252600640 1 --2- 172.21.15.83:0/3957876884 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f024c10bcf0 0x7f024c19c900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:32:22.612 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:22.610+0000 7f0252600640 1 --2- 172.21.15.83:0/3957876884 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f024c1031e0 0x7f024c198570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:32:22.613 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:22.610+0000 7f0252600640 1 --2- 172.21.15.83:0/3957876884 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f024c1028c0 0x7f024c198030 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:32:22.613 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:22.610+0000 7f0252600640 1 -- 172.21.15.83:0/3957876884 >> 172.21.15.83:0/3957876884 conn(0x7f024c0fc340 msgr2=0x7f024c0fd080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:32:22.613 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:22.610+0000 7f0252600640 1 -- 172.21.15.83:0/3957876884 shutdown_connections 2024-08-19T01:32:22.613 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:22.610+0000 7f0252600640 1 -- 172.21.15.83:0/3957876884 wait complete. 2024-08-19T01:32:23.702 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:32:23 smithi117 ceph-mon[22016]: pgmap v258: 1 pgs: 1 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-19T01:32:23.702 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:32:23 smithi117 ceph-mon[22016]: from='client.? 172.21.15.83:0/3957876884' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-19T01:32:23.716 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:32:23 smithi167 ceph-mon[21875]: pgmap v258: 1 pgs: 1 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-19T01:32:23.716 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:32:23 smithi167 ceph-mon[21875]: from='client.? 172.21.15.83:0/3957876884' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-19T01:32:23.724 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:32:23 smithi083 ceph-mon[17120]: pgmap v258: 1 pgs: 1 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-19T01:32:23.724 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:32:23 smithi083 ceph-mon[17120]: from='client.? 172.21.15.83:0/3957876884' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-19T01:32:24.154 INFO:journalctl@ceph.mds.g.smithi083.stdout:Aug 19 01:32:23 smithi083 systemd[1]: Started Ceph mds.g for bad8780e-5dc9-11ef-bccf-c7b262605968. 2024-08-19T01:32:25.155 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:32:24 smithi083 ceph-mon[17120]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' 2024-08-19T01:32:25.155 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:32:24 smithi083 ceph-mon[17120]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' 2024-08-19T01:32:25.155 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:32:24 smithi083 ceph-mon[17120]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' 2024-08-19T01:32:25.155 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:32:24 smithi083 ceph-mon[17120]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-08-19T01:32:25.155 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:32:24 smithi083 ceph-mon[17120]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-08-19T01:32:25.155 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:32:24 smithi083 ceph-mon[17120]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-19T01:32:25.155 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:32:24 smithi083 ceph-mon[17120]: Deploying daemon mds.a on smithi083 2024-08-19T01:32:25.155 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:32:24 smithi083 ceph-mon[17120]: pgmap v259: 1 pgs: 1 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-19T01:32:25.155 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:32:24 smithi083 ceph-mon[17120]: mds.? [v2:172.21.15.83:6838/3239261845,v1:172.21.15.83:6839/3239261845] up:boot 2024-08-19T01:32:25.155 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:32:24 smithi083 ceph-mon[17120]: fsmap 10 up:standby 2024-08-19T01:32:25.156 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:32:24 smithi083 ceph-mon[17120]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-08-19T01:32:25.202 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:32:24 smithi117 ceph-mon[22016]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' 2024-08-19T01:32:25.202 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:32:24 smithi117 ceph-mon[22016]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' 2024-08-19T01:32:25.203 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:32:24 smithi117 ceph-mon[22016]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' 2024-08-19T01:32:25.203 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:32:24 smithi117 ceph-mon[22016]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-08-19T01:32:25.203 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:32:24 smithi117 ceph-mon[22016]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-08-19T01:32:25.203 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:32:24 smithi117 ceph-mon[22016]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-19T01:32:25.203 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:32:24 smithi117 ceph-mon[22016]: Deploying daemon mds.a on smithi083 2024-08-19T01:32:25.203 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:32:24 smithi117 ceph-mon[22016]: pgmap v259: 1 pgs: 1 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-19T01:32:25.203 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:32:24 smithi117 ceph-mon[22016]: mds.? [v2:172.21.15.83:6838/3239261845,v1:172.21.15.83:6839/3239261845] up:boot 2024-08-19T01:32:25.203 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:32:24 smithi117 ceph-mon[22016]: fsmap 10 up:standby 2024-08-19T01:32:25.203 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:32:24 smithi117 ceph-mon[22016]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-08-19T01:32:25.216 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:32:24 smithi167 ceph-mon[21875]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' 2024-08-19T01:32:25.216 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:32:24 smithi167 ceph-mon[21875]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' 2024-08-19T01:32:25.216 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:32:24 smithi167 ceph-mon[21875]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' 2024-08-19T01:32:25.216 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:32:24 smithi167 ceph-mon[21875]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-08-19T01:32:25.217 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:32:24 smithi167 ceph-mon[21875]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-08-19T01:32:25.217 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:32:24 smithi167 ceph-mon[21875]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-19T01:32:25.217 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:32:24 smithi167 ceph-mon[21875]: Deploying daemon mds.a on smithi083 2024-08-19T01:32:25.217 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:32:24 smithi167 ceph-mon[21875]: pgmap v259: 1 pgs: 1 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-19T01:32:25.217 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:32:24 smithi167 ceph-mon[21875]: mds.? [v2:172.21.15.83:6838/3239261845,v1:172.21.15.83:6839/3239261845] up:boot 2024-08-19T01:32:25.217 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:32:24 smithi167 ceph-mon[21875]: fsmap 10 up:standby 2024-08-19T01:32:25.217 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:32:24 smithi167 ceph-mon[21875]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-08-19T01:32:25.475 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833 shell --fsid bad8780e-5dc9-11ef-bccf-c7b262605968 -- ceph fs dump --format=json 2024-08-19T01:32:25.932 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/bad8780e-5dc9-11ef-bccf-c7b262605968/mon.a/config 2024-08-19T01:32:26.821 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:26.758+0000 7f486b200640 1 -- 172.21.15.83:0/2257450101 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f486c1076f0 msgr2=0x7f486c107ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:32:26.822 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:26.758+0000 7f486b200640 1 --2- 172.21.15.83:0/2257450101 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f486c1076f0 0x7f486c107ad0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f485c009890 tx=0x7f485c02a380 comp rx=0 tx=0).stop 2024-08-19T01:32:26.822 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:26.758+0000 7f486b200640 1 -- 172.21.15.83:0/2257450101 shutdown_connections 2024-08-19T01:32:26.822 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:26.758+0000 7f486b200640 1 --2- 172.21.15.83:0/2257450101 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f486c102090 0x7f486c10e5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:32:26.822 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:26.758+0000 7f486b200640 1 --2- 172.21.15.83:0/2257450101 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f486c1016f0 0x7f486c101b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:32:26.822 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:26.758+0000 7f486b200640 1 --2- 172.21.15.83:0/2257450101 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f486c1076f0 0x7f486c107ad0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:32:26.822 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:26.758+0000 7f486b200640 1 -- 172.21.15.83:0/2257450101 >> 172.21.15.83:0/2257450101 conn(0x7f486c0fd470 msgr2=0x7f486c0ff890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:32:26.822 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:26.758+0000 7f486b200640 1 -- 172.21.15.83:0/2257450101 shutdown_connections 2024-08-19T01:32:26.822 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:26.758+0000 7f486b200640 1 -- 172.21.15.83:0/2257450101 wait complete. 2024-08-19T01:32:26.822 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:26.760+0000 7f486b200640 1 Processor -- start 2024-08-19T01:32:26.822 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:26.760+0000 7f486b200640 1 -- start start 2024-08-19T01:32:26.822 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:26.760+0000 7f486b200640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f486c1016f0 0x7f486c19a210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:32:26.822 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:26.760+0000 7f486b200640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f486c102090 0x7f486c19a750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:32:26.823 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:26.760+0000 7f486b200640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f486c1076f0 0x7f486c19eae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:32:26.823 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:26.760+0000 7f486b200640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f486c111af0 con 0x7f486c1076f0 2024-08-19T01:32:26.823 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:26.760+0000 7f486b200640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f486c111970 con 0x7f486c102090 2024-08-19T01:32:26.823 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:26.760+0000 7f486b200640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f486c111c70 con 0x7f486c1016f0 2024-08-19T01:32:26.823 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:26.761+0000 7f4863e00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f486c102090 0x7f486c19a750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:32:26.823 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:26.761+0000 7f4863e00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f486c102090 0x7f486c19a750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.83:36548/0 (socket says 172.21.15.83:36548) 2024-08-19T01:32:26.823 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:26.761+0000 7f486a800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f486c1076f0 0x7f486c19eae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:32:26.823 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:26.761+0000 7f4863e00640 1 -- 172.21.15.83:0/1323017529 learned_addr learned my addr 172.21.15.83:0/1323017529 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-08-19T01:32:26.823 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:26.761+0000 7f4869e00640 1 --2- 172.21.15.83:0/1323017529 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f486c1016f0 0x7f486c19a210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:32:26.823 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:26.761+0000 7f4863e00640 1 -- 172.21.15.83:0/1323017529 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f486c1016f0 msgr2=0x7f486c19a210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:32:26.823 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:26.761+0000 7f4863e00640 1 --2- 172.21.15.83:0/1323017529 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f486c1016f0 0x7f486c19a210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:32:26.823 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:26.761+0000 7f4863e00640 1 -- 172.21.15.83:0/1323017529 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f486c1076f0 msgr2=0x7f486c19eae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:32:26.823 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:26.761+0000 7f4863e00640 1 --2- 172.21.15.83:0/1323017529 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f486c1076f0 0x7f486c19eae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:32:26.824 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:26.761+0000 7f4863e00640 1 -- 172.21.15.83:0/1323017529 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f486c19f1c0 con 0x7f486c102090 2024-08-19T01:32:26.824 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:26.761+0000 7f486a800640 1 --2- 172.21.15.83:0/1323017529 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f486c1076f0 0x7f486c19eae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-19T01:32:26.824 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:26.761+0000 7f4863e00640 1 --2- 172.21.15.83:0/1323017529 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f486c102090 0x7f486c19a750 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f485400d770 tx=0x7f485400dc30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:32:26.824 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:26.762+0000 7f4862a00640 1 -- 172.21.15.83:0/1323017529 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4854017070 con 0x7f486c102090 2024-08-19T01:32:26.824 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:26.762+0000 7f4862a00640 1 -- 172.21.15.83:0/1323017529 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4854005500 con 0x7f486c102090 2024-08-19T01:32:26.824 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:26.762+0000 7f4862a00640 1 -- 172.21.15.83:0/1323017529 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4854004e00 con 0x7f486c102090 2024-08-19T01:32:26.824 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:26.762+0000 7f486b200640 1 -- 172.21.15.83:0/1323017529 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f486c19f4b0 con 0x7f486c102090 2024-08-19T01:32:26.824 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:26.762+0000 7f486b200640 1 -- 172.21.15.83:0/1323017529 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f486c1a6d90 con 0x7f486c102090 2024-08-19T01:32:26.824 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:26.763+0000 7f486b200640 1 -- 172.21.15.83:0/1323017529 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f486c101b50 con 0x7f486c102090 2024-08-19T01:32:26.824 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:26.765+0000 7f4862a00640 1 -- 172.21.15.83:0/1323017529 <== mon.2 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f4854004fa0 con 0x7f486c102090 2024-08-19T01:32:26.824 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:26.766+0000 7f4862a00640 1 --2- 172.21.15.83:0/1323017529 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f48500a2b80 0x7f48500a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:32:26.824 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:26.766+0000 7f4862a00640 1 -- 172.21.15.83:0/1323017529 <== mon.2 v2:172.21.15.117:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 7127+0+0 (secure 0 0 0) 0x7f48540ce610 con 0x7f486c102090 2024-08-19T01:32:26.824 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:26.766+0000 7f4869e00640 1 --2- 172.21.15.83:0/1323017529 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f48500a2b80 0x7f48500a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:32:26.825 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:26.767+0000 7f4869e00640 1 --2- 172.21.15.83:0/1323017529 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f48500a2b80 0x7f48500a5040 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f485c035040 tx=0x7f485c0056d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:32:26.825 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:26.770+0000 7f4862a00640 1 -- 172.21.15.83:0/1323017529 <== mon.2 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7f485409b320 con 0x7f486c102090 2024-08-19T01:32:26.999 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:26.997+0000 7f486b200640 1 -- 172.21.15.83:0/1323017529 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f486c1a6ff0 con 0x7f486c102090 2024-08-19T01:32:27.000 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:26.998+0000 7f4862a00640 1 -- 172.21.15.83:0/1323017529 <== mon.2 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 11 v11) ==== 94+0+8878 (secure 0 0 0) 0x7f485409f620 con 0x7f486c102090 2024-08-19T01:32:27.000 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-08-19T01:32:27.000 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":11,"btime":"2024-08-19T01:32:24:407229+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14421,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6835/988556067","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":988556067},{"type":"v1","addr":"172.21.15.83:6835","nonce":988556067}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14433,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6837/869702632","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6836","nonce":869702632},{"type":"v1","addr":"172.21.15.83:6837","nonce":869702632}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24430,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6833/2505076046","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6832","nonce":2505076046},{"type":"v1","addr":"172.21.15.167:6833","nonce":2505076046}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24449,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6833/3557520478","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":3557520478},{"type":"v1","addr":"172.21.15.117:6833","nonce":3557520478}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24466,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6835/3482045966","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":3482045966},{"type":"v1","addr":"172.21.15.167:6835","nonce":3482045966}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24478,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6837/1762130408","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6836","nonce":1762130408},{"type":"v1","addr":"172.21.15.167:6837","nonce":1762130408}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24485,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6835/1024213994","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":1024213994},{"type":"v1","addr":"172.21.15.117:6835","nonce":1024213994}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24496,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6839/3507387682","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6838","nonce":3507387682},{"type":"v1","addr":"172.21.15.167:6839","nonce":3507387682}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":24509,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6837/3185224684","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":3185224684},{"type":"v1","addr":"172.21.15.117:6837","nonce":3185224684}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24527,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6839/3239261845","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6838","nonce":3239261845},{"type":"v1","addr":"172.21.15.83:6839","nonce":3239261845}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11}],"filesystems":[]} 2024-08-19T01:32:27.000 INFO:teuthology.orchestra.run.smithi083.stderr:dumped fsmap epoch 11 2024-08-19T01:32:27.002 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:27.000+0000 7f486b200640 1 -- 172.21.15.83:0/1323017529 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f48500a2b80 msgr2=0x7f48500a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:32:27.002 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:27.000+0000 7f486b200640 1 --2- 172.21.15.83:0/1323017529 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f48500a2b80 0x7f48500a5040 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f485c035040 tx=0x7f485c0056d0 comp rx=0 tx=0).stop 2024-08-19T01:32:27.002 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:27.000+0000 7f486b200640 1 -- 172.21.15.83:0/1323017529 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f486c102090 msgr2=0x7f486c19a750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:32:27.002 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:27.000+0000 7f486b200640 1 --2- 172.21.15.83:0/1323017529 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f486c102090 0x7f486c19a750 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f485400d770 tx=0x7f485400dc30 comp rx=0 tx=0).stop 2024-08-19T01:32:27.003 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:27.001+0000 7f486b200640 1 -- 172.21.15.83:0/1323017529 shutdown_connections 2024-08-19T01:32:27.003 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:27.001+0000 7f486b200640 1 --2- 172.21.15.83:0/1323017529 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f48500a2b80 0x7f48500a5040 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:32:27.003 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:27.001+0000 7f486b200640 1 --2- 172.21.15.83:0/1323017529 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f486c1076f0 0x7f486c19eae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:32:27.003 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:27.001+0000 7f486b200640 1 --2- 172.21.15.83:0/1323017529 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f486c102090 0x7f486c19a750 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:32:27.003 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:27.001+0000 7f486b200640 1 --2- 172.21.15.83:0/1323017529 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f486c1016f0 0x7f486c19a210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:32:27.003 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:27.001+0000 7f486b200640 1 -- 172.21.15.83:0/1323017529 >> 172.21.15.83:0/1323017529 conn(0x7f486c0fd470 msgr2=0x7f486c10cd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:32:27.003 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:27.001+0000 7f486b200640 1 -- 172.21.15.83:0/1323017529 shutdown_connections 2024-08-19T01:32:27.003 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:27.001+0000 7f486b200640 1 -- 172.21.15.83:0/1323017529 wait complete. 2024-08-19T01:32:27.293 INFO:journalctl@ceph.mds.a.smithi083.stdout:Aug 19 01:32:27 smithi083 podman[58036]: 2024-08-19 01:32:27.147077463 +0000 UTC m=+0.172308933 container create 16051d2af2ea3f9f2d320e2248da4cf2fb41acbbdcef5355fc8a263984ad9120 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c9dd28080a631be25c77c36c63b17a3ef340ffb2e21c87d4d7b56b6398436d9, name=ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-mds-a, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, ceph=True, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-de09dd0, org.label-schema.schema-version=1.0, io.buildah.version=1.36.0, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=) 2024-08-19T01:32:27.702 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:32:27 smithi117 ceph-mon[22016]: pgmap v260: 1 pgs: 1 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-19T01:32:27.702 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:32:27 smithi117 ceph-mon[22016]: from='client.? 172.21.15.83:0/1323017529' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-19T01:32:27.716 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:32:27 smithi167 ceph-mon[21875]: pgmap v260: 1 pgs: 1 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-19T01:32:27.716 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:32:27 smithi167 ceph-mon[21875]: from='client.? 172.21.15.83:0/1323017529' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-19T01:32:27.905 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:32:27 smithi083 ceph-mon[17120]: pgmap v260: 1 pgs: 1 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-19T01:32:27.905 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:32:27 smithi083 ceph-mon[17120]: from='client.? 172.21.15.83:0/1323017529' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-19T01:32:27.905 INFO:journalctl@ceph.mds.a.smithi083.stdout:Aug 19 01:32:27 smithi083 podman[58036]: 2024-08-19 01:32:27.420935983 +0000 UTC m=+0.446167455 container init 16051d2af2ea3f9f2d320e2248da4cf2fb41acbbdcef5355fc8a263984ad9120 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c9dd28080a631be25c77c36c63b17a3ef340ffb2e21c87d4d7b56b6398436d9, name=ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-mds-a, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_CLEAN=True, maintainer=Guillaume Abrioux , RELEASE=squid-de09dd0, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.36.0, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0) 2024-08-19T01:32:27.905 INFO:journalctl@ceph.mds.a.smithi083.stdout:Aug 19 01:32:27 smithi083 podman[58036]: 2024-08-19 01:32:27.42701346 +0000 UTC m=+0.452244931 container start 16051d2af2ea3f9f2d320e2248da4cf2fb41acbbdcef5355fc8a263984ad9120 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c9dd28080a631be25c77c36c63b17a3ef340ffb2e21c87d4d7b56b6398436d9, name=ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-mds-a, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, GIT_CLEAN=True, RELEASE=squid-de09dd0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, io.buildah.version=1.36.0, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-08-19T01:32:27.905 INFO:journalctl@ceph.mds.a.smithi083.stdout:Aug 19 01:32:27 smithi083 ceph-mds[58067]: -- 172.21.15.83:0/2894679254 <== mon.0 v2:172.21.15.83:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x555acbe75c20 con 0x555acbe53400 2024-08-19T01:32:27.906 INFO:journalctl@ceph.mds.a.smithi083.stdout:Aug 19 01:32:27 smithi083 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-mds-a[58063]: starting mds.a at 2024-08-19T01:32:27.906 INFO:journalctl@ceph.mds.a.smithi083.stdout:Aug 19 01:32:27 smithi083 bash[58036]: 16051d2af2ea3f9f2d320e2248da4cf2fb41acbbdcef5355fc8a263984ad9120 2024-08-19T01:32:27.906 INFO:journalctl@ceph.mds.a.smithi083.stdout:Aug 19 01:32:27 smithi083 systemd[1]: Started Ceph mds.a for bad8780e-5dc9-11ef-bccf-c7b262605968. 2024-08-19T01:32:28.905 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:32:28 smithi083 ceph-mon[17120]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' 2024-08-19T01:32:28.905 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:32:28 smithi083 ceph-mon[17120]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' 2024-08-19T01:32:28.905 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:32:28 smithi083 ceph-mon[17120]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' 2024-08-19T01:32:28.905 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:32:28 smithi083 ceph-mon[17120]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-08-19T01:32:28.905 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:32:28 smithi083 ceph-mon[17120]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-08-19T01:32:28.905 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:32:28 smithi083 ceph-mon[17120]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-19T01:32:28.905 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:32:28 smithi083 ceph-mon[17120]: Deploying daemon mds.b on smithi117 2024-08-19T01:32:28.905 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:32:28 smithi083 ceph-mon[17120]: pgmap v261: 1 pgs: 1 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-19T01:32:28.905 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:32:28 smithi083 ceph-mon[17120]: mds.? [v2:172.21.15.83:6840/1795975999,v1:172.21.15.83:6841/1795975999] up:boot 2024-08-19T01:32:28.905 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:32:28 smithi083 ceph-mon[17120]: fsmap 11 up:standby 2024-08-19T01:32:28.905 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:32:28 smithi083 ceph-mon[17120]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-08-19T01:32:28.990 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:32:28 smithi167 ceph-mon[21875]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' 2024-08-19T01:32:28.991 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:32:28 smithi167 ceph-mon[21875]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' 2024-08-19T01:32:28.991 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:32:28 smithi167 ceph-mon[21875]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' 2024-08-19T01:32:28.991 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:32:28 smithi167 ceph-mon[21875]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-08-19T01:32:28.991 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:32:28 smithi167 ceph-mon[21875]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-08-19T01:32:28.991 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:32:28 smithi167 ceph-mon[21875]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-19T01:32:28.991 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:32:28 smithi167 ceph-mon[21875]: Deploying daemon mds.b on smithi117 2024-08-19T01:32:28.991 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:32:28 smithi167 ceph-mon[21875]: pgmap v261: 1 pgs: 1 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-19T01:32:28.991 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:32:28 smithi167 ceph-mon[21875]: mds.? [v2:172.21.15.83:6840/1795975999,v1:172.21.15.83:6841/1795975999] up:boot 2024-08-19T01:32:28.991 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:32:28 smithi167 ceph-mon[21875]: fsmap 11 up:standby 2024-08-19T01:32:28.991 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:32:28 smithi167 ceph-mon[21875]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-08-19T01:32:29.082 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:32:28 smithi117 ceph-mon[22016]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' 2024-08-19T01:32:29.082 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:32:28 smithi117 ceph-mon[22016]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' 2024-08-19T01:32:29.082 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:32:28 smithi117 ceph-mon[22016]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' 2024-08-19T01:32:29.082 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:32:28 smithi117 ceph-mon[22016]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-08-19T01:32:29.082 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:32:28 smithi117 ceph-mon[22016]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-08-19T01:32:29.082 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:32:28 smithi117 ceph-mon[22016]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-19T01:32:29.082 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:32:28 smithi117 ceph-mon[22016]: Deploying daemon mds.b on smithi117 2024-08-19T01:32:29.082 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:32:28 smithi117 ceph-mon[22016]: pgmap v261: 1 pgs: 1 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-19T01:32:29.082 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:32:28 smithi117 ceph-mon[22016]: mds.? [v2:172.21.15.83:6840/1795975999,v1:172.21.15.83:6841/1795975999] up:boot 2024-08-19T01:32:29.082 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:32:28 smithi117 ceph-mon[22016]: fsmap 11 up:standby 2024-08-19T01:32:29.083 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:32:28 smithi117 ceph-mon[22016]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-08-19T01:32:29.703 INFO:journalctl@ceph.mds.b.smithi117.stdout:Aug 19 01:32:29 smithi117 systemd[1]: Starting Ceph mds.b for bad8780e-5dc9-11ef-bccf-c7b262605968... 2024-08-19T01:32:30.153 INFO:journalctl@ceph.mds.b.smithi117.stdout:Aug 19 01:32:29 smithi117 podman[51890]: 2024-08-19 01:32:29.729389707 +0000 UTC m=+0.017143593 image pull ea7a5189f30bad3aca9524788e2965d2368d7ce786df09d4edcd8c3edf9cefba quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c9dd28080a631be25c77c36c63b17a3ef340ffb2e21c87d4d7b56b6398436d9 2024-08-19T01:32:30.153 INFO:journalctl@ceph.mds.b.smithi117.stdout:Aug 19 01:32:29 smithi117 podman[51890]: 2024-08-19 01:32:29.841624559 +0000 UTC m=+0.129378425 container create 6c1e5cbab019b0b8c63b841b6f19872a440fdb4a45b9f93dbc7371afe2a6e47c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c9dd28080a631be25c77c36c63b17a3ef340ffb2e21c87d4d7b56b6398436d9, name=ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-mds-b, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , RELEASE=squid-de09dd0, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, io.buildah.version=1.36.0) 2024-08-19T01:32:30.153 INFO:journalctl@ceph.mds.b.smithi117.stdout:Aug 19 01:32:30 smithi117 podman[51890]: 2024-08-19 01:32:30.043944239 +0000 UTC m=+0.331698114 container init 6c1e5cbab019b0b8c63b841b6f19872a440fdb4a45b9f93dbc7371afe2a6e47c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c9dd28080a631be25c77c36c63b17a3ef340ffb2e21c87d4d7b56b6398436d9, name=ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-mds-b, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, RELEASE=squid-de09dd0, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.36.0, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux ) 2024-08-19T01:32:30.153 INFO:journalctl@ceph.mds.b.smithi117.stdout:Aug 19 01:32:30 smithi117 podman[51890]: 2024-08-19 01:32:30.046739409 +0000 UTC m=+0.334493277 container start 6c1e5cbab019b0b8c63b841b6f19872a440fdb4a45b9f93dbc7371afe2a6e47c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c9dd28080a631be25c77c36c63b17a3ef340ffb2e21c87d4d7b56b6398436d9, name=ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-mds-b, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, io.buildah.version=1.36.0, RELEASE=squid-de09dd0, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731) 2024-08-19T01:32:30.153 INFO:journalctl@ceph.mds.b.smithi117.stdout:Aug 19 01:32:30 smithi117 ceph-mds[51909]: -- 172.21.15.117:0/1393648120 <== mon.2 v2:172.21.15.117:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55b5941cd860 con 0x55b5941aa800 2024-08-19T01:32:30.153 INFO:journalctl@ceph.mds.b.smithi117.stdout:Aug 19 01:32:30 smithi117 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-mds-b[51905]: starting mds.b at 2024-08-19T01:32:30.153 INFO:journalctl@ceph.mds.b.smithi117.stdout:Aug 19 01:32:30 smithi117 bash[51890]: 6c1e5cbab019b0b8c63b841b6f19872a440fdb4a45b9f93dbc7371afe2a6e47c 2024-08-19T01:32:30.452 INFO:journalctl@ceph.mds.b.smithi117.stdout:Aug 19 01:32:30 smithi117 systemd[1]: Started Ceph mds.b for bad8780e-5dc9-11ef-bccf-c7b262605968. 2024-08-19T01:32:30.536 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833 shell --fsid bad8780e-5dc9-11ef-bccf-c7b262605968 -- ceph fs dump --format=json 2024-08-19T01:32:30.930 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/bad8780e-5dc9-11ef-bccf-c7b262605968/mon.a/config 2024-08-19T01:32:31.216 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:32:31 smithi167 ceph-mon[21875]: mds.? [v2:172.21.15.117:6838/1967595704,v1:172.21.15.117:6839/1967595704] up:boot 2024-08-19T01:32:31.216 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:32:31 smithi167 ceph-mon[21875]: fsmap 12 up:standby 2024-08-19T01:32:31.217 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:32:31 smithi167 ceph-mon[21875]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-08-19T01:32:31.217 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:32:31 smithi167 ceph-mon[21875]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' 2024-08-19T01:32:31.217 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:32:31 smithi167 ceph-mon[21875]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' 2024-08-19T01:32:31.217 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:32:31 smithi167 ceph-mon[21875]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' 2024-08-19T01:32:31.217 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:32:31 smithi167 ceph-mon[21875]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' 2024-08-19T01:32:31.217 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:32:31 smithi167 ceph-mon[21875]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' 2024-08-19T01:32:31.217 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:32:31 smithi167 ceph-mon[21875]: pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-19T01:32:31.217 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:32:31 smithi167 ceph-mon[21875]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-19T01:32:31.405 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:32:31 smithi083 ceph-mon[17120]: mds.? [v2:172.21.15.117:6838/1967595704,v1:172.21.15.117:6839/1967595704] up:boot 2024-08-19T01:32:31.405 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:32:31 smithi083 ceph-mon[17120]: fsmap 12 up:standby 2024-08-19T01:32:31.405 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:32:31 smithi083 ceph-mon[17120]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-08-19T01:32:31.405 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:32:31 smithi083 ceph-mon[17120]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' 2024-08-19T01:32:31.405 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:32:31 smithi083 ceph-mon[17120]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' 2024-08-19T01:32:31.405 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:32:31 smithi083 ceph-mon[17120]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' 2024-08-19T01:32:31.405 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:32:31 smithi083 ceph-mon[17120]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' 2024-08-19T01:32:31.405 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:32:31 smithi083 ceph-mon[17120]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' 2024-08-19T01:32:31.406 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:32:31 smithi083 ceph-mon[17120]: pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-19T01:32:31.406 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:32:31 smithi083 ceph-mon[17120]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-19T01:32:31.440 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:32:31 smithi117 ceph-mon[22016]: mds.? [v2:172.21.15.117:6838/1967595704,v1:172.21.15.117:6839/1967595704] up:boot 2024-08-19T01:32:31.441 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:32:31 smithi117 ceph-mon[22016]: fsmap 12 up:standby 2024-08-19T01:32:31.441 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:32:31 smithi117 ceph-mon[22016]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-08-19T01:32:31.441 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:32:31 smithi117 ceph-mon[22016]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' 2024-08-19T01:32:31.441 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:32:31 smithi117 ceph-mon[22016]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' 2024-08-19T01:32:31.441 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:32:31 smithi117 ceph-mon[22016]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' 2024-08-19T01:32:31.441 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:32:31 smithi117 ceph-mon[22016]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' 2024-08-19T01:32:31.441 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:32:31 smithi117 ceph-mon[22016]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' 2024-08-19T01:32:31.441 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:32:31 smithi117 ceph-mon[22016]: pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-19T01:32:31.441 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:32:31 smithi117 ceph-mon[22016]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-19T01:32:32.168 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:32.165+0000 7f8a4a400640 1 -- 172.21.15.83:0/2990688585 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8a44074c10 msgr2=0x7f8a4410da50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:32:32.168 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:32.165+0000 7f8a4a400640 1 --2- 172.21.15.83:0/2990688585 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8a44074c10 0x7f8a4410da50 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f8a30009900 tx=0x7f8a3002a480 comp rx=0 tx=0).stop 2024-08-19T01:32:32.168 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:32.165+0000 7f8a4a400640 1 -- 172.21.15.83:0/2990688585 shutdown_connections 2024-08-19T01:32:32.168 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:32.165+0000 7f8a4a400640 1 --2- 172.21.15.83:0/2990688585 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8a4410df90 0x7f8a44110380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:32:32.168 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:32.165+0000 7f8a4a400640 1 --2- 172.21.15.83:0/2990688585 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8a44074c10 0x7f8a4410da50 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:32:32.168 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:32.165+0000 7f8a4a400640 1 --2- 172.21.15.83:0/2990688585 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8a440742f0 0x7f8a440746d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:32:32.168 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:32.165+0000 7f8a4a400640 1 -- 172.21.15.83:0/2990688585 >> 172.21.15.83:0/2990688585 conn(0x7f8a440fd470 msgr2=0x7f8a440ff890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:32:32.168 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:32.165+0000 7f8a4a400640 1 -- 172.21.15.83:0/2990688585 shutdown_connections 2024-08-19T01:32:32.168 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:32.165+0000 7f8a4a400640 1 -- 172.21.15.83:0/2990688585 wait complete. 2024-08-19T01:32:32.168 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:32.166+0000 7f8a4a400640 1 Processor -- start 2024-08-19T01:32:32.170 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:32.167+0000 7f8a4a400640 1 -- start start 2024-08-19T01:32:32.170 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:32.167+0000 7f8a4a400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8a440742f0 0x7f8a4419a2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:32:32.170 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:32.167+0000 7f8a4a400640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8a44074c10 0x7f8a4419a7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:32:32.170 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:32.167+0000 7f8a4a400640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8a4410df90 0x7f8a4419eb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:32:32.170 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:32.167+0000 7f8a4a400640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f8a44113ce0 con 0x7f8a440742f0 2024-08-19T01:32:32.170 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:32.167+0000 7f8a4a400640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f8a44113b60 con 0x7f8a44074c10 2024-08-19T01:32:32.170 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:32.167+0000 7f8a4a400640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f8a44113e60 con 0x7f8a4410df90 2024-08-19T01:32:32.170 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:32.167+0000 7f8a49000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8a440742f0 0x7f8a4419a2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:32:32.170 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:32.167+0000 7f8a43e00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8a44074c10 0x7f8a4419a7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:32:32.170 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:32.167+0000 7f8a43e00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8a44074c10 0x7f8a4419a7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.83:55064/0 (socket says 172.21.15.83:55064) 2024-08-19T01:32:32.170 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:32.167+0000 7f8a43e00640 1 -- 172.21.15.83:0/2282532573 learned_addr learned my addr 172.21.15.83:0/2282532573 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-08-19T01:32:32.170 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:32.168+0000 7f8a49a00640 1 --2- 172.21.15.83:0/2282532573 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8a4410df90 0x7f8a4419eb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:32:32.171 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:32.168+0000 7f8a43e00640 1 -- 172.21.15.83:0/2282532573 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8a4410df90 msgr2=0x7f8a4419eb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:32:32.171 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:32.168+0000 7f8a43e00640 1 --2- 172.21.15.83:0/2282532573 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8a4410df90 0x7f8a4419eb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:32:32.171 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:32.168+0000 7f8a43e00640 1 -- 172.21.15.83:0/2282532573 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8a440742f0 msgr2=0x7f8a4419a2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:32:32.171 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:32.168+0000 7f8a43e00640 1 --2- 172.21.15.83:0/2282532573 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8a440742f0 0x7f8a4419a2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:32:32.171 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:32.168+0000 7f8a43e00640 1 -- 172.21.15.83:0/2282532573 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8a4419f250 con 0x7f8a44074c10 2024-08-19T01:32:32.171 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:32.168+0000 7f8a49000640 1 --2- 172.21.15.83:0/2282532573 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8a440742f0 0x7f8a4419a2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-19T01:32:32.172 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:32.168+0000 7f8a43e00640 1 --2- 172.21.15.83:0/2282532573 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8a44074c10 0x7f8a4419a7e0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f8a30006010 tx=0x7f8a30004040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:32:32.172 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:32.169+0000 7f8a41600640 1 -- 172.21.15.83:0/2282532573 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a3003f030 con 0x7f8a44074c10 2024-08-19T01:32:32.172 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:32.169+0000 7f8a41600640 1 -- 172.21.15.83:0/2282532573 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8a30004440 con 0x7f8a44074c10 2024-08-19T01:32:32.172 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:32.169+0000 7f8a4a400640 1 -- 172.21.15.83:0/2282532573 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8a4419f4e0 con 0x7f8a44074c10 2024-08-19T01:32:32.172 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:32.169+0000 7f8a41600640 1 -- 172.21.15.83:0/2282532573 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a30035070 con 0x7f8a44074c10 2024-08-19T01:32:32.172 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:32.169+0000 7f8a4a400640 1 -- 172.21.15.83:0/2282532573 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8a441a6e20 con 0x7f8a44074c10 2024-08-19T01:32:32.172 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:32.170+0000 7f8a4a400640 1 -- 172.21.15.83:0/2282532573 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8a440746d0 con 0x7f8a44074c10 2024-08-19T01:32:32.176 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:32.174+0000 7f8a41600640 1 -- 172.21.15.83:0/2282532573 <== mon.2 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f8a30042030 con 0x7f8a44074c10 2024-08-19T01:32:32.178 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:32.176+0000 7f8a41600640 1 --2- 172.21.15.83:0/2282532573 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f8a2c0a2c00 0x7f8a2c0a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:32:32.178 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:32.177+0000 7f8a41600640 1 -- 172.21.15.83:0/2282532573 <== mon.2 v2:172.21.15.117:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 7127+0+0 (secure 0 0 0) 0x7f8a300ef6a0 con 0x7f8a44074c10 2024-08-19T01:32:32.178 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:32.177+0000 7f8a49000640 1 --2- 172.21.15.83:0/2282532573 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f8a2c0a2c00 0x7f8a2c0a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:32:32.179 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:32.177+0000 7f8a41600640 1 -- 172.21.15.83:0/2282532573 <== mon.2 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7f8a30049030 con 0x7f8a44074c10 2024-08-19T01:32:32.179 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:32.178+0000 7f8a49000640 1 --2- 172.21.15.83:0/2282532573 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f8a2c0a2c00 0x7f8a2c0a50c0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f8a38009fc0 tx=0x7f8a38005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:32:32.405 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:32.403+0000 7f8a4a400640 1 -- 172.21.15.83:0/2282532573 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f8a4419b600 con 0x7f8a44074c10 2024-08-19T01:32:32.406 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-08-19T01:32:32.407 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":13,"btime":"2024-08-19T01:32:30:125415+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14421,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6835/988556067","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":988556067},{"type":"v1","addr":"172.21.15.83:6835","nonce":988556067}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14433,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6837/869702632","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6836","nonce":869702632},{"type":"v1","addr":"172.21.15.83:6837","nonce":869702632}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14475,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6841/1795975999","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6840","nonce":1795975999},{"type":"v1","addr":"172.21.15.83:6841","nonce":1795975999}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24430,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6833/2505076046","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6832","nonce":2505076046},{"type":"v1","addr":"172.21.15.167:6833","nonce":2505076046}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24449,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6833/3557520478","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":3557520478},{"type":"v1","addr":"172.21.15.117:6833","nonce":3557520478}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24466,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6835/3482045966","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":3482045966},{"type":"v1","addr":"172.21.15.167:6835","nonce":3482045966}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24478,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6837/1762130408","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6836","nonce":1762130408},{"type":"v1","addr":"172.21.15.167:6837","nonce":1762130408}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24485,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6835/1024213994","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":1024213994},{"type":"v1","addr":"172.21.15.117:6835","nonce":1024213994}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24496,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6839/3507387682","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6838","nonce":3507387682},{"type":"v1","addr":"172.21.15.167:6839","nonce":3507387682}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":24509,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6837/3185224684","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":3185224684},{"type":"v1","addr":"172.21.15.117:6837","nonce":3185224684}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24527,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6839/3239261845","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6838","nonce":3239261845},{"type":"v1","addr":"172.21.15.83:6839","nonce":3239261845}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":24545,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6839/1967595704","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6838","nonce":1967595704},{"type":"v1","addr":"172.21.15.117:6839","nonce":1967595704}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13}],"filesystems":[]} 2024-08-19T01:32:32.407 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:32.404+0000 7f8a41600640 1 -- 172.21.15.83:0/2282532573 <== mon.2 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 13 v13) ==== 94+0+10533 (secure 0 0 0) 0x7f8a4419b600 con 0x7f8a44074c10 2024-08-19T01:32:32.407 INFO:teuthology.orchestra.run.smithi083.stderr:dumped fsmap epoch 13 2024-08-19T01:32:32.409 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:32.407+0000 7f8a4a400640 1 -- 172.21.15.83:0/2282532573 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f8a2c0a2c00 msgr2=0x7f8a2c0a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:32:32.410 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:32.407+0000 7f8a4a400640 1 --2- 172.21.15.83:0/2282532573 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f8a2c0a2c00 0x7f8a2c0a50c0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f8a38009fc0 tx=0x7f8a38005f90 comp rx=0 tx=0).stop 2024-08-19T01:32:32.410 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:32.407+0000 7f8a4a400640 1 -- 172.21.15.83:0/2282532573 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8a44074c10 msgr2=0x7f8a4419a7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:32:32.410 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:32.407+0000 7f8a4a400640 1 --2- 172.21.15.83:0/2282532573 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8a44074c10 0x7f8a4419a7e0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f8a30006010 tx=0x7f8a30004040 comp rx=0 tx=0).stop 2024-08-19T01:32:32.410 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:32.407+0000 7f8a4a400640 1 -- 172.21.15.83:0/2282532573 shutdown_connections 2024-08-19T01:32:32.410 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:32.407+0000 7f8a4a400640 1 --2- 172.21.15.83:0/2282532573 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f8a2c0a2c00 0x7f8a2c0a50c0 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:32:32.410 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:32.408+0000 7f8a4a400640 1 --2- 172.21.15.83:0/2282532573 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8a4410df90 0x7f8a4419eb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:32:32.410 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:32.408+0000 7f8a4a400640 1 --2- 172.21.15.83:0/2282532573 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8a44074c10 0x7f8a4419a7e0 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:32:32.410 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:32.408+0000 7f8a4a400640 1 --2- 172.21.15.83:0/2282532573 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8a440742f0 0x7f8a4419a2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:32:32.410 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:32.408+0000 7f8a4a400640 1 -- 172.21.15.83:0/2282532573 >> 172.21.15.83:0/2282532573 conn(0x7f8a440fd470 msgr2=0x7f8a440fe1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:32:32.411 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:32.408+0000 7f8a4a400640 1 -- 172.21.15.83:0/2282532573 shutdown_connections 2024-08-19T01:32:32.411 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:32.408+0000 7f8a4a400640 1 -- 172.21.15.83:0/2282532573 wait complete. 2024-08-19T01:32:32.952 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:32:32 smithi117 ceph-mon[22016]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' 2024-08-19T01:32:32.952 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:32:32 smithi117 ceph-mon[22016]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' 2024-08-19T01:32:32.952 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:32:32 smithi117 ceph-mon[22016]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' 2024-08-19T01:32:32.953 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:32:32 smithi117 ceph-mon[22016]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' 2024-08-19T01:32:32.953 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:32:32 smithi117 ceph-mon[22016]: pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-19T01:32:32.953 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:32:32 smithi117 ceph-mon[22016]: from='client.? 172.21.15.83:0/2282532573' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-19T01:32:33.155 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:32:32 smithi083 ceph-mon[17120]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' 2024-08-19T01:32:33.155 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:32:32 smithi083 ceph-mon[17120]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' 2024-08-19T01:32:33.155 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:32:32 smithi083 ceph-mon[17120]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' 2024-08-19T01:32:33.155 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:32:32 smithi083 ceph-mon[17120]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' 2024-08-19T01:32:33.155 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:32:32 smithi083 ceph-mon[17120]: pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-19T01:32:33.155 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:32:32 smithi083 ceph-mon[17120]: from='client.? 172.21.15.83:0/2282532573' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-19T01:32:33.216 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:32:32 smithi167 ceph-mon[21875]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' 2024-08-19T01:32:33.216 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:32:32 smithi167 ceph-mon[21875]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' 2024-08-19T01:32:33.216 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:32:32 smithi167 ceph-mon[21875]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' 2024-08-19T01:32:33.216 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:32:32 smithi167 ceph-mon[21875]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' 2024-08-19T01:32:33.216 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:32:32 smithi167 ceph-mon[21875]: pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-19T01:32:33.216 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:32:32 smithi167 ceph-mon[21875]: from='client.? 172.21.15.83:0/2282532573' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-19T01:32:33.240 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833 shell --fsid bad8780e-5dc9-11ef-bccf-c7b262605968 -- ceph --format=json-pretty osd lspools 2024-08-19T01:32:33.633 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/bad8780e-5dc9-11ef-bccf-c7b262605968/mon.a/config 2024-08-19T01:32:34.235 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:34.233+0000 7f1cb9800640 1 -- 172.21.15.83:0/2805572958 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1cb41016f0 msgr2=0x7f1cb4101b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:32:34.236 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:34.233+0000 7f1cb9800640 1 --2- 172.21.15.83:0/2805572958 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1cb41016f0 0x7f1cb4101b50 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f1ca0009900 tx=0x7f1ca002a3f0 comp rx=0 tx=0).stop 2024-08-19T01:32:34.236 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:34.233+0000 7f1cb9800640 1 -- 172.21.15.83:0/2805572958 shutdown_connections 2024-08-19T01:32:34.236 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:34.233+0000 7f1cb9800640 1 --2- 172.21.15.83:0/2805572958 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1cb4102090 0x7f1cb410e5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:32:34.236 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:34.233+0000 7f1cb9800640 1 --2- 172.21.15.83:0/2805572958 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1cb41016f0 0x7f1cb4101b50 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:32:34.236 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:34.233+0000 7f1cb9800640 1 --2- 172.21.15.83:0/2805572958 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1cb41076f0 0x7f1cb4107ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:32:34.236 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:34.233+0000 7f1cb9800640 1 -- 172.21.15.83:0/2805572958 >> 172.21.15.83:0/2805572958 conn(0x7f1cb40fd470 msgr2=0x7f1cb40ff890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:32:34.236 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:34.233+0000 7f1cb9800640 1 -- 172.21.15.83:0/2805572958 shutdown_connections 2024-08-19T01:32:34.236 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:34.233+0000 7f1cb9800640 1 -- 172.21.15.83:0/2805572958 wait complete. 2024-08-19T01:32:34.236 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:34.234+0000 7f1cb9800640 1 Processor -- start 2024-08-19T01:32:34.236 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:34.234+0000 7f1cb9800640 1 -- start start 2024-08-19T01:32:34.238 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:34.234+0000 7f1cb9800640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1cb41016f0 0x7f1cb419a1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:32:34.238 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:34.234+0000 7f1cb9800640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1cb4102090 0x7f1cb419a730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:32:34.239 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:34.234+0000 7f1cb9800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1cb41076f0 0x7f1cb419eac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:32:34.239 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:34.234+0000 7f1cb9800640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f1cb4111b00 con 0x7f1cb41076f0 2024-08-19T01:32:34.239 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:34.234+0000 7f1cb9800640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f1cb4111980 con 0x7f1cb4102090 2024-08-19T01:32:34.239 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:34.234+0000 7f1cb9800640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f1cb4111c80 con 0x7f1cb41016f0 2024-08-19T01:32:34.239 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:34.234+0000 7f1cb3400640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1cb4102090 0x7f1cb419a730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:32:34.239 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:34.235+0000 7f1cb3400640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1cb4102090 0x7f1cb419a730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.83:55078/0 (socket says 172.21.15.83:55078) 2024-08-19T01:32:34.239 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:34.235+0000 7f1cb3400640 1 -- 172.21.15.83:0/1774723108 learned_addr learned my addr 172.21.15.83:0/1774723108 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-08-19T01:32:34.239 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:34.235+0000 7f1cb3e00640 1 --2- 172.21.15.83:0/1774723108 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1cb41016f0 0x7f1cb419a1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:32:34.239 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:34.235+0000 7f1cb8e00640 1 --2- 172.21.15.83:0/1774723108 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1cb41076f0 0x7f1cb419eac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:32:34.239 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:34.235+0000 7f1cb3400640 1 -- 172.21.15.83:0/1774723108 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1cb41016f0 msgr2=0x7f1cb419a1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:32:34.239 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:34.235+0000 7f1cb3400640 1 --2- 172.21.15.83:0/1774723108 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1cb41016f0 0x7f1cb419a1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:32:34.239 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:34.235+0000 7f1cb3400640 1 -- 172.21.15.83:0/1774723108 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1cb41076f0 msgr2=0x7f1cb419eac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:32:34.239 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:34.235+0000 7f1cb3400640 1 --2- 172.21.15.83:0/1774723108 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1cb41076f0 0x7f1cb419eac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:32:34.239 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:34.235+0000 7f1cb3400640 1 -- 172.21.15.83:0/1774723108 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1cb419f1a0 con 0x7f1cb4102090 2024-08-19T01:32:34.240 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:34.235+0000 7f1cb3400640 1 --2- 172.21.15.83:0/1774723108 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1cb4102090 0x7f1cb419a730 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f1ca002a9e0 tx=0x7f1ca0003c10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:32:34.240 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:34.236+0000 7f1cb0c00640 1 -- 172.21.15.83:0/1774723108 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ca003f070 con 0x7f1cb4102090 2024-08-19T01:32:34.240 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:34.236+0000 7f1cb0c00640 1 -- 172.21.15.83:0/1774723108 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1ca0031c30 con 0x7f1cb4102090 2024-08-19T01:32:34.240 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:34.236+0000 7f1cb9800640 1 -- 172.21.15.83:0/1774723108 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1cb419f430 con 0x7f1cb4102090 2024-08-19T01:32:34.240 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:34.236+0000 7f1cb0c00640 1 -- 172.21.15.83:0/1774723108 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ca0035030 con 0x7f1cb4102090 2024-08-19T01:32:34.240 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:34.236+0000 7f1cb9800640 1 -- 172.21.15.83:0/1774723108 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1cb41a6d70 con 0x7f1cb4102090 2024-08-19T01:32:34.240 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:34.237+0000 7f1cb9800640 1 -- 172.21.15.83:0/1774723108 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1cb4101b50 con 0x7f1cb4102090 2024-08-19T01:32:34.241 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:34.240+0000 7f1cb0c00640 1 -- 172.21.15.83:0/1774723108 <== mon.2 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f1ca0042070 con 0x7f1cb4102090 2024-08-19T01:32:34.243 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:34.241+0000 7f1cb0c00640 1 --2- 172.21.15.83:0/1774723108 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f1c980a2c00 0x7f1c980a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:32:34.243 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:34.241+0000 7f1cb0c00640 1 -- 172.21.15.83:0/1774723108 <== mon.2 v2:172.21.15.117:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 7127+0+0 (secure 0 0 0) 0x7f1ca00ef1c0 con 0x7f1cb4102090 2024-08-19T01:32:34.243 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:34.241+0000 7f1cb3e00640 1 --2- 172.21.15.83:0/1774723108 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f1c980a2c00 0x7f1c980a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:32:34.243 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:34.242+0000 7f1cb3e00640 1 --2- 172.21.15.83:0/1774723108 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f1c980a2c00 0x7f1c980a50c0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f1ca4009b50 tx=0x7f1ca4005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:32:34.246 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:34.245+0000 7f1cb0c00640 1 -- 172.21.15.83:0/1774723108 <== mon.2 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7f1ca0031de0 con 0x7f1cb4102090 2024-08-19T01:32:34.440 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:34.438+0000 7f1cb9800640 1 -- 172.21.15.83:0/1774723108 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7f1cb41a6fd0 con 0x7f1cb4102090 2024-08-19T01:32:34.441 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:34.439+0000 7f1cb0c00640 1 -- 172.21.15.83:0/1774723108 <== mon.2 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v79) ==== 84+0+65 (secure 0 0 0) 0x7f1ca00bbf70 con 0x7f1cb4102090 2024-08-19T01:32:34.441 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-08-19T01:32:34.441 INFO:teuthology.orchestra.run.smithi083.stdout:[ 2024-08-19T01:32:34.441 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-08-19T01:32:34.441 INFO:teuthology.orchestra.run.smithi083.stdout: "poolnum": 1, 2024-08-19T01:32:34.441 INFO:teuthology.orchestra.run.smithi083.stdout: "poolname": ".mgr" 2024-08-19T01:32:34.441 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-08-19T01:32:34.442 INFO:teuthology.orchestra.run.smithi083.stdout:] 2024-08-19T01:32:34.444 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:34.442+0000 7f1cb9800640 1 -- 172.21.15.83:0/1774723108 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f1c980a2c00 msgr2=0x7f1c980a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:32:34.444 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:34.442+0000 7f1cb9800640 1 --2- 172.21.15.83:0/1774723108 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f1c980a2c00 0x7f1c980a50c0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f1ca4009b50 tx=0x7f1ca4005f90 comp rx=0 tx=0).stop 2024-08-19T01:32:34.444 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:34.442+0000 7f1cb9800640 1 -- 172.21.15.83:0/1774723108 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1cb4102090 msgr2=0x7f1cb419a730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:32:34.444 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:34.442+0000 7f1cb9800640 1 --2- 172.21.15.83:0/1774723108 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1cb4102090 0x7f1cb419a730 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f1ca002a9e0 tx=0x7f1ca0003c10 comp rx=0 tx=0).stop 2024-08-19T01:32:34.444 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:34.442+0000 7f1cb9800640 1 -- 172.21.15.83:0/1774723108 shutdown_connections 2024-08-19T01:32:34.444 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:34.442+0000 7f1cb9800640 1 --2- 172.21.15.83:0/1774723108 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f1c980a2c00 0x7f1c980a50c0 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:32:34.444 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:34.442+0000 7f1cb9800640 1 --2- 172.21.15.83:0/1774723108 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1cb41076f0 0x7f1cb419eac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:32:34.444 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:34.442+0000 7f1cb9800640 1 --2- 172.21.15.83:0/1774723108 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1cb4102090 0x7f1cb419a730 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:32:34.444 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:34.442+0000 7f1cb9800640 1 --2- 172.21.15.83:0/1774723108 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1cb41016f0 0x7f1cb419a1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:32:34.444 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:34.442+0000 7f1cb9800640 1 -- 172.21.15.83:0/1774723108 >> 172.21.15.83:0/1774723108 conn(0x7f1cb40fd470 msgr2=0x7f1cb410cd80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:32:34.444 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:34.442+0000 7f1cb9800640 1 -- 172.21.15.83:0/1774723108 shutdown_connections 2024-08-19T01:32:34.444 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:34.442+0000 7f1cb9800640 1 -- 172.21.15.83:0/1774723108 wait complete. 2024-08-19T01:32:34.703 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:32:34 smithi117 ceph-mon[22016]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' 2024-08-19T01:32:34.703 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:32:34 smithi117 ceph-mon[22016]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' 2024-08-19T01:32:34.703 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:32:34 smithi117 ceph-mon[22016]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-19T01:32:34.703 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:32:34 smithi117 ceph-mon[22016]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-19T01:32:34.703 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:32:34 smithi117 ceph-mon[22016]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' 2024-08-19T01:32:34.703 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:32:34 smithi117 ceph-mon[22016]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-19T01:32:34.703 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:32:34 smithi117 ceph-mon[22016]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' 2024-08-19T01:32:34.703 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:32:34 smithi117 ceph-mon[22016]: from='client.? 172.21.15.83:0/1774723108' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-08-19T01:32:34.790 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:32:34 smithi083 ceph-mon[17120]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' 2024-08-19T01:32:34.790 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:32:34 smithi083 ceph-mon[17120]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' 2024-08-19T01:32:34.790 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:32:34 smithi083 ceph-mon[17120]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-19T01:32:34.790 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:32:34 smithi083 ceph-mon[17120]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-19T01:32:34.790 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:32:34 smithi083 ceph-mon[17120]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' 2024-08-19T01:32:34.790 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:32:34 smithi083 ceph-mon[17120]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-19T01:32:34.790 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:32:34 smithi083 ceph-mon[17120]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' 2024-08-19T01:32:34.790 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:32:34 smithi083 ceph-mon[17120]: from='client.? 172.21.15.83:0/1774723108' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-08-19T01:32:34.966 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:32:34 smithi167 ceph-mon[21875]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' 2024-08-19T01:32:34.966 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:32:34 smithi167 ceph-mon[21875]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' 2024-08-19T01:32:34.966 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:32:34 smithi167 ceph-mon[21875]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-19T01:32:34.966 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:32:34 smithi167 ceph-mon[21875]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-19T01:32:34.966 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:32:34 smithi167 ceph-mon[21875]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' 2024-08-19T01:32:34.966 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:32:34 smithi167 ceph-mon[21875]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-19T01:32:34.966 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:32:34 smithi167 ceph-mon[21875]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' 2024-08-19T01:32:34.966 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:32:34 smithi167 ceph-mon[21875]: from='client.? 172.21.15.83:0/1774723108' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-08-19T01:32:35.781 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:32:35 smithi083 ceph-mon[17120]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-19T01:32:35.781 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:32:35 smithi083 ceph-mon[17120]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' 2024-08-19T01:32:35.781 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:32:35 smithi083 ceph-mon[17120]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' 2024-08-19T01:32:35.781 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:32:35 smithi083 ceph-mon[17120]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' 2024-08-19T01:32:35.781 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:32:35 smithi083 ceph-mon[17120]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' 2024-08-19T01:32:35.783 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-08-19T01:32:35.783 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833 shell --fsid bad8780e-5dc9-11ef-bccf-c7b262605968 -- ceph osd pool create cephfs_metadata --pg_num_min 64 2024-08-19T01:32:35.952 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:32:35 smithi117 ceph-mon[22016]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-19T01:32:35.952 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:32:35 smithi117 ceph-mon[22016]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' 2024-08-19T01:32:35.953 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:32:35 smithi117 ceph-mon[22016]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' 2024-08-19T01:32:35.953 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:32:35 smithi117 ceph-mon[22016]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' 2024-08-19T01:32:35.953 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:32:35 smithi117 ceph-mon[22016]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' 2024-08-19T01:32:35.966 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:32:35 smithi167 ceph-mon[21875]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-19T01:32:35.966 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:32:35 smithi167 ceph-mon[21875]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' 2024-08-19T01:32:35.966 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:32:35 smithi167 ceph-mon[21875]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' 2024-08-19T01:32:35.966 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:32:35 smithi167 ceph-mon[21875]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' 2024-08-19T01:32:35.966 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:32:35 smithi167 ceph-mon[21875]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' 2024-08-19T01:32:36.179 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/bad8780e-5dc9-11ef-bccf-c7b262605968/mon.a/config 2024-08-19T01:32:36.738 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:36.737+0000 7f8a5ea00640 1 -- 172.21.15.83:0/3312299291 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8a58103ae0 msgr2=0x7f8a58103f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:32:36.739 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:36.737+0000 7f8a5ea00640 1 --2- 172.21.15.83:0/3312299291 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8a58103ae0 0x7f8a58103f40 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f8a44009930 tx=0x7f8a4402a420 comp rx=0 tx=0).stop 2024-08-19T01:32:36.739 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:36.737+0000 7f8a5ea00640 1 -- 172.21.15.83:0/3312299291 shutdown_connections 2024-08-19T01:32:36.739 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:36.737+0000 7f8a5ea00640 1 --2- 172.21.15.83:0/3312299291 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8a58104480 0x7f8a581109b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:32:36.739 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:36.737+0000 7f8a5ea00640 1 --2- 172.21.15.83:0/3312299291 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8a58103ae0 0x7f8a58103f40 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:32:36.739 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:36.737+0000 7f8a5ea00640 1 --2- 172.21.15.83:0/3312299291 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8a58109b70 0x7f8a58109f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:32:36.739 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:36.737+0000 7f8a5ea00640 1 -- 172.21.15.83:0/3312299291 >> 172.21.15.83:0/3312299291 conn(0x7f8a580ff780 msgr2=0x7f8a58101ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:32:36.739 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:36.737+0000 7f8a5ea00640 1 -- 172.21.15.83:0/3312299291 shutdown_connections 2024-08-19T01:32:36.739 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:36.737+0000 7f8a5ea00640 1 -- 172.21.15.83:0/3312299291 wait complete. 2024-08-19T01:32:36.740 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:36.738+0000 7f8a5ea00640 1 Processor -- start 2024-08-19T01:32:36.740 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:36.738+0000 7f8a5ea00640 1 -- start start 2024-08-19T01:32:36.740 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:36.739+0000 7f8a5ea00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8a58103ae0 0x7f8a5819a3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:32:36.740 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:36.739+0000 7f8a5ea00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8a58104480 0x7f8a5819a910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:32:36.740 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:36.739+0000 7f8a5ea00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8a58109b70 0x7f8a5819eca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:32:36.740 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:36.739+0000 7f8a5ea00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f8a58113ef0 con 0x7f8a58104480 2024-08-19T01:32:36.740 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:36.739+0000 7f8a5ea00640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f8a58113d70 con 0x7f8a58109b70 2024-08-19T01:32:36.740 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:36.739+0000 7f8a5ea00640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f8a58114070 con 0x7f8a58103ae0 2024-08-19T01:32:36.740 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:36.739+0000 7f8a57e00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8a58103ae0 0x7f8a5819a3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:32:36.741 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:36.739+0000 7f8a57e00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8a58103ae0 0x7f8a5819a3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.83:52692/0 (socket says 172.21.15.83:52692) 2024-08-19T01:32:36.741 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:36.739+0000 7f8a57e00640 1 -- 172.21.15.83:0/1488631096 learned_addr learned my addr 172.21.15.83:0/1488631096 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-08-19T01:32:36.741 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:36.739+0000 7f8a57400640 1 --2- 172.21.15.83:0/1488631096 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8a58104480 0x7f8a5819a910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:32:36.741 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:36.739+0000 7f8a5ca00640 1 --2- 172.21.15.83:0/1488631096 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8a58109b70 0x7f8a5819eca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:32:36.742 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:36.740+0000 7f8a57e00640 1 -- 172.21.15.83:0/1488631096 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8a58109b70 msgr2=0x7f8a5819eca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:32:36.742 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:36.740+0000 7f8a57e00640 1 --2- 172.21.15.83:0/1488631096 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8a58109b70 0x7f8a5819eca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:32:36.742 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:36.740+0000 7f8a57e00640 1 -- 172.21.15.83:0/1488631096 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8a58104480 msgr2=0x7f8a5819a910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:32:36.742 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:36.740+0000 7f8a57e00640 1 --2- 172.21.15.83:0/1488631096 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8a58104480 0x7f8a5819a910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:32:36.742 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:36.740+0000 7f8a57e00640 1 -- 172.21.15.83:0/1488631096 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8a5819f380 con 0x7f8a58103ae0 2024-08-19T01:32:36.742 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:36.740+0000 7f8a57e00640 1 --2- 172.21.15.83:0/1488631096 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8a58103ae0 0x7f8a5819a3d0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f8a4800d950 tx=0x7f8a4800de10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:32:36.742 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:36.740+0000 7f8a54c00640 1 -- 172.21.15.83:0/1488631096 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a48017070 con 0x7f8a58103ae0 2024-08-19T01:32:36.742 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:36.740+0000 7f8a54c00640 1 -- 172.21.15.83:0/1488631096 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8a48009c30 con 0x7f8a58103ae0 2024-08-19T01:32:36.742 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:36.741+0000 7f8a5ea00640 1 -- 172.21.15.83:0/1488631096 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8a5819f670 con 0x7f8a58103ae0 2024-08-19T01:32:36.742 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:36.741+0000 7f8a54c00640 1 -- 172.21.15.83:0/1488631096 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a4800f6b0 con 0x7f8a58103ae0 2024-08-19T01:32:36.742 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:36.741+0000 7f8a5ea00640 1 -- 172.21.15.83:0/1488631096 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8a581a6f50 con 0x7f8a58103ae0 2024-08-19T01:32:36.743 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:36.742+0000 7f8a5ea00640 1 -- 172.21.15.83:0/1488631096 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8a58103f40 con 0x7f8a58103ae0 2024-08-19T01:32:36.744 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:36.743+0000 7f8a54c00640 1 -- 172.21.15.83:0/1488631096 <== mon.1 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f8a480109d0 con 0x7f8a58103ae0 2024-08-19T01:32:36.745 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:36.745+0000 7f8a54c00640 1 --2- 172.21.15.83:0/1488631096 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f8a3c0a2c00 0x7f8a3c0a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:32:36.745 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:36.745+0000 7f8a54c00640 1 -- 172.21.15.83:0/1488631096 <== mon.1 v2:172.21.15.167:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 7127+0+0 (secure 0 0 0) 0x7f8a480ce6c0 con 0x7f8a58103ae0 2024-08-19T01:32:36.746 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:36.745+0000 7f8a57400640 1 --2- 172.21.15.83:0/1488631096 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f8a3c0a2c00 0x7f8a3c0a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:32:36.746 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:36.746+0000 7f8a57400640 1 --2- 172.21.15.83:0/1488631096 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f8a3c0a2c00 0x7f8a3c0a50c0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f8a44006010 tx=0x7f8a44000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:32:36.748 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:36.747+0000 7f8a54c00640 1 -- 172.21.15.83:0/1488631096 <== mon.1 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7f8a48009de0 con 0x7f8a58103ae0 2024-08-19T01:32:36.953 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:36.952+0000 7f8a5ea00640 1 -- 172.21.15.83:0/1488631096 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7f8a5810c0a0 con 0x7f8a58103ae0 2024-08-19T01:32:37.897 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:37.896+0000 7f8a54c00640 1 -- 172.21.15.83:0/1488631096 <== mon.1 v2:172.21.15.167:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v80) ==== 138+0+0 (secure 0 0 0) 0x7f8a4809b450 con 0x7f8a58103ae0 2024-08-19T01:32:37.897 INFO:teuthology.orchestra.run.smithi083.stderr:pool 'cephfs_metadata' created 2024-08-19T01:32:37.897 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:32:37 smithi083 ceph-mon[17120]: pgmap v265: 1 pgs: 1 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-19T01:32:37.897 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:32:37 smithi083 ceph-mon[17120]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' 2024-08-19T01:32:37.898 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:32:37 smithi083 ceph-mon[17120]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' 2024-08-19T01:32:37.898 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:32:37 smithi083 ceph-mon[17120]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-19T01:32:37.898 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:32:37 smithi083 ceph-mon[17120]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-19T01:32:37.898 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:32:37 smithi083 ceph-mon[17120]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' 2024-08-19T01:32:37.898 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:32:37 smithi083 ceph-mon[17120]: from='client.? 172.21.15.83:0/1488631096' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-08-19T01:32:37.898 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:32:37 smithi083 ceph-mon[17120]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-08-19T01:32:37.902 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:37.900+0000 7f8a5ea00640 1 -- 172.21.15.83:0/1488631096 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f8a3c0a2c00 msgr2=0x7f8a3c0a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:32:37.902 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:37.900+0000 7f8a5ea00640 1 --2- 172.21.15.83:0/1488631096 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f8a3c0a2c00 0x7f8a3c0a50c0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f8a44006010 tx=0x7f8a44000bc0 comp rx=0 tx=0).stop 2024-08-19T01:32:37.902 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:37.900+0000 7f8a5ea00640 1 -- 172.21.15.83:0/1488631096 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8a58103ae0 msgr2=0x7f8a5819a3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:32:37.902 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:37.900+0000 7f8a5ea00640 1 --2- 172.21.15.83:0/1488631096 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8a58103ae0 0x7f8a5819a3d0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f8a4800d950 tx=0x7f8a4800de10 comp rx=0 tx=0).stop 2024-08-19T01:32:37.902 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:37.901+0000 7f8a5ea00640 1 -- 172.21.15.83:0/1488631096 shutdown_connections 2024-08-19T01:32:37.902 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:37.901+0000 7f8a5ea00640 1 --2- 172.21.15.83:0/1488631096 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f8a3c0a2c00 0x7f8a3c0a50c0 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:32:37.902 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:37.901+0000 7f8a5ea00640 1 --2- 172.21.15.83:0/1488631096 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8a58109b70 0x7f8a5819eca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:32:37.902 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:37.901+0000 7f8a5ea00640 1 --2- 172.21.15.83:0/1488631096 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8a58104480 0x7f8a5819a910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:32:37.903 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:37.901+0000 7f8a5ea00640 1 --2- 172.21.15.83:0/1488631096 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8a58103ae0 0x7f8a5819a3d0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:32:37.903 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:37.901+0000 7f8a5ea00640 1 -- 172.21.15.83:0/1488631096 >> 172.21.15.83:0/1488631096 conn(0x7f8a580ff780 msgr2=0x7f8a580fffb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:32:37.903 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:37.901+0000 7f8a5ea00640 1 -- 172.21.15.83:0/1488631096 shutdown_connections 2024-08-19T01:32:37.903 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:37.902+0000 7f8a5ea00640 1 -- 172.21.15.83:0/1488631096 wait complete. 2024-08-19T01:32:37.952 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:32:37 smithi117 ceph-mon[22016]: pgmap v265: 1 pgs: 1 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-19T01:32:37.952 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:32:37 smithi117 ceph-mon[22016]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' 2024-08-19T01:32:37.952 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:32:37 smithi117 ceph-mon[22016]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' 2024-08-19T01:32:37.953 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:32:37 smithi117 ceph-mon[22016]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-19T01:32:37.953 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:32:37 smithi117 ceph-mon[22016]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-19T01:32:37.953 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:32:37 smithi117 ceph-mon[22016]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' 2024-08-19T01:32:37.953 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:32:37 smithi117 ceph-mon[22016]: from='client.? 172.21.15.83:0/1488631096' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-08-19T01:32:37.953 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:32:37 smithi117 ceph-mon[22016]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-08-19T01:32:37.966 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:32:37 smithi167 ceph-mon[21875]: pgmap v265: 1 pgs: 1 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-19T01:32:37.966 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:32:37 smithi167 ceph-mon[21875]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' 2024-08-19T01:32:37.966 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:32:37 smithi167 ceph-mon[21875]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' 2024-08-19T01:32:37.966 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:32:37 smithi167 ceph-mon[21875]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-19T01:32:37.966 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:32:37 smithi167 ceph-mon[21875]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-19T01:32:37.966 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:32:37 smithi167 ceph-mon[21875]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' 2024-08-19T01:32:37.966 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:32:37 smithi167 ceph-mon[21875]: from='client.? 172.21.15.83:0/1488631096' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-08-19T01:32:37.966 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:32:37 smithi167 ceph-mon[21875]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-08-19T01:32:38.618 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833 shell --fsid bad8780e-5dc9-11ef-bccf-c7b262605968 -- ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-08-19T01:32:39.012 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/bad8780e-5dc9-11ef-bccf-c7b262605968/mon.a/config 2024-08-19T01:32:39.154 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:32:38 smithi083 ceph-mon[17120]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-08-19T01:32:39.154 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:32:38 smithi083 ceph-mon[17120]: osdmap e80: 12 total, 12 up, 12 in 2024-08-19T01:32:39.154 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:32:38 smithi083 ceph-mon[17120]: pgmap v267: 33 pgs: 32 unknown, 1 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-19T01:32:39.202 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:32:38 smithi117 ceph-mon[22016]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-08-19T01:32:39.202 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:32:38 smithi117 ceph-mon[22016]: osdmap e80: 12 total, 12 up, 12 in 2024-08-19T01:32:39.202 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:32:38 smithi117 ceph-mon[22016]: pgmap v267: 33 pgs: 32 unknown, 1 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-19T01:32:39.216 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:32:38 smithi167 ceph-mon[21875]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-08-19T01:32:39.216 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:32:38 smithi167 ceph-mon[21875]: osdmap e80: 12 total, 12 up, 12 in 2024-08-19T01:32:39.216 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:32:38 smithi167 ceph-mon[21875]: pgmap v267: 33 pgs: 32 unknown, 1 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-19T01:32:39.516 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:39.515+0000 7f8737e00640 1 -- 172.21.15.83:0/2037430940 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f87381042a0 msgr2=0x7f87381107d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:32:39.517 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:39.515+0000 7f8737e00640 1 --2- 172.21.15.83:0/2037430940 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f87381042a0 0x7f87381107d0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f8730009cd0 tx=0x7f873002a8e0 comp rx=0 tx=0).stop 2024-08-19T01:32:39.517 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:39.516+0000 7f8737e00640 1 -- 172.21.15.83:0/2037430940 shutdown_connections 2024-08-19T01:32:39.517 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:39.516+0000 7f8737e00640 1 --2- 172.21.15.83:0/2037430940 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f87381042a0 0x7f87381107d0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:32:39.517 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:39.516+0000 7f8737e00640 1 --2- 172.21.15.83:0/2037430940 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8738103900 0x7f8738103d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:32:39.517 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:39.516+0000 7f8737e00640 1 --2- 172.21.15.83:0/2037430940 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8738109900 0x7f8738109ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:32:39.517 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:39.516+0000 7f8737e00640 1 -- 172.21.15.83:0/2037430940 >> 172.21.15.83:0/2037430940 conn(0x7f87380ff600 msgr2=0x7f8738101a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:32:39.517 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:39.516+0000 7f8737e00640 1 -- 172.21.15.83:0/2037430940 shutdown_connections 2024-08-19T01:32:39.517 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:39.516+0000 7f8737e00640 1 -- 172.21.15.83:0/2037430940 wait complete. 2024-08-19T01:32:39.519 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:39.516+0000 7f8737e00640 1 Processor -- start 2024-08-19T01:32:39.519 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:39.517+0000 7f8737e00640 1 -- start start 2024-08-19T01:32:39.519 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:39.517+0000 7f8737e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8738103900 0x7f873819a140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:32:39.519 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:39.517+0000 7f8737e00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f87381042a0 0x7f873819a680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:32:39.519 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:39.517+0000 7f8737e00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8738109900 0x7f873819ea10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:32:39.520 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:39.517+0000 7f8737e00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f8738113d00 con 0x7f8738103900 2024-08-19T01:32:39.520 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:39.517+0000 7f8737e00640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f8738113b80 con 0x7f8738109900 2024-08-19T01:32:39.520 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:39.517+0000 7f8737e00640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f8738113e80 con 0x7f87381042a0 2024-08-19T01:32:39.520 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:39.517+0000 7f8736a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8738103900 0x7f873819a140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:32:39.520 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:39.517+0000 7f872e000640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f87381042a0 0x7f873819a680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:32:39.520 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:39.517+0000 7f872e000640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f87381042a0 0x7f873819a680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.83:52700/0 (socket says 172.21.15.83:52700) 2024-08-19T01:32:39.520 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:39.517+0000 7f872e000640 1 -- 172.21.15.83:0/2867314764 learned_addr learned my addr 172.21.15.83:0/2867314764 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-08-19T01:32:39.520 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:39.517+0000 7f8736a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8738103900 0x7f873819a140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:58150/0 (socket says 172.21.15.83:58150) 2024-08-19T01:32:39.520 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:39.517+0000 7f8737400640 1 --2- 172.21.15.83:0/2867314764 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8738109900 0x7f873819ea10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:32:39.520 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:39.518+0000 7f872e000640 1 -- 172.21.15.83:0/2867314764 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8738109900 msgr2=0x7f873819ea10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:32:39.520 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:39.518+0000 7f872e000640 1 --2- 172.21.15.83:0/2867314764 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8738109900 0x7f873819ea10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:32:39.521 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:39.518+0000 7f872e000640 1 -- 172.21.15.83:0/2867314764 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8738103900 msgr2=0x7f873819a140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:32:39.521 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:39.518+0000 7f872e000640 1 --2- 172.21.15.83:0/2867314764 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8738103900 0x7f873819a140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:32:39.521 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:39.518+0000 7f872e000640 1 -- 172.21.15.83:0/2867314764 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f873819f0f0 con 0x7f87381042a0 2024-08-19T01:32:39.521 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:39.518+0000 7f872e000640 1 --2- 172.21.15.83:0/2867314764 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f87381042a0 0x7f873819a680 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f872000e9f0 tx=0x7f872000eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:32:39.521 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:39.518+0000 7f872fe00640 1 -- 172.21.15.83:0/2867314764 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f872000cd80 con 0x7f87381042a0 2024-08-19T01:32:39.521 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:39.518+0000 7f8737e00640 1 -- 172.21.15.83:0/2867314764 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f873819f3e0 con 0x7f87381042a0 2024-08-19T01:32:39.521 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:39.518+0000 7f872fe00640 1 -- 172.21.15.83:0/2867314764 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8720010910 con 0x7f87381042a0 2024-08-19T01:32:39.521 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:39.518+0000 7f872fe00640 1 -- 172.21.15.83:0/2867314764 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87200188f0 con 0x7f87381042a0 2024-08-19T01:32:39.521 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:39.518+0000 7f8737e00640 1 -- 172.21.15.83:0/2867314764 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f87381a6cc0 con 0x7f87381042a0 2024-08-19T01:32:39.522 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:39.519+0000 7f8737e00640 1 -- 172.21.15.83:0/2867314764 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8738103d60 con 0x7f87381042a0 2024-08-19T01:32:39.522 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:39.521+0000 7f872fe00640 1 -- 172.21.15.83:0/2867314764 <== mon.1 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f8720010ac0 con 0x7f87381042a0 2024-08-19T01:32:39.523 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:39.523+0000 7f872fe00640 1 --2- 172.21.15.83:0/2867314764 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f871c0a2b80 0x7f871c0a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:32:39.523 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:39.523+0000 7f872fe00640 1 -- 172.21.15.83:0/2867314764 <== mon.1 v2:172.21.15.167:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7497+0+0 (secure 0 0 0) 0x7f87200cf540 con 0x7f87381042a0 2024-08-19T01:32:39.524 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:39.523+0000 7f8736a00640 1 --2- 172.21.15.83:0/2867314764 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f871c0a2b80 0x7f871c0a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:32:39.524 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:39.524+0000 7f8736a00640 1 --2- 172.21.15.83:0/2867314764 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f871c0a2b80 0x7f871c0a5040 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f8728009b50 tx=0x7f8728005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:32:39.525 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:39.525+0000 7f872fe00640 1 -- 172.21.15.83:0/2867314764 <== mon.1 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7f872009c0e0 con 0x7f87381042a0 2024-08-19T01:32:39.701 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:39.700+0000 7f8737e00640 1 -- 172.21.15.83:0/2867314764 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7f87381a6ff0 con 0x7f87381042a0 2024-08-19T01:32:40.214 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:40.213+0000 7f872fe00640 1 -- 172.21.15.83:0/2867314764 <== mon.1 v2:172.21.15.167:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v82) ==== 170+0+0 (secure 0 0 0) 0x7f87200a03e0 con 0x7f87381042a0 2024-08-19T01:32:40.214 INFO:teuthology.orchestra.run.smithi083.stderr:pool 'cephfs_data' created 2024-08-19T01:32:40.220 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:40.218+0000 7f8737e00640 1 -- 172.21.15.83:0/2867314764 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f871c0a2b80 msgr2=0x7f871c0a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:32:40.220 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:40.218+0000 7f8737e00640 1 --2- 172.21.15.83:0/2867314764 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f871c0a2b80 0x7f871c0a5040 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f8728009b50 tx=0x7f8728005f90 comp rx=0 tx=0).stop 2024-08-19T01:32:40.220 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:40.218+0000 7f8737e00640 1 -- 172.21.15.83:0/2867314764 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f87381042a0 msgr2=0x7f873819a680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:32:40.220 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:40.218+0000 7f8737e00640 1 --2- 172.21.15.83:0/2867314764 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f87381042a0 0x7f873819a680 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f872000e9f0 tx=0x7f872000eeb0 comp rx=0 tx=0).stop 2024-08-19T01:32:40.220 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:40.219+0000 7f8737e00640 1 -- 172.21.15.83:0/2867314764 shutdown_connections 2024-08-19T01:32:40.220 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:40.219+0000 7f8737e00640 1 --2- 172.21.15.83:0/2867314764 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f871c0a2b80 0x7f871c0a5040 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:32:40.221 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:40.219+0000 7f8737e00640 1 --2- 172.21.15.83:0/2867314764 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8738109900 0x7f873819ea10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:32:40.221 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:40.219+0000 7f8737e00640 1 --2- 172.21.15.83:0/2867314764 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f87381042a0 0x7f873819a680 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:32:40.221 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:40.219+0000 7f8737e00640 1 --2- 172.21.15.83:0/2867314764 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8738103900 0x7f873819a140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:32:40.221 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:40.219+0000 7f8737e00640 1 -- 172.21.15.83:0/2867314764 >> 172.21.15.83:0/2867314764 conn(0x7f87380ff600 msgr2=0x7f8738100cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:32:40.221 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:40.220+0000 7f8737e00640 1 -- 172.21.15.83:0/2867314764 shutdown_connections 2024-08-19T01:32:40.221 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:40.220+0000 7f8737e00640 1 -- 172.21.15.83:0/2867314764 wait complete. 2024-08-19T01:32:40.466 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:32:40 smithi167 ceph-mon[21875]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-08-19T01:32:40.466 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:32:40 smithi167 ceph-mon[21875]: osdmap e81: 12 total, 12 up, 12 in 2024-08-19T01:32:40.546 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:32:40 smithi083 ceph-mon[17120]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-08-19T01:32:40.547 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:32:40 smithi083 ceph-mon[17120]: osdmap e81: 12 total, 12 up, 12 in 2024-08-19T01:32:40.702 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:32:40 smithi117 ceph-mon[22016]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-08-19T01:32:40.702 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:32:40 smithi117 ceph-mon[22016]: osdmap e81: 12 total, 12 up, 12 in 2024-08-19T01:32:40.884 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833 shell --fsid bad8780e-5dc9-11ef-bccf-c7b262605968 -- ceph fs new cephfs cephfs_metadata cephfs_data 2024-08-19T01:32:41.275 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/bad8780e-5dc9-11ef-bccf-c7b262605968/mon.a/config 2024-08-19T01:32:41.466 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:32:41 smithi167 ceph-mon[21875]: from='client.? 172.21.15.83:0/2867314764' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-08-19T01:32:41.466 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:32:41 smithi167 ceph-mon[21875]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-08-19T01:32:41.466 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:32:41 smithi167 ceph-mon[21875]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-08-19T01:32:41.466 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:32:41 smithi167 ceph-mon[21875]: osdmap e82: 12 total, 12 up, 12 in 2024-08-19T01:32:41.466 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:32:41 smithi167 ceph-mon[21875]: pgmap v270: 97 pgs: 6 creating+peering, 82 unknown, 9 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-19T01:32:41.548 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:32:41 smithi083 ceph-mon[17120]: from='client.? 172.21.15.83:0/2867314764' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-08-19T01:32:41.548 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:32:41 smithi083 ceph-mon[17120]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-08-19T01:32:41.548 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:32:41 smithi083 ceph-mon[17120]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-08-19T01:32:41.548 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:32:41 smithi083 ceph-mon[17120]: osdmap e82: 12 total, 12 up, 12 in 2024-08-19T01:32:41.548 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:32:41 smithi083 ceph-mon[17120]: pgmap v270: 97 pgs: 6 creating+peering, 82 unknown, 9 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-19T01:32:41.702 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:32:41 smithi117 ceph-mon[22016]: from='client.? 172.21.15.83:0/2867314764' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-08-19T01:32:41.703 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:32:41 smithi117 ceph-mon[22016]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-08-19T01:32:41.703 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:32:41 smithi117 ceph-mon[22016]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-08-19T01:32:41.703 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:32:41 smithi117 ceph-mon[22016]: osdmap e82: 12 total, 12 up, 12 in 2024-08-19T01:32:41.703 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:32:41 smithi117 ceph-mon[22016]: pgmap v270: 97 pgs: 6 creating+peering, 82 unknown, 9 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-19T01:32:41.829 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:41.828+0000 7f5fd2000640 1 -- 172.21.15.83:0/4244726967 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5fcc1078b0 msgr2=0x7f5fcc107c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:32:41.829 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:41.828+0000 7f5fd2000640 1 --2- 172.21.15.83:0/4244726967 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5fcc1078b0 0x7f5fcc107c90 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f5fbc009930 tx=0x7f5fbc02a440 comp rx=0 tx=0).stop 2024-08-19T01:32:41.830 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:41.828+0000 7f5fd2000640 1 -- 172.21.15.83:0/4244726967 shutdown_connections 2024-08-19T01:32:41.830 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:41.828+0000 7f5fd2000640 1 --2- 172.21.15.83:0/4244726967 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5fcc102250 0x7f5fcc10e780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:32:41.830 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:41.828+0000 7f5fd2000640 1 --2- 172.21.15.83:0/4244726967 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5fcc1018b0 0x7f5fcc101d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:32:41.830 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:41.828+0000 7f5fd2000640 1 --2- 172.21.15.83:0/4244726967 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5fcc1078b0 0x7f5fcc107c90 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:32:41.830 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:41.828+0000 7f5fd2000640 1 -- 172.21.15.83:0/4244726967 >> 172.21.15.83:0/4244726967 conn(0x7f5fcc0fd5f0 msgr2=0x7f5fcc0ffa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:32:41.830 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:41.828+0000 7f5fd2000640 1 -- 172.21.15.83:0/4244726967 shutdown_connections 2024-08-19T01:32:41.830 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:41.828+0000 7f5fd2000640 1 -- 172.21.15.83:0/4244726967 wait complete. 2024-08-19T01:32:41.832 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:41.829+0000 7f5fd2000640 1 Processor -- start 2024-08-19T01:32:41.832 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:41.829+0000 7f5fd2000640 1 -- start start 2024-08-19T01:32:41.832 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:41.830+0000 7f5fd2000640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5fcc1018b0 0x7f5fcc074220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:32:41.832 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:41.830+0000 7f5fd2000640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5fcc102250 0x7f5fcc074760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:32:41.833 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:41.830+0000 7f5fd2000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5fcc1078b0 0x7f5fcc074ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:32:41.833 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:41.830+0000 7f5fd2000640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f5fcc111cc0 con 0x7f5fcc1078b0 2024-08-19T01:32:41.833 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:41.830+0000 7f5fd2000640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f5fcc111b40 con 0x7f5fcc102250 2024-08-19T01:32:41.833 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:41.830+0000 7f5fd2000640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f5fcc111e40 con 0x7f5fcc1018b0 2024-08-19T01:32:41.833 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:41.830+0000 7f5fcbe00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5fcc1078b0 0x7f5fcc074ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:32:41.833 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:41.830+0000 7f5fcbe00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5fcc1078b0 0x7f5fcc074ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:60668/0 (socket says 172.21.15.83:60668) 2024-08-19T01:32:41.833 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:41.830+0000 7f5fcbe00640 1 -- 172.21.15.83:0/2664477276 learned_addr learned my addr 172.21.15.83:0/2664477276 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-08-19T01:32:41.833 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:41.830+0000 7f5fcb400640 1 --2- 172.21.15.83:0/2664477276 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5fcc1018b0 0x7f5fcc074220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:32:41.833 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:41.830+0000 7f5fc3e00640 1 --2- 172.21.15.83:0/2664477276 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5fcc102250 0x7f5fcc074760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:32:41.833 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:41.830+0000 7f5fcbe00640 1 -- 172.21.15.83:0/2664477276 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5fcc1018b0 msgr2=0x7f5fcc074220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:32:41.833 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:41.830+0000 7f5fcbe00640 1 --2- 172.21.15.83:0/2664477276 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5fcc1018b0 0x7f5fcc074220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:32:41.833 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:41.830+0000 7f5fcbe00640 1 -- 172.21.15.83:0/2664477276 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5fcc102250 msgr2=0x7f5fcc074760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:32:41.833 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:41.830+0000 7f5fcbe00640 1 --2- 172.21.15.83:0/2664477276 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5fcc102250 0x7f5fcc074760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:32:41.834 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:41.830+0000 7f5fcbe00640 1 -- 172.21.15.83:0/2664477276 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5fcc1a6660 con 0x7f5fcc1078b0 2024-08-19T01:32:41.834 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:41.831+0000 7f5fcbe00640 1 --2- 172.21.15.83:0/2664477276 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5fcc1078b0 0x7f5fcc074ca0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f5fc400eaa0 tx=0x7f5fc400ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:32:41.834 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:41.831+0000 7f5fc8c00640 1 -- 172.21.15.83:0/2664477276 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5fc400cdb0 con 0x7f5fcc1078b0 2024-08-19T01:32:41.834 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:41.831+0000 7f5fc8c00640 1 -- 172.21.15.83:0/2664477276 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5fc4005020 con 0x7f5fcc1078b0 2024-08-19T01:32:41.834 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:41.831+0000 7f5fd2000640 1 -- 172.21.15.83:0/2664477276 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5fcc1a6950 con 0x7f5fcc1078b0 2024-08-19T01:32:41.835 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:41.831+0000 7f5fc8c00640 1 -- 172.21.15.83:0/2664477276 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5fc40116b0 con 0x7f5fcc1078b0 2024-08-19T01:32:41.836 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:41.831+0000 7f5fd2000640 1 -- 172.21.15.83:0/2664477276 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5fcc1a6e40 con 0x7f5fcc1078b0 2024-08-19T01:32:41.836 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:41.832+0000 7f5fd2000640 1 -- 172.21.15.83:0/2664477276 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5fcc101d10 con 0x7f5fcc1078b0 2024-08-19T01:32:41.836 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:41.834+0000 7f5fc8c00640 1 -- 172.21.15.83:0/2664477276 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f5fc4005560 con 0x7f5fcc1078b0 2024-08-19T01:32:41.837 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:41.835+0000 7f5fc8c00640 1 --2- 172.21.15.83:0/2664477276 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f5fb00a2b80 0x7f5fb00a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:32:41.837 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:41.836+0000 7f5fc8c00640 1 -- 172.21.15.83:0/2664477276 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 7879+0+0 (secure 0 0 0) 0x7f5fc40cf090 con 0x7f5fcc1078b0 2024-08-19T01:32:41.837 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:41.836+0000 7f5fcb400640 1 --2- 172.21.15.83:0/2664477276 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f5fb00a2b80 0x7f5fb00a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:32:41.838 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:41.836+0000 7f5fcb400640 1 --2- 172.21.15.83:0/2664477276 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f5fb00a2b80 0x7f5fb00a5040 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f5fbc006010 tx=0x7f5fbc000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:32:41.840 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:41.840+0000 7f5fc8c00640 1 -- 172.21.15.83:0/2664477276 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7f5fc4016030 con 0x7f5fcc1078b0 2024-08-19T01:32:42.054 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:42.053+0000 7f5fd2000640 1 -- 172.21.15.83:0/2664477276 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7f5fcc109890 con 0x7f5fcc1078b0 2024-08-19T01:32:42.216 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:42.215+0000 7f5fc8c00640 1 -- 172.21.15.83:0/2664477276 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 2 and data pool 3 v14) ==== 172+0+0 (secure 0 0 0) 0x7f5fc409bab0 con 0x7f5fcc1078b0 2024-08-19T01:32:42.217 INFO:teuthology.orchestra.run.smithi083.stderr:new fs with metadata pool 2 and data pool 3 2024-08-19T01:32:42.220 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:42.219+0000 7f5fd2000640 1 -- 172.21.15.83:0/2664477276 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f5fb00a2b80 msgr2=0x7f5fb00a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:32:42.220 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:42.219+0000 7f5fd2000640 1 --2- 172.21.15.83:0/2664477276 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f5fb00a2b80 0x7f5fb00a5040 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f5fbc006010 tx=0x7f5fbc000bc0 comp rx=0 tx=0).stop 2024-08-19T01:32:42.220 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:42.219+0000 7f5fd2000640 1 -- 172.21.15.83:0/2664477276 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5fcc1078b0 msgr2=0x7f5fcc074ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:32:42.220 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:42.219+0000 7f5fd2000640 1 --2- 172.21.15.83:0/2664477276 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5fcc1078b0 0x7f5fcc074ca0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f5fc400eaa0 tx=0x7f5fc400ef60 comp rx=0 tx=0).stop 2024-08-19T01:32:42.220 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:42.219+0000 7f5fd2000640 1 -- 172.21.15.83:0/2664477276 shutdown_connections 2024-08-19T01:32:42.220 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:42.219+0000 7f5fd2000640 1 --2- 172.21.15.83:0/2664477276 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f5fb00a2b80 0x7f5fb00a5040 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:32:42.220 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:42.219+0000 7f5fd2000640 1 --2- 172.21.15.83:0/2664477276 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5fcc1078b0 0x7f5fcc074ca0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:32:42.220 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:42.219+0000 7f5fd2000640 1 --2- 172.21.15.83:0/2664477276 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5fcc102250 0x7f5fcc074760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:32:42.220 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:42.219+0000 7f5fd2000640 1 --2- 172.21.15.83:0/2664477276 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5fcc1018b0 0x7f5fcc074220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:32:42.220 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:42.219+0000 7f5fd2000640 1 -- 172.21.15.83:0/2664477276 >> 172.21.15.83:0/2664477276 conn(0x7f5fcc0fd5f0 msgr2=0x7f5fcc0fdde0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:32:42.220 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:42.219+0000 7f5fd2000640 1 -- 172.21.15.83:0/2664477276 shutdown_connections 2024-08-19T01:32:42.220 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:42.219+0000 7f5fd2000640 1 -- 172.21.15.83:0/2664477276 wait complete. 2024-08-19T01:32:42.404 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:32:42 smithi083 ceph-mon[17120]: osdmap e83: 12 total, 12 up, 12 in 2024-08-19T01:32:42.405 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:32:42 smithi083 ceph-mon[17120]: from='client.? 172.21.15.83:0/2664477276' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-08-19T01:32:42.405 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:32:42 smithi083 ceph-mon[17120]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-08-19T01:32:42.452 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:32:42 smithi117 ceph-mon[22016]: osdmap e83: 12 total, 12 up, 12 in 2024-08-19T01:32:42.452 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:32:42 smithi117 ceph-mon[22016]: from='client.? 172.21.15.83:0/2664477276' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-08-19T01:32:42.452 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:32:42 smithi117 ceph-mon[22016]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-08-19T01:32:42.716 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:32:42 smithi167 ceph-mon[21875]: osdmap e83: 12 total, 12 up, 12 in 2024-08-19T01:32:42.716 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:32:42 smithi167 ceph-mon[21875]: from='client.? 172.21.15.83:0/2664477276' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-08-19T01:32:42.716 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:32:42 smithi167 ceph-mon[21875]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-08-19T01:32:42.991 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833 shell --fsid bad8780e-5dc9-11ef-bccf-c7b262605968 -- ceph osd dump --format=json 2024-08-19T01:32:43.363 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/bad8780e-5dc9-11ef-bccf-c7b262605968/mon.a/config 2024-08-19T01:32:43.504 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:32:43 smithi083 ceph-mon[17120]: from='client.? 172.21.15.83:0/2664477276' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-08-19T01:32:43.505 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:32:43 smithi083 ceph-mon[17120]: osdmap e84: 12 total, 12 up, 12 in 2024-08-19T01:32:43.505 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:32:43 smithi083 ceph-mon[17120]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-08-19T01:32:43.505 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:32:43 smithi083 ceph-mon[17120]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-08-19T01:32:43.505 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:32:43 smithi083 ceph-mon[17120]: pgmap v273: 97 pgs: 12 creating+peering, 68 unknown, 17 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-19T01:32:43.702 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:32:43 smithi117 ceph-mon[22016]: from='client.? 172.21.15.83:0/2664477276' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-08-19T01:32:43.702 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:32:43 smithi117 ceph-mon[22016]: osdmap e84: 12 total, 12 up, 12 in 2024-08-19T01:32:43.703 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:32:43 smithi117 ceph-mon[22016]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-08-19T01:32:43.703 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:32:43 smithi117 ceph-mon[22016]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-08-19T01:32:43.703 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:32:43 smithi117 ceph-mon[22016]: pgmap v273: 97 pgs: 12 creating+peering, 68 unknown, 17 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-19T01:32:43.716 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:32:43 smithi167 ceph-mon[21875]: from='client.? 172.21.15.83:0/2664477276' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-08-19T01:32:43.716 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:32:43 smithi167 ceph-mon[21875]: osdmap e84: 12 total, 12 up, 12 in 2024-08-19T01:32:43.716 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:32:43 smithi167 ceph-mon[21875]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-08-19T01:32:43.716 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:32:43 smithi167 ceph-mon[21875]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-08-19T01:32:43.716 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:32:43 smithi167 ceph-mon[21875]: pgmap v273: 97 pgs: 12 creating+peering, 68 unknown, 17 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-19T01:32:43.891 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:43.874+0000 7f61da800640 1 -- 172.21.15.83:0/1633120067 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f61d4109900 msgr2=0x7f61d4109ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:32:43.892 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:43.874+0000 7f61da800640 1 --2- 172.21.15.83:0/1633120067 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f61d4109900 0x7f61d4109ce0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f61c8009930 tx=0x7f61c802a420 comp rx=0 tx=0).stop 2024-08-19T01:32:43.892 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:43.874+0000 7f61da800640 1 -- 172.21.15.83:0/1633120067 shutdown_connections 2024-08-19T01:32:43.892 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:43.874+0000 7f61da800640 1 --2- 172.21.15.83:0/1633120067 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f61d41042a0 0x7f61d41107d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:32:43.892 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:43.874+0000 7f61da800640 1 --2- 172.21.15.83:0/1633120067 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f61d4103900 0x7f61d4103d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:32:43.892 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:43.874+0000 7f61da800640 1 --2- 172.21.15.83:0/1633120067 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f61d4109900 0x7f61d4109ce0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:32:43.892 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:43.874+0000 7f61da800640 1 -- 172.21.15.83:0/1633120067 >> 172.21.15.83:0/1633120067 conn(0x7f61d40ff640 msgr2=0x7f61d4101a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:32:43.892 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:43.874+0000 7f61da800640 1 -- 172.21.15.83:0/1633120067 shutdown_connections 2024-08-19T01:32:43.892 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:43.874+0000 7f61da800640 1 -- 172.21.15.83:0/1633120067 wait complete. 2024-08-19T01:32:43.892 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:43.875+0000 7f61da800640 1 Processor -- start 2024-08-19T01:32:43.892 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:43.875+0000 7f61da800640 1 -- start start 2024-08-19T01:32:43.892 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:43.875+0000 7f61da800640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f61d4103900 0x7f61d419a290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:32:43.892 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:43.875+0000 7f61da800640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f61d41042a0 0x7f61d419a7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:32:43.893 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:43.875+0000 7f61da800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f61d4109900 0x7f61d419eb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:32:43.893 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:43.875+0000 7f61da800640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f61d4113d50 con 0x7f61d4109900 2024-08-19T01:32:43.893 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:43.875+0000 7f61da800640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f61d4113bd0 con 0x7f61d4103900 2024-08-19T01:32:43.893 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:43.875+0000 7f61da800640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f61d4113ed0 con 0x7f61d41042a0 2024-08-19T01:32:43.893 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:43.876+0000 7f61d9400640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f61d4103900 0x7f61d419a290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:32:43.893 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:43.876+0000 7f61d8a00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f61d41042a0 0x7f61d419a7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:32:43.893 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:43.876+0000 7f61d9400640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f61d4103900 0x7f61d419a290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.83:43788/0 (socket says 172.21.15.83:43788) 2024-08-19T01:32:43.893 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:43.876+0000 7f61d9400640 1 -- 172.21.15.83:0/304713519 learned_addr learned my addr 172.21.15.83:0/304713519 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-08-19T01:32:43.893 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:43.876+0000 7f61d9e00640 1 --2- 172.21.15.83:0/304713519 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f61d4109900 0x7f61d419eb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:32:43.893 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:43.876+0000 7f61d9400640 1 -- 172.21.15.83:0/304713519 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f61d41042a0 msgr2=0x7f61d419a7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:32:43.893 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:43.876+0000 7f61d9400640 1 --2- 172.21.15.83:0/304713519 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f61d41042a0 0x7f61d419a7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:32:43.893 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:43.876+0000 7f61d9400640 1 -- 172.21.15.83:0/304713519 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f61d4109900 msgr2=0x7f61d419eb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:32:43.893 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:43.876+0000 7f61d9400640 1 --2- 172.21.15.83:0/304713519 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f61d4109900 0x7f61d419eb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:32:43.893 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:43.876+0000 7f61d9400640 1 -- 172.21.15.83:0/304713519 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f61d419f240 con 0x7f61d4103900 2024-08-19T01:32:43.894 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:43.877+0000 7f61d9400640 1 --2- 172.21.15.83:0/304713519 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f61d4103900 0x7f61d419a290 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f61c802a940 tx=0x7f61c8003b20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:32:43.894 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:43.877+0000 7f61d2000640 1 -- 172.21.15.83:0/304713519 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61c803f070 con 0x7f61d4103900 2024-08-19T01:32:43.894 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:43.877+0000 7f61da800640 1 -- 172.21.15.83:0/304713519 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f61d419f4d0 con 0x7f61d4103900 2024-08-19T01:32:43.894 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:43.877+0000 7f61d2000640 1 -- 172.21.15.83:0/304713519 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f61c8031c90 con 0x7f61d4103900 2024-08-19T01:32:43.894 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:43.877+0000 7f61d2000640 1 -- 172.21.15.83:0/304713519 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61c8035030 con 0x7f61d4103900 2024-08-19T01:32:43.894 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:43.877+0000 7f61da800640 1 -- 172.21.15.83:0/304713519 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f61d41a6e10 con 0x7f61d4103900 2024-08-19T01:32:43.894 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:43.878+0000 7f61da800640 1 -- 172.21.15.83:0/304713519 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f61d4105040 con 0x7f61d4103900 2024-08-19T01:32:43.894 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:43.880+0000 7f61d2000640 1 -- 172.21.15.83:0/304713519 <== mon.2 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f61c8042070 con 0x7f61d4103900 2024-08-19T01:32:43.894 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:43.881+0000 7f61d2000640 1 --2- 172.21.15.83:0/304713519 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f61bc0a2c00 0x7f61bc0a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:32:43.894 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:43.881+0000 7f61d2000640 1 -- 172.21.15.83:0/304713519 <== mon.2 v2:172.21.15.117:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 7979+0+0 (secure 0 0 0) 0x7f61c80efcc0 con 0x7f61d4103900 2024-08-19T01:32:43.894 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:43.882+0000 7f61d8a00640 1 --2- 172.21.15.83:0/304713519 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f61bc0a2c00 0x7f61bc0a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:32:43.894 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:43.882+0000 7f61d8a00640 1 --2- 172.21.15.83:0/304713519 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f61bc0a2c00 0x7f61bc0a50c0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f61d4101d00 tx=0x7f61c0005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:32:43.894 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:43.884+0000 7f61d2000640 1 -- 172.21.15.83:0/304713519 <== mon.2 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7f61c80bc680 con 0x7f61d4103900 2024-08-19T01:32:44.063 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:44.061+0000 7f61da800640 1 -- 172.21.15.83:0/304713519 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f61d410be70 con 0x7f61d4103900 2024-08-19T01:32:44.065 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:44.063+0000 7f61d2000640 1 -- 172.21.15.83:0/304713519 <== mon.2 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v84) ==== 74+0+22194 (secure 0 0 0) 0x7f61c80c0980 con 0x7f61d4103900 2024-08-19T01:32:44.065 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-08-19T01:32:44.065 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":84,"fsid":"bad8780e-5dc9-11ef-bccf-c7b262605968","created":"2024-08-19T01:25:12.626841+0000","modified":"2024-08-19T01:32:42.054551+0000","last_up_change":"2024-08-19T01:31:47.254559+0000","last_in_change":"2024-08-19T01:31:27.652847+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-08-19T01:28:16.862007+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-08-19T01:32:37.455535+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-08-19T01:32:40.201935+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ffa74041-f9d2-4555-bb43-508b8c581487","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6802","nonce":1082526824},{"type":"v1","addr":"172.21.15.83:6803","nonce":1082526824}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6804","nonce":1082526824},{"type":"v1","addr":"172.21.15.83:6805","nonce":1082526824}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6808","nonce":1082526824},{"type":"v1","addr":"172.21.15.83:6809","nonce":1082526824}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6806","nonce":1082526824},{"type":"v1","addr":"172.21.15.83:6807","nonce":1082526824}]},"public_addr":"172.21.15.83:6803/1082526824","cluster_addr":"172.21.15.83:6805/1082526824","heartbeat_back_addr":"172.21.15.83:6809/1082526824","heartbeat_front_addr":"172.21.15.83:6807/1082526824","state":["exists","up"]},{"osd":1,"uuid":"1977570b-b4f5-44fd-bb1e-3a31318be649","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6800","nonce":1925419802},{"type":"v1","addr":"172.21.15.117:6801","nonce":1925419802}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6802","nonce":1925419802},{"type":"v1","addr":"172.21.15.117:6803","nonce":1925419802}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6806","nonce":1925419802},{"type":"v1","addr":"172.21.15.117:6807","nonce":1925419802}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6804","nonce":1925419802},{"type":"v1","addr":"172.21.15.117:6805","nonce":1925419802}]},"public_addr":"172.21.15.117:6801/1925419802","cluster_addr":"172.21.15.117:6803/1925419802","heartbeat_back_addr":"172.21.15.117:6807/1925419802","heartbeat_front_addr":"172.21.15.117:6805/1925419802","state":["exists","up"]},{"osd":2,"uuid":"a75278d1-161f-41d5-a92f-058d1077bf2a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6800","nonce":551625328},{"type":"v1","addr":"172.21.15.167:6801","nonce":551625328}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6802","nonce":551625328},{"type":"v1","addr":"172.21.15.167:6803","nonce":551625328}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6806","nonce":551625328},{"type":"v1","addr":"172.21.15.167:6807","nonce":551625328}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6804","nonce":551625328},{"type":"v1","addr":"172.21.15.167:6805","nonce":551625328}]},"public_addr":"172.21.15.167:6801/551625328","cluster_addr":"172.21.15.167:6803/551625328","heartbeat_back_addr":"172.21.15.167:6807/551625328","heartbeat_front_addr":"172.21.15.167:6805/551625328","state":["exists","up"]},{"osd":3,"uuid":"d1fb16f9-910e-46fa-b1e3-8e5bef4d346d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6810","nonce":3138942062},{"type":"v1","addr":"172.21.15.83:6811","nonce":3138942062}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6812","nonce":3138942062},{"type":"v1","addr":"172.21.15.83:6813","nonce":3138942062}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6816","nonce":3138942062},{"type":"v1","addr":"172.21.15.83:6817","nonce":3138942062}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6814","nonce":3138942062},{"type":"v1","addr":"172.21.15.83:6815","nonce":3138942062}]},"public_addr":"172.21.15.83:6811/3138942062","cluster_addr":"172.21.15.83:6813/3138942062","heartbeat_back_addr":"172.21.15.83:6817/3138942062","heartbeat_front_addr":"172.21.15.83:6815/3138942062","state":["exists","up"]},{"osd":4,"uuid":"8176ed18-1fe2-446d-8994-8c60f6e08f38","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6808","nonce":3978863827},{"type":"v1","addr":"172.21.15.117:6809","nonce":3978863827}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6810","nonce":3978863827},{"type":"v1","addr":"172.21.15.117:6811","nonce":3978863827}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6814","nonce":3978863827},{"type":"v1","addr":"172.21.15.117:6815","nonce":3978863827}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6812","nonce":3978863827},{"type":"v1","addr":"172.21.15.117:6813","nonce":3978863827}]},"public_addr":"172.21.15.117:6809/3978863827","cluster_addr":"172.21.15.117:6811/3978863827","heartbeat_back_addr":"172.21.15.117:6815/3978863827","heartbeat_front_addr":"172.21.15.117:6813/3978863827","state":["exists","up"]},{"osd":5,"uuid":"e01e3f15-25c8-4d2c-be6f-a170f036064c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6808","nonce":2761858035},{"type":"v1","addr":"172.21.15.167:6809","nonce":2761858035}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6810","nonce":2761858035},{"type":"v1","addr":"172.21.15.167:6811","nonce":2761858035}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6814","nonce":2761858035},{"type":"v1","addr":"172.21.15.167:6815","nonce":2761858035}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6812","nonce":2761858035},{"type":"v1","addr":"172.21.15.167:6813","nonce":2761858035}]},"public_addr":"172.21.15.167:6809/2761858035","cluster_addr":"172.21.15.167:6811/2761858035","heartbeat_back_addr":"172.21.15.167:6815/2761858035","heartbeat_front_addr":"172.21.15.167:6813/2761858035","state":["exists","up"]},{"osd":6,"uuid":"78cf0a9b-9165-444f-b073-ebe13016de9d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6818","nonce":260644552},{"type":"v1","addr":"172.21.15.83:6819","nonce":260644552}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6820","nonce":260644552},{"type":"v1","addr":"172.21.15.83:6821","nonce":260644552}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6824","nonce":260644552},{"type":"v1","addr":"172.21.15.83:6825","nonce":260644552}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6822","nonce":260644552},{"type":"v1","addr":"172.21.15.83:6823","nonce":260644552}]},"public_addr":"172.21.15.83:6819/260644552","cluster_addr":"172.21.15.83:6821/260644552","heartbeat_back_addr":"172.21.15.83:6825/260644552","heartbeat_front_addr":"172.21.15.83:6823/260644552","state":["exists","up"]},{"osd":7,"uuid":"ee524f7f-672e-430e-9f58-1a9117513466","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6816","nonce":104113252},{"type":"v1","addr":"172.21.15.117:6817","nonce":104113252}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6818","nonce":104113252},{"type":"v1","addr":"172.21.15.117:6819","nonce":104113252}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6822","nonce":104113252},{"type":"v1","addr":"172.21.15.117:6823","nonce":104113252}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6820","nonce":104113252},{"type":"v1","addr":"172.21.15.117:6821","nonce":104113252}]},"public_addr":"172.21.15.117:6817/104113252","cluster_addr":"172.21.15.117:6819/104113252","heartbeat_back_addr":"172.21.15.117:6823/104113252","heartbeat_front_addr":"172.21.15.117:6821/104113252","state":["exists","up"]},{"osd":8,"uuid":"76d5317d-4845-4138-b210-e7bd2d80472a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":58,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6816","nonce":2196898857},{"type":"v1","addr":"172.21.15.167:6817","nonce":2196898857}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6818","nonce":2196898857},{"type":"v1","addr":"172.21.15.167:6819","nonce":2196898857}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6822","nonce":2196898857},{"type":"v1","addr":"172.21.15.167:6823","nonce":2196898857}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6820","nonce":2196898857},{"type":"v1","addr":"172.21.15.167:6821","nonce":2196898857}]},"public_addr":"172.21.15.167:6817/2196898857","cluster_addr":"172.21.15.167:6819/2196898857","heartbeat_back_addr":"172.21.15.167:6823/2196898857","heartbeat_front_addr":"172.21.15.167:6821/2196898857","state":["exists","up"]},{"osd":9,"uuid":"0151bc2a-3ad2-4db3-8505-5855792cc9bb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6826","nonce":903463450},{"type":"v1","addr":"172.21.15.83:6827","nonce":903463450}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6828","nonce":903463450},{"type":"v1","addr":"172.21.15.83:6829","nonce":903463450}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6832","nonce":903463450},{"type":"v1","addr":"172.21.15.83:6833","nonce":903463450}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6830","nonce":903463450},{"type":"v1","addr":"172.21.15.83:6831","nonce":903463450}]},"public_addr":"172.21.15.83:6827/903463450","cluster_addr":"172.21.15.83:6829/903463450","heartbeat_back_addr":"172.21.15.83:6833/903463450","heartbeat_front_addr":"172.21.15.83:6831/903463450","state":["exists","up"]},{"osd":10,"uuid":"437ed848-0da4-4cd4-8716-0c55bb99b676","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6824","nonce":1183570837},{"type":"v1","addr":"172.21.15.117:6825","nonce":1183570837}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6826","nonce":1183570837},{"type":"v1","addr":"172.21.15.117:6827","nonce":1183570837}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6830","nonce":1183570837},{"type":"v1","addr":"172.21.15.117:6831","nonce":1183570837}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6828","nonce":1183570837},{"type":"v1","addr":"172.21.15.117:6829","nonce":1183570837}]},"public_addr":"172.21.15.117:6825/1183570837","cluster_addr":"172.21.15.117:6827/1183570837","heartbeat_back_addr":"172.21.15.117:6831/1183570837","heartbeat_front_addr":"172.21.15.117:6829/1183570837","state":["exists","up"]},{"osd":11,"uuid":"13345f05-cdc0-4dae-a58b-551abbec1915","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":78,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6824","nonce":187027350},{"type":"v1","addr":"172.21.15.167:6825","nonce":187027350}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6826","nonce":187027350},{"type":"v1","addr":"172.21.15.167:6827","nonce":187027350}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6830","nonce":187027350},{"type":"v1","addr":"172.21.15.167:6831","nonce":187027350}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6828","nonce":187027350},{"type":"v1","addr":"172.21.15.167:6829","nonce":187027350}]},"public_addr":"172.21.15.167:6825/187027350","cluster_addr":"172.21.15.167:6827/187027350","heartbeat_back_addr":"172.21.15.167:6831/187027350","heartbeat_front_addr":"172.21.15.167:6829/187027350","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-19T01:27:28.760517+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-19T01:27:50.493028+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-19T01:28:11.997674+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-19T01:28:38.109934+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-19T01:29:00.173065+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-19T01:29:21.336085+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-19T01:29:47.871498+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-19T01:30:10.720498+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-19T01:30:32.425438+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-19T01:30:59.524989+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-19T01:31:22.806162+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-19T01:31:45.043246+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.83:0/3138095466":"2024-08-20T01:26:04.277716+0000","172.21.15.83:0/3835005198":"2024-08-20T01:26:04.277716+0000","172.21.15.83:0/1790374088":"2024-08-20T01:25:37.228583+0000","172.21.15.83:6801/573907319":"2024-08-20T01:26:04.277716+0000","172.21.15.83:6800/573907319":"2024-08-20T01:26:04.277716+0000","172.21.15.83:0/664676002":"2024-08-20T01:25:37.228583+0000","172.21.15.83:0/2144611449":"2024-08-20T01:25:37.228583+0000","172.21.15.83:6801/3261152818":"2024-08-20T01:25:37.228583+0000","172.21.15.83:0/1609552666":"2024-08-20T01:26:04.277716+0000","172.21.15.83:6800/3261152818":"2024-08-20T01:25:37.228583+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-08-19T01:32:44.068 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:44.066+0000 7f61da800640 1 -- 172.21.15.83:0/304713519 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f61bc0a2c00 msgr2=0x7f61bc0a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:32:44.068 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:44.066+0000 7f61da800640 1 --2- 172.21.15.83:0/304713519 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f61bc0a2c00 0x7f61bc0a50c0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f61d4101d00 tx=0x7f61c0005f90 comp rx=0 tx=0).stop 2024-08-19T01:32:44.068 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:44.066+0000 7f61da800640 1 -- 172.21.15.83:0/304713519 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f61d4103900 msgr2=0x7f61d419a290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:32:44.068 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:44.066+0000 7f61da800640 1 --2- 172.21.15.83:0/304713519 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f61d4103900 0x7f61d419a290 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f61c802a940 tx=0x7f61c8003b20 comp rx=0 tx=0).stop 2024-08-19T01:32:44.068 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:44.066+0000 7f61da800640 1 -- 172.21.15.83:0/304713519 shutdown_connections 2024-08-19T01:32:44.068 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:44.066+0000 7f61da800640 1 --2- 172.21.15.83:0/304713519 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f61bc0a2c00 0x7f61bc0a50c0 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:32:44.068 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:44.066+0000 7f61da800640 1 --2- 172.21.15.83:0/304713519 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f61d4109900 0x7f61d419eb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:32:44.068 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:44.066+0000 7f61da800640 1 --2- 172.21.15.83:0/304713519 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f61d41042a0 0x7f61d419a7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:32:44.068 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:44.066+0000 7f61da800640 1 --2- 172.21.15.83:0/304713519 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f61d4103900 0x7f61d419a290 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:32:44.068 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:44.066+0000 7f61da800640 1 -- 172.21.15.83:0/304713519 >> 172.21.15.83:0/304713519 conn(0x7f61d40ff640 msgr2=0x7f61d40fffa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:32:44.068 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:44.066+0000 7f61da800640 1 -- 172.21.15.83:0/304713519 shutdown_connections 2024-08-19T01:32:44.069 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:44.066+0000 7f61da800640 1 -- 172.21.15.83:0/304713519 wait complete. 2024-08-19T01:32:44.401 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:32:44 smithi083 ceph-mon[17120]: Health check cleared: POOL_APP_NOT_ENABLED (was: 2 pool(s) do not have an application enabled) 2024-08-19T01:32:44.401 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:32:44 smithi083 ceph-mon[17120]: Cluster is now healthy 2024-08-19T01:32:44.401 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:32:44 smithi083 ceph-mon[17120]: mds.? [v2:172.21.15.117:6838/1967595704,v1:172.21.15.117:6839/1967595704] up:active 2024-08-19T01:32:44.401 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:32:44 smithi083 ceph-mon[17120]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-08-19T01:32:44.401 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:32:44 smithi083 ceph-mon[17120]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-08-19T01:32:44.402 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:32:44 smithi083 ceph-mon[17120]: from='client.? 172.21.15.83:0/304713519' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-08-19T01:32:44.702 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:32:44 smithi117 ceph-mon[22016]: Health check cleared: POOL_APP_NOT_ENABLED (was: 2 pool(s) do not have an application enabled) 2024-08-19T01:32:44.703 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:32:44 smithi117 ceph-mon[22016]: Cluster is now healthy 2024-08-19T01:32:44.703 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:32:44 smithi117 ceph-mon[22016]: mds.? [v2:172.21.15.117:6838/1967595704,v1:172.21.15.117:6839/1967595704] up:active 2024-08-19T01:32:44.703 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:32:44 smithi117 ceph-mon[22016]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-08-19T01:32:44.703 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:32:44 smithi117 ceph-mon[22016]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-08-19T01:32:44.703 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:32:44 smithi117 ceph-mon[22016]: from='client.? 172.21.15.83:0/304713519' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-08-19T01:32:44.716 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:32:44 smithi167 ceph-mon[21875]: Health check cleared: POOL_APP_NOT_ENABLED (was: 2 pool(s) do not have an application enabled) 2024-08-19T01:32:44.716 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:32:44 smithi167 ceph-mon[21875]: Cluster is now healthy 2024-08-19T01:32:44.716 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:32:44 smithi167 ceph-mon[21875]: mds.? [v2:172.21.15.117:6838/1967595704,v1:172.21.15.117:6839/1967595704] up:active 2024-08-19T01:32:44.716 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:32:44 smithi167 ceph-mon[21875]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-08-19T01:32:44.716 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:32:44 smithi167 ceph-mon[21875]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-08-19T01:32:44.716 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:32:44 smithi167 ceph-mon[21875]: from='client.? 172.21.15.83:0/304713519' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-08-19T01:32:44.754 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833 shell --fsid bad8780e-5dc9-11ef-bccf-c7b262605968 -- ceph osd dump --format=json 2024-08-19T01:32:45.130 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/bad8780e-5dc9-11ef-bccf-c7b262605968/mon.a/config 2024-08-19T01:32:45.422 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:32:45 smithi083 ceph-mon[17120]: pgmap v274: 97 pgs: 6 creating+peering, 8 unknown, 83 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 563 B/s wr, 5 op/s 2024-08-19T01:32:45.452 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:32:45 smithi117 ceph-mon[22016]: pgmap v274: 97 pgs: 6 creating+peering, 8 unknown, 83 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 563 B/s wr, 5 op/s 2024-08-19T01:32:45.453 INFO:journalctl@ceph.mds.b.smithi117.stdout:Aug 19 01:32:45 smithi117 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-mds-b[51905]: 2024-08-19T01:32:45.133+0000 7f79e1eb8640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-08-19T01:32:45.666 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:45.663+0000 7f3fcf400640 1 -- 172.21.15.83:0/614582007 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3fc8102240 msgr2=0x7f3fc810e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:32:45.666 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:45.663+0000 7f3fcf400640 1 --2- 172.21.15.83:0/614582007 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3fc8102240 0x7f3fc810e770 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f3fc0009cd0 tx=0x7f3fc002a8e0 comp rx=0 tx=0).stop 2024-08-19T01:32:45.666 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:45.664+0000 7f3fcf400640 1 -- 172.21.15.83:0/614582007 shutdown_connections 2024-08-19T01:32:45.666 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:45.664+0000 7f3fcf400640 1 --2- 172.21.15.83:0/614582007 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3fc8102240 0x7f3fc810e770 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:32:45.666 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:45.664+0000 7f3fcf400640 1 --2- 172.21.15.83:0/614582007 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3fc81018a0 0x7f3fc8101d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:32:45.666 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:45.664+0000 7f3fcf400640 1 --2- 172.21.15.83:0/614582007 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3fc81078a0 0x7f3fc8107c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:32:45.666 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:45.664+0000 7f3fcf400640 1 -- 172.21.15.83:0/614582007 >> 172.21.15.83:0/614582007 conn(0x7f3fc80fd5e0 msgr2=0x7f3fc80ffa00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:32:45.666 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:45.664+0000 7f3fcf400640 1 -- 172.21.15.83:0/614582007 shutdown_connections 2024-08-19T01:32:45.666 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:45.664+0000 7f3fcf400640 1 -- 172.21.15.83:0/614582007 wait complete. 2024-08-19T01:32:45.668 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:45.665+0000 7f3fcf400640 1 Processor -- start 2024-08-19T01:32:45.668 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:45.665+0000 7f3fcf400640 1 -- start start 2024-08-19T01:32:45.668 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:45.665+0000 7f3fcf400640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3fc81018a0 0x7f3fc819a350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:32:45.668 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:45.665+0000 7f3fcf400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3fc8102240 0x7f3fc819a890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:32:45.668 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:45.665+0000 7f3fcf400640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3fc81078a0 0x7f3fc819ec20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:32:45.668 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:45.665+0000 7f3fcf400640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f3fc8111cf0 con 0x7f3fc8102240 2024-08-19T01:32:45.669 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:45.665+0000 7f3fcf400640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f3fc8111b70 con 0x7f3fc81078a0 2024-08-19T01:32:45.669 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:45.665+0000 7f3fcf400640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f3fc8111e70 con 0x7f3fc81018a0 2024-08-19T01:32:45.669 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:45.665+0000 7f3fbfe00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3fc8102240 0x7f3fc819a890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:32:45.669 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:45.665+0000 7f3fbfe00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3fc8102240 0x7f3fc819a890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:60706/0 (socket says 172.21.15.83:60706) 2024-08-19T01:32:45.669 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:45.665+0000 7f3fbfe00640 1 -- 172.21.15.83:0/3942845811 learned_addr learned my addr 172.21.15.83:0/3942845811 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-08-19T01:32:45.669 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:45.665+0000 7f3fcca00640 1 --2- 172.21.15.83:0/3942845811 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3fc81018a0 0x7f3fc819a350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:32:45.669 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:45.665+0000 7f3fcd400640 1 --2- 172.21.15.83:0/3942845811 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3fc81078a0 0x7f3fc819ec20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:32:45.669 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:45.666+0000 7f3fcca00640 1 -- 172.21.15.83:0/3942845811 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3fc81078a0 msgr2=0x7f3fc819ec20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:32:45.669 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:45.666+0000 7f3fcca00640 1 --2- 172.21.15.83:0/3942845811 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3fc81078a0 0x7f3fc819ec20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:32:45.669 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:45.666+0000 7f3fcca00640 1 -- 172.21.15.83:0/3942845811 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3fc8102240 msgr2=0x7f3fc819a890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:32:45.669 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:45.666+0000 7f3fcca00640 1 --2- 172.21.15.83:0/3942845811 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3fc8102240 0x7f3fc819a890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:32:45.669 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:45.666+0000 7f3fcca00640 1 -- 172.21.15.83:0/3942845811 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3fc819f300 con 0x7f3fc81018a0 2024-08-19T01:32:45.669 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:45.666+0000 7f3fbfe00640 1 --2- 172.21.15.83:0/3942845811 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3fc8102240 0x7f3fc819a890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-19T01:32:45.669 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:45.666+0000 7f3fcca00640 1 --2- 172.21.15.83:0/3942845811 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3fc81018a0 0x7f3fc819a350 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f3fb800e750 tx=0x7f3fb800ec10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:32:45.670 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:45.666+0000 7f3fc6000640 1 -- 172.21.15.83:0/3942845811 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3fb800ccc0 con 0x7f3fc81018a0 2024-08-19T01:32:45.670 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:45.666+0000 7f3fcf400640 1 -- 172.21.15.83:0/3942845811 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3fc819f5f0 con 0x7f3fc81018a0 2024-08-19T01:32:45.670 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:45.667+0000 7f3fc6000640 1 -- 172.21.15.83:0/3942845811 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3fb800ee60 con 0x7f3fc81018a0 2024-08-19T01:32:45.670 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:45.667+0000 7f3fcf400640 1 -- 172.21.15.83:0/3942845811 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3fc81a6ed0 con 0x7f3fc81018a0 2024-08-19T01:32:45.670 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:45.667+0000 7f3fc6000640 1 -- 172.21.15.83:0/3942845811 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3fb801c070 con 0x7f3fc81018a0 2024-08-19T01:32:45.670 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:45.668+0000 7f3fcf400640 1 -- 172.21.15.83:0/3942845811 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3fa4005290 con 0x7f3fc81018a0 2024-08-19T01:32:45.671 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:45.669+0000 7f3fc6000640 1 -- 172.21.15.83:0/3942845811 <== mon.1 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f3fb8042ae0 con 0x7f3fc81018a0 2024-08-19T01:32:45.672 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:45.671+0000 7f3fc6000640 1 --2- 172.21.15.83:0/3942845811 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f3fac0a2c00 0x7f3fac0a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:32:45.672 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:45.671+0000 7f3fc6000640 1 -- 172.21.15.83:0/3942845811 <== mon.1 v2:172.21.15.167:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 7979+0+0 (secure 0 0 0) 0x7f3fb80cecb0 con 0x7f3fc81018a0 2024-08-19T01:32:45.672 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:45.671+0000 7f3fbfe00640 1 --2- 172.21.15.83:0/3942845811 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f3fac0a2c00 0x7f3fac0a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:32:45.673 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:45.672+0000 7f3fbfe00640 1 --2- 172.21.15.83:0/3942845811 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f3fac0a2c00 0x7f3fac0a50c0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f3fc810deb0 tx=0x7f3fb0005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:32:45.675 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:45.673+0000 7f3fc6000640 1 -- 172.21.15.83:0/3942845811 <== mon.1 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7f3fb809bcc0 con 0x7f3fc81018a0 2024-08-19T01:32:45.716 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:32:45 smithi167 ceph-mon[21875]: pgmap v274: 97 pgs: 6 creating+peering, 8 unknown, 83 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 563 B/s wr, 5 op/s 2024-08-19T01:32:45.851 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:45.849+0000 7f3fcf400640 1 -- 172.21.15.83:0/3942845811 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f3fa4005850 con 0x7f3fc81018a0 2024-08-19T01:32:45.853 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:45.851+0000 7f3fc6000640 1 -- 172.21.15.83:0/3942845811 <== mon.1 v2:172.21.15.167:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v84) ==== 74+0+22194 (secure 0 0 0) 0x7f3fb809b5c0 con 0x7f3fc81018a0 2024-08-19T01:32:45.854 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-08-19T01:32:45.854 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":84,"fsid":"bad8780e-5dc9-11ef-bccf-c7b262605968","created":"2024-08-19T01:25:12.626841+0000","modified":"2024-08-19T01:32:42.054551+0000","last_up_change":"2024-08-19T01:31:47.254559+0000","last_in_change":"2024-08-19T01:31:27.652847+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-08-19T01:28:16.862007+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-08-19T01:32:37.455535+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-08-19T01:32:40.201935+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ffa74041-f9d2-4555-bb43-508b8c581487","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6802","nonce":1082526824},{"type":"v1","addr":"172.21.15.83:6803","nonce":1082526824}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6804","nonce":1082526824},{"type":"v1","addr":"172.21.15.83:6805","nonce":1082526824}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6808","nonce":1082526824},{"type":"v1","addr":"172.21.15.83:6809","nonce":1082526824}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6806","nonce":1082526824},{"type":"v1","addr":"172.21.15.83:6807","nonce":1082526824}]},"public_addr":"172.21.15.83:6803/1082526824","cluster_addr":"172.21.15.83:6805/1082526824","heartbeat_back_addr":"172.21.15.83:6809/1082526824","heartbeat_front_addr":"172.21.15.83:6807/1082526824","state":["exists","up"]},{"osd":1,"uuid":"1977570b-b4f5-44fd-bb1e-3a31318be649","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6800","nonce":1925419802},{"type":"v1","addr":"172.21.15.117:6801","nonce":1925419802}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6802","nonce":1925419802},{"type":"v1","addr":"172.21.15.117:6803","nonce":1925419802}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6806","nonce":1925419802},{"type":"v1","addr":"172.21.15.117:6807","nonce":1925419802}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6804","nonce":1925419802},{"type":"v1","addr":"172.21.15.117:6805","nonce":1925419802}]},"public_addr":"172.21.15.117:6801/1925419802","cluster_addr":"172.21.15.117:6803/1925419802","heartbeat_back_addr":"172.21.15.117:6807/1925419802","heartbeat_front_addr":"172.21.15.117:6805/1925419802","state":["exists","up"]},{"osd":2,"uuid":"a75278d1-161f-41d5-a92f-058d1077bf2a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6800","nonce":551625328},{"type":"v1","addr":"172.21.15.167:6801","nonce":551625328}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6802","nonce":551625328},{"type":"v1","addr":"172.21.15.167:6803","nonce":551625328}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6806","nonce":551625328},{"type":"v1","addr":"172.21.15.167:6807","nonce":551625328}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6804","nonce":551625328},{"type":"v1","addr":"172.21.15.167:6805","nonce":551625328}]},"public_addr":"172.21.15.167:6801/551625328","cluster_addr":"172.21.15.167:6803/551625328","heartbeat_back_addr":"172.21.15.167:6807/551625328","heartbeat_front_addr":"172.21.15.167:6805/551625328","state":["exists","up"]},{"osd":3,"uuid":"d1fb16f9-910e-46fa-b1e3-8e5bef4d346d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6810","nonce":3138942062},{"type":"v1","addr":"172.21.15.83:6811","nonce":3138942062}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6812","nonce":3138942062},{"type":"v1","addr":"172.21.15.83:6813","nonce":3138942062}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6816","nonce":3138942062},{"type":"v1","addr":"172.21.15.83:6817","nonce":3138942062}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6814","nonce":3138942062},{"type":"v1","addr":"172.21.15.83:6815","nonce":3138942062}]},"public_addr":"172.21.15.83:6811/3138942062","cluster_addr":"172.21.15.83:6813/3138942062","heartbeat_back_addr":"172.21.15.83:6817/3138942062","heartbeat_front_addr":"172.21.15.83:6815/3138942062","state":["exists","up"]},{"osd":4,"uuid":"8176ed18-1fe2-446d-8994-8c60f6e08f38","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6808","nonce":3978863827},{"type":"v1","addr":"172.21.15.117:6809","nonce":3978863827}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6810","nonce":3978863827},{"type":"v1","addr":"172.21.15.117:6811","nonce":3978863827}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6814","nonce":3978863827},{"type":"v1","addr":"172.21.15.117:6815","nonce":3978863827}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6812","nonce":3978863827},{"type":"v1","addr":"172.21.15.117:6813","nonce":3978863827}]},"public_addr":"172.21.15.117:6809/3978863827","cluster_addr":"172.21.15.117:6811/3978863827","heartbeat_back_addr":"172.21.15.117:6815/3978863827","heartbeat_front_addr":"172.21.15.117:6813/3978863827","state":["exists","up"]},{"osd":5,"uuid":"e01e3f15-25c8-4d2c-be6f-a170f036064c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6808","nonce":2761858035},{"type":"v1","addr":"172.21.15.167:6809","nonce":2761858035}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6810","nonce":2761858035},{"type":"v1","addr":"172.21.15.167:6811","nonce":2761858035}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6814","nonce":2761858035},{"type":"v1","addr":"172.21.15.167:6815","nonce":2761858035}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6812","nonce":2761858035},{"type":"v1","addr":"172.21.15.167:6813","nonce":2761858035}]},"public_addr":"172.21.15.167:6809/2761858035","cluster_addr":"172.21.15.167:6811/2761858035","heartbeat_back_addr":"172.21.15.167:6815/2761858035","heartbeat_front_addr":"172.21.15.167:6813/2761858035","state":["exists","up"]},{"osd":6,"uuid":"78cf0a9b-9165-444f-b073-ebe13016de9d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6818","nonce":260644552},{"type":"v1","addr":"172.21.15.83:6819","nonce":260644552}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6820","nonce":260644552},{"type":"v1","addr":"172.21.15.83:6821","nonce":260644552}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6824","nonce":260644552},{"type":"v1","addr":"172.21.15.83:6825","nonce":260644552}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6822","nonce":260644552},{"type":"v1","addr":"172.21.15.83:6823","nonce":260644552}]},"public_addr":"172.21.15.83:6819/260644552","cluster_addr":"172.21.15.83:6821/260644552","heartbeat_back_addr":"172.21.15.83:6825/260644552","heartbeat_front_addr":"172.21.15.83:6823/260644552","state":["exists","up"]},{"osd":7,"uuid":"ee524f7f-672e-430e-9f58-1a9117513466","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6816","nonce":104113252},{"type":"v1","addr":"172.21.15.117:6817","nonce":104113252}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6818","nonce":104113252},{"type":"v1","addr":"172.21.15.117:6819","nonce":104113252}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6822","nonce":104113252},{"type":"v1","addr":"172.21.15.117:6823","nonce":104113252}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6820","nonce":104113252},{"type":"v1","addr":"172.21.15.117:6821","nonce":104113252}]},"public_addr":"172.21.15.117:6817/104113252","cluster_addr":"172.21.15.117:6819/104113252","heartbeat_back_addr":"172.21.15.117:6823/104113252","heartbeat_front_addr":"172.21.15.117:6821/104113252","state":["exists","up"]},{"osd":8,"uuid":"76d5317d-4845-4138-b210-e7bd2d80472a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":58,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6816","nonce":2196898857},{"type":"v1","addr":"172.21.15.167:6817","nonce":2196898857}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6818","nonce":2196898857},{"type":"v1","addr":"172.21.15.167:6819","nonce":2196898857}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6822","nonce":2196898857},{"type":"v1","addr":"172.21.15.167:6823","nonce":2196898857}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6820","nonce":2196898857},{"type":"v1","addr":"172.21.15.167:6821","nonce":2196898857}]},"public_addr":"172.21.15.167:6817/2196898857","cluster_addr":"172.21.15.167:6819/2196898857","heartbeat_back_addr":"172.21.15.167:6823/2196898857","heartbeat_front_addr":"172.21.15.167:6821/2196898857","state":["exists","up"]},{"osd":9,"uuid":"0151bc2a-3ad2-4db3-8505-5855792cc9bb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6826","nonce":903463450},{"type":"v1","addr":"172.21.15.83:6827","nonce":903463450}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6828","nonce":903463450},{"type":"v1","addr":"172.21.15.83:6829","nonce":903463450}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6832","nonce":903463450},{"type":"v1","addr":"172.21.15.83:6833","nonce":903463450}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6830","nonce":903463450},{"type":"v1","addr":"172.21.15.83:6831","nonce":903463450}]},"public_addr":"172.21.15.83:6827/903463450","cluster_addr":"172.21.15.83:6829/903463450","heartbeat_back_addr":"172.21.15.83:6833/903463450","heartbeat_front_addr":"172.21.15.83:6831/903463450","state":["exists","up"]},{"osd":10,"uuid":"437ed848-0da4-4cd4-8716-0c55bb99b676","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6824","nonce":1183570837},{"type":"v1","addr":"172.21.15.117:6825","nonce":1183570837}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6826","nonce":1183570837},{"type":"v1","addr":"172.21.15.117:6827","nonce":1183570837}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6830","nonce":1183570837},{"type":"v1","addr":"172.21.15.117:6831","nonce":1183570837}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6828","nonce":1183570837},{"type":"v1","addr":"172.21.15.117:6829","nonce":1183570837}]},"public_addr":"172.21.15.117:6825/1183570837","cluster_addr":"172.21.15.117:6827/1183570837","heartbeat_back_addr":"172.21.15.117:6831/1183570837","heartbeat_front_addr":"172.21.15.117:6829/1183570837","state":["exists","up"]},{"osd":11,"uuid":"13345f05-cdc0-4dae-a58b-551abbec1915","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":78,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6824","nonce":187027350},{"type":"v1","addr":"172.21.15.167:6825","nonce":187027350}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6826","nonce":187027350},{"type":"v1","addr":"172.21.15.167:6827","nonce":187027350}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6830","nonce":187027350},{"type":"v1","addr":"172.21.15.167:6831","nonce":187027350}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6828","nonce":187027350},{"type":"v1","addr":"172.21.15.167:6829","nonce":187027350}]},"public_addr":"172.21.15.167:6825/187027350","cluster_addr":"172.21.15.167:6827/187027350","heartbeat_back_addr":"172.21.15.167:6831/187027350","heartbeat_front_addr":"172.21.15.167:6829/187027350","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-19T01:27:28.760517+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-19T01:27:50.493028+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-19T01:28:11.997674+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-19T01:28:38.109934+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-19T01:29:00.173065+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-19T01:29:21.336085+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-19T01:29:47.871498+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-19T01:30:10.720498+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-19T01:30:32.425438+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-19T01:30:59.524989+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-19T01:31:22.806162+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-19T01:31:45.043246+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.83:0/3138095466":"2024-08-20T01:26:04.277716+0000","172.21.15.83:0/3835005198":"2024-08-20T01:26:04.277716+0000","172.21.15.83:0/1790374088":"2024-08-20T01:25:37.228583+0000","172.21.15.83:6801/573907319":"2024-08-20T01:26:04.277716+0000","172.21.15.83:6800/573907319":"2024-08-20T01:26:04.277716+0000","172.21.15.83:0/664676002":"2024-08-20T01:25:37.228583+0000","172.21.15.83:0/2144611449":"2024-08-20T01:25:37.228583+0000","172.21.15.83:6801/3261152818":"2024-08-20T01:25:37.228583+0000","172.21.15.83:0/1609552666":"2024-08-20T01:26:04.277716+0000","172.21.15.83:6800/3261152818":"2024-08-20T01:25:37.228583+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-08-19T01:32:45.855 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:45.853+0000 7f3fcf400640 1 -- 172.21.15.83:0/3942845811 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f3fac0a2c00 msgr2=0x7f3fac0a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:32:45.856 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:45.854+0000 7f3fcf400640 1 --2- 172.21.15.83:0/3942845811 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f3fac0a2c00 0x7f3fac0a50c0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f3fc810deb0 tx=0x7f3fb0005f90 comp rx=0 tx=0).stop 2024-08-19T01:32:45.856 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:45.854+0000 7f3fcf400640 1 -- 172.21.15.83:0/3942845811 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3fc81018a0 msgr2=0x7f3fc819a350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:32:45.856 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:45.854+0000 7f3fcf400640 1 --2- 172.21.15.83:0/3942845811 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3fc81018a0 0x7f3fc819a350 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f3fb800e750 tx=0x7f3fb800ec10 comp rx=0 tx=0).stop 2024-08-19T01:32:45.856 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:45.854+0000 7f3fcf400640 1 -- 172.21.15.83:0/3942845811 shutdown_connections 2024-08-19T01:32:45.856 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:45.854+0000 7f3fcf400640 1 --2- 172.21.15.83:0/3942845811 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f3fac0a2c00 0x7f3fac0a50c0 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:32:45.856 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:45.854+0000 7f3fcf400640 1 --2- 172.21.15.83:0/3942845811 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3fc81078a0 0x7f3fc819ec20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:32:45.856 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:45.854+0000 7f3fcf400640 1 --2- 172.21.15.83:0/3942845811 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3fc8102240 0x7f3fc819a890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:32:45.857 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:45.854+0000 7f3fcf400640 1 --2- 172.21.15.83:0/3942845811 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3fc81018a0 0x7f3fc819a350 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:32:45.857 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:45.854+0000 7f3fcf400640 1 -- 172.21.15.83:0/3942845811 >> 172.21.15.83:0/3942845811 conn(0x7f3fc80fd5e0 msgr2=0x7f3fc810cef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:32:45.857 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:45.854+0000 7f3fcf400640 1 -- 172.21.15.83:0/3942845811 shutdown_connections 2024-08-19T01:32:45.857 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:45.854+0000 7f3fcf400640 1 -- 172.21.15.83:0/3942845811 wait complete. 2024-08-19T01:32:46.604 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:32:46 smithi083 ceph-mon[17120]: from='client.? 172.21.15.83:0/3942845811' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-08-19T01:32:46.645 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833 shell --fsid bad8780e-5dc9-11ef-bccf-c7b262605968 -- ceph fs set cephfs standby_count_wanted 0 2024-08-19T01:32:46.702 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:32:46 smithi117 ceph-mon[22016]: from='client.? 172.21.15.83:0/3942845811' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-08-19T01:32:46.716 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:32:46 smithi167 ceph-mon[21875]: from='client.? 172.21.15.83:0/3942845811' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-08-19T01:32:47.045 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/bad8780e-5dc9-11ef-bccf-c7b262605968/mon.a/config 2024-08-19T01:32:47.550 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:32:47 smithi083 ceph-mon[17120]: pgmap v275: 97 pgs: 97 active+clean; 578 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s wr, 6 op/s 2024-08-19T01:32:47.565 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:47.563+0000 7f8004a00640 1 -- 172.21.15.83:0/4046006842 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8000101700 msgr2=0x7f8000101b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:32:47.565 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:47.563+0000 7f8004a00640 1 --2- 172.21.15.83:0/4046006842 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8000101700 0x7f8000101b60 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f7fe8009860 tx=0x7f7fe802a340 comp rx=0 tx=0).stop 2024-08-19T01:32:47.566 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:47.563+0000 7f8004a00640 1 -- 172.21.15.83:0/4046006842 shutdown_connections 2024-08-19T01:32:47.566 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:47.563+0000 7f8004a00640 1 --2- 172.21.15.83:0/4046006842 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f80001020a0 0x7f800010e5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:32:47.566 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:47.563+0000 7f8004a00640 1 --2- 172.21.15.83:0/4046006842 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8000101700 0x7f8000101b60 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:32:47.566 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:47.563+0000 7f8004a00640 1 --2- 172.21.15.83:0/4046006842 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8000107700 0x7f8000107ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:32:47.566 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:47.563+0000 7f8004a00640 1 -- 172.21.15.83:0/4046006842 >> 172.21.15.83:0/4046006842 conn(0x7f80000fd440 msgr2=0x7f80000ff860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:32:47.566 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:47.564+0000 7f8004a00640 1 -- 172.21.15.83:0/4046006842 shutdown_connections 2024-08-19T01:32:47.566 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:47.564+0000 7f8004a00640 1 -- 172.21.15.83:0/4046006842 wait complete. 2024-08-19T01:32:47.568 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:47.564+0000 7f8004a00640 1 Processor -- start 2024-08-19T01:32:47.568 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:47.565+0000 7f8004a00640 1 -- start start 2024-08-19T01:32:47.568 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:47.565+0000 7f8004a00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8000101700 0x7f8000074220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:32:47.568 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:47.565+0000 7f8004a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f80001020a0 0x7f8000074760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:32:47.569 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:47.565+0000 7f8004a00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8000107700 0x7f8000074ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:32:47.569 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:47.565+0000 7f8004a00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f8000111b00 con 0x7f80001020a0 2024-08-19T01:32:47.569 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:47.565+0000 7f8004a00640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f8000111980 con 0x7f8000107700 2024-08-19T01:32:47.569 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:47.565+0000 7f8004a00640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f8000111c80 con 0x7f8000101700 2024-08-19T01:32:47.569 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:47.565+0000 7f7fff400640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8000101700 0x7f8000074220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:32:47.569 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:47.565+0000 7f7ff7e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f80001020a0 0x7f8000074760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:32:47.569 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:47.565+0000 7f7fff400640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8000101700 0x7f8000074220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.83:42960/0 (socket says 172.21.15.83:42960) 2024-08-19T01:32:47.569 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:47.565+0000 7f7fff400640 1 -- 172.21.15.83:0/4085878473 learned_addr learned my addr 172.21.15.83:0/4085878473 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-08-19T01:32:47.569 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:47.565+0000 7f7fffe00640 1 --2- 172.21.15.83:0/4085878473 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8000107700 0x7f8000074ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:32:47.569 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:47.566+0000 7f7fff400640 1 -- 172.21.15.83:0/4085878473 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8000107700 msgr2=0x7f8000074ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:32:47.569 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:47.566+0000 7f7fff400640 1 --2- 172.21.15.83:0/4085878473 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8000107700 0x7f8000074ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:32:47.569 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:47.566+0000 7f7fff400640 1 -- 172.21.15.83:0/4085878473 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f80001020a0 msgr2=0x7f8000074760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:32:47.569 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:47.566+0000 7f7fff400640 1 --2- 172.21.15.83:0/4085878473 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f80001020a0 0x7f8000074760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:32:47.570 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:47.566+0000 7f7fff400640 1 -- 172.21.15.83:0/4085878473 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f80001a6500 con 0x7f8000101700 2024-08-19T01:32:47.570 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:47.566+0000 7f7fff400640 1 --2- 172.21.15.83:0/4085878473 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8000101700 0x7f8000074220 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f7ff000e9f0 tx=0x7f7ff000edf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:32:47.570 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:47.566+0000 7f7ffcc00640 1 -- 172.21.15.83:0/4085878473 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ff000cd80 con 0x7f8000101700 2024-08-19T01:32:47.570 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:47.566+0000 7f8004a00640 1 -- 172.21.15.83:0/4085878473 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f80001a67f0 con 0x7f8000101700 2024-08-19T01:32:47.570 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:47.566+0000 7f7ffcc00640 1 -- 172.21.15.83:0/4085878473 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7ff0005500 con 0x7f8000101700 2024-08-19T01:32:47.570 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:47.566+0000 7f7ffcc00640 1 -- 172.21.15.83:0/4085878473 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ff0010690 con 0x7f8000101700 2024-08-19T01:32:47.570 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:47.566+0000 7f8004a00640 1 -- 172.21.15.83:0/4085878473 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f80001a6d00 con 0x7f8000101700 2024-08-19T01:32:47.570 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:47.567+0000 7f8004a00640 1 -- 172.21.15.83:0/4085878473 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8000101b60 con 0x7f8000101700 2024-08-19T01:32:47.571 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:47.569+0000 7f7ffcc00640 1 -- 172.21.15.83:0/4085878473 <== mon.1 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f7ff00046c0 con 0x7f8000101700 2024-08-19T01:32:47.572 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:47.571+0000 7f7ffcc00640 1 --2- 172.21.15.83:0/4085878473 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f7fe40a2c00 0x7f7fe40a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:32:47.572 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:47.571+0000 7f7ffcc00640 1 -- 172.21.15.83:0/4085878473 <== mon.1 v2:172.21.15.167:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 7979+0+0 (secure 0 0 0) 0x7f7ff00cf3f0 con 0x7f8000101700 2024-08-19T01:32:47.573 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:47.571+0000 7f7ff7e00640 1 --2- 172.21.15.83:0/4085878473 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f7fe40a2c00 0x7f7fe40a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:32:47.573 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:47.572+0000 7f7ff7e00640 1 --2- 172.21.15.83:0/4085878473 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f7fe40a2c00 0x7f7fe40a50c0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f7fe802a930 tx=0x7f7fe80056d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:32:47.574 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:47.573+0000 7f7ffcc00640 1 -- 172.21.15.83:0/4085878473 <== mon.1 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7f7ff0053df0 con 0x7f8000101700 2024-08-19T01:32:47.603 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:32:47 smithi117 ceph-mon[22016]: pgmap v275: 97 pgs: 97 active+clean; 578 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s wr, 6 op/s 2024-08-19T01:32:47.617 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:32:47 smithi167 ceph-mon[21875]: pgmap v275: 97 pgs: 97 active+clean; 578 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s wr, 6 op/s 2024-08-19T01:32:47.794 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:47.792+0000 7f8004a00640 1 -- 172.21.15.83:0/4085878473 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7f8000109c70 con 0x7f8000101700 2024-08-19T01:32:48.258 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:48.256+0000 7f7ffcc00640 1 -- 172.21.15.83:0/4085878473 <== mon.1 v2:172.21.15.167:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v17) ==== 118+0+0 (secure 0 0 0) 0x7f7ff009c4a0 con 0x7f8000101700 2024-08-19T01:32:48.260 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:48.258+0000 7f8004a00640 1 -- 172.21.15.83:0/4085878473 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f7fe40a2c00 msgr2=0x7f7fe40a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:32:48.260 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:48.258+0000 7f8004a00640 1 --2- 172.21.15.83:0/4085878473 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f7fe40a2c00 0x7f7fe40a50c0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f7fe802a930 tx=0x7f7fe80056d0 comp rx=0 tx=0).stop 2024-08-19T01:32:48.260 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:48.258+0000 7f8004a00640 1 -- 172.21.15.83:0/4085878473 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8000101700 msgr2=0x7f8000074220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:32:48.261 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:48.258+0000 7f8004a00640 1 --2- 172.21.15.83:0/4085878473 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8000101700 0x7f8000074220 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f7ff000e9f0 tx=0x7f7ff000edf0 comp rx=0 tx=0).stop 2024-08-19T01:32:48.261 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:48.258+0000 7f8004a00640 1 -- 172.21.15.83:0/4085878473 shutdown_connections 2024-08-19T01:32:48.261 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:48.258+0000 7f8004a00640 1 --2- 172.21.15.83:0/4085878473 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f7fe40a2c00 0x7f7fe40a50c0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:32:48.261 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:48.258+0000 7f8004a00640 1 --2- 172.21.15.83:0/4085878473 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8000107700 0x7f8000074ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:32:48.261 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:48.258+0000 7f8004a00640 1 --2- 172.21.15.83:0/4085878473 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f80001020a0 0x7f8000074760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:32:48.261 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:48.258+0000 7f8004a00640 1 --2- 172.21.15.83:0/4085878473 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8000101700 0x7f8000074220 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:32:48.261 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:48.258+0000 7f8004a00640 1 -- 172.21.15.83:0/4085878473 >> 172.21.15.83:0/4085878473 conn(0x7f80000fd440 msgr2=0x7f80000fdc70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:32:48.261 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:48.259+0000 7f8004a00640 1 -- 172.21.15.83:0/4085878473 shutdown_connections 2024-08-19T01:32:48.261 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:48.259+0000 7f8004a00640 1 -- 172.21.15.83:0/4085878473 wait complete. 2024-08-19T01:32:48.404 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:32:48 smithi083 ceph-mon[17120]: from='client.? 172.21.15.83:0/4085878473' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-08-19T01:32:48.405 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:32:48 smithi083 ceph-mon[17120]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-08-19T01:32:48.702 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:32:48 smithi117 ceph-mon[22016]: from='client.? 172.21.15.83:0/4085878473' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-08-19T01:32:48.702 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:32:48 smithi117 ceph-mon[22016]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-08-19T01:32:48.716 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:32:48 smithi167 ceph-mon[21875]: from='client.? 172.21.15.83:0/4085878473' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-08-19T01:32:48.716 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:32:48 smithi167 ceph-mon[21875]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-08-19T01:32:49.041 DEBUG:tasks.cephfs.filesystem:fs_config: {'max_mds': 3, 'session_timeout': 300, 'standby_replay': True, 'subvols': {'create': 2, 'subvol_options': '--size 25000000000'}} 2024-08-19T01:32:49.041 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833 shell --fsid bad8780e-5dc9-11ef-bccf-c7b262605968 -- ceph fs set cephfs max_mds 3 2024-08-19T01:32:49.424 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/bad8780e-5dc9-11ef-bccf-c7b262605968/mon.a/config 2024-08-19T01:32:49.564 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:32:49 smithi083 ceph-mon[17120]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-08-19T01:32:49.565 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:32:49 smithi083 ceph-mon[17120]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-08-19T01:32:49.565 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:32:49 smithi083 ceph-mon[17120]: pgmap v276: 97 pgs: 97 active+clean; 579 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s wr, 5 op/s 2024-08-19T01:32:49.702 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:32:49 smithi117 ceph-mon[22016]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-08-19T01:32:49.702 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:32:49 smithi117 ceph-mon[22016]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-08-19T01:32:49.703 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:32:49 smithi117 ceph-mon[22016]: pgmap v276: 97 pgs: 97 active+clean; 579 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s wr, 5 op/s 2024-08-19T01:32:49.716 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:32:49 smithi167 ceph-mon[21875]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-08-19T01:32:49.716 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:32:49 smithi167 ceph-mon[21875]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-08-19T01:32:49.716 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:32:49 smithi167 ceph-mon[21875]: pgmap v276: 97 pgs: 97 active+clean; 579 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s wr, 5 op/s 2024-08-19T01:32:49.954 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:49.952+0000 7f3fb8000640 1 -- 172.21.15.83:0/410670034 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3fb010beb0 msgr2=0x7f3fb010e350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:32:49.954 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:49.952+0000 7f3fb8000640 1 --2- 172.21.15.83:0/410670034 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3fb010beb0 0x7f3fb010e350 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f3fa8009be0 tx=0x7f3fa802a750 comp rx=0 tx=0).stop 2024-08-19T01:32:49.954 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:49.952+0000 7f3fb8000640 1 -- 172.21.15.83:0/410670034 shutdown_connections 2024-08-19T01:32:49.954 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:49.952+0000 7f3fb8000640 1 --2- 172.21.15.83:0/410670034 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3fb010beb0 0x7f3fb010e350 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:32:49.954 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:49.952+0000 7f3fb8000640 1 --2- 172.21.15.83:0/410670034 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3fb01033a0 0x7f3fb010b890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:32:49.954 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:49.952+0000 7f3fb8000640 1 --2- 172.21.15.83:0/410670034 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3fb0102a80 0x7f3fb0102e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:32:49.955 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:49.952+0000 7f3fb8000640 1 -- 172.21.15.83:0/410670034 >> 172.21.15.83:0/410670034 conn(0x7f3fb00fc500 msgr2=0x7f3fb00fe920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:32:49.955 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:49.952+0000 7f3fb8000640 1 -- 172.21.15.83:0/410670034 shutdown_connections 2024-08-19T01:32:49.955 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:49.952+0000 7f3fb8000640 1 -- 172.21.15.83:0/410670034 wait complete. 2024-08-19T01:32:49.956 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:49.953+0000 7f3fb8000640 1 Processor -- start 2024-08-19T01:32:49.957 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:49.953+0000 7f3fb8000640 1 -- start start 2024-08-19T01:32:49.957 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:49.953+0000 7f3fb8000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3fb0102a80 0x7f3fb019a450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:32:49.957 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:49.954+0000 7f3fb8000640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3fb01033a0 0x7f3fb019a990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:32:49.957 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:49.954+0000 7f3fb8000640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3fb010beb0 0x7f3fb019ed20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:32:49.957 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:49.954+0000 7f3fb8000640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f3fb0111cb0 con 0x7f3fb0102a80 2024-08-19T01:32:49.957 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:49.954+0000 7f3fb8000640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f3fb0111b30 con 0x7f3fb010beb0 2024-08-19T01:32:49.957 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:49.954+0000 7f3fb8000640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f3fb0111e30 con 0x7f3fb01033a0 2024-08-19T01:32:49.957 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:49.954+0000 7f3fb5600640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3fb0102a80 0x7f3fb019a450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:32:49.957 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:49.954+0000 7f3fb5600640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3fb0102a80 0x7f3fb019a450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:60748/0 (socket says 172.21.15.83:60748) 2024-08-19T01:32:49.957 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:49.954+0000 7f3fb5600640 1 -- 172.21.15.83:0/2522153480 learned_addr learned my addr 172.21.15.83:0/2522153480 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-08-19T01:32:49.957 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:49.954+0000 7f3fb4c00640 1 --2- 172.21.15.83:0/2522153480 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3fb01033a0 0x7f3fb019a990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:32:49.957 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:49.954+0000 7f3fb6000640 1 --2- 172.21.15.83:0/2522153480 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3fb010beb0 0x7f3fb019ed20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:32:49.958 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:49.954+0000 7f3fb5600640 1 -- 172.21.15.83:0/2522153480 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3fb01033a0 msgr2=0x7f3fb019a990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:32:49.958 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:49.954+0000 7f3fb5600640 1 --2- 172.21.15.83:0/2522153480 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3fb01033a0 0x7f3fb019a990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:32:49.958 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:49.954+0000 7f3fb5600640 1 -- 172.21.15.83:0/2522153480 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3fb010beb0 msgr2=0x7f3fb019ed20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:32:49.958 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:49.954+0000 7f3fb5600640 1 --2- 172.21.15.83:0/2522153480 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3fb010beb0 0x7f3fb019ed20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:32:49.958 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:49.954+0000 7f3fb5600640 1 -- 172.21.15.83:0/2522153480 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3fb019f400 con 0x7f3fb0102a80 2024-08-19T01:32:49.958 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:49.955+0000 7f3fb5600640 1 --2- 172.21.15.83:0/2522153480 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3fb0102a80 0x7f3fb019a450 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f3fa400e9f0 tx=0x7f3fa400edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:32:49.958 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:49.955+0000 7f3fae000640 1 -- 172.21.15.83:0/2522153480 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3fa400cd80 con 0x7f3fb0102a80 2024-08-19T01:32:49.958 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:49.955+0000 7f3fae000640 1 -- 172.21.15.83:0/2522153480 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3fa4005500 con 0x7f3fb0102a80 2024-08-19T01:32:49.958 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:49.955+0000 7f3fb8000640 1 -- 172.21.15.83:0/2522153480 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3fb019f6f0 con 0x7f3fb0102a80 2024-08-19T01:32:49.958 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:49.955+0000 7f3fae000640 1 -- 172.21.15.83:0/2522153480 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3fa4010690 con 0x7f3fb0102a80 2024-08-19T01:32:49.959 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:49.955+0000 7f3fb8000640 1 -- 172.21.15.83:0/2522153480 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3fb01a6fd0 con 0x7f3fb0102a80 2024-08-19T01:32:49.959 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:49.956+0000 7f3fb8000640 1 -- 172.21.15.83:0/2522153480 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3fb0102e60 con 0x7f3fb0102a80 2024-08-19T01:32:49.960 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:49.958+0000 7f3fae000640 1 -- 172.21.15.83:0/2522153480 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f3fa40046c0 con 0x7f3fb0102a80 2024-08-19T01:32:49.961 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:49.959+0000 7f3fae000640 1 --2- 172.21.15.83:0/2522153480 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f3f980a2c00 0x7f3f980a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:32:49.961 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:49.960+0000 7f3fae000640 1 -- 172.21.15.83:0/2522153480 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 7979+0+0 (secure 0 0 0) 0x7f3fa40cf4f0 con 0x7f3fb0102a80 2024-08-19T01:32:49.961 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:49.960+0000 7f3fb4c00640 1 --2- 172.21.15.83:0/2522153480 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f3f980a2c00 0x7f3f980a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:32:49.962 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:49.960+0000 7f3fb4c00640 1 --2- 172.21.15.83:0/2522153480 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f3f980a2c00 0x7f3f980a50c0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f3f9c0096c0 tx=0x7f3f9c009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:32:49.963 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:49.962+0000 7f3fae000640 1 -- 172.21.15.83:0/2522153480 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7f3fa4053df0 con 0x7f3fb0102a80 2024-08-19T01:32:50.175 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:50.174+0000 7f3fb8000640 1 -- 172.21.15.83:0/2522153480 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"} v 0) -- 0x7f3fb019b7b0 con 0x7f3fb0102a80 2024-08-19T01:32:50.264 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:50.262+0000 7f3fae000640 1 -- 172.21.15.83:0/2522153480 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]=0 v18) ==== 105+0+0 (secure 0 0 0) 0x7f3fa409c500 con 0x7f3fb0102a80 2024-08-19T01:32:50.266 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:50.265+0000 7f3fb8000640 1 -- 172.21.15.83:0/2522153480 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f3f980a2c00 msgr2=0x7f3f980a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:32:50.266 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:50.265+0000 7f3fb8000640 1 --2- 172.21.15.83:0/2522153480 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f3f980a2c00 0x7f3f980a50c0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f3f9c0096c0 tx=0x7f3f9c009260 comp rx=0 tx=0).stop 2024-08-19T01:32:50.267 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:50.265+0000 7f3fb8000640 1 -- 172.21.15.83:0/2522153480 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3fb0102a80 msgr2=0x7f3fb019a450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:32:50.267 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:50.265+0000 7f3fb8000640 1 --2- 172.21.15.83:0/2522153480 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3fb0102a80 0x7f3fb019a450 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f3fa400e9f0 tx=0x7f3fa400edf0 comp rx=0 tx=0).stop 2024-08-19T01:32:50.267 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:50.265+0000 7f3fb8000640 1 -- 172.21.15.83:0/2522153480 shutdown_connections 2024-08-19T01:32:50.267 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:50.265+0000 7f3fb8000640 1 --2- 172.21.15.83:0/2522153480 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f3f980a2c00 0x7f3f980a50c0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:32:50.267 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:50.265+0000 7f3fb8000640 1 --2- 172.21.15.83:0/2522153480 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3fb010beb0 0x7f3fb019ed20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:32:50.267 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:50.265+0000 7f3fb8000640 1 --2- 172.21.15.83:0/2522153480 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3fb01033a0 0x7f3fb019a990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:32:50.267 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:50.265+0000 7f3fb8000640 1 --2- 172.21.15.83:0/2522153480 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3fb0102a80 0x7f3fb019a450 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:32:50.268 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:50.265+0000 7f3fb8000640 1 -- 172.21.15.83:0/2522153480 >> 172.21.15.83:0/2522153480 conn(0x7f3fb00fc500 msgr2=0x7f3fb010d120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:32:50.268 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:50.265+0000 7f3fb8000640 1 -- 172.21.15.83:0/2522153480 shutdown_connections 2024-08-19T01:32:50.268 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:50.265+0000 7f3fb8000640 1 -- 172.21.15.83:0/2522153480 wait complete. 2024-08-19T01:32:50.629 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:32:50 smithi083 ceph-mon[17120]: from='client.? 172.21.15.83:0/2522153480' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]: dispatch 2024-08-19T01:32:50.702 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:32:50 smithi117 ceph-mon[22016]: from='client.? 172.21.15.83:0/2522153480' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]: dispatch 2024-08-19T01:32:50.716 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:32:50 smithi167 ceph-mon[21875]: from='client.? 172.21.15.83:0/2522153480' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]: dispatch 2024-08-19T01:32:50.996 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833 shell --fsid bad8780e-5dc9-11ef-bccf-c7b262605968 -- ceph fs set cephfs allow_standby_replay true 2024-08-19T01:32:51.378 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/bad8780e-5dc9-11ef-bccf-c7b262605968/mon.a/config 2024-08-19T01:32:51.519 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:32:51 smithi083 ceph-mon[17120]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-08-19T01:32:51.519 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:32:51 smithi083 ceph-mon[17120]: from='client.? 172.21.15.83:0/2522153480' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]': finished 2024-08-19T01:32:51.520 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:32:51 smithi083 ceph-mon[17120]: daemon mds.g assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-08-19T01:32:51.520 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:32:51 smithi083 ceph-mon[17120]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-08-19T01:32:51.520 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:32:51 smithi083 ceph-mon[17120]: fsmap cephfs:2 {0=b=up:active,1=g=up:creating} 10 up:standby 2024-08-19T01:32:51.520 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:32:51 smithi083 ceph-mon[17120]: daemon mds.g is now active in filesystem cephfs as rank 1 2024-08-19T01:32:51.520 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:32:51 smithi083 ceph-mon[17120]: pgmap v277: 97 pgs: 97 active+clean; 579 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 KiB/s wr, 4 op/s 2024-08-19T01:32:51.702 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:32:51 smithi117 ceph-mon[22016]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-08-19T01:32:51.702 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:32:51 smithi117 ceph-mon[22016]: from='client.? 172.21.15.83:0/2522153480' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]': finished 2024-08-19T01:32:51.703 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:32:51 smithi117 ceph-mon[22016]: daemon mds.g assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-08-19T01:32:51.703 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:32:51 smithi117 ceph-mon[22016]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-08-19T01:32:51.703 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:32:51 smithi117 ceph-mon[22016]: fsmap cephfs:2 {0=b=up:active,1=g=up:creating} 10 up:standby 2024-08-19T01:32:51.703 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:32:51 smithi117 ceph-mon[22016]: daemon mds.g is now active in filesystem cephfs as rank 1 2024-08-19T01:32:51.703 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:32:51 smithi117 ceph-mon[22016]: pgmap v277: 97 pgs: 97 active+clean; 579 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 KiB/s wr, 4 op/s 2024-08-19T01:32:51.716 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:32:51 smithi167 ceph-mon[21875]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-08-19T01:32:51.716 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:32:51 smithi167 ceph-mon[21875]: from='client.? 172.21.15.83:0/2522153480' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]': finished 2024-08-19T01:32:51.716 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:32:51 smithi167 ceph-mon[21875]: daemon mds.g assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-08-19T01:32:51.716 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:32:51 smithi167 ceph-mon[21875]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-08-19T01:32:51.716 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:32:51 smithi167 ceph-mon[21875]: fsmap cephfs:2 {0=b=up:active,1=g=up:creating} 10 up:standby 2024-08-19T01:32:51.716 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:32:51 smithi167 ceph-mon[21875]: daemon mds.g is now active in filesystem cephfs as rank 1 2024-08-19T01:32:51.716 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:32:51 smithi167 ceph-mon[21875]: pgmap v277: 97 pgs: 97 active+clean; 579 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 KiB/s wr, 4 op/s 2024-08-19T01:32:51.945 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:51.943+0000 7fd085400640 1 -- 172.21.15.83:0/939328023 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd080104480 msgr2=0x7fd0801109b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:32:51.945 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:51.943+0000 7fd085400640 1 --2- 172.21.15.83:0/939328023 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd080104480 0x7fd0801109b0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fd078009930 tx=0x7fd07802a420 comp rx=0 tx=0).stop 2024-08-19T01:32:51.945 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:51.943+0000 7fd085400640 1 -- 172.21.15.83:0/939328023 shutdown_connections 2024-08-19T01:32:51.945 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:51.943+0000 7fd085400640 1 --2- 172.21.15.83:0/939328023 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd080104480 0x7fd0801109b0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:32:51.945 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:51.943+0000 7fd085400640 1 --2- 172.21.15.83:0/939328023 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd080103ae0 0x7fd080103f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:32:51.945 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:51.943+0000 7fd085400640 1 --2- 172.21.15.83:0/939328023 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd080109b70 0x7fd080109f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:32:51.945 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:51.943+0000 7fd085400640 1 -- 172.21.15.83:0/939328023 >> 172.21.15.83:0/939328023 conn(0x7fd0800ff780 msgr2=0x7fd080101ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:32:51.945 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:51.943+0000 7fd085400640 1 -- 172.21.15.83:0/939328023 shutdown_connections 2024-08-19T01:32:51.946 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:51.943+0000 7fd085400640 1 -- 172.21.15.83:0/939328023 wait complete. 2024-08-19T01:32:51.947 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:51.944+0000 7fd085400640 1 Processor -- start 2024-08-19T01:32:51.947 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:51.944+0000 7fd085400640 1 -- start start 2024-08-19T01:32:51.947 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:51.944+0000 7fd085400640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd080103ae0 0x7fd08019a370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:32:51.947 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:51.944+0000 7fd085400640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd080104480 0x7fd08019a8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:32:51.947 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:51.944+0000 7fd085400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd080109b70 0x7fd08019ec40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:32:51.947 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:51.945+0000 7fd085400640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fd080113f30 con 0x7fd080109b70 2024-08-19T01:32:51.948 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:51.945+0000 7fd085400640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fd080113db0 con 0x7fd080104480 2024-08-19T01:32:51.948 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:51.945+0000 7fd085400640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fd0801140b0 con 0x7fd080103ae0 2024-08-19T01:32:51.948 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:51.945+0000 7fd07e800640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd080103ae0 0x7fd08019a370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:32:51.948 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:51.945+0000 7fd07e800640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd080103ae0 0x7fd08019a370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.83:33786/0 (socket says 172.21.15.83:33786) 2024-08-19T01:32:51.948 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:51.945+0000 7fd07de00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd080104480 0x7fd08019a8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:32:51.948 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:51.945+0000 7fd07e800640 1 -- 172.21.15.83:0/3624333860 learned_addr learned my addr 172.21.15.83:0/3624333860 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-08-19T01:32:51.948 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:51.945+0000 7fd07f200640 1 --2- 172.21.15.83:0/3624333860 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd080109b70 0x7fd08019ec40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:32:51.948 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:51.945+0000 7fd07e800640 1 -- 172.21.15.83:0/3624333860 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd080104480 msgr2=0x7fd08019a8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:32:51.948 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:51.945+0000 7fd07e800640 1 --2- 172.21.15.83:0/3624333860 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd080104480 0x7fd08019a8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:32:51.948 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:51.945+0000 7fd07e800640 1 -- 172.21.15.83:0/3624333860 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd080109b70 msgr2=0x7fd08019ec40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:32:51.948 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:51.945+0000 7fd07e800640 1 --2- 172.21.15.83:0/3624333860 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd080109b70 0x7fd08019ec40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:32:51.948 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:51.945+0000 7fd07e800640 1 -- 172.21.15.83:0/3624333860 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd08019f320 con 0x7fd080103ae0 2024-08-19T01:32:51.948 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:51.945+0000 7fd07de00640 1 --2- 172.21.15.83:0/3624333860 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd080104480 0x7fd08019a8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-19T01:32:51.948 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:51.946+0000 7fd07e800640 1 --2- 172.21.15.83:0/3624333860 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd080103ae0 0x7fd08019a370 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fd070009d50 tx=0x7fd07000c6a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:32:51.949 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:51.946+0000 7fd077400640 1 -- 172.21.15.83:0/3624333860 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd070003b70 con 0x7fd080103ae0 2024-08-19T01:32:51.949 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:51.946+0000 7fd085400640 1 -- 172.21.15.83:0/3624333860 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd08019f610 con 0x7fd080103ae0 2024-08-19T01:32:51.949 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:51.946+0000 7fd077400640 1 -- 172.21.15.83:0/3624333860 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd070003d10 con 0x7fd080103ae0 2024-08-19T01:32:51.949 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:51.946+0000 7fd077400640 1 -- 172.21.15.83:0/3624333860 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd070010a20 con 0x7fd080103ae0 2024-08-19T01:32:51.949 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:51.946+0000 7fd085400640 1 -- 172.21.15.83:0/3624333860 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd0801a6ef0 con 0x7fd080103ae0 2024-08-19T01:32:51.950 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:51.947+0000 7fd085400640 1 -- 172.21.15.83:0/3624333860 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd080103f40 con 0x7fd080103ae0 2024-08-19T01:32:51.951 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:51.949+0000 7fd077400640 1 -- 172.21.15.83:0/3624333860 <== mon.1 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7fd070010bc0 con 0x7fd080103ae0 2024-08-19T01:32:51.952 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:51.950+0000 7fd077400640 1 --2- 172.21.15.83:0/3624333860 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7fd0640a2b80 0x7fd0640a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:32:51.952 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:51.951+0000 7fd077400640 1 -- 172.21.15.83:0/3624333860 <== mon.1 v2:172.21.15.167:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 7979+0+0 (secure 0 0 0) 0x7fd0700cf660 con 0x7fd080103ae0 2024-08-19T01:32:51.952 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:51.951+0000 7fd07de00640 1 --2- 172.21.15.83:0/3624333860 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7fd0640a2b80 0x7fd0640a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:32:51.953 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:51.951+0000 7fd07de00640 1 --2- 172.21.15.83:0/3624333860 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7fd0640a2b80 0x7fd0640a5040 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fd080101c80 tx=0x7fd06c005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:32:51.954 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:51.952+0000 7fd077400640 1 -- 172.21.15.83:0/3624333860 <== mon.1 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7fd07009c700 con 0x7fd080103ae0 2024-08-19T01:32:52.173 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:52.171+0000 7fd085400640 1 -- 172.21.15.83:0/3624333860 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"} v 0) -- 0x7fd0801a7190 con 0x7fd080103ae0 2024-08-19T01:32:52.288 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:52.286+0000 7fd077400640 1 -- 172.21.15.83:0/3624333860 <== mon.1 v2:172.21.15.167:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]=0 v22) ==== 121+0+0 (secure 0 0 0) 0x7fd07009c000 con 0x7fd080103ae0 2024-08-19T01:32:52.290 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:52.288+0000 7fd085400640 1 -- 172.21.15.83:0/3624333860 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7fd0640a2b80 msgr2=0x7fd0640a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:32:52.290 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:52.288+0000 7fd085400640 1 --2- 172.21.15.83:0/3624333860 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7fd0640a2b80 0x7fd0640a5040 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fd080101c80 tx=0x7fd06c005f90 comp rx=0 tx=0).stop 2024-08-19T01:32:52.290 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:52.288+0000 7fd085400640 1 -- 172.21.15.83:0/3624333860 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd080103ae0 msgr2=0x7fd08019a370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:32:52.290 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:52.288+0000 7fd085400640 1 --2- 172.21.15.83:0/3624333860 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd080103ae0 0x7fd08019a370 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fd070009d50 tx=0x7fd07000c6a0 comp rx=0 tx=0).stop 2024-08-19T01:32:52.291 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:52.288+0000 7fd085400640 1 -- 172.21.15.83:0/3624333860 shutdown_connections 2024-08-19T01:32:52.291 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:52.288+0000 7fd085400640 1 --2- 172.21.15.83:0/3624333860 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7fd0640a2b80 0x7fd0640a5040 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:32:52.291 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:52.288+0000 7fd085400640 1 --2- 172.21.15.83:0/3624333860 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd080109b70 0x7fd08019ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:32:52.291 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:52.288+0000 7fd085400640 1 --2- 172.21.15.83:0/3624333860 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd080104480 0x7fd08019a8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:32:52.291 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:52.288+0000 7fd085400640 1 --2- 172.21.15.83:0/3624333860 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd080103ae0 0x7fd08019a370 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:32:52.291 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:52.288+0000 7fd085400640 1 -- 172.21.15.83:0/3624333860 >> 172.21.15.83:0/3624333860 conn(0x7fd0800ff780 msgr2=0x7fd0800ffeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:32:52.291 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:52.289+0000 7fd085400640 1 -- 172.21.15.83:0/3624333860 shutdown_connections 2024-08-19T01:32:52.291 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:52.289+0000 7fd085400640 1 -- 172.21.15.83:0/3624333860 wait complete. 2024-08-19T01:32:52.655 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:32:52 smithi083 ceph-mon[17120]: mds.? [v2:172.21.15.83:6838/3239261845,v1:172.21.15.83:6839/3239261845] up:active 2024-08-19T01:32:52.655 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:32:52 smithi083 ceph-mon[17120]: daemon mds.h assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-08-19T01:32:52.655 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:32:52 smithi083 ceph-mon[17120]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-08-19T01:32:52.655 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:32:52 smithi083 ceph-mon[17120]: Cluster is now healthy 2024-08-19T01:32:52.655 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:32:52 smithi083 ceph-mon[17120]: fsmap cephfs:2 {0=b=up:active,1=g=up:active} 10 up:standby 2024-08-19T01:32:52.655 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:32:52 smithi083 ceph-mon[17120]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=h=up:creating} 9 up:standby 2024-08-19T01:32:52.655 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:32:52 smithi083 ceph-mon[17120]: daemon mds.h is now active in filesystem cephfs as rank 2 2024-08-19T01:32:52.655 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:32:52 smithi083 ceph-mon[17120]: from='client.? 172.21.15.83:0/3624333860' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-08-19T01:32:52.655 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:32:52 smithi083 ceph-mon[17120]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-08-19T01:32:52.702 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:32:52 smithi117 ceph-mon[22016]: mds.? [v2:172.21.15.83:6838/3239261845,v1:172.21.15.83:6839/3239261845] up:active 2024-08-19T01:32:52.702 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:32:52 smithi117 ceph-mon[22016]: daemon mds.h assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-08-19T01:32:52.702 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:32:52 smithi117 ceph-mon[22016]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-08-19T01:32:52.702 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:32:52 smithi117 ceph-mon[22016]: Cluster is now healthy 2024-08-19T01:32:52.703 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:32:52 smithi117 ceph-mon[22016]: fsmap cephfs:2 {0=b=up:active,1=g=up:active} 10 up:standby 2024-08-19T01:32:52.703 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:32:52 smithi117 ceph-mon[22016]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=h=up:creating} 9 up:standby 2024-08-19T01:32:52.703 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:32:52 smithi117 ceph-mon[22016]: daemon mds.h is now active in filesystem cephfs as rank 2 2024-08-19T01:32:52.703 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:32:52 smithi117 ceph-mon[22016]: from='client.? 172.21.15.83:0/3624333860' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-08-19T01:32:52.703 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:32:52 smithi117 ceph-mon[22016]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-08-19T01:32:52.716 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:32:52 smithi167 ceph-mon[21875]: mds.? [v2:172.21.15.83:6838/3239261845,v1:172.21.15.83:6839/3239261845] up:active 2024-08-19T01:32:52.716 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:32:52 smithi167 ceph-mon[21875]: daemon mds.h assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-08-19T01:32:52.716 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:32:52 smithi167 ceph-mon[21875]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-08-19T01:32:52.716 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:32:52 smithi167 ceph-mon[21875]: Cluster is now healthy 2024-08-19T01:32:52.716 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:32:52 smithi167 ceph-mon[21875]: fsmap cephfs:2 {0=b=up:active,1=g=up:active} 10 up:standby 2024-08-19T01:32:52.716 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:32:52 smithi167 ceph-mon[21875]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=h=up:creating} 9 up:standby 2024-08-19T01:32:52.716 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:32:52 smithi167 ceph-mon[21875]: daemon mds.h is now active in filesystem cephfs as rank 2 2024-08-19T01:32:52.716 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:32:52 smithi167 ceph-mon[21875]: from='client.? 172.21.15.83:0/3624333860' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-08-19T01:32:52.717 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:32:52 smithi167 ceph-mon[21875]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-08-19T01:32:53.091 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833 shell --fsid bad8780e-5dc9-11ef-bccf-c7b262605968 -- ceph fs set cephfs session_timeout 300 2024-08-19T01:32:53.404 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:32:53 smithi083 ceph-mon[17120]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-08-19T01:32:53.405 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:32:53 smithi083 ceph-mon[17120]: mds.? [v2:172.21.15.117:6836/3185224684,v1:172.21.15.117:6837/3185224684] up:active 2024-08-19T01:32:53.405 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:32:53 smithi083 ceph-mon[17120]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=h=up:active} 9 up:standby 2024-08-19T01:32:53.405 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:32:53 smithi083 ceph-mon[17120]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-08-19T01:32:53.405 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:32:53 smithi083 ceph-mon[17120]: pgmap v278: 97 pgs: 97 active+clean; 579 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s wr, 5 op/s 2024-08-19T01:32:53.468 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/bad8780e-5dc9-11ef-bccf-c7b262605968/mon.a/config 2024-08-19T01:32:53.702 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:32:53 smithi117 ceph-mon[22016]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-08-19T01:32:53.702 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:32:53 smithi117 ceph-mon[22016]: mds.? [v2:172.21.15.117:6836/3185224684,v1:172.21.15.117:6837/3185224684] up:active 2024-08-19T01:32:53.703 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:32:53 smithi117 ceph-mon[22016]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=h=up:active} 9 up:standby 2024-08-19T01:32:53.703 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:32:53 smithi117 ceph-mon[22016]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-08-19T01:32:53.703 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:32:53 smithi117 ceph-mon[22016]: pgmap v278: 97 pgs: 97 active+clean; 579 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s wr, 5 op/s 2024-08-19T01:32:53.716 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:32:53 smithi167 ceph-mon[21875]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-08-19T01:32:53.716 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:32:53 smithi167 ceph-mon[21875]: mds.? [v2:172.21.15.117:6836/3185224684,v1:172.21.15.117:6837/3185224684] up:active 2024-08-19T01:32:53.716 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:32:53 smithi167 ceph-mon[21875]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=h=up:active} 9 up:standby 2024-08-19T01:32:53.716 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:32:53 smithi167 ceph-mon[21875]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-08-19T01:32:53.716 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:32:53 smithi167 ceph-mon[21875]: pgmap v278: 97 pgs: 97 active+clean; 579 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s wr, 5 op/s 2024-08-19T01:32:53.987 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:53.985+0000 7fefdb000640 1 -- 172.21.15.83:0/2618737973 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fefd4102a60 msgr2=0x7fefd4102e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:32:53.987 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:53.985+0000 7fefdb000640 1 --2- 172.21.15.83:0/2618737973 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fefd4102a60 0x7fefd4102e40 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fefc4009900 tx=0x7fefc402a480 comp rx=0 tx=0).stop 2024-08-19T01:32:53.987 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:53.985+0000 7fefdb000640 1 -- 172.21.15.83:0/2618737973 shutdown_connections 2024-08-19T01:32:53.987 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:53.985+0000 7fefdb000640 1 --2- 172.21.15.83:0/2618737973 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fefd410be90 0x7fefd410e330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:32:53.987 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:53.985+0000 7fefdb000640 1 --2- 172.21.15.83:0/2618737973 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fefd4103380 0x7fefd410b870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:32:53.987 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:53.985+0000 7fefdb000640 1 --2- 172.21.15.83:0/2618737973 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fefd4102a60 0x7fefd4102e40 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:32:53.987 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:53.985+0000 7fefdb000640 1 -- 172.21.15.83:0/2618737973 >> 172.21.15.83:0/2618737973 conn(0x7fefd40fc500 msgr2=0x7fefd40fe920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:32:53.987 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:53.985+0000 7fefdb000640 1 -- 172.21.15.83:0/2618737973 shutdown_connections 2024-08-19T01:32:53.988 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:53.985+0000 7fefdb000640 1 -- 172.21.15.83:0/2618737973 wait complete. 2024-08-19T01:32:53.989 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:53.986+0000 7fefdb000640 1 Processor -- start 2024-08-19T01:32:53.989 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:53.986+0000 7fefdb000640 1 -- start start 2024-08-19T01:32:53.990 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:53.986+0000 7fefdb000640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fefd4102a60 0x7fefd419a380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:32:53.990 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:53.986+0000 7fefdb000640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fefd4103380 0x7fefd419a8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:32:53.990 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:53.986+0000 7fefdb000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fefd410be90 0x7fefd419ec50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:32:53.990 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:53.986+0000 7fefdb000640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fefd4111ce0 con 0x7fefd410be90 2024-08-19T01:32:53.990 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:53.986+0000 7fefdb000640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fefd4111b60 con 0x7fefd4103380 2024-08-19T01:32:53.990 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:53.986+0000 7fefdb000640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fefd4111e60 con 0x7fefd4102a60 2024-08-19T01:32:53.990 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:53.987+0000 7fefd9000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fefd410be90 0x7fefd419ec50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:32:53.990 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:53.987+0000 7fefd3e00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fefd4102a60 0x7fefd419a380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:32:53.990 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:53.987+0000 7fefd9000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fefd410be90 0x7fefd419ec50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:39730/0 (socket says 172.21.15.83:39730) 2024-08-19T01:32:53.990 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:53.987+0000 7fefd9000640 1 -- 172.21.15.83:0/3363425959 learned_addr learned my addr 172.21.15.83:0/3363425959 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-08-19T01:32:53.990 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:53.987+0000 7fefd3400640 1 --2- 172.21.15.83:0/3363425959 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fefd4103380 0x7fefd419a8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:32:53.990 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:53.987+0000 7fefd9000640 1 -- 172.21.15.83:0/3363425959 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fefd4102a60 msgr2=0x7fefd419a380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:32:53.991 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:53.987+0000 7fefd9000640 1 --2- 172.21.15.83:0/3363425959 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fefd4102a60 0x7fefd419a380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:32:53.991 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:53.987+0000 7fefd9000640 1 -- 172.21.15.83:0/3363425959 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fefd4103380 msgr2=0x7fefd419a8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:32:53.991 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:53.987+0000 7fefd9000640 1 --2- 172.21.15.83:0/3363425959 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fefd4103380 0x7fefd419a8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:32:53.991 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:53.987+0000 7fefd9000640 1 -- 172.21.15.83:0/3363425959 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fefd419f330 con 0x7fefd410be90 2024-08-19T01:32:53.991 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:53.987+0000 7fefd3400640 1 --2- 172.21.15.83:0/3363425959 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fefd4103380 0x7fefd419a8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-19T01:32:53.991 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:53.987+0000 7fefd9000640 1 --2- 172.21.15.83:0/3363425959 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fefd410be90 0x7fefd419ec50 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fefcc00eb10 tx=0x7fefcc00c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:32:53.991 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:53.988+0000 7fefd0c00640 1 -- 172.21.15.83:0/3363425959 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fefcc004b20 con 0x7fefd410be90 2024-08-19T01:32:53.991 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:53.988+0000 7fefd0c00640 1 -- 172.21.15.83:0/3363425959 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fefcc004cc0 con 0x7fefd410be90 2024-08-19T01:32:53.991 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:53.988+0000 7fefd0c00640 1 -- 172.21.15.83:0/3363425959 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fefcc010760 con 0x7fefd410be90 2024-08-19T01:32:53.991 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:53.988+0000 7fefdb000640 1 -- 172.21.15.83:0/3363425959 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fefd419f620 con 0x7fefd410be90 2024-08-19T01:32:53.991 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:53.988+0000 7fefdb000640 1 -- 172.21.15.83:0/3363425959 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fefd41a6f00 con 0x7fefd410be90 2024-08-19T01:32:53.992 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:53.989+0000 7fefdb000640 1 -- 172.21.15.83:0/3363425959 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fefd4102e40 con 0x7fefd410be90 2024-08-19T01:32:53.992 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:53.991+0000 7fefd0c00640 1 -- 172.21.15.83:0/3363425959 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7fefcc005020 con 0x7fefd410be90 2024-08-19T01:32:53.994 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:53.992+0000 7fefd0c00640 1 --2- 172.21.15.83:0/3363425959 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7fefb80a2c00 0x7fefb80a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:32:53.994 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:53.992+0000 7fefd0c00640 1 -- 172.21.15.83:0/3363425959 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 7979+0+0 (secure 0 0 0) 0x7fefcc0cf540 con 0x7fefd410be90 2024-08-19T01:32:53.994 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:53.992+0000 7fefd3e00640 1 --2- 172.21.15.83:0/3363425959 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7fefb80a2c00 0x7fefb80a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:32:53.995 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:53.993+0000 7fefd3e00640 1 --2- 172.21.15.83:0/3363425959 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7fefb80a2c00 0x7fefb80a50c0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fefc4000c00 tx=0x7fefc4005b50 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:32:53.998 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:53.996+0000 7fefd0c00640 1 -- 172.21.15.83:0/3363425959 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7fefcc09c550 con 0x7fefd410be90 2024-08-19T01:32:54.209 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:54.207+0000 7fefdb000640 1 -- 172.21.15.83:0/3363425959 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"} v 0) -- 0x7fefd41a7270 con 0x7fefd410be90 2024-08-19T01:32:54.299 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:54.297+0000 7fefd0c00640 1 -- 172.21.15.83:0/3363425959 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]=0 v24) ==== 115+0+0 (secure 0 0 0) 0x7fefcc09be50 con 0x7fefd410be90 2024-08-19T01:32:54.301 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:54.299+0000 7fefdb000640 1 -- 172.21.15.83:0/3363425959 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7fefb80a2c00 msgr2=0x7fefb80a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:32:54.302 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:54.299+0000 7fefdb000640 1 --2- 172.21.15.83:0/3363425959 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7fefb80a2c00 0x7fefb80a50c0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fefc4000c00 tx=0x7fefc4005b50 comp rx=0 tx=0).stop 2024-08-19T01:32:54.302 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:54.299+0000 7fefdb000640 1 -- 172.21.15.83:0/3363425959 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fefd410be90 msgr2=0x7fefd419ec50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:32:54.302 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:54.299+0000 7fefdb000640 1 --2- 172.21.15.83:0/3363425959 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fefd410be90 0x7fefd419ec50 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fefcc00eb10 tx=0x7fefcc00c2d0 comp rx=0 tx=0).stop 2024-08-19T01:32:54.302 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:54.300+0000 7fefdb000640 1 -- 172.21.15.83:0/3363425959 shutdown_connections 2024-08-19T01:32:54.302 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:54.300+0000 7fefdb000640 1 --2- 172.21.15.83:0/3363425959 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7fefb80a2c00 0x7fefb80a50c0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:32:54.302 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:54.300+0000 7fefdb000640 1 --2- 172.21.15.83:0/3363425959 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fefd410be90 0x7fefd419ec50 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:32:54.302 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:54.300+0000 7fefdb000640 1 --2- 172.21.15.83:0/3363425959 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fefd4103380 0x7fefd419a8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:32:54.302 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:54.300+0000 7fefdb000640 1 --2- 172.21.15.83:0/3363425959 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fefd4102a60 0x7fefd419a380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:32:54.302 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:54.300+0000 7fefdb000640 1 -- 172.21.15.83:0/3363425959 >> 172.21.15.83:0/3363425959 conn(0x7fefd40fc500 msgr2=0x7fefd40fd100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:32:54.302 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:54.300+0000 7fefdb000640 1 -- 172.21.15.83:0/3363425959 shutdown_connections 2024-08-19T01:32:54.302 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:54.300+0000 7fefdb000640 1 -- 172.21.15.83:0/3363425959 wait complete. 2024-08-19T01:32:54.654 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:32:54 smithi083 ceph-mon[17120]: from='client.? 172.21.15.83:0/3363425959' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-08-19T01:32:54.702 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:32:54 smithi117 ceph-mon[22016]: from='client.? 172.21.15.83:0/3363425959' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-08-19T01:32:54.716 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:32:54 smithi167 ceph-mon[21875]: from='client.? 172.21.15.83:0/3363425959' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-08-19T01:32:55.041 DEBUG:tasks.cephfs.filesystem:Creating {'create': 2, 'subvol_options': '--size 25000000000'} subvols for filesystem 'cephfs' 2024-08-19T01:32:55.041 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833 shell --fsid bad8780e-5dc9-11ef-bccf-c7b262605968 -- ceph fs subvolumegroup create cephfs qa 2024-08-19T01:32:55.416 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/bad8780e-5dc9-11ef-bccf-c7b262605968/mon.a/config 2024-08-19T01:32:55.556 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:32:55 smithi083 ceph-mon[17120]: from='client.? 172.21.15.83:0/3363425959' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-08-19T01:32:55.556 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:32:55 smithi083 ceph-mon[17120]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-08-19T01:32:55.556 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:32:55 smithi083 ceph-mon[17120]: pgmap v279: 97 pgs: 97 active+clean; 581 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2.9 KiB/s wr, 9 op/s 2024-08-19T01:32:55.702 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:32:55 smithi117 ceph-mon[22016]: from='client.? 172.21.15.83:0/3363425959' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-08-19T01:32:55.703 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:32:55 smithi117 ceph-mon[22016]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-08-19T01:32:55.703 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:32:55 smithi117 ceph-mon[22016]: pgmap v279: 97 pgs: 97 active+clean; 581 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2.9 KiB/s wr, 9 op/s 2024-08-19T01:32:55.716 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:32:55 smithi167 ceph-mon[21875]: from='client.? 172.21.15.83:0/3363425959' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-08-19T01:32:55.716 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:32:55 smithi167 ceph-mon[21875]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-08-19T01:32:55.716 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:32:55 smithi167 ceph-mon[21875]: pgmap v279: 97 pgs: 97 active+clean; 581 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2.9 KiB/s wr, 9 op/s 2024-08-19T01:32:55.961 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:55.959+0000 7f6416800640 1 -- 172.21.15.83:0/3458623245 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6410103ac0 msgr2=0x7f6410103f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:32:55.961 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:55.959+0000 7f6416800640 1 --2- 172.21.15.83:0/3458623245 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6410103ac0 0x7f6410103f20 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f63fc009930 tx=0x7f63fc02a420 comp rx=0 tx=0).stop 2024-08-19T01:32:55.961 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:55.959+0000 7f6416800640 1 -- 172.21.15.83:0/3458623245 shutdown_connections 2024-08-19T01:32:55.961 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:55.959+0000 7f6416800640 1 --2- 172.21.15.83:0/3458623245 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6410104460 0x7f6410110990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:32:55.961 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:55.959+0000 7f6416800640 1 --2- 172.21.15.83:0/3458623245 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6410103ac0 0x7f6410103f20 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:32:55.961 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:55.959+0000 7f6416800640 1 --2- 172.21.15.83:0/3458623245 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6410109ac0 0x7f6410109ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:32:55.961 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:55.959+0000 7f6416800640 1 -- 172.21.15.83:0/3458623245 >> 172.21.15.83:0/3458623245 conn(0x7f64100ff7c0 msgr2=0x7f6410101be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:32:55.962 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:55.959+0000 7f6416800640 1 -- 172.21.15.83:0/3458623245 shutdown_connections 2024-08-19T01:32:55.962 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:55.959+0000 7f6416800640 1 -- 172.21.15.83:0/3458623245 wait complete. 2024-08-19T01:32:55.963 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:55.960+0000 7f6416800640 1 Processor -- start 2024-08-19T01:32:55.963 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:55.960+0000 7f6416800640 1 -- start start 2024-08-19T01:32:55.963 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:55.960+0000 7f6416800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6410103ac0 0x7f641019a3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:32:55.963 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:55.960+0000 7f6416800640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6410104460 0x7f641019a8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:32:55.963 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:55.961+0000 7f6416800640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6410109ac0 0x7f641019ec70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:32:55.963 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:55.961+0000 7f6416800640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f6410113ec0 con 0x7f6410103ac0 2024-08-19T01:32:55.963 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:55.961+0000 7f6416800640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f6410113d40 con 0x7f6410109ac0 2024-08-19T01:32:55.964 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:55.961+0000 7f6416800640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f6410114040 con 0x7f6410104460 2024-08-19T01:32:55.964 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:55.961+0000 7f640f400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6410103ac0 0x7f641019a3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:32:55.964 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:55.961+0000 7f640f400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6410103ac0 0x7f641019a3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:39750/0 (socket says 172.21.15.83:39750) 2024-08-19T01:32:55.964 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:55.961+0000 7f640f400640 1 -- 172.21.15.83:0/1144396185 learned_addr learned my addr 172.21.15.83:0/1144396185 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-08-19T01:32:55.964 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:55.961+0000 7f640fe00640 1 --2- 172.21.15.83:0/1144396185 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6410109ac0 0x7f641019ec70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:32:55.964 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:55.961+0000 7f640ea00640 1 --2- 172.21.15.83:0/1144396185 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6410104460 0x7f641019a8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:32:55.964 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:55.961+0000 7f640f400640 1 -- 172.21.15.83:0/1144396185 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6410104460 msgr2=0x7f641019a8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:32:55.964 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:55.961+0000 7f640f400640 1 --2- 172.21.15.83:0/1144396185 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6410104460 0x7f641019a8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:32:55.964 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:55.961+0000 7f640f400640 1 -- 172.21.15.83:0/1144396185 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6410109ac0 msgr2=0x7f641019ec70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:32:55.964 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:55.961+0000 7f640f400640 1 --2- 172.21.15.83:0/1144396185 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6410109ac0 0x7f641019ec70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:32:55.964 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:55.961+0000 7f640f400640 1 -- 172.21.15.83:0/1144396185 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f641019f350 con 0x7f6410103ac0 2024-08-19T01:32:55.964 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:55.962+0000 7f640f400640 1 --2- 172.21.15.83:0/1144396185 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6410103ac0 0x7f641019a3a0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f640000e9f0 tx=0x7f640000edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:32:55.965 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:55.962+0000 7f6407e00640 1 -- 172.21.15.83:0/1144396185 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f640000cd80 con 0x7f6410103ac0 2024-08-19T01:32:55.965 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:55.962+0000 7f6416800640 1 -- 172.21.15.83:0/1144396185 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f641019f640 con 0x7f6410103ac0 2024-08-19T01:32:55.965 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:55.962+0000 7f6407e00640 1 -- 172.21.15.83:0/1144396185 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6400005500 con 0x7f6410103ac0 2024-08-19T01:32:55.965 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:55.962+0000 7f6407e00640 1 -- 172.21.15.83:0/1144396185 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6400010690 con 0x7f6410103ac0 2024-08-19T01:32:55.965 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:55.962+0000 7f6416800640 1 -- 172.21.15.83:0/1144396185 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f64101a6f20 con 0x7f6410103ac0 2024-08-19T01:32:55.966 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:55.963+0000 7f6416800640 1 -- 172.21.15.83:0/1144396185 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6410103f20 con 0x7f6410103ac0 2024-08-19T01:32:55.966 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:55.965+0000 7f6407e00640 1 -- 172.21.15.83:0/1144396185 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f64000046c0 con 0x7f6410103ac0 2024-08-19T01:32:55.968 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:55.966+0000 7f6407e00640 1 --2- 172.21.15.83:0/1144396185 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f63f40a2b80 0x7f63f40a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:32:55.968 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:55.966+0000 7f6407e00640 1 -- 172.21.15.83:0/1144396185 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 7979+0+0 (secure 0 0 0) 0x7f64000cf340 con 0x7f6410103ac0 2024-08-19T01:32:55.968 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:55.967+0000 7f640ea00640 1 --2- 172.21.15.83:0/1144396185 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f63f40a2b80 0x7f63f40a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:32:55.969 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:55.967+0000 7f640ea00640 1 --2- 172.21.15.83:0/1144396185 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f63f40a2b80 0x7f63f40a5040 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f63fc009900 tx=0x7f63fc0058d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:32:55.972 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:55.970+0000 7f6407e00640 1 -- 172.21.15.83:0/1144396185 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7f640009c350 con 0x7f6410103ac0 2024-08-19T01:32:56.171 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:56.169+0000 7f6416800640 1 -- 172.21.15.83:0/1144396185 --> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] -- mgr_command(tid 0: {"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7f641010c0a0 con 0x7f63f40a2b80 2024-08-19T01:32:56.204 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:56.202+0000 7f6407e00640 1 -- 172.21.15.83:0/1144396185 <== mgr.14150 v2:172.21.15.83:6800/683740916 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f63fc031630 con 0x7f63f40a2b80 2024-08-19T01:32:56.207 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:56.205+0000 7f6416800640 1 -- 172.21.15.83:0/1144396185 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f63f40a2b80 msgr2=0x7f63f40a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:32:56.207 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:56.205+0000 7f6416800640 1 --2- 172.21.15.83:0/1144396185 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f63f40a2b80 0x7f63f40a5040 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f63fc009900 tx=0x7f63fc0058d0 comp rx=0 tx=0).stop 2024-08-19T01:32:56.207 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:56.205+0000 7f6416800640 1 -- 172.21.15.83:0/1144396185 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6410103ac0 msgr2=0x7f641019a3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:32:56.207 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:56.205+0000 7f6416800640 1 --2- 172.21.15.83:0/1144396185 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6410103ac0 0x7f641019a3a0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f640000e9f0 tx=0x7f640000edf0 comp rx=0 tx=0).stop 2024-08-19T01:32:56.207 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:56.205+0000 7f6416800640 1 -- 172.21.15.83:0/1144396185 shutdown_connections 2024-08-19T01:32:56.207 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:56.205+0000 7f6416800640 1 --2- 172.21.15.83:0/1144396185 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f63f40a2b80 0x7f63f40a5040 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:32:56.207 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:56.205+0000 7f6416800640 1 --2- 172.21.15.83:0/1144396185 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6410109ac0 0x7f641019ec70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:32:56.207 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:56.205+0000 7f6416800640 1 --2- 172.21.15.83:0/1144396185 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6410104460 0x7f641019a8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:32:56.207 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:56.205+0000 7f6416800640 1 --2- 172.21.15.83:0/1144396185 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6410103ac0 0x7f641019a3a0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:32:56.207 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:56.205+0000 7f6416800640 1 -- 172.21.15.83:0/1144396185 >> 172.21.15.83:0/1144396185 conn(0x7f64100ff7c0 msgr2=0x7f64100fffd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:32:56.208 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:56.205+0000 7f6416800640 1 -- 172.21.15.83:0/1144396185 shutdown_connections 2024-08-19T01:32:56.208 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:56.205+0000 7f6416800640 1 -- 172.21.15.83:0/1144396185 wait complete. 2024-08-19T01:32:56.546 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Aug 19 01:32:56 smithi083 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-mgr-x[17344]: 2024-08-19T01:32:56.176+0000 7f0272800640 -1 client.0 error registering admin socket command: (17) File exists 2024-08-19T01:32:56.546 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Aug 19 01:32:56 smithi083 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-mgr-x[17344]: 2024-08-19T01:32:56.176+0000 7f0272800640 -1 client.0 error registering admin socket command: (17) File exists 2024-08-19T01:32:56.546 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Aug 19 01:32:56 smithi083 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-mgr-x[17344]: 2024-08-19T01:32:56.176+0000 7f0272800640 -1 client.0 error registering admin socket command: (17) File exists 2024-08-19T01:32:56.547 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Aug 19 01:32:56 smithi083 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-mgr-x[17344]: 2024-08-19T01:32:56.176+0000 7f0272800640 -1 client.0 error registering admin socket command: (17) File exists 2024-08-19T01:32:56.547 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Aug 19 01:32:56 smithi083 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-mgr-x[17344]: 2024-08-19T01:32:56.176+0000 7f0272800640 -1 client.0 error registering admin socket command: (17) File exists 2024-08-19T01:32:56.902 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833 shell --fsid bad8780e-5dc9-11ef-bccf-c7b262605968 -- ceph fs subvolume create cephfs sv_0 --group_name qa 2024-08-19T01:32:57.282 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/bad8780e-5dc9-11ef-bccf-c7b262605968/mon.a/config 2024-08-19T01:32:57.703 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:32:57 smithi117 ceph-mon[22016]: from='client.14547 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-08-19T01:32:57.703 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:32:57 smithi117 ceph-mon[22016]: pgmap v280: 97 pgs: 97 active+clean; 582 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 2.9 KiB/s wr, 10 op/s 2024-08-19T01:32:57.716 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:32:57 smithi167 ceph-mon[21875]: from='client.14547 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-08-19T01:32:57.716 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:32:57 smithi167 ceph-mon[21875]: pgmap v280: 97 pgs: 97 active+clean; 582 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 2.9 KiB/s wr, 10 op/s 2024-08-19T01:32:57.741 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:32:57 smithi083 ceph-mon[17120]: from='client.14547 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-08-19T01:32:57.741 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:32:57 smithi083 ceph-mon[17120]: pgmap v280: 97 pgs: 97 active+clean; 582 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 2.9 KiB/s wr, 10 op/s 2024-08-19T01:32:57.875 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:57.854+0000 7f4975400640 1 -- 172.21.15.83:0/2756977390 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f49701078b0 msgr2=0x7f4970107c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:32:57.875 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:57.854+0000 7f4975400640 1 --2- 172.21.15.83:0/2756977390 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f49701078b0 0x7f4970107c90 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f4960009cd0 tx=0x7f496002a8e0 comp rx=0 tx=0).stop 2024-08-19T01:32:57.875 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:57.854+0000 7f4975400640 1 -- 172.21.15.83:0/2756977390 shutdown_connections 2024-08-19T01:32:57.875 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:57.854+0000 7f4975400640 1 --2- 172.21.15.83:0/2756977390 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4970102250 0x7f497010e780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:32:57.875 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:57.854+0000 7f4975400640 1 --2- 172.21.15.83:0/2756977390 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f49701018b0 0x7f4970101d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:32:57.875 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:57.854+0000 7f4975400640 1 --2- 172.21.15.83:0/2756977390 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f49701078b0 0x7f4970107c90 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:32:57.875 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:57.854+0000 7f4975400640 1 -- 172.21.15.83:0/2756977390 >> 172.21.15.83:0/2756977390 conn(0x7f49700fd5f0 msgr2=0x7f49700ffa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:32:57.875 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:57.854+0000 7f4975400640 1 -- 172.21.15.83:0/2756977390 shutdown_connections 2024-08-19T01:32:57.875 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:57.854+0000 7f4975400640 1 -- 172.21.15.83:0/2756977390 wait complete. 2024-08-19T01:32:57.876 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:57.855+0000 7f4975400640 1 Processor -- start 2024-08-19T01:32:57.876 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:57.855+0000 7f4975400640 1 -- start start 2024-08-19T01:32:57.876 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:57.856+0000 7f4975400640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f49701018b0 0x7f497019a3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:32:57.876 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:57.856+0000 7f4975400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4970102250 0x7f497019a8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:32:57.876 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:57.856+0000 7f4975400640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f49701078b0 0x7f497019ec80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:32:57.876 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:57.856+0000 7f4975400640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f4970111cb0 con 0x7f4970102250 2024-08-19T01:32:57.876 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:57.856+0000 7f4975400640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f4970111b30 con 0x7f49701018b0 2024-08-19T01:32:57.876 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:57.856+0000 7f4975400640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f4970111e30 con 0x7f49701078b0 2024-08-19T01:32:57.876 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:57.856+0000 7f496de00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4970102250 0x7f497019a8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:32:57.876 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:57.856+0000 7f496de00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4970102250 0x7f497019a8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:39808/0 (socket says 172.21.15.83:39808) 2024-08-19T01:32:57.876 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:57.856+0000 7f496de00640 1 -- 172.21.15.83:0/1645261364 learned_addr learned my addr 172.21.15.83:0/1645261364 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-08-19T01:32:57.876 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:57.856+0000 7f496f200640 1 --2- 172.21.15.83:0/1645261364 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f49701078b0 0x7f497019ec80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:32:57.877 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:57.856+0000 7f496e800640 1 --2- 172.21.15.83:0/1645261364 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f49701018b0 0x7f497019a3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:32:57.877 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:57.856+0000 7f496de00640 1 -- 172.21.15.83:0/1645261364 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f49701078b0 msgr2=0x7f497019ec80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:32:57.877 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:57.856+0000 7f496de00640 1 --2- 172.21.15.83:0/1645261364 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f49701078b0 0x7f497019ec80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:32:57.877 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:57.856+0000 7f496de00640 1 -- 172.21.15.83:0/1645261364 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f49701018b0 msgr2=0x7f497019a3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:32:57.877 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:57.856+0000 7f496de00640 1 --2- 172.21.15.83:0/1645261364 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f49701018b0 0x7f497019a3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:32:57.877 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:57.856+0000 7f496de00640 1 -- 172.21.15.83:0/1645261364 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f497019f360 con 0x7f4970102250 2024-08-19T01:32:57.877 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:57.857+0000 7f496de00640 1 --2- 172.21.15.83:0/1645261364 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4970102250 0x7f497019a8f0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f495c00e9f0 tx=0x7f495c00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:32:57.877 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:57.857+0000 7f4967400640 1 -- 172.21.15.83:0/1645261364 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f495c00cd80 con 0x7f4970102250 2024-08-19T01:32:57.877 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:57.857+0000 7f4967400640 1 -- 172.21.15.83:0/1645261364 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f495c010910 con 0x7f4970102250 2024-08-19T01:32:57.877 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:57.857+0000 7f4975400640 1 -- 172.21.15.83:0/1645261364 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f497019f650 con 0x7f4970102250 2024-08-19T01:32:57.877 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:57.857+0000 7f4967400640 1 -- 172.21.15.83:0/1645261364 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f495c018990 con 0x7f4970102250 2024-08-19T01:32:57.877 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:57.857+0000 7f4975400640 1 -- 172.21.15.83:0/1645261364 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f49701a6f30 con 0x7f4970102250 2024-08-19T01:32:57.877 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:57.858+0000 7f4975400640 1 -- 172.21.15.83:0/1645261364 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4970101d10 con 0x7f4970102250 2024-08-19T01:32:57.877 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:57.860+0000 7f4967400640 1 -- 172.21.15.83:0/1645261364 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f495c010ac0 con 0x7f4970102250 2024-08-19T01:32:57.878 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:57.862+0000 7f4967400640 1 --2- 172.21.15.83:0/1645261364 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f49540a2c10 0x7f49540a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:32:57.878 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:57.862+0000 7f4967400640 1 -- 172.21.15.83:0/1645261364 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 7979+0+0 (secure 0 0 0) 0x7f495c0cf430 con 0x7f4970102250 2024-08-19T01:32:57.878 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:57.862+0000 7f496e800640 1 --2- 172.21.15.83:0/1645261364 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f49540a2c10 0x7f49540a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:32:57.878 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:57.863+0000 7f496e800640 1 --2- 172.21.15.83:0/1645261364 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f49540a2c10 0x7f49540a50d0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f4960006010 tx=0x7f4960009360 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:32:57.878 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:57.866+0000 7f4967400640 1 -- 172.21.15.83:0/1645261364 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7f495c09bc90 con 0x7f4970102250 2024-08-19T01:32:58.070 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:58.069+0000 7f4975400640 1 -- 172.21.15.83:0/1645261364 --> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7f497010dbd0 con 0x7f49540a2c10 2024-08-19T01:32:58.099 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:58.094+0000 7f4967400640 1 -- 172.21.15.83:0/1645261364 <== mgr.14150 v2:172.21.15.83:6800/683740916 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f497010dbd0 con 0x7f49540a2c10 2024-08-19T01:32:58.099 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:58.096+0000 7f4975400640 1 -- 172.21.15.83:0/1645261364 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f49540a2c10 msgr2=0x7f49540a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:32:58.099 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:58.096+0000 7f4975400640 1 --2- 172.21.15.83:0/1645261364 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f49540a2c10 0x7f49540a50d0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f4960006010 tx=0x7f4960009360 comp rx=0 tx=0).stop 2024-08-19T01:32:58.099 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:58.096+0000 7f4975400640 1 -- 172.21.15.83:0/1645261364 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4970102250 msgr2=0x7f497019a8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:32:58.099 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:58.096+0000 7f4975400640 1 --2- 172.21.15.83:0/1645261364 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4970102250 0x7f497019a8f0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f495c00e9f0 tx=0x7f495c00eeb0 comp rx=0 tx=0).stop 2024-08-19T01:32:58.099 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:58.096+0000 7f4975400640 1 -- 172.21.15.83:0/1645261364 shutdown_connections 2024-08-19T01:32:58.099 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:58.096+0000 7f4975400640 1 --2- 172.21.15.83:0/1645261364 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f49540a2c10 0x7f49540a50d0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:32:58.099 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:58.096+0000 7f4975400640 1 --2- 172.21.15.83:0/1645261364 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f49701078b0 0x7f497019ec80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:32:58.099 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:58.096+0000 7f4975400640 1 --2- 172.21.15.83:0/1645261364 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4970102250 0x7f497019a8f0 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:32:58.099 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:58.096+0000 7f4975400640 1 --2- 172.21.15.83:0/1645261364 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f49701018b0 0x7f497019a3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:32:58.099 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:58.096+0000 7f4975400640 1 -- 172.21.15.83:0/1645261364 >> 172.21.15.83:0/1645261364 conn(0x7f49700fd5f0 msgr2=0x7f497010d4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:32:58.099 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:58.097+0000 7f4975400640 1 -- 172.21.15.83:0/1645261364 shutdown_connections 2024-08-19T01:32:58.100 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:58.097+0000 7f4975400640 1 -- 172.21.15.83:0/1645261364 wait complete. 2024-08-19T01:32:58.702 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:32:58 smithi117 ceph-mon[22016]: mgrmap e17: x(active, since 6m), standbys: z, y 2024-08-19T01:32:58.716 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:32:58 smithi167 ceph-mon[21875]: mgrmap e17: x(active, since 6m), standbys: z, y 2024-08-19T01:32:58.863 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:32:58 smithi083 ceph-mon[17120]: mgrmap e17: x(active, since 6m), standbys: z, y 2024-08-19T01:32:58.902 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833 shell --fsid bad8780e-5dc9-11ef-bccf-c7b262605968 -- ceph fs subvolume getpath cephfs --group_name qa sv_0 2024-08-19T01:32:59.278 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/bad8780e-5dc9-11ef-bccf-c7b262605968/mon.a/config 2024-08-19T01:32:59.702 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:32:59 smithi117 ceph-mon[22016]: from='client.14559 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-08-19T01:32:59.703 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:32:59 smithi117 ceph-mon[22016]: pgmap v281: 97 pgs: 97 active+clean; 582 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 2.3 KiB/s wr, 11 op/s 2024-08-19T01:32:59.716 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:32:59 smithi167 ceph-mon[21875]: from='client.14559 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-08-19T01:32:59.716 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:32:59 smithi167 ceph-mon[21875]: pgmap v281: 97 pgs: 97 active+clean; 582 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 2.3 KiB/s wr, 11 op/s 2024-08-19T01:32:59.741 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:32:59 smithi083 ceph-mon[17120]: from='client.14559 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-08-19T01:32:59.741 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:32:59 smithi083 ceph-mon[17120]: pgmap v281: 97 pgs: 97 active+clean; 582 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 2.3 KiB/s wr, 11 op/s 2024-08-19T01:32:59.883 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:59.856+0000 7f6917e00640 1 -- 172.21.15.83:0/2307007731 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f69180742f0 msgr2=0x7f69180746d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:32:59.883 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:59.856+0000 7f6917e00640 1 --2- 172.21.15.83:0/2307007731 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f69180742f0 0x7f69180746d0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f6908009900 tx=0x7f690802a480 comp rx=0 tx=0).stop 2024-08-19T01:32:59.883 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:59.856+0000 7f6917e00640 1 -- 172.21.15.83:0/2307007731 shutdown_connections 2024-08-19T01:32:59.883 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:59.856+0000 7f6917e00640 1 --2- 172.21.15.83:0/2307007731 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f691810df90 0x7f6918110380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:32:59.883 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:59.856+0000 7f6917e00640 1 --2- 172.21.15.83:0/2307007731 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6918074c10 0x7f691810da50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:32:59.883 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:59.856+0000 7f6917e00640 1 --2- 172.21.15.83:0/2307007731 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f69180742f0 0x7f69180746d0 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:32:59.884 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:59.856+0000 7f6917e00640 1 -- 172.21.15.83:0/2307007731 >> 172.21.15.83:0/2307007731 conn(0x7f69180fd470 msgr2=0x7f69180ff890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:32:59.884 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:59.856+0000 7f6917e00640 1 -- 172.21.15.83:0/2307007731 shutdown_connections 2024-08-19T01:32:59.884 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:59.856+0000 7f6917e00640 1 -- 172.21.15.83:0/2307007731 wait complete. 2024-08-19T01:32:59.884 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:59.857+0000 7f6917e00640 1 Processor -- start 2024-08-19T01:32:59.884 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:59.857+0000 7f6917e00640 1 -- start start 2024-08-19T01:32:59.884 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:59.857+0000 7f6917e00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f69180742f0 0x7f691819a2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:32:59.884 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:59.857+0000 7f6917e00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6918074c10 0x7f691819a7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:32:59.884 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:59.857+0000 7f6917e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f691810df90 0x7f691819eb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:32:59.884 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:59.857+0000 7f6917e00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f6918113d30 con 0x7f691810df90 2024-08-19T01:32:59.884 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:59.857+0000 7f6917e00640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f6918113bb0 con 0x7f69180742f0 2024-08-19T01:32:59.884 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:59.857+0000 7f6917e00640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f6918113eb0 con 0x7f6918074c10 2024-08-19T01:32:59.884 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:59.857+0000 7f6916a00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f69180742f0 0x7f691819a2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:32:59.885 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:59.857+0000 7f6917400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f691810df90 0x7f691819eb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:32:59.885 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:59.858+0000 7f6916a00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f69180742f0 0x7f691819a2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.83:56430/0 (socket says 172.21.15.83:56430) 2024-08-19T01:32:59.885 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:59.858+0000 7f6916a00640 1 -- 172.21.15.83:0/1239160348 learned_addr learned my addr 172.21.15.83:0/1239160348 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-08-19T01:32:59.885 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:59.858+0000 7f6916000640 1 --2- 172.21.15.83:0/1239160348 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6918074c10 0x7f691819a7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:32:59.885 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:59.858+0000 7f6916a00640 1 -- 172.21.15.83:0/1239160348 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6918074c10 msgr2=0x7f691819a7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:32:59.885 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:59.858+0000 7f6916a00640 1 --2- 172.21.15.83:0/1239160348 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6918074c10 0x7f691819a7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:32:59.885 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:59.858+0000 7f6916a00640 1 -- 172.21.15.83:0/1239160348 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f691810df90 msgr2=0x7f691819eb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:32:59.885 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:59.858+0000 7f6916a00640 1 --2- 172.21.15.83:0/1239160348 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f691810df90 0x7f691819eb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:32:59.885 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:59.858+0000 7f6916a00640 1 -- 172.21.15.83:0/1239160348 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f691819f260 con 0x7f69180742f0 2024-08-19T01:32:59.885 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:59.858+0000 7f6916a00640 1 --2- 172.21.15.83:0/1239160348 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f69180742f0 0x7f691819a2b0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f6908004770 tx=0x7f6908003c10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:32:59.885 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:59.859+0000 7f690f400640 1 -- 172.21.15.83:0/1239160348 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f690803f070 con 0x7f69180742f0 2024-08-19T01:32:59.885 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:59.859+0000 7f690f400640 1 -- 172.21.15.83:0/1239160348 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6908031ca0 con 0x7f69180742f0 2024-08-19T01:32:59.886 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:59.859+0000 7f6917e00640 1 -- 172.21.15.83:0/1239160348 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f691819f4f0 con 0x7f69180742f0 2024-08-19T01:32:59.886 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:59.859+0000 7f690f400640 1 -- 172.21.15.83:0/1239160348 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6908035030 con 0x7f69180742f0 2024-08-19T01:32:59.886 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:59.859+0000 7f6917e00640 1 -- 172.21.15.83:0/1239160348 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f69181a6e30 con 0x7f69180742f0 2024-08-19T01:32:59.886 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:59.860+0000 7f6917e00640 1 -- 172.21.15.83:0/1239160348 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f69180759a0 con 0x7f69180742f0 2024-08-19T01:32:59.886 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:59.862+0000 7f690f400640 1 -- 172.21.15.83:0/1239160348 <== mon.2 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f6908042070 con 0x7f69180742f0 2024-08-19T01:32:59.886 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:59.863+0000 7f690f400640 1 --2- 172.21.15.83:0/1239160348 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f68fc0a2c10 0x7f68fc0a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:32:59.886 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:59.863+0000 7f690f400640 1 -- 172.21.15.83:0/1239160348 <== mon.2 v2:172.21.15.117:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 7979+0+0 (secure 0 0 0) 0x7f69080efaf0 con 0x7f69180742f0 2024-08-19T01:32:59.886 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:59.863+0000 7f6916000640 1 --2- 172.21.15.83:0/1239160348 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f68fc0a2c10 0x7f68fc0a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:32:59.886 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:59.864+0000 7f6916000640 1 --2- 172.21.15.83:0/1239160348 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f68fc0a2c10 0x7f68fc0a50d0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f691810f380 tx=0x7f690400b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:32:59.886 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:32:59.865+0000 7f690f400640 1 -- 172.21.15.83:0/1239160348 <== mon.2 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7f69080bc350 con 0x7f69180742f0 2024-08-19T01:33:00.070 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:00.068+0000 7f6917e00640 1 -- 172.21.15.83:0/1239160348 --> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}) -- 0x7f6918062360 con 0x7f68fc0a2c10 2024-08-19T01:33:00.075 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:00.073+0000 7f690f400640 1 -- 172.21.15.83:0/1239160348 <== mgr.14150 v2:172.21.15.83:6800/683740916 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+54 (secure 0 0 0) 0x7f6918062360 con 0x7f68fc0a2c10 2024-08-19T01:33:00.075 INFO:teuthology.orchestra.run.smithi083.stdout:/volumes/qa/sv_0/a6934629-8ec9-4d98-b176-46bc4ffe3d7f 2024-08-19T01:33:00.077 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:00.075+0000 7f6917e00640 1 -- 172.21.15.83:0/1239160348 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f68fc0a2c10 msgr2=0x7f68fc0a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:33:00.077 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:00.075+0000 7f6917e00640 1 --2- 172.21.15.83:0/1239160348 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f68fc0a2c10 0x7f68fc0a50d0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f691810f380 tx=0x7f690400b410 comp rx=0 tx=0).stop 2024-08-19T01:33:00.077 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:00.075+0000 7f6917e00640 1 -- 172.21.15.83:0/1239160348 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f69180742f0 msgr2=0x7f691819a2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:33:00.078 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:00.075+0000 7f6917e00640 1 --2- 172.21.15.83:0/1239160348 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f69180742f0 0x7f691819a2b0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f6908004770 tx=0x7f6908003c10 comp rx=0 tx=0).stop 2024-08-19T01:33:00.078 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:00.075+0000 7f6917e00640 1 -- 172.21.15.83:0/1239160348 shutdown_connections 2024-08-19T01:33:00.078 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:00.075+0000 7f6917e00640 1 --2- 172.21.15.83:0/1239160348 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f68fc0a2c10 0x7f68fc0a50d0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:33:00.078 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:00.075+0000 7f6917e00640 1 --2- 172.21.15.83:0/1239160348 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f691810df90 0x7f691819eb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:33:00.078 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:00.075+0000 7f6917e00640 1 --2- 172.21.15.83:0/1239160348 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6918074c10 0x7f691819a7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:33:00.078 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:00.076+0000 7f6917e00640 1 --2- 172.21.15.83:0/1239160348 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f69180742f0 0x7f691819a2b0 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:33:00.078 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:00.076+0000 7f6917e00640 1 -- 172.21.15.83:0/1239160348 >> 172.21.15.83:0/1239160348 conn(0x7f69180fd470 msgr2=0x7f69180fe110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:33:00.078 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:00.076+0000 7f6917e00640 1 -- 172.21.15.83:0/1239160348 shutdown_connections 2024-08-19T01:33:00.078 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:00.076+0000 7f6917e00640 1 -- 172.21.15.83:0/1239160348 wait complete. 2024-08-19T01:33:00.824 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833 shell --fsid bad8780e-5dc9-11ef-bccf-c7b262605968 -- ceph fs subvolume create cephfs sv_1 --group_name qa 2024-08-19T01:33:01.199 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/bad8780e-5dc9-11ef-bccf-c7b262605968/mon.a/config 2024-08-19T01:33:01.655 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:33:01 smithi083 ceph-mon[17120]: from='client.24635 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-08-19T01:33:01.655 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:33:01 smithi083 ceph-mon[17120]: pgmap v282: 97 pgs: 97 active+clean; 603 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 4.2 KiB/s wr, 13 op/s 2024-08-19T01:33:01.702 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:33:01 smithi117 ceph-mon[22016]: from='client.24635 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-08-19T01:33:01.703 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:33:01 smithi117 ceph-mon[22016]: pgmap v282: 97 pgs: 97 active+clean; 603 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 4.2 KiB/s wr, 13 op/s 2024-08-19T01:33:01.716 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:33:01 smithi167 ceph-mon[21875]: from='client.24635 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-08-19T01:33:01.716 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:33:01 smithi167 ceph-mon[21875]: pgmap v282: 97 pgs: 97 active+clean; 603 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 4.2 KiB/s wr, 13 op/s 2024-08-19T01:33:01.884 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:01.797+0000 7f390cc00640 1 -- 172.21.15.83:0/2696074259 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f39081016f0 msgr2=0x7f3908101b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:33:01.884 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:01.797+0000 7f390cc00640 1 --2- 172.21.15.83:0/2696074259 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f39081016f0 0x7f3908101b50 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f38f4009930 tx=0x7f38f402a420 comp rx=0 tx=0).stop 2024-08-19T01:33:01.884 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:01.797+0000 7f390cc00640 1 -- 172.21.15.83:0/2696074259 shutdown_connections 2024-08-19T01:33:01.884 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:01.797+0000 7f390cc00640 1 --2- 172.21.15.83:0/2696074259 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3908102090 0x7f390810e5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:33:01.884 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:01.797+0000 7f390cc00640 1 --2- 172.21.15.83:0/2696074259 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f39081016f0 0x7f3908101b50 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:33:01.884 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:01.797+0000 7f390cc00640 1 --2- 172.21.15.83:0/2696074259 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f39081076f0 0x7f3908107ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:33:01.884 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:01.797+0000 7f390cc00640 1 -- 172.21.15.83:0/2696074259 >> 172.21.15.83:0/2696074259 conn(0x7f39080fd470 msgr2=0x7f39080ff890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:33:01.884 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:01.797+0000 7f390cc00640 1 -- 172.21.15.83:0/2696074259 shutdown_connections 2024-08-19T01:33:01.885 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:01.797+0000 7f390cc00640 1 -- 172.21.15.83:0/2696074259 wait complete. 2024-08-19T01:33:01.885 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:01.798+0000 7f390cc00640 1 Processor -- start 2024-08-19T01:33:01.885 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:01.798+0000 7f390cc00640 1 -- start start 2024-08-19T01:33:01.885 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:01.799+0000 7f390cc00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f39081016f0 0x7f3908074220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:33:01.885 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:01.799+0000 7f390cc00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3908102090 0x7f3908074760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:33:01.885 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:01.799+0000 7f390cc00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f39081076f0 0x7f3908074ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:33:01.885 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:01.799+0000 7f390cc00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f3908111b40 con 0x7f3908102090 2024-08-19T01:33:01.885 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:01.799+0000 7f390cc00640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f39081119c0 con 0x7f39081076f0 2024-08-19T01:33:01.885 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:01.799+0000 7f390cc00640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f3908111cc0 con 0x7f39081016f0 2024-08-19T01:33:01.885 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:01.799+0000 7f3907400640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f39081016f0 0x7f3908074220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:33:01.885 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:01.799+0000 7f3907400640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f39081016f0 0x7f3908074220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.83:37370/0 (socket says 172.21.15.83:37370) 2024-08-19T01:33:01.886 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:01.799+0000 7f3907400640 1 -- 172.21.15.83:0/2337724693 learned_addr learned my addr 172.21.15.83:0/2337724693 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-08-19T01:33:01.886 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:01.799+0000 7f3906a00640 1 --2- 172.21.15.83:0/2337724693 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3908102090 0x7f3908074760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:33:01.886 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:01.799+0000 7f3907e00640 1 --2- 172.21.15.83:0/2337724693 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f39081076f0 0x7f3908074ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:33:01.886 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:01.799+0000 7f3907400640 1 -- 172.21.15.83:0/2337724693 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f39081076f0 msgr2=0x7f3908074ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:33:01.886 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:01.799+0000 7f3907400640 1 --2- 172.21.15.83:0/2337724693 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f39081076f0 0x7f3908074ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:33:01.886 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:01.799+0000 7f3907400640 1 -- 172.21.15.83:0/2337724693 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3908102090 msgr2=0x7f3908074760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:33:01.886 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:01.799+0000 7f3907400640 1 --2- 172.21.15.83:0/2337724693 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3908102090 0x7f3908074760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:33:01.886 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:01.799+0000 7f3907400640 1 -- 172.21.15.83:0/2337724693 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f39081a6480 con 0x7f39081016f0 2024-08-19T01:33:01.886 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:01.799+0000 7f3907e00640 1 --2- 172.21.15.83:0/2337724693 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f39081076f0 0x7f3908074ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-19T01:33:01.886 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:01.800+0000 7f3907400640 1 --2- 172.21.15.83:0/2337724693 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f39081016f0 0x7f3908074220 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f38f800e9f0 tx=0x7f38f800eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:33:01.887 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:01.800+0000 7f38ffe00640 1 -- 172.21.15.83:0/2337724693 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38f800cd80 con 0x7f39081016f0 2024-08-19T01:33:01.887 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:01.800+0000 7f390cc00640 1 -- 172.21.15.83:0/2337724693 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f39081a6770 con 0x7f39081016f0 2024-08-19T01:33:01.887 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:01.800+0000 7f38ffe00640 1 -- 172.21.15.83:0/2337724693 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f38f8010910 con 0x7f39081016f0 2024-08-19T01:33:01.887 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:01.800+0000 7f38ffe00640 1 -- 172.21.15.83:0/2337724693 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38f80188f0 con 0x7f39081016f0 2024-08-19T01:33:01.887 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:01.800+0000 7f390cc00640 1 -- 172.21.15.83:0/2337724693 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f39081a6cd0 con 0x7f39081016f0 2024-08-19T01:33:01.887 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:01.801+0000 7f390cc00640 1 -- 172.21.15.83:0/2337724693 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3908101b50 con 0x7f39081016f0 2024-08-19T01:33:01.887 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:01.803+0000 7f38ffe00640 1 -- 172.21.15.83:0/2337724693 <== mon.1 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f38f8010ac0 con 0x7f39081016f0 2024-08-19T01:33:01.887 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:01.804+0000 7f38ffe00640 1 --2- 172.21.15.83:0/2337724693 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f38ec0a2c90 0x7f38ec0a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:33:01.887 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:01.804+0000 7f38ffe00640 1 -- 172.21.15.83:0/2337724693 <== mon.1 v2:172.21.15.167:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 7979+0+0 (secure 0 0 0) 0x7f38f80cf4a0 con 0x7f39081016f0 2024-08-19T01:33:01.887 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:01.805+0000 7f3906a00640 1 --2- 172.21.15.83:0/2337724693 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f38ec0a2c90 0x7f38ec0a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:33:01.887 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:01.805+0000 7f3906a00640 1 --2- 172.21.15.83:0/2337724693 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f38ec0a2c90 0x7f38ec0a5150 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f38f4006010 tx=0x7f38f4000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:33:01.887 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:01.808+0000 7f38ffe00640 1 -- 172.21.15.83:0/2337724693 <== mon.1 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7f38f809c350 con 0x7f39081016f0 2024-08-19T01:33:02.013 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:02.011+0000 7f390cc00640 1 -- 172.21.15.83:0/2337724693 --> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7f3908109cd0 con 0x7f38ec0a2c90 2024-08-19T01:33:02.038 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:02.036+0000 7f38ffe00640 1 -- 172.21.15.83:0/2337724693 <== mgr.14150 v2:172.21.15.83:6800/683740916 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f38f4004190 con 0x7f38ec0a2c90 2024-08-19T01:33:02.041 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:02.039+0000 7f390cc00640 1 -- 172.21.15.83:0/2337724693 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f38ec0a2c90 msgr2=0x7f38ec0a5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:33:02.041 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:02.039+0000 7f390cc00640 1 --2- 172.21.15.83:0/2337724693 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f38ec0a2c90 0x7f38ec0a5150 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f38f4006010 tx=0x7f38f4000bc0 comp rx=0 tx=0).stop 2024-08-19T01:33:02.041 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:02.039+0000 7f390cc00640 1 -- 172.21.15.83:0/2337724693 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f39081016f0 msgr2=0x7f3908074220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:33:02.041 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:02.039+0000 7f390cc00640 1 --2- 172.21.15.83:0/2337724693 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f39081016f0 0x7f3908074220 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f38f800e9f0 tx=0x7f38f800eeb0 comp rx=0 tx=0).stop 2024-08-19T01:33:02.041 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:02.039+0000 7f390cc00640 1 -- 172.21.15.83:0/2337724693 shutdown_connections 2024-08-19T01:33:02.041 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:02.039+0000 7f390cc00640 1 --2- 172.21.15.83:0/2337724693 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f38ec0a2c90 0x7f38ec0a5150 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:33:02.041 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:02.039+0000 7f390cc00640 1 --2- 172.21.15.83:0/2337724693 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f39081076f0 0x7f3908074ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:33:02.041 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:02.039+0000 7f390cc00640 1 --2- 172.21.15.83:0/2337724693 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3908102090 0x7f3908074760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:33:02.041 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:02.039+0000 7f390cc00640 1 --2- 172.21.15.83:0/2337724693 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f39081016f0 0x7f3908074220 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:33:02.041 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:02.039+0000 7f390cc00640 1 -- 172.21.15.83:0/2337724693 >> 172.21.15.83:0/2337724693 conn(0x7f39080fd470 msgr2=0x7f39080fdb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:33:02.041 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:02.039+0000 7f390cc00640 1 -- 172.21.15.83:0/2337724693 shutdown_connections 2024-08-19T01:33:02.041 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:02.039+0000 7f390cc00640 1 -- 172.21.15.83:0/2337724693 wait complete. 2024-08-19T01:33:02.825 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833 shell --fsid bad8780e-5dc9-11ef-bccf-c7b262605968 -- ceph fs subvolume getpath cephfs --group_name qa sv_1 2024-08-19T01:33:03.202 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/bad8780e-5dc9-11ef-bccf-c7b262605968/mon.a/config 2024-08-19T01:33:03.608 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:33:03 smithi083 ceph-mon[17120]: from='client.24622 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-08-19T01:33:03.608 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:33:03 smithi083 ceph-mon[17120]: pgmap v283: 97 pgs: 97 active+clean; 603 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 4.4 KiB/s wr, 13 op/s 2024-08-19T01:33:03.702 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:33:03 smithi117 ceph-mon[22016]: from='client.24622 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-08-19T01:33:03.702 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:33:03 smithi117 ceph-mon[22016]: pgmap v283: 97 pgs: 97 active+clean; 603 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 4.4 KiB/s wr, 13 op/s 2024-08-19T01:33:03.716 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:33:03 smithi167 ceph-mon[21875]: from='client.24622 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-08-19T01:33:03.716 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:33:03 smithi167 ceph-mon[21875]: pgmap v283: 97 pgs: 97 active+clean; 603 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 4.4 KiB/s wr, 13 op/s 2024-08-19T01:33:03.816 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:03.814+0000 7f318ce00640 1 -- 172.21.15.83:0/3443737555 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3188109b70 msgr2=0x7f3188109f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:33:03.817 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:03.814+0000 7f318ce00640 1 --2- 172.21.15.83:0/3443737555 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3188109b70 0x7f3188109f50 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f3178009930 tx=0x7f317802a420 comp rx=0 tx=0).stop 2024-08-19T01:33:03.817 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:03.815+0000 7f318ce00640 1 -- 172.21.15.83:0/3443737555 shutdown_connections 2024-08-19T01:33:03.817 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:03.815+0000 7f318ce00640 1 --2- 172.21.15.83:0/3443737555 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3188104480 0x7f31881109b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:33:03.817 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:03.815+0000 7f318ce00640 1 --2- 172.21.15.83:0/3443737555 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3188103ae0 0x7f3188103f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:33:03.817 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:03.815+0000 7f318ce00640 1 --2- 172.21.15.83:0/3443737555 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3188109b70 0x7f3188109f50 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:33:03.817 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:03.815+0000 7f318ce00640 1 -- 172.21.15.83:0/3443737555 >> 172.21.15.83:0/3443737555 conn(0x7f31880ff780 msgr2=0x7f3188101ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:33:03.817 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:03.815+0000 7f318ce00640 1 -- 172.21.15.83:0/3443737555 shutdown_connections 2024-08-19T01:33:03.817 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:03.815+0000 7f318ce00640 1 -- 172.21.15.83:0/3443737555 wait complete. 2024-08-19T01:33:03.818 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:03.816+0000 7f318ce00640 1 Processor -- start 2024-08-19T01:33:03.819 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:03.816+0000 7f318ce00640 1 -- start start 2024-08-19T01:33:03.819 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:03.816+0000 7f318ce00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3188103ae0 0x7f318819a3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:33:03.819 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:03.816+0000 7f318ce00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3188104480 0x7f318819a900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:33:03.819 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:03.817+0000 7f318ce00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3188109b70 0x7f318819ec90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:33:03.819 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:03.817+0000 7f318ce00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f3188113ee0 con 0x7f3188104480 2024-08-19T01:33:03.819 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:03.817+0000 7f318ce00640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f3188113d60 con 0x7f3188109b70 2024-08-19T01:33:03.819 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:03.817+0000 7f318ce00640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f3188114060 con 0x7f3188103ae0 2024-08-19T01:33:03.819 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:03.817+0000 7f317d400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3188104480 0x7f318819a900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:33:03.819 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:03.817+0000 7f317d400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3188104480 0x7f318819a900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:44796/0 (socket says 172.21.15.83:44796) 2024-08-19T01:33:03.819 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:03.817+0000 7f317d400640 1 -- 172.21.15.83:0/3198145213 learned_addr learned my addr 172.21.15.83:0/3198145213 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-08-19T01:33:03.819 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:03.817+0000 7f3185e00640 1 --2- 172.21.15.83:0/3198145213 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3188103ae0 0x7f318819a3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:33:03.819 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:03.817+0000 7f3186800640 1 --2- 172.21.15.83:0/3198145213 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3188109b70 0x7f318819ec90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:33:03.820 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:03.817+0000 7f317d400640 1 -- 172.21.15.83:0/3198145213 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3188103ae0 msgr2=0x7f318819a3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:33:03.820 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:03.817+0000 7f317d400640 1 --2- 172.21.15.83:0/3198145213 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3188103ae0 0x7f318819a3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:33:03.820 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:03.817+0000 7f317d400640 1 -- 172.21.15.83:0/3198145213 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3188109b70 msgr2=0x7f318819ec90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:33:03.820 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:03.817+0000 7f317d400640 1 --2- 172.21.15.83:0/3198145213 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3188109b70 0x7f318819ec90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:33:03.821 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:03.817+0000 7f317d400640 1 -- 172.21.15.83:0/3198145213 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f318819f370 con 0x7f3188104480 2024-08-19T01:33:03.821 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:03.818+0000 7f317d400640 1 --2- 172.21.15.83:0/3198145213 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3188104480 0x7f318819a900 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f317000ea60 tx=0x7f317000ef20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:33:03.821 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:03.818+0000 7f317f400640 1 -- 172.21.15.83:0/3198145213 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f317000cde0 con 0x7f3188104480 2024-08-19T01:33:03.821 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:03.818+0000 7f318ce00640 1 -- 172.21.15.83:0/3198145213 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f318819f660 con 0x7f3188104480 2024-08-19T01:33:03.821 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:03.818+0000 7f317f400640 1 -- 172.21.15.83:0/3198145213 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f31700035f0 con 0x7f3188104480 2024-08-19T01:33:03.821 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:03.818+0000 7f317f400640 1 -- 172.21.15.83:0/3198145213 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3170010670 con 0x7f3188104480 2024-08-19T01:33:03.821 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:03.818+0000 7f318ce00640 1 -- 172.21.15.83:0/3198145213 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f31881a6f40 con 0x7f3188104480 2024-08-19T01:33:03.821 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:03.819+0000 7f318ce00640 1 -- 172.21.15.83:0/3198145213 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3188103f40 con 0x7f3188104480 2024-08-19T01:33:03.823 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:03.821+0000 7f317f400640 1 -- 172.21.15.83:0/3198145213 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f31700037a0 con 0x7f3188104480 2024-08-19T01:33:03.824 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:03.823+0000 7f317f400640 1 --2- 172.21.15.83:0/3198145213 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f316c0a2c10 0x7f316c0a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:33:03.825 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:03.823+0000 7f317f400640 1 -- 172.21.15.83:0/3198145213 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 7979+0+0 (secure 0 0 0) 0x7f31700cf410 con 0x7f3188104480 2024-08-19T01:33:03.825 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:03.823+0000 7f3185e00640 1 --2- 172.21.15.83:0/3198145213 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f316c0a2c10 0x7f316c0a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:33:03.825 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:03.824+0000 7f3185e00640 1 --2- 172.21.15.83:0/3198145213 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f316c0a2c10 0x7f316c0a50d0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f3178009900 tx=0x7f31780058d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:33:03.828 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:03.827+0000 7f317f400640 1 -- 172.21.15.83:0/3198145213 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7f317009c2c0 con 0x7f3188104480 2024-08-19T01:33:04.029 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:04.027+0000 7f318ce00640 1 -- 172.21.15.83:0/3198145213 --> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}) -- 0x7f3188062360 con 0x7f316c0a2c10 2024-08-19T01:33:04.033 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:04.032+0000 7f317f400640 1 -- 172.21.15.83:0/3198145213 <== mgr.14150 v2:172.21.15.83:6800/683740916 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+54 (secure 0 0 0) 0x7f3188062360 con 0x7f316c0a2c10 2024-08-19T01:33:04.034 INFO:teuthology.orchestra.run.smithi083.stdout:/volumes/qa/sv_1/05b773d9-8346-4397-99ec-d49959a55703 2024-08-19T01:33:04.036 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:04.034+0000 7f318ce00640 1 -- 172.21.15.83:0/3198145213 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f316c0a2c10 msgr2=0x7f316c0a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:33:04.036 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:04.034+0000 7f318ce00640 1 --2- 172.21.15.83:0/3198145213 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f316c0a2c10 0x7f316c0a50d0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f3178009900 tx=0x7f31780058d0 comp rx=0 tx=0).stop 2024-08-19T01:33:04.036 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:04.034+0000 7f318ce00640 1 -- 172.21.15.83:0/3198145213 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3188104480 msgr2=0x7f318819a900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:33:04.036 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:04.034+0000 7f318ce00640 1 --2- 172.21.15.83:0/3198145213 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3188104480 0x7f318819a900 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f317000ea60 tx=0x7f317000ef20 comp rx=0 tx=0).stop 2024-08-19T01:33:04.036 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:04.034+0000 7f318ce00640 1 -- 172.21.15.83:0/3198145213 shutdown_connections 2024-08-19T01:33:04.036 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:04.034+0000 7f318ce00640 1 --2- 172.21.15.83:0/3198145213 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f316c0a2c10 0x7f316c0a50d0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:33:04.036 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:04.034+0000 7f318ce00640 1 --2- 172.21.15.83:0/3198145213 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3188109b70 0x7f318819ec90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:33:04.036 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:04.034+0000 7f318ce00640 1 --2- 172.21.15.83:0/3198145213 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3188104480 0x7f318819a900 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:33:04.037 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:04.034+0000 7f318ce00640 1 --2- 172.21.15.83:0/3198145213 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3188103ae0 0x7f318819a3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:33:04.037 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:04.034+0000 7f318ce00640 1 -- 172.21.15.83:0/3198145213 >> 172.21.15.83:0/3198145213 conn(0x7f31880ff780 msgr2=0x7f3188100000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:33:04.037 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:04.034+0000 7f318ce00640 1 -- 172.21.15.83:0/3198145213 shutdown_connections 2024-08-19T01:33:04.037 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:04.034+0000 7f318ce00640 1 -- 172.21.15.83:0/3198145213 wait complete. 2024-08-19T01:33:04.764 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:33:04 smithi083 ceph-mon[17120]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.a", "id": [2, 11]}]: dispatch 2024-08-19T01:33:04.764 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:33:04 smithi083 ceph-mon[17120]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1a", "id": [8, 1]}]: dispatch 2024-08-19T01:33:04.764 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:33:04 smithi083 ceph-mon[17120]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1b", "id": [2, 4]}]: dispatch 2024-08-19T01:33:04.764 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:33:04 smithi083 ceph-mon[17120]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1d", "id": [8, 11]}]: dispatch 2024-08-19T01:33:04.764 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:33:04 smithi083 ceph-mon[17120]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.6", "id": [4, 6]}]: dispatch 2024-08-19T01:33:04.764 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:33:04 smithi083 ceph-mon[17120]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.11", "id": [4, 8]}]: dispatch 2024-08-19T01:33:04.764 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:33:04 smithi083 ceph-mon[17120]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1b", "id": [5, 9]}]: dispatch 2024-08-19T01:33:04.764 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:33:04 smithi083 ceph-mon[17120]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.20", "id": [5, 9]}]: dispatch 2024-08-19T01:33:04.764 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:33:04 smithi083 ceph-mon[17120]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.31", "id": [4, 7]}]: dispatch 2024-08-19T01:33:04.764 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:33:04 smithi083 ceph-mon[17120]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3b", "id": [3, 9]}]: dispatch 2024-08-19T01:33:04.806 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833 shell --fsid bad8780e-5dc9-11ef-bccf-c7b262605968 -- ceph fs dump --format=json 2024-08-19T01:33:04.952 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:33:04 smithi117 ceph-mon[22016]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.a", "id": [2, 11]}]: dispatch 2024-08-19T01:33:04.953 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:33:04 smithi117 ceph-mon[22016]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1a", "id": [8, 1]}]: dispatch 2024-08-19T01:33:04.953 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:33:04 smithi117 ceph-mon[22016]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1b", "id": [2, 4]}]: dispatch 2024-08-19T01:33:04.953 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:33:04 smithi117 ceph-mon[22016]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1d", "id": [8, 11]}]: dispatch 2024-08-19T01:33:04.953 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:33:04 smithi117 ceph-mon[22016]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.6", "id": [4, 6]}]: dispatch 2024-08-19T01:33:04.953 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:33:04 smithi117 ceph-mon[22016]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.11", "id": [4, 8]}]: dispatch 2024-08-19T01:33:04.953 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:33:04 smithi117 ceph-mon[22016]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1b", "id": [5, 9]}]: dispatch 2024-08-19T01:33:04.953 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:33:04 smithi117 ceph-mon[22016]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.20", "id": [5, 9]}]: dispatch 2024-08-19T01:33:04.953 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:33:04 smithi117 ceph-mon[22016]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.31", "id": [4, 7]}]: dispatch 2024-08-19T01:33:04.953 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:33:04 smithi117 ceph-mon[22016]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3b", "id": [3, 9]}]: dispatch 2024-08-19T01:33:04.966 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:33:04 smithi167 ceph-mon[21875]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.a", "id": [2, 11]}]: dispatch 2024-08-19T01:33:04.966 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:33:04 smithi167 ceph-mon[21875]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1a", "id": [8, 1]}]: dispatch 2024-08-19T01:33:04.966 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:33:04 smithi167 ceph-mon[21875]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1b", "id": [2, 4]}]: dispatch 2024-08-19T01:33:04.966 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:33:04 smithi167 ceph-mon[21875]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1d", "id": [8, 11]}]: dispatch 2024-08-19T01:33:04.966 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:33:04 smithi167 ceph-mon[21875]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.6", "id": [4, 6]}]: dispatch 2024-08-19T01:33:04.967 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:33:04 smithi167 ceph-mon[21875]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.11", "id": [4, 8]}]: dispatch 2024-08-19T01:33:04.967 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:33:04 smithi167 ceph-mon[21875]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1b", "id": [5, 9]}]: dispatch 2024-08-19T01:33:04.967 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:33:04 smithi167 ceph-mon[21875]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.20", "id": [5, 9]}]: dispatch 2024-08-19T01:33:04.967 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:33:04 smithi167 ceph-mon[21875]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.31", "id": [4, 7]}]: dispatch 2024-08-19T01:33:04.967 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:33:04 smithi167 ceph-mon[21875]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3b", "id": [3, 9]}]: dispatch 2024-08-19T01:33:05.181 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/bad8780e-5dc9-11ef-bccf-c7b262605968/mon.a/config 2024-08-19T01:33:05.593 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:33:05 smithi083 ceph-mon[17120]: from='client.14577 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-08-19T01:33:05.593 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:33:05 smithi083 ceph-mon[17120]: pgmap v284: 97 pgs: 97 active+clean; 603 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 4.5 KiB/s wr, 13 op/s 2024-08-19T01:33:05.593 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:33:05 smithi083 ceph-mon[17120]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.a", "id": [2, 11]}]': finished 2024-08-19T01:33:05.593 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:33:05 smithi083 ceph-mon[17120]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1a", "id": [8, 1]}]': finished 2024-08-19T01:33:05.593 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:33:05 smithi083 ceph-mon[17120]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1b", "id": [2, 4]}]': finished 2024-08-19T01:33:05.593 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:33:05 smithi083 ceph-mon[17120]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1d", "id": [8, 11]}]': finished 2024-08-19T01:33:05.593 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:33:05 smithi083 ceph-mon[17120]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.6", "id": [4, 6]}]': finished 2024-08-19T01:33:05.593 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:33:05 smithi083 ceph-mon[17120]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.11", "id": [4, 8]}]': finished 2024-08-19T01:33:05.593 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:33:05 smithi083 ceph-mon[17120]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1b", "id": [5, 9]}]': finished 2024-08-19T01:33:05.593 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:33:05 smithi083 ceph-mon[17120]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.20", "id": [5, 9]}]': finished 2024-08-19T01:33:05.594 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:33:05 smithi083 ceph-mon[17120]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.31", "id": [4, 7]}]': finished 2024-08-19T01:33:05.594 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:33:05 smithi083 ceph-mon[17120]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3b", "id": [3, 9]}]': finished 2024-08-19T01:33:05.594 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:33:05 smithi083 ceph-mon[17120]: osdmap e85: 12 total, 12 up, 12 in 2024-08-19T01:33:05.709 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:05.699+0000 7f0074000640 1 -- 172.21.15.83:0/4205844814 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f006c104480 msgr2=0x7f006c1109b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:33:05.709 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:05.699+0000 7f0074000640 1 --2- 172.21.15.83:0/4205844814 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f006c104480 0x7f006c1109b0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f0064009930 tx=0x7f006402a420 comp rx=0 tx=0).stop 2024-08-19T01:33:05.709 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:05.700+0000 7f0074000640 1 -- 172.21.15.83:0/4205844814 shutdown_connections 2024-08-19T01:33:05.709 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:05.700+0000 7f0074000640 1 --2- 172.21.15.83:0/4205844814 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f006c104480 0x7f006c1109b0 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:33:05.709 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:05.700+0000 7f0074000640 1 --2- 172.21.15.83:0/4205844814 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f006c103ae0 0x7f006c103f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:33:05.709 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:05.700+0000 7f0074000640 1 --2- 172.21.15.83:0/4205844814 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f006c109b70 0x7f006c109f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:33:05.709 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:05.700+0000 7f0074000640 1 -- 172.21.15.83:0/4205844814 >> 172.21.15.83:0/4205844814 conn(0x7f006c0ff780 msgr2=0x7f006c101ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:33:05.709 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:05.700+0000 7f0074000640 1 -- 172.21.15.83:0/4205844814 shutdown_connections 2024-08-19T01:33:05.709 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:05.700+0000 7f0074000640 1 -- 172.21.15.83:0/4205844814 wait complete. 2024-08-19T01:33:05.709 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:05.701+0000 7f0074000640 1 Processor -- start 2024-08-19T01:33:05.709 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:05.701+0000 7f0074000640 1 -- start start 2024-08-19T01:33:05.710 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:05.701+0000 7f0074000640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f006c103ae0 0x7f006c19a390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:33:05.710 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:05.702+0000 7f0074000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f006c104480 0x7f006c19a8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:33:05.710 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:05.702+0000 7f0074000640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f006c109b70 0x7f006c19ec60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:33:05.710 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:05.702+0000 7f0074000640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f006c113f30 con 0x7f006c104480 2024-08-19T01:33:05.710 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:05.702+0000 7f0074000640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f006c113db0 con 0x7f006c103ae0 2024-08-19T01:33:05.710 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:05.702+0000 7f0074000640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f006c1140b0 con 0x7f006c109b70 2024-08-19T01:33:05.710 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:05.702+0000 7f0071600640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f006c103ae0 0x7f006c19a390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:33:05.710 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:05.702+0000 7f0070c00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f006c104480 0x7f006c19a8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:33:05.710 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:05.702+0000 7f0071600640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f006c103ae0 0x7f006c19a390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.83:55536/0 (socket says 172.21.15.83:55536) 2024-08-19T01:33:05.710 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:05.702+0000 7f0071600640 1 -- 172.21.15.83:0/1589002656 learned_addr learned my addr 172.21.15.83:0/1589002656 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-08-19T01:33:05.710 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:05.702+0000 7f0072000640 1 --2- 172.21.15.83:0/1589002656 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f006c109b70 0x7f006c19ec60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:33:05.710 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:05.702+0000 7f0070c00640 1 -- 172.21.15.83:0/1589002656 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f006c109b70 msgr2=0x7f006c19ec60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:33:05.711 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:05.702+0000 7f0070c00640 1 --2- 172.21.15.83:0/1589002656 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f006c109b70 0x7f006c19ec60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:33:05.711 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:05.702+0000 7f0070c00640 1 -- 172.21.15.83:0/1589002656 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f006c103ae0 msgr2=0x7f006c19a390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:33:05.711 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:05.702+0000 7f0070c00640 1 --2- 172.21.15.83:0/1589002656 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f006c103ae0 0x7f006c19a390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:33:05.711 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:05.702+0000 7f0070c00640 1 -- 172.21.15.83:0/1589002656 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f006c19f340 con 0x7f006c104480 2024-08-19T01:33:05.711 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:05.703+0000 7f0070c00640 1 --2- 172.21.15.83:0/1589002656 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f006c104480 0x7f006c19a8d0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f005800d720 tx=0x7f005800dbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:33:05.711 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:05.703+0000 7f006a000640 1 -- 172.21.15.83:0/1589002656 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0058017070 con 0x7f006c104480 2024-08-19T01:33:05.711 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:05.703+0000 7f0074000640 1 -- 172.21.15.83:0/1589002656 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f006c19f630 con 0x7f006c104480 2024-08-19T01:33:05.711 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:05.703+0000 7f006a000640 1 -- 172.21.15.83:0/1589002656 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f005800de30 con 0x7f006c104480 2024-08-19T01:33:05.711 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:05.703+0000 7f006a000640 1 -- 172.21.15.83:0/1589002656 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00580047a0 con 0x7f006c104480 2024-08-19T01:33:05.711 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:05.703+0000 7f0074000640 1 -- 172.21.15.83:0/1589002656 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f006c1a6f10 con 0x7f006c104480 2024-08-19T01:33:05.711 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:05.704+0000 7f0074000640 1 -- 172.21.15.83:0/1589002656 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f006c103f40 con 0x7f006c104480 2024-08-19T01:33:05.711 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:05.706+0000 7f006a000640 1 -- 172.21.15.83:0/1589002656 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f00580099d0 con 0x7f006c104480 2024-08-19T01:33:05.712 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:05.708+0000 7f006a000640 1 --2- 172.21.15.83:0/1589002656 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f00540a2c10 0x7f00540a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:33:05.712 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:05.708+0000 7f006a000640 1 -- 172.21.15.83:0/1589002656 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8269+0+0 (secure 0 0 0) 0x7f00580ceb10 con 0x7f006c104480 2024-08-19T01:33:05.712 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:05.708+0000 7f0071600640 1 --2- 172.21.15.83:0/1589002656 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f00540a2c10 0x7f00540a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:33:05.712 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:05.709+0000 7f0071600640 1 --2- 172.21.15.83:0/1589002656 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f00540a2c10 0x7f00540a50d0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f0060009fa0 tx=0x7f0060009500 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:33:05.713 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:05.711+0000 7f006a000640 1 -- 172.21.15.83:0/1589002656 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7f0058020020 con 0x7f006c104480 2024-08-19T01:33:05.926 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:05.924+0000 7f0074000640 1 -- 172.21.15.83:0/1589002656 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f006c1a7140 con 0x7f006c104480 2024-08-19T01:33:05.927 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:05.925+0000 7f006a000640 1 -- 172.21.15.83:0/1589002656 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) ==== 94+0+11920 (secure 0 0 0) 0x7f005809b890 con 0x7f006c104480 2024-08-19T01:33:05.927 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-08-19T01:33:05.927 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":24,"btime":"2024-08-19T01:32:54:291077+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14421,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6835/988556067","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":988556067},{"type":"v1","addr":"172.21.15.83:6835","nonce":988556067}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14433,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6837/869702632","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6836","nonce":869702632},{"type":"v1","addr":"172.21.15.83:6837","nonce":869702632}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14475,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6841/1795975999","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6840","nonce":1795975999},{"type":"v1","addr":"172.21.15.83:6841","nonce":1795975999}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24430,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6833/2505076046","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6832","nonce":2505076046},{"type":"v1","addr":"172.21.15.167:6833","nonce":2505076046}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24449,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6833/3557520478","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":3557520478},{"type":"v1","addr":"172.21.15.117:6833","nonce":3557520478}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24466,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6835/3482045966","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":3482045966},{"type":"v1","addr":"172.21.15.167:6835","nonce":3482045966}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7}],"filesystems":[{"mdsmap":{"epoch":24,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-08-19T01:32:42.054769+0000","modified":"2024-08-19T01:32:54.209056+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24545,"mds_1":24527,"mds_2":24509},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24478":{"gid":24478,"name":"c","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.167:6837/1762130408","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6836","nonce":1762130408},{"type":"v1","addr":"172.21.15.167:6837","nonce":1762130408}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24485":{"gid":24485,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.117:6835/1024213994","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":1024213994},{"type":"v1","addr":"172.21.15.117:6835","nonce":1024213994}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24496":{"gid":24496,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.167:6839/3507387682","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6838","nonce":3507387682},{"type":"v1","addr":"172.21.15.167:6839","nonce":3507387682}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24509":{"gid":24509,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":10,"addr":"172.21.15.117:6837/3185224684","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":3185224684},{"type":"v1","addr":"172.21.15.117:6837","nonce":3185224684}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24527":{"gid":24527,"name":"g","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.83:6839/3239261845","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6838","nonce":3239261845},{"type":"v1","addr":"172.21.15.83:6839","nonce":3239261845}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24545":{"gid":24545,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.117:6839/1967595704","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6838","nonce":1967595704},{"type":"v1","addr":"172.21.15.117:6839","nonce":1967595704}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24545,"qdb_cluster":[24509,24527,24545]},"id":1}]} 2024-08-19T01:33:05.927 INFO:teuthology.orchestra.run.smithi083.stderr:dumped fsmap epoch 24 2024-08-19T01:33:05.929 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:05.927+0000 7f0074000640 1 -- 172.21.15.83:0/1589002656 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f00540a2c10 msgr2=0x7f00540a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:33:05.929 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:05.927+0000 7f0074000640 1 --2- 172.21.15.83:0/1589002656 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f00540a2c10 0x7f00540a50d0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f0060009fa0 tx=0x7f0060009500 comp rx=0 tx=0).stop 2024-08-19T01:33:05.929 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:05.927+0000 7f0074000640 1 -- 172.21.15.83:0/1589002656 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f006c104480 msgr2=0x7f006c19a8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:33:05.930 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:05.927+0000 7f0074000640 1 --2- 172.21.15.83:0/1589002656 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f006c104480 0x7f006c19a8d0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f005800d720 tx=0x7f005800dbe0 comp rx=0 tx=0).stop 2024-08-19T01:33:05.930 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:05.928+0000 7f0074000640 1 -- 172.21.15.83:0/1589002656 shutdown_connections 2024-08-19T01:33:05.930 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:05.928+0000 7f0074000640 1 --2- 172.21.15.83:0/1589002656 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f00540a2c10 0x7f00540a50d0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:33:05.930 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:05.928+0000 7f0074000640 1 --2- 172.21.15.83:0/1589002656 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f006c109b70 0x7f006c19ec60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:33:05.930 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:05.928+0000 7f0074000640 1 --2- 172.21.15.83:0/1589002656 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f006c104480 0x7f006c19a8d0 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:33:05.930 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:05.928+0000 7f0074000640 1 --2- 172.21.15.83:0/1589002656 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f006c103ae0 0x7f006c19a390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:33:05.930 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:05.928+0000 7f0074000640 1 -- 172.21.15.83:0/1589002656 >> 172.21.15.83:0/1589002656 conn(0x7f006c0ff780 msgr2=0x7f006c0fff20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:33:05.930 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:05.928+0000 7f0074000640 1 -- 172.21.15.83:0/1589002656 shutdown_connections 2024-08-19T01:33:05.930 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:05.928+0000 7f0074000640 1 -- 172.21.15.83:0/1589002656 wait complete. 2024-08-19T01:33:05.952 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:33:05 smithi117 ceph-mon[22016]: from='client.14577 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-08-19T01:33:05.953 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:33:05 smithi117 ceph-mon[22016]: pgmap v284: 97 pgs: 97 active+clean; 603 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 4.5 KiB/s wr, 13 op/s 2024-08-19T01:33:05.953 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:33:05 smithi117 ceph-mon[22016]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.a", "id": [2, 11]}]': finished 2024-08-19T01:33:05.953 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:33:05 smithi117 ceph-mon[22016]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1a", "id": [8, 1]}]': finished 2024-08-19T01:33:05.953 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:33:05 smithi117 ceph-mon[22016]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1b", "id": [2, 4]}]': finished 2024-08-19T01:33:05.953 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:33:05 smithi117 ceph-mon[22016]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1d", "id": [8, 11]}]': finished 2024-08-19T01:33:05.953 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:33:05 smithi117 ceph-mon[22016]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.6", "id": [4, 6]}]': finished 2024-08-19T01:33:05.953 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:33:05 smithi117 ceph-mon[22016]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.11", "id": [4, 8]}]': finished 2024-08-19T01:33:05.953 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:33:05 smithi117 ceph-mon[22016]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1b", "id": [5, 9]}]': finished 2024-08-19T01:33:05.953 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:33:05 smithi117 ceph-mon[22016]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.20", "id": [5, 9]}]': finished 2024-08-19T01:33:05.953 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:33:05 smithi117 ceph-mon[22016]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.31", "id": [4, 7]}]': finished 2024-08-19T01:33:05.953 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:33:05 smithi117 ceph-mon[22016]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3b", "id": [3, 9]}]': finished 2024-08-19T01:33:05.953 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:33:05 smithi117 ceph-mon[22016]: osdmap e85: 12 total, 12 up, 12 in 2024-08-19T01:33:05.966 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:33:05 smithi167 ceph-mon[21875]: from='client.14577 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-08-19T01:33:05.966 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:33:05 smithi167 ceph-mon[21875]: pgmap v284: 97 pgs: 97 active+clean; 603 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 4.5 KiB/s wr, 13 op/s 2024-08-19T01:33:05.966 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:33:05 smithi167 ceph-mon[21875]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.a", "id": [2, 11]}]': finished 2024-08-19T01:33:05.966 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:33:05 smithi167 ceph-mon[21875]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1a", "id": [8, 1]}]': finished 2024-08-19T01:33:05.966 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:33:05 smithi167 ceph-mon[21875]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1b", "id": [2, 4]}]': finished 2024-08-19T01:33:05.966 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:33:05 smithi167 ceph-mon[21875]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1d", "id": [8, 11]}]': finished 2024-08-19T01:33:05.966 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:33:05 smithi167 ceph-mon[21875]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.6", "id": [4, 6]}]': finished 2024-08-19T01:33:05.967 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:33:05 smithi167 ceph-mon[21875]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.11", "id": [4, 8]}]': finished 2024-08-19T01:33:05.967 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:33:05 smithi167 ceph-mon[21875]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1b", "id": [5, 9]}]': finished 2024-08-19T01:33:05.967 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:33:05 smithi167 ceph-mon[21875]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.20", "id": [5, 9]}]': finished 2024-08-19T01:33:05.967 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:33:05 smithi167 ceph-mon[21875]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.31", "id": [4, 7]}]': finished 2024-08-19T01:33:05.967 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:33:05 smithi167 ceph-mon[21875]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3b", "id": [3, 9]}]': finished 2024-08-19T01:33:05.967 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:33:05 smithi167 ceph-mon[21875]: osdmap e85: 12 total, 12 up, 12 in 2024-08-19T01:33:06.613 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:33:06 smithi083 ceph-mon[17120]: osdmap e86: 12 total, 12 up, 12 in 2024-08-19T01:33:06.613 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:33:06 smithi083 ceph-mon[17120]: from='client.? 172.21.15.83:0/1589002656' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-19T01:33:06.650 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833 shell --fsid bad8780e-5dc9-11ef-bccf-c7b262605968 -- ceph osd dump --format=json 2024-08-19T01:33:06.952 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:33:06 smithi117 ceph-mon[22016]: osdmap e86: 12 total, 12 up, 12 in 2024-08-19T01:33:06.953 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:33:06 smithi117 ceph-mon[22016]: from='client.? 172.21.15.83:0/1589002656' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-19T01:33:06.966 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:33:06 smithi167 ceph-mon[21875]: osdmap e86: 12 total, 12 up, 12 in 2024-08-19T01:33:06.966 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:33:06 smithi167 ceph-mon[21875]: from='client.? 172.21.15.83:0/1589002656' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-19T01:33:07.024 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/bad8780e-5dc9-11ef-bccf-c7b262605968/mon.a/config 2024-08-19T01:33:07.575 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:07.551+0000 7f43f6800640 1 -- 172.21.15.83:0/1251883386 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f43f0103ae0 msgr2=0x7f43f0103f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:33:07.575 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:07.551+0000 7f43f6800640 1 --2- 172.21.15.83:0/1251883386 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f43f0103ae0 0x7f43f0103f40 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f43dc009900 tx=0x7f43dc02a3f0 comp rx=0 tx=0).stop 2024-08-19T01:33:07.575 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:07.551+0000 7f43f6800640 1 -- 172.21.15.83:0/1251883386 shutdown_connections 2024-08-19T01:33:07.575 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:07.551+0000 7f43f6800640 1 --2- 172.21.15.83:0/1251883386 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f43f0104480 0x7f43f01109b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:33:07.575 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:07.551+0000 7f43f6800640 1 --2- 172.21.15.83:0/1251883386 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f43f0103ae0 0x7f43f0103f40 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:33:07.575 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:07.551+0000 7f43f6800640 1 --2- 172.21.15.83:0/1251883386 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f43f0109b70 0x7f43f0109f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:33:07.575 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:07.551+0000 7f43f6800640 1 -- 172.21.15.83:0/1251883386 >> 172.21.15.83:0/1251883386 conn(0x7f43f00ff780 msgr2=0x7f43f0101ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:33:07.575 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:07.551+0000 7f43f6800640 1 -- 172.21.15.83:0/1251883386 shutdown_connections 2024-08-19T01:33:07.575 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:07.551+0000 7f43f6800640 1 -- 172.21.15.83:0/1251883386 wait complete. 2024-08-19T01:33:07.575 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:07.552+0000 7f43f6800640 1 Processor -- start 2024-08-19T01:33:07.575 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:07.552+0000 7f43f6800640 1 -- start start 2024-08-19T01:33:07.576 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:07.552+0000 7f43f6800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f43f0103ae0 0x7f43f019a380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:33:07.576 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:07.552+0000 7f43f6800640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f43f0104480 0x7f43f019a8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:33:07.576 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:07.552+0000 7f43f6800640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f43f0109b70 0x7f43f019ec50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:33:07.576 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:07.552+0000 7f43f6800640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f43f0113ee0 con 0x7f43f0103ae0 2024-08-19T01:33:07.576 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:07.552+0000 7f43f6800640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f43f0113d60 con 0x7f43f0104480 2024-08-19T01:33:07.576 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:07.552+0000 7f43f6800640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f43f0114060 con 0x7f43f0109b70 2024-08-19T01:33:07.576 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:07.553+0000 7f43ef400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f43f0103ae0 0x7f43f019a380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:33:07.576 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:07.553+0000 7f43ef400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f43f0103ae0 0x7f43f019a380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:44826/0 (socket says 172.21.15.83:44826) 2024-08-19T01:33:07.576 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:07.553+0000 7f43ef400640 1 -- 172.21.15.83:0/4069608898 learned_addr learned my addr 172.21.15.83:0/4069608898 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-08-19T01:33:07.576 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:07.553+0000 7f43eea00640 1 --2- 172.21.15.83:0/4069608898 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f43f0104480 0x7f43f019a8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:33:07.576 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:07.553+0000 7f43efe00640 1 --2- 172.21.15.83:0/4069608898 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f43f0109b70 0x7f43f019ec50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:33:07.576 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:07.553+0000 7f43ef400640 1 -- 172.21.15.83:0/4069608898 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f43f0109b70 msgr2=0x7f43f019ec50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:33:07.576 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:07.553+0000 7f43ef400640 1 --2- 172.21.15.83:0/4069608898 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f43f0109b70 0x7f43f019ec50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:33:07.577 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:07.553+0000 7f43ef400640 1 -- 172.21.15.83:0/4069608898 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f43f0104480 msgr2=0x7f43f019a8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:33:07.577 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:07.553+0000 7f43ef400640 1 --2- 172.21.15.83:0/4069608898 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f43f0104480 0x7f43f019a8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:33:07.577 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:07.553+0000 7f43ef400640 1 -- 172.21.15.83:0/4069608898 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f43f019f330 con 0x7f43f0103ae0 2024-08-19T01:33:07.577 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:07.553+0000 7f43eea00640 1 --2- 172.21.15.83:0/4069608898 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f43f0104480 0x7f43f019a8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-19T01:33:07.577 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:07.553+0000 7f43ef400640 1 --2- 172.21.15.83:0/4069608898 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f43f0103ae0 0x7f43f019a380 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f43e000e9f0 tx=0x7f43e000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:33:07.577 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:07.554+0000 7f43e7e00640 1 -- 172.21.15.83:0/4069608898 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43e000cd80 con 0x7f43f0103ae0 2024-08-19T01:33:07.577 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:07.554+0000 7f43f6800640 1 -- 172.21.15.83:0/4069608898 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f43f019f620 con 0x7f43f0103ae0 2024-08-19T01:33:07.577 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:07.554+0000 7f43e7e00640 1 -- 172.21.15.83:0/4069608898 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f43e0010910 con 0x7f43f0103ae0 2024-08-19T01:33:07.577 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:07.554+0000 7f43e7e00640 1 -- 172.21.15.83:0/4069608898 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43e00188f0 con 0x7f43f0103ae0 2024-08-19T01:33:07.577 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:07.554+0000 7f43f6800640 1 -- 172.21.15.83:0/4069608898 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f43f01a6f00 con 0x7f43f0103ae0 2024-08-19T01:33:07.577 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:07.555+0000 7f43f6800640 1 -- 172.21.15.83:0/4069608898 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f43f01051e0 con 0x7f43f0103ae0 2024-08-19T01:33:07.577 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:07.557+0000 7f43e7e00640 1 -- 172.21.15.83:0/4069608898 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f43e0010ac0 con 0x7f43f0103ae0 2024-08-19T01:33:07.577 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:07.559+0000 7f43e7e00640 1 --2- 172.21.15.83:0/4069608898 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f43d40a2c10 0x7f43d40a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:33:07.578 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:07.559+0000 7f43e7e00640 1 -- 172.21.15.83:0/4069608898 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8269+0+0 (secure 0 0 0) 0x7f43e00cf5c0 con 0x7f43f0103ae0 2024-08-19T01:33:07.578 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:07.559+0000 7f43eea00640 1 --2- 172.21.15.83:0/4069608898 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f43d40a2c10 0x7f43d40a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:33:07.578 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:07.560+0000 7f43eea00640 1 --2- 172.21.15.83:0/4069608898 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f43d40a2c10 0x7f43d40a50d0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f43dc009230 tx=0x7f43dc0058d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:33:07.578 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:07.563+0000 7f43e7e00640 1 -- 172.21.15.83:0/4069608898 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7f43e009bcf0 con 0x7f43f0103ae0 2024-08-19T01:33:07.739 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:07.737+0000 7f43f6800640 1 -- 172.21.15.83:0/4069608898 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f43f010c050 con 0x7f43f0103ae0 2024-08-19T01:33:07.739 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:33:07 smithi083 ceph-mon[17120]: pgmap v287: 97 pgs: 1 peering, 96 active+clean; 621 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 5.7 KiB/s wr, 9 op/s 2024-08-19T01:33:07.740 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:07.739+0000 7f43e7e00640 1 -- 172.21.15.83:0/4069608898 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v86) ==== 74+0+22722 (secure 0 0 0) 0x7f43e009fff0 con 0x7f43f0103ae0 2024-08-19T01:33:07.741 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-08-19T01:33:07.741 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":86,"fsid":"bad8780e-5dc9-11ef-bccf-c7b262605968","created":"2024-08-19T01:25:12.626841+0000","modified":"2024-08-19T01:33:05.461998+0000","last_up_change":"2024-08-19T01:31:47.254559+0000","last_in_change":"2024-08-19T01:31:27.652847+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-08-19T01:28:16.862007+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-08-19T01:32:37.455535+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-08-19T01:32:40.201935+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ffa74041-f9d2-4555-bb43-508b8c581487","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6802","nonce":1082526824},{"type":"v1","addr":"172.21.15.83:6803","nonce":1082526824}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6804","nonce":1082526824},{"type":"v1","addr":"172.21.15.83:6805","nonce":1082526824}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6808","nonce":1082526824},{"type":"v1","addr":"172.21.15.83:6809","nonce":1082526824}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6806","nonce":1082526824},{"type":"v1","addr":"172.21.15.83:6807","nonce":1082526824}]},"public_addr":"172.21.15.83:6803/1082526824","cluster_addr":"172.21.15.83:6805/1082526824","heartbeat_back_addr":"172.21.15.83:6809/1082526824","heartbeat_front_addr":"172.21.15.83:6807/1082526824","state":["exists","up"]},{"osd":1,"uuid":"1977570b-b4f5-44fd-bb1e-3a31318be649","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6800","nonce":1925419802},{"type":"v1","addr":"172.21.15.117:6801","nonce":1925419802}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6802","nonce":1925419802},{"type":"v1","addr":"172.21.15.117:6803","nonce":1925419802}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6806","nonce":1925419802},{"type":"v1","addr":"172.21.15.117:6807","nonce":1925419802}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6804","nonce":1925419802},{"type":"v1","addr":"172.21.15.117:6805","nonce":1925419802}]},"public_addr":"172.21.15.117:6801/1925419802","cluster_addr":"172.21.15.117:6803/1925419802","heartbeat_back_addr":"172.21.15.117:6807/1925419802","heartbeat_front_addr":"172.21.15.117:6805/1925419802","state":["exists","up"]},{"osd":2,"uuid":"a75278d1-161f-41d5-a92f-058d1077bf2a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6800","nonce":551625328},{"type":"v1","addr":"172.21.15.167:6801","nonce":551625328}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6802","nonce":551625328},{"type":"v1","addr":"172.21.15.167:6803","nonce":551625328}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6806","nonce":551625328},{"type":"v1","addr":"172.21.15.167:6807","nonce":551625328}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6804","nonce":551625328},{"type":"v1","addr":"172.21.15.167:6805","nonce":551625328}]},"public_addr":"172.21.15.167:6801/551625328","cluster_addr":"172.21.15.167:6803/551625328","heartbeat_back_addr":"172.21.15.167:6807/551625328","heartbeat_front_addr":"172.21.15.167:6805/551625328","state":["exists","up"]},{"osd":3,"uuid":"d1fb16f9-910e-46fa-b1e3-8e5bef4d346d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6810","nonce":3138942062},{"type":"v1","addr":"172.21.15.83:6811","nonce":3138942062}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6812","nonce":3138942062},{"type":"v1","addr":"172.21.15.83:6813","nonce":3138942062}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6816","nonce":3138942062},{"type":"v1","addr":"172.21.15.83:6817","nonce":3138942062}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6814","nonce":3138942062},{"type":"v1","addr":"172.21.15.83:6815","nonce":3138942062}]},"public_addr":"172.21.15.83:6811/3138942062","cluster_addr":"172.21.15.83:6813/3138942062","heartbeat_back_addr":"172.21.15.83:6817/3138942062","heartbeat_front_addr":"172.21.15.83:6815/3138942062","state":["exists","up"]},{"osd":4,"uuid":"8176ed18-1fe2-446d-8994-8c60f6e08f38","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6808","nonce":3978863827},{"type":"v1","addr":"172.21.15.117:6809","nonce":3978863827}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6810","nonce":3978863827},{"type":"v1","addr":"172.21.15.117:6811","nonce":3978863827}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6814","nonce":3978863827},{"type":"v1","addr":"172.21.15.117:6815","nonce":3978863827}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6812","nonce":3978863827},{"type":"v1","addr":"172.21.15.117:6813","nonce":3978863827}]},"public_addr":"172.21.15.117:6809/3978863827","cluster_addr":"172.21.15.117:6811/3978863827","heartbeat_back_addr":"172.21.15.117:6815/3978863827","heartbeat_front_addr":"172.21.15.117:6813/3978863827","state":["exists","up"]},{"osd":5,"uuid":"e01e3f15-25c8-4d2c-be6f-a170f036064c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6808","nonce":2761858035},{"type":"v1","addr":"172.21.15.167:6809","nonce":2761858035}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6810","nonce":2761858035},{"type":"v1","addr":"172.21.15.167:6811","nonce":2761858035}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6814","nonce":2761858035},{"type":"v1","addr":"172.21.15.167:6815","nonce":2761858035}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6812","nonce":2761858035},{"type":"v1","addr":"172.21.15.167:6813","nonce":2761858035}]},"public_addr":"172.21.15.167:6809/2761858035","cluster_addr":"172.21.15.167:6811/2761858035","heartbeat_back_addr":"172.21.15.167:6815/2761858035","heartbeat_front_addr":"172.21.15.167:6813/2761858035","state":["exists","up"]},{"osd":6,"uuid":"78cf0a9b-9165-444f-b073-ebe13016de9d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6818","nonce":260644552},{"type":"v1","addr":"172.21.15.83:6819","nonce":260644552}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6820","nonce":260644552},{"type":"v1","addr":"172.21.15.83:6821","nonce":260644552}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6824","nonce":260644552},{"type":"v1","addr":"172.21.15.83:6825","nonce":260644552}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6822","nonce":260644552},{"type":"v1","addr":"172.21.15.83:6823","nonce":260644552}]},"public_addr":"172.21.15.83:6819/260644552","cluster_addr":"172.21.15.83:6821/260644552","heartbeat_back_addr":"172.21.15.83:6825/260644552","heartbeat_front_addr":"172.21.15.83:6823/260644552","state":["exists","up"]},{"osd":7,"uuid":"ee524f7f-672e-430e-9f58-1a9117513466","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6816","nonce":104113252},{"type":"v1","addr":"172.21.15.117:6817","nonce":104113252}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6818","nonce":104113252},{"type":"v1","addr":"172.21.15.117:6819","nonce":104113252}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6822","nonce":104113252},{"type":"v1","addr":"172.21.15.117:6823","nonce":104113252}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6820","nonce":104113252},{"type":"v1","addr":"172.21.15.117:6821","nonce":104113252}]},"public_addr":"172.21.15.117:6817/104113252","cluster_addr":"172.21.15.117:6819/104113252","heartbeat_back_addr":"172.21.15.117:6823/104113252","heartbeat_front_addr":"172.21.15.117:6821/104113252","state":["exists","up"]},{"osd":8,"uuid":"76d5317d-4845-4138-b210-e7bd2d80472a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":58,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6816","nonce":2196898857},{"type":"v1","addr":"172.21.15.167:6817","nonce":2196898857}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6818","nonce":2196898857},{"type":"v1","addr":"172.21.15.167:6819","nonce":2196898857}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6822","nonce":2196898857},{"type":"v1","addr":"172.21.15.167:6823","nonce":2196898857}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6820","nonce":2196898857},{"type":"v1","addr":"172.21.15.167:6821","nonce":2196898857}]},"public_addr":"172.21.15.167:6817/2196898857","cluster_addr":"172.21.15.167:6819/2196898857","heartbeat_back_addr":"172.21.15.167:6823/2196898857","heartbeat_front_addr":"172.21.15.167:6821/2196898857","state":["exists","up"]},{"osd":9,"uuid":"0151bc2a-3ad2-4db3-8505-5855792cc9bb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6826","nonce":903463450},{"type":"v1","addr":"172.21.15.83:6827","nonce":903463450}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6828","nonce":903463450},{"type":"v1","addr":"172.21.15.83:6829","nonce":903463450}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6832","nonce":903463450},{"type":"v1","addr":"172.21.15.83:6833","nonce":903463450}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6830","nonce":903463450},{"type":"v1","addr":"172.21.15.83:6831","nonce":903463450}]},"public_addr":"172.21.15.83:6827/903463450","cluster_addr":"172.21.15.83:6829/903463450","heartbeat_back_addr":"172.21.15.83:6833/903463450","heartbeat_front_addr":"172.21.15.83:6831/903463450","state":["exists","up"]},{"osd":10,"uuid":"437ed848-0da4-4cd4-8716-0c55bb99b676","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6824","nonce":1183570837},{"type":"v1","addr":"172.21.15.117:6825","nonce":1183570837}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6826","nonce":1183570837},{"type":"v1","addr":"172.21.15.117:6827","nonce":1183570837}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6830","nonce":1183570837},{"type":"v1","addr":"172.21.15.117:6831","nonce":1183570837}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6828","nonce":1183570837},{"type":"v1","addr":"172.21.15.117:6829","nonce":1183570837}]},"public_addr":"172.21.15.117:6825/1183570837","cluster_addr":"172.21.15.117:6827/1183570837","heartbeat_back_addr":"172.21.15.117:6831/1183570837","heartbeat_front_addr":"172.21.15.117:6829/1183570837","state":["exists","up"]},{"osd":11,"uuid":"13345f05-cdc0-4dae-a58b-551abbec1915","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":78,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6824","nonce":187027350},{"type":"v1","addr":"172.21.15.167:6825","nonce":187027350}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6826","nonce":187027350},{"type":"v1","addr":"172.21.15.167:6827","nonce":187027350}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6830","nonce":187027350},{"type":"v1","addr":"172.21.15.167:6831","nonce":187027350}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6828","nonce":187027350},{"type":"v1","addr":"172.21.15.167:6829","nonce":187027350}]},"public_addr":"172.21.15.167:6825/187027350","cluster_addr":"172.21.15.167:6827/187027350","heartbeat_back_addr":"172.21.15.167:6831/187027350","heartbeat_front_addr":"172.21.15.167:6829/187027350","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-19T01:27:28.760517+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-19T01:27:50.493028+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-19T01:28:11.997674+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-19T01:28:38.109934+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-19T01:29:00.173065+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-19T01:29:21.336085+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-19T01:29:47.871498+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-19T01:30:10.720498+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-19T01:30:32.425438+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-19T01:30:59.524989+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-19T01:31:22.806162+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-19T01:31:45.043246+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.1a","mappings":[{"from":8,"to":1}]},{"pgid":"2.1b","mappings":[{"from":2,"to":4}]},{"pgid":"2.1d","mappings":[{"from":8,"to":11}]},{"pgid":"3.6","mappings":[{"from":4,"to":6}]},{"pgid":"3.11","mappings":[{"from":4,"to":8}]},{"pgid":"3.1b","mappings":[{"from":5,"to":9}]},{"pgid":"3.20","mappings":[{"from":5,"to":9}]},{"pgid":"3.31","mappings":[{"from":4,"to":7}]},{"pgid":"3.3b","mappings":[{"from":3,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.83:0/3138095466":"2024-08-20T01:26:04.277716+0000","172.21.15.83:0/3835005198":"2024-08-20T01:26:04.277716+0000","172.21.15.83:0/1790374088":"2024-08-20T01:25:37.228583+0000","172.21.15.83:6801/573907319":"2024-08-20T01:26:04.277716+0000","172.21.15.83:6800/573907319":"2024-08-20T01:26:04.277716+0000","172.21.15.83:0/664676002":"2024-08-20T01:25:37.228583+0000","172.21.15.83:0/2144611449":"2024-08-20T01:25:37.228583+0000","172.21.15.83:6801/3261152818":"2024-08-20T01:25:37.228583+0000","172.21.15.83:0/1609552666":"2024-08-20T01:26:04.277716+0000","172.21.15.83:6800/3261152818":"2024-08-20T01:25:37.228583+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-08-19T01:33:07.743 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:07.741+0000 7f43f6800640 1 -- 172.21.15.83:0/4069608898 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f43d40a2c10 msgr2=0x7f43d40a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:33:07.743 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:07.741+0000 7f43f6800640 1 --2- 172.21.15.83:0/4069608898 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f43d40a2c10 0x7f43d40a50d0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f43dc009230 tx=0x7f43dc0058d0 comp rx=0 tx=0).stop 2024-08-19T01:33:07.743 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:07.741+0000 7f43f6800640 1 -- 172.21.15.83:0/4069608898 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f43f0103ae0 msgr2=0x7f43f019a380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:33:07.743 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:07.741+0000 7f43f6800640 1 --2- 172.21.15.83:0/4069608898 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f43f0103ae0 0x7f43f019a380 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f43e000e9f0 tx=0x7f43e000eeb0 comp rx=0 tx=0).stop 2024-08-19T01:33:07.743 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:07.741+0000 7f43f6800640 1 -- 172.21.15.83:0/4069608898 shutdown_connections 2024-08-19T01:33:07.743 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:07.741+0000 7f43f6800640 1 --2- 172.21.15.83:0/4069608898 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f43d40a2c10 0x7f43d40a50d0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:33:07.743 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:07.741+0000 7f43f6800640 1 --2- 172.21.15.83:0/4069608898 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f43f0109b70 0x7f43f019ec50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:33:07.743 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:07.741+0000 7f43f6800640 1 --2- 172.21.15.83:0/4069608898 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f43f0104480 0x7f43f019a8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:33:07.743 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:07.741+0000 7f43f6800640 1 --2- 172.21.15.83:0/4069608898 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f43f0103ae0 0x7f43f019a380 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:33:07.743 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:07.741+0000 7f43f6800640 1 -- 172.21.15.83:0/4069608898 >> 172.21.15.83:0/4069608898 conn(0x7f43f00ff780 msgr2=0x7f43f00ffff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:33:07.743 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:07.741+0000 7f43f6800640 1 -- 172.21.15.83:0/4069608898 shutdown_connections 2024-08-19T01:33:07.743 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:07.741+0000 7f43f6800640 1 -- 172.21.15.83:0/4069608898 wait complete. 2024-08-19T01:33:07.952 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:33:07 smithi117 ceph-mon[22016]: pgmap v287: 97 pgs: 1 peering, 96 active+clean; 621 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 5.7 KiB/s wr, 9 op/s 2024-08-19T01:33:07.966 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:33:07 smithi167 ceph-mon[21875]: pgmap v287: 97 pgs: 1 peering, 96 active+clean; 621 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 5.7 KiB/s wr, 9 op/s 2024-08-19T01:33:08.568 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2024-08-19T01:33:08.568 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833 shell --fsid bad8780e-5dc9-11ef-bccf-c7b262605968 -- ceph pg dump --format=json 2024-08-19T01:33:08.904 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:33:08 smithi083 ceph-mon[17120]: from='client.? 172.21.15.83:0/4069608898' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-08-19T01:33:08.940 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/bad8780e-5dc9-11ef-bccf-c7b262605968/mon.a/config 2024-08-19T01:33:08.952 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:33:08 smithi117 ceph-mon[22016]: from='client.? 172.21.15.83:0/4069608898' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-08-19T01:33:08.966 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:33:08 smithi167 ceph-mon[21875]: from='client.? 172.21.15.83:0/4069608898' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-08-19T01:33:09.508 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:09.472+0000 7f483b400640 1 -- 172.21.15.83:0/59157879 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4834102250 msgr2=0x7f483410e780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:33:09.508 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:09.472+0000 7f483b400640 1 --2- 172.21.15.83:0/59157879 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4834102250 0x7f483410e780 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f482c009cd0 tx=0x7f482c02a8e0 comp rx=0 tx=0).stop 2024-08-19T01:33:09.508 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:09.472+0000 7f483b400640 1 -- 172.21.15.83:0/59157879 shutdown_connections 2024-08-19T01:33:09.508 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:09.472+0000 7f483b400640 1 --2- 172.21.15.83:0/59157879 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4834102250 0x7f483410e780 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:33:09.509 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:09.472+0000 7f483b400640 1 --2- 172.21.15.83:0/59157879 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f48341018b0 0x7f4834101d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:33:09.509 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:09.472+0000 7f483b400640 1 --2- 172.21.15.83:0/59157879 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f48341078b0 0x7f4834107c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:33:09.509 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:09.472+0000 7f483b400640 1 -- 172.21.15.83:0/59157879 >> 172.21.15.83:0/59157879 conn(0x7f48340fd5f0 msgr2=0x7f48340ffa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:33:09.509 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:09.473+0000 7f483b400640 1 -- 172.21.15.83:0/59157879 shutdown_connections 2024-08-19T01:33:09.509 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:09.473+0000 7f483b400640 1 -- 172.21.15.83:0/59157879 wait complete. 2024-08-19T01:33:09.509 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:09.473+0000 7f483b400640 1 Processor -- start 2024-08-19T01:33:09.509 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:09.474+0000 7f483b400640 1 -- start start 2024-08-19T01:33:09.509 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:09.474+0000 7f483b400640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f48341018b0 0x7f483419a370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:33:09.509 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:09.474+0000 7f483b400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4834102250 0x7f483419a8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:33:09.509 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:09.474+0000 7f483b400640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f48341078b0 0x7f483419ec40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:33:09.509 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:09.474+0000 7f483b400640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f4834111d00 con 0x7f4834102250 2024-08-19T01:33:09.509 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:09.474+0000 7f483b400640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f4834111b80 con 0x7f48341078b0 2024-08-19T01:33:09.509 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:09.474+0000 7f483b400640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f4834111e80 con 0x7f48341018b0 2024-08-19T01:33:09.509 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:09.474+0000 7f4838a00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f48341018b0 0x7f483419a370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:33:09.510 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:09.474+0000 7f4838a00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f48341018b0 0x7f483419a370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.83:37442/0 (socket says 172.21.15.83:37442) 2024-08-19T01:33:09.510 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:09.474+0000 7f4838a00640 1 -- 172.21.15.83:0/905640870 learned_addr learned my addr 172.21.15.83:0/905640870 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-08-19T01:33:09.510 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:09.474+0000 7f482be00640 1 --2- 172.21.15.83:0/905640870 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4834102250 0x7f483419a8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:33:09.510 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:09.474+0000 7f4839400640 1 --2- 172.21.15.83:0/905640870 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f48341078b0 0x7f483419ec40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:33:09.510 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:09.475+0000 7f4838a00640 1 -- 172.21.15.83:0/905640870 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f48341078b0 msgr2=0x7f483419ec40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:33:09.510 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:09.475+0000 7f4838a00640 1 --2- 172.21.15.83:0/905640870 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f48341078b0 0x7f483419ec40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:33:09.510 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:09.475+0000 7f4838a00640 1 -- 172.21.15.83:0/905640870 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4834102250 msgr2=0x7f483419a8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:33:09.510 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:09.475+0000 7f4838a00640 1 --2- 172.21.15.83:0/905640870 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4834102250 0x7f483419a8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:33:09.510 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:09.475+0000 7f4838a00640 1 -- 172.21.15.83:0/905640870 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f483419f320 con 0x7f48341018b0 2024-08-19T01:33:09.510 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:09.475+0000 7f4838a00640 1 --2- 172.21.15.83:0/905640870 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f48341018b0 0x7f483419a370 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f482400e9f0 tx=0x7f482400edf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:33:09.510 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:09.475+0000 7f4832000640 1 -- 172.21.15.83:0/905640870 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f482400cd80 con 0x7f48341018b0 2024-08-19T01:33:09.510 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:09.475+0000 7f483b400640 1 -- 172.21.15.83:0/905640870 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f483419f610 con 0x7f48341018b0 2024-08-19T01:33:09.511 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:09.475+0000 7f4832000640 1 -- 172.21.15.83:0/905640870 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4824005500 con 0x7f48341018b0 2024-08-19T01:33:09.511 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:09.475+0000 7f4832000640 1 -- 172.21.15.83:0/905640870 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4824010690 con 0x7f48341018b0 2024-08-19T01:33:09.511 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:09.475+0000 7f483b400640 1 -- 172.21.15.83:0/905640870 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f48341a6ef0 con 0x7f48341018b0 2024-08-19T01:33:09.511 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:09.476+0000 7f483b400640 1 -- 172.21.15.83:0/905640870 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4834101d10 con 0x7f48341018b0 2024-08-19T01:33:09.511 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:09.479+0000 7f4832000640 1 -- 172.21.15.83:0/905640870 <== mon.1 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f48240046c0 con 0x7f48341018b0 2024-08-19T01:33:09.511 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:09.480+0000 7f4832000640 1 --2- 172.21.15.83:0/905640870 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f48180a2c90 0x7f48180a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:33:09.511 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:09.480+0000 7f4832000640 1 -- 172.21.15.83:0/905640870 <== mon.1 v2:172.21.15.167:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8269+0+0 (secure 0 0 0) 0x7f48240cf750 con 0x7f48341018b0 2024-08-19T01:33:09.511 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:09.480+0000 7f482be00640 1 --2- 172.21.15.83:0/905640870 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f48180a2c90 0x7f48180a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:33:09.511 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:09.480+0000 7f482be00640 1 --2- 172.21.15.83:0/905640870 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f48180a2c90 0x7f48180a5150 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f483410e3d0 tx=0x7f481c005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:33:09.511 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:09.482+0000 7f4832000640 1 -- 172.21.15.83:0/905640870 <== mon.1 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7f4824053df0 con 0x7f48341018b0 2024-08-19T01:33:09.654 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:33:09 smithi083 ceph-mon[17120]: pgmap v288: 97 pgs: 1 peering, 96 active+clean; 621 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 3.0 KiB/s wr, 7 op/s; 126 B/s, 1 keys/s, 0 objects/s recovering 2024-08-19T01:33:09.655 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:09.653+0000 7f483b400640 1 -- 172.21.15.83:0/905640870 --> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f483410da90 con 0x7f48180a2c90 2024-08-19T01:33:09.663 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:09.661+0000 7f4832000640 1 -- 172.21.15.83:0/905640870 <== mgr.14150 v2:172.21.15.83:6800/683740916 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258710 (secure 0 0 0) 0x7f483410da90 con 0x7f48180a2c90 2024-08-19T01:33:09.666 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-08-19T01:33:09.667 INFO:teuthology.orchestra.run.smithi083.stderr:dumped all 2024-08-19T01:33:09.670 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:09.665+0000 7f483b400640 1 -- 172.21.15.83:0/905640870 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f48180a2c90 msgr2=0x7f48180a5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:33:09.670 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:09.665+0000 7f483b400640 1 --2- 172.21.15.83:0/905640870 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f48180a2c90 0x7f48180a5150 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f483410e3d0 tx=0x7f481c005f90 comp rx=0 tx=0).stop 2024-08-19T01:33:09.670 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:09.665+0000 7f483b400640 1 -- 172.21.15.83:0/905640870 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f48341018b0 msgr2=0x7f483419a370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:33:09.670 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:09.665+0000 7f483b400640 1 --2- 172.21.15.83:0/905640870 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f48341018b0 0x7f483419a370 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f482400e9f0 tx=0x7f482400edf0 comp rx=0 tx=0).stop 2024-08-19T01:33:09.670 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:09.665+0000 7f483b400640 1 -- 172.21.15.83:0/905640870 shutdown_connections 2024-08-19T01:33:09.670 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:09.665+0000 7f483b400640 1 --2- 172.21.15.83:0/905640870 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f48180a2c90 0x7f48180a5150 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:33:09.670 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:09.665+0000 7f483b400640 1 --2- 172.21.15.83:0/905640870 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f48341078b0 0x7f483419ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:33:09.671 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:09.665+0000 7f483b400640 1 --2- 172.21.15.83:0/905640870 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4834102250 0x7f483419a8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:33:09.671 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:09.665+0000 7f483b400640 1 --2- 172.21.15.83:0/905640870 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f48341018b0 0x7f483419a370 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:33:09.671 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:09.665+0000 7f483b400640 1 -- 172.21.15.83:0/905640870 >> 172.21.15.83:0/905640870 conn(0x7f48340fd5f0 msgr2=0x7f483410d380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:33:09.671 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:09.666+0000 7f483b400640 1 -- 172.21.15.83:0/905640870 shutdown_connections 2024-08-19T01:33:09.671 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:09.666+0000 7f483b400640 1 -- 172.21.15.83:0/905640870 wait complete. 2024-08-19T01:33:09.952 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:33:09 smithi117 ceph-mon[22016]: pgmap v288: 97 pgs: 1 peering, 96 active+clean; 621 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 3.0 KiB/s wr, 7 op/s; 126 B/s, 1 keys/s, 0 objects/s recovering 2024-08-19T01:33:09.966 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:33:09 smithi167 ceph-mon[21875]: pgmap v288: 97 pgs: 1 peering, 96 active+clean; 621 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 3.0 KiB/s wr, 7 op/s; 126 B/s, 1 keys/s, 0 objects/s recovering 2024-08-19T01:33:10.366 INFO:teuthology.orchestra.run.smithi083.stdout:{"pg_ready":true,"pg_map":{"version":288,"stamp":"2024-08-19T01:33:08.364262+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":636054,"num_objects":64,"num_object_clones":0,"num_object_copies":192,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":235,"num_read_kb":211,"num_write":336,"num_write_kb":3027,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9,"num_bytes_recovered":1181748,"num_keys_recovered":10,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":269,"ondisk_log_size":269,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":290,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":374620,"kb_used_data":12932,"kb_used_omap":18,"kb_used_meta":361517,"kb_avail":1124567204,"statfs":{"total":1151940427776,"available":1151556816896,"internally_reserved":0,"allocated":13242368,"data_stored":7246233,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19143,"internal_metadata":370193721},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":18850,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":45,"num_read_kb":39,"num_write":17,"num_write_kb":24,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":1012,"num_keys_recovered":10,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.002470"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:33:05.470153+0000","last_change":"2024-08-19T01:32:41.244214+0000","last_active":"2024-08-19T01:33:05.470153+0000","last_peered":"2024-08-19T01:33:05.470153+0000","last_clean":"2024-08-19T01:33:05.470153+0000","last_became_active":"2024-08-19T01:32:41.243889+0000","last_became_peered":"2024-08-19T01:32:41.243889+0000","last_unstale":"2024-08-19T01:33:05.470153+0000","last_undegraded":"2024-08-19T01:33:05.470153+0000","last_fullsized":"2024-08-19T01:33:05.470153+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_clean_scrub_stamp":"2024-08-19T01:32:40.202519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T13:10:18.230055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:33:05.468954+0000","last_change":"2024-08-19T01:32:41.242729+0000","last_active":"2024-08-19T01:33:05.468954+0000","last_peered":"2024-08-19T01:33:05.468954+0000","last_clean":"2024-08-19T01:33:05.468954+0000","last_became_active":"2024-08-19T01:32:41.242436+0000","last_became_peered":"2024-08-19T01:32:41.242436+0000","last_unstale":"2024-08-19T01:33:05.468954+0000","last_undegraded":"2024-08-19T01:33:05.468954+0000","last_fullsized":"2024-08-19T01:33:05.468954+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_clean_scrub_stamp":"2024-08-19T01:32:40.202519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T13:19:32.010576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:33:05.468943+0000","last_change":"2024-08-19T01:32:41.247377+0000","last_active":"2024-08-19T01:33:05.468943+0000","last_peered":"2024-08-19T01:33:05.468943+0000","last_clean":"2024-08-19T01:33:05.468943+0000","last_became_active":"2024-08-19T01:32:41.246804+0000","last_became_peered":"2024-08-19T01:32:41.246804+0000","last_unstale":"2024-08-19T01:33:05.468943+0000","last_undegraded":"2024-08-19T01:33:05.468943+0000","last_fullsized":"2024-08-19T01:33:05.468943+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_clean_scrub_stamp":"2024-08-19T01:32:40.202519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T04:48:52.382687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:33:05.470397+0000","last_change":"2024-08-19T01:32:41.232900+0000","last_active":"2024-08-19T01:33:05.470397+0000","last_peered":"2024-08-19T01:33:05.470397+0000","last_clean":"2024-08-19T01:33:05.470397+0000","last_became_active":"2024-08-19T01:32:41.232573+0000","last_became_peered":"2024-08-19T01:32:41.232573+0000","last_unstale":"2024-08-19T01:33:05.470397+0000","last_undegraded":"2024-08-19T01:33:05.470397+0000","last_fullsized":"2024-08-19T01:33:05.470397+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_clean_scrub_stamp":"2024-08-19T01:32:40.202519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T09:42:36.309834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-08-19T01:32:42.218961+0000","last_change":"2024-08-19T01:32:41.233923+0000","last_active":"2024-08-19T01:32:42.218961+0000","last_peered":"2024-08-19T01:32:42.218961+0000","last_clean":"2024-08-19T01:32:42.218961+0000","last_became_active":"2024-08-19T01:32:41.233509+0000","last_became_peered":"2024-08-19T01:32:41.233509+0000","last_unstale":"2024-08-19T01:32:42.218961+0000","last_undegraded":"2024-08-19T01:32:42.218961+0000","last_fullsized":"2024-08-19T01:32:42.218961+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_clean_scrub_stamp":"2024-08-19T01:32:40.202519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T13:27:24.737584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-08-19T01:33:04.468554+0000","last_change":"2024-08-19T01:32:41.228263+0000","last_active":"2024-08-19T01:33:04.468554+0000","last_peered":"2024-08-19T01:33:04.468554+0000","last_clean":"2024-08-19T01:33:04.468554+0000","last_became_active":"2024-08-19T01:32:41.227765+0000","last_became_peered":"2024-08-19T01:32:41.227765+0000","last_unstale":"2024-08-19T01:33:04.468554+0000","last_undegraded":"2024-08-19T01:33:04.468554+0000","last_fullsized":"2024-08-19T01:33:04.468554+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_clean_scrub_stamp":"2024-08-19T01:32:40.202519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T11:21:18.817886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:33:05.474641+0000","last_change":"2024-08-19T01:32:41.235680+0000","last_active":"2024-08-19T01:33:05.474641+0000","last_peered":"2024-08-19T01:33:05.474641+0000","last_clean":"2024-08-19T01:33:05.474641+0000","last_became_active":"2024-08-19T01:32:41.234150+0000","last_became_peered":"2024-08-19T01:32:41.234150+0000","last_unstale":"2024-08-19T01:33:05.474641+0000","last_undegraded":"2024-08-19T01:33:05.474641+0000","last_fullsized":"2024-08-19T01:33:05.474641+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_clean_scrub_stamp":"2024-08-19T01:32:40.202519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T10:28:08.908632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-08-19T01:32:42.227107+0000","last_change":"2024-08-19T01:32:41.230170+0000","last_active":"2024-08-19T01:32:42.227107+0000","last_peered":"2024-08-19T01:32:42.227107+0000","last_clean":"2024-08-19T01:32:42.227107+0000","last_became_active":"2024-08-19T01:32:41.229467+0000","last_became_peered":"2024-08-19T01:32:41.229467+0000","last_unstale":"2024-08-19T01:32:42.227107+0000","last_undegraded":"2024-08-19T01:32:42.227107+0000","last_fullsized":"2024-08-19T01:32:42.227107+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_clean_scrub_stamp":"2024-08-19T01:32:40.202519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T10:24:15.832910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:33:05.701955+0000","last_change":"2024-08-19T01:32:41.230607+0000","last_active":"2024-08-19T01:33:05.701955+0000","last_peered":"2024-08-19T01:33:05.701955+0000","last_clean":"2024-08-19T01:33:05.701955+0000","last_became_active":"2024-08-19T01:32:41.229979+0000","last_became_peered":"2024-08-19T01:32:41.229979+0000","last_unstale":"2024-08-19T01:33:05.701955+0000","last_undegraded":"2024-08-19T01:33:05.701955+0000","last_fullsized":"2024-08-19T01:33:05.701955+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_clean_scrub_stamp":"2024-08-19T01:32:40.202519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T02:23:46.736139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-08-19T01:33:04.468912+0000","last_change":"2024-08-19T01:32:41.233576+0000","last_active":"2024-08-19T01:33:04.468912+0000","last_peered":"2024-08-19T01:33:04.468912+0000","last_clean":"2024-08-19T01:33:04.468912+0000","last_became_active":"2024-08-19T01:32:41.231279+0000","last_became_peered":"2024-08-19T01:32:41.231279+0000","last_unstale":"2024-08-19T01:33:04.468912+0000","last_undegraded":"2024-08-19T01:33:04.468912+0000","last_fullsized":"2024-08-19T01:33:04.468912+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_clean_scrub_stamp":"2024-08-19T01:32:40.202519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T07:32:45.341628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:33:05.473054+0000","last_change":"2024-08-19T01:32:41.233192+0000","last_active":"2024-08-19T01:33:05.473054+0000","last_peered":"2024-08-19T01:33:05.473054+0000","last_clean":"2024-08-19T01:33:05.473054+0000","last_became_active":"2024-08-19T01:32:41.230944+0000","last_became_peered":"2024-08-19T01:32:41.230944+0000","last_unstale":"2024-08-19T01:33:05.473054+0000","last_undegraded":"2024-08-19T01:33:05.473054+0000","last_fullsized":"2024-08-19T01:33:05.473054+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_clean_scrub_stamp":"2024-08-19T01:32:40.202519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T08:55:29.036252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:33:05.470408+0000","last_change":"2024-08-19T01:32:41.234216+0000","last_active":"2024-08-19T01:33:05.470408+0000","last_peered":"2024-08-19T01:33:05.470408+0000","last_clean":"2024-08-19T01:33:05.470408+0000","last_became_active":"2024-08-19T01:32:41.233906+0000","last_became_peered":"2024-08-19T01:32:41.233906+0000","last_unstale":"2024-08-19T01:33:05.470408+0000","last_undegraded":"2024-08-19T01:33:05.470408+0000","last_fullsized":"2024-08-19T01:33:05.470408+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_clean_scrub_stamp":"2024-08-19T01:32:40.202519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T08:35:01.764494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:33:05.477816+0000","last_change":"2024-08-19T01:32:41.230651+0000","last_active":"2024-08-19T01:33:05.477816+0000","last_peered":"2024-08-19T01:33:05.477816+0000","last_clean":"2024-08-19T01:33:05.477816+0000","last_became_active":"2024-08-19T01:32:41.230253+0000","last_became_peered":"2024-08-19T01:32:41.230253+0000","last_unstale":"2024-08-19T01:33:05.477816+0000","last_undegraded":"2024-08-19T01:33:05.477816+0000","last_fullsized":"2024-08-19T01:33:05.477816+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_clean_scrub_stamp":"2024-08-19T01:32:40.202519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T09:41:59.942821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:33:05.468782+0000","last_change":"2024-08-19T01:32:41.240082+0000","last_active":"2024-08-19T01:33:05.468782+0000","last_peered":"2024-08-19T01:33:05.468782+0000","last_clean":"2024-08-19T01:33:05.468782+0000","last_became_active":"2024-08-19T01:32:41.239711+0000","last_became_peered":"2024-08-19T01:32:41.239711+0000","last_unstale":"2024-08-19T01:33:05.468782+0000","last_undegraded":"2024-08-19T01:33:05.468782+0000","last_fullsized":"2024-08-19T01:33:05.468782+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_clean_scrub_stamp":"2024-08-19T01:32:40.202519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T12:48:59.122307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":23,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:33:05.474883+0000","last_change":"2024-08-19T01:33:05.474883+0000","last_active":"2024-08-19T01:33:05.474883+0000","last_peered":"2024-08-19T01:33:05.474883+0000","last_clean":"2024-08-19T01:33:05.474883+0000","last_became_active":"2024-08-19T01:33:05.474502+0000","last_became_peered":"2024-08-19T01:33:05.474502+0000","last_unstale":"2024-08-19T01:33:05.474883+0000","last_undegraded":"2024-08-19T01:33:05.474883+0000","last_fullsized":"2024-08-19T01:33:05.474883+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_clean_scrub_stamp":"2024-08-19T01:32:40.202519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T07:02:01.171103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2],"acting":[0,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:33:05.701404+0000","last_change":"2024-08-19T01:32:41.240899+0000","last_active":"2024-08-19T01:33:05.701404+0000","last_peered":"2024-08-19T01:33:05.701404+0000","last_clean":"2024-08-19T01:33:05.701404+0000","last_became_active":"2024-08-19T01:32:41.240532+0000","last_became_peered":"2024-08-19T01:32:41.240532+0000","last_unstale":"2024-08-19T01:33:05.701404+0000","last_undegraded":"2024-08-19T01:33:05.701404+0000","last_fullsized":"2024-08-19T01:33:05.701404+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_clean_scrub_stamp":"2024-08-19T01:32:40.202519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T08:17:47.035887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:33:05.470183+0000","last_change":"2024-08-19T01:32:41.237977+0000","last_active":"2024-08-19T01:33:05.470183+0000","last_peered":"2024-08-19T01:33:05.470183+0000","last_clean":"2024-08-19T01:33:05.470183+0000","last_became_active":"2024-08-19T01:32:41.237634+0000","last_became_peered":"2024-08-19T01:32:41.237634+0000","last_unstale":"2024-08-19T01:33:05.470183+0000","last_undegraded":"2024-08-19T01:33:05.470183+0000","last_fullsized":"2024-08-19T01:33:05.470183+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_clean_scrub_stamp":"2024-08-19T01:32:40.202519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T12:48:37.005972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:33:05.466272+0000","last_change":"2024-08-19T01:32:41.236498+0000","last_active":"2024-08-19T01:33:05.466272+0000","last_peered":"2024-08-19T01:33:05.466272+0000","last_clean":"2024-08-19T01:33:05.466272+0000","last_became_active":"2024-08-19T01:32:41.236122+0000","last_became_peered":"2024-08-19T01:32:41.236122+0000","last_unstale":"2024-08-19T01:33:05.466272+0000","last_undegraded":"2024-08-19T01:33:05.466272+0000","last_fullsized":"2024-08-19T01:33:05.466272+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_clean_scrub_stamp":"2024-08-19T01:32:40.202519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T02:31:23.648064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-08-19T01:33:04.472873+0000","last_change":"2024-08-19T01:32:41.234607+0000","last_active":"2024-08-19T01:33:04.472873+0000","last_peered":"2024-08-19T01:33:04.472873+0000","last_clean":"2024-08-19T01:33:04.472873+0000","last_became_active":"2024-08-19T01:32:41.232867+0000","last_became_peered":"2024-08-19T01:32:41.232867+0000","last_unstale":"2024-08-19T01:33:04.472873+0000","last_undegraded":"2024-08-19T01:33:04.472873+0000","last_fullsized":"2024-08-19T01:33:04.472873+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_clean_scrub_stamp":"2024-08-19T01:32:40.202519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T12:32:38.441848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:33:05.469342+0000","last_change":"2024-08-19T01:32:41.231705+0000","last_active":"2024-08-19T01:33:05.469342+0000","last_peered":"2024-08-19T01:33:05.469342+0000","last_clean":"2024-08-19T01:33:05.469342+0000","last_became_active":"2024-08-19T01:32:41.231262+0000","last_became_peered":"2024-08-19T01:32:41.231262+0000","last_unstale":"2024-08-19T01:33:05.469342+0000","last_undegraded":"2024-08-19T01:33:05.469342+0000","last_fullsized":"2024-08-19T01:33:05.469342+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_clean_scrub_stamp":"2024-08-19T01:32:40.202519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T10:41:15.025471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:33:05.477712+0000","last_change":"2024-08-19T01:32:41.243192+0000","last_active":"2024-08-19T01:33:05.477712+0000","last_peered":"2024-08-19T01:33:05.477712+0000","last_clean":"2024-08-19T01:33:05.477712+0000","last_became_active":"2024-08-19T01:32:41.242704+0000","last_became_peered":"2024-08-19T01:32:41.242704+0000","last_unstale":"2024-08-19T01:33:05.477712+0000","last_undegraded":"2024-08-19T01:33:05.477712+0000","last_fullsized":"2024-08-19T01:33:05.477712+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_clean_scrub_stamp":"2024-08-19T01:32:40.202519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T01:44:43.270213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:33:05.701950+0000","last_change":"2024-08-19T01:32:41.239908+0000","last_active":"2024-08-19T01:33:05.701950+0000","last_peered":"2024-08-19T01:33:05.701950+0000","last_clean":"2024-08-19T01:33:05.701950+0000","last_became_active":"2024-08-19T01:32:41.239438+0000","last_became_peered":"2024-08-19T01:32:41.239438+0000","last_unstale":"2024-08-19T01:33:05.701950+0000","last_undegraded":"2024-08-19T01:33:05.701950+0000","last_fullsized":"2024-08-19T01:33:05.701950+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_clean_scrub_stamp":"2024-08-19T01:32:40.202519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T11:03:26.822111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-08-19T01:32:42.227571+0000","last_change":"2024-08-19T01:32:41.226373+0000","last_active":"2024-08-19T01:32:42.227571+0000","last_peered":"2024-08-19T01:32:42.227571+0000","last_clean":"2024-08-19T01:32:42.227571+0000","last_became_active":"2024-08-19T01:32:41.226089+0000","last_became_peered":"2024-08-19T01:32:41.226089+0000","last_unstale":"2024-08-19T01:32:42.227571+0000","last_undegraded":"2024-08-19T01:32:42.227571+0000","last_fullsized":"2024-08-19T01:32:42.227571+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_clean_scrub_stamp":"2024-08-19T01:32:40.202519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T01:33:56.639300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:33:05.469289+0000","last_change":"2024-08-19T01:32:41.237829+0000","last_active":"2024-08-19T01:33:05.469289+0000","last_peered":"2024-08-19T01:33:05.469289+0000","last_clean":"2024-08-19T01:33:05.469289+0000","last_became_active":"2024-08-19T01:32:41.237518+0000","last_became_peered":"2024-08-19T01:32:41.237518+0000","last_unstale":"2024-08-19T01:33:05.469289+0000","last_undegraded":"2024-08-19T01:33:05.469289+0000","last_fullsized":"2024-08-19T01:33:05.469289+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_clean_scrub_stamp":"2024-08-19T01:32:40.202519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T02:25:11.307658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-08-19T01:32:42.229621+0000","last_change":"2024-08-19T01:32:41.234508+0000","last_active":"2024-08-19T01:32:42.229621+0000","last_peered":"2024-08-19T01:32:42.229621+0000","last_clean":"2024-08-19T01:32:42.229621+0000","last_became_active":"2024-08-19T01:32:41.232869+0000","last_became_peered":"2024-08-19T01:32:41.232869+0000","last_unstale":"2024-08-19T01:32:42.229621+0000","last_undegraded":"2024-08-19T01:32:42.229621+0000","last_fullsized":"2024-08-19T01:32:42.229621+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_clean_scrub_stamp":"2024-08-19T01:32:40.202519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T11:20:55.097576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-08-19T01:33:04.472566+0000","last_change":"2024-08-19T01:32:41.230536+0000","last_active":"2024-08-19T01:33:04.472566+0000","last_peered":"2024-08-19T01:33:04.472566+0000","last_clean":"2024-08-19T01:33:04.472566+0000","last_became_active":"2024-08-19T01:32:41.230179+0000","last_became_peered":"2024-08-19T01:32:41.230179+0000","last_unstale":"2024-08-19T01:33:04.472566+0000","last_undegraded":"2024-08-19T01:33:04.472566+0000","last_fullsized":"2024-08-19T01:33:04.472566+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_clean_scrub_stamp":"2024-08-19T01:32:40.202519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T02:25:09.441363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:33:05.469818+0000","last_change":"2024-08-19T01:32:41.230695+0000","last_active":"2024-08-19T01:33:05.469818+0000","last_peered":"2024-08-19T01:33:05.469818+0000","last_clean":"2024-08-19T01:33:05.469818+0000","last_became_active":"2024-08-19T01:32:41.230324+0000","last_became_peered":"2024-08-19T01:32:41.230324+0000","last_unstale":"2024-08-19T01:33:05.469818+0000","last_undegraded":"2024-08-19T01:33:05.469818+0000","last_fullsized":"2024-08-19T01:33:05.469818+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_clean_scrub_stamp":"2024-08-19T01:32:40.202519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T05:38:09.317605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:33:05.477784+0000","last_change":"2024-08-19T01:32:41.238325+0000","last_active":"2024-08-19T01:33:05.477784+0000","last_peered":"2024-08-19T01:33:05.477784+0000","last_clean":"2024-08-19T01:33:05.477784+0000","last_became_active":"2024-08-19T01:32:41.237768+0000","last_became_peered":"2024-08-19T01:32:41.237768+0000","last_unstale":"2024-08-19T01:33:05.477784+0000","last_undegraded":"2024-08-19T01:33:05.477784+0000","last_fullsized":"2024-08-19T01:33:05.477784+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_clean_scrub_stamp":"2024-08-19T01:32:40.202519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T08:45:06.866241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:33:05.468967+0000","last_change":"2024-08-19T01:32:41.246950+0000","last_active":"2024-08-19T01:33:05.468967+0000","last_peered":"2024-08-19T01:33:05.468967+0000","last_clean":"2024-08-19T01:33:05.468967+0000","last_became_active":"2024-08-19T01:32:41.245441+0000","last_became_peered":"2024-08-19T01:32:41.245441+0000","last_unstale":"2024-08-19T01:33:05.468967+0000","last_undegraded":"2024-08-19T01:33:05.468967+0000","last_fullsized":"2024-08-19T01:33:05.468967+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_clean_scrub_stamp":"2024-08-19T01:32:40.202519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T03:11:06.921258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-08-19T01:33:04.473945+0000","last_change":"2024-08-19T01:32:41.244810+0000","last_active":"2024-08-19T01:33:04.473945+0000","last_peered":"2024-08-19T01:33:04.473945+0000","last_clean":"2024-08-19T01:33:04.473945+0000","last_became_active":"2024-08-19T01:32:41.244401+0000","last_became_peered":"2024-08-19T01:32:41.244401+0000","last_unstale":"2024-08-19T01:33:04.473945+0000","last_undegraded":"2024-08-19T01:33:04.473945+0000","last_fullsized":"2024-08-19T01:33:04.473945+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_clean_scrub_stamp":"2024-08-19T01:32:40.202519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T04:54:57.728181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-08-19T01:33:04.473006+0000","last_change":"2024-08-19T01:32:41.235977+0000","last_active":"2024-08-19T01:33:04.473006+0000","last_peered":"2024-08-19T01:33:04.473006+0000","last_clean":"2024-08-19T01:33:04.473006+0000","last_became_active":"2024-08-19T01:32:41.235693+0000","last_became_peered":"2024-08-19T01:32:41.235693+0000","last_unstale":"2024-08-19T01:33:04.473006+0000","last_undegraded":"2024-08-19T01:33:04.473006+0000","last_fullsized":"2024-08-19T01:33:04.473006+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_clean_scrub_stamp":"2024-08-19T01:32:40.202519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T04:59:15.766532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":23,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:33:05.473318+0000","last_change":"2024-08-19T01:33:05.473318+0000","last_active":"2024-08-19T01:33:05.473318+0000","last_peered":"2024-08-19T01:33:05.473318+0000","last_clean":"2024-08-19T01:33:05.473318+0000","last_became_active":"2024-08-19T01:33:05.472885+0000","last_became_peered":"2024-08-19T01:33:05.472885+0000","last_unstale":"2024-08-19T01:33:05.473318+0000","last_undegraded":"2024-08-19T01:33:05.473318+0000","last_fullsized":"2024-08-19T01:33:05.473318+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_clean_scrub_stamp":"2024-08-19T01:32:40.202519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T07:49:35.525930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,9],"acting":[11,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-08-19T01:32:42.218958+0000","last_change":"2024-08-19T01:32:41.230846+0000","last_active":"2024-08-19T01:32:42.218958+0000","last_peered":"2024-08-19T01:32:42.218958+0000","last_clean":"2024-08-19T01:32:42.218958+0000","last_became_active":"2024-08-19T01:32:41.230405+0000","last_became_peered":"2024-08-19T01:32:41.230405+0000","last_unstale":"2024-08-19T01:32:42.218958+0000","last_undegraded":"2024-08-19T01:32:42.218958+0000","last_fullsized":"2024-08-19T01:32:42.218958+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_clean_scrub_stamp":"2024-08-19T01:32:40.202519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T08:59:46.241898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1b","version":"84'6","reported_seq":59,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:33:06.478738+0000","last_change":"2024-08-19T01:33:05.479889+0000","last_active":"2024-08-19T01:33:06.478738+0000","last_peered":"2024-08-19T01:33:06.478738+0000","last_clean":"2024-08-19T01:33:06.478738+0000","last_became_active":"2024-08-19T01:33:05.474698+0000","last_became_peered":"2024-08-19T01:33:05.474698+0000","last_unstale":"2024-08-19T01:33:06.478738+0000","last_undegraded":"2024-08-19T01:33:06.478738+0000","last_fullsized":"2024-08-19T01:33:06.478738+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:32:37.889224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:32:37.889224+0000","last_clean_scrub_stamp":"2024-08-19T01:32:37.889224+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T08:41:29.053471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1012,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":17,"num_read_kb":3,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":1012,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7],"acting":[5,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":11,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:33:05.475899+0000","last_change":"2024-08-19T01:33:05.474083+0000","last_active":"2024-08-19T01:33:05.475899+0000","last_peered":"2024-08-19T01:33:05.475899+0000","last_clean":"2024-08-19T01:33:05.475899+0000","last_became_active":"2024-08-19T01:33:05.473718+0000","last_became_peered":"2024-08-19T01:33:05.473718+0000","last_unstale":"2024-08-19T01:33:05.475899+0000","last_undegraded":"2024-08-19T01:33:05.475899+0000","last_fullsized":"2024-08-19T01:33:05.475899+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_clean_scrub_stamp":"2024-08-19T01:32:40.202519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T02:25:11.955170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,4,3],"acting":[9,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.1a","version":"84'3","reported_seq":22,"reported_epoch":84,"state":"peering","last_fresh":"2024-08-19T01:32:51.290137+0000","last_change":"2024-08-19T01:33:04.468784+0000","last_active":"2024-08-19T01:33:04.468784+0000","last_peered":"2024-08-19T01:32:51.290137+0000","last_clean":"2024-08-19T01:32:51.290137+0000","last_became_active":"2024-08-19T01:32:38.911397+0000","last_became_peered":"2024-08-19T01:32:38.911397+0000","last_unstale":"2024-08-19T01:32:51.290137+0000","last_undegraded":"2024-08-19T01:32:51.290137+0000","last_fullsized":"2024-08-19T01:32:51.290137+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:32:37.889224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:32:37.889224+0000","last_clean_scrub_stamp":"2024-08-19T01:32:37.889224+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,6],"acting":[10,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1,6,8],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.19","version":"84'2","reported_seq":24,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:33:05.477490+0000","last_change":"2024-08-19T01:32:38.911757+0000","last_active":"2024-08-19T01:33:05.477490+0000","last_peered":"2024-08-19T01:33:05.477490+0000","last_clean":"2024-08-19T01:33:05.477490+0000","last_became_active":"2024-08-19T01:32:38.910832+0000","last_became_peered":"2024-08-19T01:32:38.910832+0000","last_unstale":"2024-08-19T01:33:05.477490+0000","last_undegraded":"2024-08-19T01:33:05.477490+0000","last_fullsized":"2024-08-19T01:33:05.477490+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:32:37.889224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:32:37.889224+0000","last_clean_scrub_stamp":"2024-08-19T01:32:37.889224+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T04:46:54.986693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-08-19T01:32:42.227161+0000","last_change":"2024-08-19T01:32:41.231130+0000","last_active":"2024-08-19T01:32:42.227161+0000","last_peered":"2024-08-19T01:32:42.227161+0000","last_clean":"2024-08-19T01:32:42.227161+0000","last_became_active":"2024-08-19T01:32:41.230883+0000","last_became_peered":"2024-08-19T01:32:41.230883+0000","last_unstale":"2024-08-19T01:32:42.227161+0000","last_undegraded":"2024-08-19T01:32:42.227161+0000","last_fullsized":"2024-08-19T01:32:42.227161+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_clean_scrub_stamp":"2024-08-19T01:32:40.202519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T06:33:11.856202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"84'1","reported_seq":23,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:33:05.470862+0000","last_change":"2024-08-19T01:32:38.916868+0000","last_active":"2024-08-19T01:33:05.470862+0000","last_peered":"2024-08-19T01:33:05.470862+0000","last_clean":"2024-08-19T01:33:05.470862+0000","last_became_active":"2024-08-19T01:32:38.916500+0000","last_became_peered":"2024-08-19T01:32:38.916500+0000","last_unstale":"2024-08-19T01:33:05.470862+0000","last_undegraded":"2024-08-19T01:33:05.470862+0000","last_fullsized":"2024-08-19T01:33:05.470862+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:32:37.889224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:32:37.889224+0000","last_clean_scrub_stamp":"2024-08-19T01:32:37.889224+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T12:30:07.085829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:33:05.465832+0000","last_change":"2024-08-19T01:32:41.235290+0000","last_active":"2024-08-19T01:33:05.465832+0000","last_peered":"2024-08-19T01:33:05.465832+0000","last_clean":"2024-08-19T01:33:05.465832+0000","last_became_active":"2024-08-19T01:32:41.235013+0000","last_became_peered":"2024-08-19T01:32:41.235013+0000","last_unstale":"2024-08-19T01:33:05.465832+0000","last_undegraded":"2024-08-19T01:33:05.465832+0000","last_fullsized":"2024-08-19T01:33:05.465832+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_clean_scrub_stamp":"2024-08-19T01:32:40.202519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T09:50:14.837080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"84'3","reported_seq":36,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-08-19T01:33:04.468627+0000","last_change":"2024-08-19T01:32:38.912023+0000","last_active":"2024-08-19T01:33:04.468627+0000","last_peered":"2024-08-19T01:33:04.468627+0000","last_clean":"2024-08-19T01:33:04.468627+0000","last_became_active":"2024-08-19T01:32:38.911679+0000","last_became_peered":"2024-08-19T01:32:38.911679+0000","last_unstale":"2024-08-19T01:33:04.468627+0000","last_undegraded":"2024-08-19T01:33:04.468627+0000","last_fullsized":"2024-08-19T01:33:04.468627+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:32:37.889224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:32:37.889224+0000","last_clean_scrub_stamp":"2024-08-19T01:32:37.889224+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T12:08:51.118338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":13,"num_read_kb":13,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:33:05.701747+0000","last_change":"2024-08-19T01:32:41.232264+0000","last_active":"2024-08-19T01:33:05.701747+0000","last_peered":"2024-08-19T01:33:05.701747+0000","last_clean":"2024-08-19T01:33:05.701747+0000","last_became_active":"2024-08-19T01:32:41.231788+0000","last_became_peered":"2024-08-19T01:32:41.231788+0000","last_unstale":"2024-08-19T01:33:05.701747+0000","last_undegraded":"2024-08-19T01:33:05.701747+0000","last_fullsized":"2024-08-19T01:33:05.701747+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_clean_scrub_stamp":"2024-08-19T01:32:40.202519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T04:59:06.443992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"84'2","reported_seq":24,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:33:05.472954+0000","last_change":"2024-08-19T01:32:38.915770+0000","last_active":"2024-08-19T01:33:05.472954+0000","last_peered":"2024-08-19T01:33:05.472954+0000","last_clean":"2024-08-19T01:33:05.472954+0000","last_became_active":"2024-08-19T01:32:38.915378+0000","last_became_peered":"2024-08-19T01:32:38.915378+0000","last_unstale":"2024-08-19T01:33:05.472954+0000","last_undegraded":"2024-08-19T01:33:05.472954+0000","last_fullsized":"2024-08-19T01:33:05.472954+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:32:37.889224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:32:37.889224+0000","last_clean_scrub_stamp":"2024-08-19T01:32:37.889224+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T09:16:14.513172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:33:05.469512+0000","last_change":"2024-08-19T01:32:41.243476+0000","last_active":"2024-08-19T01:33:05.469512+0000","last_peered":"2024-08-19T01:33:05.469512+0000","last_clean":"2024-08-19T01:33:05.469512+0000","last_became_active":"2024-08-19T01:32:41.243190+0000","last_became_peered":"2024-08-19T01:32:41.243190+0000","last_unstale":"2024-08-19T01:33:05.469512+0000","last_undegraded":"2024-08-19T01:33:05.469512+0000","last_fullsized":"2024-08-19T01:33:05.469512+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_clean_scrub_stamp":"2024-08-19T01:32:40.202519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T12:07:34.096919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"84'4","reported_seq":27,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:33:05.470664+0000","last_change":"2024-08-19T01:32:39.051458+0000","last_active":"2024-08-19T01:33:05.470664+0000","last_peered":"2024-08-19T01:33:05.470664+0000","last_clean":"2024-08-19T01:33:05.470664+0000","last_became_active":"2024-08-19T01:32:39.051113+0000","last_became_peered":"2024-08-19T01:32:39.051113+0000","last_unstale":"2024-08-19T01:33:05.470664+0000","last_undegraded":"2024-08-19T01:33:05.470664+0000","last_fullsized":"2024-08-19T01:33:05.470664+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:32:37.889224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:32:37.889224+0000","last_clean_scrub_stamp":"2024-08-19T01:32:37.889224+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T02:15:51.371054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:33:05.701579+0000","last_change":"2024-08-19T01:32:41.236453+0000","last_active":"2024-08-19T01:33:05.701579+0000","last_peered":"2024-08-19T01:33:05.701579+0000","last_clean":"2024-08-19T01:33:05.701579+0000","last_became_active":"2024-08-19T01:32:41.236135+0000","last_became_peered":"2024-08-19T01:32:41.236135+0000","last_unstale":"2024-08-19T01:33:05.701579+0000","last_undegraded":"2024-08-19T01:33:05.701579+0000","last_fullsized":"2024-08-19T01:33:05.701579+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_clean_scrub_stamp":"2024-08-19T01:32:40.202519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T12:38:41.398299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"84'21","reported_seq":73,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-08-19T01:33:04.468623+0000","last_change":"2024-08-19T01:32:38.914831+0000","last_active":"2024-08-19T01:33:04.468623+0000","last_peered":"2024-08-19T01:33:04.468623+0000","last_clean":"2024-08-19T01:33:04.468623+0000","last_became_active":"2024-08-19T01:32:38.914326+0000","last_became_peered":"2024-08-19T01:32:38.914326+0000","last_unstale":"2024-08-19T01:33:04.468623+0000","last_undegraded":"2024-08-19T01:33:04.468623+0000","last_fullsized":"2024-08-19T01:33:04.468623+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:32:37.889224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:32:37.889224+0000","last_clean_scrub_stamp":"2024-08-19T01:32:37.889224+0000","objects_scrubbed":0,"log_size":21,"log_dups_size":0,"ondisk_log_size":21,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T02:15:25.263974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41062,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":32,"num_read_kb":55,"num_write":22,"num_write_kb":45,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-08-19T01:32:42.219179+0000","last_change":"2024-08-19T01:32:41.237945+0000","last_active":"2024-08-19T01:32:42.219179+0000","last_peered":"2024-08-19T01:32:42.219179+0000","last_clean":"2024-08-19T01:32:42.219179+0000","last_became_active":"2024-08-19T01:32:41.237362+0000","last_became_peered":"2024-08-19T01:32:41.237362+0000","last_unstale":"2024-08-19T01:32:42.219179+0000","last_undegraded":"2024-08-19T01:32:42.219179+0000","last_fullsized":"2024-08-19T01:32:42.219179+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_clean_scrub_stamp":"2024-08-19T01:32:40.202519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T07:29:17.180054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"84'6","reported_seq":27,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-08-19T01:33:04.473010+0000","last_change":"2024-08-19T01:32:38.912453+0000","last_active":"2024-08-19T01:33:04.473010+0000","last_peered":"2024-08-19T01:33:04.473010+0000","last_clean":"2024-08-19T01:33:04.473010+0000","last_became_active":"2024-08-19T01:32:38.912051+0000","last_became_peered":"2024-08-19T01:32:38.912051+0000","last_unstale":"2024-08-19T01:33:04.473010+0000","last_undegraded":"2024-08-19T01:33:04.473010+0000","last_fullsized":"2024-08-19T01:33:04.473010+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:32:37.889224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:32:37.889224+0000","last_clean_scrub_stamp":"2024-08-19T01:32:37.889224+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T06:49:58.810134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:33:05.477644+0000","last_change":"2024-08-19T01:32:41.236282+0000","last_active":"2024-08-19T01:33:05.477644+0000","last_peered":"2024-08-19T01:33:05.477644+0000","last_clean":"2024-08-19T01:33:05.477644+0000","last_became_active":"2024-08-19T01:32:41.235559+0000","last_became_peered":"2024-08-19T01:32:41.235559+0000","last_unstale":"2024-08-19T01:33:05.477644+0000","last_undegraded":"2024-08-19T01:33:05.477644+0000","last_fullsized":"2024-08-19T01:33:05.477644+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_clean_scrub_stamp":"2024-08-19T01:32:40.202519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T07:51:52.157094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"84'3","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:33:05.470566+0000","last_change":"2024-08-19T01:32:38.915688+0000","last_active":"2024-08-19T01:33:05.470566+0000","last_peered":"2024-08-19T01:33:05.470566+0000","last_clean":"2024-08-19T01:33:05.470566+0000","last_became_active":"2024-08-19T01:32:38.915297+0000","last_became_peered":"2024-08-19T01:32:38.915297+0000","last_unstale":"2024-08-19T01:33:05.470566+0000","last_undegraded":"2024-08-19T01:33:05.470566+0000","last_fullsized":"2024-08-19T01:33:05.470566+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:32:37.889224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:32:37.889224+0000","last_clean_scrub_stamp":"2024-08-19T01:32:37.889224+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T01:38:12.780588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:33:05.472886+0000","last_change":"2024-08-19T01:32:41.233766+0000","last_active":"2024-08-19T01:33:05.472886+0000","last_peered":"2024-08-19T01:33:05.472886+0000","last_clean":"2024-08-19T01:33:05.472886+0000","last_became_active":"2024-08-19T01:32:41.233425+0000","last_became_peered":"2024-08-19T01:32:41.233425+0000","last_unstale":"2024-08-19T01:33:05.472886+0000","last_undegraded":"2024-08-19T01:33:05.472886+0000","last_fullsized":"2024-08-19T01:33:05.472886+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_clean_scrub_stamp":"2024-08-19T01:32:40.202519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T10:29:15.471639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"84'4","reported_seq":45,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:33:06.319643+0000","last_change":"2024-08-19T01:32:38.912610+0000","last_active":"2024-08-19T01:33:06.319643+0000","last_peered":"2024-08-19T01:33:06.319643+0000","last_clean":"2024-08-19T01:33:06.319643+0000","last_became_active":"2024-08-19T01:32:38.912040+0000","last_became_peered":"2024-08-19T01:32:38.912040+0000","last_unstale":"2024-08-19T01:33:06.319643+0000","last_undegraded":"2024-08-19T01:33:06.319643+0000","last_fullsized":"2024-08-19T01:33:06.319643+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:32:37.889224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:32:37.889224+0000","last_clean_scrub_stamp":"2024-08-19T01:32:37.889224+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T09:26:05.966391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":978,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":20,"num_read_kb":4,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:33:05.472892+0000","last_change":"2024-08-19T01:32:41.235945+0000","last_active":"2024-08-19T01:33:05.472892+0000","last_peered":"2024-08-19T01:33:05.472892+0000","last_clean":"2024-08-19T01:33:05.472892+0000","last_became_active":"2024-08-19T01:32:41.235633+0000","last_became_peered":"2024-08-19T01:32:41.235633+0000","last_unstale":"2024-08-19T01:33:05.472892+0000","last_undegraded":"2024-08-19T01:33:05.472892+0000","last_fullsized":"2024-08-19T01:33:05.472892+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_clean_scrub_stamp":"2024-08-19T01:32:40.202519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T08:37:37.682175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"84'15","reported_seq":34,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-08-19T01:32:51.292993+0000","last_change":"2024-08-19T01:32:39.052850+0000","last_active":"2024-08-19T01:32:51.292993+0000","last_peered":"2024-08-19T01:32:51.292993+0000","last_clean":"2024-08-19T01:32:51.292993+0000","last_became_active":"2024-08-19T01:32:39.052455+0000","last_became_peered":"2024-08-19T01:32:39.052455+0000","last_unstale":"2024-08-19T01:32:51.292993+0000","last_undegraded":"2024-08-19T01:32:51.292993+0000","last_fullsized":"2024-08-19T01:32:51.292993+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:32:37.889224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:32:37.889224+0000","last_clean_scrub_stamp":"2024-08-19T01:32:37.889224+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T03:25:08.279911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":23,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:33:05.702575+0000","last_change":"2024-08-19T01:33:05.702575+0000","last_active":"2024-08-19T01:33:05.702575+0000","last_peered":"2024-08-19T01:33:05.702575+0000","last_clean":"2024-08-19T01:33:05.702575+0000","last_became_active":"2024-08-19T01:33:05.702110+0000","last_became_peered":"2024-08-19T01:33:05.702110+0000","last_unstale":"2024-08-19T01:33:05.702575+0000","last_undegraded":"2024-08-19T01:33:05.702575+0000","last_fullsized":"2024-08-19T01:33:05.702575+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_clean_scrub_stamp":"2024-08-19T01:32:40.202519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T12:18:51.102180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,1],"acting":[5,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"84'2","reported_seq":21,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-08-19T01:32:52.294363+0000","last_change":"2024-08-19T01:32:38.912090+0000","last_active":"2024-08-19T01:32:52.294363+0000","last_peered":"2024-08-19T01:32:52.294363+0000","last_clean":"2024-08-19T01:32:52.294363+0000","last_became_active":"2024-08-19T01:32:38.911753+0000","last_became_peered":"2024-08-19T01:32:38.911753+0000","last_unstale":"2024-08-19T01:32:52.294363+0000","last_undegraded":"2024-08-19T01:32:52.294363+0000","last_fullsized":"2024-08-19T01:32:52.294363+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:32:37.889224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:32:37.889224+0000","last_clean_scrub_stamp":"2024-08-19T01:32:37.889224+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T12:56:42.982062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:33:05.469805+0000","last_change":"2024-08-19T01:32:41.230429+0000","last_active":"2024-08-19T01:33:05.469805+0000","last_peered":"2024-08-19T01:33:05.469805+0000","last_clean":"2024-08-19T01:33:05.469805+0000","last_became_active":"2024-08-19T01:32:41.230115+0000","last_became_peered":"2024-08-19T01:32:41.230115+0000","last_unstale":"2024-08-19T01:33:05.469805+0000","last_undegraded":"2024-08-19T01:33:05.469805+0000","last_fullsized":"2024-08-19T01:33:05.469805+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_clean_scrub_stamp":"2024-08-19T01:32:40.202519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T01:50:51.084130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.e","version":"84'2","reported_seq":24,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:33:05.477826+0000","last_change":"2024-08-19T01:32:38.914557+0000","last_active":"2024-08-19T01:33:05.477826+0000","last_peered":"2024-08-19T01:33:05.477826+0000","last_clean":"2024-08-19T01:33:05.477826+0000","last_became_active":"2024-08-19T01:32:38.914115+0000","last_became_peered":"2024-08-19T01:32:38.914115+0000","last_unstale":"2024-08-19T01:33:05.477826+0000","last_undegraded":"2024-08-19T01:33:05.477826+0000","last_fullsized":"2024-08-19T01:33:05.477826+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:32:37.889224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:32:37.889224+0000","last_clean_scrub_stamp":"2024-08-19T01:32:37.889224+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T06:30:00.523513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:33:05.473436+0000","last_change":"2024-08-19T01:32:41.234883+0000","last_active":"2024-08-19T01:33:05.473436+0000","last_peered":"2024-08-19T01:33:05.473436+0000","last_clean":"2024-08-19T01:33:05.473436+0000","last_became_active":"2024-08-19T01:32:41.233901+0000","last_became_peered":"2024-08-19T01:32:41.233901+0000","last_unstale":"2024-08-19T01:33:05.473436+0000","last_undegraded":"2024-08-19T01:33:05.473436+0000","last_fullsized":"2024-08-19T01:33:05.473436+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_clean_scrub_stamp":"2024-08-19T01:32:40.202519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T12:08:46.305259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"84'1","reported_seq":24,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:33:05.470777+0000","last_change":"2024-08-19T01:32:38.913441+0000","last_active":"2024-08-19T01:33:05.470777+0000","last_peered":"2024-08-19T01:33:05.470777+0000","last_clean":"2024-08-19T01:33:05.470777+0000","last_became_active":"2024-08-19T01:32:38.913115+0000","last_became_peered":"2024-08-19T01:32:38.913115+0000","last_unstale":"2024-08-19T01:33:05.470777+0000","last_undegraded":"2024-08-19T01:33:05.470777+0000","last_fullsized":"2024-08-19T01:33:05.470777+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:32:37.889224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:32:37.889224+0000","last_clean_scrub_stamp":"2024-08-19T01:32:37.889224+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T11:43:15.290980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"84'1","reported_seq":18,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:33:05.472934+0000","last_change":"2024-08-19T01:32:41.235370+0000","last_active":"2024-08-19T01:33:05.472934+0000","last_peered":"2024-08-19T01:33:05.472934+0000","last_clean":"2024-08-19T01:33:05.472934+0000","last_became_active":"2024-08-19T01:32:41.235012+0000","last_became_peered":"2024-08-19T01:32:41.235012+0000","last_unstale":"2024-08-19T01:33:05.472934+0000","last_undegraded":"2024-08-19T01:33:05.472934+0000","last_fullsized":"2024-08-19T01:33:05.472934+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_clean_scrub_stamp":"2024-08-19T01:32:40.202519+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T02:18:31.134974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"84'3","reported_seq":24,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-08-19T01:33:04.473553+0000","last_change":"2024-08-19T01:32:39.245560+0000","last_active":"2024-08-19T01:33:04.473553+0000","last_peered":"2024-08-19T01:33:04.473553+0000","last_clean":"2024-08-19T01:33:04.473553+0000","last_became_active":"2024-08-19T01:32:39.245118+0000","last_became_peered":"2024-08-19T01:32:39.245118+0000","last_unstale":"2024-08-19T01:33:04.473553+0000","last_undegraded":"2024-08-19T01:33:04.473553+0000","last_fullsized":"2024-08-19T01:33:04.473553+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:32:37.889224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:32:37.889224+0000","last_clean_scrub_stamp":"2024-08-19T01:32:37.889224+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T02:18:24.605709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:33:05.473113+0000","last_change":"2024-08-19T01:32:41.234452+0000","last_active":"2024-08-19T01:33:05.473113+0000","last_peered":"2024-08-19T01:33:05.473113+0000","last_clean":"2024-08-19T01:33:05.473113+0000","last_became_active":"2024-08-19T01:32:41.233852+0000","last_became_peered":"2024-08-19T01:32:41.233852+0000","last_unstale":"2024-08-19T01:33:05.473113+0000","last_undegraded":"2024-08-19T01:33:05.473113+0000","last_fullsized":"2024-08-19T01:33:05.473113+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_clean_scrub_stamp":"2024-08-19T01:32:40.202519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T06:18:34.457239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"22'137","reported_seq":338,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:33:05.473232+0000","last_change":"2024-08-19T01:31:27.659394+0000","last_active":"2024-08-19T01:33:05.473232+0000","last_peered":"2024-08-19T01:33:05.473232+0000","last_clean":"2024-08-19T01:33:05.473232+0000","last_became_active":"2024-08-19T01:31:27.659007+0000","last_became_peered":"2024-08-19T01:31:27.659007+0000","last_unstale":"2024-08-19T01:33:05.473232+0000","last_undegraded":"2024-08-19T01:33:05.473232+0000","last_fullsized":"2024-08-19T01:33:05.473232+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:28:16.912251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:28:16.912251+0000","last_clean_scrub_stamp":"2024-08-19T01:28:16.912251+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T05:53:01.660619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"84'3","reported_seq":27,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:33:05.470433+0000","last_change":"2024-08-19T01:32:38.913408+0000","last_active":"2024-08-19T01:33:05.470433+0000","last_peered":"2024-08-19T01:33:05.470433+0000","last_clean":"2024-08-19T01:33:05.470433+0000","last_became_active":"2024-08-19T01:32:38.912242+0000","last_became_peered":"2024-08-19T01:32:38.912242+0000","last_unstale":"2024-08-19T01:33:05.470433+0000","last_undegraded":"2024-08-19T01:33:05.470433+0000","last_fullsized":"2024-08-19T01:33:05.470433+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:32:37.889224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:32:37.889224+0000","last_clean_scrub_stamp":"2024-08-19T01:32:37.889224+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T07:26:48.121366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-08-19T01:32:42.228024+0000","last_change":"2024-08-19T01:32:41.232611+0000","last_active":"2024-08-19T01:32:42.228024+0000","last_peered":"2024-08-19T01:32:42.228024+0000","last_clean":"2024-08-19T01:32:42.228024+0000","last_became_active":"2024-08-19T01:32:41.232261+0000","last_became_peered":"2024-08-19T01:32:41.232261+0000","last_unstale":"2024-08-19T01:32:42.228024+0000","last_undegraded":"2024-08-19T01:32:42.228024+0000","last_fullsized":"2024-08-19T01:32:42.228024+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_clean_scrub_stamp":"2024-08-19T01:32:40.202519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T08:04:16.793933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"84'3","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:33:05.473104+0000","last_change":"2024-08-19T01:32:38.912765+0000","last_active":"2024-08-19T01:33:05.473104+0000","last_peered":"2024-08-19T01:33:05.473104+0000","last_clean":"2024-08-19T01:33:05.473104+0000","last_became_active":"2024-08-19T01:32:38.912274+0000","last_became_peered":"2024-08-19T01:32:38.912274+0000","last_unstale":"2024-08-19T01:33:05.473104+0000","last_undegraded":"2024-08-19T01:33:05.473104+0000","last_fullsized":"2024-08-19T01:33:05.473104+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:32:37.889224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:32:37.889224+0000","last_clean_scrub_stamp":"2024-08-19T01:32:37.889224+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T09:57:49.714827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:33:05.477455+0000","last_change":"2024-08-19T01:32:41.224449+0000","last_active":"2024-08-19T01:33:05.477455+0000","last_peered":"2024-08-19T01:33:05.477455+0000","last_clean":"2024-08-19T01:33:05.477455+0000","last_became_active":"2024-08-19T01:32:41.224064+0000","last_became_peered":"2024-08-19T01:32:41.224064+0000","last_unstale":"2024-08-19T01:33:05.477455+0000","last_undegraded":"2024-08-19T01:33:05.477455+0000","last_fullsized":"2024-08-19T01:33:05.477455+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_clean_scrub_stamp":"2024-08-19T01:32:40.202519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T08:53:33.948504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"84'4","reported_seq":40,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:33:05.477561+0000","last_change":"2024-08-19T01:32:39.050869+0000","last_active":"2024-08-19T01:33:05.477561+0000","last_peered":"2024-08-19T01:33:05.477561+0000","last_clean":"2024-08-19T01:33:05.477561+0000","last_became_active":"2024-08-19T01:32:39.050529+0000","last_became_peered":"2024-08-19T01:32:39.050529+0000","last_unstale":"2024-08-19T01:33:05.477561+0000","last_undegraded":"2024-08-19T01:33:05.477561+0000","last_fullsized":"2024-08-19T01:33:05.477561+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:32:37.889224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:32:37.889224+0000","last_clean_scrub_stamp":"2024-08-19T01:32:37.889224+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T12:52:52.788360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":14,"num_read_kb":14,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:33:05.470330+0000","last_change":"2024-08-19T01:32:41.237093+0000","last_active":"2024-08-19T01:33:05.470330+0000","last_peered":"2024-08-19T01:33:05.470330+0000","last_clean":"2024-08-19T01:33:05.470330+0000","last_became_active":"2024-08-19T01:32:41.236738+0000","last_became_peered":"2024-08-19T01:32:41.236738+0000","last_unstale":"2024-08-19T01:33:05.470330+0000","last_undegraded":"2024-08-19T01:33:05.470330+0000","last_fullsized":"2024-08-19T01:33:05.470330+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_clean_scrub_stamp":"2024-08-19T01:32:40.202519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T08:37:33.117344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"84'1","reported_seq":23,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:33:05.473189+0000","last_change":"2024-08-19T01:32:38.915010+0000","last_active":"2024-08-19T01:33:05.473189+0000","last_peered":"2024-08-19T01:33:05.473189+0000","last_clean":"2024-08-19T01:33:05.473189+0000","last_became_active":"2024-08-19T01:32:38.914687+0000","last_became_peered":"2024-08-19T01:32:38.914687+0000","last_unstale":"2024-08-19T01:33:05.473189+0000","last_undegraded":"2024-08-19T01:33:05.473189+0000","last_fullsized":"2024-08-19T01:33:05.473189+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:32:37.889224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:32:37.889224+0000","last_clean_scrub_stamp":"2024-08-19T01:32:37.889224+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T04:46:07.881778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-08-19T01:33:04.468552+0000","last_change":"2024-08-19T01:32:41.244069+0000","last_active":"2024-08-19T01:33:04.468552+0000","last_peered":"2024-08-19T01:33:04.468552+0000","last_clean":"2024-08-19T01:33:04.468552+0000","last_became_active":"2024-08-19T01:32:41.243482+0000","last_became_peered":"2024-08-19T01:32:41.243482+0000","last_unstale":"2024-08-19T01:33:04.468552+0000","last_undegraded":"2024-08-19T01:33:04.468552+0000","last_fullsized":"2024-08-19T01:33:04.468552+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_clean_scrub_stamp":"2024-08-19T01:32:40.202519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T02:08:28.044421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":22,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:33:05.466068+0000","last_change":"2024-08-19T01:32:39.051383+0000","last_active":"2024-08-19T01:33:05.466068+0000","last_peered":"2024-08-19T01:33:05.466068+0000","last_clean":"2024-08-19T01:33:05.466068+0000","last_became_active":"2024-08-19T01:32:39.050970+0000","last_became_peered":"2024-08-19T01:32:39.050970+0000","last_unstale":"2024-08-19T01:33:05.466068+0000","last_undegraded":"2024-08-19T01:33:05.466068+0000","last_fullsized":"2024-08-19T01:33:05.466068+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:32:37.889224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:32:37.889224+0000","last_clean_scrub_stamp":"2024-08-19T01:32:37.889224+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T11:31:30.461651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"84'1","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-08-19T01:33:04.468448+0000","last_change":"2024-08-19T01:32:41.232656+0000","last_active":"2024-08-19T01:33:04.468448+0000","last_peered":"2024-08-19T01:33:04.468448+0000","last_clean":"2024-08-19T01:33:04.468448+0000","last_became_active":"2024-08-19T01:32:41.232274+0000","last_became_peered":"2024-08-19T01:32:41.232274+0000","last_unstale":"2024-08-19T01:33:04.468448+0000","last_undegraded":"2024-08-19T01:33:04.468448+0000","last_fullsized":"2024-08-19T01:33:04.468448+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_clean_scrub_stamp":"2024-08-19T01:32:40.202519+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T03:38:40.695652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.5","version":"84'1","reported_seq":20,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-08-19T01:32:50.279442+0000","last_change":"2024-08-19T01:32:39.050809+0000","last_active":"2024-08-19T01:32:50.279442+0000","last_peered":"2024-08-19T01:32:50.279442+0000","last_clean":"2024-08-19T01:32:50.279442+0000","last_became_active":"2024-08-19T01:32:39.050434+0000","last_became_peered":"2024-08-19T01:32:39.050434+0000","last_unstale":"2024-08-19T01:32:50.279442+0000","last_undegraded":"2024-08-19T01:32:50.279442+0000","last_fullsized":"2024-08-19T01:32:50.279442+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:32:37.889224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:32:37.889224+0000","last_clean_scrub_stamp":"2024-08-19T01:32:37.889224+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T13:04:28.828117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:33:05.477935+0000","last_change":"2024-08-19T01:32:41.228485+0000","last_active":"2024-08-19T01:33:05.477935+0000","last_peered":"2024-08-19T01:33:05.477935+0000","last_clean":"2024-08-19T01:33:05.477935+0000","last_became_active":"2024-08-19T01:32:41.227238+0000","last_became_peered":"2024-08-19T01:32:41.227238+0000","last_unstale":"2024-08-19T01:33:05.477935+0000","last_undegraded":"2024-08-19T01:33:05.477935+0000","last_fullsized":"2024-08-19T01:33:05.477935+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_clean_scrub_stamp":"2024-08-19T01:32:40.202519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T10:55:55.939382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"85'9","reported_seq":31,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:33:05.466322+0000","last_change":"2024-08-19T01:32:38.915994+0000","last_active":"2024-08-19T01:33:05.466322+0000","last_peered":"2024-08-19T01:33:05.466322+0000","last_clean":"2024-08-19T01:33:05.466322+0000","last_became_active":"2024-08-19T01:32:38.914714+0000","last_became_peered":"2024-08-19T01:32:38.914714+0000","last_unstale":"2024-08-19T01:33:05.466322+0000","last_undegraded":"2024-08-19T01:33:05.466322+0000","last_fullsized":"2024-08-19T01:33:05.466322+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:32:37.889224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:32:37.889224+0000","last_clean_scrub_stamp":"2024-08-19T01:32:37.889224+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T05:38:07.274274+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-08-19T01:32:42.229771+0000","last_change":"2024-08-19T01:32:41.236611+0000","last_active":"2024-08-19T01:32:42.229771+0000","last_peered":"2024-08-19T01:32:42.229771+0000","last_clean":"2024-08-19T01:32:42.229771+0000","last_became_active":"2024-08-19T01:32:41.233821+0000","last_became_peered":"2024-08-19T01:32:41.233821+0000","last_unstale":"2024-08-19T01:32:42.229771+0000","last_undegraded":"2024-08-19T01:32:42.229771+0000","last_fullsized":"2024-08-19T01:32:42.229771+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_clean_scrub_stamp":"2024-08-19T01:32:40.202519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T11:34:07.879034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"84'4","reported_seq":26,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:33:05.470567+0000","last_change":"2024-08-19T01:32:39.245411+0000","last_active":"2024-08-19T01:33:05.470567+0000","last_peered":"2024-08-19T01:33:05.470567+0000","last_clean":"2024-08-19T01:33:05.470567+0000","last_became_active":"2024-08-19T01:32:39.244980+0000","last_became_peered":"2024-08-19T01:32:39.244980+0000","last_unstale":"2024-08-19T01:33:05.470567+0000","last_undegraded":"2024-08-19T01:33:05.470567+0000","last_fullsized":"2024-08-19T01:33:05.470567+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:32:37.889224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:32:37.889224+0000","last_clean_scrub_stamp":"2024-08-19T01:32:37.889224+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T02:06:57.878051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":23,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:33:05.478955+0000","last_change":"2024-08-19T01:33:05.478955+0000","last_active":"2024-08-19T01:33:05.478955+0000","last_peered":"2024-08-19T01:33:05.478955+0000","last_clean":"2024-08-19T01:33:05.478955+0000","last_became_active":"2024-08-19T01:33:05.478446+0000","last_became_peered":"2024-08-19T01:33:05.478446+0000","last_unstale":"2024-08-19T01:33:05.478955+0000","last_undegraded":"2024-08-19T01:33:05.478955+0000","last_fullsized":"2024-08-19T01:33:05.478955+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_clean_scrub_stamp":"2024-08-19T01:32:40.202519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T11:58:24.285438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,6],"acting":[0,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"84'2","reported_seq":24,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:33:05.701553+0000","last_change":"2024-08-19T01:32:39.050100+0000","last_active":"2024-08-19T01:33:05.701553+0000","last_peered":"2024-08-19T01:33:05.701553+0000","last_clean":"2024-08-19T01:33:05.701553+0000","last_became_active":"2024-08-19T01:32:39.049807+0000","last_became_peered":"2024-08-19T01:32:39.049807+0000","last_unstale":"2024-08-19T01:33:05.701553+0000","last_undegraded":"2024-08-19T01:33:05.701553+0000","last_fullsized":"2024-08-19T01:33:05.701553+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:32:37.889224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:32:37.889224+0000","last_clean_scrub_stamp":"2024-08-19T01:32:37.889224+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T07:06:22.361609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-08-19T01:32:42.218877+0000","last_change":"2024-08-19T01:32:41.228982+0000","last_active":"2024-08-19T01:32:42.218877+0000","last_peered":"2024-08-19T01:32:42.218877+0000","last_clean":"2024-08-19T01:32:42.218877+0000","last_became_active":"2024-08-19T01:32:41.228562+0000","last_became_peered":"2024-08-19T01:32:41.228562+0000","last_unstale":"2024-08-19T01:32:42.218877+0000","last_undegraded":"2024-08-19T01:32:42.218877+0000","last_fullsized":"2024-08-19T01:32:42.218877+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_clean_scrub_stamp":"2024-08-19T01:32:40.202519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T03:14:42.480359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"84'3","reported_seq":21,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-08-19T01:32:50.279832+0000","last_change":"2024-08-19T01:32:38.913677+0000","last_active":"2024-08-19T01:32:50.279832+0000","last_peered":"2024-08-19T01:32:50.279832+0000","last_clean":"2024-08-19T01:32:50.279832+0000","last_became_active":"2024-08-19T01:32:38.913091+0000","last_became_peered":"2024-08-19T01:32:38.913091+0000","last_unstale":"2024-08-19T01:32:50.279832+0000","last_undegraded":"2024-08-19T01:32:50.279832+0000","last_fullsized":"2024-08-19T01:32:50.279832+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:32:37.889224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:32:37.889224+0000","last_clean_scrub_stamp":"2024-08-19T01:32:37.889224+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T12:39:16.489246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:33:05.469625+0000","last_change":"2024-08-19T01:32:41.239380+0000","last_active":"2024-08-19T01:33:05.469625+0000","last_peered":"2024-08-19T01:33:05.469625+0000","last_clean":"2024-08-19T01:33:05.469625+0000","last_became_active":"2024-08-19T01:32:41.239060+0000","last_became_peered":"2024-08-19T01:32:41.239060+0000","last_unstale":"2024-08-19T01:33:05.469625+0000","last_undegraded":"2024-08-19T01:33:05.469625+0000","last_fullsized":"2024-08-19T01:33:05.469625+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_clean_scrub_stamp":"2024-08-19T01:32:40.202519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T02:21:59.630705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"84'1","reported_seq":15,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:33:05.479101+0000","last_change":"2024-08-19T01:33:05.478578+0000","last_active":"2024-08-19T01:33:05.479101+0000","last_peered":"2024-08-19T01:33:05.479101+0000","last_clean":"2024-08-19T01:33:05.479101+0000","last_became_active":"2024-08-19T01:33:05.473433+0000","last_became_peered":"2024-08-19T01:33:05.473433+0000","last_unstale":"2024-08-19T01:33:05.479101+0000","last_undegraded":"2024-08-19T01:33:05.479101+0000","last_fullsized":"2024-08-19T01:33:05.479101+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:32:37.889224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:32:37.889224+0000","last_clean_scrub_stamp":"2024-08-19T01:32:37.889224+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T05:06:29.975613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,3,8],"acting":[11,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-08-19T01:32:42.228004+0000","last_change":"2024-08-19T01:32:41.236012+0000","last_active":"2024-08-19T01:32:42.228004+0000","last_peered":"2024-08-19T01:32:42.228004+0000","last_clean":"2024-08-19T01:32:42.228004+0000","last_became_active":"2024-08-19T01:32:41.235724+0000","last_became_peered":"2024-08-19T01:32:41.235724+0000","last_unstale":"2024-08-19T01:32:42.228004+0000","last_undegraded":"2024-08-19T01:32:42.228004+0000","last_fullsized":"2024-08-19T01:32:42.228004+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_clean_scrub_stamp":"2024-08-19T01:32:40.202519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T08:45:08.865144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"84'4","reported_seq":23,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-08-19T01:32:50.280025+0000","last_change":"2024-08-19T01:32:38.915170+0000","last_active":"2024-08-19T01:32:50.280025+0000","last_peered":"2024-08-19T01:32:50.280025+0000","last_clean":"2024-08-19T01:32:50.280025+0000","last_became_active":"2024-08-19T01:32:38.914812+0000","last_became_peered":"2024-08-19T01:32:38.914812+0000","last_unstale":"2024-08-19T01:32:50.280025+0000","last_undegraded":"2024-08-19T01:32:50.280025+0000","last_fullsized":"2024-08-19T01:32:50.280025+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:32:37.889224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:32:37.889224+0000","last_clean_scrub_stamp":"2024-08-19T01:32:37.889224+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T11:46:39.409065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:33:05.472797+0000","last_change":"2024-08-19T01:32:41.234133+0000","last_active":"2024-08-19T01:33:05.472797+0000","last_peered":"2024-08-19T01:33:05.472797+0000","last_clean":"2024-08-19T01:33:05.472797+0000","last_became_active":"2024-08-19T01:32:41.233492+0000","last_became_peered":"2024-08-19T01:32:41.233492+0000","last_unstale":"2024-08-19T01:33:05.472797+0000","last_undegraded":"2024-08-19T01:33:05.472797+0000","last_fullsized":"2024-08-19T01:33:05.472797+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_clean_scrub_stamp":"2024-08-19T01:32:40.202519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T06:11:12.587678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:33:05.469674+0000","last_change":"2024-08-19T01:32:41.232628+0000","last_active":"2024-08-19T01:33:05.469674+0000","last_peered":"2024-08-19T01:33:05.469674+0000","last_clean":"2024-08-19T01:33:05.469674+0000","last_became_active":"2024-08-19T01:32:41.232295+0000","last_became_peered":"2024-08-19T01:32:41.232295+0000","last_unstale":"2024-08-19T01:33:05.469674+0000","last_undegraded":"2024-08-19T01:33:05.469674+0000","last_fullsized":"2024-08-19T01:33:05.469674+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_clean_scrub_stamp":"2024-08-19T01:32:40.202519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T02:01:09.490553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"84'2","reported_seq":20,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-08-19T01:32:42.232899+0000","last_change":"2024-08-19T01:32:38.912000+0000","last_active":"2024-08-19T01:32:42.232899+0000","last_peered":"2024-08-19T01:32:42.232899+0000","last_clean":"2024-08-19T01:32:42.232899+0000","last_became_active":"2024-08-19T01:32:38.911535+0000","last_became_peered":"2024-08-19T01:32:38.911535+0000","last_unstale":"2024-08-19T01:32:42.232899+0000","last_undegraded":"2024-08-19T01:32:42.232899+0000","last_fullsized":"2024-08-19T01:32:42.232899+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:32:37.889224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:32:37.889224+0000","last_clean_scrub_stamp":"2024-08-19T01:32:37.889224+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T07:12:49.358384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:33:05.701576+0000","last_change":"2024-08-19T01:32:41.233507+0000","last_active":"2024-08-19T01:33:05.701576+0000","last_peered":"2024-08-19T01:33:05.701576+0000","last_clean":"2024-08-19T01:33:05.701576+0000","last_became_active":"2024-08-19T01:32:41.233179+0000","last_became_peered":"2024-08-19T01:32:41.233179+0000","last_unstale":"2024-08-19T01:33:05.701576+0000","last_undegraded":"2024-08-19T01:33:05.701576+0000","last_fullsized":"2024-08-19T01:33:05.701576+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_clean_scrub_stamp":"2024-08-19T01:32:40.202519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T11:47:44.780859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1d","version":"84'4","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:33:05.483750+0000","last_change":"2024-08-19T01:33:05.483322+0000","last_active":"2024-08-19T01:33:05.483750+0000","last_peered":"2024-08-19T01:33:05.483750+0000","last_clean":"2024-08-19T01:33:05.483750+0000","last_became_active":"2024-08-19T01:33:05.475496+0000","last_became_peered":"2024-08-19T01:33:05.475496+0000","last_unstale":"2024-08-19T01:33:05.483750+0000","last_undegraded":"2024-08-19T01:33:05.483750+0000","last_fullsized":"2024-08-19T01:33:05.483750+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:32:37.889224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:32:37.889224+0000","last_clean_scrub_stamp":"2024-08-19T01:32:37.889224+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T01:57:48.143495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":10,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,7],"acting":[11,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:33:05.469647+0000","last_change":"2024-08-19T01:32:41.244641+0000","last_active":"2024-08-19T01:33:05.469647+0000","last_peered":"2024-08-19T01:33:05.469647+0000","last_clean":"2024-08-19T01:33:05.469647+0000","last_became_active":"2024-08-19T01:32:41.244311+0000","last_became_peered":"2024-08-19T01:32:41.244311+0000","last_unstale":"2024-08-19T01:33:05.469647+0000","last_undegraded":"2024-08-19T01:33:05.469647+0000","last_fullsized":"2024-08-19T01:33:05.469647+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_clean_scrub_stamp":"2024-08-19T01:32:40.202519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T10:39:12.591092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"84'2","reported_seq":20,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-08-19T01:32:42.233986+0000","last_change":"2024-08-19T01:32:39.051314+0000","last_active":"2024-08-19T01:32:42.233986+0000","last_peered":"2024-08-19T01:32:42.233986+0000","last_clean":"2024-08-19T01:32:42.233986+0000","last_became_active":"2024-08-19T01:32:39.050998+0000","last_became_peered":"2024-08-19T01:32:39.050998+0000","last_unstale":"2024-08-19T01:32:42.233986+0000","last_undegraded":"2024-08-19T01:32:42.233986+0000","last_fullsized":"2024-08-19T01:32:42.233986+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:32:37.889224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:32:37.889224+0000","last_clean_scrub_stamp":"2024-08-19T01:32:37.889224+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T09:21:15.689149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-08-19T01:32:42.229192+0000","last_change":"2024-08-19T01:32:41.234762+0000","last_active":"2024-08-19T01:32:42.229192+0000","last_peered":"2024-08-19T01:32:42.229192+0000","last_clean":"2024-08-19T01:32:42.229192+0000","last_became_active":"2024-08-19T01:32:41.234471+0000","last_became_peered":"2024-08-19T01:32:41.234471+0000","last_unstale":"2024-08-19T01:32:42.229192+0000","last_undegraded":"2024-08-19T01:32:42.229192+0000","last_fullsized":"2024-08-19T01:32:42.229192+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_clean_scrub_stamp":"2024-08-19T01:32:40.202519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T08:34:33.110938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"84'9","reported_seq":32,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:33:05.466470+0000","last_change":"2024-08-19T01:32:38.917068+0000","last_active":"2024-08-19T01:33:05.466470+0000","last_peered":"2024-08-19T01:33:05.466470+0000","last_clean":"2024-08-19T01:33:05.466470+0000","last_became_active":"2024-08-19T01:32:38.915399+0000","last_became_peered":"2024-08-19T01:32:38.915399+0000","last_unstale":"2024-08-19T01:33:05.466470+0000","last_undegraded":"2024-08-19T01:33:05.466470+0000","last_fullsized":"2024-08-19T01:33:05.466470+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:32:37.889224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:32:37.889224+0000","last_clean_scrub_stamp":"2024-08-19T01:32:37.889224+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T04:17:25.061208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":45452,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":109,"num_read_kb":102,"num_write":153,"num_write_kb":87,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":1012,"num_keys_recovered":10,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":376832,"data_stored":146420,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":130,"ondisk_log_size":130,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4751360,"data_stored":4722944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8}],"osd_stats":[{"osd":11,"up_from":78,"seq":335007449106,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32100,"kb_used_data":916,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713052,"statfs":{"total":95995035648,"available":95962165248,"internally_reserved":0,"allocated":937984,"data_stored":451051,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":71,"seq":304942678038,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28560,"kb_used_data":1544,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716592,"statfs":{"total":95995035648,"available":95965790208,"internally_reserved":0,"allocated":1581056,"data_stored":1066708,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":66,"seq":283467841563,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32116,"kb_used_data":932,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713036,"statfs":{"total":95995035648,"available":95962148864,"internally_reserved":0,"allocated":954368,"data_stored":451287,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":8,"up_from":58,"seq":249108103200,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32076,"kb_used_data":900,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713076,"statfs":{"total":95995035648,"available":95962189824,"internally_reserved":0,"allocated":921600,"data_stored":433931,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":51,"seq":219043332133,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32696,"kb_used_data":1512,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712456,"statfs":{"total":95995035648,"available":95961554944,"internally_reserved":0,"allocated":1548288,"data_stored":1043526,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":46,"seq":197568495658,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27924,"kb_used_data":904,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717228,"statfs":{"total":95995035648,"available":95966441472,"internally_reserved":0,"allocated":925696,"data_stored":442159,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":38,"seq":163208757295,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28548,"kb_used_data":1524,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716604,"statfs":{"total":95995035648,"available":95965802496,"internally_reserved":0,"allocated":1560576,"data_stored":1042820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":33,"seq":141733920819,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32112,"kb_used_data":932,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713040,"statfs":{"total":95995035648,"available":95962152960,"internally_reserved":0,"allocated":954368,"data_stored":452515,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182455,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27920,"kb_used_data":904,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717232,"statfs":{"total":95995035648,"available":95966445568,"internally_reserved":0,"allocated":925696,"data_stored":433711,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411389,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36288,"kb_used_data":952,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708864,"statfs":{"total":95995035648,"available":95957876736,"internally_reserved":0,"allocated":974848,"data_stored":475995,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574912,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27996,"kb_used_data":972,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717156,"statfs":{"total":95995035648,"available":95966367744,"internally_reserved":0,"allocated":995328,"data_stored":500583,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738438,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36284,"kb_used_data":940,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708868,"statfs":{"total":95995035648,"available":95957880832,"internally_reserved":0,"allocated":962560,"data_stored":451947,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":50114,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":42802,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2046,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1866,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2572,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":81920,"data_stored":43030,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-08-19T01:33:10.371 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833 shell --fsid bad8780e-5dc9-11ef-bccf-c7b262605968 -- ceph pg dump --format=json 2024-08-19T01:33:10.747 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/bad8780e-5dc9-11ef-bccf-c7b262605968/mon.a/config 2024-08-19T01:33:11.292 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:11.290+0000 7f85ae000640 1 -- 172.21.15.83:0/11729139 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f85a81042a0 msgr2=0x7f85a81107d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:33:11.292 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:11.290+0000 7f85ae000640 1 --2- 172.21.15.83:0/11729139 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f85a81042a0 0x7f85a81107d0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f85a0009cd0 tx=0x7f85a002a8e0 comp rx=0 tx=0).stop 2024-08-19T01:33:11.292 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:11.290+0000 7f85ae000640 1 -- 172.21.15.83:0/11729139 shutdown_connections 2024-08-19T01:33:11.292 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:11.290+0000 7f85ae000640 1 --2- 172.21.15.83:0/11729139 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f85a81042a0 0x7f85a81107d0 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:33:11.292 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:11.290+0000 7f85ae000640 1 --2- 172.21.15.83:0/11729139 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f85a8103900 0x7f85a8103d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:33:11.292 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:11.290+0000 7f85ae000640 1 --2- 172.21.15.83:0/11729139 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f85a8109900 0x7f85a8109ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:33:11.292 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:11.290+0000 7f85ae000640 1 -- 172.21.15.83:0/11729139 >> 172.21.15.83:0/11729139 conn(0x7f85a80ff640 msgr2=0x7f85a8101a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:33:11.292 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:11.290+0000 7f85ae000640 1 -- 172.21.15.83:0/11729139 shutdown_connections 2024-08-19T01:33:11.292 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:11.290+0000 7f85ae000640 1 -- 172.21.15.83:0/11729139 wait complete. 2024-08-19T01:33:11.293 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:11.291+0000 7f85ae000640 1 Processor -- start 2024-08-19T01:33:11.293 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:11.291+0000 7f85ae000640 1 -- start start 2024-08-19T01:33:11.294 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:11.292+0000 7f85ae000640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f85a8103900 0x7f85a819a170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:33:11.294 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:11.292+0000 7f85ae000640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f85a81042a0 0x7f85a819a6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:33:11.294 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:11.292+0000 7f85ae000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f85a8109900 0x7f85a819ea40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:33:11.294 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:11.292+0000 7f85ae000640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f85a8113d50 con 0x7f85a8109900 2024-08-19T01:33:11.295 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:11.292+0000 7f85ae000640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f85a8113bd0 con 0x7f85a8103900 2024-08-19T01:33:11.295 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:11.292+0000 7f85ae000640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f85a8113ed0 con 0x7f85a81042a0 2024-08-19T01:33:11.295 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:11.292+0000 7f85acc00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f85a8103900 0x7f85a819a170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:33:11.295 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:11.292+0000 7f85acc00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f85a8103900 0x7f85a819a170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.83:37666/0 (socket says 172.21.15.83:37666) 2024-08-19T01:33:11.295 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:11.292+0000 7f85acc00640 1 -- 172.21.15.83:0/589894048 learned_addr learned my addr 172.21.15.83:0/589894048 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-08-19T01:33:11.295 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:11.292+0000 7f85ad600640 1 --2- 172.21.15.83:0/589894048 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f85a8109900 0x7f85a819ea40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:33:11.295 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:11.292+0000 7f859fe00640 1 --2- 172.21.15.83:0/589894048 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f85a81042a0 0x7f85a819a6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:33:11.295 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:11.293+0000 7f85acc00640 1 -- 172.21.15.83:0/589894048 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f85a81042a0 msgr2=0x7f85a819a6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:33:11.295 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:11.293+0000 7f85acc00640 1 --2- 172.21.15.83:0/589894048 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f85a81042a0 0x7f85a819a6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:33:11.295 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:11.293+0000 7f85acc00640 1 -- 172.21.15.83:0/589894048 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f85a8109900 msgr2=0x7f85a819ea40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:33:11.296 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:11.293+0000 7f85acc00640 1 --2- 172.21.15.83:0/589894048 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f85a8109900 0x7f85a819ea40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:33:11.296 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:11.293+0000 7f85acc00640 1 -- 172.21.15.83:0/589894048 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f85a819f120 con 0x7f85a8103900 2024-08-19T01:33:11.296 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:11.293+0000 7f85acc00640 1 --2- 172.21.15.83:0/589894048 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f85a8103900 0x7f85a819a170 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f859800d950 tx=0x7f859800de10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:33:11.297 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:11.293+0000 7f85a6000640 1 -- 172.21.15.83:0/589894048 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8598017030 con 0x7f85a8103900 2024-08-19T01:33:11.297 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:11.293+0000 7f85a6000640 1 -- 172.21.15.83:0/589894048 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8598004020 con 0x7f85a8103900 2024-08-19T01:33:11.297 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:11.293+0000 7f85a6000640 1 -- 172.21.15.83:0/589894048 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85980047a0 con 0x7f85a8103900 2024-08-19T01:33:11.297 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:11.293+0000 7f85ae000640 1 -- 172.21.15.83:0/589894048 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f85a819f410 con 0x7f85a8103900 2024-08-19T01:33:11.297 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:11.293+0000 7f85ae000640 1 -- 172.21.15.83:0/589894048 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f85a81a6cf0 con 0x7f85a8103900 2024-08-19T01:33:11.297 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:11.294+0000 7f85ae000640 1 -- 172.21.15.83:0/589894048 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f85a8105040 con 0x7f85a8103900 2024-08-19T01:33:11.298 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:11.296+0000 7f85a6000640 1 -- 172.21.15.83:0/589894048 <== mon.2 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f85980041d0 con 0x7f85a8103900 2024-08-19T01:33:11.299 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:11.298+0000 7f85a6000640 1 --2- 172.21.15.83:0/589894048 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f858c0a2c90 0x7f858c0a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:33:11.299 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:11.298+0000 7f85a6000640 1 -- 172.21.15.83:0/589894048 <== mon.2 v2:172.21.15.117:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8269+0+0 (secure 0 0 0) 0x7f85980ced20 con 0x7f85a8103900 2024-08-19T01:33:11.299 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:11.298+0000 7f859fe00640 1 --2- 172.21.15.83:0/589894048 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f858c0a2c90 0x7f858c0a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:33:11.300 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:11.299+0000 7f859fe00640 1 --2- 172.21.15.83:0/589894048 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f858c0a2c90 0x7f858c0a5150 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f85a8108ef0 tx=0x7f8590005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:33:11.302 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:11.300+0000 7f85a6000640 1 -- 172.21.15.83:0/589894048 <== mon.2 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7f859809bb20 con 0x7f85a8103900 2024-08-19T01:33:11.475 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:11.473+0000 7f85ae000640 1 -- 172.21.15.83:0/589894048 --> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f85a81013e0 con 0x7f858c0a2c90 2024-08-19T01:33:11.482 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:11.480+0000 7f85a6000640 1 -- 172.21.15.83:0/589894048 <== mgr.14150 v2:172.21.15.83:6800/683740916 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258747 (secure 0 0 0) 0x7f85a81013e0 con 0x7f858c0a2c90 2024-08-19T01:33:11.484 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-08-19T01:33:11.485 INFO:teuthology.orchestra.run.smithi083.stderr:dumped all 2024-08-19T01:33:11.487 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:11.485+0000 7f85ae000640 1 -- 172.21.15.83:0/589894048 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f858c0a2c90 msgr2=0x7f858c0a5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:33:11.488 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:11.485+0000 7f85ae000640 1 --2- 172.21.15.83:0/589894048 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f858c0a2c90 0x7f858c0a5150 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f85a8108ef0 tx=0x7f8590005f90 comp rx=0 tx=0).stop 2024-08-19T01:33:11.488 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:11.485+0000 7f85ae000640 1 -- 172.21.15.83:0/589894048 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f85a8103900 msgr2=0x7f85a819a170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:33:11.488 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:11.485+0000 7f85ae000640 1 --2- 172.21.15.83:0/589894048 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f85a8103900 0x7f85a819a170 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f859800d950 tx=0x7f859800de10 comp rx=0 tx=0).stop 2024-08-19T01:33:11.488 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:11.485+0000 7f85ae000640 1 -- 172.21.15.83:0/589894048 shutdown_connections 2024-08-19T01:33:11.488 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:11.485+0000 7f85ae000640 1 --2- 172.21.15.83:0/589894048 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f858c0a2c90 0x7f858c0a5150 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:33:11.488 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:11.485+0000 7f85ae000640 1 --2- 172.21.15.83:0/589894048 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f85a8109900 0x7f85a819ea40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:33:11.488 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:11.485+0000 7f85ae000640 1 --2- 172.21.15.83:0/589894048 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f85a81042a0 0x7f85a819a6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:33:11.488 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:11.485+0000 7f85ae000640 1 --2- 172.21.15.83:0/589894048 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f85a8103900 0x7f85a819a170 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:33:11.488 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:11.485+0000 7f85ae000640 1 -- 172.21.15.83:0/589894048 >> 172.21.15.83:0/589894048 conn(0x7f85a80ff640 msgr2=0x7f85a8100cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:33:11.488 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:11.485+0000 7f85ae000640 1 -- 172.21.15.83:0/589894048 shutdown_connections 2024-08-19T01:33:11.488 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:11.485+0000 7f85ae000640 1 -- 172.21.15.83:0/589894048 wait complete. 2024-08-19T01:33:11.829 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:33:11 smithi083 ceph-mon[17120]: from='client.24646 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-08-19T01:33:11.829 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:33:11 smithi083 ceph-mon[17120]: pgmap v289: 97 pgs: 97 active+clean; 630 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 4.1 KiB/s wr, 9 op/s; 126 B/s, 1 keys/s, 0 objects/s recovering 2024-08-19T01:33:11.952 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:33:11 smithi117 ceph-mon[22016]: from='client.24646 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-08-19T01:33:11.952 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:33:11 smithi117 ceph-mon[22016]: pgmap v289: 97 pgs: 97 active+clean; 630 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 4.1 KiB/s wr, 9 op/s; 126 B/s, 1 keys/s, 0 objects/s recovering 2024-08-19T01:33:11.966 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:33:11 smithi167 ceph-mon[21875]: from='client.24646 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-08-19T01:33:11.966 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:33:11 smithi167 ceph-mon[21875]: pgmap v289: 97 pgs: 97 active+clean; 630 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 4.1 KiB/s wr, 9 op/s; 126 B/s, 1 keys/s, 0 objects/s recovering 2024-08-19T01:33:12.301 INFO:teuthology.orchestra.run.smithi083.stdout:{"pg_ready":true,"pg_map":{"version":289,"stamp":"2024-08-19T01:33:10.364905+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":645106,"num_objects":64,"num_object_clones":0,"num_object_copies":192,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":251,"num_read_kb":230,"num_write":340,"num_write_kb":3038,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":1181748,"num_keys_recovered":10,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":273,"ondisk_log_size":273,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":286,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":374792,"kb_used_data":13076,"kb_used_omap":18,"kb_used_meta":361517,"kb_avail":1124567032,"statfs":{"total":1151940427776,"available":1151556640768,"internally_reserved":0,"allocated":13389824,"data_stored":7332429,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19143,"internal_metadata":370193721},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":27902,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":56,"num_read_kb":53,"num_write":17,"num_write_kb":33,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":1012,"num_keys_recovered":10,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.002482"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:33:05.470153+0000","last_change":"2024-08-19T01:32:41.244214+0000","last_active":"2024-08-19T01:33:05.470153+0000","last_peered":"2024-08-19T01:33:05.470153+0000","last_clean":"2024-08-19T01:33:05.470153+0000","last_became_active":"2024-08-19T01:32:41.243889+0000","last_became_peered":"2024-08-19T01:32:41.243889+0000","last_unstale":"2024-08-19T01:33:05.470153+0000","last_undegraded":"2024-08-19T01:33:05.470153+0000","last_fullsized":"2024-08-19T01:33:05.470153+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_clean_scrub_stamp":"2024-08-19T01:32:40.202519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T13:10:18.230055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:33:05.468954+0000","last_change":"2024-08-19T01:32:41.242729+0000","last_active":"2024-08-19T01:33:05.468954+0000","last_peered":"2024-08-19T01:33:05.468954+0000","last_clean":"2024-08-19T01:33:05.468954+0000","last_became_active":"2024-08-19T01:32:41.242436+0000","last_became_peered":"2024-08-19T01:32:41.242436+0000","last_unstale":"2024-08-19T01:33:05.468954+0000","last_undegraded":"2024-08-19T01:33:05.468954+0000","last_fullsized":"2024-08-19T01:33:05.468954+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_clean_scrub_stamp":"2024-08-19T01:32:40.202519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T13:19:32.010576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:33:05.468943+0000","last_change":"2024-08-19T01:32:41.247377+0000","last_active":"2024-08-19T01:33:05.468943+0000","last_peered":"2024-08-19T01:33:05.468943+0000","last_clean":"2024-08-19T01:33:05.468943+0000","last_became_active":"2024-08-19T01:32:41.246804+0000","last_became_peered":"2024-08-19T01:32:41.246804+0000","last_unstale":"2024-08-19T01:33:05.468943+0000","last_undegraded":"2024-08-19T01:33:05.468943+0000","last_fullsized":"2024-08-19T01:33:05.468943+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_clean_scrub_stamp":"2024-08-19T01:32:40.202519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T04:48:52.382687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:33:05.470397+0000","last_change":"2024-08-19T01:32:41.232900+0000","last_active":"2024-08-19T01:33:05.470397+0000","last_peered":"2024-08-19T01:33:05.470397+0000","last_clean":"2024-08-19T01:33:05.470397+0000","last_became_active":"2024-08-19T01:32:41.232573+0000","last_became_peered":"2024-08-19T01:32:41.232573+0000","last_unstale":"2024-08-19T01:33:05.470397+0000","last_undegraded":"2024-08-19T01:33:05.470397+0000","last_fullsized":"2024-08-19T01:33:05.470397+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_clean_scrub_stamp":"2024-08-19T01:32:40.202519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T09:42:36.309834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":23,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:33:05.478964+0000","last_change":"2024-08-19T01:33:05.478964+0000","last_active":"2024-08-19T01:33:05.478964+0000","last_peered":"2024-08-19T01:33:05.478964+0000","last_clean":"2024-08-19T01:33:05.478964+0000","last_became_active":"2024-08-19T01:33:05.478430+0000","last_became_peered":"2024-08-19T01:33:05.478430+0000","last_unstale":"2024-08-19T01:33:05.478964+0000","last_undegraded":"2024-08-19T01:33:05.478964+0000","last_fullsized":"2024-08-19T01:33:05.478964+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_clean_scrub_stamp":"2024-08-19T01:32:40.202519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T04:10:49.267550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,6],"acting":[8,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:33:05.474307+0000","last_change":"2024-08-19T01:32:41.228263+0000","last_active":"2024-08-19T01:33:05.474307+0000","last_peered":"2024-08-19T01:33:05.474307+0000","last_clean":"2024-08-19T01:33:05.474307+0000","last_became_active":"2024-08-19T01:32:41.227765+0000","last_became_peered":"2024-08-19T01:32:41.227765+0000","last_unstale":"2024-08-19T01:33:05.474307+0000","last_undegraded":"2024-08-19T01:33:05.474307+0000","last_fullsized":"2024-08-19T01:33:05.474307+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_clean_scrub_stamp":"2024-08-19T01:32:40.202519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T11:21:18.817886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:33:05.474641+0000","last_change":"2024-08-19T01:32:41.235680+0000","last_active":"2024-08-19T01:33:05.474641+0000","last_peered":"2024-08-19T01:33:05.474641+0000","last_clean":"2024-08-19T01:33:05.474641+0000","last_became_active":"2024-08-19T01:32:41.234150+0000","last_became_peered":"2024-08-19T01:32:41.234150+0000","last_unstale":"2024-08-19T01:33:05.474641+0000","last_undegraded":"2024-08-19T01:33:05.474641+0000","last_fullsized":"2024-08-19T01:33:05.474641+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_clean_scrub_stamp":"2024-08-19T01:32:40.202519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T10:28:08.908632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:33:05.471317+0000","last_change":"2024-08-19T01:32:41.230170+0000","last_active":"2024-08-19T01:33:05.471317+0000","last_peered":"2024-08-19T01:33:05.471317+0000","last_clean":"2024-08-19T01:33:05.471317+0000","last_became_active":"2024-08-19T01:32:41.229467+0000","last_became_peered":"2024-08-19T01:32:41.229467+0000","last_unstale":"2024-08-19T01:33:05.471317+0000","last_undegraded":"2024-08-19T01:33:05.471317+0000","last_fullsized":"2024-08-19T01:33:05.471317+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_clean_scrub_stamp":"2024-08-19T01:32:40.202519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T10:24:15.832910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:33:05.701955+0000","last_change":"2024-08-19T01:32:41.230607+0000","last_active":"2024-08-19T01:33:05.701955+0000","last_peered":"2024-08-19T01:33:05.701955+0000","last_clean":"2024-08-19T01:33:05.701955+0000","last_became_active":"2024-08-19T01:32:41.229979+0000","last_became_peered":"2024-08-19T01:32:41.229979+0000","last_unstale":"2024-08-19T01:33:05.701955+0000","last_undegraded":"2024-08-19T01:33:05.701955+0000","last_fullsized":"2024-08-19T01:33:05.701955+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_clean_scrub_stamp":"2024-08-19T01:32:40.202519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T02:23:46.736139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:33:05.474802+0000","last_change":"2024-08-19T01:32:41.233576+0000","last_active":"2024-08-19T01:33:05.474802+0000","last_peered":"2024-08-19T01:33:05.474802+0000","last_clean":"2024-08-19T01:33:05.474802+0000","last_became_active":"2024-08-19T01:32:41.231279+0000","last_became_peered":"2024-08-19T01:32:41.231279+0000","last_unstale":"2024-08-19T01:33:05.474802+0000","last_undegraded":"2024-08-19T01:33:05.474802+0000","last_fullsized":"2024-08-19T01:33:05.474802+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_clean_scrub_stamp":"2024-08-19T01:32:40.202519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T07:32:45.341628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:33:05.473054+0000","last_change":"2024-08-19T01:32:41.233192+0000","last_active":"2024-08-19T01:33:05.473054+0000","last_peered":"2024-08-19T01:33:05.473054+0000","last_clean":"2024-08-19T01:33:05.473054+0000","last_became_active":"2024-08-19T01:32:41.230944+0000","last_became_peered":"2024-08-19T01:32:41.230944+0000","last_unstale":"2024-08-19T01:33:05.473054+0000","last_undegraded":"2024-08-19T01:33:05.473054+0000","last_fullsized":"2024-08-19T01:33:05.473054+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_clean_scrub_stamp":"2024-08-19T01:32:40.202519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T08:55:29.036252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:33:05.470408+0000","last_change":"2024-08-19T01:32:41.234216+0000","last_active":"2024-08-19T01:33:05.470408+0000","last_peered":"2024-08-19T01:33:05.470408+0000","last_clean":"2024-08-19T01:33:05.470408+0000","last_became_active":"2024-08-19T01:32:41.233906+0000","last_became_peered":"2024-08-19T01:32:41.233906+0000","last_unstale":"2024-08-19T01:33:05.470408+0000","last_undegraded":"2024-08-19T01:33:05.470408+0000","last_fullsized":"2024-08-19T01:33:05.470408+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_clean_scrub_stamp":"2024-08-19T01:32:40.202519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T08:35:01.764494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:33:05.477816+0000","last_change":"2024-08-19T01:32:41.230651+0000","last_active":"2024-08-19T01:33:05.477816+0000","last_peered":"2024-08-19T01:33:05.477816+0000","last_clean":"2024-08-19T01:33:05.477816+0000","last_became_active":"2024-08-19T01:32:41.230253+0000","last_became_peered":"2024-08-19T01:32:41.230253+0000","last_unstale":"2024-08-19T01:33:05.477816+0000","last_undegraded":"2024-08-19T01:33:05.477816+0000","last_fullsized":"2024-08-19T01:33:05.477816+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_clean_scrub_stamp":"2024-08-19T01:32:40.202519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T09:41:59.942821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:33:05.468782+0000","last_change":"2024-08-19T01:32:41.240082+0000","last_active":"2024-08-19T01:33:05.468782+0000","last_peered":"2024-08-19T01:33:05.468782+0000","last_clean":"2024-08-19T01:33:05.468782+0000","last_became_active":"2024-08-19T01:32:41.239711+0000","last_became_peered":"2024-08-19T01:32:41.239711+0000","last_unstale":"2024-08-19T01:33:05.468782+0000","last_undegraded":"2024-08-19T01:33:05.468782+0000","last_fullsized":"2024-08-19T01:33:05.468782+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_clean_scrub_stamp":"2024-08-19T01:32:40.202519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T12:48:59.122307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":23,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:33:05.474883+0000","last_change":"2024-08-19T01:33:05.474883+0000","last_active":"2024-08-19T01:33:05.474883+0000","last_peered":"2024-08-19T01:33:05.474883+0000","last_clean":"2024-08-19T01:33:05.474883+0000","last_became_active":"2024-08-19T01:33:05.474502+0000","last_became_peered":"2024-08-19T01:33:05.474502+0000","last_unstale":"2024-08-19T01:33:05.474883+0000","last_undegraded":"2024-08-19T01:33:05.474883+0000","last_fullsized":"2024-08-19T01:33:05.474883+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_clean_scrub_stamp":"2024-08-19T01:32:40.202519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T07:02:01.171103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2],"acting":[0,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:33:05.701404+0000","last_change":"2024-08-19T01:32:41.240899+0000","last_active":"2024-08-19T01:33:05.701404+0000","last_peered":"2024-08-19T01:33:05.701404+0000","last_clean":"2024-08-19T01:33:05.701404+0000","last_became_active":"2024-08-19T01:32:41.240532+0000","last_became_peered":"2024-08-19T01:32:41.240532+0000","last_unstale":"2024-08-19T01:33:05.701404+0000","last_undegraded":"2024-08-19T01:33:05.701404+0000","last_fullsized":"2024-08-19T01:33:05.701404+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_clean_scrub_stamp":"2024-08-19T01:32:40.202519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T08:17:47.035887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:33:05.470183+0000","last_change":"2024-08-19T01:32:41.237977+0000","last_active":"2024-08-19T01:33:05.470183+0000","last_peered":"2024-08-19T01:33:05.470183+0000","last_clean":"2024-08-19T01:33:05.470183+0000","last_became_active":"2024-08-19T01:32:41.237634+0000","last_became_peered":"2024-08-19T01:32:41.237634+0000","last_unstale":"2024-08-19T01:33:05.470183+0000","last_undegraded":"2024-08-19T01:33:05.470183+0000","last_fullsized":"2024-08-19T01:33:05.470183+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_clean_scrub_stamp":"2024-08-19T01:32:40.202519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T12:48:37.005972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:33:05.466272+0000","last_change":"2024-08-19T01:32:41.236498+0000","last_active":"2024-08-19T01:33:05.466272+0000","last_peered":"2024-08-19T01:33:05.466272+0000","last_clean":"2024-08-19T01:33:05.466272+0000","last_became_active":"2024-08-19T01:32:41.236122+0000","last_became_peered":"2024-08-19T01:32:41.236122+0000","last_unstale":"2024-08-19T01:33:05.466272+0000","last_undegraded":"2024-08-19T01:33:05.466272+0000","last_fullsized":"2024-08-19T01:33:05.466272+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_clean_scrub_stamp":"2024-08-19T01:32:40.202519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T02:31:23.648064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:33:05.466818+0000","last_change":"2024-08-19T01:32:41.234607+0000","last_active":"2024-08-19T01:33:05.466818+0000","last_peered":"2024-08-19T01:33:05.466818+0000","last_clean":"2024-08-19T01:33:05.466818+0000","last_became_active":"2024-08-19T01:32:41.232867+0000","last_became_peered":"2024-08-19T01:32:41.232867+0000","last_unstale":"2024-08-19T01:33:05.466818+0000","last_undegraded":"2024-08-19T01:33:05.466818+0000","last_fullsized":"2024-08-19T01:33:05.466818+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_clean_scrub_stamp":"2024-08-19T01:32:40.202519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T12:32:38.441848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:33:05.469342+0000","last_change":"2024-08-19T01:32:41.231705+0000","last_active":"2024-08-19T01:33:05.469342+0000","last_peered":"2024-08-19T01:33:05.469342+0000","last_clean":"2024-08-19T01:33:05.469342+0000","last_became_active":"2024-08-19T01:32:41.231262+0000","last_became_peered":"2024-08-19T01:32:41.231262+0000","last_unstale":"2024-08-19T01:33:05.469342+0000","last_undegraded":"2024-08-19T01:33:05.469342+0000","last_fullsized":"2024-08-19T01:33:05.469342+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_clean_scrub_stamp":"2024-08-19T01:32:40.202519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T10:41:15.025471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:33:05.477712+0000","last_change":"2024-08-19T01:32:41.243192+0000","last_active":"2024-08-19T01:33:05.477712+0000","last_peered":"2024-08-19T01:33:05.477712+0000","last_clean":"2024-08-19T01:33:05.477712+0000","last_became_active":"2024-08-19T01:32:41.242704+0000","last_became_peered":"2024-08-19T01:32:41.242704+0000","last_unstale":"2024-08-19T01:33:05.477712+0000","last_undegraded":"2024-08-19T01:33:05.477712+0000","last_fullsized":"2024-08-19T01:33:05.477712+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_clean_scrub_stamp":"2024-08-19T01:32:40.202519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T01:44:43.270213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:33:05.701950+0000","last_change":"2024-08-19T01:32:41.239908+0000","last_active":"2024-08-19T01:33:05.701950+0000","last_peered":"2024-08-19T01:33:05.701950+0000","last_clean":"2024-08-19T01:33:05.701950+0000","last_became_active":"2024-08-19T01:32:41.239438+0000","last_became_peered":"2024-08-19T01:32:41.239438+0000","last_unstale":"2024-08-19T01:33:05.701950+0000","last_undegraded":"2024-08-19T01:33:05.701950+0000","last_fullsized":"2024-08-19T01:33:05.701950+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_clean_scrub_stamp":"2024-08-19T01:32:40.202519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T11:03:26.822111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:33:05.471582+0000","last_change":"2024-08-19T01:32:41.226373+0000","last_active":"2024-08-19T01:33:05.471582+0000","last_peered":"2024-08-19T01:33:05.471582+0000","last_clean":"2024-08-19T01:33:05.471582+0000","last_became_active":"2024-08-19T01:32:41.226089+0000","last_became_peered":"2024-08-19T01:32:41.226089+0000","last_unstale":"2024-08-19T01:33:05.471582+0000","last_undegraded":"2024-08-19T01:33:05.471582+0000","last_fullsized":"2024-08-19T01:33:05.471582+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_clean_scrub_stamp":"2024-08-19T01:32:40.202519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T01:33:56.639300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:33:05.469289+0000","last_change":"2024-08-19T01:32:41.237829+0000","last_active":"2024-08-19T01:33:05.469289+0000","last_peered":"2024-08-19T01:33:05.469289+0000","last_clean":"2024-08-19T01:33:05.469289+0000","last_became_active":"2024-08-19T01:32:41.237518+0000","last_became_peered":"2024-08-19T01:32:41.237518+0000","last_unstale":"2024-08-19T01:33:05.469289+0000","last_undegraded":"2024-08-19T01:33:05.469289+0000","last_fullsized":"2024-08-19T01:33:05.469289+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_clean_scrub_stamp":"2024-08-19T01:32:40.202519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T02:25:11.307658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:33:05.472158+0000","last_change":"2024-08-19T01:32:41.234508+0000","last_active":"2024-08-19T01:33:05.472158+0000","last_peered":"2024-08-19T01:33:05.472158+0000","last_clean":"2024-08-19T01:33:05.472158+0000","last_became_active":"2024-08-19T01:32:41.232869+0000","last_became_peered":"2024-08-19T01:32:41.232869+0000","last_unstale":"2024-08-19T01:33:05.472158+0000","last_undegraded":"2024-08-19T01:33:05.472158+0000","last_fullsized":"2024-08-19T01:33:05.472158+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_clean_scrub_stamp":"2024-08-19T01:32:40.202519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T11:20:55.097576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:33:05.466993+0000","last_change":"2024-08-19T01:32:41.230536+0000","last_active":"2024-08-19T01:33:05.466993+0000","last_peered":"2024-08-19T01:33:05.466993+0000","last_clean":"2024-08-19T01:33:05.466993+0000","last_became_active":"2024-08-19T01:32:41.230179+0000","last_became_peered":"2024-08-19T01:32:41.230179+0000","last_unstale":"2024-08-19T01:33:05.466993+0000","last_undegraded":"2024-08-19T01:33:05.466993+0000","last_fullsized":"2024-08-19T01:33:05.466993+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_clean_scrub_stamp":"2024-08-19T01:32:40.202519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T02:25:09.441363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:33:05.469818+0000","last_change":"2024-08-19T01:32:41.230695+0000","last_active":"2024-08-19T01:33:05.469818+0000","last_peered":"2024-08-19T01:33:05.469818+0000","last_clean":"2024-08-19T01:33:05.469818+0000","last_became_active":"2024-08-19T01:32:41.230324+0000","last_became_peered":"2024-08-19T01:32:41.230324+0000","last_unstale":"2024-08-19T01:33:05.469818+0000","last_undegraded":"2024-08-19T01:33:05.469818+0000","last_fullsized":"2024-08-19T01:33:05.469818+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_clean_scrub_stamp":"2024-08-19T01:32:40.202519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T05:38:09.317605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:33:05.477784+0000","last_change":"2024-08-19T01:32:41.238325+0000","last_active":"2024-08-19T01:33:05.477784+0000","last_peered":"2024-08-19T01:33:05.477784+0000","last_clean":"2024-08-19T01:33:05.477784+0000","last_became_active":"2024-08-19T01:32:41.237768+0000","last_became_peered":"2024-08-19T01:32:41.237768+0000","last_unstale":"2024-08-19T01:33:05.477784+0000","last_undegraded":"2024-08-19T01:33:05.477784+0000","last_fullsized":"2024-08-19T01:33:05.477784+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_clean_scrub_stamp":"2024-08-19T01:32:40.202519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T08:45:06.866241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:33:05.468967+0000","last_change":"2024-08-19T01:32:41.246950+0000","last_active":"2024-08-19T01:33:05.468967+0000","last_peered":"2024-08-19T01:33:05.468967+0000","last_clean":"2024-08-19T01:33:05.468967+0000","last_became_active":"2024-08-19T01:32:41.245441+0000","last_became_peered":"2024-08-19T01:32:41.245441+0000","last_unstale":"2024-08-19T01:33:05.468967+0000","last_undegraded":"2024-08-19T01:33:05.468967+0000","last_fullsized":"2024-08-19T01:33:05.468967+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_clean_scrub_stamp":"2024-08-19T01:32:40.202519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T03:11:06.921258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:33:05.466512+0000","last_change":"2024-08-19T01:32:41.244810+0000","last_active":"2024-08-19T01:33:05.466512+0000","last_peered":"2024-08-19T01:33:05.466512+0000","last_clean":"2024-08-19T01:33:05.466512+0000","last_became_active":"2024-08-19T01:32:41.244401+0000","last_became_peered":"2024-08-19T01:32:41.244401+0000","last_unstale":"2024-08-19T01:33:05.466512+0000","last_undegraded":"2024-08-19T01:33:05.466512+0000","last_fullsized":"2024-08-19T01:33:05.466512+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_clean_scrub_stamp":"2024-08-19T01:32:40.202519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T04:54:57.728181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:33:05.466793+0000","last_change":"2024-08-19T01:32:41.235977+0000","last_active":"2024-08-19T01:33:05.466793+0000","last_peered":"2024-08-19T01:33:05.466793+0000","last_clean":"2024-08-19T01:33:05.466793+0000","last_became_active":"2024-08-19T01:32:41.235693+0000","last_became_peered":"2024-08-19T01:32:41.235693+0000","last_unstale":"2024-08-19T01:33:05.466793+0000","last_undegraded":"2024-08-19T01:33:05.466793+0000","last_fullsized":"2024-08-19T01:33:05.466793+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_clean_scrub_stamp":"2024-08-19T01:32:40.202519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T04:59:15.766532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":23,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:33:05.473318+0000","last_change":"2024-08-19T01:33:05.473318+0000","last_active":"2024-08-19T01:33:05.473318+0000","last_peered":"2024-08-19T01:33:05.473318+0000","last_clean":"2024-08-19T01:33:05.473318+0000","last_became_active":"2024-08-19T01:33:05.472885+0000","last_became_peered":"2024-08-19T01:33:05.472885+0000","last_unstale":"2024-08-19T01:33:05.473318+0000","last_undegraded":"2024-08-19T01:33:05.473318+0000","last_fullsized":"2024-08-19T01:33:05.473318+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_clean_scrub_stamp":"2024-08-19T01:32:40.202519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T07:49:35.525930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,9],"acting":[11,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:33:05.471500+0000","last_change":"2024-08-19T01:32:41.230846+0000","last_active":"2024-08-19T01:33:05.471500+0000","last_peered":"2024-08-19T01:33:05.471500+0000","last_clean":"2024-08-19T01:33:05.471500+0000","last_became_active":"2024-08-19T01:32:41.230405+0000","last_became_peered":"2024-08-19T01:32:41.230405+0000","last_unstale":"2024-08-19T01:33:05.471500+0000","last_undegraded":"2024-08-19T01:33:05.471500+0000","last_fullsized":"2024-08-19T01:33:05.471500+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_clean_scrub_stamp":"2024-08-19T01:32:40.202519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T08:59:46.241898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1b","version":"84'6","reported_seq":59,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:33:06.478738+0000","last_change":"2024-08-19T01:33:05.479889+0000","last_active":"2024-08-19T01:33:06.478738+0000","last_peered":"2024-08-19T01:33:06.478738+0000","last_clean":"2024-08-19T01:33:06.478738+0000","last_became_active":"2024-08-19T01:33:05.474698+0000","last_became_peered":"2024-08-19T01:33:05.474698+0000","last_unstale":"2024-08-19T01:33:06.478738+0000","last_undegraded":"2024-08-19T01:33:06.478738+0000","last_fullsized":"2024-08-19T01:33:06.478738+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:32:37.889224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:32:37.889224+0000","last_clean_scrub_stamp":"2024-08-19T01:32:37.889224+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T08:41:29.053471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1012,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":17,"num_read_kb":3,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":1012,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7],"acting":[5,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":11,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:33:05.475899+0000","last_change":"2024-08-19T01:33:05.474083+0000","last_active":"2024-08-19T01:33:05.475899+0000","last_peered":"2024-08-19T01:33:05.475899+0000","last_clean":"2024-08-19T01:33:05.475899+0000","last_became_active":"2024-08-19T01:33:05.473718+0000","last_became_peered":"2024-08-19T01:33:05.473718+0000","last_unstale":"2024-08-19T01:33:05.475899+0000","last_undegraded":"2024-08-19T01:33:05.475899+0000","last_fullsized":"2024-08-19T01:33:05.475899+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_clean_scrub_stamp":"2024-08-19T01:32:40.202519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T02:25:11.955170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,4,3],"acting":[9,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.1a","version":"84'3","reported_seq":35,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:33:05.705917+0000","last_change":"2024-08-19T01:33:05.705917+0000","last_active":"2024-08-19T01:33:05.705917+0000","last_peered":"2024-08-19T01:33:05.705917+0000","last_clean":"2024-08-19T01:33:05.705917+0000","last_became_active":"2024-08-19T01:33:05.702775+0000","last_became_peered":"2024-08-19T01:33:05.702775+0000","last_unstale":"2024-08-19T01:33:05.705917+0000","last_undegraded":"2024-08-19T01:33:05.705917+0000","last_fullsized":"2024-08-19T01:33:05.705917+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:32:37.889224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:32:37.889224+0000","last_clean_scrub_stamp":"2024-08-19T01:32:37.889224+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T03:58:29.989815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,6],"acting":[10,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.19","version":"84'2","reported_seq":24,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:33:05.477490+0000","last_change":"2024-08-19T01:32:38.911757+0000","last_active":"2024-08-19T01:33:05.477490+0000","last_peered":"2024-08-19T01:33:05.477490+0000","last_clean":"2024-08-19T01:33:05.477490+0000","last_became_active":"2024-08-19T01:32:38.910832+0000","last_became_peered":"2024-08-19T01:32:38.910832+0000","last_unstale":"2024-08-19T01:33:05.477490+0000","last_undegraded":"2024-08-19T01:33:05.477490+0000","last_fullsized":"2024-08-19T01:33:05.477490+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:32:37.889224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:32:37.889224+0000","last_clean_scrub_stamp":"2024-08-19T01:32:37.889224+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T04:46:54.986693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:33:05.471462+0000","last_change":"2024-08-19T01:32:41.231130+0000","last_active":"2024-08-19T01:33:05.471462+0000","last_peered":"2024-08-19T01:33:05.471462+0000","last_clean":"2024-08-19T01:33:05.471462+0000","last_became_active":"2024-08-19T01:32:41.230883+0000","last_became_peered":"2024-08-19T01:32:41.230883+0000","last_unstale":"2024-08-19T01:33:05.471462+0000","last_undegraded":"2024-08-19T01:33:05.471462+0000","last_fullsized":"2024-08-19T01:33:05.471462+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_clean_scrub_stamp":"2024-08-19T01:32:40.202519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T06:33:11.856202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"84'1","reported_seq":23,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:33:05.470862+0000","last_change":"2024-08-19T01:32:38.916868+0000","last_active":"2024-08-19T01:33:05.470862+0000","last_peered":"2024-08-19T01:33:05.470862+0000","last_clean":"2024-08-19T01:33:05.470862+0000","last_became_active":"2024-08-19T01:32:38.916500+0000","last_became_peered":"2024-08-19T01:32:38.916500+0000","last_unstale":"2024-08-19T01:33:05.470862+0000","last_undegraded":"2024-08-19T01:33:05.470862+0000","last_fullsized":"2024-08-19T01:33:05.470862+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:32:37.889224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:32:37.889224+0000","last_clean_scrub_stamp":"2024-08-19T01:32:37.889224+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T12:30:07.085829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:33:05.465832+0000","last_change":"2024-08-19T01:32:41.235290+0000","last_active":"2024-08-19T01:33:05.465832+0000","last_peered":"2024-08-19T01:33:05.465832+0000","last_clean":"2024-08-19T01:33:05.465832+0000","last_became_active":"2024-08-19T01:32:41.235013+0000","last_became_peered":"2024-08-19T01:32:41.235013+0000","last_unstale":"2024-08-19T01:33:05.465832+0000","last_undegraded":"2024-08-19T01:33:05.465832+0000","last_fullsized":"2024-08-19T01:33:05.465832+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_clean_scrub_stamp":"2024-08-19T01:32:40.202519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T09:50:14.837080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"84'3","reported_seq":43,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:33:08.480633+0000","last_change":"2024-08-19T01:32:38.912023+0000","last_active":"2024-08-19T01:33:08.480633+0000","last_peered":"2024-08-19T01:33:08.480633+0000","last_clean":"2024-08-19T01:33:08.480633+0000","last_became_active":"2024-08-19T01:32:38.911679+0000","last_became_peered":"2024-08-19T01:32:38.911679+0000","last_unstale":"2024-08-19T01:33:08.480633+0000","last_undegraded":"2024-08-19T01:33:08.480633+0000","last_fullsized":"2024-08-19T01:33:08.480633+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:32:37.889224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:32:37.889224+0000","last_clean_scrub_stamp":"2024-08-19T01:32:37.889224+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T12:08:51.118338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":18,"num_read_kb":18,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:33:05.701747+0000","last_change":"2024-08-19T01:32:41.232264+0000","last_active":"2024-08-19T01:33:05.701747+0000","last_peered":"2024-08-19T01:33:05.701747+0000","last_clean":"2024-08-19T01:33:05.701747+0000","last_became_active":"2024-08-19T01:32:41.231788+0000","last_became_peered":"2024-08-19T01:32:41.231788+0000","last_unstale":"2024-08-19T01:33:05.701747+0000","last_undegraded":"2024-08-19T01:33:05.701747+0000","last_fullsized":"2024-08-19T01:33:05.701747+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_clean_scrub_stamp":"2024-08-19T01:32:40.202519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T04:59:06.443992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"84'2","reported_seq":24,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:33:05.472954+0000","last_change":"2024-08-19T01:32:38.915770+0000","last_active":"2024-08-19T01:33:05.472954+0000","last_peered":"2024-08-19T01:33:05.472954+0000","last_clean":"2024-08-19T01:33:05.472954+0000","last_became_active":"2024-08-19T01:32:38.915378+0000","last_became_peered":"2024-08-19T01:32:38.915378+0000","last_unstale":"2024-08-19T01:33:05.472954+0000","last_undegraded":"2024-08-19T01:33:05.472954+0000","last_fullsized":"2024-08-19T01:33:05.472954+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:32:37.889224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:32:37.889224+0000","last_clean_scrub_stamp":"2024-08-19T01:32:37.889224+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T09:16:14.513172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:33:05.469512+0000","last_change":"2024-08-19T01:32:41.243476+0000","last_active":"2024-08-19T01:33:05.469512+0000","last_peered":"2024-08-19T01:33:05.469512+0000","last_clean":"2024-08-19T01:33:05.469512+0000","last_became_active":"2024-08-19T01:32:41.243190+0000","last_became_peered":"2024-08-19T01:32:41.243190+0000","last_unstale":"2024-08-19T01:33:05.469512+0000","last_undegraded":"2024-08-19T01:33:05.469512+0000","last_fullsized":"2024-08-19T01:33:05.469512+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_clean_scrub_stamp":"2024-08-19T01:32:40.202519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T12:07:34.096919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"84'4","reported_seq":27,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:33:05.470664+0000","last_change":"2024-08-19T01:32:39.051458+0000","last_active":"2024-08-19T01:33:05.470664+0000","last_peered":"2024-08-19T01:33:05.470664+0000","last_clean":"2024-08-19T01:33:05.470664+0000","last_became_active":"2024-08-19T01:32:39.051113+0000","last_became_peered":"2024-08-19T01:32:39.051113+0000","last_unstale":"2024-08-19T01:33:05.470664+0000","last_undegraded":"2024-08-19T01:33:05.470664+0000","last_fullsized":"2024-08-19T01:33:05.470664+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:32:37.889224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:32:37.889224+0000","last_clean_scrub_stamp":"2024-08-19T01:32:37.889224+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T02:15:51.371054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:33:05.701579+0000","last_change":"2024-08-19T01:32:41.236453+0000","last_active":"2024-08-19T01:33:05.701579+0000","last_peered":"2024-08-19T01:33:05.701579+0000","last_clean":"2024-08-19T01:33:05.701579+0000","last_became_active":"2024-08-19T01:32:41.236135+0000","last_became_peered":"2024-08-19T01:32:41.236135+0000","last_unstale":"2024-08-19T01:33:05.701579+0000","last_undegraded":"2024-08-19T01:33:05.701579+0000","last_fullsized":"2024-08-19T01:33:05.701579+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_clean_scrub_stamp":"2024-08-19T01:32:40.202519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T12:38:41.398299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"85'25","reported_seq":90,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:33:09.482063+0000","last_change":"2024-08-19T01:32:38.914831+0000","last_active":"2024-08-19T01:33:09.482063+0000","last_peered":"2024-08-19T01:33:09.482063+0000","last_clean":"2024-08-19T01:33:09.482063+0000","last_became_active":"2024-08-19T01:32:38.914326+0000","last_became_peered":"2024-08-19T01:32:38.914326+0000","last_unstale":"2024-08-19T01:33:09.482063+0000","last_undegraded":"2024-08-19T01:33:09.482063+0000","last_fullsized":"2024-08-19T01:33:09.482063+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:32:37.889224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:32:37.889224+0000","last_clean_scrub_stamp":"2024-08-19T01:32:37.889224+0000","objects_scrubbed":0,"log_size":25,"log_dups_size":0,"ondisk_log_size":25,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T02:15:25.263974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":50114,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":43,"num_read_kb":69,"num_write":26,"num_write_kb":56,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:33:05.472530+0000","last_change":"2024-08-19T01:32:41.237945+0000","last_active":"2024-08-19T01:33:05.472530+0000","last_peered":"2024-08-19T01:33:05.472530+0000","last_clean":"2024-08-19T01:33:05.472530+0000","last_became_active":"2024-08-19T01:32:41.237362+0000","last_became_peered":"2024-08-19T01:32:41.237362+0000","last_unstale":"2024-08-19T01:33:05.472530+0000","last_undegraded":"2024-08-19T01:33:05.472530+0000","last_fullsized":"2024-08-19T01:33:05.472530+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_clean_scrub_stamp":"2024-08-19T01:32:40.202519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T07:29:17.180054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"84'6","reported_seq":29,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:33:05.466940+0000","last_change":"2024-08-19T01:32:38.912453+0000","last_active":"2024-08-19T01:33:05.466940+0000","last_peered":"2024-08-19T01:33:05.466940+0000","last_clean":"2024-08-19T01:33:05.466940+0000","last_became_active":"2024-08-19T01:32:38.912051+0000","last_became_peered":"2024-08-19T01:32:38.912051+0000","last_unstale":"2024-08-19T01:33:05.466940+0000","last_undegraded":"2024-08-19T01:33:05.466940+0000","last_fullsized":"2024-08-19T01:33:05.466940+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:32:37.889224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:32:37.889224+0000","last_clean_scrub_stamp":"2024-08-19T01:32:37.889224+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T06:49:58.810134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:33:05.477644+0000","last_change":"2024-08-19T01:32:41.236282+0000","last_active":"2024-08-19T01:33:05.477644+0000","last_peered":"2024-08-19T01:33:05.477644+0000","last_clean":"2024-08-19T01:33:05.477644+0000","last_became_active":"2024-08-19T01:32:41.235559+0000","last_became_peered":"2024-08-19T01:32:41.235559+0000","last_unstale":"2024-08-19T01:33:05.477644+0000","last_undegraded":"2024-08-19T01:33:05.477644+0000","last_fullsized":"2024-08-19T01:33:05.477644+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_clean_scrub_stamp":"2024-08-19T01:32:40.202519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T07:51:52.157094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"84'3","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:33:05.470566+0000","last_change":"2024-08-19T01:32:38.915688+0000","last_active":"2024-08-19T01:33:05.470566+0000","last_peered":"2024-08-19T01:33:05.470566+0000","last_clean":"2024-08-19T01:33:05.470566+0000","last_became_active":"2024-08-19T01:32:38.915297+0000","last_became_peered":"2024-08-19T01:32:38.915297+0000","last_unstale":"2024-08-19T01:33:05.470566+0000","last_undegraded":"2024-08-19T01:33:05.470566+0000","last_fullsized":"2024-08-19T01:33:05.470566+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:32:37.889224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:32:37.889224+0000","last_clean_scrub_stamp":"2024-08-19T01:32:37.889224+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T01:38:12.780588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:33:05.472886+0000","last_change":"2024-08-19T01:32:41.233766+0000","last_active":"2024-08-19T01:33:05.472886+0000","last_peered":"2024-08-19T01:33:05.472886+0000","last_clean":"2024-08-19T01:33:05.472886+0000","last_became_active":"2024-08-19T01:32:41.233425+0000","last_became_peered":"2024-08-19T01:32:41.233425+0000","last_unstale":"2024-08-19T01:33:05.472886+0000","last_undegraded":"2024-08-19T01:33:05.472886+0000","last_fullsized":"2024-08-19T01:33:05.472886+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_clean_scrub_stamp":"2024-08-19T01:32:40.202519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T10:29:15.471639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"84'4","reported_seq":45,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:33:06.319643+0000","last_change":"2024-08-19T01:32:38.912610+0000","last_active":"2024-08-19T01:33:06.319643+0000","last_peered":"2024-08-19T01:33:06.319643+0000","last_clean":"2024-08-19T01:33:06.319643+0000","last_became_active":"2024-08-19T01:32:38.912040+0000","last_became_peered":"2024-08-19T01:32:38.912040+0000","last_unstale":"2024-08-19T01:33:06.319643+0000","last_undegraded":"2024-08-19T01:33:06.319643+0000","last_fullsized":"2024-08-19T01:33:06.319643+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:32:37.889224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:32:37.889224+0000","last_clean_scrub_stamp":"2024-08-19T01:32:37.889224+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T09:26:05.966391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":978,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":20,"num_read_kb":4,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:33:05.472892+0000","last_change":"2024-08-19T01:32:41.235945+0000","last_active":"2024-08-19T01:33:05.472892+0000","last_peered":"2024-08-19T01:33:05.472892+0000","last_clean":"2024-08-19T01:33:05.472892+0000","last_became_active":"2024-08-19T01:32:41.235633+0000","last_became_peered":"2024-08-19T01:32:41.235633+0000","last_unstale":"2024-08-19T01:33:05.472892+0000","last_undegraded":"2024-08-19T01:33:05.472892+0000","last_fullsized":"2024-08-19T01:33:05.472892+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_clean_scrub_stamp":"2024-08-19T01:32:40.202519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T08:37:37.682175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"84'15","reported_seq":38,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:33:05.471087+0000","last_change":"2024-08-19T01:32:39.052850+0000","last_active":"2024-08-19T01:33:05.471087+0000","last_peered":"2024-08-19T01:33:05.471087+0000","last_clean":"2024-08-19T01:33:05.471087+0000","last_became_active":"2024-08-19T01:32:39.052455+0000","last_became_peered":"2024-08-19T01:32:39.052455+0000","last_unstale":"2024-08-19T01:33:05.471087+0000","last_undegraded":"2024-08-19T01:33:05.471087+0000","last_fullsized":"2024-08-19T01:33:05.471087+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:32:37.889224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:32:37.889224+0000","last_clean_scrub_stamp":"2024-08-19T01:32:37.889224+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T03:25:08.279911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":23,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:33:05.702575+0000","last_change":"2024-08-19T01:33:05.702575+0000","last_active":"2024-08-19T01:33:05.702575+0000","last_peered":"2024-08-19T01:33:05.702575+0000","last_clean":"2024-08-19T01:33:05.702575+0000","last_became_active":"2024-08-19T01:33:05.702110+0000","last_became_peered":"2024-08-19T01:33:05.702110+0000","last_unstale":"2024-08-19T01:33:05.702575+0000","last_undegraded":"2024-08-19T01:33:05.702575+0000","last_fullsized":"2024-08-19T01:33:05.702575+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_clean_scrub_stamp":"2024-08-19T01:32:40.202519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T12:18:51.102180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,1],"acting":[5,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"84'2","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:33:05.472730+0000","last_change":"2024-08-19T01:32:38.912090+0000","last_active":"2024-08-19T01:33:05.472730+0000","last_peered":"2024-08-19T01:33:05.472730+0000","last_clean":"2024-08-19T01:33:05.472730+0000","last_became_active":"2024-08-19T01:32:38.911753+0000","last_became_peered":"2024-08-19T01:32:38.911753+0000","last_unstale":"2024-08-19T01:33:05.472730+0000","last_undegraded":"2024-08-19T01:33:05.472730+0000","last_fullsized":"2024-08-19T01:33:05.472730+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:32:37.889224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:32:37.889224+0000","last_clean_scrub_stamp":"2024-08-19T01:32:37.889224+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T12:56:42.982062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:33:05.469805+0000","last_change":"2024-08-19T01:32:41.230429+0000","last_active":"2024-08-19T01:33:05.469805+0000","last_peered":"2024-08-19T01:33:05.469805+0000","last_clean":"2024-08-19T01:33:05.469805+0000","last_became_active":"2024-08-19T01:32:41.230115+0000","last_became_peered":"2024-08-19T01:32:41.230115+0000","last_unstale":"2024-08-19T01:33:05.469805+0000","last_undegraded":"2024-08-19T01:33:05.469805+0000","last_fullsized":"2024-08-19T01:33:05.469805+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_clean_scrub_stamp":"2024-08-19T01:32:40.202519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T01:50:51.084130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.e","version":"84'2","reported_seq":24,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:33:05.477826+0000","last_change":"2024-08-19T01:32:38.914557+0000","last_active":"2024-08-19T01:33:05.477826+0000","last_peered":"2024-08-19T01:33:05.477826+0000","last_clean":"2024-08-19T01:33:05.477826+0000","last_became_active":"2024-08-19T01:32:38.914115+0000","last_became_peered":"2024-08-19T01:32:38.914115+0000","last_unstale":"2024-08-19T01:33:05.477826+0000","last_undegraded":"2024-08-19T01:33:05.477826+0000","last_fullsized":"2024-08-19T01:33:05.477826+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:32:37.889224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:32:37.889224+0000","last_clean_scrub_stamp":"2024-08-19T01:32:37.889224+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T06:30:00.523513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:33:05.473436+0000","last_change":"2024-08-19T01:32:41.234883+0000","last_active":"2024-08-19T01:33:05.473436+0000","last_peered":"2024-08-19T01:33:05.473436+0000","last_clean":"2024-08-19T01:33:05.473436+0000","last_became_active":"2024-08-19T01:32:41.233901+0000","last_became_peered":"2024-08-19T01:32:41.233901+0000","last_unstale":"2024-08-19T01:33:05.473436+0000","last_undegraded":"2024-08-19T01:33:05.473436+0000","last_fullsized":"2024-08-19T01:33:05.473436+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_clean_scrub_stamp":"2024-08-19T01:32:40.202519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T12:08:46.305259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"84'1","reported_seq":24,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:33:05.470777+0000","last_change":"2024-08-19T01:32:38.913441+0000","last_active":"2024-08-19T01:33:05.470777+0000","last_peered":"2024-08-19T01:33:05.470777+0000","last_clean":"2024-08-19T01:33:05.470777+0000","last_became_active":"2024-08-19T01:32:38.913115+0000","last_became_peered":"2024-08-19T01:32:38.913115+0000","last_unstale":"2024-08-19T01:33:05.470777+0000","last_undegraded":"2024-08-19T01:33:05.470777+0000","last_fullsized":"2024-08-19T01:33:05.470777+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:32:37.889224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:32:37.889224+0000","last_clean_scrub_stamp":"2024-08-19T01:32:37.889224+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T11:43:15.290980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"84'1","reported_seq":18,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:33:05.472934+0000","last_change":"2024-08-19T01:32:41.235370+0000","last_active":"2024-08-19T01:33:05.472934+0000","last_peered":"2024-08-19T01:33:05.472934+0000","last_clean":"2024-08-19T01:33:05.472934+0000","last_became_active":"2024-08-19T01:32:41.235012+0000","last_became_peered":"2024-08-19T01:32:41.235012+0000","last_unstale":"2024-08-19T01:33:05.472934+0000","last_undegraded":"2024-08-19T01:33:05.472934+0000","last_fullsized":"2024-08-19T01:33:05.472934+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_clean_scrub_stamp":"2024-08-19T01:32:40.202519+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T02:18:31.134974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"84'3","reported_seq":26,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:33:05.466872+0000","last_change":"2024-08-19T01:32:39.245560+0000","last_active":"2024-08-19T01:33:05.466872+0000","last_peered":"2024-08-19T01:33:05.466872+0000","last_clean":"2024-08-19T01:33:05.466872+0000","last_became_active":"2024-08-19T01:32:39.245118+0000","last_became_peered":"2024-08-19T01:32:39.245118+0000","last_unstale":"2024-08-19T01:33:05.466872+0000","last_undegraded":"2024-08-19T01:33:05.466872+0000","last_fullsized":"2024-08-19T01:33:05.466872+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:32:37.889224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:32:37.889224+0000","last_clean_scrub_stamp":"2024-08-19T01:32:37.889224+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T02:18:24.605709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:33:05.473113+0000","last_change":"2024-08-19T01:32:41.234452+0000","last_active":"2024-08-19T01:33:05.473113+0000","last_peered":"2024-08-19T01:33:05.473113+0000","last_clean":"2024-08-19T01:33:05.473113+0000","last_became_active":"2024-08-19T01:32:41.233852+0000","last_became_peered":"2024-08-19T01:32:41.233852+0000","last_unstale":"2024-08-19T01:33:05.473113+0000","last_undegraded":"2024-08-19T01:33:05.473113+0000","last_fullsized":"2024-08-19T01:33:05.473113+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_clean_scrub_stamp":"2024-08-19T01:32:40.202519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T06:18:34.457239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"22'137","reported_seq":338,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:33:05.473232+0000","last_change":"2024-08-19T01:31:27.659394+0000","last_active":"2024-08-19T01:33:05.473232+0000","last_peered":"2024-08-19T01:33:05.473232+0000","last_clean":"2024-08-19T01:33:05.473232+0000","last_became_active":"2024-08-19T01:31:27.659007+0000","last_became_peered":"2024-08-19T01:31:27.659007+0000","last_unstale":"2024-08-19T01:33:05.473232+0000","last_undegraded":"2024-08-19T01:33:05.473232+0000","last_fullsized":"2024-08-19T01:33:05.473232+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:28:16.912251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:28:16.912251+0000","last_clean_scrub_stamp":"2024-08-19T01:28:16.912251+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T05:53:01.660619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"84'3","reported_seq":27,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:33:05.470433+0000","last_change":"2024-08-19T01:32:38.913408+0000","last_active":"2024-08-19T01:33:05.470433+0000","last_peered":"2024-08-19T01:33:05.470433+0000","last_clean":"2024-08-19T01:33:05.470433+0000","last_became_active":"2024-08-19T01:32:38.912242+0000","last_became_peered":"2024-08-19T01:32:38.912242+0000","last_unstale":"2024-08-19T01:33:05.470433+0000","last_undegraded":"2024-08-19T01:33:05.470433+0000","last_fullsized":"2024-08-19T01:33:05.470433+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:32:37.889224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:32:37.889224+0000","last_clean_scrub_stamp":"2024-08-19T01:32:37.889224+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T07:26:48.121366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:33:05.471752+0000","last_change":"2024-08-19T01:32:41.232611+0000","last_active":"2024-08-19T01:33:05.471752+0000","last_peered":"2024-08-19T01:33:05.471752+0000","last_clean":"2024-08-19T01:33:05.471752+0000","last_became_active":"2024-08-19T01:32:41.232261+0000","last_became_peered":"2024-08-19T01:32:41.232261+0000","last_unstale":"2024-08-19T01:33:05.471752+0000","last_undegraded":"2024-08-19T01:33:05.471752+0000","last_fullsized":"2024-08-19T01:33:05.471752+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_clean_scrub_stamp":"2024-08-19T01:32:40.202519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T08:04:16.793933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"84'3","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:33:05.473104+0000","last_change":"2024-08-19T01:32:38.912765+0000","last_active":"2024-08-19T01:33:05.473104+0000","last_peered":"2024-08-19T01:33:05.473104+0000","last_clean":"2024-08-19T01:33:05.473104+0000","last_became_active":"2024-08-19T01:32:38.912274+0000","last_became_peered":"2024-08-19T01:32:38.912274+0000","last_unstale":"2024-08-19T01:33:05.473104+0000","last_undegraded":"2024-08-19T01:33:05.473104+0000","last_fullsized":"2024-08-19T01:33:05.473104+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:32:37.889224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:32:37.889224+0000","last_clean_scrub_stamp":"2024-08-19T01:32:37.889224+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T09:57:49.714827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:33:05.477455+0000","last_change":"2024-08-19T01:32:41.224449+0000","last_active":"2024-08-19T01:33:05.477455+0000","last_peered":"2024-08-19T01:33:05.477455+0000","last_clean":"2024-08-19T01:33:05.477455+0000","last_became_active":"2024-08-19T01:32:41.224064+0000","last_became_peered":"2024-08-19T01:32:41.224064+0000","last_unstale":"2024-08-19T01:33:05.477455+0000","last_undegraded":"2024-08-19T01:33:05.477455+0000","last_fullsized":"2024-08-19T01:33:05.477455+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_clean_scrub_stamp":"2024-08-19T01:32:40.202519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T08:53:33.948504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"84'4","reported_seq":40,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:33:05.477561+0000","last_change":"2024-08-19T01:32:39.050869+0000","last_active":"2024-08-19T01:33:05.477561+0000","last_peered":"2024-08-19T01:33:05.477561+0000","last_clean":"2024-08-19T01:33:05.477561+0000","last_became_active":"2024-08-19T01:32:39.050529+0000","last_became_peered":"2024-08-19T01:32:39.050529+0000","last_unstale":"2024-08-19T01:33:05.477561+0000","last_undegraded":"2024-08-19T01:33:05.477561+0000","last_fullsized":"2024-08-19T01:33:05.477561+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:32:37.889224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:32:37.889224+0000","last_clean_scrub_stamp":"2024-08-19T01:32:37.889224+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T12:52:52.788360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":14,"num_read_kb":14,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:33:05.470330+0000","last_change":"2024-08-19T01:32:41.237093+0000","last_active":"2024-08-19T01:33:05.470330+0000","last_peered":"2024-08-19T01:33:05.470330+0000","last_clean":"2024-08-19T01:33:05.470330+0000","last_became_active":"2024-08-19T01:32:41.236738+0000","last_became_peered":"2024-08-19T01:32:41.236738+0000","last_unstale":"2024-08-19T01:33:05.470330+0000","last_undegraded":"2024-08-19T01:33:05.470330+0000","last_fullsized":"2024-08-19T01:33:05.470330+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_clean_scrub_stamp":"2024-08-19T01:32:40.202519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T08:37:33.117344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"84'1","reported_seq":23,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:33:05.473189+0000","last_change":"2024-08-19T01:32:38.915010+0000","last_active":"2024-08-19T01:33:05.473189+0000","last_peered":"2024-08-19T01:33:05.473189+0000","last_clean":"2024-08-19T01:33:05.473189+0000","last_became_active":"2024-08-19T01:32:38.914687+0000","last_became_peered":"2024-08-19T01:32:38.914687+0000","last_unstale":"2024-08-19T01:33:05.473189+0000","last_undegraded":"2024-08-19T01:33:05.473189+0000","last_fullsized":"2024-08-19T01:33:05.473189+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:32:37.889224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:32:37.889224+0000","last_clean_scrub_stamp":"2024-08-19T01:32:37.889224+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T04:46:07.881778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:33:05.474056+0000","last_change":"2024-08-19T01:32:41.244069+0000","last_active":"2024-08-19T01:33:05.474056+0000","last_peered":"2024-08-19T01:33:05.474056+0000","last_clean":"2024-08-19T01:33:05.474056+0000","last_became_active":"2024-08-19T01:32:41.243482+0000","last_became_peered":"2024-08-19T01:32:41.243482+0000","last_unstale":"2024-08-19T01:33:05.474056+0000","last_undegraded":"2024-08-19T01:33:05.474056+0000","last_fullsized":"2024-08-19T01:33:05.474056+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_clean_scrub_stamp":"2024-08-19T01:32:40.202519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T02:08:28.044421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":22,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:33:05.466068+0000","last_change":"2024-08-19T01:32:39.051383+0000","last_active":"2024-08-19T01:33:05.466068+0000","last_peered":"2024-08-19T01:33:05.466068+0000","last_clean":"2024-08-19T01:33:05.466068+0000","last_became_active":"2024-08-19T01:32:39.050970+0000","last_became_peered":"2024-08-19T01:32:39.050970+0000","last_unstale":"2024-08-19T01:33:05.466068+0000","last_undegraded":"2024-08-19T01:33:05.466068+0000","last_fullsized":"2024-08-19T01:33:05.466068+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:32:37.889224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:32:37.889224+0000","last_clean_scrub_stamp":"2024-08-19T01:32:37.889224+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T11:31:30.461651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"84'1","reported_seq":18,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:33:05.474420+0000","last_change":"2024-08-19T01:32:41.232656+0000","last_active":"2024-08-19T01:33:05.474420+0000","last_peered":"2024-08-19T01:33:05.474420+0000","last_clean":"2024-08-19T01:33:05.474420+0000","last_became_active":"2024-08-19T01:32:41.232274+0000","last_became_peered":"2024-08-19T01:32:41.232274+0000","last_unstale":"2024-08-19T01:33:05.474420+0000","last_undegraded":"2024-08-19T01:33:05.474420+0000","last_fullsized":"2024-08-19T01:33:05.474420+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_clean_scrub_stamp":"2024-08-19T01:32:40.202519+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T03:38:40.695652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.5","version":"84'1","reported_seq":24,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:33:05.472592+0000","last_change":"2024-08-19T01:32:39.050809+0000","last_active":"2024-08-19T01:33:05.472592+0000","last_peered":"2024-08-19T01:33:05.472592+0000","last_clean":"2024-08-19T01:33:05.472592+0000","last_became_active":"2024-08-19T01:32:39.050434+0000","last_became_peered":"2024-08-19T01:32:39.050434+0000","last_unstale":"2024-08-19T01:33:05.472592+0000","last_undegraded":"2024-08-19T01:33:05.472592+0000","last_fullsized":"2024-08-19T01:33:05.472592+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:32:37.889224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:32:37.889224+0000","last_clean_scrub_stamp":"2024-08-19T01:32:37.889224+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T13:04:28.828117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:33:05.477935+0000","last_change":"2024-08-19T01:32:41.228485+0000","last_active":"2024-08-19T01:33:05.477935+0000","last_peered":"2024-08-19T01:33:05.477935+0000","last_clean":"2024-08-19T01:33:05.477935+0000","last_became_active":"2024-08-19T01:32:41.227238+0000","last_became_peered":"2024-08-19T01:32:41.227238+0000","last_unstale":"2024-08-19T01:33:05.477935+0000","last_undegraded":"2024-08-19T01:33:05.477935+0000","last_fullsized":"2024-08-19T01:33:05.477935+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_clean_scrub_stamp":"2024-08-19T01:32:40.202519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T10:55:55.939382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"85'9","reported_seq":31,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:33:05.466322+0000","last_change":"2024-08-19T01:32:38.915994+0000","last_active":"2024-08-19T01:33:05.466322+0000","last_peered":"2024-08-19T01:33:05.466322+0000","last_clean":"2024-08-19T01:33:05.466322+0000","last_became_active":"2024-08-19T01:32:38.914714+0000","last_became_peered":"2024-08-19T01:32:38.914714+0000","last_unstale":"2024-08-19T01:33:05.466322+0000","last_undegraded":"2024-08-19T01:33:05.466322+0000","last_fullsized":"2024-08-19T01:33:05.466322+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:32:37.889224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:32:37.889224+0000","last_clean_scrub_stamp":"2024-08-19T01:32:37.889224+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T05:38:07.274274+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:33:05.472281+0000","last_change":"2024-08-19T01:32:41.236611+0000","last_active":"2024-08-19T01:33:05.472281+0000","last_peered":"2024-08-19T01:33:05.472281+0000","last_clean":"2024-08-19T01:33:05.472281+0000","last_became_active":"2024-08-19T01:32:41.233821+0000","last_became_peered":"2024-08-19T01:32:41.233821+0000","last_unstale":"2024-08-19T01:33:05.472281+0000","last_undegraded":"2024-08-19T01:33:05.472281+0000","last_fullsized":"2024-08-19T01:33:05.472281+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_clean_scrub_stamp":"2024-08-19T01:32:40.202519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T11:34:07.879034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"84'4","reported_seq":26,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:33:05.470567+0000","last_change":"2024-08-19T01:32:39.245411+0000","last_active":"2024-08-19T01:33:05.470567+0000","last_peered":"2024-08-19T01:33:05.470567+0000","last_clean":"2024-08-19T01:33:05.470567+0000","last_became_active":"2024-08-19T01:32:39.244980+0000","last_became_peered":"2024-08-19T01:32:39.244980+0000","last_unstale":"2024-08-19T01:33:05.470567+0000","last_undegraded":"2024-08-19T01:33:05.470567+0000","last_fullsized":"2024-08-19T01:33:05.470567+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:32:37.889224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:32:37.889224+0000","last_clean_scrub_stamp":"2024-08-19T01:32:37.889224+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T02:06:57.878051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":23,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:33:05.478955+0000","last_change":"2024-08-19T01:33:05.478955+0000","last_active":"2024-08-19T01:33:05.478955+0000","last_peered":"2024-08-19T01:33:05.478955+0000","last_clean":"2024-08-19T01:33:05.478955+0000","last_became_active":"2024-08-19T01:33:05.478446+0000","last_became_peered":"2024-08-19T01:33:05.478446+0000","last_unstale":"2024-08-19T01:33:05.478955+0000","last_undegraded":"2024-08-19T01:33:05.478955+0000","last_fullsized":"2024-08-19T01:33:05.478955+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_clean_scrub_stamp":"2024-08-19T01:32:40.202519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T11:58:24.285438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,6],"acting":[0,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"84'2","reported_seq":24,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:33:05.701553+0000","last_change":"2024-08-19T01:32:39.050100+0000","last_active":"2024-08-19T01:33:05.701553+0000","last_peered":"2024-08-19T01:33:05.701553+0000","last_clean":"2024-08-19T01:33:05.701553+0000","last_became_active":"2024-08-19T01:32:39.049807+0000","last_became_peered":"2024-08-19T01:32:39.049807+0000","last_unstale":"2024-08-19T01:33:05.701553+0000","last_undegraded":"2024-08-19T01:33:05.701553+0000","last_fullsized":"2024-08-19T01:33:05.701553+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:32:37.889224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:32:37.889224+0000","last_clean_scrub_stamp":"2024-08-19T01:32:37.889224+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T07:06:22.361609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:33:05.471209+0000","last_change":"2024-08-19T01:32:41.228982+0000","last_active":"2024-08-19T01:33:05.471209+0000","last_peered":"2024-08-19T01:33:05.471209+0000","last_clean":"2024-08-19T01:33:05.471209+0000","last_became_active":"2024-08-19T01:32:41.228562+0000","last_became_peered":"2024-08-19T01:32:41.228562+0000","last_unstale":"2024-08-19T01:33:05.471209+0000","last_undegraded":"2024-08-19T01:33:05.471209+0000","last_fullsized":"2024-08-19T01:33:05.471209+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_clean_scrub_stamp":"2024-08-19T01:32:40.202519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T03:14:42.480359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"84'3","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:33:05.471722+0000","last_change":"2024-08-19T01:32:38.913677+0000","last_active":"2024-08-19T01:33:05.471722+0000","last_peered":"2024-08-19T01:33:05.471722+0000","last_clean":"2024-08-19T01:33:05.471722+0000","last_became_active":"2024-08-19T01:32:38.913091+0000","last_became_peered":"2024-08-19T01:32:38.913091+0000","last_unstale":"2024-08-19T01:33:05.471722+0000","last_undegraded":"2024-08-19T01:33:05.471722+0000","last_fullsized":"2024-08-19T01:33:05.471722+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:32:37.889224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:32:37.889224+0000","last_clean_scrub_stamp":"2024-08-19T01:32:37.889224+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T12:39:16.489246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:33:05.469625+0000","last_change":"2024-08-19T01:32:41.239380+0000","last_active":"2024-08-19T01:33:05.469625+0000","last_peered":"2024-08-19T01:33:05.469625+0000","last_clean":"2024-08-19T01:33:05.469625+0000","last_became_active":"2024-08-19T01:32:41.239060+0000","last_became_peered":"2024-08-19T01:32:41.239060+0000","last_unstale":"2024-08-19T01:33:05.469625+0000","last_undegraded":"2024-08-19T01:33:05.469625+0000","last_fullsized":"2024-08-19T01:33:05.469625+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_clean_scrub_stamp":"2024-08-19T01:32:40.202519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T02:21:59.630705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"84'1","reported_seq":15,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:33:05.479101+0000","last_change":"2024-08-19T01:33:05.478578+0000","last_active":"2024-08-19T01:33:05.479101+0000","last_peered":"2024-08-19T01:33:05.479101+0000","last_clean":"2024-08-19T01:33:05.479101+0000","last_became_active":"2024-08-19T01:33:05.473433+0000","last_became_peered":"2024-08-19T01:33:05.473433+0000","last_unstale":"2024-08-19T01:33:05.479101+0000","last_undegraded":"2024-08-19T01:33:05.479101+0000","last_fullsized":"2024-08-19T01:33:05.479101+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:32:37.889224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:32:37.889224+0000","last_clean_scrub_stamp":"2024-08-19T01:32:37.889224+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T05:06:29.975613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,3,8],"acting":[11,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:33:05.472017+0000","last_change":"2024-08-19T01:32:41.236012+0000","last_active":"2024-08-19T01:33:05.472017+0000","last_peered":"2024-08-19T01:33:05.472017+0000","last_clean":"2024-08-19T01:33:05.472017+0000","last_became_active":"2024-08-19T01:32:41.235724+0000","last_became_peered":"2024-08-19T01:32:41.235724+0000","last_unstale":"2024-08-19T01:33:05.472017+0000","last_undegraded":"2024-08-19T01:33:05.472017+0000","last_fullsized":"2024-08-19T01:33:05.472017+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_clean_scrub_stamp":"2024-08-19T01:32:40.202519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T08:45:08.865144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"84'4","reported_seq":27,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:33:05.471339+0000","last_change":"2024-08-19T01:32:38.915170+0000","last_active":"2024-08-19T01:33:05.471339+0000","last_peered":"2024-08-19T01:33:05.471339+0000","last_clean":"2024-08-19T01:33:05.471339+0000","last_became_active":"2024-08-19T01:32:38.914812+0000","last_became_peered":"2024-08-19T01:32:38.914812+0000","last_unstale":"2024-08-19T01:33:05.471339+0000","last_undegraded":"2024-08-19T01:33:05.471339+0000","last_fullsized":"2024-08-19T01:33:05.471339+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:32:37.889224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:32:37.889224+0000","last_clean_scrub_stamp":"2024-08-19T01:32:37.889224+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T11:46:39.409065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:33:05.472797+0000","last_change":"2024-08-19T01:32:41.234133+0000","last_active":"2024-08-19T01:33:05.472797+0000","last_peered":"2024-08-19T01:33:05.472797+0000","last_clean":"2024-08-19T01:33:05.472797+0000","last_became_active":"2024-08-19T01:32:41.233492+0000","last_became_peered":"2024-08-19T01:32:41.233492+0000","last_unstale":"2024-08-19T01:33:05.472797+0000","last_undegraded":"2024-08-19T01:33:05.472797+0000","last_fullsized":"2024-08-19T01:33:05.472797+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_clean_scrub_stamp":"2024-08-19T01:32:40.202519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T06:11:12.587678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:33:05.469674+0000","last_change":"2024-08-19T01:32:41.232628+0000","last_active":"2024-08-19T01:33:05.469674+0000","last_peered":"2024-08-19T01:33:05.469674+0000","last_clean":"2024-08-19T01:33:05.469674+0000","last_became_active":"2024-08-19T01:32:41.232295+0000","last_became_peered":"2024-08-19T01:32:41.232295+0000","last_unstale":"2024-08-19T01:33:05.469674+0000","last_undegraded":"2024-08-19T01:33:05.469674+0000","last_fullsized":"2024-08-19T01:33:05.469674+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_clean_scrub_stamp":"2024-08-19T01:32:40.202519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T02:01:09.490553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"84'2","reported_seq":24,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:33:05.472666+0000","last_change":"2024-08-19T01:32:38.912000+0000","last_active":"2024-08-19T01:33:05.472666+0000","last_peered":"2024-08-19T01:33:05.472666+0000","last_clean":"2024-08-19T01:33:05.472666+0000","last_became_active":"2024-08-19T01:32:38.911535+0000","last_became_peered":"2024-08-19T01:32:38.911535+0000","last_unstale":"2024-08-19T01:33:05.472666+0000","last_undegraded":"2024-08-19T01:33:05.472666+0000","last_fullsized":"2024-08-19T01:33:05.472666+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:32:37.889224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:32:37.889224+0000","last_clean_scrub_stamp":"2024-08-19T01:32:37.889224+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T07:12:49.358384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:33:05.701576+0000","last_change":"2024-08-19T01:32:41.233507+0000","last_active":"2024-08-19T01:33:05.701576+0000","last_peered":"2024-08-19T01:33:05.701576+0000","last_clean":"2024-08-19T01:33:05.701576+0000","last_became_active":"2024-08-19T01:32:41.233179+0000","last_became_peered":"2024-08-19T01:32:41.233179+0000","last_unstale":"2024-08-19T01:33:05.701576+0000","last_undegraded":"2024-08-19T01:33:05.701576+0000","last_fullsized":"2024-08-19T01:33:05.701576+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_clean_scrub_stamp":"2024-08-19T01:32:40.202519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T11:47:44.780859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1d","version":"84'4","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:33:05.483750+0000","last_change":"2024-08-19T01:33:05.483322+0000","last_active":"2024-08-19T01:33:05.483750+0000","last_peered":"2024-08-19T01:33:05.483750+0000","last_clean":"2024-08-19T01:33:05.483750+0000","last_became_active":"2024-08-19T01:33:05.475496+0000","last_became_peered":"2024-08-19T01:33:05.475496+0000","last_unstale":"2024-08-19T01:33:05.483750+0000","last_undegraded":"2024-08-19T01:33:05.483750+0000","last_fullsized":"2024-08-19T01:33:05.483750+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:32:37.889224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:32:37.889224+0000","last_clean_scrub_stamp":"2024-08-19T01:32:37.889224+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T01:57:48.143495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":10,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,7],"acting":[11,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:33:05.469647+0000","last_change":"2024-08-19T01:32:41.244641+0000","last_active":"2024-08-19T01:33:05.469647+0000","last_peered":"2024-08-19T01:33:05.469647+0000","last_clean":"2024-08-19T01:33:05.469647+0000","last_became_active":"2024-08-19T01:32:41.244311+0000","last_became_peered":"2024-08-19T01:32:41.244311+0000","last_unstale":"2024-08-19T01:33:05.469647+0000","last_undegraded":"2024-08-19T01:33:05.469647+0000","last_fullsized":"2024-08-19T01:33:05.469647+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_clean_scrub_stamp":"2024-08-19T01:32:40.202519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T10:39:12.591092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"84'2","reported_seq":24,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:33:05.472352+0000","last_change":"2024-08-19T01:32:39.051314+0000","last_active":"2024-08-19T01:33:05.472352+0000","last_peered":"2024-08-19T01:33:05.472352+0000","last_clean":"2024-08-19T01:33:05.472352+0000","last_became_active":"2024-08-19T01:32:39.050998+0000","last_became_peered":"2024-08-19T01:32:39.050998+0000","last_unstale":"2024-08-19T01:33:05.472352+0000","last_undegraded":"2024-08-19T01:33:05.472352+0000","last_fullsized":"2024-08-19T01:33:05.472352+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:32:37.889224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:32:37.889224+0000","last_clean_scrub_stamp":"2024-08-19T01:32:37.889224+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T09:21:15.689149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:33:05.472112+0000","last_change":"2024-08-19T01:32:41.234762+0000","last_active":"2024-08-19T01:33:05.472112+0000","last_peered":"2024-08-19T01:33:05.472112+0000","last_clean":"2024-08-19T01:33:05.472112+0000","last_became_active":"2024-08-19T01:32:41.234471+0000","last_became_peered":"2024-08-19T01:32:41.234471+0000","last_unstale":"2024-08-19T01:33:05.472112+0000","last_undegraded":"2024-08-19T01:33:05.472112+0000","last_fullsized":"2024-08-19T01:33:05.472112+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_clean_scrub_stamp":"2024-08-19T01:32:40.202519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T08:34:33.110938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"84'9","reported_seq":32,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:33:05.466470+0000","last_change":"2024-08-19T01:32:38.917068+0000","last_active":"2024-08-19T01:33:05.466470+0000","last_peered":"2024-08-19T01:33:05.466470+0000","last_clean":"2024-08-19T01:33:05.466470+0000","last_became_active":"2024-08-19T01:32:38.915399+0000","last_became_peered":"2024-08-19T01:32:38.915399+0000","last_unstale":"2024-08-19T01:33:05.466470+0000","last_undegraded":"2024-08-19T01:33:05.466470+0000","last_fullsized":"2024-08-19T01:33:05.466470+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:32:37.889224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:32:37.889224+0000","last_clean_scrub_stamp":"2024-08-19T01:32:37.889224+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T04:17:25.061208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":54504,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":125,"num_read_kb":121,"num_write":157,"num_write_kb":98,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":1012,"num_keys_recovered":10,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":393216,"data_stored":163512,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":134,"ondisk_log_size":134,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4751360,"data_stored":4722944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8}],"osd_stats":[{"osd":11,"up_from":78,"seq":335007449106,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32100,"kb_used_data":916,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713052,"statfs":{"total":95995035648,"available":95962165248,"internally_reserved":0,"allocated":937984,"data_stored":451051,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":71,"seq":304942678039,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28612,"kb_used_data":1588,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716540,"statfs":{"total":95995035648,"available":95965736960,"internally_reserved":0,"allocated":1626112,"data_stored":1093036,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":66,"seq":283467841563,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32116,"kb_used_data":932,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713036,"statfs":{"total":95995035648,"available":95962148864,"internally_reserved":0,"allocated":954368,"data_stored":451287,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":8,"up_from":58,"seq":249108103201,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32116,"kb_used_data":932,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713036,"statfs":{"total":95995035648,"available":95962148864,"internally_reserved":0,"allocated":954368,"data_stored":451207,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":51,"seq":219043332133,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32696,"kb_used_data":1512,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712456,"statfs":{"total":95995035648,"available":95961554944,"internally_reserved":0,"allocated":1548288,"data_stored":1043526,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":46,"seq":197568495658,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27924,"kb_used_data":904,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717228,"statfs":{"total":95995035648,"available":95966441472,"internally_reserved":0,"allocated":925696,"data_stored":442159,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":38,"seq":163208757295,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28548,"kb_used_data":1524,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716604,"statfs":{"total":95995035648,"available":95965802496,"internally_reserved":0,"allocated":1560576,"data_stored":1042820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":33,"seq":141733920819,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32112,"kb_used_data":932,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713040,"statfs":{"total":95995035648,"available":95962152960,"internally_reserved":0,"allocated":954368,"data_stored":452515,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182456,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27956,"kb_used_data":936,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717196,"statfs":{"total":95995035648,"available":95966408704,"internally_reserved":0,"allocated":958464,"data_stored":450987,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411390,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36332,"kb_used_data":988,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708820,"statfs":{"total":95995035648,"available":95957831680,"internally_reserved":0,"allocated":1011712,"data_stored":501311,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574912,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27996,"kb_used_data":972,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717156,"statfs":{"total":95995035648,"available":95966367744,"internally_reserved":0,"allocated":995328,"data_stored":500583,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738438,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36284,"kb_used_data":940,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708868,"statfs":{"total":95995035648,"available":95957880832,"internally_reserved":0,"allocated":962560,"data_stored":451947,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":50114,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":77824,"data_stored":50842,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2046,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1866,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2572,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":94208,"data_stored":52082,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-08-19T01:33:12.307 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2024-08-19T01:33:12.308 INFO:tasks.cephadm:Setting up client nodes... 2024-08-19T01:33:12.308 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid bad8780e-5dc9-11ef-bccf-c7b262605968 -- ceph auth get-or-create client.0 mon 'allow *' osd 'allow *' mds 'allow *' mgr 'allow *' 2024-08-19T01:33:12.654 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:33:12 smithi083 ceph-mon[17120]: from='client.24671 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-08-19T01:33:12.686 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/bad8780e-5dc9-11ef-bccf-c7b262605968/mon.a/config 2024-08-19T01:33:12.952 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:33:12 smithi117 ceph-mon[22016]: from='client.24671 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-08-19T01:33:12.966 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:33:12 smithi167 ceph-mon[21875]: from='client.24671 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-08-19T01:33:13.243 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:13.241+0000 7f1c1de00640 1 -- 172.21.15.83:0/1376293544 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1c18103b30 msgr2=0x7f1c18103f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:33:13.243 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:13.241+0000 7f1c1de00640 1 --2- 172.21.15.83:0/1376293544 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1c18103b30 0x7f1c18103f30 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f1c08009900 tx=0x7f1c0802a410 comp rx=0 tx=0).stop 2024-08-19T01:33:13.243 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:13.241+0000 7f1c1de00640 1 -- 172.21.15.83:0/1376293544 shutdown_connections 2024-08-19T01:33:13.244 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:13.241+0000 7f1c1de00640 1 --2- 172.21.15.83:0/1376293544 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1c181056f0 0x7f1c1810bf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:33:13.244 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:13.241+0000 7f1c1de00640 1 --2- 172.21.15.83:0/1376293544 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1c18104d30 0x7f1c181051b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:33:13.244 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:13.241+0000 7f1c1de00640 1 --2- 172.21.15.83:0/1376293544 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1c18103b30 0x7f1c18103f30 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:33:13.244 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:13.241+0000 7f1c1de00640 1 -- 172.21.15.83:0/1376293544 >> 172.21.15.83:0/1376293544 conn(0x7f1c180ff2e0 msgr2=0x7f1c18101700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:33:13.244 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:13.241+0000 7f1c1de00640 1 -- 172.21.15.83:0/1376293544 shutdown_connections 2024-08-19T01:33:13.244 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:13.241+0000 7f1c1de00640 1 -- 172.21.15.83:0/1376293544 wait complete. 2024-08-19T01:33:13.245 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:13.242+0000 7f1c1de00640 1 Processor -- start 2024-08-19T01:33:13.246 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:13.242+0000 7f1c1de00640 1 -- start start 2024-08-19T01:33:13.246 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:13.242+0000 7f1c1de00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1c18103b30 0x7f1c18199e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:33:13.246 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:13.242+0000 7f1c1de00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1c18104d30 0x7f1c1819a390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:33:13.246 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:13.243+0000 7f1c1de00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1c181056f0 0x7f1c181a1410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:33:13.246 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:13.243+0000 7f1c1de00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f1c1810f800 con 0x7f1c18103b30 2024-08-19T01:33:13.246 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:13.243+0000 7f1c1de00640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f1c1810f680 con 0x7f1c181056f0 2024-08-19T01:33:13.246 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:13.243+0000 7f1c1de00640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f1c1810f980 con 0x7f1c18104d30 2024-08-19T01:33:13.246 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:13.243+0000 7f1c16a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1c18103b30 0x7f1c18199e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:33:13.246 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:13.243+0000 7f1c16000640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1c18104d30 0x7f1c1819a390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:33:13.246 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:13.243+0000 7f1c16a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1c18103b30 0x7f1c18199e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:42076/0 (socket says 172.21.15.83:42076) 2024-08-19T01:33:13.246 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:13.243+0000 7f1c16000640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1c18104d30 0x7f1c1819a390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.83:39168/0 (socket says 172.21.15.83:39168) 2024-08-19T01:33:13.246 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:13.243+0000 7f1c16a00640 1 -- 172.21.15.83:0/2085287220 learned_addr learned my addr 172.21.15.83:0/2085287220 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-08-19T01:33:13.247 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:13.243+0000 7f1c17400640 1 --2- 172.21.15.83:0/2085287220 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1c181056f0 0x7f1c181a1410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:33:13.247 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:13.243+0000 7f1c17400640 1 -- 172.21.15.83:0/2085287220 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1c18104d30 msgr2=0x7f1c1819a390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:33:13.247 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:13.243+0000 7f1c17400640 1 --2- 172.21.15.83:0/2085287220 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1c18104d30 0x7f1c1819a390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:33:13.247 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:13.243+0000 7f1c17400640 1 -- 172.21.15.83:0/2085287220 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1c18103b30 msgr2=0x7f1c18199e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:33:13.247 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:13.243+0000 7f1c17400640 1 --2- 172.21.15.83:0/2085287220 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1c18103b30 0x7f1c18199e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:33:13.247 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:13.243+0000 7f1c17400640 1 -- 172.21.15.83:0/2085287220 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1c181a1a80 con 0x7f1c181056f0 2024-08-19T01:33:13.247 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:13.244+0000 7f1c16a00640 1 --2- 172.21.15.83:0/2085287220 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1c18103b30 0x7f1c18199e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-19T01:33:13.247 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:13.244+0000 7f1c17400640 1 --2- 172.21.15.83:0/2085287220 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1c181056f0 0x7f1c181a1410 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f1c10009fd0 tx=0x7f1c1000ef10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:33:13.247 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:13.244+0000 7f1c1ca00640 1 -- 172.21.15.83:0/2085287220 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c1000ce20 con 0x7f1c181056f0 2024-08-19T01:33:13.247 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:13.244+0000 7f1c1ca00640 1 -- 172.21.15.83:0/2085287220 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1c10005020 con 0x7f1c181056f0 2024-08-19T01:33:13.247 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:13.244+0000 7f1c1de00640 1 -- 172.21.15.83:0/2085287220 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1c181a1d70 con 0x7f1c181056f0 2024-08-19T01:33:13.247 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:13.244+0000 7f1c1ca00640 1 -- 172.21.15.83:0/2085287220 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c10010720 con 0x7f1c181056f0 2024-08-19T01:33:13.248 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:13.244+0000 7f1c1de00640 1 -- 172.21.15.83:0/2085287220 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1c181a2250 con 0x7f1c181056f0 2024-08-19T01:33:13.248 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:13.245+0000 7f1c1de00640 1 -- 172.21.15.83:0/2085287220 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1c1810aee0 con 0x7f1c181056f0 2024-08-19T01:33:13.248 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:13.247+0000 7f1c1ca00640 1 -- 172.21.15.83:0/2085287220 <== mon.2 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f1c10005540 con 0x7f1c181056f0 2024-08-19T01:33:13.250 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:13.248+0000 7f1c1ca00640 1 --2- 172.21.15.83:0/2085287220 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f1bfc0a2cc0 0x7f1bfc0a5180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:33:13.250 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:13.249+0000 7f1c1ca00640 1 -- 172.21.15.83:0/2085287220 <== mon.2 v2:172.21.15.117:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8269+0+0 (secure 0 0 0) 0x7f1c100cf870 con 0x7f1c181056f0 2024-08-19T01:33:13.251 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:13.249+0000 7f1c16a00640 1 --2- 172.21.15.83:0/2085287220 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f1bfc0a2cc0 0x7f1bfc0a5180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:33:13.251 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:13.249+0000 7f1c16a00640 1 --2- 172.21.15.83:0/2085287220 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f1bfc0a2cc0 0x7f1bfc0a5180 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f1c08000c00 tx=0x7f1c080315b0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:33:13.254 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:13.253+0000 7f1c1ca00640 1 -- 172.21.15.83:0/2085287220 <== mon.2 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7f1c1009c5f0 con 0x7f1c181056f0 2024-08-19T01:33:13.499 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:13.497+0000 7f1c1de00640 1 -- 172.21.15.83:0/2085287220 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]} v 0) -- 0x7f1c1802d5e0 con 0x7f1c181056f0 2024-08-19T01:33:13.508 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:13.506+0000 7f1c1ca00640 1 -- 172.21.15.83:0/2085287220 <== mon.2 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]=0 v31) ==== 170+0+59 (secure 0 0 0) 0x7f1c1009bef0 con 0x7f1c181056f0 2024-08-19T01:33:13.508 INFO:teuthology.orchestra.run.smithi083.stdout:[client.0] 2024-08-19T01:33:13.508 INFO:teuthology.orchestra.run.smithi083.stdout: key = AQDZoMJmzc3/HRAAfLzEZHGCGOcOQvjIE/d/ag== 2024-08-19T01:33:13.511 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:13.509+0000 7f1c1de00640 1 -- 172.21.15.83:0/2085287220 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f1bfc0a2cc0 msgr2=0x7f1bfc0a5180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:33:13.511 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:13.509+0000 7f1c1de00640 1 --2- 172.21.15.83:0/2085287220 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f1bfc0a2cc0 0x7f1bfc0a5180 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f1c08000c00 tx=0x7f1c080315b0 comp rx=0 tx=0).stop 2024-08-19T01:33:13.511 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:13.509+0000 7f1c1de00640 1 -- 172.21.15.83:0/2085287220 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1c181056f0 msgr2=0x7f1c181a1410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:33:13.511 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:13.509+0000 7f1c1de00640 1 --2- 172.21.15.83:0/2085287220 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1c181056f0 0x7f1c181a1410 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f1c10009fd0 tx=0x7f1c1000ef10 comp rx=0 tx=0).stop 2024-08-19T01:33:13.511 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:13.509+0000 7f1c1de00640 1 -- 172.21.15.83:0/2085287220 shutdown_connections 2024-08-19T01:33:13.511 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:13.509+0000 7f1c1de00640 1 --2- 172.21.15.83:0/2085287220 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f1bfc0a2cc0 0x7f1bfc0a5180 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:33:13.511 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:13.509+0000 7f1c1de00640 1 --2- 172.21.15.83:0/2085287220 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1c181056f0 0x7f1c181a1410 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:33:13.511 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:13.509+0000 7f1c1de00640 1 --2- 172.21.15.83:0/2085287220 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1c18104d30 0x7f1c1819a390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:33:13.511 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:13.509+0000 7f1c1de00640 1 --2- 172.21.15.83:0/2085287220 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1c18103b30 0x7f1c18199e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:33:13.511 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:13.509+0000 7f1c1de00640 1 -- 172.21.15.83:0/2085287220 >> 172.21.15.83:0/2085287220 conn(0x7f1c180ff2e0 msgr2=0x7f1c181080b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:33:13.512 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:13.509+0000 7f1c1de00640 1 -- 172.21.15.83:0/2085287220 shutdown_connections 2024-08-19T01:33:13.512 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:13.509+0000 7f1c1de00640 1 -- 172.21.15.83:0/2085287220 wait complete. 2024-08-19T01:33:13.854 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:33:13 smithi083 ceph-mon[17120]: pgmap v290: 97 pgs: 97 active+clean; 630 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 4.0 KiB/s wr, 8 op/s; 126 B/s, 1 keys/s, 0 objects/s recovering 2024-08-19T01:33:13.952 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:33:13 smithi117 ceph-mon[22016]: pgmap v290: 97 pgs: 97 active+clean; 630 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 4.0 KiB/s wr, 8 op/s; 126 B/s, 1 keys/s, 0 objects/s recovering 2024-08-19T01:33:13.966 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:33:13 smithi167 ceph-mon[21875]: pgmap v290: 97 pgs: 97 active+clean; 630 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 4.0 KiB/s wr, 8 op/s; 126 B/s, 1 keys/s, 0 objects/s recovering 2024-08-19T01:33:14.315 DEBUG:teuthology.orchestra.run.smithi083:> set -ex 2024-08-19T01:33:14.315 DEBUG:teuthology.orchestra.run.smithi083:> sudo dd of=/etc/ceph/ceph.client.0.keyring 2024-08-19T01:33:14.316 DEBUG:teuthology.orchestra.run.smithi083:> sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-08-19T01:33:14.395 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-08-19T01:33:14.395 INFO:tasks.cephadm.ceph_manager.ceph:waiting for mgr available 2024-08-19T01:33:14.395 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833 shell --fsid bad8780e-5dc9-11ef-bccf-c7b262605968 -- ceph mgr dump --format=json 2024-08-19T01:33:14.654 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:33:14 smithi083 ceph-mon[17120]: from='client.? 172.21.15.83:0/2085287220' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-08-19T01:33:14.654 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:33:14 smithi083 ceph-mon[17120]: from='client.? ' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-08-19T01:33:14.655 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:33:14 smithi083 ceph-mon[17120]: from='client.? ' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-08-19T01:33:14.809 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/bad8780e-5dc9-11ef-bccf-c7b262605968/mon.a/config 2024-08-19T01:33:14.952 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:33:14 smithi117 ceph-mon[22016]: from='client.? 172.21.15.83:0/2085287220' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-08-19T01:33:14.953 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:33:14 smithi117 ceph-mon[22016]: from='client.? ' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-08-19T01:33:14.953 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:33:14 smithi117 ceph-mon[22016]: from='client.? ' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-08-19T01:33:14.966 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:33:14 smithi167 ceph-mon[21875]: from='client.? 172.21.15.83:0/2085287220' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-08-19T01:33:14.966 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:33:14 smithi167 ceph-mon[21875]: from='client.? ' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-08-19T01:33:14.966 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:33:14 smithi167 ceph-mon[21875]: from='client.? ' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-08-19T01:33:15.354 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:15.352+0000 7f25a1600640 1 -- 172.21.15.83:0/2632822686 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f259c1076f0 msgr2=0x7f259c107ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:33:15.355 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:15.352+0000 7f25a1600640 1 --2- 172.21.15.83:0/2632822686 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f259c1076f0 0x7f259c107ad0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f258c009930 tx=0x7f258c02a420 comp rx=0 tx=0).stop 2024-08-19T01:33:15.355 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:15.352+0000 7f25a1600640 1 -- 172.21.15.83:0/2632822686 shutdown_connections 2024-08-19T01:33:15.355 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:15.352+0000 7f25a1600640 1 --2- 172.21.15.83:0/2632822686 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f259c102090 0x7f259c10e5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:33:15.355 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:15.352+0000 7f25a1600640 1 --2- 172.21.15.83:0/2632822686 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f259c1016f0 0x7f259c101b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:33:15.355 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:15.352+0000 7f25a1600640 1 --2- 172.21.15.83:0/2632822686 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f259c1076f0 0x7f259c107ad0 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:33:15.355 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:15.352+0000 7f25a1600640 1 -- 172.21.15.83:0/2632822686 >> 172.21.15.83:0/2632822686 conn(0x7f259c0fd470 msgr2=0x7f259c0ff890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:33:15.355 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:15.352+0000 7f25a1600640 1 -- 172.21.15.83:0/2632822686 shutdown_connections 2024-08-19T01:33:15.355 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:15.352+0000 7f25a1600640 1 -- 172.21.15.83:0/2632822686 wait complete. 2024-08-19T01:33:15.357 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:15.354+0000 7f25a1600640 1 Processor -- start 2024-08-19T01:33:15.358 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:15.354+0000 7f25a1600640 1 -- start start 2024-08-19T01:33:15.358 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:15.354+0000 7f25a1600640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f259c1016f0 0x7f259c197fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:33:15.358 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:15.354+0000 7f25a1600640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f259c102090 0x7f259c198500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:33:15.358 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:15.354+0000 7f25a1600640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f259c1076f0 0x7f259c19c890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:33:15.358 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:15.354+0000 7f25a1600640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f259c111b40 con 0x7f259c1016f0 2024-08-19T01:33:15.358 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:15.354+0000 7f25a1600640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f259c1119c0 con 0x7f259c1076f0 2024-08-19T01:33:15.358 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:15.354+0000 7f25a1600640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f259c111cc0 con 0x7f259c102090 2024-08-19T01:33:15.358 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:15.355+0000 7f259be00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f259c1016f0 0x7f259c197fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:33:15.359 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:15.355+0000 7f259be00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f259c1016f0 0x7f259c197fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:42088/0 (socket says 172.21.15.83:42088) 2024-08-19T01:33:15.359 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:15.355+0000 7f259be00640 1 -- 172.21.15.83:0/3344846952 learned_addr learned my addr 172.21.15.83:0/3344846952 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-08-19T01:33:15.359 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:15.355+0000 7f2593e00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f259c102090 0x7f259c198500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:33:15.359 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:15.355+0000 7f25a0c00640 1 --2- 172.21.15.83:0/3344846952 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f259c1076f0 0x7f259c19c890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:33:15.359 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:15.355+0000 7f259be00640 1 -- 172.21.15.83:0/3344846952 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f259c102090 msgr2=0x7f259c198500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:33:15.359 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:15.355+0000 7f259be00640 1 --2- 172.21.15.83:0/3344846952 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f259c102090 0x7f259c198500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:33:15.359 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:15.355+0000 7f259be00640 1 -- 172.21.15.83:0/3344846952 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f259c1076f0 msgr2=0x7f259c19c890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:33:15.360 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:15.355+0000 7f259be00640 1 --2- 172.21.15.83:0/3344846952 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f259c1076f0 0x7f259c19c890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:33:15.360 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:15.355+0000 7f259be00640 1 -- 172.21.15.83:0/3344846952 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f259c19cf70 con 0x7f259c1016f0 2024-08-19T01:33:15.360 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:15.355+0000 7f2593e00640 1 --2- 172.21.15.83:0/3344846952 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f259c102090 0x7f259c198500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-19T01:33:15.360 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:15.355+0000 7f259be00640 1 --2- 172.21.15.83:0/3344846952 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f259c1016f0 0x7f259c197fc0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f258c02a940 tx=0x7f258c003b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:33:15.361 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:15.356+0000 7f2599600640 1 -- 172.21.15.83:0/3344846952 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f258c03f070 con 0x7f259c1016f0 2024-08-19T01:33:15.361 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:15.356+0000 7f2599600640 1 -- 172.21.15.83:0/3344846952 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f258c031c90 con 0x7f259c1016f0 2024-08-19T01:33:15.361 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:15.356+0000 7f25a1600640 1 -- 172.21.15.83:0/3344846952 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f259c19d200 con 0x7f259c1016f0 2024-08-19T01:33:15.361 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:15.356+0000 7f2599600640 1 -- 172.21.15.83:0/3344846952 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f258c035030 con 0x7f259c1016f0 2024-08-19T01:33:15.361 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:15.356+0000 7f25a1600640 1 -- 172.21.15.83:0/3344846952 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f259c1a4b40 con 0x7f259c1016f0 2024-08-19T01:33:15.362 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:15.357+0000 7f25a1600640 1 -- 172.21.15.83:0/3344846952 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f259c102e30 con 0x7f259c1016f0 2024-08-19T01:33:15.363 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:15.359+0000 7f2599600640 1 -- 172.21.15.83:0/3344846952 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f258c042070 con 0x7f259c1016f0 2024-08-19T01:33:15.363 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:15.360+0000 7f2599600640 1 --2- 172.21.15.83:0/3344846952 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f25800a2c10 0x7f25800a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:33:15.363 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:15.360+0000 7f2599600640 1 -- 172.21.15.83:0/3344846952 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8269+0+0 (secure 0 0 0) 0x7f258c0ef850 con 0x7f259c1016f0 2024-08-19T01:33:15.363 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:15.360+0000 7f2593e00640 1 --2- 172.21.15.83:0/3344846952 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f25800a2c10 0x7f25800a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:33:15.363 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:15.361+0000 7f2593e00640 1 --2- 172.21.15.83:0/3344846952 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f25800a2c10 0x7f25800a50d0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f25840095f0 tx=0x7f2584005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:33:15.366 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:15.364+0000 7f2599600640 1 -- 172.21.15.83:0/3344846952 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7f258c0bbf80 con 0x7f259c1016f0 2024-08-19T01:33:15.598 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:15.596+0000 7f25a1600640 1 -- 172.21.15.83:0/3344846952 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) -- 0x7f259c109c60 con 0x7f259c1016f0 2024-08-19T01:33:15.603 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:15.601+0000 7f2599600640 1 -- 172.21.15.83:0/3344846952 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v17) ==== 74+0+263198 (secure 0 0 0) 0x7f258c0c0280 con 0x7f259c1016f0 2024-08-19T01:33:15.611 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-08-19T01:33:15.614 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:15.607+0000 7f25a1600640 1 -- 172.21.15.83:0/3344846952 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f25800a2c10 msgr2=0x7f25800a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:33:15.614 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:15.607+0000 7f25a1600640 1 --2- 172.21.15.83:0/3344846952 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f25800a2c10 0x7f25800a50d0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f25840095f0 tx=0x7f2584005f90 comp rx=0 tx=0).stop 2024-08-19T01:33:15.614 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:15.607+0000 7f25a1600640 1 -- 172.21.15.83:0/3344846952 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f259c1016f0 msgr2=0x7f259c197fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:33:15.614 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:15.607+0000 7f25a1600640 1 --2- 172.21.15.83:0/3344846952 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f259c1016f0 0x7f259c197fc0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f258c02a940 tx=0x7f258c003b20 comp rx=0 tx=0).stop 2024-08-19T01:33:15.614 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:15.607+0000 7f25a1600640 1 -- 172.21.15.83:0/3344846952 shutdown_connections 2024-08-19T01:33:15.615 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:15.607+0000 7f25a1600640 1 --2- 172.21.15.83:0/3344846952 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f25800a2c10 0x7f25800a50d0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:33:15.615 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:15.607+0000 7f25a1600640 1 --2- 172.21.15.83:0/3344846952 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f259c1076f0 0x7f259c19c890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:33:15.615 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:15.607+0000 7f25a1600640 1 --2- 172.21.15.83:0/3344846952 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f259c102090 0x7f259c198500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:33:15.615 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:15.607+0000 7f25a1600640 1 --2- 172.21.15.83:0/3344846952 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f259c1016f0 0x7f259c197fc0 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:33:15.616 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:15.607+0000 7f25a1600640 1 -- 172.21.15.83:0/3344846952 >> 172.21.15.83:0/3344846952 conn(0x7f259c0fd470 msgr2=0x7f259c0fdc80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:33:15.616 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:15.607+0000 7f25a1600640 1 -- 172.21.15.83:0/3344846952 shutdown_connections 2024-08-19T01:33:15.616 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:15.607+0000 7f25a1600640 1 -- 172.21.15.83:0/3344846952 wait complete. 2024-08-19T01:33:15.854 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:33:15 smithi083 ceph-mon[17120]: pgmap v291: 97 pgs: 97 active+clean; 630 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 3.2 KiB/s wr, 8 op/s; 102 B/s, 1 keys/s, 0 objects/s recovering 2024-08-19T01:33:15.952 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:33:15 smithi117 ceph-mon[22016]: pgmap v291: 97 pgs: 97 active+clean; 630 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 3.2 KiB/s wr, 8 op/s; 102 B/s, 1 keys/s, 0 objects/s recovering 2024-08-19T01:33:15.966 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:33:15 smithi167 ceph-mon[21875]: pgmap v291: 97 pgs: 97 active+clean; 630 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 3.2 KiB/s wr, 8 op/s; 102 B/s, 1 keys/s, 0 objects/s recovering 2024-08-19T01:33:16.492 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":17,"flags":0,"active_gid":14150,"active_name":"x","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6800","nonce":683740916},{"type":"v1","addr":"172.21.15.83:6801","nonce":683740916}]},"active_addr":"172.21.15.83:6801/683740916","active_change":"2024-08-19T01:26:04.277823+0000","active_mgr_features":4540701547738038271,"available":true,"standbys":[{"gid":24109,"name":"z","mgr_features":4540701547738038271,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.5","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]},{"gid":24122,"name":"y","mgr_features":4540701547738038271,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.5","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["cephadm","dashboard","iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.5","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{"dashboard":"https://172.21.15.83:8443/"},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"squid":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":3,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.83:0","nonce":2373054747}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.83:0","nonce":3822765168}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.83:0","nonce":2728865953}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.83:0","nonce":3847157033}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.83:0","nonce":1864709888}]}]} 2024-08-19T01:33:16.500 INFO:tasks.cephadm.ceph_manager.ceph:mgr available! 2024-08-19T01:33:16.500 INFO:tasks.cephadm.ceph_manager.ceph:waiting for all up 2024-08-19T01:33:16.500 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833 shell --fsid bad8780e-5dc9-11ef-bccf-c7b262605968 -- ceph osd dump --format=json 2024-08-19T01:33:16.715 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:33:16 smithi083 ceph-mon[17120]: from='client.? 172.21.15.83:0/3344846952' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-08-19T01:33:16.886 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/bad8780e-5dc9-11ef-bccf-c7b262605968/mon.a/config 2024-08-19T01:33:16.952 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:33:16 smithi117 ceph-mon[22016]: from='client.? 172.21.15.83:0/3344846952' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-08-19T01:33:16.966 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:33:16 smithi167 ceph-mon[21875]: from='client.? 172.21.15.83:0/3344846952' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-08-19T01:33:17.477 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:17.476+0000 7f4b97a00640 1 -- 172.21.15.83:0/3175162537 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4b90076450 msgr2=0x7f4b90076830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:33:17.477 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:17.476+0000 7f4b97a00640 1 --2- 172.21.15.83:0/3175162537 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4b90076450 0x7f4b90076830 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f4b80009900 tx=0x7f4b8002a3f0 comp rx=0 tx=0).stop 2024-08-19T01:33:17.477 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:17.476+0000 7f4b97a00640 1 -- 172.21.15.83:0/3175162537 shutdown_connections 2024-08-19T01:33:17.477 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:17.476+0000 7f4b97a00640 1 --2- 172.21.15.83:0/3175162537 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4b90112780 0x7f4b90114b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:33:17.477 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:17.476+0000 7f4b97a00640 1 --2- 172.21.15.83:0/3175162537 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4b90076d70 0x7f4b90112240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:33:17.477 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:17.476+0000 7f4b97a00640 1 --2- 172.21.15.83:0/3175162537 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4b90076450 0x7f4b90076830 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:33:17.477 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:17.476+0000 7f4b97a00640 1 -- 172.21.15.83:0/3175162537 >> 172.21.15.83:0/3175162537 conn(0x7f4b900ff780 msgr2=0x7f4b90101ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:33:17.477 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:17.476+0000 7f4b97a00640 1 -- 172.21.15.83:0/3175162537 shutdown_connections 2024-08-19T01:33:17.478 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:17.476+0000 7f4b97a00640 1 -- 172.21.15.83:0/3175162537 wait complete. 2024-08-19T01:33:17.479 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:17.477+0000 7f4b97a00640 1 Processor -- start 2024-08-19T01:33:17.479 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:17.477+0000 7f4b97a00640 1 -- start start 2024-08-19T01:33:17.479 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:17.477+0000 7f4b97a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4b90076450 0x7f4b901a4640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:33:17.479 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:17.477+0000 7f4b97a00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4b90076d70 0x7f4b901a4b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:33:17.479 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:17.477+0000 7f4b97a00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4b90112780 0x7f4b9019e710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:33:17.479 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:17.477+0000 7f4b97a00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f4b90118360 con 0x7f4b90076450 2024-08-19T01:33:17.479 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:17.477+0000 7f4b97a00640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f4b901181e0 con 0x7f4b90076d70 2024-08-19T01:33:17.479 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:17.477+0000 7f4b97a00640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f4b901184e0 con 0x7f4b90112780 2024-08-19T01:33:17.480 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:17.478+0000 7f4b95000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4b90076450 0x7f4b901a4640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:33:17.480 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:17.478+0000 7f4b87e00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4b90076d70 0x7f4b901a4b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:33:17.480 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:17.478+0000 7f4b95000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4b90076450 0x7f4b901a4640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:42112/0 (socket says 172.21.15.83:42112) 2024-08-19T01:33:17.480 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:17.478+0000 7f4b95000640 1 -- 172.21.15.83:0/505583985 learned_addr learned my addr 172.21.15.83:0/505583985 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-08-19T01:33:17.480 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:17.478+0000 7f4b95a00640 1 --2- 172.21.15.83:0/505583985 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4b90112780 0x7f4b9019e710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:33:17.480 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:17.478+0000 7f4b87e00640 1 -- 172.21.15.83:0/505583985 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4b90112780 msgr2=0x7f4b9019e710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:33:17.480 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:17.478+0000 7f4b87e00640 1 --2- 172.21.15.83:0/505583985 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4b90112780 0x7f4b9019e710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:33:17.480 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:17.478+0000 7f4b87e00640 1 -- 172.21.15.83:0/505583985 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4b90076450 msgr2=0x7f4b901a4640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:33:17.480 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:17.478+0000 7f4b87e00640 1 --2- 172.21.15.83:0/505583985 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4b90076450 0x7f4b901a4640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:33:17.480 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:17.478+0000 7f4b87e00640 1 -- 172.21.15.83:0/505583985 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4b9019ef80 con 0x7f4b90076d70 2024-08-19T01:33:17.480 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:17.478+0000 7f4b87e00640 1 --2- 172.21.15.83:0/505583985 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4b90076d70 0x7f4b901a4b80 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f4b7800e9f0 tx=0x7f4b7800eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:33:17.480 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:17.479+0000 7f4b8e000640 1 -- 172.21.15.83:0/505583985 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b7800ccc0 con 0x7f4b90076d70 2024-08-19T01:33:17.480 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:17.479+0000 7f4b8e000640 1 -- 172.21.15.83:0/505583985 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4b78005020 con 0x7f4b90076d70 2024-08-19T01:33:17.481 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:17.479+0000 7f4b97a00640 1 -- 172.21.15.83:0/505583985 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4b9019f270 con 0x7f4b90076d70 2024-08-19T01:33:17.481 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:17.479+0000 7f4b8e000640 1 -- 172.21.15.83:0/505583985 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b780049e0 con 0x7f4b90076d70 2024-08-19T01:33:17.481 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:17.479+0000 7f4b97a00640 1 -- 172.21.15.83:0/505583985 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4b901ab5c0 con 0x7f4b90076d70 2024-08-19T01:33:17.481 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:17.480+0000 7f4b97a00640 1 -- 172.21.15.83:0/505583985 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4b90077b90 con 0x7f4b90076d70 2024-08-19T01:33:17.482 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:17.482+0000 7f4b8e000640 1 -- 172.21.15.83:0/505583985 <== mon.2 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f4b78005560 con 0x7f4b90076d70 2024-08-19T01:33:17.483 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:17.483+0000 7f4b8e000640 1 --2- 172.21.15.83:0/505583985 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f4b740a2c90 0x7f4b740a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:33:17.484 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:17.483+0000 7f4b8e000640 1 -- 172.21.15.83:0/505583985 <== mon.2 v2:172.21.15.117:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8269+0+0 (secure 0 0 0) 0x7f4b780cf1f0 con 0x7f4b90076d70 2024-08-19T01:33:17.484 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:17.483+0000 7f4b95000640 1 --2- 172.21.15.83:0/505583985 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f4b740a2c90 0x7f4b740a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:33:17.484 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:17.484+0000 7f4b95000640 1 --2- 172.21.15.83:0/505583985 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f4b740a2c90 0x7f4b740a5150 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f4b800096c0 tx=0x7f4b800046c0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:33:17.485 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:17.485+0000 7f4b8e000640 1 -- 172.21.15.83:0/505583985 <== mon.2 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7f4b780051d0 con 0x7f4b90076d70 2024-08-19T01:33:17.654 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:33:17 smithi083 ceph-mon[17120]: pgmap v292: 97 pgs: 97 active+clean; 638 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 2.0 KiB/s wr, 7 op/s; 92 B/s, 0 keys/s, 0 objects/s recovering 2024-08-19T01:33:17.670 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:17.670+0000 7f4b97a00640 1 -- 172.21.15.83:0/505583985 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f4b9019fc40 con 0x7f4b90076d70 2024-08-19T01:33:17.672 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:17.672+0000 7f4b8e000640 1 -- 172.21.15.83:0/505583985 <== mon.2 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v86) ==== 74+0+22722 (secure 0 0 0) 0x7f4b7809c000 con 0x7f4b90076d70 2024-08-19T01:33:17.673 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-08-19T01:33:17.673 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":86,"fsid":"bad8780e-5dc9-11ef-bccf-c7b262605968","created":"2024-08-19T01:25:12.626841+0000","modified":"2024-08-19T01:33:05.461998+0000","last_up_change":"2024-08-19T01:31:47.254559+0000","last_in_change":"2024-08-19T01:31:27.652847+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-08-19T01:28:16.862007+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-08-19T01:32:37.455535+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-08-19T01:32:40.201935+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ffa74041-f9d2-4555-bb43-508b8c581487","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6802","nonce":1082526824},{"type":"v1","addr":"172.21.15.83:6803","nonce":1082526824}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6804","nonce":1082526824},{"type":"v1","addr":"172.21.15.83:6805","nonce":1082526824}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6808","nonce":1082526824},{"type":"v1","addr":"172.21.15.83:6809","nonce":1082526824}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6806","nonce":1082526824},{"type":"v1","addr":"172.21.15.83:6807","nonce":1082526824}]},"public_addr":"172.21.15.83:6803/1082526824","cluster_addr":"172.21.15.83:6805/1082526824","heartbeat_back_addr":"172.21.15.83:6809/1082526824","heartbeat_front_addr":"172.21.15.83:6807/1082526824","state":["exists","up"]},{"osd":1,"uuid":"1977570b-b4f5-44fd-bb1e-3a31318be649","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6800","nonce":1925419802},{"type":"v1","addr":"172.21.15.117:6801","nonce":1925419802}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6802","nonce":1925419802},{"type":"v1","addr":"172.21.15.117:6803","nonce":1925419802}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6806","nonce":1925419802},{"type":"v1","addr":"172.21.15.117:6807","nonce":1925419802}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6804","nonce":1925419802},{"type":"v1","addr":"172.21.15.117:6805","nonce":1925419802}]},"public_addr":"172.21.15.117:6801/1925419802","cluster_addr":"172.21.15.117:6803/1925419802","heartbeat_back_addr":"172.21.15.117:6807/1925419802","heartbeat_front_addr":"172.21.15.117:6805/1925419802","state":["exists","up"]},{"osd":2,"uuid":"a75278d1-161f-41d5-a92f-058d1077bf2a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6800","nonce":551625328},{"type":"v1","addr":"172.21.15.167:6801","nonce":551625328}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6802","nonce":551625328},{"type":"v1","addr":"172.21.15.167:6803","nonce":551625328}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6806","nonce":551625328},{"type":"v1","addr":"172.21.15.167:6807","nonce":551625328}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6804","nonce":551625328},{"type":"v1","addr":"172.21.15.167:6805","nonce":551625328}]},"public_addr":"172.21.15.167:6801/551625328","cluster_addr":"172.21.15.167:6803/551625328","heartbeat_back_addr":"172.21.15.167:6807/551625328","heartbeat_front_addr":"172.21.15.167:6805/551625328","state":["exists","up"]},{"osd":3,"uuid":"d1fb16f9-910e-46fa-b1e3-8e5bef4d346d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6810","nonce":3138942062},{"type":"v1","addr":"172.21.15.83:6811","nonce":3138942062}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6812","nonce":3138942062},{"type":"v1","addr":"172.21.15.83:6813","nonce":3138942062}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6816","nonce":3138942062},{"type":"v1","addr":"172.21.15.83:6817","nonce":3138942062}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6814","nonce":3138942062},{"type":"v1","addr":"172.21.15.83:6815","nonce":3138942062}]},"public_addr":"172.21.15.83:6811/3138942062","cluster_addr":"172.21.15.83:6813/3138942062","heartbeat_back_addr":"172.21.15.83:6817/3138942062","heartbeat_front_addr":"172.21.15.83:6815/3138942062","state":["exists","up"]},{"osd":4,"uuid":"8176ed18-1fe2-446d-8994-8c60f6e08f38","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6808","nonce":3978863827},{"type":"v1","addr":"172.21.15.117:6809","nonce":3978863827}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6810","nonce":3978863827},{"type":"v1","addr":"172.21.15.117:6811","nonce":3978863827}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6814","nonce":3978863827},{"type":"v1","addr":"172.21.15.117:6815","nonce":3978863827}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6812","nonce":3978863827},{"type":"v1","addr":"172.21.15.117:6813","nonce":3978863827}]},"public_addr":"172.21.15.117:6809/3978863827","cluster_addr":"172.21.15.117:6811/3978863827","heartbeat_back_addr":"172.21.15.117:6815/3978863827","heartbeat_front_addr":"172.21.15.117:6813/3978863827","state":["exists","up"]},{"osd":5,"uuid":"e01e3f15-25c8-4d2c-be6f-a170f036064c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6808","nonce":2761858035},{"type":"v1","addr":"172.21.15.167:6809","nonce":2761858035}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6810","nonce":2761858035},{"type":"v1","addr":"172.21.15.167:6811","nonce":2761858035}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6814","nonce":2761858035},{"type":"v1","addr":"172.21.15.167:6815","nonce":2761858035}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6812","nonce":2761858035},{"type":"v1","addr":"172.21.15.167:6813","nonce":2761858035}]},"public_addr":"172.21.15.167:6809/2761858035","cluster_addr":"172.21.15.167:6811/2761858035","heartbeat_back_addr":"172.21.15.167:6815/2761858035","heartbeat_front_addr":"172.21.15.167:6813/2761858035","state":["exists","up"]},{"osd":6,"uuid":"78cf0a9b-9165-444f-b073-ebe13016de9d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6818","nonce":260644552},{"type":"v1","addr":"172.21.15.83:6819","nonce":260644552}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6820","nonce":260644552},{"type":"v1","addr":"172.21.15.83:6821","nonce":260644552}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6824","nonce":260644552},{"type":"v1","addr":"172.21.15.83:6825","nonce":260644552}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6822","nonce":260644552},{"type":"v1","addr":"172.21.15.83:6823","nonce":260644552}]},"public_addr":"172.21.15.83:6819/260644552","cluster_addr":"172.21.15.83:6821/260644552","heartbeat_back_addr":"172.21.15.83:6825/260644552","heartbeat_front_addr":"172.21.15.83:6823/260644552","state":["exists","up"]},{"osd":7,"uuid":"ee524f7f-672e-430e-9f58-1a9117513466","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6816","nonce":104113252},{"type":"v1","addr":"172.21.15.117:6817","nonce":104113252}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6818","nonce":104113252},{"type":"v1","addr":"172.21.15.117:6819","nonce":104113252}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6822","nonce":104113252},{"type":"v1","addr":"172.21.15.117:6823","nonce":104113252}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6820","nonce":104113252},{"type":"v1","addr":"172.21.15.117:6821","nonce":104113252}]},"public_addr":"172.21.15.117:6817/104113252","cluster_addr":"172.21.15.117:6819/104113252","heartbeat_back_addr":"172.21.15.117:6823/104113252","heartbeat_front_addr":"172.21.15.117:6821/104113252","state":["exists","up"]},{"osd":8,"uuid":"76d5317d-4845-4138-b210-e7bd2d80472a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":58,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6816","nonce":2196898857},{"type":"v1","addr":"172.21.15.167:6817","nonce":2196898857}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6818","nonce":2196898857},{"type":"v1","addr":"172.21.15.167:6819","nonce":2196898857}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6822","nonce":2196898857},{"type":"v1","addr":"172.21.15.167:6823","nonce":2196898857}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6820","nonce":2196898857},{"type":"v1","addr":"172.21.15.167:6821","nonce":2196898857}]},"public_addr":"172.21.15.167:6817/2196898857","cluster_addr":"172.21.15.167:6819/2196898857","heartbeat_back_addr":"172.21.15.167:6823/2196898857","heartbeat_front_addr":"172.21.15.167:6821/2196898857","state":["exists","up"]},{"osd":9,"uuid":"0151bc2a-3ad2-4db3-8505-5855792cc9bb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6826","nonce":903463450},{"type":"v1","addr":"172.21.15.83:6827","nonce":903463450}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6828","nonce":903463450},{"type":"v1","addr":"172.21.15.83:6829","nonce":903463450}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6832","nonce":903463450},{"type":"v1","addr":"172.21.15.83:6833","nonce":903463450}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6830","nonce":903463450},{"type":"v1","addr":"172.21.15.83:6831","nonce":903463450}]},"public_addr":"172.21.15.83:6827/903463450","cluster_addr":"172.21.15.83:6829/903463450","heartbeat_back_addr":"172.21.15.83:6833/903463450","heartbeat_front_addr":"172.21.15.83:6831/903463450","state":["exists","up"]},{"osd":10,"uuid":"437ed848-0da4-4cd4-8716-0c55bb99b676","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6824","nonce":1183570837},{"type":"v1","addr":"172.21.15.117:6825","nonce":1183570837}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6826","nonce":1183570837},{"type":"v1","addr":"172.21.15.117:6827","nonce":1183570837}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6830","nonce":1183570837},{"type":"v1","addr":"172.21.15.117:6831","nonce":1183570837}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6828","nonce":1183570837},{"type":"v1","addr":"172.21.15.117:6829","nonce":1183570837}]},"public_addr":"172.21.15.117:6825/1183570837","cluster_addr":"172.21.15.117:6827/1183570837","heartbeat_back_addr":"172.21.15.117:6831/1183570837","heartbeat_front_addr":"172.21.15.117:6829/1183570837","state":["exists","up"]},{"osd":11,"uuid":"13345f05-cdc0-4dae-a58b-551abbec1915","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":78,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6824","nonce":187027350},{"type":"v1","addr":"172.21.15.167:6825","nonce":187027350}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6826","nonce":187027350},{"type":"v1","addr":"172.21.15.167:6827","nonce":187027350}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6830","nonce":187027350},{"type":"v1","addr":"172.21.15.167:6831","nonce":187027350}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6828","nonce":187027350},{"type":"v1","addr":"172.21.15.167:6829","nonce":187027350}]},"public_addr":"172.21.15.167:6825/187027350","cluster_addr":"172.21.15.167:6827/187027350","heartbeat_back_addr":"172.21.15.167:6831/187027350","heartbeat_front_addr":"172.21.15.167:6829/187027350","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-19T01:27:28.760517+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-19T01:27:50.493028+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-19T01:28:11.997674+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-19T01:28:38.109934+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-19T01:29:00.173065+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-19T01:29:21.336085+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-19T01:29:47.871498+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-19T01:30:10.720498+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-19T01:30:32.425438+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-19T01:30:59.524989+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-19T01:31:22.806162+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-19T01:31:45.043246+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.1a","mappings":[{"from":8,"to":1}]},{"pgid":"2.1b","mappings":[{"from":2,"to":4}]},{"pgid":"2.1d","mappings":[{"from":8,"to":11}]},{"pgid":"3.6","mappings":[{"from":4,"to":6}]},{"pgid":"3.11","mappings":[{"from":4,"to":8}]},{"pgid":"3.1b","mappings":[{"from":5,"to":9}]},{"pgid":"3.20","mappings":[{"from":5,"to":9}]},{"pgid":"3.31","mappings":[{"from":4,"to":7}]},{"pgid":"3.3b","mappings":[{"from":3,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.83:0/3138095466":"2024-08-20T01:26:04.277716+0000","172.21.15.83:0/3835005198":"2024-08-20T01:26:04.277716+0000","172.21.15.83:0/1790374088":"2024-08-20T01:25:37.228583+0000","172.21.15.83:6801/573907319":"2024-08-20T01:26:04.277716+0000","172.21.15.83:6800/573907319":"2024-08-20T01:26:04.277716+0000","172.21.15.83:0/664676002":"2024-08-20T01:25:37.228583+0000","172.21.15.83:0/2144611449":"2024-08-20T01:25:37.228583+0000","172.21.15.83:6801/3261152818":"2024-08-20T01:25:37.228583+0000","172.21.15.83:0/1609552666":"2024-08-20T01:26:04.277716+0000","172.21.15.83:6800/3261152818":"2024-08-20T01:25:37.228583+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-08-19T01:33:17.675 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:17.674+0000 7f4b97a00640 1 -- 172.21.15.83:0/505583985 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f4b740a2c90 msgr2=0x7f4b740a5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:33:17.675 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:17.674+0000 7f4b97a00640 1 --2- 172.21.15.83:0/505583985 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f4b740a2c90 0x7f4b740a5150 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f4b800096c0 tx=0x7f4b800046c0 comp rx=0 tx=0).stop 2024-08-19T01:33:17.676 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:17.674+0000 7f4b97a00640 1 -- 172.21.15.83:0/505583985 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4b90076d70 msgr2=0x7f4b901a4b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:33:17.676 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:17.674+0000 7f4b97a00640 1 --2- 172.21.15.83:0/505583985 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4b90076d70 0x7f4b901a4b80 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f4b7800e9f0 tx=0x7f4b7800eeb0 comp rx=0 tx=0).stop 2024-08-19T01:33:17.676 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:17.674+0000 7f4b97a00640 1 -- 172.21.15.83:0/505583985 shutdown_connections 2024-08-19T01:33:17.676 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:17.675+0000 7f4b97a00640 1 --2- 172.21.15.83:0/505583985 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f4b740a2c90 0x7f4b740a5150 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:33:17.676 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:17.675+0000 7f4b97a00640 1 --2- 172.21.15.83:0/505583985 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4b90112780 0x7f4b9019e710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:33:17.676 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:17.675+0000 7f4b97a00640 1 --2- 172.21.15.83:0/505583985 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4b90076d70 0x7f4b901a4b80 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:33:17.676 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:17.675+0000 7f4b97a00640 1 --2- 172.21.15.83:0/505583985 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4b90076450 0x7f4b901a4640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:33:17.676 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:17.675+0000 7f4b97a00640 1 -- 172.21.15.83:0/505583985 >> 172.21.15.83:0/505583985 conn(0x7f4b900ff780 msgr2=0x7f4b90101b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:33:17.676 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:17.675+0000 7f4b97a00640 1 -- 172.21.15.83:0/505583985 shutdown_connections 2024-08-19T01:33:17.676 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:17.675+0000 7f4b97a00640 1 -- 172.21.15.83:0/505583985 wait complete. 2024-08-19T01:33:17.854 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:33:17 smithi117 ceph-mon[22016]: pgmap v292: 97 pgs: 97 active+clean; 638 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 2.0 KiB/s wr, 7 op/s; 92 B/s, 0 keys/s, 0 objects/s recovering 2024-08-19T01:33:17.867 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:33:17 smithi167 ceph-mon[21875]: pgmap v292: 97 pgs: 97 active+clean; 638 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 2.0 KiB/s wr, 7 op/s; 92 B/s, 0 keys/s, 0 objects/s recovering 2024-08-19T01:33:18.399 INFO:tasks.cephadm.ceph_manager.ceph:all up! 2024-08-19T01:33:18.399 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833 shell --fsid bad8780e-5dc9-11ef-bccf-c7b262605968 -- ceph osd dump --format=json 2024-08-19T01:33:18.654 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:33:18 smithi083 ceph-mon[17120]: from='client.? 172.21.15.83:0/505583985' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-08-19T01:33:18.779 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/bad8780e-5dc9-11ef-bccf-c7b262605968/mon.a/config 2024-08-19T01:33:18.952 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:33:18 smithi117 ceph-mon[22016]: from='client.? 172.21.15.83:0/505583985' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-08-19T01:33:18.966 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:33:18 smithi167 ceph-mon[21875]: from='client.? 172.21.15.83:0/505583985' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-08-19T01:33:19.351 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:19.350+0000 7f5ed1000640 1 -- 172.21.15.83:0/1885248176 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5ecc109900 msgr2=0x7f5ecc109ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:33:19.351 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:19.350+0000 7f5ed1000640 1 --2- 172.21.15.83:0/1885248176 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5ecc109900 0x7f5ecc109ce0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f5ebc009900 tx=0x7f5ebc02a410 comp rx=0 tx=0).stop 2024-08-19T01:33:19.351 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:19.350+0000 7f5ed1000640 1 -- 172.21.15.83:0/1885248176 shutdown_connections 2024-08-19T01:33:19.352 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:19.350+0000 7f5ed1000640 1 --2- 172.21.15.83:0/1885248176 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5ecc1042a0 0x7f5ecc1107d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:33:19.352 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:19.350+0000 7f5ed1000640 1 --2- 172.21.15.83:0/1885248176 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5ecc103900 0x7f5ecc103d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:33:19.352 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:19.350+0000 7f5ed1000640 1 --2- 172.21.15.83:0/1885248176 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5ecc109900 0x7f5ecc109ce0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:33:19.352 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:19.350+0000 7f5ed1000640 1 -- 172.21.15.83:0/1885248176 >> 172.21.15.83:0/1885248176 conn(0x7f5ecc0ff600 msgr2=0x7f5ecc101a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:33:19.352 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:19.350+0000 7f5ed1000640 1 -- 172.21.15.83:0/1885248176 shutdown_connections 2024-08-19T01:33:19.352 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:19.350+0000 7f5ed1000640 1 -- 172.21.15.83:0/1885248176 wait complete. 2024-08-19T01:33:19.354 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:19.351+0000 7f5ed1000640 1 Processor -- start 2024-08-19T01:33:19.354 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:19.352+0000 7f5ed1000640 1 -- start start 2024-08-19T01:33:19.354 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:19.352+0000 7f5ed1000640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5ecc103900 0x7f5ecc19a1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:33:19.354 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:19.352+0000 7f5ed1000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5ecc1042a0 0x7f5ecc19a700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:33:19.355 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:19.352+0000 7f5ed1000640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5ecc109900 0x7f5ecc19ea90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:33:19.355 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:19.352+0000 7f5ed1000640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f5ecc113d00 con 0x7f5ecc1042a0 2024-08-19T01:33:19.355 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:19.352+0000 7f5ed1000640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f5ecc113b80 con 0x7f5ecc109900 2024-08-19T01:33:19.355 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:19.352+0000 7f5ed1000640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f5ecc113e80 con 0x7f5ecc103900 2024-08-19T01:33:19.355 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:19.352+0000 7f5ecb400640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5ecc103900 0x7f5ecc19a1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:33:19.355 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:19.352+0000 7f5ecb400640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5ecc103900 0x7f5ecc19a1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.83:39216/0 (socket says 172.21.15.83:39216) 2024-08-19T01:33:19.355 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:19.352+0000 7f5ecb400640 1 -- 172.21.15.83:0/4102653211 learned_addr learned my addr 172.21.15.83:0/4102653211 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-08-19T01:33:19.355 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:19.352+0000 7f5ecaa00640 1 --2- 172.21.15.83:0/4102653211 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5ecc1042a0 0x7f5ecc19a700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:33:19.355 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:19.353+0000 7f5ecbe00640 1 --2- 172.21.15.83:0/4102653211 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5ecc109900 0x7f5ecc19ea90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:33:19.355 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:19.353+0000 7f5ecbe00640 1 -- 172.21.15.83:0/4102653211 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5ecc103900 msgr2=0x7f5ecc19a1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:33:19.355 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:19.353+0000 7f5ecbe00640 1 --2- 172.21.15.83:0/4102653211 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5ecc103900 0x7f5ecc19a1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:33:19.358 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:19.353+0000 7f5ecbe00640 1 -- 172.21.15.83:0/4102653211 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5ecc1042a0 msgr2=0x7f5ecc19a700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:33:19.359 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:19.353+0000 7f5ecbe00640 1 --2- 172.21.15.83:0/4102653211 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5ecc1042a0 0x7f5ecc19a700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:33:19.359 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:19.353+0000 7f5ecbe00640 1 -- 172.21.15.83:0/4102653211 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5ecc19f170 con 0x7f5ecc109900 2024-08-19T01:33:19.359 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:19.353+0000 7f5ecb400640 1 --2- 172.21.15.83:0/4102653211 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5ecc103900 0x7f5ecc19a1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-19T01:33:19.359 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:19.353+0000 7f5ecbe00640 1 --2- 172.21.15.83:0/4102653211 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5ecc109900 0x7f5ecc19ea90 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f5ec4009fd0 tx=0x7f5ec400ef10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:33:19.359 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:19.353+0000 7f5ec3e00640 1 -- 172.21.15.83:0/4102653211 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ec400ce20 con 0x7f5ecc109900 2024-08-19T01:33:19.359 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:19.353+0000 7f5ec3e00640 1 -- 172.21.15.83:0/4102653211 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5ec4005020 con 0x7f5ecc109900 2024-08-19T01:33:19.359 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:19.353+0000 7f5ed1000640 1 -- 172.21.15.83:0/4102653211 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5ecc19f460 con 0x7f5ecc109900 2024-08-19T01:33:19.359 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:19.353+0000 7f5ec3e00640 1 -- 172.21.15.83:0/4102653211 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ec4010720 con 0x7f5ecc109900 2024-08-19T01:33:19.359 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:19.354+0000 7f5ed1000640 1 -- 172.21.15.83:0/4102653211 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5ecc1a6d40 con 0x7f5ecc109900 2024-08-19T01:33:19.361 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:19.355+0000 7f5ed1000640 1 -- 172.21.15.83:0/4102653211 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5ecc103d60 con 0x7f5ecc109900 2024-08-19T01:33:19.361 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:19.356+0000 7f5ec3e00640 1 -- 172.21.15.83:0/4102653211 <== mon.2 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f5ec4005540 con 0x7f5ecc109900 2024-08-19T01:33:19.361 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:19.358+0000 7f5ec3e00640 1 --2- 172.21.15.83:0/4102653211 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f5eb00a2c10 0x7f5eb00a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:33:19.361 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:19.358+0000 7f5ec3e00640 1 -- 172.21.15.83:0/4102653211 <== mon.2 v2:172.21.15.117:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8269+0+0 (secure 0 0 0) 0x7f5ec40cf7e0 con 0x7f5ecc109900 2024-08-19T01:33:19.361 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:19.358+0000 7f5ecb400640 1 --2- 172.21.15.83:0/4102653211 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f5eb00a2c10 0x7f5eb00a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:33:19.361 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:19.359+0000 7f5ecb400640 1 --2- 172.21.15.83:0/4102653211 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f5eb00a2c10 0x7f5eb00a50d0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f5ebc0096c0 tx=0x7f5ebc005a20 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:33:19.362 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:19.362+0000 7f5ec3e00640 1 -- 172.21.15.83:0/4102653211 <== mon.2 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7f5ec409c560 con 0x7f5ecc109900 2024-08-19T01:33:19.544 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:19.543+0000 7f5ed1000640 1 -- 172.21.15.83:0/4102653211 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f5ecc10b8e0 con 0x7f5ecc109900 2024-08-19T01:33:19.546 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:19.545+0000 7f5ec3e00640 1 -- 172.21.15.83:0/4102653211 <== mon.2 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v86) ==== 74+0+22722 (secure 0 0 0) 0x7f5ec409be60 con 0x7f5ecc109900 2024-08-19T01:33:19.547 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-08-19T01:33:19.548 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":86,"fsid":"bad8780e-5dc9-11ef-bccf-c7b262605968","created":"2024-08-19T01:25:12.626841+0000","modified":"2024-08-19T01:33:05.461998+0000","last_up_change":"2024-08-19T01:31:47.254559+0000","last_in_change":"2024-08-19T01:31:27.652847+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-08-19T01:28:16.862007+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-08-19T01:32:37.455535+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-08-19T01:32:40.201935+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ffa74041-f9d2-4555-bb43-508b8c581487","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6802","nonce":1082526824},{"type":"v1","addr":"172.21.15.83:6803","nonce":1082526824}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6804","nonce":1082526824},{"type":"v1","addr":"172.21.15.83:6805","nonce":1082526824}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6808","nonce":1082526824},{"type":"v1","addr":"172.21.15.83:6809","nonce":1082526824}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6806","nonce":1082526824},{"type":"v1","addr":"172.21.15.83:6807","nonce":1082526824}]},"public_addr":"172.21.15.83:6803/1082526824","cluster_addr":"172.21.15.83:6805/1082526824","heartbeat_back_addr":"172.21.15.83:6809/1082526824","heartbeat_front_addr":"172.21.15.83:6807/1082526824","state":["exists","up"]},{"osd":1,"uuid":"1977570b-b4f5-44fd-bb1e-3a31318be649","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6800","nonce":1925419802},{"type":"v1","addr":"172.21.15.117:6801","nonce":1925419802}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6802","nonce":1925419802},{"type":"v1","addr":"172.21.15.117:6803","nonce":1925419802}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6806","nonce":1925419802},{"type":"v1","addr":"172.21.15.117:6807","nonce":1925419802}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6804","nonce":1925419802},{"type":"v1","addr":"172.21.15.117:6805","nonce":1925419802}]},"public_addr":"172.21.15.117:6801/1925419802","cluster_addr":"172.21.15.117:6803/1925419802","heartbeat_back_addr":"172.21.15.117:6807/1925419802","heartbeat_front_addr":"172.21.15.117:6805/1925419802","state":["exists","up"]},{"osd":2,"uuid":"a75278d1-161f-41d5-a92f-058d1077bf2a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6800","nonce":551625328},{"type":"v1","addr":"172.21.15.167:6801","nonce":551625328}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6802","nonce":551625328},{"type":"v1","addr":"172.21.15.167:6803","nonce":551625328}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6806","nonce":551625328},{"type":"v1","addr":"172.21.15.167:6807","nonce":551625328}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6804","nonce":551625328},{"type":"v1","addr":"172.21.15.167:6805","nonce":551625328}]},"public_addr":"172.21.15.167:6801/551625328","cluster_addr":"172.21.15.167:6803/551625328","heartbeat_back_addr":"172.21.15.167:6807/551625328","heartbeat_front_addr":"172.21.15.167:6805/551625328","state":["exists","up"]},{"osd":3,"uuid":"d1fb16f9-910e-46fa-b1e3-8e5bef4d346d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6810","nonce":3138942062},{"type":"v1","addr":"172.21.15.83:6811","nonce":3138942062}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6812","nonce":3138942062},{"type":"v1","addr":"172.21.15.83:6813","nonce":3138942062}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6816","nonce":3138942062},{"type":"v1","addr":"172.21.15.83:6817","nonce":3138942062}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6814","nonce":3138942062},{"type":"v1","addr":"172.21.15.83:6815","nonce":3138942062}]},"public_addr":"172.21.15.83:6811/3138942062","cluster_addr":"172.21.15.83:6813/3138942062","heartbeat_back_addr":"172.21.15.83:6817/3138942062","heartbeat_front_addr":"172.21.15.83:6815/3138942062","state":["exists","up"]},{"osd":4,"uuid":"8176ed18-1fe2-446d-8994-8c60f6e08f38","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6808","nonce":3978863827},{"type":"v1","addr":"172.21.15.117:6809","nonce":3978863827}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6810","nonce":3978863827},{"type":"v1","addr":"172.21.15.117:6811","nonce":3978863827}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6814","nonce":3978863827},{"type":"v1","addr":"172.21.15.117:6815","nonce":3978863827}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6812","nonce":3978863827},{"type":"v1","addr":"172.21.15.117:6813","nonce":3978863827}]},"public_addr":"172.21.15.117:6809/3978863827","cluster_addr":"172.21.15.117:6811/3978863827","heartbeat_back_addr":"172.21.15.117:6815/3978863827","heartbeat_front_addr":"172.21.15.117:6813/3978863827","state":["exists","up"]},{"osd":5,"uuid":"e01e3f15-25c8-4d2c-be6f-a170f036064c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6808","nonce":2761858035},{"type":"v1","addr":"172.21.15.167:6809","nonce":2761858035}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6810","nonce":2761858035},{"type":"v1","addr":"172.21.15.167:6811","nonce":2761858035}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6814","nonce":2761858035},{"type":"v1","addr":"172.21.15.167:6815","nonce":2761858035}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6812","nonce":2761858035},{"type":"v1","addr":"172.21.15.167:6813","nonce":2761858035}]},"public_addr":"172.21.15.167:6809/2761858035","cluster_addr":"172.21.15.167:6811/2761858035","heartbeat_back_addr":"172.21.15.167:6815/2761858035","heartbeat_front_addr":"172.21.15.167:6813/2761858035","state":["exists","up"]},{"osd":6,"uuid":"78cf0a9b-9165-444f-b073-ebe13016de9d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6818","nonce":260644552},{"type":"v1","addr":"172.21.15.83:6819","nonce":260644552}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6820","nonce":260644552},{"type":"v1","addr":"172.21.15.83:6821","nonce":260644552}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6824","nonce":260644552},{"type":"v1","addr":"172.21.15.83:6825","nonce":260644552}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6822","nonce":260644552},{"type":"v1","addr":"172.21.15.83:6823","nonce":260644552}]},"public_addr":"172.21.15.83:6819/260644552","cluster_addr":"172.21.15.83:6821/260644552","heartbeat_back_addr":"172.21.15.83:6825/260644552","heartbeat_front_addr":"172.21.15.83:6823/260644552","state":["exists","up"]},{"osd":7,"uuid":"ee524f7f-672e-430e-9f58-1a9117513466","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6816","nonce":104113252},{"type":"v1","addr":"172.21.15.117:6817","nonce":104113252}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6818","nonce":104113252},{"type":"v1","addr":"172.21.15.117:6819","nonce":104113252}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6822","nonce":104113252},{"type":"v1","addr":"172.21.15.117:6823","nonce":104113252}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6820","nonce":104113252},{"type":"v1","addr":"172.21.15.117:6821","nonce":104113252}]},"public_addr":"172.21.15.117:6817/104113252","cluster_addr":"172.21.15.117:6819/104113252","heartbeat_back_addr":"172.21.15.117:6823/104113252","heartbeat_front_addr":"172.21.15.117:6821/104113252","state":["exists","up"]},{"osd":8,"uuid":"76d5317d-4845-4138-b210-e7bd2d80472a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":58,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6816","nonce":2196898857},{"type":"v1","addr":"172.21.15.167:6817","nonce":2196898857}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6818","nonce":2196898857},{"type":"v1","addr":"172.21.15.167:6819","nonce":2196898857}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6822","nonce":2196898857},{"type":"v1","addr":"172.21.15.167:6823","nonce":2196898857}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6820","nonce":2196898857},{"type":"v1","addr":"172.21.15.167:6821","nonce":2196898857}]},"public_addr":"172.21.15.167:6817/2196898857","cluster_addr":"172.21.15.167:6819/2196898857","heartbeat_back_addr":"172.21.15.167:6823/2196898857","heartbeat_front_addr":"172.21.15.167:6821/2196898857","state":["exists","up"]},{"osd":9,"uuid":"0151bc2a-3ad2-4db3-8505-5855792cc9bb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6826","nonce":903463450},{"type":"v1","addr":"172.21.15.83:6827","nonce":903463450}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6828","nonce":903463450},{"type":"v1","addr":"172.21.15.83:6829","nonce":903463450}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6832","nonce":903463450},{"type":"v1","addr":"172.21.15.83:6833","nonce":903463450}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6830","nonce":903463450},{"type":"v1","addr":"172.21.15.83:6831","nonce":903463450}]},"public_addr":"172.21.15.83:6827/903463450","cluster_addr":"172.21.15.83:6829/903463450","heartbeat_back_addr":"172.21.15.83:6833/903463450","heartbeat_front_addr":"172.21.15.83:6831/903463450","state":["exists","up"]},{"osd":10,"uuid":"437ed848-0da4-4cd4-8716-0c55bb99b676","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6824","nonce":1183570837},{"type":"v1","addr":"172.21.15.117:6825","nonce":1183570837}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6826","nonce":1183570837},{"type":"v1","addr":"172.21.15.117:6827","nonce":1183570837}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6830","nonce":1183570837},{"type":"v1","addr":"172.21.15.117:6831","nonce":1183570837}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6828","nonce":1183570837},{"type":"v1","addr":"172.21.15.117:6829","nonce":1183570837}]},"public_addr":"172.21.15.117:6825/1183570837","cluster_addr":"172.21.15.117:6827/1183570837","heartbeat_back_addr":"172.21.15.117:6831/1183570837","heartbeat_front_addr":"172.21.15.117:6829/1183570837","state":["exists","up"]},{"osd":11,"uuid":"13345f05-cdc0-4dae-a58b-551abbec1915","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":78,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6824","nonce":187027350},{"type":"v1","addr":"172.21.15.167:6825","nonce":187027350}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6826","nonce":187027350},{"type":"v1","addr":"172.21.15.167:6827","nonce":187027350}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6830","nonce":187027350},{"type":"v1","addr":"172.21.15.167:6831","nonce":187027350}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6828","nonce":187027350},{"type":"v1","addr":"172.21.15.167:6829","nonce":187027350}]},"public_addr":"172.21.15.167:6825/187027350","cluster_addr":"172.21.15.167:6827/187027350","heartbeat_back_addr":"172.21.15.167:6831/187027350","heartbeat_front_addr":"172.21.15.167:6829/187027350","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-19T01:27:28.760517+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-19T01:27:50.493028+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-19T01:28:11.997674+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-19T01:28:38.109934+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-19T01:29:00.173065+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-19T01:29:21.336085+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-19T01:29:47.871498+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-19T01:30:10.720498+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-19T01:30:32.425438+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-19T01:30:59.524989+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-19T01:31:22.806162+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-19T01:31:45.043246+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.1a","mappings":[{"from":8,"to":1}]},{"pgid":"2.1b","mappings":[{"from":2,"to":4}]},{"pgid":"2.1d","mappings":[{"from":8,"to":11}]},{"pgid":"3.6","mappings":[{"from":4,"to":6}]},{"pgid":"3.11","mappings":[{"from":4,"to":8}]},{"pgid":"3.1b","mappings":[{"from":5,"to":9}]},{"pgid":"3.20","mappings":[{"from":5,"to":9}]},{"pgid":"3.31","mappings":[{"from":4,"to":7}]},{"pgid":"3.3b","mappings":[{"from":3,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.83:0/3138095466":"2024-08-20T01:26:04.277716+0000","172.21.15.83:0/3835005198":"2024-08-20T01:26:04.277716+0000","172.21.15.83:0/1790374088":"2024-08-20T01:25:37.228583+0000","172.21.15.83:6801/573907319":"2024-08-20T01:26:04.277716+0000","172.21.15.83:6800/573907319":"2024-08-20T01:26:04.277716+0000","172.21.15.83:0/664676002":"2024-08-20T01:25:37.228583+0000","172.21.15.83:0/2144611449":"2024-08-20T01:25:37.228583+0000","172.21.15.83:6801/3261152818":"2024-08-20T01:25:37.228583+0000","172.21.15.83:0/1609552666":"2024-08-20T01:26:04.277716+0000","172.21.15.83:6800/3261152818":"2024-08-20T01:25:37.228583+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-08-19T01:33:19.549 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:19.547+0000 7f5ed1000640 1 -- 172.21.15.83:0/4102653211 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f5eb00a2c10 msgr2=0x7f5eb00a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:33:19.550 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:19.547+0000 7f5ed1000640 1 --2- 172.21.15.83:0/4102653211 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f5eb00a2c10 0x7f5eb00a50d0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f5ebc0096c0 tx=0x7f5ebc005a20 comp rx=0 tx=0).stop 2024-08-19T01:33:19.550 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:19.547+0000 7f5ed1000640 1 -- 172.21.15.83:0/4102653211 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5ecc109900 msgr2=0x7f5ecc19ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:33:19.550 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:19.547+0000 7f5ed1000640 1 --2- 172.21.15.83:0/4102653211 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5ecc109900 0x7f5ecc19ea90 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f5ec4009fd0 tx=0x7f5ec400ef10 comp rx=0 tx=0).stop 2024-08-19T01:33:19.550 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:19.548+0000 7f5ed1000640 1 -- 172.21.15.83:0/4102653211 shutdown_connections 2024-08-19T01:33:19.550 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:19.548+0000 7f5ed1000640 1 --2- 172.21.15.83:0/4102653211 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f5eb00a2c10 0x7f5eb00a50d0 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:33:19.551 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:19.548+0000 7f5ed1000640 1 --2- 172.21.15.83:0/4102653211 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5ecc109900 0x7f5ecc19ea90 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:33:19.551 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:19.548+0000 7f5ed1000640 1 --2- 172.21.15.83:0/4102653211 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5ecc1042a0 0x7f5ecc19a700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:33:19.551 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:19.548+0000 7f5ed1000640 1 --2- 172.21.15.83:0/4102653211 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5ecc103900 0x7f5ecc19a1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:33:19.551 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:19.548+0000 7f5ed1000640 1 -- 172.21.15.83:0/4102653211 >> 172.21.15.83:0/4102653211 conn(0x7f5ecc0ff600 msgr2=0x7f5ecc0ffe90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:33:19.551 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:19.548+0000 7f5ed1000640 1 -- 172.21.15.83:0/4102653211 shutdown_connections 2024-08-19T01:33:19.551 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:19.548+0000 7f5ed1000640 1 -- 172.21.15.83:0/4102653211 wait complete. 2024-08-19T01:33:19.871 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:33:19 smithi083 ceph-mon[17120]: pgmap v293: 97 pgs: 97 active+clean; 638 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 1.8 KiB/s wr, 7 op/s; 84 B/s, 0 keys/s, 0 objects/s recovering 2024-08-19T01:33:19.952 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:33:19 smithi117 ceph-mon[22016]: pgmap v293: 97 pgs: 97 active+clean; 638 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 1.8 KiB/s wr, 7 op/s; 84 B/s, 0 keys/s, 0 objects/s recovering 2024-08-19T01:33:19.966 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:33:19 smithi167 ceph-mon[21875]: pgmap v293: 97 pgs: 97 active+clean; 638 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 1.8 KiB/s wr, 7 op/s; 84 B/s, 0 keys/s, 0 objects/s recovering 2024-08-19T01:33:20.303 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833 shell --fsid bad8780e-5dc9-11ef-bccf-c7b262605968 -- ceph tell osd.0 flush_pg_stats 2024-08-19T01:33:20.303 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833 shell --fsid bad8780e-5dc9-11ef-bccf-c7b262605968 -- ceph tell osd.1 flush_pg_stats 2024-08-19T01:33:20.304 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833 shell --fsid bad8780e-5dc9-11ef-bccf-c7b262605968 -- ceph tell osd.2 flush_pg_stats 2024-08-19T01:33:20.304 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833 shell --fsid bad8780e-5dc9-11ef-bccf-c7b262605968 -- ceph tell osd.3 flush_pg_stats 2024-08-19T01:33:20.305 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833 shell --fsid bad8780e-5dc9-11ef-bccf-c7b262605968 -- ceph tell osd.4 flush_pg_stats 2024-08-19T01:33:20.305 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833 shell --fsid bad8780e-5dc9-11ef-bccf-c7b262605968 -- ceph tell osd.5 flush_pg_stats 2024-08-19T01:33:20.305 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833 shell --fsid bad8780e-5dc9-11ef-bccf-c7b262605968 -- ceph tell osd.6 flush_pg_stats 2024-08-19T01:33:20.306 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833 shell --fsid bad8780e-5dc9-11ef-bccf-c7b262605968 -- ceph tell osd.7 flush_pg_stats 2024-08-19T01:33:20.306 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833 shell --fsid bad8780e-5dc9-11ef-bccf-c7b262605968 -- ceph tell osd.8 flush_pg_stats 2024-08-19T01:33:20.307 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833 shell --fsid bad8780e-5dc9-11ef-bccf-c7b262605968 -- ceph tell osd.9 flush_pg_stats 2024-08-19T01:33:20.307 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833 shell --fsid bad8780e-5dc9-11ef-bccf-c7b262605968 -- ceph tell osd.10 flush_pg_stats 2024-08-19T01:33:20.307 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833 shell --fsid bad8780e-5dc9-11ef-bccf-c7b262605968 -- ceph tell osd.11 flush_pg_stats 2024-08-19T01:33:20.906 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:33:20 smithi083 ceph-mon[17120]: from='client.? 172.21.15.83:0/4102653211' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-08-19T01:33:20.952 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:33:20 smithi117 ceph-mon[22016]: from='client.? 172.21.15.83:0/4102653211' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-08-19T01:33:20.966 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:33:20 smithi167 ceph-mon[21875]: from='client.? 172.21.15.83:0/4102653211' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-08-19T01:33:21.240 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/bad8780e-5dc9-11ef-bccf-c7b262605968/mon.a/config 2024-08-19T01:33:21.777 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:33:21 smithi083 ceph-mon[17120]: pgmap v294: 97 pgs: 97 active+clean; 640 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 1.9 KiB/s wr, 7 op/s; 0 B/s, 0 objects/s recovering 2024-08-19T01:33:21.889 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:21.888+0000 7f1f93e00640 1 -- 172.21.15.83:0/2724884343 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1f8c074350 msgr2=0x7f1f8c074730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:33:21.889 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:21.888+0000 7f1f93e00640 1 --2- 172.21.15.83:0/2724884343 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1f8c074350 0x7f1f8c074730 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f1f80009ca0 tx=0x7f1f8002a950 comp rx=0 tx=0).stop 2024-08-19T01:33:21.889 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:21.888+0000 7f1f93e00640 1 -- 172.21.15.83:0/2724884343 shutdown_connections 2024-08-19T01:33:21.889 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:21.888+0000 7f1f93e00640 1 --2- 172.21.15.83:0/2724884343 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1f8c10e0f0 0x7f1f8c110540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:33:21.890 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:21.888+0000 7f1f93e00640 1 --2- 172.21.15.83:0/2724884343 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1f8c074c70 0x7f1f8c10dbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:33:21.890 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:21.888+0000 7f1f93e00640 1 --2- 172.21.15.83:0/2724884343 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1f8c074350 0x7f1f8c074730 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:33:21.890 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:21.888+0000 7f1f93e00640 1 -- 172.21.15.83:0/2724884343 >> 172.21.15.83:0/2724884343 conn(0x7f1f8c0fd5f0 msgr2=0x7f1f8c0ffa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:33:21.890 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:21.888+0000 7f1f93e00640 1 -- 172.21.15.83:0/2724884343 shutdown_connections 2024-08-19T01:33:21.890 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:21.888+0000 7f1f93e00640 1 -- 172.21.15.83:0/2724884343 wait complete. 2024-08-19T01:33:21.891 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:21.889+0000 7f1f93e00640 1 Processor -- start 2024-08-19T01:33:21.892 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:21.889+0000 7f1f93e00640 1 -- start start 2024-08-19T01:33:21.892 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:21.890+0000 7f1f93e00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1f8c074350 0x7f1f8c19a3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:33:21.892 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:21.890+0000 7f1f93e00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1f8c074c70 0x7f1f8c19a910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:33:21.892 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:21.890+0000 7f1f93e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1f8c10e0f0 0x7f1f8c19eca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:33:21.892 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:21.890+0000 7f1f93e00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f1f8c113ef0 con 0x7f1f8c10e0f0 2024-08-19T01:33:21.892 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:21.890+0000 7f1f93e00640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f1f8c113d70 con 0x7f1f8c074350 2024-08-19T01:33:21.892 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:21.890+0000 7f1f93e00640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f1f8c114070 con 0x7f1f8c074c70 2024-08-19T01:33:21.892 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:21.890+0000 7f1f91e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1f8c10e0f0 0x7f1f8c19eca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:33:21.892 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:21.890+0000 7f1f91e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1f8c10e0f0 0x7f1f8c19eca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:60688/0 (socket says 172.21.15.83:60688) 2024-08-19T01:33:21.892 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:21.890+0000 7f1f91e00640 1 -- 172.21.15.83:0/3061250378 learned_addr learned my addr 172.21.15.83:0/3061250378 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-08-19T01:33:21.892 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:21.890+0000 7f1f91400640 1 --2- 172.21.15.83:0/3061250378 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1f8c074350 0x7f1f8c19a3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:33:21.893 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:21.890+0000 7f1f90a00640 1 --2- 172.21.15.83:0/3061250378 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1f8c074c70 0x7f1f8c19a910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:33:21.893 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:21.890+0000 7f1f91e00640 1 -- 172.21.15.83:0/3061250378 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1f8c074c70 msgr2=0x7f1f8c19a910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:33:21.893 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:21.890+0000 7f1f91e00640 1 --2- 172.21.15.83:0/3061250378 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1f8c074c70 0x7f1f8c19a910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:33:21.893 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:21.890+0000 7f1f91e00640 1 -- 172.21.15.83:0/3061250378 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1f8c074350 msgr2=0x7f1f8c19a3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:33:21.893 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:21.890+0000 7f1f91e00640 1 --2- 172.21.15.83:0/3061250378 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1f8c074350 0x7f1f8c19a3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:33:21.893 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:21.890+0000 7f1f91e00640 1 -- 172.21.15.83:0/3061250378 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1f8c19f380 con 0x7f1f8c10e0f0 2024-08-19T01:33:21.893 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:21.890+0000 7f1f91400640 1 --2- 172.21.15.83:0/3061250378 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1f8c074350 0x7f1f8c19a3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-19T01:33:21.893 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:21.891+0000 7f1f91e00640 1 --2- 172.21.15.83:0/3061250378 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1f8c10e0f0 0x7f1f8c19eca0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f1f8400e9b0 tx=0x7f1f8400ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:33:21.893 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:21.891+0000 7f1f8a000640 1 -- 172.21.15.83:0/3061250378 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f8400cd80 con 0x7f1f8c10e0f0 2024-08-19T01:33:21.893 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:21.891+0000 7f1f8a000640 1 -- 172.21.15.83:0/3061250378 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1f840035f0 con 0x7f1f8c10e0f0 2024-08-19T01:33:21.893 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:21.891+0000 7f1f93e00640 1 -- 172.21.15.83:0/3061250378 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1f8c19f670 con 0x7f1f8c10e0f0 2024-08-19T01:33:21.894 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:21.891+0000 7f1f8a000640 1 -- 172.21.15.83:0/3061250378 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f84010670 con 0x7f1f8c10e0f0 2024-08-19T01:33:21.894 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:21.891+0000 7f1f93e00640 1 -- 172.21.15.83:0/3061250378 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1f8c1a6f50 con 0x7f1f8c10e0f0 2024-08-19T01:33:21.894 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:21.892+0000 7f1f93e00640 1 -- 172.21.15.83:0/3061250378 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f1f8c062670 con 0x7f1f8c10e0f0 2024-08-19T01:33:21.895 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:21.894+0000 7f1f8a000640 1 -- 172.21.15.83:0/3061250378 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f1f840037a0 con 0x7f1f8c10e0f0 2024-08-19T01:33:21.896 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:21.895+0000 7f1f8a000640 1 --2- 172.21.15.83:0/3061250378 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f1f740a2c90 0x7f1f740a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:33:21.896 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:21.896+0000 7f1f8a000640 1 -- 172.21.15.83:0/3061250378 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8269+0+0 (secure 0 0 0) 0x7f1f840cf670 con 0x7f1f8c10e0f0 2024-08-19T01:33:21.896 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:21.896+0000 7f1f91400640 1 --2- 172.21.15.83:0/3061250378 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f1f740a2c90 0x7f1f740a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:33:21.897 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:21.896+0000 7f1f8a000640 1 --2- 172.21.15.83:0/3061250378 >> [v2:172.21.15.83:6818/260644552,v1:172.21.15.83:6819/260644552] conn(0x7f1f740ac940 0x7f1f740aede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:33:21.897 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:21.896+0000 7f1f8a000640 1 -- 172.21.15.83:0/3061250378 --> [v2:172.21.15.83:6818/260644552,v1:172.21.15.83:6819/260644552] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f1f84017e50 con 0x7f1f740ac940 2024-08-19T01:33:21.897 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:21.896+0000 7f1f8a000640 1 -- 172.21.15.83:0/3061250378 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_get_version_reply(handle=1 version=86) ==== 24+0+0 (secure 0 0 0) 0x7f1f840cfac0 con 0x7f1f8c10e0f0 2024-08-19T01:33:21.897 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:21.896+0000 7f1f90a00640 1 --2- 172.21.15.83:0/3061250378 >> [v2:172.21.15.83:6818/260644552,v1:172.21.15.83:6819/260644552] conn(0x7f1f740ac940 0x7f1f740aede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:33:21.898 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:21.896+0000 7f1f91400640 1 --2- 172.21.15.83:0/3061250378 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f1f740a2c90 0x7f1f740a5150 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f1f8002af40 tx=0x7f1f80005d80 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:33:21.898 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:21.897+0000 7f1f90a00640 1 --2- 172.21.15.83:0/3061250378 >> [v2:172.21.15.83:6818/260644552,v1:172.21.15.83:6819/260644552] conn(0x7f1f740ac940 0x7f1f740aede0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:33:21.898 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:21.898+0000 7f1f8a000640 1 -- 172.21.15.83:0/3061250378 <== osd.6 v2:172.21.15.83:6818/260644552 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f1f84017e50 con 0x7f1f740ac940 2024-08-19T01:33:21.916 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/bad8780e-5dc9-11ef-bccf-c7b262605968/mon.a/config 2024-08-19T01:33:21.916 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/bad8780e-5dc9-11ef-bccf-c7b262605968/mon.a/config 2024-08-19T01:33:21.921 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:21.918+0000 7f1f93e00640 1 -- 172.21.15.83:0/3061250378 --> [v2:172.21.15.83:6818/260644552,v1:172.21.15.83:6819/260644552] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f1f8c062af0 con 0x7f1f740ac940 2024-08-19T01:33:21.922 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:21.921+0000 7f1f8a000640 1 -- 172.21.15.83:0/3061250378 <== osd.6 v2:172.21.15.83:6818/260644552 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f1f8c062af0 con 0x7f1f740ac940 2024-08-19T01:33:21.922 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:21.921+0000 7f1f93e00640 1 -- 172.21.15.83:0/3061250378 >> [v2:172.21.15.83:6818/260644552,v1:172.21.15.83:6819/260644552] conn(0x7f1f740ac940 msgr2=0x7f1f740aede0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:33:21.922 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:21.921+0000 7f1f93e00640 1 --2- 172.21.15.83:0/3061250378 >> [v2:172.21.15.83:6818/260644552,v1:172.21.15.83:6819/260644552] conn(0x7f1f740ac940 0x7f1f740aede0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:33:21.923 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:21.922+0000 7f1f93e00640 1 -- 172.21.15.83:0/3061250378 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f1f740a2c90 msgr2=0x7f1f740a5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:33:21.923 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:21.922+0000 7f1f93e00640 1 --2- 172.21.15.83:0/3061250378 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f1f740a2c90 0x7f1f740a5150 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f1f8002af40 tx=0x7f1f80005d80 comp rx=0 tx=0).stop 2024-08-19T01:33:21.926 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:21.922+0000 7f1f93e00640 1 -- 172.21.15.83:0/3061250378 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1f8c10e0f0 msgr2=0x7f1f8c19eca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:33:21.926 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:21.922+0000 7f1f93e00640 1 --2- 172.21.15.83:0/3061250378 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1f8c10e0f0 0x7f1f8c19eca0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f1f8400e9b0 tx=0x7f1f8400ee70 comp rx=0 tx=0).stop 2024-08-19T01:33:21.930 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:21.922+0000 7f1f91e00640 1 -- 172.21.15.83:0/3061250378 reap_dead start 2024-08-19T01:33:21.930 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:21.923+0000 7f1f93e00640 1 -- 172.21.15.83:0/3061250378 shutdown_connections 2024-08-19T01:33:21.930 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:21.923+0000 7f1f93e00640 1 -- 172.21.15.83:0/3061250378 >> 172.21.15.83:0/3061250378 conn(0x7f1f8c0fd5f0 msgr2=0x7f1f8c0fe2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:33:21.931 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:21.923+0000 7f1f93e00640 1 -- 172.21.15.83:0/3061250378 shutdown_connections 2024-08-19T01:33:21.931 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:21.923+0000 7f1f93e00640 1 -- 172.21.15.83:0/3061250378 wait complete. 2024-08-19T01:33:21.937 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/bad8780e-5dc9-11ef-bccf-c7b262605968/mon.a/config 2024-08-19T01:33:21.937 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/bad8780e-5dc9-11ef-bccf-c7b262605968/mon.a/config 2024-08-19T01:33:21.950 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/bad8780e-5dc9-11ef-bccf-c7b262605968/mon.a/config 2024-08-19T01:33:21.952 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:33:21 smithi117 ceph-mon[22016]: pgmap v294: 97 pgs: 97 active+clean; 640 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 1.9 KiB/s wr, 7 op/s; 0 B/s, 0 objects/s recovering 2024-08-19T01:33:21.956 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/bad8780e-5dc9-11ef-bccf-c7b262605968/mon.a/config 2024-08-19T01:33:21.956 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/bad8780e-5dc9-11ef-bccf-c7b262605968/mon.a/config 2024-08-19T01:33:21.957 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/bad8780e-5dc9-11ef-bccf-c7b262605968/mon.a/config 2024-08-19T01:33:21.957 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/bad8780e-5dc9-11ef-bccf-c7b262605968/mon.a/config 2024-08-19T01:33:21.960 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/bad8780e-5dc9-11ef-bccf-c7b262605968/mon.a/config 2024-08-19T01:33:21.966 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:33:21 smithi167 ceph-mon[21875]: pgmap v294: 97 pgs: 97 active+clean; 640 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 1.9 KiB/s wr, 7 op/s; 0 B/s, 0 objects/s recovering 2024-08-19T01:33:22.141 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/bad8780e-5dc9-11ef-bccf-c7b262605968/mon.a/config 2024-08-19T01:33:22.878 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:33:22 smithi083 ceph-mon[17120]: pgmap v295: 97 pgs: 97 active+clean; 640 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 1023 B/s wr, 6 op/s 2024-08-19T01:33:22.952 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:33:22 smithi117 ceph-mon[22016]: pgmap v295: 97 pgs: 97 active+clean; 640 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 1023 B/s wr, 6 op/s 2024-08-19T01:33:22.966 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:33:22 smithi167 ceph-mon[21875]: pgmap v295: 97 pgs: 97 active+clean; 640 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 1023 B/s wr, 6 op/s 2024-08-19T01:33:23.696 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:23.695+0000 7fec9ea00640 1 -- 172.21.15.83:0/3001537962 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fec9810df40 msgr2=0x7fec98110390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:33:23.696 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:23.695+0000 7fec9ea00640 1 --2- 172.21.15.83:0/3001537962 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fec9810df40 0x7fec98110390 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fec90009ca0 tx=0x7fec9002a950 comp rx=0 tx=0).stop 2024-08-19T01:33:23.696 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:23.695+0000 7fec9ea00640 1 -- 172.21.15.83:0/3001537962 shutdown_connections 2024-08-19T01:33:23.696 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:23.695+0000 7fec9ea00640 1 --2- 172.21.15.83:0/3001537962 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fec9810df40 0x7fec98110390 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:33:23.696 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:23.695+0000 7fec9ea00640 1 --2- 172.21.15.83:0/3001537962 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fec98074c70 0x7fec9810da00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:33:23.696 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:23.695+0000 7fec9ea00640 1 --2- 172.21.15.83:0/3001537962 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fec98074350 0x7fec98074730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:33:23.696 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:23.695+0000 7fec9ea00640 1 -- 172.21.15.83:0/3001537962 >> 172.21.15.83:0/3001537962 conn(0x7fec980fd440 msgr2=0x7fec980ff860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:33:23.697 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:23.695+0000 7fec9ea00640 1 -- 172.21.15.83:0/3001537962 shutdown_connections 2024-08-19T01:33:23.697 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:23.695+0000 7fec9ea00640 1 -- 172.21.15.83:0/3001537962 wait complete. 2024-08-19T01:33:23.697 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:23.696+0000 7fec9ea00640 1 Processor -- start 2024-08-19T01:33:23.698 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:23.696+0000 7fec9ea00640 1 -- start start 2024-08-19T01:33:23.698 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:23.697+0000 7fec9ea00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fec98074350 0x7fec9819a1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:33:23.698 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:23.697+0000 7fec9ea00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fec98074c70 0x7fec9819a6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:33:23.698 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:23.697+0000 7fec9ea00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fec9810df40 0x7fec9819ea80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:33:23.698 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:23.697+0000 7fec9ea00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fec98113d40 con 0x7fec98074350 2024-08-19T01:33:23.698 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:23.697+0000 7fec9ea00640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fec98113bc0 con 0x7fec98074c70 2024-08-19T01:33:23.698 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:23.697+0000 7fec9ea00640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fec98113ec0 con 0x7fec9810df40 2024-08-19T01:33:23.698 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:23.697+0000 7fec9d600640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fec98074350 0x7fec9819a1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:33:23.698 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:23.697+0000 7fec9d600640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fec98074350 0x7fec9819a1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:60712/0 (socket says 172.21.15.83:60712) 2024-08-19T01:33:23.698 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:23.697+0000 7fec9d600640 1 -- 172.21.15.83:0/1172472079 learned_addr learned my addr 172.21.15.83:0/1172472079 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-08-19T01:33:23.698 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:23.697+0000 7fec9cc00640 1 --2- 172.21.15.83:0/1172472079 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fec98074c70 0x7fec9819a6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:33:23.698 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:23.697+0000 7fec9e000640 1 --2- 172.21.15.83:0/1172472079 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fec9810df40 0x7fec9819ea80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:33:23.699 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:23.697+0000 7fec9cc00640 1 -- 172.21.15.83:0/1172472079 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fec9810df40 msgr2=0x7fec9819ea80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:33:23.700 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:23.697+0000 7fec9cc00640 1 --2- 172.21.15.83:0/1172472079 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fec9810df40 0x7fec9819ea80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:33:23.700 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:23.697+0000 7fec9cc00640 1 -- 172.21.15.83:0/1172472079 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fec98074350 msgr2=0x7fec9819a1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:33:23.700 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:23.697+0000 7fec9cc00640 1 --2- 172.21.15.83:0/1172472079 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fec98074350 0x7fec9819a1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:33:23.700 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:23.697+0000 7fec9cc00640 1 -- 172.21.15.83:0/1172472079 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fec9819f160 con 0x7fec98074c70 2024-08-19T01:33:23.700 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:23.697+0000 7fec9d600640 1 --2- 172.21.15.83:0/1172472079 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fec98074350 0x7fec9819a1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-19T01:33:23.700 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:23.698+0000 7fec9cc00640 1 --2- 172.21.15.83:0/1172472079 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fec98074c70 0x7fec9819a6f0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fec8400d720 tx=0x7fec8400dbe0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:33:23.700 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:23.698+0000 7fec96000640 1 -- 172.21.15.83:0/1172472079 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec84017070 con 0x7fec98074c70 2024-08-19T01:33:23.700 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:23.698+0000 7fec96000640 1 -- 172.21.15.83:0/1172472079 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fec8400de30 con 0x7fec98074c70 2024-08-19T01:33:23.700 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:23.698+0000 7fec9ea00640 1 -- 172.21.15.83:0/1172472079 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fec9819f450 con 0x7fec98074c70 2024-08-19T01:33:23.700 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:23.698+0000 7fec96000640 1 -- 172.21.15.83:0/1172472079 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec840047a0 con 0x7fec98074c70 2024-08-19T01:33:23.700 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:23.698+0000 7fec9ea00640 1 -- 172.21.15.83:0/1172472079 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fec981a6d30 con 0x7fec98074c70 2024-08-19T01:33:23.700 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:23.699+0000 7fec9ea00640 1 -- 172.21.15.83:0/1172472079 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fec980626b0 con 0x7fec98074c70 2024-08-19T01:33:23.702 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:23.702+0000 7fec96000640 1 -- 172.21.15.83:0/1172472079 <== mon.2 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7fec840099d0 con 0x7fec98074c70 2024-08-19T01:33:23.704 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:23.703+0000 7fec96000640 1 --2- 172.21.15.83:0/1172472079 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7fec800a2c10 0x7fec800a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:33:23.704 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:23.703+0000 7fec96000640 1 -- 172.21.15.83:0/1172472079 <== mon.2 v2:172.21.15.117:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8269+0+0 (secure 0 0 0) 0x7fec840cf310 con 0x7fec98074c70 2024-08-19T01:33:23.704 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:23.703+0000 7fec9d600640 1 --2- 172.21.15.83:0/1172472079 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7fec800a2c10 0x7fec800a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:33:23.704 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:23.704+0000 7fec96000640 1 --2- 172.21.15.83:0/1172472079 >> [v2:172.21.15.83:6810/3138942062,v1:172.21.15.83:6811/3138942062] conn(0x7fec800ac8c0 0x7fec800aed60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:33:23.705 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:23.704+0000 7fec96000640 1 -- 172.21.15.83:0/1172472079 --> [v2:172.21.15.83:6810/3138942062,v1:172.21.15.83:6811/3138942062] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fec84015e80 con 0x7fec800ac8c0 2024-08-19T01:33:23.705 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:23.704+0000 7fec96000640 1 -- 172.21.15.83:0/1172472079 <== mon.2 v2:172.21.15.117:3300/0 6 ==== mon_get_version_reply(handle=1 version=86) ==== 24+0+0 (secure 0 0 0) 0x7fec840cf6c0 con 0x7fec98074c70 2024-08-19T01:33:23.705 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:23.704+0000 7fec9e000640 1 --2- 172.21.15.83:0/1172472079 >> [v2:172.21.15.83:6810/3138942062,v1:172.21.15.83:6811/3138942062] conn(0x7fec800ac8c0 0x7fec800aed60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:33:23.705 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:23.704+0000 7fec9d600640 1 --2- 172.21.15.83:0/1172472079 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7fec800a2c10 0x7fec800a50d0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fec8c011550 tx=0x7fec8c005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:33:23.706 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:23.704+0000 7fec9e000640 1 --2- 172.21.15.83:0/1172472079 >> [v2:172.21.15.83:6810/3138942062,v1:172.21.15.83:6811/3138942062] conn(0x7fec800ac8c0 0x7fec800aed60 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:33:23.706 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:23.705+0000 7fec96000640 1 -- 172.21.15.83:0/1172472079 <== osd.3 v2:172.21.15.83:6810/3138942062 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fec84015e80 con 0x7fec800ac8c0 2024-08-19T01:33:23.720 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:23.720+0000 7fec9ea00640 1 -- 172.21.15.83:0/1172472079 --> [v2:172.21.15.83:6810/3138942062,v1:172.21.15.83:6811/3138942062] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fec98062ab0 con 0x7fec800ac8c0 2024-08-19T01:33:23.721 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:23.720+0000 7fec96000640 1 -- 172.21.15.83:0/1172472079 <== osd.3 v2:172.21.15.83:6810/3138942062 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7fec98062ab0 con 0x7fec800ac8c0 2024-08-19T01:33:23.722 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:23.721+0000 7fec9ea00640 1 -- 172.21.15.83:0/1172472079 >> [v2:172.21.15.83:6810/3138942062,v1:172.21.15.83:6811/3138942062] conn(0x7fec800ac8c0 msgr2=0x7fec800aed60 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:33:23.722 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:23.721+0000 7fec9ea00640 1 --2- 172.21.15.83:0/1172472079 >> [v2:172.21.15.83:6810/3138942062,v1:172.21.15.83:6811/3138942062] conn(0x7fec800ac8c0 0x7fec800aed60 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:33:23.722 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:23.721+0000 7fec9ea00640 1 -- 172.21.15.83:0/1172472079 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7fec800a2c10 msgr2=0x7fec800a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:33:23.722 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:23.721+0000 7fec9ea00640 1 --2- 172.21.15.83:0/1172472079 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7fec800a2c10 0x7fec800a50d0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fec8c011550 tx=0x7fec8c005f90 comp rx=0 tx=0).stop 2024-08-19T01:33:23.722 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:23.721+0000 7fec9ea00640 1 -- 172.21.15.83:0/1172472079 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fec98074c70 msgr2=0x7fec9819a6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:33:23.722 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:23.721+0000 7fec9ea00640 1 --2- 172.21.15.83:0/1172472079 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fec98074c70 0x7fec9819a6f0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fec8400d720 tx=0x7fec8400dbe0 comp rx=0 tx=0).stop 2024-08-19T01:33:23.722 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:23.721+0000 7fec9e000640 1 -- 172.21.15.83:0/1172472079 reap_dead start 2024-08-19T01:33:23.722 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:23.721+0000 7fec9ea00640 1 -- 172.21.15.83:0/1172472079 shutdown_connections 2024-08-19T01:33:23.722 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:23.721+0000 7fec9ea00640 1 -- 172.21.15.83:0/1172472079 >> 172.21.15.83:0/1172472079 conn(0x7fec980fd440 msgr2=0x7fec980fe0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:33:23.722 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:23.721+0000 7fec9ea00640 1 -- 172.21.15.83:0/1172472079 shutdown_connections 2024-08-19T01:33:23.723 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:23.721+0000 7fec9ea00640 1 -- 172.21.15.83:0/1172472079 wait complete. 2024-08-19T01:33:24.586 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:24.243+0000 7f85fb200640 1 -- 172.21.15.83:0/3153426550 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f85f4102250 msgr2=0x7f85f410e780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:33:24.586 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:24.243+0000 7f85fb200640 1 --2- 172.21.15.83:0/3153426550 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f85f4102250 0x7f85f410e780 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f85ec009860 tx=0x7f85ec02a340 comp rx=0 tx=0).stop 2024-08-19T01:33:24.586 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:24.243+0000 7f85fb200640 1 -- 172.21.15.83:0/3153426550 shutdown_connections 2024-08-19T01:33:24.586 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:24.243+0000 7f85fb200640 1 --2- 172.21.15.83:0/3153426550 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f85f4102250 0x7f85f410e780 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:33:24.587 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:24.243+0000 7f85fb200640 1 --2- 172.21.15.83:0/3153426550 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f85f41018b0 0x7f85f4101d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:33:24.587 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:24.243+0000 7f85fb200640 1 --2- 172.21.15.83:0/3153426550 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f85f41078b0 0x7f85f4107c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:33:24.587 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:24.243+0000 7f85fb200640 1 -- 172.21.15.83:0/3153426550 >> 172.21.15.83:0/3153426550 conn(0x7f85f40fd5f0 msgr2=0x7f85f40ffa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:33:24.587 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:24.243+0000 7f85fb200640 1 -- 172.21.15.83:0/3153426550 shutdown_connections 2024-08-19T01:33:24.587 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:24.243+0000 7f85fb200640 1 -- 172.21.15.83:0/3153426550 wait complete. 2024-08-19T01:33:24.587 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:24.244+0000 7f85fb200640 1 Processor -- start 2024-08-19T01:33:24.587 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:24.244+0000 7f85fb200640 1 -- start start 2024-08-19T01:33:24.587 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:24.244+0000 7f85fb200640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f85f41018b0 0x7f85f4198110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:33:24.587 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:24.244+0000 7f85fb200640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f85f4102250 0x7f85f4198650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:33:24.587 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:24.244+0000 7f85fb200640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f85f41078b0 0x7f85f419c9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:33:24.587 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:24.244+0000 7f85fb200640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f85f4111cb0 con 0x7f85f4102250 2024-08-19T01:33:24.588 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:24.244+0000 7f85fb200640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f85f4111b30 con 0x7f85f41018b0 2024-08-19T01:33:24.588 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:24.244+0000 7f85fb200640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f85f4111e30 con 0x7f85f41078b0 2024-08-19T01:33:24.588 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:24.244+0000 7f85f3e00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f85f41018b0 0x7f85f4198110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:33:24.588 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:24.245+0000 7f85f3e00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f85f41018b0 0x7f85f4198110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.83:45784/0 (socket says 172.21.15.83:45784) 2024-08-19T01:33:24.588 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:24.245+0000 7f85f3e00640 1 -- 172.21.15.83:0/3860080650 learned_addr learned my addr 172.21.15.83:0/3860080650 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-08-19T01:33:24.588 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:24.245+0000 7f85eb400640 1 --2- 172.21.15.83:0/3860080650 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f85f4102250 0x7f85f4198650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:33:24.588 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:24.245+0000 7f85f9200640 1 --2- 172.21.15.83:0/3860080650 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f85f41078b0 0x7f85f419c9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:33:24.588 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:24.245+0000 7f85f3e00640 1 -- 172.21.15.83:0/3860080650 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f85f41078b0 msgr2=0x7f85f419c9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:33:24.588 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:24.245+0000 7f85f3e00640 1 --2- 172.21.15.83:0/3860080650 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f85f41078b0 0x7f85f419c9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:33:24.589 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:24.245+0000 7f85f3e00640 1 -- 172.21.15.83:0/3860080650 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f85f4102250 msgr2=0x7f85f4198650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:33:24.589 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:24.245+0000 7f85f3e00640 1 --2- 172.21.15.83:0/3860080650 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f85f4102250 0x7f85f4198650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:33:24.589 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:24.245+0000 7f85f3e00640 1 -- 172.21.15.83:0/3860080650 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f85f419d0c0 con 0x7f85f41018b0 2024-08-19T01:33:24.589 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:24.245+0000 7f85f3e00640 1 --2- 172.21.15.83:0/3860080650 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f85f41018b0 0x7f85f4198110 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f85e400db80 tx=0x7f85e400b630 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:33:24.589 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:24.246+0000 7f85f1600640 1 -- 172.21.15.83:0/3860080650 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85e4017070 con 0x7f85f41018b0 2024-08-19T01:33:24.589 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:24.246+0000 7f85f1600640 1 -- 172.21.15.83:0/3860080650 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f85e4003b70 con 0x7f85f41018b0 2024-08-19T01:33:24.589 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:24.246+0000 7f85f1600640 1 -- 172.21.15.83:0/3860080650 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85e40106c0 con 0x7f85f41018b0 2024-08-19T01:33:24.589 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:24.246+0000 7f85fb200640 1 -- 172.21.15.83:0/3860080650 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f85f419d3b0 con 0x7f85f41018b0 2024-08-19T01:33:24.589 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:24.246+0000 7f85fb200640 1 -- 172.21.15.83:0/3860080650 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f85f41a4c90 con 0x7f85f41018b0 2024-08-19T01:33:24.589 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:24.246+0000 7f85fb200640 1 -- 172.21.15.83:0/3860080650 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f85f40626b0 con 0x7f85f41018b0 2024-08-19T01:33:24.589 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:24.249+0000 7f85f1600640 1 -- 172.21.15.83:0/3860080650 <== mon.2 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f85e4004020 con 0x7f85f41018b0 2024-08-19T01:33:24.589 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:24.250+0000 7f85f1600640 1 --2- 172.21.15.83:0/3860080650 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f85d80a2c10 0x7f85d80a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:33:24.589 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:24.250+0000 7f85f1600640 1 -- 172.21.15.83:0/3860080650 <== mon.2 v2:172.21.15.117:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8269+0+0 (secure 0 0 0) 0x7f85e40cf5e0 con 0x7f85f41018b0 2024-08-19T01:33:24.589 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:24.250+0000 7f85eb400640 1 --2- 172.21.15.83:0/3860080650 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f85d80a2c10 0x7f85d80a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:33:24.590 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:24.250+0000 7f85f1600640 1 --2- 172.21.15.83:0/3860080650 >> [v2:172.21.15.117:6816/104113252,v1:172.21.15.117:6817/104113252] conn(0x7f85d80ac8c0 0x7f85d80aed60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:33:24.590 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:24.250+0000 7f85f1600640 1 -- 172.21.15.83:0/3860080650 --> [v2:172.21.15.117:6816/104113252,v1:172.21.15.117:6817/104113252] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f85e401fe50 con 0x7f85d80ac8c0 2024-08-19T01:33:24.590 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:24.250+0000 7f85f1600640 1 -- 172.21.15.83:0/3860080650 <== mon.2 v2:172.21.15.117:3300/0 6 ==== mon_get_version_reply(handle=1 version=86) ==== 24+0+0 (secure 0 0 0) 0x7f85e40cf990 con 0x7f85f41018b0 2024-08-19T01:33:24.590 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:24.251+0000 7f85f9200640 1 --2- 172.21.15.83:0/3860080650 >> [v2:172.21.15.117:6816/104113252,v1:172.21.15.117:6817/104113252] conn(0x7f85d80ac8c0 0x7f85d80aed60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:33:24.590 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:24.251+0000 7f85eb400640 1 --2- 172.21.15.83:0/3860080650 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f85d80a2c10 0x7f85d80a50d0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f85f40ffb90 tx=0x7f85dc009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:33:24.590 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:24.251+0000 7f85f9200640 1 --2- 172.21.15.83:0/3860080650 >> [v2:172.21.15.117:6816/104113252,v1:172.21.15.117:6817/104113252] conn(0x7f85d80ac8c0 0x7f85d80aed60 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:33:24.590 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:24.252+0000 7f85f1600640 1 -- 172.21.15.83:0/3860080650 <== osd.7 v2:172.21.15.117:6816/104113252 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f85e401fe50 con 0x7f85d80ac8c0 2024-08-19T01:33:24.590 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:24.266+0000 7f85fb200640 1 -- 172.21.15.83:0/3860080650 --> [v2:172.21.15.117:6816/104113252,v1:172.21.15.117:6817/104113252] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f85f4062b30 con 0x7f85d80ac8c0 2024-08-19T01:33:24.591 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:24.267+0000 7f85f1600640 1 -- 172.21.15.83:0/3860080650 <== osd.7 v2:172.21.15.117:6816/104113252 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f85f4062b30 con 0x7f85d80ac8c0 2024-08-19T01:33:24.591 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:24.267+0000 7f85fb200640 1 -- 172.21.15.83:0/3860080650 >> [v2:172.21.15.117:6816/104113252,v1:172.21.15.117:6817/104113252] conn(0x7f85d80ac8c0 msgr2=0x7f85d80aed60 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:33:24.591 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:24.267+0000 7f85fb200640 1 --2- 172.21.15.83:0/3860080650 >> [v2:172.21.15.117:6816/104113252,v1:172.21.15.117:6817/104113252] conn(0x7f85d80ac8c0 0x7f85d80aed60 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:33:24.591 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:24.267+0000 7f85fb200640 1 -- 172.21.15.83:0/3860080650 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f85d80a2c10 msgr2=0x7f85d80a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:33:24.591 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:24.267+0000 7f85fb200640 1 --2- 172.21.15.83:0/3860080650 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f85d80a2c10 0x7f85d80a50d0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f85f40ffb90 tx=0x7f85dc009260 comp rx=0 tx=0).stop 2024-08-19T01:33:24.591 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:24.267+0000 7f85fb200640 1 -- 172.21.15.83:0/3860080650 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f85f41018b0 msgr2=0x7f85f4198110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:33:24.591 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:24.267+0000 7f85fb200640 1 --2- 172.21.15.83:0/3860080650 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f85f41018b0 0x7f85f4198110 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f85e400db80 tx=0x7f85e400b630 comp rx=0 tx=0).stop 2024-08-19T01:33:24.591 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:24.267+0000 7f85f9200640 1 -- 172.21.15.83:0/3860080650 reap_dead start 2024-08-19T01:33:24.591 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:24.267+0000 7f85fb200640 1 -- 172.21.15.83:0/3860080650 shutdown_connections 2024-08-19T01:33:24.591 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:24.267+0000 7f85fb200640 1 -- 172.21.15.83:0/3860080650 >> 172.21.15.83:0/3860080650 conn(0x7f85f40fd5f0 msgr2=0x7f85f40fdd70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:33:24.591 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:24.267+0000 7f85fb200640 1 -- 172.21.15.83:0/3860080650 shutdown_connections 2024-08-19T01:33:24.591 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:24.268+0000 7f85fb200640 1 -- 172.21.15.83:0/3860080650 wait complete. 2024-08-19T01:33:25.059 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:25.057+0000 7f71b9c00640 1 -- 172.21.15.83:0/2198057092 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f71b41042a0 msgr2=0x7f71b41107d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:33:25.059 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:25.057+0000 7f71b9c00640 1 --2- 172.21.15.83:0/2198057092 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f71b41042a0 0x7f71b41107d0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f71ac009cd0 tx=0x7f71ac02a8f0 comp rx=0 tx=0).stop 2024-08-19T01:33:25.059 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:25.057+0000 7f71b9c00640 1 -- 172.21.15.83:0/2198057092 shutdown_connections 2024-08-19T01:33:25.059 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:25.057+0000 7f71b9c00640 1 --2- 172.21.15.83:0/2198057092 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f71b41042a0 0x7f71b41107d0 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:33:25.060 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:25.057+0000 7f71b9c00640 1 --2- 172.21.15.83:0/2198057092 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f71b4103900 0x7f71b4103d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:33:25.060 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:25.057+0000 7f71b9c00640 1 --2- 172.21.15.83:0/2198057092 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f71b4109900 0x7f71b4109ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:33:25.060 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:25.057+0000 7f71b9c00640 1 -- 172.21.15.83:0/2198057092 >> 172.21.15.83:0/2198057092 conn(0x7f71b40ff600 msgr2=0x7f71b4101a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:33:25.060 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:25.057+0000 7f71b9c00640 1 -- 172.21.15.83:0/2198057092 shutdown_connections 2024-08-19T01:33:25.060 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:25.057+0000 7f71b9c00640 1 -- 172.21.15.83:0/2198057092 wait complete. 2024-08-19T01:33:25.060 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:25.059+0000 7f71b9c00640 1 Processor -- start 2024-08-19T01:33:25.061 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:25.059+0000 7f71b9c00640 1 -- start start 2024-08-19T01:33:25.061 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:25.059+0000 7f71b9c00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f71b4103900 0x7f71b419a1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:33:25.061 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:25.059+0000 7f71b9c00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f71b41042a0 0x7f71b419a710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:33:25.061 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:25.059+0000 7f71b9c00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f71b4109900 0x7f71b419eaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:33:25.061 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:25.059+0000 7f71b9c00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f71b4113d00 con 0x7f71b4109900 2024-08-19T01:33:25.061 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:25.059+0000 7f71b9c00640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f71b4113b80 con 0x7f71b4103900 2024-08-19T01:33:25.061 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:25.059+0000 7f71b9c00640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f71b4113e80 con 0x7f71b41042a0 2024-08-19T01:33:25.062 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:25.059+0000 7f71b3e00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f71b4103900 0x7f71b419a1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:33:25.062 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:25.059+0000 7f71b9200640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f71b4109900 0x7f71b419eaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:33:25.063 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:25.059+0000 7f71b3e00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f71b4103900 0x7f71b419a1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.83:45816/0 (socket says 172.21.15.83:45816) 2024-08-19T01:33:25.063 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:25.059+0000 7f71b3e00640 1 -- 172.21.15.83:0/3850821829 learned_addr learned my addr 172.21.15.83:0/3850821829 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-08-19T01:33:25.063 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:25.059+0000 7f71b3400640 1 --2- 172.21.15.83:0/3850821829 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f71b41042a0 0x7f71b419a710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:33:25.063 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:25.060+0000 7f71b3e00640 1 -- 172.21.15.83:0/3850821829 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f71b41042a0 msgr2=0x7f71b419a710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:33:25.063 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:25.060+0000 7f71b3e00640 1 --2- 172.21.15.83:0/3850821829 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f71b41042a0 0x7f71b419a710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:33:25.063 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:25.060+0000 7f71b3e00640 1 -- 172.21.15.83:0/3850821829 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f71b4109900 msgr2=0x7f71b419eaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:33:25.063 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:25.060+0000 7f71b3e00640 1 --2- 172.21.15.83:0/3850821829 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f71b4109900 0x7f71b419eaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:33:25.063 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:25.060+0000 7f71b3e00640 1 -- 172.21.15.83:0/3850821829 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f71b419f180 con 0x7f71b4103900 2024-08-19T01:33:25.063 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:25.060+0000 7f71b9200640 1 --2- 172.21.15.83:0/3850821829 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f71b4109900 0x7f71b419eaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-19T01:33:25.063 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:25.060+0000 7f71b3e00640 1 --2- 172.21.15.83:0/3850821829 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f71b4103900 0x7f71b419a1d0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f71a400e9f0 tx=0x7f71a400edf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:33:25.063 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:25.060+0000 7f71b0c00640 1 -- 172.21.15.83:0/3850821829 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71a400cd80 con 0x7f71b4103900 2024-08-19T01:33:25.063 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:25.061+0000 7f71b0c00640 1 -- 172.21.15.83:0/3850821829 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f71a4005500 con 0x7f71b4103900 2024-08-19T01:33:25.063 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:25.061+0000 7f71b0c00640 1 -- 172.21.15.83:0/3850821829 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71a4010690 con 0x7f71b4103900 2024-08-19T01:33:25.064 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:25.061+0000 7f71b9c00640 1 -- 172.21.15.83:0/3850821829 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f71b419f470 con 0x7f71b4103900 2024-08-19T01:33:25.064 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:25.061+0000 7f71b9c00640 1 -- 172.21.15.83:0/3850821829 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f71b41a6d50 con 0x7f71b4103900 2024-08-19T01:33:25.064 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:25.062+0000 7f71b9c00640 1 -- 172.21.15.83:0/3850821829 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f71b40626b0 con 0x7f71b4103900 2024-08-19T01:33:25.065 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:25.064+0000 7f71b0c00640 1 -- 172.21.15.83:0/3850821829 <== mon.2 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f71a40046c0 con 0x7f71b4103900 2024-08-19T01:33:25.066 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:25.065+0000 7f71b0c00640 1 --2- 172.21.15.83:0/3850821829 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f71980a2c10 0x7f71980a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:33:25.067 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:25.065+0000 7f71b0c00640 1 -- 172.21.15.83:0/3850821829 <== mon.2 v2:172.21.15.117:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8269+0+0 (secure 0 0 0) 0x7f71a40cfa30 con 0x7f71b4103900 2024-08-19T01:33:25.067 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:25.065+0000 7f71b3400640 1 --2- 172.21.15.83:0/3850821829 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f71980a2c10 0x7f71980a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:33:25.067 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:25.065+0000 7f71b0c00640 1 --2- 172.21.15.83:0/3850821829 >> [v2:172.21.15.117:6808/3978863827,v1:172.21.15.117:6809/3978863827] conn(0x7f71980ac8c0 0x7f71980aed60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:33:25.067 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:25.065+0000 7f71b0c00640 1 -- 172.21.15.83:0/3850821829 --> [v2:172.21.15.117:6808/3978863827,v1:172.21.15.117:6809/3978863827] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f71a4004470 con 0x7f71980ac8c0 2024-08-19T01:33:25.067 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:25.065+0000 7f71b0c00640 1 -- 172.21.15.83:0/3850821829 <== mon.2 v2:172.21.15.117:3300/0 6 ==== mon_get_version_reply(handle=1 version=86) ==== 24+0+0 (secure 0 0 0) 0x7f71a40cfde0 con 0x7f71b4103900 2024-08-19T01:33:25.068 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:25.066+0000 7f71b9200640 1 --2- 172.21.15.83:0/3850821829 >> [v2:172.21.15.117:6808/3978863827,v1:172.21.15.117:6809/3978863827] conn(0x7f71980ac8c0 0x7f71980aed60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:33:25.068 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:25.066+0000 7f71b3400640 1 --2- 172.21.15.83:0/3850821829 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f71980a2c10 0x7f71980a50d0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f71b4101bf0 tx=0x7f71a0005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:33:25.068 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:25.066+0000 7f71b9200640 1 --2- 172.21.15.83:0/3850821829 >> [v2:172.21.15.117:6808/3978863827,v1:172.21.15.117:6809/3978863827] conn(0x7f71980ac8c0 0x7f71980aed60 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:33:25.069 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:25.067+0000 7f71b0c00640 1 -- 172.21.15.83:0/3850821829 <== osd.4 v2:172.21.15.117:6808/3978863827 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f71a4004470 con 0x7f71980ac8c0 2024-08-19T01:33:25.083 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:25.081+0000 7f71b9c00640 1 -- 172.21.15.83:0/3850821829 --> [v2:172.21.15.117:6808/3978863827,v1:172.21.15.117:6809/3978863827] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f71b4062b30 con 0x7f71980ac8c0 2024-08-19T01:33:25.084 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:25.082+0000 7f71b0c00640 1 -- 172.21.15.83:0/3850821829 <== osd.4 v2:172.21.15.117:6808/3978863827 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f71b4062b30 con 0x7f71980ac8c0 2024-08-19T01:33:25.085 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:25.082+0000 7f71b9c00640 1 -- 172.21.15.83:0/3850821829 >> [v2:172.21.15.117:6808/3978863827,v1:172.21.15.117:6809/3978863827] conn(0x7f71980ac8c0 msgr2=0x7f71980aed60 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:33:25.085 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:25.082+0000 7f71b9c00640 1 --2- 172.21.15.83:0/3850821829 >> [v2:172.21.15.117:6808/3978863827,v1:172.21.15.117:6809/3978863827] conn(0x7f71980ac8c0 0x7f71980aed60 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:33:25.085 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:25.083+0000 7f71b9c00640 1 -- 172.21.15.83:0/3850821829 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f71980a2c10 msgr2=0x7f71980a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:33:25.085 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:25.083+0000 7f71b9c00640 1 --2- 172.21.15.83:0/3850821829 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f71980a2c10 0x7f71980a50d0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f71b4101bf0 tx=0x7f71a0005f90 comp rx=0 tx=0).stop 2024-08-19T01:33:25.085 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:25.083+0000 7f71b9c00640 1 -- 172.21.15.83:0/3850821829 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f71b4103900 msgr2=0x7f71b419a1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:33:25.085 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:25.083+0000 7f71b9c00640 1 --2- 172.21.15.83:0/3850821829 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f71b4103900 0x7f71b419a1d0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f71a400e9f0 tx=0x7f71a400edf0 comp rx=0 tx=0).stop 2024-08-19T01:33:25.085 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:25.083+0000 7f71b9200640 1 -- 172.21.15.83:0/3850821829 reap_dead start 2024-08-19T01:33:25.085 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:25.083+0000 7f71b9c00640 1 -- 172.21.15.83:0/3850821829 shutdown_connections 2024-08-19T01:33:25.085 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:25.083+0000 7f71b9c00640 1 -- 172.21.15.83:0/3850821829 >> 172.21.15.83:0/3850821829 conn(0x7f71b40ff600 msgr2=0x7f71b40ffe90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:33:25.085 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:25.083+0000 7f71b9c00640 1 -- 172.21.15.83:0/3850821829 shutdown_connections 2024-08-19T01:33:25.085 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:25.083+0000 7f71b9c00640 1 -- 172.21.15.83:0/3850821829 wait complete. 2024-08-19T01:33:25.526 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:24.579+0000 7f508fe00640 1 -- 172.21.15.83:0/1240992230 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5088104480 msgr2=0x7f50881109b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:33:25.526 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:24.579+0000 7f508fe00640 1 --2- 172.21.15.83:0/1240992230 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5088104480 0x7f50881109b0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f5080009cd0 tx=0x7f508002a8f0 comp rx=0 tx=0).stop 2024-08-19T01:33:25.526 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:24.579+0000 7f508fe00640 1 -- 172.21.15.83:0/1240992230 shutdown_connections 2024-08-19T01:33:25.526 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:24.579+0000 7f508fe00640 1 --2- 172.21.15.83:0/1240992230 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5088104480 0x7f50881109b0 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:33:25.526 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:24.579+0000 7f508fe00640 1 --2- 172.21.15.83:0/1240992230 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5088103ae0 0x7f5088103f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:33:25.526 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:24.579+0000 7f508fe00640 1 --2- 172.21.15.83:0/1240992230 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5088109b70 0x7f5088109f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:33:25.526 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:24.579+0000 7f508fe00640 1 -- 172.21.15.83:0/1240992230 >> 172.21.15.83:0/1240992230 conn(0x7f50880ff780 msgr2=0x7f5088101ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:33:25.526 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:24.580+0000 7f508fe00640 1 -- 172.21.15.83:0/1240992230 shutdown_connections 2024-08-19T01:33:25.526 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:24.580+0000 7f508fe00640 1 -- 172.21.15.83:0/1240992230 wait complete. 2024-08-19T01:33:25.526 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:24.580+0000 7f508fe00640 1 Processor -- start 2024-08-19T01:33:25.527 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:24.581+0000 7f508fe00640 1 -- start start 2024-08-19T01:33:25.527 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:24.581+0000 7f508fe00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5088103ae0 0x7f508819a380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:33:25.527 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:24.581+0000 7f508fe00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5088104480 0x7f508819a8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:33:25.527 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:24.581+0000 7f508fe00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5088109b70 0x7f508819ec50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:33:25.527 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:24.581+0000 7f508fe00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f5088113ee0 con 0x7f5088109b70 2024-08-19T01:33:25.527 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:24.581+0000 7f508fe00640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f5088113d60 con 0x7f5088104480 2024-08-19T01:33:25.527 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:24.581+0000 7f508fe00640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f5088114060 con 0x7f5088103ae0 2024-08-19T01:33:25.527 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:24.581+0000 7f508de00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5088109b70 0x7f508819ec50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:33:25.527 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:24.581+0000 7f508de00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5088109b70 0x7f508819ec50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:60742/0 (socket says 172.21.15.83:60742) 2024-08-19T01:33:25.527 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:24.581+0000 7f508de00640 1 -- 172.21.15.83:0/552012498 learned_addr learned my addr 172.21.15.83:0/552012498 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-08-19T01:33:25.527 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:24.581+0000 7f508d400640 1 --2- 172.21.15.83:0/552012498 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5088103ae0 0x7f508819a380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:33:25.527 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:24.581+0000 7f508ca00640 1 --2- 172.21.15.83:0/552012498 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5088104480 0x7f508819a8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:33:25.527 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:24.582+0000 7f508de00640 1 -- 172.21.15.83:0/552012498 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5088103ae0 msgr2=0x7f508819a380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:33:25.527 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:24.582+0000 7f508de00640 1 --2- 172.21.15.83:0/552012498 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5088103ae0 0x7f508819a380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:33:25.528 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:24.582+0000 7f508de00640 1 -- 172.21.15.83:0/552012498 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5088104480 msgr2=0x7f508819a8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:33:25.528 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:24.582+0000 7f508de00640 1 --2- 172.21.15.83:0/552012498 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5088104480 0x7f508819a8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:33:25.528 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:24.582+0000 7f508de00640 1 -- 172.21.15.83:0/552012498 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f508819f330 con 0x7f5088109b70 2024-08-19T01:33:25.528 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:24.582+0000 7f508de00640 1 --2- 172.21.15.83:0/552012498 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5088109b70 0x7f508819ec50 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f5080009ca0 tx=0x7f5080004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:33:25.528 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:24.582+0000 7f5086000640 1 -- 172.21.15.83:0/552012498 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5080003d80 con 0x7f5088109b70 2024-08-19T01:33:25.528 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:24.582+0000 7f5086000640 1 -- 172.21.15.83:0/552012498 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5080040070 con 0x7f5088109b70 2024-08-19T01:33:25.528 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:24.582+0000 7f508fe00640 1 -- 172.21.15.83:0/552012498 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f508819f5c0 con 0x7f5088109b70 2024-08-19T01:33:25.528 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:24.582+0000 7f5086000640 1 -- 172.21.15.83:0/552012498 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5080045a10 con 0x7f5088109b70 2024-08-19T01:33:25.528 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:24.582+0000 7f508fe00640 1 -- 172.21.15.83:0/552012498 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f50881a6f00 con 0x7f5088109b70 2024-08-19T01:33:25.528 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:24.583+0000 7f508fe00640 1 -- 172.21.15.83:0/552012498 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f50880626b0 con 0x7f5088109b70 2024-08-19T01:33:25.528 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:24.585+0000 7f5086000640 1 -- 172.21.15.83:0/552012498 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f5080045bb0 con 0x7f5088109b70 2024-08-19T01:33:25.528 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:24.586+0000 7f5086000640 1 --2- 172.21.15.83:0/552012498 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f50700a2c10 0x7f50700a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:33:25.528 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:24.587+0000 7f5086000640 1 -- 172.21.15.83:0/552012498 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8269+0+0 (secure 0 0 0) 0x7f50800f04e0 con 0x7f5088109b70 2024-08-19T01:33:25.528 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:24.587+0000 7f508d400640 1 --2- 172.21.15.83:0/552012498 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f50700a2c10 0x7f50700a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:33:25.529 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:24.587+0000 7f5086000640 1 --2- 172.21.15.83:0/552012498 >> [v2:172.21.15.117:6824/1183570837,v1:172.21.15.117:6825/1183570837] conn(0x7f50700ac790 0x7f50700aec30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:33:25.529 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:24.587+0000 7f5086000640 1 -- 172.21.15.83:0/552012498 --> [v2:172.21.15.117:6824/1183570837,v1:172.21.15.117:6825/1183570837] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f508002fec0 con 0x7f50700ac790 2024-08-19T01:33:25.529 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:24.587+0000 7f5086000640 1 -- 172.21.15.83:0/552012498 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_get_version_reply(handle=1 version=86) ==== 24+0+0 (secure 0 0 0) 0x7f50800f0930 con 0x7f5088109b70 2024-08-19T01:33:25.529 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:24.587+0000 7f508ca00640 1 --2- 172.21.15.83:0/552012498 >> [v2:172.21.15.117:6824/1183570837,v1:172.21.15.117:6825/1183570837] conn(0x7f50700ac790 0x7f50700aec30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:33:25.529 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:24.587+0000 7f508d400640 1 --2- 172.21.15.83:0/552012498 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f50700a2c10 0x7f50700a50d0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f507c009b50 tx=0x7f507c005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:33:25.529 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:24.588+0000 7f508ca00640 1 --2- 172.21.15.83:0/552012498 >> [v2:172.21.15.117:6824/1183570837,v1:172.21.15.117:6825/1183570837] conn(0x7f50700ac790 0x7f50700aec30 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.10 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:33:25.530 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:24.589+0000 7f5086000640 1 -- 172.21.15.83:0/552012498 <== osd.10 v2:172.21.15.117:6824/1183570837 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f508002fec0 con 0x7f50700ac790 2024-08-19T01:33:25.530 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:24.603+0000 7f508fe00640 1 -- 172.21.15.83:0/552012498 --> [v2:172.21.15.117:6824/1183570837,v1:172.21.15.117:6825/1183570837] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f5088062b30 con 0x7f50700ac790 2024-08-19T01:33:25.530 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:24.604+0000 7f5086000640 1 -- 172.21.15.83:0/552012498 <== osd.10 v2:172.21.15.117:6824/1183570837 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f5088062b30 con 0x7f50700ac790 2024-08-19T01:33:25.530 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:24.604+0000 7f508fe00640 1 -- 172.21.15.83:0/552012498 >> [v2:172.21.15.117:6824/1183570837,v1:172.21.15.117:6825/1183570837] conn(0x7f50700ac790 msgr2=0x7f50700aec30 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:33:25.530 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:24.604+0000 7f508fe00640 1 --2- 172.21.15.83:0/552012498 >> [v2:172.21.15.117:6824/1183570837,v1:172.21.15.117:6825/1183570837] conn(0x7f50700ac790 0x7f50700aec30 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:33:25.530 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:24.604+0000 7f508fe00640 1 -- 172.21.15.83:0/552012498 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f50700a2c10 msgr2=0x7f50700a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:33:25.530 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:24.604+0000 7f508fe00640 1 --2- 172.21.15.83:0/552012498 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f50700a2c10 0x7f50700a50d0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f507c009b50 tx=0x7f507c005f90 comp rx=0 tx=0).stop 2024-08-19T01:33:25.530 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:24.604+0000 7f508fe00640 1 -- 172.21.15.83:0/552012498 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5088109b70 msgr2=0x7f508819ec50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:33:25.530 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:24.604+0000 7f508fe00640 1 --2- 172.21.15.83:0/552012498 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5088109b70 0x7f508819ec50 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f5080009ca0 tx=0x7f5080004a40 comp rx=0 tx=0).stop 2024-08-19T01:33:25.530 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:24.604+0000 7f508de00640 1 -- 172.21.15.83:0/552012498 reap_dead start 2024-08-19T01:33:25.530 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:24.604+0000 7f508fe00640 1 -- 172.21.15.83:0/552012498 shutdown_connections 2024-08-19T01:33:25.530 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:24.604+0000 7f508fe00640 1 -- 172.21.15.83:0/552012498 >> 172.21.15.83:0/552012498 conn(0x7f50880ff780 msgr2=0x7f50880ffff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:33:25.531 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:24.605+0000 7f508fe00640 1 -- 172.21.15.83:0/552012498 shutdown_connections 2024-08-19T01:33:25.531 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:24.605+0000 7f508fe00640 1 -- 172.21.15.83:0/552012498 wait complete. 2024-08-19T01:33:25.702 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:33:25 smithi117 ceph-mon[22016]: pgmap v296: 97 pgs: 97 active+clean; 640 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 1023 B/s wr, 6 op/s 2024-08-19T01:33:25.716 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:33:25 smithi167 ceph-mon[21875]: pgmap v296: 97 pgs: 97 active+clean; 640 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 1023 B/s wr, 6 op/s 2024-08-19T01:33:25.904 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:33:25 smithi083 ceph-mon[17120]: pgmap v296: 97 pgs: 97 active+clean; 640 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 1023 B/s wr, 6 op/s 2024-08-19T01:33:26.307 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:26.305+0000 7f8e0a200640 1 -- 172.21.15.83:0/3598185217 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8e041018b0 msgr2=0x7f8e04101d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:33:26.307 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:26.305+0000 7f8e0a200640 1 --2- 172.21.15.83:0/3598185217 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8e041018b0 0x7f8e04101d10 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f8dec009ca0 tx=0x7f8dec02a8c0 comp rx=0 tx=0).stop 2024-08-19T01:33:26.307 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:26.305+0000 7f8e0a200640 1 -- 172.21.15.83:0/3598185217 shutdown_connections 2024-08-19T01:33:26.307 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:26.305+0000 7f8e0a200640 1 --2- 172.21.15.83:0/3598185217 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8e04102250 0x7f8e0410e780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:33:26.307 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:26.305+0000 7f8e0a200640 1 --2- 172.21.15.83:0/3598185217 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8e041018b0 0x7f8e04101d10 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:33:26.307 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:26.305+0000 7f8e0a200640 1 --2- 172.21.15.83:0/3598185217 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8e041078b0 0x7f8e04107c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:33:26.307 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:26.305+0000 7f8e0a200640 1 -- 172.21.15.83:0/3598185217 >> 172.21.15.83:0/3598185217 conn(0x7f8e040fd5f0 msgr2=0x7f8e040ffa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:33:26.307 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:26.305+0000 7f8e0a200640 1 -- 172.21.15.83:0/3598185217 shutdown_connections 2024-08-19T01:33:26.307 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:26.305+0000 7f8e0a200640 1 -- 172.21.15.83:0/3598185217 wait complete. 2024-08-19T01:33:26.308 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:26.306+0000 7f8e0a200640 1 Processor -- start 2024-08-19T01:33:26.309 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:26.306+0000 7f8e0a200640 1 -- start start 2024-08-19T01:33:26.309 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:26.307+0000 7f8e0a200640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8e041018b0 0x7f8e041981c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:33:26.309 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:26.307+0000 7f8e0a200640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8e04102250 0x7f8e04198700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:33:26.309 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:26.307+0000 7f8e0a200640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8e041078b0 0x7f8e0419ca90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:33:26.309 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:26.307+0000 7f8e0a200640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f8e04111cc0 con 0x7f8e041018b0 2024-08-19T01:33:26.309 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:26.307+0000 7f8e0a200640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f8e04111b40 con 0x7f8e04102250 2024-08-19T01:33:26.309 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:26.307+0000 7f8e0a200640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f8e04111e40 con 0x7f8e041078b0 2024-08-19T01:33:26.309 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:26.307+0000 7f8e03400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8e041018b0 0x7f8e041981c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:33:26.309 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:26.307+0000 7f8e03400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8e041018b0 0x7f8e041981c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:60782/0 (socket says 172.21.15.83:60782) 2024-08-19T01:33:26.309 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:26.307+0000 7f8e03400640 1 -- 172.21.15.83:0/1926361571 learned_addr learned my addr 172.21.15.83:0/1926361571 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-08-19T01:33:26.309 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:26.307+0000 7f8dfaa00640 1 --2- 172.21.15.83:0/1926361571 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8e04102250 0x7f8e04198700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:33:26.309 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:26.307+0000 7f8e03e00640 1 --2- 172.21.15.83:0/1926361571 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8e041078b0 0x7f8e0419ca90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:33:26.310 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:26.307+0000 7f8e03400640 1 -- 172.21.15.83:0/1926361571 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8e041078b0 msgr2=0x7f8e0419ca90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:33:26.310 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:26.307+0000 7f8e03400640 1 --2- 172.21.15.83:0/1926361571 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8e041078b0 0x7f8e0419ca90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:33:26.311 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:26.307+0000 7f8e03400640 1 -- 172.21.15.83:0/1926361571 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8e04102250 msgr2=0x7f8e04198700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:33:26.311 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:26.307+0000 7f8e03400640 1 --2- 172.21.15.83:0/1926361571 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8e04102250 0x7f8e04198700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:33:26.311 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:26.308+0000 7f8e03400640 1 -- 172.21.15.83:0/1926361571 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8e0419d170 con 0x7f8e041018b0 2024-08-19T01:33:26.311 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:26.308+0000 7f8e03400640 1 --2- 172.21.15.83:0/1926361571 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8e041018b0 0x7f8e041981c0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f8df400ea60 tx=0x7f8df400ef20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:33:26.311 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:26.308+0000 7f8e00c00640 1 -- 172.21.15.83:0/1926361571 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8df400cde0 con 0x7f8e041018b0 2024-08-19T01:33:26.311 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:26.308+0000 7f8e00c00640 1 -- 172.21.15.83:0/1926361571 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8df40035f0 con 0x7f8e041018b0 2024-08-19T01:33:26.311 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:26.308+0000 7f8e0a200640 1 -- 172.21.15.83:0/1926361571 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8e0419d460 con 0x7f8e041018b0 2024-08-19T01:33:26.311 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:26.308+0000 7f8e00c00640 1 -- 172.21.15.83:0/1926361571 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8df4010670 con 0x7f8e041018b0 2024-08-19T01:33:26.311 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:26.308+0000 7f8e0a200640 1 -- 172.21.15.83:0/1926361571 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8e041a4d40 con 0x7f8e041018b0 2024-08-19T01:33:26.311 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:26.309+0000 7f8e0a200640 1 -- 172.21.15.83:0/1926361571 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f8e04062670 con 0x7f8e041018b0 2024-08-19T01:33:26.312 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:26.311+0000 7f8e00c00640 1 -- 172.21.15.83:0/1926361571 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f8df40037a0 con 0x7f8e041018b0 2024-08-19T01:33:26.314 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:26.312+0000 7f8e00c00640 1 --2- 172.21.15.83:0/1926361571 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f8de80a2c10 0x7f8de80a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:33:26.314 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:26.313+0000 7f8e00c00640 1 -- 172.21.15.83:0/1926361571 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8269+0+0 (secure 0 0 0) 0x7f8df40cf890 con 0x7f8e041018b0 2024-08-19T01:33:26.315 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:26.313+0000 7f8dfaa00640 1 --2- 172.21.15.83:0/1926361571 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f8de80a2c10 0x7f8de80a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:33:26.315 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:26.313+0000 7f8e00c00640 1 --2- 172.21.15.83:0/1926361571 >> [v2:172.21.15.167:6824/187027350,v1:172.21.15.167:6825/187027350] conn(0x7f8de80ac8c0 0x7f8de80aed60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:33:26.315 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:26.313+0000 7f8e00c00640 1 -- 172.21.15.83:0/1926361571 --> [v2:172.21.15.167:6824/187027350,v1:172.21.15.167:6825/187027350] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f8df401ee70 con 0x7f8de80ac8c0 2024-08-19T01:33:26.315 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:26.313+0000 7f8e00c00640 1 -- 172.21.15.83:0/1926361571 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_get_version_reply(handle=1 version=86) ==== 24+0+0 (secure 0 0 0) 0x7f8df40cfc40 con 0x7f8e041018b0 2024-08-19T01:33:26.315 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:26.313+0000 7f8dfaa00640 1 --2- 172.21.15.83:0/1926361571 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f8de80a2c10 0x7f8de80a50d0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f8dec006010 tx=0x7f8dec009820 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:33:26.316 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:26.313+0000 7f8e03e00640 1 --2- 172.21.15.83:0/1926361571 >> [v2:172.21.15.167:6824/187027350,v1:172.21.15.167:6825/187027350] conn(0x7f8de80ac8c0 0x7f8de80aed60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:33:26.316 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:26.314+0000 7f8e03e00640 1 --2- 172.21.15.83:0/1926361571 >> [v2:172.21.15.167:6824/187027350,v1:172.21.15.167:6825/187027350] conn(0x7f8de80ac8c0 0x7f8de80aed60 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.11 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:33:26.316 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:26.315+0000 7f8e00c00640 1 -- 172.21.15.83:0/1926361571 <== osd.11 v2:172.21.15.167:6824/187027350 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f8df401ee70 con 0x7f8de80ac8c0 2024-08-19T01:33:26.330 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:26.329+0000 7f8e0a200640 1 -- 172.21.15.83:0/1926361571 --> [v2:172.21.15.167:6824/187027350,v1:172.21.15.167:6825/187027350] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f8e04062a70 con 0x7f8de80ac8c0 2024-08-19T01:33:26.331 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:26.330+0000 7f8e00c00640 1 -- 172.21.15.83:0/1926361571 <== osd.11 v2:172.21.15.167:6824/187027350 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f8e04062a70 con 0x7f8de80ac8c0 2024-08-19T01:33:26.332 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:26.330+0000 7f8e0a200640 1 -- 172.21.15.83:0/1926361571 >> [v2:172.21.15.167:6824/187027350,v1:172.21.15.167:6825/187027350] conn(0x7f8de80ac8c0 msgr2=0x7f8de80aed60 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:33:26.332 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:26.330+0000 7f8e0a200640 1 --2- 172.21.15.83:0/1926361571 >> [v2:172.21.15.167:6824/187027350,v1:172.21.15.167:6825/187027350] conn(0x7f8de80ac8c0 0x7f8de80aed60 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:33:26.332 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:26.330+0000 7f8e0a200640 1 -- 172.21.15.83:0/1926361571 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f8de80a2c10 msgr2=0x7f8de80a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:33:26.332 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:26.330+0000 7f8e0a200640 1 --2- 172.21.15.83:0/1926361571 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f8de80a2c10 0x7f8de80a50d0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f8dec006010 tx=0x7f8dec009820 comp rx=0 tx=0).stop 2024-08-19T01:33:26.332 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:26.330+0000 7f8e0a200640 1 -- 172.21.15.83:0/1926361571 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8e041018b0 msgr2=0x7f8e041981c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:33:26.332 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:26.330+0000 7f8e0a200640 1 --2- 172.21.15.83:0/1926361571 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8e041018b0 0x7f8e041981c0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f8df400ea60 tx=0x7f8df400ef20 comp rx=0 tx=0).stop 2024-08-19T01:33:26.332 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:26.330+0000 7f8e03e00640 1 -- 172.21.15.83:0/1926361571 reap_dead start 2024-08-19T01:33:26.333 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:26.330+0000 7f8e0a200640 1 -- 172.21.15.83:0/1926361571 shutdown_connections 2024-08-19T01:33:26.333 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:26.330+0000 7f8e0a200640 1 -- 172.21.15.83:0/1926361571 >> 172.21.15.83:0/1926361571 conn(0x7f8e040fd5f0 msgr2=0x7f8e040fdda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:33:26.333 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:26.330+0000 7f8e0a200640 1 -- 172.21.15.83:0/1926361571 shutdown_connections 2024-08-19T01:33:26.333 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:26.331+0000 7f8e0a200640 1 -- 172.21.15.83:0/1926361571 wait complete. 2024-08-19T01:33:26.957 INFO:teuthology.orchestra.run.smithi083.stdout:304942678043 2024-08-19T01:33:26.957 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833 shell --fsid bad8780e-5dc9-11ef-bccf-c7b262605968 -- ceph osd last-stat-seq osd.10 2024-08-19T01:33:27.495 INFO:teuthology.orchestra.run.smithi083.stdout:219043332137 2024-08-19T01:33:27.496 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833 shell --fsid bad8780e-5dc9-11ef-bccf-c7b262605968 -- ceph osd last-stat-seq osd.7 2024-08-19T01:33:27.702 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:33:27 smithi117 ceph-mon[22016]: pgmap v297: 97 pgs: 97 active+clean; 640 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 1023 B/s wr, 7 op/s 2024-08-19T01:33:27.716 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:33:27 smithi167 ceph-mon[21875]: pgmap v297: 97 pgs: 97 active+clean; 640 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 1023 B/s wr, 7 op/s 2024-08-19T01:33:27.729 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:33:27 smithi083 ceph-mon[17120]: pgmap v297: 97 pgs: 97 active+clean; 640 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 1023 B/s wr, 7 op/s 2024-08-19T01:33:28.115 INFO:teuthology.orchestra.run.smithi083.stdout:107374182459 2024-08-19T01:33:28.115 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833 shell --fsid bad8780e-5dc9-11ef-bccf-c7b262605968 -- ceph osd last-stat-seq osd.3 2024-08-19T01:33:28.495 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:28.488+0000 7f1759000640 1 -- 172.21.15.83:0/717552301 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f17541016f0 msgr2=0x7f1754101b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:33:28.496 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:28.488+0000 7f1759000640 1 --2- 172.21.15.83:0/717552301 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f17541016f0 0x7f1754101b50 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f1740009930 tx=0x7f174002a420 comp rx=0 tx=0).stop 2024-08-19T01:33:28.496 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:28.489+0000 7f1759000640 1 -- 172.21.15.83:0/717552301 shutdown_connections 2024-08-19T01:33:28.496 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:28.489+0000 7f1759000640 1 --2- 172.21.15.83:0/717552301 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1754102090 0x7f175410e5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:33:28.496 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:28.489+0000 7f1759000640 1 --2- 172.21.15.83:0/717552301 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f17541016f0 0x7f1754101b50 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:33:28.496 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:28.489+0000 7f1759000640 1 --2- 172.21.15.83:0/717552301 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f17541076f0 0x7f1754107ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:33:28.496 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:28.489+0000 7f1759000640 1 -- 172.21.15.83:0/717552301 >> 172.21.15.83:0/717552301 conn(0x7f17540fd470 msgr2=0x7f17540ff890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:33:28.496 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:28.489+0000 7f1759000640 1 -- 172.21.15.83:0/717552301 shutdown_connections 2024-08-19T01:33:28.496 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:28.489+0000 7f1759000640 1 -- 172.21.15.83:0/717552301 wait complete. 2024-08-19T01:33:28.496 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:28.489+0000 7f1759000640 1 Processor -- start 2024-08-19T01:33:28.496 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:28.490+0000 7f1759000640 1 -- start start 2024-08-19T01:33:28.496 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:28.490+0000 7f1759000640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f17541016f0 0x7f175419a210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:33:28.496 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:28.490+0000 7f1759000640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1754102090 0x7f175419a750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:33:28.497 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:28.490+0000 7f1759000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f17541076f0 0x7f175419eae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:33:28.497 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:28.490+0000 7f1759000640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f1754111b40 con 0x7f17541076f0 2024-08-19T01:33:28.497 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:28.490+0000 7f1759000640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f17541119c0 con 0x7f1754102090 2024-08-19T01:33:28.497 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:28.490+0000 7f1759000640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f1754111cc0 con 0x7f17541016f0 2024-08-19T01:33:28.497 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:28.490+0000 7f1753e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f17541076f0 0x7f175419eae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:33:28.497 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:28.490+0000 7f1753e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f17541076f0 0x7f175419eae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:60822/0 (socket says 172.21.15.83:60822) 2024-08-19T01:33:28.497 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:28.490+0000 7f1753e00640 1 -- 172.21.15.83:0/2742838961 learned_addr learned my addr 172.21.15.83:0/2742838961 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-08-19T01:33:28.497 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:28.490+0000 7f1753400640 1 --2- 172.21.15.83:0/2742838961 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f17541016f0 0x7f175419a210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:33:28.497 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:28.490+0000 7f1752a00640 1 --2- 172.21.15.83:0/2742838961 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1754102090 0x7f175419a750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:33:28.497 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:28.491+0000 7f1753e00640 1 -- 172.21.15.83:0/2742838961 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f17541016f0 msgr2=0x7f175419a210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:33:28.497 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:28.491+0000 7f1753e00640 1 --2- 172.21.15.83:0/2742838961 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f17541016f0 0x7f175419a210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:33:28.498 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:28.491+0000 7f1753e00640 1 -- 172.21.15.83:0/2742838961 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1754102090 msgr2=0x7f175419a750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:33:28.498 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:28.491+0000 7f1753e00640 1 --2- 172.21.15.83:0/2742838961 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1754102090 0x7f175419a750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:33:28.498 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:28.491+0000 7f1753e00640 1 -- 172.21.15.83:0/2742838961 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f175419f1c0 con 0x7f17541076f0 2024-08-19T01:33:28.498 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:28.491+0000 7f1753e00640 1 --2- 172.21.15.83:0/2742838961 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f17541076f0 0x7f175419eae0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f174c00efd0 tx=0x7f174c00c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:33:28.498 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:28.491+0000 7f174be00640 1 -- 172.21.15.83:0/2742838961 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f174c003b70 con 0x7f17541076f0 2024-08-19T01:33:28.498 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:28.491+0000 7f174be00640 1 -- 172.21.15.83:0/2742838961 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f174c003d10 con 0x7f17541076f0 2024-08-19T01:33:28.498 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:28.491+0000 7f1759000640 1 -- 172.21.15.83:0/2742838961 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f175419f4b0 con 0x7f17541076f0 2024-08-19T01:33:28.498 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:28.491+0000 7f174be00640 1 -- 172.21.15.83:0/2742838961 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f174c0109f0 con 0x7f17541076f0 2024-08-19T01:33:28.498 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:28.491+0000 7f1759000640 1 -- 172.21.15.83:0/2742838961 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f17541a6d90 con 0x7f17541076f0 2024-08-19T01:33:28.498 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:28.492+0000 7f1759000640 1 -- 172.21.15.83:0/2742838961 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f17540fee30 con 0x7f17541076f0 2024-08-19T01:33:28.499 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:28.494+0000 7f174be00640 1 -- 172.21.15.83:0/2742838961 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f174c010b90 con 0x7f17541076f0 2024-08-19T01:33:28.500 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:28.496+0000 7f174be00640 1 --2- 172.21.15.83:0/2742838961 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f17380a2c10 0x7f17380a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:33:28.500 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:28.496+0000 7f174be00640 1 -- 172.21.15.83:0/2742838961 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8269+0+0 (secure 0 0 0) 0x7f174c0cfa60 con 0x7f17541076f0 2024-08-19T01:33:28.500 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:28.496+0000 7f1753400640 1 --2- 172.21.15.83:0/2742838961 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f17380a2c10 0x7f17380a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:33:28.500 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:28.496+0000 7f174be00640 1 --2- 172.21.15.83:0/2742838961 >> [v2:172.21.15.167:6808/2761858035,v1:172.21.15.167:6809/2761858035] conn(0x7f17380ac8c0 0x7f17380aed60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:33:28.500 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:28.496+0000 7f174be00640 1 -- 172.21.15.83:0/2742838961 --> [v2:172.21.15.167:6808/2761858035,v1:172.21.15.167:6809/2761858035] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f174c009820 con 0x7f17380ac8c0 2024-08-19T01:33:28.500 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:28.496+0000 7f174be00640 1 -- 172.21.15.83:0/2742838961 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_get_version_reply(handle=1 version=86) ==== 24+0+0 (secure 0 0 0) 0x7f174c0cfe10 con 0x7f17541076f0 2024-08-19T01:33:28.500 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:28.497+0000 7f1752a00640 1 --2- 172.21.15.83:0/2742838961 >> [v2:172.21.15.167:6808/2761858035,v1:172.21.15.167:6809/2761858035] conn(0x7f17380ac8c0 0x7f17380aed60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:33:28.501 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:28.497+0000 7f1753400640 1 --2- 172.21.15.83:0/2742838961 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f17380a2c10 0x7f17380a50d0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f1744009b50 tx=0x7f1744005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:33:28.501 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:28.497+0000 7f1752a00640 1 --2- 172.21.15.83:0/2742838961 >> [v2:172.21.15.167:6808/2761858035,v1:172.21.15.167:6809/2761858035] conn(0x7f17380ac8c0 0x7f17380aed60 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:33:28.501 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:28.498+0000 7f174be00640 1 -- 172.21.15.83:0/2742838961 <== osd.5 v2:172.21.15.167:6808/2761858035 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f174c009820 con 0x7f17380ac8c0 2024-08-19T01:33:28.519 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:28.518+0000 7f1759000640 1 -- 172.21.15.83:0/2742838961 --> [v2:172.21.15.167:6808/2761858035,v1:172.21.15.167:6809/2761858035] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f1754102e30 con 0x7f17380ac8c0 2024-08-19T01:33:28.520 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:28.519+0000 7f174be00640 1 -- 172.21.15.83:0/2742838961 <== osd.5 v2:172.21.15.167:6808/2761858035 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f1754102e30 con 0x7f17380ac8c0 2024-08-19T01:33:28.521 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:28.519+0000 7f1759000640 1 -- 172.21.15.83:0/2742838961 >> [v2:172.21.15.167:6808/2761858035,v1:172.21.15.167:6809/2761858035] conn(0x7f17380ac8c0 msgr2=0x7f17380aed60 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:33:28.521 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:28.519+0000 7f1759000640 1 --2- 172.21.15.83:0/2742838961 >> [v2:172.21.15.167:6808/2761858035,v1:172.21.15.167:6809/2761858035] conn(0x7f17380ac8c0 0x7f17380aed60 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:33:28.521 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:28.519+0000 7f1759000640 1 -- 172.21.15.83:0/2742838961 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f17380a2c10 msgr2=0x7f17380a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:33:28.521 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:28.519+0000 7f1759000640 1 --2- 172.21.15.83:0/2742838961 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f17380a2c10 0x7f17380a50d0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f1744009b50 tx=0x7f1744005f90 comp rx=0 tx=0).stop 2024-08-19T01:33:28.521 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:28.519+0000 7f1759000640 1 -- 172.21.15.83:0/2742838961 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f17541076f0 msgr2=0x7f175419eae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:33:28.521 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:28.519+0000 7f1759000640 1 --2- 172.21.15.83:0/2742838961 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f17541076f0 0x7f175419eae0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f174c00efd0 tx=0x7f174c00c6a0 comp rx=0 tx=0).stop 2024-08-19T01:33:28.521 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:28.519+0000 7f1753e00640 1 -- 172.21.15.83:0/2742838961 reap_dead start 2024-08-19T01:33:28.522 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:28.519+0000 7f1759000640 1 -- 172.21.15.83:0/2742838961 shutdown_connections 2024-08-19T01:33:28.522 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:28.519+0000 7f1759000640 1 -- 172.21.15.83:0/2742838961 >> 172.21.15.83:0/2742838961 conn(0x7f17540fd470 msgr2=0x7f175410cca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:33:28.522 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:28.519+0000 7f1759000640 1 -- 172.21.15.83:0/2742838961 shutdown_connections 2024-08-19T01:33:28.522 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:28.520+0000 7f1759000640 1 -- 172.21.15.83:0/2742838961 wait complete. 2024-08-19T01:33:28.613 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:28.384+0000 7f3871e00640 1 -- 172.21.15.83:0/3386500430 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f386c1020a0 msgr2=0x7f386c10e5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:33:28.613 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:28.384+0000 7f3871e00640 1 --2- 172.21.15.83:0/3386500430 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f386c1020a0 0x7f386c10e5d0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f3864009be0 tx=0x7f386402a6c0 comp rx=0 tx=0).stop 2024-08-19T01:33:28.613 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:28.384+0000 7f3871e00640 1 -- 172.21.15.83:0/3386500430 shutdown_connections 2024-08-19T01:33:28.613 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:28.384+0000 7f3871e00640 1 --2- 172.21.15.83:0/3386500430 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f386c1020a0 0x7f386c10e5d0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:33:28.614 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:28.384+0000 7f3871e00640 1 --2- 172.21.15.83:0/3386500430 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f386c101700 0x7f386c101b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:33:28.614 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:28.384+0000 7f3871e00640 1 --2- 172.21.15.83:0/3386500430 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f386c107700 0x7f386c107ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:33:28.614 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:28.384+0000 7f3871e00640 1 -- 172.21.15.83:0/3386500430 >> 172.21.15.83:0/3386500430 conn(0x7f386c0fd440 msgr2=0x7f386c0ff860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:33:28.614 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:28.384+0000 7f3871e00640 1 -- 172.21.15.83:0/3386500430 shutdown_connections 2024-08-19T01:33:28.614 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:28.384+0000 7f3871e00640 1 -- 172.21.15.83:0/3386500430 wait complete. 2024-08-19T01:33:28.614 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:28.385+0000 7f3871e00640 1 Processor -- start 2024-08-19T01:33:28.614 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:28.385+0000 7f3871e00640 1 -- start start 2024-08-19T01:33:28.614 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:28.386+0000 7f3871e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f386c101700 0x7f386c19a190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:33:28.614 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:28.386+0000 7f3871e00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f386c1020a0 0x7f386c19a6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:33:28.614 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:28.386+0000 7f3871e00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f386c107700 0x7f386c19ea60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:33:28.614 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:28.386+0000 7f3871e00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f386c111b00 con 0x7f386c101700 2024-08-19T01:33:28.615 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:28.386+0000 7f3871e00640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f386c111980 con 0x7f386c1020a0 2024-08-19T01:33:28.615 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:28.386+0000 7f3871e00640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f386c111c80 con 0x7f386c107700 2024-08-19T01:33:28.615 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:28.386+0000 7f3870a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f386c101700 0x7f386c19a190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:33:28.615 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:28.386+0000 7f3863e00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f386c1020a0 0x7f386c19a6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:33:28.615 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:28.386+0000 7f3870a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f386c101700 0x7f386c19a190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:60810/0 (socket says 172.21.15.83:60810) 2024-08-19T01:33:28.615 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:28.386+0000 7f3870a00640 1 -- 172.21.15.83:0/26967945 learned_addr learned my addr 172.21.15.83:0/26967945 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-08-19T01:33:28.615 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:28.386+0000 7f3871400640 1 --2- 172.21.15.83:0/26967945 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f386c107700 0x7f386c19ea60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:33:28.615 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:28.387+0000 7f3870a00640 1 -- 172.21.15.83:0/26967945 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f386c107700 msgr2=0x7f386c19ea60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:33:28.615 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:28.387+0000 7f3870a00640 1 --2- 172.21.15.83:0/26967945 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f386c107700 0x7f386c19ea60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:33:28.615 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:28.387+0000 7f3870a00640 1 -- 172.21.15.83:0/26967945 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f386c1020a0 msgr2=0x7f386c19a6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:33:28.615 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:28.387+0000 7f3870a00640 1 --2- 172.21.15.83:0/26967945 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f386c1020a0 0x7f386c19a6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:33:28.615 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:28.387+0000 7f3870a00640 1 -- 172.21.15.83:0/26967945 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f386c19f140 con 0x7f386c101700 2024-08-19T01:33:28.615 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:28.387+0000 7f3863e00640 1 --2- 172.21.15.83:0/26967945 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f386c1020a0 0x7f386c19a6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-19T01:33:28.615 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:28.387+0000 7f3870a00640 1 --2- 172.21.15.83:0/26967945 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f386c101700 0x7f386c19a190 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f385c00e9c0 tx=0x7f385c00ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:33:28.616 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:28.387+0000 7f386a000640 1 -- 172.21.15.83:0/26967945 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f385c00ccc0 con 0x7f386c101700 2024-08-19T01:33:28.616 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:28.387+0000 7f3871e00640 1 -- 172.21.15.83:0/26967945 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f386c19f430 con 0x7f386c101700 2024-08-19T01:33:28.616 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:28.387+0000 7f386a000640 1 -- 172.21.15.83:0/26967945 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f385c005500 con 0x7f386c101700 2024-08-19T01:33:28.616 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:28.387+0000 7f3871e00640 1 -- 172.21.15.83:0/26967945 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f386c1a6d10 con 0x7f386c101700 2024-08-19T01:33:28.616 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:28.388+0000 7f386a000640 1 -- 172.21.15.83:0/26967945 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f385c018430 con 0x7f386c101700 2024-08-19T01:33:28.616 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:28.388+0000 7f3871e00640 1 -- 172.21.15.83:0/26967945 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f386c0ff410 con 0x7f386c101700 2024-08-19T01:33:28.616 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:28.390+0000 7f386a000640 1 -- 172.21.15.83:0/26967945 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f385c0046c0 con 0x7f386c101700 2024-08-19T01:33:28.616 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:28.391+0000 7f386a000640 1 --2- 172.21.15.83:0/26967945 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f38500a2c10 0x7f38500a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:33:28.616 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:28.391+0000 7f386a000640 1 -- 172.21.15.83:0/26967945 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8269+0+0 (secure 0 0 0) 0x7f385c0cf590 con 0x7f386c101700 2024-08-19T01:33:28.616 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:28.392+0000 7f3863e00640 1 --2- 172.21.15.83:0/26967945 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f38500a2c10 0x7f38500a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:33:28.616 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:28.392+0000 7f386a000640 1 --2- 172.21.15.83:0/26967945 >> [v2:172.21.15.167:6800/551625328,v1:172.21.15.167:6801/551625328] conn(0x7f38500ac8c0 0x7f38500aed60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:33:28.616 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:28.392+0000 7f386a000640 1 -- 172.21.15.83:0/26967945 --> [v2:172.21.15.167:6800/551625328,v1:172.21.15.167:6801/551625328] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f385c004470 con 0x7f38500ac8c0 2024-08-19T01:33:28.616 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:28.392+0000 7f386a000640 1 -- 172.21.15.83:0/26967945 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_get_version_reply(handle=1 version=86) ==== 24+0+0 (secure 0 0 0) 0x7f385c0d6050 con 0x7f386c101700 2024-08-19T01:33:28.616 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:28.392+0000 7f3871400640 1 --2- 172.21.15.83:0/26967945 >> [v2:172.21.15.167:6800/551625328,v1:172.21.15.167:6801/551625328] conn(0x7f38500ac8c0 0x7f38500aed60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:33:28.617 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:28.392+0000 7f3863e00640 1 --2- 172.21.15.83:0/26967945 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f38500a2c10 0x7f38500a50d0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f386c10e270 tx=0x7f3854005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:33:28.617 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:28.393+0000 7f3871400640 1 --2- 172.21.15.83:0/26967945 >> [v2:172.21.15.167:6800/551625328,v1:172.21.15.167:6801/551625328] conn(0x7f38500ac8c0 0x7f38500aed60 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:33:28.617 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:28.394+0000 7f386a000640 1 -- 172.21.15.83:0/26967945 <== osd.2 v2:172.21.15.167:6800/551625328 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f385c004470 con 0x7f38500ac8c0 2024-08-19T01:33:28.617 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:28.408+0000 7f3871e00640 1 -- 172.21.15.83:0/26967945 --> [v2:172.21.15.167:6800/551625328,v1:172.21.15.167:6801/551625328] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f386c102e40 con 0x7f38500ac8c0 2024-08-19T01:33:28.617 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:28.408+0000 7f386a000640 1 -- 172.21.15.83:0/26967945 <== osd.2 v2:172.21.15.167:6800/551625328 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f386c102e40 con 0x7f38500ac8c0 2024-08-19T01:33:28.617 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:28.409+0000 7f3871e00640 1 -- 172.21.15.83:0/26967945 >> [v2:172.21.15.167:6800/551625328,v1:172.21.15.167:6801/551625328] conn(0x7f38500ac8c0 msgr2=0x7f38500aed60 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:33:28.617 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:28.409+0000 7f3871e00640 1 --2- 172.21.15.83:0/26967945 >> [v2:172.21.15.167:6800/551625328,v1:172.21.15.167:6801/551625328] conn(0x7f38500ac8c0 0x7f38500aed60 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:33:28.617 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:28.409+0000 7f3871e00640 1 -- 172.21.15.83:0/26967945 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f38500a2c10 msgr2=0x7f38500a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:33:28.618 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:28.409+0000 7f3871e00640 1 --2- 172.21.15.83:0/26967945 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f38500a2c10 0x7f38500a50d0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f386c10e270 tx=0x7f3854005f90 comp rx=0 tx=0).stop 2024-08-19T01:33:28.618 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:28.409+0000 7f3871e00640 1 -- 172.21.15.83:0/26967945 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f386c101700 msgr2=0x7f386c19a190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:33:28.618 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:28.409+0000 7f3871e00640 1 --2- 172.21.15.83:0/26967945 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f386c101700 0x7f386c19a190 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f385c00e9c0 tx=0x7f385c00ee80 comp rx=0 tx=0).stop 2024-08-19T01:33:28.618 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:28.409+0000 7f3871400640 1 -- 172.21.15.83:0/26967945 reap_dead start 2024-08-19T01:33:28.618 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:28.409+0000 7f3871e00640 1 -- 172.21.15.83:0/26967945 shutdown_connections 2024-08-19T01:33:28.618 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:28.409+0000 7f3871e00640 1 -- 172.21.15.83:0/26967945 >> 172.21.15.83:0/26967945 conn(0x7f386c0fd440 msgr2=0x7f386c0fe9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:33:28.618 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:28.409+0000 7f3871e00640 1 -- 172.21.15.83:0/26967945 shutdown_connections 2024-08-19T01:33:28.618 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:28.409+0000 7f3871e00640 1 -- 172.21.15.83:0/26967945 wait complete. 2024-08-19T01:33:28.991 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:26.885+0000 7f4f07200640 1 -- 172.21.15.83:0/358253029 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4f001042a0 msgr2=0x7f4f001107d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:33:28.991 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:26.885+0000 7f4f07200640 1 --2- 172.21.15.83:0/358253029 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4f001042a0 0x7f4f001107d0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f4efc009cd0 tx=0x7f4efc02a8e0 comp rx=0 tx=0).stop 2024-08-19T01:33:28.991 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:26.885+0000 7f4f07200640 1 -- 172.21.15.83:0/358253029 shutdown_connections 2024-08-19T01:33:28.991 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:26.885+0000 7f4f07200640 1 --2- 172.21.15.83:0/358253029 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4f001042a0 0x7f4f001107d0 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:33:28.991 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:26.885+0000 7f4f07200640 1 --2- 172.21.15.83:0/358253029 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4f00103900 0x7f4f00103d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:33:28.991 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:26.885+0000 7f4f07200640 1 --2- 172.21.15.83:0/358253029 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4f00109900 0x7f4f00109ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:33:28.991 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:26.885+0000 7f4f07200640 1 -- 172.21.15.83:0/358253029 >> 172.21.15.83:0/358253029 conn(0x7f4f000ff600 msgr2=0x7f4f00101a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:33:28.991 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:26.885+0000 7f4f07200640 1 -- 172.21.15.83:0/358253029 shutdown_connections 2024-08-19T01:33:28.992 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:26.885+0000 7f4f07200640 1 -- 172.21.15.83:0/358253029 wait complete. 2024-08-19T01:33:28.992 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:26.886+0000 7f4f07200640 1 Processor -- start 2024-08-19T01:33:28.992 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:26.886+0000 7f4f07200640 1 -- start start 2024-08-19T01:33:28.992 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:26.886+0000 7f4f07200640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4f00103900 0x7f4f0019a1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:33:28.992 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:26.886+0000 7f4f07200640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4f001042a0 0x7f4f0019a6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:33:28.992 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:26.886+0000 7f4f07200640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4f00109900 0x7f4f0019ea80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:33:28.992 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:26.887+0000 7f4f07200640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f4f00113d50 con 0x7f4f00109900 2024-08-19T01:33:28.992 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:26.887+0000 7f4f07200640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f4f00113bd0 con 0x7f4f00103900 2024-08-19T01:33:28.992 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:26.887+0000 7f4f07200640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f4f00113ed0 con 0x7f4f001042a0 2024-08-19T01:33:28.992 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:26.887+0000 7f4f06800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4f00109900 0x7f4f0019ea80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:33:28.992 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:26.887+0000 7f4f05e00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4f00103900 0x7f4f0019a1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:33:28.992 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:26.887+0000 7f4f05e00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4f00103900 0x7f4f0019a1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.83:45852/0 (socket says 172.21.15.83:45852) 2024-08-19T01:33:28.993 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:26.887+0000 7f4f06800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4f00109900 0x7f4f0019ea80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:60802/0 (socket says 172.21.15.83:60802) 2024-08-19T01:33:28.993 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:26.887+0000 7f4f05e00640 1 -- 172.21.15.83:0/4066648863 learned_addr learned my addr 172.21.15.83:0/4066648863 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-08-19T01:33:28.993 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:26.887+0000 7f4f05400640 1 --2- 172.21.15.83:0/4066648863 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4f001042a0 0x7f4f0019a6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:33:28.993 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:26.887+0000 7f4f06800640 1 -- 172.21.15.83:0/4066648863 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4f001042a0 msgr2=0x7f4f0019a6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:33:28.993 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:26.887+0000 7f4f06800640 1 --2- 172.21.15.83:0/4066648863 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4f001042a0 0x7f4f0019a6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:33:28.993 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:26.887+0000 7f4f06800640 1 -- 172.21.15.83:0/4066648863 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4f00103900 msgr2=0x7f4f0019a1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:33:28.993 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:26.887+0000 7f4f06800640 1 --2- 172.21.15.83:0/4066648863 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4f00103900 0x7f4f0019a1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:33:28.993 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:26.887+0000 7f4f06800640 1 -- 172.21.15.83:0/4066648863 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4f0019f160 con 0x7f4f00109900 2024-08-19T01:33:28.993 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:26.888+0000 7f4f06800640 1 --2- 172.21.15.83:0/4066648863 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4f00109900 0x7f4f0019ea80 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f4efc009ca0 tx=0x7f4efc031e60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:33:28.993 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:26.888+0000 7f4efaa00640 1 -- 172.21.15.83:0/4066648863 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4efc004040 con 0x7f4f00109900 2024-08-19T01:33:28.993 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:26.888+0000 7f4efaa00640 1 -- 172.21.15.83:0/4066648863 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4efc0046c0 con 0x7f4f00109900 2024-08-19T01:33:28.993 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:26.888+0000 7f4f07200640 1 -- 172.21.15.83:0/4066648863 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4f0019f3f0 con 0x7f4f00109900 2024-08-19T01:33:28.993 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:26.888+0000 7f4efaa00640 1 -- 172.21.15.83:0/4066648863 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4efc030e40 con 0x7f4f00109900 2024-08-19T01:33:28.993 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:26.888+0000 7f4f07200640 1 -- 172.21.15.83:0/4066648863 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4f001a6d30 con 0x7f4f00109900 2024-08-19T01:33:28.993 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:26.889+0000 7f4f07200640 1 -- 172.21.15.83:0/4066648863 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f4f001007d0 con 0x7f4f00109900 2024-08-19T01:33:28.994 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:26.891+0000 7f4efaa00640 1 -- 172.21.15.83:0/4066648863 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f4efc0305a0 con 0x7f4f00109900 2024-08-19T01:33:28.994 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:26.892+0000 7f4efaa00640 1 --2- 172.21.15.83:0/4066648863 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f4ee80a2c90 0x7f4ee80a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:33:28.994 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:26.892+0000 7f4efaa00640 1 -- 172.21.15.83:0/4066648863 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8269+0+0 (secure 0 0 0) 0x7f4efc02e030 con 0x7f4f00109900 2024-08-19T01:33:28.994 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:26.892+0000 7f4f05e00640 1 --2- 172.21.15.83:0/4066648863 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f4ee80a2c90 0x7f4ee80a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:33:28.994 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:26.893+0000 7f4efaa00640 1 --2- 172.21.15.83:0/4066648863 >> [v2:172.21.15.83:6826/903463450,v1:172.21.15.83:6827/903463450] conn(0x7f4ee80ac940 0x7f4ee80aede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:33:28.994 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:26.893+0000 7f4efaa00640 1 -- 172.21.15.83:0/4066648863 --> [v2:172.21.15.83:6826/903463450,v1:172.21.15.83:6827/903463450] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f4efc0059a0 con 0x7f4ee80ac940 2024-08-19T01:33:28.994 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:26.893+0000 7f4efaa00640 1 -- 172.21.15.83:0/4066648863 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_get_version_reply(handle=1 version=86) ==== 24+0+0 (secure 0 0 0) 0x7f4efc0f0ab0 con 0x7f4f00109900 2024-08-19T01:33:28.994 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:26.893+0000 7f4f05400640 1 --2- 172.21.15.83:0/4066648863 >> [v2:172.21.15.83:6826/903463450,v1:172.21.15.83:6827/903463450] conn(0x7f4ee80ac940 0x7f4ee80aede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:33:28.994 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:26.893+0000 7f4f05e00640 1 --2- 172.21.15.83:0/4066648863 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f4ee80a2c90 0x7f4ee80a5150 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f4ef4009c10 tx=0x7f4ef4005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:33:28.994 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:26.893+0000 7f4f05400640 1 --2- 172.21.15.83:0/4066648863 >> [v2:172.21.15.83:6826/903463450,v1:172.21.15.83:6827/903463450] conn(0x7f4ee80ac940 0x7f4ee80aede0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.9 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:33:28.994 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:26.894+0000 7f4efaa00640 1 -- 172.21.15.83:0/4066648863 <== osd.9 v2:172.21.15.83:6826/903463450 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f4efc0059a0 con 0x7f4ee80ac940 2024-08-19T01:33:28.994 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:26.909+0000 7f4f07200640 1 -- 172.21.15.83:0/4066648863 --> [v2:172.21.15.83:6826/903463450,v1:172.21.15.83:6827/903463450] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f4f00105040 con 0x7f4ee80ac940 2024-08-19T01:33:28.994 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:26.909+0000 7f4efaa00640 1 -- 172.21.15.83:0/4066648863 <== osd.9 v2:172.21.15.83:6826/903463450 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f4f00105040 con 0x7f4ee80ac940 2024-08-19T01:33:28.995 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:26.909+0000 7f4f07200640 1 -- 172.21.15.83:0/4066648863 >> [v2:172.21.15.83:6826/903463450,v1:172.21.15.83:6827/903463450] conn(0x7f4ee80ac940 msgr2=0x7f4ee80aede0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:33:28.995 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:26.909+0000 7f4f07200640 1 --2- 172.21.15.83:0/4066648863 >> [v2:172.21.15.83:6826/903463450,v1:172.21.15.83:6827/903463450] conn(0x7f4ee80ac940 0x7f4ee80aede0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:33:28.995 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:26.909+0000 7f4f07200640 1 -- 172.21.15.83:0/4066648863 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f4ee80a2c90 msgr2=0x7f4ee80a5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:33:28.995 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:26.909+0000 7f4f07200640 1 --2- 172.21.15.83:0/4066648863 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f4ee80a2c90 0x7f4ee80a5150 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f4ef4009c10 tx=0x7f4ef4005f90 comp rx=0 tx=0).stop 2024-08-19T01:33:28.995 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:26.910+0000 7f4f07200640 1 -- 172.21.15.83:0/4066648863 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4f00109900 msgr2=0x7f4f0019ea80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:33:28.995 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:26.910+0000 7f4f07200640 1 --2- 172.21.15.83:0/4066648863 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4f00109900 0x7f4f0019ea80 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f4efc009ca0 tx=0x7f4efc031e60 comp rx=0 tx=0).stop 2024-08-19T01:33:28.996 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:26.910+0000 7f4f06800640 1 -- 172.21.15.83:0/4066648863 reap_dead start 2024-08-19T01:33:28.996 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:26.910+0000 7f4f07200640 1 -- 172.21.15.83:0/4066648863 shutdown_connections 2024-08-19T01:33:28.996 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:26.910+0000 7f4f07200640 1 -- 172.21.15.83:0/4066648863 >> 172.21.15.83:0/4066648863 conn(0x7f4f000ff600 msgr2=0x7f4f000ffd70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:33:28.996 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:26.910+0000 7f4f07200640 1 -- 172.21.15.83:0/4066648863 shutdown_connections 2024-08-19T01:33:28.996 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:26.910+0000 7f4f07200640 1 -- 172.21.15.83:0/4066648863 wait complete. 2024-08-19T01:33:29.511 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:33:29 smithi083 ceph-mon[17120]: pgmap v298: 97 pgs: 97 active+clean; 640 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 255 B/s wr, 6 op/s 2024-08-19T01:33:29.702 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:33:29 smithi117 ceph-mon[22016]: pgmap v298: 97 pgs: 97 active+clean; 640 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 255 B/s wr, 6 op/s 2024-08-19T01:33:29.716 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:33:29 smithi167 ceph-mon[21875]: pgmap v298: 97 pgs: 97 active+clean; 640 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 255 B/s wr, 6 op/s 2024-08-19T01:33:30.302 INFO:teuthology.orchestra.run.smithi083.stdout:283467841567 2024-08-19T01:33:30.302 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833 shell --fsid bad8780e-5dc9-11ef-bccf-c7b262605968 -- ceph osd last-stat-seq osd.9 2024-08-19T01:33:31.654 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:33:31 smithi083 ceph-mon[17120]: pgmap v299: 97 pgs: 97 active+clean; 640 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 255 B/s wr, 6 op/s 2024-08-19T01:33:31.702 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:33:31 smithi117 ceph-mon[22016]: pgmap v299: 97 pgs: 97 active+clean; 640 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 255 B/s wr, 6 op/s 2024-08-19T01:33:31.716 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:33:31 smithi167 ceph-mon[21875]: pgmap v299: 97 pgs: 97 active+clean; 640 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 255 B/s wr, 6 op/s 2024-08-19T01:33:31.841 INFO:teuthology.orchestra.run.smithi083.stdout:163208757300 2024-08-19T01:33:31.841 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833 shell --fsid bad8780e-5dc9-11ef-bccf-c7b262605968 -- ceph osd last-stat-seq osd.5 2024-08-19T01:33:31.969 INFO:teuthology.orchestra.run.smithi083.stdout:141733920823 2024-08-19T01:33:31.970 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833 shell --fsid bad8780e-5dc9-11ef-bccf-c7b262605968 -- ceph osd last-stat-seq osd.4 2024-08-19T01:33:32.015 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:30.907+0000 7f9926c00640 1 -- 172.21.15.83:0/541305610 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9920109ac0 msgr2=0x7f9920109ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:33:32.015 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:30.907+0000 7f9926c00640 1 --2- 172.21.15.83:0/541305610 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9920109ac0 0x7f9920109ea0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f9910009930 tx=0x7f991002a420 comp rx=0 tx=0).stop 2024-08-19T01:33:32.015 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:30.907+0000 7f9926c00640 1 -- 172.21.15.83:0/541305610 shutdown_connections 2024-08-19T01:33:32.015 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:30.907+0000 7f9926c00640 1 --2- 172.21.15.83:0/541305610 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9920104460 0x7f9920110990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:33:32.015 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:30.907+0000 7f9926c00640 1 --2- 172.21.15.83:0/541305610 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9920103ac0 0x7f9920103f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:33:32.015 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:30.907+0000 7f9926c00640 1 --2- 172.21.15.83:0/541305610 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9920109ac0 0x7f9920109ea0 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:33:32.015 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:30.907+0000 7f9926c00640 1 -- 172.21.15.83:0/541305610 >> 172.21.15.83:0/541305610 conn(0x7f99200ff7c0 msgr2=0x7f9920101be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:33:32.015 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:30.908+0000 7f9926c00640 1 -- 172.21.15.83:0/541305610 shutdown_connections 2024-08-19T01:33:32.015 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:30.908+0000 7f9926c00640 1 -- 172.21.15.83:0/541305610 wait complete. 2024-08-19T01:33:32.015 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:30.909+0000 7f9926c00640 1 Processor -- start 2024-08-19T01:33:32.015 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:30.909+0000 7f9926c00640 1 -- start start 2024-08-19T01:33:32.015 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:30.909+0000 7f9926c00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9920103ac0 0x7f992019a410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:33:32.016 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:30.909+0000 7f9926c00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9920104460 0x7f992019a950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:33:32.016 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:30.909+0000 7f9926c00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9920109ac0 0x7f992019ece0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:33:32.016 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:30.909+0000 7f9926c00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f9920113f10 con 0x7f9920109ac0 2024-08-19T01:33:32.016 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:30.909+0000 7f9926c00640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f9920113d90 con 0x7f9920103ac0 2024-08-19T01:33:32.016 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:30.909+0000 7f9926c00640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f9920114090 con 0x7f9920104460 2024-08-19T01:33:32.016 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:30.910+0000 7f9924c00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9920109ac0 0x7f992019ece0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:33:32.016 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:30.910+0000 7f9924c00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9920109ac0 0x7f992019ece0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:60400/0 (socket says 172.21.15.83:60400) 2024-08-19T01:33:32.016 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:30.910+0000 7f991fe00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9920103ac0 0x7f992019a410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:33:32.016 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:30.910+0000 7f9924c00640 1 -- 172.21.15.83:0/727606893 learned_addr learned my addr 172.21.15.83:0/727606893 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-08-19T01:33:32.016 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:30.910+0000 7f991f400640 1 --2- 172.21.15.83:0/727606893 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9920104460 0x7f992019a950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:33:32.016 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:30.910+0000 7f9924c00640 1 -- 172.21.15.83:0/727606893 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9920104460 msgr2=0x7f992019a950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:33:32.016 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:30.910+0000 7f9924c00640 1 --2- 172.21.15.83:0/727606893 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9920104460 0x7f992019a950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:33:32.016 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:30.910+0000 7f9924c00640 1 -- 172.21.15.83:0/727606893 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9920103ac0 msgr2=0x7f992019a410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:33:32.017 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:30.910+0000 7f9924c00640 1 --2- 172.21.15.83:0/727606893 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9920103ac0 0x7f992019a410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:33:32.017 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:30.910+0000 7f9924c00640 1 -- 172.21.15.83:0/727606893 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f992019f3c0 con 0x7f9920109ac0 2024-08-19T01:33:32.017 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:30.910+0000 7f991fe00640 1 --2- 172.21.15.83:0/727606893 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9920103ac0 0x7f992019a410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-19T01:33:32.017 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:30.910+0000 7f9924c00640 1 --2- 172.21.15.83:0/727606893 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9920109ac0 0x7f992019ece0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f9918009fd0 tx=0x7f991800b260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:33:32.017 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:30.911+0000 7f991cc00640 1 -- 172.21.15.83:0/727606893 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9918017070 con 0x7f9920109ac0 2024-08-19T01:33:32.017 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:30.911+0000 7f991cc00640 1 -- 172.21.15.83:0/727606893 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9918003e50 con 0x7f9920109ac0 2024-08-19T01:33:32.017 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:30.911+0000 7f9926c00640 1 -- 172.21.15.83:0/727606893 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f992019f6b0 con 0x7f9920109ac0 2024-08-19T01:33:32.017 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:30.911+0000 7f991cc00640 1 -- 172.21.15.83:0/727606893 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99180106c0 con 0x7f9920109ac0 2024-08-19T01:33:32.017 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:30.911+0000 7f9926c00640 1 -- 172.21.15.83:0/727606893 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f99201a6f90 con 0x7f9920109ac0 2024-08-19T01:33:32.017 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:30.912+0000 7f9926c00640 1 -- 172.21.15.83:0/727606893 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f99200626b0 con 0x7f9920109ac0 2024-08-19T01:33:32.017 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:30.914+0000 7f991cc00640 1 -- 172.21.15.83:0/727606893 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f9918003970 con 0x7f9920109ac0 2024-08-19T01:33:32.017 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:30.915+0000 7f991cc00640 1 --2- 172.21.15.83:0/727606893 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f99040a2c10 0x7f99040a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:33:32.017 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:30.915+0000 7f991cc00640 1 -- 172.21.15.83:0/727606893 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8269+0+0 (secure 0 0 0) 0x7f99180cf230 con 0x7f9920109ac0 2024-08-19T01:33:32.017 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:30.915+0000 7f991fe00640 1 --2- 172.21.15.83:0/727606893 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f99040a2c10 0x7f99040a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:33:32.017 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:30.916+0000 7f991cc00640 1 --2- 172.21.15.83:0/727606893 >> [v2:172.21.15.83:6802/1082526824,v1:172.21.15.83:6803/1082526824] conn(0x7f99040ac8c0 0x7f99040aed60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:33:32.018 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:30.916+0000 7f991cc00640 1 -- 172.21.15.83:0/727606893 --> [v2:172.21.15.83:6802/1082526824,v1:172.21.15.83:6803/1082526824] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f99040af430 con 0x7f99040ac8c0 2024-08-19T01:33:32.018 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:30.916+0000 7f991cc00640 1 -- 172.21.15.83:0/727606893 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_get_version_reply(handle=1 version=86) ==== 24+0+0 (secure 0 0 0) 0x7f99180cf680 con 0x7f9920109ac0 2024-08-19T01:33:32.018 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:30.916+0000 7f991f400640 1 --2- 172.21.15.83:0/727606893 >> [v2:172.21.15.83:6802/1082526824,v1:172.21.15.83:6803/1082526824] conn(0x7f99040ac8c0 0x7f99040aed60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:33:32.018 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:30.916+0000 7f991fe00640 1 --2- 172.21.15.83:0/727606893 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f99040a2c10 0x7f99040a50d0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f9910006010 tx=0x7f9910031510 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:33:32.018 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:30.917+0000 7f991f400640 1 --2- 172.21.15.83:0/727606893 >> [v2:172.21.15.83:6802/1082526824,v1:172.21.15.83:6803/1082526824] conn(0x7f99040ac8c0 0x7f99040aed60 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:33:32.018 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:30.918+0000 7f991cc00640 1 -- 172.21.15.83:0/727606893 <== osd.0 v2:172.21.15.83:6802/1082526824 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f99040af430 con 0x7f99040ac8c0 2024-08-19T01:33:32.018 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:30.934+0000 7f9926c00640 1 -- 172.21.15.83:0/727606893 --> [v2:172.21.15.83:6802/1082526824,v1:172.21.15.83:6803/1082526824] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f9920062b30 con 0x7f99040ac8c0 2024-08-19T01:33:32.019 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:30.935+0000 7f991cc00640 1 -- 172.21.15.83:0/727606893 <== osd.0 v2:172.21.15.83:6802/1082526824 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f9920062b30 con 0x7f99040ac8c0 2024-08-19T01:33:32.019 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:30.935+0000 7f9926c00640 1 -- 172.21.15.83:0/727606893 >> [v2:172.21.15.83:6802/1082526824,v1:172.21.15.83:6803/1082526824] conn(0x7f99040ac8c0 msgr2=0x7f99040aed60 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:33:32.019 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:30.935+0000 7f9926c00640 1 --2- 172.21.15.83:0/727606893 >> [v2:172.21.15.83:6802/1082526824,v1:172.21.15.83:6803/1082526824] conn(0x7f99040ac8c0 0x7f99040aed60 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:33:32.019 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:30.935+0000 7f9926c00640 1 -- 172.21.15.83:0/727606893 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f99040a2c10 msgr2=0x7f99040a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:33:32.019 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:30.935+0000 7f9926c00640 1 --2- 172.21.15.83:0/727606893 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f99040a2c10 0x7f99040a50d0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f9910006010 tx=0x7f9910031510 comp rx=0 tx=0).stop 2024-08-19T01:33:32.019 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:30.935+0000 7f9926c00640 1 -- 172.21.15.83:0/727606893 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9920109ac0 msgr2=0x7f992019ece0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:33:32.019 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:30.935+0000 7f9926c00640 1 --2- 172.21.15.83:0/727606893 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9920109ac0 0x7f992019ece0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f9918009fd0 tx=0x7f991800b260 comp rx=0 tx=0).stop 2024-08-19T01:33:32.019 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:30.935+0000 7f9924c00640 1 -- 172.21.15.83:0/727606893 reap_dead start 2024-08-19T01:33:32.019 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:30.935+0000 7f9926c00640 1 -- 172.21.15.83:0/727606893 shutdown_connections 2024-08-19T01:33:32.019 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:30.935+0000 7f9926c00640 1 -- 172.21.15.83:0/727606893 >> 172.21.15.83:0/727606893 conn(0x7f99200ff7c0 msgr2=0x7f9920100070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:33:32.019 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:30.935+0000 7f9926c00640 1 -- 172.21.15.83:0/727606893 shutdown_connections 2024-08-19T01:33:32.019 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:30.935+0000 7f9926c00640 1 -- 172.21.15.83:0/727606893 wait complete. 2024-08-19T01:33:32.349 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:31.029+0000 7f9843200640 1 -- 172.21.15.83:0/2101881284 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9844103280 msgr2=0x7f984410b770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:33:32.349 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:31.029+0000 7f9843200640 1 --2- 172.21.15.83:0/2101881284 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9844103280 0x7f984410b770 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f9830009900 tx=0x7f983002a480 comp rx=0 tx=0).stop 2024-08-19T01:33:32.350 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:31.029+0000 7f9843200640 1 -- 172.21.15.83:0/2101881284 shutdown_connections 2024-08-19T01:33:32.350 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:31.029+0000 7f9843200640 1 --2- 172.21.15.83:0/2101881284 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f984410bd00 0x7f984410e190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:33:32.350 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:31.029+0000 7f9843200640 1 --2- 172.21.15.83:0/2101881284 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9844103280 0x7f984410b770 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:33:32.350 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:31.029+0000 7f9843200640 1 --2- 172.21.15.83:0/2101881284 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9844102960 0x7f9844102d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:33:32.350 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:31.029+0000 7f9843200640 1 -- 172.21.15.83:0/2101881284 >> 172.21.15.83:0/2101881284 conn(0x7f98440fc340 msgr2=0x7f98440fe760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:33:32.350 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:31.029+0000 7f9843200640 1 -- 172.21.15.83:0/2101881284 shutdown_connections 2024-08-19T01:33:32.350 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:31.029+0000 7f9843200640 1 -- 172.21.15.83:0/2101881284 wait complete. 2024-08-19T01:33:32.350 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:31.030+0000 7f9843200640 1 Processor -- start 2024-08-19T01:33:32.350 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:31.030+0000 7f9843200640 1 -- start start 2024-08-19T01:33:32.350 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:31.031+0000 7f9843200640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9844102960 0x7f9844197fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:33:32.350 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:31.031+0000 7f9843200640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9844103280 0x7f9844198510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:33:32.350 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:31.031+0000 7f9843200640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f984410bd00 0x7f984419c8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:33:32.350 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:31.031+0000 7f9843200640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f9844111ab0 con 0x7f984410bd00 2024-08-19T01:33:32.351 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:31.031+0000 7f9843200640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f9844111930 con 0x7f9844103280 2024-08-19T01:33:32.351 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:31.031+0000 7f9843200640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f9844111c30 con 0x7f9844102960 2024-08-19T01:33:32.351 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:31.031+0000 7f9842800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f984410bd00 0x7f984419c8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:33:32.351 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:31.031+0000 7f9842800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f984410bd00 0x7f984419c8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:60426/0 (socket says 172.21.15.83:60426) 2024-08-19T01:33:32.351 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:31.031+0000 7f9842800640 1 -- 172.21.15.83:0/502630633 learned_addr learned my addr 172.21.15.83:0/502630633 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-08-19T01:33:32.351 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:31.031+0000 7f9841400640 1 --2- 172.21.15.83:0/502630633 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9844103280 0x7f9844198510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:33:32.351 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:31.031+0000 7f9841e00640 1 --2- 172.21.15.83:0/502630633 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9844102960 0x7f9844197fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:33:32.351 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:31.031+0000 7f9842800640 1 -- 172.21.15.83:0/502630633 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9844102960 msgr2=0x7f9844197fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:33:32.351 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:31.031+0000 7f9842800640 1 --2- 172.21.15.83:0/502630633 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9844102960 0x7f9844197fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:33:32.351 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:31.031+0000 7f9842800640 1 -- 172.21.15.83:0/502630633 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9844103280 msgr2=0x7f9844198510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:33:32.351 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:31.031+0000 7f9842800640 1 --2- 172.21.15.83:0/502630633 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9844103280 0x7f9844198510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:33:32.351 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:31.031+0000 7f9842800640 1 -- 172.21.15.83:0/502630633 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f984419cf80 con 0x7f984410bd00 2024-08-19T01:33:32.351 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:31.032+0000 7f9842800640 1 --2- 172.21.15.83:0/502630633 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f984410bd00 0x7f984419c8a0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f983c00efd0 tx=0x7f983c00c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:33:32.352 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:31.032+0000 7f983aa00640 1 -- 172.21.15.83:0/502630633 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f983c003b70 con 0x7f984410bd00 2024-08-19T01:33:32.352 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:31.032+0000 7f983aa00640 1 -- 172.21.15.83:0/502630633 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f983c003d10 con 0x7f984410bd00 2024-08-19T01:33:32.352 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:31.032+0000 7f9843200640 1 -- 172.21.15.83:0/502630633 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f984419d270 con 0x7f984410bd00 2024-08-19T01:33:32.352 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:31.032+0000 7f983aa00640 1 -- 172.21.15.83:0/502630633 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f983c0109f0 con 0x7f984410bd00 2024-08-19T01:33:32.352 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:31.032+0000 7f9843200640 1 -- 172.21.15.83:0/502630633 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f98441a4b50 con 0x7f984410bd00 2024-08-19T01:33:32.352 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:31.033+0000 7f9843200640 1 -- 172.21.15.83:0/502630633 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f9844062670 con 0x7f984410bd00 2024-08-19T01:33:32.352 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:31.035+0000 7f983aa00640 1 -- 172.21.15.83:0/502630633 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f983c010b90 con 0x7f984410bd00 2024-08-19T01:33:32.352 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:31.036+0000 7f983aa00640 1 --2- 172.21.15.83:0/502630633 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f98280a2c90 0x7f98280a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:33:32.352 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:31.036+0000 7f983aa00640 1 -- 172.21.15.83:0/502630633 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8269+0+0 (secure 0 0 0) 0x7f983c0cf8f0 con 0x7f984410bd00 2024-08-19T01:33:32.352 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:31.036+0000 7f9841e00640 1 --2- 172.21.15.83:0/502630633 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f98280a2c90 0x7f98280a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:33:32.353 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:31.037+0000 7f983aa00640 1 --2- 172.21.15.83:0/502630633 >> [v2:172.21.15.117:6800/1925419802,v1:172.21.15.117:6801/1925419802] conn(0x7f98280ac940 0x7f98280aede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:33:32.353 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:31.037+0000 7f983aa00640 1 -- 172.21.15.83:0/502630633 --> [v2:172.21.15.117:6800/1925419802,v1:172.21.15.117:6801/1925419802] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f983c009820 con 0x7f98280ac940 2024-08-19T01:33:32.353 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:31.037+0000 7f983aa00640 1 -- 172.21.15.83:0/502630633 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_get_version_reply(handle=1 version=86) ==== 24+0+0 (secure 0 0 0) 0x7f983c0cfd40 con 0x7f984410bd00 2024-08-19T01:33:32.353 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:31.037+0000 7f9841e00640 1 --2- 172.21.15.83:0/502630633 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f98280a2c90 0x7f98280a5150 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f9834009bd0 tx=0x7f9834005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:33:32.353 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:31.037+0000 7f9841400640 1 --2- 172.21.15.83:0/502630633 >> [v2:172.21.15.117:6800/1925419802,v1:172.21.15.117:6801/1925419802] conn(0x7f98280ac940 0x7f98280aede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:33:32.353 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:31.038+0000 7f9841400640 1 --2- 172.21.15.83:0/502630633 >> [v2:172.21.15.117:6800/1925419802,v1:172.21.15.117:6801/1925419802] conn(0x7f98280ac940 0x7f98280aede0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:33:32.353 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:31.038+0000 7f983aa00640 1 -- 172.21.15.83:0/502630633 <== osd.1 v2:172.21.15.117:6800/1925419802 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f983c009820 con 0x7f98280ac940 2024-08-19T01:33:32.353 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:31.052+0000 7f9843200640 1 -- 172.21.15.83:0/502630633 --> [v2:172.21.15.117:6800/1925419802,v1:172.21.15.117:6801/1925419802] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f9844062af0 con 0x7f98280ac940 2024-08-19T01:33:32.354 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:31.053+0000 7f983aa00640 1 -- 172.21.15.83:0/502630633 <== osd.1 v2:172.21.15.117:6800/1925419802 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f9844062af0 con 0x7f98280ac940 2024-08-19T01:33:32.354 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:31.053+0000 7f9843200640 1 -- 172.21.15.83:0/502630633 >> [v2:172.21.15.117:6800/1925419802,v1:172.21.15.117:6801/1925419802] conn(0x7f98280ac940 msgr2=0x7f98280aede0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:33:32.354 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:31.053+0000 7f9843200640 1 --2- 172.21.15.83:0/502630633 >> [v2:172.21.15.117:6800/1925419802,v1:172.21.15.117:6801/1925419802] conn(0x7f98280ac940 0x7f98280aede0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:33:32.354 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:31.054+0000 7f9843200640 1 -- 172.21.15.83:0/502630633 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f98280a2c90 msgr2=0x7f98280a5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:33:32.354 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:31.054+0000 7f9843200640 1 --2- 172.21.15.83:0/502630633 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f98280a2c90 0x7f98280a5150 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f9834009bd0 tx=0x7f9834005f90 comp rx=0 tx=0).stop 2024-08-19T01:33:32.354 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:31.054+0000 7f9843200640 1 -- 172.21.15.83:0/502630633 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f984410bd00 msgr2=0x7f984419c8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:33:32.354 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:31.054+0000 7f9843200640 1 --2- 172.21.15.83:0/502630633 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f984410bd00 0x7f984419c8a0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f983c00efd0 tx=0x7f983c00c6a0 comp rx=0 tx=0).stop 2024-08-19T01:33:32.354 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:31.054+0000 7f9842800640 1 -- 172.21.15.83:0/502630633 reap_dead start 2024-08-19T01:33:32.354 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:31.054+0000 7f9843200640 1 -- 172.21.15.83:0/502630633 shutdown_connections 2024-08-19T01:33:32.354 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:31.054+0000 7f9843200640 1 -- 172.21.15.83:0/502630633 >> 172.21.15.83:0/502630633 conn(0x7f98440fc340 msgr2=0x7f98440fdc00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:33:32.354 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:31.054+0000 7f9843200640 1 -- 172.21.15.83:0/502630633 shutdown_connections 2024-08-19T01:33:32.354 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:31.054+0000 7f9843200640 1 -- 172.21.15.83:0/502630633 wait complete. 2024-08-19T01:33:32.814 INFO:teuthology.orchestra.run.smithi083.stdout:197568495662 2024-08-19T01:33:32.814 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833 shell --fsid bad8780e-5dc9-11ef-bccf-c7b262605968 -- ceph osd last-stat-seq osd.6 2024-08-19T01:33:33.584 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:33.581+0000 7fa006a00640 1 -- 172.21.15.83:0/3366644900 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa000109b70 msgr2=0x7fa000109f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:33:33.584 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:33.581+0000 7fa006a00640 1 --2- 172.21.15.83:0/3366644900 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa000109b70 0x7fa000109f50 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f9ff0009930 tx=0x7f9ff002a420 comp rx=0 tx=0).stop 2024-08-19T01:33:33.584 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:33.581+0000 7fa006a00640 1 -- 172.21.15.83:0/3366644900 shutdown_connections 2024-08-19T01:33:33.584 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:33.581+0000 7fa006a00640 1 --2- 172.21.15.83:0/3366644900 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa000104480 0x7fa0001109b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:33:33.584 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:33.581+0000 7fa006a00640 1 --2- 172.21.15.83:0/3366644900 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa000103ae0 0x7fa000103f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:33:33.585 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:33.581+0000 7fa006a00640 1 --2- 172.21.15.83:0/3366644900 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa000109b70 0x7fa000109f50 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:33:33.585 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:33.581+0000 7fa006a00640 1 -- 172.21.15.83:0/3366644900 >> 172.21.15.83:0/3366644900 conn(0x7fa0000ff780 msgr2=0x7fa000101ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:33:33.585 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:33.581+0000 7fa006a00640 1 -- 172.21.15.83:0/3366644900 shutdown_connections 2024-08-19T01:33:33.585 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:33.581+0000 7fa006a00640 1 -- 172.21.15.83:0/3366644900 wait complete. 2024-08-19T01:33:33.585 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:33.582+0000 7fa006a00640 1 Processor -- start 2024-08-19T01:33:33.585 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:33.582+0000 7fa006a00640 1 -- start start 2024-08-19T01:33:33.587 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:33.583+0000 7fa006a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa000103ae0 0x7fa00019a3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:33:33.587 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:33.583+0000 7fa006a00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa000104480 0x7fa00019a920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:33:33.587 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:33.583+0000 7fa006a00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa000109b70 0x7fa00019ecb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:33:33.587 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:33.583+0000 7fa006a00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fa000113ee0 con 0x7fa000103ae0 2024-08-19T01:33:33.587 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:33.583+0000 7fa006a00640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fa000113d60 con 0x7fa000109b70 2024-08-19T01:33:33.588 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:33.583+0000 7fa006a00640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fa000114060 con 0x7fa000104480 2024-08-19T01:33:33.588 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:33.583+0000 7f9fffe00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa000103ae0 0x7fa00019a3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:33:33.588 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:33.583+0000 7f9fffe00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa000103ae0 0x7fa00019a3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:60448/0 (socket says 172.21.15.83:60448) 2024-08-19T01:33:33.588 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:33.583+0000 7f9fffe00640 1 -- 172.21.15.83:0/148405899 learned_addr learned my addr 172.21.15.83:0/148405899 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-08-19T01:33:33.588 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:33.583+0000 7f9fff400640 1 --2- 172.21.15.83:0/148405899 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa000104480 0x7fa00019a920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:33:33.588 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:33.583+0000 7fa004a00640 1 --2- 172.21.15.83:0/148405899 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa000109b70 0x7fa00019ecb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:33:33.588 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:33.584+0000 7f9fff400640 1 -- 172.21.15.83:0/148405899 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa000109b70 msgr2=0x7fa00019ecb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:33:33.588 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:33.584+0000 7f9fff400640 1 --2- 172.21.15.83:0/148405899 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa000109b70 0x7fa00019ecb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:33:33.589 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:33.584+0000 7f9fff400640 1 -- 172.21.15.83:0/148405899 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa000103ae0 msgr2=0x7fa00019a3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:33:33.589 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:33.584+0000 7f9fff400640 1 --2- 172.21.15.83:0/148405899 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa000103ae0 0x7fa00019a3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:33:33.589 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:33.584+0000 7f9fff400640 1 -- 172.21.15.83:0/148405899 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa00019f390 con 0x7fa000104480 2024-08-19T01:33:33.589 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:33.584+0000 7fa004a00640 1 --2- 172.21.15.83:0/148405899 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa000109b70 0x7fa00019ecb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-19T01:33:33.589 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:33.584+0000 7f9fff400640 1 --2- 172.21.15.83:0/148405899 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa000104480 0x7fa00019a920 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f9fec00d720 tx=0x7f9fec00dbe0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:33:33.589 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:33.584+0000 7f9ffcc00640 1 -- 172.21.15.83:0/148405899 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9fec017070 con 0x7fa000104480 2024-08-19T01:33:33.589 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:33.584+0000 7fa006a00640 1 -- 172.21.15.83:0/148405899 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa00019f680 con 0x7fa000104480 2024-08-19T01:33:33.589 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:33.584+0000 7f9ffcc00640 1 -- 172.21.15.83:0/148405899 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9fec00de30 con 0x7fa000104480 2024-08-19T01:33:33.589 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:33.584+0000 7f9ffcc00640 1 -- 172.21.15.83:0/148405899 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9fec0047a0 con 0x7fa000104480 2024-08-19T01:33:33.590 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:33.584+0000 7fa006a00640 1 -- 172.21.15.83:0/148405899 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa0001a6f60 con 0x7fa000104480 2024-08-19T01:33:33.590 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:33.585+0000 7fa006a00640 1 -- 172.21.15.83:0/148405899 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fa000062670 con 0x7fa000104480 2024-08-19T01:33:33.590 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:33.588+0000 7f9ffcc00640 1 -- 172.21.15.83:0/148405899 <== mon.1 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f9fec0099d0 con 0x7fa000104480 2024-08-19T01:33:33.591 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:33.589+0000 7f9ffcc00640 1 --2- 172.21.15.83:0/148405899 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f9fe40a2c10 0x7f9fe40a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:33:33.591 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:33.589+0000 7f9ffcc00640 1 -- 172.21.15.83:0/148405899 <== mon.1 v2:172.21.15.167:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8269+0+0 (secure 0 0 0) 0x7f9fec0cf440 con 0x7fa000104480 2024-08-19T01:33:33.591 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:33.589+0000 7f9fffe00640 1 --2- 172.21.15.83:0/148405899 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f9fe40a2c10 0x7f9fe40a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:33:33.591 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:33.590+0000 7f9ffcc00640 1 --2- 172.21.15.83:0/148405899 >> [v2:172.21.15.167:6816/2196898857,v1:172.21.15.167:6817/2196898857] conn(0x7f9fe40ac8c0 0x7f9fe40aed60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:33:33.591 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:33.590+0000 7f9ffcc00640 1 -- 172.21.15.83:0/148405899 --> [v2:172.21.15.167:6816/2196898857,v1:172.21.15.167:6817/2196898857] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f9fe40af430 con 0x7f9fe40ac8c0 2024-08-19T01:33:33.592 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:33.590+0000 7f9ffcc00640 1 -- 172.21.15.83:0/148405899 <== mon.1 v2:172.21.15.167:3300/0 6 ==== mon_get_version_reply(handle=1 version=86) ==== 24+0+0 (secure 0 0 0) 0x7f9fec0cf7f0 con 0x7fa000104480 2024-08-19T01:33:33.592 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:33.590+0000 7fa004a00640 1 --2- 172.21.15.83:0/148405899 >> [v2:172.21.15.167:6816/2196898857,v1:172.21.15.167:6817/2196898857] conn(0x7f9fe40ac8c0 0x7f9fe40aed60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:33:33.592 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:33.590+0000 7f9fffe00640 1 --2- 172.21.15.83:0/148405899 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f9fe40a2c10 0x7f9fe40a50d0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f9ff0000c00 tx=0x7f9ff0009560 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:33:33.593 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:33.591+0000 7fa004a00640 1 --2- 172.21.15.83:0/148405899 >> [v2:172.21.15.167:6816/2196898857,v1:172.21.15.167:6817/2196898857] conn(0x7f9fe40ac8c0 0x7f9fe40aed60 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:33:33.593 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:33.591+0000 7f9ffcc00640 1 -- 172.21.15.83:0/148405899 <== osd.8 v2:172.21.15.167:6816/2196898857 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f9fe40af430 con 0x7f9fe40ac8c0 2024-08-19T01:33:33.607 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:33.606+0000 7fa006a00640 1 -- 172.21.15.83:0/148405899 --> [v2:172.21.15.167:6816/2196898857,v1:172.21.15.167:6817/2196898857] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fa000062af0 con 0x7f9fe40ac8c0 2024-08-19T01:33:33.608 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:33.606+0000 7f9ffcc00640 1 -- 172.21.15.83:0/148405899 <== osd.8 v2:172.21.15.167:6816/2196898857 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7fa000062af0 con 0x7f9fe40ac8c0 2024-08-19T01:33:33.609 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:33.607+0000 7fa006a00640 1 -- 172.21.15.83:0/148405899 >> [v2:172.21.15.167:6816/2196898857,v1:172.21.15.167:6817/2196898857] conn(0x7f9fe40ac8c0 msgr2=0x7f9fe40aed60 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:33:33.609 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:33.607+0000 7fa006a00640 1 --2- 172.21.15.83:0/148405899 >> [v2:172.21.15.167:6816/2196898857,v1:172.21.15.167:6817/2196898857] conn(0x7f9fe40ac8c0 0x7f9fe40aed60 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:33:33.609 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:33.607+0000 7fa006a00640 1 -- 172.21.15.83:0/148405899 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f9fe40a2c10 msgr2=0x7f9fe40a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:33:33.609 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:33.607+0000 7fa006a00640 1 --2- 172.21.15.83:0/148405899 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f9fe40a2c10 0x7f9fe40a50d0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f9ff0000c00 tx=0x7f9ff0009560 comp rx=0 tx=0).stop 2024-08-19T01:33:33.609 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:33.607+0000 7fa006a00640 1 -- 172.21.15.83:0/148405899 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa000104480 msgr2=0x7fa00019a920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:33:33.609 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:33.607+0000 7fa006a00640 1 --2- 172.21.15.83:0/148405899 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa000104480 0x7fa00019a920 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f9fec00d720 tx=0x7f9fec00dbe0 comp rx=0 tx=0).stop 2024-08-19T01:33:33.609 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:33.607+0000 7fa004a00640 1 -- 172.21.15.83:0/148405899 reap_dead start 2024-08-19T01:33:33.610 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:33.607+0000 7fa006a00640 1 -- 172.21.15.83:0/148405899 shutdown_connections 2024-08-19T01:33:33.610 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:33.607+0000 7fa006a00640 1 -- 172.21.15.83:0/148405899 >> 172.21.15.83:0/148405899 conn(0x7fa0000ff780 msgr2=0x7fa000100020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:33:33.610 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:33.607+0000 7fa006a00640 1 -- 172.21.15.83:0/148405899 shutdown_connections 2024-08-19T01:33:33.610 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:33.607+0000 7fa006a00640 1 -- 172.21.15.83:0/148405899 wait complete. 2024-08-19T01:33:33.702 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:33:33 smithi117 ceph-mon[22016]: pgmap v300: 97 pgs: 97 active+clean; 640 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-08-19T01:33:33.715 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:33:33 smithi167 ceph-mon[21875]: pgmap v300: 97 pgs: 97 active+clean; 640 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-08-19T01:33:33.800 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:33:33 smithi083 ceph-mon[17120]: pgmap v300: 97 pgs: 97 active+clean; 640 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-08-19T01:33:34.176 INFO:teuthology.orchestra.run.smithi083.stdout:335007449110 2024-08-19T01:33:34.176 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833 shell --fsid bad8780e-5dc9-11ef-bccf-c7b262605968 -- ceph osd last-stat-seq osd.11 2024-08-19T01:33:34.448 INFO:teuthology.orchestra.run.smithi083.stdout:55834574917 2024-08-19T01:33:34.448 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833 shell --fsid bad8780e-5dc9-11ef-bccf-c7b262605968 -- ceph osd last-stat-seq osd.1 2024-08-19T01:33:35.354 INFO:teuthology.orchestra.run.smithi083.stdout:34359738443 2024-08-19T01:33:35.354 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833 shell --fsid bad8780e-5dc9-11ef-bccf-c7b262605968 -- ceph osd last-stat-seq osd.0 2024-08-19T01:33:35.654 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:33:35 smithi083 ceph-mon[17120]: pgmap v301: 97 pgs: 97 active+clean; 640 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 KiB/s rd, 5 op/s 2024-08-19T01:33:35.702 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:33:35 smithi117 ceph-mon[22016]: pgmap v301: 97 pgs: 97 active+clean; 640 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 KiB/s rd, 5 op/s 2024-08-19T01:33:35.716 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:33:35 smithi167 ceph-mon[21875]: pgmap v301: 97 pgs: 97 active+clean; 640 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 KiB/s rd, 5 op/s 2024-08-19T01:33:35.849 INFO:teuthology.orchestra.run.smithi083.stdout:77309411394 2024-08-19T01:33:35.850 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833 shell --fsid bad8780e-5dc9-11ef-bccf-c7b262605968 -- ceph osd last-stat-seq osd.2 2024-08-19T01:33:36.140 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/bad8780e-5dc9-11ef-bccf-c7b262605968/mon.a/config 2024-08-19T01:33:36.190 INFO:teuthology.orchestra.run.smithi083.stdout:249108103206 2024-08-19T01:33:36.190 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833 shell --fsid bad8780e-5dc9-11ef-bccf-c7b262605968 -- ceph osd last-stat-seq osd.8 2024-08-19T01:33:36.340 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/bad8780e-5dc9-11ef-bccf-c7b262605968/mon.a/config 2024-08-19T01:33:36.353 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/bad8780e-5dc9-11ef-bccf-c7b262605968/mon.a/config 2024-08-19T01:33:36.353 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/bad8780e-5dc9-11ef-bccf-c7b262605968/mon.a/config 2024-08-19T01:33:36.359 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/bad8780e-5dc9-11ef-bccf-c7b262605968/mon.a/config 2024-08-19T01:33:36.361 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/bad8780e-5dc9-11ef-bccf-c7b262605968/mon.a/config 2024-08-19T01:33:36.366 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/bad8780e-5dc9-11ef-bccf-c7b262605968/mon.a/config 2024-08-19T01:33:36.594 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/bad8780e-5dc9-11ef-bccf-c7b262605968/mon.a/config 2024-08-19T01:33:36.608 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/bad8780e-5dc9-11ef-bccf-c7b262605968/mon.a/config 2024-08-19T01:33:37.205 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:37.203+0000 7f0977200640 1 -- 172.21.15.83:0/3819628904 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f09781076f0 msgr2=0x7f0978107ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:33:37.206 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:37.203+0000 7f0977200640 1 --2- 172.21.15.83:0/3819628904 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f09781076f0 0x7f0978107ad0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f0968009930 tx=0x7f096802a420 comp rx=0 tx=0).stop 2024-08-19T01:33:37.206 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:37.203+0000 7f0977200640 1 -- 172.21.15.83:0/3819628904 shutdown_connections 2024-08-19T01:33:37.206 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:37.203+0000 7f0977200640 1 --2- 172.21.15.83:0/3819628904 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0978102090 0x7f097810e5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:33:37.206 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:37.203+0000 7f0977200640 1 --2- 172.21.15.83:0/3819628904 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f09781016f0 0x7f0978101b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:33:37.206 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:37.203+0000 7f0977200640 1 --2- 172.21.15.83:0/3819628904 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f09781076f0 0x7f0978107ad0 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:33:37.206 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:37.203+0000 7f0977200640 1 -- 172.21.15.83:0/3819628904 >> 172.21.15.83:0/3819628904 conn(0x7f09780fd470 msgr2=0x7f09780ff890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:33:37.206 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:37.203+0000 7f0977200640 1 -- 172.21.15.83:0/3819628904 shutdown_connections 2024-08-19T01:33:37.206 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:37.203+0000 7f0977200640 1 -- 172.21.15.83:0/3819628904 wait complete. 2024-08-19T01:33:37.207 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:37.204+0000 7f0977200640 1 Processor -- start 2024-08-19T01:33:37.209 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:37.205+0000 7f0977200640 1 -- start start 2024-08-19T01:33:37.209 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:37.205+0000 7f0977200640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f09781016f0 0x7f0978198020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:33:37.209 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:37.205+0000 7f0977200640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0978102090 0x7f0978198560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:33:37.209 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:37.205+0000 7f0977200640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f09781076f0 0x7f097819c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:33:37.209 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:37.205+0000 7f0977200640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f0978111b00 con 0x7f09781016f0 2024-08-19T01:33:37.209 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:37.205+0000 7f0977200640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f0978111980 con 0x7f0978102090 2024-08-19T01:33:37.209 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:37.205+0000 7f0977200640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f0978111c80 con 0x7f09781076f0 2024-08-19T01:33:37.210 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:37.206+0000 7f0975e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f09781016f0 0x7f0978198020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:33:37.210 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:37.206+0000 7f0976800640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f09781076f0 0x7f097819c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:33:37.210 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:37.206+0000 7f0975e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f09781016f0 0x7f0978198020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:60472/0 (socket says 172.21.15.83:60472) 2024-08-19T01:33:37.210 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:37.206+0000 7f0975e00640 1 -- 172.21.15.83:0/3398814825 learned_addr learned my addr 172.21.15.83:0/3398814825 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-08-19T01:33:37.210 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:37.206+0000 7f0975400640 1 --2- 172.21.15.83:0/3398814825 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0978102090 0x7f0978198560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:33:37.210 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:37.207+0000 7f0975e00640 1 -- 172.21.15.83:0/3398814825 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f09781076f0 msgr2=0x7f097819c8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:33:37.210 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:37.207+0000 7f0975e00640 1 --2- 172.21.15.83:0/3398814825 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f09781076f0 0x7f097819c8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:33:37.210 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:37.207+0000 7f0975e00640 1 -- 172.21.15.83:0/3398814825 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0978102090 msgr2=0x7f0978198560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:33:37.210 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:37.207+0000 7f0975e00640 1 --2- 172.21.15.83:0/3398814825 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0978102090 0x7f0978198560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:33:37.210 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:37.207+0000 7f0975e00640 1 -- 172.21.15.83:0/3398814825 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f097819cfd0 con 0x7f09781016f0 2024-08-19T01:33:37.211 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:37.207+0000 7f0975e00640 1 --2- 172.21.15.83:0/3398814825 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f09781016f0 0x7f0978198020 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f096802a940 tx=0x7f0968003b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:33:37.211 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:37.208+0000 7f096ea00640 1 -- 172.21.15.83:0/3398814825 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f096803f070 con 0x7f09781016f0 2024-08-19T01:33:37.211 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:37.208+0000 7f096ea00640 1 -- 172.21.15.83:0/3398814825 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0968031c90 con 0x7f09781016f0 2024-08-19T01:33:37.211 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:37.208+0000 7f096ea00640 1 -- 172.21.15.83:0/3398814825 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0968035030 con 0x7f09781016f0 2024-08-19T01:33:37.211 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:37.208+0000 7f0977200640 1 -- 172.21.15.83:0/3398814825 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f097819d260 con 0x7f09781016f0 2024-08-19T01:33:37.211 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:37.208+0000 7f0977200640 1 -- 172.21.15.83:0/3398814825 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f097819d6f0 con 0x7f09781016f0 2024-08-19T01:33:37.213 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:37.210+0000 7f0977200640 1 -- 172.21.15.83:0/3398814825 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0978101b50 con 0x7f09781016f0 2024-08-19T01:33:37.215 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:37.214+0000 7f096ea00640 1 -- 172.21.15.83:0/3398814825 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f0968042070 con 0x7f09781016f0 2024-08-19T01:33:37.218 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:37.216+0000 7f096ea00640 1 --2- 172.21.15.83:0/3398814825 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f095c0a2d60 0x7f095c0a5220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:33:37.218 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:37.216+0000 7f096ea00640 1 -- 172.21.15.83:0/3398814825 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8269+0+0 (secure 0 0 0) 0x7f09680effa0 con 0x7f09781016f0 2024-08-19T01:33:37.219 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:37.217+0000 7f0975400640 1 --2- 172.21.15.83:0/3398814825 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f095c0a2d60 0x7f095c0a5220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:33:37.220 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:37.218+0000 7f096ea00640 1 -- 172.21.15.83:0/3398814825 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7f0968049030 con 0x7f09781016f0 2024-08-19T01:33:37.220 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:37.219+0000 7f0975400640 1 --2- 172.21.15.83:0/3398814825 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f095c0a2d60 0x7f095c0a5220 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f09640095f0 tx=0x7f0964005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:33:37.415 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:37.413+0000 7f0977200640 1 -- 172.21.15.83:0/3398814825 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 10} v 0) -- 0x7f09781096d0 con 0x7f09781016f0 2024-08-19T01:33:37.415 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:37.414+0000 7f096ea00640 1 -- 172.21.15.83:0/3398814825 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 10}]=0 v0) ==== 75+0+13 (secure 0 0 0) 0x7f0968042360 con 0x7f09781016f0 2024-08-19T01:33:37.416 INFO:teuthology.orchestra.run.smithi083.stdout:304942678045 2024-08-19T01:33:37.418 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:37.416+0000 7f0977200640 1 -- 172.21.15.83:0/3398814825 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f095c0a2d60 msgr2=0x7f095c0a5220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:33:37.418 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:37.416+0000 7f0977200640 1 --2- 172.21.15.83:0/3398814825 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f095c0a2d60 0x7f095c0a5220 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f09640095f0 tx=0x7f0964005f90 comp rx=0 tx=0).stop 2024-08-19T01:33:37.418 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:37.416+0000 7f0977200640 1 -- 172.21.15.83:0/3398814825 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f09781016f0 msgr2=0x7f0978198020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:33:37.418 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:37.416+0000 7f0977200640 1 --2- 172.21.15.83:0/3398814825 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f09781016f0 0x7f0978198020 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f096802a940 tx=0x7f0968003b20 comp rx=0 tx=0).stop 2024-08-19T01:33:37.419 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:37.417+0000 7f0977200640 1 -- 172.21.15.83:0/3398814825 shutdown_connections 2024-08-19T01:33:37.419 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:37.417+0000 7f0977200640 1 --2- 172.21.15.83:0/3398814825 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f095c0a2d60 0x7f095c0a5220 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:33:37.419 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:37.417+0000 7f0977200640 1 --2- 172.21.15.83:0/3398814825 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f09781076f0 0x7f097819c8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:33:37.419 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:37.417+0000 7f0977200640 1 --2- 172.21.15.83:0/3398814825 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0978102090 0x7f0978198560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:33:37.419 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:37.417+0000 7f0977200640 1 --2- 172.21.15.83:0/3398814825 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f09781016f0 0x7f0978198020 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:33:37.419 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:37.417+0000 7f0977200640 1 -- 172.21.15.83:0/3398814825 >> 172.21.15.83:0/3398814825 conn(0x7f09780fd470 msgr2=0x7f09780fdce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:33:37.419 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:37.417+0000 7f0977200640 1 -- 172.21.15.83:0/3398814825 shutdown_connections 2024-08-19T01:33:37.419 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:37.417+0000 7f0977200640 1 -- 172.21.15.83:0/3398814825 wait complete. 2024-08-19T01:33:37.467 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:33:37 smithi083 ceph-mon[17120]: pgmap v302: 97 pgs: 97 active+clean; 640 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 6 op/s 2024-08-19T01:33:37.467 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:33:37 smithi083 ceph-mon[17120]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-19T01:33:37.467 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:33:37 smithi083 ceph-mon[17120]: from='client.? 172.21.15.83:0/3124923670' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-08-19T01:33:37.467 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:33:37 smithi083 ceph-mon[17120]: from='client.? 172.21.15.83:0/3398814825' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-08-19T01:33:37.698 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:33:37 smithi117 ceph-mon[22016]: pgmap v302: 97 pgs: 97 active+clean; 640 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 6 op/s 2024-08-19T01:33:37.698 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:33:37 smithi117 ceph-mon[22016]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-19T01:33:37.699 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:33:37 smithi117 ceph-mon[22016]: from='client.? 172.21.15.83:0/3124923670' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-08-19T01:33:37.699 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:33:37 smithi117 ceph-mon[22016]: from='client.? 172.21.15.83:0/3398814825' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-08-19T01:33:37.699 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:33:37 smithi167 ceph-mon[21875]: pgmap v302: 97 pgs: 97 active+clean; 640 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 6 op/s 2024-08-19T01:33:37.699 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:33:37 smithi167 ceph-mon[21875]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-19T01:33:37.699 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:33:37 smithi167 ceph-mon[21875]: from='client.? 172.21.15.83:0/3124923670' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-08-19T01:33:37.699 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:33:37 smithi167 ceph-mon[21875]: from='client.? 172.21.15.83:0/3398814825' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-08-19T01:33:38.341 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:37.128+0000 7fbc39000640 1 -- 172.21.15.83:0/1049150619 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbc34074350 msgr2=0x7fbc34074730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:33:38.341 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:37.128+0000 7fbc39000640 1 --2- 172.21.15.83:0/1049150619 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbc34074350 0x7fbc34074730 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fbc24009900 tx=0x7fbc2402a480 comp rx=0 tx=0).stop 2024-08-19T01:33:38.341 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:37.128+0000 7fbc39000640 1 -- 172.21.15.83:0/1049150619 shutdown_connections 2024-08-19T01:33:38.341 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:37.128+0000 7fbc39000640 1 --2- 172.21.15.83:0/1049150619 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbc3410e0e0 0x7fbc34110530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:33:38.341 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:37.128+0000 7fbc39000640 1 --2- 172.21.15.83:0/1049150619 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbc34074c70 0x7fbc3410dba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:33:38.341 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:37.128+0000 7fbc39000640 1 --2- 172.21.15.83:0/1049150619 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbc34074350 0x7fbc34074730 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:33:38.341 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:37.128+0000 7fbc39000640 1 -- 172.21.15.83:0/1049150619 >> 172.21.15.83:0/1049150619 conn(0x7fbc340fd5c0 msgr2=0x7fbc340ff9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:33:38.341 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:37.128+0000 7fbc39000640 1 -- 172.21.15.83:0/1049150619 shutdown_connections 2024-08-19T01:33:38.342 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:37.128+0000 7fbc39000640 1 -- 172.21.15.83:0/1049150619 wait complete. 2024-08-19T01:33:38.342 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:37.129+0000 7fbc39000640 1 Processor -- start 2024-08-19T01:33:38.342 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:37.129+0000 7fbc39000640 1 -- start start 2024-08-19T01:33:38.342 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:37.130+0000 7fbc39000640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbc34074350 0x7fbc3419a3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:33:38.342 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:37.130+0000 7fbc39000640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbc34074c70 0x7fbc3419a910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:33:38.342 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:37.130+0000 7fbc39000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbc3410e0e0 0x7fbc3419eca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:33:38.342 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:37.130+0000 7fbc39000640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fbc34113e50 con 0x7fbc3410e0e0 2024-08-19T01:33:38.342 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:37.130+0000 7fbc39000640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fbc34113cd0 con 0x7fbc34074350 2024-08-19T01:33:38.342 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:37.130+0000 7fbc39000640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fbc34113fd0 con 0x7fbc34074c70 2024-08-19T01:33:38.342 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:37.130+0000 7fbc31400640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbc34074c70 0x7fbc3419a910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:33:38.342 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:37.130+0000 7fbc31400640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbc34074c70 0x7fbc3419a910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.83:37502/0 (socket says 172.21.15.83:37502) 2024-08-19T01:33:38.342 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:37.130+0000 7fbc31e00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbc34074350 0x7fbc3419a3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:33:38.343 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:37.130+0000 7fbc31400640 1 -- 172.21.15.83:0/3124923670 learned_addr learned my addr 172.21.15.83:0/3124923670 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-08-19T01:33:38.343 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:37.130+0000 7fbc32800640 1 --2- 172.21.15.83:0/3124923670 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbc3410e0e0 0x7fbc3419eca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:33:38.343 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:37.130+0000 7fbc31e00640 1 -- 172.21.15.83:0/3124923670 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbc34074c70 msgr2=0x7fbc3419a910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:33:38.343 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:37.130+0000 7fbc31e00640 1 --2- 172.21.15.83:0/3124923670 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbc34074c70 0x7fbc3419a910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:33:38.343 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:37.130+0000 7fbc31e00640 1 -- 172.21.15.83:0/3124923670 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbc3410e0e0 msgr2=0x7fbc3419eca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:33:38.343 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:37.130+0000 7fbc31e00640 1 --2- 172.21.15.83:0/3124923670 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbc3410e0e0 0x7fbc3419eca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:33:38.343 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:37.130+0000 7fbc31e00640 1 -- 172.21.15.83:0/3124923670 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbc3419f380 con 0x7fbc34074350 2024-08-19T01:33:38.343 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:37.131+0000 7fbc31e00640 1 --2- 172.21.15.83:0/3124923670 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbc34074350 0x7fbc3419a3d0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fbc24005940 tx=0x7fbc240040c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:33:38.343 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:37.131+0000 7fbc2aa00640 1 -- 172.21.15.83:0/3124923670 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc2403f030 con 0x7fbc34074350 2024-08-19T01:33:38.343 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:37.131+0000 7fbc39000640 1 -- 172.21.15.83:0/3124923670 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbc3419f610 con 0x7fbc34074350 2024-08-19T01:33:38.343 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:37.131+0000 7fbc2aa00640 1 -- 172.21.15.83:0/3124923670 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbc240044c0 con 0x7fbc34074350 2024-08-19T01:33:38.343 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:37.131+0000 7fbc2aa00640 1 -- 172.21.15.83:0/3124923670 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc24035070 con 0x7fbc34074350 2024-08-19T01:33:38.343 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:37.131+0000 7fbc39000640 1 -- 172.21.15.83:0/3124923670 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbc341a6f50 con 0x7fbc34074350 2024-08-19T01:33:38.344 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:37.133+0000 7fbc39000640 1 -- 172.21.15.83:0/3124923670 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbc34074730 con 0x7fbc34074350 2024-08-19T01:33:38.344 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:37.134+0000 7fbc2aa00640 1 -- 172.21.15.83:0/3124923670 <== mon.2 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7fbc24042030 con 0x7fbc34074350 2024-08-19T01:33:38.344 INFO:teuthology.orchestra.run.smithi083.stdout:219043332139 2024-08-19T01:33:38.344 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:37.136+0000 7fbc2aa00640 1 --2- 172.21.15.83:0/3124923670 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7fbc180a2c90 0x7fbc180a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:33:38.344 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:37.136+0000 7fbc2aa00640 1 -- 172.21.15.83:0/3124923670 <== mon.2 v2:172.21.15.117:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8269+0+0 (secure 0 0 0) 0x7fbc240ef480 con 0x7fbc34074350 2024-08-19T01:33:38.345 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:37.137+0000 7fbc31400640 1 --2- 172.21.15.83:0/3124923670 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7fbc180a2c90 0x7fbc180a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:33:38.345 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:37.137+0000 7fbc31400640 1 --2- 172.21.15.83:0/3124923670 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7fbc180a2c90 0x7fbc180a5150 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fbc200095f0 tx=0x7fbc20005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:33:38.345 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:37.140+0000 7fbc2aa00640 1 -- 172.21.15.83:0/3124923670 <== mon.2 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7fbc240bc200 con 0x7fbc34074350 2024-08-19T01:33:38.345 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:37.342+0000 7fbc39000640 1 -- 172.21.15.83:0/3124923670 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) -- 0x7fbc34109260 con 0x7fbc34074350 2024-08-19T01:33:38.345 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:37.342+0000 7fbc2aa00640 1 -- 172.21.15.83:0/3124923670 <== mon.2 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7fbc240bbb00 con 0x7fbc34074350 2024-08-19T01:33:38.345 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:37.345+0000 7fbc39000640 1 -- 172.21.15.83:0/3124923670 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7fbc180a2c90 msgr2=0x7fbc180a5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:33:38.345 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:37.345+0000 7fbc39000640 1 --2- 172.21.15.83:0/3124923670 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7fbc180a2c90 0x7fbc180a5150 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fbc200095f0 tx=0x7fbc20005f90 comp rx=0 tx=0).stop 2024-08-19T01:33:38.345 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:37.345+0000 7fbc39000640 1 -- 172.21.15.83:0/3124923670 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbc34074350 msgr2=0x7fbc3419a3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:33:38.345 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:37.345+0000 7fbc39000640 1 --2- 172.21.15.83:0/3124923670 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbc34074350 0x7fbc3419a3d0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fbc24005940 tx=0x7fbc240040c0 comp rx=0 tx=0).stop 2024-08-19T01:33:38.345 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:37.345+0000 7fbc39000640 1 -- 172.21.15.83:0/3124923670 shutdown_connections 2024-08-19T01:33:38.345 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:37.345+0000 7fbc39000640 1 --2- 172.21.15.83:0/3124923670 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7fbc180a2c90 0x7fbc180a5150 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:33:38.345 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:37.345+0000 7fbc39000640 1 --2- 172.21.15.83:0/3124923670 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbc3410e0e0 0x7fbc3419eca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:33:38.346 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:37.345+0000 7fbc39000640 1 --2- 172.21.15.83:0/3124923670 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbc34074c70 0x7fbc3419a910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:33:38.346 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:37.345+0000 7fbc39000640 1 --2- 172.21.15.83:0/3124923670 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbc34074350 0x7fbc3419a3d0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:33:38.346 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:37.345+0000 7fbc39000640 1 -- 172.21.15.83:0/3124923670 >> 172.21.15.83:0/3124923670 conn(0x7fbc340fd5c0 msgr2=0x7fbc340fedc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:33:38.346 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:37.345+0000 7fbc39000640 1 -- 172.21.15.83:0/3124923670 shutdown_connections 2024-08-19T01:33:38.346 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:37.345+0000 7fbc39000640 1 -- 172.21.15.83:0/3124923670 wait complete. 2024-08-19T01:33:38.702 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:33:38 smithi117 ceph-mon[22016]: from='client.? 172.21.15.83:0/2966560491' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-08-19T01:33:38.715 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:33:38 smithi167 ceph-mon[21875]: from='client.? 172.21.15.83:0/2966560491' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-08-19T01:33:38.777 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:33:38 smithi083 ceph-mon[17120]: from='client.? 172.21.15.83:0/2966560491' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-08-19T01:33:38.903 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:38.585+0000 7f7293e00640 1 -- 172.21.15.83:0/372460976 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7294074350 msgr2=0x7f7294074730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:33:38.903 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:38.585+0000 7f7293e00640 1 --2- 172.21.15.83:0/372460976 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7294074350 0x7f7294074730 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f7284009900 tx=0x7f728402a480 comp rx=0 tx=0).stop 2024-08-19T01:33:38.903 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:38.585+0000 7f7293e00640 1 -- 172.21.15.83:0/372460976 shutdown_connections 2024-08-19T01:33:38.903 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:38.585+0000 7f7293e00640 1 --2- 172.21.15.83:0/372460976 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f729410df40 0x7f7294110390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:33:38.903 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:38.585+0000 7f7293e00640 1 --2- 172.21.15.83:0/372460976 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7294074c70 0x7f729410da00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:33:38.903 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:38.585+0000 7f7293e00640 1 --2- 172.21.15.83:0/372460976 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7294074350 0x7f7294074730 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:33:38.903 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:38.585+0000 7f7293e00640 1 -- 172.21.15.83:0/372460976 >> 172.21.15.83:0/372460976 conn(0x7f72940fd440 msgr2=0x7f72940ff860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:33:38.903 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:38.585+0000 7f7293e00640 1 -- 172.21.15.83:0/372460976 shutdown_connections 2024-08-19T01:33:38.903 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:38.585+0000 7f7293e00640 1 -- 172.21.15.83:0/372460976 wait complete. 2024-08-19T01:33:38.904 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:38.586+0000 7f7293e00640 1 Processor -- start 2024-08-19T01:33:38.904 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:38.586+0000 7f7293e00640 1 -- start start 2024-08-19T01:33:38.904 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:38.587+0000 7f7293e00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7294074350 0x7f729419a330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:33:38.904 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:38.587+0000 7f7293e00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7294074c70 0x7f729419a870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:33:38.904 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:38.587+0000 7f7293e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f729410df40 0x7f729419ec00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:33:38.904 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:38.587+0000 7f7293e00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f7294113d40 con 0x7f729410df40 2024-08-19T01:33:38.904 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:38.587+0000 7f7293e00640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f7294113bc0 con 0x7f7294074350 2024-08-19T01:33:38.904 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:38.587+0000 7f7293e00640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f7294113ec0 con 0x7f7294074c70 2024-08-19T01:33:38.904 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:38.587+0000 7f7292a00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7294074350 0x7f729419a330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:33:38.904 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:38.587+0000 7f7292a00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7294074350 0x7f729419a330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.83:56654/0 (socket says 172.21.15.83:56654) 2024-08-19T01:33:38.904 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:38.587+0000 7f7292a00640 1 -- 172.21.15.83:0/1650596604 learned_addr learned my addr 172.21.15.83:0/1650596604 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-08-19T01:33:38.904 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:38.587+0000 7f7292000640 1 --2- 172.21.15.83:0/1650596604 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7294074c70 0x7f729419a870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:33:38.904 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:38.587+0000 7f7293400640 1 --2- 172.21.15.83:0/1650596604 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f729410df40 0x7f729419ec00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:33:38.904 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:38.587+0000 7f7292a00640 1 -- 172.21.15.83:0/1650596604 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7294074c70 msgr2=0x7f729419a870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:33:38.905 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:38.587+0000 7f7292a00640 1 --2- 172.21.15.83:0/1650596604 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7294074c70 0x7f729419a870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:33:38.905 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:38.587+0000 7f7292a00640 1 -- 172.21.15.83:0/1650596604 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f729410df40 msgr2=0x7f729419ec00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:33:38.905 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:38.587+0000 7f7292a00640 1 --2- 172.21.15.83:0/1650596604 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f729410df40 0x7f729419ec00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:33:38.905 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:38.587+0000 7f7292a00640 1 -- 172.21.15.83:0/1650596604 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f729419f2e0 con 0x7f7294074350 2024-08-19T01:33:38.905 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:38.588+0000 7f7292a00640 1 --2- 172.21.15.83:0/1650596604 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7294074350 0x7f729419a330 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f7284004770 tx=0x7f7284003c10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:33:38.905 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:38.588+0000 7f7298a00640 1 -- 172.21.15.83:0/1650596604 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f728403f070 con 0x7f7294074350 2024-08-19T01:33:38.905 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:38.588+0000 7f7298a00640 1 -- 172.21.15.83:0/1650596604 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7284031ca0 con 0x7f7294074350 2024-08-19T01:33:38.905 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:38.588+0000 7f7293e00640 1 -- 172.21.15.83:0/1650596604 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f729419f570 con 0x7f7294074350 2024-08-19T01:33:38.905 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:38.588+0000 7f7298a00640 1 -- 172.21.15.83:0/1650596604 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7284035030 con 0x7f7294074350 2024-08-19T01:33:38.905 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:38.588+0000 7f7293e00640 1 -- 172.21.15.83:0/1650596604 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f72941a6eb0 con 0x7f7294074350 2024-08-19T01:33:38.905 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:38.589+0000 7f7293e00640 1 -- 172.21.15.83:0/1650596604 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7294074730 con 0x7f7294074350 2024-08-19T01:33:38.905 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:38.591+0000 7f7298a00640 1 -- 172.21.15.83:0/1650596604 <== mon.2 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f7284042070 con 0x7f7294074350 2024-08-19T01:33:38.905 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:38.592+0000 7f7298a00640 1 --2- 172.21.15.83:0/1650596604 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f72780a2c90 0x7f72780a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:33:38.905 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:38.592+0000 7f7298a00640 1 -- 172.21.15.83:0/1650596604 <== mon.2 v2:172.21.15.117:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8269+0+0 (secure 0 0 0) 0x7f72840ef7a0 con 0x7f7294074350 2024-08-19T01:33:38.905 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:38.592+0000 7f7292000640 1 --2- 172.21.15.83:0/1650596604 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f72780a2c90 0x7f72780a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:33:38.906 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:38.593+0000 7f7292000640 1 --2- 172.21.15.83:0/1650596604 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f72780a2c90 0x7f72780a5150 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f729410f390 tx=0x7f7280005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:33:38.906 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:38.596+0000 7f7298a00640 1 -- 172.21.15.83:0/1650596604 <== mon.2 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7f72840bc520 con 0x7f7294074350 2024-08-19T01:33:38.906 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:38.775+0000 7f7293e00640 1 -- 172.21.15.83:0/1650596604 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 11} v 0) -- 0x7f72941090c0 con 0x7f7294074350 2024-08-19T01:33:38.906 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:38.775+0000 7f7298a00640 1 -- 172.21.15.83:0/1650596604 <== mon.2 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 11}]=0 v0) ==== 75+0+13 (secure 0 0 0) 0x7f72840bbe20 con 0x7f7294074350 2024-08-19T01:33:38.906 INFO:teuthology.orchestra.run.smithi083.stdout:335007449113 2024-08-19T01:33:38.906 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:38.778+0000 7f7293e00640 1 -- 172.21.15.83:0/1650596604 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f72780a2c90 msgr2=0x7f72780a5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:33:38.906 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:38.778+0000 7f7293e00640 1 --2- 172.21.15.83:0/1650596604 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f72780a2c90 0x7f72780a5150 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f729410f390 tx=0x7f7280005f90 comp rx=0 tx=0).stop 2024-08-19T01:33:38.906 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:38.778+0000 7f7293e00640 1 -- 172.21.15.83:0/1650596604 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7294074350 msgr2=0x7f729419a330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:33:38.907 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:38.778+0000 7f7293e00640 1 --2- 172.21.15.83:0/1650596604 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7294074350 0x7f729419a330 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f7284004770 tx=0x7f7284003c10 comp rx=0 tx=0).stop 2024-08-19T01:33:38.907 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:38.778+0000 7f7293e00640 1 -- 172.21.15.83:0/1650596604 shutdown_connections 2024-08-19T01:33:38.907 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:38.778+0000 7f7293e00640 1 --2- 172.21.15.83:0/1650596604 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f72780a2c90 0x7f72780a5150 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:33:38.907 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:38.778+0000 7f7293e00640 1 --2- 172.21.15.83:0/1650596604 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f729410df40 0x7f729419ec00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:33:38.907 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:38.778+0000 7f7293e00640 1 --2- 172.21.15.83:0/1650596604 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7294074c70 0x7f729419a870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:33:38.907 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:38.778+0000 7f7293e00640 1 --2- 172.21.15.83:0/1650596604 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7294074350 0x7f729419a330 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:33:38.907 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:38.778+0000 7f7293e00640 1 -- 172.21.15.83:0/1650596604 >> 172.21.15.83:0/1650596604 conn(0x7f72940fd440 msgr2=0x7f72940fe180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:33:38.907 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:38.778+0000 7f7293e00640 1 -- 172.21.15.83:0/1650596604 shutdown_connections 2024-08-19T01:33:38.907 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:38.778+0000 7f7293e00640 1 -- 172.21.15.83:0/1650596604 wait complete. 2024-08-19T01:33:39.844 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:33:39 smithi083 ceph-mon[17120]: pgmap v303: 97 pgs: 97 active+clean; 640 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s rd, 5 op/s 2024-08-19T01:33:39.845 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:33:39 smithi083 ceph-mon[17120]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' 2024-08-19T01:33:39.845 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:33:39 smithi083 ceph-mon[17120]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' 2024-08-19T01:33:39.845 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:33:39 smithi083 ceph-mon[17120]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' 2024-08-19T01:33:39.845 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:33:39 smithi083 ceph-mon[17120]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' 2024-08-19T01:33:39.845 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:33:39 smithi083 ceph-mon[17120]: from='client.? 172.21.15.83:0/1650596604' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-08-19T01:33:39.952 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:33:39 smithi117 ceph-mon[22016]: pgmap v303: 97 pgs: 97 active+clean; 640 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s rd, 5 op/s 2024-08-19T01:33:39.952 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:33:39 smithi117 ceph-mon[22016]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' 2024-08-19T01:33:39.952 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:33:39 smithi117 ceph-mon[22016]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' 2024-08-19T01:33:39.952 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:33:39 smithi117 ceph-mon[22016]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' 2024-08-19T01:33:39.952 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:33:39 smithi117 ceph-mon[22016]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' 2024-08-19T01:33:39.953 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:33:39 smithi117 ceph-mon[22016]: from='client.? 172.21.15.83:0/1650596604' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-08-19T01:33:39.960 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:39.958+0000 7f0422400640 1 -- 172.21.15.83:0/327392668 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f041c103ac0 msgr2=0x7f041c103f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:33:39.960 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:39.958+0000 7f0422400640 1 --2- 172.21.15.83:0/327392668 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f041c103ac0 0x7f041c103f20 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f0404009900 tx=0x7f040402a3f0 comp rx=0 tx=0).stop 2024-08-19T01:33:39.960 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:39.958+0000 7f0422400640 1 -- 172.21.15.83:0/327392668 shutdown_connections 2024-08-19T01:33:39.960 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:39.958+0000 7f0422400640 1 --2- 172.21.15.83:0/327392668 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f041c104460 0x7f041c110990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:33:39.960 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:39.958+0000 7f0422400640 1 --2- 172.21.15.83:0/327392668 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f041c103ac0 0x7f041c103f20 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:33:39.960 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:39.958+0000 7f0422400640 1 --2- 172.21.15.83:0/327392668 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f041c109ac0 0x7f041c109ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:33:39.960 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:39.958+0000 7f0422400640 1 -- 172.21.15.83:0/327392668 >> 172.21.15.83:0/327392668 conn(0x7f041c0ff7c0 msgr2=0x7f041c101be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:33:39.960 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:39.958+0000 7f0422400640 1 -- 172.21.15.83:0/327392668 shutdown_connections 2024-08-19T01:33:39.961 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:39.958+0000 7f0422400640 1 -- 172.21.15.83:0/327392668 wait complete. 2024-08-19T01:33:39.961 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:39.959+0000 7f0422400640 1 Processor -- start 2024-08-19T01:33:39.961 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:39.959+0000 7f0422400640 1 -- start start 2024-08-19T01:33:39.961 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:39.960+0000 7f0422400640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f041c103ac0 0x7f041c19a3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:33:39.962 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:39.960+0000 7f0422400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f041c104460 0x7f041c19a8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:33:39.962 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:39.960+0000 7f0422400640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f041c109ac0 0x7f041c19ec70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:33:39.962 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:39.960+0000 7f0422400640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f041c113ec0 con 0x7f041c104460 2024-08-19T01:33:39.962 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:39.960+0000 7f0422400640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f041c113d40 con 0x7f041c103ac0 2024-08-19T01:33:39.962 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:39.960+0000 7f0422400640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f041c114040 con 0x7f041c109ac0 2024-08-19T01:33:39.962 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:39.960+0000 7f041b400640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f041c103ac0 0x7f041c19a3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:33:39.962 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:39.960+0000 7f0413e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f041c104460 0x7f041c19a8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:33:39.963 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:39.960+0000 7f041b400640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f041c103ac0 0x7f041c19a3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.83:56680/0 (socket says 172.21.15.83:56680) 2024-08-19T01:33:39.963 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:39.960+0000 7f041b400640 1 -- 172.21.15.83:0/1262685269 learned_addr learned my addr 172.21.15.83:0/1262685269 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-08-19T01:33:39.963 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:39.960+0000 7f041be00640 1 --2- 172.21.15.83:0/1262685269 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f041c109ac0 0x7f041c19ec70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:33:39.963 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:39.960+0000 7f041b400640 1 -- 172.21.15.83:0/1262685269 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f041c109ac0 msgr2=0x7f041c19ec70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:33:39.963 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:39.960+0000 7f041b400640 1 --2- 172.21.15.83:0/1262685269 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f041c109ac0 0x7f041c19ec70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:33:39.963 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:39.961+0000 7f041b400640 1 -- 172.21.15.83:0/1262685269 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f041c104460 msgr2=0x7f041c19a8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:33:39.963 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:39.961+0000 7f041b400640 1 --2- 172.21.15.83:0/1262685269 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f041c104460 0x7f041c19a8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:33:39.963 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:39.961+0000 7f041b400640 1 -- 172.21.15.83:0/1262685269 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f041c19f350 con 0x7f041c103ac0 2024-08-19T01:33:39.964 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:39.961+0000 7f041b400640 1 --2- 172.21.15.83:0/1262685269 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f041c103ac0 0x7f041c19a3a0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f040c00e9f0 tx=0x7f040c00edf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:33:39.964 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:39.961+0000 7f0418c00640 1 -- 172.21.15.83:0/1262685269 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f040c00ccc0 con 0x7f041c103ac0 2024-08-19T01:33:39.964 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:39.961+0000 7f0422400640 1 -- 172.21.15.83:0/1262685269 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f041c19f640 con 0x7f041c103ac0 2024-08-19T01:33:39.964 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:39.961+0000 7f0418c00640 1 -- 172.21.15.83:0/1262685269 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f040c00ce60 con 0x7f041c103ac0 2024-08-19T01:33:39.964 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:39.961+0000 7f0418c00640 1 -- 172.21.15.83:0/1262685269 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f040c010690 con 0x7f041c103ac0 2024-08-19T01:33:39.964 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:39.961+0000 7f0422400640 1 -- 172.21.15.83:0/1262685269 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f041c1a6f20 con 0x7f041c103ac0 2024-08-19T01:33:39.964 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:39.962+0000 7f0422400640 1 -- 172.21.15.83:0/1262685269 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f041c103f20 con 0x7f041c103ac0 2024-08-19T01:33:39.965 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:39.964+0000 7f0418c00640 1 -- 172.21.15.83:0/1262685269 <== mon.2 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f040c0046c0 con 0x7f041c103ac0 2024-08-19T01:33:39.966 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:33:39 smithi167 ceph-mon[21875]: pgmap v303: 97 pgs: 97 active+clean; 640 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s rd, 5 op/s 2024-08-19T01:33:39.966 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:33:39 smithi167 ceph-mon[21875]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' 2024-08-19T01:33:39.966 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:33:39 smithi167 ceph-mon[21875]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' 2024-08-19T01:33:39.966 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:33:39 smithi167 ceph-mon[21875]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' 2024-08-19T01:33:39.966 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:33:39 smithi167 ceph-mon[21875]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' 2024-08-19T01:33:39.966 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:33:39 smithi167 ceph-mon[21875]: from='client.? 172.21.15.83:0/1650596604' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-08-19T01:33:39.967 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:39.965+0000 7f0418c00640 1 --2- 172.21.15.83:0/1262685269 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f04000a2c10 0x7f04000a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:33:39.967 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:39.965+0000 7f0418c00640 1 -- 172.21.15.83:0/1262685269 <== mon.2 v2:172.21.15.117:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8269+0+0 (secure 0 0 0) 0x7f040c0cf5a0 con 0x7f041c103ac0 2024-08-19T01:33:39.967 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:39.966+0000 7f0413e00640 1 --2- 172.21.15.83:0/1262685269 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f04000a2c10 0x7f04000a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:33:39.967 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:39.966+0000 7f0413e00640 1 --2- 172.21.15.83:0/1262685269 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f04000a2c10 0x7f04000a50d0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f0404009230 tx=0x7f04040058d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:33:39.969 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:39.968+0000 7f0418c00640 1 -- 172.21.15.83:0/1262685269 <== mon.2 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7f040c09c320 con 0x7f041c103ac0 2024-08-19T01:33:40.148 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:40.146+0000 7f0422400640 1 -- 172.21.15.83:0/1262685269 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) -- 0x7f041c10c030 con 0x7f041c103ac0 2024-08-19T01:33:40.148 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:40.147+0000 7f0418c00640 1 -- 172.21.15.83:0/1262685269 <== mon.2 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f040c086070 con 0x7f041c103ac0 2024-08-19T01:33:40.149 INFO:teuthology.orchestra.run.smithi083.stdout:55834574919 2024-08-19T01:33:40.151 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:40.149+0000 7f0422400640 1 -- 172.21.15.83:0/1262685269 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f04000a2c10 msgr2=0x7f04000a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:33:40.151 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:40.149+0000 7f0422400640 1 --2- 172.21.15.83:0/1262685269 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f04000a2c10 0x7f04000a50d0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f0404009230 tx=0x7f04040058d0 comp rx=0 tx=0).stop 2024-08-19T01:33:40.151 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:40.149+0000 7f0422400640 1 -- 172.21.15.83:0/1262685269 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f041c103ac0 msgr2=0x7f041c19a3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:33:40.151 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:40.149+0000 7f0422400640 1 --2- 172.21.15.83:0/1262685269 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f041c103ac0 0x7f041c19a3a0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f040c00e9f0 tx=0x7f040c00edf0 comp rx=0 tx=0).stop 2024-08-19T01:33:40.151 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:40.149+0000 7f0422400640 1 -- 172.21.15.83:0/1262685269 shutdown_connections 2024-08-19T01:33:40.151 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:40.149+0000 7f0422400640 1 --2- 172.21.15.83:0/1262685269 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f04000a2c10 0x7f04000a50d0 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:33:40.151 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:40.149+0000 7f0422400640 1 --2- 172.21.15.83:0/1262685269 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f041c109ac0 0x7f041c19ec70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:33:40.151 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:40.149+0000 7f0422400640 1 --2- 172.21.15.83:0/1262685269 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f041c104460 0x7f041c19a8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:33:40.151 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:40.149+0000 7f0422400640 1 --2- 172.21.15.83:0/1262685269 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f041c103ac0 0x7f041c19a3a0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:33:40.151 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:40.149+0000 7f0422400640 1 -- 172.21.15.83:0/1262685269 >> 172.21.15.83:0/1262685269 conn(0x7f041c0ff7c0 msgr2=0x7f041c100020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:33:40.152 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:40.149+0000 7f0422400640 1 -- 172.21.15.83:0/1262685269 shutdown_connections 2024-08-19T01:33:40.152 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:40.149+0000 7f0422400640 1 -- 172.21.15.83:0/1262685269 wait complete. 2024-08-19T01:33:40.399 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/bad8780e-5dc9-11ef-bccf-c7b262605968/mon.a/config 2024-08-19T01:33:40.428 INFO:tasks.cephadm.ceph_manager.ceph:need seq 304942678043 got 304942678045 for osd.10 2024-08-19T01:33:40.428 DEBUG:teuthology.parallel:result is None 2024-08-19T01:33:40.629 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/bad8780e-5dc9-11ef-bccf-c7b262605968/mon.a/config 2024-08-19T01:33:40.720 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:33:40 smithi083 ceph-mon[17120]: from='client.? 172.21.15.83:0/1262685269' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-08-19T01:33:40.952 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:33:40 smithi117 ceph-mon[22016]: from='client.? 172.21.15.83:0/1262685269' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-08-19T01:33:40.966 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:33:40 smithi167 ceph-mon[21875]: from='client.? 172.21.15.83:0/1262685269' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-08-19T01:33:41.156 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:41.132+0000 7f9f0ba00640 1 -- 172.21.15.83:0/333133215 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9f041018b0 msgr2=0x7f9f04101d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:33:41.156 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:41.132+0000 7f9f0ba00640 1 --2- 172.21.15.83:0/333133215 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9f041018b0 0x7f9f04101d10 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f9ef0009930 tx=0x7f9ef002a420 comp rx=0 tx=0).stop 2024-08-19T01:33:41.156 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:41.133+0000 7f9f0ba00640 1 -- 172.21.15.83:0/333133215 shutdown_connections 2024-08-19T01:33:41.156 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:41.133+0000 7f9f0ba00640 1 --2- 172.21.15.83:0/333133215 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9f04102250 0x7f9f0410e780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:33:41.156 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:41.133+0000 7f9f0ba00640 1 --2- 172.21.15.83:0/333133215 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9f041018b0 0x7f9f04101d10 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:33:41.156 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:41.133+0000 7f9f0ba00640 1 --2- 172.21.15.83:0/333133215 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9f041078b0 0x7f9f04107c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:33:41.156 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:41.133+0000 7f9f0ba00640 1 -- 172.21.15.83:0/333133215 >> 172.21.15.83:0/333133215 conn(0x7f9f040fd5f0 msgr2=0x7f9f040ffa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:33:41.156 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:41.133+0000 7f9f0ba00640 1 -- 172.21.15.83:0/333133215 shutdown_connections 2024-08-19T01:33:41.156 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:41.133+0000 7f9f0ba00640 1 -- 172.21.15.83:0/333133215 wait complete. 2024-08-19T01:33:41.157 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:41.134+0000 7f9f0ba00640 1 Processor -- start 2024-08-19T01:33:41.157 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:41.134+0000 7f9f0ba00640 1 -- start start 2024-08-19T01:33:41.157 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:41.135+0000 7f9f0ba00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9f041018b0 0x7f9f04074220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:33:41.157 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:41.135+0000 7f9f0ba00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9f04102250 0x7f9f04074760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:33:41.157 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:41.135+0000 7f9f0ba00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9f041078b0 0x7f9f04074ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:33:41.157 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:41.135+0000 7f9f0ba00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f9f04111d00 con 0x7f9f041078b0 2024-08-19T01:33:41.157 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:41.135+0000 7f9f0ba00640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f9f04111b80 con 0x7f9f04102250 2024-08-19T01:33:41.157 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:41.135+0000 7f9f0ba00640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f9f04111e80 con 0x7f9f041018b0 2024-08-19T01:33:41.157 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:41.135+0000 7f9f09a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9f041078b0 0x7f9f04074ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:33:41.157 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:41.135+0000 7f9f09a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9f041078b0 0x7f9f04074ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:46292/0 (socket says 172.21.15.83:46292) 2024-08-19T01:33:41.157 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:41.135+0000 7f9f09a00640 1 -- 172.21.15.83:0/973871758 learned_addr learned my addr 172.21.15.83:0/973871758 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-08-19T01:33:41.157 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:41.135+0000 7f9f09000640 1 --2- 172.21.15.83:0/973871758 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9f041018b0 0x7f9f04074220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:33:41.157 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:41.135+0000 7f9f03e00640 1 --2- 172.21.15.83:0/973871758 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9f04102250 0x7f9f04074760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:33:41.158 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:41.135+0000 7f9f09a00640 1 -- 172.21.15.83:0/973871758 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9f041018b0 msgr2=0x7f9f04074220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:33:41.158 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:41.135+0000 7f9f09a00640 1 --2- 172.21.15.83:0/973871758 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9f041018b0 0x7f9f04074220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:33:41.158 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:41.135+0000 7f9f09a00640 1 -- 172.21.15.83:0/973871758 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9f04102250 msgr2=0x7f9f04074760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:33:41.158 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:41.135+0000 7f9f09a00640 1 --2- 172.21.15.83:0/973871758 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9f04102250 0x7f9f04074760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:33:41.158 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:41.135+0000 7f9f09a00640 1 -- 172.21.15.83:0/973871758 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9f041a66c0 con 0x7f9f041078b0 2024-08-19T01:33:41.158 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:41.135+0000 7f9f03e00640 1 --2- 172.21.15.83:0/973871758 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9f04102250 0x7f9f04074760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-19T01:33:41.158 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:41.136+0000 7f9f09a00640 1 --2- 172.21.15.83:0/973871758 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9f041078b0 0x7f9f04074ca0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f9efc009fd0 tx=0x7f9efc00b260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:33:41.158 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:41.136+0000 7f9f01600640 1 -- 172.21.15.83:0/973871758 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9efc017070 con 0x7f9f041078b0 2024-08-19T01:33:41.158 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:41.136+0000 7f9f01600640 1 -- 172.21.15.83:0/973871758 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9efc003e50 con 0x7f9f041078b0 2024-08-19T01:33:41.158 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:41.136+0000 7f9f0ba00640 1 -- 172.21.15.83:0/973871758 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9f041a69b0 con 0x7f9f041078b0 2024-08-19T01:33:41.158 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:41.136+0000 7f9f01600640 1 -- 172.21.15.83:0/973871758 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9efc0106c0 con 0x7f9f041078b0 2024-08-19T01:33:41.158 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:41.136+0000 7f9f0ba00640 1 -- 172.21.15.83:0/973871758 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9f041a6e70 con 0x7f9f041078b0 2024-08-19T01:33:41.158 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:41.137+0000 7f9f0ba00640 1 -- 172.21.15.83:0/973871758 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9f04101d10 con 0x7f9f041078b0 2024-08-19T01:33:41.159 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:41.139+0000 7f9f01600640 1 -- 172.21.15.83:0/973871758 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f9efc003970 con 0x7f9f041078b0 2024-08-19T01:33:41.159 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:41.140+0000 7f9f01600640 1 --2- 172.21.15.83:0/973871758 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f9ee80a2c10 0x7f9ee80a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:33:41.159 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:41.140+0000 7f9f01600640 1 -- 172.21.15.83:0/973871758 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8269+0+0 (secure 0 0 0) 0x7f9efc0ceff0 con 0x7f9f041078b0 2024-08-19T01:33:41.159 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:41.140+0000 7f9f09000640 1 --2- 172.21.15.83:0/973871758 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f9ee80a2c10 0x7f9ee80a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:33:41.159 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:41.141+0000 7f9f09000640 1 --2- 172.21.15.83:0/973871758 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f9ee80a2c10 0x7f9ee80a50d0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f9ef4005fd0 tx=0x7f9ef400b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:33:41.159 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:41.144+0000 7f9f01600640 1 -- 172.21.15.83:0/973871758 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7f9efc09bd90 con 0x7f9f041078b0 2024-08-19T01:33:41.237 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:38.232+0000 7f29aa600640 1 -- 172.21.15.83:0/2916689334 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f29a4102250 msgr2=0x7f29a410e780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:33:41.237 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:38.232+0000 7f29aa600640 1 --2- 172.21.15.83:0/2916689334 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f29a4102250 0x7f29a410e780 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f299c009ca0 tx=0x7f299c02a8c0 comp rx=0 tx=0).stop 2024-08-19T01:33:41.237 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:38.232+0000 7f29aa600640 1 -- 172.21.15.83:0/2916689334 shutdown_connections 2024-08-19T01:33:41.238 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:38.232+0000 7f29aa600640 1 --2- 172.21.15.83:0/2916689334 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f29a4102250 0x7f29a410e780 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:33:41.238 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:38.232+0000 7f29aa600640 1 --2- 172.21.15.83:0/2916689334 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f29a41018b0 0x7f29a4101d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:33:41.238 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:38.232+0000 7f29aa600640 1 --2- 172.21.15.83:0/2916689334 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f29a41078b0 0x7f29a4107c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:33:41.238 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:38.232+0000 7f29aa600640 1 -- 172.21.15.83:0/2916689334 >> 172.21.15.83:0/2916689334 conn(0x7f29a40fd5f0 msgr2=0x7f29a40ffa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:33:41.238 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:38.232+0000 7f29aa600640 1 -- 172.21.15.83:0/2916689334 shutdown_connections 2024-08-19T01:33:41.238 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:38.232+0000 7f29aa600640 1 -- 172.21.15.83:0/2916689334 wait complete. 2024-08-19T01:33:41.238 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:38.233+0000 7f29aa600640 1 Processor -- start 2024-08-19T01:33:41.238 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:38.233+0000 7f29aa600640 1 -- start start 2024-08-19T01:33:41.238 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:38.233+0000 7f29aa600640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f29a41018b0 0x7f29a4198100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:33:41.238 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:38.233+0000 7f29aa600640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f29a4102250 0x7f29a4198640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:33:41.238 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:38.233+0000 7f29aa600640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f29a41078b0 0x7f29a419c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:33:41.238 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:38.233+0000 7f29aa600640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f29a4111d00 con 0x7f29a4102250 2024-08-19T01:33:41.239 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:38.233+0000 7f29aa600640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f29a4111b80 con 0x7f29a41078b0 2024-08-19T01:33:41.239 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:38.233+0000 7f29aa600640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f29a4111e80 con 0x7f29a41018b0 2024-08-19T01:33:41.239 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:38.233+0000 7f29a3e00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f29a41078b0 0x7f29a419c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:33:41.239 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:38.233+0000 7f29a3e00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f29a41078b0 0x7f29a419c9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.83:56636/0 (socket says 172.21.15.83:56636) 2024-08-19T01:33:41.239 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:38.233+0000 7f29a3e00640 1 -- 172.21.15.83:0/2966560491 learned_addr learned my addr 172.21.15.83:0/2966560491 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-08-19T01:33:41.239 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:38.233+0000 7f29a3400640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f29a41018b0 0x7f29a4198100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:33:41.239 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:38.234+0000 7f299aa00640 1 --2- 172.21.15.83:0/2966560491 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f29a4102250 0x7f29a4198640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:33:41.239 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:38.234+0000 7f29a3400640 1 -- 172.21.15.83:0/2966560491 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f29a41078b0 msgr2=0x7f29a419c9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:33:41.239 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:38.234+0000 7f29a3400640 1 --2- 172.21.15.83:0/2966560491 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f29a41078b0 0x7f29a419c9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:33:41.239 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:38.234+0000 7f29a3400640 1 -- 172.21.15.83:0/2966560491 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f29a4102250 msgr2=0x7f29a4198640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:33:41.239 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:38.234+0000 7f29a3400640 1 --2- 172.21.15.83:0/2966560491 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f29a4102250 0x7f29a4198640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:33:41.239 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:38.234+0000 7f29a3400640 1 -- 172.21.15.83:0/2966560491 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f29a419d0b0 con 0x7f29a41018b0 2024-08-19T01:33:41.239 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:38.234+0000 7f29a3400640 1 --2- 172.21.15.83:0/2966560491 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f29a41018b0 0x7f29a4198100 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f299400e9f0 tx=0x7f299400eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:33:41.240 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:38.234+0000 7f29a0c00640 1 -- 172.21.15.83:0/2966560491 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f299400ccc0 con 0x7f29a41018b0 2024-08-19T01:33:41.240 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:38.234+0000 7f29aa600640 1 -- 172.21.15.83:0/2966560491 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f29a419d3a0 con 0x7f29a41018b0 2024-08-19T01:33:41.240 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:38.234+0000 7f29a0c00640 1 -- 172.21.15.83:0/2966560491 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2994010910 con 0x7f29a41018b0 2024-08-19T01:33:41.240 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:38.234+0000 7f29a0c00640 1 -- 172.21.15.83:0/2966560491 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2994018960 con 0x7f29a41018b0 2024-08-19T01:33:41.240 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:38.235+0000 7f29aa600640 1 -- 172.21.15.83:0/2966560491 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f29a41a4c80 con 0x7f29a41018b0 2024-08-19T01:33:41.240 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:38.235+0000 7f29aa600640 1 -- 172.21.15.83:0/2966560491 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f29a4101d10 con 0x7f29a41018b0 2024-08-19T01:33:41.240 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:38.237+0000 7f29a0c00640 1 -- 172.21.15.83:0/2966560491 <== mon.1 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f2994010ac0 con 0x7f29a41018b0 2024-08-19T01:33:41.240 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:38.239+0000 7f29a0c00640 1 --2- 172.21.15.83:0/2966560491 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f29880a2c10 0x7f29880a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:33:41.240 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:38.239+0000 7f29a0c00640 1 -- 172.21.15.83:0/2966560491 <== mon.1 v2:172.21.15.167:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8269+0+0 (secure 0 0 0) 0x7f29940cf570 con 0x7f29a41018b0 2024-08-19T01:33:41.240 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:38.239+0000 7f299aa00640 1 --2- 172.21.15.83:0/2966560491 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f29880a2c10 0x7f29880a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:33:41.240 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:38.239+0000 7f299aa00640 1 --2- 172.21.15.83:0/2966560491 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f29880a2c10 0x7f29880a50d0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f29a40ffb40 tx=0x7f298c005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:33:41.240 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:38.243+0000 7f29a0c00640 1 -- 172.21.15.83:0/2966560491 <== mon.1 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7f299409bca0 con 0x7f29a41018b0 2024-08-19T01:33:41.240 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:38.429+0000 7f29aa600640 1 -- 172.21.15.83:0/2966560491 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) -- 0x7f29a41a4f70 con 0x7f29a41018b0 2024-08-19T01:33:41.240 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:38.429+0000 7f29a0c00640 1 -- 172.21.15.83:0/2966560491 <== mon.1 v2:172.21.15.167:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f299409ffa0 con 0x7f29a41018b0 2024-08-19T01:33:41.241 INFO:teuthology.orchestra.run.smithi083.stdout:163208757301 2024-08-19T01:33:41.241 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:38.432+0000 7f29aa600640 1 -- 172.21.15.83:0/2966560491 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f29880a2c10 msgr2=0x7f29880a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:33:41.241 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:38.432+0000 7f29aa600640 1 --2- 172.21.15.83:0/2966560491 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f29880a2c10 0x7f29880a50d0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f29a40ffb40 tx=0x7f298c005f90 comp rx=0 tx=0).stop 2024-08-19T01:33:41.241 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:38.432+0000 7f29aa600640 1 -- 172.21.15.83:0/2966560491 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f29a41018b0 msgr2=0x7f29a4198100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:33:41.241 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:38.432+0000 7f29aa600640 1 --2- 172.21.15.83:0/2966560491 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f29a41018b0 0x7f29a4198100 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f299400e9f0 tx=0x7f299400eeb0 comp rx=0 tx=0).stop 2024-08-19T01:33:41.241 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:38.432+0000 7f29aa600640 1 -- 172.21.15.83:0/2966560491 shutdown_connections 2024-08-19T01:33:41.241 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:38.432+0000 7f29aa600640 1 --2- 172.21.15.83:0/2966560491 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f29880a2c10 0x7f29880a50d0 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:33:41.241 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:38.432+0000 7f29aa600640 1 --2- 172.21.15.83:0/2966560491 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f29a41078b0 0x7f29a419c9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:33:41.242 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:38.432+0000 7f29aa600640 1 --2- 172.21.15.83:0/2966560491 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f29a4102250 0x7f29a4198640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:33:41.242 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:38.432+0000 7f29aa600640 1 --2- 172.21.15.83:0/2966560491 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f29a41018b0 0x7f29a4198100 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:33:41.242 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:38.432+0000 7f29aa600640 1 -- 172.21.15.83:0/2966560491 >> 172.21.15.83:0/2966560491 conn(0x7f29a40fd5f0 msgr2=0x7f29a40fdc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:33:41.242 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:38.432+0000 7f29aa600640 1 -- 172.21.15.83:0/2966560491 shutdown_connections 2024-08-19T01:33:41.242 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:38.432+0000 7f29aa600640 1 -- 172.21.15.83:0/2966560491 wait complete. 2024-08-19T01:33:41.327 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:41.325+0000 7f9f0ba00640 1 -- 172.21.15.83:0/973871758 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) -- 0x7f9f04109e20 con 0x7f9f041078b0 2024-08-19T01:33:41.327 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:41.325+0000 7f9f01600640 1 -- 172.21.15.83:0/973871758 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f9efc09b690 con 0x7f9f041078b0 2024-08-19T01:33:41.327 INFO:teuthology.orchestra.run.smithi083.stdout:34359738445 2024-08-19T01:33:41.329 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:41.328+0000 7f9f0ba00640 1 -- 172.21.15.83:0/973871758 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f9ee80a2c10 msgr2=0x7f9ee80a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:33:41.329 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:41.328+0000 7f9f0ba00640 1 --2- 172.21.15.83:0/973871758 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f9ee80a2c10 0x7f9ee80a50d0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f9ef4005fd0 tx=0x7f9ef400b390 comp rx=0 tx=0).stop 2024-08-19T01:33:41.329 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:41.328+0000 7f9f0ba00640 1 -- 172.21.15.83:0/973871758 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9f041078b0 msgr2=0x7f9f04074ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:33:41.329 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:41.328+0000 7f9f0ba00640 1 --2- 172.21.15.83:0/973871758 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9f041078b0 0x7f9f04074ca0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f9efc009fd0 tx=0x7f9efc00b260 comp rx=0 tx=0).stop 2024-08-19T01:33:41.330 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:41.328+0000 7f9f0ba00640 1 -- 172.21.15.83:0/973871758 shutdown_connections 2024-08-19T01:33:41.330 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:41.328+0000 7f9f0ba00640 1 --2- 172.21.15.83:0/973871758 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f9ee80a2c10 0x7f9ee80a50d0 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:33:41.330 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:41.328+0000 7f9f0ba00640 1 --2- 172.21.15.83:0/973871758 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9f041078b0 0x7f9f04074ca0 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:33:41.330 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:41.328+0000 7f9f0ba00640 1 --2- 172.21.15.83:0/973871758 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9f04102250 0x7f9f04074760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:33:41.330 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:41.328+0000 7f9f0ba00640 1 --2- 172.21.15.83:0/973871758 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9f041018b0 0x7f9f04074220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:33:41.330 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:41.328+0000 7f9f0ba00640 1 -- 172.21.15.83:0/973871758 >> 172.21.15.83:0/973871758 conn(0x7f9f040fd5f0 msgr2=0x7f9f040fde70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:33:41.330 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:41.328+0000 7f9f0ba00640 1 -- 172.21.15.83:0/973871758 shutdown_connections 2024-08-19T01:33:41.330 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:41.328+0000 7f9f0ba00640 1 -- 172.21.15.83:0/973871758 wait complete. 2024-08-19T01:33:41.578 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:33:41 smithi083 ceph-mon[17120]: pgmap v304: 97 pgs: 97 active+clean; 640 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 5 op/s 2024-08-19T01:33:41.578 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:33:41 smithi083 ceph-mon[17120]: from='client.? 172.21.15.83:0/973871758' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-08-19T01:33:41.952 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:33:41 smithi117 ceph-mon[22016]: pgmap v304: 97 pgs: 97 active+clean; 640 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 5 op/s 2024-08-19T01:33:41.952 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:33:41 smithi117 ceph-mon[22016]: from='client.? 172.21.15.83:0/973871758' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-08-19T01:33:41.966 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:33:41 smithi167 ceph-mon[21875]: pgmap v304: 97 pgs: 97 active+clean; 640 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 5 op/s 2024-08-19T01:33:41.966 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:33:41 smithi167 ceph-mon[21875]: from='client.? 172.21.15.83:0/973871758' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-08-19T01:33:42.301 INFO:tasks.cephadm.ceph_manager.ceph:need seq 55834574917 got 55834574919 for osd.1 2024-08-19T01:33:42.301 DEBUG:teuthology.parallel:result is None 2024-08-19T01:33:42.805 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:42.803+0000 7f43a5200640 1 -- 172.21.15.83:0/945533976 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f43a01076f0 msgr2=0x7f43a0107ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:33:42.805 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:42.803+0000 7f43a5200640 1 --2- 172.21.15.83:0/945533976 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f43a01076f0 0x7f43a0107ad0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f4390009930 tx=0x7f439002a420 comp rx=0 tx=0).stop 2024-08-19T01:33:42.805 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:42.803+0000 7f43a5200640 1 -- 172.21.15.83:0/945533976 shutdown_connections 2024-08-19T01:33:42.805 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:42.803+0000 7f43a5200640 1 --2- 172.21.15.83:0/945533976 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f43a0102090 0x7f43a010e5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:33:42.806 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:33:42 smithi083 ceph-mon[17120]: from='client.? 172.21.15.83:0/3929494168' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-08-19T01:33:42.806 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:42.803+0000 7f43a5200640 1 --2- 172.21.15.83:0/945533976 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f43a01016f0 0x7f43a0101b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:33:42.806 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:42.803+0000 7f43a5200640 1 --2- 172.21.15.83:0/945533976 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f43a01076f0 0x7f43a0107ad0 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:33:42.806 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:42.803+0000 7f43a5200640 1 -- 172.21.15.83:0/945533976 >> 172.21.15.83:0/945533976 conn(0x7f43a00fd470 msgr2=0x7f43a00ff890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:33:42.807 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:42.803+0000 7f43a5200640 1 -- 172.21.15.83:0/945533976 shutdown_connections 2024-08-19T01:33:42.807 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:42.805+0000 7f43a5200640 1 -- 172.21.15.83:0/945533976 wait complete. 2024-08-19T01:33:42.807 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:42.806+0000 7f43a5200640 1 Processor -- start 2024-08-19T01:33:42.807 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:42.806+0000 7f43a5200640 1 -- start start 2024-08-19T01:33:42.808 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:42.806+0000 7f43a5200640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f43a01016f0 0x7f43a0197ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:33:42.808 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:42.806+0000 7f43a5200640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f43a0102090 0x7f43a0198530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:33:42.808 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:42.806+0000 7f43a5200640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f43a01076f0 0x7f43a019c8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:33:42.808 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:42.806+0000 7f43a5200640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f43a0111af0 con 0x7f43a01076f0 2024-08-19T01:33:42.809 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:42.806+0000 7f43a5200640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f43a0111970 con 0x7f43a0102090 2024-08-19T01:33:42.809 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:42.806+0000 7f43a5200640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f43a0111c70 con 0x7f43a01016f0 2024-08-19T01:33:42.809 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:42.807+0000 7f439f400640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f43a01016f0 0x7f43a0197ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:33:42.809 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:42.807+0000 7f439ea00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f43a0102090 0x7f43a0198530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:33:42.809 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:42.807+0000 7f439f400640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f43a01016f0 0x7f43a0197ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.83:36206/0 (socket says 172.21.15.83:36206) 2024-08-19T01:33:42.809 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:42.807+0000 7f439f400640 1 -- 172.21.15.83:0/55356964 learned_addr learned my addr 172.21.15.83:0/55356964 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-08-19T01:33:42.809 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:42.807+0000 7f439fe00640 1 --2- 172.21.15.83:0/55356964 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f43a01076f0 0x7f43a019c8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:33:42.809 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:42.807+0000 7f439f400640 1 -- 172.21.15.83:0/55356964 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f43a0102090 msgr2=0x7f43a0198530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:33:42.809 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:42.807+0000 7f439f400640 1 --2- 172.21.15.83:0/55356964 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f43a0102090 0x7f43a0198530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:33:42.809 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:42.807+0000 7f439f400640 1 -- 172.21.15.83:0/55356964 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f43a01076f0 msgr2=0x7f43a019c8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:33:42.810 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:42.807+0000 7f439f400640 1 --2- 172.21.15.83:0/55356964 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f43a01076f0 0x7f43a019c8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:33:42.810 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:42.807+0000 7f439f400640 1 -- 172.21.15.83:0/55356964 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f43a019cfa0 con 0x7f43a01016f0 2024-08-19T01:33:42.810 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:42.807+0000 7f439ea00640 1 --2- 172.21.15.83:0/55356964 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f43a0102090 0x7f43a0198530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-19T01:33:42.810 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:42.808+0000 7f439f400640 1 --2- 172.21.15.83:0/55356964 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f43a01016f0 0x7f43a0197ff0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f4390009230 tx=0x7f4390004040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:33:42.811 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:42.808+0000 7f4397e00640 1 -- 172.21.15.83:0/55356964 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f439003f030 con 0x7f43a01016f0 2024-08-19T01:33:42.811 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:42.808+0000 7f43a5200640 1 -- 172.21.15.83:0/55356964 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f43a019d230 con 0x7f43a01016f0 2024-08-19T01:33:42.811 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:42.808+0000 7f4397e00640 1 -- 172.21.15.83:0/55356964 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4390004440 con 0x7f43a01016f0 2024-08-19T01:33:42.811 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:42.808+0000 7f4397e00640 1 -- 172.21.15.83:0/55356964 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4390035070 con 0x7f43a01016f0 2024-08-19T01:33:42.811 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:42.808+0000 7f43a5200640 1 -- 172.21.15.83:0/55356964 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f43a01a4b70 con 0x7f43a01016f0 2024-08-19T01:33:42.811 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:42.809+0000 7f43a5200640 1 -- 172.21.15.83:0/55356964 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f43a0101b50 con 0x7f43a01016f0 2024-08-19T01:33:42.812 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:42.811+0000 7f4397e00640 1 -- 172.21.15.83:0/55356964 <== mon.1 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f4390042030 con 0x7f43a01016f0 2024-08-19T01:33:42.814 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:42.812+0000 7f4397e00640 1 --2- 172.21.15.83:0/55356964 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f43840a2c10 0x7f43840a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:33:42.814 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:42.812+0000 7f4397e00640 1 -- 172.21.15.83:0/55356964 <== mon.1 v2:172.21.15.167:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8269+0+0 (secure 0 0 0) 0x7f43900ef550 con 0x7f43a01016f0 2024-08-19T01:33:42.814 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:42.813+0000 7f439ea00640 1 --2- 172.21.15.83:0/55356964 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f43840a2c10 0x7f43840a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:33:42.815 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:42.813+0000 7f439ea00640 1 --2- 172.21.15.83:0/55356964 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f43840a2c10 0x7f43840a50d0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f43a0100a30 tx=0x7f438c005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:33:42.817 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:42.816+0000 7f4397e00640 1 -- 172.21.15.83:0/55356964 <== mon.1 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7f43900bbc80 con 0x7f43a01016f0 2024-08-19T01:33:42.952 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:33:42 smithi117 ceph-mon[22016]: from='client.? 172.21.15.83:0/3929494168' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-08-19T01:33:42.966 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:33:42 smithi167 ceph-mon[21875]: from='client.? 172.21.15.83:0/3929494168' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-08-19T01:33:43.007 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:43.005+0000 7f43a5200640 1 -- 172.21.15.83:0/55356964 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) -- 0x7f43a0109c60 con 0x7f43a01016f0 2024-08-19T01:33:43.007 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:43.005+0000 7f4397e00640 1 -- 172.21.15.83:0/55356964 <== mon.1 v2:172.21.15.167:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f43900bff80 con 0x7f43a01016f0 2024-08-19T01:33:43.007 INFO:teuthology.orchestra.run.smithi083.stdout:107374182463 2024-08-19T01:33:43.009 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:43.007+0000 7f43a5200640 1 -- 172.21.15.83:0/55356964 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f43840a2c10 msgr2=0x7f43840a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:33:43.009 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:43.007+0000 7f43a5200640 1 --2- 172.21.15.83:0/55356964 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f43840a2c10 0x7f43840a50d0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f43a0100a30 tx=0x7f438c005f90 comp rx=0 tx=0).stop 2024-08-19T01:33:43.009 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:43.007+0000 7f43a5200640 1 -- 172.21.15.83:0/55356964 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f43a01016f0 msgr2=0x7f43a0197ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:33:43.009 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:43.007+0000 7f43a5200640 1 --2- 172.21.15.83:0/55356964 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f43a01016f0 0x7f43a0197ff0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f4390009230 tx=0x7f4390004040 comp rx=0 tx=0).stop 2024-08-19T01:33:43.010 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:43.008+0000 7f43a5200640 1 -- 172.21.15.83:0/55356964 shutdown_connections 2024-08-19T01:33:43.010 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:43.008+0000 7f43a5200640 1 --2- 172.21.15.83:0/55356964 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f43840a2c10 0x7f43840a50d0 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:33:43.010 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:43.008+0000 7f43a5200640 1 --2- 172.21.15.83:0/55356964 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f43a01076f0 0x7f43a019c8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:33:43.010 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:43.008+0000 7f43a5200640 1 --2- 172.21.15.83:0/55356964 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f43a0102090 0x7f43a0198530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:33:43.010 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:43.008+0000 7f43a5200640 1 --2- 172.21.15.83:0/55356964 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f43a01016f0 0x7f43a0197ff0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:33:43.010 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:43.008+0000 7f43a5200640 1 -- 172.21.15.83:0/55356964 >> 172.21.15.83:0/55356964 conn(0x7f43a00fd470 msgr2=0x7f43a00fed60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:33:43.010 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:43.008+0000 7f43a5200640 1 -- 172.21.15.83:0/55356964 shutdown_connections 2024-08-19T01:33:43.010 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:43.008+0000 7f43a5200640 1 -- 172.21.15.83:0/55356964 wait complete. 2024-08-19T01:33:43.339 INFO:tasks.cephadm.ceph_manager.ceph:need seq 34359738443 got 34359738445 for osd.0 2024-08-19T01:33:43.340 DEBUG:teuthology.parallel:result is None 2024-08-19T01:33:43.632 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:33:43 smithi083 ceph-mon[17120]: pgmap v305: 97 pgs: 97 active+clean; 640 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-08-19T01:33:43.632 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:33:43 smithi083 ceph-mon[17120]: from='client.? 172.21.15.83:0/345329434' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-08-19T01:33:43.632 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:33:43 smithi083 ceph-mon[17120]: from='client.? 172.21.15.83:0/55356964' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-08-19T01:33:43.670 INFO:tasks.cephadm.ceph_manager.ceph:need seq 335007449110 got 335007449113 for osd.11 2024-08-19T01:33:43.670 DEBUG:teuthology.parallel:result is None 2024-08-19T01:33:43.952 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:33:43 smithi117 ceph-mon[22016]: pgmap v305: 97 pgs: 97 active+clean; 640 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-08-19T01:33:43.952 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:33:43 smithi117 ceph-mon[22016]: from='client.? 172.21.15.83:0/345329434' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-08-19T01:33:43.953 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:33:43 smithi117 ceph-mon[22016]: from='client.? 172.21.15.83:0/55356964' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-08-19T01:33:43.966 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:33:43 smithi167 ceph-mon[21875]: pgmap v305: 97 pgs: 97 active+clean; 640 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-08-19T01:33:43.966 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:33:43 smithi167 ceph-mon[21875]: from='client.? 172.21.15.83:0/345329434' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-08-19T01:33:43.966 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:33:43 smithi167 ceph-mon[21875]: from='client.? 172.21.15.83:0/55356964' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-08-19T01:33:44.010 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:42.684+0000 7f6cc6800640 1 -- 172.21.15.83:0/2963937235 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6cc0109900 msgr2=0x7f6cc0109ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:33:44.010 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:42.684+0000 7f6cc6800640 1 --2- 172.21.15.83:0/2963937235 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6cc0109900 0x7f6cc0109ce0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f6cb4009930 tx=0x7f6cb402a420 comp rx=0 tx=0).stop 2024-08-19T01:33:44.010 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:42.684+0000 7f6cc6800640 1 -- 172.21.15.83:0/2963937235 shutdown_connections 2024-08-19T01:33:44.010 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:42.684+0000 7f6cc6800640 1 --2- 172.21.15.83:0/2963937235 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6cc01042a0 0x7f6cc01107d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:33:44.010 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:42.684+0000 7f6cc6800640 1 --2- 172.21.15.83:0/2963937235 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6cc0103900 0x7f6cc0103d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:33:44.011 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:42.684+0000 7f6cc6800640 1 --2- 172.21.15.83:0/2963937235 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6cc0109900 0x7f6cc0109ce0 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:33:44.011 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:42.684+0000 7f6cc6800640 1 -- 172.21.15.83:0/2963937235 >> 172.21.15.83:0/2963937235 conn(0x7f6cc00ff600 msgr2=0x7f6cc0101a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:33:44.011 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:42.684+0000 7f6cc6800640 1 -- 172.21.15.83:0/2963937235 shutdown_connections 2024-08-19T01:33:44.011 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:42.684+0000 7f6cc6800640 1 -- 172.21.15.83:0/2963937235 wait complete. 2024-08-19T01:33:44.011 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:42.685+0000 7f6cc6800640 1 Processor -- start 2024-08-19T01:33:44.011 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:42.685+0000 7f6cc6800640 1 -- start start 2024-08-19T01:33:44.011 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:42.685+0000 7f6cc6800640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6cc0103900 0x7f6cc019a1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:33:44.011 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:42.685+0000 7f6cc6800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6cc01042a0 0x7f6cc019a700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:33:44.011 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:42.685+0000 7f6cc6800640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6cc0109900 0x7f6cc019ea90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:33:44.011 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:42.685+0000 7f6cc6800640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f6cc0113d00 con 0x7f6cc01042a0 2024-08-19T01:33:44.011 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:42.685+0000 7f6cc6800640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f6cc0113b80 con 0x7f6cc0109900 2024-08-19T01:33:44.011 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:42.685+0000 7f6cc6800640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f6cc0113e80 con 0x7f6cc0103900 2024-08-19T01:33:44.012 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:42.685+0000 7f6cc4a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6cc01042a0 0x7f6cc019a700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:33:44.012 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:42.685+0000 7f6cc4a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6cc01042a0 0x7f6cc019a700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:46330/0 (socket says 172.21.15.83:46330) 2024-08-19T01:33:44.012 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:42.685+0000 7f6cc4a00640 1 -- 172.21.15.83:0/345329434 learned_addr learned my addr 172.21.15.83:0/345329434 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-08-19T01:33:44.012 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:42.685+0000 7f6cc5400640 1 --2- 172.21.15.83:0/345329434 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6cc0103900 0x7f6cc019a1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:33:44.012 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:42.686+0000 7f6cc5e00640 1 --2- 172.21.15.83:0/345329434 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6cc0109900 0x7f6cc019ea90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:33:44.012 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:42.686+0000 7f6cc4a00640 1 -- 172.21.15.83:0/345329434 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6cc0103900 msgr2=0x7f6cc019a1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:33:44.012 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:42.686+0000 7f6cc4a00640 1 --2- 172.21.15.83:0/345329434 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6cc0103900 0x7f6cc019a1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:33:44.012 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:42.686+0000 7f6cc4a00640 1 -- 172.21.15.83:0/345329434 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6cc0109900 msgr2=0x7f6cc019ea90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:33:44.012 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:42.686+0000 7f6cc4a00640 1 --2- 172.21.15.83:0/345329434 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6cc0109900 0x7f6cc019ea90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:33:44.012 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:42.686+0000 7f6cc4a00640 1 -- 172.21.15.83:0/345329434 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6cc019f170 con 0x7f6cc01042a0 2024-08-19T01:33:44.012 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:42.686+0000 7f6cc4a00640 1 --2- 172.21.15.83:0/345329434 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6cc01042a0 0x7f6cc019a700 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f6cac00e9f0 tx=0x7f6cac00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:33:44.012 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:42.686+0000 7f6cbe000640 1 -- 172.21.15.83:0/345329434 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6cac00cd80 con 0x7f6cc01042a0 2024-08-19T01:33:44.012 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:42.686+0000 7f6cbe000640 1 -- 172.21.15.83:0/345329434 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6cac010910 con 0x7f6cc01042a0 2024-08-19T01:33:44.013 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:42.686+0000 7f6cc6800640 1 -- 172.21.15.83:0/345329434 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6cc019f460 con 0x7f6cc01042a0 2024-08-19T01:33:44.013 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:42.686+0000 7f6cbe000640 1 -- 172.21.15.83:0/345329434 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6cac018990 con 0x7f6cc01042a0 2024-08-19T01:33:44.013 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:42.687+0000 7f6cc6800640 1 -- 172.21.15.83:0/345329434 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6cc01a6d40 con 0x7f6cc01042a0 2024-08-19T01:33:44.013 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:42.688+0000 7f6cc6800640 1 -- 172.21.15.83:0/345329434 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6cc0103d60 con 0x7f6cc01042a0 2024-08-19T01:33:44.013 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:42.689+0000 7f6cbe000640 1 -- 172.21.15.83:0/345329434 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f6cac010ac0 con 0x7f6cc01042a0 2024-08-19T01:33:44.013 INFO:teuthology.orchestra.run.smithi083.stdout:141733920826 2024-08-19T01:33:44.013 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:42.691+0000 7f6cbe000640 1 --2- 172.21.15.83:0/345329434 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f6ca80a2c90 0x7f6ca80a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:33:44.013 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:42.691+0000 7f6cbe000640 1 -- 172.21.15.83:0/345329434 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8269+0+0 (secure 0 0 0) 0x7f6cac0cf5d0 con 0x7f6cc01042a0 2024-08-19T01:33:44.014 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:42.692+0000 7f6cc5400640 1 --2- 172.21.15.83:0/345329434 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f6ca80a2c90 0x7f6ca80a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:33:44.014 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:42.692+0000 7f6cc5400640 1 --2- 172.21.15.83:0/345329434 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f6ca80a2c90 0x7f6ca80a5150 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f6cb4009900 tx=0x7f6cb40058d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:33:44.014 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:42.695+0000 7f6cbe000640 1 -- 172.21.15.83:0/345329434 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7f6cac09bd00 con 0x7f6cc01042a0 2024-08-19T01:33:44.014 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:42.886+0000 7f6cc6800640 1 -- 172.21.15.83:0/345329434 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) -- 0x7f6cc002d5e0 con 0x7f6cc01042a0 2024-08-19T01:33:44.014 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:42.886+0000 7f6cbe000640 1 -- 172.21.15.83:0/345329434 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f6cac0a0000 con 0x7f6cc01042a0 2024-08-19T01:33:44.014 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:42.888+0000 7f6cc6800640 1 -- 172.21.15.83:0/345329434 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f6ca80a2c90 msgr2=0x7f6ca80a5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:33:44.014 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:42.889+0000 7f6cc6800640 1 --2- 172.21.15.83:0/345329434 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f6ca80a2c90 0x7f6ca80a5150 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f6cb4009900 tx=0x7f6cb40058d0 comp rx=0 tx=0).stop 2024-08-19T01:33:44.014 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:42.889+0000 7f6cc6800640 1 -- 172.21.15.83:0/345329434 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6cc01042a0 msgr2=0x7f6cc019a700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:33:44.015 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:42.889+0000 7f6cc6800640 1 --2- 172.21.15.83:0/345329434 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6cc01042a0 0x7f6cc019a700 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f6cac00e9f0 tx=0x7f6cac00eeb0 comp rx=0 tx=0).stop 2024-08-19T01:33:44.015 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:42.889+0000 7f6cc6800640 1 -- 172.21.15.83:0/345329434 shutdown_connections 2024-08-19T01:33:44.015 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:42.889+0000 7f6cc6800640 1 --2- 172.21.15.83:0/345329434 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f6ca80a2c90 0x7f6ca80a5150 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:33:44.015 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:42.889+0000 7f6cc6800640 1 --2- 172.21.15.83:0/345329434 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6cc0109900 0x7f6cc019ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:33:44.015 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:42.889+0000 7f6cc6800640 1 --2- 172.21.15.83:0/345329434 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6cc01042a0 0x7f6cc019a700 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:33:44.015 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:42.889+0000 7f6cc6800640 1 --2- 172.21.15.83:0/345329434 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6cc0103900 0x7f6cc019a1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:33:44.015 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:42.889+0000 7f6cc6800640 1 -- 172.21.15.83:0/345329434 >> 172.21.15.83:0/345329434 conn(0x7f6cc00ff600 msgr2=0x7f6cc0100e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:33:44.015 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:42.889+0000 7f6cc6800640 1 -- 172.21.15.83:0/345329434 shutdown_connections 2024-08-19T01:33:44.015 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:42.889+0000 7f6cc6800640 1 -- 172.21.15.83:0/345329434 wait complete. 2024-08-19T01:33:44.449 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:44.447+0000 7fb2c9a00640 1 -- 172.21.15.83:0/325705515 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb2c4074350 msgr2=0x7fb2c4074730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:33:44.449 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:44.447+0000 7fb2c9a00640 1 --2- 172.21.15.83:0/325705515 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb2c4074350 0x7fb2c4074730 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fb2b4009900 tx=0x7fb2b402a480 comp rx=0 tx=0).stop 2024-08-19T01:33:44.449 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:44.447+0000 7fb2c9a00640 1 -- 172.21.15.83:0/325705515 shutdown_connections 2024-08-19T01:33:44.449 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:44.447+0000 7fb2c9a00640 1 --2- 172.21.15.83:0/325705515 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb2c410e0f0 0x7fb2c4110540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:33:44.449 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:44.447+0000 7fb2c9a00640 1 --2- 172.21.15.83:0/325705515 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb2c4074c70 0x7fb2c410dbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:33:44.449 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:44.447+0000 7fb2c9a00640 1 --2- 172.21.15.83:0/325705515 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb2c4074350 0x7fb2c4074730 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:33:44.449 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:44.447+0000 7fb2c9a00640 1 -- 172.21.15.83:0/325705515 >> 172.21.15.83:0/325705515 conn(0x7fb2c40fd5f0 msgr2=0x7fb2c40ffa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:33:44.449 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:44.447+0000 7fb2c9a00640 1 -- 172.21.15.83:0/325705515 shutdown_connections 2024-08-19T01:33:44.450 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:44.447+0000 7fb2c9a00640 1 -- 172.21.15.83:0/325705515 wait complete. 2024-08-19T01:33:44.451 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:44.448+0000 7fb2c9a00640 1 Processor -- start 2024-08-19T01:33:44.451 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:44.448+0000 7fb2c9a00640 1 -- start start 2024-08-19T01:33:44.451 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:44.449+0000 7fb2c9a00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb2c4074350 0x7fb2c419a3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:33:44.451 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:44.449+0000 7fb2c9a00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb2c4074c70 0x7fb2c419a900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:33:44.451 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:44.449+0000 7fb2c9a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb2c410e0f0 0x7fb2c419ec90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:33:44.451 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:44.449+0000 7fb2c9a00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fb2c4113ea0 con 0x7fb2c410e0f0 2024-08-19T01:33:44.451 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:44.449+0000 7fb2c9a00640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fb2c4113d20 con 0x7fb2c4074350 2024-08-19T01:33:44.451 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:44.449+0000 7fb2c9a00640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fb2c4114020 con 0x7fb2c4074c70 2024-08-19T01:33:44.451 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:44.449+0000 7fb2c3400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb2c410e0f0 0x7fb2c419ec90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:33:44.452 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:44.449+0000 7fb2c2a00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb2c4074350 0x7fb2c419a3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:33:44.452 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:44.449+0000 7fb2c3400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb2c410e0f0 0x7fb2c419ec90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:46374/0 (socket says 172.21.15.83:46374) 2024-08-19T01:33:44.452 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:44.449+0000 7fb2c3400640 1 -- 172.21.15.83:0/1770058293 learned_addr learned my addr 172.21.15.83:0/1770058293 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-08-19T01:33:44.452 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:44.449+0000 7fb2c2000640 1 --2- 172.21.15.83:0/1770058293 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb2c4074c70 0x7fb2c419a900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:33:44.452 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:44.449+0000 7fb2c3400640 1 -- 172.21.15.83:0/1770058293 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb2c4074c70 msgr2=0x7fb2c419a900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:33:44.452 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:44.449+0000 7fb2c3400640 1 --2- 172.21.15.83:0/1770058293 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb2c4074c70 0x7fb2c419a900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:33:44.452 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:44.449+0000 7fb2c3400640 1 -- 172.21.15.83:0/1770058293 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb2c4074350 msgr2=0x7fb2c419a3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:33:44.452 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:44.449+0000 7fb2c3400640 1 --2- 172.21.15.83:0/1770058293 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb2c4074350 0x7fb2c419a3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:33:44.452 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:44.449+0000 7fb2c3400640 1 -- 172.21.15.83:0/1770058293 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb2c419f370 con 0x7fb2c410e0f0 2024-08-19T01:33:44.452 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:44.449+0000 7fb2c2000640 1 --2- 172.21.15.83:0/1770058293 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb2c4074c70 0x7fb2c419a900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-19T01:33:44.452 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:44.450+0000 7fb2c3400640 1 --2- 172.21.15.83:0/1770058293 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb2c410e0f0 0x7fb2c419ec90 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fb2bc00eb10 tx=0x7fb2bc00c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:33:44.453 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:44.450+0000 7fb2bb400640 1 -- 172.21.15.83:0/1770058293 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2bc004b20 con 0x7fb2c410e0f0 2024-08-19T01:33:44.453 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:44.450+0000 7fb2bb400640 1 -- 172.21.15.83:0/1770058293 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb2bc004cc0 con 0x7fb2c410e0f0 2024-08-19T01:33:44.453 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:44.450+0000 7fb2bb400640 1 -- 172.21.15.83:0/1770058293 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2bc010760 con 0x7fb2c410e0f0 2024-08-19T01:33:44.453 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:44.450+0000 7fb2c9a00640 1 -- 172.21.15.83:0/1770058293 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb2c419f660 con 0x7fb2c410e0f0 2024-08-19T01:33:44.453 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:44.450+0000 7fb2c9a00640 1 -- 172.21.15.83:0/1770058293 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb2c41a6f40 con 0x7fb2c410e0f0 2024-08-19T01:33:44.453 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:44.451+0000 7fb2c9a00640 1 -- 172.21.15.83:0/1770058293 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb2c4074730 con 0x7fb2c410e0f0 2024-08-19T01:33:44.454 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:44.453+0000 7fb2bb400640 1 -- 172.21.15.83:0/1770058293 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7fb2bc005020 con 0x7fb2c410e0f0 2024-08-19T01:33:44.456 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:44.454+0000 7fb2bb400640 1 --2- 172.21.15.83:0/1770058293 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7fb2a80a2c90 0x7fb2a80a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:33:44.456 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:44.454+0000 7fb2bb400640 1 -- 172.21.15.83:0/1770058293 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8269+0+0 (secure 0 0 0) 0x7fb2bc0cf790 con 0x7fb2c410e0f0 2024-08-19T01:33:44.456 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:44.455+0000 7fb2c2a00640 1 --2- 172.21.15.83:0/1770058293 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7fb2a80a2c90 0x7fb2a80a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:33:44.457 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:44.455+0000 7fb2c2a00640 1 --2- 172.21.15.83:0/1770058293 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7fb2a80a2c90 0x7fb2a80a5150 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fb2b4009230 tx=0x7fb2b40058d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:33:44.460 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:44.459+0000 7fb2bb400640 1 -- 172.21.15.83:0/1770058293 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7fb2bc09c510 con 0x7fb2c410e0f0 2024-08-19T01:33:44.655 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:44.653+0000 7fb2c9a00640 1 -- 172.21.15.83:0/1770058293 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) -- 0x7fb2c41a71e0 con 0x7fb2c410e0f0 2024-08-19T01:33:44.655 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:44.654+0000 7fb2bb400640 1 -- 172.21.15.83:0/1770058293 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fb2bc09be10 con 0x7fb2c410e0f0 2024-08-19T01:33:44.656 INFO:teuthology.orchestra.run.smithi083.stdout:77309411397 2024-08-19T01:33:44.657 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:44.656+0000 7fb2c9a00640 1 -- 172.21.15.83:0/1770058293 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7fb2a80a2c90 msgr2=0x7fb2a80a5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:33:44.658 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:44.656+0000 7fb2c9a00640 1 --2- 172.21.15.83:0/1770058293 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7fb2a80a2c90 0x7fb2a80a5150 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fb2b4009230 tx=0x7fb2b40058d0 comp rx=0 tx=0).stop 2024-08-19T01:33:44.658 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:44.656+0000 7fb2c9a00640 1 -- 172.21.15.83:0/1770058293 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb2c410e0f0 msgr2=0x7fb2c419ec90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:33:44.658 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:44.656+0000 7fb2c9a00640 1 --2- 172.21.15.83:0/1770058293 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb2c410e0f0 0x7fb2c419ec90 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fb2bc00eb10 tx=0x7fb2bc00c2d0 comp rx=0 tx=0).stop 2024-08-19T01:33:44.658 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:44.656+0000 7fb2c9a00640 1 -- 172.21.15.83:0/1770058293 shutdown_connections 2024-08-19T01:33:44.658 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:44.656+0000 7fb2c9a00640 1 --2- 172.21.15.83:0/1770058293 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7fb2a80a2c90 0x7fb2a80a5150 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:33:44.658 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:44.656+0000 7fb2c9a00640 1 --2- 172.21.15.83:0/1770058293 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb2c410e0f0 0x7fb2c419ec90 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:33:44.658 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:44.656+0000 7fb2c9a00640 1 --2- 172.21.15.83:0/1770058293 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb2c4074c70 0x7fb2c419a900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:33:44.658 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:44.656+0000 7fb2c9a00640 1 --2- 172.21.15.83:0/1770058293 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb2c4074350 0x7fb2c419a3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:33:44.658 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:44.656+0000 7fb2c9a00640 1 -- 172.21.15.83:0/1770058293 >> 172.21.15.83:0/1770058293 conn(0x7fb2c40fd5f0 msgr2=0x7fb2c40fecd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:33:44.658 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:44.656+0000 7fb2c9a00640 1 -- 172.21.15.83:0/1770058293 shutdown_connections 2024-08-19T01:33:44.658 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:44.656+0000 7fb2c9a00640 1 -- 172.21.15.83:0/1770058293 wait complete. 2024-08-19T01:33:45.314 INFO:tasks.cephadm.ceph_manager.ceph:need seq 219043332137 got 219043332139 for osd.7 2024-08-19T01:33:45.314 DEBUG:teuthology.parallel:result is None 2024-08-19T01:33:45.537 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:42.094+0000 7f3bb8e00640 1 -- 172.21.15.83:0/135352323 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3bb4102250 msgr2=0x7f3bb410e780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:33:45.537 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:42.094+0000 7f3bb8e00640 1 --2- 172.21.15.83:0/135352323 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3bb4102250 0x7f3bb410e780 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f3bac009cd0 tx=0x7f3bac02a8e0 comp rx=0 tx=0).stop 2024-08-19T01:33:45.537 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:42.094+0000 7f3bb8e00640 1 -- 172.21.15.83:0/135352323 shutdown_connections 2024-08-19T01:33:45.537 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:42.094+0000 7f3bb8e00640 1 --2- 172.21.15.83:0/135352323 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3bb4102250 0x7f3bb410e780 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:33:45.537 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:42.094+0000 7f3bb8e00640 1 --2- 172.21.15.83:0/135352323 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3bb41018b0 0x7f3bb4101d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:33:45.537 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:42.094+0000 7f3bb8e00640 1 --2- 172.21.15.83:0/135352323 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3bb41078b0 0x7f3bb4107c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:33:45.538 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:42.094+0000 7f3bb8e00640 1 -- 172.21.15.83:0/135352323 >> 172.21.15.83:0/135352323 conn(0x7f3bb40fd5f0 msgr2=0x7f3bb40ffa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:33:45.538 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:42.094+0000 7f3bb8e00640 1 -- 172.21.15.83:0/135352323 shutdown_connections 2024-08-19T01:33:45.538 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:42.094+0000 7f3bb8e00640 1 -- 172.21.15.83:0/135352323 wait complete. 2024-08-19T01:33:45.538 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:42.095+0000 7f3bb8e00640 1 Processor -- start 2024-08-19T01:33:45.538 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:42.096+0000 7f3bb8e00640 1 -- start start 2024-08-19T01:33:45.538 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:42.096+0000 7f3bb8e00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3bb41018b0 0x7f3bb4198230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:33:45.538 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:42.096+0000 7f3bb8e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3bb4102250 0x7f3bb4198770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:33:45.538 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:42.096+0000 7f3bb8e00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3bb41078b0 0x7f3bb419cb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:33:45.538 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:42.096+0000 7f3bb8e00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f3bb4111cb0 con 0x7f3bb4102250 2024-08-19T01:33:45.538 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:42.096+0000 7f3bb8e00640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f3bb4111b30 con 0x7f3bb41018b0 2024-08-19T01:33:45.539 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:42.096+0000 7f3bb8e00640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f3bb4111e30 con 0x7f3bb41078b0 2024-08-19T01:33:45.539 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:42.096+0000 7f3bb1400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3bb4102250 0x7f3bb4198770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:33:45.539 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:42.096+0000 7f3bb1400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3bb4102250 0x7f3bb4198770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:46306/0 (socket says 172.21.15.83:46306) 2024-08-19T01:33:45.539 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:42.096+0000 7f3bb1400640 1 -- 172.21.15.83:0/3929494168 learned_addr learned my addr 172.21.15.83:0/3929494168 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-08-19T01:33:45.539 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:42.096+0000 7f3bb1e00640 1 --2- 172.21.15.83:0/3929494168 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3bb41018b0 0x7f3bb4198230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:33:45.539 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:42.096+0000 7f3bb2800640 1 --2- 172.21.15.83:0/3929494168 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3bb41078b0 0x7f3bb419cb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:33:45.539 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:42.097+0000 7f3bb1400640 1 -- 172.21.15.83:0/3929494168 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3bb41078b0 msgr2=0x7f3bb419cb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:33:45.539 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:42.097+0000 7f3bb1400640 1 --2- 172.21.15.83:0/3929494168 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3bb41078b0 0x7f3bb419cb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:33:45.539 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:42.097+0000 7f3bb1400640 1 -- 172.21.15.83:0/3929494168 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3bb41018b0 msgr2=0x7f3bb4198230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:33:45.539 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:42.097+0000 7f3bb1400640 1 --2- 172.21.15.83:0/3929494168 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3bb41018b0 0x7f3bb4198230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:33:45.539 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:42.097+0000 7f3bb1400640 1 -- 172.21.15.83:0/3929494168 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3bb419d1e0 con 0x7f3bb4102250 2024-08-19T01:33:45.540 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:42.097+0000 7f3bb1400640 1 --2- 172.21.15.83:0/3929494168 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3bb4102250 0x7f3bb4198770 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f3ba000e9f0 tx=0x7f3ba000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:33:45.540 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:42.097+0000 7f3baaa00640 1 -- 172.21.15.83:0/3929494168 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ba000cd80 con 0x7f3bb4102250 2024-08-19T01:33:45.540 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:42.097+0000 7f3baaa00640 1 -- 172.21.15.83:0/3929494168 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3ba0010910 con 0x7f3bb4102250 2024-08-19T01:33:45.540 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:42.097+0000 7f3baaa00640 1 -- 172.21.15.83:0/3929494168 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ba0018990 con 0x7f3bb4102250 2024-08-19T01:33:45.540 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:42.097+0000 7f3bb8e00640 1 -- 172.21.15.83:0/3929494168 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3bb419d4d0 con 0x7f3bb4102250 2024-08-19T01:33:45.540 INFO:teuthology.orchestra.run.smithi083.stdout:283467841570 2024-08-19T01:33:45.541 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:42.097+0000 7f3bb8e00640 1 -- 172.21.15.83:0/3929494168 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3bb41a4db0 con 0x7f3bb4102250 2024-08-19T01:33:45.541 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:42.098+0000 7f3bb8e00640 1 -- 172.21.15.83:0/3929494168 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3bb4101d10 con 0x7f3bb4102250 2024-08-19T01:33:45.541 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:42.100+0000 7f3baaa00640 1 -- 172.21.15.83:0/3929494168 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f3ba0010ac0 con 0x7f3bb4102250 2024-08-19T01:33:45.541 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:42.101+0000 7f3baaa00640 1 --2- 172.21.15.83:0/3929494168 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f3b980a2c10 0x7f3b980a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:33:45.541 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:42.102+0000 7f3baaa00640 1 -- 172.21.15.83:0/3929494168 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8269+0+0 (secure 0 0 0) 0x7f3ba00cf560 con 0x7f3bb4102250 2024-08-19T01:33:45.541 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:42.102+0000 7f3bb1e00640 1 --2- 172.21.15.83:0/3929494168 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f3b980a2c10 0x7f3b980a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:33:45.541 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:42.102+0000 7f3bb1e00640 1 --2- 172.21.15.83:0/3929494168 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f3b980a2c10 0x7f3b980a50d0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f3ba400bb60 tx=0x7f3ba400b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:33:45.541 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:42.105+0000 7f3baaa00640 1 -- 172.21.15.83:0/3929494168 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7f3ba009bc90 con 0x7f3bb4102250 2024-08-19T01:33:45.542 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:42.294+0000 7f3bb8e00640 1 -- 172.21.15.83:0/3929494168 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 9} v 0) -- 0x7f3bb4109e70 con 0x7f3bb4102250 2024-08-19T01:33:45.542 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:42.295+0000 7f3baaa00640 1 -- 172.21.15.83:0/3929494168 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 9}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f3ba009ff90 con 0x7f3bb4102250 2024-08-19T01:33:45.542 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:42.297+0000 7f3bb8e00640 1 -- 172.21.15.83:0/3929494168 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f3b980a2c10 msgr2=0x7f3b980a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:33:45.542 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:42.297+0000 7f3bb8e00640 1 --2- 172.21.15.83:0/3929494168 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f3b980a2c10 0x7f3b980a50d0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f3ba400bb60 tx=0x7f3ba400b390 comp rx=0 tx=0).stop 2024-08-19T01:33:45.542 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:42.297+0000 7f3bb8e00640 1 -- 172.21.15.83:0/3929494168 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3bb4102250 msgr2=0x7f3bb4198770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:33:45.542 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:42.297+0000 7f3bb8e00640 1 --2- 172.21.15.83:0/3929494168 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3bb4102250 0x7f3bb4198770 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f3ba000e9f0 tx=0x7f3ba000eeb0 comp rx=0 tx=0).stop 2024-08-19T01:33:45.542 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:42.297+0000 7f3bb8e00640 1 -- 172.21.15.83:0/3929494168 shutdown_connections 2024-08-19T01:33:45.542 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:42.297+0000 7f3bb8e00640 1 --2- 172.21.15.83:0/3929494168 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f3b980a2c10 0x7f3b980a50d0 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:33:45.542 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:42.297+0000 7f3bb8e00640 1 --2- 172.21.15.83:0/3929494168 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3bb41078b0 0x7f3bb419cb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:33:45.542 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:42.297+0000 7f3bb8e00640 1 --2- 172.21.15.83:0/3929494168 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3bb4102250 0x7f3bb4198770 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:33:45.542 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:42.297+0000 7f3bb8e00640 1 --2- 172.21.15.83:0/3929494168 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3bb41018b0 0x7f3bb4198230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:33:45.543 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:42.297+0000 7f3bb8e00640 1 -- 172.21.15.83:0/3929494168 >> 172.21.15.83:0/3929494168 conn(0x7f3bb40fd5f0 msgr2=0x7f3bb40fedd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:33:45.543 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:42.297+0000 7f3bb8e00640 1 -- 172.21.15.83:0/3929494168 shutdown_connections 2024-08-19T01:33:45.543 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:42.297+0000 7f3bb8e00640 1 -- 172.21.15.83:0/3929494168 wait complete. 2024-08-19T01:33:45.834 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:33:45 smithi083 ceph-mon[17120]: pgmap v306: 97 pgs: 97 active+clean; 640 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-08-19T01:33:45.834 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:33:45 smithi083 ceph-mon[17120]: from='client.? 172.21.15.83:0/1770058293' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-08-19T01:33:45.952 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:33:45 smithi117 ceph-mon[22016]: pgmap v306: 97 pgs: 97 active+clean; 640 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-08-19T01:33:45.952 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:33:45 smithi117 ceph-mon[22016]: from='client.? 172.21.15.83:0/1770058293' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-08-19T01:33:45.966 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:33:45 smithi167 ceph-mon[21875]: pgmap v306: 97 pgs: 97 active+clean; 640 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-08-19T01:33:45.966 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:33:45 smithi167 ceph-mon[21875]: from='client.? 172.21.15.83:0/1770058293' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-08-19T01:33:46.041 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:45.530+0000 7f267f200640 1 -- 172.21.15.83:0/3014976322 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f26801042a0 msgr2=0x7f26801107d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:33:46.041 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:45.530+0000 7f267f200640 1 --2- 172.21.15.83:0/3014976322 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f26801042a0 0x7f26801107d0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f2678009cd0 tx=0x7f267802a8e0 comp rx=0 tx=0).stop 2024-08-19T01:33:46.041 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:45.530+0000 7f267f200640 1 -- 172.21.15.83:0/3014976322 shutdown_connections 2024-08-19T01:33:46.042 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:45.530+0000 7f267f200640 1 --2- 172.21.15.83:0/3014976322 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f26801042a0 0x7f26801107d0 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:33:46.042 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:45.530+0000 7f267f200640 1 --2- 172.21.15.83:0/3014976322 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2680103900 0x7f2680103d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:33:46.042 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:45.530+0000 7f267f200640 1 --2- 172.21.15.83:0/3014976322 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2680109900 0x7f2680109ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:33:46.042 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:45.530+0000 7f267f200640 1 -- 172.21.15.83:0/3014976322 >> 172.21.15.83:0/3014976322 conn(0x7f26800ff600 msgr2=0x7f2680101a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:33:46.042 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:45.530+0000 7f267f200640 1 -- 172.21.15.83:0/3014976322 shutdown_connections 2024-08-19T01:33:46.042 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:45.531+0000 7f267f200640 1 -- 172.21.15.83:0/3014976322 wait complete. 2024-08-19T01:33:46.042 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:45.532+0000 7f267f200640 1 Processor -- start 2024-08-19T01:33:46.042 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:45.532+0000 7f267f200640 1 -- start start 2024-08-19T01:33:46.042 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:45.532+0000 7f267f200640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2680103900 0x7f268019a150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:33:46.042 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:45.532+0000 7f267f200640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f26801042a0 0x7f268019a690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:33:46.042 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:45.532+0000 7f267f200640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2680109900 0x7f268019ea20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:33:46.042 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:45.532+0000 7f267f200640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f2680113d00 con 0x7f26801042a0 2024-08-19T01:33:46.043 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:45.532+0000 7f267f200640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f2680113b80 con 0x7f2680103900 2024-08-19T01:33:46.043 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:45.532+0000 7f267f200640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f2680113e80 con 0x7f2680109900 2024-08-19T01:33:46.043 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:45.532+0000 7f267e800640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2680109900 0x7f268019ea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:33:46.043 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:45.532+0000 7f267d400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f26801042a0 0x7f268019a690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:33:46.043 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:45.532+0000 7f267e800640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2680109900 0x7f268019ea20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.83:36244/0 (socket says 172.21.15.83:36244) 2024-08-19T01:33:46.043 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:45.532+0000 7f267e800640 1 -- 172.21.15.83:0/1897404243 learned_addr learned my addr 172.21.15.83:0/1897404243 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-08-19T01:33:46.043 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:45.532+0000 7f267de00640 1 --2- 172.21.15.83:0/1897404243 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2680103900 0x7f268019a150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:33:46.043 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:45.533+0000 7f267d400640 1 -- 172.21.15.83:0/1897404243 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2680109900 msgr2=0x7f268019ea20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:33:46.043 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:45.533+0000 7f267d400640 1 --2- 172.21.15.83:0/1897404243 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2680109900 0x7f268019ea20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:33:46.043 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:45.533+0000 7f267d400640 1 -- 172.21.15.83:0/1897404243 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2680103900 msgr2=0x7f268019a150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:33:46.043 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:45.533+0000 7f267d400640 1 --2- 172.21.15.83:0/1897404243 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2680103900 0x7f268019a150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:33:46.043 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:45.533+0000 7f267d400640 1 -- 172.21.15.83:0/1897404243 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f268019f100 con 0x7f26801042a0 2024-08-19T01:33:46.043 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:45.533+0000 7f267d400640 1 --2- 172.21.15.83:0/1897404243 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f26801042a0 0x7f268019a690 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f266c00d6d0 tx=0x7f266c00db90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:33:46.044 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:45.533+0000 7f2676a00640 1 -- 172.21.15.83:0/1897404243 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f266c017070 con 0x7f26801042a0 2024-08-19T01:33:46.044 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:45.533+0000 7f2676a00640 1 -- 172.21.15.83:0/1897404243 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f266c009820 con 0x7f26801042a0 2024-08-19T01:33:46.044 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:45.533+0000 7f2676a00640 1 -- 172.21.15.83:0/1897404243 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f266c004d00 con 0x7f26801042a0 2024-08-19T01:33:46.044 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:45.533+0000 7f267f200640 1 -- 172.21.15.83:0/1897404243 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f268019f3f0 con 0x7f26801042a0 2024-08-19T01:33:46.044 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:45.534+0000 7f267f200640 1 -- 172.21.15.83:0/1897404243 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f26801a6cd0 con 0x7f26801042a0 2024-08-19T01:33:46.044 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:45.535+0000 7f267f200640 1 -- 172.21.15.83:0/1897404243 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2680103d60 con 0x7f26801042a0 2024-08-19T01:33:46.044 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:45.538+0000 7f2676a00640 1 -- 172.21.15.83:0/1897404243 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f266c0099d0 con 0x7f26801042a0 2024-08-19T01:33:46.044 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:45.539+0000 7f2676a00640 1 --2- 172.21.15.83:0/1897404243 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f26640a2c90 0x7f26640a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:33:46.044 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:45.539+0000 7f2676a00640 1 -- 172.21.15.83:0/1897404243 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8269+0+0 (secure 0 0 0) 0x7f266c0cefe0 con 0x7f26801042a0 2024-08-19T01:33:46.044 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:45.540+0000 7f267de00640 1 --2- 172.21.15.83:0/1897404243 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f26640a2c90 0x7f26640a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:33:46.044 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:45.540+0000 7f267de00640 1 --2- 172.21.15.83:0/1897404243 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f26640a2c90 0x7f26640a5150 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f26700058c0 tx=0x7f267000a2d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:33:46.044 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:45.541+0000 7f2676a00640 1 -- 172.21.15.83:0/1897404243 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7f266c09b890 con 0x7f26801042a0 2024-08-19T01:33:46.044 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:45.720+0000 7f267f200640 1 -- 172.21.15.83:0/1897404243 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) -- 0x7f268002d5e0 con 0x7f26801042a0 2024-08-19T01:33:46.045 INFO:teuthology.orchestra.run.smithi083.stdout:197568495666 2024-08-19T01:33:46.045 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:45.720+0000 7f2676a00640 1 -- 172.21.15.83:0/1897404243 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f266c09fb90 con 0x7f26801042a0 2024-08-19T01:33:46.045 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:45.722+0000 7f267f200640 1 -- 172.21.15.83:0/1897404243 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f26640a2c90 msgr2=0x7f26640a5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:33:46.045 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:45.722+0000 7f267f200640 1 --2- 172.21.15.83:0/1897404243 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f26640a2c90 0x7f26640a5150 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f26700058c0 tx=0x7f267000a2d0 comp rx=0 tx=0).stop 2024-08-19T01:33:46.045 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:45.722+0000 7f267f200640 1 -- 172.21.15.83:0/1897404243 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f26801042a0 msgr2=0x7f268019a690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:33:46.046 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:45.722+0000 7f267f200640 1 --2- 172.21.15.83:0/1897404243 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f26801042a0 0x7f268019a690 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f266c00d6d0 tx=0x7f266c00db90 comp rx=0 tx=0).stop 2024-08-19T01:33:46.046 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:45.722+0000 7f267f200640 1 -- 172.21.15.83:0/1897404243 shutdown_connections 2024-08-19T01:33:46.046 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:45.722+0000 7f267f200640 1 --2- 172.21.15.83:0/1897404243 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f26640a2c90 0x7f26640a5150 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:33:46.046 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:45.722+0000 7f267f200640 1 --2- 172.21.15.83:0/1897404243 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2680109900 0x7f268019ea20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:33:46.046 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:45.722+0000 7f267f200640 1 --2- 172.21.15.83:0/1897404243 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f26801042a0 0x7f268019a690 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:33:46.046 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:45.723+0000 7f267f200640 1 --2- 172.21.15.83:0/1897404243 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2680103900 0x7f268019a150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:33:46.046 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:45.723+0000 7f267f200640 1 -- 172.21.15.83:0/1897404243 >> 172.21.15.83:0/1897404243 conn(0x7f26800ff600 msgr2=0x7f2680100e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:33:46.046 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:45.723+0000 7f267f200640 1 -- 172.21.15.83:0/1897404243 shutdown_connections 2024-08-19T01:33:46.046 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:45.723+0000 7f267f200640 1 -- 172.21.15.83:0/1897404243 wait complete. 2024-08-19T01:33:46.905 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:33:46 smithi083 ceph-mon[17120]: from='client.? 172.21.15.83:0/1897404243' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-08-19T01:33:46.905 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:33:46 smithi083 ceph-mon[17120]: pgmap v307: 97 pgs: 97 active+clean; 640 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-08-19T01:33:46.947 INFO:tasks.cephadm.ceph_manager.ceph:need seq 141733920823 got 141733920826 for osd.4 2024-08-19T01:33:46.947 DEBUG:teuthology.parallel:result is None 2024-08-19T01:33:46.952 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:33:46 smithi117 ceph-mon[22016]: from='client.? 172.21.15.83:0/1897404243' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-08-19T01:33:46.952 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:33:46 smithi117 ceph-mon[22016]: pgmap v307: 97 pgs: 97 active+clean; 640 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-08-19T01:33:46.966 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:33:46 smithi167 ceph-mon[21875]: from='client.? 172.21.15.83:0/1897404243' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-08-19T01:33:46.966 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:33:46 smithi167 ceph-mon[21875]: pgmap v307: 97 pgs: 97 active+clean; 640 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-08-19T01:33:47.268 INFO:tasks.cephadm.ceph_manager.ceph:need seq 77309411394 got 77309411397 for osd.2 2024-08-19T01:33:47.268 DEBUG:teuthology.parallel:result is None 2024-08-19T01:33:47.599 INFO:tasks.cephadm.ceph_manager.ceph:need seq 107374182459 got 107374182463 for osd.3 2024-08-19T01:33:47.599 DEBUG:teuthology.parallel:result is None 2024-08-19T01:33:48.078 INFO:tasks.cephadm.ceph_manager.ceph:need seq 163208757300 got 163208757301 for osd.5 2024-08-19T01:33:48.079 DEBUG:teuthology.parallel:result is None 2024-08-19T01:33:48.441 INFO:tasks.cephadm.ceph_manager.ceph:need seq 283467841567 got 283467841570 for osd.9 2024-08-19T01:33:48.441 DEBUG:teuthology.parallel:result is None 2024-08-19T01:33:49.015 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/bad8780e-5dc9-11ef-bccf-c7b262605968/mon.a/config 2024-08-19T01:33:49.044 INFO:tasks.cephadm.ceph_manager.ceph:need seq 197568495662 got 197568495666 for osd.6 2024-08-19T01:33:49.044 DEBUG:teuthology.parallel:result is None 2024-08-19T01:33:49.574 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:33:49 smithi083 ceph-mon[17120]: pgmap v308: 97 pgs: 97 active+clean; 640 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-08-19T01:33:49.575 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:49.572+0000 7fb72ce00640 1 -- 172.21.15.83:0/645434792 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb7281042a0 msgr2=0x7fb7281107d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:33:49.575 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:49.572+0000 7fb72ce00640 1 --2- 172.21.15.83:0/645434792 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb7281042a0 0x7fb7281107d0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fb720009cd0 tx=0x7fb72002a8e0 comp rx=0 tx=0).stop 2024-08-19T01:33:49.575 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:49.573+0000 7fb72ce00640 1 -- 172.21.15.83:0/645434792 shutdown_connections 2024-08-19T01:33:49.575 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:49.573+0000 7fb72ce00640 1 --2- 172.21.15.83:0/645434792 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb7281042a0 0x7fb7281107d0 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:33:49.575 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:49.573+0000 7fb72ce00640 1 --2- 172.21.15.83:0/645434792 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb728103900 0x7fb728103d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:33:49.575 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:49.573+0000 7fb72ce00640 1 --2- 172.21.15.83:0/645434792 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb728109900 0x7fb728109ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:33:49.575 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:49.573+0000 7fb72ce00640 1 -- 172.21.15.83:0/645434792 >> 172.21.15.83:0/645434792 conn(0x7fb7280ff600 msgr2=0x7fb728101a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:33:49.575 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:49.573+0000 7fb72ce00640 1 -- 172.21.15.83:0/645434792 shutdown_connections 2024-08-19T01:33:49.575 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:49.573+0000 7fb72ce00640 1 -- 172.21.15.83:0/645434792 wait complete. 2024-08-19T01:33:49.577 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:49.574+0000 7fb72ce00640 1 Processor -- start 2024-08-19T01:33:49.577 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:49.574+0000 7fb72ce00640 1 -- start start 2024-08-19T01:33:49.577 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:49.574+0000 7fb72ce00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb728103900 0x7fb72819a1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:33:49.577 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:49.574+0000 7fb72ce00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb7281042a0 0x7fb72819a6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:33:49.577 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:49.574+0000 7fb72ce00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb728109900 0x7fb72819ea80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:33:49.577 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:49.574+0000 7fb72ce00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fb728113d50 con 0x7fb7281042a0 2024-08-19T01:33:49.577 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:49.574+0000 7fb72ce00640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fb728113bd0 con 0x7fb728103900 2024-08-19T01:33:49.577 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:49.574+0000 7fb72ce00640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fb728113ed0 con 0x7fb728109900 2024-08-19T01:33:49.577 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:49.575+0000 7fb727400640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb728103900 0x7fb72819a1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:33:49.577 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:49.575+0000 7fb71ea00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb7281042a0 0x7fb72819a6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:33:49.577 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:49.575+0000 7fb727400640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb728103900 0x7fb72819a1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.83:53924/0 (socket says 172.21.15.83:53924) 2024-08-19T01:33:49.577 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:49.575+0000 7fb727400640 1 -- 172.21.15.83:0/926450907 learned_addr learned my addr 172.21.15.83:0/926450907 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-08-19T01:33:49.578 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:49.575+0000 7fb727e00640 1 --2- 172.21.15.83:0/926450907 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb728109900 0x7fb72819ea80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:33:49.578 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:49.575+0000 7fb727400640 1 -- 172.21.15.83:0/926450907 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb728109900 msgr2=0x7fb72819ea80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:33:49.578 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:49.575+0000 7fb727400640 1 --2- 172.21.15.83:0/926450907 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb728109900 0x7fb72819ea80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:33:49.578 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:49.575+0000 7fb727400640 1 -- 172.21.15.83:0/926450907 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb7281042a0 msgr2=0x7fb72819a6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:33:49.578 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:49.575+0000 7fb727400640 1 --2- 172.21.15.83:0/926450907 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb7281042a0 0x7fb72819a6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:33:49.578 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:49.575+0000 7fb727400640 1 -- 172.21.15.83:0/926450907 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb72819f160 con 0x7fb728103900 2024-08-19T01:33:49.578 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:49.575+0000 7fb727e00640 1 --2- 172.21.15.83:0/926450907 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb728109900 0x7fb72819ea80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-19T01:33:49.578 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:49.575+0000 7fb727400640 1 --2- 172.21.15.83:0/926450907 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb728103900 0x7fb72819a1b0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fb71800e9f0 tx=0x7fb71800eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:33:49.579 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:49.576+0000 7fb724c00640 1 -- 172.21.15.83:0/926450907 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb71800cd80 con 0x7fb728103900 2024-08-19T01:33:49.579 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:49.576+0000 7fb724c00640 1 -- 172.21.15.83:0/926450907 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb718010910 con 0x7fb728103900 2024-08-19T01:33:49.579 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:49.576+0000 7fb72ce00640 1 -- 172.21.15.83:0/926450907 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb72819f450 con 0x7fb728103900 2024-08-19T01:33:49.579 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:49.576+0000 7fb724c00640 1 -- 172.21.15.83:0/926450907 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb718018990 con 0x7fb728103900 2024-08-19T01:33:49.579 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:49.576+0000 7fb72ce00640 1 -- 172.21.15.83:0/926450907 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb7281a6d30 con 0x7fb728103900 2024-08-19T01:33:49.579 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:49.577+0000 7fb72ce00640 1 -- 172.21.15.83:0/926450907 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb728103d60 con 0x7fb728103900 2024-08-19T01:33:49.580 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:49.579+0000 7fb724c00640 1 -- 172.21.15.83:0/926450907 <== mon.2 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7fb718010ac0 con 0x7fb728103900 2024-08-19T01:33:49.581 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:49.580+0000 7fb724c00640 1 --2- 172.21.15.83:0/926450907 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7fb70c0a2c10 0x7fb70c0a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:33:49.582 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:49.580+0000 7fb724c00640 1 -- 172.21.15.83:0/926450907 <== mon.2 v2:172.21.15.117:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8269+0+0 (secure 0 0 0) 0x7fb7180cf590 con 0x7fb728103900 2024-08-19T01:33:49.582 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:49.580+0000 7fb71ea00640 1 --2- 172.21.15.83:0/926450907 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7fb70c0a2c10 0x7fb70c0a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:33:49.582 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:49.581+0000 7fb71ea00640 1 --2- 172.21.15.83:0/926450907 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7fb70c0a2c10 0x7fb70c0a50d0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fb728101bd0 tx=0x7fb71000b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:33:49.585 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:49.584+0000 7fb724c00640 1 -- 172.21.15.83:0/926450907 <== mon.2 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7fb71809c310 con 0x7fb728103900 2024-08-19T01:33:49.685 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:33:49 smithi167 ceph-mon[21875]: pgmap v308: 97 pgs: 97 active+clean; 640 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-08-19T01:33:49.702 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:33:49 smithi117 ceph-mon[22016]: pgmap v308: 97 pgs: 97 active+clean; 640 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-08-19T01:33:49.784 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:49.782+0000 7fb72ce00640 1 -- 172.21.15.83:0/926450907 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 8} v 0) -- 0x7fb72819b510 con 0x7fb728103900 2024-08-19T01:33:49.784 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:49.783+0000 7fb724c00640 1 -- 172.21.15.83:0/926450907 <== mon.2 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 8}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7fb71809bc10 con 0x7fb728103900 2024-08-19T01:33:49.785 INFO:teuthology.orchestra.run.smithi083.stdout:249108103209 2024-08-19T01:33:49.787 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:49.785+0000 7fb72ce00640 1 -- 172.21.15.83:0/926450907 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7fb70c0a2c10 msgr2=0x7fb70c0a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:33:49.787 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:49.785+0000 7fb72ce00640 1 --2- 172.21.15.83:0/926450907 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7fb70c0a2c10 0x7fb70c0a50d0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fb728101bd0 tx=0x7fb71000b410 comp rx=0 tx=0).stop 2024-08-19T01:33:49.787 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:49.785+0000 7fb72ce00640 1 -- 172.21.15.83:0/926450907 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb728103900 msgr2=0x7fb72819a1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:33:49.787 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:49.785+0000 7fb72ce00640 1 --2- 172.21.15.83:0/926450907 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb728103900 0x7fb72819a1b0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fb71800e9f0 tx=0x7fb71800eeb0 comp rx=0 tx=0).stop 2024-08-19T01:33:49.787 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:49.785+0000 7fb72ce00640 1 -- 172.21.15.83:0/926450907 shutdown_connections 2024-08-19T01:33:49.787 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:49.785+0000 7fb72ce00640 1 --2- 172.21.15.83:0/926450907 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7fb70c0a2c10 0x7fb70c0a50d0 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:33:49.788 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:49.785+0000 7fb72ce00640 1 --2- 172.21.15.83:0/926450907 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb728109900 0x7fb72819ea80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:33:49.788 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:49.785+0000 7fb72ce00640 1 --2- 172.21.15.83:0/926450907 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb7281042a0 0x7fb72819a6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:33:49.788 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:49.785+0000 7fb72ce00640 1 --2- 172.21.15.83:0/926450907 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb728103900 0x7fb72819a1b0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:33:49.788 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:49.785+0000 7fb72ce00640 1 -- 172.21.15.83:0/926450907 >> 172.21.15.83:0/926450907 conn(0x7fb7280ff600 msgr2=0x7fb7280ffdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:33:49.788 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:49.785+0000 7fb72ce00640 1 -- 172.21.15.83:0/926450907 shutdown_connections 2024-08-19T01:33:49.788 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:49.786+0000 7fb72ce00640 1 -- 172.21.15.83:0/926450907 wait complete. 2024-08-19T01:33:50.654 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:33:50 smithi083 ceph-mon[17120]: from='client.? 172.21.15.83:0/926450907' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-08-19T01:33:50.702 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:33:50 smithi117 ceph-mon[22016]: from='client.? 172.21.15.83:0/926450907' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-08-19T01:33:50.715 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:33:50 smithi167 ceph-mon[21875]: from='client.? 172.21.15.83:0/926450907' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-08-19T01:33:51.045 INFO:tasks.cephadm.ceph_manager.ceph:need seq 249108103206 got 249108103209 for osd.8 2024-08-19T01:33:51.045 DEBUG:teuthology.parallel:result is None 2024-08-19T01:33:51.045 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2024-08-19T01:33:51.045 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833 shell --fsid bad8780e-5dc9-11ef-bccf-c7b262605968 -- ceph pg dump --format=json 2024-08-19T01:33:51.443 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/bad8780e-5dc9-11ef-bccf-c7b262605968/mon.a/config 2024-08-19T01:33:51.702 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:33:51 smithi117 ceph-mon[22016]: pgmap v309: 97 pgs: 97 active+clean; 640 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-08-19T01:33:51.716 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:33:51 smithi167 ceph-mon[21875]: pgmap v309: 97 pgs: 97 active+clean; 640 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-08-19T01:33:51.739 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:33:51 smithi083 ceph-mon[17120]: pgmap v309: 97 pgs: 97 active+clean; 640 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-08-19T01:33:52.009 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:51.998+0000 7f47db200640 1 -- 172.21.15.83:0/3333454357 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f47dc1031e0 msgr2=0x7f47dc10b6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:33:52.009 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:51.998+0000 7f47db200640 1 --2- 172.21.15.83:0/3333454357 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f47dc1031e0 0x7f47dc10b6d0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f47c4009900 tx=0x7f47c402a480 comp rx=0 tx=0).stop 2024-08-19T01:33:52.009 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:51.998+0000 7f47db200640 1 -- 172.21.15.83:0/3333454357 shutdown_connections 2024-08-19T01:33:52.009 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:51.998+0000 7f47db200640 1 --2- 172.21.15.83:0/3333454357 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f47dc10bcf0 0x7f47dc10e190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:33:52.009 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:51.998+0000 7f47db200640 1 --2- 172.21.15.83:0/3333454357 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f47dc1031e0 0x7f47dc10b6d0 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:33:52.009 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:51.998+0000 7f47db200640 1 --2- 172.21.15.83:0/3333454357 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f47dc1028c0 0x7f47dc102ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:33:52.009 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:51.998+0000 7f47db200640 1 -- 172.21.15.83:0/3333454357 >> 172.21.15.83:0/3333454357 conn(0x7f47dc0fc340 msgr2=0x7f47dc0fe760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:33:52.009 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:51.998+0000 7f47db200640 1 -- 172.21.15.83:0/3333454357 shutdown_connections 2024-08-19T01:33:52.009 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:51.998+0000 7f47db200640 1 -- 172.21.15.83:0/3333454357 wait complete. 2024-08-19T01:33:52.010 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:51.999+0000 7f47db200640 1 Processor -- start 2024-08-19T01:33:52.010 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:51.999+0000 7f47db200640 1 -- start start 2024-08-19T01:33:52.010 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:51.999+0000 7f47db200640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f47dc1028c0 0x7f47dc1a3240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:33:52.010 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:51.999+0000 7f47db200640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f47dc1031e0 0x7f47dc1a3780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:33:52.010 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:51.999+0000 7f47db200640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f47dc10bcf0 0x7f47dc10a880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:33:52.010 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:51.999+0000 7f47db200640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f47dc111b40 con 0x7f47dc1028c0 2024-08-19T01:33:52.010 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:51.999+0000 7f47db200640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f47dc1119c0 con 0x7f47dc1031e0 2024-08-19T01:33:52.010 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:51.999+0000 7f47db200640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f47dc111cc0 con 0x7f47dc10bcf0 2024-08-19T01:33:52.010 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:52.000+0000 7f47d9e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f47dc1028c0 0x7f47dc1a3240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:33:52.010 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:52.000+0000 7f47d9e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f47dc1028c0 0x7f47dc1a3240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:57636/0 (socket says 172.21.15.83:57636) 2024-08-19T01:33:52.010 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:52.000+0000 7f47d9e00640 1 -- 172.21.15.83:0/4004944920 learned_addr learned my addr 172.21.15.83:0/4004944920 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-08-19T01:33:52.010 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:52.000+0000 7f47da800640 1 --2- 172.21.15.83:0/4004944920 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f47dc10bcf0 0x7f47dc10a880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:33:52.010 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:52.000+0000 7f47d3e00640 1 --2- 172.21.15.83:0/4004944920 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f47dc1031e0 0x7f47dc1a3780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:33:52.011 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:52.000+0000 7f47d9e00640 1 -- 172.21.15.83:0/4004944920 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f47dc10bcf0 msgr2=0x7f47dc10a880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:33:52.011 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:52.000+0000 7f47d9e00640 1 --2- 172.21.15.83:0/4004944920 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f47dc10bcf0 0x7f47dc10a880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:33:52.011 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:52.000+0000 7f47d9e00640 1 -- 172.21.15.83:0/4004944920 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f47dc1031e0 msgr2=0x7f47dc1a3780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:33:52.011 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:52.000+0000 7f47d9e00640 1 --2- 172.21.15.83:0/4004944920 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f47dc1031e0 0x7f47dc1a3780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:33:52.011 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:52.000+0000 7f47d9e00640 1 -- 172.21.15.83:0/4004944920 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f47dc10b000 con 0x7f47dc1028c0 2024-08-19T01:33:52.011 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:52.000+0000 7f47d3e00640 1 --2- 172.21.15.83:0/4004944920 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f47dc1031e0 0x7f47dc1a3780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-19T01:33:52.011 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:52.001+0000 7f47d9e00640 1 --2- 172.21.15.83:0/4004944920 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f47dc1028c0 0x7f47dc1a3240 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f47cc00e9f0 tx=0x7f47cc00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:33:52.011 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:52.001+0000 7f47d2a00640 1 -- 172.21.15.83:0/4004944920 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47cc00cd80 con 0x7f47dc1028c0 2024-08-19T01:33:52.011 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:52.001+0000 7f47db200640 1 -- 172.21.15.83:0/4004944920 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f47dc10b2f0 con 0x7f47dc1028c0 2024-08-19T01:33:52.011 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:52.001+0000 7f47d2a00640 1 -- 172.21.15.83:0/4004944920 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f47cc010910 con 0x7f47dc1028c0 2024-08-19T01:33:52.011 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:52.001+0000 7f47d2a00640 1 -- 172.21.15.83:0/4004944920 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47cc0188f0 con 0x7f47dc1028c0 2024-08-19T01:33:52.011 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:52.001+0000 7f47db200640 1 -- 172.21.15.83:0/4004944920 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f47dc1ab170 con 0x7f47dc1028c0 2024-08-19T01:33:52.011 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:52.002+0000 7f47db200640 1 -- 172.21.15.83:0/4004944920 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f47dc102ca0 con 0x7f47dc1028c0 2024-08-19T01:33:52.012 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:52.004+0000 7f47d2a00640 1 -- 172.21.15.83:0/4004944920 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f47cc010ac0 con 0x7f47dc1028c0 2024-08-19T01:33:52.012 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:52.005+0000 7f47d2a00640 1 --2- 172.21.15.83:0/4004944920 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f47c00a2c10 0x7f47c00a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:33:52.012 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:52.006+0000 7f47d2a00640 1 -- 172.21.15.83:0/4004944920 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8269+0+0 (secure 0 0 0) 0x7f47cc0cf5e0 con 0x7f47dc1028c0 2024-08-19T01:33:52.012 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:52.006+0000 7f47d3e00640 1 --2- 172.21.15.83:0/4004944920 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f47c00a2c10 0x7f47c00a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:33:52.012 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:52.006+0000 7f47d3e00640 1 --2- 172.21.15.83:0/4004944920 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f47c00a2c10 0x7f47c00a50d0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f47c4000c00 tx=0x7f47c4031510 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:33:52.012 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:52.010+0000 7f47d2a00640 1 -- 172.21.15.83:0/4004944920 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7f47cc09bd10 con 0x7f47dc1028c0 2024-08-19T01:33:52.199 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:52.197+0000 7f47db200640 1 -- 172.21.15.83:0/4004944920 --> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f47dc0fe660 con 0x7f47c00a2c10 2024-08-19T01:33:52.206 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:52.204+0000 7f47d2a00640 1 -- 172.21.15.83:0/4004944920 <== mgr.14150 v2:172.21.15.83:6800/683740916 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258723 (secure 0 0 0) 0x7f47dc0fe660 con 0x7f47c00a2c10 2024-08-19T01:33:52.207 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-08-19T01:33:52.212 INFO:teuthology.orchestra.run.smithi083.stderr:dumped all 2024-08-19T01:33:52.213 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:52.209+0000 7f47db200640 1 -- 172.21.15.83:0/4004944920 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f47c00a2c10 msgr2=0x7f47c00a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:33:52.214 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:52.209+0000 7f47db200640 1 --2- 172.21.15.83:0/4004944920 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f47c00a2c10 0x7f47c00a50d0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f47c4000c00 tx=0x7f47c4031510 comp rx=0 tx=0).stop 2024-08-19T01:33:52.214 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:52.209+0000 7f47db200640 1 -- 172.21.15.83:0/4004944920 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f47dc1028c0 msgr2=0x7f47dc1a3240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:33:52.214 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:52.209+0000 7f47db200640 1 --2- 172.21.15.83:0/4004944920 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f47dc1028c0 0x7f47dc1a3240 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f47cc00e9f0 tx=0x7f47cc00eeb0 comp rx=0 tx=0).stop 2024-08-19T01:33:52.214 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:52.209+0000 7f47db200640 1 -- 172.21.15.83:0/4004944920 shutdown_connections 2024-08-19T01:33:52.214 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:52.209+0000 7f47db200640 1 --2- 172.21.15.83:0/4004944920 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f47c00a2c10 0x7f47c00a50d0 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:33:52.214 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:52.209+0000 7f47db200640 1 --2- 172.21.15.83:0/4004944920 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f47dc10bcf0 0x7f47dc10a880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:33:52.214 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:52.209+0000 7f47db200640 1 --2- 172.21.15.83:0/4004944920 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f47dc1031e0 0x7f47dc1a3780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:33:52.214 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:52.209+0000 7f47db200640 1 --2- 172.21.15.83:0/4004944920 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f47dc1028c0 0x7f47dc1a3240 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:33:52.214 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:52.209+0000 7f47db200640 1 -- 172.21.15.83:0/4004944920 >> 172.21.15.83:0/4004944920 conn(0x7f47dc0fc340 msgr2=0x7f47dc0fcf40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:33:52.214 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:52.209+0000 7f47db200640 1 -- 172.21.15.83:0/4004944920 shutdown_connections 2024-08-19T01:33:52.214 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:52.209+0000 7f47db200640 1 -- 172.21.15.83:0/4004944920 wait complete. 2024-08-19T01:33:52.979 INFO:teuthology.orchestra.run.smithi083.stdout:{"pg_ready":true,"pg_map":{"version":309,"stamp":"2024-08-19T01:33:50.375104+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":655608,"num_objects":64,"num_object_clones":0,"num_object_copies":192,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":490,"num_read_kb":359,"num_write":347,"num_write_kb":3050,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":1181748,"num_keys_recovered":10,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":278,"ondisk_log_size":278,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":374792,"kb_used_data":13116,"kb_used_omap":18,"kb_used_meta":361517,"kb_avail":1124567032,"statfs":{"total":1151940427776,"available":1151556640768,"internally_reserved":0,"allocated":13430784,"data_stored":7372452,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19143,"internal_metadata":370193721},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":75,"num_read_kb":40,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002982"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:33:05.470153+0000","last_change":"2024-08-19T01:32:41.244214+0000","last_active":"2024-08-19T01:33:05.470153+0000","last_peered":"2024-08-19T01:33:05.470153+0000","last_clean":"2024-08-19T01:33:05.470153+0000","last_became_active":"2024-08-19T01:32:41.243889+0000","last_became_peered":"2024-08-19T01:32:41.243889+0000","last_unstale":"2024-08-19T01:33:05.470153+0000","last_undegraded":"2024-08-19T01:33:05.470153+0000","last_fullsized":"2024-08-19T01:33:05.470153+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_clean_scrub_stamp":"2024-08-19T01:32:40.202519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T13:10:18.230055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:33:05.468954+0000","last_change":"2024-08-19T01:32:41.242729+0000","last_active":"2024-08-19T01:33:05.468954+0000","last_peered":"2024-08-19T01:33:05.468954+0000","last_clean":"2024-08-19T01:33:05.468954+0000","last_became_active":"2024-08-19T01:32:41.242436+0000","last_became_peered":"2024-08-19T01:32:41.242436+0000","last_unstale":"2024-08-19T01:33:05.468954+0000","last_undegraded":"2024-08-19T01:33:05.468954+0000","last_fullsized":"2024-08-19T01:33:05.468954+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_clean_scrub_stamp":"2024-08-19T01:32:40.202519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T13:19:32.010576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:33:05.468943+0000","last_change":"2024-08-19T01:32:41.247377+0000","last_active":"2024-08-19T01:33:05.468943+0000","last_peered":"2024-08-19T01:33:05.468943+0000","last_clean":"2024-08-19T01:33:05.468943+0000","last_became_active":"2024-08-19T01:32:41.246804+0000","last_became_peered":"2024-08-19T01:32:41.246804+0000","last_unstale":"2024-08-19T01:33:05.468943+0000","last_undegraded":"2024-08-19T01:33:05.468943+0000","last_fullsized":"2024-08-19T01:33:05.468943+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_clean_scrub_stamp":"2024-08-19T01:32:40.202519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T04:48:52.382687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:33:05.470397+0000","last_change":"2024-08-19T01:32:41.232900+0000","last_active":"2024-08-19T01:33:05.470397+0000","last_peered":"2024-08-19T01:33:05.470397+0000","last_clean":"2024-08-19T01:33:05.470397+0000","last_became_active":"2024-08-19T01:32:41.232573+0000","last_became_peered":"2024-08-19T01:32:41.232573+0000","last_unstale":"2024-08-19T01:33:05.470397+0000","last_undegraded":"2024-08-19T01:33:05.470397+0000","last_fullsized":"2024-08-19T01:33:05.470397+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_clean_scrub_stamp":"2024-08-19T01:32:40.202519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T09:42:36.309834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":23,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:33:05.478964+0000","last_change":"2024-08-19T01:33:05.478964+0000","last_active":"2024-08-19T01:33:05.478964+0000","last_peered":"2024-08-19T01:33:05.478964+0000","last_clean":"2024-08-19T01:33:05.478964+0000","last_became_active":"2024-08-19T01:33:05.478430+0000","last_became_peered":"2024-08-19T01:33:05.478430+0000","last_unstale":"2024-08-19T01:33:05.478964+0000","last_undegraded":"2024-08-19T01:33:05.478964+0000","last_fullsized":"2024-08-19T01:33:05.478964+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_clean_scrub_stamp":"2024-08-19T01:32:40.202519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T04:10:49.267550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,6],"acting":[8,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:33:05.474307+0000","last_change":"2024-08-19T01:32:41.228263+0000","last_active":"2024-08-19T01:33:05.474307+0000","last_peered":"2024-08-19T01:33:05.474307+0000","last_clean":"2024-08-19T01:33:05.474307+0000","last_became_active":"2024-08-19T01:32:41.227765+0000","last_became_peered":"2024-08-19T01:32:41.227765+0000","last_unstale":"2024-08-19T01:33:05.474307+0000","last_undegraded":"2024-08-19T01:33:05.474307+0000","last_fullsized":"2024-08-19T01:33:05.474307+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_clean_scrub_stamp":"2024-08-19T01:32:40.202519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T11:21:18.817886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:33:05.474641+0000","last_change":"2024-08-19T01:32:41.235680+0000","last_active":"2024-08-19T01:33:05.474641+0000","last_peered":"2024-08-19T01:33:05.474641+0000","last_clean":"2024-08-19T01:33:05.474641+0000","last_became_active":"2024-08-19T01:32:41.234150+0000","last_became_peered":"2024-08-19T01:32:41.234150+0000","last_unstale":"2024-08-19T01:33:05.474641+0000","last_undegraded":"2024-08-19T01:33:05.474641+0000","last_fullsized":"2024-08-19T01:33:05.474641+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_clean_scrub_stamp":"2024-08-19T01:32:40.202519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T10:28:08.908632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:33:05.471317+0000","last_change":"2024-08-19T01:32:41.230170+0000","last_active":"2024-08-19T01:33:05.471317+0000","last_peered":"2024-08-19T01:33:05.471317+0000","last_clean":"2024-08-19T01:33:05.471317+0000","last_became_active":"2024-08-19T01:32:41.229467+0000","last_became_peered":"2024-08-19T01:32:41.229467+0000","last_unstale":"2024-08-19T01:33:05.471317+0000","last_undegraded":"2024-08-19T01:33:05.471317+0000","last_fullsized":"2024-08-19T01:33:05.471317+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_clean_scrub_stamp":"2024-08-19T01:32:40.202519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T10:24:15.832910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:33:05.701955+0000","last_change":"2024-08-19T01:32:41.230607+0000","last_active":"2024-08-19T01:33:05.701955+0000","last_peered":"2024-08-19T01:33:05.701955+0000","last_clean":"2024-08-19T01:33:05.701955+0000","last_became_active":"2024-08-19T01:32:41.229979+0000","last_became_peered":"2024-08-19T01:32:41.229979+0000","last_unstale":"2024-08-19T01:33:05.701955+0000","last_undegraded":"2024-08-19T01:33:05.701955+0000","last_fullsized":"2024-08-19T01:33:05.701955+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_clean_scrub_stamp":"2024-08-19T01:32:40.202519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T02:23:46.736139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:33:05.474802+0000","last_change":"2024-08-19T01:32:41.233576+0000","last_active":"2024-08-19T01:33:05.474802+0000","last_peered":"2024-08-19T01:33:05.474802+0000","last_clean":"2024-08-19T01:33:05.474802+0000","last_became_active":"2024-08-19T01:32:41.231279+0000","last_became_peered":"2024-08-19T01:32:41.231279+0000","last_unstale":"2024-08-19T01:33:05.474802+0000","last_undegraded":"2024-08-19T01:33:05.474802+0000","last_fullsized":"2024-08-19T01:33:05.474802+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_clean_scrub_stamp":"2024-08-19T01:32:40.202519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T07:32:45.341628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:33:05.473054+0000","last_change":"2024-08-19T01:32:41.233192+0000","last_active":"2024-08-19T01:33:05.473054+0000","last_peered":"2024-08-19T01:33:05.473054+0000","last_clean":"2024-08-19T01:33:05.473054+0000","last_became_active":"2024-08-19T01:32:41.230944+0000","last_became_peered":"2024-08-19T01:32:41.230944+0000","last_unstale":"2024-08-19T01:33:05.473054+0000","last_undegraded":"2024-08-19T01:33:05.473054+0000","last_fullsized":"2024-08-19T01:33:05.473054+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_clean_scrub_stamp":"2024-08-19T01:32:40.202519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T08:55:29.036252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:33:05.470408+0000","last_change":"2024-08-19T01:32:41.234216+0000","last_active":"2024-08-19T01:33:05.470408+0000","last_peered":"2024-08-19T01:33:05.470408+0000","last_clean":"2024-08-19T01:33:05.470408+0000","last_became_active":"2024-08-19T01:32:41.233906+0000","last_became_peered":"2024-08-19T01:32:41.233906+0000","last_unstale":"2024-08-19T01:33:05.470408+0000","last_undegraded":"2024-08-19T01:33:05.470408+0000","last_fullsized":"2024-08-19T01:33:05.470408+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_clean_scrub_stamp":"2024-08-19T01:32:40.202519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T08:35:01.764494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:33:05.477816+0000","last_change":"2024-08-19T01:32:41.230651+0000","last_active":"2024-08-19T01:33:05.477816+0000","last_peered":"2024-08-19T01:33:05.477816+0000","last_clean":"2024-08-19T01:33:05.477816+0000","last_became_active":"2024-08-19T01:32:41.230253+0000","last_became_peered":"2024-08-19T01:32:41.230253+0000","last_unstale":"2024-08-19T01:33:05.477816+0000","last_undegraded":"2024-08-19T01:33:05.477816+0000","last_fullsized":"2024-08-19T01:33:05.477816+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_clean_scrub_stamp":"2024-08-19T01:32:40.202519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T09:41:59.942821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:33:05.468782+0000","last_change":"2024-08-19T01:32:41.240082+0000","last_active":"2024-08-19T01:33:05.468782+0000","last_peered":"2024-08-19T01:33:05.468782+0000","last_clean":"2024-08-19T01:33:05.468782+0000","last_became_active":"2024-08-19T01:32:41.239711+0000","last_became_peered":"2024-08-19T01:32:41.239711+0000","last_unstale":"2024-08-19T01:33:05.468782+0000","last_undegraded":"2024-08-19T01:33:05.468782+0000","last_fullsized":"2024-08-19T01:33:05.468782+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_clean_scrub_stamp":"2024-08-19T01:32:40.202519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T12:48:59.122307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":23,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:33:05.474883+0000","last_change":"2024-08-19T01:33:05.474883+0000","last_active":"2024-08-19T01:33:05.474883+0000","last_peered":"2024-08-19T01:33:05.474883+0000","last_clean":"2024-08-19T01:33:05.474883+0000","last_became_active":"2024-08-19T01:33:05.474502+0000","last_became_peered":"2024-08-19T01:33:05.474502+0000","last_unstale":"2024-08-19T01:33:05.474883+0000","last_undegraded":"2024-08-19T01:33:05.474883+0000","last_fullsized":"2024-08-19T01:33:05.474883+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_clean_scrub_stamp":"2024-08-19T01:32:40.202519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T07:02:01.171103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2],"acting":[0,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:33:05.701404+0000","last_change":"2024-08-19T01:32:41.240899+0000","last_active":"2024-08-19T01:33:05.701404+0000","last_peered":"2024-08-19T01:33:05.701404+0000","last_clean":"2024-08-19T01:33:05.701404+0000","last_became_active":"2024-08-19T01:32:41.240532+0000","last_became_peered":"2024-08-19T01:32:41.240532+0000","last_unstale":"2024-08-19T01:33:05.701404+0000","last_undegraded":"2024-08-19T01:33:05.701404+0000","last_fullsized":"2024-08-19T01:33:05.701404+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_clean_scrub_stamp":"2024-08-19T01:32:40.202519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T08:17:47.035887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:33:05.470183+0000","last_change":"2024-08-19T01:32:41.237977+0000","last_active":"2024-08-19T01:33:05.470183+0000","last_peered":"2024-08-19T01:33:05.470183+0000","last_clean":"2024-08-19T01:33:05.470183+0000","last_became_active":"2024-08-19T01:32:41.237634+0000","last_became_peered":"2024-08-19T01:32:41.237634+0000","last_unstale":"2024-08-19T01:33:05.470183+0000","last_undegraded":"2024-08-19T01:33:05.470183+0000","last_fullsized":"2024-08-19T01:33:05.470183+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_clean_scrub_stamp":"2024-08-19T01:32:40.202519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T12:48:37.005972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:33:05.466272+0000","last_change":"2024-08-19T01:32:41.236498+0000","last_active":"2024-08-19T01:33:05.466272+0000","last_peered":"2024-08-19T01:33:05.466272+0000","last_clean":"2024-08-19T01:33:05.466272+0000","last_became_active":"2024-08-19T01:32:41.236122+0000","last_became_peered":"2024-08-19T01:32:41.236122+0000","last_unstale":"2024-08-19T01:33:05.466272+0000","last_undegraded":"2024-08-19T01:33:05.466272+0000","last_fullsized":"2024-08-19T01:33:05.466272+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_clean_scrub_stamp":"2024-08-19T01:32:40.202519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T02:31:23.648064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:33:05.466818+0000","last_change":"2024-08-19T01:32:41.234607+0000","last_active":"2024-08-19T01:33:05.466818+0000","last_peered":"2024-08-19T01:33:05.466818+0000","last_clean":"2024-08-19T01:33:05.466818+0000","last_became_active":"2024-08-19T01:32:41.232867+0000","last_became_peered":"2024-08-19T01:32:41.232867+0000","last_unstale":"2024-08-19T01:33:05.466818+0000","last_undegraded":"2024-08-19T01:33:05.466818+0000","last_fullsized":"2024-08-19T01:33:05.466818+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_clean_scrub_stamp":"2024-08-19T01:32:40.202519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T12:32:38.441848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:33:05.469342+0000","last_change":"2024-08-19T01:32:41.231705+0000","last_active":"2024-08-19T01:33:05.469342+0000","last_peered":"2024-08-19T01:33:05.469342+0000","last_clean":"2024-08-19T01:33:05.469342+0000","last_became_active":"2024-08-19T01:32:41.231262+0000","last_became_peered":"2024-08-19T01:32:41.231262+0000","last_unstale":"2024-08-19T01:33:05.469342+0000","last_undegraded":"2024-08-19T01:33:05.469342+0000","last_fullsized":"2024-08-19T01:33:05.469342+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_clean_scrub_stamp":"2024-08-19T01:32:40.202519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T10:41:15.025471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:33:05.477712+0000","last_change":"2024-08-19T01:32:41.243192+0000","last_active":"2024-08-19T01:33:05.477712+0000","last_peered":"2024-08-19T01:33:05.477712+0000","last_clean":"2024-08-19T01:33:05.477712+0000","last_became_active":"2024-08-19T01:32:41.242704+0000","last_became_peered":"2024-08-19T01:32:41.242704+0000","last_unstale":"2024-08-19T01:33:05.477712+0000","last_undegraded":"2024-08-19T01:33:05.477712+0000","last_fullsized":"2024-08-19T01:33:05.477712+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_clean_scrub_stamp":"2024-08-19T01:32:40.202519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T01:44:43.270213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:33:05.701950+0000","last_change":"2024-08-19T01:32:41.239908+0000","last_active":"2024-08-19T01:33:05.701950+0000","last_peered":"2024-08-19T01:33:05.701950+0000","last_clean":"2024-08-19T01:33:05.701950+0000","last_became_active":"2024-08-19T01:32:41.239438+0000","last_became_peered":"2024-08-19T01:32:41.239438+0000","last_unstale":"2024-08-19T01:33:05.701950+0000","last_undegraded":"2024-08-19T01:33:05.701950+0000","last_fullsized":"2024-08-19T01:33:05.701950+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_clean_scrub_stamp":"2024-08-19T01:32:40.202519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T11:03:26.822111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:33:05.471582+0000","last_change":"2024-08-19T01:32:41.226373+0000","last_active":"2024-08-19T01:33:05.471582+0000","last_peered":"2024-08-19T01:33:05.471582+0000","last_clean":"2024-08-19T01:33:05.471582+0000","last_became_active":"2024-08-19T01:32:41.226089+0000","last_became_peered":"2024-08-19T01:32:41.226089+0000","last_unstale":"2024-08-19T01:33:05.471582+0000","last_undegraded":"2024-08-19T01:33:05.471582+0000","last_fullsized":"2024-08-19T01:33:05.471582+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_clean_scrub_stamp":"2024-08-19T01:32:40.202519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T01:33:56.639300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:33:05.469289+0000","last_change":"2024-08-19T01:32:41.237829+0000","last_active":"2024-08-19T01:33:05.469289+0000","last_peered":"2024-08-19T01:33:05.469289+0000","last_clean":"2024-08-19T01:33:05.469289+0000","last_became_active":"2024-08-19T01:32:41.237518+0000","last_became_peered":"2024-08-19T01:32:41.237518+0000","last_unstale":"2024-08-19T01:33:05.469289+0000","last_undegraded":"2024-08-19T01:33:05.469289+0000","last_fullsized":"2024-08-19T01:33:05.469289+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_clean_scrub_stamp":"2024-08-19T01:32:40.202519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T02:25:11.307658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:33:05.472158+0000","last_change":"2024-08-19T01:32:41.234508+0000","last_active":"2024-08-19T01:33:05.472158+0000","last_peered":"2024-08-19T01:33:05.472158+0000","last_clean":"2024-08-19T01:33:05.472158+0000","last_became_active":"2024-08-19T01:32:41.232869+0000","last_became_peered":"2024-08-19T01:32:41.232869+0000","last_unstale":"2024-08-19T01:33:05.472158+0000","last_undegraded":"2024-08-19T01:33:05.472158+0000","last_fullsized":"2024-08-19T01:33:05.472158+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_clean_scrub_stamp":"2024-08-19T01:32:40.202519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T11:20:55.097576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:33:05.466993+0000","last_change":"2024-08-19T01:32:41.230536+0000","last_active":"2024-08-19T01:33:05.466993+0000","last_peered":"2024-08-19T01:33:05.466993+0000","last_clean":"2024-08-19T01:33:05.466993+0000","last_became_active":"2024-08-19T01:32:41.230179+0000","last_became_peered":"2024-08-19T01:32:41.230179+0000","last_unstale":"2024-08-19T01:33:05.466993+0000","last_undegraded":"2024-08-19T01:33:05.466993+0000","last_fullsized":"2024-08-19T01:33:05.466993+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_clean_scrub_stamp":"2024-08-19T01:32:40.202519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T02:25:09.441363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:33:05.469818+0000","last_change":"2024-08-19T01:32:41.230695+0000","last_active":"2024-08-19T01:33:05.469818+0000","last_peered":"2024-08-19T01:33:05.469818+0000","last_clean":"2024-08-19T01:33:05.469818+0000","last_became_active":"2024-08-19T01:32:41.230324+0000","last_became_peered":"2024-08-19T01:32:41.230324+0000","last_unstale":"2024-08-19T01:33:05.469818+0000","last_undegraded":"2024-08-19T01:33:05.469818+0000","last_fullsized":"2024-08-19T01:33:05.469818+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_clean_scrub_stamp":"2024-08-19T01:32:40.202519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T05:38:09.317605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:33:05.477784+0000","last_change":"2024-08-19T01:32:41.238325+0000","last_active":"2024-08-19T01:33:05.477784+0000","last_peered":"2024-08-19T01:33:05.477784+0000","last_clean":"2024-08-19T01:33:05.477784+0000","last_became_active":"2024-08-19T01:32:41.237768+0000","last_became_peered":"2024-08-19T01:32:41.237768+0000","last_unstale":"2024-08-19T01:33:05.477784+0000","last_undegraded":"2024-08-19T01:33:05.477784+0000","last_fullsized":"2024-08-19T01:33:05.477784+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_clean_scrub_stamp":"2024-08-19T01:32:40.202519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T08:45:06.866241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:33:05.468967+0000","last_change":"2024-08-19T01:32:41.246950+0000","last_active":"2024-08-19T01:33:05.468967+0000","last_peered":"2024-08-19T01:33:05.468967+0000","last_clean":"2024-08-19T01:33:05.468967+0000","last_became_active":"2024-08-19T01:32:41.245441+0000","last_became_peered":"2024-08-19T01:32:41.245441+0000","last_unstale":"2024-08-19T01:33:05.468967+0000","last_undegraded":"2024-08-19T01:33:05.468967+0000","last_fullsized":"2024-08-19T01:33:05.468967+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_clean_scrub_stamp":"2024-08-19T01:32:40.202519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T03:11:06.921258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:33:05.466512+0000","last_change":"2024-08-19T01:32:41.244810+0000","last_active":"2024-08-19T01:33:05.466512+0000","last_peered":"2024-08-19T01:33:05.466512+0000","last_clean":"2024-08-19T01:33:05.466512+0000","last_became_active":"2024-08-19T01:32:41.244401+0000","last_became_peered":"2024-08-19T01:32:41.244401+0000","last_unstale":"2024-08-19T01:33:05.466512+0000","last_undegraded":"2024-08-19T01:33:05.466512+0000","last_fullsized":"2024-08-19T01:33:05.466512+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_clean_scrub_stamp":"2024-08-19T01:32:40.202519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T04:54:57.728181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:33:05.466793+0000","last_change":"2024-08-19T01:32:41.235977+0000","last_active":"2024-08-19T01:33:05.466793+0000","last_peered":"2024-08-19T01:33:05.466793+0000","last_clean":"2024-08-19T01:33:05.466793+0000","last_became_active":"2024-08-19T01:32:41.235693+0000","last_became_peered":"2024-08-19T01:32:41.235693+0000","last_unstale":"2024-08-19T01:33:05.466793+0000","last_undegraded":"2024-08-19T01:33:05.466793+0000","last_fullsized":"2024-08-19T01:33:05.466793+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_clean_scrub_stamp":"2024-08-19T01:32:40.202519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T04:59:15.766532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":23,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:33:05.473318+0000","last_change":"2024-08-19T01:33:05.473318+0000","last_active":"2024-08-19T01:33:05.473318+0000","last_peered":"2024-08-19T01:33:05.473318+0000","last_clean":"2024-08-19T01:33:05.473318+0000","last_became_active":"2024-08-19T01:33:05.472885+0000","last_became_peered":"2024-08-19T01:33:05.472885+0000","last_unstale":"2024-08-19T01:33:05.473318+0000","last_undegraded":"2024-08-19T01:33:05.473318+0000","last_fullsized":"2024-08-19T01:33:05.473318+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_clean_scrub_stamp":"2024-08-19T01:32:40.202519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T07:49:35.525930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,9],"acting":[11,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:33:05.471500+0000","last_change":"2024-08-19T01:32:41.230846+0000","last_active":"2024-08-19T01:33:05.471500+0000","last_peered":"2024-08-19T01:33:05.471500+0000","last_clean":"2024-08-19T01:33:05.471500+0000","last_became_active":"2024-08-19T01:32:41.230405+0000","last_became_peered":"2024-08-19T01:32:41.230405+0000","last_unstale":"2024-08-19T01:33:05.471500+0000","last_undegraded":"2024-08-19T01:33:05.471500+0000","last_fullsized":"2024-08-19T01:33:05.471500+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_clean_scrub_stamp":"2024-08-19T01:32:40.202519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T08:59:46.241898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1b","version":"84'6","reported_seq":99,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:33:46.531957+0000","last_change":"2024-08-19T01:33:05.479889+0000","last_active":"2024-08-19T01:33:46.531957+0000","last_peered":"2024-08-19T01:33:46.531957+0000","last_clean":"2024-08-19T01:33:46.531957+0000","last_became_active":"2024-08-19T01:33:05.474698+0000","last_became_peered":"2024-08-19T01:33:05.474698+0000","last_unstale":"2024-08-19T01:33:46.531957+0000","last_undegraded":"2024-08-19T01:33:46.531957+0000","last_fullsized":"2024-08-19T01:33:46.531957+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:32:37.889224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:32:37.889224+0000","last_clean_scrub_stamp":"2024-08-19T01:32:37.889224+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T08:41:29.053471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1012,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":57,"num_read_kb":3,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":1012,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7],"acting":[5,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":11,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:33:05.475899+0000","last_change":"2024-08-19T01:33:05.474083+0000","last_active":"2024-08-19T01:33:05.475899+0000","last_peered":"2024-08-19T01:33:05.475899+0000","last_clean":"2024-08-19T01:33:05.475899+0000","last_became_active":"2024-08-19T01:33:05.473718+0000","last_became_peered":"2024-08-19T01:33:05.473718+0000","last_unstale":"2024-08-19T01:33:05.475899+0000","last_undegraded":"2024-08-19T01:33:05.475899+0000","last_fullsized":"2024-08-19T01:33:05.475899+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_clean_scrub_stamp":"2024-08-19T01:32:40.202519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T02:25:11.955170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,4,3],"acting":[9,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.1a","version":"84'3","reported_seq":35,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:33:05.705917+0000","last_change":"2024-08-19T01:33:05.705917+0000","last_active":"2024-08-19T01:33:05.705917+0000","last_peered":"2024-08-19T01:33:05.705917+0000","last_clean":"2024-08-19T01:33:05.705917+0000","last_became_active":"2024-08-19T01:33:05.702775+0000","last_became_peered":"2024-08-19T01:33:05.702775+0000","last_unstale":"2024-08-19T01:33:05.705917+0000","last_undegraded":"2024-08-19T01:33:05.705917+0000","last_fullsized":"2024-08-19T01:33:05.705917+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:32:37.889224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:32:37.889224+0000","last_clean_scrub_stamp":"2024-08-19T01:32:37.889224+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T03:58:29.989815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,6],"acting":[10,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.19","version":"84'2","reported_seq":24,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:33:05.477490+0000","last_change":"2024-08-19T01:32:38.911757+0000","last_active":"2024-08-19T01:33:05.477490+0000","last_peered":"2024-08-19T01:33:05.477490+0000","last_clean":"2024-08-19T01:33:05.477490+0000","last_became_active":"2024-08-19T01:32:38.910832+0000","last_became_peered":"2024-08-19T01:32:38.910832+0000","last_unstale":"2024-08-19T01:33:05.477490+0000","last_undegraded":"2024-08-19T01:33:05.477490+0000","last_fullsized":"2024-08-19T01:33:05.477490+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:32:37.889224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:32:37.889224+0000","last_clean_scrub_stamp":"2024-08-19T01:32:37.889224+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T04:46:54.986693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:33:05.471462+0000","last_change":"2024-08-19T01:32:41.231130+0000","last_active":"2024-08-19T01:33:05.471462+0000","last_peered":"2024-08-19T01:33:05.471462+0000","last_clean":"2024-08-19T01:33:05.471462+0000","last_became_active":"2024-08-19T01:32:41.230883+0000","last_became_peered":"2024-08-19T01:32:41.230883+0000","last_unstale":"2024-08-19T01:33:05.471462+0000","last_undegraded":"2024-08-19T01:33:05.471462+0000","last_fullsized":"2024-08-19T01:33:05.471462+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_clean_scrub_stamp":"2024-08-19T01:32:40.202519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T06:33:11.856202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"84'1","reported_seq":23,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:33:05.470862+0000","last_change":"2024-08-19T01:32:38.916868+0000","last_active":"2024-08-19T01:33:05.470862+0000","last_peered":"2024-08-19T01:33:05.470862+0000","last_clean":"2024-08-19T01:33:05.470862+0000","last_became_active":"2024-08-19T01:32:38.916500+0000","last_became_peered":"2024-08-19T01:32:38.916500+0000","last_unstale":"2024-08-19T01:33:05.470862+0000","last_undegraded":"2024-08-19T01:33:05.470862+0000","last_fullsized":"2024-08-19T01:33:05.470862+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:32:37.889224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:32:37.889224+0000","last_clean_scrub_stamp":"2024-08-19T01:32:37.889224+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T12:30:07.085829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:33:05.465832+0000","last_change":"2024-08-19T01:32:41.235290+0000","last_active":"2024-08-19T01:33:05.465832+0000","last_peered":"2024-08-19T01:33:05.465832+0000","last_clean":"2024-08-19T01:33:05.465832+0000","last_became_active":"2024-08-19T01:32:41.235013+0000","last_became_peered":"2024-08-19T01:32:41.235013+0000","last_unstale":"2024-08-19T01:33:05.465832+0000","last_undegraded":"2024-08-19T01:33:05.465832+0000","last_fullsized":"2024-08-19T01:33:05.465832+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_clean_scrub_stamp":"2024-08-19T01:32:40.202519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T09:50:14.837080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"84'3","reported_seq":82,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:33:47.532715+0000","last_change":"2024-08-19T01:32:38.912023+0000","last_active":"2024-08-19T01:33:47.532715+0000","last_peered":"2024-08-19T01:33:47.532715+0000","last_clean":"2024-08-19T01:33:47.532715+0000","last_became_active":"2024-08-19T01:32:38.911679+0000","last_became_peered":"2024-08-19T01:32:38.911679+0000","last_unstale":"2024-08-19T01:33:47.532715+0000","last_undegraded":"2024-08-19T01:33:47.532715+0000","last_fullsized":"2024-08-19T01:33:47.532715+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:32:37.889224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:32:37.889224+0000","last_clean_scrub_stamp":"2024-08-19T01:32:37.889224+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T12:08:51.118338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":57,"num_read_kb":57,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:33:05.701747+0000","last_change":"2024-08-19T01:32:41.232264+0000","last_active":"2024-08-19T01:33:05.701747+0000","last_peered":"2024-08-19T01:33:05.701747+0000","last_clean":"2024-08-19T01:33:05.701747+0000","last_became_active":"2024-08-19T01:32:41.231788+0000","last_became_peered":"2024-08-19T01:32:41.231788+0000","last_unstale":"2024-08-19T01:33:05.701747+0000","last_undegraded":"2024-08-19T01:33:05.701747+0000","last_fullsized":"2024-08-19T01:33:05.701747+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_clean_scrub_stamp":"2024-08-19T01:32:40.202519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T04:59:06.443992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"84'2","reported_seq":24,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:33:05.472954+0000","last_change":"2024-08-19T01:32:38.915770+0000","last_active":"2024-08-19T01:33:05.472954+0000","last_peered":"2024-08-19T01:33:05.472954+0000","last_clean":"2024-08-19T01:33:05.472954+0000","last_became_active":"2024-08-19T01:32:38.915378+0000","last_became_peered":"2024-08-19T01:32:38.915378+0000","last_unstale":"2024-08-19T01:33:05.472954+0000","last_undegraded":"2024-08-19T01:33:05.472954+0000","last_fullsized":"2024-08-19T01:33:05.472954+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:32:37.889224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:32:37.889224+0000","last_clean_scrub_stamp":"2024-08-19T01:32:37.889224+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T09:16:14.513172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:33:05.469512+0000","last_change":"2024-08-19T01:32:41.243476+0000","last_active":"2024-08-19T01:33:05.469512+0000","last_peered":"2024-08-19T01:33:05.469512+0000","last_clean":"2024-08-19T01:33:05.469512+0000","last_became_active":"2024-08-19T01:32:41.243190+0000","last_became_peered":"2024-08-19T01:32:41.243190+0000","last_unstale":"2024-08-19T01:33:05.469512+0000","last_undegraded":"2024-08-19T01:33:05.469512+0000","last_fullsized":"2024-08-19T01:33:05.469512+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_clean_scrub_stamp":"2024-08-19T01:32:40.202519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T12:07:34.096919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"84'4","reported_seq":27,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:33:05.470664+0000","last_change":"2024-08-19T01:32:39.051458+0000","last_active":"2024-08-19T01:33:05.470664+0000","last_peered":"2024-08-19T01:33:05.470664+0000","last_clean":"2024-08-19T01:33:05.470664+0000","last_became_active":"2024-08-19T01:32:39.051113+0000","last_became_peered":"2024-08-19T01:32:39.051113+0000","last_unstale":"2024-08-19T01:33:05.470664+0000","last_undegraded":"2024-08-19T01:33:05.470664+0000","last_fullsized":"2024-08-19T01:33:05.470664+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:32:37.889224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:32:37.889224+0000","last_clean_scrub_stamp":"2024-08-19T01:32:37.889224+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T02:15:51.371054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:33:05.701579+0000","last_change":"2024-08-19T01:32:41.236453+0000","last_active":"2024-08-19T01:33:05.701579+0000","last_peered":"2024-08-19T01:33:05.701579+0000","last_clean":"2024-08-19T01:33:05.701579+0000","last_became_active":"2024-08-19T01:32:41.236135+0000","last_became_peered":"2024-08-19T01:32:41.236135+0000","last_unstale":"2024-08-19T01:33:05.701579+0000","last_undegraded":"2024-08-19T01:33:05.701579+0000","last_fullsized":"2024-08-19T01:33:05.701579+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_clean_scrub_stamp":"2024-08-19T01:32:40.202519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T12:38:41.398299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"86'28","reported_seq":173,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:33:48.534218+0000","last_change":"2024-08-19T01:32:38.914831+0000","last_active":"2024-08-19T01:33:48.534218+0000","last_peered":"2024-08-19T01:33:48.534218+0000","last_clean":"2024-08-19T01:33:48.534218+0000","last_became_active":"2024-08-19T01:32:38.914326+0000","last_became_peered":"2024-08-19T01:32:38.914326+0000","last_unstale":"2024-08-19T01:33:48.534218+0000","last_undegraded":"2024-08-19T01:33:48.534218+0000","last_fullsized":"2024-08-19T01:33:48.534218+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:32:37.889224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:32:37.889224+0000","last_clean_scrub_stamp":"2024-08-19T01:32:37.889224+0000","objects_scrubbed":0,"log_size":28,"log_dups_size":0,"ondisk_log_size":28,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T02:15:25.263974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":60616,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":123,"num_read_kb":119,"num_write":29,"num_write_kb":68,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:33:05.472530+0000","last_change":"2024-08-19T01:32:41.237945+0000","last_active":"2024-08-19T01:33:05.472530+0000","last_peered":"2024-08-19T01:33:05.472530+0000","last_clean":"2024-08-19T01:33:05.472530+0000","last_became_active":"2024-08-19T01:32:41.237362+0000","last_became_peered":"2024-08-19T01:32:41.237362+0000","last_unstale":"2024-08-19T01:33:05.472530+0000","last_undegraded":"2024-08-19T01:33:05.472530+0000","last_fullsized":"2024-08-19T01:33:05.472530+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_clean_scrub_stamp":"2024-08-19T01:32:40.202519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T07:29:17.180054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"84'6","reported_seq":29,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:33:05.466940+0000","last_change":"2024-08-19T01:32:38.912453+0000","last_active":"2024-08-19T01:33:05.466940+0000","last_peered":"2024-08-19T01:33:05.466940+0000","last_clean":"2024-08-19T01:33:05.466940+0000","last_became_active":"2024-08-19T01:32:38.912051+0000","last_became_peered":"2024-08-19T01:32:38.912051+0000","last_unstale":"2024-08-19T01:33:05.466940+0000","last_undegraded":"2024-08-19T01:33:05.466940+0000","last_fullsized":"2024-08-19T01:33:05.466940+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:32:37.889224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:32:37.889224+0000","last_clean_scrub_stamp":"2024-08-19T01:32:37.889224+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T06:49:58.810134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:33:05.477644+0000","last_change":"2024-08-19T01:32:41.236282+0000","last_active":"2024-08-19T01:33:05.477644+0000","last_peered":"2024-08-19T01:33:05.477644+0000","last_clean":"2024-08-19T01:33:05.477644+0000","last_became_active":"2024-08-19T01:32:41.235559+0000","last_became_peered":"2024-08-19T01:32:41.235559+0000","last_unstale":"2024-08-19T01:33:05.477644+0000","last_undegraded":"2024-08-19T01:33:05.477644+0000","last_fullsized":"2024-08-19T01:33:05.477644+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_clean_scrub_stamp":"2024-08-19T01:32:40.202519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T07:51:52.157094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"84'3","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:33:05.470566+0000","last_change":"2024-08-19T01:32:38.915688+0000","last_active":"2024-08-19T01:33:05.470566+0000","last_peered":"2024-08-19T01:33:05.470566+0000","last_clean":"2024-08-19T01:33:05.470566+0000","last_became_active":"2024-08-19T01:32:38.915297+0000","last_became_peered":"2024-08-19T01:32:38.915297+0000","last_unstale":"2024-08-19T01:33:05.470566+0000","last_undegraded":"2024-08-19T01:33:05.470566+0000","last_fullsized":"2024-08-19T01:33:05.470566+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:32:37.889224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:32:37.889224+0000","last_clean_scrub_stamp":"2024-08-19T01:32:37.889224+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T01:38:12.780588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:33:05.472886+0000","last_change":"2024-08-19T01:32:41.233766+0000","last_active":"2024-08-19T01:33:05.472886+0000","last_peered":"2024-08-19T01:33:05.472886+0000","last_clean":"2024-08-19T01:33:05.472886+0000","last_became_active":"2024-08-19T01:32:41.233425+0000","last_became_peered":"2024-08-19T01:32:41.233425+0000","last_unstale":"2024-08-19T01:33:05.472886+0000","last_undegraded":"2024-08-19T01:33:05.472886+0000","last_fullsized":"2024-08-19T01:33:05.472886+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_clean_scrub_stamp":"2024-08-19T01:32:40.202519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T10:29:15.471639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"84'4","reported_seq":85,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:33:46.394817+0000","last_change":"2024-08-19T01:32:38.912610+0000","last_active":"2024-08-19T01:33:46.394817+0000","last_peered":"2024-08-19T01:33:46.394817+0000","last_clean":"2024-08-19T01:33:46.394817+0000","last_became_active":"2024-08-19T01:32:38.912040+0000","last_became_peered":"2024-08-19T01:32:38.912040+0000","last_unstale":"2024-08-19T01:33:46.394817+0000","last_undegraded":"2024-08-19T01:33:46.394817+0000","last_fullsized":"2024-08-19T01:33:46.394817+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:32:37.889224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:32:37.889224+0000","last_clean_scrub_stamp":"2024-08-19T01:32:37.889224+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T09:26:05.966391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":978,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":60,"num_read_kb":4,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:33:05.472892+0000","last_change":"2024-08-19T01:32:41.235945+0000","last_active":"2024-08-19T01:33:05.472892+0000","last_peered":"2024-08-19T01:33:05.472892+0000","last_clean":"2024-08-19T01:33:05.472892+0000","last_became_active":"2024-08-19T01:32:41.235633+0000","last_became_peered":"2024-08-19T01:32:41.235633+0000","last_unstale":"2024-08-19T01:33:05.472892+0000","last_undegraded":"2024-08-19T01:33:05.472892+0000","last_fullsized":"2024-08-19T01:33:05.472892+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_clean_scrub_stamp":"2024-08-19T01:32:40.202519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T08:37:37.682175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"84'15","reported_seq":38,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:33:05.471087+0000","last_change":"2024-08-19T01:32:39.052850+0000","last_active":"2024-08-19T01:33:05.471087+0000","last_peered":"2024-08-19T01:33:05.471087+0000","last_clean":"2024-08-19T01:33:05.471087+0000","last_became_active":"2024-08-19T01:32:39.052455+0000","last_became_peered":"2024-08-19T01:32:39.052455+0000","last_unstale":"2024-08-19T01:33:05.471087+0000","last_undegraded":"2024-08-19T01:33:05.471087+0000","last_fullsized":"2024-08-19T01:33:05.471087+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:32:37.889224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:32:37.889224+0000","last_clean_scrub_stamp":"2024-08-19T01:32:37.889224+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T03:25:08.279911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":23,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:33:05.702575+0000","last_change":"2024-08-19T01:33:05.702575+0000","last_active":"2024-08-19T01:33:05.702575+0000","last_peered":"2024-08-19T01:33:05.702575+0000","last_clean":"2024-08-19T01:33:05.702575+0000","last_became_active":"2024-08-19T01:33:05.702110+0000","last_became_peered":"2024-08-19T01:33:05.702110+0000","last_unstale":"2024-08-19T01:33:05.702575+0000","last_undegraded":"2024-08-19T01:33:05.702575+0000","last_fullsized":"2024-08-19T01:33:05.702575+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_clean_scrub_stamp":"2024-08-19T01:32:40.202519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T12:18:51.102180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,1],"acting":[5,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"84'2","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:33:05.472730+0000","last_change":"2024-08-19T01:32:38.912090+0000","last_active":"2024-08-19T01:33:05.472730+0000","last_peered":"2024-08-19T01:33:05.472730+0000","last_clean":"2024-08-19T01:33:05.472730+0000","last_became_active":"2024-08-19T01:32:38.911753+0000","last_became_peered":"2024-08-19T01:32:38.911753+0000","last_unstale":"2024-08-19T01:33:05.472730+0000","last_undegraded":"2024-08-19T01:33:05.472730+0000","last_fullsized":"2024-08-19T01:33:05.472730+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:32:37.889224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:32:37.889224+0000","last_clean_scrub_stamp":"2024-08-19T01:32:37.889224+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T12:56:42.982062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:33:05.469805+0000","last_change":"2024-08-19T01:32:41.230429+0000","last_active":"2024-08-19T01:33:05.469805+0000","last_peered":"2024-08-19T01:33:05.469805+0000","last_clean":"2024-08-19T01:33:05.469805+0000","last_became_active":"2024-08-19T01:32:41.230115+0000","last_became_peered":"2024-08-19T01:32:41.230115+0000","last_unstale":"2024-08-19T01:33:05.469805+0000","last_undegraded":"2024-08-19T01:33:05.469805+0000","last_fullsized":"2024-08-19T01:33:05.469805+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_clean_scrub_stamp":"2024-08-19T01:32:40.202519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T01:50:51.084130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.e","version":"84'2","reported_seq":24,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:33:05.477826+0000","last_change":"2024-08-19T01:32:38.914557+0000","last_active":"2024-08-19T01:33:05.477826+0000","last_peered":"2024-08-19T01:33:05.477826+0000","last_clean":"2024-08-19T01:33:05.477826+0000","last_became_active":"2024-08-19T01:32:38.914115+0000","last_became_peered":"2024-08-19T01:32:38.914115+0000","last_unstale":"2024-08-19T01:33:05.477826+0000","last_undegraded":"2024-08-19T01:33:05.477826+0000","last_fullsized":"2024-08-19T01:33:05.477826+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:32:37.889224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:32:37.889224+0000","last_clean_scrub_stamp":"2024-08-19T01:32:37.889224+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T06:30:00.523513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:33:05.473436+0000","last_change":"2024-08-19T01:32:41.234883+0000","last_active":"2024-08-19T01:33:05.473436+0000","last_peered":"2024-08-19T01:33:05.473436+0000","last_clean":"2024-08-19T01:33:05.473436+0000","last_became_active":"2024-08-19T01:32:41.233901+0000","last_became_peered":"2024-08-19T01:32:41.233901+0000","last_unstale":"2024-08-19T01:33:05.473436+0000","last_undegraded":"2024-08-19T01:33:05.473436+0000","last_fullsized":"2024-08-19T01:33:05.473436+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_clean_scrub_stamp":"2024-08-19T01:32:40.202519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T12:08:46.305259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"84'1","reported_seq":24,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:33:05.470777+0000","last_change":"2024-08-19T01:32:38.913441+0000","last_active":"2024-08-19T01:33:05.470777+0000","last_peered":"2024-08-19T01:33:05.470777+0000","last_clean":"2024-08-19T01:33:05.470777+0000","last_became_active":"2024-08-19T01:32:38.913115+0000","last_became_peered":"2024-08-19T01:32:38.913115+0000","last_unstale":"2024-08-19T01:33:05.470777+0000","last_undegraded":"2024-08-19T01:33:05.470777+0000","last_fullsized":"2024-08-19T01:33:05.470777+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:32:37.889224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:32:37.889224+0000","last_clean_scrub_stamp":"2024-08-19T01:32:37.889224+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T11:43:15.290980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"84'1","reported_seq":18,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:33:05.472934+0000","last_change":"2024-08-19T01:32:41.235370+0000","last_active":"2024-08-19T01:33:05.472934+0000","last_peered":"2024-08-19T01:33:05.472934+0000","last_clean":"2024-08-19T01:33:05.472934+0000","last_became_active":"2024-08-19T01:32:41.235012+0000","last_became_peered":"2024-08-19T01:32:41.235012+0000","last_unstale":"2024-08-19T01:33:05.472934+0000","last_undegraded":"2024-08-19T01:33:05.472934+0000","last_fullsized":"2024-08-19T01:33:05.472934+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_clean_scrub_stamp":"2024-08-19T01:32:40.202519+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T02:18:31.134974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"84'3","reported_seq":26,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:33:05.466872+0000","last_change":"2024-08-19T01:32:39.245560+0000","last_active":"2024-08-19T01:33:05.466872+0000","last_peered":"2024-08-19T01:33:05.466872+0000","last_clean":"2024-08-19T01:33:05.466872+0000","last_became_active":"2024-08-19T01:32:39.245118+0000","last_became_peered":"2024-08-19T01:32:39.245118+0000","last_unstale":"2024-08-19T01:33:05.466872+0000","last_undegraded":"2024-08-19T01:33:05.466872+0000","last_fullsized":"2024-08-19T01:33:05.466872+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:32:37.889224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:32:37.889224+0000","last_clean_scrub_stamp":"2024-08-19T01:32:37.889224+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T02:18:24.605709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:33:05.473113+0000","last_change":"2024-08-19T01:32:41.234452+0000","last_active":"2024-08-19T01:33:05.473113+0000","last_peered":"2024-08-19T01:33:05.473113+0000","last_clean":"2024-08-19T01:33:05.473113+0000","last_became_active":"2024-08-19T01:32:41.233852+0000","last_became_peered":"2024-08-19T01:32:41.233852+0000","last_unstale":"2024-08-19T01:33:05.473113+0000","last_undegraded":"2024-08-19T01:33:05.473113+0000","last_fullsized":"2024-08-19T01:33:05.473113+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_clean_scrub_stamp":"2024-08-19T01:32:40.202519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T06:18:34.457239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"22'137","reported_seq":338,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:33:05.473232+0000","last_change":"2024-08-19T01:31:27.659394+0000","last_active":"2024-08-19T01:33:05.473232+0000","last_peered":"2024-08-19T01:33:05.473232+0000","last_clean":"2024-08-19T01:33:05.473232+0000","last_became_active":"2024-08-19T01:31:27.659007+0000","last_became_peered":"2024-08-19T01:31:27.659007+0000","last_unstale":"2024-08-19T01:33:05.473232+0000","last_undegraded":"2024-08-19T01:33:05.473232+0000","last_fullsized":"2024-08-19T01:33:05.473232+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:28:16.912251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:28:16.912251+0000","last_clean_scrub_stamp":"2024-08-19T01:28:16.912251+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T05:53:01.660619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"84'3","reported_seq":27,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:33:05.470433+0000","last_change":"2024-08-19T01:32:38.913408+0000","last_active":"2024-08-19T01:33:05.470433+0000","last_peered":"2024-08-19T01:33:05.470433+0000","last_clean":"2024-08-19T01:33:05.470433+0000","last_became_active":"2024-08-19T01:32:38.912242+0000","last_became_peered":"2024-08-19T01:32:38.912242+0000","last_unstale":"2024-08-19T01:33:05.470433+0000","last_undegraded":"2024-08-19T01:33:05.470433+0000","last_fullsized":"2024-08-19T01:33:05.470433+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:32:37.889224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:32:37.889224+0000","last_clean_scrub_stamp":"2024-08-19T01:32:37.889224+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T07:26:48.121366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:33:05.471752+0000","last_change":"2024-08-19T01:32:41.232611+0000","last_active":"2024-08-19T01:33:05.471752+0000","last_peered":"2024-08-19T01:33:05.471752+0000","last_clean":"2024-08-19T01:33:05.471752+0000","last_became_active":"2024-08-19T01:32:41.232261+0000","last_became_peered":"2024-08-19T01:32:41.232261+0000","last_unstale":"2024-08-19T01:33:05.471752+0000","last_undegraded":"2024-08-19T01:33:05.471752+0000","last_fullsized":"2024-08-19T01:33:05.471752+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_clean_scrub_stamp":"2024-08-19T01:32:40.202519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T08:04:16.793933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"84'3","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:33:05.473104+0000","last_change":"2024-08-19T01:32:38.912765+0000","last_active":"2024-08-19T01:33:05.473104+0000","last_peered":"2024-08-19T01:33:05.473104+0000","last_clean":"2024-08-19T01:33:05.473104+0000","last_became_active":"2024-08-19T01:32:38.912274+0000","last_became_peered":"2024-08-19T01:32:38.912274+0000","last_unstale":"2024-08-19T01:33:05.473104+0000","last_undegraded":"2024-08-19T01:33:05.473104+0000","last_fullsized":"2024-08-19T01:33:05.473104+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:32:37.889224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:32:37.889224+0000","last_clean_scrub_stamp":"2024-08-19T01:32:37.889224+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T09:57:49.714827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:33:05.477455+0000","last_change":"2024-08-19T01:32:41.224449+0000","last_active":"2024-08-19T01:33:05.477455+0000","last_peered":"2024-08-19T01:33:05.477455+0000","last_clean":"2024-08-19T01:33:05.477455+0000","last_became_active":"2024-08-19T01:32:41.224064+0000","last_became_peered":"2024-08-19T01:32:41.224064+0000","last_unstale":"2024-08-19T01:33:05.477455+0000","last_undegraded":"2024-08-19T01:33:05.477455+0000","last_fullsized":"2024-08-19T01:33:05.477455+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_clean_scrub_stamp":"2024-08-19T01:32:40.202519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T08:53:33.948504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"84'4","reported_seq":80,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:33:44.390466+0000","last_change":"2024-08-19T01:32:39.050869+0000","last_active":"2024-08-19T01:33:44.390466+0000","last_peered":"2024-08-19T01:33:44.390466+0000","last_clean":"2024-08-19T01:33:44.390466+0000","last_became_active":"2024-08-19T01:32:39.050529+0000","last_became_peered":"2024-08-19T01:32:39.050529+0000","last_unstale":"2024-08-19T01:33:44.390466+0000","last_undegraded":"2024-08-19T01:33:44.390466+0000","last_fullsized":"2024-08-19T01:33:44.390466+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:32:37.889224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:32:37.889224+0000","last_clean_scrub_stamp":"2024-08-19T01:32:37.889224+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T12:52:52.788360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":54,"num_read_kb":54,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:33:05.470330+0000","last_change":"2024-08-19T01:32:41.237093+0000","last_active":"2024-08-19T01:33:05.470330+0000","last_peered":"2024-08-19T01:33:05.470330+0000","last_clean":"2024-08-19T01:33:05.470330+0000","last_became_active":"2024-08-19T01:32:41.236738+0000","last_became_peered":"2024-08-19T01:32:41.236738+0000","last_unstale":"2024-08-19T01:33:05.470330+0000","last_undegraded":"2024-08-19T01:33:05.470330+0000","last_fullsized":"2024-08-19T01:33:05.470330+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_clean_scrub_stamp":"2024-08-19T01:32:40.202519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T08:37:33.117344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"84'1","reported_seq":23,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:33:05.473189+0000","last_change":"2024-08-19T01:32:38.915010+0000","last_active":"2024-08-19T01:33:05.473189+0000","last_peered":"2024-08-19T01:33:05.473189+0000","last_clean":"2024-08-19T01:33:05.473189+0000","last_became_active":"2024-08-19T01:32:38.914687+0000","last_became_peered":"2024-08-19T01:32:38.914687+0000","last_unstale":"2024-08-19T01:33:05.473189+0000","last_undegraded":"2024-08-19T01:33:05.473189+0000","last_fullsized":"2024-08-19T01:33:05.473189+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:32:37.889224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:32:37.889224+0000","last_clean_scrub_stamp":"2024-08-19T01:32:37.889224+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T04:46:07.881778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:33:05.474056+0000","last_change":"2024-08-19T01:32:41.244069+0000","last_active":"2024-08-19T01:33:05.474056+0000","last_peered":"2024-08-19T01:33:05.474056+0000","last_clean":"2024-08-19T01:33:05.474056+0000","last_became_active":"2024-08-19T01:32:41.243482+0000","last_became_peered":"2024-08-19T01:32:41.243482+0000","last_unstale":"2024-08-19T01:33:05.474056+0000","last_undegraded":"2024-08-19T01:33:05.474056+0000","last_fullsized":"2024-08-19T01:33:05.474056+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_clean_scrub_stamp":"2024-08-19T01:32:40.202519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T02:08:28.044421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":22,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:33:05.466068+0000","last_change":"2024-08-19T01:32:39.051383+0000","last_active":"2024-08-19T01:33:05.466068+0000","last_peered":"2024-08-19T01:33:05.466068+0000","last_clean":"2024-08-19T01:33:05.466068+0000","last_became_active":"2024-08-19T01:32:39.050970+0000","last_became_peered":"2024-08-19T01:32:39.050970+0000","last_unstale":"2024-08-19T01:33:05.466068+0000","last_undegraded":"2024-08-19T01:33:05.466068+0000","last_fullsized":"2024-08-19T01:33:05.466068+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:32:37.889224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:32:37.889224+0000","last_clean_scrub_stamp":"2024-08-19T01:32:37.889224+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T11:31:30.461651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"84'1","reported_seq":18,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:33:05.474420+0000","last_change":"2024-08-19T01:32:41.232656+0000","last_active":"2024-08-19T01:33:05.474420+0000","last_peered":"2024-08-19T01:33:05.474420+0000","last_clean":"2024-08-19T01:33:05.474420+0000","last_became_active":"2024-08-19T01:32:41.232274+0000","last_became_peered":"2024-08-19T01:32:41.232274+0000","last_unstale":"2024-08-19T01:33:05.474420+0000","last_undegraded":"2024-08-19T01:33:05.474420+0000","last_fullsized":"2024-08-19T01:33:05.474420+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_clean_scrub_stamp":"2024-08-19T01:32:40.202519+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T03:38:40.695652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.5","version":"84'1","reported_seq":24,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:33:05.472592+0000","last_change":"2024-08-19T01:32:39.050809+0000","last_active":"2024-08-19T01:33:05.472592+0000","last_peered":"2024-08-19T01:33:05.472592+0000","last_clean":"2024-08-19T01:33:05.472592+0000","last_became_active":"2024-08-19T01:32:39.050434+0000","last_became_peered":"2024-08-19T01:32:39.050434+0000","last_unstale":"2024-08-19T01:33:05.472592+0000","last_undegraded":"2024-08-19T01:33:05.472592+0000","last_fullsized":"2024-08-19T01:33:05.472592+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:32:37.889224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:32:37.889224+0000","last_clean_scrub_stamp":"2024-08-19T01:32:37.889224+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T13:04:28.828117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:33:05.477935+0000","last_change":"2024-08-19T01:32:41.228485+0000","last_active":"2024-08-19T01:33:05.477935+0000","last_peered":"2024-08-19T01:33:05.477935+0000","last_clean":"2024-08-19T01:33:05.477935+0000","last_became_active":"2024-08-19T01:32:41.227238+0000","last_became_peered":"2024-08-19T01:32:41.227238+0000","last_unstale":"2024-08-19T01:33:05.477935+0000","last_undegraded":"2024-08-19T01:33:05.477935+0000","last_fullsized":"2024-08-19T01:33:05.477935+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_clean_scrub_stamp":"2024-08-19T01:32:40.202519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T10:55:55.939382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"86'11","reported_seq":33,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:33:07.225904+0000","last_change":"2024-08-19T01:32:38.915994+0000","last_active":"2024-08-19T01:33:07.225904+0000","last_peered":"2024-08-19T01:33:07.225904+0000","last_clean":"2024-08-19T01:33:07.225904+0000","last_became_active":"2024-08-19T01:32:38.914714+0000","last_became_peered":"2024-08-19T01:32:38.914714+0000","last_unstale":"2024-08-19T01:33:07.225904+0000","last_undegraded":"2024-08-19T01:33:07.225904+0000","last_fullsized":"2024-08-19T01:33:07.225904+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:32:37.889224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:32:37.889224+0000","last_clean_scrub_stamp":"2024-08-19T01:32:37.889224+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T05:38:07.274274+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:33:05.472281+0000","last_change":"2024-08-19T01:32:41.236611+0000","last_active":"2024-08-19T01:33:05.472281+0000","last_peered":"2024-08-19T01:33:05.472281+0000","last_clean":"2024-08-19T01:33:05.472281+0000","last_became_active":"2024-08-19T01:32:41.233821+0000","last_became_peered":"2024-08-19T01:32:41.233821+0000","last_unstale":"2024-08-19T01:33:05.472281+0000","last_undegraded":"2024-08-19T01:33:05.472281+0000","last_fullsized":"2024-08-19T01:33:05.472281+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_clean_scrub_stamp":"2024-08-19T01:32:40.202519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T11:34:07.879034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"84'4","reported_seq":26,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:33:05.470567+0000","last_change":"2024-08-19T01:32:39.245411+0000","last_active":"2024-08-19T01:33:05.470567+0000","last_peered":"2024-08-19T01:33:05.470567+0000","last_clean":"2024-08-19T01:33:05.470567+0000","last_became_active":"2024-08-19T01:32:39.244980+0000","last_became_peered":"2024-08-19T01:32:39.244980+0000","last_unstale":"2024-08-19T01:33:05.470567+0000","last_undegraded":"2024-08-19T01:33:05.470567+0000","last_fullsized":"2024-08-19T01:33:05.470567+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:32:37.889224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:32:37.889224+0000","last_clean_scrub_stamp":"2024-08-19T01:32:37.889224+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T02:06:57.878051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":23,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:33:05.478955+0000","last_change":"2024-08-19T01:33:05.478955+0000","last_active":"2024-08-19T01:33:05.478955+0000","last_peered":"2024-08-19T01:33:05.478955+0000","last_clean":"2024-08-19T01:33:05.478955+0000","last_became_active":"2024-08-19T01:33:05.478446+0000","last_became_peered":"2024-08-19T01:33:05.478446+0000","last_unstale":"2024-08-19T01:33:05.478955+0000","last_undegraded":"2024-08-19T01:33:05.478955+0000","last_fullsized":"2024-08-19T01:33:05.478955+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_clean_scrub_stamp":"2024-08-19T01:32:40.202519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T11:58:24.285438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,6],"acting":[0,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"84'2","reported_seq":24,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:33:05.701553+0000","last_change":"2024-08-19T01:32:39.050100+0000","last_active":"2024-08-19T01:33:05.701553+0000","last_peered":"2024-08-19T01:33:05.701553+0000","last_clean":"2024-08-19T01:33:05.701553+0000","last_became_active":"2024-08-19T01:32:39.049807+0000","last_became_peered":"2024-08-19T01:32:39.049807+0000","last_unstale":"2024-08-19T01:33:05.701553+0000","last_undegraded":"2024-08-19T01:33:05.701553+0000","last_fullsized":"2024-08-19T01:33:05.701553+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:32:37.889224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:32:37.889224+0000","last_clean_scrub_stamp":"2024-08-19T01:32:37.889224+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T07:06:22.361609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:33:05.471209+0000","last_change":"2024-08-19T01:32:41.228982+0000","last_active":"2024-08-19T01:33:05.471209+0000","last_peered":"2024-08-19T01:33:05.471209+0000","last_clean":"2024-08-19T01:33:05.471209+0000","last_became_active":"2024-08-19T01:32:41.228562+0000","last_became_peered":"2024-08-19T01:32:41.228562+0000","last_unstale":"2024-08-19T01:33:05.471209+0000","last_undegraded":"2024-08-19T01:33:05.471209+0000","last_fullsized":"2024-08-19T01:33:05.471209+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_clean_scrub_stamp":"2024-08-19T01:32:40.202519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T03:14:42.480359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"84'3","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:33:05.471722+0000","last_change":"2024-08-19T01:32:38.913677+0000","last_active":"2024-08-19T01:33:05.471722+0000","last_peered":"2024-08-19T01:33:05.471722+0000","last_clean":"2024-08-19T01:33:05.471722+0000","last_became_active":"2024-08-19T01:32:38.913091+0000","last_became_peered":"2024-08-19T01:32:38.913091+0000","last_unstale":"2024-08-19T01:33:05.471722+0000","last_undegraded":"2024-08-19T01:33:05.471722+0000","last_fullsized":"2024-08-19T01:33:05.471722+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:32:37.889224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:32:37.889224+0000","last_clean_scrub_stamp":"2024-08-19T01:32:37.889224+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T12:39:16.489246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:33:05.469625+0000","last_change":"2024-08-19T01:32:41.239380+0000","last_active":"2024-08-19T01:33:05.469625+0000","last_peered":"2024-08-19T01:33:05.469625+0000","last_clean":"2024-08-19T01:33:05.469625+0000","last_became_active":"2024-08-19T01:32:41.239060+0000","last_became_peered":"2024-08-19T01:32:41.239060+0000","last_unstale":"2024-08-19T01:33:05.469625+0000","last_undegraded":"2024-08-19T01:33:05.469625+0000","last_fullsized":"2024-08-19T01:33:05.469625+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_clean_scrub_stamp":"2024-08-19T01:32:40.202519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T02:21:59.630705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"84'1","reported_seq":15,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:33:05.479101+0000","last_change":"2024-08-19T01:33:05.478578+0000","last_active":"2024-08-19T01:33:05.479101+0000","last_peered":"2024-08-19T01:33:05.479101+0000","last_clean":"2024-08-19T01:33:05.479101+0000","last_became_active":"2024-08-19T01:33:05.473433+0000","last_became_peered":"2024-08-19T01:33:05.473433+0000","last_unstale":"2024-08-19T01:33:05.479101+0000","last_undegraded":"2024-08-19T01:33:05.479101+0000","last_fullsized":"2024-08-19T01:33:05.479101+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:32:37.889224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:32:37.889224+0000","last_clean_scrub_stamp":"2024-08-19T01:32:37.889224+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T05:06:29.975613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,3,8],"acting":[11,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:33:05.472017+0000","last_change":"2024-08-19T01:32:41.236012+0000","last_active":"2024-08-19T01:33:05.472017+0000","last_peered":"2024-08-19T01:33:05.472017+0000","last_clean":"2024-08-19T01:33:05.472017+0000","last_became_active":"2024-08-19T01:32:41.235724+0000","last_became_peered":"2024-08-19T01:32:41.235724+0000","last_unstale":"2024-08-19T01:33:05.472017+0000","last_undegraded":"2024-08-19T01:33:05.472017+0000","last_fullsized":"2024-08-19T01:33:05.472017+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_clean_scrub_stamp":"2024-08-19T01:32:40.202519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T08:45:08.865144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"84'4","reported_seq":27,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:33:05.471339+0000","last_change":"2024-08-19T01:32:38.915170+0000","last_active":"2024-08-19T01:33:05.471339+0000","last_peered":"2024-08-19T01:33:05.471339+0000","last_clean":"2024-08-19T01:33:05.471339+0000","last_became_active":"2024-08-19T01:32:38.914812+0000","last_became_peered":"2024-08-19T01:32:38.914812+0000","last_unstale":"2024-08-19T01:33:05.471339+0000","last_undegraded":"2024-08-19T01:33:05.471339+0000","last_fullsized":"2024-08-19T01:33:05.471339+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:32:37.889224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:32:37.889224+0000","last_clean_scrub_stamp":"2024-08-19T01:32:37.889224+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T11:46:39.409065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:33:05.472797+0000","last_change":"2024-08-19T01:32:41.234133+0000","last_active":"2024-08-19T01:33:05.472797+0000","last_peered":"2024-08-19T01:33:05.472797+0000","last_clean":"2024-08-19T01:33:05.472797+0000","last_became_active":"2024-08-19T01:32:41.233492+0000","last_became_peered":"2024-08-19T01:32:41.233492+0000","last_unstale":"2024-08-19T01:33:05.472797+0000","last_undegraded":"2024-08-19T01:33:05.472797+0000","last_fullsized":"2024-08-19T01:33:05.472797+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_clean_scrub_stamp":"2024-08-19T01:32:40.202519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T06:11:12.587678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:33:05.469674+0000","last_change":"2024-08-19T01:32:41.232628+0000","last_active":"2024-08-19T01:33:05.469674+0000","last_peered":"2024-08-19T01:33:05.469674+0000","last_clean":"2024-08-19T01:33:05.469674+0000","last_became_active":"2024-08-19T01:32:41.232295+0000","last_became_peered":"2024-08-19T01:32:41.232295+0000","last_unstale":"2024-08-19T01:33:05.469674+0000","last_undegraded":"2024-08-19T01:33:05.469674+0000","last_fullsized":"2024-08-19T01:33:05.469674+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_clean_scrub_stamp":"2024-08-19T01:32:40.202519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T02:01:09.490553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"84'2","reported_seq":24,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:33:05.472666+0000","last_change":"2024-08-19T01:32:38.912000+0000","last_active":"2024-08-19T01:33:05.472666+0000","last_peered":"2024-08-19T01:33:05.472666+0000","last_clean":"2024-08-19T01:33:05.472666+0000","last_became_active":"2024-08-19T01:32:38.911535+0000","last_became_peered":"2024-08-19T01:32:38.911535+0000","last_unstale":"2024-08-19T01:33:05.472666+0000","last_undegraded":"2024-08-19T01:33:05.472666+0000","last_fullsized":"2024-08-19T01:33:05.472666+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:32:37.889224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:32:37.889224+0000","last_clean_scrub_stamp":"2024-08-19T01:32:37.889224+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T07:12:49.358384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:33:05.701576+0000","last_change":"2024-08-19T01:32:41.233507+0000","last_active":"2024-08-19T01:33:05.701576+0000","last_peered":"2024-08-19T01:33:05.701576+0000","last_clean":"2024-08-19T01:33:05.701576+0000","last_became_active":"2024-08-19T01:32:41.233179+0000","last_became_peered":"2024-08-19T01:32:41.233179+0000","last_unstale":"2024-08-19T01:33:05.701576+0000","last_undegraded":"2024-08-19T01:33:05.701576+0000","last_fullsized":"2024-08-19T01:33:05.701576+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_clean_scrub_stamp":"2024-08-19T01:32:40.202519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T11:47:44.780859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1d","version":"84'4","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:33:05.483750+0000","last_change":"2024-08-19T01:33:05.483322+0000","last_active":"2024-08-19T01:33:05.483750+0000","last_peered":"2024-08-19T01:33:05.483750+0000","last_clean":"2024-08-19T01:33:05.483750+0000","last_became_active":"2024-08-19T01:33:05.475496+0000","last_became_peered":"2024-08-19T01:33:05.475496+0000","last_unstale":"2024-08-19T01:33:05.483750+0000","last_undegraded":"2024-08-19T01:33:05.483750+0000","last_fullsized":"2024-08-19T01:33:05.483750+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:32:37.889224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:32:37.889224+0000","last_clean_scrub_stamp":"2024-08-19T01:32:37.889224+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T01:57:48.143495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":10,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,7],"acting":[11,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:33:05.469647+0000","last_change":"2024-08-19T01:32:41.244641+0000","last_active":"2024-08-19T01:33:05.469647+0000","last_peered":"2024-08-19T01:33:05.469647+0000","last_clean":"2024-08-19T01:33:05.469647+0000","last_became_active":"2024-08-19T01:32:41.244311+0000","last_became_peered":"2024-08-19T01:32:41.244311+0000","last_unstale":"2024-08-19T01:33:05.469647+0000","last_undegraded":"2024-08-19T01:33:05.469647+0000","last_fullsized":"2024-08-19T01:33:05.469647+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_clean_scrub_stamp":"2024-08-19T01:32:40.202519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T10:39:12.591092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"84'2","reported_seq":24,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:33:05.472352+0000","last_change":"2024-08-19T01:32:39.051314+0000","last_active":"2024-08-19T01:33:05.472352+0000","last_peered":"2024-08-19T01:33:05.472352+0000","last_clean":"2024-08-19T01:33:05.472352+0000","last_became_active":"2024-08-19T01:32:39.050998+0000","last_became_peered":"2024-08-19T01:32:39.050998+0000","last_unstale":"2024-08-19T01:33:05.472352+0000","last_undegraded":"2024-08-19T01:33:05.472352+0000","last_fullsized":"2024-08-19T01:33:05.472352+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:32:37.889224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:32:37.889224+0000","last_clean_scrub_stamp":"2024-08-19T01:32:37.889224+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T09:21:15.689149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:33:05.472112+0000","last_change":"2024-08-19T01:32:41.234762+0000","last_active":"2024-08-19T01:33:05.472112+0000","last_peered":"2024-08-19T01:33:05.472112+0000","last_clean":"2024-08-19T01:33:05.472112+0000","last_became_active":"2024-08-19T01:32:41.234471+0000","last_became_peered":"2024-08-19T01:32:41.234471+0000","last_unstale":"2024-08-19T01:33:05.472112+0000","last_undegraded":"2024-08-19T01:33:05.472112+0000","last_fullsized":"2024-08-19T01:33:05.472112+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_clean_scrub_stamp":"2024-08-19T01:32:40.202519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T08:34:33.110938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"84'9","reported_seq":32,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:33:05.466470+0000","last_change":"2024-08-19T01:32:38.917068+0000","last_active":"2024-08-19T01:33:05.466470+0000","last_peered":"2024-08-19T01:33:05.466470+0000","last_clean":"2024-08-19T01:33:05.466470+0000","last_became_active":"2024-08-19T01:32:38.915399+0000","last_became_peered":"2024-08-19T01:32:38.915399+0000","last_unstale":"2024-08-19T01:33:05.466470+0000","last_undegraded":"2024-08-19T01:33:05.466470+0000","last_fullsized":"2024-08-19T01:33:05.466470+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:32:37.889224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:32:37.889224+0000","last_clean_scrub_stamp":"2024-08-19T01:32:37.889224+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T04:17:25.061208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":65006,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":364,"num_read_kb":250,"num_write":164,"num_write_kb":110,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":1012,"num_keys_recovered":10,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":417792,"data_stored":195018,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":139,"ondisk_log_size":139,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4751360,"data_stored":4722944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8}],"osd_stats":[{"osd":11,"up_from":78,"seq":335007449115,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32100,"kb_used_data":916,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713052,"statfs":{"total":95995035648,"available":95962165248,"internally_reserved":0,"allocated":937984,"data_stored":451051,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":71,"seq":304942678048,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28612,"kb_used_data":1596,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716540,"statfs":{"total":95995035648,"available":95965736960,"internally_reserved":0,"allocated":1634304,"data_stored":1103538,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":66,"seq":283467841572,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32108,"kb_used_data":932,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713044,"statfs":{"total":95995035648,"available":95962157056,"internally_reserved":0,"allocated":954368,"data_stored":451287,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":58,"seq":249108103210,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32108,"kb_used_data":932,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713044,"statfs":{"total":95995035648,"available":95962157056,"internally_reserved":0,"allocated":954368,"data_stored":451207,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":51,"seq":219043332142,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32696,"kb_used_data":1512,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712456,"statfs":{"total":95995035648,"available":95961554944,"internally_reserved":0,"allocated":1548288,"data_stored":1043526,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":46,"seq":197568495667,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27940,"kb_used_data":920,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717212,"statfs":{"total":95995035648,"available":95966425088,"internally_reserved":0,"allocated":942080,"data_stored":450676,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":38,"seq":163208757304,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28548,"kb_used_data":1524,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716604,"statfs":{"total":95995035648,"available":95965802496,"internally_reserved":0,"allocated":1560576,"data_stored":1042820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":33,"seq":141733920828,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32112,"kb_used_data":932,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713040,"statfs":{"total":95995035648,"available":95962152960,"internally_reserved":0,"allocated":954368,"data_stored":452515,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182465,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27956,"kb_used_data":936,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717196,"statfs":{"total":95995035648,"available":95966408704,"internally_reserved":0,"allocated":958464,"data_stored":450987,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411399,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36332,"kb_used_data":996,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708820,"statfs":{"total":95995035648,"available":95957831680,"internally_reserved":0,"allocated":1019904,"data_stored":511813,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574921,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27996,"kb_used_data":980,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717156,"statfs":{"total":95995035648,"available":95966367744,"internally_reserved":0,"allocated":1003520,"data_stored":511085,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738447,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36284,"kb_used_data":940,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708868,"statfs":{"total":95995035648,"available":95957880832,"internally_reserved":0,"allocated":962560,"data_stored":451947,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":69632,"data_stored":60616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":86016,"data_stored":61344,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2046,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1866,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2572,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":102400,"data_stored":62584,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-08-19T01:33:52.984 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833 shell --fsid bad8780e-5dc9-11ef-bccf-c7b262605968 -- ceph pg dump --format=json 2024-08-19T01:33:53.379 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/bad8780e-5dc9-11ef-bccf-c7b262605968/mon.a/config 2024-08-19T01:33:53.404 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:33:53 smithi083 ceph-mon[17120]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' 2024-08-19T01:33:53.404 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:33:53 smithi083 ceph-mon[17120]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' 2024-08-19T01:33:53.405 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:33:53 smithi083 ceph-mon[17120]: from='client.14775 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-08-19T01:33:53.405 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:33:53 smithi083 ceph-mon[17120]: pgmap v310: 97 pgs: 97 active+clean; 640 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-08-19T01:33:53.405 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:33:53 smithi083 ceph-mon[17120]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-19T01:33:53.405 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:33:53 smithi083 ceph-mon[17120]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-19T01:33:53.405 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:33:53 smithi083 ceph-mon[17120]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' 2024-08-19T01:33:53.452 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:33:53 smithi117 ceph-mon[22016]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' 2024-08-19T01:33:53.452 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:33:53 smithi117 ceph-mon[22016]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' 2024-08-19T01:33:53.453 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:33:53 smithi117 ceph-mon[22016]: from='client.14775 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-08-19T01:33:53.453 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:33:53 smithi117 ceph-mon[22016]: pgmap v310: 97 pgs: 97 active+clean; 640 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-08-19T01:33:53.453 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:33:53 smithi117 ceph-mon[22016]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-19T01:33:53.453 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:33:53 smithi117 ceph-mon[22016]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-19T01:33:53.453 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:33:53 smithi117 ceph-mon[22016]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' 2024-08-19T01:33:53.466 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:33:53 smithi167 ceph-mon[21875]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' 2024-08-19T01:33:53.466 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:33:53 smithi167 ceph-mon[21875]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' 2024-08-19T01:33:53.466 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:33:53 smithi167 ceph-mon[21875]: from='client.14775 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-08-19T01:33:53.466 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:33:53 smithi167 ceph-mon[21875]: pgmap v310: 97 pgs: 97 active+clean; 640 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-08-19T01:33:53.466 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:33:53 smithi167 ceph-mon[21875]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-19T01:33:53.466 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:33:53 smithi167 ceph-mon[21875]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-19T01:33:53.466 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:33:53 smithi167 ceph-mon[21875]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' 2024-08-19T01:33:53.933 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:53.931+0000 7f9c5b600640 1 -- 172.21.15.83:0/3000768587 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9c541078b0 msgr2=0x7f9c54107c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:33:53.933 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:53.931+0000 7f9c5b600640 1 --2- 172.21.15.83:0/3000768587 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9c541078b0 0x7f9c54107c90 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f9c48009930 tx=0x7f9c4802a420 comp rx=0 tx=0).stop 2024-08-19T01:33:53.934 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:53.931+0000 7f9c5b600640 1 -- 172.21.15.83:0/3000768587 shutdown_connections 2024-08-19T01:33:53.934 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:53.931+0000 7f9c5b600640 1 --2- 172.21.15.83:0/3000768587 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9c54102250 0x7f9c5410e780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:33:53.934 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:53.931+0000 7f9c5b600640 1 --2- 172.21.15.83:0/3000768587 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9c541018b0 0x7f9c54101d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:33:53.934 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:53.931+0000 7f9c5b600640 1 --2- 172.21.15.83:0/3000768587 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9c541078b0 0x7f9c54107c90 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:33:53.934 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:53.931+0000 7f9c5b600640 1 -- 172.21.15.83:0/3000768587 >> 172.21.15.83:0/3000768587 conn(0x7f9c540fd5f0 msgr2=0x7f9c540ffa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:33:53.934 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:53.931+0000 7f9c5b600640 1 -- 172.21.15.83:0/3000768587 shutdown_connections 2024-08-19T01:33:53.934 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:53.931+0000 7f9c5b600640 1 -- 172.21.15.83:0/3000768587 wait complete. 2024-08-19T01:33:53.934 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:53.932+0000 7f9c5b600640 1 Processor -- start 2024-08-19T01:33:53.939 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:53.932+0000 7f9c5b600640 1 -- start start 2024-08-19T01:33:53.939 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:53.933+0000 7f9c5b600640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9c541018b0 0x7f9c541982e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:33:53.939 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:53.933+0000 7f9c5b600640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9c54102250 0x7f9c54198820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:33:53.939 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:53.933+0000 7f9c5b600640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9c541078b0 0x7f9c5419cbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:33:53.939 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:53.933+0000 7f9c5b600640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f9c54111d00 con 0x7f9c541018b0 2024-08-19T01:33:53.939 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:53.933+0000 7f9c5b600640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f9c54111b80 con 0x7f9c541078b0 2024-08-19T01:33:53.939 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:53.933+0000 7f9c5b600640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f9c54111e80 con 0x7f9c54102250 2024-08-19T01:33:53.939 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:53.933+0000 7f9c58c00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9c541018b0 0x7f9c541982e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:33:53.939 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:53.933+0000 7f9c58c00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9c541018b0 0x7f9c541982e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:57656/0 (socket says 172.21.15.83:57656) 2024-08-19T01:33:53.939 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:53.933+0000 7f9c58c00640 1 -- 172.21.15.83:0/1457689926 learned_addr learned my addr 172.21.15.83:0/1457689926 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-08-19T01:33:53.939 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:53.933+0000 7f9c59600640 1 --2- 172.21.15.83:0/1457689926 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9c541078b0 0x7f9c5419cbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:33:53.940 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:53.933+0000 7f9c53e00640 1 --2- 172.21.15.83:0/1457689926 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9c54102250 0x7f9c54198820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:33:53.940 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:53.933+0000 7f9c58c00640 1 -- 172.21.15.83:0/1457689926 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9c54102250 msgr2=0x7f9c54198820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:33:53.940 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:53.933+0000 7f9c58c00640 1 --2- 172.21.15.83:0/1457689926 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9c54102250 0x7f9c54198820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:33:53.940 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:53.933+0000 7f9c58c00640 1 -- 172.21.15.83:0/1457689926 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9c541078b0 msgr2=0x7f9c5419cbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:33:53.940 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:53.933+0000 7f9c58c00640 1 --2- 172.21.15.83:0/1457689926 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9c541078b0 0x7f9c5419cbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:33:53.940 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:53.933+0000 7f9c58c00640 1 -- 172.21.15.83:0/1457689926 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9c5419d290 con 0x7f9c541018b0 2024-08-19T01:33:53.940 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:53.933+0000 7f9c59600640 1 --2- 172.21.15.83:0/1457689926 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9c541078b0 0x7f9c5419cbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-19T01:33:53.940 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:53.934+0000 7f9c58c00640 1 --2- 172.21.15.83:0/1457689926 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9c541018b0 0x7f9c541982e0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f9c4802a940 tx=0x7f9c48003b90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:33:53.940 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:53.934+0000 7f9c51600640 1 -- 172.21.15.83:0/1457689926 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c4803f070 con 0x7f9c541018b0 2024-08-19T01:33:53.940 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:53.934+0000 7f9c51600640 1 -- 172.21.15.83:0/1457689926 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9c48031c90 con 0x7f9c541018b0 2024-08-19T01:33:53.940 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:53.934+0000 7f9c5b600640 1 -- 172.21.15.83:0/1457689926 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9c5419d520 con 0x7f9c541018b0 2024-08-19T01:33:53.940 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:53.934+0000 7f9c51600640 1 -- 172.21.15.83:0/1457689926 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c48035030 con 0x7f9c541018b0 2024-08-19T01:33:53.940 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:53.934+0000 7f9c5b600640 1 -- 172.21.15.83:0/1457689926 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9c541a4e60 con 0x7f9c541018b0 2024-08-19T01:33:53.940 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:53.935+0000 7f9c5b600640 1 -- 172.21.15.83:0/1457689926 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9c54102ff0 con 0x7f9c541018b0 2024-08-19T01:33:53.941 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:53.937+0000 7f9c51600640 1 -- 172.21.15.83:0/1457689926 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f9c48042070 con 0x7f9c541018b0 2024-08-19T01:33:53.941 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:53.938+0000 7f9c51600640 1 --2- 172.21.15.83:0/1457689926 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f9c3c0a2c10 0x7f9c3c0a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:33:53.941 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:53.938+0000 7f9c51600640 1 -- 172.21.15.83:0/1457689926 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8269+0+0 (secure 0 0 0) 0x7f9c480efa40 con 0x7f9c541018b0 2024-08-19T01:33:53.941 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:53.938+0000 7f9c53e00640 1 --2- 172.21.15.83:0/1457689926 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f9c3c0a2c10 0x7f9c3c0a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:33:53.941 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:53.939+0000 7f9c53e00640 1 --2- 172.21.15.83:0/1457689926 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f9c3c0a2c10 0x7f9c3c0a50d0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f9c54106ec0 tx=0x7f9c40005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:33:53.944 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:53.942+0000 7f9c51600640 1 -- 172.21.15.83:0/1457689926 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7f9c480bc7c0 con 0x7f9c541018b0 2024-08-19T01:33:54.117 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:54.115+0000 7f9c5b600640 1 -- 172.21.15.83:0/1457689926 --> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f9c54062360 con 0x7f9c3c0a2c10 2024-08-19T01:33:54.124 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:54.122+0000 7f9c51600640 1 -- 172.21.15.83:0/1457689926 <== mgr.14150 v2:172.21.15.83:6800/683740916 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258723 (secure 0 0 0) 0x7f9c54062360 con 0x7f9c3c0a2c10 2024-08-19T01:33:54.127 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-08-19T01:33:54.128 INFO:teuthology.orchestra.run.smithi083.stderr:dumped all 2024-08-19T01:33:54.129 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:54.126+0000 7f9c5b600640 1 -- 172.21.15.83:0/1457689926 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f9c3c0a2c10 msgr2=0x7f9c3c0a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:33:54.130 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:54.126+0000 7f9c5b600640 1 --2- 172.21.15.83:0/1457689926 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f9c3c0a2c10 0x7f9c3c0a50d0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f9c54106ec0 tx=0x7f9c40005f90 comp rx=0 tx=0).stop 2024-08-19T01:33:54.130 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:54.126+0000 7f9c5b600640 1 -- 172.21.15.83:0/1457689926 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9c541018b0 msgr2=0x7f9c541982e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:33:54.130 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:54.126+0000 7f9c5b600640 1 --2- 172.21.15.83:0/1457689926 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9c541018b0 0x7f9c541982e0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f9c4802a940 tx=0x7f9c48003b90 comp rx=0 tx=0).stop 2024-08-19T01:33:54.130 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:54.127+0000 7f9c5b600640 1 -- 172.21.15.83:0/1457689926 shutdown_connections 2024-08-19T01:33:54.130 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:54.127+0000 7f9c5b600640 1 --2- 172.21.15.83:0/1457689926 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f9c3c0a2c10 0x7f9c3c0a50d0 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:33:54.130 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:54.127+0000 7f9c5b600640 1 --2- 172.21.15.83:0/1457689926 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9c541078b0 0x7f9c5419cbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:33:54.130 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:54.127+0000 7f9c5b600640 1 --2- 172.21.15.83:0/1457689926 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9c54102250 0x7f9c54198820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:33:54.130 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:54.127+0000 7f9c5b600640 1 --2- 172.21.15.83:0/1457689926 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9c541018b0 0x7f9c541982e0 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:33:54.130 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:54.127+0000 7f9c5b600640 1 -- 172.21.15.83:0/1457689926 >> 172.21.15.83:0/1457689926 conn(0x7f9c540fd5f0 msgr2=0x7f9c540feeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:33:54.130 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:54.127+0000 7f9c5b600640 1 -- 172.21.15.83:0/1457689926 shutdown_connections 2024-08-19T01:33:54.130 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:54.127+0000 7f9c5b600640 1 -- 172.21.15.83:0/1457689926 wait complete. 2024-08-19T01:33:54.935 INFO:teuthology.orchestra.run.smithi083.stdout:{"pg_ready":true,"pg_map":{"version":310,"stamp":"2024-08-19T01:33:52.375706+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":655608,"num_objects":64,"num_object_clones":0,"num_object_copies":192,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":495,"num_read_kb":364,"num_write":347,"num_write_kb":3050,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":1181748,"num_keys_recovered":10,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":278,"ondisk_log_size":278,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":374792,"kb_used_data":13116,"kb_used_omap":18,"kb_used_meta":361517,"kb_avail":1124567032,"statfs":{"total":1151940427776,"available":1151556640768,"internally_reserved":0,"allocated":13430784,"data_stored":7372452,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19143,"internal_metadata":370193721},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":65,"num_read_kb":35,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002869"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:33:05.470153+0000","last_change":"2024-08-19T01:32:41.244214+0000","last_active":"2024-08-19T01:33:05.470153+0000","last_peered":"2024-08-19T01:33:05.470153+0000","last_clean":"2024-08-19T01:33:05.470153+0000","last_became_active":"2024-08-19T01:32:41.243889+0000","last_became_peered":"2024-08-19T01:32:41.243889+0000","last_unstale":"2024-08-19T01:33:05.470153+0000","last_undegraded":"2024-08-19T01:33:05.470153+0000","last_fullsized":"2024-08-19T01:33:05.470153+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_clean_scrub_stamp":"2024-08-19T01:32:40.202519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T13:10:18.230055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:33:05.468954+0000","last_change":"2024-08-19T01:32:41.242729+0000","last_active":"2024-08-19T01:33:05.468954+0000","last_peered":"2024-08-19T01:33:05.468954+0000","last_clean":"2024-08-19T01:33:05.468954+0000","last_became_active":"2024-08-19T01:32:41.242436+0000","last_became_peered":"2024-08-19T01:32:41.242436+0000","last_unstale":"2024-08-19T01:33:05.468954+0000","last_undegraded":"2024-08-19T01:33:05.468954+0000","last_fullsized":"2024-08-19T01:33:05.468954+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_clean_scrub_stamp":"2024-08-19T01:32:40.202519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T13:19:32.010576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:33:05.468943+0000","last_change":"2024-08-19T01:32:41.247377+0000","last_active":"2024-08-19T01:33:05.468943+0000","last_peered":"2024-08-19T01:33:05.468943+0000","last_clean":"2024-08-19T01:33:05.468943+0000","last_became_active":"2024-08-19T01:32:41.246804+0000","last_became_peered":"2024-08-19T01:32:41.246804+0000","last_unstale":"2024-08-19T01:33:05.468943+0000","last_undegraded":"2024-08-19T01:33:05.468943+0000","last_fullsized":"2024-08-19T01:33:05.468943+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_clean_scrub_stamp":"2024-08-19T01:32:40.202519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T04:48:52.382687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:33:05.470397+0000","last_change":"2024-08-19T01:32:41.232900+0000","last_active":"2024-08-19T01:33:05.470397+0000","last_peered":"2024-08-19T01:33:05.470397+0000","last_clean":"2024-08-19T01:33:05.470397+0000","last_became_active":"2024-08-19T01:32:41.232573+0000","last_became_peered":"2024-08-19T01:32:41.232573+0000","last_unstale":"2024-08-19T01:33:05.470397+0000","last_undegraded":"2024-08-19T01:33:05.470397+0000","last_fullsized":"2024-08-19T01:33:05.470397+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_clean_scrub_stamp":"2024-08-19T01:32:40.202519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T09:42:36.309834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":23,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:33:05.478964+0000","last_change":"2024-08-19T01:33:05.478964+0000","last_active":"2024-08-19T01:33:05.478964+0000","last_peered":"2024-08-19T01:33:05.478964+0000","last_clean":"2024-08-19T01:33:05.478964+0000","last_became_active":"2024-08-19T01:33:05.478430+0000","last_became_peered":"2024-08-19T01:33:05.478430+0000","last_unstale":"2024-08-19T01:33:05.478964+0000","last_undegraded":"2024-08-19T01:33:05.478964+0000","last_fullsized":"2024-08-19T01:33:05.478964+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_clean_scrub_stamp":"2024-08-19T01:32:40.202519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T04:10:49.267550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,6],"acting":[8,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:33:05.474307+0000","last_change":"2024-08-19T01:32:41.228263+0000","last_active":"2024-08-19T01:33:05.474307+0000","last_peered":"2024-08-19T01:33:05.474307+0000","last_clean":"2024-08-19T01:33:05.474307+0000","last_became_active":"2024-08-19T01:32:41.227765+0000","last_became_peered":"2024-08-19T01:32:41.227765+0000","last_unstale":"2024-08-19T01:33:05.474307+0000","last_undegraded":"2024-08-19T01:33:05.474307+0000","last_fullsized":"2024-08-19T01:33:05.474307+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_clean_scrub_stamp":"2024-08-19T01:32:40.202519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T11:21:18.817886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:33:05.474641+0000","last_change":"2024-08-19T01:32:41.235680+0000","last_active":"2024-08-19T01:33:05.474641+0000","last_peered":"2024-08-19T01:33:05.474641+0000","last_clean":"2024-08-19T01:33:05.474641+0000","last_became_active":"2024-08-19T01:32:41.234150+0000","last_became_peered":"2024-08-19T01:32:41.234150+0000","last_unstale":"2024-08-19T01:33:05.474641+0000","last_undegraded":"2024-08-19T01:33:05.474641+0000","last_fullsized":"2024-08-19T01:33:05.474641+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_clean_scrub_stamp":"2024-08-19T01:32:40.202519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T10:28:08.908632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:33:05.471317+0000","last_change":"2024-08-19T01:32:41.230170+0000","last_active":"2024-08-19T01:33:05.471317+0000","last_peered":"2024-08-19T01:33:05.471317+0000","last_clean":"2024-08-19T01:33:05.471317+0000","last_became_active":"2024-08-19T01:32:41.229467+0000","last_became_peered":"2024-08-19T01:32:41.229467+0000","last_unstale":"2024-08-19T01:33:05.471317+0000","last_undegraded":"2024-08-19T01:33:05.471317+0000","last_fullsized":"2024-08-19T01:33:05.471317+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_clean_scrub_stamp":"2024-08-19T01:32:40.202519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T10:24:15.832910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:33:05.701955+0000","last_change":"2024-08-19T01:32:41.230607+0000","last_active":"2024-08-19T01:33:05.701955+0000","last_peered":"2024-08-19T01:33:05.701955+0000","last_clean":"2024-08-19T01:33:05.701955+0000","last_became_active":"2024-08-19T01:32:41.229979+0000","last_became_peered":"2024-08-19T01:32:41.229979+0000","last_unstale":"2024-08-19T01:33:05.701955+0000","last_undegraded":"2024-08-19T01:33:05.701955+0000","last_fullsized":"2024-08-19T01:33:05.701955+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_clean_scrub_stamp":"2024-08-19T01:32:40.202519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T02:23:46.736139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:33:05.474802+0000","last_change":"2024-08-19T01:32:41.233576+0000","last_active":"2024-08-19T01:33:05.474802+0000","last_peered":"2024-08-19T01:33:05.474802+0000","last_clean":"2024-08-19T01:33:05.474802+0000","last_became_active":"2024-08-19T01:32:41.231279+0000","last_became_peered":"2024-08-19T01:32:41.231279+0000","last_unstale":"2024-08-19T01:33:05.474802+0000","last_undegraded":"2024-08-19T01:33:05.474802+0000","last_fullsized":"2024-08-19T01:33:05.474802+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_clean_scrub_stamp":"2024-08-19T01:32:40.202519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T07:32:45.341628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:33:05.473054+0000","last_change":"2024-08-19T01:32:41.233192+0000","last_active":"2024-08-19T01:33:05.473054+0000","last_peered":"2024-08-19T01:33:05.473054+0000","last_clean":"2024-08-19T01:33:05.473054+0000","last_became_active":"2024-08-19T01:32:41.230944+0000","last_became_peered":"2024-08-19T01:32:41.230944+0000","last_unstale":"2024-08-19T01:33:05.473054+0000","last_undegraded":"2024-08-19T01:33:05.473054+0000","last_fullsized":"2024-08-19T01:33:05.473054+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_clean_scrub_stamp":"2024-08-19T01:32:40.202519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T08:55:29.036252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:33:05.470408+0000","last_change":"2024-08-19T01:32:41.234216+0000","last_active":"2024-08-19T01:33:05.470408+0000","last_peered":"2024-08-19T01:33:05.470408+0000","last_clean":"2024-08-19T01:33:05.470408+0000","last_became_active":"2024-08-19T01:32:41.233906+0000","last_became_peered":"2024-08-19T01:32:41.233906+0000","last_unstale":"2024-08-19T01:33:05.470408+0000","last_undegraded":"2024-08-19T01:33:05.470408+0000","last_fullsized":"2024-08-19T01:33:05.470408+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_clean_scrub_stamp":"2024-08-19T01:32:40.202519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T08:35:01.764494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:33:05.477816+0000","last_change":"2024-08-19T01:32:41.230651+0000","last_active":"2024-08-19T01:33:05.477816+0000","last_peered":"2024-08-19T01:33:05.477816+0000","last_clean":"2024-08-19T01:33:05.477816+0000","last_became_active":"2024-08-19T01:32:41.230253+0000","last_became_peered":"2024-08-19T01:32:41.230253+0000","last_unstale":"2024-08-19T01:33:05.477816+0000","last_undegraded":"2024-08-19T01:33:05.477816+0000","last_fullsized":"2024-08-19T01:33:05.477816+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_clean_scrub_stamp":"2024-08-19T01:32:40.202519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T09:41:59.942821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:33:05.468782+0000","last_change":"2024-08-19T01:32:41.240082+0000","last_active":"2024-08-19T01:33:05.468782+0000","last_peered":"2024-08-19T01:33:05.468782+0000","last_clean":"2024-08-19T01:33:05.468782+0000","last_became_active":"2024-08-19T01:32:41.239711+0000","last_became_peered":"2024-08-19T01:32:41.239711+0000","last_unstale":"2024-08-19T01:33:05.468782+0000","last_undegraded":"2024-08-19T01:33:05.468782+0000","last_fullsized":"2024-08-19T01:33:05.468782+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_clean_scrub_stamp":"2024-08-19T01:32:40.202519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T12:48:59.122307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":23,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:33:05.474883+0000","last_change":"2024-08-19T01:33:05.474883+0000","last_active":"2024-08-19T01:33:05.474883+0000","last_peered":"2024-08-19T01:33:05.474883+0000","last_clean":"2024-08-19T01:33:05.474883+0000","last_became_active":"2024-08-19T01:33:05.474502+0000","last_became_peered":"2024-08-19T01:33:05.474502+0000","last_unstale":"2024-08-19T01:33:05.474883+0000","last_undegraded":"2024-08-19T01:33:05.474883+0000","last_fullsized":"2024-08-19T01:33:05.474883+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_clean_scrub_stamp":"2024-08-19T01:32:40.202519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T07:02:01.171103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2],"acting":[0,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:33:05.701404+0000","last_change":"2024-08-19T01:32:41.240899+0000","last_active":"2024-08-19T01:33:05.701404+0000","last_peered":"2024-08-19T01:33:05.701404+0000","last_clean":"2024-08-19T01:33:05.701404+0000","last_became_active":"2024-08-19T01:32:41.240532+0000","last_became_peered":"2024-08-19T01:32:41.240532+0000","last_unstale":"2024-08-19T01:33:05.701404+0000","last_undegraded":"2024-08-19T01:33:05.701404+0000","last_fullsized":"2024-08-19T01:33:05.701404+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_clean_scrub_stamp":"2024-08-19T01:32:40.202519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T08:17:47.035887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:33:05.470183+0000","last_change":"2024-08-19T01:32:41.237977+0000","last_active":"2024-08-19T01:33:05.470183+0000","last_peered":"2024-08-19T01:33:05.470183+0000","last_clean":"2024-08-19T01:33:05.470183+0000","last_became_active":"2024-08-19T01:32:41.237634+0000","last_became_peered":"2024-08-19T01:32:41.237634+0000","last_unstale":"2024-08-19T01:33:05.470183+0000","last_undegraded":"2024-08-19T01:33:05.470183+0000","last_fullsized":"2024-08-19T01:33:05.470183+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_clean_scrub_stamp":"2024-08-19T01:32:40.202519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T12:48:37.005972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:33:05.466272+0000","last_change":"2024-08-19T01:32:41.236498+0000","last_active":"2024-08-19T01:33:05.466272+0000","last_peered":"2024-08-19T01:33:05.466272+0000","last_clean":"2024-08-19T01:33:05.466272+0000","last_became_active":"2024-08-19T01:32:41.236122+0000","last_became_peered":"2024-08-19T01:32:41.236122+0000","last_unstale":"2024-08-19T01:33:05.466272+0000","last_undegraded":"2024-08-19T01:33:05.466272+0000","last_fullsized":"2024-08-19T01:33:05.466272+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_clean_scrub_stamp":"2024-08-19T01:32:40.202519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T02:31:23.648064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:33:05.466818+0000","last_change":"2024-08-19T01:32:41.234607+0000","last_active":"2024-08-19T01:33:05.466818+0000","last_peered":"2024-08-19T01:33:05.466818+0000","last_clean":"2024-08-19T01:33:05.466818+0000","last_became_active":"2024-08-19T01:32:41.232867+0000","last_became_peered":"2024-08-19T01:32:41.232867+0000","last_unstale":"2024-08-19T01:33:05.466818+0000","last_undegraded":"2024-08-19T01:33:05.466818+0000","last_fullsized":"2024-08-19T01:33:05.466818+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_clean_scrub_stamp":"2024-08-19T01:32:40.202519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T12:32:38.441848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:33:05.469342+0000","last_change":"2024-08-19T01:32:41.231705+0000","last_active":"2024-08-19T01:33:05.469342+0000","last_peered":"2024-08-19T01:33:05.469342+0000","last_clean":"2024-08-19T01:33:05.469342+0000","last_became_active":"2024-08-19T01:32:41.231262+0000","last_became_peered":"2024-08-19T01:32:41.231262+0000","last_unstale":"2024-08-19T01:33:05.469342+0000","last_undegraded":"2024-08-19T01:33:05.469342+0000","last_fullsized":"2024-08-19T01:33:05.469342+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_clean_scrub_stamp":"2024-08-19T01:32:40.202519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T10:41:15.025471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:33:05.477712+0000","last_change":"2024-08-19T01:32:41.243192+0000","last_active":"2024-08-19T01:33:05.477712+0000","last_peered":"2024-08-19T01:33:05.477712+0000","last_clean":"2024-08-19T01:33:05.477712+0000","last_became_active":"2024-08-19T01:32:41.242704+0000","last_became_peered":"2024-08-19T01:32:41.242704+0000","last_unstale":"2024-08-19T01:33:05.477712+0000","last_undegraded":"2024-08-19T01:33:05.477712+0000","last_fullsized":"2024-08-19T01:33:05.477712+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_clean_scrub_stamp":"2024-08-19T01:32:40.202519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T01:44:43.270213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:33:05.701950+0000","last_change":"2024-08-19T01:32:41.239908+0000","last_active":"2024-08-19T01:33:05.701950+0000","last_peered":"2024-08-19T01:33:05.701950+0000","last_clean":"2024-08-19T01:33:05.701950+0000","last_became_active":"2024-08-19T01:32:41.239438+0000","last_became_peered":"2024-08-19T01:32:41.239438+0000","last_unstale":"2024-08-19T01:33:05.701950+0000","last_undegraded":"2024-08-19T01:33:05.701950+0000","last_fullsized":"2024-08-19T01:33:05.701950+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_clean_scrub_stamp":"2024-08-19T01:32:40.202519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T11:03:26.822111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:33:05.471582+0000","last_change":"2024-08-19T01:32:41.226373+0000","last_active":"2024-08-19T01:33:05.471582+0000","last_peered":"2024-08-19T01:33:05.471582+0000","last_clean":"2024-08-19T01:33:05.471582+0000","last_became_active":"2024-08-19T01:32:41.226089+0000","last_became_peered":"2024-08-19T01:32:41.226089+0000","last_unstale":"2024-08-19T01:33:05.471582+0000","last_undegraded":"2024-08-19T01:33:05.471582+0000","last_fullsized":"2024-08-19T01:33:05.471582+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_clean_scrub_stamp":"2024-08-19T01:32:40.202519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T01:33:56.639300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:33:05.469289+0000","last_change":"2024-08-19T01:32:41.237829+0000","last_active":"2024-08-19T01:33:05.469289+0000","last_peered":"2024-08-19T01:33:05.469289+0000","last_clean":"2024-08-19T01:33:05.469289+0000","last_became_active":"2024-08-19T01:32:41.237518+0000","last_became_peered":"2024-08-19T01:32:41.237518+0000","last_unstale":"2024-08-19T01:33:05.469289+0000","last_undegraded":"2024-08-19T01:33:05.469289+0000","last_fullsized":"2024-08-19T01:33:05.469289+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_clean_scrub_stamp":"2024-08-19T01:32:40.202519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T02:25:11.307658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:33:05.472158+0000","last_change":"2024-08-19T01:32:41.234508+0000","last_active":"2024-08-19T01:33:05.472158+0000","last_peered":"2024-08-19T01:33:05.472158+0000","last_clean":"2024-08-19T01:33:05.472158+0000","last_became_active":"2024-08-19T01:32:41.232869+0000","last_became_peered":"2024-08-19T01:32:41.232869+0000","last_unstale":"2024-08-19T01:33:05.472158+0000","last_undegraded":"2024-08-19T01:33:05.472158+0000","last_fullsized":"2024-08-19T01:33:05.472158+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_clean_scrub_stamp":"2024-08-19T01:32:40.202519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T11:20:55.097576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:33:05.466993+0000","last_change":"2024-08-19T01:32:41.230536+0000","last_active":"2024-08-19T01:33:05.466993+0000","last_peered":"2024-08-19T01:33:05.466993+0000","last_clean":"2024-08-19T01:33:05.466993+0000","last_became_active":"2024-08-19T01:32:41.230179+0000","last_became_peered":"2024-08-19T01:32:41.230179+0000","last_unstale":"2024-08-19T01:33:05.466993+0000","last_undegraded":"2024-08-19T01:33:05.466993+0000","last_fullsized":"2024-08-19T01:33:05.466993+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_clean_scrub_stamp":"2024-08-19T01:32:40.202519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T02:25:09.441363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:33:05.469818+0000","last_change":"2024-08-19T01:32:41.230695+0000","last_active":"2024-08-19T01:33:05.469818+0000","last_peered":"2024-08-19T01:33:05.469818+0000","last_clean":"2024-08-19T01:33:05.469818+0000","last_became_active":"2024-08-19T01:32:41.230324+0000","last_became_peered":"2024-08-19T01:32:41.230324+0000","last_unstale":"2024-08-19T01:33:05.469818+0000","last_undegraded":"2024-08-19T01:33:05.469818+0000","last_fullsized":"2024-08-19T01:33:05.469818+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_clean_scrub_stamp":"2024-08-19T01:32:40.202519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T05:38:09.317605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:33:05.477784+0000","last_change":"2024-08-19T01:32:41.238325+0000","last_active":"2024-08-19T01:33:05.477784+0000","last_peered":"2024-08-19T01:33:05.477784+0000","last_clean":"2024-08-19T01:33:05.477784+0000","last_became_active":"2024-08-19T01:32:41.237768+0000","last_became_peered":"2024-08-19T01:32:41.237768+0000","last_unstale":"2024-08-19T01:33:05.477784+0000","last_undegraded":"2024-08-19T01:33:05.477784+0000","last_fullsized":"2024-08-19T01:33:05.477784+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_clean_scrub_stamp":"2024-08-19T01:32:40.202519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T08:45:06.866241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:33:05.468967+0000","last_change":"2024-08-19T01:32:41.246950+0000","last_active":"2024-08-19T01:33:05.468967+0000","last_peered":"2024-08-19T01:33:05.468967+0000","last_clean":"2024-08-19T01:33:05.468967+0000","last_became_active":"2024-08-19T01:32:41.245441+0000","last_became_peered":"2024-08-19T01:32:41.245441+0000","last_unstale":"2024-08-19T01:33:05.468967+0000","last_undegraded":"2024-08-19T01:33:05.468967+0000","last_fullsized":"2024-08-19T01:33:05.468967+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_clean_scrub_stamp":"2024-08-19T01:32:40.202519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T03:11:06.921258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:33:05.466512+0000","last_change":"2024-08-19T01:32:41.244810+0000","last_active":"2024-08-19T01:33:05.466512+0000","last_peered":"2024-08-19T01:33:05.466512+0000","last_clean":"2024-08-19T01:33:05.466512+0000","last_became_active":"2024-08-19T01:32:41.244401+0000","last_became_peered":"2024-08-19T01:32:41.244401+0000","last_unstale":"2024-08-19T01:33:05.466512+0000","last_undegraded":"2024-08-19T01:33:05.466512+0000","last_fullsized":"2024-08-19T01:33:05.466512+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_clean_scrub_stamp":"2024-08-19T01:32:40.202519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T04:54:57.728181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:33:05.466793+0000","last_change":"2024-08-19T01:32:41.235977+0000","last_active":"2024-08-19T01:33:05.466793+0000","last_peered":"2024-08-19T01:33:05.466793+0000","last_clean":"2024-08-19T01:33:05.466793+0000","last_became_active":"2024-08-19T01:32:41.235693+0000","last_became_peered":"2024-08-19T01:32:41.235693+0000","last_unstale":"2024-08-19T01:33:05.466793+0000","last_undegraded":"2024-08-19T01:33:05.466793+0000","last_fullsized":"2024-08-19T01:33:05.466793+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_clean_scrub_stamp":"2024-08-19T01:32:40.202519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T04:59:15.766532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":23,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:33:05.473318+0000","last_change":"2024-08-19T01:33:05.473318+0000","last_active":"2024-08-19T01:33:05.473318+0000","last_peered":"2024-08-19T01:33:05.473318+0000","last_clean":"2024-08-19T01:33:05.473318+0000","last_became_active":"2024-08-19T01:33:05.472885+0000","last_became_peered":"2024-08-19T01:33:05.472885+0000","last_unstale":"2024-08-19T01:33:05.473318+0000","last_undegraded":"2024-08-19T01:33:05.473318+0000","last_fullsized":"2024-08-19T01:33:05.473318+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_clean_scrub_stamp":"2024-08-19T01:32:40.202519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T07:49:35.525930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,9],"acting":[11,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:33:05.471500+0000","last_change":"2024-08-19T01:32:41.230846+0000","last_active":"2024-08-19T01:33:05.471500+0000","last_peered":"2024-08-19T01:33:05.471500+0000","last_clean":"2024-08-19T01:33:05.471500+0000","last_became_active":"2024-08-19T01:32:41.230405+0000","last_became_peered":"2024-08-19T01:32:41.230405+0000","last_unstale":"2024-08-19T01:33:05.471500+0000","last_undegraded":"2024-08-19T01:33:05.471500+0000","last_fullsized":"2024-08-19T01:33:05.471500+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_clean_scrub_stamp":"2024-08-19T01:32:40.202519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T08:59:46.241898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1b","version":"84'6","reported_seq":99,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:33:46.531957+0000","last_change":"2024-08-19T01:33:05.479889+0000","last_active":"2024-08-19T01:33:46.531957+0000","last_peered":"2024-08-19T01:33:46.531957+0000","last_clean":"2024-08-19T01:33:46.531957+0000","last_became_active":"2024-08-19T01:33:05.474698+0000","last_became_peered":"2024-08-19T01:33:05.474698+0000","last_unstale":"2024-08-19T01:33:46.531957+0000","last_undegraded":"2024-08-19T01:33:46.531957+0000","last_fullsized":"2024-08-19T01:33:46.531957+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:32:37.889224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:32:37.889224+0000","last_clean_scrub_stamp":"2024-08-19T01:32:37.889224+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T08:41:29.053471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1012,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":57,"num_read_kb":3,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":1012,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7],"acting":[5,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":11,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:33:05.475899+0000","last_change":"2024-08-19T01:33:05.474083+0000","last_active":"2024-08-19T01:33:05.475899+0000","last_peered":"2024-08-19T01:33:05.475899+0000","last_clean":"2024-08-19T01:33:05.475899+0000","last_became_active":"2024-08-19T01:33:05.473718+0000","last_became_peered":"2024-08-19T01:33:05.473718+0000","last_unstale":"2024-08-19T01:33:05.475899+0000","last_undegraded":"2024-08-19T01:33:05.475899+0000","last_fullsized":"2024-08-19T01:33:05.475899+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_clean_scrub_stamp":"2024-08-19T01:32:40.202519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T02:25:11.955170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,4,3],"acting":[9,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.1a","version":"84'3","reported_seq":35,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:33:05.705917+0000","last_change":"2024-08-19T01:33:05.705917+0000","last_active":"2024-08-19T01:33:05.705917+0000","last_peered":"2024-08-19T01:33:05.705917+0000","last_clean":"2024-08-19T01:33:05.705917+0000","last_became_active":"2024-08-19T01:33:05.702775+0000","last_became_peered":"2024-08-19T01:33:05.702775+0000","last_unstale":"2024-08-19T01:33:05.705917+0000","last_undegraded":"2024-08-19T01:33:05.705917+0000","last_fullsized":"2024-08-19T01:33:05.705917+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:32:37.889224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:32:37.889224+0000","last_clean_scrub_stamp":"2024-08-19T01:32:37.889224+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T03:58:29.989815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,6],"acting":[10,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.19","version":"84'2","reported_seq":24,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:33:05.477490+0000","last_change":"2024-08-19T01:32:38.911757+0000","last_active":"2024-08-19T01:33:05.477490+0000","last_peered":"2024-08-19T01:33:05.477490+0000","last_clean":"2024-08-19T01:33:05.477490+0000","last_became_active":"2024-08-19T01:32:38.910832+0000","last_became_peered":"2024-08-19T01:32:38.910832+0000","last_unstale":"2024-08-19T01:33:05.477490+0000","last_undegraded":"2024-08-19T01:33:05.477490+0000","last_fullsized":"2024-08-19T01:33:05.477490+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:32:37.889224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:32:37.889224+0000","last_clean_scrub_stamp":"2024-08-19T01:32:37.889224+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T04:46:54.986693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:33:05.471462+0000","last_change":"2024-08-19T01:32:41.231130+0000","last_active":"2024-08-19T01:33:05.471462+0000","last_peered":"2024-08-19T01:33:05.471462+0000","last_clean":"2024-08-19T01:33:05.471462+0000","last_became_active":"2024-08-19T01:32:41.230883+0000","last_became_peered":"2024-08-19T01:32:41.230883+0000","last_unstale":"2024-08-19T01:33:05.471462+0000","last_undegraded":"2024-08-19T01:33:05.471462+0000","last_fullsized":"2024-08-19T01:33:05.471462+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_clean_scrub_stamp":"2024-08-19T01:32:40.202519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T06:33:11.856202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"84'1","reported_seq":23,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:33:05.470862+0000","last_change":"2024-08-19T01:32:38.916868+0000","last_active":"2024-08-19T01:33:05.470862+0000","last_peered":"2024-08-19T01:33:05.470862+0000","last_clean":"2024-08-19T01:33:05.470862+0000","last_became_active":"2024-08-19T01:32:38.916500+0000","last_became_peered":"2024-08-19T01:32:38.916500+0000","last_unstale":"2024-08-19T01:33:05.470862+0000","last_undegraded":"2024-08-19T01:33:05.470862+0000","last_fullsized":"2024-08-19T01:33:05.470862+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:32:37.889224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:32:37.889224+0000","last_clean_scrub_stamp":"2024-08-19T01:32:37.889224+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T12:30:07.085829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:33:05.465832+0000","last_change":"2024-08-19T01:32:41.235290+0000","last_active":"2024-08-19T01:33:05.465832+0000","last_peered":"2024-08-19T01:33:05.465832+0000","last_clean":"2024-08-19T01:33:05.465832+0000","last_became_active":"2024-08-19T01:32:41.235013+0000","last_became_peered":"2024-08-19T01:32:41.235013+0000","last_unstale":"2024-08-19T01:33:05.465832+0000","last_undegraded":"2024-08-19T01:33:05.465832+0000","last_fullsized":"2024-08-19T01:33:05.465832+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_clean_scrub_stamp":"2024-08-19T01:32:40.202519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T09:50:14.837080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"84'3","reported_seq":82,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:33:47.532715+0000","last_change":"2024-08-19T01:32:38.912023+0000","last_active":"2024-08-19T01:33:47.532715+0000","last_peered":"2024-08-19T01:33:47.532715+0000","last_clean":"2024-08-19T01:33:47.532715+0000","last_became_active":"2024-08-19T01:32:38.911679+0000","last_became_peered":"2024-08-19T01:32:38.911679+0000","last_unstale":"2024-08-19T01:33:47.532715+0000","last_undegraded":"2024-08-19T01:33:47.532715+0000","last_fullsized":"2024-08-19T01:33:47.532715+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:32:37.889224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:32:37.889224+0000","last_clean_scrub_stamp":"2024-08-19T01:32:37.889224+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T12:08:51.118338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":57,"num_read_kb":57,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:33:05.701747+0000","last_change":"2024-08-19T01:32:41.232264+0000","last_active":"2024-08-19T01:33:05.701747+0000","last_peered":"2024-08-19T01:33:05.701747+0000","last_clean":"2024-08-19T01:33:05.701747+0000","last_became_active":"2024-08-19T01:32:41.231788+0000","last_became_peered":"2024-08-19T01:32:41.231788+0000","last_unstale":"2024-08-19T01:33:05.701747+0000","last_undegraded":"2024-08-19T01:33:05.701747+0000","last_fullsized":"2024-08-19T01:33:05.701747+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_clean_scrub_stamp":"2024-08-19T01:32:40.202519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T04:59:06.443992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"84'2","reported_seq":24,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:33:05.472954+0000","last_change":"2024-08-19T01:32:38.915770+0000","last_active":"2024-08-19T01:33:05.472954+0000","last_peered":"2024-08-19T01:33:05.472954+0000","last_clean":"2024-08-19T01:33:05.472954+0000","last_became_active":"2024-08-19T01:32:38.915378+0000","last_became_peered":"2024-08-19T01:32:38.915378+0000","last_unstale":"2024-08-19T01:33:05.472954+0000","last_undegraded":"2024-08-19T01:33:05.472954+0000","last_fullsized":"2024-08-19T01:33:05.472954+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:32:37.889224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:32:37.889224+0000","last_clean_scrub_stamp":"2024-08-19T01:32:37.889224+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T09:16:14.513172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:33:05.469512+0000","last_change":"2024-08-19T01:32:41.243476+0000","last_active":"2024-08-19T01:33:05.469512+0000","last_peered":"2024-08-19T01:33:05.469512+0000","last_clean":"2024-08-19T01:33:05.469512+0000","last_became_active":"2024-08-19T01:32:41.243190+0000","last_became_peered":"2024-08-19T01:32:41.243190+0000","last_unstale":"2024-08-19T01:33:05.469512+0000","last_undegraded":"2024-08-19T01:33:05.469512+0000","last_fullsized":"2024-08-19T01:33:05.469512+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_clean_scrub_stamp":"2024-08-19T01:32:40.202519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T12:07:34.096919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"84'4","reported_seq":27,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:33:05.470664+0000","last_change":"2024-08-19T01:32:39.051458+0000","last_active":"2024-08-19T01:33:05.470664+0000","last_peered":"2024-08-19T01:33:05.470664+0000","last_clean":"2024-08-19T01:33:05.470664+0000","last_became_active":"2024-08-19T01:32:39.051113+0000","last_became_peered":"2024-08-19T01:32:39.051113+0000","last_unstale":"2024-08-19T01:33:05.470664+0000","last_undegraded":"2024-08-19T01:33:05.470664+0000","last_fullsized":"2024-08-19T01:33:05.470664+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:32:37.889224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:32:37.889224+0000","last_clean_scrub_stamp":"2024-08-19T01:32:37.889224+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T02:15:51.371054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:33:05.701579+0000","last_change":"2024-08-19T01:32:41.236453+0000","last_active":"2024-08-19T01:33:05.701579+0000","last_peered":"2024-08-19T01:33:05.701579+0000","last_clean":"2024-08-19T01:33:05.701579+0000","last_became_active":"2024-08-19T01:32:41.236135+0000","last_became_peered":"2024-08-19T01:32:41.236135+0000","last_unstale":"2024-08-19T01:33:05.701579+0000","last_undegraded":"2024-08-19T01:33:05.701579+0000","last_fullsized":"2024-08-19T01:33:05.701579+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_clean_scrub_stamp":"2024-08-19T01:32:40.202519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T12:38:41.398299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"86'28","reported_seq":173,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:33:48.534218+0000","last_change":"2024-08-19T01:32:38.914831+0000","last_active":"2024-08-19T01:33:48.534218+0000","last_peered":"2024-08-19T01:33:48.534218+0000","last_clean":"2024-08-19T01:33:48.534218+0000","last_became_active":"2024-08-19T01:32:38.914326+0000","last_became_peered":"2024-08-19T01:32:38.914326+0000","last_unstale":"2024-08-19T01:33:48.534218+0000","last_undegraded":"2024-08-19T01:33:48.534218+0000","last_fullsized":"2024-08-19T01:33:48.534218+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:32:37.889224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:32:37.889224+0000","last_clean_scrub_stamp":"2024-08-19T01:32:37.889224+0000","objects_scrubbed":0,"log_size":28,"log_dups_size":0,"ondisk_log_size":28,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T02:15:25.263974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":60616,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":123,"num_read_kb":119,"num_write":29,"num_write_kb":68,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:33:05.472530+0000","last_change":"2024-08-19T01:32:41.237945+0000","last_active":"2024-08-19T01:33:05.472530+0000","last_peered":"2024-08-19T01:33:05.472530+0000","last_clean":"2024-08-19T01:33:05.472530+0000","last_became_active":"2024-08-19T01:32:41.237362+0000","last_became_peered":"2024-08-19T01:32:41.237362+0000","last_unstale":"2024-08-19T01:33:05.472530+0000","last_undegraded":"2024-08-19T01:33:05.472530+0000","last_fullsized":"2024-08-19T01:33:05.472530+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_clean_scrub_stamp":"2024-08-19T01:32:40.202519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T07:29:17.180054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"84'6","reported_seq":29,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:33:05.466940+0000","last_change":"2024-08-19T01:32:38.912453+0000","last_active":"2024-08-19T01:33:05.466940+0000","last_peered":"2024-08-19T01:33:05.466940+0000","last_clean":"2024-08-19T01:33:05.466940+0000","last_became_active":"2024-08-19T01:32:38.912051+0000","last_became_peered":"2024-08-19T01:32:38.912051+0000","last_unstale":"2024-08-19T01:33:05.466940+0000","last_undegraded":"2024-08-19T01:33:05.466940+0000","last_fullsized":"2024-08-19T01:33:05.466940+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:32:37.889224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:32:37.889224+0000","last_clean_scrub_stamp":"2024-08-19T01:32:37.889224+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T06:49:58.810134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:33:05.477644+0000","last_change":"2024-08-19T01:32:41.236282+0000","last_active":"2024-08-19T01:33:05.477644+0000","last_peered":"2024-08-19T01:33:05.477644+0000","last_clean":"2024-08-19T01:33:05.477644+0000","last_became_active":"2024-08-19T01:32:41.235559+0000","last_became_peered":"2024-08-19T01:32:41.235559+0000","last_unstale":"2024-08-19T01:33:05.477644+0000","last_undegraded":"2024-08-19T01:33:05.477644+0000","last_fullsized":"2024-08-19T01:33:05.477644+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_clean_scrub_stamp":"2024-08-19T01:32:40.202519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T07:51:52.157094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"84'3","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:33:05.470566+0000","last_change":"2024-08-19T01:32:38.915688+0000","last_active":"2024-08-19T01:33:05.470566+0000","last_peered":"2024-08-19T01:33:05.470566+0000","last_clean":"2024-08-19T01:33:05.470566+0000","last_became_active":"2024-08-19T01:32:38.915297+0000","last_became_peered":"2024-08-19T01:32:38.915297+0000","last_unstale":"2024-08-19T01:33:05.470566+0000","last_undegraded":"2024-08-19T01:33:05.470566+0000","last_fullsized":"2024-08-19T01:33:05.470566+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:32:37.889224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:32:37.889224+0000","last_clean_scrub_stamp":"2024-08-19T01:32:37.889224+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T01:38:12.780588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:33:05.472886+0000","last_change":"2024-08-19T01:32:41.233766+0000","last_active":"2024-08-19T01:33:05.472886+0000","last_peered":"2024-08-19T01:33:05.472886+0000","last_clean":"2024-08-19T01:33:05.472886+0000","last_became_active":"2024-08-19T01:32:41.233425+0000","last_became_peered":"2024-08-19T01:32:41.233425+0000","last_unstale":"2024-08-19T01:33:05.472886+0000","last_undegraded":"2024-08-19T01:33:05.472886+0000","last_fullsized":"2024-08-19T01:33:05.472886+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_clean_scrub_stamp":"2024-08-19T01:32:40.202519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T10:29:15.471639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"84'4","reported_seq":85,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:33:46.394817+0000","last_change":"2024-08-19T01:32:38.912610+0000","last_active":"2024-08-19T01:33:46.394817+0000","last_peered":"2024-08-19T01:33:46.394817+0000","last_clean":"2024-08-19T01:33:46.394817+0000","last_became_active":"2024-08-19T01:32:38.912040+0000","last_became_peered":"2024-08-19T01:32:38.912040+0000","last_unstale":"2024-08-19T01:33:46.394817+0000","last_undegraded":"2024-08-19T01:33:46.394817+0000","last_fullsized":"2024-08-19T01:33:46.394817+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:32:37.889224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:32:37.889224+0000","last_clean_scrub_stamp":"2024-08-19T01:32:37.889224+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T09:26:05.966391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":978,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":60,"num_read_kb":4,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:33:05.472892+0000","last_change":"2024-08-19T01:32:41.235945+0000","last_active":"2024-08-19T01:33:05.472892+0000","last_peered":"2024-08-19T01:33:05.472892+0000","last_clean":"2024-08-19T01:33:05.472892+0000","last_became_active":"2024-08-19T01:32:41.235633+0000","last_became_peered":"2024-08-19T01:32:41.235633+0000","last_unstale":"2024-08-19T01:33:05.472892+0000","last_undegraded":"2024-08-19T01:33:05.472892+0000","last_fullsized":"2024-08-19T01:33:05.472892+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_clean_scrub_stamp":"2024-08-19T01:32:40.202519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T08:37:37.682175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"84'15","reported_seq":38,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:33:05.471087+0000","last_change":"2024-08-19T01:32:39.052850+0000","last_active":"2024-08-19T01:33:05.471087+0000","last_peered":"2024-08-19T01:33:05.471087+0000","last_clean":"2024-08-19T01:33:05.471087+0000","last_became_active":"2024-08-19T01:32:39.052455+0000","last_became_peered":"2024-08-19T01:32:39.052455+0000","last_unstale":"2024-08-19T01:33:05.471087+0000","last_undegraded":"2024-08-19T01:33:05.471087+0000","last_fullsized":"2024-08-19T01:33:05.471087+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:32:37.889224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:32:37.889224+0000","last_clean_scrub_stamp":"2024-08-19T01:32:37.889224+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T03:25:08.279911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":23,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:33:05.702575+0000","last_change":"2024-08-19T01:33:05.702575+0000","last_active":"2024-08-19T01:33:05.702575+0000","last_peered":"2024-08-19T01:33:05.702575+0000","last_clean":"2024-08-19T01:33:05.702575+0000","last_became_active":"2024-08-19T01:33:05.702110+0000","last_became_peered":"2024-08-19T01:33:05.702110+0000","last_unstale":"2024-08-19T01:33:05.702575+0000","last_undegraded":"2024-08-19T01:33:05.702575+0000","last_fullsized":"2024-08-19T01:33:05.702575+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_clean_scrub_stamp":"2024-08-19T01:32:40.202519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T12:18:51.102180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,1],"acting":[5,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"84'2","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:33:05.472730+0000","last_change":"2024-08-19T01:32:38.912090+0000","last_active":"2024-08-19T01:33:05.472730+0000","last_peered":"2024-08-19T01:33:05.472730+0000","last_clean":"2024-08-19T01:33:05.472730+0000","last_became_active":"2024-08-19T01:32:38.911753+0000","last_became_peered":"2024-08-19T01:32:38.911753+0000","last_unstale":"2024-08-19T01:33:05.472730+0000","last_undegraded":"2024-08-19T01:33:05.472730+0000","last_fullsized":"2024-08-19T01:33:05.472730+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:32:37.889224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:32:37.889224+0000","last_clean_scrub_stamp":"2024-08-19T01:32:37.889224+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T12:56:42.982062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:33:05.469805+0000","last_change":"2024-08-19T01:32:41.230429+0000","last_active":"2024-08-19T01:33:05.469805+0000","last_peered":"2024-08-19T01:33:05.469805+0000","last_clean":"2024-08-19T01:33:05.469805+0000","last_became_active":"2024-08-19T01:32:41.230115+0000","last_became_peered":"2024-08-19T01:32:41.230115+0000","last_unstale":"2024-08-19T01:33:05.469805+0000","last_undegraded":"2024-08-19T01:33:05.469805+0000","last_fullsized":"2024-08-19T01:33:05.469805+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_clean_scrub_stamp":"2024-08-19T01:32:40.202519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T01:50:51.084130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.e","version":"84'2","reported_seq":24,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:33:05.477826+0000","last_change":"2024-08-19T01:32:38.914557+0000","last_active":"2024-08-19T01:33:05.477826+0000","last_peered":"2024-08-19T01:33:05.477826+0000","last_clean":"2024-08-19T01:33:05.477826+0000","last_became_active":"2024-08-19T01:32:38.914115+0000","last_became_peered":"2024-08-19T01:32:38.914115+0000","last_unstale":"2024-08-19T01:33:05.477826+0000","last_undegraded":"2024-08-19T01:33:05.477826+0000","last_fullsized":"2024-08-19T01:33:05.477826+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:32:37.889224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:32:37.889224+0000","last_clean_scrub_stamp":"2024-08-19T01:32:37.889224+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T06:30:00.523513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:33:05.473436+0000","last_change":"2024-08-19T01:32:41.234883+0000","last_active":"2024-08-19T01:33:05.473436+0000","last_peered":"2024-08-19T01:33:05.473436+0000","last_clean":"2024-08-19T01:33:05.473436+0000","last_became_active":"2024-08-19T01:32:41.233901+0000","last_became_peered":"2024-08-19T01:32:41.233901+0000","last_unstale":"2024-08-19T01:33:05.473436+0000","last_undegraded":"2024-08-19T01:33:05.473436+0000","last_fullsized":"2024-08-19T01:33:05.473436+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_clean_scrub_stamp":"2024-08-19T01:32:40.202519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T12:08:46.305259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"84'1","reported_seq":24,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:33:05.470777+0000","last_change":"2024-08-19T01:32:38.913441+0000","last_active":"2024-08-19T01:33:05.470777+0000","last_peered":"2024-08-19T01:33:05.470777+0000","last_clean":"2024-08-19T01:33:05.470777+0000","last_became_active":"2024-08-19T01:32:38.913115+0000","last_became_peered":"2024-08-19T01:32:38.913115+0000","last_unstale":"2024-08-19T01:33:05.470777+0000","last_undegraded":"2024-08-19T01:33:05.470777+0000","last_fullsized":"2024-08-19T01:33:05.470777+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:32:37.889224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:32:37.889224+0000","last_clean_scrub_stamp":"2024-08-19T01:32:37.889224+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T11:43:15.290980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"84'1","reported_seq":18,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:33:05.472934+0000","last_change":"2024-08-19T01:32:41.235370+0000","last_active":"2024-08-19T01:33:05.472934+0000","last_peered":"2024-08-19T01:33:05.472934+0000","last_clean":"2024-08-19T01:33:05.472934+0000","last_became_active":"2024-08-19T01:32:41.235012+0000","last_became_peered":"2024-08-19T01:32:41.235012+0000","last_unstale":"2024-08-19T01:33:05.472934+0000","last_undegraded":"2024-08-19T01:33:05.472934+0000","last_fullsized":"2024-08-19T01:33:05.472934+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_clean_scrub_stamp":"2024-08-19T01:32:40.202519+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T02:18:31.134974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"84'3","reported_seq":26,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:33:05.466872+0000","last_change":"2024-08-19T01:32:39.245560+0000","last_active":"2024-08-19T01:33:05.466872+0000","last_peered":"2024-08-19T01:33:05.466872+0000","last_clean":"2024-08-19T01:33:05.466872+0000","last_became_active":"2024-08-19T01:32:39.245118+0000","last_became_peered":"2024-08-19T01:32:39.245118+0000","last_unstale":"2024-08-19T01:33:05.466872+0000","last_undegraded":"2024-08-19T01:33:05.466872+0000","last_fullsized":"2024-08-19T01:33:05.466872+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:32:37.889224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:32:37.889224+0000","last_clean_scrub_stamp":"2024-08-19T01:32:37.889224+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T02:18:24.605709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:33:05.473113+0000","last_change":"2024-08-19T01:32:41.234452+0000","last_active":"2024-08-19T01:33:05.473113+0000","last_peered":"2024-08-19T01:33:05.473113+0000","last_clean":"2024-08-19T01:33:05.473113+0000","last_became_active":"2024-08-19T01:32:41.233852+0000","last_became_peered":"2024-08-19T01:32:41.233852+0000","last_unstale":"2024-08-19T01:33:05.473113+0000","last_undegraded":"2024-08-19T01:33:05.473113+0000","last_fullsized":"2024-08-19T01:33:05.473113+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_clean_scrub_stamp":"2024-08-19T01:32:40.202519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T06:18:34.457239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"22'137","reported_seq":338,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:33:05.473232+0000","last_change":"2024-08-19T01:31:27.659394+0000","last_active":"2024-08-19T01:33:05.473232+0000","last_peered":"2024-08-19T01:33:05.473232+0000","last_clean":"2024-08-19T01:33:05.473232+0000","last_became_active":"2024-08-19T01:31:27.659007+0000","last_became_peered":"2024-08-19T01:31:27.659007+0000","last_unstale":"2024-08-19T01:33:05.473232+0000","last_undegraded":"2024-08-19T01:33:05.473232+0000","last_fullsized":"2024-08-19T01:33:05.473232+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:28:16.912251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:28:16.912251+0000","last_clean_scrub_stamp":"2024-08-19T01:28:16.912251+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T05:53:01.660619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"84'3","reported_seq":27,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:33:05.470433+0000","last_change":"2024-08-19T01:32:38.913408+0000","last_active":"2024-08-19T01:33:05.470433+0000","last_peered":"2024-08-19T01:33:05.470433+0000","last_clean":"2024-08-19T01:33:05.470433+0000","last_became_active":"2024-08-19T01:32:38.912242+0000","last_became_peered":"2024-08-19T01:32:38.912242+0000","last_unstale":"2024-08-19T01:33:05.470433+0000","last_undegraded":"2024-08-19T01:33:05.470433+0000","last_fullsized":"2024-08-19T01:33:05.470433+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:32:37.889224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:32:37.889224+0000","last_clean_scrub_stamp":"2024-08-19T01:32:37.889224+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T07:26:48.121366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:33:05.471752+0000","last_change":"2024-08-19T01:32:41.232611+0000","last_active":"2024-08-19T01:33:05.471752+0000","last_peered":"2024-08-19T01:33:05.471752+0000","last_clean":"2024-08-19T01:33:05.471752+0000","last_became_active":"2024-08-19T01:32:41.232261+0000","last_became_peered":"2024-08-19T01:32:41.232261+0000","last_unstale":"2024-08-19T01:33:05.471752+0000","last_undegraded":"2024-08-19T01:33:05.471752+0000","last_fullsized":"2024-08-19T01:33:05.471752+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_clean_scrub_stamp":"2024-08-19T01:32:40.202519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T08:04:16.793933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"84'3","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:33:05.473104+0000","last_change":"2024-08-19T01:32:38.912765+0000","last_active":"2024-08-19T01:33:05.473104+0000","last_peered":"2024-08-19T01:33:05.473104+0000","last_clean":"2024-08-19T01:33:05.473104+0000","last_became_active":"2024-08-19T01:32:38.912274+0000","last_became_peered":"2024-08-19T01:32:38.912274+0000","last_unstale":"2024-08-19T01:33:05.473104+0000","last_undegraded":"2024-08-19T01:33:05.473104+0000","last_fullsized":"2024-08-19T01:33:05.473104+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:32:37.889224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:32:37.889224+0000","last_clean_scrub_stamp":"2024-08-19T01:32:37.889224+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T09:57:49.714827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:33:05.477455+0000","last_change":"2024-08-19T01:32:41.224449+0000","last_active":"2024-08-19T01:33:05.477455+0000","last_peered":"2024-08-19T01:33:05.477455+0000","last_clean":"2024-08-19T01:33:05.477455+0000","last_became_active":"2024-08-19T01:32:41.224064+0000","last_became_peered":"2024-08-19T01:32:41.224064+0000","last_unstale":"2024-08-19T01:33:05.477455+0000","last_undegraded":"2024-08-19T01:33:05.477455+0000","last_fullsized":"2024-08-19T01:33:05.477455+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_clean_scrub_stamp":"2024-08-19T01:32:40.202519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T08:53:33.948504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"84'4","reported_seq":85,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:33:49.399626+0000","last_change":"2024-08-19T01:32:39.050869+0000","last_active":"2024-08-19T01:33:49.399626+0000","last_peered":"2024-08-19T01:33:49.399626+0000","last_clean":"2024-08-19T01:33:49.399626+0000","last_became_active":"2024-08-19T01:32:39.050529+0000","last_became_peered":"2024-08-19T01:32:39.050529+0000","last_unstale":"2024-08-19T01:33:49.399626+0000","last_undegraded":"2024-08-19T01:33:49.399626+0000","last_fullsized":"2024-08-19T01:33:49.399626+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:32:37.889224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:32:37.889224+0000","last_clean_scrub_stamp":"2024-08-19T01:32:37.889224+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T12:52:52.788360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":59,"num_read_kb":59,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:33:05.470330+0000","last_change":"2024-08-19T01:32:41.237093+0000","last_active":"2024-08-19T01:33:05.470330+0000","last_peered":"2024-08-19T01:33:05.470330+0000","last_clean":"2024-08-19T01:33:05.470330+0000","last_became_active":"2024-08-19T01:32:41.236738+0000","last_became_peered":"2024-08-19T01:32:41.236738+0000","last_unstale":"2024-08-19T01:33:05.470330+0000","last_undegraded":"2024-08-19T01:33:05.470330+0000","last_fullsized":"2024-08-19T01:33:05.470330+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_clean_scrub_stamp":"2024-08-19T01:32:40.202519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T08:37:33.117344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"84'1","reported_seq":23,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:33:05.473189+0000","last_change":"2024-08-19T01:32:38.915010+0000","last_active":"2024-08-19T01:33:05.473189+0000","last_peered":"2024-08-19T01:33:05.473189+0000","last_clean":"2024-08-19T01:33:05.473189+0000","last_became_active":"2024-08-19T01:32:38.914687+0000","last_became_peered":"2024-08-19T01:32:38.914687+0000","last_unstale":"2024-08-19T01:33:05.473189+0000","last_undegraded":"2024-08-19T01:33:05.473189+0000","last_fullsized":"2024-08-19T01:33:05.473189+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:32:37.889224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:32:37.889224+0000","last_clean_scrub_stamp":"2024-08-19T01:32:37.889224+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T04:46:07.881778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:33:05.474056+0000","last_change":"2024-08-19T01:32:41.244069+0000","last_active":"2024-08-19T01:33:05.474056+0000","last_peered":"2024-08-19T01:33:05.474056+0000","last_clean":"2024-08-19T01:33:05.474056+0000","last_became_active":"2024-08-19T01:32:41.243482+0000","last_became_peered":"2024-08-19T01:32:41.243482+0000","last_unstale":"2024-08-19T01:33:05.474056+0000","last_undegraded":"2024-08-19T01:33:05.474056+0000","last_fullsized":"2024-08-19T01:33:05.474056+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_clean_scrub_stamp":"2024-08-19T01:32:40.202519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T02:08:28.044421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":22,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:33:05.466068+0000","last_change":"2024-08-19T01:32:39.051383+0000","last_active":"2024-08-19T01:33:05.466068+0000","last_peered":"2024-08-19T01:33:05.466068+0000","last_clean":"2024-08-19T01:33:05.466068+0000","last_became_active":"2024-08-19T01:32:39.050970+0000","last_became_peered":"2024-08-19T01:32:39.050970+0000","last_unstale":"2024-08-19T01:33:05.466068+0000","last_undegraded":"2024-08-19T01:33:05.466068+0000","last_fullsized":"2024-08-19T01:33:05.466068+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:32:37.889224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:32:37.889224+0000","last_clean_scrub_stamp":"2024-08-19T01:32:37.889224+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T11:31:30.461651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"84'1","reported_seq":18,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:33:05.474420+0000","last_change":"2024-08-19T01:32:41.232656+0000","last_active":"2024-08-19T01:33:05.474420+0000","last_peered":"2024-08-19T01:33:05.474420+0000","last_clean":"2024-08-19T01:33:05.474420+0000","last_became_active":"2024-08-19T01:32:41.232274+0000","last_became_peered":"2024-08-19T01:32:41.232274+0000","last_unstale":"2024-08-19T01:33:05.474420+0000","last_undegraded":"2024-08-19T01:33:05.474420+0000","last_fullsized":"2024-08-19T01:33:05.474420+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_clean_scrub_stamp":"2024-08-19T01:32:40.202519+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T03:38:40.695652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.5","version":"84'1","reported_seq":24,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:33:05.472592+0000","last_change":"2024-08-19T01:32:39.050809+0000","last_active":"2024-08-19T01:33:05.472592+0000","last_peered":"2024-08-19T01:33:05.472592+0000","last_clean":"2024-08-19T01:33:05.472592+0000","last_became_active":"2024-08-19T01:32:39.050434+0000","last_became_peered":"2024-08-19T01:32:39.050434+0000","last_unstale":"2024-08-19T01:33:05.472592+0000","last_undegraded":"2024-08-19T01:33:05.472592+0000","last_fullsized":"2024-08-19T01:33:05.472592+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:32:37.889224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:32:37.889224+0000","last_clean_scrub_stamp":"2024-08-19T01:32:37.889224+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T13:04:28.828117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:33:05.477935+0000","last_change":"2024-08-19T01:32:41.228485+0000","last_active":"2024-08-19T01:33:05.477935+0000","last_peered":"2024-08-19T01:33:05.477935+0000","last_clean":"2024-08-19T01:33:05.477935+0000","last_became_active":"2024-08-19T01:32:41.227238+0000","last_became_peered":"2024-08-19T01:32:41.227238+0000","last_unstale":"2024-08-19T01:33:05.477935+0000","last_undegraded":"2024-08-19T01:33:05.477935+0000","last_fullsized":"2024-08-19T01:33:05.477935+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_clean_scrub_stamp":"2024-08-19T01:32:40.202519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T10:55:55.939382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"86'11","reported_seq":33,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:33:07.225904+0000","last_change":"2024-08-19T01:32:38.915994+0000","last_active":"2024-08-19T01:33:07.225904+0000","last_peered":"2024-08-19T01:33:07.225904+0000","last_clean":"2024-08-19T01:33:07.225904+0000","last_became_active":"2024-08-19T01:32:38.914714+0000","last_became_peered":"2024-08-19T01:32:38.914714+0000","last_unstale":"2024-08-19T01:33:07.225904+0000","last_undegraded":"2024-08-19T01:33:07.225904+0000","last_fullsized":"2024-08-19T01:33:07.225904+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:32:37.889224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:32:37.889224+0000","last_clean_scrub_stamp":"2024-08-19T01:32:37.889224+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T05:38:07.274274+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:33:05.472281+0000","last_change":"2024-08-19T01:32:41.236611+0000","last_active":"2024-08-19T01:33:05.472281+0000","last_peered":"2024-08-19T01:33:05.472281+0000","last_clean":"2024-08-19T01:33:05.472281+0000","last_became_active":"2024-08-19T01:32:41.233821+0000","last_became_peered":"2024-08-19T01:32:41.233821+0000","last_unstale":"2024-08-19T01:33:05.472281+0000","last_undegraded":"2024-08-19T01:33:05.472281+0000","last_fullsized":"2024-08-19T01:33:05.472281+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_clean_scrub_stamp":"2024-08-19T01:32:40.202519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T11:34:07.879034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"84'4","reported_seq":26,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:33:05.470567+0000","last_change":"2024-08-19T01:32:39.245411+0000","last_active":"2024-08-19T01:33:05.470567+0000","last_peered":"2024-08-19T01:33:05.470567+0000","last_clean":"2024-08-19T01:33:05.470567+0000","last_became_active":"2024-08-19T01:32:39.244980+0000","last_became_peered":"2024-08-19T01:32:39.244980+0000","last_unstale":"2024-08-19T01:33:05.470567+0000","last_undegraded":"2024-08-19T01:33:05.470567+0000","last_fullsized":"2024-08-19T01:33:05.470567+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:32:37.889224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:32:37.889224+0000","last_clean_scrub_stamp":"2024-08-19T01:32:37.889224+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T02:06:57.878051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":23,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:33:05.478955+0000","last_change":"2024-08-19T01:33:05.478955+0000","last_active":"2024-08-19T01:33:05.478955+0000","last_peered":"2024-08-19T01:33:05.478955+0000","last_clean":"2024-08-19T01:33:05.478955+0000","last_became_active":"2024-08-19T01:33:05.478446+0000","last_became_peered":"2024-08-19T01:33:05.478446+0000","last_unstale":"2024-08-19T01:33:05.478955+0000","last_undegraded":"2024-08-19T01:33:05.478955+0000","last_fullsized":"2024-08-19T01:33:05.478955+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_clean_scrub_stamp":"2024-08-19T01:32:40.202519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T11:58:24.285438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,6],"acting":[0,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"84'2","reported_seq":24,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:33:05.701553+0000","last_change":"2024-08-19T01:32:39.050100+0000","last_active":"2024-08-19T01:33:05.701553+0000","last_peered":"2024-08-19T01:33:05.701553+0000","last_clean":"2024-08-19T01:33:05.701553+0000","last_became_active":"2024-08-19T01:32:39.049807+0000","last_became_peered":"2024-08-19T01:32:39.049807+0000","last_unstale":"2024-08-19T01:33:05.701553+0000","last_undegraded":"2024-08-19T01:33:05.701553+0000","last_fullsized":"2024-08-19T01:33:05.701553+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:32:37.889224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:32:37.889224+0000","last_clean_scrub_stamp":"2024-08-19T01:32:37.889224+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T07:06:22.361609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:33:05.471209+0000","last_change":"2024-08-19T01:32:41.228982+0000","last_active":"2024-08-19T01:33:05.471209+0000","last_peered":"2024-08-19T01:33:05.471209+0000","last_clean":"2024-08-19T01:33:05.471209+0000","last_became_active":"2024-08-19T01:32:41.228562+0000","last_became_peered":"2024-08-19T01:32:41.228562+0000","last_unstale":"2024-08-19T01:33:05.471209+0000","last_undegraded":"2024-08-19T01:33:05.471209+0000","last_fullsized":"2024-08-19T01:33:05.471209+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_clean_scrub_stamp":"2024-08-19T01:32:40.202519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T03:14:42.480359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"84'3","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:33:05.471722+0000","last_change":"2024-08-19T01:32:38.913677+0000","last_active":"2024-08-19T01:33:05.471722+0000","last_peered":"2024-08-19T01:33:05.471722+0000","last_clean":"2024-08-19T01:33:05.471722+0000","last_became_active":"2024-08-19T01:32:38.913091+0000","last_became_peered":"2024-08-19T01:32:38.913091+0000","last_unstale":"2024-08-19T01:33:05.471722+0000","last_undegraded":"2024-08-19T01:33:05.471722+0000","last_fullsized":"2024-08-19T01:33:05.471722+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:32:37.889224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:32:37.889224+0000","last_clean_scrub_stamp":"2024-08-19T01:32:37.889224+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T12:39:16.489246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:33:05.469625+0000","last_change":"2024-08-19T01:32:41.239380+0000","last_active":"2024-08-19T01:33:05.469625+0000","last_peered":"2024-08-19T01:33:05.469625+0000","last_clean":"2024-08-19T01:33:05.469625+0000","last_became_active":"2024-08-19T01:32:41.239060+0000","last_became_peered":"2024-08-19T01:32:41.239060+0000","last_unstale":"2024-08-19T01:33:05.469625+0000","last_undegraded":"2024-08-19T01:33:05.469625+0000","last_fullsized":"2024-08-19T01:33:05.469625+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_clean_scrub_stamp":"2024-08-19T01:32:40.202519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T02:21:59.630705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"84'1","reported_seq":15,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:33:05.479101+0000","last_change":"2024-08-19T01:33:05.478578+0000","last_active":"2024-08-19T01:33:05.479101+0000","last_peered":"2024-08-19T01:33:05.479101+0000","last_clean":"2024-08-19T01:33:05.479101+0000","last_became_active":"2024-08-19T01:33:05.473433+0000","last_became_peered":"2024-08-19T01:33:05.473433+0000","last_unstale":"2024-08-19T01:33:05.479101+0000","last_undegraded":"2024-08-19T01:33:05.479101+0000","last_fullsized":"2024-08-19T01:33:05.479101+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:32:37.889224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:32:37.889224+0000","last_clean_scrub_stamp":"2024-08-19T01:32:37.889224+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T05:06:29.975613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,3,8],"acting":[11,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:33:05.472017+0000","last_change":"2024-08-19T01:32:41.236012+0000","last_active":"2024-08-19T01:33:05.472017+0000","last_peered":"2024-08-19T01:33:05.472017+0000","last_clean":"2024-08-19T01:33:05.472017+0000","last_became_active":"2024-08-19T01:32:41.235724+0000","last_became_peered":"2024-08-19T01:32:41.235724+0000","last_unstale":"2024-08-19T01:33:05.472017+0000","last_undegraded":"2024-08-19T01:33:05.472017+0000","last_fullsized":"2024-08-19T01:33:05.472017+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_clean_scrub_stamp":"2024-08-19T01:32:40.202519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T08:45:08.865144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"84'4","reported_seq":27,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:33:05.471339+0000","last_change":"2024-08-19T01:32:38.915170+0000","last_active":"2024-08-19T01:33:05.471339+0000","last_peered":"2024-08-19T01:33:05.471339+0000","last_clean":"2024-08-19T01:33:05.471339+0000","last_became_active":"2024-08-19T01:32:38.914812+0000","last_became_peered":"2024-08-19T01:32:38.914812+0000","last_unstale":"2024-08-19T01:33:05.471339+0000","last_undegraded":"2024-08-19T01:33:05.471339+0000","last_fullsized":"2024-08-19T01:33:05.471339+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:32:37.889224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:32:37.889224+0000","last_clean_scrub_stamp":"2024-08-19T01:32:37.889224+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T11:46:39.409065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:33:05.472797+0000","last_change":"2024-08-19T01:32:41.234133+0000","last_active":"2024-08-19T01:33:05.472797+0000","last_peered":"2024-08-19T01:33:05.472797+0000","last_clean":"2024-08-19T01:33:05.472797+0000","last_became_active":"2024-08-19T01:32:41.233492+0000","last_became_peered":"2024-08-19T01:32:41.233492+0000","last_unstale":"2024-08-19T01:33:05.472797+0000","last_undegraded":"2024-08-19T01:33:05.472797+0000","last_fullsized":"2024-08-19T01:33:05.472797+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_clean_scrub_stamp":"2024-08-19T01:32:40.202519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T06:11:12.587678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:33:05.469674+0000","last_change":"2024-08-19T01:32:41.232628+0000","last_active":"2024-08-19T01:33:05.469674+0000","last_peered":"2024-08-19T01:33:05.469674+0000","last_clean":"2024-08-19T01:33:05.469674+0000","last_became_active":"2024-08-19T01:32:41.232295+0000","last_became_peered":"2024-08-19T01:32:41.232295+0000","last_unstale":"2024-08-19T01:33:05.469674+0000","last_undegraded":"2024-08-19T01:33:05.469674+0000","last_fullsized":"2024-08-19T01:33:05.469674+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_clean_scrub_stamp":"2024-08-19T01:32:40.202519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T02:01:09.490553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"84'2","reported_seq":24,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:33:05.472666+0000","last_change":"2024-08-19T01:32:38.912000+0000","last_active":"2024-08-19T01:33:05.472666+0000","last_peered":"2024-08-19T01:33:05.472666+0000","last_clean":"2024-08-19T01:33:05.472666+0000","last_became_active":"2024-08-19T01:32:38.911535+0000","last_became_peered":"2024-08-19T01:32:38.911535+0000","last_unstale":"2024-08-19T01:33:05.472666+0000","last_undegraded":"2024-08-19T01:33:05.472666+0000","last_fullsized":"2024-08-19T01:33:05.472666+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:32:37.889224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:32:37.889224+0000","last_clean_scrub_stamp":"2024-08-19T01:32:37.889224+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T07:12:49.358384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:33:05.701576+0000","last_change":"2024-08-19T01:32:41.233507+0000","last_active":"2024-08-19T01:33:05.701576+0000","last_peered":"2024-08-19T01:33:05.701576+0000","last_clean":"2024-08-19T01:33:05.701576+0000","last_became_active":"2024-08-19T01:32:41.233179+0000","last_became_peered":"2024-08-19T01:32:41.233179+0000","last_unstale":"2024-08-19T01:33:05.701576+0000","last_undegraded":"2024-08-19T01:33:05.701576+0000","last_fullsized":"2024-08-19T01:33:05.701576+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_clean_scrub_stamp":"2024-08-19T01:32:40.202519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T11:47:44.780859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1d","version":"84'4","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:33:05.483750+0000","last_change":"2024-08-19T01:33:05.483322+0000","last_active":"2024-08-19T01:33:05.483750+0000","last_peered":"2024-08-19T01:33:05.483750+0000","last_clean":"2024-08-19T01:33:05.483750+0000","last_became_active":"2024-08-19T01:33:05.475496+0000","last_became_peered":"2024-08-19T01:33:05.475496+0000","last_unstale":"2024-08-19T01:33:05.483750+0000","last_undegraded":"2024-08-19T01:33:05.483750+0000","last_fullsized":"2024-08-19T01:33:05.483750+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:32:37.889224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:32:37.889224+0000","last_clean_scrub_stamp":"2024-08-19T01:32:37.889224+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T01:57:48.143495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":10,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,7],"acting":[11,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:33:05.469647+0000","last_change":"2024-08-19T01:32:41.244641+0000","last_active":"2024-08-19T01:33:05.469647+0000","last_peered":"2024-08-19T01:33:05.469647+0000","last_clean":"2024-08-19T01:33:05.469647+0000","last_became_active":"2024-08-19T01:32:41.244311+0000","last_became_peered":"2024-08-19T01:32:41.244311+0000","last_unstale":"2024-08-19T01:33:05.469647+0000","last_undegraded":"2024-08-19T01:33:05.469647+0000","last_fullsized":"2024-08-19T01:33:05.469647+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_clean_scrub_stamp":"2024-08-19T01:32:40.202519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T10:39:12.591092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"84'2","reported_seq":24,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:33:05.472352+0000","last_change":"2024-08-19T01:32:39.051314+0000","last_active":"2024-08-19T01:33:05.472352+0000","last_peered":"2024-08-19T01:33:05.472352+0000","last_clean":"2024-08-19T01:33:05.472352+0000","last_became_active":"2024-08-19T01:32:39.050998+0000","last_became_peered":"2024-08-19T01:32:39.050998+0000","last_unstale":"2024-08-19T01:33:05.472352+0000","last_undegraded":"2024-08-19T01:33:05.472352+0000","last_fullsized":"2024-08-19T01:33:05.472352+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:32:37.889224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:32:37.889224+0000","last_clean_scrub_stamp":"2024-08-19T01:32:37.889224+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T09:21:15.689149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:33:05.472112+0000","last_change":"2024-08-19T01:32:41.234762+0000","last_active":"2024-08-19T01:33:05.472112+0000","last_peered":"2024-08-19T01:33:05.472112+0000","last_clean":"2024-08-19T01:33:05.472112+0000","last_became_active":"2024-08-19T01:32:41.234471+0000","last_became_peered":"2024-08-19T01:32:41.234471+0000","last_unstale":"2024-08-19T01:33:05.472112+0000","last_undegraded":"2024-08-19T01:33:05.472112+0000","last_fullsized":"2024-08-19T01:33:05.472112+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_clean_scrub_stamp":"2024-08-19T01:32:40.202519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T08:34:33.110938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"84'9","reported_seq":32,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:33:05.466470+0000","last_change":"2024-08-19T01:32:38.917068+0000","last_active":"2024-08-19T01:33:05.466470+0000","last_peered":"2024-08-19T01:33:05.466470+0000","last_clean":"2024-08-19T01:33:05.466470+0000","last_became_active":"2024-08-19T01:32:38.915399+0000","last_became_peered":"2024-08-19T01:32:38.915399+0000","last_unstale":"2024-08-19T01:33:05.466470+0000","last_undegraded":"2024-08-19T01:33:05.466470+0000","last_fullsized":"2024-08-19T01:33:05.466470+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:32:37.889224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:32:37.889224+0000","last_clean_scrub_stamp":"2024-08-19T01:32:37.889224+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T04:17:25.061208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":65006,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":369,"num_read_kb":255,"num_write":164,"num_write_kb":110,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":1012,"num_keys_recovered":10,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":417792,"data_stored":195018,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":139,"ondisk_log_size":139,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4751360,"data_stored":4722944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8}],"osd_stats":[{"osd":11,"up_from":78,"seq":335007449115,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32100,"kb_used_data":916,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713052,"statfs":{"total":95995035648,"available":95962165248,"internally_reserved":0,"allocated":937984,"data_stored":451051,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":71,"seq":304942678048,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28612,"kb_used_data":1596,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716540,"statfs":{"total":95995035648,"available":95965736960,"internally_reserved":0,"allocated":1634304,"data_stored":1103538,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":66,"seq":283467841573,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32108,"kb_used_data":932,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713044,"statfs":{"total":95995035648,"available":95962157056,"internally_reserved":0,"allocated":954368,"data_stored":451287,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":58,"seq":249108103210,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32108,"kb_used_data":932,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713044,"statfs":{"total":95995035648,"available":95962157056,"internally_reserved":0,"allocated":954368,"data_stored":451207,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":51,"seq":219043332142,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32696,"kb_used_data":1512,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712456,"statfs":{"total":95995035648,"available":95961554944,"internally_reserved":0,"allocated":1548288,"data_stored":1043526,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":46,"seq":197568495668,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27940,"kb_used_data":920,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717212,"statfs":{"total":95995035648,"available":95966425088,"internally_reserved":0,"allocated":942080,"data_stored":450676,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":38,"seq":163208757304,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28548,"kb_used_data":1524,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716604,"statfs":{"total":95995035648,"available":95965802496,"internally_reserved":0,"allocated":1560576,"data_stored":1042820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":33,"seq":141733920828,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32112,"kb_used_data":932,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713040,"statfs":{"total":95995035648,"available":95962152960,"internally_reserved":0,"allocated":954368,"data_stored":452515,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182465,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27956,"kb_used_data":936,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717196,"statfs":{"total":95995035648,"available":95966408704,"internally_reserved":0,"allocated":958464,"data_stored":450987,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411399,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36332,"kb_used_data":996,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708820,"statfs":{"total":95995035648,"available":95957831680,"internally_reserved":0,"allocated":1019904,"data_stored":511813,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574921,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27996,"kb_used_data":980,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717156,"statfs":{"total":95995035648,"available":95966367744,"internally_reserved":0,"allocated":1003520,"data_stored":511085,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738447,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36284,"kb_used_data":940,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708868,"statfs":{"total":95995035648,"available":95957880832,"internally_reserved":0,"allocated":962560,"data_stored":451947,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":69632,"data_stored":60616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":86016,"data_stored":61344,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2046,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1866,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2572,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":102400,"data_stored":62584,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-08-19T01:33:54.939 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2024-08-19T01:33:54.940 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-08-19T01:33:54.940 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy 2024-08-19T01:33:54.940 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833 shell --fsid bad8780e-5dc9-11ef-bccf-c7b262605968 -- ceph health --format=json 2024-08-19T01:33:55.350 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/bad8780e-5dc9-11ef-bccf-c7b262605968/mon.a/config 2024-08-19T01:33:55.687 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:33:55 smithi083 ceph-mon[17120]: from='client.14781 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-08-19T01:33:55.687 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:33:55 smithi083 ceph-mon[17120]: pgmap v311: 97 pgs: 97 active+clean; 640 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-08-19T01:33:55.702 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:33:55 smithi117 ceph-mon[22016]: from='client.14781 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-08-19T01:33:55.702 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:33:55 smithi117 ceph-mon[22016]: pgmap v311: 97 pgs: 97 active+clean; 640 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-08-19T01:33:55.716 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:33:55 smithi167 ceph-mon[21875]: from='client.14781 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-08-19T01:33:55.716 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:33:55 smithi167 ceph-mon[21875]: pgmap v311: 97 pgs: 97 active+clean; 640 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-08-19T01:33:55.920 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:55.918+0000 7fb8a5400640 1 -- 172.21.15.83:0/1421241345 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb8a0074be0 msgr2=0x7fb8a0112380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:33:55.920 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:55.918+0000 7fb8a5400640 1 --2- 172.21.15.83:0/1421241345 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb8a0074be0 0x7fb8a0112380 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fb888009900 tx=0x7fb88802a480 comp rx=0 tx=0).stop 2024-08-19T01:33:55.920 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:55.918+0000 7fb8a5400640 1 -- 172.21.15.83:0/1421241345 shutdown_connections 2024-08-19T01:33:55.920 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:55.918+0000 7fb8a5400640 1 --2- 172.21.15.83:0/1421241345 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb8a01128c0 0x7fb8a0114cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:33:55.920 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:55.918+0000 7fb8a5400640 1 --2- 172.21.15.83:0/1421241345 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb8a0074be0 0x7fb8a0112380 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:33:55.920 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:55.918+0000 7fb8a5400640 1 --2- 172.21.15.83:0/1421241345 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb8a00742c0 0x7fb8a00746a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:33:55.920 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:55.918+0000 7fb8a5400640 1 -- 172.21.15.83:0/1421241345 >> 172.21.15.83:0/1421241345 conn(0x7fb8a00fd5f0 msgr2=0x7fb8a00ffa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:33:55.921 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:55.918+0000 7fb8a5400640 1 -- 172.21.15.83:0/1421241345 shutdown_connections 2024-08-19T01:33:55.921 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:55.918+0000 7fb8a5400640 1 -- 172.21.15.83:0/1421241345 wait complete. 2024-08-19T01:33:55.921 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:55.919+0000 7fb8a5400640 1 Processor -- start 2024-08-19T01:33:55.921 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:55.920+0000 7fb8a5400640 1 -- start start 2024-08-19T01:33:55.922 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:55.920+0000 7fb8a5400640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb8a00742c0 0x7fb8a01a45f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:33:55.923 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:55.920+0000 7fb8a5400640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb8a0074be0 0x7fb8a01a4b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:33:55.923 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:55.920+0000 7fb8a5400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb8a01128c0 0x7fb8a019e750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:33:55.923 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:55.920+0000 7fb8a5400640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fb8a01182e0 con 0x7fb8a01128c0 2024-08-19T01:33:55.923 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:55.920+0000 7fb8a5400640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fb8a0118160 con 0x7fb8a0074be0 2024-08-19T01:33:55.923 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:55.920+0000 7fb8a5400640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fb8a0118460 con 0x7fb8a00742c0 2024-08-19T01:33:55.923 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:55.920+0000 7fb89e800640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb8a00742c0 0x7fb8a01a45f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:33:55.923 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:55.920+0000 7fb89e800640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb8a00742c0 0x7fb8a01a45f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.83:49548/0 (socket says 172.21.15.83:49548) 2024-08-19T01:33:55.923 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:55.920+0000 7fb89e800640 1 -- 172.21.15.83:0/3971930264 learned_addr learned my addr 172.21.15.83:0/3971930264 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-08-19T01:33:55.923 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:55.920+0000 7fb89f200640 1 --2- 172.21.15.83:0/3971930264 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb8a01128c0 0x7fb8a019e750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:33:55.923 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:55.920+0000 7fb897e00640 1 --2- 172.21.15.83:0/3971930264 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb8a0074be0 0x7fb8a01a4b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:33:55.923 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:55.921+0000 7fb89e800640 1 -- 172.21.15.83:0/3971930264 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb8a0074be0 msgr2=0x7fb8a01a4b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:33:55.923 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:55.921+0000 7fb89e800640 1 --2- 172.21.15.83:0/3971930264 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb8a0074be0 0x7fb8a01a4b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:33:55.924 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:55.921+0000 7fb89e800640 1 -- 172.21.15.83:0/3971930264 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb8a01128c0 msgr2=0x7fb8a019e750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:33:55.924 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:55.921+0000 7fb89e800640 1 --2- 172.21.15.83:0/3971930264 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb8a01128c0 0x7fb8a019e750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:33:55.924 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:55.921+0000 7fb89e800640 1 -- 172.21.15.83:0/3971930264 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb8a019ef80 con 0x7fb8a00742c0 2024-08-19T01:33:55.924 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:55.921+0000 7fb89f200640 1 --2- 172.21.15.83:0/3971930264 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb8a01128c0 0x7fb8a019e750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-19T01:33:55.924 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:55.921+0000 7fb89e800640 1 --2- 172.21.15.83:0/3971930264 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb8a00742c0 0x7fb8a01a45f0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fb89000e9f0 tx=0x7fb89000eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:33:55.924 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:55.921+0000 7fb897400640 1 -- 172.21.15.83:0/3971930264 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb89000cd80 con 0x7fb8a00742c0 2024-08-19T01:33:55.924 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:55.921+0000 7fb897400640 1 -- 172.21.15.83:0/3971930264 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb890010910 con 0x7fb8a00742c0 2024-08-19T01:33:55.924 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:55.921+0000 7fb8a5400640 1 -- 172.21.15.83:0/3971930264 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb8a019f270 con 0x7fb8a00742c0 2024-08-19T01:33:55.924 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:55.921+0000 7fb897400640 1 -- 172.21.15.83:0/3971930264 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb890018990 con 0x7fb8a00742c0 2024-08-19T01:33:55.925 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:55.921+0000 7fb8a5400640 1 -- 172.21.15.83:0/3971930264 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb8a01ab5e0 con 0x7fb8a00742c0 2024-08-19T01:33:55.925 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:55.922+0000 7fb8a5400640 1 -- 172.21.15.83:0/3971930264 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb8a0075a00 con 0x7fb8a00742c0 2024-08-19T01:33:55.926 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:55.924+0000 7fb897400640 1 -- 172.21.15.83:0/3971930264 <== mon.1 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7fb890010ac0 con 0x7fb8a00742c0 2024-08-19T01:33:55.927 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:55.926+0000 7fb897400640 1 --2- 172.21.15.83:0/3971930264 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7fb8840a2c10 0x7fb8840a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:33:55.927 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:55.926+0000 7fb897400640 1 -- 172.21.15.83:0/3971930264 <== mon.1 v2:172.21.15.167:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8269+0+0 (secure 0 0 0) 0x7fb8900cfc30 con 0x7fb8a00742c0 2024-08-19T01:33:55.927 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:55.926+0000 7fb897e00640 1 --2- 172.21.15.83:0/3971930264 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7fb8840a2c10 0x7fb8840a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:33:55.928 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:55.926+0000 7fb897e00640 1 --2- 172.21.15.83:0/3971930264 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7fb8840a2c10 0x7fb8840a50d0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fb888000c00 tx=0x7fb888005a20 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:33:55.929 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:55.928+0000 7fb897400640 1 -- 172.21.15.83:0/3971930264 <== mon.1 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7fb89009c9b0 con 0x7fb8a00742c0 2024-08-19T01:33:56.144 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:56.142+0000 7fb8a5400640 1 -- 172.21.15.83:0/3971930264 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7fb8a0104d00 con 0x7fb8a00742c0 2024-08-19T01:33:56.145 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:56.143+0000 7fb897400640 1 -- 172.21.15.83:0/3971930264 <== mon.1 v2:172.21.15.167:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+46 (secure 0 0 0) 0x7fb890096e20 con 0x7fb8a00742c0 2024-08-19T01:33:56.145 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-08-19T01:33:56.145 INFO:teuthology.orchestra.run.smithi083.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-08-19T01:33:56.148 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:56.146+0000 7fb8a5400640 1 -- 172.21.15.83:0/3971930264 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7fb8840a2c10 msgr2=0x7fb8840a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:33:56.149 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:56.146+0000 7fb8a5400640 1 --2- 172.21.15.83:0/3971930264 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7fb8840a2c10 0x7fb8840a50d0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fb888000c00 tx=0x7fb888005a20 comp rx=0 tx=0).stop 2024-08-19T01:33:56.149 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:56.146+0000 7fb8a5400640 1 -- 172.21.15.83:0/3971930264 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb8a00742c0 msgr2=0x7fb8a01a45f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:33:56.149 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:56.146+0000 7fb8a5400640 1 --2- 172.21.15.83:0/3971930264 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb8a00742c0 0x7fb8a01a45f0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fb89000e9f0 tx=0x7fb89000eeb0 comp rx=0 tx=0).stop 2024-08-19T01:33:56.149 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:56.147+0000 7fb8a5400640 1 -- 172.21.15.83:0/3971930264 shutdown_connections 2024-08-19T01:33:56.149 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:56.147+0000 7fb8a5400640 1 --2- 172.21.15.83:0/3971930264 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7fb8840a2c10 0x7fb8840a50d0 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:33:56.149 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:56.147+0000 7fb8a5400640 1 --2- 172.21.15.83:0/3971930264 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb8a01128c0 0x7fb8a019e750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:33:56.149 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:56.147+0000 7fb8a5400640 1 --2- 172.21.15.83:0/3971930264 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb8a0074be0 0x7fb8a01a4b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:33:56.149 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:56.147+0000 7fb8a5400640 1 --2- 172.21.15.83:0/3971930264 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb8a00742c0 0x7fb8a01a45f0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:33:56.149 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:56.147+0000 7fb8a5400640 1 -- 172.21.15.83:0/3971930264 >> 172.21.15.83:0/3971930264 conn(0x7fb8a00fd5f0 msgr2=0x7fb8a00fee20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:33:56.149 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:56.147+0000 7fb8a5400640 1 -- 172.21.15.83:0/3971930264 shutdown_connections 2024-08-19T01:33:56.149 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:56.147+0000 7fb8a5400640 1 -- 172.21.15.83:0/3971930264 wait complete. 2024-08-19T01:33:56.702 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:33:56 smithi117 ceph-mon[22016]: from='client.? 172.21.15.83:0/3971930264' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-08-19T01:33:56.716 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:33:56 smithi167 ceph-mon[21875]: from='client.? 172.21.15.83:0/3971930264' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-08-19T01:33:56.870 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:33:56 smithi083 ceph-mon[17120]: from='client.? 172.21.15.83:0/3971930264' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-08-19T01:33:56.907 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy done 2024-08-19T01:33:56.907 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833 shell --fsid bad8780e-5dc9-11ef-bccf-c7b262605968 -- ceph fs dump --format=json 2024-08-19T01:33:57.279 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/bad8780e-5dc9-11ef-bccf-c7b262605968/mon.a/config 2024-08-19T01:33:57.702 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:33:57 smithi117 ceph-mon[22016]: pgmap v312: 97 pgs: 97 active+clean; 640 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-08-19T01:33:57.716 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:33:57 smithi167 ceph-mon[21875]: pgmap v312: 97 pgs: 97 active+clean; 640 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-08-19T01:33:57.770 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:33:57 smithi083 ceph-mon[17120]: pgmap v312: 97 pgs: 97 active+clean; 640 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-08-19T01:33:57.879 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:57.877+0000 7f0b2be00640 1 -- 172.21.15.83:0/2299552150 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0b2c076450 msgr2=0x7f0b2c076830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:33:57.880 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:57.877+0000 7f0b2be00640 1 --2- 172.21.15.83:0/2299552150 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0b2c076450 0x7f0b2c076830 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f0b1c009900 tx=0x7f0b1c02a3f0 comp rx=0 tx=0).stop 2024-08-19T01:33:57.880 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:57.878+0000 7f0b2be00640 1 -- 172.21.15.83:0/2299552150 shutdown_connections 2024-08-19T01:33:57.880 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:57.878+0000 7f0b2be00640 1 --2- 172.21.15.83:0/2299552150 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0b2c1125a0 0x7f0b2c114990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:33:57.880 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:57.878+0000 7f0b2be00640 1 --2- 172.21.15.83:0/2299552150 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0b2c076d70 0x7f0b2c112060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:33:57.880 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:57.878+0000 7f0b2be00640 1 --2- 172.21.15.83:0/2299552150 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0b2c076450 0x7f0b2c076830 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:33:57.880 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:57.878+0000 7f0b2be00640 1 -- 172.21.15.83:0/2299552150 >> 172.21.15.83:0/2299552150 conn(0x7f0b2c0ff600 msgr2=0x7f0b2c101a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:33:57.880 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:57.878+0000 7f0b2be00640 1 -- 172.21.15.83:0/2299552150 shutdown_connections 2024-08-19T01:33:57.880 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:57.878+0000 7f0b2be00640 1 -- 172.21.15.83:0/2299552150 wait complete. 2024-08-19T01:33:57.882 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:57.879+0000 7f0b2be00640 1 Processor -- start 2024-08-19T01:33:57.882 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:57.879+0000 7f0b2be00640 1 -- start start 2024-08-19T01:33:57.882 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:57.879+0000 7f0b2be00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0b2c076450 0x7f0b2c19e5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:33:57.882 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:57.879+0000 7f0b2be00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0b2c076d70 0x7f0b2c19eb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:33:57.882 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:57.879+0000 7f0b2be00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0b2c1125a0 0x7f0b2c1a2eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:33:57.882 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:57.879+0000 7f0b2be00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f0b2c118180 con 0x7f0b2c076450 2024-08-19T01:33:57.883 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:57.879+0000 7f0b2be00640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f0b2c118000 con 0x7f0b2c076d70 2024-08-19T01:33:57.883 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:57.879+0000 7f0b2be00640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f0b2c118300 con 0x7f0b2c1125a0 2024-08-19T01:33:57.883 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:57.879+0000 7f0b2aa00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0b2c076450 0x7f0b2c19e5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:33:57.883 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:57.879+0000 7f0b2aa00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0b2c076450 0x7f0b2c19e5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:57696/0 (socket says 172.21.15.83:57696) 2024-08-19T01:33:57.883 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:57.879+0000 7f0b2aa00640 1 -- 172.21.15.83:0/4114136562 learned_addr learned my addr 172.21.15.83:0/4114136562 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-08-19T01:33:57.883 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:57.880+0000 7f0b22000640 1 --2- 172.21.15.83:0/4114136562 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0b2c076d70 0x7f0b2c19eb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:33:57.883 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:57.880+0000 7f0b2b400640 1 --2- 172.21.15.83:0/4114136562 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0b2c1125a0 0x7f0b2c1a2eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:33:57.883 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:57.880+0000 7f0b2aa00640 1 -- 172.21.15.83:0/4114136562 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0b2c1125a0 msgr2=0x7f0b2c1a2eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:33:57.883 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:57.880+0000 7f0b2aa00640 1 --2- 172.21.15.83:0/4114136562 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0b2c1125a0 0x7f0b2c1a2eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:33:57.883 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:57.880+0000 7f0b2aa00640 1 -- 172.21.15.83:0/4114136562 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0b2c076d70 msgr2=0x7f0b2c19eb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:33:57.883 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:57.880+0000 7f0b2aa00640 1 --2- 172.21.15.83:0/4114136562 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0b2c076d70 0x7f0b2c19eb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:33:57.883 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:57.880+0000 7f0b2aa00640 1 -- 172.21.15.83:0/4114136562 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0b2c1a3590 con 0x7f0b2c076450 2024-08-19T01:33:57.883 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:57.880+0000 7f0b2aa00640 1 --2- 172.21.15.83:0/4114136562 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0b2c076450 0x7f0b2c19e5e0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f0b1c004770 tx=0x7f0b1c003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:33:57.883 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:57.881+0000 7f0b23e00640 1 -- 172.21.15.83:0/4114136562 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b1c03f070 con 0x7f0b2c076450 2024-08-19T01:33:57.884 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:57.881+0000 7f0b23e00640 1 -- 172.21.15.83:0/4114136562 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0b1c031ca0 con 0x7f0b2c076450 2024-08-19T01:33:57.884 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:57.881+0000 7f0b2be00640 1 -- 172.21.15.83:0/4114136562 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0b2c1a3820 con 0x7f0b2c076450 2024-08-19T01:33:57.884 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:57.881+0000 7f0b23e00640 1 -- 172.21.15.83:0/4114136562 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b1c035030 con 0x7f0b2c076450 2024-08-19T01:33:57.884 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:57.881+0000 7f0b2be00640 1 -- 172.21.15.83:0/4114136562 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0b2c1ab160 con 0x7f0b2c076450 2024-08-19T01:33:57.884 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:57.882+0000 7f0b2be00640 1 -- 172.21.15.83:0/4114136562 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0b2c076830 con 0x7f0b2c076450 2024-08-19T01:33:57.885 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:57.883+0000 7f0b23e00640 1 -- 172.21.15.83:0/4114136562 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f0b1c042070 con 0x7f0b2c076450 2024-08-19T01:33:57.886 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:57.885+0000 7f0b23e00640 1 --2- 172.21.15.83:0/4114136562 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f0b100a2c90 0x7f0b100a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:33:57.886 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:57.885+0000 7f0b23e00640 1 -- 172.21.15.83:0/4114136562 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8269+0+0 (secure 0 0 0) 0x7f0b1c0ef760 con 0x7f0b2c076450 2024-08-19T01:33:57.886 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:57.885+0000 7f0b22000640 1 --2- 172.21.15.83:0/4114136562 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f0b100a2c90 0x7f0b100a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:33:57.887 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:57.886+0000 7f0b22000640 1 --2- 172.21.15.83:0/4114136562 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f0b100a2c90 0x7f0b100a5150 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f0b2c079460 tx=0x7f0b14005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:33:57.890 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:57.889+0000 7f0b23e00640 1 -- 172.21.15.83:0/4114136562 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7f0b1c0bbf50 con 0x7f0b2c076450 2024-08-19T01:33:58.096 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:58.094+0000 7f0b2be00640 1 -- 172.21.15.83:0/4114136562 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f0b2c1ab330 con 0x7f0b2c076450 2024-08-19T01:33:58.097 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:58.095+0000 7f0b23e00640 1 -- 172.21.15.83:0/4114136562 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) ==== 94+0+11920 (secure 0 0 0) 0x7f0b1c0c0250 con 0x7f0b2c076450 2024-08-19T01:33:58.098 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-08-19T01:33:58.098 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":24,"btime":"2024-08-19T01:32:54:291077+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14421,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6835/988556067","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":988556067},{"type":"v1","addr":"172.21.15.83:6835","nonce":988556067}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14433,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6837/869702632","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6836","nonce":869702632},{"type":"v1","addr":"172.21.15.83:6837","nonce":869702632}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14475,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6841/1795975999","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6840","nonce":1795975999},{"type":"v1","addr":"172.21.15.83:6841","nonce":1795975999}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24430,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6833/2505076046","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6832","nonce":2505076046},{"type":"v1","addr":"172.21.15.167:6833","nonce":2505076046}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24449,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6833/3557520478","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":3557520478},{"type":"v1","addr":"172.21.15.117:6833","nonce":3557520478}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24466,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6835/3482045966","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":3482045966},{"type":"v1","addr":"172.21.15.167:6835","nonce":3482045966}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7}],"filesystems":[{"mdsmap":{"epoch":24,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-08-19T01:32:42.054769+0000","modified":"2024-08-19T01:32:54.209056+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24545,"mds_1":24527,"mds_2":24509},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24478":{"gid":24478,"name":"c","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.167:6837/1762130408","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6836","nonce":1762130408},{"type":"v1","addr":"172.21.15.167:6837","nonce":1762130408}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24485":{"gid":24485,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.117:6835/1024213994","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":1024213994},{"type":"v1","addr":"172.21.15.117:6835","nonce":1024213994}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24496":{"gid":24496,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.167:6839/3507387682","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6838","nonce":3507387682},{"type":"v1","addr":"172.21.15.167:6839","nonce":3507387682}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24509":{"gid":24509,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":10,"addr":"172.21.15.117:6837/3185224684","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":3185224684},{"type":"v1","addr":"172.21.15.117:6837","nonce":3185224684}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24527":{"gid":24527,"name":"g","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.83:6839/3239261845","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6838","nonce":3239261845},{"type":"v1","addr":"172.21.15.83:6839","nonce":3239261845}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24545":{"gid":24545,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.117:6839/1967595704","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6838","nonce":1967595704},{"type":"v1","addr":"172.21.15.117:6839","nonce":1967595704}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24545,"qdb_cluster":[24509,24527,24545]},"id":1}]} 2024-08-19T01:33:58.099 INFO:teuthology.orchestra.run.smithi083.stderr:dumped fsmap epoch 24 2024-08-19T01:33:58.100 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:58.098+0000 7f0b2be00640 1 -- 172.21.15.83:0/4114136562 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f0b100a2c90 msgr2=0x7f0b100a5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:33:58.101 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:58.098+0000 7f0b2be00640 1 --2- 172.21.15.83:0/4114136562 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f0b100a2c90 0x7f0b100a5150 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f0b2c079460 tx=0x7f0b14005f90 comp rx=0 tx=0).stop 2024-08-19T01:33:58.101 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:58.098+0000 7f0b2be00640 1 -- 172.21.15.83:0/4114136562 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0b2c076450 msgr2=0x7f0b2c19e5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:33:58.101 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:58.098+0000 7f0b2be00640 1 --2- 172.21.15.83:0/4114136562 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0b2c076450 0x7f0b2c19e5e0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f0b1c004770 tx=0x7f0b1c003c10 comp rx=0 tx=0).stop 2024-08-19T01:33:58.101 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:58.098+0000 7f0b2be00640 1 -- 172.21.15.83:0/4114136562 shutdown_connections 2024-08-19T01:33:58.101 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:58.098+0000 7f0b2be00640 1 --2- 172.21.15.83:0/4114136562 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f0b100a2c90 0x7f0b100a5150 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:33:58.101 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:58.098+0000 7f0b2be00640 1 --2- 172.21.15.83:0/4114136562 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0b2c1125a0 0x7f0b2c1a2eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:33:58.102 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:58.098+0000 7f0b2be00640 1 --2- 172.21.15.83:0/4114136562 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0b2c076d70 0x7f0b2c19eb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:33:58.102 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:58.098+0000 7f0b2be00640 1 --2- 172.21.15.83:0/4114136562 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0b2c076450 0x7f0b2c19e5e0 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:33:58.102 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:58.098+0000 7f0b2be00640 1 -- 172.21.15.83:0/4114136562 >> 172.21.15.83:0/4114136562 conn(0x7f0b2c0ff600 msgr2=0x7f0b2c1019f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:33:58.102 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:58.099+0000 7f0b2be00640 1 -- 172.21.15.83:0/4114136562 shutdown_connections 2024-08-19T01:33:58.102 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:58.099+0000 7f0b2be00640 1 -- 172.21.15.83:0/4114136562 wait complete. 2024-08-19T01:33:58.702 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:33:58 smithi117 ceph-mon[22016]: from='client.? 172.21.15.83:0/4114136562' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-19T01:33:58.716 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:33:58 smithi167 ceph-mon[21875]: from='client.? 172.21.15.83:0/4114136562' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-19T01:33:58.783 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:33:58 smithi083 ceph-mon[17120]: from='client.? 172.21.15.83:0/4114136562' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-19T01:33:58.825 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833 shell --fsid bad8780e-5dc9-11ef-bccf-c7b262605968 -- ceph fs dump --format=json 2024-08-19T01:33:59.200 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/bad8780e-5dc9-11ef-bccf-c7b262605968/mon.a/config 2024-08-19T01:33:59.628 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:33:59 smithi083 ceph-mon[17120]: pgmap v313: 97 pgs: 97 active+clean; 640 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-08-19T01:33:59.702 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:33:59 smithi117 ceph-mon[22016]: pgmap v313: 97 pgs: 97 active+clean; 640 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-08-19T01:33:59.716 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:33:59 smithi167 ceph-mon[21875]: pgmap v313: 97 pgs: 97 active+clean; 640 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-08-19T01:33:59.761 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:59.736+0000 7f6b88c00640 1 -- 172.21.15.83:0/1080533398 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6b841018b0 msgr2=0x7f6b84101d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:33:59.762 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:59.736+0000 7f6b88c00640 1 --2- 172.21.15.83:0/1080533398 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6b841018b0 0x7f6b84101d10 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f6b70009930 tx=0x7f6b7002a420 comp rx=0 tx=0).stop 2024-08-19T01:33:59.762 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:59.736+0000 7f6b88c00640 1 -- 172.21.15.83:0/1080533398 shutdown_connections 2024-08-19T01:33:59.762 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:59.736+0000 7f6b88c00640 1 --2- 172.21.15.83:0/1080533398 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6b84102250 0x7f6b8410e780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:33:59.762 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:59.736+0000 7f6b88c00640 1 --2- 172.21.15.83:0/1080533398 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6b841018b0 0x7f6b84101d10 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:33:59.762 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:59.736+0000 7f6b88c00640 1 --2- 172.21.15.83:0/1080533398 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6b841078b0 0x7f6b84107c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:33:59.762 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:59.736+0000 7f6b88c00640 1 -- 172.21.15.83:0/1080533398 >> 172.21.15.83:0/1080533398 conn(0x7f6b840fd5f0 msgr2=0x7f6b840ffa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:33:59.762 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:59.736+0000 7f6b88c00640 1 -- 172.21.15.83:0/1080533398 shutdown_connections 2024-08-19T01:33:59.762 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:59.736+0000 7f6b88c00640 1 -- 172.21.15.83:0/1080533398 wait complete. 2024-08-19T01:33:59.762 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:59.736+0000 7f6b88c00640 1 Processor -- start 2024-08-19T01:33:59.762 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:59.738+0000 7f6b88c00640 1 -- start start 2024-08-19T01:33:59.763 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:59.738+0000 7f6b88c00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6b841018b0 0x7f6b8419a3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:33:59.763 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:59.738+0000 7f6b88c00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6b84102250 0x7f6b8419a920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:33:59.763 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:59.738+0000 7f6b88c00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6b841078b0 0x7f6b8419ecb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:33:59.763 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:59.738+0000 7f6b88c00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f6b84111cb0 con 0x7f6b841018b0 2024-08-19T01:33:59.763 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:59.738+0000 7f6b88c00640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f6b84111b30 con 0x7f6b84102250 2024-08-19T01:33:59.763 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:59.738+0000 7f6b88c00640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f6b84111e30 con 0x7f6b841078b0 2024-08-19T01:33:59.763 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:59.738+0000 7f6b81e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6b841018b0 0x7f6b8419a3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:33:59.763 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:59.738+0000 7f6b81e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6b841018b0 0x7f6b8419a3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:57716/0 (socket says 172.21.15.83:57716) 2024-08-19T01:33:59.763 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:59.738+0000 7f6b81e00640 1 -- 172.21.15.83:0/3875460963 learned_addr learned my addr 172.21.15.83:0/3875460963 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-08-19T01:33:59.763 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:59.738+0000 7f6b82800640 1 --2- 172.21.15.83:0/3875460963 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6b841078b0 0x7f6b8419ecb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:33:59.763 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:59.738+0000 7f6b81400640 1 --2- 172.21.15.83:0/3875460963 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6b84102250 0x7f6b8419a920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:33:59.763 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:59.739+0000 7f6b81e00640 1 -- 172.21.15.83:0/3875460963 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6b841078b0 msgr2=0x7f6b8419ecb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:33:59.763 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:59.739+0000 7f6b81e00640 1 --2- 172.21.15.83:0/3875460963 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6b841078b0 0x7f6b8419ecb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:33:59.763 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:59.739+0000 7f6b81e00640 1 -- 172.21.15.83:0/3875460963 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6b84102250 msgr2=0x7f6b8419a920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:33:59.763 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:59.739+0000 7f6b81e00640 1 --2- 172.21.15.83:0/3875460963 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6b84102250 0x7f6b8419a920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:33:59.764 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:59.739+0000 7f6b81e00640 1 -- 172.21.15.83:0/3875460963 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6b8419f390 con 0x7f6b841018b0 2024-08-19T01:33:59.764 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:59.739+0000 7f6b81e00640 1 --2- 172.21.15.83:0/3875460963 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6b841018b0 0x7f6b8419a3e0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f6b7400d950 tx=0x7f6b7400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:33:59.764 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:59.739+0000 7f6b7aa00640 1 -- 172.21.15.83:0/3875460963 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b74017030 con 0x7f6b841018b0 2024-08-19T01:33:59.764 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:59.739+0000 7f6b7aa00640 1 -- 172.21.15.83:0/3875460963 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6b74004020 con 0x7f6b841018b0 2024-08-19T01:33:59.764 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:59.739+0000 7f6b7aa00640 1 -- 172.21.15.83:0/3875460963 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b740047a0 con 0x7f6b841018b0 2024-08-19T01:33:59.764 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:59.739+0000 7f6b88c00640 1 -- 172.21.15.83:0/3875460963 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6b8419f680 con 0x7f6b841018b0 2024-08-19T01:33:59.764 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:59.739+0000 7f6b88c00640 1 -- 172.21.15.83:0/3875460963 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6b841a6f60 con 0x7f6b841018b0 2024-08-19T01:33:59.764 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:59.741+0000 7f6b88c00640 1 -- 172.21.15.83:0/3875460963 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6b84101d10 con 0x7f6b841018b0 2024-08-19T01:33:59.764 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:59.742+0000 7f6b7aa00640 1 -- 172.21.15.83:0/3875460963 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f6b740041d0 con 0x7f6b841018b0 2024-08-19T01:33:59.764 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:59.744+0000 7f6b7aa00640 1 --2- 172.21.15.83:0/3875460963 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f6b680a2c90 0x7f6b680a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:33:59.764 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:59.744+0000 7f6b7aa00640 1 -- 172.21.15.83:0/3875460963 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8269+0+0 (secure 0 0 0) 0x7f6b740ceab0 con 0x7f6b841018b0 2024-08-19T01:33:59.764 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:59.744+0000 7f6b81400640 1 --2- 172.21.15.83:0/3875460963 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f6b680a2c90 0x7f6b680a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:33:59.764 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:59.744+0000 7f6b81400640 1 --2- 172.21.15.83:0/3875460963 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f6b680a2c90 0x7f6b680a5150 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f6b70009900 tx=0x7f6b700058d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:33:59.764 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:59.748+0000 7f6b7aa00640 1 -- 172.21.15.83:0/3875460963 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7f6b7409b830 con 0x7f6b841018b0 2024-08-19T01:33:59.960 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:59.959+0000 7f6b88c00640 1 -- 172.21.15.83:0/3875460963 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f6b841a7200 con 0x7f6b841018b0 2024-08-19T01:33:59.960 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:59.960+0000 7f6b7aa00640 1 -- 172.21.15.83:0/3875460963 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) ==== 94+0+11920 (secure 0 0 0) 0x7f6b7409b130 con 0x7f6b841018b0 2024-08-19T01:33:59.961 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-08-19T01:33:59.961 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":24,"btime":"2024-08-19T01:32:54:291077+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14421,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6835/988556067","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":988556067},{"type":"v1","addr":"172.21.15.83:6835","nonce":988556067}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14433,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6837/869702632","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6836","nonce":869702632},{"type":"v1","addr":"172.21.15.83:6837","nonce":869702632}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14475,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6841/1795975999","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6840","nonce":1795975999},{"type":"v1","addr":"172.21.15.83:6841","nonce":1795975999}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24430,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6833/2505076046","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6832","nonce":2505076046},{"type":"v1","addr":"172.21.15.167:6833","nonce":2505076046}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24449,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6833/3557520478","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":3557520478},{"type":"v1","addr":"172.21.15.117:6833","nonce":3557520478}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24466,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6835/3482045966","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":3482045966},{"type":"v1","addr":"172.21.15.167:6835","nonce":3482045966}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7}],"filesystems":[{"mdsmap":{"epoch":24,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-08-19T01:32:42.054769+0000","modified":"2024-08-19T01:32:54.209056+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24545,"mds_1":24527,"mds_2":24509},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24478":{"gid":24478,"name":"c","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.167:6837/1762130408","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6836","nonce":1762130408},{"type":"v1","addr":"172.21.15.167:6837","nonce":1762130408}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24485":{"gid":24485,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.117:6835/1024213994","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":1024213994},{"type":"v1","addr":"172.21.15.117:6835","nonce":1024213994}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24496":{"gid":24496,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.167:6839/3507387682","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6838","nonce":3507387682},{"type":"v1","addr":"172.21.15.167:6839","nonce":3507387682}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24509":{"gid":24509,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":10,"addr":"172.21.15.117:6837/3185224684","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":3185224684},{"type":"v1","addr":"172.21.15.117:6837","nonce":3185224684}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24527":{"gid":24527,"name":"g","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.83:6839/3239261845","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6838","nonce":3239261845},{"type":"v1","addr":"172.21.15.83:6839","nonce":3239261845}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24545":{"gid":24545,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.117:6839/1967595704","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6838","nonce":1967595704},{"type":"v1","addr":"172.21.15.117:6839","nonce":1967595704}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24545,"qdb_cluster":[24509,24527,24545]},"id":1}]} 2024-08-19T01:33:59.961 INFO:teuthology.orchestra.run.smithi083.stderr:dumped fsmap epoch 24 2024-08-19T01:33:59.963 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:59.962+0000 7f6b88c00640 1 -- 172.21.15.83:0/3875460963 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f6b680a2c90 msgr2=0x7f6b680a5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:33:59.963 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:59.962+0000 7f6b88c00640 1 --2- 172.21.15.83:0/3875460963 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f6b680a2c90 0x7f6b680a5150 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f6b70009900 tx=0x7f6b700058d0 comp rx=0 tx=0).stop 2024-08-19T01:33:59.963 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:59.962+0000 7f6b88c00640 1 -- 172.21.15.83:0/3875460963 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6b841018b0 msgr2=0x7f6b8419a3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:33:59.963 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:59.962+0000 7f6b88c00640 1 --2- 172.21.15.83:0/3875460963 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6b841018b0 0x7f6b8419a3e0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f6b7400d950 tx=0x7f6b7400de10 comp rx=0 tx=0).stop 2024-08-19T01:33:59.963 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:59.962+0000 7f6b88c00640 1 -- 172.21.15.83:0/3875460963 shutdown_connections 2024-08-19T01:33:59.964 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:59.962+0000 7f6b88c00640 1 --2- 172.21.15.83:0/3875460963 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f6b680a2c90 0x7f6b680a5150 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:33:59.964 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:59.962+0000 7f6b88c00640 1 --2- 172.21.15.83:0/3875460963 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6b841078b0 0x7f6b8419ecb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:33:59.964 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:59.962+0000 7f6b88c00640 1 --2- 172.21.15.83:0/3875460963 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6b84102250 0x7f6b8419a920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:33:59.964 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:59.962+0000 7f6b88c00640 1 --2- 172.21.15.83:0/3875460963 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6b841018b0 0x7f6b8419a3e0 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:33:59.964 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:59.962+0000 7f6b88c00640 1 -- 172.21.15.83:0/3875460963 >> 172.21.15.83:0/3875460963 conn(0x7f6b840fd5f0 msgr2=0x7f6b840fdc90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:33:59.964 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:59.963+0000 7f6b88c00640 1 -- 172.21.15.83:0/3875460963 shutdown_connections 2024-08-19T01:33:59.964 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:33:59.963+0000 7f6b88c00640 1 -- 172.21.15.83:0/3875460963 wait complete. 2024-08-19T01:34:00.654 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:34:00 smithi083 ceph-mon[17120]: from='client.? 172.21.15.83:0/3875460963' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-19T01:34:00.702 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:34:00 smithi117 ceph-mon[22016]: from='client.? 172.21.15.83:0/3875460963' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-19T01:34:00.716 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:34:00 smithi167 ceph-mon[21875]: from='client.? 172.21.15.83:0/3875460963' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-19T01:34:00.748 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833 shell --fsid bad8780e-5dc9-11ef-bccf-c7b262605968 -- ceph osd dump --format=json 2024-08-19T01:34:01.131 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/bad8780e-5dc9-11ef-bccf-c7b262605968/mon.a/config 2024-08-19T01:34:01.713 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:01.712+0000 7f73fd800640 1 -- 172.21.15.83:0/4271073631 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f73f8103ae0 msgr2=0x7f73f8103f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:34:01.816 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:01.712+0000 7f73fd800640 1 --2- 172.21.15.83:0/4271073631 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f73f8103ae0 0x7f73f8103f40 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f73e4009930 tx=0x7f73e402a420 comp rx=0 tx=0).stop 2024-08-19T01:34:01.816 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:01.712+0000 7f73fd800640 1 -- 172.21.15.83:0/4271073631 shutdown_connections 2024-08-19T01:34:01.816 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:01.712+0000 7f73fd800640 1 --2- 172.21.15.83:0/4271073631 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f73f8104480 0x7f73f81109b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:01.817 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:01.712+0000 7f73fd800640 1 --2- 172.21.15.83:0/4271073631 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f73f8103ae0 0x7f73f8103f40 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:01.817 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:01.712+0000 7f73fd800640 1 --2- 172.21.15.83:0/4271073631 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f73f8109b70 0x7f73f8109f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:01.817 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:01.712+0000 7f73fd800640 1 -- 172.21.15.83:0/4271073631 >> 172.21.15.83:0/4271073631 conn(0x7f73f80ff780 msgr2=0x7f73f8101ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:34:01.817 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:01.712+0000 7f73fd800640 1 -- 172.21.15.83:0/4271073631 shutdown_connections 2024-08-19T01:34:01.817 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:01.712+0000 7f73fd800640 1 -- 172.21.15.83:0/4271073631 wait complete. 2024-08-19T01:34:01.817 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:34:01 smithi083 ceph-mon[17120]: pgmap v314: 97 pgs: 97 active+clean; 640 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-08-19T01:34:01.819 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:01.713+0000 7f73fd800640 1 Processor -- start 2024-08-19T01:34:01.819 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:01.713+0000 7f73fd800640 1 -- start start 2024-08-19T01:34:01.819 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:01.714+0000 7f73fd800640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f73f8103ae0 0x7f73f819a3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:34:01.819 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:01.714+0000 7f73fd800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f73f8104480 0x7f73f819a910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:34:01.819 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:01.714+0000 7f73fd800640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f73f8109b70 0x7f73f819eca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:34:01.819 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:01.714+0000 7f73fd800640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f73f8113f30 con 0x7f73f8104480 2024-08-19T01:34:01.820 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:01.714+0000 7f73fd800640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f73f8113db0 con 0x7f73f8109b70 2024-08-19T01:34:01.820 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:01.714+0000 7f73fd800640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f73f81140b0 con 0x7f73f8103ae0 2024-08-19T01:34:01.820 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:01.714+0000 7f73f6000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f73f8104480 0x7f73f819a910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:34:01.820 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:01.714+0000 7f73f6a00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f73f8103ae0 0x7f73f819a3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:34:01.820 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:01.714+0000 7f73f6a00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f73f8103ae0 0x7f73f819a3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.83:56436/0 (socket says 172.21.15.83:56436) 2024-08-19T01:34:01.820 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:01.714+0000 7f73f6a00640 1 -- 172.21.15.83:0/6875364 learned_addr learned my addr 172.21.15.83:0/6875364 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-08-19T01:34:01.820 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:01.714+0000 7f73f7400640 1 --2- 172.21.15.83:0/6875364 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f73f8109b70 0x7f73f819eca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:34:01.820 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:01.715+0000 7f73f6a00640 1 -- 172.21.15.83:0/6875364 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f73f8109b70 msgr2=0x7f73f819eca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:34:01.820 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:01.715+0000 7f73f6a00640 1 --2- 172.21.15.83:0/6875364 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f73f8109b70 0x7f73f819eca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:01.820 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:01.715+0000 7f73f6a00640 1 -- 172.21.15.83:0/6875364 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f73f8104480 msgr2=0x7f73f819a910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:34:01.820 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:01.715+0000 7f73f6a00640 1 --2- 172.21.15.83:0/6875364 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f73f8104480 0x7f73f819a910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:01.820 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:01.715+0000 7f73f6a00640 1 -- 172.21.15.83:0/6875364 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f73f819f380 con 0x7f73f8103ae0 2024-08-19T01:34:01.820 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:01.715+0000 7f73f6a00640 1 --2- 172.21.15.83:0/6875364 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f73f8103ae0 0x7f73f819a3d0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f73e800eb90 tx=0x7f73e800c6a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:34:01.820 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:01.715+0000 7f73ef400640 1 -- 172.21.15.83:0/6875364 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73e8004d20 con 0x7f73f8103ae0 2024-08-19T01:34:01.821 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:01.715+0000 7f73fd800640 1 -- 172.21.15.83:0/6875364 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f73f819f670 con 0x7f73f8103ae0 2024-08-19T01:34:01.821 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:01.715+0000 7f73ef400640 1 -- 172.21.15.83:0/6875364 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f73e800ee90 con 0x7f73f8103ae0 2024-08-19T01:34:01.821 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:01.715+0000 7f73ef400640 1 -- 172.21.15.83:0/6875364 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73e8010780 con 0x7f73f8103ae0 2024-08-19T01:34:01.821 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:01.715+0000 7f73fd800640 1 -- 172.21.15.83:0/6875364 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f73f81a6f50 con 0x7f73f8103ae0 2024-08-19T01:34:01.821 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:01.716+0000 7f73ecc00640 1 -- 172.21.15.83:0/6875364 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f73f8103f40 con 0x7f73f8103ae0 2024-08-19T01:34:01.821 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:01.718+0000 7f73ef400640 1 -- 172.21.15.83:0/6875364 <== mon.1 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f73e8044b60 con 0x7f73f8103ae0 2024-08-19T01:34:01.821 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:01.719+0000 7f73ef400640 1 --2- 172.21.15.83:0/6875364 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f73dc0a2c10 0x7f73dc0a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:34:01.821 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:01.720+0000 7f73ef400640 1 -- 172.21.15.83:0/6875364 <== mon.1 v2:172.21.15.167:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8269+0+0 (secure 0 0 0) 0x7f73e80cf7e0 con 0x7f73f8103ae0 2024-08-19T01:34:01.821 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:01.720+0000 7f73f6000640 1 --2- 172.21.15.83:0/6875364 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f73dc0a2c10 0x7f73dc0a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:34:01.821 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:01.720+0000 7f73f6000640 1 --2- 172.21.15.83:0/6875364 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f73dc0a2c10 0x7f73dc0a50d0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f73e4006010 tx=0x7f73e4000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:34:01.821 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:01.722+0000 7f73ef400640 1 -- 172.21.15.83:0/6875364 <== mon.1 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7f73e809c560 con 0x7f73f8103ae0 2024-08-19T01:34:01.822 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:34:01 smithi167 ceph-mon[21875]: pgmap v314: 97 pgs: 97 active+clean; 640 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-08-19T01:34:01.898 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:01.897+0000 7f73ecc00640 1 -- 172.21.15.83:0/6875364 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f73f819b730 con 0x7f73f8103ae0 2024-08-19T01:34:01.899 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:01.899+0000 7f73ef400640 1 -- 172.21.15.83:0/6875364 <== mon.1 v2:172.21.15.167:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v86) ==== 74+0+22722 (secure 0 0 0) 0x7f73e809be60 con 0x7f73f8103ae0 2024-08-19T01:34:01.900 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-08-19T01:34:01.900 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":86,"fsid":"bad8780e-5dc9-11ef-bccf-c7b262605968","created":"2024-08-19T01:25:12.626841+0000","modified":"2024-08-19T01:33:05.461998+0000","last_up_change":"2024-08-19T01:31:47.254559+0000","last_in_change":"2024-08-19T01:31:27.652847+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-08-19T01:28:16.862007+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-08-19T01:32:37.455535+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-08-19T01:32:40.201935+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ffa74041-f9d2-4555-bb43-508b8c581487","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6802","nonce":1082526824},{"type":"v1","addr":"172.21.15.83:6803","nonce":1082526824}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6804","nonce":1082526824},{"type":"v1","addr":"172.21.15.83:6805","nonce":1082526824}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6808","nonce":1082526824},{"type":"v1","addr":"172.21.15.83:6809","nonce":1082526824}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6806","nonce":1082526824},{"type":"v1","addr":"172.21.15.83:6807","nonce":1082526824}]},"public_addr":"172.21.15.83:6803/1082526824","cluster_addr":"172.21.15.83:6805/1082526824","heartbeat_back_addr":"172.21.15.83:6809/1082526824","heartbeat_front_addr":"172.21.15.83:6807/1082526824","state":["exists","up"]},{"osd":1,"uuid":"1977570b-b4f5-44fd-bb1e-3a31318be649","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6800","nonce":1925419802},{"type":"v1","addr":"172.21.15.117:6801","nonce":1925419802}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6802","nonce":1925419802},{"type":"v1","addr":"172.21.15.117:6803","nonce":1925419802}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6806","nonce":1925419802},{"type":"v1","addr":"172.21.15.117:6807","nonce":1925419802}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6804","nonce":1925419802},{"type":"v1","addr":"172.21.15.117:6805","nonce":1925419802}]},"public_addr":"172.21.15.117:6801/1925419802","cluster_addr":"172.21.15.117:6803/1925419802","heartbeat_back_addr":"172.21.15.117:6807/1925419802","heartbeat_front_addr":"172.21.15.117:6805/1925419802","state":["exists","up"]},{"osd":2,"uuid":"a75278d1-161f-41d5-a92f-058d1077bf2a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6800","nonce":551625328},{"type":"v1","addr":"172.21.15.167:6801","nonce":551625328}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6802","nonce":551625328},{"type":"v1","addr":"172.21.15.167:6803","nonce":551625328}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6806","nonce":551625328},{"type":"v1","addr":"172.21.15.167:6807","nonce":551625328}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6804","nonce":551625328},{"type":"v1","addr":"172.21.15.167:6805","nonce":551625328}]},"public_addr":"172.21.15.167:6801/551625328","cluster_addr":"172.21.15.167:6803/551625328","heartbeat_back_addr":"172.21.15.167:6807/551625328","heartbeat_front_addr":"172.21.15.167:6805/551625328","state":["exists","up"]},{"osd":3,"uuid":"d1fb16f9-910e-46fa-b1e3-8e5bef4d346d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6810","nonce":3138942062},{"type":"v1","addr":"172.21.15.83:6811","nonce":3138942062}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6812","nonce":3138942062},{"type":"v1","addr":"172.21.15.83:6813","nonce":3138942062}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6816","nonce":3138942062},{"type":"v1","addr":"172.21.15.83:6817","nonce":3138942062}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6814","nonce":3138942062},{"type":"v1","addr":"172.21.15.83:6815","nonce":3138942062}]},"public_addr":"172.21.15.83:6811/3138942062","cluster_addr":"172.21.15.83:6813/3138942062","heartbeat_back_addr":"172.21.15.83:6817/3138942062","heartbeat_front_addr":"172.21.15.83:6815/3138942062","state":["exists","up"]},{"osd":4,"uuid":"8176ed18-1fe2-446d-8994-8c60f6e08f38","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6808","nonce":3978863827},{"type":"v1","addr":"172.21.15.117:6809","nonce":3978863827}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6810","nonce":3978863827},{"type":"v1","addr":"172.21.15.117:6811","nonce":3978863827}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6814","nonce":3978863827},{"type":"v1","addr":"172.21.15.117:6815","nonce":3978863827}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6812","nonce":3978863827},{"type":"v1","addr":"172.21.15.117:6813","nonce":3978863827}]},"public_addr":"172.21.15.117:6809/3978863827","cluster_addr":"172.21.15.117:6811/3978863827","heartbeat_back_addr":"172.21.15.117:6815/3978863827","heartbeat_front_addr":"172.21.15.117:6813/3978863827","state":["exists","up"]},{"osd":5,"uuid":"e01e3f15-25c8-4d2c-be6f-a170f036064c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6808","nonce":2761858035},{"type":"v1","addr":"172.21.15.167:6809","nonce":2761858035}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6810","nonce":2761858035},{"type":"v1","addr":"172.21.15.167:6811","nonce":2761858035}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6814","nonce":2761858035},{"type":"v1","addr":"172.21.15.167:6815","nonce":2761858035}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6812","nonce":2761858035},{"type":"v1","addr":"172.21.15.167:6813","nonce":2761858035}]},"public_addr":"172.21.15.167:6809/2761858035","cluster_addr":"172.21.15.167:6811/2761858035","heartbeat_back_addr":"172.21.15.167:6815/2761858035","heartbeat_front_addr":"172.21.15.167:6813/2761858035","state":["exists","up"]},{"osd":6,"uuid":"78cf0a9b-9165-444f-b073-ebe13016de9d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6818","nonce":260644552},{"type":"v1","addr":"172.21.15.83:6819","nonce":260644552}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6820","nonce":260644552},{"type":"v1","addr":"172.21.15.83:6821","nonce":260644552}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6824","nonce":260644552},{"type":"v1","addr":"172.21.15.83:6825","nonce":260644552}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6822","nonce":260644552},{"type":"v1","addr":"172.21.15.83:6823","nonce":260644552}]},"public_addr":"172.21.15.83:6819/260644552","cluster_addr":"172.21.15.83:6821/260644552","heartbeat_back_addr":"172.21.15.83:6825/260644552","heartbeat_front_addr":"172.21.15.83:6823/260644552","state":["exists","up"]},{"osd":7,"uuid":"ee524f7f-672e-430e-9f58-1a9117513466","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6816","nonce":104113252},{"type":"v1","addr":"172.21.15.117:6817","nonce":104113252}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6818","nonce":104113252},{"type":"v1","addr":"172.21.15.117:6819","nonce":104113252}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6822","nonce":104113252},{"type":"v1","addr":"172.21.15.117:6823","nonce":104113252}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6820","nonce":104113252},{"type":"v1","addr":"172.21.15.117:6821","nonce":104113252}]},"public_addr":"172.21.15.117:6817/104113252","cluster_addr":"172.21.15.117:6819/104113252","heartbeat_back_addr":"172.21.15.117:6823/104113252","heartbeat_front_addr":"172.21.15.117:6821/104113252","state":["exists","up"]},{"osd":8,"uuid":"76d5317d-4845-4138-b210-e7bd2d80472a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":58,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6816","nonce":2196898857},{"type":"v1","addr":"172.21.15.167:6817","nonce":2196898857}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6818","nonce":2196898857},{"type":"v1","addr":"172.21.15.167:6819","nonce":2196898857}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6822","nonce":2196898857},{"type":"v1","addr":"172.21.15.167:6823","nonce":2196898857}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6820","nonce":2196898857},{"type":"v1","addr":"172.21.15.167:6821","nonce":2196898857}]},"public_addr":"172.21.15.167:6817/2196898857","cluster_addr":"172.21.15.167:6819/2196898857","heartbeat_back_addr":"172.21.15.167:6823/2196898857","heartbeat_front_addr":"172.21.15.167:6821/2196898857","state":["exists","up"]},{"osd":9,"uuid":"0151bc2a-3ad2-4db3-8505-5855792cc9bb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6826","nonce":903463450},{"type":"v1","addr":"172.21.15.83:6827","nonce":903463450}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6828","nonce":903463450},{"type":"v1","addr":"172.21.15.83:6829","nonce":903463450}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6832","nonce":903463450},{"type":"v1","addr":"172.21.15.83:6833","nonce":903463450}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6830","nonce":903463450},{"type":"v1","addr":"172.21.15.83:6831","nonce":903463450}]},"public_addr":"172.21.15.83:6827/903463450","cluster_addr":"172.21.15.83:6829/903463450","heartbeat_back_addr":"172.21.15.83:6833/903463450","heartbeat_front_addr":"172.21.15.83:6831/903463450","state":["exists","up"]},{"osd":10,"uuid":"437ed848-0da4-4cd4-8716-0c55bb99b676","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6824","nonce":1183570837},{"type":"v1","addr":"172.21.15.117:6825","nonce":1183570837}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6826","nonce":1183570837},{"type":"v1","addr":"172.21.15.117:6827","nonce":1183570837}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6830","nonce":1183570837},{"type":"v1","addr":"172.21.15.117:6831","nonce":1183570837}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6828","nonce":1183570837},{"type":"v1","addr":"172.21.15.117:6829","nonce":1183570837}]},"public_addr":"172.21.15.117:6825/1183570837","cluster_addr":"172.21.15.117:6827/1183570837","heartbeat_back_addr":"172.21.15.117:6831/1183570837","heartbeat_front_addr":"172.21.15.117:6829/1183570837","state":["exists","up"]},{"osd":11,"uuid":"13345f05-cdc0-4dae-a58b-551abbec1915","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":78,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6824","nonce":187027350},{"type":"v1","addr":"172.21.15.167:6825","nonce":187027350}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6826","nonce":187027350},{"type":"v1","addr":"172.21.15.167:6827","nonce":187027350}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6830","nonce":187027350},{"type":"v1","addr":"172.21.15.167:6831","nonce":187027350}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6828","nonce":187027350},{"type":"v1","addr":"172.21.15.167:6829","nonce":187027350}]},"public_addr":"172.21.15.167:6825/187027350","cluster_addr":"172.21.15.167:6827/187027350","heartbeat_back_addr":"172.21.15.167:6831/187027350","heartbeat_front_addr":"172.21.15.167:6829/187027350","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-19T01:27:28.760517+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-19T01:27:50.493028+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-19T01:28:11.997674+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-19T01:28:38.109934+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-19T01:29:00.173065+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-19T01:29:21.336085+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-19T01:29:47.871498+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-19T01:30:10.720498+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-19T01:30:32.425438+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-19T01:30:59.524989+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-19T01:31:22.806162+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-19T01:31:45.043246+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.1a","mappings":[{"from":8,"to":1}]},{"pgid":"2.1b","mappings":[{"from":2,"to":4}]},{"pgid":"2.1d","mappings":[{"from":8,"to":11}]},{"pgid":"3.6","mappings":[{"from":4,"to":6}]},{"pgid":"3.11","mappings":[{"from":4,"to":8}]},{"pgid":"3.1b","mappings":[{"from":5,"to":9}]},{"pgid":"3.20","mappings":[{"from":5,"to":9}]},{"pgid":"3.31","mappings":[{"from":4,"to":7}]},{"pgid":"3.3b","mappings":[{"from":3,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.83:0/3138095466":"2024-08-20T01:26:04.277716+0000","172.21.15.83:0/3835005198":"2024-08-20T01:26:04.277716+0000","172.21.15.83:0/1790374088":"2024-08-20T01:25:37.228583+0000","172.21.15.83:6801/573907319":"2024-08-20T01:26:04.277716+0000","172.21.15.83:6800/573907319":"2024-08-20T01:26:04.277716+0000","172.21.15.83:0/664676002":"2024-08-20T01:25:37.228583+0000","172.21.15.83:0/2144611449":"2024-08-20T01:25:37.228583+0000","172.21.15.83:6801/3261152818":"2024-08-20T01:25:37.228583+0000","172.21.15.83:0/1609552666":"2024-08-20T01:26:04.277716+0000","172.21.15.83:6800/3261152818":"2024-08-20T01:25:37.228583+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-08-19T01:34:01.902 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:01.901+0000 7f73ecc00640 1 -- 172.21.15.83:0/6875364 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f73dc0a2c10 msgr2=0x7f73dc0a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:34:01.902 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:01.901+0000 7f73ecc00640 1 --2- 172.21.15.83:0/6875364 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f73dc0a2c10 0x7f73dc0a50d0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f73e4006010 tx=0x7f73e4000bc0 comp rx=0 tx=0).stop 2024-08-19T01:34:01.902 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:01.901+0000 7f73ecc00640 1 -- 172.21.15.83:0/6875364 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f73f8103ae0 msgr2=0x7f73f819a3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:34:01.902 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:01.901+0000 7f73ecc00640 1 --2- 172.21.15.83:0/6875364 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f73f8103ae0 0x7f73f819a3d0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f73e800eb90 tx=0x7f73e800c6a0 comp rx=0 tx=0).stop 2024-08-19T01:34:01.902 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:01.901+0000 7f73ecc00640 1 -- 172.21.15.83:0/6875364 shutdown_connections 2024-08-19T01:34:01.902 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:01.901+0000 7f73ecc00640 1 --2- 172.21.15.83:0/6875364 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f73dc0a2c10 0x7f73dc0a50d0 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:01.902 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:01.901+0000 7f73ecc00640 1 --2- 172.21.15.83:0/6875364 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f73f8109b70 0x7f73f819eca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:01.902 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:01.901+0000 7f73ecc00640 1 --2- 172.21.15.83:0/6875364 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f73f8104480 0x7f73f819a910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:01.903 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:01.901+0000 7f73ecc00640 1 --2- 172.21.15.83:0/6875364 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f73f8103ae0 0x7f73f819a3d0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:01.903 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:01.901+0000 7f73ecc00640 1 -- 172.21.15.83:0/6875364 >> 172.21.15.83:0/6875364 conn(0x7f73f80ff780 msgr2=0x7f73f80fff70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:34:01.903 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:01.901+0000 7f73ecc00640 1 -- 172.21.15.83:0/6875364 shutdown_connections 2024-08-19T01:34:01.903 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:01.901+0000 7f73ecc00640 1 -- 172.21.15.83:0/6875364 wait complete. 2024-08-19T01:34:01.952 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:34:01 smithi117 ceph-mon[22016]: pgmap v314: 97 pgs: 97 active+clean; 640 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-08-19T01:34:02.680 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833 shell --fsid bad8780e-5dc9-11ef-bccf-c7b262605968 -- ceph fs dump --format=json 2024-08-19T01:34:02.716 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:34:02 smithi167 ceph-mon[21875]: from='client.? 172.21.15.83:0/6875364' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-08-19T01:34:02.730 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:34:02 smithi083 ceph-mon[17120]: from='client.? 172.21.15.83:0/6875364' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-08-19T01:34:02.952 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:34:02 smithi117 ceph-mon[22016]: from='client.? 172.21.15.83:0/6875364' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-08-19T01:34:03.054 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/bad8780e-5dc9-11ef-bccf-c7b262605968/mon.a/config 2024-08-19T01:34:03.563 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:03.562+0000 7fb018a00640 1 -- 172.21.15.83:0/1698367859 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb014104460 msgr2=0x7fb014110990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:34:03.564 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:03.562+0000 7fb018a00640 1 --2- 172.21.15.83:0/1698367859 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb014104460 0x7fb014110990 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fb00c009cd0 tx=0x7fb00c02a8e0 comp rx=0 tx=0).stop 2024-08-19T01:34:03.564 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:03.563+0000 7fb018a00640 1 -- 172.21.15.83:0/1698367859 shutdown_connections 2024-08-19T01:34:03.564 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:03.563+0000 7fb018a00640 1 --2- 172.21.15.83:0/1698367859 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb014104460 0x7fb014110990 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:03.564 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:03.563+0000 7fb018a00640 1 --2- 172.21.15.83:0/1698367859 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb014103ac0 0x7fb014103f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:03.564 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:03.563+0000 7fb018a00640 1 --2- 172.21.15.83:0/1698367859 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb014109ac0 0x7fb014109ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:03.564 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:03.563+0000 7fb018a00640 1 -- 172.21.15.83:0/1698367859 >> 172.21.15.83:0/1698367859 conn(0x7fb0140ff7c0 msgr2=0x7fb014101be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:34:03.564 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:03.563+0000 7fb018a00640 1 -- 172.21.15.83:0/1698367859 shutdown_connections 2024-08-19T01:34:03.564 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:03.563+0000 7fb018a00640 1 -- 172.21.15.83:0/1698367859 wait complete. 2024-08-19T01:34:03.565 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:03.564+0000 7fb018a00640 1 Processor -- start 2024-08-19T01:34:03.566 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:03.564+0000 7fb018a00640 1 -- start start 2024-08-19T01:34:03.566 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:03.564+0000 7fb018a00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb014103ac0 0x7fb01419a350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:34:03.566 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:03.564+0000 7fb018a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb014104460 0x7fb01419a890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:34:03.566 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:03.564+0000 7fb018a00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb014109ac0 0x7fb01419ec20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:34:03.566 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:03.564+0000 7fb018a00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fb014113f10 con 0x7fb014104460 2024-08-19T01:34:03.566 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:03.564+0000 7fb018a00640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fb014113d90 con 0x7fb014109ac0 2024-08-19T01:34:03.566 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:03.564+0000 7fb018a00640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fb014114090 con 0x7fb014103ac0 2024-08-19T01:34:03.566 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:03.564+0000 7fb011400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb014104460 0x7fb01419a890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:34:03.566 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:03.564+0000 7fb011e00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb014103ac0 0x7fb01419a350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:34:03.566 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:03.564+0000 7fb011400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb014104460 0x7fb01419a890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:52888/0 (socket says 172.21.15.83:52888) 2024-08-19T01:34:03.566 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:03.564+0000 7fb011400640 1 -- 172.21.15.83:0/3804808095 learned_addr learned my addr 172.21.15.83:0/3804808095 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-08-19T01:34:03.566 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:03.564+0000 7fb012800640 1 --2- 172.21.15.83:0/3804808095 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb014109ac0 0x7fb01419ec20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:34:03.567 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:03.565+0000 7fb011e00640 1 -- 172.21.15.83:0/3804808095 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb014109ac0 msgr2=0x7fb01419ec20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:34:03.567 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:03.565+0000 7fb011e00640 1 --2- 172.21.15.83:0/3804808095 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb014109ac0 0x7fb01419ec20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:03.567 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:03.565+0000 7fb011e00640 1 -- 172.21.15.83:0/3804808095 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb014104460 msgr2=0x7fb01419a890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:34:03.567 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:03.565+0000 7fb011e00640 1 --2- 172.21.15.83:0/3804808095 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb014104460 0x7fb01419a890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:03.567 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:03.565+0000 7fb011e00640 1 -- 172.21.15.83:0/3804808095 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb01419f300 con 0x7fb014103ac0 2024-08-19T01:34:03.567 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:03.565+0000 7fb011400640 1 --2- 172.21.15.83:0/3804808095 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb014104460 0x7fb01419a890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-19T01:34:03.567 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:03.565+0000 7fb011e00640 1 --2- 172.21.15.83:0/3804808095 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb014103ac0 0x7fb01419a350 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fb00400e9f0 tx=0x7fb00400eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:34:03.568 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:03.565+0000 7fb00aa00640 1 -- 172.21.15.83:0/3804808095 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb00400ccc0 con 0x7fb014103ac0 2024-08-19T01:34:03.568 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:03.566+0000 7fb018a00640 1 -- 172.21.15.83:0/3804808095 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb01419f5f0 con 0x7fb014103ac0 2024-08-19T01:34:03.568 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:03.566+0000 7fb00aa00640 1 -- 172.21.15.83:0/3804808095 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb004010910 con 0x7fb014103ac0 2024-08-19T01:34:03.568 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:03.566+0000 7fb00aa00640 1 -- 172.21.15.83:0/3804808095 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb004018880 con 0x7fb014103ac0 2024-08-19T01:34:03.568 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:03.566+0000 7fb018a00640 1 -- 172.21.15.83:0/3804808095 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb0141a6ed0 con 0x7fb014103ac0 2024-08-19T01:34:03.568 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:03.567+0000 7fb018a00640 1 -- 172.21.15.83:0/3804808095 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb014103f20 con 0x7fb014103ac0 2024-08-19T01:34:03.569 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:03.569+0000 7fb00aa00640 1 -- 172.21.15.83:0/3804808095 <== mon.1 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7fb004010ac0 con 0x7fb014103ac0 2024-08-19T01:34:03.570 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:03.570+0000 7fb00aa00640 1 --2- 172.21.15.83:0/3804808095 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7faff80a2c10 0x7faff80a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:34:03.571 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:03.570+0000 7fb00aa00640 1 -- 172.21.15.83:0/3804808095 <== mon.1 v2:172.21.15.167:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8269+0+0 (secure 0 0 0) 0x7fb0040cf720 con 0x7fb014103ac0 2024-08-19T01:34:03.571 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:03.570+0000 7fb011400640 1 --2- 172.21.15.83:0/3804808095 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7faff80a2c10 0x7faff80a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:34:03.571 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:03.571+0000 7fb011400640 1 --2- 172.21.15.83:0/3804808095 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7faff80a2c10 0x7faff80a50d0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fb014101d30 tx=0x7fb000005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:34:03.572 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:03.572+0000 7fb00aa00640 1 -- 172.21.15.83:0/3804808095 <== mon.1 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7fb00409be50 con 0x7fb014103ac0 2024-08-19T01:34:03.716 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:34:03 smithi167 ceph-mon[21875]: pgmap v315: 97 pgs: 97 active+clean; 640 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-08-19T01:34:03.780 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:03.779+0000 7fb018a00640 1 -- 172.21.15.83:0/3804808095 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fb01419b6b0 con 0x7fb014103ac0 2024-08-19T01:34:03.780 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:34:03 smithi083 ceph-mon[17120]: pgmap v315: 97 pgs: 97 active+clean; 640 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-08-19T01:34:03.781 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:03.780+0000 7fb00aa00640 1 -- 172.21.15.83:0/3804808095 <== mon.1 v2:172.21.15.167:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) ==== 94+0+11920 (secure 0 0 0) 0x7fb0040a0150 con 0x7fb014103ac0 2024-08-19T01:34:03.781 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-08-19T01:34:03.781 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":24,"btime":"2024-08-19T01:32:54:291077+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14421,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6835/988556067","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":988556067},{"type":"v1","addr":"172.21.15.83:6835","nonce":988556067}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14433,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6837/869702632","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6836","nonce":869702632},{"type":"v1","addr":"172.21.15.83:6837","nonce":869702632}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14475,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6841/1795975999","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6840","nonce":1795975999},{"type":"v1","addr":"172.21.15.83:6841","nonce":1795975999}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24430,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6833/2505076046","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6832","nonce":2505076046},{"type":"v1","addr":"172.21.15.167:6833","nonce":2505076046}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24449,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6833/3557520478","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":3557520478},{"type":"v1","addr":"172.21.15.117:6833","nonce":3557520478}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24466,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6835/3482045966","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":3482045966},{"type":"v1","addr":"172.21.15.167:6835","nonce":3482045966}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7}],"filesystems":[{"mdsmap":{"epoch":24,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-08-19T01:32:42.054769+0000","modified":"2024-08-19T01:32:54.209056+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24545,"mds_1":24527,"mds_2":24509},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24478":{"gid":24478,"name":"c","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.167:6837/1762130408","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6836","nonce":1762130408},{"type":"v1","addr":"172.21.15.167:6837","nonce":1762130408}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24485":{"gid":24485,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.117:6835/1024213994","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":1024213994},{"type":"v1","addr":"172.21.15.117:6835","nonce":1024213994}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24496":{"gid":24496,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.167:6839/3507387682","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6838","nonce":3507387682},{"type":"v1","addr":"172.21.15.167:6839","nonce":3507387682}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24509":{"gid":24509,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":10,"addr":"172.21.15.117:6837/3185224684","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":3185224684},{"type":"v1","addr":"172.21.15.117:6837","nonce":3185224684}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24527":{"gid":24527,"name":"g","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.83:6839/3239261845","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6838","nonce":3239261845},{"type":"v1","addr":"172.21.15.83:6839","nonce":3239261845}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24545":{"gid":24545,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.117:6839/1967595704","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6838","nonce":1967595704},{"type":"v1","addr":"172.21.15.117:6839","nonce":1967595704}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24545,"qdb_cluster":[24509,24527,24545]},"id":1}]} 2024-08-19T01:34:03.781 INFO:teuthology.orchestra.run.smithi083.stderr:dumped fsmap epoch 24 2024-08-19T01:34:03.784 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:03.783+0000 7fb018a00640 1 -- 172.21.15.83:0/3804808095 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7faff80a2c10 msgr2=0x7faff80a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:34:03.784 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:03.783+0000 7fb018a00640 1 --2- 172.21.15.83:0/3804808095 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7faff80a2c10 0x7faff80a50d0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fb014101d30 tx=0x7fb000005f90 comp rx=0 tx=0).stop 2024-08-19T01:34:03.784 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:03.783+0000 7fb018a00640 1 -- 172.21.15.83:0/3804808095 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb014103ac0 msgr2=0x7fb01419a350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:34:03.784 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:03.783+0000 7fb018a00640 1 --2- 172.21.15.83:0/3804808095 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb014103ac0 0x7fb01419a350 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fb00400e9f0 tx=0x7fb00400eeb0 comp rx=0 tx=0).stop 2024-08-19T01:34:03.785 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:03.783+0000 7fb018a00640 1 -- 172.21.15.83:0/3804808095 shutdown_connections 2024-08-19T01:34:03.785 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:03.783+0000 7fb018a00640 1 --2- 172.21.15.83:0/3804808095 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7faff80a2c10 0x7faff80a50d0 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:03.785 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:03.783+0000 7fb018a00640 1 --2- 172.21.15.83:0/3804808095 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb014109ac0 0x7fb01419ec20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:03.785 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:03.783+0000 7fb018a00640 1 --2- 172.21.15.83:0/3804808095 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb014104460 0x7fb01419a890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:03.785 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:03.783+0000 7fb018a00640 1 --2- 172.21.15.83:0/3804808095 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb014103ac0 0x7fb01419a350 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:03.785 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:03.783+0000 7fb018a00640 1 -- 172.21.15.83:0/3804808095 >> 172.21.15.83:0/3804808095 conn(0x7fb0140ff7c0 msgr2=0x7fb0140fff10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:34:03.785 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:03.784+0000 7fb018a00640 1 -- 172.21.15.83:0/3804808095 shutdown_connections 2024-08-19T01:34:03.785 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:03.784+0000 7fb018a00640 1 -- 172.21.15.83:0/3804808095 wait complete. 2024-08-19T01:34:03.952 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:34:03 smithi117 ceph-mon[22016]: pgmap v315: 97 pgs: 97 active+clean; 640 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-08-19T01:34:04.546 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 24, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-08-19T01:32:42.054769+0000', 'modified': '2024-08-19T01:32:54.209056+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 3, 'in': [0, 1, 2], 'up': {'mds_0': 24545, 'mds_1': 24527, 'mds_2': 24509}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_24478': {'gid': 24478, 'name': 'c', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.167:6837/1762130408', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.167:6836', 'nonce': 1762130408}, {'type': 'v1', 'addr': '172.21.15.167:6837', 'nonce': 1762130408}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24485': {'gid': 24485, 'name': 'e', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.117:6835/1024213994', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.117:6834', 'nonce': 1024213994}, {'type': 'v1', 'addr': '172.21.15.117:6835', 'nonce': 1024213994}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24496': {'gid': 24496, 'name': 'i', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.167:6839/3507387682', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.167:6838', 'nonce': 3507387682}, {'type': 'v1', 'addr': '172.21.15.167:6839', 'nonce': 3507387682}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24509': {'gid': 24509, 'name': 'h', 'rank': 2, 'incarnation': 21, 'state': 'up:active', 'state_seq': 10, 'addr': '172.21.15.117:6837/3185224684', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.117:6836', 'nonce': 3185224684}, {'type': 'v1', 'addr': '172.21.15.117:6837', 'nonce': 3185224684}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24527': {'gid': 24527, 'name': 'g', 'rank': 1, 'incarnation': 19, 'state': 'up:active', 'state_seq': 8, 'addr': '172.21.15.83:6839/3239261845', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.83:6838', 'nonce': 3239261845}, {'type': 'v1', 'addr': '172.21.15.83:6839', 'nonce': 3239261845}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24545': {'gid': 24545, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 5, 'addr': '172.21.15.117:6839/1967595704', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.117:6838', 'nonce': 1967595704}, {'type': 'v1', 'addr': '172.21.15.117:6839', 'nonce': 1967595704}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 24545, 'qdb_cluster': [24509, 24527, 24545]} 2024-08-19T01:34:04.546 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 3/3 2024-08-19T01:34:04.547 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833 shell --fsid bad8780e-5dc9-11ef-bccf-c7b262605968 -- ceph tell mds.h status 2024-08-19T01:34:04.904 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:34:04 smithi083 ceph-mon[17120]: from='client.? 172.21.15.83:0/3804808095' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-19T01:34:04.921 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/bad8780e-5dc9-11ef-bccf-c7b262605968/mon.a/config 2024-08-19T01:34:04.952 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:34:04 smithi117 ceph-mon[22016]: from='client.? 172.21.15.83:0/3804808095' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-19T01:34:04.966 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:34:04 smithi167 ceph-mon[21875]: from='client.? 172.21.15.83:0/3804808095' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-19T01:34:05.483 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:05.481+0000 7f5b62c00640 1 -- 172.21.15.83:0/3290016552 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5b5c1018b0 msgr2=0x7f5b5c101d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:34:05.483 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:05.481+0000 7f5b62c00640 1 --2- 172.21.15.83:0/3290016552 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5b5c1018b0 0x7f5b5c101d10 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f5b48009930 tx=0x7f5b4802a420 comp rx=0 tx=0).stop 2024-08-19T01:34:05.483 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:05.481+0000 7f5b62c00640 1 -- 172.21.15.83:0/3290016552 shutdown_connections 2024-08-19T01:34:05.483 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:05.481+0000 7f5b62c00640 1 --2- 172.21.15.83:0/3290016552 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5b5c102250 0x7f5b5c10e780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:05.483 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:05.481+0000 7f5b62c00640 1 --2- 172.21.15.83:0/3290016552 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5b5c1018b0 0x7f5b5c101d10 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:05.483 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:05.481+0000 7f5b62c00640 1 --2- 172.21.15.83:0/3290016552 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5b5c1078b0 0x7f5b5c107c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:05.484 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:05.481+0000 7f5b62c00640 1 -- 172.21.15.83:0/3290016552 >> 172.21.15.83:0/3290016552 conn(0x7f5b5c0fd5f0 msgr2=0x7f5b5c0ffa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:34:05.484 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:05.481+0000 7f5b62c00640 1 -- 172.21.15.83:0/3290016552 shutdown_connections 2024-08-19T01:34:05.484 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:05.482+0000 7f5b62c00640 1 -- 172.21.15.83:0/3290016552 wait complete. 2024-08-19T01:34:05.484 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:05.482+0000 7f5b62c00640 1 Processor -- start 2024-08-19T01:34:05.484 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:05.483+0000 7f5b62c00640 1 -- start start 2024-08-19T01:34:05.485 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:05.483+0000 7f5b62c00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5b5c1018b0 0x7f5b5c19a3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:34:05.485 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:05.483+0000 7f5b62c00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5b5c102250 0x7f5b5c19a8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:34:05.486 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:05.483+0000 7f5b62c00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5b5c1078b0 0x7f5b5c19ec80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:34:05.486 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:05.483+0000 7f5b62c00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f5b5c111cc0 con 0x7f5b5c1078b0 2024-08-19T01:34:05.486 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:05.483+0000 7f5b62c00640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f5b5c111b40 con 0x7f5b5c1018b0 2024-08-19T01:34:05.486 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:05.483+0000 7f5b62c00640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f5b5c111e40 con 0x7f5b5c102250 2024-08-19T01:34:05.486 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:05.483+0000 7f5b60c00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5b5c1078b0 0x7f5b5c19ec80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:34:05.486 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:05.483+0000 7f5b60c00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5b5c1078b0 0x7f5b5c19ec80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:52920/0 (socket says 172.21.15.83:52920) 2024-08-19T01:34:05.486 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:05.483+0000 7f5b5b400640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5b5c102250 0x7f5b5c19a8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:34:05.486 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:05.483+0000 7f5b60c00640 1 -- 172.21.15.83:0/2769699094 learned_addr learned my addr 172.21.15.83:0/2769699094 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-08-19T01:34:05.486 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:05.483+0000 7f5b5be00640 1 --2- 172.21.15.83:0/2769699094 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5b5c1018b0 0x7f5b5c19a3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:34:05.486 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:05.484+0000 7f5b5b400640 1 -- 172.21.15.83:0/2769699094 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5b5c1018b0 msgr2=0x7f5b5c19a3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:34:05.486 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:05.484+0000 7f5b5b400640 1 --2- 172.21.15.83:0/2769699094 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5b5c1018b0 0x7f5b5c19a3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:05.486 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:05.484+0000 7f5b5b400640 1 -- 172.21.15.83:0/2769699094 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5b5c1078b0 msgr2=0x7f5b5c19ec80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:34:05.487 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:05.484+0000 7f5b5b400640 1 --2- 172.21.15.83:0/2769699094 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5b5c1078b0 0x7f5b5c19ec80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:05.487 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:05.484+0000 7f5b5b400640 1 -- 172.21.15.83:0/2769699094 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5b5c19f360 con 0x7f5b5c102250 2024-08-19T01:34:05.487 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:05.484+0000 7f5b5b400640 1 --2- 172.21.15.83:0/2769699094 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5b5c102250 0x7f5b5c19a8f0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f5b4802a940 tx=0x7f5b48003b20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:34:05.487 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:05.484+0000 7f5b58c00640 1 -- 172.21.15.83:0/2769699094 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b4803f070 con 0x7f5b5c102250 2024-08-19T01:34:05.487 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:05.484+0000 7f5b62c00640 1 -- 172.21.15.83:0/2769699094 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5b5c19f5f0 con 0x7f5b5c102250 2024-08-19T01:34:05.487 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:05.484+0000 7f5b58c00640 1 -- 172.21.15.83:0/2769699094 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5b48031c90 con 0x7f5b5c102250 2024-08-19T01:34:05.487 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:05.484+0000 7f5b58c00640 1 -- 172.21.15.83:0/2769699094 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b48035030 con 0x7f5b5c102250 2024-08-19T01:34:05.487 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:05.485+0000 7f5b62c00640 1 -- 172.21.15.83:0/2769699094 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5b5c1a6f30 con 0x7f5b5c102250 2024-08-19T01:34:05.488 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:05.488+0000 7f5b65868740 1 Processor -- start 2024-08-19T01:34:05.488 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:05.488+0000 7f5b58c00640 1 -- 172.21.15.83:0/2769699094 <== mon.1 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f5b48042070 con 0x7f5b5c102250 2024-08-19T01:34:05.488 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:05.488+0000 7f5b65868740 1 -- start start 2024-08-19T01:34:05.489 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:05.488+0000 7f5b65868740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559ca688f880 0x559ca688ddd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:34:05.489 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:05.488+0000 7f5b65868740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559ca688e310 0x559ca688e750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:34:05.489 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:05.488+0000 7f5b65868740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559ca68990b0 0x559ca689b4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:34:05.489 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:05.488+0000 7f5b65868740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x559ca67f4940 con 0x559ca688e310 2024-08-19T01:34:05.489 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:05.488+0000 7f5b65868740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x559ca678c9a0 con 0x559ca688f880 2024-08-19T01:34:05.489 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:05.488+0000 7f5b65868740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x559ca67f0900 con 0x559ca68990b0 2024-08-19T01:34:05.490 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:05.488+0000 7f5b5be00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559ca688e310 0x559ca688e750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:34:05.490 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:05.488+0000 7f5b5be00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559ca688e310 0x559ca688e750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:52930/0 (socket says 172.21.15.83:52930) 2024-08-19T01:34:05.490 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:05.488+0000 7f5b5be00640 1 -- 172.21.15.83:0/2433630890 learned_addr learned my addr 172.21.15.83:0/2433630890 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-08-19T01:34:05.490 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:05.488+0000 7f5b60c00640 1 --2- 172.21.15.83:0/2433630890 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559ca688f880 0x559ca688ddd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:34:05.490 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:05.489+0000 7f5b5b400640 1 --2- 172.21.15.83:0/2433630890 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559ca68990b0 0x559ca689b4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:34:05.490 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:05.489+0000 7f5b5be00640 1 -- 172.21.15.83:0/2433630890 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559ca68990b0 msgr2=0x559ca689b4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:34:05.491 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:05.489+0000 7f5b5be00640 1 --2- 172.21.15.83:0/2433630890 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559ca68990b0 0x559ca689b4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:05.492 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:05.489+0000 7f5b5be00640 1 -- 172.21.15.83:0/2433630890 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559ca688f880 msgr2=0x559ca688ddd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:34:05.492 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:05.489+0000 7f5b5be00640 1 --2- 172.21.15.83:0/2433630890 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559ca688f880 0x559ca688ddd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:05.492 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:05.489+0000 7f5b5be00640 1 -- 172.21.15.83:0/2433630890 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559ca677aef0 con 0x559ca688e310 2024-08-19T01:34:05.492 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:05.489+0000 7f5b60c00640 1 --2- 172.21.15.83:0/2433630890 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559ca688f880 0x559ca688ddd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-19T01:34:05.492 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:05.489+0000 7f5b5b400640 1 --2- 172.21.15.83:0/2433630890 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559ca68990b0 0x559ca689b4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-19T01:34:05.492 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:05.489+0000 7f5b58c00640 1 --2- 172.21.15.83:0/2769699094 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f5b400a2c00 0x7f5b400a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:34:05.492 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:05.489+0000 7f5b5be00640 1 --2- 172.21.15.83:0/2433630890 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559ca688e310 0x559ca688e750 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f5b4c009c70 tx=0x7f5b4c011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b37dcfad43a9eb7 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:34:05.492 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:05.489+0000 7f5b58c00640 1 -- 172.21.15.83:0/2769699094 <== mon.1 v2:172.21.15.167:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8269+0+0 (secure 0 0 0) 0x7f5b480ef830 con 0x7f5b5c102250 2024-08-19T01:34:05.492 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:05.489+0000 7f5b60c00640 1 --2- 172.21.15.83:0/2769699094 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f5b400a2c00 0x7f5b400a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:34:05.493 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:05.490+0000 7f5b51200640 1 -- 172.21.15.83:0/2433630890 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b4c004230 con 0x559ca688e310 2024-08-19T01:34:05.493 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:05.490+0000 7f5b51200640 1 -- 172.21.15.83:0/2433630890 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5b4c009e20 con 0x559ca688e310 2024-08-19T01:34:05.493 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:05.490+0000 7f5b51200640 1 -- 172.21.15.83:0/2433630890 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b4c019700 con 0x559ca688e310 2024-08-19T01:34:05.493 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:05.490+0000 7f5b65868740 1 -- 172.21.15.83:0/2433630890 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559ca688e310 msgr2=0x559ca688e750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:34:05.493 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:05.490+0000 7f5b65868740 1 --2- 172.21.15.83:0/2433630890 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559ca688e310 0x559ca688e750 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f5b4c009c70 tx=0x7f5b4c011810 comp rx=0 tx=0).stop 2024-08-19T01:34:05.493 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:05.490+0000 7f5b60c00640 1 --2- 172.21.15.83:0/2769699094 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f5b400a2c00 0x7f5b400a50c0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f5b540049d0 tx=0x7f5b54009500 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:34:05.493 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:05.490+0000 7f5b65868740 1 -- 172.21.15.83:0/2433630890 shutdown_connections 2024-08-19T01:34:05.493 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:05.490+0000 7f5b65868740 1 --2- 172.21.15.83:0/2433630890 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559ca68990b0 0x559ca689b4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:05.493 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:05.490+0000 7f5b65868740 1 --2- 172.21.15.83:0/2433630890 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559ca688e310 0x559ca688e750 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:05.493 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:05.490+0000 7f5b65868740 1 --2- 172.21.15.83:0/2433630890 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559ca688f880 0x559ca688ddd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:05.493 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:05.490+0000 7f5b65868740 1 -- 172.21.15.83:0/2433630890 >> 172.21.15.83:0/2433630890 conn(0x559ca6856a00 msgr2=0x559ca688a9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:34:05.493 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:05.491+0000 7f5b65868740 1 -- 172.21.15.83:0/2433630890 shutdown_connections 2024-08-19T01:34:05.493 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:05.491+0000 7f5b65868740 1 -- 172.21.15.83:0/2433630890 wait complete. 2024-08-19T01:34:05.494 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:05.491+0000 7f5b65868740 1 -- start start 2024-08-19T01:34:05.494 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:05.491+0000 7f5b65868740 1 Processor -- start 2024-08-19T01:34:05.496 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:05.492+0000 7f5b65868740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559ca688e310 0x559ca6895f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:34:05.497 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:05.492+0000 7f5b65868740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559ca688f880 0x559ca68964c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:34:05.497 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:05.492+0000 7f5b65868740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559ca68990b0 0x559ca6896a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:34:05.497 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:05.492+0000 7f5b65868740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x559ca689bdb0 con 0x559ca68990b0 2024-08-19T01:34:05.497 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:05.492+0000 7f5b65868740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x559ca67afa80 con 0x559ca688f880 2024-08-19T01:34:05.497 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:05.492+0000 7f5b65868740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x559ca689bf30 con 0x559ca688e310 2024-08-19T01:34:05.497 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:05.492+0000 7f5b65868740 10 client.0 _ll_register_callbacks cb 0x559ca6778140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-19T01:34:05.497 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:05.492+0000 7f5b60c00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559ca68990b0 0x559ca6896a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:34:05.497 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:05.492+0000 7f5b5b400640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559ca688f880 0x559ca68964c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:34:05.497 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:05.492+0000 7f5b5be00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559ca688e310 0x559ca6895f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:34:05.497 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:05.492+0000 7f5b5b400640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559ca688f880 0x559ca68964c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.83:53770/0 (socket says 172.21.15.83:53770) 2024-08-19T01:34:05.497 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:05.492+0000 7f5b5b400640 1 -- 172.21.15.83:0/2595456326 learned_addr learned my addr 172.21.15.83:0/2595456326 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-08-19T01:34:05.497 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:05.492+0000 7f5b60c00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559ca68990b0 0x559ca6896a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:52944/0 (socket says 172.21.15.83:52944) 2024-08-19T01:34:05.498 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:05.492+0000 7f5b5b400640 1 -- 172.21.15.83:0/2595456326 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559ca688e310 msgr2=0x559ca6895f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:34:05.498 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:05.492+0000 7f5b5b400640 1 --2- 172.21.15.83:0/2595456326 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559ca688e310 0x559ca6895f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:05.498 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:05.492+0000 7f5b5b400640 1 -- 172.21.15.83:0/2595456326 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559ca68990b0 msgr2=0x559ca6896a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:34:05.498 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:05.492+0000 7f5b5b400640 1 --2- 172.21.15.83:0/2595456326 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559ca68990b0 0x559ca6896a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:05.498 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:05.493+0000 7f5b5b400640 1 -- 172.21.15.83:0/2595456326 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559ca6793700 con 0x559ca688f880 2024-08-19T01:34:05.498 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:05.493+0000 7f5b5be00640 1 --2- 172.21.15.83:0/2595456326 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559ca688e310 0x559ca6895f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-19T01:34:05.498 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:05.493+0000 7f5b60c00640 1 --2- 172.21.15.83:0/2595456326 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559ca68990b0 0x559ca6896a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-19T01:34:05.498 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:05.493+0000 7f5b5b400640 1 --2- 172.21.15.83:0/2595456326 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559ca688f880 0x559ca68964c0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f5b480b8af0 tx=0x7f5b48031490 comp rx=0 tx=0).ready entity=mon.2 client_cookie=41dd9f9b72bdbdf6 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:34:05.498 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:05.493+0000 7f5b45600640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-08-19T01:34:05.498 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:05.493+0000 7f5b45600640 1 -- 172.21.15.83:0/2595456326 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b48049070 con 0x559ca688f880 2024-08-19T01:34:05.498 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:05.493+0000 7f5b45600640 1 -- 172.21.15.83:0/2595456326 <== mon.2 v2:172.21.15.117:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8269+0+0 (secure 0 0 0) 0x7f5b4803b460 con 0x559ca688f880 2024-08-19T01:34:05.498 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:05.493+0000 7f5b65868740 1 -- 172.21.15.83:0/2595456326 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5b4c009e20 con 0x559ca688f880 2024-08-19T01:34:05.498 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:05.493+0000 7f5b45600640 1 -- 172.21.15.83:0/2595456326 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f5b48031e40 con 0x559ca688f880 2024-08-19T01:34:05.499 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:05.493+0000 7f5b65868740 10 client.24890 fetch_fsmap learned FSMap version 24 2024-08-19T01:34:05.499 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:05.493+0000 7f5b65868740 1 -- 172.21.15.83:0/2595456326 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x559ca65662b0 con 0x559ca688f880 2024-08-19T01:34:05.499 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:05.493+0000 7f5b45600640 1 -- 172.21.15.83:0/2595456326 <== mon.2 v2:172.21.15.117:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5b4c009e20 con 0x559ca688f880 2024-08-19T01:34:05.499 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:05.494+0000 7f5b45600640 1 -- 172.21.15.83:0/2595456326 <== mon.2 v2:172.21.15.117:3300/0 5 ==== fsmap(e 24) ==== 8346+0+0 (secure 0 0 0) 0x7f5b480b8790 con 0x559ca688f880 2024-08-19T01:34:05.499 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:05.494+0000 7f5b65868740 10 client.24890 fetch_fsmap finished waiting for FSMap version 24 2024-08-19T01:34:05.499 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:05.494+0000 7f5b65868740 10 client.24890 resolve_mds: resolved name 'h' to daemon mds.h 2024-08-19T01:34:05.499 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:05.494+0000 7f5b65868740 20 client.24890 populate_metadata read hostname 'smithi083' 2024-08-19T01:34:05.499 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:05.494+0000 7f5b65868740 1 --2- 172.21.15.83:0/2595456326 >> [v2:172.21.15.117:6836/3185224684,v1:172.21.15.117:6837/3185224684] conn(0x559ca68a9de0 0x559ca68ab250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:34:05.499 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:05.494+0000 7f5b65868740 4 client.24890 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-19T01:34:05.499 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:05.494+0000 7f5b65868740 1 -- 172.21.15.83:0/2595456326 --> [v2:172.21.15.117:6836/3185224684,v1:172.21.15.117:6837/3185224684] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559ca6801eb0 con 0x559ca68a9de0 2024-08-19T01:34:05.499 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:05.494+0000 7f5b5be00640 1 --2- 172.21.15.83:0/2595456326 >> [v2:172.21.15.117:6836/3185224684,v1:172.21.15.117:6837/3185224684] conn(0x559ca68a9de0 0x559ca68ab250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:34:05.499 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:05.495+0000 7f5b5be00640 1 --2- 172.21.15.83:0/2595456326 >> [v2:172.21.15.117:6836/3185224684,v1:172.21.15.117:6837/3185224684] conn(0x559ca68a9de0 0x559ca68ab250 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=82ca19db39a9a10 server_cookie=84261d8834237468 in_seq=0 out_seq=0 2024-08-19T01:34:05.499 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:05.495+0000 7f5b45600640 10 client.24890 ms_handle_connect on v2:172.21.15.117:6836/3185224684 2024-08-19T01:34:05.499 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:05.495+0000 7f5b45600640 1 -- 172.21.15.83:0/2595456326 <== mds.2 v2:172.21.15.117:6836/3185224684 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x559ca6801eb0 con 0x559ca68a9de0 2024-08-19T01:34:05.500 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:05.495+0000 7f5b45600640 10 client.24890 handle_command_reply: tid=0 2024-08-19T01:34:05.500 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:05.496+0000 7f5b65868740 1 client.24890 shutdown 2024-08-19T01:34:05.500 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:05.496+0000 7f5b65868740 1 -- 172.21.15.83:0/2595456326 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559ca688f880 msgr2=0x559ca68964c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:34:05.500 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:05.496+0000 7f5b65868740 1 --2- 172.21.15.83:0/2595456326 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559ca688f880 0x559ca68964c0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f5b480b8af0 tx=0x7f5b48031490 comp rx=0 tx=0).stop 2024-08-19T01:34:05.500 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:05.496+0000 7f5b65868740 1 -- 172.21.15.83:0/2595456326 shutdown_connections 2024-08-19T01:34:05.500 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:05.496+0000 7f5b65868740 1 --2- 172.21.15.83:0/2595456326 >> [v2:172.21.15.117:6836/3185224684,v1:172.21.15.117:6837/3185224684] conn(0x559ca68a9de0 0x559ca68ab250 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:05.500 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:05.496+0000 7f5b65868740 1 --2- 172.21.15.83:0/2595456326 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559ca68990b0 0x559ca6896a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:05.500 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:05.496+0000 7f5b65868740 1 --2- 172.21.15.83:0/2595456326 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559ca688f880 0x559ca68964c0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:05.500 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:05.496+0000 7f5b45600640 0 client.24890 ms_handle_reset on v2:172.21.15.117:6836/3185224684 2024-08-19T01:34:05.500 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:05.496+0000 7f5b65868740 1 --2- 172.21.15.83:0/2595456326 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559ca688e310 0x559ca6895f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:05.500 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:05.496+0000 7f5b65868740 1 -- 172.21.15.83:0/2595456326 >> 172.21.15.83:0/2595456326 conn(0x559ca6856a00 msgr2=0x559ca6890670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:34:05.500 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:05.496+0000 7f5b65868740 1 -- 172.21.15.83:0/2595456326 shutdown_connections 2024-08-19T01:34:05.500 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:05.496+0000 7f5b65868740 1 -- 172.21.15.83:0/2595456326 wait complete. 2024-08-19T01:34:05.500 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:05.496+0000 7f5b65868740 20 client.24890 trim_cache size 0 max 16384 2024-08-19T01:34:05.516 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:05.514+0000 7f5b65868740 1 Processor -- start 2024-08-19T01:34:05.516 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:05.514+0000 7f5b65868740 1 -- start start 2024-08-19T01:34:05.516 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:05.515+0000 7f5b65868740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559ca688e310 0x559ca6896850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:34:05.516 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:05.515+0000 7f5b65868740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559ca688f880 0x559ca689a4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:34:05.516 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:05.515+0000 7f5b65868740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559ca68990b0 0x559ca689a9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:34:05.516 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:05.515+0000 7f5b65868740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x559ca68a9590 con 0x559ca68990b0 2024-08-19T01:34:05.516 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:05.515+0000 7f5b65868740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x559ca68a9c60 con 0x559ca688e310 2024-08-19T01:34:05.516 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:05.515+0000 7f5b65868740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x559ca68aa330 con 0x559ca688f880 2024-08-19T01:34:05.516 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:05.515+0000 7f5b60c00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559ca68990b0 0x559ca689a9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:34:05.516 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:05.515+0000 7f5b60c00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559ca68990b0 0x559ca689a9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:52954/0 (socket says 172.21.15.83:52954) 2024-08-19T01:34:05.516 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:05.515+0000 7f5b60c00640 1 -- 172.21.15.83:0/394173263 learned_addr learned my addr 172.21.15.83:0/394173263 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-08-19T01:34:05.517 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:05.515+0000 7f5b5b400640 1 --2- 172.21.15.83:0/394173263 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559ca688f880 0x559ca689a4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:34:05.517 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:05.515+0000 7f5b5be00640 1 --2- 172.21.15.83:0/394173263 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559ca688e310 0x559ca6896850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:34:05.518 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:05.516+0000 7f5b5be00640 1 -- 172.21.15.83:0/394173263 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559ca688f880 msgr2=0x559ca689a4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:34:05.518 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:05.516+0000 7f5b5be00640 1 --2- 172.21.15.83:0/394173263 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559ca688f880 0x559ca689a4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:05.518 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:05.516+0000 7f5b5be00640 1 -- 172.21.15.83:0/394173263 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559ca68990b0 msgr2=0x559ca689a9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:34:05.518 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:05.516+0000 7f5b5be00640 1 --2- 172.21.15.83:0/394173263 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559ca68990b0 0x559ca689a9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:05.518 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:05.516+0000 7f5b5be00640 1 -- 172.21.15.83:0/394173263 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559ca689bc90 con 0x559ca688e310 2024-08-19T01:34:05.519 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:05.516+0000 7f5b60c00640 1 --2- 172.21.15.83:0/394173263 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559ca68990b0 0x559ca689a9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-19T01:34:05.519 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:05.516+0000 7f5b5be00640 1 --2- 172.21.15.83:0/394173263 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559ca688e310 0x559ca6896850 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f5b4c000c00 tx=0x7f5b4c003e80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=19d985d5c85d8e07 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:34:05.519 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:05.516+0000 7f5b47e00640 1 -- 172.21.15.83:0/394173263 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b4c009e20 con 0x559ca688e310 2024-08-19T01:34:05.519 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:05.516+0000 7f5b47e00640 1 -- 172.21.15.83:0/394173263 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5b4c029070 con 0x559ca688e310 2024-08-19T01:34:05.519 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:05.516+0000 7f5b47e00640 1 -- 172.21.15.83:0/394173263 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b4c003bc0 con 0x559ca688e310 2024-08-19T01:34:05.519 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:05.517+0000 7f5b65868740 1 -- 172.21.15.83:0/394173263 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559ca688e310 msgr2=0x559ca6896850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:34:05.519 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:05.517+0000 7f5b65868740 1 --2- 172.21.15.83:0/394173263 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559ca688e310 0x559ca6896850 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f5b4c000c00 tx=0x7f5b4c003e80 comp rx=0 tx=0).stop 2024-08-19T01:34:05.519 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:05.517+0000 7f5b65868740 1 -- 172.21.15.83:0/394173263 shutdown_connections 2024-08-19T01:34:05.519 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:05.517+0000 7f5b65868740 1 --2- 172.21.15.83:0/394173263 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559ca68990b0 0x559ca689a9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:05.519 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:05.517+0000 7f5b65868740 1 --2- 172.21.15.83:0/394173263 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559ca688f880 0x559ca689a4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:05.519 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:05.517+0000 7f5b65868740 1 --2- 172.21.15.83:0/394173263 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559ca688e310 0x559ca6896850 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:05.519 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:05.517+0000 7f5b65868740 1 -- 172.21.15.83:0/394173263 >> 172.21.15.83:0/394173263 conn(0x559ca6856a00 msgr2=0x559ca68a8ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:34:05.520 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:05.517+0000 7f5b65868740 1 -- 172.21.15.83:0/394173263 shutdown_connections 2024-08-19T01:34:05.520 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:05.517+0000 7f5b65868740 1 -- 172.21.15.83:0/394173263 wait complete. 2024-08-19T01:34:05.520 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:05.517+0000 7f5b65868740 1 -- start start 2024-08-19T01:34:05.520 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:05.517+0000 7f5b65868740 1 Processor -- start 2024-08-19T01:34:05.523 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:05.518+0000 7f5b65868740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559ca688e310 0x559ca68b3510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:34:05.523 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:05.518+0000 7f5b65868740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559ca688f880 0x559ca68b3a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:34:05.523 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:05.518+0000 7f5b65868740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559ca68990b0 0x559ca6893220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:34:05.523 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:05.518+0000 7f5b65868740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x559ca68ab0d0 con 0x559ca688f880 2024-08-19T01:34:05.523 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:05.518+0000 7f5b65868740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x559ca68aaa00 con 0x559ca688e310 2024-08-19T01:34:05.523 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:05.518+0000 7f5b65868740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x559ca68ab7a0 con 0x559ca68990b0 2024-08-19T01:34:05.523 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:05.518+0000 7f5b65868740 10 client.0 _ll_register_callbacks cb 0x7f5b48005de0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-19T01:34:05.523 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:05.518+0000 7f5b5b400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559ca688f880 0x559ca68b3a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:34:05.523 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:05.518+0000 7f5b5b400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559ca688f880 0x559ca68b3a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:52962/0 (socket says 172.21.15.83:52962) 2024-08-19T01:34:05.523 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:05.518+0000 7f5b5b400640 1 -- 172.21.15.83:0/1361885279 learned_addr learned my addr 172.21.15.83:0/1361885279 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-08-19T01:34:05.524 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:05.518+0000 7f5b5be00640 1 --2- 172.21.15.83:0/1361885279 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559ca688e310 0x559ca68b3510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:34:05.524 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:05.518+0000 7f5b60c00640 1 --2- 172.21.15.83:0/1361885279 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559ca68990b0 0x559ca6893220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:34:05.524 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:05.519+0000 7f5b5b400640 1 -- 172.21.15.83:0/1361885279 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559ca68990b0 msgr2=0x559ca6893220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:34:05.524 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:05.519+0000 7f5b5b400640 1 --2- 172.21.15.83:0/1361885279 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559ca68990b0 0x559ca6893220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:05.524 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:05.519+0000 7f5b5b400640 1 -- 172.21.15.83:0/1361885279 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559ca688e310 msgr2=0x559ca68b3510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:34:05.524 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:05.519+0000 7f5b5b400640 1 --2- 172.21.15.83:0/1361885279 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559ca688e310 0x559ca68b3510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:05.524 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:05.519+0000 7f5b5b400640 1 -- 172.21.15.83:0/1361885279 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559ca688f4d0 con 0x559ca688f880 2024-08-19T01:34:05.524 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:05.519+0000 7f5b5b400640 1 --2- 172.21.15.83:0/1361885279 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559ca688f880 0x559ca68b3a50 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f5b480ba3d0 tx=0x7f5b48009560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=148511f706a8ed19 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:34:05.524 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:05.519+0000 7f5b45600640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-08-19T01:34:05.525 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:05.519+0000 7f5b45600640 1 -- 172.21.15.83:0/1361885279 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b48049070 con 0x559ca688f880 2024-08-19T01:34:05.525 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:05.519+0000 7f5b45600640 1 -- 172.21.15.83:0/1361885279 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8269+0+0 (secure 0 0 0) 0x7f5b480bb570 con 0x559ca688f880 2024-08-19T01:34:05.525 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:05.519+0000 7f5b65868740 1 -- 172.21.15.83:0/1361885279 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5b4c029070 con 0x559ca688f880 2024-08-19T01:34:05.525 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:05.519+0000 7f5b45600640 1 -- 172.21.15.83:0/1361885279 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f5b480bbc60 con 0x559ca688f880 2024-08-19T01:34:05.525 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:05.519+0000 7f5b65868740 10 client.14829 fetch_fsmap learned FSMap version 24 2024-08-19T01:34:05.525 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:05.519+0000 7f5b65868740 1 -- 172.21.15.83:0/1361885279 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x559ca68998c0 con 0x559ca688f880 2024-08-19T01:34:05.525 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:05.520+0000 7f5b45600640 1 -- 172.21.15.83:0/1361885279 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5b4c029070 con 0x559ca688f880 2024-08-19T01:34:05.525 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:05.520+0000 7f5b45600640 1 -- 172.21.15.83:0/1361885279 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 24) ==== 8346+0+0 (secure 0 0 0) 0x7f5b480b7e70 con 0x559ca688f880 2024-08-19T01:34:05.525 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:05.520+0000 7f5b65868740 10 client.14829 fetch_fsmap finished waiting for FSMap version 24 2024-08-19T01:34:05.525 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:05.520+0000 7f5b65868740 10 client.14829 resolve_mds: resolved name 'h' to daemon mds.h 2024-08-19T01:34:05.525 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:05.520+0000 7f5b65868740 20 client.14829 populate_metadata read hostname 'smithi083' 2024-08-19T01:34:05.526 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:05.520+0000 7f5b65868740 1 --2- 172.21.15.83:0/1361885279 >> [v2:172.21.15.117:6836/3185224684,v1:172.21.15.117:6837/3185224684] conn(0x559ca69182c0 0x559ca6916fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:34:05.526 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:05.520+0000 7f5b65868740 4 client.14829 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "status"}] 2024-08-19T01:34:05.526 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:05.520+0000 7f5b65868740 1 -- 172.21.15.83:0/1361885279 --> [v2:172.21.15.117:6836/3185224684,v1:172.21.15.117:6837/3185224684] -- command(tid 0: {"prefix": "status"}) -- 0x559ca67fc3c0 con 0x559ca69182c0 2024-08-19T01:34:05.526 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:05.520+0000 7f5b5be00640 1 --2- 172.21.15.83:0/1361885279 >> [v2:172.21.15.117:6836/3185224684,v1:172.21.15.117:6837/3185224684] conn(0x559ca69182c0 0x559ca6916fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:34:05.526 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:05.521+0000 7f5b5be00640 1 --2- 172.21.15.83:0/1361885279 >> [v2:172.21.15.117:6836/3185224684,v1:172.21.15.117:6837/3185224684] conn(0x559ca69182c0 0x559ca6916fd0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=c99303b87f6b246 server_cookie=32bb7c591ad43410 in_seq=0 out_seq=0 2024-08-19T01:34:05.526 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:05.521+0000 7f5b45600640 10 client.14829 ms_handle_connect on v2:172.21.15.117:6836/3185224684 2024-08-19T01:34:05.526 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:05.521+0000 7f5b45600640 1 -- 172.21.15.83:0/1361885279 <== mds.2 v2:172.21.15.117:6836/3185224684 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x559ca67fc3c0 con 0x559ca69182c0 2024-08-19T01:34:05.526 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:05.521+0000 7f5b45600640 10 client.14829 handle_command_reply: tid=0 2024-08-19T01:34:05.526 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:05.521+0000 7f5b65868740 1 client.14829 shutdown 2024-08-19T01:34:05.526 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:05.521+0000 7f5b65868740 1 -- 172.21.15.83:0/1361885279 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559ca688f880 msgr2=0x559ca68b3a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:34:05.526 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:05.521+0000 7f5b65868740 1 --2- 172.21.15.83:0/1361885279 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559ca688f880 0x559ca68b3a50 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f5b480ba3d0 tx=0x7f5b48009560 comp rx=0 tx=0).stop 2024-08-19T01:34:05.526 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:05.521+0000 7f5b65868740 1 -- 172.21.15.83:0/1361885279 shutdown_connections 2024-08-19T01:34:05.527 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:05.521+0000 7f5b65868740 1 --2- 172.21.15.83:0/1361885279 >> [v2:172.21.15.117:6836/3185224684,v1:172.21.15.117:6837/3185224684] conn(0x559ca69182c0 0x559ca6916fd0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:05.527 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:05.521+0000 7f5b65868740 1 --2- 172.21.15.83:0/1361885279 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559ca68990b0 0x559ca6893220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:05.527 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:05.521+0000 7f5b65868740 1 --2- 172.21.15.83:0/1361885279 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559ca688f880 0x559ca68b3a50 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:05.527 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:05.521+0000 7f5b65868740 1 --2- 172.21.15.83:0/1361885279 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559ca688e310 0x559ca68b3510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:05.527 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:05.521+0000 7f5b65868740 1 -- 172.21.15.83:0/1361885279 >> 172.21.15.83:0/1361885279 conn(0x559ca6856a00 msgr2=0x559ca688a400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:34:05.527 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:05.521+0000 7f5b45600640 0 client.14829 ms_handle_reset on v2:172.21.15.117:6836/3185224684 2024-08-19T01:34:05.527 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:05.522+0000 7f5b65868740 1 -- 172.21.15.83:0/1361885279 shutdown_connections 2024-08-19T01:34:05.527 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:05.522+0000 7f5b65868740 1 -- 172.21.15.83:0/1361885279 wait complete. 2024-08-19T01:34:05.527 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:05.522+0000 7f5b65868740 20 client.14829 trim_cache size 0 max 16384 2024-08-19T01:34:05.528 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-08-19T01:34:05.528 INFO:teuthology.orchestra.run.smithi083.stdout: "cluster_fsid": "bad8780e-5dc9-11ef-bccf-c7b262605968", 2024-08-19T01:34:05.528 INFO:teuthology.orchestra.run.smithi083.stdout: "whoami": 2, 2024-08-19T01:34:05.528 INFO:teuthology.orchestra.run.smithi083.stdout: "id": 24509, 2024-08-19T01:34:05.528 INFO:teuthology.orchestra.run.smithi083.stdout: "want_state": "up:active", 2024-08-19T01:34:05.528 INFO:teuthology.orchestra.run.smithi083.stdout: "state": "up:active", 2024-08-19T01:34:05.528 INFO:teuthology.orchestra.run.smithi083.stdout: "fs_name": "cephfs", 2024-08-19T01:34:05.528 INFO:teuthology.orchestra.run.smithi083.stdout: "rank_uptime": 74.239656111000002, 2024-08-19T01:34:05.528 INFO:teuthology.orchestra.run.smithi083.stdout: "mdsmap_epoch": 24, 2024-08-19T01:34:05.528 INFO:teuthology.orchestra.run.smithi083.stdout: "osdmap_epoch": 86, 2024-08-19T01:34:05.528 INFO:teuthology.orchestra.run.smithi083.stdout: "osdmap_epoch_barrier": 84, 2024-08-19T01:34:05.528 INFO:teuthology.orchestra.run.smithi083.stdout: "uptime": 108.658189705 2024-08-19T01:34:05.529 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-08-19T01:34:05.529 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:05.522+0000 7f5b62c00640 1 -- 172.21.15.83:0/2769699094 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f5b400a2c00 msgr2=0x7f5b400a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:34:05.529 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:05.522+0000 7f5b62c00640 1 --2- 172.21.15.83:0/2769699094 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f5b400a2c00 0x7f5b400a50c0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f5b540049d0 tx=0x7f5b54009500 comp rx=0 tx=0).stop 2024-08-19T01:34:05.529 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:05.522+0000 7f5b62c00640 1 -- 172.21.15.83:0/2769699094 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5b5c102250 msgr2=0x7f5b5c19a8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:34:05.529 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:05.522+0000 7f5b62c00640 1 --2- 172.21.15.83:0/2769699094 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5b5c102250 0x7f5b5c19a8f0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f5b4802a940 tx=0x7f5b48003b20 comp rx=0 tx=0).stop 2024-08-19T01:34:05.530 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:05.522+0000 7f5b62c00640 1 -- 172.21.15.83:0/2769699094 shutdown_connections 2024-08-19T01:34:05.530 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:05.522+0000 7f5b62c00640 1 --2- 172.21.15.83:0/2769699094 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f5b400a2c00 0x7f5b400a50c0 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:05.530 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:05.522+0000 7f5b62c00640 1 --2- 172.21.15.83:0/2769699094 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5b5c1078b0 0x7f5b5c19ec80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:05.530 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:05.522+0000 7f5b62c00640 1 --2- 172.21.15.83:0/2769699094 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5b5c102250 0x7f5b5c19a8f0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:05.530 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:05.522+0000 7f5b62c00640 1 --2- 172.21.15.83:0/2769699094 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5b5c1018b0 0x7f5b5c19a3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:05.530 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:05.522+0000 7f5b62c00640 1 -- 172.21.15.83:0/2769699094 >> 172.21.15.83:0/2769699094 conn(0x7f5b5c0fd5f0 msgr2=0x7f5b5c10cf80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:34:05.530 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:05.523+0000 7f5b62c00640 1 -- 172.21.15.83:0/2769699094 shutdown_connections 2024-08-19T01:34:05.530 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:05.523+0000 7f5b62c00640 1 -- 172.21.15.83:0/2769699094 wait complete. 2024-08-19T01:34:05.654 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:34:05 smithi083 ceph-mon[17120]: pgmap v316: 97 pgs: 97 active+clean; 640 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-08-19T01:34:05.655 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:34:05 smithi083 ceph-mon[17120]: mgrmap e18: x(active, since 8m), standbys: z, y 2024-08-19T01:34:05.952 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:34:05 smithi117 ceph-mon[22016]: pgmap v316: 97 pgs: 97 active+clean; 640 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-08-19T01:34:05.952 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:34:05 smithi117 ceph-mon[22016]: mgrmap e18: x(active, since 8m), standbys: z, y 2024-08-19T01:34:05.966 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:34:05 smithi167 ceph-mon[21875]: pgmap v316: 97 pgs: 97 active+clean; 640 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-08-19T01:34:05.966 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:34:05 smithi167 ceph-mon[21875]: mgrmap e18: x(active, since 8m), standbys: z, y 2024-08-19T01:34:06.506 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833 shell --fsid bad8780e-5dc9-11ef-bccf-c7b262605968 -- ceph tell mds.g status 2024-08-19T01:34:06.880 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/bad8780e-5dc9-11ef-bccf-c7b262605968/mon.a/config 2024-08-19T01:34:07.412 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:07.385+0000 7f38d1200640 1 -- 172.21.15.83:0/4248254530 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f38cc109900 msgr2=0x7f38cc109ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:34:07.412 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:07.385+0000 7f38d1200640 1 --2- 172.21.15.83:0/4248254530 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f38cc109900 0x7f38cc109ce0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f38bc009930 tx=0x7f38bc02a420 comp rx=0 tx=0).stop 2024-08-19T01:34:07.412 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:07.385+0000 7f38d1200640 1 -- 172.21.15.83:0/4248254530 shutdown_connections 2024-08-19T01:34:07.412 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:07.385+0000 7f38d1200640 1 --2- 172.21.15.83:0/4248254530 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f38cc1042a0 0x7f38cc1107d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:07.413 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:07.385+0000 7f38d1200640 1 --2- 172.21.15.83:0/4248254530 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f38cc103900 0x7f38cc103d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:07.413 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:07.385+0000 7f38d1200640 1 --2- 172.21.15.83:0/4248254530 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f38cc109900 0x7f38cc109ce0 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:07.413 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:07.385+0000 7f38d1200640 1 -- 172.21.15.83:0/4248254530 >> 172.21.15.83:0/4248254530 conn(0x7f38cc0ff600 msgr2=0x7f38cc101a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:34:07.413 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:07.385+0000 7f38d1200640 1 -- 172.21.15.83:0/4248254530 shutdown_connections 2024-08-19T01:34:07.413 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:07.385+0000 7f38d1200640 1 -- 172.21.15.83:0/4248254530 wait complete. 2024-08-19T01:34:07.413 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:07.386+0000 7f38d1200640 1 Processor -- start 2024-08-19T01:34:07.413 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:07.386+0000 7f38d1200640 1 -- start start 2024-08-19T01:34:07.413 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:07.387+0000 7f38d1200640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f38cc103900 0x7f38cc19a1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:34:07.413 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:07.387+0000 7f38d1200640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f38cc1042a0 0x7f38cc19a710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:34:07.413 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:07.387+0000 7f38d1200640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f38cc109900 0x7f38cc19eaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:34:07.413 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:07.387+0000 7f38d1200640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f38cc113d50 con 0x7f38cc109900 2024-08-19T01:34:07.413 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:07.387+0000 7f38d1200640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f38cc113bd0 con 0x7f38cc1042a0 2024-08-19T01:34:07.413 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:07.387+0000 7f38d1200640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f38cc113ed0 con 0x7f38cc103900 2024-08-19T01:34:07.414 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:07.387+0000 7f38cbe00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f38cc109900 0x7f38cc19eaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:34:07.414 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:07.387+0000 7f38cbe00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f38cc109900 0x7f38cc19eaa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:52986/0 (socket says 172.21.15.83:52986) 2024-08-19T01:34:07.414 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:07.387+0000 7f38cbe00640 1 -- 172.21.15.83:0/1876331344 learned_addr learned my addr 172.21.15.83:0/1876331344 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-08-19T01:34:07.414 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:07.387+0000 7f38c3e00640 1 --2- 172.21.15.83:0/1876331344 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f38cc1042a0 0x7f38cc19a710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:34:07.414 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:07.387+0000 7f38cb400640 1 --2- 172.21.15.83:0/1876331344 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f38cc103900 0x7f38cc19a1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:34:07.414 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:07.387+0000 7f38cbe00640 1 -- 172.21.15.83:0/1876331344 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f38cc103900 msgr2=0x7f38cc19a1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:34:07.414 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:07.387+0000 7f38cbe00640 1 --2- 172.21.15.83:0/1876331344 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f38cc103900 0x7f38cc19a1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:07.414 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:07.387+0000 7f38cbe00640 1 -- 172.21.15.83:0/1876331344 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f38cc1042a0 msgr2=0x7f38cc19a710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:34:07.414 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:07.387+0000 7f38cbe00640 1 --2- 172.21.15.83:0/1876331344 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f38cc1042a0 0x7f38cc19a710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:07.414 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:07.387+0000 7f38cbe00640 1 -- 172.21.15.83:0/1876331344 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f38cc19f180 con 0x7f38cc109900 2024-08-19T01:34:07.414 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:07.387+0000 7f38c3e00640 1 --2- 172.21.15.83:0/1876331344 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f38cc1042a0 0x7f38cc19a710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-19T01:34:07.414 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:07.388+0000 7f38cbe00640 1 --2- 172.21.15.83:0/1876331344 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f38cc109900 0x7f38cc19eaa0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f38c400eaa0 tx=0x7f38c400ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:34:07.414 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:07.388+0000 7f38c8c00640 1 -- 172.21.15.83:0/1876331344 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38c400cdb0 con 0x7f38cc109900 2024-08-19T01:34:07.415 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:07.388+0000 7f38d1200640 1 -- 172.21.15.83:0/1876331344 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f38cc19f470 con 0x7f38cc109900 2024-08-19T01:34:07.415 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:07.388+0000 7f38c8c00640 1 -- 172.21.15.83:0/1876331344 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f38c4005020 con 0x7f38cc109900 2024-08-19T01:34:07.415 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:07.388+0000 7f38c8c00640 1 -- 172.21.15.83:0/1876331344 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38c40116d0 con 0x7f38cc109900 2024-08-19T01:34:07.415 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:07.388+0000 7f38d1200640 1 -- 172.21.15.83:0/1876331344 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f38cc1a6d50 con 0x7f38cc109900 2024-08-19T01:34:07.415 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:07.391+0000 7f38d5483740 1 Processor -- start 2024-08-19T01:34:07.415 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:07.391+0000 7f38c8c00640 1 -- 172.21.15.83:0/1876331344 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f38c40054d0 con 0x7f38cc109900 2024-08-19T01:34:07.415 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:07.391+0000 7f38d5483740 1 -- start start 2024-08-19T01:34:07.415 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:07.391+0000 7f38d5483740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5558115a8a50 0x5558115a6fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:34:07.415 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:07.391+0000 7f38d5483740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5558115a74e0 0x5558115a7920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:34:07.415 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:07.391+0000 7f38d5483740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5558115b2280 0x5558115b4670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:34:07.415 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:07.391+0000 7f38d5483740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55581150d940 con 0x5558115b2280 2024-08-19T01:34:07.415 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:07.391+0000 7f38d5483740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x5558114a59a0 con 0x5558115a8a50 2024-08-19T01:34:07.415 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:07.391+0000 7f38d5483740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x555811509900 con 0x5558115a74e0 2024-08-19T01:34:07.415 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:07.392+0000 7f38c3e00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5558115a8a50 0x5558115a6fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:34:07.416 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:07.392+0000 7f38c3e00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5558115a8a50 0x5558115a6fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.83:53802/0 (socket says 172.21.15.83:53802) 2024-08-19T01:34:07.416 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:07.392+0000 7f38c3e00640 1 -- 172.21.15.83:0/2944202062 learned_addr learned my addr 172.21.15.83:0/2944202062 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-08-19T01:34:07.416 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:07.392+0000 7f38c3e00640 1 --2- 172.21.15.83:0/2944202062 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5558115b2280 0x5558115b4670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:34:07.416 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:07.392+0000 7f38cb400640 1 --2- 172.21.15.83:0/2944202062 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5558115a74e0 0x5558115a7920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:34:07.416 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:07.392+0000 7f38c3e00640 1 -- 172.21.15.83:0/2944202062 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5558115a74e0 msgr2=0x5558115a7920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:34:07.416 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:07.392+0000 7f38c3e00640 1 --2- 172.21.15.83:0/2944202062 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5558115a74e0 0x5558115a7920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:07.416 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:07.392+0000 7f38c3e00640 1 -- 172.21.15.83:0/2944202062 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5558115b2280 msgr2=0x5558115b4670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:34:07.416 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:07.392+0000 7f38c3e00640 1 --2- 172.21.15.83:0/2944202062 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5558115b2280 0x5558115b4670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:07.416 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:07.392+0000 7f38c3e00640 1 -- 172.21.15.83:0/2944202062 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555811493ef0 con 0x5558115a8a50 2024-08-19T01:34:07.416 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:07.392+0000 7f38c3e00640 1 --2- 172.21.15.83:0/2944202062 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5558115a8a50 0x5558115a6fa0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f38b4009fd0 tx=0x7f38b4009ba0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fc03c0acc4b20da6 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:34:07.416 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:07.393+0000 7f38c0c00640 1 -- 172.21.15.83:0/2944202062 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38b400fc50 con 0x5558115a8a50 2024-08-19T01:34:07.416 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:07.393+0000 7f38c0c00640 1 -- 172.21.15.83:0/2944202062 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f38b400fdf0 con 0x5558115a8a50 2024-08-19T01:34:07.416 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:07.393+0000 7f38c0c00640 1 -- 172.21.15.83:0/2944202062 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38b400e6f0 con 0x5558115a8a50 2024-08-19T01:34:07.416 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:07.393+0000 7f38c8c00640 1 --2- 172.21.15.83:0/1876331344 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f38b00a2b80 0x7f38b00a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:34:07.417 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:07.393+0000 7f38cb400640 1 --2- 172.21.15.83:0/1876331344 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f38b00a2b80 0x7f38b00a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:34:07.417 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:07.393+0000 7f38c8c00640 1 -- 172.21.15.83:0/1876331344 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8269+0+0 (secure 0 0 0) 0x7f38c40cf210 con 0x7f38cc109900 2024-08-19T01:34:07.417 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:07.393+0000 7f38d5483740 1 -- 172.21.15.83:0/2944202062 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5558115a8a50 msgr2=0x5558115a6fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:34:07.417 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:07.393+0000 7f38d5483740 1 --2- 172.21.15.83:0/2944202062 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5558115a8a50 0x5558115a6fa0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f38b4009fd0 tx=0x7f38b4009ba0 comp rx=0 tx=0).stop 2024-08-19T01:34:07.417 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:07.394+0000 7f38d5483740 1 -- 172.21.15.83:0/2944202062 shutdown_connections 2024-08-19T01:34:07.417 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:07.394+0000 7f38d5483740 1 --2- 172.21.15.83:0/2944202062 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5558115b2280 0x5558115b4670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:07.417 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:07.394+0000 7f38d5483740 1 --2- 172.21.15.83:0/2944202062 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5558115a74e0 0x5558115a7920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:07.417 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:07.394+0000 7f38d5483740 1 --2- 172.21.15.83:0/2944202062 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5558115a8a50 0x5558115a6fa0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:07.417 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:07.394+0000 7f38d5483740 1 -- 172.21.15.83:0/2944202062 >> 172.21.15.83:0/2944202062 conn(0x55581156fa00 msgr2=0x5558115a3b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:34:07.417 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:07.394+0000 7f38d5483740 1 -- 172.21.15.83:0/2944202062 shutdown_connections 2024-08-19T01:34:07.417 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:07.394+0000 7f38cb400640 1 --2- 172.21.15.83:0/1876331344 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f38b00a2b80 0x7f38b00a5040 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f38bc006010 tx=0x7f38bc000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:34:07.417 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:07.394+0000 7f38d5483740 1 -- 172.21.15.83:0/2944202062 wait complete. 2024-08-19T01:34:07.417 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:07.394+0000 7f38d5483740 1 -- start start 2024-08-19T01:34:07.418 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:07.394+0000 7f38d5483740 1 Processor -- start 2024-08-19T01:34:07.418 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:07.395+0000 7f38d5483740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5558115a74e0 0x5558115af260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:34:07.418 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:07.395+0000 7f38d5483740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5558115a8a50 0x5558115af7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:34:07.418 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:07.395+0000 7f38d5483740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5558115b2280 0x5558115afce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:34:07.418 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:07.395+0000 7f38d5483740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x5558115b4f80 con 0x5558115b2280 2024-08-19T01:34:07.418 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:07.395+0000 7f38d5483740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x5558114c8a80 con 0x5558115a74e0 2024-08-19T01:34:07.418 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:07.395+0000 7f38d5483740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x5558115b5100 con 0x5558115a8a50 2024-08-19T01:34:07.418 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:07.395+0000 7f38d5483740 10 client.0 _ll_register_callbacks cb 0x555811491140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-19T01:34:07.418 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:07.395+0000 7f38cb400640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5558115a74e0 0x5558115af260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:34:07.418 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:07.395+0000 7f38cbe00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5558115b2280 0x5558115afce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:34:07.418 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:07.395+0000 7f38cb400640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5558115a74e0 0x5558115af260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.83:53810/0 (socket says 172.21.15.83:53810) 2024-08-19T01:34:07.418 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:07.395+0000 7f38cb400640 1 -- 172.21.15.83:0/710197248 learned_addr learned my addr 172.21.15.83:0/710197248 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-08-19T01:34:07.418 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:07.395+0000 7f38c3e00640 1 --2- 172.21.15.83:0/710197248 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5558115a8a50 0x5558115af7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:34:07.419 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:07.395+0000 7f38cb400640 1 -- 172.21.15.83:0/710197248 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5558115a8a50 msgr2=0x5558115af7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:34:07.419 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:07.395+0000 7f38cb400640 1 --2- 172.21.15.83:0/710197248 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5558115a8a50 0x5558115af7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:07.419 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:07.395+0000 7f38cb400640 1 -- 172.21.15.83:0/710197248 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5558115b2280 msgr2=0x5558115afce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:34:07.419 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:07.395+0000 7f38cb400640 1 --2- 172.21.15.83:0/710197248 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5558115b2280 0x5558115afce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:07.419 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:07.395+0000 7f38cb400640 1 -- 172.21.15.83:0/710197248 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5558114ac700 con 0x5558115a74e0 2024-08-19T01:34:07.419 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:07.396+0000 7f38cb400640 1 --2- 172.21.15.83:0/710197248 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5558115a74e0 0x5558115af260 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f38bc009930 tx=0x7f38bc009260 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e11a82f4a2b0b136 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:34:07.419 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:07.396+0000 7f38b9600640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-08-19T01:34:07.419 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:07.396+0000 7f38b9600640 1 -- 172.21.15.83:0/710197248 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38bc04e070 con 0x5558115a74e0 2024-08-19T01:34:07.419 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:07.396+0000 7f38b9600640 1 -- 172.21.15.83:0/710197248 <== mon.2 v2:172.21.15.117:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8269+0+0 (secure 0 0 0) 0x7f38bc004cd0 con 0x5558115a74e0 2024-08-19T01:34:07.419 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:07.396+0000 7f38d5483740 1 -- 172.21.15.83:0/710197248 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f38b400fdf0 con 0x5558115a74e0 2024-08-19T01:34:07.419 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:07.396+0000 7f38b9600640 1 -- 172.21.15.83:0/710197248 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f38bc05a050 con 0x5558115a74e0 2024-08-19T01:34:07.419 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:07.396+0000 7f38d5483740 10 client.24908 fetch_fsmap learned FSMap version 24 2024-08-19T01:34:07.419 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:07.396+0000 7f38d5483740 1 -- 172.21.15.83:0/710197248 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x55581127f2b0 con 0x5558115a74e0 2024-08-19T01:34:07.420 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:07.396+0000 7f38b9600640 1 -- 172.21.15.83:0/710197248 <== mon.2 v2:172.21.15.117:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f38b400fdf0 con 0x5558115a74e0 2024-08-19T01:34:07.420 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:07.397+0000 7f38b9600640 1 -- 172.21.15.83:0/710197248 <== mon.2 v2:172.21.15.117:3300/0 5 ==== fsmap(e 24) ==== 8346+0+0 (secure 0 0 0) 0x7f38bc003850 con 0x5558115a74e0 2024-08-19T01:34:07.420 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:07.397+0000 7f38d5483740 10 client.24908 fetch_fsmap finished waiting for FSMap version 24 2024-08-19T01:34:07.420 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:07.397+0000 7f38d5483740 10 client.24908 resolve_mds: resolved name 'g' to daemon mds.g 2024-08-19T01:34:07.420 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:07.397+0000 7f38d5483740 20 client.24908 populate_metadata read hostname 'smithi083' 2024-08-19T01:34:07.420 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:07.397+0000 7f38d5483740 1 --2- 172.21.15.83:0/710197248 >> [v2:172.21.15.83:6838/3239261845,v1:172.21.15.83:6839/3239261845] conn(0x5558115c31a0 0x5558115c4610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:34:07.420 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:07.397+0000 7f38d5483740 4 client.24908 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-19T01:34:07.420 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:07.397+0000 7f38d5483740 1 -- 172.21.15.83:0/710197248 --> [v2:172.21.15.83:6838/3239261845,v1:172.21.15.83:6839/3239261845] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55581151aeb0 con 0x5558115c31a0 2024-08-19T01:34:07.420 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:07.397+0000 7f38c3e00640 1 --2- 172.21.15.83:0/710197248 >> [v2:172.21.15.83:6838/3239261845,v1:172.21.15.83:6839/3239261845] conn(0x5558115c31a0 0x5558115c4610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:34:07.420 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:07.397+0000 7f38c3e00640 1 --2- 172.21.15.83:0/710197248 >> [v2:172.21.15.83:6838/3239261845,v1:172.21.15.83:6839/3239261845] conn(0x5558115c31a0 0x5558115c4610 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=f38c2a17a011fd3f server_cookie=8bd89ca544c811af in_seq=0 out_seq=0 2024-08-19T01:34:07.420 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:07.398+0000 7f38b9600640 10 client.24908 ms_handle_connect on v2:172.21.15.83:6838/3239261845 2024-08-19T01:34:07.420 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:07.398+0000 7f38b9600640 1 -- 172.21.15.83:0/710197248 <== mds.1 v2:172.21.15.83:6838/3239261845 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55581151aeb0 con 0x5558115c31a0 2024-08-19T01:34:07.420 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:07.398+0000 7f38b9600640 10 client.24908 handle_command_reply: tid=0 2024-08-19T01:34:07.421 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:07.398+0000 7f38d5483740 1 client.24908 shutdown 2024-08-19T01:34:07.421 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:07.399+0000 7f38d5483740 1 -- 172.21.15.83:0/710197248 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5558115a74e0 msgr2=0x5558115af260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:34:07.421 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:07.399+0000 7f38d5483740 1 --2- 172.21.15.83:0/710197248 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5558115a74e0 0x5558115af260 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f38bc009930 tx=0x7f38bc009260 comp rx=0 tx=0).stop 2024-08-19T01:34:07.421 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:07.399+0000 7f38d5483740 1 -- 172.21.15.83:0/710197248 shutdown_connections 2024-08-19T01:34:07.421 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:07.399+0000 7f38d5483740 1 --2- 172.21.15.83:0/710197248 >> [v2:172.21.15.83:6838/3239261845,v1:172.21.15.83:6839/3239261845] conn(0x5558115c31a0 0x5558115c4610 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:07.421 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:07.399+0000 7f38d5483740 1 --2- 172.21.15.83:0/710197248 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5558115b2280 0x5558115afce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:07.421 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:07.399+0000 7f38d5483740 1 --2- 172.21.15.83:0/710197248 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5558115a8a50 0x5558115af7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:07.421 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:07.399+0000 7f38d5483740 1 --2- 172.21.15.83:0/710197248 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5558115a74e0 0x5558115af260 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:07.421 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:07.399+0000 7f38b9600640 0 client.24908 ms_handle_reset on v2:172.21.15.83:6838/3239261845 2024-08-19T01:34:07.421 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:07.399+0000 7f38d5483740 1 -- 172.21.15.83:0/710197248 >> 172.21.15.83:0/710197248 conn(0x55581156fa00 msgr2=0x5558115a9870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:34:07.421 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:07.399+0000 7f38d5483740 1 -- 172.21.15.83:0/710197248 shutdown_connections 2024-08-19T01:34:07.421 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:07.399+0000 7f38d5483740 1 -- 172.21.15.83:0/710197248 wait complete. 2024-08-19T01:34:07.421 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:07.399+0000 7f38d5483740 20 client.24908 trim_cache size 0 max 16384 2024-08-19T01:34:07.425 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:07.417+0000 7f38d5483740 1 Processor -- start 2024-08-19T01:34:07.425 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:07.418+0000 7f38d5483740 1 -- start start 2024-08-19T01:34:07.426 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:07.418+0000 7f38d5483740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5558115a74e0 0x5558115c3f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:34:07.426 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:07.418+0000 7f38d5483740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5558115a8a50 0x5558115afb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:34:07.426 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:07.418+0000 7f38d5483740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5558115b2280 0x5558115ac4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:34:07.426 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:07.418+0000 7f38d5483740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x5558115c2950 con 0x5558115a74e0 2024-08-19T01:34:07.426 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:07.418+0000 7f38d5483740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x5558115c3020 con 0x5558115b2280 2024-08-19T01:34:07.426 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:07.418+0000 7f38d5483740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x5558115c36f0 con 0x5558115a8a50 2024-08-19T01:34:07.426 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:07.418+0000 7f38cb400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5558115a74e0 0x5558115c3f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:34:07.426 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:07.418+0000 7f38cb400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5558115a74e0 0x5558115c3f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:53014/0 (socket says 172.21.15.83:53014) 2024-08-19T01:34:07.426 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:07.418+0000 7f38cb400640 1 -- 172.21.15.83:0/1281227447 learned_addr learned my addr 172.21.15.83:0/1281227447 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-08-19T01:34:07.426 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:07.418+0000 7f38cbe00640 1 --2- 172.21.15.83:0/1281227447 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5558115b2280 0x5558115ac4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:34:07.426 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:07.418+0000 7f38c3e00640 1 --2- 172.21.15.83:0/1281227447 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5558115a8a50 0x5558115afb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:34:07.426 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:07.418+0000 7f38cb400640 1 -- 172.21.15.83:0/1281227447 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5558115a8a50 msgr2=0x5558115afb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:34:07.427 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:07.418+0000 7f38cb400640 1 --2- 172.21.15.83:0/1281227447 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5558115a8a50 0x5558115afb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:07.427 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:07.418+0000 7f38cb400640 1 -- 172.21.15.83:0/1281227447 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5558115b2280 msgr2=0x5558115ac4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:34:07.427 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:07.418+0000 7f38cb400640 1 --2- 172.21.15.83:0/1281227447 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5558115b2280 0x5558115ac4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:07.427 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:07.419+0000 7f38cb400640 1 -- 172.21.15.83:0/1281227447 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5558115a4870 con 0x5558115a74e0 2024-08-19T01:34:07.427 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:07.419+0000 7f38cbe00640 1 --2- 172.21.15.83:0/1281227447 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5558115b2280 0x5558115ac4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-19T01:34:07.427 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:07.419+0000 7f38cb400640 1 --2- 172.21.15.83:0/1281227447 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5558115a74e0 0x5558115c3f40 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f38bc005d00 tx=0x7f38bc0045a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=455b590bcaf9ed90 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:34:07.427 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:07.419+0000 7f38bbe00640 1 -- 172.21.15.83:0/1281227447 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38bc04e070 con 0x5558115a74e0 2024-08-19T01:34:07.427 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:07.419+0000 7f38bbe00640 1 -- 172.21.15.83:0/1281227447 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f38bc063070 con 0x5558115a74e0 2024-08-19T01:34:07.427 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:07.419+0000 7f38bbe00640 1 -- 172.21.15.83:0/1281227447 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38bc035040 con 0x5558115a74e0 2024-08-19T01:34:07.427 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:07.420+0000 7f38d5483740 1 -- 172.21.15.83:0/1281227447 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5558115a74e0 msgr2=0x5558115c3f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:34:07.427 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:07.420+0000 7f38d5483740 1 --2- 172.21.15.83:0/1281227447 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5558115a74e0 0x5558115c3f40 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f38bc005d00 tx=0x7f38bc0045a0 comp rx=0 tx=0).stop 2024-08-19T01:34:07.427 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:07.420+0000 7f38d5483740 1 -- 172.21.15.83:0/1281227447 shutdown_connections 2024-08-19T01:34:07.427 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:07.420+0000 7f38d5483740 1 --2- 172.21.15.83:0/1281227447 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5558115b2280 0x5558115ac4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:07.427 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:07.420+0000 7f38d5483740 1 --2- 172.21.15.83:0/1281227447 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5558115a8a50 0x5558115afb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:07.428 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:07.420+0000 7f38d5483740 1 --2- 172.21.15.83:0/1281227447 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5558115a74e0 0x5558115c3f40 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:07.428 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:07.420+0000 7f38d5483740 1 -- 172.21.15.83:0/1281227447 >> 172.21.15.83:0/1281227447 conn(0x55581156fa00 msgr2=0x5558115adb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:34:07.428 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:07.420+0000 7f38d5483740 1 -- 172.21.15.83:0/1281227447 shutdown_connections 2024-08-19T01:34:07.428 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:07.420+0000 7f38d5483740 1 -- 172.21.15.83:0/1281227447 wait complete. 2024-08-19T01:34:07.428 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:07.421+0000 7f38d5483740 1 -- start start 2024-08-19T01:34:07.428 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:07.421+0000 7f38d5483740 1 Processor -- start 2024-08-19T01:34:07.428 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:07.422+0000 7f38d5483740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5558115a74e0 0x5558115ccd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:34:07.428 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:07.422+0000 7f38d5483740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5558115a8a50 0x5558115c6090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:34:07.428 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:07.422+0000 7f38d5483740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5558115b2280 0x5558115c65d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:34:07.428 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:07.422+0000 7f38d5483740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x5558115c4490 con 0x5558115a8a50 2024-08-19T01:34:07.428 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:07.422+0000 7f38d5483740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x5558115c3dc0 con 0x5558115b2280 2024-08-19T01:34:07.428 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:07.422+0000 7f38d5483740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x5558115c4b60 con 0x5558115a74e0 2024-08-19T01:34:07.428 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:07.422+0000 7f38d5483740 10 client.0 _ll_register_callbacks cb 0x7f38bc0042b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-19T01:34:07.428 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:07.422+0000 7f38c3e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5558115a8a50 0x5558115c6090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:34:07.429 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:07.422+0000 7f38c3e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5558115a8a50 0x5558115c6090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:53022/0 (socket says 172.21.15.83:53022) 2024-08-19T01:34:07.429 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:07.422+0000 7f38c3e00640 1 -- 172.21.15.83:0/3401391435 learned_addr learned my addr 172.21.15.83:0/3401391435 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-08-19T01:34:07.429 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:07.422+0000 7f38cb400640 1 --2- 172.21.15.83:0/3401391435 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5558115a74e0 0x5558115ccd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:34:07.429 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:07.422+0000 7f38cbe00640 1 --2- 172.21.15.83:0/3401391435 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5558115b2280 0x5558115c65d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:34:07.429 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:07.422+0000 7f38c3e00640 1 -- 172.21.15.83:0/3401391435 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5558115a74e0 msgr2=0x5558115ccd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:34:07.429 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:07.422+0000 7f38c3e00640 1 --2- 172.21.15.83:0/3401391435 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5558115a74e0 0x5558115ccd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:07.429 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:07.422+0000 7f38c3e00640 1 -- 172.21.15.83:0/3401391435 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5558115b2280 msgr2=0x5558115c65d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:34:07.429 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:07.422+0000 7f38c3e00640 1 --2- 172.21.15.83:0/3401391435 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5558115b2280 0x5558115c65d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:07.429 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:07.422+0000 7f38c3e00640 1 -- 172.21.15.83:0/3401391435 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5558115c6b10 con 0x5558115a8a50 2024-08-19T01:34:07.429 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:07.422+0000 7f38cbe00640 1 --2- 172.21.15.83:0/3401391435 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5558115b2280 0x5558115c65d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-19T01:34:07.429 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:07.423+0000 7f38c3e00640 1 --2- 172.21.15.83:0/3401391435 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5558115a8a50 0x5558115c6090 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f38b401ff00 tx=0x7f38b4009f00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bf5f4cb215ce5497 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:34:07.429 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:07.423+0000 7f38b9600640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-08-19T01:34:07.429 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:07.423+0000 7f38b9600640 1 -- 172.21.15.83:0/3401391435 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38b4011030 con 0x5558115a8a50 2024-08-19T01:34:07.430 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:07.423+0000 7f38d5483740 1 -- 172.21.15.83:0/3401391435 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f38bc063070 con 0x5558115a8a50 2024-08-19T01:34:07.430 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:07.423+0000 7f38b9600640 1 -- 172.21.15.83:0/3401391435 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8269+0+0 (secure 0 0 0) 0x7f38b40111d0 con 0x5558115a8a50 2024-08-19T01:34:07.430 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:07.423+0000 7f38b9600640 1 -- 172.21.15.83:0/3401391435 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f38b40113d0 con 0x5558115a8a50 2024-08-19T01:34:07.430 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:07.423+0000 7f38d5483740 10 client.14847 fetch_fsmap learned FSMap version 24 2024-08-19T01:34:07.430 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:07.423+0000 7f38d5483740 1 -- 172.21.15.83:0/3401391435 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x5558115cb580 con 0x5558115a8a50 2024-08-19T01:34:07.430 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:07.423+0000 7f38b9600640 1 -- 172.21.15.83:0/3401391435 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f38bc063070 con 0x5558115a8a50 2024-08-19T01:34:07.430 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:07.424+0000 7f38b9600640 1 -- 172.21.15.83:0/3401391435 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 24) ==== 8346+0+0 (secure 0 0 0) 0x7f38b401d020 con 0x5558115a8a50 2024-08-19T01:34:07.430 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:07.424+0000 7f38d5483740 10 client.14847 fetch_fsmap finished waiting for FSMap version 24 2024-08-19T01:34:07.430 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:07.424+0000 7f38d5483740 10 client.14847 resolve_mds: resolved name 'g' to daemon mds.g 2024-08-19T01:34:07.430 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:07.424+0000 7f38d5483740 20 client.14847 populate_metadata read hostname 'smithi083' 2024-08-19T01:34:07.430 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:07.424+0000 7f38d5483740 1 --2- 172.21.15.83:0/3401391435 >> [v2:172.21.15.83:6838/3239261845,v1:172.21.15.83:6839/3239261845] conn(0x555811631f60 0x555811630c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:34:07.430 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:07.424+0000 7f38d5483740 4 client.14847 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "status"}] 2024-08-19T01:34:07.430 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:07.424+0000 7f38d5483740 1 -- 172.21.15.83:0/3401391435 --> [v2:172.21.15.83:6838/3239261845,v1:172.21.15.83:6839/3239261845] -- command(tid 0: {"prefix": "status"}) -- 0x5558115153c0 con 0x555811631f60 2024-08-19T01:34:07.431 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:07.424+0000 7f38cb400640 1 --2- 172.21.15.83:0/3401391435 >> [v2:172.21.15.83:6838/3239261845,v1:172.21.15.83:6839/3239261845] conn(0x555811631f60 0x555811630c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:34:07.432 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:07.425+0000 7f38cb400640 1 --2- 172.21.15.83:0/3401391435 >> [v2:172.21.15.83:6838/3239261845,v1:172.21.15.83:6839/3239261845] conn(0x555811631f60 0x555811630c70 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=18b3c3da212fc7a7 server_cookie=29fca2da6a5d9107 in_seq=0 out_seq=0 2024-08-19T01:34:07.432 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:07.425+0000 7f38b9600640 10 client.14847 ms_handle_connect on v2:172.21.15.83:6838/3239261845 2024-08-19T01:34:07.432 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:07.425+0000 7f38b9600640 1 -- 172.21.15.83:0/3401391435 <== mds.1 v2:172.21.15.83:6838/3239261845 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x5558115153c0 con 0x555811631f60 2024-08-19T01:34:07.432 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:07.425+0000 7f38b9600640 10 client.14847 handle_command_reply: tid=0 2024-08-19T01:34:07.432 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:07.425+0000 7f38d5483740 1 client.14847 shutdown 2024-08-19T01:34:07.432 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:07.425+0000 7f38d5483740 1 -- 172.21.15.83:0/3401391435 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5558115a8a50 msgr2=0x5558115c6090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:34:07.432 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:07.425+0000 7f38d5483740 1 --2- 172.21.15.83:0/3401391435 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5558115a8a50 0x5558115c6090 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f38b401ff00 tx=0x7f38b4009f00 comp rx=0 tx=0).stop 2024-08-19T01:34:07.432 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:07.425+0000 7f38d5483740 1 -- 172.21.15.83:0/3401391435 shutdown_connections 2024-08-19T01:34:07.433 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:07.425+0000 7f38d5483740 1 --2- 172.21.15.83:0/3401391435 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5558115b2280 0x5558115c65d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:07.433 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:07.425+0000 7f38d5483740 1 --2- 172.21.15.83:0/3401391435 >> [v2:172.21.15.83:6838/3239261845,v1:172.21.15.83:6839/3239261845] conn(0x555811631f60 0x555811630c70 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:07.433 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:07.425+0000 7f38d5483740 1 --2- 172.21.15.83:0/3401391435 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5558115a8a50 0x5558115c6090 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:07.433 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:07.425+0000 7f38d5483740 1 --2- 172.21.15.83:0/3401391435 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5558115a74e0 0x5558115ccd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:07.433 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:07.425+0000 7f38d5483740 1 -- 172.21.15.83:0/3401391435 >> 172.21.15.83:0/3401391435 conn(0x55581156fa00 msgr2=0x5558115bd390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:34:07.433 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:07.425+0000 7f38b9600640 0 client.14847 ms_handle_reset on v2:172.21.15.83:6838/3239261845 2024-08-19T01:34:07.433 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:07.425+0000 7f38d5483740 1 -- 172.21.15.83:0/3401391435 shutdown_connections 2024-08-19T01:34:07.433 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:07.425+0000 7f38d5483740 1 -- 172.21.15.83:0/3401391435 wait complete. 2024-08-19T01:34:07.433 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:07.425+0000 7f38d5483740 20 client.14847 trim_cache size 0 max 16384 2024-08-19T01:34:07.433 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-08-19T01:34:07.433 INFO:teuthology.orchestra.run.smithi083.stdout: "cluster_fsid": "bad8780e-5dc9-11ef-bccf-c7b262605968", 2024-08-19T01:34:07.433 INFO:teuthology.orchestra.run.smithi083.stdout: "whoami": 1, 2024-08-19T01:34:07.434 INFO:teuthology.orchestra.run.smithi083.stdout: "id": 24527, 2024-08-19T01:34:07.434 INFO:teuthology.orchestra.run.smithi083.stdout: "want_state": "up:active", 2024-08-19T01:34:07.434 INFO:teuthology.orchestra.run.smithi083.stdout: "state": "up:active", 2024-08-19T01:34:07.434 INFO:teuthology.orchestra.run.smithi083.stdout: "fs_name": "cephfs", 2024-08-19T01:34:07.434 INFO:teuthology.orchestra.run.smithi083.stdout: "rank_uptime": 77.156406001999997, 2024-08-19T01:34:07.434 INFO:teuthology.orchestra.run.smithi083.stdout: "mdsmap_epoch": 24, 2024-08-19T01:34:07.434 INFO:teuthology.orchestra.run.smithi083.stdout: "osdmap_epoch": 86, 2024-08-19T01:34:07.434 INFO:teuthology.orchestra.run.smithi083.stdout: "osdmap_epoch_barrier": 84, 2024-08-19T01:34:07.434 INFO:teuthology.orchestra.run.smithi083.stdout: "uptime": 103.654518079 2024-08-19T01:34:07.434 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-08-19T01:34:07.435 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:07.426+0000 7f38d1200640 1 -- 172.21.15.83:0/1876331344 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f38b00a2b80 msgr2=0x7f38b00a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:34:07.435 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:07.426+0000 7f38d1200640 1 --2- 172.21.15.83:0/1876331344 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f38b00a2b80 0x7f38b00a5040 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f38bc006010 tx=0x7f38bc000bc0 comp rx=0 tx=0).stop 2024-08-19T01:34:07.435 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:07.426+0000 7f38d1200640 1 -- 172.21.15.83:0/1876331344 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f38cc109900 msgr2=0x7f38cc19eaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:34:07.438 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:07.426+0000 7f38d1200640 1 --2- 172.21.15.83:0/1876331344 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f38cc109900 0x7f38cc19eaa0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f38c400eaa0 tx=0x7f38c400ef60 comp rx=0 tx=0).stop 2024-08-19T01:34:07.438 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:07.426+0000 7f38d1200640 1 -- 172.21.15.83:0/1876331344 shutdown_connections 2024-08-19T01:34:07.438 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:07.426+0000 7f38d1200640 1 --2- 172.21.15.83:0/1876331344 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f38b00a2b80 0x7f38b00a5040 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:07.438 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:07.426+0000 7f38d1200640 1 --2- 172.21.15.83:0/1876331344 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f38cc109900 0x7f38cc19eaa0 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:07.438 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:07.426+0000 7f38d1200640 1 --2- 172.21.15.83:0/1876331344 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f38cc1042a0 0x7f38cc19a710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:07.438 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:07.426+0000 7f38d1200640 1 --2- 172.21.15.83:0/1876331344 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f38cc103900 0x7f38cc19a1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:07.438 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:07.426+0000 7f38d1200640 1 -- 172.21.15.83:0/1876331344 >> 172.21.15.83:0/1876331344 conn(0x7f38cc0ff600 msgr2=0x7f38cc100dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:34:07.438 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:07.426+0000 7f38d1200640 1 -- 172.21.15.83:0/1876331344 shutdown_connections 2024-08-19T01:34:07.438 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:07.427+0000 7f38d1200640 1 -- 172.21.15.83:0/1876331344 wait complete. 2024-08-19T01:34:07.654 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:34:07 smithi083 ceph-mon[17120]: pgmap v317: 97 pgs: 97 active+clean; 640 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 426 B/s wr, 7 op/s 2024-08-19T01:34:07.952 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:34:07 smithi117 ceph-mon[22016]: pgmap v317: 97 pgs: 97 active+clean; 640 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 426 B/s wr, 7 op/s 2024-08-19T01:34:07.966 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:34:07 smithi167 ceph-mon[21875]: pgmap v317: 97 pgs: 97 active+clean; 640 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 426 B/s wr, 7 op/s 2024-08-19T01:34:08.207 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833 shell --fsid bad8780e-5dc9-11ef-bccf-c7b262605968 -- ceph tell mds.b status 2024-08-19T01:34:08.580 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/bad8780e-5dc9-11ef-bccf-c7b262605968/mon.a/config 2024-08-19T01:34:09.158 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:09.156+0000 7f0d7d000640 1 -- 172.21.15.83:0/2794656073 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0d78102250 msgr2=0x7f0d7810e780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:34:09.158 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:09.156+0000 7f0d7d000640 1 --2- 172.21.15.83:0/2794656073 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0d78102250 0x7f0d7810e780 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f0d70009cd0 tx=0x7f0d7002a8e0 comp rx=0 tx=0).stop 2024-08-19T01:34:09.158 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:09.156+0000 7f0d7d000640 1 -- 172.21.15.83:0/2794656073 shutdown_connections 2024-08-19T01:34:09.158 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:09.156+0000 7f0d7d000640 1 --2- 172.21.15.83:0/2794656073 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0d78102250 0x7f0d7810e780 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:09.158 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:09.156+0000 7f0d7d000640 1 --2- 172.21.15.83:0/2794656073 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0d781018b0 0x7f0d78101d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:09.158 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:09.156+0000 7f0d7d000640 1 --2- 172.21.15.83:0/2794656073 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0d781078b0 0x7f0d78107c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:09.158 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:09.156+0000 7f0d7d000640 1 -- 172.21.15.83:0/2794656073 >> 172.21.15.83:0/2794656073 conn(0x7f0d780fd5f0 msgr2=0x7f0d780ffa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:34:09.158 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:09.156+0000 7f0d7d000640 1 -- 172.21.15.83:0/2794656073 shutdown_connections 2024-08-19T01:34:09.158 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:09.156+0000 7f0d7d000640 1 -- 172.21.15.83:0/2794656073 wait complete. 2024-08-19T01:34:09.160 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:09.157+0000 7f0d7d000640 1 Processor -- start 2024-08-19T01:34:09.160 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:09.157+0000 7f0d7d000640 1 -- start start 2024-08-19T01:34:09.160 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:09.157+0000 7f0d7d000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0d781018b0 0x7f0d78074220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:34:09.160 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:09.157+0000 7f0d7d000640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0d78102250 0x7f0d78074760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:34:09.160 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:09.157+0000 7f0d7d000640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0d781078b0 0x7f0d78074ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:34:09.161 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:09.157+0000 7f0d7d000640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f0d78111d00 con 0x7f0d781018b0 2024-08-19T01:34:09.161 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:09.157+0000 7f0d7d000640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f0d78111b80 con 0x7f0d78102250 2024-08-19T01:34:09.161 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:09.157+0000 7f0d7d000640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f0d78111e80 con 0x7f0d781078b0 2024-08-19T01:34:09.161 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:09.158+0000 7f0d75e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0d781018b0 0x7f0d78074220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:34:09.161 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:09.158+0000 7f0d75e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0d781018b0 0x7f0d78074220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:53038/0 (socket says 172.21.15.83:53038) 2024-08-19T01:34:09.161 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:09.158+0000 7f0d75e00640 1 -- 172.21.15.83:0/1155008954 learned_addr learned my addr 172.21.15.83:0/1155008954 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-08-19T01:34:09.161 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:09.158+0000 7f0d75400640 1 --2- 172.21.15.83:0/1155008954 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0d78102250 0x7f0d78074760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:34:09.161 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:09.158+0000 7f0d76800640 1 --2- 172.21.15.83:0/1155008954 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0d781078b0 0x7f0d78074ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:34:09.161 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:09.158+0000 7f0d75e00640 1 -- 172.21.15.83:0/1155008954 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0d781078b0 msgr2=0x7f0d78074ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:34:09.161 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:09.158+0000 7f0d75e00640 1 --2- 172.21.15.83:0/1155008954 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0d781078b0 0x7f0d78074ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:09.161 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:09.158+0000 7f0d75e00640 1 -- 172.21.15.83:0/1155008954 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0d78102250 msgr2=0x7f0d78074760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:34:09.161 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:09.158+0000 7f0d75e00640 1 --2- 172.21.15.83:0/1155008954 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0d78102250 0x7f0d78074760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:09.161 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:09.158+0000 7f0d75e00640 1 -- 172.21.15.83:0/1155008954 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0d781a6620 con 0x7f0d781018b0 2024-08-19T01:34:09.161 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:09.158+0000 7f0d76800640 1 --2- 172.21.15.83:0/1155008954 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0d781078b0 0x7f0d78074ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-19T01:34:09.161 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:09.158+0000 7f0d75e00640 1 --2- 172.21.15.83:0/1155008954 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0d781018b0 0x7f0d78074220 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f0d6800e9f0 tx=0x7f0d6800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:34:09.162 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:09.158+0000 7f0d6ea00640 1 -- 172.21.15.83:0/1155008954 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d6800cd80 con 0x7f0d781018b0 2024-08-19T01:34:09.162 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:09.159+0000 7f0d6ea00640 1 -- 172.21.15.83:0/1155008954 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0d68010910 con 0x7f0d781018b0 2024-08-19T01:34:09.162 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:09.159+0000 7f0d7d000640 1 -- 172.21.15.83:0/1155008954 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0d781a6910 con 0x7f0d781018b0 2024-08-19T01:34:09.162 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:09.159+0000 7f0d6ea00640 1 -- 172.21.15.83:0/1155008954 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d68018990 con 0x7f0d781018b0 2024-08-19T01:34:09.162 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:09.159+0000 7f0d7d000640 1 -- 172.21.15.83:0/1155008954 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0d781a6e50 con 0x7f0d781018b0 2024-08-19T01:34:09.163 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:09.162+0000 7f0d6ea00640 1 -- 172.21.15.83:0/1155008954 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f0d68010ac0 con 0x7f0d781018b0 2024-08-19T01:34:09.163 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:09.162+0000 7f0d7fa0d740 1 Processor -- start 2024-08-19T01:34:09.163 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:09.162+0000 7f0d7fa0d740 1 -- start start 2024-08-19T01:34:09.164 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:09.162+0000 7f0d7fa0d740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5612f8dac880 0x5612f8daadd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:34:09.164 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:09.162+0000 7f0d7fa0d740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5612f8dab310 0x5612f8dab750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:34:09.164 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:09.162+0000 7f0d7fa0d740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5612f8db60b0 0x5612f8db84a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:34:09.164 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:09.162+0000 7f0d7fa0d740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x5612f8d11940 con 0x5612f8db60b0 2024-08-19T01:34:09.164 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:09.162+0000 7f0d7fa0d740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x5612f8ca99a0 con 0x5612f8dac880 2024-08-19T01:34:09.164 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:09.162+0000 7f0d7fa0d740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x5612f8d0d900 con 0x5612f8dab310 2024-08-19T01:34:09.165 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:09.163+0000 7f0d76800640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5612f8dac880 0x5612f8daadd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:34:09.165 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:09.163+0000 7f0d75400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5612f8db60b0 0x5612f8db84a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:34:09.165 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:09.163+0000 7f0d76800640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5612f8dac880 0x5612f8daadd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.83:53834/0 (socket says 172.21.15.83:53834) 2024-08-19T01:34:09.165 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:09.163+0000 7f0d76800640 1 -- 172.21.15.83:0/2150724506 learned_addr learned my addr 172.21.15.83:0/2150724506 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-08-19T01:34:09.166 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:09.163+0000 7f0d75e00640 1 --2- 172.21.15.83:0/2150724506 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5612f8dab310 0x5612f8dab750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:34:09.166 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:09.163+0000 7f0d76800640 1 -- 172.21.15.83:0/2150724506 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5612f8dab310 msgr2=0x5612f8dab750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:34:09.166 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:09.163+0000 7f0d76800640 1 --2- 172.21.15.83:0/2150724506 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5612f8dab310 0x5612f8dab750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:09.166 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:09.163+0000 7f0d76800640 1 -- 172.21.15.83:0/2150724506 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5612f8db60b0 msgr2=0x5612f8db84a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:34:09.166 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:09.163+0000 7f0d76800640 1 --2- 172.21.15.83:0/2150724506 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5612f8db60b0 0x5612f8db84a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:09.166 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:09.163+0000 7f0d76800640 1 -- 172.21.15.83:0/2150724506 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5612f8c97ef0 con 0x5612f8dac880 2024-08-19T01:34:09.166 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:09.163+0000 7f0d6ea00640 1 --2- 172.21.15.83:0/1155008954 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f0d5c0a2b80 0x7f0d5c0a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:34:09.166 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:09.164+0000 7f0d76800640 1 --2- 172.21.15.83:0/2150724506 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5612f8dac880 0x5612f8daadd0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f0d70009cd0 tx=0x7f0d70009a20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=864fa0de12586f7c server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:34:09.166 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:09.164+0000 7f0d6ea00640 1 -- 172.21.15.83:0/1155008954 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8269+0+0 (secure 0 0 0) 0x7f0d680cf400 con 0x7f0d781018b0 2024-08-19T01:34:09.166 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:09.164+0000 7f0d75e00640 1 --2- 172.21.15.83:0/1155008954 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f0d5c0a2b80 0x7f0d5c0a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:34:09.166 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:09.164+0000 7f0d63400640 1 -- 172.21.15.83:0/2150724506 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d700318c0 con 0x5612f8dac880 2024-08-19T01:34:09.167 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:09.164+0000 7f0d63400640 1 -- 172.21.15.83:0/2150724506 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0d70036040 con 0x5612f8dac880 2024-08-19T01:34:09.167 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:09.164+0000 7f0d63400640 1 -- 172.21.15.83:0/2150724506 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d700045a0 con 0x5612f8dac880 2024-08-19T01:34:09.168 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:09.165+0000 7f0d7fa0d740 1 -- 172.21.15.83:0/2150724506 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5612f8dac880 msgr2=0x5612f8daadd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:34:09.168 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:09.165+0000 7f0d75e00640 1 --2- 172.21.15.83:0/1155008954 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f0d5c0a2b80 0x7f0d5c0a5040 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f0d68098040 tx=0x7f0d68000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:34:09.168 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:09.165+0000 7f0d7fa0d740 1 --2- 172.21.15.83:0/2150724506 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5612f8dac880 0x5612f8daadd0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f0d70009cd0 tx=0x7f0d70009a20 comp rx=0 tx=0).stop 2024-08-19T01:34:09.168 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:09.165+0000 7f0d7fa0d740 1 -- 172.21.15.83:0/2150724506 shutdown_connections 2024-08-19T01:34:09.168 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:09.165+0000 7f0d7fa0d740 1 --2- 172.21.15.83:0/2150724506 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5612f8db60b0 0x5612f8db84a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:09.168 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:09.165+0000 7f0d7fa0d740 1 --2- 172.21.15.83:0/2150724506 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5612f8dab310 0x5612f8dab750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:09.168 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:09.165+0000 7f0d7fa0d740 1 --2- 172.21.15.83:0/2150724506 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5612f8dac880 0x5612f8daadd0 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:09.169 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:09.165+0000 7f0d7fa0d740 1 -- 172.21.15.83:0/2150724506 >> 172.21.15.83:0/2150724506 conn(0x5612f8d73a00 msgr2=0x5612f8da79a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:34:09.169 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:09.165+0000 7f0d7fa0d740 1 -- 172.21.15.83:0/2150724506 shutdown_connections 2024-08-19T01:34:09.169 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:09.165+0000 7f0d7fa0d740 1 -- 172.21.15.83:0/2150724506 wait complete. 2024-08-19T01:34:09.169 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:09.165+0000 7f0d7fa0d740 1 -- start start 2024-08-19T01:34:09.169 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:09.166+0000 7f0d7fa0d740 1 Processor -- start 2024-08-19T01:34:09.169 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:09.166+0000 7f0d7fa0d740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5612f8dab310 0x5612f8db3020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:34:09.169 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:09.166+0000 7f0d7fa0d740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5612f8dac880 0x5612f8db3560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:34:09.169 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:09.166+0000 7f0d7fa0d740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5612f8db60b0 0x5612f8db3aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:34:09.169 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:09.166+0000 7f0d7fa0d740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x5612f8db8d50 con 0x5612f8dab310 2024-08-19T01:34:09.169 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:09.166+0000 7f0d7fa0d740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x5612f8ccca80 con 0x5612f8db60b0 2024-08-19T01:34:09.169 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:09.166+0000 7f0d7fa0d740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x5612f8db8ed0 con 0x5612f8dac880 2024-08-19T01:34:09.169 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:09.166+0000 7f0d7fa0d740 10 client.0 _ll_register_callbacks cb 0x5612f8c95140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-19T01:34:09.169 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:09.167+0000 7f0d76800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5612f8dab310 0x5612f8db3020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:34:09.169 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:09.167+0000 7f0d76800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5612f8dab310 0x5612f8db3020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:53050/0 (socket says 172.21.15.83:53050) 2024-08-19T01:34:09.169 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:09.167+0000 7f0d76800640 1 -- 172.21.15.83:0/2237823369 learned_addr learned my addr 172.21.15.83:0/2237823369 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-08-19T01:34:09.169 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:09.167+0000 7f0d75400640 1 --2- 172.21.15.83:0/2237823369 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5612f8dac880 0x5612f8db3560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:34:09.170 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:09.167+0000 7f0d76800640 1 --2- 172.21.15.83:0/2237823369 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5612f8db60b0 0x5612f8db3aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:34:09.171 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:09.167+0000 7f0d76800640 1 -- 172.21.15.83:0/2237823369 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5612f8dac880 msgr2=0x5612f8db3560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:34:09.171 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:09.167+0000 7f0d76800640 1 --2- 172.21.15.83:0/2237823369 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5612f8dac880 0x5612f8db3560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:09.171 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:09.167+0000 7f0d76800640 1 -- 172.21.15.83:0/2237823369 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5612f8db60b0 msgr2=0x5612f8db3aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:34:09.171 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:09.167+0000 7f0d76800640 1 --2- 172.21.15.83:0/2237823369 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5612f8db60b0 0x5612f8db3aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:09.171 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:09.167+0000 7f0d76800640 1 -- 172.21.15.83:0/2237823369 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5612f8cb0700 con 0x5612f8dab310 2024-08-19T01:34:09.171 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:09.167+0000 7f0d76800640 1 --2- 172.21.15.83:0/2237823369 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5612f8dab310 0x5612f8db3020 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f0d70006010 tx=0x7f0d700040e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=49e753a83cae78f7 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:34:09.172 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:09.167+0000 7f0d57e00640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-08-19T01:34:09.172 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:09.168+0000 7f0d57e00640 1 -- 172.21.15.83:0/2237823369 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d70053060 con 0x5612f8dab310 2024-08-19T01:34:09.172 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:09.168+0000 7f0d57e00640 1 -- 172.21.15.83:0/2237823369 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8269+0+0 (secure 0 0 0) 0x7f0d70004730 con 0x5612f8dab310 2024-08-19T01:34:09.172 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:09.168+0000 7f0d7fa0d740 1 -- 172.21.15.83:0/2237823369 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0d70036040 con 0x5612f8dab310 2024-08-19T01:34:09.172 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:09.168+0000 7f0d57e00640 1 -- 172.21.15.83:0/2237823369 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f0d70003970 con 0x5612f8dab310 2024-08-19T01:34:09.172 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:09.168+0000 7f0d7fa0d740 10 client.14859 fetch_fsmap learned FSMap version 24 2024-08-19T01:34:09.172 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:09.168+0000 7f0d7fa0d740 1 -- 172.21.15.83:0/2237823369 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x5612f8a832b0 con 0x5612f8dab310 2024-08-19T01:34:09.172 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:09.169+0000 7f0d57e00640 1 -- 172.21.15.83:0/2237823369 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0d70036040 con 0x5612f8dab310 2024-08-19T01:34:09.172 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:09.169+0000 7f0d57e00640 1 -- 172.21.15.83:0/2237823369 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 24) ==== 8346+0+0 (secure 0 0 0) 0x7f0d70030070 con 0x5612f8dab310 2024-08-19T01:34:09.172 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:09.169+0000 7f0d7fa0d740 10 client.14859 fetch_fsmap finished waiting for FSMap version 24 2024-08-19T01:34:09.172 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:09.169+0000 7f0d7fa0d740 10 client.14859 resolve_mds: resolved name 'b' to daemon mds.b 2024-08-19T01:34:09.172 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:09.169+0000 7f0d7fa0d740 20 client.14859 populate_metadata read hostname 'smithi083' 2024-08-19T01:34:09.172 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:09.169+0000 7f0d7fa0d740 1 --2- 172.21.15.83:0/2237823369 >> [v2:172.21.15.117:6838/1967595704,v1:172.21.15.117:6839/1967595704] conn(0x5612f8dc5bd0 0x5612f8dc7d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:34:09.172 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:09.169+0000 7f0d7fa0d740 4 client.14859 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-19T01:34:09.173 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:09.169+0000 7f0d7fa0d740 1 -- 172.21.15.83:0/2237823369 --> [v2:172.21.15.117:6838/1967595704,v1:172.21.15.117:6839/1967595704] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5612f8d1eeb0 con 0x5612f8dc5bd0 2024-08-19T01:34:09.173 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:09.169+0000 7f0d75400640 1 --2- 172.21.15.83:0/2237823369 >> [v2:172.21.15.117:6838/1967595704,v1:172.21.15.117:6839/1967595704] conn(0x5612f8dc5bd0 0x5612f8dc7d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:34:09.173 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:09.170+0000 7f0d75400640 1 --2- 172.21.15.83:0/2237823369 >> [v2:172.21.15.117:6838/1967595704,v1:172.21.15.117:6839/1967595704] conn(0x5612f8dc5bd0 0x5612f8dc7d80 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=52d3cc5252c8a2e8 server_cookie=cce1daf7cc5954b0 in_seq=0 out_seq=0 2024-08-19T01:34:09.173 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:09.170+0000 7f0d57e00640 10 client.14859 ms_handle_connect on v2:172.21.15.117:6838/1967595704 2024-08-19T01:34:09.174 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:09.171+0000 7f0d57e00640 1 -- 172.21.15.83:0/2237823369 <== mds.0 v2:172.21.15.117:6838/1967595704 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5612f8d1eeb0 con 0x5612f8dc5bd0 2024-08-19T01:34:09.174 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:09.171+0000 7f0d57e00640 10 client.14859 handle_command_reply: tid=0 2024-08-19T01:34:09.174 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:09.171+0000 7f0d7fa0d740 1 client.14859 shutdown 2024-08-19T01:34:09.174 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:09.171+0000 7f0d7fa0d740 1 -- 172.21.15.83:0/2237823369 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5612f8dab310 msgr2=0x5612f8db3020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:34:09.174 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:09.171+0000 7f0d7fa0d740 1 --2- 172.21.15.83:0/2237823369 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5612f8dab310 0x5612f8db3020 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f0d70006010 tx=0x7f0d700040e0 comp rx=0 tx=0).stop 2024-08-19T01:34:09.174 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:09.171+0000 7f0d7fa0d740 1 -- 172.21.15.83:0/2237823369 shutdown_connections 2024-08-19T01:34:09.174 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:09.171+0000 7f0d7fa0d740 1 --2- 172.21.15.83:0/2237823369 >> [v2:172.21.15.117:6838/1967595704,v1:172.21.15.117:6839/1967595704] conn(0x5612f8dc5bd0 0x5612f8dc7d80 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:09.174 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:09.171+0000 7f0d7fa0d740 1 --2- 172.21.15.83:0/2237823369 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5612f8db60b0 0x5612f8db3aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:09.174 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:09.171+0000 7f0d7fa0d740 1 --2- 172.21.15.83:0/2237823369 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5612f8dac880 0x5612f8db3560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:09.175 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:09.171+0000 7f0d7fa0d740 1 --2- 172.21.15.83:0/2237823369 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5612f8dab310 0x5612f8db3020 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:09.175 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:09.171+0000 7f0d7fa0d740 1 -- 172.21.15.83:0/2237823369 >> 172.21.15.83:0/2237823369 conn(0x5612f8d73a00 msgr2=0x5612f8dad6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:34:09.175 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:09.171+0000 7f0d57e00640 0 client.14859 ms_handle_reset on v2:172.21.15.117:6838/1967595704 2024-08-19T01:34:09.175 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:09.171+0000 7f0d7fa0d740 1 -- 172.21.15.83:0/2237823369 shutdown_connections 2024-08-19T01:34:09.175 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:09.171+0000 7f0d7fa0d740 1 -- 172.21.15.83:0/2237823369 wait complete. 2024-08-19T01:34:09.175 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:09.171+0000 7f0d7fa0d740 20 client.14859 trim_cache size 0 max 16384 2024-08-19T01:34:09.192 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:09.190+0000 7f0d7fa0d740 1 Processor -- start 2024-08-19T01:34:09.192 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:09.190+0000 7f0d7fa0d740 1 -- start start 2024-08-19T01:34:09.192 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:09.190+0000 7f0d7fa0d740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5612f8dab310 0x5612f8dc6910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:34:09.192 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:09.190+0000 7f0d7fa0d740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5612f8dac880 0x5612f8dac040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:34:09.192 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:09.190+0000 7f0d7fa0d740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5612f8db60b0 0x5612f8db18f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:34:09.192 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:09.190+0000 7f0d7fa0d740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x5612f8dc60c0 con 0x5612f8dac880 2024-08-19T01:34:09.192 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:09.190+0000 7f0d7fa0d740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x5612f8dc6790 con 0x5612f8db60b0 2024-08-19T01:34:09.192 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:09.190+0000 7f0d7fa0d740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x5612f8dc6e60 con 0x5612f8dab310 2024-08-19T01:34:09.193 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:09.190+0000 7f0d75400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5612f8dac880 0x5612f8dac040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:34:09.193 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:09.190+0000 7f0d75400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5612f8dac880 0x5612f8dac040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:53052/0 (socket says 172.21.15.83:53052) 2024-08-19T01:34:09.193 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:09.190+0000 7f0d75400640 1 -- 172.21.15.83:0/413962937 learned_addr learned my addr 172.21.15.83:0/413962937 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-08-19T01:34:09.193 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:09.190+0000 7f0d76800640 1 --2- 172.21.15.83:0/413962937 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5612f8dab310 0x5612f8dc6910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:34:09.193 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:09.190+0000 7f0d76800640 1 --2- 172.21.15.83:0/413962937 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5612f8db60b0 0x5612f8db18f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:34:09.193 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:09.191+0000 7f0d75400640 1 -- 172.21.15.83:0/413962937 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5612f8dab310 msgr2=0x5612f8dc6910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:34:09.193 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:09.191+0000 7f0d75400640 1 --2- 172.21.15.83:0/413962937 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5612f8dab310 0x5612f8dc6910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:09.193 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:09.191+0000 7f0d75400640 1 -- 172.21.15.83:0/413962937 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5612f8db60b0 msgr2=0x5612f8db18f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:34:09.193 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:09.191+0000 7f0d75400640 1 --2- 172.21.15.83:0/413962937 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5612f8db60b0 0x5612f8db18f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:09.193 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:09.191+0000 7f0d75400640 1 -- 172.21.15.83:0/413962937 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5612f8dade60 con 0x5612f8dac880 2024-08-19T01:34:09.193 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:09.191+0000 7f0d76800640 1 --2- 172.21.15.83:0/413962937 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5612f8dab310 0x5612f8dc6910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-19T01:34:09.196 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:09.191+0000 7f0d75400640 1 --2- 172.21.15.83:0/413962937 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5612f8dac880 0x5612f8dac040 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f0d64000c00 tx=0x7f0d6400b410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eb955b2f06d92c5f server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:34:09.196 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:09.191+0000 7f0d62a00640 1 -- 172.21.15.83:0/413962937 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d64004220 con 0x5612f8dac880 2024-08-19T01:34:09.196 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:09.191+0000 7f0d62a00640 1 -- 172.21.15.83:0/413962937 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0d640043c0 con 0x5612f8dac880 2024-08-19T01:34:09.196 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:09.191+0000 7f0d62a00640 1 -- 172.21.15.83:0/413962937 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d6401db10 con 0x5612f8dac880 2024-08-19T01:34:09.196 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:09.192+0000 7f0d7fa0d740 1 -- 172.21.15.83:0/413962937 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5612f8dac880 msgr2=0x5612f8dac040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:34:09.196 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:09.192+0000 7f0d7fa0d740 1 --2- 172.21.15.83:0/413962937 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5612f8dac880 0x5612f8dac040 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f0d64000c00 tx=0x7f0d6400b410 comp rx=0 tx=0).stop 2024-08-19T01:34:09.196 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:09.192+0000 7f0d7fa0d740 1 -- 172.21.15.83:0/413962937 shutdown_connections 2024-08-19T01:34:09.196 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:09.192+0000 7f0d7fa0d740 1 --2- 172.21.15.83:0/413962937 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5612f8db60b0 0x5612f8db18f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:09.196 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:09.192+0000 7f0d7fa0d740 1 --2- 172.21.15.83:0/413962937 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5612f8dac880 0x5612f8dac040 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:09.196 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:09.192+0000 7f0d7fa0d740 1 --2- 172.21.15.83:0/413962937 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5612f8dab310 0x5612f8dc6910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:09.196 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:09.192+0000 7f0d7fa0d740 1 -- 172.21.15.83:0/413962937 >> 172.21.15.83:0/413962937 conn(0x5612f8d73a00 msgr2=0x5612f8da2660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:34:09.197 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:09.192+0000 7f0d7fa0d740 1 -- 172.21.15.83:0/413962937 shutdown_connections 2024-08-19T01:34:09.197 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:09.192+0000 7f0d7fa0d740 1 -- 172.21.15.83:0/413962937 wait complete. 2024-08-19T01:34:09.197 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:09.192+0000 7f0d7fa0d740 1 -- start start 2024-08-19T01:34:09.197 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:09.193+0000 7f0d7fa0d740 1 Processor -- start 2024-08-19T01:34:09.197 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:09.193+0000 7f0d7fa0d740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5612f8dab310 0x5612f8dd0530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:34:09.197 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:09.193+0000 7f0d7fa0d740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5612f8dac880 0x5612f8dc9860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:34:09.197 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:09.193+0000 7f0d7fa0d740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5612f8db60b0 0x5612f8dc9da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:34:09.197 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:09.193+0000 7f0d7fa0d740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x5612f8dc7c00 con 0x5612f8dab310 2024-08-19T01:34:09.197 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:09.193+0000 7f0d7fa0d740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x5612f8dc7530 con 0x5612f8db60b0 2024-08-19T01:34:09.197 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:09.193+0000 7f0d7fa0d740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x5612f8dc82d0 con 0x5612f8dac880 2024-08-19T01:34:09.197 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:09.193+0000 7f0d7fa0d740 10 client.0 _ll_register_callbacks cb 0x7f0d6400b810 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-19T01:34:09.197 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:09.193+0000 7f0d76800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5612f8dab310 0x5612f8dd0530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:34:09.197 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:09.193+0000 7f0d76800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5612f8dab310 0x5612f8dd0530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:53064/0 (socket says 172.21.15.83:53064) 2024-08-19T01:34:09.197 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:09.193+0000 7f0d76800640 1 -- 172.21.15.83:0/1788067955 learned_addr learned my addr 172.21.15.83:0/1788067955 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-08-19T01:34:09.198 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:09.193+0000 7f0d75400640 1 --2- 172.21.15.83:0/1788067955 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5612f8dac880 0x5612f8dc9860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:34:09.198 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:09.194+0000 7f0d76800640 1 --2- 172.21.15.83:0/1788067955 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5612f8db60b0 0x5612f8dc9da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:34:09.198 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:09.194+0000 7f0d75400640 1 -- 172.21.15.83:0/1788067955 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5612f8db60b0 msgr2=0x5612f8dc9da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:34:09.198 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:09.194+0000 7f0d75400640 1 --2- 172.21.15.83:0/1788067955 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5612f8db60b0 0x5612f8dc9da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:09.198 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:09.194+0000 7f0d75400640 1 -- 172.21.15.83:0/1788067955 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5612f8dab310 msgr2=0x5612f8dd0530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:34:09.198 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:09.194+0000 7f0d75400640 1 --2- 172.21.15.83:0/1788067955 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5612f8dab310 0x5612f8dd0530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:09.198 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:09.194+0000 7f0d75400640 1 -- 172.21.15.83:0/1788067955 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5612f8dca310 con 0x5612f8dac880 2024-08-19T01:34:09.198 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:09.194+0000 7f0d76800640 1 --2- 172.21.15.83:0/1788067955 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5612f8dab310 0x5612f8dd0530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-19T01:34:09.198 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:09.194+0000 7f0d75400640 1 --2- 172.21.15.83:0/1788067955 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5612f8dac880 0x5612f8dc9860 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x5612f8db5e00 tx=0x7f0d6400bb40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=112a3fa214c69e16 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:34:09.198 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:09.194+0000 7f0d57e00640 10 client.0 ms_handle_connect on v2:172.21.15.167:3300/0 2024-08-19T01:34:09.198 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:09.194+0000 7f0d57e00640 1 -- 172.21.15.83:0/1788067955 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d6401da90 con 0x5612f8dac880 2024-08-19T01:34:09.198 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:09.194+0000 7f0d7fa0d740 1 -- 172.21.15.83:0/1788067955 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0d640043c0 con 0x5612f8dac880 2024-08-19T01:34:09.198 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:09.194+0000 7f0d57e00640 1 -- 172.21.15.83:0/1788067955 <== mon.1 v2:172.21.15.167:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8269+0+0 (secure 0 0 0) 0x7f0d6401dc30 con 0x5612f8dac880 2024-08-19T01:34:09.200 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:09.195+0000 7f0d57e00640 1 -- 172.21.15.83:0/1788067955 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f0d6401de30 con 0x5612f8dac880 2024-08-19T01:34:09.200 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:09.195+0000 7f0d7fa0d740 10 client.24907 fetch_fsmap learned FSMap version 24 2024-08-19T01:34:09.200 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:09.195+0000 7f0d7fa0d740 1 -- 172.21.15.83:0/1788067955 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x5612f8dcedb0 con 0x5612f8dac880 2024-08-19T01:34:09.200 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:09.195+0000 7f0d57e00640 1 -- 172.21.15.83:0/1788067955 <== mon.1 v2:172.21.15.167:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0d640043c0 con 0x5612f8dac880 2024-08-19T01:34:09.200 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:09.195+0000 7f0d57e00640 1 -- 172.21.15.83:0/1788067955 <== mon.1 v2:172.21.15.167:3300/0 5 ==== fsmap(e 24) ==== 8346+0+0 (secure 0 0 0) 0x7f0d64004cb0 con 0x5612f8dac880 2024-08-19T01:34:09.200 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:09.195+0000 7f0d7fa0d740 10 client.24907 fetch_fsmap finished waiting for FSMap version 24 2024-08-19T01:34:09.201 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:09.195+0000 7f0d7fa0d740 10 client.24907 resolve_mds: resolved name 'b' to daemon mds.b 2024-08-19T01:34:09.201 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:09.195+0000 7f0d7fa0d740 20 client.24907 populate_metadata read hostname 'smithi083' 2024-08-19T01:34:09.201 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:09.195+0000 7f0d7fa0d740 1 --2- 172.21.15.83:0/1788067955 >> [v2:172.21.15.117:6838/1967595704,v1:172.21.15.117:6839/1967595704] conn(0x5612f8e35500 0x5612f8e34210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:34:09.201 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:09.195+0000 7f0d7fa0d740 4 client.24907 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "status"}] 2024-08-19T01:34:09.201 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:09.196+0000 7f0d7fa0d740 1 -- 172.21.15.83:0/1788067955 --> [v2:172.21.15.117:6838/1967595704,v1:172.21.15.117:6839/1967595704] -- command(tid 0: {"prefix": "status"}) -- 0x5612f8d193c0 con 0x5612f8e35500 2024-08-19T01:34:09.201 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:09.196+0000 7f0d76800640 1 --2- 172.21.15.83:0/1788067955 >> [v2:172.21.15.117:6838/1967595704,v1:172.21.15.117:6839/1967595704] conn(0x5612f8e35500 0x5612f8e34210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:34:09.201 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:09.196+0000 7f0d76800640 1 --2- 172.21.15.83:0/1788067955 >> [v2:172.21.15.117:6838/1967595704,v1:172.21.15.117:6839/1967595704] conn(0x5612f8e35500 0x5612f8e34210 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5ab3a68026234648 server_cookie=a7b7c43b848be4ed in_seq=0 out_seq=0 2024-08-19T01:34:09.201 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:09.196+0000 7f0d57e00640 10 client.24907 ms_handle_connect on v2:172.21.15.117:6838/1967595704 2024-08-19T01:34:09.201 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:09.197+0000 7f0d57e00640 1 -- 172.21.15.83:0/1788067955 <== mds.0 v2:172.21.15.117:6838/1967595704 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x5612f8d193c0 con 0x5612f8e35500 2024-08-19T01:34:09.201 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:09.197+0000 7f0d57e00640 10 client.24907 handle_command_reply: tid=0 2024-08-19T01:34:09.201 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:09.197+0000 7f0d7fa0d740 1 client.24907 shutdown 2024-08-19T01:34:09.201 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:09.197+0000 7f0d7fa0d740 1 -- 172.21.15.83:0/1788067955 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5612f8dac880 msgr2=0x5612f8dc9860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:34:09.201 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:09.197+0000 7f0d7fa0d740 1 --2- 172.21.15.83:0/1788067955 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5612f8dac880 0x5612f8dc9860 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x5612f8db5e00 tx=0x7f0d6400bb40 comp rx=0 tx=0).stop 2024-08-19T01:34:09.201 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:09.197+0000 7f0d7fa0d740 1 -- 172.21.15.83:0/1788067955 shutdown_connections 2024-08-19T01:34:09.201 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:09.197+0000 7f0d7fa0d740 1 --2- 172.21.15.83:0/1788067955 >> [v2:172.21.15.117:6838/1967595704,v1:172.21.15.117:6839/1967595704] conn(0x5612f8e35500 0x5612f8e34210 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:09.202 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:09.197+0000 7f0d7fa0d740 1 --2- 172.21.15.83:0/1788067955 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5612f8db60b0 0x5612f8dc9da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:09.202 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:09.197+0000 7f0d57e00640 0 client.24907 ms_handle_reset on v2:172.21.15.117:6838/1967595704 2024-08-19T01:34:09.202 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:09.197+0000 7f0d7fa0d740 1 --2- 172.21.15.83:0/1788067955 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5612f8dac880 0x5612f8dc9860 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:09.202 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:09.197+0000 7f0d7fa0d740 1 --2- 172.21.15.83:0/1788067955 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5612f8dab310 0x5612f8dd0530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:09.202 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:09.197+0000 7f0d7fa0d740 1 -- 172.21.15.83:0/1788067955 >> 172.21.15.83:0/1788067955 conn(0x5612f8d73a00 msgr2=0x5612f8db1820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:34:09.202 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:09.197+0000 7f0d7fa0d740 1 -- 172.21.15.83:0/1788067955 shutdown_connections 2024-08-19T01:34:09.202 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:09.197+0000 7f0d7fa0d740 1 -- 172.21.15.83:0/1788067955 wait complete. 2024-08-19T01:34:09.202 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:09.197+0000 7f0d7fa0d740 20 client.24907 trim_cache size 0 max 16384 2024-08-19T01:34:09.202 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-08-19T01:34:09.202 INFO:teuthology.orchestra.run.smithi083.stdout: "cluster_fsid": "bad8780e-5dc9-11ef-bccf-c7b262605968", 2024-08-19T01:34:09.202 INFO:teuthology.orchestra.run.smithi083.stdout: "whoami": 0, 2024-08-19T01:34:09.202 INFO:teuthology.orchestra.run.smithi083.stdout: "id": 24545, 2024-08-19T01:34:09.203 INFO:teuthology.orchestra.run.smithi083.stdout: "want_state": "up:active", 2024-08-19T01:34:09.203 INFO:teuthology.orchestra.run.smithi083.stdout: "state": "up:active", 2024-08-19T01:34:09.203 INFO:teuthology.orchestra.run.smithi083.stdout: "fs_name": "cephfs", 2024-08-19T01:34:09.203 INFO:teuthology.orchestra.run.smithi083.stdout: "rank_uptime": 86.980836115000002, 2024-08-19T01:34:09.203 INFO:teuthology.orchestra.run.smithi083.stdout: "mdsmap_epoch": 24, 2024-08-19T01:34:09.203 INFO:teuthology.orchestra.run.smithi083.stdout: "osdmap_epoch": 86, 2024-08-19T01:34:09.203 INFO:teuthology.orchestra.run.smithi083.stdout: "osdmap_epoch_barrier": 84, 2024-08-19T01:34:09.203 INFO:teuthology.orchestra.run.smithi083.stdout: "uptime": 99.125555716999997 2024-08-19T01:34:09.203 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-08-19T01:34:09.204 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:09.198+0000 7f0d7d000640 1 -- 172.21.15.83:0/1155008954 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f0d5c0a2b80 msgr2=0x7f0d5c0a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:34:09.204 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:09.198+0000 7f0d7d000640 1 --2- 172.21.15.83:0/1155008954 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f0d5c0a2b80 0x7f0d5c0a5040 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f0d68098040 tx=0x7f0d68000bc0 comp rx=0 tx=0).stop 2024-08-19T01:34:09.204 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:09.198+0000 7f0d7d000640 1 -- 172.21.15.83:0/1155008954 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0d781018b0 msgr2=0x7f0d78074220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:34:09.204 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:09.198+0000 7f0d7d000640 1 --2- 172.21.15.83:0/1155008954 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0d781018b0 0x7f0d78074220 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f0d6800e9f0 tx=0x7f0d6800eeb0 comp rx=0 tx=0).stop 2024-08-19T01:34:09.204 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:09.198+0000 7f0d7d000640 1 -- 172.21.15.83:0/1155008954 shutdown_connections 2024-08-19T01:34:09.204 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:09.198+0000 7f0d7d000640 1 --2- 172.21.15.83:0/1155008954 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f0d5c0a2b80 0x7f0d5c0a5040 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:09.204 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:09.198+0000 7f0d7d000640 1 --2- 172.21.15.83:0/1155008954 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0d781078b0 0x7f0d78074ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:09.204 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:09.198+0000 7f0d7d000640 1 --2- 172.21.15.83:0/1155008954 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0d78102250 0x7f0d78074760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:09.204 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:09.198+0000 7f0d7d000640 1 --2- 172.21.15.83:0/1155008954 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0d781018b0 0x7f0d78074220 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:09.204 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:09.198+0000 7f0d7d000640 1 -- 172.21.15.83:0/1155008954 >> 172.21.15.83:0/1155008954 conn(0x7f0d780fd5f0 msgr2=0x7f0d780fde30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:34:09.204 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:09.198+0000 7f0d7d000640 1 -- 172.21.15.83:0/1155008954 shutdown_connections 2024-08-19T01:34:09.204 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:09.198+0000 7f0d7d000640 1 -- 172.21.15.83:0/1155008954 wait complete. 2024-08-19T01:34:09.904 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:34:09 smithi083 ceph-mon[17120]: pgmap v318: 97 pgs: 97 active+clean; 641 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 597 B/s wr, 6 op/s 2024-08-19T01:34:09.952 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:34:09 smithi117 ceph-mon[22016]: pgmap v318: 97 pgs: 97 active+clean; 641 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 597 B/s wr, 6 op/s 2024-08-19T01:34:09.966 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:34:09 smithi167 ceph-mon[21875]: pgmap v318: 97 pgs: 97 active+clean; 641 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 597 B/s wr, 6 op/s 2024-08-19T01:34:09.996 INFO:tasks.cephadm:Setup complete, yielding 2024-08-19T01:34:09.996 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-08-19T01:34:10.006 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi083.front.sepia.ceph.com 2024-08-19T01:34:10.006 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid bad8780e-5dc9-11ef-bccf-c7b262605968 -- bash -c 'ceph orch status' 2024-08-19T01:34:10.380 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/bad8780e-5dc9-11ef-bccf-c7b262605968/mon.a/config 2024-08-19T01:34:10.950 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:10.948+0000 7f156a200640 1 -- 172.21.15.83:0/221110461 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1564102b30 msgr2=0x7f1564102fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:34:10.950 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:10.948+0000 7f156a200640 1 --2- 172.21.15.83:0/221110461 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1564102b30 0x7f1564102fb0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f154c009900 tx=0x7f154c02a3f0 comp rx=0 tx=0).stop 2024-08-19T01:34:10.951 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:10.948+0000 7f156a200640 1 -- 172.21.15.83:0/221110461 shutdown_connections 2024-08-19T01:34:10.951 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:10.948+0000 7f156a200640 1 --2- 172.21.15.83:0/221110461 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f15641034f0 0x7f1564109d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:10.951 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:10.948+0000 7f156a200640 1 --2- 172.21.15.83:0/221110461 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1564102b30 0x7f1564102fb0 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:10.951 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:10.948+0000 7f156a200640 1 --2- 172.21.15.83:0/221110461 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1564101930 0x7f1564101d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:10.951 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:10.948+0000 7f156a200640 1 -- 172.21.15.83:0/221110461 >> 172.21.15.83:0/221110461 conn(0x7f15640fd0c0 msgr2=0x7f15640ff500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:34:10.951 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:10.948+0000 7f156a200640 1 -- 172.21.15.83:0/221110461 shutdown_connections 2024-08-19T01:34:10.951 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:10.948+0000 7f156a200640 1 -- 172.21.15.83:0/221110461 wait complete. 2024-08-19T01:34:10.953 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:10.949+0000 7f156a200640 1 Processor -- start 2024-08-19T01:34:10.953 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:10.949+0000 7f156a200640 1 -- start start 2024-08-19T01:34:10.954 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:10.950+0000 7f156a200640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1564101930 0x7f1564197c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:34:10.954 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:10.950+0000 7f156a200640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1564102b30 0x7f1564198150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:34:10.954 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:10.950+0000 7f156a200640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f15641034f0 0x7f156419f190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:34:10.954 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:10.950+0000 7f156a200640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f156410d600 con 0x7f1564102b30 2024-08-19T01:34:10.954 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:10.950+0000 7f156a200640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f156410d480 con 0x7f15641034f0 2024-08-19T01:34:10.954 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:10.950+0000 7f156a200640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f156410d780 con 0x7f1564101930 2024-08-19T01:34:10.954 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:10.950+0000 7f155be00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1564102b30 0x7f1564198150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:34:10.954 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:10.950+0000 7f1563400640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1564101930 0x7f1564197c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:34:10.954 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:10.950+0000 7f155be00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1564102b30 0x7f1564198150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:45496/0 (socket says 172.21.15.83:45496) 2024-08-19T01:34:10.954 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:10.950+0000 7f155be00640 1 -- 172.21.15.83:0/1697623405 learned_addr learned my addr 172.21.15.83:0/1697623405 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-08-19T01:34:10.955 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:10.950+0000 7f1563e00640 1 --2- 172.21.15.83:0/1697623405 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f15641034f0 0x7f156419f190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:34:10.955 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:10.951+0000 7f1563400640 1 -- 172.21.15.83:0/1697623405 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f15641034f0 msgr2=0x7f156419f190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:34:10.955 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:10.951+0000 7f1563400640 1 --2- 172.21.15.83:0/1697623405 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f15641034f0 0x7f156419f190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:10.955 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:10.951+0000 7f1563400640 1 -- 172.21.15.83:0/1697623405 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1564102b30 msgr2=0x7f1564198150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:34:10.955 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:10.951+0000 7f1563400640 1 --2- 172.21.15.83:0/1697623405 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1564102b30 0x7f1564198150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:10.955 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:10.951+0000 7f1563400640 1 -- 172.21.15.83:0/1697623405 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f156419f830 con 0x7f1564101930 2024-08-19T01:34:10.955 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:10.951+0000 7f1563400640 1 --2- 172.21.15.83:0/1697623405 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1564101930 0x7f1564197c10 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f1564102990 tx=0x7f155400dde0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:34:10.955 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:10.951+0000 7f1560c00640 1 -- 172.21.15.83:0/1697623405 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1554017070 con 0x7f1564101930 2024-08-19T01:34:10.955 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:10.951+0000 7f1560c00640 1 -- 172.21.15.83:0/1697623405 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1554009c30 con 0x7f1564101930 2024-08-19T01:34:10.955 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:10.951+0000 7f156a200640 1 -- 172.21.15.83:0/1697623405 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f156419fb20 con 0x7f1564101930 2024-08-19T01:34:10.955 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:10.951+0000 7f1560c00640 1 -- 172.21.15.83:0/1697623405 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f155400f620 con 0x7f1564101930 2024-08-19T01:34:10.955 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:10.951+0000 7f156a200640 1 -- 172.21.15.83:0/1697623405 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f15641a0060 con 0x7f1564101930 2024-08-19T01:34:10.956 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:10.952+0000 7f156a200640 1 -- 172.21.15.83:0/1697623405 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1564108ce0 con 0x7f1564101930 2024-08-19T01:34:10.956 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:10.954+0000 7f1560c00640 1 -- 172.21.15.83:0/1697623405 <== mon.1 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f155401a070 con 0x7f1564101930 2024-08-19T01:34:10.957 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:10.956+0000 7f1560c00640 1 --2- 172.21.15.83:0/1697623405 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f15480a2b80 0x7f15480a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:34:10.957 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:10.956+0000 7f1560c00640 1 -- 172.21.15.83:0/1697623405 <== mon.1 v2:172.21.15.167:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8269+0+0 (secure 0 0 0) 0x7f15540cea90 con 0x7f1564101930 2024-08-19T01:34:10.958 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:10.956+0000 7f155be00640 1 --2- 172.21.15.83:0/1697623405 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f15480a2b80 0x7f15480a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:34:10.958 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:10.956+0000 7f155be00640 1 --2- 172.21.15.83:0/1697623405 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f15480a2b80 0x7f15480a5040 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f154c0096c0 tx=0x7f154c0315b0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:34:10.961 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:10.960+0000 7f1560c00640 1 -- 172.21.15.83:0/1697623405 <== mon.1 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7f155409b320 con 0x7f1564101930 2024-08-19T01:34:11.151 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:11.149+0000 7f156a200640 1 -- 172.21.15.83:0/1697623405 --> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] -- mgr_command(tid 0: {"prefix": "orch status", "target": ["mon-mgr", ""]}) -- 0x7f1564062360 con 0x7f15480a2b80 2024-08-19T01:34:11.153 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:11.151+0000 7f1560c00640 1 -- 172.21.15.83:0/1697623405 <== mgr.14150 v2:172.21.15.83:6800/683740916 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+43 (secure 0 0 0) 0x7f1564062360 con 0x7f15480a2b80 2024-08-19T01:34:11.153 INFO:teuthology.orchestra.run.smithi083.stdout:Backend: cephadm 2024-08-19T01:34:11.153 INFO:teuthology.orchestra.run.smithi083.stdout:Available: Yes 2024-08-19T01:34:11.153 INFO:teuthology.orchestra.run.smithi083.stdout:Paused: No 2024-08-19T01:34:11.155 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:11.153+0000 7f156a200640 1 -- 172.21.15.83:0/1697623405 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f15480a2b80 msgr2=0x7f15480a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:34:11.155 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:11.153+0000 7f156a200640 1 --2- 172.21.15.83:0/1697623405 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f15480a2b80 0x7f15480a5040 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f154c0096c0 tx=0x7f154c0315b0 comp rx=0 tx=0).stop 2024-08-19T01:34:11.155 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:11.153+0000 7f156a200640 1 -- 172.21.15.83:0/1697623405 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1564101930 msgr2=0x7f1564197c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:34:11.155 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:11.153+0000 7f156a200640 1 --2- 172.21.15.83:0/1697623405 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1564101930 0x7f1564197c10 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f1564102990 tx=0x7f155400dde0 comp rx=0 tx=0).stop 2024-08-19T01:34:11.155 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:11.153+0000 7f156a200640 1 -- 172.21.15.83:0/1697623405 shutdown_connections 2024-08-19T01:34:11.155 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:11.153+0000 7f156a200640 1 --2- 172.21.15.83:0/1697623405 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f15480a2b80 0x7f15480a5040 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:11.155 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:11.153+0000 7f156a200640 1 --2- 172.21.15.83:0/1697623405 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f15641034f0 0x7f156419f190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:11.155 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:11.153+0000 7f156a200640 1 --2- 172.21.15.83:0/1697623405 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1564102b30 0x7f1564198150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:11.155 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:11.153+0000 7f156a200640 1 --2- 172.21.15.83:0/1697623405 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1564101930 0x7f1564197c10 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:11.155 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:11.153+0000 7f156a200640 1 -- 172.21.15.83:0/1697623405 >> 172.21.15.83:0/1697623405 conn(0x7f15640fd0c0 msgr2=0x7f1564105eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:34:11.155 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:11.153+0000 7f156a200640 1 -- 172.21.15.83:0/1697623405 shutdown_connections 2024-08-19T01:34:11.155 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:11.153+0000 7f156a200640 1 -- 172.21.15.83:0/1697623405 wait complete. 2024-08-19T01:34:11.883 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:34:11 smithi083 ceph-mon[17120]: pgmap v319: 97 pgs: 97 active+clean; 641 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 597 B/s wr, 7 op/s 2024-08-19T01:34:11.920 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid bad8780e-5dc9-11ef-bccf-c7b262605968 -- bash -c 'ceph orch ps' 2024-08-19T01:34:11.952 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:34:11 smithi117 ceph-mon[22016]: pgmap v319: 97 pgs: 97 active+clean; 641 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 597 B/s wr, 7 op/s 2024-08-19T01:34:11.966 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:34:11 smithi167 ceph-mon[21875]: pgmap v319: 97 pgs: 97 active+clean; 641 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 597 B/s wr, 7 op/s 2024-08-19T01:34:12.295 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/bad8780e-5dc9-11ef-bccf-c7b262605968/mon.a/config 2024-08-19T01:34:12.703 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:34:12 smithi083 ceph-mon[17120]: from='client.24913 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-08-19T01:34:12.814 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:12.812+0000 7ff867800640 1 -- 172.21.15.83:0/3492931878 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff860102b30 msgr2=0x7ff860102fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:34:12.815 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:12.812+0000 7ff867800640 1 --2- 172.21.15.83:0/3492931878 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff860102b30 0x7ff860102fb0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7ff84c009900 tx=0x7ff84c02a410 comp rx=0 tx=0).stop 2024-08-19T01:34:12.815 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:12.812+0000 7ff867800640 1 -- 172.21.15.83:0/3492931878 shutdown_connections 2024-08-19T01:34:12.815 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:12.812+0000 7ff867800640 1 --2- 172.21.15.83:0/3492931878 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff8601034f0 0x7ff860109d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:12.815 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:12.812+0000 7ff867800640 1 --2- 172.21.15.83:0/3492931878 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff860102b30 0x7ff860102fb0 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:12.815 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:12.812+0000 7ff867800640 1 --2- 172.21.15.83:0/3492931878 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff860101930 0x7ff860101d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:12.815 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:12.812+0000 7ff867800640 1 -- 172.21.15.83:0/3492931878 >> 172.21.15.83:0/3492931878 conn(0x7ff8600fd0c0 msgr2=0x7ff8600ff500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:34:12.815 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:12.812+0000 7ff867800640 1 -- 172.21.15.83:0/3492931878 shutdown_connections 2024-08-19T01:34:12.815 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:12.813+0000 7ff867800640 1 -- 172.21.15.83:0/3492931878 wait complete. 2024-08-19T01:34:12.816 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:12.813+0000 7ff867800640 1 Processor -- start 2024-08-19T01:34:12.816 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:12.814+0000 7ff867800640 1 -- start start 2024-08-19T01:34:12.816 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:12.814+0000 7ff867800640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff860101930 0x7ff860075cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:34:12.817 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:12.814+0000 7ff867800640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff860102b30 0x7ff860074310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:34:12.817 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:12.814+0000 7ff867800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff8601034f0 0x7ff860074850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:34:12.817 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:12.814+0000 7ff867800640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7ff86010d590 con 0x7ff8601034f0 2024-08-19T01:34:12.817 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:12.814+0000 7ff867800640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7ff86010d410 con 0x7ff860101930 2024-08-19T01:34:12.817 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:12.814+0000 7ff867800640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7ff86010d710 con 0x7ff860102b30 2024-08-19T01:34:12.817 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:12.814+0000 7ff865800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff8601034f0 0x7ff860074850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:34:12.817 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:12.814+0000 7ff865800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff8601034f0 0x7ff860074850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:45528/0 (socket says 172.21.15.83:45528) 2024-08-19T01:34:12.817 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:12.814+0000 7ff865800640 1 -- 172.21.15.83:0/4277373701 learned_addr learned my addr 172.21.15.83:0/4277373701 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-08-19T01:34:12.817 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:12.814+0000 7ff864e00640 1 --2- 172.21.15.83:0/4277373701 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff860101930 0x7ff860075cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:34:12.817 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:12.814+0000 7ff85fe00640 1 --2- 172.21.15.83:0/4277373701 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff860102b30 0x7ff860074310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:34:12.817 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:12.815+0000 7ff865800640 1 -- 172.21.15.83:0/4277373701 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff860102b30 msgr2=0x7ff860074310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:34:12.817 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:12.815+0000 7ff865800640 1 --2- 172.21.15.83:0/4277373701 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff860102b30 0x7ff860074310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:12.817 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:12.815+0000 7ff865800640 1 -- 172.21.15.83:0/4277373701 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff860101930 msgr2=0x7ff860075cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:34:12.817 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:12.815+0000 7ff865800640 1 --2- 172.21.15.83:0/4277373701 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff860101930 0x7ff860075cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:12.818 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:12.815+0000 7ff865800640 1 -- 172.21.15.83:0/4277373701 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff860074d90 con 0x7ff8601034f0 2024-08-19T01:34:12.818 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:12.815+0000 7ff864e00640 1 --2- 172.21.15.83:0/4277373701 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff860101930 0x7ff860075cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-19T01:34:12.818 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:12.815+0000 7ff865800640 1 --2- 172.21.15.83:0/4277373701 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff8601034f0 0x7ff860074850 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7ff85800eaa0 tx=0x7ff85800ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:34:12.819 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:12.815+0000 7ff85d600640 1 -- 172.21.15.83:0/4277373701 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff85800cdb0 con 0x7ff8601034f0 2024-08-19T01:34:12.819 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:12.815+0000 7ff85d600640 1 -- 172.21.15.83:0/4277373701 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff858005020 con 0x7ff8601034f0 2024-08-19T01:34:12.819 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:12.815+0000 7ff85d600640 1 -- 172.21.15.83:0/4277373701 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8580116b0 con 0x7ff8601034f0 2024-08-19T01:34:12.819 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:12.815+0000 7ff867800640 1 -- 172.21.15.83:0/4277373701 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff860075080 con 0x7ff8601034f0 2024-08-19T01:34:12.819 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:12.816+0000 7ff867800640 1 -- 172.21.15.83:0/4277373701 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff8601a6670 con 0x7ff8601034f0 2024-08-19T01:34:12.819 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:12.817+0000 7ff867800640 1 -- 172.21.15.83:0/4277373701 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff860101d30 con 0x7ff8601034f0 2024-08-19T01:34:12.820 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:12.818+0000 7ff85d600640 1 -- 172.21.15.83:0/4277373701 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7ff858005560 con 0x7ff8601034f0 2024-08-19T01:34:12.821 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:12.820+0000 7ff85d600640 1 --2- 172.21.15.83:0/4277373701 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7ff8440a2c00 0x7ff8440a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:34:12.822 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:12.820+0000 7ff85d600640 1 -- 172.21.15.83:0/4277373701 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8269+0+0 (secure 0 0 0) 0x7ff8580cf200 con 0x7ff8601034f0 2024-08-19T01:34:12.822 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:12.820+0000 7ff864e00640 1 --2- 172.21.15.83:0/4277373701 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7ff8440a2c00 0x7ff8440a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:34:12.822 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:12.821+0000 7ff864e00640 1 --2- 172.21.15.83:0/4277373701 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7ff8440a2c00 0x7ff8440a50c0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7ff860102990 tx=0x7ff850005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:34:12.825 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:12.824+0000 7ff85d600640 1 -- 172.21.15.83:0/4277373701 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7ff858016030 con 0x7ff8601034f0 2024-08-19T01:34:12.952 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:34:12 smithi117 ceph-mon[22016]: from='client.24913 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-08-19T01:34:12.966 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:34:12 smithi167 ceph-mon[21875]: from='client.24913 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-08-19T01:34:13.007 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:13.005+0000 7ff867800640 1 -- 172.21.15.83:0/4277373701 --> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) -- 0x7ff860062360 con 0x7ff8440a2c00 2024-08-19T01:34:13.021 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:13.020+0000 7ff85d600640 1 -- 172.21.15.83:0/4277373701 <== mgr.14150 v2:172.21.15.83:6800/683740916 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+4185 (secure 0 0 0) 0x7ff860062360 con 0x7ff8440a2c00 2024-08-19T01:34:13.022 INFO:teuthology.orchestra.run.smithi083.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-08-19T01:34:13.022 INFO:teuthology.orchestra.run.smithi083.stdout:mds.a smithi083 running (106s) 20s ago 105s 12.6M - 19.1.0-1382-gde09dd01 ea7a5189f30b 16051d2af2ea 2024-08-19T01:34:13.022 INFO:teuthology.orchestra.run.smithi083.stdout:mds.b smithi117 running (103s) 34s ago 102s 28.6M - 19.1.0-1382-gde09dd01 ea7a5189f30b 6c1e5cbab019 2024-08-19T01:34:13.022 INFO:teuthology.orchestra.run.smithi083.stdout:mds.c smithi167 running (119s) 34s ago 118s 16.9M - 19.1.0-1382-gde09dd01 ea7a5189f30b 2ce4263dad56 2024-08-19T01:34:13.022 INFO:teuthology.orchestra.run.smithi083.stdout:mds.d smithi083 running (2m) 20s ago 2m 12.8M - 19.1.0-1382-gde09dd01 ea7a5189f30b c3e18ff80221 2024-08-19T01:34:13.022 INFO:teuthology.orchestra.run.smithi083.stdout:mds.e smithi117 running (2m) 34s ago 2m 18.3M - 19.1.0-1382-gde09dd01 ea7a5189f30b 8f8b5c7000c4 2024-08-19T01:34:13.022 INFO:teuthology.orchestra.run.smithi083.stdout:mds.f smithi167 running (2m) 34s ago 2m 17.7M - 19.1.0-1382-gde09dd01 ea7a5189f30b 15e840bcee59 2024-08-19T01:34:13.022 INFO:teuthology.orchestra.run.smithi083.stdout:mds.g smithi083 running (110s) 20s ago 109s 16.4M - 19.1.0-1382-gde09dd01 ea7a5189f30b dbdad0fb1007 2024-08-19T01:34:13.022 INFO:teuthology.orchestra.run.smithi083.stdout:mds.h smithi117 running (116s) 34s ago 116s 17.8M - 19.1.0-1382-gde09dd01 ea7a5189f30b 682ad742e1a1 2024-08-19T01:34:13.023 INFO:teuthology.orchestra.run.smithi083.stdout:mds.i smithi167 running (113s) 34s ago 113s 18.0M - 19.1.0-1382-gde09dd01 ea7a5189f30b 7b30511a1090 2024-08-19T01:34:13.023 INFO:teuthology.orchestra.run.smithi083.stdout:mds.j smithi083 running (2m) 20s ago 2m 12.8M - 19.1.0-1382-gde09dd01 ea7a5189f30b 86e4712742a3 2024-08-19T01:34:13.023 INFO:teuthology.orchestra.run.smithi083.stdout:mds.k smithi117 running (2m) 34s ago 2m 14.3M - 19.1.0-1382-gde09dd01 ea7a5189f30b 13892578aec1 2024-08-19T01:34:13.023 INFO:teuthology.orchestra.run.smithi083.stdout:mds.l smithi167 running (2m) 34s ago 2m 14.4M - 19.1.0-1382-gde09dd01 ea7a5189f30b 3d38eebe4a75 2024-08-19T01:34:13.023 INFO:teuthology.orchestra.run.smithi083.stdout:mgr.x smithi083 *:9283,8765 running (8m) 20s ago 8m 509M - 19.1.0-1382-gde09dd01 ea7a5189f30b c168c67385d5 2024-08-19T01:34:13.023 INFO:teuthology.orchestra.run.smithi083.stdout:mgr.y smithi117 *:8443,8765 running (7m) 34s ago 7m 443M - 19.1.0-1382-gde09dd01 ea7a5189f30b 01f29b2e497c 2024-08-19T01:34:13.023 INFO:teuthology.orchestra.run.smithi083.stdout:mgr.z smithi167 *:8443,8765 running (7m) 34s ago 7m 443M - 19.1.0-1382-gde09dd01 ea7a5189f30b e7079f51b08d 2024-08-19T01:34:13.023 INFO:teuthology.orchestra.run.smithi083.stdout:mon.a smithi083 running (8m) 20s ago 9m 48.6M 2048M 19.1.0-1382-gde09dd01 ea7a5189f30b dbe1fc960772 2024-08-19T01:34:13.023 INFO:teuthology.orchestra.run.smithi083.stdout:mon.b smithi117 running (7m) 34s ago 7m 41.2M 2048M 19.1.0-1382-gde09dd01 ea7a5189f30b b45592b97ea9 2024-08-19T01:34:13.024 INFO:teuthology.orchestra.run.smithi083.stdout:mon.c smithi167 running (7m) 34s ago 7m 40.2M 2048M 19.1.0-1382-gde09dd01 ea7a5189f30b 16aa4382925d 2024-08-19T01:34:13.024 INFO:teuthology.orchestra.run.smithi083.stdout:osd.0 smithi083 running (6m) 20s ago 6m 54.0M 4272M 19.1.0-1382-gde09dd01 ea7a5189f30b 2b55f821138b 2024-08-19T01:34:13.024 INFO:teuthology.orchestra.run.smithi083.stdout:osd.1 smithi117 running (6m) 34s ago 6m 48.8M 4272M 19.1.0-1382-gde09dd01 ea7a5189f30b a14802c52aa6 2024-08-19T01:34:13.024 INFO:teuthology.orchestra.run.smithi083.stdout:osd.2 smithi167 running (6m) 34s ago 6m 56.8M 4272M 19.1.0-1382-gde09dd01 ea7a5189f30b 415027713ec5 2024-08-19T01:34:13.024 INFO:teuthology.orchestra.run.smithi083.stdout:osd.3 smithi083 running (5m) 20s ago 5m 45.3M 4272M 19.1.0-1382-gde09dd01 ea7a5189f30b ee290eb0504a 2024-08-19T01:34:13.024 INFO:teuthology.orchestra.run.smithi083.stdout:osd.4 smithi117 running (5m) 34s ago 5m 54.3M 4272M 19.1.0-1382-gde09dd01 ea7a5189f30b b3cffe3e9096 2024-08-19T01:34:13.024 INFO:teuthology.orchestra.run.smithi083.stdout:osd.5 smithi167 running (4m) 34s ago 4m 45.7M 4272M 19.1.0-1382-gde09dd01 ea7a5189f30b 75eb620dc5c5 2024-08-19T01:34:13.024 INFO:teuthology.orchestra.run.smithi083.stdout:osd.6 smithi083 running (4m) 20s ago 4m 42.8M 4272M 19.1.0-1382-gde09dd01 ea7a5189f30b 1a4817db6ba3 2024-08-19T01:34:13.024 INFO:teuthology.orchestra.run.smithi083.stdout:osd.7 smithi117 running (4m) 34s ago 4m 54.4M 4272M 19.1.0-1382-gde09dd01 ea7a5189f30b 98bb53d2daa4 2024-08-19T01:34:13.024 INFO:teuthology.orchestra.run.smithi083.stdout:osd.8 smithi167 running (3m) 34s ago 3m 52.7M 4272M 19.1.0-1382-gde09dd01 ea7a5189f30b f2b1c182317c 2024-08-19T01:34:13.024 INFO:teuthology.orchestra.run.smithi083.stdout:osd.9 smithi083 running (3m) 20s ago 3m 52.3M 4272M 19.1.0-1382-gde09dd01 ea7a5189f30b 09811e0a09a5 2024-08-19T01:34:13.024 INFO:teuthology.orchestra.run.smithi083.stdout:osd.10 smithi117 running (2m) 34s ago 2m 49.8M 4272M 19.1.0-1382-gde09dd01 ea7a5189f30b 64fc4fdcde48 2024-08-19T01:34:13.025 INFO:teuthology.orchestra.run.smithi083.stdout:osd.11 smithi167 running (2m) 34s ago 2m 50.4M 4272M 19.1.0-1382-gde09dd01 ea7a5189f30b a6866e0c738d 2024-08-19T01:34:13.025 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:13.022+0000 7ff867800640 1 -- 172.21.15.83:0/4277373701 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7ff8440a2c00 msgr2=0x7ff8440a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:34:13.026 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:13.022+0000 7ff867800640 1 --2- 172.21.15.83:0/4277373701 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7ff8440a2c00 0x7ff8440a50c0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7ff860102990 tx=0x7ff850005f90 comp rx=0 tx=0).stop 2024-08-19T01:34:13.026 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:13.022+0000 7ff867800640 1 -- 172.21.15.83:0/4277373701 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff8601034f0 msgr2=0x7ff860074850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:34:13.026 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:13.022+0000 7ff867800640 1 --2- 172.21.15.83:0/4277373701 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff8601034f0 0x7ff860074850 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7ff85800eaa0 tx=0x7ff85800ef60 comp rx=0 tx=0).stop 2024-08-19T01:34:13.026 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:13.022+0000 7ff867800640 1 -- 172.21.15.83:0/4277373701 shutdown_connections 2024-08-19T01:34:13.026 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:13.022+0000 7ff867800640 1 --2- 172.21.15.83:0/4277373701 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7ff8440a2c00 0x7ff8440a50c0 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:13.026 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:13.022+0000 7ff867800640 1 --2- 172.21.15.83:0/4277373701 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff8601034f0 0x7ff860074850 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:13.026 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:13.022+0000 7ff867800640 1 --2- 172.21.15.83:0/4277373701 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff860102b30 0x7ff860074310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:13.026 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:13.022+0000 7ff867800640 1 --2- 172.21.15.83:0/4277373701 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff860101930 0x7ff860075cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:13.026 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:13.022+0000 7ff867800640 1 -- 172.21.15.83:0/4277373701 >> 172.21.15.83:0/4277373701 conn(0x7ff8600fd0c0 msgr2=0x7ff860105f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:34:13.026 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:13.022+0000 7ff867800640 1 -- 172.21.15.83:0/4277373701 shutdown_connections 2024-08-19T01:34:13.027 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:13.022+0000 7ff867800640 1 -- 172.21.15.83:0/4277373701 wait complete. 2024-08-19T01:34:13.654 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:34:13 smithi083 ceph-mon[17120]: pgmap v320: 97 pgs: 97 active+clean; 641 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 597 B/s wr, 6 op/s 2024-08-19T01:34:13.719 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid bad8780e-5dc9-11ef-bccf-c7b262605968 -- bash -c 'ceph orch ls' 2024-08-19T01:34:13.952 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:34:13 smithi117 ceph-mon[22016]: pgmap v320: 97 pgs: 97 active+clean; 641 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 597 B/s wr, 6 op/s 2024-08-19T01:34:13.966 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:34:13 smithi167 ceph-mon[21875]: pgmap v320: 97 pgs: 97 active+clean; 641 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 597 B/s wr, 6 op/s 2024-08-19T01:34:14.094 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/bad8780e-5dc9-11ef-bccf-c7b262605968/mon.a/config 2024-08-19T01:34:14.695 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:14.661+0000 7f3b34200640 1 -- 172.21.15.83:0/3987201796 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3b2c103b50 msgr2=0x7f3b2c103f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:34:14.695 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:14.661+0000 7f3b34200640 1 --2- 172.21.15.83:0/3987201796 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3b2c103b50 0x7f3b2c103f50 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f3b20009900 tx=0x7f3b2002a3f0 comp rx=0 tx=0).stop 2024-08-19T01:34:14.695 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:14.661+0000 7f3b34200640 1 -- 172.21.15.83:0/3987201796 shutdown_connections 2024-08-19T01:34:14.695 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:14.661+0000 7f3b34200640 1 --2- 172.21.15.83:0/3987201796 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3b2c105710 0x7f3b2c10bfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:14.695 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:14.661+0000 7f3b34200640 1 --2- 172.21.15.83:0/3987201796 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3b2c104d50 0x7f3b2c1051d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:14.696 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:14.661+0000 7f3b34200640 1 --2- 172.21.15.83:0/3987201796 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3b2c103b50 0x7f3b2c103f50 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:14.696 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:14.661+0000 7f3b34200640 1 -- 172.21.15.83:0/3987201796 >> 172.21.15.83:0/3987201796 conn(0x7f3b2c0ff2e0 msgr2=0x7f3b2c101720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:34:14.696 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:14.661+0000 7f3b34200640 1 -- 172.21.15.83:0/3987201796 shutdown_connections 2024-08-19T01:34:14.696 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:14.661+0000 7f3b34200640 1 -- 172.21.15.83:0/3987201796 wait complete. 2024-08-19T01:34:14.696 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:14.663+0000 7f3b34200640 1 Processor -- start 2024-08-19T01:34:14.696 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:14.663+0000 7f3b34200640 1 -- start start 2024-08-19T01:34:14.696 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:14.663+0000 7f3b34200640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3b2c103b50 0x7f3b2c199e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:34:14.696 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:14.663+0000 7f3b34200640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3b2c104d50 0x7f3b2c19a370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:34:14.696 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:14.663+0000 7f3b34200640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3b2c105710 0x7f3b2c1a13f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:34:14.696 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:14.663+0000 7f3b34200640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f3b2c10f7e0 con 0x7f3b2c105710 2024-08-19T01:34:14.696 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:14.663+0000 7f3b34200640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f3b2c10f660 con 0x7f3b2c103b50 2024-08-19T01:34:14.696 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:14.663+0000 7f3b34200640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f3b2c10f960 con 0x7f3b2c104d50 2024-08-19T01:34:14.696 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:14.663+0000 7f3b32200640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3b2c105710 0x7f3b2c1a13f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:34:14.697 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:14.663+0000 7f3b31800640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3b2c103b50 0x7f3b2c199e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:34:14.697 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:14.663+0000 7f3b32200640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3b2c105710 0x7f3b2c1a13f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:45550/0 (socket says 172.21.15.83:45550) 2024-08-19T01:34:14.697 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:14.663+0000 7f3b32200640 1 -- 172.21.15.83:0/3031251780 learned_addr learned my addr 172.21.15.83:0/3031251780 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-08-19T01:34:14.697 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:14.663+0000 7f3b30e00640 1 --2- 172.21.15.83:0/3031251780 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3b2c104d50 0x7f3b2c19a370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:34:14.697 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:14.664+0000 7f3b31800640 1 -- 172.21.15.83:0/3031251780 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3b2c104d50 msgr2=0x7f3b2c19a370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:34:14.697 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:14.664+0000 7f3b31800640 1 --2- 172.21.15.83:0/3031251780 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3b2c104d50 0x7f3b2c19a370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:14.697 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:14.664+0000 7f3b31800640 1 -- 172.21.15.83:0/3031251780 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3b2c105710 msgr2=0x7f3b2c1a13f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:34:14.697 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:14.664+0000 7f3b31800640 1 --2- 172.21.15.83:0/3031251780 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3b2c105710 0x7f3b2c1a13f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:14.697 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:14.664+0000 7f3b31800640 1 -- 172.21.15.83:0/3031251780 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3b2c1a1a60 con 0x7f3b2c103b50 2024-08-19T01:34:14.697 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:14.664+0000 7f3b31800640 1 --2- 172.21.15.83:0/3031251780 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3b2c103b50 0x7f3b2c199e30 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f3b20009230 tx=0x7f3b20031b60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:34:14.697 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:14.664+0000 7f3b2a000640 1 -- 172.21.15.83:0/3031251780 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b2003f070 con 0x7f3b2c103b50 2024-08-19T01:34:14.697 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:14.664+0000 7f3b34200640 1 -- 172.21.15.83:0/3031251780 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3b2c1a1cf0 con 0x7f3b2c103b50 2024-08-19T01:34:14.697 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:14.664+0000 7f3b2a000640 1 -- 172.21.15.83:0/3031251780 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3b20003cd0 con 0x7f3b2c103b50 2024-08-19T01:34:14.698 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:14.664+0000 7f3b2a000640 1 -- 172.21.15.83:0/3031251780 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b20035070 con 0x7f3b2c103b50 2024-08-19T01:34:14.698 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:14.665+0000 7f3b34200640 1 -- 172.21.15.83:0/3031251780 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3b2c1a21a0 con 0x7f3b2c103b50 2024-08-19T01:34:14.698 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:14.665+0000 7f3b34200640 1 -- 172.21.15.83:0/3031251780 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3b2c10af70 con 0x7f3b2c103b50 2024-08-19T01:34:14.698 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:14.667+0000 7f3b2a000640 1 -- 172.21.15.83:0/3031251780 <== mon.2 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f3b2003b4f0 con 0x7f3b2c103b50 2024-08-19T01:34:14.698 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:14.669+0000 7f3b2a000640 1 --2- 172.21.15.83:0/3031251780 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f3b140a2bb0 0x7f3b140a5070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:34:14.698 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:14.669+0000 7f3b2a000640 1 -- 172.21.15.83:0/3031251780 <== mon.2 v2:172.21.15.117:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8269+0+0 (secure 0 0 0) 0x7f3b2006ce00 con 0x7f3b2c103b50 2024-08-19T01:34:14.698 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:14.669+0000 7f3b30e00640 1 --2- 172.21.15.83:0/3031251780 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f3b140a2bb0 0x7f3b140a5070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:34:14.698 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:14.670+0000 7f3b30e00640 1 --2- 172.21.15.83:0/3031251780 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f3b140a2bb0 0x7f3b140a5070 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f3b2c19b350 tx=0x7f3b18005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:34:14.698 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:14.671+0000 7f3b2a000640 1 -- 172.21.15.83:0/3031251780 <== mon.2 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7f3b200bbfb0 con 0x7f3b2c103b50 2024-08-19T01:34:14.852 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:14.850+0000 7f3b34200640 1 -- 172.21.15.83:0/3031251780 --> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] -- mgr_command(tid 0: {"prefix": "orch ls", "target": ["mon-mgr", ""]}) -- 0x7f3b2c062360 con 0x7f3b140a2bb0 2024-08-19T01:34:14.852 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:34:14 smithi083 ceph-mon[17120]: from='client.14877 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-08-19T01:34:14.858 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:14.856+0000 7f3b2a000640 1 -- 172.21.15.83:0/3031251780 <== mgr.14150 v2:172.21.15.83:6800/683740916 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+980 (secure 0 0 0) 0x7f3b2c062360 con 0x7f3b140a2bb0 2024-08-19T01:34:14.858 INFO:teuthology.orchestra.run.smithi083.stdout:NAME PORTS RUNNING REFRESHED AGE PLACEMENT 2024-08-19T01:34:14.858 INFO:teuthology.orchestra.run.smithi083.stdout:mds.all 12/12 36s ago 2m smithi083=a;smithi083=d;smithi083=g;smithi083=j;smithi117=b;smithi117=e;smithi117=h;smithi117=k;smithi167=c;smithi167=f;smithi167=i;smithi167=l;count:12 2024-08-19T01:34:14.858 INFO:teuthology.orchestra.run.smithi083.stdout:mgr 3/3 36s ago 7m smithi083=x;smithi117=y;smithi167=z;count:3 2024-08-19T01:34:14.858 INFO:teuthology.orchestra.run.smithi083.stdout:mon 3/3 36s ago 7m smithi083:172.21.15.83=a;smithi117:172.21.15.117=b;smithi167:172.21.15.167=c;count:3 2024-08-19T01:34:14.859 INFO:teuthology.orchestra.run.smithi083.stdout:osd 12 36s ago - 2024-08-19T01:34:14.860 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:14.859+0000 7f3b34200640 1 -- 172.21.15.83:0/3031251780 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f3b140a2bb0 msgr2=0x7f3b140a5070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:34:14.861 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:14.859+0000 7f3b34200640 1 --2- 172.21.15.83:0/3031251780 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f3b140a2bb0 0x7f3b140a5070 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f3b2c19b350 tx=0x7f3b18005f90 comp rx=0 tx=0).stop 2024-08-19T01:34:14.861 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:14.859+0000 7f3b34200640 1 -- 172.21.15.83:0/3031251780 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3b2c103b50 msgr2=0x7f3b2c199e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:34:14.861 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:14.859+0000 7f3b34200640 1 --2- 172.21.15.83:0/3031251780 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3b2c103b50 0x7f3b2c199e30 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f3b20009230 tx=0x7f3b20031b60 comp rx=0 tx=0).stop 2024-08-19T01:34:14.861 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:14.859+0000 7f3b34200640 1 -- 172.21.15.83:0/3031251780 shutdown_connections 2024-08-19T01:34:14.861 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:14.859+0000 7f3b34200640 1 --2- 172.21.15.83:0/3031251780 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f3b140a2bb0 0x7f3b140a5070 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:14.861 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:14.859+0000 7f3b34200640 1 --2- 172.21.15.83:0/3031251780 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3b2c105710 0x7f3b2c1a13f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:14.861 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:14.859+0000 7f3b34200640 1 --2- 172.21.15.83:0/3031251780 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3b2c104d50 0x7f3b2c19a370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:14.861 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:14.859+0000 7f3b34200640 1 --2- 172.21.15.83:0/3031251780 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3b2c103b50 0x7f3b2c199e30 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:14.861 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:14.859+0000 7f3b34200640 1 -- 172.21.15.83:0/3031251780 >> 172.21.15.83:0/3031251780 conn(0x7f3b2c0ff2e0 msgr2=0x7f3b2c108140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:34:14.861 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:14.859+0000 7f3b34200640 1 -- 172.21.15.83:0/3031251780 shutdown_connections 2024-08-19T01:34:14.861 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:14.859+0000 7f3b34200640 1 -- 172.21.15.83:0/3031251780 wait complete. 2024-08-19T01:34:14.952 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:34:14 smithi117 ceph-mon[22016]: from='client.14877 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-08-19T01:34:14.966 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:34:14 smithi167 ceph-mon[21875]: from='client.14877 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-08-19T01:34:15.661 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid bad8780e-5dc9-11ef-bccf-c7b262605968 -- bash -c 'ceph orch host ls' 2024-08-19T01:34:15.904 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:34:15 smithi083 ceph-mon[17120]: pgmap v321: 97 pgs: 97 active+clean; 641 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 597 B/s wr, 6 op/s 2024-08-19T01:34:15.952 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:34:15 smithi117 ceph-mon[22016]: pgmap v321: 97 pgs: 97 active+clean; 641 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 597 B/s wr, 6 op/s 2024-08-19T01:34:15.965 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:34:15 smithi167 ceph-mon[21875]: pgmap v321: 97 pgs: 97 active+clean; 641 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 597 B/s wr, 6 op/s 2024-08-19T01:34:16.038 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/bad8780e-5dc9-11ef-bccf-c7b262605968/mon.a/config 2024-08-19T01:34:16.595 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:16.593+0000 7f1c78a00640 1 -- 172.21.15.83:0/4248175598 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1c74103330 msgr2=0x7f1c74109bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:34:16.595 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:16.593+0000 7f1c78a00640 1 --2- 172.21.15.83:0/4248175598 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1c74103330 0x7f1c74109bc0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f1c6c009ca0 tx=0x7f1c6c02a8c0 comp rx=0 tx=0).stop 2024-08-19T01:34:16.595 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:16.593+0000 7f1c78a00640 1 -- 172.21.15.83:0/4248175598 shutdown_connections 2024-08-19T01:34:16.595 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:16.593+0000 7f1c78a00640 1 --2- 172.21.15.83:0/4248175598 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1c74103330 0x7f1c74109bc0 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:16.595 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:16.593+0000 7f1c78a00640 1 --2- 172.21.15.83:0/4248175598 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1c74102970 0x7f1c74102df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:16.595 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:16.593+0000 7f1c78a00640 1 --2- 172.21.15.83:0/4248175598 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1c74101770 0x7f1c74101b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:16.596 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:16.593+0000 7f1c78a00640 1 -- 172.21.15.83:0/4248175598 >> 172.21.15.83:0/4248175598 conn(0x7f1c740fcf20 msgr2=0x7f1c740ff340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:34:16.596 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:16.593+0000 7f1c78a00640 1 -- 172.21.15.83:0/4248175598 shutdown_connections 2024-08-19T01:34:16.596 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:16.593+0000 7f1c78a00640 1 -- 172.21.15.83:0/4248175598 wait complete. 2024-08-19T01:34:16.597 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:16.594+0000 7f1c78a00640 1 Processor -- start 2024-08-19T01:34:16.597 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:16.594+0000 7f1c78a00640 1 -- start start 2024-08-19T01:34:16.597 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:16.595+0000 7f1c78a00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1c74101770 0x7f1c74075cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:34:16.597 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:16.595+0000 7f1c78a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1c74102970 0x7f1c74074310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:34:16.597 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:16.595+0000 7f1c78a00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1c74103330 0x7f1c74074850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:34:16.597 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:16.595+0000 7f1c78a00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f1c7410d3f0 con 0x7f1c74102970 2024-08-19T01:34:16.597 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:16.595+0000 7f1c78a00640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f1c7410d270 con 0x7f1c74101770 2024-08-19T01:34:16.597 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:16.595+0000 7f1c78a00640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f1c7410d570 con 0x7f1c74103330 2024-08-19T01:34:16.597 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:16.595+0000 7f1c73400640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1c74101770 0x7f1c74075cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:34:16.598 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:16.595+0000 7f1c72a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1c74102970 0x7f1c74074310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:34:16.598 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:16.595+0000 7f1c73400640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1c74101770 0x7f1c74075cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.83:43574/0 (socket says 172.21.15.83:43574) 2024-08-19T01:34:16.598 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:16.595+0000 7f1c73400640 1 -- 172.21.15.83:0/383606511 learned_addr learned my addr 172.21.15.83:0/383606511 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-08-19T01:34:16.598 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:16.595+0000 7f1c73e00640 1 --2- 172.21.15.83:0/383606511 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1c74103330 0x7f1c74074850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:34:16.598 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:16.595+0000 7f1c73400640 1 -- 172.21.15.83:0/383606511 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1c74103330 msgr2=0x7f1c74074850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:34:16.598 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:16.595+0000 7f1c73400640 1 --2- 172.21.15.83:0/383606511 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1c74103330 0x7f1c74074850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:16.598 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:16.595+0000 7f1c73400640 1 -- 172.21.15.83:0/383606511 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1c74102970 msgr2=0x7f1c74074310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:34:16.598 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:16.595+0000 7f1c73400640 1 --2- 172.21.15.83:0/383606511 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1c74102970 0x7f1c74074310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:16.598 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:16.595+0000 7f1c73400640 1 -- 172.21.15.83:0/383606511 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1c74075110 con 0x7f1c74101770 2024-08-19T01:34:16.598 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:16.595+0000 7f1c72a00640 1 --2- 172.21.15.83:0/383606511 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1c74102970 0x7f1c74074310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-19T01:34:16.599 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:16.596+0000 7f1c73400640 1 --2- 172.21.15.83:0/383606511 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1c74101770 0x7f1c74075cc0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f1c6400e9c0 tx=0x7f1c6400ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:34:16.599 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:16.596+0000 7f1c6be00640 1 -- 172.21.15.83:0/383606511 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c6400cd80 con 0x7f1c74101770 2024-08-19T01:34:16.599 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:16.596+0000 7f1c6be00640 1 -- 172.21.15.83:0/383606511 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1c640035f0 con 0x7f1c74101770 2024-08-19T01:34:16.599 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:16.596+0000 7f1c6be00640 1 -- 172.21.15.83:0/383606511 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c64010690 con 0x7f1c74101770 2024-08-19T01:34:16.599 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:16.596+0000 7f1c78a00640 1 -- 172.21.15.83:0/383606511 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1c741a1b50 con 0x7f1c74101770 2024-08-19T01:34:16.599 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:16.596+0000 7f1c78a00640 1 -- 172.21.15.83:0/383606511 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1c741a2010 con 0x7f1c74101770 2024-08-19T01:34:16.599 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:16.597+0000 7f1c78a00640 1 -- 172.21.15.83:0/383606511 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1c74108aa0 con 0x7f1c74101770 2024-08-19T01:34:16.602 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:16.601+0000 7f1c6be00640 1 -- 172.21.15.83:0/383606511 <== mon.2 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f1c640037a0 con 0x7f1c74101770 2024-08-19T01:34:16.604 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:16.602+0000 7f1c6be00640 1 --2- 172.21.15.83:0/383606511 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f1c580a2b80 0x7f1c580a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:34:16.604 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:16.602+0000 7f1c6be00640 1 -- 172.21.15.83:0/383606511 <== mon.2 v2:172.21.15.117:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8269+0+0 (secure 0 0 0) 0x7f1c640cf4f0 con 0x7f1c74101770 2024-08-19T01:34:16.604 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:16.602+0000 7f1c72a00640 1 --2- 172.21.15.83:0/383606511 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f1c580a2b80 0x7f1c580a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:34:16.604 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:16.603+0000 7f1c72a00640 1 --2- 172.21.15.83:0/383606511 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f1c580a2b80 0x7f1c580a5040 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f1c600095f0 tx=0x7f1c60005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:34:16.608 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:16.606+0000 7f1c6be00640 1 -- 172.21.15.83:0/383606511 <== mon.2 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7f1c6409c3d0 con 0x7f1c74101770 2024-08-19T01:34:16.796 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:34:16 smithi083 ceph-mon[17120]: from='client.24947 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-08-19T01:34:16.797 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:16.794+0000 7f1c78a00640 1 -- 172.21.15.83:0/383606511 --> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""]}) -- 0x7f1c74062360 con 0x7f1c580a2b80 2024-08-19T01:34:16.797 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:16.796+0000 7f1c6be00640 1 -- 172.21.15.83:0/383606511 <== mgr.14150 v2:172.21.15.83:6800/683740916 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+191 (secure 0 0 0) 0x7f1c74062360 con 0x7f1c580a2b80 2024-08-19T01:34:16.798 INFO:teuthology.orchestra.run.smithi083.stdout:HOST ADDR LABELS STATUS 2024-08-19T01:34:16.798 INFO:teuthology.orchestra.run.smithi083.stdout:smithi083 172.21.15.83 2024-08-19T01:34:16.798 INFO:teuthology.orchestra.run.smithi083.stdout:smithi117 172.21.15.117 2024-08-19T01:34:16.798 INFO:teuthology.orchestra.run.smithi083.stdout:smithi167 172.21.15.167 2024-08-19T01:34:16.798 INFO:teuthology.orchestra.run.smithi083.stdout:3 hosts in cluster 2024-08-19T01:34:16.800 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:16.798+0000 7f1c78a00640 1 -- 172.21.15.83:0/383606511 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f1c580a2b80 msgr2=0x7f1c580a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:34:16.800 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:16.798+0000 7f1c78a00640 1 --2- 172.21.15.83:0/383606511 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f1c580a2b80 0x7f1c580a5040 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f1c600095f0 tx=0x7f1c60005f90 comp rx=0 tx=0).stop 2024-08-19T01:34:16.801 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:16.799+0000 7f1c78a00640 1 -- 172.21.15.83:0/383606511 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1c74101770 msgr2=0x7f1c74075cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:34:16.801 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:16.799+0000 7f1c78a00640 1 --2- 172.21.15.83:0/383606511 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1c74101770 0x7f1c74075cc0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f1c6400e9c0 tx=0x7f1c6400ee80 comp rx=0 tx=0).stop 2024-08-19T01:34:16.801 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:16.799+0000 7f1c78a00640 1 -- 172.21.15.83:0/383606511 shutdown_connections 2024-08-19T01:34:16.801 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:16.799+0000 7f1c78a00640 1 --2- 172.21.15.83:0/383606511 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f1c580a2b80 0x7f1c580a5040 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:16.801 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:16.799+0000 7f1c78a00640 1 --2- 172.21.15.83:0/383606511 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1c74103330 0x7f1c74074850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:16.801 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:16.799+0000 7f1c78a00640 1 --2- 172.21.15.83:0/383606511 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1c74102970 0x7f1c74074310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:16.801 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:16.799+0000 7f1c78a00640 1 --2- 172.21.15.83:0/383606511 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1c74101770 0x7f1c74075cc0 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:16.801 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:16.799+0000 7f1c78a00640 1 -- 172.21.15.83:0/383606511 >> 172.21.15.83:0/383606511 conn(0x7f1c740fcf20 msgr2=0x7f1c74105c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:34:16.801 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:16.799+0000 7f1c78a00640 1 -- 172.21.15.83:0/383606511 shutdown_connections 2024-08-19T01:34:16.801 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:16.799+0000 7f1c78a00640 1 -- 172.21.15.83:0/383606511 wait complete. 2024-08-19T01:34:16.952 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:34:16 smithi117 ceph-mon[22016]: from='client.24947 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-08-19T01:34:16.966 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:34:16 smithi167 ceph-mon[21875]: from='client.24947 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-08-19T01:34:17.590 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid bad8780e-5dc9-11ef-bccf-c7b262605968 -- bash -c 'ceph orch device ls' 2024-08-19T01:34:17.836 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:34:17 smithi083 ceph-mon[17120]: pgmap v322: 97 pgs: 97 active+clean; 641 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 597 B/s wr, 7 op/s 2024-08-19T01:34:17.952 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:34:17 smithi117 ceph-mon[22016]: pgmap v322: 97 pgs: 97 active+clean; 641 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 597 B/s wr, 7 op/s 2024-08-19T01:34:17.966 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:34:17 smithi167 ceph-mon[21875]: pgmap v322: 97 pgs: 97 active+clean; 641 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 597 B/s wr, 7 op/s 2024-08-19T01:34:17.973 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/bad8780e-5dc9-11ef-bccf-c7b262605968/mon.a/config 2024-08-19T01:34:18.562 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:18.560+0000 7fbab6600640 1 -- 172.21.15.83:0/3523657240 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbab01056f0 msgr2=0x7fbab010bf80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:34:18.562 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:18.560+0000 7fbab6600640 1 --2- 172.21.15.83:0/3523657240 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbab01056f0 0x7fbab010bf80 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fbaa8009ca0 tx=0x7fbaa802a8c0 comp rx=0 tx=0).stop 2024-08-19T01:34:18.562 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:18.560+0000 7fbab6600640 1 -- 172.21.15.83:0/3523657240 shutdown_connections 2024-08-19T01:34:18.563 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:18.560+0000 7fbab6600640 1 --2- 172.21.15.83:0/3523657240 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbab01056f0 0x7fbab010bf80 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:18.563 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:18.560+0000 7fbab6600640 1 --2- 172.21.15.83:0/3523657240 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbab0104d30 0x7fbab01051b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:18.563 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:18.560+0000 7fbab6600640 1 --2- 172.21.15.83:0/3523657240 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbab0103b30 0x7fbab0103f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:18.563 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:18.560+0000 7fbab6600640 1 -- 172.21.15.83:0/3523657240 >> 172.21.15.83:0/3523657240 conn(0x7fbab00ff2e0 msgr2=0x7fbab0101700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:34:18.563 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:18.560+0000 7fbab6600640 1 -- 172.21.15.83:0/3523657240 shutdown_connections 2024-08-19T01:34:18.563 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:18.560+0000 7fbab6600640 1 -- 172.21.15.83:0/3523657240 wait complete. 2024-08-19T01:34:18.564 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:18.561+0000 7fbab6600640 1 Processor -- start 2024-08-19T01:34:18.565 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:18.561+0000 7fbab6600640 1 -- start start 2024-08-19T01:34:18.565 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:18.562+0000 7fbab6600640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbab0103b30 0x7fbab0199db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:34:18.565 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:18.562+0000 7fbab6600640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbab0104d30 0x7fbab019a2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:34:18.565 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:18.562+0000 7fbab6600640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbab01056f0 0x7fbab01a1330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:34:18.565 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:18.562+0000 7fbab6600640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fbab010f790 con 0x7fbab0104d30 2024-08-19T01:34:18.565 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:18.562+0000 7fbab6600640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fbab010f610 con 0x7fbab0103b30 2024-08-19T01:34:18.565 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:18.562+0000 7fbab6600640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fbab010f910 con 0x7fbab01056f0 2024-08-19T01:34:18.565 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:18.562+0000 7fbaaea00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbab0104d30 0x7fbab019a2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:34:18.565 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:18.562+0000 7fbaaea00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbab0104d30 0x7fbab019a2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:45584/0 (socket says 172.21.15.83:45584) 2024-08-19T01:34:18.565 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:18.562+0000 7fbaaea00640 1 -- 172.21.15.83:0/3915728082 learned_addr learned my addr 172.21.15.83:0/3915728082 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-08-19T01:34:18.565 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:18.562+0000 7fbaaf400640 1 --2- 172.21.15.83:0/3915728082 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbab0103b30 0x7fbab0199db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:34:18.565 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:18.562+0000 7fbaafe00640 1 --2- 172.21.15.83:0/3915728082 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbab01056f0 0x7fbab01a1330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:34:18.566 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:18.562+0000 7fbaaea00640 1 -- 172.21.15.83:0/3915728082 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbab01056f0 msgr2=0x7fbab01a1330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:34:18.566 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:18.562+0000 7fbaaea00640 1 --2- 172.21.15.83:0/3915728082 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbab01056f0 0x7fbab01a1330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:18.566 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:18.562+0000 7fbaaea00640 1 -- 172.21.15.83:0/3915728082 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbab0103b30 msgr2=0x7fbab0199db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:34:18.566 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:18.562+0000 7fbaaea00640 1 --2- 172.21.15.83:0/3915728082 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbab0103b30 0x7fbab0199db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:18.566 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:18.562+0000 7fbaaea00640 1 -- 172.21.15.83:0/3915728082 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbab01a19d0 con 0x7fbab0104d30 2024-08-19T01:34:18.566 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:18.562+0000 7fbaafe00640 1 --2- 172.21.15.83:0/3915728082 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbab01056f0 0x7fbab01a1330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-19T01:34:18.566 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:18.563+0000 7fbaaea00640 1 --2- 172.21.15.83:0/3915728082 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbab0104d30 0x7fbab019a2f0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fba9c00e9f0 tx=0x7fba9c00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:34:18.566 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:18.563+0000 7fbaa7e00640 1 -- 172.21.15.83:0/3915728082 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba9c00cd80 con 0x7fbab0104d30 2024-08-19T01:34:18.566 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:18.563+0000 7fbab6600640 1 -- 172.21.15.83:0/3915728082 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbab01a1cc0 con 0x7fbab0104d30 2024-08-19T01:34:18.566 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:18.563+0000 7fbaa7e00640 1 -- 172.21.15.83:0/3915728082 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fba9c010910 con 0x7fbab0104d30 2024-08-19T01:34:18.566 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:18.563+0000 7fbaa7e00640 1 -- 172.21.15.83:0/3915728082 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba9c0188f0 con 0x7fbab0104d30 2024-08-19T01:34:18.566 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:18.563+0000 7fbab6600640 1 -- 172.21.15.83:0/3915728082 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbab01a21b0 con 0x7fbab0104d30 2024-08-19T01:34:18.567 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:18.564+0000 7fbab6600640 1 -- 172.21.15.83:0/3915728082 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbab010aeb0 con 0x7fbab0104d30 2024-08-19T01:34:18.568 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:18.566+0000 7fbaa7e00640 1 -- 172.21.15.83:0/3915728082 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fba9c010ac0 con 0x7fbab0104d30 2024-08-19T01:34:18.569 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:18.567+0000 7fbaa7e00640 1 --2- 172.21.15.83:0/3915728082 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7fba940a2b80 0x7fba940a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:34:18.569 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:18.568+0000 7fbaa7e00640 1 -- 172.21.15.83:0/3915728082 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8269+0+0 (secure 0 0 0) 0x7fba9c0cf500 con 0x7fbab0104d30 2024-08-19T01:34:18.569 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:18.568+0000 7fbaaf400640 1 --2- 172.21.15.83:0/3915728082 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7fba940a2b80 0x7fba940a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:34:18.570 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:18.568+0000 7fbaaf400640 1 --2- 172.21.15.83:0/3915728082 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7fba940a2b80 0x7fba940a5040 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fbab019b290 tx=0x7fbaa0005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:34:18.573 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:18.572+0000 7fbaa7e00640 1 -- 172.21.15.83:0/3915728082 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7fba9c09c3e0 con 0x7fbab0104d30 2024-08-19T01:34:18.757 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:18.755+0000 7fbab6600640 1 -- 172.21.15.83:0/3915728082 --> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] -- mgr_command(tid 0: {"prefix": "orch device ls", "target": ["mon-mgr", ""]}) -- 0x7fbab0062360 con 0x7fba940a2b80 2024-08-19T01:34:18.757 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:34:18 smithi083 ceph-mon[17120]: from='client.24953 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-08-19T01:34:18.759 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:18.757+0000 7fbaa7e00640 1 -- 172.21.15.83:0/3915728082 <== mgr.14150 v2:172.21.15.83:6800/683740916 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+532 (secure 0 0 0) 0x7fbab0062360 con 0x7fba940a2b80 2024-08-19T01:34:18.759 INFO:teuthology.orchestra.run.smithi083.stdout:HOST PATH TYPE DEVICE ID SIZE AVAILABLE REFRESHED REJECT REASONS 2024-08-19T01:34:18.759 INFO:teuthology.orchestra.run.smithi083.stdout:smithi083 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_PHFT62040148400BGN 372G No 3m ago Has a FileSystem, LVM detected 2024-08-19T01:34:18.759 INFO:teuthology.orchestra.run.smithi083.stdout:smithi117 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_PHFT620400V6400BGN 372G No 2m ago Has a FileSystem, LVM detected 2024-08-19T01:34:18.759 INFO:teuthology.orchestra.run.smithi083.stdout:smithi167 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_CVFT62330080400BGN 372G No 2m ago Has a FileSystem, LVM detected 2024-08-19T01:34:18.761 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:18.759+0000 7fbab6600640 1 -- 172.21.15.83:0/3915728082 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7fba940a2b80 msgr2=0x7fba940a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:34:18.761 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:18.759+0000 7fbab6600640 1 --2- 172.21.15.83:0/3915728082 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7fba940a2b80 0x7fba940a5040 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fbab019b290 tx=0x7fbaa0005f90 comp rx=0 tx=0).stop 2024-08-19T01:34:18.762 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:18.759+0000 7fbab6600640 1 -- 172.21.15.83:0/3915728082 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbab0104d30 msgr2=0x7fbab019a2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:34:18.762 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:18.759+0000 7fbab6600640 1 --2- 172.21.15.83:0/3915728082 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbab0104d30 0x7fbab019a2f0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fba9c00e9f0 tx=0x7fba9c00eeb0 comp rx=0 tx=0).stop 2024-08-19T01:34:18.762 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:18.760+0000 7fbab6600640 1 -- 172.21.15.83:0/3915728082 shutdown_connections 2024-08-19T01:34:18.762 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:18.760+0000 7fbab6600640 1 --2- 172.21.15.83:0/3915728082 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7fba940a2b80 0x7fba940a5040 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:18.762 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:18.760+0000 7fbab6600640 1 --2- 172.21.15.83:0/3915728082 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbab01056f0 0x7fbab01a1330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:18.762 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:18.760+0000 7fbab6600640 1 --2- 172.21.15.83:0/3915728082 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbab0104d30 0x7fbab019a2f0 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:18.762 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:18.760+0000 7fbab6600640 1 --2- 172.21.15.83:0/3915728082 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbab0103b30 0x7fbab0199db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:18.762 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:18.760+0000 7fbab6600640 1 -- 172.21.15.83:0/3915728082 >> 172.21.15.83:0/3915728082 conn(0x7fbab00ff2e0 msgr2=0x7fbab0108080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:34:18.762 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:18.760+0000 7fbab6600640 1 -- 172.21.15.83:0/3915728082 shutdown_connections 2024-08-19T01:34:18.762 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:18.760+0000 7fbab6600640 1 -- 172.21.15.83:0/3915728082 wait complete. 2024-08-19T01:34:18.952 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:34:18 smithi117 ceph-mon[22016]: from='client.24953 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-08-19T01:34:18.966 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:34:18 smithi167 ceph-mon[21875]: from='client.24953 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-08-19T01:34:19.510 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-08-19T01:34:19.522 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi083.front.sepia.ceph.com 2024-08-19T01:34:19.522 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid bad8780e-5dc9-11ef-bccf-c7b262605968 -- bash -c 'ceph fs dump' 2024-08-19T01:34:19.853 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:34:19 smithi117 ceph-mon[22016]: pgmap v323: 97 pgs: 97 active+clean; 641 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 170 B/s wr, 6 op/s 2024-08-19T01:34:19.867 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:34:19 smithi167 ceph-mon[21875]: pgmap v323: 97 pgs: 97 active+clean; 641 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 170 B/s wr, 6 op/s 2024-08-19T01:34:19.896 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/bad8780e-5dc9-11ef-bccf-c7b262605968/mon.a/config 2024-08-19T01:34:19.904 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:34:19 smithi083 ceph-mon[17120]: pgmap v323: 97 pgs: 97 active+clean; 641 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 170 B/s wr, 6 op/s 2024-08-19T01:34:20.436 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:20.428+0000 7fd611c00640 1 -- 172.21.15.83:0/789493156 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd60c1034f0 msgr2=0x7fd60c109d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:34:20.436 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:20.428+0000 7fd611c00640 1 --2- 172.21.15.83:0/789493156 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd60c1034f0 0x7fd60c109d80 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fd604009ca0 tx=0x7fd60402a8c0 comp rx=0 tx=0).stop 2024-08-19T01:34:20.436 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:20.428+0000 7fd611c00640 1 -- 172.21.15.83:0/789493156 shutdown_connections 2024-08-19T01:34:20.436 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:20.428+0000 7fd611c00640 1 --2- 172.21.15.83:0/789493156 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd60c1034f0 0x7fd60c109d80 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:20.436 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:20.428+0000 7fd611c00640 1 --2- 172.21.15.83:0/789493156 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd60c102b30 0x7fd60c102fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:20.436 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:20.428+0000 7fd611c00640 1 --2- 172.21.15.83:0/789493156 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd60c101930 0x7fd60c101d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:20.437 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:20.428+0000 7fd611c00640 1 -- 172.21.15.83:0/789493156 >> 172.21.15.83:0/789493156 conn(0x7fd60c0fd0c0 msgr2=0x7fd60c0ff500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:34:20.437 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:20.428+0000 7fd611c00640 1 -- 172.21.15.83:0/789493156 shutdown_connections 2024-08-19T01:34:20.437 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:20.428+0000 7fd611c00640 1 -- 172.21.15.83:0/789493156 wait complete. 2024-08-19T01:34:20.437 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:20.429+0000 7fd611c00640 1 Processor -- start 2024-08-19T01:34:20.437 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:20.429+0000 7fd611c00640 1 -- start start 2024-08-19T01:34:20.437 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:20.429+0000 7fd611c00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd60c101930 0x7fd60c199da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:34:20.437 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:20.429+0000 7fd611c00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd60c102b30 0x7fd60c19a2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:34:20.437 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:20.430+0000 7fd611c00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd60c1034f0 0x7fd60c1a1360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:34:20.437 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:20.430+0000 7fd611c00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fd60c10d600 con 0x7fd60c1034f0 2024-08-19T01:34:20.437 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:20.430+0000 7fd611c00640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fd60c10d480 con 0x7fd60c102b30 2024-08-19T01:34:20.437 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:20.430+0000 7fd611c00640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fd60c10d780 con 0x7fd60c101930 2024-08-19T01:34:20.437 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:20.430+0000 7fd60b400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd60c1034f0 0x7fd60c1a1360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:34:20.437 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:20.430+0000 7fd60b400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd60c1034f0 0x7fd60c1a1360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:45602/0 (socket says 172.21.15.83:45602) 2024-08-19T01:34:20.438 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:20.430+0000 7fd60b400640 1 -- 172.21.15.83:0/2097064386 learned_addr learned my addr 172.21.15.83:0/2097064386 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-08-19T01:34:20.438 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:20.430+0000 7fd60aa00640 1 --2- 172.21.15.83:0/2097064386 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd60c101930 0x7fd60c199da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:34:20.438 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:20.430+0000 7fd60a000640 1 --2- 172.21.15.83:0/2097064386 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd60c102b30 0x7fd60c19a2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:34:20.438 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:20.430+0000 7fd60b400640 1 -- 172.21.15.83:0/2097064386 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd60c101930 msgr2=0x7fd60c199da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:34:20.438 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:20.430+0000 7fd60b400640 1 --2- 172.21.15.83:0/2097064386 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd60c101930 0x7fd60c199da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:20.438 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:20.430+0000 7fd60b400640 1 -- 172.21.15.83:0/2097064386 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd60c102b30 msgr2=0x7fd60c19a2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:34:20.438 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:20.430+0000 7fd60b400640 1 --2- 172.21.15.83:0/2097064386 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd60c102b30 0x7fd60c19a2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:20.438 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:20.430+0000 7fd60b400640 1 -- 172.21.15.83:0/2097064386 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd60c1a19d0 con 0x7fd60c1034f0 2024-08-19T01:34:20.438 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:20.431+0000 7fd60b400640 1 --2- 172.21.15.83:0/2097064386 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd60c1034f0 0x7fd60c1a1360 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fd604009430 tx=0x7fd6040319f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:34:20.438 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:20.431+0000 7fd603400640 1 -- 172.21.15.83:0/2097064386 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd604003b70 con 0x7fd60c1034f0 2024-08-19T01:34:20.438 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:20.431+0000 7fd603400640 1 -- 172.21.15.83:0/2097064386 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd604003d10 con 0x7fd60c1034f0 2024-08-19T01:34:20.438 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:20.431+0000 7fd603400640 1 -- 172.21.15.83:0/2097064386 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd60402fc50 con 0x7fd60c1034f0 2024-08-19T01:34:20.438 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:20.431+0000 7fd611c00640 1 -- 172.21.15.83:0/2097064386 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd60c1a1c60 con 0x7fd60c1034f0 2024-08-19T01:34:20.439 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:20.431+0000 7fd611c00640 1 -- 172.21.15.83:0/2097064386 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd60c1a21a0 con 0x7fd60c1034f0 2024-08-19T01:34:20.439 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:20.432+0000 7fd611c00640 1 -- 172.21.15.83:0/2097064386 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd60c1a2370 con 0x7fd60c1034f0 2024-08-19T01:34:20.439 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:20.434+0000 7fd603400640 1 -- 172.21.15.83:0/2097064386 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fd604004040 con 0x7fd60c1034f0 2024-08-19T01:34:20.439 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:20.435+0000 7fd603400640 1 --2- 172.21.15.83:0/2097064386 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7fd5f00a2bb0 0x7fd5f00a5070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:34:20.439 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:20.436+0000 7fd603400640 1 -- 172.21.15.83:0/2097064386 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8269+0+0 (secure 0 0 0) 0x7fd6040f0550 con 0x7fd60c1034f0 2024-08-19T01:34:20.440 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:20.436+0000 7fd60aa00640 1 --2- 172.21.15.83:0/2097064386 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7fd5f00a2bb0 0x7fd5f00a5070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:34:20.440 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:20.436+0000 7fd60aa00640 1 --2- 172.21.15.83:0/2097064386 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7fd5f00a2bb0 0x7fd5f00a5070 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fd60c102990 tx=0x7fd5fc005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:34:20.441 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:20.440+0000 7fd603400640 1 -- 172.21.15.83:0/2097064386 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7fd6040bd430 con 0x7fd60c1034f0 2024-08-19T01:34:20.654 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:34:20 smithi083 ceph-mon[17120]: from='client.14895 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-08-19T01:34:20.655 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:20.653+0000 7fd611c00640 1 -- 172.21.15.83:0/2097064386 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs dump"} v 0) -- 0x7fd60c1a2540 con 0x7fd60c1034f0 2024-08-19T01:34:20.655 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:20.654+0000 7fd603400640 1 -- 172.21.15.83:0/2097064386 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 24 v24) ==== 76+0+3093 (secure 0 0 0) 0x7fd6040bcd30 con 0x7fd60c1034f0 2024-08-19T01:34:20.656 INFO:teuthology.orchestra.run.smithi083.stdout:e24 2024-08-19T01:34:20.656 INFO:teuthology.orchestra.run.smithi083.stdout:btime 2024-08-19T01:32:54:291077+0000 2024-08-19T01:34:20.656 INFO:teuthology.orchestra.run.smithi083.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-08-19T01:34:20.656 INFO:teuthology.orchestra.run.smithi083.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-08-19T01:34:20.656 INFO:teuthology.orchestra.run.smithi083.stdout:legacy client fscid: 1 2024-08-19T01:34:20.656 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-08-19T01:34:20.656 INFO:teuthology.orchestra.run.smithi083.stdout:Filesystem 'cephfs' (1) 2024-08-19T01:34:20.657 INFO:teuthology.orchestra.run.smithi083.stdout:fs_name cephfs 2024-08-19T01:34:20.657 INFO:teuthology.orchestra.run.smithi083.stdout:epoch 24 2024-08-19T01:34:20.657 INFO:teuthology.orchestra.run.smithi083.stdout:flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2024-08-19T01:34:20.657 INFO:teuthology.orchestra.run.smithi083.stdout:created 2024-08-19T01:32:42.054769+0000 2024-08-19T01:34:20.657 INFO:teuthology.orchestra.run.smithi083.stdout:modified 2024-08-19T01:32:54.209056+0000 2024-08-19T01:34:20.657 INFO:teuthology.orchestra.run.smithi083.stdout:tableserver 0 2024-08-19T01:34:20.657 INFO:teuthology.orchestra.run.smithi083.stdout:root 0 2024-08-19T01:34:20.657 INFO:teuthology.orchestra.run.smithi083.stdout:session_timeout 300 2024-08-19T01:34:20.657 INFO:teuthology.orchestra.run.smithi083.stdout:session_autoclose 300 2024-08-19T01:34:20.657 INFO:teuthology.orchestra.run.smithi083.stdout:max_file_size 1099511627776 2024-08-19T01:34:20.657 INFO:teuthology.orchestra.run.smithi083.stdout:max_xattr_size 65536 2024-08-19T01:34:20.657 INFO:teuthology.orchestra.run.smithi083.stdout:required_client_features {} 2024-08-19T01:34:20.658 INFO:teuthology.orchestra.run.smithi083.stdout:last_failure 0 2024-08-19T01:34:20.658 INFO:teuthology.orchestra.run.smithi083.stdout:last_failure_osd_epoch 0 2024-08-19T01:34:20.658 INFO:teuthology.orchestra.run.smithi083.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-08-19T01:34:20.658 INFO:teuthology.orchestra.run.smithi083.stdout:max_mds 3 2024-08-19T01:34:20.658 INFO:teuthology.orchestra.run.smithi083.stdout:in 0,1,2 2024-08-19T01:34:20.658 INFO:teuthology.orchestra.run.smithi083.stdout:up {0=24545,1=24527,2=24509} 2024-08-19T01:34:20.658 INFO:teuthology.orchestra.run.smithi083.stdout:failed 2024-08-19T01:34:20.658 INFO:teuthology.orchestra.run.smithi083.stdout:damaged 2024-08-19T01:34:20.658 INFO:teuthology.orchestra.run.smithi083.stdout:stopped 2024-08-19T01:34:20.658 INFO:teuthology.orchestra.run.smithi083.stdout:data_pools [3] 2024-08-19T01:34:20.658 INFO:teuthology.orchestra.run.smithi083.stdout:metadata_pool 2 2024-08-19T01:34:20.659 INFO:teuthology.orchestra.run.smithi083.stdout:inline_data disabled 2024-08-19T01:34:20.659 INFO:teuthology.orchestra.run.smithi083.stdout:balancer 2024-08-19T01:34:20.659 INFO:teuthology.orchestra.run.smithi083.stdout:bal_rank_mask -1 2024-08-19T01:34:20.659 INFO:teuthology.orchestra.run.smithi083.stdout:standby_count_wanted 0 2024-08-19T01:34:20.659 INFO:teuthology.orchestra.run.smithi083.stdout:qdb_cluster leader: 24545 members: 24509,24527,24545 2024-08-19T01:34:20.659 INFO:teuthology.orchestra.run.smithi083.stdout:[mds.b{0:24545} state up:active seq 5 addr [v2:172.21.15.117:6838/1967595704,v1:172.21.15.117:6839/1967595704] compat {c=[1],r=[1],i=[1fff]}] 2024-08-19T01:34:20.659 INFO:teuthology.orchestra.run.smithi083.stdout:[mds.i{0:24496} state up:standby-replay seq 1 addr [v2:172.21.15.167:6838/3507387682,v1:172.21.15.167:6839/3507387682] compat {c=[1],r=[1],i=[1fff]}] 2024-08-19T01:34:20.659 INFO:teuthology.orchestra.run.smithi083.stdout:[mds.g{1:24527} state up:active seq 8 addr [v2:172.21.15.83:6838/3239261845,v1:172.21.15.83:6839/3239261845] compat {c=[1],r=[1],i=[1fff]}] 2024-08-19T01:34:20.659 INFO:teuthology.orchestra.run.smithi083.stdout:[mds.e{1:24485} state up:standby-replay seq 1 addr [v2:172.21.15.117:6834/1024213994,v1:172.21.15.117:6835/1024213994] compat {c=[1],r=[1],i=[1fff]}] 2024-08-19T01:34:20.659 INFO:teuthology.orchestra.run.smithi083.stdout:[mds.h{2:24509} state up:active seq 10 addr [v2:172.21.15.117:6836/3185224684,v1:172.21.15.117:6837/3185224684] compat {c=[1],r=[1],i=[1fff]}] 2024-08-19T01:34:20.659 INFO:teuthology.orchestra.run.smithi083.stdout:[mds.c{2:24478} state up:standby-replay seq 1 addr [v2:172.21.15.167:6836/1762130408,v1:172.21.15.167:6837/1762130408] compat {c=[1],r=[1],i=[1fff]}] 2024-08-19T01:34:20.659 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-08-19T01:34:20.659 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-08-19T01:34:20.660 INFO:teuthology.orchestra.run.smithi083.stdout:Standby daemons: 2024-08-19T01:34:20.660 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-08-19T01:34:20.660 INFO:teuthology.orchestra.run.smithi083.stdout:[mds.j{-1:14421} state up:standby seq 1 addr [v2:172.21.15.83:6834/988556067,v1:172.21.15.83:6835/988556067] compat {c=[1],r=[1],i=[1fff]}] 2024-08-19T01:34:20.660 INFO:teuthology.orchestra.run.smithi083.stdout:[mds.d{-1:14433} state up:standby seq 1 addr [v2:172.21.15.83:6836/869702632,v1:172.21.15.83:6837/869702632] compat {c=[1],r=[1],i=[1fff]}] 2024-08-19T01:34:20.660 INFO:teuthology.orchestra.run.smithi083.stdout:[mds.a{-1:14475} state up:standby seq 1 addr [v2:172.21.15.83:6840/1795975999,v1:172.21.15.83:6841/1795975999] compat {c=[1],r=[1],i=[1fff]}] 2024-08-19T01:34:20.660 INFO:teuthology.orchestra.run.smithi083.stdout:[mds.l{-1:24430} state up:standby seq 1 addr [v2:172.21.15.167:6832/2505076046,v1:172.21.15.167:6833/2505076046] compat {c=[1],r=[1],i=[1fff]}] 2024-08-19T01:34:20.660 INFO:teuthology.orchestra.run.smithi083.stdout:[mds.k{-1:24449} state up:standby seq 1 addr [v2:172.21.15.117:6832/3557520478,v1:172.21.15.117:6833/3557520478] compat {c=[1],r=[1],i=[1fff]}] 2024-08-19T01:34:20.660 INFO:teuthology.orchestra.run.smithi083.stdout:[mds.f{-1:24466} state up:standby seq 1 addr [v2:172.21.15.167:6834/3482045966,v1:172.21.15.167:6835/3482045966] compat {c=[1],r=[1],i=[1fff]}] 2024-08-19T01:34:20.661 INFO:teuthology.orchestra.run.smithi083.stderr:dumped fsmap epoch 24 2024-08-19T01:34:20.661 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:20.656+0000 7fd611c00640 1 -- 172.21.15.83:0/2097064386 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7fd5f00a2bb0 msgr2=0x7fd5f00a5070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:34:20.661 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:20.656+0000 7fd611c00640 1 --2- 172.21.15.83:0/2097064386 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7fd5f00a2bb0 0x7fd5f00a5070 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fd60c102990 tx=0x7fd5fc005f90 comp rx=0 tx=0).stop 2024-08-19T01:34:20.661 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:20.656+0000 7fd611c00640 1 -- 172.21.15.83:0/2097064386 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd60c1034f0 msgr2=0x7fd60c1a1360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:34:20.661 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:20.656+0000 7fd611c00640 1 --2- 172.21.15.83:0/2097064386 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd60c1034f0 0x7fd60c1a1360 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fd604009430 tx=0x7fd6040319f0 comp rx=0 tx=0).stop 2024-08-19T01:34:20.661 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:20.656+0000 7fd611c00640 1 -- 172.21.15.83:0/2097064386 shutdown_connections 2024-08-19T01:34:20.661 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:20.656+0000 7fd611c00640 1 --2- 172.21.15.83:0/2097064386 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7fd5f00a2bb0 0x7fd5f00a5070 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:20.661 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:20.656+0000 7fd611c00640 1 --2- 172.21.15.83:0/2097064386 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd60c1034f0 0x7fd60c1a1360 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:20.662 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:20.656+0000 7fd611c00640 1 --2- 172.21.15.83:0/2097064386 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd60c102b30 0x7fd60c19a2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:20.662 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:20.656+0000 7fd611c00640 1 --2- 172.21.15.83:0/2097064386 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd60c101930 0x7fd60c199da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:20.662 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:20.656+0000 7fd611c00640 1 -- 172.21.15.83:0/2097064386 >> 172.21.15.83:0/2097064386 conn(0x7fd60c0fd0c0 msgr2=0x7fd60c074150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:34:20.662 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:20.657+0000 7fd611c00640 1 -- 172.21.15.83:0/2097064386 shutdown_connections 2024-08-19T01:34:20.662 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:20.657+0000 7fd611c00640 1 -- 172.21.15.83:0/2097064386 wait complete. 2024-08-19T01:34:20.952 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:34:20 smithi117 ceph-mon[22016]: from='client.14895 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-08-19T01:34:20.966 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:34:20 smithi167 ceph-mon[21875]: from='client.14895 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-08-19T01:34:21.520 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid bad8780e-5dc9-11ef-bccf-c7b262605968 -- bash -c 'ceph osd dump' 2024-08-19T01:34:21.895 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/bad8780e-5dc9-11ef-bccf-c7b262605968/mon.a/config 2024-08-19T01:34:21.904 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:34:21 smithi083 ceph-mon[17120]: pgmap v324: 97 pgs: 97 active+clean; 641 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 6 op/s 2024-08-19T01:34:21.904 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:34:21 smithi083 ceph-mon[17120]: from='client.? 172.21.15.83:0/2097064386' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-08-19T01:34:21.952 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:34:21 smithi117 ceph-mon[22016]: pgmap v324: 97 pgs: 97 active+clean; 641 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 6 op/s 2024-08-19T01:34:21.952 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:34:21 smithi117 ceph-mon[22016]: from='client.? 172.21.15.83:0/2097064386' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-08-19T01:34:21.966 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:34:21 smithi167 ceph-mon[21875]: pgmap v324: 97 pgs: 97 active+clean; 641 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 6 op/s 2024-08-19T01:34:21.966 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:34:21 smithi167 ceph-mon[21875]: from='client.? 172.21.15.83:0/2097064386' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-08-19T01:34:22.457 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:22.455+0000 7f0374000640 1 -- 172.21.15.83:0/2817794032 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f036c0768d0 msgr2=0x7f036c076cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:34:22.457 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:22.455+0000 7f0374000640 1 --2- 172.21.15.83:0/2817794032 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f036c0768d0 0x7f036c076cd0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f0360009900 tx=0x7f036002a3f0 comp rx=0 tx=0).stop 2024-08-19T01:34:22.458 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:22.455+0000 7f0374000640 1 -- 172.21.15.83:0/2817794032 shutdown_connections 2024-08-19T01:34:22.458 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:22.455+0000 7f0374000640 1 --2- 172.21.15.83:0/2817794032 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f036c0784c0 0x7f036c078d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:22.458 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:22.455+0000 7f0374000640 1 --2- 172.21.15.83:0/2817794032 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f036c077b20 0x7f036c077f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:22.458 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:22.455+0000 7f0374000640 1 --2- 172.21.15.83:0/2817794032 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f036c0768d0 0x7f036c076cd0 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:22.458 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:22.455+0000 7f0374000640 1 -- 172.21.15.83:0/2817794032 >> 172.21.15.83:0/2817794032 conn(0x7f036c0ff2e0 msgr2=0x7f036c101700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:34:22.458 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:22.455+0000 7f0374000640 1 -- 172.21.15.83:0/2817794032 shutdown_connections 2024-08-19T01:34:22.458 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:22.455+0000 7f0374000640 1 -- 172.21.15.83:0/2817794032 wait complete. 2024-08-19T01:34:22.459 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:22.456+0000 7f0374000640 1 Processor -- start 2024-08-19T01:34:22.459 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:22.456+0000 7f0374000640 1 -- start start 2024-08-19T01:34:22.459 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:22.457+0000 7f0374000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f036c0768d0 0x7f036c19e150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:34:22.459 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:22.457+0000 7f0374000640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f036c077b20 0x7f036c19e690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:34:22.459 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:22.457+0000 7f0374000640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f036c0784c0 0x7f036c1a5710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:34:22.459 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:22.457+0000 7f0374000640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f036c113c00 con 0x7f036c0768d0 2024-08-19T01:34:22.459 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:22.457+0000 7f0374000640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f036c113a80 con 0x7f036c077b20 2024-08-19T01:34:22.460 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:22.457+0000 7f0374000640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f036c113d80 con 0x7f036c0784c0 2024-08-19T01:34:22.460 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:22.457+0000 7f0370c00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f036c077b20 0x7f036c19e690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:34:22.460 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:22.457+0000 7f0370c00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f036c077b20 0x7f036c19e690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.83:43874/0 (socket says 172.21.15.83:43874) 2024-08-19T01:34:22.460 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:22.457+0000 7f0370c00640 1 -- 172.21.15.83:0/3973118036 learned_addr learned my addr 172.21.15.83:0/3973118036 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-08-19T01:34:22.460 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:22.457+0000 7f0372000640 1 --2- 172.21.15.83:0/3973118036 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f036c0784c0 0x7f036c1a5710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:34:22.460 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:22.457+0000 7f0371600640 1 --2- 172.21.15.83:0/3973118036 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f036c0768d0 0x7f036c19e150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:34:22.461 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:22.458+0000 7f0370c00640 1 -- 172.21.15.83:0/3973118036 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f036c0784c0 msgr2=0x7f036c1a5710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:34:22.461 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:22.458+0000 7f0370c00640 1 --2- 172.21.15.83:0/3973118036 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f036c0784c0 0x7f036c1a5710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:22.461 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:22.458+0000 7f0370c00640 1 -- 172.21.15.83:0/3973118036 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f036c0768d0 msgr2=0x7f036c19e150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:34:22.461 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:22.458+0000 7f0370c00640 1 --2- 172.21.15.83:0/3973118036 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f036c0768d0 0x7f036c19e150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:22.461 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:22.458+0000 7f0370c00640 1 -- 172.21.15.83:0/3973118036 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f036c1a5d80 con 0x7f036c077b20 2024-08-19T01:34:22.461 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:22.458+0000 7f0372000640 1 --2- 172.21.15.83:0/3973118036 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f036c0784c0 0x7f036c1a5710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-19T01:34:22.461 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:22.458+0000 7f0370c00640 1 --2- 172.21.15.83:0/3973118036 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f036c077b20 0x7f036c19e690 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f035800c920 tx=0x7f035800cde0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:34:22.461 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:22.458+0000 7f036a000640 1 -- 172.21.15.83:0/3973118036 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0358007c70 con 0x7f036c077b20 2024-08-19T01:34:22.461 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:22.459+0000 7f0374000640 1 -- 172.21.15.83:0/3973118036 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f036c1a6070 con 0x7f036c077b20 2024-08-19T01:34:22.461 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:22.459+0000 7f036a000640 1 -- 172.21.15.83:0/3973118036 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0358007e10 con 0x7f036c077b20 2024-08-19T01:34:22.461 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:22.459+0000 7f036a000640 1 -- 172.21.15.83:0/3973118036 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0358018890 con 0x7f036c077b20 2024-08-19T01:34:22.462 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:22.459+0000 7f0374000640 1 -- 172.21.15.83:0/3973118036 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f036c1a65b0 con 0x7f036c077b20 2024-08-19T01:34:22.462 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:22.460+0000 7f0374000640 1 -- 172.21.15.83:0/3973118036 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f036c1104b0 con 0x7f036c077b20 2024-08-19T01:34:22.464 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:22.463+0000 7f036a000640 1 -- 172.21.15.83:0/3973118036 <== mon.2 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f0358018a30 con 0x7f036c077b20 2024-08-19T01:34:22.465 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:22.464+0000 7f036a000640 1 --2- 172.21.15.83:0/3973118036 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f03540a2bb0 0x7f03540a5070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:34:22.466 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:22.464+0000 7f036a000640 1 -- 172.21.15.83:0/3973118036 <== mon.2 v2:172.21.15.117:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8269+0+0 (secure 0 0 0) 0x7f03580cf3d0 con 0x7f036c077b20 2024-08-19T01:34:22.466 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:22.464+0000 7f0371600640 1 --2- 172.21.15.83:0/3973118036 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f03540a2bb0 0x7f03540a5070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:34:22.466 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:22.465+0000 7f0371600640 1 --2- 172.21.15.83:0/3973118036 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f03540a2bb0 0x7f03540a5070 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f0360031440 tx=0x7f03600315b0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:34:22.467 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:22.465+0000 7f036a000640 1 -- 172.21.15.83:0/3973118036 <== mon.2 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7f0358014030 con 0x7f036c077b20 2024-08-19T01:34:22.645 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:22.643+0000 7f0374000640 1 -- 172.21.15.83:0/3973118036 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "osd dump"} v 0) -- 0x7f036c02d830 con 0x7f036c077b20 2024-08-19T01:34:22.646 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:22.644+0000 7f036a000640 1 -- 172.21.15.83:0/3973118036 <== mon.2 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump"}]=0 v86) ==== 56+0+5327 (secure 0 0 0) 0x7f035809c340 con 0x7f036c077b20 2024-08-19T01:34:22.646 INFO:teuthology.orchestra.run.smithi083.stdout:epoch 86 2024-08-19T01:34:22.646 INFO:teuthology.orchestra.run.smithi083.stdout:fsid bad8780e-5dc9-11ef-bccf-c7b262605968 2024-08-19T01:34:22.646 INFO:teuthology.orchestra.run.smithi083.stdout:created 2024-08-19T01:25:12.626841+0000 2024-08-19T01:34:22.647 INFO:teuthology.orchestra.run.smithi083.stdout:modified 2024-08-19T01:33:05.461998+0000 2024-08-19T01:34:22.647 INFO:teuthology.orchestra.run.smithi083.stdout:flags sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit 2024-08-19T01:34:22.647 INFO:teuthology.orchestra.run.smithi083.stdout:crush_version 26 2024-08-19T01:34:22.647 INFO:teuthology.orchestra.run.smithi083.stdout:full_ratio 0.95 2024-08-19T01:34:22.647 INFO:teuthology.orchestra.run.smithi083.stdout:backfillfull_ratio 0.9 2024-08-19T01:34:22.647 INFO:teuthology.orchestra.run.smithi083.stdout:nearfull_ratio 0.85 2024-08-19T01:34:22.647 INFO:teuthology.orchestra.run.smithi083.stdout:require_min_compat_client luminous 2024-08-19T01:34:22.647 INFO:teuthology.orchestra.run.smithi083.stdout:min_compat_client luminous 2024-08-19T01:34:22.647 INFO:teuthology.orchestra.run.smithi083.stdout:require_osd_release squid 2024-08-19T01:34:22.647 INFO:teuthology.orchestra.run.smithi083.stdout:stretch_mode_enabled false 2024-08-19T01:34:22.647 INFO:teuthology.orchestra.run.smithi083.stdout:pool 1 '.mgr' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 1 pgp_num 1 autoscale_mode off last_change 22 flags hashpspool stripe_width 0 pg_num_max 32 pg_num_min 1 application mgr read_balance_score 12.00 2024-08-19T01:34:22.647 INFO:teuthology.orchestra.run.smithi083.stdout:pool 2 'cephfs_metadata' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 32 pgp_num 32 autoscale_mode off last_change 84 flags hashpspool stripe_width 0 pg_autoscale_bias 4 pg_num_min 16 recovery_priority 5 application cephfs read_balance_score 1.88 2024-08-19T01:34:22.647 INFO:teuthology.orchestra.run.smithi083.stdout:pool 3 'cephfs_data' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 64 pgp_num 64 autoscale_mode off last_change 84 flags hashpspool stripe_width 0 pg_num_min 64 target_size_ratio 0.9 application cephfs read_balance_score 1.50 2024-08-19T01:34:22.647 INFO:teuthology.orchestra.run.smithi083.stdout:max_osd 12 2024-08-19T01:34:22.648 INFO:teuthology.orchestra.run.smithi083.stdout:osd.0 up in weight 1 up_from 8 up_thru 85 down_at 0 last_clean_interval [0,0) [v2:172.21.15.83:6802/1082526824,v1:172.21.15.83:6803/1082526824] [v2:172.21.15.83:6804/1082526824,v1:172.21.15.83:6805/1082526824] exists,up ffa74041-f9d2-4555-bb43-508b8c581487 2024-08-19T01:34:22.648 INFO:teuthology.orchestra.run.smithi083.stdout:osd.1 up in weight 1 up_from 13 up_thru 82 down_at 0 last_clean_interval [0,0) [v2:172.21.15.117:6800/1925419802,v1:172.21.15.117:6801/1925419802] [v2:172.21.15.117:6802/1925419802,v1:172.21.15.117:6803/1925419802] exists,up 1977570b-b4f5-44fd-bb1e-3a31318be649 2024-08-19T01:34:22.648 INFO:teuthology.orchestra.run.smithi083.stdout:osd.2 up in weight 1 up_from 18 up_thru 82 down_at 0 last_clean_interval [0,0) [v2:172.21.15.167:6800/551625328,v1:172.21.15.167:6801/551625328] [v2:172.21.15.167:6802/551625328,v1:172.21.15.167:6803/551625328] exists,up a75278d1-161f-41d5-a92f-058d1077bf2a 2024-08-19T01:34:22.648 INFO:teuthology.orchestra.run.smithi083.stdout:osd.3 up in weight 1 up_from 25 up_thru 82 down_at 0 last_clean_interval [0,0) [v2:172.21.15.83:6810/3138942062,v1:172.21.15.83:6811/3138942062] [v2:172.21.15.83:6812/3138942062,v1:172.21.15.83:6813/3138942062] exists,up d1fb16f9-910e-46fa-b1e3-8e5bef4d346d 2024-08-19T01:34:22.648 INFO:teuthology.orchestra.run.smithi083.stdout:osd.4 up in weight 1 up_from 33 up_thru 82 down_at 0 last_clean_interval [0,0) [v2:172.21.15.117:6808/3978863827,v1:172.21.15.117:6809/3978863827] [v2:172.21.15.117:6810/3978863827,v1:172.21.15.117:6811/3978863827] exists,up 8176ed18-1fe2-446d-8994-8c60f6e08f38 2024-08-19T01:34:22.648 INFO:teuthology.orchestra.run.smithi083.stdout:osd.5 up in weight 1 up_from 38 up_thru 85 down_at 0 last_clean_interval [0,0) [v2:172.21.15.167:6808/2761858035,v1:172.21.15.167:6809/2761858035] [v2:172.21.15.167:6810/2761858035,v1:172.21.15.167:6811/2761858035] exists,up e01e3f15-25c8-4d2c-be6f-a170f036064c 2024-08-19T01:34:22.648 INFO:teuthology.orchestra.run.smithi083.stdout:osd.6 up in weight 1 up_from 46 up_thru 82 down_at 0 last_clean_interval [0,0) [v2:172.21.15.83:6818/260644552,v1:172.21.15.83:6819/260644552] [v2:172.21.15.83:6820/260644552,v1:172.21.15.83:6821/260644552] exists,up 78cf0a9b-9165-444f-b073-ebe13016de9d 2024-08-19T01:34:22.648 INFO:teuthology.orchestra.run.smithi083.stdout:osd.7 up in weight 1 up_from 51 up_thru 82 down_at 0 last_clean_interval [0,0) [v2:172.21.15.117:6816/104113252,v1:172.21.15.117:6817/104113252] [v2:172.21.15.117:6818/104113252,v1:172.21.15.117:6819/104113252] exists,up ee524f7f-672e-430e-9f58-1a9117513466 2024-08-19T01:34:22.648 INFO:teuthology.orchestra.run.smithi083.stdout:osd.8 up in weight 1 up_from 58 up_thru 85 down_at 0 last_clean_interval [0,0) [v2:172.21.15.167:6816/2196898857,v1:172.21.15.167:6817/2196898857] [v2:172.21.15.167:6818/2196898857,v1:172.21.15.167:6819/2196898857] exists,up 76d5317d-4845-4138-b210-e7bd2d80472a 2024-08-19T01:34:22.648 INFO:teuthology.orchestra.run.smithi083.stdout:osd.9 up in weight 1 up_from 66 up_thru 85 down_at 0 last_clean_interval [0,0) [v2:172.21.15.83:6826/903463450,v1:172.21.15.83:6827/903463450] [v2:172.21.15.83:6828/903463450,v1:172.21.15.83:6829/903463450] exists,up 0151bc2a-3ad2-4db3-8505-5855792cc9bb 2024-08-19T01:34:22.648 INFO:teuthology.orchestra.run.smithi083.stdout:osd.10 up in weight 1 up_from 71 up_thru 85 down_at 0 last_clean_interval [0,0) [v2:172.21.15.117:6824/1183570837,v1:172.21.15.117:6825/1183570837] [v2:172.21.15.117:6826/1183570837,v1:172.21.15.117:6827/1183570837] exists,up 437ed848-0da4-4cd4-8716-0c55bb99b676 2024-08-19T01:34:22.648 INFO:teuthology.orchestra.run.smithi083.stdout:osd.11 up in weight 1 up_from 78 up_thru 85 down_at 0 last_clean_interval [0,0) [v2:172.21.15.167:6824/187027350,v1:172.21.15.167:6825/187027350] [v2:172.21.15.167:6826/187027350,v1:172.21.15.167:6827/187027350] exists,up 13345f05-cdc0-4dae-a58b-551abbec1915 2024-08-19T01:34:22.648 INFO:teuthology.orchestra.run.smithi083.stdout:pg_upmap_items 2.a [2,11] 2024-08-19T01:34:22.649 INFO:teuthology.orchestra.run.smithi083.stdout:pg_upmap_items 2.1a [8,1] 2024-08-19T01:34:22.649 INFO:teuthology.orchestra.run.smithi083.stdout:pg_upmap_items 2.1b [2,4] 2024-08-19T01:34:22.649 INFO:teuthology.orchestra.run.smithi083.stdout:pg_upmap_items 2.1d [8,11] 2024-08-19T01:34:22.649 INFO:teuthology.orchestra.run.smithi083.stdout:pg_upmap_items 3.6 [4,6] 2024-08-19T01:34:22.649 INFO:teuthology.orchestra.run.smithi083.stdout:pg_upmap_items 3.11 [4,8] 2024-08-19T01:34:22.649 INFO:teuthology.orchestra.run.smithi083.stdout:pg_upmap_items 3.1b [5,9] 2024-08-19T01:34:22.649 INFO:teuthology.orchestra.run.smithi083.stdout:pg_upmap_items 3.20 [5,9] 2024-08-19T01:34:22.649 INFO:teuthology.orchestra.run.smithi083.stdout:pg_upmap_items 3.31 [4,7] 2024-08-19T01:34:22.649 INFO:teuthology.orchestra.run.smithi083.stdout:pg_upmap_items 3.3b [3,9] 2024-08-19T01:34:22.649 INFO:teuthology.orchestra.run.smithi083.stdout:blocklist 172.21.15.83:0/3138095466 expires 2024-08-20T01:26:04.277716+0000 2024-08-19T01:34:22.649 INFO:teuthology.orchestra.run.smithi083.stdout:blocklist 172.21.15.83:0/3835005198 expires 2024-08-20T01:26:04.277716+0000 2024-08-19T01:34:22.649 INFO:teuthology.orchestra.run.smithi083.stdout:blocklist 172.21.15.83:0/1790374088 expires 2024-08-20T01:25:37.228583+0000 2024-08-19T01:34:22.649 INFO:teuthology.orchestra.run.smithi083.stdout:blocklist 172.21.15.83:6801/573907319 expires 2024-08-20T01:26:04.277716+0000 2024-08-19T01:34:22.650 INFO:teuthology.orchestra.run.smithi083.stdout:blocklist 172.21.15.83:6800/573907319 expires 2024-08-20T01:26:04.277716+0000 2024-08-19T01:34:22.650 INFO:teuthology.orchestra.run.smithi083.stdout:blocklist 172.21.15.83:0/664676002 expires 2024-08-20T01:25:37.228583+0000 2024-08-19T01:34:22.650 INFO:teuthology.orchestra.run.smithi083.stdout:blocklist 172.21.15.83:0/2144611449 expires 2024-08-20T01:25:37.228583+0000 2024-08-19T01:34:22.650 INFO:teuthology.orchestra.run.smithi083.stdout:blocklist 172.21.15.83:6801/3261152818 expires 2024-08-20T01:25:37.228583+0000 2024-08-19T01:34:22.650 INFO:teuthology.orchestra.run.smithi083.stdout:blocklist 172.21.15.83:0/1609552666 expires 2024-08-20T01:26:04.277716+0000 2024-08-19T01:34:22.650 INFO:teuthology.orchestra.run.smithi083.stdout:blocklist 172.21.15.83:6800/3261152818 expires 2024-08-20T01:25:37.228583+0000 2024-08-19T01:34:22.651 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:22.647+0000 7f0374000640 1 -- 172.21.15.83:0/3973118036 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f03540a2bb0 msgr2=0x7f03540a5070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:34:22.651 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:22.647+0000 7f0374000640 1 --2- 172.21.15.83:0/3973118036 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f03540a2bb0 0x7f03540a5070 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f0360031440 tx=0x7f03600315b0 comp rx=0 tx=0).stop 2024-08-19T01:34:22.651 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:22.647+0000 7f0374000640 1 -- 172.21.15.83:0/3973118036 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f036c077b20 msgr2=0x7f036c19e690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:34:22.651 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:22.647+0000 7f0374000640 1 --2- 172.21.15.83:0/3973118036 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f036c077b20 0x7f036c19e690 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f035800c920 tx=0x7f035800cde0 comp rx=0 tx=0).stop 2024-08-19T01:34:22.651 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:22.647+0000 7f0374000640 1 -- 172.21.15.83:0/3973118036 shutdown_connections 2024-08-19T01:34:22.651 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:22.647+0000 7f0374000640 1 --2- 172.21.15.83:0/3973118036 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f03540a2bb0 0x7f03540a5070 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:22.651 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:22.647+0000 7f0374000640 1 --2- 172.21.15.83:0/3973118036 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f036c0784c0 0x7f036c1a5710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:22.651 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:22.647+0000 7f0374000640 1 --2- 172.21.15.83:0/3973118036 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f036c077b20 0x7f036c19e690 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:22.651 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:22.647+0000 7f0374000640 1 --2- 172.21.15.83:0/3973118036 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f036c0768d0 0x7f036c19e150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:22.651 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:22.647+0000 7f0374000640 1 -- 172.21.15.83:0/3973118036 >> 172.21.15.83:0/3973118036 conn(0x7f036c0ff2e0 msgr2=0x7f036c100d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:34:22.651 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:22.647+0000 7f0374000640 1 -- 172.21.15.83:0/3973118036 shutdown_connections 2024-08-19T01:34:22.652 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:22.647+0000 7f0374000640 1 -- 172.21.15.83:0/3973118036 wait complete. 2024-08-19T01:34:23.491 INFO:teuthology.run_tasks:Running task fs.ready... 2024-08-19T01:34:23.504 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833 shell --fsid bad8780e-5dc9-11ef-bccf-c7b262605968 -- ceph fs dump --format=json 2024-08-19T01:34:23.879 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/bad8780e-5dc9-11ef-bccf-c7b262605968/mon.a/config 2024-08-19T01:34:23.904 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:34:23 smithi083 ceph-mon[17120]: pgmap v325: 97 pgs: 97 active+clean; 641 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-08-19T01:34:23.904 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:34:23 smithi083 ceph-mon[17120]: from='client.? 172.21.15.83:0/3973118036' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-08-19T01:34:23.953 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:34:23 smithi117 ceph-mon[22016]: pgmap v325: 97 pgs: 97 active+clean; 641 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-08-19T01:34:23.953 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:34:23 smithi117 ceph-mon[22016]: from='client.? 172.21.15.83:0/3973118036' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-08-19T01:34:23.966 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:34:23 smithi167 ceph-mon[21875]: pgmap v325: 97 pgs: 97 active+clean; 641 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-08-19T01:34:23.966 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:34:23 smithi167 ceph-mon[21875]: from='client.? 172.21.15.83:0/3973118036' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-08-19T01:34:24.407 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:24.405+0000 7f88a6e00640 1 -- 172.21.15.83:0/4203568892 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f88a0102090 msgr2=0x7f88a010e5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:34:24.407 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:24.405+0000 7f88a6e00640 1 --2- 172.21.15.83:0/4203568892 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f88a0102090 0x7f88a010e5c0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f8898009cd0 tx=0x7f889802a8e0 comp rx=0 tx=0).stop 2024-08-19T01:34:24.407 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:24.405+0000 7f88a6e00640 1 -- 172.21.15.83:0/4203568892 shutdown_connections 2024-08-19T01:34:24.407 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:24.405+0000 7f88a6e00640 1 --2- 172.21.15.83:0/4203568892 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f88a0102090 0x7f88a010e5c0 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:24.407 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:24.405+0000 7f88a6e00640 1 --2- 172.21.15.83:0/4203568892 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f88a01016f0 0x7f88a0101b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:24.408 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:24.405+0000 7f88a6e00640 1 --2- 172.21.15.83:0/4203568892 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f88a01076f0 0x7f88a0107ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:24.408 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:24.405+0000 7f88a6e00640 1 -- 172.21.15.83:0/4203568892 >> 172.21.15.83:0/4203568892 conn(0x7f88a00fd410 msgr2=0x7f88a00ff830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:34:24.408 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:24.405+0000 7f88a6e00640 1 -- 172.21.15.83:0/4203568892 shutdown_connections 2024-08-19T01:34:24.408 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:24.405+0000 7f88a6e00640 1 -- 172.21.15.83:0/4203568892 wait complete. 2024-08-19T01:34:24.409 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:24.406+0000 7f88a6e00640 1 Processor -- start 2024-08-19T01:34:24.409 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:24.406+0000 7f88a6e00640 1 -- start start 2024-08-19T01:34:24.410 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:24.406+0000 7f88a6e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f88a01016f0 0x7f88a019a140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:34:24.410 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:24.406+0000 7f88a6e00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f88a0102090 0x7f88a019a680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:34:24.410 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:24.406+0000 7f88a6e00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f88a01076f0 0x7f88a019ea10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:34:24.410 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:24.406+0000 7f88a6e00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f88a0111b40 con 0x7f88a01016f0 2024-08-19T01:34:24.410 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:24.406+0000 7f88a6e00640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f88a01119c0 con 0x7f88a0102090 2024-08-19T01:34:24.410 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:24.406+0000 7f88a6e00640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f88a0111cc0 con 0x7f88a01076f0 2024-08-19T01:34:24.410 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:24.407+0000 7f88a5a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f88a01016f0 0x7f88a019a140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:34:24.410 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:24.407+0000 7f88a5a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f88a01016f0 0x7f88a019a140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:53438/0 (socket says 172.21.15.83:53438) 2024-08-19T01:34:24.410 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:24.407+0000 7f88a5a00640 1 -- 172.21.15.83:0/4265455598 learned_addr learned my addr 172.21.15.83:0/4265455598 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-08-19T01:34:24.410 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:24.407+0000 7f88a5000640 1 --2- 172.21.15.83:0/4265455598 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f88a0102090 0x7f88a019a680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:34:24.410 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:24.407+0000 7f88a6400640 1 --2- 172.21.15.83:0/4265455598 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f88a01076f0 0x7f88a019ea10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:34:24.410 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:24.407+0000 7f88a5a00640 1 -- 172.21.15.83:0/4265455598 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f88a01076f0 msgr2=0x7f88a019ea10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:34:24.411 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:24.407+0000 7f88a5a00640 1 --2- 172.21.15.83:0/4265455598 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f88a01076f0 0x7f88a019ea10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:24.411 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:24.407+0000 7f88a5a00640 1 -- 172.21.15.83:0/4265455598 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f88a0102090 msgr2=0x7f88a019a680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:34:24.411 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:24.407+0000 7f88a5a00640 1 --2- 172.21.15.83:0/4265455598 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f88a0102090 0x7f88a019a680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:24.411 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:24.407+0000 7f88a5a00640 1 -- 172.21.15.83:0/4265455598 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f88a019f0f0 con 0x7f88a01016f0 2024-08-19T01:34:24.411 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:24.407+0000 7f88a5a00640 1 --2- 172.21.15.83:0/4265455598 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f88a01016f0 0x7f88a019a140 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f889400d950 tx=0x7f889400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:34:24.411 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:24.408+0000 7f889e000640 1 -- 172.21.15.83:0/4265455598 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8894017070 con 0x7f88a01016f0 2024-08-19T01:34:24.411 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:24.408+0000 7f889e000640 1 -- 172.21.15.83:0/4265455598 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8894009c30 con 0x7f88a01016f0 2024-08-19T01:34:24.411 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:24.408+0000 7f88a6e00640 1 -- 172.21.15.83:0/4265455598 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f88a019f3e0 con 0x7f88a01016f0 2024-08-19T01:34:24.411 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:24.408+0000 7f889e000640 1 -- 172.21.15.83:0/4265455598 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f889400f6b0 con 0x7f88a01016f0 2024-08-19T01:34:24.411 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:24.408+0000 7f88a6e00640 1 -- 172.21.15.83:0/4265455598 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f88a01a6cc0 con 0x7f88a01016f0 2024-08-19T01:34:24.412 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:24.409+0000 7f88a6e00640 1 -- 172.21.15.83:0/4265455598 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f88a0101b50 con 0x7f88a01016f0 2024-08-19T01:34:24.412 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:24.411+0000 7f889e000640 1 -- 172.21.15.83:0/4265455598 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f88940109d0 con 0x7f88a01016f0 2024-08-19T01:34:24.413 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:24.412+0000 7f889e000640 1 --2- 172.21.15.83:0/4265455598 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f88880a2c00 0x7f88880a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:34:24.414 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:24.412+0000 7f889e000640 1 -- 172.21.15.83:0/4265455598 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8269+0+0 (secure 0 0 0) 0x7f88940cec10 con 0x7f88a01016f0 2024-08-19T01:34:24.414 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:24.413+0000 7f88a5000640 1 --2- 172.21.15.83:0/4265455598 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f88880a2c00 0x7f88880a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:34:24.414 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:24.413+0000 7f88a5000640 1 --2- 172.21.15.83:0/4265455598 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f88880a2c00 0x7f88880a50c0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f88a010e840 tx=0x7f888c005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:34:24.418 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:24.416+0000 7f889e000640 1 -- 172.21.15.83:0/4265455598 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7f8894009de0 con 0x7f88a01016f0 2024-08-19T01:34:24.628 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:24.627+0000 7f88a6e00640 1 -- 172.21.15.83:0/4265455598 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f88a01a6ef0 con 0x7f88a01016f0 2024-08-19T01:34:24.629 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:24.627+0000 7f889e000640 1 -- 172.21.15.83:0/4265455598 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) ==== 94+0+11920 (secure 0 0 0) 0x7f889409baf0 con 0x7f88a01016f0 2024-08-19T01:34:24.629 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-08-19T01:34:24.629 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":24,"btime":"2024-08-19T01:32:54:291077+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14421,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6835/988556067","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":988556067},{"type":"v1","addr":"172.21.15.83:6835","nonce":988556067}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14433,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6837/869702632","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6836","nonce":869702632},{"type":"v1","addr":"172.21.15.83:6837","nonce":869702632}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14475,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6841/1795975999","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6840","nonce":1795975999},{"type":"v1","addr":"172.21.15.83:6841","nonce":1795975999}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24430,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6833/2505076046","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6832","nonce":2505076046},{"type":"v1","addr":"172.21.15.167:6833","nonce":2505076046}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24449,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6833/3557520478","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":3557520478},{"type":"v1","addr":"172.21.15.117:6833","nonce":3557520478}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24466,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6835/3482045966","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":3482045966},{"type":"v1","addr":"172.21.15.167:6835","nonce":3482045966}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7}],"filesystems":[{"mdsmap":{"epoch":24,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-08-19T01:32:42.054769+0000","modified":"2024-08-19T01:32:54.209056+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24545,"mds_1":24527,"mds_2":24509},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24478":{"gid":24478,"name":"c","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.167:6837/1762130408","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6836","nonce":1762130408},{"type":"v1","addr":"172.21.15.167:6837","nonce":1762130408}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24485":{"gid":24485,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.117:6835/1024213994","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":1024213994},{"type":"v1","addr":"172.21.15.117:6835","nonce":1024213994}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24496":{"gid":24496,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.167:6839/3507387682","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6838","nonce":3507387682},{"type":"v1","addr":"172.21.15.167:6839","nonce":3507387682}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24509":{"gid":24509,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":10,"addr":"172.21.15.117:6837/3185224684","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":3185224684},{"type":"v1","addr":"172.21.15.117:6837","nonce":3185224684}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24527":{"gid":24527,"name":"g","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.83:6839/3239261845","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6838","nonce":3239261845},{"type":"v1","addr":"172.21.15.83:6839","nonce":3239261845}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24545":{"gid":24545,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.117:6839/1967595704","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6838","nonce":1967595704},{"type":"v1","addr":"172.21.15.117:6839","nonce":1967595704}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24545,"qdb_cluster":[24509,24527,24545]},"id":1}]} 2024-08-19T01:34:24.630 INFO:teuthology.orchestra.run.smithi083.stderr:dumped fsmap epoch 24 2024-08-19T01:34:24.631 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:24.630+0000 7f88a6e00640 1 -- 172.21.15.83:0/4265455598 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f88880a2c00 msgr2=0x7f88880a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:34:24.632 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:24.630+0000 7f88a6e00640 1 --2- 172.21.15.83:0/4265455598 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f88880a2c00 0x7f88880a50c0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f88a010e840 tx=0x7f888c005f90 comp rx=0 tx=0).stop 2024-08-19T01:34:24.632 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:24.630+0000 7f88a6e00640 1 -- 172.21.15.83:0/4265455598 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f88a01016f0 msgr2=0x7f88a019a140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:34:24.632 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:24.630+0000 7f88a6e00640 1 --2- 172.21.15.83:0/4265455598 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f88a01016f0 0x7f88a019a140 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f889400d950 tx=0x7f889400de10 comp rx=0 tx=0).stop 2024-08-19T01:34:24.632 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:24.630+0000 7f88a6e00640 1 -- 172.21.15.83:0/4265455598 shutdown_connections 2024-08-19T01:34:24.632 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:24.630+0000 7f88a6e00640 1 --2- 172.21.15.83:0/4265455598 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f88880a2c00 0x7f88880a50c0 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:24.632 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:24.630+0000 7f88a6e00640 1 --2- 172.21.15.83:0/4265455598 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f88a01076f0 0x7f88a019ea10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:24.632 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:24.630+0000 7f88a6e00640 1 --2- 172.21.15.83:0/4265455598 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f88a0102090 0x7f88a019a680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:24.632 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:24.630+0000 7f88a6e00640 1 --2- 172.21.15.83:0/4265455598 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f88a01016f0 0x7f88a019a140 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:24.632 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:24.630+0000 7f88a6e00640 1 -- 172.21.15.83:0/4265455598 >> 172.21.15.83:0/4265455598 conn(0x7f88a00fd410 msgr2=0x7f88a010cdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:34:24.632 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:24.630+0000 7f88a6e00640 1 -- 172.21.15.83:0/4265455598 shutdown_connections 2024-08-19T01:34:24.632 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:24.630+0000 7f88a6e00640 1 -- 172.21.15.83:0/4265455598 wait complete. 2024-08-19T01:34:25.372 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833 shell --fsid bad8780e-5dc9-11ef-bccf-c7b262605968 -- ceph fs dump --format=json 2024-08-19T01:34:25.654 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:34:25 smithi083 ceph-mon[17120]: pgmap v326: 97 pgs: 97 active+clean; 641 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-08-19T01:34:25.654 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:34:25 smithi083 ceph-mon[17120]: from='client.? 172.21.15.83:0/4265455598' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-19T01:34:25.749 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/bad8780e-5dc9-11ef-bccf-c7b262605968/mon.a/config 2024-08-19T01:34:25.952 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:34:25 smithi117 ceph-mon[22016]: pgmap v326: 97 pgs: 97 active+clean; 641 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-08-19T01:34:25.952 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:34:25 smithi117 ceph-mon[22016]: from='client.? 172.21.15.83:0/4265455598' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-19T01:34:25.966 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:34:25 smithi167 ceph-mon[21875]: pgmap v326: 97 pgs: 97 active+clean; 641 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-08-19T01:34:25.966 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:34:25 smithi167 ceph-mon[21875]: from='client.? 172.21.15.83:0/4265455598' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-19T01:34:26.344 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:26.317+0000 7fd820c00640 1 -- 172.21.15.83:0/2502395919 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd81c1020a0 msgr2=0x7fd81c10e5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:34:26.344 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:26.317+0000 7fd820c00640 1 --2- 172.21.15.83:0/2502395919 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd81c1020a0 0x7fd81c10e5d0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fd814009cd0 tx=0x7fd81402a8e0 comp rx=0 tx=0).stop 2024-08-19T01:34:26.344 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:26.317+0000 7fd820c00640 1 -- 172.21.15.83:0/2502395919 shutdown_connections 2024-08-19T01:34:26.345 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:26.317+0000 7fd820c00640 1 --2- 172.21.15.83:0/2502395919 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd81c1020a0 0x7fd81c10e5d0 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:26.345 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:26.317+0000 7fd820c00640 1 --2- 172.21.15.83:0/2502395919 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd81c101700 0x7fd81c101b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:26.345 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:26.317+0000 7fd820c00640 1 --2- 172.21.15.83:0/2502395919 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd81c107700 0x7fd81c107ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:26.345 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:26.317+0000 7fd820c00640 1 -- 172.21.15.83:0/2502395919 >> 172.21.15.83:0/2502395919 conn(0x7fd81c0fd440 msgr2=0x7fd81c0ff860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:34:26.345 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:26.317+0000 7fd820c00640 1 -- 172.21.15.83:0/2502395919 shutdown_connections 2024-08-19T01:34:26.345 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:26.317+0000 7fd820c00640 1 -- 172.21.15.83:0/2502395919 wait complete. 2024-08-19T01:34:26.345 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:26.318+0000 7fd820c00640 1 Processor -- start 2024-08-19T01:34:26.345 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:26.318+0000 7fd820c00640 1 -- start start 2024-08-19T01:34:26.345 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:26.318+0000 7fd820c00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd81c101700 0x7fd81c074220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:34:26.345 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:26.318+0000 7fd820c00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd81c1020a0 0x7fd81c074760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:34:26.345 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:26.318+0000 7fd820c00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd81c107700 0x7fd81c074ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:34:26.345 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:26.318+0000 7fd820c00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fd81c111b50 con 0x7fd81c1020a0 2024-08-19T01:34:26.346 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:26.318+0000 7fd820c00640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fd81c1119d0 con 0x7fd81c101700 2024-08-19T01:34:26.346 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:26.318+0000 7fd820c00640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fd81c111cd0 con 0x7fd81c107700 2024-08-19T01:34:26.346 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:26.319+0000 7fd813e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd81c1020a0 0x7fd81c074760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:34:26.346 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:26.319+0000 7fd813e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd81c1020a0 0x7fd81c074760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:53462/0 (socket says 172.21.15.83:53462) 2024-08-19T01:34:26.346 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:26.319+0000 7fd813e00640 1 -- 172.21.15.83:0/1313362147 learned_addr learned my addr 172.21.15.83:0/1313362147 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-08-19T01:34:26.346 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:26.319+0000 7fd81b400640 1 --2- 172.21.15.83:0/1313362147 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd81c101700 0x7fd81c074220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:34:26.346 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:26.319+0000 7fd81be00640 1 --2- 172.21.15.83:0/1313362147 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd81c107700 0x7fd81c074ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:34:26.346 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:26.319+0000 7fd813e00640 1 -- 172.21.15.83:0/1313362147 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd81c107700 msgr2=0x7fd81c074ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:34:26.346 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:26.319+0000 7fd813e00640 1 --2- 172.21.15.83:0/1313362147 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd81c107700 0x7fd81c074ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:26.346 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:26.319+0000 7fd813e00640 1 -- 172.21.15.83:0/1313362147 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd81c101700 msgr2=0x7fd81c074220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:34:26.346 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:26.319+0000 7fd813e00640 1 --2- 172.21.15.83:0/1313362147 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd81c101700 0x7fd81c074220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:26.346 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:26.319+0000 7fd813e00640 1 -- 172.21.15.83:0/1313362147 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd81c1a6490 con 0x7fd81c1020a0 2024-08-19T01:34:26.346 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:26.319+0000 7fd813e00640 1 --2- 172.21.15.83:0/1313362147 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd81c1020a0 0x7fd81c074760 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fd80400d770 tx=0x7fd80400dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:34:26.347 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:26.320+0000 7fd818c00640 1 -- 172.21.15.83:0/1313362147 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd804017070 con 0x7fd81c1020a0 2024-08-19T01:34:26.347 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:26.320+0000 7fd818c00640 1 -- 172.21.15.83:0/1313362147 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd804005500 con 0x7fd81c1020a0 2024-08-19T01:34:26.347 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:26.320+0000 7fd818c00640 1 -- 172.21.15.83:0/1313362147 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd804004e00 con 0x7fd81c1020a0 2024-08-19T01:34:26.347 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:26.320+0000 7fd820c00640 1 -- 172.21.15.83:0/1313362147 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd81c1a6780 con 0x7fd81c1020a0 2024-08-19T01:34:26.347 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:26.320+0000 7fd820c00640 1 -- 172.21.15.83:0/1313362147 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd81c1a6c70 con 0x7fd81c1020a0 2024-08-19T01:34:26.347 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:26.321+0000 7fd820c00640 1 -- 172.21.15.83:0/1313362147 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd81c101b60 con 0x7fd81c1020a0 2024-08-19T01:34:26.347 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:26.323+0000 7fd818c00640 1 -- 172.21.15.83:0/1313362147 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fd804004fa0 con 0x7fd81c1020a0 2024-08-19T01:34:26.347 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:26.325+0000 7fd818c00640 1 --2- 172.21.15.83:0/1313362147 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7fd8000a2c00 0x7fd8000a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:34:26.347 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:26.325+0000 7fd818c00640 1 -- 172.21.15.83:0/1313362147 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8269+0+0 (secure 0 0 0) 0x7fd8040ce9e0 con 0x7fd81c1020a0 2024-08-19T01:34:26.347 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:26.325+0000 7fd81b400640 1 --2- 172.21.15.83:0/1313362147 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7fd8000a2c00 0x7fd8000a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:34:26.347 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:26.325+0000 7fd81b400640 1 --2- 172.21.15.83:0/1313362147 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7fd8000a2c00 0x7fd8000a50c0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fd80c005d00 tx=0x7fd80c00b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:34:26.347 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:26.328+0000 7fd818c00640 1 -- 172.21.15.83:0/1313362147 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7fd80409b270 con 0x7fd81c1020a0 2024-08-19T01:34:26.541 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:26.539+0000 7fd820c00640 1 -- 172.21.15.83:0/1313362147 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd81c1a6f00 con 0x7fd81c1020a0 2024-08-19T01:34:26.541 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:26.540+0000 7fd818c00640 1 -- 172.21.15.83:0/1313362147 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) ==== 94+0+11920 (secure 0 0 0) 0x7fd80409f570 con 0x7fd81c1020a0 2024-08-19T01:34:26.542 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-08-19T01:34:26.542 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":24,"btime":"2024-08-19T01:32:54:291077+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14421,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6835/988556067","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":988556067},{"type":"v1","addr":"172.21.15.83:6835","nonce":988556067}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14433,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6837/869702632","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6836","nonce":869702632},{"type":"v1","addr":"172.21.15.83:6837","nonce":869702632}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14475,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6841/1795975999","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6840","nonce":1795975999},{"type":"v1","addr":"172.21.15.83:6841","nonce":1795975999}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24430,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6833/2505076046","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6832","nonce":2505076046},{"type":"v1","addr":"172.21.15.167:6833","nonce":2505076046}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24449,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6833/3557520478","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":3557520478},{"type":"v1","addr":"172.21.15.117:6833","nonce":3557520478}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24466,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6835/3482045966","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":3482045966},{"type":"v1","addr":"172.21.15.167:6835","nonce":3482045966}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7}],"filesystems":[{"mdsmap":{"epoch":24,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-08-19T01:32:42.054769+0000","modified":"2024-08-19T01:32:54.209056+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24545,"mds_1":24527,"mds_2":24509},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24478":{"gid":24478,"name":"c","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.167:6837/1762130408","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6836","nonce":1762130408},{"type":"v1","addr":"172.21.15.167:6837","nonce":1762130408}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24485":{"gid":24485,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.117:6835/1024213994","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":1024213994},{"type":"v1","addr":"172.21.15.117:6835","nonce":1024213994}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24496":{"gid":24496,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.167:6839/3507387682","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6838","nonce":3507387682},{"type":"v1","addr":"172.21.15.167:6839","nonce":3507387682}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24509":{"gid":24509,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":10,"addr":"172.21.15.117:6837/3185224684","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":3185224684},{"type":"v1","addr":"172.21.15.117:6837","nonce":3185224684}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24527":{"gid":24527,"name":"g","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.83:6839/3239261845","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6838","nonce":3239261845},{"type":"v1","addr":"172.21.15.83:6839","nonce":3239261845}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24545":{"gid":24545,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.117:6839/1967595704","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6838","nonce":1967595704},{"type":"v1","addr":"172.21.15.117:6839","nonce":1967595704}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24545,"qdb_cluster":[24509,24527,24545]},"id":1}]} 2024-08-19T01:34:26.542 INFO:teuthology.orchestra.run.smithi083.stderr:dumped fsmap epoch 24 2024-08-19T01:34:26.545 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:26.542+0000 7fd820c00640 1 -- 172.21.15.83:0/1313362147 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7fd8000a2c00 msgr2=0x7fd8000a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:34:26.545 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:26.542+0000 7fd820c00640 1 --2- 172.21.15.83:0/1313362147 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7fd8000a2c00 0x7fd8000a50c0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fd80c005d00 tx=0x7fd80c00b410 comp rx=0 tx=0).stop 2024-08-19T01:34:26.545 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:26.542+0000 7fd820c00640 1 -- 172.21.15.83:0/1313362147 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd81c1020a0 msgr2=0x7fd81c074760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:34:26.545 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:26.542+0000 7fd820c00640 1 --2- 172.21.15.83:0/1313362147 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd81c1020a0 0x7fd81c074760 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fd80400d770 tx=0x7fd80400dc30 comp rx=0 tx=0).stop 2024-08-19T01:34:26.545 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:26.543+0000 7fd820c00640 1 -- 172.21.15.83:0/1313362147 shutdown_connections 2024-08-19T01:34:26.545 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:26.543+0000 7fd820c00640 1 --2- 172.21.15.83:0/1313362147 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7fd8000a2c00 0x7fd8000a50c0 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:26.545 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:26.543+0000 7fd820c00640 1 --2- 172.21.15.83:0/1313362147 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd81c107700 0x7fd81c074ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:26.545 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:26.543+0000 7fd820c00640 1 --2- 172.21.15.83:0/1313362147 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd81c1020a0 0x7fd81c074760 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:26.545 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:26.543+0000 7fd820c00640 1 --2- 172.21.15.83:0/1313362147 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd81c101700 0x7fd81c074220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:26.545 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:26.543+0000 7fd820c00640 1 -- 172.21.15.83:0/1313362147 >> 172.21.15.83:0/1313362147 conn(0x7fd81c0fd440 msgr2=0x7fd81c0fdaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:34:26.545 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:26.543+0000 7fd820c00640 1 -- 172.21.15.83:0/1313362147 shutdown_connections 2024-08-19T01:34:26.545 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:26.543+0000 7fd820c00640 1 -- 172.21.15.83:0/1313362147 wait complete. 2024-08-19T01:34:27.362 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833 shell --fsid bad8780e-5dc9-11ef-bccf-c7b262605968 -- ceph osd dump --format=json 2024-08-19T01:34:27.654 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:34:27 smithi083 ceph-mon[17120]: pgmap v327: 97 pgs: 97 active+clean; 641 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-08-19T01:34:27.654 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:34:27 smithi083 ceph-mon[17120]: from='client.? 172.21.15.83:0/1313362147' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-19T01:34:27.739 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/bad8780e-5dc9-11ef-bccf-c7b262605968/mon.a/config 2024-08-19T01:34:27.952 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:34:27 smithi117 ceph-mon[22016]: pgmap v327: 97 pgs: 97 active+clean; 641 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-08-19T01:34:27.952 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:34:27 smithi117 ceph-mon[22016]: from='client.? 172.21.15.83:0/1313362147' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-19T01:34:27.966 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:34:27 smithi167 ceph-mon[21875]: pgmap v327: 97 pgs: 97 active+clean; 641 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-08-19T01:34:27.966 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:34:27 smithi167 ceph-mon[21875]: from='client.? 172.21.15.83:0/1313362147' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-19T01:34:28.257 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:28.235+0000 7fb41be00640 1 -- 172.21.15.83:0/401709918 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb41c107700 msgr2=0x7fb41c107ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:34:28.257 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:28.235+0000 7fb41be00640 1 --2- 172.21.15.83:0/401709918 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb41c107700 0x7fb41c107ae0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fb40c009930 tx=0x7fb40c02a440 comp rx=0 tx=0).stop 2024-08-19T01:34:28.257 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:28.235+0000 7fb41be00640 1 -- 172.21.15.83:0/401709918 shutdown_connections 2024-08-19T01:34:28.257 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:28.235+0000 7fb41be00640 1 --2- 172.21.15.83:0/401709918 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb41c1020a0 0x7fb41c10e5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:28.257 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:28.235+0000 7fb41be00640 1 --2- 172.21.15.83:0/401709918 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb41c101700 0x7fb41c101b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:28.257 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:28.235+0000 7fb41be00640 1 --2- 172.21.15.83:0/401709918 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb41c107700 0x7fb41c107ae0 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:28.257 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:28.235+0000 7fb41be00640 1 -- 172.21.15.83:0/401709918 >> 172.21.15.83:0/401709918 conn(0x7fb41c0fd440 msgr2=0x7fb41c0ff860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:34:28.257 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:28.235+0000 7fb41be00640 1 -- 172.21.15.83:0/401709918 shutdown_connections 2024-08-19T01:34:28.258 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:28.235+0000 7fb41be00640 1 -- 172.21.15.83:0/401709918 wait complete. 2024-08-19T01:34:28.258 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:28.236+0000 7fb41be00640 1 Processor -- start 2024-08-19T01:34:28.258 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:28.236+0000 7fb41be00640 1 -- start start 2024-08-19T01:34:28.258 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:28.236+0000 7fb41be00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb41c101700 0x7fb41c19a250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:34:28.258 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:28.236+0000 7fb41be00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb41c1020a0 0x7fb41c19a790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:34:28.258 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:28.236+0000 7fb41be00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb41c107700 0x7fb41c19eb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:34:28.258 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:28.236+0000 7fb41be00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fb41c111b00 con 0x7fb41c1020a0 2024-08-19T01:34:28.258 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:28.236+0000 7fb41be00640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fb41c111980 con 0x7fb41c107700 2024-08-19T01:34:28.258 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:28.236+0000 7fb41be00640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fb41c111c80 con 0x7fb41c101700 2024-08-19T01:34:28.258 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:28.237+0000 7fb41a000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb41c1020a0 0x7fb41c19a790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:34:28.258 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:28.237+0000 7fb41a000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb41c1020a0 0x7fb41c19a790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:53482/0 (socket says 172.21.15.83:53482) 2024-08-19T01:34:28.258 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:28.237+0000 7fb41a000640 1 -- 172.21.15.83:0/1091178831 learned_addr learned my addr 172.21.15.83:0/1091178831 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-08-19T01:34:28.258 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:28.237+0000 7fb41aa00640 1 --2- 172.21.15.83:0/1091178831 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb41c101700 0x7fb41c19a250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:34:28.259 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:28.237+0000 7fb41b400640 1 --2- 172.21.15.83:0/1091178831 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb41c107700 0x7fb41c19eb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:34:28.259 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:28.237+0000 7fb41a000640 1 -- 172.21.15.83:0/1091178831 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb41c101700 msgr2=0x7fb41c19a250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:34:28.259 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:28.237+0000 7fb41a000640 1 --2- 172.21.15.83:0/1091178831 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb41c101700 0x7fb41c19a250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:28.259 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:28.237+0000 7fb41a000640 1 -- 172.21.15.83:0/1091178831 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb41c107700 msgr2=0x7fb41c19eb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:34:28.259 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:28.237+0000 7fb41a000640 1 --2- 172.21.15.83:0/1091178831 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb41c107700 0x7fb41c19eb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:28.259 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:28.237+0000 7fb41a000640 1 -- 172.21.15.83:0/1091178831 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb41c19f200 con 0x7fb41c1020a0 2024-08-19T01:34:28.259 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:28.237+0000 7fb41a000640 1 --2- 172.21.15.83:0/1091178831 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb41c1020a0 0x7fb41c19a790 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fb40800e9f0 tx=0x7fb40800edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:34:28.259 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:28.237+0000 7fb413400640 1 -- 172.21.15.83:0/1091178831 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb40800cd80 con 0x7fb41c1020a0 2024-08-19T01:34:28.259 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:28.237+0000 7fb413400640 1 -- 172.21.15.83:0/1091178831 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb408005500 con 0x7fb41c1020a0 2024-08-19T01:34:28.259 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:28.237+0000 7fb41be00640 1 -- 172.21.15.83:0/1091178831 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb41c19f4f0 con 0x7fb41c1020a0 2024-08-19T01:34:28.259 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:28.238+0000 7fb413400640 1 -- 172.21.15.83:0/1091178831 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb408010690 con 0x7fb41c1020a0 2024-08-19T01:34:28.259 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:28.238+0000 7fb41be00640 1 -- 172.21.15.83:0/1091178831 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb41c1a6dd0 con 0x7fb41c1020a0 2024-08-19T01:34:28.259 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:28.239+0000 7fb41be00640 1 -- 172.21.15.83:0/1091178831 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb41c101b60 con 0x7fb41c1020a0 2024-08-19T01:34:28.259 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:28.241+0000 7fb413400640 1 -- 172.21.15.83:0/1091178831 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fb4080046c0 con 0x7fb41c1020a0 2024-08-19T01:34:28.260 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:28.242+0000 7fb413400640 1 --2- 172.21.15.83:0/1091178831 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7fb4000a2b80 0x7fb4000a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:34:28.260 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:28.242+0000 7fb413400640 1 -- 172.21.15.83:0/1091178831 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8269+0+0 (secure 0 0 0) 0x7fb4080cf4a0 con 0x7fb41c1020a0 2024-08-19T01:34:28.260 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:28.242+0000 7fb41aa00640 1 --2- 172.21.15.83:0/1091178831 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7fb4000a2b80 0x7fb4000a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:34:28.260 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:28.243+0000 7fb41aa00640 1 --2- 172.21.15.83:0/1091178831 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7fb4000a2b80 0x7fb4000a5040 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fb40c009900 tx=0x7fb40c0058d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:34:28.260 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:28.245+0000 7fb413400640 1 -- 172.21.15.83:0/1091178831 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7fb40809c380 con 0x7fb41c1020a0 2024-08-19T01:34:28.422 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:28.420+0000 7fb41be00640 1 -- 172.21.15.83:0/1091178831 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fb41c1a7100 con 0x7fb41c1020a0 2024-08-19T01:34:28.423 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:28.422+0000 7fb413400640 1 -- 172.21.15.83:0/1091178831 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v86) ==== 74+0+22722 (secure 0 0 0) 0x7fb40809bc80 con 0x7fb41c1020a0 2024-08-19T01:34:28.424 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-08-19T01:34:28.424 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":86,"fsid":"bad8780e-5dc9-11ef-bccf-c7b262605968","created":"2024-08-19T01:25:12.626841+0000","modified":"2024-08-19T01:33:05.461998+0000","last_up_change":"2024-08-19T01:31:47.254559+0000","last_in_change":"2024-08-19T01:31:27.652847+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-08-19T01:28:16.862007+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-08-19T01:32:37.455535+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-08-19T01:32:40.201935+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ffa74041-f9d2-4555-bb43-508b8c581487","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6802","nonce":1082526824},{"type":"v1","addr":"172.21.15.83:6803","nonce":1082526824}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6804","nonce":1082526824},{"type":"v1","addr":"172.21.15.83:6805","nonce":1082526824}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6808","nonce":1082526824},{"type":"v1","addr":"172.21.15.83:6809","nonce":1082526824}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6806","nonce":1082526824},{"type":"v1","addr":"172.21.15.83:6807","nonce":1082526824}]},"public_addr":"172.21.15.83:6803/1082526824","cluster_addr":"172.21.15.83:6805/1082526824","heartbeat_back_addr":"172.21.15.83:6809/1082526824","heartbeat_front_addr":"172.21.15.83:6807/1082526824","state":["exists","up"]},{"osd":1,"uuid":"1977570b-b4f5-44fd-bb1e-3a31318be649","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6800","nonce":1925419802},{"type":"v1","addr":"172.21.15.117:6801","nonce":1925419802}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6802","nonce":1925419802},{"type":"v1","addr":"172.21.15.117:6803","nonce":1925419802}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6806","nonce":1925419802},{"type":"v1","addr":"172.21.15.117:6807","nonce":1925419802}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6804","nonce":1925419802},{"type":"v1","addr":"172.21.15.117:6805","nonce":1925419802}]},"public_addr":"172.21.15.117:6801/1925419802","cluster_addr":"172.21.15.117:6803/1925419802","heartbeat_back_addr":"172.21.15.117:6807/1925419802","heartbeat_front_addr":"172.21.15.117:6805/1925419802","state":["exists","up"]},{"osd":2,"uuid":"a75278d1-161f-41d5-a92f-058d1077bf2a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6800","nonce":551625328},{"type":"v1","addr":"172.21.15.167:6801","nonce":551625328}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6802","nonce":551625328},{"type":"v1","addr":"172.21.15.167:6803","nonce":551625328}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6806","nonce":551625328},{"type":"v1","addr":"172.21.15.167:6807","nonce":551625328}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6804","nonce":551625328},{"type":"v1","addr":"172.21.15.167:6805","nonce":551625328}]},"public_addr":"172.21.15.167:6801/551625328","cluster_addr":"172.21.15.167:6803/551625328","heartbeat_back_addr":"172.21.15.167:6807/551625328","heartbeat_front_addr":"172.21.15.167:6805/551625328","state":["exists","up"]},{"osd":3,"uuid":"d1fb16f9-910e-46fa-b1e3-8e5bef4d346d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6810","nonce":3138942062},{"type":"v1","addr":"172.21.15.83:6811","nonce":3138942062}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6812","nonce":3138942062},{"type":"v1","addr":"172.21.15.83:6813","nonce":3138942062}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6816","nonce":3138942062},{"type":"v1","addr":"172.21.15.83:6817","nonce":3138942062}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6814","nonce":3138942062},{"type":"v1","addr":"172.21.15.83:6815","nonce":3138942062}]},"public_addr":"172.21.15.83:6811/3138942062","cluster_addr":"172.21.15.83:6813/3138942062","heartbeat_back_addr":"172.21.15.83:6817/3138942062","heartbeat_front_addr":"172.21.15.83:6815/3138942062","state":["exists","up"]},{"osd":4,"uuid":"8176ed18-1fe2-446d-8994-8c60f6e08f38","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6808","nonce":3978863827},{"type":"v1","addr":"172.21.15.117:6809","nonce":3978863827}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6810","nonce":3978863827},{"type":"v1","addr":"172.21.15.117:6811","nonce":3978863827}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6814","nonce":3978863827},{"type":"v1","addr":"172.21.15.117:6815","nonce":3978863827}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6812","nonce":3978863827},{"type":"v1","addr":"172.21.15.117:6813","nonce":3978863827}]},"public_addr":"172.21.15.117:6809/3978863827","cluster_addr":"172.21.15.117:6811/3978863827","heartbeat_back_addr":"172.21.15.117:6815/3978863827","heartbeat_front_addr":"172.21.15.117:6813/3978863827","state":["exists","up"]},{"osd":5,"uuid":"e01e3f15-25c8-4d2c-be6f-a170f036064c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6808","nonce":2761858035},{"type":"v1","addr":"172.21.15.167:6809","nonce":2761858035}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6810","nonce":2761858035},{"type":"v1","addr":"172.21.15.167:6811","nonce":2761858035}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6814","nonce":2761858035},{"type":"v1","addr":"172.21.15.167:6815","nonce":2761858035}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6812","nonce":2761858035},{"type":"v1","addr":"172.21.15.167:6813","nonce":2761858035}]},"public_addr":"172.21.15.167:6809/2761858035","cluster_addr":"172.21.15.167:6811/2761858035","heartbeat_back_addr":"172.21.15.167:6815/2761858035","heartbeat_front_addr":"172.21.15.167:6813/2761858035","state":["exists","up"]},{"osd":6,"uuid":"78cf0a9b-9165-444f-b073-ebe13016de9d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6818","nonce":260644552},{"type":"v1","addr":"172.21.15.83:6819","nonce":260644552}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6820","nonce":260644552},{"type":"v1","addr":"172.21.15.83:6821","nonce":260644552}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6824","nonce":260644552},{"type":"v1","addr":"172.21.15.83:6825","nonce":260644552}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6822","nonce":260644552},{"type":"v1","addr":"172.21.15.83:6823","nonce":260644552}]},"public_addr":"172.21.15.83:6819/260644552","cluster_addr":"172.21.15.83:6821/260644552","heartbeat_back_addr":"172.21.15.83:6825/260644552","heartbeat_front_addr":"172.21.15.83:6823/260644552","state":["exists","up"]},{"osd":7,"uuid":"ee524f7f-672e-430e-9f58-1a9117513466","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6816","nonce":104113252},{"type":"v1","addr":"172.21.15.117:6817","nonce":104113252}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6818","nonce":104113252},{"type":"v1","addr":"172.21.15.117:6819","nonce":104113252}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6822","nonce":104113252},{"type":"v1","addr":"172.21.15.117:6823","nonce":104113252}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6820","nonce":104113252},{"type":"v1","addr":"172.21.15.117:6821","nonce":104113252}]},"public_addr":"172.21.15.117:6817/104113252","cluster_addr":"172.21.15.117:6819/104113252","heartbeat_back_addr":"172.21.15.117:6823/104113252","heartbeat_front_addr":"172.21.15.117:6821/104113252","state":["exists","up"]},{"osd":8,"uuid":"76d5317d-4845-4138-b210-e7bd2d80472a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":58,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6816","nonce":2196898857},{"type":"v1","addr":"172.21.15.167:6817","nonce":2196898857}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6818","nonce":2196898857},{"type":"v1","addr":"172.21.15.167:6819","nonce":2196898857}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6822","nonce":2196898857},{"type":"v1","addr":"172.21.15.167:6823","nonce":2196898857}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6820","nonce":2196898857},{"type":"v1","addr":"172.21.15.167:6821","nonce":2196898857}]},"public_addr":"172.21.15.167:6817/2196898857","cluster_addr":"172.21.15.167:6819/2196898857","heartbeat_back_addr":"172.21.15.167:6823/2196898857","heartbeat_front_addr":"172.21.15.167:6821/2196898857","state":["exists","up"]},{"osd":9,"uuid":"0151bc2a-3ad2-4db3-8505-5855792cc9bb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6826","nonce":903463450},{"type":"v1","addr":"172.21.15.83:6827","nonce":903463450}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6828","nonce":903463450},{"type":"v1","addr":"172.21.15.83:6829","nonce":903463450}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6832","nonce":903463450},{"type":"v1","addr":"172.21.15.83:6833","nonce":903463450}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6830","nonce":903463450},{"type":"v1","addr":"172.21.15.83:6831","nonce":903463450}]},"public_addr":"172.21.15.83:6827/903463450","cluster_addr":"172.21.15.83:6829/903463450","heartbeat_back_addr":"172.21.15.83:6833/903463450","heartbeat_front_addr":"172.21.15.83:6831/903463450","state":["exists","up"]},{"osd":10,"uuid":"437ed848-0da4-4cd4-8716-0c55bb99b676","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6824","nonce":1183570837},{"type":"v1","addr":"172.21.15.117:6825","nonce":1183570837}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6826","nonce":1183570837},{"type":"v1","addr":"172.21.15.117:6827","nonce":1183570837}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6830","nonce":1183570837},{"type":"v1","addr":"172.21.15.117:6831","nonce":1183570837}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6828","nonce":1183570837},{"type":"v1","addr":"172.21.15.117:6829","nonce":1183570837}]},"public_addr":"172.21.15.117:6825/1183570837","cluster_addr":"172.21.15.117:6827/1183570837","heartbeat_back_addr":"172.21.15.117:6831/1183570837","heartbeat_front_addr":"172.21.15.117:6829/1183570837","state":["exists","up"]},{"osd":11,"uuid":"13345f05-cdc0-4dae-a58b-551abbec1915","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":78,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6824","nonce":187027350},{"type":"v1","addr":"172.21.15.167:6825","nonce":187027350}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6826","nonce":187027350},{"type":"v1","addr":"172.21.15.167:6827","nonce":187027350}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6830","nonce":187027350},{"type":"v1","addr":"172.21.15.167:6831","nonce":187027350}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6828","nonce":187027350},{"type":"v1","addr":"172.21.15.167:6829","nonce":187027350}]},"public_addr":"172.21.15.167:6825/187027350","cluster_addr":"172.21.15.167:6827/187027350","heartbeat_back_addr":"172.21.15.167:6831/187027350","heartbeat_front_addr":"172.21.15.167:6829/187027350","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-19T01:27:28.760517+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-19T01:27:50.493028+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-19T01:28:11.997674+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-19T01:28:38.109934+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-19T01:29:00.173065+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-19T01:29:21.336085+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-19T01:29:47.871498+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-19T01:30:10.720498+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-19T01:30:32.425438+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-19T01:30:59.524989+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-19T01:31:22.806162+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-19T01:31:45.043246+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.1a","mappings":[{"from":8,"to":1}]},{"pgid":"2.1b","mappings":[{"from":2,"to":4}]},{"pgid":"2.1d","mappings":[{"from":8,"to":11}]},{"pgid":"3.6","mappings":[{"from":4,"to":6}]},{"pgid":"3.11","mappings":[{"from":4,"to":8}]},{"pgid":"3.1b","mappings":[{"from":5,"to":9}]},{"pgid":"3.20","mappings":[{"from":5,"to":9}]},{"pgid":"3.31","mappings":[{"from":4,"to":7}]},{"pgid":"3.3b","mappings":[{"from":3,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.83:0/3138095466":"2024-08-20T01:26:04.277716+0000","172.21.15.83:0/3835005198":"2024-08-20T01:26:04.277716+0000","172.21.15.83:0/1790374088":"2024-08-20T01:25:37.228583+0000","172.21.15.83:6801/573907319":"2024-08-20T01:26:04.277716+0000","172.21.15.83:6800/573907319":"2024-08-20T01:26:04.277716+0000","172.21.15.83:0/664676002":"2024-08-20T01:25:37.228583+0000","172.21.15.83:0/2144611449":"2024-08-20T01:25:37.228583+0000","172.21.15.83:6801/3261152818":"2024-08-20T01:25:37.228583+0000","172.21.15.83:0/1609552666":"2024-08-20T01:26:04.277716+0000","172.21.15.83:6800/3261152818":"2024-08-20T01:25:37.228583+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-08-19T01:34:28.426 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:28.424+0000 7fb41be00640 1 -- 172.21.15.83:0/1091178831 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7fb4000a2b80 msgr2=0x7fb4000a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:34:28.426 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:28.424+0000 7fb41be00640 1 --2- 172.21.15.83:0/1091178831 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7fb4000a2b80 0x7fb4000a5040 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fb40c009900 tx=0x7fb40c0058d0 comp rx=0 tx=0).stop 2024-08-19T01:34:28.426 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:28.424+0000 7fb41be00640 1 -- 172.21.15.83:0/1091178831 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb41c1020a0 msgr2=0x7fb41c19a790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:34:28.426 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:28.424+0000 7fb41be00640 1 --2- 172.21.15.83:0/1091178831 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb41c1020a0 0x7fb41c19a790 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fb40800e9f0 tx=0x7fb40800edf0 comp rx=0 tx=0).stop 2024-08-19T01:34:28.426 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:28.424+0000 7fb41be00640 1 -- 172.21.15.83:0/1091178831 shutdown_connections 2024-08-19T01:34:28.426 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:28.424+0000 7fb41be00640 1 --2- 172.21.15.83:0/1091178831 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7fb4000a2b80 0x7fb4000a5040 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:28.426 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:28.424+0000 7fb41be00640 1 --2- 172.21.15.83:0/1091178831 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb41c107700 0x7fb41c19eb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:28.427 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:28.424+0000 7fb41be00640 1 --2- 172.21.15.83:0/1091178831 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb41c1020a0 0x7fb41c19a790 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:28.427 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:28.424+0000 7fb41be00640 1 --2- 172.21.15.83:0/1091178831 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb41c101700 0x7fb41c19a250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:28.427 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:28.424+0000 7fb41be00640 1 -- 172.21.15.83:0/1091178831 >> 172.21.15.83:0/1091178831 conn(0x7fb41c0fd440 msgr2=0x7fb41c10cd50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:34:28.427 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:28.425+0000 7fb41be00640 1 -- 172.21.15.83:0/1091178831 shutdown_connections 2024-08-19T01:34:28.427 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:28.425+0000 7fb41be00640 1 -- 172.21.15.83:0/1091178831 wait complete. 2024-08-19T01:34:28.796 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:34:28 smithi083 ceph-mon[17120]: from='client.? 172.21.15.83:0/1091178831' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-08-19T01:34:28.952 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:34:28 smithi117 ceph-mon[22016]: from='client.? 172.21.15.83:0/1091178831' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-08-19T01:34:28.965 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:34:28 smithi167 ceph-mon[21875]: from='client.? 172.21.15.83:0/1091178831' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-08-19T01:34:29.095 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 24, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-08-19T01:32:42.054769+0000', 'modified': '2024-08-19T01:32:54.209056+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 3, 'in': [0, 1, 2], 'up': {'mds_0': 24545, 'mds_1': 24527, 'mds_2': 24509}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_24478': {'gid': 24478, 'name': 'c', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.167:6837/1762130408', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.167:6836', 'nonce': 1762130408}, {'type': 'v1', 'addr': '172.21.15.167:6837', 'nonce': 1762130408}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24485': {'gid': 24485, 'name': 'e', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.117:6835/1024213994', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.117:6834', 'nonce': 1024213994}, {'type': 'v1', 'addr': '172.21.15.117:6835', 'nonce': 1024213994}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24496': {'gid': 24496, 'name': 'i', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.167:6839/3507387682', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.167:6838', 'nonce': 3507387682}, {'type': 'v1', 'addr': '172.21.15.167:6839', 'nonce': 3507387682}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24509': {'gid': 24509, 'name': 'h', 'rank': 2, 'incarnation': 21, 'state': 'up:active', 'state_seq': 10, 'addr': '172.21.15.117:6837/3185224684', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.117:6836', 'nonce': 3185224684}, {'type': 'v1', 'addr': '172.21.15.117:6837', 'nonce': 3185224684}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24527': {'gid': 24527, 'name': 'g', 'rank': 1, 'incarnation': 19, 'state': 'up:active', 'state_seq': 8, 'addr': '172.21.15.83:6839/3239261845', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.83:6838', 'nonce': 3239261845}, {'type': 'v1', 'addr': '172.21.15.83:6839', 'nonce': 3239261845}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24545': {'gid': 24545, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 5, 'addr': '172.21.15.117:6839/1967595704', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.117:6838', 'nonce': 1967595704}, {'type': 'v1', 'addr': '172.21.15.117:6839', 'nonce': 1967595704}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 24545, 'qdb_cluster': [24509, 24527, 24545]} 2024-08-19T01:34:29.095 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 3/3 2024-08-19T01:34:29.095 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833 shell --fsid bad8780e-5dc9-11ef-bccf-c7b262605968 -- ceph tell mds.h status 2024-08-19T01:34:29.470 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/bad8780e-5dc9-11ef-bccf-c7b262605968/mon.a/config 2024-08-19T01:34:29.904 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:34:29 smithi083 ceph-mon[17120]: pgmap v328: 97 pgs: 97 active+clean; 641 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-08-19T01:34:29.952 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:34:29 smithi117 ceph-mon[22016]: pgmap v328: 97 pgs: 97 active+clean; 641 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-08-19T01:34:29.966 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:34:29 smithi167 ceph-mon[21875]: pgmap v328: 97 pgs: 97 active+clean; 641 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-08-19T01:34:30.028 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:30.027+0000 7f9ad1a00640 1 -- 172.21.15.83:0/626909033 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9acc1018b0 msgr2=0x7f9acc101d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:34:30.029 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:30.027+0000 7f9ad1a00640 1 --2- 172.21.15.83:0/626909033 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9acc1018b0 0x7f9acc101d10 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f9ab4009900 tx=0x7f9ab402a3f0 comp rx=0 tx=0).stop 2024-08-19T01:34:30.029 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:30.027+0000 7f9ad1a00640 1 -- 172.21.15.83:0/626909033 shutdown_connections 2024-08-19T01:34:30.029 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:30.027+0000 7f9ad1a00640 1 --2- 172.21.15.83:0/626909033 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9acc102250 0x7f9acc10e780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:30.029 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:30.027+0000 7f9ad1a00640 1 --2- 172.21.15.83:0/626909033 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9acc1018b0 0x7f9acc101d10 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:30.029 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:30.027+0000 7f9ad1a00640 1 --2- 172.21.15.83:0/626909033 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9acc1078b0 0x7f9acc107c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:30.029 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:30.027+0000 7f9ad1a00640 1 -- 172.21.15.83:0/626909033 >> 172.21.15.83:0/626909033 conn(0x7f9acc0fd5f0 msgr2=0x7f9acc0ffa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:34:30.029 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:30.027+0000 7f9ad1a00640 1 -- 172.21.15.83:0/626909033 shutdown_connections 2024-08-19T01:34:30.029 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:30.027+0000 7f9ad1a00640 1 -- 172.21.15.83:0/626909033 wait complete. 2024-08-19T01:34:30.030 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:30.028+0000 7f9ad1a00640 1 Processor -- start 2024-08-19T01:34:30.030 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:30.028+0000 7f9ad1a00640 1 -- start start 2024-08-19T01:34:30.031 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:30.028+0000 7f9ad1a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9acc1018b0 0x7f9acc19a370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:34:30.031 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:30.028+0000 7f9ad1a00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9acc102250 0x7f9acc19a8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:34:30.031 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:30.028+0000 7f9ad1a00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9acc1078b0 0x7f9acc19ec40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:34:30.031 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:30.028+0000 7f9ad1a00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f9acc111d00 con 0x7f9acc1018b0 2024-08-19T01:34:30.031 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:30.028+0000 7f9ad1a00640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f9acc111b80 con 0x7f9acc102250 2024-08-19T01:34:30.031 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:30.028+0000 7f9ad1a00640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f9acc111e80 con 0x7f9acc1078b0 2024-08-19T01:34:30.031 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:30.028+0000 7f9acaa00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9acc1018b0 0x7f9acc19a370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:34:30.031 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:30.028+0000 7f9acaa00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9acc1018b0 0x7f9acc19a370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:53500/0 (socket says 172.21.15.83:53500) 2024-08-19T01:34:30.031 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:30.028+0000 7f9acaa00640 1 -- 172.21.15.83:0/2053656325 learned_addr learned my addr 172.21.15.83:0/2053656325 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-08-19T01:34:30.031 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:30.029+0000 7f9ac2000640 1 --2- 172.21.15.83:0/2053656325 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9acc102250 0x7f9acc19a8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:34:30.031 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:30.029+0000 7f9acb400640 1 --2- 172.21.15.83:0/2053656325 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9acc1078b0 0x7f9acc19ec40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:34:30.031 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:30.029+0000 7f9acaa00640 1 -- 172.21.15.83:0/2053656325 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9acc1078b0 msgr2=0x7f9acc19ec40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:34:30.031 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:30.029+0000 7f9acaa00640 1 --2- 172.21.15.83:0/2053656325 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9acc1078b0 0x7f9acc19ec40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:30.031 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:30.029+0000 7f9acaa00640 1 -- 172.21.15.83:0/2053656325 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9acc102250 msgr2=0x7f9acc19a8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:34:30.032 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:30.029+0000 7f9acaa00640 1 --2- 172.21.15.83:0/2053656325 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9acc102250 0x7f9acc19a8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:30.032 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:30.029+0000 7f9acaa00640 1 -- 172.21.15.83:0/2053656325 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9acc19f320 con 0x7f9acc1018b0 2024-08-19T01:34:30.032 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:30.029+0000 7f9acaa00640 1 --2- 172.21.15.83:0/2053656325 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9acc1018b0 0x7f9acc19a370 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f9abc00ea60 tx=0x7f9abc00ef20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:34:30.032 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:30.030+0000 7f9ac3e00640 1 -- 172.21.15.83:0/2053656325 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9abc00cde0 con 0x7f9acc1018b0 2024-08-19T01:34:30.032 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:30.030+0000 7f9ad1a00640 1 -- 172.21.15.83:0/2053656325 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9acc19f610 con 0x7f9acc1018b0 2024-08-19T01:34:30.032 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:30.030+0000 7f9ac3e00640 1 -- 172.21.15.83:0/2053656325 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9abc0035f0 con 0x7f9acc1018b0 2024-08-19T01:34:30.033 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:30.030+0000 7f9ac3e00640 1 -- 172.21.15.83:0/2053656325 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9abc010670 con 0x7f9acc1018b0 2024-08-19T01:34:30.033 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:30.030+0000 7f9ad1a00640 1 -- 172.21.15.83:0/2053656325 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9acc1a6ef0 con 0x7f9acc1018b0 2024-08-19T01:34:30.034 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:30.033+0000 7f9ac3e00640 1 -- 172.21.15.83:0/2053656325 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f9abc0037a0 con 0x7f9acc1018b0 2024-08-19T01:34:30.034 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:30.033+0000 7f9ad4436740 1 Processor -- start 2024-08-19T01:34:30.034 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:30.033+0000 7f9ad4436740 1 -- start start 2024-08-19T01:34:30.035 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:30.033+0000 7f9ad4436740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5590b0ba4880 0x5590b0ba2dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:34:30.035 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:30.033+0000 7f9ad4436740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5590b0ba3310 0x5590b0ba3750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:34:30.035 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:30.033+0000 7f9ad4436740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5590b0bae0b0 0x5590b0bb04a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:34:30.035 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:30.033+0000 7f9ad4436740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x5590b0b09940 con 0x5590b0ba3310 2024-08-19T01:34:30.035 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:30.033+0000 7f9ad4436740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x5590b0aa19a0 con 0x5590b0ba4880 2024-08-19T01:34:30.036 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:30.033+0000 7f9ad4436740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x5590b0b05900 con 0x5590b0bae0b0 2024-08-19T01:34:30.036 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:30.034+0000 7f9acb400640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5590b0ba4880 0x5590b0ba2dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:34:30.036 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:30.034+0000 7f9acb400640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5590b0ba4880 0x5590b0ba2dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.83:43958/0 (socket says 172.21.15.83:43958) 2024-08-19T01:34:30.036 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:30.034+0000 7f9acaa00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5590b0ba3310 0x5590b0ba3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:34:30.036 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:30.034+0000 7f9acb400640 1 -- 172.21.15.83:0/3137942151 learned_addr learned my addr 172.21.15.83:0/3137942151 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-08-19T01:34:30.036 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:30.034+0000 7f9ac2000640 1 --2- 172.21.15.83:0/3137942151 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5590b0bae0b0 0x5590b0bb04a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:34:30.037 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:30.034+0000 7f9acaa00640 1 -- 172.21.15.83:0/3137942151 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5590b0bae0b0 msgr2=0x5590b0bb04a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:34:30.037 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:30.034+0000 7f9acaa00640 1 --2- 172.21.15.83:0/3137942151 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5590b0bae0b0 0x5590b0bb04a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:30.037 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:30.034+0000 7f9acaa00640 1 -- 172.21.15.83:0/3137942151 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5590b0ba4880 msgr2=0x5590b0ba2dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:34:30.037 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:30.034+0000 7f9acaa00640 1 --2- 172.21.15.83:0/3137942151 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5590b0ba4880 0x5590b0ba2dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:30.037 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:30.034+0000 7f9acaa00640 1 -- 172.21.15.83:0/3137942151 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5590b0a8fef0 con 0x5590b0ba3310 2024-08-19T01:34:30.037 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:30.034+0000 7f9acb400640 1 --2- 172.21.15.83:0/3137942151 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5590b0ba4880 0x5590b0ba2dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-19T01:34:30.037 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:30.034+0000 7f9ac3e00640 1 --2- 172.21.15.83:0/2053656325 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f9ab00a2b80 0x7f9ab00a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:34:30.037 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:30.034+0000 7f9acaa00640 1 --2- 172.21.15.83:0/3137942151 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5590b0ba3310 0x5590b0ba3750 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f9abc0ccdd0 tx=0x7f9abc0d7820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=876fd542e0770330 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:34:30.037 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:30.035+0000 7f9acb400640 1 --2- 172.21.15.83:0/2053656325 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f9ab00a2b80 0x7f9ab00a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:34:30.037 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:30.035+0000 7f9ac3e00640 1 -- 172.21.15.83:0/2053656325 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8269+0+0 (secure 0 0 0) 0x7f9abc0cf320 con 0x7f9acc1018b0 2024-08-19T01:34:30.037 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:30.035+0000 7f9abbe00640 1 -- 172.21.15.83:0/3137942151 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9abc097e00 con 0x5590b0ba3310 2024-08-19T01:34:30.037 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:30.035+0000 7f9abbe00640 1 -- 172.21.15.83:0/3137942151 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9abc09c260 con 0x5590b0ba3310 2024-08-19T01:34:30.038 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:30.035+0000 7f9abbe00640 1 -- 172.21.15.83:0/3137942151 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9abc09b4c0 con 0x5590b0ba3310 2024-08-19T01:34:30.039 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:30.036+0000 7f9ad4436740 1 -- 172.21.15.83:0/3137942151 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5590b0ba3310 msgr2=0x5590b0ba3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:34:30.039 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:30.036+0000 7f9ad4436740 1 --2- 172.21.15.83:0/3137942151 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5590b0ba3310 0x5590b0ba3750 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f9abc0ccdd0 tx=0x7f9abc0d7820 comp rx=0 tx=0).stop 2024-08-19T01:34:30.039 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:30.036+0000 7f9acb400640 1 --2- 172.21.15.83:0/2053656325 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f9ab00a2b80 0x7f9ab00a5040 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f9ac4011890 tx=0x7f9ac4009500 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:34:30.039 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:30.036+0000 7f9ad4436740 1 -- 172.21.15.83:0/3137942151 shutdown_connections 2024-08-19T01:34:30.039 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:30.036+0000 7f9ad4436740 1 --2- 172.21.15.83:0/3137942151 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5590b0bae0b0 0x5590b0bb04a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:30.039 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:30.036+0000 7f9ad4436740 1 --2- 172.21.15.83:0/3137942151 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5590b0ba3310 0x5590b0ba3750 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:30.039 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:30.036+0000 7f9ad4436740 1 --2- 172.21.15.83:0/3137942151 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5590b0ba4880 0x5590b0ba2dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:30.039 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:30.036+0000 7f9ad4436740 1 -- 172.21.15.83:0/3137942151 >> 172.21.15.83:0/3137942151 conn(0x5590b0b6ba00 msgr2=0x5590b0b9f9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:34:30.039 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:30.036+0000 7f9ad4436740 1 -- 172.21.15.83:0/3137942151 shutdown_connections 2024-08-19T01:34:30.040 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:30.036+0000 7f9ad4436740 1 -- 172.21.15.83:0/3137942151 wait complete. 2024-08-19T01:34:30.040 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:30.036+0000 7f9ad4436740 1 -- start start 2024-08-19T01:34:30.040 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:30.037+0000 7f9ad4436740 1 Processor -- start 2024-08-19T01:34:30.040 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:30.037+0000 7f9ad4436740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5590b0ba3310 0x5590b0bab060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:34:30.040 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:30.037+0000 7f9ad4436740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5590b0ba4880 0x5590b0bab5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:34:30.040 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:30.037+0000 7f9ad4436740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5590b0bae0b0 0x5590b0babae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:34:30.040 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:30.037+0000 7f9ad4436740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x5590b0bb0db0 con 0x5590b0ba3310 2024-08-19T01:34:30.040 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:30.037+0000 7f9ad4436740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x5590b0ac4a80 con 0x5590b0ba4880 2024-08-19T01:34:30.040 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:30.037+0000 7f9ad4436740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x5590b0bb0f30 con 0x5590b0bae0b0 2024-08-19T01:34:30.040 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:30.037+0000 7f9ad4436740 10 client.0 _ll_register_callbacks cb 0x5590b0a8d140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-19T01:34:30.040 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:30.037+0000 7f9acaa00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5590b0ba3310 0x5590b0bab060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:34:30.040 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:30.037+0000 7f9acaa00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5590b0ba3310 0x5590b0bab060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:53518/0 (socket says 172.21.15.83:53518) 2024-08-19T01:34:30.040 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:30.037+0000 7f9acaa00640 1 -- 172.21.15.83:0/3900691463 learned_addr learned my addr 172.21.15.83:0/3900691463 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-08-19T01:34:30.040 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:30.037+0000 7f9ac2000640 1 --2- 172.21.15.83:0/3900691463 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5590b0ba4880 0x5590b0bab5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:34:30.041 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:30.038+0000 7f9acb400640 1 --2- 172.21.15.83:0/3900691463 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5590b0bae0b0 0x5590b0babae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:34:30.042 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:30.038+0000 7f9acaa00640 1 -- 172.21.15.83:0/3900691463 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5590b0bae0b0 msgr2=0x5590b0babae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:34:30.042 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:30.038+0000 7f9acaa00640 1 --2- 172.21.15.83:0/3900691463 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5590b0bae0b0 0x5590b0babae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:30.042 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:30.038+0000 7f9acaa00640 1 -- 172.21.15.83:0/3900691463 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5590b0ba4880 msgr2=0x5590b0bab5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:34:30.043 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:30.038+0000 7f9acaa00640 1 --2- 172.21.15.83:0/3900691463 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5590b0ba4880 0x5590b0bab5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:30.043 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:30.038+0000 7f9acaa00640 1 -- 172.21.15.83:0/3900691463 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5590b0aa8700 con 0x5590b0ba3310 2024-08-19T01:34:30.043 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:30.038+0000 7f9acaa00640 1 --2- 172.21.15.83:0/3900691463 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5590b0ba3310 0x5590b0bab060 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f9abc09c410 tx=0x7f9abc0a47d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d047f20d05ac04e4 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:34:30.043 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:30.038+0000 7f9ab8c00640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-08-19T01:34:30.043 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:30.038+0000 7f9ab8c00640 1 -- 172.21.15.83:0/3900691463 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9abc09b6a0 con 0x5590b0ba3310 2024-08-19T01:34:30.043 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:30.038+0000 7f9ab8c00640 1 -- 172.21.15.83:0/3900691463 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8269+0+0 (secure 0 0 0) 0x7f9abc003b30 con 0x5590b0ba3310 2024-08-19T01:34:30.043 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:30.038+0000 7f9ad4436740 1 -- 172.21.15.83:0/3900691463 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9abc09c260 con 0x5590b0ba3310 2024-08-19T01:34:30.043 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:30.038+0000 7f9ab8c00640 1 -- 172.21.15.83:0/3900691463 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f9abc0b3050 con 0x5590b0ba3310 2024-08-19T01:34:30.043 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:30.039+0000 7f9ad4436740 10 client.14937 fetch_fsmap learned FSMap version 24 2024-08-19T01:34:30.043 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:30.039+0000 7f9ad4436740 1 -- 172.21.15.83:0/3900691463 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x5590b087b2b0 con 0x5590b0ba3310 2024-08-19T01:34:30.043 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:30.039+0000 7f9ab8c00640 1 -- 172.21.15.83:0/3900691463 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9abc0ae070 con 0x5590b0ba3310 2024-08-19T01:34:30.043 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:30.039+0000 7f9ab8c00640 1 -- 172.21.15.83:0/3900691463 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 24) ==== 8346+0+0 (secure 0 0 0) 0x7f9abc09bdc0 con 0x5590b0ba3310 2024-08-19T01:34:30.043 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:30.039+0000 7f9ad4436740 10 client.14937 fetch_fsmap finished waiting for FSMap version 24 2024-08-19T01:34:30.043 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:30.039+0000 7f9ad4436740 10 client.14937 resolve_mds: resolved name 'h' to daemon mds.h 2024-08-19T01:34:30.044 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:30.039+0000 7f9ad4436740 20 client.14937 populate_metadata read hostname 'smithi083' 2024-08-19T01:34:30.044 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:30.039+0000 7f9ad4436740 1 --2- 172.21.15.83:0/3900691463 >> [v2:172.21.15.117:6836/3185224684,v1:172.21.15.117:6837/3185224684] conn(0x5590b0bbdc50 0x5590b0bbfcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:34:30.044 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:30.039+0000 7f9ad4436740 4 client.14937 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-19T01:34:30.044 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:30.039+0000 7f9ad4436740 1 -- 172.21.15.83:0/3900691463 --> [v2:172.21.15.117:6836/3185224684,v1:172.21.15.117:6837/3185224684] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5590b0b16eb0 con 0x5590b0bbdc50 2024-08-19T01:34:30.044 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:30.040+0000 7f9ac2000640 1 --2- 172.21.15.83:0/3900691463 >> [v2:172.21.15.117:6836/3185224684,v1:172.21.15.117:6837/3185224684] conn(0x5590b0bbdc50 0x5590b0bbfcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:34:30.044 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:30.040+0000 7f9ac2000640 1 --2- 172.21.15.83:0/3900691463 >> [v2:172.21.15.117:6836/3185224684,v1:172.21.15.117:6837/3185224684] conn(0x5590b0bbdc50 0x5590b0bbfcf0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=4a8c2cf4c7876f3d server_cookie=43244a18fa6981b2 in_seq=0 out_seq=0 2024-08-19T01:34:30.044 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:30.040+0000 7f9ab8c00640 10 client.14937 ms_handle_connect on v2:172.21.15.117:6836/3185224684 2024-08-19T01:34:30.044 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:30.041+0000 7f9ab8c00640 1 -- 172.21.15.83:0/3900691463 <== mds.2 v2:172.21.15.117:6836/3185224684 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5590b0b16eb0 con 0x5590b0bbdc50 2024-08-19T01:34:30.044 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:30.041+0000 7f9ab8c00640 10 client.14937 handle_command_reply: tid=0 2024-08-19T01:34:30.044 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:30.041+0000 7f9ad4436740 1 client.14937 shutdown 2024-08-19T01:34:30.045 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:30.041+0000 7f9ad4436740 1 -- 172.21.15.83:0/3900691463 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5590b0ba3310 msgr2=0x5590b0bab060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:34:30.045 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:30.041+0000 7f9ad4436740 1 --2- 172.21.15.83:0/3900691463 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5590b0ba3310 0x5590b0bab060 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f9abc09c410 tx=0x7f9abc0a47d0 comp rx=0 tx=0).stop 2024-08-19T01:34:30.045 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:30.041+0000 7f9ad4436740 1 -- 172.21.15.83:0/3900691463 shutdown_connections 2024-08-19T01:34:30.045 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:30.041+0000 7f9ad4436740 1 --2- 172.21.15.83:0/3900691463 >> [v2:172.21.15.117:6836/3185224684,v1:172.21.15.117:6837/3185224684] conn(0x5590b0bbdc50 0x5590b0bbfcf0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:30.045 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:30.041+0000 7f9ad4436740 1 --2- 172.21.15.83:0/3900691463 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5590b0bae0b0 0x5590b0babae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:30.046 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:30.041+0000 7f9ad4436740 1 --2- 172.21.15.83:0/3900691463 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5590b0ba4880 0x5590b0bab5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:30.046 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:30.041+0000 7f9ad4436740 1 --2- 172.21.15.83:0/3900691463 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5590b0ba3310 0x5590b0bab060 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:30.046 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:30.041+0000 7f9ad4436740 1 -- 172.21.15.83:0/3900691463 >> 172.21.15.83:0/3900691463 conn(0x5590b0b6ba00 msgr2=0x5590b0ba56a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:34:30.046 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:30.041+0000 7f9ab8c00640 0 client.14937 ms_handle_reset on v2:172.21.15.117:6836/3185224684 2024-08-19T01:34:30.046 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:30.042+0000 7f9ad4436740 1 -- 172.21.15.83:0/3900691463 shutdown_connections 2024-08-19T01:34:30.046 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:30.042+0000 7f9ad4436740 1 -- 172.21.15.83:0/3900691463 wait complete. 2024-08-19T01:34:30.046 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:30.042+0000 7f9ad4436740 20 client.14937 trim_cache size 0 max 16384 2024-08-19T01:34:30.063 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:30.060+0000 7f9ad4436740 1 Processor -- start 2024-08-19T01:34:30.064 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:30.060+0000 7f9ad4436740 1 -- start start 2024-08-19T01:34:30.064 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:30.061+0000 7f9ad4436740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5590b0ba3310 0x5590b0ba4040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:34:30.064 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:30.061+0000 7f9ad4436740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5590b0ba4880 0x5590b0b9f400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:34:30.064 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:30.061+0000 7f9ad4436740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5590b0bae0b0 0x5590b0b9f940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:34:30.064 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:30.061+0000 7f9ad4436740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x5590b0bbe030 con 0x5590b0ba3310 2024-08-19T01:34:30.064 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:30.061+0000 7f9ad4436740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x5590b0bbe700 con 0x5590b0ba4880 2024-08-19T01:34:30.064 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:30.061+0000 7f9ad4436740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x5590b0bbedd0 con 0x5590b0bae0b0 2024-08-19T01:34:30.064 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:30.061+0000 7f9acaa00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5590b0ba3310 0x5590b0ba4040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:34:30.064 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:30.061+0000 7f9acaa00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5590b0ba3310 0x5590b0ba4040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:53534/0 (socket says 172.21.15.83:53534) 2024-08-19T01:34:30.064 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:30.061+0000 7f9acaa00640 1 -- 172.21.15.83:0/370054143 learned_addr learned my addr 172.21.15.83:0/370054143 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-08-19T01:34:30.065 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:30.061+0000 7f9ac2000640 1 --2- 172.21.15.83:0/370054143 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5590b0ba4880 0x5590b0b9f400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:34:30.065 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:30.061+0000 7f9acb400640 1 --2- 172.21.15.83:0/370054143 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5590b0bae0b0 0x5590b0b9f940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:34:30.065 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:30.061+0000 7f9acaa00640 1 -- 172.21.15.83:0/370054143 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5590b0bae0b0 msgr2=0x5590b0b9f940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:34:30.065 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:30.061+0000 7f9acaa00640 1 --2- 172.21.15.83:0/370054143 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5590b0bae0b0 0x5590b0b9f940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:30.065 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:30.061+0000 7f9acaa00640 1 -- 172.21.15.83:0/370054143 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5590b0ba4880 msgr2=0x5590b0b9f400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:34:30.065 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:30.061+0000 7f9acaa00640 1 --2- 172.21.15.83:0/370054143 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5590b0ba4880 0x5590b0b9f400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:30.065 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:30.061+0000 7f9acaa00640 1 -- 172.21.15.83:0/370054143 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5590b0ba6c40 con 0x5590b0ba3310 2024-08-19T01:34:30.065 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:30.061+0000 7f9ac2000640 1 --2- 172.21.15.83:0/370054143 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5590b0ba4880 0x5590b0b9f400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-19T01:34:30.065 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:30.062+0000 7f9acaa00640 1 --2- 172.21.15.83:0/370054143 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5590b0ba3310 0x5590b0ba4040 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f9abc000c00 tx=0x7f9abc0d7820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5263d777e8200bcc server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:34:30.065 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:30.062+0000 7f9abb400640 1 -- 172.21.15.83:0/370054143 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9abc09b930 con 0x5590b0ba3310 2024-08-19T01:34:30.065 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:30.062+0000 7f9abb400640 1 -- 172.21.15.83:0/370054143 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9abc0ae070 con 0x5590b0ba3310 2024-08-19T01:34:30.065 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:30.062+0000 7f9abb400640 1 -- 172.21.15.83:0/370054143 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9abc09b6a0 con 0x5590b0ba3310 2024-08-19T01:34:30.069 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:30.062+0000 7f9ad4436740 1 -- 172.21.15.83:0/370054143 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5590b0ba3310 msgr2=0x5590b0ba4040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:34:30.069 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:30.062+0000 7f9ad4436740 1 --2- 172.21.15.83:0/370054143 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5590b0ba3310 0x5590b0ba4040 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f9abc000c00 tx=0x7f9abc0d7820 comp rx=0 tx=0).stop 2024-08-19T01:34:30.070 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:30.063+0000 7f9ad4436740 1 -- 172.21.15.83:0/370054143 shutdown_connections 2024-08-19T01:34:30.070 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:30.063+0000 7f9ad4436740 1 --2- 172.21.15.83:0/370054143 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5590b0bae0b0 0x5590b0b9f940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:30.070 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:30.063+0000 7f9ad4436740 1 --2- 172.21.15.83:0/370054143 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5590b0ba4880 0x5590b0b9f400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:30.071 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:30.063+0000 7f9ad4436740 1 --2- 172.21.15.83:0/370054143 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5590b0ba3310 0x5590b0ba4040 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:30.071 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:30.063+0000 7f9ad4436740 1 -- 172.21.15.83:0/370054143 >> 172.21.15.83:0/370054143 conn(0x5590b0b6ba00 msgr2=0x5590b0b9a580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:34:30.071 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:30.063+0000 7f9ad4436740 1 -- 172.21.15.83:0/370054143 shutdown_connections 2024-08-19T01:34:30.071 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:30.063+0000 7f9ad4436740 1 -- 172.21.15.83:0/370054143 wait complete. 2024-08-19T01:34:30.071 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:30.063+0000 7f9ad4436740 1 -- start start 2024-08-19T01:34:30.071 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:30.063+0000 7f9ad4436740 1 Processor -- start 2024-08-19T01:34:30.071 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:30.064+0000 7f9ad4436740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5590b0ba3310 0x5590b0bc8460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:34:30.071 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:30.064+0000 7f9ad4436740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5590b0ba4880 0x5590b0bc17e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:34:30.071 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:30.064+0000 7f9ad4436740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5590b0bae0b0 0x5590b0bc1d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:34:30.071 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:30.064+0000 7f9ad4436740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x5590b0bbfb70 con 0x5590b0ba3310 2024-08-19T01:34:30.071 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:30.064+0000 7f9ad4436740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x5590b0bbf4a0 con 0x5590b0ba4880 2024-08-19T01:34:30.071 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:30.064+0000 7f9ad4436740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x5590b0bc0240 con 0x5590b0bae0b0 2024-08-19T01:34:30.071 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:30.064+0000 7f9ad4436740 10 client.0 _ll_register_callbacks cb 0x7f9abc0ac070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-19T01:34:30.072 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:30.064+0000 7f9acaa00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5590b0ba3310 0x5590b0bc8460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:34:30.072 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:30.064+0000 7f9acaa00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5590b0ba3310 0x5590b0bc8460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:53550/0 (socket says 172.21.15.83:53550) 2024-08-19T01:34:30.072 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:30.064+0000 7f9acaa00640 1 -- 172.21.15.83:0/896552882 learned_addr learned my addr 172.21.15.83:0/896552882 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-08-19T01:34:30.072 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:30.064+0000 7f9acb400640 1 --2- 172.21.15.83:0/896552882 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5590b0bae0b0 0x5590b0bc1d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:34:30.072 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:30.064+0000 7f9ac2000640 1 --2- 172.21.15.83:0/896552882 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5590b0ba4880 0x5590b0bc17e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:34:30.072 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:30.064+0000 7f9acaa00640 1 -- 172.21.15.83:0/896552882 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5590b0bae0b0 msgr2=0x5590b0bc1d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:34:30.072 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:30.064+0000 7f9acaa00640 1 --2- 172.21.15.83:0/896552882 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5590b0bae0b0 0x5590b0bc1d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:30.072 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:30.064+0000 7f9acaa00640 1 -- 172.21.15.83:0/896552882 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5590b0ba4880 msgr2=0x5590b0bc17e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:34:30.072 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:30.064+0000 7f9acaa00640 1 --2- 172.21.15.83:0/896552882 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5590b0ba4880 0x5590b0bc17e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:30.072 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:30.064+0000 7f9acaa00640 1 -- 172.21.15.83:0/896552882 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5590b0bc2260 con 0x5590b0ba3310 2024-08-19T01:34:30.072 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:30.065+0000 7f9acaa00640 1 --2- 172.21.15.83:0/896552882 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5590b0ba3310 0x5590b0bc8460 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f9abc0cf6d0 tx=0x7f9abc09c130 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a4901cb1d059d5cc server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:34:30.072 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:30.065+0000 7f9ab8c00640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-08-19T01:34:30.072 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:30.065+0000 7f9ab8c00640 1 -- 172.21.15.83:0/896552882 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9abc0b9070 con 0x5590b0ba3310 2024-08-19T01:34:30.073 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:30.065+0000 7f9ab8c00640 1 -- 172.21.15.83:0/896552882 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8269+0+0 (secure 0 0 0) 0x7f9abc0d2910 con 0x5590b0ba3310 2024-08-19T01:34:30.073 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:30.065+0000 7f9ad4436740 1 -- 172.21.15.83:0/896552882 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9abc0ae070 con 0x5590b0ba3310 2024-08-19T01:34:30.073 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:30.065+0000 7f9ab8c00640 1 -- 172.21.15.83:0/896552882 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f9abc09b4c0 con 0x5590b0ba3310 2024-08-19T01:34:30.073 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:30.065+0000 7f9ad4436740 10 client.14943 fetch_fsmap learned FSMap version 24 2024-08-19T01:34:30.073 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:30.065+0000 7f9ad4436740 1 -- 172.21.15.83:0/896552882 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x5590b0bc2260 con 0x5590b0ba3310 2024-08-19T01:34:30.073 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:30.065+0000 7f9ab8c00640 1 -- 172.21.15.83:0/896552882 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9abc0ae070 con 0x5590b0ba3310 2024-08-19T01:34:30.073 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:30.065+0000 7f9ab8c00640 1 -- 172.21.15.83:0/896552882 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 24) ==== 8346+0+0 (secure 0 0 0) 0x7f9abc0ba2b0 con 0x5590b0ba3310 2024-08-19T01:34:30.073 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:30.065+0000 7f9ad4436740 10 client.14943 fetch_fsmap finished waiting for FSMap version 24 2024-08-19T01:34:30.073 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:30.066+0000 7f9ad4436740 10 client.14943 resolve_mds: resolved name 'h' to daemon mds.h 2024-08-19T01:34:30.074 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:30.066+0000 7f9ad4436740 20 client.14943 populate_metadata read hostname 'smithi083' 2024-08-19T01:34:30.074 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:30.066+0000 7f9ad4436740 1 --2- 172.21.15.83:0/896552882 >> [v2:172.21.15.117:6836/3185224684,v1:172.21.15.117:6837/3185224684] conn(0x5590b0c2d060 0x5590b0c2bd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:34:30.074 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:30.066+0000 7f9ad4436740 4 client.14943 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "status"}] 2024-08-19T01:34:30.074 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:30.066+0000 7f9ad4436740 1 -- 172.21.15.83:0/896552882 --> [v2:172.21.15.117:6836/3185224684,v1:172.21.15.117:6837/3185224684] -- command(tid 0: {"prefix": "status"}) -- 0x5590b0b113c0 con 0x5590b0c2d060 2024-08-19T01:34:30.074 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:30.066+0000 7f9ac2000640 1 --2- 172.21.15.83:0/896552882 >> [v2:172.21.15.117:6836/3185224684,v1:172.21.15.117:6837/3185224684] conn(0x5590b0c2d060 0x5590b0c2bd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:34:30.074 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:30.066+0000 7f9ac2000640 1 --2- 172.21.15.83:0/896552882 >> [v2:172.21.15.117:6836/3185224684,v1:172.21.15.117:6837/3185224684] conn(0x5590b0c2d060 0x5590b0c2bd70 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=5ec559e0620b4434 server_cookie=579585015449975f in_seq=0 out_seq=0 2024-08-19T01:34:30.074 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:30.066+0000 7f9ab8c00640 10 client.14943 ms_handle_connect on v2:172.21.15.117:6836/3185224684 2024-08-19T01:34:30.074 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:30.067+0000 7f9ab8c00640 1 -- 172.21.15.83:0/896552882 <== mds.2 v2:172.21.15.117:6836/3185224684 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x5590b0b113c0 con 0x5590b0c2d060 2024-08-19T01:34:30.074 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:30.067+0000 7f9ab8c00640 10 client.14943 handle_command_reply: tid=0 2024-08-19T01:34:30.074 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:30.067+0000 7f9ad4436740 1 client.14943 shutdown 2024-08-19T01:34:30.075 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:30.067+0000 7f9ad4436740 1 -- 172.21.15.83:0/896552882 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5590b0ba3310 msgr2=0x5590b0bc8460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:34:30.075 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:30.067+0000 7f9ad4436740 1 --2- 172.21.15.83:0/896552882 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5590b0ba3310 0x5590b0bc8460 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f9abc0cf6d0 tx=0x7f9abc09c130 comp rx=0 tx=0).stop 2024-08-19T01:34:30.075 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:30.067+0000 7f9ad4436740 1 -- 172.21.15.83:0/896552882 shutdown_connections 2024-08-19T01:34:30.075 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:30.067+0000 7f9ad4436740 1 --2- 172.21.15.83:0/896552882 >> [v2:172.21.15.117:6836/3185224684,v1:172.21.15.117:6837/3185224684] conn(0x5590b0c2d060 0x5590b0c2bd70 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:30.075 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:30.067+0000 7f9ad4436740 1 --2- 172.21.15.83:0/896552882 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5590b0bae0b0 0x5590b0bc1d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:30.075 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:30.067+0000 7f9ad4436740 1 --2- 172.21.15.83:0/896552882 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5590b0ba4880 0x5590b0bc17e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:30.075 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:30.067+0000 7f9ad4436740 1 --2- 172.21.15.83:0/896552882 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5590b0ba3310 0x5590b0bc8460 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:30.075 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:30.067+0000 7f9ab8c00640 0 client.14943 ms_handle_reset on v2:172.21.15.117:6836/3185224684 2024-08-19T01:34:30.075 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:30.067+0000 7f9ad4436740 1 -- 172.21.15.83:0/896552882 >> 172.21.15.83:0/896552882 conn(0x5590b0b6ba00 msgr2=0x5590b0ba94f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:34:30.075 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:30.067+0000 7f9ad4436740 1 -- 172.21.15.83:0/896552882 shutdown_connections 2024-08-19T01:34:30.075 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:30.067+0000 7f9ad4436740 1 -- 172.21.15.83:0/896552882 wait complete. 2024-08-19T01:34:30.075 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:30.067+0000 7f9ad4436740 20 client.14943 trim_cache size 0 max 16384 2024-08-19T01:34:30.076 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-08-19T01:34:30.076 INFO:teuthology.orchestra.run.smithi083.stdout: "cluster_fsid": "bad8780e-5dc9-11ef-bccf-c7b262605968", 2024-08-19T01:34:30.076 INFO:teuthology.orchestra.run.smithi083.stdout: "whoami": 2, 2024-08-19T01:34:30.076 INFO:teuthology.orchestra.run.smithi083.stdout: "id": 24509, 2024-08-19T01:34:30.076 INFO:teuthology.orchestra.run.smithi083.stdout: "want_state": "up:active", 2024-08-19T01:34:30.076 INFO:teuthology.orchestra.run.smithi083.stdout: "state": "up:active", 2024-08-19T01:34:30.076 INFO:teuthology.orchestra.run.smithi083.stdout: "fs_name": "cephfs", 2024-08-19T01:34:30.076 INFO:teuthology.orchestra.run.smithi083.stdout: "rank_uptime": 98.786194578000007, 2024-08-19T01:34:30.076 INFO:teuthology.orchestra.run.smithi083.stdout: "mdsmap_epoch": 24, 2024-08-19T01:34:30.076 INFO:teuthology.orchestra.run.smithi083.stdout: "osdmap_epoch": 86, 2024-08-19T01:34:30.076 INFO:teuthology.orchestra.run.smithi083.stdout: "osdmap_epoch_barrier": 84, 2024-08-19T01:34:30.077 INFO:teuthology.orchestra.run.smithi083.stdout: "uptime": 133.20473004900001 2024-08-19T01:34:30.077 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-08-19T01:34:30.077 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:30.068+0000 7f9ad1a00640 1 -- 172.21.15.83:0/2053656325 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f9ab00a2b80 msgr2=0x7f9ab00a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:34:30.077 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:30.068+0000 7f9ad1a00640 1 --2- 172.21.15.83:0/2053656325 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f9ab00a2b80 0x7f9ab00a5040 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f9ac4011890 tx=0x7f9ac4009500 comp rx=0 tx=0).stop 2024-08-19T01:34:30.077 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:30.068+0000 7f9ad1a00640 1 -- 172.21.15.83:0/2053656325 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9acc1018b0 msgr2=0x7f9acc19a370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:34:30.078 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:30.068+0000 7f9ad1a00640 1 --2- 172.21.15.83:0/2053656325 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9acc1018b0 0x7f9acc19a370 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f9abc00ea60 tx=0x7f9abc00ef20 comp rx=0 tx=0).stop 2024-08-19T01:34:30.078 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:30.068+0000 7f9ad1a00640 1 -- 172.21.15.83:0/2053656325 shutdown_connections 2024-08-19T01:34:30.078 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:30.068+0000 7f9ad1a00640 1 --2- 172.21.15.83:0/2053656325 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f9ab00a2b80 0x7f9ab00a5040 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:30.078 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:30.068+0000 7f9ad1a00640 1 --2- 172.21.15.83:0/2053656325 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9acc1078b0 0x7f9acc19ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:30.078 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:30.068+0000 7f9ad1a00640 1 --2- 172.21.15.83:0/2053656325 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9acc102250 0x7f9acc19a8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:30.078 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:30.068+0000 7f9ad1a00640 1 --2- 172.21.15.83:0/2053656325 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9acc1018b0 0x7f9acc19a370 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:30.078 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:30.068+0000 7f9ad1a00640 1 -- 172.21.15.83:0/2053656325 >> 172.21.15.83:0/2053656325 conn(0x7f9acc0fd5f0 msgr2=0x7f9acc10cf40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:34:30.078 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:30.068+0000 7f9ad1a00640 1 -- 172.21.15.83:0/2053656325 shutdown_connections 2024-08-19T01:34:30.078 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:30.068+0000 7f9ad1a00640 1 -- 172.21.15.83:0/2053656325 wait complete. 2024-08-19T01:34:30.789 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:34:30 smithi083 ceph-mon[17120]: pgmap v329: 97 pgs: 97 active+clean; 641 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-08-19T01:34:30.828 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833 shell --fsid bad8780e-5dc9-11ef-bccf-c7b262605968 -- ceph tell mds.g status 2024-08-19T01:34:30.952 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:34:30 smithi117 ceph-mon[22016]: pgmap v329: 97 pgs: 97 active+clean; 641 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-08-19T01:34:30.966 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:34:30 smithi167 ceph-mon[21875]: pgmap v329: 97 pgs: 97 active+clean; 641 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-08-19T01:34:31.203 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/bad8780e-5dc9-11ef-bccf-c7b262605968/mon.a/config 2024-08-19T01:34:31.710 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:31.708+0000 7fc5dd200640 1 -- 172.21.15.83:0/3124002950 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc5d8102250 msgr2=0x7fc5d810e780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:34:31.711 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:31.708+0000 7fc5dd200640 1 --2- 172.21.15.83:0/3124002950 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc5d8102250 0x7fc5d810e780 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fc5d0009cd0 tx=0x7fc5d002a8f0 comp rx=0 tx=0).stop 2024-08-19T01:34:31.711 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:31.708+0000 7fc5dd200640 1 -- 172.21.15.83:0/3124002950 shutdown_connections 2024-08-19T01:34:31.711 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:31.708+0000 7fc5dd200640 1 --2- 172.21.15.83:0/3124002950 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc5d8102250 0x7fc5d810e780 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:31.711 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:31.708+0000 7fc5dd200640 1 --2- 172.21.15.83:0/3124002950 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc5d81018b0 0x7fc5d8101d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:31.711 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:31.708+0000 7fc5dd200640 1 --2- 172.21.15.83:0/3124002950 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc5d81078b0 0x7fc5d8107c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:31.711 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:31.708+0000 7fc5dd200640 1 -- 172.21.15.83:0/3124002950 >> 172.21.15.83:0/3124002950 conn(0x7fc5d80fd5f0 msgr2=0x7fc5d80ffa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:34:31.711 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:31.708+0000 7fc5dd200640 1 -- 172.21.15.83:0/3124002950 shutdown_connections 2024-08-19T01:34:31.711 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:31.709+0000 7fc5dd200640 1 -- 172.21.15.83:0/3124002950 wait complete. 2024-08-19T01:34:31.712 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:31.709+0000 7fc5dd200640 1 Processor -- start 2024-08-19T01:34:31.713 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:31.710+0000 7fc5dd200640 1 -- start start 2024-08-19T01:34:31.713 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:31.710+0000 7fc5dd200640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc5d81018b0 0x7fc5d819a3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:34:31.713 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:31.710+0000 7fc5dd200640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc5d8102250 0x7fc5d819a900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:34:31.713 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:31.710+0000 7fc5dd200640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc5d81078b0 0x7fc5d819ec90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:34:31.713 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:31.710+0000 7fc5dd200640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fc5d8111d00 con 0x7fc5d8102250 2024-08-19T01:34:31.713 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:31.710+0000 7fc5dd200640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fc5d8111b80 con 0x7fc5d81018b0 2024-08-19T01:34:31.713 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:31.710+0000 7fc5dd200640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fc5d8111e80 con 0x7fc5d81078b0 2024-08-19T01:34:31.713 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:31.710+0000 7fc5cd400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc5d8102250 0x7fc5d819a900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:34:31.713 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:31.710+0000 7fc5d5e00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc5d81018b0 0x7fc5d819a3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:34:31.713 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:31.710+0000 7fc5cd400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc5d8102250 0x7fc5d819a900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:60160/0 (socket says 172.21.15.83:60160) 2024-08-19T01:34:31.713 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:31.710+0000 7fc5cd400640 1 -- 172.21.15.83:0/1182325017 learned_addr learned my addr 172.21.15.83:0/1182325017 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-08-19T01:34:31.713 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:31.710+0000 7fc5d6800640 1 --2- 172.21.15.83:0/1182325017 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc5d81078b0 0x7fc5d819ec90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:34:31.713 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:31.711+0000 7fc5cd400640 1 -- 172.21.15.83:0/1182325017 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc5d81078b0 msgr2=0x7fc5d819ec90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:34:31.713 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:31.711+0000 7fc5cd400640 1 --2- 172.21.15.83:0/1182325017 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc5d81078b0 0x7fc5d819ec90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:31.713 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:31.711+0000 7fc5cd400640 1 -- 172.21.15.83:0/1182325017 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc5d81018b0 msgr2=0x7fc5d819a3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:34:31.714 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:31.711+0000 7fc5cd400640 1 --2- 172.21.15.83:0/1182325017 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc5d81018b0 0x7fc5d819a3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:31.714 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:31.711+0000 7fc5cd400640 1 -- 172.21.15.83:0/1182325017 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc5d819f370 con 0x7fc5d8102250 2024-08-19T01:34:31.714 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:31.711+0000 7fc5d5e00640 1 --2- 172.21.15.83:0/1182325017 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc5d81018b0 0x7fc5d819a3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-19T01:34:31.714 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:31.711+0000 7fc5cd400640 1 --2- 172.21.15.83:0/1182325017 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc5d8102250 0x7fc5d819a900 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fc5c000e9f0 tx=0x7fc5c000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:34:31.714 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:31.711+0000 7fc5cf400640 1 -- 172.21.15.83:0/1182325017 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5c000cd80 con 0x7fc5d8102250 2024-08-19T01:34:31.714 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:31.711+0000 7fc5cf400640 1 -- 172.21.15.83:0/1182325017 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc5c0010910 con 0x7fc5d8102250 2024-08-19T01:34:31.714 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:31.711+0000 7fc5dd200640 1 -- 172.21.15.83:0/1182325017 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc5d819f660 con 0x7fc5d8102250 2024-08-19T01:34:31.715 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:31.711+0000 7fc5cf400640 1 -- 172.21.15.83:0/1182325017 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5c0018990 con 0x7fc5d8102250 2024-08-19T01:34:31.715 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:31.712+0000 7fc5dd200640 1 -- 172.21.15.83:0/1182325017 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc5d81a6f40 con 0x7fc5d8102250 2024-08-19T01:34:31.716 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:31.714+0000 7fc5cf400640 1 -- 172.21.15.83:0/1182325017 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fc5c0010ac0 con 0x7fc5d8102250 2024-08-19T01:34:31.716 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:31.714+0000 7fc5dfdcc740 1 Processor -- start 2024-08-19T01:34:31.717 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:31.715+0000 7fc5dfdcc740 1 -- start start 2024-08-19T01:34:31.717 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:31.715+0000 7fc5dfdcc740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555b228c7880 0x555b228c5dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:34:31.717 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:31.715+0000 7fc5dfdcc740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x555b228c6310 0x555b228c6750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:34:31.717 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:31.715+0000 7fc5dfdcc740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555b228d10b0 0x555b228d34a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:34:31.717 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:31.715+0000 7fc5dfdcc740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x555b2282c940 con 0x555b228d10b0 2024-08-19T01:34:31.717 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:31.715+0000 7fc5dfdcc740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x555b227c49a0 con 0x555b228c7880 2024-08-19T01:34:31.717 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:31.715+0000 7fc5dfdcc740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x555b22828900 con 0x555b228c6310 2024-08-19T01:34:31.720 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:31.715+0000 7fc5d5e00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x555b228c6310 0x555b228c6750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:34:31.720 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:31.715+0000 7fc5d5e00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x555b228c6310 0x555b228c6750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.83:35098/0 (socket says 172.21.15.83:35098) 2024-08-19T01:34:31.720 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:31.715+0000 7fc5d5e00640 1 -- 172.21.15.83:0/1405499500 learned_addr learned my addr 172.21.15.83:0/1405499500 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-08-19T01:34:31.720 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:31.715+0000 7fc5d6800640 1 --2- 172.21.15.83:0/1405499500 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555b228c7880 0x555b228c5dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:34:31.720 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:31.715+0000 7fc5cd400640 1 --2- 172.21.15.83:0/1405499500 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555b228d10b0 0x555b228d34a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:34:31.720 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:31.716+0000 7fc5d5e00640 1 -- 172.21.15.83:0/1405499500 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555b228c7880 msgr2=0x555b228c5dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:34:31.720 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:31.716+0000 7fc5d5e00640 1 --2- 172.21.15.83:0/1405499500 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555b228c7880 0x555b228c5dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:31.720 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:31.716+0000 7fc5d5e00640 1 -- 172.21.15.83:0/1405499500 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555b228d10b0 msgr2=0x555b228d34a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:34:31.720 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:31.716+0000 7fc5d5e00640 1 --2- 172.21.15.83:0/1405499500 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555b228d10b0 0x555b228d34a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:31.720 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:31.716+0000 7fc5d5e00640 1 -- 172.21.15.83:0/1405499500 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555b227b2ef0 con 0x555b228c6310 2024-08-19T01:34:31.721 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:31.716+0000 7fc5d5e00640 1 --2- 172.21.15.83:0/1405499500 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x555b228c6310 0x555b228c6750 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fc5c80114c0 tx=0x7fc5c8011980 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9185fc0b20b6dcfe server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:34:31.721 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:31.716+0000 7fc5c7400640 1 -- 172.21.15.83:0/1405499500 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5c8004020 con 0x555b228c6310 2024-08-19T01:34:31.721 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:31.716+0000 7fc5c7400640 1 -- 172.21.15.83:0/1405499500 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc5c80041c0 con 0x555b228c6310 2024-08-19T01:34:31.721 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:31.716+0000 7fc5c7400640 1 -- 172.21.15.83:0/1405499500 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5c800f8f0 con 0x555b228c6310 2024-08-19T01:34:31.721 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:31.716+0000 7fc5cf400640 1 --2- 172.21.15.83:0/1182325017 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7fc5bc0a2c00 0x7fc5bc0a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:34:31.721 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:31.717+0000 7fc5cf400640 1 -- 172.21.15.83:0/1182325017 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8269+0+0 (secure 0 0 0) 0x7fc5c00cf330 con 0x7fc5d8102250 2024-08-19T01:34:31.721 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:31.717+0000 7fc5d6800640 1 --2- 172.21.15.83:0/1182325017 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7fc5bc0a2c00 0x7fc5bc0a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:34:31.721 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:31.717+0000 7fc5dfdcc740 1 -- 172.21.15.83:0/1405499500 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x555b228c6310 msgr2=0x555b228c6750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:34:31.721 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:31.717+0000 7fc5dfdcc740 1 --2- 172.21.15.83:0/1405499500 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x555b228c6310 0x555b228c6750 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fc5c80114c0 tx=0x7fc5c8011980 comp rx=0 tx=0).stop 2024-08-19T01:34:31.721 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:31.717+0000 7fc5dfdcc740 1 -- 172.21.15.83:0/1405499500 shutdown_connections 2024-08-19T01:34:31.721 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:31.717+0000 7fc5dfdcc740 1 --2- 172.21.15.83:0/1405499500 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555b228d10b0 0x555b228d34a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:31.722 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:31.717+0000 7fc5dfdcc740 1 --2- 172.21.15.83:0/1405499500 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x555b228c6310 0x555b228c6750 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:31.722 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:31.717+0000 7fc5dfdcc740 1 --2- 172.21.15.83:0/1405499500 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555b228c7880 0x555b228c5dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:31.722 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:31.717+0000 7fc5dfdcc740 1 -- 172.21.15.83:0/1405499500 >> 172.21.15.83:0/1405499500 conn(0x555b2288ea00 msgr2=0x555b228c29a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:34:31.722 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:31.717+0000 7fc5dfdcc740 1 -- 172.21.15.83:0/1405499500 shutdown_connections 2024-08-19T01:34:31.722 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:31.717+0000 7fc5d6800640 1 --2- 172.21.15.83:0/1182325017 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7fc5bc0a2c00 0x7fc5bc0a50c0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fc5d0004af0 tx=0x7fc5d0005d80 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:34:31.722 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:31.718+0000 7fc5dfdcc740 1 -- 172.21.15.83:0/1405499500 wait complete. 2024-08-19T01:34:31.722 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:31.718+0000 7fc5dfdcc740 1 -- start start 2024-08-19T01:34:31.722 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:31.718+0000 7fc5dfdcc740 1 Processor -- start 2024-08-19T01:34:31.726 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:31.718+0000 7fc5dfdcc740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555b228c6310 0x555b228cdfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:34:31.726 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:31.718+0000 7fc5dfdcc740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555b228c7880 0x555b228ce510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:34:31.726 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:31.719+0000 7fc5dfdcc740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x555b228d10b0 0x555b228cea50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:34:31.726 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:31.719+0000 7fc5dfdcc740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x555b228d3db0 con 0x555b228c7880 2024-08-19T01:34:31.726 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:31.719+0000 7fc5dfdcc740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x555b227e7a80 con 0x555b228c6310 2024-08-19T01:34:31.726 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:31.719+0000 7fc5dfdcc740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x555b228d3f30 con 0x555b228d10b0 2024-08-19T01:34:31.726 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:31.719+0000 7fc5dfdcc740 10 client.0 _ll_register_callbacks cb 0x555b227b0140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-19T01:34:31.726 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:31.719+0000 7fc5cd400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555b228c7880 0x555b228ce510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:34:31.727 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:31.719+0000 7fc5d5e00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555b228c6310 0x555b228cdfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:34:31.727 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:31.719+0000 7fc5d5e00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555b228c6310 0x555b228cdfd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.83:57474/0 (socket says 172.21.15.83:57474) 2024-08-19T01:34:31.727 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:31.719+0000 7fc5d5e00640 1 -- 172.21.15.83:0/18954057 learned_addr learned my addr 172.21.15.83:0/18954057 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-08-19T01:34:31.727 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:31.719+0000 7fc5d6800640 1 --2- 172.21.15.83:0/18954057 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x555b228d10b0 0x555b228cea50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:34:31.727 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:31.719+0000 7fc5d5e00640 1 -- 172.21.15.83:0/18954057 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x555b228d10b0 msgr2=0x555b228cea50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:34:31.727 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:31.719+0000 7fc5d5e00640 1 --2- 172.21.15.83:0/18954057 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x555b228d10b0 0x555b228cea50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:31.727 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:31.719+0000 7fc5d5e00640 1 -- 172.21.15.83:0/18954057 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555b228c7880 msgr2=0x555b228ce510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:34:31.728 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:31.719+0000 7fc5d5e00640 1 --2- 172.21.15.83:0/18954057 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555b228c7880 0x555b228ce510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:31.728 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:31.719+0000 7fc5d5e00640 1 -- 172.21.15.83:0/18954057 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555b227cb700 con 0x555b228c6310 2024-08-19T01:34:31.728 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:31.719+0000 7fc5cd400640 1 --2- 172.21.15.83:0/18954057 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555b228c7880 0x555b228ce510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-19T01:34:31.728 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:31.720+0000 7fc5d5e00640 1 --2- 172.21.15.83:0/18954057 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555b228c6310 0x555b228cdfd0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fc5c80096e0 tx=0x7fc5c8009de0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=67855e77abe62779 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:34:31.728 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:31.720+0000 7fc5bbe00640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-08-19T01:34:31.728 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:31.720+0000 7fc5bbe00640 1 -- 172.21.15.83:0/18954057 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5c801d040 con 0x555b228c6310 2024-08-19T01:34:31.729 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:31.720+0000 7fc5dfdcc740 1 -- 172.21.15.83:0/18954057 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc5c80041c0 con 0x555b228c6310 2024-08-19T01:34:31.729 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:31.720+0000 7fc5bbe00640 1 -- 172.21.15.83:0/18954057 <== mon.2 v2:172.21.15.117:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8269+0+0 (secure 0 0 0) 0x7fc5c80043f0 con 0x555b228c6310 2024-08-19T01:34:31.729 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:31.720+0000 7fc5bbe00640 1 -- 172.21.15.83:0/18954057 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7fc5c8003c60 con 0x555b228c6310 2024-08-19T01:34:31.729 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:31.720+0000 7fc5dfdcc740 10 client.25019 fetch_fsmap learned FSMap version 24 2024-08-19T01:34:31.729 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:31.720+0000 7fc5dfdcc740 1 -- 172.21.15.83:0/18954057 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x555b2259e2b0 con 0x555b228c6310 2024-08-19T01:34:31.729 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:31.721+0000 7fc5bbe00640 1 -- 172.21.15.83:0/18954057 <== mon.2 v2:172.21.15.117:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc5c80041c0 con 0x555b228c6310 2024-08-19T01:34:31.729 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:31.721+0000 7fc5bbe00640 1 -- 172.21.15.83:0/18954057 <== mon.2 v2:172.21.15.117:3300/0 5 ==== fsmap(e 24) ==== 8346+0+0 (secure 0 0 0) 0x7fc5c8020070 con 0x555b228c6310 2024-08-19T01:34:31.729 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:31.721+0000 7fc5dfdcc740 10 client.25019 fetch_fsmap finished waiting for FSMap version 24 2024-08-19T01:34:31.729 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:31.721+0000 7fc5dfdcc740 10 client.25019 resolve_mds: resolved name 'g' to daemon mds.g 2024-08-19T01:34:31.729 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:31.721+0000 7fc5dfdcc740 20 client.25019 populate_metadata read hostname 'smithi083' 2024-08-19T01:34:31.729 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:31.721+0000 7fc5dfdcc740 1 --2- 172.21.15.83:0/18954057 >> [v2:172.21.15.83:6838/3239261845,v1:172.21.15.83:6839/3239261845] conn(0x555b228e1c70 0x555b228e30e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:34:31.729 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:31.721+0000 7fc5dfdcc740 4 client.25019 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-19T01:34:31.730 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:31.721+0000 7fc5dfdcc740 1 -- 172.21.15.83:0/18954057 --> [v2:172.21.15.83:6838/3239261845,v1:172.21.15.83:6839/3239261845] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555b22839eb0 con 0x555b228e1c70 2024-08-19T01:34:31.730 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:31.721+0000 7fc5cd400640 1 --2- 172.21.15.83:0/18954057 >> [v2:172.21.15.83:6838/3239261845,v1:172.21.15.83:6839/3239261845] conn(0x555b228e1c70 0x555b228e30e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:34:31.730 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:31.722+0000 7fc5cd400640 1 --2- 172.21.15.83:0/18954057 >> [v2:172.21.15.83:6838/3239261845,v1:172.21.15.83:6839/3239261845] conn(0x555b228e1c70 0x555b228e30e0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=9025b042f4199bc5 server_cookie=af33c012a041bdc0 in_seq=0 out_seq=0 2024-08-19T01:34:31.730 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:31.722+0000 7fc5bbe00640 10 client.25019 ms_handle_connect on v2:172.21.15.83:6838/3239261845 2024-08-19T01:34:31.730 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:31.722+0000 7fc5bbe00640 1 -- 172.21.15.83:0/18954057 <== mds.1 v2:172.21.15.83:6838/3239261845 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x555b22839eb0 con 0x555b228e1c70 2024-08-19T01:34:31.730 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:31.722+0000 7fc5bbe00640 10 client.25019 handle_command_reply: tid=0 2024-08-19T01:34:31.730 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:31.722+0000 7fc5dfdcc740 1 client.25019 shutdown 2024-08-19T01:34:31.730 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:31.723+0000 7fc5dfdcc740 1 -- 172.21.15.83:0/18954057 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555b228c6310 msgr2=0x555b228cdfd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:34:31.730 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:31.723+0000 7fc5dfdcc740 1 --2- 172.21.15.83:0/18954057 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555b228c6310 0x555b228cdfd0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fc5c80096e0 tx=0x7fc5c8009de0 comp rx=0 tx=0).stop 2024-08-19T01:34:31.730 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:31.723+0000 7fc5dfdcc740 1 -- 172.21.15.83:0/18954057 shutdown_connections 2024-08-19T01:34:31.730 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:31.723+0000 7fc5dfdcc740 1 --2- 172.21.15.83:0/18954057 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x555b228d10b0 0x555b228cea50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:31.730 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:31.723+0000 7fc5dfdcc740 1 --2- 172.21.15.83:0/18954057 >> [v2:172.21.15.83:6838/3239261845,v1:172.21.15.83:6839/3239261845] conn(0x555b228e1c70 0x555b228e30e0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:31.730 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:31.723+0000 7fc5dfdcc740 1 --2- 172.21.15.83:0/18954057 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555b228c7880 0x555b228ce510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:31.731 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:31.723+0000 7fc5dfdcc740 1 --2- 172.21.15.83:0/18954057 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555b228c6310 0x555b228cdfd0 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:31.731 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:31.723+0000 7fc5bbe00640 0 client.25019 ms_handle_reset on v2:172.21.15.83:6838/3239261845 2024-08-19T01:34:31.731 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:31.723+0000 7fc5dfdcc740 1 -- 172.21.15.83:0/18954057 >> 172.21.15.83:0/18954057 conn(0x555b2288ea00 msgr2=0x555b228c86a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:34:31.731 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:31.723+0000 7fc5dfdcc740 1 -- 172.21.15.83:0/18954057 shutdown_connections 2024-08-19T01:34:31.731 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:31.723+0000 7fc5dfdcc740 1 -- 172.21.15.83:0/18954057 wait complete. 2024-08-19T01:34:31.731 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:31.723+0000 7fc5dfdcc740 20 client.25019 trim_cache size 0 max 16384 2024-08-19T01:34:31.743 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:31.741+0000 7fc5dfdcc740 1 Processor -- start 2024-08-19T01:34:31.743 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:31.742+0000 7fc5dfdcc740 1 -- start start 2024-08-19T01:34:31.744 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:31.742+0000 7fc5dfdcc740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555b2288ea00 0x555b228e15a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:34:31.744 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:31.742+0000 7fc5dfdcc740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x555b228c6310 0x555b228c2b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:34:31.744 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:31.742+0000 7fc5dfdcc740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555b228c7880 0x555b228bd580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:34:31.744 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:31.742+0000 7fc5dfdcc740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x555b228e1420 con 0x555b228c7880 2024-08-19T01:34:31.744 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:31.742+0000 7fc5dfdcc740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x555b228e1af0 con 0x555b2288ea00 2024-08-19T01:34:31.745 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:31.742+0000 7fc5dfdcc740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x555b228e21c0 con 0x555b228c6310 2024-08-19T01:34:31.745 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:31.742+0000 7fc5d6800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555b228c7880 0x555b228bd580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:34:31.745 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:31.742+0000 7fc5d6800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555b228c7880 0x555b228bd580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:60188/0 (socket says 172.21.15.83:60188) 2024-08-19T01:34:31.745 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:31.742+0000 7fc5d6800640 1 -- 172.21.15.83:0/3416179851 learned_addr learned my addr 172.21.15.83:0/3416179851 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-08-19T01:34:31.745 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:31.742+0000 7fc5cd400640 1 --2- 172.21.15.83:0/3416179851 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x555b228c6310 0x555b228c2b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:34:31.745 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:31.742+0000 7fc5d5e00640 1 --2- 172.21.15.83:0/3416179851 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555b2288ea00 0x555b228e15a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:34:31.745 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:31.743+0000 7fc5d6800640 1 -- 172.21.15.83:0/3416179851 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x555b228c6310 msgr2=0x555b228c2b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:34:31.745 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:31.743+0000 7fc5d6800640 1 --2- 172.21.15.83:0/3416179851 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x555b228c6310 0x555b228c2b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:31.745 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:31.743+0000 7fc5d6800640 1 -- 172.21.15.83:0/3416179851 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555b2288ea00 msgr2=0x555b228e15a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:34:31.745 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:31.743+0000 7fc5d6800640 1 --2- 172.21.15.83:0/3416179851 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555b2288ea00 0x555b228e15a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:31.745 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:31.743+0000 7fc5d6800640 1 -- 172.21.15.83:0/3416179851 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555b228ccf80 con 0x555b228c7880 2024-08-19T01:34:31.745 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:31.743+0000 7fc5d6800640 1 --2- 172.21.15.83:0/3416179851 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555b228c7880 0x555b228bd580 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fc5d0035000 tx=0x7fc5d0041000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2eddba1e0255da0b server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:34:31.747 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:31.743+0000 7fc5c6a00640 1 -- 172.21.15.83:0/3416179851 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5d0003c40 con 0x555b228c7880 2024-08-19T01:34:31.747 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:31.743+0000 7fc5c6a00640 1 -- 172.21.15.83:0/3416179851 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc5d0036050 con 0x555b228c7880 2024-08-19T01:34:31.747 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:31.743+0000 7fc5c6a00640 1 -- 172.21.15.83:0/3416179851 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5d0004630 con 0x555b228c7880 2024-08-19T01:34:31.747 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:31.744+0000 7fc5dfdcc740 1 -- 172.21.15.83:0/3416179851 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555b228c7880 msgr2=0x555b228bd580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:34:31.747 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:31.744+0000 7fc5dfdcc740 1 --2- 172.21.15.83:0/3416179851 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555b228c7880 0x555b228bd580 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fc5d0035000 tx=0x7fc5d0041000 comp rx=0 tx=0).stop 2024-08-19T01:34:31.747 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:31.744+0000 7fc5dfdcc740 1 -- 172.21.15.83:0/3416179851 shutdown_connections 2024-08-19T01:34:31.748 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:31.744+0000 7fc5dfdcc740 1 --2- 172.21.15.83:0/3416179851 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555b228c7880 0x555b228bd580 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:31.748 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:31.744+0000 7fc5dfdcc740 1 --2- 172.21.15.83:0/3416179851 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x555b228c6310 0x555b228c2b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:31.748 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:31.744+0000 7fc5dfdcc740 1 --2- 172.21.15.83:0/3416179851 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555b2288ea00 0x555b228e15a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:31.748 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:31.744+0000 7fc5dfdcc740 1 -- 172.21.15.83:0/3416179851 >> 172.21.15.83:0/3416179851 conn(0x555b228e1040 msgr2=0x555b228c2440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:34:31.748 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:31.744+0000 7fc5dfdcc740 1 -- 172.21.15.83:0/3416179851 shutdown_connections 2024-08-19T01:34:31.748 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:31.744+0000 7fc5dfdcc740 1 -- 172.21.15.83:0/3416179851 wait complete. 2024-08-19T01:34:31.748 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:31.744+0000 7fc5dfdcc740 1 -- start start 2024-08-19T01:34:31.748 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:31.745+0000 7fc5dfdcc740 1 Processor -- start 2024-08-19T01:34:31.748 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:31.745+0000 7fc5dfdcc740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555b2288ea00 0x555b228cb740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:34:31.748 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:31.745+0000 7fc5dfdcc740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x555b228c6310 0x555b228cbc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:34:31.749 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:31.745+0000 7fc5dfdcc740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555b228c7880 0x555b228ca500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:34:31.749 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:31.745+0000 7fc5dfdcc740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x555b228e2f60 con 0x555b228c7880 2024-08-19T01:34:31.749 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:31.745+0000 7fc5dfdcc740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x555b228e2890 con 0x555b2288ea00 2024-08-19T01:34:31.749 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:31.745+0000 7fc5dfdcc740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x555b228e3630 con 0x555b228c6310 2024-08-19T01:34:31.749 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:31.745+0000 7fc5dfdcc740 10 client.0 _ll_register_callbacks cb 0x7fc5c8005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-19T01:34:31.749 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:31.745+0000 7fc5d6800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555b228c7880 0x555b228ca500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:34:31.749 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:31.745+0000 7fc5d6800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555b228c7880 0x555b228ca500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:60192/0 (socket says 172.21.15.83:60192) 2024-08-19T01:34:31.749 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:31.745+0000 7fc5cd400640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x555b228c6310 0x555b228cbc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:34:31.749 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:31.745+0000 7fc5d6800640 1 -- 172.21.15.83:0/1458951130 learned_addr learned my addr 172.21.15.83:0/1458951130 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-08-19T01:34:31.749 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:31.745+0000 7fc5d5e00640 1 --2- 172.21.15.83:0/1458951130 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555b2288ea00 0x555b228cb740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:34:31.752 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:31.746+0000 7fc5d6800640 1 -- 172.21.15.83:0/1458951130 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x555b228c6310 msgr2=0x555b228cbc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:34:31.753 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:31.746+0000 7fc5d6800640 1 --2- 172.21.15.83:0/1458951130 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x555b228c6310 0x555b228cbc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:31.753 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:31.746+0000 7fc5d6800640 1 -- 172.21.15.83:0/1458951130 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555b2288ea00 msgr2=0x555b228cb740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:34:31.753 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:31.746+0000 7fc5d6800640 1 --2- 172.21.15.83:0/1458951130 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555b2288ea00 0x555b228cb740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:31.753 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:31.746+0000 7fc5d6800640 1 -- 172.21.15.83:0/1458951130 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555b228cad30 con 0x555b228c7880 2024-08-19T01:34:31.753 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:31.746+0000 7fc5cd400640 1 --2- 172.21.15.83:0/1458951130 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x555b228c6310 0x555b228cbc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-19T01:34:31.753 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:31.746+0000 7fc5d6800640 1 --2- 172.21.15.83:0/1458951130 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555b228c7880 0x555b228ca500 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fc5d002ae10 tx=0x7fc5d0009880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=38b4d16ceb700df9 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:34:31.753 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:31.746+0000 7fc5bbe00640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-08-19T01:34:31.753 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:31.746+0000 7fc5bbe00640 1 -- 172.21.15.83:0/1458951130 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5d0004740 con 0x555b228c7880 2024-08-19T01:34:31.753 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:31.746+0000 7fc5bbe00640 1 -- 172.21.15.83:0/1458951130 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8269+0+0 (secure 0 0 0) 0x7fc5d0050040 con 0x555b228c7880 2024-08-19T01:34:31.753 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:31.746+0000 7fc5dfdcc740 1 -- 172.21.15.83:0/1458951130 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc5d0036050 con 0x555b228c7880 2024-08-19T01:34:31.753 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:31.747+0000 7fc5bbe00640 1 -- 172.21.15.83:0/1458951130 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7fc5d0004b40 con 0x555b228c7880 2024-08-19T01:34:31.753 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:31.747+0000 7fc5dfdcc740 10 client.14961 fetch_fsmap learned FSMap version 24 2024-08-19T01:34:31.754 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:31.747+0000 7fc5dfdcc740 1 -- 172.21.15.83:0/1458951130 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x555b228ebd10 con 0x555b228c7880 2024-08-19T01:34:31.754 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:31.747+0000 7fc5bbe00640 1 -- 172.21.15.83:0/1458951130 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc5d0036050 con 0x555b228c7880 2024-08-19T01:34:31.754 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:31.747+0000 7fc5bbe00640 1 -- 172.21.15.83:0/1458951130 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 24) ==== 8346+0+0 (secure 0 0 0) 0x7fc5d0005040 con 0x555b228c7880 2024-08-19T01:34:31.754 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:31.747+0000 7fc5dfdcc740 10 client.14961 fetch_fsmap finished waiting for FSMap version 24 2024-08-19T01:34:31.754 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:31.747+0000 7fc5dfdcc740 10 client.14961 resolve_mds: resolved name 'g' to daemon mds.g 2024-08-19T01:34:31.754 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:31.747+0000 7fc5dfdcc740 20 client.14961 populate_metadata read hostname 'smithi083' 2024-08-19T01:34:31.754 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:31.747+0000 7fc5dfdcc740 1 --2- 172.21.15.83:0/1458951130 >> [v2:172.21.15.83:6838/3239261845,v1:172.21.15.83:6839/3239261845] conn(0x555b228d10b0 0x555b2294f230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:34:31.754 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:31.747+0000 7fc5dfdcc740 4 client.14961 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "status"}] 2024-08-19T01:34:31.754 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:31.747+0000 7fc5dfdcc740 1 -- 172.21.15.83:0/1458951130 --> [v2:172.21.15.83:6838/3239261845,v1:172.21.15.83:6839/3239261845] -- command(tid 0: {"prefix": "status"}) -- 0x555b228343c0 con 0x555b228d10b0 2024-08-19T01:34:31.755 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:31.748+0000 7fc5d5e00640 1 --2- 172.21.15.83:0/1458951130 >> [v2:172.21.15.83:6838/3239261845,v1:172.21.15.83:6839/3239261845] conn(0x555b228d10b0 0x555b2294f230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:34:31.755 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:31.748+0000 7fc5d5e00640 1 --2- 172.21.15.83:0/1458951130 >> [v2:172.21.15.83:6838/3239261845,v1:172.21.15.83:6839/3239261845] conn(0x555b228d10b0 0x555b2294f230 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=c46068cc99824a89 server_cookie=923dabf254d32d26 in_seq=0 out_seq=0 2024-08-19T01:34:31.755 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:31.748+0000 7fc5bbe00640 10 client.14961 ms_handle_connect on v2:172.21.15.83:6838/3239261845 2024-08-19T01:34:31.755 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:31.748+0000 7fc5bbe00640 1 -- 172.21.15.83:0/1458951130 <== mds.1 v2:172.21.15.83:6838/3239261845 1 ==== command_reply(tid 0: 0 ) ==== 8+0+322 (crc 0 0 0) 0x555b228343c0 con 0x555b228d10b0 2024-08-19T01:34:31.755 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:31.748+0000 7fc5bbe00640 10 client.14961 handle_command_reply: tid=0 2024-08-19T01:34:31.755 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:31.748+0000 7fc5dfdcc740 1 client.14961 shutdown 2024-08-19T01:34:31.755 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:31.748+0000 7fc5dfdcc740 1 -- 172.21.15.83:0/1458951130 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555b228c7880 msgr2=0x555b228ca500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:34:31.755 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:31.748+0000 7fc5dfdcc740 1 --2- 172.21.15.83:0/1458951130 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555b228c7880 0x555b228ca500 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fc5d002ae10 tx=0x7fc5d0009880 comp rx=0 tx=0).stop 2024-08-19T01:34:31.755 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:31.749+0000 7fc5dfdcc740 1 -- 172.21.15.83:0/1458951130 shutdown_connections 2024-08-19T01:34:31.755 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:31.749+0000 7fc5dfdcc740 1 --2- 172.21.15.83:0/1458951130 >> [v2:172.21.15.83:6838/3239261845,v1:172.21.15.83:6839/3239261845] conn(0x555b228d10b0 0x555b2294f230 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:31.755 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:31.749+0000 7fc5dfdcc740 1 --2- 172.21.15.83:0/1458951130 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555b228c7880 0x555b228ca500 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:31.755 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:31.749+0000 7fc5dfdcc740 1 --2- 172.21.15.83:0/1458951130 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x555b228c6310 0x555b228cbc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:31.755 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:31.749+0000 7fc5dfdcc740 1 --2- 172.21.15.83:0/1458951130 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555b2288ea00 0x555b228cb740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:31.755 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:31.749+0000 7fc5dfdcc740 1 -- 172.21.15.83:0/1458951130 >> 172.21.15.83:0/1458951130 conn(0x555b228e1040 msgr2=0x555b228dc1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:34:31.756 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:31.749+0000 7fc5bbe00640 0 client.14961 ms_handle_reset on v2:172.21.15.83:6838/3239261845 2024-08-19T01:34:31.756 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:31.749+0000 7fc5dfdcc740 1 -- 172.21.15.83:0/1458951130 shutdown_connections 2024-08-19T01:34:31.756 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:31.749+0000 7fc5dfdcc740 1 -- 172.21.15.83:0/1458951130 wait complete. 2024-08-19T01:34:31.756 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:31.749+0000 7fc5dfdcc740 20 client.14961 trim_cache size 0 max 16384 2024-08-19T01:34:31.756 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-08-19T01:34:31.756 INFO:teuthology.orchestra.run.smithi083.stdout: "cluster_fsid": "bad8780e-5dc9-11ef-bccf-c7b262605968", 2024-08-19T01:34:31.756 INFO:teuthology.orchestra.run.smithi083.stdout: "whoami": 1, 2024-08-19T01:34:31.756 INFO:teuthology.orchestra.run.smithi083.stdout: "id": 24527, 2024-08-19T01:34:31.757 INFO:teuthology.orchestra.run.smithi083.stdout: "want_state": "up:active", 2024-08-19T01:34:31.757 INFO:teuthology.orchestra.run.smithi083.stdout: "state": "up:active", 2024-08-19T01:34:31.757 INFO:teuthology.orchestra.run.smithi083.stdout: "fs_name": "cephfs", 2024-08-19T01:34:31.757 INFO:teuthology.orchestra.run.smithi083.stdout: "rank_uptime": 101.479815909, 2024-08-19T01:34:31.757 INFO:teuthology.orchestra.run.smithi083.stdout: "mdsmap_epoch": 24, 2024-08-19T01:34:31.757 INFO:teuthology.orchestra.run.smithi083.stdout: "osdmap_epoch": 86, 2024-08-19T01:34:31.757 INFO:teuthology.orchestra.run.smithi083.stdout: "osdmap_epoch_barrier": 84, 2024-08-19T01:34:31.757 INFO:teuthology.orchestra.run.smithi083.stdout: "uptime": 127.977927861 2024-08-19T01:34:31.757 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-08-19T01:34:31.758 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:31.750+0000 7fc5dd200640 1 -- 172.21.15.83:0/1182325017 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7fc5bc0a2c00 msgr2=0x7fc5bc0a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:34:31.758 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:31.750+0000 7fc5dd200640 1 --2- 172.21.15.83:0/1182325017 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7fc5bc0a2c00 0x7fc5bc0a50c0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fc5d0004af0 tx=0x7fc5d0005d80 comp rx=0 tx=0).stop 2024-08-19T01:34:31.758 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:31.750+0000 7fc5dd200640 1 -- 172.21.15.83:0/1182325017 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc5d8102250 msgr2=0x7fc5d819a900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:34:31.758 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:31.750+0000 7fc5dd200640 1 --2- 172.21.15.83:0/1182325017 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc5d8102250 0x7fc5d819a900 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fc5c000e9f0 tx=0x7fc5c000eeb0 comp rx=0 tx=0).stop 2024-08-19T01:34:31.758 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:31.750+0000 7fc5dd200640 1 -- 172.21.15.83:0/1182325017 shutdown_connections 2024-08-19T01:34:31.759 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:31.750+0000 7fc5dd200640 1 --2- 172.21.15.83:0/1182325017 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7fc5bc0a2c00 0x7fc5bc0a50c0 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:31.759 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:31.750+0000 7fc5dd200640 1 --2- 172.21.15.83:0/1182325017 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc5d81078b0 0x7fc5d819ec90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:31.759 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:31.750+0000 7fc5dd200640 1 --2- 172.21.15.83:0/1182325017 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc5d8102250 0x7fc5d819a900 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:31.759 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:31.750+0000 7fc5dd200640 1 --2- 172.21.15.83:0/1182325017 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc5d81018b0 0x7fc5d819a3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:31.759 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:31.750+0000 7fc5dd200640 1 -- 172.21.15.83:0/1182325017 >> 172.21.15.83:0/1182325017 conn(0x7fc5d80fd5f0 msgr2=0x7fc5d810cb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:34:31.759 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:31.750+0000 7fc5dd200640 1 -- 172.21.15.83:0/1182325017 shutdown_connections 2024-08-19T01:34:31.759 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:31.750+0000 7fc5dd200640 1 -- 172.21.15.83:0/1182325017 wait complete. 2024-08-19T01:34:32.754 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833 shell --fsid bad8780e-5dc9-11ef-bccf-c7b262605968 -- ceph tell mds.b status 2024-08-19T01:34:33.130 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/bad8780e-5dc9-11ef-bccf-c7b262605968/mon.a/config 2024-08-19T01:34:33.586 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:34:33 smithi083 ceph-mon[17120]: pgmap v330: 97 pgs: 97 active+clean; 641 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-08-19T01:34:33.695 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:33.694+0000 7f48b3e00640 1 -- 172.21.15.83:0/3665085514 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f48b4101700 msgr2=0x7f48b4101b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:34:33.696 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:33.694+0000 7f48b3e00640 1 --2- 172.21.15.83:0/3665085514 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f48b4101700 0x7f48b4101b60 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f48a0009930 tx=0x7f48a002a440 comp rx=0 tx=0).stop 2024-08-19T01:34:33.696 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:33.694+0000 7f48b3e00640 1 -- 172.21.15.83:0/3665085514 shutdown_connections 2024-08-19T01:34:33.696 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:33.694+0000 7f48b3e00640 1 --2- 172.21.15.83:0/3665085514 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f48b41020a0 0x7f48b410e5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:33.696 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:33.694+0000 7f48b3e00640 1 --2- 172.21.15.83:0/3665085514 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f48b4101700 0x7f48b4101b60 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:33.696 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:33.694+0000 7f48b3e00640 1 --2- 172.21.15.83:0/3665085514 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f48b4107700 0x7f48b4107ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:33.696 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:33.694+0000 7f48b3e00640 1 -- 172.21.15.83:0/3665085514 >> 172.21.15.83:0/3665085514 conn(0x7f48b40fd440 msgr2=0x7f48b40ff860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:34:33.696 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:33.694+0000 7f48b3e00640 1 -- 172.21.15.83:0/3665085514 shutdown_connections 2024-08-19T01:34:33.696 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:33.694+0000 7f48b3e00640 1 -- 172.21.15.83:0/3665085514 wait complete. 2024-08-19T01:34:33.697 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:33.695+0000 7f48b3e00640 1 Processor -- start 2024-08-19T01:34:33.697 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:33.695+0000 7f48b3e00640 1 -- start start 2024-08-19T01:34:33.697 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:33.695+0000 7f48b3e00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f48b4101700 0x7f48b419a210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:34:33.698 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:33.695+0000 7f48b3e00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f48b41020a0 0x7f48b419a750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:34:33.698 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:33.695+0000 7f48b3e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f48b4107700 0x7f48b419eae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:34:33.698 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:33.695+0000 7f48b3e00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f48b4111b50 con 0x7f48b4107700 2024-08-19T01:34:33.698 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:33.695+0000 7f48b3e00640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f48b41119d0 con 0x7f48b4101700 2024-08-19T01:34:33.698 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:33.695+0000 7f48b3e00640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f48b4111cd0 con 0x7f48b41020a0 2024-08-19T01:34:33.698 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:33.695+0000 7f48b2a00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f48b4101700 0x7f48b419a210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:34:33.698 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:33.696+0000 7f48b2a00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f48b4101700 0x7f48b419a210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.83:57508/0 (socket says 172.21.15.83:57508) 2024-08-19T01:34:33.698 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:33.696+0000 7f48b3400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f48b4107700 0x7f48b419eae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:34:33.698 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:33.696+0000 7f48b2a00640 1 -- 172.21.15.83:0/3411565137 learned_addr learned my addr 172.21.15.83:0/3411565137 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-08-19T01:34:33.698 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:33.696+0000 7f48b2000640 1 --2- 172.21.15.83:0/3411565137 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f48b41020a0 0x7f48b419a750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:34:33.698 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:33.696+0000 7f48b2a00640 1 -- 172.21.15.83:0/3411565137 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f48b41020a0 msgr2=0x7f48b419a750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:34:33.699 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:33.696+0000 7f48b2a00640 1 --2- 172.21.15.83:0/3411565137 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f48b41020a0 0x7f48b419a750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:33.699 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:33.696+0000 7f48b2a00640 1 -- 172.21.15.83:0/3411565137 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f48b4107700 msgr2=0x7f48b419eae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:34:33.699 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:33.696+0000 7f48b2a00640 1 --2- 172.21.15.83:0/3411565137 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f48b4107700 0x7f48b419eae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:33.699 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:33.696+0000 7f48b2a00640 1 -- 172.21.15.83:0/3411565137 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f48b419f1c0 con 0x7f48b4101700 2024-08-19T01:34:33.699 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:33.696+0000 7f48b3400640 1 --2- 172.21.15.83:0/3411565137 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f48b4107700 0x7f48b419eae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-19T01:34:33.699 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:33.696+0000 7f48b2a00640 1 --2- 172.21.15.83:0/3411565137 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f48b4101700 0x7f48b419a210 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f48a400ea60 tx=0x7f48a400ef20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:34:33.699 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:33.697+0000 7f48ab400640 1 -- 172.21.15.83:0/3411565137 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48a400cde0 con 0x7f48b4101700 2024-08-19T01:34:33.699 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:33.697+0000 7f48b3e00640 1 -- 172.21.15.83:0/3411565137 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f48b419f4b0 con 0x7f48b4101700 2024-08-19T01:34:33.700 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:33.697+0000 7f48ab400640 1 -- 172.21.15.83:0/3411565137 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f48a40035f0 con 0x7f48b4101700 2024-08-19T01:34:33.700 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:33.697+0000 7f48ab400640 1 -- 172.21.15.83:0/3411565137 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48a4010670 con 0x7f48b4101700 2024-08-19T01:34:33.700 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:33.697+0000 7f48b3e00640 1 -- 172.21.15.83:0/3411565137 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f48b41a6d90 con 0x7f48b4101700 2024-08-19T01:34:33.702 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:33.700+0000 7f48bc024740 1 Processor -- start 2024-08-19T01:34:33.702 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:33.701+0000 7f48bc024740 1 -- start start 2024-08-19T01:34:33.703 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:34:33 smithi117 ceph-mon[22016]: pgmap v330: 97 pgs: 97 active+clean; 641 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-08-19T01:34:33.703 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:33.701+0000 7f48bc024740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5574c2011a50 0x5574c200ffa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:34:33.703 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:33.701+0000 7f48bc024740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5574c20104e0 0x5574c2010920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:34:33.704 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:33.701+0000 7f48bc024740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5574c201b280 0x5574c201d670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:34:33.704 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:33.701+0000 7f48bc024740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x5574c1f76940 con 0x5574c20104e0 2024-08-19T01:34:33.704 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:33.701+0000 7f48bc024740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x5574c1f0e9a0 con 0x5574c201b280 2024-08-19T01:34:33.704 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:33.701+0000 7f48bc024740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x5574c1f72900 con 0x5574c2011a50 2024-08-19T01:34:33.704 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:33.701+0000 7f48ab400640 1 -- 172.21.15.83:0/3411565137 <== mon.2 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f48a40037a0 con 0x7f48b4101700 2024-08-19T01:34:33.704 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:33.701+0000 7f48b3400640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5574c2011a50 0x5574c200ffa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:34:33.704 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:33.701+0000 7f48b2a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5574c20104e0 0x5574c2010920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:34:33.704 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:33.701+0000 7f48b3400640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5574c2011a50 0x5574c200ffa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.83:35146/0 (socket says 172.21.15.83:35146) 2024-08-19T01:34:33.704 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:33.701+0000 7f48b3400640 1 -- 172.21.15.83:0/3106129211 learned_addr learned my addr 172.21.15.83:0/3106129211 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-08-19T01:34:33.704 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:33.702+0000 7f48b2000640 1 --2- 172.21.15.83:0/3106129211 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5574c201b280 0x5574c201d670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:34:33.704 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:33.702+0000 7f48b3400640 1 -- 172.21.15.83:0/3106129211 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5574c201b280 msgr2=0x5574c201d670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:34:33.704 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:33.702+0000 7f48b3400640 1 --2- 172.21.15.83:0/3106129211 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5574c201b280 0x5574c201d670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:33.704 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:33.702+0000 7f48b3400640 1 -- 172.21.15.83:0/3106129211 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5574c20104e0 msgr2=0x5574c2010920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:34:33.705 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:33.702+0000 7f48b3400640 1 --2- 172.21.15.83:0/3106129211 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5574c20104e0 0x5574c2010920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:33.705 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:33.702+0000 7f48b3400640 1 -- 172.21.15.83:0/3106129211 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5574c1efcef0 con 0x5574c2011a50 2024-08-19T01:34:33.706 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:33.702+0000 7f48b3400640 1 --2- 172.21.15.83:0/3106129211 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5574c2011a50 0x5574c200ffa0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f48ac010750 tx=0x7f48ac010c10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b5557946a7c6e912 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:34:33.706 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:33.703+0000 7f489f400640 1 -- 172.21.15.83:0/3106129211 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48ac00ead0 con 0x5574c2011a50 2024-08-19T01:34:33.706 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:33.703+0000 7f489f400640 1 -- 172.21.15.83:0/3106129211 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f48ac010e60 con 0x5574c2011a50 2024-08-19T01:34:33.706 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:33.703+0000 7f489f400640 1 -- 172.21.15.83:0/3106129211 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48ac011640 con 0x5574c2011a50 2024-08-19T01:34:33.706 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:33.703+0000 7f48ab400640 1 --2- 172.21.15.83:0/3411565137 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f48980a2c00 0x7f48980a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:34:33.706 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:33.703+0000 7f48b2a00640 1 --2- 172.21.15.83:0/3411565137 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f48980a2c00 0x7f48980a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:34:33.706 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:33.703+0000 7f48ab400640 1 -- 172.21.15.83:0/3411565137 <== mon.2 v2:172.21.15.117:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8269+0+0 (secure 0 0 0) 0x7f48a40cf1b0 con 0x7f48b4101700 2024-08-19T01:34:33.707 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:33.703+0000 7f48bc024740 1 -- 172.21.15.83:0/3106129211 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5574c2011a50 msgr2=0x5574c200ffa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:34:33.707 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:33.703+0000 7f48bc024740 1 --2- 172.21.15.83:0/3106129211 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5574c2011a50 0x5574c200ffa0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f48ac010750 tx=0x7f48ac010c10 comp rx=0 tx=0).stop 2024-08-19T01:34:33.707 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:33.704+0000 7f48bc024740 1 -- 172.21.15.83:0/3106129211 shutdown_connections 2024-08-19T01:34:33.707 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:33.704+0000 7f48bc024740 1 --2- 172.21.15.83:0/3106129211 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5574c201b280 0x5574c201d670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:33.707 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:33.704+0000 7f48bc024740 1 --2- 172.21.15.83:0/3106129211 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5574c20104e0 0x5574c2010920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:33.707 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:33.704+0000 7f48bc024740 1 --2- 172.21.15.83:0/3106129211 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5574c2011a50 0x5574c200ffa0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:33.707 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:33.704+0000 7f48bc024740 1 -- 172.21.15.83:0/3106129211 >> 172.21.15.83:0/3106129211 conn(0x5574c1fd8a00 msgr2=0x5574c200cb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:34:33.707 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:33.704+0000 7f48b2a00640 1 --2- 172.21.15.83:0/3411565137 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f48980a2c00 0x7f48980a50c0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f48a40cd000 tx=0x7f48a40d2c90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:34:33.707 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:33.704+0000 7f48bc024740 1 -- 172.21.15.83:0/3106129211 shutdown_connections 2024-08-19T01:34:33.707 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:33.704+0000 7f48bc024740 1 -- 172.21.15.83:0/3106129211 wait complete. 2024-08-19T01:34:33.707 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:33.704+0000 7f48bc024740 1 -- start start 2024-08-19T01:34:33.707 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:33.705+0000 7f48bc024740 1 Processor -- start 2024-08-19T01:34:33.709 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:33.705+0000 7f48bc024740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5574c20104e0 0x5574c2018190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:34:33.710 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:33.705+0000 7f48bc024740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5574c2011a50 0x5574c20186d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:34:33.710 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:33.705+0000 7f48bc024740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5574c201b280 0x5574c2018c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:34:33.710 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:33.705+0000 7f48bc024740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x5574c201df80 con 0x5574c201b280 2024-08-19T01:34:33.710 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:33.705+0000 7f48bc024740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x5574c1f31a80 con 0x5574c2011a50 2024-08-19T01:34:33.710 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:33.705+0000 7f48bc024740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x5574c201e100 con 0x5574c20104e0 2024-08-19T01:34:33.710 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:33.705+0000 7f48bc024740 10 client.0 _ll_register_callbacks cb 0x5574c1efa140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-19T01:34:33.710 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:33.705+0000 7f48b2000640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5574c2011a50 0x5574c20186d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:34:33.710 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:33.705+0000 7f48b2000640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5574c2011a50 0x5574c20186d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.83:57528/0 (socket says 172.21.15.83:57528) 2024-08-19T01:34:33.710 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:33.705+0000 7f48b2000640 1 -- 172.21.15.83:0/2558531675 learned_addr learned my addr 172.21.15.83:0/2558531675 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-08-19T01:34:33.710 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:33.705+0000 7f48b3400640 1 --2- 172.21.15.83:0/2558531675 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5574c20104e0 0x5574c2018190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:34:33.710 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:33.705+0000 7f48b3400640 1 --2- 172.21.15.83:0/2558531675 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5574c201b280 0x5574c2018c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:34:33.710 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:33.706+0000 7f48b2000640 1 -- 172.21.15.83:0/2558531675 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5574c20104e0 msgr2=0x5574c2018190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:34:33.710 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:33.706+0000 7f48b2000640 1 --2- 172.21.15.83:0/2558531675 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5574c20104e0 0x5574c2018190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:33.711 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:33.706+0000 7f48b2000640 1 -- 172.21.15.83:0/2558531675 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5574c201b280 msgr2=0x5574c2018c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:34:33.711 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:33.706+0000 7f48b2000640 1 --2- 172.21.15.83:0/2558531675 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5574c201b280 0x5574c2018c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:33.711 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:33.706+0000 7f48b2000640 1 -- 172.21.15.83:0/2558531675 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5574c1f15700 con 0x5574c2011a50 2024-08-19T01:34:33.711 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:33.706+0000 7f48b3400640 1 --2- 172.21.15.83:0/2558531675 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5574c201b280 0x5574c2018c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-19T01:34:33.711 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:33.706+0000 7f48b2000640 1 --2- 172.21.15.83:0/2558531675 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5574c2011a50 0x5574c20186d0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f48a0033000 tx=0x7f48a0004440 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fd94cdfb726f1372 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:34:33.711 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:33.706+0000 7f4893e00640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-08-19T01:34:33.711 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:33.706+0000 7f4893e00640 1 -- 172.21.15.83:0/2558531675 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48a0035040 con 0x5574c2011a50 2024-08-19T01:34:33.711 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:33.706+0000 7f48bc024740 1 -- 172.21.15.83:0/2558531675 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f48ac010e60 con 0x5574c2011a50 2024-08-19T01:34:33.711 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:33.706+0000 7f4893e00640 1 -- 172.21.15.83:0/2558531675 <== mon.2 v2:172.21.15.117:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8269+0+0 (secure 0 0 0) 0x7f48a0005450 con 0x5574c2011a50 2024-08-19T01:34:33.711 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:33.707+0000 7f4893e00640 1 -- 172.21.15.83:0/2558531675 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f48a004c070 con 0x5574c2011a50 2024-08-19T01:34:33.711 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:33.707+0000 7f48bc024740 10 client.25037 fetch_fsmap learned FSMap version 24 2024-08-19T01:34:33.711 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:33.707+0000 7f48bc024740 1 -- 172.21.15.83:0/2558531675 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x5574c1ce82b0 con 0x5574c2011a50 2024-08-19T01:34:33.711 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:33.707+0000 7f4893e00640 1 -- 172.21.15.83:0/2558531675 <== mon.2 v2:172.21.15.117:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f48ac010e60 con 0x5574c2011a50 2024-08-19T01:34:33.711 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:33.707+0000 7f4893e00640 1 -- 172.21.15.83:0/2558531675 <== mon.2 v2:172.21.15.117:3300/0 5 ==== fsmap(e 24) ==== 8346+0+0 (secure 0 0 0) 0x7f48a00318e0 con 0x5574c2011a50 2024-08-19T01:34:33.712 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:33.707+0000 7f48bc024740 10 client.25037 fetch_fsmap finished waiting for FSMap version 24 2024-08-19T01:34:33.712 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:33.707+0000 7f48bc024740 10 client.25037 resolve_mds: resolved name 'b' to daemon mds.b 2024-08-19T01:34:33.712 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:33.707+0000 7f48bc024740 20 client.25037 populate_metadata read hostname 'smithi083' 2024-08-19T01:34:33.712 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:33.707+0000 7f48bc024740 1 --2- 172.21.15.83:0/2558531675 >> [v2:172.21.15.117:6838/1967595704,v1:172.21.15.117:6839/1967595704] conn(0x5574c202c020 0x5574c202d490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:34:33.712 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:33.707+0000 7f48bc024740 4 client.25037 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-19T01:34:33.712 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:33.707+0000 7f48bc024740 1 -- 172.21.15.83:0/2558531675 --> [v2:172.21.15.117:6838/1967595704,v1:172.21.15.117:6839/1967595704] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5574c1f83eb0 con 0x5574c202c020 2024-08-19T01:34:33.712 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:33.708+0000 7f48b3400640 1 --2- 172.21.15.83:0/2558531675 >> [v2:172.21.15.117:6838/1967595704,v1:172.21.15.117:6839/1967595704] conn(0x5574c202c020 0x5574c202d490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:34:33.712 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:33.708+0000 7f48b3400640 1 --2- 172.21.15.83:0/2558531675 >> [v2:172.21.15.117:6838/1967595704,v1:172.21.15.117:6839/1967595704] conn(0x5574c202c020 0x5574c202d490 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4f620b3c9e188d03 server_cookie=babf0234395a4d6c in_seq=0 out_seq=0 2024-08-19T01:34:33.712 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:33.708+0000 7f4893e00640 10 client.25037 ms_handle_connect on v2:172.21.15.117:6838/1967595704 2024-08-19T01:34:33.713 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:33.709+0000 7f4893e00640 1 -- 172.21.15.83:0/2558531675 <== mds.0 v2:172.21.15.117:6838/1967595704 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5574c1f83eb0 con 0x5574c202c020 2024-08-19T01:34:33.713 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:33.709+0000 7f4893e00640 10 client.25037 handle_command_reply: tid=0 2024-08-19T01:34:33.713 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:33.709+0000 7f48bc024740 1 client.25037 shutdown 2024-08-19T01:34:33.713 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:33.709+0000 7f48bc024740 1 -- 172.21.15.83:0/2558531675 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5574c2011a50 msgr2=0x5574c20186d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:34:33.713 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:33.709+0000 7f48bc024740 1 --2- 172.21.15.83:0/2558531675 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5574c2011a50 0x5574c20186d0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f48a0033000 tx=0x7f48a0004440 comp rx=0 tx=0).stop 2024-08-19T01:34:33.713 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:33.709+0000 7f48bc024740 1 -- 172.21.15.83:0/2558531675 shutdown_connections 2024-08-19T01:34:33.713 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:33.709+0000 7f48bc024740 1 --2- 172.21.15.83:0/2558531675 >> [v2:172.21.15.117:6838/1967595704,v1:172.21.15.117:6839/1967595704] conn(0x5574c202c020 0x5574c202d490 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:33.714 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:33.709+0000 7f48bc024740 1 --2- 172.21.15.83:0/2558531675 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5574c201b280 0x5574c2018c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:33.714 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:33.709+0000 7f48bc024740 1 --2- 172.21.15.83:0/2558531675 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5574c2011a50 0x5574c20186d0 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:33.714 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:33.709+0000 7f48bc024740 1 --2- 172.21.15.83:0/2558531675 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5574c20104e0 0x5574c2018190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:33.714 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:33.709+0000 7f4893e00640 0 client.25037 ms_handle_reset on v2:172.21.15.117:6838/1967595704 2024-08-19T01:34:33.714 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:33.709+0000 7f48bc024740 1 -- 172.21.15.83:0/2558531675 >> 172.21.15.83:0/2558531675 conn(0x5574c1fd8a00 msgr2=0x5574c2012870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:34:33.714 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:33.710+0000 7f48bc024740 1 -- 172.21.15.83:0/2558531675 shutdown_connections 2024-08-19T01:34:33.714 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:33.710+0000 7f48bc024740 1 -- 172.21.15.83:0/2558531675 wait complete. 2024-08-19T01:34:33.714 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:33.710+0000 7f48bc024740 20 client.25037 trim_cache size 0 max 16384 2024-08-19T01:34:33.715 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:34:33 smithi167 ceph-mon[21875]: pgmap v330: 97 pgs: 97 active+clean; 641 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-08-19T01:34:33.730 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:33.728+0000 7f48bc024740 1 Processor -- start 2024-08-19T01:34:33.730 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:33.728+0000 7f48bc024740 1 -- start start 2024-08-19T01:34:33.730 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:33.728+0000 7f48bc024740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5574c20104e0 0x5574c202b950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:34:33.730 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:33.729+0000 7f48bc024740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5574c2011a50 0x5574c200fcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:34:33.730 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:33.729+0000 7f48bc024740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5574c201b280 0x5574c2016aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:34:33.730 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:33.729+0000 7f48bc024740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x5574c202b7d0 con 0x5574c20104e0 2024-08-19T01:34:33.730 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:33.729+0000 7f48bc024740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x5574c202bea0 con 0x5574c2011a50 2024-08-19T01:34:33.731 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:33.729+0000 7f48bc024740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x5574c202c570 con 0x5574c201b280 2024-08-19T01:34:33.732 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:33.729+0000 7f48b3400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5574c20104e0 0x5574c202b950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:34:33.732 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:33.729+0000 7f48b3400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5574c20104e0 0x5574c202b950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:60262/0 (socket says 172.21.15.83:60262) 2024-08-19T01:34:33.732 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:33.729+0000 7f48b3400640 1 -- 172.21.15.83:0/2889086611 learned_addr learned my addr 172.21.15.83:0/2889086611 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-08-19T01:34:33.732 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:33.729+0000 7f48b2000640 1 --2- 172.21.15.83:0/2889086611 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5574c2011a50 0x5574c200fcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:34:33.732 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:33.729+0000 7f48b3400640 1 --2- 172.21.15.83:0/2889086611 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5574c201b280 0x5574c2016aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:34:33.732 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:33.729+0000 7f48b3400640 1 -- 172.21.15.83:0/2889086611 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5574c201b280 msgr2=0x5574c2016aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:34:33.732 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:33.729+0000 7f48b3400640 1 --2- 172.21.15.83:0/2889086611 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5574c201b280 0x5574c2016aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:33.732 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:33.729+0000 7f48b3400640 1 -- 172.21.15.83:0/2889086611 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5574c2011a50 msgr2=0x5574c200fcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:34:33.733 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:33.729+0000 7f48b3400640 1 --2- 172.21.15.83:0/2889086611 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5574c2011a50 0x5574c200fcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:33.733 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:33.729+0000 7f48b3400640 1 -- 172.21.15.83:0/2889086611 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5574c2013030 con 0x5574c20104e0 2024-08-19T01:34:33.733 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:33.729+0000 7f48b3400640 1 --2- 172.21.15.83:0/2889086611 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5574c20104e0 0x5574c202b950 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f48ac00e6c0 tx=0x7f48ac012cf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6d04b48debc2931e server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:34:33.733 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:33.730+0000 7f489ea00640 1 -- 172.21.15.83:0/2889086611 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48ac02f070 con 0x5574c20104e0 2024-08-19T01:34:33.733 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:33.730+0000 7f489ea00640 1 -- 172.21.15.83:0/2889086611 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f48ac012900 con 0x5574c20104e0 2024-08-19T01:34:33.733 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:33.730+0000 7f489ea00640 1 -- 172.21.15.83:0/2889086611 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48ac025040 con 0x5574c20104e0 2024-08-19T01:34:33.733 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:33.730+0000 7f48bc024740 1 -- 172.21.15.83:0/2889086611 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5574c20104e0 msgr2=0x5574c202b950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:34:33.733 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:33.730+0000 7f48bc024740 1 --2- 172.21.15.83:0/2889086611 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5574c20104e0 0x5574c202b950 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f48ac00e6c0 tx=0x7f48ac012cf0 comp rx=0 tx=0).stop 2024-08-19T01:34:33.733 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:33.730+0000 7f48bc024740 1 -- 172.21.15.83:0/2889086611 shutdown_connections 2024-08-19T01:34:33.733 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:33.730+0000 7f48bc024740 1 --2- 172.21.15.83:0/2889086611 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5574c201b280 0x5574c2016aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:33.733 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:33.730+0000 7f48bc024740 1 --2- 172.21.15.83:0/2889086611 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5574c2011a50 0x5574c200fcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:33.733 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:33.730+0000 7f48bc024740 1 --2- 172.21.15.83:0/2889086611 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5574c20104e0 0x5574c202b950 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:33.733 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:33.730+0000 7f48bc024740 1 -- 172.21.15.83:0/2889086611 >> 172.21.15.83:0/2889086611 conn(0x5574c1fd8a00 msgr2=0x5574c200c610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:34:33.733 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:33.731+0000 7f48bc024740 1 -- 172.21.15.83:0/2889086611 shutdown_connections 2024-08-19T01:34:33.733 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:33.731+0000 7f48bc024740 1 -- 172.21.15.83:0/2889086611 wait complete. 2024-08-19T01:34:33.736 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:33.731+0000 7f48bc024740 1 -- start start 2024-08-19T01:34:33.736 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:33.731+0000 7f48bc024740 1 Processor -- start 2024-08-19T01:34:33.736 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:33.731+0000 7f48bc024740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5574c20104e0 0x5574c2033b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:34:33.736 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:33.731+0000 7f48bc024740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5574c2011a50 0x5574c2034040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:34:33.736 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:33.732+0000 7f48bc024740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5574c201b280 0x5574c2034580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:34:33.736 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:33.732+0000 7f48bc024740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x5574c202d310 con 0x5574c20104e0 2024-08-19T01:34:33.736 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:33.732+0000 7f48bc024740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x5574c202cc40 con 0x5574c201b280 2024-08-19T01:34:33.737 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:33.732+0000 7f48bc024740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x5574c202d9e0 con 0x5574c2011a50 2024-08-19T01:34:33.737 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:33.732+0000 7f48bc024740 10 client.0 _ll_register_callbacks cb 0x7f48a0009560 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-19T01:34:33.737 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:33.732+0000 7f48b3400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5574c20104e0 0x5574c2033b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:34:33.737 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:33.732+0000 7f48b3400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5574c20104e0 0x5574c2033b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:60266/0 (socket says 172.21.15.83:60266) 2024-08-19T01:34:33.737 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:33.732+0000 7f48b3400640 1 -- 172.21.15.83:0/2996701387 learned_addr learned my addr 172.21.15.83:0/2996701387 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-08-19T01:34:33.737 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:33.732+0000 7f48b2000640 1 --2- 172.21.15.83:0/2996701387 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5574c2011a50 0x5574c2034040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:34:33.737 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:33.732+0000 7f48b3400640 1 --2- 172.21.15.83:0/2996701387 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5574c201b280 0x5574c2034580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:34:33.737 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:33.732+0000 7f48b3400640 1 -- 172.21.15.83:0/2996701387 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5574c2011a50 msgr2=0x5574c2034040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:34:33.737 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:33.732+0000 7f48b3400640 1 --2- 172.21.15.83:0/2996701387 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5574c2011a50 0x5574c2034040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:33.737 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:33.732+0000 7f48b3400640 1 -- 172.21.15.83:0/2996701387 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5574c201b280 msgr2=0x5574c2034580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:34:33.737 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:33.732+0000 7f48b3400640 1 --2- 172.21.15.83:0/2996701387 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5574c201b280 0x5574c2034580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:33.737 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:33.732+0000 7f48b3400640 1 -- 172.21.15.83:0/2996701387 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5574c2018150 con 0x5574c20104e0 2024-08-19T01:34:33.737 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:33.733+0000 7f48b3400640 1 --2- 172.21.15.83:0/2996701387 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5574c20104e0 0x5574c2033b00 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f48ac009770 tx=0x7f48ac00edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9b34a301cca38f53 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:34:33.738 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:33.733+0000 7f4893e00640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-08-19T01:34:33.738 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:33.733+0000 7f4893e00640 1 -- 172.21.15.83:0/2996701387 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48ac02f070 con 0x5574c20104e0 2024-08-19T01:34:33.738 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:33.733+0000 7f48bc024740 1 -- 172.21.15.83:0/2996701387 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f48ac012900 con 0x5574c20104e0 2024-08-19T01:34:33.738 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:33.733+0000 7f4893e00640 1 -- 172.21.15.83:0/2996701387 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8269+0+0 (secure 0 0 0) 0x7f48ac00e870 con 0x5574c20104e0 2024-08-19T01:34:33.738 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:33.733+0000 7f4893e00640 1 -- 172.21.15.83:0/2996701387 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f48ac010e60 con 0x5574c20104e0 2024-08-19T01:34:33.738 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:33.733+0000 7f48bc024740 10 client.14979 fetch_fsmap learned FSMap version 24 2024-08-19T01:34:33.738 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:33.733+0000 7f48bc024740 1 -- 172.21.15.83:0/2996701387 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x5574c2096910 con 0x5574c20104e0 2024-08-19T01:34:33.738 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:33.733+0000 7f4893e00640 1 -- 172.21.15.83:0/2996701387 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f48ac012900 con 0x5574c20104e0 2024-08-19T01:34:33.738 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:33.734+0000 7f4893e00640 1 -- 172.21.15.83:0/2996701387 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 24) ==== 8346+0+0 (secure 0 0 0) 0x7f48ac040030 con 0x5574c20104e0 2024-08-19T01:34:33.738 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:33.734+0000 7f48bc024740 10 client.14979 fetch_fsmap finished waiting for FSMap version 24 2024-08-19T01:34:33.738 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:33.734+0000 7f48bc024740 10 client.14979 resolve_mds: resolved name 'b' to daemon mds.b 2024-08-19T01:34:33.738 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:33.734+0000 7f48bc024740 20 client.14979 populate_metadata read hostname 'smithi083' 2024-08-19T01:34:33.738 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:33.734+0000 7f48bc024740 1 --2- 172.21.15.83:0/2996701387 >> [v2:172.21.15.117:6838/1967595704,v1:172.21.15.117:6839/1967595704] conn(0x5574c209abd0 0x5574c20998e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:34:33.739 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:33.734+0000 7f48bc024740 4 client.14979 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "status"}] 2024-08-19T01:34:33.739 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:33.734+0000 7f48bc024740 1 -- 172.21.15.83:0/2996701387 --> [v2:172.21.15.117:6838/1967595704,v1:172.21.15.117:6839/1967595704] -- command(tid 0: {"prefix": "status"}) -- 0x5574c1f7e3c0 con 0x5574c209abd0 2024-08-19T01:34:33.739 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:33.734+0000 7f48b2000640 1 --2- 172.21.15.83:0/2996701387 >> [v2:172.21.15.117:6838/1967595704,v1:172.21.15.117:6839/1967595704] conn(0x5574c209abd0 0x5574c20998e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:34:33.740 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:33.735+0000 7f48b2000640 1 --2- 172.21.15.83:0/2996701387 >> [v2:172.21.15.117:6838/1967595704,v1:172.21.15.117:6839/1967595704] conn(0x5574c209abd0 0x5574c20998e0 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bcf8d9c7f7631ad7 server_cookie=74e592cec47e1cfd in_seq=0 out_seq=0 2024-08-19T01:34:33.740 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:33.735+0000 7f4893e00640 10 client.14979 ms_handle_connect on v2:172.21.15.117:6838/1967595704 2024-08-19T01:34:33.740 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:33.735+0000 7f4893e00640 1 -- 172.21.15.83:0/2996701387 <== mds.0 v2:172.21.15.117:6838/1967595704 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x5574c1f7e3c0 con 0x5574c209abd0 2024-08-19T01:34:33.740 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:33.735+0000 7f4893e00640 10 client.14979 handle_command_reply: tid=0 2024-08-19T01:34:33.740 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:33.735+0000 7f48bc024740 1 client.14979 shutdown 2024-08-19T01:34:33.740 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:33.735+0000 7f48bc024740 1 -- 172.21.15.83:0/2996701387 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5574c20104e0 msgr2=0x5574c2033b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:34:33.740 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:33.735+0000 7f48bc024740 1 --2- 172.21.15.83:0/2996701387 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5574c20104e0 0x5574c2033b00 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f48ac009770 tx=0x7f48ac00edf0 comp rx=0 tx=0).stop 2024-08-19T01:34:33.740 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:33.735+0000 7f48bc024740 1 -- 172.21.15.83:0/2996701387 shutdown_connections 2024-08-19T01:34:33.740 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:33.735+0000 7f48bc024740 1 --2- 172.21.15.83:0/2996701387 >> [v2:172.21.15.117:6838/1967595704,v1:172.21.15.117:6839/1967595704] conn(0x5574c209abd0 0x5574c20998e0 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:33.740 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:33.735+0000 7f48bc024740 1 --2- 172.21.15.83:0/2996701387 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5574c201b280 0x5574c2034580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:33.740 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:33.735+0000 7f4893e00640 0 client.14979 ms_handle_reset on v2:172.21.15.117:6838/1967595704 2024-08-19T01:34:33.741 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:33.735+0000 7f48bc024740 1 --2- 172.21.15.83:0/2996701387 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5574c2011a50 0x5574c2034040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:33.741 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:33.735+0000 7f48bc024740 1 --2- 172.21.15.83:0/2996701387 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5574c20104e0 0x5574c2033b00 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:33.741 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:33.735+0000 7f48bc024740 1 -- 172.21.15.83:0/2996701387 >> 172.21.15.83:0/2996701387 conn(0x5574c1fd8a00 msgr2=0x5574c2016a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:34:33.741 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:33.735+0000 7f48bc024740 1 -- 172.21.15.83:0/2996701387 shutdown_connections 2024-08-19T01:34:33.741 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:33.735+0000 7f48bc024740 1 -- 172.21.15.83:0/2996701387 wait complete. 2024-08-19T01:34:33.741 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:33.736+0000 7f48bc024740 20 client.14979 trim_cache size 0 max 16384 2024-08-19T01:34:33.741 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-08-19T01:34:33.741 INFO:teuthology.orchestra.run.smithi083.stdout: "cluster_fsid": "bad8780e-5dc9-11ef-bccf-c7b262605968", 2024-08-19T01:34:33.741 INFO:teuthology.orchestra.run.smithi083.stdout: "whoami": 0, 2024-08-19T01:34:33.741 INFO:teuthology.orchestra.run.smithi083.stdout: "id": 24545, 2024-08-19T01:34:33.742 INFO:teuthology.orchestra.run.smithi083.stdout: "want_state": "up:active", 2024-08-19T01:34:33.742 INFO:teuthology.orchestra.run.smithi083.stdout: "state": "up:active", 2024-08-19T01:34:33.742 INFO:teuthology.orchestra.run.smithi083.stdout: "fs_name": "cephfs", 2024-08-19T01:34:33.742 INFO:teuthology.orchestra.run.smithi083.stdout: "rank_uptime": 111.518963301, 2024-08-19T01:34:33.742 INFO:teuthology.orchestra.run.smithi083.stdout: "mdsmap_epoch": 24, 2024-08-19T01:34:33.742 INFO:teuthology.orchestra.run.smithi083.stdout: "osdmap_epoch": 86, 2024-08-19T01:34:33.742 INFO:teuthology.orchestra.run.smithi083.stdout: "osdmap_epoch_barrier": 84, 2024-08-19T01:34:33.742 INFO:teuthology.orchestra.run.smithi083.stdout: "uptime": 123.66368240200001 2024-08-19T01:34:33.742 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-08-19T01:34:33.743 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:33.736+0000 7f48b3e00640 1 -- 172.21.15.83:0/3411565137 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f48980a2c00 msgr2=0x7f48980a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:34:33.743 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:33.736+0000 7f48b3e00640 1 --2- 172.21.15.83:0/3411565137 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f48980a2c00 0x7f48980a50c0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f48a40cd000 tx=0x7f48a40d2c90 comp rx=0 tx=0).stop 2024-08-19T01:34:33.743 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:33.736+0000 7f48b3e00640 1 -- 172.21.15.83:0/3411565137 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f48b4101700 msgr2=0x7f48b419a210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:34:33.743 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:33.736+0000 7f48b3e00640 1 --2- 172.21.15.83:0/3411565137 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f48b4101700 0x7f48b419a210 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f48a400ea60 tx=0x7f48a400ef20 comp rx=0 tx=0).stop 2024-08-19T01:34:33.743 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:33.736+0000 7f48b3e00640 1 -- 172.21.15.83:0/3411565137 shutdown_connections 2024-08-19T01:34:33.743 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:33.736+0000 7f48b3e00640 1 --2- 172.21.15.83:0/3411565137 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f48980a2c00 0x7f48980a50c0 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:33.743 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:33.736+0000 7f48b3e00640 1 --2- 172.21.15.83:0/3411565137 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f48b4107700 0x7f48b419eae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:33.743 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:33.736+0000 7f48b3e00640 1 --2- 172.21.15.83:0/3411565137 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f48b41020a0 0x7f48b419a750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:33.743 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:33.736+0000 7f48b3e00640 1 --2- 172.21.15.83:0/3411565137 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f48b4101700 0x7f48b419a210 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:33.744 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:33.736+0000 7f48b3e00640 1 -- 172.21.15.83:0/3411565137 >> 172.21.15.83:0/3411565137 conn(0x7f48b40fd440 msgr2=0x7f48b410ccb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:34:33.744 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:33.736+0000 7f48b3e00640 1 -- 172.21.15.83:0/3411565137 shutdown_connections 2024-08-19T01:34:33.744 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:33.736+0000 7f48b3e00640 1 -- 172.21.15.83:0/3411565137 wait complete. 2024-08-19T01:34:34.579 INFO:teuthology.run_tasks:Running task sequential... 2024-08-19T01:34:34.589 INFO:teuthology.task.sequential:In sequential, running task sequential... 2024-08-19T01:34:34.589 INFO:teuthology.task.sequential:In sequential, running task print... 2024-08-19T01:34:34.590 INFO:teuthology.task.print:Enabling mgr modules 2024-08-19T01:34:34.590 INFO:teuthology.run_tasks:Running task kclient... 2024-08-19T01:34:34.624 INFO:tasks.kclient:Mounting kernel clients... 2024-08-19T01:34:34.624 INFO:tasks.kclient:config is {'client.0': {}} 2024-08-19T01:34:34.624 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-08-19T01:34:34.625 DEBUG:teuthology.orchestra.run.smithi083:> ip netns list 2024-08-19T01:34:34.662 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-08-19T01:34:34.662 DEBUG:teuthology.orchestra.run.smithi083:> sudo ip link delete ceph-brx 2024-08-19T01:34:34.762 INFO:teuthology.orchestra.run.smithi083.stderr:Cannot find device "ceph-brx" 2024-08-19T01:34:34.764 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-08-19T01:34:34.765 INFO:tasks.kclient:client.0 config is {'mntopts': ['ms_mode=legacy', 'nowsync'], 'syntax': 'v2', 'mount_subvol_num': 1} 2024-08-19T01:34:34.765 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-08-19T01:34:34.765 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-08-19T01:34:34.765 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833 shell --fsid bad8780e-5dc9-11ef-bccf-c7b262605968 -- ceph fs ls 2024-08-19T01:34:35.137 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/bad8780e-5dc9-11ef-bccf-c7b262605968/mon.a/config 2024-08-19T01:34:35.578 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:34:35 smithi083 ceph-mon[17120]: pgmap v331: 97 pgs: 97 active+clean; 641 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-08-19T01:34:35.690 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:35.688+0000 7f16f9c00640 1 -- 172.21.15.83:0/1995463527 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f16f4109ac0 msgr2=0x7f16f4109ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:34:35.691 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:35.688+0000 7f16f9c00640 1 --2- 172.21.15.83:0/1995463527 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f16f4109ac0 0x7f16f4109ea0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f16e4009930 tx=0x7f16e402a420 comp rx=0 tx=0).stop 2024-08-19T01:34:35.691 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:35.688+0000 7f16f9c00640 1 -- 172.21.15.83:0/1995463527 shutdown_connections 2024-08-19T01:34:35.691 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:35.688+0000 7f16f9c00640 1 --2- 172.21.15.83:0/1995463527 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f16f4104460 0x7f16f4110990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:35.691 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:35.688+0000 7f16f9c00640 1 --2- 172.21.15.83:0/1995463527 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f16f4103ac0 0x7f16f4103f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:35.691 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:35.688+0000 7f16f9c00640 1 --2- 172.21.15.83:0/1995463527 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f16f4109ac0 0x7f16f4109ea0 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:35.691 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:35.688+0000 7f16f9c00640 1 -- 172.21.15.83:0/1995463527 >> 172.21.15.83:0/1995463527 conn(0x7f16f40ff7c0 msgr2=0x7f16f4101be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:34:35.691 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:35.688+0000 7f16f9c00640 1 -- 172.21.15.83:0/1995463527 shutdown_connections 2024-08-19T01:34:35.691 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:35.688+0000 7f16f9c00640 1 -- 172.21.15.83:0/1995463527 wait complete. 2024-08-19T01:34:35.691 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:35.689+0000 7f16f9c00640 1 Processor -- start 2024-08-19T01:34:35.693 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:35.689+0000 7f16f9c00640 1 -- start start 2024-08-19T01:34:35.693 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:35.690+0000 7f16f9c00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f16f4103ac0 0x7f16f419a3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:34:35.693 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:35.690+0000 7f16f9c00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f16f4104460 0x7f16f419a8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:34:35.693 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:35.690+0000 7f16f9c00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f16f4109ac0 0x7f16f419ec80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:34:35.693 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:35.690+0000 7f16f9c00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f16f4113f10 con 0x7f16f4104460 2024-08-19T01:34:35.693 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:35.690+0000 7f16f9c00640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f16f4113d90 con 0x7f16f4103ac0 2024-08-19T01:34:35.693 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:35.690+0000 7f16f9c00640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f16f4114090 con 0x7f16f4109ac0 2024-08-19T01:34:35.693 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:35.690+0000 7f16f2000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f16f4104460 0x7f16f419a8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:34:35.693 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:35.690+0000 7f16f2000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f16f4104460 0x7f16f419a8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:60288/0 (socket says 172.21.15.83:60288) 2024-08-19T01:34:35.693 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:35.690+0000 7f16f2a00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f16f4103ac0 0x7f16f419a3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:34:35.693 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:35.690+0000 7f16f2000640 1 -- 172.21.15.83:0/3203330629 learned_addr learned my addr 172.21.15.83:0/3203330629 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-08-19T01:34:35.693 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:35.690+0000 7f16f3400640 1 --2- 172.21.15.83:0/3203330629 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f16f4109ac0 0x7f16f419ec80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:34:35.693 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:35.691+0000 7f16f2000640 1 -- 172.21.15.83:0/3203330629 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f16f4109ac0 msgr2=0x7f16f419ec80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:34:35.693 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:35.691+0000 7f16f2000640 1 --2- 172.21.15.83:0/3203330629 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f16f4109ac0 0x7f16f419ec80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:35.694 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:35.691+0000 7f16f2000640 1 -- 172.21.15.83:0/3203330629 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f16f4103ac0 msgr2=0x7f16f419a3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:34:35.694 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:35.691+0000 7f16f2000640 1 --2- 172.21.15.83:0/3203330629 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f16f4103ac0 0x7f16f419a3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:35.694 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:35.691+0000 7f16f2000640 1 -- 172.21.15.83:0/3203330629 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f16f419f360 con 0x7f16f4104460 2024-08-19T01:34:35.694 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:35.691+0000 7f16f2a00640 1 --2- 172.21.15.83:0/3203330629 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f16f4103ac0 0x7f16f419a3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-19T01:34:35.694 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:35.691+0000 7f16f2000640 1 --2- 172.21.15.83:0/3203330629 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f16f4104460 0x7f16f419a8f0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f16e000d770 tx=0x7f16e000dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:34:35.694 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:35.691+0000 7f16eb400640 1 -- 172.21.15.83:0/3203330629 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16e0017070 con 0x7f16f4104460 2024-08-19T01:34:35.694 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:35.691+0000 7f16eb400640 1 -- 172.21.15.83:0/3203330629 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f16e0005500 con 0x7f16f4104460 2024-08-19T01:34:35.694 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:35.691+0000 7f16eb400640 1 -- 172.21.15.83:0/3203330629 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16e0004e00 con 0x7f16f4104460 2024-08-19T01:34:35.694 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:35.691+0000 7f16f9c00640 1 -- 172.21.15.83:0/3203330629 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f16f419f650 con 0x7f16f4104460 2024-08-19T01:34:35.695 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:35.691+0000 7f16f9c00640 1 -- 172.21.15.83:0/3203330629 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f16f41a6f30 con 0x7f16f4104460 2024-08-19T01:34:35.695 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:35.692+0000 7f16f9c00640 1 -- 172.21.15.83:0/3203330629 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f16f4103f20 con 0x7f16f4104460 2024-08-19T01:34:35.696 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:35.694+0000 7f16eb400640 1 -- 172.21.15.83:0/3203330629 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f16e0004fa0 con 0x7f16f4104460 2024-08-19T01:34:35.697 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:35.695+0000 7f16eb400640 1 --2- 172.21.15.83:0/3203330629 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f16d80a2c00 0x7f16d80a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:34:35.697 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:35.696+0000 7f16eb400640 1 -- 172.21.15.83:0/3203330629 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8269+0+0 (secure 0 0 0) 0x7f16e00ce9e0 con 0x7f16f4104460 2024-08-19T01:34:35.697 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:35.696+0000 7f16f2a00640 1 --2- 172.21.15.83:0/3203330629 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f16d80a2c00 0x7f16d80a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:34:35.698 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:35.696+0000 7f16f2a00640 1 --2- 172.21.15.83:0/3203330629 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f16d80a2c00 0x7f16d80a50c0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f16e4006010 tx=0x7f16e4031510 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:34:35.701 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:35.700+0000 7f16eb400640 1 -- 172.21.15.83:0/3203330629 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7f16e009b270 con 0x7f16f4104460 2024-08-19T01:34:35.702 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:34:35 smithi117 ceph-mon[22016]: pgmap v331: 97 pgs: 97 active+clean; 641 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-08-19T01:34:35.716 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:34:35 smithi167 ceph-mon[21875]: pgmap v331: 97 pgs: 97 active+clean; 641 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-08-19T01:34:35.909 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:35.907+0000 7f16f9c00640 1 -- 172.21.15.83:0/3203330629 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f16f410c090 con 0x7f16f4104460 2024-08-19T01:34:35.909 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:35.907+0000 7f16eb400640 1 -- 172.21.15.83:0/3203330629 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v24) ==== 53+0+73 (secure 0 0 0) 0x7f16e009f570 con 0x7f16f4104460 2024-08-19T01:34:35.909 INFO:teuthology.orchestra.run.smithi083.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-08-19T01:34:35.912 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:35.910+0000 7f16f9c00640 1 -- 172.21.15.83:0/3203330629 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f16d80a2c00 msgr2=0x7f16d80a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:34:35.912 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:35.910+0000 7f16f9c00640 1 --2- 172.21.15.83:0/3203330629 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f16d80a2c00 0x7f16d80a50c0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f16e4006010 tx=0x7f16e4031510 comp rx=0 tx=0).stop 2024-08-19T01:34:35.912 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:35.910+0000 7f16f9c00640 1 -- 172.21.15.83:0/3203330629 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f16f4104460 msgr2=0x7f16f419a8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:34:35.912 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:35.910+0000 7f16f9c00640 1 --2- 172.21.15.83:0/3203330629 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f16f4104460 0x7f16f419a8f0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f16e000d770 tx=0x7f16e000dc30 comp rx=0 tx=0).stop 2024-08-19T01:34:35.912 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:35.910+0000 7f16f9c00640 1 -- 172.21.15.83:0/3203330629 shutdown_connections 2024-08-19T01:34:35.912 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:35.910+0000 7f16f9c00640 1 --2- 172.21.15.83:0/3203330629 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f16d80a2c00 0x7f16d80a50c0 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:35.912 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:35.910+0000 7f16f9c00640 1 --2- 172.21.15.83:0/3203330629 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f16f4109ac0 0x7f16f419ec80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:35.912 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:35.910+0000 7f16f9c00640 1 --2- 172.21.15.83:0/3203330629 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f16f4104460 0x7f16f419a8f0 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:35.912 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:35.910+0000 7f16f9c00640 1 --2- 172.21.15.83:0/3203330629 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f16f4103ac0 0x7f16f419a3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:35.912 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:35.910+0000 7f16f9c00640 1 -- 172.21.15.83:0/3203330629 >> 172.21.15.83:0/3203330629 conn(0x7f16f40ff7c0 msgr2=0x7f16f40fffa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:34:35.912 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:35.910+0000 7f16f9c00640 1 -- 172.21.15.83:0/3203330629 shutdown_connections 2024-08-19T01:34:35.912 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:35.910+0000 7f16f9c00640 1 -- 172.21.15.83:0/3203330629 wait complete. 2024-08-19T01:34:36.654 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:34:36 smithi083 ceph-mon[17120]: from='client.? 172.21.15.83:0/3203330629' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-08-19T01:34:36.702 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:34:36 smithi117 ceph-mon[22016]: from='client.? 172.21.15.83:0/3203330629' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-08-19T01:34:36.715 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:34:36 smithi167 ceph-mon[21875]: from='client.? 172.21.15.83:0/3203330629' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-08-19T01:34:36.820 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-08-19T01:34:36.820 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-08-19T01:34:36.820 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi083.front.sepia.ceph.com 2024-08-19T01:34:36.820 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-08-19T01:34:36.820 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-08-19T01:34:36.820 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-08-19T01:34:36.820 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-08-19T01:34:36.820 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-08-19T01:34:36.820 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.0' 2024-08-19T01:34:36.821 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-08-19T01:34:36.821 DEBUG:teuthology.orchestra.run.smithi083:> ip addr 2024-08-19T01:34:36.856 INFO:teuthology.orchestra.run.smithi083.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-08-19T01:34:36.856 INFO:teuthology.orchestra.run.smithi083.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-08-19T01:34:36.856 INFO:teuthology.orchestra.run.smithi083.stdout: inet 127.0.0.1/8 scope host lo 2024-08-19T01:34:36.856 INFO:teuthology.orchestra.run.smithi083.stdout: valid_lft forever preferred_lft forever 2024-08-19T01:34:36.856 INFO:teuthology.orchestra.run.smithi083.stdout: inet6 ::1/128 scope host 2024-08-19T01:34:36.856 INFO:teuthology.orchestra.run.smithi083.stdout: valid_lft forever preferred_lft forever 2024-08-19T01:34:36.856 INFO:teuthology.orchestra.run.smithi083.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-08-19T01:34:36.856 INFO:teuthology.orchestra.run.smithi083.stdout: link/ether 0c:c4:7a:d8:da:1e brd ff:ff:ff:ff:ff:ff 2024-08-19T01:34:36.856 INFO:teuthology.orchestra.run.smithi083.stdout: altname enp7s0f0 2024-08-19T01:34:36.857 INFO:teuthology.orchestra.run.smithi083.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-08-19T01:34:36.857 INFO:teuthology.orchestra.run.smithi083.stdout: link/ether 0c:c4:7a:d8:da:1f brd ff:ff:ff:ff:ff:ff 2024-08-19T01:34:36.857 INFO:teuthology.orchestra.run.smithi083.stdout: altname enp7s0f1 2024-08-19T01:34:36.857 INFO:teuthology.orchestra.run.smithi083.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-08-19T01:34:36.857 INFO:teuthology.orchestra.run.smithi083.stdout: link/ether 0c:c4:7a:8f:ce:90 brd ff:ff:ff:ff:ff:ff 2024-08-19T01:34:36.857 INFO:teuthology.orchestra.run.smithi083.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-08-19T01:34:36.857 INFO:teuthology.orchestra.run.smithi083.stdout: link/ether 0c:c4:7a:8f:ce:91 brd ff:ff:ff:ff:ff:ff 2024-08-19T01:34:36.857 INFO:teuthology.orchestra.run.smithi083.stdout: inet 172.21.15.83/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-08-19T01:34:36.857 INFO:teuthology.orchestra.run.smithi083.stdout: valid_lft 42331sec preferred_lft 42331sec 2024-08-19T01:34:36.857 INFO:teuthology.orchestra.run.smithi083.stdout: inet6 fe80::ec4:7aff:fe8f:ce91/64 scope link noprefixroute 2024-08-19T01:34:36.857 INFO:teuthology.orchestra.run.smithi083.stdout: valid_lft forever preferred_lft forever 2024-08-19T01:34:36.858 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-08-19T01:34:36.858 DEBUG:teuthology.orchestra.run.smithi083:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-08-19T01:34:36.858 DEBUG:teuthology.orchestra.run.smithi083:> set -e 2024-08-19T01:34:36.858 DEBUG:teuthology.orchestra.run.smithi083:> sudo ip link add name ceph-brx type bridge 2024-08-19T01:34:36.858 DEBUG:teuthology.orchestra.run.smithi083:> sudo ip addr flush dev ceph-brx 2024-08-19T01:34:36.859 DEBUG:teuthology.orchestra.run.smithi083:> sudo ip link set ceph-brx up 2024-08-19T01:34:36.859 DEBUG:teuthology.orchestra.run.smithi083:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-08-19T01:34:36.859 DEBUG:teuthology.orchestra.run.smithi083:> ') 2024-08-19T01:34:36.976 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:36 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-08-19T01:34:37.266 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:37 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-08-19T01:34:37.277 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-08-19T01:34:37.277 DEBUG:teuthology.orchestra.run.smithi083:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-08-19T01:34:37.334 INFO:teuthology.orchestra.run.smithi083.stdout:1 2024-08-19T01:34:37.337 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-08-19T01:34:37.337 DEBUG:teuthology.orchestra.run.smithi083:> route 2024-08-19T01:34:37.385 INFO:teuthology.orchestra.run.smithi083.stdout:Kernel IP routing table 2024-08-19T01:34:37.385 INFO:teuthology.orchestra.run.smithi083.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-08-19T01:34:37.385 INFO:teuthology.orchestra.run.smithi083.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-08-19T01:34:37.385 INFO:teuthology.orchestra.run.smithi083.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-08-19T01:34:37.385 INFO:teuthology.orchestra.run.smithi083.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-08-19T01:34:37.386 DEBUG:teuthology.orchestra.run.smithi083:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-08-19T01:34:37.386 DEBUG:teuthology.orchestra.run.smithi083:> set -e 2024-08-19T01:34:37.386 DEBUG:teuthology.orchestra.run.smithi083:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-08-19T01:34:37.386 DEBUG:teuthology.orchestra.run.smithi083:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-08-19T01:34:37.386 DEBUG:teuthology.orchestra.run.smithi083:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-08-19T01:34:37.386 DEBUG:teuthology.orchestra.run.smithi083:> ') 2024-08-19T01:34:37.465 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:37 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-08-19T01:34:37.716 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:34:37 smithi167 ceph-mon[21875]: pgmap v332: 97 pgs: 97 active+clean; 641 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-08-19T01:34:37.718 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:34:37 smithi083 ceph-mon[17120]: pgmap v332: 97 pgs: 97 active+clean; 641 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-08-19T01:34:37.952 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:34:37 smithi117 ceph-mon[22016]: pgmap v332: 97 pgs: 97 active+clean; 641 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-08-19T01:34:37.961 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:37 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-08-19T01:34:37.969 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-08-19T01:34:37.970 DEBUG:teuthology.orchestra.run.smithi083:> ip netns list 2024-08-19T01:34:38.006 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-08-19T01:34:38.006 DEBUG:teuthology.orchestra.run.smithi083:> ip netns list-id 2024-08-19T01:34:38.081 DEBUG:teuthology.orchestra.run.smithi083:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-08-19T01:34:38.082 DEBUG:teuthology.orchestra.run.smithi083:> set -e 2024-08-19T01:34:38.082 DEBUG:teuthology.orchestra.run.smithi083:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.0 2024-08-19T01:34:38.082 DEBUG:teuthology.orchestra.run.smithi083:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.0 0 2024-08-19T01:34:38.082 DEBUG:teuthology.orchestra.run.smithi083:> ') 2024-08-19T01:34:38.198 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:38 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-08-19T01:34:38.250 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:38 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-08-19T01:34:38.258 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.0' with 192.168.0.1/16 2024-08-19T01:34:38.258 DEBUG:teuthology.orchestra.run.smithi083:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-08-19T01:34:38.258 DEBUG:teuthology.orchestra.run.smithi083:> set -e 2024-08-19T01:34:38.258 DEBUG:teuthology.orchestra.run.smithi083:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.0 type veth peer name brx.0 2024-08-19T01:34:38.258 DEBUG:teuthology.orchestra.run.smithi083:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-08-19T01:34:38.258 DEBUG:teuthology.orchestra.run.smithi083:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set veth0 up 2024-08-19T01:34:38.258 DEBUG:teuthology.orchestra.run.smithi083:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set lo up 2024-08-19T01:34:38.258 DEBUG:teuthology.orchestra.run.smithi083:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip route add default via 192.168.255.254 2024-08-19T01:34:38.259 DEBUG:teuthology.orchestra.run.smithi083:> ') 2024-08-19T01:34:38.337 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:38 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-08-19T01:34:38.494 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:38 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-08-19T01:34:38.502 DEBUG:teuthology.orchestra.run.smithi083:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-08-19T01:34:38.502 DEBUG:teuthology.orchestra.run.smithi083:> set -e 2024-08-19T01:34:38.502 DEBUG:teuthology.orchestra.run.smithi083:> sudo ip link set brx.0 up 2024-08-19T01:34:38.502 DEBUG:teuthology.orchestra.run.smithi083:> sudo ip link set dev brx.0 master ceph-brx 2024-08-19T01:34:38.502 DEBUG:teuthology.orchestra.run.smithi083:> ') 2024-08-19T01:34:38.581 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:38 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-08-19T01:34:38.660 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:38 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-08-19T01:34:38.668 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-08-19T01:34:38.669 DEBUG:teuthology.orchestra.run.smithi083:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-08-19T01:34:38.704 INFO:teuthology.orchestra.run.smithi083.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-08-19T01:34:38.704 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-08-19T01:34:38.704 DEBUG:teuthology.orchestra.run.smithi083:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-08-19T01:34:38.779 WARNING:tasks.cephfs.mount:You cannot specify both: cephfs_mntpt and mount_subvol_num 2024-08-19T01:34:38.779 INFO:tasks.cephfs.mount:Mounting subvol 1 for now 2024-08-19T01:34:38.779 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/volumes/qa/sv_1/05b773d9-8346-4397-99ec-d49959a55703 2024-08-19T01:34:38.779 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-08-19T01:34:38.779 DEBUG:teuthology.orchestra.run.smithi083:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/volumes/qa/sv_1/05b773d9-8346-4397-99ec-d49959a55703 /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,ms_mode=legacy,nowsync,nofallback 2024-08-19T01:34:38.930 INFO:teuthology.orchestra.run.smithi083.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,ms_mode=legacy,nowsync,nofallback 2024-08-19T01:34:38.930 INFO:teuthology.orchestra.run.smithi083.stdout:mount.ceph: options "norequire_active_mds,norbytes,ms_mode=legacy,nowsync". 2024-08-19T01:34:39.569 INFO:teuthology.orchestra.run.smithi083.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,ms_mode=legacy,nowsync,nofallback 2024-08-19T01:34:39.570 INFO:teuthology.orchestra.run.smithi083.stdout:mount.ceph: options "norequire_active_mds,norbytes,ms_mode=legacy,nowsync". 2024-08-19T01:34:39.570 INFO:teuthology.orchestra.run.smithi083.stdout:mount.ceph: resolved to: "172.21.15.83:6789,172.21.15.117:6789,172.21.15.167:6789" 2024-08-19T01:34:39.570 INFO:teuthology.orchestra.run.smithi083.stdout:mount.ceph: trying mount with new device syntax: 0@bad8780e-5dc9-11ef-bccf-c7b262605968.cephfs=/volumes/qa/sv_1/05b773d9-8346-4397-99ec-d49959a55703 2024-08-19T01:34:39.570 INFO:teuthology.orchestra.run.smithi083.stdout:mount.ceph: options "norequire_active_mds,norbytes,ms_mode=legacy,nowsync,name=0,key=0,mon_addr=172.21.15.83:6789/172.21.15.117:6789/172.21.15.167:6789" will pass to kernel 2024-08-19T01:34:39.573 INFO:teuthology.orchestra.run.smithi083.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-08-19T01:34:39.573 INFO:teuthology.orchestra.run.smithi083.stdout: You just mounted a file system that supports labels which does not 2024-08-19T01:34:39.573 INFO:teuthology.orchestra.run.smithi083.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-08-19T01:34:39.573 INFO:teuthology.orchestra.run.smithi083.stdout: applications will generate AVC messages and not be allowed access to 2024-08-19T01:34:39.573 INFO:teuthology.orchestra.run.smithi083.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-08-19T01:34:39.576 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-08-19T01:34:39.576 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-08-19T01:34:39.576 DEBUG:teuthology.orchestra.run.smithi083:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-08-19T01:34:39.637 DEBUG:teuthology.orchestra.run.smithi083:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-08-19T01:34:39.716 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:34:39 smithi167 ceph-mon[21875]: pgmap v333: 97 pgs: 97 active+clean; 641 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-08-19T01:34:39.717 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:39 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-08-19T01:34:39.721 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:39 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-08-19T01:34:39.729 INFO:teuthology.orchestra.run.smithi083.stdout:client25052 2024-08-19T01:34:39.729 DEBUG:teuthology.orchestra.run.smithi083:> set -ex 2024-08-19T01:34:39.729 DEBUG:teuthology.orchestra.run.smithi083:> dd if=/proc/self/mounts of=/dev/stdout 2024-08-19T01:34:39.768 DEBUG:teuthology.orchestra.run.smithi083:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-08-19T01:34:39.886 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:39 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-08-19T01:34:39.889 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:39 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-08-19T01:34:39.897 INFO:teuthology.orchestra.run.smithi083.stdout:client25052 2024-08-19T01:34:39.898 DEBUG:teuthology.orchestra.run.smithi083:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-08-19T01:34:39.904 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:34:39 smithi083 ceph-mon[17120]: pgmap v333: 97 pgs: 97 active+clean; 641 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-08-19T01:34:39.952 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:34:39 smithi117 ceph-mon[22016]: pgmap v333: 97 pgs: 97 active+clean; 641 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-08-19T01:34:39.976 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:39 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-08-19T01:34:39.980 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:39 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-08-19T01:34:39.987 INFO:teuthology.orchestra.run.smithi083.stdout:client25052 2024-08-19T01:34:39.988 DEBUG:teuthology.orchestra.run.smithi083:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=300 -- bash -c 'sudo dd if=/sys/kernel/debug/ceph/bad8780e-5dc9-11ef-bccf-c7b262605968.client25052/status') 2024-08-19T01:34:40.105 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:40 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-08-19T01:34:40.126 INFO:teuthology.orchestra.run.smithi083.stdout:instance: client.25052 (3)172.21.15.83:0/798945717 2024-08-19T01:34:40.127 INFO:teuthology.orchestra.run.smithi083.stdout:blocklisted: false 2024-08-19T01:34:40.127 INFO:teuthology.orchestra.run.smithi083.stderr:0+1 records in 2024-08-19T01:34:40.127 INFO:teuthology.orchestra.run.smithi083.stderr:0+1 records out 2024-08-19T01:34:40.127 INFO:teuthology.orchestra.run.smithi083.stderr:70 bytes copied, 5.8098e-05 s, 1.2 MB/s 2024-08-19T01:34:40.129 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:40 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-08-19T01:34:40.137 INFO:teuthology.run_tasks:Running task exec... 2024-08-19T01:34:40.148 INFO:teuthology.task.exec:Executing custom commands... 2024-08-19T01:34:40.149 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi083.front.sepia.ceph.com 2024-08-19T01:34:40.149 DEBUG:teuthology.orchestra.run.smithi083:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs set cephfs balance_automate false' 2024-08-19T01:34:40.303 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:40.301+0000 7f49a2800640 1 -- 172.21.15.83:0/3135710191 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f499c075f60 msgr2=0x7f499c0743b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:34:40.303 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:40.301+0000 7f49a2800640 1 --2- 172.21.15.83:0/3135710191 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f499c075f60 0x7f499c0743b0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f498c009900 tx=0x7f498c02a3f0 comp rx=0 tx=0).stop 2024-08-19T01:34:40.303 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:40.301+0000 7f49a2800640 1 -- 172.21.15.83:0/3135710191 shutdown_connections 2024-08-19T01:34:40.303 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:40.301+0000 7f49a2800640 1 --2- 172.21.15.83:0/3135710191 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f499c10de60 0x7f499c110250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:40.303 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:40.301+0000 7f49a2800640 1 --2- 172.21.15.83:0/3135710191 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f499c074980 0x7f499c074de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:40.303 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:40.301+0000 7f49a2800640 1 --2- 172.21.15.83:0/3135710191 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f499c075f60 0x7f499c0743b0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:40.303 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:40.301+0000 7f49a2800640 1 -- 172.21.15.83:0/3135710191 >> 172.21.15.83:0/3135710191 conn(0x7f499c0fd010 msgr2=0x7f499c0ff430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:34:40.304 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:40.301+0000 7f49a2800640 1 -- 172.21.15.83:0/3135710191 shutdown_connections 2024-08-19T01:34:40.304 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:40.301+0000 7f49a2800640 1 -- 172.21.15.83:0/3135710191 wait complete. 2024-08-19T01:34:40.306 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:40.302+0000 7f49a2800640 1 Processor -- start 2024-08-19T01:34:40.306 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:40.302+0000 7f49a2800640 1 -- start start 2024-08-19T01:34:40.306 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:40.302+0000 7f49a2800640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f499c074980 0x7f499c19e0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:34:40.306 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:40.302+0000 7f49a2800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f499c075f60 0x7f499c19e620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:34:40.306 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:40.302+0000 7f49a2800640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f499c10de60 0x7f499c1a5690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:34:40.306 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:40.302+0000 7f49a2800640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f499c113b50 con 0x7f499c075f60 2024-08-19T01:34:40.306 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:40.302+0000 7f49a2800640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f499c1139d0 con 0x7f499c10de60 2024-08-19T01:34:40.306 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:40.302+0000 7f49a2800640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f499c113cd0 con 0x7f499c074980 2024-08-19T01:34:40.306 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:40.303+0000 7f4992a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f499c075f60 0x7f499c19e620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:34:40.307 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:40.303+0000 7f4992a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f499c075f60 0x7f499c19e620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:60314/0 (socket says 172.21.15.83:60314) 2024-08-19T01:34:40.307 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:40.303+0000 7f4992a00640 1 -- 172.21.15.83:0/3489126117 learned_addr learned my addr 172.21.15.83:0/3489126117 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-08-19T01:34:40.307 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:40.303+0000 7f499be00640 1 --2- 172.21.15.83:0/3489126117 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f499c10de60 0x7f499c1a5690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:34:40.307 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:40.303+0000 7f499b400640 1 --2- 172.21.15.83:0/3489126117 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f499c074980 0x7f499c19e0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:34:40.307 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:40.303+0000 7f4992a00640 1 -- 172.21.15.83:0/3489126117 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f499c074980 msgr2=0x7f499c19e0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:34:40.307 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:40.303+0000 7f4992a00640 1 --2- 172.21.15.83:0/3489126117 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f499c074980 0x7f499c19e0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:40.307 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:40.303+0000 7f4992a00640 1 -- 172.21.15.83:0/3489126117 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f499c10de60 msgr2=0x7f499c1a5690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:34:40.307 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:40.303+0000 7f4992a00640 1 --2- 172.21.15.83:0/3489126117 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f499c10de60 0x7f499c1a5690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:40.307 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:40.303+0000 7f4992a00640 1 -- 172.21.15.83:0/3489126117 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f499c1a5d00 con 0x7f499c075f60 2024-08-19T01:34:40.307 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:40.303+0000 7f499b400640 1 --2- 172.21.15.83:0/3489126117 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f499c074980 0x7f499c19e0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-19T01:34:40.307 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:40.303+0000 7f499be00640 1 --2- 172.21.15.83:0/3489126117 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f499c10de60 0x7f499c1a5690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-19T01:34:40.307 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:40.303+0000 7f4992a00640 1 --2- 172.21.15.83:0/3489126117 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f499c075f60 0x7f499c19e620 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f498400ea60 tx=0x7f498400ef20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:34:40.307 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:40.304+0000 7f4998c00640 1 -- 172.21.15.83:0/3489126117 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f498400cde0 con 0x7f499c075f60 2024-08-19T01:34:40.308 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:40.304+0000 7f4998c00640 1 -- 172.21.15.83:0/3489126117 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f49840035f0 con 0x7f499c075f60 2024-08-19T01:34:40.308 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:40.304+0000 7f49a2800640 1 -- 172.21.15.83:0/3489126117 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f499c1a5ff0 con 0x7f499c075f60 2024-08-19T01:34:40.308 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:40.304+0000 7f4998c00640 1 -- 172.21.15.83:0/3489126117 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4984010670 con 0x7f499c075f60 2024-08-19T01:34:40.308 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:40.304+0000 7f49a2800640 1 -- 172.21.15.83:0/3489126117 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f499c1a64b0 con 0x7f499c075f60 2024-08-19T01:34:40.308 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:40.305+0000 7f49a2800640 1 -- 172.21.15.83:0/3489126117 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f499c103f10 con 0x7f499c075f60 2024-08-19T01:34:40.308 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:40.307+0000 7f4998c00640 1 -- 172.21.15.83:0/3489126117 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f49840037a0 con 0x7f499c075f60 2024-08-19T01:34:40.309 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:40.308+0000 7f4998c00640 1 --2- 172.21.15.83:0/3489126117 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f49800a2c30 0x7f49800a50f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:34:40.310 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:40.308+0000 7f4998c00640 1 -- 172.21.15.83:0/3489126117 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8269+0+0 (secure 0 0 0) 0x7f49840cf420 con 0x7f499c075f60 2024-08-19T01:34:40.310 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:40.308+0000 7f499b400640 1 --2- 172.21.15.83:0/3489126117 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f49800a2c30 0x7f49800a50f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:34:40.310 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:40.309+0000 7f499b400640 1 --2- 172.21.15.83:0/3489126117 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f49800a2c30 0x7f49800a50f0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f498c006010 tx=0x7f498c000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:34:40.313 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:40.312+0000 7f4998c00640 1 -- 172.21.15.83:0/3489126117 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7f498409c300 con 0x7f499c075f60 2024-08-19T01:34:40.528 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:40.527+0000 7f49a2800640 1 -- 172.21.15.83:0/3489126117 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"} v 0) -- 0x7f499c062b50 con 0x7f499c075f60 2024-08-19T01:34:41.465 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:41.463+0000 7f4998c00640 1 -- 172.21.15.83:0/3489126117 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]=0 v25) ==== 118+0+0 (secure 0 0 0) 0x7f498409bc00 con 0x7f499c075f60 2024-08-19T01:34:41.467 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:41.466+0000 7f49a2800640 1 -- 172.21.15.83:0/3489126117 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f49800a2c30 msgr2=0x7f49800a50f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:34:41.467 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:41.466+0000 7f49a2800640 1 --2- 172.21.15.83:0/3489126117 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f49800a2c30 0x7f49800a50f0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f498c006010 tx=0x7f498c000bc0 comp rx=0 tx=0).stop 2024-08-19T01:34:41.468 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:41.466+0000 7f49a2800640 1 -- 172.21.15.83:0/3489126117 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f499c075f60 msgr2=0x7f499c19e620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:34:41.468 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:41.466+0000 7f49a2800640 1 --2- 172.21.15.83:0/3489126117 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f499c075f60 0x7f499c19e620 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f498400ea60 tx=0x7f498400ef20 comp rx=0 tx=0).stop 2024-08-19T01:34:41.468 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:41.466+0000 7f49a2800640 1 -- 172.21.15.83:0/3489126117 shutdown_connections 2024-08-19T01:34:41.468 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:41.466+0000 7f49a2800640 1 --2- 172.21.15.83:0/3489126117 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f49800a2c30 0x7f49800a50f0 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:41.469 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:41.466+0000 7f49a2800640 1 --2- 172.21.15.83:0/3489126117 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f499c10de60 0x7f499c1a5690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:41.469 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:41.466+0000 7f49a2800640 1 --2- 172.21.15.83:0/3489126117 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f499c075f60 0x7f499c19e620 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:41.469 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:41.466+0000 7f49a2800640 1 --2- 172.21.15.83:0/3489126117 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f499c074980 0x7f499c19e0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:41.469 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:41.466+0000 7f49a2800640 1 -- 172.21.15.83:0/3489126117 >> 172.21.15.83:0/3489126117 conn(0x7f499c0fd010 msgr2=0x7f499c1010e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:34:41.469 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:41.467+0000 7f49a2800640 1 -- 172.21.15.83:0/3489126117 shutdown_connections 2024-08-19T01:34:41.469 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:41.467+0000 7f49a2800640 1 -- 172.21.15.83:0/3489126117 wait complete. 2024-08-19T01:34:41.486 DEBUG:teuthology.orchestra.run.smithi083:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs subvolumegroup pin cephfs qa random 0.10' 2024-08-19T01:34:41.644 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:41.642+0000 7f96d7200640 1 -- 172.21.15.83:0/2638753187 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f96d8074cb0 msgr2=0x7f96d81094c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:34:41.644 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:41.642+0000 7f96d7200640 1 --2- 172.21.15.83:0/2638753187 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f96d8074cb0 0x7f96d81094c0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f96c0009900 tx=0x7f96c002a410 comp rx=0 tx=0).stop 2024-08-19T01:34:41.644 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:41.642+0000 7f96d7200640 1 -- 172.21.15.83:0/2638753187 shutdown_connections 2024-08-19T01:34:41.644 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:41.642+0000 7f96d7200640 1 --2- 172.21.15.83:0/2638753187 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f96d8109a00 0x7f96d810bdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:41.644 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:41.642+0000 7f96d7200640 1 --2- 172.21.15.83:0/2638753187 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f96d8074cb0 0x7f96d81094c0 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:41.644 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:41.642+0000 7f96d7200640 1 --2- 172.21.15.83:0/2638753187 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f96d8074370 0x7f96d8074770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:41.644 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:41.642+0000 7f96d7200640 1 -- 172.21.15.83:0/2638753187 >> 172.21.15.83:0/2638753187 conn(0x7f96d80fce00 msgr2=0x7f96d80ff240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:34:41.644 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:41.642+0000 7f96d7200640 1 -- 172.21.15.83:0/2638753187 shutdown_connections 2024-08-19T01:34:41.644 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:41.642+0000 7f96d7200640 1 -- 172.21.15.83:0/2638753187 wait complete. 2024-08-19T01:34:41.646 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:41.643+0000 7f96d7200640 1 Processor -- start 2024-08-19T01:34:41.647 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:41.643+0000 7f96d7200640 1 -- start start 2024-08-19T01:34:41.647 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:41.643+0000 7f96d7200640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f96d8074370 0x7f96d8199cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:34:41.647 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:41.643+0000 7f96d7200640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f96d8074cb0 0x7f96d819a1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:34:41.647 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:41.643+0000 7f96d7200640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f96d8109a00 0x7f96d81a1250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:34:41.647 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:41.643+0000 7f96d7200640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f96d810f660 con 0x7f96d8109a00 2024-08-19T01:34:41.647 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:41.643+0000 7f96d7200640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f96d810f4e0 con 0x7f96d8074cb0 2024-08-19T01:34:41.647 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:41.643+0000 7f96d7200640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f96d810f7e0 con 0x7f96d8074370 2024-08-19T01:34:41.647 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:41.644+0000 7f96d6800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f96d8109a00 0x7f96d81a1250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:34:41.647 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:41.644+0000 7f96d6800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f96d8109a00 0x7f96d81a1250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:57688/0 (socket says 172.21.15.83:57688) 2024-08-19T01:34:41.647 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:41.644+0000 7f96d6800640 1 -- 172.21.15.83:0/3794750732 learned_addr learned my addr 172.21.15.83:0/3794750732 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-08-19T01:34:41.647 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:41.644+0000 7f96cfe00640 1 --2- 172.21.15.83:0/3794750732 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f96d8074cb0 0x7f96d819a1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:34:41.648 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:41.644+0000 7f96d5e00640 1 --2- 172.21.15.83:0/3794750732 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f96d8074370 0x7f96d8199cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:34:41.648 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:41.644+0000 7f96d6800640 1 -- 172.21.15.83:0/3794750732 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f96d8074370 msgr2=0x7f96d8199cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:34:41.648 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:41.644+0000 7f96d6800640 1 --2- 172.21.15.83:0/3794750732 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f96d8074370 0x7f96d8199cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:41.648 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:41.644+0000 7f96d6800640 1 -- 172.21.15.83:0/3794750732 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f96d8074cb0 msgr2=0x7f96d819a1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:34:41.648 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:41.644+0000 7f96d6800640 1 --2- 172.21.15.83:0/3794750732 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f96d8074cb0 0x7f96d819a1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:41.648 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:41.644+0000 7f96d6800640 1 -- 172.21.15.83:0/3794750732 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f96d81a18f0 con 0x7f96d8109a00 2024-08-19T01:34:41.648 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:41.644+0000 7f96d6800640 1 --2- 172.21.15.83:0/3794750732 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f96d8109a00 0x7f96d81a1250 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f96d000eb10 tx=0x7f96d000c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:34:41.648 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:41.645+0000 7f96cea00640 1 -- 172.21.15.83:0/3794750732 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96d0004b20 con 0x7f96d8109a00 2024-08-19T01:34:41.648 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:41.645+0000 7f96cea00640 1 -- 172.21.15.83:0/3794750732 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f96d0004cc0 con 0x7f96d8109a00 2024-08-19T01:34:41.648 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:41.645+0000 7f96cea00640 1 -- 172.21.15.83:0/3794750732 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96d0010760 con 0x7f96d8109a00 2024-08-19T01:34:41.648 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:41.645+0000 7f96d7200640 1 -- 172.21.15.83:0/3794750732 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f96d81a1be0 con 0x7f96d8109a00 2024-08-19T01:34:41.648 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:41.645+0000 7f96d7200640 1 -- 172.21.15.83:0/3794750732 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f96d81a20b0 con 0x7f96d8109a00 2024-08-19T01:34:41.649 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:41.646+0000 7f96d7200640 1 -- 172.21.15.83:0/3794750732 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f96d8074770 con 0x7f96d8109a00 2024-08-19T01:34:41.649 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:41.647+0000 7f96cea00640 1 -- 172.21.15.83:0/3794750732 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f96d0005020 con 0x7f96d8109a00 2024-08-19T01:34:41.650 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:41.649+0000 7f96cea00640 1 --2- 172.21.15.83:0/3794750732 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f96bc0a2c00 0x7f96bc0a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:34:41.650 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:41.649+0000 7f96cea00640 1 -- 172.21.15.83:0/3794750732 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8269+0+0 (secure 0 0 0) 0x7f96d00cf5f0 con 0x7f96d8109a00 2024-08-19T01:34:41.650 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:41.649+0000 7f96d5e00640 1 --2- 172.21.15.83:0/3794750732 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f96bc0a2c00 0x7f96bc0a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:34:41.651 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:41.650+0000 7f96d5e00640 1 --2- 172.21.15.83:0/3794750732 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f96bc0a2c00 0x7f96bc0a50c0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f96d819ab80 tx=0x7f96c8005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:34:41.654 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:41.652+0000 7f96cea00640 1 -- 172.21.15.83:0/3794750732 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7f96d009c4d0 con 0x7f96d8109a00 2024-08-19T01:34:41.716 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:34:41 smithi167 ceph-mon[21875]: pgmap v334: 97 pgs: 97 active+clean; 641 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-08-19T01:34:41.716 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:34:41 smithi167 ceph-mon[21875]: from='client.? 172.21.15.83:0/3489126117' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]: dispatch 2024-08-19T01:34:41.856 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:41.855+0000 7f96d7200640 1 -- 172.21.15.83:0/3794750732 --> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] -- mgr_command(tid 0: {"prefix": "fs subvolumegroup pin", "vol_name": "cephfs", "group_name": "qa", "pin_type": "random", "pin_setting": "0.10", "target": ["mon-mgr", ""]}) -- 0x7f96d80624c0 con 0x7f96bc0a2c00 2024-08-19T01:34:41.863 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:34:41 smithi083 ceph-mon[17120]: pgmap v334: 97 pgs: 97 active+clean; 641 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-08-19T01:34:41.863 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:34:41 smithi083 ceph-mon[17120]: from='client.? 172.21.15.83:0/3489126117' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]: dispatch 2024-08-19T01:34:41.878 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:41.877+0000 7f96cea00640 1 -- 172.21.15.83:0/3794750732 <== mgr.14150 v2:172.21.15.83:6800/683740916 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+3 (secure 0 0 0) 0x7f96d80624c0 con 0x7f96bc0a2c00 2024-08-19T01:34:41.879 INFO:teuthology.orchestra.run.smithi083.stdout:{} 2024-08-19T01:34:41.881 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:41.879+0000 7f96d7200640 1 -- 172.21.15.83:0/3794750732 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f96bc0a2c00 msgr2=0x7f96bc0a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:34:41.881 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:41.879+0000 7f96d7200640 1 --2- 172.21.15.83:0/3794750732 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f96bc0a2c00 0x7f96bc0a50c0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f96d819ab80 tx=0x7f96c8005f90 comp rx=0 tx=0).stop 2024-08-19T01:34:41.881 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:41.879+0000 7f96d7200640 1 -- 172.21.15.83:0/3794750732 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f96d8109a00 msgr2=0x7f96d81a1250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:34:41.881 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:41.879+0000 7f96d7200640 1 --2- 172.21.15.83:0/3794750732 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f96d8109a00 0x7f96d81a1250 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f96d000eb10 tx=0x7f96d000c2d0 comp rx=0 tx=0).stop 2024-08-19T01:34:41.882 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:41.880+0000 7f96d7200640 1 -- 172.21.15.83:0/3794750732 shutdown_connections 2024-08-19T01:34:41.882 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:41.880+0000 7f96d7200640 1 --2- 172.21.15.83:0/3794750732 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f96bc0a2c00 0x7f96bc0a50c0 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:41.882 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:41.880+0000 7f96d7200640 1 --2- 172.21.15.83:0/3794750732 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f96d8109a00 0x7f96d81a1250 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:41.882 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:41.880+0000 7f96d7200640 1 --2- 172.21.15.83:0/3794750732 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f96d8074cb0 0x7f96d819a1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:41.882 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:41.880+0000 7f96d7200640 1 --2- 172.21.15.83:0/3794750732 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f96d8074370 0x7f96d8199cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:41.882 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:41.880+0000 7f96d7200640 1 -- 172.21.15.83:0/3794750732 >> 172.21.15.83:0/3794750732 conn(0x7f96d80fce00 msgr2=0x7f96d81054a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:34:41.882 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:41.880+0000 7f96d7200640 1 -- 172.21.15.83:0/3794750732 shutdown_connections 2024-08-19T01:34:41.882 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:41.880+0000 7f96d7200640 1 -- 172.21.15.83:0/3794750732 wait complete. 2024-08-19T01:34:41.897 INFO:teuthology.run_tasks:Running task check-counter... 2024-08-19T01:34:41.908 INFO:teuthology.run_tasks:Running task fwd_scrub... 2024-08-19T01:34:41.918 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-08-19T01:34:41.952 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:34:41 smithi117 ceph-mon[22016]: pgmap v334: 97 pgs: 97 active+clean; 641 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-08-19T01:34:41.952 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:34:41 smithi117 ceph-mon[22016]: from='client.? 172.21.15.83:0/3489126117' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]: dispatch 2024-08-19T01:34:42.073 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:42.072+0000 7fb25aa00640 1 -- 172.21.15.83:0/1088139694 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb254105610 msgr2=0x7fb25410bea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:34:42.074 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:42.072+0000 7fb25aa00640 1 --2- 172.21.15.83:0/1088139694 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb254105610 0x7fb25410bea0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fb24c009ca0 tx=0x7fb24c02a8c0 comp rx=0 tx=0).stop 2024-08-19T01:34:42.074 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:42.072+0000 7fb25aa00640 1 -- 172.21.15.83:0/1088139694 shutdown_connections 2024-08-19T01:34:42.074 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:42.072+0000 7fb25aa00640 1 --2- 172.21.15.83:0/1088139694 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb254105610 0x7fb25410bea0 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:42.074 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:42.072+0000 7fb25aa00640 1 --2- 172.21.15.83:0/1088139694 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb254104c50 0x7fb2541050d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:42.074 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:42.073+0000 7fb25aa00640 1 --2- 172.21.15.83:0/1088139694 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb254103a60 0x7fb254103e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:42.074 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:42.073+0000 7fb25aa00640 1 -- 172.21.15.83:0/1088139694 >> 172.21.15.83:0/1088139694 conn(0x7fb2540ff1e0 msgr2=0x7fb254101600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:34:42.074 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:42.073+0000 7fb25aa00640 1 -- 172.21.15.83:0/1088139694 shutdown_connections 2024-08-19T01:34:42.074 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:42.073+0000 7fb25aa00640 1 -- 172.21.15.83:0/1088139694 wait complete. 2024-08-19T01:34:42.076 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:42.073+0000 7fb25aa00640 1 Processor -- start 2024-08-19T01:34:42.076 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:42.074+0000 7fb25aa00640 1 -- start start 2024-08-19T01:34:42.077 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:42.074+0000 7fb25aa00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb254103a60 0x7fb254199d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:34:42.077 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:42.074+0000 7fb25aa00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb254104c50 0x7fb25419a290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:34:42.077 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:42.074+0000 7fb25aa00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb254105610 0x7fb2541a12c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:34:42.077 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:42.074+0000 7fb25aa00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fb25410f710 con 0x7fb254104c50 2024-08-19T01:34:42.077 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:42.074+0000 7fb25aa00640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fb25410f590 con 0x7fb254103a60 2024-08-19T01:34:42.077 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:42.074+0000 7fb25aa00640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fb25410f890 con 0x7fb254105610 2024-08-19T01:34:42.077 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:42.074+0000 7fb253400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb254104c50 0x7fb25419a290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:34:42.077 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:42.074+0000 7fb253400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb254104c50 0x7fb25419a290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:57736/0 (socket says 172.21.15.83:57736) 2024-08-19T01:34:42.077 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:42.074+0000 7fb253400640 1 -- 172.21.15.83:0/3268516603 learned_addr learned my addr 172.21.15.83:0/3268516603 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-08-19T01:34:42.077 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:42.074+0000 7fb253e00640 1 --2- 172.21.15.83:0/3268516603 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb254103a60 0x7fb254199d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:34:42.077 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:42.074+0000 7fb258a00640 1 --2- 172.21.15.83:0/3268516603 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb254105610 0x7fb2541a12c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:34:42.077 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:42.075+0000 7fb253400640 1 -- 172.21.15.83:0/3268516603 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb254105610 msgr2=0x7fb2541a12c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:34:42.078 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:42.075+0000 7fb253400640 1 --2- 172.21.15.83:0/3268516603 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb254105610 0x7fb2541a12c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:42.078 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:42.075+0000 7fb253400640 1 -- 172.21.15.83:0/3268516603 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb254103a60 msgr2=0x7fb254199d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:34:42.078 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:42.075+0000 7fb253400640 1 --2- 172.21.15.83:0/3268516603 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb254103a60 0x7fb254199d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:42.078 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:42.075+0000 7fb253400640 1 -- 172.21.15.83:0/3268516603 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb2541a1960 con 0x7fb254104c50 2024-08-19T01:34:42.078 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:42.075+0000 7fb253400640 1 --2- 172.21.15.83:0/3268516603 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb254104c50 0x7fb25419a290 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fb24000e9f0 tx=0x7fb24000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:34:42.078 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:42.075+0000 7fb250c00640 1 -- 172.21.15.83:0/3268516603 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb24000cd80 con 0x7fb254104c50 2024-08-19T01:34:42.078 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:42.075+0000 7fb250c00640 1 -- 172.21.15.83:0/3268516603 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb240010910 con 0x7fb254104c50 2024-08-19T01:34:42.078 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:42.075+0000 7fb250c00640 1 -- 172.21.15.83:0/3268516603 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb240018990 con 0x7fb254104c50 2024-08-19T01:34:42.078 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:42.075+0000 7fb25aa00640 1 -- 172.21.15.83:0/3268516603 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb2541a1c50 con 0x7fb254104c50 2024-08-19T01:34:42.078 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:42.075+0000 7fb25aa00640 1 -- 172.21.15.83:0/3268516603 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb2541a2150 con 0x7fb254104c50 2024-08-19T01:34:42.079 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:42.076+0000 7fb25aa00640 1 -- 172.21.15.83:0/3268516603 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb25410ad80 con 0x7fb254104c50 2024-08-19T01:34:42.079 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:42.079+0000 7fb250c00640 1 -- 172.21.15.83:0/3268516603 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fb240010ac0 con 0x7fb254104c50 2024-08-19T01:34:42.081 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:42.081+0000 7fb250c00640 1 --2- 172.21.15.83:0/3268516603 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7fb2380a2b90 0x7fb2380a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:34:42.081 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:42.081+0000 7fb250c00640 1 -- 172.21.15.83:0/3268516603 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8269+0+0 (secure 0 0 0) 0x7fb2400cf460 con 0x7fb254104c50 2024-08-19T01:34:42.082 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:42.081+0000 7fb253e00640 1 --2- 172.21.15.83:0/3268516603 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7fb2380a2b90 0x7fb2380a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:34:42.082 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:42.082+0000 7fb253e00640 1 --2- 172.21.15.83:0/3268516603 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7fb2380a2b90 0x7fb2380a5050 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fb25419b220 tx=0x7fb24400b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:34:42.085 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:42.085+0000 7fb250c00640 1 -- 172.21.15.83:0/3268516603 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7fb24009bcf0 con 0x7fb254104c50 2024-08-19T01:34:42.154 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Aug 19 01:34:41 smithi083 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-mgr-x[17344]: 2024-08-19T01:34:41.860+0000 7f0272800640 -1 client.0 error registering admin socket command: (17) File exists 2024-08-19T01:34:42.154 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Aug 19 01:34:41 smithi083 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-mgr-x[17344]: 2024-08-19T01:34:41.860+0000 7f0272800640 -1 client.0 error registering admin socket command: (17) File exists 2024-08-19T01:34:42.155 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Aug 19 01:34:41 smithi083 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-mgr-x[17344]: 2024-08-19T01:34:41.860+0000 7f0272800640 -1 client.0 error registering admin socket command: (17) File exists 2024-08-19T01:34:42.155 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Aug 19 01:34:41 smithi083 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-mgr-x[17344]: 2024-08-19T01:34:41.860+0000 7f0272800640 -1 client.0 error registering admin socket command: (17) File exists 2024-08-19T01:34:42.155 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Aug 19 01:34:41 smithi083 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-mgr-x[17344]: 2024-08-19T01:34:41.860+0000 7f0272800640 -1 client.0 error registering admin socket command: (17) File exists 2024-08-19T01:34:42.264 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:42.263+0000 7fb25aa00640 1 -- 172.21.15.83:0/3268516603 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fb254062bc0 con 0x7fb254104c50 2024-08-19T01:34:42.265 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:42.265+0000 7fb250c00640 1 -- 172.21.15.83:0/3268516603 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v86) ==== 74+0+22722 (secure 0 0 0) 0x7fb24009fff0 con 0x7fb254104c50 2024-08-19T01:34:42.265 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-08-19T01:34:42.266 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":86,"fsid":"bad8780e-5dc9-11ef-bccf-c7b262605968","created":"2024-08-19T01:25:12.626841+0000","modified":"2024-08-19T01:33:05.461998+0000","last_up_change":"2024-08-19T01:31:47.254559+0000","last_in_change":"2024-08-19T01:31:27.652847+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-08-19T01:28:16.862007+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-08-19T01:32:37.455535+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-08-19T01:32:40.201935+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ffa74041-f9d2-4555-bb43-508b8c581487","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6802","nonce":1082526824},{"type":"v1","addr":"172.21.15.83:6803","nonce":1082526824}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6804","nonce":1082526824},{"type":"v1","addr":"172.21.15.83:6805","nonce":1082526824}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6808","nonce":1082526824},{"type":"v1","addr":"172.21.15.83:6809","nonce":1082526824}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6806","nonce":1082526824},{"type":"v1","addr":"172.21.15.83:6807","nonce":1082526824}]},"public_addr":"172.21.15.83:6803/1082526824","cluster_addr":"172.21.15.83:6805/1082526824","heartbeat_back_addr":"172.21.15.83:6809/1082526824","heartbeat_front_addr":"172.21.15.83:6807/1082526824","state":["exists","up"]},{"osd":1,"uuid":"1977570b-b4f5-44fd-bb1e-3a31318be649","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6800","nonce":1925419802},{"type":"v1","addr":"172.21.15.117:6801","nonce":1925419802}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6802","nonce":1925419802},{"type":"v1","addr":"172.21.15.117:6803","nonce":1925419802}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6806","nonce":1925419802},{"type":"v1","addr":"172.21.15.117:6807","nonce":1925419802}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6804","nonce":1925419802},{"type":"v1","addr":"172.21.15.117:6805","nonce":1925419802}]},"public_addr":"172.21.15.117:6801/1925419802","cluster_addr":"172.21.15.117:6803/1925419802","heartbeat_back_addr":"172.21.15.117:6807/1925419802","heartbeat_front_addr":"172.21.15.117:6805/1925419802","state":["exists","up"]},{"osd":2,"uuid":"a75278d1-161f-41d5-a92f-058d1077bf2a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6800","nonce":551625328},{"type":"v1","addr":"172.21.15.167:6801","nonce":551625328}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6802","nonce":551625328},{"type":"v1","addr":"172.21.15.167:6803","nonce":551625328}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6806","nonce":551625328},{"type":"v1","addr":"172.21.15.167:6807","nonce":551625328}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6804","nonce":551625328},{"type":"v1","addr":"172.21.15.167:6805","nonce":551625328}]},"public_addr":"172.21.15.167:6801/551625328","cluster_addr":"172.21.15.167:6803/551625328","heartbeat_back_addr":"172.21.15.167:6807/551625328","heartbeat_front_addr":"172.21.15.167:6805/551625328","state":["exists","up"]},{"osd":3,"uuid":"d1fb16f9-910e-46fa-b1e3-8e5bef4d346d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6810","nonce":3138942062},{"type":"v1","addr":"172.21.15.83:6811","nonce":3138942062}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6812","nonce":3138942062},{"type":"v1","addr":"172.21.15.83:6813","nonce":3138942062}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6816","nonce":3138942062},{"type":"v1","addr":"172.21.15.83:6817","nonce":3138942062}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6814","nonce":3138942062},{"type":"v1","addr":"172.21.15.83:6815","nonce":3138942062}]},"public_addr":"172.21.15.83:6811/3138942062","cluster_addr":"172.21.15.83:6813/3138942062","heartbeat_back_addr":"172.21.15.83:6817/3138942062","heartbeat_front_addr":"172.21.15.83:6815/3138942062","state":["exists","up"]},{"osd":4,"uuid":"8176ed18-1fe2-446d-8994-8c60f6e08f38","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6808","nonce":3978863827},{"type":"v1","addr":"172.21.15.117:6809","nonce":3978863827}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6810","nonce":3978863827},{"type":"v1","addr":"172.21.15.117:6811","nonce":3978863827}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6814","nonce":3978863827},{"type":"v1","addr":"172.21.15.117:6815","nonce":3978863827}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6812","nonce":3978863827},{"type":"v1","addr":"172.21.15.117:6813","nonce":3978863827}]},"public_addr":"172.21.15.117:6809/3978863827","cluster_addr":"172.21.15.117:6811/3978863827","heartbeat_back_addr":"172.21.15.117:6815/3978863827","heartbeat_front_addr":"172.21.15.117:6813/3978863827","state":["exists","up"]},{"osd":5,"uuid":"e01e3f15-25c8-4d2c-be6f-a170f036064c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6808","nonce":2761858035},{"type":"v1","addr":"172.21.15.167:6809","nonce":2761858035}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6810","nonce":2761858035},{"type":"v1","addr":"172.21.15.167:6811","nonce":2761858035}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6814","nonce":2761858035},{"type":"v1","addr":"172.21.15.167:6815","nonce":2761858035}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6812","nonce":2761858035},{"type":"v1","addr":"172.21.15.167:6813","nonce":2761858035}]},"public_addr":"172.21.15.167:6809/2761858035","cluster_addr":"172.21.15.167:6811/2761858035","heartbeat_back_addr":"172.21.15.167:6815/2761858035","heartbeat_front_addr":"172.21.15.167:6813/2761858035","state":["exists","up"]},{"osd":6,"uuid":"78cf0a9b-9165-444f-b073-ebe13016de9d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6818","nonce":260644552},{"type":"v1","addr":"172.21.15.83:6819","nonce":260644552}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6820","nonce":260644552},{"type":"v1","addr":"172.21.15.83:6821","nonce":260644552}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6824","nonce":260644552},{"type":"v1","addr":"172.21.15.83:6825","nonce":260644552}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6822","nonce":260644552},{"type":"v1","addr":"172.21.15.83:6823","nonce":260644552}]},"public_addr":"172.21.15.83:6819/260644552","cluster_addr":"172.21.15.83:6821/260644552","heartbeat_back_addr":"172.21.15.83:6825/260644552","heartbeat_front_addr":"172.21.15.83:6823/260644552","state":["exists","up"]},{"osd":7,"uuid":"ee524f7f-672e-430e-9f58-1a9117513466","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6816","nonce":104113252},{"type":"v1","addr":"172.21.15.117:6817","nonce":104113252}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6818","nonce":104113252},{"type":"v1","addr":"172.21.15.117:6819","nonce":104113252}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6822","nonce":104113252},{"type":"v1","addr":"172.21.15.117:6823","nonce":104113252}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6820","nonce":104113252},{"type":"v1","addr":"172.21.15.117:6821","nonce":104113252}]},"public_addr":"172.21.15.117:6817/104113252","cluster_addr":"172.21.15.117:6819/104113252","heartbeat_back_addr":"172.21.15.117:6823/104113252","heartbeat_front_addr":"172.21.15.117:6821/104113252","state":["exists","up"]},{"osd":8,"uuid":"76d5317d-4845-4138-b210-e7bd2d80472a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":58,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6816","nonce":2196898857},{"type":"v1","addr":"172.21.15.167:6817","nonce":2196898857}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6818","nonce":2196898857},{"type":"v1","addr":"172.21.15.167:6819","nonce":2196898857}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6822","nonce":2196898857},{"type":"v1","addr":"172.21.15.167:6823","nonce":2196898857}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6820","nonce":2196898857},{"type":"v1","addr":"172.21.15.167:6821","nonce":2196898857}]},"public_addr":"172.21.15.167:6817/2196898857","cluster_addr":"172.21.15.167:6819/2196898857","heartbeat_back_addr":"172.21.15.167:6823/2196898857","heartbeat_front_addr":"172.21.15.167:6821/2196898857","state":["exists","up"]},{"osd":9,"uuid":"0151bc2a-3ad2-4db3-8505-5855792cc9bb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6826","nonce":903463450},{"type":"v1","addr":"172.21.15.83:6827","nonce":903463450}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6828","nonce":903463450},{"type":"v1","addr":"172.21.15.83:6829","nonce":903463450}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6832","nonce":903463450},{"type":"v1","addr":"172.21.15.83:6833","nonce":903463450}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6830","nonce":903463450},{"type":"v1","addr":"172.21.15.83:6831","nonce":903463450}]},"public_addr":"172.21.15.83:6827/903463450","cluster_addr":"172.21.15.83:6829/903463450","heartbeat_back_addr":"172.21.15.83:6833/903463450","heartbeat_front_addr":"172.21.15.83:6831/903463450","state":["exists","up"]},{"osd":10,"uuid":"437ed848-0da4-4cd4-8716-0c55bb99b676","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6824","nonce":1183570837},{"type":"v1","addr":"172.21.15.117:6825","nonce":1183570837}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6826","nonce":1183570837},{"type":"v1","addr":"172.21.15.117:6827","nonce":1183570837}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6830","nonce":1183570837},{"type":"v1","addr":"172.21.15.117:6831","nonce":1183570837}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6828","nonce":1183570837},{"type":"v1","addr":"172.21.15.117:6829","nonce":1183570837}]},"public_addr":"172.21.15.117:6825/1183570837","cluster_addr":"172.21.15.117:6827/1183570837","heartbeat_back_addr":"172.21.15.117:6831/1183570837","heartbeat_front_addr":"172.21.15.117:6829/1183570837","state":["exists","up"]},{"osd":11,"uuid":"13345f05-cdc0-4dae-a58b-551abbec1915","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":78,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6824","nonce":187027350},{"type":"v1","addr":"172.21.15.167:6825","nonce":187027350}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6826","nonce":187027350},{"type":"v1","addr":"172.21.15.167:6827","nonce":187027350}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6830","nonce":187027350},{"type":"v1","addr":"172.21.15.167:6831","nonce":187027350}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6828","nonce":187027350},{"type":"v1","addr":"172.21.15.167:6829","nonce":187027350}]},"public_addr":"172.21.15.167:6825/187027350","cluster_addr":"172.21.15.167:6827/187027350","heartbeat_back_addr":"172.21.15.167:6831/187027350","heartbeat_front_addr":"172.21.15.167:6829/187027350","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-19T01:27:28.760517+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-19T01:27:50.493028+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-19T01:28:11.997674+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-19T01:28:38.109934+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-19T01:29:00.173065+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-19T01:29:21.336085+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-19T01:29:47.871498+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-19T01:30:10.720498+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-19T01:30:32.425438+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-19T01:30:59.524989+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-19T01:31:22.806162+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-19T01:31:45.043246+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.1a","mappings":[{"from":8,"to":1}]},{"pgid":"2.1b","mappings":[{"from":2,"to":4}]},{"pgid":"2.1d","mappings":[{"from":8,"to":11}]},{"pgid":"3.6","mappings":[{"from":4,"to":6}]},{"pgid":"3.11","mappings":[{"from":4,"to":8}]},{"pgid":"3.1b","mappings":[{"from":5,"to":9}]},{"pgid":"3.20","mappings":[{"from":5,"to":9}]},{"pgid":"3.31","mappings":[{"from":4,"to":7}]},{"pgid":"3.3b","mappings":[{"from":3,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.83:0/3138095466":"2024-08-20T01:26:04.277716+0000","172.21.15.83:0/3835005198":"2024-08-20T01:26:04.277716+0000","172.21.15.83:0/1790374088":"2024-08-20T01:25:37.228583+0000","172.21.15.83:6801/573907319":"2024-08-20T01:26:04.277716+0000","172.21.15.83:6800/573907319":"2024-08-20T01:26:04.277716+0000","172.21.15.83:0/664676002":"2024-08-20T01:25:37.228583+0000","172.21.15.83:0/2144611449":"2024-08-20T01:25:37.228583+0000","172.21.15.83:6801/3261152818":"2024-08-20T01:25:37.228583+0000","172.21.15.83:0/1609552666":"2024-08-20T01:26:04.277716+0000","172.21.15.83:6800/3261152818":"2024-08-20T01:25:37.228583+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-08-19T01:34:42.268 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:42.267+0000 7fb25aa00640 1 -- 172.21.15.83:0/3268516603 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7fb2380a2b90 msgr2=0x7fb2380a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:34:42.268 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:42.267+0000 7fb25aa00640 1 --2- 172.21.15.83:0/3268516603 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7fb2380a2b90 0x7fb2380a5050 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fb25419b220 tx=0x7fb24400b390 comp rx=0 tx=0).stop 2024-08-19T01:34:42.268 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:42.267+0000 7fb25aa00640 1 -- 172.21.15.83:0/3268516603 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb254104c50 msgr2=0x7fb25419a290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:34:42.268 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:42.267+0000 7fb25aa00640 1 --2- 172.21.15.83:0/3268516603 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb254104c50 0x7fb25419a290 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fb24000e9f0 tx=0x7fb24000eeb0 comp rx=0 tx=0).stop 2024-08-19T01:34:42.268 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:42.267+0000 7fb25aa00640 1 -- 172.21.15.83:0/3268516603 shutdown_connections 2024-08-19T01:34:42.268 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:42.267+0000 7fb25aa00640 1 --2- 172.21.15.83:0/3268516603 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7fb2380a2b90 0x7fb2380a5050 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:42.268 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:42.267+0000 7fb25aa00640 1 --2- 172.21.15.83:0/3268516603 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb254105610 0x7fb2541a12c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:42.268 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:42.267+0000 7fb25aa00640 1 --2- 172.21.15.83:0/3268516603 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb254104c50 0x7fb25419a290 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:42.268 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:42.267+0000 7fb25aa00640 1 --2- 172.21.15.83:0/3268516603 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb254103a60 0x7fb254199d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:42.268 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:42.267+0000 7fb25aa00640 1 -- 172.21.15.83:0/3268516603 >> 172.21.15.83:0/3268516603 conn(0x7fb2540ff1e0 msgr2=0x7fb254107f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:34:42.268 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:42.267+0000 7fb25aa00640 1 -- 172.21.15.83:0/3268516603 shutdown_connections 2024-08-19T01:34:42.269 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:42.267+0000 7fb25aa00640 1 -- 172.21.15.83:0/3268516603 wait complete. 2024-08-19T01:34:42.284 INFO:tasks.fwd_scrub.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-08-19T01:28:16.862007+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-08-19T01:32:37.455535+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '84', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-08-19T01:32:40.201935+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '84', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-08-19T01:34:42.284 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-08-19T01:34:42.443 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:42.442+0000 7fa5a7c00640 1 -- 172.21.15.83:0/4206819757 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa5a0105610 msgr2=0x7fa5a010bea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:34:42.444 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:42.442+0000 7fa5a7c00640 1 --2- 172.21.15.83:0/4206819757 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa5a0105610 0x7fa5a010bea0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fa598009ca0 tx=0x7fa59802a8c0 comp rx=0 tx=0).stop 2024-08-19T01:34:42.444 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:42.443+0000 7fa5a7c00640 1 -- 172.21.15.83:0/4206819757 shutdown_connections 2024-08-19T01:34:42.444 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:42.443+0000 7fa5a7c00640 1 --2- 172.21.15.83:0/4206819757 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa5a0105610 0x7fa5a010bea0 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:42.444 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:42.443+0000 7fa5a7c00640 1 --2- 172.21.15.83:0/4206819757 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa5a0104c50 0x7fa5a01050d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:42.444 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:42.443+0000 7fa5a7c00640 1 --2- 172.21.15.83:0/4206819757 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa5a0103a60 0x7fa5a0103e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:42.444 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:42.443+0000 7fa5a7c00640 1 -- 172.21.15.83:0/4206819757 >> 172.21.15.83:0/4206819757 conn(0x7fa5a00ff1a0 msgr2=0x7fa5a0101600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:34:42.444 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:42.443+0000 7fa5a7c00640 1 -- 172.21.15.83:0/4206819757 shutdown_connections 2024-08-19T01:34:42.444 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:42.443+0000 7fa5a7c00640 1 -- 172.21.15.83:0/4206819757 wait complete. 2024-08-19T01:34:42.446 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:42.443+0000 7fa5a7c00640 1 Processor -- start 2024-08-19T01:34:42.446 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:42.444+0000 7fa5a7c00640 1 -- start start 2024-08-19T01:34:42.446 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:42.444+0000 7fa5a7c00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa5a0103a60 0x7fa5a0199da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:34:42.446 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:42.444+0000 7fa5a7c00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa5a0104c50 0x7fa5a019a2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:34:42.447 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:42.444+0000 7fa5a7c00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa5a0105610 0x7fa5a01a1310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:34:42.447 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:42.444+0000 7fa5a7c00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fa5a010f730 con 0x7fa5a0104c50 2024-08-19T01:34:42.447 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:42.444+0000 7fa5a7c00640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fa5a010f5b0 con 0x7fa5a0103a60 2024-08-19T01:34:42.447 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:42.444+0000 7fa5a7c00640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fa5a010f8b0 con 0x7fa5a0105610 2024-08-19T01:34:42.447 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:42.444+0000 7fa597e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa5a0104c50 0x7fa5a019a2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:34:42.447 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:42.444+0000 7fa597e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa5a0104c50 0x7fa5a019a2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:57768/0 (socket says 172.21.15.83:57768) 2024-08-19T01:34:42.447 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:42.444+0000 7fa597e00640 1 -- 172.21.15.83:0/1070418318 learned_addr learned my addr 172.21.15.83:0/1070418318 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-08-19T01:34:42.447 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:42.444+0000 7fa5a5c00640 1 --2- 172.21.15.83:0/1070418318 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa5a0105610 0x7fa5a01a1310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:34:42.447 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:42.444+0000 7fa5a5200640 1 --2- 172.21.15.83:0/1070418318 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa5a0103a60 0x7fa5a0199da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:34:42.447 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:42.445+0000 7fa597e00640 1 -- 172.21.15.83:0/1070418318 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa5a0105610 msgr2=0x7fa5a01a1310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:34:42.447 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:42.445+0000 7fa597e00640 1 --2- 172.21.15.83:0/1070418318 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa5a0105610 0x7fa5a01a1310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:42.448 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:42.445+0000 7fa597e00640 1 -- 172.21.15.83:0/1070418318 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa5a0103a60 msgr2=0x7fa5a0199da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:34:42.448 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:42.445+0000 7fa597e00640 1 --2- 172.21.15.83:0/1070418318 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa5a0103a60 0x7fa5a0199da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:42.448 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:42.445+0000 7fa597e00640 1 -- 172.21.15.83:0/1070418318 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa5a01a19b0 con 0x7fa5a0104c50 2024-08-19T01:34:42.448 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:42.445+0000 7fa5a5200640 1 --2- 172.21.15.83:0/1070418318 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa5a0103a60 0x7fa5a0199da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-19T01:34:42.448 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:42.445+0000 7fa597e00640 1 --2- 172.21.15.83:0/1070418318 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa5a0104c50 0x7fa5a019a2e0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fa58800e9f0 tx=0x7fa58800edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:34:42.448 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:42.445+0000 7fa59e000640 1 -- 172.21.15.83:0/1070418318 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa58800cd80 con 0x7fa5a0104c50 2024-08-19T01:34:42.448 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:42.445+0000 7fa59e000640 1 -- 172.21.15.83:0/1070418318 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa588005500 con 0x7fa5a0104c50 2024-08-19T01:34:42.448 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:42.445+0000 7fa5a7c00640 1 -- 172.21.15.83:0/1070418318 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa5a01a1ca0 con 0x7fa5a0104c50 2024-08-19T01:34:42.448 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:42.445+0000 7fa59e000640 1 -- 172.21.15.83:0/1070418318 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa588010690 con 0x7fa5a0104c50 2024-08-19T01:34:42.448 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:42.445+0000 7fa5a7c00640 1 -- 172.21.15.83:0/1070418318 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa5a01a21a0 con 0x7fa5a0104c50 2024-08-19T01:34:42.449 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:42.446+0000 7fa5a7c00640 1 -- 172.21.15.83:0/1070418318 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa5a010ae00 con 0x7fa5a0104c50 2024-08-19T01:34:42.449 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:42.448+0000 7fa59e000640 1 -- 172.21.15.83:0/1070418318 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fa5880046c0 con 0x7fa5a0104c50 2024-08-19T01:34:42.450 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:42.449+0000 7fa59e000640 1 --2- 172.21.15.83:0/1070418318 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7fa5840a2b90 0x7fa5840a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:34:42.450 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:42.450+0000 7fa59e000640 1 -- 172.21.15.83:0/1070418318 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8269+0+0 (secure 0 0 0) 0x7fa5880cf530 con 0x7fa5a0104c50 2024-08-19T01:34:42.450 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:42.450+0000 7fa5a5200640 1 --2- 172.21.15.83:0/1070418318 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7fa5840a2b90 0x7fa5840a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:34:42.451 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:42.451+0000 7fa5a5200640 1 --2- 172.21.15.83:0/1070418318 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7fa5840a2b90 0x7fa5840a5050 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fa5a019b270 tx=0x7fa590005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:34:42.454 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:42.453+0000 7fa59e000640 1 -- 172.21.15.83:0/1070418318 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7fa58809c410 con 0x7fa5a0104c50 2024-08-19T01:34:42.634 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:42.630+0000 7fa5a7c00640 1 -- 172.21.15.83:0/1070418318 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fa5a0062b50 con 0x7fa5a0104c50 2024-08-19T01:34:42.635 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:42.631+0000 7fa59e000640 1 -- 172.21.15.83:0/1070418318 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v86) ==== 93+0+10 (secure 0 0 0) 0x7fa58809bd10 con 0x7fa5a0104c50 2024-08-19T01:34:42.635 INFO:teuthology.orchestra.run.smithi083.stdout:pg_num: 1 2024-08-19T01:34:42.636 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:42.633+0000 7fa5a7c00640 1 -- 172.21.15.83:0/1070418318 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7fa5840a2b90 msgr2=0x7fa5840a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:34:42.636 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:42.633+0000 7fa5a7c00640 1 --2- 172.21.15.83:0/1070418318 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7fa5840a2b90 0x7fa5840a5050 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fa5a019b270 tx=0x7fa590005f90 comp rx=0 tx=0).stop 2024-08-19T01:34:42.636 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:42.633+0000 7fa5a7c00640 1 -- 172.21.15.83:0/1070418318 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa5a0104c50 msgr2=0x7fa5a019a2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:34:42.636 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:42.633+0000 7fa5a7c00640 1 --2- 172.21.15.83:0/1070418318 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa5a0104c50 0x7fa5a019a2e0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fa58800e9f0 tx=0x7fa58800edf0 comp rx=0 tx=0).stop 2024-08-19T01:34:42.636 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:42.633+0000 7fa5a7c00640 1 -- 172.21.15.83:0/1070418318 shutdown_connections 2024-08-19T01:34:42.636 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:42.633+0000 7fa5a7c00640 1 --2- 172.21.15.83:0/1070418318 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7fa5840a2b90 0x7fa5840a5050 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:42.636 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:42.633+0000 7fa5a7c00640 1 --2- 172.21.15.83:0/1070418318 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa5a0105610 0x7fa5a01a1310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:42.636 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:42.633+0000 7fa5a7c00640 1 --2- 172.21.15.83:0/1070418318 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa5a0104c50 0x7fa5a019a2e0 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:42.636 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:42.633+0000 7fa5a7c00640 1 --2- 172.21.15.83:0/1070418318 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa5a0103a60 0x7fa5a0199da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:42.636 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:42.633+0000 7fa5a7c00640 1 -- 172.21.15.83:0/1070418318 >> 172.21.15.83:0/1070418318 conn(0x7fa5a00ff1a0 msgr2=0x7fa5a0107fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:34:42.636 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:42.633+0000 7fa5a7c00640 1 -- 172.21.15.83:0/1070418318 shutdown_connections 2024-08-19T01:34:42.636 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:42.633+0000 7fa5a7c00640 1 -- 172.21.15.83:0/1070418318 wait complete. 2024-08-19T01:34:42.648 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:34:42 smithi083 ceph-mon[17120]: from='client.? 172.21.15.83:0/3489126117' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]': finished 2024-08-19T01:34:42.648 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:34:42 smithi083 ceph-mon[17120]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-08-19T01:34:42.648 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:34:42 smithi083 ceph-mon[17120]: from='client.? 172.21.15.83:0/3268516603' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-08-19T01:34:42.650 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-08-19T01:34:42.807 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:42.806+0000 7fd293a00640 1 -- 172.21.15.83:0/2244821120 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd28c105600 msgr2=0x7fd28c10be90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:34:42.807 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:42.806+0000 7fd293a00640 1 --2- 172.21.15.83:0/2244821120 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd28c105600 0x7fd28c10be90 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fd284009ca0 tx=0x7fd28402a8c0 comp rx=0 tx=0).stop 2024-08-19T01:34:42.807 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:42.806+0000 7fd293a00640 1 -- 172.21.15.83:0/2244821120 shutdown_connections 2024-08-19T01:34:42.807 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:42.806+0000 7fd293a00640 1 --2- 172.21.15.83:0/2244821120 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd28c105600 0x7fd28c10be90 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:42.807 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:42.806+0000 7fd293a00640 1 --2- 172.21.15.83:0/2244821120 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd28c104c40 0x7fd28c1050c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:42.807 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:42.806+0000 7fd293a00640 1 --2- 172.21.15.83:0/2244821120 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd28c103a50 0x7fd28c103e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:42.807 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:42.806+0000 7fd293a00640 1 -- 172.21.15.83:0/2244821120 >> 172.21.15.83:0/2244821120 conn(0x7fd28c0ff1d0 msgr2=0x7fd28c1015f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:34:42.807 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:42.806+0000 7fd293a00640 1 -- 172.21.15.83:0/2244821120 shutdown_connections 2024-08-19T01:34:42.807 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:42.806+0000 7fd293a00640 1 -- 172.21.15.83:0/2244821120 wait complete. 2024-08-19T01:34:42.809 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:42.807+0000 7fd293a00640 1 Processor -- start 2024-08-19T01:34:42.809 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:42.807+0000 7fd293a00640 1 -- start start 2024-08-19T01:34:42.809 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:42.807+0000 7fd293a00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd28c103a50 0x7fd28c199d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:34:42.810 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:42.807+0000 7fd293a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd28c104c40 0x7fd28c19a290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:34:42.810 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:42.807+0000 7fd293a00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd28c105600 0x7fd28c1a1300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:34:42.810 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:42.807+0000 7fd293a00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fd28c10f6d0 con 0x7fd28c104c40 2024-08-19T01:34:42.810 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:42.807+0000 7fd293a00640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fd28c10f550 con 0x7fd28c105600 2024-08-19T01:34:42.810 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:42.807+0000 7fd293a00640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fd28c10f850 con 0x7fd28c103a50 2024-08-19T01:34:42.810 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:42.808+0000 7fd28be00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd28c104c40 0x7fd28c19a290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:34:42.810 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:42.808+0000 7fd291a00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd28c105600 0x7fd28c1a1300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:34:42.810 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:42.808+0000 7fd28be00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd28c104c40 0x7fd28c19a290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:57784/0 (socket says 172.21.15.83:57784) 2024-08-19T01:34:42.810 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:42.808+0000 7fd28be00640 1 -- 172.21.15.83:0/2869374896 learned_addr learned my addr 172.21.15.83:0/2869374896 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-08-19T01:34:42.810 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:42.808+0000 7fd291000640 1 --2- 172.21.15.83:0/2869374896 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd28c103a50 0x7fd28c199d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:34:42.810 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:42.808+0000 7fd28be00640 1 -- 172.21.15.83:0/2869374896 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd28c103a50 msgr2=0x7fd28c199d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:34:42.810 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:42.808+0000 7fd28be00640 1 --2- 172.21.15.83:0/2869374896 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd28c103a50 0x7fd28c199d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:42.811 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:42.808+0000 7fd28be00640 1 -- 172.21.15.83:0/2869374896 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd28c105600 msgr2=0x7fd28c1a1300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:34:42.811 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:42.808+0000 7fd28be00640 1 --2- 172.21.15.83:0/2869374896 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd28c105600 0x7fd28c1a1300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:42.811 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:42.808+0000 7fd28be00640 1 -- 172.21.15.83:0/2869374896 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd28c1a1970 con 0x7fd28c104c40 2024-08-19T01:34:42.811 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:42.808+0000 7fd291a00640 1 --2- 172.21.15.83:0/2869374896 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd28c105600 0x7fd28c1a1300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-19T01:34:42.811 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:42.808+0000 7fd28be00640 1 --2- 172.21.15.83:0/2869374896 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd28c104c40 0x7fd28c19a290 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7fd27800e9f0 tx=0x7fd27800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:34:42.811 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:42.808+0000 7fd289600640 1 -- 172.21.15.83:0/2869374896 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd27800cd80 con 0x7fd28c104c40 2024-08-19T01:34:42.811 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:42.809+0000 7fd289600640 1 -- 172.21.15.83:0/2869374896 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd278010910 con 0x7fd28c104c40 2024-08-19T01:34:42.811 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:42.809+0000 7fd289600640 1 -- 172.21.15.83:0/2869374896 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd278018990 con 0x7fd28c104c40 2024-08-19T01:34:42.811 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:42.809+0000 7fd293a00640 1 -- 172.21.15.83:0/2869374896 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd28c1a1c60 con 0x7fd28c104c40 2024-08-19T01:34:42.811 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:42.809+0000 7fd293a00640 1 -- 172.21.15.83:0/2869374896 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd28c1a2160 con 0x7fd28c104c40 2024-08-19T01:34:42.812 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:42.810+0000 7fd293a00640 1 -- 172.21.15.83:0/2869374896 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd28c10ae10 con 0x7fd28c104c40 2024-08-19T01:34:42.812 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:42.811+0000 7fd289600640 1 -- 172.21.15.83:0/2869374896 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fd278010ac0 con 0x7fd28c104c40 2024-08-19T01:34:42.813 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:42.813+0000 7fd289600640 1 --2- 172.21.15.83:0/2869374896 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7fd2740a2c10 0x7fd2740a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:34:42.813 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:42.813+0000 7fd289600640 1 -- 172.21.15.83:0/2869374896 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8269+0+0 (secure 0 0 0) 0x7fd2780cf2c0 con 0x7fd28c104c40 2024-08-19T01:34:42.814 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:42.813+0000 7fd291000640 1 --2- 172.21.15.83:0/2869374896 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7fd2740a2c10 0x7fd2740a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:34:42.814 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:42.814+0000 7fd291000640 1 --2- 172.21.15.83:0/2869374896 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7fd2740a2c10 0x7fd2740a50d0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fd28c104aa0 tx=0x7fd280005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:34:42.817 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:42.817+0000 7fd289600640 1 -- 172.21.15.83:0/2869374896 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7fd27809bbb0 con 0x7fd28c104c40 2024-08-19T01:34:42.952 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:34:42 smithi117 ceph-mon[22016]: from='client.? 172.21.15.83:0/3489126117' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]': finished 2024-08-19T01:34:42.952 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:34:42 smithi117 ceph-mon[22016]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-08-19T01:34:42.952 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:34:42 smithi117 ceph-mon[22016]: from='client.? 172.21.15.83:0/3268516603' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-08-19T01:34:42.966 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:34:42 smithi167 ceph-mon[21875]: from='client.? 172.21.15.83:0/3489126117' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]': finished 2024-08-19T01:34:42.966 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:34:42 smithi167 ceph-mon[21875]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-08-19T01:34:42.966 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:34:42 smithi167 ceph-mon[21875]: from='client.? 172.21.15.83:0/3268516603' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-08-19T01:34:42.996 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:42.996+0000 7fd293a00640 1 -- 172.21.15.83:0/2869374896 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fd28c062bc0 con 0x7fd28c104c40 2024-08-19T01:34:42.997 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:42.996+0000 7fd289600640 1 -- 172.21.15.83:0/2869374896 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v86) ==== 104+0+11 (secure 0 0 0) 0x7fd27809feb0 con 0x7fd28c104c40 2024-08-19T01:34:42.997 INFO:teuthology.orchestra.run.smithi083.stdout:pg_num: 32 2024-08-19T01:34:42.999 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:42.998+0000 7fd293a00640 1 -- 172.21.15.83:0/2869374896 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7fd2740a2c10 msgr2=0x7fd2740a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:34:42.999 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:42.998+0000 7fd293a00640 1 --2- 172.21.15.83:0/2869374896 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7fd2740a2c10 0x7fd2740a50d0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fd28c104aa0 tx=0x7fd280005f90 comp rx=0 tx=0).stop 2024-08-19T01:34:42.999 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:42.998+0000 7fd293a00640 1 -- 172.21.15.83:0/2869374896 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd28c104c40 msgr2=0x7fd28c19a290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:34:42.999 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:42.998+0000 7fd293a00640 1 --2- 172.21.15.83:0/2869374896 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd28c104c40 0x7fd28c19a290 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7fd27800e9f0 tx=0x7fd27800eeb0 comp rx=0 tx=0).stop 2024-08-19T01:34:42.999 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:42.998+0000 7fd293a00640 1 -- 172.21.15.83:0/2869374896 shutdown_connections 2024-08-19T01:34:42.999 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:42.998+0000 7fd293a00640 1 --2- 172.21.15.83:0/2869374896 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7fd2740a2c10 0x7fd2740a50d0 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:43.000 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:42.998+0000 7fd293a00640 1 --2- 172.21.15.83:0/2869374896 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd28c105600 0x7fd28c1a1300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:43.000 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:42.998+0000 7fd293a00640 1 --2- 172.21.15.83:0/2869374896 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd28c104c40 0x7fd28c19a290 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:43.000 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:42.998+0000 7fd293a00640 1 --2- 172.21.15.83:0/2869374896 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd28c103a50 0x7fd28c199d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:43.000 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:42.998+0000 7fd293a00640 1 -- 172.21.15.83:0/2869374896 >> 172.21.15.83:0/2869374896 conn(0x7fd28c0ff1d0 msgr2=0x7fd28c107fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:34:43.000 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:42.999+0000 7fd293a00640 1 -- 172.21.15.83:0/2869374896 shutdown_connections 2024-08-19T01:34:43.000 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:42.999+0000 7fd293a00640 1 -- 172.21.15.83:0/2869374896 wait complete. 2024-08-19T01:34:43.016 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-08-19T01:34:43.174 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:43.172+0000 7fcf04000640 1 -- 172.21.15.83:0/3726779340 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcefc103a50 msgr2=0x7fcefc103e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:34:43.174 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:43.172+0000 7fcf04000640 1 --2- 172.21.15.83:0/3726779340 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcefc103a50 0x7fcefc103e50 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fcef0009900 tx=0x7fcef002a410 comp rx=0 tx=0).stop 2024-08-19T01:34:43.174 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:43.172+0000 7fcf04000640 1 -- 172.21.15.83:0/3726779340 shutdown_connections 2024-08-19T01:34:43.174 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:43.172+0000 7fcf04000640 1 --2- 172.21.15.83:0/3726779340 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fcefc105600 0x7fcefc10be90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:43.174 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:43.172+0000 7fcf04000640 1 --2- 172.21.15.83:0/3726779340 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcefc104c40 0x7fcefc1050c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:43.174 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:43.172+0000 7fcf04000640 1 --2- 172.21.15.83:0/3726779340 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcefc103a50 0x7fcefc103e50 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:43.174 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:43.172+0000 7fcf04000640 1 -- 172.21.15.83:0/3726779340 >> 172.21.15.83:0/3726779340 conn(0x7fcefc0ff1d0 msgr2=0x7fcefc1015f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:34:43.174 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:43.173+0000 7fcf04000640 1 -- 172.21.15.83:0/3726779340 shutdown_connections 2024-08-19T01:34:43.174 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:43.173+0000 7fcf04000640 1 -- 172.21.15.83:0/3726779340 wait complete. 2024-08-19T01:34:43.176 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:43.173+0000 7fcf04000640 1 Processor -- start 2024-08-19T01:34:43.176 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:43.174+0000 7fcf04000640 1 -- start start 2024-08-19T01:34:43.176 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:43.174+0000 7fcf04000640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fcefc103a50 0x7fcefc199dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:34:43.176 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:43.174+0000 7fcf04000640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcefc104c40 0x7fcefc19a310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:34:43.177 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:43.174+0000 7fcf04000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcefc105600 0x7fcefc1a1380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:34:43.177 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:43.174+0000 7fcf04000640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fcefc10f720 con 0x7fcefc105600 2024-08-19T01:34:43.177 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:43.174+0000 7fcf04000640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fcefc10f5a0 con 0x7fcefc104c40 2024-08-19T01:34:43.177 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:43.174+0000 7fcf04000640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fcefc10f8a0 con 0x7fcefc103a50 2024-08-19T01:34:43.177 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:43.174+0000 7fcf02000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcefc105600 0x7fcefc1a1380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:34:43.177 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:43.174+0000 7fcf02000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcefc105600 0x7fcefc1a1380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:57804/0 (socket says 172.21.15.83:57804) 2024-08-19T01:34:43.177 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:43.174+0000 7fcf02000640 1 -- 172.21.15.83:0/184213440 learned_addr learned my addr 172.21.15.83:0/184213440 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-08-19T01:34:43.177 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:43.174+0000 7fcf00c00640 1 --2- 172.21.15.83:0/184213440 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcefc104c40 0x7fcefc19a310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:34:43.177 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:43.174+0000 7fcf01600640 1 --2- 172.21.15.83:0/184213440 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fcefc103a50 0x7fcefc199dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:34:43.177 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:43.175+0000 7fcf02000640 1 -- 172.21.15.83:0/184213440 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fcefc103a50 msgr2=0x7fcefc199dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:34:43.178 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:43.175+0000 7fcf02000640 1 --2- 172.21.15.83:0/184213440 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fcefc103a50 0x7fcefc199dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:43.178 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:43.175+0000 7fcf02000640 1 -- 172.21.15.83:0/184213440 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcefc104c40 msgr2=0x7fcefc19a310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:34:43.178 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:43.175+0000 7fcf02000640 1 --2- 172.21.15.83:0/184213440 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcefc104c40 0x7fcefc19a310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:43.178 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:43.175+0000 7fcf02000640 1 -- 172.21.15.83:0/184213440 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcefc1a19f0 con 0x7fcefc105600 2024-08-19T01:34:43.178 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:43.175+0000 7fcf02000640 1 --2- 172.21.15.83:0/184213440 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcefc105600 0x7fcefc1a1380 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fcef400eb10 tx=0x7fcef400c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:34:43.178 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:43.175+0000 7fcefa000640 1 -- 172.21.15.83:0/184213440 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcef4004b20 con 0x7fcefc105600 2024-08-19T01:34:43.178 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:43.175+0000 7fcefa000640 1 -- 172.21.15.83:0/184213440 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcef4004cc0 con 0x7fcefc105600 2024-08-19T01:34:43.178 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:43.175+0000 7fcf04000640 1 -- 172.21.15.83:0/184213440 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcefc1a1ce0 con 0x7fcefc105600 2024-08-19T01:34:43.178 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:43.175+0000 7fcefa000640 1 -- 172.21.15.83:0/184213440 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcef4010760 con 0x7fcefc105600 2024-08-19T01:34:43.178 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:43.175+0000 7fcf04000640 1 -- 172.21.15.83:0/184213440 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcefc1a2220 con 0x7fcefc105600 2024-08-19T01:34:43.179 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:43.176+0000 7fcf04000640 1 -- 172.21.15.83:0/184213440 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcefc10ae40 con 0x7fcefc105600 2024-08-19T01:34:43.179 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:43.178+0000 7fcefa000640 1 -- 172.21.15.83:0/184213440 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fcef4005020 con 0x7fcefc105600 2024-08-19T01:34:43.180 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:43.179+0000 7fcefa000640 1 --2- 172.21.15.83:0/184213440 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7fcee40a2c10 0x7fcee40a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:34:43.180 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:43.180+0000 7fcefa000640 1 -- 172.21.15.83:0/184213440 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8269+0+0 (secure 0 0 0) 0x7fcef40cf6e0 con 0x7fcefc105600 2024-08-19T01:34:43.180 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:43.180+0000 7fcf01600640 1 --2- 172.21.15.83:0/184213440 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7fcee40a2c10 0x7fcee40a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:34:43.180 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:43.180+0000 7fcf01600640 1 --2- 172.21.15.83:0/184213440 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7fcee40a2c10 0x7fcee40a50d0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fcef002aa00 tx=0x7fcef00315b0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:34:43.183 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:43.183+0000 7fcefa000640 1 -- 172.21.15.83:0/184213440 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7fcef409c5c0 con 0x7fcefc105600 2024-08-19T01:34:43.357 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:43.356+0000 7fcf04000640 1 -- 172.21.15.83:0/184213440 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fcefc062bc0 con 0x7fcefc105600 2024-08-19T01:34:43.357 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:43.357+0000 7fcefa000640 1 -- 172.21.15.83:0/184213440 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v86) ==== 100+0+11 (secure 0 0 0) 0x7fcef409bec0 con 0x7fcefc105600 2024-08-19T01:34:43.358 INFO:teuthology.orchestra.run.smithi083.stdout:pg_num: 64 2024-08-19T01:34:43.359 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:43.359+0000 7fcf04000640 1 -- 172.21.15.83:0/184213440 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7fcee40a2c10 msgr2=0x7fcee40a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:34:43.359 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:43.359+0000 7fcf04000640 1 --2- 172.21.15.83:0/184213440 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7fcee40a2c10 0x7fcee40a50d0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fcef002aa00 tx=0x7fcef00315b0 comp rx=0 tx=0).stop 2024-08-19T01:34:43.360 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:43.359+0000 7fcf04000640 1 -- 172.21.15.83:0/184213440 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcefc105600 msgr2=0x7fcefc1a1380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:34:43.360 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:43.359+0000 7fcf04000640 1 --2- 172.21.15.83:0/184213440 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcefc105600 0x7fcefc1a1380 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fcef400eb10 tx=0x7fcef400c2d0 comp rx=0 tx=0).stop 2024-08-19T01:34:43.360 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:43.359+0000 7fcf04000640 1 -- 172.21.15.83:0/184213440 shutdown_connections 2024-08-19T01:34:43.360 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:43.359+0000 7fcf04000640 1 --2- 172.21.15.83:0/184213440 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7fcee40a2c10 0x7fcee40a50d0 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:43.360 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:43.359+0000 7fcf04000640 1 --2- 172.21.15.83:0/184213440 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcefc105600 0x7fcefc1a1380 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:43.360 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:43.359+0000 7fcf04000640 1 --2- 172.21.15.83:0/184213440 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcefc104c40 0x7fcefc19a310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:43.360 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:43.359+0000 7fcf04000640 1 --2- 172.21.15.83:0/184213440 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fcefc103a50 0x7fcefc199dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:43.360 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:43.359+0000 7fcf04000640 1 -- 172.21.15.83:0/184213440 >> 172.21.15.83:0/184213440 conn(0x7fcefc0ff1d0 msgr2=0x7fcefc108010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:34:43.361 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:43.360+0000 7fcf04000640 1 -- 172.21.15.83:0/184213440 shutdown_connections 2024-08-19T01:34:43.361 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:43.360+0000 7fcf04000640 1 -- 172.21.15.83:0/184213440 wait complete. 2024-08-19T01:34:43.377 INFO:tasks.fwd_scrub:Wait for all MDSs to reach steady state... 2024-08-19T01:34:43.377 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833 shell --fsid bad8780e-5dc9-11ef-bccf-c7b262605968 -- ceph fs dump --format=json 2024-08-19T01:34:43.654 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:34:43 smithi083 ceph-mon[17120]: from='client.14997 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup pin", "vol_name": "cephfs", "group_name": "qa", "pin_type": "random", "pin_setting": "0.10", "target": ["mon-mgr", ""]}]: dispatch 2024-08-19T01:34:43.655 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:34:43 smithi083 ceph-mon[17120]: pgmap v335: 97 pgs: 97 active+clean; 641 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 170 B/s wr, 5 op/s 2024-08-19T01:34:43.655 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:34:43 smithi083 ceph-mon[17120]: from='client.? 172.21.15.83:0/1070418318' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-08-19T01:34:43.655 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:34:43 smithi083 ceph-mon[17120]: from='client.? 172.21.15.83:0/2869374896' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-08-19T01:34:43.655 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:34:43 smithi083 ceph-mon[17120]: from='client.? 172.21.15.83:0/184213440' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-08-19T01:34:43.764 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/bad8780e-5dc9-11ef-bccf-c7b262605968/mon.a/config 2024-08-19T01:34:43.952 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:34:43 smithi117 ceph-mon[22016]: from='client.14997 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup pin", "vol_name": "cephfs", "group_name": "qa", "pin_type": "random", "pin_setting": "0.10", "target": ["mon-mgr", ""]}]: dispatch 2024-08-19T01:34:43.953 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:34:43 smithi117 ceph-mon[22016]: pgmap v335: 97 pgs: 97 active+clean; 641 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 170 B/s wr, 5 op/s 2024-08-19T01:34:43.953 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:34:43 smithi117 ceph-mon[22016]: from='client.? 172.21.15.83:0/1070418318' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-08-19T01:34:43.953 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:34:43 smithi117 ceph-mon[22016]: from='client.? 172.21.15.83:0/2869374896' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-08-19T01:34:43.953 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:34:43 smithi117 ceph-mon[22016]: from='client.? 172.21.15.83:0/184213440' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-08-19T01:34:43.966 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:34:43 smithi167 ceph-mon[21875]: from='client.14997 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup pin", "vol_name": "cephfs", "group_name": "qa", "pin_type": "random", "pin_setting": "0.10", "target": ["mon-mgr", ""]}]: dispatch 2024-08-19T01:34:43.966 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:34:43 smithi167 ceph-mon[21875]: pgmap v335: 97 pgs: 97 active+clean; 641 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 170 B/s wr, 5 op/s 2024-08-19T01:34:43.966 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:34:43 smithi167 ceph-mon[21875]: from='client.? 172.21.15.83:0/1070418318' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-08-19T01:34:43.966 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:34:43 smithi167 ceph-mon[21875]: from='client.? 172.21.15.83:0/2869374896' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-08-19T01:34:43.966 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:34:43 smithi167 ceph-mon[21875]: from='client.? 172.21.15.83:0/184213440' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-08-19T01:34:44.325 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:44.323+0000 7ff7aa600640 1 -- 172.21.15.83:0/1493217233 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff7a41018b0 msgr2=0x7ff7a4101d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:34:44.325 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:44.323+0000 7ff7aa600640 1 --2- 172.21.15.83:0/1493217233 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff7a41018b0 0x7ff7a4101d10 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7ff790009930 tx=0x7ff79002a420 comp rx=0 tx=0).stop 2024-08-19T01:34:44.325 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:44.323+0000 7ff7aa600640 1 -- 172.21.15.83:0/1493217233 shutdown_connections 2024-08-19T01:34:44.325 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:44.323+0000 7ff7aa600640 1 --2- 172.21.15.83:0/1493217233 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff7a4102250 0x7ff7a410e780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:44.325 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:44.323+0000 7ff7aa600640 1 --2- 172.21.15.83:0/1493217233 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff7a41018b0 0x7ff7a4101d10 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:44.325 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:44.323+0000 7ff7aa600640 1 --2- 172.21.15.83:0/1493217233 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff7a41078b0 0x7ff7a4107c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:44.325 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:44.323+0000 7ff7aa600640 1 -- 172.21.15.83:0/1493217233 >> 172.21.15.83:0/1493217233 conn(0x7ff7a40fd5f0 msgr2=0x7ff7a40ffa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:34:44.326 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:44.323+0000 7ff7aa600640 1 -- 172.21.15.83:0/1493217233 shutdown_connections 2024-08-19T01:34:44.326 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:44.323+0000 7ff7aa600640 1 -- 172.21.15.83:0/1493217233 wait complete. 2024-08-19T01:34:44.326 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:44.324+0000 7ff7aa600640 1 Processor -- start 2024-08-19T01:34:44.326 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:44.325+0000 7ff7aa600640 1 -- start start 2024-08-19T01:34:44.327 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:44.325+0000 7ff7aa600640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff7a41018b0 0x7ff7a4074220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:34:44.327 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:44.325+0000 7ff7aa600640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff7a4102250 0x7ff7a4074760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:34:44.327 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:44.325+0000 7ff7aa600640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff7a41078b0 0x7ff7a4074ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:34:44.327 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:44.325+0000 7ff7aa600640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7ff7a4111cc0 con 0x7ff7a41018b0 2024-08-19T01:34:44.327 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:44.325+0000 7ff7aa600640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7ff7a4111b40 con 0x7ff7a4102250 2024-08-19T01:34:44.327 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:44.325+0000 7ff7aa600640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7ff7a4111e40 con 0x7ff7a41078b0 2024-08-19T01:34:44.327 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:44.325+0000 7ff7a3400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff7a41018b0 0x7ff7a4074220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:34:44.328 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:44.325+0000 7ff7a3e00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff7a41078b0 0x7ff7a4074ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:34:44.328 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:44.325+0000 7ff7a3400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff7a41018b0 0x7ff7a4074220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:57814/0 (socket says 172.21.15.83:57814) 2024-08-19T01:34:44.328 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:44.325+0000 7ff7a3400640 1 -- 172.21.15.83:0/1390910346 learned_addr learned my addr 172.21.15.83:0/1390910346 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-08-19T01:34:44.328 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:44.325+0000 7ff7a2a00640 1 --2- 172.21.15.83:0/1390910346 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff7a4102250 0x7ff7a4074760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:34:44.328 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:44.326+0000 7ff7a3400640 1 -- 172.21.15.83:0/1390910346 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff7a41078b0 msgr2=0x7ff7a4074ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:34:44.328 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:44.326+0000 7ff7a3400640 1 --2- 172.21.15.83:0/1390910346 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff7a41078b0 0x7ff7a4074ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:44.328 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:44.326+0000 7ff7a3400640 1 -- 172.21.15.83:0/1390910346 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff7a4102250 msgr2=0x7ff7a4074760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:34:44.328 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:44.326+0000 7ff7a3400640 1 --2- 172.21.15.83:0/1390910346 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff7a4102250 0x7ff7a4074760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:44.328 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:44.326+0000 7ff7a3400640 1 -- 172.21.15.83:0/1390910346 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff7a41a6680 con 0x7ff7a41018b0 2024-08-19T01:34:44.328 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:44.326+0000 7ff7a3400640 1 --2- 172.21.15.83:0/1390910346 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff7a41018b0 0x7ff7a4074220 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7ff79400d950 tx=0x7ff79400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:34:44.328 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:44.326+0000 7ff79be00640 1 -- 172.21.15.83:0/1390910346 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff794017070 con 0x7ff7a41018b0 2024-08-19T01:34:44.328 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:44.326+0000 7ff79be00640 1 -- 172.21.15.83:0/1390910346 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff794009c30 con 0x7ff7a41018b0 2024-08-19T01:34:44.328 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:44.326+0000 7ff79be00640 1 -- 172.21.15.83:0/1390910346 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff79400f6b0 con 0x7ff7a41018b0 2024-08-19T01:34:44.328 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:44.326+0000 7ff7aa600640 1 -- 172.21.15.83:0/1390910346 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff7a41a6970 con 0x7ff7a41018b0 2024-08-19T01:34:44.329 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:44.327+0000 7ff7aa600640 1 -- 172.21.15.83:0/1390910346 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff7a41a6ed0 con 0x7ff7a41018b0 2024-08-19T01:34:44.329 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:44.328+0000 7ff7aa600640 1 -- 172.21.15.83:0/1390910346 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff7a4101d10 con 0x7ff7a41018b0 2024-08-19T01:34:44.330 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:44.329+0000 7ff79be00640 1 -- 172.21.15.83:0/1390910346 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 19) ==== 137884+0+0 (secure 0 0 0) 0x7ff7940109d0 con 0x7ff7a41018b0 2024-08-19T01:34:44.331 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:44.331+0000 7ff79be00640 1 --2- 172.21.15.83:0/1390910346 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7ff7880a2c90 0x7ff7880a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:34:44.331 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:44.331+0000 7ff79be00640 1 -- 172.21.15.83:0/1390910346 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8269+0+0 (secure 0 0 0) 0x7ff7940cecf0 con 0x7ff7a41018b0 2024-08-19T01:34:44.331 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:44.331+0000 7ff7a2a00640 1 --2- 172.21.15.83:0/1390910346 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7ff7880a2c90 0x7ff7880a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:34:44.332 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:44.332+0000 7ff7a2a00640 1 --2- 172.21.15.83:0/1390910346 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7ff7880a2c90 0x7ff7880a5150 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7ff790006010 tx=0x7ff790000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:34:44.335 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:44.335+0000 7ff79be00640 1 -- 172.21.15.83:0/1390910346 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7ff794009de0 con 0x7ff7a41018b0 2024-08-19T01:34:44.547 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:44.546+0000 7ff7aa600640 1 -- 172.21.15.83:0/1390910346 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff7a4109e40 con 0x7ff7a41018b0 2024-08-19T01:34:44.548 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:44.547+0000 7ff79be00640 1 -- 172.21.15.83:0/1390910346 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+11920 (secure 0 0 0) 0x7ff79409b420 con 0x7ff7a41018b0 2024-08-19T01:34:44.548 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-08-19T01:34:44.548 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":25,"btime":"2024-08-19T01:34:41:456953+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14421,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6835/988556067","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":988556067},{"type":"v1","addr":"172.21.15.83:6835","nonce":988556067}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14433,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6837/869702632","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6836","nonce":869702632},{"type":"v1","addr":"172.21.15.83:6837","nonce":869702632}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14475,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6841/1795975999","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6840","nonce":1795975999},{"type":"v1","addr":"172.21.15.83:6841","nonce":1795975999}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24430,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6833/2505076046","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6832","nonce":2505076046},{"type":"v1","addr":"172.21.15.167:6833","nonce":2505076046}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24449,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6833/3557520478","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":3557520478},{"type":"v1","addr":"172.21.15.117:6833","nonce":3557520478}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24466,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6835/3482045966","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":3482045966},{"type":"v1","addr":"172.21.15.167:6835","nonce":3482045966}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7}],"filesystems":[{"mdsmap":{"epoch":25,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-08-19T01:32:42.054769+0000","modified":"2024-08-19T01:34:40.528901+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24545,"mds_1":24527,"mds_2":24509},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24478":{"gid":24478,"name":"c","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.167:6837/1762130408","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6836","nonce":1762130408},{"type":"v1","addr":"172.21.15.167:6837","nonce":1762130408}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24485":{"gid":24485,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.117:6835/1024213994","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":1024213994},{"type":"v1","addr":"172.21.15.117:6835","nonce":1024213994}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24496":{"gid":24496,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.167:6839/3507387682","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6838","nonce":3507387682},{"type":"v1","addr":"172.21.15.167:6839","nonce":3507387682}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24509":{"gid":24509,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":10,"addr":"172.21.15.117:6837/3185224684","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":3185224684},{"type":"v1","addr":"172.21.15.117:6837","nonce":3185224684}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24527":{"gid":24527,"name":"g","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.83:6839/3239261845","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6838","nonce":3239261845},{"type":"v1","addr":"172.21.15.83:6839","nonce":3239261845}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24545":{"gid":24545,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.117:6839/1967595704","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6838","nonce":1967595704},{"type":"v1","addr":"172.21.15.117:6839","nonce":1967595704}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24545,"qdb_cluster":[24545,24527,24509]},"id":1}]} 2024-08-19T01:34:44.548 INFO:teuthology.orchestra.run.smithi083.stderr:dumped fsmap epoch 25 2024-08-19T01:34:44.550 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:44.549+0000 7ff7aa600640 1 -- 172.21.15.83:0/1390910346 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7ff7880a2c90 msgr2=0x7ff7880a5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:34:44.550 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:44.549+0000 7ff7aa600640 1 --2- 172.21.15.83:0/1390910346 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7ff7880a2c90 0x7ff7880a5150 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7ff790006010 tx=0x7ff790000bc0 comp rx=0 tx=0).stop 2024-08-19T01:34:44.550 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:44.549+0000 7ff7aa600640 1 -- 172.21.15.83:0/1390910346 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff7a41018b0 msgr2=0x7ff7a4074220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:34:44.550 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:44.549+0000 7ff7aa600640 1 --2- 172.21.15.83:0/1390910346 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff7a41018b0 0x7ff7a4074220 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7ff79400d950 tx=0x7ff79400de10 comp rx=0 tx=0).stop 2024-08-19T01:34:44.551 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:44.550+0000 7ff7aa600640 1 -- 172.21.15.83:0/1390910346 shutdown_connections 2024-08-19T01:34:44.551 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:44.550+0000 7ff7aa600640 1 --2- 172.21.15.83:0/1390910346 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7ff7880a2c90 0x7ff7880a5150 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:44.551 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:44.550+0000 7ff7aa600640 1 --2- 172.21.15.83:0/1390910346 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff7a41078b0 0x7ff7a4074ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:44.551 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:44.550+0000 7ff7aa600640 1 --2- 172.21.15.83:0/1390910346 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff7a4102250 0x7ff7a4074760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:44.551 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:44.550+0000 7ff7aa600640 1 --2- 172.21.15.83:0/1390910346 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff7a41018b0 0x7ff7a4074220 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:44.551 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:44.550+0000 7ff7aa600640 1 -- 172.21.15.83:0/1390910346 >> 172.21.15.83:0/1390910346 conn(0x7ff7a40fd5f0 msgr2=0x7ff7a40fdda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:34:44.551 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:44.550+0000 7ff7aa600640 1 -- 172.21.15.83:0/1390910346 shutdown_connections 2024-08-19T01:34:44.551 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:44.550+0000 7ff7aa600640 1 -- 172.21.15.83:0/1390910346 wait complete. 2024-08-19T01:34:44.904 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:34:44 smithi083 ceph-mon[17120]: mgrmap e19: x(active, since 8m), standbys: z, y 2024-08-19T01:34:44.952 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:34:44 smithi117 ceph-mon[22016]: mgrmap e19: x(active, since 8m), standbys: z, y 2024-08-19T01:34:44.966 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:34:44 smithi167 ceph-mon[21875]: mgrmap e19: x(active, since 8m), standbys: z, y 2024-08-19T01:34:45.442 INFO:tasks.fwd_scrub:Ready to start scrub thrashing 2024-08-19T01:34:45.442 INFO:tasks.fwd_scrub.ceph_manager:waiting for clean 2024-08-19T01:34:45.442 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-08-19T01:34:45.599 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:45.598+0000 7f0820c00640 1 -- 172.21.15.83:0/4237627777 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f081c102a50 msgr2=0x7f081c102ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:34:45.602 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:45.598+0000 7f0820c00640 1 --2- 172.21.15.83:0/4237627777 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f081c102a50 0x7f081c102ed0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f0808009900 tx=0x7f080802a410 comp rx=0 tx=0).stop 2024-08-19T01:34:45.602 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:45.598+0000 7f0820c00640 1 -- 172.21.15.83:0/4237627777 shutdown_connections 2024-08-19T01:34:45.602 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:45.598+0000 7f0820c00640 1 --2- 172.21.15.83:0/4237627777 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f081c103410 0x7f081c109ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:45.602 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:45.598+0000 7f0820c00640 1 --2- 172.21.15.83:0/4237627777 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f081c102a50 0x7f081c102ed0 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:45.602 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:45.598+0000 7f0820c00640 1 --2- 172.21.15.83:0/4237627777 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f081c101860 0x7f081c101c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:45.602 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:45.598+0000 7f0820c00640 1 -- 172.21.15.83:0/4237627777 >> 172.21.15.83:0/4237627777 conn(0x7f081c0fcfe0 msgr2=0x7f081c0ff400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:34:45.602 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:45.598+0000 7f0820c00640 1 -- 172.21.15.83:0/4237627777 shutdown_connections 2024-08-19T01:34:45.602 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:45.598+0000 7f0820c00640 1 -- 172.21.15.83:0/4237627777 wait complete. 2024-08-19T01:34:45.604 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:45.599+0000 7f0820c00640 1 Processor -- start 2024-08-19T01:34:45.605 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:45.599+0000 7f0820c00640 1 -- start start 2024-08-19T01:34:45.605 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:45.600+0000 7f0820c00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f081c101860 0x7f081c199de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:34:45.605 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:45.600+0000 7f0820c00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f081c102a50 0x7f081c19a320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:34:45.605 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:45.600+0000 7f0820c00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f081c103410 0x7f081c1a1390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:34:45.605 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:45.600+0000 7f0820c00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f081c10d530 con 0x7f081c103410 2024-08-19T01:34:45.605 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:45.600+0000 7f0820c00640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f081c10d3b0 con 0x7f081c101860 2024-08-19T01:34:45.605 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:45.600+0000 7f0820c00640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f081c10d6b0 con 0x7f081c102a50 2024-08-19T01:34:45.605 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:45.600+0000 7f081a800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f081c103410 0x7f081c1a1390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:34:45.605 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:45.600+0000 7f081a800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f081c103410 0x7f081c1a1390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:57828/0 (socket says 172.21.15.83:57828) 2024-08-19T01:34:45.605 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:45.600+0000 7f081a800640 1 -- 172.21.15.83:0/3732036326 learned_addr learned my addr 172.21.15.83:0/3732036326 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-08-19T01:34:45.605 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:45.600+0000 7f0819400640 1 --2- 172.21.15.83:0/3732036326 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f081c102a50 0x7f081c19a320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:34:45.605 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:45.600+0000 7f0819e00640 1 --2- 172.21.15.83:0/3732036326 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f081c101860 0x7f081c199de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:34:45.605 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:45.600+0000 7f081a800640 1 -- 172.21.15.83:0/3732036326 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f081c102a50 msgr2=0x7f081c19a320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:34:45.606 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:45.600+0000 7f081a800640 1 --2- 172.21.15.83:0/3732036326 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f081c102a50 0x7f081c19a320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:45.606 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:45.600+0000 7f081a800640 1 -- 172.21.15.83:0/3732036326 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f081c101860 msgr2=0x7f081c199de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:34:45.606 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:45.600+0000 7f081a800640 1 --2- 172.21.15.83:0/3732036326 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f081c101860 0x7f081c199de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:45.606 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:45.600+0000 7f081a800640 1 -- 172.21.15.83:0/3732036326 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f081c1a1a00 con 0x7f081c103410 2024-08-19T01:34:45.606 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:45.600+0000 7f0819e00640 1 --2- 172.21.15.83:0/3732036326 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f081c101860 0x7f081c199de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-19T01:34:45.606 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:45.600+0000 7f0819400640 1 --2- 172.21.15.83:0/3732036326 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f081c102a50 0x7f081c19a320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-19T01:34:45.606 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:45.601+0000 7f081a800640 1 --2- 172.21.15.83:0/3732036326 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f081c103410 0x7f081c1a1390 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f081400eb10 tx=0x7f081400c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:34:45.606 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:45.601+0000 7f0812a00640 1 -- 172.21.15.83:0/3732036326 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0814004b20 con 0x7f081c103410 2024-08-19T01:34:45.606 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:45.601+0000 7f0812a00640 1 -- 172.21.15.83:0/3732036326 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0814004cc0 con 0x7f081c103410 2024-08-19T01:34:45.606 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:45.601+0000 7f0812a00640 1 -- 172.21.15.83:0/3732036326 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0814010760 con 0x7f081c103410 2024-08-19T01:34:45.606 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:45.601+0000 7f0820c00640 1 -- 172.21.15.83:0/3732036326 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f081c1a1cf0 con 0x7f081c103410 2024-08-19T01:34:45.606 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:45.601+0000 7f0820c00640 1 -- 172.21.15.83:0/3732036326 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f081c1a21c0 con 0x7f081c103410 2024-08-19T01:34:45.606 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:45.602+0000 7f0820c00640 1 -- 172.21.15.83:0/3732036326 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f081c1a23f0 con 0x7f081c103410 2024-08-19T01:34:45.607 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:45.604+0000 7f0812a00640 1 -- 172.21.15.83:0/3732036326 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 19) ==== 137884+0+0 (secure 0 0 0) 0x7f0814005020 con 0x7f081c103410 2024-08-19T01:34:45.607 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:45.605+0000 7f0812a00640 1 --2- 172.21.15.83:0/3732036326 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f08000a2ca0 0x7f08000a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:34:45.607 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:45.605+0000 7f0812a00640 1 -- 172.21.15.83:0/3732036326 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8269+0+0 (secure 0 0 0) 0x7f08140cf9e0 con 0x7f081c103410 2024-08-19T01:34:45.607 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:45.605+0000 7f0819e00640 1 --2- 172.21.15.83:0/3732036326 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f08000a2ca0 0x7f08000a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:34:45.607 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:45.606+0000 7f0819e00640 1 --2- 172.21.15.83:0/3732036326 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f08000a2ca0 0x7f08000a5160 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f081c1028b0 tx=0x7f080c005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:34:45.609 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:45.609+0000 7f0812a00640 1 -- 172.21.15.83:0/3732036326 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7f081409c760 con 0x7f081c103410 2024-08-19T01:34:45.654 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:34:45 smithi083 ceph-mon[17120]: pgmap v336: 97 pgs: 97 active+clean; 642 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 597 B/s wr, 6 op/s 2024-08-19T01:34:45.655 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:34:45 smithi083 ceph-mon[17120]: from='client.? 172.21.15.83:0/1390910346' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-19T01:34:45.782 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:45.781+0000 7f0820c00640 1 -- 172.21.15.83:0/3732036326 --> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f081c075870 con 0x7f08000a2ca0 2024-08-19T01:34:45.789 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:45.788+0000 7f0812a00640 1 -- 172.21.15.83:0/3732036326 <== mgr.14150 v2:172.21.15.83:6800/683740916 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258843 (secure 0 0 0) 0x7f081c075870 con 0x7f08000a2ca0 2024-08-19T01:34:45.792 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-08-19T01:34:45.794 INFO:teuthology.orchestra.run.smithi083.stderr:dumped all 2024-08-19T01:34:45.795 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:45.792+0000 7f0820c00640 1 -- 172.21.15.83:0/3732036326 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f08000a2ca0 msgr2=0x7f08000a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:34:45.795 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:45.792+0000 7f0820c00640 1 --2- 172.21.15.83:0/3732036326 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f08000a2ca0 0x7f08000a5160 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f081c1028b0 tx=0x7f080c005f90 comp rx=0 tx=0).stop 2024-08-19T01:34:45.795 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:45.792+0000 7f0820c00640 1 -- 172.21.15.83:0/3732036326 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f081c103410 msgr2=0x7f081c1a1390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:34:45.795 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:45.792+0000 7f0820c00640 1 --2- 172.21.15.83:0/3732036326 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f081c103410 0x7f081c1a1390 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f081400eb10 tx=0x7f081400c2d0 comp rx=0 tx=0).stop 2024-08-19T01:34:45.795 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:45.792+0000 7f0820c00640 1 -- 172.21.15.83:0/3732036326 shutdown_connections 2024-08-19T01:34:45.796 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:45.792+0000 7f0820c00640 1 --2- 172.21.15.83:0/3732036326 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f08000a2ca0 0x7f08000a5160 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:45.796 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:45.792+0000 7f0820c00640 1 --2- 172.21.15.83:0/3732036326 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f081c103410 0x7f081c1a1390 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:45.796 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:45.792+0000 7f0820c00640 1 --2- 172.21.15.83:0/3732036326 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f081c102a50 0x7f081c19a320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:45.796 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:45.792+0000 7f0820c00640 1 --2- 172.21.15.83:0/3732036326 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f081c101860 0x7f081c199de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:45.796 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:45.792+0000 7f0820c00640 1 -- 172.21.15.83:0/3732036326 >> 172.21.15.83:0/3732036326 conn(0x7f081c0fcfe0 msgr2=0x7f081c074150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:34:45.796 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:45.792+0000 7f0820c00640 1 -- 172.21.15.83:0/3732036326 shutdown_connections 2024-08-19T01:34:45.796 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:45.792+0000 7f0820c00640 1 -- 172.21.15.83:0/3732036326 wait complete. 2024-08-19T01:34:45.809 INFO:teuthology.orchestra.run.smithi083.stdout:{"pg_ready":true,"pg_map":{"version":336,"stamp":"2024-08-19T01:34:44.389285+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":657230,"num_objects":64,"num_object_clones":0,"num_object_copies":192,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":811,"num_read_kb":520,"num_write":366,"num_write_kb":3064,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":1181748,"num_keys_recovered":10,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":793,"ondisk_log_size":793,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":374752,"kb_used_data":13120,"kb_used_omap":18,"kb_used_meta":361517,"kb_avail":1124567072,"statfs":{"total":1151940427776,"available":1151556681728,"internally_reserved":0,"allocated":13434880,"data_stored":7380852,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19143,"internal_metadata":370193721},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":1091,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":73,"num_read_kb":33,"num_write":10,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002860"},"pg_stats":[{"pgid":"3.3f","version":"86'8","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:34:04.396225+0000","last_change":"2024-08-19T01:32:41.244214+0000","last_active":"2024-08-19T01:34:04.396225+0000","last_peered":"2024-08-19T01:34:04.396225+0000","last_clean":"2024-08-19T01:34:04.396225+0000","last_became_active":"2024-08-19T01:32:41.243889+0000","last_became_peered":"2024-08-19T01:32:41.243889+0000","last_unstale":"2024-08-19T01:34:04.396225+0000","last_undegraded":"2024-08-19T01:34:04.396225+0000","last_fullsized":"2024-08-19T01:34:04.396225+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_clean_scrub_stamp":"2024-08-19T01:32:40.202519+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T13:10:18.230055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"86'4","reported_seq":21,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:34:04.396539+0000","last_change":"2024-08-19T01:32:41.242729+0000","last_active":"2024-08-19T01:34:04.396539+0000","last_peered":"2024-08-19T01:34:04.396539+0000","last_clean":"2024-08-19T01:34:04.396539+0000","last_became_active":"2024-08-19T01:32:41.242436+0000","last_became_peered":"2024-08-19T01:32:41.242436+0000","last_unstale":"2024-08-19T01:34:04.396539+0000","last_undegraded":"2024-08-19T01:34:04.396539+0000","last_fullsized":"2024-08-19T01:34:04.396539+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_clean_scrub_stamp":"2024-08-19T01:32:40.202519+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T13:19:32.010576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"86'7","reported_seq":24,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:34:04.399256+0000","last_change":"2024-08-19T01:32:41.247377+0000","last_active":"2024-08-19T01:34:04.399256+0000","last_peered":"2024-08-19T01:34:04.399256+0000","last_clean":"2024-08-19T01:34:04.399256+0000","last_became_active":"2024-08-19T01:32:41.246804+0000","last_became_peered":"2024-08-19T01:32:41.246804+0000","last_unstale":"2024-08-19T01:34:04.399256+0000","last_undegraded":"2024-08-19T01:34:04.399256+0000","last_fullsized":"2024-08-19T01:34:04.399256+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_clean_scrub_stamp":"2024-08-19T01:32:40.202519+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T04:48:52.382687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"86'8","reported_seq":26,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:34:04.389960+0000","last_change":"2024-08-19T01:32:41.232900+0000","last_active":"2024-08-19T01:34:04.389960+0000","last_peered":"2024-08-19T01:34:04.389960+0000","last_clean":"2024-08-19T01:34:04.389960+0000","last_became_active":"2024-08-19T01:32:41.232573+0000","last_became_peered":"2024-08-19T01:32:41.232573+0000","last_unstale":"2024-08-19T01:34:04.389960+0000","last_undegraded":"2024-08-19T01:34:04.389960+0000","last_fullsized":"2024-08-19T01:34:04.389960+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_clean_scrub_stamp":"2024-08-19T01:32:40.202519+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T09:42:36.309834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"86'4","reported_seq":27,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:34:04.393723+0000","last_change":"2024-08-19T01:33:05.478964+0000","last_active":"2024-08-19T01:34:04.393723+0000","last_peered":"2024-08-19T01:34:04.393723+0000","last_clean":"2024-08-19T01:34:04.393723+0000","last_became_active":"2024-08-19T01:33:05.478430+0000","last_became_peered":"2024-08-19T01:33:05.478430+0000","last_unstale":"2024-08-19T01:34:04.393723+0000","last_undegraded":"2024-08-19T01:34:04.393723+0000","last_fullsized":"2024-08-19T01:34:04.393723+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_clean_scrub_stamp":"2024-08-19T01:32:40.202519+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T04:10:49.267550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,6],"acting":[8,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"86'6","reported_seq":23,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:34:04.395690+0000","last_change":"2024-08-19T01:32:41.228263+0000","last_active":"2024-08-19T01:34:04.395690+0000","last_peered":"2024-08-19T01:34:04.395690+0000","last_clean":"2024-08-19T01:34:04.395690+0000","last_became_active":"2024-08-19T01:32:41.227765+0000","last_became_peered":"2024-08-19T01:32:41.227765+0000","last_unstale":"2024-08-19T01:34:04.395690+0000","last_undegraded":"2024-08-19T01:34:04.395690+0000","last_fullsized":"2024-08-19T01:34:04.395690+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_clean_scrub_stamp":"2024-08-19T01:32:40.202519+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T11:21:18.817886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"86'5","reported_seq":22,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:34:04.391175+0000","last_change":"2024-08-19T01:32:41.235680+0000","last_active":"2024-08-19T01:34:04.391175+0000","last_peered":"2024-08-19T01:34:04.391175+0000","last_clean":"2024-08-19T01:34:04.391175+0000","last_became_active":"2024-08-19T01:32:41.234150+0000","last_became_peered":"2024-08-19T01:32:41.234150+0000","last_unstale":"2024-08-19T01:34:04.391175+0000","last_undegraded":"2024-08-19T01:34:04.391175+0000","last_fullsized":"2024-08-19T01:34:04.391175+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_clean_scrub_stamp":"2024-08-19T01:32:40.202519+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T10:28:08.908632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"86'9","reported_seq":26,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:34:04.394271+0000","last_change":"2024-08-19T01:32:41.230170+0000","last_active":"2024-08-19T01:34:04.394271+0000","last_peered":"2024-08-19T01:34:04.394271+0000","last_clean":"2024-08-19T01:34:04.394271+0000","last_became_active":"2024-08-19T01:32:41.229467+0000","last_became_peered":"2024-08-19T01:32:41.229467+0000","last_unstale":"2024-08-19T01:34:04.394271+0000","last_undegraded":"2024-08-19T01:34:04.394271+0000","last_fullsized":"2024-08-19T01:34:04.394271+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_clean_scrub_stamp":"2024-08-19T01:32:40.202519+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T10:24:15.832910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"86'11","reported_seq":28,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:34:04.398861+0000","last_change":"2024-08-19T01:32:41.230607+0000","last_active":"2024-08-19T01:34:04.398861+0000","last_peered":"2024-08-19T01:34:04.398861+0000","last_clean":"2024-08-19T01:34:04.398861+0000","last_became_active":"2024-08-19T01:32:41.229979+0000","last_became_peered":"2024-08-19T01:32:41.229979+0000","last_unstale":"2024-08-19T01:34:04.398861+0000","last_undegraded":"2024-08-19T01:34:04.398861+0000","last_fullsized":"2024-08-19T01:34:04.398861+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_clean_scrub_stamp":"2024-08-19T01:32:40.202519+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T02:23:46.736139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"86'16","reported_seq":33,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:34:04.395510+0000","last_change":"2024-08-19T01:32:41.233576+0000","last_active":"2024-08-19T01:34:04.395510+0000","last_peered":"2024-08-19T01:34:04.395510+0000","last_clean":"2024-08-19T01:34:04.395510+0000","last_became_active":"2024-08-19T01:32:41.231279+0000","last_became_peered":"2024-08-19T01:32:41.231279+0000","last_unstale":"2024-08-19T01:34:04.395510+0000","last_undegraded":"2024-08-19T01:34:04.395510+0000","last_fullsized":"2024-08-19T01:34:04.395510+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_clean_scrub_stamp":"2024-08-19T01:32:40.202519+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T07:32:45.341628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"86'6","reported_seq":23,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:34:04.396391+0000","last_change":"2024-08-19T01:32:41.233192+0000","last_active":"2024-08-19T01:34:04.396391+0000","last_peered":"2024-08-19T01:34:04.396391+0000","last_clean":"2024-08-19T01:34:04.396391+0000","last_became_active":"2024-08-19T01:32:41.230944+0000","last_became_peered":"2024-08-19T01:32:41.230944+0000","last_unstale":"2024-08-19T01:34:04.396391+0000","last_undegraded":"2024-08-19T01:34:04.396391+0000","last_fullsized":"2024-08-19T01:34:04.396391+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_clean_scrub_stamp":"2024-08-19T01:32:40.202519+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T08:55:29.036252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"86'7","reported_seq":24,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:34:04.386560+0000","last_change":"2024-08-19T01:32:41.234216+0000","last_active":"2024-08-19T01:34:04.386560+0000","last_peered":"2024-08-19T01:34:04.386560+0000","last_clean":"2024-08-19T01:34:04.386560+0000","last_became_active":"2024-08-19T01:32:41.233906+0000","last_became_peered":"2024-08-19T01:32:41.233906+0000","last_unstale":"2024-08-19T01:34:04.386560+0000","last_undegraded":"2024-08-19T01:34:04.386560+0000","last_fullsized":"2024-08-19T01:34:04.386560+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_clean_scrub_stamp":"2024-08-19T01:32:40.202519+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T08:35:01.764494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"86'10","reported_seq":27,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:34:04.398217+0000","last_change":"2024-08-19T01:32:41.230651+0000","last_active":"2024-08-19T01:34:04.398217+0000","last_peered":"2024-08-19T01:34:04.398217+0000","last_clean":"2024-08-19T01:34:04.398217+0000","last_became_active":"2024-08-19T01:32:41.230253+0000","last_became_peered":"2024-08-19T01:32:41.230253+0000","last_unstale":"2024-08-19T01:34:04.398217+0000","last_undegraded":"2024-08-19T01:34:04.398217+0000","last_fullsized":"2024-08-19T01:34:04.398217+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_clean_scrub_stamp":"2024-08-19T01:32:40.202519+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T09:41:59.942821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"86'7","reported_seq":24,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:34:04.396983+0000","last_change":"2024-08-19T01:32:41.240082+0000","last_active":"2024-08-19T01:34:04.396983+0000","last_peered":"2024-08-19T01:34:04.396983+0000","last_clean":"2024-08-19T01:34:04.396983+0000","last_became_active":"2024-08-19T01:32:41.239711+0000","last_became_peered":"2024-08-19T01:32:41.239711+0000","last_unstale":"2024-08-19T01:34:04.396983+0000","last_undegraded":"2024-08-19T01:34:04.396983+0000","last_fullsized":"2024-08-19T01:34:04.396983+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_clean_scrub_stamp":"2024-08-19T01:32:40.202519+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T12:48:59.122307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"86'7","reported_seq":30,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:34:04.390223+0000","last_change":"2024-08-19T01:33:05.474883+0000","last_active":"2024-08-19T01:34:04.390223+0000","last_peered":"2024-08-19T01:34:04.390223+0000","last_clean":"2024-08-19T01:34:04.390223+0000","last_became_active":"2024-08-19T01:33:05.474502+0000","last_became_peered":"2024-08-19T01:33:05.474502+0000","last_unstale":"2024-08-19T01:34:04.390223+0000","last_undegraded":"2024-08-19T01:34:04.390223+0000","last_fullsized":"2024-08-19T01:34:04.390223+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_clean_scrub_stamp":"2024-08-19T01:32:40.202519+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T07:02:01.171103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2],"acting":[0,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"86'9","reported_seq":26,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:34:04.399368+0000","last_change":"2024-08-19T01:32:41.240899+0000","last_active":"2024-08-19T01:34:04.399368+0000","last_peered":"2024-08-19T01:34:04.399368+0000","last_clean":"2024-08-19T01:34:04.399368+0000","last_became_active":"2024-08-19T01:32:41.240532+0000","last_became_peered":"2024-08-19T01:32:41.240532+0000","last_unstale":"2024-08-19T01:34:04.399368+0000","last_undegraded":"2024-08-19T01:34:04.399368+0000","last_fullsized":"2024-08-19T01:34:04.399368+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_clean_scrub_stamp":"2024-08-19T01:32:40.202519+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T08:17:47.035887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"86'8","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:34:04.396273+0000","last_change":"2024-08-19T01:32:41.237977+0000","last_active":"2024-08-19T01:34:04.396273+0000","last_peered":"2024-08-19T01:34:04.396273+0000","last_clean":"2024-08-19T01:34:04.396273+0000","last_became_active":"2024-08-19T01:32:41.237634+0000","last_became_peered":"2024-08-19T01:32:41.237634+0000","last_unstale":"2024-08-19T01:34:04.396273+0000","last_undegraded":"2024-08-19T01:34:04.396273+0000","last_fullsized":"2024-08-19T01:34:04.396273+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_clean_scrub_stamp":"2024-08-19T01:32:40.202519+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T12:48:37.005972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"86'10","reported_seq":27,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:34:04.391627+0000","last_change":"2024-08-19T01:32:41.236498+0000","last_active":"2024-08-19T01:34:04.391627+0000","last_peered":"2024-08-19T01:34:04.391627+0000","last_clean":"2024-08-19T01:34:04.391627+0000","last_became_active":"2024-08-19T01:32:41.236122+0000","last_became_peered":"2024-08-19T01:32:41.236122+0000","last_unstale":"2024-08-19T01:34:04.391627+0000","last_undegraded":"2024-08-19T01:34:04.391627+0000","last_fullsized":"2024-08-19T01:34:04.391627+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_clean_scrub_stamp":"2024-08-19T01:32:40.202519+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T02:31:23.648064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"86'7","reported_seq":24,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:34:04.390365+0000","last_change":"2024-08-19T01:32:41.234607+0000","last_active":"2024-08-19T01:34:04.390365+0000","last_peered":"2024-08-19T01:34:04.390365+0000","last_clean":"2024-08-19T01:34:04.390365+0000","last_became_active":"2024-08-19T01:32:41.232867+0000","last_became_peered":"2024-08-19T01:32:41.232867+0000","last_unstale":"2024-08-19T01:34:04.390365+0000","last_undegraded":"2024-08-19T01:34:04.390365+0000","last_fullsized":"2024-08-19T01:34:04.390365+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_clean_scrub_stamp":"2024-08-19T01:32:40.202519+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T12:32:38.441848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2c","version":"86'8","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:34:04.384758+0000","last_change":"2024-08-19T01:32:41.231705+0000","last_active":"2024-08-19T01:34:04.384758+0000","last_peered":"2024-08-19T01:34:04.384758+0000","last_clean":"2024-08-19T01:34:04.384758+0000","last_became_active":"2024-08-19T01:32:41.231262+0000","last_became_peered":"2024-08-19T01:32:41.231262+0000","last_unstale":"2024-08-19T01:34:04.384758+0000","last_undegraded":"2024-08-19T01:34:04.384758+0000","last_fullsized":"2024-08-19T01:34:04.384758+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_clean_scrub_stamp":"2024-08-19T01:32:40.202519+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T10:41:15.025471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2b","version":"86'11","reported_seq":28,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:34:04.397040+0000","last_change":"2024-08-19T01:32:41.243192+0000","last_active":"2024-08-19T01:34:04.397040+0000","last_peered":"2024-08-19T01:34:04.397040+0000","last_clean":"2024-08-19T01:34:04.397040+0000","last_became_active":"2024-08-19T01:32:41.242704+0000","last_became_peered":"2024-08-19T01:32:41.242704+0000","last_unstale":"2024-08-19T01:34:04.397040+0000","last_undegraded":"2024-08-19T01:34:04.397040+0000","last_fullsized":"2024-08-19T01:34:04.397040+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_clean_scrub_stamp":"2024-08-19T01:32:40.202519+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T01:44:43.270213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2a","version":"86'10","reported_seq":27,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:34:04.399726+0000","last_change":"2024-08-19T01:32:41.239908+0000","last_active":"2024-08-19T01:34:04.399726+0000","last_peered":"2024-08-19T01:34:04.399726+0000","last_clean":"2024-08-19T01:34:04.399726+0000","last_became_active":"2024-08-19T01:32:41.239438+0000","last_became_peered":"2024-08-19T01:32:41.239438+0000","last_unstale":"2024-08-19T01:34:04.399726+0000","last_undegraded":"2024-08-19T01:34:04.399726+0000","last_fullsized":"2024-08-19T01:34:04.399726+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_clean_scrub_stamp":"2024-08-19T01:32:40.202519+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T11:03:26.822111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.29","version":"86'8","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:34:04.389344+0000","last_change":"2024-08-19T01:32:41.226373+0000","last_active":"2024-08-19T01:34:04.389344+0000","last_peered":"2024-08-19T01:34:04.389344+0000","last_clean":"2024-08-19T01:34:04.389344+0000","last_became_active":"2024-08-19T01:32:41.226089+0000","last_became_peered":"2024-08-19T01:32:41.226089+0000","last_unstale":"2024-08-19T01:34:04.389344+0000","last_undegraded":"2024-08-19T01:34:04.389344+0000","last_fullsized":"2024-08-19T01:34:04.389344+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_clean_scrub_stamp":"2024-08-19T01:32:40.202519+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T01:33:56.639300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"86'6","reported_seq":23,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:34:04.389272+0000","last_change":"2024-08-19T01:32:41.237829+0000","last_active":"2024-08-19T01:34:04.389272+0000","last_peered":"2024-08-19T01:34:04.389272+0000","last_clean":"2024-08-19T01:34:04.389272+0000","last_became_active":"2024-08-19T01:32:41.237518+0000","last_became_peered":"2024-08-19T01:32:41.237518+0000","last_unstale":"2024-08-19T01:34:04.389272+0000","last_undegraded":"2024-08-19T01:34:04.389272+0000","last_fullsized":"2024-08-19T01:34:04.389272+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_clean_scrub_stamp":"2024-08-19T01:32:40.202519+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T02:25:11.307658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"86'2","reported_seq":19,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:34:04.386117+0000","last_change":"2024-08-19T01:32:41.234508+0000","last_active":"2024-08-19T01:34:04.386117+0000","last_peered":"2024-08-19T01:34:04.386117+0000","last_clean":"2024-08-19T01:34:04.386117+0000","last_became_active":"2024-08-19T01:32:41.232869+0000","last_became_peered":"2024-08-19T01:32:41.232869+0000","last_unstale":"2024-08-19T01:34:04.386117+0000","last_undegraded":"2024-08-19T01:34:04.386117+0000","last_fullsized":"2024-08-19T01:34:04.386117+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_clean_scrub_stamp":"2024-08-19T01:32:40.202519+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T11:20:55.097576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"86'15","reported_seq":32,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:34:04.390238+0000","last_change":"2024-08-19T01:32:41.230536+0000","last_active":"2024-08-19T01:34:04.390238+0000","last_peered":"2024-08-19T01:34:04.390238+0000","last_clean":"2024-08-19T01:34:04.390238+0000","last_became_active":"2024-08-19T01:32:41.230179+0000","last_became_peered":"2024-08-19T01:32:41.230179+0000","last_unstale":"2024-08-19T01:34:04.390238+0000","last_undegraded":"2024-08-19T01:34:04.390238+0000","last_fullsized":"2024-08-19T01:34:04.390238+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_clean_scrub_stamp":"2024-08-19T01:32:40.202519+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T02:25:09.441363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.25","version":"86'6","reported_seq":23,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:34:04.387595+0000","last_change":"2024-08-19T01:32:41.230695+0000","last_active":"2024-08-19T01:34:04.387595+0000","last_peered":"2024-08-19T01:34:04.387595+0000","last_clean":"2024-08-19T01:34:04.387595+0000","last_became_active":"2024-08-19T01:32:41.230324+0000","last_became_peered":"2024-08-19T01:32:41.230324+0000","last_unstale":"2024-08-19T01:34:04.387595+0000","last_undegraded":"2024-08-19T01:34:04.387595+0000","last_fullsized":"2024-08-19T01:34:04.387595+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_clean_scrub_stamp":"2024-08-19T01:32:40.202519+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T05:38:09.317605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.24","version":"86'4","reported_seq":21,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:34:04.391939+0000","last_change":"2024-08-19T01:32:41.238325+0000","last_active":"2024-08-19T01:34:04.391939+0000","last_peered":"2024-08-19T01:34:04.391939+0000","last_clean":"2024-08-19T01:34:04.391939+0000","last_became_active":"2024-08-19T01:32:41.237768+0000","last_became_peered":"2024-08-19T01:32:41.237768+0000","last_unstale":"2024-08-19T01:34:04.391939+0000","last_undegraded":"2024-08-19T01:34:04.391939+0000","last_fullsized":"2024-08-19T01:34:04.391939+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_clean_scrub_stamp":"2024-08-19T01:32:40.202519+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T08:45:06.866241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.23","version":"86'7","reported_seq":24,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:34:04.396309+0000","last_change":"2024-08-19T01:32:41.246950+0000","last_active":"2024-08-19T01:34:04.396309+0000","last_peered":"2024-08-19T01:34:04.396309+0000","last_clean":"2024-08-19T01:34:04.396309+0000","last_became_active":"2024-08-19T01:32:41.245441+0000","last_became_peered":"2024-08-19T01:32:41.245441+0000","last_unstale":"2024-08-19T01:34:04.396309+0000","last_undegraded":"2024-08-19T01:34:04.396309+0000","last_fullsized":"2024-08-19T01:34:04.396309+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_clean_scrub_stamp":"2024-08-19T01:32:40.202519+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T03:11:06.921258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"86'8","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:34:04.373200+0000","last_change":"2024-08-19T01:32:41.244810+0000","last_active":"2024-08-19T01:34:04.373200+0000","last_peered":"2024-08-19T01:34:04.373200+0000","last_clean":"2024-08-19T01:34:04.373200+0000","last_became_active":"2024-08-19T01:32:41.244401+0000","last_became_peered":"2024-08-19T01:32:41.244401+0000","last_unstale":"2024-08-19T01:34:04.373200+0000","last_undegraded":"2024-08-19T01:34:04.373200+0000","last_fullsized":"2024-08-19T01:34:04.373200+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_clean_scrub_stamp":"2024-08-19T01:32:40.202519+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T04:54:57.728181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.21","version":"86'12","reported_seq":29,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:34:04.387129+0000","last_change":"2024-08-19T01:32:41.235977+0000","last_active":"2024-08-19T01:34:04.387129+0000","last_peered":"2024-08-19T01:34:04.387129+0000","last_clean":"2024-08-19T01:34:04.387129+0000","last_became_active":"2024-08-19T01:32:41.235693+0000","last_became_peered":"2024-08-19T01:32:41.235693+0000","last_unstale":"2024-08-19T01:34:04.387129+0000","last_undegraded":"2024-08-19T01:34:04.387129+0000","last_fullsized":"2024-08-19T01:34:04.387129+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_clean_scrub_stamp":"2024-08-19T01:32:40.202519+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T04:59:15.766532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"86'9","reported_seq":32,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:34:04.386756+0000","last_change":"2024-08-19T01:33:05.473318+0000","last_active":"2024-08-19T01:34:04.386756+0000","last_peered":"2024-08-19T01:34:04.386756+0000","last_clean":"2024-08-19T01:34:04.386756+0000","last_became_active":"2024-08-19T01:33:05.472885+0000","last_became_peered":"2024-08-19T01:33:05.472885+0000","last_unstale":"2024-08-19T01:34:04.386756+0000","last_undegraded":"2024-08-19T01:34:04.386756+0000","last_fullsized":"2024-08-19T01:34:04.386756+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_clean_scrub_stamp":"2024-08-19T01:32:40.202519+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T07:49:35.525930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,9],"acting":[11,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1a","version":"86'7","reported_seq":24,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:34:04.376075+0000","last_change":"2024-08-19T01:32:41.230846+0000","last_active":"2024-08-19T01:34:04.376075+0000","last_peered":"2024-08-19T01:34:04.376075+0000","last_clean":"2024-08-19T01:34:04.376075+0000","last_became_active":"2024-08-19T01:32:41.230405+0000","last_became_peered":"2024-08-19T01:32:41.230405+0000","last_unstale":"2024-08-19T01:34:04.376075+0000","last_undegraded":"2024-08-19T01:34:04.376075+0000","last_fullsized":"2024-08-19T01:34:04.376075+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_clean_scrub_stamp":"2024-08-19T01:32:40.202519+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T08:59:46.241898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1b","version":"86'8","reported_seq":158,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:34:42.617354+0000","last_change":"2024-08-19T01:33:05.479889+0000","last_active":"2024-08-19T01:34:42.617354+0000","last_peered":"2024-08-19T01:34:42.617354+0000","last_clean":"2024-08-19T01:34:42.617354+0000","last_became_active":"2024-08-19T01:33:05.474698+0000","last_became_peered":"2024-08-19T01:33:05.474698+0000","last_unstale":"2024-08-19T01:34:42.617354+0000","last_undegraded":"2024-08-19T01:34:42.617354+0000","last_fullsized":"2024-08-19T01:34:42.617354+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:32:37.889224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:32:37.889224+0000","last_clean_scrub_stamp":"2024-08-19T01:32:37.889224+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T08:41:29.053471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1543,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":114,"num_read_kb":5,"num_write":5,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":1012,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7],"acting":[5,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"86'8","reported_seq":19,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:34:04.387536+0000","last_change":"2024-08-19T01:33:05.474083+0000","last_active":"2024-08-19T01:34:04.387536+0000","last_peered":"2024-08-19T01:34:04.387536+0000","last_clean":"2024-08-19T01:34:04.387536+0000","last_became_active":"2024-08-19T01:33:05.473718+0000","last_became_peered":"2024-08-19T01:33:05.473718+0000","last_unstale":"2024-08-19T01:34:04.387536+0000","last_undegraded":"2024-08-19T01:34:04.387536+0000","last_fullsized":"2024-08-19T01:34:04.387536+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_clean_scrub_stamp":"2024-08-19T01:32:40.202519+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T02:25:11.955170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,4,3],"acting":[9,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.1a","version":"84'3","reported_seq":35,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:33:05.705917+0000","last_change":"2024-08-19T01:33:05.705917+0000","last_active":"2024-08-19T01:33:05.705917+0000","last_peered":"2024-08-19T01:33:05.705917+0000","last_clean":"2024-08-19T01:33:05.705917+0000","last_became_active":"2024-08-19T01:33:05.702775+0000","last_became_peered":"2024-08-19T01:33:05.702775+0000","last_unstale":"2024-08-19T01:33:05.705917+0000","last_undegraded":"2024-08-19T01:33:05.705917+0000","last_fullsized":"2024-08-19T01:33:05.705917+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:32:37.889224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:32:37.889224+0000","last_clean_scrub_stamp":"2024-08-19T01:32:37.889224+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T03:58:29.989815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,6],"acting":[10,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.19","version":"84'2","reported_seq":24,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:33:05.477490+0000","last_change":"2024-08-19T01:32:38.911757+0000","last_active":"2024-08-19T01:33:05.477490+0000","last_peered":"2024-08-19T01:33:05.477490+0000","last_clean":"2024-08-19T01:33:05.477490+0000","last_became_active":"2024-08-19T01:32:38.910832+0000","last_became_peered":"2024-08-19T01:32:38.910832+0000","last_unstale":"2024-08-19T01:33:05.477490+0000","last_undegraded":"2024-08-19T01:33:05.477490+0000","last_fullsized":"2024-08-19T01:33:05.477490+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:32:37.889224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:32:37.889224+0000","last_clean_scrub_stamp":"2024-08-19T01:32:37.889224+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T04:46:54.986693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"86'8","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:34:04.392576+0000","last_change":"2024-08-19T01:32:41.231130+0000","last_active":"2024-08-19T01:34:04.392576+0000","last_peered":"2024-08-19T01:34:04.392576+0000","last_clean":"2024-08-19T01:34:04.392576+0000","last_became_active":"2024-08-19T01:32:41.230883+0000","last_became_peered":"2024-08-19T01:32:41.230883+0000","last_unstale":"2024-08-19T01:34:04.392576+0000","last_undegraded":"2024-08-19T01:34:04.392576+0000","last_fullsized":"2024-08-19T01:34:04.392576+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_clean_scrub_stamp":"2024-08-19T01:32:40.202519+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T06:33:11.856202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"84'1","reported_seq":23,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:33:05.470862+0000","last_change":"2024-08-19T01:32:38.916868+0000","last_active":"2024-08-19T01:33:05.470862+0000","last_peered":"2024-08-19T01:33:05.470862+0000","last_clean":"2024-08-19T01:33:05.470862+0000","last_became_active":"2024-08-19T01:32:38.916500+0000","last_became_peered":"2024-08-19T01:32:38.916500+0000","last_unstale":"2024-08-19T01:33:05.470862+0000","last_undegraded":"2024-08-19T01:33:05.470862+0000","last_fullsized":"2024-08-19T01:33:05.470862+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:32:37.889224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:32:37.889224+0000","last_clean_scrub_stamp":"2024-08-19T01:32:37.889224+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T12:30:07.085829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"86'11","reported_seq":28,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:34:04.391892+0000","last_change":"2024-08-19T01:32:41.235290+0000","last_active":"2024-08-19T01:34:04.391892+0000","last_peered":"2024-08-19T01:34:04.391892+0000","last_clean":"2024-08-19T01:34:04.391892+0000","last_became_active":"2024-08-19T01:32:41.235013+0000","last_became_peered":"2024-08-19T01:32:41.235013+0000","last_unstale":"2024-08-19T01:34:04.391892+0000","last_undegraded":"2024-08-19T01:34:04.391892+0000","last_fullsized":"2024-08-19T01:34:04.391892+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_clean_scrub_stamp":"2024-08-19T01:32:40.202519+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T09:50:14.837080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"86'4","reported_seq":133,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:34:37.609085+0000","last_change":"2024-08-19T01:32:38.912023+0000","last_active":"2024-08-19T01:34:37.609085+0000","last_peered":"2024-08-19T01:34:37.609085+0000","last_clean":"2024-08-19T01:34:37.609085+0000","last_became_active":"2024-08-19T01:32:38.911679+0000","last_became_peered":"2024-08-19T01:32:38.911679+0000","last_unstale":"2024-08-19T01:34:37.609085+0000","last_undegraded":"2024-08-19T01:34:37.609085+0000","last_fullsized":"2024-08-19T01:34:37.609085+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:32:37.889224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:32:37.889224+0000","last_clean_scrub_stamp":"2024-08-19T01:32:37.889224+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T12:08:51.118338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":107,"num_read_kb":107,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"86'10","reported_seq":27,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:34:04.399254+0000","last_change":"2024-08-19T01:32:41.232264+0000","last_active":"2024-08-19T01:34:04.399254+0000","last_peered":"2024-08-19T01:34:04.399254+0000","last_clean":"2024-08-19T01:34:04.399254+0000","last_became_active":"2024-08-19T01:32:41.231788+0000","last_became_peered":"2024-08-19T01:32:41.231788+0000","last_unstale":"2024-08-19T01:34:04.399254+0000","last_undegraded":"2024-08-19T01:34:04.399254+0000","last_fullsized":"2024-08-19T01:34:04.399254+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_clean_scrub_stamp":"2024-08-19T01:32:40.202519+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T04:59:06.443992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"84'2","reported_seq":24,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:33:05.472954+0000","last_change":"2024-08-19T01:32:38.915770+0000","last_active":"2024-08-19T01:33:05.472954+0000","last_peered":"2024-08-19T01:33:05.472954+0000","last_clean":"2024-08-19T01:33:05.472954+0000","last_became_active":"2024-08-19T01:32:38.915378+0000","last_became_peered":"2024-08-19T01:32:38.915378+0000","last_unstale":"2024-08-19T01:33:05.472954+0000","last_undegraded":"2024-08-19T01:33:05.472954+0000","last_fullsized":"2024-08-19T01:33:05.472954+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:32:37.889224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:32:37.889224+0000","last_clean_scrub_stamp":"2024-08-19T01:32:37.889224+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T09:16:14.513172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"86'9","reported_seq":26,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:34:04.395227+0000","last_change":"2024-08-19T01:32:41.243476+0000","last_active":"2024-08-19T01:34:04.395227+0000","last_peered":"2024-08-19T01:34:04.395227+0000","last_clean":"2024-08-19T01:34:04.395227+0000","last_became_active":"2024-08-19T01:32:41.243190+0000","last_became_peered":"2024-08-19T01:32:41.243190+0000","last_unstale":"2024-08-19T01:34:04.395227+0000","last_undegraded":"2024-08-19T01:34:04.395227+0000","last_fullsized":"2024-08-19T01:34:04.395227+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_clean_scrub_stamp":"2024-08-19T01:32:40.202519+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T12:07:34.096919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"84'4","reported_seq":27,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:33:05.470664+0000","last_change":"2024-08-19T01:32:39.051458+0000","last_active":"2024-08-19T01:33:05.470664+0000","last_peered":"2024-08-19T01:33:05.470664+0000","last_clean":"2024-08-19T01:33:05.470664+0000","last_became_active":"2024-08-19T01:32:39.051113+0000","last_became_peered":"2024-08-19T01:32:39.051113+0000","last_unstale":"2024-08-19T01:33:05.470664+0000","last_undegraded":"2024-08-19T01:33:05.470664+0000","last_fullsized":"2024-08-19T01:33:05.470664+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:32:37.889224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:32:37.889224+0000","last_clean_scrub_stamp":"2024-08-19T01:32:37.889224+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T02:15:51.371054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"86'11","reported_seq":28,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:34:04.398787+0000","last_change":"2024-08-19T01:32:41.236453+0000","last_active":"2024-08-19T01:34:04.398787+0000","last_peered":"2024-08-19T01:34:04.398787+0000","last_clean":"2024-08-19T01:34:04.398787+0000","last_became_active":"2024-08-19T01:32:41.236135+0000","last_became_peered":"2024-08-19T01:32:41.236135+0000","last_unstale":"2024-08-19T01:34:04.398787+0000","last_undegraded":"2024-08-19T01:34:04.398787+0000","last_fullsized":"2024-08-19T01:34:04.398787+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_clean_scrub_stamp":"2024-08-19T01:32:40.202519+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T12:38:41.398299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"86'31","reported_seq":277,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:34:38.610606+0000","last_change":"2024-08-19T01:32:38.914831+0000","last_active":"2024-08-19T01:34:38.610606+0000","last_peered":"2024-08-19T01:34:38.610606+0000","last_clean":"2024-08-19T01:34:38.610606+0000","last_became_active":"2024-08-19T01:32:38.914326+0000","last_became_peered":"2024-08-19T01:32:38.914326+0000","last_unstale":"2024-08-19T01:34:38.610606+0000","last_undegraded":"2024-08-19T01:34:38.610606+0000","last_fullsized":"2024-08-19T01:34:38.610606+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:32:37.889224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:32:37.889224+0000","last_clean_scrub_stamp":"2024-08-19T01:32:37.889224+0000","objects_scrubbed":0,"log_size":31,"log_dups_size":0,"ondisk_log_size":31,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T02:15:25.263974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":60865,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":224,"num_read_kb":170,"num_write":32,"num_write_kb":71,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"86'6","reported_seq":23,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:34:04.390164+0000","last_change":"2024-08-19T01:32:41.237945+0000","last_active":"2024-08-19T01:34:04.390164+0000","last_peered":"2024-08-19T01:34:04.390164+0000","last_clean":"2024-08-19T01:34:04.390164+0000","last_became_active":"2024-08-19T01:32:41.237362+0000","last_became_peered":"2024-08-19T01:32:41.237362+0000","last_unstale":"2024-08-19T01:34:04.390164+0000","last_undegraded":"2024-08-19T01:34:04.390164+0000","last_fullsized":"2024-08-19T01:34:04.390164+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_clean_scrub_stamp":"2024-08-19T01:32:40.202519+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T07:29:17.180054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"84'6","reported_seq":29,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:33:05.466940+0000","last_change":"2024-08-19T01:32:38.912453+0000","last_active":"2024-08-19T01:33:05.466940+0000","last_peered":"2024-08-19T01:33:05.466940+0000","last_clean":"2024-08-19T01:33:05.466940+0000","last_became_active":"2024-08-19T01:32:38.912051+0000","last_became_peered":"2024-08-19T01:32:38.912051+0000","last_unstale":"2024-08-19T01:33:05.466940+0000","last_undegraded":"2024-08-19T01:33:05.466940+0000","last_fullsized":"2024-08-19T01:33:05.466940+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:32:37.889224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:32:37.889224+0000","last_clean_scrub_stamp":"2024-08-19T01:32:37.889224+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T06:49:58.810134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"86'9","reported_seq":26,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:34:04.391203+0000","last_change":"2024-08-19T01:32:41.236282+0000","last_active":"2024-08-19T01:34:04.391203+0000","last_peered":"2024-08-19T01:34:04.391203+0000","last_clean":"2024-08-19T01:34:04.391203+0000","last_became_active":"2024-08-19T01:32:41.235559+0000","last_became_peered":"2024-08-19T01:32:41.235559+0000","last_unstale":"2024-08-19T01:34:04.391203+0000","last_undegraded":"2024-08-19T01:34:04.391203+0000","last_fullsized":"2024-08-19T01:34:04.391203+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_clean_scrub_stamp":"2024-08-19T01:32:40.202519+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T07:51:52.157094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"84'3","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:33:05.470566+0000","last_change":"2024-08-19T01:32:38.915688+0000","last_active":"2024-08-19T01:33:05.470566+0000","last_peered":"2024-08-19T01:33:05.470566+0000","last_clean":"2024-08-19T01:33:05.470566+0000","last_became_active":"2024-08-19T01:32:38.915297+0000","last_became_peered":"2024-08-19T01:32:38.915297+0000","last_unstale":"2024-08-19T01:33:05.470566+0000","last_undegraded":"2024-08-19T01:33:05.470566+0000","last_fullsized":"2024-08-19T01:33:05.470566+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:32:37.889224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:32:37.889224+0000","last_clean_scrub_stamp":"2024-08-19T01:32:37.889224+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T01:38:12.780588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"86'15","reported_seq":32,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:34:04.395033+0000","last_change":"2024-08-19T01:32:41.233766+0000","last_active":"2024-08-19T01:34:04.395033+0000","last_peered":"2024-08-19T01:34:04.395033+0000","last_clean":"2024-08-19T01:34:04.395033+0000","last_became_active":"2024-08-19T01:32:41.233425+0000","last_became_peered":"2024-08-19T01:32:41.233425+0000","last_unstale":"2024-08-19T01:34:04.395033+0000","last_undegraded":"2024-08-19T01:34:04.395033+0000","last_fullsized":"2024-08-19T01:34:04.395033+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_clean_scrub_stamp":"2024-08-19T01:32:40.202519+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T10:29:15.471639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"86'7","reported_seq":146,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:34:42.493710+0000","last_change":"2024-08-19T01:32:38.912610+0000","last_active":"2024-08-19T01:34:42.493710+0000","last_peered":"2024-08-19T01:34:42.493710+0000","last_clean":"2024-08-19T01:34:42.493710+0000","last_became_active":"2024-08-19T01:32:38.912040+0000","last_became_peered":"2024-08-19T01:32:38.912040+0000","last_unstale":"2024-08-19T01:34:42.493710+0000","last_undegraded":"2024-08-19T01:34:42.493710+0000","last_fullsized":"2024-08-19T01:34:42.493710+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:32:37.889224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:32:37.889224+0000","last_clean_scrub_stamp":"2024-08-19T01:32:37.889224+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T09:26:05.966391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1820,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":118,"num_read_kb":7,"num_write":6,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"86'6","reported_seq":23,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:34:04.393529+0000","last_change":"2024-08-19T01:32:41.235945+0000","last_active":"2024-08-19T01:34:04.393529+0000","last_peered":"2024-08-19T01:34:04.393529+0000","last_clean":"2024-08-19T01:34:04.393529+0000","last_became_active":"2024-08-19T01:32:41.235633+0000","last_became_peered":"2024-08-19T01:32:41.235633+0000","last_unstale":"2024-08-19T01:34:04.393529+0000","last_undegraded":"2024-08-19T01:34:04.393529+0000","last_fullsized":"2024-08-19T01:34:04.393529+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_clean_scrub_stamp":"2024-08-19T01:32:40.202519+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T08:37:37.682175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"84'15","reported_seq":38,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:33:05.471087+0000","last_change":"2024-08-19T01:32:39.052850+0000","last_active":"2024-08-19T01:33:05.471087+0000","last_peered":"2024-08-19T01:33:05.471087+0000","last_clean":"2024-08-19T01:33:05.471087+0000","last_became_active":"2024-08-19T01:32:39.052455+0000","last_became_peered":"2024-08-19T01:32:39.052455+0000","last_unstale":"2024-08-19T01:33:05.471087+0000","last_undegraded":"2024-08-19T01:33:05.471087+0000","last_fullsized":"2024-08-19T01:33:05.471087+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:32:37.889224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:32:37.889224+0000","last_clean_scrub_stamp":"2024-08-19T01:32:37.889224+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T03:25:08.279911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"86'6","reported_seq":29,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:34:04.394643+0000","last_change":"2024-08-19T01:33:05.702575+0000","last_active":"2024-08-19T01:34:04.394643+0000","last_peered":"2024-08-19T01:34:04.394643+0000","last_clean":"2024-08-19T01:34:04.394643+0000","last_became_active":"2024-08-19T01:33:05.702110+0000","last_became_peered":"2024-08-19T01:33:05.702110+0000","last_unstale":"2024-08-19T01:34:04.394643+0000","last_undegraded":"2024-08-19T01:34:04.394643+0000","last_fullsized":"2024-08-19T01:34:04.394643+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_clean_scrub_stamp":"2024-08-19T01:32:40.202519+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T12:18:51.102180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,1],"acting":[5,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"86'3","reported_seq":26,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:33:05.472787+0000","last_change":"2024-08-19T01:32:38.912090+0000","last_active":"2024-08-19T01:33:05.472787+0000","last_peered":"2024-08-19T01:33:05.472787+0000","last_clean":"2024-08-19T01:33:05.472787+0000","last_became_active":"2024-08-19T01:32:38.911753+0000","last_became_peered":"2024-08-19T01:32:38.911753+0000","last_unstale":"2024-08-19T01:33:05.472787+0000","last_undegraded":"2024-08-19T01:33:05.472787+0000","last_fullsized":"2024-08-19T01:33:05.472787+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:32:37.889224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:32:37.889224+0000","last_clean_scrub_stamp":"2024-08-19T01:32:37.889224+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T12:56:42.982062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"86'5","reported_seq":22,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:34:04.387010+0000","last_change":"2024-08-19T01:32:41.230429+0000","last_active":"2024-08-19T01:34:04.387010+0000","last_peered":"2024-08-19T01:34:04.387010+0000","last_clean":"2024-08-19T01:34:04.387010+0000","last_became_active":"2024-08-19T01:32:41.230115+0000","last_became_peered":"2024-08-19T01:32:41.230115+0000","last_unstale":"2024-08-19T01:34:04.387010+0000","last_undegraded":"2024-08-19T01:34:04.387010+0000","last_fullsized":"2024-08-19T01:34:04.387010+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_clean_scrub_stamp":"2024-08-19T01:32:40.202519+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T01:50:51.084130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.e","version":"84'2","reported_seq":24,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:33:05.477826+0000","last_change":"2024-08-19T01:32:38.914557+0000","last_active":"2024-08-19T01:33:05.477826+0000","last_peered":"2024-08-19T01:33:05.477826+0000","last_clean":"2024-08-19T01:33:05.477826+0000","last_became_active":"2024-08-19T01:32:38.914115+0000","last_became_peered":"2024-08-19T01:32:38.914115+0000","last_unstale":"2024-08-19T01:33:05.477826+0000","last_undegraded":"2024-08-19T01:33:05.477826+0000","last_fullsized":"2024-08-19T01:33:05.477826+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:32:37.889224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:32:37.889224+0000","last_clean_scrub_stamp":"2024-08-19T01:32:37.889224+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T06:30:00.523513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"86'9","reported_seq":26,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:34:04.391088+0000","last_change":"2024-08-19T01:32:41.234883+0000","last_active":"2024-08-19T01:34:04.391088+0000","last_peered":"2024-08-19T01:34:04.391088+0000","last_clean":"2024-08-19T01:34:04.391088+0000","last_became_active":"2024-08-19T01:32:41.233901+0000","last_became_peered":"2024-08-19T01:32:41.233901+0000","last_unstale":"2024-08-19T01:34:04.391088+0000","last_undegraded":"2024-08-19T01:34:04.391088+0000","last_fullsized":"2024-08-19T01:34:04.391088+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_clean_scrub_stamp":"2024-08-19T01:32:40.202519+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T12:08:46.305259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"84'1","reported_seq":24,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:33:05.470777+0000","last_change":"2024-08-19T01:32:38.913441+0000","last_active":"2024-08-19T01:33:05.470777+0000","last_peered":"2024-08-19T01:33:05.470777+0000","last_clean":"2024-08-19T01:33:05.470777+0000","last_became_active":"2024-08-19T01:32:38.913115+0000","last_became_peered":"2024-08-19T01:32:38.913115+0000","last_unstale":"2024-08-19T01:33:05.470777+0000","last_undegraded":"2024-08-19T01:33:05.470777+0000","last_fullsized":"2024-08-19T01:33:05.470777+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:32:37.889224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:32:37.889224+0000","last_clean_scrub_stamp":"2024-08-19T01:32:37.889224+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T11:43:15.290980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"86'7","reported_seq":24,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:34:04.391219+0000","last_change":"2024-08-19T01:32:41.235370+0000","last_active":"2024-08-19T01:34:04.391219+0000","last_peered":"2024-08-19T01:34:04.391219+0000","last_clean":"2024-08-19T01:34:04.391219+0000","last_became_active":"2024-08-19T01:32:41.235012+0000","last_became_peered":"2024-08-19T01:32:41.235012+0000","last_unstale":"2024-08-19T01:34:04.391219+0000","last_undegraded":"2024-08-19T01:34:04.391219+0000","last_fullsized":"2024-08-19T01:34:04.391219+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_clean_scrub_stamp":"2024-08-19T01:32:40.202519+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T02:18:31.134974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"84'3","reported_seq":26,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:33:05.466872+0000","last_change":"2024-08-19T01:32:39.245560+0000","last_active":"2024-08-19T01:33:05.466872+0000","last_peered":"2024-08-19T01:33:05.466872+0000","last_clean":"2024-08-19T01:33:05.466872+0000","last_became_active":"2024-08-19T01:32:39.245118+0000","last_became_peered":"2024-08-19T01:32:39.245118+0000","last_unstale":"2024-08-19T01:33:05.466872+0000","last_undegraded":"2024-08-19T01:33:05.466872+0000","last_fullsized":"2024-08-19T01:33:05.466872+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:32:37.889224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:32:37.889224+0000","last_clean_scrub_stamp":"2024-08-19T01:32:37.889224+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T02:18:24.605709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"86'7","reported_seq":24,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:34:04.396513+0000","last_change":"2024-08-19T01:32:41.234452+0000","last_active":"2024-08-19T01:34:04.396513+0000","last_peered":"2024-08-19T01:34:04.396513+0000","last_clean":"2024-08-19T01:34:04.396513+0000","last_became_active":"2024-08-19T01:32:41.233852+0000","last_became_peered":"2024-08-19T01:32:41.233852+0000","last_unstale":"2024-08-19T01:34:04.396513+0000","last_undegraded":"2024-08-19T01:34:04.396513+0000","last_fullsized":"2024-08-19T01:34:04.396513+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_clean_scrub_stamp":"2024-08-19T01:32:40.202519+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T06:18:34.457239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"22'137","reported_seq":339,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:33:05.473281+0000","last_change":"2024-08-19T01:31:27.659394+0000","last_active":"2024-08-19T01:33:05.473281+0000","last_peered":"2024-08-19T01:33:05.473281+0000","last_clean":"2024-08-19T01:33:05.473281+0000","last_became_active":"2024-08-19T01:31:27.659007+0000","last_became_peered":"2024-08-19T01:31:27.659007+0000","last_unstale":"2024-08-19T01:33:05.473281+0000","last_undegraded":"2024-08-19T01:33:05.473281+0000","last_fullsized":"2024-08-19T01:33:05.473281+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:28:16.912251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:28:16.912251+0000","last_clean_scrub_stamp":"2024-08-19T01:28:16.912251+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T05:53:01.660619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"84'3","reported_seq":27,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:33:05.470433+0000","last_change":"2024-08-19T01:32:38.913408+0000","last_active":"2024-08-19T01:33:05.470433+0000","last_peered":"2024-08-19T01:33:05.470433+0000","last_clean":"2024-08-19T01:33:05.470433+0000","last_became_active":"2024-08-19T01:32:38.912242+0000","last_became_peered":"2024-08-19T01:32:38.912242+0000","last_unstale":"2024-08-19T01:33:05.470433+0000","last_undegraded":"2024-08-19T01:33:05.470433+0000","last_fullsized":"2024-08-19T01:33:05.470433+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:32:37.889224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:32:37.889224+0000","last_clean_scrub_stamp":"2024-08-19T01:32:37.889224+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T07:26:48.121366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2","version":"86'8","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:34:04.390396+0000","last_change":"2024-08-19T01:32:41.232611+0000","last_active":"2024-08-19T01:34:04.390396+0000","last_peered":"2024-08-19T01:34:04.390396+0000","last_clean":"2024-08-19T01:34:04.390396+0000","last_became_active":"2024-08-19T01:32:41.232261+0000","last_became_peered":"2024-08-19T01:32:41.232261+0000","last_unstale":"2024-08-19T01:34:04.390396+0000","last_undegraded":"2024-08-19T01:34:04.390396+0000","last_fullsized":"2024-08-19T01:34:04.390396+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_clean_scrub_stamp":"2024-08-19T01:32:40.202519+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T08:04:16.793933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"84'3","reported_seq":26,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:33:05.473161+0000","last_change":"2024-08-19T01:32:38.912765+0000","last_active":"2024-08-19T01:33:05.473161+0000","last_peered":"2024-08-19T01:33:05.473161+0000","last_clean":"2024-08-19T01:33:05.473161+0000","last_became_active":"2024-08-19T01:32:38.912274+0000","last_became_peered":"2024-08-19T01:32:38.912274+0000","last_unstale":"2024-08-19T01:33:05.473161+0000","last_undegraded":"2024-08-19T01:33:05.473161+0000","last_fullsized":"2024-08-19T01:33:05.473161+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:32:37.889224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:32:37.889224+0000","last_clean_scrub_stamp":"2024-08-19T01:32:37.889224+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T09:57:49.714827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"86'5","reported_seq":22,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:34:04.390678+0000","last_change":"2024-08-19T01:32:41.224449+0000","last_active":"2024-08-19T01:34:04.390678+0000","last_peered":"2024-08-19T01:34:04.390678+0000","last_clean":"2024-08-19T01:34:04.390678+0000","last_became_active":"2024-08-19T01:32:41.224064+0000","last_became_peered":"2024-08-19T01:32:41.224064+0000","last_unstale":"2024-08-19T01:34:04.390678+0000","last_undegraded":"2024-08-19T01:34:04.390678+0000","last_fullsized":"2024-08-19T01:34:04.390678+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_clean_scrub_stamp":"2024-08-19T01:32:40.202519+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T08:53:33.948504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"86'6","reported_seq":137,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:34:39.566576+0000","last_change":"2024-08-19T01:32:39.050869+0000","last_active":"2024-08-19T01:34:39.566576+0000","last_peered":"2024-08-19T01:34:39.566576+0000","last_clean":"2024-08-19T01:34:39.566576+0000","last_became_active":"2024-08-19T01:32:39.050529+0000","last_became_peered":"2024-08-19T01:32:39.050529+0000","last_unstale":"2024-08-19T01:34:39.566576+0000","last_undegraded":"2024-08-19T01:34:39.566576+0000","last_fullsized":"2024-08-19T01:34:39.566576+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:32:37.889224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:32:37.889224+0000","last_clean_scrub_stamp":"2024-08-19T01:32:37.889224+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T12:52:52.788360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":109,"num_read_kb":109,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"86'8","reported_seq":26,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:34:04.387624+0000","last_change":"2024-08-19T01:32:41.237093+0000","last_active":"2024-08-19T01:34:04.387624+0000","last_peered":"2024-08-19T01:34:04.387624+0000","last_clean":"2024-08-19T01:34:04.387624+0000","last_became_active":"2024-08-19T01:32:41.236738+0000","last_became_peered":"2024-08-19T01:32:41.236738+0000","last_unstale":"2024-08-19T01:34:04.387624+0000","last_undegraded":"2024-08-19T01:34:04.387624+0000","last_fullsized":"2024-08-19T01:34:04.387624+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_clean_scrub_stamp":"2024-08-19T01:32:40.202519+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T08:37:33.117344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"84'1","reported_seq":23,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:33:05.473189+0000","last_change":"2024-08-19T01:32:38.915010+0000","last_active":"2024-08-19T01:33:05.473189+0000","last_peered":"2024-08-19T01:33:05.473189+0000","last_clean":"2024-08-19T01:33:05.473189+0000","last_became_active":"2024-08-19T01:32:38.914687+0000","last_became_peered":"2024-08-19T01:32:38.914687+0000","last_unstale":"2024-08-19T01:33:05.473189+0000","last_undegraded":"2024-08-19T01:33:05.473189+0000","last_fullsized":"2024-08-19T01:33:05.473189+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:32:37.889224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:32:37.889224+0000","last_clean_scrub_stamp":"2024-08-19T01:32:37.889224+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T04:46:07.881778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"86'7","reported_seq":24,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:34:04.397148+0000","last_change":"2024-08-19T01:32:41.244069+0000","last_active":"2024-08-19T01:34:04.397148+0000","last_peered":"2024-08-19T01:34:04.397148+0000","last_clean":"2024-08-19T01:34:04.397148+0000","last_became_active":"2024-08-19T01:32:41.243482+0000","last_became_peered":"2024-08-19T01:32:41.243482+0000","last_unstale":"2024-08-19T01:34:04.397148+0000","last_undegraded":"2024-08-19T01:34:04.397148+0000","last_fullsized":"2024-08-19T01:34:04.397148+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_clean_scrub_stamp":"2024-08-19T01:32:40.202519+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T02:08:28.044421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":22,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:33:05.466068+0000","last_change":"2024-08-19T01:32:39.051383+0000","last_active":"2024-08-19T01:33:05.466068+0000","last_peered":"2024-08-19T01:33:05.466068+0000","last_clean":"2024-08-19T01:33:05.466068+0000","last_became_active":"2024-08-19T01:32:39.050970+0000","last_became_peered":"2024-08-19T01:32:39.050970+0000","last_unstale":"2024-08-19T01:33:05.466068+0000","last_undegraded":"2024-08-19T01:33:05.466068+0000","last_fullsized":"2024-08-19T01:33:05.466068+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:32:37.889224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:32:37.889224+0000","last_clean_scrub_stamp":"2024-08-19T01:32:37.889224+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T11:31:30.461651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"86'5","reported_seq":22,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:34:04.395486+0000","last_change":"2024-08-19T01:32:41.232656+0000","last_active":"2024-08-19T01:34:04.395486+0000","last_peered":"2024-08-19T01:34:04.395486+0000","last_clean":"2024-08-19T01:34:04.395486+0000","last_became_active":"2024-08-19T01:32:41.232274+0000","last_became_peered":"2024-08-19T01:32:41.232274+0000","last_unstale":"2024-08-19T01:34:04.395486+0000","last_undegraded":"2024-08-19T01:34:04.395486+0000","last_fullsized":"2024-08-19T01:34:04.395486+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_clean_scrub_stamp":"2024-08-19T01:32:40.202519+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T03:38:40.695652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.5","version":"84'1","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:33:05.472646+0000","last_change":"2024-08-19T01:32:39.050809+0000","last_active":"2024-08-19T01:33:05.472646+0000","last_peered":"2024-08-19T01:33:05.472646+0000","last_clean":"2024-08-19T01:33:05.472646+0000","last_became_active":"2024-08-19T01:32:39.050434+0000","last_became_peered":"2024-08-19T01:32:39.050434+0000","last_unstale":"2024-08-19T01:33:05.472646+0000","last_undegraded":"2024-08-19T01:33:05.472646+0000","last_fullsized":"2024-08-19T01:33:05.472646+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:32:37.889224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:32:37.889224+0000","last_clean_scrub_stamp":"2024-08-19T01:32:37.889224+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T13:04:28.828117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"86'9","reported_seq":26,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:34:04.390057+0000","last_change":"2024-08-19T01:32:41.228485+0000","last_active":"2024-08-19T01:34:04.390057+0000","last_peered":"2024-08-19T01:34:04.390057+0000","last_clean":"2024-08-19T01:34:04.390057+0000","last_became_active":"2024-08-19T01:32:41.227238+0000","last_became_peered":"2024-08-19T01:32:41.227238+0000","last_unstale":"2024-08-19T01:34:04.390057+0000","last_undegraded":"2024-08-19T01:34:04.390057+0000","last_fullsized":"2024-08-19T01:34:04.390057+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_clean_scrub_stamp":"2024-08-19T01:32:40.202519+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T10:55:55.939382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"86'13","reported_seq":35,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:34:05.234056+0000","last_change":"2024-08-19T01:32:38.915994+0000","last_active":"2024-08-19T01:34:05.234056+0000","last_peered":"2024-08-19T01:34:05.234056+0000","last_clean":"2024-08-19T01:34:05.234056+0000","last_became_active":"2024-08-19T01:32:38.914714+0000","last_became_peered":"2024-08-19T01:32:38.914714+0000","last_unstale":"2024-08-19T01:34:05.234056+0000","last_undegraded":"2024-08-19T01:34:05.234056+0000","last_fullsized":"2024-08-19T01:34:05.234056+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:32:37.889224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:32:37.889224+0000","last_clean_scrub_stamp":"2024-08-19T01:32:37.889224+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T05:38:07.274274+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":25,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"86'5","reported_seq":22,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:34:04.386255+0000","last_change":"2024-08-19T01:32:41.236611+0000","last_active":"2024-08-19T01:34:04.386255+0000","last_peered":"2024-08-19T01:34:04.386255+0000","last_clean":"2024-08-19T01:34:04.386255+0000","last_became_active":"2024-08-19T01:32:41.233821+0000","last_became_peered":"2024-08-19T01:32:41.233821+0000","last_unstale":"2024-08-19T01:34:04.386255+0000","last_undegraded":"2024-08-19T01:34:04.386255+0000","last_fullsized":"2024-08-19T01:34:04.386255+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_clean_scrub_stamp":"2024-08-19T01:32:40.202519+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T11:34:07.879034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"84'4","reported_seq":26,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:33:05.470567+0000","last_change":"2024-08-19T01:32:39.245411+0000","last_active":"2024-08-19T01:33:05.470567+0000","last_peered":"2024-08-19T01:33:05.470567+0000","last_clean":"2024-08-19T01:33:05.470567+0000","last_became_active":"2024-08-19T01:32:39.244980+0000","last_became_peered":"2024-08-19T01:32:39.244980+0000","last_unstale":"2024-08-19T01:33:05.470567+0000","last_undegraded":"2024-08-19T01:33:05.470567+0000","last_fullsized":"2024-08-19T01:33:05.470567+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:32:37.889224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:32:37.889224+0000","last_clean_scrub_stamp":"2024-08-19T01:32:37.889224+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T02:06:57.878051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"86'7","reported_seq":30,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:34:04.395910+0000","last_change":"2024-08-19T01:33:05.478955+0000","last_active":"2024-08-19T01:34:04.395910+0000","last_peered":"2024-08-19T01:34:04.395910+0000","last_clean":"2024-08-19T01:34:04.395910+0000","last_became_active":"2024-08-19T01:33:05.478446+0000","last_became_peered":"2024-08-19T01:33:05.478446+0000","last_unstale":"2024-08-19T01:34:04.395910+0000","last_undegraded":"2024-08-19T01:34:04.395910+0000","last_fullsized":"2024-08-19T01:34:04.395910+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_clean_scrub_stamp":"2024-08-19T01:32:40.202519+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T11:58:24.285438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,6],"acting":[0,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"84'2","reported_seq":24,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:33:05.701553+0000","last_change":"2024-08-19T01:32:39.050100+0000","last_active":"2024-08-19T01:33:05.701553+0000","last_peered":"2024-08-19T01:33:05.701553+0000","last_clean":"2024-08-19T01:33:05.701553+0000","last_became_active":"2024-08-19T01:32:39.049807+0000","last_became_peered":"2024-08-19T01:32:39.049807+0000","last_unstale":"2024-08-19T01:33:05.701553+0000","last_undegraded":"2024-08-19T01:33:05.701553+0000","last_fullsized":"2024-08-19T01:33:05.701553+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:32:37.889224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:32:37.889224+0000","last_clean_scrub_stamp":"2024-08-19T01:32:37.889224+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T07:06:22.361609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"86'2","reported_seq":19,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:34:04.355286+0000","last_change":"2024-08-19T01:32:41.228982+0000","last_active":"2024-08-19T01:34:04.355286+0000","last_peered":"2024-08-19T01:34:04.355286+0000","last_clean":"2024-08-19T01:34:04.355286+0000","last_became_active":"2024-08-19T01:32:41.228562+0000","last_became_peered":"2024-08-19T01:32:41.228562+0000","last_unstale":"2024-08-19T01:34:04.355286+0000","last_undegraded":"2024-08-19T01:34:04.355286+0000","last_fullsized":"2024-08-19T01:34:04.355286+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_clean_scrub_stamp":"2024-08-19T01:32:40.202519+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T03:14:42.480359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"84'3","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:33:05.471722+0000","last_change":"2024-08-19T01:32:38.913677+0000","last_active":"2024-08-19T01:33:05.471722+0000","last_peered":"2024-08-19T01:33:05.471722+0000","last_clean":"2024-08-19T01:33:05.471722+0000","last_became_active":"2024-08-19T01:32:38.913091+0000","last_became_peered":"2024-08-19T01:32:38.913091+0000","last_unstale":"2024-08-19T01:33:05.471722+0000","last_undegraded":"2024-08-19T01:33:05.471722+0000","last_fullsized":"2024-08-19T01:33:05.471722+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:32:37.889224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:32:37.889224+0000","last_clean_scrub_stamp":"2024-08-19T01:32:37.889224+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T12:39:16.489246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"86'9","reported_seq":26,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:34:04.395672+0000","last_change":"2024-08-19T01:32:41.239380+0000","last_active":"2024-08-19T01:34:04.395672+0000","last_peered":"2024-08-19T01:34:04.395672+0000","last_clean":"2024-08-19T01:34:04.395672+0000","last_became_active":"2024-08-19T01:32:41.239060+0000","last_became_peered":"2024-08-19T01:32:41.239060+0000","last_unstale":"2024-08-19T01:34:04.395672+0000","last_undegraded":"2024-08-19T01:34:04.395672+0000","last_fullsized":"2024-08-19T01:34:04.395672+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_clean_scrub_stamp":"2024-08-19T01:32:40.202519+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T02:21:59.630705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"86'2","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:33:07.449044+0000","last_change":"2024-08-19T01:33:05.478578+0000","last_active":"2024-08-19T01:33:07.449044+0000","last_peered":"2024-08-19T01:33:07.449044+0000","last_clean":"2024-08-19T01:33:07.449044+0000","last_became_active":"2024-08-19T01:33:05.473433+0000","last_became_peered":"2024-08-19T01:33:05.473433+0000","last_unstale":"2024-08-19T01:33:07.449044+0000","last_undegraded":"2024-08-19T01:33:07.449044+0000","last_fullsized":"2024-08-19T01:33:07.449044+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:32:37.889224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:32:37.889224+0000","last_clean_scrub_stamp":"2024-08-19T01:32:37.889224+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T05:06:29.975613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,3,8],"acting":[11,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.b","version":"86'8","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:34:04.393696+0000","last_change":"2024-08-19T01:32:41.236012+0000","last_active":"2024-08-19T01:34:04.393696+0000","last_peered":"2024-08-19T01:34:04.393696+0000","last_clean":"2024-08-19T01:34:04.393696+0000","last_became_active":"2024-08-19T01:32:41.235724+0000","last_became_peered":"2024-08-19T01:32:41.235724+0000","last_unstale":"2024-08-19T01:34:04.393696+0000","last_undegraded":"2024-08-19T01:34:04.393696+0000","last_fullsized":"2024-08-19T01:34:04.393696+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_clean_scrub_stamp":"2024-08-19T01:32:40.202519+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T08:45:08.865144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"84'4","reported_seq":27,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:33:05.471339+0000","last_change":"2024-08-19T01:32:38.915170+0000","last_active":"2024-08-19T01:33:05.471339+0000","last_peered":"2024-08-19T01:33:05.471339+0000","last_clean":"2024-08-19T01:33:05.471339+0000","last_became_active":"2024-08-19T01:32:38.914812+0000","last_became_peered":"2024-08-19T01:32:38.914812+0000","last_unstale":"2024-08-19T01:33:05.471339+0000","last_undegraded":"2024-08-19T01:33:05.471339+0000","last_fullsized":"2024-08-19T01:33:05.471339+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:32:37.889224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:32:37.889224+0000","last_clean_scrub_stamp":"2024-08-19T01:32:37.889224+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T11:46:39.409065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"86'5","reported_seq":22,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:34:04.396323+0000","last_change":"2024-08-19T01:32:41.234133+0000","last_active":"2024-08-19T01:34:04.396323+0000","last_peered":"2024-08-19T01:34:04.396323+0000","last_clean":"2024-08-19T01:34:04.396323+0000","last_became_active":"2024-08-19T01:32:41.233492+0000","last_became_peered":"2024-08-19T01:32:41.233492+0000","last_unstale":"2024-08-19T01:34:04.396323+0000","last_undegraded":"2024-08-19T01:34:04.396323+0000","last_fullsized":"2024-08-19T01:34:04.396323+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_clean_scrub_stamp":"2024-08-19T01:32:40.202519+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T06:11:12.587678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"86'15","reported_seq":32,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:34:04.395111+0000","last_change":"2024-08-19T01:32:41.232628+0000","last_active":"2024-08-19T01:34:04.395111+0000","last_peered":"2024-08-19T01:34:04.395111+0000","last_clean":"2024-08-19T01:34:04.395111+0000","last_became_active":"2024-08-19T01:32:41.232295+0000","last_became_peered":"2024-08-19T01:32:41.232295+0000","last_unstale":"2024-08-19T01:34:04.395111+0000","last_undegraded":"2024-08-19T01:34:04.395111+0000","last_fullsized":"2024-08-19T01:34:04.395111+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_clean_scrub_stamp":"2024-08-19T01:32:40.202519+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T02:01:09.490553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"84'2","reported_seq":24,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:33:05.472666+0000","last_change":"2024-08-19T01:32:38.912000+0000","last_active":"2024-08-19T01:33:05.472666+0000","last_peered":"2024-08-19T01:33:05.472666+0000","last_clean":"2024-08-19T01:33:05.472666+0000","last_became_active":"2024-08-19T01:32:38.911535+0000","last_became_peered":"2024-08-19T01:32:38.911535+0000","last_unstale":"2024-08-19T01:33:05.472666+0000","last_undegraded":"2024-08-19T01:33:05.472666+0000","last_fullsized":"2024-08-19T01:33:05.472666+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:32:37.889224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:32:37.889224+0000","last_clean_scrub_stamp":"2024-08-19T01:32:37.889224+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T07:12:49.358384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"86'9","reported_seq":26,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:34:04.399042+0000","last_change":"2024-08-19T01:32:41.233507+0000","last_active":"2024-08-19T01:34:04.399042+0000","last_peered":"2024-08-19T01:34:04.399042+0000","last_clean":"2024-08-19T01:34:04.399042+0000","last_became_active":"2024-08-19T01:32:41.233179+0000","last_became_peered":"2024-08-19T01:32:41.233179+0000","last_unstale":"2024-08-19T01:34:04.399042+0000","last_undegraded":"2024-08-19T01:34:04.399042+0000","last_fullsized":"2024-08-19T01:34:04.399042+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_clean_scrub_stamp":"2024-08-19T01:32:40.202519+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T11:47:44.780859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1d","version":"84'4","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:33:05.483750+0000","last_change":"2024-08-19T01:33:05.483322+0000","last_active":"2024-08-19T01:33:05.483750+0000","last_peered":"2024-08-19T01:33:05.483750+0000","last_clean":"2024-08-19T01:33:05.483750+0000","last_became_active":"2024-08-19T01:33:05.475496+0000","last_became_peered":"2024-08-19T01:33:05.475496+0000","last_unstale":"2024-08-19T01:33:05.483750+0000","last_undegraded":"2024-08-19T01:33:05.483750+0000","last_fullsized":"2024-08-19T01:33:05.483750+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:32:37.889224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:32:37.889224+0000","last_clean_scrub_stamp":"2024-08-19T01:32:37.889224+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T01:57:48.143495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":10,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,7],"acting":[11,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1f","version":"86'6","reported_seq":23,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:34:04.399271+0000","last_change":"2024-08-19T01:32:41.244641+0000","last_active":"2024-08-19T01:34:04.399271+0000","last_peered":"2024-08-19T01:34:04.399271+0000","last_clean":"2024-08-19T01:34:04.399271+0000","last_became_active":"2024-08-19T01:32:41.244311+0000","last_became_peered":"2024-08-19T01:32:41.244311+0000","last_unstale":"2024-08-19T01:34:04.399271+0000","last_undegraded":"2024-08-19T01:34:04.399271+0000","last_fullsized":"2024-08-19T01:34:04.399271+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_clean_scrub_stamp":"2024-08-19T01:32:40.202519+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T10:39:12.591092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"84'2","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:33:05.472403+0000","last_change":"2024-08-19T01:32:39.051314+0000","last_active":"2024-08-19T01:33:05.472403+0000","last_peered":"2024-08-19T01:33:05.472403+0000","last_clean":"2024-08-19T01:33:05.472403+0000","last_became_active":"2024-08-19T01:32:39.050998+0000","last_became_peered":"2024-08-19T01:32:39.050998+0000","last_unstale":"2024-08-19T01:33:05.472403+0000","last_undegraded":"2024-08-19T01:33:05.472403+0000","last_fullsized":"2024-08-19T01:33:05.472403+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:32:37.889224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:32:37.889224+0000","last_clean_scrub_stamp":"2024-08-19T01:32:37.889224+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T09:21:15.689149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"86'5","reported_seq":23,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:34:04.390896+0000","last_change":"2024-08-19T01:32:41.234762+0000","last_active":"2024-08-19T01:34:04.390896+0000","last_peered":"2024-08-19T01:34:04.390896+0000","last_clean":"2024-08-19T01:34:04.390896+0000","last_became_active":"2024-08-19T01:32:41.234471+0000","last_became_peered":"2024-08-19T01:32:41.234471+0000","last_unstale":"2024-08-19T01:34:04.390896+0000","last_undegraded":"2024-08-19T01:34:04.390896+0000","last_fullsized":"2024-08-19T01:34:04.390896+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_clean_scrub_stamp":"2024-08-19T01:32:40.202519+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T08:34:33.110938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"84'9","reported_seq":32,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:33:05.466470+0000","last_change":"2024-08-19T01:32:38.917068+0000","last_active":"2024-08-19T01:33:05.466470+0000","last_peered":"2024-08-19T01:33:05.466470+0000","last_clean":"2024-08-19T01:33:05.466470+0000","last_became_active":"2024-08-19T01:32:38.915399+0000","last_became_peered":"2024-08-19T01:32:38.915399+0000","last_unstale":"2024-08-19T01:33:05.466470+0000","last_undegraded":"2024-08-19T01:33:05.466470+0000","last_fullsized":"2024-08-19T01:33:05.466470+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:32:37.889224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:32:37.889224+0000","last_clean_scrub_stamp":"2024-08-19T01:32:37.889224+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T04:17:25.061208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":502,"ondisk_log_size":502,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":66628,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":685,"num_read_kb":411,"num_write":183,"num_write_kb":124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":1012,"num_keys_recovered":10,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":421888,"data_stored":204198,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":154,"ondisk_log_size":154,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4751360,"data_stored":4722944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8}],"osd_stats":[{"osd":11,"up_from":78,"seq":335007449126,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32092,"kb_used_data":916,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713060,"statfs":{"total":95995035648,"available":95962173440,"internally_reserved":0,"allocated":937984,"data_stored":451051,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":71,"seq":304942678058,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28612,"kb_used_data":1596,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716540,"statfs":{"total":95995035648,"available":95965736960,"internally_reserved":0,"allocated":1634304,"data_stored":1103928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":66,"seq":283467841583,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32108,"kb_used_data":932,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713044,"statfs":{"total":95995035648,"available":95962157056,"internally_reserved":0,"allocated":954368,"data_stored":451287,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":58,"seq":249108103221,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32108,"kb_used_data":932,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713044,"statfs":{"total":95995035648,"available":95962157056,"internally_reserved":0,"allocated":954368,"data_stored":451207,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":51,"seq":219043332153,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32688,"kb_used_data":1512,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712464,"statfs":{"total":95995035648,"available":95961563136,"internally_reserved":0,"allocated":1548288,"data_stored":1044119,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":46,"seq":197568495678,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27936,"kb_used_data":920,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717216,"statfs":{"total":95995035648,"available":95966429184,"internally_reserved":0,"allocated":942080,"data_stored":450676,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":38,"seq":163208757315,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28540,"kb_used_data":1524,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716612,"statfs":{"total":95995035648,"available":95965810688,"internally_reserved":0,"allocated":1560576,"data_stored":1043351,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":33,"seq":141733920839,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32108,"kb_used_data":932,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713044,"statfs":{"total":95995035648,"available":95962157056,"internally_reserved":0,"allocated":954368,"data_stored":453888,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182476,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27952,"kb_used_data":936,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717200,"statfs":{"total":95995035648,"available":95966412800,"internally_reserved":0,"allocated":958464,"data_stored":450987,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411409,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36332,"kb_used_data":996,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708820,"statfs":{"total":95995035648,"available":95957831680,"internally_reserved":0,"allocated":1019904,"data_stored":512062,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574932,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28000,"kb_used_data":984,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717152,"statfs":{"total":95995035648,"available":95966363648,"internally_reserved":0,"allocated":1007616,"data_stored":516349,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738458,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36276,"kb_used_data":940,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708876,"statfs":{"total":95995035648,"available":95957889024,"internally_reserved":0,"allocated":962560,"data_stored":451947,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":65880,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":86016,"data_stored":61593,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":3419,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":2397,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":3945,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":102400,"data_stored":62974,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-08-19T01:34:45.814 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-08-19T01:34:45.952 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:34:45 smithi117 ceph-mon[22016]: pgmap v336: 97 pgs: 97 active+clean; 642 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 597 B/s wr, 6 op/s 2024-08-19T01:34:45.952 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:34:45 smithi117 ceph-mon[22016]: from='client.? 172.21.15.83:0/1390910346' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-19T01:34:45.966 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:34:45 smithi167 ceph-mon[21875]: pgmap v336: 97 pgs: 97 active+clean; 642 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 597 B/s wr, 6 op/s 2024-08-19T01:34:45.966 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:34:45 smithi167 ceph-mon[21875]: from='client.? 172.21.15.83:0/1390910346' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-19T01:34:46.003 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:46.002+0000 7f32e0600640 1 -- 172.21.15.83:0/4034713935 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f32d8103a60 msgr2=0x7f32d8103e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:34:46.004 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:46.002+0000 7f32e0600640 1 --2- 172.21.15.83:0/4034713935 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f32d8103a60 0x7f32d8103e60 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f32cc009860 tx=0x7f32cc02a340 comp rx=0 tx=0).stop 2024-08-19T01:34:46.004 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:46.003+0000 7f32e0600640 1 -- 172.21.15.83:0/4034713935 shutdown_connections 2024-08-19T01:34:46.004 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:46.003+0000 7f32e0600640 1 --2- 172.21.15.83:0/4034713935 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f32d8105610 0x7f32d810bea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:46.004 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:46.003+0000 7f32e0600640 1 --2- 172.21.15.83:0/4034713935 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f32d8104c50 0x7f32d81050d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:46.004 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:46.003+0000 7f32e0600640 1 --2- 172.21.15.83:0/4034713935 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f32d8103a60 0x7f32d8103e60 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:46.004 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:46.003+0000 7f32e0600640 1 -- 172.21.15.83:0/4034713935 >> 172.21.15.83:0/4034713935 conn(0x7f32d80ff1e0 msgr2=0x7f32d8101600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:34:46.004 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:46.003+0000 7f32e0600640 1 -- 172.21.15.83:0/4034713935 shutdown_connections 2024-08-19T01:34:46.004 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:46.003+0000 7f32e0600640 1 -- 172.21.15.83:0/4034713935 wait complete. 2024-08-19T01:34:46.006 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:46.004+0000 7f32e0600640 1 Processor -- start 2024-08-19T01:34:46.007 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:46.004+0000 7f32e0600640 1 -- start start 2024-08-19T01:34:46.007 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:46.004+0000 7f32e0600640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f32d8103a60 0x7f32d8199dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:34:46.007 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:46.004+0000 7f32e0600640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f32d8104c50 0x7f32d819a310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:34:46.007 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:46.004+0000 7f32e0600640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f32d8105610 0x7f32d81a1380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:34:46.007 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:46.004+0000 7f32e0600640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f32d810f6e0 con 0x7f32d8103a60 2024-08-19T01:34:46.007 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:46.004+0000 7f32e0600640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f32d810f560 con 0x7f32d8104c50 2024-08-19T01:34:46.007 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:46.004+0000 7f32e0600640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f32d810f860 con 0x7f32d8105610 2024-08-19T01:34:46.007 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:46.004+0000 7f32ddc00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f32d8103a60 0x7f32d8199dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:34:46.007 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:46.004+0000 7f32de600640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f32d8105610 0x7f32d81a1380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:34:46.007 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:46.004+0000 7f32de600640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f32d8105610 0x7f32d81a1380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.83:47554/0 (socket says 172.21.15.83:47554) 2024-08-19T01:34:46.007 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:46.004+0000 7f32de600640 1 -- 172.21.15.83:0/1188953001 learned_addr learned my addr 172.21.15.83:0/1188953001 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-08-19T01:34:46.007 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:46.004+0000 7f32dd200640 1 --2- 172.21.15.83:0/1188953001 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f32d8104c50 0x7f32d819a310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:34:46.008 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:46.005+0000 7f32de600640 1 -- 172.21.15.83:0/1188953001 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f32d8104c50 msgr2=0x7f32d819a310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:34:46.008 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:46.005+0000 7f32de600640 1 --2- 172.21.15.83:0/1188953001 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f32d8104c50 0x7f32d819a310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:46.008 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:46.005+0000 7f32de600640 1 -- 172.21.15.83:0/1188953001 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f32d8103a60 msgr2=0x7f32d8199dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:34:46.008 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:46.005+0000 7f32de600640 1 --2- 172.21.15.83:0/1188953001 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f32d8103a60 0x7f32d8199dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:46.008 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:46.005+0000 7f32de600640 1 -- 172.21.15.83:0/1188953001 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f32d81a19f0 con 0x7f32d8105610 2024-08-19T01:34:46.008 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:46.005+0000 7f32dd200640 1 --2- 172.21.15.83:0/1188953001 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f32d8104c50 0x7f32d819a310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-19T01:34:46.008 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:46.005+0000 7f32de600640 1 --2- 172.21.15.83:0/1188953001 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f32d8105610 0x7f32d81a1380 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f32d000efc0 tx=0x7f32d000c800 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:34:46.008 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:46.005+0000 7f32d6000640 1 -- 172.21.15.83:0/1188953001 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32d0003970 con 0x7f32d8105610 2024-08-19T01:34:46.008 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:46.005+0000 7f32d6000640 1 -- 172.21.15.83:0/1188953001 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f32d0004020 con 0x7f32d8105610 2024-08-19T01:34:46.008 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:46.005+0000 7f32d6000640 1 -- 172.21.15.83:0/1188953001 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32d000b9f0 con 0x7f32d8105610 2024-08-19T01:34:46.008 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:46.005+0000 7f32e0600640 1 -- 172.21.15.83:0/1188953001 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f32d81a1ce0 con 0x7f32d8105610 2024-08-19T01:34:46.008 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:46.006+0000 7f32e0600640 1 -- 172.21.15.83:0/1188953001 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f32d81a2180 con 0x7f32d8105610 2024-08-19T01:34:46.009 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:46.007+0000 7f32e0600640 1 -- 172.21.15.83:0/1188953001 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f32d810aec0 con 0x7f32d8105610 2024-08-19T01:34:46.009 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:46.009+0000 7f32d6000640 1 -- 172.21.15.83:0/1188953001 <== mon.1 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 19) ==== 137884+0+0 (secure 0 0 0) 0x7f32d001d020 con 0x7f32d8105610 2024-08-19T01:34:46.010 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:46.010+0000 7f32d6000640 1 --2- 172.21.15.83:0/1188953001 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f32c00a2ca0 0x7f32c00a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:34:46.011 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:46.010+0000 7f32d6000640 1 -- 172.21.15.83:0/1188953001 <== mon.1 v2:172.21.15.167:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8269+0+0 (secure 0 0 0) 0x7f32d00cf6e0 con 0x7f32d8105610 2024-08-19T01:34:46.011 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:46.011+0000 7f32ddc00640 1 --2- 172.21.15.83:0/1188953001 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f32c00a2ca0 0x7f32c00a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:34:46.011 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:46.011+0000 7f32ddc00640 1 --2- 172.21.15.83:0/1188953001 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f32c00a2ca0 0x7f32c00a5160 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f32cc030410 tx=0x7f32cc030550 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:34:46.013 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:46.012+0000 7f32d6000640 1 -- 172.21.15.83:0/1188953001 <== mon.1 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7f32d009c460 con 0x7f32d8105610 2024-08-19T01:34:46.185 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:46.184+0000 7f32e0600640 1 -- 172.21.15.83:0/1188953001 --> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f32d8062360 con 0x7f32c00a2ca0 2024-08-19T01:34:46.191 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:46.191+0000 7f32d6000640 1 -- 172.21.15.83:0/1188953001 <== mgr.14150 v2:172.21.15.83:6800/683740916 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258843 (secure 0 0 0) 0x7f32d8062360 con 0x7f32c00a2ca0 2024-08-19T01:34:46.195 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-08-19T01:34:46.196 INFO:teuthology.orchestra.run.smithi083.stderr:dumped all 2024-08-19T01:34:46.198 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:46.195+0000 7f32e0600640 1 -- 172.21.15.83:0/1188953001 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f32c00a2ca0 msgr2=0x7f32c00a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:34:46.198 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:46.195+0000 7f32e0600640 1 --2- 172.21.15.83:0/1188953001 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f32c00a2ca0 0x7f32c00a5160 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f32cc030410 tx=0x7f32cc030550 comp rx=0 tx=0).stop 2024-08-19T01:34:46.198 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:46.195+0000 7f32e0600640 1 -- 172.21.15.83:0/1188953001 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f32d8105610 msgr2=0x7f32d81a1380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:34:46.198 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:46.195+0000 7f32e0600640 1 --2- 172.21.15.83:0/1188953001 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f32d8105610 0x7f32d81a1380 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f32d000efc0 tx=0x7f32d000c800 comp rx=0 tx=0).stop 2024-08-19T01:34:46.198 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:46.195+0000 7f32e0600640 1 -- 172.21.15.83:0/1188953001 shutdown_connections 2024-08-19T01:34:46.198 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:46.195+0000 7f32e0600640 1 --2- 172.21.15.83:0/1188953001 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f32c00a2ca0 0x7f32c00a5160 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:46.198 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:46.195+0000 7f32e0600640 1 --2- 172.21.15.83:0/1188953001 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f32d8105610 0x7f32d81a1380 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:46.198 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:46.195+0000 7f32e0600640 1 --2- 172.21.15.83:0/1188953001 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f32d8104c50 0x7f32d819a310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:46.198 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:46.195+0000 7f32e0600640 1 --2- 172.21.15.83:0/1188953001 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f32d8103a60 0x7f32d8199dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:46.198 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:46.195+0000 7f32e0600640 1 -- 172.21.15.83:0/1188953001 >> 172.21.15.83:0/1188953001 conn(0x7f32d80ff1e0 msgr2=0x7f32d8108090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:34:46.198 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:46.195+0000 7f32e0600640 1 -- 172.21.15.83:0/1188953001 shutdown_connections 2024-08-19T01:34:46.198 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:46.195+0000 7f32e0600640 1 -- 172.21.15.83:0/1188953001 wait complete. 2024-08-19T01:34:46.212 INFO:teuthology.orchestra.run.smithi083.stdout:{"pg_ready":true,"pg_map":{"version":336,"stamp":"2024-08-19T01:34:44.389285+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":657230,"num_objects":64,"num_object_clones":0,"num_object_copies":192,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":811,"num_read_kb":520,"num_write":366,"num_write_kb":3064,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":1181748,"num_keys_recovered":10,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":793,"ondisk_log_size":793,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":374752,"kb_used_data":13120,"kb_used_omap":18,"kb_used_meta":361517,"kb_avail":1124567072,"statfs":{"total":1151940427776,"available":1151556681728,"internally_reserved":0,"allocated":13434880,"data_stored":7380852,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19143,"internal_metadata":370193721},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":1091,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":73,"num_read_kb":33,"num_write":10,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002860"},"pg_stats":[{"pgid":"3.3f","version":"86'8","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:34:04.396225+0000","last_change":"2024-08-19T01:32:41.244214+0000","last_active":"2024-08-19T01:34:04.396225+0000","last_peered":"2024-08-19T01:34:04.396225+0000","last_clean":"2024-08-19T01:34:04.396225+0000","last_became_active":"2024-08-19T01:32:41.243889+0000","last_became_peered":"2024-08-19T01:32:41.243889+0000","last_unstale":"2024-08-19T01:34:04.396225+0000","last_undegraded":"2024-08-19T01:34:04.396225+0000","last_fullsized":"2024-08-19T01:34:04.396225+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_clean_scrub_stamp":"2024-08-19T01:32:40.202519+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T13:10:18.230055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"86'4","reported_seq":21,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:34:04.396539+0000","last_change":"2024-08-19T01:32:41.242729+0000","last_active":"2024-08-19T01:34:04.396539+0000","last_peered":"2024-08-19T01:34:04.396539+0000","last_clean":"2024-08-19T01:34:04.396539+0000","last_became_active":"2024-08-19T01:32:41.242436+0000","last_became_peered":"2024-08-19T01:32:41.242436+0000","last_unstale":"2024-08-19T01:34:04.396539+0000","last_undegraded":"2024-08-19T01:34:04.396539+0000","last_fullsized":"2024-08-19T01:34:04.396539+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_clean_scrub_stamp":"2024-08-19T01:32:40.202519+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T13:19:32.010576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"86'7","reported_seq":24,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:34:04.399256+0000","last_change":"2024-08-19T01:32:41.247377+0000","last_active":"2024-08-19T01:34:04.399256+0000","last_peered":"2024-08-19T01:34:04.399256+0000","last_clean":"2024-08-19T01:34:04.399256+0000","last_became_active":"2024-08-19T01:32:41.246804+0000","last_became_peered":"2024-08-19T01:32:41.246804+0000","last_unstale":"2024-08-19T01:34:04.399256+0000","last_undegraded":"2024-08-19T01:34:04.399256+0000","last_fullsized":"2024-08-19T01:34:04.399256+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_clean_scrub_stamp":"2024-08-19T01:32:40.202519+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T04:48:52.382687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"86'8","reported_seq":26,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:34:04.389960+0000","last_change":"2024-08-19T01:32:41.232900+0000","last_active":"2024-08-19T01:34:04.389960+0000","last_peered":"2024-08-19T01:34:04.389960+0000","last_clean":"2024-08-19T01:34:04.389960+0000","last_became_active":"2024-08-19T01:32:41.232573+0000","last_became_peered":"2024-08-19T01:32:41.232573+0000","last_unstale":"2024-08-19T01:34:04.389960+0000","last_undegraded":"2024-08-19T01:34:04.389960+0000","last_fullsized":"2024-08-19T01:34:04.389960+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_clean_scrub_stamp":"2024-08-19T01:32:40.202519+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T09:42:36.309834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"86'4","reported_seq":27,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:34:04.393723+0000","last_change":"2024-08-19T01:33:05.478964+0000","last_active":"2024-08-19T01:34:04.393723+0000","last_peered":"2024-08-19T01:34:04.393723+0000","last_clean":"2024-08-19T01:34:04.393723+0000","last_became_active":"2024-08-19T01:33:05.478430+0000","last_became_peered":"2024-08-19T01:33:05.478430+0000","last_unstale":"2024-08-19T01:34:04.393723+0000","last_undegraded":"2024-08-19T01:34:04.393723+0000","last_fullsized":"2024-08-19T01:34:04.393723+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_clean_scrub_stamp":"2024-08-19T01:32:40.202519+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T04:10:49.267550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,6],"acting":[8,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"86'6","reported_seq":23,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:34:04.395690+0000","last_change":"2024-08-19T01:32:41.228263+0000","last_active":"2024-08-19T01:34:04.395690+0000","last_peered":"2024-08-19T01:34:04.395690+0000","last_clean":"2024-08-19T01:34:04.395690+0000","last_became_active":"2024-08-19T01:32:41.227765+0000","last_became_peered":"2024-08-19T01:32:41.227765+0000","last_unstale":"2024-08-19T01:34:04.395690+0000","last_undegraded":"2024-08-19T01:34:04.395690+0000","last_fullsized":"2024-08-19T01:34:04.395690+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_clean_scrub_stamp":"2024-08-19T01:32:40.202519+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T11:21:18.817886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"86'5","reported_seq":22,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:34:04.391175+0000","last_change":"2024-08-19T01:32:41.235680+0000","last_active":"2024-08-19T01:34:04.391175+0000","last_peered":"2024-08-19T01:34:04.391175+0000","last_clean":"2024-08-19T01:34:04.391175+0000","last_became_active":"2024-08-19T01:32:41.234150+0000","last_became_peered":"2024-08-19T01:32:41.234150+0000","last_unstale":"2024-08-19T01:34:04.391175+0000","last_undegraded":"2024-08-19T01:34:04.391175+0000","last_fullsized":"2024-08-19T01:34:04.391175+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_clean_scrub_stamp":"2024-08-19T01:32:40.202519+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T10:28:08.908632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"86'9","reported_seq":26,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:34:04.394271+0000","last_change":"2024-08-19T01:32:41.230170+0000","last_active":"2024-08-19T01:34:04.394271+0000","last_peered":"2024-08-19T01:34:04.394271+0000","last_clean":"2024-08-19T01:34:04.394271+0000","last_became_active":"2024-08-19T01:32:41.229467+0000","last_became_peered":"2024-08-19T01:32:41.229467+0000","last_unstale":"2024-08-19T01:34:04.394271+0000","last_undegraded":"2024-08-19T01:34:04.394271+0000","last_fullsized":"2024-08-19T01:34:04.394271+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_clean_scrub_stamp":"2024-08-19T01:32:40.202519+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T10:24:15.832910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"86'11","reported_seq":28,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:34:04.398861+0000","last_change":"2024-08-19T01:32:41.230607+0000","last_active":"2024-08-19T01:34:04.398861+0000","last_peered":"2024-08-19T01:34:04.398861+0000","last_clean":"2024-08-19T01:34:04.398861+0000","last_became_active":"2024-08-19T01:32:41.229979+0000","last_became_peered":"2024-08-19T01:32:41.229979+0000","last_unstale":"2024-08-19T01:34:04.398861+0000","last_undegraded":"2024-08-19T01:34:04.398861+0000","last_fullsized":"2024-08-19T01:34:04.398861+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_clean_scrub_stamp":"2024-08-19T01:32:40.202519+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T02:23:46.736139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"86'16","reported_seq":33,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:34:04.395510+0000","last_change":"2024-08-19T01:32:41.233576+0000","last_active":"2024-08-19T01:34:04.395510+0000","last_peered":"2024-08-19T01:34:04.395510+0000","last_clean":"2024-08-19T01:34:04.395510+0000","last_became_active":"2024-08-19T01:32:41.231279+0000","last_became_peered":"2024-08-19T01:32:41.231279+0000","last_unstale":"2024-08-19T01:34:04.395510+0000","last_undegraded":"2024-08-19T01:34:04.395510+0000","last_fullsized":"2024-08-19T01:34:04.395510+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_clean_scrub_stamp":"2024-08-19T01:32:40.202519+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T07:32:45.341628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"86'6","reported_seq":23,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:34:04.396391+0000","last_change":"2024-08-19T01:32:41.233192+0000","last_active":"2024-08-19T01:34:04.396391+0000","last_peered":"2024-08-19T01:34:04.396391+0000","last_clean":"2024-08-19T01:34:04.396391+0000","last_became_active":"2024-08-19T01:32:41.230944+0000","last_became_peered":"2024-08-19T01:32:41.230944+0000","last_unstale":"2024-08-19T01:34:04.396391+0000","last_undegraded":"2024-08-19T01:34:04.396391+0000","last_fullsized":"2024-08-19T01:34:04.396391+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_clean_scrub_stamp":"2024-08-19T01:32:40.202519+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T08:55:29.036252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"86'7","reported_seq":24,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:34:04.386560+0000","last_change":"2024-08-19T01:32:41.234216+0000","last_active":"2024-08-19T01:34:04.386560+0000","last_peered":"2024-08-19T01:34:04.386560+0000","last_clean":"2024-08-19T01:34:04.386560+0000","last_became_active":"2024-08-19T01:32:41.233906+0000","last_became_peered":"2024-08-19T01:32:41.233906+0000","last_unstale":"2024-08-19T01:34:04.386560+0000","last_undegraded":"2024-08-19T01:34:04.386560+0000","last_fullsized":"2024-08-19T01:34:04.386560+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_clean_scrub_stamp":"2024-08-19T01:32:40.202519+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T08:35:01.764494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"86'10","reported_seq":27,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:34:04.398217+0000","last_change":"2024-08-19T01:32:41.230651+0000","last_active":"2024-08-19T01:34:04.398217+0000","last_peered":"2024-08-19T01:34:04.398217+0000","last_clean":"2024-08-19T01:34:04.398217+0000","last_became_active":"2024-08-19T01:32:41.230253+0000","last_became_peered":"2024-08-19T01:32:41.230253+0000","last_unstale":"2024-08-19T01:34:04.398217+0000","last_undegraded":"2024-08-19T01:34:04.398217+0000","last_fullsized":"2024-08-19T01:34:04.398217+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_clean_scrub_stamp":"2024-08-19T01:32:40.202519+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T09:41:59.942821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"86'7","reported_seq":24,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:34:04.396983+0000","last_change":"2024-08-19T01:32:41.240082+0000","last_active":"2024-08-19T01:34:04.396983+0000","last_peered":"2024-08-19T01:34:04.396983+0000","last_clean":"2024-08-19T01:34:04.396983+0000","last_became_active":"2024-08-19T01:32:41.239711+0000","last_became_peered":"2024-08-19T01:32:41.239711+0000","last_unstale":"2024-08-19T01:34:04.396983+0000","last_undegraded":"2024-08-19T01:34:04.396983+0000","last_fullsized":"2024-08-19T01:34:04.396983+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_clean_scrub_stamp":"2024-08-19T01:32:40.202519+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T12:48:59.122307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"86'7","reported_seq":30,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:34:04.390223+0000","last_change":"2024-08-19T01:33:05.474883+0000","last_active":"2024-08-19T01:34:04.390223+0000","last_peered":"2024-08-19T01:34:04.390223+0000","last_clean":"2024-08-19T01:34:04.390223+0000","last_became_active":"2024-08-19T01:33:05.474502+0000","last_became_peered":"2024-08-19T01:33:05.474502+0000","last_unstale":"2024-08-19T01:34:04.390223+0000","last_undegraded":"2024-08-19T01:34:04.390223+0000","last_fullsized":"2024-08-19T01:34:04.390223+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_clean_scrub_stamp":"2024-08-19T01:32:40.202519+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T07:02:01.171103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2],"acting":[0,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"86'9","reported_seq":26,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:34:04.399368+0000","last_change":"2024-08-19T01:32:41.240899+0000","last_active":"2024-08-19T01:34:04.399368+0000","last_peered":"2024-08-19T01:34:04.399368+0000","last_clean":"2024-08-19T01:34:04.399368+0000","last_became_active":"2024-08-19T01:32:41.240532+0000","last_became_peered":"2024-08-19T01:32:41.240532+0000","last_unstale":"2024-08-19T01:34:04.399368+0000","last_undegraded":"2024-08-19T01:34:04.399368+0000","last_fullsized":"2024-08-19T01:34:04.399368+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_clean_scrub_stamp":"2024-08-19T01:32:40.202519+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T08:17:47.035887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"86'8","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:34:04.396273+0000","last_change":"2024-08-19T01:32:41.237977+0000","last_active":"2024-08-19T01:34:04.396273+0000","last_peered":"2024-08-19T01:34:04.396273+0000","last_clean":"2024-08-19T01:34:04.396273+0000","last_became_active":"2024-08-19T01:32:41.237634+0000","last_became_peered":"2024-08-19T01:32:41.237634+0000","last_unstale":"2024-08-19T01:34:04.396273+0000","last_undegraded":"2024-08-19T01:34:04.396273+0000","last_fullsized":"2024-08-19T01:34:04.396273+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_clean_scrub_stamp":"2024-08-19T01:32:40.202519+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T12:48:37.005972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"86'10","reported_seq":27,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:34:04.391627+0000","last_change":"2024-08-19T01:32:41.236498+0000","last_active":"2024-08-19T01:34:04.391627+0000","last_peered":"2024-08-19T01:34:04.391627+0000","last_clean":"2024-08-19T01:34:04.391627+0000","last_became_active":"2024-08-19T01:32:41.236122+0000","last_became_peered":"2024-08-19T01:32:41.236122+0000","last_unstale":"2024-08-19T01:34:04.391627+0000","last_undegraded":"2024-08-19T01:34:04.391627+0000","last_fullsized":"2024-08-19T01:34:04.391627+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_clean_scrub_stamp":"2024-08-19T01:32:40.202519+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T02:31:23.648064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"86'7","reported_seq":24,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:34:04.390365+0000","last_change":"2024-08-19T01:32:41.234607+0000","last_active":"2024-08-19T01:34:04.390365+0000","last_peered":"2024-08-19T01:34:04.390365+0000","last_clean":"2024-08-19T01:34:04.390365+0000","last_became_active":"2024-08-19T01:32:41.232867+0000","last_became_peered":"2024-08-19T01:32:41.232867+0000","last_unstale":"2024-08-19T01:34:04.390365+0000","last_undegraded":"2024-08-19T01:34:04.390365+0000","last_fullsized":"2024-08-19T01:34:04.390365+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_clean_scrub_stamp":"2024-08-19T01:32:40.202519+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T12:32:38.441848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2c","version":"86'8","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:34:04.384758+0000","last_change":"2024-08-19T01:32:41.231705+0000","last_active":"2024-08-19T01:34:04.384758+0000","last_peered":"2024-08-19T01:34:04.384758+0000","last_clean":"2024-08-19T01:34:04.384758+0000","last_became_active":"2024-08-19T01:32:41.231262+0000","last_became_peered":"2024-08-19T01:32:41.231262+0000","last_unstale":"2024-08-19T01:34:04.384758+0000","last_undegraded":"2024-08-19T01:34:04.384758+0000","last_fullsized":"2024-08-19T01:34:04.384758+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_clean_scrub_stamp":"2024-08-19T01:32:40.202519+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T10:41:15.025471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2b","version":"86'11","reported_seq":28,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:34:04.397040+0000","last_change":"2024-08-19T01:32:41.243192+0000","last_active":"2024-08-19T01:34:04.397040+0000","last_peered":"2024-08-19T01:34:04.397040+0000","last_clean":"2024-08-19T01:34:04.397040+0000","last_became_active":"2024-08-19T01:32:41.242704+0000","last_became_peered":"2024-08-19T01:32:41.242704+0000","last_unstale":"2024-08-19T01:34:04.397040+0000","last_undegraded":"2024-08-19T01:34:04.397040+0000","last_fullsized":"2024-08-19T01:34:04.397040+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_clean_scrub_stamp":"2024-08-19T01:32:40.202519+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T01:44:43.270213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2a","version":"86'10","reported_seq":27,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:34:04.399726+0000","last_change":"2024-08-19T01:32:41.239908+0000","last_active":"2024-08-19T01:34:04.399726+0000","last_peered":"2024-08-19T01:34:04.399726+0000","last_clean":"2024-08-19T01:34:04.399726+0000","last_became_active":"2024-08-19T01:32:41.239438+0000","last_became_peered":"2024-08-19T01:32:41.239438+0000","last_unstale":"2024-08-19T01:34:04.399726+0000","last_undegraded":"2024-08-19T01:34:04.399726+0000","last_fullsized":"2024-08-19T01:34:04.399726+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_clean_scrub_stamp":"2024-08-19T01:32:40.202519+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T11:03:26.822111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.29","version":"86'8","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:34:04.389344+0000","last_change":"2024-08-19T01:32:41.226373+0000","last_active":"2024-08-19T01:34:04.389344+0000","last_peered":"2024-08-19T01:34:04.389344+0000","last_clean":"2024-08-19T01:34:04.389344+0000","last_became_active":"2024-08-19T01:32:41.226089+0000","last_became_peered":"2024-08-19T01:32:41.226089+0000","last_unstale":"2024-08-19T01:34:04.389344+0000","last_undegraded":"2024-08-19T01:34:04.389344+0000","last_fullsized":"2024-08-19T01:34:04.389344+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_clean_scrub_stamp":"2024-08-19T01:32:40.202519+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T01:33:56.639300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"86'6","reported_seq":23,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:34:04.389272+0000","last_change":"2024-08-19T01:32:41.237829+0000","last_active":"2024-08-19T01:34:04.389272+0000","last_peered":"2024-08-19T01:34:04.389272+0000","last_clean":"2024-08-19T01:34:04.389272+0000","last_became_active":"2024-08-19T01:32:41.237518+0000","last_became_peered":"2024-08-19T01:32:41.237518+0000","last_unstale":"2024-08-19T01:34:04.389272+0000","last_undegraded":"2024-08-19T01:34:04.389272+0000","last_fullsized":"2024-08-19T01:34:04.389272+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_clean_scrub_stamp":"2024-08-19T01:32:40.202519+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T02:25:11.307658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"86'2","reported_seq":19,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:34:04.386117+0000","last_change":"2024-08-19T01:32:41.234508+0000","last_active":"2024-08-19T01:34:04.386117+0000","last_peered":"2024-08-19T01:34:04.386117+0000","last_clean":"2024-08-19T01:34:04.386117+0000","last_became_active":"2024-08-19T01:32:41.232869+0000","last_became_peered":"2024-08-19T01:32:41.232869+0000","last_unstale":"2024-08-19T01:34:04.386117+0000","last_undegraded":"2024-08-19T01:34:04.386117+0000","last_fullsized":"2024-08-19T01:34:04.386117+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_clean_scrub_stamp":"2024-08-19T01:32:40.202519+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T11:20:55.097576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"86'15","reported_seq":32,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:34:04.390238+0000","last_change":"2024-08-19T01:32:41.230536+0000","last_active":"2024-08-19T01:34:04.390238+0000","last_peered":"2024-08-19T01:34:04.390238+0000","last_clean":"2024-08-19T01:34:04.390238+0000","last_became_active":"2024-08-19T01:32:41.230179+0000","last_became_peered":"2024-08-19T01:32:41.230179+0000","last_unstale":"2024-08-19T01:34:04.390238+0000","last_undegraded":"2024-08-19T01:34:04.390238+0000","last_fullsized":"2024-08-19T01:34:04.390238+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_clean_scrub_stamp":"2024-08-19T01:32:40.202519+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T02:25:09.441363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.25","version":"86'6","reported_seq":23,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:34:04.387595+0000","last_change":"2024-08-19T01:32:41.230695+0000","last_active":"2024-08-19T01:34:04.387595+0000","last_peered":"2024-08-19T01:34:04.387595+0000","last_clean":"2024-08-19T01:34:04.387595+0000","last_became_active":"2024-08-19T01:32:41.230324+0000","last_became_peered":"2024-08-19T01:32:41.230324+0000","last_unstale":"2024-08-19T01:34:04.387595+0000","last_undegraded":"2024-08-19T01:34:04.387595+0000","last_fullsized":"2024-08-19T01:34:04.387595+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_clean_scrub_stamp":"2024-08-19T01:32:40.202519+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T05:38:09.317605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.24","version":"86'4","reported_seq":21,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:34:04.391939+0000","last_change":"2024-08-19T01:32:41.238325+0000","last_active":"2024-08-19T01:34:04.391939+0000","last_peered":"2024-08-19T01:34:04.391939+0000","last_clean":"2024-08-19T01:34:04.391939+0000","last_became_active":"2024-08-19T01:32:41.237768+0000","last_became_peered":"2024-08-19T01:32:41.237768+0000","last_unstale":"2024-08-19T01:34:04.391939+0000","last_undegraded":"2024-08-19T01:34:04.391939+0000","last_fullsized":"2024-08-19T01:34:04.391939+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_clean_scrub_stamp":"2024-08-19T01:32:40.202519+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T08:45:06.866241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.23","version":"86'7","reported_seq":24,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:34:04.396309+0000","last_change":"2024-08-19T01:32:41.246950+0000","last_active":"2024-08-19T01:34:04.396309+0000","last_peered":"2024-08-19T01:34:04.396309+0000","last_clean":"2024-08-19T01:34:04.396309+0000","last_became_active":"2024-08-19T01:32:41.245441+0000","last_became_peered":"2024-08-19T01:32:41.245441+0000","last_unstale":"2024-08-19T01:34:04.396309+0000","last_undegraded":"2024-08-19T01:34:04.396309+0000","last_fullsized":"2024-08-19T01:34:04.396309+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_clean_scrub_stamp":"2024-08-19T01:32:40.202519+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T03:11:06.921258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"86'8","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:34:04.373200+0000","last_change":"2024-08-19T01:32:41.244810+0000","last_active":"2024-08-19T01:34:04.373200+0000","last_peered":"2024-08-19T01:34:04.373200+0000","last_clean":"2024-08-19T01:34:04.373200+0000","last_became_active":"2024-08-19T01:32:41.244401+0000","last_became_peered":"2024-08-19T01:32:41.244401+0000","last_unstale":"2024-08-19T01:34:04.373200+0000","last_undegraded":"2024-08-19T01:34:04.373200+0000","last_fullsized":"2024-08-19T01:34:04.373200+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_clean_scrub_stamp":"2024-08-19T01:32:40.202519+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T04:54:57.728181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.21","version":"86'12","reported_seq":29,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:34:04.387129+0000","last_change":"2024-08-19T01:32:41.235977+0000","last_active":"2024-08-19T01:34:04.387129+0000","last_peered":"2024-08-19T01:34:04.387129+0000","last_clean":"2024-08-19T01:34:04.387129+0000","last_became_active":"2024-08-19T01:32:41.235693+0000","last_became_peered":"2024-08-19T01:32:41.235693+0000","last_unstale":"2024-08-19T01:34:04.387129+0000","last_undegraded":"2024-08-19T01:34:04.387129+0000","last_fullsized":"2024-08-19T01:34:04.387129+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_clean_scrub_stamp":"2024-08-19T01:32:40.202519+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T04:59:15.766532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"86'9","reported_seq":32,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:34:04.386756+0000","last_change":"2024-08-19T01:33:05.473318+0000","last_active":"2024-08-19T01:34:04.386756+0000","last_peered":"2024-08-19T01:34:04.386756+0000","last_clean":"2024-08-19T01:34:04.386756+0000","last_became_active":"2024-08-19T01:33:05.472885+0000","last_became_peered":"2024-08-19T01:33:05.472885+0000","last_unstale":"2024-08-19T01:34:04.386756+0000","last_undegraded":"2024-08-19T01:34:04.386756+0000","last_fullsized":"2024-08-19T01:34:04.386756+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_clean_scrub_stamp":"2024-08-19T01:32:40.202519+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T07:49:35.525930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,9],"acting":[11,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1a","version":"86'7","reported_seq":24,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:34:04.376075+0000","last_change":"2024-08-19T01:32:41.230846+0000","last_active":"2024-08-19T01:34:04.376075+0000","last_peered":"2024-08-19T01:34:04.376075+0000","last_clean":"2024-08-19T01:34:04.376075+0000","last_became_active":"2024-08-19T01:32:41.230405+0000","last_became_peered":"2024-08-19T01:32:41.230405+0000","last_unstale":"2024-08-19T01:34:04.376075+0000","last_undegraded":"2024-08-19T01:34:04.376075+0000","last_fullsized":"2024-08-19T01:34:04.376075+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_clean_scrub_stamp":"2024-08-19T01:32:40.202519+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T08:59:46.241898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1b","version":"86'8","reported_seq":158,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:34:42.617354+0000","last_change":"2024-08-19T01:33:05.479889+0000","last_active":"2024-08-19T01:34:42.617354+0000","last_peered":"2024-08-19T01:34:42.617354+0000","last_clean":"2024-08-19T01:34:42.617354+0000","last_became_active":"2024-08-19T01:33:05.474698+0000","last_became_peered":"2024-08-19T01:33:05.474698+0000","last_unstale":"2024-08-19T01:34:42.617354+0000","last_undegraded":"2024-08-19T01:34:42.617354+0000","last_fullsized":"2024-08-19T01:34:42.617354+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:32:37.889224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:32:37.889224+0000","last_clean_scrub_stamp":"2024-08-19T01:32:37.889224+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T08:41:29.053471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1543,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":114,"num_read_kb":5,"num_write":5,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":1012,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7],"acting":[5,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"86'8","reported_seq":19,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:34:04.387536+0000","last_change":"2024-08-19T01:33:05.474083+0000","last_active":"2024-08-19T01:34:04.387536+0000","last_peered":"2024-08-19T01:34:04.387536+0000","last_clean":"2024-08-19T01:34:04.387536+0000","last_became_active":"2024-08-19T01:33:05.473718+0000","last_became_peered":"2024-08-19T01:33:05.473718+0000","last_unstale":"2024-08-19T01:34:04.387536+0000","last_undegraded":"2024-08-19T01:34:04.387536+0000","last_fullsized":"2024-08-19T01:34:04.387536+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_clean_scrub_stamp":"2024-08-19T01:32:40.202519+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T02:25:11.955170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,4,3],"acting":[9,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.1a","version":"84'3","reported_seq":35,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:33:05.705917+0000","last_change":"2024-08-19T01:33:05.705917+0000","last_active":"2024-08-19T01:33:05.705917+0000","last_peered":"2024-08-19T01:33:05.705917+0000","last_clean":"2024-08-19T01:33:05.705917+0000","last_became_active":"2024-08-19T01:33:05.702775+0000","last_became_peered":"2024-08-19T01:33:05.702775+0000","last_unstale":"2024-08-19T01:33:05.705917+0000","last_undegraded":"2024-08-19T01:33:05.705917+0000","last_fullsized":"2024-08-19T01:33:05.705917+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:32:37.889224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:32:37.889224+0000","last_clean_scrub_stamp":"2024-08-19T01:32:37.889224+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T03:58:29.989815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,6],"acting":[10,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.19","version":"84'2","reported_seq":24,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:33:05.477490+0000","last_change":"2024-08-19T01:32:38.911757+0000","last_active":"2024-08-19T01:33:05.477490+0000","last_peered":"2024-08-19T01:33:05.477490+0000","last_clean":"2024-08-19T01:33:05.477490+0000","last_became_active":"2024-08-19T01:32:38.910832+0000","last_became_peered":"2024-08-19T01:32:38.910832+0000","last_unstale":"2024-08-19T01:33:05.477490+0000","last_undegraded":"2024-08-19T01:33:05.477490+0000","last_fullsized":"2024-08-19T01:33:05.477490+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:32:37.889224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:32:37.889224+0000","last_clean_scrub_stamp":"2024-08-19T01:32:37.889224+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T04:46:54.986693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"86'8","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:34:04.392576+0000","last_change":"2024-08-19T01:32:41.231130+0000","last_active":"2024-08-19T01:34:04.392576+0000","last_peered":"2024-08-19T01:34:04.392576+0000","last_clean":"2024-08-19T01:34:04.392576+0000","last_became_active":"2024-08-19T01:32:41.230883+0000","last_became_peered":"2024-08-19T01:32:41.230883+0000","last_unstale":"2024-08-19T01:34:04.392576+0000","last_undegraded":"2024-08-19T01:34:04.392576+0000","last_fullsized":"2024-08-19T01:34:04.392576+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_clean_scrub_stamp":"2024-08-19T01:32:40.202519+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T06:33:11.856202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"84'1","reported_seq":23,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:33:05.470862+0000","last_change":"2024-08-19T01:32:38.916868+0000","last_active":"2024-08-19T01:33:05.470862+0000","last_peered":"2024-08-19T01:33:05.470862+0000","last_clean":"2024-08-19T01:33:05.470862+0000","last_became_active":"2024-08-19T01:32:38.916500+0000","last_became_peered":"2024-08-19T01:32:38.916500+0000","last_unstale":"2024-08-19T01:33:05.470862+0000","last_undegraded":"2024-08-19T01:33:05.470862+0000","last_fullsized":"2024-08-19T01:33:05.470862+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:32:37.889224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:32:37.889224+0000","last_clean_scrub_stamp":"2024-08-19T01:32:37.889224+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T12:30:07.085829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"86'11","reported_seq":28,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:34:04.391892+0000","last_change":"2024-08-19T01:32:41.235290+0000","last_active":"2024-08-19T01:34:04.391892+0000","last_peered":"2024-08-19T01:34:04.391892+0000","last_clean":"2024-08-19T01:34:04.391892+0000","last_became_active":"2024-08-19T01:32:41.235013+0000","last_became_peered":"2024-08-19T01:32:41.235013+0000","last_unstale":"2024-08-19T01:34:04.391892+0000","last_undegraded":"2024-08-19T01:34:04.391892+0000","last_fullsized":"2024-08-19T01:34:04.391892+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_clean_scrub_stamp":"2024-08-19T01:32:40.202519+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T09:50:14.837080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"86'4","reported_seq":133,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:34:37.609085+0000","last_change":"2024-08-19T01:32:38.912023+0000","last_active":"2024-08-19T01:34:37.609085+0000","last_peered":"2024-08-19T01:34:37.609085+0000","last_clean":"2024-08-19T01:34:37.609085+0000","last_became_active":"2024-08-19T01:32:38.911679+0000","last_became_peered":"2024-08-19T01:32:38.911679+0000","last_unstale":"2024-08-19T01:34:37.609085+0000","last_undegraded":"2024-08-19T01:34:37.609085+0000","last_fullsized":"2024-08-19T01:34:37.609085+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:32:37.889224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:32:37.889224+0000","last_clean_scrub_stamp":"2024-08-19T01:32:37.889224+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T12:08:51.118338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":107,"num_read_kb":107,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"86'10","reported_seq":27,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:34:04.399254+0000","last_change":"2024-08-19T01:32:41.232264+0000","last_active":"2024-08-19T01:34:04.399254+0000","last_peered":"2024-08-19T01:34:04.399254+0000","last_clean":"2024-08-19T01:34:04.399254+0000","last_became_active":"2024-08-19T01:32:41.231788+0000","last_became_peered":"2024-08-19T01:32:41.231788+0000","last_unstale":"2024-08-19T01:34:04.399254+0000","last_undegraded":"2024-08-19T01:34:04.399254+0000","last_fullsized":"2024-08-19T01:34:04.399254+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_clean_scrub_stamp":"2024-08-19T01:32:40.202519+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T04:59:06.443992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"84'2","reported_seq":24,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:33:05.472954+0000","last_change":"2024-08-19T01:32:38.915770+0000","last_active":"2024-08-19T01:33:05.472954+0000","last_peered":"2024-08-19T01:33:05.472954+0000","last_clean":"2024-08-19T01:33:05.472954+0000","last_became_active":"2024-08-19T01:32:38.915378+0000","last_became_peered":"2024-08-19T01:32:38.915378+0000","last_unstale":"2024-08-19T01:33:05.472954+0000","last_undegraded":"2024-08-19T01:33:05.472954+0000","last_fullsized":"2024-08-19T01:33:05.472954+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:32:37.889224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:32:37.889224+0000","last_clean_scrub_stamp":"2024-08-19T01:32:37.889224+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T09:16:14.513172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"86'9","reported_seq":26,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:34:04.395227+0000","last_change":"2024-08-19T01:32:41.243476+0000","last_active":"2024-08-19T01:34:04.395227+0000","last_peered":"2024-08-19T01:34:04.395227+0000","last_clean":"2024-08-19T01:34:04.395227+0000","last_became_active":"2024-08-19T01:32:41.243190+0000","last_became_peered":"2024-08-19T01:32:41.243190+0000","last_unstale":"2024-08-19T01:34:04.395227+0000","last_undegraded":"2024-08-19T01:34:04.395227+0000","last_fullsized":"2024-08-19T01:34:04.395227+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_clean_scrub_stamp":"2024-08-19T01:32:40.202519+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T12:07:34.096919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"84'4","reported_seq":27,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:33:05.470664+0000","last_change":"2024-08-19T01:32:39.051458+0000","last_active":"2024-08-19T01:33:05.470664+0000","last_peered":"2024-08-19T01:33:05.470664+0000","last_clean":"2024-08-19T01:33:05.470664+0000","last_became_active":"2024-08-19T01:32:39.051113+0000","last_became_peered":"2024-08-19T01:32:39.051113+0000","last_unstale":"2024-08-19T01:33:05.470664+0000","last_undegraded":"2024-08-19T01:33:05.470664+0000","last_fullsized":"2024-08-19T01:33:05.470664+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:32:37.889224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:32:37.889224+0000","last_clean_scrub_stamp":"2024-08-19T01:32:37.889224+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T02:15:51.371054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"86'11","reported_seq":28,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:34:04.398787+0000","last_change":"2024-08-19T01:32:41.236453+0000","last_active":"2024-08-19T01:34:04.398787+0000","last_peered":"2024-08-19T01:34:04.398787+0000","last_clean":"2024-08-19T01:34:04.398787+0000","last_became_active":"2024-08-19T01:32:41.236135+0000","last_became_peered":"2024-08-19T01:32:41.236135+0000","last_unstale":"2024-08-19T01:34:04.398787+0000","last_undegraded":"2024-08-19T01:34:04.398787+0000","last_fullsized":"2024-08-19T01:34:04.398787+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_clean_scrub_stamp":"2024-08-19T01:32:40.202519+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T12:38:41.398299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"86'31","reported_seq":277,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:34:38.610606+0000","last_change":"2024-08-19T01:32:38.914831+0000","last_active":"2024-08-19T01:34:38.610606+0000","last_peered":"2024-08-19T01:34:38.610606+0000","last_clean":"2024-08-19T01:34:38.610606+0000","last_became_active":"2024-08-19T01:32:38.914326+0000","last_became_peered":"2024-08-19T01:32:38.914326+0000","last_unstale":"2024-08-19T01:34:38.610606+0000","last_undegraded":"2024-08-19T01:34:38.610606+0000","last_fullsized":"2024-08-19T01:34:38.610606+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:32:37.889224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:32:37.889224+0000","last_clean_scrub_stamp":"2024-08-19T01:32:37.889224+0000","objects_scrubbed":0,"log_size":31,"log_dups_size":0,"ondisk_log_size":31,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T02:15:25.263974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":60865,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":224,"num_read_kb":170,"num_write":32,"num_write_kb":71,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"86'6","reported_seq":23,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:34:04.390164+0000","last_change":"2024-08-19T01:32:41.237945+0000","last_active":"2024-08-19T01:34:04.390164+0000","last_peered":"2024-08-19T01:34:04.390164+0000","last_clean":"2024-08-19T01:34:04.390164+0000","last_became_active":"2024-08-19T01:32:41.237362+0000","last_became_peered":"2024-08-19T01:32:41.237362+0000","last_unstale":"2024-08-19T01:34:04.390164+0000","last_undegraded":"2024-08-19T01:34:04.390164+0000","last_fullsized":"2024-08-19T01:34:04.390164+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_clean_scrub_stamp":"2024-08-19T01:32:40.202519+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T07:29:17.180054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"84'6","reported_seq":29,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:33:05.466940+0000","last_change":"2024-08-19T01:32:38.912453+0000","last_active":"2024-08-19T01:33:05.466940+0000","last_peered":"2024-08-19T01:33:05.466940+0000","last_clean":"2024-08-19T01:33:05.466940+0000","last_became_active":"2024-08-19T01:32:38.912051+0000","last_became_peered":"2024-08-19T01:32:38.912051+0000","last_unstale":"2024-08-19T01:33:05.466940+0000","last_undegraded":"2024-08-19T01:33:05.466940+0000","last_fullsized":"2024-08-19T01:33:05.466940+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:32:37.889224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:32:37.889224+0000","last_clean_scrub_stamp":"2024-08-19T01:32:37.889224+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T06:49:58.810134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"86'9","reported_seq":26,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:34:04.391203+0000","last_change":"2024-08-19T01:32:41.236282+0000","last_active":"2024-08-19T01:34:04.391203+0000","last_peered":"2024-08-19T01:34:04.391203+0000","last_clean":"2024-08-19T01:34:04.391203+0000","last_became_active":"2024-08-19T01:32:41.235559+0000","last_became_peered":"2024-08-19T01:32:41.235559+0000","last_unstale":"2024-08-19T01:34:04.391203+0000","last_undegraded":"2024-08-19T01:34:04.391203+0000","last_fullsized":"2024-08-19T01:34:04.391203+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_clean_scrub_stamp":"2024-08-19T01:32:40.202519+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T07:51:52.157094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"84'3","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:33:05.470566+0000","last_change":"2024-08-19T01:32:38.915688+0000","last_active":"2024-08-19T01:33:05.470566+0000","last_peered":"2024-08-19T01:33:05.470566+0000","last_clean":"2024-08-19T01:33:05.470566+0000","last_became_active":"2024-08-19T01:32:38.915297+0000","last_became_peered":"2024-08-19T01:32:38.915297+0000","last_unstale":"2024-08-19T01:33:05.470566+0000","last_undegraded":"2024-08-19T01:33:05.470566+0000","last_fullsized":"2024-08-19T01:33:05.470566+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:32:37.889224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:32:37.889224+0000","last_clean_scrub_stamp":"2024-08-19T01:32:37.889224+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T01:38:12.780588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"86'15","reported_seq":32,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:34:04.395033+0000","last_change":"2024-08-19T01:32:41.233766+0000","last_active":"2024-08-19T01:34:04.395033+0000","last_peered":"2024-08-19T01:34:04.395033+0000","last_clean":"2024-08-19T01:34:04.395033+0000","last_became_active":"2024-08-19T01:32:41.233425+0000","last_became_peered":"2024-08-19T01:32:41.233425+0000","last_unstale":"2024-08-19T01:34:04.395033+0000","last_undegraded":"2024-08-19T01:34:04.395033+0000","last_fullsized":"2024-08-19T01:34:04.395033+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_clean_scrub_stamp":"2024-08-19T01:32:40.202519+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T10:29:15.471639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"86'7","reported_seq":146,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:34:42.493710+0000","last_change":"2024-08-19T01:32:38.912610+0000","last_active":"2024-08-19T01:34:42.493710+0000","last_peered":"2024-08-19T01:34:42.493710+0000","last_clean":"2024-08-19T01:34:42.493710+0000","last_became_active":"2024-08-19T01:32:38.912040+0000","last_became_peered":"2024-08-19T01:32:38.912040+0000","last_unstale":"2024-08-19T01:34:42.493710+0000","last_undegraded":"2024-08-19T01:34:42.493710+0000","last_fullsized":"2024-08-19T01:34:42.493710+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:32:37.889224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:32:37.889224+0000","last_clean_scrub_stamp":"2024-08-19T01:32:37.889224+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T09:26:05.966391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1820,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":118,"num_read_kb":7,"num_write":6,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"86'6","reported_seq":23,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:34:04.393529+0000","last_change":"2024-08-19T01:32:41.235945+0000","last_active":"2024-08-19T01:34:04.393529+0000","last_peered":"2024-08-19T01:34:04.393529+0000","last_clean":"2024-08-19T01:34:04.393529+0000","last_became_active":"2024-08-19T01:32:41.235633+0000","last_became_peered":"2024-08-19T01:32:41.235633+0000","last_unstale":"2024-08-19T01:34:04.393529+0000","last_undegraded":"2024-08-19T01:34:04.393529+0000","last_fullsized":"2024-08-19T01:34:04.393529+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_clean_scrub_stamp":"2024-08-19T01:32:40.202519+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T08:37:37.682175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"84'15","reported_seq":38,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:33:05.471087+0000","last_change":"2024-08-19T01:32:39.052850+0000","last_active":"2024-08-19T01:33:05.471087+0000","last_peered":"2024-08-19T01:33:05.471087+0000","last_clean":"2024-08-19T01:33:05.471087+0000","last_became_active":"2024-08-19T01:32:39.052455+0000","last_became_peered":"2024-08-19T01:32:39.052455+0000","last_unstale":"2024-08-19T01:33:05.471087+0000","last_undegraded":"2024-08-19T01:33:05.471087+0000","last_fullsized":"2024-08-19T01:33:05.471087+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:32:37.889224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:32:37.889224+0000","last_clean_scrub_stamp":"2024-08-19T01:32:37.889224+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T03:25:08.279911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"86'6","reported_seq":29,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:34:04.394643+0000","last_change":"2024-08-19T01:33:05.702575+0000","last_active":"2024-08-19T01:34:04.394643+0000","last_peered":"2024-08-19T01:34:04.394643+0000","last_clean":"2024-08-19T01:34:04.394643+0000","last_became_active":"2024-08-19T01:33:05.702110+0000","last_became_peered":"2024-08-19T01:33:05.702110+0000","last_unstale":"2024-08-19T01:34:04.394643+0000","last_undegraded":"2024-08-19T01:34:04.394643+0000","last_fullsized":"2024-08-19T01:34:04.394643+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_clean_scrub_stamp":"2024-08-19T01:32:40.202519+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T12:18:51.102180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,1],"acting":[5,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"86'3","reported_seq":26,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:33:05.472787+0000","last_change":"2024-08-19T01:32:38.912090+0000","last_active":"2024-08-19T01:33:05.472787+0000","last_peered":"2024-08-19T01:33:05.472787+0000","last_clean":"2024-08-19T01:33:05.472787+0000","last_became_active":"2024-08-19T01:32:38.911753+0000","last_became_peered":"2024-08-19T01:32:38.911753+0000","last_unstale":"2024-08-19T01:33:05.472787+0000","last_undegraded":"2024-08-19T01:33:05.472787+0000","last_fullsized":"2024-08-19T01:33:05.472787+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:32:37.889224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:32:37.889224+0000","last_clean_scrub_stamp":"2024-08-19T01:32:37.889224+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T12:56:42.982062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"86'5","reported_seq":22,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:34:04.387010+0000","last_change":"2024-08-19T01:32:41.230429+0000","last_active":"2024-08-19T01:34:04.387010+0000","last_peered":"2024-08-19T01:34:04.387010+0000","last_clean":"2024-08-19T01:34:04.387010+0000","last_became_active":"2024-08-19T01:32:41.230115+0000","last_became_peered":"2024-08-19T01:32:41.230115+0000","last_unstale":"2024-08-19T01:34:04.387010+0000","last_undegraded":"2024-08-19T01:34:04.387010+0000","last_fullsized":"2024-08-19T01:34:04.387010+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_clean_scrub_stamp":"2024-08-19T01:32:40.202519+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T01:50:51.084130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.e","version":"84'2","reported_seq":24,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:33:05.477826+0000","last_change":"2024-08-19T01:32:38.914557+0000","last_active":"2024-08-19T01:33:05.477826+0000","last_peered":"2024-08-19T01:33:05.477826+0000","last_clean":"2024-08-19T01:33:05.477826+0000","last_became_active":"2024-08-19T01:32:38.914115+0000","last_became_peered":"2024-08-19T01:32:38.914115+0000","last_unstale":"2024-08-19T01:33:05.477826+0000","last_undegraded":"2024-08-19T01:33:05.477826+0000","last_fullsized":"2024-08-19T01:33:05.477826+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:32:37.889224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:32:37.889224+0000","last_clean_scrub_stamp":"2024-08-19T01:32:37.889224+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T06:30:00.523513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"86'9","reported_seq":26,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:34:04.391088+0000","last_change":"2024-08-19T01:32:41.234883+0000","last_active":"2024-08-19T01:34:04.391088+0000","last_peered":"2024-08-19T01:34:04.391088+0000","last_clean":"2024-08-19T01:34:04.391088+0000","last_became_active":"2024-08-19T01:32:41.233901+0000","last_became_peered":"2024-08-19T01:32:41.233901+0000","last_unstale":"2024-08-19T01:34:04.391088+0000","last_undegraded":"2024-08-19T01:34:04.391088+0000","last_fullsized":"2024-08-19T01:34:04.391088+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_clean_scrub_stamp":"2024-08-19T01:32:40.202519+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T12:08:46.305259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"84'1","reported_seq":24,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:33:05.470777+0000","last_change":"2024-08-19T01:32:38.913441+0000","last_active":"2024-08-19T01:33:05.470777+0000","last_peered":"2024-08-19T01:33:05.470777+0000","last_clean":"2024-08-19T01:33:05.470777+0000","last_became_active":"2024-08-19T01:32:38.913115+0000","last_became_peered":"2024-08-19T01:32:38.913115+0000","last_unstale":"2024-08-19T01:33:05.470777+0000","last_undegraded":"2024-08-19T01:33:05.470777+0000","last_fullsized":"2024-08-19T01:33:05.470777+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:32:37.889224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:32:37.889224+0000","last_clean_scrub_stamp":"2024-08-19T01:32:37.889224+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T11:43:15.290980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"86'7","reported_seq":24,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:34:04.391219+0000","last_change":"2024-08-19T01:32:41.235370+0000","last_active":"2024-08-19T01:34:04.391219+0000","last_peered":"2024-08-19T01:34:04.391219+0000","last_clean":"2024-08-19T01:34:04.391219+0000","last_became_active":"2024-08-19T01:32:41.235012+0000","last_became_peered":"2024-08-19T01:32:41.235012+0000","last_unstale":"2024-08-19T01:34:04.391219+0000","last_undegraded":"2024-08-19T01:34:04.391219+0000","last_fullsized":"2024-08-19T01:34:04.391219+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_clean_scrub_stamp":"2024-08-19T01:32:40.202519+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T02:18:31.134974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"84'3","reported_seq":26,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:33:05.466872+0000","last_change":"2024-08-19T01:32:39.245560+0000","last_active":"2024-08-19T01:33:05.466872+0000","last_peered":"2024-08-19T01:33:05.466872+0000","last_clean":"2024-08-19T01:33:05.466872+0000","last_became_active":"2024-08-19T01:32:39.245118+0000","last_became_peered":"2024-08-19T01:32:39.245118+0000","last_unstale":"2024-08-19T01:33:05.466872+0000","last_undegraded":"2024-08-19T01:33:05.466872+0000","last_fullsized":"2024-08-19T01:33:05.466872+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:32:37.889224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:32:37.889224+0000","last_clean_scrub_stamp":"2024-08-19T01:32:37.889224+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T02:18:24.605709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"86'7","reported_seq":24,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:34:04.396513+0000","last_change":"2024-08-19T01:32:41.234452+0000","last_active":"2024-08-19T01:34:04.396513+0000","last_peered":"2024-08-19T01:34:04.396513+0000","last_clean":"2024-08-19T01:34:04.396513+0000","last_became_active":"2024-08-19T01:32:41.233852+0000","last_became_peered":"2024-08-19T01:32:41.233852+0000","last_unstale":"2024-08-19T01:34:04.396513+0000","last_undegraded":"2024-08-19T01:34:04.396513+0000","last_fullsized":"2024-08-19T01:34:04.396513+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_clean_scrub_stamp":"2024-08-19T01:32:40.202519+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T06:18:34.457239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"22'137","reported_seq":339,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:33:05.473281+0000","last_change":"2024-08-19T01:31:27.659394+0000","last_active":"2024-08-19T01:33:05.473281+0000","last_peered":"2024-08-19T01:33:05.473281+0000","last_clean":"2024-08-19T01:33:05.473281+0000","last_became_active":"2024-08-19T01:31:27.659007+0000","last_became_peered":"2024-08-19T01:31:27.659007+0000","last_unstale":"2024-08-19T01:33:05.473281+0000","last_undegraded":"2024-08-19T01:33:05.473281+0000","last_fullsized":"2024-08-19T01:33:05.473281+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:28:16.912251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:28:16.912251+0000","last_clean_scrub_stamp":"2024-08-19T01:28:16.912251+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T05:53:01.660619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"84'3","reported_seq":27,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:33:05.470433+0000","last_change":"2024-08-19T01:32:38.913408+0000","last_active":"2024-08-19T01:33:05.470433+0000","last_peered":"2024-08-19T01:33:05.470433+0000","last_clean":"2024-08-19T01:33:05.470433+0000","last_became_active":"2024-08-19T01:32:38.912242+0000","last_became_peered":"2024-08-19T01:32:38.912242+0000","last_unstale":"2024-08-19T01:33:05.470433+0000","last_undegraded":"2024-08-19T01:33:05.470433+0000","last_fullsized":"2024-08-19T01:33:05.470433+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:32:37.889224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:32:37.889224+0000","last_clean_scrub_stamp":"2024-08-19T01:32:37.889224+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T07:26:48.121366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2","version":"86'8","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:34:04.390396+0000","last_change":"2024-08-19T01:32:41.232611+0000","last_active":"2024-08-19T01:34:04.390396+0000","last_peered":"2024-08-19T01:34:04.390396+0000","last_clean":"2024-08-19T01:34:04.390396+0000","last_became_active":"2024-08-19T01:32:41.232261+0000","last_became_peered":"2024-08-19T01:32:41.232261+0000","last_unstale":"2024-08-19T01:34:04.390396+0000","last_undegraded":"2024-08-19T01:34:04.390396+0000","last_fullsized":"2024-08-19T01:34:04.390396+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_clean_scrub_stamp":"2024-08-19T01:32:40.202519+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T08:04:16.793933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"84'3","reported_seq":26,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:33:05.473161+0000","last_change":"2024-08-19T01:32:38.912765+0000","last_active":"2024-08-19T01:33:05.473161+0000","last_peered":"2024-08-19T01:33:05.473161+0000","last_clean":"2024-08-19T01:33:05.473161+0000","last_became_active":"2024-08-19T01:32:38.912274+0000","last_became_peered":"2024-08-19T01:32:38.912274+0000","last_unstale":"2024-08-19T01:33:05.473161+0000","last_undegraded":"2024-08-19T01:33:05.473161+0000","last_fullsized":"2024-08-19T01:33:05.473161+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:32:37.889224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:32:37.889224+0000","last_clean_scrub_stamp":"2024-08-19T01:32:37.889224+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T09:57:49.714827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"86'5","reported_seq":22,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:34:04.390678+0000","last_change":"2024-08-19T01:32:41.224449+0000","last_active":"2024-08-19T01:34:04.390678+0000","last_peered":"2024-08-19T01:34:04.390678+0000","last_clean":"2024-08-19T01:34:04.390678+0000","last_became_active":"2024-08-19T01:32:41.224064+0000","last_became_peered":"2024-08-19T01:32:41.224064+0000","last_unstale":"2024-08-19T01:34:04.390678+0000","last_undegraded":"2024-08-19T01:34:04.390678+0000","last_fullsized":"2024-08-19T01:34:04.390678+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_clean_scrub_stamp":"2024-08-19T01:32:40.202519+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T08:53:33.948504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"86'6","reported_seq":137,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:34:39.566576+0000","last_change":"2024-08-19T01:32:39.050869+0000","last_active":"2024-08-19T01:34:39.566576+0000","last_peered":"2024-08-19T01:34:39.566576+0000","last_clean":"2024-08-19T01:34:39.566576+0000","last_became_active":"2024-08-19T01:32:39.050529+0000","last_became_peered":"2024-08-19T01:32:39.050529+0000","last_unstale":"2024-08-19T01:34:39.566576+0000","last_undegraded":"2024-08-19T01:34:39.566576+0000","last_fullsized":"2024-08-19T01:34:39.566576+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:32:37.889224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:32:37.889224+0000","last_clean_scrub_stamp":"2024-08-19T01:32:37.889224+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T12:52:52.788360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":109,"num_read_kb":109,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"86'8","reported_seq":26,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:34:04.387624+0000","last_change":"2024-08-19T01:32:41.237093+0000","last_active":"2024-08-19T01:34:04.387624+0000","last_peered":"2024-08-19T01:34:04.387624+0000","last_clean":"2024-08-19T01:34:04.387624+0000","last_became_active":"2024-08-19T01:32:41.236738+0000","last_became_peered":"2024-08-19T01:32:41.236738+0000","last_unstale":"2024-08-19T01:34:04.387624+0000","last_undegraded":"2024-08-19T01:34:04.387624+0000","last_fullsized":"2024-08-19T01:34:04.387624+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_clean_scrub_stamp":"2024-08-19T01:32:40.202519+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T08:37:33.117344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"84'1","reported_seq":23,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:33:05.473189+0000","last_change":"2024-08-19T01:32:38.915010+0000","last_active":"2024-08-19T01:33:05.473189+0000","last_peered":"2024-08-19T01:33:05.473189+0000","last_clean":"2024-08-19T01:33:05.473189+0000","last_became_active":"2024-08-19T01:32:38.914687+0000","last_became_peered":"2024-08-19T01:32:38.914687+0000","last_unstale":"2024-08-19T01:33:05.473189+0000","last_undegraded":"2024-08-19T01:33:05.473189+0000","last_fullsized":"2024-08-19T01:33:05.473189+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:32:37.889224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:32:37.889224+0000","last_clean_scrub_stamp":"2024-08-19T01:32:37.889224+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T04:46:07.881778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"86'7","reported_seq":24,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:34:04.397148+0000","last_change":"2024-08-19T01:32:41.244069+0000","last_active":"2024-08-19T01:34:04.397148+0000","last_peered":"2024-08-19T01:34:04.397148+0000","last_clean":"2024-08-19T01:34:04.397148+0000","last_became_active":"2024-08-19T01:32:41.243482+0000","last_became_peered":"2024-08-19T01:32:41.243482+0000","last_unstale":"2024-08-19T01:34:04.397148+0000","last_undegraded":"2024-08-19T01:34:04.397148+0000","last_fullsized":"2024-08-19T01:34:04.397148+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_clean_scrub_stamp":"2024-08-19T01:32:40.202519+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T02:08:28.044421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":22,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:33:05.466068+0000","last_change":"2024-08-19T01:32:39.051383+0000","last_active":"2024-08-19T01:33:05.466068+0000","last_peered":"2024-08-19T01:33:05.466068+0000","last_clean":"2024-08-19T01:33:05.466068+0000","last_became_active":"2024-08-19T01:32:39.050970+0000","last_became_peered":"2024-08-19T01:32:39.050970+0000","last_unstale":"2024-08-19T01:33:05.466068+0000","last_undegraded":"2024-08-19T01:33:05.466068+0000","last_fullsized":"2024-08-19T01:33:05.466068+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:32:37.889224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:32:37.889224+0000","last_clean_scrub_stamp":"2024-08-19T01:32:37.889224+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T11:31:30.461651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"86'5","reported_seq":22,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:34:04.395486+0000","last_change":"2024-08-19T01:32:41.232656+0000","last_active":"2024-08-19T01:34:04.395486+0000","last_peered":"2024-08-19T01:34:04.395486+0000","last_clean":"2024-08-19T01:34:04.395486+0000","last_became_active":"2024-08-19T01:32:41.232274+0000","last_became_peered":"2024-08-19T01:32:41.232274+0000","last_unstale":"2024-08-19T01:34:04.395486+0000","last_undegraded":"2024-08-19T01:34:04.395486+0000","last_fullsized":"2024-08-19T01:34:04.395486+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_clean_scrub_stamp":"2024-08-19T01:32:40.202519+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T03:38:40.695652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.5","version":"84'1","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:33:05.472646+0000","last_change":"2024-08-19T01:32:39.050809+0000","last_active":"2024-08-19T01:33:05.472646+0000","last_peered":"2024-08-19T01:33:05.472646+0000","last_clean":"2024-08-19T01:33:05.472646+0000","last_became_active":"2024-08-19T01:32:39.050434+0000","last_became_peered":"2024-08-19T01:32:39.050434+0000","last_unstale":"2024-08-19T01:33:05.472646+0000","last_undegraded":"2024-08-19T01:33:05.472646+0000","last_fullsized":"2024-08-19T01:33:05.472646+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:32:37.889224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:32:37.889224+0000","last_clean_scrub_stamp":"2024-08-19T01:32:37.889224+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T13:04:28.828117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"86'9","reported_seq":26,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:34:04.390057+0000","last_change":"2024-08-19T01:32:41.228485+0000","last_active":"2024-08-19T01:34:04.390057+0000","last_peered":"2024-08-19T01:34:04.390057+0000","last_clean":"2024-08-19T01:34:04.390057+0000","last_became_active":"2024-08-19T01:32:41.227238+0000","last_became_peered":"2024-08-19T01:32:41.227238+0000","last_unstale":"2024-08-19T01:34:04.390057+0000","last_undegraded":"2024-08-19T01:34:04.390057+0000","last_fullsized":"2024-08-19T01:34:04.390057+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_clean_scrub_stamp":"2024-08-19T01:32:40.202519+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T10:55:55.939382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"86'13","reported_seq":35,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:34:05.234056+0000","last_change":"2024-08-19T01:32:38.915994+0000","last_active":"2024-08-19T01:34:05.234056+0000","last_peered":"2024-08-19T01:34:05.234056+0000","last_clean":"2024-08-19T01:34:05.234056+0000","last_became_active":"2024-08-19T01:32:38.914714+0000","last_became_peered":"2024-08-19T01:32:38.914714+0000","last_unstale":"2024-08-19T01:34:05.234056+0000","last_undegraded":"2024-08-19T01:34:05.234056+0000","last_fullsized":"2024-08-19T01:34:05.234056+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:32:37.889224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:32:37.889224+0000","last_clean_scrub_stamp":"2024-08-19T01:32:37.889224+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T05:38:07.274274+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":25,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"86'5","reported_seq":22,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:34:04.386255+0000","last_change":"2024-08-19T01:32:41.236611+0000","last_active":"2024-08-19T01:34:04.386255+0000","last_peered":"2024-08-19T01:34:04.386255+0000","last_clean":"2024-08-19T01:34:04.386255+0000","last_became_active":"2024-08-19T01:32:41.233821+0000","last_became_peered":"2024-08-19T01:32:41.233821+0000","last_unstale":"2024-08-19T01:34:04.386255+0000","last_undegraded":"2024-08-19T01:34:04.386255+0000","last_fullsized":"2024-08-19T01:34:04.386255+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_clean_scrub_stamp":"2024-08-19T01:32:40.202519+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T11:34:07.879034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"84'4","reported_seq":26,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:33:05.470567+0000","last_change":"2024-08-19T01:32:39.245411+0000","last_active":"2024-08-19T01:33:05.470567+0000","last_peered":"2024-08-19T01:33:05.470567+0000","last_clean":"2024-08-19T01:33:05.470567+0000","last_became_active":"2024-08-19T01:32:39.244980+0000","last_became_peered":"2024-08-19T01:32:39.244980+0000","last_unstale":"2024-08-19T01:33:05.470567+0000","last_undegraded":"2024-08-19T01:33:05.470567+0000","last_fullsized":"2024-08-19T01:33:05.470567+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:32:37.889224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:32:37.889224+0000","last_clean_scrub_stamp":"2024-08-19T01:32:37.889224+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T02:06:57.878051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"86'7","reported_seq":30,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:34:04.395910+0000","last_change":"2024-08-19T01:33:05.478955+0000","last_active":"2024-08-19T01:34:04.395910+0000","last_peered":"2024-08-19T01:34:04.395910+0000","last_clean":"2024-08-19T01:34:04.395910+0000","last_became_active":"2024-08-19T01:33:05.478446+0000","last_became_peered":"2024-08-19T01:33:05.478446+0000","last_unstale":"2024-08-19T01:34:04.395910+0000","last_undegraded":"2024-08-19T01:34:04.395910+0000","last_fullsized":"2024-08-19T01:34:04.395910+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_clean_scrub_stamp":"2024-08-19T01:32:40.202519+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T11:58:24.285438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,6],"acting":[0,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"84'2","reported_seq":24,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:33:05.701553+0000","last_change":"2024-08-19T01:32:39.050100+0000","last_active":"2024-08-19T01:33:05.701553+0000","last_peered":"2024-08-19T01:33:05.701553+0000","last_clean":"2024-08-19T01:33:05.701553+0000","last_became_active":"2024-08-19T01:32:39.049807+0000","last_became_peered":"2024-08-19T01:32:39.049807+0000","last_unstale":"2024-08-19T01:33:05.701553+0000","last_undegraded":"2024-08-19T01:33:05.701553+0000","last_fullsized":"2024-08-19T01:33:05.701553+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:32:37.889224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:32:37.889224+0000","last_clean_scrub_stamp":"2024-08-19T01:32:37.889224+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T07:06:22.361609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"86'2","reported_seq":19,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:34:04.355286+0000","last_change":"2024-08-19T01:32:41.228982+0000","last_active":"2024-08-19T01:34:04.355286+0000","last_peered":"2024-08-19T01:34:04.355286+0000","last_clean":"2024-08-19T01:34:04.355286+0000","last_became_active":"2024-08-19T01:32:41.228562+0000","last_became_peered":"2024-08-19T01:32:41.228562+0000","last_unstale":"2024-08-19T01:34:04.355286+0000","last_undegraded":"2024-08-19T01:34:04.355286+0000","last_fullsized":"2024-08-19T01:34:04.355286+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_clean_scrub_stamp":"2024-08-19T01:32:40.202519+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T03:14:42.480359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"84'3","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:33:05.471722+0000","last_change":"2024-08-19T01:32:38.913677+0000","last_active":"2024-08-19T01:33:05.471722+0000","last_peered":"2024-08-19T01:33:05.471722+0000","last_clean":"2024-08-19T01:33:05.471722+0000","last_became_active":"2024-08-19T01:32:38.913091+0000","last_became_peered":"2024-08-19T01:32:38.913091+0000","last_unstale":"2024-08-19T01:33:05.471722+0000","last_undegraded":"2024-08-19T01:33:05.471722+0000","last_fullsized":"2024-08-19T01:33:05.471722+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:32:37.889224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:32:37.889224+0000","last_clean_scrub_stamp":"2024-08-19T01:32:37.889224+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T12:39:16.489246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"86'9","reported_seq":26,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:34:04.395672+0000","last_change":"2024-08-19T01:32:41.239380+0000","last_active":"2024-08-19T01:34:04.395672+0000","last_peered":"2024-08-19T01:34:04.395672+0000","last_clean":"2024-08-19T01:34:04.395672+0000","last_became_active":"2024-08-19T01:32:41.239060+0000","last_became_peered":"2024-08-19T01:32:41.239060+0000","last_unstale":"2024-08-19T01:34:04.395672+0000","last_undegraded":"2024-08-19T01:34:04.395672+0000","last_fullsized":"2024-08-19T01:34:04.395672+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_clean_scrub_stamp":"2024-08-19T01:32:40.202519+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T02:21:59.630705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"86'2","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:33:07.449044+0000","last_change":"2024-08-19T01:33:05.478578+0000","last_active":"2024-08-19T01:33:07.449044+0000","last_peered":"2024-08-19T01:33:07.449044+0000","last_clean":"2024-08-19T01:33:07.449044+0000","last_became_active":"2024-08-19T01:33:05.473433+0000","last_became_peered":"2024-08-19T01:33:05.473433+0000","last_unstale":"2024-08-19T01:33:07.449044+0000","last_undegraded":"2024-08-19T01:33:07.449044+0000","last_fullsized":"2024-08-19T01:33:07.449044+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:32:37.889224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:32:37.889224+0000","last_clean_scrub_stamp":"2024-08-19T01:32:37.889224+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T05:06:29.975613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,3,8],"acting":[11,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.b","version":"86'8","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:34:04.393696+0000","last_change":"2024-08-19T01:32:41.236012+0000","last_active":"2024-08-19T01:34:04.393696+0000","last_peered":"2024-08-19T01:34:04.393696+0000","last_clean":"2024-08-19T01:34:04.393696+0000","last_became_active":"2024-08-19T01:32:41.235724+0000","last_became_peered":"2024-08-19T01:32:41.235724+0000","last_unstale":"2024-08-19T01:34:04.393696+0000","last_undegraded":"2024-08-19T01:34:04.393696+0000","last_fullsized":"2024-08-19T01:34:04.393696+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_clean_scrub_stamp":"2024-08-19T01:32:40.202519+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T08:45:08.865144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"84'4","reported_seq":27,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:33:05.471339+0000","last_change":"2024-08-19T01:32:38.915170+0000","last_active":"2024-08-19T01:33:05.471339+0000","last_peered":"2024-08-19T01:33:05.471339+0000","last_clean":"2024-08-19T01:33:05.471339+0000","last_became_active":"2024-08-19T01:32:38.914812+0000","last_became_peered":"2024-08-19T01:32:38.914812+0000","last_unstale":"2024-08-19T01:33:05.471339+0000","last_undegraded":"2024-08-19T01:33:05.471339+0000","last_fullsized":"2024-08-19T01:33:05.471339+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:32:37.889224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:32:37.889224+0000","last_clean_scrub_stamp":"2024-08-19T01:32:37.889224+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T11:46:39.409065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"86'5","reported_seq":22,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:34:04.396323+0000","last_change":"2024-08-19T01:32:41.234133+0000","last_active":"2024-08-19T01:34:04.396323+0000","last_peered":"2024-08-19T01:34:04.396323+0000","last_clean":"2024-08-19T01:34:04.396323+0000","last_became_active":"2024-08-19T01:32:41.233492+0000","last_became_peered":"2024-08-19T01:32:41.233492+0000","last_unstale":"2024-08-19T01:34:04.396323+0000","last_undegraded":"2024-08-19T01:34:04.396323+0000","last_fullsized":"2024-08-19T01:34:04.396323+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_clean_scrub_stamp":"2024-08-19T01:32:40.202519+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T06:11:12.587678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"86'15","reported_seq":32,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:34:04.395111+0000","last_change":"2024-08-19T01:32:41.232628+0000","last_active":"2024-08-19T01:34:04.395111+0000","last_peered":"2024-08-19T01:34:04.395111+0000","last_clean":"2024-08-19T01:34:04.395111+0000","last_became_active":"2024-08-19T01:32:41.232295+0000","last_became_peered":"2024-08-19T01:32:41.232295+0000","last_unstale":"2024-08-19T01:34:04.395111+0000","last_undegraded":"2024-08-19T01:34:04.395111+0000","last_fullsized":"2024-08-19T01:34:04.395111+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_clean_scrub_stamp":"2024-08-19T01:32:40.202519+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T02:01:09.490553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"84'2","reported_seq":24,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:33:05.472666+0000","last_change":"2024-08-19T01:32:38.912000+0000","last_active":"2024-08-19T01:33:05.472666+0000","last_peered":"2024-08-19T01:33:05.472666+0000","last_clean":"2024-08-19T01:33:05.472666+0000","last_became_active":"2024-08-19T01:32:38.911535+0000","last_became_peered":"2024-08-19T01:32:38.911535+0000","last_unstale":"2024-08-19T01:33:05.472666+0000","last_undegraded":"2024-08-19T01:33:05.472666+0000","last_fullsized":"2024-08-19T01:33:05.472666+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:32:37.889224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:32:37.889224+0000","last_clean_scrub_stamp":"2024-08-19T01:32:37.889224+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T07:12:49.358384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"86'9","reported_seq":26,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:34:04.399042+0000","last_change":"2024-08-19T01:32:41.233507+0000","last_active":"2024-08-19T01:34:04.399042+0000","last_peered":"2024-08-19T01:34:04.399042+0000","last_clean":"2024-08-19T01:34:04.399042+0000","last_became_active":"2024-08-19T01:32:41.233179+0000","last_became_peered":"2024-08-19T01:32:41.233179+0000","last_unstale":"2024-08-19T01:34:04.399042+0000","last_undegraded":"2024-08-19T01:34:04.399042+0000","last_fullsized":"2024-08-19T01:34:04.399042+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_clean_scrub_stamp":"2024-08-19T01:32:40.202519+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T11:47:44.780859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1d","version":"84'4","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:33:05.483750+0000","last_change":"2024-08-19T01:33:05.483322+0000","last_active":"2024-08-19T01:33:05.483750+0000","last_peered":"2024-08-19T01:33:05.483750+0000","last_clean":"2024-08-19T01:33:05.483750+0000","last_became_active":"2024-08-19T01:33:05.475496+0000","last_became_peered":"2024-08-19T01:33:05.475496+0000","last_unstale":"2024-08-19T01:33:05.483750+0000","last_undegraded":"2024-08-19T01:33:05.483750+0000","last_fullsized":"2024-08-19T01:33:05.483750+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:32:37.889224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:32:37.889224+0000","last_clean_scrub_stamp":"2024-08-19T01:32:37.889224+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T01:57:48.143495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":10,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,7],"acting":[11,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1f","version":"86'6","reported_seq":23,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:34:04.399271+0000","last_change":"2024-08-19T01:32:41.244641+0000","last_active":"2024-08-19T01:34:04.399271+0000","last_peered":"2024-08-19T01:34:04.399271+0000","last_clean":"2024-08-19T01:34:04.399271+0000","last_became_active":"2024-08-19T01:32:41.244311+0000","last_became_peered":"2024-08-19T01:32:41.244311+0000","last_unstale":"2024-08-19T01:34:04.399271+0000","last_undegraded":"2024-08-19T01:34:04.399271+0000","last_fullsized":"2024-08-19T01:34:04.399271+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_clean_scrub_stamp":"2024-08-19T01:32:40.202519+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T10:39:12.591092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"84'2","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:33:05.472403+0000","last_change":"2024-08-19T01:32:39.051314+0000","last_active":"2024-08-19T01:33:05.472403+0000","last_peered":"2024-08-19T01:33:05.472403+0000","last_clean":"2024-08-19T01:33:05.472403+0000","last_became_active":"2024-08-19T01:32:39.050998+0000","last_became_peered":"2024-08-19T01:32:39.050998+0000","last_unstale":"2024-08-19T01:33:05.472403+0000","last_undegraded":"2024-08-19T01:33:05.472403+0000","last_fullsized":"2024-08-19T01:33:05.472403+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:32:37.889224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:32:37.889224+0000","last_clean_scrub_stamp":"2024-08-19T01:32:37.889224+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T09:21:15.689149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"86'5","reported_seq":23,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:34:04.390896+0000","last_change":"2024-08-19T01:32:41.234762+0000","last_active":"2024-08-19T01:34:04.390896+0000","last_peered":"2024-08-19T01:34:04.390896+0000","last_clean":"2024-08-19T01:34:04.390896+0000","last_became_active":"2024-08-19T01:32:41.234471+0000","last_became_peered":"2024-08-19T01:32:41.234471+0000","last_unstale":"2024-08-19T01:34:04.390896+0000","last_undegraded":"2024-08-19T01:34:04.390896+0000","last_fullsized":"2024-08-19T01:34:04.390896+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_clean_scrub_stamp":"2024-08-19T01:32:40.202519+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T08:34:33.110938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"84'9","reported_seq":32,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:33:05.466470+0000","last_change":"2024-08-19T01:32:38.917068+0000","last_active":"2024-08-19T01:33:05.466470+0000","last_peered":"2024-08-19T01:33:05.466470+0000","last_clean":"2024-08-19T01:33:05.466470+0000","last_became_active":"2024-08-19T01:32:38.915399+0000","last_became_peered":"2024-08-19T01:32:38.915399+0000","last_unstale":"2024-08-19T01:33:05.466470+0000","last_undegraded":"2024-08-19T01:33:05.466470+0000","last_fullsized":"2024-08-19T01:33:05.466470+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:32:37.889224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:32:37.889224+0000","last_clean_scrub_stamp":"2024-08-19T01:32:37.889224+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T04:17:25.061208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":502,"ondisk_log_size":502,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":66628,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":685,"num_read_kb":411,"num_write":183,"num_write_kb":124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":1012,"num_keys_recovered":10,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":421888,"data_stored":204198,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":154,"ondisk_log_size":154,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4751360,"data_stored":4722944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8}],"osd_stats":[{"osd":11,"up_from":78,"seq":335007449126,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32092,"kb_used_data":916,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713060,"statfs":{"total":95995035648,"available":95962173440,"internally_reserved":0,"allocated":937984,"data_stored":451051,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":71,"seq":304942678058,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28612,"kb_used_data":1596,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716540,"statfs":{"total":95995035648,"available":95965736960,"internally_reserved":0,"allocated":1634304,"data_stored":1103928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":66,"seq":283467841583,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32108,"kb_used_data":932,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713044,"statfs":{"total":95995035648,"available":95962157056,"internally_reserved":0,"allocated":954368,"data_stored":451287,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":58,"seq":249108103221,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32108,"kb_used_data":932,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713044,"statfs":{"total":95995035648,"available":95962157056,"internally_reserved":0,"allocated":954368,"data_stored":451207,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":51,"seq":219043332153,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32688,"kb_used_data":1512,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712464,"statfs":{"total":95995035648,"available":95961563136,"internally_reserved":0,"allocated":1548288,"data_stored":1044119,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":46,"seq":197568495678,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27936,"kb_used_data":920,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717216,"statfs":{"total":95995035648,"available":95966429184,"internally_reserved":0,"allocated":942080,"data_stored":450676,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":38,"seq":163208757315,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28540,"kb_used_data":1524,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716612,"statfs":{"total":95995035648,"available":95965810688,"internally_reserved":0,"allocated":1560576,"data_stored":1043351,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":33,"seq":141733920839,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32108,"kb_used_data":932,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713044,"statfs":{"total":95995035648,"available":95962157056,"internally_reserved":0,"allocated":954368,"data_stored":453888,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182476,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27952,"kb_used_data":936,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717200,"statfs":{"total":95995035648,"available":95966412800,"internally_reserved":0,"allocated":958464,"data_stored":450987,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411409,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36332,"kb_used_data":996,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708820,"statfs":{"total":95995035648,"available":95957831680,"internally_reserved":0,"allocated":1019904,"data_stored":512062,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574932,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28000,"kb_used_data":984,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717152,"statfs":{"total":95995035648,"available":95966363648,"internally_reserved":0,"allocated":1007616,"data_stored":516349,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738458,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36276,"kb_used_data":940,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708876,"statfs":{"total":95995035648,"available":95957889024,"internally_reserved":0,"allocated":962560,"data_stored":451947,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":65880,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":86016,"data_stored":61593,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":3419,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":2397,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":3945,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":102400,"data_stored":62974,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-08-19T01:34:46.267 INFO:tasks.fwd_scrub.ceph_manager:clean! 2024-08-19T01:34:46.267 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-08-19T01:34:46.425 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:46.425+0000 7f4f9f000640 1 -- 172.21.15.83:0/3677042751 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4f98102d80 msgr2=0x7f4f98103180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:34:46.426 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:46.425+0000 7f4f9f000640 1 --2- 172.21.15.83:0/3677042751 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4f98102d80 0x7f4f98103180 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f4f8c009900 tx=0x7f4f8c02a3f0 comp rx=0 tx=0).stop 2024-08-19T01:34:46.426 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:46.425+0000 7f4f9f000640 1 -- 172.21.15.83:0/3677042751 shutdown_connections 2024-08-19T01:34:46.426 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:46.425+0000 7f4f9f000640 1 --2- 172.21.15.83:0/3677042751 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4f98101c70 0x7f4f98109ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:46.426 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:46.425+0000 7f4f9f000640 1 --2- 172.21.15.83:0/3677042751 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4f981012b0 0x7f4f98101730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:46.426 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:46.425+0000 7f4f9f000640 1 --2- 172.21.15.83:0/3677042751 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4f98102d80 0x7f4f98103180 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:46.426 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:46.425+0000 7f4f9f000640 1 -- 172.21.15.83:0/3677042751 >> 172.21.15.83:0/3677042751 conn(0x7f4f980fce40 msgr2=0x7f4f980ff260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:34:46.426 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:46.425+0000 7f4f9f000640 1 -- 172.21.15.83:0/3677042751 shutdown_connections 2024-08-19T01:34:46.426 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:46.425+0000 7f4f9f000640 1 -- 172.21.15.83:0/3677042751 wait complete. 2024-08-19T01:34:46.429 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:46.426+0000 7f4f9f000640 1 Processor -- start 2024-08-19T01:34:46.429 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:46.426+0000 7f4f9f000640 1 -- start start 2024-08-19T01:34:46.429 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:46.426+0000 7f4f9f000640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4f981012b0 0x7f4f98199d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:34:46.429 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:46.426+0000 7f4f9f000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4f98101c70 0x7f4f9819a2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:34:46.430 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:46.426+0000 7f4f9f000640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4f98102d80 0x7f4f981a1310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:34:46.430 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:46.426+0000 7f4f9f000640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f4f9810d420 con 0x7f4f98101c70 2024-08-19T01:34:46.430 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:46.426+0000 7f4f9f000640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f4f9810d2a0 con 0x7f4f981012b0 2024-08-19T01:34:46.430 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:46.426+0000 7f4f9f000640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f4f9810d5a0 con 0x7f4f98102d80 2024-08-19T01:34:46.430 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:46.426+0000 7f4f9d200640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4f98101c70 0x7f4f9819a2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:34:46.430 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:46.427+0000 7f4f9dc00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4f981012b0 0x7f4f98199d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:34:46.430 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:46.427+0000 7f4f9d200640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4f98101c70 0x7f4f9819a2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:57870/0 (socket says 172.21.15.83:57870) 2024-08-19T01:34:46.430 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:46.427+0000 7f4f9d200640 1 -- 172.21.15.83:0/130211398 learned_addr learned my addr 172.21.15.83:0/130211398 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-08-19T01:34:46.430 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:46.427+0000 7f4f9e600640 1 --2- 172.21.15.83:0/130211398 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4f98102d80 0x7f4f981a1310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:34:46.430 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:46.427+0000 7f4f9dc00640 1 -- 172.21.15.83:0/130211398 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4f98102d80 msgr2=0x7f4f981a1310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:34:46.430 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:46.427+0000 7f4f9dc00640 1 --2- 172.21.15.83:0/130211398 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4f98102d80 0x7f4f981a1310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:46.430 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:46.427+0000 7f4f9dc00640 1 -- 172.21.15.83:0/130211398 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4f98101c70 msgr2=0x7f4f9819a2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:34:46.431 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:46.427+0000 7f4f9dc00640 1 --2- 172.21.15.83:0/130211398 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4f98101c70 0x7f4f9819a2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:46.431 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:46.427+0000 7f4f9dc00640 1 -- 172.21.15.83:0/130211398 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4f981a1980 con 0x7f4f981012b0 2024-08-19T01:34:46.431 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:46.427+0000 7f4f9e600640 1 --2- 172.21.15.83:0/130211398 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4f98102d80 0x7f4f981a1310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-19T01:34:46.431 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:46.427+0000 7f4f9dc00640 1 --2- 172.21.15.83:0/130211398 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4f981012b0 0x7f4f98199d60 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f4f8c009230 tx=0x7f4f8c004270 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:34:46.431 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:46.428+0000 7f4f96000640 1 -- 172.21.15.83:0/130211398 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f8c03f070 con 0x7f4f981012b0 2024-08-19T01:34:46.431 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:46.428+0000 7f4f9f000640 1 -- 172.21.15.83:0/130211398 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4f981a1c10 con 0x7f4f981012b0 2024-08-19T01:34:46.431 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:46.428+0000 7f4f96000640 1 -- 172.21.15.83:0/130211398 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4f8c003cd0 con 0x7f4f981012b0 2024-08-19T01:34:46.431 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:46.428+0000 7f4f96000640 1 -- 172.21.15.83:0/130211398 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f8c035070 con 0x7f4f981012b0 2024-08-19T01:34:46.431 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:46.428+0000 7f4f9f000640 1 -- 172.21.15.83:0/130211398 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4f981a20d0 con 0x7f4f981012b0 2024-08-19T01:34:46.431 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:46.429+0000 7f4f9f000640 1 -- 172.21.15.83:0/130211398 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4f98101730 con 0x7f4f981012b0 2024-08-19T01:34:46.431 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:46.431+0000 7f4f96000640 1 -- 172.21.15.83:0/130211398 <== mon.2 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 19) ==== 137884+0+0 (secure 0 0 0) 0x7f4f8c03b460 con 0x7f4f981012b0 2024-08-19T01:34:46.432 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:46.432+0000 7f4f96000640 1 --2- 172.21.15.83:0/130211398 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f4f800a2c20 0x7f4f800a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:34:46.432 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:46.432+0000 7f4f96000640 1 -- 172.21.15.83:0/130211398 <== mon.2 v2:172.21.15.117:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8269+0+0 (secure 0 0 0) 0x7f4f8c0ef980 con 0x7f4f981012b0 2024-08-19T01:34:46.432 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:46.432+0000 7f4f9d200640 1 --2- 172.21.15.83:0/130211398 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f4f800a2c20 0x7f4f800a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:34:46.433 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:46.433+0000 7f4f9d200640 1 --2- 172.21.15.83:0/130211398 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f4f800a2c20 0x7f4f800a50e0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f4f840095f0 tx=0x7f4f84005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:34:46.434 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:46.434+0000 7f4f96000640 1 -- 172.21.15.83:0/130211398 <== mon.2 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7f4f8c0bc0b0 con 0x7f4f981012b0 2024-08-19T01:34:46.617 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:46.616+0000 7f4f9f000640 1 -- 172.21.15.83:0/130211398 --> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f4f98109780 con 0x7f4f800a2c20 2024-08-19T01:34:46.623 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:46.623+0000 7f4f96000640 1 -- 172.21.15.83:0/130211398 <== mgr.14150 v2:172.21.15.83:6800/683740916 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258845 (secure 0 0 0) 0x7f4f98109780 con 0x7f4f800a2c20 2024-08-19T01:34:46.627 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-08-19T01:34:46.628 INFO:teuthology.orchestra.run.smithi083.stderr:dumped all 2024-08-19T01:34:46.629 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:46.627+0000 7f4f9f000640 1 -- 172.21.15.83:0/130211398 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f4f800a2c20 msgr2=0x7f4f800a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:34:46.629 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:46.627+0000 7f4f9f000640 1 --2- 172.21.15.83:0/130211398 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f4f800a2c20 0x7f4f800a50e0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f4f840095f0 tx=0x7f4f84005f90 comp rx=0 tx=0).stop 2024-08-19T01:34:46.629 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:46.627+0000 7f4f9f000640 1 -- 172.21.15.83:0/130211398 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4f981012b0 msgr2=0x7f4f98199d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:34:46.629 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:46.627+0000 7f4f9f000640 1 --2- 172.21.15.83:0/130211398 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4f981012b0 0x7f4f98199d60 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f4f8c009230 tx=0x7f4f8c004270 comp rx=0 tx=0).stop 2024-08-19T01:34:46.630 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:46.627+0000 7f4f9f000640 1 -- 172.21.15.83:0/130211398 shutdown_connections 2024-08-19T01:34:46.630 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:46.627+0000 7f4f9f000640 1 --2- 172.21.15.83:0/130211398 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f4f800a2c20 0x7f4f800a50e0 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:46.630 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:46.627+0000 7f4f9f000640 1 --2- 172.21.15.83:0/130211398 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4f98102d80 0x7f4f981a1310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:46.630 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:46.627+0000 7f4f9f000640 1 --2- 172.21.15.83:0/130211398 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4f98101c70 0x7f4f9819a2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:46.630 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:46.627+0000 7f4f9f000640 1 --2- 172.21.15.83:0/130211398 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4f981012b0 0x7f4f98199d60 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:46.630 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:46.627+0000 7f4f9f000640 1 -- 172.21.15.83:0/130211398 >> 172.21.15.83:0/130211398 conn(0x7f4f980fce40 msgr2=0x7f4f98108060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:34:46.630 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:46.627+0000 7f4f9f000640 1 -- 172.21.15.83:0/130211398 shutdown_connections 2024-08-19T01:34:46.630 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:46.627+0000 7f4f9f000640 1 -- 172.21.15.83:0/130211398 wait complete. 2024-08-19T01:34:46.645 INFO:teuthology.orchestra.run.smithi083.stdout:{"pg_ready":true,"pg_map":{"version":337,"stamp":"2024-08-19T01:34:46.389812+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":662245,"num_objects":64,"num_object_clones":0,"num_object_copies":192,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":833,"num_read_kb":541,"num_write":371,"num_write_kb":3073,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":1181748,"num_keys_recovered":10,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":798,"ondisk_log_size":798,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":374760,"kb_used_data":13128,"kb_used_omap":18,"kb_used_meta":361517,"kb_avail":1124567064,"statfs":{"total":1151940427776,"available":1151556673536,"internally_reserved":0,"allocated":13443072,"data_stored":7391583,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19143,"internal_metadata":370193721},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":6106,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":85,"num_read_kb":54,"num_write":15,"num_write_kb":16,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002871"},"pg_stats":[{"pgid":"3.3f","version":"86'8","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:34:04.396225+0000","last_change":"2024-08-19T01:32:41.244214+0000","last_active":"2024-08-19T01:34:04.396225+0000","last_peered":"2024-08-19T01:34:04.396225+0000","last_clean":"2024-08-19T01:34:04.396225+0000","last_became_active":"2024-08-19T01:32:41.243889+0000","last_became_peered":"2024-08-19T01:32:41.243889+0000","last_unstale":"2024-08-19T01:34:04.396225+0000","last_undegraded":"2024-08-19T01:34:04.396225+0000","last_fullsized":"2024-08-19T01:34:04.396225+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_clean_scrub_stamp":"2024-08-19T01:32:40.202519+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T13:10:18.230055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"86'4","reported_seq":21,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:34:04.396539+0000","last_change":"2024-08-19T01:32:41.242729+0000","last_active":"2024-08-19T01:34:04.396539+0000","last_peered":"2024-08-19T01:34:04.396539+0000","last_clean":"2024-08-19T01:34:04.396539+0000","last_became_active":"2024-08-19T01:32:41.242436+0000","last_became_peered":"2024-08-19T01:32:41.242436+0000","last_unstale":"2024-08-19T01:34:04.396539+0000","last_undegraded":"2024-08-19T01:34:04.396539+0000","last_fullsized":"2024-08-19T01:34:04.396539+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_clean_scrub_stamp":"2024-08-19T01:32:40.202519+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T13:19:32.010576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"86'7","reported_seq":24,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:34:04.399256+0000","last_change":"2024-08-19T01:32:41.247377+0000","last_active":"2024-08-19T01:34:04.399256+0000","last_peered":"2024-08-19T01:34:04.399256+0000","last_clean":"2024-08-19T01:34:04.399256+0000","last_became_active":"2024-08-19T01:32:41.246804+0000","last_became_peered":"2024-08-19T01:32:41.246804+0000","last_unstale":"2024-08-19T01:34:04.399256+0000","last_undegraded":"2024-08-19T01:34:04.399256+0000","last_fullsized":"2024-08-19T01:34:04.399256+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_clean_scrub_stamp":"2024-08-19T01:32:40.202519+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T04:48:52.382687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"86'8","reported_seq":26,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:34:04.389960+0000","last_change":"2024-08-19T01:32:41.232900+0000","last_active":"2024-08-19T01:34:04.389960+0000","last_peered":"2024-08-19T01:34:04.389960+0000","last_clean":"2024-08-19T01:34:04.389960+0000","last_became_active":"2024-08-19T01:32:41.232573+0000","last_became_peered":"2024-08-19T01:32:41.232573+0000","last_unstale":"2024-08-19T01:34:04.389960+0000","last_undegraded":"2024-08-19T01:34:04.389960+0000","last_fullsized":"2024-08-19T01:34:04.389960+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_clean_scrub_stamp":"2024-08-19T01:32:40.202519+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T09:42:36.309834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"86'4","reported_seq":27,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:34:04.393723+0000","last_change":"2024-08-19T01:33:05.478964+0000","last_active":"2024-08-19T01:34:04.393723+0000","last_peered":"2024-08-19T01:34:04.393723+0000","last_clean":"2024-08-19T01:34:04.393723+0000","last_became_active":"2024-08-19T01:33:05.478430+0000","last_became_peered":"2024-08-19T01:33:05.478430+0000","last_unstale":"2024-08-19T01:34:04.393723+0000","last_undegraded":"2024-08-19T01:34:04.393723+0000","last_fullsized":"2024-08-19T01:34:04.393723+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_clean_scrub_stamp":"2024-08-19T01:32:40.202519+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T04:10:49.267550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,6],"acting":[8,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"86'6","reported_seq":23,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:34:04.395690+0000","last_change":"2024-08-19T01:32:41.228263+0000","last_active":"2024-08-19T01:34:04.395690+0000","last_peered":"2024-08-19T01:34:04.395690+0000","last_clean":"2024-08-19T01:34:04.395690+0000","last_became_active":"2024-08-19T01:32:41.227765+0000","last_became_peered":"2024-08-19T01:32:41.227765+0000","last_unstale":"2024-08-19T01:34:04.395690+0000","last_undegraded":"2024-08-19T01:34:04.395690+0000","last_fullsized":"2024-08-19T01:34:04.395690+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_clean_scrub_stamp":"2024-08-19T01:32:40.202519+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T11:21:18.817886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"86'5","reported_seq":22,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:34:04.391175+0000","last_change":"2024-08-19T01:32:41.235680+0000","last_active":"2024-08-19T01:34:04.391175+0000","last_peered":"2024-08-19T01:34:04.391175+0000","last_clean":"2024-08-19T01:34:04.391175+0000","last_became_active":"2024-08-19T01:32:41.234150+0000","last_became_peered":"2024-08-19T01:32:41.234150+0000","last_unstale":"2024-08-19T01:34:04.391175+0000","last_undegraded":"2024-08-19T01:34:04.391175+0000","last_fullsized":"2024-08-19T01:34:04.391175+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_clean_scrub_stamp":"2024-08-19T01:32:40.202519+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T10:28:08.908632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"86'9","reported_seq":26,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:34:04.394271+0000","last_change":"2024-08-19T01:32:41.230170+0000","last_active":"2024-08-19T01:34:04.394271+0000","last_peered":"2024-08-19T01:34:04.394271+0000","last_clean":"2024-08-19T01:34:04.394271+0000","last_became_active":"2024-08-19T01:32:41.229467+0000","last_became_peered":"2024-08-19T01:32:41.229467+0000","last_unstale":"2024-08-19T01:34:04.394271+0000","last_undegraded":"2024-08-19T01:34:04.394271+0000","last_fullsized":"2024-08-19T01:34:04.394271+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_clean_scrub_stamp":"2024-08-19T01:32:40.202519+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T10:24:15.832910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"86'11","reported_seq":28,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:34:04.398861+0000","last_change":"2024-08-19T01:32:41.230607+0000","last_active":"2024-08-19T01:34:04.398861+0000","last_peered":"2024-08-19T01:34:04.398861+0000","last_clean":"2024-08-19T01:34:04.398861+0000","last_became_active":"2024-08-19T01:32:41.229979+0000","last_became_peered":"2024-08-19T01:32:41.229979+0000","last_unstale":"2024-08-19T01:34:04.398861+0000","last_undegraded":"2024-08-19T01:34:04.398861+0000","last_fullsized":"2024-08-19T01:34:04.398861+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_clean_scrub_stamp":"2024-08-19T01:32:40.202519+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T02:23:46.736139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"86'16","reported_seq":33,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:34:04.395510+0000","last_change":"2024-08-19T01:32:41.233576+0000","last_active":"2024-08-19T01:34:04.395510+0000","last_peered":"2024-08-19T01:34:04.395510+0000","last_clean":"2024-08-19T01:34:04.395510+0000","last_became_active":"2024-08-19T01:32:41.231279+0000","last_became_peered":"2024-08-19T01:32:41.231279+0000","last_unstale":"2024-08-19T01:34:04.395510+0000","last_undegraded":"2024-08-19T01:34:04.395510+0000","last_fullsized":"2024-08-19T01:34:04.395510+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_clean_scrub_stamp":"2024-08-19T01:32:40.202519+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T07:32:45.341628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"86'6","reported_seq":23,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:34:04.396391+0000","last_change":"2024-08-19T01:32:41.233192+0000","last_active":"2024-08-19T01:34:04.396391+0000","last_peered":"2024-08-19T01:34:04.396391+0000","last_clean":"2024-08-19T01:34:04.396391+0000","last_became_active":"2024-08-19T01:32:41.230944+0000","last_became_peered":"2024-08-19T01:32:41.230944+0000","last_unstale":"2024-08-19T01:34:04.396391+0000","last_undegraded":"2024-08-19T01:34:04.396391+0000","last_fullsized":"2024-08-19T01:34:04.396391+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_clean_scrub_stamp":"2024-08-19T01:32:40.202519+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T08:55:29.036252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"86'7","reported_seq":24,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:34:04.386560+0000","last_change":"2024-08-19T01:32:41.234216+0000","last_active":"2024-08-19T01:34:04.386560+0000","last_peered":"2024-08-19T01:34:04.386560+0000","last_clean":"2024-08-19T01:34:04.386560+0000","last_became_active":"2024-08-19T01:32:41.233906+0000","last_became_peered":"2024-08-19T01:32:41.233906+0000","last_unstale":"2024-08-19T01:34:04.386560+0000","last_undegraded":"2024-08-19T01:34:04.386560+0000","last_fullsized":"2024-08-19T01:34:04.386560+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_clean_scrub_stamp":"2024-08-19T01:32:40.202519+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T08:35:01.764494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"86'10","reported_seq":27,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:34:04.398217+0000","last_change":"2024-08-19T01:32:41.230651+0000","last_active":"2024-08-19T01:34:04.398217+0000","last_peered":"2024-08-19T01:34:04.398217+0000","last_clean":"2024-08-19T01:34:04.398217+0000","last_became_active":"2024-08-19T01:32:41.230253+0000","last_became_peered":"2024-08-19T01:32:41.230253+0000","last_unstale":"2024-08-19T01:34:04.398217+0000","last_undegraded":"2024-08-19T01:34:04.398217+0000","last_fullsized":"2024-08-19T01:34:04.398217+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_clean_scrub_stamp":"2024-08-19T01:32:40.202519+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T09:41:59.942821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"86'7","reported_seq":24,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:34:04.396983+0000","last_change":"2024-08-19T01:32:41.240082+0000","last_active":"2024-08-19T01:34:04.396983+0000","last_peered":"2024-08-19T01:34:04.396983+0000","last_clean":"2024-08-19T01:34:04.396983+0000","last_became_active":"2024-08-19T01:32:41.239711+0000","last_became_peered":"2024-08-19T01:32:41.239711+0000","last_unstale":"2024-08-19T01:34:04.396983+0000","last_undegraded":"2024-08-19T01:34:04.396983+0000","last_fullsized":"2024-08-19T01:34:04.396983+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_clean_scrub_stamp":"2024-08-19T01:32:40.202519+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T12:48:59.122307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"86'7","reported_seq":30,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:34:04.390223+0000","last_change":"2024-08-19T01:33:05.474883+0000","last_active":"2024-08-19T01:34:04.390223+0000","last_peered":"2024-08-19T01:34:04.390223+0000","last_clean":"2024-08-19T01:34:04.390223+0000","last_became_active":"2024-08-19T01:33:05.474502+0000","last_became_peered":"2024-08-19T01:33:05.474502+0000","last_unstale":"2024-08-19T01:34:04.390223+0000","last_undegraded":"2024-08-19T01:34:04.390223+0000","last_fullsized":"2024-08-19T01:34:04.390223+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_clean_scrub_stamp":"2024-08-19T01:32:40.202519+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T07:02:01.171103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2],"acting":[0,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"86'9","reported_seq":26,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:34:04.399368+0000","last_change":"2024-08-19T01:32:41.240899+0000","last_active":"2024-08-19T01:34:04.399368+0000","last_peered":"2024-08-19T01:34:04.399368+0000","last_clean":"2024-08-19T01:34:04.399368+0000","last_became_active":"2024-08-19T01:32:41.240532+0000","last_became_peered":"2024-08-19T01:32:41.240532+0000","last_unstale":"2024-08-19T01:34:04.399368+0000","last_undegraded":"2024-08-19T01:34:04.399368+0000","last_fullsized":"2024-08-19T01:34:04.399368+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_clean_scrub_stamp":"2024-08-19T01:32:40.202519+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T08:17:47.035887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"86'8","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:34:04.396273+0000","last_change":"2024-08-19T01:32:41.237977+0000","last_active":"2024-08-19T01:34:04.396273+0000","last_peered":"2024-08-19T01:34:04.396273+0000","last_clean":"2024-08-19T01:34:04.396273+0000","last_became_active":"2024-08-19T01:32:41.237634+0000","last_became_peered":"2024-08-19T01:32:41.237634+0000","last_unstale":"2024-08-19T01:34:04.396273+0000","last_undegraded":"2024-08-19T01:34:04.396273+0000","last_fullsized":"2024-08-19T01:34:04.396273+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_clean_scrub_stamp":"2024-08-19T01:32:40.202519+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T12:48:37.005972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"86'10","reported_seq":27,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:34:04.391627+0000","last_change":"2024-08-19T01:32:41.236498+0000","last_active":"2024-08-19T01:34:04.391627+0000","last_peered":"2024-08-19T01:34:04.391627+0000","last_clean":"2024-08-19T01:34:04.391627+0000","last_became_active":"2024-08-19T01:32:41.236122+0000","last_became_peered":"2024-08-19T01:32:41.236122+0000","last_unstale":"2024-08-19T01:34:04.391627+0000","last_undegraded":"2024-08-19T01:34:04.391627+0000","last_fullsized":"2024-08-19T01:34:04.391627+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_clean_scrub_stamp":"2024-08-19T01:32:40.202519+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T02:31:23.648064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"86'7","reported_seq":24,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:34:04.390365+0000","last_change":"2024-08-19T01:32:41.234607+0000","last_active":"2024-08-19T01:34:04.390365+0000","last_peered":"2024-08-19T01:34:04.390365+0000","last_clean":"2024-08-19T01:34:04.390365+0000","last_became_active":"2024-08-19T01:32:41.232867+0000","last_became_peered":"2024-08-19T01:32:41.232867+0000","last_unstale":"2024-08-19T01:34:04.390365+0000","last_undegraded":"2024-08-19T01:34:04.390365+0000","last_fullsized":"2024-08-19T01:34:04.390365+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_clean_scrub_stamp":"2024-08-19T01:32:40.202519+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T12:32:38.441848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2c","version":"86'8","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:34:04.384758+0000","last_change":"2024-08-19T01:32:41.231705+0000","last_active":"2024-08-19T01:34:04.384758+0000","last_peered":"2024-08-19T01:34:04.384758+0000","last_clean":"2024-08-19T01:34:04.384758+0000","last_became_active":"2024-08-19T01:32:41.231262+0000","last_became_peered":"2024-08-19T01:32:41.231262+0000","last_unstale":"2024-08-19T01:34:04.384758+0000","last_undegraded":"2024-08-19T01:34:04.384758+0000","last_fullsized":"2024-08-19T01:34:04.384758+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_clean_scrub_stamp":"2024-08-19T01:32:40.202519+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T10:41:15.025471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2b","version":"86'11","reported_seq":28,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:34:04.397040+0000","last_change":"2024-08-19T01:32:41.243192+0000","last_active":"2024-08-19T01:34:04.397040+0000","last_peered":"2024-08-19T01:34:04.397040+0000","last_clean":"2024-08-19T01:34:04.397040+0000","last_became_active":"2024-08-19T01:32:41.242704+0000","last_became_peered":"2024-08-19T01:32:41.242704+0000","last_unstale":"2024-08-19T01:34:04.397040+0000","last_undegraded":"2024-08-19T01:34:04.397040+0000","last_fullsized":"2024-08-19T01:34:04.397040+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_clean_scrub_stamp":"2024-08-19T01:32:40.202519+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T01:44:43.270213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2a","version":"86'10","reported_seq":27,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:34:04.399726+0000","last_change":"2024-08-19T01:32:41.239908+0000","last_active":"2024-08-19T01:34:04.399726+0000","last_peered":"2024-08-19T01:34:04.399726+0000","last_clean":"2024-08-19T01:34:04.399726+0000","last_became_active":"2024-08-19T01:32:41.239438+0000","last_became_peered":"2024-08-19T01:32:41.239438+0000","last_unstale":"2024-08-19T01:34:04.399726+0000","last_undegraded":"2024-08-19T01:34:04.399726+0000","last_fullsized":"2024-08-19T01:34:04.399726+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_clean_scrub_stamp":"2024-08-19T01:32:40.202519+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T11:03:26.822111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.29","version":"86'8","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:34:04.389344+0000","last_change":"2024-08-19T01:32:41.226373+0000","last_active":"2024-08-19T01:34:04.389344+0000","last_peered":"2024-08-19T01:34:04.389344+0000","last_clean":"2024-08-19T01:34:04.389344+0000","last_became_active":"2024-08-19T01:32:41.226089+0000","last_became_peered":"2024-08-19T01:32:41.226089+0000","last_unstale":"2024-08-19T01:34:04.389344+0000","last_undegraded":"2024-08-19T01:34:04.389344+0000","last_fullsized":"2024-08-19T01:34:04.389344+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_clean_scrub_stamp":"2024-08-19T01:32:40.202519+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T01:33:56.639300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"86'6","reported_seq":23,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:34:04.389272+0000","last_change":"2024-08-19T01:32:41.237829+0000","last_active":"2024-08-19T01:34:04.389272+0000","last_peered":"2024-08-19T01:34:04.389272+0000","last_clean":"2024-08-19T01:34:04.389272+0000","last_became_active":"2024-08-19T01:32:41.237518+0000","last_became_peered":"2024-08-19T01:32:41.237518+0000","last_unstale":"2024-08-19T01:34:04.389272+0000","last_undegraded":"2024-08-19T01:34:04.389272+0000","last_fullsized":"2024-08-19T01:34:04.389272+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_clean_scrub_stamp":"2024-08-19T01:32:40.202519+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T02:25:11.307658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"86'2","reported_seq":19,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:34:04.386117+0000","last_change":"2024-08-19T01:32:41.234508+0000","last_active":"2024-08-19T01:34:04.386117+0000","last_peered":"2024-08-19T01:34:04.386117+0000","last_clean":"2024-08-19T01:34:04.386117+0000","last_became_active":"2024-08-19T01:32:41.232869+0000","last_became_peered":"2024-08-19T01:32:41.232869+0000","last_unstale":"2024-08-19T01:34:04.386117+0000","last_undegraded":"2024-08-19T01:34:04.386117+0000","last_fullsized":"2024-08-19T01:34:04.386117+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_clean_scrub_stamp":"2024-08-19T01:32:40.202519+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T11:20:55.097576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"86'15","reported_seq":32,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:34:04.390238+0000","last_change":"2024-08-19T01:32:41.230536+0000","last_active":"2024-08-19T01:34:04.390238+0000","last_peered":"2024-08-19T01:34:04.390238+0000","last_clean":"2024-08-19T01:34:04.390238+0000","last_became_active":"2024-08-19T01:32:41.230179+0000","last_became_peered":"2024-08-19T01:32:41.230179+0000","last_unstale":"2024-08-19T01:34:04.390238+0000","last_undegraded":"2024-08-19T01:34:04.390238+0000","last_fullsized":"2024-08-19T01:34:04.390238+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_clean_scrub_stamp":"2024-08-19T01:32:40.202519+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T02:25:09.441363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.25","version":"86'6","reported_seq":23,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:34:04.387595+0000","last_change":"2024-08-19T01:32:41.230695+0000","last_active":"2024-08-19T01:34:04.387595+0000","last_peered":"2024-08-19T01:34:04.387595+0000","last_clean":"2024-08-19T01:34:04.387595+0000","last_became_active":"2024-08-19T01:32:41.230324+0000","last_became_peered":"2024-08-19T01:32:41.230324+0000","last_unstale":"2024-08-19T01:34:04.387595+0000","last_undegraded":"2024-08-19T01:34:04.387595+0000","last_fullsized":"2024-08-19T01:34:04.387595+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_clean_scrub_stamp":"2024-08-19T01:32:40.202519+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T05:38:09.317605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.24","version":"86'4","reported_seq":21,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:34:04.391939+0000","last_change":"2024-08-19T01:32:41.238325+0000","last_active":"2024-08-19T01:34:04.391939+0000","last_peered":"2024-08-19T01:34:04.391939+0000","last_clean":"2024-08-19T01:34:04.391939+0000","last_became_active":"2024-08-19T01:32:41.237768+0000","last_became_peered":"2024-08-19T01:32:41.237768+0000","last_unstale":"2024-08-19T01:34:04.391939+0000","last_undegraded":"2024-08-19T01:34:04.391939+0000","last_fullsized":"2024-08-19T01:34:04.391939+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_clean_scrub_stamp":"2024-08-19T01:32:40.202519+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T08:45:06.866241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.23","version":"86'7","reported_seq":24,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:34:04.396309+0000","last_change":"2024-08-19T01:32:41.246950+0000","last_active":"2024-08-19T01:34:04.396309+0000","last_peered":"2024-08-19T01:34:04.396309+0000","last_clean":"2024-08-19T01:34:04.396309+0000","last_became_active":"2024-08-19T01:32:41.245441+0000","last_became_peered":"2024-08-19T01:32:41.245441+0000","last_unstale":"2024-08-19T01:34:04.396309+0000","last_undegraded":"2024-08-19T01:34:04.396309+0000","last_fullsized":"2024-08-19T01:34:04.396309+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_clean_scrub_stamp":"2024-08-19T01:32:40.202519+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T03:11:06.921258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"86'8","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:34:04.373200+0000","last_change":"2024-08-19T01:32:41.244810+0000","last_active":"2024-08-19T01:34:04.373200+0000","last_peered":"2024-08-19T01:34:04.373200+0000","last_clean":"2024-08-19T01:34:04.373200+0000","last_became_active":"2024-08-19T01:32:41.244401+0000","last_became_peered":"2024-08-19T01:32:41.244401+0000","last_unstale":"2024-08-19T01:34:04.373200+0000","last_undegraded":"2024-08-19T01:34:04.373200+0000","last_fullsized":"2024-08-19T01:34:04.373200+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_clean_scrub_stamp":"2024-08-19T01:32:40.202519+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T04:54:57.728181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.21","version":"86'12","reported_seq":29,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:34:04.387129+0000","last_change":"2024-08-19T01:32:41.235977+0000","last_active":"2024-08-19T01:34:04.387129+0000","last_peered":"2024-08-19T01:34:04.387129+0000","last_clean":"2024-08-19T01:34:04.387129+0000","last_became_active":"2024-08-19T01:32:41.235693+0000","last_became_peered":"2024-08-19T01:32:41.235693+0000","last_unstale":"2024-08-19T01:34:04.387129+0000","last_undegraded":"2024-08-19T01:34:04.387129+0000","last_fullsized":"2024-08-19T01:34:04.387129+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_clean_scrub_stamp":"2024-08-19T01:32:40.202519+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T04:59:15.766532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"86'9","reported_seq":32,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:34:04.386756+0000","last_change":"2024-08-19T01:33:05.473318+0000","last_active":"2024-08-19T01:34:04.386756+0000","last_peered":"2024-08-19T01:34:04.386756+0000","last_clean":"2024-08-19T01:34:04.386756+0000","last_became_active":"2024-08-19T01:33:05.472885+0000","last_became_peered":"2024-08-19T01:33:05.472885+0000","last_unstale":"2024-08-19T01:34:04.386756+0000","last_undegraded":"2024-08-19T01:34:04.386756+0000","last_fullsized":"2024-08-19T01:34:04.386756+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_clean_scrub_stamp":"2024-08-19T01:32:40.202519+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T07:49:35.525930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,9],"acting":[11,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1a","version":"86'7","reported_seq":24,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:34:04.376075+0000","last_change":"2024-08-19T01:32:41.230846+0000","last_active":"2024-08-19T01:34:04.376075+0000","last_peered":"2024-08-19T01:34:04.376075+0000","last_clean":"2024-08-19T01:34:04.376075+0000","last_became_active":"2024-08-19T01:32:41.230405+0000","last_became_peered":"2024-08-19T01:32:41.230405+0000","last_unstale":"2024-08-19T01:34:04.376075+0000","last_undegraded":"2024-08-19T01:34:04.376075+0000","last_fullsized":"2024-08-19T01:34:04.376075+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_clean_scrub_stamp":"2024-08-19T01:32:40.202519+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T08:59:46.241898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1b","version":"86'8","reported_seq":158,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:34:42.617354+0000","last_change":"2024-08-19T01:33:05.479889+0000","last_active":"2024-08-19T01:34:42.617354+0000","last_peered":"2024-08-19T01:34:42.617354+0000","last_clean":"2024-08-19T01:34:42.617354+0000","last_became_active":"2024-08-19T01:33:05.474698+0000","last_became_peered":"2024-08-19T01:33:05.474698+0000","last_unstale":"2024-08-19T01:34:42.617354+0000","last_undegraded":"2024-08-19T01:34:42.617354+0000","last_fullsized":"2024-08-19T01:34:42.617354+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:32:37.889224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:32:37.889224+0000","last_clean_scrub_stamp":"2024-08-19T01:32:37.889224+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T08:41:29.053471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1543,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":114,"num_read_kb":5,"num_write":5,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":1012,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7],"acting":[5,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"86'8","reported_seq":19,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:34:04.387536+0000","last_change":"2024-08-19T01:33:05.474083+0000","last_active":"2024-08-19T01:34:04.387536+0000","last_peered":"2024-08-19T01:34:04.387536+0000","last_clean":"2024-08-19T01:34:04.387536+0000","last_became_active":"2024-08-19T01:33:05.473718+0000","last_became_peered":"2024-08-19T01:33:05.473718+0000","last_unstale":"2024-08-19T01:34:04.387536+0000","last_undegraded":"2024-08-19T01:34:04.387536+0000","last_fullsized":"2024-08-19T01:34:04.387536+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_clean_scrub_stamp":"2024-08-19T01:32:40.202519+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T02:25:11.955170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,4,3],"acting":[9,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.1a","version":"84'3","reported_seq":35,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:33:05.705917+0000","last_change":"2024-08-19T01:33:05.705917+0000","last_active":"2024-08-19T01:33:05.705917+0000","last_peered":"2024-08-19T01:33:05.705917+0000","last_clean":"2024-08-19T01:33:05.705917+0000","last_became_active":"2024-08-19T01:33:05.702775+0000","last_became_peered":"2024-08-19T01:33:05.702775+0000","last_unstale":"2024-08-19T01:33:05.705917+0000","last_undegraded":"2024-08-19T01:33:05.705917+0000","last_fullsized":"2024-08-19T01:33:05.705917+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:32:37.889224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:32:37.889224+0000","last_clean_scrub_stamp":"2024-08-19T01:32:37.889224+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T03:58:29.989815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,6],"acting":[10,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.19","version":"84'2","reported_seq":24,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:33:05.477490+0000","last_change":"2024-08-19T01:32:38.911757+0000","last_active":"2024-08-19T01:33:05.477490+0000","last_peered":"2024-08-19T01:33:05.477490+0000","last_clean":"2024-08-19T01:33:05.477490+0000","last_became_active":"2024-08-19T01:32:38.910832+0000","last_became_peered":"2024-08-19T01:32:38.910832+0000","last_unstale":"2024-08-19T01:33:05.477490+0000","last_undegraded":"2024-08-19T01:33:05.477490+0000","last_fullsized":"2024-08-19T01:33:05.477490+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:32:37.889224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:32:37.889224+0000","last_clean_scrub_stamp":"2024-08-19T01:32:37.889224+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T04:46:54.986693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"86'8","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:34:04.392576+0000","last_change":"2024-08-19T01:32:41.231130+0000","last_active":"2024-08-19T01:34:04.392576+0000","last_peered":"2024-08-19T01:34:04.392576+0000","last_clean":"2024-08-19T01:34:04.392576+0000","last_became_active":"2024-08-19T01:32:41.230883+0000","last_became_peered":"2024-08-19T01:32:41.230883+0000","last_unstale":"2024-08-19T01:34:04.392576+0000","last_undegraded":"2024-08-19T01:34:04.392576+0000","last_fullsized":"2024-08-19T01:34:04.392576+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_clean_scrub_stamp":"2024-08-19T01:32:40.202519+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T06:33:11.856202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"84'1","reported_seq":23,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:33:05.470862+0000","last_change":"2024-08-19T01:32:38.916868+0000","last_active":"2024-08-19T01:33:05.470862+0000","last_peered":"2024-08-19T01:33:05.470862+0000","last_clean":"2024-08-19T01:33:05.470862+0000","last_became_active":"2024-08-19T01:32:38.916500+0000","last_became_peered":"2024-08-19T01:32:38.916500+0000","last_unstale":"2024-08-19T01:33:05.470862+0000","last_undegraded":"2024-08-19T01:33:05.470862+0000","last_fullsized":"2024-08-19T01:33:05.470862+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:32:37.889224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:32:37.889224+0000","last_clean_scrub_stamp":"2024-08-19T01:32:37.889224+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T12:30:07.085829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"86'11","reported_seq":28,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:34:04.391892+0000","last_change":"2024-08-19T01:32:41.235290+0000","last_active":"2024-08-19T01:34:04.391892+0000","last_peered":"2024-08-19T01:34:04.391892+0000","last_clean":"2024-08-19T01:34:04.391892+0000","last_became_active":"2024-08-19T01:32:41.235013+0000","last_became_peered":"2024-08-19T01:32:41.235013+0000","last_unstale":"2024-08-19T01:34:04.391892+0000","last_undegraded":"2024-08-19T01:34:04.391892+0000","last_fullsized":"2024-08-19T01:34:04.391892+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_clean_scrub_stamp":"2024-08-19T01:32:40.202519+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T09:50:14.837080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"86'5","reported_seq":139,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:34:42.616731+0000","last_change":"2024-08-19T01:32:38.912023+0000","last_active":"2024-08-19T01:34:42.616731+0000","last_peered":"2024-08-19T01:34:42.616731+0000","last_clean":"2024-08-19T01:34:42.616731+0000","last_became_active":"2024-08-19T01:32:38.911679+0000","last_became_peered":"2024-08-19T01:32:38.911679+0000","last_unstale":"2024-08-19T01:34:42.616731+0000","last_undegraded":"2024-08-19T01:34:42.616731+0000","last_fullsized":"2024-08-19T01:34:42.616731+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:32:37.889224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:32:37.889224+0000","last_clean_scrub_stamp":"2024-08-19T01:32:37.889224+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T12:08:51.118338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":112,"num_read_kb":112,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"86'10","reported_seq":27,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:34:04.399254+0000","last_change":"2024-08-19T01:32:41.232264+0000","last_active":"2024-08-19T01:34:04.399254+0000","last_peered":"2024-08-19T01:34:04.399254+0000","last_clean":"2024-08-19T01:34:04.399254+0000","last_became_active":"2024-08-19T01:32:41.231788+0000","last_became_peered":"2024-08-19T01:32:41.231788+0000","last_unstale":"2024-08-19T01:34:04.399254+0000","last_undegraded":"2024-08-19T01:34:04.399254+0000","last_fullsized":"2024-08-19T01:34:04.399254+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_clean_scrub_stamp":"2024-08-19T01:32:40.202519+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T04:59:06.443992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"84'2","reported_seq":24,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:33:05.472954+0000","last_change":"2024-08-19T01:32:38.915770+0000","last_active":"2024-08-19T01:33:05.472954+0000","last_peered":"2024-08-19T01:33:05.472954+0000","last_clean":"2024-08-19T01:33:05.472954+0000","last_became_active":"2024-08-19T01:32:38.915378+0000","last_became_peered":"2024-08-19T01:32:38.915378+0000","last_unstale":"2024-08-19T01:33:05.472954+0000","last_undegraded":"2024-08-19T01:33:05.472954+0000","last_fullsized":"2024-08-19T01:33:05.472954+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:32:37.889224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:32:37.889224+0000","last_clean_scrub_stamp":"2024-08-19T01:32:37.889224+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T09:16:14.513172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"86'9","reported_seq":26,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:34:04.395227+0000","last_change":"2024-08-19T01:32:41.243476+0000","last_active":"2024-08-19T01:34:04.395227+0000","last_peered":"2024-08-19T01:34:04.395227+0000","last_clean":"2024-08-19T01:34:04.395227+0000","last_became_active":"2024-08-19T01:32:41.243190+0000","last_became_peered":"2024-08-19T01:32:41.243190+0000","last_unstale":"2024-08-19T01:34:04.395227+0000","last_undegraded":"2024-08-19T01:34:04.395227+0000","last_fullsized":"2024-08-19T01:34:04.395227+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_clean_scrub_stamp":"2024-08-19T01:32:40.202519+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T12:07:34.096919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"84'4","reported_seq":27,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:33:05.470664+0000","last_change":"2024-08-19T01:32:39.051458+0000","last_active":"2024-08-19T01:33:05.470664+0000","last_peered":"2024-08-19T01:33:05.470664+0000","last_clean":"2024-08-19T01:33:05.470664+0000","last_became_active":"2024-08-19T01:32:39.051113+0000","last_became_peered":"2024-08-19T01:32:39.051113+0000","last_unstale":"2024-08-19T01:33:05.470664+0000","last_undegraded":"2024-08-19T01:33:05.470664+0000","last_fullsized":"2024-08-19T01:33:05.470664+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:32:37.889224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:32:37.889224+0000","last_clean_scrub_stamp":"2024-08-19T01:32:37.889224+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T02:15:51.371054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"86'11","reported_seq":28,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:34:04.398787+0000","last_change":"2024-08-19T01:32:41.236453+0000","last_active":"2024-08-19T01:34:04.398787+0000","last_peered":"2024-08-19T01:34:04.398787+0000","last_clean":"2024-08-19T01:34:04.398787+0000","last_became_active":"2024-08-19T01:32:41.236135+0000","last_became_peered":"2024-08-19T01:32:41.236135+0000","last_unstale":"2024-08-19T01:34:04.398787+0000","last_undegraded":"2024-08-19T01:34:04.398787+0000","last_fullsized":"2024-08-19T01:34:04.398787+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_clean_scrub_stamp":"2024-08-19T01:32:40.202519+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T12:38:41.398299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"86'35","reported_seq":293,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:34:43.619166+0000","last_change":"2024-08-19T01:32:38.914831+0000","last_active":"2024-08-19T01:34:43.619166+0000","last_peered":"2024-08-19T01:34:43.619166+0000","last_clean":"2024-08-19T01:34:43.619166+0000","last_became_active":"2024-08-19T01:32:38.914326+0000","last_became_peered":"2024-08-19T01:32:38.914326+0000","last_unstale":"2024-08-19T01:34:43.619166+0000","last_undegraded":"2024-08-19T01:34:43.619166+0000","last_fullsized":"2024-08-19T01:34:43.619166+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:32:37.889224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:32:37.889224+0000","last_clean_scrub_stamp":"2024-08-19T01:32:37.889224+0000","objects_scrubbed":0,"log_size":35,"log_dups_size":0,"ondisk_log_size":35,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T02:15:25.263974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":65880,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":236,"num_read_kb":181,"num_write":36,"num_write_kb":79,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"86'6","reported_seq":23,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:34:04.390164+0000","last_change":"2024-08-19T01:32:41.237945+0000","last_active":"2024-08-19T01:34:04.390164+0000","last_peered":"2024-08-19T01:34:04.390164+0000","last_clean":"2024-08-19T01:34:04.390164+0000","last_became_active":"2024-08-19T01:32:41.237362+0000","last_became_peered":"2024-08-19T01:32:41.237362+0000","last_unstale":"2024-08-19T01:34:04.390164+0000","last_undegraded":"2024-08-19T01:34:04.390164+0000","last_fullsized":"2024-08-19T01:34:04.390164+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_clean_scrub_stamp":"2024-08-19T01:32:40.202519+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T07:29:17.180054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"84'6","reported_seq":29,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:33:05.466940+0000","last_change":"2024-08-19T01:32:38.912453+0000","last_active":"2024-08-19T01:33:05.466940+0000","last_peered":"2024-08-19T01:33:05.466940+0000","last_clean":"2024-08-19T01:33:05.466940+0000","last_became_active":"2024-08-19T01:32:38.912051+0000","last_became_peered":"2024-08-19T01:32:38.912051+0000","last_unstale":"2024-08-19T01:33:05.466940+0000","last_undegraded":"2024-08-19T01:33:05.466940+0000","last_fullsized":"2024-08-19T01:33:05.466940+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:32:37.889224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:32:37.889224+0000","last_clean_scrub_stamp":"2024-08-19T01:32:37.889224+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T06:49:58.810134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"86'9","reported_seq":26,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:34:04.391203+0000","last_change":"2024-08-19T01:32:41.236282+0000","last_active":"2024-08-19T01:34:04.391203+0000","last_peered":"2024-08-19T01:34:04.391203+0000","last_clean":"2024-08-19T01:34:04.391203+0000","last_became_active":"2024-08-19T01:32:41.235559+0000","last_became_peered":"2024-08-19T01:32:41.235559+0000","last_unstale":"2024-08-19T01:34:04.391203+0000","last_undegraded":"2024-08-19T01:34:04.391203+0000","last_fullsized":"2024-08-19T01:34:04.391203+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_clean_scrub_stamp":"2024-08-19T01:32:40.202519+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T07:51:52.157094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"84'3","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:33:05.470566+0000","last_change":"2024-08-19T01:32:38.915688+0000","last_active":"2024-08-19T01:33:05.470566+0000","last_peered":"2024-08-19T01:33:05.470566+0000","last_clean":"2024-08-19T01:33:05.470566+0000","last_became_active":"2024-08-19T01:32:38.915297+0000","last_became_peered":"2024-08-19T01:32:38.915297+0000","last_unstale":"2024-08-19T01:33:05.470566+0000","last_undegraded":"2024-08-19T01:33:05.470566+0000","last_fullsized":"2024-08-19T01:33:05.470566+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:32:37.889224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:32:37.889224+0000","last_clean_scrub_stamp":"2024-08-19T01:32:37.889224+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T01:38:12.780588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"86'15","reported_seq":32,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:34:04.395033+0000","last_change":"2024-08-19T01:32:41.233766+0000","last_active":"2024-08-19T01:34:04.395033+0000","last_peered":"2024-08-19T01:34:04.395033+0000","last_clean":"2024-08-19T01:34:04.395033+0000","last_became_active":"2024-08-19T01:32:41.233425+0000","last_became_peered":"2024-08-19T01:32:41.233425+0000","last_unstale":"2024-08-19T01:34:04.395033+0000","last_undegraded":"2024-08-19T01:34:04.395033+0000","last_fullsized":"2024-08-19T01:34:04.395033+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_clean_scrub_stamp":"2024-08-19T01:32:40.202519+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T10:29:15.471639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"86'7","reported_seq":146,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:34:42.493710+0000","last_change":"2024-08-19T01:32:38.912610+0000","last_active":"2024-08-19T01:34:42.493710+0000","last_peered":"2024-08-19T01:34:42.493710+0000","last_clean":"2024-08-19T01:34:42.493710+0000","last_became_active":"2024-08-19T01:32:38.912040+0000","last_became_peered":"2024-08-19T01:32:38.912040+0000","last_unstale":"2024-08-19T01:34:42.493710+0000","last_undegraded":"2024-08-19T01:34:42.493710+0000","last_fullsized":"2024-08-19T01:34:42.493710+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:32:37.889224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:32:37.889224+0000","last_clean_scrub_stamp":"2024-08-19T01:32:37.889224+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T09:26:05.966391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1820,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":118,"num_read_kb":7,"num_write":6,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"86'6","reported_seq":23,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:34:04.393529+0000","last_change":"2024-08-19T01:32:41.235945+0000","last_active":"2024-08-19T01:34:04.393529+0000","last_peered":"2024-08-19T01:34:04.393529+0000","last_clean":"2024-08-19T01:34:04.393529+0000","last_became_active":"2024-08-19T01:32:41.235633+0000","last_became_peered":"2024-08-19T01:32:41.235633+0000","last_unstale":"2024-08-19T01:34:04.393529+0000","last_undegraded":"2024-08-19T01:34:04.393529+0000","last_fullsized":"2024-08-19T01:34:04.393529+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_clean_scrub_stamp":"2024-08-19T01:32:40.202519+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T08:37:37.682175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"84'15","reported_seq":38,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:33:05.471087+0000","last_change":"2024-08-19T01:32:39.052850+0000","last_active":"2024-08-19T01:33:05.471087+0000","last_peered":"2024-08-19T01:33:05.471087+0000","last_clean":"2024-08-19T01:33:05.471087+0000","last_became_active":"2024-08-19T01:32:39.052455+0000","last_became_peered":"2024-08-19T01:32:39.052455+0000","last_unstale":"2024-08-19T01:33:05.471087+0000","last_undegraded":"2024-08-19T01:33:05.471087+0000","last_fullsized":"2024-08-19T01:33:05.471087+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:32:37.889224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:32:37.889224+0000","last_clean_scrub_stamp":"2024-08-19T01:32:37.889224+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T03:25:08.279911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"86'6","reported_seq":29,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:34:04.394643+0000","last_change":"2024-08-19T01:33:05.702575+0000","last_active":"2024-08-19T01:34:04.394643+0000","last_peered":"2024-08-19T01:34:04.394643+0000","last_clean":"2024-08-19T01:34:04.394643+0000","last_became_active":"2024-08-19T01:33:05.702110+0000","last_became_peered":"2024-08-19T01:33:05.702110+0000","last_unstale":"2024-08-19T01:34:04.394643+0000","last_undegraded":"2024-08-19T01:34:04.394643+0000","last_fullsized":"2024-08-19T01:34:04.394643+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_clean_scrub_stamp":"2024-08-19T01:32:40.202519+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T12:18:51.102180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,1],"acting":[5,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"86'3","reported_seq":26,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:33:05.472787+0000","last_change":"2024-08-19T01:32:38.912090+0000","last_active":"2024-08-19T01:33:05.472787+0000","last_peered":"2024-08-19T01:33:05.472787+0000","last_clean":"2024-08-19T01:33:05.472787+0000","last_became_active":"2024-08-19T01:32:38.911753+0000","last_became_peered":"2024-08-19T01:32:38.911753+0000","last_unstale":"2024-08-19T01:33:05.472787+0000","last_undegraded":"2024-08-19T01:33:05.472787+0000","last_fullsized":"2024-08-19T01:33:05.472787+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:32:37.889224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:32:37.889224+0000","last_clean_scrub_stamp":"2024-08-19T01:32:37.889224+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T12:56:42.982062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"86'5","reported_seq":22,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:34:04.387010+0000","last_change":"2024-08-19T01:32:41.230429+0000","last_active":"2024-08-19T01:34:04.387010+0000","last_peered":"2024-08-19T01:34:04.387010+0000","last_clean":"2024-08-19T01:34:04.387010+0000","last_became_active":"2024-08-19T01:32:41.230115+0000","last_became_peered":"2024-08-19T01:32:41.230115+0000","last_unstale":"2024-08-19T01:34:04.387010+0000","last_undegraded":"2024-08-19T01:34:04.387010+0000","last_fullsized":"2024-08-19T01:34:04.387010+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_clean_scrub_stamp":"2024-08-19T01:32:40.202519+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T01:50:51.084130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.e","version":"84'2","reported_seq":24,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:33:05.477826+0000","last_change":"2024-08-19T01:32:38.914557+0000","last_active":"2024-08-19T01:33:05.477826+0000","last_peered":"2024-08-19T01:33:05.477826+0000","last_clean":"2024-08-19T01:33:05.477826+0000","last_became_active":"2024-08-19T01:32:38.914115+0000","last_became_peered":"2024-08-19T01:32:38.914115+0000","last_unstale":"2024-08-19T01:33:05.477826+0000","last_undegraded":"2024-08-19T01:33:05.477826+0000","last_fullsized":"2024-08-19T01:33:05.477826+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:32:37.889224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:32:37.889224+0000","last_clean_scrub_stamp":"2024-08-19T01:32:37.889224+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T06:30:00.523513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"86'9","reported_seq":26,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:34:04.391088+0000","last_change":"2024-08-19T01:32:41.234883+0000","last_active":"2024-08-19T01:34:04.391088+0000","last_peered":"2024-08-19T01:34:04.391088+0000","last_clean":"2024-08-19T01:34:04.391088+0000","last_became_active":"2024-08-19T01:32:41.233901+0000","last_became_peered":"2024-08-19T01:32:41.233901+0000","last_unstale":"2024-08-19T01:34:04.391088+0000","last_undegraded":"2024-08-19T01:34:04.391088+0000","last_fullsized":"2024-08-19T01:34:04.391088+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_clean_scrub_stamp":"2024-08-19T01:32:40.202519+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T12:08:46.305259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"84'1","reported_seq":24,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:33:05.470777+0000","last_change":"2024-08-19T01:32:38.913441+0000","last_active":"2024-08-19T01:33:05.470777+0000","last_peered":"2024-08-19T01:33:05.470777+0000","last_clean":"2024-08-19T01:33:05.470777+0000","last_became_active":"2024-08-19T01:32:38.913115+0000","last_became_peered":"2024-08-19T01:32:38.913115+0000","last_unstale":"2024-08-19T01:33:05.470777+0000","last_undegraded":"2024-08-19T01:33:05.470777+0000","last_fullsized":"2024-08-19T01:33:05.470777+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:32:37.889224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:32:37.889224+0000","last_clean_scrub_stamp":"2024-08-19T01:32:37.889224+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T11:43:15.290980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"86'7","reported_seq":24,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:34:04.391219+0000","last_change":"2024-08-19T01:32:41.235370+0000","last_active":"2024-08-19T01:34:04.391219+0000","last_peered":"2024-08-19T01:34:04.391219+0000","last_clean":"2024-08-19T01:34:04.391219+0000","last_became_active":"2024-08-19T01:32:41.235012+0000","last_became_peered":"2024-08-19T01:32:41.235012+0000","last_unstale":"2024-08-19T01:34:04.391219+0000","last_undegraded":"2024-08-19T01:34:04.391219+0000","last_fullsized":"2024-08-19T01:34:04.391219+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_clean_scrub_stamp":"2024-08-19T01:32:40.202519+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T02:18:31.134974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"84'3","reported_seq":26,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:33:05.466872+0000","last_change":"2024-08-19T01:32:39.245560+0000","last_active":"2024-08-19T01:33:05.466872+0000","last_peered":"2024-08-19T01:33:05.466872+0000","last_clean":"2024-08-19T01:33:05.466872+0000","last_became_active":"2024-08-19T01:32:39.245118+0000","last_became_peered":"2024-08-19T01:32:39.245118+0000","last_unstale":"2024-08-19T01:33:05.466872+0000","last_undegraded":"2024-08-19T01:33:05.466872+0000","last_fullsized":"2024-08-19T01:33:05.466872+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:32:37.889224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:32:37.889224+0000","last_clean_scrub_stamp":"2024-08-19T01:32:37.889224+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T02:18:24.605709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"86'7","reported_seq":24,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:34:04.396513+0000","last_change":"2024-08-19T01:32:41.234452+0000","last_active":"2024-08-19T01:34:04.396513+0000","last_peered":"2024-08-19T01:34:04.396513+0000","last_clean":"2024-08-19T01:34:04.396513+0000","last_became_active":"2024-08-19T01:32:41.233852+0000","last_became_peered":"2024-08-19T01:32:41.233852+0000","last_unstale":"2024-08-19T01:34:04.396513+0000","last_undegraded":"2024-08-19T01:34:04.396513+0000","last_fullsized":"2024-08-19T01:34:04.396513+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_clean_scrub_stamp":"2024-08-19T01:32:40.202519+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T06:18:34.457239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"22'137","reported_seq":339,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:33:05.473281+0000","last_change":"2024-08-19T01:31:27.659394+0000","last_active":"2024-08-19T01:33:05.473281+0000","last_peered":"2024-08-19T01:33:05.473281+0000","last_clean":"2024-08-19T01:33:05.473281+0000","last_became_active":"2024-08-19T01:31:27.659007+0000","last_became_peered":"2024-08-19T01:31:27.659007+0000","last_unstale":"2024-08-19T01:33:05.473281+0000","last_undegraded":"2024-08-19T01:33:05.473281+0000","last_fullsized":"2024-08-19T01:33:05.473281+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:28:16.912251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:28:16.912251+0000","last_clean_scrub_stamp":"2024-08-19T01:28:16.912251+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T05:53:01.660619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"84'3","reported_seq":27,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:33:05.470433+0000","last_change":"2024-08-19T01:32:38.913408+0000","last_active":"2024-08-19T01:33:05.470433+0000","last_peered":"2024-08-19T01:33:05.470433+0000","last_clean":"2024-08-19T01:33:05.470433+0000","last_became_active":"2024-08-19T01:32:38.912242+0000","last_became_peered":"2024-08-19T01:32:38.912242+0000","last_unstale":"2024-08-19T01:33:05.470433+0000","last_undegraded":"2024-08-19T01:33:05.470433+0000","last_fullsized":"2024-08-19T01:33:05.470433+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:32:37.889224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:32:37.889224+0000","last_clean_scrub_stamp":"2024-08-19T01:32:37.889224+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T07:26:48.121366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2","version":"86'8","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:34:04.390396+0000","last_change":"2024-08-19T01:32:41.232611+0000","last_active":"2024-08-19T01:34:04.390396+0000","last_peered":"2024-08-19T01:34:04.390396+0000","last_clean":"2024-08-19T01:34:04.390396+0000","last_became_active":"2024-08-19T01:32:41.232261+0000","last_became_peered":"2024-08-19T01:32:41.232261+0000","last_unstale":"2024-08-19T01:34:04.390396+0000","last_undegraded":"2024-08-19T01:34:04.390396+0000","last_fullsized":"2024-08-19T01:34:04.390396+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_clean_scrub_stamp":"2024-08-19T01:32:40.202519+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T08:04:16.793933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"84'3","reported_seq":26,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:33:05.473161+0000","last_change":"2024-08-19T01:32:38.912765+0000","last_active":"2024-08-19T01:33:05.473161+0000","last_peered":"2024-08-19T01:33:05.473161+0000","last_clean":"2024-08-19T01:33:05.473161+0000","last_became_active":"2024-08-19T01:32:38.912274+0000","last_became_peered":"2024-08-19T01:32:38.912274+0000","last_unstale":"2024-08-19T01:33:05.473161+0000","last_undegraded":"2024-08-19T01:33:05.473161+0000","last_fullsized":"2024-08-19T01:33:05.473161+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:32:37.889224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:32:37.889224+0000","last_clean_scrub_stamp":"2024-08-19T01:32:37.889224+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T09:57:49.714827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"86'5","reported_seq":23,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:34:04.391551+0000","last_change":"2024-08-19T01:32:41.224449+0000","last_active":"2024-08-19T01:34:04.391551+0000","last_peered":"2024-08-19T01:34:04.391551+0000","last_clean":"2024-08-19T01:34:04.391551+0000","last_became_active":"2024-08-19T01:32:41.224064+0000","last_became_peered":"2024-08-19T01:32:41.224064+0000","last_unstale":"2024-08-19T01:34:04.391551+0000","last_undegraded":"2024-08-19T01:34:04.391551+0000","last_fullsized":"2024-08-19T01:34:04.391551+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_clean_scrub_stamp":"2024-08-19T01:32:40.202519+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T08:53:33.948504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"86'6","reported_seq":143,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:34:45.498277+0000","last_change":"2024-08-19T01:32:39.050869+0000","last_active":"2024-08-19T01:34:45.498277+0000","last_peered":"2024-08-19T01:34:45.498277+0000","last_clean":"2024-08-19T01:34:45.498277+0000","last_became_active":"2024-08-19T01:32:39.050529+0000","last_became_peered":"2024-08-19T01:32:39.050529+0000","last_unstale":"2024-08-19T01:34:45.498277+0000","last_undegraded":"2024-08-19T01:34:45.498277+0000","last_fullsized":"2024-08-19T01:34:45.498277+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:32:37.889224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:32:37.889224+0000","last_clean_scrub_stamp":"2024-08-19T01:32:37.889224+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T12:52:52.788360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":114,"num_read_kb":114,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"86'8","reported_seq":26,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:34:04.387624+0000","last_change":"2024-08-19T01:32:41.237093+0000","last_active":"2024-08-19T01:34:04.387624+0000","last_peered":"2024-08-19T01:34:04.387624+0000","last_clean":"2024-08-19T01:34:04.387624+0000","last_became_active":"2024-08-19T01:32:41.236738+0000","last_became_peered":"2024-08-19T01:32:41.236738+0000","last_unstale":"2024-08-19T01:34:04.387624+0000","last_undegraded":"2024-08-19T01:34:04.387624+0000","last_fullsized":"2024-08-19T01:34:04.387624+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_clean_scrub_stamp":"2024-08-19T01:32:40.202519+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T08:37:33.117344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"84'1","reported_seq":23,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:33:05.473189+0000","last_change":"2024-08-19T01:32:38.915010+0000","last_active":"2024-08-19T01:33:05.473189+0000","last_peered":"2024-08-19T01:33:05.473189+0000","last_clean":"2024-08-19T01:33:05.473189+0000","last_became_active":"2024-08-19T01:32:38.914687+0000","last_became_peered":"2024-08-19T01:32:38.914687+0000","last_unstale":"2024-08-19T01:33:05.473189+0000","last_undegraded":"2024-08-19T01:33:05.473189+0000","last_fullsized":"2024-08-19T01:33:05.473189+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:32:37.889224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:32:37.889224+0000","last_clean_scrub_stamp":"2024-08-19T01:32:37.889224+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T04:46:07.881778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"86'7","reported_seq":24,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:34:04.397148+0000","last_change":"2024-08-19T01:32:41.244069+0000","last_active":"2024-08-19T01:34:04.397148+0000","last_peered":"2024-08-19T01:34:04.397148+0000","last_clean":"2024-08-19T01:34:04.397148+0000","last_became_active":"2024-08-19T01:32:41.243482+0000","last_became_peered":"2024-08-19T01:32:41.243482+0000","last_unstale":"2024-08-19T01:34:04.397148+0000","last_undegraded":"2024-08-19T01:34:04.397148+0000","last_fullsized":"2024-08-19T01:34:04.397148+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_clean_scrub_stamp":"2024-08-19T01:32:40.202519+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T02:08:28.044421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":22,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:33:05.466068+0000","last_change":"2024-08-19T01:32:39.051383+0000","last_active":"2024-08-19T01:33:05.466068+0000","last_peered":"2024-08-19T01:33:05.466068+0000","last_clean":"2024-08-19T01:33:05.466068+0000","last_became_active":"2024-08-19T01:32:39.050970+0000","last_became_peered":"2024-08-19T01:32:39.050970+0000","last_unstale":"2024-08-19T01:33:05.466068+0000","last_undegraded":"2024-08-19T01:33:05.466068+0000","last_fullsized":"2024-08-19T01:33:05.466068+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:32:37.889224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:32:37.889224+0000","last_clean_scrub_stamp":"2024-08-19T01:32:37.889224+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T11:31:30.461651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"86'5","reported_seq":22,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:34:04.395486+0000","last_change":"2024-08-19T01:32:41.232656+0000","last_active":"2024-08-19T01:34:04.395486+0000","last_peered":"2024-08-19T01:34:04.395486+0000","last_clean":"2024-08-19T01:34:04.395486+0000","last_became_active":"2024-08-19T01:32:41.232274+0000","last_became_peered":"2024-08-19T01:32:41.232274+0000","last_unstale":"2024-08-19T01:34:04.395486+0000","last_undegraded":"2024-08-19T01:34:04.395486+0000","last_fullsized":"2024-08-19T01:34:04.395486+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_clean_scrub_stamp":"2024-08-19T01:32:40.202519+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T03:38:40.695652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.5","version":"84'1","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:33:05.472646+0000","last_change":"2024-08-19T01:32:39.050809+0000","last_active":"2024-08-19T01:33:05.472646+0000","last_peered":"2024-08-19T01:33:05.472646+0000","last_clean":"2024-08-19T01:33:05.472646+0000","last_became_active":"2024-08-19T01:32:39.050434+0000","last_became_peered":"2024-08-19T01:32:39.050434+0000","last_unstale":"2024-08-19T01:33:05.472646+0000","last_undegraded":"2024-08-19T01:33:05.472646+0000","last_fullsized":"2024-08-19T01:33:05.472646+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:32:37.889224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:32:37.889224+0000","last_clean_scrub_stamp":"2024-08-19T01:32:37.889224+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T13:04:28.828117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"86'9","reported_seq":27,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:34:04.391061+0000","last_change":"2024-08-19T01:32:41.228485+0000","last_active":"2024-08-19T01:34:04.391061+0000","last_peered":"2024-08-19T01:34:04.391061+0000","last_clean":"2024-08-19T01:34:04.391061+0000","last_became_active":"2024-08-19T01:32:41.227238+0000","last_became_peered":"2024-08-19T01:32:41.227238+0000","last_unstale":"2024-08-19T01:34:04.391061+0000","last_undegraded":"2024-08-19T01:34:04.391061+0000","last_fullsized":"2024-08-19T01:34:04.391061+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_clean_scrub_stamp":"2024-08-19T01:32:40.202519+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T10:55:55.939382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"86'13","reported_seq":35,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:34:05.234056+0000","last_change":"2024-08-19T01:32:38.915994+0000","last_active":"2024-08-19T01:34:05.234056+0000","last_peered":"2024-08-19T01:34:05.234056+0000","last_clean":"2024-08-19T01:34:05.234056+0000","last_became_active":"2024-08-19T01:32:38.914714+0000","last_became_peered":"2024-08-19T01:32:38.914714+0000","last_unstale":"2024-08-19T01:34:05.234056+0000","last_undegraded":"2024-08-19T01:34:05.234056+0000","last_fullsized":"2024-08-19T01:34:05.234056+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:32:37.889224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:32:37.889224+0000","last_clean_scrub_stamp":"2024-08-19T01:32:37.889224+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T05:38:07.274274+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":25,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"86'5","reported_seq":22,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:34:04.386255+0000","last_change":"2024-08-19T01:32:41.236611+0000","last_active":"2024-08-19T01:34:04.386255+0000","last_peered":"2024-08-19T01:34:04.386255+0000","last_clean":"2024-08-19T01:34:04.386255+0000","last_became_active":"2024-08-19T01:32:41.233821+0000","last_became_peered":"2024-08-19T01:32:41.233821+0000","last_unstale":"2024-08-19T01:34:04.386255+0000","last_undegraded":"2024-08-19T01:34:04.386255+0000","last_fullsized":"2024-08-19T01:34:04.386255+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_clean_scrub_stamp":"2024-08-19T01:32:40.202519+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T11:34:07.879034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"84'4","reported_seq":26,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:33:05.470567+0000","last_change":"2024-08-19T01:32:39.245411+0000","last_active":"2024-08-19T01:33:05.470567+0000","last_peered":"2024-08-19T01:33:05.470567+0000","last_clean":"2024-08-19T01:33:05.470567+0000","last_became_active":"2024-08-19T01:32:39.244980+0000","last_became_peered":"2024-08-19T01:32:39.244980+0000","last_unstale":"2024-08-19T01:33:05.470567+0000","last_undegraded":"2024-08-19T01:33:05.470567+0000","last_fullsized":"2024-08-19T01:33:05.470567+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:32:37.889224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:32:37.889224+0000","last_clean_scrub_stamp":"2024-08-19T01:32:37.889224+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T02:06:57.878051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"86'7","reported_seq":30,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:34:04.395910+0000","last_change":"2024-08-19T01:33:05.478955+0000","last_active":"2024-08-19T01:34:04.395910+0000","last_peered":"2024-08-19T01:34:04.395910+0000","last_clean":"2024-08-19T01:34:04.395910+0000","last_became_active":"2024-08-19T01:33:05.478446+0000","last_became_peered":"2024-08-19T01:33:05.478446+0000","last_unstale":"2024-08-19T01:34:04.395910+0000","last_undegraded":"2024-08-19T01:34:04.395910+0000","last_fullsized":"2024-08-19T01:34:04.395910+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_clean_scrub_stamp":"2024-08-19T01:32:40.202519+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T11:58:24.285438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,6],"acting":[0,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"84'2","reported_seq":24,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:33:05.701553+0000","last_change":"2024-08-19T01:32:39.050100+0000","last_active":"2024-08-19T01:33:05.701553+0000","last_peered":"2024-08-19T01:33:05.701553+0000","last_clean":"2024-08-19T01:33:05.701553+0000","last_became_active":"2024-08-19T01:32:39.049807+0000","last_became_peered":"2024-08-19T01:32:39.049807+0000","last_unstale":"2024-08-19T01:33:05.701553+0000","last_undegraded":"2024-08-19T01:33:05.701553+0000","last_fullsized":"2024-08-19T01:33:05.701553+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:32:37.889224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:32:37.889224+0000","last_clean_scrub_stamp":"2024-08-19T01:32:37.889224+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T07:06:22.361609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"86'2","reported_seq":19,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:34:04.355286+0000","last_change":"2024-08-19T01:32:41.228982+0000","last_active":"2024-08-19T01:34:04.355286+0000","last_peered":"2024-08-19T01:34:04.355286+0000","last_clean":"2024-08-19T01:34:04.355286+0000","last_became_active":"2024-08-19T01:32:41.228562+0000","last_became_peered":"2024-08-19T01:32:41.228562+0000","last_unstale":"2024-08-19T01:34:04.355286+0000","last_undegraded":"2024-08-19T01:34:04.355286+0000","last_fullsized":"2024-08-19T01:34:04.355286+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_clean_scrub_stamp":"2024-08-19T01:32:40.202519+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T03:14:42.480359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"84'3","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:33:05.471722+0000","last_change":"2024-08-19T01:32:38.913677+0000","last_active":"2024-08-19T01:33:05.471722+0000","last_peered":"2024-08-19T01:33:05.471722+0000","last_clean":"2024-08-19T01:33:05.471722+0000","last_became_active":"2024-08-19T01:32:38.913091+0000","last_became_peered":"2024-08-19T01:32:38.913091+0000","last_unstale":"2024-08-19T01:33:05.471722+0000","last_undegraded":"2024-08-19T01:33:05.471722+0000","last_fullsized":"2024-08-19T01:33:05.471722+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:32:37.889224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:32:37.889224+0000","last_clean_scrub_stamp":"2024-08-19T01:32:37.889224+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T12:39:16.489246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"86'9","reported_seq":26,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:34:04.395672+0000","last_change":"2024-08-19T01:32:41.239380+0000","last_active":"2024-08-19T01:34:04.395672+0000","last_peered":"2024-08-19T01:34:04.395672+0000","last_clean":"2024-08-19T01:34:04.395672+0000","last_became_active":"2024-08-19T01:32:41.239060+0000","last_became_peered":"2024-08-19T01:32:41.239060+0000","last_unstale":"2024-08-19T01:34:04.395672+0000","last_undegraded":"2024-08-19T01:34:04.395672+0000","last_fullsized":"2024-08-19T01:34:04.395672+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_clean_scrub_stamp":"2024-08-19T01:32:40.202519+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T02:21:59.630705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"86'2","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:33:07.449044+0000","last_change":"2024-08-19T01:33:05.478578+0000","last_active":"2024-08-19T01:33:07.449044+0000","last_peered":"2024-08-19T01:33:07.449044+0000","last_clean":"2024-08-19T01:33:07.449044+0000","last_became_active":"2024-08-19T01:33:05.473433+0000","last_became_peered":"2024-08-19T01:33:05.473433+0000","last_unstale":"2024-08-19T01:33:07.449044+0000","last_undegraded":"2024-08-19T01:33:07.449044+0000","last_fullsized":"2024-08-19T01:33:07.449044+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:32:37.889224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:32:37.889224+0000","last_clean_scrub_stamp":"2024-08-19T01:32:37.889224+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T05:06:29.975613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,3,8],"acting":[11,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.b","version":"86'8","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:34:04.393696+0000","last_change":"2024-08-19T01:32:41.236012+0000","last_active":"2024-08-19T01:34:04.393696+0000","last_peered":"2024-08-19T01:34:04.393696+0000","last_clean":"2024-08-19T01:34:04.393696+0000","last_became_active":"2024-08-19T01:32:41.235724+0000","last_became_peered":"2024-08-19T01:32:41.235724+0000","last_unstale":"2024-08-19T01:34:04.393696+0000","last_undegraded":"2024-08-19T01:34:04.393696+0000","last_fullsized":"2024-08-19T01:34:04.393696+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_clean_scrub_stamp":"2024-08-19T01:32:40.202519+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T08:45:08.865144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"84'4","reported_seq":27,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:33:05.471339+0000","last_change":"2024-08-19T01:32:38.915170+0000","last_active":"2024-08-19T01:33:05.471339+0000","last_peered":"2024-08-19T01:33:05.471339+0000","last_clean":"2024-08-19T01:33:05.471339+0000","last_became_active":"2024-08-19T01:32:38.914812+0000","last_became_peered":"2024-08-19T01:32:38.914812+0000","last_unstale":"2024-08-19T01:33:05.471339+0000","last_undegraded":"2024-08-19T01:33:05.471339+0000","last_fullsized":"2024-08-19T01:33:05.471339+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:32:37.889224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:32:37.889224+0000","last_clean_scrub_stamp":"2024-08-19T01:32:37.889224+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T11:46:39.409065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"86'5","reported_seq":22,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:34:04.396323+0000","last_change":"2024-08-19T01:32:41.234133+0000","last_active":"2024-08-19T01:34:04.396323+0000","last_peered":"2024-08-19T01:34:04.396323+0000","last_clean":"2024-08-19T01:34:04.396323+0000","last_became_active":"2024-08-19T01:32:41.233492+0000","last_became_peered":"2024-08-19T01:32:41.233492+0000","last_unstale":"2024-08-19T01:34:04.396323+0000","last_undegraded":"2024-08-19T01:34:04.396323+0000","last_fullsized":"2024-08-19T01:34:04.396323+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_clean_scrub_stamp":"2024-08-19T01:32:40.202519+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T06:11:12.587678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"86'15","reported_seq":32,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:34:04.395111+0000","last_change":"2024-08-19T01:32:41.232628+0000","last_active":"2024-08-19T01:34:04.395111+0000","last_peered":"2024-08-19T01:34:04.395111+0000","last_clean":"2024-08-19T01:34:04.395111+0000","last_became_active":"2024-08-19T01:32:41.232295+0000","last_became_peered":"2024-08-19T01:32:41.232295+0000","last_unstale":"2024-08-19T01:34:04.395111+0000","last_undegraded":"2024-08-19T01:34:04.395111+0000","last_fullsized":"2024-08-19T01:34:04.395111+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_clean_scrub_stamp":"2024-08-19T01:32:40.202519+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T02:01:09.490553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"84'2","reported_seq":24,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:33:05.472666+0000","last_change":"2024-08-19T01:32:38.912000+0000","last_active":"2024-08-19T01:33:05.472666+0000","last_peered":"2024-08-19T01:33:05.472666+0000","last_clean":"2024-08-19T01:33:05.472666+0000","last_became_active":"2024-08-19T01:32:38.911535+0000","last_became_peered":"2024-08-19T01:32:38.911535+0000","last_unstale":"2024-08-19T01:33:05.472666+0000","last_undegraded":"2024-08-19T01:33:05.472666+0000","last_fullsized":"2024-08-19T01:33:05.472666+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:32:37.889224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:32:37.889224+0000","last_clean_scrub_stamp":"2024-08-19T01:32:37.889224+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T07:12:49.358384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"86'9","reported_seq":26,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:34:04.399042+0000","last_change":"2024-08-19T01:32:41.233507+0000","last_active":"2024-08-19T01:34:04.399042+0000","last_peered":"2024-08-19T01:34:04.399042+0000","last_clean":"2024-08-19T01:34:04.399042+0000","last_became_active":"2024-08-19T01:32:41.233179+0000","last_became_peered":"2024-08-19T01:32:41.233179+0000","last_unstale":"2024-08-19T01:34:04.399042+0000","last_undegraded":"2024-08-19T01:34:04.399042+0000","last_fullsized":"2024-08-19T01:34:04.399042+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_clean_scrub_stamp":"2024-08-19T01:32:40.202519+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T11:47:44.780859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1d","version":"84'4","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:33:05.483750+0000","last_change":"2024-08-19T01:33:05.483322+0000","last_active":"2024-08-19T01:33:05.483750+0000","last_peered":"2024-08-19T01:33:05.483750+0000","last_clean":"2024-08-19T01:33:05.483750+0000","last_became_active":"2024-08-19T01:33:05.475496+0000","last_became_peered":"2024-08-19T01:33:05.475496+0000","last_unstale":"2024-08-19T01:33:05.483750+0000","last_undegraded":"2024-08-19T01:33:05.483750+0000","last_fullsized":"2024-08-19T01:33:05.483750+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:32:37.889224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:32:37.889224+0000","last_clean_scrub_stamp":"2024-08-19T01:32:37.889224+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T01:57:48.143495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":10,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,7],"acting":[11,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1f","version":"86'6","reported_seq":23,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:34:04.399271+0000","last_change":"2024-08-19T01:32:41.244641+0000","last_active":"2024-08-19T01:34:04.399271+0000","last_peered":"2024-08-19T01:34:04.399271+0000","last_clean":"2024-08-19T01:34:04.399271+0000","last_became_active":"2024-08-19T01:32:41.244311+0000","last_became_peered":"2024-08-19T01:32:41.244311+0000","last_unstale":"2024-08-19T01:34:04.399271+0000","last_undegraded":"2024-08-19T01:34:04.399271+0000","last_fullsized":"2024-08-19T01:34:04.399271+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_clean_scrub_stamp":"2024-08-19T01:32:40.202519+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T10:39:12.591092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"84'2","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:33:05.472403+0000","last_change":"2024-08-19T01:32:39.051314+0000","last_active":"2024-08-19T01:33:05.472403+0000","last_peered":"2024-08-19T01:33:05.472403+0000","last_clean":"2024-08-19T01:33:05.472403+0000","last_became_active":"2024-08-19T01:32:39.050998+0000","last_became_peered":"2024-08-19T01:32:39.050998+0000","last_unstale":"2024-08-19T01:33:05.472403+0000","last_undegraded":"2024-08-19T01:33:05.472403+0000","last_fullsized":"2024-08-19T01:33:05.472403+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:32:37.889224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:32:37.889224+0000","last_clean_scrub_stamp":"2024-08-19T01:32:37.889224+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T09:21:15.689149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"86'5","reported_seq":23,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:34:04.390896+0000","last_change":"2024-08-19T01:32:41.234762+0000","last_active":"2024-08-19T01:34:04.390896+0000","last_peered":"2024-08-19T01:34:04.390896+0000","last_clean":"2024-08-19T01:34:04.390896+0000","last_became_active":"2024-08-19T01:32:41.234471+0000","last_became_peered":"2024-08-19T01:32:41.234471+0000","last_unstale":"2024-08-19T01:34:04.390896+0000","last_undegraded":"2024-08-19T01:34:04.390896+0000","last_fullsized":"2024-08-19T01:34:04.390896+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:32:40.202519+0000","last_clean_scrub_stamp":"2024-08-19T01:32:40.202519+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T08:34:33.110938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"84'9","reported_seq":32,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:33:05.466470+0000","last_change":"2024-08-19T01:32:38.917068+0000","last_active":"2024-08-19T01:33:05.466470+0000","last_peered":"2024-08-19T01:33:05.466470+0000","last_clean":"2024-08-19T01:33:05.466470+0000","last_became_active":"2024-08-19T01:32:38.915399+0000","last_became_peered":"2024-08-19T01:32:38.915399+0000","last_unstale":"2024-08-19T01:33:05.466470+0000","last_undegraded":"2024-08-19T01:33:05.466470+0000","last_fullsized":"2024-08-19T01:33:05.466470+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:32:37.889224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:32:37.889224+0000","last_clean_scrub_stamp":"2024-08-19T01:32:37.889224+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T04:17:25.061208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":502,"ondisk_log_size":502,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":71643,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":707,"num_read_kb":432,"num_write":188,"num_write_kb":133,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":1012,"num_keys_recovered":10,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":430080,"data_stored":214929,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":159,"ondisk_log_size":159,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4751360,"data_stored":4722944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8}],"osd_stats":[{"osd":11,"up_from":78,"seq":335007449126,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32092,"kb_used_data":916,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713060,"statfs":{"total":95995035648,"available":95962173440,"internally_reserved":0,"allocated":937984,"data_stored":451051,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":71,"seq":304942678059,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28616,"kb_used_data":1600,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716536,"statfs":{"total":95995035648,"available":95965732864,"internally_reserved":0,"allocated":1638400,"data_stored":1109644,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":66,"seq":283467841583,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32108,"kb_used_data":932,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713044,"statfs":{"total":95995035648,"available":95962157056,"internally_reserved":0,"allocated":954368,"data_stored":451287,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":58,"seq":249108103221,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32108,"kb_used_data":932,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713044,"statfs":{"total":95995035648,"available":95962157056,"internally_reserved":0,"allocated":954368,"data_stored":451207,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":51,"seq":219043332153,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32688,"kb_used_data":1512,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712464,"statfs":{"total":95995035648,"available":95961563136,"internally_reserved":0,"allocated":1548288,"data_stored":1044119,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":46,"seq":197568495679,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27936,"kb_used_data":920,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717216,"statfs":{"total":95995035648,"available":95966429184,"internally_reserved":0,"allocated":942080,"data_stored":450676,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":38,"seq":163208757315,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28540,"kb_used_data":1524,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716612,"statfs":{"total":95995035648,"available":95965810688,"internally_reserved":0,"allocated":1560576,"data_stored":1043351,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":33,"seq":141733920839,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32108,"kb_used_data":932,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713044,"statfs":{"total":95995035648,"available":95962157056,"internally_reserved":0,"allocated":954368,"data_stored":453888,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182476,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27952,"kb_used_data":936,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717200,"statfs":{"total":95995035648,"available":95966412800,"internally_reserved":0,"allocated":958464,"data_stored":450987,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411410,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36336,"kb_used_data":1000,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708816,"statfs":{"total":95995035648,"available":95957827584,"internally_reserved":0,"allocated":1024000,"data_stored":517077,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574932,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28000,"kb_used_data":984,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717152,"statfs":{"total":95995035648,"available":95966363648,"internally_reserved":0,"allocated":1007616,"data_stored":516349,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738458,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36276,"kb_used_data":940,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708876,"statfs":{"total":95995035648,"available":95957889024,"internally_reserved":0,"allocated":962560,"data_stored":451947,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":65880,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":66608,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":3419,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":2397,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":3945,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":106496,"data_stored":68690,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-08-19T01:34:46.649 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833 shell --fsid bad8780e-5dc9-11ef-bccf-c7b262605968 -- ceph fs dump --format=json 2024-08-19T01:34:47.082 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/bad8780e-5dc9-11ef-bccf-c7b262605968/mon.a/config 2024-08-19T01:34:47.674 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:47.652+0000 7f5577200640 1 -- 172.21.15.83:0/892274184 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5578074350 msgr2=0x7f5578074730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:34:47.674 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:47.652+0000 7f5577200640 1 --2- 172.21.15.83:0/892274184 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5578074350 0x7f5578074730 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f5568009900 tx=0x7f556802a480 comp rx=0 tx=0).stop 2024-08-19T01:34:47.674 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:47.652+0000 7f5577200640 1 -- 172.21.15.83:0/892274184 shutdown_connections 2024-08-19T01:34:47.674 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:47.652+0000 7f5577200640 1 --2- 172.21.15.83:0/892274184 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f557810df40 0x7f5578110390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:47.674 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:47.652+0000 7f5577200640 1 --2- 172.21.15.83:0/892274184 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5578074c70 0x7f557810da00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:47.675 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:47.652+0000 7f5577200640 1 --2- 172.21.15.83:0/892274184 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5578074350 0x7f5578074730 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:47.675 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:47.652+0000 7f5577200640 1 -- 172.21.15.83:0/892274184 >> 172.21.15.83:0/892274184 conn(0x7f55780fd440 msgr2=0x7f55780ff860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:34:47.675 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:47.652+0000 7f5577200640 1 -- 172.21.15.83:0/892274184 shutdown_connections 2024-08-19T01:34:47.675 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:47.652+0000 7f5577200640 1 -- 172.21.15.83:0/892274184 wait complete. 2024-08-19T01:34:47.675 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:47.653+0000 7f5577200640 1 Processor -- start 2024-08-19T01:34:47.675 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:47.654+0000 7f5577200640 1 -- start start 2024-08-19T01:34:47.675 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:47.654+0000 7f5577200640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5578074350 0x7f557819a1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:34:47.675 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:47.654+0000 7f5577200640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5578074c70 0x7f557819a6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:34:47.675 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:47.654+0000 7f5577200640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f557810df40 0x7f557819ea80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:34:47.675 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:47.654+0000 7f5577200640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f5578113cf0 con 0x7f5578074c70 2024-08-19T01:34:47.675 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:47.654+0000 7f5577200640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f5578113b70 con 0x7f557810df40 2024-08-19T01:34:47.675 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:47.654+0000 7f5577200640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f5578113e70 con 0x7f5578074350 2024-08-19T01:34:47.675 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:47.654+0000 7f5575e00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5578074350 0x7f557819a1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:34:47.676 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:47.654+0000 7f5575400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5578074c70 0x7f557819a6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:34:47.676 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:47.654+0000 7f5575e00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5578074350 0x7f557819a1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.83:47584/0 (socket says 172.21.15.83:47584) 2024-08-19T01:34:47.676 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:47.654+0000 7f5575e00640 1 -- 172.21.15.83:0/1046216372 learned_addr learned my addr 172.21.15.83:0/1046216372 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-08-19T01:34:47.676 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:47.654+0000 7f5576800640 1 --2- 172.21.15.83:0/1046216372 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f557810df40 0x7f557819ea80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:34:47.676 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:47.655+0000 7f5575400640 1 -- 172.21.15.83:0/1046216372 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5578074350 msgr2=0x7f557819a1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:34:47.676 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:47.655+0000 7f5575400640 1 --2- 172.21.15.83:0/1046216372 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5578074350 0x7f557819a1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:47.676 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:47.655+0000 7f5575400640 1 -- 172.21.15.83:0/1046216372 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f557810df40 msgr2=0x7f557819ea80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:34:47.676 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:47.655+0000 7f5575400640 1 --2- 172.21.15.83:0/1046216372 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f557810df40 0x7f557819ea80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:47.676 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:47.655+0000 7f5575400640 1 -- 172.21.15.83:0/1046216372 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f557819f160 con 0x7f5578074c70 2024-08-19T01:34:47.676 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:47.655+0000 7f5575400640 1 --2- 172.21.15.83:0/1046216372 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5578074c70 0x7f557819a6f0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f556400e9f0 tx=0x7f556400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:34:47.676 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:47.655+0000 7f556ea00640 1 -- 172.21.15.83:0/1046216372 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f556400ccc0 con 0x7f5578074c70 2024-08-19T01:34:47.676 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:47.655+0000 7f556ea00640 1 -- 172.21.15.83:0/1046216372 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5564010910 con 0x7f5578074c70 2024-08-19T01:34:47.676 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:47.655+0000 7f5577200640 1 -- 172.21.15.83:0/1046216372 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f557819f450 con 0x7f5578074c70 2024-08-19T01:34:47.677 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:47.656+0000 7f556ea00640 1 -- 172.21.15.83:0/1046216372 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5564018960 con 0x7f5578074c70 2024-08-19T01:34:47.677 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:47.656+0000 7f5577200640 1 -- 172.21.15.83:0/1046216372 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f55781a6d30 con 0x7f5578074c70 2024-08-19T01:34:47.677 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:47.657+0000 7f5577200640 1 -- 172.21.15.83:0/1046216372 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5578074730 con 0x7f5578074c70 2024-08-19T01:34:47.677 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:47.660+0000 7f556ea00640 1 -- 172.21.15.83:0/1046216372 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 19) ==== 137884+0+0 (secure 0 0 0) 0x7f5564010ac0 con 0x7f5578074c70 2024-08-19T01:34:47.677 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:47.661+0000 7f556ea00640 1 --2- 172.21.15.83:0/1046216372 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f555c0a2c90 0x7f555c0a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:34:47.677 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:47.661+0000 7f556ea00640 1 -- 172.21.15.83:0/1046216372 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8269+0+0 (secure 0 0 0) 0x7f55640cf9c0 con 0x7f5578074c70 2024-08-19T01:34:47.677 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:47.661+0000 7f5575e00640 1 --2- 172.21.15.83:0/1046216372 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f555c0a2c90 0x7f555c0a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:34:47.677 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:47.662+0000 7f5575e00640 1 --2- 172.21.15.83:0/1046216372 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f555c0a2c90 0x7f555c0a5150 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f55680096c0 tx=0x7f5568005a20 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:34:47.677 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:47.663+0000 7f556ea00640 1 -- 172.21.15.83:0/1046216372 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7f556409c150 con 0x7f5578074c70 2024-08-19T01:34:47.877 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:47.875+0000 7f5577200640 1 -- 172.21.15.83:0/1046216372 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f557819b510 con 0x7f5578074c70 2024-08-19T01:34:47.877 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:34:47 smithi083 ceph-mon[17120]: from='client.15039 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-08-19T01:34:47.878 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:34:47 smithi083 ceph-mon[17120]: from='client.25084 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-08-19T01:34:47.878 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:34:47 smithi083 ceph-mon[17120]: pgmap v337: 97 pgs: 97 active+clean; 647 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 1.3 KiB/s wr, 8 op/s 2024-08-19T01:34:47.878 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:47.876+0000 7f556ea00640 1 -- 172.21.15.83:0/1046216372 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+11920 (secure 0 0 0) 0x7f557819b510 con 0x7f5578074c70 2024-08-19T01:34:47.883 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-08-19T01:34:47.883 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":25,"btime":"2024-08-19T01:34:41:456953+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14421,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6835/988556067","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":988556067},{"type":"v1","addr":"172.21.15.83:6835","nonce":988556067}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14433,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6837/869702632","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6836","nonce":869702632},{"type":"v1","addr":"172.21.15.83:6837","nonce":869702632}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14475,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6841/1795975999","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6840","nonce":1795975999},{"type":"v1","addr":"172.21.15.83:6841","nonce":1795975999}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24430,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6833/2505076046","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6832","nonce":2505076046},{"type":"v1","addr":"172.21.15.167:6833","nonce":2505076046}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24449,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6833/3557520478","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":3557520478},{"type":"v1","addr":"172.21.15.117:6833","nonce":3557520478}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24466,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6835/3482045966","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":3482045966},{"type":"v1","addr":"172.21.15.167:6835","nonce":3482045966}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7}],"filesystems":[{"mdsmap":{"epoch":25,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-08-19T01:32:42.054769+0000","modified":"2024-08-19T01:34:40.528901+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24545,"mds_1":24527,"mds_2":24509},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24478":{"gid":24478,"name":"c","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.167:6837/1762130408","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6836","nonce":1762130408},{"type":"v1","addr":"172.21.15.167:6837","nonce":1762130408}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24485":{"gid":24485,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.117:6835/1024213994","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":1024213994},{"type":"v1","addr":"172.21.15.117:6835","nonce":1024213994}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24496":{"gid":24496,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.167:6839/3507387682","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6838","nonce":3507387682},{"type":"v1","addr":"172.21.15.167:6839","nonce":3507387682}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24509":{"gid":24509,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":10,"addr":"172.21.15.117:6837/3185224684","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":3185224684},{"type":"v1","addr":"172.21.15.117:6837","nonce":3185224684}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24527":{"gid":24527,"name":"g","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.83:6839/3239261845","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6838","nonce":3239261845},{"type":"v1","addr":"172.21.15.83:6839","nonce":3239261845}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24545":{"gid":24545,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.117:6839/1967595704","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6838","nonce":1967595704},{"type":"v1","addr":"172.21.15.117:6839","nonce":1967595704}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24545,"qdb_cluster":[24545,24527,24509]},"id":1}]} 2024-08-19T01:34:47.883 INFO:teuthology.orchestra.run.smithi083.stderr:dumped fsmap epoch 25 2024-08-19T01:34:47.884 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:47.878+0000 7f5577200640 1 -- 172.21.15.83:0/1046216372 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f555c0a2c90 msgr2=0x7f555c0a5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:34:47.884 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:47.878+0000 7f5577200640 1 --2- 172.21.15.83:0/1046216372 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f555c0a2c90 0x7f555c0a5150 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f55680096c0 tx=0x7f5568005a20 comp rx=0 tx=0).stop 2024-08-19T01:34:47.885 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:47.878+0000 7f5577200640 1 -- 172.21.15.83:0/1046216372 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5578074c70 msgr2=0x7f557819a6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:34:47.885 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:47.878+0000 7f5577200640 1 --2- 172.21.15.83:0/1046216372 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5578074c70 0x7f557819a6f0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f556400e9f0 tx=0x7f556400eeb0 comp rx=0 tx=0).stop 2024-08-19T01:34:47.885 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:47.878+0000 7f5577200640 1 -- 172.21.15.83:0/1046216372 shutdown_connections 2024-08-19T01:34:47.885 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:47.878+0000 7f5577200640 1 --2- 172.21.15.83:0/1046216372 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f555c0a2c90 0x7f555c0a5150 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:47.885 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:47.878+0000 7f5577200640 1 --2- 172.21.15.83:0/1046216372 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f557810df40 0x7f557819ea80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:47.885 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:47.878+0000 7f5577200640 1 --2- 172.21.15.83:0/1046216372 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5578074c70 0x7f557819a6f0 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:47.885 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:47.878+0000 7f5577200640 1 --2- 172.21.15.83:0/1046216372 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5578074350 0x7f557819a1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:47.885 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:47.878+0000 7f5577200640 1 -- 172.21.15.83:0/1046216372 >> 172.21.15.83:0/1046216372 conn(0x7f55780fd440 msgr2=0x7f55780fe0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:34:47.885 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:47.879+0000 7f5577200640 1 -- 172.21.15.83:0/1046216372 shutdown_connections 2024-08-19T01:34:47.885 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:47.879+0000 7f5577200640 1 -- 172.21.15.83:0/1046216372 wait complete. 2024-08-19T01:34:47.957 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:34:47 smithi117 ceph-mon[22016]: from='client.15039 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-08-19T01:34:47.957 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:34:47 smithi117 ceph-mon[22016]: from='client.25084 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-08-19T01:34:47.957 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:34:47 smithi117 ceph-mon[22016]: pgmap v337: 97 pgs: 97 active+clean; 647 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 1.3 KiB/s wr, 8 op/s 2024-08-19T01:34:47.966 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:34:47 smithi167 ceph-mon[21875]: from='client.15039 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-08-19T01:34:47.966 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:34:47 smithi167 ceph-mon[21875]: from='client.25084 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-08-19T01:34:47.966 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:34:47 smithi167 ceph-mon[21875]: pgmap v337: 97 pgs: 97 active+clean; 647 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 1.3 KiB/s wr, 8 op/s 2024-08-19T01:34:48.654 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:34:48 smithi083 ceph-mon[17120]: from='client.25118 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-08-19T01:34:48.655 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:34:48 smithi083 ceph-mon[17120]: from='client.? 172.21.15.83:0/1046216372' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-19T01:34:48.920 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833 shell --fsid bad8780e-5dc9-11ef-bccf-c7b262605968 -- ceph osd dump --format=json 2024-08-19T01:34:48.952 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:34:48 smithi117 ceph-mon[22016]: from='client.25118 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-08-19T01:34:48.953 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:34:48 smithi117 ceph-mon[22016]: from='client.? 172.21.15.83:0/1046216372' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-19T01:34:48.966 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:34:48 smithi167 ceph-mon[21875]: from='client.25118 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-08-19T01:34:48.966 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:34:48 smithi167 ceph-mon[21875]: from='client.? 172.21.15.83:0/1046216372' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-19T01:34:49.303 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/bad8780e-5dc9-11ef-bccf-c7b262605968/mon.a/config 2024-08-19T01:34:49.766 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:34:49 smithi083 ceph-mon[17120]: pgmap v338: 97 pgs: 97 active+clean; 647 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 1.4 KiB/s wr, 7 op/s 2024-08-19T01:34:49.854 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:34:49 smithi117 ceph-mon[22016]: pgmap v338: 97 pgs: 97 active+clean; 647 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 1.4 KiB/s wr, 7 op/s 2024-08-19T01:34:49.867 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:34:49 smithi167 ceph-mon[21875]: pgmap v338: 97 pgs: 97 active+clean; 647 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 1.4 KiB/s wr, 7 op/s 2024-08-19T01:34:49.877 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:49.875+0000 7f5f3a800640 1 -- 172.21.15.83:0/1650083852 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5f341020a0 msgr2=0x7f5f3410e5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:34:49.877 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:49.875+0000 7f5f3a800640 1 --2- 172.21.15.83:0/1650083852 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5f341020a0 0x7f5f3410e5d0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f5f2c009cd0 tx=0x7f5f2c02a8e0 comp rx=0 tx=0).stop 2024-08-19T01:34:49.877 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:49.875+0000 7f5f3a800640 1 -- 172.21.15.83:0/1650083852 shutdown_connections 2024-08-19T01:34:49.878 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:49.875+0000 7f5f3a800640 1 --2- 172.21.15.83:0/1650083852 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5f341020a0 0x7f5f3410e5d0 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:49.878 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:49.875+0000 7f5f3a800640 1 --2- 172.21.15.83:0/1650083852 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5f34101700 0x7f5f34101b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:49.878 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:49.875+0000 7f5f3a800640 1 --2- 172.21.15.83:0/1650083852 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5f34107700 0x7f5f34107ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:49.878 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:49.875+0000 7f5f3a800640 1 -- 172.21.15.83:0/1650083852 >> 172.21.15.83:0/1650083852 conn(0x7f5f340fd440 msgr2=0x7f5f340ff860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:34:49.878 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:49.876+0000 7f5f3a800640 1 -- 172.21.15.83:0/1650083852 shutdown_connections 2024-08-19T01:34:49.878 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:49.876+0000 7f5f3a800640 1 -- 172.21.15.83:0/1650083852 wait complete. 2024-08-19T01:34:49.879 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:49.877+0000 7f5f3a800640 1 Processor -- start 2024-08-19T01:34:49.879 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:49.877+0000 7f5f3a800640 1 -- start start 2024-08-19T01:34:49.879 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:49.877+0000 7f5f3a800640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5f34101700 0x7f5f34074220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:34:49.879 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:49.877+0000 7f5f3a800640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5f341020a0 0x7f5f34074760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:34:49.879 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:49.877+0000 7f5f3a800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5f34107700 0x7f5f34074ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:34:49.879 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:49.877+0000 7f5f3a800640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f5f34111b50 con 0x7f5f34107700 2024-08-19T01:34:49.879 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:49.877+0000 7f5f3a800640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f5f341119d0 con 0x7f5f34101700 2024-08-19T01:34:49.879 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:49.877+0000 7f5f3a800640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f5f34111cd0 con 0x7f5f341020a0 2024-08-19T01:34:49.880 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:49.877+0000 7f5f39e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5f34107700 0x7f5f34074ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:34:49.880 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:49.877+0000 7f5f39e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5f34107700 0x7f5f34074ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:57910/0 (socket says 172.21.15.83:57910) 2024-08-19T01:34:49.880 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:49.878+0000 7f5f39e00640 1 -- 172.21.15.83:0/178731420 learned_addr learned my addr 172.21.15.83:0/178731420 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-08-19T01:34:49.880 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:49.878+0000 7f5f39400640 1 --2- 172.21.15.83:0/178731420 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5f34101700 0x7f5f34074220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:34:49.880 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:49.878+0000 7f5f38a00640 1 --2- 172.21.15.83:0/178731420 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5f341020a0 0x7f5f34074760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:34:49.881 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:49.878+0000 7f5f39400640 1 -- 172.21.15.83:0/178731420 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5f341020a0 msgr2=0x7f5f34074760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:34:49.881 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:49.878+0000 7f5f39400640 1 --2- 172.21.15.83:0/178731420 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5f341020a0 0x7f5f34074760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:49.881 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:49.878+0000 7f5f39400640 1 -- 172.21.15.83:0/178731420 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5f34107700 msgr2=0x7f5f34074ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:34:49.881 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:49.878+0000 7f5f39400640 1 --2- 172.21.15.83:0/178731420 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5f34107700 0x7f5f34074ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:49.881 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:49.878+0000 7f5f39400640 1 -- 172.21.15.83:0/178731420 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5f341a6490 con 0x7f5f34101700 2024-08-19T01:34:49.881 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:49.878+0000 7f5f39400640 1 --2- 172.21.15.83:0/178731420 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5f34101700 0x7f5f34074220 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f5f2800e9f0 tx=0x7f5f2800eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:34:49.881 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:49.879+0000 7f5f32000640 1 -- 172.21.15.83:0/178731420 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f2800ccc0 con 0x7f5f34101700 2024-08-19T01:34:49.882 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:49.879+0000 7f5f3a800640 1 -- 172.21.15.83:0/178731420 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5f341a6780 con 0x7f5f34101700 2024-08-19T01:34:49.882 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:49.879+0000 7f5f32000640 1 -- 172.21.15.83:0/178731420 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5f28010910 con 0x7f5f34101700 2024-08-19T01:34:49.882 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:49.879+0000 7f5f32000640 1 -- 172.21.15.83:0/178731420 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f28018960 con 0x7f5f34101700 2024-08-19T01:34:49.882 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:49.879+0000 7f5f3a800640 1 -- 172.21.15.83:0/178731420 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5f341a6cc0 con 0x7f5f34101700 2024-08-19T01:34:49.882 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:49.880+0000 7f5f3a800640 1 -- 172.21.15.83:0/178731420 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5f34101b60 con 0x7f5f34101700 2024-08-19T01:34:49.883 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:49.882+0000 7f5f32000640 1 -- 172.21.15.83:0/178731420 <== mon.2 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 19) ==== 137884+0+0 (secure 0 0 0) 0x7f5f28010ac0 con 0x7f5f34101700 2024-08-19T01:34:49.885 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:49.883+0000 7f5f32000640 1 --2- 172.21.15.83:0/178731420 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f5f1c0a2c10 0x7f5f1c0a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:34:49.885 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:49.883+0000 7f5f32000640 1 -- 172.21.15.83:0/178731420 <== mon.2 v2:172.21.15.117:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8269+0+0 (secure 0 0 0) 0x7f5f280cf8f0 con 0x7f5f34101700 2024-08-19T01:34:49.885 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:49.883+0000 7f5f38a00640 1 --2- 172.21.15.83:0/178731420 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f5f1c0a2c10 0x7f5f1c0a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:34:49.885 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:49.884+0000 7f5f38a00640 1 --2- 172.21.15.83:0/178731420 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f5f1c0a2c10 0x7f5f1c0a50d0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f5f34075cb0 tx=0x7f5f20005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:34:49.887 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:49.885+0000 7f5f32000640 1 -- 172.21.15.83:0/178731420 <== mon.2 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7f5f2809c020 con 0x7f5f34101700 2024-08-19T01:34:50.061 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:50.059+0000 7f5f3a800640 1 -- 172.21.15.83:0/178731420 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f5f341a6f20 con 0x7f5f34101700 2024-08-19T01:34:50.063 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:50.061+0000 7f5f32000640 1 -- 172.21.15.83:0/178731420 <== mon.2 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v86) ==== 74+0+22722 (secure 0 0 0) 0x7f5f280a0320 con 0x7f5f34101700 2024-08-19T01:34:50.063 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-08-19T01:34:50.063 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":86,"fsid":"bad8780e-5dc9-11ef-bccf-c7b262605968","created":"2024-08-19T01:25:12.626841+0000","modified":"2024-08-19T01:33:05.461998+0000","last_up_change":"2024-08-19T01:31:47.254559+0000","last_in_change":"2024-08-19T01:31:27.652847+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-08-19T01:28:16.862007+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-08-19T01:32:37.455535+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-08-19T01:32:40.201935+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ffa74041-f9d2-4555-bb43-508b8c581487","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6802","nonce":1082526824},{"type":"v1","addr":"172.21.15.83:6803","nonce":1082526824}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6804","nonce":1082526824},{"type":"v1","addr":"172.21.15.83:6805","nonce":1082526824}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6808","nonce":1082526824},{"type":"v1","addr":"172.21.15.83:6809","nonce":1082526824}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6806","nonce":1082526824},{"type":"v1","addr":"172.21.15.83:6807","nonce":1082526824}]},"public_addr":"172.21.15.83:6803/1082526824","cluster_addr":"172.21.15.83:6805/1082526824","heartbeat_back_addr":"172.21.15.83:6809/1082526824","heartbeat_front_addr":"172.21.15.83:6807/1082526824","state":["exists","up"]},{"osd":1,"uuid":"1977570b-b4f5-44fd-bb1e-3a31318be649","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6800","nonce":1925419802},{"type":"v1","addr":"172.21.15.117:6801","nonce":1925419802}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6802","nonce":1925419802},{"type":"v1","addr":"172.21.15.117:6803","nonce":1925419802}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6806","nonce":1925419802},{"type":"v1","addr":"172.21.15.117:6807","nonce":1925419802}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6804","nonce":1925419802},{"type":"v1","addr":"172.21.15.117:6805","nonce":1925419802}]},"public_addr":"172.21.15.117:6801/1925419802","cluster_addr":"172.21.15.117:6803/1925419802","heartbeat_back_addr":"172.21.15.117:6807/1925419802","heartbeat_front_addr":"172.21.15.117:6805/1925419802","state":["exists","up"]},{"osd":2,"uuid":"a75278d1-161f-41d5-a92f-058d1077bf2a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6800","nonce":551625328},{"type":"v1","addr":"172.21.15.167:6801","nonce":551625328}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6802","nonce":551625328},{"type":"v1","addr":"172.21.15.167:6803","nonce":551625328}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6806","nonce":551625328},{"type":"v1","addr":"172.21.15.167:6807","nonce":551625328}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6804","nonce":551625328},{"type":"v1","addr":"172.21.15.167:6805","nonce":551625328}]},"public_addr":"172.21.15.167:6801/551625328","cluster_addr":"172.21.15.167:6803/551625328","heartbeat_back_addr":"172.21.15.167:6807/551625328","heartbeat_front_addr":"172.21.15.167:6805/551625328","state":["exists","up"]},{"osd":3,"uuid":"d1fb16f9-910e-46fa-b1e3-8e5bef4d346d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6810","nonce":3138942062},{"type":"v1","addr":"172.21.15.83:6811","nonce":3138942062}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6812","nonce":3138942062},{"type":"v1","addr":"172.21.15.83:6813","nonce":3138942062}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6816","nonce":3138942062},{"type":"v1","addr":"172.21.15.83:6817","nonce":3138942062}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6814","nonce":3138942062},{"type":"v1","addr":"172.21.15.83:6815","nonce":3138942062}]},"public_addr":"172.21.15.83:6811/3138942062","cluster_addr":"172.21.15.83:6813/3138942062","heartbeat_back_addr":"172.21.15.83:6817/3138942062","heartbeat_front_addr":"172.21.15.83:6815/3138942062","state":["exists","up"]},{"osd":4,"uuid":"8176ed18-1fe2-446d-8994-8c60f6e08f38","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6808","nonce":3978863827},{"type":"v1","addr":"172.21.15.117:6809","nonce":3978863827}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6810","nonce":3978863827},{"type":"v1","addr":"172.21.15.117:6811","nonce":3978863827}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6814","nonce":3978863827},{"type":"v1","addr":"172.21.15.117:6815","nonce":3978863827}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6812","nonce":3978863827},{"type":"v1","addr":"172.21.15.117:6813","nonce":3978863827}]},"public_addr":"172.21.15.117:6809/3978863827","cluster_addr":"172.21.15.117:6811/3978863827","heartbeat_back_addr":"172.21.15.117:6815/3978863827","heartbeat_front_addr":"172.21.15.117:6813/3978863827","state":["exists","up"]},{"osd":5,"uuid":"e01e3f15-25c8-4d2c-be6f-a170f036064c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6808","nonce":2761858035},{"type":"v1","addr":"172.21.15.167:6809","nonce":2761858035}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6810","nonce":2761858035},{"type":"v1","addr":"172.21.15.167:6811","nonce":2761858035}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6814","nonce":2761858035},{"type":"v1","addr":"172.21.15.167:6815","nonce":2761858035}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6812","nonce":2761858035},{"type":"v1","addr":"172.21.15.167:6813","nonce":2761858035}]},"public_addr":"172.21.15.167:6809/2761858035","cluster_addr":"172.21.15.167:6811/2761858035","heartbeat_back_addr":"172.21.15.167:6815/2761858035","heartbeat_front_addr":"172.21.15.167:6813/2761858035","state":["exists","up"]},{"osd":6,"uuid":"78cf0a9b-9165-444f-b073-ebe13016de9d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6818","nonce":260644552},{"type":"v1","addr":"172.21.15.83:6819","nonce":260644552}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6820","nonce":260644552},{"type":"v1","addr":"172.21.15.83:6821","nonce":260644552}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6824","nonce":260644552},{"type":"v1","addr":"172.21.15.83:6825","nonce":260644552}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6822","nonce":260644552},{"type":"v1","addr":"172.21.15.83:6823","nonce":260644552}]},"public_addr":"172.21.15.83:6819/260644552","cluster_addr":"172.21.15.83:6821/260644552","heartbeat_back_addr":"172.21.15.83:6825/260644552","heartbeat_front_addr":"172.21.15.83:6823/260644552","state":["exists","up"]},{"osd":7,"uuid":"ee524f7f-672e-430e-9f58-1a9117513466","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6816","nonce":104113252},{"type":"v1","addr":"172.21.15.117:6817","nonce":104113252}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6818","nonce":104113252},{"type":"v1","addr":"172.21.15.117:6819","nonce":104113252}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6822","nonce":104113252},{"type":"v1","addr":"172.21.15.117:6823","nonce":104113252}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6820","nonce":104113252},{"type":"v1","addr":"172.21.15.117:6821","nonce":104113252}]},"public_addr":"172.21.15.117:6817/104113252","cluster_addr":"172.21.15.117:6819/104113252","heartbeat_back_addr":"172.21.15.117:6823/104113252","heartbeat_front_addr":"172.21.15.117:6821/104113252","state":["exists","up"]},{"osd":8,"uuid":"76d5317d-4845-4138-b210-e7bd2d80472a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":58,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6816","nonce":2196898857},{"type":"v1","addr":"172.21.15.167:6817","nonce":2196898857}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6818","nonce":2196898857},{"type":"v1","addr":"172.21.15.167:6819","nonce":2196898857}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6822","nonce":2196898857},{"type":"v1","addr":"172.21.15.167:6823","nonce":2196898857}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6820","nonce":2196898857},{"type":"v1","addr":"172.21.15.167:6821","nonce":2196898857}]},"public_addr":"172.21.15.167:6817/2196898857","cluster_addr":"172.21.15.167:6819/2196898857","heartbeat_back_addr":"172.21.15.167:6823/2196898857","heartbeat_front_addr":"172.21.15.167:6821/2196898857","state":["exists","up"]},{"osd":9,"uuid":"0151bc2a-3ad2-4db3-8505-5855792cc9bb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6826","nonce":903463450},{"type":"v1","addr":"172.21.15.83:6827","nonce":903463450}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6828","nonce":903463450},{"type":"v1","addr":"172.21.15.83:6829","nonce":903463450}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6832","nonce":903463450},{"type":"v1","addr":"172.21.15.83:6833","nonce":903463450}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6830","nonce":903463450},{"type":"v1","addr":"172.21.15.83:6831","nonce":903463450}]},"public_addr":"172.21.15.83:6827/903463450","cluster_addr":"172.21.15.83:6829/903463450","heartbeat_back_addr":"172.21.15.83:6833/903463450","heartbeat_front_addr":"172.21.15.83:6831/903463450","state":["exists","up"]},{"osd":10,"uuid":"437ed848-0da4-4cd4-8716-0c55bb99b676","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6824","nonce":1183570837},{"type":"v1","addr":"172.21.15.117:6825","nonce":1183570837}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6826","nonce":1183570837},{"type":"v1","addr":"172.21.15.117:6827","nonce":1183570837}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6830","nonce":1183570837},{"type":"v1","addr":"172.21.15.117:6831","nonce":1183570837}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6828","nonce":1183570837},{"type":"v1","addr":"172.21.15.117:6829","nonce":1183570837}]},"public_addr":"172.21.15.117:6825/1183570837","cluster_addr":"172.21.15.117:6827/1183570837","heartbeat_back_addr":"172.21.15.117:6831/1183570837","heartbeat_front_addr":"172.21.15.117:6829/1183570837","state":["exists","up"]},{"osd":11,"uuid":"13345f05-cdc0-4dae-a58b-551abbec1915","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":78,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6824","nonce":187027350},{"type":"v1","addr":"172.21.15.167:6825","nonce":187027350}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6826","nonce":187027350},{"type":"v1","addr":"172.21.15.167:6827","nonce":187027350}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6830","nonce":187027350},{"type":"v1","addr":"172.21.15.167:6831","nonce":187027350}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6828","nonce":187027350},{"type":"v1","addr":"172.21.15.167:6829","nonce":187027350}]},"public_addr":"172.21.15.167:6825/187027350","cluster_addr":"172.21.15.167:6827/187027350","heartbeat_back_addr":"172.21.15.167:6831/187027350","heartbeat_front_addr":"172.21.15.167:6829/187027350","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-19T01:27:28.760517+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-19T01:27:50.493028+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-19T01:28:11.997674+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-19T01:28:38.109934+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-19T01:29:00.173065+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-19T01:29:21.336085+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-19T01:29:47.871498+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-19T01:30:10.720498+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-19T01:30:32.425438+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-19T01:30:59.524989+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-19T01:31:22.806162+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-19T01:31:45.043246+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.1a","mappings":[{"from":8,"to":1}]},{"pgid":"2.1b","mappings":[{"from":2,"to":4}]},{"pgid":"2.1d","mappings":[{"from":8,"to":11}]},{"pgid":"3.6","mappings":[{"from":4,"to":6}]},{"pgid":"3.11","mappings":[{"from":4,"to":8}]},{"pgid":"3.1b","mappings":[{"from":5,"to":9}]},{"pgid":"3.20","mappings":[{"from":5,"to":9}]},{"pgid":"3.31","mappings":[{"from":4,"to":7}]},{"pgid":"3.3b","mappings":[{"from":3,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.83:0/3138095466":"2024-08-20T01:26:04.277716+0000","172.21.15.83:0/3835005198":"2024-08-20T01:26:04.277716+0000","172.21.15.83:0/1790374088":"2024-08-20T01:25:37.228583+0000","172.21.15.83:6801/573907319":"2024-08-20T01:26:04.277716+0000","172.21.15.83:6800/573907319":"2024-08-20T01:26:04.277716+0000","172.21.15.83:0/664676002":"2024-08-20T01:25:37.228583+0000","172.21.15.83:0/2144611449":"2024-08-20T01:25:37.228583+0000","172.21.15.83:6801/3261152818":"2024-08-20T01:25:37.228583+0000","172.21.15.83:0/1609552666":"2024-08-20T01:26:04.277716+0000","172.21.15.83:6800/3261152818":"2024-08-20T01:25:37.228583+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-08-19T01:34:50.065 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:50.063+0000 7f5f3a800640 1 -- 172.21.15.83:0/178731420 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f5f1c0a2c10 msgr2=0x7f5f1c0a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:34:50.066 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:50.063+0000 7f5f3a800640 1 --2- 172.21.15.83:0/178731420 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f5f1c0a2c10 0x7f5f1c0a50d0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f5f34075cb0 tx=0x7f5f20005f90 comp rx=0 tx=0).stop 2024-08-19T01:34:50.066 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:50.063+0000 7f5f3a800640 1 -- 172.21.15.83:0/178731420 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5f34101700 msgr2=0x7f5f34074220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:34:50.066 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:50.063+0000 7f5f3a800640 1 --2- 172.21.15.83:0/178731420 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5f34101700 0x7f5f34074220 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f5f2800e9f0 tx=0x7f5f2800eeb0 comp rx=0 tx=0).stop 2024-08-19T01:34:50.066 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:50.064+0000 7f5f3a800640 1 -- 172.21.15.83:0/178731420 shutdown_connections 2024-08-19T01:34:50.066 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:50.064+0000 7f5f3a800640 1 --2- 172.21.15.83:0/178731420 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f5f1c0a2c10 0x7f5f1c0a50d0 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:50.066 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:50.064+0000 7f5f3a800640 1 --2- 172.21.15.83:0/178731420 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5f34107700 0x7f5f34074ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:50.066 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:50.064+0000 7f5f3a800640 1 --2- 172.21.15.83:0/178731420 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5f341020a0 0x7f5f34074760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:50.066 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:50.064+0000 7f5f3a800640 1 --2- 172.21.15.83:0/178731420 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5f34101700 0x7f5f34074220 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:50.066 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:50.064+0000 7f5f3a800640 1 -- 172.21.15.83:0/178731420 >> 172.21.15.83:0/178731420 conn(0x7f5f340fd440 msgr2=0x7f5f340fdaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:34:50.066 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:50.064+0000 7f5f3a800640 1 -- 172.21.15.83:0/178731420 shutdown_connections 2024-08-19T01:34:50.066 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:50.064+0000 7f5f3a800640 1 -- 172.21.15.83:0/178731420 wait complete. 2024-08-19T01:34:50.889 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:34:50 smithi083 ceph-mon[17120]: from='client.? 172.21.15.83:0/178731420' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-08-19T01:34:50.928 DEBUG:tasks.fwd_scrub:Yielding 2024-08-19T01:34:50.928 INFO:teuthology.run_tasks:Running task background_exec... 2024-08-19T01:34:50.938 INFO:teuthology.task.background_exec:Running background command on role mon.a host ubuntu@smithi083.front.sepia.ceph.com 2024-08-19T01:34:50.939 DEBUG:teuthology.orchestra.run.smithi083:> sudo TESTDIR=/home/ubuntu/cephtest daemon-helper kill --kill-group bash -c 'while sleep 13; do ceph tell mds.cephfs:0 flush journal; done' 2024-08-19T01:34:50.939 INFO:tasks.fwd_scrub.fs.[cephfs]:start scrubbing fs: cephfs 2024-08-19T01:34:50.939 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-08-19T01:34:50.939 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833 shell --fsid bad8780e-5dc9-11ef-bccf-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-08-19T01:34:50.943 INFO:teuthology.run_tasks:Running task workunit... 2024-08-19T01:34:50.957 INFO:tasks.workunit:Pulling workunits from ref 78215f5a1cf5eaf321d506a66539942e12851f34 2024-08-19T01:34:50.957 INFO:tasks.workunit:Making a separate scratch dir for every client... 2024-08-19T01:34:50.957 INFO:tasks.workunit:timeout=3h 2024-08-19T01:34:50.957 INFO:tasks.workunit:cleanup=True 2024-08-19T01:34:50.958 DEBUG:teuthology.orchestra.run.smithi083:> stat -- /home/ubuntu/cephtest/mnt.0 2024-08-19T01:34:50.959 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:34:50 smithi117 ceph-mon[22016]: from='client.? 172.21.15.83:0/178731420' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-08-19T01:34:50.966 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:34:50 smithi167 ceph-mon[21875]: from='client.? 172.21.15.83:0/178731420' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-08-19T01:34:50.995 INFO:teuthology.orchestra.run.smithi083.stdout: File: /home/ubuntu/cephtest/mnt.0 2024-08-19T01:34:50.995 INFO:teuthology.orchestra.run.smithi083.stdout: Size: 0 Blocks: 0 IO Block: 65536 directory 2024-08-19T01:34:50.995 INFO:teuthology.orchestra.run.smithi083.stdout:Device: 6ch/108d Inode: 1099511628283 Links: 2 2024-08-19T01:34:50.995 INFO:teuthology.orchestra.run.smithi083.stdout:Access: (1777/drwxrwxrwt) Uid: ( 0/ root) Gid: ( 0/ root) 2024-08-19T01:34:50.995 INFO:teuthology.orchestra.run.smithi083.stdout:Context: system_u:object_r:unlabeled_t:s0 2024-08-19T01:34:50.996 INFO:teuthology.orchestra.run.smithi083.stdout:Access: 2024-08-19 01:33:02.017383647 +0000 2024-08-19T01:34:50.996 INFO:teuthology.orchestra.run.smithi083.stdout:Modify: 2024-08-19 01:33:02.017383647 +0000 2024-08-19T01:34:50.996 INFO:teuthology.orchestra.run.smithi083.stdout:Change: 2024-08-19 01:34:39.627230066 +0000 2024-08-19T01:34:50.996 INFO:teuthology.orchestra.run.smithi083.stdout: Birth: 2024-08-19 01:33:02.017383647 +0000 2024-08-19T01:34:50.996 INFO:tasks.workunit:Did not need to create dir /home/ubuntu/cephtest/mnt.0 2024-08-19T01:34:50.996 DEBUG:teuthology.orchestra.run.smithi083:> cd -- /home/ubuntu/cephtest/mnt.0 && sudo install -d -m 0755 --owner=ubuntu -- client.0 2024-08-19T01:34:51.108 DEBUG:teuthology.orchestra.run.smithi083:> rm -rf /home/ubuntu/cephtest/clone.client.0 && git clone https://git.ceph.com/ceph.git /home/ubuntu/cephtest/clone.client.0 && cd /home/ubuntu/cephtest/clone.client.0 && git checkout 78215f5a1cf5eaf321d506a66539942e12851f34 2024-08-19T01:34:51.146 INFO:tasks.workunit.client.0.smithi083.stderr:Cloning into '/home/ubuntu/cephtest/clone.client.0'... 2024-08-19T01:34:51.333 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/bad8780e-5dc9-11ef-bccf-c7b262605968/mon.a/config 2024-08-19T01:34:51.767 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:34:51 smithi083 ceph-mon[17120]: pgmap v339: 97 pgs: 97 active+clean; 649 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 1.7 KiB/s wr, 8 op/s 2024-08-19T01:34:51.885 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:51.883+0000 7fe151e00640 1 -- 172.21.15.83:0/3256123785 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe14c1016f0 msgr2=0x7fe14c101b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:34:51.886 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:51.883+0000 7fe151e00640 1 --2- 172.21.15.83:0/3256123785 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe14c1016f0 0x7fe14c101b50 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7fe138009930 tx=0x7fe13802a420 comp rx=0 tx=0).stop 2024-08-19T01:34:51.886 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:51.883+0000 7fe151e00640 1 -- 172.21.15.83:0/3256123785 shutdown_connections 2024-08-19T01:34:51.886 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:51.884+0000 7fe151e00640 1 --2- 172.21.15.83:0/3256123785 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe14c102090 0x7fe14c10e5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:51.886 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:51.884+0000 7fe151e00640 1 --2- 172.21.15.83:0/3256123785 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe14c1016f0 0x7fe14c101b50 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:51.886 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:51.884+0000 7fe151e00640 1 --2- 172.21.15.83:0/3256123785 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe14c1076f0 0x7fe14c107ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:51.886 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:51.884+0000 7fe151e00640 1 -- 172.21.15.83:0/3256123785 >> 172.21.15.83:0/3256123785 conn(0x7fe14c0fd470 msgr2=0x7fe14c0ff890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:34:51.886 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:51.884+0000 7fe151e00640 1 -- 172.21.15.83:0/3256123785 shutdown_connections 2024-08-19T01:34:51.886 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:51.884+0000 7fe151e00640 1 -- 172.21.15.83:0/3256123785 wait complete. 2024-08-19T01:34:51.887 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:51.885+0000 7fe151e00640 1 Processor -- start 2024-08-19T01:34:51.887 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:51.885+0000 7fe151e00640 1 -- start start 2024-08-19T01:34:51.887 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:51.885+0000 7fe151e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe14c1016f0 0x7fe14c19a1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:34:51.887 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:51.885+0000 7fe151e00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe14c102090 0x7fe14c19a710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:34:51.887 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:51.885+0000 7fe151e00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe14c1076f0 0x7fe14c19eaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:34:51.887 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:51.885+0000 7fe151e00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fe14c111b40 con 0x7fe14c1016f0 2024-08-19T01:34:51.887 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:51.885+0000 7fe151e00640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fe14c1119c0 con 0x7fe14c102090 2024-08-19T01:34:51.888 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:51.885+0000 7fe151e00640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fe14c111cc0 con 0x7fe14c1076f0 2024-08-19T01:34:51.888 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:51.885+0000 7fe150a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe14c1016f0 0x7fe14c19a1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:34:51.888 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:51.885+0000 7fe150a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe14c1016f0 0x7fe14c19a1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:39240/0 (socket says 172.21.15.83:39240) 2024-08-19T01:34:51.888 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:51.885+0000 7fe150a00640 1 -- 172.21.15.83:0/869774115 learned_addr learned my addr 172.21.15.83:0/869774115 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-08-19T01:34:51.888 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:51.885+0000 7fe14be00640 1 --2- 172.21.15.83:0/869774115 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe14c102090 0x7fe14c19a710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:34:51.888 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:51.885+0000 7fe151400640 1 --2- 172.21.15.83:0/869774115 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe14c1076f0 0x7fe14c19eaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:34:51.889 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:51.886+0000 7fe151400640 1 -- 172.21.15.83:0/869774115 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe14c102090 msgr2=0x7fe14c19a710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:34:51.889 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:51.886+0000 7fe151400640 1 --2- 172.21.15.83:0/869774115 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe14c102090 0x7fe14c19a710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:51.889 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:51.886+0000 7fe151400640 1 -- 172.21.15.83:0/869774115 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe14c1016f0 msgr2=0x7fe14c19a1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:34:51.889 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:51.886+0000 7fe151400640 1 --2- 172.21.15.83:0/869774115 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe14c1016f0 0x7fe14c19a1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:51.889 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:51.886+0000 7fe151400640 1 -- 172.21.15.83:0/869774115 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe14c19f180 con 0x7fe14c1076f0 2024-08-19T01:34:51.889 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:51.886+0000 7fe151400640 1 --2- 172.21.15.83:0/869774115 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe14c1076f0 0x7fe14c19eaa0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fe144009fd0 tx=0x7fe14400ef10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:34:51.889 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:51.886+0000 7fe149600640 1 -- 172.21.15.83:0/869774115 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe14400ce20 con 0x7fe14c1076f0 2024-08-19T01:34:51.889 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:51.887+0000 7fe149600640 1 -- 172.21.15.83:0/869774115 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe144005020 con 0x7fe14c1076f0 2024-08-19T01:34:51.889 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:51.887+0000 7fe151e00640 1 -- 172.21.15.83:0/869774115 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe14c19f470 con 0x7fe14c1076f0 2024-08-19T01:34:51.889 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:51.887+0000 7fe149600640 1 -- 172.21.15.83:0/869774115 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe144010720 con 0x7fe14c1076f0 2024-08-19T01:34:51.889 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:51.887+0000 7fe151e00640 1 -- 172.21.15.83:0/869774115 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe14c1a6d50 con 0x7fe14c1076f0 2024-08-19T01:34:51.891 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:51.890+0000 7fe155e06740 1 Processor -- start 2024-08-19T01:34:51.891 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:51.890+0000 7fe149600640 1 -- 172.21.15.83:0/869774115 <== mon.1 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 19) ==== 137884+0+0 (secure 0 0 0) 0x7fe144005540 con 0x7fe14c1076f0 2024-08-19T01:34:51.891 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:51.890+0000 7fe155e06740 1 -- start start 2024-08-19T01:34:51.892 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:51.890+0000 7fe155e06740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5588d5caa650 0x5588d5caaa30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:34:51.892 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:51.890+0000 7fe155e06740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5588d5caaf70 0x5588d5cb52f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:34:51.892 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:51.890+0000 7fe155e06740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5588d5cb5830 0x5588d5cb7c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:34:51.893 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:51.890+0000 7fe155e06740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x5588d5bc57e0 con 0x5588d5caa650 2024-08-19T01:34:51.893 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:51.890+0000 7fe155e06740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x5588d5c10ac0 con 0x5588d5cb5830 2024-08-19T01:34:51.893 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:51.890+0000 7fe155e06740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x5588d5ba88c0 con 0x5588d5caaf70 2024-08-19T01:34:51.894 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:51.891+0000 7fe14be00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5588d5caa650 0x5588d5caaa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:34:51.894 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:51.891+0000 7fe14be00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5588d5caa650 0x5588d5caaa30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:39242/0 (socket says 172.21.15.83:39242) 2024-08-19T01:34:51.894 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:51.891+0000 7fe14be00640 1 -- 172.21.15.83:0/876926789 learned_addr learned my addr 172.21.15.83:0/876926789 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-08-19T01:34:51.894 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:51.891+0000 7fe150a00640 1 --2- 172.21.15.83:0/876926789 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5588d5caaf70 0x5588d5cb52f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:34:51.894 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:51.891+0000 7fe14be00640 1 --2- 172.21.15.83:0/876926789 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5588d5cb5830 0x5588d5cb7c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:34:51.894 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:51.891+0000 7fe14be00640 1 -- 172.21.15.83:0/876926789 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5588d5caaf70 msgr2=0x5588d5cb52f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:34:51.894 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:51.891+0000 7fe14be00640 1 --2- 172.21.15.83:0/876926789 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5588d5caaf70 0x5588d5cb52f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:51.894 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:51.891+0000 7fe14be00640 1 -- 172.21.15.83:0/876926789 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5588d5cb5830 msgr2=0x5588d5cb7c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:34:51.895 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:51.891+0000 7fe14be00640 1 --2- 172.21.15.83:0/876926789 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5588d5cb5830 0x5588d5cb7c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:51.895 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:51.891+0000 7fe14be00640 1 -- 172.21.15.83:0/876926789 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5588d5baf760 con 0x5588d5caa650 2024-08-19T01:34:51.895 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:51.892+0000 7fe149600640 1 --2- 172.21.15.83:0/869774115 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7fe1340a2c10 0x7fe1340a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:34:51.895 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:51.892+0000 7fe14be00640 1 --2- 172.21.15.83:0/876926789 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5588d5caa650 0x5588d5caaa30 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7fe13802aa10 tx=0x7fe138005b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2e8636eca0e798e2 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:34:51.895 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:51.892+0000 7fe149600640 1 -- 172.21.15.83:0/869774115 <== mon.1 v2:172.21.15.167:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8269+0+0 (secure 0 0 0) 0x7fe1440cf850 con 0x7fe14c1076f0 2024-08-19T01:34:51.895 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:51.892+0000 7fe150a00640 1 --2- 172.21.15.83:0/869774115 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7fe1340a2c10 0x7fe1340a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:34:51.895 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:51.892+0000 7fe13dc00640 1 -- 172.21.15.83:0/876926789 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe138044020 con 0x5588d5caa650 2024-08-19T01:34:51.895 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:51.892+0000 7fe13dc00640 1 -- 172.21.15.83:0/876926789 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe1380044b0 con 0x5588d5caa650 2024-08-19T01:34:51.895 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:51.892+0000 7fe13dc00640 1 -- 172.21.15.83:0/876926789 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe138035040 con 0x5588d5caa650 2024-08-19T01:34:51.897 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:51.893+0000 7fe155e06740 1 -- 172.21.15.83:0/876926789 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5588d5caa650 msgr2=0x5588d5caaa30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:34:51.897 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:51.893+0000 7fe150a00640 1 --2- 172.21.15.83:0/869774115 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7fe1340a2c10 0x7fe1340a50d0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x5588d5ca7a40 tx=0x7fe14000b4a0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:34:51.897 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:51.893+0000 7fe155e06740 1 --2- 172.21.15.83:0/876926789 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5588d5caa650 0x5588d5caaa30 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7fe13802aa10 tx=0x7fe138005b50 comp rx=0 tx=0).stop 2024-08-19T01:34:51.897 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:51.893+0000 7fe155e06740 1 -- 172.21.15.83:0/876926789 shutdown_connections 2024-08-19T01:34:51.897 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:51.893+0000 7fe155e06740 1 --2- 172.21.15.83:0/876926789 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5588d5cb5830 0x5588d5cb7c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:51.897 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:51.893+0000 7fe155e06740 1 --2- 172.21.15.83:0/876926789 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5588d5caaf70 0x5588d5cb52f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:51.897 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:51.893+0000 7fe155e06740 1 --2- 172.21.15.83:0/876926789 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5588d5caa650 0x5588d5caaa30 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:51.897 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:51.893+0000 7fe155e06740 1 -- 172.21.15.83:0/876926789 >> 172.21.15.83:0/876926789 conn(0x5588d5ca6be0 msgr2=0x5588d5ca6fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:34:51.897 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:51.893+0000 7fe155e06740 1 -- 172.21.15.83:0/876926789 shutdown_connections 2024-08-19T01:34:51.897 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:51.893+0000 7fe155e06740 1 -- 172.21.15.83:0/876926789 wait complete. 2024-08-19T01:34:51.897 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:51.894+0000 7fe155e06740 1 -- start start 2024-08-19T01:34:51.898 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:51.894+0000 7fe155e06740 1 Processor -- start 2024-08-19T01:34:51.898 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:51.895+0000 7fe155e06740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5588d5caa650 0x5588d5cb27c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:34:51.898 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:51.895+0000 7fe155e06740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5588d5caaf70 0x5588d5cb2d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:34:51.898 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:51.895+0000 7fe155e06740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5588d5cb5830 0x5588d5cb3240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:34:51.898 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:51.895+0000 7fe155e06740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x5588d5bcbae0 con 0x5588d5caaf70 2024-08-19T01:34:51.898 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:51.895+0000 7fe155e06740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x5588d5c0ca80 con 0x5588d5caa650 2024-08-19T01:34:51.898 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:51.895+0000 7fe155e06740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x5588d5cb84a0 con 0x5588d5cb5830 2024-08-19T01:34:51.898 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:51.895+0000 7fe155e06740 10 client.0 _ll_register_callbacks cb 0x5588d5bcd190 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-19T01:34:51.898 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:51.895+0000 7fe150a00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5588d5caa650 0x5588d5cb27c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:34:51.898 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:51.895+0000 7fe150a00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5588d5caa650 0x5588d5cb27c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.83:39422/0 (socket says 172.21.15.83:39422) 2024-08-19T01:34:51.898 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:51.895+0000 7fe150a00640 1 -- 172.21.15.83:0/832681850 learned_addr learned my addr 172.21.15.83:0/832681850 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-08-19T01:34:51.898 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:51.895+0000 7fe14be00640 1 --2- 172.21.15.83:0/832681850 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5588d5caaf70 0x5588d5cb2d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:34:51.898 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:51.895+0000 7fe151400640 1 --2- 172.21.15.83:0/832681850 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5588d5cb5830 0x5588d5cb3240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:34:51.900 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:51.895+0000 7fe150a00640 1 -- 172.21.15.83:0/832681850 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5588d5cb5830 msgr2=0x5588d5cb3240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:34:51.900 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:51.895+0000 7fe150a00640 1 --2- 172.21.15.83:0/832681850 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5588d5cb5830 0x5588d5cb3240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:51.900 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:51.895+0000 7fe150a00640 1 -- 172.21.15.83:0/832681850 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5588d5caaf70 msgr2=0x5588d5cb2d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:34:51.900 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:51.895+0000 7fe150a00640 1 --2- 172.21.15.83:0/832681850 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5588d5caaf70 0x5588d5cb2d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:51.900 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:51.895+0000 7fe150a00640 1 -- 172.21.15.83:0/832681850 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5588d59826a0 con 0x5588d5caa650 2024-08-19T01:34:51.900 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:51.896+0000 7fe150a00640 1 --2- 172.21.15.83:0/832681850 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5588d5caa650 0x5588d5cb27c0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fe14000bfd0 tx=0x7fe1400046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fcd3dde07fcbd9de server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:34:51.900 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:51.896+0000 7fe132000640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-08-19T01:34:51.900 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:51.896+0000 7fe132000640 1 -- 172.21.15.83:0/832681850 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe140005220 con 0x5588d5caa650 2024-08-19T01:34:51.900 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:51.896+0000 7fe132000640 1 -- 172.21.15.83:0/832681850 <== mon.2 v2:172.21.15.117:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8269+0+0 (secure 0 0 0) 0x7fe1400053c0 con 0x5588d5caa650 2024-08-19T01:34:51.900 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:51.896+0000 7fe155e06740 1 -- 172.21.15.83:0/832681850 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe1380044b0 con 0x5588d5caa650 2024-08-19T01:34:51.900 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:51.896+0000 7fe132000640 1 -- 172.21.15.83:0/832681850 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fe140005600 con 0x5588d5caa650 2024-08-19T01:34:51.901 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:51.896+0000 7fe155e06740 10 client.25142 fetch_fsmap learned FSMap version 25 2024-08-19T01:34:51.901 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:51.896+0000 7fe155e06740 1 -- 172.21.15.83:0/832681850 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5588d5ccc8e0 con 0x5588d5caa650 2024-08-19T01:34:51.901 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:51.897+0000 7fe132000640 1 -- 172.21.15.83:0/832681850 <== mon.2 v2:172.21.15.117:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe1380044b0 con 0x5588d5caa650 2024-08-19T01:34:51.901 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:51.897+0000 7fe132000640 1 -- 172.21.15.83:0/832681850 <== mon.2 v2:172.21.15.117:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fe140024070 con 0x5588d5caa650 2024-08-19T01:34:51.901 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:51.897+0000 7fe155e06740 10 client.25142 fetch_fsmap finished waiting for FSMap version 25 2024-08-19T01:34:51.901 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:51.897+0000 7fe155e06740 10 client.25142 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-19T01:34:51.901 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:51.897+0000 7fe155e06740 20 client.25142 populate_metadata read hostname 'smithi083' 2024-08-19T01:34:51.901 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:51.897+0000 7fe155e06740 1 --2- 172.21.15.83:0/832681850 >> [v2:172.21.15.117:6838/1967595704,v1:172.21.15.117:6839/1967595704] conn(0x5588d5ccdda0 0x5588d5cd01d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:34:51.901 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:51.897+0000 7fe155e06740 4 client.25142 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-19T01:34:51.901 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:51.897+0000 7fe155e06740 1 -- 172.21.15.83:0/832681850 --> [v2:172.21.15.117:6838/1967595704,v1:172.21.15.117:6839/1967595704] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5588d5c1e030 con 0x5588d5ccdda0 2024-08-19T01:34:51.901 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:51.898+0000 7fe14be00640 1 --2- 172.21.15.83:0/832681850 >> [v2:172.21.15.117:6838/1967595704,v1:172.21.15.117:6839/1967595704] conn(0x5588d5ccdda0 0x5588d5cd01d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:34:51.901 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:51.898+0000 7fe14be00640 1 --2- 172.21.15.83:0/832681850 >> [v2:172.21.15.117:6838/1967595704,v1:172.21.15.117:6839/1967595704] conn(0x5588d5ccdda0 0x5588d5cd01d0 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6c21c5c989faffe8 server_cookie=f154a1be913f063f in_seq=0 out_seq=0 2024-08-19T01:34:51.901 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:51.898+0000 7fe132000640 10 client.25142 ms_handle_connect on v2:172.21.15.117:6838/1967595704 2024-08-19T01:34:51.902 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:51.899+0000 7fe132000640 1 -- 172.21.15.83:0/832681850 <== mds.0 v2:172.21.15.117:6838/1967595704 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5588d5c1e030 con 0x5588d5ccdda0 2024-08-19T01:34:51.903 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:51.899+0000 7fe132000640 10 client.25142 handle_command_reply: tid=0 2024-08-19T01:34:51.903 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:51.899+0000 7fe155e06740 1 client.25142 shutdown 2024-08-19T01:34:51.903 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:51.899+0000 7fe155e06740 1 -- 172.21.15.83:0/832681850 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5588d5caa650 msgr2=0x5588d5cb27c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:34:51.903 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:51.899+0000 7fe155e06740 1 --2- 172.21.15.83:0/832681850 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5588d5caa650 0x5588d5cb27c0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fe14000bfd0 tx=0x7fe1400046c0 comp rx=0 tx=0).stop 2024-08-19T01:34:51.903 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:51.899+0000 7fe155e06740 1 -- 172.21.15.83:0/832681850 shutdown_connections 2024-08-19T01:34:51.903 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:51.899+0000 7fe155e06740 1 --2- 172.21.15.83:0/832681850 >> [v2:172.21.15.117:6838/1967595704,v1:172.21.15.117:6839/1967595704] conn(0x5588d5ccdda0 0x5588d5cd01d0 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:51.903 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:51.899+0000 7fe155e06740 1 --2- 172.21.15.83:0/832681850 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5588d5cb5830 0x5588d5cb3240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:51.903 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:51.899+0000 7fe132000640 0 client.25142 ms_handle_reset on v2:172.21.15.117:6838/1967595704 2024-08-19T01:34:51.903 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:51.899+0000 7fe155e06740 1 --2- 172.21.15.83:0/832681850 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5588d5caaf70 0x5588d5cb2d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:51.904 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:51.899+0000 7fe155e06740 1 --2- 172.21.15.83:0/832681850 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5588d5caa650 0x5588d5cb27c0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:51.904 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:51.899+0000 7fe155e06740 1 -- 172.21.15.83:0/832681850 >> 172.21.15.83:0/832681850 conn(0x5588d5ca6be0 msgr2=0x5588d5cac910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:34:51.904 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:51.900+0000 7fe155e06740 1 -- 172.21.15.83:0/832681850 shutdown_connections 2024-08-19T01:34:51.904 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:51.900+0000 7fe155e06740 1 -- 172.21.15.83:0/832681850 wait complete. 2024-08-19T01:34:51.904 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:51.900+0000 7fe155e06740 20 client.25142 trim_cache size 0 max 16384 2024-08-19T01:34:51.923 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:51.921+0000 7fe155e06740 1 Processor -- start 2024-08-19T01:34:51.923 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:51.921+0000 7fe155e06740 1 -- start start 2024-08-19T01:34:51.923 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:51.921+0000 7fe155e06740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5588d5caa650 0x5588d5cb2100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:34:51.923 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:51.921+0000 7fe155e06740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5588d5caaf70 0x5588d5cad0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:34:51.924 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:51.922+0000 7fe155e06740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5588d5cb5830 0x5588d5cd5aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:34:51.924 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:51.922+0000 7fe155e06740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fe14c1ad770 con 0x5588d5caa650 2024-08-19T01:34:51.924 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:51.922+0000 7fe155e06740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fe14c1ab040 con 0x5588d5caaf70 2024-08-19T01:34:51.924 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:51.922+0000 7fe155e06740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fe14c1a8ed0 con 0x5588d5cb5830 2024-08-19T01:34:51.924 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:51.922+0000 7fe150a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5588d5caa650 0x5588d5cb2100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:34:51.924 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:51.922+0000 7fe150a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5588d5caa650 0x5588d5cb2100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:39250/0 (socket says 172.21.15.83:39250) 2024-08-19T01:34:51.924 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:51.922+0000 7fe150a00640 1 -- 172.21.15.83:0/2145664967 learned_addr learned my addr 172.21.15.83:0/2145664967 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-08-19T01:34:51.924 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:51.922+0000 7fe14be00640 1 --2- 172.21.15.83:0/2145664967 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5588d5caaf70 0x5588d5cad0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:34:51.925 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:51.922+0000 7fe151400640 1 --2- 172.21.15.83:0/2145664967 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5588d5cb5830 0x5588d5cd5aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:34:51.925 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:51.922+0000 7fe150a00640 1 -- 172.21.15.83:0/2145664967 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5588d5cb5830 msgr2=0x5588d5cd5aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:34:51.925 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:51.922+0000 7fe150a00640 1 --2- 172.21.15.83:0/2145664967 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5588d5cb5830 0x5588d5cd5aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:51.925 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:51.922+0000 7fe150a00640 1 -- 172.21.15.83:0/2145664967 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5588d5caaf70 msgr2=0x5588d5cad0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:34:51.925 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:51.922+0000 7fe150a00640 1 --2- 172.21.15.83:0/2145664967 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5588d5caaf70 0x5588d5cad0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:51.926 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:51.922+0000 7fe150a00640 1 -- 172.21.15.83:0/2145664967 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5588d5ca7c90 con 0x5588d5caa650 2024-08-19T01:34:51.926 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:51.922+0000 7fe14be00640 1 --2- 172.21.15.83:0/2145664967 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5588d5caaf70 0x5588d5cad0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-19T01:34:51.926 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:51.923+0000 7fe150a00640 1 --2- 172.21.15.83:0/2145664967 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5588d5caa650 0x5588d5cb2100 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7fe1400057c0 tx=0x7fe14000bdb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c33b1a46ad8b04a0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:34:51.926 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:51.923+0000 7fe13d200640 1 -- 172.21.15.83:0/2145664967 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe14001b670 con 0x5588d5caa650 2024-08-19T01:34:51.926 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:51.923+0000 7fe13d200640 1 -- 172.21.15.83:0/2145664967 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe140033070 con 0x5588d5caa650 2024-08-19T01:34:51.926 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:51.923+0000 7fe13d200640 1 -- 172.21.15.83:0/2145664967 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe14002f670 con 0x5588d5caa650 2024-08-19T01:34:51.926 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:51.924+0000 7fe155e06740 1 -- 172.21.15.83:0/2145664967 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5588d5caa650 msgr2=0x5588d5cb2100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:34:51.926 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:51.924+0000 7fe155e06740 1 --2- 172.21.15.83:0/2145664967 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5588d5caa650 0x5588d5cb2100 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7fe1400057c0 tx=0x7fe14000bdb0 comp rx=0 tx=0).stop 2024-08-19T01:34:51.926 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:51.924+0000 7fe155e06740 1 -- 172.21.15.83:0/2145664967 shutdown_connections 2024-08-19T01:34:51.926 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:51.924+0000 7fe155e06740 1 --2- 172.21.15.83:0/2145664967 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5588d5cb5830 0x5588d5cd5aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:51.926 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:51.924+0000 7fe155e06740 1 --2- 172.21.15.83:0/2145664967 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5588d5caaf70 0x5588d5cad0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:51.926 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:51.924+0000 7fe155e06740 1 --2- 172.21.15.83:0/2145664967 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5588d5caa650 0x5588d5cb2100 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:51.926 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:51.924+0000 7fe155e06740 1 -- 172.21.15.83:0/2145664967 >> 172.21.15.83:0/2145664967 conn(0x5588d5ca6be0 msgr2=0x5588d5ca5d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:34:51.926 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:51.924+0000 7fe155e06740 1 -- 172.21.15.83:0/2145664967 shutdown_connections 2024-08-19T01:34:51.927 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:51.924+0000 7fe155e06740 1 -- 172.21.15.83:0/2145664967 wait complete. 2024-08-19T01:34:51.929 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:51.924+0000 7fe155e06740 1 -- start start 2024-08-19T01:34:51.929 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:51.924+0000 7fe155e06740 1 Processor -- start 2024-08-19T01:34:51.929 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:51.925+0000 7fe155e06740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5588d5caa650 0x5588d5cbdee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:34:51.929 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:51.925+0000 7fe155e06740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5588d5caaf70 0x5588d5caf5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:34:51.929 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:51.925+0000 7fe155e06740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5588d5cb5830 0x5588d5cafb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:34:51.929 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:51.925+0000 7fe155e06740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fe14c1a9520 con 0x5588d5caaf70 2024-08-19T01:34:51.929 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:51.925+0000 7fe155e06740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fe14c1aa970 con 0x5588d5caa650 2024-08-19T01:34:51.930 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:51.925+0000 7fe155e06740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fe14c1a9bd0 con 0x5588d5cb5830 2024-08-19T01:34:51.930 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:51.925+0000 7fe155e06740 10 client.0 _ll_register_callbacks cb 0x7fe140004320 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-19T01:34:51.930 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:51.925+0000 7fe14be00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5588d5caaf70 0x5588d5caf5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:34:51.930 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:51.925+0000 7fe14be00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5588d5caaf70 0x5588d5caf5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:39260/0 (socket says 172.21.15.83:39260) 2024-08-19T01:34:51.930 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:51.925+0000 7fe14be00640 1 -- 172.21.15.83:0/2483730447 learned_addr learned my addr 172.21.15.83:0/2483730447 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-08-19T01:34:51.930 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:51.925+0000 7fe150a00640 1 --2- 172.21.15.83:0/2483730447 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5588d5caa650 0x5588d5cbdee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:34:51.930 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:51.925+0000 7fe151400640 1 --2- 172.21.15.83:0/2483730447 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5588d5cb5830 0x5588d5cafb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:34:51.930 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:51.926+0000 7fe150a00640 1 -- 172.21.15.83:0/2483730447 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5588d5cb5830 msgr2=0x5588d5cafb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:34:51.930 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:51.926+0000 7fe150a00640 1 --2- 172.21.15.83:0/2483730447 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5588d5cb5830 0x5588d5cafb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:51.930 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:51.926+0000 7fe150a00640 1 -- 172.21.15.83:0/2483730447 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5588d5caaf70 msgr2=0x5588d5caf5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:34:51.930 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:51.926+0000 7fe150a00640 1 --2- 172.21.15.83:0/2483730447 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5588d5caaf70 0x5588d5caf5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:51.930 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:51.926+0000 7fe150a00640 1 -- 172.21.15.83:0/2483730447 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5588d5cb0390 con 0x5588d5caa650 2024-08-19T01:34:51.930 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:51.926+0000 7fe14be00640 1 --2- 172.21.15.83:0/2483730447 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5588d5caaf70 0x5588d5caf5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-19T01:34:51.930 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:51.926+0000 7fe150a00640 1 --2- 172.21.15.83:0/2483730447 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5588d5caa650 0x5588d5cbdee0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fe140017040 tx=0x7fe140003c40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6fa34b0bd49c903a server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:34:51.931 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:51.926+0000 7fe132000640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-08-19T01:34:51.931 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:51.926+0000 7fe132000640 1 -- 172.21.15.83:0/2483730447 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe140010660 con 0x5588d5caa650 2024-08-19T01:34:51.931 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:51.926+0000 7fe155e06740 1 -- 172.21.15.83:0/2483730447 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe140033070 con 0x5588d5caa650 2024-08-19T01:34:51.931 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:51.926+0000 7fe132000640 1 -- 172.21.15.83:0/2483730447 <== mon.2 v2:172.21.15.117:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8269+0+0 (secure 0 0 0) 0x7fe140010800 con 0x5588d5caa650 2024-08-19T01:34:51.931 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:51.927+0000 7fe132000640 1 -- 172.21.15.83:0/2483730447 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fe140010a00 con 0x5588d5caa650 2024-08-19T01:34:51.931 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:51.927+0000 7fe155e06740 10 client.25148 fetch_fsmap learned FSMap version 25 2024-08-19T01:34:51.931 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:51.927+0000 7fe155e06740 1 -- 172.21.15.83:0/2483730447 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5588d5d38b50 con 0x5588d5caa650 2024-08-19T01:34:51.931 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:51.927+0000 7fe132000640 1 -- 172.21.15.83:0/2483730447 <== mon.2 v2:172.21.15.117:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe140033070 con 0x5588d5caa650 2024-08-19T01:34:51.931 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:51.927+0000 7fe132000640 1 -- 172.21.15.83:0/2483730447 <== mon.2 v2:172.21.15.117:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fe140024070 con 0x5588d5caa650 2024-08-19T01:34:51.931 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:51.927+0000 7fe155e06740 10 client.25148 fetch_fsmap finished waiting for FSMap version 25 2024-08-19T01:34:51.931 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:51.927+0000 7fe155e06740 10 client.25148 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-19T01:34:51.931 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:51.927+0000 7fe155e06740 20 client.25148 populate_metadata read hostname 'smithi083' 2024-08-19T01:34:51.931 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:51.927+0000 7fe155e06740 1 --2- 172.21.15.83:0/2483730447 >> [v2:172.21.15.117:6838/1967595704,v1:172.21.15.117:6839/1967595704] conn(0x5588d5d39110 0x5588d5d3b500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:34:51.931 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:51.927+0000 7fe155e06740 4 client.25148 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-08-19T01:34:51.932 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:51.927+0000 7fe155e06740 1 -- 172.21.15.83:0/2483730447 --> [v2:172.21.15.117:6838/1967595704,v1:172.21.15.117:6839/1967595704] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x5588d5c18540 con 0x5588d5d39110 2024-08-19T01:34:51.933 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:51.928+0000 7fe14be00640 1 --2- 172.21.15.83:0/2483730447 >> [v2:172.21.15.117:6838/1967595704,v1:172.21.15.117:6839/1967595704] conn(0x5588d5d39110 0x5588d5d3b500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:34:51.933 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:51.928+0000 7fe14be00640 1 --2- 172.21.15.83:0/2483730447 >> [v2:172.21.15.117:6838/1967595704,v1:172.21.15.117:6839/1967595704] conn(0x5588d5d39110 0x5588d5d3b500 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ef571b8b4249ded1 server_cookie=13b3f3da47cda286 in_seq=0 out_seq=0 2024-08-19T01:34:51.933 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:51.928+0000 7fe132000640 10 client.25148 ms_handle_connect on v2:172.21.15.117:6838/1967595704 2024-08-19T01:34:51.933 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:51.929+0000 7fe132000640 1 -- 172.21.15.83:0/2483730447 <== mds.0 v2:172.21.15.117:6838/1967595704 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x5588d5c18540 con 0x5588d5d39110 2024-08-19T01:34:51.933 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:51.929+0000 7fe132000640 10 client.25148 handle_command_reply: tid=0 2024-08-19T01:34:51.933 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:51.929+0000 7fe155e06740 1 client.25148 shutdown 2024-08-19T01:34:51.933 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:51.930+0000 7fe155e06740 1 -- 172.21.15.83:0/2483730447 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5588d5caa650 msgr2=0x5588d5cbdee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:34:51.933 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:51.930+0000 7fe155e06740 1 --2- 172.21.15.83:0/2483730447 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5588d5caa650 0x5588d5cbdee0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fe140017040 tx=0x7fe140003c40 comp rx=0 tx=0).stop 2024-08-19T01:34:51.933 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:51.930+0000 7fe155e06740 1 -- 172.21.15.83:0/2483730447 shutdown_connections 2024-08-19T01:34:51.934 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:51.930+0000 7fe155e06740 1 --2- 172.21.15.83:0/2483730447 >> [v2:172.21.15.117:6838/1967595704,v1:172.21.15.117:6839/1967595704] conn(0x5588d5d39110 0x5588d5d3b500 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:51.934 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:51.930+0000 7fe155e06740 1 --2- 172.21.15.83:0/2483730447 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5588d5cb5830 0x5588d5cafb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:51.934 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:51.930+0000 7fe155e06740 1 --2- 172.21.15.83:0/2483730447 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5588d5caaf70 0x5588d5caf5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:51.934 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:51.930+0000 7fe155e06740 1 --2- 172.21.15.83:0/2483730447 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5588d5caa650 0x5588d5cbdee0 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:51.934 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:51.930+0000 7fe132000640 0 client.25148 ms_handle_reset on v2:172.21.15.117:6838/1967595704 2024-08-19T01:34:51.934 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:51.930+0000 7fe155e06740 1 -- 172.21.15.83:0/2483730447 >> 172.21.15.83:0/2483730447 conn(0x5588d5ca6be0 msgr2=0x5588d5cc0570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:34:51.934 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:51.930+0000 7fe155e06740 1 -- 172.21.15.83:0/2483730447 shutdown_connections 2024-08-19T01:34:51.934 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:51.930+0000 7fe155e06740 1 -- 172.21.15.83:0/2483730447 wait complete. 2024-08-19T01:34:51.934 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:51.930+0000 7fe155e06740 20 client.25148 trim_cache size 0 max 16384 2024-08-19T01:34:51.934 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-08-19T01:34:51.934 INFO:teuthology.orchestra.run.smithi083.stdout: "return_code": 0, 2024-08-19T01:34:51.935 INFO:teuthology.orchestra.run.smithi083.stdout: "scrub_tag": "757a53a7-cd2a-4afd-b9b6-548543c74c88", 2024-08-19T01:34:51.935 INFO:teuthology.orchestra.run.smithi083.stdout: "mode": "asynchronous" 2024-08-19T01:34:51.935 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-08-19T01:34:51.935 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:51.931+0000 7fe151e00640 1 -- 172.21.15.83:0/869774115 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7fe1340a2c10 msgr2=0x7fe1340a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:34:51.935 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:51.931+0000 7fe151e00640 1 --2- 172.21.15.83:0/869774115 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7fe1340a2c10 0x7fe1340a50d0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x5588d5ca7a40 tx=0x7fe14000b4a0 comp rx=0 tx=0).stop 2024-08-19T01:34:51.935 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:51.931+0000 7fe151e00640 1 -- 172.21.15.83:0/869774115 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe14c1076f0 msgr2=0x7fe14c19eaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:34:51.936 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:51.931+0000 7fe151e00640 1 --2- 172.21.15.83:0/869774115 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe14c1076f0 0x7fe14c19eaa0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fe144009fd0 tx=0x7fe14400ef10 comp rx=0 tx=0).stop 2024-08-19T01:34:51.936 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:51.931+0000 7fe151e00640 1 -- 172.21.15.83:0/869774115 shutdown_connections 2024-08-19T01:34:51.936 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:51.931+0000 7fe151e00640 1 --2- 172.21.15.83:0/869774115 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7fe1340a2c10 0x7fe1340a50d0 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:51.936 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:51.931+0000 7fe151e00640 1 --2- 172.21.15.83:0/869774115 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe14c1076f0 0x7fe14c19eaa0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:51.936 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:51.931+0000 7fe151e00640 1 --2- 172.21.15.83:0/869774115 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe14c102090 0x7fe14c19a710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:51.936 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:51.931+0000 7fe151e00640 1 --2- 172.21.15.83:0/869774115 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe14c1016f0 0x7fe14c19a1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:51.936 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:51.931+0000 7fe151e00640 1 -- 172.21.15.83:0/869774115 >> 172.21.15.83:0/869774115 conn(0x7fe14c0fd470 msgr2=0x7fe14c0fda50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:34:51.936 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:51.931+0000 7fe151e00640 1 -- 172.21.15.83:0/869774115 shutdown_connections 2024-08-19T01:34:51.936 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:51.931+0000 7fe151e00640 1 -- 172.21.15.83:0/869774115 wait complete. 2024-08-19T01:34:51.952 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:34:51 smithi117 ceph-mon[22016]: pgmap v339: 97 pgs: 97 active+clean; 649 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 1.7 KiB/s wr, 8 op/s 2024-08-19T01:34:51.966 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:34:51 smithi167 ceph-mon[21875]: pgmap v339: 97 pgs: 97 active+clean; 649 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 1.7 KiB/s wr, 8 op/s 2024-08-19T01:34:52.647 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:34:52 smithi083 ceph-mon[17120]: scrub queued for path: / 2024-08-19T01:34:52.647 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:34:52 smithi083 ceph-mon[17120]: scrub summary: idle+waiting paths [/] 2024-08-19T01:34:52.647 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:34:52 smithi083 ceph-mon[17120]: scrub summary: active paths [/] 2024-08-19T01:34:52.647 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:34:52 smithi083 ceph-mon[17120]: scrub summary: idle+waiting paths [/] 2024-08-19T01:34:52.684 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833 shell --fsid bad8780e-5dc9-11ef-bccf-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-08-19T01:34:52.952 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:34:52 smithi117 ceph-mon[22016]: scrub queued for path: / 2024-08-19T01:34:52.952 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:34:52 smithi117 ceph-mon[22016]: scrub summary: idle+waiting paths [/] 2024-08-19T01:34:52.952 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:34:52 smithi117 ceph-mon[22016]: scrub summary: active paths [/] 2024-08-19T01:34:52.952 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:34:52 smithi117 ceph-mon[22016]: scrub summary: idle+waiting paths [/] 2024-08-19T01:34:52.966 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:34:52 smithi167 ceph-mon[21875]: scrub queued for path: / 2024-08-19T01:34:52.966 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:34:52 smithi167 ceph-mon[21875]: scrub summary: idle+waiting paths [/] 2024-08-19T01:34:52.966 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:34:52 smithi167 ceph-mon[21875]: scrub summary: active paths [/] 2024-08-19T01:34:52.966 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:34:52 smithi167 ceph-mon[21875]: scrub summary: idle+waiting paths [/] 2024-08-19T01:34:53.099 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/bad8780e-5dc9-11ef-bccf-c7b262605968/mon.a/config 2024-08-19T01:34:53.705 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:53.703+0000 7f63b0400640 1 -- 172.21.15.83:0/1772897508 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f63a81018a0 msgr2=0x7f63a8101d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:34:53.705 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:53.703+0000 7f63b0400640 1 --2- 172.21.15.83:0/1772897508 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f63a81018a0 0x7f63a8101d00 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f6394009930 tx=0x7f639402a420 comp rx=0 tx=0).stop 2024-08-19T01:34:53.705 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:53.703+0000 7f63b0400640 1 -- 172.21.15.83:0/1772897508 shutdown_connections 2024-08-19T01:34:53.705 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:53.703+0000 7f63b0400640 1 --2- 172.21.15.83:0/1772897508 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f63a8102240 0x7f63a810e770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:53.705 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:53.703+0000 7f63b0400640 1 --2- 172.21.15.83:0/1772897508 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f63a81018a0 0x7f63a8101d00 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:53.705 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:53.703+0000 7f63b0400640 1 --2- 172.21.15.83:0/1772897508 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f63a81078a0 0x7f63a8107c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:53.705 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:53.703+0000 7f63b0400640 1 -- 172.21.15.83:0/1772897508 >> 172.21.15.83:0/1772897508 conn(0x7f63a80fd5e0 msgr2=0x7f63a80ffa00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:34:53.705 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:53.703+0000 7f63b0400640 1 -- 172.21.15.83:0/1772897508 shutdown_connections 2024-08-19T01:34:53.706 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:53.703+0000 7f63b0400640 1 -- 172.21.15.83:0/1772897508 wait complete. 2024-08-19T01:34:53.706 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:53.704+0000 7f63b0400640 1 Processor -- start 2024-08-19T01:34:53.706 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:53.704+0000 7f63b0400640 1 -- start start 2024-08-19T01:34:53.706 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:53.705+0000 7f63b0400640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f63a81018a0 0x7f63a819a3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:34:53.706 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:53.705+0000 7f63b0400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f63a8102240 0x7f63a819a920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:34:53.707 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:34:53 smithi083 ceph-mon[17120]: pgmap v340: 97 pgs: 97 active+clean; 649 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 1.7 KiB/s wr, 7 op/s 2024-08-19T01:34:53.707 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:34:53 smithi083 ceph-mon[17120]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-19T01:34:53.708 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:53.705+0000 7f63b0400640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f63a81078a0 0x7f63a819ecb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:34:53.708 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:53.705+0000 7f63b0400640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f63a8111ca0 con 0x7f63a8102240 2024-08-19T01:34:53.708 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:53.705+0000 7f63b0400640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f63a8111b20 con 0x7f63a81018a0 2024-08-19T01:34:53.708 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:53.705+0000 7f63b0400640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f63a8111e20 con 0x7f63a81078a0 2024-08-19T01:34:53.708 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:53.705+0000 7f63ad000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f63a8102240 0x7f63a819a920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:34:53.708 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:53.705+0000 7f63ae400640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f63a81078a0 0x7f63a819ecb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:34:53.709 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:53.705+0000 7f63ada00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f63a81018a0 0x7f63a819a3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:34:53.709 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:53.705+0000 7f63ae400640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f63a81078a0 0x7f63a819ecb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.83:58678/0 (socket says 172.21.15.83:58678) 2024-08-19T01:34:53.709 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:53.705+0000 7f63ae400640 1 -- 172.21.15.83:0/2662666822 learned_addr learned my addr 172.21.15.83:0/2662666822 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-08-19T01:34:53.709 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:53.706+0000 7f63ada00640 1 -- 172.21.15.83:0/2662666822 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f63a81078a0 msgr2=0x7f63a819ecb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:34:53.709 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:53.706+0000 7f63ada00640 1 --2- 172.21.15.83:0/2662666822 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f63a81078a0 0x7f63a819ecb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:53.709 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:53.706+0000 7f63ada00640 1 -- 172.21.15.83:0/2662666822 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f63a8102240 msgr2=0x7f63a819a920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:34:53.709 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:53.706+0000 7f63ada00640 1 --2- 172.21.15.83:0/2662666822 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f63a8102240 0x7f63a819a920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:53.709 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:53.706+0000 7f63ada00640 1 -- 172.21.15.83:0/2662666822 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f63a819f390 con 0x7f63a81018a0 2024-08-19T01:34:53.709 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:53.706+0000 7f63ae400640 1 --2- 172.21.15.83:0/2662666822 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f63a81078a0 0x7f63a819ecb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-19T01:34:53.709 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:53.706+0000 7f63ada00640 1 --2- 172.21.15.83:0/2662666822 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f63a81018a0 0x7f63a819a3e0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f639c00e9f0 tx=0x7f639c00eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:34:53.709 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:53.706+0000 7f63a6000640 1 -- 172.21.15.83:0/2662666822 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f639c00cd80 con 0x7f63a81018a0 2024-08-19T01:34:53.709 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:53.706+0000 7f63a6000640 1 -- 172.21.15.83:0/2662666822 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f639c0035f0 con 0x7f63a81018a0 2024-08-19T01:34:53.709 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:53.706+0000 7f63a6000640 1 -- 172.21.15.83:0/2662666822 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f639c010690 con 0x7f63a81018a0 2024-08-19T01:34:53.709 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:53.706+0000 7f63b0400640 1 -- 172.21.15.83:0/2662666822 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f63a819f680 con 0x7f63a81018a0 2024-08-19T01:34:53.710 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:53.707+0000 7f63b0400640 1 -- 172.21.15.83:0/2662666822 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f63a81a6f60 con 0x7f63a81018a0 2024-08-19T01:34:53.711 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:53.710+0000 7f63b2e8c740 1 Processor -- start 2024-08-19T01:34:53.712 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:53.710+0000 7f63b2e8c740 1 -- start start 2024-08-19T01:34:53.712 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:53.710+0000 7f63a6000640 1 -- 172.21.15.83:0/2662666822 <== mon.2 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 19) ==== 137884+0+0 (secure 0 0 0) 0x7f639c003b30 con 0x7f63a81018a0 2024-08-19T01:34:53.712 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:53.711+0000 7f63b2e8c740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5606afbbbb90 0x5606afbbbf70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:34:53.712 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:53.711+0000 7f63b2e8c740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5606afbbc4b0 0x5606afbc6830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:34:53.713 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:53.711+0000 7f63b2e8c740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5606afbc6d70 0x5606afbc9160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:34:53.713 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:53.711+0000 7f63b2e8c740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x5606afb21bf0 con 0x5606afbbc4b0 2024-08-19T01:34:53.713 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:53.711+0000 7f63b2e8c740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x5606afab9bd0 con 0x5606afbc6d70 2024-08-19T01:34:53.713 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:53.711+0000 7f63b2e8c740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x5606afb1dbb0 con 0x5606afbbbb90 2024-08-19T01:34:53.714 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:53.711+0000 7f63ada00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5606afbbc4b0 0x5606afbc6830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:34:53.714 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:53.711+0000 7f63ae400640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5606afbbbb90 0x5606afbbbf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:34:53.714 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:53.711+0000 7f63ae400640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5606afbbbb90 0x5606afbbbf70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.83:58694/0 (socket says 172.21.15.83:58694) 2024-08-19T01:34:53.714 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:53.711+0000 7f63ae400640 1 -- 172.21.15.83:0/1615236321 learned_addr learned my addr 172.21.15.83:0/1615236321 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-08-19T01:34:53.714 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:53.711+0000 7f63ad000640 1 --2- 172.21.15.83:0/1615236321 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5606afbc6d70 0x5606afbc9160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:34:53.714 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:53.711+0000 7f63ae400640 1 -- 172.21.15.83:0/1615236321 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5606afbc6d70 msgr2=0x5606afbc9160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:34:53.714 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:53.711+0000 7f63ae400640 1 --2- 172.21.15.83:0/1615236321 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5606afbc6d70 0x5606afbc9160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:53.714 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:53.711+0000 7f63ae400640 1 -- 172.21.15.83:0/1615236321 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5606afbbc4b0 msgr2=0x5606afbc6830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:34:53.714 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:53.711+0000 7f63ae400640 1 --2- 172.21.15.83:0/1615236321 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5606afbbc4b0 0x5606afbc6830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:53.714 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:53.711+0000 7f63ae400640 1 -- 172.21.15.83:0/1615236321 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5606afac08a0 con 0x5606afbbbb90 2024-08-19T01:34:53.714 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:53.712+0000 7f63ae400640 1 --2- 172.21.15.83:0/1615236321 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5606afbbbb90 0x5606afbbbf70 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f63a00118b0 tx=0x7f63a0011d70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c2ad89d613b5b20a server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:34:53.714 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:53.712+0000 7f639aa00640 1 -- 172.21.15.83:0/1615236321 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63a001e070 con 0x5606afbbbb90 2024-08-19T01:34:53.715 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:53.712+0000 7f639aa00640 1 -- 172.21.15.83:0/1615236321 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f63a0011310 con 0x5606afbbbb90 2024-08-19T01:34:53.715 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:53.712+0000 7f639aa00640 1 -- 172.21.15.83:0/1615236321 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63a0019e00 con 0x5606afbbbb90 2024-08-19T01:34:53.716 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:53.712+0000 7f63a6000640 1 --2- 172.21.15.83:0/2662666822 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f63900a2c90 0x7f63900a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:34:53.716 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:53.713+0000 7f63a6000640 1 -- 172.21.15.83:0/2662666822 <== mon.2 v2:172.21.15.117:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8269+0+0 (secure 0 0 0) 0x7f639c0cf3d0 con 0x7f63a81018a0 2024-08-19T01:34:53.716 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:53.713+0000 7f63ada00640 1 --2- 172.21.15.83:0/2662666822 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f63900a2c90 0x7f63900a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:34:53.716 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:53.713+0000 7f63b2e8c740 1 -- 172.21.15.83:0/1615236321 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5606afbbbb90 msgr2=0x5606afbbbf70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:34:53.716 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:53.713+0000 7f63b2e8c740 1 --2- 172.21.15.83:0/1615236321 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5606afbbbb90 0x5606afbbbf70 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f63a00118b0 tx=0x7f63a0011d70 comp rx=0 tx=0).stop 2024-08-19T01:34:53.716 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:53.713+0000 7f63b2e8c740 1 -- 172.21.15.83:0/1615236321 shutdown_connections 2024-08-19T01:34:53.716 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:53.713+0000 7f63b2e8c740 1 --2- 172.21.15.83:0/1615236321 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5606afbc6d70 0x5606afbc9160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:53.716 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:53.713+0000 7f63b2e8c740 1 --2- 172.21.15.83:0/1615236321 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5606afbbc4b0 0x5606afbc6830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:53.716 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:53.713+0000 7f63b2e8c740 1 --2- 172.21.15.83:0/1615236321 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5606afbbbb90 0x5606afbbbf70 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:53.716 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:53.713+0000 7f63b2e8c740 1 -- 172.21.15.83:0/1615236321 >> 172.21.15.83:0/1615236321 conn(0x5606afbb8120 msgr2=0x5606afbb8500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:34:53.717 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:53.713+0000 7f63b2e8c740 1 -- 172.21.15.83:0/1615236321 shutdown_connections 2024-08-19T01:34:53.717 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:53.714+0000 7f63b2e8c740 1 -- 172.21.15.83:0/1615236321 wait complete. 2024-08-19T01:34:53.717 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:53.714+0000 7f63ada00640 1 --2- 172.21.15.83:0/2662666822 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f63900a2c90 0x7f63900a5150 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f639c097eb0 tx=0x7f639c0d1820 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:34:53.717 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:53.714+0000 7f63b2e8c740 1 -- start start 2024-08-19T01:34:53.717 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:53.714+0000 7f63b2e8c740 1 Processor -- start 2024-08-19T01:34:53.718 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:53.715+0000 7f63b2e8c740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5606afbbbb90 0x5606afbc3b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:34:53.718 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:53.715+0000 7f63b2e8c740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5606afbbc4b0 0x5606afbc40b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:34:53.718 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:53.715+0000 7f63b2e8c740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5606afbc6d70 0x5606afbc45f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:34:53.718 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:53.715+0000 7f63b2e8c740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x5606afbc9940 con 0x5606afbbc4b0 2024-08-19T01:34:53.718 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:53.715+0000 7f63b2e8c740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x5606afadcd40 con 0x5606afbbbb90 2024-08-19T01:34:53.718 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:53.715+0000 7f63b2e8c740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x5606afbc9ac0 con 0x5606afbc6d70 2024-08-19T01:34:53.718 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:53.715+0000 7f63b2e8c740 10 client.0 _ll_register_callbacks cb 0x5606afaa5b10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-19T01:34:53.719 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:53.715+0000 7f63ae400640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5606afbbbb90 0x5606afbc3b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:34:53.719 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:53.715+0000 7f63ad000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5606afbbc4b0 0x5606afbc40b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:34:53.719 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:53.715+0000 7f63ae400640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5606afbbbb90 0x5606afbc3b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.83:39468/0 (socket says 172.21.15.83:39468) 2024-08-19T01:34:53.719 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:53.716+0000 7f63ae400640 1 -- 172.21.15.83:0/2660978064 learned_addr learned my addr 172.21.15.83:0/2660978064 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-08-19T01:34:53.719 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:53.716+0000 7f63ae400640 1 --2- 172.21.15.83:0/2660978064 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5606afbc6d70 0x5606afbc45f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:34:53.719 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:53.716+0000 7f63ae400640 1 -- 172.21.15.83:0/2660978064 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5606afbc6d70 msgr2=0x5606afbc45f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:34:53.719 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:53.716+0000 7f63ae400640 1 --2- 172.21.15.83:0/2660978064 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5606afbc6d70 0x5606afbc45f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:53.719 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:53.716+0000 7f63ae400640 1 -- 172.21.15.83:0/2660978064 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5606afbbc4b0 msgr2=0x5606afbc40b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:34:53.719 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:53.716+0000 7f63ae400640 1 --2- 172.21.15.83:0/2660978064 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5606afbbc4b0 0x5606afbc40b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:53.719 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:53.716+0000 7f63ae400640 1 -- 172.21.15.83:0/2660978064 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5606af8933c0 con 0x5606afbbbb90 2024-08-19T01:34:53.719 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:53.716+0000 7f63ad000640 1 --2- 172.21.15.83:0/2660978064 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5606afbbc4b0 0x5606afbc40b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-19T01:34:53.719 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:53.716+0000 7f63ae400640 1 --2- 172.21.15.83:0/2660978064 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5606afbbbb90 0x5606afbc3b70 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f63a0011880 tx=0x7f63a0009a30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b8ba941cdfbc720f server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:34:53.719 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:53.716+0000 7f638f400640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-08-19T01:34:53.719 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:53.717+0000 7f638f400640 1 -- 172.21.15.83:0/2660978064 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63a001e030 con 0x5606afbbbb90 2024-08-19T01:34:53.719 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:53.717+0000 7f638f400640 1 -- 172.21.15.83:0/2660978064 <== mon.2 v2:172.21.15.117:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8269+0+0 (secure 0 0 0) 0x7f63a000fbd0 con 0x5606afbbbb90 2024-08-19T01:34:53.720 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:53.717+0000 7f63b2e8c740 1 -- 172.21.15.83:0/2660978064 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f63a0011310 con 0x5606afbbbb90 2024-08-19T01:34:53.721 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:53.717+0000 7f638f400640 1 -- 172.21.15.83:0/2660978064 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f63a000fdd0 con 0x5606afbbbb90 2024-08-19T01:34:53.721 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:53.717+0000 7f63b2e8c740 10 client.25160 fetch_fsmap learned FSMap version 25 2024-08-19T01:34:53.721 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:53.717+0000 7f63b2e8c740 1 -- 172.21.15.83:0/2660978064 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5606afbd5540 con 0x5606afbbbb90 2024-08-19T01:34:53.722 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:53.717+0000 7f638f400640 1 -- 172.21.15.83:0/2660978064 <== mon.2 v2:172.21.15.117:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f63a0011310 con 0x5606afbbbb90 2024-08-19T01:34:53.722 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:53.717+0000 7f638f400640 1 -- 172.21.15.83:0/2660978064 <== mon.2 v2:172.21.15.117:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f63a0030030 con 0x5606afbbbb90 2024-08-19T01:34:53.722 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:53.717+0000 7f63b2e8c740 10 client.25160 fetch_fsmap finished waiting for FSMap version 25 2024-08-19T01:34:53.722 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:53.717+0000 7f63b2e8c740 10 client.25160 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-19T01:34:53.722 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:53.717+0000 7f63b2e8c740 20 client.25160 populate_metadata read hostname 'smithi083' 2024-08-19T01:34:53.722 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:53.717+0000 7f63b2e8c740 1 --2- 172.21.15.83:0/2660978064 >> [v2:172.21.15.117:6838/1967595704,v1:172.21.15.117:6839/1967595704] conn(0x5606afbd6a00 0x5606afbd8df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:34:53.722 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:53.718+0000 7f63b2e8c740 4 client.25160 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-19T01:34:53.722 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:53.718+0000 7f63b2e8c740 1 -- 172.21.15.83:0/2660978064 --> [v2:172.21.15.117:6838/1967595704,v1:172.21.15.117:6839/1967595704] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5606afb2f160 con 0x5606afbd6a00 2024-08-19T01:34:53.722 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:53.718+0000 7f63ad000640 1 --2- 172.21.15.83:0/2660978064 >> [v2:172.21.15.117:6838/1967595704,v1:172.21.15.117:6839/1967595704] conn(0x5606afbd6a00 0x5606afbd8df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:34:53.722 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:53.718+0000 7f63ad000640 1 --2- 172.21.15.83:0/2660978064 >> [v2:172.21.15.117:6838/1967595704,v1:172.21.15.117:6839/1967595704] conn(0x5606afbd6a00 0x5606afbd8df0 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=58548446481fb115 server_cookie=9bd11564886507ec in_seq=0 out_seq=0 2024-08-19T01:34:53.722 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:53.718+0000 7f638f400640 10 client.25160 ms_handle_connect on v2:172.21.15.117:6838/1967595704 2024-08-19T01:34:53.722 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:53.719+0000 7f638f400640 1 -- 172.21.15.83:0/2660978064 <== mds.0 v2:172.21.15.117:6838/1967595704 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5606afb2f160 con 0x5606afbd6a00 2024-08-19T01:34:53.723 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:53.719+0000 7f638f400640 10 client.25160 handle_command_reply: tid=0 2024-08-19T01:34:53.723 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:53.719+0000 7f63b2e8c740 1 client.25160 shutdown 2024-08-19T01:34:53.723 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:53.719+0000 7f63b2e8c740 1 -- 172.21.15.83:0/2660978064 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5606afbbbb90 msgr2=0x5606afbc3b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:34:53.723 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:53.719+0000 7f63b2e8c740 1 --2- 172.21.15.83:0/2660978064 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5606afbbbb90 0x5606afbc3b70 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f63a0011880 tx=0x7f63a0009a30 comp rx=0 tx=0).stop 2024-08-19T01:34:53.723 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:53.720+0000 7f63b2e8c740 1 -- 172.21.15.83:0/2660978064 shutdown_connections 2024-08-19T01:34:53.723 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:53.720+0000 7f63b2e8c740 1 --2- 172.21.15.83:0/2660978064 >> [v2:172.21.15.117:6838/1967595704,v1:172.21.15.117:6839/1967595704] conn(0x5606afbd6a00 0x5606afbd8df0 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:53.723 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:53.720+0000 7f63b2e8c740 1 --2- 172.21.15.83:0/2660978064 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5606afbc6d70 0x5606afbc45f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:53.723 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:53.720+0000 7f63b2e8c740 1 --2- 172.21.15.83:0/2660978064 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5606afbbc4b0 0x5606afbc40b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:53.723 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:53.720+0000 7f63b2e8c740 1 --2- 172.21.15.83:0/2660978064 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5606afbbbb90 0x5606afbc3b70 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:53.723 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:53.720+0000 7f63b2e8c740 1 -- 172.21.15.83:0/2660978064 >> 172.21.15.83:0/2660978064 conn(0x5606afbb8120 msgr2=0x5606afbbde50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:34:53.723 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:53.720+0000 7f638f400640 0 client.25160 ms_handle_reset on v2:172.21.15.117:6838/1967595704 2024-08-19T01:34:53.723 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:53.720+0000 7f63b2e8c740 1 -- 172.21.15.83:0/2660978064 shutdown_connections 2024-08-19T01:34:53.723 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:53.720+0000 7f63b2e8c740 1 -- 172.21.15.83:0/2660978064 wait complete. 2024-08-19T01:34:53.723 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:53.720+0000 7f63b2e8c740 20 client.25160 trim_cache size 0 max 16384 2024-08-19T01:34:53.741 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:53.739+0000 7f63b2e8c740 1 Processor -- start 2024-08-19T01:34:53.741 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:53.739+0000 7f63b2e8c740 1 -- start start 2024-08-19T01:34:53.741 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:53.740+0000 7f63b2e8c740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5606afbbbb90 0x5606afbde530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:34:53.741 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:53.740+0000 7f63b2e8c740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5606afbbc4b0 0x5606afbdea70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:34:53.742 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:53.740+0000 7f63b2e8c740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5606afbc6d70 0x5606afbc0e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:34:53.742 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:53.740+0000 7f63b2e8c740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f63a81b1f20 con 0x5606afbbc4b0 2024-08-19T01:34:53.742 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:53.740+0000 7f63b2e8c740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f63a81af7f0 con 0x5606afbc6d70 2024-08-19T01:34:53.742 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:53.740+0000 7f63b2e8c740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f63a81ad640 con 0x5606afbbbb90 2024-08-19T01:34:53.743 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:53.740+0000 7f63ae400640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5606afbbbb90 0x5606afbde530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:34:53.744 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:53.740+0000 7f63ad000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5606afbbc4b0 0x5606afbdea70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:34:53.744 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:53.740+0000 7f63ae400640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5606afbbbb90 0x5606afbde530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.83:58698/0 (socket says 172.21.15.83:58698) 2024-08-19T01:34:53.744 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:53.740+0000 7f63ae400640 1 -- 172.21.15.83:0/291670772 learned_addr learned my addr 172.21.15.83:0/291670772 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-08-19T01:34:53.744 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:53.740+0000 7f63ae400640 1 --2- 172.21.15.83:0/291670772 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5606afbc6d70 0x5606afbc0e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:34:53.744 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:53.741+0000 7f63ae400640 1 -- 172.21.15.83:0/291670772 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5606afbc6d70 msgr2=0x5606afbc0e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:34:53.744 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:53.741+0000 7f63ae400640 1 --2- 172.21.15.83:0/291670772 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5606afbc6d70 0x5606afbc0e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:53.744 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:53.741+0000 7f63ae400640 1 -- 172.21.15.83:0/291670772 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5606afbbc4b0 msgr2=0x5606afbdea70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:34:53.744 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:53.741+0000 7f63ae400640 1 --2- 172.21.15.83:0/291670772 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5606afbbc4b0 0x5606afbdea70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:53.744 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:53.741+0000 7f63ae400640 1 -- 172.21.15.83:0/291670772 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5606afbbc2e0 con 0x5606afbbbb90 2024-08-19T01:34:53.744 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:53.741+0000 7f63ad000640 1 --2- 172.21.15.83:0/291670772 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5606afbbc4b0 0x5606afbdea70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-19T01:34:53.744 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:53.741+0000 7f63ae400640 1 --2- 172.21.15.83:0/291670772 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5606afbbbb90 0x5606afbde530 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f63a0005d70 tx=0x7f63a0011d70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c8d6258e2321e38b server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:34:53.744 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:53.741+0000 7f639a000640 1 -- 172.21.15.83:0/291670772 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63a001e040 con 0x5606afbbbb90 2024-08-19T01:34:53.745 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:53.741+0000 7f639a000640 1 -- 172.21.15.83:0/291670772 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f63a0020070 con 0x5606afbbbb90 2024-08-19T01:34:53.745 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:53.741+0000 7f639a000640 1 -- 172.21.15.83:0/291670772 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63a0028900 con 0x5606afbbbb90 2024-08-19T01:34:53.745 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:53.742+0000 7f63b2e8c740 1 -- 172.21.15.83:0/291670772 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5606afbbbb90 msgr2=0x5606afbde530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:34:53.745 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:53.742+0000 7f63b2e8c740 1 --2- 172.21.15.83:0/291670772 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5606afbbbb90 0x5606afbde530 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f63a0005d70 tx=0x7f63a0011d70 comp rx=0 tx=0).stop 2024-08-19T01:34:53.745 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:53.742+0000 7f63b2e8c740 1 -- 172.21.15.83:0/291670772 shutdown_connections 2024-08-19T01:34:53.745 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:53.742+0000 7f63b2e8c740 1 --2- 172.21.15.83:0/291670772 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5606afbc6d70 0x5606afbc0e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:53.745 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:53.742+0000 7f63b2e8c740 1 --2- 172.21.15.83:0/291670772 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5606afbbc4b0 0x5606afbdea70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:53.745 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:53.742+0000 7f63b2e8c740 1 --2- 172.21.15.83:0/291670772 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5606afbbbb90 0x5606afbde530 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:53.745 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:53.742+0000 7f63b2e8c740 1 -- 172.21.15.83:0/291670772 >> 172.21.15.83:0/291670772 conn(0x5606afbb8120 msgr2=0x5606afbc5bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:34:53.745 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:53.742+0000 7f63b2e8c740 1 -- 172.21.15.83:0/291670772 shutdown_connections 2024-08-19T01:34:53.745 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:53.742+0000 7f63b2e8c740 1 -- 172.21.15.83:0/291670772 wait complete. 2024-08-19T01:34:53.747 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:53.743+0000 7f63b2e8c740 1 -- start start 2024-08-19T01:34:53.748 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:53.743+0000 7f63b2e8c740 1 Processor -- start 2024-08-19T01:34:53.748 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:53.743+0000 7f63b2e8c740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5606afbbbb90 0x5606afbc9d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:34:53.748 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:53.743+0000 7f63b2e8c740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5606afbbc4b0 0x5606afbca240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:34:53.748 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:53.743+0000 7f63b2e8c740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5606afbc6d70 0x5606afbca7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:34:53.748 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:53.743+0000 7f63b2e8c740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f63a81adcb0 con 0x5606afbbc4b0 2024-08-19T01:34:53.748 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:53.743+0000 7f63b2e8c740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f63a81af120 con 0x5606afbbbb90 2024-08-19T01:34:53.748 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:53.743+0000 7f63b2e8c740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f63a81ae380 con 0x5606afbc6d70 2024-08-19T01:34:53.748 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:53.743+0000 7f63b2e8c740 10 client.0 _ll_register_callbacks cb 0x7f63a00093a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-19T01:34:53.748 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:53.744+0000 7f63ae400640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5606afbbbb90 0x5606afbc9d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:34:53.748 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:53.744+0000 7f63ad000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5606afbbc4b0 0x5606afbca240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:34:53.748 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:53.744+0000 7f63ae400640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5606afbbbb90 0x5606afbc9d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.83:39494/0 (socket says 172.21.15.83:39494) 2024-08-19T01:34:53.749 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:53.744+0000 7f63ae400640 1 -- 172.21.15.83:0/4238216521 learned_addr learned my addr 172.21.15.83:0/4238216521 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-08-19T01:34:53.749 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:53.744+0000 7f63ae400640 1 --2- 172.21.15.83:0/4238216521 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5606afbc6d70 0x5606afbca7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:34:53.749 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:53.744+0000 7f63ae400640 1 -- 172.21.15.83:0/4238216521 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5606afbc6d70 msgr2=0x5606afbca7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:34:53.749 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:53.744+0000 7f63ae400640 1 --2- 172.21.15.83:0/4238216521 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5606afbc6d70 0x5606afbca7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:53.749 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:53.744+0000 7f63ae400640 1 -- 172.21.15.83:0/4238216521 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5606afbbc4b0 msgr2=0x5606afbca240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:34:53.749 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:53.744+0000 7f63ae400640 1 --2- 172.21.15.83:0/4238216521 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5606afbbc4b0 0x5606afbca240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:53.749 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:53.744+0000 7f63ae400640 1 -- 172.21.15.83:0/4238216521 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5606afc3f7c0 con 0x5606afbbbb90 2024-08-19T01:34:53.749 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:53.744+0000 7f63ae400640 1 --2- 172.21.15.83:0/4238216521 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5606afbbbb90 0x5606afbc9d00 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f63a00116c0 tx=0x7f63a0010c20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d00bb0f41ef1686c server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:34:53.749 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:53.744+0000 7f638f400640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-08-19T01:34:53.749 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:53.745+0000 7f638f400640 1 -- 172.21.15.83:0/4238216521 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63a001e070 con 0x5606afbbbb90 2024-08-19T01:34:53.749 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:53.745+0000 7f638f400640 1 -- 172.21.15.83:0/4238216521 <== mon.2 v2:172.21.15.117:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8269+0+0 (secure 0 0 0) 0x7f63a0010d00 con 0x5606afbbbb90 2024-08-19T01:34:53.749 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:53.745+0000 7f63b2e8c740 1 -- 172.21.15.83:0/4238216521 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f63a0020070 con 0x5606afbbbb90 2024-08-19T01:34:53.749 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:53.745+0000 7f638f400640 1 -- 172.21.15.83:0/4238216521 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f63a0033450 con 0x5606afbbbb90 2024-08-19T01:34:53.749 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:53.745+0000 7f63b2e8c740 10 client.25166 fetch_fsmap learned FSMap version 25 2024-08-19T01:34:53.749 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:53.745+0000 7f63b2e8c740 1 -- 172.21.15.83:0/4238216521 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5606afc40ad0 con 0x5606afbbbb90 2024-08-19T01:34:53.750 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:53.745+0000 7f638f400640 1 -- 172.21.15.83:0/4238216521 <== mon.2 v2:172.21.15.117:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f63a0020070 con 0x5606afbbbb90 2024-08-19T01:34:53.750 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:53.746+0000 7f638f400640 1 -- 172.21.15.83:0/4238216521 <== mon.2 v2:172.21.15.117:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f63a004a080 con 0x5606afbbbb90 2024-08-19T01:34:53.750 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:53.746+0000 7f63b2e8c740 10 client.25166 fetch_fsmap finished waiting for FSMap version 25 2024-08-19T01:34:53.750 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:53.746+0000 7f63b2e8c740 10 client.25166 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-19T01:34:53.750 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:53.746+0000 7f63b2e8c740 20 client.25166 populate_metadata read hostname 'smithi083' 2024-08-19T01:34:53.750 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:53.746+0000 7f63b2e8c740 1 --2- 172.21.15.83:0/4238216521 >> [v2:172.21.15.117:6838/1967595704,v1:172.21.15.117:6839/1967595704] conn(0x5606afc41080 0x5606afc43470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:34:53.750 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:53.746+0000 7f63b2e8c740 4 client.25166 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-08-19T01:34:53.750 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:53.746+0000 7f63b2e8c740 1 -- 172.21.15.83:0/4238216521 --> [v2:172.21.15.117:6838/1967595704,v1:172.21.15.117:6839/1967595704] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5606afb29670 con 0x5606afc41080 2024-08-19T01:34:53.750 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:53.746+0000 7f63ad000640 1 --2- 172.21.15.83:0/4238216521 >> [v2:172.21.15.117:6838/1967595704,v1:172.21.15.117:6839/1967595704] conn(0x5606afc41080 0x5606afc43470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:34:53.751 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:53.747+0000 7f63ad000640 1 --2- 172.21.15.83:0/4238216521 >> [v2:172.21.15.117:6838/1967595704,v1:172.21.15.117:6839/1967595704] conn(0x5606afc41080 0x5606afc43470 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5c42286485cc7fc8 server_cookie=11b16c8be7d55845 in_seq=0 out_seq=0 2024-08-19T01:34:53.751 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:53.747+0000 7f638f400640 10 client.25166 ms_handle_connect on v2:172.21.15.117:6838/1967595704 2024-08-19T01:34:53.752 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:53.747+0000 7f638f400640 1 -- 172.21.15.83:0/4238216521 <== mds.0 v2:172.21.15.117:6838/1967595704 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x5606afb29670 con 0x5606afc41080 2024-08-19T01:34:53.752 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:53.747+0000 7f638f400640 10 client.25166 handle_command_reply: tid=0 2024-08-19T01:34:53.752 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:53.747+0000 7f63b2e8c740 1 client.25166 shutdown 2024-08-19T01:34:53.752 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:53.747+0000 7f63b2e8c740 1 -- 172.21.15.83:0/4238216521 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5606afbbbb90 msgr2=0x5606afbc9d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:34:53.752 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:53.747+0000 7f63b2e8c740 1 --2- 172.21.15.83:0/4238216521 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5606afbbbb90 0x5606afbc9d00 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f63a00116c0 tx=0x7f63a0010c20 comp rx=0 tx=0).stop 2024-08-19T01:34:53.752 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:53.747+0000 7f63b2e8c740 1 -- 172.21.15.83:0/4238216521 shutdown_connections 2024-08-19T01:34:53.752 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:53.747+0000 7f63b2e8c740 1 --2- 172.21.15.83:0/4238216521 >> [v2:172.21.15.117:6838/1967595704,v1:172.21.15.117:6839/1967595704] conn(0x5606afc41080 0x5606afc43470 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:53.752 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:53.747+0000 7f63b2e8c740 1 --2- 172.21.15.83:0/4238216521 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5606afbc6d70 0x5606afbca7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:53.752 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:53.747+0000 7f63b2e8c740 1 --2- 172.21.15.83:0/4238216521 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5606afbbc4b0 0x5606afbca240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:53.752 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:53.747+0000 7f63b2e8c740 1 --2- 172.21.15.83:0/4238216521 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5606afbbbb90 0x5606afbc9d00 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:53.752 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:53.747+0000 7f63b2e8c740 1 -- 172.21.15.83:0/4238216521 >> 172.21.15.83:0/4238216521 conn(0x5606afbb8120 msgr2=0x5606afbc53d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:34:53.752 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:53.747+0000 7f638f400640 0 client.25166 ms_handle_reset on v2:172.21.15.117:6838/1967595704 2024-08-19T01:34:53.752 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:53.747+0000 7f63b2e8c740 1 -- 172.21.15.83:0/4238216521 shutdown_connections 2024-08-19T01:34:53.752 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:53.747+0000 7f63b2e8c740 1 -- 172.21.15.83:0/4238216521 wait complete. 2024-08-19T01:34:53.753 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:53.747+0000 7f63b2e8c740 20 client.25166 trim_cache size 0 max 16384 2024-08-19T01:34:53.753 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-08-19T01:34:53.753 INFO:teuthology.orchestra.run.smithi083.stdout: "status": "IDLE (waiting for more scrubs)", 2024-08-19T01:34:53.753 INFO:teuthology.orchestra.run.smithi083.stdout: "scrubs": { 2024-08-19T01:34:53.753 INFO:teuthology.orchestra.run.smithi083.stdout: "757a53a7-cd2a-4afd-b9b6-548543c74c88": { 2024-08-19T01:34:53.753 INFO:teuthology.orchestra.run.smithi083.stdout: "path": "/", 2024-08-19T01:34:53.753 INFO:teuthology.orchestra.run.smithi083.stdout: "tag": "757a53a7-cd2a-4afd-b9b6-548543c74c88", 2024-08-19T01:34:53.753 INFO:teuthology.orchestra.run.smithi083.stdout: "options": "recursive,force" 2024-08-19T01:34:53.753 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-08-19T01:34:53.753 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-08-19T01:34:53.754 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-08-19T01:34:53.754 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:53.748+0000 7f63b0400640 1 -- 172.21.15.83:0/2662666822 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f63900a2c90 msgr2=0x7f63900a5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:34:53.754 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:53.748+0000 7f63b0400640 1 --2- 172.21.15.83:0/2662666822 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f63900a2c90 0x7f63900a5150 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f639c097eb0 tx=0x7f639c0d1820 comp rx=0 tx=0).stop 2024-08-19T01:34:53.754 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:53.748+0000 7f63b0400640 1 -- 172.21.15.83:0/2662666822 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f63a81018a0 msgr2=0x7f63a819a3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:34:53.755 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:53.748+0000 7f63b0400640 1 --2- 172.21.15.83:0/2662666822 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f63a81018a0 0x7f63a819a3e0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f639c00e9f0 tx=0x7f639c00eeb0 comp rx=0 tx=0).stop 2024-08-19T01:34:53.755 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:53.748+0000 7f63b0400640 1 -- 172.21.15.83:0/2662666822 shutdown_connections 2024-08-19T01:34:53.755 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:53.748+0000 7f63b0400640 1 --2- 172.21.15.83:0/2662666822 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f63900a2c90 0x7f63900a5150 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:53.755 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:53.749+0000 7f63b0400640 1 --2- 172.21.15.83:0/2662666822 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f63a81078a0 0x7f63a819ecb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:53.755 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:53.749+0000 7f63b0400640 1 --2- 172.21.15.83:0/2662666822 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f63a8102240 0x7f63a819a920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:53.755 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:53.749+0000 7f63b0400640 1 --2- 172.21.15.83:0/2662666822 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f63a81018a0 0x7f63a819a3e0 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:53.755 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:53.749+0000 7f63b0400640 1 -- 172.21.15.83:0/2662666822 >> 172.21.15.83:0/2662666822 conn(0x7f63a80fd5e0 msgr2=0x7f63a810cf30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:34:53.755 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:53.749+0000 7f63b0400640 1 -- 172.21.15.83:0/2662666822 shutdown_connections 2024-08-19T01:34:53.755 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:34:53.749+0000 7f63b0400640 1 -- 172.21.15.83:0/2662666822 wait complete. 2024-08-19T01:34:53.877 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:34:53 smithi117 ceph-mon[22016]: pgmap v340: 97 pgs: 97 active+clean; 649 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 1.7 KiB/s wr, 7 op/s 2024-08-19T01:34:53.877 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:34:53 smithi117 ceph-mon[22016]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-19T01:34:53.966 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:34:53 smithi167 ceph-mon[21875]: pgmap v340: 97 pgs: 97 active+clean; 649 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 1.7 KiB/s wr, 7 op/s 2024-08-19T01:34:53.966 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:34:53 smithi167 ceph-mon[21875]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-19T01:34:54.519 INFO:tasks.cephfs.filesystem:scrub status for tag:757a53a7-cd2a-4afd-b9b6-548543c74c88 - {'path': '/', 'tag': '757a53a7-cd2a-4afd-b9b6-548543c74c88', 'options': 'recursive,force'} 2024-08-19T01:34:55.904 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:34:55 smithi083 ceph-mon[17120]: pgmap v341: 97 pgs: 97 active+clean; 649 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 1.7 KiB/s wr, 8 op/s 2024-08-19T01:34:55.905 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:34:55 smithi083 ceph-mon[17120]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' 2024-08-19T01:34:55.905 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:34:55 smithi083 ceph-mon[17120]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' 2024-08-19T01:34:55.905 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:34:55 smithi083 ceph-mon[17120]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' 2024-08-19T01:34:55.905 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:34:55 smithi083 ceph-mon[17120]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' 2024-08-19T01:34:55.952 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:34:55 smithi117 ceph-mon[22016]: pgmap v341: 97 pgs: 97 active+clean; 649 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 1.7 KiB/s wr, 8 op/s 2024-08-19T01:34:55.952 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:34:55 smithi117 ceph-mon[22016]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' 2024-08-19T01:34:55.952 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:34:55 smithi117 ceph-mon[22016]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' 2024-08-19T01:34:55.953 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:34:55 smithi117 ceph-mon[22016]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' 2024-08-19T01:34:55.953 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:34:55 smithi117 ceph-mon[22016]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' 2024-08-19T01:34:55.966 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:34:55 smithi167 ceph-mon[21875]: pgmap v341: 97 pgs: 97 active+clean; 649 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 1.7 KiB/s wr, 8 op/s 2024-08-19T01:34:55.966 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:34:55 smithi167 ceph-mon[21875]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' 2024-08-19T01:34:55.966 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:34:55 smithi167 ceph-mon[21875]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' 2024-08-19T01:34:55.966 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:34:55 smithi167 ceph-mon[21875]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' 2024-08-19T01:34:55.966 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:34:55 smithi167 ceph-mon[21875]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' 2024-08-19T01:34:57.404 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:34:57 smithi083 ceph-mon[17120]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' 2024-08-19T01:34:57.405 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:34:57 smithi083 ceph-mon[17120]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' 2024-08-19T01:34:57.405 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:34:57 smithi083 ceph-mon[17120]: pgmap v342: 97 pgs: 97 active+clean; 656 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 2.0 KiB/s wr, 8 op/s 2024-08-19T01:34:57.405 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:34:57 smithi083 ceph-mon[17120]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-19T01:34:57.405 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:34:57 smithi083 ceph-mon[17120]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-19T01:34:57.405 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:34:57 smithi083 ceph-mon[17120]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' 2024-08-19T01:34:57.452 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:34:57 smithi117 ceph-mon[22016]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' 2024-08-19T01:34:57.452 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:34:57 smithi117 ceph-mon[22016]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' 2024-08-19T01:34:57.452 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:34:57 smithi117 ceph-mon[22016]: pgmap v342: 97 pgs: 97 active+clean; 656 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 2.0 KiB/s wr, 8 op/s 2024-08-19T01:34:57.452 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:34:57 smithi117 ceph-mon[22016]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-19T01:34:57.453 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:34:57 smithi117 ceph-mon[22016]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-19T01:34:57.453 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:34:57 smithi117 ceph-mon[22016]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' 2024-08-19T01:34:57.466 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:34:57 smithi167 ceph-mon[21875]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' 2024-08-19T01:34:57.466 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:34:57 smithi167 ceph-mon[21875]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' 2024-08-19T01:34:57.466 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:34:57 smithi167 ceph-mon[21875]: pgmap v342: 97 pgs: 97 active+clean; 656 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 2.0 KiB/s wr, 8 op/s 2024-08-19T01:34:57.467 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:34:57 smithi167 ceph-mon[21875]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-19T01:34:57.467 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:34:57 smithi167 ceph-mon[21875]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-19T01:34:57.467 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:34:57 smithi167 ceph-mon[21875]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' 2024-08-19T01:34:59.702 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:34:59 smithi117 ceph-mon[22016]: pgmap v343: 97 pgs: 97 active+clean; 656 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 1.2 KiB/s wr, 7 op/s 2024-08-19T01:34:59.716 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:34:59 smithi167 ceph-mon[21875]: pgmap v343: 97 pgs: 97 active+clean; 656 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 1.2 KiB/s wr, 7 op/s 2024-08-19T01:34:59.904 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:34:59 smithi083 ceph-mon[17120]: pgmap v343: 97 pgs: 97 active+clean; 656 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 1.2 KiB/s wr, 7 op/s 2024-08-19T01:35:01.702 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:35:01 smithi117 ceph-mon[22016]: pgmap v344: 97 pgs: 97 active+clean; 656 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 1.2 KiB/s wr, 7 op/s 2024-08-19T01:35:01.716 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:35:01 smithi167 ceph-mon[21875]: pgmap v344: 97 pgs: 97 active+clean; 656 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 1.2 KiB/s wr, 7 op/s 2024-08-19T01:35:01.904 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:35:01 smithi083 ceph-mon[17120]: pgmap v344: 97 pgs: 97 active+clean; 656 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 1.2 KiB/s wr, 7 op/s 2024-08-19T01:35:03.716 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:35:03 smithi167 ceph-mon[21875]: pgmap v345: 97 pgs: 97 active+clean; 656 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 938 B/s wr, 6 op/s 2024-08-19T01:35:03.904 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:35:03 smithi083 ceph-mon[17120]: pgmap v345: 97 pgs: 97 active+clean; 656 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 938 B/s wr, 6 op/s 2024-08-19T01:35:03.952 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:35:03 smithi117 ceph-mon[22016]: pgmap v345: 97 pgs: 97 active+clean; 656 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 938 B/s wr, 6 op/s 2024-08-19T01:35:04.148 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:04.146+0000 7f5d2f400640 1 -- 172.21.15.83:0/2874160868 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5d28103a90 msgr2=0x7f5d28103e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:35:04.148 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:04.146+0000 7f5d2f400640 1 --2- 172.21.15.83:0/2874160868 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5d28103a90 0x7f5d28103e90 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f5d1c009ca0 tx=0x7f5d1c02a8c0 comp rx=0 tx=0).stop 2024-08-19T01:35:04.148 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:04.146+0000 7f5d2f400640 1 -- 172.21.15.83:0/2874160868 shutdown_connections 2024-08-19T01:35:04.148 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:04.146+0000 7f5d2f400640 1 --2- 172.21.15.83:0/2874160868 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5d28105640 0x7f5d2810bed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:35:04.148 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:04.146+0000 7f5d2f400640 1 --2- 172.21.15.83:0/2874160868 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5d28104c80 0x7f5d28105100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:35:04.148 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:04.146+0000 7f5d2f400640 1 --2- 172.21.15.83:0/2874160868 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5d28103a90 0x7f5d28103e90 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:35:04.148 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:04.146+0000 7f5d2f400640 1 -- 172.21.15.83:0/2874160868 >> 172.21.15.83:0/2874160868 conn(0x7f5d280ff210 msgr2=0x7f5d28101630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:35:04.148 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:04.146+0000 7f5d2f400640 1 -- 172.21.15.83:0/2874160868 shutdown_connections 2024-08-19T01:35:04.148 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:04.146+0000 7f5d2f400640 1 -- 172.21.15.83:0/2874160868 wait complete. 2024-08-19T01:35:04.150 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:04.147+0000 7f5d2f400640 1 Processor -- start 2024-08-19T01:35:04.150 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:04.147+0000 7f5d2f400640 1 -- start start 2024-08-19T01:35:04.151 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:04.147+0000 7f5d2f400640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5d28103a90 0x7f5d28199cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:35:04.151 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:04.147+0000 7f5d2f400640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5d28104c80 0x7f5d2819a230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:35:04.151 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:04.147+0000 7f5d2f400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5d28105640 0x7f5d281a1260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:35:04.151 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:04.147+0000 7f5d2f400640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f5d2810f740 con 0x7f5d28105640 2024-08-19T01:35:04.151 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:04.147+0000 7f5d2f400640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f5d2810f5c0 con 0x7f5d28103a90 2024-08-19T01:35:04.151 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:04.147+0000 7f5d2f400640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f5d2810f8c0 con 0x7f5d28104c80 2024-08-19T01:35:04.151 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:04.147+0000 7f5d2d400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5d28105640 0x7f5d281a1260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:35:04.151 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:04.148+0000 7f5d2d400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5d28105640 0x7f5d281a1260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:34762/0 (socket says 172.21.15.83:34762) 2024-08-19T01:35:04.151 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:04.148+0000 7f5d2d400640 1 -- 172.21.15.83:0/320926717 learned_addr learned my addr 172.21.15.83:0/320926717 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-08-19T01:35:04.152 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:04.148+0000 7f5d2ca00640 1 --2- 172.21.15.83:0/320926717 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5d28103a90 0x7f5d28199cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:35:04.152 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:04.148+0000 7f5d27e00640 1 --2- 172.21.15.83:0/320926717 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5d28104c80 0x7f5d2819a230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:35:04.152 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:04.148+0000 7f5d2d400640 1 -- 172.21.15.83:0/320926717 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5d28104c80 msgr2=0x7f5d2819a230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:35:04.152 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:04.148+0000 7f5d2d400640 1 --2- 172.21.15.83:0/320926717 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5d28104c80 0x7f5d2819a230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:35:04.152 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:04.148+0000 7f5d2d400640 1 -- 172.21.15.83:0/320926717 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5d28103a90 msgr2=0x7f5d28199cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:35:04.152 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:04.148+0000 7f5d2d400640 1 --2- 172.21.15.83:0/320926717 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5d28103a90 0x7f5d28199cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:35:04.152 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:04.148+0000 7f5d2d400640 1 -- 172.21.15.83:0/320926717 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5d281a1900 con 0x7f5d28105640 2024-08-19T01:35:04.152 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:04.148+0000 7f5d2d400640 1 --2- 172.21.15.83:0/320926717 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5d28105640 0x7f5d281a1260 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f5d2000e970 tx=0x7f5d2000ed70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:35:04.152 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:04.148+0000 7f5d25600640 1 -- 172.21.15.83:0/320926717 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d2000ccc0 con 0x7f5d28105640 2024-08-19T01:35:04.153 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:04.148+0000 7f5d25600640 1 -- 172.21.15.83:0/320926717 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5d2000ce60 con 0x7f5d28105640 2024-08-19T01:35:04.153 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:04.148+0000 7f5d25600640 1 -- 172.21.15.83:0/320926717 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d20010490 con 0x7f5d28105640 2024-08-19T01:35:04.153 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:04.149+0000 7f5d2f400640 1 -- 172.21.15.83:0/320926717 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5d281a1bf0 con 0x7f5d28105640 2024-08-19T01:35:04.153 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:04.149+0000 7f5d2f400640 1 -- 172.21.15.83:0/320926717 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5d281a20c0 con 0x7f5d28105640 2024-08-19T01:35:04.154 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:04.152+0000 7f5d25600640 1 -- 172.21.15.83:0/320926717 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 19) ==== 137884+0+0 (secure 0 0 0) 0x7f5d20005020 con 0x7f5d28105640 2024-08-19T01:35:04.154 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:04.152+0000 7f5d32238740 1 Processor -- start 2024-08-19T01:35:04.154 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:04.152+0000 7f5d32238740 1 -- start start 2024-08-19T01:35:04.155 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:04.152+0000 7f5d32238740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a0278d3470 0x55a0278d3870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:35:04.155 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:04.152+0000 7f5d32238740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a0278d3db0 0x55a0278d8870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:35:04.155 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:04.152+0000 7f5d32238740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a0278d8e40 0x55a0278db300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:35:04.155 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:04.152+0000 7f5d32238740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55a027837890 con 0x55a0278d3db0 2024-08-19T01:35:04.155 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:04.152+0000 7f5d32238740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55a0277cfd90 con 0x55a0278d3470 2024-08-19T01:35:04.155 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:04.152+0000 7f5d32238740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55a027833850 con 0x55a0278d8e40 2024-08-19T01:35:04.155 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:04.152+0000 7f5d2ca00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a0278d3db0 0x55a0278d8870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:35:04.155 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:04.152+0000 7f5d27e00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a0278d3470 0x55a0278d3870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:35:04.155 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:04.153+0000 7f5d27e00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a0278d3470 0x55a0278d3870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.83:58748/0 (socket says 172.21.15.83:58748) 2024-08-19T01:35:04.155 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:04.153+0000 7f5d27e00640 1 -- 172.21.15.83:0/3074945381 learned_addr learned my addr 172.21.15.83:0/3074945381 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-08-19T01:35:04.155 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:04.153+0000 7f5d27e00640 1 --2- 172.21.15.83:0/3074945381 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a0278d8e40 0x55a0278db300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:35:04.155 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:04.153+0000 7f5d2ca00640 1 -- 172.21.15.83:0/3074945381 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a0278d8e40 msgr2=0x55a0278db300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:35:04.156 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:04.153+0000 7f5d2ca00640 1 --2- 172.21.15.83:0/3074945381 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a0278d8e40 0x55a0278db300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:35:04.156 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:04.153+0000 7f5d2ca00640 1 -- 172.21.15.83:0/3074945381 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a0278d3470 msgr2=0x55a0278d3870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:35:04.156 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:04.153+0000 7f5d2ca00640 1 --2- 172.21.15.83:0/3074945381 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a0278d3470 0x55a0278d3870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:35:04.156 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:04.153+0000 7f5d2ca00640 1 -- 172.21.15.83:0/3074945381 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a02754b210 con 0x55a0278d3db0 2024-08-19T01:35:04.160 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:04.153+0000 7f5d2ca00640 1 --2- 172.21.15.83:0/3074945381 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a0278d3db0 0x55a0278d8870 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f5d1c02ade0 tx=0x7f5d1c009800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a08d1a56a02c0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:35:04.160 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:04.154+0000 7f5d19c00640 1 -- 172.21.15.83:0/3074945381 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d1c031650 con 0x55a0278d3db0 2024-08-19T01:35:04.160 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:04.154+0000 7f5d19c00640 1 -- 172.21.15.83:0/3074945381 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5d1c036030 con 0x55a0278d3db0 2024-08-19T01:35:04.160 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:04.154+0000 7f5d19c00640 1 -- 172.21.15.83:0/3074945381 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d1c004700 con 0x55a0278d3db0 2024-08-19T01:35:04.161 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:04.154+0000 7f5d25600640 1 --2- 172.21.15.83:0/320926717 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f5d100a2d50 0x7f5d100a5210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:35:04.161 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:04.154+0000 7f5d25600640 1 -- 172.21.15.83:0/320926717 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8269+0+0 (secure 0 0 0) 0x7f5d200cf6c0 con 0x7f5d28105640 2024-08-19T01:35:04.161 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:04.154+0000 7f5d32238740 1 -- 172.21.15.83:0/3074945381 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a0278d3db0 msgr2=0x55a0278d8870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:35:04.161 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:04.154+0000 7f5d32238740 1 --2- 172.21.15.83:0/3074945381 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a0278d3db0 0x55a0278d8870 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f5d1c02ade0 tx=0x7f5d1c009800 comp rx=0 tx=0).stop 2024-08-19T01:35:04.161 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:04.154+0000 7f5d27e00640 1 --2- 172.21.15.83:0/320926717 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f5d100a2d50 0x7f5d100a5210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:35:04.161 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:04.155+0000 7f5d32238740 1 -- 172.21.15.83:0/3074945381 shutdown_connections 2024-08-19T01:35:04.161 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:04.155+0000 7f5d32238740 1 --2- 172.21.15.83:0/3074945381 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a0278d8e40 0x55a0278db300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:35:04.161 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:04.155+0000 7f5d32238740 1 --2- 172.21.15.83:0/3074945381 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a0278d3db0 0x55a0278d8870 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:35:04.161 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:04.155+0000 7f5d32238740 1 --2- 172.21.15.83:0/3074945381 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a0278d3470 0x55a0278d3870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:35:04.161 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:04.155+0000 7f5d32238740 1 -- 172.21.15.83:0/3074945381 >> 172.21.15.83:0/3074945381 conn(0x55a0278ceae0 msgr2=0x55a0278ceee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:35:04.161 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:04.155+0000 7f5d32238740 1 -- 172.21.15.83:0/3074945381 shutdown_connections 2024-08-19T01:35:04.161 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:04.155+0000 7f5d32238740 1 -- 172.21.15.83:0/3074945381 wait complete. 2024-08-19T01:35:04.161 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:04.155+0000 7f5d27e00640 1 --2- 172.21.15.83:0/320926717 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f5d100a2d50 0x7f5d100a5210 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f5d14005850 tx=0x7f5d1400b7e0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:35:04.162 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:04.155+0000 7f5d32238740 1 -- start start 2024-08-19T01:35:04.162 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:04.156+0000 7f5d32238740 1 Processor -- start 2024-08-19T01:35:04.162 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:04.156+0000 7f5d32238740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a0278d3470 0x55a0278dd550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:35:04.162 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:04.156+0000 7f5d32238740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a0278d3db0 0x55a0278dda90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:35:04.162 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:04.156+0000 7f5d32238740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a0278d8e40 0x55a0278e4ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:35:04.162 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:04.156+0000 7f5d32238740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55a0278dbcd0 con 0x55a0278d3db0 2024-08-19T01:35:04.162 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:04.156+0000 7f5d32238740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55a0277f2f70 con 0x55a0278d3470 2024-08-19T01:35:04.162 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:04.156+0000 7f5d32238740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55a0278dbe50 con 0x55a0278d8e40 2024-08-19T01:35:04.162 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:04.156+0000 7f5d32238740 10 client.0 _ll_register_callbacks cb 0x55a02782f8d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-19T01:35:04.163 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:04.156+0000 7f5d27e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a0278d3db0 0x55a0278dda90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:35:04.163 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:04.156+0000 7f5d27e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a0278d3db0 0x55a0278dda90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:34778/0 (socket says 172.21.15.83:34778) 2024-08-19T01:35:04.163 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:04.156+0000 7f5d27e00640 1 -- 172.21.15.83:0/2835135658 learned_addr learned my addr 172.21.15.83:0/2835135658 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-08-19T01:35:04.163 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:04.156+0000 7f5d2d400640 1 --2- 172.21.15.83:0/2835135658 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a0278d8e40 0x55a0278e4ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:35:04.163 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:04.156+0000 7f5d2ca00640 1 --2- 172.21.15.83:0/2835135658 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a0278d3470 0x55a0278dd550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:35:04.163 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:04.157+0000 7f5d27e00640 1 -- 172.21.15.83:0/2835135658 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a0278d8e40 msgr2=0x55a0278e4ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:35:04.163 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:04.157+0000 7f5d27e00640 1 --2- 172.21.15.83:0/2835135658 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a0278d8e40 0x55a0278e4ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:35:04.163 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:04.157+0000 7f5d27e00640 1 -- 172.21.15.83:0/2835135658 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a0278d3470 msgr2=0x55a0278dd550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:35:04.163 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:04.157+0000 7f5d27e00640 1 --2- 172.21.15.83:0/2835135658 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a0278d3470 0x55a0278dd550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:35:04.163 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:04.157+0000 7f5d27e00640 1 -- 172.21.15.83:0/2835135658 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a027617620 con 0x55a0278d3db0 2024-08-19T01:35:04.163 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:04.157+0000 7f5d27e00640 1 --2- 172.21.15.83:0/2835135658 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a0278d3db0 0x55a0278dda90 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f5d1400e8e0 tx=0x7f5d14004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ae8470bfb7fd560e server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:35:04.163 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:04.157+0000 7f5d0e000640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-08-19T01:35:04.163 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:04.157+0000 7f5d0e000640 1 -- 172.21.15.83:0/2835135658 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d14004020 con 0x55a0278d3db0 2024-08-19T01:35:04.164 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:04.157+0000 7f5d0e000640 1 -- 172.21.15.83:0/2835135658 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8269+0+0 (secure 0 0 0) 0x7f5d140041c0 con 0x55a0278d3db0 2024-08-19T01:35:04.164 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:04.157+0000 7f5d32238740 1 -- 172.21.15.83:0/2835135658 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5d1c036030 con 0x55a0278d3db0 2024-08-19T01:35:04.164 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:04.157+0000 7f5d0e000640 1 -- 172.21.15.83:0/2835135658 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f5d140043c0 con 0x55a0278d3db0 2024-08-19T01:35:04.164 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:04.157+0000 7f5d32238740 10 client.15111 fetch_fsmap learned FSMap version 25 2024-08-19T01:35:04.164 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:04.157+0000 7f5d32238740 1 -- 172.21.15.83:0/2835135658 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55a0278e7d10 con 0x55a0278d3db0 2024-08-19T01:35:04.164 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:04.158+0000 7f5d0e000640 1 -- 172.21.15.83:0/2835135658 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5d1c036030 con 0x55a0278d3db0 2024-08-19T01:35:04.164 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:04.158+0000 7f5d0e000640 1 -- 172.21.15.83:0/2835135658 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f5d14029030 con 0x55a0278d3db0 2024-08-19T01:35:04.164 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:04.158+0000 7f5d32238740 10 client.15111 fetch_fsmap finished waiting for FSMap version 25 2024-08-19T01:35:04.164 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:04.158+0000 7f5d32238740 10 client.15111 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-08-19T01:35:04.164 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:04.158+0000 7f5d32238740 20 client.15111 populate_metadata read hostname 'smithi083' 2024-08-19T01:35:04.164 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:04.158+0000 7f5d32238740 1 --2- 172.21.15.83:0/2835135658 >> [v2:172.21.15.117:6838/1967595704,v1:172.21.15.117:6839/1967595704] conn(0x55a0278e9450 0x55a0278eb8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:35:04.164 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:04.158+0000 7f5d32238740 4 client.15111 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-19T01:35:04.164 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:04.158+0000 7f5d32238740 1 -- 172.21.15.83:0/2835135658 --> [v2:172.21.15.117:6838/1967595704,v1:172.21.15.117:6839/1967595704] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a027844fc0 con 0x55a0278e9450 2024-08-19T01:35:04.165 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:04.158+0000 7f5d2ca00640 1 --2- 172.21.15.83:0/2835135658 >> [v2:172.21.15.117:6838/1967595704,v1:172.21.15.117:6839/1967595704] conn(0x55a0278e9450 0x55a0278eb8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:35:04.165 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:04.159+0000 7f5d2ca00640 1 --2- 172.21.15.83:0/2835135658 >> [v2:172.21.15.117:6838/1967595704,v1:172.21.15.117:6839/1967595704] conn(0x55a0278e9450 0x55a0278eb8a0 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c047ea8161cd2202 server_cookie=8a215e0b00c72969 in_seq=0 out_seq=0 2024-08-19T01:35:04.165 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:04.159+0000 7f5d0e000640 10 client.15111 ms_handle_connect on v2:172.21.15.117:6838/1967595704 2024-08-19T01:35:04.166 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:04.160+0000 7f5d0e000640 1 -- 172.21.15.83:0/2835135658 <== mds.0 v2:172.21.15.117:6838/1967595704 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55a027844fc0 con 0x55a0278e9450 2024-08-19T01:35:04.166 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:04.160+0000 7f5d0e000640 10 client.15111 handle_command_reply: tid=0 2024-08-19T01:35:04.166 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:04.160+0000 7f5d32238740 1 client.15111 shutdown 2024-08-19T01:35:04.166 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:04.160+0000 7f5d32238740 1 -- 172.21.15.83:0/2835135658 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a0278d3db0 msgr2=0x55a0278dda90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:35:04.166 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:04.160+0000 7f5d32238740 1 --2- 172.21.15.83:0/2835135658 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a0278d3db0 0x55a0278dda90 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f5d1400e8e0 tx=0x7f5d14004730 comp rx=0 tx=0).stop 2024-08-19T01:35:04.166 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:04.160+0000 7f5d32238740 1 -- 172.21.15.83:0/2835135658 shutdown_connections 2024-08-19T01:35:04.166 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:04.160+0000 7f5d32238740 1 --2- 172.21.15.83:0/2835135658 >> [v2:172.21.15.117:6838/1967595704,v1:172.21.15.117:6839/1967595704] conn(0x55a0278e9450 0x55a0278eb8a0 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:35:04.166 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:04.160+0000 7f5d32238740 1 --2- 172.21.15.83:0/2835135658 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a0278d8e40 0x55a0278e4ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:35:04.166 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:04.160+0000 7f5d32238740 1 --2- 172.21.15.83:0/2835135658 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a0278d3db0 0x55a0278dda90 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:35:04.167 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:04.160+0000 7f5d32238740 1 --2- 172.21.15.83:0/2835135658 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a0278d3470 0x55a0278dd550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:35:04.167 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:04.160+0000 7f5d0e000640 0 client.15111 ms_handle_reset on v2:172.21.15.117:6838/1967595704 2024-08-19T01:35:04.167 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:04.160+0000 7f5d32238740 1 -- 172.21.15.83:0/2835135658 >> 172.21.15.83:0/2835135658 conn(0x55a0278ceae0 msgr2=0x55a0278d02f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:35:04.167 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:04.160+0000 7f5d32238740 1 -- 172.21.15.83:0/2835135658 shutdown_connections 2024-08-19T01:35:04.167 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:04.160+0000 7f5d32238740 1 -- 172.21.15.83:0/2835135658 wait complete. 2024-08-19T01:35:04.167 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:04.160+0000 7f5d32238740 20 client.15111 trim_cache size 0 max 16384 2024-08-19T01:35:04.177 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:04.175+0000 7f5d32238740 1 Processor -- start 2024-08-19T01:35:04.178 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:04.175+0000 7f5d32238740 1 -- start start 2024-08-19T01:35:04.178 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:04.175+0000 7f5d32238740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a0278d3470 0x55a0278d4f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:35:04.178 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:04.175+0000 7f5d32238740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a0278d3db0 0x55a0278e4a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:35:04.178 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:04.176+0000 7f5d32238740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a0278d8e40 0x55a0278d54d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:35:04.178 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:04.176+0000 7f5d32238740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f5d281ad730 con 0x55a0278d3db0 2024-08-19T01:35:04.178 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:04.176+0000 7f5d32238740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f5d281ab000 con 0x55a0278d8e40 2024-08-19T01:35:04.178 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:04.176+0000 7f5d32238740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f5d281a8e50 con 0x55a0278d3470 2024-08-19T01:35:04.178 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:04.176+0000 7f5d27e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a0278d3db0 0x55a0278e4a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:35:04.178 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:04.176+0000 7f5d27e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a0278d3db0 0x55a0278e4a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:34784/0 (socket says 172.21.15.83:34784) 2024-08-19T01:35:04.178 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:04.176+0000 7f5d27e00640 1 -- 172.21.15.83:0/3760765764 learned_addr learned my addr 172.21.15.83:0/3760765764 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-08-19T01:35:04.178 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:04.176+0000 7f5d2ca00640 1 --2- 172.21.15.83:0/3760765764 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a0278d3470 0x55a0278d4f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:35:04.178 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:04.176+0000 7f5d2d400640 1 --2- 172.21.15.83:0/3760765764 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a0278d8e40 0x55a0278d54d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:35:04.179 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:04.176+0000 7f5d27e00640 1 -- 172.21.15.83:0/3760765764 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a0278d3470 msgr2=0x55a0278d4f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:35:04.179 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:04.176+0000 7f5d27e00640 1 --2- 172.21.15.83:0/3760765764 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a0278d3470 0x55a0278d4f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:35:04.179 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:04.176+0000 7f5d27e00640 1 -- 172.21.15.83:0/3760765764 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a0278d8e40 msgr2=0x55a0278d54d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:35:04.179 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:04.176+0000 7f5d27e00640 1 --2- 172.21.15.83:0/3760765764 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a0278d8e40 0x55a0278d54d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:35:04.179 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:04.176+0000 7f5d27e00640 1 -- 172.21.15.83:0/3760765764 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a0278ce900 con 0x55a0278d3db0 2024-08-19T01:35:04.179 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:04.176+0000 7f5d2d400640 1 --2- 172.21.15.83:0/3760765764 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a0278d8e40 0x55a0278d54d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-19T01:35:04.179 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:04.176+0000 7f5d27e00640 1 --2- 172.21.15.83:0/3760765764 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a0278d3db0 0x55a0278e4a90 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f5d14003b00 tx=0x7f5d14005c50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cc49e4dfdd7e2aca server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:35:04.182 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:04.177+0000 7f5d19200640 1 -- 172.21.15.83:0/3760765764 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d14039020 con 0x55a0278d3db0 2024-08-19T01:35:04.182 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:04.177+0000 7f5d19200640 1 -- 172.21.15.83:0/3760765764 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5d1400e6a0 con 0x55a0278d3db0 2024-08-19T01:35:04.183 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:04.177+0000 7f5d19200640 1 -- 172.21.15.83:0/3760765764 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d14020b20 con 0x55a0278d3db0 2024-08-19T01:35:04.183 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:04.177+0000 7f5d32238740 1 -- 172.21.15.83:0/3760765764 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a0278d3db0 msgr2=0x55a0278e4a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:35:04.183 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:04.177+0000 7f5d32238740 1 --2- 172.21.15.83:0/3760765764 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a0278d3db0 0x55a0278e4a90 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f5d14003b00 tx=0x7f5d14005c50 comp rx=0 tx=0).stop 2024-08-19T01:35:04.183 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:04.177+0000 7f5d32238740 1 -- 172.21.15.83:0/3760765764 shutdown_connections 2024-08-19T01:35:04.183 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:04.177+0000 7f5d32238740 1 --2- 172.21.15.83:0/3760765764 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a0278d8e40 0x55a0278d54d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:35:04.183 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:04.177+0000 7f5d32238740 1 --2- 172.21.15.83:0/3760765764 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a0278d3db0 0x55a0278e4a90 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:35:04.183 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:04.177+0000 7f5d32238740 1 --2- 172.21.15.83:0/3760765764 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a0278d3470 0x55a0278d4f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:35:04.183 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:04.177+0000 7f5d32238740 1 -- 172.21.15.83:0/3760765764 >> 172.21.15.83:0/3760765764 conn(0x55a0278ceae0 msgr2=0x55a0278d8600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:35:04.183 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:04.177+0000 7f5d32238740 1 -- 172.21.15.83:0/3760765764 shutdown_connections 2024-08-19T01:35:04.183 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:04.177+0000 7f5d32238740 1 -- 172.21.15.83:0/3760765764 wait complete. 2024-08-19T01:35:04.183 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:04.178+0000 7f5d32238740 1 -- start start 2024-08-19T01:35:04.183 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:04.178+0000 7f5d32238740 1 Processor -- start 2024-08-19T01:35:04.183 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:04.179+0000 7f5d32238740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a0278d3470 0x55a0278e1cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:35:04.184 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:04.179+0000 7f5d32238740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a0278d3db0 0x55a0278e2210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:35:04.184 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:04.179+0000 7f5d32238740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a0278d8e40 0x55a0278e31d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:35:04.184 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:04.179+0000 7f5d32238740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f5d281a94c0 con 0x55a0278d3db0 2024-08-19T01:35:04.184 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:04.179+0000 7f5d32238740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f5d281aa930 con 0x55a0278d8e40 2024-08-19T01:35:04.184 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:04.179+0000 7f5d32238740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f5d281a9b90 con 0x55a0278d3470 2024-08-19T01:35:04.184 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:04.179+0000 7f5d32238740 10 client.0 _ll_register_callbacks cb 0x7f5d1400e5f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-19T01:35:04.184 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:04.179+0000 7f5d2ca00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a0278d3470 0x55a0278e1cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:35:04.184 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:04.179+0000 7f5d27e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a0278d3db0 0x55a0278e2210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:35:04.184 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:04.179+0000 7f5d2ca00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a0278d3470 0x55a0278e1cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.83:55386/0 (socket says 172.21.15.83:55386) 2024-08-19T01:35:04.184 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:04.179+0000 7f5d2ca00640 1 -- 172.21.15.83:0/531156702 learned_addr learned my addr 172.21.15.83:0/531156702 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-08-19T01:35:04.184 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:04.179+0000 7f5d2d400640 1 --2- 172.21.15.83:0/531156702 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a0278d8e40 0x55a0278e31d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:35:04.184 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:04.179+0000 7f5d2ca00640 1 -- 172.21.15.83:0/531156702 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a0278d8e40 msgr2=0x55a0278e31d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:35:04.184 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:04.179+0000 7f5d2ca00640 1 --2- 172.21.15.83:0/531156702 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a0278d8e40 0x55a0278e31d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:35:04.185 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:04.179+0000 7f5d2ca00640 1 -- 172.21.15.83:0/531156702 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a0278d3db0 msgr2=0x55a0278e2210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:35:04.185 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:04.179+0000 7f5d2ca00640 1 --2- 172.21.15.83:0/531156702 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a0278d3db0 0x55a0278e2210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:35:04.185 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:04.179+0000 7f5d2ca00640 1 -- 172.21.15.83:0/531156702 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a0278e37e0 con 0x55a0278d3470 2024-08-19T01:35:04.185 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:04.179+0000 7f5d27e00640 1 --2- 172.21.15.83:0/531156702 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a0278d3db0 0x55a0278e2210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-19T01:35:04.185 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:04.180+0000 7f5d2ca00640 1 --2- 172.21.15.83:0/531156702 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a0278d3470 0x55a0278e1cd0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f5d1c006010 tx=0x7f5d1c02f310 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c7235bfd284e6ae3 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:35:04.185 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:04.180+0000 7f5d0e000640 10 client.0 ms_handle_connect on v2:172.21.15.167:3300/0 2024-08-19T01:35:04.185 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:04.180+0000 7f5d0e000640 1 -- 172.21.15.83:0/531156702 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d1c04a040 con 0x55a0278d3470 2024-08-19T01:35:04.185 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:04.180+0000 7f5d32238740 1 -- 172.21.15.83:0/531156702 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55a02783f310 con 0x55a0278d3470 2024-08-19T01:35:04.185 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:04.180+0000 7f5d0e000640 1 -- 172.21.15.83:0/531156702 <== mon.1 v2:172.21.15.167:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8269+0+0 (secure 0 0 0) 0x7f5d1c0047a0 con 0x55a0278d3470 2024-08-19T01:35:04.185 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:04.180+0000 7f5d0e000640 1 -- 172.21.15.83:0/531156702 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f5d1c031450 con 0x55a0278d3470 2024-08-19T01:35:04.185 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:04.180+0000 7f5d32238740 10 client.25156 fetch_fsmap learned FSMap version 25 2024-08-19T01:35:04.185 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:04.180+0000 7f5d32238740 1 -- 172.21.15.83:0/531156702 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55a0278e4ac0 con 0x55a0278d3470 2024-08-19T01:35:04.185 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:04.180+0000 7f5d0e000640 1 -- 172.21.15.83:0/531156702 <== mon.1 v2:172.21.15.167:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x55a02783f310 con 0x55a0278d3470 2024-08-19T01:35:04.186 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:04.181+0000 7f5d0e000640 1 -- 172.21.15.83:0/531156702 <== mon.1 v2:172.21.15.167:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f5d1c043070 con 0x55a0278d3470 2024-08-19T01:35:04.186 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:04.181+0000 7f5d32238740 10 client.25156 fetch_fsmap finished waiting for FSMap version 25 2024-08-19T01:35:04.186 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:04.181+0000 7f5d32238740 10 client.25156 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-08-19T01:35:04.186 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:04.181+0000 7f5d32238740 20 client.25156 populate_metadata read hostname 'smithi083' 2024-08-19T01:35:04.186 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:04.181+0000 7f5d32238740 1 --2- 172.21.15.83:0/531156702 >> [v2:172.21.15.117:6838/1967595704,v1:172.21.15.117:6839/1967595704] conn(0x55a02795cbb0 0x55a02795efd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:35:04.186 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:04.181+0000 7f5d32238740 4 client.25156 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-08-19T01:35:04.186 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:04.181+0000 7f5d32238740 1 -- 172.21.15.83:0/531156702 --> [v2:172.21.15.117:6838/1967595704,v1:172.21.15.117:6839/1967595704] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55a0277bc7c0 con 0x55a02795cbb0 2024-08-19T01:35:04.186 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:04.181+0000 7f5d27e00640 1 --2- 172.21.15.83:0/531156702 >> [v2:172.21.15.117:6838/1967595704,v1:172.21.15.117:6839/1967595704] conn(0x55a02795cbb0 0x55a02795efd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:35:04.187 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:04.181+0000 7f5d0e000640 10 client.25156 ms_handle_connect on v2:172.21.15.117:6838/1967595704 2024-08-19T01:35:04.187 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:04.181+0000 7f5d27e00640 1 --2- 172.21.15.83:0/531156702 >> [v2:172.21.15.117:6838/1967595704,v1:172.21.15.117:6839/1967595704] conn(0x55a02795cbb0 0x55a02795efd0 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9cbeef9983cf3b32 server_cookie=d637350aac1d6267 in_seq=0 out_seq=0 2024-08-19T01:35:04.194 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:04.192+0000 7f5d0e000640 1 -- 172.21.15.83:0/531156702 <== mds.0 v2:172.21.15.117:6838/1967595704 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55a0277bc7c0 con 0x55a02795cbb0 2024-08-19T01:35:04.194 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:04.192+0000 7f5d0e000640 10 client.25156 handle_command_reply: tid=0 2024-08-19T01:35:04.194 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:04.192+0000 7f5d32238740 1 client.25156 shutdown 2024-08-19T01:35:04.194 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:04.192+0000 7f5d32238740 1 -- 172.21.15.83:0/531156702 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a0278d3470 msgr2=0x55a0278e1cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:35:04.195 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:04.192+0000 7f5d32238740 1 --2- 172.21.15.83:0/531156702 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a0278d3470 0x55a0278e1cd0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f5d1c006010 tx=0x7f5d1c02f310 comp rx=0 tx=0).stop 2024-08-19T01:35:04.195 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:04.192+0000 7f5d32238740 1 -- 172.21.15.83:0/531156702 shutdown_connections 2024-08-19T01:35:04.195 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:04.192+0000 7f5d32238740 1 --2- 172.21.15.83:0/531156702 >> [v2:172.21.15.117:6838/1967595704,v1:172.21.15.117:6839/1967595704] conn(0x55a02795cbb0 0x55a02795efd0 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:35:04.195 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:04.192+0000 7f5d32238740 1 --2- 172.21.15.83:0/531156702 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a0278d8e40 0x55a0278e31d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:35:04.195 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:04.192+0000 7f5d0e000640 0 client.25156 ms_handle_reset on v2:172.21.15.117:6838/1967595704 2024-08-19T01:35:04.195 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:04.192+0000 7f5d32238740 1 --2- 172.21.15.83:0/531156702 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a0278d3db0 0x55a0278e2210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:35:04.195 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:04.192+0000 7f5d32238740 1 --2- 172.21.15.83:0/531156702 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a0278d3470 0x55a0278e1cd0 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:35:04.195 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:04.192+0000 7f5d32238740 1 -- 172.21.15.83:0/531156702 >> 172.21.15.83:0/531156702 conn(0x55a0278ceae0 msgr2=0x55a0278d81d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:35:04.195 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:04.193+0000 7f5d32238740 1 -- 172.21.15.83:0/531156702 shutdown_connections 2024-08-19T01:35:04.195 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:04.193+0000 7f5d32238740 1 -- 172.21.15.83:0/531156702 wait complete. 2024-08-19T01:35:04.195 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:04.193+0000 7f5d32238740 20 client.25156 trim_cache size 0 max 16384 2024-08-19T01:35:04.196 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stdout:{ 2024-08-19T01:35:04.196 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stdout: "message": "", 2024-08-19T01:35:04.196 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stdout: "return_code": 0 2024-08-19T01:35:04.196 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stdout:} 2024-08-19T01:35:04.196 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:04.193+0000 7f5d2f400640 1 -- 172.21.15.83:0/320926717 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f5d100a2d50 msgr2=0x7f5d100a5210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:35:04.197 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:04.193+0000 7f5d2f400640 1 --2- 172.21.15.83:0/320926717 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f5d100a2d50 0x7f5d100a5210 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f5d14005850 tx=0x7f5d1400b7e0 comp rx=0 tx=0).stop 2024-08-19T01:35:04.197 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:04.193+0000 7f5d2f400640 1 -- 172.21.15.83:0/320926717 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5d28105640 msgr2=0x7f5d281a1260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:35:04.197 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:04.193+0000 7f5d2f400640 1 --2- 172.21.15.83:0/320926717 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5d28105640 0x7f5d281a1260 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f5d2000e970 tx=0x7f5d2000ed70 comp rx=0 tx=0).stop 2024-08-19T01:35:04.197 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:04.193+0000 7f5d2f400640 1 -- 172.21.15.83:0/320926717 shutdown_connections 2024-08-19T01:35:04.197 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:04.193+0000 7f5d2f400640 1 --2- 172.21.15.83:0/320926717 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f5d100a2d50 0x7f5d100a5210 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:35:04.197 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:04.193+0000 7f5d2f400640 1 --2- 172.21.15.83:0/320926717 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5d28105640 0x7f5d281a1260 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:35:04.197 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:04.193+0000 7f5d2f400640 1 --2- 172.21.15.83:0/320926717 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5d28104c80 0x7f5d2819a230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:35:04.197 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:04.193+0000 7f5d2f400640 1 --2- 172.21.15.83:0/320926717 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5d28103a90 0x7f5d28199cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:35:04.197 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:04.193+0000 7f5d2f400640 1 -- 172.21.15.83:0/320926717 >> 172.21.15.83:0/320926717 conn(0x7f5d280ff210 msgr2=0x7f5d28107f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:35:04.197 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:04.194+0000 7f5d2f400640 1 -- 172.21.15.83:0/320926717 shutdown_connections 2024-08-19T01:35:04.198 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:04.194+0000 7f5d2f400640 1 -- 172.21.15.83:0/320926717 wait complete. 2024-08-19T01:35:05.716 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:35:05 smithi167 ceph-mon[21875]: pgmap v346: 97 pgs: 97 active+clean; 656 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 853 B/s wr, 6 op/s 2024-08-19T01:35:05.716 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:35:05 smithi167 ceph-mon[21875]: scrub summary: idle 2024-08-19T01:35:05.904 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:35:05 smithi083 ceph-mon[17120]: pgmap v346: 97 pgs: 97 active+clean; 656 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 853 B/s wr, 6 op/s 2024-08-19T01:35:05.905 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:35:05 smithi083 ceph-mon[17120]: scrub summary: idle 2024-08-19T01:35:05.952 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:35:05 smithi117 ceph-mon[22016]: pgmap v346: 97 pgs: 97 active+clean; 656 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 853 B/s wr, 6 op/s 2024-08-19T01:35:05.952 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:35:05 smithi117 ceph-mon[22016]: scrub summary: idle 2024-08-19T01:35:07.716 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:35:07 smithi167 ceph-mon[21875]: pgmap v347: 97 pgs: 97 active+clean; 657 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 1.3 KiB/s wr, 8 op/s 2024-08-19T01:35:07.904 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:35:07 smithi083 ceph-mon[17120]: pgmap v347: 97 pgs: 97 active+clean; 657 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 1.3 KiB/s wr, 8 op/s 2024-08-19T01:35:07.952 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:35:07 smithi117 ceph-mon[22016]: pgmap v347: 97 pgs: 97 active+clean; 657 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 1.3 KiB/s wr, 8 op/s 2024-08-19T01:35:09.716 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:35:09 smithi167 ceph-mon[21875]: pgmap v348: 97 pgs: 97 active+clean; 657 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 1.2 KiB/s wr, 9 op/s 2024-08-19T01:35:09.904 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:35:09 smithi083 ceph-mon[17120]: pgmap v348: 97 pgs: 97 active+clean; 657 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 1.2 KiB/s wr, 9 op/s 2024-08-19T01:35:09.952 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:35:09 smithi117 ceph-mon[22016]: pgmap v348: 97 pgs: 97 active+clean; 657 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 1.2 KiB/s wr, 9 op/s 2024-08-19T01:35:11.904 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:35:11 smithi083 ceph-mon[17120]: pgmap v349: 97 pgs: 97 active+clean; 657 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 1.5 KiB/s wr, 10 op/s 2024-08-19T01:35:11.952 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:35:11 smithi117 ceph-mon[22016]: pgmap v349: 97 pgs: 97 active+clean; 657 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 1.5 KiB/s wr, 10 op/s 2024-08-19T01:35:11.966 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:35:11 smithi167 ceph-mon[21875]: pgmap v349: 97 pgs: 97 active+clean; 657 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 1.5 KiB/s wr, 10 op/s 2024-08-19T01:35:13.904 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:35:13 smithi083 ceph-mon[17120]: pgmap v350: 97 pgs: 97 active+clean; 657 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 1.5 KiB/s wr, 9 op/s 2024-08-19T01:35:13.952 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:35:13 smithi117 ceph-mon[22016]: pgmap v350: 97 pgs: 97 active+clean; 657 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 1.5 KiB/s wr, 9 op/s 2024-08-19T01:35:13.966 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:35:13 smithi167 ceph-mon[21875]: pgmap v350: 97 pgs: 97 active+clean; 657 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 1.5 KiB/s wr, 9 op/s 2024-08-19T01:35:15.724 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:35:15 smithi083 ceph-mon[17120]: pgmap v351: 97 pgs: 97 active+clean; 657 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 1.5 KiB/s wr, 10 op/s 2024-08-19T01:35:15.952 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:35:15 smithi117 ceph-mon[22016]: pgmap v351: 97 pgs: 97 active+clean; 657 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 1.5 KiB/s wr, 10 op/s 2024-08-19T01:35:15.966 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:35:15 smithi167 ceph-mon[21875]: pgmap v351: 97 pgs: 97 active+clean; 657 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 1.5 KiB/s wr, 10 op/s 2024-08-19T01:35:17.321 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:17.319+0000 7f286e000640 1 -- 172.21.15.83:0/2666685265 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2868103a90 msgr2=0x7f2868103e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:35:17.321 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:17.319+0000 7f286e000640 1 --2- 172.21.15.83:0/2666685265 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2868103a90 0x7f2868103e90 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f2858009900 tx=0x7f285802a3f0 comp rx=0 tx=0).stop 2024-08-19T01:35:17.321 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:17.319+0000 7f286e000640 1 -- 172.21.15.83:0/2666685265 shutdown_connections 2024-08-19T01:35:17.321 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:17.319+0000 7f286e000640 1 --2- 172.21.15.83:0/2666685265 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2868105640 0x7f286810bed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:35:17.322 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:17.319+0000 7f286e000640 1 --2- 172.21.15.83:0/2666685265 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2868104c80 0x7f2868105100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:35:17.322 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:17.319+0000 7f286e000640 1 --2- 172.21.15.83:0/2666685265 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2868103a90 0x7f2868103e90 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:35:17.322 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:17.319+0000 7f286e000640 1 -- 172.21.15.83:0/2666685265 >> 172.21.15.83:0/2666685265 conn(0x7f28680ff210 msgr2=0x7f2868101630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:35:17.322 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:17.320+0000 7f286e000640 1 -- 172.21.15.83:0/2666685265 shutdown_connections 2024-08-19T01:35:17.322 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:17.320+0000 7f286e000640 1 -- 172.21.15.83:0/2666685265 wait complete. 2024-08-19T01:35:17.324 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:17.320+0000 7f286e000640 1 Processor -- start 2024-08-19T01:35:17.324 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:17.321+0000 7f286e000640 1 -- start start 2024-08-19T01:35:17.324 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:17.321+0000 7f286e000640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2868103a90 0x7f2868199d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:35:17.324 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:17.321+0000 7f286e000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2868104c80 0x7f286819a290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:35:17.324 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:17.321+0000 7f286e000640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2868105640 0x7f28681a12c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:35:17.324 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:17.321+0000 7f286e000640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f286810f740 con 0x7f2868104c80 2024-08-19T01:35:17.325 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:17.321+0000 7f286e000640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f286810f5c0 con 0x7f2868105640 2024-08-19T01:35:17.325 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:17.321+0000 7f286e000640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f286810f8c0 con 0x7f2868103a90 2024-08-19T01:35:17.325 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:17.321+0000 7f2866a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2868104c80 0x7f286819a290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:35:17.325 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:17.321+0000 7f2867400640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2868103a90 0x7f2868199d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:35:17.325 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:17.321+0000 7f2866a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2868104c80 0x7f286819a290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:56888/0 (socket says 172.21.15.83:56888) 2024-08-19T01:35:17.325 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:17.321+0000 7f2866a00640 1 -- 172.21.15.83:0/637844192 learned_addr learned my addr 172.21.15.83:0/637844192 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-08-19T01:35:17.325 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:17.321+0000 7f2867e00640 1 --2- 172.21.15.83:0/637844192 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2868105640 0x7f28681a12c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:35:17.325 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:17.322+0000 7f2867400640 1 -- 172.21.15.83:0/637844192 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2868105640 msgr2=0x7f28681a12c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:35:17.325 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:17.322+0000 7f2867400640 1 --2- 172.21.15.83:0/637844192 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2868105640 0x7f28681a12c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:35:17.325 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:17.322+0000 7f2867400640 1 -- 172.21.15.83:0/637844192 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2868104c80 msgr2=0x7f286819a290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:35:17.325 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:17.322+0000 7f2867400640 1 --2- 172.21.15.83:0/637844192 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2868104c80 0x7f286819a290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:35:17.325 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:17.322+0000 7f2867400640 1 -- 172.21.15.83:0/637844192 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f28681a1960 con 0x7f2868103a90 2024-08-19T01:35:17.325 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:17.322+0000 7f2866a00640 1 --2- 172.21.15.83:0/637844192 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2868104c80 0x7f286819a290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-19T01:35:17.326 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:17.322+0000 7f2867400640 1 --2- 172.21.15.83:0/637844192 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2868103a90 0x7f2868199d50 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f2858009230 tx=0x7f2858032ce0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:35:17.326 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:17.322+0000 7f285fe00640 1 -- 172.21.15.83:0/637844192 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f285803f030 con 0x7f2868103a90 2024-08-19T01:35:17.326 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:17.322+0000 7f285fe00640 1 -- 172.21.15.83:0/637844192 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2858003ce0 con 0x7f2868103a90 2024-08-19T01:35:17.326 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:17.322+0000 7f285fe00640 1 -- 172.21.15.83:0/637844192 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2858035070 con 0x7f2868103a90 2024-08-19T01:35:17.326 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:17.322+0000 7f286e000640 1 -- 172.21.15.83:0/637844192 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f28681a1bf0 con 0x7f2868103a90 2024-08-19T01:35:17.326 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:17.323+0000 7f286e000640 1 -- 172.21.15.83:0/637844192 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f28681a20b0 con 0x7f2868103a90 2024-08-19T01:35:17.327 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:17.326+0000 7f285fe00640 1 -- 172.21.15.83:0/637844192 <== mon.1 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 19) ==== 137884+0+0 (secure 0 0 0) 0x7f2858042030 con 0x7f2868103a90 2024-08-19T01:35:17.327 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:17.326+0000 7f2870e25740 1 Processor -- start 2024-08-19T01:35:17.328 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:17.326+0000 7f2870e25740 1 -- start start 2024-08-19T01:35:17.328 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:17.327+0000 7f2870e25740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a269a80470 0x55a269a80870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:35:17.328 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:17.327+0000 7f2870e25740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a269a80db0 0x55a269a85870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:35:17.329 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:17.327+0000 7f2870e25740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a269a85e40 0x55a269a88300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:35:17.329 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:17.327+0000 7f2870e25740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55a2699e4890 con 0x55a269a85e40 2024-08-19T01:35:17.329 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:17.327+0000 7f2870e25740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55a26997cd90 con 0x55a269a80db0 2024-08-19T01:35:17.329 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:17.327+0000 7f2870e25740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55a2699e0850 con 0x55a269a80470 2024-08-19T01:35:17.329 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:17.327+0000 7f2867400640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a269a80db0 0x55a269a85870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:35:17.329 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:17.327+0000 7f2867e00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a269a80470 0x55a269a80870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:35:17.329 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:17.327+0000 7f2867e00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a269a80470 0x55a269a80870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.83:38010/0 (socket says 172.21.15.83:38010) 2024-08-19T01:35:17.329 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:17.327+0000 7f2867e00640 1 -- 172.21.15.83:0/3625086964 learned_addr learned my addr 172.21.15.83:0/3625086964 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-08-19T01:35:17.329 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:17.327+0000 7f2866a00640 1 --2- 172.21.15.83:0/3625086964 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a269a85e40 0x55a269a88300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:35:17.331 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:17.327+0000 7f2867e00640 1 -- 172.21.15.83:0/3625086964 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a269a80db0 msgr2=0x55a269a85870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:35:17.331 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:17.327+0000 7f2867e00640 1 --2- 172.21.15.83:0/3625086964 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a269a80db0 0x55a269a85870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:35:17.331 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:17.327+0000 7f2867e00640 1 -- 172.21.15.83:0/3625086964 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a269a85e40 msgr2=0x55a269a88300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:35:17.331 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:17.327+0000 7f2867e00640 1 --2- 172.21.15.83:0/3625086964 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a269a85e40 0x55a269a88300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:35:17.331 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:17.327+0000 7f2867e00640 1 -- 172.21.15.83:0/3625086964 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a2696f8210 con 0x55a269a80470 2024-08-19T01:35:17.331 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:17.327+0000 7f2866a00640 1 --2- 172.21.15.83:0/3625086964 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a269a85e40 0x55a269a88300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-19T01:35:17.332 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:17.327+0000 7f285fe00640 1 --2- 172.21.15.83:0/637844192 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f284c0a2cd0 0x7f284c0a5190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:35:17.332 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:17.328+0000 7f285fe00640 1 -- 172.21.15.83:0/637844192 <== mon.1 v2:172.21.15.167:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8269+0+0 (secure 0 0 0) 0x7f28580ef700 con 0x7f2868103a90 2024-08-19T01:35:17.332 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:17.328+0000 7f2867e00640 1 --2- 172.21.15.83:0/3625086964 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a269a80470 0x55a269a80870 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f2860009fd0 tx=0x7f28600118d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d70756c7698a9939 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:35:17.332 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:17.328+0000 7f2867400640 1 --2- 172.21.15.83:0/637844192 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f284c0a2cd0 0x7f284c0a5190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:35:17.332 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:17.328+0000 7f2853e00640 1 -- 172.21.15.83:0/3625086964 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2860004d20 con 0x55a269a80470 2024-08-19T01:35:17.332 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:17.328+0000 7f2853e00640 1 -- 172.21.15.83:0/3625086964 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2860004050 con 0x55a269a80470 2024-08-19T01:35:17.332 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:17.328+0000 7f2853e00640 1 -- 172.21.15.83:0/3625086964 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2860005280 con 0x55a269a80470 2024-08-19T01:35:17.332 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:17.329+0000 7f2870e25740 1 -- 172.21.15.83:0/3625086964 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a269a80470 msgr2=0x55a269a80870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:35:17.332 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:17.329+0000 7f2867400640 1 --2- 172.21.15.83:0/637844192 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f284c0a2cd0 0x7f284c0a5190 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f2858009900 tx=0x7f28580f5530 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:35:17.332 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:17.329+0000 7f2870e25740 1 --2- 172.21.15.83:0/3625086964 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a269a80470 0x55a269a80870 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f2860009fd0 tx=0x7f28600118d0 comp rx=0 tx=0).stop 2024-08-19T01:35:17.332 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:17.329+0000 7f2870e25740 1 -- 172.21.15.83:0/3625086964 shutdown_connections 2024-08-19T01:35:17.332 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:17.329+0000 7f2870e25740 1 --2- 172.21.15.83:0/3625086964 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a269a85e40 0x55a269a88300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:35:17.332 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:17.329+0000 7f2870e25740 1 --2- 172.21.15.83:0/3625086964 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a269a80db0 0x55a269a85870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:35:17.333 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:17.329+0000 7f2870e25740 1 --2- 172.21.15.83:0/3625086964 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a269a80470 0x55a269a80870 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:35:17.333 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:17.329+0000 7f2870e25740 1 -- 172.21.15.83:0/3625086964 >> 172.21.15.83:0/3625086964 conn(0x55a269a7bae0 msgr2=0x55a269a7bee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:35:17.333 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:17.329+0000 7f2870e25740 1 -- 172.21.15.83:0/3625086964 shutdown_connections 2024-08-19T01:35:17.333 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:17.329+0000 7f2870e25740 1 -- 172.21.15.83:0/3625086964 wait complete. 2024-08-19T01:35:17.333 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:17.330+0000 7f2870e25740 1 -- start start 2024-08-19T01:35:17.333 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:17.330+0000 7f2870e25740 1 Processor -- start 2024-08-19T01:35:17.335 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:17.330+0000 7f2870e25740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a269a80470 0x55a269a8a7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:35:17.336 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:17.330+0000 7f2870e25740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a269a80db0 0x55a269a8ad00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:35:17.336 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:17.330+0000 7f2870e25740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a269a85e40 0x55a269a91d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:35:17.336 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:17.330+0000 7f2870e25740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55a269a88eb0 con 0x55a269a80db0 2024-08-19T01:35:17.336 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:17.330+0000 7f2870e25740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55a26999ff70 con 0x55a269a85e40 2024-08-19T01:35:17.336 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:17.330+0000 7f2870e25740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55a269a89030 con 0x55a269a80470 2024-08-19T01:35:17.336 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:17.330+0000 7f2870e25740 10 client.0 _ll_register_callbacks cb 0x55a2699dc8d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-19T01:35:17.336 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:17.331+0000 7f2867e00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a269a80470 0x55a269a8a7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:35:17.336 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:17.331+0000 7f2867e00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a269a80470 0x55a269a8a7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.83:38018/0 (socket says 172.21.15.83:38018) 2024-08-19T01:35:17.336 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:17.331+0000 7f2867e00640 1 -- 172.21.15.83:0/228870809 learned_addr learned my addr 172.21.15.83:0/228870809 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-08-19T01:35:17.336 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:17.331+0000 7f2867e00640 1 --2- 172.21.15.83:0/228870809 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a269a85e40 0x55a269a91d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:35:17.336 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:17.331+0000 7f2866a00640 1 --2- 172.21.15.83:0/228870809 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a269a80db0 0x55a269a8ad00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:35:17.336 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:17.331+0000 7f2867e00640 1 -- 172.21.15.83:0/228870809 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a269a85e40 msgr2=0x55a269a91d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:35:17.337 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:17.331+0000 7f2867e00640 1 --2- 172.21.15.83:0/228870809 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a269a85e40 0x55a269a91d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:35:17.337 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:17.331+0000 7f2867e00640 1 -- 172.21.15.83:0/228870809 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a269a80db0 msgr2=0x55a269a8ad00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:35:17.337 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:17.331+0000 7f2867e00640 1 --2- 172.21.15.83:0/228870809 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a269a80db0 0x55a269a8ad00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:35:17.337 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:17.331+0000 7f2867e00640 1 -- 172.21.15.83:0/228870809 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a2697c4620 con 0x55a269a80470 2024-08-19T01:35:17.337 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:17.331+0000 7f2867e00640 1 --2- 172.21.15.83:0/228870809 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a269a80470 0x55a269a8a7c0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f2860004d50 tx=0x7f2860000f40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=90a38abd7f46eff server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:35:17.337 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:17.331+0000 7f2850c00640 10 client.0 ms_handle_connect on v2:172.21.15.167:3300/0 2024-08-19T01:35:17.337 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:17.332+0000 7f2850c00640 1 -- 172.21.15.83:0/228870809 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f286000f4e0 con 0x55a269a80470 2024-08-19T01:35:17.337 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:17.332+0000 7f2850c00640 1 -- 172.21.15.83:0/228870809 <== mon.1 v2:172.21.15.167:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8269+0+0 (secure 0 0 0) 0x7f286000f680 con 0x55a269a80470 2024-08-19T01:35:17.337 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:17.332+0000 7f2870e25740 1 -- 172.21.15.83:0/228870809 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2860004050 con 0x55a269a80470 2024-08-19T01:35:17.337 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:17.332+0000 7f2850c00640 1 -- 172.21.15.83:0/228870809 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f286000f880 con 0x55a269a80470 2024-08-19T01:35:17.337 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:17.332+0000 7f2870e25740 10 client.25168 fetch_fsmap learned FSMap version 25 2024-08-19T01:35:17.337 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:17.332+0000 7f2870e25740 1 -- 172.21.15.83:0/228870809 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55a269a94ec0 con 0x55a269a80470 2024-08-19T01:35:17.337 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:17.332+0000 7f2850c00640 1 -- 172.21.15.83:0/228870809 <== mon.1 v2:172.21.15.167:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2860004050 con 0x55a269a80470 2024-08-19T01:35:17.338 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:17.333+0000 7f2850c00640 1 -- 172.21.15.83:0/228870809 <== mon.1 v2:172.21.15.167:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f2860025040 con 0x55a269a80470 2024-08-19T01:35:17.338 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:17.333+0000 7f2870e25740 10 client.25168 fetch_fsmap finished waiting for FSMap version 25 2024-08-19T01:35:17.338 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:17.333+0000 7f2870e25740 10 client.25168 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-08-19T01:35:17.338 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:17.333+0000 7f2870e25740 20 client.25168 populate_metadata read hostname 'smithi083' 2024-08-19T01:35:17.338 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:17.333+0000 7f2870e25740 1 --2- 172.21.15.83:0/228870809 >> [v2:172.21.15.117:6838/1967595704,v1:172.21.15.117:6839/1967595704] conn(0x55a269a963e0 0x55a269a98800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:35:17.338 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:17.333+0000 7f2870e25740 4 client.25168 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-19T01:35:17.338 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:17.333+0000 7f2870e25740 1 -- 172.21.15.83:0/228870809 --> [v2:172.21.15.117:6838/1967595704,v1:172.21.15.117:6839/1967595704] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a2699f1fc0 con 0x55a269a963e0 2024-08-19T01:35:17.338 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:17.333+0000 7f2866a00640 1 --2- 172.21.15.83:0/228870809 >> [v2:172.21.15.117:6838/1967595704,v1:172.21.15.117:6839/1967595704] conn(0x55a269a963e0 0x55a269a98800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:35:17.338 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:17.334+0000 7f2850c00640 10 client.25168 ms_handle_connect on v2:172.21.15.117:6838/1967595704 2024-08-19T01:35:17.338 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:17.334+0000 7f2866a00640 1 --2- 172.21.15.83:0/228870809 >> [v2:172.21.15.117:6838/1967595704,v1:172.21.15.117:6839/1967595704] conn(0x55a269a963e0 0x55a269a98800 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5cd47d6711999c3d server_cookie=bcbf868a9138d3fb in_seq=0 out_seq=0 2024-08-19T01:35:17.339 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:17.335+0000 7f2850c00640 1 -- 172.21.15.83:0/228870809 <== mds.0 v2:172.21.15.117:6838/1967595704 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55a2699f1fc0 con 0x55a269a963e0 2024-08-19T01:35:17.339 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:17.335+0000 7f2850c00640 10 client.25168 handle_command_reply: tid=0 2024-08-19T01:35:17.339 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:17.335+0000 7f2870e25740 1 client.25168 shutdown 2024-08-19T01:35:17.339 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:17.335+0000 7f2870e25740 1 -- 172.21.15.83:0/228870809 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a269a80470 msgr2=0x55a269a8a7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:35:17.340 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:17.335+0000 7f2870e25740 1 --2- 172.21.15.83:0/228870809 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a269a80470 0x55a269a8a7c0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f2860004d50 tx=0x7f2860000f40 comp rx=0 tx=0).stop 2024-08-19T01:35:17.340 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:17.335+0000 7f2870e25740 1 -- 172.21.15.83:0/228870809 shutdown_connections 2024-08-19T01:35:17.340 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:17.335+0000 7f2870e25740 1 --2- 172.21.15.83:0/228870809 >> [v2:172.21.15.117:6838/1967595704,v1:172.21.15.117:6839/1967595704] conn(0x55a269a963e0 0x55a269a98800 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:35:17.340 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:17.335+0000 7f2870e25740 1 --2- 172.21.15.83:0/228870809 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a269a85e40 0x55a269a91d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:35:17.340 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:17.335+0000 7f2870e25740 1 --2- 172.21.15.83:0/228870809 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a269a80db0 0x55a269a8ad00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:35:17.340 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:17.335+0000 7f2850c00640 0 client.25168 ms_handle_reset on v2:172.21.15.117:6838/1967595704 2024-08-19T01:35:17.340 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:17.335+0000 7f2870e25740 1 --2- 172.21.15.83:0/228870809 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a269a80470 0x55a269a8a7c0 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:35:17.340 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:17.335+0000 7f2870e25740 1 -- 172.21.15.83:0/228870809 >> 172.21.15.83:0/228870809 conn(0x55a269a7bae0 msgr2=0x55a269a7d2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:35:17.340 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:17.335+0000 7f2870e25740 1 -- 172.21.15.83:0/228870809 shutdown_connections 2024-08-19T01:35:17.340 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:17.335+0000 7f2870e25740 1 -- 172.21.15.83:0/228870809 wait complete. 2024-08-19T01:35:17.340 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:17.336+0000 7f2870e25740 20 client.25168 trim_cache size 0 max 16384 2024-08-19T01:35:17.359 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:17.356+0000 7f2870e25740 1 Processor -- start 2024-08-19T01:35:17.359 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:17.356+0000 7f2870e25740 1 -- start start 2024-08-19T01:35:17.359 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:17.357+0000 7f2870e25740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a269a80470 0x55a269a8ad90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:35:17.359 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:17.357+0000 7f2870e25740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a269a80db0 0x55a269a91d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:35:17.359 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:17.357+0000 7f2870e25740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a269a85e40 0x55a269a7abe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:35:17.359 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:17.357+0000 7f2870e25740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f28681ad800 con 0x55a269a80470 2024-08-19T01:35:17.360 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:17.357+0000 7f2870e25740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f28681ab0d0 con 0x55a269a80db0 2024-08-19T01:35:17.360 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:17.357+0000 7f2870e25740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f28681a8f20 con 0x55a269a85e40 2024-08-19T01:35:17.360 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:17.357+0000 7f2867e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a269a80470 0x55a269a8ad90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:35:17.360 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:17.357+0000 7f2867e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a269a80470 0x55a269a8ad90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:56912/0 (socket says 172.21.15.83:56912) 2024-08-19T01:35:17.360 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:17.357+0000 7f2867e00640 1 -- 172.21.15.83:0/3430511283 learned_addr learned my addr 172.21.15.83:0/3430511283 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-08-19T01:35:17.360 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:17.357+0000 7f2866a00640 1 --2- 172.21.15.83:0/3430511283 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a269a80db0 0x55a269a91d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:35:17.360 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:17.357+0000 7f2867e00640 1 --2- 172.21.15.83:0/3430511283 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a269a85e40 0x55a269a7abe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:35:17.360 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:17.357+0000 7f2867e00640 1 -- 172.21.15.83:0/3430511283 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a269a85e40 msgr2=0x55a269a7abe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:35:17.360 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:17.357+0000 7f2867e00640 1 --2- 172.21.15.83:0/3430511283 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a269a85e40 0x55a269a7abe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:35:17.360 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:17.357+0000 7f2867e00640 1 -- 172.21.15.83:0/3430511283 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a269a80db0 msgr2=0x55a269a91d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:35:17.360 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:17.357+0000 7f2867e00640 1 --2- 172.21.15.83:0/3430511283 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a269a80db0 0x55a269a91d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:35:17.360 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:17.357+0000 7f2867e00640 1 -- 172.21.15.83:0/3430511283 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a269a81290 con 0x55a269a80470 2024-08-19T01:35:17.360 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:17.357+0000 7f2866a00640 1 --2- 172.21.15.83:0/3430511283 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a269a80db0 0x55a269a91d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-19T01:35:17.361 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:17.358+0000 7f2867e00640 1 --2- 172.21.15.83:0/3430511283 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a269a80470 0x55a269a8ad90 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f2860009de0 tx=0x7f2860014040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d42151b2de73a0d3 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:35:17.364 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:17.358+0000 7f2853400640 1 -- 172.21.15.83:0/3430511283 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2860003c40 con 0x55a269a80470 2024-08-19T01:35:17.364 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:17.358+0000 7f2853400640 1 -- 172.21.15.83:0/3430511283 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f286002d070 con 0x55a269a80470 2024-08-19T01:35:17.364 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:17.358+0000 7f2853400640 1 -- 172.21.15.83:0/3430511283 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2860028480 con 0x55a269a80470 2024-08-19T01:35:17.364 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:17.359+0000 7f2870e25740 1 -- 172.21.15.83:0/3430511283 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a269a80470 msgr2=0x55a269a8ad90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:35:17.365 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:17.359+0000 7f2870e25740 1 --2- 172.21.15.83:0/3430511283 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a269a80470 0x55a269a8ad90 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f2860009de0 tx=0x7f2860014040 comp rx=0 tx=0).stop 2024-08-19T01:35:17.365 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:17.359+0000 7f2870e25740 1 -- 172.21.15.83:0/3430511283 shutdown_connections 2024-08-19T01:35:17.365 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:17.359+0000 7f2870e25740 1 --2- 172.21.15.83:0/3430511283 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a269a85e40 0x55a269a7abe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:35:17.365 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:17.359+0000 7f2870e25740 1 --2- 172.21.15.83:0/3430511283 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a269a80db0 0x55a269a91d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:35:17.365 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:17.359+0000 7f2870e25740 1 --2- 172.21.15.83:0/3430511283 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a269a80470 0x55a269a8ad90 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:35:17.365 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:17.359+0000 7f2870e25740 1 -- 172.21.15.83:0/3430511283 >> 172.21.15.83:0/3430511283 conn(0x55a269a7bae0 msgr2=0x55a269a7a4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:35:17.365 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:17.359+0000 7f2870e25740 1 -- 172.21.15.83:0/3430511283 shutdown_connections 2024-08-19T01:35:17.365 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:17.359+0000 7f2870e25740 1 -- 172.21.15.83:0/3430511283 wait complete. 2024-08-19T01:35:17.365 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:17.359+0000 7f2870e25740 1 -- start start 2024-08-19T01:35:17.365 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:17.360+0000 7f2870e25740 1 Processor -- start 2024-08-19T01:35:17.365 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:17.360+0000 7f2870e25740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a269a80470 0x55a269a8f1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:35:17.365 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:17.360+0000 7f2870e25740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a269a80db0 0x55a269a8f730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:35:17.366 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:17.360+0000 7f2870e25740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a269a85e40 0x55a269a90740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:35:17.366 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:17.360+0000 7f2870e25740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f28681a9590 con 0x55a269a80470 2024-08-19T01:35:17.366 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:17.360+0000 7f2870e25740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f28681aaa00 con 0x55a269a85e40 2024-08-19T01:35:17.366 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:17.360+0000 7f2870e25740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f28681a9c60 con 0x55a269a80db0 2024-08-19T01:35:17.366 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:17.360+0000 7f2870e25740 10 client.0 _ll_register_callbacks cb 0x7f28580f5400 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-19T01:35:17.366 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:17.360+0000 7f2867e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a269a80470 0x55a269a8f1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:35:17.366 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:17.360+0000 7f2867e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a269a80470 0x55a269a8f1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:56926/0 (socket says 172.21.15.83:56926) 2024-08-19T01:35:17.366 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:17.360+0000 7f2867e00640 1 -- 172.21.15.83:0/2186344183 learned_addr learned my addr 172.21.15.83:0/2186344183 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-08-19T01:35:17.366 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:17.360+0000 7f2866a00640 1 --2- 172.21.15.83:0/2186344183 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a269a80db0 0x55a269a8f730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:35:17.366 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:17.361+0000 7f2867e00640 1 --2- 172.21.15.83:0/2186344183 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a269a85e40 0x55a269a90740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:35:17.366 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:17.361+0000 7f2867e00640 1 -- 172.21.15.83:0/2186344183 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a269a80db0 msgr2=0x55a269a8f730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:35:17.366 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:17.361+0000 7f2867e00640 1 --2- 172.21.15.83:0/2186344183 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a269a80db0 0x55a269a8f730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:35:17.367 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:17.361+0000 7f2867e00640 1 -- 172.21.15.83:0/2186344183 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a269a85e40 msgr2=0x55a269a90740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:35:17.367 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:17.361+0000 7f2867e00640 1 --2- 172.21.15.83:0/2186344183 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a269a85e40 0x55a269a90740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:35:17.367 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:17.361+0000 7f2867e00640 1 -- 172.21.15.83:0/2186344183 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a269a90d50 con 0x55a269a80470 2024-08-19T01:35:17.367 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:17.361+0000 7f2867e00640 1 --2- 172.21.15.83:0/2186344183 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a269a80470 0x55a269a8f1f0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f2860004a50 tx=0x7f28600047d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=de88839d2ed318c8 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:35:17.367 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:17.361+0000 7f2850c00640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-08-19T01:35:17.367 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:17.361+0000 7f2850c00640 1 -- 172.21.15.83:0/2186344183 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2860009b30 con 0x55a269a80470 2024-08-19T01:35:17.367 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:17.361+0000 7f2850c00640 1 -- 172.21.15.83:0/2186344183 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8269+0+0 (secure 0 0 0) 0x7f2860003f10 con 0x55a269a80470 2024-08-19T01:35:17.367 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:17.361+0000 7f2870e25740 1 -- 172.21.15.83:0/2186344183 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f286002d070 con 0x55a269a80470 2024-08-19T01:35:17.367 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:17.362+0000 7f2850c00640 1 -- 172.21.15.83:0/2186344183 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f2860028480 con 0x55a269a80470 2024-08-19T01:35:17.367 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:17.362+0000 7f2870e25740 10 client.15135 fetch_fsmap learned FSMap version 25 2024-08-19T01:35:17.367 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:17.362+0000 7f2870e25740 1 -- 172.21.15.83:0/2186344183 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55a269a92060 con 0x55a269a80470 2024-08-19T01:35:17.367 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:17.362+0000 7f2850c00640 1 -- 172.21.15.83:0/2186344183 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f286002d070 con 0x55a269a80470 2024-08-19T01:35:17.367 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:17.362+0000 7f2850c00640 1 -- 172.21.15.83:0/2186344183 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f2860019a30 con 0x55a269a80470 2024-08-19T01:35:17.368 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:17.362+0000 7f2870e25740 10 client.15135 fetch_fsmap finished waiting for FSMap version 25 2024-08-19T01:35:17.368 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:17.362+0000 7f2870e25740 10 client.15135 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-08-19T01:35:17.368 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:17.362+0000 7f2870e25740 20 client.15135 populate_metadata read hostname 'smithi083' 2024-08-19T01:35:17.368 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:17.362+0000 7f2870e25740 1 --2- 172.21.15.83:0/2186344183 >> [v2:172.21.15.117:6838/1967595704,v1:172.21.15.117:6839/1967595704] conn(0x55a269b01720 0x55a269b03b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:35:17.368 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:17.362+0000 7f2870e25740 4 client.15135 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-08-19T01:35:17.368 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:17.362+0000 7f2870e25740 1 -- 172.21.15.83:0/2186344183 --> [v2:172.21.15.117:6838/1967595704,v1:172.21.15.117:6839/1967595704] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55a2699ec310 con 0x55a269b01720 2024-08-19T01:35:17.368 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:17.362+0000 7f2866a00640 1 --2- 172.21.15.83:0/2186344183 >> [v2:172.21.15.117:6838/1967595704,v1:172.21.15.117:6839/1967595704] conn(0x55a269b01720 0x55a269b03b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:35:17.368 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:17.363+0000 7f2866a00640 1 --2- 172.21.15.83:0/2186344183 >> [v2:172.21.15.117:6838/1967595704,v1:172.21.15.117:6839/1967595704] conn(0x55a269b01720 0x55a269b03b40 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c9d6d30646187142 server_cookie=4b98a0d4746c8f11 in_seq=0 out_seq=0 2024-08-19T01:35:17.368 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:17.363+0000 7f2850c00640 10 client.15135 ms_handle_connect on v2:172.21.15.117:6838/1967595704 2024-08-19T01:35:17.369 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:17.367+0000 7f2850c00640 1 -- 172.21.15.83:0/2186344183 <== mds.0 v2:172.21.15.117:6838/1967595704 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55a2699ec310 con 0x55a269b01720 2024-08-19T01:35:17.369 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:17.367+0000 7f2850c00640 10 client.15135 handle_command_reply: tid=0 2024-08-19T01:35:17.370 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:17.367+0000 7f2870e25740 1 client.15135 shutdown 2024-08-19T01:35:17.370 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:17.367+0000 7f2870e25740 1 -- 172.21.15.83:0/2186344183 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a269a80470 msgr2=0x55a269a8f1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:35:17.370 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:17.367+0000 7f2870e25740 1 --2- 172.21.15.83:0/2186344183 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a269a80470 0x55a269a8f1f0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f2860004a50 tx=0x7f28600047d0 comp rx=0 tx=0).stop 2024-08-19T01:35:17.370 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:17.367+0000 7f2870e25740 1 -- 172.21.15.83:0/2186344183 shutdown_connections 2024-08-19T01:35:17.370 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:17.367+0000 7f2870e25740 1 --2- 172.21.15.83:0/2186344183 >> [v2:172.21.15.117:6838/1967595704,v1:172.21.15.117:6839/1967595704] conn(0x55a269b01720 0x55a269b03b40 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:35:17.370 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:17.367+0000 7f2870e25740 1 --2- 172.21.15.83:0/2186344183 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a269a85e40 0x55a269a90740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:35:17.370 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:17.367+0000 7f2870e25740 1 --2- 172.21.15.83:0/2186344183 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a269a80db0 0x55a269a8f730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:35:17.370 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:17.367+0000 7f2850c00640 0 client.15135 ms_handle_reset on v2:172.21.15.117:6838/1967595704 2024-08-19T01:35:17.370 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:17.367+0000 7f2870e25740 1 --2- 172.21.15.83:0/2186344183 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a269a80470 0x55a269a8f1f0 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:35:17.370 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:17.367+0000 7f2870e25740 1 -- 172.21.15.83:0/2186344183 >> 172.21.15.83:0/2186344183 conn(0x55a269a7bae0 msgr2=0x55a269a7acf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:35:17.370 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:17.367+0000 7f2870e25740 1 -- 172.21.15.83:0/2186344183 shutdown_connections 2024-08-19T01:35:17.370 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:17.367+0000 7f2870e25740 1 -- 172.21.15.83:0/2186344183 wait complete. 2024-08-19T01:35:17.371 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:17.367+0000 7f2870e25740 20 client.15135 trim_cache size 0 max 16384 2024-08-19T01:35:17.371 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stdout:{ 2024-08-19T01:35:17.371 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stdout: "message": "", 2024-08-19T01:35:17.371 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stdout: "return_code": 0 2024-08-19T01:35:17.371 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stdout:} 2024-08-19T01:35:17.372 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:17.368+0000 7f286e000640 1 -- 172.21.15.83:0/637844192 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f284c0a2cd0 msgr2=0x7f284c0a5190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:35:17.372 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:17.368+0000 7f286e000640 1 --2- 172.21.15.83:0/637844192 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f284c0a2cd0 0x7f284c0a5190 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f2858009900 tx=0x7f28580f5530 comp rx=0 tx=0).stop 2024-08-19T01:35:17.372 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:17.368+0000 7f286e000640 1 -- 172.21.15.83:0/637844192 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2868103a90 msgr2=0x7f2868199d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:35:17.372 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:17.368+0000 7f286e000640 1 --2- 172.21.15.83:0/637844192 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2868103a90 0x7f2868199d50 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f2858009230 tx=0x7f2858032ce0 comp rx=0 tx=0).stop 2024-08-19T01:35:17.372 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:17.368+0000 7f286e000640 1 -- 172.21.15.83:0/637844192 shutdown_connections 2024-08-19T01:35:17.373 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:17.368+0000 7f286e000640 1 --2- 172.21.15.83:0/637844192 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f284c0a2cd0 0x7f284c0a5190 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:35:17.373 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:17.368+0000 7f286e000640 1 --2- 172.21.15.83:0/637844192 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2868105640 0x7f28681a12c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:35:17.373 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:17.368+0000 7f286e000640 1 --2- 172.21.15.83:0/637844192 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2868104c80 0x7f286819a290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:35:17.373 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:17.368+0000 7f286e000640 1 --2- 172.21.15.83:0/637844192 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2868103a90 0x7f2868199d50 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:35:17.373 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:17.368+0000 7f286e000640 1 -- 172.21.15.83:0/637844192 >> 172.21.15.83:0/637844192 conn(0x7f28680ff210 msgr2=0x7f2868107f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:35:17.373 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:17.368+0000 7f286e000640 1 -- 172.21.15.83:0/637844192 shutdown_connections 2024-08-19T01:35:17.373 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:17.368+0000 7f286e000640 1 -- 172.21.15.83:0/637844192 wait complete. 2024-08-19T01:35:17.654 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:35:17 smithi083 ceph-mon[17120]: pgmap v352: 97 pgs: 97 active+clean; 657 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 1.5 KiB/s wr, 10 op/s 2024-08-19T01:35:17.952 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:35:17 smithi117 ceph-mon[22016]: pgmap v352: 97 pgs: 97 active+clean; 657 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 1.5 KiB/s wr, 10 op/s 2024-08-19T01:35:17.966 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:35:17 smithi167 ceph-mon[21875]: pgmap v352: 97 pgs: 97 active+clean; 657 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 1.5 KiB/s wr, 10 op/s 2024-08-19T01:35:19.755 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:35:19 smithi117 ceph-mon[22016]: pgmap v353: 97 pgs: 97 active+clean; 657 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 853 B/s wr, 8 op/s 2024-08-19T01:35:19.768 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:35:19 smithi167 ceph-mon[21875]: pgmap v353: 97 pgs: 97 active+clean; 657 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 853 B/s wr, 8 op/s 2024-08-19T01:35:19.904 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:35:19 smithi083 ceph-mon[17120]: pgmap v353: 97 pgs: 97 active+clean; 657 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 853 B/s wr, 8 op/s 2024-08-19T01:35:21.904 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:35:21 smithi083 ceph-mon[17120]: pgmap v354: 97 pgs: 97 active+clean; 658 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 511 B/s wr, 7 op/s 2024-08-19T01:35:21.952 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:35:21 smithi117 ceph-mon[22016]: pgmap v354: 97 pgs: 97 active+clean; 658 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 511 B/s wr, 7 op/s 2024-08-19T01:35:21.962 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:35:21 smithi167 ceph-mon[21875]: pgmap v354: 97 pgs: 97 active+clean; 658 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 511 B/s wr, 7 op/s 2024-08-19T01:35:23.904 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:35:23 smithi083 ceph-mon[17120]: pgmap v355: 97 pgs: 97 active+clean; 658 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 255 B/s wr, 5 op/s 2024-08-19T01:35:23.952 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:35:23 smithi117 ceph-mon[22016]: pgmap v355: 97 pgs: 97 active+clean; 658 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 255 B/s wr, 5 op/s 2024-08-19T01:35:23.966 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:35:23 smithi167 ceph-mon[21875]: pgmap v355: 97 pgs: 97 active+clean; 658 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 255 B/s wr, 5 op/s 2024-08-19T01:35:24.520 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833 shell --fsid bad8780e-5dc9-11ef-bccf-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-08-19T01:35:24.944 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/bad8780e-5dc9-11ef-bccf-c7b262605968/mon.a/config 2024-08-19T01:35:25.639 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:25.638+0000 7fdd93200640 1 -- 172.21.15.83:0/1612202305 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdd94074240 msgr2=0x7fdd94074620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:35:25.640 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:25.638+0000 7fdd93200640 1 --2- 172.21.15.83:0/1612202305 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdd94074240 0x7fdd94074620 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7fdd84009900 tx=0x7fdd8402a490 comp rx=0 tx=0).stop 2024-08-19T01:35:25.640 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:25.638+0000 7fdd93200640 1 -- 172.21.15.83:0/1612202305 shutdown_connections 2024-08-19T01:35:25.640 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:25.638+0000 7fdd93200640 1 --2- 172.21.15.83:0/1612202305 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdd9410dfb0 0x7fdd941103a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:35:25.640 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:25.638+0000 7fdd93200640 1 --2- 172.21.15.83:0/1612202305 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdd94074b60 0x7fdd9410da70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:35:25.640 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:25.638+0000 7fdd93200640 1 --2- 172.21.15.83:0/1612202305 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdd94074240 0x7fdd94074620 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:35:25.640 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:25.638+0000 7fdd93200640 1 -- 172.21.15.83:0/1612202305 >> 172.21.15.83:0/1612202305 conn(0x7fdd940fd470 msgr2=0x7fdd940ff890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:35:25.640 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:25.638+0000 7fdd93200640 1 -- 172.21.15.83:0/1612202305 shutdown_connections 2024-08-19T01:35:25.640 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:25.638+0000 7fdd93200640 1 -- 172.21.15.83:0/1612202305 wait complete. 2024-08-19T01:35:25.641 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:25.639+0000 7fdd93200640 1 Processor -- start 2024-08-19T01:35:25.641 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:25.640+0000 7fdd93200640 1 -- start start 2024-08-19T01:35:25.641 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:25.640+0000 7fdd93200640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdd94074240 0x7fdd9419a230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:35:25.641 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:25.640+0000 7fdd93200640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdd94074b60 0x7fdd9419a770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:35:25.641 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:25.640+0000 7fdd93200640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdd9410dfb0 0x7fdd9419eb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:35:25.641 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:25.640+0000 7fdd93200640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fdd94113d30 con 0x7fdd94074b60 2024-08-19T01:35:25.641 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:25.640+0000 7fdd93200640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fdd94113bb0 con 0x7fdd94074240 2024-08-19T01:35:25.642 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:25.640+0000 7fdd93200640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fdd94113eb0 con 0x7fdd9410dfb0 2024-08-19T01:35:25.642 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:25.641+0000 7fdd91e00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdd94074240 0x7fdd9419a230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:35:25.642 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:25.641+0000 7fdd91400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdd94074b60 0x7fdd9419a770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:35:25.642 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:25.641+0000 7fdd91e00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdd94074240 0x7fdd9419a230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.83:39060/0 (socket says 172.21.15.83:39060) 2024-08-19T01:35:25.642 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:25.641+0000 7fdd91e00640 1 -- 172.21.15.83:0/3868566560 learned_addr learned my addr 172.21.15.83:0/3868566560 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-08-19T01:35:25.643 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:25.641+0000 7fdd92800640 1 --2- 172.21.15.83:0/3868566560 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdd9410dfb0 0x7fdd9419eb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:35:25.643 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:25.641+0000 7fdd91e00640 1 -- 172.21.15.83:0/3868566560 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdd9410dfb0 msgr2=0x7fdd9419eb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:35:25.643 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:25.641+0000 7fdd91e00640 1 --2- 172.21.15.83:0/3868566560 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdd9410dfb0 0x7fdd9419eb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:35:25.643 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:25.641+0000 7fdd91e00640 1 -- 172.21.15.83:0/3868566560 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdd94074b60 msgr2=0x7fdd9419a770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:35:25.643 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:25.641+0000 7fdd91e00640 1 --2- 172.21.15.83:0/3868566560 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdd94074b60 0x7fdd9419a770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:35:25.643 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:25.641+0000 7fdd91e00640 1 -- 172.21.15.83:0/3868566560 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdd9419f1e0 con 0x7fdd94074240 2024-08-19T01:35:25.643 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:25.641+0000 7fdd92800640 1 --2- 172.21.15.83:0/3868566560 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdd9410dfb0 0x7fdd9419eb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-19T01:35:25.644 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:25.641+0000 7fdd91400640 1 --2- 172.21.15.83:0/3868566560 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdd94074b60 0x7fdd9419a770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-19T01:35:25.644 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:25.642+0000 7fdd91e00640 1 --2- 172.21.15.83:0/3868566560 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdd94074240 0x7fdd9419a230 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fdd84004770 tx=0x7fdd84003c10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:35:25.644 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:25.642+0000 7fdd8aa00640 1 -- 172.21.15.83:0/3868566560 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd8403f070 con 0x7fdd94074240 2024-08-19T01:35:25.644 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:25.642+0000 7fdd8aa00640 1 -- 172.21.15.83:0/3868566560 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdd84031ca0 con 0x7fdd94074240 2024-08-19T01:35:25.644 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:25.642+0000 7fdd93200640 1 -- 172.21.15.83:0/3868566560 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdd9419f470 con 0x7fdd94074240 2024-08-19T01:35:25.644 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:25.642+0000 7fdd8aa00640 1 -- 172.21.15.83:0/3868566560 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd84035030 con 0x7fdd94074240 2024-08-19T01:35:25.644 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:25.642+0000 7fdd93200640 1 -- 172.21.15.83:0/3868566560 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdd941a6db0 con 0x7fdd94074240 2024-08-19T01:35:25.648 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:25.647+0000 7fdd9b657740 1 Processor -- start 2024-08-19T01:35:25.648 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:25.647+0000 7fdd9b657740 1 -- start start 2024-08-19T01:35:25.648 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:25.647+0000 7fdd8aa00640 1 -- 172.21.15.83:0/3868566560 <== mon.2 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 19) ==== 137884+0+0 (secure 0 0 0) 0x7fdd84042070 con 0x7fdd94074240 2024-08-19T01:35:25.649 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:25.647+0000 7fdd9b657740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x555a48e55d80 0x555a48e56160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:35:25.649 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:25.647+0000 7fdd9b657740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555a48e566a0 0x555a48e60a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:35:25.649 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:25.647+0000 7fdd9b657740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555a48e60f60 0x555a48e63350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:35:25.649 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:25.647+0000 7fdd9b657740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x555a48dbbbf0 con 0x555a48e566a0 2024-08-19T01:35:25.649 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:25.647+0000 7fdd9b657740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x555a48d53bd0 con 0x555a48e60f60 2024-08-19T01:35:25.649 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:25.647+0000 7fdd9b657740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x555a48db7bb0 con 0x555a48e55d80 2024-08-19T01:35:25.650 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:25.648+0000 7fdd92800640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x555a48e55d80 0x555a48e56160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:35:25.650 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:25.648+0000 7fdd92800640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x555a48e55d80 0x555a48e56160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.83:46470/0 (socket says 172.21.15.83:46470) 2024-08-19T01:35:25.650 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:25.648+0000 7fdd92800640 1 -- 172.21.15.83:0/1556383147 learned_addr learned my addr 172.21.15.83:0/1556383147 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-08-19T01:35:25.650 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:25.648+0000 7fdd92800640 1 -- 172.21.15.83:0/1556383147 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555a48e60f60 msgr2=0x555a48e63350 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-08-19T01:35:25.650 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:25.648+0000 7fdd92800640 1 --2- 172.21.15.83:0/1556383147 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555a48e60f60 0x555a48e63350 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:35:25.650 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:25.648+0000 7fdd92800640 1 -- 172.21.15.83:0/1556383147 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555a48e566a0 msgr2=0x555a48e60a20 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-08-19T01:35:25.650 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:25.648+0000 7fdd92800640 1 --2- 172.21.15.83:0/1556383147 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555a48e566a0 0x555a48e60a20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:35:25.650 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:25.648+0000 7fdd92800640 1 -- 172.21.15.83:0/1556383147 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555a48d5a8a0 con 0x555a48e55d80 2024-08-19T01:35:25.651 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:25.649+0000 7fdd92800640 1 --2- 172.21.15.83:0/1556383147 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x555a48e55d80 0x555a48e56160 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fdd8c009fa0 tx=0x7fdd8c010860 comp rx=0 tx=0).ready entity=mon.1 client_cookie=aa8ae6bae03b1f2e server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:35:25.651 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:25.649+0000 7fdd8aa00640 1 --2- 172.21.15.83:0/3868566560 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7fdd780a2c10 0x7fdd780a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:35:25.651 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:25.649+0000 7fdd8aa00640 1 -- 172.21.15.83:0/3868566560 <== mon.2 v2:172.21.15.117:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8269+0+0 (secure 0 0 0) 0x7fdd840ef880 con 0x7fdd94074240 2024-08-19T01:35:25.651 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:25.649+0000 7fdd7f000640 1 -- 172.21.15.83:0/1556383147 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd8c011700 con 0x555a48e55d80 2024-08-19T01:35:25.651 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:25.649+0000 7fdd91e00640 1 --2- 172.21.15.83:0/3868566560 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7fdd780a2c10 0x7fdd780a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:35:25.651 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:25.649+0000 7fdd7f000640 1 -- 172.21.15.83:0/1556383147 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdd8c011d80 con 0x555a48e55d80 2024-08-19T01:35:25.651 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:25.649+0000 7fdd7f000640 1 -- 172.21.15.83:0/1556383147 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd8c01a380 con 0x555a48e55d80 2024-08-19T01:35:25.652 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:25.650+0000 7fdd9b657740 1 -- 172.21.15.83:0/1556383147 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x555a48e55d80 msgr2=0x555a48e56160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:35:25.652 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:25.650+0000 7fdd91e00640 1 --2- 172.21.15.83:0/3868566560 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7fdd780a2c10 0x7fdd780a50d0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fdd840bbfb0 tx=0x7fdd8402a490 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:35:25.652 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:25.650+0000 7fdd9b657740 1 --2- 172.21.15.83:0/1556383147 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x555a48e55d80 0x555a48e56160 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fdd8c009fa0 tx=0x7fdd8c010860 comp rx=0 tx=0).stop 2024-08-19T01:35:25.652 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:25.651+0000 7fdd9b657740 1 -- 172.21.15.83:0/1556383147 shutdown_connections 2024-08-19T01:35:25.652 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:25.651+0000 7fdd9b657740 1 --2- 172.21.15.83:0/1556383147 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555a48e60f60 0x555a48e63350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:35:25.652 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:25.651+0000 7fdd9b657740 1 --2- 172.21.15.83:0/1556383147 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555a48e566a0 0x555a48e60a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:35:25.652 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:25.651+0000 7fdd9b657740 1 --2- 172.21.15.83:0/1556383147 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x555a48e55d80 0x555a48e56160 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:35:25.652 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:25.651+0000 7fdd9b657740 1 -- 172.21.15.83:0/1556383147 >> 172.21.15.83:0/1556383147 conn(0x555a48e52310 msgr2=0x555a48e526f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:35:25.653 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:25.651+0000 7fdd9b657740 1 -- 172.21.15.83:0/1556383147 shutdown_connections 2024-08-19T01:35:25.653 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:25.651+0000 7fdd9b657740 1 -- 172.21.15.83:0/1556383147 wait complete. 2024-08-19T01:35:25.653 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:25.651+0000 7fdd9b657740 1 -- start start 2024-08-19T01:35:25.653 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:25.652+0000 7fdd9b657740 1 Processor -- start 2024-08-19T01:35:25.653 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:25.652+0000 7fdd9b657740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x555a48e55d80 0x555a48e5dec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:35:25.653 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:25.652+0000 7fdd9b657740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555a48e566a0 0x555a48e5e400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:35:25.653 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:25.652+0000 7fdd9b657740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555a48e60f60 0x555a48e5e940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:35:25.654 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:25.652+0000 7fdd9b657740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x555a48e63b30 con 0x555a48e60f60 2024-08-19T01:35:25.654 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:25.652+0000 7fdd9b657740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x555a48d76d40 con 0x555a48e566a0 2024-08-19T01:35:25.654 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:25.652+0000 7fdd9b657740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x555a48e63cb0 con 0x555a48e55d80 2024-08-19T01:35:25.654 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:25.653+0000 7fdd9b657740 10 client.0 _ll_register_callbacks cb 0x555a48d3fb10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-19T01:35:25.655 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:25.653+0000 7fdd92800640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x555a48e55d80 0x555a48e5dec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:35:25.655 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:25.653+0000 7fdd92800640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x555a48e55d80 0x555a48e5dec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.83:46474/0 (socket says 172.21.15.83:46474) 2024-08-19T01:35:25.655 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:25.653+0000 7fdd92800640 1 -- 172.21.15.83:0/1485632496 learned_addr learned my addr 172.21.15.83:0/1485632496 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-08-19T01:35:25.655 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:25.653+0000 7fdd92800640 1 --2- 172.21.15.83:0/1485632496 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555a48e60f60 0x555a48e5e940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:35:25.655 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:25.653+0000 7fdd91400640 1 --2- 172.21.15.83:0/1485632496 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555a48e566a0 0x555a48e5e400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:35:25.656 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:25.654+0000 7fdd92800640 1 -- 172.21.15.83:0/1485632496 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555a48e566a0 msgr2=0x555a48e5e400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:35:25.656 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:25.654+0000 7fdd92800640 1 --2- 172.21.15.83:0/1485632496 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555a48e566a0 0x555a48e5e400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:35:25.656 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:25.654+0000 7fdd92800640 1 -- 172.21.15.83:0/1485632496 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555a48e60f60 msgr2=0x555a48e5e940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:35:25.656 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:25.654+0000 7fdd92800640 1 --2- 172.21.15.83:0/1485632496 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555a48e60f60 0x555a48e5e940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:35:25.656 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:25.654+0000 7fdd92800640 1 -- 172.21.15.83:0/1485632496 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555a48b2d3c0 con 0x555a48e55d80 2024-08-19T01:35:25.656 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:25.654+0000 7fdd92800640 1 --2- 172.21.15.83:0/1485632496 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x555a48e55d80 0x555a48e5dec0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fdd8c011fd0 tx=0x7fdd8c013040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=87ddcaac44d4f0a0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:35:25.656 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:25.654+0000 7fdd73400640 10 client.0 ms_handle_connect on v2:172.21.15.167:3300/0 2024-08-19T01:35:25.656 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:25.654+0000 7fdd73400640 1 -- 172.21.15.83:0/1485632496 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd8c01e110 con 0x555a48e55d80 2024-08-19T01:35:25.656 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:25.654+0000 7fdd73400640 1 -- 172.21.15.83:0/1485632496 <== mon.1 v2:172.21.15.167:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8269+0+0 (secure 0 0 0) 0x7fdd8c01e2b0 con 0x555a48e55d80 2024-08-19T01:35:25.657 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:25.654+0000 7fdd9b657740 1 -- 172.21.15.83:0/1485632496 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdd8c011d80 con 0x555a48e55d80 2024-08-19T01:35:25.657 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:25.655+0000 7fdd73400640 1 -- 172.21.15.83:0/1485632496 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fdd8c01e4b0 con 0x555a48e55d80 2024-08-19T01:35:25.657 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:25.655+0000 7fdd9b657740 10 client.25186 fetch_fsmap learned FSMap version 25 2024-08-19T01:35:25.657 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:25.655+0000 7fdd9b657740 1 -- 172.21.15.83:0/1485632496 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x555a48e6f860 con 0x555a48e55d80 2024-08-19T01:35:25.657 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:25.655+0000 7fdd73400640 1 -- 172.21.15.83:0/1485632496 <== mon.1 v2:172.21.15.167:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdd8c011d80 con 0x555a48e55d80 2024-08-19T01:35:25.657 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:25.655+0000 7fdd73400640 1 -- 172.21.15.83:0/1485632496 <== mon.1 v2:172.21.15.167:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fdd8c02a020 con 0x555a48e55d80 2024-08-19T01:35:25.658 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:25.656+0000 7fdd9b657740 10 client.25186 fetch_fsmap finished waiting for FSMap version 25 2024-08-19T01:35:25.658 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:25.656+0000 7fdd9b657740 10 client.25186 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-19T01:35:25.658 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:25.656+0000 7fdd9b657740 20 client.25186 populate_metadata read hostname 'smithi083' 2024-08-19T01:35:25.658 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:25.656+0000 7fdd9b657740 1 --2- 172.21.15.83:0/1485632496 >> [v2:172.21.15.117:6838/1967595704,v1:172.21.15.117:6839/1967595704] conn(0x555a48e70c70 0x555a48e73060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:35:25.658 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:25.656+0000 7fdd9b657740 4 client.25186 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-19T01:35:25.658 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:25.656+0000 7fdd9b657740 1 -- 172.21.15.83:0/1485632496 --> [v2:172.21.15.117:6838/1967595704,v1:172.21.15.117:6839/1967595704] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555a48dc9160 con 0x555a48e70c70 2024-08-19T01:35:25.658 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:25.656+0000 7fdd91400640 1 --2- 172.21.15.83:0/1485632496 >> [v2:172.21.15.117:6838/1967595704,v1:172.21.15.117:6839/1967595704] conn(0x555a48e70c70 0x555a48e73060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:35:25.658 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:25.657+0000 7fdd91400640 1 --2- 172.21.15.83:0/1485632496 >> [v2:172.21.15.117:6838/1967595704,v1:172.21.15.117:6839/1967595704] conn(0x555a48e70c70 0x555a48e73060 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2e5da10dd07b6ae8 server_cookie=82e4e6ba783e7338 in_seq=0 out_seq=0 2024-08-19T01:35:25.658 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:25.657+0000 7fdd73400640 10 client.25186 ms_handle_connect on v2:172.21.15.117:6838/1967595704 2024-08-19T01:35:25.659 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:25.658+0000 7fdd73400640 1 -- 172.21.15.83:0/1485632496 <== mds.0 v2:172.21.15.117:6838/1967595704 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x555a48dc9160 con 0x555a48e70c70 2024-08-19T01:35:25.660 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:25.658+0000 7fdd73400640 10 client.25186 handle_command_reply: tid=0 2024-08-19T01:35:25.660 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:25.658+0000 7fdd9b657740 1 client.25186 shutdown 2024-08-19T01:35:25.660 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:25.658+0000 7fdd9b657740 1 -- 172.21.15.83:0/1485632496 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x555a48e55d80 msgr2=0x555a48e5dec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:35:25.660 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:25.658+0000 7fdd9b657740 1 --2- 172.21.15.83:0/1485632496 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x555a48e55d80 0x555a48e5dec0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fdd8c011fd0 tx=0x7fdd8c013040 comp rx=0 tx=0).stop 2024-08-19T01:35:25.660 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:25.658+0000 7fdd9b657740 1 -- 172.21.15.83:0/1485632496 shutdown_connections 2024-08-19T01:35:25.660 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:25.658+0000 7fdd9b657740 1 --2- 172.21.15.83:0/1485632496 >> [v2:172.21.15.117:6838/1967595704,v1:172.21.15.117:6839/1967595704] conn(0x555a48e70c70 0x555a48e73060 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:35:25.660 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:25.658+0000 7fdd9b657740 1 --2- 172.21.15.83:0/1485632496 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555a48e60f60 0x555a48e5e940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:35:25.660 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:25.658+0000 7fdd9b657740 1 --2- 172.21.15.83:0/1485632496 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555a48e566a0 0x555a48e5e400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:35:25.660 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:25.658+0000 7fdd9b657740 1 --2- 172.21.15.83:0/1485632496 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x555a48e55d80 0x555a48e5dec0 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:35:25.660 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:25.658+0000 7fdd9b657740 1 -- 172.21.15.83:0/1485632496 >> 172.21.15.83:0/1485632496 conn(0x555a48e52310 msgr2=0x555a48e58040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:35:25.660 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:25.658+0000 7fdd73400640 0 client.25186 ms_handle_reset on v2:172.21.15.117:6838/1967595704 2024-08-19T01:35:25.660 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:25.658+0000 7fdd9b657740 1 -- 172.21.15.83:0/1485632496 shutdown_connections 2024-08-19T01:35:25.660 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:25.658+0000 7fdd9b657740 1 -- 172.21.15.83:0/1485632496 wait complete. 2024-08-19T01:35:25.661 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:25.659+0000 7fdd9b657740 20 client.25186 trim_cache size 0 max 16384 2024-08-19T01:35:25.687 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:25.686+0000 7fdd9b657740 1 Processor -- start 2024-08-19T01:35:25.687 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:25.686+0000 7fdd9b657740 1 -- start start 2024-08-19T01:35:25.687 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:25.687+0000 7fdd9b657740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555a48e55d80 0x555a48e587e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:35:25.687 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:25.687+0000 7fdd9b657740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555a48e566a0 0x555a48e787a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:35:25.688 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:25.687+0000 7fdd9b657740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x555a48e60f60 0x555a48e78ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:35:25.688 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:25.687+0000 7fdd9b657740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fdd941b1d30 con 0x555a48e566a0 2024-08-19T01:35:25.688 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:25.687+0000 7fdd9b657740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fdd941af600 con 0x555a48e55d80 2024-08-19T01:35:25.688 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:25.687+0000 7fdd9b657740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fdd941ad450 con 0x555a48e60f60 2024-08-19T01:35:25.688 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:25.687+0000 7fdd91400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555a48e566a0 0x555a48e787a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:35:25.689 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:25.687+0000 7fdd92800640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555a48e55d80 0x555a48e587e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:35:25.689 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:25.687+0000 7fdd92800640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555a48e55d80 0x555a48e587e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.83:39082/0 (socket says 172.21.15.83:39082) 2024-08-19T01:35:25.689 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:25.687+0000 7fdd91400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555a48e566a0 0x555a48e787a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:51610/0 (socket says 172.21.15.83:51610) 2024-08-19T01:35:25.689 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:25.687+0000 7fdd92800640 1 -- 172.21.15.83:0/1370376034 learned_addr learned my addr 172.21.15.83:0/1370376034 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-08-19T01:35:25.689 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:25.687+0000 7fdd92800640 1 --2- 172.21.15.83:0/1370376034 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x555a48e60f60 0x555a48e78ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:35:25.689 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:25.688+0000 7fdd91400640 1 -- 172.21.15.83:0/1370376034 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x555a48e60f60 msgr2=0x555a48e78ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:35:25.689 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:25.688+0000 7fdd91400640 1 --2- 172.21.15.83:0/1370376034 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x555a48e60f60 0x555a48e78ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:35:25.689 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:25.688+0000 7fdd91400640 1 -- 172.21.15.83:0/1370376034 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555a48e55d80 msgr2=0x555a48e587e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:35:25.689 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:25.688+0000 7fdd91400640 1 --2- 172.21.15.83:0/1370376034 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555a48e55d80 0x555a48e587e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:35:25.689 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:25.688+0000 7fdd91400640 1 -- 172.21.15.83:0/1370376034 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555a48e51db0 con 0x555a48e566a0 2024-08-19T01:35:25.689 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:25.688+0000 7fdd91400640 1 --2- 172.21.15.83:0/1370376034 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555a48e566a0 0x555a48e787a0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x555a48e73740 tx=0x7fdd80013890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=477433369dcd0125 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:35:25.690 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:25.688+0000 7fdd7e600640 1 -- 172.21.15.83:0/1370376034 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd80023030 con 0x555a48e566a0 2024-08-19T01:35:25.690 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:25.689+0000 7fdd7e600640 1 -- 172.21.15.83:0/1370376034 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdd800121d0 con 0x555a48e566a0 2024-08-19T01:35:25.690 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:25.689+0000 7fdd7e600640 1 -- 172.21.15.83:0/1370376034 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd8001e990 con 0x555a48e566a0 2024-08-19T01:35:25.691 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:25.689+0000 7fdd9b657740 1 -- 172.21.15.83:0/1370376034 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555a48e566a0 msgr2=0x555a48e787a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:35:25.691 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:25.689+0000 7fdd9b657740 1 --2- 172.21.15.83:0/1370376034 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555a48e566a0 0x555a48e787a0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x555a48e73740 tx=0x7fdd80013890 comp rx=0 tx=0).stop 2024-08-19T01:35:25.691 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:25.689+0000 7fdd9b657740 1 -- 172.21.15.83:0/1370376034 shutdown_connections 2024-08-19T01:35:25.691 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:25.689+0000 7fdd9b657740 1 --2- 172.21.15.83:0/1370376034 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x555a48e60f60 0x555a48e78ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:35:25.691 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:25.689+0000 7fdd9b657740 1 --2- 172.21.15.83:0/1370376034 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555a48e566a0 0x555a48e787a0 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:35:25.691 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:25.689+0000 7fdd9b657740 1 --2- 172.21.15.83:0/1370376034 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555a48e55d80 0x555a48e587e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:35:25.691 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:25.689+0000 7fdd9b657740 1 -- 172.21.15.83:0/1370376034 >> 172.21.15.83:0/1370376034 conn(0x555a48e52310 msgr2=0x555a48e5e6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:35:25.691 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:25.690+0000 7fdd9b657740 1 -- 172.21.15.83:0/1370376034 shutdown_connections 2024-08-19T01:35:25.691 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:25.690+0000 7fdd9b657740 1 -- 172.21.15.83:0/1370376034 wait complete. 2024-08-19T01:35:25.692 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:25.691+0000 7fdd9b657740 1 -- start start 2024-08-19T01:35:25.692 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:25.691+0000 7fdd9b657740 1 Processor -- start 2024-08-19T01:35:25.692 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:25.691+0000 7fdd9b657740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555a48e55d80 0x555a48e64480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:35:25.692 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:25.691+0000 7fdd9b657740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x555a48e60f60 0x555a48e649c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:35:25.692 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:25.691+0000 7fdd9b657740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555a48ed7750 0x555a48ed9b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:35:25.692 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:25.691+0000 7fdd9b657740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fdd941adac0 con 0x555a48ed7750 2024-08-19T01:35:25.692 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:25.691+0000 7fdd9b657740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fdd941aef30 con 0x555a48e55d80 2024-08-19T01:35:25.692 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:25.691+0000 7fdd9b657740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fdd941ae190 con 0x555a48e60f60 2024-08-19T01:35:25.692 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:25.692+0000 7fdd9b657740 10 client.0 _ll_register_callbacks cb 0x7fdd8c009e20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-19T01:35:25.693 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:25.692+0000 7fdd92800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555a48ed7750 0x555a48ed9b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:35:25.693 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:25.692+0000 7fdd92800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555a48ed7750 0x555a48ed9b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:51622/0 (socket says 172.21.15.83:51622) 2024-08-19T01:35:25.693 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:25.692+0000 7fdd92800640 1 -- 172.21.15.83:0/879790883 learned_addr learned my addr 172.21.15.83:0/879790883 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-08-19T01:35:25.693 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:25.692+0000 7fdd92800640 1 --2- 172.21.15.83:0/879790883 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555a48e55d80 0x555a48e64480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:35:25.693 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:25.692+0000 7fdd91400640 1 --2- 172.21.15.83:0/879790883 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x555a48e60f60 0x555a48e649c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:35:25.693 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:25.692+0000 7fdd92800640 1 -- 172.21.15.83:0/879790883 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x555a48e60f60 msgr2=0x555a48e649c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:35:25.694 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:25.692+0000 7fdd92800640 1 --2- 172.21.15.83:0/879790883 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x555a48e60f60 0x555a48e649c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:35:25.694 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:25.692+0000 7fdd92800640 1 -- 172.21.15.83:0/879790883 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555a48e55d80 msgr2=0x555a48e64480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:35:25.694 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:25.692+0000 7fdd92800640 1 --2- 172.21.15.83:0/879790883 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555a48e55d80 0x555a48e64480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:35:25.694 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:25.692+0000 7fdd92800640 1 -- 172.21.15.83:0/879790883 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555a48eda080 con 0x555a48ed7750 2024-08-19T01:35:25.694 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:25.693+0000 7fdd92800640 1 --2- 172.21.15.83:0/879790883 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555a48ed7750 0x555a48ed9b40 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7fdd8c006010 tx=0x7fdd8c01cb30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a6ca10711cd54bf0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:35:25.695 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:25.693+0000 7fdd73400640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-08-19T01:35:25.695 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:25.693+0000 7fdd73400640 1 -- 172.21.15.83:0/879790883 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd8c01c950 con 0x555a48ed7750 2024-08-19T01:35:25.695 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:25.693+0000 7fdd9b657740 1 -- 172.21.15.83:0/879790883 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdd800121d0 con 0x555a48ed7750 2024-08-19T01:35:25.695 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:25.694+0000 7fdd73400640 1 -- 172.21.15.83:0/879790883 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8269+0+0 (secure 0 0 0) 0x7fdd8c02c650 con 0x555a48ed7750 2024-08-19T01:35:25.695 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:25.694+0000 7fdd73400640 1 -- 172.21.15.83:0/879790883 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fdd8c0049e0 con 0x555a48ed7750 2024-08-19T01:35:25.695 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:25.694+0000 7fdd9b657740 10 client.15150 fetch_fsmap learned FSMap version 25 2024-08-19T01:35:25.695 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:25.694+0000 7fdd9b657740 1 -- 172.21.15.83:0/879790883 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x555a48edb330 con 0x555a48ed7750 2024-08-19T01:35:25.695 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:25.694+0000 7fdd73400640 1 -- 172.21.15.83:0/879790883 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdd800121d0 con 0x555a48ed7750 2024-08-19T01:35:25.696 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:25.694+0000 7fdd73400640 1 -- 172.21.15.83:0/879790883 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fdd8c02a040 con 0x555a48ed7750 2024-08-19T01:35:25.696 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:25.694+0000 7fdd9b657740 10 client.15150 fetch_fsmap finished waiting for FSMap version 25 2024-08-19T01:35:25.696 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:25.694+0000 7fdd9b657740 10 client.15150 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-19T01:35:25.696 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:25.694+0000 7fdd9b657740 20 client.15150 populate_metadata read hostname 'smithi083' 2024-08-19T01:35:25.696 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:25.695+0000 7fdd9b657740 1 --2- 172.21.15.83:0/879790883 >> [v2:172.21.15.117:6838/1967595704,v1:172.21.15.117:6839/1967595704] conn(0x555a48edb8c0 0x555a48eddce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:35:25.696 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:25.695+0000 7fdd9b657740 4 client.15150 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-08-19T01:35:25.696 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:25.695+0000 7fdd9b657740 1 -- 172.21.15.83:0/879790883 --> [v2:172.21.15.117:6838/1967595704,v1:172.21.15.117:6839/1967595704] -- command(tid 0: {"prefix": "scrub status"}) -- 0x555a48dc3670 con 0x555a48edb8c0 2024-08-19T01:35:25.696 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:25.695+0000 7fdd91400640 1 --2- 172.21.15.83:0/879790883 >> [v2:172.21.15.117:6838/1967595704,v1:172.21.15.117:6839/1967595704] conn(0x555a48edb8c0 0x555a48eddce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:35:25.696 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:25.696+0000 7fdd91400640 1 --2- 172.21.15.83:0/879790883 >> [v2:172.21.15.117:6838/1967595704,v1:172.21.15.117:6839/1967595704] conn(0x555a48edb8c0 0x555a48eddce0 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=78243c6d456743a2 server_cookie=d3c8c019f2583f3e in_seq=0 out_seq=0 2024-08-19T01:35:25.697 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:25.696+0000 7fdd73400640 10 client.15150 ms_handle_connect on v2:172.21.15.117:6838/1967595704 2024-08-19T01:35:25.697 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:25.696+0000 7fdd73400640 1 -- 172.21.15.83:0/879790883 <== mds.0 v2:172.21.15.117:6838/1967595704 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x555a48dc3670 con 0x555a48edb8c0 2024-08-19T01:35:25.698 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:25.696+0000 7fdd73400640 10 client.15150 handle_command_reply: tid=0 2024-08-19T01:35:25.698 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:25.696+0000 7fdd9b657740 1 client.15150 shutdown 2024-08-19T01:35:25.698 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:25.696+0000 7fdd9b657740 1 -- 172.21.15.83:0/879790883 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555a48ed7750 msgr2=0x555a48ed9b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:35:25.698 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:25.696+0000 7fdd9b657740 1 --2- 172.21.15.83:0/879790883 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555a48ed7750 0x555a48ed9b40 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7fdd8c006010 tx=0x7fdd8c01cb30 comp rx=0 tx=0).stop 2024-08-19T01:35:25.698 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:25.696+0000 7fdd9b657740 1 -- 172.21.15.83:0/879790883 shutdown_connections 2024-08-19T01:35:25.698 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:25.696+0000 7fdd9b657740 1 --2- 172.21.15.83:0/879790883 >> [v2:172.21.15.117:6838/1967595704,v1:172.21.15.117:6839/1967595704] conn(0x555a48edb8c0 0x555a48eddce0 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:35:25.698 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:25.696+0000 7fdd9b657740 1 --2- 172.21.15.83:0/879790883 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555a48ed7750 0x555a48ed9b40 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:35:25.698 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:25.696+0000 7fdd9b657740 1 --2- 172.21.15.83:0/879790883 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x555a48e60f60 0x555a48e649c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:35:25.698 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:25.696+0000 7fdd9b657740 1 --2- 172.21.15.83:0/879790883 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555a48e55d80 0x555a48e64480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:35:25.698 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:25.696+0000 7fdd9b657740 1 -- 172.21.15.83:0/879790883 >> 172.21.15.83:0/879790883 conn(0x555a48e52310 msgr2=0x555a48e6bd80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:35:25.698 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:25.696+0000 7fdd73400640 0 client.15150 ms_handle_reset on v2:172.21.15.117:6838/1967595704 2024-08-19T01:35:25.698 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:25.697+0000 7fdd9b657740 1 -- 172.21.15.83:0/879790883 shutdown_connections 2024-08-19T01:35:25.698 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:25.697+0000 7fdd9b657740 1 -- 172.21.15.83:0/879790883 wait complete. 2024-08-19T01:35:25.699 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:25.697+0000 7fdd9b657740 20 client.15150 trim_cache size 0 max 16384 2024-08-19T01:35:25.699 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-08-19T01:35:25.699 INFO:teuthology.orchestra.run.smithi083.stdout: "status": "no active scrubs running", 2024-08-19T01:35:25.699 INFO:teuthology.orchestra.run.smithi083.stdout: "scrubs": {} 2024-08-19T01:35:25.699 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-08-19T01:35:25.700 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:25.698+0000 7fdd93200640 1 -- 172.21.15.83:0/3868566560 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7fdd780a2c10 msgr2=0x7fdd780a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:35:25.700 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:25.698+0000 7fdd93200640 1 --2- 172.21.15.83:0/3868566560 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7fdd780a2c10 0x7fdd780a50d0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fdd840bbfb0 tx=0x7fdd8402a490 comp rx=0 tx=0).stop 2024-08-19T01:35:25.700 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:25.698+0000 7fdd93200640 1 -- 172.21.15.83:0/3868566560 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdd94074240 msgr2=0x7fdd9419a230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:35:25.700 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:25.698+0000 7fdd93200640 1 --2- 172.21.15.83:0/3868566560 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdd94074240 0x7fdd9419a230 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fdd84004770 tx=0x7fdd84003c10 comp rx=0 tx=0).stop 2024-08-19T01:35:25.700 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:25.698+0000 7fdd93200640 1 -- 172.21.15.83:0/3868566560 shutdown_connections 2024-08-19T01:35:25.700 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:25.698+0000 7fdd93200640 1 --2- 172.21.15.83:0/3868566560 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7fdd780a2c10 0x7fdd780a50d0 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:35:25.700 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:25.698+0000 7fdd93200640 1 --2- 172.21.15.83:0/3868566560 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdd9410dfb0 0x7fdd9419eb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:35:25.700 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:25.698+0000 7fdd93200640 1 --2- 172.21.15.83:0/3868566560 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdd94074b60 0x7fdd9419a770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:35:25.701 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:25.698+0000 7fdd93200640 1 --2- 172.21.15.83:0/3868566560 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdd94074240 0x7fdd9419a230 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:35:25.701 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:25.698+0000 7fdd93200640 1 -- 172.21.15.83:0/3868566560 >> 172.21.15.83:0/3868566560 conn(0x7fdd940fd470 msgr2=0x7fdd940feb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:35:25.701 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:25.698+0000 7fdd93200640 1 -- 172.21.15.83:0/3868566560 shutdown_connections 2024-08-19T01:35:25.701 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:25.698+0000 7fdd93200640 1 -- 172.21.15.83:0/3868566560 wait complete. 2024-08-19T01:35:25.904 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:35:25 smithi083 ceph-mon[17120]: pgmap v356: 97 pgs: 97 active+clean; 658 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 6 op/s 2024-08-19T01:35:25.952 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:35:25 smithi117 ceph-mon[22016]: pgmap v356: 97 pgs: 97 active+clean; 658 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 6 op/s 2024-08-19T01:35:25.966 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:35:25 smithi167 ceph-mon[21875]: pgmap v356: 97 pgs: 97 active+clean; 658 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 6 op/s 2024-08-19T01:35:26.524 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-08-19T01:35:26.524 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833 shell --fsid bad8780e-5dc9-11ef-bccf-c7b262605968 -- ceph fs dump --format=json 2024-08-19T01:35:27.055 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/bad8780e-5dc9-11ef-bccf-c7b262605968/mon.a/config 2024-08-19T01:35:27.490 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:35:27 smithi083 ceph-mon[17120]: pgmap v357: 97 pgs: 97 active+clean; 667 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 1.1 KiB/s wr, 7 op/s 2024-08-19T01:35:27.690 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:27.688+0000 7f3bcb200640 1 -- 172.21.15.83:0/2664912491 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3bcc074c10 msgr2=0x7f3bcc10da50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:35:27.690 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:27.689+0000 7f3bcb200640 1 --2- 172.21.15.83:0/2664912491 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3bcc074c10 0x7f3bcc10da50 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f3bb8009900 tx=0x7f3bb802a480 comp rx=0 tx=0).stop 2024-08-19T01:35:27.690 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:27.689+0000 7f3bcb200640 1 -- 172.21.15.83:0/2664912491 shutdown_connections 2024-08-19T01:35:27.690 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:27.689+0000 7f3bcb200640 1 --2- 172.21.15.83:0/2664912491 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3bcc10df90 0x7f3bcc110380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:35:27.690 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:27.689+0000 7f3bcb200640 1 --2- 172.21.15.83:0/2664912491 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3bcc074c10 0x7f3bcc10da50 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:35:27.690 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:27.689+0000 7f3bcb200640 1 --2- 172.21.15.83:0/2664912491 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3bcc0742f0 0x7f3bcc0746d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:35:27.690 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:27.689+0000 7f3bcb200640 1 -- 172.21.15.83:0/2664912491 >> 172.21.15.83:0/2664912491 conn(0x7f3bcc0fd470 msgr2=0x7f3bcc0ff890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:35:27.691 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:27.689+0000 7f3bcb200640 1 -- 172.21.15.83:0/2664912491 shutdown_connections 2024-08-19T01:35:27.691 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:27.689+0000 7f3bcb200640 1 -- 172.21.15.83:0/2664912491 wait complete. 2024-08-19T01:35:27.694 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:27.690+0000 7f3bcb200640 1 Processor -- start 2024-08-19T01:35:27.694 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:27.690+0000 7f3bcb200640 1 -- start start 2024-08-19T01:35:27.694 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:27.691+0000 7f3bcb200640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3bcc0742f0 0x7f3bcc19a1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:35:27.694 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:27.691+0000 7f3bcb200640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3bcc074c10 0x7f3bcc19a710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:35:27.695 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:27.691+0000 7f3bcb200640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3bcc10df90 0x7f3bcc19eaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:35:27.695 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:27.691+0000 7f3bcb200640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f3bcc113d30 con 0x7f3bcc074c10 2024-08-19T01:35:27.695 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:27.691+0000 7f3bcb200640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f3bcc113bb0 con 0x7f3bcc0742f0 2024-08-19T01:35:27.695 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:27.691+0000 7f3bcb200640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f3bcc113eb0 con 0x7f3bcc10df90 2024-08-19T01:35:27.695 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:27.691+0000 7f3bc9e00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3bcc0742f0 0x7f3bcc19a1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:35:27.695 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:27.691+0000 7f3bc9e00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3bcc0742f0 0x7f3bcc19a1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.83:39116/0 (socket says 172.21.15.83:39116) 2024-08-19T01:35:27.695 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:27.691+0000 7f3bc9e00640 1 -- 172.21.15.83:0/4020976177 learned_addr learned my addr 172.21.15.83:0/4020976177 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-08-19T01:35:27.695 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:27.691+0000 7f3bca800640 1 --2- 172.21.15.83:0/4020976177 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3bcc10df90 0x7f3bcc19eaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:35:27.695 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:27.691+0000 7f3bc9400640 1 --2- 172.21.15.83:0/4020976177 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3bcc074c10 0x7f3bcc19a710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:35:27.695 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:27.692+0000 7f3bc9e00640 1 -- 172.21.15.83:0/4020976177 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3bcc10df90 msgr2=0x7f3bcc19eaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:35:27.695 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:27.692+0000 7f3bc9e00640 1 --2- 172.21.15.83:0/4020976177 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3bcc10df90 0x7f3bcc19eaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:35:27.695 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:27.692+0000 7f3bc9e00640 1 -- 172.21.15.83:0/4020976177 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3bcc074c10 msgr2=0x7f3bcc19a710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:35:27.695 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:27.692+0000 7f3bc9e00640 1 --2- 172.21.15.83:0/4020976177 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3bcc074c10 0x7f3bcc19a710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:35:27.696 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:27.692+0000 7f3bc9e00640 1 -- 172.21.15.83:0/4020976177 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3bcc19f180 con 0x7f3bcc0742f0 2024-08-19T01:35:27.696 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:27.692+0000 7f3bca800640 1 --2- 172.21.15.83:0/4020976177 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3bcc10df90 0x7f3bcc19eaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-19T01:35:27.696 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:27.692+0000 7f3bc9e00640 1 --2- 172.21.15.83:0/4020976177 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3bcc0742f0 0x7f3bcc19a1d0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f3bbc00ea60 tx=0x7f3bbc00ef20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:35:27.696 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:27.693+0000 7f3bc2a00640 1 -- 172.21.15.83:0/4020976177 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3bbc00cde0 con 0x7f3bcc0742f0 2024-08-19T01:35:27.696 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:27.693+0000 7f3bc2a00640 1 -- 172.21.15.83:0/4020976177 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3bbc0035f0 con 0x7f3bcc0742f0 2024-08-19T01:35:27.696 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:27.693+0000 7f3bcb200640 1 -- 172.21.15.83:0/4020976177 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3bcc19f470 con 0x7f3bcc0742f0 2024-08-19T01:35:27.696 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:27.693+0000 7f3bc2a00640 1 -- 172.21.15.83:0/4020976177 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3bbc010670 con 0x7f3bcc0742f0 2024-08-19T01:35:27.696 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:27.693+0000 7f3bcb200640 1 -- 172.21.15.83:0/4020976177 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3bcc1a6d50 con 0x7f3bcc0742f0 2024-08-19T01:35:27.696 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:27.694+0000 7f3bcb200640 1 -- 172.21.15.83:0/4020976177 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3bcc0746d0 con 0x7f3bcc0742f0 2024-08-19T01:35:27.697 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:27.696+0000 7f3bc2a00640 1 -- 172.21.15.83:0/4020976177 <== mon.2 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 19) ==== 137884+0+0 (secure 0 0 0) 0x7f3bbc0037a0 con 0x7f3bcc0742f0 2024-08-19T01:35:27.698 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:27.697+0000 7f3bc2a00640 1 --2- 172.21.15.83:0/4020976177 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f3bb00a2de0 0x7f3bb00a52a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:35:27.698 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:27.697+0000 7f3bc2a00640 1 -- 172.21.15.83:0/4020976177 <== mon.2 v2:172.21.15.117:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8269+0+0 (secure 0 0 0) 0x7f3bbc0cf440 con 0x7f3bcc0742f0 2024-08-19T01:35:27.698 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:27.698+0000 7f3bc9400640 1 --2- 172.21.15.83:0/4020976177 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f3bb00a2de0 0x7f3bb00a52a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:35:27.699 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:27.698+0000 7f3bc9400640 1 --2- 172.21.15.83:0/4020976177 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f3bb00a2de0 0x7f3bb00a52a0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f3bb8000c00 tx=0x7f3bb8005b50 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:35:27.701 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:27.701+0000 7f3bc2a00640 1 -- 172.21.15.83:0/4020976177 <== mon.2 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7f3bbc09c1c0 con 0x7f3bcc0742f0 2024-08-19T01:35:27.922 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:27.921+0000 7f3bcb200640 1 -- 172.21.15.83:0/4020976177 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f3bcc19b530 con 0x7f3bcc0742f0 2024-08-19T01:35:27.922 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:27.922+0000 7f3bc2a00640 1 -- 172.21.15.83:0/4020976177 <== mon.2 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+11920 (secure 0 0 0) 0x7f3bbc09bac0 con 0x7f3bcc0742f0 2024-08-19T01:35:27.923 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-08-19T01:35:27.923 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":25,"btime":"2024-08-19T01:34:41:456953+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14421,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6835/988556067","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":988556067},{"type":"v1","addr":"172.21.15.83:6835","nonce":988556067}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14433,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6837/869702632","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6836","nonce":869702632},{"type":"v1","addr":"172.21.15.83:6837","nonce":869702632}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14475,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6841/1795975999","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6840","nonce":1795975999},{"type":"v1","addr":"172.21.15.83:6841","nonce":1795975999}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24430,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6833/2505076046","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6832","nonce":2505076046},{"type":"v1","addr":"172.21.15.167:6833","nonce":2505076046}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24449,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6833/3557520478","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":3557520478},{"type":"v1","addr":"172.21.15.117:6833","nonce":3557520478}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24466,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6835/3482045966","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":3482045966},{"type":"v1","addr":"172.21.15.167:6835","nonce":3482045966}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7}],"filesystems":[{"mdsmap":{"epoch":25,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-08-19T01:32:42.054769+0000","modified":"2024-08-19T01:34:40.528901+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24545,"mds_1":24527,"mds_2":24509},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24478":{"gid":24478,"name":"c","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.167:6837/1762130408","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6836","nonce":1762130408},{"type":"v1","addr":"172.21.15.167:6837","nonce":1762130408}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24485":{"gid":24485,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.117:6835/1024213994","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":1024213994},{"type":"v1","addr":"172.21.15.117:6835","nonce":1024213994}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24496":{"gid":24496,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.167:6839/3507387682","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6838","nonce":3507387682},{"type":"v1","addr":"172.21.15.167:6839","nonce":3507387682}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24509":{"gid":24509,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":10,"addr":"172.21.15.117:6837/3185224684","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":3185224684},{"type":"v1","addr":"172.21.15.117:6837","nonce":3185224684}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24527":{"gid":24527,"name":"g","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.83:6839/3239261845","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6838","nonce":3239261845},{"type":"v1","addr":"172.21.15.83:6839","nonce":3239261845}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24545":{"gid":24545,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.117:6839/1967595704","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6838","nonce":1967595704},{"type":"v1","addr":"172.21.15.117:6839","nonce":1967595704}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24545,"qdb_cluster":[24545,24527,24509]},"id":1}]} 2024-08-19T01:35:27.923 INFO:teuthology.orchestra.run.smithi083.stderr:dumped fsmap epoch 25 2024-08-19T01:35:27.925 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:27.925+0000 7f3bcb200640 1 -- 172.21.15.83:0/4020976177 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f3bb00a2de0 msgr2=0x7f3bb00a52a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:35:27.925 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:27.925+0000 7f3bcb200640 1 --2- 172.21.15.83:0/4020976177 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f3bb00a2de0 0x7f3bb00a52a0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f3bb8000c00 tx=0x7f3bb8005b50 comp rx=0 tx=0).stop 2024-08-19T01:35:27.925 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:27.925+0000 7f3bcb200640 1 -- 172.21.15.83:0/4020976177 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3bcc0742f0 msgr2=0x7f3bcc19a1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:35:27.925 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:27.925+0000 7f3bcb200640 1 --2- 172.21.15.83:0/4020976177 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3bcc0742f0 0x7f3bcc19a1d0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f3bbc00ea60 tx=0x7f3bbc00ef20 comp rx=0 tx=0).stop 2024-08-19T01:35:27.926 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:27.925+0000 7f3bcb200640 1 -- 172.21.15.83:0/4020976177 shutdown_connections 2024-08-19T01:35:27.926 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:27.925+0000 7f3bcb200640 1 --2- 172.21.15.83:0/4020976177 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f3bb00a2de0 0x7f3bb00a52a0 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:35:27.926 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:27.925+0000 7f3bcb200640 1 --2- 172.21.15.83:0/4020976177 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3bcc10df90 0x7f3bcc19eaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:35:27.926 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:27.925+0000 7f3bcb200640 1 --2- 172.21.15.83:0/4020976177 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3bcc074c10 0x7f3bcc19a710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:35:27.926 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:27.925+0000 7f3bcb200640 1 --2- 172.21.15.83:0/4020976177 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3bcc0742f0 0x7f3bcc19a1d0 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:35:27.926 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:27.925+0000 7f3bcb200640 1 -- 172.21.15.83:0/4020976177 >> 172.21.15.83:0/4020976177 conn(0x7f3bcc0fd470 msgr2=0x7f3bcc0fe170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:35:27.926 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:27.925+0000 7f3bcb200640 1 -- 172.21.15.83:0/4020976177 shutdown_connections 2024-08-19T01:35:27.926 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:27.925+0000 7f3bcb200640 1 -- 172.21.15.83:0/4020976177 wait complete. 2024-08-19T01:35:27.952 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:35:27 smithi117 ceph-mon[22016]: pgmap v357: 97 pgs: 97 active+clean; 667 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 1.1 KiB/s wr, 7 op/s 2024-08-19T01:35:27.966 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:35:27 smithi167 ceph-mon[21875]: pgmap v357: 97 pgs: 97 active+clean; 667 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 1.1 KiB/s wr, 7 op/s 2024-08-19T01:35:28.634 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:35:28 smithi083 ceph-mon[17120]: from='client.? 172.21.15.83:0/4020976177' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-19T01:35:28.695 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833 shell --fsid bad8780e-5dc9-11ef-bccf-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-08-19T01:35:28.952 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:35:28 smithi117 ceph-mon[22016]: from='client.? 172.21.15.83:0/4020976177' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-19T01:35:28.966 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:35:28 smithi167 ceph-mon[21875]: from='client.? 172.21.15.83:0/4020976177' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-19T01:35:29.295 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/bad8780e-5dc9-11ef-bccf-c7b262605968/mon.a/config 2024-08-19T01:35:29.779 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:35:29 smithi083 ceph-mon[17120]: pgmap v358: 97 pgs: 97 active+clean; 667 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 1.2 KiB/s wr, 6 op/s 2024-08-19T01:35:29.939 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:29.937+0000 7f6257200640 1 -- 172.21.15.83:0/2341795792 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6258109900 msgr2=0x7f6258109ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:35:29.939 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:29.937+0000 7f6257200640 1 --2- 172.21.15.83:0/2341795792 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6258109900 0x7f6258109ce0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f6248009930 tx=0x7f624802a420 comp rx=0 tx=0).stop 2024-08-19T01:35:29.939 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:29.937+0000 7f6257200640 1 -- 172.21.15.83:0/2341795792 shutdown_connections 2024-08-19T01:35:29.939 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:29.937+0000 7f6257200640 1 --2- 172.21.15.83:0/2341795792 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f62581042a0 0x7f62581107d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:35:29.939 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:29.937+0000 7f6257200640 1 --2- 172.21.15.83:0/2341795792 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6258103900 0x7f6258103d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:35:29.939 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:29.937+0000 7f6257200640 1 --2- 172.21.15.83:0/2341795792 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6258109900 0x7f6258109ce0 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:35:29.939 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:29.937+0000 7f6257200640 1 -- 172.21.15.83:0/2341795792 >> 172.21.15.83:0/2341795792 conn(0x7f62580ff640 msgr2=0x7f6258101a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:35:29.939 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:29.937+0000 7f6257200640 1 -- 172.21.15.83:0/2341795792 shutdown_connections 2024-08-19T01:35:29.939 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:29.937+0000 7f6257200640 1 -- 172.21.15.83:0/2341795792 wait complete. 2024-08-19T01:35:29.940 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:29.938+0000 7f6257200640 1 Processor -- start 2024-08-19T01:35:29.940 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:29.939+0000 7f6257200640 1 -- start start 2024-08-19T01:35:29.940 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:29.939+0000 7f6257200640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6258103900 0x7f6258110720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:35:29.940 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:29.939+0000 7f6257200640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f62581042a0 0x7f625810d7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:35:29.940 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:29.939+0000 7f6257200640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6258109900 0x7f625810ddc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:35:29.940 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:29.939+0000 7f6257200640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f6258113d50 con 0x7f6258109900 2024-08-19T01:35:29.941 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:29.939+0000 7f6257200640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f6258113bd0 con 0x7f6258103900 2024-08-19T01:35:29.941 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:29.939+0000 7f6257200640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f6258113ed0 con 0x7f62581042a0 2024-08-19T01:35:29.941 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:29.939+0000 7f6256800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6258109900 0x7f625810ddc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:35:29.941 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:29.939+0000 7f6255e00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6258103900 0x7f6258110720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:35:29.941 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:29.939+0000 7f6255e00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6258103900 0x7f6258110720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.83:39126/0 (socket says 172.21.15.83:39126) 2024-08-19T01:35:29.941 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:29.940+0000 7f6255e00640 1 -- 172.21.15.83:0/1374165693 learned_addr learned my addr 172.21.15.83:0/1374165693 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-08-19T01:35:29.941 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:29.940+0000 7f6255e00640 1 -- 172.21.15.83:0/1374165693 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f62581042a0 msgr2=0x7f625810d7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:35:29.942 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:29.940+0000 7f6255e00640 1 --2- 172.21.15.83:0/1374165693 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f62581042a0 0x7f625810d7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:35:29.942 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:29.940+0000 7f6255e00640 1 -- 172.21.15.83:0/1374165693 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6258109900 msgr2=0x7f625810ddc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:35:29.942 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:29.940+0000 7f6255e00640 1 --2- 172.21.15.83:0/1374165693 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6258109900 0x7f625810ddc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:35:29.942 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:29.940+0000 7f6255e00640 1 -- 172.21.15.83:0/1374165693 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f62581aaa50 con 0x7f6258103900 2024-08-19T01:35:29.942 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:29.941+0000 7f6255e00640 1 --2- 172.21.15.83:0/1374165693 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6258103900 0x7f6258110720 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f624802a940 tx=0x7f6248003b20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:35:29.942 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:29.941+0000 7f624ea00640 1 -- 172.21.15.83:0/1374165693 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f624803f070 con 0x7f6258103900 2024-08-19T01:35:29.942 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:29.941+0000 7f6257200640 1 -- 172.21.15.83:0/1374165693 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f62581aac20 con 0x7f6258103900 2024-08-19T01:35:29.942 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:29.941+0000 7f624ea00640 1 -- 172.21.15.83:0/1374165693 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6248031c90 con 0x7f6258103900 2024-08-19T01:35:29.943 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:29.941+0000 7f624ea00640 1 -- 172.21.15.83:0/1374165693 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6248035030 con 0x7f6258103900 2024-08-19T01:35:29.943 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:29.941+0000 7f6257200640 1 -- 172.21.15.83:0/1374165693 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f62581ab0d0 con 0x7f6258103900 2024-08-19T01:35:29.946 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:29.945+0000 7f625fa06740 1 Processor -- start 2024-08-19T01:35:29.946 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:29.946+0000 7f625fa06740 1 -- start start 2024-08-19T01:35:29.948 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:29.947+0000 7f625fa06740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55df5c6da6e0 0x55df5c6daac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:35:29.948 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:29.947+0000 7f625fa06740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55df5c6db000 0x55df5c6e5380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:35:29.948 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:29.947+0000 7f625fa06740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55df5c6e58c0 0x55df5c6e7cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:35:29.948 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:29.947+0000 7f625fa06740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55df5c63fbf0 con 0x55df5c6e58c0 2024-08-19T01:35:29.948 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:29.947+0000 7f625fa06740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55df5c5d7b20 con 0x55df5c6db000 2024-08-19T01:35:29.948 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:29.947+0000 7f625fa06740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55df5c63bbb0 con 0x55df5c6da6e0 2024-08-19T01:35:29.949 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:29.947+0000 7f6256800640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55df5c6da6e0 0x55df5c6daac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:35:29.949 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:29.947+0000 7f6255400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55df5c6e58c0 0x55df5c6e7cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:35:29.949 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:29.947+0000 7f6256800640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55df5c6da6e0 0x55df5c6daac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.83:46522/0 (socket says 172.21.15.83:46522) 2024-08-19T01:35:29.949 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:29.947+0000 7f6256800640 1 -- 172.21.15.83:0/2440188851 learned_addr learned my addr 172.21.15.83:0/2440188851 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-08-19T01:35:29.949 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:29.948+0000 7f6256800640 1 -- 172.21.15.83:0/2440188851 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55df5c6db000 msgr2=0x55df5c6e5380 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-08-19T01:35:29.949 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:29.948+0000 7f6256800640 1 --2- 172.21.15.83:0/2440188851 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55df5c6db000 0x55df5c6e5380 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:35:29.950 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:29.948+0000 7f624ea00640 1 -- 172.21.15.83:0/1374165693 <== mon.2 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 19) ==== 137884+0+0 (secure 0 0 0) 0x7f6248042070 con 0x7f6258103900 2024-08-19T01:35:29.950 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:29.948+0000 7f6256800640 1 -- 172.21.15.83:0/2440188851 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55df5c6e58c0 msgr2=0x55df5c6e7cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:35:29.950 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:29.948+0000 7f6256800640 1 --2- 172.21.15.83:0/2440188851 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55df5c6e58c0 0x55df5c6e7cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:35:29.950 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:29.948+0000 7f6256800640 1 -- 172.21.15.83:0/2440188851 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55df5c5de7f0 con 0x55df5c6da6e0 2024-08-19T01:35:29.950 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:29.948+0000 7f6255400640 1 --2- 172.21.15.83:0/2440188851 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55df5c6e58c0 0x55df5c6e7cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-19T01:35:29.950 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:29.948+0000 7f6256800640 1 --2- 172.21.15.83:0/2440188851 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55df5c6da6e0 0x55df5c6daac0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f6250009e10 tx=0x7f625000f730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=85eb1de08642d94e server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:35:29.950 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:29.949+0000 7f6243400640 1 -- 172.21.15.83:0/2440188851 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6250004d20 con 0x55df5c6da6e0 2024-08-19T01:35:29.950 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:29.949+0000 7f6243400640 1 -- 172.21.15.83:0/2440188851 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6250005500 con 0x55df5c6da6e0 2024-08-19T01:35:29.951 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:29.949+0000 7f6243400640 1 -- 172.21.15.83:0/2440188851 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6250004d20 con 0x55df5c6da6e0 2024-08-19T01:35:29.951 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:29.950+0000 7f624ea00640 1 --2- 172.21.15.83:0/1374165693 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f623c0a2d10 0x7f623c0a51d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:35:29.951 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:29.950+0000 7f624ea00640 1 -- 172.21.15.83:0/1374165693 <== mon.2 v2:172.21.15.117:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8269+0+0 (secure 0 0 0) 0x7f62480ef930 con 0x7f6258103900 2024-08-19T01:35:29.951 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:29.950+0000 7f625fa06740 1 -- 172.21.15.83:0/2440188851 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55df5c6da6e0 msgr2=0x55df5c6daac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:35:29.953 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:29.950+0000 7f625fa06740 1 --2- 172.21.15.83:0/2440188851 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55df5c6da6e0 0x55df5c6daac0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f6250009e10 tx=0x7f625000f730 comp rx=0 tx=0).stop 2024-08-19T01:35:29.953 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:29.951+0000 7f6255e00640 1 --2- 172.21.15.83:0/1374165693 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f623c0a2d10 0x7f623c0a51d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:35:29.953 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:29.951+0000 7f625fa06740 1 -- 172.21.15.83:0/2440188851 shutdown_connections 2024-08-19T01:35:29.954 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:29.951+0000 7f625fa06740 1 --2- 172.21.15.83:0/2440188851 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55df5c6e58c0 0x55df5c6e7cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:35:29.954 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:35:29 smithi117 ceph-mon[22016]: pgmap v358: 97 pgs: 97 active+clean; 667 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 1.2 KiB/s wr, 6 op/s 2024-08-19T01:35:29.955 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:29.951+0000 7f625fa06740 1 --2- 172.21.15.83:0/2440188851 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55df5c6db000 0x55df5c6e5380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:35:29.955 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:29.951+0000 7f625fa06740 1 --2- 172.21.15.83:0/2440188851 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55df5c6da6e0 0x55df5c6daac0 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:35:29.955 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:29.951+0000 7f625fa06740 1 -- 172.21.15.83:0/2440188851 >> 172.21.15.83:0/2440188851 conn(0x55df5c6d6c70 msgr2=0x55df5c6d7050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:35:29.955 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:29.952+0000 7f6255e00640 1 --2- 172.21.15.83:0/1374165693 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f623c0a2d10 0x7f623c0a51d0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f62480c46f0 tx=0x7f6248009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:35:29.955 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:29.952+0000 7f625fa06740 1 -- 172.21.15.83:0/2440188851 shutdown_connections 2024-08-19T01:35:29.955 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:29.952+0000 7f625fa06740 1 -- 172.21.15.83:0/2440188851 wait complete. 2024-08-19T01:35:29.955 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:29.953+0000 7f625fa06740 1 -- start start 2024-08-19T01:35:29.955 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:29.953+0000 7f625fa06740 1 Processor -- start 2024-08-19T01:35:29.956 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:29.954+0000 7f625fa06740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55df5c6da6e0 0x55df5c6e2780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:35:29.956 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:29.954+0000 7f625fa06740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55df5c6db000 0x55df5c6e2cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:35:29.956 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:29.954+0000 7f625fa06740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55df5c6e58c0 0x55df5c6e3200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:35:29.956 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:29.954+0000 7f625fa06740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55df5c6e8490 con 0x55df5c6da6e0 2024-08-19T01:35:29.956 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:29.954+0000 7f625fa06740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55df5c5facf0 con 0x55df5c6e58c0 2024-08-19T01:35:29.956 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:29.954+0000 7f625fa06740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55df5c6e8610 con 0x55df5c6db000 2024-08-19T01:35:29.957 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:29.954+0000 7f625fa06740 10 client.0 _ll_register_callbacks cb 0x55df5c5e6e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-19T01:35:29.957 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:29.954+0000 7f6255400640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55df5c6db000 0x55df5c6e2cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:35:29.957 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:29.955+0000 7f6255400640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55df5c6db000 0x55df5c6e2cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.83:46530/0 (socket says 172.21.15.83:46530) 2024-08-19T01:35:29.957 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:29.955+0000 7f6255400640 1 -- 172.21.15.83:0/903080678 learned_addr learned my addr 172.21.15.83:0/903080678 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-08-19T01:35:29.957 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:29.955+0000 7f6256800640 1 --2- 172.21.15.83:0/903080678 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55df5c6da6e0 0x55df5c6e2780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:35:29.958 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:29.955+0000 7f6255400640 1 -- 172.21.15.83:0/903080678 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55df5c6e58c0 msgr2=0x55df5c6e3200 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-08-19T01:35:29.958 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:29.955+0000 7f6255400640 1 --2- 172.21.15.83:0/903080678 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55df5c6e58c0 0x55df5c6e3200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:35:29.958 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:29.955+0000 7f6255400640 1 -- 172.21.15.83:0/903080678 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55df5c6da6e0 msgr2=0x55df5c6e2780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:35:29.958 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:29.955+0000 7f6255400640 1 --2- 172.21.15.83:0/903080678 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55df5c6da6e0 0x55df5c6e2780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:35:29.958 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:29.955+0000 7f6255400640 1 -- 172.21.15.83:0/903080678 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55df5c3b13c0 con 0x55df5c6db000 2024-08-19T01:35:29.958 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:29.955+0000 7f6256800640 1 --2- 172.21.15.83:0/903080678 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55df5c6da6e0 0x55df5c6e2780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-19T01:35:29.958 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:29.955+0000 7f6255400640 1 --2- 172.21.15.83:0/903080678 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55df5c6db000 0x55df5c6e2cc0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f62440105f0 tx=0x7f6244010ab0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=344edbbef93efb2a server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:35:29.958 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:29.955+0000 7f6237e00640 10 client.0 ms_handle_connect on v2:172.21.15.167:3300/0 2024-08-19T01:35:29.958 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:29.956+0000 7f6237e00640 1 -- 172.21.15.83:0/903080678 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6244012920 con 0x55df5c6db000 2024-08-19T01:35:29.958 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:29.956+0000 7f625fa06740 1 -- 172.21.15.83:0/903080678 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6250005500 con 0x55df5c6db000 2024-08-19T01:35:29.958 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:29.956+0000 7f6237e00640 1 -- 172.21.15.83:0/903080678 <== mon.1 v2:172.21.15.167:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8269+0+0 (secure 0 0 0) 0x7f6244012ac0 con 0x55df5c6db000 2024-08-19T01:35:29.958 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:29.956+0000 7f6237e00640 1 -- 172.21.15.83:0/903080678 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f6244012cc0 con 0x55df5c6db000 2024-08-19T01:35:29.958 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:29.956+0000 7f625fa06740 10 client.25207 fetch_fsmap learned FSMap version 25 2024-08-19T01:35:29.959 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:29.956+0000 7f625fa06740 1 -- 172.21.15.83:0/903080678 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55df5c6f4190 con 0x55df5c6db000 2024-08-19T01:35:29.959 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:29.957+0000 7f6237e00640 1 -- 172.21.15.83:0/903080678 <== mon.1 v2:172.21.15.167:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6250005500 con 0x55df5c6db000 2024-08-19T01:35:29.960 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:29.957+0000 7f6237e00640 1 -- 172.21.15.83:0/903080678 <== mon.1 v2:172.21.15.167:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f624400f040 con 0x55df5c6db000 2024-08-19T01:35:29.960 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:29.957+0000 7f625fa06740 10 client.25207 fetch_fsmap finished waiting for FSMap version 25 2024-08-19T01:35:29.960 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:29.957+0000 7f625fa06740 10 client.25207 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-08-19T01:35:29.960 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:29.957+0000 7f625fa06740 20 client.25207 populate_metadata read hostname 'smithi083' 2024-08-19T01:35:29.960 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:29.957+0000 7f625fa06740 1 --2- 172.21.15.83:0/903080678 >> [v2:172.21.15.117:6836/3185224684,v1:172.21.15.117:6837/3185224684] conn(0x55df5c6f5640 0x55df5c6f7a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:35:29.960 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:29.957+0000 7f625fa06740 4 client.25207 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-19T01:35:29.960 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:29.957+0000 7f625fa06740 1 -- 172.21.15.83:0/903080678 --> [v2:172.21.15.117:6836/3185224684,v1:172.21.15.117:6837/3185224684] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55df5c64d160 con 0x55df5c6f5640 2024-08-19T01:35:29.960 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:29.958+0000 7f6256800640 1 --2- 172.21.15.83:0/903080678 >> [v2:172.21.15.117:6836/3185224684,v1:172.21.15.117:6837/3185224684] conn(0x55df5c6f5640 0x55df5c6f7a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:35:29.960 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:29.958+0000 7f6256800640 1 --2- 172.21.15.83:0/903080678 >> [v2:172.21.15.117:6836/3185224684,v1:172.21.15.117:6837/3185224684] conn(0x55df5c6f5640 0x55df5c6f7a80 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=bf8e07a58cfaf9aa server_cookie=1aca6bc353a88e25 in_seq=0 out_seq=0 2024-08-19T01:35:29.960 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:29.958+0000 7f6237e00640 10 client.25207 ms_handle_connect on v2:172.21.15.117:6836/3185224684 2024-08-19T01:35:29.960 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:29.959+0000 7f6237e00640 1 -- 172.21.15.83:0/903080678 <== mds.2 v2:172.21.15.117:6836/3185224684 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55df5c64d160 con 0x55df5c6f5640 2024-08-19T01:35:29.960 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:29.959+0000 7f6237e00640 10 client.25207 handle_command_reply: tid=0 2024-08-19T01:35:29.960 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:29.959+0000 7f625fa06740 1 client.25207 shutdown 2024-08-19T01:35:29.961 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:29.959+0000 7f625fa06740 1 -- 172.21.15.83:0/903080678 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55df5c6db000 msgr2=0x55df5c6e2cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:35:29.961 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:29.959+0000 7f625fa06740 1 --2- 172.21.15.83:0/903080678 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55df5c6db000 0x55df5c6e2cc0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f62440105f0 tx=0x7f6244010ab0 comp rx=0 tx=0).stop 2024-08-19T01:35:29.961 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:29.960+0000 7f625fa06740 1 -- 172.21.15.83:0/903080678 shutdown_connections 2024-08-19T01:35:29.961 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:29.960+0000 7f625fa06740 1 --2- 172.21.15.83:0/903080678 >> [v2:172.21.15.117:6836/3185224684,v1:172.21.15.117:6837/3185224684] conn(0x55df5c6f5640 0x55df5c6f7a80 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:35:29.962 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:29.960+0000 7f625fa06740 1 --2- 172.21.15.83:0/903080678 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55df5c6e58c0 0x55df5c6e3200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:35:29.962 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:29.960+0000 7f625fa06740 1 --2- 172.21.15.83:0/903080678 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55df5c6db000 0x55df5c6e2cc0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:35:29.962 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:29.960+0000 7f625fa06740 1 --2- 172.21.15.83:0/903080678 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55df5c6da6e0 0x55df5c6e2780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:35:29.962 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:29.960+0000 7f625fa06740 1 -- 172.21.15.83:0/903080678 >> 172.21.15.83:0/903080678 conn(0x55df5c6d6c70 msgr2=0x55df5c6dc950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:35:29.962 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:29.960+0000 7f6237e00640 0 client.25207 ms_handle_reset on v2:172.21.15.117:6836/3185224684 2024-08-19T01:35:29.962 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:29.960+0000 7f625fa06740 1 -- 172.21.15.83:0/903080678 shutdown_connections 2024-08-19T01:35:29.962 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:29.960+0000 7f625fa06740 1 -- 172.21.15.83:0/903080678 wait complete. 2024-08-19T01:35:29.962 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:29.960+0000 7f625fa06740 20 client.25207 trim_cache size 0 max 16384 2024-08-19T01:35:29.965 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:35:29 smithi167 ceph-mon[21875]: pgmap v358: 97 pgs: 97 active+clean; 667 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 1.2 KiB/s wr, 6 op/s 2024-08-19T01:35:29.975 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:29.974+0000 7f625fa06740 1 Processor -- start 2024-08-19T01:35:29.975 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:29.974+0000 7f625fa06740 1 -- start start 2024-08-19T01:35:29.975 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:29.975+0000 7f625fa06740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55df5c6da6e0 0x55df5c6dd140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:35:29.975 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:29.975+0000 7f625fa06740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55df5c6db000 0x55df5c6d1910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:35:29.975 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:29.975+0000 7f625fa06740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55df5c6e58c0 0x55df5c6e3050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:35:29.976 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:29.975+0000 7f625fa06740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f62581b6240 con 0x55df5c6db000 2024-08-19T01:35:29.976 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:29.975+0000 7f625fa06740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f62581b3b10 con 0x55df5c6da6e0 2024-08-19T01:35:29.976 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:29.975+0000 7f625fa06740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f62581b1960 con 0x55df5c6e58c0 2024-08-19T01:35:29.977 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:29.975+0000 7f6256800640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55df5c6da6e0 0x55df5c6dd140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:35:29.977 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:29.975+0000 7f6256800640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55df5c6da6e0 0x55df5c6dd140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.83:39156/0 (socket says 172.21.15.83:39156) 2024-08-19T01:35:29.977 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:29.975+0000 7f6256800640 1 -- 172.21.15.83:0/241046104 learned_addr learned my addr 172.21.15.83:0/241046104 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-08-19T01:35:29.977 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:29.975+0000 7f6256800640 1 --2- 172.21.15.83:0/241046104 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55df5c6e58c0 0x55df5c6e3050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:35:29.977 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:29.975+0000 7f6255400640 1 --2- 172.21.15.83:0/241046104 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55df5c6db000 0x55df5c6d1910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:35:29.977 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:29.976+0000 7f6256800640 1 -- 172.21.15.83:0/241046104 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55df5c6e58c0 msgr2=0x55df5c6e3050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:35:29.977 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:29.976+0000 7f6256800640 1 --2- 172.21.15.83:0/241046104 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55df5c6e58c0 0x55df5c6e3050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:35:29.977 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:29.976+0000 7f6256800640 1 -- 172.21.15.83:0/241046104 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55df5c6db000 msgr2=0x55df5c6d1910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:35:29.977 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:29.976+0000 7f6256800640 1 --2- 172.21.15.83:0/241046104 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55df5c6db000 0x55df5c6d1910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:35:29.977 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:29.976+0000 7f6256800640 1 -- 172.21.15.83:0/241046104 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55df5c6e1730 con 0x55df5c6da6e0 2024-08-19T01:35:29.977 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:29.976+0000 7f6256800640 1 --2- 172.21.15.83:0/241046104 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55df5c6da6e0 0x55df5c6dd140 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f6250000f80 tx=0x7f6250009c10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1a119301874e9f53 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:35:29.977 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:29.976+0000 7f6242a00640 1 -- 172.21.15.83:0/241046104 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f625002a870 con 0x55df5c6da6e0 2024-08-19T01:35:29.977 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:29.976+0000 7f6242a00640 1 -- 172.21.15.83:0/241046104 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f625002aa10 con 0x55df5c6da6e0 2024-08-19T01:35:29.978 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:29.976+0000 7f6242a00640 1 -- 172.21.15.83:0/241046104 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6250028680 con 0x55df5c6da6e0 2024-08-19T01:35:29.978 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:29.977+0000 7f625fa06740 1 -- 172.21.15.83:0/241046104 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55df5c6da6e0 msgr2=0x55df5c6dd140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:35:29.979 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:29.977+0000 7f625fa06740 1 --2- 172.21.15.83:0/241046104 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55df5c6da6e0 0x55df5c6dd140 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f6250000f80 tx=0x7f6250009c10 comp rx=0 tx=0).stop 2024-08-19T01:35:29.979 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:29.977+0000 7f625fa06740 1 -- 172.21.15.83:0/241046104 shutdown_connections 2024-08-19T01:35:29.979 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:29.977+0000 7f625fa06740 1 --2- 172.21.15.83:0/241046104 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55df5c6e58c0 0x55df5c6e3050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:35:29.979 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:29.977+0000 7f625fa06740 1 --2- 172.21.15.83:0/241046104 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55df5c6db000 0x55df5c6d1910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:35:29.979 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:29.977+0000 7f625fa06740 1 --2- 172.21.15.83:0/241046104 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55df5c6da6e0 0x55df5c6dd140 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:35:29.979 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:29.977+0000 7f625fa06740 1 -- 172.21.15.83:0/241046104 >> 172.21.15.83:0/241046104 conn(0x55df5c6d6c70 msgr2=0x55df5c6e4800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:35:29.979 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:29.977+0000 7f625fa06740 1 -- 172.21.15.83:0/241046104 shutdown_connections 2024-08-19T01:35:29.979 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:29.977+0000 7f625fa06740 1 -- 172.21.15.83:0/241046104 wait complete. 2024-08-19T01:35:29.979 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:29.978+0000 7f625fa06740 1 -- start start 2024-08-19T01:35:29.979 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:29.978+0000 7f625fa06740 1 Processor -- start 2024-08-19T01:35:29.980 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:29.979+0000 7f625fa06740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55df5c6da6e0 0x55df5c6edf40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:35:29.980 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:29.979+0000 7f625fa06740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55df5c6db000 0x55df5c6df370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:35:29.980 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:29.979+0000 7f625fa06740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55df5c6e58c0 0x55df5c6df8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:35:29.980 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:29.979+0000 7f625fa06740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f62581b1fd0 con 0x55df5c6db000 2024-08-19T01:35:29.980 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:29.979+0000 7f625fa06740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f62581b3440 con 0x55df5c6e58c0 2024-08-19T01:35:29.980 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:29.979+0000 7f625fa06740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f62581b26a0 con 0x55df5c6da6e0 2024-08-19T01:35:29.980 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:29.979+0000 7f625fa06740 10 client.0 _ll_register_callbacks cb 0x7f6244009dd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-19T01:35:29.980 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:29.979+0000 7f6255400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55df5c6db000 0x55df5c6df370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:35:29.980 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:29.979+0000 7f6256800640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55df5c6da6e0 0x55df5c6edf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:35:29.981 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:29.979+0000 7f6256800640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55df5c6e58c0 0x55df5c6df8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:35:29.981 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:29.979+0000 7f6255400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55df5c6db000 0x55df5c6df370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:51690/0 (socket says 172.21.15.83:51690) 2024-08-19T01:35:29.982 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:29.979+0000 7f6255400640 1 -- 172.21.15.83:0/2692747571 learned_addr learned my addr 172.21.15.83:0/2692747571 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-08-19T01:35:29.982 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:29.980+0000 7f6255400640 1 -- 172.21.15.83:0/2692747571 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55df5c6da6e0 msgr2=0x55df5c6edf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:35:29.982 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:29.980+0000 7f6255400640 1 --2- 172.21.15.83:0/2692747571 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55df5c6da6e0 0x55df5c6edf40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:35:29.982 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:29.980+0000 7f6255400640 1 -- 172.21.15.83:0/2692747571 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55df5c6e58c0 msgr2=0x55df5c6df8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:35:29.982 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:29.980+0000 7f6255400640 1 --2- 172.21.15.83:0/2692747571 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55df5c6e58c0 0x55df5c6df8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:35:29.982 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:29.980+0000 7f6255400640 1 -- 172.21.15.83:0/2692747571 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55df5c6dae30 con 0x55df5c6db000 2024-08-19T01:35:29.982 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:29.980+0000 7f6256800640 1 --2- 172.21.15.83:0/2692747571 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55df5c6e58c0 0x55df5c6df8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-19T01:35:29.982 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:29.980+0000 7f6255400640 1 --2- 172.21.15.83:0/2692747571 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55df5c6db000 0x55df5c6df370 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f62440105c0 tx=0x7f6244011a10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b6b13b270082d6dc server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:35:29.982 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:29.981+0000 7f6237e00640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-08-19T01:35:29.982 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:29.981+0000 7f6237e00640 1 -- 172.21.15.83:0/2692747571 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6244032020 con 0x55df5c6db000 2024-08-19T01:35:29.983 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:29.981+0000 7f6237e00640 1 -- 172.21.15.83:0/2692747571 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8269+0+0 (secure 0 0 0) 0x7f6244010fb0 con 0x55df5c6db000 2024-08-19T01:35:29.983 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:29.981+0000 7f625fa06740 1 -- 172.21.15.83:0/2692747571 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f625002aa10 con 0x55df5c6db000 2024-08-19T01:35:29.983 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:29.981+0000 7f6237e00640 1 -- 172.21.15.83:0/2692747571 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f6244029070 con 0x55df5c6db000 2024-08-19T01:35:29.983 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:29.981+0000 7f625fa06740 10 client.15168 fetch_fsmap learned FSMap version 25 2024-08-19T01:35:29.983 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:29.981+0000 7f625fa06740 1 -- 172.21.15.83:0/2692747571 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55df5c7600c0 con 0x55df5c6db000 2024-08-19T01:35:29.983 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:29.982+0000 7f6237e00640 1 -- 172.21.15.83:0/2692747571 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f625002aa10 con 0x55df5c6db000 2024-08-19T01:35:29.983 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:29.982+0000 7f6237e00640 1 -- 172.21.15.83:0/2692747571 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f6244004830 con 0x55df5c6db000 2024-08-19T01:35:29.983 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:29.982+0000 7f625fa06740 10 client.15168 fetch_fsmap finished waiting for FSMap version 25 2024-08-19T01:35:29.984 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:29.982+0000 7f625fa06740 10 client.15168 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-08-19T01:35:29.984 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:29.982+0000 7f625fa06740 20 client.15168 populate_metadata read hostname 'smithi083' 2024-08-19T01:35:29.984 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:29.982+0000 7f625fa06740 1 --2- 172.21.15.83:0/2692747571 >> [v2:172.21.15.117:6836/3185224684,v1:172.21.15.117:6837/3185224684] conn(0x55df5c7608c0 0x55df5c762ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:35:29.984 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:29.982+0000 7f625fa06740 4 client.15168 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-08-19T01:35:29.984 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:29.982+0000 7f625fa06740 1 -- 172.21.15.83:0/2692747571 --> [v2:172.21.15.117:6836/3185224684,v1:172.21.15.117:6837/3185224684] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55df5c647670 con 0x55df5c7608c0 2024-08-19T01:35:29.984 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:29.982+0000 7f6256800640 1 --2- 172.21.15.83:0/2692747571 >> [v2:172.21.15.117:6836/3185224684,v1:172.21.15.117:6837/3185224684] conn(0x55df5c7608c0 0x55df5c762ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:35:29.985 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:29.983+0000 7f6237e00640 10 client.15168 ms_handle_connect on v2:172.21.15.117:6836/3185224684 2024-08-19T01:35:29.985 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:29.983+0000 7f6256800640 1 --2- 172.21.15.83:0/2692747571 >> [v2:172.21.15.117:6836/3185224684,v1:172.21.15.117:6837/3185224684] conn(0x55df5c7608c0 0x55df5c762ce0 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=77e3cd8c3a0a1356 server_cookie=d3574b83c2e3c96e in_seq=0 out_seq=0 2024-08-19T01:35:29.985 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:29.983+0000 7f6237e00640 1 -- 172.21.15.83:0/2692747571 <== mds.2 v2:172.21.15.117:6836/3185224684 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55df5c647670 con 0x55df5c7608c0 2024-08-19T01:35:29.985 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:29.983+0000 7f6237e00640 10 client.15168 handle_command_reply: tid=0 2024-08-19T01:35:29.985 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:29.983+0000 7f625fa06740 1 client.15168 shutdown 2024-08-19T01:35:29.985 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:29.983+0000 7f625fa06740 1 -- 172.21.15.83:0/2692747571 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55df5c6db000 msgr2=0x55df5c6df370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:35:29.986 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:29.983+0000 7f625fa06740 1 --2- 172.21.15.83:0/2692747571 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55df5c6db000 0x55df5c6df370 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f62440105c0 tx=0x7f6244011a10 comp rx=0 tx=0).stop 2024-08-19T01:35:29.986 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:29.984+0000 7f625fa06740 1 -- 172.21.15.83:0/2692747571 shutdown_connections 2024-08-19T01:35:29.986 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:29.984+0000 7f625fa06740 1 --2- 172.21.15.83:0/2692747571 >> [v2:172.21.15.117:6836/3185224684,v1:172.21.15.117:6837/3185224684] conn(0x55df5c7608c0 0x55df5c762ce0 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:35:29.986 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:29.984+0000 7f6237e00640 0 client.15168 ms_handle_reset on v2:172.21.15.117:6836/3185224684 2024-08-19T01:35:29.986 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:29.984+0000 7f625fa06740 1 --2- 172.21.15.83:0/2692747571 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55df5c6e58c0 0x55df5c6df8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:35:29.986 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:29.984+0000 7f625fa06740 1 --2- 172.21.15.83:0/2692747571 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55df5c6db000 0x55df5c6df370 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:35:29.986 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:29.984+0000 7f625fa06740 1 --2- 172.21.15.83:0/2692747571 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55df5c6da6e0 0x55df5c6edf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:35:29.986 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:29.984+0000 7f625fa06740 1 -- 172.21.15.83:0/2692747571 >> 172.21.15.83:0/2692747571 conn(0x55df5c6d6c70 msgr2=0x55df5c6d54a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:35:29.986 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:29.984+0000 7f625fa06740 1 -- 172.21.15.83:0/2692747571 shutdown_connections 2024-08-19T01:35:29.986 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:29.984+0000 7f625fa06740 1 -- 172.21.15.83:0/2692747571 wait complete. 2024-08-19T01:35:29.986 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:29.985+0000 7f625fa06740 20 client.15168 trim_cache size 0 max 16384 2024-08-19T01:35:29.987 INFO:teuthology.orchestra.run.smithi083.stdout:[] 2024-08-19T01:35:29.987 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:29.986+0000 7f6257200640 1 -- 172.21.15.83:0/1374165693 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f623c0a2d10 msgr2=0x7f623c0a51d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:35:29.987 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:29.986+0000 7f6257200640 1 --2- 172.21.15.83:0/1374165693 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f623c0a2d10 0x7f623c0a51d0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f62480c46f0 tx=0x7f6248009260 comp rx=0 tx=0).stop 2024-08-19T01:35:29.988 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:29.986+0000 7f6257200640 1 -- 172.21.15.83:0/1374165693 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6258103900 msgr2=0x7f6258110720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:35:29.988 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:29.986+0000 7f6257200640 1 --2- 172.21.15.83:0/1374165693 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6258103900 0x7f6258110720 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f624802a940 tx=0x7f6248003b20 comp rx=0 tx=0).stop 2024-08-19T01:35:29.988 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:29.986+0000 7f6257200640 1 -- 172.21.15.83:0/1374165693 shutdown_connections 2024-08-19T01:35:29.988 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:29.986+0000 7f6257200640 1 --2- 172.21.15.83:0/1374165693 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f623c0a2d10 0x7f623c0a51d0 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:35:29.988 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:29.986+0000 7f6257200640 1 --2- 172.21.15.83:0/1374165693 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6258109900 0x7f625810ddc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:35:29.988 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:29.986+0000 7f6257200640 1 --2- 172.21.15.83:0/1374165693 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f62581042a0 0x7f625810d7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:35:29.988 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:29.986+0000 7f6257200640 1 --2- 172.21.15.83:0/1374165693 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6258103900 0x7f6258110720 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:35:29.988 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:29.986+0000 7f6257200640 1 -- 172.21.15.83:0/1374165693 >> 172.21.15.83:0/1374165693 conn(0x7f62580ff640 msgr2=0x7f62580fff80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:35:29.988 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:29.986+0000 7f6257200640 1 -- 172.21.15.83:0/1374165693 shutdown_connections 2024-08-19T01:35:29.988 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:29.986+0000 7f6257200640 1 -- 172.21.15.83:0/1374165693 wait complete. 2024-08-19T01:35:30.489 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:30.489+0000 7fd771e00640 1 -- 172.21.15.83:0/681759444 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd76c102830 msgr2=0x7fd76c104ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:35:30.490 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:30.489+0000 7fd771e00640 1 --2- 172.21.15.83:0/681759444 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd76c102830 0x7fd76c104ca0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fd758009900 tx=0x7fd75802a3f0 comp rx=0 tx=0).stop 2024-08-19T01:35:30.490 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:30.489+0000 7fd771e00640 1 -- 172.21.15.83:0/681759444 shutdown_connections 2024-08-19T01:35:30.490 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:30.489+0000 7fd771e00640 1 --2- 172.21.15.83:0/681759444 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd76c1051e0 0x7fd76c105690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:35:30.490 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:30.489+0000 7fd771e00640 1 --2- 172.21.15.83:0/681759444 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd76c102830 0x7fd76c104ca0 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:35:30.490 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:30.489+0000 7fd771e00640 1 --2- 172.21.15.83:0/681759444 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd76c0ffee0 0x7fd76c1022f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:35:30.490 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:30.489+0000 7fd771e00640 1 -- 172.21.15.83:0/681759444 >> 172.21.15.83:0/681759444 conn(0x7fd76c0fbcd0 msgr2=0x7fd76c0fe110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:35:30.490 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:30.489+0000 7fd771e00640 1 -- 172.21.15.83:0/681759444 shutdown_connections 2024-08-19T01:35:30.490 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:30.489+0000 7fd771e00640 1 -- 172.21.15.83:0/681759444 wait complete. 2024-08-19T01:35:30.491 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:30.490+0000 7fd771e00640 1 Processor -- start 2024-08-19T01:35:30.491 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:30.490+0000 7fd771e00640 1 -- start start 2024-08-19T01:35:30.491 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:30.490+0000 7fd771e00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd76c0ffee0 0x7fd76c19dfc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:35:30.492 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:30.490+0000 7fd771e00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd76c102830 0x7fd76c19e500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:35:30.492 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:30.490+0000 7fd771e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd76c1051e0 0x7fd76c1a5570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:35:30.492 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:30.490+0000 7fd771e00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fd76c113930 con 0x7fd76c1051e0 2024-08-19T01:35:30.492 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:30.490+0000 7fd771e00640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fd76c1137b0 con 0x7fd76c102830 2024-08-19T01:35:30.492 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:30.490+0000 7fd771e00640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fd76c113ab0 con 0x7fd76c0ffee0 2024-08-19T01:35:30.492 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:30.491+0000 7fd770a00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd76c0ffee0 0x7fd76c19dfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:35:30.492 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:30.491+0000 7fd770a00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd76c0ffee0 0x7fd76c19dfc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.83:46574/0 (socket says 172.21.15.83:46574) 2024-08-19T01:35:30.492 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:30.491+0000 7fd770a00640 1 -- 172.21.15.83:0/1738376816 learned_addr learned my addr 172.21.15.83:0/1738376816 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-08-19T01:35:30.492 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:30.491+0000 7fd76be00640 1 --2- 172.21.15.83:0/1738376816 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd76c102830 0x7fd76c19e500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:35:30.492 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:30.491+0000 7fd771400640 1 --2- 172.21.15.83:0/1738376816 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd76c1051e0 0x7fd76c1a5570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:35:30.493 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:30.491+0000 7fd770a00640 1 -- 172.21.15.83:0/1738376816 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd76c102830 msgr2=0x7fd76c19e500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:35:30.493 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:30.491+0000 7fd770a00640 1 --2- 172.21.15.83:0/1738376816 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd76c102830 0x7fd76c19e500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:35:30.493 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:30.491+0000 7fd770a00640 1 -- 172.21.15.83:0/1738376816 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd76c1051e0 msgr2=0x7fd76c1a5570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:35:30.493 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:30.491+0000 7fd770a00640 1 --2- 172.21.15.83:0/1738376816 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd76c1051e0 0x7fd76c1a5570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:35:30.493 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:30.491+0000 7fd770a00640 1 -- 172.21.15.83:0/1738376816 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd76c1a5be0 con 0x7fd76c0ffee0 2024-08-19T01:35:30.494 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:30.491+0000 7fd770a00640 1 --2- 172.21.15.83:0/1738376816 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd76c0ffee0 0x7fd76c19dfc0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fd76000e9f0 tx=0x7fd76000edf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:35:30.494 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:30.492+0000 7fd769600640 1 -- 172.21.15.83:0/1738376816 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd76000cd80 con 0x7fd76c0ffee0 2024-08-19T01:35:30.494 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:30.492+0000 7fd769600640 1 -- 172.21.15.83:0/1738376816 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd760005500 con 0x7fd76c0ffee0 2024-08-19T01:35:30.494 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:30.492+0000 7fd771e00640 1 -- 172.21.15.83:0/1738376816 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd76c1a5ed0 con 0x7fd76c0ffee0 2024-08-19T01:35:30.494 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:30.492+0000 7fd769600640 1 -- 172.21.15.83:0/1738376816 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd760010690 con 0x7fd76c0ffee0 2024-08-19T01:35:30.494 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:30.492+0000 7fd771e00640 1 -- 172.21.15.83:0/1738376816 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd76c1a62e0 con 0x7fd76c0ffee0 2024-08-19T01:35:30.497 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:30.496+0000 7fd769600640 1 -- 172.21.15.83:0/1738376816 <== mon.1 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 19) ==== 137884+0+0 (secure 0 0 0) 0x7fd7600046c0 con 0x7fd76c0ffee0 2024-08-19T01:35:30.497 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:30.496+0000 7fd776209740 1 Processor -- start 2024-08-19T01:35:30.497 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:30.497+0000 7fd776209740 1 -- start start 2024-08-19T01:35:30.498 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:30.497+0000 7fd776209740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557d4b4e0680 0x557d4b4e0a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:35:30.498 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:30.497+0000 7fd776209740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557d4b4e0fc0 0x557d4b4e5a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:35:30.498 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:30.497+0000 7fd776209740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557d4b4e6050 0x557d4b4e8510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:35:30.498 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:30.497+0000 7fd776209740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x557d4b444890 con 0x557d4b4e6050 2024-08-19T01:35:30.498 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:30.497+0000 7fd776209740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x557d4b3dcd90 con 0x557d4b4e0fc0 2024-08-19T01:35:30.498 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:30.497+0000 7fd776209740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x557d4b440850 con 0x557d4b4e0680 2024-08-19T01:35:30.499 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:30.498+0000 7fd771400640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557d4b4e0680 0x557d4b4e0a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:35:30.499 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:30.498+0000 7fd771400640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557d4b4e0680 0x557d4b4e0a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.83:46582/0 (socket says 172.21.15.83:46582) 2024-08-19T01:35:30.499 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:30.498+0000 7fd771400640 1 -- 172.21.15.83:0/3235418913 learned_addr learned my addr 172.21.15.83:0/3235418913 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-08-19T01:35:30.499 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:30.498+0000 7fd769600640 1 --2- 172.21.15.83:0/1738376816 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7fd7540a2c10 0x7fd7540a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:35:30.499 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:30.498+0000 7fd770a00640 1 --2- 172.21.15.83:0/3235418913 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557d4b4e0fc0 0x557d4b4e5a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:35:30.499 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:30.499+0000 7fd771400640 1 --2- 172.21.15.83:0/1738376816 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7fd7540a2c10 0x7fd7540a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:35:30.499 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:30.499+0000 7fd769600640 1 -- 172.21.15.83:0/1738376816 <== mon.1 v2:172.21.15.167:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8269+0+0 (secure 0 0 0) 0x7fd7600cf610 con 0x7fd76c0ffee0 2024-08-19T01:35:30.499 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:30.499+0000 7fd770a00640 1 -- 172.21.15.83:0/3235418913 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557d4b4e0680 msgr2=0x557d4b4e0a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:35:30.500 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:30.499+0000 7fd770a00640 1 --2- 172.21.15.83:0/3235418913 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557d4b4e0680 0x557d4b4e0a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:35:30.500 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:30.499+0000 7fd770a00640 1 -- 172.21.15.83:0/3235418913 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557d4b4e6050 msgr2=0x557d4b4e8510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:35:30.500 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:30.499+0000 7fd770a00640 1 --2- 172.21.15.83:0/3235418913 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557d4b4e6050 0x557d4b4e8510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:35:30.500 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:30.499+0000 7fd770a00640 1 -- 172.21.15.83:0/3235418913 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557d4b158210 con 0x557d4b4e0fc0 2024-08-19T01:35:30.500 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:30.499+0000 7fd770a00640 1 --2- 172.21.15.83:0/3235418913 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557d4b4e0fc0 0x557d4b4e5a80 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fd7600ccfa0 tx=0x7fd760004a40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f1439713d3dcb165 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:35:30.500 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:30.499+0000 7fd75dc00640 1 -- 172.21.15.83:0/3235418913 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7600a4960 con 0x557d4b4e0fc0 2024-08-19T01:35:30.501 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:30.499+0000 7fd75dc00640 1 -- 172.21.15.83:0/3235418913 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd76009c3f0 con 0x557d4b4e0fc0 2024-08-19T01:35:30.501 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:30.499+0000 7fd75dc00640 1 -- 172.21.15.83:0/3235418913 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd76009c1b0 con 0x557d4b4e0fc0 2024-08-19T01:35:30.501 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:30.500+0000 7fd776209740 1 -- 172.21.15.83:0/3235418913 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557d4b4e0fc0 msgr2=0x557d4b4e5a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:35:30.502 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:30.500+0000 7fd776209740 1 --2- 172.21.15.83:0/3235418913 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557d4b4e0fc0 0x557d4b4e5a80 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fd7600ccfa0 tx=0x7fd760004a40 comp rx=0 tx=0).stop 2024-08-19T01:35:30.502 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:30.500+0000 7fd771400640 1 --2- 172.21.15.83:0/1738376816 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7fd7540a2c10 0x7fd7540a50d0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fd764009fd0 tx=0x7fd764004ab0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:35:30.502 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:30.501+0000 7fd776209740 1 -- 172.21.15.83:0/3235418913 shutdown_connections 2024-08-19T01:35:30.502 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:30.501+0000 7fd776209740 1 --2- 172.21.15.83:0/3235418913 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557d4b4e6050 0x557d4b4e8510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:35:30.502 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:30.501+0000 7fd776209740 1 --2- 172.21.15.83:0/3235418913 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557d4b4e0fc0 0x557d4b4e5a80 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:35:30.502 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:30.501+0000 7fd776209740 1 --2- 172.21.15.83:0/3235418913 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557d4b4e0680 0x557d4b4e0a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:35:30.502 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:30.501+0000 7fd776209740 1 -- 172.21.15.83:0/3235418913 >> 172.21.15.83:0/3235418913 conn(0x557d4b4dbcf0 msgr2=0x557d4b4dc0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:35:30.502 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:30.501+0000 7fd776209740 1 -- 172.21.15.83:0/3235418913 shutdown_connections 2024-08-19T01:35:30.502 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:30.501+0000 7fd776209740 1 -- 172.21.15.83:0/3235418913 wait complete. 2024-08-19T01:35:30.503 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:30.502+0000 7fd776209740 1 -- start start 2024-08-19T01:35:30.503 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:30.502+0000 7fd776209740 1 Processor -- start 2024-08-19T01:35:30.504 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:30.503+0000 7fd776209740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557d4b4e0680 0x557d4b4ea9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:35:30.504 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:30.503+0000 7fd776209740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557d4b4e0fc0 0x557d4b4eaee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:35:30.504 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:30.503+0000 7fd776209740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557d4b4e6050 0x557d4b4f1f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:35:30.504 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:30.503+0000 7fd776209740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x557d4b4e9000 con 0x557d4b4e0680 2024-08-19T01:35:30.504 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:30.503+0000 7fd776209740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x557d4b3fff70 con 0x557d4b4e0fc0 2024-08-19T01:35:30.504 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:30.503+0000 7fd776209740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x557d4b4e9180 con 0x557d4b4e6050 2024-08-19T01:35:30.504 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:30.503+0000 7fd776209740 10 client.0 _ll_register_callbacks cb 0x557d4b43c8d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-19T01:35:30.504 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:30.503+0000 7fd771400640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557d4b4e6050 0x557d4b4f1f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:35:30.505 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:30.503+0000 7fd771400640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557d4b4e6050 0x557d4b4f1f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.83:46586/0 (socket says 172.21.15.83:46586) 2024-08-19T01:35:30.505 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:30.503+0000 7fd771400640 1 -- 172.21.15.83:0/4110371822 learned_addr learned my addr 172.21.15.83:0/4110371822 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-08-19T01:35:30.505 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:30.504+0000 7fd76be00640 1 --2- 172.21.15.83:0/4110371822 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557d4b4e0fc0 0x557d4b4eaee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:35:30.505 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:30.504+0000 7fd771400640 1 -- 172.21.15.83:0/4110371822 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557d4b4e0fc0 msgr2=0x557d4b4eaee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:35:30.505 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:30.504+0000 7fd771400640 1 --2- 172.21.15.83:0/4110371822 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557d4b4e0fc0 0x557d4b4eaee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:35:30.506 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:30.504+0000 7fd771400640 1 -- 172.21.15.83:0/4110371822 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557d4b4e0680 msgr2=0x557d4b4ea9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:35:30.506 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:30.504+0000 7fd770a00640 1 --2- 172.21.15.83:0/4110371822 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557d4b4e0680 0x557d4b4ea9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:35:30.506 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:30.504+0000 7fd771400640 1 --2- 172.21.15.83:0/4110371822 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557d4b4e0680 0x557d4b4ea9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:35:30.506 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:30.504+0000 7fd771400640 1 -- 172.21.15.83:0/4110371822 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557d4b224620 con 0x557d4b4e6050 2024-08-19T01:35:30.506 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:30.504+0000 7fd770a00640 1 --2- 172.21.15.83:0/4110371822 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557d4b4e0680 0x557d4b4ea9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-19T01:35:30.506 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:30.505+0000 7fd771400640 1 --2- 172.21.15.83:0/4110371822 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557d4b4e6050 0x557d4b4f1f20 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fd764013fa0 tx=0x7fd764004a40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=111f4540f37a9b01 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:35:30.506 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:30.505+0000 7fd752000640 10 client.0 ms_handle_connect on v2:172.21.15.167:3300/0 2024-08-19T01:35:30.508 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:30.507+0000 7fd752000640 1 -- 172.21.15.83:0/4110371822 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd764004b20 con 0x557d4b4e6050 2024-08-19T01:35:30.508 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:30.508+0000 7fd752000640 1 -- 172.21.15.83:0/4110371822 <== mon.1 v2:172.21.15.167:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8269+0+0 (secure 0 0 0) 0x7fd76401c070 con 0x557d4b4e6050 2024-08-19T01:35:30.508 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:30.508+0000 7fd776209740 1 -- 172.21.15.83:0/4110371822 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd76009c3f0 con 0x557d4b4e6050 2024-08-19T01:35:30.509 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:30.508+0000 7fd752000640 1 -- 172.21.15.83:0/4110371822 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fd764004cc0 con 0x557d4b4e6050 2024-08-19T01:35:30.509 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:30.508+0000 7fd776209740 10 client.25225 fetch_fsmap learned FSMap version 25 2024-08-19T01:35:30.509 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:30.508+0000 7fd776209740 1 -- 172.21.15.83:0/4110371822 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x557d4b506170 con 0x557d4b4e6050 2024-08-19T01:35:30.510 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:30.509+0000 7fd752000640 1 -- 172.21.15.83:0/4110371822 <== mon.1 v2:172.21.15.167:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd76009c3f0 con 0x557d4b4e6050 2024-08-19T01:35:30.510 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:30.509+0000 7fd752000640 1 -- 172.21.15.83:0/4110371822 <== mon.1 v2:172.21.15.167:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fd7640053d0 con 0x557d4b4e6050 2024-08-19T01:35:30.510 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:30.509+0000 7fd776209740 10 client.25225 fetch_fsmap finished waiting for FSMap version 25 2024-08-19T01:35:30.510 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:30.509+0000 7fd776209740 10 client.25225 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-08-19T01:35:30.510 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:30.509+0000 7fd776209740 20 client.25225 populate_metadata read hostname 'smithi083' 2024-08-19T01:35:30.510 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:30.509+0000 7fd776209740 1 --2- 172.21.15.83:0/4110371822 >> [v2:172.21.15.117:6838/1967595704,v1:172.21.15.117:6839/1967595704] conn(0x557d4b507870 0x557d4b509ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:35:30.510 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:30.509+0000 7fd776209740 4 client.25225 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-19T01:35:30.510 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:30.509+0000 7fd776209740 1 -- 172.21.15.83:0/4110371822 --> [v2:172.21.15.117:6838/1967595704,v1:172.21.15.117:6839/1967595704] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557d4b451fc0 con 0x557d4b507870 2024-08-19T01:35:30.510 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:30.509+0000 7fd770a00640 1 --2- 172.21.15.83:0/4110371822 >> [v2:172.21.15.117:6838/1967595704,v1:172.21.15.117:6839/1967595704] conn(0x557d4b507870 0x557d4b509ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:35:30.511 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:30.510+0000 7fd752000640 10 client.25225 ms_handle_connect on v2:172.21.15.117:6838/1967595704 2024-08-19T01:35:30.511 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:30.510+0000 7fd770a00640 1 --2- 172.21.15.83:0/4110371822 >> [v2:172.21.15.117:6838/1967595704,v1:172.21.15.117:6839/1967595704] conn(0x557d4b507870 0x557d4b509ca0 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7c1137c8ac8a118 server_cookie=3c6f255d584066d1 in_seq=0 out_seq=0 2024-08-19T01:35:30.511 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:30.511+0000 7fd752000640 1 -- 172.21.15.83:0/4110371822 <== mds.0 v2:172.21.15.117:6838/1967595704 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x557d4b451fc0 con 0x557d4b507870 2024-08-19T01:35:30.511 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:30.511+0000 7fd752000640 10 client.25225 handle_command_reply: tid=0 2024-08-19T01:35:30.511 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:30.511+0000 7fd776209740 1 client.25225 shutdown 2024-08-19T01:35:30.512 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:30.511+0000 7fd776209740 1 -- 172.21.15.83:0/4110371822 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557d4b4e6050 msgr2=0x557d4b4f1f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:35:30.512 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:30.511+0000 7fd776209740 1 --2- 172.21.15.83:0/4110371822 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557d4b4e6050 0x557d4b4f1f20 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fd764013fa0 tx=0x7fd764004a40 comp rx=0 tx=0).stop 2024-08-19T01:35:30.512 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:30.511+0000 7fd776209740 1 -- 172.21.15.83:0/4110371822 shutdown_connections 2024-08-19T01:35:30.513 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:30.511+0000 7fd776209740 1 --2- 172.21.15.83:0/4110371822 >> [v2:172.21.15.117:6838/1967595704,v1:172.21.15.117:6839/1967595704] conn(0x557d4b507870 0x557d4b509ca0 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:35:30.513 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:30.511+0000 7fd776209740 1 --2- 172.21.15.83:0/4110371822 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557d4b4e6050 0x557d4b4f1f20 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:35:30.513 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:30.512+0000 7fd776209740 1 --2- 172.21.15.83:0/4110371822 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557d4b4e0fc0 0x557d4b4eaee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:35:30.513 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:30.512+0000 7fd752000640 0 client.25225 ms_handle_reset on v2:172.21.15.117:6838/1967595704 2024-08-19T01:35:30.513 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:30.512+0000 7fd776209740 1 --2- 172.21.15.83:0/4110371822 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557d4b4e0680 0x557d4b4ea9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:35:30.513 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:30.512+0000 7fd776209740 1 -- 172.21.15.83:0/4110371822 >> 172.21.15.83:0/4110371822 conn(0x557d4b4dbcf0 msgr2=0x557d4b4dd500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:35:30.513 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:30.512+0000 7fd776209740 1 -- 172.21.15.83:0/4110371822 shutdown_connections 2024-08-19T01:35:30.513 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:30.512+0000 7fd776209740 1 -- 172.21.15.83:0/4110371822 wait complete. 2024-08-19T01:35:30.513 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:30.512+0000 7fd776209740 20 client.25225 trim_cache size 0 max 16384 2024-08-19T01:35:30.528 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:30.527+0000 7fd776209740 1 Processor -- start 2024-08-19T01:35:30.528 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:30.527+0000 7fd776209740 1 -- start start 2024-08-19T01:35:30.528 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:30.527+0000 7fd776209740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557d4b4e0680 0x557d4b4dae30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:35:30.528 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:30.527+0000 7fd776209740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557d4b4e0fc0 0x557d4b4f1e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:35:30.529 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:30.527+0000 7fd776209740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557d4b4e6050 0x557d4b4dd0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:35:30.529 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:30.527+0000 7fd776209740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fd76c1a92b0 con 0x557d4b4e6050 2024-08-19T01:35:30.529 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:30.527+0000 7fd776209740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fd76c1a6b80 con 0x557d4b4e0680 2024-08-19T01:35:30.529 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:30.527+0000 7fd776209740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fd76c072570 con 0x557d4b4e0fc0 2024-08-19T01:35:30.529 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:30.528+0000 7fd770a00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557d4b4e0680 0x557d4b4dae30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:35:30.529 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:30.528+0000 7fd770a00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557d4b4e0680 0x557d4b4dae30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.83:39210/0 (socket says 172.21.15.83:39210) 2024-08-19T01:35:30.529 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:30.528+0000 7fd770a00640 1 -- 172.21.15.83:0/2903947656 learned_addr learned my addr 172.21.15.83:0/2903947656 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-08-19T01:35:30.529 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:30.528+0000 7fd76be00640 1 --2- 172.21.15.83:0/2903947656 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557d4b4e0fc0 0x557d4b4f1e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:35:30.529 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:30.528+0000 7fd771400640 1 --2- 172.21.15.83:0/2903947656 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557d4b4e6050 0x557d4b4dd0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:35:30.529 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:30.528+0000 7fd770a00640 1 -- 172.21.15.83:0/2903947656 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557d4b4e0fc0 msgr2=0x557d4b4f1e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:35:30.529 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:30.528+0000 7fd770a00640 1 --2- 172.21.15.83:0/2903947656 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557d4b4e0fc0 0x557d4b4f1e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:35:30.530 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:30.528+0000 7fd770a00640 1 -- 172.21.15.83:0/2903947656 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557d4b4e6050 msgr2=0x557d4b4dd0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:35:30.530 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:30.528+0000 7fd770a00640 1 --2- 172.21.15.83:0/2903947656 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557d4b4e6050 0x557d4b4dd0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:35:30.530 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:30.528+0000 7fd770a00640 1 -- 172.21.15.83:0/2903947656 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557d4b4d77e0 con 0x557d4b4e0680 2024-08-19T01:35:30.530 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:30.528+0000 7fd76be00640 1 --2- 172.21.15.83:0/2903947656 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557d4b4e0fc0 0x557d4b4f1e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-19T01:35:30.530 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:30.528+0000 7fd771400640 1 --2- 172.21.15.83:0/2903947656 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557d4b4e6050 0x557d4b4dd0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-19T01:35:30.530 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:30.528+0000 7fd770a00640 1 --2- 172.21.15.83:0/2903947656 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557d4b4e0680 0x557d4b4dae30 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fd760098150 tx=0x7fd7600d4040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=dc10523276cb0c51 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:35:30.532 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:30.529+0000 7fd75d200640 1 -- 172.21.15.83:0/2903947656 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7600a00a0 con 0x557d4b4e0680 2024-08-19T01:35:30.532 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:30.529+0000 7fd75d200640 1 -- 172.21.15.83:0/2903947656 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x557d4b451fc0 con 0x557d4b4e0680 2024-08-19T01:35:30.532 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:30.529+0000 7fd75d200640 1 -- 172.21.15.83:0/2903947656 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd76009c1b0 con 0x557d4b4e0680 2024-08-19T01:35:30.532 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:30.529+0000 7fd776209740 1 -- 172.21.15.83:0/2903947656 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557d4b4e0680 msgr2=0x557d4b4dae30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:35:30.533 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:30.529+0000 7fd776209740 1 --2- 172.21.15.83:0/2903947656 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557d4b4e0680 0x557d4b4dae30 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fd760098150 tx=0x7fd7600d4040 comp rx=0 tx=0).stop 2024-08-19T01:35:30.533 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:30.530+0000 7fd776209740 1 -- 172.21.15.83:0/2903947656 shutdown_connections 2024-08-19T01:35:30.533 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:30.530+0000 7fd776209740 1 --2- 172.21.15.83:0/2903947656 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557d4b4e6050 0x557d4b4dd0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:35:30.533 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:30.530+0000 7fd776209740 1 --2- 172.21.15.83:0/2903947656 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557d4b4e0fc0 0x557d4b4f1e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:35:30.533 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:30.530+0000 7fd776209740 1 --2- 172.21.15.83:0/2903947656 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557d4b4e0680 0x557d4b4dae30 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:35:30.533 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:30.530+0000 7fd776209740 1 -- 172.21.15.83:0/2903947656 >> 172.21.15.83:0/2903947656 conn(0x557d4b4dbcf0 msgr2=0x557d4b4da720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:35:30.533 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:30.530+0000 7fd776209740 1 -- 172.21.15.83:0/2903947656 shutdown_connections 2024-08-19T01:35:30.533 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:30.530+0000 7fd776209740 1 -- 172.21.15.83:0/2903947656 wait complete. 2024-08-19T01:35:30.533 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:30.530+0000 7fd776209740 1 -- start start 2024-08-19T01:35:30.533 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:30.530+0000 7fd776209740 1 Processor -- start 2024-08-19T01:35:30.533 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:30.531+0000 7fd776209740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557d4b4e0680 0x557d4b4eed80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:35:30.533 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:30.531+0000 7fd776209740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557d4b4e0fc0 0x557d4b4ef2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:35:30.533 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:30.531+0000 7fd776209740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557d4b4e6050 0x557d4b570570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:35:30.534 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:30.531+0000 7fd776209740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fd76c072c20 con 0x557d4b4e6050 2024-08-19T01:35:30.534 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:30.531+0000 7fd776209740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fd76c1a6570 con 0x557d4b4e0680 2024-08-19T01:35:30.534 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:30.531+0000 7fd776209740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fd76c0732f0 con 0x557d4b4e0fc0 2024-08-19T01:35:30.534 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:30.531+0000 7fd776209740 10 client.0 _ll_register_callbacks cb 0x7fd7640135c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-19T01:35:30.534 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:30.531+0000 7fd770a00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557d4b4e0680 0x557d4b4eed80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:35:30.534 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:30.531+0000 7fd771400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557d4b4e6050 0x557d4b570570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:35:30.535 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:30.531+0000 7fd770a00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557d4b4e0680 0x557d4b4eed80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.83:39212/0 (socket says 172.21.15.83:39212) 2024-08-19T01:35:30.535 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:30.531+0000 7fd770a00640 1 -- 172.21.15.83:0/2110064924 learned_addr learned my addr 172.21.15.83:0/2110064924 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-08-19T01:35:30.535 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:30.531+0000 7fd76be00640 1 --2- 172.21.15.83:0/2110064924 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557d4b4e0fc0 0x557d4b4ef2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:35:30.535 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:30.532+0000 7fd770a00640 1 -- 172.21.15.83:0/2110064924 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557d4b4e0fc0 msgr2=0x557d4b4ef2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:35:30.535 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:30.532+0000 7fd770a00640 1 --2- 172.21.15.83:0/2110064924 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557d4b4e0fc0 0x557d4b4ef2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:35:30.535 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:30.532+0000 7fd770a00640 1 -- 172.21.15.83:0/2110064924 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557d4b4e6050 msgr2=0x557d4b570570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:35:30.535 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:30.532+0000 7fd770a00640 1 --2- 172.21.15.83:0/2110064924 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557d4b4e6050 0x557d4b570570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:35:30.535 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:30.532+0000 7fd770a00640 1 -- 172.21.15.83:0/2110064924 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557d4b570ab0 con 0x557d4b4e0680 2024-08-19T01:35:30.535 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:30.532+0000 7fd770a00640 1 --2- 172.21.15.83:0/2110064924 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557d4b4e0680 0x557d4b4eed80 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fd760096790 tx=0x7fd7600cd420 comp rx=0 tx=0).ready entity=mon.2 client_cookie=823fd93bd2a08ed8 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:35:30.535 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:30.532+0000 7fd752000640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-08-19T01:35:30.536 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:30.532+0000 7fd752000640 1 -- 172.21.15.83:0/2110064924 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7600a00a0 con 0x557d4b4e0680 2024-08-19T01:35:30.536 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:30.532+0000 7fd776209740 1 -- 172.21.15.83:0/2110064924 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x557d4b451fc0 con 0x557d4b4e0680 2024-08-19T01:35:30.536 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:30.532+0000 7fd752000640 1 -- 172.21.15.83:0/2110064924 <== mon.2 v2:172.21.15.117:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8269+0+0 (secure 0 0 0) 0x7fd76009c160 con 0x557d4b4e0680 2024-08-19T01:35:30.537 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:30.532+0000 7fd752000640 1 -- 172.21.15.83:0/2110064924 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fd7600a44d0 con 0x557d4b4e0680 2024-08-19T01:35:30.537 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:30.532+0000 7fd776209740 10 client.25250 fetch_fsmap learned FSMap version 25 2024-08-19T01:35:30.537 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:30.533+0000 7fd776209740 1 -- 172.21.15.83:0/2110064924 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x557d4b571c40 con 0x557d4b4e0680 2024-08-19T01:35:30.537 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:30.533+0000 7fd752000640 1 -- 172.21.15.83:0/2110064924 <== mon.2 v2:172.21.15.117:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x557d4b451fc0 con 0x557d4b4e0680 2024-08-19T01:35:30.537 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:30.533+0000 7fd752000640 1 -- 172.21.15.83:0/2110064924 <== mon.2 v2:172.21.15.117:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fd76009b260 con 0x557d4b4e0680 2024-08-19T01:35:30.537 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:30.533+0000 7fd776209740 10 client.25250 fetch_fsmap finished waiting for FSMap version 25 2024-08-19T01:35:30.537 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:30.533+0000 7fd776209740 10 client.25250 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-08-19T01:35:30.537 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:30.533+0000 7fd776209740 20 client.25250 populate_metadata read hostname 'smithi083' 2024-08-19T01:35:30.537 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:30.533+0000 7fd776209740 1 --2- 172.21.15.83:0/2110064924 >> [v2:172.21.15.117:6838/1967595704,v1:172.21.15.117:6839/1967595704] conn(0x557d4b572140 0x557d4b574530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:35:30.537 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:30.533+0000 7fd776209740 4 client.25250 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-08-19T01:35:30.537 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:30.533+0000 7fd776209740 1 -- 172.21.15.83:0/2110064924 --> [v2:172.21.15.117:6838/1967595704,v1:172.21.15.117:6839/1967595704] -- command(tid 0: {"prefix": "flush journal"}) -- 0x557d4b44c310 con 0x557d4b572140 2024-08-19T01:35:30.537 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:30.534+0000 7fd76be00640 1 --2- 172.21.15.83:0/2110064924 >> [v2:172.21.15.117:6838/1967595704,v1:172.21.15.117:6839/1967595704] conn(0x557d4b572140 0x557d4b574530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:35:30.537 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:30.534+0000 7fd76be00640 1 --2- 172.21.15.83:0/2110064924 >> [v2:172.21.15.117:6838/1967595704,v1:172.21.15.117:6839/1967595704] conn(0x557d4b572140 0x557d4b574530 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8069a1c5ac5322da server_cookie=3b89bfa332e2d022 in_seq=0 out_seq=0 2024-08-19T01:35:30.537 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:30.534+0000 7fd752000640 10 client.25250 ms_handle_connect on v2:172.21.15.117:6838/1967595704 2024-08-19T01:35:30.543 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:30.542+0000 7fd752000640 1 -- 172.21.15.83:0/2110064924 <== mds.0 v2:172.21.15.117:6838/1967595704 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x557d4b44c310 con 0x557d4b572140 2024-08-19T01:35:30.543 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:30.542+0000 7fd752000640 10 client.25250 handle_command_reply: tid=0 2024-08-19T01:35:30.543 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:30.542+0000 7fd776209740 1 client.25250 shutdown 2024-08-19T01:35:30.543 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:30.542+0000 7fd776209740 1 -- 172.21.15.83:0/2110064924 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557d4b4e0680 msgr2=0x557d4b4eed80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:35:30.543 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:30.542+0000 7fd776209740 1 --2- 172.21.15.83:0/2110064924 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557d4b4e0680 0x557d4b4eed80 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fd760096790 tx=0x7fd7600cd420 comp rx=0 tx=0).stop 2024-08-19T01:35:30.543 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:30.542+0000 7fd776209740 1 -- 172.21.15.83:0/2110064924 shutdown_connections 2024-08-19T01:35:30.543 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:30.542+0000 7fd776209740 1 --2- 172.21.15.83:0/2110064924 >> [v2:172.21.15.117:6838/1967595704,v1:172.21.15.117:6839/1967595704] conn(0x557d4b572140 0x557d4b574530 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:35:30.544 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:30.542+0000 7fd776209740 1 --2- 172.21.15.83:0/2110064924 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557d4b4e6050 0x557d4b570570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:35:30.544 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:30.542+0000 7fd776209740 1 --2- 172.21.15.83:0/2110064924 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557d4b4e0fc0 0x557d4b4ef2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:35:30.544 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:30.542+0000 7fd776209740 1 --2- 172.21.15.83:0/2110064924 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557d4b4e0680 0x557d4b4eed80 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:35:30.544 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:30.542+0000 7fd776209740 1 -- 172.21.15.83:0/2110064924 >> 172.21.15.83:0/2110064924 conn(0x557d4b4dbcf0 msgr2=0x557d4b4daf40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:35:30.544 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:30.542+0000 7fd752000640 0 client.25250 ms_handle_reset on v2:172.21.15.117:6838/1967595704 2024-08-19T01:35:30.544 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:30.543+0000 7fd776209740 1 -- 172.21.15.83:0/2110064924 shutdown_connections 2024-08-19T01:35:30.544 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:30.543+0000 7fd776209740 1 -- 172.21.15.83:0/2110064924 wait complete. 2024-08-19T01:35:30.544 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:30.543+0000 7fd776209740 20 client.25250 trim_cache size 0 max 16384 2024-08-19T01:35:30.546 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stdout:{ 2024-08-19T01:35:30.546 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stdout: "message": "", 2024-08-19T01:35:30.546 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stdout: "return_code": 0 2024-08-19T01:35:30.546 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stdout:} 2024-08-19T01:35:30.547 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:30.544+0000 7fd771e00640 1 -- 172.21.15.83:0/1738376816 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7fd7540a2c10 msgr2=0x7fd7540a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:35:30.547 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:30.544+0000 7fd771e00640 1 --2- 172.21.15.83:0/1738376816 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7fd7540a2c10 0x7fd7540a50d0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fd764009fd0 tx=0x7fd764004ab0 comp rx=0 tx=0).stop 2024-08-19T01:35:30.547 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:30.544+0000 7fd771e00640 1 -- 172.21.15.83:0/1738376816 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd76c0ffee0 msgr2=0x7fd76c19dfc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:35:30.547 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:30.544+0000 7fd771e00640 1 --2- 172.21.15.83:0/1738376816 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd76c0ffee0 0x7fd76c19dfc0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fd76000e9f0 tx=0x7fd76000edf0 comp rx=0 tx=0).stop 2024-08-19T01:35:30.547 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:30.544+0000 7fd771e00640 1 -- 172.21.15.83:0/1738376816 shutdown_connections 2024-08-19T01:35:30.547 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:30.544+0000 7fd771e00640 1 --2- 172.21.15.83:0/1738376816 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7fd7540a2c10 0x7fd7540a50d0 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:35:30.547 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:30.544+0000 7fd771e00640 1 --2- 172.21.15.83:0/1738376816 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd76c1051e0 0x7fd76c1a5570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:35:30.547 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:30.544+0000 7fd771e00640 1 --2- 172.21.15.83:0/1738376816 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd76c102830 0x7fd76c19e500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:35:30.547 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:30.544+0000 7fd771e00640 1 --2- 172.21.15.83:0/1738376816 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd76c0ffee0 0x7fd76c19dfc0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:35:30.547 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:30.544+0000 7fd771e00640 1 -- 172.21.15.83:0/1738376816 >> 172.21.15.83:0/1738376816 conn(0x7fd76c0fbcd0 msgr2=0x7fd76c100860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:35:30.547 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:30.544+0000 7fd771e00640 1 -- 172.21.15.83:0/1738376816 shutdown_connections 2024-08-19T01:35:30.548 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:30.544+0000 7fd771e00640 1 -- 172.21.15.83:0/1738376816 wait complete. 2024-08-19T01:35:30.868 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833 shell --fsid bad8780e-5dc9-11ef-bccf-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-08-19T01:35:31.322 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/bad8780e-5dc9-11ef-bccf-c7b262605968/mon.a/config 2024-08-19T01:35:31.731 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:35:31 smithi083 ceph-mon[17120]: pgmap v359: 97 pgs: 97 active+clean; 667 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1.2 KiB/s wr, 6 op/s 2024-08-19T01:35:31.952 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:35:31 smithi117 ceph-mon[22016]: pgmap v359: 97 pgs: 97 active+clean; 667 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1.2 KiB/s wr, 6 op/s 2024-08-19T01:35:31.966 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:35:31 smithi167 ceph-mon[21875]: pgmap v359: 97 pgs: 97 active+clean; 667 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1.2 KiB/s wr, 6 op/s 2024-08-19T01:35:31.987 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:31.968+0000 7f5413a00640 1 -- 172.21.15.83:0/1182244493 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f540c104480 msgr2=0x7f540c1109b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:35:31.988 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:31.968+0000 7f5413a00640 1 --2- 172.21.15.83:0/1182244493 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f540c104480 0x7f540c1109b0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f5404009cd0 tx=0x7f540402a8f0 comp rx=0 tx=0).stop 2024-08-19T01:35:31.988 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:31.968+0000 7f5413a00640 1 -- 172.21.15.83:0/1182244493 shutdown_connections 2024-08-19T01:35:31.988 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:31.968+0000 7f5413a00640 1 --2- 172.21.15.83:0/1182244493 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f540c104480 0x7f540c1109b0 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:35:31.988 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:31.968+0000 7f5413a00640 1 --2- 172.21.15.83:0/1182244493 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f540c103ae0 0x7f540c103f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:35:31.988 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:31.968+0000 7f5413a00640 1 --2- 172.21.15.83:0/1182244493 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f540c109b70 0x7f540c109f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:35:31.988 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:31.968+0000 7f5413a00640 1 -- 172.21.15.83:0/1182244493 >> 172.21.15.83:0/1182244493 conn(0x7f540c0ff780 msgr2=0x7f540c101ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:35:31.988 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:31.968+0000 7f5413a00640 1 -- 172.21.15.83:0/1182244493 shutdown_connections 2024-08-19T01:35:31.988 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:31.969+0000 7f5413a00640 1 -- 172.21.15.83:0/1182244493 wait complete. 2024-08-19T01:35:31.988 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:31.970+0000 7f5413a00640 1 Processor -- start 2024-08-19T01:35:31.988 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:31.970+0000 7f5413a00640 1 -- start start 2024-08-19T01:35:31.988 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:31.970+0000 7f5413a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f540c103ae0 0x7f540c19a350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:35:31.989 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:31.970+0000 7f5413a00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f540c104480 0x7f540c19a890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:35:31.989 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:31.970+0000 7f5413a00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f540c109b70 0x7f540c19ec20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:35:31.989 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:31.970+0000 7f5413a00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f540c113f30 con 0x7f540c103ae0 2024-08-19T01:35:31.989 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:31.970+0000 7f5413a00640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f540c113db0 con 0x7f540c104480 2024-08-19T01:35:31.989 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:31.970+0000 7f5413a00640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f540c1140b0 con 0x7f540c109b70 2024-08-19T01:35:31.989 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:31.970+0000 7f540be00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f540c104480 0x7f540c19a890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:35:31.989 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:31.970+0000 7f5411a00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f540c109b70 0x7f540c19ec20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:35:31.989 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:31.971+0000 7f540be00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f540c104480 0x7f540c19a890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.83:54110/0 (socket says 172.21.15.83:54110) 2024-08-19T01:35:31.989 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:31.971+0000 7f540be00640 1 -- 172.21.15.83:0/2627874702 learned_addr learned my addr 172.21.15.83:0/2627874702 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-08-19T01:35:31.989 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:31.971+0000 7f5411000640 1 --2- 172.21.15.83:0/2627874702 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f540c103ae0 0x7f540c19a350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:35:31.989 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:31.971+0000 7f540be00640 1 -- 172.21.15.83:0/2627874702 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f540c109b70 msgr2=0x7f540c19ec20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:35:31.989 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:31.971+0000 7f540be00640 1 --2- 172.21.15.83:0/2627874702 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f540c109b70 0x7f540c19ec20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:35:31.989 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:31.971+0000 7f540be00640 1 -- 172.21.15.83:0/2627874702 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f540c103ae0 msgr2=0x7f540c19a350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:35:31.990 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:31.971+0000 7f540be00640 1 --2- 172.21.15.83:0/2627874702 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f540c103ae0 0x7f540c19a350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:35:31.990 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:31.971+0000 7f540be00640 1 -- 172.21.15.83:0/2627874702 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f540c19f300 con 0x7f540c104480 2024-08-19T01:35:31.990 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:31.971+0000 7f540be00640 1 --2- 172.21.15.83:0/2627874702 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f540c104480 0x7f540c19a890 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f53f800e9f0 tx=0x7f53f800eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:35:31.990 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:31.972+0000 7f5409600640 1 -- 172.21.15.83:0/2627874702 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53f800cd80 con 0x7f540c104480 2024-08-19T01:35:31.990 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:31.972+0000 7f5409600640 1 -- 172.21.15.83:0/2627874702 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f53f8010910 con 0x7f540c104480 2024-08-19T01:35:31.990 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:31.972+0000 7f5413a00640 1 -- 172.21.15.83:0/2627874702 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f540c19f5f0 con 0x7f540c104480 2024-08-19T01:35:31.990 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:31.972+0000 7f5409600640 1 -- 172.21.15.83:0/2627874702 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53f8018990 con 0x7f540c104480 2024-08-19T01:35:31.990 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:31.972+0000 7f5413a00640 1 -- 172.21.15.83:0/2627874702 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f540c1a6ed0 con 0x7f540c104480 2024-08-19T01:35:31.990 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:31.975+0000 7f5409600640 1 -- 172.21.15.83:0/2627874702 <== mon.2 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 19) ==== 137884+0+0 (secure 0 0 0) 0x7f53f8010ac0 con 0x7f540c104480 2024-08-19T01:35:31.990 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:31.976+0000 7f54165c0740 1 Processor -- start 2024-08-19T01:35:31.990 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:31.976+0000 7f54165c0740 1 -- start start 2024-08-19T01:35:31.990 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:31.976+0000 7f54165c0740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a98e8b5410 0x55a98e8b57f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:35:31.990 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:31.976+0000 7f54165c0740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a98e8b5d30 0x55a98e8c89a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:35:31.991 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:31.976+0000 7f54165c0740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a98e8c8ee0 0x55a98e8cb2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:35:31.991 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:31.976+0000 7f54165c0740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55a98e81abf0 con 0x55a98e8b5d30 2024-08-19T01:35:31.991 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:31.976+0000 7f54165c0740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55a98e7b2b20 con 0x55a98e8c8ee0 2024-08-19T01:35:31.991 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:31.976+0000 7f54165c0740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55a98e816bb0 con 0x55a98e8b5410 2024-08-19T01:35:31.991 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:31.977+0000 7f5411a00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a98e8b5410 0x55a98e8b57f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:35:31.991 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:31.977+0000 7f5411000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a98e8b5d30 0x55a98e8c89a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:35:31.991 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:31.977+0000 7f5411a00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a98e8b5410 0x55a98e8b57f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.83:49284/0 (socket says 172.21.15.83:49284) 2024-08-19T01:35:31.991 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:31.977+0000 7f5411a00640 1 -- 172.21.15.83:0/1433822939 learned_addr learned my addr 172.21.15.83:0/1433822939 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-08-19T01:35:31.991 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:31.977+0000 7f5409600640 1 --2- 172.21.15.83:0/2627874702 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f53f40a2d10 0x7f53f40a51d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:35:31.991 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:31.977+0000 7f5409600640 1 -- 172.21.15.83:0/2627874702 <== mon.2 v2:172.21.15.117:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8269+0+0 (secure 0 0 0) 0x7f53f80cf570 con 0x7f540c104480 2024-08-19T01:35:31.991 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:31.978+0000 7f540be00640 1 --2- 172.21.15.83:0/1433822939 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a98e8c8ee0 0x55a98e8cb2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:35:31.991 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:31.978+0000 7f5411000640 1 -- 172.21.15.83:0/1433822939 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a98e8b5410 msgr2=0x55a98e8b57f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:35:31.991 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:31.978+0000 7f5411000640 1 --2- 172.21.15.83:0/1433822939 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a98e8b5410 0x55a98e8b57f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:35:31.992 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:31.978+0000 7f5411000640 1 -- 172.21.15.83:0/1433822939 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a98e8c8ee0 msgr2=0x55a98e8cb2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:35:31.992 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:31.978+0000 7f5411000640 1 --2- 172.21.15.83:0/1433822939 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a98e8c8ee0 0x55a98e8cb2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:35:31.992 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:31.978+0000 7f5411000640 1 -- 172.21.15.83:0/1433822939 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a98e7b97f0 con 0x55a98e8b5d30 2024-08-19T01:35:31.992 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:31.978+0000 7f5411a00640 1 --2- 172.21.15.83:0/2627874702 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f53f40a2d10 0x7f53f40a51d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:35:31.992 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:31.979+0000 7f5411000640 1 --2- 172.21.15.83:0/1433822939 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a98e8b5d30 0x55a98e8c89a0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f5400009af0 tx=0x7f54000113e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1643f0a833ca22b3 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:35:31.992 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:31.979+0000 7f53fe000640 1 -- 172.21.15.83:0/1433822939 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5400011e10 con 0x55a98e8b5d30 2024-08-19T01:35:31.992 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:31.979+0000 7f5411a00640 1 --2- 172.21.15.83:0/2627874702 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f53f40a2d10 0x7f53f40a51d0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f5404009ca0 tx=0x7f540402a8f0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:35:31.992 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:31.979+0000 7f53fe000640 1 -- 172.21.15.83:0/1433822939 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5400016070 con 0x55a98e8b5d30 2024-08-19T01:35:31.992 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:31.979+0000 7f53fe000640 1 -- 172.21.15.83:0/1433822939 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f540001b470 con 0x55a98e8b5d30 2024-08-19T01:35:31.992 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:31.980+0000 7f54165c0740 1 -- 172.21.15.83:0/1433822939 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a98e8b5d30 msgr2=0x55a98e8c89a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:35:31.992 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:31.980+0000 7f54165c0740 1 --2- 172.21.15.83:0/1433822939 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a98e8b5d30 0x55a98e8c89a0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f5400009af0 tx=0x7f54000113e0 comp rx=0 tx=0).stop 2024-08-19T01:35:31.992 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:31.980+0000 7f54165c0740 1 -- 172.21.15.83:0/1433822939 shutdown_connections 2024-08-19T01:35:31.992 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:31.980+0000 7f54165c0740 1 --2- 172.21.15.83:0/1433822939 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a98e8c8ee0 0x55a98e8cb2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:35:31.993 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:31.980+0000 7f54165c0740 1 --2- 172.21.15.83:0/1433822939 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a98e8b5d30 0x55a98e8c89a0 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:35:31.993 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:31.980+0000 7f54165c0740 1 --2- 172.21.15.83:0/1433822939 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a98e8b5410 0x55a98e8b57f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:35:31.993 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:31.980+0000 7f54165c0740 1 -- 172.21.15.83:0/1433822939 >> 172.21.15.83:0/1433822939 conn(0x55a98e8b1a80 msgr2=0x55a98e8b1e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:35:31.993 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:31.980+0000 7f54165c0740 1 -- 172.21.15.83:0/1433822939 shutdown_connections 2024-08-19T01:35:31.993 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:31.980+0000 7f54165c0740 1 -- 172.21.15.83:0/1433822939 wait complete. 2024-08-19T01:35:31.993 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:31.981+0000 7f54165c0740 1 -- start start 2024-08-19T01:35:31.993 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:31.981+0000 7f54165c0740 1 Processor -- start 2024-08-19T01:35:31.993 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:31.982+0000 7f54165c0740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a98e8b5410 0x55a98e8c5e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:35:31.993 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:31.982+0000 7f54165c0740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a98e8b5d30 0x55a98e8c63c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:35:31.993 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:31.982+0000 7f54165c0740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a98e8c8ee0 0x55a98e8c6900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:35:31.993 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:31.982+0000 7f54165c0740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55a98e8cba50 con 0x55a98e8c8ee0 2024-08-19T01:35:31.993 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:31.982+0000 7f54165c0740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55a98e7d5cf0 con 0x55a98e8b5410 2024-08-19T01:35:31.993 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:31.982+0000 7f54165c0740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55a98e8cbbd0 con 0x55a98e8b5d30 2024-08-19T01:35:31.994 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:31.982+0000 7f54165c0740 10 client.0 _ll_register_callbacks cb 0x55a98e7c1e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-19T01:35:31.994 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:31.982+0000 7f5411a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a98e8c8ee0 0x55a98e8c6900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:35:31.994 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:31.982+0000 7f5411a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a98e8c8ee0 0x55a98e8c6900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:58462/0 (socket says 172.21.15.83:58462) 2024-08-19T01:35:31.994 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:31.982+0000 7f5411a00640 1 -- 172.21.15.83:0/3279011051 learned_addr learned my addr 172.21.15.83:0/3279011051 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-08-19T01:35:31.994 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:31.982+0000 7f540be00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a98e8b5d30 0x55a98e8c63c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:35:31.994 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:31.982+0000 7f5411000640 1 --2- 172.21.15.83:0/3279011051 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a98e8b5410 0x55a98e8c5e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:35:31.994 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:31.983+0000 7f540be00640 1 -- 172.21.15.83:0/3279011051 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a98e8b5410 msgr2=0x55a98e8c5e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:35:31.994 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:31.983+0000 7f540be00640 1 --2- 172.21.15.83:0/3279011051 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a98e8b5410 0x55a98e8c5e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:35:31.994 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:31.983+0000 7f540be00640 1 -- 172.21.15.83:0/3279011051 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a98e8c8ee0 msgr2=0x55a98e8c6900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:35:31.994 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:31.983+0000 7f540be00640 1 --2- 172.21.15.83:0/3279011051 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a98e8c8ee0 0x55a98e8c6900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:35:31.994 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:31.983+0000 7f540be00640 1 -- 172.21.15.83:0/3279011051 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a98e58c3c0 con 0x55a98e8b5d30 2024-08-19T01:35:31.994 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:31.983+0000 7f5411000640 1 --2- 172.21.15.83:0/3279011051 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a98e8b5410 0x55a98e8c5e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-19T01:35:31.995 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:31.983+0000 7f5411a00640 1 --2- 172.21.15.83:0/3279011051 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a98e8c8ee0 0x55a98e8c6900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-19T01:35:31.995 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:31.983+0000 7f540be00640 1 --2- 172.21.15.83:0/3279011051 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a98e8b5d30 0x55a98e8c63c0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f53f80ccea0 tx=0x7f53f809c430 comp rx=0 tx=0).ready entity=mon.1 client_cookie=56bc5d8876514286 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:35:31.995 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:31.983+0000 7f53f2a00640 10 client.0 ms_handle_connect on v2:172.21.15.167:3300/0 2024-08-19T01:35:31.995 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:31.983+0000 7f53f2a00640 1 -- 172.21.15.83:0/3279011051 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53f80cf2b0 con 0x55a98e8b5d30 2024-08-19T01:35:31.995 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:31.984+0000 7f53f2a00640 1 -- 172.21.15.83:0/3279011051 <== mon.1 v2:172.21.15.167:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8269+0+0 (secure 0 0 0) 0x7f53f8018b30 con 0x55a98e8b5d30 2024-08-19T01:35:31.995 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:31.984+0000 7f54165c0740 1 -- 172.21.15.83:0/3279011051 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5400016070 con 0x55a98e8b5d30 2024-08-19T01:35:31.995 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:31.984+0000 7f53f2a00640 1 -- 172.21.15.83:0/3279011051 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f53f8018d30 con 0x55a98e8b5d30 2024-08-19T01:35:31.995 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:31.984+0000 7f54165c0740 10 client.25243 fetch_fsmap learned FSMap version 25 2024-08-19T01:35:31.995 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:31.984+0000 7f54165c0740 1 -- 172.21.15.83:0/3279011051 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55a98e8d7810 con 0x55a98e8b5d30 2024-08-19T01:35:31.995 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:31.984+0000 7f53f2a00640 1 -- 172.21.15.83:0/3279011051 <== mon.1 v2:172.21.15.167:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5400016070 con 0x55a98e8b5d30 2024-08-19T01:35:31.995 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:31.984+0000 7f53f2a00640 1 -- 172.21.15.83:0/3279011051 <== mon.1 v2:172.21.15.167:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f53f80979d0 con 0x55a98e8b5d30 2024-08-19T01:35:31.995 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:31.984+0000 7f54165c0740 10 client.25243 fetch_fsmap finished waiting for FSMap version 25 2024-08-19T01:35:31.995 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:31.984+0000 7f54165c0740 10 client.25243 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.g 2024-08-19T01:35:31.995 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:31.984+0000 7f54165c0740 20 client.25243 populate_metadata read hostname 'smithi083' 2024-08-19T01:35:31.995 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:31.985+0000 7f54165c0740 1 --2- 172.21.15.83:0/3279011051 >> [v2:172.21.15.83:6838/3239261845,v1:172.21.15.83:6839/3239261845] conn(0x55a98e8d8d20 0x55a98e8db110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:35:31.996 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:31.985+0000 7f54165c0740 4 client.25243 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-19T01:35:31.996 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:31.985+0000 7f54165c0740 1 -- 172.21.15.83:0/3279011051 --> [v2:172.21.15.83:6838/3239261845,v1:172.21.15.83:6839/3239261845] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a98e828160 con 0x55a98e8d8d20 2024-08-19T01:35:31.996 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:31.985+0000 7f5411000640 1 --2- 172.21.15.83:0/3279011051 >> [v2:172.21.15.83:6838/3239261845,v1:172.21.15.83:6839/3239261845] conn(0x55a98e8d8d20 0x55a98e8db110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:35:31.996 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:31.986+0000 7f5411000640 1 --2- 172.21.15.83:0/3279011051 >> [v2:172.21.15.83:6838/3239261845,v1:172.21.15.83:6839/3239261845] conn(0x55a98e8d8d20 0x55a98e8db110 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=a4d9fbe5b2ebf720 server_cookie=d46726a0ae98f7e7 in_seq=0 out_seq=0 2024-08-19T01:35:31.996 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:31.986+0000 7f53f2a00640 10 client.25243 ms_handle_connect on v2:172.21.15.83:6838/3239261845 2024-08-19T01:35:31.997 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:31.987+0000 7f53f2a00640 1 -- 172.21.15.83:0/3279011051 <== mds.1 v2:172.21.15.83:6838/3239261845 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55a98e828160 con 0x55a98e8d8d20 2024-08-19T01:35:31.997 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:31.987+0000 7f53f2a00640 10 client.25243 handle_command_reply: tid=0 2024-08-19T01:35:31.997 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:31.987+0000 7f54165c0740 1 client.25243 shutdown 2024-08-19T01:35:31.997 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:31.987+0000 7f54165c0740 1 -- 172.21.15.83:0/3279011051 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a98e8b5d30 msgr2=0x55a98e8c63c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:35:31.997 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:31.987+0000 7f54165c0740 1 --2- 172.21.15.83:0/3279011051 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a98e8b5d30 0x55a98e8c63c0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f53f80ccea0 tx=0x7f53f809c430 comp rx=0 tx=0).stop 2024-08-19T01:35:31.997 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:31.987+0000 7f54165c0740 1 -- 172.21.15.83:0/3279011051 shutdown_connections 2024-08-19T01:35:31.997 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:31.987+0000 7f54165c0740 1 --2- 172.21.15.83:0/3279011051 >> [v2:172.21.15.83:6838/3239261845,v1:172.21.15.83:6839/3239261845] conn(0x55a98e8d8d20 0x55a98e8db110 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:35:31.997 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:31.987+0000 7f54165c0740 1 --2- 172.21.15.83:0/3279011051 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a98e8c8ee0 0x55a98e8c6900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:35:31.997 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:31.987+0000 7f54165c0740 1 --2- 172.21.15.83:0/3279011051 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a98e8b5d30 0x55a98e8c63c0 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:35:31.997 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:31.988+0000 7f54165c0740 1 --2- 172.21.15.83:0/3279011051 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a98e8b5410 0x55a98e8c5e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:35:31.997 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:31.988+0000 7f54165c0740 1 -- 172.21.15.83:0/3279011051 >> 172.21.15.83:0/3279011051 conn(0x55a98e8b1a80 msgr2=0x55a98e8b7710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:35:31.998 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:31.988+0000 7f53f2a00640 0 client.25243 ms_handle_reset on v2:172.21.15.83:6838/3239261845 2024-08-19T01:35:31.998 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:31.988+0000 7f54165c0740 1 -- 172.21.15.83:0/3279011051 shutdown_connections 2024-08-19T01:35:31.998 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:31.988+0000 7f54165c0740 1 -- 172.21.15.83:0/3279011051 wait complete. 2024-08-19T01:35:31.998 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:31.988+0000 7f54165c0740 20 client.25243 trim_cache size 0 max 16384 2024-08-19T01:35:32.016 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:32.014+0000 7f54165c0740 1 Processor -- start 2024-08-19T01:35:32.016 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:32.015+0000 7f54165c0740 1 -- start start 2024-08-19T01:35:32.016 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:32.015+0000 7f54165c0740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a98e8b5410 0x55a98e8c6750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:35:32.017 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:32.015+0000 7f54165c0740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a98e8b5d30 0x55a98e8e09e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:35:32.017 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:32.015+0000 7f54165c0740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a98e8c8ee0 0x55a98e8e0f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:35:32.018 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:32.015+0000 7f54165c0740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f540c1adb20 con 0x55a98e8b5410 2024-08-19T01:35:32.018 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:32.015+0000 7f54165c0740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f540c1ab3f0 con 0x55a98e8c8ee0 2024-08-19T01:35:32.018 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:32.015+0000 7f54165c0740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f540c1a92c0 con 0x55a98e8b5d30 2024-08-19T01:35:32.019 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:32.016+0000 7f540be00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a98e8b5d30 0x55a98e8e09e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:35:32.019 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:32.016+0000 7f5411a00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a98e8c8ee0 0x55a98e8e0f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:35:32.019 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:32.016+0000 7f540be00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a98e8b5d30 0x55a98e8e09e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.83:49306/0 (socket says 172.21.15.83:49306) 2024-08-19T01:35:32.019 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:32.016+0000 7f540be00640 1 -- 172.21.15.83:0/869974756 learned_addr learned my addr 172.21.15.83:0/869974756 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-08-19T01:35:32.019 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:32.016+0000 7f5411a00640 1 -- 172.21.15.83:0/869974756 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a98e8b5d30 msgr2=0x55a98e8e09e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:35:32.019 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:32.016+0000 7f5411a00640 1 --2- 172.21.15.83:0/869974756 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a98e8b5d30 0x55a98e8e09e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:35:32.020 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:32.016+0000 7f5411a00640 1 -- 172.21.15.83:0/869974756 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a98e8b5410 msgr2=0x55a98e8c6750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:35:32.020 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:32.016+0000 7f5411000640 1 --2- 172.21.15.83:0/869974756 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a98e8b5410 0x55a98e8c6750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:35:32.020 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:32.016+0000 7f5411a00640 1 --2- 172.21.15.83:0/869974756 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a98e8b5410 0x55a98e8c6750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:35:32.020 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:32.016+0000 7f5411a00640 1 -- 172.21.15.83:0/869974756 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a98e8db480 con 0x55a98e8c8ee0 2024-08-19T01:35:32.020 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:32.016+0000 7f5411000640 1 --2- 172.21.15.83:0/869974756 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a98e8b5410 0x55a98e8c6750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-19T01:35:32.020 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:32.017+0000 7f5411a00640 1 --2- 172.21.15.83:0/869974756 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a98e8c8ee0 0x55a98e8e0f20 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f5404005e30 tx=0x7f5404045000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=53640f874ee02fce server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:35:32.020 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:32.017+0000 7f53fd600640 1 -- 172.21.15.83:0/869974756 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5404045070 con 0x55a98e8c8ee0 2024-08-19T01:35:32.020 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:32.017+0000 7f53fd600640 1 -- 172.21.15.83:0/869974756 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5404036040 con 0x55a98e8c8ee0 2024-08-19T01:35:32.020 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:32.017+0000 7f53fd600640 1 -- 172.21.15.83:0/869974756 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54040317e0 con 0x55a98e8c8ee0 2024-08-19T01:35:32.021 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:32.018+0000 7f54165c0740 1 -- 172.21.15.83:0/869974756 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a98e8c8ee0 msgr2=0x55a98e8e0f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:35:32.021 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:32.018+0000 7f54165c0740 1 --2- 172.21.15.83:0/869974756 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a98e8c8ee0 0x55a98e8e0f20 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f5404005e30 tx=0x7f5404045000 comp rx=0 tx=0).stop 2024-08-19T01:35:32.021 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:32.018+0000 7f54165c0740 1 -- 172.21.15.83:0/869974756 shutdown_connections 2024-08-19T01:35:32.021 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:32.018+0000 7f54165c0740 1 --2- 172.21.15.83:0/869974756 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a98e8c8ee0 0x55a98e8e0f20 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:35:32.021 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:32.018+0000 7f54165c0740 1 --2- 172.21.15.83:0/869974756 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a98e8b5d30 0x55a98e8e09e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:35:32.022 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:32.018+0000 7f54165c0740 1 --2- 172.21.15.83:0/869974756 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a98e8b5410 0x55a98e8c6750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:35:32.022 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:32.018+0000 7f54165c0740 1 -- 172.21.15.83:0/869974756 >> 172.21.15.83:0/869974756 conn(0x55a98e8b1a80 msgr2=0x55a98e8c4f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:35:32.022 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:32.019+0000 7f54165c0740 1 -- 172.21.15.83:0/869974756 shutdown_connections 2024-08-19T01:35:32.022 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:32.019+0000 7f54165c0740 1 -- 172.21.15.83:0/869974756 wait complete. 2024-08-19T01:35:32.022 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:32.019+0000 7f54165c0740 1 -- start start 2024-08-19T01:35:32.022 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:32.020+0000 7f54165c0740 1 Processor -- start 2024-08-19T01:35:32.022 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:32.020+0000 7f54165c0740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a98e8b5410 0x55a98e8cbe50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:35:32.022 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:32.020+0000 7f54165c0740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a98e8b5d30 0x55a98e8cc390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:35:32.024 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:32.020+0000 7f54165c0740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a98e8c8ee0 0x55a98e8cc8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:35:32.024 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:32.020+0000 7f54165c0740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f540c1a9910 con 0x55a98e8c8ee0 2024-08-19T01:35:32.024 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:32.020+0000 7f54165c0740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f540c1aad20 con 0x55a98e8b5410 2024-08-19T01:35:32.024 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:32.020+0000 7f54165c0740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f540c1a9f80 con 0x55a98e8b5d30 2024-08-19T01:35:32.024 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:32.020+0000 7f54165c0740 10 client.0 _ll_register_callbacks cb 0x7f53f809b550 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-19T01:35:32.024 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:32.020+0000 7f5411000640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a98e8b5410 0x55a98e8cbe50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:35:32.024 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:32.021+0000 7f5411000640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a98e8b5410 0x55a98e8cbe50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.83:54154/0 (socket says 172.21.15.83:54154) 2024-08-19T01:35:32.025 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:32.021+0000 7f5411000640 1 -- 172.21.15.83:0/2213838332 learned_addr learned my addr 172.21.15.83:0/2213838332 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-08-19T01:35:32.025 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:32.021+0000 7f5411a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a98e8c8ee0 0x55a98e8cc8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:35:32.025 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:32.021+0000 7f540be00640 1 --2- 172.21.15.83:0/2213838332 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a98e8b5d30 0x55a98e8cc390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:35:32.025 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:32.021+0000 7f5411a00640 1 -- 172.21.15.83:0/2213838332 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a98e8b5d30 msgr2=0x55a98e8cc390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:35:32.025 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:32.021+0000 7f5411a00640 1 --2- 172.21.15.83:0/2213838332 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a98e8b5d30 0x55a98e8cc390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:35:32.025 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:32.021+0000 7f5411a00640 1 -- 172.21.15.83:0/2213838332 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a98e8b5410 msgr2=0x55a98e8cbe50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:35:32.025 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:32.021+0000 7f5411a00640 1 --2- 172.21.15.83:0/2213838332 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a98e8b5410 0x55a98e8cbe50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:35:32.025 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:32.021+0000 7f5411a00640 1 -- 172.21.15.83:0/2213838332 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a98e941b00 con 0x55a98e8c8ee0 2024-08-19T01:35:32.025 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:32.022+0000 7f5411a00640 1 --2- 172.21.15.83:0/2213838332 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a98e8c8ee0 0x55a98e8cc8d0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f540403f040 tx=0x7f5404004230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=14e2950afbc40d41 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:35:32.025 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:32.022+0000 7f53f2a00640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-08-19T01:35:32.025 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:32.022+0000 7f53f2a00640 1 -- 172.21.15.83:0/2213838332 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5404031520 con 0x55a98e8c8ee0 2024-08-19T01:35:32.025 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:32.022+0000 7f53f2a00640 1 -- 172.21.15.83:0/2213838332 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8269+0+0 (secure 0 0 0) 0x7f54040316c0 con 0x55a98e8c8ee0 2024-08-19T01:35:32.025 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:32.022+0000 7f54165c0740 1 -- 172.21.15.83:0/2213838332 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5404036040 con 0x55a98e8c8ee0 2024-08-19T01:35:32.026 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:32.023+0000 7f53f2a00640 1 -- 172.21.15.83:0/2213838332 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f54040318c0 con 0x55a98e8c8ee0 2024-08-19T01:35:32.026 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:32.023+0000 7f54165c0740 10 client.15195 fetch_fsmap learned FSMap version 25 2024-08-19T01:35:32.026 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:32.023+0000 7f54165c0740 1 -- 172.21.15.83:0/2213838332 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55a98e942d30 con 0x55a98e8c8ee0 2024-08-19T01:35:32.026 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:32.023+0000 7f53f2a00640 1 -- 172.21.15.83:0/2213838332 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5404036040 con 0x55a98e8c8ee0 2024-08-19T01:35:32.027 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:32.024+0000 7f53f2a00640 1 -- 172.21.15.83:0/2213838332 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f5404056760 con 0x55a98e8c8ee0 2024-08-19T01:35:32.027 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:32.024+0000 7f54165c0740 10 client.15195 fetch_fsmap finished waiting for FSMap version 25 2024-08-19T01:35:32.027 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:32.024+0000 7f54165c0740 10 client.15195 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.g 2024-08-19T01:35:32.027 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:32.024+0000 7f54165c0740 20 client.15195 populate_metadata read hostname 'smithi083' 2024-08-19T01:35:32.027 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:32.024+0000 7f54165c0740 1 --2- 172.21.15.83:0/2213838332 >> [v2:172.21.15.83:6838/3239261845,v1:172.21.15.83:6839/3239261845] conn(0x55a98e943450 0x55a98e945870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:35:32.027 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:32.024+0000 7f54165c0740 4 client.15195 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-08-19T01:35:32.027 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:32.024+0000 7f54165c0740 1 -- 172.21.15.83:0/2213838332 --> [v2:172.21.15.83:6838/3239261845,v1:172.21.15.83:6839/3239261845] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55a98e822670 con 0x55a98e943450 2024-08-19T01:35:32.027 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:32.025+0000 7f5411000640 1 --2- 172.21.15.83:0/2213838332 >> [v2:172.21.15.83:6838/3239261845,v1:172.21.15.83:6839/3239261845] conn(0x55a98e943450 0x55a98e945870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:35:32.028 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:32.025+0000 7f5411000640 1 --2- 172.21.15.83:0/2213838332 >> [v2:172.21.15.83:6838/3239261845,v1:172.21.15.83:6839/3239261845] conn(0x55a98e943450 0x55a98e945870 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=8063287fda654297 server_cookie=9a8f71fb14cccda3 in_seq=0 out_seq=0 2024-08-19T01:35:32.028 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:32.025+0000 7f53f2a00640 10 client.15195 ms_handle_connect on v2:172.21.15.83:6838/3239261845 2024-08-19T01:35:32.028 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:32.026+0000 7f53f2a00640 1 -- 172.21.15.83:0/2213838332 <== mds.1 v2:172.21.15.83:6838/3239261845 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55a98e822670 con 0x55a98e943450 2024-08-19T01:35:32.028 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:32.026+0000 7f53f2a00640 10 client.15195 handle_command_reply: tid=0 2024-08-19T01:35:32.028 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:32.026+0000 7f54165c0740 1 client.15195 shutdown 2024-08-19T01:35:32.028 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:32.026+0000 7f54165c0740 1 -- 172.21.15.83:0/2213838332 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a98e8c8ee0 msgr2=0x55a98e8cc8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:35:32.029 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:32.026+0000 7f54165c0740 1 --2- 172.21.15.83:0/2213838332 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a98e8c8ee0 0x55a98e8cc8d0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f540403f040 tx=0x7f5404004230 comp rx=0 tx=0).stop 2024-08-19T01:35:32.029 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:32.026+0000 7f54165c0740 1 -- 172.21.15.83:0/2213838332 shutdown_connections 2024-08-19T01:35:32.029 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:32.026+0000 7f54165c0740 1 --2- 172.21.15.83:0/2213838332 >> [v2:172.21.15.83:6838/3239261845,v1:172.21.15.83:6839/3239261845] conn(0x55a98e943450 0x55a98e945870 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:35:32.029 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:32.026+0000 7f54165c0740 1 --2- 172.21.15.83:0/2213838332 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a98e8c8ee0 0x55a98e8cc8d0 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:35:32.029 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:32.026+0000 7f54165c0740 1 --2- 172.21.15.83:0/2213838332 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a98e8b5d30 0x55a98e8cc390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:35:32.029 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:32.026+0000 7f54165c0740 1 --2- 172.21.15.83:0/2213838332 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a98e8b5410 0x55a98e8cbe50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:35:32.029 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:32.026+0000 7f54165c0740 1 -- 172.21.15.83:0/2213838332 >> 172.21.15.83:0/2213838332 conn(0x55a98e8b1a80 msgr2=0x55a98e8c76e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:35:32.029 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:32.026+0000 7f53f2a00640 0 client.15195 ms_handle_reset on v2:172.21.15.83:6838/3239261845 2024-08-19T01:35:32.029 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:32.027+0000 7f54165c0740 1 -- 172.21.15.83:0/2213838332 shutdown_connections 2024-08-19T01:35:32.029 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:32.027+0000 7f54165c0740 1 -- 172.21.15.83:0/2213838332 wait complete. 2024-08-19T01:35:32.029 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:32.027+0000 7f54165c0740 20 client.15195 trim_cache size 0 max 16384 2024-08-19T01:35:32.030 INFO:teuthology.orchestra.run.smithi083.stdout:[] 2024-08-19T01:35:32.030 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:32.028+0000 7f5413a00640 1 -- 172.21.15.83:0/2627874702 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f53f40a2d10 msgr2=0x7f53f40a51d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:35:32.031 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:32.028+0000 7f5413a00640 1 --2- 172.21.15.83:0/2627874702 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f53f40a2d10 0x7f53f40a51d0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f5404009ca0 tx=0x7f540402a8f0 comp rx=0 tx=0).stop 2024-08-19T01:35:32.031 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:32.028+0000 7f5413a00640 1 -- 172.21.15.83:0/2627874702 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f540c104480 msgr2=0x7f540c19a890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:35:32.031 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:32.028+0000 7f5413a00640 1 --2- 172.21.15.83:0/2627874702 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f540c104480 0x7f540c19a890 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f53f800e9f0 tx=0x7f53f800eeb0 comp rx=0 tx=0).stop 2024-08-19T01:35:32.031 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:32.029+0000 7f5413a00640 1 -- 172.21.15.83:0/2627874702 shutdown_connections 2024-08-19T01:35:32.031 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:32.029+0000 7f5413a00640 1 --2- 172.21.15.83:0/2627874702 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f53f40a2d10 0x7f53f40a51d0 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:35:32.031 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:32.029+0000 7f5413a00640 1 --2- 172.21.15.83:0/2627874702 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f540c109b70 0x7f540c19ec20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:35:32.031 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:32.029+0000 7f5413a00640 1 --2- 172.21.15.83:0/2627874702 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f540c104480 0x7f540c19a890 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:35:32.031 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:32.029+0000 7f5413a00640 1 --2- 172.21.15.83:0/2627874702 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f540c103ae0 0x7f540c19a350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:35:32.031 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:32.029+0000 7f5413a00640 1 -- 172.21.15.83:0/2627874702 >> 172.21.15.83:0/2627874702 conn(0x7f540c0ff780 msgr2=0x7f540c100ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:35:32.031 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:32.029+0000 7f5413a00640 1 -- 172.21.15.83:0/2627874702 shutdown_connections 2024-08-19T01:35:32.031 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:32.029+0000 7f5413a00640 1 -- 172.21.15.83:0/2627874702 wait complete. 2024-08-19T01:35:33.167 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833 shell --fsid bad8780e-5dc9-11ef-bccf-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-08-19T01:35:33.592 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/bad8780e-5dc9-11ef-bccf-c7b262605968/mon.a/config 2024-08-19T01:35:33.810 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:35:33 smithi083 ceph-mon[17120]: pgmap v360: 97 pgs: 97 active+clean; 667 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 1.1 KiB/s wr, 6 op/s 2024-08-19T01:35:33.952 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:35:33 smithi117 ceph-mon[22016]: pgmap v360: 97 pgs: 97 active+clean; 667 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 1.1 KiB/s wr, 6 op/s 2024-08-19T01:35:33.966 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:35:33 smithi167 ceph-mon[21875]: pgmap v360: 97 pgs: 97 active+clean; 667 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 1.1 KiB/s wr, 6 op/s 2024-08-19T01:35:34.204 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:34.202+0000 7fb5f1800640 1 -- 172.21.15.83:0/1628435050 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb5ec100e00 msgr2=0x7fb5ec10dac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:35:34.221 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:34.202+0000 7fb5f1800640 1 --2- 172.21.15.83:0/1628435050 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb5ec100e00 0x7fb5ec10dac0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fb5d8009860 tx=0x7fb5d802a340 comp rx=0 tx=0).stop 2024-08-19T01:35:34.221 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:34.202+0000 7fb5f1800640 1 -- 172.21.15.83:0/1628435050 shutdown_connections 2024-08-19T01:35:34.221 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:34.202+0000 7fb5f1800640 1 --2- 172.21.15.83:0/1628435050 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb5ec10e000 0x7fb5ec1103f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:35:34.221 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:34.202+0000 7fb5f1800640 1 --2- 172.21.15.83:0/1628435050 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb5ec100e00 0x7fb5ec10dac0 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:35:34.221 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:34.202+0000 7fb5f1800640 1 --2- 172.21.15.83:0/1628435050 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb5ec100450 0x7fb5ec100830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:35:34.221 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:34.202+0000 7fb5f1800640 1 -- 172.21.15.83:0/1628435050 >> 172.21.15.83:0/1628435050 conn(0x7fb5ec0fc340 msgr2=0x7fb5ec0fe760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:35:34.221 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:34.203+0000 7fb5f1800640 1 -- 172.21.15.83:0/1628435050 shutdown_connections 2024-08-19T01:35:34.221 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:34.203+0000 7fb5f1800640 1 -- 172.21.15.83:0/1628435050 wait complete. 2024-08-19T01:35:34.226 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:34.204+0000 7fb5f1800640 1 Processor -- start 2024-08-19T01:35:34.226 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:34.204+0000 7fb5f1800640 1 -- start start 2024-08-19T01:35:34.226 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:34.205+0000 7fb5f1800640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb5ec100450 0x7fb5ec19a260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:35:34.226 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:34.205+0000 7fb5f1800640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb5ec100e00 0x7fb5ec19a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:35:34.226 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:34.205+0000 7fb5f1800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb5ec10e000 0x7fb5ec19eb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:35:34.226 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:34.205+0000 7fb5f1800640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fb5ec113d00 con 0x7fb5ec10e000 2024-08-19T01:35:34.226 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:34.205+0000 7fb5f1800640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fb5ec113b80 con 0x7fb5ec100450 2024-08-19T01:35:34.226 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:34.205+0000 7fb5f1800640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fb5ec113e80 con 0x7fb5ec100e00 2024-08-19T01:35:34.226 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:34.205+0000 7fb5ebe00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb5ec100450 0x7fb5ec19a260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:35:34.226 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:34.205+0000 7fb5eb400640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb5ec100e00 0x7fb5ec19a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:35:34.226 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:34.205+0000 7fb5ebe00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb5ec100450 0x7fb5ec19a260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.83:54168/0 (socket says 172.21.15.83:54168) 2024-08-19T01:35:34.227 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:34.205+0000 7fb5ebe00640 1 -- 172.21.15.83:0/2400294879 learned_addr learned my addr 172.21.15.83:0/2400294879 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-08-19T01:35:34.227 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:34.206+0000 7fb5f0e00640 1 --2- 172.21.15.83:0/2400294879 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb5ec10e000 0x7fb5ec19eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:35:34.227 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:34.206+0000 7fb5f0e00640 1 -- 172.21.15.83:0/2400294879 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb5ec100e00 msgr2=0x7fb5ec19a7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:35:34.227 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:34.206+0000 7fb5f0e00640 1 --2- 172.21.15.83:0/2400294879 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb5ec100e00 0x7fb5ec19a7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:35:34.227 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:34.206+0000 7fb5f0e00640 1 -- 172.21.15.83:0/2400294879 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb5ec100450 msgr2=0x7fb5ec19a260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:35:34.227 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:34.206+0000 7fb5f0e00640 1 --2- 172.21.15.83:0/2400294879 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb5ec100450 0x7fb5ec19a260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:35:34.227 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:34.206+0000 7fb5f0e00640 1 -- 172.21.15.83:0/2400294879 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb5ec19f210 con 0x7fb5ec10e000 2024-08-19T01:35:34.227 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:34.206+0000 7fb5f0e00640 1 --2- 172.21.15.83:0/2400294879 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb5ec10e000 0x7fb5ec19eb30 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7fb5ec102630 tx=0x7fb5e400dea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:35:34.227 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:34.207+0000 7fb5e8c00640 1 -- 172.21.15.83:0/2400294879 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5e400bd40 con 0x7fb5ec10e000 2024-08-19T01:35:34.227 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:34.207+0000 7fb5e8c00640 1 -- 172.21.15.83:0/2400294879 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb5e4003e50 con 0x7fb5ec10e000 2024-08-19T01:35:34.227 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:34.207+0000 7fb5e8c00640 1 -- 172.21.15.83:0/2400294879 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5e40106c0 con 0x7fb5ec10e000 2024-08-19T01:35:34.227 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:34.207+0000 7fb5f1800640 1 -- 172.21.15.83:0/2400294879 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb5ec19f500 con 0x7fb5ec10e000 2024-08-19T01:35:34.227 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:34.207+0000 7fb5f1800640 1 -- 172.21.15.83:0/2400294879 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb5ec1a6de0 con 0x7fb5ec10e000 2024-08-19T01:35:34.228 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:34.211+0000 7fb5f59d8740 1 Processor -- start 2024-08-19T01:35:34.228 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:34.212+0000 7fb5e8c00640 1 -- 172.21.15.83:0/2400294879 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 19) ==== 137884+0+0 (secure 0 0 0) 0x7fb5e4003970 con 0x7fb5ec10e000 2024-08-19T01:35:34.228 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:34.212+0000 7fb5f59d8740 1 -- start start 2024-08-19T01:35:34.228 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:34.213+0000 7fb5f59d8740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558cfd1e2ef0 0x558cfd1e32d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:35:34.228 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:34.213+0000 7fb5f59d8740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558cfd1e3810 0x558cfd1edb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:35:34.228 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:34.213+0000 7fb5f59d8740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558cfd1ee0d0 0x558cfd1f04c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:35:34.228 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:34.213+0000 7fb5f59d8740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x558cfd13fbf0 con 0x558cfd1e3810 2024-08-19T01:35:34.228 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:34.213+0000 7fb5f59d8740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x558cfd0d7b20 con 0x558cfd1ee0d0 2024-08-19T01:35:34.228 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:34.213+0000 7fb5f59d8740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x558cfd13bbb0 con 0x558cfd1e2ef0 2024-08-19T01:35:34.228 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:34.214+0000 7fb5ebe00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558cfd1e3810 0x558cfd1edb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:35:34.228 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:34.214+0000 7fb5ebe00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558cfd1e3810 0x558cfd1edb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:58512/0 (socket says 172.21.15.83:58512) 2024-08-19T01:35:34.228 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:34.214+0000 7fb5ebe00640 1 -- 172.21.15.83:0/2953782365 learned_addr learned my addr 172.21.15.83:0/2953782365 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-08-19T01:35:34.228 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:34.214+0000 7fb5eb400640 1 --2- 172.21.15.83:0/2953782365 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558cfd1ee0d0 0x558cfd1f04c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:35:34.229 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:34.214+0000 7fb5e8c00640 1 --2- 172.21.15.83:0/2400294879 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7fb5d00a2d60 0x7fb5d00a5220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:35:34.229 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:34.215+0000 7fb5e8c00640 1 -- 172.21.15.83:0/2400294879 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8269+0+0 (secure 0 0 0) 0x7fb5e40cf380 con 0x7fb5ec10e000 2024-08-19T01:35:34.229 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:34.215+0000 7fb5ebe00640 1 -- 172.21.15.83:0/2953782365 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558cfd1e2ef0 msgr2=0x558cfd1e32d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:35:34.229 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:34.214+0000 7fb5eb400640 1 --2- 172.21.15.83:0/2953782365 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558cfd1e2ef0 0x558cfd1e32d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:35:34.229 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:34.215+0000 7fb5ebe00640 1 --2- 172.21.15.83:0/2953782365 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558cfd1e2ef0 0x558cfd1e32d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:35:34.229 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:34.215+0000 7fb5ebe00640 1 -- 172.21.15.83:0/2953782365 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558cfd1ee0d0 msgr2=0x558cfd1f04c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:35:34.229 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:34.215+0000 7fb5ebe00640 1 --2- 172.21.15.83:0/2953782365 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558cfd1ee0d0 0x558cfd1f04c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:35:34.229 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:34.215+0000 7fb5ebe00640 1 -- 172.21.15.83:0/2953782365 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558cfd0de7f0 con 0x558cfd1e3810 2024-08-19T01:35:34.229 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:34.215+0000 7fb5eb400640 1 --2- 172.21.15.83:0/2953782365 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558cfd1e2ef0 0x558cfd1e32d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-19T01:35:34.229 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:34.215+0000 7fb5ebe00640 1 --2- 172.21.15.83:0/2953782365 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558cfd1e3810 0x558cfd1edb90 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7fb5dc009d80 tx=0x7fb5dc011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=872ea15df0101561 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:35:34.229 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:34.215+0000 7fb5e1600640 1 -- 172.21.15.83:0/2953782365 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5dc01e070 con 0x558cfd1e3810 2024-08-19T01:35:34.229 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:34.216+0000 7fb5f0e00640 1 --2- 172.21.15.83:0/2400294879 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7fb5d00a2d60 0x7fb5d00a5220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:35:34.229 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:34.216+0000 7fb5e1600640 1 -- 172.21.15.83:0/2953782365 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb5dc019b10 con 0x558cfd1e3810 2024-08-19T01:35:34.229 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:34.216+0000 7fb5e1600640 1 -- 172.21.15.83:0/2953782365 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5dc00eb10 con 0x558cfd1e3810 2024-08-19T01:35:34.230 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:34.217+0000 7fb5f59d8740 1 -- 172.21.15.83:0/2953782365 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558cfd1e3810 msgr2=0x558cfd1edb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:35:34.230 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:34.217+0000 7fb5f0e00640 1 --2- 172.21.15.83:0/2400294879 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7fb5d00a2d60 0x7fb5d00a5220 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fb5e400b840 tx=0x7fb5e4004570 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:35:34.230 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:34.217+0000 7fb5f59d8740 1 --2- 172.21.15.83:0/2953782365 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558cfd1e3810 0x558cfd1edb90 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7fb5dc009d80 tx=0x7fb5dc011810 comp rx=0 tx=0).stop 2024-08-19T01:35:34.230 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:34.217+0000 7fb5f59d8740 1 -- 172.21.15.83:0/2953782365 shutdown_connections 2024-08-19T01:35:34.230 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:34.217+0000 7fb5f59d8740 1 --2- 172.21.15.83:0/2953782365 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558cfd1ee0d0 0x558cfd1f04c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:35:34.230 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:34.217+0000 7fb5f59d8740 1 --2- 172.21.15.83:0/2953782365 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558cfd1e3810 0x558cfd1edb90 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:35:34.230 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:34.217+0000 7fb5f59d8740 1 --2- 172.21.15.83:0/2953782365 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558cfd1e2ef0 0x558cfd1e32d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:35:34.230 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:34.217+0000 7fb5f59d8740 1 -- 172.21.15.83:0/2953782365 >> 172.21.15.83:0/2953782365 conn(0x558cfd1df480 msgr2=0x558cfd1df860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:35:34.230 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:34.217+0000 7fb5f59d8740 1 -- 172.21.15.83:0/2953782365 shutdown_connections 2024-08-19T01:35:34.230 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:34.218+0000 7fb5f59d8740 1 -- 172.21.15.83:0/2953782365 wait complete. 2024-08-19T01:35:34.230 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:34.218+0000 7fb5f59d8740 1 -- start start 2024-08-19T01:35:34.230 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:34.218+0000 7fb5f59d8740 1 Processor -- start 2024-08-19T01:35:34.230 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:34.219+0000 7fb5f59d8740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558cfd1e2ef0 0x558cfd1eb030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:35:34.231 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:34.219+0000 7fb5f59d8740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558cfd1e3810 0x558cfd1eb570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:35:34.231 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:34.219+0000 7fb5f59d8740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558cfd1ee0d0 0x558cfd1ebab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:35:34.231 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:34.219+0000 7fb5f59d8740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x558cfd1f0ca0 con 0x558cfd1ee0d0 2024-08-19T01:35:34.231 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:34.219+0000 7fb5f59d8740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x558cfd0facf0 con 0x558cfd1e2ef0 2024-08-19T01:35:34.231 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:34.219+0000 7fb5f59d8740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x558cfd1f0e20 con 0x558cfd1e3810 2024-08-19T01:35:34.231 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:34.219+0000 7fb5f59d8740 10 client.0 _ll_register_callbacks cb 0x558cfd0e6e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-19T01:35:34.231 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:34.220+0000 7fb5eb400640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558cfd1e2ef0 0x558cfd1eb030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:35:34.231 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:34.220+0000 7fb5eb400640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558cfd1e2ef0 0x558cfd1eb030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.83:54186/0 (socket says 172.21.15.83:54186) 2024-08-19T01:35:34.231 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:34.220+0000 7fb5eb400640 1 -- 172.21.15.83:0/1239980579 learned_addr learned my addr 172.21.15.83:0/1239980579 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-08-19T01:35:34.231 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:34.220+0000 7fb5eb400640 1 --2- 172.21.15.83:0/1239980579 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558cfd1ee0d0 0x558cfd1ebab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:35:34.231 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:34.220+0000 7fb5ebe00640 1 --2- 172.21.15.83:0/1239980579 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558cfd1e3810 0x558cfd1eb570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:35:34.231 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:34.220+0000 7fb5eb400640 1 -- 172.21.15.83:0/1239980579 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558cfd1e3810 msgr2=0x558cfd1eb570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:35:34.231 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:34.220+0000 7fb5eb400640 1 --2- 172.21.15.83:0/1239980579 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558cfd1e3810 0x558cfd1eb570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:35:34.232 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:34.220+0000 7fb5eb400640 1 -- 172.21.15.83:0/1239980579 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558cfd1ee0d0 msgr2=0x558cfd1ebab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:35:34.232 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:34.220+0000 7fb5eb400640 1 --2- 172.21.15.83:0/1239980579 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558cfd1ee0d0 0x558cfd1ebab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:35:34.232 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:34.220+0000 7fb5eb400640 1 -- 172.21.15.83:0/1239980579 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558cfceb13c0 con 0x558cfd1e2ef0 2024-08-19T01:35:34.232 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:34.221+0000 7fb5eb400640 1 --2- 172.21.15.83:0/1239980579 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558cfd1e2ef0 0x558cfd1eb030 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fb5d8009860 tx=0x7fb5d80056d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=727610a400a6790f server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:35:34.232 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:34.221+0000 7fb5d6000640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-08-19T01:35:34.232 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:34.221+0000 7fb5d6000640 1 -- 172.21.15.83:0/1239980579 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5d8037040 con 0x558cfd1e2ef0 2024-08-19T01:35:34.232 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:34.221+0000 7fb5f59d8740 1 -- 172.21.15.83:0/1239980579 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb5dc019b10 con 0x558cfd1e2ef0 2024-08-19T01:35:34.232 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:34.221+0000 7fb5d6000640 1 -- 172.21.15.83:0/1239980579 <== mon.2 v2:172.21.15.117:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8269+0+0 (secure 0 0 0) 0x7fb5d8030960 con 0x558cfd1e2ef0 2024-08-19T01:35:34.232 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:34.221+0000 7fb5d6000640 1 -- 172.21.15.83:0/1239980579 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fb5d80035f0 con 0x558cfd1e2ef0 2024-08-19T01:35:34.232 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:34.221+0000 7fb5f59d8740 10 client.25280 fetch_fsmap learned FSMap version 25 2024-08-19T01:35:34.232 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:34.221+0000 7fb5f59d8740 1 -- 172.21.15.83:0/1239980579 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x558cfd1fc940 con 0x558cfd1e2ef0 2024-08-19T01:35:34.232 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:34.222+0000 7fb5d6000640 1 -- 172.21.15.83:0/1239980579 <== mon.2 v2:172.21.15.117:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb5dc019b10 con 0x558cfd1e2ef0 2024-08-19T01:35:34.232 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:34.223+0000 7fb5d6000640 1 -- 172.21.15.83:0/1239980579 <== mon.2 v2:172.21.15.117:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fb5d803d040 con 0x558cfd1e2ef0 2024-08-19T01:35:34.232 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:34.223+0000 7fb5f59d8740 10 client.25280 fetch_fsmap finished waiting for FSMap version 25 2024-08-19T01:35:34.233 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:34.223+0000 7fb5f59d8740 10 client.25280 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-19T01:35:34.233 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:34.223+0000 7fb5f59d8740 20 client.25280 populate_metadata read hostname 'smithi083' 2024-08-19T01:35:34.233 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:34.223+0000 7fb5f59d8740 1 --2- 172.21.15.83:0/1239980579 >> [v2:172.21.15.117:6838/1967595704,v1:172.21.15.117:6839/1967595704] conn(0x558cfd1fde00 0x558cfd2001f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:35:34.233 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:34.223+0000 7fb5f59d8740 4 client.25280 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-19T01:35:34.233 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:34.223+0000 7fb5f59d8740 1 -- 172.21.15.83:0/1239980579 --> [v2:172.21.15.117:6838/1967595704,v1:172.21.15.117:6839/1967595704] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558cfd14d160 con 0x558cfd1fde00 2024-08-19T01:35:34.233 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:34.223+0000 7fb5ebe00640 1 --2- 172.21.15.83:0/1239980579 >> [v2:172.21.15.117:6838/1967595704,v1:172.21.15.117:6839/1967595704] conn(0x558cfd1fde00 0x558cfd2001f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:35:34.233 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:34.224+0000 7fb5ebe00640 1 --2- 172.21.15.83:0/1239980579 >> [v2:172.21.15.117:6838/1967595704,v1:172.21.15.117:6839/1967595704] conn(0x558cfd1fde00 0x558cfd2001f0 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=21bd8bb6bf496c29 server_cookie=201a378131fbb284 in_seq=0 out_seq=0 2024-08-19T01:35:34.233 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:34.224+0000 7fb5d6000640 10 client.25280 ms_handle_connect on v2:172.21.15.117:6838/1967595704 2024-08-19T01:35:34.234 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:34.225+0000 7fb5d6000640 1 -- 172.21.15.83:0/1239980579 <== mds.0 v2:172.21.15.117:6838/1967595704 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x558cfd14d160 con 0x558cfd1fde00 2024-08-19T01:35:34.234 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:34.225+0000 7fb5d6000640 10 client.25280 handle_command_reply: tid=0 2024-08-19T01:35:34.234 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:34.225+0000 7fb5f59d8740 1 client.25280 shutdown 2024-08-19T01:35:34.234 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:34.225+0000 7fb5f59d8740 1 -- 172.21.15.83:0/1239980579 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558cfd1e2ef0 msgr2=0x558cfd1eb030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:35:34.234 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:34.225+0000 7fb5f59d8740 1 --2- 172.21.15.83:0/1239980579 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558cfd1e2ef0 0x558cfd1eb030 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fb5d8009860 tx=0x7fb5d80056d0 comp rx=0 tx=0).stop 2024-08-19T01:35:34.234 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:34.225+0000 7fb5f59d8740 1 -- 172.21.15.83:0/1239980579 shutdown_connections 2024-08-19T01:35:34.235 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:34.225+0000 7fb5f59d8740 1 --2- 172.21.15.83:0/1239980579 >> [v2:172.21.15.117:6838/1967595704,v1:172.21.15.117:6839/1967595704] conn(0x558cfd1fde00 0x558cfd2001f0 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:35:34.235 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:34.226+0000 7fb5f59d8740 1 --2- 172.21.15.83:0/1239980579 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558cfd1ee0d0 0x558cfd1ebab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:35:34.235 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:34.226+0000 7fb5f59d8740 1 --2- 172.21.15.83:0/1239980579 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558cfd1e3810 0x558cfd1eb570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:35:34.235 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:34.226+0000 7fb5f59d8740 1 --2- 172.21.15.83:0/1239980579 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558cfd1e2ef0 0x558cfd1eb030 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:35:34.235 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:34.226+0000 7fb5f59d8740 1 -- 172.21.15.83:0/1239980579 >> 172.21.15.83:0/1239980579 conn(0x558cfd1df480 msgr2=0x558cfd1e5160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:35:34.235 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:34.226+0000 7fb5d6000640 0 client.25280 ms_handle_reset on v2:172.21.15.117:6838/1967595704 2024-08-19T01:35:34.235 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:34.226+0000 7fb5f59d8740 1 -- 172.21.15.83:0/1239980579 shutdown_connections 2024-08-19T01:35:34.235 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:34.226+0000 7fb5f59d8740 1 -- 172.21.15.83:0/1239980579 wait complete. 2024-08-19T01:35:34.235 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:34.226+0000 7fb5f59d8740 20 client.25280 trim_cache size 0 max 16384 2024-08-19T01:35:34.254 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:34.253+0000 7fb5f59d8740 1 Processor -- start 2024-08-19T01:35:34.254 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:34.253+0000 7fb5f59d8740 1 -- start start 2024-08-19T01:35:34.255 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:34.253+0000 7fb5f59d8740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558cfd1e2ef0 0x558cfd1e5950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:35:34.256 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:34.253+0000 7fb5f59d8740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558cfd1e3810 0x558cfd1d1910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:35:34.256 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:34.253+0000 7fb5f59d8740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558cfd1ee0d0 0x558cfd1ecfa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:35:34.256 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:34.253+0000 7fb5f59d8740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fb5ec1ad9b0 con 0x558cfd1ee0d0 2024-08-19T01:35:34.256 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:34.253+0000 7fb5f59d8740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fb5ec1ab280 con 0x558cfd1e3810 2024-08-19T01:35:34.256 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:34.253+0000 7fb5f59d8740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fb5ec1a90d0 con 0x558cfd1e2ef0 2024-08-19T01:35:34.256 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:34.254+0000 7fb5ebe00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558cfd1e3810 0x558cfd1d1910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:35:34.257 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:34.254+0000 7fb5ebe00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558cfd1e3810 0x558cfd1d1910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.83:54188/0 (socket says 172.21.15.83:54188) 2024-08-19T01:35:34.257 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:34.254+0000 7fb5ebe00640 1 -- 172.21.15.83:0/761982247 learned_addr learned my addr 172.21.15.83:0/761982247 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-08-19T01:35:34.257 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:34.254+0000 7fb5eb400640 1 --2- 172.21.15.83:0/761982247 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558cfd1ee0d0 0x558cfd1ecfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:35:34.257 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:34.254+0000 7fb5eb400640 1 --2- 172.21.15.83:0/761982247 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558cfd1e2ef0 0x558cfd1e5950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:35:34.257 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:34.254+0000 7fb5ebe00640 1 -- 172.21.15.83:0/761982247 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558cfd1e2ef0 msgr2=0x558cfd1e5950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:35:34.257 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:34.254+0000 7fb5ebe00640 1 --2- 172.21.15.83:0/761982247 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558cfd1e2ef0 0x558cfd1e5950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:35:34.257 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:34.254+0000 7fb5ebe00640 1 -- 172.21.15.83:0/761982247 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558cfd1ee0d0 msgr2=0x558cfd1ecfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:35:34.257 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:34.254+0000 7fb5ebe00640 1 --2- 172.21.15.83:0/761982247 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558cfd1ee0d0 0x558cfd1ecfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:35:34.257 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:34.254+0000 7fb5ebe00640 1 -- 172.21.15.83:0/761982247 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558cfd200560 con 0x558cfd1e3810 2024-08-19T01:35:34.257 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:34.254+0000 7fb5eb400640 1 --2- 172.21.15.83:0/761982247 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558cfd1ee0d0 0x558cfd1ecfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-19T01:35:34.257 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:34.255+0000 7fb5ebe00640 1 --2- 172.21.15.83:0/761982247 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558cfd1e3810 0x558cfd1d1910 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fb5dc00e420 tx=0x7fb5dc0097b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9b074d2a0c9b36b4 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:35:34.258 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:34.255+0000 7fb5e0c00640 1 -- 172.21.15.83:0/761982247 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5dc01e040 con 0x558cfd1e3810 2024-08-19T01:35:34.258 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:34.255+0000 7fb5e0c00640 1 -- 172.21.15.83:0/761982247 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb5dc019430 con 0x558cfd1e3810 2024-08-19T01:35:34.258 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:34.255+0000 7fb5e0c00640 1 -- 172.21.15.83:0/761982247 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5dc003850 con 0x558cfd1e3810 2024-08-19T01:35:34.258 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:34.256+0000 7fb5f59d8740 1 -- 172.21.15.83:0/761982247 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558cfd1e3810 msgr2=0x558cfd1d1910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:35:34.259 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:34.256+0000 7fb5f59d8740 1 --2- 172.21.15.83:0/761982247 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558cfd1e3810 0x558cfd1d1910 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fb5dc00e420 tx=0x7fb5dc0097b0 comp rx=0 tx=0).stop 2024-08-19T01:35:34.259 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:34.257+0000 7fb5f59d8740 1 -- 172.21.15.83:0/761982247 shutdown_connections 2024-08-19T01:35:34.259 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:34.257+0000 7fb5f59d8740 1 --2- 172.21.15.83:0/761982247 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558cfd1ee0d0 0x558cfd1ecfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:35:34.259 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:34.257+0000 7fb5f59d8740 1 --2- 172.21.15.83:0/761982247 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558cfd1e3810 0x558cfd1d1910 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:35:34.259 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:34.257+0000 7fb5f59d8740 1 --2- 172.21.15.83:0/761982247 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558cfd1e2ef0 0x558cfd1e5950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:35:34.259 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:34.257+0000 7fb5f59d8740 1 -- 172.21.15.83:0/761982247 >> 172.21.15.83:0/761982247 conn(0x558cfd1df480 msgr2=0x558cfd1ec890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:35:34.259 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:34.257+0000 7fb5f59d8740 1 -- 172.21.15.83:0/761982247 shutdown_connections 2024-08-19T01:35:34.260 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:34.257+0000 7fb5f59d8740 1 -- 172.21.15.83:0/761982247 wait complete. 2024-08-19T01:35:34.260 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:34.257+0000 7fb5f59d8740 1 -- start start 2024-08-19T01:35:34.260 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:34.258+0000 7fb5f59d8740 1 Processor -- start 2024-08-19T01:35:34.260 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:34.259+0000 7fb5f59d8740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558cfd1e2ef0 0x558cfd1f6250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:35:34.260 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:34.259+0000 7fb5f59d8740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558cfd1e3810 0x558cfd1f6790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:35:34.261 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:34.259+0000 7fb5f59d8740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558cfd1ee0d0 0x558cfd1e7860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:35:34.261 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:34.259+0000 7fb5f59d8740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fb5ec1a9740 con 0x558cfd1ee0d0 2024-08-19T01:35:34.261 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:34.259+0000 7fb5f59d8740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fb5ec1aabb0 con 0x558cfd1e2ef0 2024-08-19T01:35:34.261 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:34.259+0000 7fb5f59d8740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fb5ec1a9e10 con 0x558cfd1e3810 2024-08-19T01:35:34.261 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:34.259+0000 7fb5f59d8740 10 client.0 _ll_register_callbacks cb 0x7fb5d8004520 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-19T01:35:34.263 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:34.259+0000 7fb5eb400640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558cfd1e2ef0 0x558cfd1f6250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:35:34.263 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:34.259+0000 7fb5eb400640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558cfd1e2ef0 0x558cfd1f6250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.83:54190/0 (socket says 172.21.15.83:54190) 2024-08-19T01:35:34.263 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:34.259+0000 7fb5eb400640 1 -- 172.21.15.83:0/968262782 learned_addr learned my addr 172.21.15.83:0/968262782 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-08-19T01:35:34.263 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:34.259+0000 7fb5eb400640 1 --2- 172.21.15.83:0/968262782 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558cfd1ee0d0 0x558cfd1e7860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:35:34.263 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:34.260+0000 7fb5ebe00640 1 --2- 172.21.15.83:0/968262782 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558cfd1e3810 0x558cfd1f6790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:35:34.263 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:34.260+0000 7fb5eb400640 1 -- 172.21.15.83:0/968262782 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558cfd1e3810 msgr2=0x558cfd1f6790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:35:34.263 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:34.260+0000 7fb5eb400640 1 --2- 172.21.15.83:0/968262782 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558cfd1e3810 0x558cfd1f6790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:35:34.263 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:34.260+0000 7fb5eb400640 1 -- 172.21.15.83:0/968262782 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558cfd1ee0d0 msgr2=0x558cfd1e7860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:35:34.263 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:34.260+0000 7fb5eb400640 1 --2- 172.21.15.83:0/968262782 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558cfd1ee0d0 0x558cfd1e7860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:35:34.264 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:34.260+0000 7fb5eb400640 1 -- 172.21.15.83:0/968262782 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558cfd1e7ef0 con 0x558cfd1e2ef0 2024-08-19T01:35:34.264 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:34.260+0000 7fb5ebe00640 1 --2- 172.21.15.83:0/968262782 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558cfd1e3810 0x558cfd1f6790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-19T01:35:34.264 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:34.260+0000 7fb5eb400640 1 --2- 172.21.15.83:0/968262782 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558cfd1e2ef0 0x558cfd1f6250 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fb5d8004650 tx=0x7fb5d8000bc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f74b8faf26622708 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:35:34.264 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:34.260+0000 7fb5d6000640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-08-19T01:35:34.264 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:34.261+0000 7fb5d6000640 1 -- 172.21.15.83:0/968262782 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5d8037040 con 0x558cfd1e2ef0 2024-08-19T01:35:34.264 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:34.261+0000 7fb5d6000640 1 -- 172.21.15.83:0/968262782 <== mon.2 v2:172.21.15.117:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8269+0+0 (secure 0 0 0) 0x7fb5d8003ba0 con 0x558cfd1e2ef0 2024-08-19T01:35:34.264 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:34.261+0000 7fb5f59d8740 1 -- 172.21.15.83:0/968262782 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb5dc019430 con 0x558cfd1e2ef0 2024-08-19T01:35:34.264 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:34.261+0000 7fb5d6000640 1 -- 172.21.15.83:0/968262782 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fb5d8003da0 con 0x558cfd1e2ef0 2024-08-19T01:35:34.264 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:34.261+0000 7fb5f59d8740 10 client.25286 fetch_fsmap learned FSMap version 25 2024-08-19T01:35:34.264 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:34.261+0000 7fb5f59d8740 1 -- 172.21.15.83:0/968262782 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x558cfd2683c0 con 0x558cfd1e2ef0 2024-08-19T01:35:34.264 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:34.261+0000 7fb5d6000640 1 -- 172.21.15.83:0/968262782 <== mon.2 v2:172.21.15.117:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb5dc019430 con 0x558cfd1e2ef0 2024-08-19T01:35:34.265 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:34.262+0000 7fb5d6000640 1 -- 172.21.15.83:0/968262782 <== mon.2 v2:172.21.15.117:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fb5d803d070 con 0x558cfd1e2ef0 2024-08-19T01:35:34.265 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:34.262+0000 7fb5f59d8740 10 client.25286 fetch_fsmap finished waiting for FSMap version 25 2024-08-19T01:35:34.266 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:34.262+0000 7fb5f59d8740 10 client.25286 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-19T01:35:34.266 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:34.262+0000 7fb5f59d8740 20 client.25286 populate_metadata read hostname 'smithi083' 2024-08-19T01:35:34.266 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:34.262+0000 7fb5f59d8740 1 --2- 172.21.15.83:0/968262782 >> [v2:172.21.15.117:6838/1967595704,v1:172.21.15.117:6839/1967595704] conn(0x558cfd268a10 0x558cfd26ae30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:35:34.266 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:34.262+0000 7fb5f59d8740 4 client.25286 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-08-19T01:35:34.266 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:34.262+0000 7fb5f59d8740 1 -- 172.21.15.83:0/968262782 --> [v2:172.21.15.117:6838/1967595704,v1:172.21.15.117:6839/1967595704] -- command(tid 0: {"prefix": "damage ls"}) -- 0x558cfd147670 con 0x558cfd268a10 2024-08-19T01:35:34.266 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:34.263+0000 7fb5ebe00640 1 --2- 172.21.15.83:0/968262782 >> [v2:172.21.15.117:6838/1967595704,v1:172.21.15.117:6839/1967595704] conn(0x558cfd268a10 0x558cfd26ae30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:35:34.266 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:34.263+0000 7fb5ebe00640 1 --2- 172.21.15.83:0/968262782 >> [v2:172.21.15.117:6838/1967595704,v1:172.21.15.117:6839/1967595704] conn(0x558cfd268a10 0x558cfd26ae30 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=806ff0bc688e19a1 server_cookie=3fa839057bcfd5b in_seq=0 out_seq=0 2024-08-19T01:35:34.267 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:34.263+0000 7fb5d6000640 10 client.25286 ms_handle_connect on v2:172.21.15.117:6838/1967595704 2024-08-19T01:35:34.267 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:34.263+0000 7fb5d6000640 1 -- 172.21.15.83:0/968262782 <== mds.0 v2:172.21.15.117:6838/1967595704 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x558cfd147670 con 0x558cfd268a10 2024-08-19T01:35:34.267 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:34.263+0000 7fb5d6000640 10 client.25286 handle_command_reply: tid=0 2024-08-19T01:35:34.267 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:34.264+0000 7fb5f59d8740 1 client.25286 shutdown 2024-08-19T01:35:34.267 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:34.264+0000 7fb5f59d8740 1 -- 172.21.15.83:0/968262782 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558cfd1e2ef0 msgr2=0x558cfd1f6250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:35:34.267 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:34.264+0000 7fb5f59d8740 1 --2- 172.21.15.83:0/968262782 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558cfd1e2ef0 0x558cfd1f6250 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fb5d8004650 tx=0x7fb5d8000bc0 comp rx=0 tx=0).stop 2024-08-19T01:35:34.268 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:34.264+0000 7fb5f59d8740 1 -- 172.21.15.83:0/968262782 shutdown_connections 2024-08-19T01:35:34.268 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:34.264+0000 7fb5f59d8740 1 --2- 172.21.15.83:0/968262782 >> [v2:172.21.15.117:6838/1967595704,v1:172.21.15.117:6839/1967595704] conn(0x558cfd268a10 0x558cfd26ae30 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:35:34.268 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:34.264+0000 7fb5f59d8740 1 --2- 172.21.15.83:0/968262782 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558cfd1ee0d0 0x558cfd1e7860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:35:34.268 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:34.264+0000 7fb5f59d8740 1 --2- 172.21.15.83:0/968262782 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558cfd1e3810 0x558cfd1f6790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:35:34.268 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:34.264+0000 7fb5f59d8740 1 --2- 172.21.15.83:0/968262782 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558cfd1e2ef0 0x558cfd1f6250 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:35:34.268 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:34.264+0000 7fb5f59d8740 1 -- 172.21.15.83:0/968262782 >> 172.21.15.83:0/968262782 conn(0x558cfd1df480 msgr2=0x558cfd1d5ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:35:34.268 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:34.264+0000 7fb5d6000640 0 client.25286 ms_handle_reset on v2:172.21.15.117:6838/1967595704 2024-08-19T01:35:34.268 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:34.265+0000 7fb5f59d8740 1 -- 172.21.15.83:0/968262782 shutdown_connections 2024-08-19T01:35:34.268 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:34.265+0000 7fb5f59d8740 1 -- 172.21.15.83:0/968262782 wait complete. 2024-08-19T01:35:34.268 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:34.265+0000 7fb5f59d8740 20 client.25286 trim_cache size 0 max 16384 2024-08-19T01:35:34.269 INFO:teuthology.orchestra.run.smithi083.stdout:[] 2024-08-19T01:35:34.269 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:34.266+0000 7fb5f1800640 1 -- 172.21.15.83:0/2400294879 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7fb5d00a2d60 msgr2=0x7fb5d00a5220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:35:34.269 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:34.266+0000 7fb5f1800640 1 --2- 172.21.15.83:0/2400294879 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7fb5d00a2d60 0x7fb5d00a5220 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fb5e400b840 tx=0x7fb5e4004570 comp rx=0 tx=0).stop 2024-08-19T01:35:34.269 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:34.266+0000 7fb5f1800640 1 -- 172.21.15.83:0/2400294879 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb5ec10e000 msgr2=0x7fb5ec19eb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:35:34.269 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:34.266+0000 7fb5f1800640 1 --2- 172.21.15.83:0/2400294879 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb5ec10e000 0x7fb5ec19eb30 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7fb5ec102630 tx=0x7fb5e400dea0 comp rx=0 tx=0).stop 2024-08-19T01:35:34.269 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:34.266+0000 7fb5f1800640 1 -- 172.21.15.83:0/2400294879 shutdown_connections 2024-08-19T01:35:34.269 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:34.266+0000 7fb5f1800640 1 --2- 172.21.15.83:0/2400294879 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7fb5d00a2d60 0x7fb5d00a5220 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:35:34.270 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:34.266+0000 7fb5f1800640 1 --2- 172.21.15.83:0/2400294879 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb5ec10e000 0x7fb5ec19eb30 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:35:34.270 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:34.266+0000 7fb5f1800640 1 --2- 172.21.15.83:0/2400294879 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb5ec100e00 0x7fb5ec19a7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:35:34.270 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:34.266+0000 7fb5f1800640 1 --2- 172.21.15.83:0/2400294879 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb5ec100450 0x7fb5ec19a260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:35:34.270 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:34.266+0000 7fb5f1800640 1 -- 172.21.15.83:0/2400294879 >> 172.21.15.83:0/2400294879 conn(0x7fb5ec0fc340 msgr2=0x7fb5ec0fe730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:35:34.270 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:34.266+0000 7fb5f1800640 1 -- 172.21.15.83:0/2400294879 shutdown_connections 2024-08-19T01:35:34.270 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:34.266+0000 7fb5f1800640 1 -- 172.21.15.83:0/2400294879 wait complete. 2024-08-19T01:35:35.654 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:35:35 smithi083 ceph-mon[17120]: pgmap v361: 97 pgs: 97 active+clean; 667 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 1.2 KiB/s wr, 7 op/s 2024-08-19T01:35:35.952 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:35:35 smithi117 ceph-mon[22016]: pgmap v361: 97 pgs: 97 active+clean; 667 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 1.2 KiB/s wr, 7 op/s 2024-08-19T01:35:35.966 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:35:35 smithi167 ceph-mon[21875]: pgmap v361: 97 pgs: 97 active+clean; 667 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 1.2 KiB/s wr, 7 op/s 2024-08-19T01:35:36.327 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-08-19T01:35:36.328 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833 shell --fsid bad8780e-5dc9-11ef-bccf-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-08-19T01:35:36.746 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/bad8780e-5dc9-11ef-bccf-c7b262605968/mon.a/config 2024-08-19T01:35:37.409 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:37.407+0000 7fd24e200640 1 -- 172.21.15.83:0/93531351 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd248109ac0 msgr2=0x7fd248109ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:35:37.409 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:37.407+0000 7fd24e200640 1 --2- 172.21.15.83:0/93531351 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd248109ac0 0x7fd248109ea0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7fd238009930 tx=0x7fd23802a420 comp rx=0 tx=0).stop 2024-08-19T01:35:37.410 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:37.407+0000 7fd24e200640 1 -- 172.21.15.83:0/93531351 shutdown_connections 2024-08-19T01:35:37.410 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:37.407+0000 7fd24e200640 1 --2- 172.21.15.83:0/93531351 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd248104460 0x7fd248110990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:35:37.410 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:37.407+0000 7fd24e200640 1 --2- 172.21.15.83:0/93531351 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd248103ac0 0x7fd248103f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:35:37.410 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:37.407+0000 7fd24e200640 1 --2- 172.21.15.83:0/93531351 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd248109ac0 0x7fd248109ea0 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:35:37.410 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:37.407+0000 7fd24e200640 1 -- 172.21.15.83:0/93531351 >> 172.21.15.83:0/93531351 conn(0x7fd2480ff7c0 msgr2=0x7fd248101be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:35:37.410 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:37.408+0000 7fd24e200640 1 -- 172.21.15.83:0/93531351 shutdown_connections 2024-08-19T01:35:37.410 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:37.408+0000 7fd24e200640 1 -- 172.21.15.83:0/93531351 wait complete. 2024-08-19T01:35:37.410 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:37.409+0000 7fd24e200640 1 Processor -- start 2024-08-19T01:35:37.411 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:37.409+0000 7fd24e200640 1 -- start start 2024-08-19T01:35:37.411 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:37.410+0000 7fd24e200640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd248103ac0 0x7fd24819a3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:35:37.412 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:37.410+0000 7fd24e200640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd248104460 0x7fd24819a8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:35:37.412 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:37.410+0000 7fd24e200640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd248109ac0 0x7fd24819ec80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:35:37.412 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:37.410+0000 7fd24e200640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fd248113f10 con 0x7fd248104460 2024-08-19T01:35:37.412 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:37.410+0000 7fd24e200640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fd248113d90 con 0x7fd248109ac0 2024-08-19T01:35:37.412 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:37.410+0000 7fd24e200640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fd248114090 con 0x7fd248103ac0 2024-08-19T01:35:37.412 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:37.410+0000 7fd247400640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd248103ac0 0x7fd24819a3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:35:37.412 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:37.410+0000 7fd247400640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd248103ac0 0x7fd24819a3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.83:49378/0 (socket says 172.21.15.83:49378) 2024-08-19T01:35:37.412 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:37.410+0000 7fd247400640 1 -- 172.21.15.83:0/1685393616 learned_addr learned my addr 172.21.15.83:0/1685393616 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-08-19T01:35:37.413 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:37.410+0000 7fd246a00640 1 --2- 172.21.15.83:0/1685393616 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd248104460 0x7fd24819a8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:35:37.413 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:37.411+0000 7fd247e00640 1 --2- 172.21.15.83:0/1685393616 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd248109ac0 0x7fd24819ec80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:35:37.413 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:37.411+0000 7fd247400640 1 -- 172.21.15.83:0/1685393616 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd248109ac0 msgr2=0x7fd24819ec80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:35:37.413 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:37.411+0000 7fd247400640 1 --2- 172.21.15.83:0/1685393616 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd248109ac0 0x7fd24819ec80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:35:37.413 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:37.411+0000 7fd247400640 1 -- 172.21.15.83:0/1685393616 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd248104460 msgr2=0x7fd24819a8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:35:37.413 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:37.411+0000 7fd247400640 1 --2- 172.21.15.83:0/1685393616 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd248104460 0x7fd24819a8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:35:37.413 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:37.411+0000 7fd247400640 1 -- 172.21.15.83:0/1685393616 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd24819f360 con 0x7fd248103ac0 2024-08-19T01:35:37.414 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:37.411+0000 7fd247e00640 1 --2- 172.21.15.83:0/1685393616 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd248109ac0 0x7fd24819ec80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-19T01:35:37.414 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:37.411+0000 7fd247400640 1 --2- 172.21.15.83:0/1685393616 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd248103ac0 0x7fd24819a3b0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fd23802a940 tx=0x7fd238003b20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:35:37.414 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:37.412+0000 7fd23fe00640 1 -- 172.21.15.83:0/1685393616 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd23803f070 con 0x7fd248103ac0 2024-08-19T01:35:37.414 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:37.412+0000 7fd23fe00640 1 -- 172.21.15.83:0/1685393616 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd238031c90 con 0x7fd248103ac0 2024-08-19T01:35:37.415 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:37.412+0000 7fd23fe00640 1 -- 172.21.15.83:0/1685393616 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd238035030 con 0x7fd248103ac0 2024-08-19T01:35:37.415 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:37.412+0000 7fd24e200640 1 -- 172.21.15.83:0/1685393616 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd24819f5f0 con 0x7fd248103ac0 2024-08-19T01:35:37.415 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:37.412+0000 7fd24e200640 1 -- 172.21.15.83:0/1685393616 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd24819fa80 con 0x7fd248103ac0 2024-08-19T01:35:37.417 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:37.416+0000 7fd23fe00640 1 -- 172.21.15.83:0/1685393616 <== mon.1 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 19) ==== 137884+0+0 (secure 0 0 0) 0x7fd238042070 con 0x7fd248103ac0 2024-08-19T01:35:37.419 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:37.418+0000 7fd250e78740 1 Processor -- start 2024-08-19T01:35:37.419 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:37.418+0000 7fd23fe00640 1 --2- 172.21.15.83:0/1685393616 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7fd22c0a2c10 0x7fd22c0a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:35:37.420 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:37.418+0000 7fd250e78740 1 -- start start 2024-08-19T01:35:37.420 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:37.418+0000 7fd23fe00640 1 -- 172.21.15.83:0/1685393616 <== mon.1 v2:172.21.15.167:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8269+0+0 (secure 0 0 0) 0x7fd2380ef8e0 con 0x7fd248103ac0 2024-08-19T01:35:37.420 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:37.418+0000 7fd247e00640 1 --2- 172.21.15.83:0/1685393616 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7fd22c0a2c10 0x7fd22c0a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:35:37.420 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:37.418+0000 7fd250e78740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ec4e956c90 0x55ec4e957070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:35:37.420 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:37.419+0000 7fd250e78740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ec4e9575b0 0x55ec4e961930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:35:37.421 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:37.419+0000 7fd250e78740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ec4e961e70 0x55ec4e964260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:35:37.421 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:37.419+0000 7fd250e78740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55ec4e8697e0 con 0x55ec4e961e70 2024-08-19T01:35:37.421 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:37.419+0000 7fd250e78740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55ec4e8b4ac0 con 0x55ec4e956c90 2024-08-19T01:35:37.421 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:37.419+0000 7fd250e78740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55ec4e84c8c0 con 0x55ec4e9575b0 2024-08-19T01:35:37.422 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:37.419+0000 7fd247400640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ec4e956c90 0x55ec4e957070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:35:37.422 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:37.419+0000 7fd246a00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ec4e9575b0 0x55ec4e961930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:35:37.422 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:37.419+0000 7fd246a00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ec4e9575b0 0x55ec4e961930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.83:49388/0 (socket says 172.21.15.83:49388) 2024-08-19T01:35:37.423 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:37.419+0000 7fd246a00640 1 -- 172.21.15.83:0/2102695151 learned_addr learned my addr 172.21.15.83:0/2102695151 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-08-19T01:35:37.423 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:37.420+0000 7fd247e00640 1 --2- 172.21.15.83:0/1685393616 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7fd22c0a2c10 0x7fd22c0a50d0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fd240005fd0 tx=0x7fd24000f730 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:35:37.423 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:37.420+0000 7fd247e00640 1 --2- 172.21.15.83:0/2102695151 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ec4e961e70 0x55ec4e964260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:35:37.423 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:37.420+0000 7fd247400640 1 -- 172.21.15.83:0/2102695151 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ec4e9575b0 msgr2=0x55ec4e961930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:35:37.423 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:37.420+0000 7fd247400640 1 --2- 172.21.15.83:0/2102695151 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ec4e9575b0 0x55ec4e961930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:35:37.423 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:37.420+0000 7fd247400640 1 -- 172.21.15.83:0/2102695151 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ec4e961e70 msgr2=0x55ec4e964260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:35:37.423 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:37.420+0000 7fd247400640 1 --2- 172.21.15.83:0/2102695151 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ec4e961e70 0x55ec4e964260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:35:37.423 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:37.420+0000 7fd247400640 1 -- 172.21.15.83:0/2102695151 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ec4e853760 con 0x55ec4e956c90 2024-08-19T01:35:37.423 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:37.420+0000 7fd247e00640 1 --2- 172.21.15.83:0/2102695151 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ec4e961e70 0x55ec4e964260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-19T01:35:37.423 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:37.420+0000 7fd247400640 1 --2- 172.21.15.83:0/2102695151 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ec4e956c90 0x55ec4e957070 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fd2380ed210 tx=0x7fd238009260 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c26468b8129fc513 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:35:37.423 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:37.421+0000 7fd233e00640 1 -- 172.21.15.83:0/2102695151 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2380bbf60 con 0x55ec4e956c90 2024-08-19T01:35:37.423 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:37.421+0000 7fd233e00640 1 -- 172.21.15.83:0/2102695151 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd2380c0310 con 0x55ec4e956c90 2024-08-19T01:35:37.424 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:37.421+0000 7fd233e00640 1 -- 172.21.15.83:0/2102695151 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2380c4740 con 0x55ec4e956c90 2024-08-19T01:35:37.424 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:37.421+0000 7fd250e78740 1 -- 172.21.15.83:0/2102695151 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ec4e956c90 msgr2=0x55ec4e957070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:35:37.424 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:37.421+0000 7fd250e78740 1 --2- 172.21.15.83:0/2102695151 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ec4e956c90 0x55ec4e957070 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fd2380ed210 tx=0x7fd238009260 comp rx=0 tx=0).stop 2024-08-19T01:35:37.425 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:37.422+0000 7fd250e78740 1 -- 172.21.15.83:0/2102695151 shutdown_connections 2024-08-19T01:35:37.425 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:37.422+0000 7fd250e78740 1 --2- 172.21.15.83:0/2102695151 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ec4e961e70 0x55ec4e964260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:35:37.425 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:37.422+0000 7fd250e78740 1 --2- 172.21.15.83:0/2102695151 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ec4e9575b0 0x55ec4e961930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:35:37.425 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:37.422+0000 7fd250e78740 1 --2- 172.21.15.83:0/2102695151 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ec4e956c90 0x55ec4e957070 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:35:37.425 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:37.422+0000 7fd250e78740 1 -- 172.21.15.83:0/2102695151 >> 172.21.15.83:0/2102695151 conn(0x55ec4e953220 msgr2=0x55ec4e953600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:35:37.425 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:37.422+0000 7fd250e78740 1 -- 172.21.15.83:0/2102695151 shutdown_connections 2024-08-19T01:35:37.425 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:37.422+0000 7fd250e78740 1 -- 172.21.15.83:0/2102695151 wait complete. 2024-08-19T01:35:37.425 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:37.423+0000 7fd250e78740 1 -- start start 2024-08-19T01:35:37.425 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:37.423+0000 7fd250e78740 1 Processor -- start 2024-08-19T01:35:37.426 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:37.423+0000 7fd250e78740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ec4e956c90 0x55ec4e95eeb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:35:37.426 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:37.423+0000 7fd250e78740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ec4e9575b0 0x55ec4e95f3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:35:37.426 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:37.423+0000 7fd250e78740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ec4e961e70 0x55ec4e95f930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:35:37.426 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:37.424+0000 7fd250e78740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55ec4e86fae0 con 0x55ec4e961e70 2024-08-19T01:35:37.426 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:37.424+0000 7fd250e78740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55ec4e8b0a80 con 0x55ec4e956c90 2024-08-19T01:35:37.426 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:37.424+0000 7fd250e78740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55ec4e964800 con 0x55ec4e9575b0 2024-08-19T01:35:37.427 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:37.424+0000 7fd250e78740 10 client.0 _ll_register_callbacks cb 0x55ec4e871190 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-19T01:35:37.427 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:37.424+0000 7fd247400640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ec4e956c90 0x55ec4e95eeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:35:37.427 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:37.424+0000 7fd247400640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ec4e956c90 0x55ec4e95eeb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.83:54234/0 (socket says 172.21.15.83:54234) 2024-08-19T01:35:37.427 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:37.424+0000 7fd247400640 1 -- 172.21.15.83:0/1535900093 learned_addr learned my addr 172.21.15.83:0/1535900093 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-08-19T01:35:37.427 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:37.424+0000 7fd246a00640 1 --2- 172.21.15.83:0/1535900093 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ec4e9575b0 0x55ec4e95f3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:35:37.427 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:37.424+0000 7fd247400640 1 -- 172.21.15.83:0/1535900093 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ec4e9575b0 msgr2=0x55ec4e95f3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:35:37.427 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:37.424+0000 7fd247400640 1 --2- 172.21.15.83:0/1535900093 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ec4e9575b0 0x55ec4e95f3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:35:37.427 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:37.424+0000 7fd247400640 1 -- 172.21.15.83:0/1535900093 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ec4e961e70 msgr2=0x55ec4e95f930 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-08-19T01:35:37.427 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:37.424+0000 7fd247400640 1 --2- 172.21.15.83:0/1535900093 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ec4e961e70 0x55ec4e95f930 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:35:37.428 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:37.424+0000 7fd247400640 1 -- 172.21.15.83:0/1535900093 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ec4e6266a0 con 0x55ec4e956c90 2024-08-19T01:35:37.428 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:37.425+0000 7fd246a00640 1 --2- 172.21.15.83:0/1535900093 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ec4e9575b0 0x55ec4e95f3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-19T01:35:37.428 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:37.425+0000 7fd247400640 1 --2- 172.21.15.83:0/1535900093 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ec4e956c90 0x55ec4e95eeb0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fd2380bbc60 tx=0x7fd2380ed690 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2e6785a763520bd1 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:35:37.428 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:37.425+0000 7fd230c00640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-08-19T01:35:37.429 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:37.425+0000 7fd230c00640 1 -- 172.21.15.83:0/1535900093 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2380c4bd0 con 0x55ec4e956c90 2024-08-19T01:35:37.429 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:37.425+0000 7fd230c00640 1 -- 172.21.15.83:0/1535900093 <== mon.2 v2:172.21.15.117:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8269+0+0 (secure 0 0 0) 0x7fd23803b460 con 0x55ec4e956c90 2024-08-19T01:35:37.429 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:37.425+0000 7fd250e78740 1 -- 172.21.15.83:0/1535900093 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd2380c0310 con 0x55ec4e956c90 2024-08-19T01:35:37.429 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:37.425+0000 7fd230c00640 1 -- 172.21.15.83:0/1535900093 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fd2380c4740 con 0x55ec4e956c90 2024-08-19T01:35:37.429 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:37.425+0000 7fd250e78740 10 client.25295 fetch_fsmap learned FSMap version 25 2024-08-19T01:35:37.430 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:37.425+0000 7fd250e78740 1 -- 172.21.15.83:0/1535900093 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55ec4e9703e0 con 0x55ec4e956c90 2024-08-19T01:35:37.430 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:37.426+0000 7fd230c00640 1 -- 172.21.15.83:0/1535900093 <== mon.2 v2:172.21.15.117:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd2380c0310 con 0x55ec4e956c90 2024-08-19T01:35:37.430 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:37.426+0000 7fd230c00640 1 -- 172.21.15.83:0/1535900093 <== mon.2 v2:172.21.15.117:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fd2380bf070 con 0x55ec4e956c90 2024-08-19T01:35:37.430 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:37.426+0000 7fd250e78740 10 client.25295 fetch_fsmap finished waiting for FSMap version 25 2024-08-19T01:35:37.430 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:37.426+0000 7fd250e78740 10 client.25295 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-19T01:35:37.430 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:37.426+0000 7fd250e78740 20 client.25295 populate_metadata read hostname 'smithi083' 2024-08-19T01:35:37.430 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:37.426+0000 7fd250e78740 1 --2- 172.21.15.83:0/1535900093 >> [v2:172.21.15.117:6838/1967595704,v1:172.21.15.117:6839/1967595704] conn(0x55ec4e971730 0x55ec4e973b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:35:37.430 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:37.426+0000 7fd250e78740 4 client.25295 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-19T01:35:37.430 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:37.426+0000 7fd250e78740 1 -- 172.21.15.83:0/1535900093 --> [v2:172.21.15.117:6838/1967595704,v1:172.21.15.117:6839/1967595704] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ec4e8c2030 con 0x55ec4e971730 2024-08-19T01:35:37.430 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:37.427+0000 7fd246a00640 1 --2- 172.21.15.83:0/1535900093 >> [v2:172.21.15.117:6838/1967595704,v1:172.21.15.117:6839/1967595704] conn(0x55ec4e971730 0x55ec4e973b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:35:37.430 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:37.428+0000 7fd230c00640 10 client.25295 ms_handle_connect on v2:172.21.15.117:6838/1967595704 2024-08-19T01:35:37.430 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:37.428+0000 7fd246a00640 1 --2- 172.21.15.83:0/1535900093 >> [v2:172.21.15.117:6838/1967595704,v1:172.21.15.117:6839/1967595704] conn(0x55ec4e971730 0x55ec4e973b20 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=31f73c63f55f5775 server_cookie=2f0bb3af16553124 in_seq=0 out_seq=0 2024-08-19T01:35:37.431 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:37.428+0000 7fd230c00640 1 -- 172.21.15.83:0/1535900093 <== mds.0 v2:172.21.15.117:6838/1967595704 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55ec4e8c2030 con 0x55ec4e971730 2024-08-19T01:35:37.431 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:37.428+0000 7fd230c00640 10 client.25295 handle_command_reply: tid=0 2024-08-19T01:35:37.431 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:37.429+0000 7fd250e78740 1 client.25295 shutdown 2024-08-19T01:35:37.432 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:37.429+0000 7fd250e78740 1 -- 172.21.15.83:0/1535900093 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ec4e956c90 msgr2=0x55ec4e95eeb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:35:37.432 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:37.429+0000 7fd250e78740 1 --2- 172.21.15.83:0/1535900093 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ec4e956c90 0x55ec4e95eeb0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fd2380bbc60 tx=0x7fd2380ed690 comp rx=0 tx=0).stop 2024-08-19T01:35:37.432 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:37.429+0000 7fd250e78740 1 -- 172.21.15.83:0/1535900093 shutdown_connections 2024-08-19T01:35:37.432 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:37.429+0000 7fd250e78740 1 --2- 172.21.15.83:0/1535900093 >> [v2:172.21.15.117:6838/1967595704,v1:172.21.15.117:6839/1967595704] conn(0x55ec4e971730 0x55ec4e973b20 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:35:37.432 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:37.429+0000 7fd250e78740 1 --2- 172.21.15.83:0/1535900093 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ec4e961e70 0x55ec4e95f930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:35:37.432 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:37.429+0000 7fd250e78740 1 --2- 172.21.15.83:0/1535900093 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ec4e9575b0 0x55ec4e95f3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:35:37.432 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:37.429+0000 7fd250e78740 1 --2- 172.21.15.83:0/1535900093 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ec4e956c90 0x55ec4e95eeb0 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:35:37.432 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:37.429+0000 7fd250e78740 1 -- 172.21.15.83:0/1535900093 >> 172.21.15.83:0/1535900093 conn(0x55ec4e953220 msgr2=0x55ec4e949b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:35:37.432 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:37.429+0000 7fd230c00640 0 client.25295 ms_handle_reset on v2:172.21.15.117:6838/1967595704 2024-08-19T01:35:37.432 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:37.429+0000 7fd250e78740 1 -- 172.21.15.83:0/1535900093 shutdown_connections 2024-08-19T01:35:37.432 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:37.429+0000 7fd250e78740 1 -- 172.21.15.83:0/1535900093 wait complete. 2024-08-19T01:35:37.432 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:37.429+0000 7fd250e78740 20 client.25295 trim_cache size 0 max 16384 2024-08-19T01:35:37.466 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:37.465+0000 7fd250e78740 1 Processor -- start 2024-08-19T01:35:37.467 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:37.465+0000 7fd250e78740 1 -- start start 2024-08-19T01:35:37.467 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:37.465+0000 7fd250e78740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ec4e956c90 0x55ec4e95f780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:35:37.467 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:37.465+0000 7fd250e78740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ec4e9575b0 0x55ec4e9792b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:35:37.467 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:37.465+0000 7fd250e78740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ec4e961e70 0x55ec4e9797f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:35:37.467 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:37.465+0000 7fd250e78740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fd2481adb10 con 0x55ec4e9575b0 2024-08-19T01:35:37.468 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:37.465+0000 7fd250e78740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fd2481ab3e0 con 0x55ec4e956c90 2024-08-19T01:35:37.468 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:37.465+0000 7fd250e78740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fd2481a9230 con 0x55ec4e961e70 2024-08-19T01:35:37.469 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:37.466+0000 7fd246a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ec4e9575b0 0x55ec4e9792b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:35:37.469 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:37.466+0000 7fd247400640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ec4e956c90 0x55ec4e95f780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:35:37.469 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:37.466+0000 7fd247400640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ec4e956c90 0x55ec4e95f780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.83:54242/0 (socket says 172.21.15.83:54242) 2024-08-19T01:35:37.469 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:37.466+0000 7fd247400640 1 -- 172.21.15.83:0/160187845 learned_addr learned my addr 172.21.15.83:0/160187845 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-08-19T01:35:37.469 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:37.466+0000 7fd247e00640 1 --2- 172.21.15.83:0/160187845 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ec4e961e70 0x55ec4e9797f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:35:37.469 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:37.466+0000 7fd247400640 1 -- 172.21.15.83:0/160187845 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ec4e961e70 msgr2=0x55ec4e9797f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:35:37.469 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:37.466+0000 7fd247400640 1 --2- 172.21.15.83:0/160187845 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ec4e961e70 0x55ec4e9797f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:35:37.469 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:37.466+0000 7fd247400640 1 -- 172.21.15.83:0/160187845 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ec4e9575b0 msgr2=0x55ec4e9792b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:35:37.469 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:37.467+0000 7fd247400640 1 --2- 172.21.15.83:0/160187845 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ec4e9575b0 0x55ec4e9792b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:35:37.469 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:37.467+0000 7fd247400640 1 -- 172.21.15.83:0/160187845 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ec4e953eb0 con 0x55ec4e956c90 2024-08-19T01:35:37.469 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:37.467+0000 7fd246a00640 1 --2- 172.21.15.83:0/160187845 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ec4e9575b0 0x55ec4e9792b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-19T01:35:37.470 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:37.467+0000 7fd247400640 1 --2- 172.21.15.83:0/160187845 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ec4e956c90 0x55ec4e95f780 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fd2380dc010 tx=0x7fd23803b790 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2ce0a0bbeb30e5a server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:35:37.470 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:37.467+0000 7fd233400640 1 -- 172.21.15.83:0/160187845 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2380c4bd0 con 0x55ec4e956c90 2024-08-19T01:35:37.470 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:37.467+0000 7fd233400640 1 -- 172.21.15.83:0/160187845 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd2380c0310 con 0x55ec4e956c90 2024-08-19T01:35:37.470 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:37.467+0000 7fd233400640 1 -- 172.21.15.83:0/160187845 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2380b7e50 con 0x55ec4e956c90 2024-08-19T01:35:37.471 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:37.468+0000 7fd250e78740 1 -- 172.21.15.83:0/160187845 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ec4e956c90 msgr2=0x55ec4e95f780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:35:37.471 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:37.468+0000 7fd250e78740 1 --2- 172.21.15.83:0/160187845 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ec4e956c90 0x55ec4e95f780 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fd2380dc010 tx=0x7fd23803b790 comp rx=0 tx=0).stop 2024-08-19T01:35:37.471 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:37.468+0000 7fd250e78740 1 -- 172.21.15.83:0/160187845 shutdown_connections 2024-08-19T01:35:37.471 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:37.468+0000 7fd250e78740 1 --2- 172.21.15.83:0/160187845 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ec4e961e70 0x55ec4e9797f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:35:37.471 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:37.468+0000 7fd250e78740 1 --2- 172.21.15.83:0/160187845 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ec4e9575b0 0x55ec4e9792b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:35:37.471 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:37.468+0000 7fd250e78740 1 --2- 172.21.15.83:0/160187845 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ec4e956c90 0x55ec4e95f780 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:35:37.471 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:37.468+0000 7fd250e78740 1 -- 172.21.15.83:0/160187845 >> 172.21.15.83:0/160187845 conn(0x55ec4e953220 msgr2=0x55ec4e960f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:35:37.471 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:37.469+0000 7fd250e78740 1 -- 172.21.15.83:0/160187845 shutdown_connections 2024-08-19T01:35:37.471 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:37.469+0000 7fd250e78740 1 -- 172.21.15.83:0/160187845 wait complete. 2024-08-19T01:35:37.472 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:37.469+0000 7fd250e78740 1 -- start start 2024-08-19T01:35:37.472 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:37.470+0000 7fd250e78740 1 Processor -- start 2024-08-19T01:35:37.472 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:37.470+0000 7fd250e78740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ec4e956c90 0x55ec4e96a310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:35:37.472 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:37.470+0000 7fd250e78740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ec4e9575b0 0x55ec4e95bd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:35:37.472 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:37.470+0000 7fd250e78740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ec4e961e70 0x55ec4e95c450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:35:37.472 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:37.470+0000 7fd250e78740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fd2481a98a0 con 0x55ec4e961e70 2024-08-19T01:35:37.472 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:37.470+0000 7fd250e78740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fd2481aad10 con 0x55ec4e9575b0 2024-08-19T01:35:37.473 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:37.470+0000 7fd250e78740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fd2481a9f70 con 0x55ec4e956c90 2024-08-19T01:35:37.473 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:37.471+0000 7fd250e78740 10 client.0 _ll_register_callbacks cb 0x7fd2380d3070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-19T01:35:37.473 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:37.471+0000 7fd247e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ec4e961e70 0x55ec4e95c450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:35:37.474 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:37.471+0000 7fd246a00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ec4e9575b0 0x55ec4e95bd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:35:37.474 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:37.471+0000 7fd247400640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ec4e956c90 0x55ec4e96a310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:35:37.474 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:37.471+0000 7fd247e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ec4e961e70 0x55ec4e95c450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:58588/0 (socket says 172.21.15.83:58588) 2024-08-19T01:35:37.474 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:37.471+0000 7fd247e00640 1 -- 172.21.15.83:0/4260226558 learned_addr learned my addr 172.21.15.83:0/4260226558 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-08-19T01:35:37.474 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:37.471+0000 7fd246a00640 1 --2- 172.21.15.83:0/4260226558 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ec4e9575b0 0x55ec4e95bd10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.83:54256/0 (socket says 172.21.15.83:54256) 2024-08-19T01:35:37.474 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:37.471+0000 7fd247400640 1 -- 172.21.15.83:0/4260226558 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ec4e9575b0 msgr2=0x55ec4e95bd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:35:37.474 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:37.471+0000 7fd247400640 1 --2- 172.21.15.83:0/4260226558 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ec4e9575b0 0x55ec4e95bd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:35:37.474 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:37.471+0000 7fd247400640 1 -- 172.21.15.83:0/4260226558 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ec4e961e70 msgr2=0x55ec4e95c450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:35:37.474 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:37.472+0000 7fd247400640 1 --2- 172.21.15.83:0/4260226558 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ec4e961e70 0x55ec4e95c450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:35:37.474 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:37.472+0000 7fd247400640 1 -- 172.21.15.83:0/4260226558 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ec4e9db280 con 0x55ec4e956c90 2024-08-19T01:35:37.474 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:37.472+0000 7fd247e00640 1 --2- 172.21.15.83:0/4260226558 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ec4e961e70 0x55ec4e95c450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-19T01:35:37.475 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:37.472+0000 7fd247400640 1 --2- 172.21.15.83:0/4260226558 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ec4e956c90 0x55ec4e96a310 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fd238006010 tx=0x7fd23803b6f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8d6bd7799071d23a server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:35:37.476 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:37.472+0000 7fd230c00640 10 client.0 ms_handle_connect on v2:172.21.15.167:3300/0 2024-08-19T01:35:37.476 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:37.472+0000 7fd230c00640 1 -- 172.21.15.83:0/4260226558 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2380c4bd0 con 0x55ec4e956c90 2024-08-19T01:35:37.476 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:37.473+0000 7fd230c00640 1 -- 172.21.15.83:0/4260226558 <== mon.1 v2:172.21.15.167:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8269+0+0 (secure 0 0 0) 0x7fd2380dadb0 con 0x55ec4e956c90 2024-08-19T01:35:37.476 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:37.473+0000 7fd250e78740 1 -- 172.21.15.83:0/4260226558 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd2380c0310 con 0x55ec4e956c90 2024-08-19T01:35:37.476 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:37.473+0000 7fd230c00640 1 -- 172.21.15.83:0/4260226558 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fd2380317b0 con 0x55ec4e956c90 2024-08-19T01:35:37.476 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:37.473+0000 7fd250e78740 10 client.25279 fetch_fsmap learned FSMap version 25 2024-08-19T01:35:37.476 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:37.473+0000 7fd250e78740 1 -- 172.21.15.83:0/4260226558 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55ec4e9db280 con 0x55ec4e956c90 2024-08-19T01:35:37.476 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:37.473+0000 7fd230c00640 1 -- 172.21.15.83:0/4260226558 <== mon.1 v2:172.21.15.167:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd2380c0310 con 0x55ec4e956c90 2024-08-19T01:35:37.476 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:37.474+0000 7fd230c00640 1 -- 172.21.15.83:0/4260226558 <== mon.1 v2:172.21.15.167:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fd2380a4030 con 0x55ec4e956c90 2024-08-19T01:35:37.477 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:37.474+0000 7fd250e78740 10 client.25279 fetch_fsmap finished waiting for FSMap version 25 2024-08-19T01:35:37.477 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:37.474+0000 7fd250e78740 10 client.25279 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-19T01:35:37.477 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:37.474+0000 7fd250e78740 20 client.25279 populate_metadata read hostname 'smithi083' 2024-08-19T01:35:37.477 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:37.474+0000 7fd250e78740 1 --2- 172.21.15.83:0/4260226558 >> [v2:172.21.15.117:6838/1967595704,v1:172.21.15.117:6839/1967595704] conn(0x55ec4e9dc700 0x55ec4e9deaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:35:37.477 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:37.474+0000 7fd250e78740 4 client.25279 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-08-19T01:35:37.477 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:37.474+0000 7fd250e78740 1 -- 172.21.15.83:0/4260226558 --> [v2:172.21.15.117:6838/1967595704,v1:172.21.15.117:6839/1967595704] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55ec4e8bc540 con 0x55ec4e9dc700 2024-08-19T01:35:37.477 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:37.475+0000 7fd246a00640 1 --2- 172.21.15.83:0/4260226558 >> [v2:172.21.15.117:6838/1967595704,v1:172.21.15.117:6839/1967595704] conn(0x55ec4e9dc700 0x55ec4e9deaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:35:37.477 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:37.476+0000 7fd246a00640 1 --2- 172.21.15.83:0/4260226558 >> [v2:172.21.15.117:6838/1967595704,v1:172.21.15.117:6839/1967595704] conn(0x55ec4e9dc700 0x55ec4e9deaf0 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9b9b63cdfeffaeb4 server_cookie=d401d92fbaafc1d5 in_seq=0 out_seq=0 2024-08-19T01:35:37.478 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:37.476+0000 7fd230c00640 10 client.25279 ms_handle_connect on v2:172.21.15.117:6838/1967595704 2024-08-19T01:35:37.478 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:37.477+0000 7fd230c00640 1 -- 172.21.15.83:0/4260226558 <== mds.0 v2:172.21.15.117:6838/1967595704 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55ec4e8bc540 con 0x55ec4e9dc700 2024-08-19T01:35:37.479 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:37.477+0000 7fd230c00640 10 client.25279 handle_command_reply: tid=0 2024-08-19T01:35:37.479 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:37.477+0000 7fd250e78740 1 client.25279 shutdown 2024-08-19T01:35:37.479 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:37.478+0000 7fd250e78740 1 -- 172.21.15.83:0/4260226558 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ec4e956c90 msgr2=0x55ec4e96a310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:35:37.480 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:37.478+0000 7fd250e78740 1 --2- 172.21.15.83:0/4260226558 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ec4e956c90 0x55ec4e96a310 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fd238006010 tx=0x7fd23803b6f0 comp rx=0 tx=0).stop 2024-08-19T01:35:37.480 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:37.478+0000 7fd250e78740 1 -- 172.21.15.83:0/4260226558 shutdown_connections 2024-08-19T01:35:37.480 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:37.478+0000 7fd250e78740 1 --2- 172.21.15.83:0/4260226558 >> [v2:172.21.15.117:6838/1967595704,v1:172.21.15.117:6839/1967595704] conn(0x55ec4e9dc700 0x55ec4e9deaf0 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:35:37.480 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:37.478+0000 7fd250e78740 1 --2- 172.21.15.83:0/4260226558 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ec4e961e70 0x55ec4e95c450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:35:37.480 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:37.478+0000 7fd250e78740 1 --2- 172.21.15.83:0/4260226558 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ec4e9575b0 0x55ec4e95bd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:35:37.480 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:37.478+0000 7fd250e78740 1 --2- 172.21.15.83:0/4260226558 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ec4e956c90 0x55ec4e96a310 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:35:37.480 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:37.478+0000 7fd250e78740 1 -- 172.21.15.83:0/4260226558 >> 172.21.15.83:0/4260226558 conn(0x55ec4e953220 msgr2=0x55ec4e960710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:35:37.480 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:37.478+0000 7fd230c00640 0 client.25279 ms_handle_reset on v2:172.21.15.117:6838/1967595704 2024-08-19T01:35:37.481 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:37.478+0000 7fd250e78740 1 -- 172.21.15.83:0/4260226558 shutdown_connections 2024-08-19T01:35:37.481 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:37.479+0000 7fd250e78740 1 -- 172.21.15.83:0/4260226558 wait complete. 2024-08-19T01:35:37.481 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:37.479+0000 7fd250e78740 20 client.25279 trim_cache size 0 max 16384 2024-08-19T01:35:37.481 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-08-19T01:35:37.481 INFO:teuthology.orchestra.run.smithi083.stdout: "return_code": 0, 2024-08-19T01:35:37.481 INFO:teuthology.orchestra.run.smithi083.stdout: "scrub_tag": "70de75de-6b90-41fb-a490-767dc312a1dc", 2024-08-19T01:35:37.482 INFO:teuthology.orchestra.run.smithi083.stdout: "mode": "asynchronous" 2024-08-19T01:35:37.482 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-08-19T01:35:37.483 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:37.480+0000 7fd24e200640 1 -- 172.21.15.83:0/1685393616 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7fd22c0a2c10 msgr2=0x7fd22c0a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:35:37.483 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:37.480+0000 7fd24e200640 1 --2- 172.21.15.83:0/1685393616 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7fd22c0a2c10 0x7fd22c0a50d0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fd240005fd0 tx=0x7fd24000f730 comp rx=0 tx=0).stop 2024-08-19T01:35:37.483 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:37.480+0000 7fd24e200640 1 -- 172.21.15.83:0/1685393616 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd248103ac0 msgr2=0x7fd24819a3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:35:37.483 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:37.480+0000 7fd24e200640 1 --2- 172.21.15.83:0/1685393616 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd248103ac0 0x7fd24819a3b0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fd23802a940 tx=0x7fd238003b20 comp rx=0 tx=0).stop 2024-08-19T01:35:37.483 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:37.480+0000 7fd24e200640 1 -- 172.21.15.83:0/1685393616 shutdown_connections 2024-08-19T01:35:37.483 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:37.480+0000 7fd24e200640 1 --2- 172.21.15.83:0/1685393616 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7fd22c0a2c10 0x7fd22c0a50d0 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:35:37.483 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:37.480+0000 7fd24e200640 1 --2- 172.21.15.83:0/1685393616 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd248109ac0 0x7fd24819ec80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:35:37.483 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:37.480+0000 7fd24e200640 1 --2- 172.21.15.83:0/1685393616 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd248104460 0x7fd24819a8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:35:37.483 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:37.480+0000 7fd24e200640 1 --2- 172.21.15.83:0/1685393616 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd248103ac0 0x7fd24819a3b0 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:35:37.483 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:37.480+0000 7fd24e200640 1 -- 172.21.15.83:0/1685393616 >> 172.21.15.83:0/1685393616 conn(0x7fd2480ff7c0 msgr2=0x7fd2480fffe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:35:37.483 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:37.481+0000 7fd24e200640 1 -- 172.21.15.83:0/1685393616 shutdown_connections 2024-08-19T01:35:37.483 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:37.481+0000 7fd24e200640 1 -- 172.21.15.83:0/1685393616 wait complete. 2024-08-19T01:35:37.912 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:35:37 smithi083 ceph-mon[17120]: pgmap v362: 97 pgs: 97 active+clean; 668 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 1.7 KiB/s wr, 8 op/s 2024-08-19T01:35:37.952 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:35:37 smithi117 ceph-mon[22016]: pgmap v362: 97 pgs: 97 active+clean; 668 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 1.7 KiB/s wr, 8 op/s 2024-08-19T01:35:37.966 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:35:37 smithi167 ceph-mon[21875]: pgmap v362: 97 pgs: 97 active+clean; 668 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 1.7 KiB/s wr, 8 op/s 2024-08-19T01:35:38.416 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833 shell --fsid bad8780e-5dc9-11ef-bccf-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-08-19T01:35:38.655 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:35:38 smithi083 ceph-mon[17120]: scrub queued for path: / 2024-08-19T01:35:38.655 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:35:38 smithi083 ceph-mon[17120]: scrub summary: idle+waiting paths [/] 2024-08-19T01:35:38.655 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:35:38 smithi083 ceph-mon[17120]: scrub summary: active paths [/] 2024-08-19T01:35:38.656 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:35:38 smithi083 ceph-mon[17120]: scrub summary: idle+waiting paths [/] 2024-08-19T01:35:38.924 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/bad8780e-5dc9-11ef-bccf-c7b262605968/mon.a/config 2024-08-19T01:35:38.952 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:35:38 smithi117 ceph-mon[22016]: scrub queued for path: / 2024-08-19T01:35:38.952 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:35:38 smithi117 ceph-mon[22016]: scrub summary: idle+waiting paths [/] 2024-08-19T01:35:38.952 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:35:38 smithi117 ceph-mon[22016]: scrub summary: active paths [/] 2024-08-19T01:35:38.952 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:35:38 smithi117 ceph-mon[22016]: scrub summary: idle+waiting paths [/] 2024-08-19T01:35:38.966 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:35:38 smithi167 ceph-mon[21875]: scrub queued for path: / 2024-08-19T01:35:38.966 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:35:38 smithi167 ceph-mon[21875]: scrub summary: idle+waiting paths [/] 2024-08-19T01:35:38.966 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:35:38 smithi167 ceph-mon[21875]: scrub summary: active paths [/] 2024-08-19T01:35:38.966 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:35:38 smithi167 ceph-mon[21875]: scrub summary: idle+waiting paths [/] 2024-08-19T01:35:39.578 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:39.555+0000 7f382d600640 1 -- 172.21.15.83:0/3098121315 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3828103ac0 msgr2=0x7f3828103f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:35:39.578 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:39.555+0000 7f382d600640 1 --2- 172.21.15.83:0/3098121315 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3828103ac0 0x7f3828103f20 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f3814009930 tx=0x7f381402a420 comp rx=0 tx=0).stop 2024-08-19T01:35:39.578 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:39.555+0000 7f382d600640 1 -- 172.21.15.83:0/3098121315 shutdown_connections 2024-08-19T01:35:39.578 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:39.555+0000 7f382d600640 1 --2- 172.21.15.83:0/3098121315 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3828104460 0x7f3828110990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:35:39.579 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:39.555+0000 7f382d600640 1 --2- 172.21.15.83:0/3098121315 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3828103ac0 0x7f3828103f20 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:35:39.579 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:39.555+0000 7f382d600640 1 --2- 172.21.15.83:0/3098121315 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3828109ac0 0x7f3828109ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:35:39.579 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:39.555+0000 7f382d600640 1 -- 172.21.15.83:0/3098121315 >> 172.21.15.83:0/3098121315 conn(0x7f38280ff7c0 msgr2=0x7f3828101be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:35:39.579 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:39.556+0000 7f382d600640 1 -- 172.21.15.83:0/3098121315 shutdown_connections 2024-08-19T01:35:39.579 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:39.556+0000 7f382d600640 1 -- 172.21.15.83:0/3098121315 wait complete. 2024-08-19T01:35:39.579 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:39.557+0000 7f382d600640 1 Processor -- start 2024-08-19T01:35:39.579 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:39.557+0000 7f382d600640 1 -- start start 2024-08-19T01:35:39.579 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:39.557+0000 7f382d600640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3828103ac0 0x7f382819a390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:35:39.579 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:39.558+0000 7f382d600640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3828104460 0x7f382819a8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:35:39.579 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:39.558+0000 7f382d600640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3828109ac0 0x7f382819ec60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:35:39.579 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:39.558+0000 7f382d600640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f3828113f10 con 0x7f3828109ac0 2024-08-19T01:35:39.579 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:39.558+0000 7f382d600640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f3828113d90 con 0x7f3828103ac0 2024-08-19T01:35:39.579 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:39.558+0000 7f382d600640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f3828114090 con 0x7f3828104460 2024-08-19T01:35:39.580 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:39.558+0000 7f3826a00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3828103ac0 0x7f382819a390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:35:39.580 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:39.558+0000 7f3826a00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3828103ac0 0x7f382819a390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.83:54284/0 (socket says 172.21.15.83:54284) 2024-08-19T01:35:39.580 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:39.558+0000 7f3826a00640 1 -- 172.21.15.83:0/3072912181 learned_addr learned my addr 172.21.15.83:0/3072912181 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-08-19T01:35:39.580 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:39.558+0000 7f3827400640 1 --2- 172.21.15.83:0/3072912181 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3828109ac0 0x7f382819ec60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:35:39.580 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:39.558+0000 7f3826000640 1 --2- 172.21.15.83:0/3072912181 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3828104460 0x7f382819a8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:35:39.580 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:39.559+0000 7f3827400640 1 -- 172.21.15.83:0/3072912181 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3828104460 msgr2=0x7f382819a8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:35:39.580 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:39.559+0000 7f3827400640 1 --2- 172.21.15.83:0/3072912181 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3828104460 0x7f382819a8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:35:39.580 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:39.559+0000 7f3827400640 1 -- 172.21.15.83:0/3072912181 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3828103ac0 msgr2=0x7f382819a390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:35:39.580 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:39.559+0000 7f3827400640 1 --2- 172.21.15.83:0/3072912181 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3828103ac0 0x7f382819a390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:35:39.580 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:39.559+0000 7f3827400640 1 -- 172.21.15.83:0/3072912181 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f382819f340 con 0x7f3828109ac0 2024-08-19T01:35:39.580 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:39.559+0000 7f3826a00640 1 --2- 172.21.15.83:0/3072912181 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3828103ac0 0x7f382819a390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-19T01:35:39.580 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:39.559+0000 7f3826000640 1 --2- 172.21.15.83:0/3072912181 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3828104460 0x7f382819a8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-19T01:35:39.581 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:39.559+0000 7f3827400640 1 --2- 172.21.15.83:0/3072912181 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3828109ac0 0x7f382819ec60 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f3820009fd0 tx=0x7f382000ef10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:35:39.581 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:39.559+0000 7f381f400640 1 -- 172.21.15.83:0/3072912181 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f382000ce20 con 0x7f3828109ac0 2024-08-19T01:35:39.581 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:39.560+0000 7f381f400640 1 -- 172.21.15.83:0/3072912181 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3820005020 con 0x7f3828109ac0 2024-08-19T01:35:39.581 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:39.560+0000 7f382d600640 1 -- 172.21.15.83:0/3072912181 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f382819f630 con 0x7f3828109ac0 2024-08-19T01:35:39.581 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:39.560+0000 7f382d600640 1 -- 172.21.15.83:0/3072912181 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f38281a6f10 con 0x7f3828109ac0 2024-08-19T01:35:39.581 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:39.560+0000 7f381f400640 1 -- 172.21.15.83:0/3072912181 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3820010720 con 0x7f3828109ac0 2024-08-19T01:35:39.581 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:39.564+0000 7f382ffda740 1 Processor -- start 2024-08-19T01:35:39.581 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:39.564+0000 7f381f400640 1 -- 172.21.15.83:0/3072912181 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 19) ==== 137884+0+0 (secure 0 0 0) 0x7f3820004a40 con 0x7f3828109ac0 2024-08-19T01:35:39.581 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:39.564+0000 7f382ffda740 1 -- start start 2024-08-19T01:35:39.581 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:39.565+0000 7f382ffda740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563e0c298b90 0x563e0c298f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:35:39.581 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:39.565+0000 7f382ffda740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563e0c2994b0 0x563e0c2a3830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:35:39.581 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:39.565+0000 7f382ffda740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563e0c2a3d70 0x563e0c2a6160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:35:39.581 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:39.565+0000 7f382ffda740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x563e0c1febf0 con 0x563e0c298b90 2024-08-19T01:35:39.582 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:39.565+0000 7f382ffda740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x563e0c196bd0 con 0x563e0c2994b0 2024-08-19T01:35:39.582 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:39.565+0000 7f382ffda740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x563e0c1fabb0 con 0x563e0c2a3d70 2024-08-19T01:35:39.582 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:39.565+0000 7f3826a00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563e0c2994b0 0x563e0c2a3830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:35:39.582 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:39.565+0000 7f3826a00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563e0c2994b0 0x563e0c2a3830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.83:54296/0 (socket says 172.21.15.83:54296) 2024-08-19T01:35:39.582 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:39.565+0000 7f3826a00640 1 -- 172.21.15.83:0/1121055043 learned_addr learned my addr 172.21.15.83:0/1121055043 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-08-19T01:35:39.582 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:39.566+0000 7f3826000640 1 --2- 172.21.15.83:0/1121055043 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563e0c2a3d70 0x563e0c2a6160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:35:39.582 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:39.566+0000 7f3826000640 1 --2- 172.21.15.83:0/1121055043 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563e0c298b90 0x563e0c298f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:35:39.582 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:39.566+0000 7f3826a00640 1 -- 172.21.15.83:0/1121055043 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563e0c2a3d70 msgr2=0x563e0c2a6160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:35:39.582 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:39.566+0000 7f3826a00640 1 --2- 172.21.15.83:0/1121055043 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563e0c2a3d70 0x563e0c2a6160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:35:39.582 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:39.566+0000 7f3826a00640 1 -- 172.21.15.83:0/1121055043 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563e0c298b90 msgr2=0x563e0c298f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:35:39.582 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:39.566+0000 7f3826a00640 1 --2- 172.21.15.83:0/1121055043 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563e0c298b90 0x563e0c298f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:35:39.582 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:39.566+0000 7f3826a00640 1 -- 172.21.15.83:0/1121055043 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563e0c19d8a0 con 0x563e0c2994b0 2024-08-19T01:35:39.583 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:39.566+0000 7f3826000640 1 --2- 172.21.15.83:0/1121055043 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563e0c298b90 0x563e0c298f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-19T01:35:39.583 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:39.566+0000 7f381f400640 1 --2- 172.21.15.83:0/3072912181 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f380c0a2c90 0x7f380c0a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:35:39.583 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:39.566+0000 7f381f400640 1 -- 172.21.15.83:0/3072912181 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8269+0+0 (secure 0 0 0) 0x7f38200cf770 con 0x7f3828109ac0 2024-08-19T01:35:39.583 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:39.567+0000 7f3826000640 1 --2- 172.21.15.83:0/3072912181 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f380c0a2c90 0x7f380c0a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:35:39.583 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:39.567+0000 7f3826a00640 1 --2- 172.21.15.83:0/1121055043 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563e0c2994b0 0x563e0c2a3830 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f38180115c0 tx=0x7f3818011a80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=aca0b45b22c17191 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:35:39.583 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:39.567+0000 7f3813400640 1 -- 172.21.15.83:0/1121055043 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f381800f660 con 0x563e0c2994b0 2024-08-19T01:35:39.583 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:39.567+0000 7f3813400640 1 -- 172.21.15.83:0/1121055043 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f381800fce0 con 0x563e0c2994b0 2024-08-19T01:35:39.583 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:39.567+0000 7f3813400640 1 -- 172.21.15.83:0/1121055043 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38180183b0 con 0x563e0c2994b0 2024-08-19T01:35:39.583 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:39.568+0000 7f382ffda740 1 -- 172.21.15.83:0/1121055043 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563e0c2994b0 msgr2=0x563e0c2a3830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:35:39.583 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:39.568+0000 7f382ffda740 1 --2- 172.21.15.83:0/1121055043 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563e0c2994b0 0x563e0c2a3830 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f38180115c0 tx=0x7f3818011a80 comp rx=0 tx=0).stop 2024-08-19T01:35:39.583 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:39.568+0000 7f3826000640 1 --2- 172.21.15.83:0/3072912181 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f380c0a2c90 0x7f380c0a5150 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f3814006010 tx=0x7f3814031510 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:35:39.583 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:39.568+0000 7f382ffda740 1 -- 172.21.15.83:0/1121055043 shutdown_connections 2024-08-19T01:35:39.583 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:39.568+0000 7f382ffda740 1 --2- 172.21.15.83:0/1121055043 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563e0c2a3d70 0x563e0c2a6160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:35:39.584 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:39.568+0000 7f382ffda740 1 --2- 172.21.15.83:0/1121055043 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563e0c2994b0 0x563e0c2a3830 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:35:39.584 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:39.568+0000 7f382ffda740 1 --2- 172.21.15.83:0/1121055043 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563e0c298b90 0x563e0c298f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:35:39.584 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:39.568+0000 7f382ffda740 1 -- 172.21.15.83:0/1121055043 >> 172.21.15.83:0/1121055043 conn(0x563e0c295120 msgr2=0x563e0c295500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:35:39.586 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:39.569+0000 7f382ffda740 1 -- 172.21.15.83:0/1121055043 shutdown_connections 2024-08-19T01:35:39.586 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:39.569+0000 7f382ffda740 1 -- 172.21.15.83:0/1121055043 wait complete. 2024-08-19T01:35:39.586 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:39.569+0000 7f382ffda740 1 -- start start 2024-08-19T01:35:39.586 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:39.569+0000 7f382ffda740 1 Processor -- start 2024-08-19T01:35:39.586 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:39.570+0000 7f382ffda740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563e0c298b90 0x563e0c2a0c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:35:39.586 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:39.570+0000 7f382ffda740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563e0c2994b0 0x563e0c2a1150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:35:39.586 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:39.570+0000 7f382ffda740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563e0c2a3d70 0x563e0c2a1690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:35:39.586 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:39.570+0000 7f382ffda740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x563e0c2a6940 con 0x563e0c2a3d70 2024-08-19T01:35:39.586 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:39.570+0000 7f382ffda740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x563e0c1b9d40 con 0x563e0c298b90 2024-08-19T01:35:39.586 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:39.570+0000 7f382ffda740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x563e0c2a6ac0 con 0x563e0c2994b0 2024-08-19T01:35:39.587 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:39.570+0000 7f382ffda740 10 client.0 _ll_register_callbacks cb 0x563e0c182b10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-19T01:35:39.587 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:39.570+0000 7f3826a00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563e0c298b90 0x563e0c2a0c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:35:39.587 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:39.570+0000 7f3826a00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563e0c298b90 0x563e0c2a0c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.83:54308/0 (socket says 172.21.15.83:54308) 2024-08-19T01:35:39.587 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:39.570+0000 7f3826a00640 1 -- 172.21.15.83:0/1449861937 learned_addr learned my addr 172.21.15.83:0/1449861937 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-08-19T01:35:39.587 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:39.571+0000 7f3826000640 1 --2- 172.21.15.83:0/1449861937 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563e0c2994b0 0x563e0c2a1150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:35:39.587 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:39.571+0000 7f3827400640 1 --2- 172.21.15.83:0/1449861937 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563e0c2a3d70 0x563e0c2a1690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:35:39.587 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:39.571+0000 7f3826a00640 1 -- 172.21.15.83:0/1449861937 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563e0c2994b0 msgr2=0x563e0c2a1150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:35:39.587 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:39.571+0000 7f3826a00640 1 --2- 172.21.15.83:0/1449861937 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563e0c2994b0 0x563e0c2a1150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:35:39.587 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:39.571+0000 7f3826a00640 1 -- 172.21.15.83:0/1449861937 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563e0c2a3d70 msgr2=0x563e0c2a1690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:35:39.587 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:39.571+0000 7f3826a00640 1 --2- 172.21.15.83:0/1449861937 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563e0c2a3d70 0x563e0c2a1690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:35:39.587 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:39.571+0000 7f3826a00640 1 -- 172.21.15.83:0/1449861937 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563e0bf703c0 con 0x563e0c298b90 2024-08-19T01:35:39.587 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:39.571+0000 7f3827400640 1 --2- 172.21.15.83:0/1449861937 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563e0c2a3d70 0x563e0c2a1690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-19T01:35:39.588 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:39.571+0000 7f3826a00640 1 --2- 172.21.15.83:0/1449861937 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563e0c298b90 0x563e0c2a0c10 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f38180199f0 tx=0x7f3818019640 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b1509193fbcb93da server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:35:39.588 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:39.571+0000 7f3807e00640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-08-19T01:35:39.588 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:39.572+0000 7f3807e00640 1 -- 172.21.15.83:0/1449861937 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3818027060 con 0x563e0c298b90 2024-08-19T01:35:39.588 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:39.572+0000 7f3807e00640 1 -- 172.21.15.83:0/1449861937 <== mon.2 v2:172.21.15.117:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8269+0+0 (secure 0 0 0) 0x7f381800f870 con 0x563e0c298b90 2024-08-19T01:35:39.588 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:39.572+0000 7f382ffda740 1 -- 172.21.15.83:0/1449861937 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f381800fce0 con 0x563e0c298b90 2024-08-19T01:35:39.588 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:39.572+0000 7f3807e00640 1 -- 172.21.15.83:0/1449861937 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f381800fe90 con 0x563e0c298b90 2024-08-19T01:35:39.588 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:39.572+0000 7f382ffda740 10 client.25313 fetch_fsmap learned FSMap version 25 2024-08-19T01:35:39.588 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:39.572+0000 7f382ffda740 1 -- 172.21.15.83:0/1449861937 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x563e0c2b2570 con 0x563e0c298b90 2024-08-19T01:35:39.588 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:39.573+0000 7f3807e00640 1 -- 172.21.15.83:0/1449861937 <== mon.2 v2:172.21.15.117:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f381800fce0 con 0x563e0c298b90 2024-08-19T01:35:39.588 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:39.573+0000 7f3807e00640 1 -- 172.21.15.83:0/1449861937 <== mon.2 v2:172.21.15.117:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f38180049c0 con 0x563e0c298b90 2024-08-19T01:35:39.588 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:39.573+0000 7f382ffda740 10 client.25313 fetch_fsmap finished waiting for FSMap version 25 2024-08-19T01:35:39.588 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:39.573+0000 7f382ffda740 10 client.25313 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-19T01:35:39.588 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:39.573+0000 7f382ffda740 20 client.25313 populate_metadata read hostname 'smithi083' 2024-08-19T01:35:39.589 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:39.573+0000 7f382ffda740 1 --2- 172.21.15.83:0/1449861937 >> [v2:172.21.15.117:6838/1967595704,v1:172.21.15.117:6839/1967595704] conn(0x563e0c2b3920 0x563e0c2b5d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:35:39.589 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:39.573+0000 7f382ffda740 4 client.25313 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-19T01:35:39.589 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:39.573+0000 7f382ffda740 1 -- 172.21.15.83:0/1449861937 --> [v2:172.21.15.117:6838/1967595704,v1:172.21.15.117:6839/1967595704] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563e0c20c160 con 0x563e0c2b3920 2024-08-19T01:35:39.589 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:39.574+0000 7f3826000640 1 --2- 172.21.15.83:0/1449861937 >> [v2:172.21.15.117:6838/1967595704,v1:172.21.15.117:6839/1967595704] conn(0x563e0c2b3920 0x563e0c2b5d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:35:39.589 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:39.574+0000 7f3826000640 1 --2- 172.21.15.83:0/1449861937 >> [v2:172.21.15.117:6838/1967595704,v1:172.21.15.117:6839/1967595704] conn(0x563e0c2b3920 0x563e0c2b5d10 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=aa2814dc32455752 server_cookie=534d83f33eef1aef in_seq=0 out_seq=0 2024-08-19T01:35:39.589 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:39.574+0000 7f3807e00640 10 client.25313 ms_handle_connect on v2:172.21.15.117:6838/1967595704 2024-08-19T01:35:39.589 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:39.575+0000 7f3807e00640 1 -- 172.21.15.83:0/1449861937 <== mds.0 v2:172.21.15.117:6838/1967595704 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x563e0c20c160 con 0x563e0c2b3920 2024-08-19T01:35:39.590 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:39.575+0000 7f3807e00640 10 client.25313 handle_command_reply: tid=0 2024-08-19T01:35:39.590 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:39.575+0000 7f382ffda740 1 client.25313 shutdown 2024-08-19T01:35:39.590 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:39.576+0000 7f382ffda740 1 -- 172.21.15.83:0/1449861937 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563e0c298b90 msgr2=0x563e0c2a0c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:35:39.590 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:39.576+0000 7f382ffda740 1 --2- 172.21.15.83:0/1449861937 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563e0c298b90 0x563e0c2a0c10 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f38180199f0 tx=0x7f3818019640 comp rx=0 tx=0).stop 2024-08-19T01:35:39.590 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:39.576+0000 7f382ffda740 1 -- 172.21.15.83:0/1449861937 shutdown_connections 2024-08-19T01:35:39.590 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:39.576+0000 7f382ffda740 1 --2- 172.21.15.83:0/1449861937 >> [v2:172.21.15.117:6838/1967595704,v1:172.21.15.117:6839/1967595704] conn(0x563e0c2b3920 0x563e0c2b5d10 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:35:39.591 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:39.576+0000 7f382ffda740 1 --2- 172.21.15.83:0/1449861937 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563e0c2a3d70 0x563e0c2a1690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:35:39.591 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:39.576+0000 7f382ffda740 1 --2- 172.21.15.83:0/1449861937 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563e0c2994b0 0x563e0c2a1150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:35:39.591 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:39.576+0000 7f382ffda740 1 --2- 172.21.15.83:0/1449861937 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563e0c298b90 0x563e0c2a0c10 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:35:39.591 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:39.576+0000 7f382ffda740 1 -- 172.21.15.83:0/1449861937 >> 172.21.15.83:0/1449861937 conn(0x563e0c295120 msgr2=0x563e0c29ae50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:35:39.591 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:39.576+0000 7f3807e00640 0 client.25313 ms_handle_reset on v2:172.21.15.117:6838/1967595704 2024-08-19T01:35:39.592 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:39.577+0000 7f382ffda740 1 -- 172.21.15.83:0/1449861937 shutdown_connections 2024-08-19T01:35:39.592 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:39.577+0000 7f382ffda740 1 -- 172.21.15.83:0/1449861937 wait complete. 2024-08-19T01:35:39.592 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:39.577+0000 7f382ffda740 20 client.25313 trim_cache size 0 max 16384 2024-08-19T01:35:39.597 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:39.595+0000 7f382ffda740 1 Processor -- start 2024-08-19T01:35:39.597 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:39.595+0000 7f382ffda740 1 -- start start 2024-08-19T01:35:39.598 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:39.595+0000 7f382ffda740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563e0c298b90 0x563e0c29b5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:35:39.598 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:39.595+0000 7f382ffda740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563e0c2994b0 0x563e0c294170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:35:39.598 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:39.596+0000 7f382ffda740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563e0c2a3d70 0x563e0c2bb450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:35:39.598 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:39.596+0000 7f382ffda740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f3808010dc0 con 0x563e0c2a3d70 2024-08-19T01:35:39.598 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:39.596+0000 7f382ffda740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f380800e690 con 0x563e0c298b90 2024-08-19T01:35:39.598 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:39.596+0000 7f382ffda740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f380800c4e0 con 0x563e0c2994b0 2024-08-19T01:35:39.599 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:39.596+0000 7f3826000640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563e0c2994b0 0x563e0c294170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:35:39.599 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:39.596+0000 7f3826000640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563e0c2994b0 0x563e0c294170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.83:49446/0 (socket says 172.21.15.83:49446) 2024-08-19T01:35:39.599 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:39.596+0000 7f3826000640 1 -- 172.21.15.83:0/1098584001 learned_addr learned my addr 172.21.15.83:0/1098584001 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-08-19T01:35:39.599 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:39.596+0000 7f3826a00640 1 --2- 172.21.15.83:0/1098584001 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563e0c298b90 0x563e0c29b5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:35:39.600 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:39.596+0000 7f3827400640 1 --2- 172.21.15.83:0/1098584001 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563e0c2a3d70 0x563e0c2bb450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:35:39.600 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:39.597+0000 7f3826a00640 1 -- 172.21.15.83:0/1098584001 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563e0c2994b0 msgr2=0x563e0c294170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:35:39.600 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:39.597+0000 7f3826a00640 1 --2- 172.21.15.83:0/1098584001 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563e0c2994b0 0x563e0c294170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:35:39.600 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:39.597+0000 7f3826a00640 1 -- 172.21.15.83:0/1098584001 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563e0c2a3d70 msgr2=0x563e0c2bb450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:35:39.600 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:39.597+0000 7f3826a00640 1 --2- 172.21.15.83:0/1098584001 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563e0c2a3d70 0x563e0c2bb450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:35:39.600 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:39.597+0000 7f3826a00640 1 -- 172.21.15.83:0/1098584001 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563e0c295df0 con 0x563e0c298b90 2024-08-19T01:35:39.600 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:39.597+0000 7f3826a00640 1 --2- 172.21.15.83:0/1098584001 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563e0c298b90 0x563e0c29b5f0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f3818011510 tx=0x7f381802b3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ea9e473d5038916d server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:35:39.600 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:39.598+0000 7f3812a00640 1 -- 172.21.15.83:0/1098584001 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3818031080 con 0x563e0c298b90 2024-08-19T01:35:39.600 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:39.598+0000 7f3812a00640 1 -- 172.21.15.83:0/1098584001 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f381800f3c0 con 0x563e0c298b90 2024-08-19T01:35:39.600 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:39.598+0000 7f3812a00640 1 -- 172.21.15.83:0/1098584001 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3818005350 con 0x563e0c298b90 2024-08-19T01:35:39.601 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:39.599+0000 7f382ffda740 1 -- 172.21.15.83:0/1098584001 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563e0c298b90 msgr2=0x563e0c29b5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:35:39.602 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:39.599+0000 7f382ffda740 1 --2- 172.21.15.83:0/1098584001 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563e0c298b90 0x563e0c29b5f0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f3818011510 tx=0x7f381802b3a0 comp rx=0 tx=0).stop 2024-08-19T01:35:39.602 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:39.599+0000 7f382ffda740 1 -- 172.21.15.83:0/1098584001 shutdown_connections 2024-08-19T01:35:39.602 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:39.599+0000 7f382ffda740 1 --2- 172.21.15.83:0/1098584001 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563e0c2a3d70 0x563e0c2bb450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:35:39.602 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:39.599+0000 7f382ffda740 1 --2- 172.21.15.83:0/1098584001 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563e0c2994b0 0x563e0c294170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:35:39.602 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:39.599+0000 7f382ffda740 1 --2- 172.21.15.83:0/1098584001 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563e0c298b90 0x563e0c29b5f0 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:35:39.602 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:39.599+0000 7f382ffda740 1 -- 172.21.15.83:0/1098584001 >> 172.21.15.83:0/1098584001 conn(0x563e0c295120 msgr2=0x563e0c293a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:35:39.602 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:39.599+0000 7f382ffda740 1 -- 172.21.15.83:0/1098584001 shutdown_connections 2024-08-19T01:35:39.602 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:39.599+0000 7f382ffda740 1 -- 172.21.15.83:0/1098584001 wait complete. 2024-08-19T01:35:39.602 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:39.600+0000 7f382ffda740 1 -- start start 2024-08-19T01:35:39.603 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:39.600+0000 7f382ffda740 1 Processor -- start 2024-08-19T01:35:39.604 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:39.601+0000 7f382ffda740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563e0c298b90 0x563e0c2ac5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:35:39.604 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:39.601+0000 7f382ffda740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563e0c2994b0 0x563e0c2a78f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:35:39.604 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:39.601+0000 7f382ffda740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563e0c2a3d70 0x563e0c2a7ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:35:39.604 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:39.601+0000 7f382ffda740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f380800cb50 con 0x563e0c298b90 2024-08-19T01:35:39.604 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:39.601+0000 7f382ffda740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f380800dfc0 con 0x563e0c2994b0 2024-08-19T01:35:39.604 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:39.601+0000 7f382ffda740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f380800d220 con 0x563e0c2a3d70 2024-08-19T01:35:39.604 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:39.601+0000 7f382ffda740 10 client.0 _ll_register_callbacks cb 0x7f3818011440 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-19T01:35:39.605 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:39.601+0000 7f3827400640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563e0c2a3d70 0x563e0c2a7ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:35:39.605 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:39.601+0000 7f3827400640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563e0c2a3d70 0x563e0c2a7ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.83:49454/0 (socket says 172.21.15.83:49454) 2024-08-19T01:35:39.605 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:39.601+0000 7f3827400640 1 -- 172.21.15.83:0/3650365449 learned_addr learned my addr 172.21.15.83:0/3650365449 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-08-19T01:35:39.605 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:39.601+0000 7f3826000640 1 --2- 172.21.15.83:0/3650365449 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563e0c2994b0 0x563e0c2a78f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:35:39.605 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:39.601+0000 7f3826a00640 1 --2- 172.21.15.83:0/3650365449 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563e0c298b90 0x563e0c2ac5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:35:39.606 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:39.602+0000 7f3827400640 1 -- 172.21.15.83:0/3650365449 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563e0c2994b0 msgr2=0x563e0c2a78f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:35:39.606 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:39.602+0000 7f3827400640 1 --2- 172.21.15.83:0/3650365449 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563e0c2994b0 0x563e0c2a78f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:35:39.606 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:39.602+0000 7f3827400640 1 -- 172.21.15.83:0/3650365449 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563e0c298b90 msgr2=0x563e0c2ac5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:35:39.606 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:39.602+0000 7f3827400640 1 --2- 172.21.15.83:0/3650365449 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563e0c298b90 0x563e0c2ac5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:35:39.606 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:39.602+0000 7f3827400640 1 -- 172.21.15.83:0/3650365449 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563e0c2a86f0 con 0x563e0c2a3d70 2024-08-19T01:35:39.606 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:39.602+0000 7f3826a00640 1 --2- 172.21.15.83:0/3650365449 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563e0c298b90 0x563e0c2ac5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-19T01:35:39.610 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:39.603+0000 7f3827400640 1 --2- 172.21.15.83:0/3650365449 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563e0c2a3d70 0x563e0c2a7ec0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f38200987c0 tx=0x7f38200a0260 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9897fe21e2fe8c3 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:35:39.610 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:39.603+0000 7f3807e00640 10 client.0 ms_handle_connect on v2:172.21.15.167:3300/0 2024-08-19T01:35:39.610 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:39.603+0000 7f3807e00640 1 -- 172.21.15.83:0/3650365449 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3820004d30 con 0x563e0c2a3d70 2024-08-19T01:35:39.614 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:39.603+0000 7f3807e00640 1 -- 172.21.15.83:0/3650365449 <== mon.1 v2:172.21.15.167:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8269+0+0 (secure 0 0 0) 0x7f38200a45d0 con 0x563e0c2a3d70 2024-08-19T01:35:39.614 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:39.603+0000 7f382ffda740 1 -- 172.21.15.83:0/3650365449 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f381800f3c0 con 0x563e0c2a3d70 2024-08-19T01:35:39.614 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:39.603+0000 7f3807e00640 1 -- 172.21.15.83:0/3650365449 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f38200a47d0 con 0x563e0c2a3d70 2024-08-19T01:35:39.614 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:39.603+0000 7f382ffda740 10 client.25294 fetch_fsmap learned FSMap version 25 2024-08-19T01:35:39.615 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:39.603+0000 7f382ffda740 1 -- 172.21.15.83:0/3650365449 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x563e0c31e590 con 0x563e0c2a3d70 2024-08-19T01:35:39.615 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:39.603+0000 7f3807e00640 1 -- 172.21.15.83:0/3650365449 <== mon.1 v2:172.21.15.167:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f381800f3c0 con 0x563e0c2a3d70 2024-08-19T01:35:39.615 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:39.604+0000 7f3807e00640 1 -- 172.21.15.83:0/3650365449 <== mon.1 v2:172.21.15.167:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f382009b360 con 0x563e0c2a3d70 2024-08-19T01:35:39.615 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:39.604+0000 7f382ffda740 10 client.25294 fetch_fsmap finished waiting for FSMap version 25 2024-08-19T01:35:39.615 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:39.604+0000 7f382ffda740 10 client.25294 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-19T01:35:39.616 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:39.604+0000 7f382ffda740 20 client.25294 populate_metadata read hostname 'smithi083' 2024-08-19T01:35:39.616 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:39.604+0000 7f382ffda740 1 --2- 172.21.15.83:0/3650365449 >> [v2:172.21.15.117:6838/1967595704,v1:172.21.15.117:6839/1967595704] conn(0x563e0c31ecb0 0x563e0c3210d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:35:39.616 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:39.604+0000 7f382ffda740 4 client.25294 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-08-19T01:35:39.616 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:39.604+0000 7f382ffda740 1 -- 172.21.15.83:0/3650365449 --> [v2:172.21.15.117:6838/1967595704,v1:172.21.15.117:6839/1967595704] -- command(tid 0: {"prefix": "scrub status"}) -- 0x563e0c206670 con 0x563e0c31ecb0 2024-08-19T01:35:39.616 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:39.605+0000 7f3826a00640 1 --2- 172.21.15.83:0/3650365449 >> [v2:172.21.15.117:6838/1967595704,v1:172.21.15.117:6839/1967595704] conn(0x563e0c31ecb0 0x563e0c3210d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:35:39.617 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:39.605+0000 7f3826a00640 1 --2- 172.21.15.83:0/3650365449 >> [v2:172.21.15.117:6838/1967595704,v1:172.21.15.117:6839/1967595704] conn(0x563e0c31ecb0 0x563e0c3210d0 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=79447cf719c04908 server_cookie=d4cdf95b6a2751b9 in_seq=0 out_seq=0 2024-08-19T01:35:39.617 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:39.605+0000 7f3807e00640 10 client.25294 ms_handle_connect on v2:172.21.15.117:6838/1967595704 2024-08-19T01:35:39.617 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:39.606+0000 7f3807e00640 1 -- 172.21.15.83:0/3650365449 <== mds.0 v2:172.21.15.117:6838/1967595704 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x563e0c206670 con 0x563e0c31ecb0 2024-08-19T01:35:39.617 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:39.606+0000 7f3807e00640 10 client.25294 handle_command_reply: tid=0 2024-08-19T01:35:39.617 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:39.606+0000 7f382ffda740 1 client.25294 shutdown 2024-08-19T01:35:39.617 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:39.606+0000 7f382ffda740 1 -- 172.21.15.83:0/3650365449 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563e0c2a3d70 msgr2=0x563e0c2a7ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:35:39.618 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:39.606+0000 7f382ffda740 1 --2- 172.21.15.83:0/3650365449 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563e0c2a3d70 0x563e0c2a7ec0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f38200987c0 tx=0x7f38200a0260 comp rx=0 tx=0).stop 2024-08-19T01:35:39.618 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:39.606+0000 7f382ffda740 1 -- 172.21.15.83:0/3650365449 shutdown_connections 2024-08-19T01:35:39.618 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:39.606+0000 7f382ffda740 1 --2- 172.21.15.83:0/3650365449 >> [v2:172.21.15.117:6838/1967595704,v1:172.21.15.117:6839/1967595704] conn(0x563e0c31ecb0 0x563e0c3210d0 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:35:39.618 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:39.606+0000 7f382ffda740 1 --2- 172.21.15.83:0/3650365449 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563e0c2a3d70 0x563e0c2a7ec0 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:35:39.618 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:39.606+0000 7f382ffda740 1 --2- 172.21.15.83:0/3650365449 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563e0c2994b0 0x563e0c2a78f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:35:39.619 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:39.606+0000 7f382ffda740 1 --2- 172.21.15.83:0/3650365449 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563e0c298b90 0x563e0c2ac5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:35:39.619 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:39.606+0000 7f382ffda740 1 -- 172.21.15.83:0/3650365449 >> 172.21.15.83:0/3650365449 conn(0x563e0c295120 msgr2=0x563e0c294280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:35:39.619 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:39.606+0000 7f3807e00640 0 client.25294 ms_handle_reset on v2:172.21.15.117:6838/1967595704 2024-08-19T01:35:39.619 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:39.607+0000 7f382ffda740 1 -- 172.21.15.83:0/3650365449 shutdown_connections 2024-08-19T01:35:39.619 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:39.607+0000 7f382ffda740 1 -- 172.21.15.83:0/3650365449 wait complete. 2024-08-19T01:35:39.619 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:39.607+0000 7f382ffda740 20 client.25294 trim_cache size 0 max 16384 2024-08-19T01:35:39.620 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-08-19T01:35:39.620 INFO:teuthology.orchestra.run.smithi083.stdout: "status": "IDLE (waiting for more scrubs)", 2024-08-19T01:35:39.620 INFO:teuthology.orchestra.run.smithi083.stdout: "scrubs": { 2024-08-19T01:35:39.620 INFO:teuthology.orchestra.run.smithi083.stdout: "70de75de-6b90-41fb-a490-767dc312a1dc": { 2024-08-19T01:35:39.621 INFO:teuthology.orchestra.run.smithi083.stdout: "path": "/", 2024-08-19T01:35:39.621 INFO:teuthology.orchestra.run.smithi083.stdout: "tag": "70de75de-6b90-41fb-a490-767dc312a1dc", 2024-08-19T01:35:39.621 INFO:teuthology.orchestra.run.smithi083.stdout: "options": "recursive,force" 2024-08-19T01:35:39.621 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-08-19T01:35:39.621 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-08-19T01:35:39.622 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-08-19T01:35:39.623 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:39.608+0000 7f382d600640 1 -- 172.21.15.83:0/3072912181 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f380c0a2c90 msgr2=0x7f380c0a5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:35:39.623 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:39.608+0000 7f382d600640 1 --2- 172.21.15.83:0/3072912181 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f380c0a2c90 0x7f380c0a5150 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f3814006010 tx=0x7f3814031510 comp rx=0 tx=0).stop 2024-08-19T01:35:39.624 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:39.608+0000 7f382d600640 1 -- 172.21.15.83:0/3072912181 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3828109ac0 msgr2=0x7f382819ec60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:35:39.624 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:39.608+0000 7f382d600640 1 --2- 172.21.15.83:0/3072912181 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3828109ac0 0x7f382819ec60 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f3820009fd0 tx=0x7f382000ef10 comp rx=0 tx=0).stop 2024-08-19T01:35:39.624 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:39.608+0000 7f382d600640 1 -- 172.21.15.83:0/3072912181 shutdown_connections 2024-08-19T01:35:39.624 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:39.608+0000 7f382d600640 1 --2- 172.21.15.83:0/3072912181 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f380c0a2c90 0x7f380c0a5150 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:35:39.624 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:39.608+0000 7f382d600640 1 --2- 172.21.15.83:0/3072912181 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3828109ac0 0x7f382819ec60 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:35:39.624 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:39.608+0000 7f382d600640 1 --2- 172.21.15.83:0/3072912181 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3828104460 0x7f382819a8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:35:39.625 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:39.608+0000 7f382d600640 1 --2- 172.21.15.83:0/3072912181 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3828103ac0 0x7f382819a390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:35:39.625 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:39.608+0000 7f382d600640 1 -- 172.21.15.83:0/3072912181 >> 172.21.15.83:0/3072912181 conn(0x7f38280ff7c0 msgr2=0x7f38280fff90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:35:39.625 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:39.608+0000 7f382d600640 1 -- 172.21.15.83:0/3072912181 shutdown_connections 2024-08-19T01:35:39.625 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:35:39.609+0000 7f382d600640 1 -- 172.21.15.83:0/3072912181 wait complete. 2024-08-19T01:35:39.904 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:35:39 smithi083 ceph-mon[17120]: pgmap v363: 97 pgs: 97 active+clean; 668 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 853 B/s wr, 8 op/s 2024-08-19T01:35:39.952 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:35:39 smithi117 ceph-mon[22016]: pgmap v363: 97 pgs: 97 active+clean; 668 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 853 B/s wr, 8 op/s 2024-08-19T01:35:39.965 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:35:39 smithi167 ceph-mon[21875]: pgmap v363: 97 pgs: 97 active+clean; 668 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 853 B/s wr, 8 op/s 2024-08-19T01:35:40.372 INFO:tasks.cephfs.filesystem:scrub status for tag:70de75de-6b90-41fb-a490-767dc312a1dc - {'path': '/', 'tag': '70de75de-6b90-41fb-a490-767dc312a1dc', 'options': 'recursive,force'} 2024-08-19T01:35:40.654 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:35:40 smithi083 ceph-mon[17120]: scrub summary: idle 2024-08-19T01:35:40.952 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:35:40 smithi117 ceph-mon[22016]: scrub summary: idle 2024-08-19T01:35:40.966 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:35:40 smithi167 ceph-mon[21875]: scrub summary: idle 2024-08-19T01:35:41.904 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:35:41 smithi083 ceph-mon[17120]: pgmap v364: 97 pgs: 97 active+clean; 668 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 767 B/s wr, 8 op/s 2024-08-19T01:35:41.952 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:35:41 smithi117 ceph-mon[22016]: pgmap v364: 97 pgs: 97 active+clean; 668 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 767 B/s wr, 8 op/s 2024-08-19T01:35:41.966 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:35:41 smithi167 ceph-mon[21875]: pgmap v364: 97 pgs: 97 active+clean; 668 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 767 B/s wr, 8 op/s 2024-08-19T01:35:43.672 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:43.670+0000 7fefba000640 1 -- 172.21.15.83:0/3996142645 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fefb41020f0 msgr2=0x7fefb41044e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:35:43.672 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:43.670+0000 7fefba000640 1 --2- 172.21.15.83:0/3996142645 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fefb41020f0 0x7fefb41044e0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7fefa8009860 tx=0x7fefa802a360 comp rx=0 tx=0).stop 2024-08-19T01:35:43.672 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:43.670+0000 7fefba000640 1 -- 172.21.15.83:0/3996142645 shutdown_connections 2024-08-19T01:35:43.672 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:43.670+0000 7fefba000640 1 --2- 172.21.15.83:0/3996142645 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fefb4107350 0x7fefb41097c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:35:43.672 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:43.670+0000 7fefba000640 1 --2- 172.21.15.83:0/3996142645 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fefb4104a20 0x7fefb4106e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:35:43.672 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:43.670+0000 7fefba000640 1 --2- 172.21.15.83:0/3996142645 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fefb41020f0 0x7fefb41044e0 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:35:43.672 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:43.670+0000 7fefba000640 1 -- 172.21.15.83:0/3996142645 >> 172.21.15.83:0/3996142645 conn(0x7fefb40fbcd0 msgr2=0x7fefb40fe110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:35:43.672 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:43.670+0000 7fefba000640 1 -- 172.21.15.83:0/3996142645 shutdown_connections 2024-08-19T01:35:43.672 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:43.670+0000 7fefba000640 1 -- 172.21.15.83:0/3996142645 wait complete. 2024-08-19T01:35:43.673 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:43.671+0000 7fefba000640 1 Processor -- start 2024-08-19T01:35:43.673 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:43.672+0000 7fefba000640 1 -- start start 2024-08-19T01:35:43.673 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:43.672+0000 7fefba000640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fefb41020f0 0x7fefb4101a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:35:43.674 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:43.672+0000 7fefba000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fefb4104a20 0x7fefb41000a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:35:43.674 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:43.672+0000 7fefba000640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fefb4107350 0x7fefb41005e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:35:43.674 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:43.672+0000 7fefba000640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fefb410d340 con 0x7fefb4104a20 2024-08-19T01:35:43.674 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:43.672+0000 7fefba000640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fefb410d1c0 con 0x7fefb4107350 2024-08-19T01:35:43.674 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:43.672+0000 7fefba000640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fefb410d4c0 con 0x7fefb41020f0 2024-08-19T01:35:43.675 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:43.672+0000 7fefb3e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fefb4104a20 0x7fefb41000a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:35:43.675 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:43.673+0000 7fefb3e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fefb4104a20 0x7fefb41000a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:51398/0 (socket says 172.21.15.83:51398) 2024-08-19T01:35:43.675 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:43.673+0000 7fefb3e00640 1 -- 172.21.15.83:0/3773844033 learned_addr learned my addr 172.21.15.83:0/3773844033 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-08-19T01:35:43.675 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:43.673+0000 7fefb8c00640 1 --2- 172.21.15.83:0/3773844033 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fefb41020f0 0x7fefb4101a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:35:43.675 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:43.673+0000 7fefb9600640 1 --2- 172.21.15.83:0/3773844033 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fefb4107350 0x7fefb41005e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:35:43.675 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:43.673+0000 7fefb8c00640 1 -- 172.21.15.83:0/3773844033 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fefb4107350 msgr2=0x7fefb41005e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:35:43.676 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:43.673+0000 7fefb8c00640 1 --2- 172.21.15.83:0/3773844033 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fefb4107350 0x7fefb41005e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:35:43.676 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:43.673+0000 7fefb8c00640 1 -- 172.21.15.83:0/3773844033 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fefb4104a20 msgr2=0x7fefb41000a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:35:43.676 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:43.673+0000 7fefb8c00640 1 --2- 172.21.15.83:0/3773844033 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fefb4104a20 0x7fefb41000a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:35:43.676 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:43.673+0000 7fefb8c00640 1 -- 172.21.15.83:0/3773844033 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fefb4100e70 con 0x7fefb41020f0 2024-08-19T01:35:43.676 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:43.673+0000 7fefb8c00640 1 --2- 172.21.15.83:0/3773844033 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fefb41020f0 0x7fefb4101a60 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fefa802a880 tx=0x7fefa8033d10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:35:43.676 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:43.674+0000 7fefb1600640 1 -- 172.21.15.83:0/3773844033 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fefa803f020 con 0x7fefb41020f0 2024-08-19T01:35:43.676 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:43.674+0000 7fefb1600640 1 -- 172.21.15.83:0/3773844033 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fefa80308f0 con 0x7fefb41020f0 2024-08-19T01:35:43.676 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:43.674+0000 7fefba000640 1 -- 172.21.15.83:0/3773844033 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fefb41a1b10 con 0x7fefb41020f0 2024-08-19T01:35:43.676 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:43.674+0000 7fefb1600640 1 -- 172.21.15.83:0/3773844033 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fefa8037040 con 0x7fefb41020f0 2024-08-19T01:35:43.676 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:43.674+0000 7fefba000640 1 -- 172.21.15.83:0/3773844033 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fefb41a2030 con 0x7fefb41020f0 2024-08-19T01:35:43.678 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:43.677+0000 7fefbe4bb740 1 Processor -- start 2024-08-19T01:35:43.678 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:43.677+0000 7fefbe4bb740 1 -- start start 2024-08-19T01:35:43.679 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:43.677+0000 7fefbe4bb740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fb4d7e7680 0x55fb4d7e7a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:35:43.679 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:43.677+0000 7fefbe4bb740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55fb4d7e7fc0 0x55fb4d7eca80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:35:43.679 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:43.677+0000 7fefbe4bb740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55fb4d7ed050 0x55fb4d7ef510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:35:43.679 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:43.677+0000 7fefbe4bb740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55fb4d74b890 con 0x55fb4d7e7680 2024-08-19T01:35:43.679 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:43.677+0000 7fefbe4bb740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55fb4d6e3d90 con 0x55fb4d7ed050 2024-08-19T01:35:43.679 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:43.677+0000 7fefbe4bb740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55fb4d747850 con 0x55fb4d7e7fc0 2024-08-19T01:35:43.680 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:43.678+0000 7fefb9600640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fb4d7e7680 0x55fb4d7e7a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:35:43.680 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:43.678+0000 7fefb3e00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55fb4d7ed050 0x55fb4d7ef510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:35:43.680 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:43.678+0000 7fefb9600640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fb4d7e7680 0x55fb4d7e7a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:51414/0 (socket says 172.21.15.83:51414) 2024-08-19T01:35:43.680 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:43.678+0000 7fefb9600640 1 -- 172.21.15.83:0/1647750047 learned_addr learned my addr 172.21.15.83:0/1647750047 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-08-19T01:35:43.680 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:43.678+0000 7fefb3e00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55fb4d7ed050 0x55fb4d7ef510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.83:38996/0 (socket says 172.21.15.83:38996) 2024-08-19T01:35:43.683 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:43.678+0000 7fefb1600640 1 -- 172.21.15.83:0/3773844033 <== mon.1 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 19) ==== 137884+0+0 (secure 0 0 0) 0x7fefa8042020 con 0x7fefb41020f0 2024-08-19T01:35:43.683 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:43.678+0000 7fefb3e00640 1 -- 172.21.15.83:0/1647750047 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55fb4d7e7fc0 msgr2=0x55fb4d7eca80 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-08-19T01:35:43.683 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:43.678+0000 7fefb3e00640 1 --2- 172.21.15.83:0/1647750047 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55fb4d7e7fc0 0x55fb4d7eca80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:35:43.683 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:43.678+0000 7fefb3e00640 1 -- 172.21.15.83:0/1647750047 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fb4d7e7680 msgr2=0x55fb4d7e7a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:35:43.684 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:43.678+0000 7fefb3e00640 1 --2- 172.21.15.83:0/1647750047 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fb4d7e7680 0x55fb4d7e7a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:35:43.684 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:43.678+0000 7fefb3e00640 1 -- 172.21.15.83:0/1647750047 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fb4d45f210 con 0x55fb4d7ed050 2024-08-19T01:35:43.684 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:43.679+0000 7fefb3e00640 1 --2- 172.21.15.83:0/1647750047 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55fb4d7ed050 0x55fb4d7ef510 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fefa0009c90 tx=0x7fefa0011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bc308dbc3e933c37 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:35:43.684 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:43.679+0000 7fefa5c00640 1 -- 172.21.15.83:0/1647750047 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fefa001e070 con 0x55fb4d7ed050 2024-08-19T01:35:43.684 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:43.679+0000 7fefa5c00640 1 -- 172.21.15.83:0/1647750047 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fefa0009e40 con 0x55fb4d7ed050 2024-08-19T01:35:43.684 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:43.679+0000 7fefa5c00640 1 -- 172.21.15.83:0/1647750047 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fefa0019670 con 0x55fb4d7ed050 2024-08-19T01:35:43.684 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:43.680+0000 7fefbe4bb740 1 -- 172.21.15.83:0/1647750047 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55fb4d7ed050 msgr2=0x55fb4d7ef510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:35:43.684 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:43.680+0000 7fefbe4bb740 1 --2- 172.21.15.83:0/1647750047 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55fb4d7ed050 0x55fb4d7ef510 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fefa0009c90 tx=0x7fefa0011810 comp rx=0 tx=0).stop 2024-08-19T01:35:43.684 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:43.680+0000 7fefb1600640 1 --2- 172.21.15.83:0/3773844033 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7fef9c0a2c10 0x7fef9c0a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:35:43.684 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:43.681+0000 7fefb1600640 1 -- 172.21.15.83:0/3773844033 <== mon.1 v2:172.21.15.167:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8269+0+0 (secure 0 0 0) 0x7fefa80eeef0 con 0x7fefb41020f0 2024-08-19T01:35:43.684 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:43.681+0000 7fefbe4bb740 1 -- 172.21.15.83:0/1647750047 shutdown_connections 2024-08-19T01:35:43.684 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:43.681+0000 7fefbe4bb740 1 --2- 172.21.15.83:0/1647750047 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55fb4d7ed050 0x55fb4d7ef510 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:35:43.684 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:43.681+0000 7fefbe4bb740 1 --2- 172.21.15.83:0/1647750047 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55fb4d7e7fc0 0x55fb4d7eca80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:35:43.685 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:43.681+0000 7fefbe4bb740 1 --2- 172.21.15.83:0/1647750047 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fb4d7e7680 0x55fb4d7e7a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:35:43.685 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:43.681+0000 7fefbe4bb740 1 -- 172.21.15.83:0/1647750047 >> 172.21.15.83:0/1647750047 conn(0x55fb4d7e2cf0 msgr2=0x55fb4d7e30f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:35:43.685 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:43.682+0000 7fefb9600640 1 --2- 172.21.15.83:0/3773844033 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7fef9c0a2c10 0x7fef9c0a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:35:43.685 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:43.682+0000 7fefbe4bb740 1 -- 172.21.15.83:0/1647750047 shutdown_connections 2024-08-19T01:35:43.685 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:43.682+0000 7fefbe4bb740 1 -- 172.21.15.83:0/1647750047 wait complete. 2024-08-19T01:35:43.687 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:43.682+0000 7fefbe4bb740 1 -- start start 2024-08-19T01:35:43.687 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:43.682+0000 7fefb9600640 1 --2- 172.21.15.83:0/3773844033 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7fef9c0a2c10 0x7fef9c0a50d0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fefac005fd0 tx=0x7fefac009500 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:35:43.688 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:43.683+0000 7fefbe4bb740 1 Processor -- start 2024-08-19T01:35:43.688 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:43.683+0000 7fefbe4bb740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55fb4d7e7680 0x55fb4d7f1990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:35:43.688 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:43.683+0000 7fefbe4bb740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fb4d7e7fc0 0x55fb4d7f1ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:35:43.688 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:43.683+0000 7fefbe4bb740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55fb4d7ed050 0x55fb4d7f8f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:35:43.688 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:43.683+0000 7fefbe4bb740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55fb4d7f00c0 con 0x55fb4d7e7fc0 2024-08-19T01:35:43.688 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:43.683+0000 7fefbe4bb740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55fb4d706f70 con 0x55fb4d7ed050 2024-08-19T01:35:43.688 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:43.683+0000 7fefbe4bb740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55fb4d7f0240 con 0x55fb4d7e7680 2024-08-19T01:35:43.688 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:43.683+0000 7fefbe4bb740 10 client.0 _ll_register_callbacks cb 0x55fb4d7438d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-19T01:35:43.688 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:43.684+0000 7fefb8c00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55fb4d7e7680 0x55fb4d7f1990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:35:43.688 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:43.684+0000 7fefb8c00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55fb4d7e7680 0x55fb4d7f1990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.83:43764/0 (socket says 172.21.15.83:43764) 2024-08-19T01:35:43.688 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:43.684+0000 7fefb8c00640 1 -- 172.21.15.83:0/1393943454 learned_addr learned my addr 172.21.15.83:0/1393943454 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-08-19T01:35:43.689 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:43.684+0000 7fefb3e00640 1 --2- 172.21.15.83:0/1393943454 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fb4d7e7fc0 0x55fb4d7f1ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:35:43.689 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:43.684+0000 7fefb9600640 1 --2- 172.21.15.83:0/1393943454 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55fb4d7ed050 0x55fb4d7f8f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:35:43.689 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:43.684+0000 7fefb8c00640 1 -- 172.21.15.83:0/1393943454 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55fb4d7ed050 msgr2=0x55fb4d7f8f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:35:43.689 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:43.684+0000 7fefb8c00640 1 --2- 172.21.15.83:0/1393943454 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55fb4d7ed050 0x55fb4d7f8f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:35:43.689 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:43.684+0000 7fefb8c00640 1 -- 172.21.15.83:0/1393943454 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fb4d7e7fc0 msgr2=0x55fb4d7f1ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:35:43.689 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:43.684+0000 7fefb8c00640 1 --2- 172.21.15.83:0/1393943454 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fb4d7e7fc0 0x55fb4d7f1ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:35:43.689 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:43.684+0000 7fefb8c00640 1 -- 172.21.15.83:0/1393943454 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fb4d52b620 con 0x55fb4d7e7680 2024-08-19T01:35:43.689 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:43.684+0000 7fefb8c00640 1 --2- 172.21.15.83:0/1393943454 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55fb4d7e7680 0x55fb4d7f1990 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fefa8035040 tx=0x7fefa80bb450 comp rx=0 tx=0).ready entity=mon.1 client_cookie=23db18c67b0364f5 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:35:43.689 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:43.684+0000 7fef9a000640 10 client.0 ms_handle_connect on v2:172.21.15.167:3300/0 2024-08-19T01:35:43.689 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:43.685+0000 7fef9a000640 1 -- 172.21.15.83:0/1393943454 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fefa80bafe0 con 0x55fb4d7e7680 2024-08-19T01:35:43.689 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:43.685+0000 7fef9a000640 1 -- 172.21.15.83:0/1393943454 <== mon.1 v2:172.21.15.167:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8269+0+0 (secure 0 0 0) 0x7fefa80bbae0 con 0x55fb4d7e7680 2024-08-19T01:35:43.689 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:43.685+0000 7fefbe4bb740 1 -- 172.21.15.83:0/1393943454 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55fb4d758fc0 con 0x55fb4d7e7680 2024-08-19T01:35:43.689 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:43.685+0000 7fef9a000640 1 -- 172.21.15.83:0/1393943454 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fefa80bbd80 con 0x55fb4d7e7680 2024-08-19T01:35:43.689 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:43.685+0000 7fefbe4bb740 10 client.25303 fetch_fsmap learned FSMap version 25 2024-08-19T01:35:43.690 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:43.685+0000 7fefbe4bb740 1 -- 172.21.15.83:0/1393943454 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55fb4d7f01e0 con 0x55fb4d7e7680 2024-08-19T01:35:43.690 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:43.685+0000 7fef9a000640 1 -- 172.21.15.83:0/1393943454 <== mon.1 v2:172.21.15.167:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x55fb4d758fc0 con 0x55fb4d7e7680 2024-08-19T01:35:43.690 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:43.685+0000 7fef9a000640 1 -- 172.21.15.83:0/1393943454 <== mon.1 v2:172.21.15.167:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fefa803a030 con 0x55fb4d7e7680 2024-08-19T01:35:43.690 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:43.686+0000 7fefbe4bb740 10 client.25303 fetch_fsmap finished waiting for FSMap version 25 2024-08-19T01:35:43.690 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:43.686+0000 7fefbe4bb740 10 client.25303 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-08-19T01:35:43.690 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:43.686+0000 7fefbe4bb740 20 client.25303 populate_metadata read hostname 'smithi083' 2024-08-19T01:35:43.690 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:43.686+0000 7fefbe4bb740 1 --2- 172.21.15.83:0/1393943454 >> [v2:172.21.15.117:6838/1967595704,v1:172.21.15.117:6839/1967595704] conn(0x55fb4d7fd7b0 0x55fb4d7ffbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:35:43.690 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:43.686+0000 7fefbe4bb740 4 client.25303 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-19T01:35:43.690 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:43.686+0000 7fefbe4bb740 1 -- 172.21.15.83:0/1393943454 --> [v2:172.21.15.117:6838/1967595704,v1:172.21.15.117:6839/1967595704] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55fb4d753310 con 0x55fb4d7fd7b0 2024-08-19T01:35:43.690 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:43.686+0000 7fefb3e00640 1 --2- 172.21.15.83:0/1393943454 >> [v2:172.21.15.117:6838/1967595704,v1:172.21.15.117:6839/1967595704] conn(0x55fb4d7fd7b0 0x55fb4d7ffbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:35:43.691 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:43.687+0000 7fef9a000640 10 client.25303 ms_handle_connect on v2:172.21.15.117:6838/1967595704 2024-08-19T01:35:43.692 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:43.687+0000 7fefb3e00640 1 --2- 172.21.15.83:0/1393943454 >> [v2:172.21.15.117:6838/1967595704,v1:172.21.15.117:6839/1967595704] conn(0x55fb4d7fd7b0 0x55fb4d7ffbe0 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e85c5e4e6ef92eac server_cookie=9aec91a956ee41f4 in_seq=0 out_seq=0 2024-08-19T01:35:43.692 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:43.688+0000 7fef9a000640 1 -- 172.21.15.83:0/1393943454 <== mds.0 v2:172.21.15.117:6838/1967595704 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55fb4d753310 con 0x55fb4d7fd7b0 2024-08-19T01:35:43.692 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:43.688+0000 7fef9a000640 10 client.25303 handle_command_reply: tid=0 2024-08-19T01:35:43.692 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:43.688+0000 7fefbe4bb740 1 client.25303 shutdown 2024-08-19T01:35:43.692 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:43.688+0000 7fefbe4bb740 1 -- 172.21.15.83:0/1393943454 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55fb4d7e7680 msgr2=0x55fb4d7f1990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:35:43.692 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:43.688+0000 7fefbe4bb740 1 --2- 172.21.15.83:0/1393943454 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55fb4d7e7680 0x55fb4d7f1990 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fefa8035040 tx=0x7fefa80bb450 comp rx=0 tx=0).stop 2024-08-19T01:35:43.692 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:43.688+0000 7fefbe4bb740 1 -- 172.21.15.83:0/1393943454 shutdown_connections 2024-08-19T01:35:43.692 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:43.688+0000 7fefbe4bb740 1 --2- 172.21.15.83:0/1393943454 >> [v2:172.21.15.117:6838/1967595704,v1:172.21.15.117:6839/1967595704] conn(0x55fb4d7fd7b0 0x55fb4d7ffbe0 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:35:43.692 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:43.689+0000 7fefbe4bb740 1 --2- 172.21.15.83:0/1393943454 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55fb4d7ed050 0x55fb4d7f8f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:35:43.692 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:43.689+0000 7fefbe4bb740 1 --2- 172.21.15.83:0/1393943454 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fb4d7e7fc0 0x55fb4d7f1ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:35:43.692 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:43.689+0000 7fefbe4bb740 1 --2- 172.21.15.83:0/1393943454 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55fb4d7e7680 0x55fb4d7f1990 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:35:43.692 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:43.689+0000 7fefbe4bb740 1 -- 172.21.15.83:0/1393943454 >> 172.21.15.83:0/1393943454 conn(0x55fb4d7e2cf0 msgr2=0x55fb4d7e4500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:35:43.693 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:43.689+0000 7fef9a000640 0 client.25303 ms_handle_reset on v2:172.21.15.117:6838/1967595704 2024-08-19T01:35:43.693 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:43.689+0000 7fefbe4bb740 1 -- 172.21.15.83:0/1393943454 shutdown_connections 2024-08-19T01:35:43.693 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:43.689+0000 7fefbe4bb740 1 -- 172.21.15.83:0/1393943454 wait complete. 2024-08-19T01:35:43.693 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:43.689+0000 7fefbe4bb740 20 client.25303 trim_cache size 0 max 16384 2024-08-19T01:35:43.719 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:43.718+0000 7fefbe4bb740 1 Processor -- start 2024-08-19T01:35:43.719 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:43.718+0000 7fefbe4bb740 1 -- start start 2024-08-19T01:35:43.720 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:43.718+0000 7fefbe4bb740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fb4d7e2cf0 0x55fb4d7f1740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:35:43.720 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:43.718+0000 7fefbe4bb740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55fb4d7e7680 0x55fb4d7f8ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:35:43.720 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:43.718+0000 7fefbe4bb740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55fb4d7e7fc0 0x55fb4d7f1c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:35:43.720 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:43.718+0000 7fefbe4bb740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fefb41a4e80 con 0x55fb4d7e2cf0 2024-08-19T01:35:43.720 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:43.718+0000 7fefbe4bb740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fefb41a2750 con 0x55fb4d7e7fc0 2024-08-19T01:35:43.720 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:43.718+0000 7fefbe4bb740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fefb40724f0 con 0x55fb4d7e7680 2024-08-19T01:35:43.721 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:43.719+0000 7fefb8c00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fb4d7e2cf0 0x55fb4d7f1740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:35:43.721 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:43.719+0000 7fefb8c00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fb4d7e2cf0 0x55fb4d7f1740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:51438/0 (socket says 172.21.15.83:51438) 2024-08-19T01:35:43.721 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:43.719+0000 7fefb8c00640 1 -- 172.21.15.83:0/269024431 learned_addr learned my addr 172.21.15.83:0/269024431 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-08-19T01:35:43.721 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:43.719+0000 7fefb3e00640 1 --2- 172.21.15.83:0/269024431 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55fb4d7e7680 0x55fb4d7f8ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:35:43.721 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:43.719+0000 7fefb9600640 1 --2- 172.21.15.83:0/269024431 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55fb4d7e7fc0 0x55fb4d7f1c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:35:43.721 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:43.719+0000 7fefb8c00640 1 -- 172.21.15.83:0/269024431 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55fb4d7e7680 msgr2=0x55fb4d7f8ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:35:43.721 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:43.719+0000 7fefb8c00640 1 --2- 172.21.15.83:0/269024431 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55fb4d7e7680 0x55fb4d7f8ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:35:43.722 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:43.719+0000 7fefb8c00640 1 -- 172.21.15.83:0/269024431 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55fb4d7e7fc0 msgr2=0x55fb4d7f1c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:35:43.722 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:43.719+0000 7fefb8c00640 1 --2- 172.21.15.83:0/269024431 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55fb4d7e7fc0 0x55fb4d7f1c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:35:43.722 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:43.719+0000 7fefb8c00640 1 -- 172.21.15.83:0/269024431 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fb4d7ec450 con 0x55fb4d7e2cf0 2024-08-19T01:35:43.722 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:43.719+0000 7fefb8c00640 1 --2- 172.21.15.83:0/269024431 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fb4d7e2cf0 0x55fb4d7f1740 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7fefa802a950 tx=0x7fefa80b9b90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=84cc17d1d3316ad1 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:35:43.722 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:43.720+0000 7fefa5200640 1 -- 172.21.15.83:0/269024431 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fefa80bafe0 con 0x55fb4d7e2cf0 2024-08-19T01:35:43.722 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:43.720+0000 7fefa5200640 1 -- 172.21.15.83:0/269024431 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fefa80bfa30 con 0x55fb4d7e2cf0 2024-08-19T01:35:43.722 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:43.720+0000 7fefa5200640 1 -- 172.21.15.83:0/269024431 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fefa80bbb40 con 0x55fb4d7e2cf0 2024-08-19T01:35:43.723 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:43.721+0000 7fefbe4bb740 1 -- 172.21.15.83:0/269024431 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fb4d7e2cf0 msgr2=0x55fb4d7f1740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:35:43.723 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:43.721+0000 7fefbe4bb740 1 --2- 172.21.15.83:0/269024431 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fb4d7e2cf0 0x55fb4d7f1740 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7fefa802a950 tx=0x7fefa80b9b90 comp rx=0 tx=0).stop 2024-08-19T01:35:43.723 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:43.721+0000 7fefbe4bb740 1 -- 172.21.15.83:0/269024431 shutdown_connections 2024-08-19T01:35:43.723 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:43.721+0000 7fefbe4bb740 1 --2- 172.21.15.83:0/269024431 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55fb4d7e7fc0 0x55fb4d7f1c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:35:43.723 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:43.721+0000 7fefbe4bb740 1 --2- 172.21.15.83:0/269024431 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55fb4d7e7680 0x55fb4d7f8ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:35:43.723 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:43.721+0000 7fefbe4bb740 1 --2- 172.21.15.83:0/269024431 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fb4d7e2cf0 0x55fb4d7f1740 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:35:43.723 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:43.721+0000 7fefbe4bb740 1 -- 172.21.15.83:0/269024431 >> 172.21.15.83:0/269024431 conn(0x55fb4d7e3440 msgr2=0x55fb4d7e9160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:35:43.723 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:43.721+0000 7fefbe4bb740 1 -- 172.21.15.83:0/269024431 shutdown_connections 2024-08-19T01:35:43.723 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:43.721+0000 7fefbe4bb740 1 -- 172.21.15.83:0/269024431 wait complete. 2024-08-19T01:35:43.724 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:43.722+0000 7fefbe4bb740 1 -- start start 2024-08-19T01:35:43.724 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:43.722+0000 7fefbe4bb740 1 Processor -- start 2024-08-19T01:35:43.724 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:43.723+0000 7fefbe4bb740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55fb4d7e2cf0 0x55fb4d7f6140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:35:43.724 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:43.723+0000 7fefbe4bb740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55fb4d7e7680 0x55fb4d7f6680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:35:43.724 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:43.723+0000 7fefbe4bb740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fb4d7e7fc0 0x55fb4d7f7640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:35:43.725 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:43.723+0000 7fefbe4bb740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fefb4072b60 con 0x55fb4d7e7fc0 2024-08-19T01:35:43.725 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:43.723+0000 7fefbe4bb740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fefb4073fd0 con 0x55fb4d7e2cf0 2024-08-19T01:35:43.725 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:43.723+0000 7fefbe4bb740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fefb4073230 con 0x55fb4d7e7680 2024-08-19T01:35:43.725 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:43.723+0000 7fefbe4bb740 10 client.0 _ll_register_callbacks cb 0x7fefa8005ab0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-19T01:35:43.725 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:43.723+0000 7fefb3e00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55fb4d7e7680 0x55fb4d7f6680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:35:43.727 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:43.723+0000 7fefb3e00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55fb4d7e7680 0x55fb4d7f6680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.83:43784/0 (socket says 172.21.15.83:43784) 2024-08-19T01:35:43.727 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:43.723+0000 7fefb3e00640 1 -- 172.21.15.83:0/613355647 learned_addr learned my addr 172.21.15.83:0/613355647 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-08-19T01:35:43.727 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:43.723+0000 7fefb9600640 1 --2- 172.21.15.83:0/613355647 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fb4d7e7fc0 0x55fb4d7f7640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:35:43.727 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:43.723+0000 7fefb8c00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55fb4d7e2cf0 0x55fb4d7f6140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:35:43.727 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:43.723+0000 7fefb3e00640 1 -- 172.21.15.83:0/613355647 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55fb4d7e2cf0 msgr2=0x55fb4d7f6140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:35:43.727 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:43.723+0000 7fefb3e00640 1 --2- 172.21.15.83:0/613355647 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55fb4d7e2cf0 0x55fb4d7f6140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:35:43.727 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:43.723+0000 7fefb3e00640 1 -- 172.21.15.83:0/613355647 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fb4d7e7fc0 msgr2=0x55fb4d7f7640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:35:43.727 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:43.723+0000 7fefb3e00640 1 --2- 172.21.15.83:0/613355647 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fb4d7e7fc0 0x55fb4d7f7640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:35:43.727 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:43.723+0000 7fefb3e00640 1 -- 172.21.15.83:0/613355647 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fb4d7f7c50 con 0x55fb4d7e7680 2024-08-19T01:35:43.727 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:43.724+0000 7fefb3e00640 1 --2- 172.21.15.83:0/613355647 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55fb4d7e7680 0x55fb4d7f6680 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x55fb4d7e3b70 tx=0x7fefa0005d80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2511a0833c3c1830 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:35:43.727 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:43.724+0000 7fef9a000640 10 client.0 ms_handle_connect on v2:172.21.15.167:3300/0 2024-08-19T01:35:43.727 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:43.724+0000 7fef9a000640 1 -- 172.21.15.83:0/613355647 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fefa001e040 con 0x55fb4d7e7680 2024-08-19T01:35:43.728 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:43.724+0000 7fef9a000640 1 -- 172.21.15.83:0/613355647 <== mon.1 v2:172.21.15.167:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8269+0+0 (secure 0 0 0) 0x7fefa00050b0 con 0x55fb4d7e7680 2024-08-19T01:35:43.728 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:43.724+0000 7fefbe4bb740 1 -- 172.21.15.83:0/613355647 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fefa80bfa30 con 0x55fb4d7e7680 2024-08-19T01:35:43.728 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:43.724+0000 7fef9a000640 1 -- 172.21.15.83:0/613355647 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fefa00052b0 con 0x55fb4d7e7680 2024-08-19T01:35:43.728 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:43.725+0000 7fefbe4bb740 10 client.25309 fetch_fsmap learned FSMap version 25 2024-08-19T01:35:43.728 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:43.725+0000 7fefbe4bb740 1 -- 172.21.15.83:0/613355647 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55fb4d7f8fd0 con 0x55fb4d7e7680 2024-08-19T01:35:43.728 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:43.725+0000 7fef9a000640 1 -- 172.21.15.83:0/613355647 <== mon.1 v2:172.21.15.167:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fefa80bfa30 con 0x55fb4d7e7680 2024-08-19T01:35:43.728 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:43.725+0000 7fef9a000640 1 -- 172.21.15.83:0/613355647 <== mon.1 v2:172.21.15.167:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fefa0027ae0 con 0x55fb4d7e7680 2024-08-19T01:35:43.728 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:43.725+0000 7fefbe4bb740 10 client.25309 fetch_fsmap finished waiting for FSMap version 25 2024-08-19T01:35:43.728 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:43.725+0000 7fefbe4bb740 10 client.25309 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-08-19T01:35:43.728 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:43.725+0000 7fefbe4bb740 20 client.25309 populate_metadata read hostname 'smithi083' 2024-08-19T01:35:43.728 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:43.725+0000 7fefbe4bb740 1 --2- 172.21.15.83:0/613355647 >> [v2:172.21.15.117:6838/1967595704,v1:172.21.15.117:6839/1967595704] conn(0x55fb4d7ed050 0x55fb4d86a900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:35:43.728 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:43.725+0000 7fefbe4bb740 4 client.25309 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-08-19T01:35:43.728 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:43.725+0000 7fefbe4bb740 1 -- 172.21.15.83:0/613355647 --> [v2:172.21.15.117:6838/1967595704,v1:172.21.15.117:6839/1967595704] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55fb4d6d07c0 con 0x55fb4d7ed050 2024-08-19T01:35:43.729 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:43.725+0000 7fefb8c00640 1 --2- 172.21.15.83:0/613355647 >> [v2:172.21.15.117:6838/1967595704,v1:172.21.15.117:6839/1967595704] conn(0x55fb4d7ed050 0x55fb4d86a900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:35:43.729 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:43.726+0000 7fefb8c00640 1 --2- 172.21.15.83:0/613355647 >> [v2:172.21.15.117:6838/1967595704,v1:172.21.15.117:6839/1967595704] conn(0x55fb4d7ed050 0x55fb4d86a900 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2556672d81c6c540 server_cookie=bc22744f037cda01 in_seq=0 out_seq=0 2024-08-19T01:35:43.729 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:43.726+0000 7fef9a000640 10 client.25309 ms_handle_connect on v2:172.21.15.117:6838/1967595704 2024-08-19T01:35:43.732 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:43.730+0000 7fef9a000640 1 -- 172.21.15.83:0/613355647 <== mds.0 v2:172.21.15.117:6838/1967595704 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55fb4d6d07c0 con 0x55fb4d7ed050 2024-08-19T01:35:43.732 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:43.730+0000 7fef9a000640 10 client.25309 handle_command_reply: tid=0 2024-08-19T01:35:43.732 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:43.730+0000 7fefbe4bb740 1 client.25309 shutdown 2024-08-19T01:35:43.732 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:43.730+0000 7fefbe4bb740 1 -- 172.21.15.83:0/613355647 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55fb4d7e7680 msgr2=0x55fb4d7f6680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:35:43.732 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:43.730+0000 7fefbe4bb740 1 --2- 172.21.15.83:0/613355647 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55fb4d7e7680 0x55fb4d7f6680 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x55fb4d7e3b70 tx=0x7fefa0005d80 comp rx=0 tx=0).stop 2024-08-19T01:35:43.732 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:43.730+0000 7fefbe4bb740 1 -- 172.21.15.83:0/613355647 shutdown_connections 2024-08-19T01:35:43.732 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:43.730+0000 7fefbe4bb740 1 --2- 172.21.15.83:0/613355647 >> [v2:172.21.15.117:6838/1967595704,v1:172.21.15.117:6839/1967595704] conn(0x55fb4d7ed050 0x55fb4d86a900 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:35:43.733 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:43.730+0000 7fefbe4bb740 1 --2- 172.21.15.83:0/613355647 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fb4d7e7fc0 0x55fb4d7f7640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:35:43.733 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:43.730+0000 7fefbe4bb740 1 --2- 172.21.15.83:0/613355647 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55fb4d7e7680 0x55fb4d7f6680 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:35:43.733 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:43.730+0000 7fef9a000640 0 client.25309 ms_handle_reset on v2:172.21.15.117:6838/1967595704 2024-08-19T01:35:43.733 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:43.730+0000 7fefbe4bb740 1 --2- 172.21.15.83:0/613355647 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55fb4d7e2cf0 0x55fb4d7f6140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:35:43.733 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:43.730+0000 7fefbe4bb740 1 -- 172.21.15.83:0/613355647 >> 172.21.15.83:0/613355647 conn(0x55fb4d7e3440 msgr2=0x55fb4d7f1740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:35:43.733 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:43.731+0000 7fefbe4bb740 1 -- 172.21.15.83:0/613355647 shutdown_connections 2024-08-19T01:35:43.733 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:43.731+0000 7fefbe4bb740 1 -- 172.21.15.83:0/613355647 wait complete. 2024-08-19T01:35:43.733 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:43.731+0000 7fefbe4bb740 20 client.25309 trim_cache size 0 max 16384 2024-08-19T01:35:43.733 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stdout:{ 2024-08-19T01:35:43.733 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stdout: "message": "", 2024-08-19T01:35:43.734 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stdout: "return_code": 0 2024-08-19T01:35:43.734 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stdout:} 2024-08-19T01:35:43.734 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:43.732+0000 7fefba000640 1 -- 172.21.15.83:0/3773844033 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7fef9c0a2c10 msgr2=0x7fef9c0a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:35:43.734 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:43.732+0000 7fefba000640 1 --2- 172.21.15.83:0/3773844033 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7fef9c0a2c10 0x7fef9c0a50d0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fefac005fd0 tx=0x7fefac009500 comp rx=0 tx=0).stop 2024-08-19T01:35:43.734 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:43.732+0000 7fefba000640 1 -- 172.21.15.83:0/3773844033 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fefb41020f0 msgr2=0x7fefb4101a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:35:43.734 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:43.732+0000 7fefba000640 1 --2- 172.21.15.83:0/3773844033 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fefb41020f0 0x7fefb4101a60 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fefa802a880 tx=0x7fefa8033d10 comp rx=0 tx=0).stop 2024-08-19T01:35:43.734 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:43.732+0000 7fefba000640 1 -- 172.21.15.83:0/3773844033 shutdown_connections 2024-08-19T01:35:43.735 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:43.732+0000 7fefba000640 1 --2- 172.21.15.83:0/3773844033 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7fef9c0a2c10 0x7fef9c0a50d0 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:35:43.735 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:43.732+0000 7fefba000640 1 --2- 172.21.15.83:0/3773844033 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fefb4107350 0x7fefb41005e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:35:43.735 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:43.732+0000 7fefba000640 1 --2- 172.21.15.83:0/3773844033 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fefb4104a20 0x7fefb41000a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:35:43.735 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:43.732+0000 7fefba000640 1 --2- 172.21.15.83:0/3773844033 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fefb41020f0 0x7fefb4101a60 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:35:43.735 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:43.732+0000 7fefba000640 1 -- 172.21.15.83:0/3773844033 >> 172.21.15.83:0/3773844033 conn(0x7fefb40fbcd0 msgr2=0x7fefb4102af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:35:43.735 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:43.732+0000 7fefba000640 1 -- 172.21.15.83:0/3773844033 shutdown_connections 2024-08-19T01:35:43.735 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:43.733+0000 7fefba000640 1 -- 172.21.15.83:0/3773844033 wait complete. 2024-08-19T01:35:43.904 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:35:43 smithi083 ceph-mon[17120]: pgmap v365: 97 pgs: 97 active+clean; 668 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 767 B/s wr, 8 op/s 2024-08-19T01:35:43.952 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:35:43 smithi117 ceph-mon[22016]: pgmap v365: 97 pgs: 97 active+clean; 668 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 767 B/s wr, 8 op/s 2024-08-19T01:35:43.966 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:35:43 smithi167 ceph-mon[21875]: pgmap v365: 97 pgs: 97 active+clean; 668 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 767 B/s wr, 8 op/s 2024-08-19T01:35:45.904 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:35:45 smithi083 ceph-mon[17120]: pgmap v366: 97 pgs: 97 active+clean; 668 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 597 B/s wr, 8 op/s 2024-08-19T01:35:45.952 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:35:45 smithi117 ceph-mon[22016]: pgmap v366: 97 pgs: 97 active+clean; 668 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 597 B/s wr, 8 op/s 2024-08-19T01:35:45.966 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:35:45 smithi167 ceph-mon[21875]: pgmap v366: 97 pgs: 97 active+clean; 668 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 597 B/s wr, 8 op/s 2024-08-19T01:35:47.904 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:35:47 smithi083 ceph-mon[17120]: pgmap v367: 97 pgs: 97 active+clean; 669 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 682 B/s wr, 8 op/s 2024-08-19T01:35:47.952 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:35:47 smithi117 ceph-mon[22016]: pgmap v367: 97 pgs: 97 active+clean; 669 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 682 B/s wr, 8 op/s 2024-08-19T01:35:47.966 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:35:47 smithi167 ceph-mon[21875]: pgmap v367: 97 pgs: 97 active+clean; 669 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 682 B/s wr, 8 op/s 2024-08-19T01:35:49.904 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:35:49 smithi083 ceph-mon[17120]: pgmap v368: 97 pgs: 97 active+clean; 669 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 255 B/s wr, 7 op/s 2024-08-19T01:35:49.952 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:35:49 smithi117 ceph-mon[22016]: pgmap v368: 97 pgs: 97 active+clean; 669 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 255 B/s wr, 7 op/s 2024-08-19T01:35:49.966 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:35:49 smithi167 ceph-mon[21875]: pgmap v368: 97 pgs: 97 active+clean; 669 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 255 B/s wr, 7 op/s 2024-08-19T01:35:51.905 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:35:51 smithi083 ceph-mon[17120]: pgmap v369: 97 pgs: 97 active+clean; 669 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 7 op/s 2024-08-19T01:35:51.952 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:35:51 smithi117 ceph-mon[22016]: pgmap v369: 97 pgs: 97 active+clean; 669 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 7 op/s 2024-08-19T01:35:51.966 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:35:51 smithi167 ceph-mon[21875]: pgmap v369: 97 pgs: 97 active+clean; 669 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 7 op/s 2024-08-19T01:35:53.904 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:35:53 smithi083 ceph-mon[17120]: pgmap v370: 97 pgs: 97 active+clean; 669 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 255 B/s wr, 6 op/s 2024-08-19T01:35:53.952 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:35:53 smithi117 ceph-mon[22016]: pgmap v370: 97 pgs: 97 active+clean; 669 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 255 B/s wr, 6 op/s 2024-08-19T01:35:53.966 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:35:53 smithi167 ceph-mon[21875]: pgmap v370: 97 pgs: 97 active+clean; 669 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 255 B/s wr, 6 op/s 2024-08-19T01:35:55.904 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:35:55 smithi083 ceph-mon[17120]: pgmap v371: 97 pgs: 97 active+clean; 669 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 255 B/s wr, 6 op/s 2024-08-19T01:35:55.952 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:35:55 smithi117 ceph-mon[22016]: pgmap v371: 97 pgs: 97 active+clean; 669 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 255 B/s wr, 6 op/s 2024-08-19T01:35:55.966 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:35:55 smithi167 ceph-mon[21875]: pgmap v371: 97 pgs: 97 active+clean; 669 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 255 B/s wr, 6 op/s 2024-08-19T01:35:56.496 INFO:tasks.workunit.client.0.smithi083.stderr:Updating files: 65% (8458/12849) Updating files: 66% (8481/12849) Updating files: 67% (8609/12849) Updating files: 68% (8738/12849) Updating files: 69% (8866/12849) Updating files: 70% (8995/12849) Updating files: 71% (9123/12849) Updating files: 72% (9252/12849) Updating files: 73% (9380/12849) Updating files: 74% (9509/12849) Updating files: 75% (9637/12849) Updating files: 76% (9766/12849) Updating files: 77% (9894/12849) Updating files: 78% (10023/12849) Updating files: 79% (10151/12849) Updating files: 80% (10280/12849) Updating files: 81% (10408/12849) Updating files: 82% (10537/12849) Updating files: 83% (10665/12849) Updating files: 84% (10794/12849) Updating files: 85% (10922/12849) Updating files: 86% (11051/12849) Updating files: 87% (11179/12849) Updating files: 88% (11308/12849) Updating files: 89% (11436/12849) Updating files: 90% (11565/12849) Updating files: 91% (11693/12849) Updating files: 92% (11822/12849) Updating files: 93% (11950/12849) Updating files: 94% (12079/12849) Updating files: 95% (12207/12849) Updating files: 96% (12336/12849) Updating files: 97% (12464/12849) Updating files: 98% (12593/12849) Updating files: 99% (12721/12849) Updating files: 100% (12849/12849) Updating files: 100% (12849/12849), done. 2024-08-19T01:35:56.855 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:56.853+0000 7f281d200640 1 -- 172.21.15.83:0/3451841883 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f28181012b0 msgr2=0x7f2818101730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:35:56.856 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:56.853+0000 7f281d200640 1 --2- 172.21.15.83:0/3451841883 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f28181012b0 0x7f2818101730 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f2804009860 tx=0x7f280402a340 comp rx=0 tx=0).stop 2024-08-19T01:35:56.856 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:56.854+0000 7f281d200640 1 -- 172.21.15.83:0/3451841883 shutdown_connections 2024-08-19T01:35:56.856 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:56.854+0000 7f281d200640 1 --2- 172.21.15.83:0/3451841883 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2818101c70 0x7f2818109ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:35:56.856 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:56.854+0000 7f281d200640 1 --2- 172.21.15.83:0/3451841883 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f28181012b0 0x7f2818101730 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:35:56.856 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:56.854+0000 7f281d200640 1 --2- 172.21.15.83:0/3451841883 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2818102d80 0x7f2818103180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:35:56.856 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:56.854+0000 7f281d200640 1 -- 172.21.15.83:0/3451841883 >> 172.21.15.83:0/3451841883 conn(0x7f28180fce00 msgr2=0x7f28180ff240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:35:56.856 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:56.854+0000 7f281d200640 1 -- 172.21.15.83:0/3451841883 shutdown_connections 2024-08-19T01:35:56.856 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:56.854+0000 7f281d200640 1 -- 172.21.15.83:0/3451841883 wait complete. 2024-08-19T01:35:56.859 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:56.855+0000 7f281d200640 1 Processor -- start 2024-08-19T01:35:56.859 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:56.855+0000 7f281d200640 1 -- start start 2024-08-19T01:35:56.859 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:56.855+0000 7f281d200640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f28181012b0 0x7f2818075cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:35:56.859 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:56.855+0000 7f281d200640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2818101c70 0x7f2818074310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:35:56.859 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:56.855+0000 7f281d200640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2818102d80 0x7f2818074850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:35:56.859 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:56.855+0000 7f281d200640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f281810d420 con 0x7f28181012b0 2024-08-19T01:35:56.860 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:56.855+0000 7f281d200640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f281810d2a0 con 0x7f2818102d80 2024-08-19T01:35:56.860 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:56.855+0000 7f281d200640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f281810d5a0 con 0x7f2818101c70 2024-08-19T01:35:56.860 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:56.855+0000 7f2817400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f28181012b0 0x7f2818075cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:35:56.860 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:56.855+0000 7f2817400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f28181012b0 0x7f2818075cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:45416/0 (socket says 172.21.15.83:45416) 2024-08-19T01:35:56.860 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:56.855+0000 7f2817400640 1 -- 172.21.15.83:0/2321086306 learned_addr learned my addr 172.21.15.83:0/2321086306 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-08-19T01:35:56.860 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:56.855+0000 7f2816a00640 1 --2- 172.21.15.83:0/2321086306 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2818101c70 0x7f2818074310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:35:56.860 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:56.856+0000 7f2817e00640 1 --2- 172.21.15.83:0/2321086306 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2818102d80 0x7f2818074850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:35:56.860 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:56.856+0000 7f2817400640 1 -- 172.21.15.83:0/2321086306 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2818101c70 msgr2=0x7f2818074310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:35:56.860 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:56.856+0000 7f2817400640 1 --2- 172.21.15.83:0/2321086306 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2818101c70 0x7f2818074310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:35:56.860 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:56.856+0000 7f2817400640 1 -- 172.21.15.83:0/2321086306 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2818102d80 msgr2=0x7f2818074850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:35:56.860 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:56.856+0000 7f2817400640 1 --2- 172.21.15.83:0/2321086306 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2818102d80 0x7f2818074850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:35:56.861 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:56.856+0000 7f2817400640 1 -- 172.21.15.83:0/2321086306 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f28180750e0 con 0x7f28181012b0 2024-08-19T01:35:56.861 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:56.856+0000 7f2817e00640 1 --2- 172.21.15.83:0/2321086306 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2818102d80 0x7f2818074850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-19T01:35:56.861 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:56.856+0000 7f2817400640 1 --2- 172.21.15.83:0/2321086306 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f28181012b0 0x7f2818075cd0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f280800e9c0 tx=0x7f280800ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:35:56.861 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:56.856+0000 7f280fe00640 1 -- 172.21.15.83:0/2321086306 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f280800cd80 con 0x7f28181012b0 2024-08-19T01:35:56.861 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:56.856+0000 7f280fe00640 1 -- 172.21.15.83:0/2321086306 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f28080035f0 con 0x7f28181012b0 2024-08-19T01:35:56.861 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:56.856+0000 7f281d200640 1 -- 172.21.15.83:0/2321086306 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f28181a1c60 con 0x7f28181012b0 2024-08-19T01:35:56.861 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:56.856+0000 7f280fe00640 1 -- 172.21.15.83:0/2321086306 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2808010690 con 0x7f28181012b0 2024-08-19T01:35:56.861 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:56.857+0000 7f281d200640 1 -- 172.21.15.83:0/2321086306 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f28181a2180 con 0x7f28181012b0 2024-08-19T01:35:56.864 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:56.859+0000 7f2821677740 1 Processor -- start 2024-08-19T01:35:56.864 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:56.859+0000 7f280fe00640 1 -- 172.21.15.83:0/2321086306 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 19) ==== 137884+0+0 (secure 0 0 0) 0x7f28080037a0 con 0x7f28181012b0 2024-08-19T01:35:56.864 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:56.860+0000 7f2821677740 1 -- start start 2024-08-19T01:35:56.865 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:56.860+0000 7f2821677740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5557372f4680 0x5557372f4a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:35:56.865 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:56.860+0000 7f2821677740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5557372f4fc0 0x5557372f9a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:35:56.865 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:56.860+0000 7f2821677740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5557372fa050 0x5557372fc510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:35:56.865 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:56.860+0000 7f2821677740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x555737258890 con 0x5557372fa050 2024-08-19T01:35:56.865 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:56.860+0000 7f2821677740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x5557371f0d90 con 0x5557372f4680 2024-08-19T01:35:56.865 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:56.860+0000 7f2821677740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x555737254850 con 0x5557372f4fc0 2024-08-19T01:35:56.865 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:56.860+0000 7f2817400640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5557372f4fc0 0x5557372f9a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:35:56.865 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:56.860+0000 7f2816a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5557372fa050 0x5557372fc510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:35:56.865 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:56.860+0000 7f2817400640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5557372f4fc0 0x5557372f9a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.83:57024/0 (socket says 172.21.15.83:57024) 2024-08-19T01:35:56.865 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:56.860+0000 7f2817400640 1 -- 172.21.15.83:0/1086163613 learned_addr learned my addr 172.21.15.83:0/1086163613 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-08-19T01:35:56.865 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:56.860+0000 7f2817e00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5557372f4680 0x5557372f4a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:35:56.865 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:56.861+0000 7f2817e00640 1 -- 172.21.15.83:0/1086163613 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5557372f4fc0 msgr2=0x5557372f9a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:35:56.866 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:56.861+0000 7f2817e00640 1 --2- 172.21.15.83:0/1086163613 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5557372f4fc0 0x5557372f9a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:35:56.866 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:56.861+0000 7f2817e00640 1 -- 172.21.15.83:0/1086163613 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5557372fa050 msgr2=0x5557372fc510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:35:56.866 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:56.861+0000 7f2817e00640 1 --2- 172.21.15.83:0/1086163613 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5557372fa050 0x5557372fc510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:35:56.866 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:56.861+0000 7f2817e00640 1 -- 172.21.15.83:0/1086163613 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555736f6c210 con 0x5557372f4680 2024-08-19T01:35:56.866 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:56.861+0000 7f2817e00640 1 --2- 172.21.15.83:0/1086163613 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5557372f4680 0x5557372f4a80 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f2810009ed0 tx=0x7f281000f7d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e9f04c02165ec0ad server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:35:56.866 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:56.861+0000 7f280fe00640 1 --2- 172.21.15.83:0/2321086306 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f27fc0a2c90 0x7f27fc0a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:35:56.866 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:56.861+0000 7f280cc00640 1 -- 172.21.15.83:0/1086163613 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2810004b20 con 0x5557372f4680 2024-08-19T01:35:56.866 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:56.861+0000 7f280cc00640 1 -- 172.21.15.83:0/1086163613 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2810004cc0 con 0x5557372f4680 2024-08-19T01:35:56.866 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:56.861+0000 7f280cc00640 1 -- 172.21.15.83:0/1086163613 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28100054f0 con 0x5557372f4680 2024-08-19T01:35:56.866 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:56.861+0000 7f280fe00640 1 -- 172.21.15.83:0/2321086306 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8269+0+0 (secure 0 0 0) 0x7f28080cf610 con 0x7f28181012b0 2024-08-19T01:35:56.866 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:56.862+0000 7f2817400640 1 --2- 172.21.15.83:0/2321086306 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f27fc0a2c90 0x7f27fc0a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:35:56.866 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:56.862+0000 7f2821677740 1 -- 172.21.15.83:0/1086163613 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5557372f4680 msgr2=0x5557372f4a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:35:56.867 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:56.862+0000 7f2817400640 1 --2- 172.21.15.83:0/2321086306 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f27fc0a2c90 0x7f27fc0a5150 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f28080980f0 tx=0x7f28080d5530 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:35:56.867 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:56.862+0000 7f2821677740 1 --2- 172.21.15.83:0/1086163613 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5557372f4680 0x5557372f4a80 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f2810009ed0 tx=0x7f281000f7d0 comp rx=0 tx=0).stop 2024-08-19T01:35:56.867 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:56.862+0000 7f2821677740 1 -- 172.21.15.83:0/1086163613 shutdown_connections 2024-08-19T01:35:56.867 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:56.862+0000 7f2821677740 1 --2- 172.21.15.83:0/1086163613 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5557372fa050 0x5557372fc510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:35:56.867 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:56.862+0000 7f2821677740 1 --2- 172.21.15.83:0/1086163613 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5557372f4fc0 0x5557372f9a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:35:56.867 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:56.862+0000 7f2821677740 1 --2- 172.21.15.83:0/1086163613 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5557372f4680 0x5557372f4a80 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:35:56.867 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:56.862+0000 7f2821677740 1 -- 172.21.15.83:0/1086163613 >> 172.21.15.83:0/1086163613 conn(0x5557372efcf0 msgr2=0x5557372f00f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:35:56.867 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:56.862+0000 7f2821677740 1 -- 172.21.15.83:0/1086163613 shutdown_connections 2024-08-19T01:35:56.867 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:56.862+0000 7f2821677740 1 -- 172.21.15.83:0/1086163613 wait complete. 2024-08-19T01:35:56.867 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:56.863+0000 7f2821677740 1 -- start start 2024-08-19T01:35:56.867 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:56.863+0000 7f2821677740 1 Processor -- start 2024-08-19T01:35:56.871 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:56.863+0000 7f2821677740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5557372f4680 0x5557372fe8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:35:56.871 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:56.863+0000 7f2821677740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5557372f4fc0 0x5557372fee20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:35:56.871 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:56.863+0000 7f2821677740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5557372fa050 0x555737305e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:35:56.871 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:56.864+0000 7f2821677740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x5557372fd0c0 con 0x5557372f4680 2024-08-19T01:35:56.872 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:56.864+0000 7f2821677740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x555737213f70 con 0x5557372f4fc0 2024-08-19T01:35:56.872 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:56.864+0000 7f2821677740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x5557372fd240 con 0x5557372fa050 2024-08-19T01:35:56.872 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:56.864+0000 7f2821677740 10 client.0 _ll_register_callbacks cb 0x5557372508d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-19T01:35:56.872 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:56.864+0000 7f2817e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5557372f4680 0x5557372fe8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:35:56.872 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:56.864+0000 7f2817e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5557372f4680 0x5557372fe8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:45444/0 (socket says 172.21.15.83:45444) 2024-08-19T01:35:56.872 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:56.864+0000 7f2817e00640 1 -- 172.21.15.83:0/2151308448 learned_addr learned my addr 172.21.15.83:0/2151308448 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-08-19T01:35:56.872 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:56.864+0000 7f2816a00640 1 --2- 172.21.15.83:0/2151308448 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5557372f4fc0 0x5557372fee20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:35:56.872 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:56.864+0000 7f2817e00640 1 --2- 172.21.15.83:0/2151308448 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5557372fa050 0x555737305e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:35:56.872 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:56.864+0000 7f2817e00640 1 -- 172.21.15.83:0/2151308448 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5557372fa050 msgr2=0x555737305e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:35:56.872 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:56.864+0000 7f2817e00640 1 --2- 172.21.15.83:0/2151308448 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5557372fa050 0x555737305e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:35:56.872 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:56.864+0000 7f2817e00640 1 -- 172.21.15.83:0/2151308448 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5557372f4fc0 msgr2=0x5557372fee20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:35:56.872 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:56.864+0000 7f2817e00640 1 --2- 172.21.15.83:0/2151308448 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5557372f4fc0 0x5557372fee20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:35:56.872 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:56.864+0000 7f2817e00640 1 -- 172.21.15.83:0/2151308448 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555737038620 con 0x5557372f4680 2024-08-19T01:35:56.872 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:56.865+0000 7f2817e00640 1 --2- 172.21.15.83:0/2151308448 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5557372f4680 0x5557372fe8e0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f2810009a60 tx=0x7f28100099b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b1dfe096faa6f44b server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:35:56.873 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:56.865+0000 7f2801600640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-08-19T01:35:56.873 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:56.865+0000 7f2801600640 1 -- 172.21.15.83:0/2151308448 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2810003970 con 0x5557372f4680 2024-08-19T01:35:56.873 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:56.865+0000 7f2801600640 1 -- 172.21.15.83:0/2151308448 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8269+0+0 (secure 0 0 0) 0x7f2810003b10 con 0x5557372f4680 2024-08-19T01:35:56.873 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:56.865+0000 7f2821677740 1 -- 172.21.15.83:0/2151308448 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2810004cc0 con 0x5557372f4680 2024-08-19T01:35:56.873 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:56.865+0000 7f2801600640 1 -- 172.21.15.83:0/2151308448 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f2810003d10 con 0x5557372f4680 2024-08-19T01:35:56.873 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:56.865+0000 7f2821677740 10 client.15270 fetch_fsmap learned FSMap version 25 2024-08-19T01:35:56.873 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:56.865+0000 7f2821677740 1 -- 172.21.15.83:0/2151308448 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x555737308fe0 con 0x5557372f4680 2024-08-19T01:35:56.873 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:56.865+0000 7f2801600640 1 -- 172.21.15.83:0/2151308448 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2810004cc0 con 0x5557372f4680 2024-08-19T01:35:56.873 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:56.866+0000 7f2801600640 1 -- 172.21.15.83:0/2151308448 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f281000b040 con 0x5557372f4680 2024-08-19T01:35:56.873 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:56.866+0000 7f2821677740 10 client.15270 fetch_fsmap finished waiting for FSMap version 25 2024-08-19T01:35:56.873 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:56.866+0000 7f2821677740 10 client.15270 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-08-19T01:35:56.873 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:56.866+0000 7f2821677740 20 client.15270 populate_metadata read hostname 'smithi083' 2024-08-19T01:35:56.873 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:56.866+0000 7f2821677740 1 --2- 172.21.15.83:0/2151308448 >> [v2:172.21.15.117:6838/1967595704,v1:172.21.15.117:6839/1967595704] conn(0x55573730a450 0x55573730c840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:35:56.874 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:56.866+0000 7f2821677740 4 client.15270 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-19T01:35:56.874 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:56.866+0000 7f2821677740 1 -- 172.21.15.83:0/2151308448 --> [v2:172.21.15.117:6838/1967595704,v1:172.21.15.117:6839/1967595704] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555737265fc0 con 0x55573730a450 2024-08-19T01:35:56.874 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:56.866+0000 7f2816a00640 1 --2- 172.21.15.83:0/2151308448 >> [v2:172.21.15.117:6838/1967595704,v1:172.21.15.117:6839/1967595704] conn(0x55573730a450 0x55573730c840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:35:56.874 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:56.867+0000 7f2801600640 10 client.15270 ms_handle_connect on v2:172.21.15.117:6838/1967595704 2024-08-19T01:35:56.874 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:56.867+0000 7f2816a00640 1 --2- 172.21.15.83:0/2151308448 >> [v2:172.21.15.117:6838/1967595704,v1:172.21.15.117:6839/1967595704] conn(0x55573730a450 0x55573730c840 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9d4dcfd2c9d978e4 server_cookie=f4d92ce268c2ae43 in_seq=0 out_seq=0 2024-08-19T01:35:56.874 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:56.868+0000 7f2801600640 1 -- 172.21.15.83:0/2151308448 <== mds.0 v2:172.21.15.117:6838/1967595704 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x555737265fc0 con 0x55573730a450 2024-08-19T01:35:56.874 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:56.868+0000 7f2801600640 10 client.15270 handle_command_reply: tid=0 2024-08-19T01:35:56.874 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:56.868+0000 7f2821677740 1 client.15270 shutdown 2024-08-19T01:35:56.874 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:56.868+0000 7f2821677740 1 -- 172.21.15.83:0/2151308448 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5557372f4680 msgr2=0x5557372fe8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:35:56.874 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:56.868+0000 7f2821677740 1 --2- 172.21.15.83:0/2151308448 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5557372f4680 0x5557372fe8e0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f2810009a60 tx=0x7f28100099b0 comp rx=0 tx=0).stop 2024-08-19T01:35:56.874 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:56.868+0000 7f2821677740 1 -- 172.21.15.83:0/2151308448 shutdown_connections 2024-08-19T01:35:56.874 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:56.868+0000 7f2821677740 1 --2- 172.21.15.83:0/2151308448 >> [v2:172.21.15.117:6838/1967595704,v1:172.21.15.117:6839/1967595704] conn(0x55573730a450 0x55573730c840 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:35:56.875 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:56.868+0000 7f2821677740 1 --2- 172.21.15.83:0/2151308448 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5557372fa050 0x555737305e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:35:56.875 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:56.868+0000 7f2821677740 1 --2- 172.21.15.83:0/2151308448 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5557372f4fc0 0x5557372fee20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:35:56.875 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:56.868+0000 7f2801600640 0 client.15270 ms_handle_reset on v2:172.21.15.117:6838/1967595704 2024-08-19T01:35:56.875 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:56.868+0000 7f2821677740 1 --2- 172.21.15.83:0/2151308448 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5557372f4680 0x5557372fe8e0 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:35:56.875 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:56.868+0000 7f2821677740 1 -- 172.21.15.83:0/2151308448 >> 172.21.15.83:0/2151308448 conn(0x5557372efcf0 msgr2=0x5557372f1500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:35:56.875 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:56.868+0000 7f2821677740 1 -- 172.21.15.83:0/2151308448 shutdown_connections 2024-08-19T01:35:56.876 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:56.868+0000 7f2821677740 1 -- 172.21.15.83:0/2151308448 wait complete. 2024-08-19T01:35:56.876 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:56.868+0000 7f2821677740 20 client.15270 trim_cache size 0 max 16384 2024-08-19T01:35:56.885 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:56.883+0000 7f2821677740 1 Processor -- start 2024-08-19T01:35:56.885 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:56.883+0000 7f2821677740 1 -- start start 2024-08-19T01:35:56.885 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:56.883+0000 7f2821677740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5557372f4680 0x5557372f00d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:35:56.885 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:56.884+0000 7f2821677740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5557372f4fc0 0x555737305e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:35:56.885 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:56.884+0000 7f2821677740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5557372fa050 0x5557372f0610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:35:56.885 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:56.884+0000 7f2821677740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f28181ad800 con 0x5557372fa050 2024-08-19T01:35:56.885 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:56.884+0000 7f2821677740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f28181ab0d0 con 0x5557372f4680 2024-08-19T01:35:56.885 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:56.884+0000 7f2821677740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f28181a8f20 con 0x5557372f4fc0 2024-08-19T01:35:56.887 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:56.884+0000 7f2817e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5557372fa050 0x5557372f0610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:35:56.887 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:56.884+0000 7f2817e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5557372fa050 0x5557372f0610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:45454/0 (socket says 172.21.15.83:45454) 2024-08-19T01:35:56.887 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:56.884+0000 7f2817e00640 1 -- 172.21.15.83:0/2003710994 learned_addr learned my addr 172.21.15.83:0/2003710994 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-08-19T01:35:56.888 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:56.884+0000 7f2816a00640 1 --2- 172.21.15.83:0/2003710994 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5557372f4fc0 0x555737305e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:35:56.888 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:56.884+0000 7f2817e00640 1 --2- 172.21.15.83:0/2003710994 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5557372f4680 0x5557372f00d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:35:56.888 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:56.884+0000 7f2817e00640 1 -- 172.21.15.83:0/2003710994 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5557372f4fc0 msgr2=0x555737305e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:35:56.888 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:56.884+0000 7f2817e00640 1 --2- 172.21.15.83:0/2003710994 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5557372f4fc0 0x555737305e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:35:56.888 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:56.884+0000 7f2817e00640 1 -- 172.21.15.83:0/2003710994 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5557372f4680 msgr2=0x5557372f00d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:35:56.888 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:56.884+0000 7f2817e00640 1 --2- 172.21.15.83:0/2003710994 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5557372f4680 0x5557372f00d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:35:56.888 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:56.884+0000 7f2817e00640 1 -- 172.21.15.83:0/2003710994 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5557372fca30 con 0x5557372fa050 2024-08-19T01:35:56.888 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:56.885+0000 7f2817e00640 1 --2- 172.21.15.83:0/2003710994 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5557372fa050 0x5557372f0610 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f2810004300 tx=0x7f281000f7d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=46d27e892f99fdd8 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:35:56.888 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:56.885+0000 7f2803e00640 1 -- 172.21.15.83:0/2003710994 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2810037070 con 0x5557372fa050 2024-08-19T01:35:56.888 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:56.885+0000 7f2803e00640 1 -- 172.21.15.83:0/2003710994 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2810004c80 con 0x5557372fa050 2024-08-19T01:35:56.888 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:56.885+0000 7f2803e00640 1 -- 172.21.15.83:0/2003710994 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2810003970 con 0x5557372fa050 2024-08-19T01:35:56.889 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:56.886+0000 7f2821677740 1 -- 172.21.15.83:0/2003710994 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5557372fa050 msgr2=0x5557372f0610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:35:56.889 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:56.886+0000 7f2821677740 1 --2- 172.21.15.83:0/2003710994 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5557372fa050 0x5557372f0610 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f2810004300 tx=0x7f281000f7d0 comp rx=0 tx=0).stop 2024-08-19T01:35:56.889 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:56.886+0000 7f2821677740 1 -- 172.21.15.83:0/2003710994 shutdown_connections 2024-08-19T01:35:56.889 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:56.886+0000 7f2821677740 1 --2- 172.21.15.83:0/2003710994 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5557372fa050 0x5557372f0610 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:35:56.889 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:56.886+0000 7f2821677740 1 --2- 172.21.15.83:0/2003710994 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5557372f4fc0 0x555737305e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:35:56.889 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:56.886+0000 7f2821677740 1 --2- 172.21.15.83:0/2003710994 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5557372f4680 0x5557372f00d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:35:56.889 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:56.886+0000 7f2821677740 1 -- 172.21.15.83:0/2003710994 >> 172.21.15.83:0/2003710994 conn(0x5557372efcf0 msgr2=0x5557372fe690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:35:56.889 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:56.886+0000 7f2821677740 1 -- 172.21.15.83:0/2003710994 shutdown_connections 2024-08-19T01:35:56.889 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:56.886+0000 7f2821677740 1 -- 172.21.15.83:0/2003710994 wait complete. 2024-08-19T01:35:56.889 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:56.886+0000 7f2821677740 1 -- start start 2024-08-19T01:35:56.892 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:56.886+0000 7f2821677740 1 Processor -- start 2024-08-19T01:35:56.893 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:56.887+0000 7f2821677740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5557372f4680 0x5557373029f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:35:56.893 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:56.887+0000 7f2821677740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5557372f4fc0 0x555737302f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:35:56.893 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:56.887+0000 7f2821677740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5557372fa050 0x555737373000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:35:56.893 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:56.887+0000 7f2821677740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f28181a9590 con 0x5557372f4680 2024-08-19T01:35:56.893 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:56.887+0000 7f2821677740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f28181aaa00 con 0x5557372f4fc0 2024-08-19T01:35:56.893 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:56.887+0000 7f2821677740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f28181a9c60 con 0x5557372fa050 2024-08-19T01:35:56.893 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:56.887+0000 7f2821677740 10 client.0 _ll_register_callbacks cb 0x7f2810009460 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-19T01:35:56.893 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:56.887+0000 7f2817e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5557372f4680 0x5557373029f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:35:56.893 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:56.887+0000 7f2817e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5557372f4680 0x5557373029f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:45468/0 (socket says 172.21.15.83:45468) 2024-08-19T01:35:56.893 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:56.887+0000 7f2817e00640 1 -- 172.21.15.83:0/3641895295 learned_addr learned my addr 172.21.15.83:0/3641895295 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-08-19T01:35:56.893 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:56.887+0000 7f2816a00640 1 --2- 172.21.15.83:0/3641895295 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5557372f4fc0 0x555737302f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:35:56.893 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:56.887+0000 7f2817e00640 1 --2- 172.21.15.83:0/3641895295 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5557372fa050 0x555737373000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:35:56.894 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:56.887+0000 7f2817e00640 1 -- 172.21.15.83:0/3641895295 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5557372fa050 msgr2=0x555737373000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:35:56.894 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:56.887+0000 7f2817e00640 1 --2- 172.21.15.83:0/3641895295 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5557372fa050 0x555737373000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:35:56.894 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:56.887+0000 7f2817e00640 1 -- 172.21.15.83:0/3641895295 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5557372f4fc0 msgr2=0x555737302f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:35:56.894 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:56.887+0000 7f2817e00640 1 --2- 172.21.15.83:0/3641895295 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5557372f4fc0 0x555737302f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:35:56.894 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:56.887+0000 7f2817e00640 1 -- 172.21.15.83:0/3641895295 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555737306020 con 0x5557372f4680 2024-08-19T01:35:56.894 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:56.888+0000 7f2816a00640 1 --2- 172.21.15.83:0/3641895295 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5557372f4fc0 0x555737302f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-19T01:35:56.894 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:56.888+0000 7f2817e00640 1 --2- 172.21.15.83:0/3641895295 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5557372f4680 0x5557373029f0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f2810000f80 tx=0x7f2810004870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3fb7980bdde0dc20 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:35:56.894 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:56.888+0000 7f2801600640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-08-19T01:35:56.894 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:56.888+0000 7f2801600640 1 -- 172.21.15.83:0/3641895295 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2810037040 con 0x5557372f4680 2024-08-19T01:35:56.894 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:56.888+0000 7f2821677740 1 -- 172.21.15.83:0/3641895295 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2810004c80 con 0x5557372f4680 2024-08-19T01:35:56.894 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:56.888+0000 7f2801600640 1 -- 172.21.15.83:0/3641895295 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8269+0+0 (secure 0 0 0) 0x7f2810027480 con 0x5557372f4680 2024-08-19T01:35:56.894 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:56.888+0000 7f2801600640 1 -- 172.21.15.83:0/3641895295 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f2810027680 con 0x5557372f4680 2024-08-19T01:35:56.895 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:56.888+0000 7f2821677740 10 client.15276 fetch_fsmap learned FSMap version 25 2024-08-19T01:35:56.895 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:56.888+0000 7f2821677740 1 -- 172.21.15.83:0/3641895295 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x555737374400 con 0x5557372f4680 2024-08-19T01:35:56.895 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:56.888+0000 7f2801600640 1 -- 172.21.15.83:0/3641895295 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2810004c80 con 0x5557372f4680 2024-08-19T01:35:56.895 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:56.889+0000 7f2801600640 1 -- 172.21.15.83:0/3641895295 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f281000b070 con 0x5557372f4680 2024-08-19T01:35:56.895 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:56.889+0000 7f2821677740 10 client.15276 fetch_fsmap finished waiting for FSMap version 25 2024-08-19T01:35:56.895 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:56.889+0000 7f2821677740 10 client.15276 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-08-19T01:35:56.895 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:56.889+0000 7f2821677740 20 client.15276 populate_metadata read hostname 'smithi083' 2024-08-19T01:35:56.895 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:56.889+0000 7f2821677740 1 --2- 172.21.15.83:0/3641895295 >> [v2:172.21.15.117:6838/1967595704,v1:172.21.15.117:6839/1967595704] conn(0x555737374cf0 0x555737377110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:35:56.895 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:56.889+0000 7f2821677740 4 client.15276 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-08-19T01:35:56.895 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:56.889+0000 7f2821677740 1 -- 172.21.15.83:0/3641895295 --> [v2:172.21.15.117:6838/1967595704,v1:172.21.15.117:6839/1967595704] -- command(tid 0: {"prefix": "flush journal"}) -- 0x555737260310 con 0x555737374cf0 2024-08-19T01:35:56.895 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:56.889+0000 7f2816a00640 1 --2- 172.21.15.83:0/3641895295 >> [v2:172.21.15.117:6838/1967595704,v1:172.21.15.117:6839/1967595704] conn(0x555737374cf0 0x555737377110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:35:56.895 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:56.890+0000 7f2801600640 10 client.15276 ms_handle_connect on v2:172.21.15.117:6838/1967595704 2024-08-19T01:35:56.896 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:56.890+0000 7f2816a00640 1 --2- 172.21.15.83:0/3641895295 >> [v2:172.21.15.117:6838/1967595704,v1:172.21.15.117:6839/1967595704] conn(0x555737374cf0 0x555737377110 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5f3f7a8c0410ba6a server_cookie=891d2b740fad43b8 in_seq=0 out_seq=0 2024-08-19T01:35:56.897 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:56.893+0000 7f2801600640 1 -- 172.21.15.83:0/3641895295 <== mds.0 v2:172.21.15.117:6838/1967595704 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x555737260310 con 0x555737374cf0 2024-08-19T01:35:56.897 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:56.893+0000 7f2801600640 10 client.15276 handle_command_reply: tid=0 2024-08-19T01:35:56.897 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:56.893+0000 7f2821677740 1 client.15276 shutdown 2024-08-19T01:35:56.897 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:56.894+0000 7f2821677740 1 -- 172.21.15.83:0/3641895295 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5557372f4680 msgr2=0x5557373029f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:35:56.897 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:56.894+0000 7f2821677740 1 --2- 172.21.15.83:0/3641895295 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5557372f4680 0x5557373029f0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f2810000f80 tx=0x7f2810004870 comp rx=0 tx=0).stop 2024-08-19T01:35:56.897 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:56.894+0000 7f2821677740 1 -- 172.21.15.83:0/3641895295 shutdown_connections 2024-08-19T01:35:56.897 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:56.894+0000 7f2821677740 1 --2- 172.21.15.83:0/3641895295 >> [v2:172.21.15.117:6838/1967595704,v1:172.21.15.117:6839/1967595704] conn(0x555737374cf0 0x555737377110 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:35:56.897 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:56.894+0000 7f2821677740 1 --2- 172.21.15.83:0/3641895295 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5557372fa050 0x555737373000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:35:56.897 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:56.894+0000 7f2821677740 1 --2- 172.21.15.83:0/3641895295 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5557372f4fc0 0x555737302f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:35:56.897 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:56.894+0000 7f2821677740 1 --2- 172.21.15.83:0/3641895295 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5557372f4680 0x5557373029f0 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:35:56.897 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:56.894+0000 7f2801600640 0 client.15276 ms_handle_reset on v2:172.21.15.117:6838/1967595704 2024-08-19T01:35:56.897 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:56.894+0000 7f2821677740 1 -- 172.21.15.83:0/3641895295 >> 172.21.15.83:0/3641895295 conn(0x5557372efcf0 msgr2=0x5557372ee720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:35:56.898 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:56.894+0000 7f2821677740 1 -- 172.21.15.83:0/3641895295 shutdown_connections 2024-08-19T01:35:56.898 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:56.894+0000 7f2821677740 1 -- 172.21.15.83:0/3641895295 wait complete. 2024-08-19T01:35:56.898 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:56.894+0000 7f2821677740 20 client.15276 trim_cache size 0 max 16384 2024-08-19T01:35:56.898 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stdout:{ 2024-08-19T01:35:56.898 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stdout: "message": "", 2024-08-19T01:35:56.898 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stdout: "return_code": 0 2024-08-19T01:35:56.898 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stdout:} 2024-08-19T01:35:56.899 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:56.895+0000 7f281d200640 1 -- 172.21.15.83:0/2321086306 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f27fc0a2c90 msgr2=0x7f27fc0a5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:35:56.899 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:56.895+0000 7f281d200640 1 --2- 172.21.15.83:0/2321086306 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f27fc0a2c90 0x7f27fc0a5150 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f28080980f0 tx=0x7f28080d5530 comp rx=0 tx=0).stop 2024-08-19T01:35:56.899 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:56.895+0000 7f281d200640 1 -- 172.21.15.83:0/2321086306 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f28181012b0 msgr2=0x7f2818075cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:35:56.899 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:56.895+0000 7f281d200640 1 --2- 172.21.15.83:0/2321086306 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f28181012b0 0x7f2818075cd0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f280800e9c0 tx=0x7f280800ee80 comp rx=0 tx=0).stop 2024-08-19T01:35:56.899 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:56.895+0000 7f281d200640 1 -- 172.21.15.83:0/2321086306 shutdown_connections 2024-08-19T01:35:56.899 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:56.895+0000 7f281d200640 1 --2- 172.21.15.83:0/2321086306 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f27fc0a2c90 0x7f27fc0a5150 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:35:56.899 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:56.895+0000 7f281d200640 1 --2- 172.21.15.83:0/2321086306 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2818102d80 0x7f2818074850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:35:56.899 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:56.895+0000 7f281d200640 1 --2- 172.21.15.83:0/2321086306 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2818101c70 0x7f2818074310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:35:56.900 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:56.895+0000 7f281d200640 1 --2- 172.21.15.83:0/2321086306 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f28181012b0 0x7f2818075cd0 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:35:56.900 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:56.895+0000 7f281d200640 1 -- 172.21.15.83:0/2321086306 >> 172.21.15.83:0/2321086306 conn(0x7f28180fce00 msgr2=0x7f2818105d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:35:56.900 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:56.895+0000 7f281d200640 1 -- 172.21.15.83:0/2321086306 shutdown_connections 2024-08-19T01:35:56.900 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:35:56.895+0000 7f281d200640 1 -- 172.21.15.83:0/2321086306 wait complete. 2024-08-19T01:35:56.904 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:35:56 smithi083 ceph-mon[17120]: pgmap v372: 97 pgs: 97 active+clean; 669 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 255 B/s wr, 7 op/s 2024-08-19T01:35:56.952 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:35:56 smithi117 ceph-mon[22016]: pgmap v372: 97 pgs: 97 active+clean; 669 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 255 B/s wr, 7 op/s 2024-08-19T01:35:56.966 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:35:56 smithi167 ceph-mon[21875]: pgmap v372: 97 pgs: 97 active+clean; 669 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 255 B/s wr, 7 op/s 2024-08-19T01:35:57.341 INFO:tasks.workunit.client.0.smithi083.stderr:Note: switching to '78215f5a1cf5eaf321d506a66539942e12851f34'. 2024-08-19T01:35:57.341 INFO:tasks.workunit.client.0.smithi083.stderr: 2024-08-19T01:35:57.341 INFO:tasks.workunit.client.0.smithi083.stderr:You are in 'detached HEAD' state. You can look around, make experimental 2024-08-19T01:35:57.341 INFO:tasks.workunit.client.0.smithi083.stderr:changes and commit them, and you can discard any commits you make in this 2024-08-19T01:35:57.341 INFO:tasks.workunit.client.0.smithi083.stderr:state without impacting any branches by switching back to a branch. 2024-08-19T01:35:57.341 INFO:tasks.workunit.client.0.smithi083.stderr: 2024-08-19T01:35:57.341 INFO:tasks.workunit.client.0.smithi083.stderr:If you want to create a new branch to retain commits you create, you may 2024-08-19T01:35:57.341 INFO:tasks.workunit.client.0.smithi083.stderr:do so (now or later) by using -c with the switch command. Example: 2024-08-19T01:35:57.342 INFO:tasks.workunit.client.0.smithi083.stderr: 2024-08-19T01:35:57.342 INFO:tasks.workunit.client.0.smithi083.stderr: git switch -c 2024-08-19T01:35:57.342 INFO:tasks.workunit.client.0.smithi083.stderr: 2024-08-19T01:35:57.342 INFO:tasks.workunit.client.0.smithi083.stderr:Or undo this operation with: 2024-08-19T01:35:57.342 INFO:tasks.workunit.client.0.smithi083.stderr: 2024-08-19T01:35:57.342 INFO:tasks.workunit.client.0.smithi083.stderr: git switch - 2024-08-19T01:35:57.342 INFO:tasks.workunit.client.0.smithi083.stderr: 2024-08-19T01:35:57.342 INFO:tasks.workunit.client.0.smithi083.stderr:Turn off this advice by setting config variable advice.detachedHead to false 2024-08-19T01:35:57.342 INFO:tasks.workunit.client.0.smithi083.stderr: 2024-08-19T01:35:57.342 INFO:tasks.workunit.client.0.smithi083.stderr:HEAD is now at 78215f5a1cf Merge pull request #59291 from zdover23/wip-doc-2024-08-19-backport-59290-to-squid 2024-08-19T01:35:57.359 DEBUG:teuthology.orchestra.run.smithi083:> cd -- /home/ubuntu/cephtest/clone.client.0/qa/workunits && if test -e Makefile ; then make ; fi && find -executable -type f -printf '%P\0' >/home/ubuntu/cephtest/workunits.list.client.0 2024-08-19T01:35:57.458 INFO:tasks.workunit.client.0.smithi083.stdout:for d in direct_io fs ; do ( cd $d ; make all ) ; done 2024-08-19T01:35:57.462 INFO:tasks.workunit.client.0.smithi083.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-08-19T01:35:57.463 INFO:tasks.workunit.client.0.smithi083.stdout:cc -Wall -Wextra -D_GNU_SOURCE direct_io_test.c -o direct_io_test 2024-08-19T01:35:57.904 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:35:57 smithi083 ceph-mon[17120]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-19T01:35:57.905 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:35:57 smithi083 ceph-mon[17120]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' 2024-08-19T01:35:57.905 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:35:57 smithi083 ceph-mon[17120]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' 2024-08-19T01:35:57.952 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:35:57 smithi117 ceph-mon[22016]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-19T01:35:57.952 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:35:57 smithi117 ceph-mon[22016]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' 2024-08-19T01:35:57.952 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:35:57 smithi117 ceph-mon[22016]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' 2024-08-19T01:35:57.966 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:35:57 smithi167 ceph-mon[21875]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-19T01:35:57.966 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:35:57 smithi167 ceph-mon[21875]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' 2024-08-19T01:35:57.966 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:35:57 smithi167 ceph-mon[21875]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' 2024-08-19T01:35:58.297 INFO:tasks.workunit.client.0.smithi083.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_sync_io.c -o test_sync_io 2024-08-19T01:35:58.393 INFO:tasks.workunit.client.0.smithi083.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_short_dio_read.c -o test_short_dio_read 2024-08-19T01:35:58.458 INFO:tasks.workunit.client.0.smithi083.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-08-19T01:35:58.462 INFO:tasks.workunit.client.0.smithi083.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-08-19T01:35:58.462 INFO:tasks.workunit.client.0.smithi083.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_o_trunc.c -o test_o_trunc 2024-08-19T01:35:58.534 INFO:tasks.workunit.client.0.smithi083.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-08-19T01:35:58.541 DEBUG:teuthology.orchestra.run.smithi083:> set -ex 2024-08-19T01:35:58.541 DEBUG:teuthology.orchestra.run.smithi083:> dd if=/home/ubuntu/cephtest/workunits.list.client.0 of=/dev/stdout 2024-08-19T01:35:58.577 INFO:tasks.workunit:Running workunits matching suites/blogbench.sh on client.0... 2024-08-19T01:35:58.580 INFO:tasks.workunit:Running workunit suites/blogbench.sh... 2024-08-19T01:35:58.580 DEBUG:teuthology.orchestra.run.smithi083:workunit test suites/blogbench.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=78215f5a1cf5eaf321d506a66539942e12851f34 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/blogbench.sh 2024-08-19T01:35:58.667 INFO:tasks.workunit.client.0.smithi083.stderr:+ echo 'getting blogbench' 2024-08-19T01:35:58.667 INFO:tasks.workunit.client.0.smithi083.stderr:+ wget http://download.ceph.com/qa/blogbench-1.0.tar.bz2 2024-08-19T01:35:58.668 INFO:tasks.workunit.client.0.smithi083.stdout:getting blogbench 2024-08-19T01:35:58.700 INFO:tasks.workunit.client.0.smithi083.stderr:--2024-08-19 01:35:58-- http://download.ceph.com/qa/blogbench-1.0.tar.bz2 2024-08-19T01:35:58.701 INFO:tasks.workunit.client.0.smithi083.stderr:Resolving download.ceph.com (download.ceph.com)... 158.69.68.124, 2607:5300:201:2000::3:58a1 2024-08-19T01:35:58.722 INFO:tasks.workunit.client.0.smithi083.stderr:Connecting to download.ceph.com (download.ceph.com)|158.69.68.124|:80... connected. 2024-08-19T01:35:58.743 INFO:tasks.workunit.client.0.smithi083.stderr:HTTP request sent, awaiting response... 200 OK 2024-08-19T01:35:58.744 INFO:tasks.workunit.client.0.smithi083.stderr:Length: 103355 (101K) [application/octet-stream] 2024-08-19T01:35:58.745 INFO:tasks.workunit.client.0.smithi083.stderr:Saving to: ‘blogbench-1.0.tar.bz2’ 2024-08-19T01:35:58.750 INFO:tasks.workunit.client.0.smithi083.stderr: 2024-08-19T01:35:58.785 INFO:tasks.workunit.client.0.smithi083.stderr: 0K .......... .......... .......... .......... .......... 49% 1.25M 0s 2024-08-19T01:35:58.806 INFO:tasks.workunit.client.0.smithi083.stderr: 50K .......... .......... .......... .......... .......... 99% 2.39M 0s 2024-08-19T01:35:58.806 INFO:tasks.workunit.client.0.smithi083.stderr: 100K 100% 1.74T=0.06s 2024-08-19T01:35:58.806 INFO:tasks.workunit.client.0.smithi083.stderr: 2024-08-19T01:35:58.806 INFO:tasks.workunit.client.0.smithi083.stderr:2024-08-19 01:35:58 (1.66 MB/s) - ‘blogbench-1.0.tar.bz2’ saved [103355/103355] 2024-08-19T01:35:58.806 INFO:tasks.workunit.client.0.smithi083.stderr: 2024-08-19T01:35:58.806 INFO:tasks.workunit.client.0.smithi083.stderr:+ tar -xvf blogbench-1.0.tar.bz2 2024-08-19T01:35:58.842 INFO:tasks.workunit.client.0.smithi083.stdout:blogbench-1.0/ 2024-08-19T01:35:58.843 INFO:tasks.workunit.client.0.smithi083.stdout:blogbench-1.0/m4/ 2024-08-19T01:35:58.844 INFO:tasks.workunit.client.0.smithi083.stdout:blogbench-1.0/m4/acx_pthread.m4 2024-08-19T01:35:58.846 INFO:tasks.workunit.client.0.smithi083.stdout:blogbench-1.0/man/ 2024-08-19T01:35:58.847 INFO:tasks.workunit.client.0.smithi083.stdout:blogbench-1.0/man/blogbench.8 2024-08-19T01:35:58.849 INFO:tasks.workunit.client.0.smithi083.stdout:blogbench-1.0/man/Makefile.am 2024-08-19T01:35:58.850 INFO:tasks.workunit.client.0.smithi083.stdout:blogbench-1.0/man/Makefile.in 2024-08-19T01:35:58.852 INFO:tasks.workunit.client.0.smithi083.stdout:blogbench-1.0/src/ 2024-08-19T01:35:58.853 INFO:tasks.workunit.client.0.smithi083.stdout:blogbench-1.0/src/mysnprintf.c 2024-08-19T01:35:58.854 INFO:tasks.workunit.client.0.smithi083.stdout:blogbench-1.0/src/mysnprintf.h 2024-08-19T01:35:58.855 INFO:tasks.workunit.client.0.smithi083.stdout:blogbench-1.0/src/blogbench.c 2024-08-19T01:35:58.856 INFO:tasks.workunit.client.0.smithi083.stdout:blogbench-1.0/src/blogbench.h 2024-08-19T01:35:58.857 INFO:tasks.workunit.client.0.smithi083.stdout:blogbench-1.0/src/fakesnprintf.c 2024-08-19T01:35:58.862 INFO:tasks.workunit.client.0.smithi083.stdout:blogbench-1.0/src/fakesnprintf.h 2024-08-19T01:35:58.862 INFO:tasks.workunit.client.0.smithi083.stdout:blogbench-1.0/src/blogbench_p.h 2024-08-19T01:35:58.862 INFO:tasks.workunit.client.0.smithi083.stdout:blogbench-1.0/src/bsd-getopt_long.c 2024-08-19T01:35:58.863 INFO:tasks.workunit.client.0.smithi083.stdout:blogbench-1.0/src/bsd-getopt_long.h 2024-08-19T01:35:58.864 INFO:tasks.workunit.client.0.smithi083.stdout:blogbench-1.0/src/Makefile.am 2024-08-19T01:35:58.865 INFO:tasks.workunit.client.0.smithi083.stdout:blogbench-1.0/src/Makefile.in 2024-08-19T01:35:58.866 INFO:tasks.workunit.client.0.smithi083.stdout:blogbench-1.0/src/process.c 2024-08-19T01:35:58.868 INFO:tasks.workunit.client.0.smithi083.stdout:blogbench-1.0/src/globals.h 2024-08-19T01:35:58.869 INFO:tasks.workunit.client.0.smithi083.stdout:blogbench-1.0/src/commenter.c 2024-08-19T01:35:58.870 INFO:tasks.workunit.client.0.smithi083.stdout:blogbench-1.0/src/rewriter.c 2024-08-19T01:35:58.871 INFO:tasks.workunit.client.0.smithi083.stdout:blogbench-1.0/src/reader.c 2024-08-19T01:35:58.872 INFO:tasks.workunit.client.0.smithi083.stdout:blogbench-1.0/src/writer.c 2024-08-19T01:35:58.873 INFO:tasks.workunit.client.0.smithi083.stdout:blogbench-1.0/src/helpers.c 2024-08-19T01:35:58.874 INFO:tasks.workunit.client.0.smithi083.stdout:blogbench-1.0/NEWS 2024-08-19T01:35:58.876 INFO:tasks.workunit.client.0.smithi083.stdout:blogbench-1.0/compile 2024-08-19T01:35:58.877 INFO:tasks.workunit.client.0.smithi083.stdout:blogbench-1.0/depcomp 2024-08-19T01:35:58.878 INFO:tasks.workunit.client.0.smithi083.stdout:blogbench-1.0/aclocal.m4 2024-08-19T01:35:58.879 INFO:tasks.workunit.client.0.smithi083.stdout:blogbench-1.0/README 2024-08-19T01:35:58.881 INFO:tasks.workunit.client.0.smithi083.stdout:blogbench-1.0/configure 2024-08-19T01:35:58.885 INFO:tasks.workunit.client.0.smithi083.stdout:blogbench-1.0/configure.ac 2024-08-19T01:35:58.886 INFO:tasks.workunit.client.0.smithi083.stdout:blogbench-1.0/config.guess 2024-08-19T01:35:58.888 INFO:tasks.workunit.client.0.smithi083.stdout:blogbench-1.0/install-sh 2024-08-19T01:35:58.889 INFO:tasks.workunit.client.0.smithi083.stdout:blogbench-1.0/config.sub 2024-08-19T01:35:58.890 INFO:tasks.workunit.client.0.smithi083.stdout:blogbench-1.0/missing 2024-08-19T01:35:58.891 INFO:tasks.workunit.client.0.smithi083.stdout:blogbench-1.0/mkinstalldirs 2024-08-19T01:35:58.892 INFO:tasks.workunit.client.0.smithi083.stdout:blogbench-1.0/Makefile.am 2024-08-19T01:35:58.893 INFO:tasks.workunit.client.0.smithi083.stdout:blogbench-1.0/Makefile.in 2024-08-19T01:35:58.895 INFO:tasks.workunit.client.0.smithi083.stdout:blogbench-1.0/config.h.in 2024-08-19T01:35:58.896 INFO:tasks.workunit.client.0.smithi083.stdout:blogbench-1.0/AUTHORS 2024-08-19T01:35:58.897 INFO:tasks.workunit.client.0.smithi083.stdout:blogbench-1.0/INSTALL 2024-08-19T01:35:58.898 INFO:tasks.workunit.client.0.smithi083.stdout:blogbench-1.0/ChangeLog 2024-08-19T01:35:58.899 INFO:tasks.workunit.client.0.smithi083.stdout:blogbench-1.0/COPYING 2024-08-19T01:35:58.901 INFO:tasks.workunit.client.0.smithi083.stderr:+ cd blogbench-1.0/ 2024-08-19T01:35:58.901 INFO:tasks.workunit.client.0.smithi083.stdout:making blogbench 2024-08-19T01:35:58.901 INFO:tasks.workunit.client.0.smithi083.stderr:+ echo 'making blogbench' 2024-08-19T01:35:58.901 INFO:tasks.workunit.client.0.smithi083.stderr:+ ./configure 2024-08-19T01:35:58.904 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:35:58 smithi083 ceph-mon[17120]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-19T01:35:58.904 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:35:58 smithi083 ceph-mon[17120]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-19T01:35:58.904 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:35:58 smithi083 ceph-mon[17120]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' 2024-08-19T01:35:58.904 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:35:58 smithi083 ceph-mon[17120]: pgmap v373: 97 pgs: 97 active+clean; 669 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 0 B/s wr, 5 op/s 2024-08-19T01:35:58.952 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:35:58 smithi117 ceph-mon[22016]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-19T01:35:58.953 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:35:58 smithi117 ceph-mon[22016]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-19T01:35:58.953 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:35:58 smithi117 ceph-mon[22016]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' 2024-08-19T01:35:58.953 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:35:58 smithi117 ceph-mon[22016]: pgmap v373: 97 pgs: 97 active+clean; 669 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 0 B/s wr, 5 op/s 2024-08-19T01:35:58.966 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:35:58 smithi167 ceph-mon[21875]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-19T01:35:58.966 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:35:58 smithi167 ceph-mon[21875]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-19T01:35:58.966 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:35:58 smithi167 ceph-mon[21875]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' 2024-08-19T01:35:58.966 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:35:58 smithi167 ceph-mon[21875]: pgmap v373: 97 pgs: 97 active+clean; 669 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 0 B/s wr, 5 op/s 2024-08-19T01:35:59.119 INFO:tasks.workunit.client.0.smithi083.stdout:checking for a BSD-compatible install... /usr/bin/install -c 2024-08-19T01:36:00.132 INFO:tasks.workunit.client.0.smithi083.stdout:checking whether build environment is sane... yes 2024-08-19T01:36:00.148 INFO:tasks.workunit.client.0.smithi083.stdout:checking for gawk... gawk 2024-08-19T01:36:00.174 INFO:tasks.workunit.client.0.smithi083.stdout:checking whether make sets $(MAKE)... yes 2024-08-19T01:36:00.213 INFO:tasks.workunit.client.0.smithi083.stdout:checking for gcc... gcc 2024-08-19T01:36:00.301 INFO:tasks.workunit.client.0.smithi083.stdout:checking for C compiler default output file name... a.out 2024-08-19T01:36:00.305 INFO:tasks.workunit.client.0.smithi083.stdout:checking whether the C compiler works... yes 2024-08-19T01:36:00.309 INFO:tasks.workunit.client.0.smithi083.stdout:checking whether we are cross compiling... no 2024-08-19T01:36:00.369 INFO:tasks.workunit.client.0.smithi083.stdout:checking for suffix of executables... 2024-08-19T01:36:00.431 INFO:tasks.workunit.client.0.smithi083.stdout:checking for suffix of object files... o 2024-08-19T01:36:00.490 INFO:tasks.workunit.client.0.smithi083.stdout:checking whether we are using the GNU C compiler... yes 2024-08-19T01:36:00.552 INFO:tasks.workunit.client.0.smithi083.stdout:checking whether gcc accepts -g... yes 2024-08-19T01:36:00.656 INFO:tasks.workunit.client.0.smithi083.stdout:checking for gcc option to accept ANSI C... none needed 2024-08-19T01:36:00.725 INFO:tasks.workunit.client.0.smithi083.stdout:checking for style of include used by make... GNU 2024-08-19T01:36:00.857 INFO:tasks.workunit.client.0.smithi083.stdout:checking dependency style of gcc... gcc3 2024-08-19T01:36:00.857 INFO:tasks.workunit.client.0.smithi083.stdout:checking for a BSD-compatible install... /usr/bin/install -c 2024-08-19T01:36:00.956 INFO:tasks.workunit.client.0.smithi083.stdout:checking how to run the C preprocessor... gcc -E 2024-08-19T01:36:01.032 INFO:tasks.workunit.client.0.smithi083.stdout:checking for egrep... grep -E 2024-08-19T01:36:01.052 INFO:tasks.workunit.client.0.smithi083.stdout:checking for AIX... no 2024-08-19T01:36:01.151 INFO:tasks.workunit.client.0.smithi083.stdout:checking for library containing strerror... none required 2024-08-19T01:36:01.381 INFO:tasks.workunit.client.0.smithi083.stdout:checking for ANSI C header files... yes 2024-08-19T01:36:01.449 INFO:tasks.workunit.client.0.smithi083.stdout:checking whether time.h and sys/time.h may both be included... yes 2024-08-19T01:36:01.530 INFO:tasks.workunit.client.0.smithi083.stdout:checking for sys/types.h... yes 2024-08-19T01:36:01.616 INFO:tasks.workunit.client.0.smithi083.stdout:checking for sys/stat.h... yes 2024-08-19T01:36:01.701 INFO:tasks.workunit.client.0.smithi083.stdout:checking for stdlib.h... yes 2024-08-19T01:36:01.716 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:36:01 smithi167 ceph-mon[21875]: pgmap v374: 97 pgs: 97 active+clean; 796 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 11 KiB/s wr, 6 op/s 2024-08-19T01:36:01.788 INFO:tasks.workunit.client.0.smithi083.stdout:checking for string.h... yes 2024-08-19T01:36:01.875 INFO:tasks.workunit.client.0.smithi083.stdout:checking for memory.h... yes 2024-08-19T01:36:01.904 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:36:01 smithi083 ceph-mon[17120]: pgmap v374: 97 pgs: 97 active+clean; 796 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 11 KiB/s wr, 6 op/s 2024-08-19T01:36:01.952 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:36:01 smithi117 ceph-mon[22016]: pgmap v374: 97 pgs: 97 active+clean; 796 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 11 KiB/s wr, 6 op/s 2024-08-19T01:36:01.962 INFO:tasks.workunit.client.0.smithi083.stdout:checking for strings.h... yes 2024-08-19T01:36:02.051 INFO:tasks.workunit.client.0.smithi083.stdout:checking for inttypes.h... yes 2024-08-19T01:36:02.139 INFO:tasks.workunit.client.0.smithi083.stdout:checking for stdint.h... yes 2024-08-19T01:36:02.229 INFO:tasks.workunit.client.0.smithi083.stdout:checking for unistd.h... yes 2024-08-19T01:36:02.249 INFO:tasks.workunit.client.0.smithi083.stdout:checking for memory.h... (cached) yes 2024-08-19T01:36:02.263 INFO:tasks.workunit.client.0.smithi083.stdout:checking for strings.h... (cached) yes 2024-08-19T01:36:02.277 INFO:tasks.workunit.client.0.smithi083.stdout:checking for unistd.h... (cached) yes 2024-08-19T01:36:02.380 INFO:tasks.workunit.client.0.smithi083.stdout:checking locale.h usability... yes 2024-08-19T01:36:02.417 INFO:tasks.workunit.client.0.smithi083.stdout:checking locale.h presence... yes 2024-08-19T01:36:02.418 INFO:tasks.workunit.client.0.smithi083.stdout:checking for locale.h... yes 2024-08-19T01:36:02.442 INFO:tasks.workunit.client.0.smithi083.stdout:checking for stdlib.h... (cached) yes 2024-08-19T01:36:02.456 INFO:tasks.workunit.client.0.smithi083.stdout:checking for string.h... (cached) yes 2024-08-19T01:36:02.547 INFO:tasks.workunit.client.0.smithi083.stdout:checking sys/socket.h usability... yes 2024-08-19T01:36:02.587 INFO:tasks.workunit.client.0.smithi083.stdout:checking sys/socket.h presence... yes 2024-08-19T01:36:02.589 INFO:tasks.workunit.client.0.smithi083.stdout:checking for sys/socket.h... yes 2024-08-19T01:36:02.680 INFO:tasks.workunit.client.0.smithi083.stdout:checking sys/time.h usability... yes 2024-08-19T01:36:02.716 INFO:tasks.workunit.client.0.smithi083.stdout:checking sys/time.h presence... yes 2024-08-19T01:36:02.718 INFO:tasks.workunit.client.0.smithi083.stdout:checking for sys/time.h... yes 2024-08-19T01:36:02.810 INFO:tasks.workunit.client.0.smithi083.stdout:checking fcntl.h usability... yes 2024-08-19T01:36:02.846 INFO:tasks.workunit.client.0.smithi083.stdout:checking fcntl.h presence... yes 2024-08-19T01:36:02.848 INFO:tasks.workunit.client.0.smithi083.stdout:checking for fcntl.h... yes 2024-08-19T01:36:02.951 INFO:tasks.workunit.client.0.smithi083.stdout:checking sys/fcntl.h usability... yes 2024-08-19T01:36:02.988 INFO:tasks.workunit.client.0.smithi083.stdout:checking sys/fcntl.h presence... yes 2024-08-19T01:36:02.990 INFO:tasks.workunit.client.0.smithi083.stdout:checking for sys/fcntl.h... yes 2024-08-19T01:36:03.150 INFO:tasks.workunit.client.0.smithi083.stdout:checking sys/param.h usability... yes 2024-08-19T01:36:03.192 INFO:tasks.workunit.client.0.smithi083.stdout:checking sys/param.h presence... yes 2024-08-19T01:36:03.193 INFO:tasks.workunit.client.0.smithi083.stdout:checking for sys/param.h... yes 2024-08-19T01:36:03.271 INFO:tasks.workunit.client.0.smithi083.stdout:checking ioctl.h usability... no 2024-08-19T01:36:03.307 INFO:tasks.workunit.client.0.smithi083.stdout:checking ioctl.h presence... no 2024-08-19T01:36:03.309 INFO:tasks.workunit.client.0.smithi083.stdout:checking for ioctl.h... no 2024-08-19T01:36:03.403 INFO:tasks.workunit.client.0.smithi083.stdout:checking sys/ioctl.h usability... yes 2024-08-19T01:36:03.439 INFO:tasks.workunit.client.0.smithi083.stdout:checking sys/ioctl.h presence... yes 2024-08-19T01:36:03.441 INFO:tasks.workunit.client.0.smithi083.stdout:checking for sys/ioctl.h... yes 2024-08-19T01:36:03.529 INFO:tasks.workunit.client.0.smithi083.stdout:checking return type of signal handlers... void 2024-08-19T01:36:03.607 INFO:tasks.workunit.client.0.smithi083.stdout:checking for size_t... yes 2024-08-19T01:36:03.675 INFO:tasks.workunit.client.0.smithi083.stdout:checking for dev_t... yes 2024-08-19T01:36:03.752 INFO:tasks.workunit.client.0.smithi083.stdout:checking for short... yes 2024-08-19T01:36:03.877 INFO:tasks.workunit.client.0.smithi083.stdout:checking size of short... 2 2024-08-19T01:36:03.904 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:36:03 smithi083 ceph-mon[17120]: pgmap v375: 97 pgs: 97 active+clean; 796 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 12 KiB/s wr, 6 op/s 2024-08-19T01:36:03.952 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:36:03 smithi117 ceph-mon[22016]: pgmap v375: 97 pgs: 97 active+clean; 796 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 12 KiB/s wr, 6 op/s 2024-08-19T01:36:03.956 INFO:tasks.workunit.client.0.smithi083.stdout:checking for int... yes 2024-08-19T01:36:03.966 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:36:03 smithi167 ceph-mon[21875]: pgmap v375: 97 pgs: 97 active+clean; 796 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 12 KiB/s wr, 6 op/s 2024-08-19T01:36:04.089 INFO:tasks.workunit.client.0.smithi083.stdout:checking size of int... 4 2024-08-19T01:36:04.171 INFO:tasks.workunit.client.0.smithi083.stdout:checking for long... yes 2024-08-19T01:36:04.278 INFO:tasks.workunit.client.0.smithi083.stdout:checking size of long... 8 2024-08-19T01:36:04.371 INFO:tasks.workunit.client.0.smithi083.stdout:checking for long long... yes 2024-08-19T01:36:04.474 INFO:tasks.workunit.client.0.smithi083.stdout:checking size of long long... 8 2024-08-19T01:36:04.550 INFO:tasks.workunit.client.0.smithi083.stdout:checking for ssize_t... yes 2024-08-19T01:36:04.620 INFO:tasks.workunit.client.0.smithi083.stdout:checking for sig_atomic_t... yes 2024-08-19T01:36:04.691 INFO:tasks.workunit.client.0.smithi083.stdout:checking whether gcc needs -traditional... no 2024-08-19T01:36:04.754 INFO:tasks.workunit.client.0.smithi083.stdout:checking for an ANSI C-conforming const... yes 2024-08-19T01:36:04.806 INFO:tasks.workunit.client.0.smithi083.stdout:checking for inline... inline 2024-08-19T01:36:04.901 INFO:tasks.workunit.client.0.smithi083.stdout:checking for working alloca.h... yes 2024-08-19T01:36:04.989 INFO:tasks.workunit.client.0.smithi083.stdout:checking for alloca... yes 2024-08-19T01:36:05.100 INFO:tasks.workunit.client.0.smithi083.stdout:checking for working memcmp... yes 2024-08-19T01:36:05.196 INFO:tasks.workunit.client.0.smithi083.stdout:checking for strftime... yes 2024-08-19T01:36:05.316 INFO:tasks.workunit.client.0.smithi083.stdout:checking whether lstat dereferences a symlink specified with a trailing slash... yes 2024-08-19T01:36:05.413 INFO:tasks.workunit.client.0.smithi083.stdout:checking whether stat accepts an empty string... no 2024-08-19T01:36:05.514 INFO:tasks.workunit.client.0.smithi083.stdout:checking for vprintf... yes 2024-08-19T01:36:05.608 INFO:tasks.workunit.client.0.smithi083.stdout:checking for _doprnt... no 2024-08-19T01:36:05.659 INFO:tasks.workunit.client.0.smithi083.stdout:checking build system type... x86_64-unknown-linux-gnu 2024-08-19T01:36:05.689 INFO:tasks.workunit.client.0.smithi083.stdout:checking host system type... x86_64-unknown-linux-gnu 2024-08-19T01:36:05.778 INFO:tasks.workunit.client.0.smithi083.stdout:checking for the pthreads library -lpthreads... no 2024-08-19T01:36:05.869 INFO:tasks.workunit.client.0.smithi083.stdout:checking whether pthreads work without any flags... yes 2024-08-19T01:36:05.887 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:36:05 smithi083 ceph-mon[17120]: pgmap v376: 97 pgs: 97 active+clean; 863 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 18 KiB/s wr, 8 op/s 2024-08-19T01:36:05.952 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:36:05 smithi117 ceph-mon[22016]: pgmap v376: 97 pgs: 97 active+clean; 863 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 18 KiB/s wr, 8 op/s 2024-08-19T01:36:05.953 INFO:tasks.workunit.client.0.smithi083.stdout:checking for joinable pthread attribute... PTHREAD_CREATE_JOINABLE 2024-08-19T01:36:05.953 INFO:tasks.workunit.client.0.smithi083.stdout:checking if more special flags are required for pthreads... no 2024-08-19T01:36:05.953 INFO:tasks.workunit.client.0.smithi083.stdout:checking for cc_r... gcc 2024-08-19T01:36:05.966 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:36:05 smithi167 ceph-mon[21875]: pgmap v376: 97 pgs: 97 active+clean; 863 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 18 KiB/s wr, 8 op/s 2024-08-19T01:36:06.066 INFO:tasks.workunit.client.0.smithi083.stdout:checking for setrlimit... yes 2024-08-19T01:36:06.169 INFO:tasks.workunit.client.0.smithi083.stdout:checking for setproctitle... no 2024-08-19T01:36:06.268 INFO:tasks.workunit.client.0.smithi083.stdout:checking for getopt_long... yes 2024-08-19T01:36:06.374 INFO:tasks.workunit.client.0.smithi083.stdout:checking for strtoull... yes 2024-08-19T01:36:06.479 INFO:tasks.workunit.client.0.smithi083.stdout:checking for strtoq... yes 2024-08-19T01:36:06.586 INFO:tasks.workunit.client.0.smithi083.stdout:checking for snprintf... yes 2024-08-19T01:36:06.692 INFO:tasks.workunit.client.0.smithi083.stdout:checking for vsnprintf... yes 2024-08-19T01:36:06.800 INFO:tasks.workunit.client.0.smithi083.stdout:checking for vfprintf... yes 2024-08-19T01:36:06.904 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:36:06 smithi083 ceph-mon[17120]: mgrmap e20: x(active, since 10m), standbys: z, y 2024-08-19T01:36:06.906 INFO:tasks.workunit.client.0.smithi083.stdout:checking for gethostname... yes 2024-08-19T01:36:06.952 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:36:06 smithi117 ceph-mon[22016]: mgrmap e20: x(active, since 10m), standbys: z, y 2024-08-19T01:36:06.966 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:36:06 smithi167 ceph-mon[21875]: mgrmap e20: x(active, since 10m), standbys: z, y 2024-08-19T01:36:07.014 INFO:tasks.workunit.client.0.smithi083.stdout:checking for setlocale... yes 2024-08-19T01:36:07.121 INFO:tasks.workunit.client.0.smithi083.stdout:checking for sync... yes 2024-08-19T01:36:07.226 INFO:tasks.workunit.client.0.smithi083.stdout:checking whether snprintf is C99 conformant... done 2024-08-19T01:36:07.248 INFO:tasks.workunit.client.0.smithi083.stdout:configure: creating ./config.status 2024-08-19T01:36:07.534 INFO:tasks.workunit.client.0.smithi083.stdout:config.status: creating Makefile 2024-08-19T01:36:07.573 INFO:tasks.workunit.client.0.smithi083.stdout:config.status: creating src/Makefile 2024-08-19T01:36:07.613 INFO:tasks.workunit.client.0.smithi083.stdout:config.status: creating man/Makefile 2024-08-19T01:36:07.633 INFO:tasks.workunit.client.0.smithi083.stdout:config.status: creating config.h 2024-08-19T01:36:07.720 INFO:tasks.workunit.client.0.smithi083.stdout:config.status: executing depfiles commands 2024-08-19T01:36:07.904 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:36:07 smithi083 ceph-mon[17120]: pgmap v377: 97 pgs: 97 active+clean; 2.7 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 109 KiB/s rd, 182 KiB/s wr, 17 op/s 2024-08-19T01:36:07.952 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:36:07 smithi117 ceph-mon[22016]: pgmap v377: 97 pgs: 97 active+clean; 2.7 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 109 KiB/s rd, 182 KiB/s wr, 17 op/s 2024-08-19T01:36:07.966 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:36:07 smithi167 ceph-mon[21875]: pgmap v377: 97 pgs: 97 active+clean; 2.7 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 109 KiB/s rd, 182 KiB/s wr, 17 op/s 2024-08-19T01:36:07.996 INFO:tasks.workunit.client.0.smithi083.stderr:+ make 2024-08-19T01:36:08.008 INFO:tasks.workunit.client.0.smithi083.stdout:make all-recursive 2024-08-19T01:36:08.011 INFO:tasks.workunit.client.0.smithi083.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/mnt.0/client.0/tmp/blogbench-1.0' 2024-08-19T01:36:08.017 INFO:tasks.workunit.client.0.smithi083.stdout:Making all in src 2024-08-19T01:36:08.026 INFO:tasks.workunit.client.0.smithi083.stdout:make[2]: Entering directory '/home/ubuntu/cephtest/mnt.0/client.0/tmp/blogbench-1.0/src' 2024-08-19T01:36:08.026 INFO:tasks.workunit.client.0.smithi083.stdout:if gcc -DHAVE_CONFIG_H -I. -I. -I.. -g -O2 -MT blogbench.o -MD -MP -MF ".deps/blogbench.Tpo" -c -o blogbench.o blogbench.c; \ 2024-08-19T01:36:08.026 INFO:tasks.workunit.client.0.smithi083.stdout:then mv -f ".deps/blogbench.Tpo" ".deps/blogbench.Po"; else rm -f ".deps/blogbench.Tpo"; exit 1; fi 2024-08-19T01:36:08.148 INFO:tasks.workunit.client.0.smithi083.stdout:if gcc -DHAVE_CONFIG_H -I. -I. -I.. -g -O2 -MT process.o -MD -MP -MF ".deps/process.Tpo" -c -o process.o process.c; \ 2024-08-19T01:36:08.149 INFO:tasks.workunit.client.0.smithi083.stdout:then mv -f ".deps/process.Tpo" ".deps/process.Po"; else rm -f ".deps/process.Tpo"; exit 1; fi 2024-08-19T01:36:08.239 INFO:tasks.workunit.client.0.smithi083.stdout:if gcc -DHAVE_CONFIG_H -I. -I. -I.. -g -O2 -MT writer.o -MD -MP -MF ".deps/writer.Tpo" -c -o writer.o writer.c; \ 2024-08-19T01:36:08.239 INFO:tasks.workunit.client.0.smithi083.stdout:then mv -f ".deps/writer.Tpo" ".deps/writer.Po"; else rm -f ".deps/writer.Tpo"; exit 1; fi 2024-08-19T01:36:08.318 INFO:tasks.workunit.client.0.smithi083.stdout:if gcc -DHAVE_CONFIG_H -I. -I. -I.. -g -O2 -MT rewriter.o -MD -MP -MF ".deps/rewriter.Tpo" -c -o rewriter.o rewriter.c; \ 2024-08-19T01:36:08.318 INFO:tasks.workunit.client.0.smithi083.stdout:then mv -f ".deps/rewriter.Tpo" ".deps/rewriter.Po"; else rm -f ".deps/rewriter.Tpo"; exit 1; fi 2024-08-19T01:36:08.401 INFO:tasks.workunit.client.0.smithi083.stdout:if gcc -DHAVE_CONFIG_H -I. -I. -I.. -g -O2 -MT reader.o -MD -MP -MF ".deps/reader.Tpo" -c -o reader.o reader.c; \ 2024-08-19T01:36:08.401 INFO:tasks.workunit.client.0.smithi083.stdout:then mv -f ".deps/reader.Tpo" ".deps/reader.Po"; else rm -f ".deps/reader.Tpo"; exit 1; fi 2024-08-19T01:36:08.478 INFO:tasks.workunit.client.0.smithi083.stdout:if gcc -DHAVE_CONFIG_H -I. -I. -I.. -g -O2 -MT commenter.o -MD -MP -MF ".deps/commenter.Tpo" -c -o commenter.o commenter.c; \ 2024-08-19T01:36:08.478 INFO:tasks.workunit.client.0.smithi083.stdout:then mv -f ".deps/commenter.Tpo" ".deps/commenter.Po"; else rm -f ".deps/commenter.Tpo"; exit 1; fi 2024-08-19T01:36:08.552 INFO:tasks.workunit.client.0.smithi083.stdout:if gcc -DHAVE_CONFIG_H -I. -I. -I.. -g -O2 -MT helpers.o -MD -MP -MF ".deps/helpers.Tpo" -c -o helpers.o helpers.c; \ 2024-08-19T01:36:08.552 INFO:tasks.workunit.client.0.smithi083.stdout:then mv -f ".deps/helpers.Tpo" ".deps/helpers.Po"; else rm -f ".deps/helpers.Tpo"; exit 1; fi 2024-08-19T01:36:08.656 INFO:tasks.workunit.client.0.smithi083.stdout:if gcc -DHAVE_CONFIG_H -I. -I. -I.. -g -O2 -MT bsd-getopt_long.o -MD -MP -MF ".deps/bsd-getopt_long.Tpo" -c -o bsd-getopt_long.o bsd-getopt_long.c; \ 2024-08-19T01:36:08.656 INFO:tasks.workunit.client.0.smithi083.stdout:then mv -f ".deps/bsd-getopt_long.Tpo" ".deps/bsd-getopt_long.Po"; else rm -f ".deps/bsd-getopt_long.Tpo"; exit 1; fi 2024-08-19T01:36:08.716 INFO:tasks.workunit.client.0.smithi083.stdout:if gcc -DHAVE_CONFIG_H -I. -I. -I.. -g -O2 -MT fakesnprintf.o -MD -MP -MF ".deps/fakesnprintf.Tpo" -c -o fakesnprintf.o fakesnprintf.c; \ 2024-08-19T01:36:08.716 INFO:tasks.workunit.client.0.smithi083.stdout:then mv -f ".deps/fakesnprintf.Tpo" ".deps/fakesnprintf.Po"; else rm -f ".deps/fakesnprintf.Tpo"; exit 1; fi 2024-08-19T01:36:08.775 INFO:tasks.workunit.client.0.smithi083.stdout:if gcc -DHAVE_CONFIG_H -I. -I. -I.. -g -O2 -MT mysnprintf.o -MD -MP -MF ".deps/mysnprintf.Tpo" -c -o mysnprintf.o mysnprintf.c; \ 2024-08-19T01:36:08.775 INFO:tasks.workunit.client.0.smithi083.stdout:then mv -f ".deps/mysnprintf.Tpo" ".deps/mysnprintf.Po"; else rm -f ".deps/mysnprintf.Tpo"; exit 1; fi 2024-08-19T01:36:08.837 INFO:tasks.workunit.client.0.smithi083.stdout:gcc -g -O2 -o blogbench blogbench.o process.o writer.o rewriter.o reader.o commenter.o helpers.o bsd-getopt_long.o fakesnprintf.o mysnprintf.o 2024-08-19T01:36:08.866 INFO:tasks.workunit.client.0.smithi083.stdout:make[2]: Leaving directory '/home/ubuntu/cephtest/mnt.0/client.0/tmp/blogbench-1.0/src' 2024-08-19T01:36:08.866 INFO:tasks.workunit.client.0.smithi083.stdout:Making all in man 2024-08-19T01:36:08.891 INFO:tasks.workunit.client.0.smithi083.stdout:make[2]: Entering directory '/home/ubuntu/cephtest/mnt.0/client.0/tmp/blogbench-1.0/man' 2024-08-19T01:36:08.891 INFO:tasks.workunit.client.0.smithi083.stdout:make[2]: Nothing to be done for 'all'. 2024-08-19T01:36:08.892 INFO:tasks.workunit.client.0.smithi083.stdout:make[2]: Leaving directory '/home/ubuntu/cephtest/mnt.0/client.0/tmp/blogbench-1.0/man' 2024-08-19T01:36:08.894 INFO:tasks.workunit.client.0.smithi083.stdout:make[2]: Entering directory '/home/ubuntu/cephtest/mnt.0/client.0/tmp/blogbench-1.0' 2024-08-19T01:36:08.897 INFO:tasks.workunit.client.0.smithi083.stdout:make[2]: Leaving directory '/home/ubuntu/cephtest/mnt.0/client.0/tmp/blogbench-1.0' 2024-08-19T01:36:08.897 INFO:tasks.workunit.client.0.smithi083.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/mnt.0/client.0/tmp/blogbench-1.0' 2024-08-19T01:36:08.898 INFO:tasks.workunit.client.0.smithi083.stderr:+ cd src 2024-08-19T01:36:08.898 INFO:tasks.workunit.client.0.smithi083.stderr:+ mkdir blogtest_in 2024-08-19T01:36:08.903 INFO:tasks.workunit.client.0.smithi083.stdout:running blogbench 2024-08-19T01:36:08.903 INFO:tasks.workunit.client.0.smithi083.stderr:+ echo 'running blogbench' 2024-08-19T01:36:08.903 INFO:tasks.workunit.client.0.smithi083.stderr:+ ./blogbench -d blogtest_in 2024-08-19T01:36:08.904 INFO:tasks.workunit.client.0.smithi083.stdout: 2024-08-19T01:36:08.904 INFO:tasks.workunit.client.0.smithi083.stdout:Frequency = 10 secs 2024-08-19T01:36:08.904 INFO:tasks.workunit.client.0.smithi083.stdout:Scratch dir = [blogtest_in] 2024-08-19T01:36:09.905 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:36:09 smithi083 ceph-mon[17120]: pgmap v378: 97 pgs: 97 active+clean; 3.0 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 108 KiB/s rd, 225 KiB/s wr, 31 op/s 2024-08-19T01:36:09.952 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:36:09 smithi117 ceph-mon[22016]: pgmap v378: 97 pgs: 97 active+clean; 3.0 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 108 KiB/s rd, 225 KiB/s wr, 31 op/s 2024-08-19T01:36:09.966 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:36:09 smithi167 ceph-mon[21875]: pgmap v378: 97 pgs: 97 active+clean; 3.0 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 108 KiB/s rd, 225 KiB/s wr, 31 op/s 2024-08-19T01:36:10.011 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:10.010+0000 7fe0b8e00640 1 -- 172.21.15.83:0/3814624770 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe0b4104a90 msgr2=0x7fe0b4104f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:36:10.011 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:10.010+0000 7fe0b8e00640 1 --2- 172.21.15.83:0/3814624770 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe0b4104a90 0x7fe0b4104f10 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7fe0a0009900 tx=0x7fe0a002a410 comp rx=0 tx=0).stop 2024-08-19T01:36:10.012 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:10.010+0000 7fe0b8e00640 1 -- 172.21.15.83:0/3814624770 shutdown_connections 2024-08-19T01:36:10.012 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:10.010+0000 7fe0b8e00640 1 --2- 172.21.15.83:0/3814624770 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe0b4105450 0x7fe0b410bce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:10.012 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:10.010+0000 7fe0b8e00640 1 --2- 172.21.15.83:0/3814624770 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe0b4104a90 0x7fe0b4104f10 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:10.012 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:10.010+0000 7fe0b8e00640 1 --2- 172.21.15.83:0/3814624770 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe0b41038a0 0x7fe0b4103ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:10.012 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:10.010+0000 7fe0b8e00640 1 -- 172.21.15.83:0/3814624770 >> 172.21.15.83:0/3814624770 conn(0x7fe0b40ff020 msgr2=0x7fe0b4101440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:36:10.012 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:10.010+0000 7fe0b8e00640 1 -- 172.21.15.83:0/3814624770 shutdown_connections 2024-08-19T01:36:10.012 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:10.011+0000 7fe0b8e00640 1 -- 172.21.15.83:0/3814624770 wait complete. 2024-08-19T01:36:10.014 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:10.011+0000 7fe0b8e00640 1 Processor -- start 2024-08-19T01:36:10.014 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:10.011+0000 7fe0b8e00640 1 -- start start 2024-08-19T01:36:10.014 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:10.012+0000 7fe0b8e00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe0b41038a0 0x7fe0b4199b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:36:10.014 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:10.012+0000 7fe0b8e00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe0b4104a90 0x7fe0b419a0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:36:10.015 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:10.012+0000 7fe0b8e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe0b4105450 0x7fe0b41a1120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:36:10.015 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:10.012+0000 7fe0b8e00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fe0b410f550 con 0x7fe0b4105450 2024-08-19T01:36:10.015 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:10.012+0000 7fe0b8e00640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fe0b410f3d0 con 0x7fe0b41038a0 2024-08-19T01:36:10.015 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:10.012+0000 7fe0b8e00640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fe0b410f6d0 con 0x7fe0b4104a90 2024-08-19T01:36:10.015 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:10.012+0000 7fe0b3e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe0b4105450 0x7fe0b41a1120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:36:10.015 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:10.012+0000 7fe0b3400640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe0b41038a0 0x7fe0b4199b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:36:10.015 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:10.012+0000 7fe0b3e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe0b4105450 0x7fe0b41a1120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:41308/0 (socket says 172.21.15.83:41308) 2024-08-19T01:36:10.015 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:10.012+0000 7fe0b3e00640 1 -- 172.21.15.83:0/2409881573 learned_addr learned my addr 172.21.15.83:0/2409881573 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-08-19T01:36:10.015 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:10.012+0000 7fe0b2a00640 1 --2- 172.21.15.83:0/2409881573 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe0b4104a90 0x7fe0b419a0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:36:10.015 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:10.012+0000 7fe0b3e00640 1 -- 172.21.15.83:0/2409881573 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe0b4104a90 msgr2=0x7fe0b419a0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:36:10.015 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:10.012+0000 7fe0b3e00640 1 --2- 172.21.15.83:0/2409881573 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe0b4104a90 0x7fe0b419a0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:10.015 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:10.012+0000 7fe0b3e00640 1 -- 172.21.15.83:0/2409881573 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe0b41038a0 msgr2=0x7fe0b4199b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:36:10.016 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:10.012+0000 7fe0b3e00640 1 --2- 172.21.15.83:0/2409881573 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe0b41038a0 0x7fe0b4199b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:10.016 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:10.012+0000 7fe0b3e00640 1 -- 172.21.15.83:0/2409881573 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe0b41a1790 con 0x7fe0b4105450 2024-08-19T01:36:10.016 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:10.013+0000 7fe0b3e00640 1 --2- 172.21.15.83:0/2409881573 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe0b4105450 0x7fe0b41a1120 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7fe0ac00eb10 tx=0x7fe0ac00c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:36:10.016 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:10.013+0000 7fe0abe00640 1 -- 172.21.15.83:0/2409881573 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0ac004b20 con 0x7fe0b4105450 2024-08-19T01:36:10.016 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:10.013+0000 7fe0abe00640 1 -- 172.21.15.83:0/2409881573 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe0ac004cc0 con 0x7fe0b4105450 2024-08-19T01:36:10.016 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:10.013+0000 7fe0b8e00640 1 -- 172.21.15.83:0/2409881573 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe0b41a1a80 con 0x7fe0b4105450 2024-08-19T01:36:10.016 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:10.013+0000 7fe0abe00640 1 -- 172.21.15.83:0/2409881573 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0ac010760 con 0x7fe0b4105450 2024-08-19T01:36:10.016 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:10.013+0000 7fe0b8e00640 1 -- 172.21.15.83:0/2409881573 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe0b41a1f50 con 0x7fe0b4105450 2024-08-19T01:36:10.017 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:10.017+0000 7fe0bd295740 1 Processor -- start 2024-08-19T01:36:10.018 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:10.017+0000 7fe0abe00640 1 -- 172.21.15.83:0/2409881573 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7fe0ac005020 con 0x7fe0b4105450 2024-08-19T01:36:10.018 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:10.017+0000 7fe0bd295740 1 -- start start 2024-08-19T01:36:10.020 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:10.018+0000 7fe0bd295740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55624b8af680 0x55624b8afa80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:36:10.020 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:10.018+0000 7fe0bd295740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55624b8affc0 0x55624b8b4a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:36:10.020 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:10.018+0000 7fe0bd295740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55624b8b5050 0x55624b8b7510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:36:10.020 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:10.018+0000 7fe0bd295740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55624b813890 con 0x55624b8b5050 2024-08-19T01:36:10.020 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:10.018+0000 7fe0bd295740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55624b7abd90 con 0x55624b8affc0 2024-08-19T01:36:10.020 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:10.018+0000 7fe0bd295740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55624b80f850 con 0x55624b8af680 2024-08-19T01:36:10.020 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:10.018+0000 7fe0b2a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55624b8b5050 0x55624b8b7510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:36:10.020 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:10.018+0000 7fe0b2a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55624b8b5050 0x55624b8b7510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:41318/0 (socket says 172.21.15.83:41318) 2024-08-19T01:36:10.020 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:10.018+0000 7fe0b2a00640 1 -- 172.21.15.83:0/4213812428 learned_addr learned my addr 172.21.15.83:0/4213812428 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-08-19T01:36:10.021 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:10.018+0000 7fe0b3400640 1 --2- 172.21.15.83:0/4213812428 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55624b8affc0 0x55624b8b4a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:36:10.021 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:10.018+0000 7fe0b2a00640 1 --2- 172.21.15.83:0/4213812428 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55624b8af680 0x55624b8afa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:36:10.021 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:10.018+0000 7fe0b3400640 1 -- 172.21.15.83:0/4213812428 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55624b8af680 msgr2=0x55624b8afa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:36:10.021 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:10.018+0000 7fe0b3400640 1 --2- 172.21.15.83:0/4213812428 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55624b8af680 0x55624b8afa80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:10.021 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:10.018+0000 7fe0b3400640 1 -- 172.21.15.83:0/4213812428 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55624b8b5050 msgr2=0x55624b8b7510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:36:10.021 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:10.018+0000 7fe0b3400640 1 --2- 172.21.15.83:0/4213812428 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55624b8b5050 0x55624b8b7510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:10.021 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:10.019+0000 7fe0b3400640 1 -- 172.21.15.83:0/4213812428 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55624b527210 con 0x55624b8affc0 2024-08-19T01:36:10.021 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:10.019+0000 7fe0b3400640 1 --2- 172.21.15.83:0/4213812428 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55624b8affc0 0x55624b8b4a80 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fe0a400bbc0 tx=0x7fe0a40107a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=859248cb778b81e1 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:36:10.021 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:10.019+0000 7fe0abe00640 1 --2- 172.21.15.83:0/2409881573 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7fe0980a2b80 0x7fe0980a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:36:10.021 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:10.019+0000 7fe0a8c00640 1 -- 172.21.15.83:0/4213812428 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0a4004030 con 0x55624b8affc0 2024-08-19T01:36:10.021 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:10.019+0000 7fe0abe00640 1 -- 172.21.15.83:0/2409881573 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8269+0+0 (secure 0 0 0) 0x7fe0ac0cf670 con 0x7fe0b4105450 2024-08-19T01:36:10.021 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:10.019+0000 7fe0a8c00640 1 -- 172.21.15.83:0/4213812428 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe0a4005500 con 0x55624b8affc0 2024-08-19T01:36:10.021 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:10.019+0000 7fe0a8c00640 1 -- 172.21.15.83:0/4213812428 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0a4004c20 con 0x55624b8affc0 2024-08-19T01:36:10.022 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:10.019+0000 7fe0b2a00640 1 --2- 172.21.15.83:0/2409881573 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7fe0980a2b80 0x7fe0980a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:36:10.026 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:10.020+0000 7fe0bd295740 1 -- 172.21.15.83:0/4213812428 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55624b8affc0 msgr2=0x55624b8b4a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:36:10.026 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:10.020+0000 7fe0b2a00640 1 --2- 172.21.15.83:0/2409881573 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7fe0980a2b80 0x7fe0980a5040 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fe0a0000c00 tx=0x7fe0a0041040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:36:10.026 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:10.020+0000 7fe0bd295740 1 --2- 172.21.15.83:0/4213812428 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55624b8affc0 0x55624b8b4a80 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fe0a400bbc0 tx=0x7fe0a40107a0 comp rx=0 tx=0).stop 2024-08-19T01:36:10.026 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:10.020+0000 7fe0bd295740 1 -- 172.21.15.83:0/4213812428 shutdown_connections 2024-08-19T01:36:10.026 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:10.020+0000 7fe0bd295740 1 --2- 172.21.15.83:0/4213812428 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55624b8b5050 0x55624b8b7510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:10.027 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:10.020+0000 7fe0bd295740 1 --2- 172.21.15.83:0/4213812428 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55624b8affc0 0x55624b8b4a80 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:10.027 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:10.020+0000 7fe0bd295740 1 --2- 172.21.15.83:0/4213812428 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55624b8af680 0x55624b8afa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:10.027 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:10.020+0000 7fe0bd295740 1 -- 172.21.15.83:0/4213812428 >> 172.21.15.83:0/4213812428 conn(0x55624b8aacf0 msgr2=0x55624b8ab0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:36:10.027 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:10.020+0000 7fe0bd295740 1 -- 172.21.15.83:0/4213812428 shutdown_connections 2024-08-19T01:36:10.027 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:10.020+0000 7fe0bd295740 1 -- 172.21.15.83:0/4213812428 wait complete. 2024-08-19T01:36:10.027 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:10.021+0000 7fe0bd295740 1 -- start start 2024-08-19T01:36:10.027 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:10.021+0000 7fe0bd295740 1 Processor -- start 2024-08-19T01:36:10.027 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:10.021+0000 7fe0bd295740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55624b8af680 0x55624b8b98e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:36:10.027 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:10.021+0000 7fe0bd295740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55624b8affc0 0x55624b8b9e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:36:10.027 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:10.021+0000 7fe0bd295740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55624b8b5050 0x55624b8c0e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:36:10.027 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:10.021+0000 7fe0bd295740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55624b8b80c0 con 0x55624b8af680 2024-08-19T01:36:10.028 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:10.021+0000 7fe0bd295740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55624b7cef70 con 0x55624b8b5050 2024-08-19T01:36:10.028 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:10.021+0000 7fe0bd295740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55624b8b8240 con 0x55624b8affc0 2024-08-19T01:36:10.028 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:10.021+0000 7fe0bd295740 10 client.0 _ll_register_callbacks cb 0x55624b80b8d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-19T01:36:10.028 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:10.021+0000 7fe0b3400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55624b8af680 0x55624b8b98e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:36:10.028 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:10.021+0000 7fe0b3e00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55624b8b5050 0x55624b8c0e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:36:10.028 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:10.021+0000 7fe0b3e00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55624b8b5050 0x55624b8c0e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.83:34506/0 (socket says 172.21.15.83:34506) 2024-08-19T01:36:10.028 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:10.022+0000 7fe0b3e00640 1 -- 172.21.15.83:0/2135233164 learned_addr learned my addr 172.21.15.83:0/2135233164 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-08-19T01:36:10.028 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:10.021+0000 7fe0b3400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55624b8af680 0x55624b8b98e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:41322/0 (socket says 172.21.15.83:41322) 2024-08-19T01:36:10.028 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:10.022+0000 7fe0b2a00640 1 --2- 172.21.15.83:0/2135233164 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55624b8affc0 0x55624b8b9e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:36:10.028 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:10.022+0000 7fe0b3e00640 1 -- 172.21.15.83:0/2135233164 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55624b8affc0 msgr2=0x55624b8b9e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:36:10.028 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:10.022+0000 7fe0b3e00640 1 --2- 172.21.15.83:0/2135233164 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55624b8affc0 0x55624b8b9e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:10.028 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:10.022+0000 7fe0b3e00640 1 -- 172.21.15.83:0/2135233164 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55624b8af680 msgr2=0x55624b8b98e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:36:10.028 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:10.022+0000 7fe0b3e00640 1 --2- 172.21.15.83:0/2135233164 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55624b8af680 0x55624b8b98e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:10.029 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:10.022+0000 7fe0b3e00640 1 -- 172.21.15.83:0/2135233164 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55624b5f3620 con 0x55624b8b5050 2024-08-19T01:36:10.029 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:10.022+0000 7fe0b3e00640 1 --2- 172.21.15.83:0/2135233164 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55624b8b5050 0x55624b8c0e60 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fe0ac0a4590 tx=0x7fe0ac09b7b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f3b5206c21f5aa7 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:36:10.029 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:10.022+0000 7fe09d600640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-08-19T01:36:10.029 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:10.022+0000 7fe09d600640 1 -- 172.21.15.83:0/2135233164 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0ac0a4830 con 0x55624b8b5050 2024-08-19T01:36:10.029 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:10.022+0000 7fe09d600640 1 -- 172.21.15.83:0/2135233164 <== mon.2 v2:172.21.15.117:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8269+0+0 (secure 0 0 0) 0x7fe0ac005310 con 0x55624b8b5050 2024-08-19T01:36:10.029 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:10.022+0000 7fe0bd295740 1 -- 172.21.15.83:0/2135233164 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe0a4005500 con 0x55624b8b5050 2024-08-19T01:36:10.029 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:10.022+0000 7fe09d600640 1 -- 172.21.15.83:0/2135233164 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fe0ac0a0200 con 0x55624b8b5050 2024-08-19T01:36:10.029 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:10.022+0000 7fe0bd295740 10 client.25367 fetch_fsmap learned FSMap version 25 2024-08-19T01:36:10.029 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:10.023+0000 7fe0bd295740 1 -- 172.21.15.83:0/2135233164 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55624b8c40a0 con 0x55624b8b5050 2024-08-19T01:36:10.029 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:10.023+0000 7fe09d600640 1 -- 172.21.15.83:0/2135233164 <== mon.2 v2:172.21.15.117:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe0a4005500 con 0x55624b8b5050 2024-08-19T01:36:10.029 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:10.023+0000 7fe09d600640 1 -- 172.21.15.83:0/2135233164 <== mon.2 v2:172.21.15.117:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fe0ac0cf0e0 con 0x55624b8b5050 2024-08-19T01:36:10.029 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:10.023+0000 7fe0bd295740 10 client.25367 fetch_fsmap finished waiting for FSMap version 25 2024-08-19T01:36:10.029 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:10.023+0000 7fe0bd295740 10 client.25367 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-08-19T01:36:10.030 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:10.023+0000 7fe0bd295740 20 client.25367 populate_metadata read hostname 'smithi083' 2024-08-19T01:36:10.030 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:10.023+0000 7fe0bd295740 1 --2- 172.21.15.83:0/2135233164 >> [v2:172.21.15.117:6838/1967595704,v1:172.21.15.117:6839/1967595704] conn(0x55624b8c5800 0x55624b8c7bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:36:10.030 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:10.023+0000 7fe0bd295740 4 client.25367 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-19T01:36:10.030 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:10.023+0000 7fe0bd295740 1 -- 172.21.15.83:0/2135233164 --> [v2:172.21.15.117:6838/1967595704,v1:172.21.15.117:6839/1967595704] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55624b820fc0 con 0x55624b8c5800 2024-08-19T01:36:10.030 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:10.023+0000 7fe0b3400640 1 --2- 172.21.15.83:0/2135233164 >> [v2:172.21.15.117:6838/1967595704,v1:172.21.15.117:6839/1967595704] conn(0x55624b8c5800 0x55624b8c7bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:36:10.030 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:10.024+0000 7fe0b3400640 1 --2- 172.21.15.83:0/2135233164 >> [v2:172.21.15.117:6838/1967595704,v1:172.21.15.117:6839/1967595704] conn(0x55624b8c5800 0x55624b8c7bf0 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=62624ecbbebe967c server_cookie=4ed54a53eeb1cfbe in_seq=0 out_seq=0 2024-08-19T01:36:10.030 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:10.024+0000 7fe09d600640 10 client.25367 ms_handle_connect on v2:172.21.15.117:6838/1967595704 2024-08-19T01:36:10.030 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:10.025+0000 7fe09d600640 1 -- 172.21.15.83:0/2135233164 <== mds.0 v2:172.21.15.117:6838/1967595704 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55624b820fc0 con 0x55624b8c5800 2024-08-19T01:36:10.030 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:10.025+0000 7fe09d600640 10 client.25367 handle_command_reply: tid=0 2024-08-19T01:36:10.030 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:10.025+0000 7fe0bd295740 1 client.25367 shutdown 2024-08-19T01:36:10.030 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:10.025+0000 7fe0bd295740 1 -- 172.21.15.83:0/2135233164 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55624b8b5050 msgr2=0x55624b8c0e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:36:10.030 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:10.025+0000 7fe0bd295740 1 --2- 172.21.15.83:0/2135233164 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55624b8b5050 0x55624b8c0e60 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fe0ac0a4590 tx=0x7fe0ac09b7b0 comp rx=0 tx=0).stop 2024-08-19T01:36:10.030 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:10.025+0000 7fe0bd295740 1 -- 172.21.15.83:0/2135233164 shutdown_connections 2024-08-19T01:36:10.030 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:10.025+0000 7fe0bd295740 1 --2- 172.21.15.83:0/2135233164 >> [v2:172.21.15.117:6838/1967595704,v1:172.21.15.117:6839/1967595704] conn(0x55624b8c5800 0x55624b8c7bf0 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:10.031 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:10.025+0000 7fe0bd295740 1 --2- 172.21.15.83:0/2135233164 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55624b8b5050 0x55624b8c0e60 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:10.031 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:10.025+0000 7fe0bd295740 1 --2- 172.21.15.83:0/2135233164 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55624b8affc0 0x55624b8b9e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:10.031 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:10.025+0000 7fe0bd295740 1 --2- 172.21.15.83:0/2135233164 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55624b8af680 0x55624b8b98e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:10.031 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:10.025+0000 7fe09d600640 0 client.25367 ms_handle_reset on v2:172.21.15.117:6838/1967595704 2024-08-19T01:36:10.031 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:10.025+0000 7fe0bd295740 1 -- 172.21.15.83:0/2135233164 >> 172.21.15.83:0/2135233164 conn(0x55624b8aacf0 msgr2=0x55624b8ac500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:36:10.031 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:10.025+0000 7fe0bd295740 1 -- 172.21.15.83:0/2135233164 shutdown_connections 2024-08-19T01:36:10.031 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:10.025+0000 7fe0bd295740 1 -- 172.21.15.83:0/2135233164 wait complete. 2024-08-19T01:36:10.031 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:10.025+0000 7fe0bd295740 20 client.25367 trim_cache size 0 max 16384 2024-08-19T01:36:10.041 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:10.040+0000 7fe0bd295740 1 Processor -- start 2024-08-19T01:36:10.041 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:10.040+0000 7fe0bd295740 1 -- start start 2024-08-19T01:36:10.041 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:10.040+0000 7fe0bd295740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55624b8af680 0x55624b8b4740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:36:10.041 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:10.040+0000 7fe0bd295740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55624b8affc0 0x55624b8c0e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:36:10.041 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:10.040+0000 7fe0bd295740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55624b8b5050 0x55624b8abba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:36:10.042 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:10.040+0000 7fe0bd295740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fe0b41ad5a0 con 0x55624b8b5050 2024-08-19T01:36:10.042 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:10.040+0000 7fe0bd295740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fe0b41aae70 con 0x55624b8af680 2024-08-19T01:36:10.042 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:10.040+0000 7fe0bd295740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fe0b41a8cc0 con 0x55624b8affc0 2024-08-19T01:36:10.042 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:10.040+0000 7fe0b3e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55624b8b5050 0x55624b8abba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:36:10.042 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:10.040+0000 7fe0b3e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55624b8b5050 0x55624b8abba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:41334/0 (socket says 172.21.15.83:41334) 2024-08-19T01:36:10.042 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:10.040+0000 7fe0b3e00640 1 -- 172.21.15.83:0/3623777483 learned_addr learned my addr 172.21.15.83:0/3623777483 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-08-19T01:36:10.042 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:10.040+0000 7fe0b3400640 1 --2- 172.21.15.83:0/3623777483 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55624b8af680 0x55624b8b4740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:36:10.042 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:10.040+0000 7fe0b2a00640 1 --2- 172.21.15.83:0/3623777483 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55624b8affc0 0x55624b8c0e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:36:10.042 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:10.041+0000 7fe0b3e00640 1 -- 172.21.15.83:0/3623777483 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55624b8affc0 msgr2=0x55624b8c0e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:36:10.042 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:10.041+0000 7fe0b3e00640 1 --2- 172.21.15.83:0/3623777483 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55624b8affc0 0x55624b8c0e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:10.043 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:10.041+0000 7fe0b3e00640 1 -- 172.21.15.83:0/3623777483 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55624b8af680 msgr2=0x55624b8b4740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:36:10.043 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:10.041+0000 7fe0b3e00640 1 --2- 172.21.15.83:0/3623777483 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55624b8af680 0x55624b8b4740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:10.043 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:10.041+0000 7fe0b3e00640 1 -- 172.21.15.83:0/3623777483 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55624b8b9420 con 0x55624b8b5050 2024-08-19T01:36:10.043 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:10.041+0000 7fe0b3400640 1 --2- 172.21.15.83:0/3623777483 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55624b8af680 0x55624b8b4740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-19T01:36:10.047 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:10.041+0000 7fe0b3e00640 1 --2- 172.21.15.83:0/3623777483 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55624b8b5050 0x55624b8abba0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7fe0ac0049d0 tx=0x7fe0ac09bfd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3757be5e65c81ee2 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:36:10.047 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:10.041+0000 7fe09fe00640 1 -- 172.21.15.83:0/3623777483 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0ac0a4ac0 con 0x55624b8b5050 2024-08-19T01:36:10.047 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:10.041+0000 7fe09fe00640 1 -- 172.21.15.83:0/3623777483 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe0ac09e070 con 0x55624b8b5050 2024-08-19T01:36:10.047 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:10.041+0000 7fe09fe00640 1 -- 172.21.15.83:0/3623777483 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0ac0a4630 con 0x55624b8b5050 2024-08-19T01:36:10.048 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:10.042+0000 7fe0bd295740 1 -- 172.21.15.83:0/3623777483 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55624b8b5050 msgr2=0x55624b8abba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:36:10.048 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:10.042+0000 7fe0bd295740 1 --2- 172.21.15.83:0/3623777483 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55624b8b5050 0x55624b8abba0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7fe0ac0049d0 tx=0x7fe0ac09bfd0 comp rx=0 tx=0).stop 2024-08-19T01:36:10.048 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:10.042+0000 7fe0bd295740 1 -- 172.21.15.83:0/3623777483 shutdown_connections 2024-08-19T01:36:10.048 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:10.042+0000 7fe0bd295740 1 --2- 172.21.15.83:0/3623777483 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55624b8b5050 0x55624b8abba0 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:10.048 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:10.042+0000 7fe0bd295740 1 --2- 172.21.15.83:0/3623777483 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55624b8affc0 0x55624b8c0e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:10.048 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:10.042+0000 7fe0bd295740 1 --2- 172.21.15.83:0/3623777483 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55624b8af680 0x55624b8b4740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:10.048 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:10.042+0000 7fe0bd295740 1 -- 172.21.15.83:0/3623777483 >> 172.21.15.83:0/3623777483 conn(0x55624b8aacf0 msgr2=0x55624b8b98e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:36:10.048 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:10.042+0000 7fe0bd295740 1 -- 172.21.15.83:0/3623777483 shutdown_connections 2024-08-19T01:36:10.048 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:10.042+0000 7fe0bd295740 1 -- 172.21.15.83:0/3623777483 wait complete. 2024-08-19T01:36:10.048 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:10.042+0000 7fe0bd295740 1 -- start start 2024-08-19T01:36:10.048 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:10.043+0000 7fe0bd295740 1 Processor -- start 2024-08-19T01:36:10.049 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:10.043+0000 7fe0bd295740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55624b8af680 0x55624b8bdff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:36:10.049 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:10.043+0000 7fe0bd295740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55624b8affc0 0x55624b8be530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:36:10.049 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:10.043+0000 7fe0bd295740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55624b8b5050 0x55624b8bf4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:36:10.049 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:10.043+0000 7fe0bd295740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fe0b41a9330 con 0x55624b8affc0 2024-08-19T01:36:10.049 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:10.043+0000 7fe0bd295740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fe0b41aa7a0 con 0x55624b8af680 2024-08-19T01:36:10.049 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:10.043+0000 7fe0bd295740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fe0b41a9a00 con 0x55624b8b5050 2024-08-19T01:36:10.049 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:10.043+0000 7fe0bd295740 10 client.0 _ll_register_callbacks cb 0x7fe0ac09bc20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-19T01:36:10.049 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:10.043+0000 7fe0b2a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55624b8affc0 0x55624b8be530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:36:10.049 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:10.043+0000 7fe0b2a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55624b8affc0 0x55624b8be530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:41336/0 (socket says 172.21.15.83:41336) 2024-08-19T01:36:10.049 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:10.043+0000 7fe0b2a00640 1 -- 172.21.15.83:0/1155470064 learned_addr learned my addr 172.21.15.83:0/1155470064 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-08-19T01:36:10.049 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:10.043+0000 7fe0b3400640 1 --2- 172.21.15.83:0/1155470064 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55624b8af680 0x55624b8bdff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:36:10.049 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:10.043+0000 7fe0b3e00640 1 --2- 172.21.15.83:0/1155470064 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55624b8b5050 0x55624b8bf4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:36:10.049 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:10.044+0000 7fe0b2a00640 1 -- 172.21.15.83:0/1155470064 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55624b8b5050 msgr2=0x55624b8bf4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:36:10.050 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:10.044+0000 7fe0b2a00640 1 --2- 172.21.15.83:0/1155470064 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55624b8b5050 0x55624b8bf4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:10.050 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:10.044+0000 7fe0b2a00640 1 -- 172.21.15.83:0/1155470064 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55624b8af680 msgr2=0x55624b8bdff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:36:10.050 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:10.044+0000 7fe0b2a00640 1 --2- 172.21.15.83:0/1155470064 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55624b8af680 0x55624b8bdff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:10.050 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:10.044+0000 7fe0b2a00640 1 -- 172.21.15.83:0/1155470064 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55624b8bfb00 con 0x55624b8affc0 2024-08-19T01:36:10.050 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:10.044+0000 7fe0b3e00640 1 --2- 172.21.15.83:0/1155470064 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55624b8b5050 0x55624b8bf4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-19T01:36:10.050 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:10.044+0000 7fe0b2a00640 1 --2- 172.21.15.83:0/1155470064 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55624b8affc0 0x55624b8be530 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7fe0a00314e0 tx=0x7fe0a002a410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=903a43a481771a89 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:36:10.050 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:10.044+0000 7fe09d600640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-08-19T01:36:10.050 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:10.044+0000 7fe09d600640 1 -- 172.21.15.83:0/1155470064 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0a0046070 con 0x55624b8affc0 2024-08-19T01:36:10.050 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:10.044+0000 7fe0bd295740 1 -- 172.21.15.83:0/1155470064 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe0ac09e070 con 0x55624b8affc0 2024-08-19T01:36:10.050 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:10.044+0000 7fe09d600640 1 -- 172.21.15.83:0/1155470064 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8269+0+0 (secure 0 0 0) 0x7fe0a0003a00 con 0x55624b8affc0 2024-08-19T01:36:10.050 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:10.044+0000 7fe09d600640 1 -- 172.21.15.83:0/1155470064 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fe0a005e070 con 0x55624b8affc0 2024-08-19T01:36:10.051 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:10.044+0000 7fe0bd295740 10 client.15294 fetch_fsmap learned FSMap version 25 2024-08-19T01:36:10.051 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:10.044+0000 7fe0bd295740 1 -- 172.21.15.83:0/1155470064 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55624b8c0ed0 con 0x55624b8affc0 2024-08-19T01:36:10.051 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:10.045+0000 7fe09d600640 1 -- 172.21.15.83:0/1155470064 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe0ac09e070 con 0x55624b8affc0 2024-08-19T01:36:10.051 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:10.045+0000 7fe09d600640 1 -- 172.21.15.83:0/1155470064 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fe0a003d450 con 0x55624b8affc0 2024-08-19T01:36:10.051 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:10.045+0000 7fe0bd295740 10 client.15294 fetch_fsmap finished waiting for FSMap version 25 2024-08-19T01:36:10.051 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:10.045+0000 7fe0bd295740 10 client.15294 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-08-19T01:36:10.051 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:10.045+0000 7fe0bd295740 20 client.15294 populate_metadata read hostname 'smithi083' 2024-08-19T01:36:10.051 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:10.045+0000 7fe0bd295740 1 --2- 172.21.15.83:0/1155470064 >> [v2:172.21.15.117:6838/1967595704,v1:172.21.15.117:6839/1967595704] conn(0x55624b930930 0x55624b932d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:36:10.051 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:10.045+0000 7fe0bd295740 4 client.15294 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-08-19T01:36:10.051 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:10.045+0000 7fe0bd295740 1 -- 172.21.15.83:0/1155470064 --> [v2:172.21.15.117:6838/1967595704,v1:172.21.15.117:6839/1967595704] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55624b81b310 con 0x55624b930930 2024-08-19T01:36:10.052 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:10.045+0000 7fe0b3400640 1 --2- 172.21.15.83:0/1155470064 >> [v2:172.21.15.117:6838/1967595704,v1:172.21.15.117:6839/1967595704] conn(0x55624b930930 0x55624b932d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:36:10.052 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:10.046+0000 7fe0b3400640 1 --2- 172.21.15.83:0/1155470064 >> [v2:172.21.15.117:6838/1967595704,v1:172.21.15.117:6839/1967595704] conn(0x55624b930930 0x55624b932d50 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f5e487928572f635 server_cookie=7fc8e207bbbd39a2 in_seq=0 out_seq=0 2024-08-19T01:36:10.052 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:10.046+0000 7fe09d600640 10 client.15294 ms_handle_connect on v2:172.21.15.117:6838/1967595704 2024-08-19T01:36:10.095 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:10.093+0000 7fe09d600640 1 -- 172.21.15.83:0/1155470064 <== mds.0 v2:172.21.15.117:6838/1967595704 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55624b81b310 con 0x55624b930930 2024-08-19T01:36:10.095 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:10.093+0000 7fe09d600640 10 client.15294 handle_command_reply: tid=0 2024-08-19T01:36:10.095 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:10.093+0000 7fe0bd295740 1 client.15294 shutdown 2024-08-19T01:36:10.095 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:10.094+0000 7fe0bd295740 1 -- 172.21.15.83:0/1155470064 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55624b8affc0 msgr2=0x55624b8be530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:36:10.095 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:10.094+0000 7fe0bd295740 1 --2- 172.21.15.83:0/1155470064 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55624b8affc0 0x55624b8be530 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7fe0a00314e0 tx=0x7fe0a002a410 comp rx=0 tx=0).stop 2024-08-19T01:36:10.096 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:10.094+0000 7fe0bd295740 1 -- 172.21.15.83:0/1155470064 shutdown_connections 2024-08-19T01:36:10.096 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:10.094+0000 7fe0bd295740 1 --2- 172.21.15.83:0/1155470064 >> [v2:172.21.15.117:6838/1967595704,v1:172.21.15.117:6839/1967595704] conn(0x55624b930930 0x55624b932d50 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:10.096 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:10.094+0000 7fe0bd295740 1 --2- 172.21.15.83:0/1155470064 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55624b8b5050 0x55624b8bf4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:10.096 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:10.094+0000 7fe0bd295740 1 --2- 172.21.15.83:0/1155470064 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55624b8affc0 0x55624b8be530 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:10.096 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:10.094+0000 7fe09d600640 0 client.15294 ms_handle_reset on v2:172.21.15.117:6838/1967595704 2024-08-19T01:36:10.096 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:10.094+0000 7fe0bd295740 1 --2- 172.21.15.83:0/1155470064 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55624b8af680 0x55624b8bdff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:10.096 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:10.094+0000 7fe0bd295740 1 -- 172.21.15.83:0/1155470064 >> 172.21.15.83:0/1155470064 conn(0x55624b8aacf0 msgr2=0x55624b8b9880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:36:10.096 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:10.094+0000 7fe0bd295740 1 -- 172.21.15.83:0/1155470064 shutdown_connections 2024-08-19T01:36:10.097 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:10.094+0000 7fe0bd295740 1 -- 172.21.15.83:0/1155470064 wait complete. 2024-08-19T01:36:10.097 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:10.094+0000 7fe0bd295740 20 client.15294 trim_cache size 0 max 16384 2024-08-19T01:36:10.097 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stdout:{ 2024-08-19T01:36:10.097 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stdout: "message": "", 2024-08-19T01:36:10.097 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stdout: "return_code": 0 2024-08-19T01:36:10.097 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stdout:} 2024-08-19T01:36:10.098 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:10.095+0000 7fe0b8e00640 1 -- 172.21.15.83:0/2409881573 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7fe0980a2b80 msgr2=0x7fe0980a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:36:10.098 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:10.095+0000 7fe0b8e00640 1 --2- 172.21.15.83:0/2409881573 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7fe0980a2b80 0x7fe0980a5040 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fe0a0000c00 tx=0x7fe0a0041040 comp rx=0 tx=0).stop 2024-08-19T01:36:10.098 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:10.095+0000 7fe0b8e00640 1 -- 172.21.15.83:0/2409881573 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe0b4105450 msgr2=0x7fe0b41a1120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:36:10.098 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:10.095+0000 7fe0b8e00640 1 --2- 172.21.15.83:0/2409881573 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe0b4105450 0x7fe0b41a1120 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7fe0ac00eb10 tx=0x7fe0ac00c2d0 comp rx=0 tx=0).stop 2024-08-19T01:36:10.099 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:10.095+0000 7fe0b8e00640 1 -- 172.21.15.83:0/2409881573 shutdown_connections 2024-08-19T01:36:10.099 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:10.095+0000 7fe0b8e00640 1 --2- 172.21.15.83:0/2409881573 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7fe0980a2b80 0x7fe0980a5040 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:10.099 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:10.095+0000 7fe0b8e00640 1 --2- 172.21.15.83:0/2409881573 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe0b4105450 0x7fe0b41a1120 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:10.099 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:10.095+0000 7fe0b8e00640 1 --2- 172.21.15.83:0/2409881573 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe0b4104a90 0x7fe0b419a0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:10.099 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:10.095+0000 7fe0b8e00640 1 --2- 172.21.15.83:0/2409881573 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe0b41038a0 0x7fe0b4199b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:10.099 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:10.095+0000 7fe0b8e00640 1 -- 172.21.15.83:0/2409881573 >> 172.21.15.83:0/2409881573 conn(0x7fe0b40ff020 msgr2=0x7fe0b4107d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:36:10.099 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:10.095+0000 7fe0b8e00640 1 -- 172.21.15.83:0/2409881573 shutdown_connections 2024-08-19T01:36:10.099 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:10.095+0000 7fe0b8e00640 1 -- 172.21.15.83:0/2409881573 wait complete. 2024-08-19T01:36:10.373 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833 shell --fsid bad8780e-5dc9-11ef-bccf-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-08-19T01:36:10.493 INFO:tasks.workunit.client.0.smithi083.stdout:Spawning 3 writers... 2024-08-19T01:36:10.493 INFO:tasks.workunit.client.0.smithi083.stdout:Spawning 1 rewriters... 2024-08-19T01:36:10.493 INFO:tasks.workunit.client.0.smithi083.stdout:Spawning 5 commenters... 2024-08-19T01:36:10.494 INFO:tasks.workunit.client.0.smithi083.stdout:Spawning 100 readers... 2024-08-19T01:36:10.501 INFO:tasks.workunit.client.0.smithi083.stdout:Benchmarking for 30 iterations. 2024-08-19T01:36:10.501 INFO:tasks.workunit.client.0.smithi083.stdout:The test will run during 5 minutes. 2024-08-19T01:36:10.501 INFO:tasks.workunit.client.0.smithi083.stdout: 2024-08-19T01:36:10.501 INFO:tasks.workunit.client.0.smithi083.stdout: Nb blogs R articles W articles R pictures W pictures R comments W comments 2024-08-19T01:36:10.775 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/bad8780e-5dc9-11ef-bccf-c7b262605968/mon.a/config 2024-08-19T01:36:11.370 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:11.348+0000 7f1cc7200640 1 -- 172.21.15.83:0/2589166618 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1cc0103ac0 msgr2=0x7f1cc0103f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:36:11.370 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:11.348+0000 7f1cc7200640 1 --2- 172.21.15.83:0/2589166618 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1cc0103ac0 0x7f1cc0103f20 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f1cac009890 tx=0x7f1cac02a360 comp rx=0 tx=0).stop 2024-08-19T01:36:11.370 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:11.348+0000 7f1cc7200640 1 -- 172.21.15.83:0/2589166618 shutdown_connections 2024-08-19T01:36:11.370 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:11.348+0000 7f1cc7200640 1 --2- 172.21.15.83:0/2589166618 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1cc0104460 0x7f1cc0110990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:11.370 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:11.348+0000 7f1cc7200640 1 --2- 172.21.15.83:0/2589166618 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1cc0103ac0 0x7f1cc0103f20 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:11.370 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:11.348+0000 7f1cc7200640 1 --2- 172.21.15.83:0/2589166618 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1cc0109ac0 0x7f1cc0109ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:11.370 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:11.348+0000 7f1cc7200640 1 -- 172.21.15.83:0/2589166618 >> 172.21.15.83:0/2589166618 conn(0x7f1cc00ff7c0 msgr2=0x7f1cc0101be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:36:11.370 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:11.348+0000 7f1cc7200640 1 -- 172.21.15.83:0/2589166618 shutdown_connections 2024-08-19T01:36:11.370 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:11.348+0000 7f1cc7200640 1 -- 172.21.15.83:0/2589166618 wait complete. 2024-08-19T01:36:11.370 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:11.349+0000 7f1cc7200640 1 Processor -- start 2024-08-19T01:36:11.371 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:11.349+0000 7f1cc7200640 1 -- start start 2024-08-19T01:36:11.371 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:11.350+0000 7f1cc7200640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1cc0103ac0 0x7f1cc019a380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:36:11.371 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:11.350+0000 7f1cc7200640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1cc0104460 0x7f1cc019a8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:36:11.371 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:11.350+0000 7f1cc7200640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1cc0109ac0 0x7f1cc019ec50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:36:11.371 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:11.350+0000 7f1cc7200640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f1cc0113e80 con 0x7f1cc0104460 2024-08-19T01:36:11.371 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:11.350+0000 7f1cc7200640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f1cc0113d00 con 0x7f1cc0103ac0 2024-08-19T01:36:11.371 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:11.350+0000 7f1cc7200640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f1cc0114000 con 0x7f1cc0109ac0 2024-08-19T01:36:11.371 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:11.350+0000 7f1cbfe00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1cc0103ac0 0x7f1cc019a380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:36:11.371 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:11.350+0000 7f1cbfe00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1cc0103ac0 0x7f1cc019a380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.83:53196/0 (socket says 172.21.15.83:53196) 2024-08-19T01:36:11.371 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:11.350+0000 7f1cbfe00640 1 -- 172.21.15.83:0/883108940 learned_addr learned my addr 172.21.15.83:0/883108940 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-08-19T01:36:11.371 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:11.350+0000 7f1cbf400640 1 --2- 172.21.15.83:0/883108940 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1cc0104460 0x7f1cc019a8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:36:11.371 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:11.350+0000 7f1cc5200640 1 --2- 172.21.15.83:0/883108940 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1cc0109ac0 0x7f1cc019ec50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:36:11.371 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:11.350+0000 7f1cbfe00640 1 -- 172.21.15.83:0/883108940 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1cc0109ac0 msgr2=0x7f1cc019ec50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:36:11.372 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:11.351+0000 7f1cbfe00640 1 --2- 172.21.15.83:0/883108940 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1cc0109ac0 0x7f1cc019ec50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:11.372 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:11.351+0000 7f1cbfe00640 1 -- 172.21.15.83:0/883108940 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1cc0104460 msgr2=0x7f1cc019a8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:36:11.372 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:11.351+0000 7f1cbfe00640 1 --2- 172.21.15.83:0/883108940 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1cc0104460 0x7f1cc019a8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:11.372 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:11.351+0000 7f1cbfe00640 1 -- 172.21.15.83:0/883108940 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1cc019f330 con 0x7f1cc0103ac0 2024-08-19T01:36:11.372 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:11.351+0000 7f1cbfe00640 1 --2- 172.21.15.83:0/883108940 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1cc0103ac0 0x7f1cc019a380 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f1cb000e9c0 tx=0x7f1cb000ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:36:11.372 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:11.351+0000 7f1cbcc00640 1 -- 172.21.15.83:0/883108940 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1cb000cde0 con 0x7f1cc0103ac0 2024-08-19T01:36:11.372 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:11.351+0000 7f1cbcc00640 1 -- 172.21.15.83:0/883108940 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1cb00035f0 con 0x7f1cc0103ac0 2024-08-19T01:36:11.372 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:11.351+0000 7f1cc7200640 1 -- 172.21.15.83:0/883108940 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1cc019f620 con 0x7f1cc0103ac0 2024-08-19T01:36:11.372 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:11.351+0000 7f1cbcc00640 1 -- 172.21.15.83:0/883108940 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1cb0010690 con 0x7f1cc0103ac0 2024-08-19T01:36:11.372 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:11.351+0000 7f1cc7200640 1 -- 172.21.15.83:0/883108940 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1cc01a6f00 con 0x7f1cc0103ac0 2024-08-19T01:36:11.372 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:11.354+0000 7f1cc9dd9740 1 Processor -- start 2024-08-19T01:36:11.372 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:11.355+0000 7f1cc9dd9740 1 -- start start 2024-08-19T01:36:11.372 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:11.355+0000 7f1cbcc00640 1 -- 172.21.15.83:0/883108940 <== mon.2 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7f1cb00037a0 con 0x7f1cc0103ac0 2024-08-19T01:36:11.372 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:11.355+0000 7f1cc9dd9740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x555fad843b90 0x555fad843f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:36:11.372 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:11.355+0000 7f1cc9dd9740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555fad8444b0 0x555fad84e830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:36:11.373 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:11.355+0000 7f1cc9dd9740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555fad84ed70 0x555fad851160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:36:11.373 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:11.355+0000 7f1cc9dd9740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x555fad7a9bf0 con 0x555fad84ed70 2024-08-19T01:36:11.373 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:11.355+0000 7f1cc9dd9740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x555fad741bd0 con 0x555fad8444b0 2024-08-19T01:36:11.373 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:11.355+0000 7f1cc9dd9740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x555fad7a5bb0 con 0x555fad843b90 2024-08-19T01:36:11.373 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:11.355+0000 7f1cbfe00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555fad8444b0 0x555fad84e830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:36:11.373 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:11.355+0000 7f1cbfe00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555fad8444b0 0x555fad84e830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.83:53208/0 (socket says 172.21.15.83:53208) 2024-08-19T01:36:11.373 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:11.355+0000 7f1cbfe00640 1 -- 172.21.15.83:0/3319515750 learned_addr learned my addr 172.21.15.83:0/3319515750 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-08-19T01:36:11.373 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:11.355+0000 7f1cc5200640 1 --2- 172.21.15.83:0/3319515750 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x555fad843b90 0x555fad843f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:36:11.373 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:11.356+0000 7f1cbf400640 1 --2- 172.21.15.83:0/3319515750 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555fad84ed70 0x555fad851160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:36:11.373 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:11.356+0000 7f1cbfe00640 1 -- 172.21.15.83:0/3319515750 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x555fad843b90 msgr2=0x555fad843f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:36:11.373 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:11.356+0000 7f1cbfe00640 1 --2- 172.21.15.83:0/3319515750 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x555fad843b90 0x555fad843f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:11.373 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:11.356+0000 7f1cbfe00640 1 -- 172.21.15.83:0/3319515750 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555fad84ed70 msgr2=0x555fad851160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:36:11.374 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:11.356+0000 7f1cbfe00640 1 --2- 172.21.15.83:0/3319515750 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555fad84ed70 0x555fad851160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:11.374 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:11.356+0000 7f1cbfe00640 1 -- 172.21.15.83:0/3319515750 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555fad7488a0 con 0x555fad8444b0 2024-08-19T01:36:11.374 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:11.356+0000 7f1cbfe00640 1 --2- 172.21.15.83:0/3319515750 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555fad8444b0 0x555fad84e830 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f1cb00d2fd0 tx=0x7f1cb00d7780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=efd444a5f2809220 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:36:11.374 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:11.356+0000 7f1cb5600640 1 -- 172.21.15.83:0/3319515750 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1cb00e2070 con 0x555fad8444b0 2024-08-19T01:36:11.374 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:11.356+0000 7f1cb5600640 1 -- 172.21.15.83:0/3319515750 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1cb00d14e0 con 0x555fad8444b0 2024-08-19T01:36:11.374 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:11.356+0000 7f1cb5600640 1 -- 172.21.15.83:0/3319515750 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1cb00ddab0 con 0x555fad8444b0 2024-08-19T01:36:11.374 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:11.357+0000 7f1cbcc00640 1 --2- 172.21.15.83:0/883108940 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f1ca40a2c00 0x7f1ca40a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:36:11.374 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:11.357+0000 7f1cbcc00640 1 -- 172.21.15.83:0/883108940 <== mon.2 v2:172.21.15.117:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8269+0+0 (secure 0 0 0) 0x7f1cb00cf260 con 0x7f1cc0103ac0 2024-08-19T01:36:11.374 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:11.357+0000 7f1cc5200640 1 --2- 172.21.15.83:0/883108940 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f1ca40a2c00 0x7f1ca40a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:36:11.374 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:11.357+0000 7f1cc9dd9740 1 -- 172.21.15.83:0/3319515750 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555fad8444b0 msgr2=0x555fad84e830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:36:11.374 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:11.357+0000 7f1cc9dd9740 1 --2- 172.21.15.83:0/3319515750 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555fad8444b0 0x555fad84e830 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f1cb00d2fd0 tx=0x7f1cb00d7780 comp rx=0 tx=0).stop 2024-08-19T01:36:11.374 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:11.357+0000 7f1cc9dd9740 1 -- 172.21.15.83:0/3319515750 shutdown_connections 2024-08-19T01:36:11.374 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:11.357+0000 7f1cc9dd9740 1 --2- 172.21.15.83:0/3319515750 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555fad84ed70 0x555fad851160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:11.374 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:11.357+0000 7f1cc9dd9740 1 --2- 172.21.15.83:0/3319515750 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555fad8444b0 0x555fad84e830 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:11.375 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:11.358+0000 7f1cc9dd9740 1 --2- 172.21.15.83:0/3319515750 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x555fad843b90 0x555fad843f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:11.375 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:11.358+0000 7f1cc9dd9740 1 -- 172.21.15.83:0/3319515750 >> 172.21.15.83:0/3319515750 conn(0x555fad840120 msgr2=0x555fad840500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:36:11.375 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:11.358+0000 7f1cc9dd9740 1 -- 172.21.15.83:0/3319515750 shutdown_connections 2024-08-19T01:36:11.375 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:11.358+0000 7f1cc5200640 1 --2- 172.21.15.83:0/883108940 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f1ca40a2c00 0x7f1ca40a50c0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f1cb8005fd0 tx=0x7f1cb8009600 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:36:11.375 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:11.358+0000 7f1cc9dd9740 1 -- 172.21.15.83:0/3319515750 wait complete. 2024-08-19T01:36:11.375 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:11.358+0000 7f1cc9dd9740 1 -- start start 2024-08-19T01:36:11.375 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:11.358+0000 7f1cc9dd9740 1 Processor -- start 2024-08-19T01:36:11.375 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:11.359+0000 7f1cc9dd9740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555fad843b90 0x555fad84bc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:36:11.375 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:11.359+0000 7f1cc9dd9740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555fad8444b0 0x555fad84c170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:36:11.375 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:11.359+0000 7f1cc9dd9740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x555fad84ed70 0x555fad84c6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:36:11.375 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:11.359+0000 7f1cc9dd9740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x555fad851990 con 0x555fad8444b0 2024-08-19T01:36:11.375 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:11.359+0000 7f1cc9dd9740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x555fad764d40 con 0x555fad843b90 2024-08-19T01:36:11.375 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:11.359+0000 7f1cc9dd9740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x555fad851b10 con 0x555fad84ed70 2024-08-19T01:36:11.375 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:11.359+0000 7f1cc9dd9740 10 client.0 _ll_register_callbacks cb 0x555fad72db10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-19T01:36:11.376 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:11.359+0000 7f1cbfe00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555fad843b90 0x555fad84bc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:36:11.376 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:11.359+0000 7f1cbfe00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555fad843b90 0x555fad84bc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.83:53224/0 (socket says 172.21.15.83:53224) 2024-08-19T01:36:11.376 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:11.359+0000 7f1cbfe00640 1 -- 172.21.15.83:0/3266322875 learned_addr learned my addr 172.21.15.83:0/3266322875 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-08-19T01:36:11.376 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:11.359+0000 7f1cbf400640 1 --2- 172.21.15.83:0/3266322875 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555fad8444b0 0x555fad84c170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:36:11.376 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:11.359+0000 7f1cc5200640 1 --2- 172.21.15.83:0/3266322875 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x555fad84ed70 0x555fad84c6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:36:11.376 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:11.359+0000 7f1cbfe00640 1 -- 172.21.15.83:0/3266322875 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x555fad84ed70 msgr2=0x555fad84c6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:36:11.376 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:11.359+0000 7f1cbfe00640 1 --2- 172.21.15.83:0/3266322875 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x555fad84ed70 0x555fad84c6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:11.376 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:11.359+0000 7f1cbfe00640 1 -- 172.21.15.83:0/3266322875 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555fad8444b0 msgr2=0x555fad84c170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:36:11.376 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:11.359+0000 7f1cbfe00640 1 --2- 172.21.15.83:0/3266322875 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555fad8444b0 0x555fad84c170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:11.376 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:11.359+0000 7f1cbfe00640 1 -- 172.21.15.83:0/3266322875 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555fad51b3c0 con 0x555fad843b90 2024-08-19T01:36:11.376 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:11.360+0000 7f1cbfe00640 1 --2- 172.21.15.83:0/3266322875 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555fad843b90 0x555fad84bc30 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f1cb00ccd90 tx=0x7f1cb0003e10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=183f6c4876bec0fb server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:36:11.376 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:11.360+0000 7f1caa000640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-08-19T01:36:11.376 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:11.360+0000 7f1caa000640 1 -- 172.21.15.83:0/3266322875 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1cb0097d40 con 0x555fad843b90 2024-08-19T01:36:11.377 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:11.360+0000 7f1caa000640 1 -- 172.21.15.83:0/3266322875 <== mon.2 v2:172.21.15.117:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8269+0+0 (secure 0 0 0) 0x7f1cb0099da0 con 0x555fad843b90 2024-08-19T01:36:11.377 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:11.360+0000 7f1cc9dd9740 1 -- 172.21.15.83:0/3266322875 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1cb00d14e0 con 0x555fad843b90 2024-08-19T01:36:11.377 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:11.360+0000 7f1caa000640 1 -- 172.21.15.83:0/3266322875 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f1cb009b400 con 0x555fad843b90 2024-08-19T01:36:11.377 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:11.360+0000 7f1cc9dd9740 10 client.25385 fetch_fsmap learned FSMap version 25 2024-08-19T01:36:11.377 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:11.360+0000 7f1cc9dd9740 1 -- 172.21.15.83:0/3266322875 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x555fad851990 con 0x555fad843b90 2024-08-19T01:36:11.377 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:11.360+0000 7f1caa000640 1 -- 172.21.15.83:0/3266322875 <== mon.2 v2:172.21.15.117:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1cb00d14e0 con 0x555fad843b90 2024-08-19T01:36:11.377 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:11.361+0000 7f1caa000640 1 -- 172.21.15.83:0/3266322875 <== mon.2 v2:172.21.15.117:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f1cb00ba250 con 0x555fad843b90 2024-08-19T01:36:11.377 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:11.361+0000 7f1cc9dd9740 10 client.25385 fetch_fsmap finished waiting for FSMap version 25 2024-08-19T01:36:11.377 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:11.361+0000 7f1cc9dd9740 10 client.25385 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-19T01:36:11.377 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:11.361+0000 7f1cc9dd9740 20 client.25385 populate_metadata read hostname 'smithi083' 2024-08-19T01:36:11.377 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:11.361+0000 7f1cc9dd9740 1 --2- 172.21.15.83:0/3266322875 >> [v2:172.21.15.117:6838/1967595704,v1:172.21.15.117:6839/1967595704] conn(0x555fad85e790 0x555fad860b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:36:11.377 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:11.361+0000 7f1cc9dd9740 4 client.25385 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-19T01:36:11.377 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:11.361+0000 7f1cc9dd9740 1 -- 172.21.15.83:0/3266322875 --> [v2:172.21.15.117:6838/1967595704,v1:172.21.15.117:6839/1967595704] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555fad7b7160 con 0x555fad85e790 2024-08-19T01:36:11.378 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:11.361+0000 7f1cbf400640 1 --2- 172.21.15.83:0/3266322875 >> [v2:172.21.15.117:6838/1967595704,v1:172.21.15.117:6839/1967595704] conn(0x555fad85e790 0x555fad860b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:36:11.378 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:11.361+0000 7f1cbf400640 1 --2- 172.21.15.83:0/3266322875 >> [v2:172.21.15.117:6838/1967595704,v1:172.21.15.117:6839/1967595704] conn(0x555fad85e790 0x555fad860b80 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=28e71bb695fd1382 server_cookie=4faf2502289210e9 in_seq=0 out_seq=0 2024-08-19T01:36:11.378 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:11.361+0000 7f1caa000640 10 client.25385 ms_handle_connect on v2:172.21.15.117:6838/1967595704 2024-08-19T01:36:11.378 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:11.363+0000 7f1caa000640 1 -- 172.21.15.83:0/3266322875 <== mds.0 v2:172.21.15.117:6838/1967595704 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x555fad7b7160 con 0x555fad85e790 2024-08-19T01:36:11.378 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:11.363+0000 7f1caa000640 10 client.25385 handle_command_reply: tid=0 2024-08-19T01:36:11.378 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:11.363+0000 7f1cc9dd9740 1 client.25385 shutdown 2024-08-19T01:36:11.378 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:11.363+0000 7f1cc9dd9740 1 -- 172.21.15.83:0/3266322875 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555fad843b90 msgr2=0x555fad84bc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:36:11.378 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:11.363+0000 7f1cc9dd9740 1 --2- 172.21.15.83:0/3266322875 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555fad843b90 0x555fad84bc30 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f1cb00ccd90 tx=0x7f1cb0003e10 comp rx=0 tx=0).stop 2024-08-19T01:36:11.378 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:11.363+0000 7f1cc9dd9740 1 -- 172.21.15.83:0/3266322875 shutdown_connections 2024-08-19T01:36:11.378 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:11.363+0000 7f1cc9dd9740 1 --2- 172.21.15.83:0/3266322875 >> [v2:172.21.15.117:6838/1967595704,v1:172.21.15.117:6839/1967595704] conn(0x555fad85e790 0x555fad860b80 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:11.378 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:11.363+0000 7f1cc9dd9740 1 --2- 172.21.15.83:0/3266322875 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x555fad84ed70 0x555fad84c6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:11.378 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:11.363+0000 7f1cc9dd9740 1 --2- 172.21.15.83:0/3266322875 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555fad8444b0 0x555fad84c170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:11.378 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:11.363+0000 7f1cc9dd9740 1 --2- 172.21.15.83:0/3266322875 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555fad843b90 0x555fad84bc30 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:11.378 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:11.363+0000 7f1cc9dd9740 1 -- 172.21.15.83:0/3266322875 >> 172.21.15.83:0/3266322875 conn(0x555fad840120 msgr2=0x555fad845e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:36:11.379 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:11.363+0000 7f1caa000640 0 client.25385 ms_handle_reset on v2:172.21.15.117:6838/1967595704 2024-08-19T01:36:11.379 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:11.364+0000 7f1cc9dd9740 1 -- 172.21.15.83:0/3266322875 shutdown_connections 2024-08-19T01:36:11.379 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:11.364+0000 7f1cc9dd9740 1 -- 172.21.15.83:0/3266322875 wait complete. 2024-08-19T01:36:11.379 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:11.364+0000 7f1cc9dd9740 20 client.25385 trim_cache size 0 max 16384 2024-08-19T01:36:11.384 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:11.384+0000 7f1cc9dd9740 1 Processor -- start 2024-08-19T01:36:11.385 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:11.384+0000 7f1cc9dd9740 1 -- start start 2024-08-19T01:36:11.385 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:11.384+0000 7f1cc9dd9740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555fad843b90 0x555fad846f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:36:11.385 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:11.384+0000 7f1cc9dd9740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x555fad8444b0 0x555fad866230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:36:11.385 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:11.384+0000 7f1cc9dd9740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555fad84ed70 0x555fad866770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:36:11.386 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:11.384+0000 7f1cc9dd9740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f1cc01b1eb0 con 0x555fad843b90 2024-08-19T01:36:11.386 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:11.384+0000 7f1cc9dd9740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f1cc01af780 con 0x555fad84ed70 2024-08-19T01:36:11.386 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:11.384+0000 7f1cc9dd9740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f1cc01ad5d0 con 0x555fad8444b0 2024-08-19T01:36:11.386 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:11.385+0000 7f1cc5200640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555fad84ed70 0x555fad866770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:36:11.386 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:11.385+0000 7f1cc5200640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555fad84ed70 0x555fad866770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.83:53240/0 (socket says 172.21.15.83:53240) 2024-08-19T01:36:11.386 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:11.385+0000 7f1cc5200640 1 -- 172.21.15.83:0/2341973247 learned_addr learned my addr 172.21.15.83:0/2341973247 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-08-19T01:36:11.386 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:11.385+0000 7f1cbfe00640 1 --2- 172.21.15.83:0/2341973247 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555fad843b90 0x555fad846f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:36:11.386 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:11.385+0000 7f1cbf400640 1 --2- 172.21.15.83:0/2341973247 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x555fad8444b0 0x555fad866230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:36:11.386 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:11.385+0000 7f1cc5200640 1 -- 172.21.15.83:0/2341973247 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x555fad8444b0 msgr2=0x555fad866230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:36:11.386 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:11.385+0000 7f1cc5200640 1 --2- 172.21.15.83:0/2341973247 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x555fad8444b0 0x555fad866230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:11.386 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:11.385+0000 7f1cc5200640 1 -- 172.21.15.83:0/2341973247 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555fad843b90 msgr2=0x555fad846f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:36:11.386 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:11.385+0000 7f1cc5200640 1 --2- 172.21.15.83:0/2341973247 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555fad843b90 0x555fad846f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:11.386 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:11.385+0000 7f1cc5200640 1 -- 172.21.15.83:0/2341973247 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555fad83ff50 con 0x555fad84ed70 2024-08-19T01:36:11.388 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:11.385+0000 7f1cc5200640 1 --2- 172.21.15.83:0/2341973247 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555fad84ed70 0x555fad866770 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f1cb80098b0 tx=0x7f1cb8016000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=de0ce875cab3085e server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:36:11.388 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:11.386+0000 7f1cb4c00640 1 -- 172.21.15.83:0/2341973247 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1cb8004d20 con 0x555fad84ed70 2024-08-19T01:36:11.388 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:11.386+0000 7f1cb4c00640 1 -- 172.21.15.83:0/2341973247 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1cb8021a50 con 0x555fad84ed70 2024-08-19T01:36:11.388 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:11.386+0000 7f1cb4c00640 1 -- 172.21.15.83:0/2341973247 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1cb80056d0 con 0x555fad84ed70 2024-08-19T01:36:11.388 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:11.386+0000 7f1cc9dd9740 1 -- 172.21.15.83:0/2341973247 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555fad84ed70 msgr2=0x555fad866770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:36:11.388 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:11.386+0000 7f1cc9dd9740 1 --2- 172.21.15.83:0/2341973247 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555fad84ed70 0x555fad866770 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f1cb80098b0 tx=0x7f1cb8016000 comp rx=0 tx=0).stop 2024-08-19T01:36:11.388 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:11.386+0000 7f1cc9dd9740 1 -- 172.21.15.83:0/2341973247 shutdown_connections 2024-08-19T01:36:11.388 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:11.386+0000 7f1cc9dd9740 1 --2- 172.21.15.83:0/2341973247 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555fad84ed70 0x555fad866770 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:11.388 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:11.386+0000 7f1cc9dd9740 1 --2- 172.21.15.83:0/2341973247 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x555fad8444b0 0x555fad866230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:11.388 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:11.386+0000 7f1cc9dd9740 1 --2- 172.21.15.83:0/2341973247 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555fad843b90 0x555fad846f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:11.388 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:11.386+0000 7f1cc9dd9740 1 -- 172.21.15.83:0/2341973247 >> 172.21.15.83:0/2341973247 conn(0x555fad840120 msgr2=0x555fad84ad20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:36:11.388 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:11.387+0000 7f1cc9dd9740 1 -- 172.21.15.83:0/2341973247 shutdown_connections 2024-08-19T01:36:11.389 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:11.387+0000 7f1cc9dd9740 1 -- 172.21.15.83:0/2341973247 wait complete. 2024-08-19T01:36:11.389 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:11.387+0000 7f1cc9dd9740 1 -- start start 2024-08-19T01:36:11.389 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:11.387+0000 7f1cc9dd9740 1 Processor -- start 2024-08-19T01:36:11.391 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:11.388+0000 7f1cc9dd9740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x555fad843b90 0x555fad857450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:36:11.391 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:11.388+0000 7f1cc9dd9740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555fad8444b0 0x555fad852780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:36:11.391 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:11.388+0000 7f1cc9dd9740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555fad84ed70 0x555fad852cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:36:11.391 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:11.388+0000 7f1cc9dd9740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f1cc01adc40 con 0x555fad84ed70 2024-08-19T01:36:11.391 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:11.388+0000 7f1cc9dd9740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f1cc01af0b0 con 0x555fad8444b0 2024-08-19T01:36:11.391 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:11.388+0000 7f1cc9dd9740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f1cc01ae310 con 0x555fad843b90 2024-08-19T01:36:11.391 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:11.388+0000 7f1cc9dd9740 10 client.0 _ll_register_callbacks cb 0x7f1cb00e0070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-19T01:36:11.391 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:11.388+0000 7f1cbf400640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555fad8444b0 0x555fad852780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:36:11.391 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:11.388+0000 7f1cc5200640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555fad84ed70 0x555fad852cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:36:11.391 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:11.388+0000 7f1cbf400640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555fad8444b0 0x555fad852780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.83:53256/0 (socket says 172.21.15.83:53256) 2024-08-19T01:36:11.391 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:11.388+0000 7f1cbf400640 1 -- 172.21.15.83:0/3652428181 learned_addr learned my addr 172.21.15.83:0/3652428181 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-08-19T01:36:11.391 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:11.388+0000 7f1cbfe00640 1 --2- 172.21.15.83:0/3652428181 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x555fad843b90 0x555fad857450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:36:11.392 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:11.388+0000 7f1cbf400640 1 -- 172.21.15.83:0/3652428181 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x555fad843b90 msgr2=0x555fad857450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:36:11.392 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:11.388+0000 7f1cbf400640 1 --2- 172.21.15.83:0/3652428181 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x555fad843b90 0x555fad857450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:11.392 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:11.388+0000 7f1cbf400640 1 -- 172.21.15.83:0/3652428181 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555fad84ed70 msgr2=0x555fad852cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:36:11.392 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:11.388+0000 7f1cbf400640 1 --2- 172.21.15.83:0/3652428181 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555fad84ed70 0x555fad852cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:11.392 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:11.388+0000 7f1cbf400640 1 -- 172.21.15.83:0/3652428181 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555fad8534a0 con 0x555fad8444b0 2024-08-19T01:36:11.392 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:11.389+0000 7f1cbf400640 1 --2- 172.21.15.83:0/3652428181 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555fad8444b0 0x555fad852780 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f1cac02a880 tx=0x7f1cac0304d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=328a6dfd724d3aeb server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:36:11.392 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:11.389+0000 7f1caa000640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-08-19T01:36:11.392 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:11.389+0000 7f1caa000640 1 -- 172.21.15.83:0/3652428181 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1cac037040 con 0x555fad8444b0 2024-08-19T01:36:11.392 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:11.389+0000 7f1caa000640 1 -- 172.21.15.83:0/3652428181 <== mon.2 v2:172.21.15.117:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8269+0+0 (secure 0 0 0) 0x7f1cac004400 con 0x555fad8444b0 2024-08-19T01:36:11.392 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:11.389+0000 7f1cc9dd9740 1 -- 172.21.15.83:0/3652428181 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1cb8021a50 con 0x555fad8444b0 2024-08-19T01:36:11.392 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:11.389+0000 7f1caa000640 1 -- 172.21.15.83:0/3652428181 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f1cac030580 con 0x555fad8444b0 2024-08-19T01:36:11.392 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:11.389+0000 7f1cc9dd9740 10 client.25391 fetch_fsmap learned FSMap version 25 2024-08-19T01:36:11.392 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:11.389+0000 7f1cc9dd9740 1 -- 172.21.15.83:0/3652428181 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x555fad8c9220 con 0x555fad8444b0 2024-08-19T01:36:11.392 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:11.390+0000 7f1caa000640 1 -- 172.21.15.83:0/3652428181 <== mon.2 v2:172.21.15.117:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1cb8021a50 con 0x555fad8444b0 2024-08-19T01:36:11.393 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:11.390+0000 7f1caa000640 1 -- 172.21.15.83:0/3652428181 <== mon.2 v2:172.21.15.117:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f1cac04f440 con 0x555fad8444b0 2024-08-19T01:36:11.393 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:11.390+0000 7f1cc9dd9740 10 client.25391 fetch_fsmap finished waiting for FSMap version 25 2024-08-19T01:36:11.393 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:11.390+0000 7f1cc9dd9740 10 client.25391 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-19T01:36:11.393 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:11.390+0000 7f1cc9dd9740 20 client.25391 populate_metadata read hostname 'smithi083' 2024-08-19T01:36:11.393 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:11.390+0000 7f1cc9dd9740 1 --2- 172.21.15.83:0/3652428181 >> [v2:172.21.15.117:6838/1967595704,v1:172.21.15.117:6839/1967595704] conn(0x555fad8c9940 0x555fad8cbd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:36:11.393 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:11.390+0000 7f1cc9dd9740 4 client.25391 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-08-19T01:36:11.393 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:11.390+0000 7f1cc9dd9740 1 -- 172.21.15.83:0/3652428181 --> [v2:172.21.15.117:6838/1967595704,v1:172.21.15.117:6839/1967595704] -- command(tid 0: {"prefix": "scrub status"}) -- 0x555fad7b1670 con 0x555fad8c9940 2024-08-19T01:36:11.393 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:11.390+0000 7f1cbfe00640 1 --2- 172.21.15.83:0/3652428181 >> [v2:172.21.15.117:6838/1967595704,v1:172.21.15.117:6839/1967595704] conn(0x555fad8c9940 0x555fad8cbd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:36:11.394 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:11.391+0000 7f1cbfe00640 1 --2- 172.21.15.83:0/3652428181 >> [v2:172.21.15.117:6838/1967595704,v1:172.21.15.117:6839/1967595704] conn(0x555fad8c9940 0x555fad8cbd60 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=de863293feafcf86 server_cookie=ea66c421581598c2 in_seq=0 out_seq=0 2024-08-19T01:36:11.394 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:11.391+0000 7f1caa000640 10 client.25391 ms_handle_connect on v2:172.21.15.117:6838/1967595704 2024-08-19T01:36:11.394 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:11.392+0000 7f1caa000640 1 -- 172.21.15.83:0/3652428181 <== mds.0 v2:172.21.15.117:6838/1967595704 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x555fad7b1670 con 0x555fad8c9940 2024-08-19T01:36:11.394 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:11.392+0000 7f1caa000640 10 client.25391 handle_command_reply: tid=0 2024-08-19T01:36:11.394 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:11.392+0000 7f1cc9dd9740 1 client.25391 shutdown 2024-08-19T01:36:11.395 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:11.393+0000 7f1cc9dd9740 1 -- 172.21.15.83:0/3652428181 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555fad8444b0 msgr2=0x555fad852780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:36:11.395 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:11.393+0000 7f1cc9dd9740 1 --2- 172.21.15.83:0/3652428181 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555fad8444b0 0x555fad852780 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f1cac02a880 tx=0x7f1cac0304d0 comp rx=0 tx=0).stop 2024-08-19T01:36:11.395 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:11.393+0000 7f1cc9dd9740 1 -- 172.21.15.83:0/3652428181 shutdown_connections 2024-08-19T01:36:11.395 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:11.393+0000 7f1cc9dd9740 1 --2- 172.21.15.83:0/3652428181 >> [v2:172.21.15.117:6838/1967595704,v1:172.21.15.117:6839/1967595704] conn(0x555fad8c9940 0x555fad8cbd60 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:11.395 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:11.393+0000 7f1cc9dd9740 1 --2- 172.21.15.83:0/3652428181 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555fad84ed70 0x555fad852cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:11.395 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:11.393+0000 7f1cc9dd9740 1 --2- 172.21.15.83:0/3652428181 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555fad8444b0 0x555fad852780 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:11.395 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:11.393+0000 7f1caa000640 0 client.25391 ms_handle_reset on v2:172.21.15.117:6838/1967595704 2024-08-19T01:36:11.395 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:11.393+0000 7f1cc9dd9740 1 --2- 172.21.15.83:0/3652428181 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x555fad843b90 0x555fad857450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:11.395 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:11.393+0000 7f1cc9dd9740 1 -- 172.21.15.83:0/3652428181 >> 172.21.15.83:0/3652428181 conn(0x555fad840120 msgr2=0x555fad84bb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:36:11.395 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:11.393+0000 7f1cc9dd9740 1 -- 172.21.15.83:0/3652428181 shutdown_connections 2024-08-19T01:36:11.395 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:11.393+0000 7f1cc9dd9740 1 -- 172.21.15.83:0/3652428181 wait complete. 2024-08-19T01:36:11.395 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:11.393+0000 7f1cc9dd9740 20 client.25391 trim_cache size 0 max 16384 2024-08-19T01:36:11.396 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-08-19T01:36:11.396 INFO:teuthology.orchestra.run.smithi083.stdout: "status": "no active scrubs running", 2024-08-19T01:36:11.396 INFO:teuthology.orchestra.run.smithi083.stdout: "scrubs": {} 2024-08-19T01:36:11.396 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-08-19T01:36:11.396 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:11.394+0000 7f1cc7200640 1 -- 172.21.15.83:0/883108940 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f1ca40a2c00 msgr2=0x7f1ca40a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:36:11.397 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:11.394+0000 7f1cc7200640 1 --2- 172.21.15.83:0/883108940 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f1ca40a2c00 0x7f1ca40a50c0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f1cb8005fd0 tx=0x7f1cb8009600 comp rx=0 tx=0).stop 2024-08-19T01:36:11.397 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:11.394+0000 7f1cc7200640 1 -- 172.21.15.83:0/883108940 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1cc0103ac0 msgr2=0x7f1cc019a380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:36:11.397 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:11.394+0000 7f1cc7200640 1 --2- 172.21.15.83:0/883108940 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1cc0103ac0 0x7f1cc019a380 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f1cb000e9c0 tx=0x7f1cb000ee80 comp rx=0 tx=0).stop 2024-08-19T01:36:11.397 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:11.394+0000 7f1cc7200640 1 -- 172.21.15.83:0/883108940 shutdown_connections 2024-08-19T01:36:11.397 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:11.394+0000 7f1cc7200640 1 --2- 172.21.15.83:0/883108940 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f1ca40a2c00 0x7f1ca40a50c0 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:11.397 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:11.394+0000 7f1cc7200640 1 --2- 172.21.15.83:0/883108940 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1cc0109ac0 0x7f1cc019ec50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:11.397 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:11.394+0000 7f1cc7200640 1 --2- 172.21.15.83:0/883108940 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1cc0104460 0x7f1cc019a8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:11.397 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:11.394+0000 7f1cc7200640 1 --2- 172.21.15.83:0/883108940 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1cc0103ac0 0x7f1cc019a380 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:11.397 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:11.394+0000 7f1cc7200640 1 -- 172.21.15.83:0/883108940 >> 172.21.15.83:0/883108940 conn(0x7f1cc00ff7c0 msgr2=0x7f1cc0100010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:36:11.397 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:11.394+0000 7f1cc7200640 1 -- 172.21.15.83:0/883108940 shutdown_connections 2024-08-19T01:36:11.397 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:11.394+0000 7f1cc7200640 1 -- 172.21.15.83:0/883108940 wait complete. 2024-08-19T01:36:11.886 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:36:11 smithi083 ceph-mon[17120]: pgmap v379: 97 pgs: 97 active+clean; 4.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 227 KiB/s rd, 347 KiB/s wr, 41 op/s 2024-08-19T01:36:11.952 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:36:11 smithi117 ceph-mon[22016]: pgmap v379: 97 pgs: 97 active+clean; 4.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 227 KiB/s rd, 347 KiB/s wr, 41 op/s 2024-08-19T01:36:11.966 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:36:11 smithi167 ceph-mon[21875]: pgmap v379: 97 pgs: 97 active+clean; 4.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 227 KiB/s rd, 347 KiB/s wr, 41 op/s 2024-08-19T01:36:12.339 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-08-19T01:36:12.339 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833 shell --fsid bad8780e-5dc9-11ef-bccf-c7b262605968 -- ceph fs dump --format=json 2024-08-19T01:36:12.823 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/bad8780e-5dc9-11ef-bccf-c7b262605968/mon.a/config 2024-08-19T01:36:13.456 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:13.452+0000 7f9c9a400640 1 -- 172.21.15.83:0/710423933 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9c9406eb40 msgr2=0x7f9c941142a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:36:13.456 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:13.452+0000 7f9c9a400640 1 --2- 172.21.15.83:0/710423933 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9c9406eb40 0x7f9c941142a0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f9c7c013f60 tx=0x7f9c7c032a00 comp rx=0 tx=0).stop 2024-08-19T01:36:13.456 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:13.453+0000 7f9c9a400640 1 -- 172.21.15.83:0/710423933 shutdown_connections 2024-08-19T01:36:13.456 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:13.453+0000 7f9c9a400640 1 --2- 172.21.15.83:0/710423933 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9c941147e0 0x7f9c94116bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:13.456 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:13.453+0000 7f9c9a400640 1 --2- 172.21.15.83:0/710423933 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9c9406eb40 0x7f9c941142a0 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:13.456 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:13.453+0000 7f9c9a400640 1 --2- 172.21.15.83:0/710423933 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9c9406e220 0x7f9c9406e600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:13.456 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:13.453+0000 7f9c9a400640 1 -- 172.21.15.83:0/710423933 >> 172.21.15.83:0/710423933 conn(0x7f9c94072eb0 msgr2=0x7f9c940732c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:36:13.456 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:13.453+0000 7f9c9a400640 1 -- 172.21.15.83:0/710423933 shutdown_connections 2024-08-19T01:36:13.456 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:13.453+0000 7f9c9a400640 1 -- 172.21.15.83:0/710423933 wait complete. 2024-08-19T01:36:13.456 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:13.455+0000 7f9c9a400640 1 Processor -- start 2024-08-19T01:36:13.457 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:13.455+0000 7f9c9a400640 1 -- start start 2024-08-19T01:36:13.457 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:13.455+0000 7f9c9a400640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9c9406e220 0x7f9c941af420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:36:13.457 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:13.455+0000 7f9c9a400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9c9406eb40 0x7f9c941af960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:36:13.457 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:13.455+0000 7f9c9a400640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9c941147e0 0x7f9c941b3cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:36:13.457 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:13.456+0000 7f9c93e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9c9406eb40 0x7f9c941af960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:36:13.458 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:13.456+0000 7f9c99a00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9c941147e0 0x7f9c941b3cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:36:13.458 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:13.456+0000 7f9c99a00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9c941147e0 0x7f9c941b3cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.83:53274/0 (socket says 172.21.15.83:53274) 2024-08-19T01:36:13.458 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:13.456+0000 7f9c99a00640 1 -- 172.21.15.83:0/4271516355 learned_addr learned my addr 172.21.15.83:0/4271516355 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-08-19T01:36:13.458 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:13.456+0000 7f9c9a400640 1 -- 172.21.15.83:0/4271516355 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f9c94118150 con 0x7f9c9406eb40 2024-08-19T01:36:13.458 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:13.456+0000 7f9c9a400640 1 -- 172.21.15.83:0/4271516355 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f9c94117fd0 con 0x7f9c941147e0 2024-08-19T01:36:13.458 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:13.456+0000 7f9c9a400640 1 -- 172.21.15.83:0/4271516355 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f9c941182d0 con 0x7f9c9406e220 2024-08-19T01:36:13.458 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:13.456+0000 7f9c99a00640 1 -- 172.21.15.83:0/4271516355 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9c9406e220 msgr2=0x7f9c941af420 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-08-19T01:36:13.458 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:13.457+0000 7f9c99a00640 1 --2- 172.21.15.83:0/4271516355 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9c9406e220 0x7f9c941af420 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:13.459 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:13.457+0000 7f9c99a00640 1 -- 172.21.15.83:0/4271516355 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9c9406eb40 msgr2=0x7f9c941af960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:36:13.459 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:13.457+0000 7f9c99a00640 1 --2- 172.21.15.83:0/4271516355 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9c9406eb40 0x7f9c941af960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:13.459 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:13.457+0000 7f9c99a00640 1 -- 172.21.15.83:0/4271516355 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9c941b43d0 con 0x7f9c941147e0 2024-08-19T01:36:13.459 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:13.457+0000 7f9c93e00640 1 --2- 172.21.15.83:0/4271516355 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9c9406eb40 0x7f9c941af960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-19T01:36:13.459 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:13.457+0000 7f9c99a00640 1 --2- 172.21.15.83:0/4271516355 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9c941147e0 0x7f9c941b3cf0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f9c8400b410 tx=0x7f9c8400b8d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:36:13.459 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:13.457+0000 7f9c91600640 1 -- 172.21.15.83:0/4271516355 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c84015020 con 0x7f9c941147e0 2024-08-19T01:36:13.459 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:13.457+0000 7f9c9a400640 1 -- 172.21.15.83:0/4271516355 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9c941b46c0 con 0x7f9c941147e0 2024-08-19T01:36:13.459 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:13.458+0000 7f9c9a400640 1 -- 172.21.15.83:0/4271516355 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9c941bbfa0 con 0x7f9c941147e0 2024-08-19T01:36:13.459 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:13.458+0000 7f9c91600640 1 -- 172.21.15.83:0/4271516355 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9c84011d10 con 0x7f9c941147e0 2024-08-19T01:36:13.460 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:13.458+0000 7f9c91600640 1 -- 172.21.15.83:0/4271516355 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c84010e40 con 0x7f9c941147e0 2024-08-19T01:36:13.460 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:13.460+0000 7f9c9a400640 1 -- 172.21.15.83:0/4271516355 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9c70005290 con 0x7f9c941147e0 2024-08-19T01:36:13.463 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:13.463+0000 7f9c91600640 1 -- 172.21.15.83:0/4271516355 <== mon.2 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7f9c84018020 con 0x7f9c941147e0 2024-08-19T01:36:13.465 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:13.465+0000 7f9c91600640 1 --2- 172.21.15.83:0/4271516355 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f9c780a2c00 0x7f9c780a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:36:13.466 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:13.465+0000 7f9c91600640 1 -- 172.21.15.83:0/4271516355 <== mon.2 v2:172.21.15.117:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8269+0+0 (secure 0 0 0) 0x7f9c84099150 con 0x7f9c941147e0 2024-08-19T01:36:13.467 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:13.466+0000 7f9c99000640 1 --2- 172.21.15.83:0/4271516355 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f9c780a2c00 0x7f9c780a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:36:13.467 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:13.467+0000 7f9c99000640 1 --2- 172.21.15.83:0/4271516355 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f9c780a2c00 0x7f9c780a50c0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f9c8c007c10 tx=0x7f9c8c0075f0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:36:13.471 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:13.470+0000 7f9c91600640 1 -- 172.21.15.83:0/4271516355 <== mon.2 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7f9c8401f020 con 0x7f9c941147e0 2024-08-19T01:36:13.655 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:36:13 smithi083 ceph-mon[17120]: pgmap v380: 97 pgs: 97 active+clean; 4.4 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 228 KiB/s rd, 352 KiB/s wr, 50 op/s 2024-08-19T01:36:13.766 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:13.765+0000 7f9c9a400640 1 -- 172.21.15.83:0/4271516355 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f9c70005850 con 0x7f9c941147e0 2024-08-19T01:36:13.767 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:13.766+0000 7f9c91600640 1 -- 172.21.15.83:0/4271516355 <== mon.2 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+11920 (secure 0 0 0) 0x7f9c8409b7b0 con 0x7f9c941147e0 2024-08-19T01:36:13.768 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-08-19T01:36:13.768 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":25,"btime":"2024-08-19T01:34:41:456953+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14421,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6835/988556067","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":988556067},{"type":"v1","addr":"172.21.15.83:6835","nonce":988556067}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14433,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6837/869702632","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6836","nonce":869702632},{"type":"v1","addr":"172.21.15.83:6837","nonce":869702632}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14475,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6841/1795975999","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6840","nonce":1795975999},{"type":"v1","addr":"172.21.15.83:6841","nonce":1795975999}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24430,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6833/2505076046","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6832","nonce":2505076046},{"type":"v1","addr":"172.21.15.167:6833","nonce":2505076046}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24449,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6833/3557520478","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":3557520478},{"type":"v1","addr":"172.21.15.117:6833","nonce":3557520478}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24466,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6835/3482045966","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":3482045966},{"type":"v1","addr":"172.21.15.167:6835","nonce":3482045966}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7}],"filesystems":[{"mdsmap":{"epoch":25,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-08-19T01:32:42.054769+0000","modified":"2024-08-19T01:34:40.528901+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24545,"mds_1":24527,"mds_2":24509},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24478":{"gid":24478,"name":"c","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.167:6837/1762130408","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6836","nonce":1762130408},{"type":"v1","addr":"172.21.15.167:6837","nonce":1762130408}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24485":{"gid":24485,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.117:6835/1024213994","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":1024213994},{"type":"v1","addr":"172.21.15.117:6835","nonce":1024213994}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24496":{"gid":24496,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.167:6839/3507387682","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6838","nonce":3507387682},{"type":"v1","addr":"172.21.15.167:6839","nonce":3507387682}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24509":{"gid":24509,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":10,"addr":"172.21.15.117:6837/3185224684","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":3185224684},{"type":"v1","addr":"172.21.15.117:6837","nonce":3185224684}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24527":{"gid":24527,"name":"g","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.83:6839/3239261845","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6838","nonce":3239261845},{"type":"v1","addr":"172.21.15.83:6839","nonce":3239261845}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24545":{"gid":24545,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.117:6839/1967595704","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6838","nonce":1967595704},{"type":"v1","addr":"172.21.15.117:6839","nonce":1967595704}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24545,"qdb_cluster":[24545,24527,24509]},"id":1}]} 2024-08-19T01:36:13.768 INFO:teuthology.orchestra.run.smithi083.stderr:dumped fsmap epoch 25 2024-08-19T01:36:13.770 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:13.768+0000 7f9c9a400640 1 -- 172.21.15.83:0/4271516355 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f9c780a2c00 msgr2=0x7f9c780a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:36:13.770 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:13.768+0000 7f9c9a400640 1 --2- 172.21.15.83:0/4271516355 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f9c780a2c00 0x7f9c780a50c0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f9c8c007c10 tx=0x7f9c8c0075f0 comp rx=0 tx=0).stop 2024-08-19T01:36:13.770 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:13.768+0000 7f9c9a400640 1 -- 172.21.15.83:0/4271516355 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9c941147e0 msgr2=0x7f9c941b3cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:36:13.770 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:13.768+0000 7f9c9a400640 1 --2- 172.21.15.83:0/4271516355 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9c941147e0 0x7f9c941b3cf0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f9c8400b410 tx=0x7f9c8400b8d0 comp rx=0 tx=0).stop 2024-08-19T01:36:13.770 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:13.768+0000 7f9c9a400640 1 -- 172.21.15.83:0/4271516355 shutdown_connections 2024-08-19T01:36:13.770 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:13.768+0000 7f9c9a400640 1 --2- 172.21.15.83:0/4271516355 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f9c780a2c00 0x7f9c780a50c0 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:13.770 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:13.768+0000 7f9c9a400640 1 --2- 172.21.15.83:0/4271516355 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9c941147e0 0x7f9c941b3cf0 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:13.770 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:13.768+0000 7f9c9a400640 1 --2- 172.21.15.83:0/4271516355 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9c9406eb40 0x7f9c941af960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:13.770 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:13.768+0000 7f9c9a400640 1 --2- 172.21.15.83:0/4271516355 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9c9406e220 0x7f9c941af420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:13.771 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:13.768+0000 7f9c9a400640 1 -- 172.21.15.83:0/4271516355 >> 172.21.15.83:0/4271516355 conn(0x7f9c94072eb0 msgr2=0x7f9c9406b200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:36:13.771 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:13.768+0000 7f9c9a400640 1 -- 172.21.15.83:0/4271516355 shutdown_connections 2024-08-19T01:36:13.771 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:13.768+0000 7f9c9a400640 1 -- 172.21.15.83:0/4271516355 wait complete. 2024-08-19T01:36:13.952 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:36:13 smithi117 ceph-mon[22016]: pgmap v380: 97 pgs: 97 active+clean; 4.4 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 228 KiB/s rd, 352 KiB/s wr, 50 op/s 2024-08-19T01:36:13.965 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:36:13 smithi167 ceph-mon[21875]: pgmap v380: 97 pgs: 97 active+clean; 4.4 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 228 KiB/s rd, 352 KiB/s wr, 50 op/s 2024-08-19T01:36:14.556 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833 shell --fsid bad8780e-5dc9-11ef-bccf-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-08-19T01:36:14.912 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:36:14 smithi083 ceph-mon[17120]: from='client.? 172.21.15.83:0/4271516355' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-19T01:36:14.952 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:36:14 smithi117 ceph-mon[22016]: from='client.? 172.21.15.83:0/4271516355' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-19T01:36:14.966 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:36:14 smithi167 ceph-mon[21875]: from='client.? 172.21.15.83:0/4271516355' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-19T01:36:15.209 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/bad8780e-5dc9-11ef-bccf-c7b262605968/mon.a/config 2024-08-19T01:36:15.601 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:36:15 smithi083 ceph-mon[17120]: pgmap v381: 97 pgs: 97 active+clean; 9.5 MiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 355 KiB/s rd, 846 KiB/s wr, 93 op/s 2024-08-19T01:36:15.948 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:15.944+0000 7f8d3e400640 1 -- 172.21.15.83:0/1105482993 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8d38109560 msgr2=0x7f8d381099c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:36:15.949 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:15.944+0000 7f8d3e400640 1 --2- 172.21.15.83:0/1105482993 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8d38109560 0x7f8d381099c0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f8d20009860 tx=0x7f8d2002a380 comp rx=0 tx=0).stop 2024-08-19T01:36:15.949 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:15.944+0000 7f8d3e400640 1 -- 172.21.15.83:0/1105482993 shutdown_connections 2024-08-19T01:36:15.949 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:15.944+0000 7f8d3e400640 1 --2- 172.21.15.83:0/1105482993 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8d3811ae70 0x7f8d3811d260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:15.949 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:15.944+0000 7f8d3e400640 1 --2- 172.21.15.83:0/1105482993 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8d38109560 0x7f8d381099c0 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:15.949 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:15.944+0000 7f8d3e400640 1 --2- 172.21.15.83:0/1105482993 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8d38109180 0x7f8d3810ff80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:15.949 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:15.944+0000 7f8d3e400640 1 -- 172.21.15.83:0/1105482993 >> 172.21.15.83:0/1105482993 conn(0x7f8d3806de70 msgr2=0x7f8d3806e280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:36:15.949 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:15.944+0000 7f8d3e400640 1 -- 172.21.15.83:0/1105482993 shutdown_connections 2024-08-19T01:36:15.949 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:15.944+0000 7f8d3e400640 1 -- 172.21.15.83:0/1105482993 wait complete. 2024-08-19T01:36:15.951 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:15.944+0000 7f8d3e400640 1 Processor -- start 2024-08-19T01:36:15.951 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:15.944+0000 7f8d3e400640 1 -- start start 2024-08-19T01:36:15.951 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:15.944+0000 7f8d3e400640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8d38109180 0x7f8d381a7010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:36:15.951 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:15.944+0000 7f8d3e400640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8d38109560 0x7f8d381a7550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:36:15.951 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:15.944+0000 7f8d3e400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8d3811ae70 0x7f8d381ab8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:36:15.951 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:15.944+0000 7f8d3e400640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f8d38120af0 con 0x7f8d3811ae70 2024-08-19T01:36:15.951 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:15.944+0000 7f8d3e400640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f8d38120970 con 0x7f8d38109560 2024-08-19T01:36:15.951 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:15.944+0000 7f8d3e400640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f8d38120c70 con 0x7f8d38109180 2024-08-19T01:36:15.952 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:36:15 smithi117 ceph-mon[22016]: pgmap v381: 97 pgs: 97 active+clean; 9.5 MiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 355 KiB/s rd, 846 KiB/s wr, 93 op/s 2024-08-19T01:36:15.953 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:15.950+0000 7f8d37e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8d3811ae70 0x7f8d381ab8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:36:15.953 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:15.950+0000 7f8d37e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8d3811ae70 0x7f8d381ab8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:40194/0 (socket says 172.21.15.83:40194) 2024-08-19T01:36:15.953 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:15.950+0000 7f8d37e00640 1 -- 172.21.15.83:0/609940065 learned_addr learned my addr 172.21.15.83:0/609940065 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-08-19T01:36:15.954 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:15.950+0000 7f8d37e00640 1 -- 172.21.15.83:0/609940065 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8d38109180 msgr2=0x7f8d381a7010 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-08-19T01:36:15.954 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:15.950+0000 7f8d37e00640 1 --2- 172.21.15.83:0/609940065 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8d38109180 0x7f8d381a7010 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:15.954 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:15.950+0000 7f8d37e00640 1 -- 172.21.15.83:0/609940065 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8d38109560 msgr2=0x7f8d381a7550 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-08-19T01:36:15.955 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:15.950+0000 7f8d37e00640 1 --2- 172.21.15.83:0/609940065 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8d38109560 0x7f8d381a7550 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:15.955 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:15.950+0000 7f8d37e00640 1 -- 172.21.15.83:0/609940065 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8d381abfc0 con 0x7f8d3811ae70 2024-08-19T01:36:15.958 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:15.956+0000 7f8d37e00640 1 --2- 172.21.15.83:0/609940065 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8d3811ae70 0x7f8d381ab8e0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f8d3000caf0 tx=0x7f8d3000cef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:36:15.959 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:15.957+0000 7f8d34c00640 1 -- 172.21.15.83:0/609940065 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d30016030 con 0x7f8d3811ae70 2024-08-19T01:36:15.959 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:15.957+0000 7f8d3e400640 1 -- 172.21.15.83:0/609940065 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8d381ac2b0 con 0x7f8d3811ae70 2024-08-19T01:36:15.959 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:15.957+0000 7f8d34c00640 1 -- 172.21.15.83:0/609940065 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8d30003970 con 0x7f8d3811ae70 2024-08-19T01:36:15.960 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:15.957+0000 7f8d34c00640 1 -- 172.21.15.83:0/609940065 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d30004650 con 0x7f8d3811ae70 2024-08-19T01:36:15.960 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:15.957+0000 7f8d3e400640 1 -- 172.21.15.83:0/609940065 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8d381b3b90 con 0x7f8d3811ae70 2024-08-19T01:36:15.965 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:36:15 smithi167 ceph-mon[21875]: pgmap v381: 97 pgs: 97 active+clean; 9.5 MiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 355 KiB/s rd, 846 KiB/s wr, 93 op/s 2024-08-19T01:36:15.969 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:15.963+0000 7f8d34c00640 1 -- 172.21.15.83:0/609940065 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7f8d30019070 con 0x7f8d3811ae70 2024-08-19T01:36:15.969 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:15.963+0000 7f8d34c00640 1 --2- 172.21.15.83:0/609940065 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f8d1c0a2d80 0x7f8d1c0a5240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:36:15.969 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:15.963+0000 7f8d40e31740 1 Processor -- start 2024-08-19T01:36:15.969 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:15.963+0000 7f8d34c00640 1 -- 172.21.15.83:0/609940065 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8269+0+0 (secure 0 0 0) 0x7f8d300043f0 con 0x7f8d3811ae70 2024-08-19T01:36:15.969 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:15.963+0000 7f8d37400640 1 --2- 172.21.15.83:0/609940065 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f8d1c0a2d80 0x7f8d1c0a5240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:36:15.975 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:15.972+0000 7f8d40e31740 1 -- start start 2024-08-19T01:36:15.976 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:15.972+0000 7f8d40e31740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f4e23984f0 0x55f4e23988d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:36:15.981 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:15.972+0000 7f8d40e31740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f4e2398e10 0x55f4e23a3190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:36:15.981 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:15.972+0000 7f8d40e31740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f4e23a36d0 0x55f4e23a5ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:36:15.981 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:15.972+0000 7f8d40e31740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55f4e22fdbf0 con 0x55f4e23984f0 2024-08-19T01:36:15.981 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:15.972+0000 7f8d40e31740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55f4e2295b20 con 0x55f4e2398e10 2024-08-19T01:36:15.981 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:15.972+0000 7f8d40e31740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55f4e22f9bb0 con 0x55f4e23a36d0 2024-08-19T01:36:15.981 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:15.974+0000 7f8d37400640 1 --2- 172.21.15.83:0/609940065 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f8d1c0a2d80 0x7f8d1c0a5240 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f8d280058c0 tx=0x7f8d28005850 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:36:15.982 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:15.974+0000 7f8d37400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f4e23984f0 0x55f4e23988d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:36:15.982 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:15.974+0000 7f8d37e00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f4e23a36d0 0x55f4e23a5ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:36:15.982 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:15.974+0000 7f8d37e00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f4e23a36d0 0x55f4e23a5ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.83:38722/0 (socket says 172.21.15.83:38722) 2024-08-19T01:36:15.982 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:15.974+0000 7f8d37e00640 1 -- 172.21.15.83:0/3384694358 learned_addr learned my addr 172.21.15.83:0/3384694358 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-08-19T01:36:15.982 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:15.975+0000 7f8d37e00640 1 -- 172.21.15.83:0/3384694358 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f4e2398e10 msgr2=0x55f4e23a3190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:36:15.982 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:15.975+0000 7f8d37e00640 1 --2- 172.21.15.83:0/3384694358 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f4e2398e10 0x55f4e23a3190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:15.982 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:15.975+0000 7f8d37e00640 1 -- 172.21.15.83:0/3384694358 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f4e23984f0 msgr2=0x55f4e23988d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:36:15.982 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:15.975+0000 7f8d37e00640 1 --2- 172.21.15.83:0/3384694358 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f4e23984f0 0x55f4e23988d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:15.982 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:15.975+0000 7f8d37e00640 1 -- 172.21.15.83:0/3384694358 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f4e229c7f0 con 0x55f4e23a36d0 2024-08-19T01:36:15.983 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:15.975+0000 7f8d37e00640 1 --2- 172.21.15.83:0/3384694358 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f4e23a36d0 0x55f4e23a5ac0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f8d30097ce0 tx=0x7f8d300012f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=44371ab56a99a3e3 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:36:15.983 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:15.975+0000 7f8d2cc00640 1 -- 172.21.15.83:0/3384694358 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d300a3f80 con 0x55f4e23a36d0 2024-08-19T01:36:15.983 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:15.975+0000 7f8d2cc00640 1 -- 172.21.15.83:0/3384694358 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8d30099610 con 0x55f4e23a36d0 2024-08-19T01:36:15.983 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:15.975+0000 7f8d2cc00640 1 -- 172.21.15.83:0/3384694358 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d3009a9c0 con 0x55f4e23a36d0 2024-08-19T01:36:15.983 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:15.976+0000 7f8d40e31740 1 -- 172.21.15.83:0/3384694358 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f4e23a36d0 msgr2=0x55f4e23a5ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:36:15.983 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:15.977+0000 7f8d40e31740 1 --2- 172.21.15.83:0/3384694358 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f4e23a36d0 0x55f4e23a5ac0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f8d30097ce0 tx=0x7f8d300012f0 comp rx=0 tx=0).stop 2024-08-19T01:36:15.983 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:15.977+0000 7f8d40e31740 1 -- 172.21.15.83:0/3384694358 shutdown_connections 2024-08-19T01:36:15.983 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:15.977+0000 7f8d40e31740 1 --2- 172.21.15.83:0/3384694358 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f4e23a36d0 0x55f4e23a5ac0 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:15.983 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:15.977+0000 7f8d40e31740 1 --2- 172.21.15.83:0/3384694358 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f4e2398e10 0x55f4e23a3190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:15.983 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:15.977+0000 7f8d40e31740 1 --2- 172.21.15.83:0/3384694358 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f4e23984f0 0x55f4e23988d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:15.983 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:15.977+0000 7f8d40e31740 1 -- 172.21.15.83:0/3384694358 >> 172.21.15.83:0/3384694358 conn(0x55f4e2394a80 msgr2=0x55f4e2394e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:36:15.984 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:15.981+0000 7f8d40e31740 1 -- 172.21.15.83:0/3384694358 shutdown_connections 2024-08-19T01:36:15.989 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:15.983+0000 7f8d40e31740 1 -- 172.21.15.83:0/3384694358 wait complete. 2024-08-19T01:36:15.989 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:15.984+0000 7f8d40e31740 1 -- start start 2024-08-19T01:36:15.989 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:15.985+0000 7f8d40e31740 1 Processor -- start 2024-08-19T01:36:15.990 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:15.989+0000 7f8d40e31740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f4e23984f0 0x55f4e23a07c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:36:15.991 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:15.990+0000 7f8d40e31740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f4e23a36d0 0x55f4e23a0d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:36:15.992 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:15.990+0000 7f8d37400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f4e23984f0 0x55f4e23a07c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:36:15.992 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:15.990+0000 7f8d37400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f4e23984f0 0x55f4e23a07c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:40226/0 (socket says 172.21.15.83:40226) 2024-08-19T01:36:15.992 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:15.991+0000 7f8d37400640 1 -- 172.21.15.83:0/1299228352 learned_addr learned my addr 172.21.15.83:0/1299228352 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-08-19T01:36:15.993 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:15.991+0000 7f8d40e31740 1 --2- 172.21.15.83:0/1299228352 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f4e23a1240 0x55f4e23af440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:36:15.995 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:15.991+0000 7f8d2fe00640 1 --2- 172.21.15.83:0/1299228352 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f4e23a36d0 0x55f4e23a0d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:36:15.995 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:15.993+0000 7f8d40e31740 1 -- 172.21.15.83:0/1299228352 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55f4e23a6060 con 0x55f4e23984f0 2024-08-19T01:36:15.995 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:15.993+0000 7f8d37e00640 1 --2- 172.21.15.83:0/1299228352 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f4e23a1240 0x55f4e23af440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:36:15.995 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:15.993+0000 7f8d40e31740 1 -- 172.21.15.83:0/1299228352 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55f4e22b8cf0 con 0x55f4e23a36d0 2024-08-19T01:36:15.996 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:15.993+0000 7f8d40e31740 1 -- 172.21.15.83:0/1299228352 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55f4e23a61e0 con 0x55f4e23a1240 2024-08-19T01:36:15.996 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:15.994+0000 7f8d40e31740 10 client.0 _ll_register_callbacks cb 0x55f4e22a4e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-19T01:36:15.997 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:15.995+0000 7f8d2fe00640 1 -- 172.21.15.83:0/1299228352 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f4e23a1240 msgr2=0x55f4e23af440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:36:15.997 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:15.995+0000 7f8d2fe00640 1 --2- 172.21.15.83:0/1299228352 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f4e23a1240 0x55f4e23af440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:15.997 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:15.995+0000 7f8d2fe00640 1 -- 172.21.15.83:0/1299228352 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f4e23984f0 msgr2=0x55f4e23a07c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:36:15.997 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:15.995+0000 7f8d2fe00640 1 --2- 172.21.15.83:0/1299228352 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f4e23984f0 0x55f4e23a07c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:15.997 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:15.995+0000 7f8d2fe00640 1 -- 172.21.15.83:0/1299228352 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f4e206f3c0 con 0x55f4e23a36d0 2024-08-19T01:36:15.998 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:15.996+0000 7f8d2fe00640 1 --2- 172.21.15.83:0/1299228352 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f4e23a36d0 0x55f4e23a0d00 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x55f4e2395cd0 tx=0x7f8d20004040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c1e8765b81c66057 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:36:15.998 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:15.996+0000 7f8d25600640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-08-19T01:36:15.998 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:15.996+0000 7f8d25600640 1 -- 172.21.15.83:0/1299228352 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d20037040 con 0x55f4e23a36d0 2024-08-19T01:36:16.001 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:15.999+0000 7f8d25600640 1 -- 172.21.15.83:0/1299228352 <== mon.2 v2:172.21.15.117:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8269+0+0 (secure 0 0 0) 0x7f8d20004470 con 0x55f4e23a36d0 2024-08-19T01:36:16.001 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:15.999+0000 7f8d40e31740 1 -- 172.21.15.83:0/1299228352 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8d30099610 con 0x55f4e23a36d0 2024-08-19T01:36:16.002 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:15.999+0000 7f8d25600640 1 -- 172.21.15.83:0/1299228352 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f8d200305b0 con 0x55f4e23a36d0 2024-08-19T01:36:16.002 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:15.999+0000 7f8d40e31740 10 client.25400 fetch_fsmap learned FSMap version 25 2024-08-19T01:36:16.002 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:15.999+0000 7f8d40e31740 1 -- 172.21.15.83:0/1299228352 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55f4e23b2250 con 0x55f4e23a36d0 2024-08-19T01:36:16.002 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:15.999+0000 7f8d25600640 1 -- 172.21.15.83:0/1299228352 <== mon.2 v2:172.21.15.117:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8d30099610 con 0x55f4e23a36d0 2024-08-19T01:36:16.002 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:16.000+0000 7f8d25600640 1 -- 172.21.15.83:0/1299228352 <== mon.2 v2:172.21.15.117:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f8d20043030 con 0x55f4e23a36d0 2024-08-19T01:36:16.002 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:16.000+0000 7f8d40e31740 10 client.25400 fetch_fsmap finished waiting for FSMap version 25 2024-08-19T01:36:16.002 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:16.000+0000 7f8d40e31740 10 client.25400 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-08-19T01:36:16.002 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:16.000+0000 7f8d40e31740 20 client.25400 populate_metadata read hostname 'smithi083' 2024-08-19T01:36:16.002 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:16.000+0000 7f8d40e31740 1 --2- 172.21.15.83:0/1299228352 >> [v2:172.21.15.117:6836/3185224684,v1:172.21.15.117:6837/3185224684] conn(0x55f4e23b3710 0x55f4e23b5b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:36:16.002 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:16.000+0000 7f8d40e31740 4 client.25400 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-19T01:36:16.003 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:16.000+0000 7f8d40e31740 1 -- 172.21.15.83:0/1299228352 --> [v2:172.21.15.117:6836/3185224684,v1:172.21.15.117:6837/3185224684] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f4e230b160 con 0x55f4e23b3710 2024-08-19T01:36:16.003 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:16.001+0000 7f8d37400640 1 --2- 172.21.15.83:0/1299228352 >> [v2:172.21.15.117:6836/3185224684,v1:172.21.15.117:6837/3185224684] conn(0x55f4e23b3710 0x55f4e23b5b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:36:16.007 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:16.001+0000 7f8d25600640 10 client.25400 ms_handle_connect on v2:172.21.15.117:6836/3185224684 2024-08-19T01:36:16.007 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:16.001+0000 7f8d37400640 1 --2- 172.21.15.83:0/1299228352 >> [v2:172.21.15.117:6836/3185224684,v1:172.21.15.117:6837/3185224684] conn(0x55f4e23b3710 0x55f4e23b5b00 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=fa742f1659b1f30b server_cookie=423e2c8bf6c58d5e in_seq=0 out_seq=0 2024-08-19T01:36:16.009 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:16.006+0000 7f8d25600640 1 -- 172.21.15.83:0/1299228352 <== mds.2 v2:172.21.15.117:6836/3185224684 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55f4e230b160 con 0x55f4e23b3710 2024-08-19T01:36:16.010 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:16.006+0000 7f8d25600640 10 client.25400 handle_command_reply: tid=0 2024-08-19T01:36:16.010 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:16.006+0000 7f8d40e31740 1 client.25400 shutdown 2024-08-19T01:36:16.010 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:16.007+0000 7f8d40e31740 1 -- 172.21.15.83:0/1299228352 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f4e23a36d0 msgr2=0x55f4e23a0d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:36:16.010 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:16.007+0000 7f8d40e31740 1 --2- 172.21.15.83:0/1299228352 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f4e23a36d0 0x55f4e23a0d00 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x55f4e2395cd0 tx=0x7f8d20004040 comp rx=0 tx=0).stop 2024-08-19T01:36:16.010 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:16.007+0000 7f8d40e31740 1 -- 172.21.15.83:0/1299228352 shutdown_connections 2024-08-19T01:36:16.010 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:16.007+0000 7f8d40e31740 1 --2- 172.21.15.83:0/1299228352 >> [v2:172.21.15.117:6836/3185224684,v1:172.21.15.117:6837/3185224684] conn(0x55f4e23b3710 0x55f4e23b5b00 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:16.010 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:16.007+0000 7f8d40e31740 1 --2- 172.21.15.83:0/1299228352 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f4e23a1240 0x55f4e23af440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:16.010 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:16.007+0000 7f8d40e31740 1 --2- 172.21.15.83:0/1299228352 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f4e23a36d0 0x55f4e23a0d00 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:16.010 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:16.007+0000 7f8d40e31740 1 --2- 172.21.15.83:0/1299228352 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f4e23984f0 0x55f4e23a07c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:16.010 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:16.007+0000 7f8d40e31740 1 -- 172.21.15.83:0/1299228352 >> 172.21.15.83:0/1299228352 conn(0x55f4e2394a80 msgr2=0x55f4e239a7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:36:16.010 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:16.008+0000 7f8d25600640 0 client.25400 ms_handle_reset on v2:172.21.15.117:6836/3185224684 2024-08-19T01:36:16.010 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:16.008+0000 7f8d40e31740 1 -- 172.21.15.83:0/1299228352 shutdown_connections 2024-08-19T01:36:16.015 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:16.013+0000 7f8d40e31740 1 -- 172.21.15.83:0/1299228352 wait complete. 2024-08-19T01:36:16.015 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:16.013+0000 7f8d40e31740 20 client.25400 trim_cache size 0 max 16384 2024-08-19T01:36:16.042 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:16.040+0000 7f8d40e31740 1 Processor -- start 2024-08-19T01:36:16.042 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:16.041+0000 7f8d40e31740 1 -- start start 2024-08-19T01:36:16.043 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:16.041+0000 7f8d40e31740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f4e23984f0 0x55f4e239f090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:36:16.043 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:16.041+0000 7f8d40e31740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f4e23a1240 0x55f4e23af230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:36:16.043 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:16.041+0000 7f8d40e31740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f4e23a36d0 0x55f4e23bb3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:36:16.043 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:16.041+0000 7f8d40e31740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f8d381c7390 con 0x55f4e23984f0 2024-08-19T01:36:16.043 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:16.041+0000 7f8d40e31740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f8d381c4dd0 con 0x55f4e23a1240 2024-08-19T01:36:16.043 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:16.041+0000 7f8d40e31740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f8d38116b00 con 0x55f4e23a36d0 2024-08-19T01:36:16.043 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:16.041+0000 7f8d37e00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f4e23a36d0 0x55f4e23bb3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:36:16.044 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:16.042+0000 7f8d37e00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f4e23a36d0 0x55f4e23bb3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.83:38744/0 (socket says 172.21.15.83:38744) 2024-08-19T01:36:16.044 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:16.042+0000 7f8d37e00640 1 -- 172.21.15.83:0/14326731 learned_addr learned my addr 172.21.15.83:0/14326731 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-08-19T01:36:16.044 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:16.042+0000 7f8d2fe00640 1 --2- 172.21.15.83:0/14326731 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f4e23a1240 0x55f4e23af230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:36:16.044 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:16.042+0000 7f8d37e00640 1 -- 172.21.15.83:0/14326731 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f4e23a1240 msgr2=0x55f4e23af230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:36:16.044 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:16.042+0000 7f8d37e00640 1 --2- 172.21.15.83:0/14326731 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f4e23a1240 0x55f4e23af230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:16.045 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:16.042+0000 7f8d37400640 1 --2- 172.21.15.83:0/14326731 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f4e23984f0 0x55f4e239f090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:36:16.045 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:16.042+0000 7f8d37e00640 1 -- 172.21.15.83:0/14326731 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f4e23984f0 msgr2=0x55f4e239f090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:36:16.045 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:16.042+0000 7f8d37e00640 1 --2- 172.21.15.83:0/14326731 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f4e23984f0 0x55f4e239f090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:16.045 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:16.042+0000 7f8d37e00640 1 -- 172.21.15.83:0/14326731 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f4e239f770 con 0x55f4e23a36d0 2024-08-19T01:36:16.045 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:16.042+0000 7f8d2fe00640 1 --2- 172.21.15.83:0/14326731 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f4e23a1240 0x55f4e23af230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-19T01:36:16.045 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:16.042+0000 7f8d37400640 1 --2- 172.21.15.83:0/14326731 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f4e23984f0 0x55f4e239f090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-19T01:36:16.045 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:16.043+0000 7f8d37e00640 1 --2- 172.21.15.83:0/14326731 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f4e23a36d0 0x55f4e23bb3b0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f8d3009b250 tx=0x7f8d3000af50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fc5f369d2c7c3709 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:36:16.045 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:16.043+0000 7f8d27e00640 1 -- 172.21.15.83:0/14326731 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d300b8070 con 0x55f4e23a36d0 2024-08-19T01:36:16.045 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:16.043+0000 7f8d27e00640 1 -- 172.21.15.83:0/14326731 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8d30099610 con 0x55f4e23a36d0 2024-08-19T01:36:16.046 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:16.043+0000 7f8d27e00640 1 -- 172.21.15.83:0/14326731 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d300b3040 con 0x55f4e23a36d0 2024-08-19T01:36:16.046 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:16.044+0000 7f8d40e31740 1 -- 172.21.15.83:0/14326731 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f4e23a36d0 msgr2=0x55f4e23bb3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:36:16.046 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:16.044+0000 7f8d40e31740 1 --2- 172.21.15.83:0/14326731 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f4e23a36d0 0x55f4e23bb3b0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f8d3009b250 tx=0x7f8d3000af50 comp rx=0 tx=0).stop 2024-08-19T01:36:16.047 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:16.045+0000 7f8d40e31740 1 -- 172.21.15.83:0/14326731 shutdown_connections 2024-08-19T01:36:16.047 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:16.045+0000 7f8d40e31740 1 --2- 172.21.15.83:0/14326731 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f4e23a36d0 0x55f4e23bb3b0 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:16.047 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:16.045+0000 7f8d40e31740 1 --2- 172.21.15.83:0/14326731 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f4e23a1240 0x55f4e23af230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:16.047 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:16.045+0000 7f8d40e31740 1 --2- 172.21.15.83:0/14326731 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f4e23984f0 0x55f4e239f090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:16.047 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:16.046+0000 7f8d40e31740 1 -- 172.21.15.83:0/14326731 >> 172.21.15.83:0/14326731 conn(0x55f4e2394a80 msgr2=0x55f4e23a2840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:36:16.047 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:16.046+0000 7f8d40e31740 1 -- 172.21.15.83:0/14326731 shutdown_connections 2024-08-19T01:36:16.048 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:16.046+0000 7f8d40e31740 1 -- 172.21.15.83:0/14326731 wait complete. 2024-08-19T01:36:16.049 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:16.047+0000 7f8d40e31740 1 -- start start 2024-08-19T01:36:16.049 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:16.048+0000 7f8d40e31740 1 Processor -- start 2024-08-19T01:36:16.050 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:16.049+0000 7f8d40e31740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f4e23984f0 0x55f4e23ab690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:36:16.051 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:16.049+0000 7f8d40e31740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f4e23a1240 0x55f4e23abbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:36:16.051 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:16.050+0000 7f8d40e31740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f4e23a36d0 0x55f4e23a6a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:36:16.052 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:16.050+0000 7f8d40e31740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f8d381171b0 con 0x55f4e23a1240 2024-08-19T01:36:16.052 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:16.050+0000 7f8d40e31740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f8d38118620 con 0x55f4e23984f0 2024-08-19T01:36:16.052 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:16.050+0000 7f8d40e31740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f8d38117880 con 0x55f4e23a36d0 2024-08-19T01:36:16.052 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:16.050+0000 7f8d40e31740 10 client.0 _ll_register_callbacks cb 0x7f8d20030460 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-19T01:36:16.052 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:16.050+0000 7f8d37400640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f4e23984f0 0x55f4e23ab690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:36:16.052 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:16.050+0000 7f8d37400640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f4e23984f0 0x55f4e23ab690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.83:53338/0 (socket says 172.21.15.83:53338) 2024-08-19T01:36:16.052 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:16.050+0000 7f8d37400640 1 -- 172.21.15.83:0/795400851 learned_addr learned my addr 172.21.15.83:0/795400851 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-08-19T01:36:16.053 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:16.050+0000 7f8d2fe00640 1 --2- 172.21.15.83:0/795400851 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f4e23a1240 0x55f4e23abbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:36:16.053 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:16.050+0000 7f8d37e00640 1 --2- 172.21.15.83:0/795400851 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f4e23a36d0 0x55f4e23a6a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:36:16.053 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:16.050+0000 7f8d37400640 1 -- 172.21.15.83:0/795400851 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f4e23a36d0 msgr2=0x55f4e23a6a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:36:16.053 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:16.051+0000 7f8d37400640 1 --2- 172.21.15.83:0/795400851 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f4e23a36d0 0x55f4e23a6a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:16.054 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:16.051+0000 7f8d37400640 1 -- 172.21.15.83:0/795400851 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f4e23a1240 msgr2=0x55f4e23abbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:36:16.054 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:16.051+0000 7f8d37400640 1 --2- 172.21.15.83:0/795400851 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f4e23a1240 0x55f4e23abbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:16.054 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:16.051+0000 7f8d37400640 1 -- 172.21.15.83:0/795400851 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f4e23a72a0 con 0x55f4e23984f0 2024-08-19T01:36:16.054 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:16.051+0000 7f8d37e00640 1 --2- 172.21.15.83:0/795400851 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f4e23a36d0 0x55f4e23a6a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-19T01:36:16.054 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:16.051+0000 7f8d2fe00640 1 --2- 172.21.15.83:0/795400851 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f4e23a1240 0x55f4e23abbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-19T01:36:16.054 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:16.051+0000 7f8d37400640 1 --2- 172.21.15.83:0/795400851 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f4e23984f0 0x55f4e23ab690 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x55f4e23b62f0 tx=0x7f8d280046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8b00f0d52984f6b2 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:36:16.054 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:16.051+0000 7f8d25600640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-08-19T01:36:16.054 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:16.051+0000 7f8d25600640 1 -- 172.21.15.83:0/795400851 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d2801e3f0 con 0x55f4e23984f0 2024-08-19T01:36:16.054 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:16.052+0000 7f8d25600640 1 -- 172.21.15.83:0/795400851 <== mon.2 v2:172.21.15.117:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8269+0+0 (secure 0 0 0) 0x7f8d2801e590 con 0x55f4e23984f0 2024-08-19T01:36:16.054 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:16.052+0000 7f8d40e31740 1 -- 172.21.15.83:0/795400851 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8d30099610 con 0x55f4e23984f0 2024-08-19T01:36:16.054 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:16.052+0000 7f8d25600640 1 -- 172.21.15.83:0/795400851 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f8d2801e790 con 0x55f4e23984f0 2024-08-19T01:36:16.054 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:16.052+0000 7f8d40e31740 10 client.25406 fetch_fsmap learned FSMap version 25 2024-08-19T01:36:16.054 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:16.052+0000 7f8d40e31740 1 -- 172.21.15.83:0/795400851 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55f4e241def0 con 0x55f4e23984f0 2024-08-19T01:36:16.055 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:16.052+0000 7f8d25600640 1 -- 172.21.15.83:0/795400851 <== mon.2 v2:172.21.15.117:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8d30099610 con 0x55f4e23984f0 2024-08-19T01:36:16.055 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:16.054+0000 7f8d25600640 1 -- 172.21.15.83:0/795400851 <== mon.2 v2:172.21.15.117:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f8d2801e930 con 0x55f4e23984f0 2024-08-19T01:36:16.055 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:16.054+0000 7f8d40e31740 10 client.25406 fetch_fsmap finished waiting for FSMap version 25 2024-08-19T01:36:16.056 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:16.054+0000 7f8d40e31740 10 client.25406 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-08-19T01:36:16.056 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:16.054+0000 7f8d40e31740 20 client.25406 populate_metadata read hostname 'smithi083' 2024-08-19T01:36:16.056 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:16.054+0000 7f8d40e31740 1 --2- 172.21.15.83:0/795400851 >> [v2:172.21.15.117:6836/3185224684,v1:172.21.15.117:6837/3185224684] conn(0x55f4e241e570 0x55f4e2420960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:36:16.056 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:16.054+0000 7f8d40e31740 4 client.25406 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-08-19T01:36:16.056 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:16.054+0000 7f8d40e31740 1 -- 172.21.15.83:0/795400851 --> [v2:172.21.15.117:6836/3185224684,v1:172.21.15.117:6837/3185224684] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55f4e2305670 con 0x55f4e241e570 2024-08-19T01:36:16.056 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:16.054+0000 7f8d2fe00640 1 --2- 172.21.15.83:0/795400851 >> [v2:172.21.15.117:6836/3185224684,v1:172.21.15.117:6837/3185224684] conn(0x55f4e241e570 0x55f4e2420960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:36:16.056 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:16.055+0000 7f8d2fe00640 1 --2- 172.21.15.83:0/795400851 >> [v2:172.21.15.117:6836/3185224684,v1:172.21.15.117:6837/3185224684] conn(0x55f4e241e570 0x55f4e2420960 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=7122e49a8f12e074 server_cookie=47d705b14c128873 in_seq=0 out_seq=0 2024-08-19T01:36:16.057 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:16.055+0000 7f8d25600640 10 client.25406 ms_handle_connect on v2:172.21.15.117:6836/3185224684 2024-08-19T01:36:16.057 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:16.055+0000 7f8d25600640 1 -- 172.21.15.83:0/795400851 <== mds.2 v2:172.21.15.117:6836/3185224684 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55f4e2305670 con 0x55f4e241e570 2024-08-19T01:36:16.057 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:16.055+0000 7f8d25600640 10 client.25406 handle_command_reply: tid=0 2024-08-19T01:36:16.057 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:16.055+0000 7f8d40e31740 1 client.25406 shutdown 2024-08-19T01:36:16.057 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:16.056+0000 7f8d40e31740 1 -- 172.21.15.83:0/795400851 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f4e23984f0 msgr2=0x55f4e23ab690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:36:16.057 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:16.056+0000 7f8d40e31740 1 --2- 172.21.15.83:0/795400851 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f4e23984f0 0x55f4e23ab690 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x55f4e23b62f0 tx=0x7f8d280046c0 comp rx=0 tx=0).stop 2024-08-19T01:36:16.058 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:16.056+0000 7f8d40e31740 1 -- 172.21.15.83:0/795400851 shutdown_connections 2024-08-19T01:36:16.058 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:16.056+0000 7f8d40e31740 1 --2- 172.21.15.83:0/795400851 >> [v2:172.21.15.117:6836/3185224684,v1:172.21.15.117:6837/3185224684] conn(0x55f4e241e570 0x55f4e2420960 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:16.058 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:16.056+0000 7f8d40e31740 1 --2- 172.21.15.83:0/795400851 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f4e23a36d0 0x55f4e23a6a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:16.058 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:16.056+0000 7f8d40e31740 1 --2- 172.21.15.83:0/795400851 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f4e23a1240 0x55f4e23abbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:16.058 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:16.056+0000 7f8d40e31740 1 --2- 172.21.15.83:0/795400851 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f4e23984f0 0x55f4e23ab690 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:16.058 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:16.056+0000 7f8d25600640 0 client.25406 ms_handle_reset on v2:172.21.15.117:6836/3185224684 2024-08-19T01:36:16.058 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:16.056+0000 7f8d40e31740 1 -- 172.21.15.83:0/795400851 >> 172.21.15.83:0/795400851 conn(0x55f4e2394a80 msgr2=0x55f4e239a7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:36:16.059 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:16.057+0000 7f8d40e31740 1 -- 172.21.15.83:0/795400851 shutdown_connections 2024-08-19T01:36:16.059 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:16.057+0000 7f8d40e31740 1 -- 172.21.15.83:0/795400851 wait complete. 2024-08-19T01:36:16.059 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:16.057+0000 7f8d40e31740 20 client.25406 trim_cache size 0 max 16384 2024-08-19T01:36:16.059 INFO:teuthology.orchestra.run.smithi083.stdout:[] 2024-08-19T01:36:16.060 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:16.058+0000 7f8d3e400640 1 -- 172.21.15.83:0/609940065 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f8d1c0a2d80 msgr2=0x7f8d1c0a5240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:36:16.060 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:16.058+0000 7f8d3e400640 1 --2- 172.21.15.83:0/609940065 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f8d1c0a2d80 0x7f8d1c0a5240 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f8d280058c0 tx=0x7f8d28005850 comp rx=0 tx=0).stop 2024-08-19T01:36:16.060 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:16.058+0000 7f8d3e400640 1 -- 172.21.15.83:0/609940065 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8d3811ae70 msgr2=0x7f8d381ab8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:36:16.060 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:16.058+0000 7f8d3e400640 1 --2- 172.21.15.83:0/609940065 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8d3811ae70 0x7f8d381ab8e0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f8d3000caf0 tx=0x7f8d3000cef0 comp rx=0 tx=0).stop 2024-08-19T01:36:16.060 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:16.058+0000 7f8d3e400640 1 -- 172.21.15.83:0/609940065 shutdown_connections 2024-08-19T01:36:16.060 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:16.058+0000 7f8d3e400640 1 --2- 172.21.15.83:0/609940065 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f8d1c0a2d80 0x7f8d1c0a5240 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:16.060 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:16.058+0000 7f8d3e400640 1 --2- 172.21.15.83:0/609940065 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8d3811ae70 0x7f8d381ab8e0 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:16.060 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:16.058+0000 7f8d3e400640 1 --2- 172.21.15.83:0/609940065 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8d38109560 0x7f8d381a7550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:16.060 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:16.058+0000 7f8d3e400640 1 --2- 172.21.15.83:0/609940065 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8d38109180 0x7f8d381a7010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:16.060 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:16.058+0000 7f8d3e400640 1 -- 172.21.15.83:0/609940065 >> 172.21.15.83:0/609940065 conn(0x7f8d3806de70 msgr2=0x7f8d3811bf50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:36:16.060 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:16.058+0000 7f8d3e400640 1 -- 172.21.15.83:0/609940065 shutdown_connections 2024-08-19T01:36:16.061 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:16.059+0000 7f8d3e400640 1 -- 172.21.15.83:0/609940065 wait complete. 2024-08-19T01:36:16.927 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833 shell --fsid bad8780e-5dc9-11ef-bccf-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-08-19T01:36:17.352 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/bad8780e-5dc9-11ef-bccf-c7b262605968/mon.a/config 2024-08-19T01:36:17.859 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:36:17 smithi083 ceph-mon[17120]: pgmap v382: 97 pgs: 97 active+clean; 11 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 469 KiB/s rd, 1.0 MiB/s wr, 102 op/s 2024-08-19T01:36:17.952 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:36:17 smithi117 ceph-mon[22016]: pgmap v382: 97 pgs: 97 active+clean; 11 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 469 KiB/s rd, 1.0 MiB/s wr, 102 op/s 2024-08-19T01:36:17.966 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:36:17 smithi167 ceph-mon[21875]: pgmap v382: 97 pgs: 97 active+clean; 11 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 469 KiB/s rd, 1.0 MiB/s wr, 102 op/s 2024-08-19T01:36:18.083 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:18.080+0000 7ff809e00640 1 -- 172.21.15.83:0/793572356 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff80410e150 msgr2=0x7ff804110540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:36:18.083 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:18.080+0000 7ff809e00640 1 --2- 172.21.15.83:0/793572356 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff80410e150 0x7ff804110540 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7ff7f0009860 tx=0x7ff7f002a3d0 comp rx=0 tx=0).stop 2024-08-19T01:36:18.083 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:18.080+0000 7ff809e00640 1 -- 172.21.15.83:0/793572356 shutdown_connections 2024-08-19T01:36:18.083 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:18.080+0000 7ff809e00640 1 --2- 172.21.15.83:0/793572356 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff80410e150 0x7ff804110540 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:18.083 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:18.080+0000 7ff809e00640 1 --2- 172.21.15.83:0/793572356 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff804074c10 0x7ff80410dc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:18.083 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:18.080+0000 7ff809e00640 1 --2- 172.21.15.83:0/793572356 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff8040742f0 0x7ff8040746d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:18.084 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:18.080+0000 7ff809e00640 1 -- 172.21.15.83:0/793572356 >> 172.21.15.83:0/793572356 conn(0x7ff8040fd610 msgr2=0x7ff8040ffa30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:36:18.084 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:18.080+0000 7ff809e00640 1 -- 172.21.15.83:0/793572356 shutdown_connections 2024-08-19T01:36:18.090 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:18.080+0000 7ff809e00640 1 -- 172.21.15.83:0/793572356 wait complete. 2024-08-19T01:36:18.091 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:18.086+0000 7ff809e00640 1 Processor -- start 2024-08-19T01:36:18.091 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:18.086+0000 7ff809e00640 1 -- start start 2024-08-19T01:36:18.093 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:18.089+0000 7ff809e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff8040742f0 0x7ff8040734e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:36:18.093 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:18.089+0000 7ff809e00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff804074c10 0x7ff804073a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:36:18.093 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:18.090+0000 7ff809e00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff80410e150 0x7ff80406c570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:36:18.093 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:18.092+0000 7ff809e00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7ff804113eb0 con 0x7ff8040742f0 2024-08-19T01:36:18.093 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:18.092+0000 7ff809e00640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7ff804113d30 con 0x7ff80410e150 2024-08-19T01:36:18.093 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:18.092+0000 7ff809e00640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7ff804114030 con 0x7ff804074c10 2024-08-19T01:36:18.093 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:18.092+0000 7ff802000640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff804074c10 0x7ff804073a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:36:18.094 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:18.092+0000 7ff802000640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff804074c10 0x7ff804073a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.83:38756/0 (socket says 172.21.15.83:38756) 2024-08-19T01:36:18.094 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:18.092+0000 7ff802000640 1 -- 172.21.15.83:0/920826105 learned_addr learned my addr 172.21.15.83:0/920826105 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-08-19T01:36:18.094 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:18.090+0000 7ff802a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff8040742f0 0x7ff8040734e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:36:18.094 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:18.090+0000 7ff803400640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff80410e150 0x7ff80406c570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:36:18.096 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:18.092+0000 7ff802000640 1 -- 172.21.15.83:0/920826105 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff80410e150 msgr2=0x7ff80406c570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:36:18.096 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:18.092+0000 7ff802000640 1 --2- 172.21.15.83:0/920826105 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff80410e150 0x7ff80406c570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:18.096 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:18.092+0000 7ff802000640 1 -- 172.21.15.83:0/920826105 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff8040742f0 msgr2=0x7ff8040734e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:36:18.096 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:18.092+0000 7ff802000640 1 --2- 172.21.15.83:0/920826105 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff8040742f0 0x7ff8040734e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:18.096 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:18.092+0000 7ff802000640 1 -- 172.21.15.83:0/920826105 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff80406ce30 con 0x7ff804074c10 2024-08-19T01:36:18.099 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:18.095+0000 7ff802000640 1 --2- 172.21.15.83:0/920826105 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff804074c10 0x7ff804073a20 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7ff7f400d950 tx=0x7ff7f400de10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:36:18.099 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:18.095+0000 7ff808a00640 1 -- 172.21.15.83:0/920826105 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7f4017070 con 0x7ff804074c10 2024-08-19T01:36:18.099 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:18.096+0000 7ff808a00640 1 -- 172.21.15.83:0/920826105 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff7f4009c30 con 0x7ff804074c10 2024-08-19T01:36:18.099 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:18.096+0000 7ff809e00640 1 -- 172.21.15.83:0/920826105 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff80406d120 con 0x7ff804074c10 2024-08-19T01:36:18.099 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:18.096+0000 7ff808a00640 1 -- 172.21.15.83:0/920826105 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7f400f6b0 con 0x7ff804074c10 2024-08-19T01:36:18.099 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:18.096+0000 7ff809e00640 1 -- 172.21.15.83:0/920826105 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff8040711e0 con 0x7ff804074c10 2024-08-19T01:36:18.108 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:18.101+0000 7ff80c85b740 1 Processor -- start 2024-08-19T01:36:18.108 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:18.101+0000 7ff80c85b740 1 -- start start 2024-08-19T01:36:18.109 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:18.103+0000 7ff80c85b740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d5c29da410 0x55d5c29da7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:36:18.110 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:18.103+0000 7ff80c85b740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d5c29dad30 0x55d5c29ed9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:36:18.110 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:18.104+0000 7ff80c85b740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d5c29edee0 0x55d5c29f02d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:36:18.110 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:18.104+0000 7ff80c85b740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55d5c293fbf0 con 0x55d5c29dad30 2024-08-19T01:36:18.110 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:18.104+0000 7ff80c85b740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55d5c28d7b20 con 0x55d5c29edee0 2024-08-19T01:36:18.110 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:18.104+0000 7ff80c85b740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55d5c293bbb0 con 0x55d5c29da410 2024-08-19T01:36:18.110 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:18.104+0000 7ff803400640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d5c29da410 0x55d5c29da7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:36:18.110 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:18.104+0000 7ff803400640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d5c29da410 0x55d5c29da7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.83:38768/0 (socket says 172.21.15.83:38768) 2024-08-19T01:36:18.111 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:18.104+0000 7ff803400640 1 -- 172.21.15.83:0/1367919274 learned_addr learned my addr 172.21.15.83:0/1367919274 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-08-19T01:36:18.111 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:18.104+0000 7ff802a00640 1 --2- 172.21.15.83:0/1367919274 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d5c29dad30 0x55d5c29ed9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:36:18.111 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:18.105+0000 7ff808a00640 1 -- 172.21.15.83:0/920826105 <== mon.1 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7ff7f40109d0 con 0x7ff804074c10 2024-08-19T01:36:18.111 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:18.105+0000 7ff803400640 1 -- 172.21.15.83:0/1367919274 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d5c29edee0 msgr2=0x55d5c29f02d0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-08-19T01:36:18.111 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:18.105+0000 7ff803400640 1 --2- 172.21.15.83:0/1367919274 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d5c29edee0 0x55d5c29f02d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:18.111 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:18.105+0000 7ff803400640 1 -- 172.21.15.83:0/1367919274 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d5c29dad30 msgr2=0x55d5c29ed9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:36:18.111 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:18.105+0000 7ff803400640 1 --2- 172.21.15.83:0/1367919274 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d5c29dad30 0x55d5c29ed9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:18.111 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:18.105+0000 7ff803400640 1 -- 172.21.15.83:0/1367919274 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d5c28de7f0 con 0x55d5c29da410 2024-08-19T01:36:18.111 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:18.106+0000 7ff802a00640 1 --2- 172.21.15.83:0/1367919274 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d5c29dad30 0x55d5c29ed9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-19T01:36:18.111 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:18.107+0000 7ff803400640 1 --2- 172.21.15.83:0/1367919274 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d5c29da410 0x55d5c29da7f0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7ff80406a730 tx=0x7ff7f0005f10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d96cec41966603f server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:36:18.113 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:18.107+0000 7ff808a00640 1 --2- 172.21.15.83:0/920826105 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7ff7e80a2c80 0x7ff7e80a5140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:36:18.113 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:18.107+0000 7ff808a00640 1 -- 172.21.15.83:0/920826105 <== mon.1 v2:172.21.15.167:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8269+0+0 (secure 0 0 0) 0x7ff7f40ceb20 con 0x7ff804074c10 2024-08-19T01:36:18.114 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:18.107+0000 7ff7efe00640 1 -- 172.21.15.83:0/1367919274 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7f0004040 con 0x55d5c29da410 2024-08-19T01:36:18.114 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:18.107+0000 7ff7efe00640 1 -- 172.21.15.83:0/1367919274 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff7f0030880 con 0x55d5c29da410 2024-08-19T01:36:18.114 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:18.107+0000 7ff7efe00640 1 -- 172.21.15.83:0/1367919274 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7f00046c0 con 0x55d5c29da410 2024-08-19T01:36:18.114 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:18.108+0000 7ff80c85b740 1 -- 172.21.15.83:0/1367919274 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d5c29da410 msgr2=0x55d5c29da7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:36:18.114 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:18.108+0000 7ff80c85b740 1 --2- 172.21.15.83:0/1367919274 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d5c29da410 0x55d5c29da7f0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7ff80406a730 tx=0x7ff7f0005f10 comp rx=0 tx=0).stop 2024-08-19T01:36:18.114 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:18.109+0000 7ff80c85b740 1 -- 172.21.15.83:0/1367919274 shutdown_connections 2024-08-19T01:36:18.114 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:18.109+0000 7ff80c85b740 1 --2- 172.21.15.83:0/1367919274 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d5c29edee0 0x55d5c29f02d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:18.114 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:18.109+0000 7ff80c85b740 1 --2- 172.21.15.83:0/1367919274 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d5c29dad30 0x55d5c29ed9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:18.114 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:18.109+0000 7ff80c85b740 1 --2- 172.21.15.83:0/1367919274 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d5c29da410 0x55d5c29da7f0 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:18.114 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:18.109+0000 7ff80c85b740 1 -- 172.21.15.83:0/1367919274 >> 172.21.15.83:0/1367919274 conn(0x55d5c29d6a80 msgr2=0x55d5c29d6e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:36:18.114 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:18.112+0000 7ff802a00640 1 --2- 172.21.15.83:0/920826105 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7ff7e80a2c80 0x7ff7e80a5140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:36:18.114 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:18.112+0000 7ff80c85b740 1 -- 172.21.15.83:0/1367919274 shutdown_connections 2024-08-19T01:36:18.115 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:18.113+0000 7ff80c85b740 1 -- 172.21.15.83:0/1367919274 wait complete. 2024-08-19T01:36:18.115 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:18.113+0000 7ff80c85b740 1 -- start start 2024-08-19T01:36:18.115 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:18.114+0000 7ff80c85b740 1 Processor -- start 2024-08-19T01:36:18.115 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:18.114+0000 7ff802a00640 1 --2- 172.21.15.83:0/920826105 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7ff7e80a2c80 0x7ff7e80a5140 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7ff7fc005fd0 tx=0x7ff7fc011040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:36:18.117 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:18.115+0000 7ff80c85b740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d5c29da410 0x55d5c29eb040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:36:18.117 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:18.115+0000 7ff80c85b740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d5c29dad30 0x55d5c29eb580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:36:18.117 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:18.115+0000 7ff80c85b740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d5c29ebac0 0x55d5c29f9ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:36:18.118 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:18.117+0000 7ff802000640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d5c29dad30 0x55d5c29eb580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:36:18.119 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:18.117+0000 7ff802000640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d5c29dad30 0x55d5c29eb580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.83:53370/0 (socket says 172.21.15.83:53370) 2024-08-19T01:36:18.121 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:18.119+0000 7ff802a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d5c29da410 0x55d5c29eb040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:36:18.123 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:18.119+0000 7ff802a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d5c29da410 0x55d5c29eb040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:40266/0 (socket says 172.21.15.83:40266) 2024-08-19T01:36:18.123 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:18.120+0000 7ff80c85b740 1 -- 172.21.15.83:0/2601703762 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55d5c29f09f0 con 0x55d5c29da410 2024-08-19T01:36:18.123 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:18.120+0000 7ff802000640 1 -- 172.21.15.83:0/2601703762 learned_addr learned my addr 172.21.15.83:0/2601703762 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-08-19T01:36:18.123 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:18.120+0000 7ff80c85b740 1 -- 172.21.15.83:0/2601703762 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55d5c28facf0 con 0x55d5c29dad30 2024-08-19T01:36:18.123 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:18.120+0000 7ff80c85b740 1 -- 172.21.15.83:0/2601703762 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55d5c29f0b70 con 0x55d5c29ebac0 2024-08-19T01:36:18.123 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:18.120+0000 7ff803400640 1 --2- 172.21.15.83:0/2601703762 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d5c29ebac0 0x55d5c29f9ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:36:18.123 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:18.120+0000 7ff80c85b740 10 client.0 _ll_register_callbacks cb 0x55d5c28e6e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-19T01:36:18.124 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:18.121+0000 7ff803400640 1 -- 172.21.15.83:0/2601703762 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d5c29dad30 msgr2=0x55d5c29eb580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:36:18.124 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:18.121+0000 7ff803400640 1 --2- 172.21.15.83:0/2601703762 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d5c29dad30 0x55d5c29eb580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:18.124 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:18.121+0000 7ff803400640 1 -- 172.21.15.83:0/2601703762 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d5c29da410 msgr2=0x55d5c29eb040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:36:18.124 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:18.121+0000 7ff803400640 1 --2- 172.21.15.83:0/2601703762 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d5c29da410 0x55d5c29eb040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:18.124 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:18.121+0000 7ff803400640 1 -- 172.21.15.83:0/2601703762 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d5c26b13c0 con 0x55d5c29ebac0 2024-08-19T01:36:18.124 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:18.121+0000 7ff80c85b740 1 -- 172.21.15.83:0/2601703762 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55d5c294d160 con 0x55d5c29ebac0 2024-08-19T01:36:18.124 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:18.121+0000 7ff802a00640 1 --2- 172.21.15.83:0/2601703762 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d5c29da410 0x55d5c29eb040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-19T01:36:18.124 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:18.121+0000 7ff803400640 1 --2- 172.21.15.83:0/2601703762 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d5c29ebac0 0x55d5c29f9ea0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7ff7f0005b60 tx=0x7ff7f00051f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2379960978ece2ef server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:36:18.124 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:18.121+0000 7ff7ecc00640 10 client.25384 ms_handle_connect on v2:172.21.15.167:3300/0 2024-08-19T01:36:18.125 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:18.122+0000 7ff7ecc00640 1 -- 172.21.15.83:0/2601703762 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7f0055060 con 0x55d5c29ebac0 2024-08-19T01:36:18.125 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:18.122+0000 7ff7ecc00640 1 -- 172.21.15.83:0/2601703762 <== mon.1 v2:172.21.15.167:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8269+0+0 (secure 0 0 0) 0x7ff7f00305b0 con 0x55d5c29ebac0 2024-08-19T01:36:18.125 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:18.122+0000 7ff7ecc00640 1 -- 172.21.15.83:0/2601703762 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7ff7f0009450 con 0x55d5c29ebac0 2024-08-19T01:36:18.125 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:18.122+0000 7ff80c85b740 10 client.25384 fetch_fsmap learned FSMap version 25 2024-08-19T01:36:18.125 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:18.122+0000 7ff80c85b740 1 -- 172.21.15.83:0/2601703762 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({fsmap=25}) -- 0x55d5c29f09f0 con 0x55d5c29ebac0 2024-08-19T01:36:18.135 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:18.125+0000 7ff7ecc00640 1 -- 172.21.15.83:0/2601703762 <== mon.1 v2:172.21.15.167:3300/0 4 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7ff7f004f070 con 0x55d5c29ebac0 2024-08-19T01:36:18.135 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:18.125+0000 7ff80c85b740 10 client.25384 fetch_fsmap finished waiting for FSMap version 25 2024-08-19T01:36:18.135 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:18.125+0000 7ff80c85b740 10 client.25384 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.g 2024-08-19T01:36:18.135 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:18.125+0000 7ff80c85b740 20 client.25384 populate_metadata read hostname 'smithi083' 2024-08-19T01:36:18.135 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:18.125+0000 7ff80c85b740 1 --2- 172.21.15.83:0/2601703762 >> [v2:172.21.15.83:6838/3239261845,v1:172.21.15.83:6839/3239261845] conn(0x55d5c29fdd90 0x55d5c2a00180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:36:18.135 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:18.125+0000 7ff80c85b740 4 client.25384 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-19T01:36:18.135 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:18.125+0000 7ff80c85b740 1 -- 172.21.15.83:0/2601703762 --> [v2:172.21.15.83:6838/3239261845,v1:172.21.15.83:6839/3239261845] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d5c2947670 con 0x55d5c29fdd90 2024-08-19T01:36:18.136 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:18.126+0000 7ff802a00640 1 --2- 172.21.15.83:0/2601703762 >> [v2:172.21.15.83:6838/3239261845,v1:172.21.15.83:6839/3239261845] conn(0x55d5c29fdd90 0x55d5c2a00180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:36:18.136 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:18.127+0000 7ff802a00640 1 --2- 172.21.15.83:0/2601703762 >> [v2:172.21.15.83:6838/3239261845,v1:172.21.15.83:6839/3239261845] conn(0x55d5c29fdd90 0x55d5c2a00180 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=18e705ad6ab0ce8 server_cookie=5e57e05a3d385ca in_seq=0 out_seq=0 2024-08-19T01:36:18.136 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:18.127+0000 7ff7ecc00640 10 client.25384 ms_handle_connect on v2:172.21.15.83:6838/3239261845 2024-08-19T01:36:18.136 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:18.128+0000 7ff7ecc00640 1 -- 172.21.15.83:0/2601703762 <== mds.1 v2:172.21.15.83:6838/3239261845 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55d5c2947670 con 0x55d5c29fdd90 2024-08-19T01:36:18.136 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:18.128+0000 7ff7ecc00640 10 client.25384 handle_command_reply: tid=0 2024-08-19T01:36:18.136 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:18.128+0000 7ff80c85b740 1 client.25384 shutdown 2024-08-19T01:36:18.136 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:18.129+0000 7ff80c85b740 1 -- 172.21.15.83:0/2601703762 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d5c29ebac0 msgr2=0x55d5c29f9ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:36:18.136 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:18.129+0000 7ff80c85b740 1 --2- 172.21.15.83:0/2601703762 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d5c29ebac0 0x55d5c29f9ea0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7ff7f0005b60 tx=0x7ff7f00051f0 comp rx=0 tx=0).stop 2024-08-19T01:36:18.136 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:18.129+0000 7ff80c85b740 1 -- 172.21.15.83:0/2601703762 shutdown_connections 2024-08-19T01:36:18.136 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:18.129+0000 7ff80c85b740 1 --2- 172.21.15.83:0/2601703762 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d5c29ebac0 0x55d5c29f9ea0 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:18.136 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:18.129+0000 7ff80c85b740 1 --2- 172.21.15.83:0/2601703762 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d5c29dad30 0x55d5c29eb580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:18.136 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:18.129+0000 7ff80c85b740 1 --2- 172.21.15.83:0/2601703762 >> [v2:172.21.15.83:6838/3239261845,v1:172.21.15.83:6839/3239261845] conn(0x55d5c29fdd90 0x55d5c2a00180 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:18.136 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:18.129+0000 7ff80c85b740 1 --2- 172.21.15.83:0/2601703762 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d5c29da410 0x55d5c29eb040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:18.136 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:18.129+0000 7ff80c85b740 1 -- 172.21.15.83:0/2601703762 >> 172.21.15.83:0/2601703762 conn(0x55d5c29d6a80 msgr2=0x55d5c29dc790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:36:18.136 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:18.129+0000 7ff7ecc00640 0 client.25384 ms_handle_reset on v2:172.21.15.83:6838/3239261845 2024-08-19T01:36:18.137 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:18.132+0000 7ff80c85b740 1 -- 172.21.15.83:0/2601703762 shutdown_connections 2024-08-19T01:36:18.137 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:18.132+0000 7ff80c85b740 1 -- 172.21.15.83:0/2601703762 wait complete. 2024-08-19T01:36:18.137 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:18.132+0000 7ff80c85b740 20 client.25384 trim_cache size 0 max 16384 2024-08-19T01:36:18.172 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:18.169+0000 7ff80c85b740 1 Processor -- start 2024-08-19T01:36:18.172 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:18.169+0000 7ff80c85b740 1 -- start start 2024-08-19T01:36:18.172 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:18.169+0000 7ff80c85b740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d5c29da410 0x55d5c29d1b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:36:18.173 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:18.169+0000 7ff80c85b740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d5c29dad30 0x55d5c29f9c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:36:18.173 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:18.169+0000 7ff80c85b740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d5c29ebac0 0x55d5c29ea020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:36:18.173 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:18.169+0000 7ff80c85b740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7ff8041b6190 con 0x55d5c29ebac0 2024-08-19T01:36:18.173 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:18.169+0000 7ff80c85b740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7ff8041b3a60 con 0x55d5c29dad30 2024-08-19T01:36:18.173 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:18.169+0000 7ff80c85b740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7ff8041b18b0 con 0x55d5c29da410 2024-08-19T01:36:18.173 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:18.169+0000 7ff802000640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d5c29dad30 0x55d5c29f9c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:36:18.173 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:18.170+0000 7ff802000640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d5c29dad30 0x55d5c29f9c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.83:53378/0 (socket says 172.21.15.83:53378) 2024-08-19T01:36:18.173 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:18.170+0000 7ff802000640 1 -- 172.21.15.83:0/3981604862 learned_addr learned my addr 172.21.15.83:0/3981604862 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-08-19T01:36:18.173 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:18.170+0000 7ff803400640 1 --2- 172.21.15.83:0/3981604862 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d5c29ebac0 0x55d5c29ea020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:36:18.173 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:18.170+0000 7ff802000640 1 -- 172.21.15.83:0/3981604862 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d5c29da410 msgr2=0x55d5c29d1b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:36:18.173 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:18.170+0000 7ff802000640 1 --2- 172.21.15.83:0/3981604862 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d5c29da410 0x55d5c29d1b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:18.173 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:18.170+0000 7ff802000640 1 -- 172.21.15.83:0/3981604862 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d5c29ebac0 msgr2=0x55d5c29ea020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:36:18.173 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:18.170+0000 7ff802000640 1 --2- 172.21.15.83:0/3981604862 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d5c29ebac0 0x55d5c29ea020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:18.174 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:18.170+0000 7ff802000640 1 -- 172.21.15.83:0/3981604862 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d5c29e9590 con 0x55d5c29dad30 2024-08-19T01:36:18.174 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:18.170+0000 7ff803400640 1 --2- 172.21.15.83:0/3981604862 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d5c29ebac0 0x55d5c29ea020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-19T01:36:18.182 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:18.171+0000 7ff802000640 1 --2- 172.21.15.83:0/3981604862 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d5c29dad30 0x55d5c29f9c90 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7ff7f409b0b0 tx=0x7ff7f4010d70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=39015e592dadd025 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:36:18.182 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:18.171+0000 7ff7ef400640 1 -- 172.21.15.83:0/3981604862 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7f4020040 con 0x55d5c29dad30 2024-08-19T01:36:18.182 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:18.171+0000 7ff7ef400640 1 -- 172.21.15.83:0/3981604862 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff7f409b3b0 con 0x55d5c29dad30 2024-08-19T01:36:18.182 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:18.171+0000 7ff7ef400640 1 -- 172.21.15.83:0/3981604862 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7f409aa60 con 0x55d5c29dad30 2024-08-19T01:36:18.182 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:18.174+0000 7ff80c85b740 1 -- 172.21.15.83:0/3981604862 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d5c29dad30 msgr2=0x55d5c29f9c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:36:18.182 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:18.174+0000 7ff80c85b740 1 --2- 172.21.15.83:0/3981604862 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d5c29dad30 0x55d5c29f9c90 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7ff7f409b0b0 tx=0x7ff7f4010d70 comp rx=0 tx=0).stop 2024-08-19T01:36:18.182 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:18.175+0000 7ff80c85b740 1 -- 172.21.15.83:0/3981604862 shutdown_connections 2024-08-19T01:36:18.182 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:18.175+0000 7ff80c85b740 1 --2- 172.21.15.83:0/3981604862 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d5c29ebac0 0x55d5c29ea020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:18.182 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:18.175+0000 7ff80c85b740 1 --2- 172.21.15.83:0/3981604862 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d5c29dad30 0x55d5c29f9c90 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:18.183 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:18.175+0000 7ff80c85b740 1 --2- 172.21.15.83:0/3981604862 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d5c29da410 0x55d5c29d1b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:18.183 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:18.175+0000 7ff80c85b740 1 -- 172.21.15.83:0/3981604862 >> 172.21.15.83:0/3981604862 conn(0x55d5c29d6a80 msgr2=0x55d5c29e9910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:36:18.183 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:18.175+0000 7ff80c85b740 1 -- 172.21.15.83:0/3981604862 shutdown_connections 2024-08-19T01:36:18.183 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:18.175+0000 7ff80c85b740 1 -- 172.21.15.83:0/3981604862 wait complete. 2024-08-19T01:36:18.183 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:18.176+0000 7ff80c85b740 1 -- start start 2024-08-19T01:36:18.183 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:18.177+0000 7ff80c85b740 1 Processor -- start 2024-08-19T01:36:18.183 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:18.178+0000 7ff80c85b740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d5c29da410 0x55d5c29f6480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:36:18.183 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:18.178+0000 7ff80c85b740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d5c29dad30 0x55d5c29e7c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:36:18.183 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:18.178+0000 7ff80c85b740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d5c29ebac0 0x55d5c29e8170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:36:18.183 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:18.179+0000 7ff80c85b740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7ff8041b1f20 con 0x55d5c29dad30 2024-08-19T01:36:18.183 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:18.179+0000 7ff80c85b740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7ff8041b3390 con 0x55d5c29da410 2024-08-19T01:36:18.183 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:18.179+0000 7ff80c85b740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7ff8041b25f0 con 0x55d5c29ebac0 2024-08-19T01:36:18.183 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:18.179+0000 7ff80c85b740 10 client.0 _ll_register_callbacks cb 0x7ff7f000b1b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-19T01:36:18.184 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:18.179+0000 7ff802a00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d5c29da410 0x55d5c29f6480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:36:18.184 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:18.180+0000 7ff803400640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d5c29ebac0 0x55d5c29e8170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:36:18.184 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:18.180+0000 7ff803400640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d5c29ebac0 0x55d5c29e8170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.83:38786/0 (socket says 172.21.15.83:38786) 2024-08-19T01:36:18.184 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:18.180+0000 7ff803400640 1 -- 172.21.15.83:0/3398177105 learned_addr learned my addr 172.21.15.83:0/3398177105 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-08-19T01:36:18.184 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:18.180+0000 7ff802000640 1 --2- 172.21.15.83:0/3398177105 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d5c29dad30 0x55d5c29e7c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:36:18.186 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:18.181+0000 7ff803400640 1 -- 172.21.15.83:0/3398177105 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d5c29da410 msgr2=0x55d5c29f6480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:36:18.186 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:18.181+0000 7ff803400640 1 --2- 172.21.15.83:0/3398177105 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d5c29da410 0x55d5c29f6480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:18.186 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:18.181+0000 7ff803400640 1 -- 172.21.15.83:0/3398177105 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d5c29dad30 msgr2=0x55d5c29e7c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:36:18.186 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:18.181+0000 7ff803400640 1 --2- 172.21.15.83:0/3398177105 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d5c29dad30 0x55d5c29e7c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:18.186 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:18.181+0000 7ff803400640 1 -- 172.21.15.83:0/3398177105 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d5c29e89a0 con 0x55d5c29ebac0 2024-08-19T01:36:18.186 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:18.182+0000 7ff802000640 1 --2- 172.21.15.83:0/3398177105 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d5c29dad30 0x55d5c29e7c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-19T01:36:18.186 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:18.183+0000 7ff802a00640 1 --2- 172.21.15.83:0/3398177105 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d5c29da410 0x55d5c29f6480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-19T01:36:18.187 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:18.183+0000 7ff7ecc00640 10 client.0 ms_handle_connect on v2:172.21.15.167:3300/0 2024-08-19T01:36:18.187 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:18.183+0000 7ff803400640 1 --2- 172.21.15.83:0/3398177105 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d5c29ebac0 0x55d5c29e8170 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7ff7f0000c00 tx=0x7ff7f0005f10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=32c5a85a5249b10a server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:36:18.187 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:18.184+0000 7ff7ecc00640 1 -- 172.21.15.83:0/3398177105 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7f0031400 con 0x55d5c29ebac0 2024-08-19T01:36:18.187 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:18.184+0000 7ff7ecc00640 1 -- 172.21.15.83:0/3398177105 <== mon.1 v2:172.21.15.167:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8269+0+0 (secure 0 0 0) 0x7ff7f00315a0 con 0x55d5c29ebac0 2024-08-19T01:36:18.187 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:18.185+0000 7ff80c85b740 1 -- 172.21.15.83:0/3398177105 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff7f409b3b0 con 0x55d5c29ebac0 2024-08-19T01:36:18.189 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:18.185+0000 7ff7ecc00640 1 -- 172.21.15.83:0/3398177105 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7ff7f00317a0 con 0x55d5c29ebac0 2024-08-19T01:36:18.189 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:18.185+0000 7ff80c85b740 10 client.25387 fetch_fsmap learned FSMap version 25 2024-08-19T01:36:18.189 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:18.185+0000 7ff80c85b740 1 -- 172.21.15.83:0/3398177105 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55d5c2a68720 con 0x55d5c29ebac0 2024-08-19T01:36:18.189 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:18.187+0000 7ff7ecc00640 1 -- 172.21.15.83:0/3398177105 <== mon.1 v2:172.21.15.167:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff7f409b3b0 con 0x55d5c29ebac0 2024-08-19T01:36:18.189 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:18.187+0000 7ff7ecc00640 1 -- 172.21.15.83:0/3398177105 <== mon.1 v2:172.21.15.167:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7ff7f004f040 con 0x55d5c29ebac0 2024-08-19T01:36:18.189 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:18.187+0000 7ff80c85b740 10 client.25387 fetch_fsmap finished waiting for FSMap version 25 2024-08-19T01:36:18.189 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:18.187+0000 7ff80c85b740 10 client.25387 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.g 2024-08-19T01:36:18.190 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:18.187+0000 7ff80c85b740 20 client.25387 populate_metadata read hostname 'smithi083' 2024-08-19T01:36:18.190 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:18.187+0000 7ff80c85b740 1 --2- 172.21.15.83:0/3398177105 >> [v2:172.21.15.83:6838/3239261845,v1:172.21.15.83:6839/3239261845] conn(0x55d5c2a68d90 0x55d5c2a6b180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:36:18.190 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:18.187+0000 7ff80c85b740 4 client.25387 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-08-19T01:36:18.190 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:18.187+0000 7ff80c85b740 1 -- 172.21.15.83:0/3398177105 --> [v2:172.21.15.83:6838/3239261845,v1:172.21.15.83:6839/3239261845] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55d5c28c4010 con 0x55d5c2a68d90 2024-08-19T01:36:18.194 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:18.188+0000 7ff802a00640 1 --2- 172.21.15.83:0/3398177105 >> [v2:172.21.15.83:6838/3239261845,v1:172.21.15.83:6839/3239261845] conn(0x55d5c2a68d90 0x55d5c2a6b180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:36:18.194 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:18.189+0000 7ff802a00640 1 --2- 172.21.15.83:0/3398177105 >> [v2:172.21.15.83:6838/3239261845,v1:172.21.15.83:6839/3239261845] conn(0x55d5c2a68d90 0x55d5c2a6b180 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=777471229d8f1e3f server_cookie=af26faab938d3aa in_seq=0 out_seq=0 2024-08-19T01:36:18.194 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:18.190+0000 7ff7ecc00640 10 client.25387 ms_handle_connect on v2:172.21.15.83:6838/3239261845 2024-08-19T01:36:18.194 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:18.190+0000 7ff7ecc00640 1 -- 172.21.15.83:0/3398177105 <== mds.1 v2:172.21.15.83:6838/3239261845 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55d5c28c4010 con 0x55d5c2a68d90 2024-08-19T01:36:18.194 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:18.190+0000 7ff7ecc00640 10 client.25387 handle_command_reply: tid=0 2024-08-19T01:36:18.195 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:18.190+0000 7ff80c85b740 1 client.25387 shutdown 2024-08-19T01:36:18.196 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:18.194+0000 7ff80c85b740 1 -- 172.21.15.83:0/3398177105 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d5c29ebac0 msgr2=0x55d5c29e8170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:36:18.196 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:18.194+0000 7ff80c85b740 1 --2- 172.21.15.83:0/3398177105 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d5c29ebac0 0x55d5c29e8170 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7ff7f0000c00 tx=0x7ff7f0005f10 comp rx=0 tx=0).stop 2024-08-19T01:36:18.200 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:18.197+0000 7ff80c85b740 1 -- 172.21.15.83:0/3398177105 shutdown_connections 2024-08-19T01:36:18.200 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:18.197+0000 7ff80c85b740 1 --2- 172.21.15.83:0/3398177105 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d5c29ebac0 0x55d5c29e8170 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:18.200 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:18.197+0000 7ff80c85b740 1 --2- 172.21.15.83:0/3398177105 >> [v2:172.21.15.83:6838/3239261845,v1:172.21.15.83:6839/3239261845] conn(0x55d5c2a68d90 0x55d5c2a6b180 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:18.200 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:18.198+0000 7ff80c85b740 1 --2- 172.21.15.83:0/3398177105 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d5c29dad30 0x55d5c29e7c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:18.200 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:18.198+0000 7ff80c85b740 1 --2- 172.21.15.83:0/3398177105 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d5c29da410 0x55d5c29f6480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:18.200 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:18.198+0000 7ff7ecc00640 0 client.25387 ms_handle_reset on v2:172.21.15.83:6838/3239261845 2024-08-19T01:36:18.200 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:18.198+0000 7ff80c85b740 1 -- 172.21.15.83:0/3398177105 >> 172.21.15.83:0/3398177105 conn(0x55d5c29d6a80 msgr2=0x55d5c29ea130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:36:18.225 INFO:teuthology.orchestra.run.smithi083.stdout:[] 2024-08-19T01:36:18.225 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:18.199+0000 7ff80c85b740 1 -- 172.21.15.83:0/3398177105 shutdown_connections 2024-08-19T01:36:18.225 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:18.199+0000 7ff80c85b740 1 -- 172.21.15.83:0/3398177105 wait complete. 2024-08-19T01:36:18.226 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:18.199+0000 7ff80c85b740 20 client.25387 trim_cache size 0 max 16384 2024-08-19T01:36:18.226 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:18.200+0000 7ff809e00640 1 -- 172.21.15.83:0/920826105 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7ff7e80a2c80 msgr2=0x7ff7e80a5140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:36:18.226 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:18.200+0000 7ff809e00640 1 --2- 172.21.15.83:0/920826105 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7ff7e80a2c80 0x7ff7e80a5140 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7ff7fc005fd0 tx=0x7ff7fc011040 comp rx=0 tx=0).stop 2024-08-19T01:36:18.226 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:18.200+0000 7ff809e00640 1 -- 172.21.15.83:0/920826105 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff804074c10 msgr2=0x7ff804073a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:36:18.226 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:18.200+0000 7ff809e00640 1 --2- 172.21.15.83:0/920826105 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff804074c10 0x7ff804073a20 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7ff7f400d950 tx=0x7ff7f400de10 comp rx=0 tx=0).stop 2024-08-19T01:36:18.226 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:18.200+0000 7ff809e00640 1 -- 172.21.15.83:0/920826105 shutdown_connections 2024-08-19T01:36:18.226 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:18.200+0000 7ff809e00640 1 --2- 172.21.15.83:0/920826105 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7ff7e80a2c80 0x7ff7e80a5140 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:18.226 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:18.200+0000 7ff809e00640 1 --2- 172.21.15.83:0/920826105 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff80410e150 0x7ff80406c570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:18.226 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:18.200+0000 7ff809e00640 1 --2- 172.21.15.83:0/920826105 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff804074c10 0x7ff804073a20 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:18.226 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:18.200+0000 7ff809e00640 1 --2- 172.21.15.83:0/920826105 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff8040742f0 0x7ff8040734e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:18.226 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:18.200+0000 7ff809e00640 1 -- 172.21.15.83:0/920826105 >> 172.21.15.83:0/920826105 conn(0x7ff8040fd610 msgr2=0x7ff8040fe0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:36:18.226 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:18.201+0000 7ff809e00640 1 -- 172.21.15.83:0/920826105 shutdown_connections 2024-08-19T01:36:18.226 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:18.201+0000 7ff809e00640 1 -- 172.21.15.83:0/920826105 wait complete. 2024-08-19T01:36:19.076 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833 shell --fsid bad8780e-5dc9-11ef-bccf-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-08-19T01:36:19.758 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/bad8780e-5dc9-11ef-bccf-c7b262605968/mon.a/config 2024-08-19T01:36:19.903 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:36:19 smithi083 ceph-mon[17120]: pgmap v383: 97 pgs: 97 active+clean; 21 MiB data, 449 MiB used, 1.0 TiB / 1.0 TiB avail; 686 KiB/s rd, 1.7 MiB/s wr, 131 op/s 2024-08-19T01:36:19.952 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:36:19 smithi117 ceph-mon[22016]: pgmap v383: 97 pgs: 97 active+clean; 21 MiB data, 449 MiB used, 1.0 TiB / 1.0 TiB avail; 686 KiB/s rd, 1.7 MiB/s wr, 131 op/s 2024-08-19T01:36:19.966 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:36:19 smithi167 ceph-mon[21875]: pgmap v383: 97 pgs: 97 active+clean; 21 MiB data, 449 MiB used, 1.0 TiB / 1.0 TiB avail; 686 KiB/s rd, 1.7 MiB/s wr, 131 op/s 2024-08-19T01:36:20.454 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:20.445+0000 7f1e45a00640 1 -- 172.21.15.83:0/2286637350 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1e4006eb40 msgr2=0x7f1e401142a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:36:20.454 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:20.445+0000 7f1e45a00640 1 --2- 172.21.15.83:0/2286637350 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1e4006eb40 0x7f1e401142a0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f1e28009c10 tx=0x7f1e2802a6e0 comp rx=0 tx=0).stop 2024-08-19T01:36:20.454 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:20.450+0000 7f1e45a00640 1 -- 172.21.15.83:0/2286637350 shutdown_connections 2024-08-19T01:36:20.455 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:20.450+0000 7f1e45a00640 1 --2- 172.21.15.83:0/2286637350 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1e401147e0 0x7f1e40116bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:20.455 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:20.450+0000 7f1e45a00640 1 --2- 172.21.15.83:0/2286637350 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1e4006eb40 0x7f1e401142a0 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:20.455 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:20.450+0000 7f1e45a00640 1 --2- 172.21.15.83:0/2286637350 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1e4006e220 0x7f1e4006e600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:20.455 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:20.450+0000 7f1e45a00640 1 -- 172.21.15.83:0/2286637350 >> 172.21.15.83:0/2286637350 conn(0x7f1e40072eb0 msgr2=0x7f1e400732c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:36:20.455 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:20.450+0000 7f1e45a00640 1 -- 172.21.15.83:0/2286637350 shutdown_connections 2024-08-19T01:36:20.455 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:20.450+0000 7f1e45a00640 1 -- 172.21.15.83:0/2286637350 wait complete. 2024-08-19T01:36:20.455 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:20.452+0000 7f1e45a00640 1 Processor -- start 2024-08-19T01:36:20.455 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:20.452+0000 7f1e45a00640 1 -- start start 2024-08-19T01:36:20.456 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:20.454+0000 7f1e45a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1e4006e220 0x7f1e401acb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:36:20.456 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:20.454+0000 7f1e45a00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1e4006eb40 0x7f1e401ad090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:36:20.456 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:20.454+0000 7f1e45a00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1e401147e0 0x7f1e40113510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:36:20.456 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:20.454+0000 7f1e45a00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f1e4011a2a0 con 0x7f1e4006e220 2024-08-19T01:36:20.456 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:20.454+0000 7f1e45a00640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f1e4011a120 con 0x7f1e401147e0 2024-08-19T01:36:20.456 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:20.454+0000 7f1e45a00640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f1e4011a420 con 0x7f1e4006eb40 2024-08-19T01:36:20.456 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:20.454+0000 7f1e37400640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1e4006eb40 0x7f1e401ad090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:36:20.457 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:20.454+0000 7f1e37400640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1e4006eb40 0x7f1e401ad090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.83:38804/0 (socket says 172.21.15.83:38804) 2024-08-19T01:36:20.457 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:20.455+0000 7f1e37400640 1 -- 172.21.15.83:0/72786262 learned_addr learned my addr 172.21.15.83:0/72786262 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-08-19T01:36:20.457 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:20.455+0000 7f1e3fe00640 1 --2- 172.21.15.83:0/72786262 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1e4006e220 0x7f1e401acb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:36:20.457 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:20.455+0000 7f1e45000640 1 --2- 172.21.15.83:0/72786262 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1e401147e0 0x7f1e40113510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:36:20.457 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:20.455+0000 7f1e3fe00640 1 -- 172.21.15.83:0/72786262 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1e4006eb40 msgr2=0x7f1e401ad090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:36:20.457 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:20.455+0000 7f1e3fe00640 1 --2- 172.21.15.83:0/72786262 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1e4006eb40 0x7f1e401ad090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:20.457 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:20.455+0000 7f1e3fe00640 1 -- 172.21.15.83:0/72786262 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1e401147e0 msgr2=0x7f1e40113510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:36:20.458 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:20.455+0000 7f1e3fe00640 1 --2- 172.21.15.83:0/72786262 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1e401147e0 0x7f1e40113510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:20.458 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:20.455+0000 7f1e3fe00640 1 -- 172.21.15.83:0/72786262 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1e40113d40 con 0x7f1e4006e220 2024-08-19T01:36:20.458 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:20.456+0000 7f1e3fe00640 1 --2- 172.21.15.83:0/72786262 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1e4006e220 0x7f1e401acb50 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f1e3000e790 tx=0x7f1e3000ec50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:36:20.459 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:20.457+0000 7f1e3d600640 1 -- 172.21.15.83:0/72786262 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e3000eea0 con 0x7f1e4006e220 2024-08-19T01:36:20.459 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:20.457+0000 7f1e3d600640 1 -- 172.21.15.83:0/72786262 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1e3000ccc0 con 0x7f1e4006e220 2024-08-19T01:36:20.459 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:20.457+0000 7f1e3d600640 1 -- 172.21.15.83:0/72786262 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e3000eea0 con 0x7f1e4006e220 2024-08-19T01:36:20.459 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:20.457+0000 7f1e45a00640 1 -- 172.21.15.83:0/72786262 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1e40114030 con 0x7f1e4006e220 2024-08-19T01:36:20.460 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:20.459+0000 7f1e45a00640 1 -- 172.21.15.83:0/72786262 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1e401b3940 con 0x7f1e4006e220 2024-08-19T01:36:20.464 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:20.463+0000 7f1e3d600640 1 -- 172.21.15.83:0/72786262 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7f1e30010430 con 0x7f1e4006e220 2024-08-19T01:36:20.468 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:20.466+0000 7f1e3d600640 1 --2- 172.21.15.83:0/72786262 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f1e240a2e50 0x7f1e240a5310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:36:20.468 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:20.466+0000 7f1e49a8e740 1 Processor -- start 2024-08-19T01:36:20.468 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:20.466+0000 7f1e3d600640 1 -- 172.21.15.83:0/72786262 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8269+0+0 (secure 0 0 0) 0x7f1e300cefa0 con 0x7f1e4006e220 2024-08-19T01:36:20.470 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:20.466+0000 7f1e37400640 1 --2- 172.21.15.83:0/72786262 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f1e240a2e50 0x7f1e240a5310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:36:20.470 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:20.467+0000 7f1e49a8e740 1 -- start start 2024-08-19T01:36:20.472 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:20.470+0000 7f1e37400640 1 --2- 172.21.15.83:0/72786262 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f1e240a2e50 0x7f1e240a5310 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f1e40114770 tx=0x7f1e28005e30 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:36:20.472 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:20.470+0000 7f1e49a8e740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5614d46d86e0 0x5614d46d8ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:36:20.472 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:20.470+0000 7f1e49a8e740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5614d46d9000 0x5614d46e3380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:36:20.472 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:20.470+0000 7f1e49a8e740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5614d46e38c0 0x5614d46e5cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:36:20.473 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:20.470+0000 7f1e3fe00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5614d46d86e0 0x5614d46d8ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:36:20.473 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:20.471+0000 7f1e3fe00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5614d46d86e0 0x5614d46d8ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:40320/0 (socket says 172.21.15.83:40320) 2024-08-19T01:36:20.473 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:20.471+0000 7f1e3fe00640 1 -- 172.21.15.83:0/1776405755 learned_addr learned my addr 172.21.15.83:0/1776405755 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-08-19T01:36:20.473 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:20.471+0000 7f1e49a8e740 1 -- 172.21.15.83:0/1776405755 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x5614d463dbf0 con 0x5614d46d86e0 2024-08-19T01:36:20.473 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:20.471+0000 7f1e49a8e740 1 -- 172.21.15.83:0/1776405755 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x5614d45d5b20 con 0x5614d46d9000 2024-08-19T01:36:20.473 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:20.471+0000 7f1e49a8e740 1 -- 172.21.15.83:0/1776405755 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x5614d4639bb0 con 0x5614d46e38c0 2024-08-19T01:36:20.473 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:20.471+0000 7f1e45000640 1 --2- 172.21.15.83:0/1776405755 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5614d46e38c0 0x5614d46e5cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:36:20.473 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:20.471+0000 7f1e37400640 1 --2- 172.21.15.83:0/1776405755 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5614d46d9000 0x5614d46e3380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:36:20.474 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:20.472+0000 7f1e3fe00640 1 -- 172.21.15.83:0/1776405755 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5614d46e38c0 msgr2=0x5614d46e5cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:36:20.474 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:20.472+0000 7f1e3fe00640 1 --2- 172.21.15.83:0/1776405755 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5614d46e38c0 0x5614d46e5cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:20.474 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:20.472+0000 7f1e3fe00640 1 -- 172.21.15.83:0/1776405755 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5614d46d9000 msgr2=0x5614d46e3380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:36:20.474 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:20.472+0000 7f1e3fe00640 1 --2- 172.21.15.83:0/1776405755 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5614d46d9000 0x5614d46e3380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:20.474 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:20.472+0000 7f1e3fe00640 1 -- 172.21.15.83:0/1776405755 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5614d45dc7f0 con 0x5614d46d86e0 2024-08-19T01:36:20.474 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:20.472+0000 7f1e3fe00640 1 --2- 172.21.15.83:0/1776405755 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5614d46d86e0 0x5614d46d8ac0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f1e30097c40 tx=0x7f1e3009b140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=208a753041fe148d server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:36:20.475 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:20.473+0000 7f1e35600640 1 -- 172.21.15.83:0/1776405755 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e3009fb90 con 0x5614d46d86e0 2024-08-19T01:36:20.476 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:20.473+0000 7f1e35600640 1 -- 172.21.15.83:0/1776405755 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1e300a4450 con 0x5614d46d86e0 2024-08-19T01:36:20.476 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:20.473+0000 7f1e35600640 1 -- 172.21.15.83:0/1776405755 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e30010720 con 0x5614d46d86e0 2024-08-19T01:36:20.477 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:20.475+0000 7f1e49a8e740 1 -- 172.21.15.83:0/1776405755 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5614d46d86e0 msgr2=0x5614d46d8ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:36:20.477 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:20.475+0000 7f1e49a8e740 1 --2- 172.21.15.83:0/1776405755 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5614d46d86e0 0x5614d46d8ac0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f1e30097c40 tx=0x7f1e3009b140 comp rx=0 tx=0).stop 2024-08-19T01:36:20.477 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:20.476+0000 7f1e49a8e740 1 -- 172.21.15.83:0/1776405755 shutdown_connections 2024-08-19T01:36:20.477 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:20.476+0000 7f1e49a8e740 1 --2- 172.21.15.83:0/1776405755 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5614d46e38c0 0x5614d46e5cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:20.477 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:20.476+0000 7f1e49a8e740 1 --2- 172.21.15.83:0/1776405755 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5614d46d9000 0x5614d46e3380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:20.478 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:20.476+0000 7f1e49a8e740 1 --2- 172.21.15.83:0/1776405755 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5614d46d86e0 0x5614d46d8ac0 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:20.478 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:20.476+0000 7f1e49a8e740 1 -- 172.21.15.83:0/1776405755 >> 172.21.15.83:0/1776405755 conn(0x5614d46d4c70 msgr2=0x5614d46d5050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:36:20.478 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:20.476+0000 7f1e49a8e740 1 -- 172.21.15.83:0/1776405755 shutdown_connections 2024-08-19T01:36:20.478 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:20.476+0000 7f1e49a8e740 1 -- 172.21.15.83:0/1776405755 wait complete. 2024-08-19T01:36:20.478 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:20.476+0000 7f1e49a8e740 1 -- start start 2024-08-19T01:36:20.479 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:20.477+0000 7f1e49a8e740 1 Processor -- start 2024-08-19T01:36:20.480 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:20.478+0000 7f1e49a8e740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5614d46d86e0 0x5614d46e0780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:36:20.480 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:20.478+0000 7f1e49a8e740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5614d46d9000 0x5614d46e0cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:36:20.480 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:20.478+0000 7f1e49a8e740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5614d46e38c0 0x5614d46e1200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:36:20.480 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:20.478+0000 7f1e49a8e740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x5614d46e6490 con 0x5614d46e38c0 2024-08-19T01:36:20.480 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:20.478+0000 7f1e49a8e740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x5614d45f8cf0 con 0x5614d46d86e0 2024-08-19T01:36:20.480 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:20.478+0000 7f1e49a8e740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x5614d46e6610 con 0x5614d46d9000 2024-08-19T01:36:20.480 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:20.478+0000 7f1e49a8e740 10 client.0 _ll_register_callbacks cb 0x5614d45e4e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-19T01:36:20.481 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:20.478+0000 7f1e37400640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5614d46d9000 0x5614d46e0cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:36:20.481 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:20.478+0000 7f1e37400640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5614d46d9000 0x5614d46e0cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.83:38826/0 (socket says 172.21.15.83:38826) 2024-08-19T01:36:20.481 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:20.478+0000 7f1e37400640 1 -- 172.21.15.83:0/692035897 learned_addr learned my addr 172.21.15.83:0/692035897 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-08-19T01:36:20.481 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:20.478+0000 7f1e45000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5614d46e38c0 0x5614d46e1200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:36:20.481 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:20.479+0000 7f1e3fe00640 1 --2- 172.21.15.83:0/692035897 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5614d46d86e0 0x5614d46e0780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:36:20.481 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:20.479+0000 7f1e37400640 1 -- 172.21.15.83:0/692035897 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5614d46d86e0 msgr2=0x5614d46e0780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:36:20.481 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:20.479+0000 7f1e37400640 1 --2- 172.21.15.83:0/692035897 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5614d46d86e0 0x5614d46e0780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:20.481 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:20.479+0000 7f1e37400640 1 -- 172.21.15.83:0/692035897 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5614d46e38c0 msgr2=0x5614d46e1200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:36:20.481 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:20.479+0000 7f1e37400640 1 --2- 172.21.15.83:0/692035897 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5614d46e38c0 0x5614d46e1200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:20.481 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:20.479+0000 7f1e37400640 1 -- 172.21.15.83:0/692035897 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5614d43af3c0 con 0x5614d46d9000 2024-08-19T01:36:20.481 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:20.479+0000 7f1e45000640 1 --2- 172.21.15.83:0/692035897 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5614d46e38c0 0x5614d46e1200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-19T01:36:20.482 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:20.479+0000 7f1e37400640 1 --2- 172.21.15.83:0/692035897 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5614d46d9000 0x5614d46e0cc0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f1e28009be0 tx=0x7f1e28009500 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4c71c6aa518e01cc server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:36:20.482 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:20.479+0000 7f1e2e000640 10 client.0 ms_handle_connect on v2:172.21.15.167:3300/0 2024-08-19T01:36:20.482 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:20.480+0000 7f1e2e000640 1 -- 172.21.15.83:0/692035897 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e2804f020 con 0x5614d46d9000 2024-08-19T01:36:20.483 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:20.480+0000 7f1e2e000640 1 -- 172.21.15.83:0/692035897 <== mon.1 v2:172.21.15.167:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8269+0+0 (secure 0 0 0) 0x7f1e2803d660 con 0x5614d46d9000 2024-08-19T01:36:20.483 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:20.480+0000 7f1e49a8e740 1 -- 172.21.15.83:0/692035897 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1e300a4450 con 0x5614d46d9000 2024-08-19T01:36:20.483 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:20.481+0000 7f1e2e000640 1 -- 172.21.15.83:0/692035897 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f1e28036040 con 0x5614d46d9000 2024-08-19T01:36:20.483 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:20.481+0000 7f1e49a8e740 10 client.25399 fetch_fsmap learned FSMap version 25 2024-08-19T01:36:20.483 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:20.481+0000 7f1e49a8e740 1 -- 172.21.15.83:0/692035897 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5614d46f2110 con 0x5614d46d9000 2024-08-19T01:36:20.484 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:20.481+0000 7f1e2e000640 1 -- 172.21.15.83:0/692035897 <== mon.1 v2:172.21.15.167:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1e300a4450 con 0x5614d46d9000 2024-08-19T01:36:20.484 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:20.482+0000 7f1e2e000640 1 -- 172.21.15.83:0/692035897 <== mon.1 v2:172.21.15.167:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f1e2805c020 con 0x5614d46d9000 2024-08-19T01:36:20.484 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:20.482+0000 7f1e49a8e740 10 client.25399 fetch_fsmap finished waiting for FSMap version 25 2024-08-19T01:36:20.484 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:20.482+0000 7f1e49a8e740 10 client.25399 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-19T01:36:20.484 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:20.483+0000 7f1e49a8e740 20 client.25399 populate_metadata read hostname 'smithi083' 2024-08-19T01:36:20.484 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:20.483+0000 7f1e49a8e740 1 --2- 172.21.15.83:0/692035897 >> [v2:172.21.15.117:6838/1967595704,v1:172.21.15.117:6839/1967595704] conn(0x5614d46f3670 0x5614d46f5ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:36:20.484 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:20.483+0000 7f1e49a8e740 4 client.25399 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-19T01:36:20.485 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:20.483+0000 7f1e49a8e740 1 -- 172.21.15.83:0/692035897 --> [v2:172.21.15.117:6838/1967595704,v1:172.21.15.117:6839/1967595704] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5614d464b160 con 0x5614d46f3670 2024-08-19T01:36:20.485 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:20.483+0000 7f1e3fe00640 1 --2- 172.21.15.83:0/692035897 >> [v2:172.21.15.117:6838/1967595704,v1:172.21.15.117:6839/1967595704] conn(0x5614d46f3670 0x5614d46f5ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:36:20.485 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:20.484+0000 7f1e3fe00640 1 --2- 172.21.15.83:0/692035897 >> [v2:172.21.15.117:6838/1967595704,v1:172.21.15.117:6839/1967595704] conn(0x5614d46f3670 0x5614d46f5ab0 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1ab7cc51c6af62c3 server_cookie=a127e09ffd7b80bf in_seq=0 out_seq=0 2024-08-19T01:36:20.485 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:20.484+0000 7f1e2e000640 10 client.25399 ms_handle_connect on v2:172.21.15.117:6838/1967595704 2024-08-19T01:36:20.490 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:20.488+0000 7f1e2e000640 1 -- 172.21.15.83:0/692035897 <== mds.0 v2:172.21.15.117:6838/1967595704 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5614d464b160 con 0x5614d46f3670 2024-08-19T01:36:20.490 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:20.488+0000 7f1e2e000640 10 client.25399 handle_command_reply: tid=0 2024-08-19T01:36:20.490 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:20.488+0000 7f1e49a8e740 1 client.25399 shutdown 2024-08-19T01:36:20.491 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:20.489+0000 7f1e49a8e740 1 -- 172.21.15.83:0/692035897 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5614d46d9000 msgr2=0x5614d46e0cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:36:20.491 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:20.489+0000 7f1e49a8e740 1 --2- 172.21.15.83:0/692035897 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5614d46d9000 0x5614d46e0cc0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f1e28009be0 tx=0x7f1e28009500 comp rx=0 tx=0).stop 2024-08-19T01:36:20.491 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:20.489+0000 7f1e49a8e740 1 -- 172.21.15.83:0/692035897 shutdown_connections 2024-08-19T01:36:20.491 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:20.489+0000 7f1e49a8e740 1 --2- 172.21.15.83:0/692035897 >> [v2:172.21.15.117:6838/1967595704,v1:172.21.15.117:6839/1967595704] conn(0x5614d46f3670 0x5614d46f5ab0 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:20.491 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:20.489+0000 7f1e49a8e740 1 --2- 172.21.15.83:0/692035897 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5614d46e38c0 0x5614d46e1200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:20.491 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:20.489+0000 7f1e49a8e740 1 --2- 172.21.15.83:0/692035897 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5614d46d9000 0x5614d46e0cc0 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:20.491 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:20.489+0000 7f1e49a8e740 1 --2- 172.21.15.83:0/692035897 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5614d46d86e0 0x5614d46e0780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:20.491 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:20.489+0000 7f1e49a8e740 1 -- 172.21.15.83:0/692035897 >> 172.21.15.83:0/692035897 conn(0x5614d46d4c70 msgr2=0x5614d46da9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:36:20.491 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:20.489+0000 7f1e2e000640 0 client.25399 ms_handle_reset on v2:172.21.15.117:6838/1967595704 2024-08-19T01:36:20.492 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:20.490+0000 7f1e49a8e740 1 -- 172.21.15.83:0/692035897 shutdown_connections 2024-08-19T01:36:20.492 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:20.490+0000 7f1e49a8e740 1 -- 172.21.15.83:0/692035897 wait complete. 2024-08-19T01:36:20.492 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:20.490+0000 7f1e49a8e740 20 client.25399 trim_cache size 0 max 16384 2024-08-19T01:36:20.501 INFO:tasks.workunit.client.0.smithi083.stdout: 16 218565 773 173281 871 73444 1250 2024-08-19T01:36:20.511 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:20.508+0000 7f1e49a8e740 1 Processor -- start 2024-08-19T01:36:20.511 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:20.509+0000 7f1e49a8e740 1 -- start start 2024-08-19T01:36:20.511 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:20.509+0000 7f1e49a8e740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5614d46d86e0 0x5614d46db140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:36:20.511 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:20.509+0000 7f1e49a8e740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5614d46d9000 0x5614d46dbac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:36:20.511 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:20.509+0000 7f1e49a8e740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5614d46e38c0 0x5614d46fb380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:36:20.511 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:20.509+0000 7f1e49a8e740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f1e401be970 con 0x5614d46e38c0 2024-08-19T01:36:20.511 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:20.509+0000 7f1e49a8e740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f1e401bc240 con 0x5614d46d86e0 2024-08-19T01:36:20.511 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:20.509+0000 7f1e3fe00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5614d46d86e0 0x5614d46db140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:36:20.511 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:20.509+0000 7f1e49a8e740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f1e401ba090 con 0x5614d46d9000 2024-08-19T01:36:20.512 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:20.509+0000 7f1e3fe00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5614d46d86e0 0x5614d46db140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.83:53410/0 (socket says 172.21.15.83:53410) 2024-08-19T01:36:20.512 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:20.509+0000 7f1e3fe00640 1 -- 172.21.15.83:0/2984104412 learned_addr learned my addr 172.21.15.83:0/2984104412 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-08-19T01:36:20.512 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:20.509+0000 7f1e37400640 1 --2- 172.21.15.83:0/2984104412 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5614d46d9000 0x5614d46dbac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:36:20.512 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:20.510+0000 7f1e45000640 1 --2- 172.21.15.83:0/2984104412 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5614d46e38c0 0x5614d46fb380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:36:20.512 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:20.510+0000 7f1e3fe00640 1 -- 172.21.15.83:0/2984104412 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5614d46d9000 msgr2=0x5614d46dbac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:36:20.512 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:20.510+0000 7f1e3fe00640 1 --2- 172.21.15.83:0/2984104412 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5614d46d9000 0x5614d46dbac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:20.512 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:20.510+0000 7f1e3fe00640 1 -- 172.21.15.83:0/2984104412 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5614d46e38c0 msgr2=0x5614d46fb380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:36:20.513 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:20.510+0000 7f1e3fe00640 1 --2- 172.21.15.83:0/2984104412 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5614d46e38c0 0x5614d46fb380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:20.513 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:20.510+0000 7f1e3fe00640 1 -- 172.21.15.83:0/2984104412 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5614d46d31f0 con 0x5614d46d86e0 2024-08-19T01:36:20.513 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:20.510+0000 7f1e45000640 1 --2- 172.21.15.83:0/2984104412 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5614d46e38c0 0x5614d46fb380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-19T01:36:20.513 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:20.510+0000 7f1e3fe00640 1 --2- 172.21.15.83:0/2984104412 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5614d46d86e0 0x5614d46db140 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f1e300ccad0 tx=0x7f1e30010ea0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=395e846cf316cca7 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:36:20.513 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:20.510+0000 7f1e34c00640 1 -- 172.21.15.83:0/2984104412 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e300860e0 con 0x5614d46d86e0 2024-08-19T01:36:20.513 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:20.510+0000 7f1e34c00640 1 -- 172.21.15.83:0/2984104412 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1e30099880 con 0x5614d46d86e0 2024-08-19T01:36:20.513 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:20.510+0000 7f1e34c00640 1 -- 172.21.15.83:0/2984104412 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e3009ac30 con 0x5614d46d86e0 2024-08-19T01:36:20.514 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:20.511+0000 7f1e49a8e740 1 -- 172.21.15.83:0/2984104412 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5614d46d86e0 msgr2=0x5614d46db140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:36:20.514 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:20.511+0000 7f1e49a8e740 1 --2- 172.21.15.83:0/2984104412 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5614d46d86e0 0x5614d46db140 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f1e300ccad0 tx=0x7f1e30010ea0 comp rx=0 tx=0).stop 2024-08-19T01:36:20.514 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:20.512+0000 7f1e49a8e740 1 -- 172.21.15.83:0/2984104412 shutdown_connections 2024-08-19T01:36:20.514 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:20.512+0000 7f1e49a8e740 1 --2- 172.21.15.83:0/2984104412 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5614d46e38c0 0x5614d46fb380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:20.514 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:20.512+0000 7f1e49a8e740 1 --2- 172.21.15.83:0/2984104412 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5614d46d9000 0x5614d46dbac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:20.514 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:20.512+0000 7f1e49a8e740 1 --2- 172.21.15.83:0/2984104412 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5614d46d86e0 0x5614d46db140 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:20.514 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:20.512+0000 7f1e49a8e740 1 -- 172.21.15.83:0/2984104412 >> 172.21.15.83:0/2984104412 conn(0x5614d46d4c70 msgr2=0x5614d46df890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:36:20.514 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:20.512+0000 7f1e49a8e740 1 -- 172.21.15.83:0/2984104412 shutdown_connections 2024-08-19T01:36:20.514 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:20.512+0000 7f1e49a8e740 1 -- 172.21.15.83:0/2984104412 wait complete. 2024-08-19T01:36:20.514 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:20.512+0000 7f1e49a8e740 1 -- start start 2024-08-19T01:36:20.515 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:20.513+0000 7f1e49a8e740 1 Processor -- start 2024-08-19T01:36:20.517 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:20.514+0000 7f1e49a8e740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5614d46d86e0 0x5614d46eba40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:36:20.517 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:20.514+0000 7f1e49a8e740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5614d46d9000 0x5614d46ebfa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:36:20.517 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:20.514+0000 7f1e49a8e740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5614d46e38c0 0x5614d46dd030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:36:20.517 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:20.514+0000 7f1e49a8e740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f1e401ba700 con 0x5614d46d86e0 2024-08-19T01:36:20.517 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:20.514+0000 7f1e49a8e740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f1e401bbb70 con 0x5614d46e38c0 2024-08-19T01:36:20.517 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:20.514+0000 7f1e49a8e740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f1e401badd0 con 0x5614d46d9000 2024-08-19T01:36:20.518 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:20.514+0000 7f1e49a8e740 10 client.0 _ll_register_callbacks cb 0x7f1e28009a60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-19T01:36:20.518 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:20.514+0000 7f1e45000640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5614d46e38c0 0x5614d46dd030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:36:20.518 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:20.514+0000 7f1e45000640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5614d46e38c0 0x5614d46dd030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.83:53420/0 (socket says 172.21.15.83:53420) 2024-08-19T01:36:20.518 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:20.514+0000 7f1e45000640 1 -- 172.21.15.83:0/443304709 learned_addr learned my addr 172.21.15.83:0/443304709 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-08-19T01:36:20.518 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:20.514+0000 7f1e3fe00640 1 --2- 172.21.15.83:0/443304709 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5614d46d86e0 0x5614d46eba40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:36:20.518 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:20.515+0000 7f1e37400640 1 --2- 172.21.15.83:0/443304709 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5614d46d9000 0x5614d46ebfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:36:20.518 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:20.515+0000 7f1e45000640 1 -- 172.21.15.83:0/443304709 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5614d46d9000 msgr2=0x5614d46ebfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:36:20.518 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:20.515+0000 7f1e45000640 1 --2- 172.21.15.83:0/443304709 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5614d46d9000 0x5614d46ebfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:20.518 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:20.515+0000 7f1e45000640 1 -- 172.21.15.83:0/443304709 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5614d46d86e0 msgr2=0x5614d46eba40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:36:20.518 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:20.515+0000 7f1e45000640 1 --2- 172.21.15.83:0/443304709 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5614d46d86e0 0x5614d46eba40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:20.518 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:20.515+0000 7f1e45000640 1 -- 172.21.15.83:0/443304709 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5614d46dd860 con 0x5614d46e38c0 2024-08-19T01:36:20.518 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:20.515+0000 7f1e3fe00640 1 --2- 172.21.15.83:0/443304709 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5614d46d86e0 0x5614d46eba40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-19T01:36:20.518 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:20.515+0000 7f1e45000640 1 --2- 172.21.15.83:0/443304709 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5614d46e38c0 0x5614d46dd030 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f1e38012840 tx=0x7f1e38012d00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=391f7e8da0ddfcf1 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:36:20.519 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:20.515+0000 7f1e2e000640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-08-19T01:36:20.519 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:20.515+0000 7f1e2e000640 1 -- 172.21.15.83:0/443304709 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e38011c70 con 0x5614d46e38c0 2024-08-19T01:36:20.519 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:20.515+0000 7f1e49a8e740 1 -- 172.21.15.83:0/443304709 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1e30099880 con 0x5614d46e38c0 2024-08-19T01:36:20.519 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:20.516+0000 7f1e2e000640 1 -- 172.21.15.83:0/443304709 <== mon.2 v2:172.21.15.117:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8269+0+0 (secure 0 0 0) 0x7f1e38011e10 con 0x5614d46e38c0 2024-08-19T01:36:20.519 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:20.516+0000 7f1e2e000640 1 -- 172.21.15.83:0/443304709 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f1e38004020 con 0x5614d46e38c0 2024-08-19T01:36:20.519 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:20.516+0000 7f1e49a8e740 10 client.25439 fetch_fsmap learned FSMap version 25 2024-08-19T01:36:20.519 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:20.516+0000 7f1e49a8e740 1 -- 172.21.15.83:0/443304709 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5614d475dea0 con 0x5614d46e38c0 2024-08-19T01:36:20.519 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:20.516+0000 7f1e2e000640 1 -- 172.21.15.83:0/443304709 <== mon.2 v2:172.21.15.117:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1e30099880 con 0x5614d46e38c0 2024-08-19T01:36:20.520 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:20.516+0000 7f1e2e000640 1 -- 172.21.15.83:0/443304709 <== mon.2 v2:172.21.15.117:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f1e38028460 con 0x5614d46e38c0 2024-08-19T01:36:20.520 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:20.516+0000 7f1e49a8e740 10 client.25439 fetch_fsmap finished waiting for FSMap version 25 2024-08-19T01:36:20.520 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:20.516+0000 7f1e49a8e740 10 client.25439 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-19T01:36:20.520 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:20.516+0000 7f1e49a8e740 20 client.25439 populate_metadata read hostname 'smithi083' 2024-08-19T01:36:20.520 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:20.516+0000 7f1e49a8e740 1 --2- 172.21.15.83:0/443304709 >> [v2:172.21.15.117:6838/1967595704,v1:172.21.15.117:6839/1967595704] conn(0x5614d475e670 0x5614d4760a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:36:20.520 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:20.516+0000 7f1e49a8e740 4 client.25439 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-08-19T01:36:20.520 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:20.516+0000 7f1e49a8e740 1 -- 172.21.15.83:0/443304709 --> [v2:172.21.15.117:6838/1967595704,v1:172.21.15.117:6839/1967595704] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5614d4645670 con 0x5614d475e670 2024-08-19T01:36:20.520 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:20.517+0000 7f1e3fe00640 1 --2- 172.21.15.83:0/443304709 >> [v2:172.21.15.117:6838/1967595704,v1:172.21.15.117:6839/1967595704] conn(0x5614d475e670 0x5614d4760a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:36:20.520 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:20.517+0000 7f1e3fe00640 1 --2- 172.21.15.83:0/443304709 >> [v2:172.21.15.117:6838/1967595704,v1:172.21.15.117:6839/1967595704] conn(0x5614d475e670 0x5614d4760a90 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bdc0e15f0d17d3f0 server_cookie=b4bcc41c56ee0c52 in_seq=0 out_seq=0 2024-08-19T01:36:20.520 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:20.517+0000 7f1e2e000640 10 client.25439 ms_handle_connect on v2:172.21.15.117:6838/1967595704 2024-08-19T01:36:20.522 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:20.521+0000 7f1e2e000640 1 -- 172.21.15.83:0/443304709 <== mds.0 v2:172.21.15.117:6838/1967595704 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5614d4645670 con 0x5614d475e670 2024-08-19T01:36:20.523 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:20.521+0000 7f1e2e000640 10 client.25439 handle_command_reply: tid=0 2024-08-19T01:36:20.523 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:20.521+0000 7f1e49a8e740 1 client.25439 shutdown 2024-08-19T01:36:20.523 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:20.521+0000 7f1e49a8e740 1 -- 172.21.15.83:0/443304709 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5614d46e38c0 msgr2=0x5614d46dd030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:36:20.523 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:20.521+0000 7f1e49a8e740 1 --2- 172.21.15.83:0/443304709 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5614d46e38c0 0x5614d46dd030 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f1e38012840 tx=0x7f1e38012d00 comp rx=0 tx=0).stop 2024-08-19T01:36:20.523 INFO:teuthology.orchestra.run.smithi083.stdout:[] 2024-08-19T01:36:20.524 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:20.521+0000 7f1e49a8e740 1 -- 172.21.15.83:0/443304709 shutdown_connections 2024-08-19T01:36:20.524 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:20.521+0000 7f1e49a8e740 1 --2- 172.21.15.83:0/443304709 >> [v2:172.21.15.117:6838/1967595704,v1:172.21.15.117:6839/1967595704] conn(0x5614d475e670 0x5614d4760a90 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:20.524 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:20.521+0000 7f1e49a8e740 1 --2- 172.21.15.83:0/443304709 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5614d46e38c0 0x5614d46dd030 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:20.524 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:20.521+0000 7f1e49a8e740 1 --2- 172.21.15.83:0/443304709 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5614d46d9000 0x5614d46ebfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:20.524 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:20.521+0000 7f1e2e000640 0 client.25439 ms_handle_reset on v2:172.21.15.117:6838/1967595704 2024-08-19T01:36:20.524 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:20.521+0000 7f1e49a8e740 1 --2- 172.21.15.83:0/443304709 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5614d46d86e0 0x5614d46eba40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:20.524 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:20.521+0000 7f1e49a8e740 1 -- 172.21.15.83:0/443304709 >> 172.21.15.83:0/443304709 conn(0x5614d46d4c70 msgr2=0x5614d46e1fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:36:20.524 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:20.521+0000 7f1e49a8e740 1 -- 172.21.15.83:0/443304709 shutdown_connections 2024-08-19T01:36:20.524 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:20.521+0000 7f1e49a8e740 1 -- 172.21.15.83:0/443304709 wait complete. 2024-08-19T01:36:20.525 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:20.522+0000 7f1e49a8e740 20 client.25439 trim_cache size 0 max 16384 2024-08-19T01:36:20.525 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:20.522+0000 7f1e45a00640 1 -- 172.21.15.83:0/72786262 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f1e240a2e50 msgr2=0x7f1e240a5310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:36:20.525 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:20.522+0000 7f1e45a00640 1 --2- 172.21.15.83:0/72786262 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f1e240a2e50 0x7f1e240a5310 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f1e40114770 tx=0x7f1e28005e30 comp rx=0 tx=0).stop 2024-08-19T01:36:20.525 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:20.522+0000 7f1e45a00640 1 -- 172.21.15.83:0/72786262 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1e4006e220 msgr2=0x7f1e401acb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:36:20.525 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:20.522+0000 7f1e45a00640 1 --2- 172.21.15.83:0/72786262 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1e4006e220 0x7f1e401acb50 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f1e3000e790 tx=0x7f1e3000ec50 comp rx=0 tx=0).stop 2024-08-19T01:36:20.525 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:20.523+0000 7f1e45a00640 1 -- 172.21.15.83:0/72786262 shutdown_connections 2024-08-19T01:36:20.525 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:20.523+0000 7f1e45a00640 1 --2- 172.21.15.83:0/72786262 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f1e240a2e50 0x7f1e240a5310 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:20.526 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:20.523+0000 7f1e45a00640 1 --2- 172.21.15.83:0/72786262 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1e401147e0 0x7f1e40113510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:20.526 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:20.523+0000 7f1e45a00640 1 --2- 172.21.15.83:0/72786262 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1e4006eb40 0x7f1e401ad090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:20.526 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:20.523+0000 7f1e45a00640 1 --2- 172.21.15.83:0/72786262 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1e4006e220 0x7f1e401acb50 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:20.526 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:20.523+0000 7f1e45a00640 1 -- 172.21.15.83:0/72786262 >> 172.21.15.83:0/72786262 conn(0x7f1e40072eb0 msgr2=0x7f1e4006c8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:36:20.526 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:20.523+0000 7f1e45a00640 1 -- 172.21.15.83:0/72786262 shutdown_connections 2024-08-19T01:36:20.526 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:20.523+0000 7f1e45a00640 1 -- 172.21.15.83:0/72786262 wait complete. 2024-08-19T01:36:21.789 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:36:21 smithi083 ceph-mon[17120]: pgmap v384: 97 pgs: 97 active+clean; 29 MiB data, 473 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.4 MiB/s wr, 144 op/s 2024-08-19T01:36:21.952 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:36:21 smithi117 ceph-mon[22016]: pgmap v384: 97 pgs: 97 active+clean; 29 MiB data, 473 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.4 MiB/s wr, 144 op/s 2024-08-19T01:36:21.966 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:36:21 smithi167 ceph-mon[21875]: pgmap v384: 97 pgs: 97 active+clean; 29 MiB data, 473 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.4 MiB/s wr, 144 op/s 2024-08-19T01:36:22.322 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-08-19T01:36:22.322 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833 shell --fsid bad8780e-5dc9-11ef-bccf-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-08-19T01:36:22.707 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/bad8780e-5dc9-11ef-bccf-c7b262605968/mon.a/config 2024-08-19T01:36:23.219 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:23.212+0000 7f92e2a00640 1 -- 172.21.15.83:0/498178711 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f92dc104a20 msgr2=0x7f92dc106e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:36:23.219 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:23.212+0000 7f92e2a00640 1 --2- 172.21.15.83:0/498178711 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f92dc104a20 0x7f92dc106e10 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f92c8009900 tx=0x7f92c802a410 comp rx=0 tx=0).stop 2024-08-19T01:36:23.219 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:23.212+0000 7f92e2a00640 1 -- 172.21.15.83:0/498178711 shutdown_connections 2024-08-19T01:36:23.219 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:23.212+0000 7f92e2a00640 1 --2- 172.21.15.83:0/498178711 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f92dc107350 0x7f92dc1097c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:23.219 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:23.212+0000 7f92e2a00640 1 --2- 172.21.15.83:0/498178711 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f92dc104a20 0x7f92dc106e10 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:23.219 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:23.212+0000 7f92e2a00640 1 --2- 172.21.15.83:0/498178711 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f92dc1020f0 0x7f92dc1044e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:23.219 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:23.212+0000 7f92e2a00640 1 -- 172.21.15.83:0/498178711 >> 172.21.15.83:0/498178711 conn(0x7f92dc0fbcd0 msgr2=0x7f92dc0fe110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:36:23.220 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:23.212+0000 7f92e2a00640 1 -- 172.21.15.83:0/498178711 shutdown_connections 2024-08-19T01:36:23.220 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:23.212+0000 7f92e2a00640 1 -- 172.21.15.83:0/498178711 wait complete. 2024-08-19T01:36:23.221 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:23.218+0000 7f92e2a00640 1 Processor -- start 2024-08-19T01:36:23.221 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:23.218+0000 7f92e2a00640 1 -- start start 2024-08-19T01:36:23.222 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:23.218+0000 7f92e2a00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f92dc1020f0 0x7f92dc199d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:36:23.222 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:23.218+0000 7f92e2a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f92dc104a20 0x7f92dc19a250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:36:23.222 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:23.218+0000 7f92e2a00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f92dc107350 0x7f92dc1a1280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:36:23.222 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:23.218+0000 7f92e2a00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f92dc10d3f0 con 0x7f92dc104a20 2024-08-19T01:36:23.222 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:23.218+0000 7f92e2a00640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f92dc10d270 con 0x7f92dc1020f0 2024-08-19T01:36:23.222 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:23.218+0000 7f92e2a00640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f92dc10d570 con 0x7f92dc107350 2024-08-19T01:36:23.222 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:23.218+0000 7f92e1600640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f92dc1020f0 0x7f92dc199d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:36:23.222 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:23.218+0000 7f92e0c00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f92dc104a20 0x7f92dc19a250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:36:23.222 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:23.218+0000 7f92e1600640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f92dc1020f0 0x7f92dc199d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.83:33764/0 (socket says 172.21.15.83:33764) 2024-08-19T01:36:23.222 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:23.218+0000 7f92e1600640 1 -- 172.21.15.83:0/4085006560 learned_addr learned my addr 172.21.15.83:0/4085006560 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-08-19T01:36:23.222 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:23.218+0000 7f92e0c00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f92dc104a20 0x7f92dc19a250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:53456/0 (socket says 172.21.15.83:53456) 2024-08-19T01:36:23.223 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:23.218+0000 7f92e0c00640 1 -- 172.21.15.83:0/4085006560 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f92dc107350 msgr2=0x7f92dc1a1280 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-08-19T01:36:23.223 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:23.218+0000 7f92e0c00640 1 --2- 172.21.15.83:0/4085006560 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f92dc107350 0x7f92dc1a1280 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:23.223 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:23.218+0000 7f92e0c00640 1 -- 172.21.15.83:0/4085006560 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f92dc1020f0 msgr2=0x7f92dc199d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:36:23.223 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:23.218+0000 7f92e0c00640 1 --2- 172.21.15.83:0/4085006560 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f92dc1020f0 0x7f92dc199d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:23.223 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:23.218+0000 7f92e0c00640 1 -- 172.21.15.83:0/4085006560 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f92dc1a1920 con 0x7f92dc104a20 2024-08-19T01:36:23.223 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:23.218+0000 7f92e0c00640 1 --2- 172.21.15.83:0/4085006560 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f92dc104a20 0x7f92dc19a250 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f92c8009230 tx=0x7f92c8032ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:36:23.223 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:23.218+0000 7f92da000640 1 -- 172.21.15.83:0/4085006560 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92c803f030 con 0x7f92dc104a20 2024-08-19T01:36:23.224 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:23.218+0000 7f92da000640 1 -- 172.21.15.83:0/4085006560 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f92c8003ce0 con 0x7f92dc104a20 2024-08-19T01:36:23.224 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:23.218+0000 7f92da000640 1 -- 172.21.15.83:0/4085006560 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92c8035070 con 0x7f92dc104a20 2024-08-19T01:36:23.224 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:23.218+0000 7f92e2a00640 1 -- 172.21.15.83:0/4085006560 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f92dc1a1bb0 con 0x7f92dc104a20 2024-08-19T01:36:23.224 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:23.218+0000 7f92e2a00640 1 -- 172.21.15.83:0/4085006560 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f92dc1a20c0 con 0x7f92dc104a20 2024-08-19T01:36:23.225 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:23.223+0000 7f92e6fd7740 1 Processor -- start 2024-08-19T01:36:23.225 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:23.223+0000 7f92e6fd7740 1 -- start start 2024-08-19T01:36:23.226 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:23.223+0000 7f92e6fd7740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559d52835680 0x559d52835a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:36:23.226 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:23.223+0000 7f92e6fd7740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559d52835fc0 0x559d5283aa80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:36:23.226 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:23.223+0000 7f92e6fd7740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559d5283b050 0x559d5283d510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:36:23.226 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:23.223+0000 7f92e6fd7740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x559d52799890 con 0x559d52835680 2024-08-19T01:36:23.226 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:23.223+0000 7f92e6fd7740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x559d52731d90 con 0x559d5283b050 2024-08-19T01:36:23.226 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:23.223+0000 7f92e6fd7740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x559d52795850 con 0x559d52835fc0 2024-08-19T01:36:23.226 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:23.223+0000 7f92e2000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559d52835680 0x559d52835a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:36:23.226 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:23.223+0000 7f92e2000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559d52835680 0x559d52835a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:53464/0 (socket says 172.21.15.83:53464) 2024-08-19T01:36:23.226 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:23.223+0000 7f92e2000640 1 -- 172.21.15.83:0/3821258091 learned_addr learned my addr 172.21.15.83:0/3821258091 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-08-19T01:36:23.226 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:23.223+0000 7f92e1600640 1 --2- 172.21.15.83:0/3821258091 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559d52835fc0 0x559d5283aa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:36:23.226 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:23.223+0000 7f92da000640 1 -- 172.21.15.83:0/4085006560 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7f92c8042030 con 0x7f92dc104a20 2024-08-19T01:36:23.227 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:23.223+0000 7f92e2000640 1 -- 172.21.15.83:0/3821258091 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559d52835fc0 msgr2=0x559d5283aa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:36:23.227 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:23.223+0000 7f92e2000640 1 --2- 172.21.15.83:0/3821258091 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559d52835fc0 0x559d5283aa80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:23.227 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:23.223+0000 7f92e2000640 1 -- 172.21.15.83:0/3821258091 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559d5283b050 msgr2=0x559d5283d510 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-08-19T01:36:23.227 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:23.223+0000 7f92e2000640 1 --2- 172.21.15.83:0/3821258091 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559d5283b050 0x559d5283d510 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:23.227 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:23.223+0000 7f92e2000640 1 -- 172.21.15.83:0/3821258091 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559d524ad210 con 0x559d52835680 2024-08-19T01:36:23.230 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:23.223+0000 7f92e2000640 1 --2- 172.21.15.83:0/3821258091 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559d52835680 0x559d52835a80 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f92d4009e20 tx=0x7f92d400e720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dd74c84e464585d7 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:36:23.230 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:23.223+0000 7f92ce600640 1 -- 172.21.15.83:0/3821258091 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92d4003b70 con 0x559d52835680 2024-08-19T01:36:23.230 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:23.223+0000 7f92ce600640 1 -- 172.21.15.83:0/3821258091 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f92d400ee70 con 0x559d52835680 2024-08-19T01:36:23.230 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:23.223+0000 7f92ce600640 1 -- 172.21.15.83:0/3821258091 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92d40188d0 con 0x559d52835680 2024-08-19T01:36:23.230 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:23.223+0000 7f92e6fd7740 1 -- 172.21.15.83:0/3821258091 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559d52835680 msgr2=0x559d52835a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:36:23.230 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:23.223+0000 7f92e6fd7740 1 --2- 172.21.15.83:0/3821258091 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559d52835680 0x559d52835a80 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f92d4009e20 tx=0x7f92d400e720 comp rx=0 tx=0).stop 2024-08-19T01:36:23.231 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:23.223+0000 7f92e6fd7740 1 -- 172.21.15.83:0/3821258091 shutdown_connections 2024-08-19T01:36:23.231 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:23.223+0000 7f92e6fd7740 1 --2- 172.21.15.83:0/3821258091 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559d5283b050 0x559d5283d510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:23.231 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:23.223+0000 7f92e6fd7740 1 --2- 172.21.15.83:0/3821258091 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559d52835fc0 0x559d5283aa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:23.231 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:23.223+0000 7f92e6fd7740 1 --2- 172.21.15.83:0/3821258091 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559d52835680 0x559d52835a80 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:23.231 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:23.223+0000 7f92e6fd7740 1 -- 172.21.15.83:0/3821258091 >> 172.21.15.83:0/3821258091 conn(0x559d52830cf0 msgr2=0x559d528310f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:36:23.231 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:23.223+0000 7f92e6fd7740 1 -- 172.21.15.83:0/3821258091 shutdown_connections 2024-08-19T01:36:23.231 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:23.223+0000 7f92e6fd7740 1 -- 172.21.15.83:0/3821258091 wait complete. 2024-08-19T01:36:23.231 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:23.223+0000 7f92da000640 1 --2- 172.21.15.83:0/4085006560 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f92c40a2ad0 0x7f92c40a4f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:36:23.231 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:23.223+0000 7f92e6fd7740 1 -- start start 2024-08-19T01:36:23.231 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:23.223+0000 7f92da000640 1 -- 172.21.15.83:0/4085006560 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8269+0+0 (secure 0 0 0) 0x7f92c80ef600 con 0x7f92dc104a20 2024-08-19T01:36:23.232 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:23.223+0000 7f92e6fd7740 1 Processor -- start 2024-08-19T01:36:23.232 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:23.223+0000 7f92e2000640 1 --2- 172.21.15.83:0/4085006560 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f92c40a2ad0 0x7f92c40a4f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:36:23.232 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:23.223+0000 7f92e6fd7740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559d52835680 0x559d5283fa20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:36:23.232 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:23.223+0000 7f92e6fd7740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559d52835fc0 0x559d5283ff60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:36:23.232 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:23.223+0000 7f92e6fd7740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559d5283b050 0x559d52846fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:36:23.232 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:23.223+0000 7f92e6fd7740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x559d5283e0c0 con 0x559d52835fc0 2024-08-19T01:36:23.232 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:23.223+0000 7f92e6fd7740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x559d52754f70 con 0x559d5283b050 2024-08-19T01:36:23.232 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:23.223+0000 7f92e6fd7740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x559d5283e240 con 0x559d52835680 2024-08-19T01:36:23.232 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:23.223+0000 7f92e6fd7740 10 client.0 _ll_register_callbacks cb 0x559d527918d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-19T01:36:23.232 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:23.223+0000 7f92e1600640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559d52835680 0x559d5283fa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:36:23.232 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:23.223+0000 7f92e0c00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559d52835fc0 0x559d5283ff60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:36:23.232 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:23.223+0000 7f92e1600640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559d52835680 0x559d5283fa20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.83:47782/0 (socket says 172.21.15.83:47782) 2024-08-19T01:36:23.232 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:23.223+0000 7f92e1600640 1 -- 172.21.15.83:0/3283979153 learned_addr learned my addr 172.21.15.83:0/3283979153 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-08-19T01:36:23.232 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:23.223+0000 7f92e2000640 1 --2- 172.21.15.83:0/4085006560 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f92c40a2ad0 0x7f92c40a4f90 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f92d400a010 tx=0x7f92d401c040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:36:23.233 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:23.228+0000 7f92e1600640 1 -- 172.21.15.83:0/3283979153 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559d5283b050 msgr2=0x559d52846fa0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-08-19T01:36:23.233 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:23.228+0000 7f92e1600640 1 --2- 172.21.15.83:0/3283979153 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559d5283b050 0x559d52846fa0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:23.233 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:23.228+0000 7f92e1600640 1 -- 172.21.15.83:0/3283979153 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559d52835fc0 msgr2=0x559d5283ff60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:36:23.233 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:23.228+0000 7f92e1600640 1 --2- 172.21.15.83:0/3283979153 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559d52835fc0 0x559d5283ff60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:23.233 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:23.228+0000 7f92e1600640 1 -- 172.21.15.83:0/3283979153 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559d52579620 con 0x559d52835680 2024-08-19T01:36:23.233 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:23.228+0000 7f92e0c00640 1 --2- 172.21.15.83:0/3283979153 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559d52835fc0 0x559d5283ff60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-19T01:36:23.233 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:23.228+0000 7f92e1600640 1 --2- 172.21.15.83:0/3283979153 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559d52835680 0x559d5283fa20 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f92d000df30 tx=0x7f92d000f670 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2baab528fa0d08f1 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:36:23.233 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:23.228+0000 7f92c2a00640 10 client.0 ms_handle_connect on v2:172.21.15.167:3300/0 2024-08-19T01:36:23.233 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:23.228+0000 7f92c2a00640 1 -- 172.21.15.83:0/3283979153 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92d0019020 con 0x559d52835680 2024-08-19T01:36:23.233 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:23.228+0000 7f92c2a00640 1 -- 172.21.15.83:0/3283979153 <== mon.1 v2:172.21.15.167:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8269+0+0 (secure 0 0 0) 0x7f92d0014670 con 0x559d52835680 2024-08-19T01:36:23.233 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:23.228+0000 7f92e6fd7740 1 -- 172.21.15.83:0/3283979153 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f92d400ee70 con 0x559d52835680 2024-08-19T01:36:23.233 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:23.228+0000 7f92c2a00640 1 -- 172.21.15.83:0/3283979153 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f92d0024050 con 0x559d52835680 2024-08-19T01:36:23.233 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:23.228+0000 7f92e6fd7740 10 client.25411 fetch_fsmap learned FSMap version 25 2024-08-19T01:36:23.233 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:23.228+0000 7f92e6fd7740 1 -- 172.21.15.83:0/3283979153 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x559d5284a1d0 con 0x559d52835680 2024-08-19T01:36:23.234 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:23.228+0000 7f92c2a00640 1 -- 172.21.15.83:0/3283979153 <== mon.1 v2:172.21.15.167:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f92d400ee70 con 0x559d52835680 2024-08-19T01:36:23.235 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:23.228+0000 7f92c2a00640 1 -- 172.21.15.83:0/3283979153 <== mon.1 v2:172.21.15.167:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f92d0015a80 con 0x559d52835680 2024-08-19T01:36:23.235 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:23.228+0000 7f92e6fd7740 10 client.25411 fetch_fsmap finished waiting for FSMap version 25 2024-08-19T01:36:23.235 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:23.228+0000 7f92e6fd7740 10 client.25411 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-08-19T01:36:23.236 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:23.228+0000 7f92e6fd7740 20 client.25411 populate_metadata read hostname 'smithi083' 2024-08-19T01:36:23.236 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:23.228+0000 7f92e6fd7740 1 --2- 172.21.15.83:0/3283979153 >> [v2:172.21.15.117:6838/1967595704,v1:172.21.15.117:6839/1967595704] conn(0x559d5284b8c0 0x559d5284dd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:36:23.236 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:23.228+0000 7f92e6fd7740 4 client.25411 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-19T01:36:23.236 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:23.228+0000 7f92e6fd7740 1 -- 172.21.15.83:0/3283979153 --> [v2:172.21.15.117:6838/1967595704,v1:172.21.15.117:6839/1967595704] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559d527a6fc0 con 0x559d5284b8c0 2024-08-19T01:36:23.236 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:23.228+0000 7f92e0c00640 1 --2- 172.21.15.83:0/3283979153 >> [v2:172.21.15.117:6838/1967595704,v1:172.21.15.117:6839/1967595704] conn(0x559d5284b8c0 0x559d5284dd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:36:23.236 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:23.228+0000 7f92c2a00640 10 client.25411 ms_handle_connect on v2:172.21.15.117:6838/1967595704 2024-08-19T01:36:23.236 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:23.228+0000 7f92e0c00640 1 --2- 172.21.15.83:0/3283979153 >> [v2:172.21.15.117:6838/1967595704,v1:172.21.15.117:6839/1967595704] conn(0x559d5284b8c0 0x559d5284dd10 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3a6cfbdea864aab1 server_cookie=8491cf8eef5e7fdd in_seq=0 out_seq=0 2024-08-19T01:36:23.236 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:23.228+0000 7f92c2a00640 1 -- 172.21.15.83:0/3283979153 <== mds.0 v2:172.21.15.117:6838/1967595704 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x559d527a6fc0 con 0x559d5284b8c0 2024-08-19T01:36:23.236 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:23.228+0000 7f92c2a00640 10 client.25411 handle_command_reply: tid=0 2024-08-19T01:36:23.236 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:23.228+0000 7f92e6fd7740 1 client.25411 shutdown 2024-08-19T01:36:23.236 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:23.228+0000 7f92e6fd7740 1 -- 172.21.15.83:0/3283979153 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559d52835680 msgr2=0x559d5283fa20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:36:23.236 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:23.228+0000 7f92e6fd7740 1 --2- 172.21.15.83:0/3283979153 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559d52835680 0x559d5283fa20 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f92d000df30 tx=0x7f92d000f670 comp rx=0 tx=0).stop 2024-08-19T01:36:23.237 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:23.228+0000 7f92e6fd7740 1 -- 172.21.15.83:0/3283979153 shutdown_connections 2024-08-19T01:36:23.237 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:23.228+0000 7f92e6fd7740 1 --2- 172.21.15.83:0/3283979153 >> [v2:172.21.15.117:6838/1967595704,v1:172.21.15.117:6839/1967595704] conn(0x559d5284b8c0 0x559d5284dd10 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:23.237 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:23.228+0000 7f92e6fd7740 1 --2- 172.21.15.83:0/3283979153 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559d5283b050 0x559d52846fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:23.237 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:23.228+0000 7f92e6fd7740 1 --2- 172.21.15.83:0/3283979153 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559d52835fc0 0x559d5283ff60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:23.237 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:23.228+0000 7f92c2a00640 0 client.25411 ms_handle_reset on v2:172.21.15.117:6838/1967595704 2024-08-19T01:36:23.237 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:23.228+0000 7f92e6fd7740 1 --2- 172.21.15.83:0/3283979153 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559d52835680 0x559d5283fa20 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:23.237 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:23.228+0000 7f92e6fd7740 1 -- 172.21.15.83:0/3283979153 >> 172.21.15.83:0/3283979153 conn(0x559d52830cf0 msgr2=0x559d52832500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:36:23.237 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:23.228+0000 7f92e6fd7740 1 -- 172.21.15.83:0/3283979153 shutdown_connections 2024-08-19T01:36:23.237 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:23.228+0000 7f92e6fd7740 1 -- 172.21.15.83:0/3283979153 wait complete. 2024-08-19T01:36:23.238 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:23.228+0000 7f92e6fd7740 20 client.25411 trim_cache size 0 max 16384 2024-08-19T01:36:23.248 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:23.246+0000 7f92e6fd7740 1 Processor -- start 2024-08-19T01:36:23.250 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:23.246+0000 7f92e6fd7740 1 -- start start 2024-08-19T01:36:23.250 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:23.246+0000 7f92e6fd7740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559d52835680 0x559d5283f7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:36:23.250 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:23.246+0000 7f92e6fd7740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559d52835fc0 0x559d52846f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:36:23.250 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:23.246+0000 7f92e6fd7740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559d5283b050 0x559d5283fd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:36:23.250 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:23.246+0000 7f92e6fd7740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f92dc1ad7e0 con 0x559d5283b050 2024-08-19T01:36:23.250 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:23.246+0000 7f92e6fd7740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f92dc1ab0b0 con 0x559d52835680 2024-08-19T01:36:23.250 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:23.246+0000 7f92e6fd7740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f92dc1a8f00 con 0x559d52835fc0 2024-08-19T01:36:23.250 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:23.247+0000 7f92e2000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559d5283b050 0x559d5283fd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:36:23.250 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:23.247+0000 7f92e2000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559d5283b050 0x559d5283fd10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:53468/0 (socket says 172.21.15.83:53468) 2024-08-19T01:36:23.250 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:23.247+0000 7f92e2000640 1 -- 172.21.15.83:0/1969400251 learned_addr learned my addr 172.21.15.83:0/1969400251 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-08-19T01:36:23.250 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:23.247+0000 7f92e1600640 1 --2- 172.21.15.83:0/1969400251 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559d52835680 0x559d5283f7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:36:23.251 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:23.247+0000 7f92e0c00640 1 --2- 172.21.15.83:0/1969400251 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559d52835fc0 0x559d52846f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:36:23.251 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:23.247+0000 7f92e2000640 1 -- 172.21.15.83:0/1969400251 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559d52835fc0 msgr2=0x559d52846f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:36:23.251 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:23.247+0000 7f92e2000640 1 --2- 172.21.15.83:0/1969400251 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559d52835fc0 0x559d52846f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:23.251 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:23.247+0000 7f92e2000640 1 -- 172.21.15.83:0/1969400251 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559d52835680 msgr2=0x559d5283f7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:36:23.251 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:23.247+0000 7f92e2000640 1 --2- 172.21.15.83:0/1969400251 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559d52835680 0x559d5283f7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:23.251 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:23.247+0000 7f92e2000640 1 -- 172.21.15.83:0/1969400251 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559d52848440 con 0x559d5283b050 2024-08-19T01:36:23.251 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:23.247+0000 7f92e1600640 1 --2- 172.21.15.83:0/1969400251 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559d52835680 0x559d5283f7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-19T01:36:23.251 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:23.247+0000 7f92e2000640 1 --2- 172.21.15.83:0/1969400251 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559d5283b050 0x559d5283fd10 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f92d4009dc0 tx=0x7f92d4020000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2af1ed29364b3833 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:36:23.251 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:23.247+0000 7f92cdc00640 1 -- 172.21.15.83:0/1969400251 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92d4034070 con 0x559d5283b050 2024-08-19T01:36:23.251 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:23.247+0000 7f92cdc00640 1 -- 172.21.15.83:0/1969400251 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f92d4018d40 con 0x559d5283b050 2024-08-19T01:36:23.251 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:23.247+0000 7f92cdc00640 1 -- 172.21.15.83:0/1969400251 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92d40107a0 con 0x559d5283b050 2024-08-19T01:36:23.254 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:23.247+0000 7f92e6fd7740 1 -- 172.21.15.83:0/1969400251 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559d5283b050 msgr2=0x559d5283fd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:36:23.254 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:23.247+0000 7f92e6fd7740 1 --2- 172.21.15.83:0/1969400251 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559d5283b050 0x559d5283fd10 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f92d4009dc0 tx=0x7f92d4020000 comp rx=0 tx=0).stop 2024-08-19T01:36:23.255 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:23.247+0000 7f92e6fd7740 1 -- 172.21.15.83:0/1969400251 shutdown_connections 2024-08-19T01:36:23.255 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:23.247+0000 7f92e6fd7740 1 --2- 172.21.15.83:0/1969400251 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559d5283b050 0x559d5283fd10 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:23.255 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:23.247+0000 7f92e6fd7740 1 --2- 172.21.15.83:0/1969400251 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559d52835fc0 0x559d52846f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:23.255 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:23.247+0000 7f92e6fd7740 1 --2- 172.21.15.83:0/1969400251 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559d52835680 0x559d5283f7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:23.255 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:23.247+0000 7f92e6fd7740 1 -- 172.21.15.83:0/1969400251 >> 172.21.15.83:0/1969400251 conn(0x559d52830cf0 msgr2=0x559d52837240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:36:23.255 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:23.247+0000 7f92e6fd7740 1 -- 172.21.15.83:0/1969400251 shutdown_connections 2024-08-19T01:36:23.255 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:23.247+0000 7f92e6fd7740 1 -- 172.21.15.83:0/1969400251 wait complete. 2024-08-19T01:36:23.255 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:23.247+0000 7f92e6fd7740 1 -- start start 2024-08-19T01:36:23.255 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:23.247+0000 7f92e6fd7740 1 Processor -- start 2024-08-19T01:36:23.255 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:23.247+0000 7f92e6fd7740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559d52835680 0x559d52844080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:36:23.255 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:23.247+0000 7f92e6fd7740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559d52835fc0 0x559d528445c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:36:23.255 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:23.247+0000 7f92e6fd7740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559d5283b050 0x559d52845560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:36:23.256 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:23.247+0000 7f92e6fd7740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f92dc1a9570 con 0x559d5283b050 2024-08-19T01:36:23.256 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:23.247+0000 7f92e6fd7740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f92dc1aa9e0 con 0x559d52835fc0 2024-08-19T01:36:23.256 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:23.247+0000 7f92e6fd7740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f92dc1a9c40 con 0x559d52835680 2024-08-19T01:36:23.256 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:23.247+0000 7f92e6fd7740 10 client.0 _ll_register_callbacks cb 0x7f92d000d240 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-19T01:36:23.256 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:23.247+0000 7f92e2000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559d5283b050 0x559d52845560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:36:23.256 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:23.247+0000 7f92e2000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559d5283b050 0x559d52845560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:53470/0 (socket says 172.21.15.83:53470) 2024-08-19T01:36:23.257 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:23.247+0000 7f92e2000640 1 -- 172.21.15.83:0/3771905313 learned_addr learned my addr 172.21.15.83:0/3771905313 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-08-19T01:36:23.257 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:23.247+0000 7f92e1600640 1 --2- 172.21.15.83:0/3771905313 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559d52835680 0x559d52844080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:36:23.257 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:23.247+0000 7f92e2000640 1 -- 172.21.15.83:0/3771905313 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559d52835680 msgr2=0x559d52844080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:36:23.257 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:23.247+0000 7f92e2000640 1 --2- 172.21.15.83:0/3771905313 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559d52835680 0x559d52844080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:23.257 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:23.247+0000 7f92e2000640 1 -- 172.21.15.83:0/3771905313 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559d52835fc0 msgr2=0x559d528445c0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-08-19T01:36:23.257 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:23.247+0000 7f92e2000640 1 --2- 172.21.15.83:0/3771905313 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559d52835fc0 0x559d528445c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:23.257 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:23.247+0000 7f92e2000640 1 -- 172.21.15.83:0/3771905313 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559d52845b70 con 0x559d5283b050 2024-08-19T01:36:23.257 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:23.247+0000 7f92e2000640 1 --2- 172.21.15.83:0/3771905313 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559d5283b050 0x559d52845560 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f92d4025f50 tx=0x7f92d4009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=815a456b15a02228 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:36:23.257 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:23.247+0000 7f92c2a00640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-08-19T01:36:23.258 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:23.247+0000 7f92c2a00640 1 -- 172.21.15.83:0/3771905313 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92d4034040 con 0x559d5283b050 2024-08-19T01:36:23.258 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:23.247+0000 7f92c2a00640 1 -- 172.21.15.83:0/3771905313 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8269+0+0 (secure 0 0 0) 0x7f92d4025430 con 0x559d5283b050 2024-08-19T01:36:23.258 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:23.247+0000 7f92e6fd7740 1 -- 172.21.15.83:0/3771905313 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f92d4018d40 con 0x559d5283b050 2024-08-19T01:36:23.258 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:23.247+0000 7f92c2a00640 1 -- 172.21.15.83:0/3771905313 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f92d4025670 con 0x559d5283b050 2024-08-19T01:36:23.258 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:23.247+0000 7f92e6fd7740 10 client.15375 fetch_fsmap learned FSMap version 25 2024-08-19T01:36:23.258 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:23.247+0000 7f92e6fd7740 1 -- 172.21.15.83:0/3771905313 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x559d52846e80 con 0x559d5283b050 2024-08-19T01:36:23.258 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:23.247+0000 7f92c2a00640 1 -- 172.21.15.83:0/3771905313 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f92d4018d40 con 0x559d5283b050 2024-08-19T01:36:23.258 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:23.247+0000 7f92c2a00640 1 -- 172.21.15.83:0/3771905313 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f92d4040030 con 0x559d5283b050 2024-08-19T01:36:23.258 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:23.247+0000 7f92e6fd7740 10 client.15375 fetch_fsmap finished waiting for FSMap version 25 2024-08-19T01:36:23.259 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:23.247+0000 7f92e6fd7740 10 client.15375 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-08-19T01:36:23.259 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:23.247+0000 7f92e6fd7740 20 client.15375 populate_metadata read hostname 'smithi083' 2024-08-19T01:36:23.259 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:23.247+0000 7f92e6fd7740 1 --2- 172.21.15.83:0/3771905313 >> [v2:172.21.15.117:6838/1967595704,v1:172.21.15.117:6839/1967595704] conn(0x559d528b6640 0x559d528b8a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:36:23.259 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:23.247+0000 7f92e6fd7740 4 client.15375 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-08-19T01:36:23.259 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:23.247+0000 7f92e6fd7740 1 -- 172.21.15.83:0/3771905313 --> [v2:172.21.15.117:6838/1967595704,v1:172.21.15.117:6839/1967595704] -- command(tid 0: {"prefix": "flush journal"}) -- 0x559d527a1310 con 0x559d528b6640 2024-08-19T01:36:23.259 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:23.247+0000 7f92e1600640 1 --2- 172.21.15.83:0/3771905313 >> [v2:172.21.15.117:6838/1967595704,v1:172.21.15.117:6839/1967595704] conn(0x559d528b6640 0x559d528b8a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:36:23.259 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:23.247+0000 7f92e1600640 1 --2- 172.21.15.83:0/3771905313 >> [v2:172.21.15.117:6838/1967595704,v1:172.21.15.117:6839/1967595704] conn(0x559d528b6640 0x559d528b8a30 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b577c7e43fec7ce2 server_cookie=4cf47855d8402065 in_seq=0 out_seq=0 2024-08-19T01:36:23.259 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:23.247+0000 7f92c2a00640 10 client.15375 ms_handle_connect on v2:172.21.15.117:6838/1967595704 2024-08-19T01:36:23.675 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:23.673+0000 7fc51ac00640 1 -- 172.21.15.83:0/3419909982 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc514076d70 msgr2=0x7fc514112220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:36:23.676 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:23.673+0000 7fc51ac00640 1 --2- 172.21.15.83:0/3419909982 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc514076d70 0x7fc514112220 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fc500009860 tx=0x7fc50002a340 comp rx=0 tx=0).stop 2024-08-19T01:36:23.676 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:23.674+0000 7fc51ac00640 1 -- 172.21.15.83:0/3419909982 shutdown_connections 2024-08-19T01:36:23.676 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:23.674+0000 7fc51ac00640 1 --2- 172.21.15.83:0/3419909982 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc514112760 0x7fc514114b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:23.676 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:23.674+0000 7fc51ac00640 1 --2- 172.21.15.83:0/3419909982 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc514076d70 0x7fc514112220 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:23.676 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:23.674+0000 7fc51ac00640 1 --2- 172.21.15.83:0/3419909982 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc514076450 0x7fc514076830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:23.676 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:23.674+0000 7fc51ac00640 1 -- 172.21.15.83:0/3419909982 >> 172.21.15.83:0/3419909982 conn(0x7fc5140ff7c0 msgr2=0x7fc514101be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:36:23.678 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:23.674+0000 7fc51ac00640 1 -- 172.21.15.83:0/3419909982 shutdown_connections 2024-08-19T01:36:23.678 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:23.675+0000 7fc51ac00640 1 -- 172.21.15.83:0/3419909982 wait complete. 2024-08-19T01:36:23.684 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:23.682+0000 7fc51ac00640 1 Processor -- start 2024-08-19T01:36:23.691 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:23.683+0000 7fc51ac00640 1 -- start start 2024-08-19T01:36:23.691 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:23.684+0000 7fc51ac00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc514076450 0x7fc51419e590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:36:23.691 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:23.684+0000 7fc51ac00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc514076d70 0x7fc51419ead0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:36:23.691 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:23.684+0000 7fc51ac00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc514112760 0x7fc5141a2e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:36:23.691 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:23.684+0000 7fc51ac00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fc5141182d0 con 0x7fc514112760 2024-08-19T01:36:23.691 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:23.684+0000 7fc51ac00640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fc514118150 con 0x7fc514076d70 2024-08-19T01:36:23.691 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:23.684+0000 7fc51ac00640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fc514118450 con 0x7fc514076450 2024-08-19T01:36:23.691 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:23.685+0000 7fc513e00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc514076450 0x7fc51419e590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:36:23.691 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:23.685+0000 7fc513e00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc514076450 0x7fc51419e590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.83:47814/0 (socket says 172.21.15.83:47814) 2024-08-19T01:36:23.691 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:23.685+0000 7fc513e00640 1 -- 172.21.15.83:0/380904365 learned_addr learned my addr 172.21.15.83:0/380904365 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-08-19T01:36:23.692 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:23.685+0000 7fc513e00640 1 -- 172.21.15.83:0/380904365 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc514076d70 msgr2=0x7fc51419ead0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-08-19T01:36:23.692 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:23.685+0000 7fc513e00640 1 --2- 172.21.15.83:0/380904365 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc514076d70 0x7fc51419ead0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:23.692 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:23.685+0000 7fc513e00640 1 -- 172.21.15.83:0/380904365 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc514112760 msgr2=0x7fc5141a2e60 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-08-19T01:36:23.692 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:23.685+0000 7fc513e00640 1 --2- 172.21.15.83:0/380904365 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc514112760 0x7fc5141a2e60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:23.692 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:23.685+0000 7fc513e00640 1 -- 172.21.15.83:0/380904365 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc5141a3560 con 0x7fc514076450 2024-08-19T01:36:23.692 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:23.686+0000 7fc513e00640 1 --2- 172.21.15.83:0/380904365 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc514076450 0x7fc51419e590 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fc50400ea60 tx=0x7fc50400ef20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:36:23.692 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:23.686+0000 7fc510c00640 1 -- 172.21.15.83:0/380904365 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc50400cde0 con 0x7fc514076450 2024-08-19T01:36:23.692 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:23.686+0000 7fc510c00640 1 -- 172.21.15.83:0/380904365 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc5040035f0 con 0x7fc514076450 2024-08-19T01:36:23.692 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:23.686+0000 7fc510c00640 1 -- 172.21.15.83:0/380904365 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc50400cde0 con 0x7fc514076450 2024-08-19T01:36:23.692 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:23.686+0000 7fc51ac00640 1 -- 172.21.15.83:0/380904365 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc5141a3850 con 0x7fc514076450 2024-08-19T01:36:23.692 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:23.687+0000 7fc51ac00640 1 -- 172.21.15.83:0/380904365 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc5141039d0 con 0x7fc514076450 2024-08-19T01:36:23.693 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:23.691+0000 7fc510c00640 1 -- 172.21.15.83:0/380904365 <== mon.1 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7fc5040037a0 con 0x7fc514076450 2024-08-19T01:36:23.703 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:23.694+0000 7fc510c00640 1 --2- 172.21.15.83:0/380904365 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7fc4f80a2e00 0x7fc4f80a52c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:36:23.703 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:23.694+0000 7fc510c00640 1 -- 172.21.15.83:0/380904365 <== mon.1 v2:172.21.15.167:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8269+0+0 (secure 0 0 0) 0x7fc5040998c0 con 0x7fc514076450 2024-08-19T01:36:23.703 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:23.695+0000 7fc51d5fd740 1 Processor -- start 2024-08-19T01:36:23.703 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:23.695+0000 7fc513400640 1 --2- 172.21.15.83:0/380904365 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7fc4f80a2e00 0x7fc4f80a52c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:36:23.703 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:23.695+0000 7fc51d5fd740 1 -- start start 2024-08-19T01:36:23.703 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:23.696+0000 7fc51d5fd740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55909aa9f480 0x55909aa9f860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:36:23.703 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:23.696+0000 7fc51d5fd740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55909aa9fda0 0x55909aaaa120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:36:23.703 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:23.696+0000 7fc51d5fd740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55909aaaa660 0x55909aaaca50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:36:23.703 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:23.696+0000 7fc513e00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55909aa9f480 0x55909aa9f860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:36:23.703 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:23.696+0000 7fc513e00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55909aa9f480 0x55909aa9f860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.83:47820/0 (socket says 172.21.15.83:47820) 2024-08-19T01:36:23.703 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:23.697+0000 7fc51d5fd740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55909a9ba7e0 con 0x55909aa9fda0 2024-08-19T01:36:23.703 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:23.697+0000 7fc51d5fd740 1 -- 172.21.15.83:0/3381447960 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55909aa05ac0 con 0x55909aaaa660 2024-08-19T01:36:23.704 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:23.697+0000 7fc51d5fd740 1 -- 172.21.15.83:0/3381447960 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55909a99d8c0 con 0x55909aa9f480 2024-08-19T01:36:23.704 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:23.697+0000 7fc518c00640 1 --2- 172.21.15.83:0/3381447960 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55909aaaa660 0x55909aaaca50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:36:23.704 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:23.697+0000 7fc513e00640 1 -- 172.21.15.83:0/3381447960 learned_addr learned my addr 172.21.15.83:0/3381447960 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-08-19T01:36:23.704 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:23.697+0000 7fc513400640 1 --2- 172.21.15.83:0/380904365 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7fc4f80a2e00 0x7fc4f80a52c0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fc5000095f0 tx=0x7fc500005fd0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:36:23.704 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:23.697+0000 7fc518c00640 1 -- 172.21.15.83:0/3381447960 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55909aa9f480 msgr2=0x55909aa9f860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:36:23.704 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:23.697+0000 7fc518c00640 1 --2- 172.21.15.83:0/3381447960 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55909aa9f480 0x55909aa9f860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:23.704 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:23.697+0000 7fc518c00640 1 -- 172.21.15.83:0/3381447960 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55909aa9fda0 msgr2=0x55909aaaa120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:36:23.704 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:23.697+0000 7fc518c00640 1 --2- 172.21.15.83:0/3381447960 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55909aa9fda0 0x55909aaaa120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:23.704 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:23.697+0000 7fc518c00640 1 -- 172.21.15.83:0/3381447960 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55909a9a4760 con 0x55909aaaa660 2024-08-19T01:36:23.704 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:23.698+0000 7fc513e00640 1 --2- 172.21.15.83:0/3381447960 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55909aa9f480 0x55909aa9f860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello state changed while learned_addr, mark_down or replacing must be happened just now 2024-08-19T01:36:23.704 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:23.698+0000 7fc518c00640 1 --2- 172.21.15.83:0/3381447960 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55909aaaa660 0x55909aaaca50 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fc51407a740 tx=0x7fc50c00e760 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4eb22e2ced23b754 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:36:23.704 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:23.700+0000 7fc509200640 1 -- 172.21.15.83:0/3381447960 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc50c003b70 con 0x55909aaaa660 2024-08-19T01:36:23.704 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:23.700+0000 7fc509200640 1 -- 172.21.15.83:0/3381447960 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc50c003d10 con 0x55909aaaa660 2024-08-19T01:36:23.705 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:23.700+0000 7fc509200640 1 -- 172.21.15.83:0/3381447960 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc50c00f940 con 0x55909aaaa660 2024-08-19T01:36:23.712 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:23.701+0000 7fc51d5fd740 1 -- 172.21.15.83:0/3381447960 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55909aaaa660 msgr2=0x55909aaaca50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:36:23.712 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:23.701+0000 7fc51d5fd740 1 --2- 172.21.15.83:0/3381447960 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55909aaaa660 0x55909aaaca50 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fc51407a740 tx=0x7fc50c00e760 comp rx=0 tx=0).stop 2024-08-19T01:36:23.713 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:23.711+0000 7fc51d5fd740 1 -- 172.21.15.83:0/3381447960 shutdown_connections 2024-08-19T01:36:23.713 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:23.711+0000 7fc51d5fd740 1 --2- 172.21.15.83:0/3381447960 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55909aaaa660 0x55909aaaca50 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:23.713 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:23.711+0000 7fc51d5fd740 1 --2- 172.21.15.83:0/3381447960 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55909aa9fda0 0x55909aaaa120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:23.713 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:23.711+0000 7fc51d5fd740 1 --2- 172.21.15.83:0/3381447960 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55909aa9f480 0x55909aa9f860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:23.713 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:23.711+0000 7fc51d5fd740 1 -- 172.21.15.83:0/3381447960 >> 172.21.15.83:0/3381447960 conn(0x55909aa9ba10 msgr2=0x55909aa9bdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:36:23.713 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:23.711+0000 7fc51d5fd740 1 -- 172.21.15.83:0/3381447960 shutdown_connections 2024-08-19T01:36:23.717 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:23.711+0000 7fc51d5fd740 1 -- 172.21.15.83:0/3381447960 wait complete. 2024-08-19T01:36:23.717 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:23.712+0000 7fc51d5fd740 1 -- start start 2024-08-19T01:36:23.717 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:23.712+0000 7fc51d5fd740 1 Processor -- start 2024-08-19T01:36:23.717 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:23.714+0000 7fc51d5fd740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55909aa9f480 0x55909aaa7660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:36:23.719 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:23.715+0000 7fc51d5fd740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55909aa9fda0 0x55909aaa7ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:36:23.719 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:23.717+0000 7fc51d5fd740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55909aaaa660 0x55909aaa80e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:36:23.719 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:23.717+0000 7fc51d5fd740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55909a9c0ae0 con 0x55909aa9f480 2024-08-19T01:36:23.719 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:23.717+0000 7fc51d5fd740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55909aa01a80 con 0x55909aa9fda0 2024-08-19T01:36:23.719 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:23.717+0000 7fc51d5fd740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55909aaad0e0 con 0x55909aaaa660 2024-08-19T01:36:23.719 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:23.717+0000 7fc51d5fd740 10 client.0 _ll_register_callbacks cb 0x55909a9c2190 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-19T01:36:23.720 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:23.718+0000 7fc518c00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55909aaaa660 0x55909aaa80e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:36:23.721 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:23.718+0000 7fc518c00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55909aaaa660 0x55909aaa80e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.83:47836/0 (socket says 172.21.15.83:47836) 2024-08-19T01:36:23.722 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:23.718+0000 7fc518c00640 1 -- 172.21.15.83:0/26415289 learned_addr learned my addr 172.21.15.83:0/26415289 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-08-19T01:36:23.722 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:23.718+0000 7fc513400640 1 --2- 172.21.15.83:0/26415289 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55909aa9fda0 0x55909aaa7ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:36:23.722 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:23.720+0000 7fc518c00640 1 -- 172.21.15.83:0/26415289 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55909aa9fda0 msgr2=0x55909aaa7ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:36:23.722 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:23.720+0000 7fc518c00640 1 --2- 172.21.15.83:0/26415289 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55909aa9fda0 0x55909aaa7ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:23.722 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:23.720+0000 7fc518c00640 1 -- 172.21.15.83:0/26415289 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55909aa9f480 msgr2=0x55909aaa7660 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-08-19T01:36:23.722 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:23.720+0000 7fc518c00640 1 --2- 172.21.15.83:0/26415289 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55909aa9f480 0x55909aaa7660 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:23.722 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:23.720+0000 7fc518c00640 1 -- 172.21.15.83:0/26415289 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55909a7776a0 con 0x55909aaaa660 2024-08-19T01:36:23.722 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:23.720+0000 7fc513400640 1 --2- 172.21.15.83:0/26415289 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55909aa9fda0 0x55909aaa7ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-19T01:36:23.723 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:23.720+0000 7fc518c00640 1 --2- 172.21.15.83:0/26415289 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55909aaaa660 0x55909aaa80e0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fc50c009ff0 tx=0x7fc50c009740 comp rx=0 tx=0).ready entity=mon.1 client_cookie=40dd4bac7db26e7f server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:36:23.723 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:23.720+0000 7fc4fd600640 10 client.0 ms_handle_connect on v2:172.21.15.167:3300/0 2024-08-19T01:36:23.723 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:23.721+0000 7fc4fd600640 1 -- 172.21.15.83:0/26415289 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc50c026060 con 0x55909aaaa660 2024-08-19T01:36:23.723 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:23.721+0000 7fc4fd600640 1 -- 172.21.15.83:0/26415289 <== mon.1 v2:172.21.15.167:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8269+0+0 (secure 0 0 0) 0x7fc50c004c50 con 0x55909aaaa660 2024-08-19T01:36:23.723 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:23.721+0000 7fc51d5fd740 1 -- 172.21.15.83:0/26415289 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc50c003d10 con 0x55909aaaa660 2024-08-19T01:36:23.723 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:23.721+0000 7fc4fd600640 1 -- 172.21.15.83:0/26415289 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fc50c004e50 con 0x55909aaaa660 2024-08-19T01:36:23.723 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:23.721+0000 7fc51d5fd740 10 client.25426 fetch_fsmap learned FSMap version 25 2024-08-19T01:36:23.723 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:23.721+0000 7fc51d5fd740 1 -- 172.21.15.83:0/26415289 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55909aab8e10 con 0x55909aaaa660 2024-08-19T01:36:23.723 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:23.722+0000 7fc4fd600640 1 -- 172.21.15.83:0/26415289 <== mon.1 v2:172.21.15.167:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc50c003d10 con 0x55909aaaa660 2024-08-19T01:36:23.731 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:23.724+0000 7fc4fd600640 1 -- 172.21.15.83:0/26415289 <== mon.1 v2:172.21.15.167:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fc50c00b030 con 0x55909aaaa660 2024-08-19T01:36:23.731 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:23.724+0000 7fc51d5fd740 10 client.25426 fetch_fsmap finished waiting for FSMap version 25 2024-08-19T01:36:23.731 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:23.724+0000 7fc51d5fd740 10 client.25426 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-19T01:36:23.731 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:23.724+0000 7fc51d5fd740 20 client.25426 populate_metadata read hostname 'smithi083' 2024-08-19T01:36:23.731 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:23.724+0000 7fc51d5fd740 1 --2- 172.21.15.83:0/26415289 >> [v2:172.21.15.117:6838/1967595704,v1:172.21.15.117:6839/1967595704] conn(0x55909aaba2d0 0x55909aabc6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:36:23.731 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:23.724+0000 7fc51d5fd740 4 client.25426 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-19T01:36:23.731 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:23.724+0000 7fc51d5fd740 1 -- 172.21.15.83:0/26415289 --> [v2:172.21.15.117:6838/1967595704,v1:172.21.15.117:6839/1967595704] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55909aa13030 con 0x55909aaba2d0 2024-08-19T01:36:23.731 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:23.724+0000 7fc513e00640 1 --2- 172.21.15.83:0/26415289 >> [v2:172.21.15.117:6838/1967595704,v1:172.21.15.117:6839/1967595704] conn(0x55909aaba2d0 0x55909aabc6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:36:23.732 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:23.726+0000 7fc513e00640 1 --2- 172.21.15.83:0/26415289 >> [v2:172.21.15.117:6838/1967595704,v1:172.21.15.117:6839/1967595704] conn(0x55909aaba2d0 0x55909aabc6c0 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cf864e51b6e9bd6a server_cookie=5cf543bee091b829 in_seq=0 out_seq=0 2024-08-19T01:36:23.732 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:23.726+0000 7fc4fd600640 10 client.25426 ms_handle_connect on v2:172.21.15.117:6838/1967595704 2024-08-19T01:36:23.732 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:23.729+0000 7fc4fd600640 1 -- 172.21.15.83:0/26415289 <== mds.0 v2:172.21.15.117:6838/1967595704 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55909aa13030 con 0x55909aaba2d0 2024-08-19T01:36:23.732 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:23.729+0000 7fc4fd600640 10 client.25426 handle_command_reply: tid=0 2024-08-19T01:36:23.732 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:23.729+0000 7fc51d5fd740 1 client.25426 shutdown 2024-08-19T01:36:23.732 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:23.730+0000 7fc51d5fd740 1 -- 172.21.15.83:0/26415289 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55909aaaa660 msgr2=0x55909aaa80e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:36:23.732 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:23.730+0000 7fc51d5fd740 1 --2- 172.21.15.83:0/26415289 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55909aaaa660 0x55909aaa80e0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fc50c009ff0 tx=0x7fc50c009740 comp rx=0 tx=0).stop 2024-08-19T01:36:23.733 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:23.731+0000 7fc51d5fd740 1 -- 172.21.15.83:0/26415289 shutdown_connections 2024-08-19T01:36:23.733 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:23.731+0000 7fc51d5fd740 1 --2- 172.21.15.83:0/26415289 >> [v2:172.21.15.117:6838/1967595704,v1:172.21.15.117:6839/1967595704] conn(0x55909aaba2d0 0x55909aabc6c0 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:23.733 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:23.731+0000 7fc51d5fd740 1 --2- 172.21.15.83:0/26415289 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55909aaaa660 0x55909aaa80e0 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:23.733 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:23.731+0000 7fc51d5fd740 1 --2- 172.21.15.83:0/26415289 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55909aa9fda0 0x55909aaa7ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:23.733 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:23.731+0000 7fc51d5fd740 1 --2- 172.21.15.83:0/26415289 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55909aa9f480 0x55909aaa7660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:23.733 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:23.731+0000 7fc51d5fd740 1 -- 172.21.15.83:0/26415289 >> 172.21.15.83:0/26415289 conn(0x55909aa9ba10 msgr2=0x55909aaa1740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:36:23.740 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:23.735+0000 7fc4fd600640 0 client.25426 ms_handle_reset on v2:172.21.15.117:6838/1967595704 2024-08-19T01:36:23.740 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:23.735+0000 7fc51d5fd740 1 -- 172.21.15.83:0/26415289 shutdown_connections 2024-08-19T01:36:23.740 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:23.735+0000 7fc51d5fd740 1 -- 172.21.15.83:0/26415289 wait complete. 2024-08-19T01:36:23.740 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:23.735+0000 7fc51d5fd740 20 client.25426 trim_cache size 0 max 16384 2024-08-19T01:36:23.783 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:23.782+0000 7fc51d5fd740 1 Processor -- start 2024-08-19T01:36:23.784 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:23.782+0000 7fc51d5fd740 1 -- start start 2024-08-19T01:36:23.785 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:23.782+0000 7fc51d5fd740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55909aa9f480 0x55909aaa95d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:36:23.786 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:23.782+0000 7fc51d5fd740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55909aa9fda0 0x55909aabf300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:36:23.786 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:23.783+0000 7fc51d5fd740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55909aaaa660 0x55909aabf840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:36:23.786 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:23.783+0000 7fc51d5fd740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fc5141bea90 con 0x55909aa9f480 2024-08-19T01:36:23.786 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:23.783+0000 7fc51d5fd740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fc5141bc360 con 0x55909aaaa660 2024-08-19T01:36:23.786 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:23.783+0000 7fc51d5fd740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fc51419f8f0 con 0x55909aa9fda0 2024-08-19T01:36:23.788 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:23.783+0000 7fc513400640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55909aa9fda0 0x55909aabf300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:36:23.788 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:23.783+0000 7fc513400640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55909aa9fda0 0x55909aabf300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.83:47838/0 (socket says 172.21.15.83:47838) 2024-08-19T01:36:23.788 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:23.783+0000 7fc513400640 1 -- 172.21.15.83:0/468799319 learned_addr learned my addr 172.21.15.83:0/468799319 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-08-19T01:36:23.788 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:23.783+0000 7fc518c00640 1 --2- 172.21.15.83:0/468799319 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55909aaaa660 0x55909aabf840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:36:23.788 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:23.783+0000 7fc513400640 1 -- 172.21.15.83:0/468799319 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55909aaaa660 msgr2=0x55909aabf840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:36:23.788 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:23.783+0000 7fc513400640 1 --2- 172.21.15.83:0/468799319 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55909aaaa660 0x55909aabf840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:23.788 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:23.783+0000 7fc513400640 1 -- 172.21.15.83:0/468799319 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55909aa9f480 msgr2=0x55909aaa95d0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-08-19T01:36:23.789 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:23.786+0000 7fc513e00640 1 --2- 172.21.15.83:0/468799319 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55909aa9f480 0x55909aaa95d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:36:23.790 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:23.787+0000 7fc513400640 1 --2- 172.21.15.83:0/468799319 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55909aa9f480 0x55909aaa95d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:23.791 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:36:23 smithi083 ceph-mon[17120]: pgmap v385: 97 pgs: 97 active+clean; 32 MiB data, 487 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.6 MiB/s wr, 151 op/s 2024-08-19T01:36:23.791 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:23.789+0000 7fc513400640 1 -- 172.21.15.83:0/468799319 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55909aaa5c10 con 0x55909aa9fda0 2024-08-19T01:36:23.792 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:23.790+0000 7fc513e00640 1 --2- 172.21.15.83:0/468799319 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55909aa9f480 0x55909aaa95d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-19T01:36:23.793 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:23.791+0000 7fc513400640 1 --2- 172.21.15.83:0/468799319 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55909aa9fda0 0x55909aabf300 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fc514103080 tx=0x7fc500000bc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8cac420b4f9955aa server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:36:23.794 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:23.793+0000 7fc4ffe00640 1 -- 172.21.15.83:0/468799319 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc500037040 con 0x55909aa9fda0 2024-08-19T01:36:23.799 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:23.793+0000 7fc4ffe00640 1 -- 172.21.15.83:0/468799319 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc50004f070 con 0x55909aa9fda0 2024-08-19T01:36:23.799 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:23.793+0000 7fc4ffe00640 1 -- 172.21.15.83:0/468799319 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc50003f6c0 con 0x55909aa9fda0 2024-08-19T01:36:23.799 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:23.794+0000 7fc51d5fd740 1 -- 172.21.15.83:0/468799319 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55909aa9fda0 msgr2=0x55909aabf300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:36:23.799 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:23.794+0000 7fc51d5fd740 1 --2- 172.21.15.83:0/468799319 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55909aa9fda0 0x55909aabf300 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fc514103080 tx=0x7fc500000bc0 comp rx=0 tx=0).stop 2024-08-19T01:36:23.799 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:23.797+0000 7fc51d5fd740 1 -- 172.21.15.83:0/468799319 shutdown_connections 2024-08-19T01:36:23.800 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:23.797+0000 7fc51d5fd740 1 --2- 172.21.15.83:0/468799319 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55909aaaa660 0x55909aabf840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:23.800 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:23.797+0000 7fc51d5fd740 1 --2- 172.21.15.83:0/468799319 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55909aa9fda0 0x55909aabf300 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:23.800 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:23.797+0000 7fc51d5fd740 1 --2- 172.21.15.83:0/468799319 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55909aa9f480 0x55909aaa95d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:23.800 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:23.797+0000 7fc51d5fd740 1 -- 172.21.15.83:0/468799319 >> 172.21.15.83:0/468799319 conn(0x55909aa9ba10 msgr2=0x55909aaa8ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:36:23.800 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:23.797+0000 7fc51d5fd740 1 -- 172.21.15.83:0/468799319 shutdown_connections 2024-08-19T01:36:23.800 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:23.797+0000 7fc51d5fd740 1 -- 172.21.15.83:0/468799319 wait complete. 2024-08-19T01:36:23.800 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:23.798+0000 7fc51d5fd740 1 -- start start 2024-08-19T01:36:23.803 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:23.798+0000 7fc51d5fd740 1 Processor -- start 2024-08-19T01:36:23.804 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:23.802+0000 7fc51d5fd740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55909aa9f480 0x55909aabf700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:36:23.804 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:23.802+0000 7fc51d5fd740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55909aa9fda0 0x55909aabfc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:36:23.804 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:23.802+0000 7fc51d5fd740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55909aaaa660 0x55909aabcca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:36:23.804 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:23.802+0000 7fc51d5fd740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fc51410bac0 con 0x55909aaaa660 2024-08-19T01:36:23.805 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:23.802+0000 7fc51d5fd740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fc51410cf10 con 0x55909aa9f480 2024-08-19T01:36:23.805 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:23.802+0000 7fc51d5fd740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fc51410c170 con 0x55909aa9fda0 2024-08-19T01:36:23.806 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:23.802+0000 7fc51d5fd740 10 client.0 _ll_register_callbacks cb 0x7fc50c009a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-19T01:36:23.806 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:23.803+0000 7fc513e00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55909aa9f480 0x55909aabf700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:36:23.806 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:23.803+0000 7fc513400640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55909aa9fda0 0x55909aabfc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:36:23.806 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:23.803+0000 7fc513400640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55909aa9fda0 0x55909aabfc40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.83:47852/0 (socket says 172.21.15.83:47852) 2024-08-19T01:36:23.806 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:23.803+0000 7fc513400640 1 -- 172.21.15.83:0/1931302173 learned_addr learned my addr 172.21.15.83:0/1931302173 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-08-19T01:36:23.806 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:23.804+0000 7fc518c00640 1 --2- 172.21.15.83:0/1931302173 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55909aaaa660 0x55909aabcca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:36:23.809 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:23.805+0000 7fc513e00640 1 -- 172.21.15.83:0/1931302173 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55909aa9fda0 msgr2=0x55909aabfc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:36:23.809 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:23.805+0000 7fc513e00640 1 --2- 172.21.15.83:0/1931302173 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55909aa9fda0 0x55909aabfc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:23.809 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:23.805+0000 7fc513e00640 1 -- 172.21.15.83:0/1931302173 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55909aaaa660 msgr2=0x55909aabcca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:36:23.809 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:23.805+0000 7fc513e00640 1 --2- 172.21.15.83:0/1931302173 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55909aaaa660 0x55909aabcca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:23.809 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:23.805+0000 7fc513e00640 1 -- 172.21.15.83:0/1931302173 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55909aabd4d0 con 0x55909aa9f480 2024-08-19T01:36:23.809 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:23.805+0000 7fc518c00640 1 --2- 172.21.15.83:0/1931302173 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55909aaaa660 0x55909aabcca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-19T01:36:23.809 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:23.807+0000 7fc513e00640 1 --2- 172.21.15.83:0/1931302173 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55909aa9f480 0x55909aabf700 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fc50409b7b0 tx=0x7fc504096320 comp rx=0 tx=0).ready entity=mon.2 client_cookie=391a5ab67e0a5a20 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:36:23.810 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:23.807+0000 7fc4fd600640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-08-19T01:36:23.810 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:23.807+0000 7fc4fd600640 1 -- 172.21.15.83:0/1931302173 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc504098250 con 0x55909aa9f480 2024-08-19T01:36:23.810 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:23.807+0000 7fc51d5fd740 1 -- 172.21.15.83:0/1931302173 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc50004f070 con 0x55909aa9f480 2024-08-19T01:36:23.810 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:23.808+0000 7fc4fd600640 1 -- 172.21.15.83:0/1931302173 <== mon.2 v2:172.21.15.117:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8269+0+0 (secure 0 0 0) 0x7fc504003ad0 con 0x55909aa9f480 2024-08-19T01:36:23.811 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:23.809+0000 7fc4fd600640 1 -- 172.21.15.83:0/1931302173 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fc504098250 con 0x55909aa9f480 2024-08-19T01:36:23.811 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:23.809+0000 7fc51d5fd740 10 client.25460 fetch_fsmap learned FSMap version 25 2024-08-19T01:36:23.811 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:23.809+0000 7fc51d5fd740 1 -- 172.21.15.83:0/1931302173 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55909ab248c0 con 0x55909aa9f480 2024-08-19T01:36:23.812 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:23.810+0000 7fc4fd600640 1 -- 172.21.15.83:0/1931302173 <== mon.2 v2:172.21.15.117:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc50004f070 con 0x55909aa9f480 2024-08-19T01:36:23.812 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:23.810+0000 7fc4fd600640 1 -- 172.21.15.83:0/1931302173 <== mon.2 v2:172.21.15.117:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fc504097570 con 0x55909aa9f480 2024-08-19T01:36:23.812 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:23.810+0000 7fc51d5fd740 10 client.25460 fetch_fsmap finished waiting for FSMap version 25 2024-08-19T01:36:23.812 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:23.810+0000 7fc51d5fd740 10 client.25460 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-19T01:36:23.812 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:23.810+0000 7fc51d5fd740 20 client.25460 populate_metadata read hostname 'smithi083' 2024-08-19T01:36:23.812 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:23.810+0000 7fc51d5fd740 1 --2- 172.21.15.83:0/1931302173 >> [v2:172.21.15.117:6838/1967595704,v1:172.21.15.117:6839/1967595704] conn(0x55909ab24fc0 0x55909ab273e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:36:23.812 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:23.810+0000 7fc51d5fd740 4 client.25460 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-08-19T01:36:23.813 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:23.810+0000 7fc51d5fd740 1 -- 172.21.15.83:0/1931302173 --> [v2:172.21.15.117:6838/1967595704,v1:172.21.15.117:6839/1967595704] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55909aa0d540 con 0x55909ab24fc0 2024-08-19T01:36:23.813 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:23.811+0000 7fc513400640 1 --2- 172.21.15.83:0/1931302173 >> [v2:172.21.15.117:6838/1967595704,v1:172.21.15.117:6839/1967595704] conn(0x55909ab24fc0 0x55909ab273e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:36:23.813 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:23.812+0000 7fc513400640 1 --2- 172.21.15.83:0/1931302173 >> [v2:172.21.15.117:6838/1967595704,v1:172.21.15.117:6839/1967595704] conn(0x55909ab24fc0 0x55909ab273e0 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e22685671569fee2 server_cookie=6aa07194ec81c9b0 in_seq=0 out_seq=0 2024-08-19T01:36:23.814 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:23.812+0000 7fc4fd600640 10 client.25460 ms_handle_connect on v2:172.21.15.117:6838/1967595704 2024-08-19T01:36:23.836 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:23.834+0000 7fc4fd600640 1 -- 172.21.15.83:0/1931302173 <== mds.0 v2:172.21.15.117:6838/1967595704 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55909aa0d540 con 0x55909ab24fc0 2024-08-19T01:36:23.836 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:23.834+0000 7fc4fd600640 10 client.25460 handle_command_reply: tid=0 2024-08-19T01:36:23.837 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:23.835+0000 7fc51d5fd740 1 client.25460 shutdown 2024-08-19T01:36:23.837 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:23.836+0000 7fc51d5fd740 1 -- 172.21.15.83:0/1931302173 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55909aa9f480 msgr2=0x55909aabf700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:36:23.837 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:23.836+0000 7fc51d5fd740 1 --2- 172.21.15.83:0/1931302173 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55909aa9f480 0x55909aabf700 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fc50409b7b0 tx=0x7fc504096320 comp rx=0 tx=0).stop 2024-08-19T01:36:23.837 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:23.836+0000 7fc51d5fd740 1 -- 172.21.15.83:0/1931302173 shutdown_connections 2024-08-19T01:36:23.837 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:23.836+0000 7fc51d5fd740 1 --2- 172.21.15.83:0/1931302173 >> [v2:172.21.15.117:6838/1967595704,v1:172.21.15.117:6839/1967595704] conn(0x55909ab24fc0 0x55909ab273e0 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:23.837 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:23.836+0000 7fc51d5fd740 1 --2- 172.21.15.83:0/1931302173 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55909aaaa660 0x55909aabcca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:23.838 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:23.836+0000 7fc51d5fd740 1 --2- 172.21.15.83:0/1931302173 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55909aa9fda0 0x55909aabfc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:23.838 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:23.836+0000 7fc51d5fd740 1 --2- 172.21.15.83:0/1931302173 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55909aa9f480 0x55909aabf700 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:23.838 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:23.836+0000 7fc51d5fd740 1 -- 172.21.15.83:0/1931302173 >> 172.21.15.83:0/1931302173 conn(0x55909aa9ba10 msgr2=0x55909aaa7e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:36:23.839 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:23.836+0000 7fc4fd600640 0 client.25460 ms_handle_reset on v2:172.21.15.117:6838/1967595704 2024-08-19T01:36:23.839 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:23.836+0000 7fc51d5fd740 1 -- 172.21.15.83:0/1931302173 shutdown_connections 2024-08-19T01:36:23.839 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:23.837+0000 7fc51d5fd740 1 -- 172.21.15.83:0/1931302173 wait complete. 2024-08-19T01:36:23.839 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:23.837+0000 7fc51d5fd740 20 client.25460 trim_cache size 0 max 16384 2024-08-19T01:36:23.839 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-08-19T01:36:23.840 INFO:teuthology.orchestra.run.smithi083.stdout: "return_code": 0, 2024-08-19T01:36:23.840 INFO:teuthology.orchestra.run.smithi083.stdout: "scrub_tag": "a9a924bb-6b35-4256-8105-61ffbeb040cb", 2024-08-19T01:36:23.840 INFO:teuthology.orchestra.run.smithi083.stdout: "mode": "asynchronous" 2024-08-19T01:36:23.840 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-08-19T01:36:23.841 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:23.838+0000 7fc51ac00640 1 -- 172.21.15.83:0/380904365 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7fc4f80a2e00 msgr2=0x7fc4f80a52c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:36:23.841 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:23.838+0000 7fc51ac00640 1 --2- 172.21.15.83:0/380904365 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7fc4f80a2e00 0x7fc4f80a52c0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fc5000095f0 tx=0x7fc500005fd0 comp rx=0 tx=0).stop 2024-08-19T01:36:23.841 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:23.838+0000 7fc51ac00640 1 -- 172.21.15.83:0/380904365 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc514076450 msgr2=0x7fc51419e590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:36:23.841 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:23.838+0000 7fc51ac00640 1 --2- 172.21.15.83:0/380904365 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc514076450 0x7fc51419e590 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fc50400ea60 tx=0x7fc50400ef20 comp rx=0 tx=0).stop 2024-08-19T01:36:23.841 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:23.838+0000 7fc51ac00640 1 -- 172.21.15.83:0/380904365 shutdown_connections 2024-08-19T01:36:23.841 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:23.838+0000 7fc51ac00640 1 --2- 172.21.15.83:0/380904365 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7fc4f80a2e00 0x7fc4f80a52c0 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:23.841 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:23.838+0000 7fc51ac00640 1 --2- 172.21.15.83:0/380904365 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc514112760 0x7fc5141a2e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:23.841 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:23.838+0000 7fc51ac00640 1 --2- 172.21.15.83:0/380904365 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc514076d70 0x7fc51419ead0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:23.841 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:23.838+0000 7fc51ac00640 1 --2- 172.21.15.83:0/380904365 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc514076450 0x7fc51419e590 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:23.841 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:23.838+0000 7fc51ac00640 1 -- 172.21.15.83:0/380904365 >> 172.21.15.83:0/380904365 conn(0x7fc5140ff7c0 msgr2=0x7fc514101bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:36:23.842 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:23.839+0000 7fc51ac00640 1 -- 172.21.15.83:0/380904365 shutdown_connections 2024-08-19T01:36:23.842 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:23.839+0000 7fc51ac00640 1 -- 172.21.15.83:0/380904365 wait complete. 2024-08-19T01:36:23.952 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:36:23 smithi117 ceph-mon[22016]: pgmap v385: 97 pgs: 97 active+clean; 32 MiB data, 487 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.6 MiB/s wr, 151 op/s 2024-08-19T01:36:23.966 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:36:23 smithi167 ceph-mon[21875]: pgmap v385: 97 pgs: 97 active+clean; 32 MiB data, 487 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.6 MiB/s wr, 151 op/s 2024-08-19T01:36:23.999 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:23.997+0000 7f92c2a00640 1 -- 172.21.15.83:0/3771905313 <== mds.0 v2:172.21.15.117:6838/1967595704 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x559d527a1310 con 0x559d528b6640 2024-08-19T01:36:23.999 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:23.997+0000 7f92c2a00640 10 client.15375 handle_command_reply: tid=0 2024-08-19T01:36:23.999 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:23.997+0000 7f92e6fd7740 1 client.15375 shutdown 2024-08-19T01:36:23.999 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:23.997+0000 7f92e6fd7740 1 -- 172.21.15.83:0/3771905313 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559d5283b050 msgr2=0x559d52845560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:36:23.999 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:23.997+0000 7f92e6fd7740 1 --2- 172.21.15.83:0/3771905313 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559d5283b050 0x559d52845560 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f92d4025f50 tx=0x7f92d4009bd0 comp rx=0 tx=0).stop 2024-08-19T01:36:23.999 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:23.997+0000 7f92e6fd7740 1 -- 172.21.15.83:0/3771905313 shutdown_connections 2024-08-19T01:36:23.999 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:23.997+0000 7f92e6fd7740 1 --2- 172.21.15.83:0/3771905313 >> [v2:172.21.15.117:6838/1967595704,v1:172.21.15.117:6839/1967595704] conn(0x559d528b6640 0x559d528b8a30 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:23.999 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:23.997+0000 7f92e6fd7740 1 --2- 172.21.15.83:0/3771905313 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559d5283b050 0x559d52845560 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:24.000 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:23.997+0000 7f92e6fd7740 1 --2- 172.21.15.83:0/3771905313 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559d52835fc0 0x559d528445c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:24.000 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:23.997+0000 7f92e6fd7740 1 --2- 172.21.15.83:0/3771905313 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559d52835680 0x559d52844080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:24.000 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:23.997+0000 7f92e6fd7740 1 -- 172.21.15.83:0/3771905313 >> 172.21.15.83:0/3771905313 conn(0x559d52830cf0 msgr2=0x559d52837240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:36:24.000 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:23.997+0000 7f92c2a00640 0 client.15375 ms_handle_reset on v2:172.21.15.117:6838/1967595704 2024-08-19T01:36:24.000 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:23.998+0000 7f92e6fd7740 1 -- 172.21.15.83:0/3771905313 shutdown_connections 2024-08-19T01:36:24.000 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:23.998+0000 7f92e6fd7740 1 -- 172.21.15.83:0/3771905313 wait complete. 2024-08-19T01:36:24.000 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:23.998+0000 7f92e6fd7740 20 client.15375 trim_cache size 0 max 16384 2024-08-19T01:36:24.000 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stdout:{ 2024-08-19T01:36:24.000 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stdout: "message": "", 2024-08-19T01:36:24.000 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stdout: "return_code": 0 2024-08-19T01:36:24.001 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stdout:} 2024-08-19T01:36:24.001 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:23.998+0000 7f92e2a00640 1 -- 172.21.15.83:0/4085006560 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f92c40a2ad0 msgr2=0x7f92c40a4f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:36:24.001 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:23.998+0000 7f92e2a00640 1 --2- 172.21.15.83:0/4085006560 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f92c40a2ad0 0x7f92c40a4f90 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f92d400a010 tx=0x7f92d401c040 comp rx=0 tx=0).stop 2024-08-19T01:36:24.001 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:23.998+0000 7f92e2a00640 1 -- 172.21.15.83:0/4085006560 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f92dc104a20 msgr2=0x7f92dc19a250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:36:24.001 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:23.998+0000 7f92e2a00640 1 --2- 172.21.15.83:0/4085006560 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f92dc104a20 0x7f92dc19a250 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f92c8009230 tx=0x7f92c8032ce0 comp rx=0 tx=0).stop 2024-08-19T01:36:24.001 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:23.998+0000 7f92e2a00640 1 -- 172.21.15.83:0/4085006560 shutdown_connections 2024-08-19T01:36:24.001 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:23.998+0000 7f92e2a00640 1 --2- 172.21.15.83:0/4085006560 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f92c40a2ad0 0x7f92c40a4f90 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:24.001 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:23.998+0000 7f92e2a00640 1 --2- 172.21.15.83:0/4085006560 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f92dc107350 0x7f92dc1a1280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:24.001 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:23.998+0000 7f92e2a00640 1 --2- 172.21.15.83:0/4085006560 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f92dc104a20 0x7f92dc19a250 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:24.002 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:23.998+0000 7f92e2a00640 1 --2- 172.21.15.83:0/4085006560 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f92dc1020f0 0x7f92dc199d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:24.002 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:23.998+0000 7f92e2a00640 1 -- 172.21.15.83:0/4085006560 >> 172.21.15.83:0/4085006560 conn(0x7f92dc0fbcd0 msgr2=0x7f92dc101780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:36:24.002 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:23.999+0000 7f92e2a00640 1 -- 172.21.15.83:0/4085006560 shutdown_connections 2024-08-19T01:36:24.002 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:23.999+0000 7f92e2a00640 1 -- 172.21.15.83:0/4085006560 wait complete. 2024-08-19T01:36:24.780 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833 shell --fsid bad8780e-5dc9-11ef-bccf-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-08-19T01:36:24.846 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:36:24 smithi083 ceph-mon[17120]: scrub queued for path: / 2024-08-19T01:36:24.846 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:36:24 smithi083 ceph-mon[17120]: scrub summary: idle+waiting paths [/] 2024-08-19T01:36:24.846 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:36:24 smithi083 ceph-mon[17120]: scrub summary: active paths [/] 2024-08-19T01:36:24.952 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:36:24 smithi117 ceph-mon[22016]: scrub queued for path: / 2024-08-19T01:36:24.952 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:36:24 smithi117 ceph-mon[22016]: scrub summary: idle+waiting paths [/] 2024-08-19T01:36:24.952 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:36:24 smithi117 ceph-mon[22016]: scrub summary: active paths [/] 2024-08-19T01:36:24.965 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:36:24 smithi167 ceph-mon[21875]: scrub queued for path: / 2024-08-19T01:36:24.966 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:36:24 smithi167 ceph-mon[21875]: scrub summary: idle+waiting paths [/] 2024-08-19T01:36:24.966 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:36:24 smithi167 ceph-mon[21875]: scrub summary: active paths [/] 2024-08-19T01:36:25.209 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/bad8780e-5dc9-11ef-bccf-c7b262605968/mon.a/config 2024-08-19T01:36:25.801 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:25.800+0000 7fb12aa00640 1 -- 172.21.15.83:0/2339331117 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb11c0319b0 con 0x7fb12c106690 2024-08-19T01:36:25.802 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:36:25 smithi083 ceph-mon[17120]: pgmap v386: 97 pgs: 97 active+clean; 54 MiB data, 555 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 4.5 MiB/s wr, 337 op/s 2024-08-19T01:36:25.802 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:25.800+0000 7fb131a00640 1 -- 172.21.15.83:0/2339331117 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb12c106690 msgr2=0x7fb12c106a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:36:25.803 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:25.800+0000 7fb131a00640 1 --2- 172.21.15.83:0/2339331117 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb12c106690 0x7fb12c106a70 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7fb11c009930 tx=0x7fb11c02a420 comp rx=0 tx=0).stop 2024-08-19T01:36:25.803 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:25.801+0000 7fb131a00640 1 -- 172.21.15.83:0/2339331117 shutdown_connections 2024-08-19T01:36:25.803 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:25.801+0000 7fb131a00640 1 --2- 172.21.15.83:0/2339331117 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb12c100fc0 0x7fb12c10d4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:25.803 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:25.801+0000 7fb131a00640 1 --2- 172.21.15.83:0/2339331117 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb12c100600 0x7fb12c100a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:25.803 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:25.801+0000 7fb131a00640 1 --2- 172.21.15.83:0/2339331117 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb12c106690 0x7fb12c106a70 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:25.803 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:25.801+0000 7fb131a00640 1 -- 172.21.15.83:0/2339331117 >> 172.21.15.83:0/2339331117 conn(0x7fb12c0fc340 msgr2=0x7fb12c0fe760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:36:25.803 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:25.801+0000 7fb131a00640 1 -- 172.21.15.83:0/2339331117 shutdown_connections 2024-08-19T01:36:25.803 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:25.801+0000 7fb131a00640 1 -- 172.21.15.83:0/2339331117 wait complete. 2024-08-19T01:36:25.804 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:25.802+0000 7fb131a00640 1 Processor -- start 2024-08-19T01:36:25.804 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:25.802+0000 7fb131a00640 1 -- start start 2024-08-19T01:36:25.804 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:25.802+0000 7fb131a00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb12c100600 0x7fb12c198d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:36:25.804 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:25.803+0000 7fb131a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb12c100fc0 0x7fb12c1992a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:36:25.804 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:25.803+0000 7fb131a00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb12c106690 0x7fb12c1a3de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:36:25.804 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:25.803+0000 7fb131a00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fb12c1109b0 con 0x7fb12c100fc0 2024-08-19T01:36:25.804 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:25.803+0000 7fb131a00640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fb12c110830 con 0x7fb12c106690 2024-08-19T01:36:25.804 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:25.803+0000 7fb131a00640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fb12c110b30 con 0x7fb12c100600 2024-08-19T01:36:25.806 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:25.803+0000 7fb12b400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb12c100fc0 0x7fb12c1992a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:36:25.806 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:25.803+0000 7fb131000640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb12c106690 0x7fb12c1a3de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:36:25.806 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:25.803+0000 7fb12b400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb12c100fc0 0x7fb12c1992a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:53546/0 (socket says 172.21.15.83:53546) 2024-08-19T01:36:25.806 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:25.803+0000 7fb12b400640 1 -- 172.21.15.83:0/150309754 learned_addr learned my addr 172.21.15.83:0/150309754 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-08-19T01:36:25.806 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:25.803+0000 7fb12be00640 1 --2- 172.21.15.83:0/150309754 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb12c100600 0x7fb12c198d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:36:25.806 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:25.803+0000 7fb12b400640 1 -- 172.21.15.83:0/150309754 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb12c100600 msgr2=0x7fb12c198d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:36:25.806 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:25.803+0000 7fb12b400640 1 --2- 172.21.15.83:0/150309754 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb12c100600 0x7fb12c198d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:25.806 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:25.803+0000 7fb12b400640 1 -- 172.21.15.83:0/150309754 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb12c106690 msgr2=0x7fb12c1a3de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:36:25.806 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:25.803+0000 7fb12b400640 1 --2- 172.21.15.83:0/150309754 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb12c106690 0x7fb12c1a3de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:25.806 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:25.803+0000 7fb12b400640 1 -- 172.21.15.83:0/150309754 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb12c1a4320 con 0x7fb12c100fc0 2024-08-19T01:36:25.806 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:25.803+0000 7fb131000640 1 --2- 172.21.15.83:0/150309754 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb12c106690 0x7fb12c1a3de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-19T01:36:25.806 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:25.804+0000 7fb12b400640 1 --2- 172.21.15.83:0/150309754 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb12c100fc0 0x7fb12c1992a0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7fb11800e9f0 tx=0x7fb11800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:36:25.807 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:25.804+0000 7fb128c00640 1 -- 172.21.15.83:0/150309754 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb11800cd80 con 0x7fb12c100fc0 2024-08-19T01:36:25.807 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:25.804+0000 7fb128c00640 1 -- 172.21.15.83:0/150309754 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb118010910 con 0x7fb12c100fc0 2024-08-19T01:36:25.807 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:25.804+0000 7fb131a00640 1 -- 172.21.15.83:0/150309754 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb12c196b50 con 0x7fb12c100fc0 2024-08-19T01:36:25.807 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:25.804+0000 7fb128c00640 1 -- 172.21.15.83:0/150309754 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb118018990 con 0x7fb12c100fc0 2024-08-19T01:36:25.807 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:25.804+0000 7fb131a00640 1 -- 172.21.15.83:0/150309754 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb12c1970e0 con 0x7fb12c100fc0 2024-08-19T01:36:25.808 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:25.807+0000 7fb128c00640 1 -- 172.21.15.83:0/150309754 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7fb118010ac0 con 0x7fb12c100fc0 2024-08-19T01:36:25.809 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:25.807+0000 7fb135aa2740 1 Processor -- start 2024-08-19T01:36:25.809 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:25.807+0000 7fb135aa2740 1 -- start start 2024-08-19T01:36:25.810 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:25.808+0000 7fb135aa2740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5631fb37dd80 0x5631fb37e160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:36:25.810 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:25.808+0000 7fb135aa2740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5631fb37e6a0 0x5631fb388a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:36:25.810 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:25.808+0000 7fb135aa2740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5631fb388f60 0x5631fb38b350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:36:25.810 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:25.808+0000 7fb135aa2740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x5631fb2e3bf0 con 0x5631fb37e6a0 2024-08-19T01:36:25.810 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:25.808+0000 7fb135aa2740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x5631fb27bbd0 con 0x5631fb388f60 2024-08-19T01:36:25.810 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:25.808+0000 7fb135aa2740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x5631fb2dfbb0 con 0x5631fb37dd80 2024-08-19T01:36:25.810 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:25.808+0000 7fb131000640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5631fb37dd80 0x5631fb37e160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:36:25.810 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:25.808+0000 7fb131000640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5631fb37dd80 0x5631fb37e160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.83:47870/0 (socket says 172.21.15.83:47870) 2024-08-19T01:36:25.811 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:25.808+0000 7fb131000640 1 -- 172.21.15.83:0/3421911066 learned_addr learned my addr 172.21.15.83:0/3421911066 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-08-19T01:36:25.811 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:25.808+0000 7fb12be00640 1 --2- 172.21.15.83:0/3421911066 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5631fb37e6a0 0x5631fb388a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:36:25.811 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:25.808+0000 7fb131000640 1 -- 172.21.15.83:0/3421911066 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5631fb388f60 msgr2=0x5631fb38b350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:36:25.811 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:25.808+0000 7fb131000640 1 --2- 172.21.15.83:0/3421911066 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5631fb388f60 0x5631fb38b350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:25.811 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:25.808+0000 7fb131000640 1 -- 172.21.15.83:0/3421911066 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5631fb37e6a0 msgr2=0x5631fb388a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:36:25.811 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:25.808+0000 7fb131000640 1 --2- 172.21.15.83:0/3421911066 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5631fb37e6a0 0x5631fb388a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:25.811 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:25.808+0000 7fb131000640 1 -- 172.21.15.83:0/3421911066 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5631fb2828a0 con 0x5631fb37dd80 2024-08-19T01:36:25.811 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:25.809+0000 7fb131000640 1 --2- 172.21.15.83:0/3421911066 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5631fb37dd80 0x5631fb37e160 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fb124011aa0 tx=0x7fb124009500 comp rx=0 tx=0).ready entity=mon.1 client_cookie=185c8960650c7209 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:36:25.812 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:25.809+0000 7fb121600640 1 -- 172.21.15.83:0/3421911066 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb124021070 con 0x5631fb37dd80 2024-08-19T01:36:25.812 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:25.809+0000 7fb121600640 1 -- 172.21.15.83:0/3421911066 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb124003e50 con 0x5631fb37dd80 2024-08-19T01:36:25.812 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:25.809+0000 7fb121600640 1 -- 172.21.15.83:0/3421911066 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb12401c480 con 0x5631fb37dd80 2024-08-19T01:36:25.812 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:25.809+0000 7fb128c00640 1 --2- 172.21.15.83:0/150309754 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7fb1100a2c00 0x7fb1100a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:36:25.812 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:25.809+0000 7fb128c00640 1 -- 172.21.15.83:0/150309754 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8269+0+0 (secure 0 0 0) 0x7fb1180cf2d0 con 0x7fb12c100fc0 2024-08-19T01:36:25.813 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:25.810+0000 7fb12be00640 1 --2- 172.21.15.83:0/150309754 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7fb1100a2c00 0x7fb1100a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:36:25.813 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:25.810+0000 7fb135aa2740 1 -- 172.21.15.83:0/3421911066 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5631fb37dd80 msgr2=0x5631fb37e160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:36:25.813 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:25.810+0000 7fb135aa2740 1 --2- 172.21.15.83:0/3421911066 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5631fb37dd80 0x5631fb37e160 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fb124011aa0 tx=0x7fb124009500 comp rx=0 tx=0).stop 2024-08-19T01:36:25.813 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:25.810+0000 7fb135aa2740 1 -- 172.21.15.83:0/3421911066 shutdown_connections 2024-08-19T01:36:25.813 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:25.810+0000 7fb135aa2740 1 --2- 172.21.15.83:0/3421911066 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5631fb388f60 0x5631fb38b350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:25.813 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:25.810+0000 7fb135aa2740 1 --2- 172.21.15.83:0/3421911066 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5631fb37e6a0 0x5631fb388a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:25.813 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:25.810+0000 7fb135aa2740 1 --2- 172.21.15.83:0/3421911066 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5631fb37dd80 0x5631fb37e160 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:25.813 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:25.810+0000 7fb135aa2740 1 -- 172.21.15.83:0/3421911066 >> 172.21.15.83:0/3421911066 conn(0x5631fb37a310 msgr2=0x5631fb37a6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:36:25.813 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:25.810+0000 7fb135aa2740 1 -- 172.21.15.83:0/3421911066 shutdown_connections 2024-08-19T01:36:25.813 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:25.810+0000 7fb12be00640 1 --2- 172.21.15.83:0/150309754 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7fb1100a2c00 0x7fb1100a50c0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fb11c009900 tx=0x7fb11c0058d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:36:25.813 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:25.810+0000 7fb135aa2740 1 -- 172.21.15.83:0/3421911066 wait complete. 2024-08-19T01:36:25.813 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:25.811+0000 7fb135aa2740 1 -- start start 2024-08-19T01:36:25.815 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:25.811+0000 7fb135aa2740 1 Processor -- start 2024-08-19T01:36:25.815 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:25.812+0000 7fb135aa2740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5631fb37dd80 0x5631fb385f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:36:25.815 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:25.812+0000 7fb135aa2740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5631fb37e6a0 0x5631fb386470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:36:25.815 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:25.812+0000 7fb135aa2740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5631fb388f60 0x5631fb3869b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:36:25.815 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:25.812+0000 7fb135aa2740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x5631fb38bb30 con 0x5631fb37dd80 2024-08-19T01:36:25.815 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:25.812+0000 7fb135aa2740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x5631fb29ed40 con 0x5631fb388f60 2024-08-19T01:36:25.815 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:25.812+0000 7fb135aa2740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x5631fb38bcb0 con 0x5631fb37e6a0 2024-08-19T01:36:25.816 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:25.812+0000 7fb135aa2740 10 client.0 _ll_register_callbacks cb 0x5631fb267b10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-19T01:36:25.816 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:25.812+0000 7fb131000640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5631fb388f60 0x5631fb3869b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:36:25.816 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:25.812+0000 7fb131000640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5631fb388f60 0x5631fb3869b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.83:33876/0 (socket says 172.21.15.83:33876) 2024-08-19T01:36:25.816 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:25.812+0000 7fb131000640 1 -- 172.21.15.83:0/603150089 learned_addr learned my addr 172.21.15.83:0/603150089 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-08-19T01:36:25.816 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:25.812+0000 7fb12be00640 1 --2- 172.21.15.83:0/603150089 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5631fb37dd80 0x5631fb385f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:36:25.816 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:25.812+0000 7fb12b400640 1 --2- 172.21.15.83:0/603150089 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5631fb37e6a0 0x5631fb386470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:36:25.816 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:25.813+0000 7fb131000640 1 -- 172.21.15.83:0/603150089 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5631fb37e6a0 msgr2=0x5631fb386470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:36:25.816 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:25.813+0000 7fb131000640 1 --2- 172.21.15.83:0/603150089 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5631fb37e6a0 0x5631fb386470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:25.816 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:25.813+0000 7fb131000640 1 -- 172.21.15.83:0/603150089 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5631fb37dd80 msgr2=0x5631fb385f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:36:25.816 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:25.813+0000 7fb131000640 1 --2- 172.21.15.83:0/603150089 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5631fb37dd80 0x5631fb385f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:25.816 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:25.813+0000 7fb131000640 1 -- 172.21.15.83:0/603150089 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5631fb0553c0 con 0x5631fb388f60 2024-08-19T01:36:25.816 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:25.813+0000 7fb131000640 1 --2- 172.21.15.83:0/603150089 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5631fb388f60 0x5631fb3869b0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fb124006010 tx=0x7fb1240042b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=aee379c9f51652f9 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:36:25.816 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:25.813+0000 7fb116000640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-08-19T01:36:25.817 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:25.813+0000 7fb116000640 1 -- 172.21.15.83:0/603150089 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb124021070 con 0x5631fb388f60 2024-08-19T01:36:25.817 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:25.813+0000 7fb135aa2740 1 -- 172.21.15.83:0/603150089 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb124003e50 con 0x5631fb388f60 2024-08-19T01:36:25.817 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:25.813+0000 7fb116000640 1 -- 172.21.15.83:0/603150089 <== mon.2 v2:172.21.15.117:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8269+0+0 (secure 0 0 0) 0x7fb124003b70 con 0x5631fb388f60 2024-08-19T01:36:25.817 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:25.813+0000 7fb116000640 1 -- 172.21.15.83:0/603150089 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fb12400caf0 con 0x5631fb388f60 2024-08-19T01:36:25.817 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:25.813+0000 7fb135aa2740 10 client.25469 fetch_fsmap learned FSMap version 25 2024-08-19T01:36:25.817 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:25.814+0000 7fb135aa2740 1 -- 172.21.15.83:0/603150089 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5631fb3978b0 con 0x5631fb388f60 2024-08-19T01:36:25.817 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:25.814+0000 7fb116000640 1 -- 172.21.15.83:0/603150089 <== mon.2 v2:172.21.15.117:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb124003e50 con 0x5631fb388f60 2024-08-19T01:36:25.817 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:25.814+0000 7fb116000640 1 -- 172.21.15.83:0/603150089 <== mon.2 v2:172.21.15.117:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fb124027070 con 0x5631fb388f60 2024-08-19T01:36:25.818 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:25.814+0000 7fb135aa2740 10 client.25469 fetch_fsmap finished waiting for FSMap version 25 2024-08-19T01:36:25.818 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:25.814+0000 7fb135aa2740 10 client.25469 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-19T01:36:25.818 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:25.814+0000 7fb135aa2740 20 client.25469 populate_metadata read hostname 'smithi083' 2024-08-19T01:36:25.818 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:25.814+0000 7fb135aa2740 1 --2- 172.21.15.83:0/603150089 >> [v2:172.21.15.117:6838/1967595704,v1:172.21.15.117:6839/1967595704] conn(0x5631fb398cc0 0x5631fb39b0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:36:25.818 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:25.814+0000 7fb135aa2740 4 client.25469 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-19T01:36:25.818 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:25.814+0000 7fb135aa2740 1 -- 172.21.15.83:0/603150089 --> [v2:172.21.15.117:6838/1967595704,v1:172.21.15.117:6839/1967595704] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5631fb2f1160 con 0x5631fb398cc0 2024-08-19T01:36:25.818 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:25.814+0000 7fb12be00640 1 --2- 172.21.15.83:0/603150089 >> [v2:172.21.15.117:6838/1967595704,v1:172.21.15.117:6839/1967595704] conn(0x5631fb398cc0 0x5631fb39b0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:36:25.818 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:25.815+0000 7fb12be00640 1 --2- 172.21.15.83:0/603150089 >> [v2:172.21.15.117:6838/1967595704,v1:172.21.15.117:6839/1967595704] conn(0x5631fb398cc0 0x5631fb39b0b0 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=42b1159d6540e163 server_cookie=765d274e620c49d9 in_seq=0 out_seq=0 2024-08-19T01:36:25.818 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:25.815+0000 7fb116000640 10 client.25469 ms_handle_connect on v2:172.21.15.117:6838/1967595704 2024-08-19T01:36:25.819 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:25.818+0000 7fb116000640 1 -- 172.21.15.83:0/603150089 <== mds.0 v2:172.21.15.117:6838/1967595704 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5631fb2f1160 con 0x5631fb398cc0 2024-08-19T01:36:25.819 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:25.818+0000 7fb116000640 10 client.25469 handle_command_reply: tid=0 2024-08-19T01:36:25.820 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:25.818+0000 7fb135aa2740 1 client.25469 shutdown 2024-08-19T01:36:25.820 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:25.818+0000 7fb135aa2740 1 -- 172.21.15.83:0/603150089 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5631fb388f60 msgr2=0x5631fb3869b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:36:25.820 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:25.818+0000 7fb135aa2740 1 --2- 172.21.15.83:0/603150089 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5631fb388f60 0x5631fb3869b0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fb124006010 tx=0x7fb1240042b0 comp rx=0 tx=0).stop 2024-08-19T01:36:25.820 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:25.818+0000 7fb135aa2740 1 -- 172.21.15.83:0/603150089 shutdown_connections 2024-08-19T01:36:25.821 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:25.818+0000 7fb135aa2740 1 --2- 172.21.15.83:0/603150089 >> [v2:172.21.15.117:6838/1967595704,v1:172.21.15.117:6839/1967595704] conn(0x5631fb398cc0 0x5631fb39b0b0 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:25.821 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:25.819+0000 7fb135aa2740 1 --2- 172.21.15.83:0/603150089 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5631fb388f60 0x5631fb3869b0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:25.821 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:25.819+0000 7fb135aa2740 1 --2- 172.21.15.83:0/603150089 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5631fb37e6a0 0x5631fb386470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:25.821 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:25.819+0000 7fb135aa2740 1 --2- 172.21.15.83:0/603150089 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5631fb37dd80 0x5631fb385f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:25.821 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:25.819+0000 7fb116000640 0 client.25469 ms_handle_reset on v2:172.21.15.117:6838/1967595704 2024-08-19T01:36:25.821 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:25.819+0000 7fb135aa2740 1 -- 172.21.15.83:0/603150089 >> 172.21.15.83:0/603150089 conn(0x5631fb37a310 msgr2=0x5631fb380040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:36:25.821 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:25.819+0000 7fb135aa2740 1 -- 172.21.15.83:0/603150089 shutdown_connections 2024-08-19T01:36:25.821 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:25.819+0000 7fb135aa2740 1 -- 172.21.15.83:0/603150089 wait complete. 2024-08-19T01:36:25.821 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:25.819+0000 7fb135aa2740 20 client.25469 trim_cache size 0 max 16384 2024-08-19T01:36:25.859 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:25.856+0000 7fb135aa2740 1 Processor -- start 2024-08-19T01:36:25.859 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:25.856+0000 7fb135aa2740 1 -- start start 2024-08-19T01:36:25.859 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:25.857+0000 7fb135aa2740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5631fb37dd80 0x5631fb3a07d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:36:25.859 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:25.857+0000 7fb135aa2740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5631fb37e6a0 0x5631fb3a0d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:36:25.859 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:25.857+0000 7fb135aa2740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5631fb388f60 0x5631fb37f7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:36:25.859 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:25.857+0000 7fb135aa2740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fb12c1ad890 con 0x5631fb37dd80 2024-08-19T01:36:25.859 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:25.857+0000 7fb135aa2740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fb12c1ab160 con 0x5631fb388f60 2024-08-19T01:36:25.859 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:25.857+0000 7fb135aa2740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fb12c1a8fb0 con 0x5631fb37e6a0 2024-08-19T01:36:25.860 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:25.858+0000 7fb12b400640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5631fb37e6a0 0x5631fb3a0d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:36:25.860 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:25.858+0000 7fb12b400640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5631fb37e6a0 0x5631fb3a0d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.83:47878/0 (socket says 172.21.15.83:47878) 2024-08-19T01:36:25.860 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:25.858+0000 7fb12b400640 1 -- 172.21.15.83:0/89272432 learned_addr learned my addr 172.21.15.83:0/89272432 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-08-19T01:36:25.860 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:25.858+0000 7fb131000640 1 --2- 172.21.15.83:0/89272432 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5631fb388f60 0x5631fb37f7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:36:25.860 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:25.858+0000 7fb12b400640 1 -- 172.21.15.83:0/89272432 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5631fb388f60 msgr2=0x5631fb37f7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:36:25.860 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:25.858+0000 7fb12b400640 1 --2- 172.21.15.83:0/89272432 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5631fb388f60 0x5631fb37f7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:25.860 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:25.858+0000 7fb12b400640 1 -- 172.21.15.83:0/89272432 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5631fb37dd80 msgr2=0x5631fb3a07d0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-08-19T01:36:25.861 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:25.858+0000 7fb12b400640 1 --2- 172.21.15.83:0/89272432 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5631fb37dd80 0x5631fb3a07d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:25.861 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:25.858+0000 7fb12b400640 1 -- 172.21.15.83:0/89272432 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5631fb37b3c0 con 0x5631fb37e6a0 2024-08-19T01:36:25.861 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:25.858+0000 7fb12b400640 1 --2- 172.21.15.83:0/89272432 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5631fb37e6a0 0x5631fb3a0d10 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fb118098590 tx=0x7fb11809b520 comp rx=0 tx=0).ready entity=mon.1 client_cookie=950d67f7f2165a1f server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:36:25.861 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:25.859+0000 7fb120c00640 1 -- 172.21.15.83:0/89272432 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1180a4780 con 0x5631fb37e6a0 2024-08-19T01:36:25.861 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:25.859+0000 7fb120c00640 1 -- 172.21.15.83:0/89272432 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb1180b4070 con 0x5631fb37e6a0 2024-08-19T01:36:25.861 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:25.859+0000 7fb120c00640 1 -- 172.21.15.83:0/89272432 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb11809b010 con 0x5631fb37e6a0 2024-08-19T01:36:25.861 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:25.860+0000 7fb135aa2740 1 -- 172.21.15.83:0/89272432 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5631fb37e6a0 msgr2=0x5631fb3a0d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:36:25.862 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:25.860+0000 7fb135aa2740 1 --2- 172.21.15.83:0/89272432 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5631fb37e6a0 0x5631fb3a0d10 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fb118098590 tx=0x7fb11809b520 comp rx=0 tx=0).stop 2024-08-19T01:36:25.862 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:25.860+0000 7fb135aa2740 1 -- 172.21.15.83:0/89272432 shutdown_connections 2024-08-19T01:36:25.862 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:25.860+0000 7fb135aa2740 1 --2- 172.21.15.83:0/89272432 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5631fb388f60 0x5631fb37f7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:25.862 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:25.860+0000 7fb135aa2740 1 --2- 172.21.15.83:0/89272432 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5631fb37e6a0 0x5631fb3a0d10 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:25.862 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:25.860+0000 7fb135aa2740 1 --2- 172.21.15.83:0/89272432 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5631fb37dd80 0x5631fb3a07d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:25.862 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:25.860+0000 7fb135aa2740 1 -- 172.21.15.83:0/89272432 >> 172.21.15.83:0/89272432 conn(0x5631fb37a310 msgr2=0x5631fb387fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:36:25.863 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:25.861+0000 7fb135aa2740 1 -- 172.21.15.83:0/89272432 shutdown_connections 2024-08-19T01:36:25.863 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:25.861+0000 7fb135aa2740 1 -- 172.21.15.83:0/89272432 wait complete. 2024-08-19T01:36:25.863 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:25.861+0000 7fb135aa2740 1 -- start start 2024-08-19T01:36:25.863 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:25.861+0000 7fb135aa2740 1 Processor -- start 2024-08-19T01:36:25.863 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:25.862+0000 7fb135aa2740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5631fb37dd80 0x5631fb391380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:36:25.863 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:25.862+0000 7fb135aa2740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5631fb37e6a0 0x5631fb38c6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:36:25.864 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:25.862+0000 7fb135aa2740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5631fb388f60 0x5631fb38cbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:36:25.865 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:25.862+0000 7fb135aa2740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fb12c1a9620 con 0x5631fb37dd80 2024-08-19T01:36:25.865 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:25.862+0000 7fb135aa2740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fb12c1aaa90 con 0x5631fb388f60 2024-08-19T01:36:25.865 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:25.862+0000 7fb135aa2740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fb12c1a9cf0 con 0x5631fb37e6a0 2024-08-19T01:36:25.865 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:25.862+0000 7fb135aa2740 10 client.0 _ll_register_callbacks cb 0x7fb12401f070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-19T01:36:25.865 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:25.862+0000 7fb12b400640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5631fb37e6a0 0x5631fb38c6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:36:25.865 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:25.862+0000 7fb12b400640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5631fb37e6a0 0x5631fb38c6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.83:47892/0 (socket says 172.21.15.83:47892) 2024-08-19T01:36:25.865 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:25.862+0000 7fb12b400640 1 -- 172.21.15.83:0/404673908 learned_addr learned my addr 172.21.15.83:0/404673908 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-08-19T01:36:25.865 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:25.862+0000 7fb131000640 1 --2- 172.21.15.83:0/404673908 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5631fb388f60 0x5631fb38cbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:36:25.865 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:25.863+0000 7fb12be00640 1 --2- 172.21.15.83:0/404673908 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5631fb37dd80 0x5631fb391380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:36:25.865 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:25.863+0000 7fb131000640 1 -- 172.21.15.83:0/404673908 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5631fb37e6a0 msgr2=0x5631fb38c6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:36:25.865 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:25.863+0000 7fb131000640 1 --2- 172.21.15.83:0/404673908 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5631fb37e6a0 0x5631fb38c6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:25.865 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:25.863+0000 7fb131000640 1 -- 172.21.15.83:0/404673908 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5631fb37dd80 msgr2=0x5631fb391380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:36:25.865 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:25.863+0000 7fb131000640 1 --2- 172.21.15.83:0/404673908 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5631fb37dd80 0x5631fb391380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:25.865 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:25.863+0000 7fb131000640 1 -- 172.21.15.83:0/404673908 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5631fb38d130 con 0x5631fb388f60 2024-08-19T01:36:25.866 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:25.863+0000 7fb12be00640 1 --2- 172.21.15.83:0/404673908 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5631fb37dd80 0x5631fb391380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-19T01:36:25.866 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:25.864+0000 7fb131000640 1 --2- 172.21.15.83:0/404673908 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5631fb388f60 0x5631fb38cbf0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fb124000f80 tx=0x7fb1240058a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=55c26f8649476ef2 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:36:25.867 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:25.864+0000 7fb116000640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-08-19T01:36:25.867 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:25.864+0000 7fb116000640 1 -- 172.21.15.83:0/404673908 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb124021040 con 0x5631fb388f60 2024-08-19T01:36:25.867 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:25.864+0000 7fb116000640 1 -- 172.21.15.83:0/404673908 <== mon.2 v2:172.21.15.117:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8269+0+0 (secure 0 0 0) 0x7fb124002030 con 0x5631fb388f60 2024-08-19T01:36:25.867 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:25.864+0000 7fb135aa2740 1 -- 172.21.15.83:0/404673908 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb1180b4070 con 0x5631fb388f60 2024-08-19T01:36:25.867 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:25.864+0000 7fb116000640 1 -- 172.21.15.83:0/404673908 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fb1240044a0 con 0x5631fb388f60 2024-08-19T01:36:25.867 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:25.864+0000 7fb135aa2740 10 client.25475 fetch_fsmap learned FSMap version 25 2024-08-19T01:36:25.867 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:25.864+0000 7fb135aa2740 1 -- 172.21.15.83:0/404673908 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5631fb40bcf0 con 0x5631fb388f60 2024-08-19T01:36:25.867 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:25.865+0000 7fb116000640 1 -- 172.21.15.83:0/404673908 <== mon.2 v2:172.21.15.117:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb124038020 con 0x5631fb388f60 2024-08-19T01:36:25.868 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:25.865+0000 7fb116000640 1 -- 172.21.15.83:0/404673908 <== mon.2 v2:172.21.15.117:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fb124027040 con 0x5631fb388f60 2024-08-19T01:36:25.868 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:25.865+0000 7fb135aa2740 10 client.25475 fetch_fsmap finished waiting for FSMap version 25 2024-08-19T01:36:25.868 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:25.865+0000 7fb135aa2740 10 client.25475 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-19T01:36:25.868 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:25.865+0000 7fb135aa2740 20 client.25475 populate_metadata read hostname 'smithi083' 2024-08-19T01:36:25.868 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:25.865+0000 7fb135aa2740 1 --2- 172.21.15.83:0/404673908 >> [v2:172.21.15.117:6838/1967595704,v1:172.21.15.117:6839/1967595704] conn(0x5631fb40c460 0x5631fb40e880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:36:25.868 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:25.865+0000 7fb135aa2740 4 client.25475 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-08-19T01:36:25.868 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:25.865+0000 7fb135aa2740 1 -- 172.21.15.83:0/404673908 --> [v2:172.21.15.117:6838/1967595704,v1:172.21.15.117:6839/1967595704] -- command(tid 0: {"prefix": "scrub status"}) -- 0x7fb1180b4070 con 0x5631fb40c460 2024-08-19T01:36:25.869 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:25.867+0000 7fb12be00640 1 --2- 172.21.15.83:0/404673908 >> [v2:172.21.15.117:6838/1967595704,v1:172.21.15.117:6839/1967595704] conn(0x5631fb40c460 0x5631fb40e880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:36:25.869 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:25.867+0000 7fb12be00640 1 --2- 172.21.15.83:0/404673908 >> [v2:172.21.15.117:6838/1967595704,v1:172.21.15.117:6839/1967595704] conn(0x5631fb40c460 0x5631fb40e880 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f7589e9a58cab803 server_cookie=3fb77628cc7fe782 in_seq=0 out_seq=0 2024-08-19T01:36:25.869 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:25.867+0000 7fb116000640 10 client.25475 ms_handle_connect on v2:172.21.15.117:6838/1967595704 2024-08-19T01:36:25.872 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:25.870+0000 7fb116000640 1 -- 172.21.15.83:0/404673908 <== mds.0 v2:172.21.15.117:6838/1967595704 1 ==== command_reply(tid 0: 0 ) ==== 8+0+268 (crc 0 0 0) 0x7fb1180b4070 con 0x5631fb40c460 2024-08-19T01:36:25.872 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:25.870+0000 7fb116000640 10 client.25475 handle_command_reply: tid=0 2024-08-19T01:36:25.872 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:25.870+0000 7fb135aa2740 1 client.25475 shutdown 2024-08-19T01:36:25.872 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:25.870+0000 7fb135aa2740 1 -- 172.21.15.83:0/404673908 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5631fb388f60 msgr2=0x5631fb38cbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:36:25.872 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:25.870+0000 7fb135aa2740 1 --2- 172.21.15.83:0/404673908 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5631fb388f60 0x5631fb38cbf0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fb124000f80 tx=0x7fb1240058a0 comp rx=0 tx=0).stop 2024-08-19T01:36:25.873 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:25.871+0000 7fb135aa2740 1 -- 172.21.15.83:0/404673908 shutdown_connections 2024-08-19T01:36:25.873 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:25.871+0000 7fb135aa2740 1 --2- 172.21.15.83:0/404673908 >> [v2:172.21.15.117:6838/1967595704,v1:172.21.15.117:6839/1967595704] conn(0x5631fb40c460 0x5631fb40e880 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:25.873 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:25.871+0000 7fb135aa2740 1 --2- 172.21.15.83:0/404673908 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5631fb388f60 0x5631fb38cbf0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:25.873 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:25.871+0000 7fb135aa2740 1 --2- 172.21.15.83:0/404673908 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5631fb37e6a0 0x5631fb38c6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:25.873 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:25.871+0000 7fb135aa2740 1 --2- 172.21.15.83:0/404673908 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5631fb37dd80 0x5631fb391380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:25.873 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:25.871+0000 7fb135aa2740 1 -- 172.21.15.83:0/404673908 >> 172.21.15.83:0/404673908 conn(0x5631fb37a310 msgr2=0x5631fb385ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:36:25.873 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:25.871+0000 7fb116000640 0 client.25475 ms_handle_reset on v2:172.21.15.117:6838/1967595704 2024-08-19T01:36:25.873 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:25.871+0000 7fb135aa2740 1 -- 172.21.15.83:0/404673908 shutdown_connections 2024-08-19T01:36:25.873 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:25.871+0000 7fb135aa2740 1 -- 172.21.15.83:0/404673908 wait complete. 2024-08-19T01:36:25.873 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:25.871+0000 7fb135aa2740 20 client.25475 trim_cache size 0 max 16384 2024-08-19T01:36:25.874 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-08-19T01:36:25.874 INFO:teuthology.orchestra.run.smithi083.stdout: "status": "scrub active (2036 inodes in the stack)", 2024-08-19T01:36:25.874 INFO:teuthology.orchestra.run.smithi083.stdout: "scrubs": { 2024-08-19T01:36:25.874 INFO:teuthology.orchestra.run.smithi083.stdout: "a9a924bb-6b35-4256-8105-61ffbeb040cb": { 2024-08-19T01:36:25.874 INFO:teuthology.orchestra.run.smithi083.stdout: "path": "/", 2024-08-19T01:36:25.874 INFO:teuthology.orchestra.run.smithi083.stdout: "tag": "a9a924bb-6b35-4256-8105-61ffbeb040cb", 2024-08-19T01:36:25.874 INFO:teuthology.orchestra.run.smithi083.stdout: "options": "recursive,force" 2024-08-19T01:36:25.874 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-08-19T01:36:25.874 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-08-19T01:36:25.875 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-08-19T01:36:25.875 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:25.873+0000 7fb131a00640 1 -- 172.21.15.83:0/150309754 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7fb1100a2c00 msgr2=0x7fb1100a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:36:25.875 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:25.873+0000 7fb131a00640 1 --2- 172.21.15.83:0/150309754 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7fb1100a2c00 0x7fb1100a50c0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fb11c009900 tx=0x7fb11c0058d0 comp rx=0 tx=0).stop 2024-08-19T01:36:25.875 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:25.873+0000 7fb131a00640 1 -- 172.21.15.83:0/150309754 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb12c100fc0 msgr2=0x7fb12c1992a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:36:25.875 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:25.873+0000 7fb131a00640 1 --2- 172.21.15.83:0/150309754 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb12c100fc0 0x7fb12c1992a0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7fb11800e9f0 tx=0x7fb11800eeb0 comp rx=0 tx=0).stop 2024-08-19T01:36:25.875 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:25.874+0000 7fb131a00640 1 -- 172.21.15.83:0/150309754 shutdown_connections 2024-08-19T01:36:25.875 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:25.874+0000 7fb131a00640 1 --2- 172.21.15.83:0/150309754 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7fb1100a2c00 0x7fb1100a50c0 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:25.875 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:25.874+0000 7fb131a00640 1 --2- 172.21.15.83:0/150309754 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb12c106690 0x7fb12c1a3de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:25.876 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:25.874+0000 7fb131a00640 1 --2- 172.21.15.83:0/150309754 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb12c100fc0 0x7fb12c1992a0 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:25.876 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:25.874+0000 7fb131a00640 1 --2- 172.21.15.83:0/150309754 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb12c100600 0x7fb12c198d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:25.876 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:25.874+0000 7fb131a00640 1 -- 172.21.15.83:0/150309754 >> 172.21.15.83:0/150309754 conn(0x7fb12c0fc340 msgr2=0x7fb12c0fdac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:36:25.876 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:25.875+0000 7fb131a00640 1 -- 172.21.15.83:0/150309754 shutdown_connections 2024-08-19T01:36:25.876 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:36:25.875+0000 7fb131a00640 1 -- 172.21.15.83:0/150309754 wait complete. 2024-08-19T01:36:25.952 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:36:25 smithi117 ceph-mon[22016]: pgmap v386: 97 pgs: 97 active+clean; 54 MiB data, 555 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 4.5 MiB/s wr, 337 op/s 2024-08-19T01:36:25.965 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:36:25 smithi167 ceph-mon[21875]: pgmap v386: 97 pgs: 97 active+clean; 54 MiB data, 555 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 4.5 MiB/s wr, 337 op/s 2024-08-19T01:36:26.649 INFO:tasks.cephfs.filesystem:scrub status for tag:a9a924bb-6b35-4256-8105-61ffbeb040cb - {'path': '/', 'tag': 'a9a924bb-6b35-4256-8105-61ffbeb040cb', 'options': 'recursive,force'} 2024-08-19T01:36:27.904 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:36:27 smithi083 ceph-mon[17120]: pgmap v387: 97 pgs: 97 active+clean; 53 MiB data, 575 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 4.7 MiB/s wr, 449 op/s 2024-08-19T01:36:27.952 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:36:27 smithi117 ceph-mon[22016]: pgmap v387: 97 pgs: 97 active+clean; 53 MiB data, 575 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 4.7 MiB/s wr, 449 op/s 2024-08-19T01:36:27.966 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:36:27 smithi167 ceph-mon[21875]: pgmap v387: 97 pgs: 97 active+clean; 53 MiB data, 575 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 4.7 MiB/s wr, 449 op/s 2024-08-19T01:36:29.916 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:36:29 smithi083 ceph-mon[17120]: pgmap v388: 97 pgs: 97 active+clean; 49 MiB data, 555 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 5.0 MiB/s wr, 788 op/s 2024-08-19T01:36:29.952 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:36:29 smithi117 ceph-mon[22016]: pgmap v388: 97 pgs: 97 active+clean; 49 MiB data, 555 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 5.0 MiB/s wr, 788 op/s 2024-08-19T01:36:29.966 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:36:29 smithi167 ceph-mon[21875]: pgmap v388: 97 pgs: 97 active+clean; 49 MiB data, 555 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 5.0 MiB/s wr, 788 op/s 2024-08-19T01:36:30.518 INFO:tasks.workunit.client.0.smithi083.stdout: 26 195496 432 137234 413 108298 1290 2024-08-19T01:36:31.905 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:36:31 smithi083 ceph-mon[17120]: pgmap v389: 97 pgs: 97 active+clean; 44 MiB data, 545 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 4.2 MiB/s wr, 773 op/s 2024-08-19T01:36:31.952 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:36:31 smithi117 ceph-mon[22016]: pgmap v389: 97 pgs: 97 active+clean; 44 MiB data, 545 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 4.2 MiB/s wr, 773 op/s 2024-08-19T01:36:31.966 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:36:31 smithi167 ceph-mon[21875]: pgmap v389: 97 pgs: 97 active+clean; 44 MiB data, 545 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 4.2 MiB/s wr, 773 op/s 2024-08-19T01:36:33.904 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:36:33 smithi083 ceph-mon[17120]: pgmap v390: 97 pgs: 97 active+clean; 44 MiB data, 545 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 3.5 MiB/s wr, 754 op/s 2024-08-19T01:36:33.952 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:36:33 smithi117 ceph-mon[22016]: pgmap v390: 97 pgs: 97 active+clean; 44 MiB data, 545 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 3.5 MiB/s wr, 754 op/s 2024-08-19T01:36:33.966 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:36:33 smithi167 ceph-mon[21875]: pgmap v390: 97 pgs: 97 active+clean; 44 MiB data, 545 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 3.5 MiB/s wr, 754 op/s 2024-08-19T01:36:34.866 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:36:34 smithi083 ceph-mon[17120]: pgmap v391: 97 pgs: 97 active+clean; 50 MiB data, 569 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 3.7 MiB/s wr, 836 op/s 2024-08-19T01:36:34.952 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:36:34 smithi117 ceph-mon[22016]: pgmap v391: 97 pgs: 97 active+clean; 50 MiB data, 569 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 3.7 MiB/s wr, 836 op/s 2024-08-19T01:36:34.965 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:36:34 smithi167 ceph-mon[21875]: pgmap v391: 97 pgs: 97 active+clean; 50 MiB data, 569 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 3.7 MiB/s wr, 836 op/s 2024-08-19T01:36:36.419 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:36:36 smithi083 ceph-mon[17120]: scrub summary: idle+waiting paths [/] 2024-08-19T01:36:36.452 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:36:36 smithi117 ceph-mon[22016]: scrub summary: idle+waiting paths [/] 2024-08-19T01:36:36.465 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:36:36 smithi167 ceph-mon[21875]: scrub summary: idle+waiting paths [/] 2024-08-19T01:36:37.390 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:37.388+0000 7f4172600640 1 -- 172.21.15.83:0/8366360 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f416c0733b0 msgr2=0x7f416c10d220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:36:37.390 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:37.388+0000 7f4172600640 1 --2- 172.21.15.83:0/8366360 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f416c0733b0 0x7f416c10d220 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f415c01ab70 tx=0x7f415c03b6d0 comp rx=0 tx=0).stop 2024-08-19T01:36:37.415 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:37.410+0000 7f4172600640 1 -- 172.21.15.83:0/8366360 shutdown_connections 2024-08-19T01:36:37.415 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:37.410+0000 7f4172600640 1 --2- 172.21.15.83:0/8366360 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f416c10d8e0 0x7f416c10fda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:37.415 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:37.410+0000 7f4172600640 1 --2- 172.21.15.83:0/8366360 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f416c0733b0 0x7f416c10d220 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:37.415 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:37.410+0000 7f4172600640 1 --2- 172.21.15.83:0/8366360 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f416c0729e0 0x7f416c072de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:37.415 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:37.410+0000 7f4172600640 1 -- 172.21.15.83:0/8366360 >> 172.21.15.83:0/8366360 conn(0x7f416c06e880 msgr2=0x7f416c070cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:36:37.426 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:37.419+0000 7f4172600640 1 -- 172.21.15.83:0/8366360 shutdown_connections 2024-08-19T01:36:37.430 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:37.424+0000 7f4172600640 1 -- 172.21.15.83:0/8366360 wait complete. 2024-08-19T01:36:37.430 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:37.425+0000 7f4172600640 1 Processor -- start 2024-08-19T01:36:37.435 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:37.433+0000 7f4172600640 1 -- start start 2024-08-19T01:36:37.435 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:37.433+0000 7f4172600640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f416c0729e0 0x7f416c19e420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:36:37.435 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:37.433+0000 7f4172600640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f416c0733b0 0x7f416c19e9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:36:37.435 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:37.433+0000 7f4172600640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f416c10d8e0 0x7f416c1a5730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:36:37.435 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:37.433+0000 7f4172600640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f416c1133d0 con 0x7f416c10d8e0 2024-08-19T01:36:37.435 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:37.433+0000 7f4172600640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f416c113250 con 0x7f416c0733b0 2024-08-19T01:36:37.435 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:37.433+0000 7f4172600640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f416c113550 con 0x7f416c0729e0 2024-08-19T01:36:37.436 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:37.434+0000 7f416be00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f416c0733b0 0x7f416c19e9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:36:37.436 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:37.434+0000 7f416be00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f416c0733b0 0x7f416c19e9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.83:56156/0 (socket says 172.21.15.83:56156) 2024-08-19T01:36:37.436 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:37.434+0000 7f416be00640 1 -- 172.21.15.83:0/3360651124 learned_addr learned my addr 172.21.15.83:0/3360651124 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-08-19T01:36:37.436 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:37.434+0000 7f4171c00640 1 --2- 172.21.15.83:0/3360651124 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f416c10d8e0 0x7f416c1a5730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:36:37.436 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:37.434+0000 7f416be00640 1 -- 172.21.15.83:0/3360651124 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f416c0729e0 msgr2=0x7f416c19e420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:36:37.436 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:37.434+0000 7f416be00640 1 --2- 172.21.15.83:0/3360651124 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f416c0729e0 0x7f416c19e420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:37.436 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:37.434+0000 7f416be00640 1 -- 172.21.15.83:0/3360651124 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f416c10d8e0 msgr2=0x7f416c1a5730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:36:37.437 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:37.434+0000 7f416be00640 1 --2- 172.21.15.83:0/3360651124 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f416c10d8e0 0x7f416c1a5730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:37.437 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:37.434+0000 7f416be00640 1 -- 172.21.15.83:0/3360651124 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f416c1a5dd0 con 0x7f416c0733b0 2024-08-19T01:36:37.437 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:37.435+0000 7f4171c00640 1 --2- 172.21.15.83:0/3360651124 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f416c10d8e0 0x7f416c1a5730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-19T01:36:37.451 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:37.447+0000 7f416be00640 1 --2- 172.21.15.83:0/3360651124 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f416c0733b0 0x7f416c19e9e0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f415c004510 tx=0x7f415c004540 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:36:37.451 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:37.448+0000 7f4169600640 1 -- 172.21.15.83:0/3360651124 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f415c04f020 con 0x7f416c0733b0 2024-08-19T01:36:37.451 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:37.448+0000 7f4169600640 1 -- 172.21.15.83:0/3360651124 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f415c004950 con 0x7f416c0733b0 2024-08-19T01:36:37.452 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:37.450+0000 7f4169600640 1 -- 172.21.15.83:0/3360651124 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f415c046070 con 0x7f416c0733b0 2024-08-19T01:36:37.452 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:37.450+0000 7f4172600640 1 -- 172.21.15.83:0/3360651124 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f416c1a6060 con 0x7f416c0733b0 2024-08-19T01:36:37.452 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:37.450+0000 7f4172600640 1 -- 172.21.15.83:0/3360651124 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f416c1a6550 con 0x7f416c0733b0 2024-08-19T01:36:37.452 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:36:37 smithi117 ceph-mon[22016]: pgmap v392: 97 pgs: 97 active+clean; 54 MiB data, 584 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 2.2 MiB/s wr, 674 op/s 2024-08-19T01:36:37.463 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:37.460+0000 7f4169600640 1 -- 172.21.15.83:0/3360651124 <== mon.2 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7f415c052020 con 0x7f416c0733b0 2024-08-19T01:36:37.464 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:37.463+0000 7f4176a3a740 1 Processor -- start 2024-08-19T01:36:37.465 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:37.464+0000 7f4176a3a740 1 -- start start 2024-08-19T01:36:37.465 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:36:37 smithi167 ceph-mon[21875]: pgmap v392: 97 pgs: 97 active+clean; 54 MiB data, 584 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 2.2 MiB/s wr, 674 op/s 2024-08-19T01:36:37.466 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:37.465+0000 7f4176a3a740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556085b37660 0x556085b37a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:36:37.474 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:37.465+0000 7f4169600640 1 --2- 172.21.15.83:0/3360651124 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f41500a2c80 0x7f41500a5140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:36:37.474 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:37.465+0000 7f4176a3a740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556085b37fa0 0x556085b3ca60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:36:37.474 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:37.465+0000 7f4176a3a740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556085b3d030 0x556085b3f4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:36:37.474 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:37.465+0000 7f4171c00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556085b37660 0x556085b37a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:36:37.474 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:37.465+0000 7f4171c00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556085b37660 0x556085b37a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.83:49766/0 (socket says 172.21.15.83:49766) 2024-08-19T01:36:37.475 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:37.465+0000 7f4171c00640 1 -- 172.21.15.83:0/3346699544 learned_addr learned my addr 172.21.15.83:0/3346699544 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-08-19T01:36:37.475 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:37.465+0000 7f4176a3a740 1 -- 172.21.15.83:0/3346699544 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x556085a9b890 con 0x556085b3d030 2024-08-19T01:36:37.475 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:37.465+0000 7f4176a3a740 1 -- 172.21.15.83:0/3346699544 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x556085a33d90 con 0x556085b37fa0 2024-08-19T01:36:37.475 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:37.465+0000 7f4176a3a740 1 -- 172.21.15.83:0/3346699544 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x556085a97850 con 0x556085b37660 2024-08-19T01:36:37.475 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:37.466+0000 7f4171c00640 1 --2- 172.21.15.83:0/3346699544 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556085b3d030 0x556085b3f4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:36:37.475 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:37.467+0000 7f4171c00640 1 -- 172.21.15.83:0/3346699544 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556085b37660 msgr2=0x556085b37a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:36:37.475 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:37.467+0000 7f4171c00640 1 --2- 172.21.15.83:0/3346699544 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556085b37660 0x556085b37a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:37.475 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:37.467+0000 7f4171c00640 1 -- 172.21.15.83:0/3346699544 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556085b37fa0 msgr2=0x556085b3ca60 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-08-19T01:36:37.475 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:37.467+0000 7f4171c00640 1 --2- 172.21.15.83:0/3346699544 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556085b37fa0 0x556085b3ca60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:37.475 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:37.467+0000 7f4171c00640 1 -- 172.21.15.83:0/3346699544 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5560857af210 con 0x556085b3d030 2024-08-19T01:36:37.475 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:37.467+0000 7f4171c00640 1 --2- 172.21.15.83:0/3346699544 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556085b3d030 0x556085b3f4f0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f415800ae30 tx=0x7f41580046e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ae5c2c5c915ae847 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:36:37.475 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:37.469+0000 7f4171200640 1 --2- 172.21.15.83:0/3360651124 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f41500a2c80 0x7f41500a5140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:36:37.475 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:37.470+0000 7f4171200640 1 --2- 172.21.15.83:0/3360651124 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f41500a2c80 0x7f41500a5140 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f416400abe0 tx=0x7f4164009230 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:36:37.476 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:37.471+0000 7f4169600640 1 -- 172.21.15.83:0/3360651124 <== mon.2 v2:172.21.15.117:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8269+0+0 (secure 0 0 0) 0x7f415c0fff20 con 0x7f416c0733b0 2024-08-19T01:36:37.478 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:37.477+0000 7f4162000640 1 -- 172.21.15.83:0/3346699544 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f415800f040 con 0x556085b3d030 2024-08-19T01:36:37.478 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:37.477+0000 7f4162000640 1 -- 172.21.15.83:0/3346699544 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4158003b70 con 0x556085b3d030 2024-08-19T01:36:37.480 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:37.477+0000 7f4162000640 1 -- 172.21.15.83:0/3346699544 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4158016630 con 0x556085b3d030 2024-08-19T01:36:37.482 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:37.479+0000 7f4176a3a740 1 -- 172.21.15.83:0/3346699544 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556085b3d030 msgr2=0x556085b3f4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:36:37.483 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:37.479+0000 7f4176a3a740 1 --2- 172.21.15.83:0/3346699544 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556085b3d030 0x556085b3f4f0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f415800ae30 tx=0x7f41580046e0 comp rx=0 tx=0).stop 2024-08-19T01:36:37.483 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:37.482+0000 7f4176a3a740 1 -- 172.21.15.83:0/3346699544 shutdown_connections 2024-08-19T01:36:37.483 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:37.482+0000 7f4176a3a740 1 --2- 172.21.15.83:0/3346699544 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556085b3d030 0x556085b3f4f0 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:37.483 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:37.482+0000 7f4176a3a740 1 --2- 172.21.15.83:0/3346699544 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556085b37fa0 0x556085b3ca60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:37.483 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:37.482+0000 7f4176a3a740 1 --2- 172.21.15.83:0/3346699544 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556085b37660 0x556085b37a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:37.484 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:37.482+0000 7f4176a3a740 1 -- 172.21.15.83:0/3346699544 >> 172.21.15.83:0/3346699544 conn(0x556085b32cd0 msgr2=0x556085b330d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:36:37.484 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:37.482+0000 7f4176a3a740 1 -- 172.21.15.83:0/3346699544 shutdown_connections 2024-08-19T01:36:37.486 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:37.482+0000 7f4176a3a740 1 -- 172.21.15.83:0/3346699544 wait complete. 2024-08-19T01:36:37.486 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:37.483+0000 7f4176a3a740 1 -- start start 2024-08-19T01:36:37.487 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:37.483+0000 7f4176a3a740 1 Processor -- start 2024-08-19T01:36:37.487 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:37.485+0000 7f4176a3a740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556085b37660 0x556085b41a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:36:37.487 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:37.486+0000 7f4171200640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556085b37660 0x556085b41a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:36:37.487 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:37.486+0000 7f4171200640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556085b37660 0x556085b41a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.83:56168/0 (socket says 172.21.15.83:56168) 2024-08-19T01:36:37.487 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:37.486+0000 7f4171200640 1 -- 172.21.15.83:0/1948078540 learned_addr learned my addr 172.21.15.83:0/1948078540 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-08-19T01:36:37.494 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:37.486+0000 7f4176a3a740 1 --2- 172.21.15.83:0/1948078540 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556085b37fa0 0x556085b41f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:36:37.495 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:37.490+0000 7f4176a3a740 1 --2- 172.21.15.83:0/1948078540 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556085b3d030 0x556085b48f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:36:37.495 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:37.490+0000 7f4176a3a740 1 -- 172.21.15.83:0/1948078540 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x556085b400a0 con 0x556085b3d030 2024-08-19T01:36:37.495 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:37.490+0000 7f4176a3a740 1 -- 172.21.15.83:0/1948078540 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x556085a56f70 con 0x556085b37660 2024-08-19T01:36:37.495 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:37.490+0000 7f4176a3a740 1 -- 172.21.15.83:0/1948078540 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x556085b40220 con 0x556085b37fa0 2024-08-19T01:36:37.495 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:37.490+0000 7f4176a3a740 10 client.0 _ll_register_callbacks cb 0x556085a938d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-19T01:36:37.495 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:37.490+0000 7f4171c00640 1 --2- 172.21.15.83:0/1948078540 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556085b3d030 0x556085b48f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:36:37.495 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:37.491+0000 7f4171c00640 1 -- 172.21.15.83:0/1948078540 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556085b37fa0 msgr2=0x556085b41f90 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-08-19T01:36:37.495 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:37.491+0000 7f4171c00640 1 --2- 172.21.15.83:0/1948078540 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556085b37fa0 0x556085b41f90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:37.495 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:37.491+0000 7f4171c00640 1 -- 172.21.15.83:0/1948078540 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556085b37660 msgr2=0x556085b41a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:36:37.495 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:37.491+0000 7f4171c00640 1 --2- 172.21.15.83:0/1948078540 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556085b37660 0x556085b41a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:37.495 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:37.491+0000 7f4171c00640 1 -- 172.21.15.83:0/1948078540 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55608587b620 con 0x556085b3d030 2024-08-19T01:36:37.495 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:37.492+0000 7f4171c00640 1 --2- 172.21.15.83:0/1948078540 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556085b3d030 0x556085b48f60 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f4158018040 tx=0x7f4158004250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=39501c658fb0e3ed server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:36:37.496 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:37.492+0000 7f4156a00640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-08-19T01:36:37.496 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:37.492+0000 7f4156a00640 1 -- 172.21.15.83:0/1948078540 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4158025070 con 0x556085b3d030 2024-08-19T01:36:37.496 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:37.493+0000 7f4156a00640 1 -- 172.21.15.83:0/1948078540 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8269+0+0 (secure 0 0 0) 0x7f415801c040 con 0x556085b3d030 2024-08-19T01:36:37.496 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:37.493+0000 7f4171200640 1 --2- 172.21.15.83:0/1948078540 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556085b37660 0x556085b41a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-19T01:36:37.498 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:37.497+0000 7f4176a3a740 1 -- 172.21.15.83:0/1948078540 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4158003b70 con 0x556085b3d030 2024-08-19T01:36:37.499 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:37.497+0000 7f4156a00640 1 -- 172.21.15.83:0/1948078540 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f4158025070 con 0x556085b3d030 2024-08-19T01:36:37.499 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:37.497+0000 7f4176a3a740 10 client.15405 fetch_fsmap learned FSMap version 25 2024-08-19T01:36:37.500 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:37.497+0000 7f4176a3a740 1 -- 172.21.15.83:0/1948078540 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x556085b4c0e0 con 0x556085b3d030 2024-08-19T01:36:37.501 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:37.498+0000 7f4156a00640 1 -- 172.21.15.83:0/1948078540 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4158003b70 con 0x556085b3d030 2024-08-19T01:36:37.501 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:37.498+0000 7f4156a00640 1 -- 172.21.15.83:0/1948078540 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f4158015070 con 0x556085b3d030 2024-08-19T01:36:37.501 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:37.499+0000 7f4176a3a740 10 client.15405 fetch_fsmap finished waiting for FSMap version 25 2024-08-19T01:36:37.502 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:37.499+0000 7f4176a3a740 10 client.15405 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-08-19T01:36:37.502 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:37.499+0000 7f4176a3a740 20 client.15405 populate_metadata read hostname 'smithi083' 2024-08-19T01:36:37.502 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:37.499+0000 7f4176a3a740 1 --2- 172.21.15.83:0/1948078540 >> [v2:172.21.15.117:6838/1967595704,v1:172.21.15.117:6839/1967595704] conn(0x556085b4dba0 0x556085b4ffe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:36:37.502 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:37.500+0000 7f4176a3a740 4 client.15405 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-19T01:36:37.502 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:37.500+0000 7f4176a3a740 1 -- 172.21.15.83:0/1948078540 --> [v2:172.21.15.117:6838/1967595704,v1:172.21.15.117:6839/1967595704] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556085aa8fc0 con 0x556085b4dba0 2024-08-19T01:36:37.502 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:37.500+0000 7f4171200640 1 --2- 172.21.15.83:0/1948078540 >> [v2:172.21.15.117:6838/1967595704,v1:172.21.15.117:6839/1967595704] conn(0x556085b4dba0 0x556085b4ffe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:36:37.502 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:37.501+0000 7f4171200640 1 --2- 172.21.15.83:0/1948078540 >> [v2:172.21.15.117:6838/1967595704,v1:172.21.15.117:6839/1967595704] conn(0x556085b4dba0 0x556085b4ffe0 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7c5a2f049822ee3f server_cookie=d4285d38fb628796 in_seq=0 out_seq=0 2024-08-19T01:36:37.503 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:37.501+0000 7f4156a00640 10 client.15405 ms_handle_connect on v2:172.21.15.117:6838/1967595704 2024-08-19T01:36:37.506 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:37.504+0000 7f4156a00640 1 -- 172.21.15.83:0/1948078540 <== mds.0 v2:172.21.15.117:6838/1967595704 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x556085aa8fc0 con 0x556085b4dba0 2024-08-19T01:36:37.506 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:37.504+0000 7f4156a00640 10 client.15405 handle_command_reply: tid=0 2024-08-19T01:36:37.506 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:37.504+0000 7f4176a3a740 1 client.15405 shutdown 2024-08-19T01:36:37.512 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:37.509+0000 7f4176a3a740 1 -- 172.21.15.83:0/1948078540 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556085b3d030 msgr2=0x556085b48f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:36:37.512 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:37.509+0000 7f4176a3a740 1 --2- 172.21.15.83:0/1948078540 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556085b3d030 0x556085b48f60 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f4158018040 tx=0x7f4158004250 comp rx=0 tx=0).stop 2024-08-19T01:36:37.514 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:37.513+0000 7f4176a3a740 1 -- 172.21.15.83:0/1948078540 shutdown_connections 2024-08-19T01:36:37.514 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:37.513+0000 7f4176a3a740 1 --2- 172.21.15.83:0/1948078540 >> [v2:172.21.15.117:6838/1967595704,v1:172.21.15.117:6839/1967595704] conn(0x556085b4dba0 0x556085b4ffe0 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:37.514 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:37.513+0000 7f4176a3a740 1 --2- 172.21.15.83:0/1948078540 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556085b3d030 0x556085b48f60 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:37.515 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:37.513+0000 7f4176a3a740 1 --2- 172.21.15.83:0/1948078540 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556085b37fa0 0x556085b41f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:37.515 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:37.513+0000 7f4176a3a740 1 --2- 172.21.15.83:0/1948078540 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556085b37660 0x556085b41a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:37.515 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:37.513+0000 7f4176a3a740 1 -- 172.21.15.83:0/1948078540 >> 172.21.15.83:0/1948078540 conn(0x556085b32cd0 msgr2=0x556085b344e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:36:37.535 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:37.516+0000 7f4156a00640 0 client.15405 ms_handle_reset on v2:172.21.15.117:6838/1967595704 2024-08-19T01:36:37.535 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:37.517+0000 7f4176a3a740 1 -- 172.21.15.83:0/1948078540 shutdown_connections 2024-08-19T01:36:37.535 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:37.520+0000 7f4176a3a740 1 -- 172.21.15.83:0/1948078540 wait complete. 2024-08-19T01:36:37.535 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:37.520+0000 7f4176a3a740 20 client.15405 trim_cache size 0 max 16384 2024-08-19T01:36:37.611 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:36:37 smithi083 ceph-mon[17120]: pgmap v392: 97 pgs: 97 active+clean; 54 MiB data, 584 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 2.2 MiB/s wr, 674 op/s 2024-08-19T01:36:37.737 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:37.736+0000 7f4176a3a740 1 Processor -- start 2024-08-19T01:36:37.745 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:37.743+0000 7f4176a3a740 1 -- start start 2024-08-19T01:36:37.745 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:37.743+0000 7f4176a3a740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556085b37660 0x556085b330b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:36:37.745 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:37.743+0000 7f4176a3a740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556085b37fa0 0x556085b48f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:36:37.745 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:37.743+0000 7f4176a3a740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556085b3d030 0x556085b335f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:36:37.745 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:37.743+0000 7f4176a3a740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f416c1b1b40 con 0x556085b3d030 2024-08-19T01:36:37.745 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:37.743+0000 7f4176a3a740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f416c1af410 con 0x556085b37660 2024-08-19T01:36:37.745 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:37.743+0000 7f4176a3a740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f416c1ad200 con 0x556085b37fa0 2024-08-19T01:36:37.746 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:37.744+0000 7f4171200640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556085b37660 0x556085b330b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:36:37.746 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:37.744+0000 7f4171200640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556085b37660 0x556085b330b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.83:56184/0 (socket says 172.21.15.83:56184) 2024-08-19T01:36:37.746 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:37.744+0000 7f4171200640 1 -- 172.21.15.83:0/931270222 learned_addr learned my addr 172.21.15.83:0/931270222 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-08-19T01:36:37.746 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:37.744+0000 7f416be00640 1 --2- 172.21.15.83:0/931270222 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556085b37fa0 0x556085b48f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:36:37.749 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:37.748+0000 7f4171c00640 1 --2- 172.21.15.83:0/931270222 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556085b3d030 0x556085b335f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:36:37.750 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:37.748+0000 7f4171c00640 1 -- 172.21.15.83:0/931270222 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556085b37fa0 msgr2=0x556085b48f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:36:37.750 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:37.748+0000 7f4171c00640 1 --2- 172.21.15.83:0/931270222 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556085b37fa0 0x556085b48f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:37.750 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:37.748+0000 7f4171c00640 1 -- 172.21.15.83:0/931270222 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556085b37660 msgr2=0x556085b330b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:36:37.750 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:37.748+0000 7f4171c00640 1 --2- 172.21.15.83:0/931270222 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556085b37660 0x556085b330b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:37.750 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:37.748+0000 7f4171c00640 1 -- 172.21.15.83:0/931270222 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556085b32af0 con 0x556085b3d030 2024-08-19T01:36:37.751 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:37.749+0000 7f4171c00640 1 --2- 172.21.15.83:0/931270222 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556085b3d030 0x556085b335f0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f415802c040 tx=0x7f415800a700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ebb910fa5732dbd7 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:36:37.751 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:37.750+0000 7f4161600640 1 -- 172.21.15.83:0/931270222 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4158036020 con 0x556085b3d030 2024-08-19T01:36:37.751 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:37.750+0000 7f4161600640 1 -- 172.21.15.83:0/931270222 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4158003e60 con 0x556085b3d030 2024-08-19T01:36:37.751 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:37.750+0000 7f4161600640 1 -- 172.21.15.83:0/931270222 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4158005bd0 con 0x556085b3d030 2024-08-19T01:36:37.752 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:37.751+0000 7f4176a3a740 1 -- 172.21.15.83:0/931270222 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556085b3d030 msgr2=0x556085b335f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:36:37.752 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:37.751+0000 7f4176a3a740 1 --2- 172.21.15.83:0/931270222 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556085b3d030 0x556085b335f0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f415802c040 tx=0x7f415800a700 comp rx=0 tx=0).stop 2024-08-19T01:36:37.772 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:37.771+0000 7f4176a3a740 1 -- 172.21.15.83:0/931270222 shutdown_connections 2024-08-19T01:36:37.772 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:37.771+0000 7f4176a3a740 1 --2- 172.21.15.83:0/931270222 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556085b3d030 0x556085b335f0 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:37.772 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:37.771+0000 7f4176a3a740 1 --2- 172.21.15.83:0/931270222 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556085b37fa0 0x556085b48f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:37.772 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:37.771+0000 7f4176a3a740 1 --2- 172.21.15.83:0/931270222 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556085b37660 0x556085b330b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:37.772 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:37.771+0000 7f4176a3a740 1 -- 172.21.15.83:0/931270222 >> 172.21.15.83:0/931270222 conn(0x556085b32cd0 msgr2=0x556085b41a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:36:37.787 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:37.786+0000 7f4176a3a740 1 -- 172.21.15.83:0/931270222 shutdown_connections 2024-08-19T01:36:37.787 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:37.786+0000 7f4176a3a740 1 -- 172.21.15.83:0/931270222 wait complete. 2024-08-19T01:36:37.788 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:37.786+0000 7f4176a3a740 1 -- start start 2024-08-19T01:36:37.788 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:37.786+0000 7f4176a3a740 1 Processor -- start 2024-08-19T01:36:37.792 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:37.790+0000 7f4176a3a740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556085b37660 0x556085b45240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:36:37.792 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:37.790+0000 7f4176a3a740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556085b37fa0 0x556085b45780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:36:37.792 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:37.790+0000 7f4176a3a740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556085b3d030 0x556085b48670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:36:37.792 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:37.790+0000 7f4176a3a740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f416c1ad8d0 con 0x556085b37fa0 2024-08-19T01:36:37.792 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:37.790+0000 7f4176a3a740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f416c1aed40 con 0x556085b37660 2024-08-19T01:36:37.792 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:37.790+0000 7f4176a3a740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f416c1adfa0 con 0x556085b3d030 2024-08-19T01:36:37.792 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:37.790+0000 7f4176a3a740 10 client.0 _ll_register_callbacks cb 0x7f4158013490 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-19T01:36:37.796 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:37.791+0000 7f416be00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556085b37fa0 0x556085b45780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:36:37.797 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:37.791+0000 7f416be00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556085b37fa0 0x556085b45780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:46422/0 (socket says 172.21.15.83:46422) 2024-08-19T01:36:37.797 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:37.791+0000 7f4171200640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556085b37660 0x556085b45240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:36:37.797 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:37.791+0000 7f416be00640 1 -- 172.21.15.83:0/1360945176 learned_addr learned my addr 172.21.15.83:0/1360945176 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-08-19T01:36:37.797 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:37.791+0000 7f4171200640 1 -- 172.21.15.83:0/1360945176 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556085b3d030 msgr2=0x556085b48670 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-08-19T01:36:37.797 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:37.791+0000 7f4171200640 1 --2- 172.21.15.83:0/1360945176 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556085b3d030 0x556085b48670 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:37.797 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:37.791+0000 7f4171200640 1 -- 172.21.15.83:0/1360945176 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556085b37fa0 msgr2=0x556085b45780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:36:37.797 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:37.791+0000 7f4171200640 1 --2- 172.21.15.83:0/1360945176 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556085b37fa0 0x556085b45780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:37.797 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:37.791+0000 7f4171200640 1 -- 172.21.15.83:0/1360945176 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556085b48c80 con 0x556085b37660 2024-08-19T01:36:37.797 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:37.792+0000 7f4171200640 1 --2- 172.21.15.83:0/1360945176 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556085b37660 0x556085b45240 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x556085b40420 tx=0x7f416400e040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8b4dda2f9b29baaa server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:36:37.804 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:37.798+0000 7f4156a00640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-08-19T01:36:37.804 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:37.798+0000 7f4156a00640 1 -- 172.21.15.83:0/1360945176 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f416402f060 con 0x556085b37660 2024-08-19T01:36:37.804 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:37.798+0000 7f4156a00640 1 -- 172.21.15.83:0/1360945176 <== mon.2 v2:172.21.15.117:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8269+0+0 (secure 0 0 0) 0x7f41640227e0 con 0x556085b37660 2024-08-19T01:36:37.811 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:37.804+0000 7f4176a3a740 1 -- 172.21.15.83:0/1360945176 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x556085aa3310 con 0x556085b37660 2024-08-19T01:36:37.814 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:37.813+0000 7f4156a00640 1 -- 172.21.15.83:0/1360945176 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f41640229e0 con 0x556085b37660 2024-08-19T01:36:37.817 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:37.816+0000 7f4176a3a740 10 client.25484 fetch_fsmap learned FSMap version 25 2024-08-19T01:36:37.818 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:37.816+0000 7f4176a3a740 1 -- 172.21.15.83:0/1360945176 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x556085bb7430 con 0x556085b37660 2024-08-19T01:36:37.821 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:37.820+0000 7f4156a00640 1 -- 172.21.15.83:0/1360945176 <== mon.2 v2:172.21.15.117:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x556085aa3310 con 0x556085b37660 2024-08-19T01:36:37.821 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:37.820+0000 7f4156a00640 1 -- 172.21.15.83:0/1360945176 <== mon.2 v2:172.21.15.117:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f416402a400 con 0x556085b37660 2024-08-19T01:36:37.821 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:37.820+0000 7f4176a3a740 10 client.25484 fetch_fsmap finished waiting for FSMap version 25 2024-08-19T01:36:37.821 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:37.820+0000 7f4176a3a740 10 client.25484 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-08-19T01:36:37.822 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:37.820+0000 7f4176a3a740 20 client.25484 populate_metadata read hostname 'smithi083' 2024-08-19T01:36:37.822 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:37.820+0000 7f4176a3a740 1 --2- 172.21.15.83:0/1360945176 >> [v2:172.21.15.117:6838/1967595704,v1:172.21.15.117:6839/1967595704] conn(0x556085bb7d70 0x556085bba190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:36:37.822 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:37.820+0000 7f4176a3a740 4 client.25484 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-08-19T01:36:37.822 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:37.820+0000 7f4176a3a740 1 -- 172.21.15.83:0/1360945176 --> [v2:172.21.15.117:6838/1967595704,v1:172.21.15.117:6839/1967595704] -- command(tid 0: {"prefix": "flush journal"}) -- 0x556085a207c0 con 0x556085bb7d70 2024-08-19T01:36:37.830 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:37.828+0000 7f416be00640 1 --2- 172.21.15.83:0/1360945176 >> [v2:172.21.15.117:6838/1967595704,v1:172.21.15.117:6839/1967595704] conn(0x556085bb7d70 0x556085bba190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:36:37.856 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:37.845+0000 7f416be00640 1 --2- 172.21.15.83:0/1360945176 >> [v2:172.21.15.117:6838/1967595704,v1:172.21.15.117:6839/1967595704] conn(0x556085bb7d70 0x556085bba190 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cc8d2f275ff43a3a server_cookie=7c7d67ad284f8898 in_seq=0 out_seq=0 2024-08-19T01:36:37.856 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:37.847+0000 7f4156a00640 10 client.25484 ms_handle_connect on v2:172.21.15.117:6838/1967595704 2024-08-19T01:36:39.905 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:36:39 smithi083 ceph-mon[17120]: pgmap v393: 97 pgs: 97 active+clean; 62 MiB data, 598 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 2.2 MiB/s wr, 566 op/s 2024-08-19T01:36:39.952 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:36:39 smithi117 ceph-mon[22016]: pgmap v393: 97 pgs: 97 active+clean; 62 MiB data, 598 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 2.2 MiB/s wr, 566 op/s 2024-08-19T01:36:39.965 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:36:39 smithi167 ceph-mon[21875]: pgmap v393: 97 pgs: 97 active+clean; 62 MiB data, 598 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 2.2 MiB/s wr, 566 op/s 2024-08-19T01:36:40.538 INFO:tasks.workunit.client.0.smithi083.stdout: 36 555746 555 371548 558 303397 1470 2024-08-19T01:36:40.904 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:36:40 smithi083 ceph-mon[17120]: scrub summary: idle 2024-08-19T01:36:40.952 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:36:40 smithi117 ceph-mon[22016]: scrub summary: idle 2024-08-19T01:36:40.966 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:36:40 smithi167 ceph-mon[21875]: scrub summary: idle 2024-08-19T01:36:41.717 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:36:41 smithi083 ceph-mon[17120]: pgmap v394: 97 pgs: 97 active+clean; 65 MiB data, 609 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.0 MiB/s wr, 241 op/s 2024-08-19T01:36:41.717 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:36:41 smithi083 ceph-mon[17120]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-08-19T01:36:41.952 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:36:41 smithi117 ceph-mon[22016]: pgmap v394: 97 pgs: 97 active+clean; 65 MiB data, 609 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.0 MiB/s wr, 241 op/s 2024-08-19T01:36:41.952 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:36:41 smithi117 ceph-mon[22016]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-08-19T01:36:41.966 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:36:41 smithi167 ceph-mon[21875]: pgmap v394: 97 pgs: 97 active+clean; 65 MiB data, 609 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.0 MiB/s wr, 241 op/s 2024-08-19T01:36:41.966 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:36:41 smithi167 ceph-mon[21875]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-08-19T01:36:42.580 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:42.577+0000 7f4156a00640 1 -- 172.21.15.83:0/1360945176 <== mds.0 v2:172.21.15.117:6838/1967595704 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x556085a207c0 con 0x556085bb7d70 2024-08-19T01:36:42.580 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:42.577+0000 7f4156a00640 10 client.25484 handle_command_reply: tid=0 2024-08-19T01:36:42.580 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:42.577+0000 7f4176a3a740 1 client.25484 shutdown 2024-08-19T01:36:42.580 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:42.578+0000 7f4176a3a740 1 -- 172.21.15.83:0/1360945176 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556085b37660 msgr2=0x556085b45240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:36:42.580 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:42.578+0000 7f4176a3a740 1 --2- 172.21.15.83:0/1360945176 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556085b37660 0x556085b45240 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x556085b40420 tx=0x7f416400e040 comp rx=0 tx=0).stop 2024-08-19T01:36:42.581 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:42.578+0000 7f4176a3a740 1 -- 172.21.15.83:0/1360945176 shutdown_connections 2024-08-19T01:36:42.581 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:42.578+0000 7f4176a3a740 1 --2- 172.21.15.83:0/1360945176 >> [v2:172.21.15.117:6838/1967595704,v1:172.21.15.117:6839/1967595704] conn(0x556085bb7d70 0x556085bba190 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:42.581 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:42.578+0000 7f4176a3a740 1 --2- 172.21.15.83:0/1360945176 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556085b3d030 0x556085b48670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:42.581 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:42.578+0000 7f4176a3a740 1 --2- 172.21.15.83:0/1360945176 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556085b37fa0 0x556085b45780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:42.584 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:42.578+0000 7f4176a3a740 1 --2- 172.21.15.83:0/1360945176 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556085b37660 0x556085b45240 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:42.584 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:42.578+0000 7f4176a3a740 1 -- 172.21.15.83:0/1360945176 >> 172.21.15.83:0/1360945176 conn(0x556085b32cd0 msgr2=0x556085b31720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:36:42.584 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:42.578+0000 7f4156a00640 0 client.25484 ms_handle_reset on v2:172.21.15.117:6838/1967595704 2024-08-19T01:36:42.584 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:42.580+0000 7f4176a3a740 1 -- 172.21.15.83:0/1360945176 shutdown_connections 2024-08-19T01:36:42.584 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:42.580+0000 7f4176a3a740 1 -- 172.21.15.83:0/1360945176 wait complete. 2024-08-19T01:36:42.584 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:42.580+0000 7f4176a3a740 20 client.25484 trim_cache size 0 max 16384 2024-08-19T01:36:42.585 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stdout:{ 2024-08-19T01:36:42.585 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stdout: "message": "", 2024-08-19T01:36:42.585 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stdout: "return_code": 0 2024-08-19T01:36:42.585 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stdout:} 2024-08-19T01:36:42.599 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:42.594+0000 7f4172600640 1 -- 172.21.15.83:0/3360651124 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f41500a2c80 msgr2=0x7f41500a5140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:36:42.599 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:42.594+0000 7f4172600640 1 --2- 172.21.15.83:0/3360651124 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f41500a2c80 0x7f41500a5140 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f416400abe0 tx=0x7f4164009230 comp rx=0 tx=0).stop 2024-08-19T01:36:42.599 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:42.594+0000 7f4172600640 1 -- 172.21.15.83:0/3360651124 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f416c0733b0 msgr2=0x7f416c19e9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:36:42.599 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:42.594+0000 7f4172600640 1 --2- 172.21.15.83:0/3360651124 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f416c0733b0 0x7f416c19e9e0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f415c004510 tx=0x7f415c004540 comp rx=0 tx=0).stop 2024-08-19T01:36:42.599 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:42.598+0000 7f4172600640 1 -- 172.21.15.83:0/3360651124 shutdown_connections 2024-08-19T01:36:42.600 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:42.598+0000 7f4172600640 1 --2- 172.21.15.83:0/3360651124 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f41500a2c80 0x7f41500a5140 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:42.600 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:42.598+0000 7f4172600640 1 --2- 172.21.15.83:0/3360651124 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f416c10d8e0 0x7f416c1a5730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:42.600 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:42.598+0000 7f4172600640 1 --2- 172.21.15.83:0/3360651124 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f416c0733b0 0x7f416c19e9e0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:42.600 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:42.598+0000 7f4172600640 1 --2- 172.21.15.83:0/3360651124 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f416c0729e0 0x7f416c19e420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:42.600 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:42.598+0000 7f4172600640 1 -- 172.21.15.83:0/3360651124 >> 172.21.15.83:0/3360651124 conn(0x7f416c06e880 msgr2=0x7f416c070660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:36:42.602 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:42.601+0000 7f4172600640 1 -- 172.21.15.83:0/3360651124 shutdown_connections 2024-08-19T01:36:42.602 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:42.601+0000 7f4172600640 1 -- 172.21.15.83:0/3360651124 wait complete. 2024-08-19T01:36:43.681 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:36:43 smithi083 ceph-mon[17120]: pgmap v395: 97 pgs: 97 active+clean; 66 MiB data, 613 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.1 MiB/s wr, 323 op/s 2024-08-19T01:36:43.954 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:36:43 smithi117 ceph-mon[22016]: pgmap v395: 97 pgs: 97 active+clean; 66 MiB data, 613 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.1 MiB/s wr, 323 op/s 2024-08-19T01:36:43.966 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:36:43 smithi167 ceph-mon[21875]: pgmap v395: 97 pgs: 97 active+clean; 66 MiB data, 613 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.1 MiB/s wr, 323 op/s 2024-08-19T01:36:45.651 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:36:45 smithi083 ceph-mon[17120]: pgmap v396: 97 pgs: 97 active+clean; 83 MiB data, 675 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 4.2 MiB/s wr, 1.03k op/s 2024-08-19T01:36:45.952 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:36:45 smithi117 ceph-mon[22016]: pgmap v396: 97 pgs: 97 active+clean; 83 MiB data, 675 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 4.2 MiB/s wr, 1.03k op/s 2024-08-19T01:36:45.965 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:36:45 smithi167 ceph-mon[21875]: pgmap v396: 97 pgs: 97 active+clean; 83 MiB data, 675 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 4.2 MiB/s wr, 1.03k op/s 2024-08-19T01:36:47.814 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:36:47 smithi083 ceph-mon[17120]: pgmap v397: 97 pgs: 97 active+clean; 84 MiB data, 680 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 4.0 MiB/s wr, 1.08k op/s 2024-08-19T01:36:47.952 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:36:47 smithi117 ceph-mon[22016]: pgmap v397: 97 pgs: 97 active+clean; 84 MiB data, 680 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 4.0 MiB/s wr, 1.08k op/s 2024-08-19T01:36:47.965 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:36:47 smithi167 ceph-mon[21875]: pgmap v397: 97 pgs: 97 active+clean; 84 MiB data, 680 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 4.0 MiB/s wr, 1.08k op/s 2024-08-19T01:36:49.727 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:36:49 smithi083 ceph-mon[17120]: pgmap v398: 97 pgs: 97 active+clean; 87 MiB data, 678 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 6.0 MiB/s wr, 1.23k op/s 2024-08-19T01:36:49.952 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:36:49 smithi117 ceph-mon[22016]: pgmap v398: 97 pgs: 97 active+clean; 87 MiB data, 678 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 6.0 MiB/s wr, 1.23k op/s 2024-08-19T01:36:49.965 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:36:49 smithi167 ceph-mon[21875]: pgmap v398: 97 pgs: 97 active+clean; 87 MiB data, 678 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 6.0 MiB/s wr, 1.23k op/s 2024-08-19T01:36:50.538 INFO:tasks.workunit.client.0.smithi083.stdout: 38 277766 146 188402 106 153805 327 2024-08-19T01:36:51.670 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:36:51 smithi083 ceph-mon[17120]: pgmap v399: 97 pgs: 97 active+clean; 104 MiB data, 747 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 6.9 MiB/s wr, 1.32k op/s 2024-08-19T01:36:51.952 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:36:51 smithi117 ceph-mon[22016]: pgmap v399: 97 pgs: 97 active+clean; 104 MiB data, 747 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 6.9 MiB/s wr, 1.32k op/s 2024-08-19T01:36:51.966 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:36:51 smithi167 ceph-mon[21875]: pgmap v399: 97 pgs: 97 active+clean; 104 MiB data, 747 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 6.9 MiB/s wr, 1.32k op/s 2024-08-19T01:36:53.609 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:36:53 smithi083 ceph-mon[17120]: pgmap v400: 97 pgs: 97 active+clean; 111 MiB data, 780 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 7.2 MiB/s wr, 1.35k op/s 2024-08-19T01:36:53.952 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:36:53 smithi117 ceph-mon[22016]: pgmap v400: 97 pgs: 97 active+clean; 111 MiB data, 780 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 7.2 MiB/s wr, 1.35k op/s 2024-08-19T01:36:53.966 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:36:53 smithi167 ceph-mon[21875]: pgmap v400: 97 pgs: 97 active+clean; 111 MiB data, 780 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 7.2 MiB/s wr, 1.35k op/s 2024-08-19T01:36:55.577 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:36:55 smithi083 ceph-mon[17120]: pgmap v401: 97 pgs: 97 active+clean; 127 MiB data, 844 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 8.7 MiB/s wr, 1.35k op/s 2024-08-19T01:36:55.952 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:36:55 smithi117 ceph-mon[22016]: pgmap v401: 97 pgs: 97 active+clean; 127 MiB data, 844 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 8.7 MiB/s wr, 1.35k op/s 2024-08-19T01:36:55.965 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:36:55 smithi167 ceph-mon[21875]: pgmap v401: 97 pgs: 97 active+clean; 127 MiB data, 844 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 8.7 MiB/s wr, 1.35k op/s 2024-08-19T01:36:56.128 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:56.125+0000 7fd327600640 1 -- 172.21.15.83:0/625718846 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd320102300 msgr2=0x7fd3201046f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:36:56.128 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:56.125+0000 7fd327600640 1 --2- 172.21.15.83:0/625718846 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd320102300 0x7fd3201046f0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fd30c009900 tx=0x7fd30c02a3f0 comp rx=0 tx=0).stop 2024-08-19T01:36:56.128 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:56.126+0000 7fd327600640 1 -- 172.21.15.83:0/625718846 shutdown_connections 2024-08-19T01:36:56.128 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:56.126+0000 7fd327600640 1 --2- 172.21.15.83:0/625718846 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd320107560 0x7fd3201099d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:56.128 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:56.126+0000 7fd327600640 1 --2- 172.21.15.83:0/625718846 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd320104c30 0x7fd320107020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:56.128 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:56.126+0000 7fd327600640 1 --2- 172.21.15.83:0/625718846 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd320102300 0x7fd3201046f0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:56.128 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:56.126+0000 7fd327600640 1 -- 172.21.15.83:0/625718846 >> 172.21.15.83:0/625718846 conn(0x7fd3200fbee0 msgr2=0x7fd3200fe320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:36:56.129 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:56.127+0000 7fd327600640 1 -- 172.21.15.83:0/625718846 shutdown_connections 2024-08-19T01:36:56.129 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:56.128+0000 7fd327600640 1 -- 172.21.15.83:0/625718846 wait complete. 2024-08-19T01:36:56.131 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:56.130+0000 7fd327600640 1 Processor -- start 2024-08-19T01:36:56.132 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:56.131+0000 7fd327600640 1 -- start start 2024-08-19T01:36:56.132 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:56.131+0000 7fd327600640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd320102300 0x7fd3201956e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:36:56.133 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:56.131+0000 7fd327600640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd320104c30 0x7fd320195c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:36:56.133 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:56.131+0000 7fd327600640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd320107560 0x7fd3201a58c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:36:56.133 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:56.131+0000 7fd327600640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fd32010d590 con 0x7fd320102300 2024-08-19T01:36:56.133 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:56.131+0000 7fd327600640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fd32010d410 con 0x7fd320107560 2024-08-19T01:36:56.133 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:56.131+0000 7fd327600640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fd32010d710 con 0x7fd320104c30 2024-08-19T01:36:56.133 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:56.132+0000 7fd31fe00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd320104c30 0x7fd320195c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:36:56.134 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:56.132+0000 7fd31fe00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd320104c30 0x7fd320195c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.83:38758/0 (socket says 172.21.15.83:38758) 2024-08-19T01:36:56.134 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:56.132+0000 7fd31fe00640 1 -- 172.21.15.83:0/3129544608 learned_addr learned my addr 172.21.15.83:0/3129544608 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-08-19T01:36:56.134 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:56.132+0000 7fd325600640 1 --2- 172.21.15.83:0/3129544608 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd320107560 0x7fd3201a58c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:36:56.134 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:56.132+0000 7fd324c00640 1 --2- 172.21.15.83:0/3129544608 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd320102300 0x7fd3201956e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:36:56.135 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:56.133+0000 7fd325600640 1 -- 172.21.15.83:0/3129544608 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd320104c30 msgr2=0x7fd320195c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:36:56.135 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:56.133+0000 7fd325600640 1 --2- 172.21.15.83:0/3129544608 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd320104c30 0x7fd320195c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:56.135 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:56.133+0000 7fd325600640 1 -- 172.21.15.83:0/3129544608 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd320102300 msgr2=0x7fd3201956e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:36:56.135 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:56.133+0000 7fd325600640 1 --2- 172.21.15.83:0/3129544608 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd320102300 0x7fd3201956e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:56.135 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:56.133+0000 7fd325600640 1 -- 172.21.15.83:0/3129544608 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd3201a5e00 con 0x7fd320107560 2024-08-19T01:36:56.135 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:56.133+0000 7fd31fe00640 1 --2- 172.21.15.83:0/3129544608 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd320104c30 0x7fd320195c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-19T01:36:56.135 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:56.133+0000 7fd324c00640 1 --2- 172.21.15.83:0/3129544608 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd320102300 0x7fd3201956e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-19T01:36:56.136 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:56.134+0000 7fd325600640 1 --2- 172.21.15.83:0/3129544608 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd320107560 0x7fd3201a58c0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fd318005fd0 tx=0x7fd31800dea0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:36:56.136 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:56.134+0000 7fd31d600640 1 -- 172.21.15.83:0/3129544608 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd31800bd40 con 0x7fd320107560 2024-08-19T01:36:56.136 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:56.134+0000 7fd31d600640 1 -- 172.21.15.83:0/3129544608 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd318003e50 con 0x7fd320107560 2024-08-19T01:36:56.136 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:56.134+0000 7fd31d600640 1 -- 172.21.15.83:0/3129544608 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3180106c0 con 0x7fd320107560 2024-08-19T01:36:56.136 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:56.134+0000 7fd327600640 1 -- 172.21.15.83:0/3129544608 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd3201a5fd0 con 0x7fd320107560 2024-08-19T01:36:56.136 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:56.134+0000 7fd327600640 1 -- 172.21.15.83:0/3129544608 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd3201a6440 con 0x7fd320107560 2024-08-19T01:36:56.142 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:56.140+0000 7fd32a422740 1 Processor -- start 2024-08-19T01:36:56.143 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:56.141+0000 7fd31d600640 1 -- 172.21.15.83:0/3129544608 <== mon.2 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7fd318003970 con 0x7fd320107560 2024-08-19T01:36:56.143 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:56.142+0000 7fd32a422740 1 -- start start 2024-08-19T01:36:56.144 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:56.142+0000 7fd32a422740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556e488b0470 0x556e488b0870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:36:56.144 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:56.142+0000 7fd32a422740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556e488b0db0 0x556e488b5870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:36:56.144 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:56.142+0000 7fd32a422740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556e488b5e40 0x556e488b8300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:36:56.144 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:56.142+0000 7fd32a422740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x556e48814890 con 0x556e488b5e40 2024-08-19T01:36:56.144 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:56.142+0000 7fd32a422740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x556e487acd90 con 0x556e488b0470 2024-08-19T01:36:56.144 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:56.142+0000 7fd32a422740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x556e48810850 con 0x556e488b0db0 2024-08-19T01:36:56.145 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:56.143+0000 7fd31fe00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556e488b0470 0x556e488b0870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:36:56.145 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:56.143+0000 7fd31fe00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556e488b0470 0x556e488b0870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.83:56752/0 (socket says 172.21.15.83:56752) 2024-08-19T01:36:56.145 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:56.143+0000 7fd31fe00640 1 -- 172.21.15.83:0/961522182 learned_addr learned my addr 172.21.15.83:0/961522182 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-08-19T01:36:56.145 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:56.143+0000 7fd31fe00640 1 --2- 172.21.15.83:0/961522182 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556e488b5e40 0x556e488b8300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:36:56.146 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:56.143+0000 7fd324c00640 1 --2- 172.21.15.83:0/961522182 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556e488b0db0 0x556e488b5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:36:56.146 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:56.143+0000 7fd31fe00640 1 -- 172.21.15.83:0/961522182 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556e488b0db0 msgr2=0x556e488b5870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:36:56.146 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:56.143+0000 7fd31fe00640 1 --2- 172.21.15.83:0/961522182 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556e488b0db0 0x556e488b5870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:56.146 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:56.143+0000 7fd31fe00640 1 -- 172.21.15.83:0/961522182 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556e488b5e40 msgr2=0x556e488b8300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:36:56.146 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:56.143+0000 7fd31fe00640 1 --2- 172.21.15.83:0/961522182 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556e488b5e40 0x556e488b8300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:56.146 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:56.143+0000 7fd31fe00640 1 -- 172.21.15.83:0/961522182 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556e48528210 con 0x556e488b0470 2024-08-19T01:36:56.146 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:56.143+0000 7fd31fe00640 1 --2- 172.21.15.83:0/961522182 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556e488b0470 0x556e488b0870 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fd314011a60 tx=0x7fd31400abe0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f2daed263aa70be0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:36:56.146 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:56.144+0000 7fd311c00640 1 -- 172.21.15.83:0/961522182 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd31400fd90 con 0x556e488b0470 2024-08-19T01:36:56.146 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:56.144+0000 7fd311c00640 1 -- 172.21.15.83:0/961522182 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd3140035f0 con 0x556e488b0470 2024-08-19T01:36:56.146 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:56.144+0000 7fd311c00640 1 -- 172.21.15.83:0/961522182 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd31401b5f0 con 0x556e488b0470 2024-08-19T01:36:56.146 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:56.144+0000 7fd31d600640 1 --2- 172.21.15.83:0/3129544608 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7fd3080a2cb0 0x7fd3080a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:36:56.147 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:56.144+0000 7fd31d600640 1 -- 172.21.15.83:0/3129544608 <== mon.2 v2:172.21.15.117:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8269+0+0 (secure 0 0 0) 0x7fd3180cf3a0 con 0x7fd320107560 2024-08-19T01:36:56.147 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:56.144+0000 7fd324c00640 1 --2- 172.21.15.83:0/3129544608 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7fd3080a2cb0 0x7fd3080a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:36:56.147 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:56.145+0000 7fd32a422740 1 -- 172.21.15.83:0/961522182 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556e488b0470 msgr2=0x556e488b0870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:36:56.147 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:56.145+0000 7fd32a422740 1 --2- 172.21.15.83:0/961522182 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556e488b0470 0x556e488b0870 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fd314011a60 tx=0x7fd31400abe0 comp rx=0 tx=0).stop 2024-08-19T01:36:56.147 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:56.146+0000 7fd32a422740 1 -- 172.21.15.83:0/961522182 shutdown_connections 2024-08-19T01:36:56.148 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:56.146+0000 7fd32a422740 1 --2- 172.21.15.83:0/961522182 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556e488b5e40 0x556e488b8300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:56.148 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:56.146+0000 7fd32a422740 1 --2- 172.21.15.83:0/961522182 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556e488b0db0 0x556e488b5870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:56.148 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:56.146+0000 7fd32a422740 1 --2- 172.21.15.83:0/961522182 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556e488b0470 0x556e488b0870 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:56.148 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:56.146+0000 7fd32a422740 1 -- 172.21.15.83:0/961522182 >> 172.21.15.83:0/961522182 conn(0x556e488abae0 msgr2=0x556e488abee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:36:56.148 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:56.146+0000 7fd32a422740 1 -- 172.21.15.83:0/961522182 shutdown_connections 2024-08-19T01:36:56.148 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:56.146+0000 7fd324c00640 1 --2- 172.21.15.83:0/3129544608 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7fd3080a2cb0 0x7fd3080a5170 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fd30c031440 tx=0x7fd30c0315b0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:36:56.149 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:56.146+0000 7fd32a422740 1 -- 172.21.15.83:0/961522182 wait complete. 2024-08-19T01:36:56.149 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:56.147+0000 7fd32a422740 1 -- start start 2024-08-19T01:36:56.149 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:56.147+0000 7fd32a422740 1 Processor -- start 2024-08-19T01:36:56.151 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:56.148+0000 7fd32a422740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556e488b0470 0x556e488cb7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:36:56.151 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:56.148+0000 7fd32a422740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556e488b0db0 0x556e488cbcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:36:56.151 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:56.148+0000 7fd32a422740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556e488b5e40 0x556e488d2cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:36:56.152 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:56.148+0000 7fd32a422740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x556e488c9ec0 con 0x556e488b0db0 2024-08-19T01:36:56.152 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:56.148+0000 7fd32a422740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x556e487cff70 con 0x556e488b5e40 2024-08-19T01:36:56.152 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:56.148+0000 7fd32a422740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x556e488ca040 con 0x556e488b0470 2024-08-19T01:36:56.152 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:56.148+0000 7fd32a422740 10 client.0 _ll_register_callbacks cb 0x556e4880c8d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-19T01:36:56.152 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:56.148+0000 7fd31fe00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556e488b0db0 0x556e488cbcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:36:56.153 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:56.148+0000 7fd31fe00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556e488b0db0 0x556e488cbcf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:60698/0 (socket says 172.21.15.83:60698) 2024-08-19T01:36:56.153 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:56.148+0000 7fd31fe00640 1 -- 172.21.15.83:0/1227776872 learned_addr learned my addr 172.21.15.83:0/1227776872 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-08-19T01:36:56.153 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:56.148+0000 7fd324c00640 1 --2- 172.21.15.83:0/1227776872 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556e488b0470 0x556e488cb7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:36:56.153 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:56.148+0000 7fd325600640 1 --2- 172.21.15.83:0/1227776872 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556e488b5e40 0x556e488d2cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:36:56.153 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:56.149+0000 7fd31fe00640 1 -- 172.21.15.83:0/1227776872 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556e488b0470 msgr2=0x556e488cb7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:36:56.153 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:56.149+0000 7fd31fe00640 1 --2- 172.21.15.83:0/1227776872 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556e488b0470 0x556e488cb7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:56.153 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:56.149+0000 7fd31fe00640 1 -- 172.21.15.83:0/1227776872 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556e488b5e40 msgr2=0x556e488d2cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:36:56.153 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:56.149+0000 7fd31fe00640 1 --2- 172.21.15.83:0/1227776872 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556e488b5e40 0x556e488d2cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:56.153 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:56.149+0000 7fd31fe00640 1 -- 172.21.15.83:0/1227776872 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556e485f4620 con 0x556e488b0db0 2024-08-19T01:36:56.153 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:56.149+0000 7fd324c00640 1 --2- 172.21.15.83:0/1227776872 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556e488b0470 0x556e488cb7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-19T01:36:56.154 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:56.149+0000 7fd325600640 1 --2- 172.21.15.83:0/1227776872 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556e488b5e40 0x556e488d2cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-19T01:36:56.154 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:56.149+0000 7fd31fe00640 1 --2- 172.21.15.83:0/1227776872 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556e488b0db0 0x556e488cbcf0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7fd31400fdf0 tx=0x7fd31400fe20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fd35a795a648093a server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:36:56.154 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:56.149+0000 7fd306000640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-08-19T01:36:56.154 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:56.149+0000 7fd306000640 1 -- 172.21.15.83:0/1227776872 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd314004430 con 0x556e488b0db0 2024-08-19T01:36:56.154 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:56.150+0000 7fd306000640 1 -- 172.21.15.83:0/1227776872 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8269+0+0 (secure 0 0 0) 0x7fd3140037a0 con 0x556e488b0db0 2024-08-19T01:36:56.156 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:56.150+0000 7fd32a422740 1 -- 172.21.15.83:0/1227776872 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd3140035f0 con 0x556e488b0db0 2024-08-19T01:36:56.156 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:56.150+0000 7fd306000640 1 -- 172.21.15.83:0/1227776872 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fd3140039a0 con 0x556e488b0db0 2024-08-19T01:36:56.156 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:56.151+0000 7fd32a422740 10 client.15423 fetch_fsmap learned FSMap version 26 2024-08-19T01:36:56.156 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:56.151+0000 7fd32a422740 1 -- 172.21.15.83:0/1227776872 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x556e488e6e50 con 0x556e488b0db0 2024-08-19T01:36:56.156 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:56.151+0000 7fd306000640 1 -- 172.21.15.83:0/1227776872 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd3140035f0 con 0x556e488b0db0 2024-08-19T01:36:56.156 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:56.151+0000 7fd306000640 1 -- 172.21.15.83:0/1227776872 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 26) ==== 8350+0+0 (secure 0 0 0) 0x7fd314023070 con 0x556e488b0db0 2024-08-19T01:36:56.156 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:56.151+0000 7fd32a422740 10 client.15423 fetch_fsmap finished waiting for FSMap version 26 2024-08-19T01:36:56.156 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:56.151+0000 7fd32a422740 10 client.15423 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-08-19T01:36:56.157 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:56.151+0000 7fd32a422740 20 client.15423 populate_metadata read hostname 'smithi083' 2024-08-19T01:36:56.157 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:56.151+0000 7fd32a422740 1 --2- 172.21.15.83:0/1227776872 >> [v2:172.21.15.117:6838/1967595704,v1:172.21.15.117:6839/1967595704] conn(0x556e488e83f0 0x556e488ea830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:36:56.157 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:56.152+0000 7fd32a422740 4 client.15423 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-19T01:36:56.157 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:56.152+0000 7fd32a422740 1 -- 172.21.15.83:0/1227776872 --> [v2:172.21.15.117:6838/1967595704,v1:172.21.15.117:6839/1967595704] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556e48821fc0 con 0x556e488e83f0 2024-08-19T01:36:56.157 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:56.152+0000 7fd324c00640 1 --2- 172.21.15.83:0/1227776872 >> [v2:172.21.15.117:6838/1967595704,v1:172.21.15.117:6839/1967595704] conn(0x556e488e83f0 0x556e488ea830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:36:56.157 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:56.153+0000 7fd306000640 10 client.15423 ms_handle_connect on v2:172.21.15.117:6838/1967595704 2024-08-19T01:36:56.157 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:56.154+0000 7fd324c00640 1 --2- 172.21.15.83:0/1227776872 >> [v2:172.21.15.117:6838/1967595704,v1:172.21.15.117:6839/1967595704] conn(0x556e488e83f0 0x556e488ea830 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e6db671a81321c19 server_cookie=44d7d558589100a6 in_seq=0 out_seq=0 2024-08-19T01:36:56.159 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:56.156+0000 7fd306000640 1 -- 172.21.15.83:0/1227776872 <== mds.0 v2:172.21.15.117:6838/1967595704 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x556e48821fc0 con 0x556e488e83f0 2024-08-19T01:36:56.159 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:56.156+0000 7fd306000640 10 client.15423 handle_command_reply: tid=0 2024-08-19T01:36:56.159 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:56.156+0000 7fd32a422740 1 client.15423 shutdown 2024-08-19T01:36:56.159 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:56.157+0000 7fd32a422740 1 -- 172.21.15.83:0/1227776872 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556e488b0db0 msgr2=0x556e488cbcf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:36:56.159 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:56.157+0000 7fd32a422740 1 --2- 172.21.15.83:0/1227776872 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556e488b0db0 0x556e488cbcf0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7fd31400fdf0 tx=0x7fd31400fe20 comp rx=0 tx=0).stop 2024-08-19T01:36:56.159 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:56.157+0000 7fd32a422740 1 -- 172.21.15.83:0/1227776872 shutdown_connections 2024-08-19T01:36:56.159 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:56.157+0000 7fd32a422740 1 --2- 172.21.15.83:0/1227776872 >> [v2:172.21.15.117:6838/1967595704,v1:172.21.15.117:6839/1967595704] conn(0x556e488e83f0 0x556e488ea830 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:56.159 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:56.157+0000 7fd32a422740 1 --2- 172.21.15.83:0/1227776872 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556e488b5e40 0x556e488d2cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:56.160 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:56.157+0000 7fd32a422740 1 --2- 172.21.15.83:0/1227776872 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556e488b0db0 0x556e488cbcf0 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:56.160 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:56.157+0000 7fd32a422740 1 --2- 172.21.15.83:0/1227776872 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556e488b0470 0x556e488cb7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:56.160 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:56.157+0000 7fd32a422740 1 -- 172.21.15.83:0/1227776872 >> 172.21.15.83:0/1227776872 conn(0x556e488abae0 msgr2=0x556e488ad2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:36:56.160 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:56.157+0000 7fd306000640 0 client.15423 ms_handle_reset on v2:172.21.15.117:6838/1967595704 2024-08-19T01:36:56.160 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:56.157+0000 7fd32a422740 1 -- 172.21.15.83:0/1227776872 shutdown_connections 2024-08-19T01:36:56.160 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:56.157+0000 7fd32a422740 1 -- 172.21.15.83:0/1227776872 wait complete. 2024-08-19T01:36:56.160 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:56.157+0000 7fd32a422740 20 client.15423 trim_cache size 0 max 16384 2024-08-19T01:36:56.185 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:56.184+0000 7fd32a422740 1 Processor -- start 2024-08-19T01:36:56.186 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:56.184+0000 7fd32a422740 1 -- start start 2024-08-19T01:36:56.186 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:56.184+0000 7fd32a422740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556e488b0470 0x556e488abec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:36:56.186 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:56.184+0000 7fd32a422740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556e488b5e40 0x556e488d2c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:36:56.186 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:56.184+0000 7fd32a422740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556e488b2710 0x556e488ac400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:36:56.186 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:56.184+0000 7fd32a422740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fd320199e20 con 0x556e488b5e40 2024-08-19T01:36:56.186 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:56.184+0000 7fd32a422740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fd320197730 con 0x556e488b2710 2024-08-19T01:36:56.186 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:56.184+0000 7fd32a422740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fd320072630 con 0x556e488b0470 2024-08-19T01:36:56.187 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:56.185+0000 7fd31fe00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556e488b5e40 0x556e488d2c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:36:56.187 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:56.185+0000 7fd31fe00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556e488b5e40 0x556e488d2c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:60704/0 (socket says 172.21.15.83:60704) 2024-08-19T01:36:56.187 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:56.185+0000 7fd324c00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556e488b0470 0x556e488abec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:36:56.187 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:56.185+0000 7fd31fe00640 1 -- 172.21.15.83:0/3044139983 learned_addr learned my addr 172.21.15.83:0/3044139983 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-08-19T01:36:56.187 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:56.185+0000 7fd325600640 1 --2- 172.21.15.83:0/3044139983 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556e488b2710 0x556e488ac400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:36:56.188 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:56.185+0000 7fd324c00640 1 -- 172.21.15.83:0/3044139983 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556e488b2710 msgr2=0x556e488ac400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:36:56.188 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:56.185+0000 7fd324c00640 1 --2- 172.21.15.83:0/3044139983 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556e488b2710 0x556e488ac400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:56.188 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:56.185+0000 7fd324c00640 1 -- 172.21.15.83:0/3044139983 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556e488b5e40 msgr2=0x556e488d2c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:36:56.188 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:56.185+0000 7fd324c00640 1 --2- 172.21.15.83:0/3044139983 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556e488b5e40 0x556e488d2c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:56.188 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:56.185+0000 7fd324c00640 1 -- 172.21.15.83:0/3044139983 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556e488ab900 con 0x556e488b0470 2024-08-19T01:36:56.188 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:56.185+0000 7fd324c00640 1 --2- 172.21.15.83:0/3044139983 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556e488b0470 0x556e488abec0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fd30c02a910 tx=0x7fd30c03f000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e862c6ddb857259 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:36:56.188 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:56.186+0000 7fd311200640 1 -- 172.21.15.83:0/3044139983 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd30c035040 con 0x556e488b0470 2024-08-19T01:36:56.189 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:56.186+0000 7fd311200640 1 -- 172.21.15.83:0/3044139983 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd30c05a070 con 0x556e488b0470 2024-08-19T01:36:56.189 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:56.186+0000 7fd311200640 1 -- 172.21.15.83:0/3044139983 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd30c003b80 con 0x556e488b0470 2024-08-19T01:36:56.189 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:56.187+0000 7fd32a422740 1 -- 172.21.15.83:0/3044139983 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556e488b0470 msgr2=0x556e488abec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:36:56.189 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:56.187+0000 7fd32a422740 1 --2- 172.21.15.83:0/3044139983 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556e488b0470 0x556e488abec0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fd30c02a910 tx=0x7fd30c03f000 comp rx=0 tx=0).stop 2024-08-19T01:36:56.189 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:56.187+0000 7fd32a422740 1 -- 172.21.15.83:0/3044139983 shutdown_connections 2024-08-19T01:36:56.189 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:56.187+0000 7fd32a422740 1 --2- 172.21.15.83:0/3044139983 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556e488b2710 0x556e488ac400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:56.189 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:56.187+0000 7fd32a422740 1 --2- 172.21.15.83:0/3044139983 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556e488b5e40 0x556e488d2c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:56.189 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:56.187+0000 7fd32a422740 1 --2- 172.21.15.83:0/3044139983 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556e488b0470 0x556e488abec0 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:56.190 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:56.187+0000 7fd32a422740 1 -- 172.21.15.83:0/3044139983 >> 172.21.15.83:0/3044139983 conn(0x556e488abae0 msgr2=0x556e488b2000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:36:56.190 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:56.187+0000 7fd32a422740 1 -- 172.21.15.83:0/3044139983 shutdown_connections 2024-08-19T01:36:56.190 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:56.187+0000 7fd32a422740 1 -- 172.21.15.83:0/3044139983 wait complete. 2024-08-19T01:36:56.190 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:56.188+0000 7fd32a422740 1 -- start start 2024-08-19T01:36:56.190 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:56.188+0000 7fd32a422740 1 Processor -- start 2024-08-19T01:36:56.192 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:56.188+0000 7fd32a422740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556e488b0470 0x556e488cf850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:36:56.192 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:56.188+0000 7fd32a422740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556e488b2710 0x556e488cfd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:36:56.193 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:56.188+0000 7fd32a422740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556e488b5e40 0x556e489510c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:36:56.193 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:56.188+0000 7fd32a422740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fd320072ca0 con 0x556e488b0470 2024-08-19T01:36:56.193 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:56.188+0000 7fd32a422740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fd320197060 con 0x556e488b2710 2024-08-19T01:36:56.193 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:56.188+0000 7fd32a422740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fd320073370 con 0x556e488b5e40 2024-08-19T01:36:56.193 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:56.188+0000 7fd32a422740 10 client.0 _ll_register_callbacks cb 0x7fd314011930 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-19T01:36:56.193 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:56.189+0000 7fd324c00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556e488b0470 0x556e488cf850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:36:56.193 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:56.189+0000 7fd31fe00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556e488b2710 0x556e488cfd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:36:56.193 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:56.189+0000 7fd324c00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556e488b0470 0x556e488cf850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:60720/0 (socket says 172.21.15.83:60720) 2024-08-19T01:36:56.193 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:56.189+0000 7fd324c00640 1 -- 172.21.15.83:0/1227958685 learned_addr learned my addr 172.21.15.83:0/1227958685 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-08-19T01:36:56.194 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:56.189+0000 7fd31fe00640 1 -- 172.21.15.83:0/1227958685 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556e488b5e40 msgr2=0x556e489510c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:36:56.194 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:56.189+0000 7fd31fe00640 1 --2- 172.21.15.83:0/1227958685 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556e488b5e40 0x556e489510c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:56.194 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:56.189+0000 7fd31fe00640 1 -- 172.21.15.83:0/1227958685 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556e488b0470 msgr2=0x556e488cf850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:36:56.194 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:56.189+0000 7fd31fe00640 1 --2- 172.21.15.83:0/1227958685 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556e488b0470 0x556e488cf850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:56.194 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:56.189+0000 7fd31fe00640 1 -- 172.21.15.83:0/1227958685 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556e488d2e80 con 0x556e488b2710 2024-08-19T01:36:56.194 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:56.189+0000 7fd324c00640 1 --2- 172.21.15.83:0/1227958685 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556e488b0470 0x556e488cf850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-19T01:36:56.194 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:56.189+0000 7fd31fe00640 1 --2- 172.21.15.83:0/1227958685 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556e488b2710 0x556e488cfd90 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fd3140305a0 tx=0x7fd314003920 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d806a47dd81b7480 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:36:56.194 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:56.189+0000 7fd306000640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-08-19T01:36:56.194 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:56.190+0000 7fd306000640 1 -- 172.21.15.83:0/1227958685 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd314038020 con 0x556e488b2710 2024-08-19T01:36:56.194 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:56.190+0000 7fd306000640 1 -- 172.21.15.83:0/1227958685 <== mon.2 v2:172.21.15.117:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8269+0+0 (secure 0 0 0) 0x7fd314004430 con 0x556e488b2710 2024-08-19T01:36:56.194 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:56.190+0000 7fd32a422740 1 -- 172.21.15.83:0/1227958685 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd30c05a070 con 0x556e488b2710 2024-08-19T01:36:56.195 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:56.190+0000 7fd306000640 1 -- 172.21.15.83:0/1227958685 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fd31400fe90 con 0x556e488b2710 2024-08-19T01:36:56.195 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:56.190+0000 7fd32a422740 10 client.25502 fetch_fsmap learned FSMap version 26 2024-08-19T01:36:56.195 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:56.190+0000 7fd32a422740 1 -- 172.21.15.83:0/1227958685 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x556e489525e0 con 0x556e488b2710 2024-08-19T01:36:56.195 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:56.190+0000 7fd306000640 1 -- 172.21.15.83:0/1227958685 <== mon.2 v2:172.21.15.117:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd30c05a070 con 0x556e488b2710 2024-08-19T01:36:56.196 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:56.191+0000 7fd306000640 1 -- 172.21.15.83:0/1227958685 <== mon.2 v2:172.21.15.117:3300/0 5 ==== fsmap(e 26) ==== 8350+0+0 (secure 0 0 0) 0x7fd314023040 con 0x556e488b2710 2024-08-19T01:36:56.196 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:56.191+0000 7fd32a422740 10 client.25502 fetch_fsmap finished waiting for FSMap version 26 2024-08-19T01:36:56.196 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:56.191+0000 7fd32a422740 10 client.25502 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-08-19T01:36:56.196 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:56.191+0000 7fd32a422740 20 client.25502 populate_metadata read hostname 'smithi083' 2024-08-19T01:36:56.196 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:56.191+0000 7fd32a422740 1 --2- 172.21.15.83:0/1227958685 >> [v2:172.21.15.117:6838/1967595704,v1:172.21.15.117:6839/1967595704] conn(0x556e48952f70 0x556e48955390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:36:56.196 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:56.191+0000 7fd32a422740 4 client.25502 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-08-19T01:36:56.196 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:56.191+0000 7fd32a422740 1 -- 172.21.15.83:0/1227958685 --> [v2:172.21.15.117:6838/1967595704,v1:172.21.15.117:6839/1967595704] -- command(tid 0: {"prefix": "flush journal"}) -- 0x556e4881c310 con 0x556e48952f70 2024-08-19T01:36:56.196 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:56.192+0000 7fd324c00640 1 --2- 172.21.15.83:0/1227958685 >> [v2:172.21.15.117:6838/1967595704,v1:172.21.15.117:6839/1967595704] conn(0x556e48952f70 0x556e48955390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:36:56.196 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:56.192+0000 7fd324c00640 1 --2- 172.21.15.83:0/1227958685 >> [v2:172.21.15.117:6838/1967595704,v1:172.21.15.117:6839/1967595704] conn(0x556e48952f70 0x556e48955390 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fe6df021cdaca6f2 server_cookie=2205145d6ee42013 in_seq=0 out_seq=0 2024-08-19T01:36:56.196 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:56.192+0000 7fd306000640 10 client.25502 ms_handle_connect on v2:172.21.15.117:6838/1967595704 2024-08-19T01:36:56.650 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833 shell --fsid bad8780e-5dc9-11ef-bccf-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-08-19T01:36:56.867 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:36:56 smithi083 ceph-mon[17120]: pgmap v402: 97 pgs: 97 active+clean; 126 MiB data, 840 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 6.6 MiB/s wr, 645 op/s 2024-08-19T01:36:56.952 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:36:56 smithi117 ceph-mon[22016]: pgmap v402: 97 pgs: 97 active+clean; 126 MiB data, 840 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 6.6 MiB/s wr, 645 op/s 2024-08-19T01:36:56.965 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:36:56 smithi167 ceph-mon[21875]: pgmap v402: 97 pgs: 97 active+clean; 126 MiB data, 840 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 6.6 MiB/s wr, 645 op/s 2024-08-19T01:36:58.653 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:36:58 smithi083 ceph-mon[17120]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-19T01:36:58.702 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:36:58 smithi117 ceph-mon[22016]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-19T01:36:58.715 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:36:58 smithi167 ceph-mon[21875]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-19T01:36:59.321 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:59.319+0000 7fd306000640 1 -- 172.21.15.83:0/1227958685 <== mds.0 v2:172.21.15.117:6838/1967595704 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x556e4881c310 con 0x556e48952f70 2024-08-19T01:36:59.321 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:59.319+0000 7fd306000640 10 client.25502 handle_command_reply: tid=0 2024-08-19T01:36:59.328 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:59.327+0000 7fd32a422740 1 client.25502 shutdown 2024-08-19T01:36:59.352 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:59.349+0000 7fd32a422740 1 -- 172.21.15.83:0/1227958685 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556e488b2710 msgr2=0x556e488cfd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:36:59.352 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:59.349+0000 7fd32a422740 1 --2- 172.21.15.83:0/1227958685 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556e488b2710 0x556e488cfd90 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fd3140305a0 tx=0x7fd314003920 comp rx=0 tx=0).stop 2024-08-19T01:36:59.352 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:59.349+0000 7fd32a422740 1 -- 172.21.15.83:0/1227958685 shutdown_connections 2024-08-19T01:36:59.352 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:59.349+0000 7fd32a422740 1 --2- 172.21.15.83:0/1227958685 >> [v2:172.21.15.117:6838/1967595704,v1:172.21.15.117:6839/1967595704] conn(0x556e48952f70 0x556e48955390 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:59.352 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:59.350+0000 7fd32a422740 1 --2- 172.21.15.83:0/1227958685 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556e488b5e40 0x556e489510c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:59.353 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:59.350+0000 7fd32a422740 1 --2- 172.21.15.83:0/1227958685 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556e488b2710 0x556e488cfd90 secure :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fd3140305a0 tx=0x7fd314003920 comp rx=0 tx=0).stop 2024-08-19T01:36:59.353 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:59.350+0000 7fd32a422740 1 --2- 172.21.15.83:0/1227958685 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556e488b0470 0x556e488cf850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:59.353 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:59.350+0000 7fd32a422740 1 -- 172.21.15.83:0/1227958685 >> 172.21.15.83:0/1227958685 conn(0x556e488abae0 msgr2=0x556e488b5240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:36:59.353 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:59.350+0000 7fd306000640 0 client.25502 ms_handle_reset on v2:172.21.15.117:6838/1967595704 2024-08-19T01:36:59.367 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:59.365+0000 7fd32a422740 1 -- 172.21.15.83:0/1227958685 shutdown_connections 2024-08-19T01:36:59.367 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:59.366+0000 7fd32a422740 1 -- 172.21.15.83:0/1227958685 wait complete. 2024-08-19T01:36:59.405 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:59.402+0000 7fd32a422740 20 client.25502 trim_cache size 0 max 16384 2024-08-19T01:36:59.441 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stdout:{ 2024-08-19T01:36:59.442 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stdout: "message": "", 2024-08-19T01:36:59.442 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stdout: "return_code": 0 2024-08-19T01:36:59.442 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stdout:} 2024-08-19T01:36:59.445 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:59.443+0000 7fd327600640 1 -- 172.21.15.83:0/3129544608 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7fd3080a2cb0 msgr2=0x7fd3080a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:36:59.445 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:59.443+0000 7fd327600640 1 --2- 172.21.15.83:0/3129544608 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7fd3080a2cb0 0x7fd3080a5170 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fd30c031440 tx=0x7fd30c0315b0 comp rx=0 tx=0).stop 2024-08-19T01:36:59.447 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:59.444+0000 7fd327600640 1 -- 172.21.15.83:0/3129544608 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd320107560 msgr2=0x7fd3201a58c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:36:59.447 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:59.444+0000 7fd327600640 1 --2- 172.21.15.83:0/3129544608 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd320107560 0x7fd3201a58c0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fd318005fd0 tx=0x7fd31800dea0 comp rx=0 tx=0).stop 2024-08-19T01:36:59.447 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:59.444+0000 7fd327600640 1 -- 172.21.15.83:0/3129544608 shutdown_connections 2024-08-19T01:36:59.447 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:59.444+0000 7fd327600640 1 --2- 172.21.15.83:0/3129544608 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7fd3080a2cb0 0x7fd3080a5170 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:59.447 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:59.444+0000 7fd327600640 1 --2- 172.21.15.83:0/3129544608 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd320107560 0x7fd3201a58c0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:59.447 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:59.444+0000 7fd327600640 1 --2- 172.21.15.83:0/3129544608 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd320104c30 0x7fd320195c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:59.447 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:59.444+0000 7fd327600640 1 --2- 172.21.15.83:0/3129544608 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd320102300 0x7fd3201956e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:59.447 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:59.444+0000 7fd327600640 1 -- 172.21.15.83:0/3129544608 >> 172.21.15.83:0/3129544608 conn(0x7fd3200fbee0 msgr2=0x7fd320101800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:36:59.447 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:59.445+0000 7fd327600640 1 -- 172.21.15.83:0/3129544608 shutdown_connections 2024-08-19T01:36:59.447 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:36:59.445+0000 7fd327600640 1 -- 172.21.15.83:0/3129544608 wait complete. 2024-08-19T01:36:59.659 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:36:59 smithi083 ceph-mon[17120]: pgmap v403: 97 pgs: 97 active+clean; 128 MiB data, 828 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 6.5 MiB/s wr, 620 op/s 2024-08-19T01:36:59.659 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:36:59 smithi083 ceph-mon[17120]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' 2024-08-19T01:36:59.660 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:36:59 smithi083 ceph-mon[17120]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' 2024-08-19T01:36:59.660 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:36:59 smithi083 ceph-mon[17120]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' 2024-08-19T01:36:59.660 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:36:59 smithi083 ceph-mon[17120]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' 2024-08-19T01:36:59.702 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:36:59 smithi117 ceph-mon[22016]: pgmap v403: 97 pgs: 97 active+clean; 128 MiB data, 828 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 6.5 MiB/s wr, 620 op/s 2024-08-19T01:36:59.702 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:36:59 smithi117 ceph-mon[22016]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' 2024-08-19T01:36:59.702 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:36:59 smithi117 ceph-mon[22016]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' 2024-08-19T01:36:59.702 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:36:59 smithi117 ceph-mon[22016]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' 2024-08-19T01:36:59.702 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:36:59 smithi117 ceph-mon[22016]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' 2024-08-19T01:36:59.716 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:36:59 smithi167 ceph-mon[21875]: pgmap v403: 97 pgs: 97 active+clean; 128 MiB data, 828 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 6.5 MiB/s wr, 620 op/s 2024-08-19T01:36:59.716 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:36:59 smithi167 ceph-mon[21875]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' 2024-08-19T01:36:59.716 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:36:59 smithi167 ceph-mon[21875]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' 2024-08-19T01:36:59.716 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:36:59 smithi167 ceph-mon[21875]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' 2024-08-19T01:36:59.716 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:36:59 smithi167 ceph-mon[21875]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' 2024-08-19T01:37:00.536 INFO:tasks.workunit.client.0.smithi083.stdout: 44 403223 336 271605 453 220653 1032 2024-08-19T01:37:01.091 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/bad8780e-5dc9-11ef-bccf-c7b262605968/mon.a/config 2024-08-19T01:37:01.860 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:37:01 smithi083 ceph-mon[17120]: pgmap v404: 97 pgs: 97 active+clean; 124 MiB data, 837 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 4.5 MiB/s wr, 584 op/s 2024-08-19T01:37:01.952 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:37:01 smithi117 ceph-mon[22016]: pgmap v404: 97 pgs: 97 active+clean; 124 MiB data, 837 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 4.5 MiB/s wr, 584 op/s 2024-08-19T01:37:01.966 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:37:01 smithi167 ceph-mon[21875]: pgmap v404: 97 pgs: 97 active+clean; 124 MiB data, 837 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 4.5 MiB/s wr, 584 op/s 2024-08-19T01:37:02.747 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:02.745+0000 7fba05e00640 1 -- 172.21.15.83:0/2863957888 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fba00101700 msgr2=0x7fba00101b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:37:02.748 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:02.745+0000 7fba05e00640 1 --2- 172.21.15.83:0/2863957888 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fba00101700 0x7fba00101b60 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fb9e8009930 tx=0x7fb9e802a440 comp rx=0 tx=0).stop 2024-08-19T01:37:02.748 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:02.745+0000 7fba05e00640 1 -- 172.21.15.83:0/2863957888 shutdown_connections 2024-08-19T01:37:02.748 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:02.745+0000 7fba05e00640 1 --2- 172.21.15.83:0/2863957888 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fba001020a0 0x7fba0010e5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:37:02.748 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:02.745+0000 7fba05e00640 1 --2- 172.21.15.83:0/2863957888 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fba00101700 0x7fba00101b60 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:37:02.748 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:02.745+0000 7fba05e00640 1 --2- 172.21.15.83:0/2863957888 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fba00107700 0x7fba00107ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:37:02.748 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:02.745+0000 7fba05e00640 1 -- 172.21.15.83:0/2863957888 >> 172.21.15.83:0/2863957888 conn(0x7fba000fd440 msgr2=0x7fba000ff860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:37:02.748 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:02.745+0000 7fba05e00640 1 -- 172.21.15.83:0/2863957888 shutdown_connections 2024-08-19T01:37:02.753 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:02.745+0000 7fba05e00640 1 -- 172.21.15.83:0/2863957888 wait complete. 2024-08-19T01:37:02.753 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:02.747+0000 7fba05e00640 1 Processor -- start 2024-08-19T01:37:02.753 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:02.747+0000 7fba05e00640 1 -- start start 2024-08-19T01:37:02.753 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:02.748+0000 7fba05e00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fba00101700 0x7fba000735f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:37:02.753 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:02.748+0000 7fba05e00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fba001020a0 0x7fba00073b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:37:02.753 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:02.748+0000 7fba05e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fba00107700 0x7fba0006c780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:37:02.753 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:02.751+0000 7fba05400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fba00107700 0x7fba0006c780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:37:02.753 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:02.751+0000 7fba05400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fba00107700 0x7fba0006c780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:53268/0 (socket says 172.21.15.83:53268) 2024-08-19T01:37:02.754 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:02.751+0000 7fba04a00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fba00101700 0x7fba000735f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:37:02.754 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:02.751+0000 7fba04a00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fba00101700 0x7fba000735f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.83:54906/0 (socket says 172.21.15.83:54906) 2024-08-19T01:37:02.754 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:02.751+0000 7fb9ffe00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fba001020a0 0x7fba00073b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:37:02.754 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:02.751+0000 7fba05400640 1 -- 172.21.15.83:0/3110481723 learned_addr learned my addr 172.21.15.83:0/3110481723 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-08-19T01:37:02.754 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:02.751+0000 7fba05e00640 1 -- 172.21.15.83:0/3110481723 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fba00111b50 con 0x7fba00107700 2024-08-19T01:37:02.754 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:02.751+0000 7fba05e00640 1 -- 172.21.15.83:0/3110481723 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fba001119d0 con 0x7fba00101700 2024-08-19T01:37:02.754 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:02.751+0000 7fba05e00640 1 -- 172.21.15.83:0/3110481723 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fba00111cd0 con 0x7fba001020a0 2024-08-19T01:37:02.755 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:02.752+0000 7fba04a00640 1 -- 172.21.15.83:0/3110481723 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fba001020a0 msgr2=0x7fba00073b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:37:02.755 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:02.752+0000 7fba04a00640 1 --2- 172.21.15.83:0/3110481723 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fba001020a0 0x7fba00073b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:37:02.755 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:02.752+0000 7fba04a00640 1 -- 172.21.15.83:0/3110481723 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fba00107700 msgr2=0x7fba0006c780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:37:02.755 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:02.752+0000 7fba04a00640 1 --2- 172.21.15.83:0/3110481723 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fba00107700 0x7fba0006c780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:37:02.755 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:02.752+0000 7fba04a00640 1 -- 172.21.15.83:0/3110481723 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fba0006d040 con 0x7fba00101700 2024-08-19T01:37:02.755 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:02.753+0000 7fb9ffe00640 1 --2- 172.21.15.83:0/3110481723 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fba001020a0 0x7fba00073b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-19T01:37:02.773 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:02.767+0000 7fba04a00640 1 --2- 172.21.15.83:0/3110481723 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fba00101700 0x7fba000735f0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fb9f800cce0 tx=0x7fb9f800a620 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:37:02.773 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:02.770+0000 7fb9fd600640 1 -- 172.21.15.83:0/3110481723 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9f800f450 con 0x7fba00101700 2024-08-19T01:37:02.773 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:02.771+0000 7fb9fd600640 1 -- 172.21.15.83:0/3110481723 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb9f800fad0 con 0x7fba00101700 2024-08-19T01:37:02.773 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:02.771+0000 7fb9fd600640 1 -- 172.21.15.83:0/3110481723 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9f8018c10 con 0x7fba00101700 2024-08-19T01:37:02.777 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:02.771+0000 7fba05e00640 1 -- 172.21.15.83:0/3110481723 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fba0006d330 con 0x7fba00101700 2024-08-19T01:37:02.777 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:02.771+0000 7fba05e00640 1 -- 172.21.15.83:0/3110481723 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fba000712f0 con 0x7fba00101700 2024-08-19T01:37:02.777 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:02.776+0000 7fba09fbb740 1 Processor -- start 2024-08-19T01:37:02.782 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:02.776+0000 7fba09fbb740 1 -- start start 2024-08-19T01:37:02.784 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:02.777+0000 7fb9fd600640 1 -- 172.21.15.83:0/3110481723 <== mon.2 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7fb9f8018400 con 0x7fba00101700 2024-08-19T01:37:02.784 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:02.778+0000 7fba09fbb740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560446b52590 0x560446b52970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:37:02.785 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:02.778+0000 7fba09fbb740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560446b52eb0 0x560446b5d230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:37:02.785 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:02.778+0000 7fba09fbb740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560446b5d770 0x560446b5fb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:37:02.785 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:02.778+0000 7fba09fbb740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x560446aafbf0 con 0x560446b52590 2024-08-19T01:37:02.785 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:02.778+0000 7fba09fbb740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x560446a47bd0 con 0x560446b52eb0 2024-08-19T01:37:02.785 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:02.778+0000 7fba09fbb740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x560446aabbb0 con 0x560446b5d770 2024-08-19T01:37:02.785 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:02.779+0000 7fb9ffe00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560446b5d770 0x560446b5fb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:37:02.785 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:02.779+0000 7fb9ffe00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560446b5d770 0x560446b5fb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.83:55148/0 (socket says 172.21.15.83:55148) 2024-08-19T01:37:02.785 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:02.779+0000 7fb9ffe00640 1 -- 172.21.15.83:0/961747829 learned_addr learned my addr 172.21.15.83:0/961747829 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-08-19T01:37:02.785 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:02.779+0000 7fba04a00640 1 --2- 172.21.15.83:0/961747829 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560446b52eb0 0x560446b5d230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:37:02.785 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:02.779+0000 7fb9ffe00640 1 -- 172.21.15.83:0/961747829 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560446b52eb0 msgr2=0x560446b5d230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:37:02.785 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:02.779+0000 7fb9ffe00640 1 --2- 172.21.15.83:0/961747829 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560446b52eb0 0x560446b5d230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:37:02.785 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:02.779+0000 7fb9ffe00640 1 -- 172.21.15.83:0/961747829 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560446b52590 msgr2=0x560446b52970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:37:02.785 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:02.780+0000 7fb9ffe00640 1 --2- 172.21.15.83:0/961747829 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560446b52590 0x560446b52970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:37:02.786 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:02.780+0000 7fb9ffe00640 1 -- 172.21.15.83:0/961747829 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560446a4e8a0 con 0x560446b5d770 2024-08-19T01:37:02.786 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:02.780+0000 7fba04a00640 1 --2- 172.21.15.83:0/961747829 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560446b52eb0 0x560446b5d230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-19T01:37:02.786 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:02.781+0000 7fb9ffe00640 1 --2- 172.21.15.83:0/961747829 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560446b5d770 0x560446b5fb60 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fb9e802aa30 tx=0x7fb9e8031a90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3dbc1b3f1a52217b server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:37:02.786 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:02.781+0000 7fb9f5c00640 1 -- 172.21.15.83:0/961747829 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9e8040070 con 0x560446b5d770 2024-08-19T01:37:02.786 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:02.782+0000 7fb9f5c00640 1 -- 172.21.15.83:0/961747829 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb9e8031770 con 0x560446b5d770 2024-08-19T01:37:02.797 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:02.783+0000 7fba09fbb740 1 -- 172.21.15.83:0/961747829 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560446b5d770 msgr2=0x560446b5fb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:37:02.797 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:02.783+0000 7fba09fbb740 1 --2- 172.21.15.83:0/961747829 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560446b5d770 0x560446b5fb60 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fb9e802aa30 tx=0x7fb9e8031a90 comp rx=0 tx=0).stop 2024-08-19T01:37:02.797 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:02.786+0000 7fba09fbb740 1 -- 172.21.15.83:0/961747829 shutdown_connections 2024-08-19T01:37:02.797 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:02.786+0000 7fba09fbb740 1 --2- 172.21.15.83:0/961747829 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560446b5d770 0x560446b5fb60 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:37:02.797 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:02.786+0000 7fba09fbb740 1 --2- 172.21.15.83:0/961747829 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560446b52eb0 0x560446b5d230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:37:02.797 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:02.786+0000 7fba09fbb740 1 --2- 172.21.15.83:0/961747829 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560446b52590 0x560446b52970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:37:02.797 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:02.786+0000 7fb9fd600640 1 --2- 172.21.15.83:0/3110481723 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7fb9e40a2c40 0x7fb9e40a5100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:37:02.797 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:02.786+0000 7fba09fbb740 1 -- 172.21.15.83:0/961747829 >> 172.21.15.83:0/961747829 conn(0x560446b4eb20 msgr2=0x560446b4ef00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:37:02.797 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:02.786+0000 7fba05400640 1 --2- 172.21.15.83:0/3110481723 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7fb9e40a2c40 0x7fb9e40a5100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:37:02.797 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:02.787+0000 7fb9fd600640 1 -- 172.21.15.83:0/3110481723 <== mon.2 v2:172.21.15.117:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8269+0+0 (secure 0 0 0) 0x7fb9f80cf5c0 con 0x7fba00101700 2024-08-19T01:37:02.797 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:02.788+0000 7fba09fbb740 1 -- 172.21.15.83:0/961747829 shutdown_connections 2024-08-19T01:37:02.798 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:02.788+0000 7fba05400640 1 --2- 172.21.15.83:0/3110481723 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7fb9e40a2c40 0x7fb9e40a5100 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fb9f0005bc0 tx=0x7fb9f0009530 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:37:02.798 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:02.788+0000 7fba09fbb740 1 -- 172.21.15.83:0/961747829 wait complete. 2024-08-19T01:37:02.798 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:02.789+0000 7fba09fbb740 1 -- start start 2024-08-19T01:37:02.798 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:02.790+0000 7fba09fbb740 1 Processor -- start 2024-08-19T01:37:02.798 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:02.791+0000 7fba09fbb740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560446b52590 0x560446b5f990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:37:02.798 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:02.791+0000 7fba09fbb740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560446b52eb0 0x560446b5cc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:37:02.798 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:02.791+0000 7fba09fbb740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560446b5d180 0x560446b5d5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:37:02.798 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:02.792+0000 7fb9ffe00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560446b52eb0 0x560446b5cc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:37:02.798 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:02.792+0000 7fb9ffe00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560446b52eb0 0x560446b5cc40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.83:54922/0 (socket says 172.21.15.83:54922) 2024-08-19T01:37:02.798 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:02.792+0000 7fba04a00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560446b52590 0x560446b5f990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:37:02.798 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:02.792+0000 7fba04a00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560446b52590 0x560446b5f990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.83:55160/0 (socket says 172.21.15.83:55160) 2024-08-19T01:37:02.798 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:02.792+0000 7fb9ffe00640 1 -- 172.21.15.83:0/3057785037 learned_addr learned my addr 172.21.15.83:0/3057785037 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-08-19T01:37:02.799 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:02.792+0000 7fba09fbb740 1 -- 172.21.15.83:0/3057785037 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x560446b603e0 con 0x560446b5d180 2024-08-19T01:37:02.799 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:02.792+0000 7fba09fbb740 1 -- 172.21.15.83:0/3057785037 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x560446a6ad40 con 0x560446b52eb0 2024-08-19T01:37:02.799 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:02.792+0000 7fba09fbb740 1 -- 172.21.15.83:0/3057785037 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x560446b60560 con 0x560446b52590 2024-08-19T01:37:02.799 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:02.793+0000 7fba05400640 1 --2- 172.21.15.83:0/3057785037 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560446b5d180 0x560446b5d5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:37:02.799 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:02.793+0000 7fb9ffe00640 1 -- 172.21.15.83:0/3057785037 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560446b52590 msgr2=0x560446b5f990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:37:02.800 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:02.793+0000 7fba09fbb740 10 client.0 _ll_register_callbacks cb 0x560446a33b10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-19T01:37:02.800 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:02.794+0000 7fb9ffe00640 1 --2- 172.21.15.83:0/3057785037 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560446b52590 0x560446b5f990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:37:02.800 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:02.794+0000 7fb9ffe00640 1 -- 172.21.15.83:0/3057785037 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560446b5d180 msgr2=0x560446b5d5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:37:02.800 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:02.794+0000 7fb9ffe00640 1 --2- 172.21.15.83:0/3057785037 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560446b5d180 0x560446b5d5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:37:02.800 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:02.794+0000 7fb9ffe00640 1 -- 172.21.15.83:0/3057785037 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5604468213c0 con 0x560446b52eb0 2024-08-19T01:37:02.800 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:02.794+0000 7fba05400640 1 --2- 172.21.15.83:0/3057785037 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560446b5d180 0x560446b5d5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-19T01:37:02.800 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:02.794+0000 7fba09fbb740 1 -- 172.21.15.83:0/3057785037 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x560446abd160 con 0x560446b52eb0 2024-08-19T01:37:02.800 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:02.796+0000 7fb9ffe00640 1 --2- 172.21.15.83:0/3057785037 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560446b52eb0 0x560446b5cc40 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fb9e8031b20 tx=0x7fb9e80318e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2931abaf5336db23 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:37:02.800 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:02.796+0000 7fb9ee000640 10 client.25508 ms_handle_connect on v2:172.21.15.117:3300/0 2024-08-19T01:37:02.800 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:02.797+0000 7fb9ee000640 1 -- 172.21.15.83:0/3057785037 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9e8040070 con 0x560446b52eb0 2024-08-19T01:37:02.800 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:02.797+0000 7fb9ee000640 1 -- 172.21.15.83:0/3057785037 <== mon.2 v2:172.21.15.117:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8269+0+0 (secure 0 0 0) 0x7fb9e8004450 con 0x560446b52eb0 2024-08-19T01:37:02.800 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:02.797+0000 7fb9ee000640 1 -- 172.21.15.83:0/3057785037 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fb9e80035f0 con 0x560446b52eb0 2024-08-19T01:37:02.800 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:02.798+0000 7fba09fbb740 10 client.25508 fetch_fsmap learned FSMap version 26 2024-08-19T01:37:02.801 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:02.798+0000 7fba09fbb740 1 -- 172.21.15.83:0/3057785037 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({fsmap=26}) -- 0x560446b6c120 con 0x560446b52eb0 2024-08-19T01:37:02.801 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:02.798+0000 7fb9ee000640 1 -- 172.21.15.83:0/3057785037 <== mon.2 v2:172.21.15.117:3300/0 4 ==== fsmap(e 26) ==== 8350+0+0 (secure 0 0 0) 0x7fb9e8003790 con 0x560446b52eb0 2024-08-19T01:37:02.802 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:02.799+0000 7fba09fbb740 10 client.25508 fetch_fsmap finished waiting for FSMap version 26 2024-08-19T01:37:02.802 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:02.799+0000 7fba09fbb740 10 client.25508 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-19T01:37:02.802 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:02.799+0000 7fba09fbb740 20 client.25508 populate_metadata read hostname 'smithi083' 2024-08-19T01:37:02.802 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:02.799+0000 7fba09fbb740 1 --2- 172.21.15.83:0/3057785037 >> [v2:172.21.15.117:6838/1967595704,v1:172.21.15.117:6839/1967595704] conn(0x560446b6d560 0x560446b6f980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:37:02.802 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:02.799+0000 7fba09fbb740 4 client.25508 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-19T01:37:02.802 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:02.799+0000 7fba09fbb740 1 -- 172.21.15.83:0/3057785037 --> [v2:172.21.15.117:6838/1967595704,v1:172.21.15.117:6839/1967595704] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560446ab7670 con 0x560446b6d560 2024-08-19T01:37:02.803 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:02.800+0000 7fba04a00640 1 --2- 172.21.15.83:0/3057785037 >> [v2:172.21.15.117:6838/1967595704,v1:172.21.15.117:6839/1967595704] conn(0x560446b6d560 0x560446b6f980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:37:02.803 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:02.800+0000 7fba04a00640 1 --2- 172.21.15.83:0/3057785037 >> [v2:172.21.15.117:6838/1967595704,v1:172.21.15.117:6839/1967595704] conn(0x560446b6d560 0x560446b6f980 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=641efb5ccb96ad2e server_cookie=68e40c2504242588 in_seq=0 out_seq=0 2024-08-19T01:37:02.803 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:02.800+0000 7fb9ee000640 10 client.25508 ms_handle_connect on v2:172.21.15.117:6838/1967595704 2024-08-19T01:37:02.809 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:02.803+0000 7fb9ee000640 1 -- 172.21.15.83:0/3057785037 <== mds.0 v2:172.21.15.117:6838/1967595704 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x560446ab7670 con 0x560446b6d560 2024-08-19T01:37:02.809 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:02.803+0000 7fb9ee000640 10 client.25508 handle_command_reply: tid=0 2024-08-19T01:37:02.809 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:02.803+0000 7fba09fbb740 1 client.25508 shutdown 2024-08-19T01:37:02.809 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:02.803+0000 7fba09fbb740 1 -- 172.21.15.83:0/3057785037 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560446b52eb0 msgr2=0x560446b5cc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:37:02.809 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:02.803+0000 7fba09fbb740 1 --2- 172.21.15.83:0/3057785037 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560446b52eb0 0x560446b5cc40 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fb9e8031b20 tx=0x7fb9e80318e0 comp rx=0 tx=0).stop 2024-08-19T01:37:02.810 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:02.804+0000 7fba09fbb740 1 -- 172.21.15.83:0/3057785037 shutdown_connections 2024-08-19T01:37:02.810 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:02.804+0000 7fba09fbb740 1 --2- 172.21.15.83:0/3057785037 >> [v2:172.21.15.117:6838/1967595704,v1:172.21.15.117:6839/1967595704] conn(0x560446b6d560 0x560446b6f980 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:37:02.810 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:02.804+0000 7fb9ee000640 0 client.25508 ms_handle_reset on v2:172.21.15.117:6838/1967595704 2024-08-19T01:37:02.810 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:02.804+0000 7fba09fbb740 1 --2- 172.21.15.83:0/3057785037 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560446b5d180 0x560446b5d5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:37:02.810 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:02.804+0000 7fba09fbb740 1 --2- 172.21.15.83:0/3057785037 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560446b52eb0 0x560446b5cc40 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:37:02.810 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:02.804+0000 7fba09fbb740 1 --2- 172.21.15.83:0/3057785037 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560446b52590 0x560446b5f990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:37:02.810 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:02.804+0000 7fba09fbb740 1 -- 172.21.15.83:0/3057785037 >> 172.21.15.83:0/3057785037 conn(0x560446b4eb20 msgr2=0x560446b54850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:37:02.810 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:02.807+0000 7fba09fbb740 1 -- 172.21.15.83:0/3057785037 shutdown_connections 2024-08-19T01:37:02.810 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:02.807+0000 7fba09fbb740 1 -- 172.21.15.83:0/3057785037 wait complete. 2024-08-19T01:37:02.810 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:02.807+0000 7fba09fbb740 20 client.25508 trim_cache size 0 max 16384 2024-08-19T01:37:02.878 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:02.853+0000 7fba09fbb740 1 Processor -- start 2024-08-19T01:37:02.878 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:02.853+0000 7fba09fbb740 1 -- start start 2024-08-19T01:37:02.878 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:02.853+0000 7fba09fbb740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560446b52590 0x560446b572f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:37:02.878 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:02.853+0000 7fba09fbb740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560446b52eb0 0x560446b75250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:37:02.879 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:02.853+0000 7fba09fbb740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560446b5d180 0x560446b75790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:37:02.879 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:02.853+0000 7fba09fbb740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fb9dc010f00 con 0x560446b5d180 2024-08-19T01:37:02.879 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:02.853+0000 7fba09fbb740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fb9dc00e7d0 con 0x560446b52590 2024-08-19T01:37:02.879 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:02.853+0000 7fba09fbb740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fb9dc00c620 con 0x560446b52eb0 2024-08-19T01:37:02.879 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:02.854+0000 7fba05400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560446b5d180 0x560446b75790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:37:02.879 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:02.854+0000 7fba04a00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560446b52590 0x560446b572f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:37:02.879 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:02.854+0000 7fba05400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560446b5d180 0x560446b75790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:53310/0 (socket says 172.21.15.83:53310) 2024-08-19T01:37:02.879 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:02.854+0000 7fba05400640 1 -- 172.21.15.83:0/4286058351 learned_addr learned my addr 172.21.15.83:0/4286058351 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-08-19T01:37:02.879 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:02.855+0000 7fb9ffe00640 1 --2- 172.21.15.83:0/4286058351 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560446b52eb0 0x560446b75250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:37:02.879 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:02.855+0000 7fba05400640 1 -- 172.21.15.83:0/4286058351 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560446b52eb0 msgr2=0x560446b75250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:37:02.879 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:02.855+0000 7fba05400640 1 --2- 172.21.15.83:0/4286058351 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560446b52eb0 0x560446b75250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:37:02.879 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:02.855+0000 7fba05400640 1 -- 172.21.15.83:0/4286058351 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560446b52590 msgr2=0x560446b572f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:37:02.879 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:02.855+0000 7fba05400640 1 --2- 172.21.15.83:0/4286058351 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560446b52590 0x560446b572f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:37:02.880 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:02.855+0000 7fba05400640 1 -- 172.21.15.83:0/4286058351 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560446b60560 con 0x560446b5d180 2024-08-19T01:37:02.880 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:02.856+0000 7fba04a00640 1 --2- 172.21.15.83:0/4286058351 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560446b52590 0x560446b572f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-19T01:37:02.880 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:02.856+0000 7fb9ffe00640 1 --2- 172.21.15.83:0/4286058351 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560446b52eb0 0x560446b75250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-19T01:37:02.880 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:02.857+0000 7fba05400640 1 --2- 172.21.15.83:0/4286058351 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560446b5d180 0x560446b75790 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7fb9f001e5a0 tx=0x7fb9f0014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=93b1f401c14ff19f server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:37:02.880 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:02.857+0000 7fb9f5200640 1 -- 172.21.15.83:0/4286058351 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9f002b070 con 0x560446b5d180 2024-08-19T01:37:02.880 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:02.858+0000 7fb9f5200640 1 -- 172.21.15.83:0/4286058351 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb9f0026ad0 con 0x560446b5d180 2024-08-19T01:37:02.880 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:02.858+0000 7fb9f5200640 1 -- 172.21.15.83:0/4286058351 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9f002b070 con 0x560446b5d180 2024-08-19T01:37:02.880 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:02.863+0000 7fba09fbb740 1 -- 172.21.15.83:0/4286058351 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560446b5d180 msgr2=0x560446b75790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:37:02.880 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:02.863+0000 7fba09fbb740 1 --2- 172.21.15.83:0/4286058351 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560446b5d180 0x560446b75790 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7fb9f001e5a0 tx=0x7fb9f0014000 comp rx=0 tx=0).stop 2024-08-19T01:37:02.880 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:02.863+0000 7fba09fbb740 1 -- 172.21.15.83:0/4286058351 shutdown_connections 2024-08-19T01:37:02.880 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:02.863+0000 7fba09fbb740 1 --2- 172.21.15.83:0/4286058351 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560446b5d180 0x560446b75790 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:37:02.880 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:02.863+0000 7fba09fbb740 1 --2- 172.21.15.83:0/4286058351 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560446b52eb0 0x560446b75250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:37:02.880 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:02.863+0000 7fba09fbb740 1 --2- 172.21.15.83:0/4286058351 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560446b52590 0x560446b572f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:37:02.880 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:02.863+0000 7fba09fbb740 1 -- 172.21.15.83:0/4286058351 >> 172.21.15.83:0/4286058351 conn(0x560446b4eb20 msgr2=0x560446b5b510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:37:02.881 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:02.864+0000 7fba09fbb740 1 -- 172.21.15.83:0/4286058351 shutdown_connections 2024-08-19T01:37:02.881 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:02.864+0000 7fba09fbb740 1 -- 172.21.15.83:0/4286058351 wait complete. 2024-08-19T01:37:02.881 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:02.865+0000 7fba09fbb740 1 -- start start 2024-08-19T01:37:02.881 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:02.866+0000 7fba09fbb740 1 Processor -- start 2024-08-19T01:37:02.881 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:02.867+0000 7fba09fbb740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560446b52590 0x560446b61720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:37:02.881 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:02.867+0000 7fba09fbb740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560446b52eb0 0x560446b61c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:37:02.881 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:02.867+0000 7fba09fbb740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560446b5d180 0x560446b621a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:37:02.881 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:02.867+0000 7fba09fbb740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fb9dc00cc90 con 0x560446b52590 2024-08-19T01:37:02.881 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:02.867+0000 7fba09fbb740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fb9dc00e100 con 0x560446b5d180 2024-08-19T01:37:02.881 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:02.867+0000 7fba09fbb740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fb9dc00d360 con 0x560446b52eb0 2024-08-19T01:37:02.881 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:02.867+0000 7fba09fbb740 10 client.0 _ll_register_callbacks cb 0x7fb9e80319e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-19T01:37:02.881 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:02.869+0000 7fb9ffe00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560446b52eb0 0x560446b61c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:37:02.881 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:02.869+0000 7fba05400640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560446b5d180 0x560446b621a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:37:02.881 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:02.869+0000 7fba05400640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560446b5d180 0x560446b621a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.83:54950/0 (socket says 172.21.15.83:54950) 2024-08-19T01:37:02.882 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:02.869+0000 7fba05400640 1 -- 172.21.15.83:0/353009078 learned_addr learned my addr 172.21.15.83:0/353009078 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-08-19T01:37:02.882 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:02.869+0000 7fba04a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560446b52590 0x560446b61720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:37:02.882 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:02.870+0000 7fba05400640 1 -- 172.21.15.83:0/353009078 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560446b52eb0 msgr2=0x560446b61c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:37:02.882 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:02.870+0000 7fba05400640 1 --2- 172.21.15.83:0/353009078 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560446b52eb0 0x560446b61c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:37:02.882 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:02.870+0000 7fba05400640 1 -- 172.21.15.83:0/353009078 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560446b52590 msgr2=0x560446b61720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:37:02.882 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:02.870+0000 7fba05400640 1 --2- 172.21.15.83:0/353009078 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560446b52590 0x560446b61720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:37:02.882 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:02.870+0000 7fba05400640 1 -- 172.21.15.83:0/353009078 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560446bd7380 con 0x560446b5d180 2024-08-19T01:37:02.882 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:02.870+0000 7fba04a00640 1 --2- 172.21.15.83:0/353009078 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560446b52590 0x560446b61720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-19T01:37:02.882 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:02.871+0000 7fb9ee000640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-08-19T01:37:02.882 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:02.871+0000 7fba05400640 1 --2- 172.21.15.83:0/353009078 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560446b5d180 0x560446b621a0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fb9f0011ca0 tx=0x7fb9f00052f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=df4b297001f1bfe8 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:37:02.882 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:02.871+0000 7fb9ee000640 1 -- 172.21.15.83:0/353009078 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9f002b040 con 0x560446b5d180 2024-08-19T01:37:02.882 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:02.872+0000 7fb9ee000640 1 -- 172.21.15.83:0/353009078 <== mon.2 v2:172.21.15.117:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8269+0+0 (secure 0 0 0) 0x7fb9f001c8a0 con 0x560446b5d180 2024-08-19T01:37:02.882 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:02.872+0000 7fba09fbb740 1 -- 172.21.15.83:0/353009078 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb9f0026ad0 con 0x560446b5d180 2024-08-19T01:37:02.883 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:02.872+0000 7fb9ee000640 1 -- 172.21.15.83:0/353009078 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fb9f001cb20 con 0x560446b5d180 2024-08-19T01:37:02.883 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:02.872+0000 7fba09fbb740 10 client.25514 fetch_fsmap learned FSMap version 26 2024-08-19T01:37:02.883 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:02.872+0000 7fba09fbb740 1 -- 172.21.15.83:0/353009078 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x560446bd8630 con 0x560446b5d180 2024-08-19T01:37:02.883 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:02.876+0000 7fb9ee000640 1 -- 172.21.15.83:0/353009078 <== mon.2 v2:172.21.15.117:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb9f0026ad0 con 0x560446b5d180 2024-08-19T01:37:02.883 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:02.876+0000 7fb9ee000640 1 -- 172.21.15.83:0/353009078 <== mon.2 v2:172.21.15.117:3300/0 5 ==== fsmap(e 26) ==== 8350+0+0 (secure 0 0 0) 0x7fb9f0025030 con 0x560446b5d180 2024-08-19T01:37:02.883 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:02.876+0000 7fba09fbb740 10 client.25514 fetch_fsmap finished waiting for FSMap version 26 2024-08-19T01:37:02.883 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:02.876+0000 7fba09fbb740 10 client.25514 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-19T01:37:02.883 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:02.876+0000 7fba09fbb740 20 client.25514 populate_metadata read hostname 'smithi083' 2024-08-19T01:37:02.883 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:02.876+0000 7fba09fbb740 1 --2- 172.21.15.83:0/353009078 >> [v2:172.21.15.117:6838/1967595704,v1:172.21.15.117:6839/1967595704] conn(0x560446bd8ca0 0x560446bdb0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:37:02.883 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:02.876+0000 7fba09fbb740 4 client.25514 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-08-19T01:37:02.883 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:02.876+0000 7fba09fbb740 1 -- 172.21.15.83:0/353009078 --> [v2:172.21.15.117:6838/1967595704,v1:172.21.15.117:6839/1967595704] -- command(tid 0: {"prefix": "scrub status"}) -- 0x560446a340c0 con 0x560446bd8ca0 2024-08-19T01:37:02.884 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:02.879+0000 7fba04a00640 1 --2- 172.21.15.83:0/353009078 >> [v2:172.21.15.117:6838/1967595704,v1:172.21.15.117:6839/1967595704] conn(0x560446bd8ca0 0x560446bdb0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:37:02.884 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:02.879+0000 7fba04a00640 1 --2- 172.21.15.83:0/353009078 >> [v2:172.21.15.117:6838/1967595704,v1:172.21.15.117:6839/1967595704] conn(0x560446bd8ca0 0x560446bdb0c0 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8bb6cf88ef0d7375 server_cookie=70f6e443fe2e83c6 in_seq=0 out_seq=0 2024-08-19T01:37:02.884 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:02.879+0000 7fb9ee000640 10 client.25514 ms_handle_connect on v2:172.21.15.117:6838/1967595704 2024-08-19T01:37:02.884 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:02.881+0000 7fb9ee000640 1 -- 172.21.15.83:0/353009078 <== mds.0 v2:172.21.15.117:6838/1967595704 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x560446a340c0 con 0x560446bd8ca0 2024-08-19T01:37:02.884 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:02.881+0000 7fb9ee000640 10 client.25514 handle_command_reply: tid=0 2024-08-19T01:37:02.885 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:02.881+0000 7fba09fbb740 1 client.25514 shutdown 2024-08-19T01:37:02.885 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:02.882+0000 7fba09fbb740 1 -- 172.21.15.83:0/353009078 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560446b5d180 msgr2=0x560446b621a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:37:02.885 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:02.882+0000 7fba09fbb740 1 --2- 172.21.15.83:0/353009078 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560446b5d180 0x560446b621a0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fb9f0011ca0 tx=0x7fb9f00052f0 comp rx=0 tx=0).stop 2024-08-19T01:37:02.885 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:02.882+0000 7fba09fbb740 1 -- 172.21.15.83:0/353009078 shutdown_connections 2024-08-19T01:37:02.885 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:02.882+0000 7fba09fbb740 1 --2- 172.21.15.83:0/353009078 >> [v2:172.21.15.117:6838/1967595704,v1:172.21.15.117:6839/1967595704] conn(0x560446bd8ca0 0x560446bdb0c0 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:37:02.885 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:02.882+0000 7fba09fbb740 1 --2- 172.21.15.83:0/353009078 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560446b5d180 0x560446b621a0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:37:02.885 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:02.882+0000 7fba09fbb740 1 --2- 172.21.15.83:0/353009078 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560446b52eb0 0x560446b61c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:37:02.885 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:02.882+0000 7fba09fbb740 1 --2- 172.21.15.83:0/353009078 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560446b52590 0x560446b61720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:37:02.885 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:02.882+0000 7fb9ee000640 0 client.25514 ms_handle_reset on v2:172.21.15.117:6838/1967595704 2024-08-19T01:37:02.885 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:02.882+0000 7fba09fbb740 1 -- 172.21.15.83:0/353009078 >> 172.21.15.83:0/353009078 conn(0x560446b4eb20 msgr2=0x560446b5b430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:37:02.885 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:02.882+0000 7fba09fbb740 1 -- 172.21.15.83:0/353009078 shutdown_connections 2024-08-19T01:37:02.885 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:02.882+0000 7fba09fbb740 1 -- 172.21.15.83:0/353009078 wait complete. 2024-08-19T01:37:02.899 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-08-19T01:37:02.899 INFO:teuthology.orchestra.run.smithi083.stdout: "status": "no active scrubs running", 2024-08-19T01:37:02.899 INFO:teuthology.orchestra.run.smithi083.stdout: "scrubs": {} 2024-08-19T01:37:02.899 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-08-19T01:37:02.900 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:02.883+0000 7fba09fbb740 20 client.25514 trim_cache size 0 max 16384 2024-08-19T01:37:02.900 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:02.884+0000 7fba05e00640 1 -- 172.21.15.83:0/3110481723 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7fb9e40a2c40 msgr2=0x7fb9e40a5100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:37:02.900 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:02.884+0000 7fba05e00640 1 --2- 172.21.15.83:0/3110481723 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7fb9e40a2c40 0x7fb9e40a5100 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fb9f0005bc0 tx=0x7fb9f0009530 comp rx=0 tx=0).stop 2024-08-19T01:37:02.900 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:02.884+0000 7fba05e00640 1 -- 172.21.15.83:0/3110481723 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fba00101700 msgr2=0x7fba000735f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:37:02.900 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:02.884+0000 7fba05e00640 1 --2- 172.21.15.83:0/3110481723 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fba00101700 0x7fba000735f0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fb9f800cce0 tx=0x7fb9f800a620 comp rx=0 tx=0).stop 2024-08-19T01:37:02.900 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:02.884+0000 7fba05e00640 1 -- 172.21.15.83:0/3110481723 shutdown_connections 2024-08-19T01:37:02.900 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:02.884+0000 7fba05e00640 1 --2- 172.21.15.83:0/3110481723 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7fb9e40a2c40 0x7fb9e40a5100 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:37:02.900 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:02.884+0000 7fba05e00640 1 --2- 172.21.15.83:0/3110481723 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fba00107700 0x7fba0006c780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:37:02.900 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:02.884+0000 7fba05e00640 1 --2- 172.21.15.83:0/3110481723 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fba001020a0 0x7fba00073b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:37:02.900 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:02.884+0000 7fba05e00640 1 --2- 172.21.15.83:0/3110481723 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fba00101700 0x7fba000735f0 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:37:02.900 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:02.884+0000 7fba05e00640 1 -- 172.21.15.83:0/3110481723 >> 172.21.15.83:0/3110481723 conn(0x7fba000fd440 msgr2=0x7fba000fda00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:37:02.900 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:02.884+0000 7fba05e00640 1 -- 172.21.15.83:0/3110481723 shutdown_connections 2024-08-19T01:37:02.901 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:02.884+0000 7fba05e00640 1 -- 172.21.15.83:0/3110481723 wait complete. 2024-08-19T01:37:03.762 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:37:03 smithi083 ceph-mon[17120]: pgmap v405: 97 pgs: 97 active+clean; 127 MiB data, 839 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 3.3 MiB/s wr, 516 op/s 2024-08-19T01:37:03.952 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:37:03 smithi117 ceph-mon[22016]: pgmap v405: 97 pgs: 97 active+clean; 127 MiB data, 839 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 3.3 MiB/s wr, 516 op/s 2024-08-19T01:37:03.966 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:37:03 smithi167 ceph-mon[21875]: pgmap v405: 97 pgs: 97 active+clean; 127 MiB data, 839 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 3.3 MiB/s wr, 516 op/s 2024-08-19T01:37:04.108 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-08-19T01:37:04.108 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833 shell --fsid bad8780e-5dc9-11ef-bccf-c7b262605968 -- ceph fs dump --format=json 2024-08-19T01:37:05.501 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:37:05 smithi083 ceph-mon[17120]: pgmap v406: 97 pgs: 97 active+clean; 114 MiB data, 787 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 3.4 MiB/s wr, 606 op/s 2024-08-19T01:37:05.952 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:37:05 smithi117 ceph-mon[22016]: pgmap v406: 97 pgs: 97 active+clean; 114 MiB data, 787 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 3.4 MiB/s wr, 606 op/s 2024-08-19T01:37:05.965 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:37:05 smithi167 ceph-mon[21875]: pgmap v406: 97 pgs: 97 active+clean; 114 MiB data, 787 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 3.4 MiB/s wr, 606 op/s 2024-08-19T01:37:07.787 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:37:07 smithi083 ceph-mon[17120]: pgmap v407: 97 pgs: 97 active+clean; 113 MiB data, 773 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.9 MiB/s wr, 525 op/s 2024-08-19T01:37:07.787 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:37:07 smithi083 ceph-mon[17120]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-19T01:37:07.787 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:37:07 smithi083 ceph-mon[17120]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-19T01:37:07.787 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:37:07 smithi083 ceph-mon[17120]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' 2024-08-19T01:37:07.952 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:37:07 smithi117 ceph-mon[22016]: pgmap v407: 97 pgs: 97 active+clean; 113 MiB data, 773 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.9 MiB/s wr, 525 op/s 2024-08-19T01:37:07.952 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:37:07 smithi117 ceph-mon[22016]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-19T01:37:07.952 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:37:07 smithi117 ceph-mon[22016]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-19T01:37:07.952 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:37:07 smithi117 ceph-mon[22016]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' 2024-08-19T01:37:07.965 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:37:07 smithi167 ceph-mon[21875]: pgmap v407: 97 pgs: 97 active+clean; 113 MiB data, 773 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.9 MiB/s wr, 525 op/s 2024-08-19T01:37:07.966 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:37:07 smithi167 ceph-mon[21875]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-19T01:37:07.966 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:37:07 smithi167 ceph-mon[21875]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-19T01:37:07.966 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:37:07 smithi167 ceph-mon[21875]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' 2024-08-19T01:37:08.231 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/bad8780e-5dc9-11ef-bccf-c7b262605968/mon.a/config 2024-08-19T01:37:09.586 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:09.581+0000 7f63fe800640 1 -- 172.21.15.83:0/1979355898 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f63f8105ac0 msgr2=0x7f63f8105f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:37:09.586 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:09.581+0000 7f63fe800640 1 --2- 172.21.15.83:0/1979355898 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f63f8105ac0 0x7f63f8105f50 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f63f001acb0 tx=0x7f63f003b990 comp rx=0 tx=0).stop 2024-08-19T01:37:09.586 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:09.582+0000 7f63fe800640 1 -- 172.21.15.83:0/1979355898 shutdown_connections 2024-08-19T01:37:09.586 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:09.582+0000 7f63fe800640 1 --2- 172.21.15.83:0/1979355898 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f63f8105ac0 0x7f63f8105f50 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:37:09.587 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:09.582+0000 7f63fe800640 1 --2- 172.21.15.83:0/1979355898 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f63f8100fd0 0x7f63f81054a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:37:09.587 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:09.582+0000 7f63fe800640 1 --2- 172.21.15.83:0/1979355898 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f63f81006b0 0x7f63f8100a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:37:09.587 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:09.582+0000 7f63fe800640 1 -- 172.21.15.83:0/1979355898 >> 172.21.15.83:0/1979355898 conn(0x7f63f80fc340 msgr2=0x7f63f80fe760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:37:09.587 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:09.583+0000 7f63fe800640 1 -- 172.21.15.83:0/1979355898 shutdown_connections 2024-08-19T01:37:09.587 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:09.583+0000 7f63fe800640 1 -- 172.21.15.83:0/1979355898 wait complete. 2024-08-19T01:37:09.588 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:09.585+0000 7f63fe800640 1 Processor -- start 2024-08-19T01:37:09.589 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:09.586+0000 7f63fe800640 1 -- start start 2024-08-19T01:37:09.589 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:09.587+0000 7f63fe800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f63f81006b0 0x7f63f8114980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:37:09.594 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:09.588+0000 7f63fd400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f63f81006b0 0x7f63f8114980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:37:09.595 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:09.590+0000 7f63fd400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f63f81006b0 0x7f63f8114980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:53330/0 (socket says 172.21.15.83:53330) 2024-08-19T01:37:09.595 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:09.590+0000 7f63fe800640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f63f8100fd0 0x7f63f8110a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:37:09.595 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:09.590+0000 7f63fe800640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f63f8105ac0 0x7f63f8110f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:37:09.595 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:09.591+0000 7f63fe800640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f63f8109450 con 0x7f63f81006b0 2024-08-19T01:37:09.595 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:09.591+0000 7f63fde00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f63f8105ac0 0x7f63f8110f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:37:09.595 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:09.591+0000 7f63fde00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f63f8105ac0 0x7f63f8110f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.83:55190/0 (socket says 172.21.15.83:55190) 2024-08-19T01:37:09.596 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:09.591+0000 7f63fde00640 1 -- 172.21.15.83:0/1627131342 learned_addr learned my addr 172.21.15.83:0/1627131342 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-08-19T01:37:09.596 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:09.592+0000 7f63fe800640 1 -- 172.21.15.83:0/1627131342 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f63f81092d0 con 0x7f63f8100fd0 2024-08-19T01:37:09.596 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:09.591+0000 7f63fca00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f63f8100fd0 0x7f63f8110a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:37:09.596 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:09.592+0000 7f63fe800640 1 -- 172.21.15.83:0/1627131342 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f63f81095d0 con 0x7f63f8105ac0 2024-08-19T01:37:09.596 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:09.593+0000 7f63fca00640 1 -- 172.21.15.83:0/1627131342 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f63f8105ac0 msgr2=0x7f63f8110f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:37:09.596 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:09.593+0000 7f63fca00640 1 --2- 172.21.15.83:0/1627131342 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f63f8105ac0 0x7f63f8110f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:37:09.596 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:09.593+0000 7f63fca00640 1 -- 172.21.15.83:0/1627131342 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f63f81006b0 msgr2=0x7f63f8114980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:37:09.596 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:09.594+0000 7f63fca00640 1 --2- 172.21.15.83:0/1627131342 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f63f81006b0 0x7f63f8114980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:37:09.596 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:09.594+0000 7f63fca00640 1 -- 172.21.15.83:0/1627131342 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f63f8111690 con 0x7f63f8100fd0 2024-08-19T01:37:09.597 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:09.594+0000 7f63fde00640 1 --2- 172.21.15.83:0/1627131342 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f63f8105ac0 0x7f63f8110f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-19T01:37:09.597 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:09.594+0000 7f63fd400640 1 --2- 172.21.15.83:0/1627131342 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f63f81006b0 0x7f63f8114980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-19T01:37:09.597 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:09.595+0000 7f63fca00640 1 --2- 172.21.15.83:0/1627131342 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f63f8100fd0 0x7f63f8110a20 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f63e400b410 tx=0x7f63e400b8d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:37:09.602 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:09.595+0000 7f63f6000640 1 -- 172.21.15.83:0/1627131342 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63e4015020 con 0x7f63f8100fd0 2024-08-19T01:37:09.602 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:09.595+0000 7f63f6000640 1 -- 172.21.15.83:0/1627131342 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f63e4011d10 con 0x7f63f8100fd0 2024-08-19T01:37:09.602 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:09.595+0000 7f63f6000640 1 -- 172.21.15.83:0/1627131342 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63e400f2e0 con 0x7f63f8100fd0 2024-08-19T01:37:09.602 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:09.595+0000 7f63fe800640 1 -- 172.21.15.83:0/1627131342 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f63f81a24d0 con 0x7f63f8100fd0 2024-08-19T01:37:09.602 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:09.596+0000 7f63fe800640 1 -- 172.21.15.83:0/1627131342 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f63f81a2870 con 0x7f63f8100fd0 2024-08-19T01:37:09.603 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:09.598+0000 7f63fe800640 1 -- 172.21.15.83:0/1627131342 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f63f806a580 con 0x7f63f8100fd0 2024-08-19T01:37:09.607 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:09.602+0000 7f63f6000640 1 -- 172.21.15.83:0/1627131342 <== mon.2 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7f63e4018020 con 0x7f63f8100fd0 2024-08-19T01:37:09.607 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:09.604+0000 7f63f6000640 1 --2- 172.21.15.83:0/1627131342 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f63e00a2cd0 0x7f63e00a5190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:37:09.607 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:09.605+0000 7f63fd400640 1 --2- 172.21.15.83:0/1627131342 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f63e00a2cd0 0x7f63e00a5190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:37:09.608 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:09.607+0000 7f63fd400640 1 --2- 172.21.15.83:0/1627131342 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f63e00a2cd0 0x7f63e00a5190 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f63ec011490 tx=0x7f63ec005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:37:09.608 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:09.607+0000 7f63f6000640 1 -- 172.21.15.83:0/1627131342 <== mon.2 v2:172.21.15.117:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8269+0+0 (secure 0 0 0) 0x7f63e40982d0 con 0x7f63f8100fd0 2024-08-19T01:37:09.609 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:09.607+0000 7f63f6000640 1 -- 172.21.15.83:0/1627131342 <== mon.2 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7f63e404cdf0 con 0x7f63f8100fd0 2024-08-19T01:37:09.914 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:37:09 smithi083 ceph-mon[17120]: pgmap v408: 97 pgs: 97 active+clean; 117 MiB data, 783 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.2 MiB/s wr, 528 op/s 2024-08-19T01:37:09.952 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:37:09 smithi117 ceph-mon[22016]: pgmap v408: 97 pgs: 97 active+clean; 117 MiB data, 783 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.2 MiB/s wr, 528 op/s 2024-08-19T01:37:09.966 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:37:09 smithi167 ceph-mon[21875]: pgmap v408: 97 pgs: 97 active+clean; 117 MiB data, 783 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.2 MiB/s wr, 528 op/s 2024-08-19T01:37:10.063 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:10.060+0000 7f63fe800640 1 -- 172.21.15.83:0/1627131342 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f63f81054d0 con 0x7f63f8100fd0 2024-08-19T01:37:10.072 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-08-19T01:37:10.072 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":26,"btime":"2024-08-19T01:36:40:482111+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14421,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6835/988556067","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":988556067},{"type":"v1","addr":"172.21.15.83:6835","nonce":988556067}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14433,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6837/869702632","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6836","nonce":869702632},{"type":"v1","addr":"172.21.15.83:6837","nonce":869702632}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14475,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6841/1795975999","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6840","nonce":1795975999},{"type":"v1","addr":"172.21.15.83:6841","nonce":1795975999}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24430,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6833/2505076046","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6832","nonce":2505076046},{"type":"v1","addr":"172.21.15.167:6833","nonce":2505076046}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24449,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6833/3557520478","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":3557520478},{"type":"v1","addr":"172.21.15.117:6833","nonce":3557520478}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24466,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6835/3482045966","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":3482045966},{"type":"v1","addr":"172.21.15.167:6835","nonce":3482045966}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7}],"filesystems":[{"mdsmap":{"epoch":26,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-08-19T01:32:42.054769+0000","modified":"2024-08-19T01:36:40.093322+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24545,"mds_1":24527,"mds_2":24509},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24478":{"gid":24478,"name":"c","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.167:6837/1762130408","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6836","nonce":1762130408},{"type":"v1","addr":"172.21.15.167:6837","nonce":1762130408}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24485":{"gid":24485,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.117:6835/1024213994","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":1024213994},{"type":"v1","addr":"172.21.15.117:6835","nonce":1024213994}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24496":{"gid":24496,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.167:6839/3507387682","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6838","nonce":3507387682},{"type":"v1","addr":"172.21.15.167:6839","nonce":3507387682}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24509":{"gid":24509,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":10,"addr":"172.21.15.117:6837/3185224684","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":3185224684},{"type":"v1","addr":"172.21.15.117:6837","nonce":3185224684}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24527":{"gid":24527,"name":"g","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.83:6839/3239261845","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6838","nonce":3239261845},{"type":"v1","addr":"172.21.15.83:6839","nonce":3239261845}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24545":{"gid":24545,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.117:6839/1967595704","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6838","nonce":1967595704},{"type":"v1","addr":"172.21.15.117:6839","nonce":1967595704}]},"join_fscid":-1,"export_targets":[2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24545,"qdb_cluster":[24509,24527,24545]},"id":1}]} 2024-08-19T01:37:10.072 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:10.061+0000 7f63f6000640 1 -- 172.21.15.83:0/1627131342 <== mon.2 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+11921 (secure 0 0 0) 0x7f63f81054d0 con 0x7f63f8100fd0 2024-08-19T01:37:10.072 INFO:teuthology.orchestra.run.smithi083.stderr:dumped fsmap epoch 26 2024-08-19T01:37:10.073 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:10.068+0000 7f63fe800640 1 -- 172.21.15.83:0/1627131342 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f63e00a2cd0 msgr2=0x7f63e00a5190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:37:10.073 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:10.068+0000 7f63fe800640 1 --2- 172.21.15.83:0/1627131342 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f63e00a2cd0 0x7f63e00a5190 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f63ec011490 tx=0x7f63ec005f90 comp rx=0 tx=0).stop 2024-08-19T01:37:10.073 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:10.068+0000 7f63fe800640 1 -- 172.21.15.83:0/1627131342 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f63f8100fd0 msgr2=0x7f63f8110a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:37:10.073 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:10.068+0000 7f63fe800640 1 --2- 172.21.15.83:0/1627131342 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f63f8100fd0 0x7f63f8110a20 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f63e400b410 tx=0x7f63e400b8d0 comp rx=0 tx=0).stop 2024-08-19T01:37:10.073 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:10.068+0000 7f63fe800640 1 -- 172.21.15.83:0/1627131342 shutdown_connections 2024-08-19T01:37:10.073 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:10.068+0000 7f63fe800640 1 --2- 172.21.15.83:0/1627131342 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f63e00a2cd0 0x7f63e00a5190 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:37:10.073 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:10.068+0000 7f63fe800640 1 --2- 172.21.15.83:0/1627131342 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f63f8105ac0 0x7f63f8110f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:37:10.073 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:10.068+0000 7f63fe800640 1 --2- 172.21.15.83:0/1627131342 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f63f8100fd0 0x7f63f8110a20 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:37:10.073 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:10.068+0000 7f63fe800640 1 --2- 172.21.15.83:0/1627131342 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f63f81006b0 0x7f63f8114980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:37:10.073 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:10.068+0000 7f63fe800640 1 -- 172.21.15.83:0/1627131342 >> 172.21.15.83:0/1627131342 conn(0x7f63f80fc340 msgr2=0x7f63f8103440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:37:10.073 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:10.070+0000 7f63fe800640 1 -- 172.21.15.83:0/1627131342 shutdown_connections 2024-08-19T01:37:10.073 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:10.070+0000 7f63fe800640 1 -- 172.21.15.83:0/1627131342 wait complete. 2024-08-19T01:37:10.534 INFO:tasks.workunit.client.0.smithi083.stdout: 56 638271 783 433232 579 390808 1709 2024-08-19T01:37:10.754 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:37:10 smithi083 ceph-mon[17120]: from='client.? 172.21.15.83:0/1627131342' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-19T01:37:10.952 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:37:10 smithi117 ceph-mon[22016]: from='client.? 172.21.15.83:0/1627131342' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-19T01:37:10.966 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:37:10 smithi167 ceph-mon[21875]: from='client.? 172.21.15.83:0/1627131342' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-19T01:37:11.415 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833 shell --fsid bad8780e-5dc9-11ef-bccf-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-08-19T01:37:11.655 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:37:11 smithi083 ceph-mon[17120]: pgmap v409: 97 pgs: 97 active+clean; 122 MiB data, 795 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 2.4 MiB/s wr, 414 op/s 2024-08-19T01:37:11.952 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:37:11 smithi117 ceph-mon[22016]: pgmap v409: 97 pgs: 97 active+clean; 122 MiB data, 795 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 2.4 MiB/s wr, 414 op/s 2024-08-19T01:37:11.965 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:37:11 smithi167 ceph-mon[21875]: pgmap v409: 97 pgs: 97 active+clean; 122 MiB data, 795 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 2.4 MiB/s wr, 414 op/s 2024-08-19T01:37:12.089 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/bad8780e-5dc9-11ef-bccf-c7b262605968/mon.a/config 2024-08-19T01:37:12.997 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:12.992+0000 7fc1cee00640 1 -- 172.21.15.83:0/74563816 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc1c81020a0 msgr2=0x7fc1c810e5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:37:12.997 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:12.992+0000 7fc1cee00640 1 --2- 172.21.15.83:0/74563816 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc1c81020a0 0x7fc1c810e5d0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fc1c0009ca0 tx=0x7fc1c002a8e0 comp rx=0 tx=0).stop 2024-08-19T01:37:12.999 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:12.996+0000 7fc1cee00640 1 -- 172.21.15.83:0/74563816 shutdown_connections 2024-08-19T01:37:12.999 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:12.996+0000 7fc1cee00640 1 --2- 172.21.15.83:0/74563816 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc1c81020a0 0x7fc1c810e5d0 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:37:12.999 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:12.996+0000 7fc1cee00640 1 --2- 172.21.15.83:0/74563816 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc1c8101700 0x7fc1c8101b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:37:12.999 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:12.997+0000 7fc1cee00640 1 --2- 172.21.15.83:0/74563816 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc1c8107700 0x7fc1c8107ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:37:12.999 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:12.997+0000 7fc1cee00640 1 -- 172.21.15.83:0/74563816 >> 172.21.15.83:0/74563816 conn(0x7fc1c80fd440 msgr2=0x7fc1c80ff860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:37:13.006 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:13.001+0000 7fc1cee00640 1 -- 172.21.15.83:0/74563816 shutdown_connections 2024-08-19T01:37:13.006 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:13.001+0000 7fc1cee00640 1 -- 172.21.15.83:0/74563816 wait complete. 2024-08-19T01:37:13.006 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:13.003+0000 7fc1cee00640 1 Processor -- start 2024-08-19T01:37:13.006 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:13.003+0000 7fc1cee00640 1 -- start start 2024-08-19T01:37:13.006 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:13.004+0000 7fc1cee00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc1c8101700 0x7fc1c810e4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:37:13.008 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:13.004+0000 7fc1cee00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc1c81020a0 0x7fc1c810b5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:37:13.008 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:13.005+0000 7fc1cd000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc1c81020a0 0x7fc1c810b5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:37:13.008 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:13.005+0000 7fc1cd000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc1c81020a0 0x7fc1c810b5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:35716/0 (socket says 172.21.15.83:35716) 2024-08-19T01:37:13.008 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:13.005+0000 7fc1cd000640 1 -- 172.21.15.83:0/3557774724 learned_addr learned my addr 172.21.15.83:0/3557774724 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-08-19T01:37:13.008 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:13.005+0000 7fc1cee00640 1 --2- 172.21.15.83:0/3557774724 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc1c8107700 0x7fc1c810bb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:37:13.008 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:13.006+0000 7fc1cee00640 1 -- 172.21.15.83:0/3557774724 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fc1c8075cf0 con 0x7fc1c81020a0 2024-08-19T01:37:13.008 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:13.006+0000 7fc1cee00640 1 -- 172.21.15.83:0/3557774724 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fc1c8075b70 con 0x7fc1c8101700 2024-08-19T01:37:13.008 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:13.006+0000 7fc1ce400640 1 --2- 172.21.15.83:0/3557774724 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc1c8107700 0x7fc1c810bb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:37:13.009 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:13.006+0000 7fc1cda00640 1 --2- 172.21.15.83:0/3557774724 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc1c8101700 0x7fc1c810e4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:37:13.009 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:13.006+0000 7fc1cee00640 1 -- 172.21.15.83:0/3557774724 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fc1c8075e70 con 0x7fc1c8107700 2024-08-19T01:37:13.010 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:13.007+0000 7fc1cd000640 1 -- 172.21.15.83:0/3557774724 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc1c8107700 msgr2=0x7fc1c810bb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:37:13.010 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:13.007+0000 7fc1cd000640 1 --2- 172.21.15.83:0/3557774724 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc1c8107700 0x7fc1c810bb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:37:13.011 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:13.007+0000 7fc1cd000640 1 -- 172.21.15.83:0/3557774724 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc1c8101700 msgr2=0x7fc1c810e4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:37:13.011 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:13.007+0000 7fc1cd000640 1 --2- 172.21.15.83:0/3557774724 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc1c8101700 0x7fc1c810e4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:37:13.011 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:13.007+0000 7fc1cd000640 1 -- 172.21.15.83:0/3557774724 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc1c81aa940 con 0x7fc1c81020a0 2024-08-19T01:37:13.011 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:13.007+0000 7fc1ce400640 1 --2- 172.21.15.83:0/3557774724 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc1c8107700 0x7fc1c810bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-19T01:37:13.011 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:13.007+0000 7fc1cda00640 1 --2- 172.21.15.83:0/3557774724 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc1c8101700 0x7fc1c810e4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-19T01:37:13.011 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:13.008+0000 7fc1cd000640 1 --2- 172.21.15.83:0/3557774724 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc1c81020a0 0x7fc1c810b5a0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7fc1b400da70 tx=0x7fc1b400df30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:37:13.011 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:13.008+0000 7fc1c6000640 1 -- 172.21.15.83:0/3557774724 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1b4017030 con 0x7fc1c81020a0 2024-08-19T01:37:13.011 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:13.009+0000 7fc1c6000640 1 -- 172.21.15.83:0/3557774724 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc1b40099a0 con 0x7fc1c81020a0 2024-08-19T01:37:13.011 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:13.009+0000 7fc1c6000640 1 -- 172.21.15.83:0/3557774724 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1b4003d30 con 0x7fc1c81020a0 2024-08-19T01:37:13.011 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:13.009+0000 7fc1cee00640 1 -- 172.21.15.83:0/3557774724 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc1c81aab10 con 0x7fc1c81020a0 2024-08-19T01:37:13.011 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:13.009+0000 7fc1cee00640 1 -- 172.21.15.83:0/3557774724 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc1c81ab040 con 0x7fc1c81020a0 2024-08-19T01:37:13.017 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:13.016+0000 7fc1d2dfc740 1 Processor -- start 2024-08-19T01:37:13.018 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:13.016+0000 7fc1d2dfc740 1 -- start start 2024-08-19T01:37:13.018 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:13.017+0000 7fc1d2dfc740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d8f96c16e0 0x55d8f96c1ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:37:13.022 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:13.017+0000 7fc1ce400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d8f96c16e0 0x55d8f96c1ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:37:13.022 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:13.017+0000 7fc1d2dfc740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d8f96c2000 0x55d8f96cc380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:37:13.023 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:13.017+0000 7fc1d2dfc740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d8f96cc8c0 0x55d8f96cecb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:37:13.023 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:13.017+0000 7fc1d2dfc740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55d8f9626bf0 con 0x55d8f96c16e0 2024-08-19T01:37:13.023 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:13.017+0000 7fc1d2dfc740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55d8f95beb20 con 0x55d8f96c2000 2024-08-19T01:37:13.023 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:13.017+0000 7fc1d2dfc740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55d8f9622bb0 con 0x55d8f96cc8c0 2024-08-19T01:37:13.023 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:13.018+0000 7fc1cda00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d8f96c2000 0x55d8f96cc380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:37:13.023 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:13.018+0000 7fc1cda00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d8f96c2000 0x55d8f96cc380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.83:48852/0 (socket says 172.21.15.83:48852) 2024-08-19T01:37:13.023 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:13.018+0000 7fc1cda00640 1 -- 172.21.15.83:0/4011846454 learned_addr learned my addr 172.21.15.83:0/4011846454 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-08-19T01:37:13.023 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:13.019+0000 7fc1ce400640 1 -- 172.21.15.83:0/4011846454 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d8f96cc8c0 msgr2=0x55d8f96cecb0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-08-19T01:37:13.023 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:13.019+0000 7fc1ce400640 1 --2- 172.21.15.83:0/4011846454 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d8f96cc8c0 0x55d8f96cecb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:37:13.023 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:13.019+0000 7fc1ce400640 1 -- 172.21.15.83:0/4011846454 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d8f96c2000 msgr2=0x55d8f96cc380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:37:13.023 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:13.019+0000 7fc1ce400640 1 --2- 172.21.15.83:0/4011846454 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d8f96c2000 0x55d8f96cc380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:37:13.023 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:13.019+0000 7fc1ce400640 1 -- 172.21.15.83:0/4011846454 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d8f95c57f0 con 0x55d8f96c16e0 2024-08-19T01:37:13.023 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:13.019+0000 7fc1cda00640 1 --2- 172.21.15.83:0/4011846454 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d8f96c2000 0x55d8f96cc380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-19T01:37:13.024 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:13.020+0000 7fc1ce400640 1 --2- 172.21.15.83:0/4011846454 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d8f96c16e0 0x55d8f96c1ac0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7fc1c0004af0 tx=0x7fc1c0031980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e7e575f1f3b2adf5 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:37:13.024 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:13.021+0000 7fc1baa00640 1 -- 172.21.15.83:0/4011846454 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1c003f070 con 0x55d8f96c16e0 2024-08-19T01:37:13.024 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:13.021+0000 7fc1baa00640 1 -- 172.21.15.83:0/4011846454 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc1c0036040 con 0x55d8f96c16e0 2024-08-19T01:37:13.024 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:13.021+0000 7fc1baa00640 1 -- 172.21.15.83:0/4011846454 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1c0004690 con 0x55d8f96c16e0 2024-08-19T01:37:13.025 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:13.022+0000 7fc1d2dfc740 1 -- 172.21.15.83:0/4011846454 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d8f96c16e0 msgr2=0x55d8f96c1ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:37:13.025 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:13.022+0000 7fc1d2dfc740 1 --2- 172.21.15.83:0/4011846454 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d8f96c16e0 0x55d8f96c1ac0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7fc1c0004af0 tx=0x7fc1c0031980 comp rx=0 tx=0).stop 2024-08-19T01:37:13.029 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:13.023+0000 7fc1d2dfc740 1 -- 172.21.15.83:0/4011846454 shutdown_connections 2024-08-19T01:37:13.029 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:13.023+0000 7fc1d2dfc740 1 --2- 172.21.15.83:0/4011846454 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d8f96cc8c0 0x55d8f96cecb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:37:13.029 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:13.023+0000 7fc1d2dfc740 1 --2- 172.21.15.83:0/4011846454 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d8f96c2000 0x55d8f96cc380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:37:13.029 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:13.023+0000 7fc1d2dfc740 1 --2- 172.21.15.83:0/4011846454 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d8f96c16e0 0x55d8f96c1ac0 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:37:13.029 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:13.023+0000 7fc1d2dfc740 1 -- 172.21.15.83:0/4011846454 >> 172.21.15.83:0/4011846454 conn(0x55d8f96bdc70 msgr2=0x55d8f96be050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:37:13.033 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:13.031+0000 7fc1c6000640 1 -- 172.21.15.83:0/3557774724 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7fc1b401a070 con 0x7fc1c81020a0 2024-08-19T01:37:13.035 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:13.033+0000 7fc1c6000640 1 --2- 172.21.15.83:0/3557774724 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7fc1b00a2b10 0x7fc1b00a4fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:37:13.036 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:13.034+0000 7fc1c6000640 1 -- 172.21.15.83:0/3557774724 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8269+0+0 (secure 0 0 0) 0x7fc1b40ce7f0 con 0x7fc1c81020a0 2024-08-19T01:37:13.037 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:13.034+0000 7fc1d2dfc740 1 -- 172.21.15.83:0/4011846454 shutdown_connections 2024-08-19T01:37:13.038 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:13.035+0000 7fc1cda00640 1 --2- 172.21.15.83:0/3557774724 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7fc1b00a2b10 0x7fc1b00a4fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:37:13.038 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:13.036+0000 7fc1cda00640 1 --2- 172.21.15.83:0/3557774724 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7fc1b00a2b10 0x7fc1b00a4fd0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x55d8f96bead0 tx=0x7fc1bc005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:37:13.039 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:13.037+0000 7fc1d2dfc740 1 -- 172.21.15.83:0/4011846454 wait complete. 2024-08-19T01:37:13.039 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:13.037+0000 7fc1d2dfc740 1 -- start start 2024-08-19T01:37:13.039 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:13.038+0000 7fc1d2dfc740 1 Processor -- start 2024-08-19T01:37:13.058 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:13.038+0000 7fc1d2dfc740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d8f96c16e0 0x55d8f96c9960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:37:13.058 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:13.038+0000 7fc1d2dfc740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d8f96c2000 0x55d8f96c9ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:37:13.058 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:13.039+0000 7fc1d2dfc740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d8f96ca3e0 0x55d8f96d8900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:37:13.058 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:13.039+0000 7fc1cda00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d8f96c16e0 0x55d8f96c9960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:37:13.059 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:13.039+0000 7fc1cda00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d8f96c16e0 0x55d8f96c9960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:35734/0 (socket says 172.21.15.83:35734) 2024-08-19T01:37:13.059 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:13.043+0000 7fc1cda00640 1 -- 172.21.15.83:0/391774663 learned_addr learned my addr 172.21.15.83:0/391774663 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-08-19T01:37:13.059 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:13.043+0000 7fc1d2dfc740 1 -- 172.21.15.83:0/391774663 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55d8f96cf490 con 0x55d8f96c16e0 2024-08-19T01:37:13.059 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:13.043+0000 7fc1d2dfc740 1 -- 172.21.15.83:0/391774663 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55d8f95e1cf0 con 0x55d8f96c2000 2024-08-19T01:37:13.059 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:13.043+0000 7fc1d2dfc740 1 -- 172.21.15.83:0/391774663 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55d8f96cf610 con 0x55d8f96ca3e0 2024-08-19T01:37:13.059 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:13.044+0000 7fc1ce400640 1 --2- 172.21.15.83:0/391774663 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d8f96ca3e0 0x55d8f96d8900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:37:13.059 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:13.044+0000 7fc1cd000640 1 --2- 172.21.15.83:0/391774663 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d8f96c2000 0x55d8f96c9ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:37:13.059 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:13.044+0000 7fc1d2dfc740 10 client.0 _ll_register_callbacks cb 0x55d8f95cde30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-19T01:37:13.059 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:13.044+0000 7fc1cda00640 1 -- 172.21.15.83:0/391774663 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d8f96ca3e0 msgr2=0x55d8f96d8900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:37:13.059 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:13.044+0000 7fc1cda00640 1 --2- 172.21.15.83:0/391774663 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d8f96ca3e0 0x55d8f96d8900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:37:13.059 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:13.044+0000 7fc1cda00640 1 -- 172.21.15.83:0/391774663 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d8f96c2000 msgr2=0x55d8f96c9ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:37:13.059 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:13.044+0000 7fc1cda00640 1 --2- 172.21.15.83:0/391774663 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d8f96c2000 0x55d8f96c9ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:37:13.059 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:13.044+0000 7fc1cda00640 1 -- 172.21.15.83:0/391774663 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d8f93983c0 con 0x55d8f96c16e0 2024-08-19T01:37:13.060 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:13.045+0000 7fc1cda00640 1 --2- 172.21.15.83:0/391774663 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d8f96c16e0 0x55d8f96c9960 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7fc1bc01d770 tx=0x7fc1bc0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=43274beb8536c00e server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:37:13.060 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:13.045+0000 7fc1af400640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-08-19T01:37:13.060 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:13.046+0000 7fc1af400640 1 -- 172.21.15.83:0/391774663 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1bc02a060 con 0x55d8f96c16e0 2024-08-19T01:37:13.060 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:13.046+0000 7fc1af400640 1 -- 172.21.15.83:0/391774663 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8269+0+0 (secure 0 0 0) 0x7fc1bc010940 con 0x55d8f96c16e0 2024-08-19T01:37:13.060 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:13.046+0000 7fc1d2dfc740 1 -- 172.21.15.83:0/391774663 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc1c0036040 con 0x55d8f96c16e0 2024-08-19T01:37:13.060 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:13.046+0000 7fc1af400640 1 -- 172.21.15.83:0/391774663 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fc1bc010b40 con 0x55d8f96c16e0 2024-08-19T01:37:13.060 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:13.047+0000 7fc1d2dfc740 10 client.15456 fetch_fsmap learned FSMap version 26 2024-08-19T01:37:13.060 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:13.047+0000 7fc1d2dfc740 1 -- 172.21.15.83:0/391774663 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55d8f96db740 con 0x55d8f96c16e0 2024-08-19T01:37:13.060 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:13.047+0000 7fc1af400640 1 -- 172.21.15.83:0/391774663 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc1c0036040 con 0x55d8f96c16e0 2024-08-19T01:37:13.060 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:13.048+0000 7fc1af400640 1 -- 172.21.15.83:0/391774663 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 26) ==== 8350+0+0 (secure 0 0 0) 0x7fc1bc019040 con 0x55d8f96c16e0 2024-08-19T01:37:13.060 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:13.049+0000 7fc1d2dfc740 10 client.15456 fetch_fsmap finished waiting for FSMap version 26 2024-08-19T01:37:13.060 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:13.049+0000 7fc1d2dfc740 10 client.15456 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-08-19T01:37:13.060 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:13.049+0000 7fc1d2dfc740 20 client.15456 populate_metadata read hostname 'smithi083' 2024-08-19T01:37:13.060 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:13.049+0000 7fc1d2dfc740 1 --2- 172.21.15.83:0/391774663 >> [v2:172.21.15.117:6836/3185224684,v1:172.21.15.117:6837/3185224684] conn(0x55d8f96dcc00 0x55d8f96deff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:37:13.061 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:13.049+0000 7fc1d2dfc740 4 client.15456 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-19T01:37:13.061 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:13.049+0000 7fc1d2dfc740 1 -- 172.21.15.83:0/391774663 --> [v2:172.21.15.117:6836/3185224684,v1:172.21.15.117:6837/3185224684] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d8f9634160 con 0x55d8f96dcc00 2024-08-19T01:37:13.061 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:13.050+0000 7fc1cd000640 1 --2- 172.21.15.83:0/391774663 >> [v2:172.21.15.117:6836/3185224684,v1:172.21.15.117:6837/3185224684] conn(0x55d8f96dcc00 0x55d8f96deff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:37:13.061 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:13.052+0000 7fc1cd000640 1 --2- 172.21.15.83:0/391774663 >> [v2:172.21.15.117:6836/3185224684,v1:172.21.15.117:6837/3185224684] conn(0x55d8f96dcc00 0x55d8f96deff0 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=40d484a0fd4e02a3 server_cookie=a52ba8d19604b7ac in_seq=0 out_seq=0 2024-08-19T01:37:13.061 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:13.052+0000 7fc1af400640 10 client.15456 ms_handle_connect on v2:172.21.15.117:6836/3185224684 2024-08-19T01:37:13.061 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:13.053+0000 7fc1af400640 1 -- 172.21.15.83:0/391774663 <== mds.2 v2:172.21.15.117:6836/3185224684 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55d8f9634160 con 0x55d8f96dcc00 2024-08-19T01:37:13.061 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:13.053+0000 7fc1af400640 10 client.15456 handle_command_reply: tid=0 2024-08-19T01:37:13.061 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:13.054+0000 7fc1d2dfc740 1 client.15456 shutdown 2024-08-19T01:37:13.061 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:13.054+0000 7fc1d2dfc740 1 -- 172.21.15.83:0/391774663 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d8f96c16e0 msgr2=0x55d8f96c9960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:37:13.061 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:13.054+0000 7fc1d2dfc740 1 --2- 172.21.15.83:0/391774663 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d8f96c16e0 0x55d8f96c9960 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7fc1bc01d770 tx=0x7fc1bc0046c0 comp rx=0 tx=0).stop 2024-08-19T01:37:13.061 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:13.055+0000 7fc1d2dfc740 1 -- 172.21.15.83:0/391774663 shutdown_connections 2024-08-19T01:37:13.061 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:13.055+0000 7fc1d2dfc740 1 --2- 172.21.15.83:0/391774663 >> [v2:172.21.15.117:6836/3185224684,v1:172.21.15.117:6837/3185224684] conn(0x55d8f96dcc00 0x55d8f96deff0 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:37:13.061 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:13.055+0000 7fc1d2dfc740 1 --2- 172.21.15.83:0/391774663 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d8f96ca3e0 0x55d8f96d8900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:37:13.061 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:13.055+0000 7fc1d2dfc740 1 --2- 172.21.15.83:0/391774663 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d8f96c2000 0x55d8f96c9ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:37:13.062 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:13.055+0000 7fc1d2dfc740 1 --2- 172.21.15.83:0/391774663 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d8f96c16e0 0x55d8f96c9960 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:37:13.062 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:13.055+0000 7fc1d2dfc740 1 -- 172.21.15.83:0/391774663 >> 172.21.15.83:0/391774663 conn(0x55d8f96bdc70 msgr2=0x55d8f96c39d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:37:13.062 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:13.055+0000 7fc1af400640 0 client.15456 ms_handle_reset on v2:172.21.15.117:6836/3185224684 2024-08-19T01:37:13.062 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:13.055+0000 7fc1d2dfc740 1 -- 172.21.15.83:0/391774663 shutdown_connections 2024-08-19T01:37:13.062 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:13.058+0000 7fc1d2dfc740 1 -- 172.21.15.83:0/391774663 wait complete. 2024-08-19T01:37:13.062 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:13.058+0000 7fc1d2dfc740 20 client.15456 trim_cache size 0 max 16384 2024-08-19T01:37:13.142 INFO:teuthology.orchestra.run.smithi083.stdout:[] 2024-08-19T01:37:13.143 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:13.087+0000 7fc1d2dfc740 1 Processor -- start 2024-08-19T01:37:13.143 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:13.087+0000 7fc1d2dfc740 1 -- start start 2024-08-19T01:37:13.143 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:13.087+0000 7fc1d2dfc740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d8f96c16e0 0x55d8f96d86f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:37:13.143 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:13.087+0000 7fc1d2dfc740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d8f96c2000 0x55d8f96cb8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:37:13.143 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:13.087+0000 7fc1d2dfc740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d8f96ca3e0 0x55d8f96e48a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:37:13.143 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:13.087+0000 7fc1d2dfc740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fc1c81b6110 con 0x55d8f96c16e0 2024-08-19T01:37:13.143 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:13.087+0000 7fc1d2dfc740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fc1c81b39e0 con 0x55d8f96ca3e0 2024-08-19T01:37:13.143 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:13.087+0000 7fc1d2dfc740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fc1c81b1830 con 0x55d8f96c2000 2024-08-19T01:37:13.143 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:13.088+0000 7fc1ce400640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d8f96ca3e0 0x55d8f96e48a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:37:13.143 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:13.088+0000 7fc1cd000640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d8f96c2000 0x55d8f96cb8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:37:13.143 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:13.088+0000 7fc1cd000640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d8f96c2000 0x55d8f96cb8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.83:59972/0 (socket says 172.21.15.83:59972) 2024-08-19T01:37:13.144 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:13.088+0000 7fc1cd000640 1 -- 172.21.15.83:0/1521192048 learned_addr learned my addr 172.21.15.83:0/1521192048 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-08-19T01:37:13.144 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:13.089+0000 7fc1cda00640 1 --2- 172.21.15.83:0/1521192048 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d8f96c16e0 0x55d8f96d86f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:37:13.144 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:13.089+0000 7fc1ce400640 1 -- 172.21.15.83:0/1521192048 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d8f96c2000 msgr2=0x55d8f96cb8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:37:13.144 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:13.089+0000 7fc1ce400640 1 --2- 172.21.15.83:0/1521192048 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d8f96c2000 0x55d8f96cb8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:37:13.144 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:13.089+0000 7fc1ce400640 1 -- 172.21.15.83:0/1521192048 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d8f96c16e0 msgr2=0x55d8f96d86f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:37:13.144 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:13.090+0000 7fc1ce400640 1 --2- 172.21.15.83:0/1521192048 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d8f96c16e0 0x55d8f96d86f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:37:13.144 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:13.090+0000 7fc1ce400640 1 -- 172.21.15.83:0/1521192048 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d8f96ca900 con 0x55d8f96ca3e0 2024-08-19T01:37:13.144 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:13.090+0000 7fc1cda00640 1 --2- 172.21.15.83:0/1521192048 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d8f96c16e0 0x55d8f96d86f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-19T01:37:13.144 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:13.090+0000 7fc1ce400640 1 --2- 172.21.15.83:0/1521192048 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d8f96ca3e0 0x55d8f96e48a0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fc1c002a920 tx=0x7fc1c0009a20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6fa35cf05dc9a9c7 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:37:13.144 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:13.091+0000 7fc1ba000640 1 -- 172.21.15.83:0/1521192048 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1c004a070 con 0x55d8f96ca3e0 2024-08-19T01:37:13.144 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:13.091+0000 7fc1ba000640 1 -- 172.21.15.83:0/1521192048 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc1c0036040 con 0x55d8f96ca3e0 2024-08-19T01:37:13.144 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:13.091+0000 7fc1ba000640 1 -- 172.21.15.83:0/1521192048 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1c004a070 con 0x55d8f96ca3e0 2024-08-19T01:37:13.144 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:13.092+0000 7fc1d2dfc740 1 -- 172.21.15.83:0/1521192048 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d8f96ca3e0 msgr2=0x55d8f96e48a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:37:13.145 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:13.092+0000 7fc1d2dfc740 1 --2- 172.21.15.83:0/1521192048 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d8f96ca3e0 0x55d8f96e48a0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fc1c002a920 tx=0x7fc1c0009a20 comp rx=0 tx=0).stop 2024-08-19T01:37:13.145 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:13.092+0000 7fc1d2dfc740 1 -- 172.21.15.83:0/1521192048 shutdown_connections 2024-08-19T01:37:13.145 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:13.092+0000 7fc1d2dfc740 1 --2- 172.21.15.83:0/1521192048 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d8f96ca3e0 0x55d8f96e48a0 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:37:13.145 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:13.092+0000 7fc1d2dfc740 1 --2- 172.21.15.83:0/1521192048 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d8f96c2000 0x55d8f96cb8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:37:13.145 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:13.092+0000 7fc1d2dfc740 1 --2- 172.21.15.83:0/1521192048 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d8f96c16e0 0x55d8f96d86f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:37:13.145 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:13.092+0000 7fc1d2dfc740 1 -- 172.21.15.83:0/1521192048 >> 172.21.15.83:0/1521192048 conn(0x55d8f96bdc70 msgr2=0x55d8f96cb1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:37:13.145 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:13.093+0000 7fc1d2dfc740 1 -- 172.21.15.83:0/1521192048 shutdown_connections 2024-08-19T01:37:13.145 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:13.093+0000 7fc1d2dfc740 1 -- 172.21.15.83:0/1521192048 wait complete. 2024-08-19T01:37:13.145 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:13.096+0000 7fc1d2dfc740 1 -- start start 2024-08-19T01:37:13.145 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:13.097+0000 7fc1d2dfc740 1 Processor -- start 2024-08-19T01:37:13.145 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:13.101+0000 7fc1d2dfc740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d8f96c16e0 0x55d8f97430c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:37:13.145 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:13.101+0000 7fc1d2dfc740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d8f96c2000 0x55d8f96d4860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:37:13.145 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:13.101+0000 7fc1d2dfc740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d8f96ca3e0 0x55d8f96d4da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:37:13.146 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:13.101+0000 7fc1d2dfc740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fc1c81b1ea0 con 0x55d8f96c16e0 2024-08-19T01:37:13.146 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:13.101+0000 7fc1d2dfc740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fc1c81b3310 con 0x55d8f96ca3e0 2024-08-19T01:37:13.146 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:13.101+0000 7fc1d2dfc740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fc1c81b2570 con 0x55d8f96c2000 2024-08-19T01:37:13.146 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:13.101+0000 7fc1d2dfc740 10 client.0 _ll_register_callbacks cb 0x7fc1bc01d420 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-19T01:37:13.146 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:13.105+0000 7fc1cd000640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d8f96c2000 0x55d8f96d4860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:37:13.146 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:13.105+0000 7fc1ce400640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d8f96ca3e0 0x55d8f96d4da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:37:13.146 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:13.106+0000 7fc1ce400640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d8f96ca3e0 0x55d8f96d4da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.83:48878/0 (socket says 172.21.15.83:48878) 2024-08-19T01:37:13.146 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:13.106+0000 7fc1ce400640 1 -- 172.21.15.83:0/818231900 learned_addr learned my addr 172.21.15.83:0/818231900 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-08-19T01:37:13.146 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:13.106+0000 7fc1cda00640 1 --2- 172.21.15.83:0/818231900 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d8f96c16e0 0x55d8f97430c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:37:13.146 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:13.107+0000 7fc1ce400640 1 -- 172.21.15.83:0/818231900 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d8f96c2000 msgr2=0x55d8f96d4860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:37:13.146 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:13.107+0000 7fc1ce400640 1 --2- 172.21.15.83:0/818231900 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d8f96c2000 0x55d8f96d4860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:37:13.146 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:13.107+0000 7fc1ce400640 1 -- 172.21.15.83:0/818231900 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d8f96c16e0 msgr2=0x55d8f97430c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:37:13.147 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:13.107+0000 7fc1ce400640 1 --2- 172.21.15.83:0/818231900 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d8f96c16e0 0x55d8f97430c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:37:13.147 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:13.107+0000 7fc1ce400640 1 -- 172.21.15.83:0/818231900 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d8f9740410 con 0x55d8f96ca3e0 2024-08-19T01:37:13.147 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:13.107+0000 7fc1cd000640 1 --2- 172.21.15.83:0/818231900 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d8f96c2000 0x55d8f96d4860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-19T01:37:13.147 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:13.109+0000 7fc1ce400640 1 --2- 172.21.15.83:0/818231900 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d8f96ca3e0 0x55d8f96d4da0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fc1c0030810 tx=0x7fc1c0030840 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2eeb42a4f31c661f server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:37:13.147 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:13.109+0000 7fc1af400640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-08-19T01:37:13.147 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:13.109+0000 7fc1af400640 1 -- 172.21.15.83:0/818231900 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1c0055060 con 0x55d8f96ca3e0 2024-08-19T01:37:13.147 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:13.109+0000 7fc1af400640 1 -- 172.21.15.83:0/818231900 <== mon.2 v2:172.21.15.117:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8269+0+0 (secure 0 0 0) 0x7fc1c0030df0 con 0x55d8f96ca3e0 2024-08-19T01:37:13.147 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:13.109+0000 7fc1d2dfc740 1 -- 172.21.15.83:0/818231900 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc1c0036040 con 0x55d8f96ca3e0 2024-08-19T01:37:13.147 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:13.110+0000 7fc1af400640 1 -- 172.21.15.83:0/818231900 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fc1c004a040 con 0x55d8f96ca3e0 2024-08-19T01:37:13.147 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:13.111+0000 7fc1d2dfc740 10 client.25535 fetch_fsmap learned FSMap version 26 2024-08-19T01:37:13.147 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:13.111+0000 7fc1d2dfc740 1 -- 172.21.15.83:0/818231900 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55d8f97699e0 con 0x55d8f96ca3e0 2024-08-19T01:37:13.147 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:13.112+0000 7fc1af400640 1 -- 172.21.15.83:0/818231900 <== mon.2 v2:172.21.15.117:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc1c0036040 con 0x55d8f96ca3e0 2024-08-19T01:37:13.147 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:13.112+0000 7fc1af400640 1 -- 172.21.15.83:0/818231900 <== mon.2 v2:172.21.15.117:3300/0 5 ==== fsmap(e 26) ==== 8350+0+0 (secure 0 0 0) 0x7fc1c0031ce0 con 0x55d8f96ca3e0 2024-08-19T01:37:13.147 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:13.112+0000 7fc1d2dfc740 10 client.25535 fetch_fsmap finished waiting for FSMap version 26 2024-08-19T01:37:13.148 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:13.112+0000 7fc1d2dfc740 10 client.25535 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-08-19T01:37:13.148 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:13.113+0000 7fc1d2dfc740 20 client.25535 populate_metadata read hostname 'smithi083' 2024-08-19T01:37:13.148 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:13.113+0000 7fc1d2dfc740 1 --2- 172.21.15.83:0/818231900 >> [v2:172.21.15.117:6836/3185224684,v1:172.21.15.117:6837/3185224684] conn(0x55d8f9769bb0 0x55d8f976bfa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:37:13.148 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:13.113+0000 7fc1d2dfc740 4 client.25535 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-08-19T01:37:13.148 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:13.113+0000 7fc1d2dfc740 1 -- 172.21.15.83:0/818231900 --> [v2:172.21.15.117:6836/3185224684,v1:172.21.15.117:6837/3185224684] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55d8f962e670 con 0x55d8f9769bb0 2024-08-19T01:37:13.148 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:13.113+0000 7fc1cda00640 1 --2- 172.21.15.83:0/818231900 >> [v2:172.21.15.117:6836/3185224684,v1:172.21.15.117:6837/3185224684] conn(0x55d8f9769bb0 0x55d8f976bfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:37:13.148 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:13.114+0000 7fc1cda00640 1 --2- 172.21.15.83:0/818231900 >> [v2:172.21.15.117:6836/3185224684,v1:172.21.15.117:6837/3185224684] conn(0x55d8f9769bb0 0x55d8f976bfa0 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=d8685831734e4147 server_cookie=d4ddd10a0a19f5ba in_seq=0 out_seq=0 2024-08-19T01:37:13.148 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:13.114+0000 7fc1af400640 10 client.25535 ms_handle_connect on v2:172.21.15.117:6836/3185224684 2024-08-19T01:37:13.148 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:13.114+0000 7fc1af400640 1 -- 172.21.15.83:0/818231900 <== mds.2 v2:172.21.15.117:6836/3185224684 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55d8f962e670 con 0x55d8f9769bb0 2024-08-19T01:37:13.148 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:13.115+0000 7fc1af400640 10 client.25535 handle_command_reply: tid=0 2024-08-19T01:37:13.148 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:13.115+0000 7fc1d2dfc740 1 client.25535 shutdown 2024-08-19T01:37:13.148 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:13.115+0000 7fc1d2dfc740 1 -- 172.21.15.83:0/818231900 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d8f96ca3e0 msgr2=0x55d8f96d4da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:37:13.148 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:13.115+0000 7fc1d2dfc740 1 --2- 172.21.15.83:0/818231900 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d8f96ca3e0 0x55d8f96d4da0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fc1c0030810 tx=0x7fc1c0030840 comp rx=0 tx=0).stop 2024-08-19T01:37:13.148 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:13.116+0000 7fc1d2dfc740 1 -- 172.21.15.83:0/818231900 shutdown_connections 2024-08-19T01:37:13.149 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:13.116+0000 7fc1d2dfc740 1 --2- 172.21.15.83:0/818231900 >> [v2:172.21.15.117:6836/3185224684,v1:172.21.15.117:6837/3185224684] conn(0x55d8f9769bb0 0x55d8f976bfa0 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:37:13.149 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:13.116+0000 7fc1d2dfc740 1 --2- 172.21.15.83:0/818231900 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d8f96ca3e0 0x55d8f96d4da0 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:37:13.149 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:13.116+0000 7fc1d2dfc740 1 --2- 172.21.15.83:0/818231900 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d8f96c2000 0x55d8f96d4860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:37:13.149 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:13.116+0000 7fc1d2dfc740 1 --2- 172.21.15.83:0/818231900 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d8f96c16e0 0x55d8f97430c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:37:13.149 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:13.116+0000 7fc1d2dfc740 1 -- 172.21.15.83:0/818231900 >> 172.21.15.83:0/818231900 conn(0x55d8f96bdc70 msgr2=0x55d8f96c39a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:37:13.149 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:13.116+0000 7fc1af400640 0 client.25535 ms_handle_reset on v2:172.21.15.117:6836/3185224684 2024-08-19T01:37:13.149 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:13.117+0000 7fc1d2dfc740 1 -- 172.21.15.83:0/818231900 shutdown_connections 2024-08-19T01:37:13.149 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:13.117+0000 7fc1d2dfc740 1 -- 172.21.15.83:0/818231900 wait complete. 2024-08-19T01:37:13.149 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:13.118+0000 7fc1d2dfc740 20 client.25535 trim_cache size 0 max 16384 2024-08-19T01:37:13.149 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:13.123+0000 7fc1cee00640 1 -- 172.21.15.83:0/3557774724 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7fc1b00a2b10 msgr2=0x7fc1b00a4fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:37:13.149 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:13.123+0000 7fc1cee00640 1 --2- 172.21.15.83:0/3557774724 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7fc1b00a2b10 0x7fc1b00a4fd0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x55d8f96bead0 tx=0x7fc1bc005f90 comp rx=0 tx=0).stop 2024-08-19T01:37:13.149 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:13.123+0000 7fc1cee00640 1 -- 172.21.15.83:0/3557774724 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc1c81020a0 msgr2=0x7fc1c810b5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:37:13.149 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:13.123+0000 7fc1cee00640 1 --2- 172.21.15.83:0/3557774724 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc1c81020a0 0x7fc1c810b5a0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7fc1b400da70 tx=0x7fc1b400df30 comp rx=0 tx=0).stop 2024-08-19T01:37:13.150 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:13.124+0000 7fc1cee00640 1 -- 172.21.15.83:0/3557774724 shutdown_connections 2024-08-19T01:37:13.150 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:13.124+0000 7fc1cee00640 1 --2- 172.21.15.83:0/3557774724 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7fc1b00a2b10 0x7fc1b00a4fd0 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:37:13.150 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:13.124+0000 7fc1cee00640 1 --2- 172.21.15.83:0/3557774724 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc1c8107700 0x7fc1c810bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:37:13.150 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:13.124+0000 7fc1cee00640 1 --2- 172.21.15.83:0/3557774724 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc1c81020a0 0x7fc1c810b5a0 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:37:13.150 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:13.124+0000 7fc1cee00640 1 --2- 172.21.15.83:0/3557774724 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc1c8101700 0x7fc1c810e4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:37:13.150 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:13.124+0000 7fc1cee00640 1 -- 172.21.15.83:0/3557774724 >> 172.21.15.83:0/3557774724 conn(0x7fc1c80fd440 msgr2=0x7fc1c80fdb00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:37:13.150 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:13.124+0000 7fc1cee00640 1 -- 172.21.15.83:0/3557774724 shutdown_connections 2024-08-19T01:37:13.150 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:13.124+0000 7fc1cee00640 1 -- 172.21.15.83:0/3557774724 wait complete. 2024-08-19T01:37:13.674 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:37:13 smithi083 ceph-mon[17120]: pgmap v410: 97 pgs: 97 active+clean; 122 MiB data, 798 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.1 MiB/s wr, 273 op/s 2024-08-19T01:37:13.843 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:37:13.841+0000 7f463e600640 1 -- 172.21.15.83:0/3177650187 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4638072a50 msgr2=0x7f4638072eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:37:13.843 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:37:13.841+0000 7f463e600640 1 --2- 172.21.15.83:0/3177650187 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4638072a50 0x7f4638072eb0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f4628009950 tx=0x7f462802a460 comp rx=0 tx=0).stop 2024-08-19T01:37:13.846 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:37:13.844+0000 7f463e600640 1 -- 172.21.15.83:0/3177650187 shutdown_connections 2024-08-19T01:37:13.846 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:37:13.844+0000 7f463e600640 1 --2- 172.21.15.83:0/3177650187 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4638073500 0x7f4638112270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:37:13.846 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:37:13.844+0000 7f463e600640 1 --2- 172.21.15.83:0/3177650187 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4638072a50 0x7f4638072eb0 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:37:13.846 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:37:13.844+0000 7f463e600640 1 --2- 172.21.15.83:0/3177650187 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4638109660 0x7f4638109a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:37:13.846 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:37:13.844+0000 7f463e600640 1 -- 172.21.15.83:0/3177650187 >> 172.21.15.83:0/3177650187 conn(0x7f463806ea10 msgr2=0x7f4638070e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:37:13.846 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:37:13.845+0000 7f463e600640 1 -- 172.21.15.83:0/3177650187 shutdown_connections 2024-08-19T01:37:13.846 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:37:13.845+0000 7f463e600640 1 -- 172.21.15.83:0/3177650187 wait complete. 2024-08-19T01:37:13.847 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:37:13.846+0000 7f463e600640 1 Processor -- start 2024-08-19T01:37:13.848 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:37:13.846+0000 7f463e600640 1 -- start start 2024-08-19T01:37:13.848 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:37:13.847+0000 7f463e600640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4638072a50 0x7f46381a6ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:37:13.848 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:37:13.847+0000 7f463e600640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4638073500 0x7f46381a6ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:37:13.849 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:37:13.847+0000 7f463e600640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4638109660 0x7f46381ae060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:37:13.849 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:37:13.848+0000 7f4637400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4638072a50 0x7f46381a6ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:37:13.850 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:37:13.848+0000 7f4636a00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4638073500 0x7f46381a6ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:37:13.850 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:37:13.848+0000 7f4636a00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4638073500 0x7f46381a6ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.83:60020/0 (socket says 172.21.15.83:60020) 2024-08-19T01:37:13.850 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:37:13.848+0000 7f4637e00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4638109660 0x7f46381ae060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:37:13.851 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:37:13.848+0000 7f4637e00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4638109660 0x7f46381ae060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.83:48900/0 (socket says 172.21.15.83:48900) 2024-08-19T01:37:13.851 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:37:13.848+0000 7f4636a00640 1 -- 172.21.15.83:0/2235227491 learned_addr learned my addr 172.21.15.83:0/2235227491 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-08-19T01:37:13.851 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:37:13.848+0000 7f463e600640 1 -- 172.21.15.83:0/2235227491 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f4638115e30 con 0x7f4638072a50 2024-08-19T01:37:13.851 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:37:13.849+0000 7f463e600640 1 -- 172.21.15.83:0/2235227491 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f4638115cb0 con 0x7f4638109660 2024-08-19T01:37:13.851 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:37:13.849+0000 7f463e600640 1 -- 172.21.15.83:0/2235227491 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f4638115fb0 con 0x7f4638073500 2024-08-19T01:37:13.851 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:37:13.849+0000 7f4637e00640 1 -- 172.21.15.83:0/2235227491 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4638073500 msgr2=0x7f46381a6ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:37:13.852 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:37:13.850+0000 7f4637e00640 1 --2- 172.21.15.83:0/2235227491 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4638073500 0x7f46381a6ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:37:13.852 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:37:13.850+0000 7f4637e00640 1 -- 172.21.15.83:0/2235227491 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4638072a50 msgr2=0x7f46381a6ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:37:13.852 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:37:13.850+0000 7f4637e00640 1 --2- 172.21.15.83:0/2235227491 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4638072a50 0x7f46381a6ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:37:13.852 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:37:13.850+0000 7f4637e00640 1 -- 172.21.15.83:0/2235227491 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f46381ae6d0 con 0x7f4638109660 2024-08-19T01:37:13.852 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:37:13.850+0000 7f4637e00640 1 --2- 172.21.15.83:0/2235227491 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4638109660 0x7f46381ae060 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f462400b410 tx=0x7f462400b8d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:37:13.856 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:37:13.855+0000 7f462fe00640 1 -- 172.21.15.83:0/2235227491 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4624015020 con 0x7f4638109660 2024-08-19T01:37:13.857 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:37:13.855+0000 7f462fe00640 1 -- 172.21.15.83:0/2235227491 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4624011d10 con 0x7f4638109660 2024-08-19T01:37:13.857 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:37:13.855+0000 7f462fe00640 1 -- 172.21.15.83:0/2235227491 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f462400f2e0 con 0x7f4638109660 2024-08-19T01:37:13.864 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:37:13.855+0000 7f463e600640 1 -- 172.21.15.83:0/2235227491 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f46381ae9c0 con 0x7f4638109660 2024-08-19T01:37:13.864 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:37:13.855+0000 7f463e600640 1 -- 172.21.15.83:0/2235227491 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f46381aeee0 con 0x7f4638109660 2024-08-19T01:37:13.873 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:37:13.868+0000 7f4641491740 1 Processor -- start 2024-08-19T01:37:13.873 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:37:13.869+0000 7f462fe00640 1 -- 172.21.15.83:0/2235227491 <== mon.2 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7f4624018020 con 0x7f4638109660 2024-08-19T01:37:13.873 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:37:13.871+0000 7f462fe00640 1 --2- 172.21.15.83:0/2235227491 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f461c0a2e20 0x7f461c0a52e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:37:13.873 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:37:13.871+0000 7f462fe00640 1 -- 172.21.15.83:0/2235227491 <== mon.2 v2:172.21.15.117:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8269+0+0 (secure 0 0 0) 0x7f46240ce7e0 con 0x7f4638109660 2024-08-19T01:37:13.874 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:37:13.872+0000 7f4641491740 1 -- start start 2024-08-19T01:37:13.874 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:37:13.872+0000 7f4641491740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b13b928480 0x55b13b928880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:37:13.874 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:37:13.872+0000 7f4641491740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b13b928dc0 0x55b13b92d880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:37:13.875 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:37:13.872+0000 7f4641491740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b13b92de50 0x55b13b930310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:37:13.875 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:37:13.872+0000 7f4641491740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55b13b88c890 con 0x55b13b92de50 2024-08-19T01:37:13.875 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:37:13.872+0000 7f4641491740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55b13b824d90 con 0x55b13b928dc0 2024-08-19T01:37:13.875 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:37:13.873+0000 7f4641491740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55b13b888850 con 0x55b13b928480 2024-08-19T01:37:13.875 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:37:13.873+0000 7f4637400640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b13b928480 0x55b13b928880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:37:13.875 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:37:13.873+0000 7f4637400640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b13b928480 0x55b13b928880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.83:60036/0 (socket says 172.21.15.83:60036) 2024-08-19T01:37:13.875 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:37:13.873+0000 7f4637400640 1 -- 172.21.15.83:0/2022169141 learned_addr learned my addr 172.21.15.83:0/2022169141 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-08-19T01:37:13.875 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:37:13.874+0000 7f4637e00640 1 --2- 172.21.15.83:0/2022169141 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b13b92de50 0x55b13b930310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:37:13.876 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:37:13.875+0000 7f4637e00640 1 -- 172.21.15.83:0/2022169141 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b13b928480 msgr2=0x55b13b928880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:37:13.877 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:37:13.875+0000 7f4637e00640 1 --2- 172.21.15.83:0/2022169141 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b13b928480 0x55b13b928880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:37:13.878 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:37:13.875+0000 7f4637e00640 1 -- 172.21.15.83:0/2022169141 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b13b928dc0 msgr2=0x55b13b92d880 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-08-19T01:37:13.878 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:37:13.875+0000 7f4637e00640 1 --2- 172.21.15.83:0/2022169141 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b13b928dc0 0x55b13b92d880 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:37:13.878 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:37:13.875+0000 7f4637e00640 1 -- 172.21.15.83:0/2022169141 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b13b5a0210 con 0x55b13b92de50 2024-08-19T01:37:13.878 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:37:13.875+0000 7f4637400640 1 --2- 172.21.15.83:0/2022169141 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b13b928480 0x55b13b928880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-19T01:37:13.878 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:37:13.876+0000 7f4637e00640 1 --2- 172.21.15.83:0/2022169141 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b13b92de50 0x55b13b930310 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f46240cc260 tx=0x7f4624010ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3f26c4aa17d8dcdd server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:37:13.878 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:37:13.876+0000 7f4636a00640 1 --2- 172.21.15.83:0/2235227491 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f461c0a2e20 0x7f461c0a52e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:37:13.878 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:37:13.876+0000 7f462cc00640 1 -- 172.21.15.83:0/2022169141 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f462409afc0 con 0x55b13b92de50 2024-08-19T01:37:13.878 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:37:13.876+0000 7f462cc00640 1 -- 172.21.15.83:0/2022169141 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f462409f370 con 0x55b13b92de50 2024-08-19T01:37:13.878 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:37:13.876+0000 7f462cc00640 1 -- 172.21.15.83:0/2022169141 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4624095d90 con 0x55b13b92de50 2024-08-19T01:37:13.879 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:37:13.877+0000 7f4641491740 1 -- 172.21.15.83:0/2022169141 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b13b92de50 msgr2=0x55b13b930310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:37:13.879 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:37:13.877+0000 7f4641491740 1 --2- 172.21.15.83:0/2022169141 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b13b92de50 0x55b13b930310 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f46240cc260 tx=0x7f4624010ed0 comp rx=0 tx=0).stop 2024-08-19T01:37:13.883 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:37:13.882+0000 7f4636a00640 1 --2- 172.21.15.83:0/2235227491 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f461c0a2e20 0x7f461c0a52e0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f46381a7fc0 tx=0x7f4628000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:37:13.884 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:37:13.882+0000 7f4641491740 1 -- 172.21.15.83:0/2022169141 shutdown_connections 2024-08-19T01:37:13.884 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:37:13.882+0000 7f4641491740 1 --2- 172.21.15.83:0/2022169141 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b13b92de50 0x55b13b930310 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:37:13.884 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:37:13.882+0000 7f4641491740 1 --2- 172.21.15.83:0/2022169141 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b13b928dc0 0x55b13b92d880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:37:13.884 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:37:13.882+0000 7f4641491740 1 --2- 172.21.15.83:0/2022169141 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b13b928480 0x55b13b928880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:37:13.884 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:37:13.882+0000 7f4641491740 1 -- 172.21.15.83:0/2022169141 >> 172.21.15.83:0/2022169141 conn(0x55b13b923af0 msgr2=0x55b13b923ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:37:13.884 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:37:13.882+0000 7f4641491740 1 -- 172.21.15.83:0/2022169141 shutdown_connections 2024-08-19T01:37:13.885 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:37:13.883+0000 7f4641491740 1 -- 172.21.15.83:0/2022169141 wait complete. 2024-08-19T01:37:13.885 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:37:13.883+0000 7f4641491740 1 -- start start 2024-08-19T01:37:13.885 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:37:13.883+0000 7f4641491740 1 Processor -- start 2024-08-19T01:37:13.885 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:37:13.884+0000 7f4641491740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b13b928480 0x55b13b943810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:37:13.886 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:37:13.885+0000 7f4641491740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b13b928dc0 0x55b13b943d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:37:13.886 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:37:13.885+0000 7f4637400640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b13b928480 0x55b13b943810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:37:13.886 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:37:13.885+0000 7f4637400640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b13b928480 0x55b13b943810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.83:60050/0 (socket says 172.21.15.83:60050) 2024-08-19T01:37:13.887 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:37:13.885+0000 7f4637400640 1 -- 172.21.15.83:0/1684758137 learned_addr learned my addr 172.21.15.83:0/1684758137 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-08-19T01:37:13.888 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:37:13.885+0000 7f4636a00640 1 --2- 172.21.15.83:0/1684758137 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b13b928dc0 0x55b13b943d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:37:13.888 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:37:13.885+0000 7f4641491740 1 --2- 172.21.15.83:0/1684758137 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b13b92de50 0x55b13b94ad20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:37:13.888 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:37:13.886+0000 7f4637e00640 1 --2- 172.21.15.83:0/1684758137 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b13b92de50 0x55b13b94ad20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:37:13.888 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:37:13.886+0000 7f4641491740 1 -- 172.21.15.83:0/1684758137 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55b13b941e70 con 0x55b13b92de50 2024-08-19T01:37:13.888 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:37:13.887+0000 7f4641491740 1 -- 172.21.15.83:0/1684758137 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55b13b847f70 con 0x55b13b928dc0 2024-08-19T01:37:13.888 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:37:13.887+0000 7f4641491740 1 -- 172.21.15.83:0/1684758137 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55b13b941ff0 con 0x55b13b928480 2024-08-19T01:37:13.890 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:37:13.888+0000 7f4637e00640 1 -- 172.21.15.83:0/1684758137 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b13b928480 msgr2=0x55b13b943810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:37:13.891 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:37:13.888+0000 7f4637e00640 1 --2- 172.21.15.83:0/1684758137 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b13b928480 0x55b13b943810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:37:13.891 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:37:13.888+0000 7f4637e00640 1 -- 172.21.15.83:0/1684758137 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b13b928dc0 msgr2=0x55b13b943d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:37:13.891 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:37:13.888+0000 7f4637e00640 1 --2- 172.21.15.83:0/1684758137 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b13b928dc0 0x55b13b943d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:37:13.891 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:37:13.888+0000 7f4637e00640 1 -- 172.21.15.83:0/1684758137 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b13b66c620 con 0x55b13b92de50 2024-08-19T01:37:13.891 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:37:13.888+0000 7f4637e00640 1 --2- 172.21.15.83:0/1684758137 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b13b92de50 0x55b13b94ad20 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f46240cc260 tx=0x7f462409a640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=859c8dbf3bcc0ace server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:37:13.891 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:37:13.889+0000 7f4621600640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-08-19T01:37:13.891 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:37:13.889+0000 7f4621600640 1 -- 172.21.15.83:0/1684758137 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46240a8060 con 0x55b13b92de50 2024-08-19T01:37:13.891 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:37:13.889+0000 7f4621600640 1 -- 172.21.15.83:0/1684758137 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8269+0+0 (secure 0 0 0) 0x7f4624095b30 con 0x55b13b92de50 2024-08-19T01:37:13.891 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:37:13.889+0000 7f4621600640 10 client.0 inactive, discarding osd_map(86..86 src has 1..86) 2024-08-19T01:37:13.894 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:37:13.892+0000 7f4641491740 10 client.0 _ll_register_callbacks cb 0x55b13b8848d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-19T01:37:13.894 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:37:13.893+0000 7f4641491740 1 -- 172.21.15.83:0/1684758137 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f462409f370 con 0x55b13b92de50 2024-08-19T01:37:13.895 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:37:13.893+0000 7f4621600640 1 -- 172.21.15.83:0/1684758137 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f46240977a0 con 0x55b13b92de50 2024-08-19T01:37:13.895 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:37:13.893+0000 7f4641491740 10 client.15471 fetch_fsmap learned FSMap version 26 2024-08-19T01:37:13.895 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:37:13.894+0000 7f4641491740 1 -- 172.21.15.83:0/1684758137 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({fsmap=26}) -- 0x55b13b94d770 con 0x55b13b92de50 2024-08-19T01:37:13.895 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:37:13.894+0000 7f4621600640 1 -- 172.21.15.83:0/1684758137 <== mon.0 v2:172.21.15.83:3300/0 4 ==== fsmap(e 26) ==== 8350+0+0 (secure 0 0 0) 0x7f462409d070 con 0x55b13b92de50 2024-08-19T01:37:13.896 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:37:13.894+0000 7f4641491740 10 client.15471 fetch_fsmap finished waiting for FSMap version 26 2024-08-19T01:37:13.896 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:37:13.895+0000 7f4641491740 10 client.15471 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-08-19T01:37:13.896 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:37:13.895+0000 7f4641491740 20 client.15471 populate_metadata read hostname 'smithi083' 2024-08-19T01:37:13.896 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:37:13.895+0000 7f4641491740 1 --2- 172.21.15.83:0/1684758137 >> [v2:172.21.15.117:6838/1967595704,v1:172.21.15.117:6839/1967595704] conn(0x55b13b94ed30 0x55b13b9511c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:37:13.896 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:37:13.895+0000 7f4641491740 4 client.15471 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-19T01:37:13.897 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:37:13.895+0000 7f4641491740 1 -- 172.21.15.83:0/1684758137 --> [v2:172.21.15.117:6838/1967595704,v1:172.21.15.117:6839/1967595704] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f462409f370 con 0x55b13b94ed30 2024-08-19T01:37:13.897 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:37:13.895+0000 7f4637400640 1 --2- 172.21.15.83:0/1684758137 >> [v2:172.21.15.117:6838/1967595704,v1:172.21.15.117:6839/1967595704] conn(0x55b13b94ed30 0x55b13b9511c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:37:13.897 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:37:13.896+0000 7f4621600640 10 client.15471 ms_handle_connect on v2:172.21.15.117:6838/1967595704 2024-08-19T01:37:13.897 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:37:13.896+0000 7f4637400640 1 --2- 172.21.15.83:0/1684758137 >> [v2:172.21.15.117:6838/1967595704,v1:172.21.15.117:6839/1967595704] conn(0x55b13b94ed30 0x55b13b9511c0 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=82e20129fbef25f0 server_cookie=5667a27b021b47c8 in_seq=0 out_seq=0 2024-08-19T01:37:13.903 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:37:13.901+0000 7f4621600640 1 -- 172.21.15.83:0/1684758137 <== mds.0 v2:172.21.15.117:6838/1967595704 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x7f462409f370 con 0x55b13b94ed30 2024-08-19T01:37:13.903 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:37:13.901+0000 7f4621600640 10 client.15471 handle_command_reply: tid=0 2024-08-19T01:37:13.903 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:37:13.901+0000 7f4641491740 1 client.15471 shutdown 2024-08-19T01:37:13.903 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:37:13.901+0000 7f4641491740 1 -- 172.21.15.83:0/1684758137 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b13b92de50 msgr2=0x55b13b94ad20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:37:13.903 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:37:13.901+0000 7f4641491740 1 --2- 172.21.15.83:0/1684758137 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b13b92de50 0x55b13b94ad20 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f46240cc260 tx=0x7f462409a640 comp rx=0 tx=0).stop 2024-08-19T01:37:13.904 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:37:13.902+0000 7f4641491740 1 -- 172.21.15.83:0/1684758137 shutdown_connections 2024-08-19T01:37:13.904 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:37:13.902+0000 7f4641491740 1 --2- 172.21.15.83:0/1684758137 >> [v2:172.21.15.117:6838/1967595704,v1:172.21.15.117:6839/1967595704] conn(0x55b13b94ed30 0x55b13b9511c0 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:37:13.904 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:37:13.902+0000 7f4641491740 1 --2- 172.21.15.83:0/1684758137 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b13b92de50 0x55b13b94ad20 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:37:13.905 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:37:13.902+0000 7f4641491740 1 --2- 172.21.15.83:0/1684758137 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b13b928dc0 0x55b13b943d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:37:13.905 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:37:13.902+0000 7f4641491740 1 --2- 172.21.15.83:0/1684758137 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b13b928480 0x55b13b943810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:37:13.905 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:37:13.902+0000 7f4641491740 1 -- 172.21.15.83:0/1684758137 >> 172.21.15.83:0/1684758137 conn(0x55b13b923af0 msgr2=0x55b13b924ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:37:13.905 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:37:13.902+0000 7f4621600640 0 client.15471 ms_handle_reset on v2:172.21.15.117:6838/1967595704 2024-08-19T01:37:13.905 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:37:13.903+0000 7f4641491740 1 -- 172.21.15.83:0/1684758137 shutdown_connections 2024-08-19T01:37:13.905 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:37:13.903+0000 7f4641491740 1 -- 172.21.15.83:0/1684758137 wait complete. 2024-08-19T01:37:13.907 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:37:13.903+0000 7f4641491740 20 client.15471 trim_cache size 0 max 16384 2024-08-19T01:37:13.925 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:37:13.924+0000 7f4641491740 1 Processor -- start 2024-08-19T01:37:13.926 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:37:13.924+0000 7f4641491740 1 -- start start 2024-08-19T01:37:13.926 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:37:13.924+0000 7f4641491740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b13b928480 0x55b13b943cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:37:13.926 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:37:13.924+0000 7f4641491740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b13b928dc0 0x55b13b94ace0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:37:13.926 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:37:13.924+0000 7f4641491740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b13b92de50 0x55b13b923ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:37:13.926 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:37:13.924+0000 7f4641491740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f46381b1e10 con 0x55b13b92de50 2024-08-19T01:37:13.926 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:37:13.924+0000 7f4641491740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f46381af6e0 con 0x55b13b928480 2024-08-19T01:37:13.926 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:37:13.924+0000 7f4641491740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f46381a20e0 con 0x55b13b928dc0 2024-08-19T01:37:13.926 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:37:13.924+0000 7f4637400640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b13b928480 0x55b13b943cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:37:13.928 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:37:13.924+0000 7f4637e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b13b92de50 0x55b13b923ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:37:13.928 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:37:13.925+0000 7f4636a00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b13b928dc0 0x55b13b94ace0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:37:13.928 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:37:13.925+0000 7f4637400640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b13b928480 0x55b13b943cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.83:48920/0 (socket says 172.21.15.83:48920) 2024-08-19T01:37:13.928 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:37:13.925+0000 7f4637400640 1 -- 172.21.15.83:0/626267312 learned_addr learned my addr 172.21.15.83:0/626267312 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-08-19T01:37:13.928 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:37:13.925+0000 7f4637400640 1 -- 172.21.15.83:0/626267312 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b13b928dc0 msgr2=0x55b13b94ace0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:37:13.928 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:37:13.925+0000 7f4637400640 1 --2- 172.21.15.83:0/626267312 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b13b928dc0 0x55b13b94ace0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:37:13.928 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:37:13.925+0000 7f4637400640 1 -- 172.21.15.83:0/626267312 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b13b92de50 msgr2=0x55b13b923ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:37:13.928 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:37:13.925+0000 7f4637400640 1 --2- 172.21.15.83:0/626267312 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b13b92de50 0x55b13b923ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:37:13.928 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:37:13.925+0000 7f4637400640 1 -- 172.21.15.83:0/626267312 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b13b9448f0 con 0x55b13b928480 2024-08-19T01:37:13.928 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:37:13.925+0000 7f4637e00640 1 --2- 172.21.15.83:0/626267312 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b13b92de50 0x55b13b923ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-19T01:37:13.928 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:37:13.925+0000 7f4637400640 1 --2- 172.21.15.83:0/626267312 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b13b928480 0x55b13b943cd0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f4630000f80 tx=0x7f4630007700 comp rx=0 tx=0).ready entity=mon.2 client_cookie=934b1d17039b9047 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:37:13.928 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:37:13.925+0000 7f4623e00640 1 -- 172.21.15.83:0/626267312 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f463001b250 con 0x55b13b928480 2024-08-19T01:37:13.929 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:37:13.925+0000 7f4623e00640 1 -- 172.21.15.83:0/626267312 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f463000c040 con 0x55b13b928480 2024-08-19T01:37:13.929 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:37:13.926+0000 7f4623e00640 1 -- 172.21.15.83:0/626267312 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46300139a0 con 0x55b13b928480 2024-08-19T01:37:13.929 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:37:13.926+0000 7f4641491740 1 -- 172.21.15.83:0/626267312 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b13b928480 msgr2=0x55b13b943cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:37:13.929 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:37:13.926+0000 7f4641491740 1 --2- 172.21.15.83:0/626267312 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b13b928480 0x55b13b943cd0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f4630000f80 tx=0x7f4630007700 comp rx=0 tx=0).stop 2024-08-19T01:37:13.929 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:37:13.926+0000 7f4641491740 1 -- 172.21.15.83:0/626267312 shutdown_connections 2024-08-19T01:37:13.929 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:37:13.926+0000 7f4641491740 1 --2- 172.21.15.83:0/626267312 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b13b92de50 0x55b13b923ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:37:13.929 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:37:13.926+0000 7f4641491740 1 --2- 172.21.15.83:0/626267312 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b13b928dc0 0x55b13b94ace0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:37:13.929 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:37:13.926+0000 7f4641491740 1 --2- 172.21.15.83:0/626267312 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b13b928480 0x55b13b943cd0 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:37:13.929 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:37:13.926+0000 7f4641491740 1 -- 172.21.15.83:0/626267312 >> 172.21.15.83:0/626267312 conn(0x55b13b923af0 msgr2=0x55b13b9435c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:37:13.929 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:37:13.926+0000 7f4641491740 1 -- 172.21.15.83:0/626267312 shutdown_connections 2024-08-19T01:37:13.932 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:37:13.926+0000 7f4641491740 1 -- 172.21.15.83:0/626267312 wait complete. 2024-08-19T01:37:13.932 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:37:13.927+0000 7f4641491740 1 -- start start 2024-08-19T01:37:13.932 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:37:13.927+0000 7f4641491740 1 Processor -- start 2024-08-19T01:37:13.932 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:37:13.927+0000 7f4641491740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b13b928480 0x55b13b947ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:37:13.932 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:37:13.927+0000 7f4641491740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b13b928dc0 0x55b13b948420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:37:13.932 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:37:13.927+0000 7f4641491740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b13b92de50 0x55b13b949430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:37:13.932 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:37:13.927+0000 7f4641491740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f46381a2750 con 0x55b13b928dc0 2024-08-19T01:37:13.932 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:37:13.927+0000 7f4641491740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f46381af170 con 0x55b13b928480 2024-08-19T01:37:13.932 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:37:13.927+0000 7f4641491740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f46381a2e20 con 0x55b13b92de50 2024-08-19T01:37:13.932 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:37:13.927+0000 7f4641491740 10 client.0 _ll_register_callbacks cb 0x7f46240b2070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-19T01:37:13.933 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:37:13.928+0000 7f4636a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b13b928dc0 0x55b13b948420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:37:13.933 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:37:13.928+0000 7f4636a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b13b928dc0 0x55b13b948420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:35824/0 (socket says 172.21.15.83:35824) 2024-08-19T01:37:13.933 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:37:13.928+0000 7f4636a00640 1 -- 172.21.15.83:0/662223998 learned_addr learned my addr 172.21.15.83:0/662223998 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-08-19T01:37:13.933 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:37:13.928+0000 7f4637e00640 1 --2- 172.21.15.83:0/662223998 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b13b92de50 0x55b13b949430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:37:13.933 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:37:13.928+0000 7f4637400640 1 --2- 172.21.15.83:0/662223998 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b13b928480 0x55b13b947ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:37:13.933 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:37:13.928+0000 7f4637e00640 1 -- 172.21.15.83:0/662223998 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b13b928480 msgr2=0x55b13b947ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:37:13.933 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:37:13.928+0000 7f4637e00640 1 --2- 172.21.15.83:0/662223998 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b13b928480 0x55b13b947ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:37:13.933 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:37:13.928+0000 7f4637e00640 1 -- 172.21.15.83:0/662223998 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b13b928dc0 msgr2=0x55b13b948420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:37:13.933 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:37:13.928+0000 7f4637e00640 1 --2- 172.21.15.83:0/662223998 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b13b928dc0 0x55b13b948420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:37:13.933 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:37:13.928+0000 7f4637e00640 1 -- 172.21.15.83:0/662223998 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b13b949a40 con 0x55b13b92de50 2024-08-19T01:37:13.933 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:37:13.928+0000 7f4637400640 1 --2- 172.21.15.83:0/662223998 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b13b928480 0x55b13b947ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-19T01:37:13.933 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:37:13.928+0000 7f4636a00640 1 --2- 172.21.15.83:0/662223998 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b13b928dc0 0x55b13b948420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-19T01:37:13.933 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:37:13.929+0000 7f4637e00640 1 --2- 172.21.15.83:0/662223998 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b13b92de50 0x55b13b949430 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f462409af50 tx=0x7f4624010380 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d3014137a2bfa978 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:37:13.933 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:37:13.929+0000 7f4621600640 10 client.0 ms_handle_connect on v2:172.21.15.167:3300/0 2024-08-19T01:37:13.933 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:37:13.929+0000 7f4621600640 1 -- 172.21.15.83:0/662223998 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46240a8070 con 0x55b13b92de50 2024-08-19T01:37:13.934 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:37:13.929+0000 7f4621600640 1 -- 172.21.15.83:0/662223998 <== mon.1 v2:172.21.15.167:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8269+0+0 (secure 0 0 0) 0x7f4624095b30 con 0x55b13b92de50 2024-08-19T01:37:13.934 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:37:13.929+0000 7f4641491740 1 -- 172.21.15.83:0/662223998 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f463000c040 con 0x55b13b92de50 2024-08-19T01:37:13.934 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:37:13.929+0000 7f4621600640 1 -- 172.21.15.83:0/662223998 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f462407f070 con 0x55b13b92de50 2024-08-19T01:37:13.934 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:37:13.929+0000 7f4641491740 10 client.25519 fetch_fsmap learned FSMap version 26 2024-08-19T01:37:13.934 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:37:13.929+0000 7f4641491740 1 -- 172.21.15.83:0/662223998 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55b13b94ad50 con 0x55b13b92de50 2024-08-19T01:37:13.934 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:37:13.929+0000 7f4621600640 1 -- 172.21.15.83:0/662223998 <== mon.1 v2:172.21.15.167:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f463000c040 con 0x55b13b92de50 2024-08-19T01:37:13.934 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:37:13.929+0000 7f4621600640 1 -- 172.21.15.83:0/662223998 <== mon.1 v2:172.21.15.167:3300/0 5 ==== fsmap(e 26) ==== 8350+0+0 (secure 0 0 0) 0x7f462409d070 con 0x55b13b92de50 2024-08-19T01:37:13.934 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:37:13.930+0000 7f4641491740 10 client.25519 fetch_fsmap finished waiting for FSMap version 26 2024-08-19T01:37:13.934 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:37:13.930+0000 7f4641491740 10 client.25519 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-08-19T01:37:13.934 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:37:13.930+0000 7f4641491740 20 client.25519 populate_metadata read hostname 'smithi083' 2024-08-19T01:37:13.934 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:37:13.930+0000 7f4641491740 1 --2- 172.21.15.83:0/662223998 >> [v2:172.21.15.117:6838/1967595704,v1:172.21.15.117:6839/1967595704] conn(0x55b13b9b9cb0 0x55b13b9bc0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:37:13.934 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:37:13.930+0000 7f4641491740 4 client.25519 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-08-19T01:37:13.934 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:37:13.930+0000 7f4641491740 1 -- 172.21.15.83:0/662223998 --> [v2:172.21.15.117:6838/1967595704,v1:172.21.15.117:6839/1967595704] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55b13b899fc0 con 0x55b13b9b9cb0 2024-08-19T01:37:13.935 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:37:13.930+0000 7f4637400640 1 --2- 172.21.15.83:0/662223998 >> [v2:172.21.15.117:6838/1967595704,v1:172.21.15.117:6839/1967595704] conn(0x55b13b9b9cb0 0x55b13b9bc0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:37:13.935 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:37:13.930+0000 7f4637400640 1 --2- 172.21.15.83:0/662223998 >> [v2:172.21.15.117:6838/1967595704,v1:172.21.15.117:6839/1967595704] conn(0x55b13b9b9cb0 0x55b13b9bc0a0 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dc3f5634136f3b16 server_cookie=27a96eaa2a9ba71f in_seq=0 out_seq=0 2024-08-19T01:37:13.935 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:37:13.930+0000 7f4621600640 10 client.25519 ms_handle_connect on v2:172.21.15.117:6838/1967595704 2024-08-19T01:37:13.952 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:37:13 smithi117 ceph-mon[22016]: pgmap v410: 97 pgs: 97 active+clean; 122 MiB data, 798 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.1 MiB/s wr, 273 op/s 2024-08-19T01:37:13.966 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:37:13 smithi167 ceph-mon[21875]: pgmap v410: 97 pgs: 97 active+clean; 122 MiB data, 798 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.1 MiB/s wr, 273 op/s 2024-08-19T01:37:15.014 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833 shell --fsid bad8780e-5dc9-11ef-bccf-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-08-19T01:37:15.084 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:37:14 smithi083 ceph-mon[17120]: pgmap v411: 97 pgs: 97 active+clean; 128 MiB data, 822 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.4 MiB/s wr, 250 op/s 2024-08-19T01:37:15.161 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:37:15.160+0000 7f4621600640 1 -- 172.21.15.83:0/662223998 <== mds.0 v2:172.21.15.117:6838/1967595704 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55b13b899fc0 con 0x55b13b9b9cb0 2024-08-19T01:37:15.161 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:37:15.160+0000 7f4621600640 10 client.25519 handle_command_reply: tid=0 2024-08-19T01:37:15.161 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:37:15.160+0000 7f4641491740 1 client.25519 shutdown 2024-08-19T01:37:15.162 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:37:15.160+0000 7f4641491740 1 -- 172.21.15.83:0/662223998 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b13b92de50 msgr2=0x55b13b949430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:37:15.162 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:37:15.160+0000 7f4641491740 1 --2- 172.21.15.83:0/662223998 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b13b92de50 0x55b13b949430 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f462409af50 tx=0x7f4624010380 comp rx=0 tx=0).stop 2024-08-19T01:37:15.163 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:37:15.161+0000 7f4641491740 1 -- 172.21.15.83:0/662223998 shutdown_connections 2024-08-19T01:37:15.163 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:37:15.161+0000 7f4641491740 1 --2- 172.21.15.83:0/662223998 >> [v2:172.21.15.117:6838/1967595704,v1:172.21.15.117:6839/1967595704] conn(0x55b13b9b9cb0 0x55b13b9bc0a0 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:37:15.163 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:37:15.161+0000 7f4641491740 1 --2- 172.21.15.83:0/662223998 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b13b92de50 0x55b13b949430 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:37:15.163 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:37:15.162+0000 7f4621600640 0 client.25519 ms_handle_reset on v2:172.21.15.117:6838/1967595704 2024-08-19T01:37:15.163 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:37:15.162+0000 7f4641491740 1 --2- 172.21.15.83:0/662223998 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b13b928dc0 0x55b13b948420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:37:15.163 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:37:15.162+0000 7f4641491740 1 --2- 172.21.15.83:0/662223998 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b13b928480 0x55b13b947ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:37:15.163 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:37:15.162+0000 7f4641491740 1 -- 172.21.15.83:0/662223998 >> 172.21.15.83:0/662223998 conn(0x55b13b923af0 msgr2=0x55b13b922720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:37:15.164 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:37:15.162+0000 7f4641491740 1 -- 172.21.15.83:0/662223998 shutdown_connections 2024-08-19T01:37:15.164 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:37:15.162+0000 7f4641491740 1 -- 172.21.15.83:0/662223998 wait complete. 2024-08-19T01:37:15.164 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:37:15.162+0000 7f4641491740 20 client.25519 trim_cache size 0 max 16384 2024-08-19T01:37:15.164 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stdout:{ 2024-08-19T01:37:15.164 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stdout: "message": "", 2024-08-19T01:37:15.164 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stdout: "return_code": 0 2024-08-19T01:37:15.165 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stdout:} 2024-08-19T01:37:15.165 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:37:15.163+0000 7f463e600640 1 -- 172.21.15.83:0/2235227491 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f461c0a2e20 msgr2=0x7f461c0a52e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:37:15.165 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:37:15.163+0000 7f463e600640 1 --2- 172.21.15.83:0/2235227491 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f461c0a2e20 0x7f461c0a52e0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f46381a7fc0 tx=0x7f4628000bc0 comp rx=0 tx=0).stop 2024-08-19T01:37:15.165 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:37:15.163+0000 7f463e600640 1 -- 172.21.15.83:0/2235227491 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4638109660 msgr2=0x7f46381ae060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:37:15.166 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:37:15.163+0000 7f463e600640 1 --2- 172.21.15.83:0/2235227491 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4638109660 0x7f46381ae060 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f462400b410 tx=0x7f462400b8d0 comp rx=0 tx=0).stop 2024-08-19T01:37:15.166 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:37:15.163+0000 7f463e600640 1 -- 172.21.15.83:0/2235227491 shutdown_connections 2024-08-19T01:37:15.166 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:37:15.163+0000 7f463e600640 1 --2- 172.21.15.83:0/2235227491 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f461c0a2e20 0x7f461c0a52e0 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:37:15.166 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:37:15.163+0000 7f463e600640 1 --2- 172.21.15.83:0/2235227491 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4638109660 0x7f46381ae060 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:37:15.166 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:37:15.163+0000 7f463e600640 1 --2- 172.21.15.83:0/2235227491 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4638073500 0x7f46381a6ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:37:15.166 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:37:15.163+0000 7f463e600640 1 --2- 172.21.15.83:0/2235227491 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4638072a50 0x7f46381a6ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:37:15.166 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:37:15.163+0000 7f463e600640 1 -- 172.21.15.83:0/2235227491 >> 172.21.15.83:0/2235227491 conn(0x7f463806ea10 msgr2=0x7f463810a030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:37:15.166 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:37:15.164+0000 7f463e600640 1 -- 172.21.15.83:0/2235227491 shutdown_connections 2024-08-19T01:37:15.166 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:37:15.164+0000 7f463e600640 1 -- 172.21.15.83:0/2235227491 wait complete. 2024-08-19T01:37:15.202 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:37:14 smithi117 ceph-mon[22016]: pgmap v411: 97 pgs: 97 active+clean; 128 MiB data, 822 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.4 MiB/s wr, 250 op/s 2024-08-19T01:37:15.215 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:37:14 smithi167 ceph-mon[21875]: pgmap v411: 97 pgs: 97 active+clean; 128 MiB data, 822 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.4 MiB/s wr, 250 op/s 2024-08-19T01:37:15.531 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/bad8780e-5dc9-11ef-bccf-c7b262605968/mon.a/config 2024-08-19T01:37:17.231 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:17.153+0000 7ff677000640 1 -- 172.21.15.83:0/1210581327 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff6701078c0 msgr2=0x7ff670107ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:37:17.231 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:17.153+0000 7ff677000640 1 --2- 172.21.15.83:0/1210581327 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff6701078c0 0x7ff670107ca0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7ff660009860 tx=0x7ff66002a340 comp rx=0 tx=0).stop 2024-08-19T01:37:17.232 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:17.153+0000 7ff677000640 1 -- 172.21.15.83:0/1210581327 shutdown_connections 2024-08-19T01:37:17.232 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:17.154+0000 7ff677000640 1 --2- 172.21.15.83:0/1210581327 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff670102260 0x7ff67010e790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:37:17.232 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:17.154+0000 7ff677000640 1 --2- 172.21.15.83:0/1210581327 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff6701018c0 0x7ff670101d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:37:17.232 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:17.154+0000 7ff677000640 1 --2- 172.21.15.83:0/1210581327 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff6701078c0 0x7ff670107ca0 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:37:17.232 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:17.154+0000 7ff677000640 1 -- 172.21.15.83:0/1210581327 >> 172.21.15.83:0/1210581327 conn(0x7ff6700fd5e0 msgr2=0x7ff6700ffa00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:37:17.232 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:17.154+0000 7ff677000640 1 -- 172.21.15.83:0/1210581327 shutdown_connections 2024-08-19T01:37:17.232 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:17.154+0000 7ff677000640 1 -- 172.21.15.83:0/1210581327 wait complete. 2024-08-19T01:37:17.232 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:17.155+0000 7ff677000640 1 Processor -- start 2024-08-19T01:37:17.232 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:17.156+0000 7ff677000640 1 -- start start 2024-08-19T01:37:17.232 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:17.156+0000 7ff677000640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff6701018c0 0x7ff67019a360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:37:17.232 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:17.156+0000 7ff677000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff670102260 0x7ff67019a8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:37:17.232 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:17.156+0000 7ff677000640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff6701078c0 0x7ff67019ec30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:37:17.233 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:17.156+0000 7ff677000640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7ff670113f10 con 0x7ff670102260 2024-08-19T01:37:17.233 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:17.156+0000 7ff677000640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7ff670113d90 con 0x7ff6701018c0 2024-08-19T01:37:17.233 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:17.156+0000 7ff677000640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7ff670114090 con 0x7ff6701078c0 2024-08-19T01:37:17.233 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:17.156+0000 7ff66f400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff670102260 0x7ff67019a8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:37:17.233 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:17.157+0000 7ff66fe00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff6701018c0 0x7ff67019a360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:37:17.233 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:17.157+0000 7ff66f400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff670102260 0x7ff67019a8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:35834/0 (socket says 172.21.15.83:35834) 2024-08-19T01:37:17.233 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:17.157+0000 7ff66f400640 1 -- 172.21.15.83:0/2260551605 learned_addr learned my addr 172.21.15.83:0/2260551605 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-08-19T01:37:17.233 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:17.157+0000 7ff66fe00640 1 -- 172.21.15.83:0/2260551605 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff6701078c0 msgr2=0x7ff67019ec30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:37:17.233 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:17.157+0000 7ff675000640 1 --2- 172.21.15.83:0/2260551605 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff6701078c0 0x7ff67019ec30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:37:17.233 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:17.158+0000 7ff66fe00640 1 --2- 172.21.15.83:0/2260551605 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff6701078c0 0x7ff67019ec30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:37:17.233 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:17.158+0000 7ff66fe00640 1 -- 172.21.15.83:0/2260551605 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff670102260 msgr2=0x7ff67019a8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:37:17.233 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:17.158+0000 7ff66fe00640 1 --2- 172.21.15.83:0/2260551605 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff670102260 0x7ff67019a8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:37:17.233 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:17.158+0000 7ff66fe00640 1 -- 172.21.15.83:0/2260551605 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff67019f310 con 0x7ff6701018c0 2024-08-19T01:37:17.234 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:17.158+0000 7ff675000640 1 --2- 172.21.15.83:0/2260551605 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff6701078c0 0x7ff67019ec30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-19T01:37:17.234 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:17.158+0000 7ff66f400640 1 --2- 172.21.15.83:0/2260551605 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff670102260 0x7ff67019a8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-19T01:37:17.234 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:17.158+0000 7ff66fe00640 1 --2- 172.21.15.83:0/2260551605 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff6701018c0 0x7ff67019a360 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7ff66002a860 tx=0x7ff660033b60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:37:17.234 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:17.158+0000 7ff66cc00640 1 -- 172.21.15.83:0/2260551605 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff660037040 con 0x7ff6701018c0 2024-08-19T01:37:17.234 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:17.158+0000 7ff677000640 1 -- 172.21.15.83:0/2260551605 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff67019f5a0 con 0x7ff6701018c0 2024-08-19T01:37:17.234 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:17.159+0000 7ff66cc00640 1 -- 172.21.15.83:0/2260551605 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff660033d50 con 0x7ff6701018c0 2024-08-19T01:37:17.234 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:17.159+0000 7ff66cc00640 1 -- 172.21.15.83:0/2260551605 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6600039e0 con 0x7ff6701018c0 2024-08-19T01:37:17.234 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:17.160+0000 7ff677000640 1 -- 172.21.15.83:0/2260551605 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff67010efd0 con 0x7ff6701018c0 2024-08-19T01:37:17.234 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:17.163+0000 7ff66cc00640 1 -- 172.21.15.83:0/2260551605 <== mon.2 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7ff660042020 con 0x7ff6701018c0 2024-08-19T01:37:17.234 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:17.164+0000 7ff679bc4740 1 Processor -- start 2024-08-19T01:37:17.234 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:17.164+0000 7ff66cc00640 1 --2- 172.21.15.83:0/2260551605 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7ff6540a2b80 0x7ff6540a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:37:17.234 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:17.164+0000 7ff679bc4740 1 -- start start 2024-08-19T01:37:17.234 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:17.164+0000 7ff66cc00640 1 -- 172.21.15.83:0/2260551605 <== mon.2 v2:172.21.15.117:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8269+0+0 (secure 0 0 0) 0x7ff6600ee4b0 con 0x7ff6701018c0 2024-08-19T01:37:17.235 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:17.164+0000 7ff675000640 1 --2- 172.21.15.83:0/2260551605 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7ff6540a2b80 0x7ff6540a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:37:17.235 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:17.165+0000 7ff679bc4740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ba663e14f0 0x55ba663e18d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:37:17.235 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:17.165+0000 7ff679bc4740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ba663e1e10 0x55ba663ec190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:37:17.235 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:17.165+0000 7ff679bc4740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ba663ec6d0 0x55ba663eeac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:37:17.235 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:17.165+0000 7ff679bc4740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55ba66346bf0 con 0x55ba663e1e10 2024-08-19T01:37:17.235 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:17.165+0000 7ff679bc4740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55ba662deb20 con 0x55ba663ec6d0 2024-08-19T01:37:17.235 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:17.165+0000 7ff679bc4740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55ba66342bb0 con 0x55ba663e14f0 2024-08-19T01:37:17.235 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:17.166+0000 7ff675000640 1 --2- 172.21.15.83:0/2260551605 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7ff6540a2b80 0x7ff6540a5040 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7ff668005fd0 tx=0x7ff6680095c0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:37:17.235 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:17.166+0000 7ff675000640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ba663ec6d0 0x55ba663eeac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:37:17.235 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:17.166+0000 7ff66f400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ba663e1e10 0x55ba663ec190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:37:17.235 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:17.166+0000 7ff675000640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ba663ec6d0 0x55ba663eeac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.83:48958/0 (socket says 172.21.15.83:48958) 2024-08-19T01:37:17.235 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:17.166+0000 7ff675000640 1 -- 172.21.15.83:0/1806968515 learned_addr learned my addr 172.21.15.83:0/1806968515 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-08-19T01:37:17.235 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:17.166+0000 7ff675000640 1 -- 172.21.15.83:0/1806968515 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ba663e14f0 msgr2=0x55ba663e18d0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-08-19T01:37:17.236 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:17.166+0000 7ff675000640 1 --2- 172.21.15.83:0/1806968515 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ba663e14f0 0x55ba663e18d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:37:17.236 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:17.166+0000 7ff675000640 1 -- 172.21.15.83:0/1806968515 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ba663e1e10 msgr2=0x55ba663ec190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:37:17.236 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:17.166+0000 7ff675000640 1 --2- 172.21.15.83:0/1806968515 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ba663e1e10 0x55ba663ec190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:37:17.236 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:17.166+0000 7ff675000640 1 -- 172.21.15.83:0/1806968515 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ba662e57f0 con 0x55ba663ec6d0 2024-08-19T01:37:17.236 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:17.167+0000 7ff675000640 1 --2- 172.21.15.83:0/1806968515 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ba663ec6d0 0x55ba663eeac0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7ff66801b7c0 tx=0x7ff668013000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=650841b3ee03c9e9 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:37:17.236 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:17.170+0000 7ff665600640 1 -- 172.21.15.83:0/1806968515 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff66801b250 con 0x55ba663ec6d0 2024-08-19T01:37:17.236 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:17.170+0000 7ff665600640 1 -- 172.21.15.83:0/1806968515 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff66800ee80 con 0x55ba663ec6d0 2024-08-19T01:37:17.236 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:17.170+0000 7ff665600640 1 -- 172.21.15.83:0/1806968515 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff66801a3f0 con 0x55ba663ec6d0 2024-08-19T01:37:17.236 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:17.170+0000 7ff679bc4740 1 -- 172.21.15.83:0/1806968515 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ba663ec6d0 msgr2=0x55ba663eeac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:37:17.236 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:17.170+0000 7ff679bc4740 1 --2- 172.21.15.83:0/1806968515 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ba663ec6d0 0x55ba663eeac0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7ff66801b7c0 tx=0x7ff668013000 comp rx=0 tx=0).stop 2024-08-19T01:37:17.236 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:17.171+0000 7ff679bc4740 1 -- 172.21.15.83:0/1806968515 shutdown_connections 2024-08-19T01:37:17.236 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:17.171+0000 7ff679bc4740 1 --2- 172.21.15.83:0/1806968515 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ba663ec6d0 0x55ba663eeac0 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:37:17.237 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:17.171+0000 7ff679bc4740 1 --2- 172.21.15.83:0/1806968515 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ba663e1e10 0x55ba663ec190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:37:17.237 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:17.171+0000 7ff679bc4740 1 --2- 172.21.15.83:0/1806968515 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ba663e14f0 0x55ba663e18d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:37:17.237 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:17.171+0000 7ff679bc4740 1 -- 172.21.15.83:0/1806968515 >> 172.21.15.83:0/1806968515 conn(0x55ba663dda80 msgr2=0x55ba663dde60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:37:17.237 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:17.171+0000 7ff679bc4740 1 -- 172.21.15.83:0/1806968515 shutdown_connections 2024-08-19T01:37:17.237 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:17.171+0000 7ff679bc4740 1 -- 172.21.15.83:0/1806968515 wait complete. 2024-08-19T01:37:17.237 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:17.171+0000 7ff679bc4740 1 -- start start 2024-08-19T01:37:17.237 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:17.172+0000 7ff679bc4740 1 Processor -- start 2024-08-19T01:37:17.237 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:17.172+0000 7ff679bc4740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ba663e14f0 0x55ba663e9670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:37:17.237 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:17.172+0000 7ff679bc4740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ba663e1e10 0x55ba663e9bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:37:17.237 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:17.172+0000 7ff679bc4740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ba663ec6d0 0x55ba663ea0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:37:17.237 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:17.172+0000 7ff679bc4740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55ba663ef2a0 con 0x55ba663ec6d0 2024-08-19T01:37:17.237 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:17.172+0000 7ff679bc4740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55ba66301cf0 con 0x55ba663e1e10 2024-08-19T01:37:17.237 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:17.172+0000 7ff679bc4740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55ba663ef420 con 0x55ba663e14f0 2024-08-19T01:37:17.237 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:17.172+0000 7ff679bc4740 10 client.0 _ll_register_callbacks cb 0x55ba662ede30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-19T01:37:17.237 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:17.173+0000 7ff66fe00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ba663e14f0 0x55ba663e9670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:37:17.238 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:17.173+0000 7ff66fe00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ba663e14f0 0x55ba663e9670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.83:60112/0 (socket says 172.21.15.83:60112) 2024-08-19T01:37:17.238 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:17.173+0000 7ff66fe00640 1 -- 172.21.15.83:0/3501936887 learned_addr learned my addr 172.21.15.83:0/3501936887 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-08-19T01:37:17.238 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:17.173+0000 7ff66f400640 1 --2- 172.21.15.83:0/3501936887 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ba663e1e10 0x55ba663e9bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:37:17.238 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:17.173+0000 7ff675000640 1 --2- 172.21.15.83:0/3501936887 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ba663ec6d0 0x55ba663ea0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:37:17.238 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:17.173+0000 7ff66fe00640 1 -- 172.21.15.83:0/3501936887 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ba663e1e10 msgr2=0x55ba663e9bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:37:17.238 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:17.173+0000 7ff66fe00640 1 --2- 172.21.15.83:0/3501936887 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ba663e1e10 0x55ba663e9bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:37:17.238 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:17.173+0000 7ff66fe00640 1 -- 172.21.15.83:0/3501936887 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ba663ec6d0 msgr2=0x55ba663ea0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:37:17.238 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:17.173+0000 7ff66fe00640 1 --2- 172.21.15.83:0/3501936887 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ba663ec6d0 0x55ba663ea0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:37:17.238 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:17.173+0000 7ff66fe00640 1 -- 172.21.15.83:0/3501936887 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ba660b83c0 con 0x55ba663e14f0 2024-08-19T01:37:17.238 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:17.173+0000 7ff66f400640 1 --2- 172.21.15.83:0/3501936887 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ba663e1e10 0x55ba663e9bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-19T01:37:17.238 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:17.174+0000 7ff66fe00640 1 --2- 172.21.15.83:0/3501936887 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ba663e14f0 0x55ba663e9670 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7ff6600b7180 tx=0x7ff6600ba680 comp rx=0 tx=0).ready entity=mon.1 client_cookie=85e818f879e0b63b server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:37:17.238 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:17.174+0000 7ff65a000640 10 client.0 ms_handle_connect on v2:172.21.15.167:3300/0 2024-08-19T01:37:17.238 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:17.174+0000 7ff65a000640 1 -- 172.21.15.83:0/3501936887 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6600bf0d0 con 0x55ba663e14f0 2024-08-19T01:37:17.239 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:17.174+0000 7ff679bc4740 1 -- 172.21.15.83:0/3501936887 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55ba66354160 con 0x55ba663e14f0 2024-08-19T01:37:17.239 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:17.174+0000 7ff65a000640 1 -- 172.21.15.83:0/3501936887 <== mon.1 v2:172.21.15.167:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8269+0+0 (secure 0 0 0) 0x7ff6600ee0d0 con 0x55ba663e14f0 2024-08-19T01:37:17.239 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:17.174+0000 7ff65a000640 1 -- 172.21.15.83:0/3501936887 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7ff6600c3900 con 0x55ba663e14f0 2024-08-19T01:37:17.239 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:17.174+0000 7ff679bc4740 10 client.25525 fetch_fsmap learned FSMap version 26 2024-08-19T01:37:17.239 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:17.174+0000 7ff679bc4740 1 -- 172.21.15.83:0/3501936887 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55ba663ef350 con 0x55ba663e14f0 2024-08-19T01:37:17.239 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:17.175+0000 7ff65a000640 1 -- 172.21.15.83:0/3501936887 <== mon.1 v2:172.21.15.167:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x55ba66354160 con 0x55ba663e14f0 2024-08-19T01:37:17.239 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:17.175+0000 7ff65a000640 1 -- 172.21.15.83:0/3501936887 <== mon.1 v2:172.21.15.167:3300/0 5 ==== fsmap(e 26) ==== 8350+0+0 (secure 0 0 0) 0x7ff66003a070 con 0x55ba663e14f0 2024-08-19T01:37:17.239 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:17.175+0000 7ff679bc4740 10 client.25525 fetch_fsmap finished waiting for FSMap version 26 2024-08-19T01:37:17.239 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:17.175+0000 7ff679bc4740 10 client.25525 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.g 2024-08-19T01:37:17.239 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:17.175+0000 7ff679bc4740 20 client.25525 populate_metadata read hostname 'smithi083' 2024-08-19T01:37:17.239 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:17.175+0000 7ff679bc4740 1 --2- 172.21.15.83:0/3501936887 >> [v2:172.21.15.83:6838/3239261845,v1:172.21.15.83:6839/3239261845] conn(0x55ba663fc400 0x55ba663fe860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:37:17.239 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:17.175+0000 7ff679bc4740 4 client.25525 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-19T01:37:17.239 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:17.175+0000 7ff679bc4740 1 -- 172.21.15.83:0/3501936887 --> [v2:172.21.15.83:6838/3239261845,v1:172.21.15.83:6839/3239261845] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ba6634e670 con 0x55ba663fc400 2024-08-19T01:37:17.239 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:17.176+0000 7ff66f400640 1 --2- 172.21.15.83:0/3501936887 >> [v2:172.21.15.83:6838/3239261845,v1:172.21.15.83:6839/3239261845] conn(0x55ba663fc400 0x55ba663fe860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:37:17.240 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:17.176+0000 7ff66f400640 1 --2- 172.21.15.83:0/3501936887 >> [v2:172.21.15.83:6838/3239261845,v1:172.21.15.83:6839/3239261845] conn(0x55ba663fc400 0x55ba663fe860 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=c03661a763b404ec server_cookie=260a151627b504ad in_seq=0 out_seq=0 2024-08-19T01:37:17.240 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:17.176+0000 7ff65a000640 10 client.25525 ms_handle_connect on v2:172.21.15.83:6838/3239261845 2024-08-19T01:37:17.240 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:17.177+0000 7ff65a000640 1 -- 172.21.15.83:0/3501936887 <== mds.1 v2:172.21.15.83:6838/3239261845 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55ba6634e670 con 0x55ba663fc400 2024-08-19T01:37:17.240 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:17.177+0000 7ff65a000640 10 client.25525 handle_command_reply: tid=0 2024-08-19T01:37:17.240 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:17.177+0000 7ff679bc4740 1 client.25525 shutdown 2024-08-19T01:37:17.240 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:17.178+0000 7ff679bc4740 1 -- 172.21.15.83:0/3501936887 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ba663e14f0 msgr2=0x55ba663e9670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:37:17.240 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:17.178+0000 7ff679bc4740 1 --2- 172.21.15.83:0/3501936887 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ba663e14f0 0x55ba663e9670 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7ff6600b7180 tx=0x7ff6600ba680 comp rx=0 tx=0).stop 2024-08-19T01:37:17.240 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:17.178+0000 7ff679bc4740 1 -- 172.21.15.83:0/3501936887 shutdown_connections 2024-08-19T01:37:17.240 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:17.178+0000 7ff679bc4740 1 --2- 172.21.15.83:0/3501936887 >> [v2:172.21.15.83:6838/3239261845,v1:172.21.15.83:6839/3239261845] conn(0x55ba663fc400 0x55ba663fe860 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:37:17.240 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:17.178+0000 7ff679bc4740 1 --2- 172.21.15.83:0/3501936887 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ba663ec6d0 0x55ba663ea0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:37:17.240 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:17.178+0000 7ff679bc4740 1 --2- 172.21.15.83:0/3501936887 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ba663e1e10 0x55ba663e9bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:37:17.240 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:17.178+0000 7ff679bc4740 1 --2- 172.21.15.83:0/3501936887 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ba663e14f0 0x55ba663e9670 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:37:17.240 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:17.178+0000 7ff679bc4740 1 -- 172.21.15.83:0/3501936887 >> 172.21.15.83:0/3501936887 conn(0x55ba663dda80 msgr2=0x55ba663e37b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:37:17.241 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:17.178+0000 7ff65a000640 0 client.25525 ms_handle_reset on v2:172.21.15.83:6838/3239261845 2024-08-19T01:37:17.241 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:17.178+0000 7ff679bc4740 1 -- 172.21.15.83:0/3501936887 shutdown_connections 2024-08-19T01:37:17.241 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:17.178+0000 7ff679bc4740 1 -- 172.21.15.83:0/3501936887 wait complete. 2024-08-19T01:37:17.241 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:17.179+0000 7ff679bc4740 20 client.25525 trim_cache size 0 max 16384 2024-08-19T01:37:17.241 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:17.205+0000 7ff679bc4740 1 Processor -- start 2024-08-19T01:37:17.241 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:17.205+0000 7ff679bc4740 1 -- start start 2024-08-19T01:37:17.241 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:17.206+0000 7ff679bc4740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ba663e1e10 0x55ba663dcaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:37:17.241 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:17.206+0000 7ff679bc4740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ba663ec6d0 0x55ba66404170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:37:17.241 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:17.206+0000 7ff679bc4740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ba663e8760 0x55ba664046b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:37:17.241 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:17.206+0000 7ff679bc4740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7ff6701b6280 con 0x55ba663e1e10 2024-08-19T01:37:17.241 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:17.206+0000 7ff679bc4740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7ff6701b3b50 con 0x55ba663e8760 2024-08-19T01:37:17.241 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:17.206+0000 7ff679bc4740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7ff6701b19a0 con 0x55ba663ec6d0 2024-08-19T01:37:17.241 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:17.206+0000 7ff66f400640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ba663ec6d0 0x55ba66404170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:37:17.242 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:17.206+0000 7ff675000640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ba663e8760 0x55ba664046b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:37:17.242 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:17.206+0000 7ff66f400640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ba663ec6d0 0x55ba66404170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.83:60118/0 (socket says 172.21.15.83:60118) 2024-08-19T01:37:17.242 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:17.206+0000 7ff66f400640 1 -- 172.21.15.83:0/2689193796 learned_addr learned my addr 172.21.15.83:0/2689193796 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-08-19T01:37:17.242 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:17.206+0000 7ff66fe00640 1 --2- 172.21.15.83:0/2689193796 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ba663e1e10 0x55ba663dcaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:37:17.242 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:17.207+0000 7ff66f400640 1 -- 172.21.15.83:0/2689193796 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ba663e8760 msgr2=0x55ba664046b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:37:17.242 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:17.207+0000 7ff66f400640 1 --2- 172.21.15.83:0/2689193796 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ba663e8760 0x55ba664046b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:37:17.242 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:17.207+0000 7ff66f400640 1 -- 172.21.15.83:0/2689193796 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ba663e1e10 msgr2=0x55ba663dcaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:37:17.242 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:17.207+0000 7ff66f400640 1 --2- 172.21.15.83:0/2689193796 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ba663e1e10 0x55ba663dcaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:37:17.242 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:17.207+0000 7ff66f400640 1 -- 172.21.15.83:0/2689193796 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ba663e1c40 con 0x55ba663ec6d0 2024-08-19T01:37:17.242 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:17.207+0000 7ff66fe00640 1 --2- 172.21.15.83:0/2689193796 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ba663e1e10 0x55ba663dcaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-19T01:37:17.242 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:17.207+0000 7ff66f400640 1 --2- 172.21.15.83:0/2689193796 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ba663ec6d0 0x55ba66404170 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x55ba663de8e0 tx=0x7ff65c005f90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3dc6f6d1abd1915f server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:37:17.242 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:17.207+0000 7ff664c00640 1 -- 172.21.15.83:0/2689193796 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff65c004220 con 0x55ba663ec6d0 2024-08-19T01:37:17.242 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:17.208+0000 7ff664c00640 1 -- 172.21.15.83:0/2689193796 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff65c0043c0 con 0x55ba663ec6d0 2024-08-19T01:37:17.242 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:17.208+0000 7ff664c00640 1 -- 172.21.15.83:0/2689193796 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff65c0047a0 con 0x55ba663ec6d0 2024-08-19T01:37:17.243 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:17.208+0000 7ff679bc4740 1 -- 172.21.15.83:0/2689193796 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ba663ec6d0 msgr2=0x55ba66404170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:37:17.243 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:17.208+0000 7ff679bc4740 1 --2- 172.21.15.83:0/2689193796 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ba663ec6d0 0x55ba66404170 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x55ba663de8e0 tx=0x7ff65c005f90 comp rx=0 tx=0).stop 2024-08-19T01:37:17.243 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:17.209+0000 7ff679bc4740 1 -- 172.21.15.83:0/2689193796 shutdown_connections 2024-08-19T01:37:17.243 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:17.209+0000 7ff679bc4740 1 --2- 172.21.15.83:0/2689193796 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ba663e8760 0x55ba664046b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:37:17.243 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:17.209+0000 7ff679bc4740 1 --2- 172.21.15.83:0/2689193796 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ba663ec6d0 0x55ba66404170 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:37:17.243 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:17.209+0000 7ff679bc4740 1 --2- 172.21.15.83:0/2689193796 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ba663e1e10 0x55ba663dcaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:37:17.243 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:17.209+0000 7ff679bc4740 1 -- 172.21.15.83:0/2689193796 >> 172.21.15.83:0/2689193796 conn(0x55ba663dda80 msgr2=0x55ba663dc3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:37:17.243 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:17.210+0000 7ff679bc4740 1 -- 172.21.15.83:0/2689193796 shutdown_connections 2024-08-19T01:37:17.243 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:17.210+0000 7ff679bc4740 1 -- 172.21.15.83:0/2689193796 wait complete. 2024-08-19T01:37:17.243 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:17.210+0000 7ff679bc4740 1 -- start start 2024-08-19T01:37:17.243 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:17.210+0000 7ff679bc4740 1 Processor -- start 2024-08-19T01:37:17.243 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:17.211+0000 7ff679bc4740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ba663e1e10 0x55ba663f0a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:37:17.243 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:17.211+0000 7ff679bc4740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ba663e8760 0x55ba663f0fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:37:17.243 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:17.211+0000 7ff679bc4740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ba663ec6d0 0x55ba66466140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:37:17.244 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:17.211+0000 7ff679bc4740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7ff6701b2010 con 0x55ba663e1e10 2024-08-19T01:37:17.244 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:17.211+0000 7ff679bc4740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7ff6701b3480 con 0x55ba663ec6d0 2024-08-19T01:37:17.244 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:17.211+0000 7ff679bc4740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7ff6701b26e0 con 0x55ba663e8760 2024-08-19T01:37:17.244 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:17.211+0000 7ff679bc4740 10 client.0 _ll_register_callbacks cb 0x7ff6600bab10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-19T01:37:17.244 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:17.211+0000 7ff675000640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ba663ec6d0 0x55ba66466140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:37:17.244 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:17.211+0000 7ff675000640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ba663ec6d0 0x55ba66466140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.83:48994/0 (socket says 172.21.15.83:48994) 2024-08-19T01:37:17.244 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:17.211+0000 7ff675000640 1 -- 172.21.15.83:0/2491626091 learned_addr learned my addr 172.21.15.83:0/2491626091 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-08-19T01:37:17.244 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:17.212+0000 7ff66fe00640 1 --2- 172.21.15.83:0/2491626091 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ba663e1e10 0x55ba663f0a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:37:17.244 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:17.212+0000 7ff66f400640 1 --2- 172.21.15.83:0/2491626091 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ba663e8760 0x55ba663f0fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:37:17.244 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:17.212+0000 7ff675000640 1 -- 172.21.15.83:0/2491626091 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ba663e8760 msgr2=0x55ba663f0fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:37:17.244 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:17.212+0000 7ff675000640 1 --2- 172.21.15.83:0/2491626091 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ba663e8760 0x55ba663f0fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:37:17.244 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:17.212+0000 7ff675000640 1 -- 172.21.15.83:0/2491626091 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ba663e1e10 msgr2=0x55ba663f0a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:37:17.244 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:17.212+0000 7ff675000640 1 --2- 172.21.15.83:0/2491626091 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ba663e1e10 0x55ba663f0a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:37:17.244 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:17.212+0000 7ff675000640 1 -- 172.21.15.83:0/2491626091 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ba66466680 con 0x55ba663ec6d0 2024-08-19T01:37:17.245 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:17.212+0000 7ff66fe00640 1 --2- 172.21.15.83:0/2491626091 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ba663e1e10 0x55ba663f0a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-19T01:37:17.245 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:17.212+0000 7ff675000640 1 --2- 172.21.15.83:0/2491626091 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ba663ec6d0 0x55ba66466140 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7ff66801b7c0 tx=0x7ff66801b1b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=57fd512e0fc56226 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:37:17.245 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:17.212+0000 7ff65a000640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-08-19T01:37:17.245 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:17.213+0000 7ff65a000640 1 -- 172.21.15.83:0/2491626091 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff668003970 con 0x55ba663ec6d0 2024-08-19T01:37:17.245 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:17.213+0000 7ff65a000640 1 -- 172.21.15.83:0/2491626091 <== mon.2 v2:172.21.15.117:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8269+0+0 (secure 0 0 0) 0x7ff668003b10 con 0x55ba663ec6d0 2024-08-19T01:37:17.245 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:17.213+0000 7ff679bc4740 1 -- 172.21.15.83:0/2491626091 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff65c0043c0 con 0x55ba663ec6d0 2024-08-19T01:37:17.245 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:17.213+0000 7ff65a000640 1 -- 172.21.15.83:0/2491626091 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7ff66803d070 con 0x55ba663ec6d0 2024-08-19T01:37:17.245 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:17.213+0000 7ff679bc4740 10 client.25565 fetch_fsmap learned FSMap version 26 2024-08-19T01:37:17.245 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:17.213+0000 7ff679bc4740 1 -- 172.21.15.83:0/2491626091 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55ba66467890 con 0x55ba663ec6d0 2024-08-19T01:37:17.245 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:17.213+0000 7ff65a000640 1 -- 172.21.15.83:0/2491626091 <== mon.2 v2:172.21.15.117:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff65c0043c0 con 0x55ba663ec6d0 2024-08-19T01:37:17.245 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:17.214+0000 7ff65a000640 1 -- 172.21.15.83:0/2491626091 <== mon.2 v2:172.21.15.117:3300/0 5 ==== fsmap(e 26) ==== 8350+0+0 (secure 0 0 0) 0x7ff668022070 con 0x55ba663ec6d0 2024-08-19T01:37:17.245 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:17.214+0000 7ff679bc4740 10 client.25565 fetch_fsmap finished waiting for FSMap version 26 2024-08-19T01:37:17.245 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:17.214+0000 7ff679bc4740 10 client.25565 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.g 2024-08-19T01:37:17.245 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:17.214+0000 7ff679bc4740 20 client.25565 populate_metadata read hostname 'smithi083' 2024-08-19T01:37:17.245 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:17.214+0000 7ff679bc4740 1 --2- 172.21.15.83:0/2491626091 >> [v2:172.21.15.83:6838/3239261845,v1:172.21.15.83:6839/3239261845] conn(0x55ba66467c80 0x55ba6646a070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:37:17.246 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:17.214+0000 7ff679bc4740 4 client.25565 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-08-19T01:37:17.246 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:17.214+0000 7ff679bc4740 1 -- 172.21.15.83:0/2491626091 --> [v2:172.21.15.83:6838/3239261845,v1:172.21.15.83:6839/3239261845] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55ba662cb010 con 0x55ba66467c80 2024-08-19T01:37:17.246 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:17.214+0000 7ff66fe00640 1 --2- 172.21.15.83:0/2491626091 >> [v2:172.21.15.83:6838/3239261845,v1:172.21.15.83:6839/3239261845] conn(0x55ba66467c80 0x55ba6646a070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:37:17.246 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:17.215+0000 7ff66fe00640 1 --2- 172.21.15.83:0/2491626091 >> [v2:172.21.15.83:6838/3239261845,v1:172.21.15.83:6839/3239261845] conn(0x55ba66467c80 0x55ba6646a070 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=73d8b6fce24a1b35 server_cookie=d4e67a11fb19ea13 in_seq=0 out_seq=0 2024-08-19T01:37:17.246 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:17.215+0000 7ff65a000640 10 client.25565 ms_handle_connect on v2:172.21.15.83:6838/3239261845 2024-08-19T01:37:17.246 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:17.215+0000 7ff65a000640 1 -- 172.21.15.83:0/2491626091 <== mds.1 v2:172.21.15.83:6838/3239261845 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55ba662cb010 con 0x55ba66467c80 2024-08-19T01:37:17.246 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:17.215+0000 7ff65a000640 10 client.25565 handle_command_reply: tid=0 2024-08-19T01:37:17.246 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:17.215+0000 7ff679bc4740 1 client.25565 shutdown 2024-08-19T01:37:17.246 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:17.215+0000 7ff679bc4740 1 -- 172.21.15.83:0/2491626091 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ba663ec6d0 msgr2=0x55ba66466140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:37:17.246 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:17.215+0000 7ff679bc4740 1 --2- 172.21.15.83:0/2491626091 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ba663ec6d0 0x55ba66466140 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7ff66801b7c0 tx=0x7ff66801b1b0 comp rx=0 tx=0).stop 2024-08-19T01:37:17.246 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:17.215+0000 7ff679bc4740 1 -- 172.21.15.83:0/2491626091 shutdown_connections 2024-08-19T01:37:17.246 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:17.215+0000 7ff679bc4740 1 --2- 172.21.15.83:0/2491626091 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ba663ec6d0 0x55ba66466140 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:37:17.246 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:17.215+0000 7ff679bc4740 1 --2- 172.21.15.83:0/2491626091 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ba663e8760 0x55ba663f0fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:37:17.247 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:17.215+0000 7ff679bc4740 1 --2- 172.21.15.83:0/2491626091 >> [v2:172.21.15.83:6838/3239261845,v1:172.21.15.83:6839/3239261845] conn(0x55ba66467c80 0x55ba6646a070 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:37:17.247 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:17.215+0000 7ff679bc4740 1 --2- 172.21.15.83:0/2491626091 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ba663e1e10 0x55ba663f0a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:37:17.247 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:17.215+0000 7ff679bc4740 1 -- 172.21.15.83:0/2491626091 >> 172.21.15.83:0/2491626091 conn(0x55ba663dda80 msgr2=0x55ba663e3760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:37:17.247 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:17.215+0000 7ff65a000640 0 client.25565 ms_handle_reset on v2:172.21.15.83:6838/3239261845 2024-08-19T01:37:17.247 INFO:teuthology.orchestra.run.smithi083.stdout:[] 2024-08-19T01:37:17.247 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:17.216+0000 7ff679bc4740 1 -- 172.21.15.83:0/2491626091 shutdown_connections 2024-08-19T01:37:17.247 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:17.216+0000 7ff679bc4740 1 -- 172.21.15.83:0/2491626091 wait complete. 2024-08-19T01:37:17.248 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:17.217+0000 7ff679bc4740 20 client.25565 trim_cache size 0 max 16384 2024-08-19T01:37:17.248 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:17.217+0000 7ff677000640 1 -- 172.21.15.83:0/2260551605 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7ff6540a2b80 msgr2=0x7ff6540a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:37:17.248 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:17.217+0000 7ff677000640 1 --2- 172.21.15.83:0/2260551605 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7ff6540a2b80 0x7ff6540a5040 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7ff668005fd0 tx=0x7ff6680095c0 comp rx=0 tx=0).stop 2024-08-19T01:37:17.248 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:17.217+0000 7ff677000640 1 -- 172.21.15.83:0/2260551605 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff6701018c0 msgr2=0x7ff67019a360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:37:17.248 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:17.217+0000 7ff677000640 1 --2- 172.21.15.83:0/2260551605 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff6701018c0 0x7ff67019a360 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7ff66002a860 tx=0x7ff660033b60 comp rx=0 tx=0).stop 2024-08-19T01:37:17.248 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:17.218+0000 7ff677000640 1 -- 172.21.15.83:0/2260551605 shutdown_connections 2024-08-19T01:37:17.248 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:17.218+0000 7ff677000640 1 --2- 172.21.15.83:0/2260551605 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7ff6540a2b80 0x7ff6540a5040 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:37:17.248 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:17.218+0000 7ff677000640 1 --2- 172.21.15.83:0/2260551605 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff6701078c0 0x7ff67019ec30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:37:17.248 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:17.218+0000 7ff677000640 1 --2- 172.21.15.83:0/2260551605 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff670102260 0x7ff67019a8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:37:17.248 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:17.218+0000 7ff677000640 1 --2- 172.21.15.83:0/2260551605 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff6701018c0 0x7ff67019a360 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:37:17.248 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:17.218+0000 7ff677000640 1 -- 172.21.15.83:0/2260551605 >> 172.21.15.83:0/2260551605 conn(0x7ff6700fd5e0 msgr2=0x7ff6700fede0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:37:17.248 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:17.218+0000 7ff677000640 1 -- 172.21.15.83:0/2260551605 shutdown_connections 2024-08-19T01:37:17.248 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:17.218+0000 7ff677000640 1 -- 172.21.15.83:0/2260551605 wait complete. 2024-08-19T01:37:17.800 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:37:17 smithi083 ceph-mon[17120]: pgmap v412: 97 pgs: 97 active+clean; 135 MiB data, 843 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.6 MiB/s wr, 339 op/s 2024-08-19T01:37:17.952 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:37:17 smithi117 ceph-mon[22016]: pgmap v412: 97 pgs: 97 active+clean; 135 MiB data, 843 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.6 MiB/s wr, 339 op/s 2024-08-19T01:37:17.966 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:37:17 smithi167 ceph-mon[21875]: pgmap v412: 97 pgs: 97 active+clean; 135 MiB data, 843 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.6 MiB/s wr, 339 op/s 2024-08-19T01:37:19.594 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:37:19 smithi083 ceph-mon[17120]: pgmap v413: 97 pgs: 97 active+clean; 136 MiB data, 874 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 3.7 MiB/s wr, 965 op/s 2024-08-19T01:37:19.656 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833 shell --fsid bad8780e-5dc9-11ef-bccf-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-08-19T01:37:19.767 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:37:19 smithi167 ceph-mon[21875]: pgmap v413: 97 pgs: 97 active+clean; 136 MiB data, 874 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 3.7 MiB/s wr, 965 op/s 2024-08-19T01:37:19.952 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:37:19 smithi117 ceph-mon[22016]: pgmap v413: 97 pgs: 97 active+clean; 136 MiB data, 874 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 3.7 MiB/s wr, 965 op/s 2024-08-19T01:37:20.554 INFO:tasks.workunit.client.0.smithi083.stdout: 60 362637 167 238348 275 223385 771 2024-08-19T01:37:21.683 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:37:21 smithi083 ceph-mon[17120]: pgmap v414: 97 pgs: 97 active+clean; 123 MiB data, 872 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 4.1 MiB/s wr, 1.21k op/s 2024-08-19T01:37:21.952 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:37:21 smithi117 ceph-mon[22016]: pgmap v414: 97 pgs: 97 active+clean; 123 MiB data, 872 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 4.1 MiB/s wr, 1.21k op/s 2024-08-19T01:37:21.966 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:37:21 smithi167 ceph-mon[21875]: pgmap v414: 97 pgs: 97 active+clean; 123 MiB data, 872 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 4.1 MiB/s wr, 1.21k op/s 2024-08-19T01:37:23.229 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/bad8780e-5dc9-11ef-bccf-c7b262605968/mon.a/config 2024-08-19T01:37:23.604 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:37:23 smithi083 ceph-mon[17120]: pgmap v415: 97 pgs: 97 active+clean; 133 MiB data, 915 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 4.5 MiB/s wr, 1.26k op/s 2024-08-19T01:37:23.952 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:37:23 smithi117 ceph-mon[22016]: pgmap v415: 97 pgs: 97 active+clean; 133 MiB data, 915 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 4.5 MiB/s wr, 1.26k op/s 2024-08-19T01:37:23.966 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:37:23 smithi167 ceph-mon[21875]: pgmap v415: 97 pgs: 97 active+clean; 133 MiB data, 915 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 4.5 MiB/s wr, 1.26k op/s 2024-08-19T01:37:24.634 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:24.626+0000 7f5deb200640 1 -- 172.21.15.83:0/653390540 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5dec102090 msgr2=0x7f5dec10e5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:37:24.634 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:24.631+0000 7f5deb200640 1 --2- 172.21.15.83:0/653390540 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5dec102090 0x7f5dec10e5c0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f5dd8009890 tx=0x7f5dd802a370 comp rx=0 tx=0).stop 2024-08-19T01:37:24.634 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:24.631+0000 7f5deb200640 1 -- 172.21.15.83:0/653390540 shutdown_connections 2024-08-19T01:37:24.634 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:24.631+0000 7f5deb200640 1 --2- 172.21.15.83:0/653390540 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5dec102090 0x7f5dec10e5c0 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:37:24.634 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:24.631+0000 7f5deb200640 1 --2- 172.21.15.83:0/653390540 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5dec1016f0 0x7f5dec101b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:37:24.634 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:24.633+0000 7f5deb200640 1 --2- 172.21.15.83:0/653390540 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5dec1076f0 0x7f5dec107ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:37:24.634 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:24.633+0000 7f5deb200640 1 -- 172.21.15.83:0/653390540 >> 172.21.15.83:0/653390540 conn(0x7f5dec0fd470 msgr2=0x7f5dec0ff890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:37:24.634 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:24.633+0000 7f5deb200640 1 -- 172.21.15.83:0/653390540 shutdown_connections 2024-08-19T01:37:24.637 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:24.633+0000 7f5deb200640 1 -- 172.21.15.83:0/653390540 wait complete. 2024-08-19T01:37:24.664 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:24.655+0000 7f5deb200640 1 Processor -- start 2024-08-19T01:37:24.664 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:24.656+0000 7f5deb200640 1 -- start start 2024-08-19T01:37:24.664 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:24.656+0000 7f5deb200640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5dec1016f0 0x7f5dec074220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:37:24.664 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:24.656+0000 7f5deb200640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5dec102090 0x7f5dec074760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:37:24.664 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:24.656+0000 7f5deb200640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5dec1076f0 0x7f5dec074ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:37:24.665 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:24.656+0000 7f5deb200640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f5dec111b10 con 0x7f5dec102090 2024-08-19T01:37:24.665 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:24.656+0000 7f5deb200640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f5dec111990 con 0x7f5dec1076f0 2024-08-19T01:37:24.665 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:24.656+0000 7f5deb200640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f5dec111c90 con 0x7f5dec1016f0 2024-08-19T01:37:24.665 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:24.656+0000 7f5de9e00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5dec1016f0 0x7f5dec074220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:37:24.665 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:24.656+0000 7f5de9e00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5dec1016f0 0x7f5dec074220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.83:54242/0 (socket says 172.21.15.83:54242) 2024-08-19T01:37:24.665 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:24.656+0000 7f5de9e00640 1 -- 172.21.15.83:0/2018277130 learned_addr learned my addr 172.21.15.83:0/2018277130 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-08-19T01:37:24.665 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:24.656+0000 7f5de9400640 1 --2- 172.21.15.83:0/2018277130 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5dec102090 0x7f5dec074760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:37:24.665 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:24.656+0000 7f5dea800640 1 --2- 172.21.15.83:0/2018277130 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5dec1076f0 0x7f5dec074ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:37:24.665 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:24.656+0000 7f5dea800640 1 -- 172.21.15.83:0/2018277130 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5dec1016f0 msgr2=0x7f5dec074220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:37:24.665 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:24.656+0000 7f5dea800640 1 --2- 172.21.15.83:0/2018277130 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5dec1016f0 0x7f5dec074220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:37:24.665 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:24.656+0000 7f5dea800640 1 -- 172.21.15.83:0/2018277130 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5dec102090 msgr2=0x7f5dec074760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:37:24.665 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:24.656+0000 7f5dea800640 1 --2- 172.21.15.83:0/2018277130 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5dec102090 0x7f5dec074760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:37:24.665 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:24.656+0000 7f5dea800640 1 -- 172.21.15.83:0/2018277130 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5dec19c170 con 0x7f5dec1076f0 2024-08-19T01:37:24.665 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:24.656+0000 7f5de9400640 1 --2- 172.21.15.83:0/2018277130 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5dec102090 0x7f5dec074760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-19T01:37:24.666 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:24.661+0000 7f5dea800640 1 --2- 172.21.15.83:0/2018277130 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5dec1076f0 0x7f5dec074ca0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f5dd802a890 tx=0x7f5dd8033d10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:37:24.666 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:24.661+0000 7f5de2a00640 1 -- 172.21.15.83:0/2018277130 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5dd8037040 con 0x7f5dec1076f0 2024-08-19T01:37:24.666 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:24.661+0000 7f5de2a00640 1 -- 172.21.15.83:0/2018277130 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5dd8040070 con 0x7f5dec1076f0 2024-08-19T01:37:24.666 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:24.661+0000 7f5de2a00640 1 -- 172.21.15.83:0/2018277130 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5dd8003a80 con 0x7f5dec1076f0 2024-08-19T01:37:24.666 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:24.661+0000 7f5deb200640 1 -- 172.21.15.83:0/2018277130 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5dec19c400 con 0x7f5dec1076f0 2024-08-19T01:37:24.666 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:24.661+0000 7f5deb200640 1 -- 172.21.15.83:0/2018277130 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5dec19c890 con 0x7f5dec1076f0 2024-08-19T01:37:24.686 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:24.668+0000 7f5df388e740 1 Processor -- start 2024-08-19T01:37:24.686 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:24.668+0000 7f5de2a00640 1 -- 172.21.15.83:0/2018277130 <== mon.2 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7f5dd8030780 con 0x7f5dec1076f0 2024-08-19T01:37:24.686 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:24.668+0000 7f5df388e740 1 -- start start 2024-08-19T01:37:24.686 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:24.668+0000 7f5df388e740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55727c8936e0 0x55727c893ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:37:24.686 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:24.668+0000 7f5df388e740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55727c894000 0x55727c89e380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:37:24.686 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:24.668+0000 7f5df388e740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55727c89e8c0 0x55727c8a0cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:37:24.686 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:24.668+0000 7f5df388e740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55727c7f8bf0 con 0x55727c894000 2024-08-19T01:37:24.687 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:24.668+0000 7f5df388e740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55727c790b20 con 0x55727c89e8c0 2024-08-19T01:37:24.687 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:24.668+0000 7f5df388e740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55727c7f4bb0 con 0x55727c8936e0 2024-08-19T01:37:24.687 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:24.673+0000 7f5de9e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55727c894000 0x55727c89e380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:37:24.687 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:24.673+0000 7f5de9e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55727c894000 0x55727c89e380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:39048/0 (socket says 172.21.15.83:39048) 2024-08-19T01:37:24.687 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:24.673+0000 7f5de9e00640 1 -- 172.21.15.83:0/3207864510 learned_addr learned my addr 172.21.15.83:0/3207864510 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-08-19T01:37:24.687 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:24.673+0000 7f5de9e00640 1 -- 172.21.15.83:0/3207864510 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55727c8936e0 msgr2=0x55727c893ac0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-08-19T01:37:24.687 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:24.673+0000 7f5de9e00640 1 --2- 172.21.15.83:0/3207864510 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55727c8936e0 0x55727c893ac0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:37:24.687 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:24.673+0000 7f5de9e00640 1 -- 172.21.15.83:0/3207864510 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55727c89e8c0 msgr2=0x55727c8a0cb0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-08-19T01:37:24.687 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:24.673+0000 7f5de9e00640 1 --2- 172.21.15.83:0/3207864510 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55727c89e8c0 0x55727c8a0cb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:37:24.687 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:24.673+0000 7f5de9e00640 1 -- 172.21.15.83:0/3207864510 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55727c7977f0 con 0x55727c894000 2024-08-19T01:37:24.687 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:24.673+0000 7f5de9e00640 1 --2- 172.21.15.83:0/3207864510 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55727c894000 0x55727c89e380 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f5de4007780 tx=0x7f5de4007c40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dd475428f6b2c251 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:37:24.687 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:24.673+0000 7f5dd7000640 1 -- 172.21.15.83:0/3207864510 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5de400e6c0 con 0x55727c894000 2024-08-19T01:37:24.687 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:24.673+0000 7f5dd7000640 1 -- 172.21.15.83:0/3207864510 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5de4019070 con 0x55727c894000 2024-08-19T01:37:24.687 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:24.673+0000 7f5dd7000640 1 -- 172.21.15.83:0/3207864510 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5de4014ce0 con 0x55727c894000 2024-08-19T01:37:24.688 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:24.673+0000 7f5de2a00640 1 --2- 172.21.15.83:0/2018277130 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f5dd00a2cd0 0x7f5dd00a5190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:37:24.688 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:24.678+0000 7f5de9400640 1 --2- 172.21.15.83:0/2018277130 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f5dd00a2cd0 0x7f5dd00a5190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:37:24.688 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:24.681+0000 7f5de2a00640 1 -- 172.21.15.83:0/2018277130 <== mon.2 v2:172.21.15.117:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8269+0+0 (secure 0 0 0) 0x7f5dd80ee960 con 0x7f5dec1076f0 2024-08-19T01:37:24.688 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:24.681+0000 7f5de9400640 1 --2- 172.21.15.83:0/2018277130 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f5dd00a2cd0 0x7f5dd00a5190 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x55727c890ff0 tx=0x7f5ddc00b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:37:24.688 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:24.681+0000 7f5df388e740 1 -- 172.21.15.83:0/3207864510 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55727c894000 msgr2=0x55727c89e380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:37:24.688 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:24.681+0000 7f5df388e740 1 --2- 172.21.15.83:0/3207864510 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55727c894000 0x55727c89e380 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f5de4007780 tx=0x7f5de4007c40 comp rx=0 tx=0).stop 2024-08-19T01:37:24.688 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:24.684+0000 7f5df388e740 1 -- 172.21.15.83:0/3207864510 shutdown_connections 2024-08-19T01:37:24.688 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:24.684+0000 7f5df388e740 1 --2- 172.21.15.83:0/3207864510 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55727c89e8c0 0x55727c8a0cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:37:24.688 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:24.684+0000 7f5df388e740 1 --2- 172.21.15.83:0/3207864510 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55727c894000 0x55727c89e380 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:37:24.688 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:24.684+0000 7f5df388e740 1 --2- 172.21.15.83:0/3207864510 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55727c8936e0 0x55727c893ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:37:24.688 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:24.684+0000 7f5df388e740 1 -- 172.21.15.83:0/3207864510 >> 172.21.15.83:0/3207864510 conn(0x55727c88fc70 msgr2=0x55727c890050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:37:24.688 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:24.684+0000 7f5df388e740 1 -- 172.21.15.83:0/3207864510 shutdown_connections 2024-08-19T01:37:24.688 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:24.684+0000 7f5df388e740 1 -- 172.21.15.83:0/3207864510 wait complete. 2024-08-19T01:37:24.695 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:24.685+0000 7f5df388e740 1 -- start start 2024-08-19T01:37:24.696 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:24.686+0000 7f5df388e740 1 Processor -- start 2024-08-19T01:37:24.696 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:24.687+0000 7f5df388e740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55727c8936e0 0x55727c89b780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:37:24.696 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:24.688+0000 7f5df388e740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55727c894000 0x55727c89bcc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:37:24.696 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:24.688+0000 7f5df388e740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55727c89e8c0 0x55727c89c200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:37:24.696 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:24.688+0000 7f5df388e740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55727c8a1490 con 0x55727c8936e0 2024-08-19T01:37:24.696 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:24.688+0000 7f5df388e740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55727c7b3cf0 con 0x55727c894000 2024-08-19T01:37:24.696 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:24.688+0000 7f5df388e740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55727c8a1610 con 0x55727c89e8c0 2024-08-19T01:37:24.696 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:24.688+0000 7f5df388e740 10 client.0 _ll_register_callbacks cb 0x55727c79fe30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-19T01:37:24.696 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:24.688+0000 7f5de9400640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55727c894000 0x55727c89bcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:37:24.696 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:24.688+0000 7f5de9400640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55727c894000 0x55727c89bcc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.83:52294/0 (socket says 172.21.15.83:52294) 2024-08-19T01:37:24.696 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:24.688+0000 7f5de9400640 1 -- 172.21.15.83:0/1318857980 learned_addr learned my addr 172.21.15.83:0/1318857980 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-08-19T01:37:24.696 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:24.689+0000 7f5de9400640 1 -- 172.21.15.83:0/1318857980 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55727c89e8c0 msgr2=0x55727c89c200 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-08-19T01:37:24.696 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:24.689+0000 7f5de9400640 1 --2- 172.21.15.83:0/1318857980 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55727c89e8c0 0x55727c89c200 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:37:24.697 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:24.689+0000 7f5de9400640 1 -- 172.21.15.83:0/1318857980 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55727c8936e0 msgr2=0x55727c89b780 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-08-19T01:37:24.697 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:24.689+0000 7f5de9e00640 1 --2- 172.21.15.83:0/1318857980 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55727c8936e0 0x55727c89b780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:37:24.697 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:24.689+0000 7f5de9400640 1 --2- 172.21.15.83:0/1318857980 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55727c8936e0 0x55727c89b780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:37:24.697 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:24.689+0000 7f5de9400640 1 -- 172.21.15.83:0/1318857980 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55727c56a3c0 con 0x55727c894000 2024-08-19T01:37:24.697 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:24.691+0000 7f5de9400640 1 --2- 172.21.15.83:0/1318857980 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55727c894000 0x55727c89bcc0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f5ddc011830 tx=0x7f5ddc004730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c965652e1a19285e server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:37:24.697 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:24.691+0000 7f5dcb400640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-08-19T01:37:24.697 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:24.692+0000 7f5dcb400640 1 -- 172.21.15.83:0/1318857980 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ddc01cac0 con 0x55727c894000 2024-08-19T01:37:24.697 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:24.692+0000 7f5dcb400640 1 -- 172.21.15.83:0/1318857980 <== mon.2 v2:172.21.15.117:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8269+0+0 (secure 0 0 0) 0x7f5ddc01cc60 con 0x55727c894000 2024-08-19T01:37:24.697 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:24.693+0000 7f5df388e740 1 -- 172.21.15.83:0/1318857980 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5de4019070 con 0x55727c894000 2024-08-19T01:37:24.697 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:24.693+0000 7f5dcb400640 1 -- 172.21.15.83:0/1318857980 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f5ddc01ce60 con 0x55727c894000 2024-08-19T01:37:24.697 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:24.694+0000 7f5df388e740 10 client.25571 fetch_fsmap learned FSMap version 26 2024-08-19T01:37:24.697 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:24.694+0000 7f5df388e740 1 -- 172.21.15.83:0/1318857980 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55727c8ad1a0 con 0x55727c894000 2024-08-19T01:37:24.697 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:24.694+0000 7f5dcb400640 1 -- 172.21.15.83:0/1318857980 <== mon.2 v2:172.21.15.117:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5de4019070 con 0x55727c894000 2024-08-19T01:37:24.706 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:24.699+0000 7f5dcb400640 1 -- 172.21.15.83:0/1318857980 <== mon.2 v2:172.21.15.117:3300/0 5 ==== fsmap(e 26) ==== 8350+0+0 (secure 0 0 0) 0x7f5ddc027bd0 con 0x55727c894000 2024-08-19T01:37:24.706 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:24.699+0000 7f5df388e740 10 client.25571 fetch_fsmap finished waiting for FSMap version 26 2024-08-19T01:37:24.706 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:24.699+0000 7f5df388e740 10 client.25571 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-19T01:37:24.706 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:24.699+0000 7f5df388e740 20 client.25571 populate_metadata read hostname 'smithi083' 2024-08-19T01:37:24.706 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:24.699+0000 7f5df388e740 1 --2- 172.21.15.83:0/1318857980 >> [v2:172.21.15.117:6838/1967595704,v1:172.21.15.117:6839/1967595704] conn(0x55727c8ae660 0x55727c8b0ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:37:24.706 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:24.699+0000 7f5df388e740 4 client.25571 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-19T01:37:24.706 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:24.699+0000 7f5df388e740 1 -- 172.21.15.83:0/1318857980 --> [v2:172.21.15.117:6838/1967595704,v1:172.21.15.117:6839/1967595704] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55727c806160 con 0x55727c8ae660 2024-08-19T01:37:24.707 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:24.700+0000 7f5de9e00640 1 --2- 172.21.15.83:0/1318857980 >> [v2:172.21.15.117:6838/1967595704,v1:172.21.15.117:6839/1967595704] conn(0x55727c8ae660 0x55727c8b0ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:37:24.707 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:24.701+0000 7f5de9e00640 1 --2- 172.21.15.83:0/1318857980 >> [v2:172.21.15.117:6838/1967595704,v1:172.21.15.117:6839/1967595704] conn(0x55727c8ae660 0x55727c8b0ab0 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=521a48e8aee96785 server_cookie=cf2eafb7c76d5832 in_seq=0 out_seq=0 2024-08-19T01:37:24.707 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:24.701+0000 7f5dcb400640 10 client.25571 ms_handle_connect on v2:172.21.15.117:6838/1967595704 2024-08-19T01:37:24.707 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:24.701+0000 7f5dcb400640 1 -- 172.21.15.83:0/1318857980 <== mds.0 v2:172.21.15.117:6838/1967595704 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55727c806160 con 0x55727c8ae660 2024-08-19T01:37:24.707 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:24.701+0000 7f5dcb400640 10 client.25571 handle_command_reply: tid=0 2024-08-19T01:37:24.707 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:24.702+0000 7f5df388e740 1 client.25571 shutdown 2024-08-19T01:37:24.707 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:24.702+0000 7f5df388e740 1 -- 172.21.15.83:0/1318857980 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55727c894000 msgr2=0x55727c89bcc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:37:24.707 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:24.702+0000 7f5df388e740 1 --2- 172.21.15.83:0/1318857980 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55727c894000 0x55727c89bcc0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f5ddc011830 tx=0x7f5ddc004730 comp rx=0 tx=0).stop 2024-08-19T01:37:24.707 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:24.702+0000 7f5df388e740 1 -- 172.21.15.83:0/1318857980 shutdown_connections 2024-08-19T01:37:24.707 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:24.702+0000 7f5df388e740 1 --2- 172.21.15.83:0/1318857980 >> [v2:172.21.15.117:6838/1967595704,v1:172.21.15.117:6839/1967595704] conn(0x55727c8ae660 0x55727c8b0ab0 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:37:24.707 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:24.702+0000 7f5df388e740 1 --2- 172.21.15.83:0/1318857980 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55727c89e8c0 0x55727c89c200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:37:24.707 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:24.702+0000 7f5df388e740 1 --2- 172.21.15.83:0/1318857980 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55727c894000 0x55727c89bcc0 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:37:24.707 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:24.702+0000 7f5df388e740 1 --2- 172.21.15.83:0/1318857980 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55727c8936e0 0x55727c89b780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:37:24.707 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:24.702+0000 7f5df388e740 1 -- 172.21.15.83:0/1318857980 >> 172.21.15.83:0/1318857980 conn(0x55727c88fc70 msgr2=0x55727c8959a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:37:24.707 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:24.702+0000 7f5dcb400640 0 client.25571 ms_handle_reset on v2:172.21.15.117:6838/1967595704 2024-08-19T01:37:24.708 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:24.703+0000 7f5df388e740 1 -- 172.21.15.83:0/1318857980 shutdown_connections 2024-08-19T01:37:24.708 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:24.703+0000 7f5df388e740 1 -- 172.21.15.83:0/1318857980 wait complete. 2024-08-19T01:37:24.708 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:24.703+0000 7f5df388e740 20 client.25571 trim_cache size 0 max 16384 2024-08-19T01:37:24.749 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:24.733+0000 7f5df388e740 1 Processor -- start 2024-08-19T01:37:24.749 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:24.733+0000 7f5df388e740 1 -- start start 2024-08-19T01:37:24.749 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:24.733+0000 7f5df388e740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55727c8936e0 0x55727c89c050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:37:24.749 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:24.733+0000 7f5df388e740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55727c894000 0x55727c88eb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:37:24.749 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:24.733+0000 7f5df388e740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55727c89e8c0 0x55727c8b6380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:37:24.749 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:24.733+0000 7f5df388e740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f5dec1bea20 con 0x55727c8936e0 2024-08-19T01:37:24.749 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:24.733+0000 7f5df388e740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f5dec1bc2f0 con 0x55727c894000 2024-08-19T01:37:24.749 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:24.733+0000 7f5df388e740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f5dec1a3100 con 0x55727c89e8c0 2024-08-19T01:37:24.749 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:24.734+0000 7f5dea800640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55727c89e8c0 0x55727c8b6380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:37:24.750 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:24.734+0000 7f5dea800640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55727c89e8c0 0x55727c8b6380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.83:54264/0 (socket says 172.21.15.83:54264) 2024-08-19T01:37:24.750 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:24.734+0000 7f5dea800640 1 -- 172.21.15.83:0/181508543 learned_addr learned my addr 172.21.15.83:0/181508543 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-08-19T01:37:24.750 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:24.735+0000 7f5de9e00640 1 --2- 172.21.15.83:0/181508543 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55727c8936e0 0x55727c89c050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:37:24.750 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:24.735+0000 7f5dea800640 1 -- 172.21.15.83:0/181508543 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55727c894000 msgr2=0x55727c88eb90 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-08-19T01:37:24.750 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:24.735+0000 7f5dea800640 1 --2- 172.21.15.83:0/181508543 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55727c894000 0x55727c88eb90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:37:24.750 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:24.735+0000 7f5dea800640 1 -- 172.21.15.83:0/181508543 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55727c8936e0 msgr2=0x55727c89c050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:37:24.750 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:24.735+0000 7f5dea800640 1 --2- 172.21.15.83:0/181508543 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55727c8936e0 0x55727c89c050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:37:24.750 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:24.735+0000 7f5dea800640 1 -- 172.21.15.83:0/181508543 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55727c890e00 con 0x55727c89e8c0 2024-08-19T01:37:24.750 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:24.735+0000 7f5de9e00640 1 --2- 172.21.15.83:0/181508543 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55727c8936e0 0x55727c89c050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-19T01:37:24.750 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:24.735+0000 7f5dea800640 1 --2- 172.21.15.83:0/181508543 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55727c89e8c0 0x55727c8b6380 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f5dd8009780 tx=0x7f5dd80095b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b68f468e1687d77f server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:37:24.750 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:24.736+0000 7f5dd6600640 1 -- 172.21.15.83:0/181508543 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5dd80bb8a0 con 0x55727c89e8c0 2024-08-19T01:37:24.750 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:24.736+0000 7f5dd6600640 1 -- 172.21.15.83:0/181508543 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5dd80bb1a0 con 0x55727c89e8c0 2024-08-19T01:37:24.774 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:24.736+0000 7f5dd6600640 1 -- 172.21.15.83:0/181508543 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5dd80bf120 con 0x55727c89e8c0 2024-08-19T01:37:24.774 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:24.741+0000 7f5df388e740 1 -- 172.21.15.83:0/181508543 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55727c89e8c0 msgr2=0x55727c8b6380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:37:24.774 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:24.741+0000 7f5df388e740 1 --2- 172.21.15.83:0/181508543 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55727c89e8c0 0x55727c8b6380 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f5dd8009780 tx=0x7f5dd80095b0 comp rx=0 tx=0).stop 2024-08-19T01:37:24.774 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:24.741+0000 7f5df388e740 1 -- 172.21.15.83:0/181508543 shutdown_connections 2024-08-19T01:37:24.774 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:24.741+0000 7f5df388e740 1 --2- 172.21.15.83:0/181508543 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55727c89e8c0 0x55727c8b6380 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:37:24.774 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:24.741+0000 7f5df388e740 1 --2- 172.21.15.83:0/181508543 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55727c894000 0x55727c88eb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:37:24.775 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:24.741+0000 7f5df388e740 1 --2- 172.21.15.83:0/181508543 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55727c8936e0 0x55727c89c050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:37:24.775 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:24.741+0000 7f5df388e740 1 -- 172.21.15.83:0/181508543 >> 172.21.15.83:0/181508543 conn(0x55727c88fc70 msgr2=0x55727c88e480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:37:24.775 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:24.742+0000 7f5df388e740 1 -- 172.21.15.83:0/181508543 shutdown_connections 2024-08-19T01:37:24.775 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:24.742+0000 7f5df388e740 1 -- 172.21.15.83:0/181508543 wait complete. 2024-08-19T01:37:24.775 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:24.743+0000 7f5df388e740 1 -- start start 2024-08-19T01:37:24.775 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:24.743+0000 7f5df388e740 1 Processor -- start 2024-08-19T01:37:24.775 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:24.744+0000 7f5df388e740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55727c8936e0 0x55727c8a6890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:37:24.775 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:24.745+0000 7f5df388e740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55727c894000 0x55727c8a6dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:37:24.775 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:24.745+0000 7f5df388e740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55727c89e8c0 0x55727c8a3860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:37:24.775 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:24.745+0000 7f5df388e740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f5dec1a3770 con 0x55727c894000 2024-08-19T01:37:24.775 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:24.745+0000 7f5df388e740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f5dec1bbc20 con 0x55727c89e8c0 2024-08-19T01:37:24.775 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:24.745+0000 7f5df388e740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f5dec1a3e40 con 0x55727c8936e0 2024-08-19T01:37:24.775 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:24.745+0000 7f5dea800640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55727c89e8c0 0x55727c8a3860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:37:24.776 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:24.745+0000 7f5df388e740 10 client.0 _ll_register_callbacks cb 0x7f5ddc011530 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-19T01:37:24.776 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:24.745+0000 7f5dea800640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55727c89e8c0 0x55727c8a3860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.83:52306/0 (socket says 172.21.15.83:52306) 2024-08-19T01:37:24.776 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:24.745+0000 7f5dea800640 1 -- 172.21.15.83:0/766612447 learned_addr learned my addr 172.21.15.83:0/766612447 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-08-19T01:37:24.776 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:24.745+0000 7f5de9e00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55727c8936e0 0x55727c8a6890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:37:24.776 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:24.746+0000 7f5dea800640 1 -- 172.21.15.83:0/766612447 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55727c8936e0 msgr2=0x55727c8a6890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:37:24.776 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:24.746+0000 7f5dea800640 1 --2- 172.21.15.83:0/766612447 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55727c8936e0 0x55727c8a6890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:37:24.776 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:24.746+0000 7f5dea800640 1 -- 172.21.15.83:0/766612447 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55727c894000 msgr2=0x55727c8a6dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:37:24.776 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:24.747+0000 7f5dea800640 1 --2- 172.21.15.83:0/766612447 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55727c894000 0x55727c8a6dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:37:24.776 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:24.747+0000 7f5dea800640 1 -- 172.21.15.83:0/766612447 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55727c8996b0 con 0x55727c89e8c0 2024-08-19T01:37:24.777 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:24.747+0000 7f5de9e00640 1 --2- 172.21.15.83:0/766612447 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55727c8936e0 0x55727c8a6890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-19T01:37:24.777 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:24.747+0000 7f5dcb400640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-08-19T01:37:24.777 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:24.748+0000 7f5dea800640 1 --2- 172.21.15.83:0/766612447 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55727c89e8c0 0x55727c8a3860 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f5dd8009890 tx=0x7f5dd80bf660 comp rx=0 tx=0).ready entity=mon.2 client_cookie=296c943f894006a5 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:37:24.777 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:24.748+0000 7f5dcb400640 1 -- 172.21.15.83:0/766612447 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5dd80d3070 con 0x55727c89e8c0 2024-08-19T01:37:24.777 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:24.748+0000 7f5df388e740 1 -- 172.21.15.83:0/766612447 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5dd80bb1a0 con 0x55727c89e8c0 2024-08-19T01:37:24.777 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:24.748+0000 7f5dcb400640 1 -- 172.21.15.83:0/766612447 <== mon.2 v2:172.21.15.117:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8269+0+0 (secure 0 0 0) 0x7f5dd80bf120 con 0x55727c89e8c0 2024-08-19T01:37:24.777 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:24.749+0000 7f5dcb400640 1 -- 172.21.15.83:0/766612447 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f5dd80bf320 con 0x55727c89e8c0 2024-08-19T01:37:24.777 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:24.749+0000 7f5df388e740 10 client.25574 fetch_fsmap learned FSMap version 26 2024-08-19T01:37:24.777 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:24.749+0000 7f5df388e740 1 -- 172.21.15.83:0/766612447 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55727c919820 con 0x55727c89e8c0 2024-08-19T01:37:24.777 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:24.749+0000 7f5dcb400640 1 -- 172.21.15.83:0/766612447 <== mon.2 v2:172.21.15.117:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5dd80bb1a0 con 0x55727c89e8c0 2024-08-19T01:37:24.778 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:24.750+0000 7f5dcb400640 1 -- 172.21.15.83:0/766612447 <== mon.2 v2:172.21.15.117:3300/0 5 ==== fsmap(e 26) ==== 8350+0+0 (secure 0 0 0) 0x7f5dd803a070 con 0x55727c89e8c0 2024-08-19T01:37:24.778 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:24.751+0000 7f5df388e740 10 client.25574 fetch_fsmap finished waiting for FSMap version 26 2024-08-19T01:37:24.778 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:24.751+0000 7f5df388e740 10 client.25574 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-19T01:37:24.778 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:24.751+0000 7f5df388e740 20 client.25574 populate_metadata read hostname 'smithi083' 2024-08-19T01:37:24.778 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:24.751+0000 7f5df388e740 1 --2- 172.21.15.83:0/766612447 >> [v2:172.21.15.117:6838/1967595704,v1:172.21.15.117:6839/1967595704] conn(0x55727c919be0 0x55727c91c000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:37:24.778 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:24.751+0000 7f5df388e740 4 client.25574 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-08-19T01:37:24.778 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:24.751+0000 7f5df388e740 1 -- 172.21.15.83:0/766612447 --> [v2:172.21.15.117:6838/1967595704,v1:172.21.15.117:6839/1967595704] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55727c800670 con 0x55727c919be0 2024-08-19T01:37:24.778 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:24.751+0000 7f5de9e00640 1 --2- 172.21.15.83:0/766612447 >> [v2:172.21.15.117:6838/1967595704,v1:172.21.15.117:6839/1967595704] conn(0x55727c919be0 0x55727c91c000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:37:24.778 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:24.752+0000 7f5de9e00640 1 --2- 172.21.15.83:0/766612447 >> [v2:172.21.15.117:6838/1967595704,v1:172.21.15.117:6839/1967595704] conn(0x55727c919be0 0x55727c91c000 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8ef063e4d49d6d5c server_cookie=d94d438259e4ba9c in_seq=0 out_seq=0 2024-08-19T01:37:24.778 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:24.752+0000 7f5dcb400640 10 client.25574 ms_handle_connect on v2:172.21.15.117:6838/1967595704 2024-08-19T01:37:24.778 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:24.753+0000 7f5dcb400640 1 -- 172.21.15.83:0/766612447 <== mds.0 v2:172.21.15.117:6838/1967595704 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55727c800670 con 0x55727c919be0 2024-08-19T01:37:24.778 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:24.753+0000 7f5dcb400640 10 client.25574 handle_command_reply: tid=0 2024-08-19T01:37:24.778 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:24.753+0000 7f5df388e740 1 client.25574 shutdown 2024-08-19T01:37:24.779 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:24.753+0000 7f5df388e740 1 -- 172.21.15.83:0/766612447 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55727c89e8c0 msgr2=0x55727c8a3860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:37:24.779 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:24.753+0000 7f5df388e740 1 --2- 172.21.15.83:0/766612447 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55727c89e8c0 0x55727c8a3860 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f5dd8009890 tx=0x7f5dd80bf660 comp rx=0 tx=0).stop 2024-08-19T01:37:24.779 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:24.754+0000 7f5df388e740 1 -- 172.21.15.83:0/766612447 shutdown_connections 2024-08-19T01:37:24.779 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:24.754+0000 7f5df388e740 1 --2- 172.21.15.83:0/766612447 >> [v2:172.21.15.117:6838/1967595704,v1:172.21.15.117:6839/1967595704] conn(0x55727c919be0 0x55727c91c000 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:37:24.779 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:24.754+0000 7f5df388e740 1 --2- 172.21.15.83:0/766612447 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55727c89e8c0 0x55727c8a3860 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:37:24.779 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:24.754+0000 7f5df388e740 1 --2- 172.21.15.83:0/766612447 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55727c894000 0x55727c8a6dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:37:24.779 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:24.754+0000 7f5df388e740 1 --2- 172.21.15.83:0/766612447 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55727c8936e0 0x55727c8a6890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:37:24.779 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:24.754+0000 7f5df388e740 1 -- 172.21.15.83:0/766612447 >> 172.21.15.83:0/766612447 conn(0x55727c88fc70 msgr2=0x55727c88eca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:37:24.779 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:24.754+0000 7f5dcb400640 0 client.25574 ms_handle_reset on v2:172.21.15.117:6838/1967595704 2024-08-19T01:37:24.779 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:24.756+0000 7f5df388e740 1 -- 172.21.15.83:0/766612447 shutdown_connections 2024-08-19T01:37:24.779 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:24.763+0000 7f5df388e740 1 -- 172.21.15.83:0/766612447 wait complete. 2024-08-19T01:37:24.780 INFO:teuthology.orchestra.run.smithi083.stdout:[] 2024-08-19T01:37:24.780 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:24.763+0000 7f5df388e740 20 client.25574 trim_cache size 0 max 16384 2024-08-19T01:37:24.780 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:24.767+0000 7f5deb200640 1 -- 172.21.15.83:0/2018277130 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f5dd00a2cd0 msgr2=0x7f5dd00a5190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:37:24.780 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:24.767+0000 7f5deb200640 1 --2- 172.21.15.83:0/2018277130 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f5dd00a2cd0 0x7f5dd00a5190 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x55727c890ff0 tx=0x7f5ddc00b390 comp rx=0 tx=0).stop 2024-08-19T01:37:24.780 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:24.767+0000 7f5deb200640 1 -- 172.21.15.83:0/2018277130 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5dec1076f0 msgr2=0x7f5dec074ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:37:24.780 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:24.767+0000 7f5deb200640 1 --2- 172.21.15.83:0/2018277130 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5dec1076f0 0x7f5dec074ca0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f5dd802a890 tx=0x7f5dd8033d10 comp rx=0 tx=0).stop 2024-08-19T01:37:24.780 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:24.767+0000 7f5deb200640 1 -- 172.21.15.83:0/2018277130 shutdown_connections 2024-08-19T01:37:24.780 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:24.767+0000 7f5deb200640 1 --2- 172.21.15.83:0/2018277130 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f5dd00a2cd0 0x7f5dd00a5190 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:37:24.780 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:24.767+0000 7f5deb200640 1 --2- 172.21.15.83:0/2018277130 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5dec1076f0 0x7f5dec074ca0 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:37:24.780 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:24.767+0000 7f5deb200640 1 --2- 172.21.15.83:0/2018277130 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5dec102090 0x7f5dec074760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:37:24.780 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:24.767+0000 7f5deb200640 1 --2- 172.21.15.83:0/2018277130 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5dec1016f0 0x7f5dec074220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:37:24.780 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:24.767+0000 7f5deb200640 1 -- 172.21.15.83:0/2018277130 >> 172.21.15.83:0/2018277130 conn(0x7f5dec0fd470 msgr2=0x7f5dec0fdc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:37:24.781 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:24.768+0000 7f5deb200640 1 -- 172.21.15.83:0/2018277130 shutdown_connections 2024-08-19T01:37:24.781 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:24.768+0000 7f5deb200640 1 -- 172.21.15.83:0/2018277130 wait complete. 2024-08-19T01:37:25.915 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:37:25 smithi083 ceph-mon[17120]: pgmap v416: 97 pgs: 97 active+clean; 179 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 8.6 MiB/s wr, 1.51k op/s 2024-08-19T01:37:25.952 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:37:25 smithi117 ceph-mon[22016]: pgmap v416: 97 pgs: 97 active+clean; 179 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 8.6 MiB/s wr, 1.51k op/s 2024-08-19T01:37:25.965 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:37:25 smithi167 ceph-mon[21875]: pgmap v416: 97 pgs: 97 active+clean; 179 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 8.6 MiB/s wr, 1.51k op/s 2024-08-19T01:37:27.588 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-08-19T01:37:27.588 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833 shell --fsid bad8780e-5dc9-11ef-bccf-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-08-19T01:37:27.654 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:37:27 smithi083 ceph-mon[17120]: pgmap v417: 97 pgs: 97 active+clean; 188 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 8.9 MiB/s wr, 1.54k op/s 2024-08-19T01:37:27.952 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:37:27 smithi117 ceph-mon[22016]: pgmap v417: 97 pgs: 97 active+clean; 188 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 8.9 MiB/s wr, 1.54k op/s 2024-08-19T01:37:27.965 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:37:27 smithi167 ceph-mon[21875]: pgmap v417: 97 pgs: 97 active+clean; 188 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 8.9 MiB/s wr, 1.54k op/s 2024-08-19T01:37:29.598 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:37:29.595+0000 7f4ccea00640 1 -- 172.21.15.83:0/4142378339 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4cc8075f60 msgr2=0x7f4cc80743b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:37:29.598 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:37:29.595+0000 7f4ccea00640 1 --2- 172.21.15.83:0/4142378339 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4cc8075f60 0x7f4cc80743b0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f4cbc01ab70 tx=0x7f4cbc03b6d0 comp rx=0 tx=0).stop 2024-08-19T01:37:29.598 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:37:29.596+0000 7f4ccea00640 1 -- 172.21.15.83:0/4142378339 shutdown_connections 2024-08-19T01:37:29.598 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:37:29.596+0000 7f4ccea00640 1 --2- 172.21.15.83:0/4142378339 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4cc810dc50 0x7f4cc8110040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:37:29.598 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:37:29.596+0000 7f4ccea00640 1 --2- 172.21.15.83:0/4142378339 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4cc8074980 0x7f4cc8074de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:37:29.598 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:37:29.596+0000 7f4ccea00640 1 --2- 172.21.15.83:0/4142378339 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4cc8075f60 0x7f4cc80743b0 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:37:29.598 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:37:29.596+0000 7f4ccea00640 1 -- 172.21.15.83:0/4142378339 >> 172.21.15.83:0/4142378339 conn(0x7f4cc80fce00 msgr2=0x7f4cc80ff220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:37:29.614 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:37:29.603+0000 7f4ccea00640 1 -- 172.21.15.83:0/4142378339 shutdown_connections 2024-08-19T01:37:29.614 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:37:29.607+0000 7f4ccea00640 1 -- 172.21.15.83:0/4142378339 wait complete. 2024-08-19T01:37:29.614 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:37:29.608+0000 7f4ccea00640 1 Processor -- start 2024-08-19T01:37:29.614 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:37:29.608+0000 7f4ccea00640 1 -- start start 2024-08-19T01:37:29.614 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:37:29.609+0000 7f4ccea00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4cc8074980 0x7f4cc806bb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:37:29.614 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:37:29.609+0000 7f4ccea00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4cc8075f60 0x7f4cc806c0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:37:29.614 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:37:29.610+0000 7f4cccc00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4cc8075f60 0x7f4cc806c0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:37:29.614 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:37:29.610+0000 7f4cccc00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4cc8075f60 0x7f4cc806c0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:39106/0 (socket says 172.21.15.83:39106) 2024-08-19T01:37:29.615 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:37:29.610+0000 7f4ccd600640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4cc8074980 0x7f4cc806bb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:37:29.615 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:37:29.610+0000 7f4ccd600640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4cc8074980 0x7f4cc806bb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.83:52326/0 (socket says 172.21.15.83:52326) 2024-08-19T01:37:29.615 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:37:29.610+0000 7f4cccc00640 1 -- 172.21.15.83:0/3015360004 learned_addr learned my addr 172.21.15.83:0/3015360004 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-08-19T01:37:29.615 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:37:29.610+0000 7f4ccea00640 1 --2- 172.21.15.83:0/3015360004 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4cc810dc50 0x7f4cc8072f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:37:29.615 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:37:29.610+0000 7f4ccea00640 1 -- 172.21.15.83:0/3015360004 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f4cc81081c0 con 0x7f4cc8075f60 2024-08-19T01:37:29.615 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:37:29.610+0000 7f4ccea00640 1 -- 172.21.15.83:0/3015360004 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f4cc8108040 con 0x7f4cc8074980 2024-08-19T01:37:29.615 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:37:29.610+0000 7f4ccea00640 1 -- 172.21.15.83:0/3015360004 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f4cc8108340 con 0x7f4cc810dc50 2024-08-19T01:37:29.615 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:37:29.611+0000 7f4cccc00640 1 -- 172.21.15.83:0/3015360004 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4cc810dc50 msgr2=0x7f4cc8072f50 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-08-19T01:37:29.615 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:37:29.611+0000 7f4cccc00640 1 --2- 172.21.15.83:0/3015360004 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4cc810dc50 0x7f4cc8072f50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:37:29.615 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:37:29.611+0000 7f4cccc00640 1 -- 172.21.15.83:0/3015360004 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4cc8074980 msgr2=0x7f4cc806bb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:37:29.615 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:37:29.611+0000 7f4cccc00640 1 --2- 172.21.15.83:0/3015360004 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4cc8074980 0x7f4cc806bb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:37:29.615 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:37:29.611+0000 7f4cccc00640 1 -- 172.21.15.83:0/3015360004 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4cc80735c0 con 0x7f4cc8075f60 2024-08-19T01:37:29.615 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:37:29.612+0000 7f4cccc00640 1 --2- 172.21.15.83:0/3015360004 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4cc8075f60 0x7f4cc806c0a0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f4cb400b6d0 tx=0x7f4cb400bb90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:37:29.617 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:37:29.616+0000 7f4cc6000640 1 -- 172.21.15.83:0/3015360004 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4cb4019070 con 0x7f4cc8075f60 2024-08-19T01:37:29.618 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:37:29.616+0000 7f4ccea00640 1 -- 172.21.15.83:0/3015360004 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4cc80738b0 con 0x7f4cc8075f60 2024-08-19T01:37:29.618 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:37:29.616+0000 7f4cc6000640 1 -- 172.21.15.83:0/3015360004 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4cb400cc00 con 0x7f4cc8075f60 2024-08-19T01:37:29.618 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:37:29.616+0000 7f4cc6000640 1 -- 172.21.15.83:0/3015360004 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4cb4012340 con 0x7f4cc8075f60 2024-08-19T01:37:29.618 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:37:29.617+0000 7f4ccea00640 1 -- 172.21.15.83:0/3015360004 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4cc8073e80 con 0x7f4cc8075f60 2024-08-19T01:37:29.636 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:37:29.627+0000 7f4cc6000640 1 -- 172.21.15.83:0/3015360004 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7f4cb401c070 con 0x7f4cc8075f60 2024-08-19T01:37:29.636 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:37:29.630+0000 7f4cc6000640 1 --2- 172.21.15.83:0/3015360004 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f4cb00a2d10 0x7f4cb00a51d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:37:29.637 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:37:29.635+0000 7f4cd304d740 1 Processor -- start 2024-08-19T01:37:29.637 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:37:29.636+0000 7f4cd304d740 1 -- start start 2024-08-19T01:37:29.639 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:37:29.638+0000 7f4cd304d740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d673580680 0x55d673580a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:37:29.639 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:37:29.638+0000 7f4cd304d740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d673580fc0 0x55d673585a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:37:29.641 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:37:29.639+0000 7f4cccc00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d673580fc0 0x55d673585a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:37:29.641 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:37:29.639+0000 7f4cccc00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d673580fc0 0x55d673585a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:39114/0 (socket says 172.21.15.83:39114) 2024-08-19T01:37:29.641 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:37:29.639+0000 7f4cccc00640 1 -- 172.21.15.83:0/2132961032 learned_addr learned my addr 172.21.15.83:0/2132961032 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-08-19T01:37:29.641 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:37:29.639+0000 7f4cd304d740 1 --2- 172.21.15.83:0/2132961032 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d673586050 0x55d673588510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:37:29.651 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:37:29.640+0000 7f4cd304d740 1 -- 172.21.15.83:0/2132961032 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55d6734e4890 con 0x55d673580fc0 2024-08-19T01:37:29.652 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:37:29.640+0000 7f4cd304d740 1 -- 172.21.15.83:0/2132961032 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55d67347cd90 con 0x55d673586050 2024-08-19T01:37:29.652 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:37:29.640+0000 7f4cd304d740 1 -- 172.21.15.83:0/2132961032 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55d6734e0850 con 0x55d673580680 2024-08-19T01:37:29.652 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:37:29.640+0000 7f4cce000640 1 --2- 172.21.15.83:0/2132961032 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d673586050 0x55d673588510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:37:29.652 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:37:29.641+0000 7f4cccc00640 1 -- 172.21.15.83:0/2132961032 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d673580680 msgr2=0x55d673580a80 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-08-19T01:37:29.652 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:37:29.641+0000 7f4cccc00640 1 --2- 172.21.15.83:0/2132961032 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d673580680 0x55d673580a80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:37:29.652 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:37:29.641+0000 7f4cccc00640 1 -- 172.21.15.83:0/2132961032 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d673586050 msgr2=0x55d673588510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:37:29.652 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:37:29.641+0000 7f4cccc00640 1 --2- 172.21.15.83:0/2132961032 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d673586050 0x55d673588510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:37:29.652 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:37:29.641+0000 7f4cccc00640 1 -- 172.21.15.83:0/2132961032 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d6731f8210 con 0x55d673580fc0 2024-08-19T01:37:29.652 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:37:29.642+0000 7f4cc6000640 1 -- 172.21.15.83:0/3015360004 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8269+0+0 (secure 0 0 0) 0x7f4cb40d2060 con 0x7f4cc8075f60 2024-08-19T01:37:29.652 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:37:29.644+0000 7f4ccd600640 1 --2- 172.21.15.83:0/3015360004 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f4cb00a2d10 0x7f4cb00a51d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:37:29.652 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:37:29.645+0000 7f4ccd600640 1 --2- 172.21.15.83:0/3015360004 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f4cb00a2d10 0x7f4cb00a51d0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f4cbc006010 tx=0x7f4cbc0056f0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:37:29.652 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:37:29.649+0000 7f4cccc00640 1 --2- 172.21.15.83:0/2132961032 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d673580fc0 0x55d673585a80 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f4cb40cb970 tx=0x7f4cb400e070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3922c12d4ce7a29 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:37:29.652 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:37:29.650+0000 7f4cba600640 1 -- 172.21.15.83:0/2132961032 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4cb4097790 con 0x55d673580fc0 2024-08-19T01:37:29.662 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:37:29.657+0000 7f4cba600640 1 -- 172.21.15.83:0/2132961032 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4cb40d1020 con 0x55d673580fc0 2024-08-19T01:37:29.662 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:37:29.657+0000 7f4cba600640 1 -- 172.21.15.83:0/2132961032 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4cb409ae50 con 0x55d673580fc0 2024-08-19T01:37:29.669 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:37:29.665+0000 7f4cd304d740 1 -- 172.21.15.83:0/2132961032 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d673580fc0 msgr2=0x55d673585a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:37:29.669 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:37:29.665+0000 7f4cd304d740 1 --2- 172.21.15.83:0/2132961032 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d673580fc0 0x55d673585a80 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f4cb40cb970 tx=0x7f4cb400e070 comp rx=0 tx=0).stop 2024-08-19T01:37:29.675 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:37:29.674+0000 7f4cd304d740 1 -- 172.21.15.83:0/2132961032 shutdown_connections 2024-08-19T01:37:29.675 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:37:29.674+0000 7f4cd304d740 1 --2- 172.21.15.83:0/2132961032 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d673586050 0x55d673588510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:37:29.675 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:37:29.674+0000 7f4cd304d740 1 --2- 172.21.15.83:0/2132961032 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d673580fc0 0x55d673585a80 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:37:29.676 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:37:29.674+0000 7f4cd304d740 1 --2- 172.21.15.83:0/2132961032 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d673580680 0x55d673580a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:37:29.677 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:37:29.674+0000 7f4cd304d740 1 -- 172.21.15.83:0/2132961032 >> 172.21.15.83:0/2132961032 conn(0x55d67357bcf0 msgr2=0x55d67357c0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:37:29.677 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:37:29.676+0000 7f4cd304d740 1 -- 172.21.15.83:0/2132961032 shutdown_connections 2024-08-19T01:37:29.684 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:37:29.678+0000 7f4cd304d740 1 -- 172.21.15.83:0/2132961032 wait complete. 2024-08-19T01:37:29.684 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:37:29.680+0000 7f4cd304d740 1 -- start start 2024-08-19T01:37:29.684 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:37:29.680+0000 7f4cd304d740 1 Processor -- start 2024-08-19T01:37:29.684 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:37:29.681+0000 7f4cd304d740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d673580680 0x55d67358aa70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:37:29.688 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:37:29.684+0000 7f4cd304d740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d673580fc0 0x55d67358afb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:37:29.689 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:37:29.684+0000 7f4cd304d740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d673586050 0x55d673591f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:37:29.689 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:37:29.684+0000 7f4cd304d740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55d6735890c0 con 0x55d673580fc0 2024-08-19T01:37:29.689 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:37:29.684+0000 7f4cd304d740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55d67349ff70 con 0x55d673586050 2024-08-19T01:37:29.689 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:37:29.684+0000 7f4cd304d740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55d673589240 con 0x55d673580680 2024-08-19T01:37:29.689 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:37:29.684+0000 7f4cd304d740 10 client.0 _ll_register_callbacks cb 0x55d6734dc8d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-19T01:37:29.689 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:37:29.684+0000 7f4cce000640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d673586050 0x55d673591f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:37:29.689 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:37:29.684+0000 7f4cce000640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d673586050 0x55d673591f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.83:52348/0 (socket says 172.21.15.83:52348) 2024-08-19T01:37:29.689 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:37:29.684+0000 7f4cce000640 1 -- 172.21.15.83:0/1360223633 learned_addr learned my addr 172.21.15.83:0/1360223633 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-08-19T01:37:29.689 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:37:29.685+0000 7f4cccc00640 1 --2- 172.21.15.83:0/1360223633 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d673580fc0 0x55d67358afb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:37:29.689 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:37:29.685+0000 7f4cccc00640 1 -- 172.21.15.83:0/1360223633 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d673580680 msgr2=0x55d67358aa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:37:29.689 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:37:29.685+0000 7f4cccc00640 1 --2- 172.21.15.83:0/1360223633 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d673580680 0x55d67358aa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:37:29.689 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:37:29.685+0000 7f4cccc00640 1 -- 172.21.15.83:0/1360223633 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d673586050 msgr2=0x55d673591f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:37:29.690 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:37:29.685+0000 7f4cccc00640 1 --2- 172.21.15.83:0/1360223633 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d673586050 0x55d673591f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:37:29.690 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:37:29.685+0000 7f4cccc00640 1 -- 172.21.15.83:0/1360223633 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d6732c4620 con 0x55d673580fc0 2024-08-19T01:37:29.690 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:37:29.686+0000 7f4cccc00640 1 --2- 172.21.15.83:0/1360223633 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d673580fc0 0x55d67358afb0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f4cb40cb970 tx=0x7f4cb4014860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=20f4c76fc8d059d5 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:37:29.695 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:37:29.690+0000 7f4caea00640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-08-19T01:37:29.695 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:37:29.690+0000 7f4caea00640 1 -- 172.21.15.83:0/1360223633 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4cb40b4020 con 0x55d673580fc0 2024-08-19T01:37:29.695 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:37:29.690+0000 7f4caea00640 1 -- 172.21.15.83:0/1360223633 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8269+0+0 (secure 0 0 0) 0x7f4cb4014560 con 0x55d673580fc0 2024-08-19T01:37:29.699 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:37:29.698+0000 7f4cd304d740 1 -- 172.21.15.83:0/1360223633 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4cb40d1020 con 0x55d673580fc0 2024-08-19T01:37:29.699 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:37:29.698+0000 7f4caea00640 1 -- 172.21.15.83:0/1360223633 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f4cb409b030 con 0x55d673580fc0 2024-08-19T01:37:29.700 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:37:29.698+0000 7f4cd304d740 10 client.15519 fetch_fsmap learned FSMap version 26 2024-08-19T01:37:29.700 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:37:29.698+0000 7f4cd304d740 1 -- 172.21.15.83:0/1360223633 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55d673595100 con 0x55d673580fc0 2024-08-19T01:37:29.701 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:37:29.699+0000 7f4caea00640 1 -- 172.21.15.83:0/1360223633 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4cb40d1020 con 0x55d673580fc0 2024-08-19T01:37:29.701 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:37:29.699+0000 7f4caea00640 1 -- 172.21.15.83:0/1360223633 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 26) ==== 8350+0+0 (secure 0 0 0) 0x7f4cb409b750 con 0x55d673580fc0 2024-08-19T01:37:29.701 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:37:29.699+0000 7f4cd304d740 10 client.15519 fetch_fsmap finished waiting for FSMap version 26 2024-08-19T01:37:29.701 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:37:29.699+0000 7f4cd304d740 10 client.15519 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-08-19T01:37:29.701 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:37:29.700+0000 7f4cd304d740 20 client.15519 populate_metadata read hostname 'smithi083' 2024-08-19T01:37:29.701 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:37:29.700+0000 7f4cd304d740 1 --2- 172.21.15.83:0/1360223633 >> [v2:172.21.15.117:6838/1967595704,v1:172.21.15.117:6839/1967595704] conn(0x55d673596450 0x55d673598840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:37:29.701 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:37:29.700+0000 7f4cd304d740 4 client.15519 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-19T01:37:29.701 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:37:29.700+0000 7f4cd304d740 1 -- 172.21.15.83:0/1360223633 --> [v2:172.21.15.117:6838/1967595704,v1:172.21.15.117:6839/1967595704] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d6734f1fc0 con 0x55d673596450 2024-08-19T01:37:29.704 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:37:29.702+0000 7f4ccd600640 1 --2- 172.21.15.83:0/1360223633 >> [v2:172.21.15.117:6838/1967595704,v1:172.21.15.117:6839/1967595704] conn(0x55d673596450 0x55d673598840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:37:29.704 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:37:29.703+0000 7f4ccd600640 1 --2- 172.21.15.83:0/1360223633 >> [v2:172.21.15.117:6838/1967595704,v1:172.21.15.117:6839/1967595704] conn(0x55d673596450 0x55d673598840 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=71fd2c3180cdd7fa server_cookie=95eb8c645ef067e0 in_seq=0 out_seq=0 2024-08-19T01:37:29.705 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:37:29.703+0000 7f4caea00640 10 client.15519 ms_handle_connect on v2:172.21.15.117:6838/1967595704 2024-08-19T01:37:29.712 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:37:29.708+0000 7f4caea00640 1 -- 172.21.15.83:0/1360223633 <== mds.0 v2:172.21.15.117:6838/1967595704 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55d6734f1fc0 con 0x55d673596450 2024-08-19T01:37:29.712 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:37:29.708+0000 7f4caea00640 10 client.15519 handle_command_reply: tid=0 2024-08-19T01:37:29.712 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:37:29.708+0000 7f4cd304d740 1 client.15519 shutdown 2024-08-19T01:37:29.720 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:37:29.713+0000 7f4cd304d740 1 -- 172.21.15.83:0/1360223633 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d673580fc0 msgr2=0x55d67358afb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:37:29.720 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:37:29.713+0000 7f4cd304d740 1 --2- 172.21.15.83:0/1360223633 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d673580fc0 0x55d67358afb0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f4cb40cb970 tx=0x7f4cb4014860 comp rx=0 tx=0).stop 2024-08-19T01:37:29.720 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:37:29.715+0000 7f4cd304d740 1 -- 172.21.15.83:0/1360223633 shutdown_connections 2024-08-19T01:37:29.720 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:37:29.715+0000 7f4cd304d740 1 --2- 172.21.15.83:0/1360223633 >> [v2:172.21.15.117:6838/1967595704,v1:172.21.15.117:6839/1967595704] conn(0x55d673596450 0x55d673598840 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:37:29.720 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:37:29.715+0000 7f4cd304d740 1 --2- 172.21.15.83:0/1360223633 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d673586050 0x55d673591f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:37:29.720 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:37:29.715+0000 7f4cd304d740 1 --2- 172.21.15.83:0/1360223633 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d673580fc0 0x55d67358afb0 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:37:29.720 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:37:29.715+0000 7f4cd304d740 1 --2- 172.21.15.83:0/1360223633 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d673580680 0x55d67358aa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:37:29.720 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:37:29.715+0000 7f4cd304d740 1 -- 172.21.15.83:0/1360223633 >> 172.21.15.83:0/1360223633 conn(0x55d67357bcf0 msgr2=0x55d67357d500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:37:29.720 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:37:29.716+0000 7f4caea00640 0 client.15519 ms_handle_reset on v2:172.21.15.117:6838/1967595704 2024-08-19T01:37:29.720 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:37:29.716+0000 7f4cd304d740 1 -- 172.21.15.83:0/1360223633 shutdown_connections 2024-08-19T01:37:29.720 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:37:29.716+0000 7f4cd304d740 1 -- 172.21.15.83:0/1360223633 wait complete. 2024-08-19T01:37:29.721 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:37:29.716+0000 7f4cd304d740 20 client.15519 trim_cache size 0 max 16384 2024-08-19T01:37:29.754 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:37:29.749+0000 7f4cd304d740 1 Processor -- start 2024-08-19T01:37:29.754 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:37:29.749+0000 7f4cd304d740 1 -- start start 2024-08-19T01:37:29.754 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:37:29.749+0000 7f4cd304d740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d67357bcf0 0x55d67358ae90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:37:29.754 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:37:29.749+0000 7f4cd304d740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d673580680 0x55d673591f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:37:29.754 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:37:29.750+0000 7f4cccc00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d673580680 0x55d673591f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:37:29.754 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:37:29.750+0000 7f4cccc00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d673580680 0x55d673591f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:39130/0 (socket says 172.21.15.83:39130) 2024-08-19T01:37:29.755 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:37:29.750+0000 7f4ccd600640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d67357bcf0 0x55d67358ae90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:37:29.755 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:37:29.750+0000 7f4cccc00640 1 -- 172.21.15.83:0/3537752636 learned_addr learned my addr 172.21.15.83:0/3537752636 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-08-19T01:37:29.755 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:37:29.750+0000 7f4cd304d740 1 --2- 172.21.15.83:0/3537752636 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d673580fc0 0x55d67357d0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:37:29.755 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:37:29.751+0000 7f4cd304d740 1 -- 172.21.15.83:0/3537752636 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f4cc81b1db0 con 0x55d673580680 2024-08-19T01:37:29.755 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:37:29.751+0000 7f4cd304d740 1 -- 172.21.15.83:0/3537752636 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f4cc81af680 con 0x55d67357bcf0 2024-08-19T01:37:29.755 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:37:29.751+0000 7f4cd304d740 1 -- 172.21.15.83:0/3537752636 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f4cc81ad490 con 0x55d673580fc0 2024-08-19T01:37:29.755 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:37:29.751+0000 7f4cce000640 1 --2- 172.21.15.83:0/3537752636 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d673580fc0 0x55d67357d0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:37:29.755 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:37:29.751+0000 7f4cccc00640 1 -- 172.21.15.83:0/3537752636 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d673580fc0 msgr2=0x55d67357d0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:37:29.755 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:37:29.751+0000 7f4cccc00640 1 --2- 172.21.15.83:0/3537752636 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d673580fc0 0x55d67357d0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:37:29.755 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:37:29.751+0000 7f4cccc00640 1 -- 172.21.15.83:0/3537752636 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d67357bcf0 msgr2=0x55d67358ae90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:37:29.755 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:37:29.751+0000 7f4cccc00640 1 --2- 172.21.15.83:0/3537752636 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d67357bcf0 0x55d67358ae90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:37:29.755 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:37:29.751+0000 7f4cccc00640 1 -- 172.21.15.83:0/3537752636 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d67357bb10 con 0x55d673580680 2024-08-19T01:37:29.756 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:37:29.752+0000 7f4cccc00640 1 --2- 172.21.15.83:0/3537752636 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d673580680 0x55d673591f40 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f4cb4097ec0 tx=0x7f4cb4014960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=96e2c4992829c730 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:37:29.756 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:37:29.752+0000 7f4cb9c00640 1 -- 172.21.15.83:0/3537752636 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4cb409b030 con 0x55d673580680 2024-08-19T01:37:29.756 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:37:29.752+0000 7f4cb9c00640 1 -- 172.21.15.83:0/3537752636 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4cb4014560 con 0x55d673580680 2024-08-19T01:37:29.756 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:37:29.753+0000 7f4cb9c00640 1 -- 172.21.15.83:0/3537752636 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4cb409b2c0 con 0x55d673580680 2024-08-19T01:37:29.756 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:37:29.753+0000 7f4cd304d740 1 -- 172.21.15.83:0/3537752636 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d673580680 msgr2=0x55d673591f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:37:29.757 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:37:29.753+0000 7f4cd304d740 1 --2- 172.21.15.83:0/3537752636 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d673580680 0x55d673591f40 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f4cb4097ec0 tx=0x7f4cb4014960 comp rx=0 tx=0).stop 2024-08-19T01:37:29.757 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:37:29.754+0000 7f4cd304d740 1 -- 172.21.15.83:0/3537752636 shutdown_connections 2024-08-19T01:37:29.757 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:37:29.754+0000 7f4cd304d740 1 --2- 172.21.15.83:0/3537752636 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d673580fc0 0x55d67357d0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:37:29.757 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:37:29.754+0000 7f4cd304d740 1 --2- 172.21.15.83:0/3537752636 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d673580680 0x55d673591f40 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:37:29.757 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:37:29.754+0000 7f4cd304d740 1 --2- 172.21.15.83:0/3537752636 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d67357bcf0 0x55d67358ae90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:37:29.757 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:37:29.754+0000 7f4cd304d740 1 -- 172.21.15.83:0/3537752636 >> 172.21.15.83:0/3537752636 conn(0x55d673598c30 msgr2=0x55d67357af40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:37:29.757 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:37:29.754+0000 7f4cd304d740 1 -- 172.21.15.83:0/3537752636 shutdown_connections 2024-08-19T01:37:29.757 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:37:29.754+0000 7f4cd304d740 1 -- 172.21.15.83:0/3537752636 wait complete. 2024-08-19T01:37:29.762 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:37:29.755+0000 7f4cd304d740 1 -- start start 2024-08-19T01:37:29.762 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:37:29.755+0000 7f4cd304d740 1 Processor -- start 2024-08-19T01:37:29.762 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:37:29.757+0000 7f4cd304d740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d673580680 0x55d67358f5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:37:29.762 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:37:29.757+0000 7f4cd304d740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d673580fc0 0x55d67358fb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:37:29.762 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:37:29.760+0000 7f4cd304d740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d673586050 0x55d673590b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:37:29.762 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:37:29.760+0000 7f4cd304d740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f4cc81adb40 con 0x55d673580680 2024-08-19T01:37:29.762 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:37:29.760+0000 7f4cd304d740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f4cc81aefb0 con 0x55d673586050 2024-08-19T01:37:29.763 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:37:29.760+0000 7f4cd304d740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f4cc81ae210 con 0x55d673580fc0 2024-08-19T01:37:29.763 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:37:29.760+0000 7f4cd304d740 10 client.0 _ll_register_callbacks cb 0x7f4cc0009450 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-19T01:37:29.763 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:37:29.760+0000 7f4ccd600640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d673580680 0x55d67358f5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:37:29.763 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:37:29.760+0000 7f4ccd600640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d673580680 0x55d67358f5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:39146/0 (socket says 172.21.15.83:39146) 2024-08-19T01:37:29.763 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:37:29.760+0000 7f4ccd600640 1 -- 172.21.15.83:0/127666967 learned_addr learned my addr 172.21.15.83:0/127666967 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-08-19T01:37:29.766 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:37:29.761+0000 7f4ccd600640 1 -- 172.21.15.83:0/127666967 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d673580fc0 msgr2=0x55d67358fb00 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-08-19T01:37:29.766 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:37:29.761+0000 7f4ccd600640 1 --2- 172.21.15.83:0/127666967 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d673580fc0 0x55d67358fb00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:37:29.766 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:37:29.761+0000 7f4ccd600640 1 -- 172.21.15.83:0/127666967 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d673586050 msgr2=0x55d673590b10 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-08-19T01:37:29.766 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:37:29.761+0000 7f4ccd600640 1 --2- 172.21.15.83:0/127666967 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d673586050 0x55d673590b10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:37:29.766 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:37:29.761+0000 7f4ccd600640 1 -- 172.21.15.83:0/127666967 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d673591120 con 0x55d673580680 2024-08-19T01:37:29.766 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:37:29.761+0000 7f4ccd600640 1 --2- 172.21.15.83:0/127666967 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d673580680 0x55d67358f5c0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f4cbc01ab70 tx=0x7f4cbc03b6d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4b2ac8059eb9f5ca server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:37:29.770 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:37:29.765+0000 7f4caea00640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-08-19T01:37:29.770 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:37:29.765+0000 7f4caea00640 1 -- 172.21.15.83:0/127666967 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4cbc051070 con 0x55d673580680 2024-08-19T01:37:29.770 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:37:29.765+0000 7f4caea00640 1 -- 172.21.15.83:0/127666967 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8269+0+0 (secure 0 0 0) 0x7f4cbc005150 con 0x55d673580680 2024-08-19T01:37:29.773 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:37:29.772+0000 7f4cd304d740 1 -- 172.21.15.83:0/127666967 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4cb4014560 con 0x55d673580680 2024-08-19T01:37:29.774 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:37:29.772+0000 7f4caea00640 1 -- 172.21.15.83:0/127666967 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f4cbc046070 con 0x55d673580680 2024-08-19T01:37:29.779 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:37:29.777+0000 7f4cd304d740 10 client.15525 fetch_fsmap learned FSMap version 26 2024-08-19T01:37:29.779 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:37:29.778+0000 7f4cd304d740 1 -- 172.21.15.83:0/127666967 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55d673601040 con 0x55d673580680 2024-08-19T01:37:29.780 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:37:29.778+0000 7f4caea00640 1 -- 172.21.15.83:0/127666967 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4cb4014560 con 0x55d673580680 2024-08-19T01:37:29.780 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:37:29.779+0000 7f4caea00640 1 -- 172.21.15.83:0/127666967 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 26) ==== 8350+0+0 (secure 0 0 0) 0x7f4cbc042470 con 0x55d673580680 2024-08-19T01:37:29.780 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:37:29.779+0000 7f4cd304d740 10 client.15525 fetch_fsmap finished waiting for FSMap version 26 2024-08-19T01:37:29.780 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:37:29.779+0000 7f4cd304d740 10 client.15525 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-08-19T01:37:29.780 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:37:29.779+0000 7f4cd304d740 20 client.15525 populate_metadata read hostname 'smithi083' 2024-08-19T01:37:29.780 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:37:29.779+0000 7f4cd304d740 1 --2- 172.21.15.83:0/127666967 >> [v2:172.21.15.117:6838/1967595704,v1:172.21.15.117:6839/1967595704] conn(0x55d673601950 0x55d673603d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:37:29.780 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:37:29.779+0000 7f4cd304d740 4 client.15525 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-08-19T01:37:29.781 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:37:29.779+0000 7f4cd304d740 1 -- 172.21.15.83:0/127666967 --> [v2:172.21.15.117:6838/1967595704,v1:172.21.15.117:6839/1967595704] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55d6734ec310 con 0x55d673601950 2024-08-19T01:37:29.792 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:37:29.783+0000 7f4cccc00640 1 --2- 172.21.15.83:0/127666967 >> [v2:172.21.15.117:6838/1967595704,v1:172.21.15.117:6839/1967595704] conn(0x55d673601950 0x55d673603d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:37:29.796 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:37:29.791+0000 7f4cccc00640 1 --2- 172.21.15.83:0/127666967 >> [v2:172.21.15.117:6838/1967595704,v1:172.21.15.117:6839/1967595704] conn(0x55d673601950 0x55d673603d70 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e08fbc7a7702a1be server_cookie=1bb5b4d5fe1025ef in_seq=0 out_seq=0 2024-08-19T01:37:29.796 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:37:29.791+0000 7f4caea00640 10 client.15525 ms_handle_connect on v2:172.21.15.117:6838/1967595704 2024-08-19T01:37:29.905 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:37:29 smithi083 ceph-mon[17120]: pgmap v418: 97 pgs: 97 active+clean; 186 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 7.9 MiB/s wr, 1.34k op/s 2024-08-19T01:37:29.952 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:37:29 smithi117 ceph-mon[22016]: pgmap v418: 97 pgs: 97 active+clean; 186 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 7.9 MiB/s wr, 1.34k op/s 2024-08-19T01:37:29.965 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:37:29 smithi167 ceph-mon[21875]: pgmap v418: 97 pgs: 97 active+clean; 186 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 7.9 MiB/s wr, 1.34k op/s 2024-08-19T01:37:30.404 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/bad8780e-5dc9-11ef-bccf-c7b262605968/mon.a/config 2024-08-19T01:37:30.554 INFO:tasks.workunit.client.0.smithi083.stdout: 66 614048 376 412851 256 381564 1151 2024-08-19T01:37:31.355 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:31.349+0000 7f4ab4a00640 1 -- 172.21.15.83:0/1677512820 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4ab0074240 msgr2=0x7f4ab0074620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:37:31.356 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:31.349+0000 7f4ab4a00640 1 --2- 172.21.15.83:0/1677512820 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4ab0074240 0x7f4ab0074620 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f4aa0009900 tx=0x7f4aa002a4a0 comp rx=0 tx=0).stop 2024-08-19T01:37:31.356 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:31.349+0000 7f4ab4a00640 1 -- 172.21.15.83:0/1677512820 shutdown_connections 2024-08-19T01:37:31.356 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:31.349+0000 7f4ab4a00640 1 --2- 172.21.15.83:0/1677512820 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4ab010dfb0 0x7f4ab01103a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:37:31.356 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:31.349+0000 7f4ab4a00640 1 --2- 172.21.15.83:0/1677512820 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4ab0074b60 0x7f4ab010da70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:37:31.356 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:31.349+0000 7f4ab4a00640 1 --2- 172.21.15.83:0/1677512820 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4ab0074240 0x7f4ab0074620 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:37:31.356 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:31.349+0000 7f4ab4a00640 1 -- 172.21.15.83:0/1677512820 >> 172.21.15.83:0/1677512820 conn(0x7f4ab00fd470 msgr2=0x7f4ab00ff890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:37:31.356 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:31.349+0000 7f4ab4a00640 1 -- 172.21.15.83:0/1677512820 shutdown_connections 2024-08-19T01:37:31.356 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:31.349+0000 7f4ab4a00640 1 -- 172.21.15.83:0/1677512820 wait complete. 2024-08-19T01:37:31.357 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:31.351+0000 7f4ab4a00640 1 Processor -- start 2024-08-19T01:37:31.357 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:31.352+0000 7f4ab4a00640 1 -- start start 2024-08-19T01:37:31.357 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:31.352+0000 7f4ab4a00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4ab0074240 0x7f4ab0073830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:37:31.357 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:31.352+0000 7f4ab4a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4ab0074b60 0x7f4ab006c930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:37:31.357 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:31.352+0000 7f4ab4a00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4ab010dfb0 0x7f4ab006ce70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:37:31.357 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:31.352+0000 7f4ab4a00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f4ab0113d00 con 0x7f4ab0074b60 2024-08-19T01:37:31.357 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:31.352+0000 7f4ab4a00640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f4ab0113b80 con 0x7f4ab0074240 2024-08-19T01:37:31.357 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:31.352+0000 7f4aaf200640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4ab010dfb0 0x7f4ab006ce70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:37:31.357 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:31.352+0000 7f4aaf200640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4ab010dfb0 0x7f4ab006ce70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.83:34694/0 (socket says 172.21.15.83:34694) 2024-08-19T01:37:31.357 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:31.352+0000 7f4aaf200640 1 -- 172.21.15.83:0/2333871187 learned_addr learned my addr 172.21.15.83:0/2333871187 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-08-19T01:37:31.357 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:31.353+0000 7f4ab4a00640 1 -- 172.21.15.83:0/2333871187 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f4ab0113e80 con 0x7f4ab010dfb0 2024-08-19T01:37:31.357 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:31.353+0000 7f4aade00640 1 --2- 172.21.15.83:0/2333871187 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4ab0074b60 0x7f4ab006c930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:37:31.357 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:31.354+0000 7f4aade00640 1 -- 172.21.15.83:0/2333871187 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4ab010dfb0 msgr2=0x7f4ab006ce70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:37:31.358 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:31.354+0000 7f4aade00640 1 --2- 172.21.15.83:0/2333871187 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4ab010dfb0 0x7f4ab006ce70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:37:31.358 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:31.354+0000 7f4aade00640 1 -- 172.21.15.83:0/2333871187 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4ab0074240 msgr2=0x7f4ab0073830 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-08-19T01:37:31.358 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:31.354+0000 7f4aade00640 1 --2- 172.21.15.83:0/2333871187 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4ab0074240 0x7f4ab0073830 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:37:31.358 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:31.354+0000 7f4aade00640 1 -- 172.21.15.83:0/2333871187 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4ab006d510 con 0x7f4ab0074b60 2024-08-19T01:37:31.358 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:31.355+0000 7f4aaf200640 1 --2- 172.21.15.83:0/2333871187 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4ab010dfb0 0x7f4ab006ce70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-19T01:37:31.359 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:31.355+0000 7f4aade00640 1 --2- 172.21.15.83:0/2333871187 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4ab0074b60 0x7f4ab006c930 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f4a9c005fd0 tx=0x7f4a9c00dd20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:37:31.359 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:31.356+0000 7f4aa7400640 1 -- 172.21.15.83:0/2333871187 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a9c01c070 con 0x7f4ab0074b60 2024-08-19T01:37:31.359 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:31.356+0000 7f4aa7400640 1 -- 172.21.15.83:0/2333871187 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4a9c009820 con 0x7f4ab0074b60 2024-08-19T01:37:31.359 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:31.356+0000 7f4ab4a00640 1 -- 172.21.15.83:0/2333871187 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4ab00713d0 con 0x7f4ab0074b60 2024-08-19T01:37:31.359 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:31.356+0000 7f4ab4a00640 1 -- 172.21.15.83:0/2333871187 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4ab0071960 con 0x7f4ab0074b60 2024-08-19T01:37:31.359 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:31.356+0000 7f4aa7400640 1 -- 172.21.15.83:0/2333871187 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a9c0039b0 con 0x7f4ab0074b60 2024-08-19T01:37:31.370 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:31.366+0000 7f4ab7fba740 1 Processor -- start 2024-08-19T01:37:31.379 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:31.366+0000 7f4aa7400640 1 -- 172.21.15.83:0/2333871187 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7f4a9c003b50 con 0x7f4ab0074b60 2024-08-19T01:37:31.379 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:31.366+0000 7f4ab7fba740 1 -- start start 2024-08-19T01:37:31.379 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:31.367+0000 7f4ab7fba740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563175d45650 0x563175d45a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:37:31.379 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:31.367+0000 7f4ab7fba740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563175d45f70 0x563175d502f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:37:31.380 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:31.367+0000 7f4ab7fba740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563175d50830 0x563175d52c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:37:31.380 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:31.367+0000 7f4ab7fba740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x563175c607e0 con 0x563175d45650 2024-08-19T01:37:31.380 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:31.367+0000 7f4ab7fba740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x563175cabac0 con 0x563175d45f70 2024-08-19T01:37:31.380 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:31.367+0000 7f4ab7fba740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x563175c438c0 con 0x563175d50830 2024-08-19T01:37:31.380 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:31.367+0000 7f4aade00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563175d50830 0x563175d52c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:37:31.380 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:31.368+0000 7f4aade00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563175d50830 0x563175d52c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.83:34698/0 (socket says 172.21.15.83:34698) 2024-08-19T01:37:31.380 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:31.368+0000 7f4aade00640 1 -- 172.21.15.83:0/2864346815 learned_addr learned my addr 172.21.15.83:0/2864346815 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-08-19T01:37:31.380 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:31.369+0000 7f4aa7400640 1 --2- 172.21.15.83:0/2333871187 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f4a940a2c00 0x7f4a940a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:37:31.380 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:31.369+0000 7f4aa7400640 1 -- 172.21.15.83:0/2333871187 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8269+0+0 (secure 0 0 0) 0x7f4a9c0d6fc0 con 0x7f4ab0074b60 2024-08-19T01:37:31.380 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:31.369+0000 7f4aae800640 1 --2- 172.21.15.83:0/2864346815 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563175d45f70 0x563175d502f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:37:31.380 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:31.370+0000 7f4aae800640 1 -- 172.21.15.83:0/2864346815 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563175d50830 msgr2=0x563175d52c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:37:31.380 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:31.370+0000 7f4aae800640 1 --2- 172.21.15.83:0/2864346815 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563175d50830 0x563175d52c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:37:31.380 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:31.370+0000 7f4aae800640 1 -- 172.21.15.83:0/2864346815 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563175d45650 msgr2=0x563175d45a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:37:31.381 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:31.370+0000 7f4aae800640 1 --2- 172.21.15.83:0/2864346815 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563175d45650 0x563175d45a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:37:31.381 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:31.370+0000 7f4aae800640 1 -- 172.21.15.83:0/2864346815 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563175c4a760 con 0x563175d45f70 2024-08-19T01:37:31.381 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:31.371+0000 7f4aae800640 1 --2- 172.21.15.83:0/2864346815 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563175d45f70 0x563175d502f0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f4aa002a9c0 tx=0x7f4aa0005780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9cea5deb49d11513 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:37:31.381 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:31.371+0000 7f4a9b400640 1 -- 172.21.15.83:0/2864346815 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4aa003e030 con 0x563175d45f70 2024-08-19T01:37:31.381 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:31.371+0000 7f4a9b400640 1 -- 172.21.15.83:0/2864346815 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4aa00035f0 con 0x563175d45f70 2024-08-19T01:37:31.381 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:31.371+0000 7f4a9b400640 1 -- 172.21.15.83:0/2864346815 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4aa003e030 con 0x563175d45f70 2024-08-19T01:37:31.381 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:31.372+0000 7f4ab7fba740 1 -- 172.21.15.83:0/2864346815 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563175d45f70 msgr2=0x563175d502f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:37:31.381 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:31.372+0000 7f4ab7fba740 1 --2- 172.21.15.83:0/2864346815 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563175d45f70 0x563175d502f0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f4aa002a9c0 tx=0x7f4aa0005780 comp rx=0 tx=0).stop 2024-08-19T01:37:31.381 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:31.378+0000 7f4aaf200640 1 --2- 172.21.15.83:0/2333871187 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f4a940a2c00 0x7f4a940a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:37:31.385 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:31.381+0000 7f4aaf200640 1 --2- 172.21.15.83:0/2333871187 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f4a940a2c00 0x7f4a940a50c0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f4aa80117b0 tx=0x7f4aa8009f10 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:37:31.385 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:31.384+0000 7f4ab7fba740 1 -- 172.21.15.83:0/2864346815 shutdown_connections 2024-08-19T01:37:31.385 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:31.384+0000 7f4ab7fba740 1 --2- 172.21.15.83:0/2864346815 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563175d50830 0x563175d52c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:37:31.388 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:31.384+0000 7f4ab7fba740 1 --2- 172.21.15.83:0/2864346815 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563175d45f70 0x563175d502f0 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:37:31.389 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:31.384+0000 7f4ab7fba740 1 --2- 172.21.15.83:0/2864346815 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563175d45650 0x563175d45a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:37:31.389 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:31.384+0000 7f4ab7fba740 1 -- 172.21.15.83:0/2864346815 >> 172.21.15.83:0/2864346815 conn(0x563175d41be0 msgr2=0x563175d41fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:37:31.389 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:31.385+0000 7f4ab7fba740 1 -- 172.21.15.83:0/2864346815 shutdown_connections 2024-08-19T01:37:31.389 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:31.385+0000 7f4ab7fba740 1 -- 172.21.15.83:0/2864346815 wait complete. 2024-08-19T01:37:31.389 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:31.385+0000 7f4ab7fba740 1 -- start start 2024-08-19T01:37:31.389 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:31.386+0000 7f4ab7fba740 1 Processor -- start 2024-08-19T01:37:31.389 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:31.387+0000 7f4ab7fba740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563175d45650 0x563175d4d6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:37:31.389 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:31.388+0000 7f4ab7fba740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563175d45f70 0x563175d4dc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:37:31.389 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:31.388+0000 7f4ab7fba740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563175d50830 0x563175d4e140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:37:31.392 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:31.389+0000 7f4ab7fba740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x563175c66ae0 con 0x563175d45650 2024-08-19T01:37:31.392 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:31.388+0000 7f4aade00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563175d45f70 0x563175d4dc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:37:31.392 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:31.389+0000 7f4ab7fba740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x563175ca7a80 con 0x563175d50830 2024-08-19T01:37:31.392 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:31.389+0000 7f4ab7fba740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x563175d534a0 con 0x563175d45f70 2024-08-19T01:37:31.392 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:31.389+0000 7f4aade00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563175d45f70 0x563175d4dc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.83:34710/0 (socket says 172.21.15.83:34710) 2024-08-19T01:37:31.392 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:31.389+0000 7f4aade00640 1 -- 172.21.15.83:0/3333670467 learned_addr learned my addr 172.21.15.83:0/3333670467 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-08-19T01:37:31.393 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:31.389+0000 7f4ab7fba740 10 client.0 _ll_register_callbacks cb 0x563175c68190 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-19T01:37:31.393 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:31.389+0000 7f4aade00640 1 -- 172.21.15.83:0/3333670467 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563175d50830 msgr2=0x563175d4e140 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-08-19T01:37:31.393 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:31.389+0000 7f4aade00640 1 --2- 172.21.15.83:0/3333670467 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563175d50830 0x563175d4e140 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:37:31.393 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:31.389+0000 7f4aae800640 1 --2- 172.21.15.83:0/3333670467 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563175d45650 0x563175d4d6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:37:31.393 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:31.389+0000 7f4aade00640 1 -- 172.21.15.83:0/3333670467 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563175d45650 msgr2=0x563175d4d6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:37:31.393 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:31.389+0000 7f4aade00640 1 --2- 172.21.15.83:0/3333670467 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563175d45650 0x563175d4d6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:37:31.393 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:31.390+0000 7f4aade00640 1 -- 172.21.15.83:0/3333670467 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563175a1d6a0 con 0x563175d45f70 2024-08-19T01:37:31.393 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:31.390+0000 7f4aae800640 1 --2- 172.21.15.83:0/3333670467 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563175d45650 0x563175d4d6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-19T01:37:31.393 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:31.391+0000 7f4a8fe00640 10 client.0 ms_handle_connect on v2:172.21.15.167:3300/0 2024-08-19T01:37:31.393 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:31.391+0000 7f4aade00640 1 --2- 172.21.15.83:0/3333670467 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563175d45f70 0x563175d4dc00 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f4a9c09fd00 tx=0x7f4a9c0a37a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=42e008d3f9495b2 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:37:31.393 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:31.391+0000 7f4a8fe00640 1 -- 172.21.15.83:0/3333670467 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a9c0a38a0 con 0x563175d45f70 2024-08-19T01:37:31.393 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:31.391+0000 7f4a8fe00640 1 -- 172.21.15.83:0/3333670467 <== mon.1 v2:172.21.15.167:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8269+0+0 (secure 0 0 0) 0x7f4a9c08e0b0 con 0x563175d45f70 2024-08-19T01:37:31.393 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:31.391+0000 7f4ab7fba740 1 -- 172.21.15.83:0/3333670467 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4aa00035f0 con 0x563175d45f70 2024-08-19T01:37:31.410 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:31.392+0000 7f4a8fe00640 1 -- 172.21.15.83:0/3333670467 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f4a9c08e2b0 con 0x563175d45f70 2024-08-19T01:37:31.410 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:31.395+0000 7f4ab7fba740 10 client.25558 fetch_fsmap learned FSMap version 26 2024-08-19T01:37:31.410 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:31.395+0000 7f4ab7fba740 1 -- 172.21.15.83:0/3333670467 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x563175d5eff0 con 0x563175d45f70 2024-08-19T01:37:31.411 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:31.395+0000 7f4a8fe00640 1 -- 172.21.15.83:0/3333670467 <== mon.1 v2:172.21.15.167:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4aa00035f0 con 0x563175d45f70 2024-08-19T01:37:31.411 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:31.396+0000 7f4a8fe00640 1 -- 172.21.15.83:0/3333670467 <== mon.1 v2:172.21.15.167:3300/0 5 ==== fsmap(e 26) ==== 8350+0+0 (secure 0 0 0) 0x7f4a9c0ac080 con 0x563175d45f70 2024-08-19T01:37:31.411 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:31.396+0000 7f4ab7fba740 10 client.25558 fetch_fsmap finished waiting for FSMap version 26 2024-08-19T01:37:31.411 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:31.396+0000 7f4ab7fba740 10 client.25558 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-19T01:37:31.411 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:31.396+0000 7f4ab7fba740 20 client.25558 populate_metadata read hostname 'smithi083' 2024-08-19T01:37:31.411 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:31.396+0000 7f4ab7fba740 1 --2- 172.21.15.83:0/3333670467 >> [v2:172.21.15.117:6838/1967595704,v1:172.21.15.117:6839/1967595704] conn(0x563175d60450 0x563175d62910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:37:31.411 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:31.396+0000 7f4ab7fba740 4 client.25558 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-19T01:37:31.411 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:31.396+0000 7f4ab7fba740 1 -- 172.21.15.83:0/3333670467 --> [v2:172.21.15.117:6838/1967595704,v1:172.21.15.117:6839/1967595704] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563175cb9030 con 0x563175d60450 2024-08-19T01:37:31.411 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:31.397+0000 7f4aae800640 1 --2- 172.21.15.83:0/3333670467 >> [v2:172.21.15.117:6838/1967595704,v1:172.21.15.117:6839/1967595704] conn(0x563175d60450 0x563175d62910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:37:31.411 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:31.398+0000 7f4aae800640 1 --2- 172.21.15.83:0/3333670467 >> [v2:172.21.15.117:6838/1967595704,v1:172.21.15.117:6839/1967595704] conn(0x563175d60450 0x563175d62910 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b63024fcccdaf034 server_cookie=4b7b220e540697b7 in_seq=0 out_seq=0 2024-08-19T01:37:31.411 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:31.398+0000 7f4a8fe00640 10 client.25558 ms_handle_connect on v2:172.21.15.117:6838/1967595704 2024-08-19T01:37:31.411 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:31.403+0000 7f4a8fe00640 1 -- 172.21.15.83:0/3333670467 <== mds.0 v2:172.21.15.117:6838/1967595704 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x563175cb9030 con 0x563175d60450 2024-08-19T01:37:31.411 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:31.403+0000 7f4a8fe00640 10 client.25558 handle_command_reply: tid=0 2024-08-19T01:37:31.411 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:31.403+0000 7f4ab7fba740 1 client.25558 shutdown 2024-08-19T01:37:31.412 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:31.403+0000 7f4ab7fba740 1 -- 172.21.15.83:0/3333670467 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563175d45f70 msgr2=0x563175d4dc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:37:31.412 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:31.403+0000 7f4ab7fba740 1 --2- 172.21.15.83:0/3333670467 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563175d45f70 0x563175d4dc00 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f4a9c09fd00 tx=0x7f4a9c0a37a0 comp rx=0 tx=0).stop 2024-08-19T01:37:31.412 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:31.404+0000 7f4ab7fba740 1 -- 172.21.15.83:0/3333670467 shutdown_connections 2024-08-19T01:37:31.412 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:31.404+0000 7f4ab7fba740 1 --2- 172.21.15.83:0/3333670467 >> [v2:172.21.15.117:6838/1967595704,v1:172.21.15.117:6839/1967595704] conn(0x563175d60450 0x563175d62910 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:37:31.412 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:31.404+0000 7f4ab7fba740 1 --2- 172.21.15.83:0/3333670467 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563175d50830 0x563175d4e140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:37:31.412 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:31.404+0000 7f4ab7fba740 1 --2- 172.21.15.83:0/3333670467 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563175d45f70 0x563175d4dc00 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:37:31.412 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:31.404+0000 7f4ab7fba740 1 --2- 172.21.15.83:0/3333670467 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563175d45650 0x563175d4d6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:37:31.412 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:31.404+0000 7f4ab7fba740 1 -- 172.21.15.83:0/3333670467 >> 172.21.15.83:0/3333670467 conn(0x563175d41be0 msgr2=0x563175d47910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:37:31.412 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:31.404+0000 7f4a8fe00640 0 client.25558 ms_handle_reset on v2:172.21.15.117:6838/1967595704 2024-08-19T01:37:31.412 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:31.404+0000 7f4ab7fba740 1 -- 172.21.15.83:0/3333670467 shutdown_connections 2024-08-19T01:37:31.412 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:31.404+0000 7f4ab7fba740 1 -- 172.21.15.83:0/3333670467 wait complete. 2024-08-19T01:37:31.412 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:31.405+0000 7f4ab7fba740 20 client.25558 trim_cache size 0 max 16384 2024-08-19T01:37:31.467 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:31.460+0000 7f4ab7fba740 1 Processor -- start 2024-08-19T01:37:31.467 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:31.460+0000 7f4ab7fba740 1 -- start start 2024-08-19T01:37:31.467 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:31.460+0000 7f4ab7fba740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563175d45650 0x563175d480b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:37:31.467 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:31.460+0000 7f4ab7fba740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563175d45f70 0x563175d68220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:37:31.467 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:31.460+0000 7f4ab7fba740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563175d50830 0x563175d68760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:37:31.467 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:31.460+0000 7f4ab7fba740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f4a90010e30 con 0x563175d45f70 2024-08-19T01:37:31.468 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:31.460+0000 7f4ab7fba740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f4a9000e700 con 0x563175d45650 2024-08-19T01:37:31.468 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:31.460+0000 7f4ab7fba740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f4a9000c550 con 0x563175d50830 2024-08-19T01:37:31.468 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:31.461+0000 7f4aae800640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563175d45650 0x563175d480b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:37:31.468 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:31.461+0000 7f4aae800640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563175d45650 0x563175d480b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.83:41036/0 (socket says 172.21.15.83:41036) 2024-08-19T01:37:31.468 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:31.461+0000 7f4aae800640 1 -- 172.21.15.83:0/472267583 learned_addr learned my addr 172.21.15.83:0/472267583 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-08-19T01:37:31.468 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:31.461+0000 7f4aade00640 1 --2- 172.21.15.83:0/472267583 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563175d45f70 0x563175d68220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:37:31.468 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:31.461+0000 7f4aaf200640 1 --2- 172.21.15.83:0/472267583 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563175d50830 0x563175d68760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:37:31.468 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:31.461+0000 7f4aae800640 1 -- 172.21.15.83:0/472267583 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563175d50830 msgr2=0x563175d68760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:37:31.468 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:31.461+0000 7f4aae800640 1 --2- 172.21.15.83:0/472267583 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563175d50830 0x563175d68760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:37:31.468 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:31.461+0000 7f4aae800640 1 -- 172.21.15.83:0/472267583 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563175d45f70 msgr2=0x563175d68220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:37:31.468 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:31.461+0000 7f4aae800640 1 --2- 172.21.15.83:0/472267583 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563175d45f70 0x563175d68220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:37:31.468 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:31.461+0000 7f4aae800640 1 -- 172.21.15.83:0/472267583 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563175d465c0 con 0x563175d45650 2024-08-19T01:37:31.468 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:31.462+0000 7f4aae800640 1 --2- 172.21.15.83:0/472267583 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563175d45650 0x563175d480b0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f4aa0009850 tx=0x7f4aa0009680 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f70a0c818d51b178 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:37:31.468 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:31.462+0000 7f4a9aa00640 1 -- 172.21.15.83:0/472267583 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4aa0051070 con 0x563175d45650 2024-08-19T01:37:31.469 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:31.462+0000 7f4a9aa00640 1 -- 172.21.15.83:0/472267583 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x563175cb9030 con 0x563175d45650 2024-08-19T01:37:31.469 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:31.462+0000 7f4a9aa00640 1 -- 172.21.15.83:0/472267583 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4aa003e040 con 0x563175d45650 2024-08-19T01:37:31.469 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:31.463+0000 7f4ab7fba740 1 -- 172.21.15.83:0/472267583 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563175d45650 msgr2=0x563175d480b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:37:31.469 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:31.463+0000 7f4ab7fba740 1 --2- 172.21.15.83:0/472267583 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563175d45650 0x563175d480b0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f4aa0009850 tx=0x7f4aa0009680 comp rx=0 tx=0).stop 2024-08-19T01:37:31.493 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:31.476+0000 7f4ab7fba740 1 -- 172.21.15.83:0/472267583 shutdown_connections 2024-08-19T01:37:31.493 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:31.476+0000 7f4ab7fba740 1 --2- 172.21.15.83:0/472267583 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563175d50830 0x563175d68760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:37:31.493 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:31.476+0000 7f4ab7fba740 1 --2- 172.21.15.83:0/472267583 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563175d45f70 0x563175d68220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:37:31.493 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:31.476+0000 7f4ab7fba740 1 --2- 172.21.15.83:0/472267583 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563175d45650 0x563175d480b0 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:37:31.493 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:31.476+0000 7f4ab7fba740 1 -- 172.21.15.83:0/472267583 >> 172.21.15.83:0/472267583 conn(0x563175d41be0 msgr2=0x563175d489d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:37:31.493 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:31.476+0000 7f4ab7fba740 1 -- 172.21.15.83:0/472267583 shutdown_connections 2024-08-19T01:37:31.494 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:31.476+0000 7f4ab7fba740 1 -- 172.21.15.83:0/472267583 wait complete. 2024-08-19T01:37:31.494 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:31.477+0000 7f4ab7fba740 1 -- start start 2024-08-19T01:37:31.494 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:31.477+0000 7f4ab7fba740 1 Processor -- start 2024-08-19T01:37:31.494 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:31.478+0000 7f4ab7fba740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563175d45650 0x563175d58ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:37:31.494 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:31.478+0000 7f4ab7fba740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563175d45f70 0x563175d53ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:37:31.494 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:31.478+0000 7f4ab7fba740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563175d50830 0x563175d54410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:37:31.494 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:31.478+0000 7f4ab7fba740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f4a9000cbc0 con 0x563175d45f70 2024-08-19T01:37:31.494 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:31.478+0000 7f4ab7fba740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f4a9000e030 con 0x563175d50830 2024-08-19T01:37:31.494 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:31.478+0000 7f4ab7fba740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f4a9000d290 con 0x563175d45650 2024-08-19T01:37:31.494 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:31.478+0000 7f4ab7fba740 10 client.0 _ll_register_callbacks cb 0x7f4a9c005bb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-19T01:37:31.494 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:31.479+0000 7f4aaf200640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563175d50830 0x563175d54410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:37:31.495 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:31.479+0000 7f4aaf200640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563175d50830 0x563175d54410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.83:41050/0 (socket says 172.21.15.83:41050) 2024-08-19T01:37:31.495 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:31.479+0000 7f4aaf200640 1 -- 172.21.15.83:0/3973695467 learned_addr learned my addr 172.21.15.83:0/3973695467 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-08-19T01:37:31.495 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:31.479+0000 7f4aaf200640 1 -- 172.21.15.83:0/3973695467 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563175d45650 msgr2=0x563175d58ba0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-08-19T01:37:31.495 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:31.479+0000 7f4aaf200640 1 --2- 172.21.15.83:0/3973695467 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563175d45650 0x563175d58ba0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:37:31.495 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:31.479+0000 7f4aaf200640 1 -- 172.21.15.83:0/3973695467 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563175d45f70 msgr2=0x563175d53ed0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-08-19T01:37:31.495 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:31.480+0000 7f4aade00640 1 --2- 172.21.15.83:0/3973695467 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563175d45f70 0x563175d53ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:37:31.495 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:31.480+0000 7f4aaf200640 1 --2- 172.21.15.83:0/3973695467 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563175d45f70 0x563175d53ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:37:31.495 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:31.480+0000 7f4aaf200640 1 -- 172.21.15.83:0/3973695467 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563175d54950 con 0x563175d50830 2024-08-19T01:37:31.495 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:31.480+0000 7f4aade00640 1 --2- 172.21.15.83:0/3973695467 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563175d45f70 0x563175d53ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-19T01:37:31.495 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:31.481+0000 7f4aaf200640 1 --2- 172.21.15.83:0/3973695467 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563175d50830 0x563175d54410 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f4aa8009d80 tx=0x7f4aa8009640 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fd41db87e863e33a server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:37:31.495 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:31.481+0000 7f4a8fe00640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-08-19T01:37:31.495 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:31.481+0000 7f4a8fe00640 1 -- 172.21.15.83:0/3973695467 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4aa8023070 con 0x563175d50830 2024-08-19T01:37:31.495 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:31.481+0000 7f4a8fe00640 1 -- 172.21.15.83:0/3973695467 <== mon.2 v2:172.21.15.117:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8269+0+0 (secure 0 0 0) 0x7f4aa801d450 con 0x563175d50830 2024-08-19T01:37:31.495 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:31.482+0000 7f4ab7fba740 1 -- 172.21.15.83:0/3973695467 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x563175cb9030 con 0x563175d50830 2024-08-19T01:37:31.496 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:31.482+0000 7f4a8fe00640 1 -- 172.21.15.83:0/3973695467 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f4aa801d650 con 0x563175d50830 2024-08-19T01:37:31.496 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:31.482+0000 7f4ab7fba740 10 client.25598 fetch_fsmap learned FSMap version 26 2024-08-19T01:37:31.496 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:31.482+0000 7f4ab7fba740 1 -- 172.21.15.83:0/3973695467 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x563175dcaf60 con 0x563175d50830 2024-08-19T01:37:31.496 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:31.483+0000 7f4a8fe00640 1 -- 172.21.15.83:0/3973695467 <== mon.2 v2:172.21.15.117:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x563175cb9030 con 0x563175d50830 2024-08-19T01:37:31.496 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:31.483+0000 7f4a8fe00640 1 -- 172.21.15.83:0/3973695467 <== mon.2 v2:172.21.15.117:3300/0 5 ==== fsmap(e 26) ==== 8350+0+0 (secure 0 0 0) 0x7f4aa801a040 con 0x563175d50830 2024-08-19T01:37:31.496 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:31.483+0000 7f4ab7fba740 10 client.25598 fetch_fsmap finished waiting for FSMap version 26 2024-08-19T01:37:31.496 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:31.483+0000 7f4ab7fba740 10 client.25598 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-19T01:37:31.496 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:31.483+0000 7f4ab7fba740 20 client.25598 populate_metadata read hostname 'smithi083' 2024-08-19T01:37:31.496 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:31.483+0000 7f4ab7fba740 1 --2- 172.21.15.83:0/3973695467 >> [v2:172.21.15.117:6838/1967595704,v1:172.21.15.117:6839/1967595704] conn(0x563175dcb730 0x563175dcdb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:37:31.496 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:31.483+0000 7f4ab7fba740 4 client.25598 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-08-19T01:37:31.496 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:31.483+0000 7f4ab7fba740 1 -- 172.21.15.83:0/3973695467 --> [v2:172.21.15.117:6838/1967595704,v1:172.21.15.117:6839/1967595704] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x563175cb3540 con 0x563175dcb730 2024-08-19T01:37:31.496 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:31.483+0000 7f4aae800640 1 --2- 172.21.15.83:0/3973695467 >> [v2:172.21.15.117:6838/1967595704,v1:172.21.15.117:6839/1967595704] conn(0x563175dcb730 0x563175dcdb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:37:31.497 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:31.484+0000 7f4aae800640 1 --2- 172.21.15.83:0/3973695467 >> [v2:172.21.15.117:6838/1967595704,v1:172.21.15.117:6839/1967595704] conn(0x563175dcb730 0x563175dcdb50 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b71a59c669914fe3 server_cookie=39171f7fd5ce58dc in_seq=0 out_seq=0 2024-08-19T01:37:31.497 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:31.484+0000 7f4a8fe00640 10 client.25598 ms_handle_connect on v2:172.21.15.117:6838/1967595704 2024-08-19T01:37:31.620 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:37:31 smithi083 ceph-mon[17120]: pgmap v419: 97 pgs: 97 active+clean; 193 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 7.5 MiB/s wr, 715 op/s 2024-08-19T01:37:31.952 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:37:31 smithi117 ceph-mon[22016]: pgmap v419: 97 pgs: 97 active+clean; 193 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 7.5 MiB/s wr, 715 op/s 2024-08-19T01:37:31.965 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:37:31 smithi167 ceph-mon[21875]: pgmap v419: 97 pgs: 97 active+clean; 193 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 7.5 MiB/s wr, 715 op/s 2024-08-19T01:37:32.272 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:32.269+0000 7f4a8fe00640 1 -- 172.21.15.83:0/3973695467 <== mds.0 v2:172.21.15.117:6838/1967595704 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x563175cb3540 con 0x563175dcb730 2024-08-19T01:37:32.273 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:32.269+0000 7f4a8fe00640 10 client.25598 handle_command_reply: tid=0 2024-08-19T01:37:32.273 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:32.270+0000 7f4ab7fba740 1 client.25598 shutdown 2024-08-19T01:37:32.273 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:32.270+0000 7f4ab7fba740 1 -- 172.21.15.83:0/3973695467 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563175d50830 msgr2=0x563175d54410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:37:32.273 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:32.270+0000 7f4ab7fba740 1 --2- 172.21.15.83:0/3973695467 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563175d50830 0x563175d54410 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f4aa8009d80 tx=0x7f4aa8009640 comp rx=0 tx=0).stop 2024-08-19T01:37:32.273 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:32.270+0000 7f4ab7fba740 1 -- 172.21.15.83:0/3973695467 shutdown_connections 2024-08-19T01:37:32.273 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:32.270+0000 7f4ab7fba740 1 --2- 172.21.15.83:0/3973695467 >> [v2:172.21.15.117:6838/1967595704,v1:172.21.15.117:6839/1967595704] conn(0x563175dcb730 0x563175dcdb50 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:37:32.273 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:32.270+0000 7f4ab7fba740 1 --2- 172.21.15.83:0/3973695467 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563175d50830 0x563175d54410 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:37:32.273 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:32.270+0000 7f4ab7fba740 1 --2- 172.21.15.83:0/3973695467 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563175d45f70 0x563175d53ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:37:32.273 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:32.270+0000 7f4ab7fba740 1 --2- 172.21.15.83:0/3973695467 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563175d45650 0x563175d58ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:37:32.273 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:32.270+0000 7f4ab7fba740 1 -- 172.21.15.83:0/3973695467 >> 172.21.15.83:0/3973695467 conn(0x563175d41be0 msgr2=0x563175d40520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:37:32.273 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:32.271+0000 7f4a8fe00640 0 client.25598 ms_handle_reset on v2:172.21.15.117:6838/1967595704 2024-08-19T01:37:32.273 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:32.272+0000 7f4ab7fba740 1 -- 172.21.15.83:0/3973695467 shutdown_connections 2024-08-19T01:37:32.274 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:32.272+0000 7f4ab7fba740 1 -- 172.21.15.83:0/3973695467 wait complete. 2024-08-19T01:37:32.284 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-08-19T01:37:32.284 INFO:teuthology.orchestra.run.smithi083.stdout: "return_code": 0, 2024-08-19T01:37:32.284 INFO:teuthology.orchestra.run.smithi083.stdout: "scrub_tag": "28dac109-a1c6-4541-bf5f-b2c13f258789", 2024-08-19T01:37:32.284 INFO:teuthology.orchestra.run.smithi083.stdout: "mode": "asynchronous" 2024-08-19T01:37:32.284 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-08-19T01:37:32.285 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:32.272+0000 7f4ab7fba740 20 client.25598 trim_cache size 0 max 16384 2024-08-19T01:37:32.285 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:32.276+0000 7f4ab4a00640 1 -- 172.21.15.83:0/2333871187 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f4a940a2c00 msgr2=0x7f4a940a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:37:32.285 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:32.276+0000 7f4ab4a00640 1 --2- 172.21.15.83:0/2333871187 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f4a940a2c00 0x7f4a940a50c0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f4aa80117b0 tx=0x7f4aa8009f10 comp rx=0 tx=0).stop 2024-08-19T01:37:32.285 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:32.276+0000 7f4ab4a00640 1 -- 172.21.15.83:0/2333871187 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4ab0074b60 msgr2=0x7f4ab006c930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:37:32.285 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:32.276+0000 7f4ab4a00640 1 --2- 172.21.15.83:0/2333871187 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4ab0074b60 0x7f4ab006c930 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f4a9c005fd0 tx=0x7f4a9c00dd20 comp rx=0 tx=0).stop 2024-08-19T01:37:32.285 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:32.276+0000 7f4ab4a00640 1 -- 172.21.15.83:0/2333871187 shutdown_connections 2024-08-19T01:37:32.285 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:32.276+0000 7f4ab4a00640 1 --2- 172.21.15.83:0/2333871187 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f4a940a2c00 0x7f4a940a50c0 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:37:32.285 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:32.276+0000 7f4ab4a00640 1 --2- 172.21.15.83:0/2333871187 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4ab010dfb0 0x7f4ab006ce70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:37:32.285 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:32.276+0000 7f4ab4a00640 1 --2- 172.21.15.83:0/2333871187 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4ab0074b60 0x7f4ab006c930 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:37:32.285 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:32.276+0000 7f4ab4a00640 1 --2- 172.21.15.83:0/2333871187 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4ab0074240 0x7f4ab0073830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:37:32.285 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:32.276+0000 7f4ab4a00640 1 -- 172.21.15.83:0/2333871187 >> 172.21.15.83:0/2333871187 conn(0x7f4ab00fd470 msgr2=0x7f4ab00fed00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:37:32.286 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:32.276+0000 7f4ab4a00640 1 -- 172.21.15.83:0/2333871187 shutdown_connections 2024-08-19T01:37:32.286 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:32.276+0000 7f4ab4a00640 1 -- 172.21.15.83:0/2333871187 wait complete. 2024-08-19T01:37:32.760 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:37:32.759+0000 7f4cad600640 1 -- 172.21.15.83:0/127666967 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d673580680 msgr2=0x55d67358f5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:37:32.760 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:37:32.759+0000 7f4cad600640 1 --2- 172.21.15.83:0/127666967 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d673580680 0x55d67358f5c0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f4cbc01ab70 tx=0x7f4cbc03b6d0 comp rx=0 tx=0).stop 2024-08-19T01:37:32.760 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:37:32.760+0000 7f4cad600640 1 --2- 172.21.15.83:0/127666967 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d673586050 0x7f4c980056b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:37:32.761 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:37:32.760+0000 7f4cad600640 1 --2- 172.21.15.83:0/127666967 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4c98006040 0x7f4c980084d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:37:32.761 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:37:32.761+0000 7f4ccd600640 1 --2- 172.21.15.83:0/127666967 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d673586050 0x7f4c980056b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:37:32.762 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:37:32.761+0000 7f4cce000640 1 --2- 172.21.15.83:0/127666967 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4c98006040 0x7f4c980084d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:37:32.766 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:37:32.761+0000 7f4cad600640 1 --2- 172.21.15.83:0/127666967 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d673580fc0 0x7f4c9800afe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:37:32.766 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:37:32.761+0000 7f4cad600640 1 -- 172.21.15.83:0/127666967 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f4c9800b920 con 0x7f4c98006040 2024-08-19T01:37:32.766 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:37:32.761+0000 7f4cad600640 1 -- 172.21.15.83:0/127666967 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f4c9800baa0 con 0x55d673586050 2024-08-19T01:37:32.767 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:37:32.761+0000 7f4cad600640 1 -- 172.21.15.83:0/127666967 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f4c9800bca0 con 0x55d673580fc0 2024-08-19T01:37:32.767 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:37:32.761+0000 7f4cce000640 1 -- 172.21.15.83:0/127666967 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d673580fc0 msgr2=0x7f4c9800afe0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-08-19T01:37:32.767 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:37:32.761+0000 7f4cce000640 1 --2- 172.21.15.83:0/127666967 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d673580fc0 0x7f4c9800afe0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:37:32.767 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:37:32.761+0000 7f4cce000640 1 -- 172.21.15.83:0/127666967 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d673586050 msgr2=0x7f4c980056b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:37:32.767 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:37:32.761+0000 7f4cce000640 1 --2- 172.21.15.83:0/127666967 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d673586050 0x7f4c980056b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:37:32.767 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:37:32.761+0000 7f4cce000640 1 -- 172.21.15.83:0/127666967 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=4+}) -- 0x7f4c9800bf60 con 0x7f4c98006040 2024-08-19T01:37:32.767 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:37:32.761+0000 7f4cce000640 1 --2- 172.21.15.83:0/127666967 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4c98006040 0x7f4c980084d0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f4cc0009fa0 tx=0x7f4cbc05e040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ba0e1b5380126cac server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:37:32.767 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:37:32.761+0000 7f4caea00640 10 client.15525 ms_handle_connect on v2:172.21.15.83:3300/0 2024-08-19T01:37:32.767 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:37:32.761+0000 7f4caea00640 1 -- 172.21.15.83:0/127666967 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4cc001b020 con 0x7f4c98006040 2024-08-19T01:37:32.767 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:37:32.761+0000 7f4caea00640 1 -- 172.21.15.83:0/127666967 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4cc0017b30 con 0x7f4c98006040 2024-08-19T01:37:33.565 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:37:33 smithi083 ceph-mon[17120]: scrub queued for path: / 2024-08-19T01:37:33.904 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:37:33 smithi083 ceph-mon[17120]: scrub summary: idle+waiting paths [/] 2024-08-19T01:37:33.904 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:37:33 smithi083 ceph-mon[17120]: scrub summary: active paths [/] 2024-08-19T01:37:33.905 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:37:33 smithi083 ceph-mon[17120]: pgmap v420: 97 pgs: 97 active+clean; 193 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 6.7 MiB/s wr, 465 op/s 2024-08-19T01:37:33.952 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:37:33 smithi117 ceph-mon[22016]: scrub queued for path: / 2024-08-19T01:37:33.952 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:37:33 smithi117 ceph-mon[22016]: scrub summary: idle+waiting paths [/] 2024-08-19T01:37:33.952 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:37:33 smithi117 ceph-mon[22016]: scrub summary: active paths [/] 2024-08-19T01:37:33.952 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:37:33 smithi117 ceph-mon[22016]: pgmap v420: 97 pgs: 97 active+clean; 193 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 6.7 MiB/s wr, 465 op/s 2024-08-19T01:37:33.965 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:37:33 smithi167 ceph-mon[21875]: scrub queued for path: / 2024-08-19T01:37:33.965 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:37:33 smithi167 ceph-mon[21875]: scrub summary: idle+waiting paths [/] 2024-08-19T01:37:33.966 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:37:33 smithi167 ceph-mon[21875]: scrub summary: active paths [/] 2024-08-19T01:37:33.966 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:37:33 smithi167 ceph-mon[21875]: pgmap v420: 97 pgs: 97 active+clean; 193 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 6.7 MiB/s wr, 465 op/s 2024-08-19T01:37:34.248 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833 shell --fsid bad8780e-5dc9-11ef-bccf-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-08-19T01:37:35.402 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:37:35.400+0000 7f4caea00640 1 -- 172.21.15.83:0/127666967 <== mds.0 v2:172.21.15.117:6838/1967595704 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55d6734ec310 con 0x55d673601950 2024-08-19T01:37:35.402 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:37:35.400+0000 7f4caea00640 10 client.15525 handle_command_reply: tid=0 2024-08-19T01:37:35.402 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:37:35.400+0000 7f4cd304d740 1 client.15525 shutdown 2024-08-19T01:37:35.435 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:37:35.434+0000 7f4cd304d740 1 -- 172.21.15.83:0/127666967 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4c98006040 msgr2=0x7f4c980084d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:37:35.435 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:37:35.434+0000 7f4cd304d740 1 --2- 172.21.15.83:0/127666967 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4c98006040 0x7f4c980084d0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f4cc0009fa0 tx=0x7f4cbc05e040 comp rx=0 tx=0).stop 2024-08-19T01:37:35.439 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:37:35.435+0000 7f4cd304d740 1 -- 172.21.15.83:0/127666967 shutdown_connections 2024-08-19T01:37:35.439 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:37:35.435+0000 7f4cd304d740 1 --2- 172.21.15.83:0/127666967 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d673580fc0 0x7f4c9800afe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:37:35.439 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:37:35.435+0000 7f4cd304d740 1 --2- 172.21.15.83:0/127666967 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4c98006040 0x7f4c980084d0 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:37:35.439 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:37:35.435+0000 7f4cd304d740 1 --2- 172.21.15.83:0/127666967 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d673586050 0x7f4c980056b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:37:35.439 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:37:35.435+0000 7f4cd304d740 1 --2- 172.21.15.83:0/127666967 >> [v2:172.21.15.117:6838/1967595704,v1:172.21.15.117:6839/1967595704] conn(0x55d673601950 0x55d673603d70 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:37:35.439 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:37:35.435+0000 7f4cd304d740 1 -- 172.21.15.83:0/127666967 >> 172.21.15.83:0/127666967 conn(0x55d673598c30 msgr2=0x55d67357c0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:37:35.439 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:37:35.435+0000 7f4caea00640 0 client.15525 ms_handle_reset on v2:172.21.15.117:6838/1967595704 2024-08-19T01:37:35.444 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:37:35.444+0000 7f4cd304d740 1 -- 172.21.15.83:0/127666967 shutdown_connections 2024-08-19T01:37:35.447 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stdout:{ 2024-08-19T01:37:35.448 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stdout: "message": "", 2024-08-19T01:37:35.448 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stdout: "return_code": 0 2024-08-19T01:37:35.448 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stdout:} 2024-08-19T01:37:35.448 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:37:35.444+0000 7f4cd304d740 1 -- 172.21.15.83:0/127666967 wait complete. 2024-08-19T01:37:35.448 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:37:35.444+0000 7f4cd304d740 20 client.15525 trim_cache size 0 max 16384 2024-08-19T01:37:35.457 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:37:35.456+0000 7f4ccea00640 1 -- 172.21.15.83:0/3015360004 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f4cb00a2d10 msgr2=0x7f4cb00a51d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:37:35.457 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:37:35.456+0000 7f4ccea00640 1 --2- 172.21.15.83:0/3015360004 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f4cb00a2d10 0x7f4cb00a51d0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f4cbc006010 tx=0x7f4cbc0056f0 comp rx=0 tx=0).stop 2024-08-19T01:37:35.457 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:37:35.456+0000 7f4ccea00640 1 -- 172.21.15.83:0/3015360004 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4cc8075f60 msgr2=0x7f4cc806c0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:37:35.457 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:37:35.456+0000 7f4ccea00640 1 --2- 172.21.15.83:0/3015360004 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4cc8075f60 0x7f4cc806c0a0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f4cb400b6d0 tx=0x7f4cb400bb90 comp rx=0 tx=0).stop 2024-08-19T01:37:35.469 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:37:35.467+0000 7f4ccea00640 1 -- 172.21.15.83:0/3015360004 shutdown_connections 2024-08-19T01:37:35.470 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:37:35.467+0000 7f4ccea00640 1 --2- 172.21.15.83:0/3015360004 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f4cb00a2d10 0x7f4cb00a51d0 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:37:35.470 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:37:35.467+0000 7f4ccea00640 1 --2- 172.21.15.83:0/3015360004 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4cc810dc50 0x7f4cc8072f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:37:35.470 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:37:35.467+0000 7f4ccea00640 1 --2- 172.21.15.83:0/3015360004 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4cc8075f60 0x7f4cc806c0a0 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:37:35.470 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:37:35.467+0000 7f4ccea00640 1 --2- 172.21.15.83:0/3015360004 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4cc8074980 0x7f4cc806bb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:37:35.470 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:37:35.467+0000 7f4ccea00640 1 -- 172.21.15.83:0/3015360004 >> 172.21.15.83:0/3015360004 conn(0x7f4cc80fce00 msgr2=0x7f4cc8100ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:37:35.473 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:37:35.473+0000 7f4ccea00640 1 -- 172.21.15.83:0/3015360004 shutdown_connections 2024-08-19T01:37:35.473 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:37:35.473+0000 7f4ccea00640 1 -- 172.21.15.83:0/3015360004 wait complete. 2024-08-19T01:37:35.560 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:37:35 smithi083 ceph-mon[17120]: pgmap v421: 97 pgs: 97 active+clean; 198 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 6.4 MiB/s wr, 478 op/s 2024-08-19T01:37:35.952 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:37:35 smithi117 ceph-mon[22016]: pgmap v421: 97 pgs: 97 active+clean; 198 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 6.4 MiB/s wr, 478 op/s 2024-08-19T01:37:35.965 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:37:35 smithi167 ceph-mon[21875]: pgmap v421: 97 pgs: 97 active+clean; 198 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 6.4 MiB/s wr, 478 op/s 2024-08-19T01:37:37.452 INFO:journalctl@ceph.mds.b.smithi117.stdout:Aug 19 01:37:36 smithi117 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-mds-b[51905]: 2024-08-19T01:37:36.959+0000 7f79e46bd640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000001d0f [...2,head] /volumes/qa/sv_1/05b773d9-8346-4397-99ec-d49959a55703/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/ auth{2=1} fragtree_t(*^3) v123376 ap=2 RANDEPHEMERALPIN f(v3 m2024-08-19T01:37:12.988594+0000 178=178+0) n(v5 rc2024-08-19T01:37:12.988594+0000 b3024162 179=178+1) (inest sync->mix g=2) caps={25052=pAsLsXsFs/pAsLsXsFsxcral@177} | dirtyscattered=0 request=0 lock=0 dirfrag=8 caps=1 dirtyparent=0 replicated=1 dirty=0 waiter=0 authpin=1 scrubqueue=0 randepin 0x55b59af05180]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value":"(2)0x10000001d0f:[<0x10000000399/blog-34 v50963>,<0x10000000208/blogtest_in v398>,<0x10000000201/src v1681>,<0x10000000006/blogbench-1.0 v1455>,<0x10000000005/tmp v97>,<0x100000001fb/client.0 v85>,<0x100000001fa/05b773d9-8346-4397-99ec-d49959a55703 v89>,<0x10000000001/sv_1 v101>,<0x10000000000/qa v92>,<0x1/volumes v89>]//[]","memoryvalue":"(2)0x10000001d0f:[<0x10000000399/blog-34 v123376>,<0x10000000208/blogtest_in v619>,<0x10000000201/src v1781>,<0x10000000006/blogbench-1.0 v1550>,<0x10000000005/tmp v188>,<0x100000001fb/client.0 v174>,<0x100000001fa/05b773d9-8346-4397-99ec-d49959a55703 v177>,<0x10000000001/sv_1 v185>,<0x10000000000/qa v179>,<0x1/volumes v172>]//[]","error_str":""},"raw_stats":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2024-08-19T01:37:01.469867+0000 176=176+0)","ondisk_value.rstat":"n(v0 rc2024-08-19T01:37:12.988594+0000 b3024162 179=178+1)","memory_value.dirstat":"f(v3 m2024-08-19T01:37:12.988594+0000 178=178+0)","memory_value.rstat":"n(v5 rc2024-08-19T01:37:12.988594+0000 b3024162 179=178+1)","error_str":"freshly-calculated rstats don't match existing ones"},"return_code":0} 2024-08-19T01:37:37.691 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:37:37 smithi083 ceph-mon[17120]: pgmap v422: 97 pgs: 97 active+clean; 194 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.5 MiB/s wr, 392 op/s 2024-08-19T01:37:37.691 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:37:37 smithi083 ceph-mon[17120]: scrub summary: active paths [/] 2024-08-19T01:37:37.691 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:37:37 smithi083 ceph-mon[17120]: scrub summary: idle+waiting paths [/] 2024-08-19T01:37:37.691 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:37:37 smithi083 ceph-mon[17120]: scrub summary: active paths [/] 2024-08-19T01:37:37.691 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:37:37 smithi083 ceph-mon[17120]: scrub summary: idle+waiting paths [/] 2024-08-19T01:37:37.691 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:37:37 smithi083 ceph-mon[17120]: scrub summary: active paths [/] 2024-08-19T01:37:37.691 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:37:37 smithi083 ceph-mon[17120]: Scrub error on inode 0x10000001d0f (/volumes/qa/sv_1/05b773d9-8346-4397-99ec-d49959a55703/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34) see mds.b log and `damage ls` output for details 2024-08-19T01:37:37.952 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:37:37 smithi117 ceph-mon[22016]: pgmap v422: 97 pgs: 97 active+clean; 194 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.5 MiB/s wr, 392 op/s 2024-08-19T01:37:37.952 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:37:37 smithi117 ceph-mon[22016]: scrub summary: active paths [/] 2024-08-19T01:37:37.952 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:37:37 smithi117 ceph-mon[22016]: scrub summary: idle+waiting paths [/] 2024-08-19T01:37:37.953 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:37:37 smithi117 ceph-mon[22016]: scrub summary: active paths [/] 2024-08-19T01:37:37.953 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:37:37 smithi117 ceph-mon[22016]: scrub summary: idle+waiting paths [/] 2024-08-19T01:37:37.953 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:37:37 smithi117 ceph-mon[22016]: scrub summary: active paths [/] 2024-08-19T01:37:37.953 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:37:37 smithi117 ceph-mon[22016]: Scrub error on inode 0x10000001d0f (/volumes/qa/sv_1/05b773d9-8346-4397-99ec-d49959a55703/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34) see mds.b log and `damage ls` output for details 2024-08-19T01:37:37.965 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:37:37 smithi167 ceph-mon[21875]: pgmap v422: 97 pgs: 97 active+clean; 194 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.5 MiB/s wr, 392 op/s 2024-08-19T01:37:37.966 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:37:37 smithi167 ceph-mon[21875]: scrub summary: active paths [/] 2024-08-19T01:37:37.966 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:37:37 smithi167 ceph-mon[21875]: scrub summary: idle+waiting paths [/] 2024-08-19T01:37:37.966 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:37:37 smithi167 ceph-mon[21875]: scrub summary: active paths [/] 2024-08-19T01:37:37.966 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:37:37 smithi167 ceph-mon[21875]: scrub summary: idle+waiting paths [/] 2024-08-19T01:37:37.966 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:37:37 smithi167 ceph-mon[21875]: scrub summary: active paths [/] 2024-08-19T01:37:37.966 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:37:37 smithi167 ceph-mon[21875]: Scrub error on inode 0x10000001d0f (/volumes/qa/sv_1/05b773d9-8346-4397-99ec-d49959a55703/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34) see mds.b log and `damage ls` output for details 2024-08-19T01:37:38.577 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:37:38 smithi083 ceph-mon[17120]: scrub summary: idle+waiting paths [/] 2024-08-19T01:37:38.577 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:37:38 smithi083 ceph-mon[17120]: scrub summary: active paths [/] 2024-08-19T01:37:38.577 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:37:38 smithi083 ceph-mon[17120]: scrub summary: idle+waiting paths [/] 2024-08-19T01:37:38.952 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:37:38 smithi117 ceph-mon[22016]: scrub summary: idle+waiting paths [/] 2024-08-19T01:37:38.952 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:37:38 smithi117 ceph-mon[22016]: scrub summary: active paths [/] 2024-08-19T01:37:38.952 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:37:38 smithi117 ceph-mon[22016]: scrub summary: idle+waiting paths [/] 2024-08-19T01:37:38.965 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:37:38 smithi167 ceph-mon[21875]: scrub summary: idle+waiting paths [/] 2024-08-19T01:37:38.966 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:37:38 smithi167 ceph-mon[21875]: scrub summary: active paths [/] 2024-08-19T01:37:38.966 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:37:38 smithi167 ceph-mon[21875]: scrub summary: idle+waiting paths [/] 2024-08-19T01:37:39.429 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/bad8780e-5dc9-11ef-bccf-c7b262605968/mon.a/config 2024-08-19T01:37:39.870 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:37:39 smithi083 ceph-mon[17120]: pgmap v423: 97 pgs: 97 active+clean; 191 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.9 MiB/s wr, 660 op/s 2024-08-19T01:37:39.952 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:37:39 smithi117 ceph-mon[22016]: pgmap v423: 97 pgs: 97 active+clean; 191 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.9 MiB/s wr, 660 op/s 2024-08-19T01:37:39.966 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:37:39 smithi167 ceph-mon[21875]: pgmap v423: 97 pgs: 97 active+clean; 191 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.9 MiB/s wr, 660 op/s 2024-08-19T01:37:40.580 INFO:tasks.workunit.client.0.smithi083.stdout: 74 602141 499 394726 391 367436 1318 2024-08-19T01:37:40.914 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:40.911+0000 7f1a24a00640 1 -- 172.21.15.83:0/2269856755 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1a1809a550 msgr2=0x7f1a1809a930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:37:40.914 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:40.911+0000 7f1a24a00640 1 --2- 172.21.15.83:0/2269856755 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1a1809a550 0x7f1a1809a930 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f1a10009890 tx=0x7f1a1002a360 comp rx=0 tx=0).stop 2024-08-19T01:37:40.914 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:40.912+0000 7f1a24a00640 1 -- 172.21.15.83:0/2269856755 shutdown_connections 2024-08-19T01:37:40.914 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:40.912+0000 7f1a24a00640 1 --2- 172.21.15.83:0/2269856755 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1a180952d0 0x7f1a180a1320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:37:40.914 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:40.912+0000 7f1a24a00640 1 --2- 172.21.15.83:0/2269856755 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1a18094930 0x7f1a18094d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:37:40.914 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:40.912+0000 7f1a24a00640 1 --2- 172.21.15.83:0/2269856755 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1a1809a550 0x7f1a1809a930 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:37:40.914 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:40.912+0000 7f1a24a00640 1 -- 172.21.15.83:0/2269856755 >> 172.21.15.83:0/2269856755 conn(0x7f1a1800a1f0 msgr2=0x7f1a1800a600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:37:40.915 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:40.912+0000 7f1a24a00640 1 -- 172.21.15.83:0/2269856755 shutdown_connections 2024-08-19T01:37:40.915 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:40.912+0000 7f1a24a00640 1 -- 172.21.15.83:0/2269856755 wait complete. 2024-08-19T01:37:40.915 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:40.914+0000 7f1a24a00640 1 Processor -- start 2024-08-19T01:37:40.915 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:40.914+0000 7f1a24a00640 1 -- start start 2024-08-19T01:37:40.916 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:40.915+0000 7f1a24a00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1a18094930 0x7f1a180a0ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:37:40.916 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:40.915+0000 7f1a24a00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1a180952d0 0x7f1a1809d090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:37:40.917 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:40.915+0000 7f1a1e800640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1a18094930 0x7f1a180a0ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:37:40.917 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:40.915+0000 7f1a1e800640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1a18094930 0x7f1a180a0ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.83:45900/0 (socket says 172.21.15.83:45900) 2024-08-19T01:37:40.917 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:40.915+0000 7f1a1e800640 1 -- 172.21.15.83:0/257425504 learned_addr learned my addr 172.21.15.83:0/257425504 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-08-19T01:37:40.917 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:40.916+0000 7f1a1de00640 1 --2- 172.21.15.83:0/257425504 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1a180952d0 0x7f1a1809d090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:37:40.917 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:40.916+0000 7f1a24a00640 1 --2- 172.21.15.83:0/257425504 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1a1809a550 0x7f1a1809d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:37:40.932 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:40.916+0000 7f1a1f200640 1 --2- 172.21.15.83:0/257425504 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1a1809a550 0x7f1a1809d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:37:40.932 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:40.917+0000 7f1a24a00640 1 -- 172.21.15.83:0/257425504 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f1a180a3770 con 0x7f1a1809a550 2024-08-19T01:37:40.932 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:40.917+0000 7f1a24a00640 1 -- 172.21.15.83:0/257425504 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f1a180a35f0 con 0x7f1a18094930 2024-08-19T01:37:40.932 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:40.917+0000 7f1a24a00640 1 -- 172.21.15.83:0/257425504 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f1a180a38f0 con 0x7f1a180952d0 2024-08-19T01:37:40.932 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:40.918+0000 7f1a1f200640 1 -- 172.21.15.83:0/257425504 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1a180952d0 msgr2=0x7f1a1809d090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:37:40.932 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:40.918+0000 7f1a1f200640 1 --2- 172.21.15.83:0/257425504 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1a180952d0 0x7f1a1809d090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:37:40.932 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:40.918+0000 7f1a1f200640 1 -- 172.21.15.83:0/257425504 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1a18094930 msgr2=0x7f1a180a0ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:37:40.933 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:40.918+0000 7f1a1f200640 1 --2- 172.21.15.83:0/257425504 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1a18094930 0x7f1a180a0ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:37:40.933 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:40.918+0000 7f1a1f200640 1 -- 172.21.15.83:0/257425504 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1a1809dd00 con 0x7f1a1809a550 2024-08-19T01:37:40.933 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:40.922+0000 7f1a1de00640 1 --2- 172.21.15.83:0/257425504 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1a180952d0 0x7f1a1809d090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-19T01:37:40.933 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:40.922+0000 7f1a1e800640 1 --2- 172.21.15.83:0/257425504 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1a18094930 0x7f1a180a0ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-19T01:37:40.933 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:40.923+0000 7f1a1f200640 1 --2- 172.21.15.83:0/257425504 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1a1809a550 0x7f1a1809d5d0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f1a20067690 tx=0x7f1a2007ca60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:37:40.933 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:40.924+0000 7f1a17400640 1 -- 172.21.15.83:0/257425504 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a2007a8a0 con 0x7f1a1809a550 2024-08-19T01:37:40.933 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:40.924+0000 7f1a17400640 1 -- 172.21.15.83:0/257425504 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1a20087450 con 0x7f1a1809a550 2024-08-19T01:37:40.933 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:40.924+0000 7f1a24a00640 1 -- 172.21.15.83:0/257425504 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1a1813ea70 con 0x7f1a1809a550 2024-08-19T01:37:40.933 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:40.924+0000 7f1a24a00640 1 -- 172.21.15.83:0/257425504 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1a1813eed0 con 0x7f1a1809a550 2024-08-19T01:37:40.935 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:40.929+0000 7f1a17400640 1 -- 172.21.15.83:0/257425504 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a20078620 con 0x7f1a1809a550 2024-08-19T01:37:40.953 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:40.934+0000 7f1a17400640 1 -- 172.21.15.83:0/257425504 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7f1a200787c0 con 0x7f1a1809a550 2024-08-19T01:37:40.953 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:40.937+0000 7f1a28055740 1 Processor -- start 2024-08-19T01:37:40.953 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:40.938+0000 7f1a17400640 1 --2- 172.21.15.83:0/257425504 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f1a040a2d20 0x7f1a040a51e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:37:40.953 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:40.938+0000 7f1a17400640 1 -- 172.21.15.83:0/257425504 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8269+0+0 (secure 0 0 0) 0x7f1a2013b990 con 0x7f1a1809a550 2024-08-19T01:37:40.953 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:40.939+0000 7f1a28055740 1 -- start start 2024-08-19T01:37:40.953 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:40.939+0000 7f1a28055740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5624effb6ba0 0x5624effb6f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:37:40.953 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:40.940+0000 7f1a1de00640 1 --2- 172.21.15.83:0/257425504 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f1a040a2d20 0x7f1a040a51e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:37:40.953 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:40.941+0000 7f1a1e800640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5624effb6ba0 0x5624effb6f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:37:40.953 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:40.941+0000 7f1a1e800640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5624effb6ba0 0x5624effb6f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.83:55688/0 (socket says 172.21.15.83:55688) 2024-08-19T01:37:40.954 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:40.941+0000 7f1a28055740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5624effb74c0 0x5624effd29f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:37:40.954 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:40.941+0000 7f1a1e800640 1 -- 172.21.15.83:0/2040485290 learned_addr learned my addr 172.21.15.83:0/2040485290 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-08-19T01:37:40.954 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:40.941+0000 7f1a28055740 1 --2- 172.21.15.83:0/2040485290 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5624effd2f30 0x5624effd5320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:37:40.959 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:40.941+0000 7f1a1f200640 1 --2- 172.21.15.83:0/2040485290 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5624effd2f30 0x5624effd5320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:37:40.959 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:40.943+0000 7f1a28055740 1 -- 172.21.15.83:0/2040485290 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x5624eff1cbf0 con 0x5624effb74c0 2024-08-19T01:37:40.959 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:40.942+0000 7f1a1de00640 1 --2- 172.21.15.83:0/257425504 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f1a040a2d20 0x7f1a040a51e0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f1a1809e850 tx=0x7f1a0c005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:37:40.959 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:40.943+0000 7f1a28055740 1 -- 172.21.15.83:0/2040485290 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x5624efeb4bd0 con 0x5624effd2f30 2024-08-19T01:37:40.959 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:40.943+0000 7f1a28055740 1 -- 172.21.15.83:0/2040485290 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x5624eff18bb0 con 0x5624effb6ba0 2024-08-19T01:37:40.960 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:40.943+0000 7f1a1e800640 1 -- 172.21.15.83:0/2040485290 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5624effd2f30 msgr2=0x5624effd5320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:37:40.960 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:40.943+0000 7f1a1de00640 1 --2- 172.21.15.83:0/2040485290 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5624effb74c0 0x5624effd29f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:37:40.960 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:40.948+0000 7f1a1e800640 1 --2- 172.21.15.83:0/2040485290 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5624effd2f30 0x5624effd5320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:37:40.960 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:40.948+0000 7f1a1e800640 1 -- 172.21.15.83:0/2040485290 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5624effb74c0 msgr2=0x5624effd29f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:37:40.960 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:40.948+0000 7f1a1e800640 1 --2- 172.21.15.83:0/2040485290 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5624effb74c0 0x5624effd29f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:37:40.960 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:40.948+0000 7f1a1e800640 1 -- 172.21.15.83:0/2040485290 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5624efebb8a0 con 0x5624effb6ba0 2024-08-19T01:37:40.960 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:40.948+0000 7f1a1de00640 1 --2- 172.21.15.83:0/2040485290 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5624effb74c0 0x5624effd29f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-19T01:37:40.960 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:40.948+0000 7f1a1f200640 1 --2- 172.21.15.83:0/2040485290 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5624effd2f30 0x5624effd5320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-19T01:37:40.960 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:40.949+0000 7f1a1e800640 1 --2- 172.21.15.83:0/2040485290 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5624effb6ba0 0x5624effb6f80 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f1a10000c00 tx=0x7f1a10005c90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=767886086b6ed9b9 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:37:40.960 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:40.949+0000 7f1a0b400640 1 -- 172.21.15.83:0/2040485290 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a10037040 con 0x5624effb6ba0 2024-08-19T01:37:40.961 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:40.950+0000 7f1a0b400640 1 -- 172.21.15.83:0/2040485290 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1a10030c80 con 0x5624effb6ba0 2024-08-19T01:37:40.961 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:40.950+0000 7f1a0b400640 1 -- 172.21.15.83:0/2040485290 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a100046c0 con 0x5624effb6ba0 2024-08-19T01:37:40.961 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:40.951+0000 7f1a28055740 1 -- 172.21.15.83:0/2040485290 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5624effb6ba0 msgr2=0x5624effb6f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:37:40.961 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:40.951+0000 7f1a28055740 1 --2- 172.21.15.83:0/2040485290 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5624effb6ba0 0x5624effb6f80 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f1a10000c00 tx=0x7f1a10005c90 comp rx=0 tx=0).stop 2024-08-19T01:37:40.961 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:40.952+0000 7f1a28055740 1 -- 172.21.15.83:0/2040485290 shutdown_connections 2024-08-19T01:37:40.961 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:40.952+0000 7f1a28055740 1 --2- 172.21.15.83:0/2040485290 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5624effd2f30 0x5624effd5320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:37:40.961 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:40.952+0000 7f1a28055740 1 --2- 172.21.15.83:0/2040485290 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5624effb74c0 0x5624effd29f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:37:40.965 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:40.952+0000 7f1a28055740 1 --2- 172.21.15.83:0/2040485290 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5624effb6ba0 0x5624effb6f80 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:37:40.965 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:40.952+0000 7f1a28055740 1 -- 172.21.15.83:0/2040485290 >> 172.21.15.83:0/2040485290 conn(0x5624effb32d0 msgr2=0x5624effb36b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:37:40.965 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:40.952+0000 7f1a28055740 1 -- 172.21.15.83:0/2040485290 shutdown_connections 2024-08-19T01:37:40.965 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:40.953+0000 7f1a28055740 1 -- 172.21.15.83:0/2040485290 wait complete. 2024-08-19T01:37:40.965 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:40.954+0000 7f1a28055740 1 -- start start 2024-08-19T01:37:40.965 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:40.956+0000 7f1a28055740 1 Processor -- start 2024-08-19T01:37:40.965 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:40.957+0000 7f1a28055740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5624effb6ba0 0x5624effcfd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:37:40.965 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:40.958+0000 7f1a28055740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5624effb74c0 0x5624effd0240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:37:40.965 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:40.958+0000 7f1a1e800640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5624effb6ba0 0x5624effcfd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:37:40.965 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:40.958+0000 7f1a1e800640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5624effb6ba0 0x5624effcfd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.83:55704/0 (socket says 172.21.15.83:55704) 2024-08-19T01:37:40.965 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:40.958+0000 7f1a1de00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5624effb74c0 0x5624effd0240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:37:40.966 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:40.958+0000 7f1a1de00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5624effb74c0 0x5624effd0240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:48254/0 (socket says 172.21.15.83:48254) 2024-08-19T01:37:40.966 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:40.958+0000 7f1a1de00640 1 -- 172.21.15.83:0/156829341 learned_addr learned my addr 172.21.15.83:0/156829341 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-08-19T01:37:40.966 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:40.959+0000 7f1a28055740 1 --2- 172.21.15.83:0/156829341 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5624effd2f30 0x5624effd0780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:37:40.966 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:40.959+0000 7f1a28055740 1 -- 172.21.15.83:0/156829341 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x5624effd5b00 con 0x5624effb74c0 2024-08-19T01:37:40.966 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:40.959+0000 7f1a28055740 1 -- 172.21.15.83:0/156829341 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x5624efed7d40 con 0x5624effd2f30 2024-08-19T01:37:40.966 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:40.959+0000 7f1a28055740 1 -- 172.21.15.83:0/156829341 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x5624effd5c80 con 0x5624effb6ba0 2024-08-19T01:37:40.966 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:40.959+0000 7f1a1f200640 1 --2- 172.21.15.83:0/156829341 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5624effd2f30 0x5624effd0780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:37:40.966 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:40.959+0000 7f1a28055740 10 client.0 _ll_register_callbacks cb 0x5624efea0b10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-19T01:37:40.966 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:40.960+0000 7f1a1f200640 1 -- 172.21.15.83:0/156829341 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5624effb6ba0 msgr2=0x5624effcfd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:37:40.966 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:40.960+0000 7f1a1f200640 1 --2- 172.21.15.83:0/156829341 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5624effb6ba0 0x5624effcfd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:37:40.966 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:40.960+0000 7f1a1f200640 1 -- 172.21.15.83:0/156829341 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5624effb74c0 msgr2=0x5624effd0240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:37:40.966 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:40.960+0000 7f1a1f200640 1 --2- 172.21.15.83:0/156829341 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5624effb74c0 0x5624effd0240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:37:40.966 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:40.960+0000 7f1a1f200640 1 -- 172.21.15.83:0/156829341 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5624efc8e3c0 con 0x5624effd2f30 2024-08-19T01:37:40.967 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:40.960+0000 7f1a1e800640 1 --2- 172.21.15.83:0/156829341 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5624effb6ba0 0x5624effcfd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-19T01:37:40.967 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:40.960+0000 7f1a1de00640 1 --2- 172.21.15.83:0/156829341 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5624effb74c0 0x5624effd0240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-19T01:37:40.967 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:40.960+0000 7f19ffe00640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-08-19T01:37:40.967 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:40.961+0000 7f1a1f200640 1 --2- 172.21.15.83:0/156829341 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5624effd2f30 0x5624effd0780 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f1a201394c0 tx=0x7f1a201093f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b8cbfe994e42ef9e server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:37:40.967 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:40.961+0000 7f19ffe00640 1 -- 172.21.15.83:0/156829341 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a20121070 con 0x5624effd2f30 2024-08-19T01:37:40.967 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:40.961+0000 7f19ffe00640 1 -- 172.21.15.83:0/156829341 <== mon.2 v2:172.21.15.117:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8269+0+0 (secure 0 0 0) 0x7f1a20122260 con 0x5624effd2f30 2024-08-19T01:37:40.967 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:40.961+0000 7f1a28055740 1 -- 172.21.15.83:0/156829341 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1a10030c80 con 0x5624effd2f30 2024-08-19T01:37:40.967 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:40.962+0000 7f19ffe00640 1 -- 172.21.15.83:0/156829341 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f1a20109560 con 0x5624effd2f30 2024-08-19T01:37:40.967 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:40.962+0000 7f1a28055740 10 client.25604 fetch_fsmap learned FSMap version 26 2024-08-19T01:37:40.967 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:40.962+0000 7f1a28055740 1 -- 172.21.15.83:0/156829341 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5624effe16b0 con 0x5624effd2f30 2024-08-19T01:37:40.967 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:40.963+0000 7f19ffe00640 1 -- 172.21.15.83:0/156829341 <== mon.2 v2:172.21.15.117:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1a10030c80 con 0x5624effd2f30 2024-08-19T01:37:40.968 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:40.963+0000 7f19ffe00640 1 -- 172.21.15.83:0/156829341 <== mon.2 v2:172.21.15.117:3300/0 5 ==== fsmap(e 26) ==== 8350+0+0 (secure 0 0 0) 0x7f1a2008f030 con 0x5624effd2f30 2024-08-19T01:37:40.968 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:40.963+0000 7f1a28055740 10 client.25604 fetch_fsmap finished waiting for FSMap version 26 2024-08-19T01:37:40.968 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:40.963+0000 7f1a28055740 10 client.25604 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-19T01:37:40.968 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:40.963+0000 7f1a28055740 20 client.25604 populate_metadata read hostname 'smithi083' 2024-08-19T01:37:40.968 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:40.963+0000 7f1a28055740 1 --2- 172.21.15.83:0/156829341 >> [v2:172.21.15.117:6838/1967595704,v1:172.21.15.117:6839/1967595704] conn(0x5624effe2b10 0x5624effe4f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:37:40.968 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:40.963+0000 7f1a28055740 4 client.25604 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-19T01:37:40.968 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:40.963+0000 7f1a28055740 1 -- 172.21.15.83:0/156829341 --> [v2:172.21.15.117:6838/1967595704,v1:172.21.15.117:6839/1967595704] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5624eff2a160 con 0x5624effe2b10 2024-08-19T01:37:40.968 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:40.964+0000 7f1a1e800640 1 --2- 172.21.15.83:0/156829341 >> [v2:172.21.15.117:6838/1967595704,v1:172.21.15.117:6839/1967595704] conn(0x5624effe2b10 0x5624effe4f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:37:40.968 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:40.966+0000 7f1a1e800640 1 --2- 172.21.15.83:0/156829341 >> [v2:172.21.15.117:6838/1967595704,v1:172.21.15.117:6839/1967595704] conn(0x5624effe2b10 0x5624effe4f70 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a280cc0a61a6ccc4 server_cookie=5ccbf2a4301fb6d6 in_seq=0 out_seq=0 2024-08-19T01:37:40.968 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:40.966+0000 7f19ffe00640 10 client.25604 ms_handle_connect on v2:172.21.15.117:6838/1967595704 2024-08-19T01:37:40.970 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:40.968+0000 7f19ffe00640 1 -- 172.21.15.83:0/156829341 <== mds.0 v2:172.21.15.117:6838/1967595704 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5624eff2a160 con 0x5624effe2b10 2024-08-19T01:37:40.970 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:40.968+0000 7f19ffe00640 10 client.25604 handle_command_reply: tid=0 2024-08-19T01:37:40.970 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:40.968+0000 7f1a28055740 1 client.25604 shutdown 2024-08-19T01:37:40.972 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:40.969+0000 7f1a28055740 1 -- 172.21.15.83:0/156829341 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5624effd2f30 msgr2=0x5624effd0780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:37:40.972 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:40.969+0000 7f1a28055740 1 --2- 172.21.15.83:0/156829341 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5624effd2f30 0x5624effd0780 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f1a201394c0 tx=0x7f1a201093f0 comp rx=0 tx=0).stop 2024-08-19T01:37:40.972 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:40.969+0000 7f1a28055740 1 -- 172.21.15.83:0/156829341 shutdown_connections 2024-08-19T01:37:40.972 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:40.969+0000 7f1a28055740 1 --2- 172.21.15.83:0/156829341 >> [v2:172.21.15.117:6838/1967595704,v1:172.21.15.117:6839/1967595704] conn(0x5624effe2b10 0x5624effe4f70 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:37:40.972 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:40.969+0000 7f1a28055740 1 --2- 172.21.15.83:0/156829341 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5624effd2f30 0x5624effd0780 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:37:40.972 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:40.969+0000 7f1a28055740 1 --2- 172.21.15.83:0/156829341 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5624effb74c0 0x5624effd0240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:37:40.972 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:40.969+0000 7f1a28055740 1 --2- 172.21.15.83:0/156829341 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5624effb6ba0 0x5624effcfd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:37:40.972 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:40.969+0000 7f1a28055740 1 -- 172.21.15.83:0/156829341 >> 172.21.15.83:0/156829341 conn(0x5624effb32d0 msgr2=0x5624effb8ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:37:40.972 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:40.969+0000 7f19ffe00640 0 client.25604 ms_handle_reset on v2:172.21.15.117:6838/1967595704 2024-08-19T01:37:40.973 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:40.970+0000 7f1a28055740 1 -- 172.21.15.83:0/156829341 shutdown_connections 2024-08-19T01:37:40.973 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:40.970+0000 7f1a28055740 1 -- 172.21.15.83:0/156829341 wait complete. 2024-08-19T01:37:40.973 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:40.970+0000 7f1a28055740 20 client.25604 trim_cache size 0 max 16384 2024-08-19T01:37:41.022 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:41.015+0000 7f1a28055740 1 Processor -- start 2024-08-19T01:37:41.022 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:41.015+0000 7f1a28055740 1 -- start start 2024-08-19T01:37:41.022 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:41.015+0000 7f1a28055740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5624effb32d0 0x5624effcb010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:37:41.022 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:41.015+0000 7f1a28055740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5624effb6ba0 0x5624effea880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:37:41.023 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:41.015+0000 7f1a28055740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5624effb74c0 0x5624effeadc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:37:41.023 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:41.015+0000 7f1a28055740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f1a00010de0 con 0x5624effb32d0 2024-08-19T01:37:41.023 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:41.015+0000 7f1a28055740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f1a0000e6b0 con 0x5624effb74c0 2024-08-19T01:37:41.023 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:41.015+0000 7f1a28055740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f1a0000c500 con 0x5624effb6ba0 2024-08-19T01:37:41.023 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:41.016+0000 7f1a1f200640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5624effb74c0 0x5624effeadc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:37:41.023 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:41.016+0000 7f1a1f200640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5624effb74c0 0x5624effeadc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.83:45926/0 (socket says 172.21.15.83:45926) 2024-08-19T01:37:41.023 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:41.016+0000 7f1a1f200640 1 -- 172.21.15.83:0/434452696 learned_addr learned my addr 172.21.15.83:0/434452696 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-08-19T01:37:41.023 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:41.016+0000 7f1a1e800640 1 --2- 172.21.15.83:0/434452696 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5624effb32d0 0x5624effcb010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:37:41.023 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:41.017+0000 7f1a1e800640 1 -- 172.21.15.83:0/434452696 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5624effb6ba0 msgr2=0x5624effea880 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-08-19T01:37:41.023 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:41.017+0000 7f1a1e800640 1 --2- 172.21.15.83:0/434452696 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5624effb6ba0 0x5624effea880 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:37:41.023 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:41.017+0000 7f1a1e800640 1 -- 172.21.15.83:0/434452696 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5624effb74c0 msgr2=0x5624effeadc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:37:41.023 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:41.017+0000 7f1a1e800640 1 --2- 172.21.15.83:0/434452696 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5624effb74c0 0x5624effeadc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:37:41.024 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:41.017+0000 7f1a1e800640 1 -- 172.21.15.83:0/434452696 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5624effb3100 con 0x5624effb32d0 2024-08-19T01:37:41.024 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:41.018+0000 7f1a1e800640 1 --2- 172.21.15.83:0/434452696 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5624effb32d0 0x5624effcb010 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f1a1002a950 tx=0x7f1a10005240 comp rx=0 tx=0).ready entity=mon.0 client_cookie=adce44154a796e30 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:37:41.024 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:41.018+0000 7f1a0aa00640 1 -- 172.21.15.83:0/434452696 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a10037040 con 0x5624effb32d0 2024-08-19T01:37:41.024 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:41.019+0000 7f1a0aa00640 1 -- 172.21.15.83:0/434452696 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1a10048070 con 0x5624effb32d0 2024-08-19T01:37:41.024 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:41.019+0000 7f1a0aa00640 1 -- 172.21.15.83:0/434452696 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a10004920 con 0x5624effb32d0 2024-08-19T01:37:41.024 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:41.020+0000 7f1a28055740 1 -- 172.21.15.83:0/434452696 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5624effb32d0 msgr2=0x5624effcb010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:37:41.024 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:41.020+0000 7f1a28055740 1 --2- 172.21.15.83:0/434452696 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5624effb32d0 0x5624effcb010 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f1a1002a950 tx=0x7f1a10005240 comp rx=0 tx=0).stop 2024-08-19T01:37:41.024 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:41.023+0000 7f1a28055740 1 -- 172.21.15.83:0/434452696 shutdown_connections 2024-08-19T01:37:41.025 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:41.023+0000 7f1a28055740 1 --2- 172.21.15.83:0/434452696 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5624effb74c0 0x5624effeadc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:37:41.025 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:41.023+0000 7f1a28055740 1 --2- 172.21.15.83:0/434452696 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5624effb6ba0 0x5624effea880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:37:41.025 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:41.023+0000 7f1a28055740 1 --2- 172.21.15.83:0/434452696 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5624effb32d0 0x5624effcb010 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:37:41.025 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:41.023+0000 7f1a28055740 1 -- 172.21.15.83:0/434452696 >> 172.21.15.83:0/434452696 conn(0x5624effcb6f0 msgr2=0x5624effcedf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:37:41.025 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:41.023+0000 7f1a28055740 1 -- 172.21.15.83:0/434452696 shutdown_connections 2024-08-19T01:37:41.030 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:41.023+0000 7f1a28055740 1 -- 172.21.15.83:0/434452696 wait complete. 2024-08-19T01:37:41.030 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:41.024+0000 7f1a28055740 1 -- start start 2024-08-19T01:37:41.030 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:41.024+0000 7f1a28055740 1 Processor -- start 2024-08-19T01:37:41.031 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:41.025+0000 7f1a28055740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5624effb32d0 0x5624effcc330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:37:41.031 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:41.026+0000 7f1a28055740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5624effb6ba0 0x5624effcc870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:37:41.031 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:41.026+0000 7f1a28055740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5624effb74c0 0x5624effd9fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:37:41.031 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:41.026+0000 7f1a1de00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5624effb6ba0 0x5624effcc870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:37:41.031 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:41.026+0000 7f1a1de00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5624effb6ba0 0x5624effcc870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.83:55716/0 (socket says 172.21.15.83:55716) 2024-08-19T01:37:41.031 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:41.027+0000 7f1a28055740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f1a0000cb70 con 0x5624effb74c0 2024-08-19T01:37:41.031 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:41.027+0000 7f1a1e800640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5624effb32d0 0x5624effcc330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:37:41.031 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:41.027+0000 7f1a1e800640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5624effb32d0 0x5624effcc330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.83:45938/0 (socket says 172.21.15.83:45938) 2024-08-19T01:37:41.031 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:41.027+0000 7f1a1e800640 1 -- 172.21.15.83:0/217801542 learned_addr learned my addr 172.21.15.83:0/217801542 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-08-19T01:37:41.031 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:41.027+0000 7f1a28055740 1 -- 172.21.15.83:0/217801542 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f1a0000dfe0 con 0x5624effb32d0 2024-08-19T01:37:41.031 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:41.028+0000 7f1a28055740 1 -- 172.21.15.83:0/217801542 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f1a0000d240 con 0x5624effb6ba0 2024-08-19T01:37:41.031 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:41.028+0000 7f1a28055740 10 client.0 _ll_register_callbacks cb 0x7f1a200678a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-19T01:37:41.032 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:41.028+0000 7f1a1f200640 1 --2- 172.21.15.83:0/217801542 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5624effb74c0 0x5624effd9fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:37:41.034 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:41.028+0000 7f1a1e800640 1 -- 172.21.15.83:0/217801542 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5624effb6ba0 msgr2=0x5624effcc870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:37:41.034 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:41.028+0000 7f1a1e800640 1 --2- 172.21.15.83:0/217801542 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5624effb6ba0 0x5624effcc870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:37:41.034 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:41.028+0000 7f1a1e800640 1 -- 172.21.15.83:0/217801542 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5624effb74c0 msgr2=0x5624effd9fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:37:41.034 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:41.032+0000 7f1a1e800640 1 --2- 172.21.15.83:0/217801542 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5624effb74c0 0x5624effd9fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:37:41.034 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:41.032+0000 7f1a1e800640 1 -- 172.21.15.83:0/217801542 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5624effda4e0 con 0x5624effb32d0 2024-08-19T01:37:41.035 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:41.032+0000 7f1a28055740 1 -- 172.21.15.83:0/217801542 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1a10048070 con 0x5624effb32d0 2024-08-19T01:37:41.035 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:41.032+0000 7f1a1f200640 1 --2- 172.21.15.83:0/217801542 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5624effb74c0 0x5624effd9fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-19T01:37:41.040 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:41.033+0000 7f1a1e800640 1 --2- 172.21.15.83:0/217801542 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5624effb32d0 0x5624effcc330 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f1a10005cd0 tx=0x7f1a10030470 comp rx=0 tx=0).ready entity=mon.2 client_cookie=24d98d24fa1d64cb server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:37:41.040 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:41.033+0000 7f19ffe00640 10 client.25610 ms_handle_connect on v2:172.21.15.117:3300/0 2024-08-19T01:37:41.041 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:41.036+0000 7f19ffe00640 1 -- 172.21.15.83:0/217801542 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a10037070 con 0x5624effb32d0 2024-08-19T01:37:41.041 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:41.036+0000 7f19ffe00640 1 -- 172.21.15.83:0/217801542 <== mon.2 v2:172.21.15.117:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8269+0+0 (secure 0 0 0) 0x7f1a10004230 con 0x5624effb32d0 2024-08-19T01:37:41.041 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:41.036+0000 7f19ffe00640 1 -- 172.21.15.83:0/217801542 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f1a10004430 con 0x5624effb32d0 2024-08-19T01:37:41.041 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:41.036+0000 7f1a28055740 10 client.25610 fetch_fsmap learned FSMap version 26 2024-08-19T01:37:41.041 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:41.036+0000 7f1a28055740 1 -- 172.21.15.83:0/217801542 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({fsmap=26}) -- 0x5624effdb6d0 con 0x5624effb32d0 2024-08-19T01:37:41.041 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:41.037+0000 7f19ffe00640 1 -- 172.21.15.83:0/217801542 <== mon.2 v2:172.21.15.117:3300/0 4 ==== fsmap(e 26) ==== 8350+0+0 (secure 0 0 0) 0x7f1a10056030 con 0x5624effb32d0 2024-08-19T01:37:41.041 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:41.037+0000 7f1a28055740 10 client.25610 fetch_fsmap finished waiting for FSMap version 26 2024-08-19T01:37:41.041 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:41.037+0000 7f1a28055740 10 client.25610 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-19T01:37:41.042 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:41.037+0000 7f1a28055740 20 client.25610 populate_metadata read hostname 'smithi083' 2024-08-19T01:37:41.042 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:41.037+0000 7f1a28055740 1 --2- 172.21.15.83:0/217801542 >> [v2:172.21.15.117:6838/1967595704,v1:172.21.15.117:6839/1967595704] conn(0x5624effd2f30 0x5624effdb8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:37:41.042 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:41.037+0000 7f1a28055740 4 client.25610 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-08-19T01:37:41.042 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:41.037+0000 7f1a28055740 1 -- 172.21.15.83:0/217801542 --> [v2:172.21.15.117:6838/1967595704,v1:172.21.15.117:6839/1967595704] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5624eff24670 con 0x5624effd2f30 2024-08-19T01:37:41.042 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:41.038+0000 7f1a1de00640 1 --2- 172.21.15.83:0/217801542 >> [v2:172.21.15.117:6838/1967595704,v1:172.21.15.117:6839/1967595704] conn(0x5624effd2f30 0x5624effdb8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:37:41.042 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:41.039+0000 7f1a1de00640 1 --2- 172.21.15.83:0/217801542 >> [v2:172.21.15.117:6838/1967595704,v1:172.21.15.117:6839/1967595704] conn(0x5624effd2f30 0x5624effdb8a0 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=282b313b36ae188e server_cookie=9b0c2bff8c0ae0f0 in_seq=0 out_seq=0 2024-08-19T01:37:41.042 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:41.039+0000 7f19ffe00640 10 client.25610 ms_handle_connect on v2:172.21.15.117:6838/1967595704 2024-08-19T01:37:41.052 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-08-19T01:37:41.052 INFO:teuthology.orchestra.run.smithi083.stdout: "status": "scrub active (5428 inodes in the stack)", 2024-08-19T01:37:41.052 INFO:teuthology.orchestra.run.smithi083.stdout: "scrubs": { 2024-08-19T01:37:41.052 INFO:teuthology.orchestra.run.smithi083.stdout: "28dac109-a1c6-4541-bf5f-b2c13f258789": { 2024-08-19T01:37:41.052 INFO:teuthology.orchestra.run.smithi083.stdout: "path": "/", 2024-08-19T01:37:41.052 INFO:teuthology.orchestra.run.smithi083.stdout: "tag": "28dac109-a1c6-4541-bf5f-b2c13f258789", 2024-08-19T01:37:41.052 INFO:teuthology.orchestra.run.smithi083.stdout: "options": "recursive,force" 2024-08-19T01:37:41.052 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-08-19T01:37:41.053 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-08-19T01:37:41.053 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-08-19T01:37:41.053 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:41.043+0000 7f19ffe00640 1 -- 172.21.15.83:0/217801542 <== mds.0 v2:172.21.15.117:6838/1967595704 1 ==== command_reply(tid 0: 0 ) ==== 8+0+268 (crc 0 0 0) 0x5624eff24670 con 0x5624effd2f30 2024-08-19T01:37:41.053 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:41.043+0000 7f19ffe00640 10 client.25610 handle_command_reply: tid=0 2024-08-19T01:37:41.053 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:41.044+0000 7f1a28055740 1 client.25610 shutdown 2024-08-19T01:37:41.053 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:41.047+0000 7f1a28055740 1 -- 172.21.15.83:0/217801542 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5624effb32d0 msgr2=0x5624effcc330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:37:41.053 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:41.047+0000 7f1a28055740 1 --2- 172.21.15.83:0/217801542 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5624effb32d0 0x5624effcc330 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f1a10005cd0 tx=0x7f1a10030470 comp rx=0 tx=0).stop 2024-08-19T01:37:41.053 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:41.047+0000 7f1a28055740 1 -- 172.21.15.83:0/217801542 shutdown_connections 2024-08-19T01:37:41.053 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:41.047+0000 7f1a28055740 1 --2- 172.21.15.83:0/217801542 >> [v2:172.21.15.117:6838/1967595704,v1:172.21.15.117:6839/1967595704] conn(0x5624effd2f30 0x5624effdb8a0 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:37:41.053 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:41.047+0000 7f1a28055740 1 --2- 172.21.15.83:0/217801542 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5624effb74c0 0x5624effd9fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:37:41.053 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:41.047+0000 7f1a28055740 1 --2- 172.21.15.83:0/217801542 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5624effb6ba0 0x5624effcc870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:37:41.053 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:41.047+0000 7f1a28055740 1 --2- 172.21.15.83:0/217801542 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5624effb32d0 0x5624effcc330 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:37:41.054 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:41.047+0000 7f1a28055740 1 -- 172.21.15.83:0/217801542 >> 172.21.15.83:0/217801542 conn(0x5624effcb6f0 msgr2=0x5624effb87d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:37:41.054 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:41.047+0000 7f19ffe00640 0 client.25610 ms_handle_reset on v2:172.21.15.117:6838/1967595704 2024-08-19T01:37:41.054 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:41.047+0000 7f1a28055740 1 -- 172.21.15.83:0/217801542 shutdown_connections 2024-08-19T01:37:41.054 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:41.048+0000 7f1a28055740 1 -- 172.21.15.83:0/217801542 wait complete. 2024-08-19T01:37:41.054 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:41.048+0000 7f1a28055740 20 client.25610 trim_cache size 0 max 16384 2024-08-19T01:37:41.054 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:41.049+0000 7f1a24a00640 1 -- 172.21.15.83:0/257425504 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f1a040a2d20 msgr2=0x7f1a040a51e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:37:41.054 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:41.049+0000 7f1a24a00640 1 --2- 172.21.15.83:0/257425504 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f1a040a2d20 0x7f1a040a51e0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f1a1809e850 tx=0x7f1a0c005f90 comp rx=0 tx=0).stop 2024-08-19T01:37:41.054 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:41.049+0000 7f1a24a00640 1 -- 172.21.15.83:0/257425504 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1a1809a550 msgr2=0x7f1a1809d5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:37:41.054 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:41.049+0000 7f1a24a00640 1 --2- 172.21.15.83:0/257425504 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1a1809a550 0x7f1a1809d5d0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f1a20067690 tx=0x7f1a2007ca60 comp rx=0 tx=0).stop 2024-08-19T01:37:41.054 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:41.049+0000 7f1a24a00640 1 -- 172.21.15.83:0/257425504 shutdown_connections 2024-08-19T01:37:41.054 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:41.049+0000 7f1a24a00640 1 --2- 172.21.15.83:0/257425504 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f1a040a2d20 0x7f1a040a51e0 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:37:41.054 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:41.049+0000 7f1a24a00640 1 --2- 172.21.15.83:0/257425504 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1a1809a550 0x7f1a1809d5d0 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:37:41.054 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:41.049+0000 7f1a24a00640 1 --2- 172.21.15.83:0/257425504 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1a180952d0 0x7f1a1809d090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:37:41.055 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:41.049+0000 7f1a24a00640 1 --2- 172.21.15.83:0/257425504 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1a18094930 0x7f1a180a0ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:37:41.055 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:41.049+0000 7f1a24a00640 1 -- 172.21.15.83:0/257425504 >> 172.21.15.83:0/257425504 conn(0x7f1a1800a1f0 msgr2=0x7f1a18093590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:37:41.055 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:41.049+0000 7f1a24a00640 1 -- 172.21.15.83:0/257425504 shutdown_connections 2024-08-19T01:37:41.055 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:37:41.049+0000 7f1a24a00640 1 -- 172.21.15.83:0/257425504 wait complete. 2024-08-19T01:37:41.870 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:37:41 smithi083 ceph-mon[17120]: pgmap v424: 97 pgs: 97 active+clean; 178 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 2.3 MiB/s wr, 725 op/s 2024-08-19T01:37:41.952 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:37:41 smithi117 ceph-mon[22016]: pgmap v424: 97 pgs: 97 active+clean; 178 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 2.3 MiB/s wr, 725 op/s 2024-08-19T01:37:41.965 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:37:41 smithi167 ceph-mon[21875]: pgmap v424: 97 pgs: 97 active+clean; 178 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 2.3 MiB/s wr, 725 op/s 2024-08-19T01:37:42.682 INFO:tasks.cephfs.filesystem:scrub status for tag:28dac109-a1c6-4541-bf5f-b2c13f258789 - {'path': '/', 'tag': '28dac109-a1c6-4541-bf5f-b2c13f258789', 'options': 'recursive,force'} 2024-08-19T01:37:42.904 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:37:42 smithi083 ceph-mon[17120]: pgmap v425: 97 pgs: 97 active+clean; 177 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.6 MiB/s wr, 774 op/s 2024-08-19T01:37:42.952 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:37:42 smithi117 ceph-mon[22016]: pgmap v425: 97 pgs: 97 active+clean; 177 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.6 MiB/s wr, 774 op/s 2024-08-19T01:37:42.965 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:37:42 smithi167 ceph-mon[21875]: pgmap v425: 97 pgs: 97 active+clean; 177 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.6 MiB/s wr, 774 op/s 2024-08-19T01:37:45.655 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:37:45 smithi083 ceph-mon[17120]: pgmap v426: 97 pgs: 97 active+clean; 184 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 2.2 MiB/s wr, 1.08k op/s 2024-08-19T01:37:45.952 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:37:45 smithi117 ceph-mon[22016]: pgmap v426: 97 pgs: 97 active+clean; 184 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 2.2 MiB/s wr, 1.08k op/s 2024-08-19T01:37:45.965 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:37:45 smithi167 ceph-mon[21875]: pgmap v426: 97 pgs: 97 active+clean; 184 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 2.2 MiB/s wr, 1.08k op/s 2024-08-19T01:37:47.577 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:37:47 smithi083 ceph-mon[17120]: scrub summary: idle+waiting paths [/] 2024-08-19T01:37:47.577 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:37:47 smithi083 ceph-mon[17120]: pgmap v427: 97 pgs: 97 active+clean; 184 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 1.6 MiB/s wr, 1.12k op/s 2024-08-19T01:37:47.952 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:37:47 smithi117 ceph-mon[22016]: scrub summary: idle+waiting paths [/] 2024-08-19T01:37:47.952 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:37:47 smithi117 ceph-mon[22016]: pgmap v427: 97 pgs: 97 active+clean; 184 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 1.6 MiB/s wr, 1.12k op/s 2024-08-19T01:37:47.965 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:37:47 smithi167 ceph-mon[21875]: scrub summary: idle+waiting paths [/] 2024-08-19T01:37:47.965 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:37:47 smithi167 ceph-mon[21875]: pgmap v427: 97 pgs: 97 active+clean; 184 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 1.6 MiB/s wr, 1.12k op/s 2024-08-19T01:37:49.548 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:37:49 smithi083 ceph-mon[17120]: pgmap v428: 97 pgs: 97 active+clean; 200 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 3.1 MiB/s wr, 1.27k op/s 2024-08-19T01:37:49.952 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:37:49 smithi117 ceph-mon[22016]: pgmap v428: 97 pgs: 97 active+clean; 200 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 3.1 MiB/s wr, 1.27k op/s 2024-08-19T01:37:49.966 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:37:49 smithi167 ceph-mon[21875]: pgmap v428: 97 pgs: 97 active+clean; 200 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 3.1 MiB/s wr, 1.27k op/s 2024-08-19T01:37:50.056 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:37:50.055+0000 7fbbe8a00640 1 -- 172.21.15.83:0/3224926140 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbbe4109c80 msgr2=0x7fbbe4110510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:37:50.056 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:37:50.055+0000 7fbbe8a00640 1 --2- 172.21.15.83:0/3224926140 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbbe4109c80 0x7fbbe4110510 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fbbdc00a900 tx=0x7fbbdc02b6a0 comp rx=0 tx=0).stop 2024-08-19T01:37:50.074 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:37:50.073+0000 7fbbe8a00640 1 -- 172.21.15.83:0/3224926140 shutdown_connections 2024-08-19T01:37:50.075 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:37:50.073+0000 7fbbe8a00640 1 --2- 172.21.15.83:0/3224926140 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbbe4109c80 0x7fbbe4110510 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:37:50.075 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:37:50.073+0000 7fbbe8a00640 1 --2- 172.21.15.83:0/3224926140 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbbe41092c0 0x7fbbe4109740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:37:50.075 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:37:50.073+0000 7fbbe8a00640 1 --2- 172.21.15.83:0/3224926140 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbbe4073300 0x7fbbe4073700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:37:50.075 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:37:50.073+0000 7fbbe8a00640 1 -- 172.21.15.83:0/3224926140 >> 172.21.15.83:0/3224926140 conn(0x7fbbe406e800 msgr2=0x7fbbe4070c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:37:50.075 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:37:50.074+0000 7fbbe8a00640 1 -- 172.21.15.83:0/3224926140 shutdown_connections 2024-08-19T01:37:50.076 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:37:50.074+0000 7fbbe8a00640 1 -- 172.21.15.83:0/3224926140 wait complete. 2024-08-19T01:37:50.077 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:37:50.075+0000 7fbbe8a00640 1 Processor -- start 2024-08-19T01:37:50.091 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:37:50.080+0000 7fbbe8a00640 1 -- start start 2024-08-19T01:37:50.091 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:37:50.081+0000 7fbbe8a00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbbe4073300 0x7fbbe419dfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:37:50.091 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:37:50.081+0000 7fbbe8a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbbe41092c0 0x7fbbe419e510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:37:50.091 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:37:50.081+0000 7fbbe8a00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbbe4109c80 0x7fbbe41b6860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:37:50.091 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:37:50.081+0000 7fbbe8a00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fbbe4113940 con 0x7fbbe41092c0 2024-08-19T01:37:50.091 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:37:50.081+0000 7fbbe8a00640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fbbe41137c0 con 0x7fbbe4109c80 2024-08-19T01:37:50.091 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:37:50.081+0000 7fbbe8a00640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fbbe4113ac0 con 0x7fbbe4073300 2024-08-19T01:37:50.091 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:37:50.082+0000 7fbbe1400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbbe41092c0 0x7fbbe419e510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:37:50.092 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:37:50.082+0000 7fbbe1400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbbe41092c0 0x7fbbe419e510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:48292/0 (socket says 172.21.15.83:48292) 2024-08-19T01:37:50.092 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:37:50.082+0000 7fbbe1400640 1 -- 172.21.15.83:0/1199743694 learned_addr learned my addr 172.21.15.83:0/1199743694 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-08-19T01:37:50.092 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:37:50.082+0000 7fbbe1400640 1 -- 172.21.15.83:0/1199743694 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbbe4073300 msgr2=0x7fbbe419dfd0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-08-19T01:37:50.095 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:37:50.082+0000 7fbbe1400640 1 --2- 172.21.15.83:0/1199743694 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbbe4073300 0x7fbbe419dfd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:37:50.095 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:37:50.082+0000 7fbbe1400640 1 -- 172.21.15.83:0/1199743694 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbbe4109c80 msgr2=0x7fbbe41b6860 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-08-19T01:37:50.095 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:37:50.082+0000 7fbbe1400640 1 --2- 172.21.15.83:0/1199743694 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbbe4109c80 0x7fbbe41b6860 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:37:50.095 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:37:50.083+0000 7fbbe1400640 1 -- 172.21.15.83:0/1199743694 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbbe41b6da0 con 0x7fbbe41092c0 2024-08-19T01:37:50.095 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:37:50.084+0000 7fbbe1400640 1 --2- 172.21.15.83:0/1199743694 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbbe41092c0 0x7fbbe419e510 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7fbbd000e9c0 tx=0x7fbbd000ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:37:50.095 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:37:50.085+0000 7fbbdaa00640 1 -- 172.21.15.83:0/1199743694 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbbd000ccc0 con 0x7fbbe41092c0 2024-08-19T01:37:50.095 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:37:50.090+0000 7fbbe8a00640 1 -- 172.21.15.83:0/1199743694 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbbe4074150 con 0x7fbbe41092c0 2024-08-19T01:37:50.095 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:37:50.090+0000 7fbbe8a00640 1 -- 172.21.15.83:0/1199743694 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbbe4074720 con 0x7fbbe41092c0 2024-08-19T01:37:50.095 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:37:50.093+0000 7fbbdaa00640 1 -- 172.21.15.83:0/1199743694 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbbd0005500 con 0x7fbbe41092c0 2024-08-19T01:37:50.095 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:37:50.093+0000 7fbbdaa00640 1 -- 172.21.15.83:0/1199743694 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbbd0010430 con 0x7fbbe41092c0 2024-08-19T01:37:50.109 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:37:50.108+0000 7fbbeb878740 1 Processor -- start 2024-08-19T01:37:50.111 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:37:50.108+0000 7fbbeb878740 1 -- start start 2024-08-19T01:37:50.115 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:37:50.110+0000 7fbbdaa00640 1 -- 172.21.15.83:0/1199743694 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7fbbd00105d0 con 0x7fbbe41092c0 2024-08-19T01:37:50.124 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:37:50.122+0000 7fbbeb878740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556183058660 0x556183058a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:37:50.124 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:37:50.122+0000 7fbbeb878740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556183058fa0 0x55618305da60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:37:50.124 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:37:50.122+0000 7fbbeb878740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55618305e030 0x5561830604f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:37:50.124 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:37:50.122+0000 7fbbeb878740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x556182fbc890 con 0x556183058660 2024-08-19T01:37:50.124 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:37:50.122+0000 7fbbeb878740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x556182f54d90 con 0x556183058fa0 2024-08-19T01:37:50.124 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:37:50.122+0000 7fbbeb878740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x556182fb8850 con 0x55618305e030 2024-08-19T01:37:50.124 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:37:50.123+0000 7fbbe1e00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556183058fa0 0x55618305da60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:37:50.125 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:37:50.123+0000 7fbbe1e00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556183058fa0 0x55618305da60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.83:45964/0 (socket says 172.21.15.83:45964) 2024-08-19T01:37:50.125 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:37:50.123+0000 7fbbe1e00640 1 -- 172.21.15.83:0/1581340464 learned_addr learned my addr 172.21.15.83:0/1581340464 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-08-19T01:37:50.125 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:37:50.124+0000 7fbbe3200640 1 --2- 172.21.15.83:0/1581340464 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556183058660 0x556183058a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:37:50.126 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:37:50.124+0000 7fbbe1e00640 1 -- 172.21.15.83:0/1581340464 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55618305e030 msgr2=0x5561830604f0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-08-19T01:37:50.126 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:37:50.124+0000 7fbbe1e00640 1 --2- 172.21.15.83:0/1581340464 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55618305e030 0x5561830604f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:37:50.126 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:37:50.124+0000 7fbbe1e00640 1 -- 172.21.15.83:0/1581340464 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556183058660 msgr2=0x556183058a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:37:50.126 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:37:50.124+0000 7fbbe1e00640 1 --2- 172.21.15.83:0/1581340464 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556183058660 0x556183058a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:37:50.126 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:37:50.124+0000 7fbbe1e00640 1 -- 172.21.15.83:0/1581340464 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556182cd0210 con 0x556183058fa0 2024-08-19T01:37:50.129 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:37:50.127+0000 7fbbdaa00640 1 --2- 172.21.15.83:0/1199743694 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7fbbc80c4e60 0x7fbbc80c7320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:37:50.129 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:37:50.127+0000 7fbbdaa00640 1 -- 172.21.15.83:0/1199743694 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8269+0+0 (secure 0 0 0) 0x7fbbd00cf2a0 con 0x7fbbe41092c0 2024-08-19T01:37:50.129 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:37:50.128+0000 7fbbe1e00640 1 --2- 172.21.15.83:0/1581340464 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556183058fa0 0x55618305da60 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fbbd4009740 tx=0x7fbbd4009c00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2bf7739f55b2bb01 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:37:50.131 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:37:50.129+0000 7fbbe3200640 1 --2- 172.21.15.83:0/1199743694 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7fbbc80c4e60 0x7fbbc80c7320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:37:50.132 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:37:50.130+0000 7fbbe3200640 1 --2- 172.21.15.83:0/1199743694 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7fbbc80c4e60 0x7fbbc80c7320 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fbbdc00b010 tx=0x7fbbdc031990 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:37:50.136 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:37:50.132+0000 7fbbcf000640 1 -- 172.21.15.83:0/1581340464 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbbd4015020 con 0x556183058fa0 2024-08-19T01:37:50.136 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:37:50.132+0000 7fbbcf000640 1 -- 172.21.15.83:0/1581340464 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbbd4009e50 con 0x556183058fa0 2024-08-19T01:37:50.136 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:37:50.132+0000 7fbbcf000640 1 -- 172.21.15.83:0/1581340464 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbbd40105e0 con 0x556183058fa0 2024-08-19T01:37:50.138 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:37:50.137+0000 7fbbeb878740 1 -- 172.21.15.83:0/1581340464 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556183058fa0 msgr2=0x55618305da60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:37:50.138 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:37:50.137+0000 7fbbeb878740 1 --2- 172.21.15.83:0/1581340464 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556183058fa0 0x55618305da60 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fbbd4009740 tx=0x7fbbd4009c00 comp rx=0 tx=0).stop 2024-08-19T01:37:50.151 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:37:50.149+0000 7fbbeb878740 1 -- 172.21.15.83:0/1581340464 shutdown_connections 2024-08-19T01:37:50.151 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:37:50.149+0000 7fbbeb878740 1 --2- 172.21.15.83:0/1581340464 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55618305e030 0x5561830604f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:37:50.151 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:37:50.149+0000 7fbbeb878740 1 --2- 172.21.15.83:0/1581340464 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556183058fa0 0x55618305da60 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:37:50.151 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:37:50.149+0000 7fbbeb878740 1 --2- 172.21.15.83:0/1581340464 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556183058660 0x556183058a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:37:50.151 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:37:50.149+0000 7fbbeb878740 1 -- 172.21.15.83:0/1581340464 >> 172.21.15.83:0/1581340464 conn(0x556183053cd0 msgr2=0x5561830540d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:37:50.152 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:37:50.150+0000 7fbbeb878740 1 -- 172.21.15.83:0/1581340464 shutdown_connections 2024-08-19T01:37:50.153 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:37:50.150+0000 7fbbeb878740 1 -- 172.21.15.83:0/1581340464 wait complete. 2024-08-19T01:37:50.153 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:37:50.151+0000 7fbbeb878740 1 -- start start 2024-08-19T01:37:50.153 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:37:50.151+0000 7fbbeb878740 1 Processor -- start 2024-08-19T01:37:50.157 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:37:50.154+0000 7fbbeb878740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556183058660 0x556183062a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:37:50.160 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:37:50.157+0000 7fbbeb878740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556183058fa0 0x556183062fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:37:50.161 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:37:50.157+0000 7fbbe1400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556183058fa0 0x556183062fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:37:50.161 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:37:50.158+0000 7fbbe1e00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556183058660 0x556183062a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:37:50.161 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:37:50.158+0000 7fbbe1e00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556183058660 0x556183062a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.83:55746/0 (socket says 172.21.15.83:55746) 2024-08-19T01:37:50.161 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:37:50.158+0000 7fbbe1e00640 1 -- 172.21.15.83:0/1638253545 learned_addr learned my addr 172.21.15.83:0/1638253545 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-08-19T01:37:50.161 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:37:50.158+0000 7fbbeb878740 1 --2- 172.21.15.83:0/1638253545 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55618305e030 0x556183069f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:37:50.161 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:37:50.158+0000 7fbbeb878740 1 -- 172.21.15.83:0/1638253545 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x5561830610a0 con 0x556183058fa0 2024-08-19T01:37:50.161 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:37:50.158+0000 7fbbeb878740 1 -- 172.21.15.83:0/1638253545 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x556182f77f70 con 0x55618305e030 2024-08-19T01:37:50.161 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:37:50.158+0000 7fbbeb878740 1 -- 172.21.15.83:0/1638253545 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x556183061220 con 0x556183058660 2024-08-19T01:37:50.161 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:37:50.158+0000 7fbbe3200640 1 --2- 172.21.15.83:0/1638253545 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55618305e030 0x556183069f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:37:50.161 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:37:50.158+0000 7fbbeb878740 10 client.0 _ll_register_callbacks cb 0x556182fb48d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-19T01:37:50.161 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:37:50.158+0000 7fbbe3200640 1 -- 172.21.15.83:0/1638253545 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556183058660 msgr2=0x556183062a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:37:50.161 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:37:50.158+0000 7fbbe3200640 1 --2- 172.21.15.83:0/1638253545 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556183058660 0x556183062a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:37:50.162 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:37:50.159+0000 7fbbe3200640 1 -- 172.21.15.83:0/1638253545 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556183058fa0 msgr2=0x556183062fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:37:50.162 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:37:50.159+0000 7fbbe3200640 1 --2- 172.21.15.83:0/1638253545 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556183058fa0 0x556183062fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:37:50.162 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:37:50.159+0000 7fbbe3200640 1 -- 172.21.15.83:0/1638253545 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556182d9c620 con 0x55618305e030 2024-08-19T01:37:50.162 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:37:50.159+0000 7fbbeb878740 1 -- 172.21.15.83:0/1638253545 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbbd4009e50 con 0x55618305e030 2024-08-19T01:37:50.162 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:37:50.160+0000 7fbbe3200640 1 --2- 172.21.15.83:0/1638253545 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55618305e030 0x556183069f80 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fbbdc02bc90 tx=0x7fbbdc00ae10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=680e30cc571431f2 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:37:50.166 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:37:50.162+0000 7fbbc3400640 10 client.25619 ms_handle_connect on v2:172.21.15.117:3300/0 2024-08-19T01:37:50.166 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:37:50.162+0000 7fbbc3400640 1 -- 172.21.15.83:0/1638253545 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbbdc04d020 con 0x55618305e030 2024-08-19T01:37:50.167 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:37:50.162+0000 7fbbc3400640 1 -- 172.21.15.83:0/1638253545 <== mon.2 v2:172.21.15.117:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8269+0+0 (secure 0 0 0) 0x7fbbdc049440 con 0x55618305e030 2024-08-19T01:37:50.167 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:37:50.162+0000 7fbbc3400640 1 -- 172.21.15.83:0/1638253545 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fbbdc037040 con 0x55618305e030 2024-08-19T01:37:50.184 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:37:50.178+0000 7fbbeb878740 10 client.25619 fetch_fsmap learned FSMap version 26 2024-08-19T01:37:50.184 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:37:50.178+0000 7fbbeb878740 1 -- 172.21.15.83:0/1638253545 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({fsmap=26}) -- 0x55618306d070 con 0x55618305e030 2024-08-19T01:37:50.189 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:37:50.187+0000 7fbbc3400640 1 -- 172.21.15.83:0/1638253545 <== mon.2 v2:172.21.15.117:3300/0 4 ==== fsmap(e 26) ==== 8350+0+0 (secure 0 0 0) 0x7fbbdc049660 con 0x55618305e030 2024-08-19T01:37:50.189 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:37:50.187+0000 7fbbeb878740 10 client.25619 fetch_fsmap finished waiting for FSMap version 26 2024-08-19T01:37:50.189 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:37:50.187+0000 7fbbeb878740 10 client.25619 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-08-19T01:37:50.189 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:37:50.188+0000 7fbbeb878740 20 client.25619 populate_metadata read hostname 'smithi083' 2024-08-19T01:37:50.189 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:37:50.188+0000 7fbbeb878740 1 --2- 172.21.15.83:0/1638253545 >> [v2:172.21.15.117:6838/1967595704,v1:172.21.15.117:6839/1967595704] conn(0x55618306e5f0 0x556183070ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:37:50.189 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:37:50.188+0000 7fbbeb878740 4 client.25619 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-19T01:37:50.189 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:37:50.188+0000 7fbbeb878740 1 -- 172.21.15.83:0/1638253545 --> [v2:172.21.15.117:6838/1967595704,v1:172.21.15.117:6839/1967595704] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556182fc9fc0 con 0x55618306e5f0 2024-08-19T01:37:50.200 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:37:50.195+0000 7fbbe1e00640 1 --2- 172.21.15.83:0/1638253545 >> [v2:172.21.15.117:6838/1967595704,v1:172.21.15.117:6839/1967595704] conn(0x55618306e5f0 0x556183070ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:37:50.208 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:37:50.203+0000 7fbbe1e00640 1 --2- 172.21.15.83:0/1638253545 >> [v2:172.21.15.117:6838/1967595704,v1:172.21.15.117:6839/1967595704] conn(0x55618306e5f0 0x556183070ab0 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5f2c7e6bc78f3771 server_cookie=262ff05311d69b76 in_seq=0 out_seq=0 2024-08-19T01:37:50.209 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:37:50.208+0000 7fbbc3400640 10 client.25619 ms_handle_connect on v2:172.21.15.117:6838/1967595704 2024-08-19T01:37:50.209 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:37:50.208+0000 7fbbc3400640 1 -- 172.21.15.83:0/1638253545 <== mds.0 v2:172.21.15.117:6838/1967595704 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x556182fc9fc0 con 0x55618306e5f0 2024-08-19T01:37:50.209 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:37:50.208+0000 7fbbc3400640 10 client.25619 handle_command_reply: tid=0 2024-08-19T01:37:50.209 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:37:50.208+0000 7fbbeb878740 1 client.25619 shutdown 2024-08-19T01:37:50.213 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:37:50.212+0000 7fbbeb878740 1 -- 172.21.15.83:0/1638253545 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55618305e030 msgr2=0x556183069f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:37:50.213 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:37:50.212+0000 7fbbeb878740 1 --2- 172.21.15.83:0/1638253545 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55618305e030 0x556183069f80 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fbbdc02bc90 tx=0x7fbbdc00ae10 comp rx=0 tx=0).stop 2024-08-19T01:37:50.217 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:37:50.215+0000 7fbbeb878740 1 -- 172.21.15.83:0/1638253545 shutdown_connections 2024-08-19T01:37:50.217 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:37:50.215+0000 7fbbeb878740 1 --2- 172.21.15.83:0/1638253545 >> [v2:172.21.15.117:6838/1967595704,v1:172.21.15.117:6839/1967595704] conn(0x55618306e5f0 0x556183070ab0 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:37:50.217 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:37:50.215+0000 7fbbeb878740 1 --2- 172.21.15.83:0/1638253545 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55618305e030 0x556183069f80 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:37:50.217 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:37:50.215+0000 7fbbeb878740 1 --2- 172.21.15.83:0/1638253545 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556183058fa0 0x556183062fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:37:50.217 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:37:50.215+0000 7fbbeb878740 1 --2- 172.21.15.83:0/1638253545 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556183058660 0x556183062a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:37:50.217 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:37:50.215+0000 7fbbeb878740 1 -- 172.21.15.83:0/1638253545 >> 172.21.15.83:0/1638253545 conn(0x556183053cd0 msgr2=0x5561830554e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:37:50.217 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:37:50.215+0000 7fbbc3400640 0 client.25619 ms_handle_reset on v2:172.21.15.117:6838/1967595704 2024-08-19T01:37:50.220 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:37:50.219+0000 7fbbeb878740 1 -- 172.21.15.83:0/1638253545 shutdown_connections 2024-08-19T01:37:50.223 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:37:50.222+0000 7fbbeb878740 1 -- 172.21.15.83:0/1638253545 wait complete. 2024-08-19T01:37:50.243 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:37:50.235+0000 7fbbeb878740 20 client.25619 trim_cache size 0 max 16384 2024-08-19T01:37:50.379 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:37:50.375+0000 7fbbeb878740 1 Processor -- start 2024-08-19T01:37:50.390 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:37:50.388+0000 7fbbeb878740 1 -- start start 2024-08-19T01:37:50.390 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:37:50.388+0000 7fbbeb878740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556183058660 0x5561830540b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:37:50.390 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:37:50.388+0000 7fbbeb878740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556183058fa0 0x556183069f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:37:50.390 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:37:50.388+0000 7fbbeb878740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55618305e030 0x5561830545f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:37:50.390 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:37:50.388+0000 7fbbeb878740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fbbc4010e70 con 0x556183058660 2024-08-19T01:37:50.390 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:37:50.388+0000 7fbbeb878740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fbbc400e740 con 0x55618305e030 2024-08-19T01:37:50.390 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:37:50.388+0000 7fbbeb878740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fbbc400c590 con 0x556183058fa0 2024-08-19T01:37:50.391 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:37:50.389+0000 7fbbe1e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556183058660 0x5561830540b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:37:50.392 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:37:50.389+0000 7fbbe1e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556183058660 0x5561830540b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:48306/0 (socket says 172.21.15.83:48306) 2024-08-19T01:37:50.392 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:37:50.389+0000 7fbbe1e00640 1 -- 172.21.15.83:0/3044365401 learned_addr learned my addr 172.21.15.83:0/3044365401 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-08-19T01:37:50.392 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:37:50.389+0000 7fbbe1e00640 1 -- 172.21.15.83:0/3044365401 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556183058fa0 msgr2=0x556183069f40 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-08-19T01:37:50.392 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:37:50.389+0000 7fbbe1e00640 1 --2- 172.21.15.83:0/3044365401 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556183058fa0 0x556183069f40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:37:50.392 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:37:50.389+0000 7fbbe1e00640 1 -- 172.21.15.83:0/3044365401 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55618305e030 msgr2=0x5561830545f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:37:50.392 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:37:50.389+0000 7fbbe1e00640 1 --2- 172.21.15.83:0/3044365401 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55618305e030 0x5561830545f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:37:50.392 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:37:50.389+0000 7fbbe1e00640 1 -- 172.21.15.83:0/3044365401 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5561830625b0 con 0x556183058660 2024-08-19T01:37:50.392 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:37:50.390+0000 7fbbe1e00640 1 --2- 172.21.15.83:0/3044365401 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556183058660 0x5561830540b0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7fbbd4009740 tx=0x7fbbd4006dc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=11d74659d13ab7f8 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:37:50.392 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:37:50.390+0000 7fbbce600640 1 -- 172.21.15.83:0/3044365401 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbbd402d070 con 0x556183058660 2024-08-19T01:37:50.393 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:37:50.391+0000 7fbbce600640 1 -- 172.21.15.83:0/3044365401 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbbd400bdb0 con 0x556183058660 2024-08-19T01:37:50.393 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:37:50.391+0000 7fbbce600640 1 -- 172.21.15.83:0/3044365401 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbbd4015040 con 0x556183058660 2024-08-19T01:37:50.393 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:37:50.391+0000 7fbbeb878740 1 -- 172.21.15.83:0/3044365401 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556183058660 msgr2=0x5561830540b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:37:50.393 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:37:50.391+0000 7fbbeb878740 1 --2- 172.21.15.83:0/3044365401 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556183058660 0x5561830540b0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7fbbd4009740 tx=0x7fbbd4006dc0 comp rx=0 tx=0).stop 2024-08-19T01:37:50.403 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:37:50.397+0000 7fbbeb878740 1 -- 172.21.15.83:0/3044365401 shutdown_connections 2024-08-19T01:37:50.403 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:37:50.397+0000 7fbbeb878740 1 --2- 172.21.15.83:0/3044365401 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55618305e030 0x5561830545f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:37:50.403 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:37:50.397+0000 7fbbeb878740 1 --2- 172.21.15.83:0/3044365401 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556183058fa0 0x556183069f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:37:50.403 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:37:50.397+0000 7fbbeb878740 1 --2- 172.21.15.83:0/3044365401 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556183058660 0x5561830540b0 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:37:50.403 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:37:50.397+0000 7fbbeb878740 1 -- 172.21.15.83:0/3044365401 >> 172.21.15.83:0/3044365401 conn(0x556183053cd0 msgr2=0x5561830703d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:37:50.403 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:37:50.402+0000 7fbbeb878740 1 -- 172.21.15.83:0/3044365401 shutdown_connections 2024-08-19T01:37:50.406 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:37:50.402+0000 7fbbeb878740 1 -- 172.21.15.83:0/3044365401 wait complete. 2024-08-19T01:37:50.406 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:37:50.404+0000 7fbbeb878740 1 -- start start 2024-08-19T01:37:50.406 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:37:50.404+0000 7fbbeb878740 1 Processor -- start 2024-08-19T01:37:50.406 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:37:50.404+0000 7fbbeb878740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556183058660 0x556183066cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:37:50.412 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:37:50.409+0000 7fbbeb878740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556183058fa0 0x556183067230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:37:50.412 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:37:50.409+0000 7fbbeb878740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55618305e030 0x5561830d7380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:37:50.412 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:37:50.409+0000 7fbbeb878740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fbbc400cc00 con 0x556183058fa0 2024-08-19T01:37:50.412 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:37:50.409+0000 7fbbeb878740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fbbc400e070 con 0x556183058660 2024-08-19T01:37:50.412 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:37:50.409+0000 7fbbeb878740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fbbc400d2d0 con 0x55618305e030 2024-08-19T01:37:50.412 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:37:50.409+0000 7fbbeb878740 10 client.0 _ll_register_callbacks cb 0x7fbbdc035070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-19T01:37:50.412 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:37:50.409+0000 7fbbe1400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556183058fa0 0x556183067230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:37:50.412 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:37:50.409+0000 7fbbe1400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556183058fa0 0x556183067230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:48318/0 (socket says 172.21.15.83:48318) 2024-08-19T01:37:50.413 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:37:50.409+0000 7fbbe1400640 1 -- 172.21.15.83:0/2647614997 learned_addr learned my addr 172.21.15.83:0/2647614997 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-08-19T01:37:50.413 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:37:50.410+0000 7fbbe1400640 1 -- 172.21.15.83:0/2647614997 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55618305e030 msgr2=0x5561830d7380 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-08-19T01:37:50.413 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:37:50.410+0000 7fbbe1400640 1 --2- 172.21.15.83:0/2647614997 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55618305e030 0x5561830d7380 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:37:50.413 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:37:50.410+0000 7fbbe1400640 1 -- 172.21.15.83:0/2647614997 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556183058660 msgr2=0x556183066cf0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-08-19T01:37:50.413 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:37:50.410+0000 7fbbe1400640 1 --2- 172.21.15.83:0/2647614997 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556183058660 0x556183066cf0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:37:50.413 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:37:50.410+0000 7fbbe1400640 1 -- 172.21.15.83:0/2647614997 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55618306a2b0 con 0x556183058fa0 2024-08-19T01:37:50.413 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:37:50.410+0000 7fbbe1400640 1 --2- 172.21.15.83:0/2647614997 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556183058fa0 0x556183067230 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7fbbd0099500 tx=0x7fbbd009d370 comp rx=0 tx=0).ready entity=mon.0 client_cookie=270a18ab738b8f6b server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:37:50.418 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:37:50.415+0000 7fbbc3400640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-08-19T01:37:50.418 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:37:50.416+0000 7fbbc3400640 1 -- 172.21.15.83:0/2647614997 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbbd00ba060 con 0x556183058fa0 2024-08-19T01:37:50.418 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:37:50.416+0000 7fbbc3400640 1 -- 172.21.15.83:0/2647614997 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8269+0+0 (secure 0 0 0) 0x7fbbd00cce30 con 0x556183058fa0 2024-08-19T01:37:50.420 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:37:50.417+0000 7fbbeb878740 1 -- 172.21.15.83:0/2647614997 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbbd400bdb0 con 0x556183058fa0 2024-08-19T01:37:50.420 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:37:50.419+0000 7fbbc3400640 1 -- 172.21.15.83:0/2647614997 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fbbd00b5030 con 0x556183058fa0 2024-08-19T01:37:50.420 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:37:50.419+0000 7fbbeb878740 10 client.15561 fetch_fsmap learned FSMap version 26 2024-08-19T01:37:50.420 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:37:50.419+0000 7fbbeb878740 1 -- 172.21.15.83:0/2647614997 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5561830d89d0 con 0x556183058fa0 2024-08-19T01:37:50.421 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:37:50.420+0000 7fbbc3400640 1 -- 172.21.15.83:0/2647614997 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbbd00a57a0 con 0x556183058fa0 2024-08-19T01:37:50.424 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:37:50.423+0000 7fbbc3400640 1 -- 172.21.15.83:0/2647614997 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 26) ==== 8350+0+0 (secure 0 0 0) 0x7fbbd009bf80 con 0x556183058fa0 2024-08-19T01:37:50.444 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:37:50.443+0000 7fbbeb878740 10 client.15561 fetch_fsmap finished waiting for FSMap version 26 2024-08-19T01:37:50.444 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:37:50.443+0000 7fbbeb878740 10 client.15561 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-08-19T01:37:50.445 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:37:50.443+0000 7fbbeb878740 20 client.15561 populate_metadata read hostname 'smithi083' 2024-08-19T01:37:50.445 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:37:50.443+0000 7fbbeb878740 1 --2- 172.21.15.83:0/2647614997 >> [v2:172.21.15.117:6838/1967595704,v1:172.21.15.117:6839/1967595704] conn(0x5561830d9360 0x5561830db780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:37:50.445 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:37:50.443+0000 7fbbeb878740 4 client.15561 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-08-19T01:37:50.445 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:37:50.443+0000 7fbbeb878740 1 -- 172.21.15.83:0/2647614997 --> [v2:172.21.15.117:6838/1967595704,v1:172.21.15.117:6839/1967595704] -- command(tid 0: {"prefix": "flush journal"}) -- 0x7fbbd400bdb0 con 0x5561830d9360 2024-08-19T01:37:50.451 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:37:50.449+0000 7fbbe1e00640 1 --2- 172.21.15.83:0/2647614997 >> [v2:172.21.15.117:6838/1967595704,v1:172.21.15.117:6839/1967595704] conn(0x5561830d9360 0x5561830db780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:37:50.476 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:37:50.468+0000 7fbbe1e00640 1 --2- 172.21.15.83:0/2647614997 >> [v2:172.21.15.117:6838/1967595704,v1:172.21.15.117:6839/1967595704] conn(0x5561830d9360 0x5561830db780 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=aebff57ebe2244b server_cookie=d90636d4e9e15b8 in_seq=0 out_seq=0 2024-08-19T01:37:50.477 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:37:50.468+0000 7fbbc3400640 10 client.15561 ms_handle_connect on v2:172.21.15.117:6838/1967595704 2024-08-19T01:37:50.599 INFO:tasks.workunit.client.0.smithi083.stdout: 78 608427 439 402369 413 346362 1173 2024-08-19T01:37:50.885 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:37:50 smithi083 ceph-mon[17120]: scrub summary: idle 2024-08-19T01:37:50.952 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:37:50 smithi117 ceph-mon[22016]: scrub summary: idle 2024-08-19T01:37:50.965 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:37:50 smithi167 ceph-mon[21875]: scrub summary: idle 2024-08-19T01:37:51.599 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:37:51 smithi083 ceph-mon[17120]: scrub summary: idle 2024-08-19T01:37:51.600 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:37:51 smithi083 ceph-mon[17120]: pgmap v429: 97 pgs: 97 active+clean; 204 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 3.3 MiB/s wr, 1.06k op/s 2024-08-19T01:37:51.952 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:37:51 smithi117 ceph-mon[22016]: scrub summary: idle 2024-08-19T01:37:51.952 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:37:51 smithi117 ceph-mon[22016]: pgmap v429: 97 pgs: 97 active+clean; 204 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 3.3 MiB/s wr, 1.06k op/s 2024-08-19T01:37:51.966 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:37:51 smithi167 ceph-mon[21875]: scrub summary: idle 2024-08-19T01:37:51.966 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:37:51 smithi167 ceph-mon[21875]: pgmap v429: 97 pgs: 97 active+clean; 204 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 3.3 MiB/s wr, 1.06k op/s 2024-08-19T01:37:53.087 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:37:53.085+0000 7fbbd9600640 1 -- 172.21.15.83:0/1199743694 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbbe41092c0 msgr2=0x7fbbe419e510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:37:53.088 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:37:53.085+0000 7fbbd9600640 1 --2- 172.21.15.83:0/1199743694 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbbe41092c0 0x7fbbe419e510 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7fbbd000e9c0 tx=0x7fbbd000ee80 comp rx=0 tx=0).stop 2024-08-19T01:37:53.096 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:37:53.086+0000 7fbbd9600640 1 --2- 172.21.15.83:0/1199743694 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbbe4073300 0x7fbbb40056b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:37:53.096 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:37:53.086+0000 7fbbd9600640 1 --2- 172.21.15.83:0/1199743694 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbbe4109c80 0x7fbbb40081a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:37:53.096 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:37:53.087+0000 7fbbd9600640 1 --2- 172.21.15.83:0/1199743694 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbbb4008c00 0x7fbbb400b090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:37:53.096 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:37:53.087+0000 7fbbd9600640 1 -- 172.21.15.83:0/1199743694 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fbbb400bb90 con 0x7fbbb4008c00 2024-08-19T01:37:53.096 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:37:53.087+0000 7fbbd9600640 1 -- 172.21.15.83:0/1199743694 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fbbb400bd10 con 0x7fbbe4109c80 2024-08-19T01:37:53.096 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:37:53.087+0000 7fbbd9600640 1 -- 172.21.15.83:0/1199743694 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fbbb400bf10 con 0x7fbbe4073300 2024-08-19T01:37:53.096 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:37:53.088+0000 7fbbe1e00640 1 --2- 172.21.15.83:0/1199743694 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbbb4008c00 0x7fbbb400b090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:37:53.096 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:37:53.089+0000 7fbbe1e00640 1 -- 172.21.15.83:0/1199743694 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbbe4073300 msgr2=0x7fbbb40056b0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-08-19T01:37:53.096 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:37:53.089+0000 7fbbe1e00640 1 --2- 172.21.15.83:0/1199743694 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbbe4073300 0x7fbbb40056b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:37:53.096 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:37:53.089+0000 7fbbe1e00640 1 -- 172.21.15.83:0/1199743694 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbbe4109c80 msgr2=0x7fbbb40081a0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-08-19T01:37:53.096 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:37:53.089+0000 7fbbe1e00640 1 --2- 172.21.15.83:0/1199743694 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbbe4109c80 0x7fbbb40081a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:37:53.096 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:37:53.089+0000 7fbbe1e00640 1 -- 172.21.15.83:0/1199743694 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,mgrmap=0+,monmap=4+}) -- 0x7fbbb400c230 con 0x7fbbb4008c00 2024-08-19T01:37:53.097 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:37:53.092+0000 7fbbe1e00640 1 --2- 172.21.15.83:0/1199743694 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbbb4008c00 0x7fbbb400b090 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x556183063bc0 tx=0x7fbbd0009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:37:53.097 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:37:53.094+0000 7fbbdaa00640 1 -- 172.21.15.83:0/1199743694 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbbd402d040 con 0x7fbbb4008c00 2024-08-19T01:37:53.099 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:37:53.094+0000 7fbbdaa00640 1 -- 172.21.15.83:0/1199743694 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbbd400bdb0 con 0x7fbbb4008c00 2024-08-19T01:37:53.113 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:37:53.112+0000 7fbbdaa00640 1 -- 172.21.15.83:0/1199743694 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7fbbd4030030 con 0x7fbbb4008c00 2024-08-19T01:37:53.738 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:37:53 smithi083 ceph-mon[17120]: pgmap v430: 97 pgs: 97 active+clean; 213 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 3.7 MiB/s wr, 1.01k op/s 2024-08-19T01:37:53.952 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:37:53 smithi117 ceph-mon[22016]: pgmap v430: 97 pgs: 97 active+clean; 213 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 3.7 MiB/s wr, 1.01k op/s 2024-08-19T01:37:53.965 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:37:53 smithi167 ceph-mon[21875]: pgmap v430: 97 pgs: 97 active+clean; 213 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 3.7 MiB/s wr, 1.01k op/s 2024-08-19T01:37:54.885 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:37:54.883+0000 7fbbc3400640 1 -- 172.21.15.83:0/2647614997 <== mds.0 v2:172.21.15.117:6838/1967595704 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x7fbbd400bdb0 con 0x5561830d9360 2024-08-19T01:37:54.886 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:37:54.883+0000 7fbbc3400640 10 client.15561 handle_command_reply: tid=0 2024-08-19T01:37:54.886 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:37:54.883+0000 7fbbeb878740 1 client.15561 shutdown 2024-08-19T01:37:54.887 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:37:54.885+0000 7fbbeb878740 1 -- 172.21.15.83:0/2647614997 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556183058fa0 msgr2=0x556183067230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:37:54.887 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:37:54.885+0000 7fbbeb878740 1 --2- 172.21.15.83:0/2647614997 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556183058fa0 0x556183067230 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7fbbd0099500 tx=0x7fbbd009d370 comp rx=0 tx=0).stop 2024-08-19T01:37:54.887 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:37:54.885+0000 7fbbeb878740 1 -- 172.21.15.83:0/2647614997 shutdown_connections 2024-08-19T01:37:54.887 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:37:54.885+0000 7fbbeb878740 1 --2- 172.21.15.83:0/2647614997 >> [v2:172.21.15.117:6838/1967595704,v1:172.21.15.117:6839/1967595704] conn(0x5561830d9360 0x5561830db780 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:37:54.887 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:37:54.885+0000 7fbbc3400640 0 client.15561 ms_handle_reset on v2:172.21.15.117:6838/1967595704 2024-08-19T01:37:54.889 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:37:54.887+0000 7fbbeb878740 1 --2- 172.21.15.83:0/2647614997 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55618305e030 0x5561830d7380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:37:54.889 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:37:54.887+0000 7fbbeb878740 1 --2- 172.21.15.83:0/2647614997 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556183058fa0 0x556183067230 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:37:54.889 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:37:54.887+0000 7fbbeb878740 1 --2- 172.21.15.83:0/2647614997 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556183058660 0x556183066cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:37:54.889 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:37:54.887+0000 7fbbeb878740 1 -- 172.21.15.83:0/2647614997 >> 172.21.15.83:0/2647614997 conn(0x556183053cd0 msgr2=0x556183052720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:37:54.889 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:37:54.887+0000 7fbbeb878740 1 -- 172.21.15.83:0/2647614997 shutdown_connections 2024-08-19T01:37:54.889 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:37:54.887+0000 7fbbeb878740 1 -- 172.21.15.83:0/2647614997 wait complete. 2024-08-19T01:37:54.889 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:37:54.887+0000 7fbbeb878740 20 client.15561 trim_cache size 0 max 16384 2024-08-19T01:37:54.889 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stdout:{ 2024-08-19T01:37:54.890 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stdout: "message": "", 2024-08-19T01:37:54.890 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stdout: "return_code": 0 2024-08-19T01:37:54.890 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stdout:} 2024-08-19T01:37:54.891 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:37:54.887+0000 7fbbe8a00640 1 -- 172.21.15.83:0/1199743694 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7fbbc80c4e60 msgr2=0x7fbbc80c7320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:37:54.891 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:37:54.887+0000 7fbbe8a00640 1 --2- 172.21.15.83:0/1199743694 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7fbbc80c4e60 0x7fbbc80c7320 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fbbdc00b010 tx=0x7fbbdc031990 comp rx=0 tx=0).stop 2024-08-19T01:37:54.891 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:37:54.887+0000 7fbbe8a00640 1 -- 172.21.15.83:0/1199743694 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbbb4008c00 msgr2=0x7fbbb400b090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:37:54.891 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:37:54.887+0000 7fbbe8a00640 1 --2- 172.21.15.83:0/1199743694 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbbb4008c00 0x7fbbb400b090 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x556183063bc0 tx=0x7fbbd0009260 comp rx=0 tx=0).stop 2024-08-19T01:37:54.891 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:37:54.887+0000 7fbbe8a00640 1 -- 172.21.15.83:0/1199743694 shutdown_connections 2024-08-19T01:37:54.892 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:37:54.887+0000 7fbbe8a00640 1 --2- 172.21.15.83:0/1199743694 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbbb4008c00 0x7fbbb400b090 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:37:54.892 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:37:54.887+0000 7fbbe8a00640 1 --2- 172.21.15.83:0/1199743694 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbbe4109c80 0x7fbbb40081a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:37:54.892 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:37:54.887+0000 7fbbe8a00640 1 --2- 172.21.15.83:0/1199743694 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbbe4073300 0x7fbbb40056b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:37:54.892 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:37:54.887+0000 7fbbe8a00640 1 --2- 172.21.15.83:0/1199743694 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7fbbc80c4e60 0x7fbbc80c7320 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:37:54.892 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:37:54.887+0000 7fbbe8a00640 1 -- 172.21.15.83:0/1199743694 >> 172.21.15.83:0/1199743694 conn(0x7fbbe406e800 msgr2=0x7fbbe410c5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:37:54.893 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:37:54.892+0000 7fbbe8a00640 1 -- 172.21.15.83:0/1199743694 shutdown_connections 2024-08-19T01:37:54.893 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:37:54.892+0000 7fbbe8a00640 1 -- 172.21.15.83:0/1199743694 wait complete. 2024-08-19T01:37:55.904 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:37:55 smithi083 ceph-mon[17120]: pgmap v431: 97 pgs: 97 active+clean; 252 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 7.5 MiB/s wr, 1.63k op/s 2024-08-19T01:37:55.952 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:37:55 smithi117 ceph-mon[22016]: pgmap v431: 97 pgs: 97 active+clean; 252 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 7.5 MiB/s wr, 1.63k op/s 2024-08-19T01:37:55.965 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:37:55 smithi167 ceph-mon[21875]: pgmap v431: 97 pgs: 97 active+clean; 252 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 7.5 MiB/s wr, 1.63k op/s 2024-08-19T01:37:57.600 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:37:57 smithi083 ceph-mon[17120]: pgmap v432: 97 pgs: 97 active+clean; 269 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 8.3 MiB/s wr, 1.62k op/s 2024-08-19T01:37:57.952 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:37:57 smithi117 ceph-mon[22016]: pgmap v432: 97 pgs: 97 active+clean; 269 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 8.3 MiB/s wr, 1.62k op/s 2024-08-19T01:37:57.966 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:37:57 smithi167 ceph-mon[21875]: pgmap v432: 97 pgs: 97 active+clean; 269 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 8.3 MiB/s wr, 1.62k op/s 2024-08-19T01:37:59.786 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:37:59 smithi083 ceph-mon[17120]: pgmap v433: 97 pgs: 97 active+clean; 247 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 8.7 MiB/s wr, 1.81k op/s 2024-08-19T01:37:59.952 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:37:59 smithi117 ceph-mon[22016]: pgmap v433: 97 pgs: 97 active+clean; 247 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 8.7 MiB/s wr, 1.81k op/s 2024-08-19T01:37:59.965 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:37:59 smithi167 ceph-mon[21875]: pgmap v433: 97 pgs: 97 active+clean; 247 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 8.7 MiB/s wr, 1.81k op/s 2024-08-19T01:38:00.600 INFO:tasks.workunit.client.0.smithi083.stdout: 85 480057 420 320248 232 287593 851 2024-08-19T01:38:01.738 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:38:01 smithi083 ceph-mon[17120]: pgmap v434: 97 pgs: 97 active+clean; 243 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 7.7 MiB/s wr, 1.54k op/s 2024-08-19T01:38:01.952 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:38:01 smithi117 ceph-mon[22016]: pgmap v434: 97 pgs: 97 active+clean; 243 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 7.7 MiB/s wr, 1.54k op/s 2024-08-19T01:38:01.966 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:38:01 smithi167 ceph-mon[21875]: pgmap v434: 97 pgs: 97 active+clean; 243 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 7.7 MiB/s wr, 1.54k op/s 2024-08-19T01:38:02.904 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:38:02 smithi083 ceph-mon[17120]: pgmap v435: 97 pgs: 97 active+clean; 243 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 7.2 MiB/s wr, 1.41k op/s 2024-08-19T01:38:02.952 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:38:02 smithi117 ceph-mon[22016]: pgmap v435: 97 pgs: 97 active+clean; 243 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 7.2 MiB/s wr, 1.41k op/s 2024-08-19T01:38:02.965 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:38:02 smithi167 ceph-mon[21875]: pgmap v435: 97 pgs: 97 active+clean; 243 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 7.2 MiB/s wr, 1.41k op/s 2024-08-19T01:38:05.521 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:38:05 smithi083 ceph-mon[17120]: pgmap v436: 97 pgs: 97 active+clean; 244 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 6.7 MiB/s wr, 1.39k op/s 2024-08-19T01:38:05.952 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:38:05 smithi117 ceph-mon[22016]: pgmap v436: 97 pgs: 97 active+clean; 244 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 6.7 MiB/s wr, 1.39k op/s 2024-08-19T01:38:05.965 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:38:05 smithi167 ceph-mon[21875]: pgmap v436: 97 pgs: 97 active+clean; 244 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 6.7 MiB/s wr, 1.39k op/s 2024-08-19T01:38:07.910 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:38:07 smithi083 ceph-mon[17120]: pgmap v437: 97 pgs: 97 active+clean; 249 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 3.4 MiB/s wr, 719 op/s 2024-08-19T01:38:07.952 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:38:07 smithi117 ceph-mon[22016]: pgmap v437: 97 pgs: 97 active+clean; 249 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 3.4 MiB/s wr, 719 op/s 2024-08-19T01:38:07.965 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:38:07 smithi167 ceph-mon[21875]: pgmap v437: 97 pgs: 97 active+clean; 249 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 3.4 MiB/s wr, 719 op/s 2024-08-19T01:38:08.225 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:38:08.222+0000 7fb2bb200640 1 -- 172.21.15.83:0/4042451292 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb2bc0731c0 msgr2=0x7fb2bc112270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:38:08.225 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:38:08.222+0000 7fb2bb200640 1 --2- 172.21.15.83:0/4042451292 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb2bc0731c0 0x7fb2bc112270 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fb2ac009900 tx=0x7fb2ac02a420 comp rx=0 tx=0).stop 2024-08-19T01:38:08.237 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:38:08.236+0000 7fb2bb200640 1 -- 172.21.15.83:0/4042451292 shutdown_connections 2024-08-19T01:38:08.237 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:38:08.236+0000 7fb2bb200640 1 --2- 172.21.15.83:0/4042451292 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb2bc0731c0 0x7fb2bc112270 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:38:08.237 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:38:08.236+0000 7fb2bb200640 1 --2- 172.21.15.83:0/4042451292 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb2bc072800 0x7fb2bc072c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:38:08.238 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:38:08.236+0000 7fb2bb200640 1 --2- 172.21.15.83:0/4042451292 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb2bc1093e0 0x7fb2bc1097e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:38:08.238 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:38:08.236+0000 7fb2bb200640 1 -- 172.21.15.83:0/4042451292 >> 172.21.15.83:0/4042451292 conn(0x7fb2bc06e880 msgr2=0x7fb2bc070cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:38:08.243 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:38:08.242+0000 7fb2bb200640 1 -- 172.21.15.83:0/4042451292 shutdown_connections 2024-08-19T01:38:08.247 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:38:08.245+0000 7fb2bb200640 1 -- 172.21.15.83:0/4042451292 wait complete. 2024-08-19T01:38:08.247 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:38:08.246+0000 7fb2bb200640 1 Processor -- start 2024-08-19T01:38:08.255 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:38:08.246+0000 7fb2bb200640 1 -- start start 2024-08-19T01:38:08.255 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:38:08.251+0000 7fb2bb200640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb2bc072800 0x7fb2bc11aa40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:38:08.255 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:38:08.251+0000 7fb2b9e00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb2bc072800 0x7fb2bc11aa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:38:08.255 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:38:08.251+0000 7fb2b9e00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb2bc072800 0x7fb2bc11aa40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.83:38396/0 (socket says 172.21.15.83:38396) 2024-08-19T01:38:08.255 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:38:08.251+0000 7fb2bb200640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb2bc0731c0 0x7fb2bc11af80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:38:08.256 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:38:08.252+0000 7fb2b9e00640 1 -- 172.21.15.83:0/3993682201 learned_addr learned my addr 172.21.15.83:0/3993682201 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-08-19T01:38:08.256 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:38:08.252+0000 7fb2b9400640 1 --2- 172.21.15.83:0/3993682201 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb2bc0731c0 0x7fb2bc11af80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:38:08.256 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:38:08.252+0000 7fb2bb200640 1 --2- 172.21.15.83:0/3993682201 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb2bc1093e0 0x7fb2bc119080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:38:08.256 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:38:08.252+0000 7fb2bb200640 1 -- 172.21.15.83:0/3993682201 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fb2bc113920 con 0x7fb2bc1093e0 2024-08-19T01:38:08.256 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:38:08.252+0000 7fb2bb200640 1 -- 172.21.15.83:0/3993682201 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fb2bc1137a0 con 0x7fb2bc072800 2024-08-19T01:38:08.256 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:38:08.252+0000 7fb2bb200640 1 -- 172.21.15.83:0/3993682201 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fb2bc113aa0 con 0x7fb2bc0731c0 2024-08-19T01:38:08.256 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:38:08.253+0000 7fb2ba800640 1 --2- 172.21.15.83:0/3993682201 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb2bc1093e0 0x7fb2bc119080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:38:08.256 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:38:08.253+0000 7fb2b9e00640 1 -- 172.21.15.83:0/3993682201 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb2bc0731c0 msgr2=0x7fb2bc11af80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:38:08.256 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:38:08.253+0000 7fb2b9e00640 1 --2- 172.21.15.83:0/3993682201 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb2bc0731c0 0x7fb2bc11af80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:38:08.257 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:38:08.253+0000 7fb2b9e00640 1 -- 172.21.15.83:0/3993682201 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb2bc1093e0 msgr2=0x7fb2bc119080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:38:08.257 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:38:08.253+0000 7fb2b9e00640 1 --2- 172.21.15.83:0/3993682201 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb2bc1093e0 0x7fb2bc119080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:38:08.257 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:38:08.253+0000 7fb2b9e00640 1 -- 172.21.15.83:0/3993682201 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb2bc119820 con 0x7fb2bc072800 2024-08-19T01:38:08.257 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:38:08.254+0000 7fb2b9e00640 1 --2- 172.21.15.83:0/3993682201 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb2bc072800 0x7fb2bc11aa40 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fb2b4009fd0 tx=0x7fb2b400beb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:38:08.257 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:38:08.254+0000 7fb2b2a00640 1 -- 172.21.15.83:0/3993682201 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2b403b070 con 0x7fb2bc072800 2024-08-19T01:38:08.257 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:38:08.254+0000 7fb2b2a00640 1 -- 172.21.15.83:0/3993682201 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb2b402ebe0 con 0x7fb2bc072800 2024-08-19T01:38:08.257 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:38:08.254+0000 7fb2b2a00640 1 -- 172.21.15.83:0/3993682201 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2b4007730 con 0x7fb2bc072800 2024-08-19T01:38:08.257 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:38:08.256+0000 7fb2ba800640 1 --2- 172.21.15.83:0/3993682201 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb2bc1093e0 0x7fb2bc119080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-19T01:38:08.258 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:38:08.256+0000 7fb2bb200640 1 -- 172.21.15.83:0/3993682201 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb2bc119b10 con 0x7fb2bc072800 2024-08-19T01:38:08.258 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:38:08.256+0000 7fb2bb200640 1 -- 172.21.15.83:0/3993682201 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb2bc11b750 con 0x7fb2bc072800 2024-08-19T01:38:08.281 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:38:08.277+0000 7fb2b2a00640 1 -- 172.21.15.83:0/3993682201 <== mon.2 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7fb2b403e070 con 0x7fb2bc072800 2024-08-19T01:38:08.281 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:38:08.279+0000 7fb2b2a00640 1 --2- 172.21.15.83:0/3993682201 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7fb2a00a2da0 0x7fb2a00a5260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:38:08.290 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:38:08.283+0000 7fb2b9400640 1 --2- 172.21.15.83:0/3993682201 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7fb2a00a2da0 0x7fb2a00a5260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:38:08.290 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:38:08.285+0000 7fb2b9400640 1 --2- 172.21.15.83:0/3993682201 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7fb2a00a2da0 0x7fb2a00a5260 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fb2bc11a7b0 tx=0x7fb2a800b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:38:08.290 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:38:08.286+0000 7fb2c3c20740 1 Processor -- start 2024-08-19T01:38:08.290 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:38:08.286+0000 7fb2b2a00640 1 -- 172.21.15.83:0/3993682201 <== mon.2 v2:172.21.15.117:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8269+0+0 (secure 0 0 0) 0x7fb2b40f05a0 con 0x7fb2bc072800 2024-08-19T01:38:08.292 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:38:08.290+0000 7fb2c3c20740 1 -- start start 2024-08-19T01:38:08.292 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:38:08.290+0000 7fb2c3c20740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563d02a3b660 0x563d02a3ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:38:08.292 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:38:08.290+0000 7fb2c3c20740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563d02a3bfa0 0x563d02a40a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:38:08.292 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:38:08.290+0000 7fb2c3c20740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563d02a41030 0x563d02a434f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:38:08.292 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:38:08.290+0000 7fb2c3c20740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x563d0299f890 con 0x563d02a3b660 2024-08-19T01:38:08.292 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:38:08.290+0000 7fb2c3c20740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x563d02937d90 con 0x563d02a3bfa0 2024-08-19T01:38:08.292 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:38:08.290+0000 7fb2c3c20740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x563d0299b850 con 0x563d02a41030 2024-08-19T01:38:08.293 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:38:08.292+0000 7fb2b9e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563d02a3b660 0x563d02a3ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:38:08.294 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:38:08.292+0000 7fb2b9e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563d02a3b660 0x563d02a3ba60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:56820/0 (socket says 172.21.15.83:56820) 2024-08-19T01:38:08.295 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:38:08.292+0000 7fb2b9e00640 1 -- 172.21.15.83:0/1606307025 learned_addr learned my addr 172.21.15.83:0/1606307025 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-08-19T01:38:08.295 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:38:08.292+0000 7fb2b9e00640 1 -- 172.21.15.83:0/1606307025 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563d02a41030 msgr2=0x563d02a434f0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-08-19T01:38:08.295 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:38:08.292+0000 7fb2b9e00640 1 --2- 172.21.15.83:0/1606307025 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563d02a41030 0x563d02a434f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:38:08.295 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:38:08.292+0000 7fb2b9e00640 1 -- 172.21.15.83:0/1606307025 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563d02a3bfa0 msgr2=0x563d02a40a60 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-08-19T01:38:08.295 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:38:08.292+0000 7fb2b9e00640 1 --2- 172.21.15.83:0/1606307025 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563d02a3bfa0 0x563d02a40a60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:38:08.295 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:38:08.293+0000 7fb2b9e00640 1 -- 172.21.15.83:0/1606307025 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563d026b3210 con 0x563d02a3b660 2024-08-19T01:38:08.295 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:38:08.293+0000 7fb2b9e00640 1 --2- 172.21.15.83:0/1606307025 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563d02a3b660 0x563d02a3ba60 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7fb2b40b9290 tx=0x7fb2b40bca30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=560a92dac835e329 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:38:08.308 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:38:08.305+0000 7fb2a7000640 1 -- 172.21.15.83:0/1606307025 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2b40c59f0 con 0x563d02a3b660 2024-08-19T01:38:08.308 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:38:08.306+0000 7fb2a7000640 1 -- 172.21.15.83:0/1606307025 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb2b40bb130 con 0x563d02a3b660 2024-08-19T01:38:08.308 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:38:08.306+0000 7fb2a7000640 1 -- 172.21.15.83:0/1606307025 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2b40c5560 con 0x563d02a3b660 2024-08-19T01:38:08.308 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:38:08.306+0000 7fb2c3c20740 1 -- 172.21.15.83:0/1606307025 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563d02a3b660 msgr2=0x563d02a3ba60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:38:08.308 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:38:08.306+0000 7fb2c3c20740 1 --2- 172.21.15.83:0/1606307025 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563d02a3b660 0x563d02a3ba60 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7fb2b40b9290 tx=0x7fb2b40bca30 comp rx=0 tx=0).stop 2024-08-19T01:38:08.324 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:38:08.314+0000 7fb2c3c20740 1 -- 172.21.15.83:0/1606307025 shutdown_connections 2024-08-19T01:38:08.324 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:38:08.314+0000 7fb2c3c20740 1 --2- 172.21.15.83:0/1606307025 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563d02a41030 0x563d02a434f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:38:08.324 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:38:08.314+0000 7fb2c3c20740 1 --2- 172.21.15.83:0/1606307025 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563d02a3bfa0 0x563d02a40a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:38:08.324 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:38:08.314+0000 7fb2c3c20740 1 --2- 172.21.15.83:0/1606307025 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563d02a3b660 0x563d02a3ba60 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:38:08.324 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:38:08.314+0000 7fb2c3c20740 1 -- 172.21.15.83:0/1606307025 >> 172.21.15.83:0/1606307025 conn(0x563d02a36cd0 msgr2=0x563d02a370d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:38:08.329 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:38:08.328+0000 7fb2c3c20740 1 -- 172.21.15.83:0/1606307025 shutdown_connections 2024-08-19T01:38:08.330 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:38:08.328+0000 7fb2c3c20740 1 -- 172.21.15.83:0/1606307025 wait complete. 2024-08-19T01:38:08.330 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:38:08.328+0000 7fb2c3c20740 1 -- start start 2024-08-19T01:38:08.330 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:38:08.329+0000 7fb2c3c20740 1 Processor -- start 2024-08-19T01:38:08.333 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:38:08.332+0000 7fb2c3c20740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563d02a3b660 0x563d02a459d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:38:08.333 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:38:08.332+0000 7fb2c3c20740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563d02a3bfa0 0x563d02a45f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:38:08.334 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:38:08.333+0000 7fb2b9e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563d02a3b660 0x563d02a459d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:38:08.334 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:38:08.333+0000 7fb2b9e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563d02a3b660 0x563d02a459d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:56830/0 (socket says 172.21.15.83:56830) 2024-08-19T01:38:08.334 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:38:08.333+0000 7fb2b9e00640 1 -- 172.21.15.83:0/2939523707 learned_addr learned my addr 172.21.15.83:0/2939523707 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-08-19T01:38:08.334 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:38:08.333+0000 7fb2b9400640 1 --2- 172.21.15.83:0/2939523707 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563d02a3bfa0 0x563d02a45f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:38:08.338 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:38:08.335+0000 7fb2c3c20740 1 --2- 172.21.15.83:0/2939523707 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563d02a41030 0x563d02a4cee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:38:08.338 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:38:08.335+0000 7fb2c3c20740 1 -- 172.21.15.83:0/2939523707 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x563d02a440a0 con 0x563d02a3b660 2024-08-19T01:38:08.338 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:38:08.335+0000 7fb2c3c20740 1 -- 172.21.15.83:0/2939523707 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x563d0295af70 con 0x563d02a3bfa0 2024-08-19T01:38:08.338 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:38:08.335+0000 7fb2c3c20740 1 -- 172.21.15.83:0/2939523707 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x563d02a44220 con 0x563d02a41030 2024-08-19T01:38:08.339 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:38:08.335+0000 7fb2c3c20740 10 client.0 _ll_register_callbacks cb 0x563d029978d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-19T01:38:08.339 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:38:08.336+0000 7fb2b9e00640 1 -- 172.21.15.83:0/2939523707 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563d02a41030 msgr2=0x563d02a4cee0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-08-19T01:38:08.339 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:38:08.336+0000 7fb2b9e00640 1 --2- 172.21.15.83:0/2939523707 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563d02a41030 0x563d02a4cee0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:38:08.339 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:38:08.336+0000 7fb2b9e00640 1 -- 172.21.15.83:0/2939523707 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563d02a3bfa0 msgr2=0x563d02a45f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:38:08.339 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:38:08.336+0000 7fb2b9e00640 1 --2- 172.21.15.83:0/2939523707 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563d02a3bfa0 0x563d02a45f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:38:08.339 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:38:08.336+0000 7fb2b9e00640 1 -- 172.21.15.83:0/2939523707 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563d0277f620 con 0x563d02a3b660 2024-08-19T01:38:08.339 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:38:08.336+0000 7fb2b9e00640 1 --2- 172.21.15.83:0/2939523707 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563d02a3b660 0x563d02a459d0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7fb2b402e4a0 tx=0x7fb2b40c11f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=15c18c9454e2f865 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:38:08.353 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:38:08.348+0000 7fb29b400640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-08-19T01:38:08.353 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:38:08.348+0000 7fb29b400640 1 -- 172.21.15.83:0/2939523707 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2b40c5560 con 0x563d02a3b660 2024-08-19T01:38:08.353 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:38:08.348+0000 7fb29b400640 1 -- 172.21.15.83:0/2939523707 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8269+0+0 (secure 0 0 0) 0x7fb2b40c5700 con 0x563d02a3b660 2024-08-19T01:38:08.353 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:38:08.351+0000 7fb2c3c20740 1 -- 172.21.15.83:0/2939523707 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb2b40bb130 con 0x563d02a3b660 2024-08-19T01:38:08.353 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:38:08.351+0000 7fb29b400640 1 -- 172.21.15.83:0/2939523707 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fb2b40bc280 con 0x563d02a3b660 2024-08-19T01:38:08.357 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:38:08.355+0000 7fb2c3c20740 10 client.15573 fetch_fsmap learned FSMap version 26 2024-08-19T01:38:08.357 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:38:08.355+0000 7fb2c3c20740 1 -- 172.21.15.83:0/2939523707 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x563d02a440a0 con 0x563d02a3b660 2024-08-19T01:38:08.360 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:38:08.358+0000 7fb29b400640 1 -- 172.21.15.83:0/2939523707 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb2b40bb130 con 0x563d02a3b660 2024-08-19T01:38:08.360 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:38:08.358+0000 7fb29b400640 1 -- 172.21.15.83:0/2939523707 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 26) ==== 8350+0+0 (secure 0 0 0) 0x7fb2b4036440 con 0x563d02a3b660 2024-08-19T01:38:08.360 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:38:08.359+0000 7fb2c3c20740 10 client.15573 fetch_fsmap finished waiting for FSMap version 26 2024-08-19T01:38:08.361 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:38:08.359+0000 7fb2c3c20740 10 client.15573 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-08-19T01:38:08.361 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:38:08.359+0000 7fb2c3c20740 20 client.15573 populate_metadata read hostname 'smithi083' 2024-08-19T01:38:08.361 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:38:08.359+0000 7fb2c3c20740 1 --2- 172.21.15.83:0/2939523707 >> [v2:172.21.15.117:6838/1967595704,v1:172.21.15.117:6839/1967595704] conn(0x563d02a51280 0x563d02a536a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:38:08.361 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:38:08.359+0000 7fb2c3c20740 4 client.15573 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-19T01:38:08.361 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:38:08.359+0000 7fb2c3c20740 1 -- 172.21.15.83:0/2939523707 --> [v2:172.21.15.117:6838/1967595704,v1:172.21.15.117:6839/1967595704] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563d029acfc0 con 0x563d02a51280 2024-08-19T01:38:08.361 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:38:08.359+0000 7fb2b9400640 1 --2- 172.21.15.83:0/2939523707 >> [v2:172.21.15.117:6838/1967595704,v1:172.21.15.117:6839/1967595704] conn(0x563d02a51280 0x563d02a536a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:38:08.361 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:38:08.360+0000 7fb2b9400640 1 --2- 172.21.15.83:0/2939523707 >> [v2:172.21.15.117:6838/1967595704,v1:172.21.15.117:6839/1967595704] conn(0x563d02a51280 0x563d02a536a0 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e54fe99d2da89bbc server_cookie=cec44439b3a889b3 in_seq=0 out_seq=0 2024-08-19T01:38:08.362 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:38:08.360+0000 7fb29b400640 10 client.15573 ms_handle_connect on v2:172.21.15.117:6838/1967595704 2024-08-19T01:38:08.365 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:38:08.364+0000 7fb29b400640 1 -- 172.21.15.83:0/2939523707 <== mds.0 v2:172.21.15.117:6838/1967595704 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x563d029acfc0 con 0x563d02a51280 2024-08-19T01:38:08.365 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:38:08.364+0000 7fb29b400640 10 client.15573 handle_command_reply: tid=0 2024-08-19T01:38:08.366 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:38:08.364+0000 7fb2c3c20740 1 client.15573 shutdown 2024-08-19T01:38:08.374 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:38:08.373+0000 7fb2c3c20740 1 -- 172.21.15.83:0/2939523707 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563d02a3b660 msgr2=0x563d02a459d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:38:08.374 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:38:08.373+0000 7fb2c3c20740 1 --2- 172.21.15.83:0/2939523707 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563d02a3b660 0x563d02a459d0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7fb2b402e4a0 tx=0x7fb2b40c11f0 comp rx=0 tx=0).stop 2024-08-19T01:38:08.375 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:38:08.373+0000 7fb2c3c20740 1 -- 172.21.15.83:0/2939523707 shutdown_connections 2024-08-19T01:38:08.375 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:38:08.373+0000 7fb2c3c20740 1 --2- 172.21.15.83:0/2939523707 >> [v2:172.21.15.117:6838/1967595704,v1:172.21.15.117:6839/1967595704] conn(0x563d02a51280 0x563d02a536a0 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:38:08.375 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:38:08.373+0000 7fb29b400640 0 client.15573 ms_handle_reset on v2:172.21.15.117:6838/1967595704 2024-08-19T01:38:08.375 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:38:08.373+0000 7fb2c3c20740 1 --2- 172.21.15.83:0/2939523707 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563d02a41030 0x563d02a4cee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:38:08.375 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:38:08.373+0000 7fb2c3c20740 1 --2- 172.21.15.83:0/2939523707 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563d02a3bfa0 0x563d02a45f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:38:08.375 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:38:08.373+0000 7fb2c3c20740 1 --2- 172.21.15.83:0/2939523707 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563d02a3b660 0x563d02a459d0 secure :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7fb2b402e4a0 tx=0x7fb2b40c11f0 comp rx=0 tx=0).stop 2024-08-19T01:38:08.375 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:38:08.373+0000 7fb2c3c20740 1 -- 172.21.15.83:0/2939523707 >> 172.21.15.83:0/2939523707 conn(0x563d02a36cd0 msgr2=0x563d02a384e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:38:08.379 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:38:08.378+0000 7fb2c3c20740 1 -- 172.21.15.83:0/2939523707 shutdown_connections 2024-08-19T01:38:08.379 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:38:08.378+0000 7fb2c3c20740 1 -- 172.21.15.83:0/2939523707 wait complete. 2024-08-19T01:38:08.393 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:38:08.378+0000 7fb2c3c20740 20 client.15573 trim_cache size 0 max 16384 2024-08-19T01:38:08.670 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:38:08.663+0000 7fb2c3c20740 1 Processor -- start 2024-08-19T01:38:08.670 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:38:08.667+0000 7fb2c3c20740 1 -- start start 2024-08-19T01:38:08.670 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:38:08.667+0000 7fb2c3c20740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563d02a3bfa0 0x563d02a35e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:38:08.670 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:38:08.667+0000 7fb2c3c20740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563d02a41030 0x563d02a4ce70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:38:08.670 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:38:08.667+0000 7fb2c3c20740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563d02a380c0 0x563d02a38540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:38:08.671 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:38:08.667+0000 7fb2c3c20740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fb2bc126f40 con 0x563d02a380c0 2024-08-19T01:38:08.671 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:38:08.667+0000 7fb2c3c20740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fb2bc124810 con 0x563d02a3bfa0 2024-08-19T01:38:08.671 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:38:08.667+0000 7fb2c3c20740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fb2bc122660 con 0x563d02a41030 2024-08-19T01:38:08.671 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:38:08.667+0000 7fb2b9e00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563d02a3bfa0 0x563d02a35e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:38:08.671 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:38:08.667+0000 7fb2b9e00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563d02a3bfa0 0x563d02a35e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.83:38408/0 (socket says 172.21.15.83:38408) 2024-08-19T01:38:08.671 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:38:08.667+0000 7fb2b9e00640 1 -- 172.21.15.83:0/2612333983 learned_addr learned my addr 172.21.15.83:0/2612333983 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-08-19T01:38:08.671 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:38:08.668+0000 7fb2ba800640 1 --2- 172.21.15.83:0/2612333983 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563d02a380c0 0x563d02a38540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:38:08.671 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:38:08.668+0000 7fb2b9400640 1 --2- 172.21.15.83:0/2612333983 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563d02a41030 0x563d02a4ce70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:38:08.672 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:38:08.668+0000 7fb2b9400640 1 -- 172.21.15.83:0/2612333983 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563d02a3bfa0 msgr2=0x563d02a35e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:38:08.672 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:38:08.668+0000 7fb2b9400640 1 --2- 172.21.15.83:0/2612333983 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563d02a3bfa0 0x563d02a35e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:38:08.672 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:38:08.668+0000 7fb2b9400640 1 -- 172.21.15.83:0/2612333983 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563d02a380c0 msgr2=0x563d02a38540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:38:08.672 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:38:08.668+0000 7fb2b9400640 1 --2- 172.21.15.83:0/2612333983 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563d02a380c0 0x563d02a38540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:38:08.672 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:38:08.668+0000 7fb2b9400640 1 -- 172.21.15.83:0/2612333983 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563d02a36af0 con 0x563d02a41030 2024-08-19T01:38:08.672 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:38:08.669+0000 7fb2ba800640 1 --2- 172.21.15.83:0/2612333983 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563d02a380c0 0x563d02a38540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-19T01:38:08.673 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:38:08.669+0000 7fb2b9400640 1 --2- 172.21.15.83:0/2612333983 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563d02a41030 0x563d02a4ce70 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x563d02a53ef0 tx=0x7fb2a8013000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e567aaba5c584536 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:38:08.673 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:38:08.669+0000 7fb2a6600640 1 -- 172.21.15.83:0/2612333983 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2a80285a0 con 0x563d02a41030 2024-08-19T01:38:08.673 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:38:08.670+0000 7fb2a6600640 1 -- 172.21.15.83:0/2612333983 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb2a802e070 con 0x563d02a41030 2024-08-19T01:38:08.673 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:38:08.670+0000 7fb2a6600640 1 -- 172.21.15.83:0/2612333983 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2a801fd30 con 0x563d02a41030 2024-08-19T01:38:08.685 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:38:08.679+0000 7fb2c3c20740 1 -- 172.21.15.83:0/2612333983 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563d02a41030 msgr2=0x563d02a4ce70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:38:08.685 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:38:08.679+0000 7fb2c3c20740 1 --2- 172.21.15.83:0/2612333983 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563d02a41030 0x563d02a4ce70 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x563d02a53ef0 tx=0x7fb2a8013000 comp rx=0 tx=0).stop 2024-08-19T01:38:08.685 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:38:08.684+0000 7fb2c3c20740 1 -- 172.21.15.83:0/2612333983 shutdown_connections 2024-08-19T01:38:08.686 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:38:08.684+0000 7fb2c3c20740 1 --2- 172.21.15.83:0/2612333983 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563d02a380c0 0x563d02a38540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:38:08.686 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:38:08.684+0000 7fb2c3c20740 1 --2- 172.21.15.83:0/2612333983 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563d02a41030 0x563d02a4ce70 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:38:08.686 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:38:08.684+0000 7fb2c3c20740 1 --2- 172.21.15.83:0/2612333983 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563d02a3bfa0 0x563d02a35e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:38:08.686 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:38:08.684+0000 7fb2c3c20740 1 -- 172.21.15.83:0/2612333983 >> 172.21.15.83:0/2612333983 conn(0x563d02a36cd0 msgr2=0x563d02a35720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:38:08.691 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:38:08.690+0000 7fb2c3c20740 1 -- 172.21.15.83:0/2612333983 shutdown_connections 2024-08-19T01:38:08.693 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:38:08.690+0000 7fb2c3c20740 1 -- 172.21.15.83:0/2612333983 wait complete. 2024-08-19T01:38:08.693 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:38:08.691+0000 7fb2c3c20740 1 -- start start 2024-08-19T01:38:08.693 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:38:08.691+0000 7fb2c3c20740 1 Processor -- start 2024-08-19T01:38:08.693 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:38:08.691+0000 7fb2c3c20740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563d02a380c0 0x563d02a4ac00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:38:08.694 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:38:08.691+0000 7fb2c3c20740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563d02a3bfa0 0x563d02a4b140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:38:08.694 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:38:08.691+0000 7fb2c3c20740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563d02a41030 0x563d02a4c150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:38:08.694 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:38:08.691+0000 7fb2c3c20740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fb2bc122cd0 con 0x563d02a3bfa0 2024-08-19T01:38:08.694 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:38:08.691+0000 7fb2c3c20740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fb2bc124140 con 0x563d02a41030 2024-08-19T01:38:08.694 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:38:08.691+0000 7fb2c3c20740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fb2bc1233a0 con 0x563d02a380c0 2024-08-19T01:38:08.694 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:38:08.692+0000 7fb2c3c20740 10 client.0 _ll_register_callbacks cb 0x7fb2b40b90d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-19T01:38:08.695 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:38:08.692+0000 7fb2b9400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563d02a3bfa0 0x563d02a4b140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:38:08.695 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:38:08.692+0000 7fb2b9400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563d02a3bfa0 0x563d02a4b140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:56846/0 (socket says 172.21.15.83:56846) 2024-08-19T01:38:08.695 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:38:08.692+0000 7fb2b9400640 1 -- 172.21.15.83:0/1372099217 learned_addr learned my addr 172.21.15.83:0/1372099217 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-08-19T01:38:08.695 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:38:08.692+0000 7fb2b9e00640 1 --2- 172.21.15.83:0/1372099217 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563d02a380c0 0x563d02a4ac00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:38:08.695 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:38:08.693+0000 7fb2b9e00640 1 -- 172.21.15.83:0/1372099217 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563d02a41030 msgr2=0x563d02a4c150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:38:08.696 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:38:08.693+0000 7fb2ba800640 1 --2- 172.21.15.83:0/1372099217 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563d02a41030 0x563d02a4c150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:38:08.696 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:38:08.693+0000 7fb2b9e00640 1 --2- 172.21.15.83:0/1372099217 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563d02a41030 0x563d02a4c150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:38:08.697 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:38:08.693+0000 7fb2b9e00640 1 -- 172.21.15.83:0/1372099217 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563d02a3bfa0 msgr2=0x563d02a4b140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:38:08.697 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:38:08.693+0000 7fb2b9e00640 1 --2- 172.21.15.83:0/1372099217 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563d02a3bfa0 0x563d02a4b140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:38:08.697 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:38:08.693+0000 7fb2b9e00640 1 -- 172.21.15.83:0/1372099217 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563d02a4c760 con 0x563d02a380c0 2024-08-19T01:38:08.697 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:38:08.693+0000 7fb2b9400640 1 --2- 172.21.15.83:0/1372099217 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563d02a3bfa0 0x563d02a4b140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-19T01:38:08.697 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:38:08.694+0000 7fb2b9e00640 1 --2- 172.21.15.83:0/1372099217 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563d02a380c0 0x563d02a4ac00 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fb2b40bb0e0 tx=0x7fb2b40bca30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ed2e3763b5b7b2af server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:38:08.697 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:38:08.694+0000 7fb29b400640 10 client.0 ms_handle_connect on v2:172.21.15.167:3300/0 2024-08-19T01:38:08.697 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:38:08.694+0000 7fb29b400640 1 -- 172.21.15.83:0/1372099217 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2b40c59f0 con 0x563d02a380c0 2024-08-19T01:38:08.697 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:38:08.694+0000 7fb29b400640 1 -- 172.21.15.83:0/1372099217 <== mon.1 v2:172.21.15.167:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8269+0+0 (secure 0 0 0) 0x7fb2b40c5560 con 0x563d02a380c0 2024-08-19T01:38:08.698 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:38:08.695+0000 7fb2ba800640 1 --2- 172.21.15.83:0/1372099217 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563d02a41030 0x563d02a4c150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-19T01:38:08.698 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:38:08.695+0000 7fb2c3c20740 1 -- 172.21.15.83:0/1372099217 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb2a802e070 con 0x563d02a380c0 2024-08-19T01:38:08.699 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:38:08.696+0000 7fb29b400640 1 -- 172.21.15.83:0/1372099217 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fb2b409a050 con 0x563d02a380c0 2024-08-19T01:38:08.699 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:38:08.696+0000 7fb2c3c20740 10 client.25582 fetch_fsmap learned FSMap version 26 2024-08-19T01:38:08.699 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:38:08.696+0000 7fb2c3c20740 1 -- 172.21.15.83:0/1372099217 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x563d02abc4a0 con 0x563d02a380c0 2024-08-19T01:38:08.699 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:38:08.697+0000 7fb29b400640 1 -- 172.21.15.83:0/1372099217 <== mon.1 v2:172.21.15.167:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb2b409f020 con 0x563d02a380c0 2024-08-19T01:38:08.699 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:38:08.697+0000 7fb29b400640 1 -- 172.21.15.83:0/1372099217 <== mon.1 v2:172.21.15.167:3300/0 5 ==== fsmap(e 26) ==== 8350+0+0 (secure 0 0 0) 0x7fb2b40bbcf0 con 0x563d02a380c0 2024-08-19T01:38:08.700 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:38:08.698+0000 7fb2c3c20740 10 client.25582 fetch_fsmap finished waiting for FSMap version 26 2024-08-19T01:38:08.700 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:38:08.698+0000 7fb2c3c20740 10 client.25582 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-08-19T01:38:08.700 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:38:08.698+0000 7fb2c3c20740 20 client.25582 populate_metadata read hostname 'smithi083' 2024-08-19T01:38:08.700 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:38:08.698+0000 7fb2c3c20740 1 --2- 172.21.15.83:0/1372099217 >> [v2:172.21.15.117:6838/1967595704,v1:172.21.15.117:6839/1967595704] conn(0x563d02abcc20 0x563d02abf040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:38:08.700 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:38:08.698+0000 7fb2c3c20740 4 client.25582 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-08-19T01:38:08.700 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:38:08.698+0000 7fb2c3c20740 1 -- 172.21.15.83:0/1372099217 --> [v2:172.21.15.117:6838/1967595704,v1:172.21.15.117:6839/1967595704] -- command(tid 0: {"prefix": "flush journal"}) -- 0x563d029a7310 con 0x563d02abcc20 2024-08-19T01:38:08.700 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:38:08.698+0000 7fb2b9400640 1 --2- 172.21.15.83:0/1372099217 >> [v2:172.21.15.117:6838/1967595704,v1:172.21.15.117:6839/1967595704] conn(0x563d02abcc20 0x563d02abf040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:38:08.704 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:38:08.703+0000 7fb29b400640 10 client.25582 ms_handle_connect on v2:172.21.15.117:6838/1967595704 2024-08-19T01:38:08.704 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:38:08.703+0000 7fb2b9400640 1 --2- 172.21.15.83:0/1372099217 >> [v2:172.21.15.117:6838/1967595704,v1:172.21.15.117:6839/1967595704] conn(0x563d02abcc20 0x563d02abf040 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a51c6ee846fb7d9d server_cookie=f7213df2f25310df in_seq=0 out_seq=0 2024-08-19T01:38:08.809 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:38:08 smithi083 ceph-mon[17120]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-19T01:38:08.952 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:38:08 smithi117 ceph-mon[22016]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-19T01:38:08.965 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:38:08 smithi167 ceph-mon[21875]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-19T01:38:09.907 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:38:09 smithi083 ceph-mon[17120]: pgmap v438: 97 pgs: 97 active+clean; 248 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 2.1 MiB/s wr, 446 op/s 2024-08-19T01:38:09.952 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:38:09 smithi117 ceph-mon[22016]: pgmap v438: 97 pgs: 97 active+clean; 248 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 2.1 MiB/s wr, 446 op/s 2024-08-19T01:38:09.966 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:38:09 smithi167 ceph-mon[21875]: pgmap v438: 97 pgs: 97 active+clean; 248 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 2.1 MiB/s wr, 446 op/s 2024-08-19T01:38:10.599 INFO:tasks.workunit.client.0.smithi083.stdout: 90 587959 544 386560 307 341030 1302 2024-08-19T01:38:11.091 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:38:11.075+0000 7fb29b400640 1 -- 172.21.15.83:0/1372099217 <== mds.0 v2:172.21.15.117:6838/1967595704 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x563d029a7310 con 0x563d02abcc20 2024-08-19T01:38:11.091 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:38:11.075+0000 7fb29b400640 10 client.25582 handle_command_reply: tid=0 2024-08-19T01:38:11.091 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:38:11.087+0000 7fb2c3c20740 1 client.25582 shutdown 2024-08-19T01:38:11.140 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:38:11.127+0000 7fb2c3c20740 1 -- 172.21.15.83:0/1372099217 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563d02a380c0 msgr2=0x563d02a4ac00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:38:11.140 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:38:11.127+0000 7fb2c3c20740 1 --2- 172.21.15.83:0/1372099217 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563d02a380c0 0x563d02a4ac00 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fb2b40bb0e0 tx=0x7fb2b40bca30 comp rx=0 tx=0).stop 2024-08-19T01:38:11.173 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:38:11.164+0000 7fb2c3c20740 1 -- 172.21.15.83:0/1372099217 shutdown_connections 2024-08-19T01:38:11.173 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:38:11.164+0000 7fb2c3c20740 1 --2- 172.21.15.83:0/1372099217 >> [v2:172.21.15.117:6838/1967595704,v1:172.21.15.117:6839/1967595704] conn(0x563d02abcc20 0x563d02abf040 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:38:11.173 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:38:11.164+0000 7fb2c3c20740 1 --2- 172.21.15.83:0/1372099217 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563d02a41030 0x563d02a4c150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:38:11.173 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:38:11.164+0000 7fb2c3c20740 1 --2- 172.21.15.83:0/1372099217 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563d02a3bfa0 0x563d02a4b140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:38:11.173 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:38:11.164+0000 7fb2c3c20740 1 --2- 172.21.15.83:0/1372099217 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563d02a380c0 0x563d02a4ac00 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:38:11.173 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:38:11.164+0000 7fb2c3c20740 1 -- 172.21.15.83:0/1372099217 >> 172.21.15.83:0/1372099217 conn(0x563d02a36cd0 msgr2=0x563d02a38540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:38:11.173 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:38:11.168+0000 7fb29b400640 0 client.25582 ms_handle_reset on v2:172.21.15.117:6838/1967595704 2024-08-19T01:38:11.192 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:38:11.190+0000 7fb2c3c20740 1 -- 172.21.15.83:0/1372099217 shutdown_connections 2024-08-19T01:38:11.201 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:38:11.193+0000 7fb2c3c20740 1 -- 172.21.15.83:0/1372099217 wait complete. 2024-08-19T01:38:11.213 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:38:11.212+0000 7fb2c3c20740 20 client.25582 trim_cache size 0 max 16384 2024-08-19T01:38:11.258 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stdout:{ 2024-08-19T01:38:11.258 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stdout: "message": "", 2024-08-19T01:38:11.258 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stdout: "return_code": 0 2024-08-19T01:38:11.258 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stdout:} 2024-08-19T01:38:11.278 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:38:11.276+0000 7fb2bb200640 1 -- 172.21.15.83:0/3993682201 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7fb2a00a2da0 msgr2=0x7fb2a00a5260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:38:11.278 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:38:11.276+0000 7fb2bb200640 1 --2- 172.21.15.83:0/3993682201 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7fb2a00a2da0 0x7fb2a00a5260 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fb2bc11a7b0 tx=0x7fb2a800b390 comp rx=0 tx=0).stop 2024-08-19T01:38:11.278 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:38:11.276+0000 7fb2bb200640 1 -- 172.21.15.83:0/3993682201 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb2bc072800 msgr2=0x7fb2bc11aa40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:38:11.278 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:38:11.276+0000 7fb2bb200640 1 --2- 172.21.15.83:0/3993682201 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb2bc072800 0x7fb2bc11aa40 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fb2b4009fd0 tx=0x7fb2b400beb0 comp rx=0 tx=0).stop 2024-08-19T01:38:11.278 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:38:11.276+0000 7fb2bb200640 1 -- 172.21.15.83:0/3993682201 shutdown_connections 2024-08-19T01:38:11.278 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:38:11.276+0000 7fb2bb200640 1 --2- 172.21.15.83:0/3993682201 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7fb2a00a2da0 0x7fb2a00a5260 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:38:11.278 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:38:11.276+0000 7fb2bb200640 1 --2- 172.21.15.83:0/3993682201 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb2bc1093e0 0x7fb2bc119080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:38:11.278 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:38:11.276+0000 7fb2bb200640 1 --2- 172.21.15.83:0/3993682201 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb2bc0731c0 0x7fb2bc11af80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:38:11.278 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:38:11.276+0000 7fb2bb200640 1 --2- 172.21.15.83:0/3993682201 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb2bc072800 0x7fb2bc11aa40 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:38:11.279 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:38:11.276+0000 7fb2bb200640 1 -- 172.21.15.83:0/3993682201 >> 172.21.15.83:0/3993682201 conn(0x7fb2bc06e880 msgr2=0x7fb2bc10e3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:38:11.307 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:38:11.290+0000 7fb2bb200640 1 -- 172.21.15.83:0/3993682201 shutdown_connections 2024-08-19T01:38:11.307 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:38:11.290+0000 7fb2bb200640 1 -- 172.21.15.83:0/3993682201 wait complete. 2024-08-19T01:38:11.861 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:38:11 smithi083 ceph-mon[17120]: pgmap v439: 97 pgs: 97 active+clean; 250 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.8 MiB/s wr, 302 op/s 2024-08-19T01:38:11.954 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:38:11 smithi117 ceph-mon[22016]: pgmap v439: 97 pgs: 97 active+clean; 250 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.8 MiB/s wr, 302 op/s 2024-08-19T01:38:11.965 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:38:11 smithi167 ceph-mon[21875]: pgmap v439: 97 pgs: 97 active+clean; 250 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.8 MiB/s wr, 302 op/s 2024-08-19T01:38:12.683 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833 shell --fsid bad8780e-5dc9-11ef-bccf-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-08-19T01:38:13.658 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:38:13 smithi083 ceph-mon[17120]: pgmap v440: 97 pgs: 97 active+clean; 251 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.6 MiB/s wr, 488 op/s 2024-08-19T01:38:13.659 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:38:13 smithi083 ceph-mon[17120]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-19T01:38:13.659 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:38:13 smithi083 ceph-mon[17120]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-19T01:38:13.659 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:38:13 smithi083 ceph-mon[17120]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' 2024-08-19T01:38:13.952 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:38:13 smithi117 ceph-mon[22016]: pgmap v440: 97 pgs: 97 active+clean; 251 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.6 MiB/s wr, 488 op/s 2024-08-19T01:38:13.952 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:38:13 smithi117 ceph-mon[22016]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-19T01:38:13.952 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:38:13 smithi117 ceph-mon[22016]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-19T01:38:13.952 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:38:13 smithi117 ceph-mon[22016]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' 2024-08-19T01:38:13.965 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:38:13 smithi167 ceph-mon[21875]: pgmap v440: 97 pgs: 97 active+clean; 251 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.6 MiB/s wr, 488 op/s 2024-08-19T01:38:13.966 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:38:13 smithi167 ceph-mon[21875]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-19T01:38:13.966 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:38:13 smithi167 ceph-mon[21875]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-19T01:38:13.966 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:38:13 smithi167 ceph-mon[21875]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' 2024-08-19T01:38:15.795 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:38:15 smithi083 ceph-mon[17120]: pgmap v441: 97 pgs: 97 active+clean; 239 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.2 MiB/s wr, 909 op/s 2024-08-19T01:38:15.952 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:38:15 smithi117 ceph-mon[22016]: pgmap v441: 97 pgs: 97 active+clean; 239 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.2 MiB/s wr, 909 op/s 2024-08-19T01:38:15.965 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:38:15 smithi167 ceph-mon[21875]: pgmap v441: 97 pgs: 97 active+clean; 239 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.2 MiB/s wr, 909 op/s 2024-08-19T01:38:17.656 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:38:17 smithi083 ceph-mon[17120]: pgmap v442: 97 pgs: 97 active+clean; 226 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.9 MiB/s wr, 921 op/s 2024-08-19T01:38:17.861 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/bad8780e-5dc9-11ef-bccf-c7b262605968/mon.a/config 2024-08-19T01:38:17.952 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:38:17 smithi117 ceph-mon[22016]: pgmap v442: 97 pgs: 97 active+clean; 226 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.9 MiB/s wr, 921 op/s 2024-08-19T01:38:17.965 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:38:17 smithi167 ceph-mon[21875]: pgmap v442: 97 pgs: 97 active+clean; 226 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.9 MiB/s wr, 921 op/s 2024-08-19T01:38:18.959 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:18.928+0000 7f079fe00640 1 -- 172.21.15.83:0/1333155160 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f07a0102090 msgr2=0x7f07a010e5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:38:18.959 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:18.928+0000 7f079fe00640 1 --2- 172.21.15.83:0/1333155160 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f07a0102090 0x7f07a010e5c0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f0798009cd0 tx=0x7f079802a910 comp rx=0 tx=0).stop 2024-08-19T01:38:18.959 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:18.928+0000 7f079fe00640 1 -- 172.21.15.83:0/1333155160 shutdown_connections 2024-08-19T01:38:18.959 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:18.928+0000 7f079fe00640 1 --2- 172.21.15.83:0/1333155160 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f07a0102090 0x7f07a010e5c0 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:38:18.959 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:18.928+0000 7f079fe00640 1 --2- 172.21.15.83:0/1333155160 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f07a01016f0 0x7f07a0101b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:38:18.960 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:18.928+0000 7f079fe00640 1 --2- 172.21.15.83:0/1333155160 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f07a01076f0 0x7f07a0107ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:38:18.960 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:18.928+0000 7f079fe00640 1 -- 172.21.15.83:0/1333155160 >> 172.21.15.83:0/1333155160 conn(0x7f07a00fd470 msgr2=0x7f07a00ff890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:38:18.960 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:18.928+0000 7f079fe00640 1 -- 172.21.15.83:0/1333155160 shutdown_connections 2024-08-19T01:38:18.960 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:18.928+0000 7f079fe00640 1 -- 172.21.15.83:0/1333155160 wait complete. 2024-08-19T01:38:18.960 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:18.929+0000 7f079fe00640 1 Processor -- start 2024-08-19T01:38:18.960 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:18.929+0000 7f079fe00640 1 -- start start 2024-08-19T01:38:18.960 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:18.930+0000 7f079fe00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f07a01016f0 0x7f07a019a270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:38:18.960 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:18.930+0000 7f079fe00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f07a0102090 0x7f07a019a7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:38:18.960 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:18.930+0000 7f079fe00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f07a01076f0 0x7f07a019eb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:38:18.960 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:18.930+0000 7f079fe00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f07a0111b80 con 0x7f07a01076f0 2024-08-19T01:38:18.960 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:18.930+0000 7f079fe00640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f07a0111a00 con 0x7f07a01016f0 2024-08-19T01:38:18.960 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:18.930+0000 7f079fe00640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f07a0111d00 con 0x7f07a0102090 2024-08-19T01:38:18.961 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:18.930+0000 7f079f400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f07a01076f0 0x7f07a019eb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:38:18.961 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:18.930+0000 7f079f400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f07a01076f0 0x7f07a019eb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:51182/0 (socket says 172.21.15.83:51182) 2024-08-19T01:38:18.961 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:18.930+0000 7f079f400640 1 -- 172.21.15.83:0/4168169367 learned_addr learned my addr 172.21.15.83:0/4168169367 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-08-19T01:38:18.961 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:18.930+0000 7f079ea00640 1 --2- 172.21.15.83:0/4168169367 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f07a01016f0 0x7f07a019a270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:38:18.961 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:18.930+0000 7f079e000640 1 --2- 172.21.15.83:0/4168169367 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f07a0102090 0x7f07a019a7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:38:18.961 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:18.930+0000 7f079f400640 1 -- 172.21.15.83:0/4168169367 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f07a0102090 msgr2=0x7f07a019a7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:38:18.961 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:18.931+0000 7f079f400640 1 --2- 172.21.15.83:0/4168169367 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f07a0102090 0x7f07a019a7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:38:18.961 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:18.931+0000 7f079f400640 1 -- 172.21.15.83:0/4168169367 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f07a01016f0 msgr2=0x7f07a019a270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:38:18.961 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:18.931+0000 7f079f400640 1 --2- 172.21.15.83:0/4168169367 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f07a01016f0 0x7f07a019a270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:38:18.961 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:18.931+0000 7f079f400640 1 -- 172.21.15.83:0/4168169367 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f07a019f220 con 0x7f07a01076f0 2024-08-19T01:38:18.961 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:18.931+0000 7f079ea00640 1 --2- 172.21.15.83:0/4168169367 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f07a01016f0 0x7f07a019a270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-19T01:38:18.961 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:18.931+0000 7f079e000640 1 --2- 172.21.15.83:0/4168169367 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f07a0102090 0x7f07a019a7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-19T01:38:18.962 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:18.931+0000 7f079f400640 1 --2- 172.21.15.83:0/4168169367 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f07a01076f0 0x7f07a019eb40 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f0798005e30 tx=0x7f0798003f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:38:18.962 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:18.931+0000 7f0797400640 1 -- 172.21.15.83:0/4168169367 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0798031660 con 0x7f07a01076f0 2024-08-19T01:38:18.962 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:18.931+0000 7f0797400640 1 -- 172.21.15.83:0/4168169367 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0798031800 con 0x7f07a01076f0 2024-08-19T01:38:18.962 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:18.931+0000 7f079fe00640 1 -- 172.21.15.83:0/4168169367 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f07a019f4b0 con 0x7f07a01076f0 2024-08-19T01:38:18.962 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:18.931+0000 7f0797400640 1 -- 172.21.15.83:0/4168169367 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f079802fb30 con 0x7f07a01076f0 2024-08-19T01:38:18.962 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:18.931+0000 7f079fe00640 1 -- 172.21.15.83:0/4168169367 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f07a01a6df0 con 0x7f07a01076f0 2024-08-19T01:38:18.962 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:18.935+0000 7f07a843e740 1 Processor -- start 2024-08-19T01:38:18.962 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:18.935+0000 7f0797400640 1 -- 172.21.15.83:0/4168169367 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7f0798004640 con 0x7f07a01076f0 2024-08-19T01:38:18.962 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:18.935+0000 7f07a843e740 1 -- start start 2024-08-19T01:38:18.962 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:18.936+0000 7f07a843e740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5594106e6d80 0x5594106e7160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:38:18.962 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:18.936+0000 7f07a843e740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5594106e76a0 0x5594106f1a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:38:18.962 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:18.936+0000 7f07a843e740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5594106f1f60 0x5594106f4350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:38:18.962 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:18.936+0000 7f07a843e740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55941064cbf0 con 0x5594106e6d80 2024-08-19T01:38:18.963 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:18.936+0000 7f07a843e740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x5594105e4bd0 con 0x5594106e76a0 2024-08-19T01:38:18.963 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:18.936+0000 7f07a843e740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x559410648bb0 con 0x5594106f1f60 2024-08-19T01:38:18.963 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:18.936+0000 7f079e000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5594106e6d80 0x5594106e7160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:38:18.963 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:18.936+0000 7f079e000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5594106e6d80 0x5594106e7160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:51190/0 (socket says 172.21.15.83:51190) 2024-08-19T01:38:18.963 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:18.936+0000 7f079e000640 1 -- 172.21.15.83:0/1109542223 learned_addr learned my addr 172.21.15.83:0/1109542223 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-08-19T01:38:18.963 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:18.936+0000 7f079e000640 1 --2- 172.21.15.83:0/1109542223 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5594106f1f60 0x5594106f4350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:38:18.963 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:18.937+0000 7f079e000640 1 -- 172.21.15.83:0/1109542223 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5594106f1f60 msgr2=0x5594106f4350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:38:18.963 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:18.937+0000 7f079e000640 1 --2- 172.21.15.83:0/1109542223 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5594106f1f60 0x5594106f4350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:38:18.963 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:18.937+0000 7f079e000640 1 -- 172.21.15.83:0/1109542223 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5594106e76a0 msgr2=0x5594106f1a20 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-08-19T01:38:18.963 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:18.937+0000 7f079e000640 1 --2- 172.21.15.83:0/1109542223 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5594106e76a0 0x5594106f1a20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:38:18.963 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:18.937+0000 7f079e000640 1 -- 172.21.15.83:0/1109542223 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5594105eb8a0 con 0x5594106e6d80 2024-08-19T01:38:18.963 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:18.937+0000 7f0797400640 1 --2- 172.21.15.83:0/4168169367 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f07840a2b80 0x7f07840a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:38:18.963 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:18.937+0000 7f079e000640 1 --2- 172.21.15.83:0/1109542223 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5594106e6d80 0x5594106e7160 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f078c009f10 tx=0x7f078c009950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5e82db7b48250bbe server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:38:18.964 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:18.937+0000 7f0797400640 1 -- 172.21.15.83:0/4168169367 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8269+0+0 (secure 0 0 0) 0x7f07980f02f0 con 0x7f07a01076f0 2024-08-19T01:38:18.964 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:18.937+0000 7f078be00640 1 -- 172.21.15.83:0/1109542223 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f078c0107c0 con 0x5594106e6d80 2024-08-19T01:38:18.964 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:18.937+0000 7f078be00640 1 -- 172.21.15.83:0/1109542223 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f078c01c070 con 0x5594106e6d80 2024-08-19T01:38:18.964 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:18.937+0000 7f078be00640 1 -- 172.21.15.83:0/1109542223 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f078c017ce0 con 0x5594106e6d80 2024-08-19T01:38:18.964 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:18.937+0000 7f079ea00640 1 --2- 172.21.15.83:0/4168169367 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f07840a2b80 0x7f07840a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:38:18.964 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:18.938+0000 7f07a843e740 1 -- 172.21.15.83:0/1109542223 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5594106e6d80 msgr2=0x5594106e7160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:38:18.964 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:18.938+0000 7f07a843e740 1 --2- 172.21.15.83:0/1109542223 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5594106e6d80 0x5594106e7160 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f078c009f10 tx=0x7f078c009950 comp rx=0 tx=0).stop 2024-08-19T01:38:18.964 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:18.938+0000 7f07a843e740 1 -- 172.21.15.83:0/1109542223 shutdown_connections 2024-08-19T01:38:18.964 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:18.938+0000 7f07a843e740 1 --2- 172.21.15.83:0/1109542223 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5594106f1f60 0x5594106f4350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:38:18.964 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:18.938+0000 7f07a843e740 1 --2- 172.21.15.83:0/1109542223 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5594106e76a0 0x5594106f1a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:38:18.964 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:18.938+0000 7f07a843e740 1 --2- 172.21.15.83:0/1109542223 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5594106e6d80 0x5594106e7160 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:38:18.964 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:18.938+0000 7f07a843e740 1 -- 172.21.15.83:0/1109542223 >> 172.21.15.83:0/1109542223 conn(0x5594106e3310 msgr2=0x5594106e36f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:38:18.964 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:18.938+0000 7f079ea00640 1 --2- 172.21.15.83:0/4168169367 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f07840a2b80 0x7f07840a5040 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x5594106e4170 tx=0x7f0790005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:38:18.965 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:18.938+0000 7f07a843e740 1 -- 172.21.15.83:0/1109542223 shutdown_connections 2024-08-19T01:38:18.965 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:18.938+0000 7f07a843e740 1 -- 172.21.15.83:0/1109542223 wait complete. 2024-08-19T01:38:18.965 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:18.939+0000 7f07a843e740 1 -- start start 2024-08-19T01:38:18.965 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:18.939+0000 7f07a843e740 1 Processor -- start 2024-08-19T01:38:18.965 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:18.939+0000 7f07a843e740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5594106e6d80 0x5594106eee20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:38:18.965 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:18.939+0000 7f07a843e740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5594106e76a0 0x5594106ef360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:38:18.965 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:18.939+0000 7f07a843e740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5594106f1f60 0x5594106ef8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:38:18.965 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:18.939+0000 7f07a843e740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x5594106f4b80 con 0x5594106e6d80 2024-08-19T01:38:18.965 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:18.939+0000 7f07a843e740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x559410607d40 con 0x5594106e76a0 2024-08-19T01:38:18.965 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:18.939+0000 7f07a843e740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x5594106f4d00 con 0x5594106f1f60 2024-08-19T01:38:18.965 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:18.939+0000 7f07a843e740 10 client.0 _ll_register_callbacks cb 0x5594105d0b10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-19T01:38:18.965 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:18.940+0000 7f079ea00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5594106e6d80 0x5594106eee20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:38:18.965 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:18.940+0000 7f079ea00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5594106e6d80 0x5594106eee20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:51196/0 (socket says 172.21.15.83:51196) 2024-08-19T01:38:18.965 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:18.940+0000 7f079ea00640 1 -- 172.21.15.83:0/2883759555 learned_addr learned my addr 172.21.15.83:0/2883759555 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-08-19T01:38:18.966 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:18.940+0000 7f079e000640 1 --2- 172.21.15.83:0/2883759555 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5594106e76a0 0x5594106ef360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:38:18.966 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:18.940+0000 7f079ea00640 1 -- 172.21.15.83:0/2883759555 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5594106f1f60 msgr2=0x5594106ef8a0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-08-19T01:38:18.966 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:18.940+0000 7f079ea00640 1 --2- 172.21.15.83:0/2883759555 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5594106f1f60 0x5594106ef8a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:38:18.966 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:18.940+0000 7f079ea00640 1 -- 172.21.15.83:0/2883759555 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5594106e76a0 msgr2=0x5594106ef360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:38:18.966 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:18.940+0000 7f079ea00640 1 --2- 172.21.15.83:0/2883759555 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5594106e76a0 0x5594106ef360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:38:18.966 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:18.940+0000 7f079ea00640 1 -- 172.21.15.83:0/2883759555 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5594103be3c0 con 0x5594106e6d80 2024-08-19T01:38:18.966 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:18.940+0000 7f079e000640 1 --2- 172.21.15.83:0/2883759555 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5594106e76a0 0x5594106ef360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-19T01:38:18.966 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:18.940+0000 7f079ea00640 1 --2- 172.21.15.83:0/2883759555 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5594106e6d80 0x5594106eee20 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f079000fcc0 tx=0x7f0790004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=53b9d32e4ac9336c server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:38:18.966 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:18.940+0000 7f0788c00640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-08-19T01:38:18.966 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:18.940+0000 7f0788c00640 1 -- 172.21.15.83:0/2883759555 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f079001cd30 con 0x5594106e6d80 2024-08-19T01:38:18.966 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:18.941+0000 7f07a843e740 1 -- 172.21.15.83:0/2883759555 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f078c01c070 con 0x5594106e6d80 2024-08-19T01:38:18.966 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:18.941+0000 7f0788c00640 1 -- 172.21.15.83:0/2883759555 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8269+0+0 (secure 0 0 0) 0x7f0790028480 con 0x5594106e6d80 2024-08-19T01:38:18.967 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:18.941+0000 7f0788c00640 1 -- 172.21.15.83:0/2883759555 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f0790028680 con 0x5594106e6d80 2024-08-19T01:38:18.967 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:18.941+0000 7f07a843e740 10 client.15591 fetch_fsmap learned FSMap version 26 2024-08-19T01:38:18.967 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:18.941+0000 7f07a843e740 1 -- 172.21.15.83:0/2883759555 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5594107008b0 con 0x5594106e6d80 2024-08-19T01:38:18.967 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:18.941+0000 7f0788c00640 1 -- 172.21.15.83:0/2883759555 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f078c01c070 con 0x5594106e6d80 2024-08-19T01:38:18.967 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:18.941+0000 7f0788c00640 1 -- 172.21.15.83:0/2883759555 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 26) ==== 8350+0+0 (secure 0 0 0) 0x7f0790033800 con 0x5594106e6d80 2024-08-19T01:38:18.967 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:18.941+0000 7f07a843e740 10 client.15591 fetch_fsmap finished waiting for FSMap version 26 2024-08-19T01:38:18.967 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:18.941+0000 7f07a843e740 10 client.15591 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-19T01:38:18.967 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:18.941+0000 7f07a843e740 20 client.15591 populate_metadata read hostname 'smithi083' 2024-08-19T01:38:18.967 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:18.941+0000 7f07a843e740 1 --2- 172.21.15.83:0/2883759555 >> [v2:172.21.15.117:6838/1967595704,v1:172.21.15.117:6839/1967595704] conn(0x559410701e10 0x5594107042b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:38:18.967 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:18.941+0000 7f07a843e740 4 client.15591 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-19T01:38:18.967 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:18.941+0000 7f07a843e740 1 -- 172.21.15.83:0/2883759555 --> [v2:172.21.15.117:6838/1967595704,v1:172.21.15.117:6839/1967595704] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55941065a160 con 0x559410701e10 2024-08-19T01:38:18.967 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:18.942+0000 7f079e000640 1 --2- 172.21.15.83:0/2883759555 >> [v2:172.21.15.117:6838/1967595704,v1:172.21.15.117:6839/1967595704] conn(0x559410701e10 0x5594107042b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:38:18.967 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:18.942+0000 7f079e000640 1 --2- 172.21.15.83:0/2883759555 >> [v2:172.21.15.117:6838/1967595704,v1:172.21.15.117:6839/1967595704] conn(0x559410701e10 0x5594107042b0 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d4b9e6669ea677d2 server_cookie=882f8b5a320fad40 in_seq=0 out_seq=0 2024-08-19T01:38:18.968 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:18.942+0000 7f0788c00640 10 client.15591 ms_handle_connect on v2:172.21.15.117:6838/1967595704 2024-08-19T01:38:18.968 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:18.943+0000 7f0788c00640 1 -- 172.21.15.83:0/2883759555 <== mds.0 v2:172.21.15.117:6838/1967595704 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55941065a160 con 0x559410701e10 2024-08-19T01:38:18.968 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:18.943+0000 7f0788c00640 10 client.15591 handle_command_reply: tid=0 2024-08-19T01:38:18.968 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:18.943+0000 7f07a843e740 1 client.15591 shutdown 2024-08-19T01:38:18.968 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:18.943+0000 7f07a843e740 1 -- 172.21.15.83:0/2883759555 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5594106e6d80 msgr2=0x5594106eee20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:38:18.968 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:18.943+0000 7f07a843e740 1 --2- 172.21.15.83:0/2883759555 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5594106e6d80 0x5594106eee20 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f079000fcc0 tx=0x7f0790004730 comp rx=0 tx=0).stop 2024-08-19T01:38:18.968 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:18.943+0000 7f07a843e740 1 -- 172.21.15.83:0/2883759555 shutdown_connections 2024-08-19T01:38:18.968 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:18.943+0000 7f07a843e740 1 --2- 172.21.15.83:0/2883759555 >> [v2:172.21.15.117:6838/1967595704,v1:172.21.15.117:6839/1967595704] conn(0x559410701e10 0x5594107042b0 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:38:18.968 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:18.943+0000 7f07a843e740 1 --2- 172.21.15.83:0/2883759555 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5594106f1f60 0x5594106ef8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:38:18.968 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:18.943+0000 7f07a843e740 1 --2- 172.21.15.83:0/2883759555 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5594106e76a0 0x5594106ef360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:38:18.968 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:18.943+0000 7f07a843e740 1 --2- 172.21.15.83:0/2883759555 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5594106e6d80 0x5594106eee20 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:38:18.968 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:18.944+0000 7f0788c00640 0 client.15591 ms_handle_reset on v2:172.21.15.117:6838/1967595704 2024-08-19T01:38:18.968 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:18.944+0000 7f07a843e740 1 -- 172.21.15.83:0/2883759555 >> 172.21.15.83:0/2883759555 conn(0x5594106e3310 msgr2=0x5594106e9040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:38:18.968 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:18.944+0000 7f07a843e740 1 -- 172.21.15.83:0/2883759555 shutdown_connections 2024-08-19T01:38:18.969 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:18.944+0000 7f07a843e740 1 -- 172.21.15.83:0/2883759555 wait complete. 2024-08-19T01:38:18.969 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:18.944+0000 7f07a843e740 20 client.15591 trim_cache size 0 max 16384 2024-08-19T01:38:18.973 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:18.962+0000 7f07a843e740 1 Processor -- start 2024-08-19T01:38:18.973 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:18.962+0000 7f07a843e740 1 -- start start 2024-08-19T01:38:18.973 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:18.962+0000 7f07a843e740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5594106e6d80 0x5594106e8f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:38:18.973 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:18.962+0000 7f07a843e740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5594106e76a0 0x559410709bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:38:18.973 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:18.962+0000 7f07a843e740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5594106f1f60 0x55941070a100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:38:18.973 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:18.962+0000 7f07a843e740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f07a01b1bf0 con 0x5594106f1f60 2024-08-19T01:38:18.973 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:18.962+0000 7f07a843e740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f07a01af4c0 con 0x5594106e6d80 2024-08-19T01:38:18.973 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:18.962+0000 7f07a843e740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f07a01ad310 con 0x5594106e76a0 2024-08-19T01:38:18.973 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:18.962+0000 7f079f400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5594106f1f60 0x55941070a100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:38:18.973 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:18.962+0000 7f079f400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5594106f1f60 0x55941070a100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:51202/0 (socket says 172.21.15.83:51202) 2024-08-19T01:38:18.974 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:18.962+0000 7f079f400640 1 -- 172.21.15.83:0/2752414844 learned_addr learned my addr 172.21.15.83:0/2752414844 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-08-19T01:38:18.974 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:18.963+0000 7f079ea00640 1 --2- 172.21.15.83:0/2752414844 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5594106e6d80 0x5594106e8f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:38:18.974 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:18.963+0000 7f079f400640 1 -- 172.21.15.83:0/2752414844 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5594106e76a0 msgr2=0x559410709bc0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-08-19T01:38:18.974 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:18.963+0000 7f079f400640 1 --2- 172.21.15.83:0/2752414844 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5594106e76a0 0x559410709bc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:38:18.974 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:18.963+0000 7f079f400640 1 -- 172.21.15.83:0/2752414844 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5594106e6d80 msgr2=0x5594106e8f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:38:18.974 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:18.963+0000 7f079f400640 1 --2- 172.21.15.83:0/2752414844 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5594106e6d80 0x5594106e8f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:38:18.974 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:18.963+0000 7f079f400640 1 -- 172.21.15.83:0/2752414844 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5594106e74d0 con 0x5594106f1f60 2024-08-19T01:38:18.974 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:18.963+0000 7f079f400640 1 --2- 172.21.15.83:0/2752414844 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5594106f1f60 0x55941070a100 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f07980b8f30 tx=0x7f07980bc610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b4a3444fc329759b server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:38:18.974 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:18.963+0000 7f078b400640 1 -- 172.21.15.83:0/2752414844 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07980bd1d0 con 0x5594106f1f60 2024-08-19T01:38:18.974 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:18.963+0000 7f078b400640 1 -- 172.21.15.83:0/2752414844 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0798036040 con 0x5594106f1f60 2024-08-19T01:38:18.974 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:18.963+0000 7f078b400640 1 -- 172.21.15.83:0/2752414844 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07980bae60 con 0x5594106f1f60 2024-08-19T01:38:18.974 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:18.964+0000 7f07a843e740 1 -- 172.21.15.83:0/2752414844 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5594106f1f60 msgr2=0x55941070a100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:38:18.974 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:18.964+0000 7f07a843e740 1 --2- 172.21.15.83:0/2752414844 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5594106f1f60 0x55941070a100 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f07980b8f30 tx=0x7f07980bc610 comp rx=0 tx=0).stop 2024-08-19T01:38:18.975 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:18.964+0000 7f07a843e740 1 -- 172.21.15.83:0/2752414844 shutdown_connections 2024-08-19T01:38:18.975 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:18.964+0000 7f07a843e740 1 --2- 172.21.15.83:0/2752414844 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5594106f1f60 0x55941070a100 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:38:18.975 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:18.964+0000 7f07a843e740 1 --2- 172.21.15.83:0/2752414844 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5594106e76a0 0x559410709bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:38:18.975 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:18.964+0000 7f07a843e740 1 --2- 172.21.15.83:0/2752414844 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5594106e6d80 0x5594106e8f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:38:18.975 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:18.964+0000 7f07a843e740 1 -- 172.21.15.83:0/2752414844 >> 172.21.15.83:0/2752414844 conn(0x5594106e3310 msgr2=0x5594106e8820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:38:18.975 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:18.964+0000 7f07a843e740 1 -- 172.21.15.83:0/2752414844 shutdown_connections 2024-08-19T01:38:18.975 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:18.964+0000 7f07a843e740 1 -- 172.21.15.83:0/2752414844 wait complete. 2024-08-19T01:38:18.975 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:18.965+0000 7f07a843e740 1 -- start start 2024-08-19T01:38:18.975 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:18.965+0000 7f07a843e740 1 Processor -- start 2024-08-19T01:38:18.975 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:18.965+0000 7f07a843e740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5594106e6d80 0x5594106fa6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:38:18.975 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:18.965+0000 7f07a843e740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5594106e76a0 0x5594106f5a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:38:18.975 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:18.965+0000 7f07a843e740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5594106f1f60 0x5594106f5f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:38:18.975 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:18.965+0000 7f07a843e740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f07a01ad980 con 0x5594106e76a0 2024-08-19T01:38:18.975 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:18.965+0000 7f07a843e740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f07a01aedf0 con 0x5594106e6d80 2024-08-19T01:38:18.976 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:18.965+0000 7f07a843e740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f07a01ae050 con 0x5594106f1f60 2024-08-19T01:38:18.976 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:18.966+0000 7f07a843e740 10 client.0 _ll_register_callbacks cb 0x7f079000f3c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-19T01:38:18.976 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:18.966+0000 7f079ea00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5594106e6d80 0x5594106fa6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:38:18.976 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:18.966+0000 7f079e000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5594106e76a0 0x5594106f5a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:38:18.976 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:18.966+0000 7f079ea00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5594106e6d80 0x5594106fa6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.83:43286/0 (socket says 172.21.15.83:43286) 2024-08-19T01:38:18.976 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:18.966+0000 7f079ea00640 1 -- 172.21.15.83:0/1606739006 learned_addr learned my addr 172.21.15.83:0/1606739006 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-08-19T01:38:18.976 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:18.966+0000 7f079f400640 1 --2- 172.21.15.83:0/1606739006 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5594106f1f60 0x5594106f5f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:38:18.976 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:18.966+0000 7f079ea00640 1 -- 172.21.15.83:0/1606739006 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5594106f1f60 msgr2=0x5594106f5f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:38:18.976 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:18.966+0000 7f079ea00640 1 --2- 172.21.15.83:0/1606739006 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5594106f1f60 0x5594106f5f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:38:18.976 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:18.966+0000 7f079ea00640 1 -- 172.21.15.83:0/1606739006 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5594106e76a0 msgr2=0x5594106f5a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:38:18.976 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:18.966+0000 7f079ea00640 1 --2- 172.21.15.83:0/1606739006 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5594106e76a0 0x5594106f5a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:38:18.976 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:18.966+0000 7f079ea00640 1 -- 172.21.15.83:0/1606739006 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5594106f6740 con 0x5594106e6d80 2024-08-19T01:38:18.976 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:18.966+0000 7f079f400640 1 --2- 172.21.15.83:0/1606739006 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5594106f1f60 0x5594106f5f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-19T01:38:18.977 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:18.968+0000 7f079ea00640 1 --2- 172.21.15.83:0/1606739006 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5594106e6d80 0x5594106fa6d0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f0790011d10 tx=0x7f0790009de0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=25198c76cc4428d server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:38:18.977 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:18.968+0000 7f0788c00640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-08-19T01:38:18.977 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:18.968+0000 7f0788c00640 1 -- 172.21.15.83:0/1606739006 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0790038040 con 0x5594106e6d80 2024-08-19T01:38:18.977 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:18.968+0000 7f07a843e740 1 -- 172.21.15.83:0/1606739006 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0798036040 con 0x5594106e6d80 2024-08-19T01:38:18.977 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:18.968+0000 7f0788c00640 1 -- 172.21.15.83:0/1606739006 <== mon.2 v2:172.21.15.117:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8269+0+0 (secure 0 0 0) 0x7f0790028480 con 0x5594106e6d80 2024-08-19T01:38:18.977 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:18.969+0000 7f0788c00640 1 -- 172.21.15.83:0/1606739006 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f079001c770 con 0x5594106e6d80 2024-08-19T01:38:18.977 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:18.969+0000 7f07a843e740 10 client.25643 fetch_fsmap learned FSMap version 26 2024-08-19T01:38:18.977 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:18.969+0000 7f07a843e740 1 -- 172.21.15.83:0/1606739006 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55941076cbd0 con 0x5594106e6d80 2024-08-19T01:38:18.977 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:18.969+0000 7f0788c00640 1 -- 172.21.15.83:0/1606739006 <== mon.2 v2:172.21.15.117:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0798036040 con 0x5594106e6d80 2024-08-19T01:38:18.977 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:18.969+0000 7f0788c00640 1 -- 172.21.15.83:0/1606739006 <== mon.2 v2:172.21.15.117:3300/0 5 ==== fsmap(e 26) ==== 8350+0+0 (secure 0 0 0) 0x7f0790046030 con 0x5594106e6d80 2024-08-19T01:38:18.977 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:18.969+0000 7f07a843e740 10 client.25643 fetch_fsmap finished waiting for FSMap version 26 2024-08-19T01:38:18.977 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:18.969+0000 7f07a843e740 10 client.25643 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-19T01:38:18.977 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:18.969+0000 7f07a843e740 20 client.25643 populate_metadata read hostname 'smithi083' 2024-08-19T01:38:18.977 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:18.969+0000 7f07a843e740 1 --2- 172.21.15.83:0/1606739006 >> [v2:172.21.15.117:6838/1967595704,v1:172.21.15.117:6839/1967595704] conn(0x55941076d340 0x55941076f730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:38:18.978 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:18.969+0000 7f07a843e740 4 client.25643 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-08-19T01:38:18.978 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:18.969+0000 7f07a843e740 1 -- 172.21.15.83:0/1606739006 --> [v2:172.21.15.117:6838/1967595704,v1:172.21.15.117:6839/1967595704] -- command(tid 0: {"prefix": "scrub status"}) -- 0x559410654670 con 0x55941076d340 2024-08-19T01:38:18.978 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:18.970+0000 7f079e000640 1 --2- 172.21.15.83:0/1606739006 >> [v2:172.21.15.117:6838/1967595704,v1:172.21.15.117:6839/1967595704] conn(0x55941076d340 0x55941076f730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:38:18.978 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:18.970+0000 7f079e000640 1 --2- 172.21.15.83:0/1606739006 >> [v2:172.21.15.117:6838/1967595704,v1:172.21.15.117:6839/1967595704] conn(0x55941076d340 0x55941076f730 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=168c9db2f9588786 server_cookie=b8ad88a2af17b2be in_seq=0 out_seq=0 2024-08-19T01:38:18.978 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:18.970+0000 7f0788c00640 10 client.25643 ms_handle_connect on v2:172.21.15.117:6838/1967595704 2024-08-19T01:38:18.978 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:18.970+0000 7f0788c00640 1 -- 172.21.15.83:0/1606739006 <== mds.0 v2:172.21.15.117:6838/1967595704 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x559410654670 con 0x55941076d340 2024-08-19T01:38:18.978 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:18.970+0000 7f0788c00640 10 client.25643 handle_command_reply: tid=0 2024-08-19T01:38:18.978 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:18.970+0000 7f07a843e740 1 client.25643 shutdown 2024-08-19T01:38:18.978 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:18.970+0000 7f07a843e740 1 -- 172.21.15.83:0/1606739006 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5594106e6d80 msgr2=0x5594106fa6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:38:18.978 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:18.970+0000 7f07a843e740 1 --2- 172.21.15.83:0/1606739006 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5594106e6d80 0x5594106fa6d0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f0790011d10 tx=0x7f0790009de0 comp rx=0 tx=0).stop 2024-08-19T01:38:18.978 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:18.971+0000 7f07a843e740 1 -- 172.21.15.83:0/1606739006 shutdown_connections 2024-08-19T01:38:18.978 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:18.971+0000 7f07a843e740 1 --2- 172.21.15.83:0/1606739006 >> [v2:172.21.15.117:6838/1967595704,v1:172.21.15.117:6839/1967595704] conn(0x55941076d340 0x55941076f730 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:38:18.978 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:18.971+0000 7f07a843e740 1 --2- 172.21.15.83:0/1606739006 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5594106f1f60 0x5594106f5f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:38:18.979 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:18.971+0000 7f07a843e740 1 --2- 172.21.15.83:0/1606739006 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5594106e76a0 0x5594106f5a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:38:18.979 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:18.971+0000 7f07a843e740 1 --2- 172.21.15.83:0/1606739006 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5594106e6d80 0x5594106fa6d0 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:38:18.979 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:18.971+0000 7f0788c00640 0 client.25643 ms_handle_reset on v2:172.21.15.117:6838/1967595704 2024-08-19T01:38:18.979 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:18.971+0000 7f07a843e740 1 -- 172.21.15.83:0/1606739006 >> 172.21.15.83:0/1606739006 conn(0x5594106e3310 msgr2=0x5594106e9040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:38:18.979 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:18.971+0000 7f07a843e740 1 -- 172.21.15.83:0/1606739006 shutdown_connections 2024-08-19T01:38:18.979 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:18.971+0000 7f07a843e740 1 -- 172.21.15.83:0/1606739006 wait complete. 2024-08-19T01:38:18.979 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:18.971+0000 7f07a843e740 20 client.25643 trim_cache size 0 max 16384 2024-08-19T01:38:18.979 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-08-19T01:38:18.979 INFO:teuthology.orchestra.run.smithi083.stdout: "status": "no active scrubs running", 2024-08-19T01:38:18.979 INFO:teuthology.orchestra.run.smithi083.stdout: "scrubs": {} 2024-08-19T01:38:18.980 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-08-19T01:38:18.980 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:18.971+0000 7f079fe00640 1 -- 172.21.15.83:0/4168169367 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f07840a2b80 msgr2=0x7f07840a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:38:18.980 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:18.971+0000 7f079fe00640 1 --2- 172.21.15.83:0/4168169367 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f07840a2b80 0x7f07840a5040 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x5594106e4170 tx=0x7f0790005f90 comp rx=0 tx=0).stop 2024-08-19T01:38:18.980 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:18.971+0000 7f079fe00640 1 -- 172.21.15.83:0/4168169367 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f07a01076f0 msgr2=0x7f07a019eb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:38:18.980 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:18.971+0000 7f079fe00640 1 --2- 172.21.15.83:0/4168169367 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f07a01076f0 0x7f07a019eb40 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f0798005e30 tx=0x7f0798003f10 comp rx=0 tx=0).stop 2024-08-19T01:38:18.980 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:18.972+0000 7f079fe00640 1 -- 172.21.15.83:0/4168169367 shutdown_connections 2024-08-19T01:38:18.981 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:18.972+0000 7f079fe00640 1 --2- 172.21.15.83:0/4168169367 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f07840a2b80 0x7f07840a5040 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:38:18.981 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:18.972+0000 7f079fe00640 1 --2- 172.21.15.83:0/4168169367 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f07a01076f0 0x7f07a019eb40 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:38:18.981 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:18.972+0000 7f079fe00640 1 --2- 172.21.15.83:0/4168169367 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f07a0102090 0x7f07a019a7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:38:18.981 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:18.972+0000 7f079fe00640 1 --2- 172.21.15.83:0/4168169367 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f07a01016f0 0x7f07a019a270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:38:18.981 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:18.972+0000 7f079fe00640 1 -- 172.21.15.83:0/4168169367 >> 172.21.15.83:0/4168169367 conn(0x7f07a00fd470 msgr2=0x7f07a010c9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:38:18.981 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:18.972+0000 7f079fe00640 1 -- 172.21.15.83:0/4168169367 shutdown_connections 2024-08-19T01:38:18.981 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:18.972+0000 7f079fe00640 1 -- 172.21.15.83:0/4168169367 wait complete. 2024-08-19T01:38:19.912 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:38:19 smithi083 ceph-mon[17120]: pgmap v443: 97 pgs: 97 active+clean; 253 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 3.9 MiB/s wr, 1.10k op/s 2024-08-19T01:38:19.952 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:38:19 smithi117 ceph-mon[22016]: pgmap v443: 97 pgs: 97 active+clean; 253 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 3.9 MiB/s wr, 1.10k op/s 2024-08-19T01:38:19.965 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:38:19 smithi167 ceph-mon[21875]: pgmap v443: 97 pgs: 97 active+clean; 253 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 3.9 MiB/s wr, 1.10k op/s 2024-08-19T01:38:20.629 INFO:tasks.workunit.client.0.smithi083.stdout: 99 593301 508 378882 450 331910 1170 2024-08-19T01:38:20.810 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-08-19T01:38:20.811 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833 shell --fsid bad8780e-5dc9-11ef-bccf-c7b262605968 -- ceph fs dump --format=json 2024-08-19T01:38:21.868 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:38:21 smithi083 ceph-mon[17120]: pgmap v444: 97 pgs: 97 active+clean; 265 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 5.0 MiB/s wr, 1.15k op/s 2024-08-19T01:38:21.952 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:38:21 smithi117 ceph-mon[22016]: pgmap v444: 97 pgs: 97 active+clean; 265 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 5.0 MiB/s wr, 1.15k op/s 2024-08-19T01:38:21.965 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:38:21 smithi167 ceph-mon[21875]: pgmap v444: 97 pgs: 97 active+clean; 265 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 5.0 MiB/s wr, 1.15k op/s 2024-08-19T01:38:23.865 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:38:23 smithi083 ceph-mon[17120]: pgmap v445: 97 pgs: 97 active+clean; 268 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 4.9 MiB/s wr, 1.01k op/s 2024-08-19T01:38:23.952 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:38:23 smithi117 ceph-mon[22016]: pgmap v445: 97 pgs: 97 active+clean; 268 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 4.9 MiB/s wr, 1.01k op/s 2024-08-19T01:38:23.966 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:38:23 smithi167 ceph-mon[21875]: pgmap v445: 97 pgs: 97 active+clean; 268 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 4.9 MiB/s wr, 1.01k op/s 2024-08-19T01:38:25.703 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:38:25 smithi083 ceph-mon[17120]: pgmap v446: 97 pgs: 97 active+clean; 292 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 7.0 MiB/s wr, 960 op/s 2024-08-19T01:38:25.952 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:38:25 smithi117 ceph-mon[22016]: pgmap v446: 97 pgs: 97 active+clean; 292 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 7.0 MiB/s wr, 960 op/s 2024-08-19T01:38:25.965 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:38:25 smithi167 ceph-mon[21875]: pgmap v446: 97 pgs: 97 active+clean; 292 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 7.0 MiB/s wr, 960 op/s 2024-08-19T01:38:26.812 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:38:26.810+0000 7efe21e00640 1 -- 172.21.15.83:0/534699009 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7efe1c109c80 msgr2=0x7efe1c110510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:38:26.812 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:38:26.810+0000 7efe21e00640 1 --2- 172.21.15.83:0/534699009 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7efe1c109c80 0x7efe1c110510 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7efe14009cd0 tx=0x7efe1402a910 comp rx=0 tx=0).stop 2024-08-19T01:38:26.828 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:38:26.827+0000 7efe21e00640 1 -- 172.21.15.83:0/534699009 shutdown_connections 2024-08-19T01:38:26.829 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:38:26.827+0000 7efe21e00640 1 --2- 172.21.15.83:0/534699009 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7efe1c109c80 0x7efe1c110510 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:38:26.829 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:38:26.827+0000 7efe21e00640 1 --2- 172.21.15.83:0/534699009 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7efe1c1092c0 0x7efe1c109740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:38:26.829 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:38:26.827+0000 7efe21e00640 1 --2- 172.21.15.83:0/534699009 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7efe1c073290 0x7efe1c0736b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:38:26.829 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:38:26.827+0000 7efe21e00640 1 -- 172.21.15.83:0/534699009 >> 172.21.15.83:0/534699009 conn(0x7efe1c06ea40 msgr2=0x7efe1c070e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:38:26.829 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:38:26.827+0000 7efe21e00640 1 -- 172.21.15.83:0/534699009 shutdown_connections 2024-08-19T01:38:26.830 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:38:26.827+0000 7efe21e00640 1 -- 172.21.15.83:0/534699009 wait complete. 2024-08-19T01:38:26.830 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:38:26.829+0000 7efe21e00640 1 Processor -- start 2024-08-19T01:38:26.837 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:38:26.832+0000 7efe21e00640 1 -- start start 2024-08-19T01:38:26.837 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:38:26.832+0000 7efe21e00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7efe1c073290 0x7efe1c119210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:38:26.837 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:38:26.833+0000 7efe21e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7efe1c1092c0 0x7efe1c119750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:38:26.837 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:38:26.833+0000 7efe21e00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7efe1c109c80 0x7efe1c1207c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:38:26.837 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:38:26.833+0000 7efe21e00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7efe1c113900 con 0x7efe1c1092c0 2024-08-19T01:38:26.837 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:38:26.833+0000 7efe21e00640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7efe1c113780 con 0x7efe1c109c80 2024-08-19T01:38:26.837 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:38:26.833+0000 7efe21e00640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7efe1c113a80 con 0x7efe1c073290 2024-08-19T01:38:26.837 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:38:26.833+0000 7efe13e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7efe1c1092c0 0x7efe1c119750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:38:26.838 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:38:26.833+0000 7efe13e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7efe1c1092c0 0x7efe1c119750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:51688/0 (socket says 172.21.15.83:51688) 2024-08-19T01:38:26.838 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:38:26.833+0000 7efe13e00640 1 -- 172.21.15.83:0/2573168382 learned_addr learned my addr 172.21.15.83:0/2573168382 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-08-19T01:38:26.838 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:38:26.834+0000 7efe13e00640 1 -- 172.21.15.83:0/2573168382 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7efe1c073290 msgr2=0x7efe1c119210 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-08-19T01:38:26.838 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:38:26.834+0000 7efe13e00640 1 --2- 172.21.15.83:0/2573168382 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7efe1c073290 0x7efe1c119210 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:38:26.838 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:38:26.834+0000 7efe13e00640 1 -- 172.21.15.83:0/2573168382 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7efe1c109c80 msgr2=0x7efe1c1207c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-08-19T01:38:26.838 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:38:26.834+0000 7efe13e00640 1 --2- 172.21.15.83:0/2573168382 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7efe1c109c80 0x7efe1c1207c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:38:26.838 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:38:26.834+0000 7efe13e00640 1 -- 172.21.15.83:0/2573168382 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efe1c120e30 con 0x7efe1c1092c0 2024-08-19T01:38:26.838 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:38:26.835+0000 7efe13e00640 1 --2- 172.21.15.83:0/2573168382 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7efe1c1092c0 0x7efe1c119750 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7efe0400d9d0 tx=0x7efe0400de90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:38:26.850 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:38:26.849+0000 7efe1a000640 1 -- 172.21.15.83:0/2573168382 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe04017070 con 0x7efe1c1092c0 2024-08-19T01:38:26.859 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:38:26.849+0000 7efe1a000640 1 -- 172.21.15.83:0/2573168382 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efe04009e20 con 0x7efe1c1092c0 2024-08-19T01:38:26.860 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:38:26.849+0000 7efe1a000640 1 -- 172.21.15.83:0/2573168382 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe04003e10 con 0x7efe1c1092c0 2024-08-19T01:38:26.860 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:38:26.849+0000 7efe21e00640 1 -- 172.21.15.83:0/2573168382 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efe1c121120 con 0x7efe1c1092c0 2024-08-19T01:38:26.860 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:38:26.849+0000 7efe21e00640 1 -- 172.21.15.83:0/2573168382 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efe1c121620 con 0x7efe1c1092c0 2024-08-19T01:38:26.865 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:38:26.862+0000 7efe1a000640 1 -- 172.21.15.83:0/2573168382 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7efe04004220 con 0x7efe1c1092c0 2024-08-19T01:38:26.865 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:38:26.863+0000 7efe1a000640 1 --2- 172.21.15.83:0/2573168382 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7efe000a2d10 0x7efe000a51d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:38:26.867 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:38:26.864+0000 7efe20a00640 1 --2- 172.21.15.83:0/2573168382 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7efe000a2d10 0x7efe000a51d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:38:26.867 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:38:26.865+0000 7efe20a00640 1 --2- 172.21.15.83:0/2573168382 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7efe000a2d10 0x7efe000a51d0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7efe1c072fb0 tx=0x7efe0c005c20 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:38:26.867 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:38:26.866+0000 7efe2623e740 1 Processor -- start 2024-08-19T01:38:26.868 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:38:26.866+0000 7efe2623e740 1 -- start start 2024-08-19T01:38:26.868 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:38:26.866+0000 7efe2623e740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a85b9c0660 0x55a85b9c0a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:38:26.868 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:38:26.866+0000 7efe2623e740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a85b9c0fa0 0x55a85b9c5a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:38:26.868 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:38:26.866+0000 7efe2623e740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a85b9c6030 0x55a85b9c84f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:38:26.868 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:38:26.866+0000 7efe2623e740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55a85b924890 con 0x55a85b9c0660 2024-08-19T01:38:26.868 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:38:26.867+0000 7efe2623e740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55a85b8bcd90 con 0x55a85b9c6030 2024-08-19T01:38:26.868 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:38:26.867+0000 7efe2623e740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55a85b920850 con 0x55a85b9c0fa0 2024-08-19T01:38:26.870 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:38:26.867+0000 7efe21400640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a85b9c6030 0x55a85b9c84f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:38:26.870 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:38:26.867+0000 7efe21400640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a85b9c6030 0x55a85b9c84f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.83:55878/0 (socket says 172.21.15.83:55878) 2024-08-19T01:38:26.870 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:38:26.867+0000 7efe21400640 1 -- 172.21.15.83:0/330885177 learned_addr learned my addr 172.21.15.83:0/330885177 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-08-19T01:38:26.870 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:38:26.868+0000 7efe20a00640 1 --2- 172.21.15.83:0/330885177 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a85b9c0660 0x55a85b9c0a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:38:26.870 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:38:26.868+0000 7efe21400640 1 -- 172.21.15.83:0/330885177 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a85b9c0fa0 msgr2=0x55a85b9c5a60 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-08-19T01:38:26.870 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:38:26.868+0000 7efe21400640 1 --2- 172.21.15.83:0/330885177 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a85b9c0fa0 0x55a85b9c5a60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:38:26.870 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:38:26.868+0000 7efe21400640 1 -- 172.21.15.83:0/330885177 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a85b9c0660 msgr2=0x55a85b9c0a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:38:26.870 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:38:26.868+0000 7efe21400640 1 --2- 172.21.15.83:0/330885177 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a85b9c0660 0x55a85b9c0a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:38:26.870 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:38:26.868+0000 7efe21400640 1 -- 172.21.15.83:0/330885177 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a85b638210 con 0x55a85b9c6030 2024-08-19T01:38:26.871 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:38:26.869+0000 7efe20a00640 1 --2- 172.21.15.83:0/330885177 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a85b9c0660 0x55a85b9c0a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-19T01:38:26.871 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:38:26.869+0000 7efe1a000640 1 -- 172.21.15.83:0/2573168382 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8269+0+0 (secure 0 0 0) 0x7efe04099240 con 0x7efe1c1092c0 2024-08-19T01:38:26.907 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:38:26.905+0000 7efe21400640 1 --2- 172.21.15.83:0/330885177 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a85b9c6030 0x55a85b9c84f0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7efe1402af00 tx=0x7efe1402f3d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f0edcbce540b5e76 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:38:26.907 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:38:26.905+0000 7efe11c00640 1 -- 172.21.15.83:0/330885177 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe14031460 con 0x55a85b9c6030 2024-08-19T01:38:26.907 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:38:26.905+0000 7efe11c00640 1 -- 172.21.15.83:0/330885177 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efe1403f070 con 0x55a85b9c6030 2024-08-19T01:38:26.907 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:38:26.905+0000 7efe11c00640 1 -- 172.21.15.83:0/330885177 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe14004620 con 0x55a85b9c6030 2024-08-19T01:38:26.922 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:38:26.913+0000 7efe2623e740 1 -- 172.21.15.83:0/330885177 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a85b9c6030 msgr2=0x55a85b9c84f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:38:26.922 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:38:26.913+0000 7efe2623e740 1 --2- 172.21.15.83:0/330885177 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a85b9c6030 0x55a85b9c84f0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7efe1402af00 tx=0x7efe1402f3d0 comp rx=0 tx=0).stop 2024-08-19T01:38:26.922 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:38:26.914+0000 7efe2623e740 1 -- 172.21.15.83:0/330885177 shutdown_connections 2024-08-19T01:38:26.922 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:38:26.914+0000 7efe2623e740 1 --2- 172.21.15.83:0/330885177 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a85b9c6030 0x55a85b9c84f0 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:38:26.922 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:38:26.914+0000 7efe2623e740 1 --2- 172.21.15.83:0/330885177 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a85b9c0fa0 0x55a85b9c5a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:38:26.922 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:38:26.914+0000 7efe2623e740 1 --2- 172.21.15.83:0/330885177 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a85b9c0660 0x55a85b9c0a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:38:26.923 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:38:26.914+0000 7efe2623e740 1 -- 172.21.15.83:0/330885177 >> 172.21.15.83:0/330885177 conn(0x55a85b9bbcd0 msgr2=0x55a85b9bc0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:38:26.961 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:38:26.946+0000 7efe2623e740 1 -- 172.21.15.83:0/330885177 shutdown_connections 2024-08-19T01:38:26.961 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:38:26.955+0000 7efe2623e740 1 -- 172.21.15.83:0/330885177 wait complete. 2024-08-19T01:38:26.961 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:38:26.955+0000 7efe2623e740 1 -- start start 2024-08-19T01:38:26.961 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:38:26.956+0000 7efe2623e740 1 Processor -- start 2024-08-19T01:38:26.962 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:38:26.960+0000 7efe2623e740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a85b9c0660 0x55a85b9caa40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:38:26.963 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:38:26.960+0000 7efe2623e740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a85b9c0fa0 0x55a85b9caf80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:38:26.963 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:38:26.960+0000 7efe2623e740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a85b9c6030 0x55a85b9d1ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:38:26.963 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:38:26.960+0000 7efe2623e740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55a85b9c9040 con 0x55a85b9c0660 2024-08-19T01:38:26.963 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:38:26.960+0000 7efe2623e740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55a85b8dff70 con 0x55a85b9c6030 2024-08-19T01:38:26.963 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:38:26.960+0000 7efe2623e740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55a85b9c91c0 con 0x55a85b9c0fa0 2024-08-19T01:38:26.963 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:38:26.960+0000 7efe2623e740 10 client.0 _ll_register_callbacks cb 0x55a85b91c8d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-19T01:38:26.963 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:38:26.961+0000 7efe20a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a85b9c0660 0x55a85b9caa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:38:26.965 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:38:26.961+0000 7efe20a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a85b9c0660 0x55a85b9caa40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:51716/0 (socket says 172.21.15.83:51716) 2024-08-19T01:38:26.965 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:38:26.961+0000 7efe20a00640 1 -- 172.21.15.83:0/2669351533 learned_addr learned my addr 172.21.15.83:0/2669351533 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-08-19T01:38:26.965 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:38:26.962+0000 7efe20a00640 1 -- 172.21.15.83:0/2669351533 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a85b9c0fa0 msgr2=0x55a85b9caf80 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-08-19T01:38:26.965 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:38:26.962+0000 7efe20a00640 1 --2- 172.21.15.83:0/2669351533 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a85b9c0fa0 0x55a85b9caf80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:38:26.965 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:38:26.962+0000 7efe20a00640 1 -- 172.21.15.83:0/2669351533 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a85b9c6030 msgr2=0x55a85b9d1ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:38:26.965 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:38:26.962+0000 7efe20a00640 1 --2- 172.21.15.83:0/2669351533 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a85b9c6030 0x55a85b9d1ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:38:26.965 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:38:26.962+0000 7efe20a00640 1 -- 172.21.15.83:0/2669351533 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a85b704620 con 0x55a85b9c0660 2024-08-19T01:38:26.965 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:38:26.963+0000 7efe20a00640 1 --2- 172.21.15.83:0/2669351533 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a85b9c0660 0x55a85b9caa40 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7efe0c00ce80 tx=0x7efe0c004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=30792b3d8427ad4c server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:38:26.973 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:38:26.972+0000 7efe0a000640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-08-19T01:38:26.973 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:38:26.972+0000 7efe0a000640 1 -- 172.21.15.83:0/2669351533 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe0c021020 con 0x55a85b9c0660 2024-08-19T01:38:26.973 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:38:26.972+0000 7efe0a000640 1 -- 172.21.15.83:0/2669351533 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8269+0+0 (secure 0 0 0) 0x7efe0c00f650 con 0x55a85b9c0660 2024-08-19T01:38:26.988 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:38:26.986+0000 7efe2623e740 1 -- 172.21.15.83:0/2669351533 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efe1403f070 con 0x55a85b9c0660 2024-08-19T01:38:26.996 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:38:26.994+0000 7efe0a000640 1 -- 172.21.15.83:0/2669351533 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7efe0c018040 con 0x55a85b9c0660 2024-08-19T01:38:27.034 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:38:27.004+0000 7efe2623e740 10 client.15603 fetch_fsmap learned FSMap version 26 2024-08-19T01:38:27.035 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:38:27.004+0000 7efe2623e740 1 -- 172.21.15.83:0/2669351533 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55a85b9d50a0 con 0x55a85b9c0660 2024-08-19T01:38:27.035 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:38:27.004+0000 7efe0a000640 1 -- 172.21.15.83:0/2669351533 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efe1403f070 con 0x55a85b9c0660 2024-08-19T01:38:27.035 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:38:27.004+0000 7efe0a000640 1 -- 172.21.15.83:0/2669351533 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 26) ==== 8350+0+0 (secure 0 0 0) 0x7efe0c01da90 con 0x55a85b9c0660 2024-08-19T01:38:27.035 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:38:27.005+0000 7efe2623e740 10 client.15603 fetch_fsmap finished waiting for FSMap version 26 2024-08-19T01:38:27.035 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:38:27.005+0000 7efe2623e740 10 client.15603 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-08-19T01:38:27.035 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:38:27.005+0000 7efe2623e740 20 client.15603 populate_metadata read hostname 'smithi083' 2024-08-19T01:38:27.035 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:38:27.005+0000 7efe2623e740 1 --2- 172.21.15.83:0/2669351533 >> [v2:172.21.15.117:6838/1967595704,v1:172.21.15.117:6839/1967595704] conn(0x55a85b9d6810 0x55a85b9d8c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:38:27.035 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:38:27.005+0000 7efe2623e740 4 client.15603 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-19T01:38:27.035 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:38:27.005+0000 7efe2623e740 1 -- 172.21.15.83:0/2669351533 --> [v2:172.21.15.117:6838/1967595704,v1:172.21.15.117:6839/1967595704] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a85b931fc0 con 0x55a85b9d6810 2024-08-19T01:38:27.035 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:38:27.005+0000 7efe13e00640 1 --2- 172.21.15.83:0/2669351533 >> [v2:172.21.15.117:6838/1967595704,v1:172.21.15.117:6839/1967595704] conn(0x55a85b9d6810 0x55a85b9d8c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:38:27.035 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:38:27.006+0000 7efe13e00640 1 --2- 172.21.15.83:0/2669351533 >> [v2:172.21.15.117:6838/1967595704,v1:172.21.15.117:6839/1967595704] conn(0x55a85b9d6810 0x55a85b9d8c90 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8cb9918e232218f0 server_cookie=fdd5318177f96a70 in_seq=0 out_seq=0 2024-08-19T01:38:27.036 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:38:27.006+0000 7efe0a000640 10 client.15603 ms_handle_connect on v2:172.21.15.117:6838/1967595704 2024-08-19T01:38:27.036 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:38:27.006+0000 7efe0a000640 1 -- 172.21.15.83:0/2669351533 <== mds.0 v2:172.21.15.117:6838/1967595704 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55a85b931fc0 con 0x55a85b9d6810 2024-08-19T01:38:27.036 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:38:27.006+0000 7efe0a000640 10 client.15603 handle_command_reply: tid=0 2024-08-19T01:38:27.036 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:38:27.007+0000 7efe2623e740 1 client.15603 shutdown 2024-08-19T01:38:27.036 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:38:27.007+0000 7efe2623e740 1 -- 172.21.15.83:0/2669351533 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a85b9c0660 msgr2=0x55a85b9caa40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:38:27.036 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:38:27.007+0000 7efe2623e740 1 --2- 172.21.15.83:0/2669351533 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a85b9c0660 0x55a85b9caa40 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7efe0c00ce80 tx=0x7efe0c004730 comp rx=0 tx=0).stop 2024-08-19T01:38:27.036 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:38:27.007+0000 7efe2623e740 1 -- 172.21.15.83:0/2669351533 shutdown_connections 2024-08-19T01:38:27.036 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:38:27.007+0000 7efe2623e740 1 --2- 172.21.15.83:0/2669351533 >> [v2:172.21.15.117:6838/1967595704,v1:172.21.15.117:6839/1967595704] conn(0x55a85b9d6810 0x55a85b9d8c90 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:38:27.036 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:38:27.007+0000 7efe2623e740 1 --2- 172.21.15.83:0/2669351533 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a85b9c6030 0x55a85b9d1ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:38:27.036 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:38:27.007+0000 7efe2623e740 1 --2- 172.21.15.83:0/2669351533 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a85b9c0fa0 0x55a85b9caf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:38:27.036 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:38:27.007+0000 7efe2623e740 1 --2- 172.21.15.83:0/2669351533 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a85b9c0660 0x55a85b9caa40 secure :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7efe0c00ce80 tx=0x7efe0c004730 comp rx=0 tx=0).stop 2024-08-19T01:38:27.036 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:38:27.007+0000 7efe2623e740 1 -- 172.21.15.83:0/2669351533 >> 172.21.15.83:0/2669351533 conn(0x55a85b9bbcd0 msgr2=0x55a85b9bd4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:38:27.036 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:38:27.007+0000 7efe0a000640 0 client.15603 ms_handle_reset on v2:172.21.15.117:6838/1967595704 2024-08-19T01:38:27.036 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:38:27.008+0000 7efe2623e740 1 -- 172.21.15.83:0/2669351533 shutdown_connections 2024-08-19T01:38:27.036 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:38:27.008+0000 7efe2623e740 1 -- 172.21.15.83:0/2669351533 wait complete. 2024-08-19T01:38:27.037 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:38:27.008+0000 7efe2623e740 20 client.15603 trim_cache size 0 max 16384 2024-08-19T01:38:27.098 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:38:27.094+0000 7efe2623e740 1 Processor -- start 2024-08-19T01:38:27.098 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:38:27.094+0000 7efe2623e740 1 -- start start 2024-08-19T01:38:27.098 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:38:27.094+0000 7efe2623e740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a85b9c0fa0 0x55a85b9c0a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:38:27.099 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:38:27.094+0000 7efe2623e740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a85b9c6030 0x55a85b9c22c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:38:27.099 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:38:27.094+0000 7efe2623e740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a85b9c2800 0x55a85b9bcc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:38:27.099 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:38:27.094+0000 7efe2623e740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7efdf8010ed0 con 0x55a85b9c6030 2024-08-19T01:38:27.099 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:38:27.094+0000 7efe2623e740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7efdf800e7a0 con 0x55a85b9c2800 2024-08-19T01:38:27.099 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:38:27.094+0000 7efe2623e740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7efdf800c5f0 con 0x55a85b9c0fa0 2024-08-19T01:38:27.099 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:38:27.095+0000 7efe13e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a85b9c6030 0x55a85b9c22c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:38:27.099 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:38:27.095+0000 7efe13e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a85b9c6030 0x55a85b9c22c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:51726/0 (socket says 172.21.15.83:51726) 2024-08-19T01:38:27.099 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:38:27.095+0000 7efe13e00640 1 -- 172.21.15.83:0/1990374395 learned_addr learned my addr 172.21.15.83:0/1990374395 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-08-19T01:38:27.099 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:38:27.096+0000 7efe13e00640 1 -- 172.21.15.83:0/1990374395 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a85b9c0fa0 msgr2=0x55a85b9c0a40 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-08-19T01:38:27.099 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:38:27.096+0000 7efe13e00640 1 --2- 172.21.15.83:0/1990374395 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a85b9c0fa0 0x55a85b9c0a40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:38:27.099 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:38:27.096+0000 7efe13e00640 1 -- 172.21.15.83:0/1990374395 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a85b9c2800 msgr2=0x55a85b9bcc60 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-08-19T01:38:27.099 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:38:27.096+0000 7efe13e00640 1 --2- 172.21.15.83:0/1990374395 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a85b9c2800 0x55a85b9bcc60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:38:27.099 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:38:27.096+0000 7efe13e00640 1 -- 172.21.15.83:0/1990374395 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a85b9bbaf0 con 0x55a85b9c6030 2024-08-19T01:38:27.099 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:38:27.096+0000 7efe13e00640 1 --2- 172.21.15.83:0/1990374395 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a85b9c6030 0x55a85b9c22c0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7efe040980b0 tx=0x7efe040a0670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e928c1b7c0eca6d7 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:38:27.119 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:38:27.115+0000 7efe11200640 1 -- 172.21.15.83:0/1990374395 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe04097bd0 con 0x55a85b9c6030 2024-08-19T01:38:27.119 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:38:27.115+0000 7efe11200640 1 -- 172.21.15.83:0/1990374395 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efe040b4170 con 0x55a85b9c6030 2024-08-19T01:38:27.119 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:38:27.115+0000 7efe11200640 1 -- 172.21.15.83:0/1990374395 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe04097d70 con 0x55a85b9c6030 2024-08-19T01:38:27.119 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:38:27.116+0000 7efe2623e740 1 -- 172.21.15.83:0/1990374395 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a85b9c6030 msgr2=0x55a85b9c22c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:38:27.119 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:38:27.116+0000 7efe2623e740 1 --2- 172.21.15.83:0/1990374395 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a85b9c6030 0x55a85b9c22c0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7efe040980b0 tx=0x7efe040a0670 comp rx=0 tx=0).stop 2024-08-19T01:38:27.120 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:38:27.116+0000 7efe2623e740 1 -- 172.21.15.83:0/1990374395 shutdown_connections 2024-08-19T01:38:27.120 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:38:27.116+0000 7efe2623e740 1 --2- 172.21.15.83:0/1990374395 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a85b9c2800 0x55a85b9bcc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:38:27.120 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:38:27.116+0000 7efe2623e740 1 --2- 172.21.15.83:0/1990374395 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a85b9c6030 0x55a85b9c22c0 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:38:27.120 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:38:27.116+0000 7efe2623e740 1 --2- 172.21.15.83:0/1990374395 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a85b9c0fa0 0x55a85b9c0a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:38:27.120 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:38:27.116+0000 7efe2623e740 1 -- 172.21.15.83:0/1990374395 >> 172.21.15.83:0/1990374395 conn(0x55a85b9bbcd0 msgr2=0x55a85b9bc0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:38:27.120 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:38:27.117+0000 7efe2623e740 1 -- 172.21.15.83:0/1990374395 shutdown_connections 2024-08-19T01:38:27.120 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:38:27.117+0000 7efe2623e740 1 -- 172.21.15.83:0/1990374395 wait complete. 2024-08-19T01:38:27.120 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:38:27.117+0000 7efe2623e740 1 -- start start 2024-08-19T01:38:27.120 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:38:27.118+0000 7efe2623e740 1 Processor -- start 2024-08-19T01:38:27.120 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:38:27.118+0000 7efe2623e740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a85b9c0fa0 0x55a85b9e86d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:38:27.125 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:38:27.121+0000 7efe2623e740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a85b9c2800 0x55a85b9e8c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:38:27.125 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:38:27.121+0000 7efe13e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a85b9c2800 0x55a85b9e8c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:38:27.125 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:38:27.121+0000 7efe13e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a85b9c2800 0x55a85b9e8c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:51740/0 (socket says 172.21.15.83:51740) 2024-08-19T01:38:27.125 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:38:27.122+0000 7efe20a00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a85b9c0fa0 0x55a85b9e86d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:38:27.125 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:38:27.122+0000 7efe2623e740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a85b9c6030 0x55a85b9e9c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:38:27.125 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:38:27.122+0000 7efe2623e740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7efdf800cc60 con 0x55a85b9c2800 2024-08-19T01:38:27.125 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:38:27.122+0000 7efe2623e740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7efdf800e0d0 con 0x55a85b9c6030 2024-08-19T01:38:27.126 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:38:27.122+0000 7efe2623e740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7efdf800d330 con 0x55a85b9c0fa0 2024-08-19T01:38:27.126 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:38:27.122+0000 7efe20a00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a85b9c0fa0 0x55a85b9e86d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.83:46138/0 (socket says 172.21.15.83:46138) 2024-08-19T01:38:27.126 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:38:27.122+0000 7efe20a00640 1 -- 172.21.15.83:0/3197790852 learned_addr learned my addr 172.21.15.83:0/3197790852 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-08-19T01:38:27.126 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:38:27.122+0000 7efe21400640 1 --2- 172.21.15.83:0/3197790852 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a85b9c6030 0x55a85b9e9c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:38:27.126 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:38:27.123+0000 7efe20a00640 1 -- 172.21.15.83:0/3197790852 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a85b9c6030 msgr2=0x55a85b9e9c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:38:27.126 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:38:27.123+0000 7efe20a00640 1 --2- 172.21.15.83:0/3197790852 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a85b9c6030 0x55a85b9e9c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:38:27.126 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:38:27.123+0000 7efe20a00640 1 -- 172.21.15.83:0/3197790852 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a85b9c2800 msgr2=0x55a85b9e8c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:38:27.126 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:38:27.123+0000 7efe20a00640 1 --2- 172.21.15.83:0/3197790852 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a85b9c2800 0x55a85b9e8c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:38:27.126 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:38:27.123+0000 7efe20a00640 1 -- 172.21.15.83:0/3197790852 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a85b9ea230 con 0x55a85b9c0fa0 2024-08-19T01:38:27.126 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:38:27.123+0000 7efe2623e740 10 client.0 _ll_register_callbacks cb 0x7efe0c00a5c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-19T01:38:27.126 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:38:27.123+0000 7efe2623e740 1 -- 172.21.15.83:0/3197790852 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efe040b4170 con 0x55a85b9c0fa0 2024-08-19T01:38:27.126 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:38:27.123+0000 7efe13e00640 1 --2- 172.21.15.83:0/3197790852 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a85b9c2800 0x55a85b9e8c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-19T01:38:27.126 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:38:27.124+0000 7efe20a00640 1 --2- 172.21.15.83:0/3197790852 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a85b9c0fa0 0x55a85b9e86d0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7efe0c01dee0 tx=0x7efe0c00e7a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e98ec01a0f949739 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:38:27.147 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:38:27.146+0000 7efe0a000640 10 client.25597 ms_handle_connect on v2:172.21.15.167:3300/0 2024-08-19T01:38:27.148 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:38:27.146+0000 7efe0a000640 1 -- 172.21.15.83:0/3197790852 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe0c021070 con 0x55a85b9c0fa0 2024-08-19T01:38:27.148 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:38:27.146+0000 7efe0a000640 1 -- 172.21.15.83:0/3197790852 <== mon.1 v2:172.21.15.167:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8269+0+0 (secure 0 0 0) 0x7efe0c00f650 con 0x55a85b9c0fa0 2024-08-19T01:38:27.148 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:38:27.146+0000 7efe0a000640 1 -- 172.21.15.83:0/3197790852 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7efe0c00e810 con 0x55a85b9c0fa0 2024-08-19T01:38:27.148 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:38:27.146+0000 7efe2623e740 10 client.25597 fetch_fsmap learned FSMap version 26 2024-08-19T01:38:27.148 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:38:27.146+0000 7efe2623e740 1 -- 172.21.15.83:0/3197790852 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({fsmap=26}) -- 0x55a85ba436e0 con 0x55a85b9c0fa0 2024-08-19T01:38:27.148 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:38:27.147+0000 7efe0a000640 1 -- 172.21.15.83:0/3197790852 <== mon.1 v2:172.21.15.167:3300/0 4 ==== fsmap(e 26) ==== 8350+0+0 (secure 0 0 0) 0x7efe0c00e9b0 con 0x55a85b9c0fa0 2024-08-19T01:38:27.149 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:38:27.147+0000 7efe2623e740 10 client.25597 fetch_fsmap finished waiting for FSMap version 26 2024-08-19T01:38:27.149 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:38:27.147+0000 7efe2623e740 10 client.25597 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-08-19T01:38:27.149 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:38:27.147+0000 7efe2623e740 20 client.25597 populate_metadata read hostname 'smithi083' 2024-08-19T01:38:27.149 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:38:27.147+0000 7efe2623e740 1 --2- 172.21.15.83:0/3197790852 >> [v2:172.21.15.117:6838/1967595704,v1:172.21.15.117:6839/1967595704] conn(0x55a85ba43f90 0x55a85ba46430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:38:27.149 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:38:27.147+0000 7efe2623e740 4 client.25597 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-08-19T01:38:27.149 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:38:27.147+0000 7efe2623e740 1 -- 172.21.15.83:0/3197790852 --> [v2:172.21.15.117:6838/1967595704,v1:172.21.15.117:6839/1967595704] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55a85b92c310 con 0x55a85ba43f90 2024-08-19T01:38:27.149 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:38:27.147+0000 7efe13e00640 1 --2- 172.21.15.83:0/3197790852 >> [v2:172.21.15.117:6838/1967595704,v1:172.21.15.117:6839/1967595704] conn(0x55a85ba43f90 0x55a85ba46430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:38:27.149 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:38:27.148+0000 7efe13e00640 1 --2- 172.21.15.83:0/3197790852 >> [v2:172.21.15.117:6838/1967595704,v1:172.21.15.117:6839/1967595704] conn(0x55a85ba43f90 0x55a85ba46430 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=464db47615cfdad2 server_cookie=eabd815d53fdbecd in_seq=0 out_seq=0 2024-08-19T01:38:27.149 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:38:27.148+0000 7efe0a000640 10 client.25597 ms_handle_connect on v2:172.21.15.117:6838/1967595704 2024-08-19T01:38:27.176 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/bad8780e-5dc9-11ef-bccf-c7b262605968/mon.a/config 2024-08-19T01:38:27.906 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:38:27 smithi083 ceph-mon[17120]: pgmap v447: 97 pgs: 97 active+clean; 305 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 7.5 MiB/s wr, 609 op/s 2024-08-19T01:38:27.952 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:38:27 smithi117 ceph-mon[22016]: pgmap v447: 97 pgs: 97 active+clean; 305 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 7.5 MiB/s wr, 609 op/s 2024-08-19T01:38:27.965 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:38:27 smithi167 ceph-mon[21875]: pgmap v447: 97 pgs: 97 active+clean; 305 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 7.5 MiB/s wr, 609 op/s 2024-08-19T01:38:28.817 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:38:28.815+0000 7efe0a000640 1 -- 172.21.15.83:0/3197790852 <== mds.0 v2:172.21.15.117:6838/1967595704 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55a85b92c310 con 0x55a85ba43f90 2024-08-19T01:38:28.817 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:38:28.815+0000 7efe0a000640 10 client.25597 handle_command_reply: tid=0 2024-08-19T01:38:28.819 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:38:28.818+0000 7efe2623e740 1 client.25597 shutdown 2024-08-19T01:38:28.819 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:38:28.818+0000 7efe2623e740 1 -- 172.21.15.83:0/3197790852 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a85b9c0fa0 msgr2=0x55a85b9e86d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:38:28.819 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:38:28.818+0000 7efe2623e740 1 --2- 172.21.15.83:0/3197790852 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a85b9c0fa0 0x55a85b9e86d0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7efe0c01dee0 tx=0x7efe0c00e7a0 comp rx=0 tx=0).stop 2024-08-19T01:38:28.820 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:38:28.818+0000 7efe2623e740 1 -- 172.21.15.83:0/3197790852 shutdown_connections 2024-08-19T01:38:28.820 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:38:28.818+0000 7efe2623e740 1 --2- 172.21.15.83:0/3197790852 >> [v2:172.21.15.117:6838/1967595704,v1:172.21.15.117:6839/1967595704] conn(0x55a85ba43f90 0x55a85ba46430 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:38:28.820 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:38:28.818+0000 7efe2623e740 1 --2- 172.21.15.83:0/3197790852 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a85b9c6030 0x55a85b9e9c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:38:28.820 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:38:28.818+0000 7efe2623e740 1 --2- 172.21.15.83:0/3197790852 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a85b9c2800 0x55a85b9e8c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:38:28.820 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:38:28.818+0000 7efe2623e740 1 --2- 172.21.15.83:0/3197790852 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a85b9c0fa0 0x55a85b9e86d0 secure :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7efe0c01dee0 tx=0x7efe0c00e7a0 comp rx=0 tx=0).stop 2024-08-19T01:38:28.820 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:38:28.818+0000 7efe2623e740 1 -- 172.21.15.83:0/3197790852 >> 172.21.15.83:0/3197790852 conn(0x55a85b9bbcd0 msgr2=0x55a85b9bcc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:38:28.820 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:38:28.818+0000 7efe0a000640 0 client.25597 ms_handle_reset on v2:172.21.15.117:6838/1967595704 2024-08-19T01:38:28.821 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stdout:{ 2024-08-19T01:38:28.821 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stdout: "message": "", 2024-08-19T01:38:28.821 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stdout: "return_code": 0 2024-08-19T01:38:28.821 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stdout:} 2024-08-19T01:38:28.821 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:38:28.819+0000 7efe2623e740 1 -- 172.21.15.83:0/3197790852 shutdown_connections 2024-08-19T01:38:28.822 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:38:28.819+0000 7efe2623e740 1 -- 172.21.15.83:0/3197790852 wait complete. 2024-08-19T01:38:28.822 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:38:28.819+0000 7efe2623e740 20 client.25597 trim_cache size 0 max 16384 2024-08-19T01:38:28.822 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:38:28.820+0000 7efe21e00640 1 -- 172.21.15.83:0/2573168382 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7efe000a2d10 msgr2=0x7efe000a51d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:38:28.822 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:38:28.820+0000 7efe21e00640 1 --2- 172.21.15.83:0/2573168382 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7efe000a2d10 0x7efe000a51d0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7efe1c072fb0 tx=0x7efe0c005c20 comp rx=0 tx=0).stop 2024-08-19T01:38:28.823 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:38:28.820+0000 7efe21e00640 1 -- 172.21.15.83:0/2573168382 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7efe1c1092c0 msgr2=0x7efe1c119750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:38:28.823 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:38:28.820+0000 7efe21e00640 1 --2- 172.21.15.83:0/2573168382 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7efe1c1092c0 0x7efe1c119750 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7efe0400d9d0 tx=0x7efe0400de90 comp rx=0 tx=0).stop 2024-08-19T01:38:28.823 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:38:28.821+0000 7efe21e00640 1 -- 172.21.15.83:0/2573168382 shutdown_connections 2024-08-19T01:38:28.823 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:38:28.821+0000 7efe21e00640 1 --2- 172.21.15.83:0/2573168382 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7efe000a2d10 0x7efe000a51d0 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:38:28.823 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:38:28.821+0000 7efe21e00640 1 --2- 172.21.15.83:0/2573168382 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7efe1c109c80 0x7efe1c1207c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:38:28.823 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:38:28.821+0000 7efe21e00640 1 --2- 172.21.15.83:0/2573168382 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7efe1c1092c0 0x7efe1c119750 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:38:28.824 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:38:28.821+0000 7efe21e00640 1 --2- 172.21.15.83:0/2573168382 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7efe1c073290 0x7efe1c119210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:38:28.824 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:38:28.821+0000 7efe21e00640 1 -- 172.21.15.83:0/2573168382 >> 172.21.15.83:0/2573168382 conn(0x7efe1c06ea40 msgr2=0x7efe1c10c5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:38:28.824 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:38:28.821+0000 7efe21e00640 1 -- 172.21.15.83:0/2573168382 shutdown_connections 2024-08-19T01:38:28.824 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:38:28.822+0000 7efe21e00640 1 -- 172.21.15.83:0/2573168382 wait complete. 2024-08-19T01:38:29.332 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:29.329+0000 7f5269400640 1 -- 172.21.15.83:0/1356944759 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f52641031e0 msgr2=0x7f526410b6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:38:29.332 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:29.329+0000 7f5269400640 1 --2- 172.21.15.83:0/1356944759 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f52641031e0 0x7f526410b6d0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f5250009900 tx=0x7f525002a4d0 comp rx=0 tx=0).stop 2024-08-19T01:38:29.333 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:29.331+0000 7f5269400640 1 -- 172.21.15.83:0/1356944759 shutdown_connections 2024-08-19T01:38:29.333 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:29.331+0000 7f5269400640 1 --2- 172.21.15.83:0/1356944759 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f526410bcf0 0x7f526410e190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:38:29.333 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:29.331+0000 7f5269400640 1 --2- 172.21.15.83:0/1356944759 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f52641031e0 0x7f526410b6d0 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:38:29.333 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:29.331+0000 7f5269400640 1 --2- 172.21.15.83:0/1356944759 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f52641028c0 0x7f5264102ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:38:29.333 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:29.331+0000 7f5269400640 1 -- 172.21.15.83:0/1356944759 >> 172.21.15.83:0/1356944759 conn(0x7f52640fc340 msgr2=0x7f52640fe760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:38:29.333 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:29.331+0000 7f5269400640 1 -- 172.21.15.83:0/1356944759 shutdown_connections 2024-08-19T01:38:29.333 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:29.331+0000 7f5269400640 1 -- 172.21.15.83:0/1356944759 wait complete. 2024-08-19T01:38:29.336 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:29.333+0000 7f5269400640 1 Processor -- start 2024-08-19T01:38:29.336 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:29.334+0000 7f5269400640 1 -- start start 2024-08-19T01:38:29.336 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:29.334+0000 7f5269400640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f52641028c0 0x7f526419a1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:38:29.336 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:29.334+0000 7f5269400640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f52641031e0 0x7f526419a720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:38:29.337 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:29.334+0000 7f5269400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f526410bcf0 0x7f526419eab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:38:29.337 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:29.334+0000 7f5269400640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f5264111b40 con 0x7f526410bcf0 2024-08-19T01:38:29.337 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:29.334+0000 7f5269400640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f52641119c0 con 0x7f52641028c0 2024-08-19T01:38:29.337 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:29.334+0000 7f5269400640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f5264111cc0 con 0x7f52641031e0 2024-08-19T01:38:29.338 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:29.335+0000 7f5263400640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f52641031e0 0x7f526419a720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:38:29.338 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:29.335+0000 7f5263400640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f52641031e0 0x7f526419a720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.83:46156/0 (socket says 172.21.15.83:46156) 2024-08-19T01:38:29.338 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:29.335+0000 7f5263400640 1 -- 172.21.15.83:0/577680609 learned_addr learned my addr 172.21.15.83:0/577680609 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-08-19T01:38:29.338 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:29.336+0000 7f5268a00640 1 --2- 172.21.15.83:0/577680609 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f526410bcf0 0x7f526419eab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:38:29.339 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:29.337+0000 7f5263e00640 1 --2- 172.21.15.83:0/577680609 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f52641028c0 0x7f526419a1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:38:29.339 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:29.337+0000 7f5263400640 1 -- 172.21.15.83:0/577680609 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f52641028c0 msgr2=0x7f526419a1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:38:29.339 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:29.337+0000 7f5263400640 1 --2- 172.21.15.83:0/577680609 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f52641028c0 0x7f526419a1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:38:29.343 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:29.341+0000 7f5263400640 1 -- 172.21.15.83:0/577680609 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f526410bcf0 msgr2=0x7f526419eab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:38:29.343 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:29.341+0000 7f5263400640 1 --2- 172.21.15.83:0/577680609 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f526410bcf0 0x7f526419eab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:38:29.343 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:29.341+0000 7f5263400640 1 -- 172.21.15.83:0/577680609 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f526419f190 con 0x7f52641031e0 2024-08-19T01:38:29.343 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:29.341+0000 7f5268a00640 1 --2- 172.21.15.83:0/577680609 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f526410bcf0 0x7f526419eab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-19T01:38:29.343 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:29.341+0000 7f5263e00640 1 --2- 172.21.15.83:0/577680609 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f52641028c0 0x7f526419a1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-19T01:38:29.344 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:29.342+0000 7f5263400640 1 --2- 172.21.15.83:0/577680609 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f52641031e0 0x7f526419a720 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f5250005ab0 tx=0x7f52500052b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:38:29.344 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:29.342+0000 7f5260c00640 1 -- 172.21.15.83:0/577680609 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5250040030 con 0x7f52641031e0 2024-08-19T01:38:29.346 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:29.343+0000 7f5260c00640 1 -- 172.21.15.83:0/577680609 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f525003c920 con 0x7f52641031e0 2024-08-19T01:38:29.346 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:29.343+0000 7f5260c00640 1 -- 172.21.15.83:0/577680609 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52500318d0 con 0x7f52641031e0 2024-08-19T01:38:29.346 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:29.343+0000 7f5269400640 1 -- 172.21.15.83:0/577680609 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f526419f420 con 0x7f52641031e0 2024-08-19T01:38:29.346 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:29.343+0000 7f5269400640 1 -- 172.21.15.83:0/577680609 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f526419f8b0 con 0x7f52641031e0 2024-08-19T01:38:29.347 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:29.346+0000 7f5269400640 1 -- 172.21.15.83:0/577680609 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5264102ca0 con 0x7f52641031e0 2024-08-19T01:38:29.353 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:29.348+0000 7f5260c00640 1 -- 172.21.15.83:0/577680609 <== mon.1 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7f5250043030 con 0x7f52641031e0 2024-08-19T01:38:29.354 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:29.353+0000 7f5260c00640 1 --2- 172.21.15.83:0/577680609 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f52480a2c00 0x7f52480a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:38:29.355 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:29.353+0000 7f5260c00640 1 -- 172.21.15.83:0/577680609 <== mon.1 v2:172.21.15.167:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8269+0+0 (secure 0 0 0) 0x7f52500ef680 con 0x7f52641031e0 2024-08-19T01:38:29.358 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:29.356+0000 7f5260c00640 1 -- 172.21.15.83:0/577680609 <== mon.1 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7f52500bc5c0 con 0x7f52641031e0 2024-08-19T01:38:29.360 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:29.357+0000 7f5263e00640 1 --2- 172.21.15.83:0/577680609 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f52480a2c00 0x7f52480a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:38:29.360 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:29.358+0000 7f5263e00640 1 --2- 172.21.15.83:0/577680609 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f52480a2c00 0x7f52480a50c0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f5254009b40 tx=0x7f5254005e50 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:38:29.851 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:38:29 smithi083 ceph-mon[17120]: pgmap v448: 97 pgs: 97 active+clean; 322 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 9.2 MiB/s wr, 863 op/s 2024-08-19T01:38:29.851 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-08-19T01:38:29.851 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":26,"btime":"2024-08-19T01:36:40:482111+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14421,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6835/988556067","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":988556067},{"type":"v1","addr":"172.21.15.83:6835","nonce":988556067}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14433,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6837/869702632","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6836","nonce":869702632},{"type":"v1","addr":"172.21.15.83:6837","nonce":869702632}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14475,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6841/1795975999","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6840","nonce":1795975999},{"type":"v1","addr":"172.21.15.83:6841","nonce":1795975999}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24430,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6833/2505076046","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6832","nonce":2505076046},{"type":"v1","addr":"172.21.15.167:6833","nonce":2505076046}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24449,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6833/3557520478","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":3557520478},{"type":"v1","addr":"172.21.15.117:6833","nonce":3557520478}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24466,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6835/3482045966","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":3482045966},{"type":"v1","addr":"172.21.15.167:6835","nonce":3482045966}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7}],"filesystems":[{"mdsmap":{"epoch":26,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-08-19T01:32:42.054769+0000","modified":"2024-08-19T01:36:40.093322+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24545,"mds_1":24527,"mds_2":24509},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24478":{"gid":24478,"name":"c","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.167:6837/1762130408","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6836","nonce":1762130408},{"type":"v1","addr":"172.21.15.167:6837","nonce":1762130408}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24485":{"gid":24485,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.117:6835/1024213994","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":1024213994},{"type":"v1","addr":"172.21.15.117:6835","nonce":1024213994}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24496":{"gid":24496,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.167:6839/3507387682","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6838","nonce":3507387682},{"type":"v1","addr":"172.21.15.167:6839","nonce":3507387682}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24509":{"gid":24509,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":10,"addr":"172.21.15.117:6837/3185224684","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":3185224684},{"type":"v1","addr":"172.21.15.117:6837","nonce":3185224684}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24527":{"gid":24527,"name":"g","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.83:6839/3239261845","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6838","nonce":3239261845},{"type":"v1","addr":"172.21.15.83:6839","nonce":3239261845}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24545":{"gid":24545,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.117:6839/1967595704","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6838","nonce":1967595704},{"type":"v1","addr":"172.21.15.117:6839","nonce":1967595704}]},"join_fscid":-1,"export_targets":[2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24545,"qdb_cluster":[24509,24527,24545]},"id":1}]} 2024-08-19T01:38:29.852 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:29.837+0000 7f5269400640 1 -- 172.21.15.83:0/577680609 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f52641a6f20 con 0x7f52641031e0 2024-08-19T01:38:29.852 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:29.838+0000 7f5260c00640 1 -- 172.21.15.83:0/577680609 <== mon.1 v2:172.21.15.167:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+11921 (secure 0 0 0) 0x7f52500bbec0 con 0x7f52641031e0 2024-08-19T01:38:29.852 INFO:teuthology.orchestra.run.smithi083.stderr:dumped fsmap epoch 26 2024-08-19T01:38:29.852 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:29.843+0000 7f5269400640 1 -- 172.21.15.83:0/577680609 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f52480a2c00 msgr2=0x7f52480a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:38:29.852 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:29.843+0000 7f5269400640 1 --2- 172.21.15.83:0/577680609 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f52480a2c00 0x7f52480a50c0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f5254009b40 tx=0x7f5254005e50 comp rx=0 tx=0).stop 2024-08-19T01:38:29.852 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:29.843+0000 7f5269400640 1 -- 172.21.15.83:0/577680609 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f52641031e0 msgr2=0x7f526419a720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:38:29.852 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:29.843+0000 7f5269400640 1 --2- 172.21.15.83:0/577680609 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f52641031e0 0x7f526419a720 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f5250005ab0 tx=0x7f52500052b0 comp rx=0 tx=0).stop 2024-08-19T01:38:29.852 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:29.843+0000 7f5269400640 1 -- 172.21.15.83:0/577680609 shutdown_connections 2024-08-19T01:38:29.852 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:29.843+0000 7f5269400640 1 --2- 172.21.15.83:0/577680609 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f52480a2c00 0x7f52480a50c0 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:38:29.852 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:29.843+0000 7f5269400640 1 --2- 172.21.15.83:0/577680609 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f526410bcf0 0x7f526419eab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:38:29.853 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:29.843+0000 7f5269400640 1 --2- 172.21.15.83:0/577680609 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f52641031e0 0x7f526419a720 secure :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f5250005ab0 tx=0x7f52500052b0 comp rx=0 tx=0).stop 2024-08-19T01:38:29.853 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:29.843+0000 7f5269400640 1 --2- 172.21.15.83:0/577680609 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f52641028c0 0x7f526419a1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:38:29.853 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:29.843+0000 7f5269400640 1 -- 172.21.15.83:0/577680609 >> 172.21.15.83:0/577680609 conn(0x7f52640fc340 msgr2=0x7f526410d010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:38:29.853 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:29.844+0000 7f5269400640 1 -- 172.21.15.83:0/577680609 shutdown_connections 2024-08-19T01:38:29.853 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:29.845+0000 7f5269400640 1 -- 172.21.15.83:0/577680609 wait complete. 2024-08-19T01:38:29.952 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:38:29 smithi117 ceph-mon[22016]: pgmap v448: 97 pgs: 97 active+clean; 322 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 9.2 MiB/s wr, 863 op/s 2024-08-19T01:38:29.965 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:38:29 smithi167 ceph-mon[21875]: pgmap v448: 97 pgs: 97 active+clean; 322 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 9.2 MiB/s wr, 863 op/s 2024-08-19T01:38:30.612 INFO:tasks.workunit.client.0.smithi083.stdout: 103 570945 271 370215 313 321382 749 2024-08-19T01:38:30.904 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:38:30 smithi083 ceph-mon[17120]: from='client.? 172.21.15.83:0/577680609' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-19T01:38:30.952 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:38:30 smithi117 ceph-mon[22016]: from='client.? 172.21.15.83:0/577680609' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-19T01:38:30.965 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:38:30 smithi167 ceph-mon[21875]: from='client.? 172.21.15.83:0/577680609' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-19T01:38:31.808 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:38:31 smithi083 ceph-mon[17120]: pgmap v449: 97 pgs: 97 active+clean; 318 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 6.9 MiB/s wr, 946 op/s 2024-08-19T01:38:31.952 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:38:31 smithi117 ceph-mon[22016]: pgmap v449: 97 pgs: 97 active+clean; 318 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 6.9 MiB/s wr, 946 op/s 2024-08-19T01:38:31.966 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:38:31 smithi167 ceph-mon[21875]: pgmap v449: 97 pgs: 97 active+clean; 318 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 6.9 MiB/s wr, 946 op/s 2024-08-19T01:38:32.143 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833 shell --fsid bad8780e-5dc9-11ef-bccf-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-08-19T01:38:33.904 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:38:33 smithi083 ceph-mon[17120]: pgmap v450: 97 pgs: 97 active+clean; 318 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 5.8 MiB/s wr, 922 op/s 2024-08-19T01:38:33.952 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:38:33 smithi117 ceph-mon[22016]: pgmap v450: 97 pgs: 97 active+clean; 318 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 5.8 MiB/s wr, 922 op/s 2024-08-19T01:38:33.966 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:38:33 smithi167 ceph-mon[21875]: pgmap v450: 97 pgs: 97 active+clean; 318 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 5.8 MiB/s wr, 922 op/s 2024-08-19T01:38:35.655 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:38:35 smithi083 ceph-mon[17120]: pgmap v451: 97 pgs: 97 active+clean; 291 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 5.9 MiB/s wr, 1.04k op/s 2024-08-19T01:38:35.952 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:38:35 smithi117 ceph-mon[22016]: pgmap v451: 97 pgs: 97 active+clean; 291 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 5.9 MiB/s wr, 1.04k op/s 2024-08-19T01:38:35.966 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:38:35 smithi167 ceph-mon[21875]: pgmap v451: 97 pgs: 97 active+clean; 291 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 5.9 MiB/s wr, 1.04k op/s 2024-08-19T01:38:37.312 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/bad8780e-5dc9-11ef-bccf-c7b262605968/mon.a/config 2024-08-19T01:38:37.905 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:38:37 smithi083 ceph-mon[17120]: pgmap v452: 97 pgs: 97 active+clean; 298 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 4.2 MiB/s wr, 886 op/s 2024-08-19T01:38:37.952 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:38:37 smithi117 ceph-mon[22016]: pgmap v452: 97 pgs: 97 active+clean; 298 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 4.2 MiB/s wr, 886 op/s 2024-08-19T01:38:37.965 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:38:37 smithi167 ceph-mon[21875]: pgmap v452: 97 pgs: 97 active+clean; 298 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 4.2 MiB/s wr, 886 op/s 2024-08-19T01:38:38.634 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:38.620+0000 7fd983200640 1 -- 172.21.15.83:0/1144227339 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd984100fc0 msgr2=0x7fd98410d4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:38:38.634 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:38.620+0000 7fd983200640 1 --2- 172.21.15.83:0/1144227339 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd984100fc0 0x7fd98410d4d0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fd97c009c10 tx=0x7fd97c02a720 comp rx=0 tx=0).stop 2024-08-19T01:38:38.634 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:38.622+0000 7fd983200640 1 -- 172.21.15.83:0/1144227339 shutdown_connections 2024-08-19T01:38:38.634 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:38.622+0000 7fd983200640 1 --2- 172.21.15.83:0/1144227339 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd984100fc0 0x7fd98410d4d0 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:38:38.634 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:38.622+0000 7fd983200640 1 --2- 172.21.15.83:0/1144227339 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd984100600 0x7fd984100a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:38:38.634 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:38.623+0000 7fd983200640 1 --2- 172.21.15.83:0/1144227339 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd984106690 0x7fd984106a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:38:38.635 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:38.623+0000 7fd983200640 1 -- 172.21.15.83:0/1144227339 >> 172.21.15.83:0/1144227339 conn(0x7fd9840fc340 msgr2=0x7fd9840fe760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:38:38.635 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:38.623+0000 7fd983200640 1 -- 172.21.15.83:0/1144227339 shutdown_connections 2024-08-19T01:38:38.635 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:38.624+0000 7fd983200640 1 -- 172.21.15.83:0/1144227339 wait complete. 2024-08-19T01:38:38.635 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:38.626+0000 7fd983200640 1 Processor -- start 2024-08-19T01:38:38.635 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:38.626+0000 7fd983200640 1 -- start start 2024-08-19T01:38:38.635 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:38.627+0000 7fd983200640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd984100600 0x7fd98419e460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:38:38.635 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:38.628+0000 7fd983200640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd984100fc0 0x7fd98419e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:38:38.635 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:38.628+0000 7fd983200640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd984106690 0x7fd9841a2d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:38:38.635 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:38.628+0000 7fd981e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd984100600 0x7fd98419e460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:38:38.635 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:38.628+0000 7fd981e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd984100600 0x7fd98419e460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:46670/0 (socket says 172.21.15.83:46670) 2024-08-19T01:38:38.635 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:38.629+0000 7fd981e00640 1 -- 172.21.15.83:0/1370631152 learned_addr learned my addr 172.21.15.83:0/1370631152 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-08-19T01:38:38.635 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:38.629+0000 7fd981400640 1 --2- 172.21.15.83:0/1370631152 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd984100fc0 0x7fd98419e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:38:38.636 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:38.630+0000 7fd983200640 1 -- 172.21.15.83:0/1370631152 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fd98410f930 con 0x7fd984100600 2024-08-19T01:38:38.636 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:38.630+0000 7fd983200640 1 -- 172.21.15.83:0/1370631152 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fd98410f7b0 con 0x7fd984100fc0 2024-08-19T01:38:38.636 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:38.630+0000 7fd982800640 1 --2- 172.21.15.83:0/1370631152 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd984106690 0x7fd9841a2d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:38:38.636 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:38.630+0000 7fd983200640 1 -- 172.21.15.83:0/1370631152 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fd98410fab0 con 0x7fd984106690 2024-08-19T01:38:38.636 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:38.631+0000 7fd982800640 1 -- 172.21.15.83:0/1370631152 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd984100fc0 msgr2=0x7fd98419e9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:38:38.636 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:38.631+0000 7fd982800640 1 --2- 172.21.15.83:0/1370631152 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd984100fc0 0x7fd98419e9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:38:38.636 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:38.631+0000 7fd982800640 1 -- 172.21.15.83:0/1370631152 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd984100600 msgr2=0x7fd98419e460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:38:38.636 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:38.631+0000 7fd982800640 1 --2- 172.21.15.83:0/1370631152 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd984100600 0x7fd98419e460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:38:38.636 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:38.631+0000 7fd982800640 1 -- 172.21.15.83:0/1370631152 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd9841a3410 con 0x7fd984106690 2024-08-19T01:38:38.636 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:38.631+0000 7fd981e00640 1 --2- 172.21.15.83:0/1370631152 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd984100600 0x7fd98419e460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-19T01:38:38.636 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:38.631+0000 7fd981400640 1 --2- 172.21.15.83:0/1370631152 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd984100fc0 0x7fd98419e9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-19T01:38:38.637 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:38.632+0000 7fd982800640 1 --2- 172.21.15.83:0/1370631152 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd984106690 0x7fd9841a2d30 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fd97c009b30 tx=0x7fd97c030890 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:38:38.637 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:38.635+0000 7fd97aa00640 1 -- 172.21.15.83:0/1370631152 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd97c041020 con 0x7fd984106690 2024-08-19T01:38:38.637 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:38.635+0000 7fd97aa00640 1 -- 172.21.15.83:0/1370631152 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd97c03cb50 con 0x7fd984106690 2024-08-19T01:38:38.637 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:38.635+0000 7fd97aa00640 1 -- 172.21.15.83:0/1370631152 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd97c036040 con 0x7fd984106690 2024-08-19T01:38:38.638 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:38.635+0000 7fd983200640 1 -- 172.21.15.83:0/1370631152 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd9841a36a0 con 0x7fd984106690 2024-08-19T01:38:38.638 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:38.636+0000 7fd983200640 1 -- 172.21.15.83:0/1370631152 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd9841a3b00 con 0x7fd984106690 2024-08-19T01:38:38.680 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:38.657+0000 7fd97aa00640 1 -- 172.21.15.83:0/1370631152 <== mon.1 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7fd97c044020 con 0x7fd984106690 2024-08-19T01:38:38.681 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:38.659+0000 7fd98b9f4740 1 Processor -- start 2024-08-19T01:38:38.681 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:38.666+0000 7fd97aa00640 1 --2- 172.21.15.83:0/1370631152 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7fd9680a2d90 0x7fd9680a5250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:38:38.681 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:38.666+0000 7fd97aa00640 1 -- 172.21.15.83:0/1370631152 <== mon.1 v2:172.21.15.167:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8269+0+0 (secure 0 0 0) 0x7fd97c0efff0 con 0x7fd984106690 2024-08-19T01:38:38.681 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:38.667+0000 7fd981400640 1 --2- 172.21.15.83:0/1370631152 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7fd9680a2d90 0x7fd9680a5250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:38:38.681 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:38.664+0000 7fd98b9f4740 1 -- start start 2024-08-19T01:38:38.681 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:38.670+0000 7fd98b9f4740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5588e9de56e0 0x5588e9de5ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:38:38.681 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:38.670+0000 7fd98b9f4740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5588e9de6000 0x5588e9df0380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:38:38.681 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:38.670+0000 7fd98b9f4740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5588e9e018d0 0x5588e9e03cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:38:38.681 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:38.670+0000 7fd98b9f4740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x5588e9d4abf0 con 0x5588e9e018d0 2024-08-19T01:38:38.681 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:38.670+0000 7fd98b9f4740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x5588e9ce2b20 con 0x5588e9de6000 2024-08-19T01:38:38.681 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:38.670+0000 7fd98b9f4740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x5588e9d46bb0 con 0x5588e9de56e0 2024-08-19T01:38:38.682 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:38.671+0000 7fd982800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5588e9e018d0 0x5588e9e03cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:38:38.682 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:38.671+0000 7fd981e00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5588e9de56e0 0x5588e9de5ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:38:38.682 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:38.671+0000 7fd982800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5588e9e018d0 0x5588e9e03cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:46682/0 (socket says 172.21.15.83:46682) 2024-08-19T01:38:38.682 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:38.671+0000 7fd982800640 1 -- 172.21.15.83:0/2363148032 learned_addr learned my addr 172.21.15.83:0/2363148032 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-08-19T01:38:38.682 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:38.672+0000 7fd982800640 1 -- 172.21.15.83:0/2363148032 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5588e9de56e0 msgr2=0x5588e9de5ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:38:38.682 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:38.672+0000 7fd982800640 1 --2- 172.21.15.83:0/2363148032 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5588e9de56e0 0x5588e9de5ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:38:38.682 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:38.672+0000 7fd982800640 1 -- 172.21.15.83:0/2363148032 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5588e9de6000 msgr2=0x5588e9df0380 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-08-19T01:38:38.682 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:38.672+0000 7fd982800640 1 --2- 172.21.15.83:0/2363148032 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5588e9de6000 0x5588e9df0380 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:38:38.682 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:38.672+0000 7fd982800640 1 -- 172.21.15.83:0/2363148032 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5588e9ce97f0 con 0x5588e9e018d0 2024-08-19T01:38:38.682 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:38.672+0000 7fd982800640 1 --2- 172.21.15.83:0/2363148032 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5588e9e018d0 0x5588e9e03cc0 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7fd97c0bd540 tx=0x7fd97c0098e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=87362d7f4ef379f2 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:38:38.682 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:38.674+0000 7fd981400640 1 --2- 172.21.15.83:0/1370631152 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7fd9680a2d90 0x7fd9680a5250 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fd984105b60 tx=0x7fd970005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:38:38.682 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:38.675+0000 7fd96f400640 1 -- 172.21.15.83:0/2363148032 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd97c0bbb40 con 0x5588e9e018d0 2024-08-19T01:38:38.683 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:38.675+0000 7fd96f400640 1 -- 172.21.15.83:0/2363148032 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd97c0edb80 con 0x5588e9e018d0 2024-08-19T01:38:38.683 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:38.676+0000 7fd98b9f4740 1 -- 172.21.15.83:0/2363148032 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5588e9e018d0 msgr2=0x5588e9e03cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:38:38.683 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:38.676+0000 7fd98b9f4740 1 --2- 172.21.15.83:0/2363148032 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5588e9e018d0 0x5588e9e03cc0 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7fd97c0bd540 tx=0x7fd97c0098e0 comp rx=0 tx=0).stop 2024-08-19T01:38:38.683 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:38.678+0000 7fd98b9f4740 1 -- 172.21.15.83:0/2363148032 shutdown_connections 2024-08-19T01:38:38.683 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:38.678+0000 7fd98b9f4740 1 --2- 172.21.15.83:0/2363148032 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5588e9e018d0 0x5588e9e03cc0 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:38:38.683 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:38.678+0000 7fd98b9f4740 1 --2- 172.21.15.83:0/2363148032 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5588e9de6000 0x5588e9df0380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:38:38.683 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:38.679+0000 7fd98b9f4740 1 --2- 172.21.15.83:0/2363148032 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5588e9de56e0 0x5588e9de5ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:38:38.683 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:38.679+0000 7fd98b9f4740 1 -- 172.21.15.83:0/2363148032 >> 172.21.15.83:0/2363148032 conn(0x5588e9de1c70 msgr2=0x5588e9de2050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:38:38.683 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:38.680+0000 7fd98b9f4740 1 -- 172.21.15.83:0/2363148032 shutdown_connections 2024-08-19T01:38:38.684 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:38.680+0000 7fd98b9f4740 1 -- 172.21.15.83:0/2363148032 wait complete. 2024-08-19T01:38:38.685 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:38.683+0000 7fd98b9f4740 1 -- start start 2024-08-19T01:38:38.688 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:38.684+0000 7fd98b9f4740 1 Processor -- start 2024-08-19T01:38:38.688 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:38.685+0000 7fd98b9f4740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5588e9de56e0 0x5588e9ded7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:38:38.688 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:38.685+0000 7fd98b9f4740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5588e9de6000 0x5588e9dedce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:38:38.688 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:38.685+0000 7fd98b9f4740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5588e9e018d0 0x5588e9dee220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:38:38.688 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:38.685+0000 7fd98b9f4740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x5588e9e04260 con 0x5588e9de6000 2024-08-19T01:38:38.688 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:38.685+0000 7fd98b9f4740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x5588e9d05cf0 con 0x5588e9e018d0 2024-08-19T01:38:38.689 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:38.685+0000 7fd98b9f4740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x5588e9e043e0 con 0x5588e9de56e0 2024-08-19T01:38:38.689 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:38.685+0000 7fd98b9f4740 10 client.0 _ll_register_callbacks cb 0x5588e9cf1e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-19T01:38:38.689 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:38.686+0000 7fd981e00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5588e9de56e0 0x5588e9ded7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:38:38.689 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:38.686+0000 7fd981e00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5588e9de56e0 0x5588e9ded7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.83:42280/0 (socket says 172.21.15.83:42280) 2024-08-19T01:38:38.689 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:38.686+0000 7fd981e00640 1 -- 172.21.15.83:0/2742276701 learned_addr learned my addr 172.21.15.83:0/2742276701 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-08-19T01:38:38.689 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:38.686+0000 7fd981e00640 1 -- 172.21.15.83:0/2742276701 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5588e9e018d0 msgr2=0x5588e9dee220 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-08-19T01:38:38.689 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:38.686+0000 7fd981e00640 1 --2- 172.21.15.83:0/2742276701 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5588e9e018d0 0x5588e9dee220 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:38:38.689 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:38.686+0000 7fd981e00640 1 -- 172.21.15.83:0/2742276701 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5588e9de6000 msgr2=0x5588e9dedce0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-08-19T01:38:38.689 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:38.686+0000 7fd981e00640 1 --2- 172.21.15.83:0/2742276701 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5588e9de6000 0x5588e9dedce0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:38:38.689 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:38.686+0000 7fd981e00640 1 -- 172.21.15.83:0/2742276701 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5588e9abc3c0 con 0x5588e9de56e0 2024-08-19T01:38:38.691 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:38.687+0000 7fd981e00640 1 --2- 172.21.15.83:0/2742276701 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5588e9de56e0 0x5588e9ded7a0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fd974012870 tx=0x7fd974012d30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5bb6e3a1995b1781 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:38:38.692 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:38.687+0000 7fd963e00640 10 client.0 ms_handle_connect on v2:172.21.15.167:3300/0 2024-08-19T01:38:38.692 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:38.688+0000 7fd963e00640 1 -- 172.21.15.83:0/2742276701 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd974011bb0 con 0x5588e9de56e0 2024-08-19T01:38:38.692 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:38.688+0000 7fd963e00640 1 -- 172.21.15.83:0/2742276701 <== mon.1 v2:172.21.15.167:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8269+0+0 (secure 0 0 0) 0x7fd974011d50 con 0x5588e9de56e0 2024-08-19T01:38:38.692 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:38.688+0000 7fd98b9f4740 1 -- 172.21.15.83:0/2742276701 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5588e9d58160 con 0x5588e9de56e0 2024-08-19T01:38:38.714 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:38.690+0000 7fd963e00640 1 -- 172.21.15.83:0/2742276701 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fd97400f460 con 0x5588e9de56e0 2024-08-19T01:38:38.714 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:38.691+0000 7fd98b9f4740 10 client.25615 fetch_fsmap learned FSMap version 26 2024-08-19T01:38:38.714 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:38.691+0000 7fd98b9f4740 1 -- 172.21.15.83:0/2742276701 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5588e9e10080 con 0x5588e9de56e0 2024-08-19T01:38:38.714 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:38.691+0000 7fd963e00640 1 -- 172.21.15.83:0/2742276701 <== mon.1 v2:172.21.15.167:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd97400fae0 con 0x5588e9de56e0 2024-08-19T01:38:38.714 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:38.693+0000 7fd963e00640 1 -- 172.21.15.83:0/2742276701 <== mon.1 v2:172.21.15.167:3300/0 5 ==== fsmap(e 26) ==== 8350+0+0 (secure 0 0 0) 0x7fd9740268c0 con 0x5588e9de56e0 2024-08-19T01:38:38.714 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:38.693+0000 7fd98b9f4740 10 client.25615 fetch_fsmap finished waiting for FSMap version 26 2024-08-19T01:38:38.714 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:38.693+0000 7fd98b9f4740 10 client.25615 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-08-19T01:38:38.714 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:38.693+0000 7fd98b9f4740 20 client.25615 populate_metadata read hostname 'smithi083' 2024-08-19T01:38:38.715 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:38.693+0000 7fd98b9f4740 1 --2- 172.21.15.83:0/2742276701 >> [v2:172.21.15.117:6836/3185224684,v1:172.21.15.117:6837/3185224684] conn(0x5588e9e11500 0x5588e9e139c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:38:38.715 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:38.693+0000 7fd98b9f4740 4 client.25615 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-19T01:38:38.715 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:38.693+0000 7fd98b9f4740 1 -- 172.21.15.83:0/2742276701 --> [v2:172.21.15.117:6836/3185224684,v1:172.21.15.117:6837/3185224684] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5588e9d58160 con 0x5588e9e11500 2024-08-19T01:38:38.715 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:38.694+0000 7fd981400640 1 --2- 172.21.15.83:0/2742276701 >> [v2:172.21.15.117:6836/3185224684,v1:172.21.15.117:6837/3185224684] conn(0x5588e9e11500 0x5588e9e139c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:38:38.715 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:38.695+0000 7fd963e00640 10 client.25615 ms_handle_connect on v2:172.21.15.117:6836/3185224684 2024-08-19T01:38:38.715 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:38.695+0000 7fd981400640 1 --2- 172.21.15.83:0/2742276701 >> [v2:172.21.15.117:6836/3185224684,v1:172.21.15.117:6837/3185224684] conn(0x5588e9e11500 0x5588e9e139c0 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=542666b5f9aa2f86 server_cookie=a5ecffa03d5588e in_seq=0 out_seq=0 2024-08-19T01:38:38.715 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:38.696+0000 7fd963e00640 1 -- 172.21.15.83:0/2742276701 <== mds.2 v2:172.21.15.117:6836/3185224684 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5588e9d58160 con 0x5588e9e11500 2024-08-19T01:38:38.715 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:38.696+0000 7fd963e00640 10 client.25615 handle_command_reply: tid=0 2024-08-19T01:38:38.715 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:38.696+0000 7fd98b9f4740 1 client.25615 shutdown 2024-08-19T01:38:38.715 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:38.696+0000 7fd98b9f4740 1 -- 172.21.15.83:0/2742276701 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5588e9de56e0 msgr2=0x5588e9ded7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:38:38.715 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:38.696+0000 7fd98b9f4740 1 --2- 172.21.15.83:0/2742276701 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5588e9de56e0 0x5588e9ded7a0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fd974012870 tx=0x7fd974012d30 comp rx=0 tx=0).stop 2024-08-19T01:38:38.715 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:38.697+0000 7fd98b9f4740 1 -- 172.21.15.83:0/2742276701 shutdown_connections 2024-08-19T01:38:38.715 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:38.697+0000 7fd98b9f4740 1 --2- 172.21.15.83:0/2742276701 >> [v2:172.21.15.117:6836/3185224684,v1:172.21.15.117:6837/3185224684] conn(0x5588e9e11500 0x5588e9e139c0 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:38:38.716 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:38.697+0000 7fd963e00640 0 client.25615 ms_handle_reset on v2:172.21.15.117:6836/3185224684 2024-08-19T01:38:38.716 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:38.697+0000 7fd98b9f4740 1 --2- 172.21.15.83:0/2742276701 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5588e9e018d0 0x5588e9dee220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:38:38.716 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:38.697+0000 7fd98b9f4740 1 --2- 172.21.15.83:0/2742276701 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5588e9de6000 0x5588e9dedce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:38:38.716 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:38.697+0000 7fd98b9f4740 1 --2- 172.21.15.83:0/2742276701 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5588e9de56e0 0x5588e9ded7a0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:38:38.716 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:38.697+0000 7fd98b9f4740 1 -- 172.21.15.83:0/2742276701 >> 172.21.15.83:0/2742276701 conn(0x5588e9de1c70 msgr2=0x5588e9de79d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:38:38.716 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:38.697+0000 7fd98b9f4740 1 -- 172.21.15.83:0/2742276701 shutdown_connections 2024-08-19T01:38:38.716 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:38.697+0000 7fd98b9f4740 1 -- 172.21.15.83:0/2742276701 wait complete. 2024-08-19T01:38:38.716 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:38.698+0000 7fd98b9f4740 20 client.25615 trim_cache size 0 max 16384 2024-08-19T01:38:38.731 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:38.726+0000 7fd98b9f4740 1 Processor -- start 2024-08-19T01:38:38.731 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:38.726+0000 7fd98b9f4740 1 -- start start 2024-08-19T01:38:38.731 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:38.727+0000 7fd98b9f4740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5588e9de56e0 0x5588e9e192d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:38:38.731 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:38.727+0000 7fd98b9f4740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5588e9de6000 0x5588e9e19810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:38:38.731 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:38.727+0000 7fd98b9f4740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5588e9e018d0 0x5588e9def000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:38:38.731 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:38.727+0000 7fd98b9f4740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fd964010e30 con 0x5588e9de6000 2024-08-19T01:38:38.732 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:38.727+0000 7fd98b9f4740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fd96400e700 con 0x5588e9e018d0 2024-08-19T01:38:38.732 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:38.727+0000 7fd98b9f4740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fd96400c550 con 0x5588e9de56e0 2024-08-19T01:38:38.732 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:38.728+0000 7fd982800640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5588e9e018d0 0x5588e9def000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:38:38.732 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:38.728+0000 7fd982800640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5588e9e018d0 0x5588e9def000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.83:40694/0 (socket says 172.21.15.83:40694) 2024-08-19T01:38:38.732 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:38.728+0000 7fd982800640 1 -- 172.21.15.83:0/1874662955 learned_addr learned my addr 172.21.15.83:0/1874662955 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-08-19T01:38:38.732 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:38.728+0000 7fd981e00640 1 --2- 172.21.15.83:0/1874662955 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5588e9de56e0 0x5588e9e192d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:38:38.732 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:38.729+0000 7fd982800640 1 -- 172.21.15.83:0/1874662955 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5588e9de56e0 msgr2=0x5588e9e192d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:38:38.732 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:38.729+0000 7fd982800640 1 --2- 172.21.15.83:0/1874662955 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5588e9de56e0 0x5588e9e192d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:38:38.732 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:38.729+0000 7fd982800640 1 -- 172.21.15.83:0/1874662955 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5588e9de6000 msgr2=0x5588e9e19810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:38:38.732 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:38.729+0000 7fd981400640 1 --2- 172.21.15.83:0/1874662955 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5588e9de6000 0x5588e9e19810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:38:38.732 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:38.729+0000 7fd982800640 1 --2- 172.21.15.83:0/1874662955 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5588e9de6000 0x5588e9e19810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:38:38.732 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:38.729+0000 7fd982800640 1 -- 172.21.15.83:0/1874662955 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5588e9de2e00 con 0x5588e9e018d0 2024-08-19T01:38:38.732 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:38.729+0000 7fd981e00640 1 --2- 172.21.15.83:0/1874662955 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5588e9de56e0 0x5588e9e192d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-19T01:38:38.733 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:38.729+0000 7fd981400640 1 --2- 172.21.15.83:0/1874662955 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5588e9de6000 0x5588e9e19810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-19T01:38:38.739 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:38.730+0000 7fd982800640 1 --2- 172.21.15.83:0/1874662955 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5588e9e018d0 0x5588e9def000 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fd97c030920 tx=0x7fd97c0c5630 comp rx=0 tx=0).ready entity=mon.2 client_cookie=39bec68cd0c04a4a server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:38:38.740 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:38.730+0000 7fd96ea00640 1 -- 172.21.15.83:0/1874662955 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd97c0bbb40 con 0x5588e9e018d0 2024-08-19T01:38:38.740 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:38.731+0000 7fd96ea00640 1 -- 172.21.15.83:0/1874662955 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd97c0edb80 con 0x5588e9e018d0 2024-08-19T01:38:38.741 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:38.731+0000 7fd96ea00640 1 -- 172.21.15.83:0/1874662955 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd97c0bbb40 con 0x5588e9e018d0 2024-08-19T01:38:38.741 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:38.732+0000 7fd98b9f4740 1 -- 172.21.15.83:0/1874662955 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5588e9e018d0 msgr2=0x5588e9def000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:38:38.741 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:38.732+0000 7fd98b9f4740 1 --2- 172.21.15.83:0/1874662955 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5588e9e018d0 0x5588e9def000 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fd97c030920 tx=0x7fd97c0c5630 comp rx=0 tx=0).stop 2024-08-19T01:38:38.741 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:38.732+0000 7fd98b9f4740 1 -- 172.21.15.83:0/1874662955 shutdown_connections 2024-08-19T01:38:38.741 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:38.732+0000 7fd98b9f4740 1 --2- 172.21.15.83:0/1874662955 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5588e9e018d0 0x5588e9def000 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:38:38.742 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:38.732+0000 7fd98b9f4740 1 --2- 172.21.15.83:0/1874662955 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5588e9de6000 0x5588e9e19810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:38:38.742 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:38.733+0000 7fd98b9f4740 1 --2- 172.21.15.83:0/1874662955 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5588e9de56e0 0x5588e9e192d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:38:38.742 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:38.733+0000 7fd98b9f4740 1 -- 172.21.15.83:0/1874662955 >> 172.21.15.83:0/1874662955 conn(0x5588e9de1c70 msgr2=0x5588e9dec890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:38:38.742 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:38.733+0000 7fd98b9f4740 1 -- 172.21.15.83:0/1874662955 shutdown_connections 2024-08-19T01:38:38.743 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:38.733+0000 7fd98b9f4740 1 -- 172.21.15.83:0/1874662955 wait complete. 2024-08-19T01:38:38.743 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:38.734+0000 7fd98b9f4740 1 -- start start 2024-08-19T01:38:38.743 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:38.734+0000 7fd98b9f4740 1 Processor -- start 2024-08-19T01:38:38.743 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:38.735+0000 7fd98b9f4740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5588e9de56e0 0x5588e9e09a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:38:38.743 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:38.736+0000 7fd98b9f4740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5588e9de6000 0x5588e9e09f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:38:38.743 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:38.736+0000 7fd98b9f4740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5588e9e018d0 0x5588e9e04fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:38:38.743 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:38.736+0000 7fd981e00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5588e9de56e0 0x5588e9e09a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:38:38.743 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:38.736+0000 7fd981e00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5588e9de56e0 0x5588e9e09a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.83:42296/0 (socket says 172.21.15.83:42296) 2024-08-19T01:38:38.743 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:38.737+0000 7fd98b9f4740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fd96400cbc0 con 0x5588e9de6000 2024-08-19T01:38:38.743 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:38.737+0000 7fd98b9f4740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fd96400e030 con 0x5588e9e018d0 2024-08-19T01:38:38.743 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:38.737+0000 7fd98b9f4740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fd96400d290 con 0x5588e9de56e0 2024-08-19T01:38:38.743 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:38.737+0000 7fd98b9f4740 10 client.0 _ll_register_callbacks cb 0x7fd974012450 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-19T01:38:38.743 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:38.737+0000 7fd982800640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5588e9e018d0 0x5588e9e04fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:38:38.744 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:38.737+0000 7fd982800640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5588e9e018d0 0x5588e9e04fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.83:40702/0 (socket says 172.21.15.83:40702) 2024-08-19T01:38:38.744 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:38.737+0000 7fd982800640 1 -- 172.21.15.83:0/3979399113 learned_addr learned my addr 172.21.15.83:0/3979399113 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-08-19T01:38:38.744 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:38.737+0000 7fd982800640 1 -- 172.21.15.83:0/3979399113 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5588e9de56e0 msgr2=0x5588e9e09a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:38:38.744 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:38.737+0000 7fd982800640 1 --2- 172.21.15.83:0/3979399113 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5588e9de56e0 0x5588e9e09a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:38:38.744 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:38.737+0000 7fd982800640 1 -- 172.21.15.83:0/3979399113 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5588e9de6000 msgr2=0x5588e9e09f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:38:38.744 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:38.738+0000 7fd981400640 1 --2- 172.21.15.83:0/3979399113 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5588e9de6000 0x5588e9e09f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:38:38.744 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:38.738+0000 7fd982800640 1 --2- 172.21.15.83:0/3979399113 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5588e9de6000 0x5588e9e09f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:38:38.744 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:38.738+0000 7fd982800640 1 -- 172.21.15.83:0/3979399113 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5588e9e057e0 con 0x5588e9e018d0 2024-08-19T01:38:38.744 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:38.738+0000 7fd981400640 1 --2- 172.21.15.83:0/3979399113 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5588e9de6000 0x5588e9e09f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-19T01:38:38.744 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:38.739+0000 7fd982800640 1 --2- 172.21.15.83:0/3979399113 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5588e9e018d0 0x5588e9e04fb0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fd97c0b9440 tx=0x7fd97c0098e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=387d801b8cd5d7b1 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:38:38.744 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:38.739+0000 7fd963e00640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-08-19T01:38:38.744 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:38.740+0000 7fd963e00640 1 -- 172.21.15.83:0/3979399113 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd97c03ce40 con 0x5588e9e018d0 2024-08-19T01:38:38.744 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:38.740+0000 7fd963e00640 1 -- 172.21.15.83:0/3979399113 <== mon.2 v2:172.21.15.117:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8269+0+0 (secure 0 0 0) 0x7fd97c04dbb0 con 0x5588e9e018d0 2024-08-19T01:38:38.744 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:38.741+0000 7fd98b9f4740 1 -- 172.21.15.83:0/3979399113 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd97c0edb80 con 0x5588e9e018d0 2024-08-19T01:38:38.745 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:38.741+0000 7fd963e00640 1 -- 172.21.15.83:0/3979399113 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fd97c04ddb0 con 0x5588e9e018d0 2024-08-19T01:38:38.745 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:38.743+0000 7fd98b9f4740 10 client.25667 fetch_fsmap learned FSMap version 26 2024-08-19T01:38:38.745 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:38.743+0000 7fd98b9f4740 1 -- 172.21.15.83:0/3979399113 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5588e9e7be80 con 0x5588e9e018d0 2024-08-19T01:38:38.746 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:38.743+0000 7fd963e00640 1 -- 172.21.15.83:0/3979399113 <== mon.2 v2:172.21.15.117:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd97c0cb030 con 0x5588e9e018d0 2024-08-19T01:38:38.746 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:38.745+0000 7fd963e00640 1 -- 172.21.15.83:0/3979399113 <== mon.2 v2:172.21.15.117:3300/0 5 ==== fsmap(e 26) ==== 8350+0+0 (secure 0 0 0) 0x7fd97c0bcc90 con 0x5588e9e018d0 2024-08-19T01:38:38.752 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:38.748+0000 7fd98b9f4740 10 client.25667 fetch_fsmap finished waiting for FSMap version 26 2024-08-19T01:38:38.752 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:38.748+0000 7fd98b9f4740 10 client.25667 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-08-19T01:38:38.752 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:38.748+0000 7fd98b9f4740 20 client.25667 populate_metadata read hostname 'smithi083' 2024-08-19T01:38:38.752 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:38.748+0000 7fd98b9f4740 1 --2- 172.21.15.83:0/3979399113 >> [v2:172.21.15.117:6836/3185224684,v1:172.21.15.117:6837/3185224684] conn(0x5588e9e11500 0x5588e9e7e110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:38:38.752 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:38.748+0000 7fd98b9f4740 4 client.25667 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-08-19T01:38:38.752 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:38.748+0000 7fd98b9f4740 1 -- 172.21.15.83:0/3979399113 --> [v2:172.21.15.117:6836/3185224684,v1:172.21.15.117:6837/3185224684] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5588e9d52670 con 0x5588e9e11500 2024-08-19T01:38:38.752 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:38.750+0000 7fd981e00640 1 --2- 172.21.15.83:0/3979399113 >> [v2:172.21.15.117:6836/3185224684,v1:172.21.15.117:6837/3185224684] conn(0x5588e9e11500 0x5588e9e7e110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:38:38.760 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:38.754+0000 7fd981e00640 1 --2- 172.21.15.83:0/3979399113 >> [v2:172.21.15.117:6836/3185224684,v1:172.21.15.117:6837/3185224684] conn(0x5588e9e11500 0x5588e9e7e110 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=9e81ec0d4d5b9289 server_cookie=220216dfe64fa149 in_seq=0 out_seq=0 2024-08-19T01:38:38.762 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:38.754+0000 7fd963e00640 10 client.25667 ms_handle_connect on v2:172.21.15.117:6836/3185224684 2024-08-19T01:38:38.762 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:38.754+0000 7fd963e00640 1 -- 172.21.15.83:0/3979399113 <== mds.2 v2:172.21.15.117:6836/3185224684 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5588e9d52670 con 0x5588e9e11500 2024-08-19T01:38:38.762 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:38.755+0000 7fd963e00640 10 client.25667 handle_command_reply: tid=0 2024-08-19T01:38:38.762 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:38.755+0000 7fd98b9f4740 1 client.25667 shutdown 2024-08-19T01:38:38.762 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:38.755+0000 7fd98b9f4740 1 -- 172.21.15.83:0/3979399113 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5588e9e018d0 msgr2=0x5588e9e04fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:38:38.762 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:38.755+0000 7fd98b9f4740 1 --2- 172.21.15.83:0/3979399113 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5588e9e018d0 0x5588e9e04fb0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fd97c0b9440 tx=0x7fd97c0098e0 comp rx=0 tx=0).stop 2024-08-19T01:38:38.762 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:38.755+0000 7fd98b9f4740 1 -- 172.21.15.83:0/3979399113 shutdown_connections 2024-08-19T01:38:38.762 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:38.755+0000 7fd98b9f4740 1 --2- 172.21.15.83:0/3979399113 >> [v2:172.21.15.117:6836/3185224684,v1:172.21.15.117:6837/3185224684] conn(0x5588e9e11500 0x5588e9e7e110 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:38:38.762 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:38.755+0000 7fd98b9f4740 1 --2- 172.21.15.83:0/3979399113 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5588e9e018d0 0x5588e9e04fb0 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:38:38.762 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:38.755+0000 7fd98b9f4740 1 --2- 172.21.15.83:0/3979399113 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5588e9de6000 0x5588e9e09f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:38:38.762 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:38.755+0000 7fd98b9f4740 1 --2- 172.21.15.83:0/3979399113 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5588e9de56e0 0x5588e9e09a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:38:38.763 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:38.755+0000 7fd98b9f4740 1 -- 172.21.15.83:0/3979399113 >> 172.21.15.83:0/3979399113 conn(0x5588e9de1c70 msgr2=0x5588e9de0480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:38:38.763 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:38.755+0000 7fd963e00640 0 client.25667 ms_handle_reset on v2:172.21.15.117:6836/3185224684 2024-08-19T01:38:38.763 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:38.757+0000 7fd98b9f4740 1 -- 172.21.15.83:0/3979399113 shutdown_connections 2024-08-19T01:38:38.777 INFO:teuthology.orchestra.run.smithi083.stdout:[] 2024-08-19T01:38:38.777 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:38.757+0000 7fd98b9f4740 1 -- 172.21.15.83:0/3979399113 wait complete. 2024-08-19T01:38:38.777 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:38.757+0000 7fd98b9f4740 20 client.25667 trim_cache size 0 max 16384 2024-08-19T01:38:38.777 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:38.759+0000 7fd983200640 1 -- 172.21.15.83:0/1370631152 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7fd9680a2d90 msgr2=0x7fd9680a5250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:38:38.777 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:38.759+0000 7fd983200640 1 --2- 172.21.15.83:0/1370631152 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7fd9680a2d90 0x7fd9680a5250 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fd984105b60 tx=0x7fd970005f90 comp rx=0 tx=0).stop 2024-08-19T01:38:38.777 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:38.759+0000 7fd983200640 1 -- 172.21.15.83:0/1370631152 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd984106690 msgr2=0x7fd9841a2d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:38:38.778 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:38.759+0000 7fd983200640 1 --2- 172.21.15.83:0/1370631152 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd984106690 0x7fd9841a2d30 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fd97c009b30 tx=0x7fd97c030890 comp rx=0 tx=0).stop 2024-08-19T01:38:38.778 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:38.759+0000 7fd983200640 1 -- 172.21.15.83:0/1370631152 shutdown_connections 2024-08-19T01:38:38.778 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:38.759+0000 7fd983200640 1 --2- 172.21.15.83:0/1370631152 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7fd9680a2d90 0x7fd9680a5250 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:38:38.778 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:38.759+0000 7fd983200640 1 --2- 172.21.15.83:0/1370631152 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd984106690 0x7fd9841a2d30 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:38:38.778 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:38.759+0000 7fd983200640 1 --2- 172.21.15.83:0/1370631152 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd984100fc0 0x7fd98419e9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:38:38.778 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:38.759+0000 7fd983200640 1 --2- 172.21.15.83:0/1370631152 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd984100600 0x7fd98419e460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:38:38.778 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:38.759+0000 7fd983200640 1 -- 172.21.15.83:0/1370631152 >> 172.21.15.83:0/1370631152 conn(0x7fd9840fc340 msgr2=0x7fd9840fd7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:38:38.778 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:38.759+0000 7fd983200640 1 -- 172.21.15.83:0/1370631152 shutdown_connections 2024-08-19T01:38:38.778 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:38.759+0000 7fd983200640 1 -- 172.21.15.83:0/1370631152 wait complete. 2024-08-19T01:38:38.845 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:38:38 smithi083 ceph-mon[17120]: pgmap v453: 97 pgs: 97 active+clean; 302 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 3.6 MiB/s wr, 844 op/s 2024-08-19T01:38:38.952 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:38:38 smithi117 ceph-mon[22016]: pgmap v453: 97 pgs: 97 active+clean; 302 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 3.6 MiB/s wr, 844 op/s 2024-08-19T01:38:38.966 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:38:38 smithi167 ceph-mon[21875]: pgmap v453: 97 pgs: 97 active+clean; 302 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 3.6 MiB/s wr, 844 op/s 2024-08-19T01:38:40.590 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833 shell --fsid bad8780e-5dc9-11ef-bccf-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-08-19T01:38:40.629 INFO:tasks.workunit.client.0.smithi083.stdout: 113 638610 631 413028 570 378909 1798 2024-08-19T01:38:41.657 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:38:41 smithi083 ceph-mon[17120]: pgmap v454: 97 pgs: 97 active+clean; 301 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.9 MiB/s wr, 566 op/s 2024-08-19T01:38:41.952 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:38:41 smithi117 ceph-mon[22016]: pgmap v454: 97 pgs: 97 active+clean; 301 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.9 MiB/s wr, 566 op/s 2024-08-19T01:38:41.966 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:38:41 smithi167 ceph-mon[21875]: pgmap v454: 97 pgs: 97 active+clean; 301 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.9 MiB/s wr, 566 op/s 2024-08-19T01:38:43.781 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:38:43 smithi083 ceph-mon[17120]: pgmap v455: 97 pgs: 97 active+clean; 301 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.7 MiB/s wr, 296 op/s 2024-08-19T01:38:43.952 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:38:43 smithi117 ceph-mon[22016]: pgmap v455: 97 pgs: 97 active+clean; 301 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.7 MiB/s wr, 296 op/s 2024-08-19T01:38:43.965 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:38:43 smithi167 ceph-mon[21875]: pgmap v455: 97 pgs: 97 active+clean; 301 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.7 MiB/s wr, 296 op/s 2024-08-19T01:38:44.373 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:38:44.366+0000 7f5447200640 1 -- 172.21.15.83:0/687731179 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f54480733b0 msgr2=0x7f544810d220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:38:44.374 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:38:44.366+0000 7f5447200640 1 --2- 172.21.15.83:0/687731179 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f54480733b0 0x7f544810d220 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f54340098b0 tx=0x7f543402a3d0 comp rx=0 tx=0).stop 2024-08-19T01:38:44.374 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:38:44.372+0000 7f5447200640 1 -- 172.21.15.83:0/687731179 shutdown_connections 2024-08-19T01:38:44.374 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:38:44.372+0000 7f5447200640 1 --2- 172.21.15.83:0/687731179 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f544810d8e0 0x7f544810fda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:38:44.374 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:38:44.372+0000 7f5447200640 1 --2- 172.21.15.83:0/687731179 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f54480733b0 0x7f544810d220 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:38:44.374 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:38:44.372+0000 7f5447200640 1 --2- 172.21.15.83:0/687731179 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f54480729e0 0x7f5448072de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:38:44.374 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:38:44.372+0000 7f5447200640 1 -- 172.21.15.83:0/687731179 >> 172.21.15.83:0/687731179 conn(0x7f544806e880 msgr2=0x7f5448070cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:38:44.380 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:38:44.379+0000 7f5447200640 1 -- 172.21.15.83:0/687731179 shutdown_connections 2024-08-19T01:38:44.391 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:38:44.385+0000 7f5447200640 1 -- 172.21.15.83:0/687731179 wait complete. 2024-08-19T01:38:44.391 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:38:44.387+0000 7f5447200640 1 Processor -- start 2024-08-19T01:38:44.396 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:38:44.391+0000 7f5447200640 1 -- start start 2024-08-19T01:38:44.396 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:38:44.391+0000 7f5447200640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f54480729e0 0x7f54481a68b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:38:44.397 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:38:44.391+0000 7f5447200640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f54480733b0 0x7f54481a6df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:38:44.397 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:38:44.391+0000 7f5447200640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f544810d8e0 0x7f54481ade60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:38:44.397 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:38:44.391+0000 7f5447200640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f544811c1d0 con 0x7f54480729e0 2024-08-19T01:38:44.397 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:38:44.391+0000 7f5447200640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f544811c050 con 0x7f544810d8e0 2024-08-19T01:38:44.397 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:38:44.391+0000 7f5447200640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f544811c350 con 0x7f54480733b0 2024-08-19T01:38:44.397 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:38:44.392+0000 7f5446800640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f544810d8e0 0x7f54481ade60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:38:44.397 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:38:44.392+0000 7f5445e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f54480729e0 0x7f54481a68b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:38:44.397 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:38:44.392+0000 7f5446800640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f544810d8e0 0x7f54481ade60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.83:37204/0 (socket says 172.21.15.83:37204) 2024-08-19T01:38:44.397 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:38:44.392+0000 7f5446800640 1 -- 172.21.15.83:0/3774815061 learned_addr learned my addr 172.21.15.83:0/3774815061 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-08-19T01:38:44.397 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:38:44.393+0000 7f5445400640 1 --2- 172.21.15.83:0/3774815061 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f54480733b0 0x7f54481a6df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:38:44.397 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:38:44.393+0000 7f5446800640 1 -- 172.21.15.83:0/3774815061 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f54480733b0 msgr2=0x7f54481a6df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:38:44.397 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:38:44.393+0000 7f5446800640 1 --2- 172.21.15.83:0/3774815061 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f54480733b0 0x7f54481a6df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:38:44.397 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:38:44.393+0000 7f5446800640 1 -- 172.21.15.83:0/3774815061 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f54480729e0 msgr2=0x7f54481a68b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:38:44.398 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:38:44.393+0000 7f5446800640 1 --2- 172.21.15.83:0/3774815061 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f54480729e0 0x7f54481a68b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:38:44.398 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:38:44.393+0000 7f5446800640 1 -- 172.21.15.83:0/3774815061 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5448110780 con 0x7f544810d8e0 2024-08-19T01:38:44.398 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:38:44.393+0000 7f5445e00640 1 --2- 172.21.15.83:0/3774815061 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f54480729e0 0x7f54481a68b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-19T01:38:44.398 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:38:44.395+0000 7f5446800640 1 --2- 172.21.15.83:0/3774815061 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f544810d8e0 0x7f54481ade60 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f5438009e60 tx=0x7f543800ed10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:38:44.398 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:38:44.396+0000 7f543ea00640 1 -- 172.21.15.83:0/3774815061 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f543800ccc0 con 0x7f544810d8e0 2024-08-19T01:38:44.398 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:38:44.396+0000 7f543ea00640 1 -- 172.21.15.83:0/3774815061 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5438005500 con 0x7f544810d8e0 2024-08-19T01:38:44.399 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:38:44.396+0000 7f543ea00640 1 -- 172.21.15.83:0/3774815061 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54380326a0 con 0x7f544810d8e0 2024-08-19T01:38:44.406 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:38:44.400+0000 7f5447200640 1 -- 172.21.15.83:0/3774815061 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5448110a10 con 0x7f544810d8e0 2024-08-19T01:38:44.406 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:38:44.400+0000 7f5447200640 1 -- 172.21.15.83:0/3774815061 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5448110ee0 con 0x7f544810d8e0 2024-08-19T01:38:44.416 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:38:44.414+0000 7f544fc84740 1 Processor -- start 2024-08-19T01:38:44.434 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:38:44.432+0000 7f544fc84740 1 -- start start 2024-08-19T01:38:44.434 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:38:44.433+0000 7f544fc84740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562f7b0aa660 0x562f7b0aaa60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:38:44.443 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:38:44.433+0000 7f544fc84740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562f7b0aafa0 0x562f7b0afa60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:38:44.443 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:38:44.437+0000 7f544fc84740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562f7b0b0030 0x562f7b0b24f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:38:44.443 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:38:44.437+0000 7f544fc84740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x562f7b00e890 con 0x562f7b0aa660 2024-08-19T01:38:44.444 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:38:44.437+0000 7f544fc84740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x562f7afa6d90 con 0x562f7b0aafa0 2024-08-19T01:38:44.444 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:38:44.437+0000 7f544fc84740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x562f7b00a850 con 0x562f7b0b0030 2024-08-19T01:38:44.444 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:38:44.441+0000 7f5445e00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562f7b0aafa0 0x562f7b0afa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:38:44.444 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:38:44.441+0000 7f5445e00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562f7b0aafa0 0x562f7b0afa60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.83:37218/0 (socket says 172.21.15.83:37218) 2024-08-19T01:38:44.444 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:38:44.441+0000 7f5445e00640 1 -- 172.21.15.83:0/3611724003 learned_addr learned my addr 172.21.15.83:0/3611724003 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-08-19T01:38:44.444 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:38:44.434+0000 7f5445400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562f7b0aa660 0x562f7b0aaa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:38:44.444 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:38:44.436+0000 7f543ea00640 1 -- 172.21.15.83:0/3774815061 <== mon.2 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7f54380046c0 con 0x7f544810d8e0 2024-08-19T01:38:44.444 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:38:44.442+0000 7f5445e00640 1 -- 172.21.15.83:0/3611724003 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562f7b0b0030 msgr2=0x562f7b0b24f0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-08-19T01:38:44.444 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:38:44.442+0000 7f5445e00640 1 --2- 172.21.15.83:0/3611724003 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562f7b0b0030 0x562f7b0b24f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:38:44.444 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:38:44.442+0000 7f5445e00640 1 -- 172.21.15.83:0/3611724003 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562f7b0aa660 msgr2=0x562f7b0aaa60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:38:44.445 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:38:44.443+0000 7f5445e00640 1 --2- 172.21.15.83:0/3611724003 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562f7b0aa660 0x562f7b0aaa60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:38:44.445 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:38:44.443+0000 7f5445e00640 1 -- 172.21.15.83:0/3611724003 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562f7ad22210 con 0x562f7b0aafa0 2024-08-19T01:38:44.445 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:38:44.443+0000 7f5445400640 1 --2- 172.21.15.83:0/3611724003 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562f7b0aa660 0x562f7b0aaa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-19T01:38:44.456 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:38:44.452+0000 7f543ea00640 1 --2- 172.21.15.83:0/3774815061 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f542c0a2d90 0x7f542c0a5250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:38:44.456 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:38:44.453+0000 7f5445400640 1 --2- 172.21.15.83:0/3774815061 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f542c0a2d90 0x7f542c0a5250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:38:44.456 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:38:44.454+0000 7f543ea00640 1 -- 172.21.15.83:0/3774815061 <== mon.2 v2:172.21.15.117:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8269+0+0 (secure 0 0 0) 0x7f54380f1490 con 0x7f544810d8e0 2024-08-19T01:38:44.456 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:38:44.455+0000 7f5445400640 1 --2- 172.21.15.83:0/3774815061 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f542c0a2d90 0x7f542c0a5250 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f543402a9c0 tx=0x7f5434000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:38:44.456 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:38:44.445+0000 7f5445e00640 1 --2- 172.21.15.83:0/3611724003 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562f7b0aafa0 0x562f7b0afa60 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f544000c4a0 tx=0x7f544000c8a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b202a1c04d8b06ee server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:38:44.458 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:38:44.455+0000 7f5433000640 1 -- 172.21.15.83:0/3611724003 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f544001a070 con 0x562f7b0aafa0 2024-08-19T01:38:44.458 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:38:44.455+0000 7f5433000640 1 -- 172.21.15.83:0/3611724003 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f54400158f0 con 0x562f7b0aafa0 2024-08-19T01:38:44.458 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:38:44.455+0000 7f5433000640 1 -- 172.21.15.83:0/3611724003 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f544000a9a0 con 0x562f7b0aafa0 2024-08-19T01:38:44.458 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:38:44.456+0000 7f544fc84740 1 -- 172.21.15.83:0/3611724003 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562f7b0aafa0 msgr2=0x562f7b0afa60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:38:44.458 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:38:44.456+0000 7f544fc84740 1 --2- 172.21.15.83:0/3611724003 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562f7b0aafa0 0x562f7b0afa60 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f544000c4a0 tx=0x7f544000c8a0 comp rx=0 tx=0).stop 2024-08-19T01:38:44.460 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:38:44.459+0000 7f544fc84740 1 -- 172.21.15.83:0/3611724003 shutdown_connections 2024-08-19T01:38:44.460 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:38:44.459+0000 7f544fc84740 1 --2- 172.21.15.83:0/3611724003 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562f7b0b0030 0x562f7b0b24f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:38:44.460 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:38:44.459+0000 7f544fc84740 1 --2- 172.21.15.83:0/3611724003 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562f7b0aafa0 0x562f7b0afa60 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:38:44.460 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:38:44.459+0000 7f544fc84740 1 --2- 172.21.15.83:0/3611724003 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562f7b0aa660 0x562f7b0aaa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:38:44.461 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:38:44.459+0000 7f544fc84740 1 -- 172.21.15.83:0/3611724003 >> 172.21.15.83:0/3611724003 conn(0x562f7b0a5cd0 msgr2=0x562f7b0a60d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:38:44.473 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:38:44.465+0000 7f544fc84740 1 -- 172.21.15.83:0/3611724003 shutdown_connections 2024-08-19T01:38:44.473 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:38:44.471+0000 7f544fc84740 1 -- 172.21.15.83:0/3611724003 wait complete. 2024-08-19T01:38:44.474 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:38:44.473+0000 7f544fc84740 1 -- start start 2024-08-19T01:38:44.474 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:38:44.473+0000 7f544fc84740 1 Processor -- start 2024-08-19T01:38:44.475 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:38:44.474+0000 7f544fc84740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562f7b0aa660 0x562f7b0b49e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:38:44.482 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:38:44.477+0000 7f5445e00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562f7b0aa660 0x562f7b0b49e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:38:44.482 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:38:44.477+0000 7f5445e00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562f7b0aa660 0x562f7b0b49e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.83:37230/0 (socket says 172.21.15.83:37230) 2024-08-19T01:38:44.483 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:38:44.477+0000 7f544fc84740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562f7b0aafa0 0x562f7b0b4f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:38:44.483 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:38:44.478+0000 7f5445400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562f7b0aafa0 0x562f7b0b4f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:38:44.483 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:38:44.478+0000 7f5445400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562f7b0aafa0 0x562f7b0b4f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:35780/0 (socket says 172.21.15.83:35780) 2024-08-19T01:38:44.483 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:38:44.478+0000 7f5445e00640 1 -- 172.21.15.83:0/3649721890 learned_addr learned my addr 172.21.15.83:0/3649721890 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-08-19T01:38:44.483 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:38:44.478+0000 7f544fc84740 1 --2- 172.21.15.83:0/3649721890 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562f7b0b0030 0x562f7b0bbef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:38:44.483 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:38:44.478+0000 7f544fc84740 1 -- 172.21.15.83:0/3649721890 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x562f7b0b30a0 con 0x562f7b0aafa0 2024-08-19T01:38:44.483 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:38:44.478+0000 7f544fc84740 1 -- 172.21.15.83:0/3649721890 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x562f7afc9f70 con 0x562f7b0aa660 2024-08-19T01:38:44.483 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:38:44.478+0000 7f544fc84740 1 -- 172.21.15.83:0/3649721890 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x562f7b0b3220 con 0x562f7b0b0030 2024-08-19T01:38:44.483 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:38:44.478+0000 7f544fc84740 10 client.0 _ll_register_callbacks cb 0x562f7b0068d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-19T01:38:44.483 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:38:44.478+0000 7f5446800640 1 --2- 172.21.15.83:0/3649721890 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562f7b0b0030 0x562f7b0bbef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:38:44.483 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:38:44.479+0000 7f5446800640 1 -- 172.21.15.83:0/3649721890 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562f7b0aa660 msgr2=0x562f7b0b49e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:38:44.483 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:38:44.479+0000 7f5446800640 1 --2- 172.21.15.83:0/3649721890 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562f7b0aa660 0x562f7b0b49e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:38:44.483 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:38:44.479+0000 7f5446800640 1 -- 172.21.15.83:0/3649721890 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562f7b0aafa0 msgr2=0x562f7b0b4f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:38:44.484 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:38:44.480+0000 7f5446800640 1 --2- 172.21.15.83:0/3649721890 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562f7b0aafa0 0x562f7b0b4f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:38:44.484 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:38:44.481+0000 7f5446800640 1 -- 172.21.15.83:0/3649721890 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562f7adee620 con 0x562f7b0b0030 2024-08-19T01:38:44.484 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:38:44.481+0000 7f5445e00640 1 --2- 172.21.15.83:0/3649721890 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562f7b0aa660 0x562f7b0b49e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-19T01:38:44.484 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:38:44.481+0000 7f5445400640 1 --2- 172.21.15.83:0/3649721890 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562f7b0aafa0 0x562f7b0b4f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-19T01:38:44.493 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:38:44.490+0000 7f5446800640 1 --2- 172.21.15.83:0/3649721890 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562f7b0b0030 0x562f7b0bbef0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f54380bd970 tx=0x7f54380056b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6d450f71ece150e server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:38:44.493 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:38:44.492+0000 7f5427400640 10 client.0 ms_handle_connect on v2:172.21.15.167:3300/0 2024-08-19T01:38:44.496 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:38:44.495+0000 7f5427400640 1 -- 172.21.15.83:0/3649721890 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54380db060 con 0x562f7b0b0030 2024-08-19T01:38:44.496 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:38:44.495+0000 7f5427400640 1 -- 172.21.15.83:0/3649721890 <== mon.1 v2:172.21.15.167:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8269+0+0 (secure 0 0 0) 0x7f54380ba2e0 con 0x562f7b0b0030 2024-08-19T01:38:44.508 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:38:44.501+0000 7f544fc84740 1 -- 172.21.15.83:0/3649721890 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f54400158f0 con 0x562f7b0b0030 2024-08-19T01:38:44.508 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:38:44.501+0000 7f5427400640 1 -- 172.21.15.83:0/3649721890 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f54380c2020 con 0x562f7b0b0030 2024-08-19T01:38:44.508 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:38:44.501+0000 7f544fc84740 10 client.25624 fetch_fsmap learned FSMap version 26 2024-08-19T01:38:44.508 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:38:44.501+0000 7f544fc84740 1 -- 172.21.15.83:0/3649721890 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x562f7b0bf0a0 con 0x562f7b0b0030 2024-08-19T01:38:44.509 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:38:44.502+0000 7f5427400640 1 -- 172.21.15.83:0/3649721890 <== mon.1 v2:172.21.15.167:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f54400158f0 con 0x562f7b0b0030 2024-08-19T01:38:44.509 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:38:44.502+0000 7f5427400640 1 -- 172.21.15.83:0/3649721890 <== mon.1 v2:172.21.15.167:3300/0 5 ==== fsmap(e 26) ==== 8350+0+0 (secure 0 0 0) 0x7f54380bd0c0 con 0x562f7b0b0030 2024-08-19T01:38:44.509 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:38:44.504+0000 7f544fc84740 10 client.25624 fetch_fsmap finished waiting for FSMap version 26 2024-08-19T01:38:44.509 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:38:44.504+0000 7f544fc84740 10 client.25624 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-08-19T01:38:44.509 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:38:44.504+0000 7f544fc84740 20 client.25624 populate_metadata read hostname 'smithi083' 2024-08-19T01:38:44.509 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:38:44.504+0000 7f544fc84740 1 --2- 172.21.15.83:0/3649721890 >> [v2:172.21.15.117:6838/1967595704,v1:172.21.15.117:6839/1967595704] conn(0x562f7b0c0720 0x562f7b0c2b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:38:44.509 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:38:44.504+0000 7f544fc84740 4 client.25624 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-19T01:38:44.509 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:38:44.504+0000 7f544fc84740 1 -- 172.21.15.83:0/3649721890 --> [v2:172.21.15.117:6838/1967595704,v1:172.21.15.117:6839/1967595704] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562f7b01bfc0 con 0x562f7b0c0720 2024-08-19T01:38:44.509 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:38:44.505+0000 7f5445e00640 1 --2- 172.21.15.83:0/3649721890 >> [v2:172.21.15.117:6838/1967595704,v1:172.21.15.117:6839/1967595704] conn(0x562f7b0c0720 0x562f7b0c2b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:38:44.509 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:38:44.505+0000 7f5445e00640 1 --2- 172.21.15.83:0/3649721890 >> [v2:172.21.15.117:6838/1967595704,v1:172.21.15.117:6839/1967595704] conn(0x562f7b0c0720 0x562f7b0c2b10 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2e40f09099e3838 server_cookie=ac59537007872c5a in_seq=0 out_seq=0 2024-08-19T01:38:44.509 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:38:44.505+0000 7f5427400640 10 client.25624 ms_handle_connect on v2:172.21.15.117:6838/1967595704 2024-08-19T01:38:44.526 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:38:44.525+0000 7f5427400640 1 -- 172.21.15.83:0/3649721890 <== mds.0 v2:172.21.15.117:6838/1967595704 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x562f7b01bfc0 con 0x562f7b0c0720 2024-08-19T01:38:44.526 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:38:44.525+0000 7f5427400640 10 client.25624 handle_command_reply: tid=0 2024-08-19T01:38:44.529 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:38:44.527+0000 7f544fc84740 1 client.25624 shutdown 2024-08-19T01:38:44.529 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:38:44.528+0000 7f544fc84740 1 -- 172.21.15.83:0/3649721890 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562f7b0b0030 msgr2=0x562f7b0bbef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:38:44.529 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:38:44.528+0000 7f544fc84740 1 --2- 172.21.15.83:0/3649721890 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562f7b0b0030 0x562f7b0bbef0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f54380bd970 tx=0x7f54380056b0 comp rx=0 tx=0).stop 2024-08-19T01:38:44.530 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:38:44.528+0000 7f544fc84740 1 -- 172.21.15.83:0/3649721890 shutdown_connections 2024-08-19T01:38:44.531 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:38:44.528+0000 7f544fc84740 1 --2- 172.21.15.83:0/3649721890 >> [v2:172.21.15.117:6838/1967595704,v1:172.21.15.117:6839/1967595704] conn(0x562f7b0c0720 0x562f7b0c2b10 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:38:44.531 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:38:44.528+0000 7f544fc84740 1 --2- 172.21.15.83:0/3649721890 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562f7b0b0030 0x562f7b0bbef0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:38:44.531 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:38:44.528+0000 7f544fc84740 1 --2- 172.21.15.83:0/3649721890 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562f7b0aafa0 0x562f7b0b4f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:38:44.531 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:38:44.528+0000 7f544fc84740 1 --2- 172.21.15.83:0/3649721890 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562f7b0aa660 0x562f7b0b49e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:38:44.531 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:38:44.528+0000 7f544fc84740 1 -- 172.21.15.83:0/3649721890 >> 172.21.15.83:0/3649721890 conn(0x562f7b0a5cd0 msgr2=0x562f7b0a74e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:38:44.531 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:38:44.528+0000 7f5427400640 0 client.25624 ms_handle_reset on v2:172.21.15.117:6838/1967595704 2024-08-19T01:38:44.538 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:38:44.532+0000 7f544fc84740 1 -- 172.21.15.83:0/3649721890 shutdown_connections 2024-08-19T01:38:44.538 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:38:44.532+0000 7f544fc84740 1 -- 172.21.15.83:0/3649721890 wait complete. 2024-08-19T01:38:44.538 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:38:44.532+0000 7f544fc84740 20 client.25624 trim_cache size 0 max 16384 2024-08-19T01:38:44.706 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:38:44.704+0000 7f544fc84740 1 Processor -- start 2024-08-19T01:38:44.716 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:38:44.715+0000 7f544fc84740 1 -- start start 2024-08-19T01:38:44.716 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:38:44.715+0000 7f544fc84740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562f7b0aa660 0x562f7b0b49e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:38:44.717 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:38:44.715+0000 7f544fc84740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562f7b0aafa0 0x562f7b0bbeb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:38:44.717 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:38:44.716+0000 7f5445e00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562f7b0aa660 0x562f7b0b49e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:38:44.717 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:38:44.716+0000 7f5445e00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562f7b0aa660 0x562f7b0b49e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.83:37244/0 (socket says 172.21.15.83:37244) 2024-08-19T01:38:44.717 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:38:44.716+0000 7f5445e00640 1 -- 172.21.15.83:0/604808257 learned_addr learned my addr 172.21.15.83:0/604808257 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-08-19T01:38:44.718 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:38:44.716+0000 7f544fc84740 1 --2- 172.21.15.83:0/604808257 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562f7b0b0030 0x562f7b0b4f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:38:44.718 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:38:44.716+0000 7f544fc84740 1 -- 172.21.15.83:0/604808257 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f54481b1b70 con 0x562f7b0b0030 2024-08-19T01:38:44.718 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:38:44.716+0000 7f544fc84740 1 -- 172.21.15.83:0/604808257 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f54481af440 con 0x562f7b0aa660 2024-08-19T01:38:44.718 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:38:44.716+0000 7f544fc84740 1 -- 172.21.15.83:0/604808257 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f5448117d50 con 0x562f7b0aafa0 2024-08-19T01:38:44.719 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:38:44.716+0000 7f5445e00640 1 -- 172.21.15.83:0/604808257 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562f7b0aafa0 msgr2=0x562f7b0bbeb0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-08-19T01:38:44.719 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:38:44.716+0000 7f5445e00640 1 --2- 172.21.15.83:0/604808257 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562f7b0aafa0 0x562f7b0bbeb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:38:44.719 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:38:44.717+0000 7f5445e00640 1 -- 172.21.15.83:0/604808257 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562f7b0b0030 msgr2=0x562f7b0b4f20 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-08-19T01:38:44.719 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:38:44.717+0000 7f5446800640 1 --2- 172.21.15.83:0/604808257 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562f7b0b0030 0x562f7b0b4f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:38:44.719 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:38:44.717+0000 7f5445e00640 1 --2- 172.21.15.83:0/604808257 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562f7b0b0030 0x562f7b0b4f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:38:44.719 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:38:44.717+0000 7f5445e00640 1 -- 172.21.15.83:0/604808257 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562f7b0c2e80 con 0x562f7b0aa660 2024-08-19T01:38:44.719 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:38:44.718+0000 7f5445e00640 1 --2- 172.21.15.83:0/604808257 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562f7b0aa660 0x562f7b0b49e0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f5440018040 tx=0x7f5440009850 comp rx=0 tx=0).ready entity=mon.2 client_cookie=dbd4e150fad90665 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:38:44.728 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:38:44.727+0000 7f5432600640 1 -- 172.21.15.83:0/604808257 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f544001a040 con 0x562f7b0aa660 2024-08-19T01:38:44.728 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:38:44.727+0000 7f5432600640 1 -- 172.21.15.83:0/604808257 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5440015410 con 0x562f7b0aa660 2024-08-19T01:38:44.728 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:38:44.727+0000 7f5432600640 1 -- 172.21.15.83:0/604808257 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5440004400 con 0x562f7b0aa660 2024-08-19T01:38:44.742 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:38:44.736+0000 7f544fc84740 1 -- 172.21.15.83:0/604808257 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562f7b0aa660 msgr2=0x562f7b0b49e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:38:44.742 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:38:44.736+0000 7f544fc84740 1 --2- 172.21.15.83:0/604808257 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562f7b0aa660 0x562f7b0b49e0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f5440018040 tx=0x7f5440009850 comp rx=0 tx=0).stop 2024-08-19T01:38:44.743 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:38:44.738+0000 7f544fc84740 1 -- 172.21.15.83:0/604808257 shutdown_connections 2024-08-19T01:38:44.743 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:38:44.738+0000 7f544fc84740 1 --2- 172.21.15.83:0/604808257 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562f7b0b0030 0x562f7b0b4f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:38:44.743 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:38:44.738+0000 7f544fc84740 1 --2- 172.21.15.83:0/604808257 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562f7b0aafa0 0x562f7b0bbeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:38:44.743 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:38:44.738+0000 7f544fc84740 1 --2- 172.21.15.83:0/604808257 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562f7b0aa660 0x562f7b0b49e0 secure :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f5440018040 tx=0x7f5440009850 comp rx=0 tx=0).stop 2024-08-19T01:38:44.743 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:38:44.738+0000 7f544fc84740 1 -- 172.21.15.83:0/604808257 >> 172.21.15.83:0/604808257 conn(0x562f7b0a5cd0 msgr2=0x562f7b0af820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:38:44.743 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:38:44.739+0000 7f544fc84740 1 -- 172.21.15.83:0/604808257 shutdown_connections 2024-08-19T01:38:44.743 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:38:44.740+0000 7f544fc84740 1 -- 172.21.15.83:0/604808257 wait complete. 2024-08-19T01:38:44.743 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:38:44.740+0000 7f544fc84740 1 -- start start 2024-08-19T01:38:44.743 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:38:44.740+0000 7f544fc84740 1 Processor -- start 2024-08-19T01:38:44.748 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:38:44.745+0000 7f544fc84740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562f7b0aa660 0x562f7b0b8f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:38:44.748 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:38:44.745+0000 7f544fc84740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562f7b0aafa0 0x562f7b0b94b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:38:44.748 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:38:44.745+0000 7f544fc84740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562f7b0b0030 0x562f7b0ba420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:38:44.748 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:38:44.745+0000 7f544fc84740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f54481183c0 con 0x562f7b0aafa0 2024-08-19T01:38:44.748 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:38:44.745+0000 7f544fc84740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f54481aed70 con 0x562f7b0aa660 2024-08-19T01:38:44.748 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:38:44.745+0000 7f544fc84740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f5448118a90 con 0x562f7b0b0030 2024-08-19T01:38:44.748 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:38:44.745+0000 7f544fc84740 10 client.0 _ll_register_callbacks cb 0x7f543800ef00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-19T01:38:44.748 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:38:44.746+0000 7f5446800640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562f7b0b0030 0x562f7b0ba420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:38:44.748 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:38:44.746+0000 7f5446800640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562f7b0b0030 0x562f7b0ba420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.83:51888/0 (socket says 172.21.15.83:51888) 2024-08-19T01:38:44.749 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:38:44.746+0000 7f5446800640 1 -- 172.21.15.83:0/158553601 learned_addr learned my addr 172.21.15.83:0/158553601 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-08-19T01:38:44.749 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:38:44.746+0000 7f5445e00640 1 --2- 172.21.15.83:0/158553601 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562f7b0aa660 0x562f7b0b8f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:38:44.749 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:38:44.746+0000 7f5446800640 1 -- 172.21.15.83:0/158553601 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562f7b0aa660 msgr2=0x562f7b0b8f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:38:44.749 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:38:44.746+0000 7f5446800640 1 --2- 172.21.15.83:0/158553601 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562f7b0aa660 0x562f7b0b8f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:38:44.749 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:38:44.746+0000 7f5446800640 1 -- 172.21.15.83:0/158553601 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562f7b0aafa0 msgr2=0x562f7b0b94b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:38:44.749 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:38:44.746+0000 7f5446800640 1 --2- 172.21.15.83:0/158553601 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562f7b0aafa0 0x562f7b0b94b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:38:44.749 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:38:44.747+0000 7f5446800640 1 -- 172.21.15.83:0/158553601 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562f7b0baa30 con 0x562f7b0b0030 2024-08-19T01:38:44.750 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:38:44.747+0000 7f5446800640 1 --2- 172.21.15.83:0/158553601 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562f7b0b0030 0x562f7b0ba420 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f54380bbf20 tx=0x7f5438042000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d48967d1bfdbe334 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:38:44.750 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:38:44.747+0000 7f5427400640 10 client.0 ms_handle_connect on v2:172.21.15.167:3300/0 2024-08-19T01:38:44.751 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:38:44.747+0000 7f5427400640 1 -- 172.21.15.83:0/158553601 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54380db040 con 0x562f7b0b0030 2024-08-19T01:38:44.751 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:38:44.747+0000 7f5427400640 1 -- 172.21.15.83:0/158553601 <== mon.1 v2:172.21.15.167:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8269+0+0 (secure 0 0 0) 0x7f54380c6b30 con 0x562f7b0b0030 2024-08-19T01:38:44.751 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:38:44.748+0000 7f544fc84740 1 -- 172.21.15.83:0/158553601 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5440015410 con 0x562f7b0b0030 2024-08-19T01:38:44.751 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:38:44.748+0000 7f5427400640 1 -- 172.21.15.83:0/158553601 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f54380b9f10 con 0x562f7b0b0030 2024-08-19T01:38:44.751 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:38:44.748+0000 7f544fc84740 10 client.25627 fetch_fsmap learned FSMap version 26 2024-08-19T01:38:44.751 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:38:44.748+0000 7f544fc84740 1 -- 172.21.15.83:0/158553601 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x562f7b0bbdb0 con 0x562f7b0b0030 2024-08-19T01:38:44.751 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:38:44.748+0000 7f5427400640 1 -- 172.21.15.83:0/158553601 <== mon.1 v2:172.21.15.167:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5440015410 con 0x562f7b0b0030 2024-08-19T01:38:44.770 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:38:44.752+0000 7f5427400640 1 -- 172.21.15.83:0/158553601 <== mon.1 v2:172.21.15.167:3300/0 5 ==== fsmap(e 26) ==== 8350+0+0 (secure 0 0 0) 0x7f54380c33b0 con 0x562f7b0b0030 2024-08-19T01:38:44.770 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:38:44.752+0000 7f544fc84740 10 client.25627 fetch_fsmap finished waiting for FSMap version 26 2024-08-19T01:38:44.770 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:38:44.752+0000 7f544fc84740 10 client.25627 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-08-19T01:38:44.770 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:38:44.752+0000 7f544fc84740 20 client.25627 populate_metadata read hostname 'smithi083' 2024-08-19T01:38:44.770 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:38:44.752+0000 7f544fc84740 1 --2- 172.21.15.83:0/158553601 >> [v2:172.21.15.117:6838/1967595704,v1:172.21.15.117:6839/1967595704] conn(0x562f7b12b590 0x562f7b12d980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:38:44.770 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:38:44.752+0000 7f544fc84740 4 client.25627 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-08-19T01:38:44.770 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:38:44.752+0000 7f544fc84740 1 -- 172.21.15.83:0/158553601 --> [v2:172.21.15.117:6838/1967595704,v1:172.21.15.117:6839/1967595704] -- command(tid 0: {"prefix": "flush journal"}) -- 0x562f7b016310 con 0x562f7b12b590 2024-08-19T01:38:44.770 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:38:44.752+0000 7f5445e00640 1 --2- 172.21.15.83:0/158553601 >> [v2:172.21.15.117:6838/1967595704,v1:172.21.15.117:6839/1967595704] conn(0x562f7b12b590 0x562f7b12d980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:38:44.770 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:38:44.753+0000 7f5445e00640 1 --2- 172.21.15.83:0/158553601 >> [v2:172.21.15.117:6838/1967595704,v1:172.21.15.117:6839/1967595704] conn(0x562f7b12b590 0x562f7b12d980 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=196de61c168533d4 server_cookie=45effc5b0582d1a4 in_seq=0 out_seq=0 2024-08-19T01:38:44.770 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:38:44.753+0000 7f5427400640 10 client.25627 ms_handle_connect on v2:172.21.15.117:6838/1967595704 2024-08-19T01:38:45.914 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:38:45 smithi083 ceph-mon[17120]: pgmap v456: 97 pgs: 97 active+clean; 311 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 2.7 MiB/s wr, 280 op/s 2024-08-19T01:38:45.952 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:38:45 smithi117 ceph-mon[22016]: pgmap v456: 97 pgs: 97 active+clean; 311 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 2.7 MiB/s wr, 280 op/s 2024-08-19T01:38:45.965 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:38:45 smithi167 ceph-mon[21875]: pgmap v456: 97 pgs: 97 active+clean; 311 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 2.7 MiB/s wr, 280 op/s 2024-08-19T01:38:46.905 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:38:46 smithi083 ceph-mon[17120]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-08-19T01:38:46.952 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:38:46 smithi117 ceph-mon[22016]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-08-19T01:38:46.965 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:38:46 smithi167 ceph-mon[21875]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-08-19T01:38:47.905 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:38:47 smithi083 ceph-mon[17120]: pgmap v457: 97 pgs: 97 active+clean; 314 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.7 MiB/s wr, 166 op/s 2024-08-19T01:38:47.952 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:38:47 smithi117 ceph-mon[22016]: pgmap v457: 97 pgs: 97 active+clean; 314 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.7 MiB/s wr, 166 op/s 2024-08-19T01:38:47.965 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:38:47 smithi167 ceph-mon[21875]: pgmap v457: 97 pgs: 97 active+clean; 314 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.7 MiB/s wr, 166 op/s 2024-08-19T01:38:48.347 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:38:48.346+0000 7f5427400640 1 -- 172.21.15.83:0/158553601 <== mds.0 v2:172.21.15.117:6838/1967595704 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x562f7b016310 con 0x562f7b12b590 2024-08-19T01:38:48.347 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:38:48.346+0000 7f5427400640 10 client.25627 handle_command_reply: tid=0 2024-08-19T01:38:48.360 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:38:48.346+0000 7f544fc84740 1 client.25627 shutdown 2024-08-19T01:38:48.399 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:38:48.395+0000 7f544fc84740 1 -- 172.21.15.83:0/158553601 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562f7b0b0030 msgr2=0x562f7b0ba420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:38:48.399 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:38:48.395+0000 7f544fc84740 1 --2- 172.21.15.83:0/158553601 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562f7b0b0030 0x562f7b0ba420 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f54380bbf20 tx=0x7f5438042000 comp rx=0 tx=0).stop 2024-08-19T01:38:48.400 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:38:48.398+0000 7f544fc84740 1 -- 172.21.15.83:0/158553601 shutdown_connections 2024-08-19T01:38:48.401 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:38:48.398+0000 7f544fc84740 1 --2- 172.21.15.83:0/158553601 >> [v2:172.21.15.117:6838/1967595704,v1:172.21.15.117:6839/1967595704] conn(0x562f7b12b590 0x562f7b12d980 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:38:48.401 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:38:48.398+0000 7f544fc84740 1 --2- 172.21.15.83:0/158553601 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562f7b0b0030 0x562f7b0ba420 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:38:48.401 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:38:48.398+0000 7f544fc84740 1 --2- 172.21.15.83:0/158553601 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562f7b0aafa0 0x562f7b0b94b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:38:48.401 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:38:48.398+0000 7f544fc84740 1 --2- 172.21.15.83:0/158553601 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562f7b0aa660 0x562f7b0b8f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:38:48.401 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:38:48.398+0000 7f544fc84740 1 -- 172.21.15.83:0/158553601 >> 172.21.15.83:0/158553601 conn(0x562f7b0a5cd0 msgr2=0x562f7b0af420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:38:48.401 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:38:48.398+0000 7f5427400640 0 client.25627 ms_handle_reset on v2:172.21.15.117:6838/1967595704 2024-08-19T01:38:48.406 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:38:48.405+0000 7f544fc84740 1 -- 172.21.15.83:0/158553601 shutdown_connections 2024-08-19T01:38:48.406 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:38:48.405+0000 7f544fc84740 1 -- 172.21.15.83:0/158553601 wait complete. 2024-08-19T01:38:48.407 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:38:48.405+0000 7f544fc84740 20 client.25627 trim_cache size 0 max 16384 2024-08-19T01:38:48.407 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stdout:{ 2024-08-19T01:38:48.407 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stdout: "message": "", 2024-08-19T01:38:48.407 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stdout: "return_code": 0 2024-08-19T01:38:48.407 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stdout:} 2024-08-19T01:38:48.463 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:38:48.437+0000 7f5447200640 1 -- 172.21.15.83:0/3774815061 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f542c0a2d90 msgr2=0x7f542c0a5250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:38:48.463 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:38:48.437+0000 7f5447200640 1 --2- 172.21.15.83:0/3774815061 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f542c0a2d90 0x7f542c0a5250 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f543402a9c0 tx=0x7f5434000bc0 comp rx=0 tx=0).stop 2024-08-19T01:38:48.463 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:38:48.437+0000 7f5447200640 1 -- 172.21.15.83:0/3774815061 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f544810d8e0 msgr2=0x7f54481ade60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:38:48.463 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:38:48.437+0000 7f5447200640 1 --2- 172.21.15.83:0/3774815061 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f544810d8e0 0x7f54481ade60 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f5438009e60 tx=0x7f543800ed10 comp rx=0 tx=0).stop 2024-08-19T01:38:48.463 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:38:48.437+0000 7f5447200640 1 -- 172.21.15.83:0/3774815061 shutdown_connections 2024-08-19T01:38:48.463 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:38:48.437+0000 7f5447200640 1 --2- 172.21.15.83:0/3774815061 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f542c0a2d90 0x7f542c0a5250 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:38:48.463 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:38:48.437+0000 7f5447200640 1 --2- 172.21.15.83:0/3774815061 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f544810d8e0 0x7f54481ade60 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:38:48.464 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:38:48.437+0000 7f5447200640 1 --2- 172.21.15.83:0/3774815061 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f54480733b0 0x7f54481a6df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:38:48.464 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:38:48.437+0000 7f5447200640 1 --2- 172.21.15.83:0/3774815061 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f54480729e0 0x7f54481a68b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:38:48.464 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:38:48.437+0000 7f5447200640 1 -- 172.21.15.83:0/3774815061 >> 172.21.15.83:0/3774815061 conn(0x7f544806e880 msgr2=0x7f5448070620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:38:48.464 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:38:48.437+0000 7f5447200640 1 -- 172.21.15.83:0/3774815061 shutdown_connections 2024-08-19T01:38:48.464 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:38:48.437+0000 7f5447200640 1 -- 172.21.15.83:0/3774815061 wait complete. 2024-08-19T01:38:49.877 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:38:49 smithi083 ceph-mon[17120]: pgmap v458: 97 pgs: 97 active+clean; 328 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 3.6 MiB/s wr, 741 op/s 2024-08-19T01:38:49.952 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:38:49 smithi117 ceph-mon[22016]: pgmap v458: 97 pgs: 97 active+clean; 328 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 3.6 MiB/s wr, 741 op/s 2024-08-19T01:38:49.965 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:38:49 smithi167 ceph-mon[21875]: pgmap v458: 97 pgs: 97 active+clean; 328 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 3.6 MiB/s wr, 741 op/s 2024-08-19T01:38:50.019 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/bad8780e-5dc9-11ef-bccf-c7b262605968/mon.a/config 2024-08-19T01:38:50.616 INFO:tasks.workunit.client.0.smithi083.stdout: 116 521056 326 336208 333 304421 967 2024-08-19T01:38:51.635 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:38:51 smithi083 ceph-mon[17120]: pgmap v459: 97 pgs: 97 active+clean; 332 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 4.6 MiB/s wr, 1.22k op/s 2024-08-19T01:38:51.748 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:51.742+0000 7f8945400640 1 -- 172.21.15.83:0/1746932391 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8940102090 msgr2=0x7f894010e5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:38:51.748 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:51.742+0000 7f8945400640 1 --2- 172.21.15.83:0/1746932391 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8940102090 0x7f894010e5c0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f8938009c10 tx=0x7f893802a700 comp rx=0 tx=0).stop 2024-08-19T01:38:51.748 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:51.744+0000 7f8945400640 1 -- 172.21.15.83:0/1746932391 shutdown_connections 2024-08-19T01:38:51.748 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:51.744+0000 7f8945400640 1 --2- 172.21.15.83:0/1746932391 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8940102090 0x7f894010e5c0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:38:51.748 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:51.744+0000 7f8945400640 1 --2- 172.21.15.83:0/1746932391 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f89401016f0 0x7f8940101b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:38:51.748 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:51.744+0000 7f8945400640 1 --2- 172.21.15.83:0/1746932391 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f89401076f0 0x7f8940107ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:38:51.748 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:51.744+0000 7f8945400640 1 -- 172.21.15.83:0/1746932391 >> 172.21.15.83:0/1746932391 conn(0x7f89400fd470 msgr2=0x7f89400ff890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:38:51.748 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:51.744+0000 7f8945400640 1 -- 172.21.15.83:0/1746932391 shutdown_connections 2024-08-19T01:38:51.749 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:51.747+0000 7f8945400640 1 -- 172.21.15.83:0/1746932391 wait complete. 2024-08-19T01:38:51.750 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:51.749+0000 7f8945400640 1 Processor -- start 2024-08-19T01:38:51.750 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:51.749+0000 7f8945400640 1 -- start start 2024-08-19T01:38:51.753 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:51.751+0000 7f8945400640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f89401016f0 0x7f8940115e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:38:51.754 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:51.751+0000 7f8945400640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8940102090 0x7f8940116350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:38:51.754 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:51.752+0000 7f8945400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f89401076f0 0x7f894011bb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:38:51.760 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:51.752+0000 7f8944a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f89401076f0 0x7f894011bb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:38:51.760 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:51.752+0000 7f8944a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f89401076f0 0x7f894011bb30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:33390/0 (socket says 172.21.15.83:33390) 2024-08-19T01:38:51.760 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:51.753+0000 7f8944a00640 1 -- 172.21.15.83:0/3480776426 learned_addr learned my addr 172.21.15.83:0/3480776426 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-08-19T01:38:51.760 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:51.756+0000 7f8945400640 1 -- 172.21.15.83:0/3480776426 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f8940075e10 con 0x7f89401076f0 2024-08-19T01:38:51.760 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:51.756+0000 7f8945400640 1 -- 172.21.15.83:0/3480776426 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f8940075c90 con 0x7f89401016f0 2024-08-19T01:38:51.760 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:51.756+0000 7f8945400640 1 -- 172.21.15.83:0/3480776426 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f8940075f90 con 0x7f8940102090 2024-08-19T01:38:51.760 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:51.756+0000 7f893f400640 1 --2- 172.21.15.83:0/3480776426 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8940102090 0x7f8940116350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:38:51.761 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:51.757+0000 7f8944a00640 1 -- 172.21.15.83:0/3480776426 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8940102090 msgr2=0x7f8940116350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:38:51.761 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:51.757+0000 7f8944a00640 1 --2- 172.21.15.83:0/3480776426 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8940102090 0x7f8940116350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:38:51.761 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:51.757+0000 7f8944a00640 1 -- 172.21.15.83:0/3480776426 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f89401016f0 msgr2=0x7f8940115e10 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-08-19T01:38:51.761 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:51.757+0000 7f8944a00640 1 --2- 172.21.15.83:0/3480776426 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f89401016f0 0x7f8940115e10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:38:51.761 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:51.757+0000 7f8944a00640 1 -- 172.21.15.83:0/3480776426 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f894011c3f0 con 0x7f89401076f0 2024-08-19T01:38:51.761 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:51.758+0000 7f8944a00640 1 --2- 172.21.15.83:0/3480776426 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f89401076f0 0x7f894011bb30 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f8938034040 tx=0x7f8938003f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:38:51.761 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:51.758+0000 7f893cc00640 1 -- 172.21.15.83:0/3480776426 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8938040020 con 0x7f89401076f0 2024-08-19T01:38:51.761 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:51.759+0000 7f893cc00640 1 -- 172.21.15.83:0/3480776426 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8938031920 con 0x7f89401076f0 2024-08-19T01:38:51.761 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:51.759+0000 7f893cc00640 1 -- 172.21.15.83:0/3480776426 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8938036040 con 0x7f89401076f0 2024-08-19T01:38:51.765 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:51.761+0000 7f8945400640 1 -- 172.21.15.83:0/3480776426 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f894011c680 con 0x7f89401076f0 2024-08-19T01:38:51.765 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:51.761+0000 7f8945400640 1 -- 172.21.15.83:0/3480776426 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8940111100 con 0x7f89401076f0 2024-08-19T01:38:51.778 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:51.771+0000 7f8949463740 1 Processor -- start 2024-08-19T01:38:51.779 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:51.772+0000 7f893cc00640 1 -- 172.21.15.83:0/3480776426 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7f8938043020 con 0x7f89401076f0 2024-08-19T01:38:51.779 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:51.774+0000 7f8949463740 1 -- start start 2024-08-19T01:38:51.779 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:51.774+0000 7f8949463740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55db8d2c06e0 0x55db8d2c0ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:38:51.779 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:51.774+0000 7f8949463740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55db8d2c1000 0x55db8d2cb380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:38:51.779 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:51.774+0000 7f8949463740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55db8d2cb8c0 0x55db8d2cdcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:38:51.779 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:51.774+0000 7f8949463740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55db8d225bf0 con 0x55db8d2cb8c0 2024-08-19T01:38:51.780 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:51.774+0000 7f8949463740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55db8d1bdb20 con 0x55db8d2c1000 2024-08-19T01:38:51.780 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:51.774+0000 7f8949463740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55db8d221bb0 con 0x55db8d2c06e0 2024-08-19T01:38:51.780 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:51.776+0000 7f893f400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55db8d2cb8c0 0x55db8d2cdcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:38:51.780 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:51.776+0000 7f893f400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55db8d2cb8c0 0x55db8d2cdcb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:33398/0 (socket says 172.21.15.83:33398) 2024-08-19T01:38:51.780 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:51.776+0000 7f893f400640 1 -- 172.21.15.83:0/885971319 learned_addr learned my addr 172.21.15.83:0/885971319 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-08-19T01:38:51.780 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:51.777+0000 7f893f400640 1 -- 172.21.15.83:0/885971319 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55db8d2c06e0 msgr2=0x55db8d2c0ac0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-08-19T01:38:51.780 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:51.777+0000 7f893f400640 1 --2- 172.21.15.83:0/885971319 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55db8d2c06e0 0x55db8d2c0ac0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:38:51.780 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:51.778+0000 7f893f400640 1 -- 172.21.15.83:0/885971319 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55db8d2c1000 msgr2=0x55db8d2cb380 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-08-19T01:38:51.780 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:51.778+0000 7f893f400640 1 --2- 172.21.15.83:0/885971319 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55db8d2c1000 0x55db8d2cb380 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:38:51.781 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:51.778+0000 7f893f400640 1 -- 172.21.15.83:0/885971319 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55db8d1c47f0 con 0x55db8d2cb8c0 2024-08-19T01:38:51.781 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:51.779+0000 7f893f400640 1 --2- 172.21.15.83:0/885971319 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55db8d2cb8c0 0x55db8d2cdcb0 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f893000b6e0 tx=0x7f893000bba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=61899bf01c948fab server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:38:51.782 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:51.780+0000 7f8935200640 1 -- 172.21.15.83:0/885971319 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8930003bd0 con 0x55db8d2cb8c0 2024-08-19T01:38:51.782 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:51.780+0000 7f8935200640 1 -- 172.21.15.83:0/885971319 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8930015070 con 0x55db8d2cb8c0 2024-08-19T01:38:51.783 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:51.780+0000 7f8935200640 1 -- 172.21.15.83:0/885971319 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8930011730 con 0x55db8d2cb8c0 2024-08-19T01:38:51.784 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:51.782+0000 7f893cc00640 1 --2- 172.21.15.83:0/3480776426 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f89240a2cc0 0x7f89240a5180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:38:51.784 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:51.782+0000 7f893cc00640 1 -- 172.21.15.83:0/3480776426 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8269+0+0 (secure 0 0 0) 0x7f89380efaf0 con 0x7f89401076f0 2024-08-19T01:38:51.785 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:51.783+0000 7f893fe00640 1 --2- 172.21.15.83:0/3480776426 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f89240a2cc0 0x7f89240a5180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:38:51.786 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:51.784+0000 7f8949463740 1 -- 172.21.15.83:0/885971319 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55db8d2cb8c0 msgr2=0x55db8d2cdcb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:38:51.786 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:51.784+0000 7f8949463740 1 --2- 172.21.15.83:0/885971319 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55db8d2cb8c0 0x55db8d2cdcb0 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f893000b6e0 tx=0x7f893000bba0 comp rx=0 tx=0).stop 2024-08-19T01:38:51.786 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:51.784+0000 7f8949463740 1 -- 172.21.15.83:0/885971319 shutdown_connections 2024-08-19T01:38:51.787 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:51.784+0000 7f8949463740 1 --2- 172.21.15.83:0/885971319 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55db8d2cb8c0 0x55db8d2cdcb0 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:38:51.787 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:51.784+0000 7f8949463740 1 --2- 172.21.15.83:0/885971319 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55db8d2c1000 0x55db8d2cb380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:38:51.788 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:51.784+0000 7f8949463740 1 --2- 172.21.15.83:0/885971319 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55db8d2c06e0 0x55db8d2c0ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:38:51.788 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:51.784+0000 7f8949463740 1 -- 172.21.15.83:0/885971319 >> 172.21.15.83:0/885971319 conn(0x55db8d2bcc70 msgr2=0x55db8d2bd050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:38:51.788 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:51.784+0000 7f8949463740 1 -- 172.21.15.83:0/885971319 shutdown_connections 2024-08-19T01:38:51.788 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:51.784+0000 7f8949463740 1 -- 172.21.15.83:0/885971319 wait complete. 2024-08-19T01:38:51.789 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:51.785+0000 7f8949463740 1 -- start start 2024-08-19T01:38:51.789 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:51.785+0000 7f8949463740 1 Processor -- start 2024-08-19T01:38:51.789 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:51.787+0000 7f8949463740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55db8d2c06e0 0x55db8d2c8720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:38:51.789 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:51.787+0000 7f8949463740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55db8d2c1000 0x55db8d2c8c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:38:51.789 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:51.787+0000 7f8949463740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55db8d2cb8c0 0x55db8d2c91a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:38:51.789 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:51.787+0000 7f8949463740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55db8d2ce490 con 0x55db8d2cb8c0 2024-08-19T01:38:51.789 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:51.787+0000 7f8949463740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55db8d1e0cf0 con 0x55db8d2c1000 2024-08-19T01:38:51.790 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:51.787+0000 7f8949463740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55db8d2ce610 con 0x55db8d2c06e0 2024-08-19T01:38:51.790 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:51.787+0000 7f8949463740 10 client.0 _ll_register_callbacks cb 0x55db8d1cce30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-19T01:38:51.790 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:51.787+0000 7f8944a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55db8d2cb8c0 0x55db8d2c91a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:38:51.791 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:51.787+0000 7f893fe00640 1 --2- 172.21.15.83:0/3480776426 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f89240a2cc0 0x7f89240a5180 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x55db8d2bdad0 tx=0x7f892c005ed0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:38:51.791 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:51.787+0000 7f8944a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55db8d2cb8c0 0x55db8d2c91a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:33408/0 (socket says 172.21.15.83:33408) 2024-08-19T01:38:51.792 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:51.788+0000 7f8944a00640 1 -- 172.21.15.83:0/2513687496 learned_addr learned my addr 172.21.15.83:0/2513687496 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-08-19T01:38:51.792 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:51.788+0000 7f893f400640 1 --2- 172.21.15.83:0/2513687496 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55db8d2c1000 0x55db8d2c8c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:38:51.792 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:51.788+0000 7f893fe00640 1 --2- 172.21.15.83:0/2513687496 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55db8d2c06e0 0x55db8d2c8720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:38:51.792 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:51.788+0000 7f8944a00640 1 -- 172.21.15.83:0/2513687496 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55db8d2c06e0 msgr2=0x55db8d2c8720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:38:51.792 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:51.788+0000 7f8944a00640 1 --2- 172.21.15.83:0/2513687496 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55db8d2c06e0 0x55db8d2c8720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:38:51.792 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:51.788+0000 7f8944a00640 1 -- 172.21.15.83:0/2513687496 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55db8d2c1000 msgr2=0x55db8d2c8c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:38:51.792 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:51.788+0000 7f8944a00640 1 --2- 172.21.15.83:0/2513687496 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55db8d2c1000 0x55db8d2c8c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:38:51.792 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:51.788+0000 7f8944a00640 1 -- 172.21.15.83:0/2513687496 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55db8cf973c0 con 0x55db8d2cb8c0 2024-08-19T01:38:51.792 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:51.789+0000 7f8944a00640 1 --2- 172.21.15.83:0/2513687496 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55db8d2cb8c0 0x55db8d2c91a0 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f89380c4a10 tx=0x7f89380bbc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1441176f29cee983 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:38:51.792 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:51.789+0000 7f8929600640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-08-19T01:38:51.792 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:51.789+0000 7f8929600640 1 -- 172.21.15.83:0/2513687496 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89380bc2d0 con 0x55db8d2cb8c0 2024-08-19T01:38:51.792 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:51.789+0000 7f8949463740 1 -- 172.21.15.83:0/2513687496 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8930015070 con 0x55db8d2cb8c0 2024-08-19T01:38:51.792 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:51.790+0000 7f8929600640 1 -- 172.21.15.83:0/2513687496 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8269+0+0 (secure 0 0 0) 0x7f89380d5210 con 0x55db8d2cb8c0 2024-08-19T01:38:51.792 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:51.790+0000 7f8929600640 1 -- 172.21.15.83:0/2513687496 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f89380c0680 con 0x55db8d2cb8c0 2024-08-19T01:38:51.793 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:51.790+0000 7f8949463740 10 client.15645 fetch_fsmap learned FSMap version 27 2024-08-19T01:38:51.793 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:51.790+0000 7f8949463740 1 -- 172.21.15.83:0/2513687496 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55db8d2ce490 con 0x55db8d2cb8c0 2024-08-19T01:38:51.793 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:51.791+0000 7f8929600640 1 -- 172.21.15.83:0/2513687496 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8930015070 con 0x55db8d2cb8c0 2024-08-19T01:38:51.793 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:51.792+0000 7f8929600640 1 -- 172.21.15.83:0/2513687496 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f893804d060 con 0x55db8d2cb8c0 2024-08-19T01:38:51.794 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:51.793+0000 7f8949463740 10 client.15645 fetch_fsmap finished waiting for FSMap version 27 2024-08-19T01:38:51.794 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:51.793+0000 7f8949463740 10 client.15645 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.g 2024-08-19T01:38:51.794 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:51.793+0000 7f8949463740 20 client.15645 populate_metadata read hostname 'smithi083' 2024-08-19T01:38:51.795 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:51.793+0000 7f8949463740 1 --2- 172.21.15.83:0/2513687496 >> [v2:172.21.15.83:6838/3239261845,v1:172.21.15.83:6839/3239261845] conn(0x55db8d2db6f0 0x55db8d2ddb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:38:51.795 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:51.793+0000 7f8949463740 4 client.15645 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-19T01:38:51.795 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:51.793+0000 7f8949463740 1 -- 172.21.15.83:0/2513687496 --> [v2:172.21.15.83:6838/3239261845,v1:172.21.15.83:6839/3239261845] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55db8d233160 con 0x55db8d2db6f0 2024-08-19T01:38:51.795 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:51.794+0000 7f893fe00640 1 --2- 172.21.15.83:0/2513687496 >> [v2:172.21.15.83:6838/3239261845,v1:172.21.15.83:6839/3239261845] conn(0x55db8d2db6f0 0x55db8d2ddb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:38:51.796 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:51.795+0000 7f893fe00640 1 --2- 172.21.15.83:0/2513687496 >> [v2:172.21.15.83:6838/3239261845,v1:172.21.15.83:6839/3239261845] conn(0x55db8d2db6f0 0x55db8d2ddb20 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=b51e813bf6f5d651 server_cookie=8a2492fb25cb46cc in_seq=0 out_seq=0 2024-08-19T01:38:51.796 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:51.795+0000 7f8929600640 10 client.15645 ms_handle_connect on v2:172.21.15.83:6838/3239261845 2024-08-19T01:38:51.797 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:51.796+0000 7f8929600640 1 -- 172.21.15.83:0/2513687496 <== mds.1 v2:172.21.15.83:6838/3239261845 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55db8d233160 con 0x55db8d2db6f0 2024-08-19T01:38:51.798 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:51.796+0000 7f8929600640 10 client.15645 handle_command_reply: tid=0 2024-08-19T01:38:51.798 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:51.796+0000 7f8949463740 1 client.15645 shutdown 2024-08-19T01:38:51.798 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:51.797+0000 7f8949463740 1 -- 172.21.15.83:0/2513687496 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55db8d2cb8c0 msgr2=0x55db8d2c91a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:38:51.798 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:51.797+0000 7f8949463740 1 --2- 172.21.15.83:0/2513687496 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55db8d2cb8c0 0x55db8d2c91a0 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f89380c4a10 tx=0x7f89380bbc30 comp rx=0 tx=0).stop 2024-08-19T01:38:51.799 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:51.797+0000 7f8949463740 1 -- 172.21.15.83:0/2513687496 shutdown_connections 2024-08-19T01:38:51.799 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:51.797+0000 7f8949463740 1 --2- 172.21.15.83:0/2513687496 >> [v2:172.21.15.83:6838/3239261845,v1:172.21.15.83:6839/3239261845] conn(0x55db8d2db6f0 0x55db8d2ddb20 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:38:51.799 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:51.797+0000 7f8949463740 1 --2- 172.21.15.83:0/2513687496 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55db8d2cb8c0 0x55db8d2c91a0 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:38:51.799 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:51.797+0000 7f8949463740 1 --2- 172.21.15.83:0/2513687496 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55db8d2c1000 0x55db8d2c8c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:38:51.799 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:51.797+0000 7f8949463740 1 --2- 172.21.15.83:0/2513687496 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55db8d2c06e0 0x55db8d2c8720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:38:51.799 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:51.797+0000 7f8949463740 1 -- 172.21.15.83:0/2513687496 >> 172.21.15.83:0/2513687496 conn(0x55db8d2bcc70 msgr2=0x55db8d2c2950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:38:51.799 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:51.797+0000 7f8929600640 0 client.15645 ms_handle_reset on v2:172.21.15.83:6838/3239261845 2024-08-19T01:38:51.800 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:51.798+0000 7f8949463740 1 -- 172.21.15.83:0/2513687496 shutdown_connections 2024-08-19T01:38:51.800 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:51.798+0000 7f8949463740 1 -- 172.21.15.83:0/2513687496 wait complete. 2024-08-19T01:38:51.800 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:51.798+0000 7f8949463740 20 client.15645 trim_cache size 0 max 16384 2024-08-19T01:38:51.827 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:51.825+0000 7f8949463740 1 Processor -- start 2024-08-19T01:38:51.828 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:51.826+0000 7f8949463740 1 -- start start 2024-08-19T01:38:51.828 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:51.826+0000 7f8949463740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55db8d2c06e0 0x55db8d2b7910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:38:51.828 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:51.826+0000 7f8949463740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55db8d2c1000 0x55db8d2c8ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:38:51.828 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:51.826+0000 7f8949463740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55db8d2cb8c0 0x55db8d2e3400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:38:51.829 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:51.826+0000 7f8949463740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f89401be970 con 0x55db8d2c1000 2024-08-19T01:38:51.829 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:51.826+0000 7f8949463740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f89401bc240 con 0x55db8d2cb8c0 2024-08-19T01:38:51.829 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:51.826+0000 7f8949463740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f89401ba030 con 0x55db8d2c06e0 2024-08-19T01:38:51.829 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:51.827+0000 7f893fe00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55db8d2c06e0 0x55db8d2b7910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:38:51.829 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:51.827+0000 7f893fe00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55db8d2c06e0 0x55db8d2b7910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.83:52170/0 (socket says 172.21.15.83:52170) 2024-08-19T01:38:51.829 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:51.827+0000 7f893fe00640 1 -- 172.21.15.83:0/50399396 learned_addr learned my addr 172.21.15.83:0/50399396 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-08-19T01:38:51.829 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:51.828+0000 7f8944a00640 1 --2- 172.21.15.83:0/50399396 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55db8d2cb8c0 0x55db8d2e3400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:38:51.830 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:51.828+0000 7f8944a00640 1 -- 172.21.15.83:0/50399396 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55db8d2c06e0 msgr2=0x55db8d2b7910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:38:51.830 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:51.828+0000 7f8944a00640 1 --2- 172.21.15.83:0/50399396 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55db8d2c06e0 0x55db8d2b7910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:38:51.830 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:51.828+0000 7f8944a00640 1 -- 172.21.15.83:0/50399396 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55db8d2c1000 msgr2=0x55db8d2c8ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:38:51.830 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:51.828+0000 7f8944a00640 1 --2- 172.21.15.83:0/50399396 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55db8d2c1000 0x55db8d2c8ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:38:51.830 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:51.828+0000 7f8944a00640 1 -- 172.21.15.83:0/50399396 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55db8d2c7710 con 0x55db8d2cb8c0 2024-08-19T01:38:51.831 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:51.828+0000 7f8944a00640 1 --2- 172.21.15.83:0/50399396 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55db8d2cb8c0 0x55db8d2e3400 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f8938004af0 tx=0x7f89380ba7e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1be5a5e794480b55 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:38:51.831 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:51.829+0000 7f892be00640 1 -- 172.21.15.83:0/50399396 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89380c0680 con 0x55db8d2cb8c0 2024-08-19T01:38:51.831 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:51.829+0000 7f892be00640 1 -- 172.21.15.83:0/50399396 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f89380c4f40 con 0x55db8d2cb8c0 2024-08-19T01:38:51.831 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:51.829+0000 7f892be00640 1 -- 172.21.15.83:0/50399396 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8938004da0 con 0x55db8d2cb8c0 2024-08-19T01:38:51.832 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:51.830+0000 7f8949463740 1 -- 172.21.15.83:0/50399396 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55db8d2cb8c0 msgr2=0x55db8d2e3400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:38:51.832 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:51.830+0000 7f8949463740 1 --2- 172.21.15.83:0/50399396 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55db8d2cb8c0 0x55db8d2e3400 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f8938004af0 tx=0x7f89380ba7e0 comp rx=0 tx=0).stop 2024-08-19T01:38:51.832 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:51.830+0000 7f8949463740 1 -- 172.21.15.83:0/50399396 shutdown_connections 2024-08-19T01:38:51.832 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:51.830+0000 7f8949463740 1 --2- 172.21.15.83:0/50399396 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55db8d2cb8c0 0x55db8d2e3400 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:38:51.832 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:51.830+0000 7f8949463740 1 --2- 172.21.15.83:0/50399396 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55db8d2c1000 0x55db8d2c8ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:38:51.833 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:51.830+0000 7f8949463740 1 --2- 172.21.15.83:0/50399396 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55db8d2c06e0 0x55db8d2b7910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:38:51.833 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:51.830+0000 7f8949463740 1 -- 172.21.15.83:0/50399396 >> 172.21.15.83:0/50399396 conn(0x55db8d2bcc70 msgr2=0x55db8d2ca7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:38:51.833 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:51.830+0000 7f8949463740 1 -- 172.21.15.83:0/50399396 shutdown_connections 2024-08-19T01:38:51.833 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:51.831+0000 7f8949463740 1 -- 172.21.15.83:0/50399396 wait complete. 2024-08-19T01:38:51.833 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:51.831+0000 7f8949463740 1 -- start start 2024-08-19T01:38:51.833 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:51.831+0000 7f8949463740 1 Processor -- start 2024-08-19T01:38:51.834 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:51.832+0000 7f8949463740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55db8d2c06e0 0x55db8d2d3eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:38:51.834 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:51.832+0000 7f8949463740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55db8d2c1000 0x55db8d2c5320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:38:51.835 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:51.833+0000 7f8949463740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55db8d2cb8c0 0x55db8d2c5860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:38:51.835 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:51.833+0000 7f8949463740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f89401ba700 con 0x55db8d2c1000 2024-08-19T01:38:51.835 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:51.833+0000 7f893fe00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55db8d2c06e0 0x55db8d2d3eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:38:51.835 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:51.833+0000 7f8944a00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55db8d2cb8c0 0x55db8d2c5860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:38:51.835 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:51.833+0000 7f8944a00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55db8d2cb8c0 0x55db8d2c5860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.83:35320/0 (socket says 172.21.15.83:35320) 2024-08-19T01:38:51.835 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:51.833+0000 7f8944a00640 1 -- 172.21.15.83:0/1106857337 learned_addr learned my addr 172.21.15.83:0/1106857337 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-08-19T01:38:51.835 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:51.834+0000 7f8949463740 1 -- 172.21.15.83:0/1106857337 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f89401bbb70 con 0x55db8d2cb8c0 2024-08-19T01:38:51.835 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:51.834+0000 7f893f400640 1 --2- 172.21.15.83:0/1106857337 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55db8d2c1000 0x55db8d2c5320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:38:51.836 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:51.834+0000 7f8949463740 1 -- 172.21.15.83:0/1106857337 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f89401badd0 con 0x55db8d2c06e0 2024-08-19T01:38:51.836 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:51.835+0000 7f8944a00640 1 -- 172.21.15.83:0/1106857337 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55db8d2c06e0 msgr2=0x55db8d2d3eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:38:51.836 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:51.835+0000 7f8949463740 10 client.0 _ll_register_callbacks cb 0x7f89380bc150 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-19T01:38:51.836 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:51.835+0000 7f8944a00640 1 --2- 172.21.15.83:0/1106857337 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55db8d2c06e0 0x55db8d2d3eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:38:51.837 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:51.835+0000 7f8944a00640 1 -- 172.21.15.83:0/1106857337 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55db8d2c1000 msgr2=0x55db8d2c5320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:38:51.837 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:51.835+0000 7f8944a00640 1 --2- 172.21.15.83:0/1106857337 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55db8d2c1000 0x55db8d2c5320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:38:51.837 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:51.835+0000 7f8944a00640 1 -- 172.21.15.83:0/1106857337 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55db8d2c5f00 con 0x55db8d2cb8c0 2024-08-19T01:38:51.837 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:51.835+0000 7f893f400640 1 --2- 172.21.15.83:0/1106857337 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55db8d2c1000 0x55db8d2c5320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-19T01:38:51.838 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:51.835+0000 7f8949463740 1 -- 172.21.15.83:0/1106857337 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f89380c4f40 con 0x55db8d2cb8c0 2024-08-19T01:38:51.849 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:51.847+0000 7f8944a00640 1 --2- 172.21.15.83:0/1106857337 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55db8d2cb8c0 0x55db8d2c5860 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f89380c4eb0 tx=0x7f8938005f70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f2405559f24984a4 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:38:51.849 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:51.847+0000 7f8929600640 10 client.25697 ms_handle_connect on v2:172.21.15.117:3300/0 2024-08-19T01:38:51.849 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:51.848+0000 7f8929600640 1 -- 172.21.15.83:0/1106857337 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89380c4cb0 con 0x55db8d2cb8c0 2024-08-19T01:38:51.849 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:51.848+0000 7f8929600640 1 -- 172.21.15.83:0/1106857337 <== mon.2 v2:172.21.15.117:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8269+0+0 (secure 0 0 0) 0x7f89380dc0b0 con 0x55db8d2cb8c0 2024-08-19T01:38:51.850 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:51.848+0000 7f8929600640 1 -- 172.21.15.83:0/1106857337 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f89380dc2b0 con 0x55db8d2cb8c0 2024-08-19T01:38:51.850 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:51.848+0000 7f8949463740 10 client.25697 fetch_fsmap learned FSMap version 27 2024-08-19T01:38:51.850 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:51.848+0000 7f8949463740 1 -- 172.21.15.83:0/1106857337 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({fsmap=27}) -- 0x55db8d2c5f00 con 0x55db8d2cb8c0 2024-08-19T01:38:51.850 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:51.849+0000 7f8929600640 1 -- 172.21.15.83:0/1106857337 <== mon.2 v2:172.21.15.117:3300/0 4 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f89380bb720 con 0x55db8d2cb8c0 2024-08-19T01:38:51.851 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:51.849+0000 7f8949463740 10 client.25697 fetch_fsmap finished waiting for FSMap version 27 2024-08-19T01:38:51.851 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:51.849+0000 7f8949463740 10 client.25697 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.g 2024-08-19T01:38:51.851 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:51.850+0000 7f8949463740 20 client.25697 populate_metadata read hostname 'smithi083' 2024-08-19T01:38:51.851 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:51.850+0000 7f8949463740 1 --2- 172.21.15.83:0/1106857337 >> [v2:172.21.15.83:6838/3239261845,v1:172.21.15.83:6839/3239261845] conn(0x55db8d3462c0 0x55db8d3486b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:38:51.852 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:51.850+0000 7f8949463740 4 client.25697 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-08-19T01:38:51.852 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:51.850+0000 7f8949463740 1 -- 172.21.15.83:0/1106857337 --> [v2:172.21.15.83:6838/3239261845,v1:172.21.15.83:6839/3239261845] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55db8d22d670 con 0x55db8d3462c0 2024-08-19T01:38:51.852 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:51.851+0000 7f893fe00640 1 --2- 172.21.15.83:0/1106857337 >> [v2:172.21.15.83:6838/3239261845,v1:172.21.15.83:6839/3239261845] conn(0x55db8d3462c0 0x55db8d3486b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:38:51.853 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:51.851+0000 7f893fe00640 1 --2- 172.21.15.83:0/1106857337 >> [v2:172.21.15.83:6838/3239261845,v1:172.21.15.83:6839/3239261845] conn(0x55db8d3462c0 0x55db8d3486b0 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=ab7f2d5c26fbedb2 server_cookie=d812c0c90e16033a in_seq=0 out_seq=0 2024-08-19T01:38:51.853 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:51.852+0000 7f8929600640 10 client.25697 ms_handle_connect on v2:172.21.15.83:6838/3239261845 2024-08-19T01:38:51.854 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:51.852+0000 7f8929600640 1 -- 172.21.15.83:0/1106857337 <== mds.1 v2:172.21.15.83:6838/3239261845 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55db8d22d670 con 0x55db8d3462c0 2024-08-19T01:38:51.854 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:51.852+0000 7f8929600640 10 client.25697 handle_command_reply: tid=0 2024-08-19T01:38:51.854 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:51.852+0000 7f8949463740 1 client.25697 shutdown 2024-08-19T01:38:51.854 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:51.853+0000 7f8949463740 1 -- 172.21.15.83:0/1106857337 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55db8d2cb8c0 msgr2=0x55db8d2c5860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:38:51.855 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:51.853+0000 7f8949463740 1 --2- 172.21.15.83:0/1106857337 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55db8d2cb8c0 0x55db8d2c5860 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f89380c4eb0 tx=0x7f8938005f70 comp rx=0 tx=0).stop 2024-08-19T01:38:51.856 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:51.853+0000 7f8949463740 1 -- 172.21.15.83:0/1106857337 shutdown_connections 2024-08-19T01:38:51.856 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:51.853+0000 7f8949463740 1 --2- 172.21.15.83:0/1106857337 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55db8d2cb8c0 0x55db8d2c5860 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:38:51.856 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:51.853+0000 7f8949463740 1 --2- 172.21.15.83:0/1106857337 >> [v2:172.21.15.83:6838/3239261845,v1:172.21.15.83:6839/3239261845] conn(0x55db8d3462c0 0x55db8d3486b0 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:38:51.856 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:51.853+0000 7f8949463740 1 --2- 172.21.15.83:0/1106857337 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55db8d2c1000 0x55db8d2c5320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:38:51.856 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:51.853+0000 7f8949463740 1 --2- 172.21.15.83:0/1106857337 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55db8d2c06e0 0x55db8d2d3eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:38:51.856 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:51.853+0000 7f8949463740 1 -- 172.21.15.83:0/1106857337 >> 172.21.15.83:0/1106857337 conn(0x55db8d2bcc70 msgr2=0x55db8d2bb480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:38:51.856 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:51.854+0000 7f8929600640 0 client.25697 ms_handle_reset on v2:172.21.15.83:6838/3239261845 2024-08-19T01:38:51.856 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:51.854+0000 7f8949463740 1 -- 172.21.15.83:0/1106857337 shutdown_connections 2024-08-19T01:38:51.856 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:51.854+0000 7f8949463740 1 -- 172.21.15.83:0/1106857337 wait complete. 2024-08-19T01:38:51.856 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:51.854+0000 7f8949463740 20 client.25697 trim_cache size 0 max 16384 2024-08-19T01:38:51.859 INFO:teuthology.orchestra.run.smithi083.stdout:[] 2024-08-19T01:38:51.860 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:51.858+0000 7f8945400640 1 -- 172.21.15.83:0/3480776426 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f89240a2cc0 msgr2=0x7f89240a5180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:38:51.860 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:51.858+0000 7f8945400640 1 --2- 172.21.15.83:0/3480776426 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f89240a2cc0 0x7f89240a5180 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x55db8d2bdad0 tx=0x7f892c005ed0 comp rx=0 tx=0).stop 2024-08-19T01:38:51.860 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:51.858+0000 7f8945400640 1 -- 172.21.15.83:0/3480776426 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f89401076f0 msgr2=0x7f894011bb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:38:51.860 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:51.858+0000 7f8945400640 1 --2- 172.21.15.83:0/3480776426 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f89401076f0 0x7f894011bb30 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f8938034040 tx=0x7f8938003f10 comp rx=0 tx=0).stop 2024-08-19T01:38:51.861 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:51.859+0000 7f8945400640 1 -- 172.21.15.83:0/3480776426 shutdown_connections 2024-08-19T01:38:51.861 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:51.859+0000 7f8945400640 1 --2- 172.21.15.83:0/3480776426 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f89240a2cc0 0x7f89240a5180 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:38:51.861 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:51.859+0000 7f8945400640 1 --2- 172.21.15.83:0/3480776426 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f89401076f0 0x7f894011bb30 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:38:51.861 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:51.859+0000 7f8945400640 1 --2- 172.21.15.83:0/3480776426 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8940102090 0x7f8940116350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:38:51.861 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:51.859+0000 7f8945400640 1 --2- 172.21.15.83:0/3480776426 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f89401016f0 0x7f8940115e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:38:51.862 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:51.859+0000 7f8945400640 1 -- 172.21.15.83:0/3480776426 >> 172.21.15.83:0/3480776426 conn(0x7f89400fd470 msgr2=0x7f89400fdcc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:38:51.862 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:51.859+0000 7f8945400640 1 -- 172.21.15.83:0/3480776426 shutdown_connections 2024-08-19T01:38:51.862 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:51.860+0000 7f8945400640 1 -- 172.21.15.83:0/3480776426 wait complete. 2024-08-19T01:38:51.952 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:38:51 smithi117 ceph-mon[22016]: pgmap v459: 97 pgs: 97 active+clean; 332 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 4.6 MiB/s wr, 1.22k op/s 2024-08-19T01:38:51.965 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:38:51 smithi167 ceph-mon[21875]: pgmap v459: 97 pgs: 97 active+clean; 332 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 4.6 MiB/s wr, 1.22k op/s 2024-08-19T01:38:53.229 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833 shell --fsid bad8780e-5dc9-11ef-bccf-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-08-19T01:38:53.905 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:38:53 smithi083 ceph-mon[17120]: pgmap v460: 97 pgs: 97 active+clean; 333 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 5.1 MiB/s wr, 1.28k op/s 2024-08-19T01:38:53.952 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:38:53 smithi117 ceph-mon[22016]: pgmap v460: 97 pgs: 97 active+clean; 333 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 5.1 MiB/s wr, 1.28k op/s 2024-08-19T01:38:53.965 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:38:53 smithi167 ceph-mon[21875]: pgmap v460: 97 pgs: 97 active+clean; 333 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 5.1 MiB/s wr, 1.28k op/s 2024-08-19T01:38:54.349 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/bad8780e-5dc9-11ef-bccf-c7b262605968/mon.a/config 2024-08-19T01:38:55.464 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:55.463+0000 7f7cb9e00640 1 -- 172.21.15.83:0/1838121184 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7cb410bcf0 msgr2=0x7f7cb410e190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:38:55.464 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:55.463+0000 7f7cb9e00640 1 --2- 172.21.15.83:0/1838121184 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7cb410bcf0 0x7f7cb410e190 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f7cac009be0 tx=0x7f7cac02a750 comp rx=0 tx=0).stop 2024-08-19T01:38:55.465 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:55.463+0000 7f7cb9e00640 1 -- 172.21.15.83:0/1838121184 shutdown_connections 2024-08-19T01:38:55.465 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:55.463+0000 7f7cb9e00640 1 --2- 172.21.15.83:0/1838121184 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7cb410bcf0 0x7f7cb410e190 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:38:55.465 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:55.463+0000 7f7cb9e00640 1 --2- 172.21.15.83:0/1838121184 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7cb41031e0 0x7f7cb410b6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:38:55.465 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:55.463+0000 7f7cb9e00640 1 --2- 172.21.15.83:0/1838121184 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7cb41028c0 0x7f7cb4102ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:38:55.465 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:55.463+0000 7f7cb9e00640 1 -- 172.21.15.83:0/1838121184 >> 172.21.15.83:0/1838121184 conn(0x7f7cb40fc340 msgr2=0x7f7cb40fe760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:38:55.465 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:55.463+0000 7f7cb9e00640 1 -- 172.21.15.83:0/1838121184 shutdown_connections 2024-08-19T01:38:55.465 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:55.463+0000 7f7cb9e00640 1 -- 172.21.15.83:0/1838121184 wait complete. 2024-08-19T01:38:55.466 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:55.466+0000 7f7cb9e00640 1 Processor -- start 2024-08-19T01:38:55.466 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:55.466+0000 7f7cb9e00640 1 -- start start 2024-08-19T01:38:55.468 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:55.467+0000 7f7cb9e00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7cb41028c0 0x7f7cb4100520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:38:55.469 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:55.467+0000 7f7cb9e00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7cb41031e0 0x7f7cb4100a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:38:55.469 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:55.467+0000 7f7cb9e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7cb410bcf0 0x7f7cb4100fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:38:55.469 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:55.467+0000 7f7cb9e00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f7cb4111af0 con 0x7f7cb410bcf0 2024-08-19T01:38:55.469 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:55.467+0000 7f7cb9e00640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f7cb4111970 con 0x7f7cb41028c0 2024-08-19T01:38:55.469 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:55.467+0000 7f7cb8a00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7cb41028c0 0x7f7cb4100520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:38:55.469 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:55.467+0000 7f7cb8a00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7cb41028c0 0x7f7cb4100520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.83:35334/0 (socket says 172.21.15.83:35334) 2024-08-19T01:38:55.469 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:55.467+0000 7f7cb8a00640 1 -- 172.21.15.83:0/2058632191 learned_addr learned my addr 172.21.15.83:0/2058632191 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-08-19T01:38:55.469 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:55.467+0000 7f7cb9e00640 1 -- 172.21.15.83:0/2058632191 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f7cb4111c70 con 0x7f7cb41031e0 2024-08-19T01:38:55.469 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:55.467+0000 7f7cb3e00640 1 --2- 172.21.15.83:0/2058632191 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7cb41031e0 0x7f7cb4100a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:38:55.469 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:55.468+0000 7f7cb9400640 1 --2- 172.21.15.83:0/2058632191 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7cb410bcf0 0x7f7cb4100fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:38:55.469 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:55.468+0000 7f7cb3e00640 1 -- 172.21.15.83:0/2058632191 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7cb41028c0 msgr2=0x7f7cb4100520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:38:55.469 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:55.468+0000 7f7cb3e00640 1 --2- 172.21.15.83:0/2058632191 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7cb41028c0 0x7f7cb4100520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:38:55.470 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:55.468+0000 7f7cb3e00640 1 -- 172.21.15.83:0/2058632191 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7cb410bcf0 msgr2=0x7f7cb4100fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:38:55.470 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:55.468+0000 7f7cb3e00640 1 --2- 172.21.15.83:0/2058632191 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7cb410bcf0 0x7f7cb4100fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:38:55.470 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:55.468+0000 7f7cb3e00640 1 -- 172.21.15.83:0/2058632191 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7cb41a6410 con 0x7f7cb41031e0 2024-08-19T01:38:55.470 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:55.468+0000 7f7cb9400640 1 --2- 172.21.15.83:0/2058632191 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7cb410bcf0 0x7f7cb4100fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-19T01:38:55.470 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:55.468+0000 7f7cb3e00640 1 --2- 172.21.15.83:0/2058632191 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7cb41031e0 0x7f7cb4100a60 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f7ca000e6c0 tx=0x7f7ca000eb80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:38:55.470 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:55.468+0000 7f7cb1600640 1 -- 172.21.15.83:0/2058632191 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ca000cab0 con 0x7f7cb41031e0 2024-08-19T01:38:55.471 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:55.468+0000 7f7cb1600640 1 -- 172.21.15.83:0/2058632191 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7ca000cc50 con 0x7f7cb41031e0 2024-08-19T01:38:55.471 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:55.469+0000 7f7cb9e00640 1 -- 172.21.15.83:0/2058632191 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7cb41a6700 con 0x7f7cb41031e0 2024-08-19T01:38:55.471 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:55.469+0000 7f7cb9e00640 1 -- 172.21.15.83:0/2058632191 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7cb41a6c40 con 0x7f7cb41031e0 2024-08-19T01:38:55.471 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:55.469+0000 7f7cb1600640 1 -- 172.21.15.83:0/2058632191 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ca0018840 con 0x7f7cb41031e0 2024-08-19T01:38:55.473 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:55.472+0000 7f7cbdbfd740 1 Processor -- start 2024-08-19T01:38:55.473 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:55.472+0000 7f7cbdbfd740 1 -- start start 2024-08-19T01:38:55.473 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:55.473+0000 7f7cbdbfd740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ac33711ef0 0x55ac337122d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:38:55.474 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:55.473+0000 7f7cbdbfd740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ac33712810 0x55ac3371cb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:38:55.474 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:55.473+0000 7f7cbdbfd740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ac3371d0d0 0x55ac3371f4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:38:55.474 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:55.473+0000 7f7cbdbfd740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55ac3366ebf0 con 0x55ac3371d0d0 2024-08-19T01:38:55.474 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:55.473+0000 7f7cbdbfd740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55ac33606b20 con 0x55ac33711ef0 2024-08-19T01:38:55.474 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:55.473+0000 7f7cbdbfd740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55ac3366abb0 con 0x55ac33712810 2024-08-19T01:38:55.474 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:55.473+0000 7f7cb1600640 1 -- 172.21.15.83:0/2058632191 <== mon.1 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7f7ca0010430 con 0x7f7cb41031e0 2024-08-19T01:38:55.474 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:55.473+0000 7f7cb9400640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ac33711ef0 0x55ac337122d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:38:55.475 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:55.473+0000 7f7cb9400640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ac33711ef0 0x55ac337122d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.83:35338/0 (socket says 172.21.15.83:35338) 2024-08-19T01:38:55.475 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:55.473+0000 7f7cb9400640 1 -- 172.21.15.83:0/1881655711 learned_addr learned my addr 172.21.15.83:0/1881655711 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-08-19T01:38:55.475 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:55.473+0000 7f7cb8a00640 1 --2- 172.21.15.83:0/1881655711 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ac33712810 0x55ac3371cb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:38:55.475 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:55.474+0000 7f7cb3e00640 1 --2- 172.21.15.83:0/1881655711 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ac3371d0d0 0x55ac3371f4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:38:55.475 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:55.474+0000 7f7cb8a00640 1 -- 172.21.15.83:0/1881655711 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ac33711ef0 msgr2=0x55ac337122d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:38:55.476 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:55.474+0000 7f7cb8a00640 1 --2- 172.21.15.83:0/1881655711 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ac33711ef0 0x55ac337122d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:38:55.476 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:55.474+0000 7f7cb8a00640 1 -- 172.21.15.83:0/1881655711 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ac3371d0d0 msgr2=0x55ac3371f4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:38:55.476 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:55.474+0000 7f7cb8a00640 1 --2- 172.21.15.83:0/1881655711 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ac3371d0d0 0x55ac3371f4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:38:55.476 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:55.474+0000 7f7cb8a00640 1 -- 172.21.15.83:0/1881655711 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ac3360d7f0 con 0x55ac33712810 2024-08-19T01:38:55.476 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:55.475+0000 7f7cb8a00640 1 --2- 172.21.15.83:0/1881655711 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ac33712810 0x55ac3371cb90 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f7ca8009c80 tx=0x7f7ca8011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=de2801dab0870eac server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:38:55.476 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:55.475+0000 7f7ca5c00640 1 -- 172.21.15.83:0/1881655711 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ca8004230 con 0x55ac33712810 2024-08-19T01:38:55.476 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:55.475+0000 7f7ca5c00640 1 -- 172.21.15.83:0/1881655711 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7ca80043d0 con 0x55ac33712810 2024-08-19T01:38:55.477 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:55.475+0000 7f7ca5c00640 1 -- 172.21.15.83:0/1881655711 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ca8004df0 con 0x55ac33712810 2024-08-19T01:38:55.478 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:55.476+0000 7f7cbdbfd740 1 -- 172.21.15.83:0/1881655711 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ac33712810 msgr2=0x55ac3371cb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:38:55.478 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:55.476+0000 7f7cbdbfd740 1 --2- 172.21.15.83:0/1881655711 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ac33712810 0x55ac3371cb90 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f7ca8009c80 tx=0x7f7ca8011810 comp rx=0 tx=0).stop 2024-08-19T01:38:55.478 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:55.476+0000 7f7cbdbfd740 1 -- 172.21.15.83:0/1881655711 shutdown_connections 2024-08-19T01:38:55.478 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:55.476+0000 7f7cbdbfd740 1 --2- 172.21.15.83:0/1881655711 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ac3371d0d0 0x55ac3371f4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:38:55.478 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:55.476+0000 7f7cbdbfd740 1 --2- 172.21.15.83:0/1881655711 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ac33712810 0x55ac3371cb90 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:38:55.478 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:55.476+0000 7f7cbdbfd740 1 --2- 172.21.15.83:0/1881655711 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ac33711ef0 0x55ac337122d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:38:55.479 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:55.476+0000 7f7cbdbfd740 1 -- 172.21.15.83:0/1881655711 >> 172.21.15.83:0/1881655711 conn(0x55ac3370e480 msgr2=0x55ac3370e860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:38:55.479 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:55.477+0000 7f7cb1600640 1 --2- 172.21.15.83:0/2058632191 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f7c9c0a2c00 0x7f7c9c0a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:38:55.479 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:55.477+0000 7f7cbdbfd740 1 -- 172.21.15.83:0/1881655711 shutdown_connections 2024-08-19T01:38:55.479 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:55.477+0000 7f7cbdbfd740 1 -- 172.21.15.83:0/1881655711 wait complete. 2024-08-19T01:38:55.479 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:55.477+0000 7f7cb1600640 1 -- 172.21.15.83:0/2058632191 <== mon.1 v2:172.21.15.167:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8269+0+0 (secure 0 0 0) 0x7f7ca00cf080 con 0x7f7cb41031e0 2024-08-19T01:38:55.479 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:55.477+0000 7f7cb8a00640 1 --2- 172.21.15.83:0/2058632191 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f7c9c0a2c00 0x7f7c9c0a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:38:55.479 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:55.477+0000 7f7cbdbfd740 1 -- start start 2024-08-19T01:38:55.479 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:55.478+0000 7f7cbdbfd740 1 Processor -- start 2024-08-19T01:38:55.480 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:55.478+0000 7f7cbdbfd740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ac33711ef0 0x55ac33719f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:38:55.480 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:55.478+0000 7f7cbdbfd740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ac33712810 0x55ac3371a4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:38:55.482 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:55.478+0000 7f7cbdbfd740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ac3371d0d0 0x55ac3371aa00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:38:55.482 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:55.479+0000 7f7cbdbfd740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55ac3371fca0 con 0x55ac33711ef0 2024-08-19T01:38:55.482 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:55.479+0000 7f7cbdbfd740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55ac33629cf0 con 0x55ac33712810 2024-08-19T01:38:55.482 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:55.479+0000 7f7cbdbfd740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55ac3371fe20 con 0x55ac3371d0d0 2024-08-19T01:38:55.482 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:55.479+0000 7f7cbdbfd740 10 client.0 _ll_register_callbacks cb 0x55ac33615e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-19T01:38:55.482 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:55.479+0000 7f7cb8a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ac33711ef0 0x55ac33719f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:38:55.483 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:55.479+0000 7f7cb3e00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ac33712810 0x55ac3371a4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:38:55.483 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:55.479+0000 7f7cb3e00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ac33712810 0x55ac3371a4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.83:35350/0 (socket says 172.21.15.83:35350) 2024-08-19T01:38:55.483 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:55.479+0000 7f7cb8a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ac33711ef0 0x55ac33719f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:33472/0 (socket says 172.21.15.83:33472) 2024-08-19T01:38:55.483 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:55.479+0000 7f7cb3e00640 1 -- 172.21.15.83:0/2390506636 learned_addr learned my addr 172.21.15.83:0/2390506636 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-08-19T01:38:55.483 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:55.479+0000 7f7cb9400640 1 --2- 172.21.15.83:0/2390506636 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ac3371d0d0 0x55ac3371aa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:38:55.483 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:55.479+0000 7f7cb8a00640 1 --2- 172.21.15.83:0/2058632191 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f7c9c0a2c00 0x7f7c9c0a50c0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f7ca8009620 tx=0x7f7ca8004950 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:38:55.483 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:55.480+0000 7f7cb9400640 1 -- 172.21.15.83:0/2390506636 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ac33712810 msgr2=0x55ac3371a4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:38:55.483 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:55.480+0000 7f7cb9400640 1 --2- 172.21.15.83:0/2390506636 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ac33712810 0x55ac3371a4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:38:55.483 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:55.480+0000 7f7cb9400640 1 -- 172.21.15.83:0/2390506636 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ac33711ef0 msgr2=0x55ac33719f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:38:55.483 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:55.480+0000 7f7cb9400640 1 --2- 172.21.15.83:0/2390506636 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ac33711ef0 0x55ac33719f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:38:55.483 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:55.480+0000 7f7cb9400640 1 -- 172.21.15.83:0/2390506636 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ac333e03c0 con 0x55ac3371d0d0 2024-08-19T01:38:55.484 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:55.480+0000 7f7cb9400640 1 --2- 172.21.15.83:0/2390506636 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ac3371d0d0 0x55ac3371aa00 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f7cac034040 tx=0x7f7cac030fb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2fc078c2cb6f65a6 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:38:55.484 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:55.480+0000 7f7c9a000640 10 client.0 ms_handle_connect on v2:172.21.15.167:3300/0 2024-08-19T01:38:55.484 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:55.481+0000 7f7c9a000640 1 -- 172.21.15.83:0/2390506636 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7cac042070 con 0x55ac3371d0d0 2024-08-19T01:38:55.484 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:55.481+0000 7f7c9a000640 1 -- 172.21.15.83:0/2390506636 <== mon.1 v2:172.21.15.167:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8269+0+0 (secure 0 0 0) 0x7f7cac0307b0 con 0x55ac3371d0d0 2024-08-19T01:38:55.484 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:55.481+0000 7f7cbdbfd740 1 -- 172.21.15.83:0/2390506636 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7ca80043d0 con 0x55ac3371d0d0 2024-08-19T01:38:55.484 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:55.481+0000 7f7c9a000640 1 -- 172.21.15.83:0/2390506636 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f7cac04c050 con 0x55ac3371d0d0 2024-08-19T01:38:55.484 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:55.481+0000 7f7cbdbfd740 10 client.25654 fetch_fsmap learned FSMap version 27 2024-08-19T01:38:55.484 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:55.481+0000 7f7cbdbfd740 1 -- 172.21.15.83:0/2390506636 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55ac3372b990 con 0x55ac3371d0d0 2024-08-19T01:38:55.485 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:55.482+0000 7f7c9a000640 1 -- 172.21.15.83:0/2390506636 <== mon.1 v2:172.21.15.167:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7ca80043d0 con 0x55ac3371d0d0 2024-08-19T01:38:55.485 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:55.482+0000 7f7c9a000640 1 -- 172.21.15.83:0/2390506636 <== mon.1 v2:172.21.15.167:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f7cac03c040 con 0x55ac3371d0d0 2024-08-19T01:38:55.485 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:55.482+0000 7f7cbdbfd740 10 client.25654 fetch_fsmap finished waiting for FSMap version 27 2024-08-19T01:38:55.486 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:55.482+0000 7f7cbdbfd740 10 client.25654 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-19T01:38:55.486 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:55.482+0000 7f7cbdbfd740 20 client.25654 populate_metadata read hostname 'smithi083' 2024-08-19T01:38:55.486 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:55.482+0000 7f7cbdbfd740 1 --2- 172.21.15.83:0/2390506636 >> [v2:172.21.15.117:6838/1967595704,v1:172.21.15.117:6839/1967595704] conn(0x55ac3372cef0 0x55ac3372f340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:38:55.486 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:55.482+0000 7f7cbdbfd740 4 client.25654 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-19T01:38:55.486 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:55.482+0000 7f7cbdbfd740 1 -- 172.21.15.83:0/2390506636 --> [v2:172.21.15.117:6838/1967595704,v1:172.21.15.117:6839/1967595704] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ac3367c160 con 0x55ac3372cef0 2024-08-19T01:38:55.486 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:55.482+0000 7f7cb8a00640 1 --2- 172.21.15.83:0/2390506636 >> [v2:172.21.15.117:6838/1967595704,v1:172.21.15.117:6839/1967595704] conn(0x55ac3372cef0 0x55ac3372f340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:38:55.486 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:55.483+0000 7f7c9a000640 10 client.25654 ms_handle_connect on v2:172.21.15.117:6838/1967595704 2024-08-19T01:38:55.486 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:55.484+0000 7f7cb8a00640 1 --2- 172.21.15.83:0/2390506636 >> [v2:172.21.15.117:6838/1967595704,v1:172.21.15.117:6839/1967595704] conn(0x55ac3372cef0 0x55ac3372f340 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9535ecd439c99786 server_cookie=14ebfa6acd5dc9f in_seq=0 out_seq=0 2024-08-19T01:38:55.488 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:55.488+0000 7f7c9a000640 1 -- 172.21.15.83:0/2390506636 <== mds.0 v2:172.21.15.117:6838/1967595704 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55ac3367c160 con 0x55ac3372cef0 2024-08-19T01:38:55.488 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:55.488+0000 7f7c9a000640 10 client.25654 handle_command_reply: tid=0 2024-08-19T01:38:55.489 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:55.488+0000 7f7cbdbfd740 1 client.25654 shutdown 2024-08-19T01:38:55.490 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:55.488+0000 7f7cbdbfd740 1 -- 172.21.15.83:0/2390506636 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ac3371d0d0 msgr2=0x55ac3371aa00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:38:55.490 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:55.488+0000 7f7cbdbfd740 1 --2- 172.21.15.83:0/2390506636 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ac3371d0d0 0x55ac3371aa00 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f7cac034040 tx=0x7f7cac030fb0 comp rx=0 tx=0).stop 2024-08-19T01:38:55.490 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:55.488+0000 7f7cbdbfd740 1 -- 172.21.15.83:0/2390506636 shutdown_connections 2024-08-19T01:38:55.490 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:55.488+0000 7f7cbdbfd740 1 --2- 172.21.15.83:0/2390506636 >> [v2:172.21.15.117:6838/1967595704,v1:172.21.15.117:6839/1967595704] conn(0x55ac3372cef0 0x55ac3372f340 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:38:55.490 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:55.488+0000 7f7cbdbfd740 1 --2- 172.21.15.83:0/2390506636 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ac3371d0d0 0x55ac3371aa00 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:38:55.490 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:55.488+0000 7f7cbdbfd740 1 --2- 172.21.15.83:0/2390506636 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ac33712810 0x55ac3371a4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:38:55.491 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:55.489+0000 7f7c9a000640 0 client.25654 ms_handle_reset on v2:172.21.15.117:6838/1967595704 2024-08-19T01:38:55.491 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:55.488+0000 7f7cbdbfd740 1 --2- 172.21.15.83:0/2390506636 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ac33711ef0 0x55ac33719f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:38:55.491 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:55.489+0000 7f7cbdbfd740 1 -- 172.21.15.83:0/2390506636 >> 172.21.15.83:0/2390506636 conn(0x55ac3370e480 msgr2=0x55ac33714160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:38:55.491 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:55.489+0000 7f7cbdbfd740 1 -- 172.21.15.83:0/2390506636 shutdown_connections 2024-08-19T01:38:55.491 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:55.489+0000 7f7cbdbfd740 1 -- 172.21.15.83:0/2390506636 wait complete. 2024-08-19T01:38:55.491 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:55.490+0000 7f7cbdbfd740 20 client.25654 trim_cache size 0 max 16384 2024-08-19T01:38:55.519 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:55.518+0000 7f7cbdbfd740 1 Processor -- start 2024-08-19T01:38:55.519 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:55.518+0000 7f7cbdbfd740 1 -- start start 2024-08-19T01:38:55.519 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:55.518+0000 7f7cbdbfd740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ac33711ef0 0x55ac33714950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:38:55.519 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:55.518+0000 7f7cbdbfd740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ac33712810 0x55ac33734c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:38:55.519 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:55.518+0000 7f7cbdbfd740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ac3371d0d0 0x55ac33735190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:38:55.519 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:55.518+0000 7f7cbdbfd740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f7c94010dc0 con 0x55ac33712810 2024-08-19T01:38:55.519 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:55.518+0000 7f7cbdbfd740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f7c9400e690 con 0x55ac33711ef0 2024-08-19T01:38:55.520 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:55.518+0000 7f7cbdbfd740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f7c9400c4e0 con 0x55ac3371d0d0 2024-08-19T01:38:55.520 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:55.519+0000 7f7cb3e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ac33712810 0x55ac33734c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:38:55.520 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:55.519+0000 7f7cb3e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ac33712810 0x55ac33734c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:33486/0 (socket says 172.21.15.83:33486) 2024-08-19T01:38:55.520 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:55.519+0000 7f7cb3e00640 1 -- 172.21.15.83:0/1272355016 learned_addr learned my addr 172.21.15.83:0/1272355016 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-08-19T01:38:55.520 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:55.519+0000 7f7cb8a00640 1 --2- 172.21.15.83:0/1272355016 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ac33711ef0 0x55ac33714950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:38:55.521 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:55.519+0000 7f7cb9400640 1 --2- 172.21.15.83:0/1272355016 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ac3371d0d0 0x55ac33735190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:38:55.521 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:55.519+0000 7f7cb9400640 1 -- 172.21.15.83:0/1272355016 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ac33711ef0 msgr2=0x55ac33714950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:38:55.521 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:55.519+0000 7f7cb9400640 1 --2- 172.21.15.83:0/1272355016 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ac33711ef0 0x55ac33714950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:38:55.521 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:55.519+0000 7f7cb9400640 1 -- 172.21.15.83:0/1272355016 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ac33712810 msgr2=0x55ac33734c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:38:55.521 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:55.519+0000 7f7cb9400640 1 --2- 172.21.15.83:0/1272355016 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ac33712810 0x55ac33734c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:38:55.521 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:55.520+0000 7f7cb9400640 1 -- 172.21.15.83:0/1272355016 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ac33715ba0 con 0x55ac3371d0d0 2024-08-19T01:38:55.521 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:55.520+0000 7f7cb9400640 1 --2- 172.21.15.83:0/1272355016 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ac3371d0d0 0x55ac33735190 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f7cac006010 tx=0x7f7cac009280 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8441ea2a75e3c2fe server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:38:55.522 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:55.520+0000 7f7ca5200640 1 -- 172.21.15.83:0/1272355016 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7cac042070 con 0x55ac3371d0d0 2024-08-19T01:38:55.522 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:55.520+0000 7f7ca5200640 1 -- 172.21.15.83:0/1272355016 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7cac009800 con 0x55ac3371d0d0 2024-08-19T01:38:55.522 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:55.520+0000 7f7ca5200640 1 -- 172.21.15.83:0/1272355016 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7cac036040 con 0x55ac3371d0d0 2024-08-19T01:38:55.522 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:55.521+0000 7f7cbdbfd740 1 -- 172.21.15.83:0/1272355016 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ac3371d0d0 msgr2=0x55ac33735190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:38:55.522 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:55.521+0000 7f7cbdbfd740 1 --2- 172.21.15.83:0/1272355016 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ac3371d0d0 0x55ac33735190 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f7cac006010 tx=0x7f7cac009280 comp rx=0 tx=0).stop 2024-08-19T01:38:55.522 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:55.521+0000 7f7cbdbfd740 1 -- 172.21.15.83:0/1272355016 shutdown_connections 2024-08-19T01:38:55.522 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:55.521+0000 7f7cbdbfd740 1 --2- 172.21.15.83:0/1272355016 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ac3371d0d0 0x55ac33735190 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:38:55.522 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:55.521+0000 7f7cbdbfd740 1 --2- 172.21.15.83:0/1272355016 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ac33712810 0x55ac33734c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:38:55.522 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:55.521+0000 7f7cbdbfd740 1 --2- 172.21.15.83:0/1272355016 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ac33711ef0 0x55ac33714950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:38:55.522 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:55.521+0000 7f7cbdbfd740 1 -- 172.21.15.83:0/1272355016 >> 172.21.15.83:0/1272355016 conn(0x55ac3370e480 msgr2=0x55ac33718870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:38:55.524 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:55.522+0000 7f7cbdbfd740 1 -- 172.21.15.83:0/1272355016 shutdown_connections 2024-08-19T01:38:55.524 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:55.522+0000 7f7cbdbfd740 1 -- 172.21.15.83:0/1272355016 wait complete. 2024-08-19T01:38:55.524 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:55.522+0000 7f7cbdbfd740 1 -- start start 2024-08-19T01:38:55.524 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:55.522+0000 7f7cbdbfd740 1 Processor -- start 2024-08-19T01:38:55.524 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:55.523+0000 7f7cbdbfd740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ac33711ef0 0x55ac33725930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:38:55.524 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:55.523+0000 7f7cbdbfd740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ac33712810 0x55ac33716df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:38:55.524 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:55.523+0000 7f7cbdbfd740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ac3371d0d0 0x55ac337173e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:38:55.524 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:55.523+0000 7f7cbdbfd740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f7c9400cb50 con 0x55ac33711ef0 2024-08-19T01:38:55.524 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:55.523+0000 7f7cbdbfd740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f7c9400dfc0 con 0x55ac3371d0d0 2024-08-19T01:38:55.524 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:55.523+0000 7f7cbdbfd740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f7c9400d220 con 0x55ac33712810 2024-08-19T01:38:55.524 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:55.523+0000 7f7cbdbfd740 10 client.0 _ll_register_callbacks cb 0x7f7ca8005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-19T01:38:55.525 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:55.524+0000 7f7cb8a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ac33711ef0 0x55ac33725930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:38:55.525 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:55.524+0000 7f7cb8a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ac33711ef0 0x55ac33725930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:33500/0 (socket says 172.21.15.83:33500) 2024-08-19T01:38:55.526 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:55.524+0000 7f7cb8a00640 1 -- 172.21.15.83:0/3268275556 learned_addr learned my addr 172.21.15.83:0/3268275556 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-08-19T01:38:55.526 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:55.524+0000 7f7cb9400640 1 --2- 172.21.15.83:0/3268275556 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ac3371d0d0 0x55ac337173e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:38:55.526 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:55.524+0000 7f7cb3e00640 1 --2- 172.21.15.83:0/3268275556 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ac33712810 0x55ac33716df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:38:55.526 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:55.524+0000 7f7cb3e00640 1 -- 172.21.15.83:0/3268275556 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ac3371d0d0 msgr2=0x55ac337173e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:38:55.526 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:55.524+0000 7f7cb3e00640 1 --2- 172.21.15.83:0/3268275556 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ac3371d0d0 0x55ac337173e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:38:55.526 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:55.524+0000 7f7cb3e00640 1 -- 172.21.15.83:0/3268275556 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ac33711ef0 msgr2=0x55ac33725930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:38:55.526 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:55.524+0000 7f7cb3e00640 1 --2- 172.21.15.83:0/3268275556 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ac33711ef0 0x55ac33725930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:38:55.526 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:55.524+0000 7f7cb3e00640 1 -- 172.21.15.83:0/3268275556 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ac33717c10 con 0x55ac33712810 2024-08-19T01:38:55.526 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:55.524+0000 7f7cb8a00640 1 --2- 172.21.15.83:0/3268275556 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ac33711ef0 0x55ac33725930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-19T01:38:55.527 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:55.524+0000 7f7cb3e00640 1 --2- 172.21.15.83:0/3268275556 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ac33712810 0x55ac33716df0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f7ca00a3fa0 tx=0x7f7ca009b460 comp rx=0 tx=0).ready entity=mon.1 client_cookie=77cf95a05b942509 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:38:55.527 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:55.524+0000 7f7c9a000640 10 client.0 ms_handle_connect on v2:172.21.15.167:3300/0 2024-08-19T01:38:55.527 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:55.525+0000 7f7c9a000640 1 -- 172.21.15.83:0/3268275556 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ca009b860 con 0x55ac33712810 2024-08-19T01:38:55.527 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:55.525+0000 7f7c9a000640 1 -- 172.21.15.83:0/3268275556 <== mon.1 v2:172.21.15.167:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8269+0+0 (secure 0 0 0) 0x7f7ca00860b0 con 0x55ac33712810 2024-08-19T01:38:55.527 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:55.525+0000 7f7cbdbfd740 1 -- 172.21.15.83:0/3268275556 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7cac009800 con 0x55ac33712810 2024-08-19T01:38:55.528 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:55.525+0000 7f7c9a000640 1 -- 172.21.15.83:0/3268275556 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f7ca00862b0 con 0x55ac33712810 2024-08-19T01:38:55.528 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:55.525+0000 7f7cbdbfd740 10 client.25660 fetch_fsmap learned FSMap version 27 2024-08-19T01:38:55.528 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:55.525+0000 7f7cbdbfd740 1 -- 172.21.15.83:0/3268275556 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55ac33797d00 con 0x55ac33712810 2024-08-19T01:38:55.528 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:55.526+0000 7f7c9a000640 1 -- 172.21.15.83:0/3268275556 <== mon.1 v2:172.21.15.167:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7cac009800 con 0x55ac33712810 2024-08-19T01:38:55.529 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:55.526+0000 7f7c9a000640 1 -- 172.21.15.83:0/3268275556 <== mon.1 v2:172.21.15.167:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f7ca009cc00 con 0x55ac33712810 2024-08-19T01:38:55.529 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:55.526+0000 7f7cbdbfd740 10 client.25660 fetch_fsmap finished waiting for FSMap version 27 2024-08-19T01:38:55.529 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:55.526+0000 7f7cbdbfd740 10 client.25660 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-19T01:38:55.529 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:55.526+0000 7f7cbdbfd740 20 client.25660 populate_metadata read hostname 'smithi083' 2024-08-19T01:38:55.529 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:55.526+0000 7f7cbdbfd740 1 --2- 172.21.15.83:0/3268275556 >> [v2:172.21.15.117:6838/1967595704,v1:172.21.15.117:6839/1967595704] conn(0x55ac3372cef0 0x55ac3379a500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:38:55.529 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:55.526+0000 7f7cbdbfd740 4 client.25660 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-08-19T01:38:55.529 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:55.526+0000 7f7cbdbfd740 1 -- 172.21.15.83:0/3268275556 --> [v2:172.21.15.117:6838/1967595704,v1:172.21.15.117:6839/1967595704] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55ac33676670 con 0x55ac3372cef0 2024-08-19T01:38:55.529 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:55.526+0000 7f7cb8a00640 1 --2- 172.21.15.83:0/3268275556 >> [v2:172.21.15.117:6838/1967595704,v1:172.21.15.117:6839/1967595704] conn(0x55ac3372cef0 0x55ac3379a500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:38:55.529 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:55.527+0000 7f7c9a000640 10 client.25660 ms_handle_connect on v2:172.21.15.117:6838/1967595704 2024-08-19T01:38:55.530 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:55.527+0000 7f7cb8a00640 1 --2- 172.21.15.83:0/3268275556 >> [v2:172.21.15.117:6838/1967595704,v1:172.21.15.117:6839/1967595704] conn(0x55ac3372cef0 0x55ac3379a500 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c86ddc0aa75e589d server_cookie=7113166973fe333a in_seq=0 out_seq=0 2024-08-19T01:38:55.531 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:55.530+0000 7f7c9a000640 1 -- 172.21.15.83:0/3268275556 <== mds.0 v2:172.21.15.117:6838/1967595704 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55ac33676670 con 0x55ac3372cef0 2024-08-19T01:38:55.531 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:55.530+0000 7f7c9a000640 10 client.25660 handle_command_reply: tid=0 2024-08-19T01:38:55.531 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:55.530+0000 7f7cbdbfd740 1 client.25660 shutdown 2024-08-19T01:38:55.531 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:55.530+0000 7f7cbdbfd740 1 -- 172.21.15.83:0/3268275556 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ac33712810 msgr2=0x55ac33716df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:38:55.531 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:55.530+0000 7f7cbdbfd740 1 --2- 172.21.15.83:0/3268275556 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ac33712810 0x55ac33716df0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f7ca00a3fa0 tx=0x7f7ca009b460 comp rx=0 tx=0).stop 2024-08-19T01:38:55.532 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:55.530+0000 7f7cbdbfd740 1 -- 172.21.15.83:0/3268275556 shutdown_connections 2024-08-19T01:38:55.532 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:55.530+0000 7f7cbdbfd740 1 --2- 172.21.15.83:0/3268275556 >> [v2:172.21.15.117:6838/1967595704,v1:172.21.15.117:6839/1967595704] conn(0x55ac3372cef0 0x55ac3379a500 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:38:55.532 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:55.530+0000 7f7c9a000640 0 client.25660 ms_handle_reset on v2:172.21.15.117:6838/1967595704 2024-08-19T01:38:55.532 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:55.530+0000 7f7cbdbfd740 1 --2- 172.21.15.83:0/3268275556 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ac3371d0d0 0x55ac337173e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:38:55.532 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:55.530+0000 7f7cbdbfd740 1 --2- 172.21.15.83:0/3268275556 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ac33712810 0x55ac33716df0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:38:55.532 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:55.530+0000 7f7cbdbfd740 1 --2- 172.21.15.83:0/3268275556 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ac33711ef0 0x55ac33725930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:38:55.532 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:55.530+0000 7f7cbdbfd740 1 -- 172.21.15.83:0/3268275556 >> 172.21.15.83:0/3268275556 conn(0x55ac3370e480 msgr2=0x55ac33719f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:38:55.533 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:55.531+0000 7f7cbdbfd740 1 -- 172.21.15.83:0/3268275556 shutdown_connections 2024-08-19T01:38:55.533 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:55.531+0000 7f7cbdbfd740 1 -- 172.21.15.83:0/3268275556 wait complete. 2024-08-19T01:38:55.533 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:55.531+0000 7f7cbdbfd740 20 client.25660 trim_cache size 0 max 16384 2024-08-19T01:38:55.533 INFO:teuthology.orchestra.run.smithi083.stdout:[] 2024-08-19T01:38:55.534 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:55.532+0000 7f7cb9e00640 1 -- 172.21.15.83:0/2058632191 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f7c9c0a2c00 msgr2=0x7f7c9c0a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:38:55.534 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:55.532+0000 7f7cb9e00640 1 --2- 172.21.15.83:0/2058632191 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f7c9c0a2c00 0x7f7c9c0a50c0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f7ca8009620 tx=0x7f7ca8004950 comp rx=0 tx=0).stop 2024-08-19T01:38:55.534 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:55.532+0000 7f7cb9e00640 1 -- 172.21.15.83:0/2058632191 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7cb41031e0 msgr2=0x7f7cb4100a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:38:55.534 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:55.532+0000 7f7cb9e00640 1 --2- 172.21.15.83:0/2058632191 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7cb41031e0 0x7f7cb4100a60 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f7ca000e6c0 tx=0x7f7ca000eb80 comp rx=0 tx=0).stop 2024-08-19T01:38:55.534 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:55.532+0000 7f7cb9e00640 1 -- 172.21.15.83:0/2058632191 shutdown_connections 2024-08-19T01:38:55.535 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:55.532+0000 7f7cb9e00640 1 --2- 172.21.15.83:0/2058632191 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f7c9c0a2c00 0x7f7c9c0a50c0 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:38:55.535 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:55.532+0000 7f7cb9e00640 1 --2- 172.21.15.83:0/2058632191 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7cb410bcf0 0x7f7cb4100fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:38:55.535 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:55.532+0000 7f7cb9e00640 1 --2- 172.21.15.83:0/2058632191 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7cb41031e0 0x7f7cb4100a60 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:38:55.535 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:55.532+0000 7f7cb9e00640 1 --2- 172.21.15.83:0/2058632191 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7cb41028c0 0x7f7cb4100520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:38:55.535 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:55.532+0000 7f7cb9e00640 1 -- 172.21.15.83:0/2058632191 >> 172.21.15.83:0/2058632191 conn(0x7f7cb40fc340 msgr2=0x7f7cb40fda20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:38:55.535 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:55.532+0000 7f7cb9e00640 1 -- 172.21.15.83:0/2058632191 shutdown_connections 2024-08-19T01:38:55.535 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:55.533+0000 7f7cb9e00640 1 -- 172.21.15.83:0/2058632191 wait complete. 2024-08-19T01:38:55.904 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:38:55 smithi083 ceph-mon[17120]: pgmap v461: 97 pgs: 97 active+clean; 365 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 9.1 MiB/s wr, 1.54k op/s 2024-08-19T01:38:55.952 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:38:55 smithi117 ceph-mon[22016]: pgmap v461: 97 pgs: 97 active+clean; 365 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 9.1 MiB/s wr, 1.54k op/s 2024-08-19T01:38:55.965 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:38:55 smithi167 ceph-mon[21875]: pgmap v461: 97 pgs: 97 active+clean; 365 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 9.1 MiB/s wr, 1.54k op/s 2024-08-19T01:38:57.611 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-08-19T01:38:57.612 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833 shell --fsid bad8780e-5dc9-11ef-bccf-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-08-19T01:38:57.904 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:38:57 smithi083 ceph-mon[17120]: pgmap v462: 97 pgs: 97 active+clean; 367 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 8.3 MiB/s wr, 1.53k op/s 2024-08-19T01:38:57.952 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:38:57 smithi117 ceph-mon[22016]: pgmap v462: 97 pgs: 97 active+clean; 367 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 8.3 MiB/s wr, 1.53k op/s 2024-08-19T01:38:57.965 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:38:57 smithi167 ceph-mon[21875]: pgmap v462: 97 pgs: 97 active+clean; 367 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 8.3 MiB/s wr, 1.53k op/s 2024-08-19T01:38:58.115 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/bad8780e-5dc9-11ef-bccf-c7b262605968/mon.a/config 2024-08-19T01:38:59.113 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:59.105+0000 7f6d73e00640 1 -- 172.21.15.83:0/886920920 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6d6c094920 msgr2=0x7f6d6c094d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:38:59.113 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:59.105+0000 7f6d73e00640 1 --2- 172.21.15.83:0/886920920 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6d6c094920 0x7f6d6c094d80 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f6d6000b9f0 tx=0x7f6d6002c0a0 comp rx=0 tx=0).stop 2024-08-19T01:38:59.113 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:59.106+0000 7f6d73e00640 1 -- 172.21.15.83:0/886920920 shutdown_connections 2024-08-19T01:38:59.113 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:59.106+0000 7f6d73e00640 1 --2- 172.21.15.83:0/886920920 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6d6c0952c0 0x7f6d6c0a1310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:38:59.113 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:59.106+0000 7f6d73e00640 1 --2- 172.21.15.83:0/886920920 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6d6c094920 0x7f6d6c094d80 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:38:59.113 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:59.106+0000 7f6d73e00640 1 --2- 172.21.15.83:0/886920920 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6d6c09a540 0x7f6d6c09a920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:38:59.113 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:59.106+0000 7f6d73e00640 1 -- 172.21.15.83:0/886920920 >> 172.21.15.83:0/886920920 conn(0x7f6d6c00a1e0 msgr2=0x7f6d6c00a5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:38:59.114 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:59.106+0000 7f6d73e00640 1 -- 172.21.15.83:0/886920920 shutdown_connections 2024-08-19T01:38:59.114 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:59.106+0000 7f6d73e00640 1 -- 172.21.15.83:0/886920920 wait complete. 2024-08-19T01:38:59.114 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:59.107+0000 7f6d73e00640 1 Processor -- start 2024-08-19T01:38:59.114 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:59.108+0000 7f6d73e00640 1 -- start start 2024-08-19T01:38:59.114 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:59.109+0000 7f6d73e00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6d6c094920 0x7f6d6c0a9ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:38:59.114 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:59.109+0000 7f6d73e00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6d6c0952c0 0x7f6d6c0aa1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:38:59.114 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:59.109+0000 7f6d72a00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6d6c094920 0x7f6d6c0a9ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:38:59.114 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:59.109+0000 7f6d72a00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6d6c094920 0x7f6d6c0a9ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.83:52252/0 (socket says 172.21.15.83:52252) 2024-08-19T01:38:59.114 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:59.109+0000 7f6d72a00640 1 -- 172.21.15.83:0/2114161914 learned_addr learned my addr 172.21.15.83:0/2114161914 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-08-19T01:38:59.114 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:59.109+0000 7f6d73e00640 1 --2- 172.21.15.83:0/2114161914 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6d6c09a540 0x7f6d6c0a4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:38:59.114 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:59.109+0000 7f6d73e00640 1 -- 172.21.15.83:0/2114161914 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f6d6c0a3790 con 0x7f6d6c09a540 2024-08-19T01:38:59.114 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:59.109+0000 7f6d72000640 1 --2- 172.21.15.83:0/2114161914 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6d6c0952c0 0x7f6d6c0aa1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:38:59.114 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:59.110+0000 7f6d73e00640 1 -- 172.21.15.83:0/2114161914 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f6d6c0a3610 con 0x7f6d6c0952c0 2024-08-19T01:38:59.115 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:59.110+0000 7f6d73e00640 1 -- 172.21.15.83:0/2114161914 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f6d6c0a3910 con 0x7f6d6c094920 2024-08-19T01:38:59.115 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:59.110+0000 7f6d73400640 1 --2- 172.21.15.83:0/2114161914 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6d6c09a540 0x7f6d6c0a4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:38:59.115 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:59.111+0000 7f6d72000640 1 -- 172.21.15.83:0/2114161914 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6d6c094920 msgr2=0x7f6d6c0a9ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:38:59.115 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:59.111+0000 7f6d72000640 1 --2- 172.21.15.83:0/2114161914 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6d6c094920 0x7f6d6c0a9ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:38:59.115 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:59.111+0000 7f6d72000640 1 -- 172.21.15.83:0/2114161914 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6d6c09a540 msgr2=0x7f6d6c0a4dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:38:59.115 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:59.111+0000 7f6d72000640 1 --2- 172.21.15.83:0/2114161914 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6d6c09a540 0x7f6d6c0a4dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:38:59.115 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:59.111+0000 7f6d72000640 1 -- 172.21.15.83:0/2114161914 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6d6c0a5680 con 0x7f6d6c0952c0 2024-08-19T01:38:59.115 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:59.111+0000 7f6d73400640 1 --2- 172.21.15.83:0/2114161914 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6d6c09a540 0x7f6d6c0a4dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-19T01:38:59.115 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:59.111+0000 7f6d72000640 1 --2- 172.21.15.83:0/2114161914 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6d6c0952c0 0x7f6d6c0aa1e0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f6d6002c610 tx=0x7f6d60004040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:38:59.116 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:59.111+0000 7f6d72a00640 1 --2- 172.21.15.83:0/2114161914 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6d6c094920 0x7f6d6c0a9ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-19T01:38:59.116 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:59.112+0000 7f6d6b400640 1 -- 172.21.15.83:0/2114161914 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d60039070 con 0x7f6d6c0952c0 2024-08-19T01:38:59.116 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:59.112+0000 7f6d6b400640 1 -- 172.21.15.83:0/2114161914 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6d60004430 con 0x7f6d6c0952c0 2024-08-19T01:38:59.116 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:59.112+0000 7f6d73e00640 1 -- 172.21.15.83:0/2114161914 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6d6c0a5910 con 0x7f6d6c0952c0 2024-08-19T01:38:59.116 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:59.112+0000 7f6d6b400640 1 -- 172.21.15.83:0/2114161914 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d600035f0 con 0x7f6d6c0952c0 2024-08-19T01:38:59.116 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:59.113+0000 7f6d73e00640 1 -- 172.21.15.83:0/2114161914 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6d6c0ad9b0 con 0x7f6d6c0952c0 2024-08-19T01:38:59.121 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:59.120+0000 7f6d7c443740 1 Processor -- start 2024-08-19T01:38:59.122 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:59.121+0000 7f6d7c443740 1 -- start start 2024-08-19T01:38:59.123 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:59.121+0000 7f6d7c443740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ad93aff650 0x55ad93affa30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:38:59.124 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:59.121+0000 7f6d6b400640 1 -- 172.21.15.83:0/2114161914 <== mon.2 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7f6d600046c0 con 0x7f6d6c0952c0 2024-08-19T01:38:59.124 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:59.124+0000 7f6d6b400640 1 --2- 172.21.15.83:0/2114161914 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f6d580a2c60 0x7f6d580a5120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:38:59.125 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:59.124+0000 7f6d73400640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ad93aff650 0x55ad93affa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:38:59.125 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:59.124+0000 7f6d7c443740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ad93afff70 0x55ad93b0a2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:38:59.125 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:59.124+0000 7f6d72a00640 1 --2- 172.21.15.83:0/2114161914 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f6d580a2c60 0x7f6d580a5120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:38:59.125 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:59.125+0000 7f6d7c443740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ad93b0a830 0x55ad93b0cc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:38:59.125 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:59.125+0000 7f6d7c443740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55ad93a1a7e0 con 0x55ad93b0a830 2024-08-19T01:38:59.125 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:59.125+0000 7f6d7c443740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55ad93a65ac0 con 0x55ad93afff70 2024-08-19T01:38:59.126 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:59.125+0000 7f6d7c443740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55ad939fd8c0 con 0x55ad93aff650 2024-08-19T01:38:59.126 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:59.125+0000 7f6d6b400640 1 -- 172.21.15.83:0/2114161914 <== mon.2 v2:172.21.15.117:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8269+0+0 (secure 0 0 0) 0x7f6d600b91f0 con 0x7f6d6c0952c0 2024-08-19T01:38:59.126 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:59.125+0000 7f6d72000640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ad93afff70 0x55ad93b0a2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:38:59.126 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:59.125+0000 7f6d72000640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ad93afff70 0x55ad93b0a2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.83:35394/0 (socket says 172.21.15.83:35394) 2024-08-19T01:38:59.126 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:59.125+0000 7f6d72000640 1 -- 172.21.15.83:0/855349979 learned_addr learned my addr 172.21.15.83:0/855349979 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-08-19T01:38:59.127 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:59.125+0000 7f6d73400640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ad93aff650 0x55ad93affa30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.83:52266/0 (socket says 172.21.15.83:52266) 2024-08-19T01:38:59.127 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:59.125+0000 7f6d72000640 1 -- 172.21.15.83:0/855349979 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ad93aff650 msgr2=0x55ad93affa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:38:59.127 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:59.125+0000 7f6d72000640 1 --2- 172.21.15.83:0/855349979 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ad93aff650 0x55ad93affa30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:38:59.127 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:59.125+0000 7f6d72000640 1 -- 172.21.15.83:0/855349979 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ad93b0a830 msgr2=0x55ad93b0cc20 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-08-19T01:38:59.127 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:59.126+0000 7f6d73400640 1 --2- 172.21.15.83:0/855349979 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ad93b0a830 0x55ad93b0cc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:38:59.127 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:59.126+0000 7f6d72000640 1 --2- 172.21.15.83:0/855349979 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ad93b0a830 0x55ad93b0cc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:38:59.127 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:59.126+0000 7f6d72000640 1 -- 172.21.15.83:0/855349979 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ad93a04760 con 0x55ad93afff70 2024-08-19T01:38:59.129 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:59.126+0000 7f6d72a00640 1 --2- 172.21.15.83:0/2114161914 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f6d580a2c60 0x7f6d580a5120 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f6d640096c0 tx=0x7f6d64009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:38:59.129 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:59.126+0000 7f6d72000640 1 --2- 172.21.15.83:0/855349979 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ad93afff70 0x55ad93b0a2f0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f6d6000b9f0 tx=0x7f6d600c2810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1df8c3c8f618d37f server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:38:59.129 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:59.128+0000 7f6d73400640 1 --2- 172.21.15.83:0/855349979 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ad93b0a830 0x55ad93b0cc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-19T01:38:59.130 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:59.129+0000 7f6d5fe00640 1 -- 172.21.15.83:0/855349979 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d600b7b80 con 0x55ad93afff70 2024-08-19T01:38:59.130 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:59.129+0000 7f6d5fe00640 1 -- 172.21.15.83:0/855349979 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6d6003eec0 con 0x55ad93afff70 2024-08-19T01:38:59.131 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:59.130+0000 7f6d5fe00640 1 -- 172.21.15.83:0/855349979 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d600b7d20 con 0x55ad93afff70 2024-08-19T01:38:59.134 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:59.134+0000 7f6d7c443740 1 -- 172.21.15.83:0/855349979 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ad93afff70 msgr2=0x55ad93b0a2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:38:59.134 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:59.134+0000 7f6d7c443740 1 --2- 172.21.15.83:0/855349979 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ad93afff70 0x55ad93b0a2f0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f6d6000b9f0 tx=0x7f6d600c2810 comp rx=0 tx=0).stop 2024-08-19T01:38:59.134 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:59.134+0000 7f6d7c443740 1 -- 172.21.15.83:0/855349979 shutdown_connections 2024-08-19T01:38:59.135 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:59.134+0000 7f6d7c443740 1 --2- 172.21.15.83:0/855349979 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ad93b0a830 0x55ad93b0cc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:38:59.135 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:59.134+0000 7f6d7c443740 1 --2- 172.21.15.83:0/855349979 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ad93afff70 0x55ad93b0a2f0 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:38:59.135 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:59.134+0000 7f6d7c443740 1 --2- 172.21.15.83:0/855349979 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ad93aff650 0x55ad93affa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:38:59.135 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:59.134+0000 7f6d7c443740 1 -- 172.21.15.83:0/855349979 >> 172.21.15.83:0/855349979 conn(0x55ad93afbbe0 msgr2=0x55ad93afbfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:38:59.135 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:59.134+0000 7f6d7c443740 1 -- 172.21.15.83:0/855349979 shutdown_connections 2024-08-19T01:38:59.135 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:59.134+0000 7f6d7c443740 1 -- 172.21.15.83:0/855349979 wait complete. 2024-08-19T01:38:59.136 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:59.135+0000 7f6d7c443740 1 -- start start 2024-08-19T01:38:59.138 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:59.137+0000 7f6d7c443740 1 Processor -- start 2024-08-19T01:38:59.142 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:59.139+0000 7f6d7c443740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ad93aff650 0x55ad93b07640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:38:59.142 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:59.139+0000 7f6d7c443740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ad93afff70 0x55ad93b07b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:38:59.142 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:59.140+0000 7f6d7c443740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ad93b0a830 0x55ad93b080c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:38:59.142 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:59.139+0000 7f6d72a00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ad93aff650 0x55ad93b07640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:38:59.142 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:59.140+0000 7f6d7c443740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55ad93a20ae0 con 0x55ad93b0a830 2024-08-19T01:38:59.142 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:59.140+0000 7f6d7c443740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55ad93a61a80 con 0x55ad93aff650 2024-08-19T01:38:59.142 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:59.140+0000 7f6d7c443740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55ad93b0d1c0 con 0x55ad93afff70 2024-08-19T01:38:59.143 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:59.140+0000 7f6d7c443740 10 client.0 _ll_register_callbacks cb 0x55ad93a22190 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-19T01:38:59.143 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:59.140+0000 7f6d72a00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ad93aff650 0x55ad93b07640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.83:35406/0 (socket says 172.21.15.83:35406) 2024-08-19T01:38:59.143 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:59.140+0000 7f6d72a00640 1 -- 172.21.15.83:0/1918423153 learned_addr learned my addr 172.21.15.83:0/1918423153 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-08-19T01:38:59.143 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:59.140+0000 7f6d72000640 1 --2- 172.21.15.83:0/1918423153 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ad93afff70 0x55ad93b07b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:38:59.143 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:59.140+0000 7f6d73400640 1 --2- 172.21.15.83:0/1918423153 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ad93b0a830 0x55ad93b080c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:38:59.143 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:59.141+0000 7f6d72a00640 1 -- 172.21.15.83:0/1918423153 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ad93afff70 msgr2=0x55ad93b07b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:38:59.143 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:59.141+0000 7f6d72a00640 1 --2- 172.21.15.83:0/1918423153 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ad93afff70 0x55ad93b07b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:38:59.143 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:59.141+0000 7f6d72a00640 1 -- 172.21.15.83:0/1918423153 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ad93b0a830 msgr2=0x55ad93b080c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:38:59.143 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:59.141+0000 7f6d72a00640 1 --2- 172.21.15.83:0/1918423153 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ad93b0a830 0x55ad93b080c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:38:59.143 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:59.141+0000 7f6d72a00640 1 -- 172.21.15.83:0/1918423153 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ad937d76a0 con 0x55ad93aff650 2024-08-19T01:38:59.143 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:59.141+0000 7f6d72000640 1 --2- 172.21.15.83:0/1918423153 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ad93afff70 0x55ad93b07b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-19T01:38:59.143 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:59.141+0000 7f6d72a00640 1 --2- 172.21.15.83:0/1918423153 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ad93aff650 0x55ad93b07640 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f6d6400fcb0 tx=0x7f6d64004730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fb3b1503dad8f2fe server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:38:59.143 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:59.141+0000 7f6d5cc00640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-08-19T01:38:59.148 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:59.141+0000 7f6d5cc00640 1 -- 172.21.15.83:0/1918423153 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d6401de10 con 0x55ad93aff650 2024-08-19T01:38:59.148 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:59.142+0000 7f6d5cc00640 1 -- 172.21.15.83:0/1918423153 <== mon.2 v2:172.21.15.117:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8269+0+0 (secure 0 0 0) 0x7f6d6401e430 con 0x55ad93aff650 2024-08-19T01:38:59.148 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:59.142+0000 7f6d7c443740 1 -- 172.21.15.83:0/1918423153 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6d6003eec0 con 0x55ad93aff650 2024-08-19T01:38:59.148 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:59.142+0000 7f6d5cc00640 1 -- 172.21.15.83:0/1918423153 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f6d6401e630 con 0x55ad93aff650 2024-08-19T01:38:59.148 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:59.143+0000 7f6d7c443740 10 client.25727 fetch_fsmap learned FSMap version 27 2024-08-19T01:38:59.148 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:59.143+0000 7f6d7c443740 1 -- 172.21.15.83:0/1918423153 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55ad93b18c20 con 0x55ad93aff650 2024-08-19T01:38:59.148 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:59.143+0000 7f6d5cc00640 1 -- 172.21.15.83:0/1918423153 <== mon.2 v2:172.21.15.117:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6d6003eec0 con 0x55ad93aff650 2024-08-19T01:38:59.148 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:59.145+0000 7f6d5cc00640 1 -- 172.21.15.83:0/1918423153 <== mon.2 v2:172.21.15.117:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f6d6401b990 con 0x55ad93aff650 2024-08-19T01:38:59.148 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:59.145+0000 7f6d7c443740 10 client.25727 fetch_fsmap finished waiting for FSMap version 27 2024-08-19T01:38:59.149 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:59.145+0000 7f6d7c443740 10 client.25727 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-19T01:38:59.149 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:59.145+0000 7f6d7c443740 20 client.25727 populate_metadata read hostname 'smithi083' 2024-08-19T01:38:59.149 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:59.146+0000 7f6d7c443740 1 --2- 172.21.15.83:0/1918423153 >> [v2:172.21.15.117:6838/1967595704,v1:172.21.15.117:6839/1967595704] conn(0x55ad93b1a180 0x55ad93b1c600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:38:59.149 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:59.146+0000 7f6d7c443740 4 client.25727 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-19T01:38:59.149 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:59.146+0000 7f6d7c443740 1 -- 172.21.15.83:0/1918423153 --> [v2:172.21.15.117:6838/1967595704,v1:172.21.15.117:6839/1967595704] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ad93a73030 con 0x55ad93b1a180 2024-08-19T01:38:59.149 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:59.146+0000 7f6d72000640 1 --2- 172.21.15.83:0/1918423153 >> [v2:172.21.15.117:6838/1967595704,v1:172.21.15.117:6839/1967595704] conn(0x55ad93b1a180 0x55ad93b1c600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:38:59.149 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:59.147+0000 7f6d72000640 1 --2- 172.21.15.83:0/1918423153 >> [v2:172.21.15.117:6838/1967595704,v1:172.21.15.117:6839/1967595704] conn(0x55ad93b1a180 0x55ad93b1c600 crc :-1 s=READY pgs=89 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a7196cc0d002a9bc server_cookie=48e32456cd4215dd in_seq=0 out_seq=0 2024-08-19T01:38:59.149 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:59.147+0000 7f6d5cc00640 10 client.25727 ms_handle_connect on v2:172.21.15.117:6838/1967595704 2024-08-19T01:38:59.151 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:59.151+0000 7f6d5cc00640 1 -- 172.21.15.83:0/1918423153 <== mds.0 v2:172.21.15.117:6838/1967595704 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55ad93a73030 con 0x55ad93b1a180 2024-08-19T01:38:59.151 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:59.151+0000 7f6d5cc00640 10 client.25727 handle_command_reply: tid=0 2024-08-19T01:38:59.152 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:59.151+0000 7f6d7c443740 1 client.25727 shutdown 2024-08-19T01:38:59.152 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:59.152+0000 7f6d7c443740 1 -- 172.21.15.83:0/1918423153 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ad93aff650 msgr2=0x55ad93b07640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:38:59.152 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:59.152+0000 7f6d7c443740 1 --2- 172.21.15.83:0/1918423153 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ad93aff650 0x55ad93b07640 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f6d6400fcb0 tx=0x7f6d64004730 comp rx=0 tx=0).stop 2024-08-19T01:38:59.155 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:59.154+0000 7f6d7c443740 1 -- 172.21.15.83:0/1918423153 shutdown_connections 2024-08-19T01:38:59.155 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:59.155+0000 7f6d7c443740 1 --2- 172.21.15.83:0/1918423153 >> [v2:172.21.15.117:6838/1967595704,v1:172.21.15.117:6839/1967595704] conn(0x55ad93b1a180 0x55ad93b1c600 crc :-1 s=READY pgs=89 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:38:59.155 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:59.155+0000 7f6d7c443740 1 --2- 172.21.15.83:0/1918423153 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ad93b0a830 0x55ad93b080c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:38:59.155 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:59.155+0000 7f6d7c443740 1 --2- 172.21.15.83:0/1918423153 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ad93afff70 0x55ad93b07b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:38:59.155 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:59.155+0000 7f6d7c443740 1 --2- 172.21.15.83:0/1918423153 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ad93aff650 0x55ad93b07640 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:38:59.156 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:59.155+0000 7f6d7c443740 1 -- 172.21.15.83:0/1918423153 >> 172.21.15.83:0/1918423153 conn(0x55ad93afbbe0 msgr2=0x55ad93afad40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:38:59.156 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:59.155+0000 7f6d5cc00640 0 client.25727 ms_handle_reset on v2:172.21.15.117:6838/1967595704 2024-08-19T01:38:59.156 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:59.156+0000 7f6d7c443740 1 -- 172.21.15.83:0/1918423153 shutdown_connections 2024-08-19T01:38:59.156 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:59.156+0000 7f6d7c443740 1 -- 172.21.15.83:0/1918423153 wait complete. 2024-08-19T01:38:59.160 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:59.156+0000 7f6d7c443740 20 client.25727 trim_cache size 0 max 16384 2024-08-19T01:38:59.200 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:59.199+0000 7f6d7c443740 1 Processor -- start 2024-08-19T01:38:59.200 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:59.199+0000 7f6d7c443740 1 -- start start 2024-08-19T01:38:59.201 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:59.200+0000 7f6d7c443740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ad93aff650 0x55ad93b064f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:38:59.201 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:59.200+0000 7f6d7c443740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ad93afff70 0x55ad93b21ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:38:59.201 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:59.200+0000 7f6d7c443740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ad93b0a830 0x55ad93b22410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:38:59.201 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:59.200+0000 7f6d7c443740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f6d6c163610 con 0x55ad93b0a830 2024-08-19T01:38:59.201 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:59.200+0000 7f6d7c443740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f6d6c160ee0 con 0x55ad93afff70 2024-08-19T01:38:59.201 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:59.200+0000 7f6d7c443740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f6d6c15ede0 con 0x55ad93aff650 2024-08-19T01:38:59.201 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:59.200+0000 7f6d72000640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ad93afff70 0x55ad93b21ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:38:59.201 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:59.200+0000 7f6d72a00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ad93aff650 0x55ad93b064f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:38:59.202 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:59.200+0000 7f6d72000640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ad93afff70 0x55ad93b21ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.83:35412/0 (socket says 172.21.15.83:35412) 2024-08-19T01:38:59.202 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:59.200+0000 7f6d72000640 1 -- 172.21.15.83:0/777733889 learned_addr learned my addr 172.21.15.83:0/777733889 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-08-19T01:38:59.202 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:59.200+0000 7f6d72a00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ad93aff650 0x55ad93b064f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.83:52278/0 (socket says 172.21.15.83:52278) 2024-08-19T01:38:59.203 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:59.201+0000 7f6d73400640 1 --2- 172.21.15.83:0/777733889 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ad93b0a830 0x55ad93b22410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:38:59.203 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:59.201+0000 7f6d72a00640 1 -- 172.21.15.83:0/777733889 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ad93afff70 msgr2=0x55ad93b21ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:38:59.203 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:59.201+0000 7f6d72a00640 1 --2- 172.21.15.83:0/777733889 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ad93afff70 0x55ad93b21ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:38:59.203 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:59.201+0000 7f6d72a00640 1 -- 172.21.15.83:0/777733889 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ad93b0a830 msgr2=0x55ad93b22410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:38:59.203 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:59.202+0000 7f6d72a00640 1 --2- 172.21.15.83:0/777733889 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ad93b0a830 0x55ad93b22410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:38:59.203 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:59.202+0000 7f6d72a00640 1 -- 172.21.15.83:0/777733889 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ad93b16320 con 0x55ad93aff650 2024-08-19T01:38:59.204 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:59.202+0000 7f6d73400640 1 --2- 172.21.15.83:0/777733889 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ad93b0a830 0x55ad93b22410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-19T01:38:59.204 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:59.204+0000 7f6d72a00640 1 --2- 172.21.15.83:0/777733889 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ad93aff650 0x55ad93b064f0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f6d6401bd50 tx=0x7f6d6401dfd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cc8253894e6ee483 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:38:59.206 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:59.204+0000 7f6d5f400640 1 -- 172.21.15.83:0/777733889 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d64035070 con 0x55ad93aff650 2024-08-19T01:38:59.206 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:59.204+0000 7f6d5f400640 1 -- 172.21.15.83:0/777733889 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6d6400faf0 con 0x55ad93aff650 2024-08-19T01:38:59.206 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:59.204+0000 7f6d5f400640 1 -- 172.21.15.83:0/777733889 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d6401ea30 con 0x55ad93aff650 2024-08-19T01:38:59.207 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:59.205+0000 7f6d7c443740 1 -- 172.21.15.83:0/777733889 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ad93aff650 msgr2=0x55ad93b064f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:38:59.207 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:59.205+0000 7f6d7c443740 1 --2- 172.21.15.83:0/777733889 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ad93aff650 0x55ad93b064f0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f6d6401bd50 tx=0x7f6d6401dfd0 comp rx=0 tx=0).stop 2024-08-19T01:38:59.207 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:59.206+0000 7f6d7c443740 1 -- 172.21.15.83:0/777733889 shutdown_connections 2024-08-19T01:38:59.207 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:59.206+0000 7f6d7c443740 1 --2- 172.21.15.83:0/777733889 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ad93b0a830 0x55ad93b22410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:38:59.207 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:59.206+0000 7f6d7c443740 1 --2- 172.21.15.83:0/777733889 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ad93afff70 0x55ad93b21ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:38:59.208 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:59.206+0000 7f6d7c443740 1 --2- 172.21.15.83:0/777733889 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ad93aff650 0x55ad93b064f0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:38:59.208 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:59.206+0000 7f6d7c443740 1 -- 172.21.15.83:0/777733889 >> 172.21.15.83:0/777733889 conn(0x55ad93afbbe0 msgr2=0x55ad93b05de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:38:59.208 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:59.206+0000 7f6d7c443740 1 -- 172.21.15.83:0/777733889 shutdown_connections 2024-08-19T01:38:59.208 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:59.206+0000 7f6d7c443740 1 -- 172.21.15.83:0/777733889 wait complete. 2024-08-19T01:38:59.208 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:59.206+0000 7f6d7c443740 1 -- start start 2024-08-19T01:38:59.208 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:59.207+0000 7f6d7c443740 1 Processor -- start 2024-08-19T01:38:59.210 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:59.208+0000 7f6d7c443740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ad93aff650 0x55ad93b12d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:38:59.210 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:59.208+0000 7f6d7c443740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ad93afff70 0x55ad93b0e0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:38:59.210 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:59.208+0000 7f6d7c443740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ad93b0a830 0x55ad93b0e850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:38:59.211 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:59.208+0000 7f6d7c443740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f6d6c15f3a0 con 0x55ad93afff70 2024-08-19T01:38:59.211 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:59.208+0000 7f6d7c443740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f6d6c160810 con 0x55ad93aff650 2024-08-19T01:38:59.211 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:59.208+0000 7f6d7c443740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f6d6c15fa70 con 0x55ad93b0a830 2024-08-19T01:38:59.211 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:59.208+0000 7f6d7c443740 10 client.0 _ll_register_callbacks cb 0x7f6d6400f560 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-19T01:38:59.211 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:59.209+0000 7f6d72a00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ad93aff650 0x55ad93b12d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:38:59.212 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:59.209+0000 7f6d72a00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ad93aff650 0x55ad93b12d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.83:35422/0 (socket says 172.21.15.83:35422) 2024-08-19T01:38:59.212 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:59.209+0000 7f6d72000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ad93afff70 0x55ad93b0e0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:38:59.212 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:59.209+0000 7f6d73400640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ad93b0a830 0x55ad93b0e850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:38:59.212 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:59.209+0000 7f6d73400640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ad93b0a830 0x55ad93b0e850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.83:52284/0 (socket says 172.21.15.83:52284) 2024-08-19T01:38:59.212 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:59.209+0000 7f6d73400640 1 -- 172.21.15.83:0/3346592518 learned_addr learned my addr 172.21.15.83:0/3346592518 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-08-19T01:38:59.212 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:59.210+0000 7f6d73400640 1 -- 172.21.15.83:0/3346592518 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ad93aff650 msgr2=0x55ad93b12d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:38:59.212 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:59.210+0000 7f6d73400640 1 --2- 172.21.15.83:0/3346592518 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ad93aff650 0x55ad93b12d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:38:59.212 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:59.210+0000 7f6d73400640 1 -- 172.21.15.83:0/3346592518 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ad93afff70 msgr2=0x55ad93b0e0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:38:59.213 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:59.211+0000 7f6d73400640 1 --2- 172.21.15.83:0/3346592518 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ad93afff70 0x55ad93b0e0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:38:59.213 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:59.211+0000 7f6d73400640 1 -- 172.21.15.83:0/3346592518 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ad93b83ea0 con 0x55ad93b0a830 2024-08-19T01:38:59.213 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:59.211+0000 7f6d72a00640 1 --2- 172.21.15.83:0/3346592518 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ad93aff650 0x55ad93b12d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-19T01:38:59.216 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:59.211+0000 7f6d72000640 1 --2- 172.21.15.83:0/3346592518 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ad93afff70 0x55ad93b0e0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-19T01:38:59.217 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:59.213+0000 7f6d73400640 1 --2- 172.21.15.83:0/3346592518 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ad93b0a830 0x55ad93b0e850 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f6d74067920 tx=0x7f6d740793d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c7ec87f4d3f4c7bc server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:38:59.217 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:59.214+0000 7f6d5cc00640 10 client.0 ms_handle_connect on v2:172.21.15.167:3300/0 2024-08-19T01:38:59.219 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:59.217+0000 7f6d5cc00640 1 -- 172.21.15.83:0/3346592518 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d74079ea0 con 0x55ad93b0a830 2024-08-19T01:38:59.220 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:59.217+0000 7f6d5cc00640 1 -- 172.21.15.83:0/3346592518 <== mon.1 v2:172.21.15.167:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8269+0+0 (secure 0 0 0) 0x7f6d740773c0 con 0x55ad93b0a830 2024-08-19T01:38:59.220 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:59.217+0000 7f6d7c443740 1 -- 172.21.15.83:0/3346592518 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55ad93af6e50 con 0x55ad93b0a830 2024-08-19T01:38:59.220 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:59.218+0000 7f6d5cc00640 1 -- 172.21.15.83:0/3346592518 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f6d740835f0 con 0x55ad93b0a830 2024-08-19T01:38:59.220 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:59.218+0000 7f6d7c443740 10 client.25678 fetch_fsmap learned FSMap version 27 2024-08-19T01:38:59.220 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:59.218+0000 7f6d7c443740 1 -- 172.21.15.83:0/3346592518 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55ad93b85100 con 0x55ad93b0a830 2024-08-19T01:38:59.220 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:59.218+0000 7f6d5cc00640 1 -- 172.21.15.83:0/3346592518 <== mon.1 v2:172.21.15.167:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x55ad93af6e50 con 0x55ad93b0a830 2024-08-19T01:38:59.220 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:59.218+0000 7f6d5cc00640 1 -- 172.21.15.83:0/3346592518 <== mon.1 v2:172.21.15.167:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f6d74082070 con 0x55ad93b0a830 2024-08-19T01:38:59.220 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:59.218+0000 7f6d7c443740 10 client.25678 fetch_fsmap finished waiting for FSMap version 27 2024-08-19T01:38:59.221 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:59.218+0000 7f6d7c443740 10 client.25678 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-19T01:38:59.221 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:59.218+0000 7f6d7c443740 20 client.25678 populate_metadata read hostname 'smithi083' 2024-08-19T01:38:59.221 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:59.218+0000 7f6d7c443740 1 --2- 172.21.15.83:0/3346592518 >> [v2:172.21.15.117:6838/1967595704,v1:172.21.15.117:6839/1967595704] conn(0x55ad93b85850 0x55ad93b87c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:38:59.221 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:59.218+0000 7f6d7c443740 4 client.25678 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-08-19T01:38:59.221 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:59.218+0000 7f6d7c443740 1 -- 172.21.15.83:0/3346592518 --> [v2:172.21.15.117:6838/1967595704,v1:172.21.15.117:6839/1967595704] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55ad93a6d540 con 0x55ad93b85850 2024-08-19T01:38:59.221 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:59.219+0000 7f6d72a00640 1 --2- 172.21.15.83:0/3346592518 >> [v2:172.21.15.117:6838/1967595704,v1:172.21.15.117:6839/1967595704] conn(0x55ad93b85850 0x55ad93b87c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:38:59.227 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:59.221+0000 7f6d72a00640 1 --2- 172.21.15.83:0/3346592518 >> [v2:172.21.15.117:6838/1967595704,v1:172.21.15.117:6839/1967595704] conn(0x55ad93b85850 0x55ad93b87c70 crc :-1 s=READY pgs=90 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f00146ec87ad4e2f server_cookie=74bb771dac26471 in_seq=0 out_seq=0 2024-08-19T01:38:59.227 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:59.221+0000 7f6d5cc00640 10 client.25678 ms_handle_connect on v2:172.21.15.117:6838/1967595704 2024-08-19T01:38:59.421 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:59.417+0000 7f6d5cc00640 1 -- 172.21.15.83:0/3346592518 <== mds.0 v2:172.21.15.117:6838/1967595704 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55ad93a6d540 con 0x55ad93b85850 2024-08-19T01:38:59.421 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:59.417+0000 7f6d5cc00640 10 client.25678 handle_command_reply: tid=0 2024-08-19T01:38:59.422 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-08-19T01:38:59.422 INFO:teuthology.orchestra.run.smithi083.stdout: "return_code": 0, 2024-08-19T01:38:59.422 INFO:teuthology.orchestra.run.smithi083.stdout: "scrub_tag": "0e436353-7799-40e1-97b3-a89eaf3bf27d", 2024-08-19T01:38:59.422 INFO:teuthology.orchestra.run.smithi083.stdout: "mode": "asynchronous" 2024-08-19T01:38:59.422 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-08-19T01:38:59.423 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:59.417+0000 7f6d7c443740 1 client.25678 shutdown 2024-08-19T01:38:59.423 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:59.417+0000 7f6d7c443740 1 -- 172.21.15.83:0/3346592518 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ad93b0a830 msgr2=0x55ad93b0e850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:38:59.423 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:59.417+0000 7f6d7c443740 1 --2- 172.21.15.83:0/3346592518 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ad93b0a830 0x55ad93b0e850 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f6d74067920 tx=0x7f6d740793d0 comp rx=0 tx=0).stop 2024-08-19T01:38:59.423 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:59.418+0000 7f6d7c443740 1 -- 172.21.15.83:0/3346592518 shutdown_connections 2024-08-19T01:38:59.423 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:59.418+0000 7f6d7c443740 1 --2- 172.21.15.83:0/3346592518 >> [v2:172.21.15.117:6838/1967595704,v1:172.21.15.117:6839/1967595704] conn(0x55ad93b85850 0x55ad93b87c70 crc :-1 s=READY pgs=90 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:38:59.423 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:59.418+0000 7f6d7c443740 1 --2- 172.21.15.83:0/3346592518 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ad93b0a830 0x55ad93b0e850 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:38:59.423 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:59.418+0000 7f6d7c443740 1 --2- 172.21.15.83:0/3346592518 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ad93afff70 0x55ad93b0e0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:38:59.423 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:59.418+0000 7f6d7c443740 1 --2- 172.21.15.83:0/3346592518 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ad93aff650 0x55ad93b12d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:38:59.423 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:59.418+0000 7f6d7c443740 1 -- 172.21.15.83:0/3346592518 >> 172.21.15.83:0/3346592518 conn(0x55ad93afbbe0 msgr2=0x55ad93b15b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:38:59.423 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:59.418+0000 7f6d5cc00640 0 client.25678 ms_handle_reset on v2:172.21.15.117:6838/1967595704 2024-08-19T01:38:59.423 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:59.418+0000 7f6d7c443740 1 -- 172.21.15.83:0/3346592518 shutdown_connections 2024-08-19T01:38:59.424 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:59.418+0000 7f6d7c443740 1 -- 172.21.15.83:0/3346592518 wait complete. 2024-08-19T01:38:59.424 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:59.418+0000 7f6d7c443740 20 client.25678 trim_cache size 0 max 16384 2024-08-19T01:38:59.424 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:59.420+0000 7f6d73e00640 1 -- 172.21.15.83:0/2114161914 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f6d580a2c60 msgr2=0x7f6d580a5120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:38:59.424 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:59.420+0000 7f6d73e00640 1 --2- 172.21.15.83:0/2114161914 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f6d580a2c60 0x7f6d580a5120 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f6d640096c0 tx=0x7f6d64009260 comp rx=0 tx=0).stop 2024-08-19T01:38:59.424 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:59.420+0000 7f6d73e00640 1 -- 172.21.15.83:0/2114161914 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6d6c0952c0 msgr2=0x7f6d6c0aa1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:38:59.424 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:59.420+0000 7f6d73e00640 1 --2- 172.21.15.83:0/2114161914 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6d6c0952c0 0x7f6d6c0aa1e0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f6d6002c610 tx=0x7f6d60004040 comp rx=0 tx=0).stop 2024-08-19T01:38:59.424 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:59.420+0000 7f6d73e00640 1 -- 172.21.15.83:0/2114161914 shutdown_connections 2024-08-19T01:38:59.424 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:59.420+0000 7f6d73e00640 1 --2- 172.21.15.83:0/2114161914 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f6d580a2c60 0x7f6d580a5120 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:38:59.424 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:59.420+0000 7f6d73e00640 1 --2- 172.21.15.83:0/2114161914 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6d6c09a540 0x7f6d6c0a4dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:38:59.424 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:59.420+0000 7f6d73e00640 1 --2- 172.21.15.83:0/2114161914 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6d6c0952c0 0x7f6d6c0aa1e0 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:38:59.424 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:59.420+0000 7f6d73e00640 1 --2- 172.21.15.83:0/2114161914 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6d6c094920 0x7f6d6c0a9ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:38:59.424 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:59.420+0000 7f6d73e00640 1 -- 172.21.15.83:0/2114161914 >> 172.21.15.83:0/2114161914 conn(0x7f6d6c00a1e0 msgr2=0x7f6d6c093610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:38:59.424 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:59.421+0000 7f6d73e00640 1 -- 172.21.15.83:0/2114161914 shutdown_connections 2024-08-19T01:38:59.424 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:38:59.421+0000 7f6d73e00640 1 -- 172.21.15.83:0/2114161914 wait complete. 2024-08-19T01:38:59.907 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:38:59 smithi083 ceph-mon[17120]: pgmap v463: 97 pgs: 97 active+clean; 368 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 8.2 MiB/s wr, 1.54k op/s 2024-08-19T01:38:59.952 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:38:59 smithi117 ceph-mon[22016]: pgmap v463: 97 pgs: 97 active+clean; 368 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 8.2 MiB/s wr, 1.54k op/s 2024-08-19T01:38:59.965 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:38:59 smithi167 ceph-mon[21875]: pgmap v463: 97 pgs: 97 active+clean; 368 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 8.2 MiB/s wr, 1.54k op/s 2024-08-19T01:39:00.627 INFO:tasks.workunit.client.0.smithi083.stdout: 124 359753 508 236583 349 207498 1357 2024-08-19T01:39:00.905 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:39:00 smithi083 ceph-mon[17120]: scrub queued for path: / 2024-08-19T01:39:00.906 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:39:00 smithi083 ceph-mon[17120]: scrub summary: idle+waiting paths [/] 2024-08-19T01:39:00.906 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:39:00 smithi083 ceph-mon[17120]: scrub summary: active paths [/] 2024-08-19T01:39:00.906 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:39:00 smithi083 ceph-mon[17120]: pgmap v464: 97 pgs: 97 active+clean; 377 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 7.5 MiB/s wr, 976 op/s 2024-08-19T01:39:00.952 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:39:00 smithi117 ceph-mon[22016]: scrub queued for path: / 2024-08-19T01:39:00.952 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:39:00 smithi117 ceph-mon[22016]: scrub summary: idle+waiting paths [/] 2024-08-19T01:39:00.952 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:39:00 smithi117 ceph-mon[22016]: scrub summary: active paths [/] 2024-08-19T01:39:00.952 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:39:00 smithi117 ceph-mon[22016]: pgmap v464: 97 pgs: 97 active+clean; 377 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 7.5 MiB/s wr, 976 op/s 2024-08-19T01:39:00.965 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:39:00 smithi167 ceph-mon[21875]: scrub queued for path: / 2024-08-19T01:39:00.965 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:39:00 smithi167 ceph-mon[21875]: scrub summary: idle+waiting paths [/] 2024-08-19T01:39:00.966 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:39:00 smithi167 ceph-mon[21875]: scrub summary: active paths [/] 2024-08-19T01:39:00.966 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:39:00 smithi167 ceph-mon[21875]: pgmap v464: 97 pgs: 97 active+clean; 377 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 7.5 MiB/s wr, 976 op/s 2024-08-19T01:39:01.240 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833 shell --fsid bad8780e-5dc9-11ef-bccf-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-08-19T01:39:01.905 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:39:01 smithi083 ceph-mon[17120]: scrub summary: active paths [/] 2024-08-19T01:39:01.905 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:39:01 smithi083 ceph-mon[17120]: scrub summary: idle+waiting paths [/] 2024-08-19T01:39:01.952 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:39:01 smithi117 ceph-mon[22016]: scrub summary: active paths [/] 2024-08-19T01:39:01.952 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:39:01 smithi117 ceph-mon[22016]: scrub summary: idle+waiting paths [/] 2024-08-19T01:39:01.965 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:39:01 smithi167 ceph-mon[21875]: scrub summary: active paths [/] 2024-08-19T01:39:01.966 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:39:01 smithi167 ceph-mon[21875]: scrub summary: idle+waiting paths [/] 2024-08-19T01:39:02.952 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:39:02 smithi117 ceph-mon[22016]: scrub summary: active paths [/] 2024-08-19T01:39:02.952 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:39:02 smithi117 ceph-mon[22016]: scrub summary: idle+waiting paths [/] 2024-08-19T01:39:02.952 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:39:02 smithi117 ceph-mon[22016]: scrub summary: active paths [/] 2024-08-19T01:39:02.953 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:39:02 smithi117 ceph-mon[22016]: scrub summary: idle+waiting paths [/] 2024-08-19T01:39:02.953 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:39:02 smithi117 ceph-mon[22016]: pgmap v465: 97 pgs: 97 active+clean; 376 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 5.9 MiB/s wr, 464 op/s 2024-08-19T01:39:02.965 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:39:02 smithi167 ceph-mon[21875]: scrub summary: active paths [/] 2024-08-19T01:39:02.965 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:39:02 smithi167 ceph-mon[21875]: scrub summary: idle+waiting paths [/] 2024-08-19T01:39:02.965 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:39:02 smithi167 ceph-mon[21875]: scrub summary: active paths [/] 2024-08-19T01:39:02.966 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:39:02 smithi167 ceph-mon[21875]: scrub summary: idle+waiting paths [/] 2024-08-19T01:39:02.966 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:39:02 smithi167 ceph-mon[21875]: pgmap v465: 97 pgs: 97 active+clean; 376 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 5.9 MiB/s wr, 464 op/s 2024-08-19T01:39:03.037 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:39:02 smithi083 ceph-mon[17120]: scrub summary: active paths [/] 2024-08-19T01:39:03.037 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:39:02 smithi083 ceph-mon[17120]: scrub summary: idle+waiting paths [/] 2024-08-19T01:39:03.037 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:39:02 smithi083 ceph-mon[17120]: scrub summary: active paths [/] 2024-08-19T01:39:03.037 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:39:02 smithi083 ceph-mon[17120]: scrub summary: idle+waiting paths [/] 2024-08-19T01:39:03.037 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:39:02 smithi083 ceph-mon[17120]: pgmap v465: 97 pgs: 97 active+clean; 376 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 5.9 MiB/s wr, 464 op/s 2024-08-19T01:39:03.423 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:39:03.422+0000 7fe379c00640 1 -- 172.21.15.83:0/462033800 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe3740729e0 msgr2=0x7fe374072de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:39:03.424 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:39:03.422+0000 7fe379c00640 1 --2- 172.21.15.83:0/462033800 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe3740729e0 0x7fe374072de0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7fe36c01af10 tx=0x7fe36c03bb60 comp rx=0 tx=0).stop 2024-08-19T01:39:03.442 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:39:03.437+0000 7fe379c00640 1 -- 172.21.15.83:0/462033800 shutdown_connections 2024-08-19T01:39:03.442 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:39:03.437+0000 7fe379c00640 1 --2- 172.21.15.83:0/462033800 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe37410d8e0 0x7fe37410fda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:39:03.442 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:39:03.437+0000 7fe379c00640 1 --2- 172.21.15.83:0/462033800 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe3740733b0 0x7fe37410d220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:39:03.442 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:39:03.437+0000 7fe379c00640 1 --2- 172.21.15.83:0/462033800 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe3740729e0 0x7fe374072de0 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:39:03.442 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:39:03.437+0000 7fe379c00640 1 -- 172.21.15.83:0/462033800 >> 172.21.15.83:0/462033800 conn(0x7fe37406e880 msgr2=0x7fe374070cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:39:03.456 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:39:03.452+0000 7fe379c00640 1 -- 172.21.15.83:0/462033800 shutdown_connections 2024-08-19T01:39:03.457 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:39:03.452+0000 7fe379c00640 1 -- 172.21.15.83:0/462033800 wait complete. 2024-08-19T01:39:03.457 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:39:03.453+0000 7fe379c00640 1 Processor -- start 2024-08-19T01:39:03.457 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:39:03.455+0000 7fe379c00640 1 -- start start 2024-08-19T01:39:03.457 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:39:03.456+0000 7fe379c00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe3740729e0 0x7fe37419e420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:39:03.458 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:39:03.457+0000 7fe373e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe3740729e0 0x7fe37419e420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:39:03.458 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:39:03.457+0000 7fe373e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe3740729e0 0x7fe37419e420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:55066/0 (socket says 172.21.15.83:55066) 2024-08-19T01:39:03.458 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:39:03.457+0000 7fe373e00640 1 -- 172.21.15.83:0/3775152390 learned_addr learned my addr 172.21.15.83:0/3775152390 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-08-19T01:39:03.458 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:39:03.457+0000 7fe379c00640 1 --2- 172.21.15.83:0/3775152390 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe3740733b0 0x7fe37419e960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:39:03.459 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:39:03.457+0000 7fe379c00640 1 --2- 172.21.15.83:0/3775152390 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe37410d8e0 0x7fe3741a5520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:39:03.466 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:39:03.458+0000 7fe373400640 1 --2- 172.21.15.83:0/3775152390 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe3740733b0 0x7fe37419e960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:39:03.466 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:39:03.458+0000 7fe379c00640 1 -- 172.21.15.83:0/3775152390 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fe374113330 con 0x7fe3740729e0 2024-08-19T01:39:03.466 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:39:03.458+0000 7fe379c00640 1 -- 172.21.15.83:0/3775152390 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fe3741131b0 con 0x7fe3740733b0 2024-08-19T01:39:03.466 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:39:03.458+0000 7fe379c00640 1 -- 172.21.15.83:0/3775152390 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fe3741134b0 con 0x7fe37410d8e0 2024-08-19T01:39:03.466 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:39:03.459+0000 7fe373e00640 1 -- 172.21.15.83:0/3775152390 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe37410d8e0 msgr2=0x7fe3741a5520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:39:03.467 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:39:03.459+0000 7fe373e00640 1 --2- 172.21.15.83:0/3775152390 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe37410d8e0 0x7fe3741a5520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:39:03.467 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:39:03.459+0000 7fe373e00640 1 -- 172.21.15.83:0/3775152390 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe3740733b0 msgr2=0x7fe37419e960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:39:03.467 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:39:03.459+0000 7fe373e00640 1 --2- 172.21.15.83:0/3775152390 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe3740733b0 0x7fe37419e960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:39:03.467 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:39:03.459+0000 7fe373e00640 1 -- 172.21.15.83:0/3775152390 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe3741a5bc0 con 0x7fe3740729e0 2024-08-19T01:39:03.467 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:39:03.459+0000 7fe373e00640 1 --2- 172.21.15.83:0/3775152390 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe3740729e0 0x7fe37419e420 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7fe36c006010 tx=0x7fe36c045790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:39:03.483 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:39:03.471+0000 7fe370c00640 1 -- 172.21.15.83:0/3775152390 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe36c054030 con 0x7fe3740729e0 2024-08-19T01:39:03.483 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:39:03.471+0000 7fe370c00640 1 -- 172.21.15.83:0/3775152390 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe36c003970 con 0x7fe3740729e0 2024-08-19T01:39:03.483 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:39:03.472+0000 7fe370c00640 1 -- 172.21.15.83:0/3775152390 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe36c004380 con 0x7fe3740729e0 2024-08-19T01:39:03.493 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:39:03.487+0000 7fe379c00640 1 -- 172.21.15.83:0/3775152390 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe3741a5e50 con 0x7fe3740729e0 2024-08-19T01:39:03.494 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:39:03.488+0000 7fe379c00640 1 -- 172.21.15.83:0/3775152390 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe3741a6280 con 0x7fe3740729e0 2024-08-19T01:39:03.534 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:39:03.531+0000 7fe37e0a2740 1 Processor -- start 2024-08-19T01:39:03.534 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:39:03.531+0000 7fe37e0a2740 1 -- start start 2024-08-19T01:39:03.534 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:39:03.532+0000 7fe37e0a2740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5575e6cf9660 0x5575e6cf9a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:39:03.534 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:39:03.532+0000 7fe37e0a2740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5575e6cf9fa0 0x5575e6cfea60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:39:03.534 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:39:03.532+0000 7fe37e0a2740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5575e6cff030 0x5575e6d014f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:39:03.534 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:39:03.532+0000 7fe37e0a2740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x5575e6c5d890 con 0x5575e6cf9660 2024-08-19T01:39:03.534 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:39:03.532+0000 7fe37e0a2740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x5575e6bf5d90 con 0x5575e6cf9fa0 2024-08-19T01:39:03.534 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:39:03.532+0000 7fe37e0a2740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x5575e6c59850 con 0x5575e6cff030 2024-08-19T01:39:03.535 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:39:03.532+0000 7fe379200640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5575e6cf9660 0x5575e6cf9a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:39:03.535 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:39:03.533+0000 7fe379200640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5575e6cf9660 0x5575e6cf9a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:55080/0 (socket says 172.21.15.83:55080) 2024-08-19T01:39:03.535 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:39:03.533+0000 7fe379200640 1 -- 172.21.15.83:0/3194567872 learned_addr learned my addr 172.21.15.83:0/3194567872 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-08-19T01:39:03.536 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:39:03.533+0000 7fe379200640 1 -- 172.21.15.83:0/3194567872 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5575e6cff030 msgr2=0x5575e6d014f0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-08-19T01:39:03.536 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:39:03.533+0000 7fe379200640 1 --2- 172.21.15.83:0/3194567872 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5575e6cff030 0x5575e6d014f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:39:03.536 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:39:03.533+0000 7fe379200640 1 -- 172.21.15.83:0/3194567872 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5575e6cf9fa0 msgr2=0x5575e6cfea60 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-08-19T01:39:03.536 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:39:03.533+0000 7fe379200640 1 --2- 172.21.15.83:0/3194567872 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5575e6cf9fa0 0x5575e6cfea60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:39:03.536 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:39:03.533+0000 7fe379200640 1 -- 172.21.15.83:0/3194567872 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5575e6971210 con 0x5575e6cf9660 2024-08-19T01:39:03.536 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:39:03.534+0000 7fe379200640 1 --2- 172.21.15.83:0/3194567872 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5575e6cf9660 0x5575e6cf9a60 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7fe36000ac50 tx=0x7fe36000c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6083e305e34d1b55 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:39:03.543 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:39:03.539+0000 7fe370c00640 1 -- 172.21.15.83:0/3775152390 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7fe36c057070 con 0x7fe3740729e0 2024-08-19T01:39:03.543 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:39:03.541+0000 7fe370c00640 1 --2- 172.21.15.83:0/3775152390 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7fe3580a2d90 0x7fe3580a5250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:39:03.543 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:39:03.541+0000 7fe370c00640 1 -- 172.21.15.83:0/3775152390 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8269+0+0 (secure 0 0 0) 0x7fe36c019040 con 0x7fe3740729e0 2024-08-19T01:39:03.549 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:39:03.546+0000 7fe373e00640 1 --2- 172.21.15.83:0/3775152390 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7fe3580a2d90 0x7fe3580a5250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:39:03.549 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:39:03.547+0000 7fe373e00640 1 --2- 172.21.15.83:0/3775152390 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7fe3580a2d90 0x7fe3580a5250 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fe36c0d58e0 tx=0x7fe36c01a5d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:39:03.556 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:39:03.555+0000 7fe369600640 1 -- 172.21.15.83:0/3194567872 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe36000f450 con 0x5575e6cf9660 2024-08-19T01:39:03.556 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:39:03.555+0000 7fe369600640 1 -- 172.21.15.83:0/3194567872 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe36000fad0 con 0x5575e6cf9660 2024-08-19T01:39:03.556 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:39:03.555+0000 7fe369600640 1 -- 172.21.15.83:0/3194567872 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe360018b80 con 0x5575e6cf9660 2024-08-19T01:39:03.558 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:39:03.556+0000 7fe37e0a2740 1 -- 172.21.15.83:0/3194567872 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5575e6cf9660 msgr2=0x5575e6cf9a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:39:03.558 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:39:03.556+0000 7fe37e0a2740 1 --2- 172.21.15.83:0/3194567872 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5575e6cf9660 0x5575e6cf9a60 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7fe36000ac50 tx=0x7fe36000c6a0 comp rx=0 tx=0).stop 2024-08-19T01:39:03.564 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:39:03.563+0000 7fe37e0a2740 1 -- 172.21.15.83:0/3194567872 shutdown_connections 2024-08-19T01:39:03.564 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:39:03.563+0000 7fe37e0a2740 1 --2- 172.21.15.83:0/3194567872 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5575e6cff030 0x5575e6d014f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:39:03.564 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:39:03.563+0000 7fe37e0a2740 1 --2- 172.21.15.83:0/3194567872 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5575e6cf9fa0 0x5575e6cfea60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:39:03.565 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:39:03.563+0000 7fe37e0a2740 1 --2- 172.21.15.83:0/3194567872 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5575e6cf9660 0x5575e6cf9a60 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:39:03.565 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:39:03.563+0000 7fe37e0a2740 1 -- 172.21.15.83:0/3194567872 >> 172.21.15.83:0/3194567872 conn(0x5575e6cf4cd0 msgr2=0x5575e6cf50d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:39:03.572 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:39:03.565+0000 7fe37e0a2740 1 -- 172.21.15.83:0/3194567872 shutdown_connections 2024-08-19T01:39:03.574 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:39:03.571+0000 7fe37e0a2740 1 -- 172.21.15.83:0/3194567872 wait complete. 2024-08-19T01:39:03.575 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:39:03.572+0000 7fe37e0a2740 1 -- start start 2024-08-19T01:39:03.575 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:39:03.573+0000 7fe37e0a2740 1 Processor -- start 2024-08-19T01:39:03.575 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:39:03.574+0000 7fe37e0a2740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5575e6cf9660 0x5575e6d03a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:39:03.575 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:39:03.574+0000 7fe37e0a2740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5575e6cf9fa0 0x5575e6d03fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:39:03.578 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:39:03.574+0000 7fe37e0a2740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5575e6cff030 0x5575e6d0af80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:39:03.579 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:39:03.574+0000 7fe37e0a2740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x5575e6d020a0 con 0x5575e6cf9660 2024-08-19T01:39:03.579 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:39:03.574+0000 7fe37e0a2740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x5575e6c18f70 con 0x5575e6cf9fa0 2024-08-19T01:39:03.579 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:39:03.574+0000 7fe37e0a2740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x5575e6d02220 con 0x5575e6cff030 2024-08-19T01:39:03.579 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:39:03.575+0000 7fe37e0a2740 10 client.0 _ll_register_callbacks cb 0x5575e6c558d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-19T01:39:03.579 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:39:03.575+0000 7fe379200640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5575e6cf9660 0x5575e6d03a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:39:03.579 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:39:03.575+0000 7fe373400640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5575e6cf9fa0 0x5575e6d03fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:39:03.579 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:39:03.575+0000 7fe373400640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5575e6cf9fa0 0x5575e6d03fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.83:57654/0 (socket says 172.21.15.83:57654) 2024-08-19T01:39:03.579 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:39:03.575+0000 7fe373400640 1 -- 172.21.15.83:0/1931884147 learned_addr learned my addr 172.21.15.83:0/1931884147 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-08-19T01:39:03.579 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:39:03.576+0000 7fe379200640 1 -- 172.21.15.83:0/1931884147 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5575e6cff030 msgr2=0x5575e6d0af80 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-08-19T01:39:03.579 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:39:03.576+0000 7fe379200640 1 --2- 172.21.15.83:0/1931884147 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5575e6cff030 0x5575e6d0af80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:39:03.579 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:39:03.576+0000 7fe379200640 1 -- 172.21.15.83:0/1931884147 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5575e6cf9fa0 msgr2=0x5575e6d03fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:39:03.579 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:39:03.576+0000 7fe379200640 1 --2- 172.21.15.83:0/1931884147 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5575e6cf9fa0 0x5575e6d03fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:39:03.579 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:39:03.576+0000 7fe379200640 1 -- 172.21.15.83:0/1931884147 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5575e6a3d620 con 0x5575e6cf9660 2024-08-19T01:39:03.579 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:39:03.576+0000 7fe379200640 1 --2- 172.21.15.83:0/1931884147 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5575e6cf9660 0x5575e6d03a70 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7fe360017000 tx=0x7fe36000af70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7db0330c7f0ff048 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:39:03.580 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:39:03.577+0000 7fe35e000640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-08-19T01:39:03.580 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:39:03.577+0000 7fe35e000640 1 -- 172.21.15.83:0/1931884147 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe360027060 con 0x5575e6cf9660 2024-08-19T01:39:03.580 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:39:03.578+0000 7fe35e000640 1 -- 172.21.15.83:0/1931884147 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8269+0+0 (secure 0 0 0) 0x7fe36000f660 con 0x5575e6cf9660 2024-08-19T01:39:03.590 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:39:03.585+0000 7fe37e0a2740 1 -- 172.21.15.83:0/1931884147 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe36000fad0 con 0x5575e6cf9660 2024-08-19T01:39:03.596 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:39:03.595+0000 7fe35e000640 1 -- 172.21.15.83:0/1931884147 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fe36000f860 con 0x5575e6cf9660 2024-08-19T01:39:03.623 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:39:03.615+0000 7fe37e0a2740 10 client.15693 fetch_fsmap learned FSMap version 27 2024-08-19T01:39:03.623 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:39:03.615+0000 7fe37e0a2740 1 -- 172.21.15.83:0/1931884147 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5575e6d0e100 con 0x5575e6cf9660 2024-08-19T01:39:03.623 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:39:03.615+0000 7fe35e000640 1 -- 172.21.15.83:0/1931884147 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe36000fad0 con 0x5575e6cf9660 2024-08-19T01:39:03.623 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:39:03.615+0000 7fe35e000640 1 -- 172.21.15.83:0/1931884147 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7fe360003760 con 0x5575e6cf9660 2024-08-19T01:39:03.623 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:39:03.616+0000 7fe37e0a2740 10 client.15693 fetch_fsmap finished waiting for FSMap version 27 2024-08-19T01:39:03.623 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:39:03.616+0000 7fe37e0a2740 10 client.15693 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-08-19T01:39:03.624 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:39:03.616+0000 7fe37e0a2740 20 client.15693 populate_metadata read hostname 'smithi083' 2024-08-19T01:39:03.624 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:39:03.616+0000 7fe37e0a2740 1 --2- 172.21.15.83:0/1931884147 >> [v2:172.21.15.117:6838/1967595704,v1:172.21.15.117:6839/1967595704] conn(0x5575e6d0f510 0x5575e6d11900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:39:03.624 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:39:03.616+0000 7fe37e0a2740 4 client.15693 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-19T01:39:03.624 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:39:03.616+0000 7fe37e0a2740 1 -- 172.21.15.83:0/1931884147 --> [v2:172.21.15.117:6838/1967595704,v1:172.21.15.117:6839/1967595704] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5575e6c6afc0 con 0x5575e6d0f510 2024-08-19T01:39:03.624 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:39:03.620+0000 7fe373400640 1 --2- 172.21.15.83:0/1931884147 >> [v2:172.21.15.117:6838/1967595704,v1:172.21.15.117:6839/1967595704] conn(0x5575e6d0f510 0x5575e6d11900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:39:03.625 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:39:03.624+0000 7fe373400640 1 --2- 172.21.15.83:0/1931884147 >> [v2:172.21.15.117:6838/1967595704,v1:172.21.15.117:6839/1967595704] conn(0x5575e6d0f510 0x5575e6d11900 crc :-1 s=READY pgs=91 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=56c10445d8fc6de server_cookie=367059503a2ebde9 in_seq=0 out_seq=0 2024-08-19T01:39:03.626 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:39:03.624+0000 7fe35e000640 10 client.15693 ms_handle_connect on v2:172.21.15.117:6838/1967595704 2024-08-19T01:39:03.640 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:39:03.639+0000 7fe35e000640 1 -- 172.21.15.83:0/1931884147 <== mds.0 v2:172.21.15.117:6838/1967595704 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5575e6c6afc0 con 0x5575e6d0f510 2024-08-19T01:39:03.640 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:39:03.639+0000 7fe35e000640 10 client.15693 handle_command_reply: tid=0 2024-08-19T01:39:03.640 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:39:03.639+0000 7fe37e0a2740 1 client.15693 shutdown 2024-08-19T01:39:03.641 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:39:03.639+0000 7fe37e0a2740 1 -- 172.21.15.83:0/1931884147 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5575e6cf9660 msgr2=0x5575e6d03a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:39:03.641 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:39:03.639+0000 7fe37e0a2740 1 --2- 172.21.15.83:0/1931884147 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5575e6cf9660 0x5575e6d03a70 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7fe360017000 tx=0x7fe36000af70 comp rx=0 tx=0).stop 2024-08-19T01:39:03.641 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:39:03.640+0000 7fe37e0a2740 1 -- 172.21.15.83:0/1931884147 shutdown_connections 2024-08-19T01:39:03.641 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:39:03.640+0000 7fe37e0a2740 1 --2- 172.21.15.83:0/1931884147 >> [v2:172.21.15.117:6838/1967595704,v1:172.21.15.117:6839/1967595704] conn(0x5575e6d0f510 0x5575e6d11900 crc :-1 s=READY pgs=91 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:39:03.642 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:39:03.640+0000 7fe37e0a2740 1 --2- 172.21.15.83:0/1931884147 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5575e6cff030 0x5575e6d0af80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:39:03.642 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:39:03.640+0000 7fe37e0a2740 1 --2- 172.21.15.83:0/1931884147 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5575e6cf9fa0 0x5575e6d03fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:39:03.642 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:39:03.640+0000 7fe37e0a2740 1 --2- 172.21.15.83:0/1931884147 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5575e6cf9660 0x5575e6d03a70 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:39:03.642 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:39:03.640+0000 7fe37e0a2740 1 -- 172.21.15.83:0/1931884147 >> 172.21.15.83:0/1931884147 conn(0x5575e6cf4cd0 msgr2=0x5575e6cf64e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:39:03.642 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:39:03.640+0000 7fe35e000640 0 client.15693 ms_handle_reset on v2:172.21.15.117:6838/1967595704 2024-08-19T01:39:03.658 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:39:03.651+0000 7fe37e0a2740 1 -- 172.21.15.83:0/1931884147 shutdown_connections 2024-08-19T01:39:03.658 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:39:03.657+0000 7fe37e0a2740 1 -- 172.21.15.83:0/1931884147 wait complete. 2024-08-19T01:39:03.660 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:39:03.659+0000 7fe37e0a2740 20 client.15693 trim_cache size 0 max 16384 2024-08-19T01:39:03.952 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:39:03 smithi117 ceph-mon[22016]: scrub summary: active paths [/] 2024-08-19T01:39:03.952 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:39:03 smithi117 ceph-mon[22016]: scrub summary: idle+waiting paths [/] 2024-08-19T01:39:03.965 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:39:03 smithi167 ceph-mon[21875]: scrub summary: active paths [/] 2024-08-19T01:39:03.965 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:39:03 smithi167 ceph-mon[21875]: scrub summary: idle+waiting paths [/] 2024-08-19T01:39:03.982 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:39:03.970+0000 7fe37e0a2740 1 Processor -- start 2024-08-19T01:39:03.982 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:39:03.976+0000 7fe37e0a2740 1 -- start start 2024-08-19T01:39:03.982 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:39:03.976+0000 7fe37e0a2740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5575e6cf9660 0x5575e6d03f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:39:03.982 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:39:03.976+0000 7fe37e0a2740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5575e6cf9fa0 0x5575e6d0af10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:39:03.982 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:39:03.976+0000 7fe37e0a2740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5575e6cff030 0x5575e6cf50b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:39:03.982 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:39:03.976+0000 7fe37e0a2740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fe3741b1b30 con 0x5575e6cf9fa0 2024-08-19T01:39:03.982 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:39:03.976+0000 7fe37e0a2740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fe3741af400 con 0x5575e6cf9660 2024-08-19T01:39:03.983 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:39:03.976+0000 7fe37e0a2740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fe3741ad250 con 0x5575e6cff030 2024-08-19T01:39:03.983 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:39:03.977+0000 7fe373400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5575e6cf9fa0 0x5575e6d0af10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:39:03.983 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:39:03.977+0000 7fe373400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5575e6cf9fa0 0x5575e6d0af10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:55106/0 (socket says 172.21.15.83:55106) 2024-08-19T01:39:03.983 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:39:03.977+0000 7fe373400640 1 -- 172.21.15.83:0/3777289359 learned_addr learned my addr 172.21.15.83:0/3777289359 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-08-19T01:39:03.983 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:39:03.978+0000 7fe373400640 1 -- 172.21.15.83:0/3777289359 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5575e6cff030 msgr2=0x5575e6cf50b0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-08-19T01:39:03.983 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:39:03.978+0000 7fe373400640 1 --2- 172.21.15.83:0/3777289359 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5575e6cff030 0x5575e6cf50b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:39:03.983 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:39:03.978+0000 7fe373400640 1 -- 172.21.15.83:0/3777289359 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5575e6cf9660 msgr2=0x5575e6d03f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:39:03.983 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:39:03.978+0000 7fe373400640 1 --2- 172.21.15.83:0/3777289359 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5575e6cf9660 0x5575e6d03f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:39:03.984 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:39:03.978+0000 7fe373400640 1 -- 172.21.15.83:0/3777289359 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5575e6cf4af0 con 0x5575e6cf9fa0 2024-08-19T01:39:03.984 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:39:03.978+0000 7fe373400640 1 --2- 172.21.15.83:0/3777289359 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5575e6cf9fa0 0x5575e6d0af10 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7fe36400ba50 tx=0x7fe36400b570 comp rx=0 tx=0).ready entity=mon.0 client_cookie=adb0705a7e818903 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:39:03.984 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:39:03.979+0000 7fe368c00640 1 -- 172.21.15.83:0/3777289359 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe364041070 con 0x5575e6cf9fa0 2024-08-19T01:39:03.984 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:39:03.979+0000 7fe368c00640 1 -- 172.21.15.83:0/3777289359 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe364015db0 con 0x5575e6cf9fa0 2024-08-19T01:39:03.984 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:39:03.979+0000 7fe368c00640 1 -- 172.21.15.83:0/3777289359 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe364044cd0 con 0x5575e6cf9fa0 2024-08-19T01:39:03.995 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:39:03.985+0000 7fe37e0a2740 1 -- 172.21.15.83:0/3777289359 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5575e6cf9fa0 msgr2=0x5575e6d0af10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:39:03.995 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:39:03.985+0000 7fe37e0a2740 1 --2- 172.21.15.83:0/3777289359 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5575e6cf9fa0 0x5575e6d0af10 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7fe36400ba50 tx=0x7fe36400b570 comp rx=0 tx=0).stop 2024-08-19T01:39:03.995 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:39:03.991+0000 7fe37e0a2740 1 -- 172.21.15.83:0/3777289359 shutdown_connections 2024-08-19T01:39:03.995 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:39:03.991+0000 7fe37e0a2740 1 --2- 172.21.15.83:0/3777289359 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5575e6cff030 0x5575e6cf50b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:39:03.995 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:39:03.991+0000 7fe37e0a2740 1 --2- 172.21.15.83:0/3777289359 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5575e6cf9fa0 0x5575e6d0af10 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:39:03.995 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:39:03.991+0000 7fe37e0a2740 1 --2- 172.21.15.83:0/3777289359 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5575e6cf9660 0x5575e6d03f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:39:03.996 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:39:03.991+0000 7fe37e0a2740 1 -- 172.21.15.83:0/3777289359 >> 172.21.15.83:0/3777289359 conn(0x5575e6cf4cd0 msgr2=0x5575e6d03820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:39:04.001 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:39:04.000+0000 7fe37e0a2740 1 -- 172.21.15.83:0/3777289359 shutdown_connections 2024-08-19T01:39:04.009 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:39:04.007+0000 7fe37e0a2740 1 -- 172.21.15.83:0/3777289359 wait complete. 2024-08-19T01:39:04.009 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:39:04.007+0000 7fe37e0a2740 1 -- start start 2024-08-19T01:39:04.009 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:39:04.007+0000 7fe37e0a2740 1 Processor -- start 2024-08-19T01:39:04.009 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:39:04.008+0000 7fe37e0a2740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5575e6cf9660 0x5575e6d0ad10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:39:04.009 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:39:04.008+0000 7fe37e0a2740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5575e6cf9fa0 0x5575e6d07820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:39:04.010 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:39:04.008+0000 7fe37e0a2740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5575e6cff030 0x5575e6d07d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:39:04.010 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:39:04.009+0000 7fe373400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5575e6cf9fa0 0x5575e6d07820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:39:04.010 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:39:04.009+0000 7fe373400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5575e6cf9fa0 0x5575e6d07820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:55110/0 (socket says 172.21.15.83:55110) 2024-08-19T01:39:04.011 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:39:04.009+0000 7fe37e0a2740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fe3741ad8c0 con 0x5575e6cf9fa0 2024-08-19T01:39:04.012 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:39:04.009+0000 7fe37e0a2740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fe3741aed30 con 0x5575e6cff030 2024-08-19T01:39:04.012 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:39:04.009+0000 7fe37e0a2740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fe3741adf90 con 0x5575e6cf9660 2024-08-19T01:39:04.012 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:39:04.009+0000 7fe37e0a2740 10 client.0 _ll_register_callbacks cb 0x7fe36000aa50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-19T01:39:04.012 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:39:04.009+0000 7fe373400640 1 -- 172.21.15.83:0/3955296112 learned_addr learned my addr 172.21.15.83:0/3955296112 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-08-19T01:39:04.012 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:39:04.009+0000 7fe379200640 1 --2- 172.21.15.83:0/3955296112 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5575e6cf9660 0x5575e6d0ad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:39:04.012 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:39:04.010+0000 7fe379200640 1 --2- 172.21.15.83:0/3955296112 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5575e6cff030 0x5575e6d07d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:39:04.012 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:39:04.010+0000 7fe373400640 1 -- 172.21.15.83:0/3955296112 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5575e6cf9660 msgr2=0x5575e6d0ad10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:39:04.012 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:39:04.010+0000 7fe373400640 1 --2- 172.21.15.83:0/3955296112 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5575e6cf9660 0x5575e6d0ad10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:39:04.012 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:39:04.010+0000 7fe373400640 1 -- 172.21.15.83:0/3955296112 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5575e6cff030 msgr2=0x5575e6d07d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:39:04.012 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:39:04.010+0000 7fe373400640 1 --2- 172.21.15.83:0/3955296112 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5575e6cff030 0x5575e6d07d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:39:04.012 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:39:04.010+0000 7fe373400640 1 -- 172.21.15.83:0/3955296112 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5575e6d08400 con 0x5575e6cf9fa0 2024-08-19T01:39:04.013 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:39:04.010+0000 7fe373400640 1 --2- 172.21.15.83:0/3955296112 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5575e6cf9fa0 0x5575e6d07820 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7fe364008010 tx=0x7fe364015b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fc7288d7952733d server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:39:04.025 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:39:04.022+0000 7fe35e000640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-08-19T01:39:04.025 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:39:04.022+0000 7fe35e000640 1 -- 172.21.15.83:0/3955296112 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe36404b020 con 0x5575e6cf9fa0 2024-08-19T01:39:04.025 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:39:04.022+0000 7fe35e000640 1 -- 172.21.15.83:0/3955296112 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8269+0+0 (secure 0 0 0) 0x7fe364044c60 con 0x5575e6cf9fa0 2024-08-19T01:39:04.027 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:39:04.026+0000 7fe37e0a2740 1 -- 172.21.15.83:0/3955296112 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe364015db0 con 0x5575e6cf9fa0 2024-08-19T01:39:04.028 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:39:04.026+0000 7fe35e000640 1 -- 172.21.15.83:0/3955296112 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fe364041040 con 0x5575e6cf9fa0 2024-08-19T01:39:04.028 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:39:04.026+0000 7fe37e0a2740 10 client.15699 fetch_fsmap learned FSMap version 27 2024-08-19T01:39:04.028 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:39:04.026+0000 7fe37e0a2740 1 -- 172.21.15.83:0/3955296112 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5575e6d79180 con 0x5575e6cf9fa0 2024-08-19T01:39:04.029 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:39:04.027+0000 7fe35e000640 1 -- 172.21.15.83:0/3955296112 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe364015db0 con 0x5575e6cf9fa0 2024-08-19T01:39:04.029 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:39:04.027+0000 7fe35e000640 1 -- 172.21.15.83:0/3955296112 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7fe36404f540 con 0x5575e6cf9fa0 2024-08-19T01:39:04.029 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:39:04.027+0000 7fe37e0a2740 10 client.15699 fetch_fsmap finished waiting for FSMap version 27 2024-08-19T01:39:04.029 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:39:04.027+0000 7fe37e0a2740 10 client.15699 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-08-19T01:39:04.029 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:39:04.027+0000 7fe37e0a2740 20 client.15699 populate_metadata read hostname 'smithi083' 2024-08-19T01:39:04.029 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:39:04.027+0000 7fe37e0a2740 1 --2- 172.21.15.83:0/3955296112 >> [v2:172.21.15.117:6838/1967595704,v1:172.21.15.117:6839/1967595704] conn(0x5575e6d79b20 0x5575e6d7bf40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:39:04.029 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:39:04.027+0000 7fe37e0a2740 4 client.15699 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-08-19T01:39:04.029 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:39:04.027+0000 7fe37e0a2740 1 -- 172.21.15.83:0/3955296112 --> [v2:172.21.15.117:6838/1967595704,v1:172.21.15.117:6839/1967595704] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5575e6c65310 con 0x5575e6d79b20 2024-08-19T01:39:04.030 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:39:03 smithi083 ceph-mon[17120]: scrub summary: active paths [/] 2024-08-19T01:39:04.030 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:39:03 smithi083 ceph-mon[17120]: scrub summary: idle+waiting paths [/] 2024-08-19T01:39:04.035 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:39:04.028+0000 7fe379200640 1 --2- 172.21.15.83:0/3955296112 >> [v2:172.21.15.117:6838/1967595704,v1:172.21.15.117:6839/1967595704] conn(0x5575e6d79b20 0x5575e6d7bf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:39:04.035 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:39:04.029+0000 7fe379200640 1 --2- 172.21.15.83:0/3955296112 >> [v2:172.21.15.117:6838/1967595704,v1:172.21.15.117:6839/1967595704] conn(0x5575e6d79b20 0x5575e6d7bf40 crc :-1 s=READY pgs=92 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6f01eff7145c7c82 server_cookie=d080f25dc14f8f89 in_seq=0 out_seq=0 2024-08-19T01:39:04.035 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:39:04.029+0000 7fe35e000640 10 client.15699 ms_handle_connect on v2:172.21.15.117:6838/1967595704 2024-08-19T01:39:04.915 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:39:04 smithi083 ceph-mon[17120]: pgmap v466: 97 pgs: 97 active+clean; 374 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 5.5 MiB/s wr, 449 op/s 2024-08-19T01:39:04.965 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:39:04 smithi167 ceph-mon[21875]: pgmap v466: 97 pgs: 97 active+clean; 374 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 5.5 MiB/s wr, 449 op/s 2024-08-19T01:39:05.202 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:39:04 smithi117 ceph-mon[22016]: pgmap v466: 97 pgs: 97 active+clean; 374 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 5.5 MiB/s wr, 449 op/s 2024-08-19T01:39:07.360 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:39:07.350+0000 7fe35e000640 1 -- 172.21.15.83:0/3955296112 <== mds.0 v2:172.21.15.117:6838/1967595704 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5575e6c65310 con 0x5575e6d79b20 2024-08-19T01:39:07.360 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:39:07.350+0000 7fe35e000640 10 client.15699 handle_command_reply: tid=0 2024-08-19T01:39:07.360 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:39:07.350+0000 7fe37e0a2740 1 client.15699 shutdown 2024-08-19T01:39:07.363 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:39:07.361+0000 7fe37e0a2740 1 -- 172.21.15.83:0/3955296112 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5575e6cf9fa0 msgr2=0x5575e6d07820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:39:07.363 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:39:07.361+0000 7fe37e0a2740 1 --2- 172.21.15.83:0/3955296112 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5575e6cf9fa0 0x5575e6d07820 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7fe364008010 tx=0x7fe364015b00 comp rx=0 tx=0).stop 2024-08-19T01:39:07.368 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:39:07.362+0000 7fe37e0a2740 1 -- 172.21.15.83:0/3955296112 shutdown_connections 2024-08-19T01:39:07.368 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:39:07.362+0000 7fe37e0a2740 1 --2- 172.21.15.83:0/3955296112 >> [v2:172.21.15.117:6838/1967595704,v1:172.21.15.117:6839/1967595704] conn(0x5575e6d79b20 0x5575e6d7bf40 crc :-1 s=READY pgs=92 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:39:07.368 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:39:07.362+0000 7fe37e0a2740 1 --2- 172.21.15.83:0/3955296112 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5575e6cff030 0x5575e6d07d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:39:07.369 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:39:07.362+0000 7fe35e000640 0 client.15699 ms_handle_reset on v2:172.21.15.117:6838/1967595704 2024-08-19T01:39:07.369 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:39:07.362+0000 7fe37e0a2740 1 --2- 172.21.15.83:0/3955296112 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5575e6cf9fa0 0x5575e6d07820 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:39:07.369 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:39:07.362+0000 7fe37e0a2740 1 --2- 172.21.15.83:0/3955296112 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5575e6cf9660 0x5575e6d0ad10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:39:07.369 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:39:07.362+0000 7fe37e0a2740 1 -- 172.21.15.83:0/3955296112 >> 172.21.15.83:0/3955296112 conn(0x5575e6cf4cd0 msgr2=0x5575e6cf3720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:39:07.369 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:39:07.364+0000 7fe37e0a2740 1 -- 172.21.15.83:0/3955296112 shutdown_connections 2024-08-19T01:39:07.369 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:39:07.366+0000 7fe37e0a2740 1 -- 172.21.15.83:0/3955296112 wait complete. 2024-08-19T01:39:07.370 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:39:07.368+0000 7fe37e0a2740 20 client.15699 trim_cache size 0 max 16384 2024-08-19T01:39:07.370 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stdout:{ 2024-08-19T01:39:07.371 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stdout: "message": "", 2024-08-19T01:39:07.371 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stdout: "return_code": 0 2024-08-19T01:39:07.371 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stdout:} 2024-08-19T01:39:07.450 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:39:07.415+0000 7fe379c00640 1 -- 172.21.15.83:0/3775152390 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7fe3580a2d90 msgr2=0x7fe3580a5250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:39:07.450 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:39:07.415+0000 7fe379c00640 1 --2- 172.21.15.83:0/3775152390 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7fe3580a2d90 0x7fe3580a5250 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fe36c0d58e0 tx=0x7fe36c01a5d0 comp rx=0 tx=0).stop 2024-08-19T01:39:07.450 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:39:07.415+0000 7fe379c00640 1 -- 172.21.15.83:0/3775152390 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe3740729e0 msgr2=0x7fe37419e420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:39:07.450 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:39:07.415+0000 7fe379c00640 1 --2- 172.21.15.83:0/3775152390 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe3740729e0 0x7fe37419e420 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7fe36c006010 tx=0x7fe36c045790 comp rx=0 tx=0).stop 2024-08-19T01:39:07.450 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:39:07.434+0000 7fe379c00640 1 -- 172.21.15.83:0/3775152390 shutdown_connections 2024-08-19T01:39:07.451 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:39:07.434+0000 7fe379c00640 1 --2- 172.21.15.83:0/3775152390 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7fe3580a2d90 0x7fe3580a5250 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:39:07.451 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:39:07.434+0000 7fe379c00640 1 --2- 172.21.15.83:0/3775152390 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe37410d8e0 0x7fe3741a5520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:39:07.451 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:39:07.434+0000 7fe379c00640 1 --2- 172.21.15.83:0/3775152390 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe3740733b0 0x7fe37419e960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:39:07.451 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:39:07.434+0000 7fe379c00640 1 --2- 172.21.15.83:0/3775152390 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe3740729e0 0x7fe37419e420 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:39:07.451 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:39:07.434+0000 7fe379c00640 1 -- 172.21.15.83:0/3775152390 >> 172.21.15.83:0/3775152390 conn(0x7fe37406e880 msgr2=0x7fe374070510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:39:07.451 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:39:07.434+0000 7fe379c00640 1 -- 172.21.15.83:0/3775152390 shutdown_connections 2024-08-19T01:39:07.451 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:39:07.441+0000 7fe379c00640 1 -- 172.21.15.83:0/3775152390 wait complete. 2024-08-19T01:39:07.548 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:39:07 smithi083 ceph-mon[17120]: pgmap v467: 97 pgs: 97 active+clean; 379 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 1.9 MiB/s wr, 262 op/s 2024-08-19T01:39:07.952 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:39:07 smithi117 ceph-mon[22016]: pgmap v467: 97 pgs: 97 active+clean; 379 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 1.9 MiB/s wr, 262 op/s 2024-08-19T01:39:07.965 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:39:07 smithi167 ceph-mon[21875]: pgmap v467: 97 pgs: 97 active+clean; 379 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 1.9 MiB/s wr, 262 op/s 2024-08-19T01:39:08.852 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/bad8780e-5dc9-11ef-bccf-c7b262605968/mon.a/config 2024-08-19T01:39:09.869 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:39:09 smithi083 ceph-mon[17120]: pgmap v468: 97 pgs: 97 active+clean; 375 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.3 MiB/s wr, 790 op/s 2024-08-19T01:39:09.952 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:39:09 smithi117 ceph-mon[22016]: pgmap v468: 97 pgs: 97 active+clean; 375 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.3 MiB/s wr, 790 op/s 2024-08-19T01:39:09.965 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:39:09 smithi167 ceph-mon[21875]: pgmap v468: 97 pgs: 97 active+clean; 375 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.3 MiB/s wr, 790 op/s 2024-08-19T01:39:10.620 INFO:tasks.workunit.client.0.smithi083.stdout: 131 596843 545 390925 315 336376 1198 2024-08-19T01:39:11.401 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:39:11.398+0000 7f6e1ec00640 1 -- 172.21.15.83:0/590865836 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6e1806e220 msgr2=0x7f6e1806e660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:39:11.401 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:39:11.398+0000 7f6e1ec00640 1 --2- 172.21.15.83:0/590865836 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6e1806e220 0x7f6e1806e660 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f6e080097d0 tx=0x7f6e0802a510 comp rx=0 tx=0).stop 2024-08-19T01:39:11.401 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:39:11.398+0000 7f6e1ec00640 1 -- 172.21.15.83:0/590865836 shutdown_connections 2024-08-19T01:39:11.401 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:39:11.398+0000 7f6e1ec00640 1 --2- 172.21.15.83:0/590865836 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6e1806eba0 0x7f6e1807a180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:39:11.401 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:39:11.398+0000 7f6e1ec00640 1 --2- 172.21.15.83:0/590865836 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6e1806e220 0x7f6e1806e660 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:39:11.401 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:39:11.398+0000 7f6e1ec00640 1 --2- 172.21.15.83:0/590865836 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6e1810ddc0 0x7f6e1810e1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:39:11.401 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:39:11.398+0000 7f6e1ec00640 1 -- 172.21.15.83:0/590865836 >> 172.21.15.83:0/590865836 conn(0x7f6e18072eb0 msgr2=0x7f6e180732c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:39:11.404 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:39:11.401+0000 7f6e1ec00640 1 -- 172.21.15.83:0/590865836 shutdown_connections 2024-08-19T01:39:11.404 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:39:11.401+0000 7f6e1ec00640 1 -- 172.21.15.83:0/590865836 wait complete. 2024-08-19T01:39:11.404 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:39:11.402+0000 7f6e1ec00640 1 Processor -- start 2024-08-19T01:39:11.404 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:39:11.403+0000 7f6e1ec00640 1 -- start start 2024-08-19T01:39:11.410 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:39:11.407+0000 7f6e1ec00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6e1806e220 0x7f6e18084b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:39:11.410 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:39:11.407+0000 7f6e1ec00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6e1806eba0 0x7f6e1807ecc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:39:11.410 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:39:11.407+0000 7f6e1ec00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6e1810ddc0 0x7f6e1807f400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:39:11.411 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:39:11.407+0000 7f6e1ec00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f6e1807d7b0 con 0x7f6e1806eba0 2024-08-19T01:39:11.411 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:39:11.407+0000 7f6e1ec00640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f6e1807d630 con 0x7f6e1806e220 2024-08-19T01:39:11.411 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:39:11.407+0000 7f6e1ec00640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f6e1807d930 con 0x7f6e1810ddc0 2024-08-19T01:39:11.411 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:39:11.407+0000 7f6e1d800640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6e1806e220 0x7f6e18084b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:39:11.411 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:39:11.408+0000 7f6e1d800640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6e1806e220 0x7f6e18084b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.83:41922/0 (socket says 172.21.15.83:41922) 2024-08-19T01:39:11.411 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:39:11.408+0000 7f6e1d800640 1 -- 172.21.15.83:0/3402408160 learned_addr learned my addr 172.21.15.83:0/3402408160 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-08-19T01:39:11.411 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:39:11.408+0000 7f6e1d800640 1 -- 172.21.15.83:0/3402408160 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6e1810ddc0 msgr2=0x7f6e1807f400 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-08-19T01:39:11.411 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:39:11.408+0000 7f6e1d800640 1 --2- 172.21.15.83:0/3402408160 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6e1810ddc0 0x7f6e1807f400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:39:11.411 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:39:11.408+0000 7f6e1d800640 1 -- 172.21.15.83:0/3402408160 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6e1806eba0 msgr2=0x7f6e1807ecc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:39:11.415 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:39:11.411+0000 7f6e1ce00640 1 --2- 172.21.15.83:0/3402408160 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6e1806eba0 0x7f6e1807ecc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:39:11.415 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:39:11.411+0000 7f6e1d800640 1 --2- 172.21.15.83:0/3402408160 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6e1806eba0 0x7f6e1807ecc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:39:11.415 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:39:11.411+0000 7f6e1d800640 1 -- 172.21.15.83:0/3402408160 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6e181c40c0 con 0x7f6e1806e220 2024-08-19T01:39:11.415 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:39:11.413+0000 7f6e1ce00640 1 --2- 172.21.15.83:0/3402408160 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6e1806eba0 0x7f6e1807ecc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-19T01:39:11.416 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:39:11.415+0000 7f6e1d800640 1 --2- 172.21.15.83:0/3402408160 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6e1806e220 0x7f6e18084b90 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f6e1000eb90 tx=0x7f6e1000c6a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:39:11.419 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:39:11.418+0000 7f6e16000640 1 -- 172.21.15.83:0/3402408160 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e10004d20 con 0x7f6e1806e220 2024-08-19T01:39:11.419 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:39:11.418+0000 7f6e16000640 1 -- 172.21.15.83:0/3402408160 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6e1000ee90 con 0x7f6e1806e220 2024-08-19T01:39:11.419 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:39:11.418+0000 7f6e16000640 1 -- 172.21.15.83:0/3402408160 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e10010830 con 0x7f6e1806e220 2024-08-19T01:39:11.422 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:39:11.419+0000 7f6e1ec00640 1 -- 172.21.15.83:0/3402408160 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6e181c4350 con 0x7f6e1806e220 2024-08-19T01:39:11.423 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:39:11.419+0000 7f6e1ec00640 1 -- 172.21.15.83:0/3402408160 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6e181c4810 con 0x7f6e1806e220 2024-08-19T01:39:11.456 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:39:11.437+0000 7f6e22ca5740 1 Processor -- start 2024-08-19T01:39:11.458 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:39:11.437+0000 7f6e22ca5740 1 -- start start 2024-08-19T01:39:11.458 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:39:11.437+0000 7f6e22ca5740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556b07874c40 0x556b07875020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:39:11.459 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:39:11.437+0000 7f6e22ca5740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556b07875560 0x556b07890a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:39:11.459 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:39:11.438+0000 7f6e22ca5740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556b07890f70 0x556b07893360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:39:11.459 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:39:11.438+0000 7f6e1e200640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556b07874c40 0x556b07875020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:39:11.459 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:39:11.438+0000 7f6e1e200640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556b07874c40 0x556b07875020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.83:41932/0 (socket says 172.21.15.83:41932) 2024-08-19T01:39:11.459 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:39:11.438+0000 7f6e1e200640 1 -- 172.21.15.83:0/2498105483 learned_addr learned my addr 172.21.15.83:0/2498105483 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-08-19T01:39:11.459 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:39:11.438+0000 7f6e22ca5740 1 -- 172.21.15.83:0/2498105483 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x556b077dabf0 con 0x556b07875560 2024-08-19T01:39:11.459 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:39:11.438+0000 7f6e22ca5740 1 -- 172.21.15.83:0/2498105483 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x556b07772bd0 con 0x556b07874c40 2024-08-19T01:39:11.459 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:39:11.438+0000 7f6e22ca5740 1 -- 172.21.15.83:0/2498105483 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x556b077d6bb0 con 0x556b07890f70 2024-08-19T01:39:11.459 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:39:11.438+0000 7f6e1ce00640 1 --2- 172.21.15.83:0/2498105483 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556b07890f70 0x556b07893360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:39:11.459 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:39:11.439+0000 7f6e1e200640 1 -- 172.21.15.83:0/2498105483 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556b07890f70 msgr2=0x556b07893360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:39:11.459 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:39:11.439+0000 7f6e1e200640 1 --2- 172.21.15.83:0/2498105483 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556b07890f70 0x556b07893360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:39:11.459 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:39:11.439+0000 7f6e1e200640 1 -- 172.21.15.83:0/2498105483 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556b07875560 msgr2=0x556b07890a30 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-08-19T01:39:11.459 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:39:11.439+0000 7f6e1e200640 1 --2- 172.21.15.83:0/2498105483 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556b07875560 0x556b07890a30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:39:11.459 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:39:11.439+0000 7f6e1e200640 1 -- 172.21.15.83:0/2498105483 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556b077798a0 con 0x556b07874c40 2024-08-19T01:39:11.460 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:39:11.439+0000 7f6e1e200640 1 --2- 172.21.15.83:0/2498105483 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556b07874c40 0x556b07875020 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f6e0c00b9c0 tx=0x7f6e0c00be80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7881fc26d99c9c21 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:39:11.460 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:39:11.439+0000 7f6e06a00640 1 -- 172.21.15.83:0/2498105483 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e0c03a020 con 0x556b07874c40 2024-08-19T01:39:11.460 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:39:11.439+0000 7f6e06a00640 1 -- 172.21.15.83:0/2498105483 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6e0c005500 con 0x556b07874c40 2024-08-19T01:39:11.460 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:39:11.440+0000 7f6e22ca5740 1 -- 172.21.15.83:0/2498105483 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556b07874c40 msgr2=0x556b07875020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:39:11.460 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:39:11.440+0000 7f6e22ca5740 1 --2- 172.21.15.83:0/2498105483 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556b07874c40 0x556b07875020 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f6e0c00b9c0 tx=0x7f6e0c00be80 comp rx=0 tx=0).stop 2024-08-19T01:39:11.460 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:39:11.441+0000 7f6e16000640 1 -- 172.21.15.83:0/3402408160 <== mon.2 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7f6e10005000 con 0x7f6e1806e220 2024-08-19T01:39:11.460 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:39:11.441+0000 7f6e22ca5740 1 -- 172.21.15.83:0/2498105483 shutdown_connections 2024-08-19T01:39:11.460 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:39:11.441+0000 7f6e22ca5740 1 --2- 172.21.15.83:0/2498105483 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556b07890f70 0x556b07893360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:39:11.460 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:39:11.441+0000 7f6e22ca5740 1 --2- 172.21.15.83:0/2498105483 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556b07875560 0x556b07890a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:39:11.461 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:39:11.441+0000 7f6e22ca5740 1 --2- 172.21.15.83:0/2498105483 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556b07874c40 0x556b07875020 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:39:11.461 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:39:11.441+0000 7f6e22ca5740 1 -- 172.21.15.83:0/2498105483 >> 172.21.15.83:0/2498105483 conn(0x556b07871310 msgr2=0x556b078716f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:39:11.461 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:39:11.443+0000 7f6e16000640 1 --2- 172.21.15.83:0/3402408160 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f6e000a2c00 0x7f6e000a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:39:11.461 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:39:11.443+0000 7f6e1ce00640 1 --2- 172.21.15.83:0/3402408160 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f6e000a2c00 0x7f6e000a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:39:11.461 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:39:11.445+0000 7f6e1ce00640 1 --2- 172.21.15.83:0/3402408160 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f6e000a2c00 0x7f6e000a50c0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f6e08005ab0 tx=0x7f6e08005a20 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:39:11.463 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:39:11.459+0000 7f6e16000640 1 -- 172.21.15.83:0/3402408160 <== mon.2 v2:172.21.15.117:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8269+0+0 (secure 0 0 0) 0x7f6e1009a070 con 0x7f6e1806e220 2024-08-19T01:39:11.463 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:39:11.460+0000 7f6e22ca5740 1 -- 172.21.15.83:0/2498105483 shutdown_connections 2024-08-19T01:39:11.463 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:39:11.460+0000 7f6e22ca5740 1 -- 172.21.15.83:0/2498105483 wait complete. 2024-08-19T01:39:11.463 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:39:11.460+0000 7f6e22ca5740 1 -- start start 2024-08-19T01:39:11.463 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:39:11.461+0000 7f6e22ca5740 1 Processor -- start 2024-08-19T01:39:11.468 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:39:11.462+0000 7f6e22ca5740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556b07874c40 0x556b07882310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:39:11.468 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:39:11.463+0000 7f6e22ca5740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556b07890f70 0x556b07882850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:39:11.468 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:39:11.463+0000 7f6e1d800640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556b07874c40 0x556b07882310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:39:11.468 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:39:11.463+0000 7f6e1d800640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556b07874c40 0x556b07882310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.83:41936/0 (socket says 172.21.15.83:41936) 2024-08-19T01:39:11.468 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:39:11.464+0000 7f6e22ca5740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556b0787d630 0x556b0787da90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:39:11.468 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:39:11.464+0000 7f6e1e200640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556b0787d630 0x556b0787da90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:39:11.468 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:39:11.465+0000 7f6e1ce00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556b07890f70 0x556b07882850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:39:11.468 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:39:11.465+0000 7f6e1ce00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556b07890f70 0x556b07882850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.83:45418/0 (socket says 172.21.15.83:45418) 2024-08-19T01:39:11.469 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:39:11.465+0000 7f6e1e200640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556b0787d630 0x556b0787da90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:45622/0 (socket says 172.21.15.83:45622) 2024-08-19T01:39:11.469 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:39:11.465+0000 7f6e22ca5740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x556b07877ca0 con 0x556b0787d630 2024-08-19T01:39:11.469 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:39:11.465+0000 7f6e22ca5740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x556b07795d40 con 0x556b07874c40 2024-08-19T01:39:11.469 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:39:11.465+0000 7f6e22ca5740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x556b07877e20 con 0x556b07890f70 2024-08-19T01:39:11.469 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:39:11.466+0000 7f6e1d800640 1 -- 172.21.15.83:0/723643608 learned_addr learned my addr 172.21.15.83:0/723643608 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-08-19T01:39:11.469 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:39:11.466+0000 7f6e22ca5740 10 client.0 _ll_register_callbacks cb 0x556b0775eb10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-19T01:39:11.470 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:39:11.467+0000 7f6e1e200640 1 -- 172.21.15.83:0/723643608 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556b07890f70 msgr2=0x556b07882850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:39:11.470 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:39:11.467+0000 7f6e1e200640 1 --2- 172.21.15.83:0/723643608 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556b07890f70 0x556b07882850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:39:11.470 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:39:11.467+0000 7f6e1e200640 1 -- 172.21.15.83:0/723643608 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556b07874c40 msgr2=0x556b07882310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:39:11.471 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:39:11.467+0000 7f6e1e200640 1 --2- 172.21.15.83:0/723643608 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556b07874c40 0x556b07882310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:39:11.471 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:39:11.467+0000 7f6e1e200640 1 -- 172.21.15.83:0/723643608 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556b0754c3c0 con 0x556b0787d630 2024-08-19T01:39:11.471 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:39:11.468+0000 7f6e1d800640 1 --2- 172.21.15.83:0/723643608 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556b07874c40 0x556b07882310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-19T01:39:11.482 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:39:11.472+0000 7f6e1e200640 1 --2- 172.21.15.83:0/723643608 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556b0787d630 0x556b0787da90 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f6e0c004a00 tx=0x7f6e0c004f80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e3a43a9fd77c4c3e server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:39:11.482 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:39:11.472+0000 7f6dfb400640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-08-19T01:39:11.482 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:39:11.472+0000 7f6dfb400640 1 -- 172.21.15.83:0/723643608 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e0c046020 con 0x556b0787d630 2024-08-19T01:39:11.482 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:39:11.472+0000 7f6e22ca5740 1 -- 172.21.15.83:0/723643608 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6e0c005500 con 0x556b0787d630 2024-08-19T01:39:11.482 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:39:11.472+0000 7f6dfb400640 1 -- 172.21.15.83:0/723643608 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8269+0+0 (secure 0 0 0) 0x7f6e0c00ab60 con 0x556b0787d630 2024-08-19T01:39:11.482 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:39:11.473+0000 7f6dfb400640 1 -- 172.21.15.83:0/723643608 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f6e0c046020 con 0x556b0787d630 2024-08-19T01:39:11.482 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:39:11.473+0000 7f6e22ca5740 10 client.15705 fetch_fsmap learned FSMap version 27 2024-08-19T01:39:11.482 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:39:11.473+0000 7f6e22ca5740 1 -- 172.21.15.83:0/723643608 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x556b0788c390 con 0x556b0787d630 2024-08-19T01:39:11.483 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:39:11.474+0000 7f6dfb400640 1 -- 172.21.15.83:0/723643608 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6e0c005500 con 0x556b0787d630 2024-08-19T01:39:11.483 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:39:11.474+0000 7f6dfb400640 1 -- 172.21.15.83:0/723643608 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f6e0c03f050 con 0x556b0787d630 2024-08-19T01:39:11.483 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:39:11.474+0000 7f6e22ca5740 10 client.15705 fetch_fsmap finished waiting for FSMap version 27 2024-08-19T01:39:11.483 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:39:11.475+0000 7f6e22ca5740 10 client.15705 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-19T01:39:11.483 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:39:11.475+0000 7f6e22ca5740 20 client.15705 populate_metadata read hostname 'smithi083' 2024-08-19T01:39:11.483 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:39:11.475+0000 7f6e22ca5740 1 --2- 172.21.15.83:0/723643608 >> [v2:172.21.15.117:6838/1967595704,v1:172.21.15.117:6839/1967595704] conn(0x556b0788d740 0x556b0788fbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:39:11.483 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:39:11.475+0000 7f6e22ca5740 4 client.15705 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-19T01:39:11.483 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:39:11.475+0000 7f6e22ca5740 1 -- 172.21.15.83:0/723643608 --> [v2:172.21.15.117:6838/1967595704,v1:172.21.15.117:6839/1967595704] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556b077e8160 con 0x556b0788d740 2024-08-19T01:39:11.483 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:39:11.476+0000 7f6e1d800640 1 --2- 172.21.15.83:0/723643608 >> [v2:172.21.15.117:6838/1967595704,v1:172.21.15.117:6839/1967595704] conn(0x556b0788d740 0x556b0788fbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:39:11.483 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:39:11.476+0000 7f6e1d800640 1 --2- 172.21.15.83:0/723643608 >> [v2:172.21.15.117:6838/1967595704,v1:172.21.15.117:6839/1967595704] conn(0x556b0788d740 0x556b0788fbc0 crc :-1 s=READY pgs=93 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ebd2f2f19d4f9257 server_cookie=e278e198d265fa05 in_seq=0 out_seq=0 2024-08-19T01:39:11.483 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:39:11.476+0000 7f6dfb400640 10 client.15705 ms_handle_connect on v2:172.21.15.117:6838/1967595704 2024-08-19T01:39:11.487 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:39:11.482+0000 7f6dfb400640 1 -- 172.21.15.83:0/723643608 <== mds.0 v2:172.21.15.117:6838/1967595704 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x556b077e8160 con 0x556b0788d740 2024-08-19T01:39:11.487 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:39:11.482+0000 7f6dfb400640 10 client.15705 handle_command_reply: tid=0 2024-08-19T01:39:11.487 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:39:11.482+0000 7f6e22ca5740 1 client.15705 shutdown 2024-08-19T01:39:11.487 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:39:11.483+0000 7f6e22ca5740 1 -- 172.21.15.83:0/723643608 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556b0787d630 msgr2=0x556b0787da90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:39:11.487 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:39:11.483+0000 7f6e22ca5740 1 --2- 172.21.15.83:0/723643608 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556b0787d630 0x556b0787da90 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f6e0c004a00 tx=0x7f6e0c004f80 comp rx=0 tx=0).stop 2024-08-19T01:39:11.487 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:39:11.483+0000 7f6e22ca5740 1 -- 172.21.15.83:0/723643608 shutdown_connections 2024-08-19T01:39:11.488 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:39:11.483+0000 7f6e22ca5740 1 --2- 172.21.15.83:0/723643608 >> [v2:172.21.15.117:6838/1967595704,v1:172.21.15.117:6839/1967595704] conn(0x556b0788d740 0x556b0788fbc0 crc :-1 s=READY pgs=93 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:39:11.488 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:39:11.483+0000 7f6e22ca5740 1 --2- 172.21.15.83:0/723643608 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556b0787d630 0x556b0787da90 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:39:11.488 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:39:11.483+0000 7f6e22ca5740 1 --2- 172.21.15.83:0/723643608 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556b07890f70 0x556b07882850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:39:11.488 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:39:11.483+0000 7f6dfb400640 0 client.15705 ms_handle_reset on v2:172.21.15.117:6838/1967595704 2024-08-19T01:39:11.488 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:39:11.483+0000 7f6e22ca5740 1 --2- 172.21.15.83:0/723643608 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556b07874c40 0x556b07882310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:39:11.488 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:39:11.483+0000 7f6e22ca5740 1 -- 172.21.15.83:0/723643608 >> 172.21.15.83:0/723643608 conn(0x556b07871310 msgr2=0x556b07877070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:39:11.488 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:39:11.484+0000 7f6e22ca5740 1 -- 172.21.15.83:0/723643608 shutdown_connections 2024-08-19T01:39:11.488 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:39:11.484+0000 7f6e22ca5740 1 -- 172.21.15.83:0/723643608 wait complete. 2024-08-19T01:39:11.488 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:39:11.484+0000 7f6e22ca5740 20 client.15705 trim_cache size 0 max 16384 2024-08-19T01:39:11.528 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:39:11.519+0000 7f6e22ca5740 1 Processor -- start 2024-08-19T01:39:11.528 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:39:11.519+0000 7f6e22ca5740 1 -- start start 2024-08-19T01:39:11.528 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:39:11.519+0000 7f6e22ca5740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556b07874c40 0x556b07882820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:39:11.529 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:39:11.519+0000 7f6e22ca5740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556b0787d630 0x556b07898930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:39:11.529 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:39:11.519+0000 7f6e22ca5740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556b07890f70 0x556b07898e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:39:11.529 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:39:11.519+0000 7f6e22ca5740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f6e181c71c0 con 0x556b07890f70 2024-08-19T01:39:11.529 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:39:11.519+0000 7f6e22ca5740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f6e181c4a90 con 0x556b07874c40 2024-08-19T01:39:11.529 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:39:11.519+0000 7f6e22ca5740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f6e1806e6a0 con 0x556b0787d630 2024-08-19T01:39:11.529 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:39:11.520+0000 7f6e1d800640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556b07874c40 0x556b07882820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:39:11.529 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:39:11.520+0000 7f6e1d800640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556b07874c40 0x556b07882820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.83:41948/0 (socket says 172.21.15.83:41948) 2024-08-19T01:39:11.529 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:39:11.520+0000 7f6e1d800640 1 -- 172.21.15.83:0/1521559376 learned_addr learned my addr 172.21.15.83:0/1521559376 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-08-19T01:39:11.529 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:39:11.520+0000 7f6e1e200640 1 --2- 172.21.15.83:0/1521559376 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556b07890f70 0x556b07898e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:39:11.529 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:39:11.520+0000 7f6e1d800640 1 -- 172.21.15.83:0/1521559376 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556b0787d630 msgr2=0x556b07898930 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-08-19T01:39:11.529 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:39:11.520+0000 7f6e1d800640 1 --2- 172.21.15.83:0/1521559376 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556b0787d630 0x556b07898930 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:39:11.529 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:39:11.520+0000 7f6e1d800640 1 -- 172.21.15.83:0/1521559376 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556b07890f70 msgr2=0x556b07898e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:39:11.529 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:39:11.520+0000 7f6e1d800640 1 --2- 172.21.15.83:0/1521559376 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556b07890f70 0x556b07898e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:39:11.530 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:39:11.520+0000 7f6e1d800640 1 -- 172.21.15.83:0/1521559376 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556b078776f0 con 0x556b07874c40 2024-08-19T01:39:11.530 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:39:11.520+0000 7f6e1e200640 1 --2- 172.21.15.83:0/1521559376 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556b07890f70 0x556b07898e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-19T01:39:11.530 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:39:11.521+0000 7f6e1d800640 1 --2- 172.21.15.83:0/1521559376 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556b07874c40 0x556b07882820 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f6e10098430 tx=0x7f6e100a3690 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cde73c5b58bda8a5 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:39:11.530 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:39:11.521+0000 7f6e06000640 1 -- 172.21.15.83:0/1521559376 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e10098a00 con 0x556b07874c40 2024-08-19T01:39:11.530 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:39:11.521+0000 7f6e06000640 1 -- 172.21.15.83:0/1521559376 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6e100bc0b0 con 0x556b07874c40 2024-08-19T01:39:11.530 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:39:11.521+0000 7f6e06000640 1 -- 172.21.15.83:0/1521559376 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e10098a00 con 0x556b07874c40 2024-08-19T01:39:11.530 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:39:11.522+0000 7f6e22ca5740 1 -- 172.21.15.83:0/1521559376 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556b07874c40 msgr2=0x556b07882820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:39:11.530 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:39:11.522+0000 7f6e22ca5740 1 --2- 172.21.15.83:0/1521559376 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556b07874c40 0x556b07882820 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f6e10098430 tx=0x7f6e100a3690 comp rx=0 tx=0).stop 2024-08-19T01:39:11.530 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:39:11.522+0000 7f6e22ca5740 1 -- 172.21.15.83:0/1521559376 shutdown_connections 2024-08-19T01:39:11.530 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:39:11.522+0000 7f6e22ca5740 1 --2- 172.21.15.83:0/1521559376 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556b07890f70 0x556b07898e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:39:11.530 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:39:11.522+0000 7f6e22ca5740 1 --2- 172.21.15.83:0/1521559376 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556b0787d630 0x556b07898930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:39:11.530 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:39:11.522+0000 7f6e22ca5740 1 --2- 172.21.15.83:0/1521559376 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556b07874c40 0x556b07882820 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:39:11.530 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:39:11.522+0000 7f6e22ca5740 1 -- 172.21.15.83:0/1521559376 >> 172.21.15.83:0/1521559376 conn(0x556b07871310 msgr2=0x556b07882110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:39:11.531 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:39:11.523+0000 7f6e22ca5740 1 -- 172.21.15.83:0/1521559376 shutdown_connections 2024-08-19T01:39:11.531 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:39:11.523+0000 7f6e22ca5740 1 -- 172.21.15.83:0/1521559376 wait complete. 2024-08-19T01:39:11.531 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:39:11.523+0000 7f6e22ca5740 1 -- start start 2024-08-19T01:39:11.531 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:39:11.524+0000 7f6e22ca5740 1 Processor -- start 2024-08-19T01:39:11.531 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:39:11.527+0000 7f6e22ca5740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556b07874c40 0x556b07884e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:39:11.531 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:39:11.527+0000 7f6e22ca5740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556b0787d630 0x556b07885380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:39:11.531 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:39:11.527+0000 7f6e22ca5740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556b07890f70 0x556b078858c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:39:11.532 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:39:11.527+0000 7f6e22ca5740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f6e181bc590 con 0x556b07890f70 2024-08-19T01:39:11.532 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:39:11.527+0000 7f6e22ca5740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f6e181bda00 con 0x556b0787d630 2024-08-19T01:39:11.532 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:39:11.527+0000 7f6e22ca5740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f6e181bcc60 con 0x556b07874c40 2024-08-19T01:39:11.532 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:39:11.528+0000 7f6e22ca5740 10 client.0 _ll_register_callbacks cb 0x7f6e08034070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-19T01:39:11.532 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:39:11.528+0000 7f6e1e200640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556b07890f70 0x556b078858c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:39:11.532 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:39:11.528+0000 7f6e1e200640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556b07890f70 0x556b078858c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:45628/0 (socket says 172.21.15.83:45628) 2024-08-19T01:39:11.532 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:39:11.528+0000 7f6e1e200640 1 -- 172.21.15.83:0/2115916687 learned_addr learned my addr 172.21.15.83:0/2115916687 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-08-19T01:39:11.532 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:39:11.529+0000 7f6e1e200640 1 -- 172.21.15.83:0/2115916687 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556b07874c40 msgr2=0x556b07884e40 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-08-19T01:39:11.532 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:39:11.529+0000 7f6e1e200640 1 --2- 172.21.15.83:0/2115916687 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556b07874c40 0x556b07884e40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:39:11.532 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:39:11.529+0000 7f6e1e200640 1 -- 172.21.15.83:0/2115916687 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556b0787d630 msgr2=0x556b07885380 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-08-19T01:39:11.533 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:39:11.529+0000 7f6e1e200640 1 --2- 172.21.15.83:0/2115916687 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556b0787d630 0x556b07885380 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:39:11.533 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:39:11.529+0000 7f6e1e200640 1 -- 172.21.15.83:0/2115916687 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556b078f98f0 con 0x556b07890f70 2024-08-19T01:39:11.533 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:39:11.530+0000 7f6e1e200640 1 --2- 172.21.15.83:0/2115916687 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556b07890f70 0x556b078858c0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f6e0c001eb0 tx=0x7f6e0c00af90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e2ca98323a754f38 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:39:11.533 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:39:11.530+0000 7f6dfb400640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-08-19T01:39:11.533 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:39:11.530+0000 7f6dfb400640 1 -- 172.21.15.83:0/2115916687 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e0c03a070 con 0x556b07890f70 2024-08-19T01:39:11.536 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:39:11.534+0000 7f6dfb400640 1 -- 172.21.15.83:0/2115916687 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8269+0+0 (secure 0 0 0) 0x7f6e0c042a20 con 0x556b07890f70 2024-08-19T01:39:11.536 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:39:11.534+0000 7f6e22ca5740 1 -- 172.21.15.83:0/2115916687 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x556b077e2670 con 0x556b07890f70 2024-08-19T01:39:11.536 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:39:11.534+0000 7f6dfb400640 1 -- 172.21.15.83:0/2115916687 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f6e0c063050 con 0x556b07890f70 2024-08-19T01:39:11.536 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:39:11.534+0000 7f6e22ca5740 10 client.15708 fetch_fsmap learned FSMap version 27 2024-08-19T01:39:11.536 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:39:11.534+0000 7f6e22ca5740 1 -- 172.21.15.83:0/2115916687 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x556b078fab20 con 0x556b07890f70 2024-08-19T01:39:11.537 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:39:11.535+0000 7f6dfb400640 1 -- 172.21.15.83:0/2115916687 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x556b077e2670 con 0x556b07890f70 2024-08-19T01:39:11.538 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:39:11.535+0000 7f6dfb400640 1 -- 172.21.15.83:0/2115916687 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f6e0c03f070 con 0x556b07890f70 2024-08-19T01:39:11.538 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:39:11.535+0000 7f6e22ca5740 10 client.15708 fetch_fsmap finished waiting for FSMap version 27 2024-08-19T01:39:11.539 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:39:11.535+0000 7f6e22ca5740 10 client.15708 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-19T01:39:11.539 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:39:11.536+0000 7f6e22ca5740 20 client.15708 populate_metadata read hostname 'smithi083' 2024-08-19T01:39:11.539 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:39:11.536+0000 7f6e22ca5740 1 --2- 172.21.15.83:0/2115916687 >> [v2:172.21.15.117:6838/1967595704,v1:172.21.15.117:6839/1967595704] conn(0x556b078fb270 0x556b078fd690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:39:11.539 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:39:11.536+0000 7f6e22ca5740 4 client.15708 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-08-19T01:39:11.539 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:39:11.536+0000 7f6e22ca5740 1 -- 172.21.15.83:0/2115916687 --> [v2:172.21.15.117:6838/1967595704,v1:172.21.15.117:6839/1967595704] -- command(tid 0: {"prefix": "scrub status"}) -- 0x556b0775f0c0 con 0x556b078fb270 2024-08-19T01:39:11.539 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:39:11.536+0000 7f6e1d800640 1 --2- 172.21.15.83:0/2115916687 >> [v2:172.21.15.117:6838/1967595704,v1:172.21.15.117:6839/1967595704] conn(0x556b078fb270 0x556b078fd690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:39:11.539 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:39:11.537+0000 7f6dfb400640 10 client.15708 ms_handle_connect on v2:172.21.15.117:6838/1967595704 2024-08-19T01:39:11.539 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:39:11.537+0000 7f6e1d800640 1 --2- 172.21.15.83:0/2115916687 >> [v2:172.21.15.117:6838/1967595704,v1:172.21.15.117:6839/1967595704] conn(0x556b078fb270 0x556b078fd690 crc :-1 s=READY pgs=94 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5a2367b72c50449a server_cookie=e31f0ac3897ee21e in_seq=0 out_seq=0 2024-08-19T01:39:11.546 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-08-19T01:39:11.546 INFO:teuthology.orchestra.run.smithi083.stdout: "status": "scrub active (16361 inodes in the stack)", 2024-08-19T01:39:11.546 INFO:teuthology.orchestra.run.smithi083.stdout: "scrubs": { 2024-08-19T01:39:11.546 INFO:teuthology.orchestra.run.smithi083.stdout: "0e436353-7799-40e1-97b3-a89eaf3bf27d": { 2024-08-19T01:39:11.546 INFO:teuthology.orchestra.run.smithi083.stdout: "path": "/", 2024-08-19T01:39:11.546 INFO:teuthology.orchestra.run.smithi083.stdout: "tag": "0e436353-7799-40e1-97b3-a89eaf3bf27d", 2024-08-19T01:39:11.547 INFO:teuthology.orchestra.run.smithi083.stdout: "options": "recursive,force" 2024-08-19T01:39:11.547 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-08-19T01:39:11.547 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-08-19T01:39:11.547 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-08-19T01:39:11.547 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:39:11.538+0000 7f6dfb400640 1 -- 172.21.15.83:0/2115916687 <== mds.0 v2:172.21.15.117:6838/1967595704 1 ==== command_reply(tid 0: 0 ) ==== 8+0+269 (crc 0 0 0) 0x556b0775f0c0 con 0x556b078fb270 2024-08-19T01:39:11.547 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:39:11.538+0000 7f6dfb400640 10 client.15708 handle_command_reply: tid=0 2024-08-19T01:39:11.547 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:39:11.538+0000 7f6e22ca5740 1 client.15708 shutdown 2024-08-19T01:39:11.547 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:39:11.539+0000 7f6e22ca5740 1 -- 172.21.15.83:0/2115916687 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556b07890f70 msgr2=0x556b078858c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:39:11.547 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:39:11.539+0000 7f6e22ca5740 1 --2- 172.21.15.83:0/2115916687 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556b07890f70 0x556b078858c0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f6e0c001eb0 tx=0x7f6e0c00af90 comp rx=0 tx=0).stop 2024-08-19T01:39:11.547 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:39:11.539+0000 7f6e22ca5740 1 -- 172.21.15.83:0/2115916687 shutdown_connections 2024-08-19T01:39:11.547 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:39:11.539+0000 7f6e22ca5740 1 --2- 172.21.15.83:0/2115916687 >> [v2:172.21.15.117:6838/1967595704,v1:172.21.15.117:6839/1967595704] conn(0x556b078fb270 0x556b078fd690 crc :-1 s=READY pgs=94 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:39:11.547 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:39:11.539+0000 7f6e22ca5740 1 --2- 172.21.15.83:0/2115916687 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556b07890f70 0x556b078858c0 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:39:11.548 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:39:11.539+0000 7f6e22ca5740 1 --2- 172.21.15.83:0/2115916687 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556b0787d630 0x556b07885380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:39:11.548 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:39:11.539+0000 7f6dfb400640 0 client.15708 ms_handle_reset on v2:172.21.15.117:6838/1967595704 2024-08-19T01:39:11.548 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:39:11.539+0000 7f6e22ca5740 1 --2- 172.21.15.83:0/2115916687 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556b07874c40 0x556b07884e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:39:11.548 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:39:11.539+0000 7f6e22ca5740 1 -- 172.21.15.83:0/2115916687 >> 172.21.15.83:0/2115916687 conn(0x556b07871310 msgr2=0x556b07876e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:39:11.548 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:39:11.540+0000 7f6e22ca5740 1 -- 172.21.15.83:0/2115916687 shutdown_connections 2024-08-19T01:39:11.548 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:39:11.540+0000 7f6e22ca5740 1 -- 172.21.15.83:0/2115916687 wait complete. 2024-08-19T01:39:11.548 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:39:11.540+0000 7f6e22ca5740 20 client.15708 trim_cache size 0 max 16384 2024-08-19T01:39:11.548 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:39:11.542+0000 7f6e1ec00640 1 -- 172.21.15.83:0/3402408160 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f6e000a2c00 msgr2=0x7f6e000a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:39:11.548 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:39:11.542+0000 7f6e1ec00640 1 --2- 172.21.15.83:0/3402408160 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f6e000a2c00 0x7f6e000a50c0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f6e08005ab0 tx=0x7f6e08005a20 comp rx=0 tx=0).stop 2024-08-19T01:39:11.548 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:39:11.542+0000 7f6e1ec00640 1 -- 172.21.15.83:0/3402408160 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6e1806e220 msgr2=0x7f6e18084b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:39:11.548 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:39:11.542+0000 7f6e1ec00640 1 --2- 172.21.15.83:0/3402408160 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6e1806e220 0x7f6e18084b90 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f6e1000eb90 tx=0x7f6e1000c6a0 comp rx=0 tx=0).stop 2024-08-19T01:39:11.548 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:39:11.543+0000 7f6e1ec00640 1 -- 172.21.15.83:0/3402408160 shutdown_connections 2024-08-19T01:39:11.548 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:39:11.543+0000 7f6e1ec00640 1 --2- 172.21.15.83:0/3402408160 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f6e000a2c00 0x7f6e000a50c0 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:39:11.548 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:39:11.543+0000 7f6e1ec00640 1 --2- 172.21.15.83:0/3402408160 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6e1810ddc0 0x7f6e1807f400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:39:11.549 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:39:11.543+0000 7f6e1ec00640 1 --2- 172.21.15.83:0/3402408160 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6e1806eba0 0x7f6e1807ecc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:39:11.549 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:39:11.543+0000 7f6e1ec00640 1 --2- 172.21.15.83:0/3402408160 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6e1806e220 0x7f6e18084b90 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:39:11.549 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:39:11.543+0000 7f6e1ec00640 1 -- 172.21.15.83:0/3402408160 >> 172.21.15.83:0/3402408160 conn(0x7f6e18072eb0 msgr2=0x7f6e1806c7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:39:11.549 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:39:11.543+0000 7f6e1ec00640 1 -- 172.21.15.83:0/3402408160 shutdown_connections 2024-08-19T01:39:11.549 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:39:11.544+0000 7f6e1ec00640 1 -- 172.21.15.83:0/3402408160 wait complete. 2024-08-19T01:39:11.655 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:39:11 smithi083 ceph-mon[17120]: pgmap v469: 97 pgs: 97 active+clean; 361 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 2.3 MiB/s wr, 1.09k op/s 2024-08-19T01:39:11.952 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:39:11 smithi117 ceph-mon[22016]: pgmap v469: 97 pgs: 97 active+clean; 361 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 2.3 MiB/s wr, 1.09k op/s 2024-08-19T01:39:11.965 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:39:11 smithi167 ceph-mon[21875]: pgmap v469: 97 pgs: 97 active+clean; 361 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 2.3 MiB/s wr, 1.09k op/s 2024-08-19T01:39:12.978 INFO:tasks.cephfs.filesystem:scrub status for tag:0e436353-7799-40e1-97b3-a89eaf3bf27d - {'path': '/', 'tag': '0e436353-7799-40e1-97b3-a89eaf3bf27d', 'options': 'recursive,force'} 2024-08-19T01:39:13.816 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:39:13 smithi083 ceph-mon[17120]: pgmap v470: 97 pgs: 97 active+clean; 353 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.8 MiB/s wr, 1.17k op/s 2024-08-19T01:39:13.952 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:39:13 smithi117 ceph-mon[22016]: pgmap v470: 97 pgs: 97 active+clean; 353 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.8 MiB/s wr, 1.17k op/s 2024-08-19T01:39:13.965 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:39:13 smithi167 ceph-mon[21875]: pgmap v470: 97 pgs: 97 active+clean; 353 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.8 MiB/s wr, 1.17k op/s 2024-08-19T01:39:14.584 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:39:14 smithi083 ceph-mon[17120]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-19T01:39:14.952 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:39:14 smithi117 ceph-mon[22016]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-19T01:39:14.965 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:39:14 smithi167 ceph-mon[21875]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-19T01:39:15.684 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:39:15 smithi083 ceph-mon[17120]: pgmap v471: 97 pgs: 97 active+clean; 369 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 3.3 MiB/s wr, 1.38k op/s 2024-08-19T01:39:15.952 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:39:15 smithi117 ceph-mon[22016]: pgmap v471: 97 pgs: 97 active+clean; 369 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 3.3 MiB/s wr, 1.38k op/s 2024-08-19T01:39:15.965 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:39:15 smithi167 ceph-mon[21875]: pgmap v471: 97 pgs: 97 active+clean; 369 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 3.3 MiB/s wr, 1.38k op/s 2024-08-19T01:39:17.755 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:39:17 smithi083 ceph-mon[17120]: pgmap v472: 97 pgs: 97 active+clean; 371 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 3.5 MiB/s wr, 1.35k op/s 2024-08-19T01:39:17.952 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:39:17 smithi117 ceph-mon[22016]: pgmap v472: 97 pgs: 97 active+clean; 371 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 3.5 MiB/s wr, 1.35k op/s 2024-08-19T01:39:17.965 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:39:17 smithi167 ceph-mon[21875]: pgmap v472: 97 pgs: 97 active+clean; 371 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 3.5 MiB/s wr, 1.35k op/s 2024-08-19T01:39:19.904 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:39:19 smithi083 ceph-mon[17120]: pgmap v473: 97 pgs: 97 active+clean; 406 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 6.1 MiB/s wr, 1.56k op/s 2024-08-19T01:39:19.953 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:39:19 smithi117 ceph-mon[22016]: pgmap v473: 97 pgs: 97 active+clean; 406 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 6.1 MiB/s wr, 1.56k op/s 2024-08-19T01:39:19.966 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:39:19 smithi167 ceph-mon[21875]: pgmap v473: 97 pgs: 97 active+clean; 406 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 6.1 MiB/s wr, 1.56k op/s 2024-08-19T01:39:20.621 INFO:tasks.workunit.client.0.smithi083.stdout: 136 541489 291 358019 382 319431 956 2024-08-19T01:39:21.817 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:39:21.814+0000 7f8b26000640 1 -- 172.21.15.83:0/3979525286 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8b180a5670 msgr2=0x7f8b180a5af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:39:21.817 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:39:21.814+0000 7f8b26000640 1 --2- 172.21.15.83:0/3979525286 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8b180a5670 0x7f8b180a5af0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f8b0c009950 tx=0x7f8b0c02a4f0 comp rx=0 tx=0).stop 2024-08-19T01:39:21.906 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:39:21 smithi083 ceph-mon[17120]: pgmap v474: 97 pgs: 97 active+clean; 428 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 7.5 MiB/s wr, 1.24k op/s 2024-08-19T01:39:21.912 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:39:21.903+0000 7f8b26000640 1 -- 172.21.15.83:0/3979525286 shutdown_connections 2024-08-19T01:39:21.913 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:39:21.904+0000 7f8b26000640 1 --2- 172.21.15.83:0/3979525286 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8b180a6030 0x7f8b180aa8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:39:21.913 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:39:21.904+0000 7f8b26000640 1 --2- 172.21.15.83:0/3979525286 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8b180a5670 0x7f8b180a5af0 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:39:21.913 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:39:21.904+0000 7f8b26000640 1 --2- 172.21.15.83:0/3979525286 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8b180a4540 0x7f8b180a4940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:39:21.913 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:39:21.904+0000 7f8b26000640 1 -- 172.21.15.83:0/3979525286 >> 172.21.15.83:0/3979525286 conn(0x7f8b1809fba0 msgr2=0x7f8b180a2000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:39:21.952 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:39:21 smithi117 ceph-mon[22016]: pgmap v474: 97 pgs: 97 active+clean; 428 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 7.5 MiB/s wr, 1.24k op/s 2024-08-19T01:39:21.966 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:39:21 smithi167 ceph-mon[21875]: pgmap v474: 97 pgs: 97 active+clean; 428 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 7.5 MiB/s wr, 1.24k op/s 2024-08-19T01:39:21.966 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:39:21.940+0000 7f8b26000640 1 -- 172.21.15.83:0/3979525286 shutdown_connections 2024-08-19T01:39:21.966 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:39:21.954+0000 7f8b26000640 1 -- 172.21.15.83:0/3979525286 wait complete. 2024-08-19T01:39:21.966 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:39:21.956+0000 7f8b26000640 1 Processor -- start 2024-08-19T01:39:21.982 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:39:21.971+0000 7f8b26000640 1 -- start start 2024-08-19T01:39:21.982 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:39:21.971+0000 7f8b26000640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8b180a4540 0x7f8b180b1110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:39:21.982 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:39:21.971+0000 7f8b26000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8b180a5670 0x7f8b180b1650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:39:21.982 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:39:21.971+0000 7f8b26000640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8b180a6030 0x7f8b18153060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:39:21.982 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:39:21.971+0000 7f8b26000640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f8b180b8970 con 0x7f8b180a5670 2024-08-19T01:39:21.982 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:39:21.971+0000 7f8b26000640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f8b180b87f0 con 0x7f8b180a4540 2024-08-19T01:39:21.982 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:39:21.971+0000 7f8b26000640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f8b180b8af0 con 0x7f8b180a6030 2024-08-19T01:39:21.983 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:39:21.975+0000 7f8b1fe00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8b180a5670 0x7f8b180b1650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:39:21.983 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:39:21.975+0000 7f8b1fe00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8b180a5670 0x7f8b180b1650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:47648/0 (socket says 172.21.15.83:47648) 2024-08-19T01:39:21.983 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:39:21.975+0000 7f8b1fe00640 1 -- 172.21.15.83:0/1239596116 learned_addr learned my addr 172.21.15.83:0/1239596116 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-08-19T01:39:21.983 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:39:21.975+0000 7f8b1fe00640 1 -- 172.21.15.83:0/1239596116 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8b180a6030 msgr2=0x7f8b18153060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:39:21.983 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:39:21.975+0000 7f8b1fe00640 1 --2- 172.21.15.83:0/1239596116 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8b180a6030 0x7f8b18153060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:39:21.983 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:39:21.975+0000 7f8b1fe00640 1 -- 172.21.15.83:0/1239596116 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8b180a4540 msgr2=0x7f8b180b1110 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-08-19T01:39:21.983 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:39:21.975+0000 7f8b1fe00640 1 --2- 172.21.15.83:0/1239596116 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8b180a4540 0x7f8b180b1110 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:39:21.983 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:39:21.975+0000 7f8b1fe00640 1 -- 172.21.15.83:0/1239596116 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8b181535a0 con 0x7f8b180a5670 2024-08-19T01:39:21.983 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:39:21.976+0000 7f8b1fe00640 1 --2- 172.21.15.83:0/1239596116 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8b180a5670 0x7f8b180b1650 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f8b0c0314e0 tx=0x7f8b0c004ef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:39:22.008 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:39:22.003+0000 7f8b1d600640 1 -- 172.21.15.83:0/1239596116 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b0c03f030 con 0x7f8b180a5670 2024-08-19T01:39:22.008 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:39:22.003+0000 7f8b1d600640 1 -- 172.21.15.83:0/1239596116 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8b0c0316b0 con 0x7f8b180a5670 2024-08-19T01:39:22.008 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:39:22.003+0000 7f8b1d600640 1 -- 172.21.15.83:0/1239596116 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b0c003850 con 0x7f8b180a5670 2024-08-19T01:39:22.039 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:39:22.038+0000 7f8b26000640 1 -- 172.21.15.83:0/1239596116 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8b18153770 con 0x7f8b180a5670 2024-08-19T01:39:22.039 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:39:22.038+0000 7f8b26000640 1 -- 172.21.15.83:0/1239596116 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8b18153b20 con 0x7f8b180a5670 2024-08-19T01:39:22.075 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:39:22.074+0000 7f8b2a63d740 1 Processor -- start 2024-08-19T01:39:22.090 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:39:22.085+0000 7f8b2a63d740 1 -- start start 2024-08-19T01:39:22.090 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:39:22.085+0000 7f8b2a63d740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5648a64bc660 0x5648a64bca60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:39:22.090 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:39:22.085+0000 7f8b2a63d740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5648a64bcfa0 0x5648a64c1a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:39:22.090 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:39:22.085+0000 7f8b2a63d740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5648a64c2030 0x5648a64c44f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:39:22.090 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:39:22.085+0000 7f8b2a63d740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x5648a6420890 con 0x5648a64bcfa0 2024-08-19T01:39:22.091 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:39:22.085+0000 7f8b2a63d740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x5648a63b8d90 con 0x5648a64c2030 2024-08-19T01:39:22.091 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:39:22.085+0000 7f8b2a63d740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x5648a641c850 con 0x5648a64bc660 2024-08-19T01:39:22.091 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:39:22.086+0000 7f8b24c00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5648a64bcfa0 0x5648a64c1a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:39:22.091 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:39:22.086+0000 7f8b24c00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5648a64bcfa0 0x5648a64c1a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:47664/0 (socket says 172.21.15.83:47664) 2024-08-19T01:39:22.091 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:39:22.086+0000 7f8b24c00640 1 -- 172.21.15.83:0/3207694821 learned_addr learned my addr 172.21.15.83:0/3207694821 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-08-19T01:39:22.091 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:39:22.087+0000 7f8b24c00640 1 -- 172.21.15.83:0/3207694821 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5648a64bc660 msgr2=0x5648a64bca60 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-08-19T01:39:22.091 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:39:22.087+0000 7f8b24c00640 1 --2- 172.21.15.83:0/3207694821 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5648a64bc660 0x5648a64bca60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:39:22.091 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:39:22.087+0000 7f8b24c00640 1 -- 172.21.15.83:0/3207694821 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5648a64c2030 msgr2=0x5648a64c44f0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-08-19T01:39:22.091 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:39:22.087+0000 7f8b24c00640 1 --2- 172.21.15.83:0/3207694821 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5648a64c2030 0x5648a64c44f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:39:22.091 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:39:22.087+0000 7f8b24c00640 1 -- 172.21.15.83:0/3207694821 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5648a6134210 con 0x5648a64bcfa0 2024-08-19T01:39:22.091 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:39:22.087+0000 7f8b24c00640 1 --2- 172.21.15.83:0/3207694821 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5648a64bcfa0 0x5648a64c1a60 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f8b14009c20 tx=0x7f8b1400e720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=48e7dcaeb8f46752 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:39:22.092 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:39:22.091+0000 7f8b11c00640 1 -- 172.21.15.83:0/3207694821 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b14005020 con 0x5648a64bcfa0 2024-08-19T01:39:22.092 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:39:22.091+0000 7f8b11c00640 1 -- 172.21.15.83:0/3207694821 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8b140047a0 con 0x5648a64bcfa0 2024-08-19T01:39:22.092 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:39:22.091+0000 7f8b11c00640 1 -- 172.21.15.83:0/3207694821 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b14010700 con 0x5648a64bcfa0 2024-08-19T01:39:22.093 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:39:22.092+0000 7f8b2a63d740 1 -- 172.21.15.83:0/3207694821 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5648a64bcfa0 msgr2=0x5648a64c1a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:39:22.093 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:39:22.092+0000 7f8b2a63d740 1 --2- 172.21.15.83:0/3207694821 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5648a64bcfa0 0x5648a64c1a60 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f8b14009c20 tx=0x7f8b1400e720 comp rx=0 tx=0).stop 2024-08-19T01:39:22.109 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:39:22.093+0000 7f8b1d600640 1 -- 172.21.15.83:0/1239596116 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7f8b0c02ad10 con 0x7f8b180a5670 2024-08-19T01:39:22.109 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:39:22.095+0000 7f8b1d600640 1 --2- 172.21.15.83:0/1239596116 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f8b080a2d90 0x7f8b080a5250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:39:22.109 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:39:22.103+0000 7f8b25600640 1 --2- 172.21.15.83:0/1239596116 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f8b080a2d90 0x7f8b080a5250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:39:22.109 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:39:22.103+0000 7f8b1d600640 1 -- 172.21.15.83:0/1239596116 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8269+0+0 (secure 0 0 0) 0x7f8b0c0ef8a0 con 0x7f8b180a5670 2024-08-19T01:39:22.109 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:39:22.104+0000 7f8b25600640 1 --2- 172.21.15.83:0/1239596116 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f8b080a2d90 0x7f8b080a5250 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f8b20067800 tx=0x7f8b20068650 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:39:22.109 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:39:22.104+0000 7f8b2a63d740 1 -- 172.21.15.83:0/3207694821 shutdown_connections 2024-08-19T01:39:22.110 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:39:22.104+0000 7f8b2a63d740 1 --2- 172.21.15.83:0/3207694821 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5648a64c2030 0x5648a64c44f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:39:22.110 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:39:22.104+0000 7f8b2a63d740 1 --2- 172.21.15.83:0/3207694821 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5648a64bcfa0 0x5648a64c1a60 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:39:22.110 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:39:22.104+0000 7f8b2a63d740 1 --2- 172.21.15.83:0/3207694821 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5648a64bc660 0x5648a64bca60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:39:22.110 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:39:22.104+0000 7f8b2a63d740 1 -- 172.21.15.83:0/3207694821 >> 172.21.15.83:0/3207694821 conn(0x5648a64b7cd0 msgr2=0x5648a64b80d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:39:22.110 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:39:22.107+0000 7f8b2a63d740 1 -- 172.21.15.83:0/3207694821 shutdown_connections 2024-08-19T01:39:22.110 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:39:22.107+0000 7f8b2a63d740 1 -- 172.21.15.83:0/3207694821 wait complete. 2024-08-19T01:39:22.110 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:39:22.107+0000 7f8b2a63d740 1 -- start start 2024-08-19T01:39:22.111 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:39:22.110+0000 7f8b2a63d740 1 Processor -- start 2024-08-19T01:39:22.121 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:39:22.111+0000 7f8b2a63d740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5648a64bc660 0x5648a64c6a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:39:22.121 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:39:22.111+0000 7f8b2a63d740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5648a64bcfa0 0x5648a64c6fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:39:22.121 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:39:22.111+0000 7f8b2a63d740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5648a64c2030 0x5648a64cdf80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:39:22.121 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:39:22.111+0000 7f8b2a63d740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x5648a64c50a0 con 0x5648a64c2030 2024-08-19T01:39:22.121 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:39:22.111+0000 7f8b2a63d740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x5648a63dbf70 con 0x5648a64bc660 2024-08-19T01:39:22.121 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:39:22.111+0000 7f8b2a63d740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x5648a64c5220 con 0x5648a64bcfa0 2024-08-19T01:39:22.126 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:39:22.111+0000 7f8b2a63d740 10 client.0 _ll_register_callbacks cb 0x5648a64188d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-19T01:39:22.126 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:39:22.112+0000 7f8b1fe00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5648a64bcfa0 0x5648a64c6fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:39:22.127 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:39:22.112+0000 7f8b1fe00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5648a64bcfa0 0x5648a64c6fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.83:52508/0 (socket says 172.21.15.83:52508) 2024-08-19T01:39:22.127 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:39:22.112+0000 7f8b1fe00640 1 -- 172.21.15.83:0/3511907324 learned_addr learned my addr 172.21.15.83:0/3511907324 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-08-19T01:39:22.127 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:39:22.112+0000 7f8b25600640 1 --2- 172.21.15.83:0/3511907324 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5648a64c2030 0x5648a64cdf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:39:22.127 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:39:22.113+0000 7f8b1fe00640 1 -- 172.21.15.83:0/3511907324 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5648a64bc660 msgr2=0x5648a64c6a70 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-08-19T01:39:22.127 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:39:22.113+0000 7f8b1fe00640 1 --2- 172.21.15.83:0/3511907324 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5648a64bc660 0x5648a64c6a70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:39:22.127 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:39:22.113+0000 7f8b1fe00640 1 -- 172.21.15.83:0/3511907324 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5648a64c2030 msgr2=0x5648a64cdf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:39:22.127 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:39:22.113+0000 7f8b1fe00640 1 --2- 172.21.15.83:0/3511907324 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5648a64c2030 0x5648a64cdf80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:39:22.127 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:39:22.113+0000 7f8b1fe00640 1 -- 172.21.15.83:0/3511907324 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5648a6200620 con 0x5648a64bcfa0 2024-08-19T01:39:22.127 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:39:22.113+0000 7f8b25600640 1 --2- 172.21.15.83:0/3511907324 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5648a64c2030 0x5648a64cdf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-19T01:39:22.127 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:39:22.114+0000 7f8b1fe00640 1 --2- 172.21.15.83:0/3511907324 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5648a64bcfa0 0x5648a64c6fb0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f8b0c0bc010 tx=0x7f8b0c0bc840 comp rx=0 tx=0).ready entity=mon.1 client_cookie=945fc85e673aaa8e server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:39:22.128 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:39:22.125+0000 7f8b06000640 10 client.0 ms_handle_connect on v2:172.21.15.167:3300/0 2024-08-19T01:39:22.128 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:39:22.125+0000 7f8b06000640 1 -- 172.21.15.83:0/3511907324 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b0c0c4cf0 con 0x5648a64bcfa0 2024-08-19T01:39:22.128 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:39:22.125+0000 7f8b06000640 1 -- 172.21.15.83:0/3511907324 <== mon.1 v2:172.21.15.167:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8269+0+0 (secure 0 0 0) 0x7f8b0c0dc070 con 0x5648a64bcfa0 2024-08-19T01:39:22.128 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:39:22.126+0000 7f8b2a63d740 1 -- 172.21.15.83:0/3511907324 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8b140047a0 con 0x5648a64bcfa0 2024-08-19T01:39:22.165 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:39:22.163+0000 7f8b06000640 1 -- 172.21.15.83:0/3511907324 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f8b0c0c00d0 con 0x5648a64bcfa0 2024-08-19T01:39:22.176 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:39:22.174+0000 7f8b2a63d740 10 client.25696 fetch_fsmap learned FSMap version 27 2024-08-19T01:39:22.176 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:39:22.174+0000 7f8b2a63d740 1 -- 172.21.15.83:0/3511907324 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5648a64c5110 con 0x5648a64bcfa0 2024-08-19T01:39:22.178 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:39:22.177+0000 7f8b06000640 1 -- 172.21.15.83:0/3511907324 <== mon.1 v2:172.21.15.167:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8b140047a0 con 0x5648a64bcfa0 2024-08-19T01:39:22.178 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:39:22.177+0000 7f8b06000640 1 -- 172.21.15.83:0/3511907324 <== mon.1 v2:172.21.15.167:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f8b0c031bd0 con 0x5648a64bcfa0 2024-08-19T01:39:22.184 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:39:22.177+0000 7f8b2a63d740 10 client.25696 fetch_fsmap finished waiting for FSMap version 27 2024-08-19T01:39:22.184 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:39:22.177+0000 7f8b2a63d740 10 client.25696 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-08-19T01:39:22.184 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:39:22.177+0000 7f8b2a63d740 20 client.25696 populate_metadata read hostname 'smithi083' 2024-08-19T01:39:22.184 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:39:22.177+0000 7f8b2a63d740 1 --2- 172.21.15.83:0/3511907324 >> [v2:172.21.15.117:6838/1967595704,v1:172.21.15.117:6839/1967595704] conn(0x5648a64d2870 0x5648a64d4cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:39:22.184 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:39:22.177+0000 7f8b2a63d740 4 client.25696 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-19T01:39:22.184 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:39:22.177+0000 7f8b2a63d740 1 -- 172.21.15.83:0/3511907324 --> [v2:172.21.15.117:6838/1967595704,v1:172.21.15.117:6839/1967595704] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5648a642dfc0 con 0x5648a64d2870 2024-08-19T01:39:22.184 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:39:22.181+0000 7f8b24c00640 1 --2- 172.21.15.83:0/3511907324 >> [v2:172.21.15.117:6838/1967595704,v1:172.21.15.117:6839/1967595704] conn(0x5648a64d2870 0x5648a64d4cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:39:22.185 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:39:22.183+0000 7f8b24c00640 1 --2- 172.21.15.83:0/3511907324 >> [v2:172.21.15.117:6838/1967595704,v1:172.21.15.117:6839/1967595704] conn(0x5648a64d2870 0x5648a64d4cf0 crc :-1 s=READY pgs=95 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d5eca4a86ead3e21 server_cookie=21db618c0c557c7f in_seq=0 out_seq=0 2024-08-19T01:39:22.185 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:39:22.183+0000 7f8b06000640 10 client.25696 ms_handle_connect on v2:172.21.15.117:6838/1967595704 2024-08-19T01:39:22.189 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:39:22.188+0000 7f8b06000640 1 -- 172.21.15.83:0/3511907324 <== mds.0 v2:172.21.15.117:6838/1967595704 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5648a642dfc0 con 0x5648a64d2870 2024-08-19T01:39:22.189 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:39:22.188+0000 7f8b06000640 10 client.25696 handle_command_reply: tid=0 2024-08-19T01:39:22.195 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:39:22.194+0000 7f8b2a63d740 1 client.25696 shutdown 2024-08-19T01:39:22.203 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:39:22.202+0000 7f8b2a63d740 1 -- 172.21.15.83:0/3511907324 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5648a64bcfa0 msgr2=0x5648a64c6fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:39:22.203 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:39:22.202+0000 7f8b2a63d740 1 --2- 172.21.15.83:0/3511907324 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5648a64bcfa0 0x5648a64c6fb0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f8b0c0bc010 tx=0x7f8b0c0bc840 comp rx=0 tx=0).stop 2024-08-19T01:39:22.204 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:39:22.202+0000 7f8b2a63d740 1 -- 172.21.15.83:0/3511907324 shutdown_connections 2024-08-19T01:39:22.204 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:39:22.202+0000 7f8b2a63d740 1 --2- 172.21.15.83:0/3511907324 >> [v2:172.21.15.117:6838/1967595704,v1:172.21.15.117:6839/1967595704] conn(0x5648a64d2870 0x5648a64d4cf0 crc :-1 s=READY pgs=95 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:39:22.204 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:39:22.202+0000 7f8b2a63d740 1 --2- 172.21.15.83:0/3511907324 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5648a64c2030 0x5648a64cdf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:39:22.204 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:39:22.202+0000 7f8b2a63d740 1 --2- 172.21.15.83:0/3511907324 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5648a64bcfa0 0x5648a64c6fb0 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:39:22.204 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:39:22.202+0000 7f8b2a63d740 1 --2- 172.21.15.83:0/3511907324 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5648a64bc660 0x5648a64c6a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:39:22.204 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:39:22.202+0000 7f8b2a63d740 1 -- 172.21.15.83:0/3511907324 >> 172.21.15.83:0/3511907324 conn(0x5648a64b7cd0 msgr2=0x5648a64b94e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:39:22.204 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:39:22.202+0000 7f8b06000640 0 client.25696 ms_handle_reset on v2:172.21.15.117:6838/1967595704 2024-08-19T01:39:22.218 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:39:22.216+0000 7f8b2a63d740 1 -- 172.21.15.83:0/3511907324 shutdown_connections 2024-08-19T01:39:22.218 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:39:22.216+0000 7f8b2a63d740 1 -- 172.21.15.83:0/3511907324 wait complete. 2024-08-19T01:39:22.220 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:39:22.219+0000 7f8b2a63d740 20 client.25696 trim_cache size 0 max 16384 2024-08-19T01:39:22.378 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:39:22.368+0000 7f8b2a63d740 1 Processor -- start 2024-08-19T01:39:22.378 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:39:22.368+0000 7f8b2a63d740 1 -- start start 2024-08-19T01:39:22.378 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:39:22.374+0000 7f8b2a63d740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5648a64bc660 0x5648a64c1780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:39:22.378 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:39:22.375+0000 7f8b2a63d740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5648a64bcfa0 0x5648a64cdf40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:39:22.378 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:39:22.375+0000 7f8b2a63d740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5648a64c2030 0x5648a64b8bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:39:22.379 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:39:22.375+0000 7f8b1fe00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5648a64bcfa0 0x5648a64cdf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:39:22.379 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:39:22.375+0000 7f8b1fe00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5648a64bcfa0 0x5648a64cdf40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.83:59122/0 (socket says 172.21.15.83:59122) 2024-08-19T01:39:22.379 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:39:22.375+0000 7f8b1fe00640 1 -- 172.21.15.83:0/1055794095 learned_addr learned my addr 172.21.15.83:0/1055794095 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-08-19T01:39:22.379 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:39:22.375+0000 7f8b2a63d740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f8b1815f370 con 0x5648a64c2030 2024-08-19T01:39:22.379 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:39:22.375+0000 7f8b2a63d740 1 -- 172.21.15.83:0/1055794095 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f8b1815cc40 con 0x5648a64bcfa0 2024-08-19T01:39:22.379 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:39:22.375+0000 7f8b2a63d740 1 -- 172.21.15.83:0/1055794095 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f8b1815aa90 con 0x5648a64bc660 2024-08-19T01:39:22.379 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:39:22.376+0000 7f8b1fe00640 1 -- 172.21.15.83:0/1055794095 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5648a64bc660 msgr2=0x5648a64c1780 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-08-19T01:39:22.379 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:39:22.376+0000 7f8b1fe00640 1 --2- 172.21.15.83:0/1055794095 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5648a64bc660 0x5648a64c1780 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:39:22.379 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:39:22.376+0000 7f8b1fe00640 1 -- 172.21.15.83:0/1055794095 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5648a64c2030 msgr2=0x5648a64b8bd0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-08-19T01:39:22.379 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:39:22.376+0000 7f8b25600640 1 --2- 172.21.15.83:0/1055794095 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5648a64c2030 0x5648a64b8bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:39:22.379 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:39:22.376+0000 7f8b1fe00640 1 --2- 172.21.15.83:0/1055794095 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5648a64c2030 0x5648a64b8bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:39:22.379 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:39:22.376+0000 7f8b1fe00640 1 -- 172.21.15.83:0/1055794095 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5648a64bd7f0 con 0x5648a64bcfa0 2024-08-19T01:39:22.380 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:39:22.377+0000 7f8b1fe00640 1 --2- 172.21.15.83:0/1055794095 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5648a64bcfa0 0x5648a64cdf40 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f8b0c005c40 tx=0x7f8b0c0c05b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=640e67d84b520d8 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:39:22.389 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:39:22.380+0000 7f8b11200640 1 -- 172.21.15.83:0/1055794095 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b0c0b8320 con 0x5648a64bcfa0 2024-08-19T01:39:22.389 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:39:22.380+0000 7f8b11200640 1 -- 172.21.15.83:0/1055794095 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8b0c0c4cf0 con 0x5648a64bcfa0 2024-08-19T01:39:22.389 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:39:22.380+0000 7f8b11200640 1 -- 172.21.15.83:0/1055794095 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b0c031bd0 con 0x5648a64bcfa0 2024-08-19T01:39:22.390 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:39:22.389+0000 7f8b2a63d740 1 -- 172.21.15.83:0/1055794095 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5648a64bcfa0 msgr2=0x5648a64cdf40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:39:22.390 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:39:22.389+0000 7f8b2a63d740 1 --2- 172.21.15.83:0/1055794095 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5648a64bcfa0 0x5648a64cdf40 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f8b0c005c40 tx=0x7f8b0c0c05b0 comp rx=0 tx=0).stop 2024-08-19T01:39:22.410 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:39:22.409+0000 7f8b2a63d740 1 -- 172.21.15.83:0/1055794095 shutdown_connections 2024-08-19T01:39:22.410 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:39:22.409+0000 7f8b2a63d740 1 --2- 172.21.15.83:0/1055794095 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5648a64c2030 0x5648a64b8bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:39:22.411 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:39:22.409+0000 7f8b2a63d740 1 --2- 172.21.15.83:0/1055794095 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5648a64bcfa0 0x5648a64cdf40 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:39:22.411 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:39:22.409+0000 7f8b2a63d740 1 --2- 172.21.15.83:0/1055794095 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5648a64bc660 0x5648a64c1780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:39:22.411 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:39:22.409+0000 7f8b2a63d740 1 -- 172.21.15.83:0/1055794095 >> 172.21.15.83:0/1055794095 conn(0x5648a64b7cd0 msgr2=0x5648a64be220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:39:22.411 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:39:22.409+0000 7f8b2a63d740 1 -- 172.21.15.83:0/1055794095 shutdown_connections 2024-08-19T01:39:22.412 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:39:22.410+0000 7f8b2a63d740 1 -- 172.21.15.83:0/1055794095 wait complete. 2024-08-19T01:39:22.412 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:39:22.410+0000 7f8b2a63d740 1 -- start start 2024-08-19T01:39:22.412 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:39:22.410+0000 7f8b2a63d740 1 Processor -- start 2024-08-19T01:39:22.420 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:39:22.415+0000 7f8b2a63d740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5648a64bc660 0x5648a64cb110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:39:22.420 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:39:22.419+0000 7f8b2a63d740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5648a64bcfa0 0x5648a64cb650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:39:22.421 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:39:22.420+0000 7f8b1fe00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5648a64bcfa0 0x5648a64cb650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:39:22.421 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:39:22.420+0000 7f8b1fe00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5648a64bcfa0 0x5648a64cb650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:47690/0 (socket says 172.21.15.83:47690) 2024-08-19T01:39:22.421 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:39:22.420+0000 7f8b1fe00640 1 -- 172.21.15.83:0/2270475521 learned_addr learned my addr 172.21.15.83:0/2270475521 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-08-19T01:39:22.425 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:39:22.420+0000 7f8b2a63d740 1 --2- 172.21.15.83:0/2270475521 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5648a64c2030 0x5648a64cc550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:39:22.425 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:39:22.424+0000 7f8b2a63d740 1 -- 172.21.15.83:0/2270475521 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f8b1815b100 con 0x5648a64bcfa0 2024-08-19T01:39:22.425 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:39:22.424+0000 7f8b2a63d740 1 -- 172.21.15.83:0/2270475521 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f8b1815c570 con 0x5648a64c2030 2024-08-19T01:39:22.425 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:39:22.424+0000 7f8b2a63d740 1 -- 172.21.15.83:0/2270475521 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f8b1815b7d0 con 0x5648a64bc660 2024-08-19T01:39:22.426 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:39:22.424+0000 7f8b2a63d740 10 client.0 _ll_register_callbacks cb 0x7f8b0c005b70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-19T01:39:22.436 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:39:22.426+0000 7f8b24c00640 1 --2- 172.21.15.83:0/2270475521 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5648a64bc660 0x5648a64cb110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:39:22.436 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:39:22.434+0000 7f8b25600640 1 --2- 172.21.15.83:0/2270475521 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5648a64c2030 0x5648a64cc550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:39:22.436 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:39:22.434+0000 7f8b24c00640 1 -- 172.21.15.83:0/2270475521 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5648a64c2030 msgr2=0x5648a64cc550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:39:22.437 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:39:22.434+0000 7f8b24c00640 1 --2- 172.21.15.83:0/2270475521 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5648a64c2030 0x5648a64cc550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:39:22.437 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:39:22.434+0000 7f8b24c00640 1 -- 172.21.15.83:0/2270475521 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5648a64bcfa0 msgr2=0x5648a64cb650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:39:22.437 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:39:22.434+0000 7f8b24c00640 1 --2- 172.21.15.83:0/2270475521 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5648a64bcfa0 0x5648a64cb650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:39:22.437 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:39:22.434+0000 7f8b24c00640 1 -- 172.21.15.83:0/2270475521 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5648a64ccb60 con 0x5648a64bc660 2024-08-19T01:39:22.437 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:39:22.435+0000 7f8b24c00640 1 --2- 172.21.15.83:0/2270475521 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5648a64bc660 0x5648a64cb110 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x5648a64c5fa0 tx=0x7f8b14009fd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6a049f4c02dba4f6 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:39:22.437 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:39:22.435+0000 7f8b06000640 10 client.0 ms_handle_connect on v2:172.21.15.167:3300/0 2024-08-19T01:39:22.440 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:39:22.439+0000 7f8b06000640 1 -- 172.21.15.83:0/2270475521 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b1401d040 con 0x5648a64bc660 2024-08-19T01:39:22.440 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:39:22.439+0000 7f8b06000640 1 -- 172.21.15.83:0/2270475521 <== mon.1 v2:172.21.15.167:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8269+0+0 (secure 0 0 0) 0x7f8b14005040 con 0x5648a64bc660 2024-08-19T01:39:22.451 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:39:22.450+0000 7f8b2a63d740 1 -- 172.21.15.83:0/2270475521 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5648a6428310 con 0x5648a64bc660 2024-08-19T01:39:22.462 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:39:22.457+0000 7f8b06000640 1 -- 172.21.15.83:0/2270475521 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f8b14005260 con 0x5648a64bc660 2024-08-19T01:39:22.465 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:39:22.464+0000 7f8b2a63d740 10 client.25699 fetch_fsmap learned FSMap version 27 2024-08-19T01:39:22.465 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:39:22.464+0000 7f8b2a63d740 1 -- 172.21.15.83:0/2270475521 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5648a64cdf40 con 0x5648a64bc660 2024-08-19T01:39:22.466 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:39:22.465+0000 7f8b06000640 1 -- 172.21.15.83:0/2270475521 <== mon.1 v2:172.21.15.167:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x5648a6428310 con 0x5648a64bc660 2024-08-19T01:39:22.466 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:39:22.465+0000 7f8b06000640 1 -- 172.21.15.83:0/2270475521 <== mon.1 v2:172.21.15.167:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f8b1402b020 con 0x5648a64bc660 2024-08-19T01:39:22.482 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:39:22.475+0000 7f8b2a63d740 10 client.25699 fetch_fsmap finished waiting for FSMap version 27 2024-08-19T01:39:22.482 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:39:22.475+0000 7f8b2a63d740 10 client.25699 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-08-19T01:39:22.482 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:39:22.475+0000 7f8b2a63d740 20 client.25699 populate_metadata read hostname 'smithi083' 2024-08-19T01:39:22.482 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:39:22.475+0000 7f8b2a63d740 1 --2- 172.21.15.83:0/2270475521 >> [v2:172.21.15.117:6838/1967595704,v1:172.21.15.117:6839/1967595704] conn(0x5648a653d710 0x5648a653fb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:39:22.483 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:39:22.475+0000 7f8b2a63d740 4 client.25699 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-08-19T01:39:22.483 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:39:22.475+0000 7f8b2a63d740 1 -- 172.21.15.83:0/2270475521 --> [v2:172.21.15.117:6838/1967595704,v1:172.21.15.117:6839/1967595704] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5648a63a57c0 con 0x5648a653d710 2024-08-19T01:39:22.483 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:39:22.476+0000 7f8b1fe00640 1 --2- 172.21.15.83:0/2270475521 >> [v2:172.21.15.117:6838/1967595704,v1:172.21.15.117:6839/1967595704] conn(0x5648a653d710 0x5648a653fb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:39:22.483 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:39:22.482+0000 7f8b1fe00640 1 --2- 172.21.15.83:0/2270475521 >> [v2:172.21.15.117:6838/1967595704,v1:172.21.15.117:6839/1967595704] conn(0x5648a653d710 0x5648a653fb00 crc :-1 s=READY pgs=96 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d74de6901983c2f6 server_cookie=762a06eea9b40f8a in_seq=0 out_seq=0 2024-08-19T01:39:22.487 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:39:22.486+0000 7f8b06000640 10 client.25699 ms_handle_connect on v2:172.21.15.117:6838/1967595704 2024-08-19T01:39:23.607 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:39:23 smithi083 ceph-mon[17120]: pgmap v475: 97 pgs: 97 active+clean; 434 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 7.8 MiB/s wr, 1.05k op/s 2024-08-19T01:39:23.952 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:39:23 smithi117 ceph-mon[22016]: pgmap v475: 97 pgs: 97 active+clean; 434 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 7.8 MiB/s wr, 1.05k op/s 2024-08-19T01:39:23.965 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:39:23 smithi167 ceph-mon[21875]: pgmap v475: 97 pgs: 97 active+clean; 434 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 7.8 MiB/s wr, 1.05k op/s 2024-08-19T01:39:25.905 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:39:25 smithi083 ceph-mon[17120]: pgmap v476: 97 pgs: 97 active+clean; 433 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 7.7 MiB/s wr, 1.24k op/s 2024-08-19T01:39:25.953 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:39:25 smithi117 ceph-mon[22016]: pgmap v476: 97 pgs: 97 active+clean; 433 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 7.7 MiB/s wr, 1.24k op/s 2024-08-19T01:39:25.965 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:39:25 smithi167 ceph-mon[21875]: pgmap v476: 97 pgs: 97 active+clean; 433 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 7.7 MiB/s wr, 1.24k op/s 2024-08-19T01:39:27.904 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:39:27 smithi083 ceph-mon[17120]: pgmap v477: 97 pgs: 97 active+clean; 433 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 6.2 MiB/s wr, 1.05k op/s 2024-08-19T01:39:27.952 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:39:27 smithi117 ceph-mon[22016]: pgmap v477: 97 pgs: 97 active+clean; 433 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 6.2 MiB/s wr, 1.05k op/s 2024-08-19T01:39:27.965 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:39:27 smithi167 ceph-mon[21875]: pgmap v477: 97 pgs: 97 active+clean; 433 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 6.2 MiB/s wr, 1.05k op/s 2024-08-19T01:39:29.350 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:39:29.349+0000 7f8b06000640 1 -- 172.21.15.83:0/2270475521 <== mds.0 v2:172.21.15.117:6838/1967595704 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x7f8b0c0c8050 con 0x5648a653d710 2024-08-19T01:39:29.350 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:39:29.349+0000 7f8b06000640 10 client.25699 handle_command_reply: tid=0 2024-08-19T01:39:29.350 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:39:29.349+0000 7f8b2a63d740 1 client.25699 shutdown 2024-08-19T01:39:29.354 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:39:29.353+0000 7f8b2a63d740 1 -- 172.21.15.83:0/2270475521 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5648a64bc660 msgr2=0x5648a64cb110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:39:29.355 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:39:29.353+0000 7f8b2a63d740 1 --2- 172.21.15.83:0/2270475521 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5648a64bc660 0x5648a64cb110 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x5648a64c5fa0 tx=0x7f8b14009fd0 comp rx=0 tx=0).stop 2024-08-19T01:39:29.356 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:39:29.355+0000 7f8b2a63d740 1 -- 172.21.15.83:0/2270475521 shutdown_connections 2024-08-19T01:39:29.356 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:39:29.355+0000 7f8b2a63d740 1 --2- 172.21.15.83:0/2270475521 >> [v2:172.21.15.117:6838/1967595704,v1:172.21.15.117:6839/1967595704] conn(0x5648a653d710 0x5648a653fb00 crc :-1 s=READY pgs=96 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:39:29.356 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:39:29.355+0000 7f8b2a63d740 1 --2- 172.21.15.83:0/2270475521 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5648a64c2030 0x5648a64cc550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:39:29.356 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:39:29.355+0000 7f8b2a63d740 1 --2- 172.21.15.83:0/2270475521 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5648a64bcfa0 0x5648a64cb650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:39:29.356 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:39:29.355+0000 7f8b2a63d740 1 --2- 172.21.15.83:0/2270475521 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5648a64bc660 0x5648a64cb110 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:39:29.356 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:39:29.355+0000 7f8b2a63d740 1 -- 172.21.15.83:0/2270475521 >> 172.21.15.83:0/2270475521 conn(0x5648a64b7cd0 msgr2=0x5648a64b6720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:39:29.356 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:39:29.355+0000 7f8b06000640 0 client.25699 ms_handle_reset on v2:172.21.15.117:6838/1967595704 2024-08-19T01:39:29.361 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:39:29.358+0000 7f8b2a63d740 1 -- 172.21.15.83:0/2270475521 shutdown_connections 2024-08-19T01:39:29.361 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:39:29.360+0000 7f8b2a63d740 1 -- 172.21.15.83:0/2270475521 wait complete. 2024-08-19T01:39:29.364 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:39:29.363+0000 7f8b2a63d740 20 client.25699 trim_cache size 0 max 16384 2024-08-19T01:39:29.373 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stdout:{ 2024-08-19T01:39:29.373 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stdout: "message": "", 2024-08-19T01:39:29.373 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stdout: "return_code": 0 2024-08-19T01:39:29.373 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stdout:} 2024-08-19T01:39:29.392 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:39:29.390+0000 7f8b26000640 1 -- 172.21.15.83:0/1239596116 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f8b080a2d90 msgr2=0x7f8b080a5250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:39:29.392 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:39:29.390+0000 7f8b26000640 1 --2- 172.21.15.83:0/1239596116 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f8b080a2d90 0x7f8b080a5250 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f8b20067800 tx=0x7f8b20068650 comp rx=0 tx=0).stop 2024-08-19T01:39:29.392 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:39:29.390+0000 7f8b26000640 1 -- 172.21.15.83:0/1239596116 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8b180a5670 msgr2=0x7f8b180b1650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:39:29.392 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:39:29.390+0000 7f8b26000640 1 --2- 172.21.15.83:0/1239596116 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8b180a5670 0x7f8b180b1650 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f8b0c0314e0 tx=0x7f8b0c004ef0 comp rx=0 tx=0).stop 2024-08-19T01:39:29.393 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:39:29.391+0000 7f8b26000640 1 -- 172.21.15.83:0/1239596116 shutdown_connections 2024-08-19T01:39:29.393 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:39:29.391+0000 7f8b26000640 1 --2- 172.21.15.83:0/1239596116 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f8b080a2d90 0x7f8b080a5250 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:39:29.393 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:39:29.391+0000 7f8b26000640 1 --2- 172.21.15.83:0/1239596116 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8b180a6030 0x7f8b18153060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:39:29.393 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:39:29.391+0000 7f8b26000640 1 --2- 172.21.15.83:0/1239596116 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8b180a5670 0x7f8b180b1650 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:39:29.393 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:39:29.391+0000 7f8b26000640 1 --2- 172.21.15.83:0/1239596116 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8b180a4540 0x7f8b180b1110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:39:29.393 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:39:29.391+0000 7f8b26000640 1 -- 172.21.15.83:0/1239596116 >> 172.21.15.83:0/1239596116 conn(0x7f8b1809fba0 msgr2=0x7f8b180a68b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:39:29.395 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:39:29.394+0000 7f8b26000640 1 -- 172.21.15.83:0/1239596116 shutdown_connections 2024-08-19T01:39:29.405 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:39:29.403+0000 7f8b26000640 1 -- 172.21.15.83:0/1239596116 wait complete. 2024-08-19T01:39:29.908 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:39:29 smithi083 ceph-mon[17120]: pgmap v478: 97 pgs: 97 active+clean; 437 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 6.7 MiB/s wr, 1.66k op/s 2024-08-19T01:39:29.952 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:39:29 smithi117 ceph-mon[22016]: pgmap v478: 97 pgs: 97 active+clean; 437 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 6.7 MiB/s wr, 1.66k op/s 2024-08-19T01:39:29.965 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:39:29 smithi167 ceph-mon[21875]: pgmap v478: 97 pgs: 97 active+clean; 437 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 6.7 MiB/s wr, 1.66k op/s 2024-08-19T01:39:30.622 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:39:30 smithi083 ceph-mon[17120]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-19T01:39:30.622 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:39:30 smithi083 ceph-mon[17120]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-19T01:39:30.622 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:39:30 smithi083 ceph-mon[17120]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' 2024-08-19T01:39:30.622 INFO:tasks.workunit.client.0.smithi083.stdout: 146 610640 659 401984 409 359819 1646 2024-08-19T01:39:30.952 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:39:30 smithi117 ceph-mon[22016]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-19T01:39:30.952 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:39:30 smithi117 ceph-mon[22016]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-19T01:39:30.952 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:39:30 smithi117 ceph-mon[22016]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' 2024-08-19T01:39:30.965 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:39:30 smithi167 ceph-mon[21875]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-19T01:39:30.965 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:39:30 smithi167 ceph-mon[21875]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-19T01:39:30.965 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:39:30 smithi167 ceph-mon[21875]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' 2024-08-19T01:39:31.912 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:39:31 smithi083 ceph-mon[17120]: pgmap v479: 97 pgs: 97 active+clean; 432 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 4.0 MiB/s wr, 1.66k op/s 2024-08-19T01:39:31.952 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:39:31 smithi117 ceph-mon[22016]: pgmap v479: 97 pgs: 97 active+clean; 432 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 4.0 MiB/s wr, 1.66k op/s 2024-08-19T01:39:31.965 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:39:31 smithi167 ceph-mon[21875]: pgmap v479: 97 pgs: 97 active+clean; 432 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 4.0 MiB/s wr, 1.66k op/s 2024-08-19T01:39:33.904 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:39:33 smithi083 ceph-mon[17120]: pgmap v480: 97 pgs: 97 active+clean; 421 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.1 MiB/s wr, 1.49k op/s 2024-08-19T01:39:33.952 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:39:33 smithi117 ceph-mon[22016]: pgmap v480: 97 pgs: 97 active+clean; 421 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.1 MiB/s wr, 1.49k op/s 2024-08-19T01:39:33.965 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:39:33 smithi167 ceph-mon[21875]: pgmap v480: 97 pgs: 97 active+clean; 421 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.1 MiB/s wr, 1.49k op/s 2024-08-19T01:39:35.908 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:39:35 smithi083 ceph-mon[17120]: pgmap v481: 97 pgs: 97 active+clean; 418 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.1 MiB/s wr, 1.44k op/s 2024-08-19T01:39:35.952 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:39:35 smithi117 ceph-mon[22016]: pgmap v481: 97 pgs: 97 active+clean; 418 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.1 MiB/s wr, 1.44k op/s 2024-08-19T01:39:35.965 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:39:35 smithi167 ceph-mon[21875]: pgmap v481: 97 pgs: 97 active+clean; 418 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.1 MiB/s wr, 1.44k op/s 2024-08-19T01:39:37.906 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:39:37 smithi083 ceph-mon[17120]: pgmap v482: 97 pgs: 97 active+clean; 417 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.9 MiB/s wr, 1.15k op/s 2024-08-19T01:39:37.952 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:39:37 smithi117 ceph-mon[22016]: pgmap v482: 97 pgs: 97 active+clean; 417 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.9 MiB/s wr, 1.15k op/s 2024-08-19T01:39:37.965 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:39:37 smithi167 ceph-mon[21875]: pgmap v482: 97 pgs: 97 active+clean; 417 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.9 MiB/s wr, 1.15k op/s 2024-08-19T01:39:39.904 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:39:39 smithi083 ceph-mon[17120]: pgmap v483: 97 pgs: 97 active+clean; 423 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 2.6 MiB/s wr, 1.24k op/s 2024-08-19T01:39:39.952 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:39:39 smithi117 ceph-mon[22016]: pgmap v483: 97 pgs: 97 active+clean; 423 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 2.6 MiB/s wr, 1.24k op/s 2024-08-19T01:39:39.965 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:39:39 smithi167 ceph-mon[21875]: pgmap v483: 97 pgs: 97 active+clean; 423 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 2.6 MiB/s wr, 1.24k op/s 2024-08-19T01:39:40.631 INFO:tasks.workunit.client.0.smithi083.stdout: 153 674206 646 438983 530 385986 1579 2024-08-19T01:39:40.903 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:39:40 smithi083 ceph-mon[17120]: pgmap v484: 97 pgs: 97 active+clean; 422 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.9 MiB/s wr, 639 op/s 2024-08-19T01:39:41.202 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:39:40 smithi117 ceph-mon[22016]: pgmap v484: 97 pgs: 97 active+clean; 422 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.9 MiB/s wr, 639 op/s 2024-08-19T01:39:41.215 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:39:40 smithi167 ceph-mon[21875]: pgmap v484: 97 pgs: 97 active+clean; 422 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.9 MiB/s wr, 639 op/s 2024-08-19T01:39:42.979 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833 shell --fsid bad8780e-5dc9-11ef-bccf-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-08-19T01:39:43.904 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:39:43 smithi083 ceph-mon[17120]: pgmap v485: 97 pgs: 97 active+clean; 429 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 2.1 MiB/s wr, 419 op/s 2024-08-19T01:39:43.952 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:39:43 smithi117 ceph-mon[22016]: pgmap v485: 97 pgs: 97 active+clean; 429 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 2.1 MiB/s wr, 419 op/s 2024-08-19T01:39:43.965 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:39:43 smithi167 ceph-mon[21875]: pgmap v485: 97 pgs: 97 active+clean; 429 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 2.1 MiB/s wr, 419 op/s 2024-08-19T01:39:45.359 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:39:45.354+0000 7f7873200640 1 -- 172.21.15.83:0/884670577 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f78740733b0 msgr2=0x7f787410d220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:39:45.359 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:39:45.354+0000 7f7873200640 1 --2- 172.21.15.83:0/884670577 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f78740733b0 0x7f787410d220 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f7860009900 tx=0x7f786002a440 comp rx=0 tx=0).stop 2024-08-19T01:39:45.403 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:39:45.402+0000 7f7873200640 1 -- 172.21.15.83:0/884670577 shutdown_connections 2024-08-19T01:39:45.403 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:39:45.402+0000 7f7873200640 1 --2- 172.21.15.83:0/884670577 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f787410d8e0 0x7f787410fda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:39:45.403 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:39:45.402+0000 7f7873200640 1 --2- 172.21.15.83:0/884670577 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f78740733b0 0x7f787410d220 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:39:45.403 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:39:45.402+0000 7f7873200640 1 --2- 172.21.15.83:0/884670577 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f78740729e0 0x7f7874072de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:39:45.403 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:39:45.402+0000 7f7873200640 1 -- 172.21.15.83:0/884670577 >> 172.21.15.83:0/884670577 conn(0x7f787406e880 msgr2=0x7f7874070cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:39:45.410 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:39:45.409+0000 7f7873200640 1 -- 172.21.15.83:0/884670577 shutdown_connections 2024-08-19T01:39:45.420 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:39:45.417+0000 7f7873200640 1 -- 172.21.15.83:0/884670577 wait complete. 2024-08-19T01:39:45.420 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:39:45.418+0000 7f7873200640 1 Processor -- start 2024-08-19T01:39:45.456 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:39:45.446+0000 7f7873200640 1 -- start start 2024-08-19T01:39:45.456 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:39:45.446+0000 7f7873200640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f78740729e0 0x7f78741156e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:39:45.456 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:39:45.446+0000 7f7873200640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f78740733b0 0x7f7874115c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:39:45.456 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:39:45.446+0000 7f7873200640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f787410d8e0 0x7f78741b6980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:39:45.456 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:39:45.446+0000 7f7873200640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f787411c270 con 0x7f78740733b0 2024-08-19T01:39:45.456 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:39:45.446+0000 7f7873200640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f787411c0f0 con 0x7f78740729e0 2024-08-19T01:39:45.457 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:39:45.446+0000 7f7873200640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f787411c3f0 con 0x7f787410d8e0 2024-08-19T01:39:45.457 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:39:45.451+0000 7f7871400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f78740733b0 0x7f7874115c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:39:45.457 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:39:45.451+0000 7f7871400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f78740733b0 0x7f7874115c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:38156/0 (socket says 172.21.15.83:38156) 2024-08-19T01:39:45.457 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:39:45.451+0000 7f7871400640 1 -- 172.21.15.83:0/1700971206 learned_addr learned my addr 172.21.15.83:0/1700971206 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-08-19T01:39:45.457 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:39:45.451+0000 7f7871400640 1 -- 172.21.15.83:0/1700971206 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f787410d8e0 msgr2=0x7f78741b6980 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-08-19T01:39:45.457 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:39:45.451+0000 7f7871400640 1 --2- 172.21.15.83:0/1700971206 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f787410d8e0 0x7f78741b6980 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:39:45.457 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:39:45.451+0000 7f7871400640 1 -- 172.21.15.83:0/1700971206 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f78740729e0 msgr2=0x7f78741156e0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-08-19T01:39:45.457 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:39:45.451+0000 7f7871400640 1 --2- 172.21.15.83:0/1700971206 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f78740729e0 0x7f78741156e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:39:45.457 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:39:45.451+0000 7f7871400640 1 -- 172.21.15.83:0/1700971206 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f78741b6ec0 con 0x7f78740733b0 2024-08-19T01:39:45.457 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:39:45.456+0000 7f7871400640 1 --2- 172.21.15.83:0/1700971206 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f78740733b0 0x7f7874115c20 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f7860009850 tx=0x7f7860032dc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:39:45.466 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:39:45.465+0000 7f786aa00640 1 -- 172.21.15.83:0/1700971206 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f786003f030 con 0x7f78740733b0 2024-08-19T01:39:45.466 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:39:45.465+0000 7f786aa00640 1 -- 172.21.15.83:0/1700971206 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7860003ce0 con 0x7f78740733b0 2024-08-19T01:39:45.466 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:39:45.465+0000 7f786aa00640 1 -- 172.21.15.83:0/1700971206 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7860035070 con 0x7f78740733b0 2024-08-19T01:39:45.470 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:39:45.465+0000 7f7873200640 1 -- 172.21.15.83:0/1700971206 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f78741b70f0 con 0x7f78740733b0 2024-08-19T01:39:45.470 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:39:45.465+0000 7f7873200640 1 -- 172.21.15.83:0/1700971206 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f78741b75e0 con 0x7f78740733b0 2024-08-19T01:39:45.514 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:39:45.507+0000 7f787c014740 1 Processor -- start 2024-08-19T01:39:45.529 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:39:45.521+0000 7f787c014740 1 -- start start 2024-08-19T01:39:45.529 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:39:45.521+0000 7f787c014740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557d0edb1660 0x557d0edb1a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:39:45.529 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:39:45.521+0000 7f787c014740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557d0edb1fa0 0x557d0edb6a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:39:45.529 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:39:45.521+0000 7f787c014740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557d0edb7030 0x557d0edb94f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:39:45.529 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:39:45.521+0000 7f787c014740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x557d0ed15890 con 0x557d0edb1fa0 2024-08-19T01:39:45.529 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:39:45.521+0000 7f787c014740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x557d0ecadd90 con 0x557d0edb1660 2024-08-19T01:39:45.530 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:39:45.521+0000 7f787c014740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x557d0ed11850 con 0x557d0edb7030 2024-08-19T01:39:45.530 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:39:45.522+0000 7f7871e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557d0edb1fa0 0x557d0edb6a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:39:45.530 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:39:45.522+0000 7f7871e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557d0edb1fa0 0x557d0edb6a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:38158/0 (socket says 172.21.15.83:38158) 2024-08-19T01:39:45.530 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:39:45.522+0000 7f7871e00640 1 -- 172.21.15.83:0/2931042999 learned_addr learned my addr 172.21.15.83:0/2931042999 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-08-19T01:39:45.530 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:39:45.522+0000 7f7872800640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557d0edb1660 0x557d0edb1a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:39:45.530 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:39:45.522+0000 7f7871400640 1 --2- 172.21.15.83:0/2931042999 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557d0edb7030 0x557d0edb94f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:39:45.530 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:39:45.523+0000 7f7872800640 1 -- 172.21.15.83:0/2931042999 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557d0edb7030 msgr2=0x557d0edb94f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:39:45.530 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:39:45.523+0000 7f7872800640 1 --2- 172.21.15.83:0/2931042999 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557d0edb7030 0x557d0edb94f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:39:45.530 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:39:45.523+0000 7f7872800640 1 -- 172.21.15.83:0/2931042999 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557d0edb1fa0 msgr2=0x557d0edb6a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:39:45.530 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:39:45.523+0000 7f7872800640 1 --2- 172.21.15.83:0/2931042999 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557d0edb1fa0 0x557d0edb6a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:39:45.530 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:39:45.523+0000 7f7872800640 1 -- 172.21.15.83:0/2931042999 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557d0ea29210 con 0x557d0edb1660 2024-08-19T01:39:45.530 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:39:45.523+0000 7f7872800640 1 --2- 172.21.15.83:0/2931042999 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557d0edb1660 0x557d0edb1a60 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f786c009cb0 tx=0x7f786c00b7e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=aac6bbf60fa3f379 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:39:45.530 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:39:45.524+0000 7f785f400640 1 -- 172.21.15.83:0/2931042999 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f786c009e60 con 0x557d0edb1660 2024-08-19T01:39:45.535 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:39:45.532+0000 7f786aa00640 1 -- 172.21.15.83:0/1700971206 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7f7860042030 con 0x7f78740733b0 2024-08-19T01:39:45.535 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:39:45.533+0000 7f785f400640 1 -- 172.21.15.83:0/2931042999 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f786c035b00 con 0x557d0edb1660 2024-08-19T01:39:45.535 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:39:45.533+0000 7f785f400640 1 -- 172.21.15.83:0/2931042999 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f786c034c00 con 0x557d0edb1660 2024-08-19T01:39:45.535 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:39:45.534+0000 7f786aa00640 1 --2- 172.21.15.83:0/1700971206 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f78580a2d90 0x7f78580a5250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:39:45.539 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:39:45.534+0000 7f787c014740 1 -- 172.21.15.83:0/2931042999 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557d0edb1660 msgr2=0x557d0edb1a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:39:45.539 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:39:45.534+0000 7f787c014740 1 --2- 172.21.15.83:0/2931042999 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557d0edb1660 0x557d0edb1a60 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f786c009cb0 tx=0x7f786c00b7e0 comp rx=0 tx=0).stop 2024-08-19T01:39:45.544 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:39:45.542+0000 7f786aa00640 1 -- 172.21.15.83:0/1700971206 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8269+0+0 (secure 0 0 0) 0x7f78600ef7d0 con 0x7f78740733b0 2024-08-19T01:39:45.545 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:39:45.543+0000 7f7871e00640 1 --2- 172.21.15.83:0/1700971206 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f78580a2d90 0x7f78580a5250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:39:45.546 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:39:45.544+0000 7f7871e00640 1 --2- 172.21.15.83:0/1700971206 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f78580a2d90 0x7f78580a5250 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f7864006fd0 tx=0x7f7864008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:39:45.549 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:39:45.548+0000 7f787c014740 1 -- 172.21.15.83:0/2931042999 shutdown_connections 2024-08-19T01:39:45.549 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:39:45.548+0000 7f787c014740 1 --2- 172.21.15.83:0/2931042999 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557d0edb7030 0x557d0edb94f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:39:45.549 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:39:45.548+0000 7f787c014740 1 --2- 172.21.15.83:0/2931042999 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557d0edb1fa0 0x557d0edb6a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:39:45.549 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:39:45.548+0000 7f787c014740 1 --2- 172.21.15.83:0/2931042999 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557d0edb1660 0x557d0edb1a60 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:39:45.549 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:39:45.548+0000 7f787c014740 1 -- 172.21.15.83:0/2931042999 >> 172.21.15.83:0/2931042999 conn(0x557d0edaccd0 msgr2=0x557d0edad0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:39:45.562 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:39:45.561+0000 7f787c014740 1 -- 172.21.15.83:0/2931042999 shutdown_connections 2024-08-19T01:39:45.566 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:39:45.561+0000 7f787c014740 1 -- 172.21.15.83:0/2931042999 wait complete. 2024-08-19T01:39:45.566 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:39:45.562+0000 7f787c014740 1 -- start start 2024-08-19T01:39:45.566 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:39:45.562+0000 7f787c014740 1 Processor -- start 2024-08-19T01:39:45.567 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:39:45.563+0000 7f787c014740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557d0edb1660 0x557d0edbb9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:39:45.574 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:39:45.570+0000 7f787c014740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557d0edb1fa0 0x557d0edbbf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:39:45.574 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:39:45.572+0000 7f7871e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557d0edb1660 0x557d0edbb9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:39:45.574 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:39:45.572+0000 7f7871e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557d0edb1660 0x557d0edbb9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:38168/0 (socket says 172.21.15.83:38168) 2024-08-19T01:39:45.574 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:39:45.572+0000 7f7871e00640 1 -- 172.21.15.83:0/773256221 learned_addr learned my addr 172.21.15.83:0/773256221 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-08-19T01:39:45.574 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:39:45.572+0000 7f787c014740 1 --2- 172.21.15.83:0/773256221 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557d0edb7030 0x557d0edc2f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:39:45.575 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:39:45.573+0000 7f7871400640 1 --2- 172.21.15.83:0/773256221 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557d0edb1fa0 0x557d0edbbf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:39:45.586 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:39:45.577+0000 7f787c014740 1 -- 172.21.15.83:0/773256221 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x557d0edba0a0 con 0x557d0edb1660 2024-08-19T01:39:45.586 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:39:45.577+0000 7f787c014740 1 -- 172.21.15.83:0/773256221 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x557d0ecd0f70 con 0x557d0edb7030 2024-08-19T01:39:45.586 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:39:45.577+0000 7f787c014740 1 -- 172.21.15.83:0/773256221 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x557d0edba220 con 0x557d0edb1fa0 2024-08-19T01:39:45.587 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:39:45.577+0000 7f787c014740 10 client.0 _ll_register_callbacks cb 0x557d0ed0d8d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-19T01:39:45.587 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:39:45.577+0000 7f7872800640 1 --2- 172.21.15.83:0/773256221 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557d0edb7030 0x557d0edc2f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:39:45.587 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:39:45.578+0000 7f7871400640 1 -- 172.21.15.83:0/773256221 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557d0edb7030 msgr2=0x557d0edc2f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:39:45.587 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:39:45.578+0000 7f7871400640 1 --2- 172.21.15.83:0/773256221 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557d0edb7030 0x557d0edc2f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:39:45.587 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:39:45.578+0000 7f7871400640 1 -- 172.21.15.83:0/773256221 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557d0edb1660 msgr2=0x557d0edbb9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:39:45.587 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:39:45.578+0000 7f7871400640 1 --2- 172.21.15.83:0/773256221 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557d0edb1660 0x557d0edbb9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:39:45.587 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:39:45.578+0000 7f7871400640 1 -- 172.21.15.83:0/773256221 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557d0eaf5620 con 0x557d0edb1fa0 2024-08-19T01:39:45.587 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:39:45.579+0000 7f7871e00640 1 --2- 172.21.15.83:0/773256221 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557d0edb1660 0x557d0edbb9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-19T01:39:45.587 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:39:45.579+0000 7f7871400640 1 --2- 172.21.15.83:0/773256221 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557d0edb1fa0 0x557d0edbbf30 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f786002aa30 tx=0x7f78600bbe60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2c9595f26ef920a1 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:39:45.587 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:39:45.583+0000 7f7853e00640 10 client.0 ms_handle_connect on v2:172.21.15.167:3300/0 2024-08-19T01:39:45.588 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:39:45.585+0000 7f7853e00640 1 -- 172.21.15.83:0/773256221 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78600ba430 con 0x557d0edb1fa0 2024-08-19T01:39:45.588 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:39:45.585+0000 7f7853e00640 1 -- 172.21.15.83:0/773256221 <== mon.1 v2:172.21.15.167:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8269+0+0 (secure 0 0 0) 0x7f78600c4790 con 0x557d0edb1fa0 2024-08-19T01:39:45.595 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:39:45.594+0000 7f787c014740 1 -- 172.21.15.83:0/773256221 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f786c035b00 con 0x557d0edb1fa0 2024-08-19T01:39:45.602 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:39:45.601+0000 7f7853e00640 1 -- 172.21.15.83:0/773256221 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f78600dc070 con 0x557d0edb1fa0 2024-08-19T01:39:45.606 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:39:45.605+0000 7f787c014740 10 client.25705 fetch_fsmap learned FSMap version 27 2024-08-19T01:39:45.606 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:39:45.605+0000 7f787c014740 1 -- 172.21.15.83:0/773256221 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x557d0edba170 con 0x557d0edb1fa0 2024-08-19T01:39:45.611 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:39:45.606+0000 7f7853e00640 1 -- 172.21.15.83:0/773256221 <== mon.1 v2:172.21.15.167:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f786c035b00 con 0x557d0edb1fa0 2024-08-19T01:39:45.611 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:39:45.606+0000 7f7853e00640 1 -- 172.21.15.83:0/773256221 <== mon.1 v2:172.21.15.167:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f78600bb500 con 0x557d0edb1fa0 2024-08-19T01:39:45.611 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:39:45.607+0000 7f787c014740 10 client.25705 fetch_fsmap finished waiting for FSMap version 27 2024-08-19T01:39:45.611 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:39:45.607+0000 7f787c014740 10 client.25705 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-08-19T01:39:45.611 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:39:45.607+0000 7f787c014740 20 client.25705 populate_metadata read hostname 'smithi083' 2024-08-19T01:39:45.611 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:39:45.607+0000 7f787c014740 1 --2- 172.21.15.83:0/773256221 >> [v2:172.21.15.117:6838/1967595704,v1:172.21.15.117:6839/1967595704] conn(0x557d0edc7800 0x557d0edc9c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:39:45.611 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:39:45.607+0000 7f787c014740 4 client.25705 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-19T01:39:45.611 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:39:45.607+0000 7f787c014740 1 -- 172.21.15.83:0/773256221 --> [v2:172.21.15.117:6838/1967595704,v1:172.21.15.117:6839/1967595704] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557d0ed22fc0 con 0x557d0edc7800 2024-08-19T01:39:45.615 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:39:45.614+0000 7f7871e00640 1 --2- 172.21.15.83:0/773256221 >> [v2:172.21.15.117:6838/1967595704,v1:172.21.15.117:6839/1967595704] conn(0x557d0edc7800 0x557d0edc9c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:39:45.623 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:39:45.622+0000 7f7871e00640 1 --2- 172.21.15.83:0/773256221 >> [v2:172.21.15.117:6838/1967595704,v1:172.21.15.117:6839/1967595704] conn(0x557d0edc7800 0x557d0edc9c60 crc :-1 s=READY pgs=97 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b7de40f2e17de909 server_cookie=c57be24bfa9fa907 in_seq=0 out_seq=0 2024-08-19T01:39:45.623 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:39:45.622+0000 7f7853e00640 10 client.25705 ms_handle_connect on v2:172.21.15.117:6838/1967595704 2024-08-19T01:39:45.627 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:39:45.626+0000 7f7853e00640 1 -- 172.21.15.83:0/773256221 <== mds.0 v2:172.21.15.117:6838/1967595704 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x557d0ed22fc0 con 0x557d0edc7800 2024-08-19T01:39:45.627 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:39:45.626+0000 7f7853e00640 10 client.25705 handle_command_reply: tid=0 2024-08-19T01:39:45.643 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:39:45.634+0000 7f787c014740 1 client.25705 shutdown 2024-08-19T01:39:45.661 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:39:45.660+0000 7f787c014740 1 -- 172.21.15.83:0/773256221 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557d0edb1fa0 msgr2=0x557d0edbbf30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:39:45.661 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:39:45.660+0000 7f787c014740 1 --2- 172.21.15.83:0/773256221 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557d0edb1fa0 0x557d0edbbf30 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f786002aa30 tx=0x7f78600bbe60 comp rx=0 tx=0).stop 2024-08-19T01:39:45.666 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:39:45.665+0000 7f787c014740 1 -- 172.21.15.83:0/773256221 shutdown_connections 2024-08-19T01:39:45.666 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:39:45.665+0000 7f787c014740 1 --2- 172.21.15.83:0/773256221 >> [v2:172.21.15.117:6838/1967595704,v1:172.21.15.117:6839/1967595704] conn(0x557d0edc7800 0x557d0edc9c60 crc :-1 s=READY pgs=97 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:39:45.666 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:39:45.665+0000 7f787c014740 1 --2- 172.21.15.83:0/773256221 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557d0edb7030 0x557d0edc2f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:39:45.666 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:39:45.665+0000 7f787c014740 1 --2- 172.21.15.83:0/773256221 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557d0edb1fa0 0x557d0edbbf30 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:39:45.666 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:39:45.665+0000 7f787c014740 1 --2- 172.21.15.83:0/773256221 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557d0edb1660 0x557d0edbb9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:39:45.667 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:39:45.665+0000 7f787c014740 1 -- 172.21.15.83:0/773256221 >> 172.21.15.83:0/773256221 conn(0x557d0edaccd0 msgr2=0x557d0edae4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:39:45.674 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:39:45.669+0000 7f7853e00640 0 client.25705 ms_handle_reset on v2:172.21.15.117:6838/1967595704 2024-08-19T01:39:45.674 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:39:45.673+0000 7f787c014740 1 -- 172.21.15.83:0/773256221 shutdown_connections 2024-08-19T01:39:45.674 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:39:45.673+0000 7f787c014740 1 -- 172.21.15.83:0/773256221 wait complete. 2024-08-19T01:39:45.675 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:39:45.673+0000 7f787c014740 20 client.25705 trim_cache size 0 max 16384 2024-08-19T01:39:45.817 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:39:45 smithi083 ceph-mon[17120]: pgmap v486: 97 pgs: 97 active+clean; 440 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 3.3 MiB/s wr, 556 op/s 2024-08-19T01:39:45.952 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:39:45 smithi117 ceph-mon[22016]: pgmap v486: 97 pgs: 97 active+clean; 440 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 3.3 MiB/s wr, 556 op/s 2024-08-19T01:39:45.965 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:39:45 smithi167 ceph-mon[21875]: pgmap v486: 97 pgs: 97 active+clean; 440 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 3.3 MiB/s wr, 556 op/s 2024-08-19T01:39:46.243 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:39:46.236+0000 7f787c014740 1 Processor -- start 2024-08-19T01:39:46.248 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:39:46.242+0000 7f787c014740 1 -- start start 2024-08-19T01:39:46.248 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:39:46.242+0000 7f787c014740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557d0edb1660 0x557d0eda7e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:39:46.248 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:39:46.242+0000 7f787c014740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557d0edb1fa0 0x557d0edc2ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:39:46.248 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:39:46.242+0000 7f787c014740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557d0edb7030 0x557d0edad0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:39:46.248 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:39:46.242+0000 7f787c014740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f7854010eb0 con 0x557d0edb1660 2024-08-19T01:39:46.248 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:39:46.242+0000 7f787c014740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f785400e780 con 0x557d0edb7030 2024-08-19T01:39:46.248 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:39:46.242+0000 7f787c014740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f785400c5d0 con 0x557d0edb1fa0 2024-08-19T01:39:46.249 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:39:46.244+0000 7f7872800640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557d0edb7030 0x557d0edad0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:39:46.249 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:39:46.244+0000 7f7872800640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557d0edb7030 0x557d0edad0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.83:42952/0 (socket says 172.21.15.83:42952) 2024-08-19T01:39:46.249 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:39:46.244+0000 7f7872800640 1 -- 172.21.15.83:0/1066887435 learned_addr learned my addr 172.21.15.83:0/1066887435 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-08-19T01:39:46.249 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:39:46.244+0000 7f7871400640 1 --2- 172.21.15.83:0/1066887435 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557d0edb1fa0 0x557d0edc2ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:39:46.249 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:39:46.247+0000 7f7872800640 1 -- 172.21.15.83:0/1066887435 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557d0edb1fa0 msgr2=0x557d0edc2ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:39:46.249 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:39:46.247+0000 7f7872800640 1 --2- 172.21.15.83:0/1066887435 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557d0edb1fa0 0x557d0edc2ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:39:46.249 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:39:46.247+0000 7f7872800640 1 -- 172.21.15.83:0/1066887435 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557d0edb1660 msgr2=0x557d0eda7e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:39:46.250 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:39:46.247+0000 7f7872800640 1 --2- 172.21.15.83:0/1066887435 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557d0edb1660 0x557d0eda7e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:39:46.250 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:39:46.247+0000 7f7872800640 1 -- 172.21.15.83:0/1066887435 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557d0edb2480 con 0x557d0edb7030 2024-08-19T01:39:46.250 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:39:46.248+0000 7f7872800640 1 --2- 172.21.15.83:0/1066887435 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557d0edb7030 0x557d0edad0b0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f786c00bf90 tx=0x7f786c009e60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cd61311c1a3c660 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:39:46.285 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:39:46.284+0000 7f785ea00640 1 -- 172.21.15.83:0/1066887435 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f786c003c10 con 0x557d0edb7030 2024-08-19T01:39:46.285 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:39:46.284+0000 7f785ea00640 1 -- 172.21.15.83:0/1066887435 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f786c044070 con 0x557d0edb7030 2024-08-19T01:39:46.285 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:39:46.284+0000 7f785ea00640 1 -- 172.21.15.83:0/1066887435 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f786c035680 con 0x557d0edb7030 2024-08-19T01:39:46.293 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:39:46.291+0000 7f787c014740 1 -- 172.21.15.83:0/1066887435 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557d0edb7030 msgr2=0x557d0edad0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:39:46.293 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:39:46.291+0000 7f787c014740 1 --2- 172.21.15.83:0/1066887435 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557d0edb7030 0x557d0edad0b0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f786c00bf90 tx=0x7f786c009e60 comp rx=0 tx=0).stop 2024-08-19T01:39:46.317 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:39:46.313+0000 7f787c014740 1 -- 172.21.15.83:0/1066887435 shutdown_connections 2024-08-19T01:39:46.317 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:39:46.313+0000 7f787c014740 1 --2- 172.21.15.83:0/1066887435 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557d0edb7030 0x557d0edad0b0 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:39:46.317 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:39:46.313+0000 7f787c014740 1 --2- 172.21.15.83:0/1066887435 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557d0edb1fa0 0x557d0edc2ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:39:46.317 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:39:46.313+0000 7f787c014740 1 --2- 172.21.15.83:0/1066887435 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557d0edb1660 0x557d0eda7e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:39:46.317 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:39:46.313+0000 7f787c014740 1 -- 172.21.15.83:0/1066887435 >> 172.21.15.83:0/1066887435 conn(0x557d0edaccd0 msgr2=0x557d0edbb9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:39:46.335 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:39:46.334+0000 7f787c014740 1 -- 172.21.15.83:0/1066887435 shutdown_connections 2024-08-19T01:39:46.336 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:39:46.334+0000 7f787c014740 1 -- 172.21.15.83:0/1066887435 wait complete. 2024-08-19T01:39:46.336 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:39:46.335+0000 7f787c014740 1 -- start start 2024-08-19T01:39:46.336 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:39:46.335+0000 7f787c014740 1 Processor -- start 2024-08-19T01:39:46.360 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:39:46.358+0000 7f787c014740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557d0edb1660 0x557d0edbff10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:39:46.360 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:39:46.358+0000 7f787c014740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557d0edb1fa0 0x557d0edc0450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:39:46.361 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:39:46.358+0000 7f787c014740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557d0edb7030 0x557d0ee304f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:39:46.361 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:39:46.358+0000 7f787c014740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f785400cc40 con 0x557d0edb1660 2024-08-19T01:39:46.361 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:39:46.358+0000 7f787c014740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f785400e0b0 con 0x557d0edb1fa0 2024-08-19T01:39:46.361 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:39:46.358+0000 7f787c014740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f785400d310 con 0x557d0edb7030 2024-08-19T01:39:46.361 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:39:46.358+0000 7f787c014740 10 client.0 _ll_register_callbacks cb 0x7f786003bec0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-19T01:39:46.365 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:39:46.359+0000 7f7871e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557d0edb1660 0x557d0edbff10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:39:46.365 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:39:46.360+0000 7f7871e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557d0edb1660 0x557d0edbff10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:38184/0 (socket says 172.21.15.83:38184) 2024-08-19T01:39:46.365 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:39:46.360+0000 7f7871e00640 1 -- 172.21.15.83:0/3297194087 learned_addr learned my addr 172.21.15.83:0/3297194087 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-08-19T01:39:46.388 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:39:46.370+0000 7f7872800640 1 --2- 172.21.15.83:0/3297194087 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557d0edb7030 0x557d0ee304f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:39:46.388 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:39:46.374+0000 7f7871400640 1 --2- 172.21.15.83:0/3297194087 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557d0edb1fa0 0x557d0edc0450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:39:46.389 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:39:46.387+0000 7f7872800640 1 -- 172.21.15.83:0/3297194087 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557d0edb1fa0 msgr2=0x557d0edc0450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:39:46.389 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:39:46.387+0000 7f7872800640 1 --2- 172.21.15.83:0/3297194087 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557d0edb1fa0 0x557d0edc0450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:39:46.389 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:39:46.387+0000 7f7872800640 1 -- 172.21.15.83:0/3297194087 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557d0edb1660 msgr2=0x557d0edbff10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:39:46.389 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:39:46.387+0000 7f7872800640 1 --2- 172.21.15.83:0/3297194087 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557d0edb1660 0x557d0edbff10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:39:46.389 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:39:46.387+0000 7f7872800640 1 -- 172.21.15.83:0/3297194087 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557d0ee30ae0 con 0x557d0edb7030 2024-08-19T01:39:46.389 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:39:46.387+0000 7f7872800640 1 --2- 172.21.15.83:0/3297194087 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557d0edb7030 0x557d0ee304f0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f786c004750 tx=0x7f786c035b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b7be353b725cb5 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:39:46.390 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:39:46.388+0000 7f7853e00640 10 client.0 ms_handle_connect on v2:172.21.15.167:3300/0 2024-08-19T01:39:46.390 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:39:46.388+0000 7f7853e00640 1 -- 172.21.15.83:0/3297194087 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f786c049670 con 0x557d0edb7030 2024-08-19T01:39:46.390 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:39:46.388+0000 7f7853e00640 1 -- 172.21.15.83:0/3297194087 <== mon.1 v2:172.21.15.167:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8269+0+0 (secure 0 0 0) 0x7f786c049810 con 0x557d0edb7030 2024-08-19T01:39:46.414 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:39:46.413+0000 7f787c014740 1 -- 172.21.15.83:0/3297194087 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f786c044070 con 0x557d0edb7030 2024-08-19T01:39:46.415 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:39:46.414+0000 7f7853e00640 1 -- 172.21.15.83:0/3297194087 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f786c004950 con 0x557d0edb7030 2024-08-19T01:39:46.415 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:39:46.414+0000 7f787c014740 10 client.25711 fetch_fsmap learned FSMap version 27 2024-08-19T01:39:46.415 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:39:46.414+0000 7f787c014740 1 -- 172.21.15.83:0/3297194087 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x557d0ee31df0 con 0x557d0edb7030 2024-08-19T01:39:46.437 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:39:46.433+0000 7f7853e00640 1 -- 172.21.15.83:0/3297194087 <== mon.1 v2:172.21.15.167:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f786c044070 con 0x557d0edb7030 2024-08-19T01:39:46.437 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:39:46.433+0000 7f7853e00640 1 -- 172.21.15.83:0/3297194087 <== mon.1 v2:172.21.15.167:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f786c051a10 con 0x557d0edb7030 2024-08-19T01:39:46.459 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:39:46.442+0000 7f787c014740 10 client.25711 fetch_fsmap finished waiting for FSMap version 27 2024-08-19T01:39:46.460 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:39:46.442+0000 7f787c014740 10 client.25711 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-08-19T01:39:46.460 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:39:46.442+0000 7f787c014740 20 client.25711 populate_metadata read hostname 'smithi083' 2024-08-19T01:39:46.460 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:39:46.442+0000 7f787c014740 1 --2- 172.21.15.83:0/3297194087 >> [v2:172.21.15.117:6838/1967595704,v1:172.21.15.117:6839/1967595704] conn(0x557d0ee32450 0x557d0ee34840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:39:46.460 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:39:46.442+0000 7f787c014740 4 client.25711 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-08-19T01:39:46.460 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:39:46.442+0000 7f787c014740 1 -- 172.21.15.83:0/3297194087 --> [v2:172.21.15.117:6838/1967595704,v1:172.21.15.117:6839/1967595704] -- command(tid 0: {"prefix": "flush journal"}) -- 0x557d0ed1d310 con 0x557d0ee32450 2024-08-19T01:39:46.460 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:39:46.448+0000 7f7871e00640 1 --2- 172.21.15.83:0/3297194087 >> [v2:172.21.15.117:6838/1967595704,v1:172.21.15.117:6839/1967595704] conn(0x557d0ee32450 0x557d0ee34840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:39:46.463 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:39:46.462+0000 7f7871e00640 1 --2- 172.21.15.83:0/3297194087 >> [v2:172.21.15.117:6838/1967595704,v1:172.21.15.117:6839/1967595704] conn(0x557d0ee32450 0x557d0ee34840 crc :-1 s=READY pgs=98 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a8a8e23f2c9cedf server_cookie=5f308e2edccc8ed3 in_seq=0 out_seq=0 2024-08-19T01:39:46.477 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:39:46.476+0000 7f7853e00640 10 client.25711 ms_handle_connect on v2:172.21.15.117:6838/1967595704 2024-08-19T01:39:47.693 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:39:47 smithi083 ceph-mon[17120]: pgmap v487: 97 pgs: 97 active+clean; 442 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.9 MiB/s wr, 485 op/s 2024-08-19T01:39:47.952 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:39:47 smithi117 ceph-mon[22016]: pgmap v487: 97 pgs: 97 active+clean; 442 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.9 MiB/s wr, 485 op/s 2024-08-19T01:39:47.965 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:39:47 smithi167 ceph-mon[21875]: pgmap v487: 97 pgs: 97 active+clean; 442 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.9 MiB/s wr, 485 op/s 2024-08-19T01:39:49.605 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:39:49 smithi083 ceph-mon[17120]: pgmap v488: 97 pgs: 97 active+clean; 490 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 7.4 MiB/s wr, 804 op/s 2024-08-19T01:39:49.952 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:39:49 smithi117 ceph-mon[22016]: pgmap v488: 97 pgs: 97 active+clean; 490 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 7.4 MiB/s wr, 804 op/s 2024-08-19T01:39:49.965 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:39:49 smithi167 ceph-mon[21875]: pgmap v488: 97 pgs: 97 active+clean; 490 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 7.4 MiB/s wr, 804 op/s 2024-08-19T01:39:50.139 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/bad8780e-5dc9-11ef-bccf-c7b262605968/mon.a/config 2024-08-19T01:39:50.632 INFO:tasks.workunit.client.0.smithi083.stdout: 156 499725 325 326088 166 288331 796 2024-08-19T01:39:51.167 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:39:51.164+0000 7f0a45c00640 1 -- 172.21.15.83:0/1526734678 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0a4010bd00 msgr2=0x7f0a4010e180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:39:51.167 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:39:51.164+0000 7f0a45c00640 1 --2- 172.21.15.83:0/1526734678 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0a4010bd00 0x7f0a4010e180 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f0a38009be0 tx=0x7f0a3802a750 comp rx=0 tx=0).stop 2024-08-19T01:39:51.169 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:39:51.164+0000 7f0a45c00640 1 -- 172.21.15.83:0/1526734678 shutdown_connections 2024-08-19T01:39:51.169 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:39:51.164+0000 7f0a45c00640 1 --2- 172.21.15.83:0/1526734678 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0a4010bd00 0x7f0a4010e180 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:39:51.170 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:39:51.164+0000 7f0a45c00640 1 --2- 172.21.15.83:0/1526734678 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0a40103240 0x7f0a4010b730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:39:51.170 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:39:51.164+0000 7f0a45c00640 1 --2- 172.21.15.83:0/1526734678 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0a40102920 0x7f0a40102d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:39:51.170 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:39:51.164+0000 7f0a45c00640 1 -- 172.21.15.83:0/1526734678 >> 172.21.15.83:0/1526734678 conn(0x7f0a400fc340 msgr2=0x7f0a400fe760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:39:51.170 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:39:51.164+0000 7f0a45c00640 1 -- 172.21.15.83:0/1526734678 shutdown_connections 2024-08-19T01:39:51.171 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:39:51.164+0000 7f0a45c00640 1 -- 172.21.15.83:0/1526734678 wait complete. 2024-08-19T01:39:51.172 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:39:51.170+0000 7f0a45c00640 1 Processor -- start 2024-08-19T01:39:51.173 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:39:51.170+0000 7f0a45c00640 1 -- start start 2024-08-19T01:39:51.173 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:39:51.170+0000 7f0a45c00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0a40102920 0x7f0a401004b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:39:51.173 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:39:51.170+0000 7f0a45c00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0a40103240 0x7f0a401009f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:39:51.174 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:39:51.170+0000 7f0a45c00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0a4010bd00 0x7f0a40100f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:39:51.174 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:39:51.170+0000 7f0a3f400640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0a40103240 0x7f0a401009f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:39:51.174 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:39:51.170+0000 7f0a3f400640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0a40103240 0x7f0a401009f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.83:45846/0 (socket says 172.21.15.83:45846) 2024-08-19T01:39:51.175 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:39:51.170+0000 7f0a45200640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0a4010bd00 0x7f0a40100f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:39:51.176 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:39:51.170+0000 7f0a45200640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0a4010bd00 0x7f0a40100f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:57788/0 (socket says 172.21.15.83:57788) 2024-08-19T01:39:51.176 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:39:51.170+0000 7f0a3fe00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0a40102920 0x7f0a401004b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:39:51.176 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:39:51.170+0000 7f0a3fe00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0a40102920 0x7f0a401004b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.83:41498/0 (socket says 172.21.15.83:41498) 2024-08-19T01:39:51.176 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:39:51.170+0000 7f0a3fe00640 1 -- 172.21.15.83:0/1022357715 learned_addr learned my addr 172.21.15.83:0/1022357715 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-08-19T01:39:51.177 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:39:51.175+0000 7f0a45c00640 1 -- 172.21.15.83:0/1022357715 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f0a40111ae0 con 0x7f0a4010bd00 2024-08-19T01:39:51.177 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:39:51.175+0000 7f0a45c00640 1 -- 172.21.15.83:0/1022357715 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f0a40111960 con 0x7f0a40102920 2024-08-19T01:39:51.177 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:39:51.175+0000 7f0a45c00640 1 -- 172.21.15.83:0/1022357715 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f0a40111c60 con 0x7f0a40103240 2024-08-19T01:39:51.178 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:39:51.176+0000 7f0a3fe00640 1 -- 172.21.15.83:0/1022357715 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0a40103240 msgr2=0x7f0a401009f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:39:51.179 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:39:51.176+0000 7f0a3fe00640 1 --2- 172.21.15.83:0/1022357715 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0a40103240 0x7f0a401009f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:39:51.179 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:39:51.176+0000 7f0a3fe00640 1 -- 172.21.15.83:0/1022357715 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0a4010bd00 msgr2=0x7f0a40100f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:39:51.180 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:39:51.176+0000 7f0a3fe00640 1 --2- 172.21.15.83:0/1022357715 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0a4010bd00 0x7f0a40100f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:39:51.180 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:39:51.176+0000 7f0a3fe00640 1 -- 172.21.15.83:0/1022357715 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0a40072810 con 0x7f0a40102920 2024-08-19T01:39:51.180 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:39:51.176+0000 7f0a3f400640 1 --2- 172.21.15.83:0/1022357715 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0a40103240 0x7f0a401009f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-19T01:39:51.180 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:39:51.176+0000 7f0a45200640 1 --2- 172.21.15.83:0/1022357715 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0a4010bd00 0x7f0a40100f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-19T01:39:51.180 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:39:51.176+0000 7f0a3fe00640 1 --2- 172.21.15.83:0/1022357715 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0a40102920 0x7f0a401004b0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f0a3000d950 tx=0x7f0a3000de10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:39:51.180 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:39:51.176+0000 7f0a3cc00640 1 -- 172.21.15.83:0/1022357715 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a30017070 con 0x7f0a40102920 2024-08-19T01:39:51.181 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:39:51.176+0000 7f0a3cc00640 1 -- 172.21.15.83:0/1022357715 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0a30009c30 con 0x7f0a40102920 2024-08-19T01:39:51.181 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:39:51.176+0000 7f0a45c00640 1 -- 172.21.15.83:0/1022357715 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0a40072b00 con 0x7f0a40102920 2024-08-19T01:39:51.181 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:39:51.176+0000 7f0a3cc00640 1 -- 172.21.15.83:0/1022357715 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a3000f6b0 con 0x7f0a40102920 2024-08-19T01:39:51.195 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:39:51.176+0000 7f0a45c00640 1 -- 172.21.15.83:0/1022357715 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0a40073040 con 0x7f0a40102920 2024-08-19T01:39:51.195 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:39:51.187+0000 7f0a49e37740 1 Processor -- start 2024-08-19T01:39:51.202 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:39:51.189+0000 7f0a3cc00640 1 -- 172.21.15.83:0/1022357715 <== mon.2 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7f0a300109d0 con 0x7f0a40102920 2024-08-19T01:39:51.203 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:39:51.194+0000 7f0a49e37740 1 -- start start 2024-08-19T01:39:51.203 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:39:51.194+0000 7f0a49e37740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55654cecbd80 0x55654cecc160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:39:51.203 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:39:51.194+0000 7f0a49e37740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55654cecc6a0 0x55654ced6a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:39:51.203 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:39:51.194+0000 7f0a49e37740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55654ced6f60 0x55654ced9350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:39:51.204 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:39:51.194+0000 7f0a49e37740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55654ce31bf0 con 0x55654cecc6a0 2024-08-19T01:39:51.204 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:39:51.194+0000 7f0a49e37740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55654cdc9bd0 con 0x55654ced6f60 2024-08-19T01:39:51.204 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:39:51.194+0000 7f0a49e37740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55654ce2dbb0 con 0x55654cecbd80 2024-08-19T01:39:51.204 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:39:51.194+0000 7f0a45200640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55654cecbd80 0x55654cecc160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:39:51.204 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:39:51.194+0000 7f0a45200640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55654cecbd80 0x55654cecc160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.83:45850/0 (socket says 172.21.15.83:45850) 2024-08-19T01:39:51.204 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:39:51.194+0000 7f0a45200640 1 -- 172.21.15.83:0/2674414263 learned_addr learned my addr 172.21.15.83:0/2674414263 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-08-19T01:39:51.205 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:39:51.194+0000 7f0a3f400640 1 --2- 172.21.15.83:0/2674414263 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55654ced6f60 0x55654ced9350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:39:51.205 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:39:51.194+0000 7f0a3f400640 1 -- 172.21.15.83:0/2674414263 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55654cecbd80 msgr2=0x55654cecc160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:39:51.205 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:39:51.194+0000 7f0a3f400640 1 --2- 172.21.15.83:0/2674414263 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55654cecbd80 0x55654cecc160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:39:51.205 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:39:51.194+0000 7f0a3f400640 1 -- 172.21.15.83:0/2674414263 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55654cecc6a0 msgr2=0x55654ced6a20 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-08-19T01:39:51.205 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:39:51.194+0000 7f0a3f400640 1 --2- 172.21.15.83:0/2674414263 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55654cecc6a0 0x55654ced6a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:39:51.205 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:39:51.194+0000 7f0a3f400640 1 -- 172.21.15.83:0/2674414263 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55654cdd08a0 con 0x55654ced6f60 2024-08-19T01:39:51.205 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:39:51.194+0000 7f0a3f400640 1 --2- 172.21.15.83:0/2674414263 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55654ced6f60 0x55654ced9350 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f0a2c005fd0 tx=0x7f0a2c00f730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2aecbea48dfdf861 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:39:51.206 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:39:51.199+0000 7f0a35600640 1 -- 172.21.15.83:0/2674414263 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a2c037070 con 0x55654ced6f60 2024-08-19T01:39:51.206 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:39:51.199+0000 7f0a35600640 1 -- 172.21.15.83:0/2674414263 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0a2c0035f0 con 0x55654ced6f60 2024-08-19T01:39:51.206 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:39:51.199+0000 7f0a35600640 1 -- 172.21.15.83:0/2674414263 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a2c0336b0 con 0x55654ced6f60 2024-08-19T01:39:51.206 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:39:51.201+0000 7f0a3cc00640 1 --2- 172.21.15.83:0/1022357715 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f0a240a2cd0 0x7f0a240a5190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:39:51.206 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:39:51.201+0000 7f0a49e37740 1 -- 172.21.15.83:0/2674414263 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55654ced6f60 msgr2=0x55654ced9350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:39:51.206 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:39:51.201+0000 7f0a49e37740 1 --2- 172.21.15.83:0/2674414263 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55654ced6f60 0x55654ced9350 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f0a2c005fd0 tx=0x7f0a2c00f730 comp rx=0 tx=0).stop 2024-08-19T01:39:51.209 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:39:51.201+0000 7f0a3cc00640 1 -- 172.21.15.83:0/1022357715 <== mon.2 v2:172.21.15.117:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8269+0+0 (secure 0 0 0) 0x7f0a300cecf0 con 0x7f0a40102920 2024-08-19T01:39:51.209 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:39:51.201+0000 7f0a49e37740 1 -- 172.21.15.83:0/2674414263 shutdown_connections 2024-08-19T01:39:51.209 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:39:51.201+0000 7f0a49e37740 1 --2- 172.21.15.83:0/2674414263 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55654ced6f60 0x55654ced9350 secure :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f0a2c005fd0 tx=0x7f0a2c00f730 comp rx=0 tx=0).stop 2024-08-19T01:39:51.209 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:39:51.201+0000 7f0a49e37740 1 --2- 172.21.15.83:0/2674414263 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55654cecc6a0 0x55654ced6a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:39:51.209 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:39:51.201+0000 7f0a49e37740 1 --2- 172.21.15.83:0/2674414263 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55654cecbd80 0x55654cecc160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:39:51.209 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:39:51.201+0000 7f0a49e37740 1 -- 172.21.15.83:0/2674414263 >> 172.21.15.83:0/2674414263 conn(0x55654cec8310 msgr2=0x55654cec86f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:39:51.210 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:39:51.201+0000 7f0a45200640 1 --2- 172.21.15.83:0/1022357715 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f0a240a2cd0 0x7f0a240a5190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:39:51.210 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:39:51.201+0000 7f0a49e37740 1 -- 172.21.15.83:0/2674414263 shutdown_connections 2024-08-19T01:39:51.210 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:39:51.201+0000 7f0a49e37740 1 -- 172.21.15.83:0/2674414263 wait complete. 2024-08-19T01:39:51.210 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:39:51.201+0000 7f0a45200640 1 --2- 172.21.15.83:0/1022357715 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f0a240a2cd0 0x7f0a240a5190 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f0a380099a0 tx=0x7f0a3800b040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:39:51.210 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:39:51.201+0000 7f0a49e37740 1 -- start start 2024-08-19T01:39:51.210 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:39:51.201+0000 7f0a49e37740 1 Processor -- start 2024-08-19T01:39:51.211 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:39:51.201+0000 7f0a49e37740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55654cecbd80 0x55654ced4120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:39:51.211 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:39:51.201+0000 7f0a49e37740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55654cecc6a0 0x55654ced4660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:39:51.211 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:39:51.207+0000 7f0a3fe00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55654cecbd80 0x55654ced4120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:39:51.211 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:39:51.207+0000 7f0a3fe00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55654cecbd80 0x55654ced4120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.83:45864/0 (socket says 172.21.15.83:45864) 2024-08-19T01:39:51.213 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:39:51.207+0000 7f0a3fe00640 1 -- 172.21.15.83:0/3409925032 learned_addr learned my addr 172.21.15.83:0/3409925032 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-08-19T01:39:51.213 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:39:51.207+0000 7f0a49e37740 1 --2- 172.21.15.83:0/3409925032 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55654ced4bf0 0x55654cee2fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:39:51.213 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:39:51.207+0000 7f0a3f400640 1 --2- 172.21.15.83:0/3409925032 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55654cecc6a0 0x55654ced4660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:39:51.213 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:39:51.207+0000 7f0a45200640 1 --2- 172.21.15.83:0/3409925032 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55654ced4bf0 0x55654cee2fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:39:51.213 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:39:51.207+0000 7f0a49e37740 1 -- 172.21.15.83:0/3409925032 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55654ced9b30 con 0x55654cecc6a0 2024-08-19T01:39:51.213 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:39:51.207+0000 7f0a49e37740 1 -- 172.21.15.83:0/3409925032 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55654cdecd40 con 0x55654ced4bf0 2024-08-19T01:39:51.214 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:39:51.207+0000 7f0a49e37740 1 -- 172.21.15.83:0/3409925032 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55654ced9cb0 con 0x55654cecbd80 2024-08-19T01:39:51.214 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:39:51.207+0000 7f0a49e37740 10 client.0 _ll_register_callbacks cb 0x55654cdb5b10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-19T01:39:51.215 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:39:51.212+0000 7f0a3f400640 1 -- 172.21.15.83:0/3409925032 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55654cecbd80 msgr2=0x55654ced4120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:39:51.215 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:39:51.212+0000 7f0a3f400640 1 --2- 172.21.15.83:0/3409925032 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55654cecbd80 0x55654ced4120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:39:51.216 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:39:51.212+0000 7f0a3f400640 1 -- 172.21.15.83:0/3409925032 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55654ced4bf0 msgr2=0x55654cee2fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:39:51.216 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:39:51.212+0000 7f0a3f400640 1 --2- 172.21.15.83:0/3409925032 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55654ced4bf0 0x55654cee2fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:39:51.216 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:39:51.212+0000 7f0a3f400640 1 -- 172.21.15.83:0/3409925032 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55654cba33c0 con 0x55654cecc6a0 2024-08-19T01:39:51.216 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:39:51.212+0000 7f0a45200640 1 --2- 172.21.15.83:0/3409925032 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55654ced4bf0 0x55654cee2fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-19T01:39:51.216 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:39:51.212+0000 7f0a3fe00640 1 --2- 172.21.15.83:0/3409925032 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55654cecbd80 0x55654ced4120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-19T01:39:51.217 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:39:51.213+0000 7f0a3f400640 1 --2- 172.21.15.83:0/3409925032 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55654cecc6a0 0x55654ced4660 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x55654cec93c0 tx=0x7f0a2c00b850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7e94f673d12e639e server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:39:51.218 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:39:51.213+0000 7f0a2a000640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-08-19T01:39:51.218 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:39:51.213+0000 7f0a2a000640 1 -- 172.21.15.83:0/3409925032 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a2c037070 con 0x55654cecc6a0 2024-08-19T01:39:51.218 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:39:51.213+0000 7f0a49e37740 1 -- 172.21.15.83:0/3409925032 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0a2c0035f0 con 0x55654cecc6a0 2024-08-19T01:39:51.219 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:39:51.213+0000 7f0a2a000640 1 -- 172.21.15.83:0/3409925032 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8269+0+0 (secure 0 0 0) 0x7f0a2c0039b0 con 0x55654cecc6a0 2024-08-19T01:39:51.219 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:39:51.213+0000 7f0a2a000640 1 -- 172.21.15.83:0/3409925032 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f0a2c004000 con 0x55654cecc6a0 2024-08-19T01:39:51.220 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:39:51.213+0000 7f0a49e37740 10 client.15747 fetch_fsmap learned FSMap version 27 2024-08-19T01:39:51.220 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:39:51.213+0000 7f0a49e37740 1 -- 172.21.15.83:0/3409925032 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55654cee5ea0 con 0x55654cecc6a0 2024-08-19T01:39:51.220 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:39:51.213+0000 7f0a2a000640 1 -- 172.21.15.83:0/3409925032 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0a2c0035f0 con 0x55654cecc6a0 2024-08-19T01:39:51.221 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:39:51.213+0000 7f0a2a000640 1 -- 172.21.15.83:0/3409925032 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f0a2c033a10 con 0x55654cecc6a0 2024-08-19T01:39:51.221 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:39:51.213+0000 7f0a49e37740 10 client.15747 fetch_fsmap finished waiting for FSMap version 27 2024-08-19T01:39:51.222 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:39:51.213+0000 7f0a49e37740 10 client.15747 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-19T01:39:51.222 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:39:51.219+0000 7f0a49e37740 20 client.15747 populate_metadata read hostname 'smithi083' 2024-08-19T01:39:51.222 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:39:51.219+0000 7f0a49e37740 1 --2- 172.21.15.83:0/3409925032 >> [v2:172.21.15.117:6838/1967595704,v1:172.21.15.117:6839/1967595704] conn(0x55654cee7250 0x55654cee9640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:39:51.222 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:39:51.219+0000 7f0a49e37740 4 client.15747 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-19T01:39:51.222 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:39:51.219+0000 7f0a49e37740 1 -- 172.21.15.83:0/3409925032 --> [v2:172.21.15.117:6838/1967595704,v1:172.21.15.117:6839/1967595704] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55654ce3f160 con 0x55654cee7250 2024-08-19T01:39:51.222 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:39:51.219+0000 7f0a3fe00640 1 --2- 172.21.15.83:0/3409925032 >> [v2:172.21.15.117:6838/1967595704,v1:172.21.15.117:6839/1967595704] conn(0x55654cee7250 0x55654cee9640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:39:51.223 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:39:51.219+0000 7f0a2a000640 10 client.15747 ms_handle_connect on v2:172.21.15.117:6838/1967595704 2024-08-19T01:39:51.223 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:39:51.219+0000 7f0a3fe00640 1 --2- 172.21.15.83:0/3409925032 >> [v2:172.21.15.117:6838/1967595704,v1:172.21.15.117:6839/1967595704] conn(0x55654cee7250 0x55654cee9640 crc :-1 s=READY pgs=99 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1c1891e5b1c33cf5 server_cookie=fe1ea41922d852ac in_seq=0 out_seq=0 2024-08-19T01:39:51.225 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:39:51.219+0000 7f0a2a000640 1 -- 172.21.15.83:0/3409925032 <== mds.0 v2:172.21.15.117:6838/1967595704 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55654ce3f160 con 0x55654cee7250 2024-08-19T01:39:51.225 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:39:51.219+0000 7f0a2a000640 10 client.15747 handle_command_reply: tid=0 2024-08-19T01:39:51.225 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:39:51.224+0000 7f0a49e37740 1 client.15747 shutdown 2024-08-19T01:39:51.227 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:39:51.224+0000 7f0a49e37740 1 -- 172.21.15.83:0/3409925032 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55654cecc6a0 msgr2=0x55654ced4660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:39:51.227 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:39:51.224+0000 7f0a49e37740 1 --2- 172.21.15.83:0/3409925032 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55654cecc6a0 0x55654ced4660 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x55654cec93c0 tx=0x7f0a2c00b850 comp rx=0 tx=0).stop 2024-08-19T01:39:51.227 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:39:51.224+0000 7f0a49e37740 1 -- 172.21.15.83:0/3409925032 shutdown_connections 2024-08-19T01:39:51.227 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:39:51.224+0000 7f0a49e37740 1 --2- 172.21.15.83:0/3409925032 >> [v2:172.21.15.117:6838/1967595704,v1:172.21.15.117:6839/1967595704] conn(0x55654cee7250 0x55654cee9640 crc :-1 s=READY pgs=99 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:39:51.228 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:39:51.224+0000 7f0a49e37740 1 --2- 172.21.15.83:0/3409925032 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55654ced4bf0 0x55654cee2fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:39:51.228 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:39:51.224+0000 7f0a49e37740 1 --2- 172.21.15.83:0/3409925032 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55654cecc6a0 0x55654ced4660 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:39:51.228 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:39:51.224+0000 7f0a49e37740 1 --2- 172.21.15.83:0/3409925032 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55654cecbd80 0x55654ced4120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:39:51.228 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:39:51.224+0000 7f0a49e37740 1 -- 172.21.15.83:0/3409925032 >> 172.21.15.83:0/3409925032 conn(0x55654cec8310 msgr2=0x55654cece070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:39:51.228 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:39:51.224+0000 7f0a2a000640 0 client.15747 ms_handle_reset on v2:172.21.15.117:6838/1967595704 2024-08-19T01:39:51.228 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:39:51.225+0000 7f0a49e37740 1 -- 172.21.15.83:0/3409925032 shutdown_connections 2024-08-19T01:39:51.228 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:39:51.225+0000 7f0a49e37740 1 -- 172.21.15.83:0/3409925032 wait complete. 2024-08-19T01:39:51.229 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:39:51.225+0000 7f0a49e37740 20 client.15747 trim_cache size 0 max 16384 2024-08-19T01:39:51.275 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:39:51.273+0000 7f0a49e37740 1 Processor -- start 2024-08-19T01:39:51.276 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:39:51.273+0000 7f0a49e37740 1 -- start start 2024-08-19T01:39:51.276 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:39:51.273+0000 7f0a49e37740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55654cecbd80 0x55654cee2f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:39:51.276 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:39:51.273+0000 7f0a49e37740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55654ced4bf0 0x55654ced3a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:39:51.276 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:39:51.273+0000 7f0a49e37740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55654cecfa90 0x55654ced13e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:39:51.277 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:39:51.273+0000 7f0a49e37740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f0a401a1090 con 0x55654ced4bf0 2024-08-19T01:39:51.277 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:39:51.273+0000 7f0a49e37740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f0a4019e960 con 0x55654cecfa90 2024-08-19T01:39:51.278 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:39:51.273+0000 7f0a49e37740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f0a4019c750 con 0x55654cecbd80 2024-08-19T01:39:51.278 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:39:51.273+0000 7f0a3fe00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55654cecbd80 0x55654cee2f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:39:51.278 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:39:51.273+0000 7f0a3fe00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55654cecbd80 0x55654cee2f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.83:45870/0 (socket says 172.21.15.83:45870) 2024-08-19T01:39:51.278 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:39:51.273+0000 7f0a3fe00640 1 -- 172.21.15.83:0/2702260047 learned_addr learned my addr 172.21.15.83:0/2702260047 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-08-19T01:39:51.278 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:39:51.273+0000 7f0a3fe00640 1 -- 172.21.15.83:0/2702260047 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55654cecfa90 msgr2=0x55654ced13e0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-08-19T01:39:51.278 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:39:51.273+0000 7f0a3fe00640 1 --2- 172.21.15.83:0/2702260047 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55654cecfa90 0x55654ced13e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:39:51.279 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:39:51.273+0000 7f0a3fe00640 1 -- 172.21.15.83:0/2702260047 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55654ced4bf0 msgr2=0x55654ced3a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:39:51.279 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:39:51.273+0000 7f0a3fe00640 1 --2- 172.21.15.83:0/2702260047 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55654ced4bf0 0x55654ced3a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:39:51.279 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:39:51.273+0000 7f0a3fe00640 1 -- 172.21.15.83:0/2702260047 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55654ced96c0 con 0x55654cecbd80 2024-08-19T01:39:51.280 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:39:51.275+0000 7f0a3fe00640 1 --2- 172.21.15.83:0/2702260047 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55654cecbd80 0x55654cee2f40 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f0a3009b4e0 tx=0x7f0a30000bc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6fe262a92ff109c server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:39:51.280 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:39:51.275+0000 7f0a34c00640 1 -- 172.21.15.83:0/2702260047 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a300b9070 con 0x55654cecbd80 2024-08-19T01:39:51.280 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:39:51.275+0000 7f0a34c00640 1 -- 172.21.15.83:0/2702260047 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0a3009f900 con 0x55654cecbd80 2024-08-19T01:39:51.280 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:39:51.275+0000 7f0a34c00640 1 -- 172.21.15.83:0/2702260047 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a300a3d30 con 0x55654cecbd80 2024-08-19T01:39:51.283 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:39:51.275+0000 7f0a49e37740 1 -- 172.21.15.83:0/2702260047 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55654cecbd80 msgr2=0x55654cee2f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:39:51.283 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:39:51.281+0000 7f0a49e37740 1 --2- 172.21.15.83:0/2702260047 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55654cecbd80 0x55654cee2f40 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f0a3009b4e0 tx=0x7f0a30000bc0 comp rx=0 tx=0).stop 2024-08-19T01:39:51.284 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:39:51.281+0000 7f0a49e37740 1 -- 172.21.15.83:0/2702260047 shutdown_connections 2024-08-19T01:39:51.284 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:39:51.281+0000 7f0a49e37740 1 --2- 172.21.15.83:0/2702260047 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55654cecfa90 0x55654ced13e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:39:51.284 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:39:51.281+0000 7f0a49e37740 1 --2- 172.21.15.83:0/2702260047 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55654ced4bf0 0x55654ced3a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:39:51.284 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:39:51.281+0000 7f0a49e37740 1 --2- 172.21.15.83:0/2702260047 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55654cecbd80 0x55654cee2f40 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:39:51.284 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:39:51.281+0000 7f0a49e37740 1 -- 172.21.15.83:0/2702260047 >> 172.21.15.83:0/2702260047 conn(0x55654cec8310 msgr2=0x55654cecf380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:39:51.284 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:39:51.281+0000 7f0a49e37740 1 -- 172.21.15.83:0/2702260047 shutdown_connections 2024-08-19T01:39:51.284 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:39:51.281+0000 7f0a49e37740 1 -- 172.21.15.83:0/2702260047 wait complete. 2024-08-19T01:39:51.285 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:39:51.281+0000 7f0a49e37740 1 -- start start 2024-08-19T01:39:51.285 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:39:51.281+0000 7f0a49e37740 1 Processor -- start 2024-08-19T01:39:51.285 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:39:51.281+0000 7f0a49e37740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55654cecfa90 0x55654ceec890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:39:51.286 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:39:51.281+0000 7f0a49e37740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55654ced4bf0 0x55654cee9b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:39:51.286 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:39:51.281+0000 7f0a49e37740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55654ceea0d0 0x55654ceea550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:39:51.286 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:39:51.281+0000 7f0a49e37740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f0a4019ce20 con 0x55654ceea0d0 2024-08-19T01:39:51.286 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:39:51.281+0000 7f0a49e37740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f0a4019e290 con 0x55654cecfa90 2024-08-19T01:39:51.286 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:39:51.281+0000 7f0a49e37740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f0a4019d4f0 con 0x55654ced4bf0 2024-08-19T01:39:51.287 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:39:51.281+0000 7f0a49e37740 10 client.0 _ll_register_callbacks cb 0x7f0a38034070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-19T01:39:51.290 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:39:51.281+0000 7f0a3f400640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55654ced4bf0 0x55654cee9b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:39:51.290 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:39:51.281+0000 7f0a3fe00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55654cecfa90 0x55654ceec890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:39:51.290 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:39:51.281+0000 7f0a3f400640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55654ced4bf0 0x55654cee9b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.83:45874/0 (socket says 172.21.15.83:45874) 2024-08-19T01:39:51.290 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:39:51.286+0000 7f0a3f400640 1 -- 172.21.15.83:0/4106305296 learned_addr learned my addr 172.21.15.83:0/4106305296 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-08-19T01:39:51.290 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:39:51.286+0000 7f0a3f400640 1 -- 172.21.15.83:0/4106305296 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55654cecfa90 msgr2=0x55654ceec890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:39:51.291 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:39:51.286+0000 7f0a3f400640 1 --2- 172.21.15.83:0/4106305296 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55654cecfa90 0x55654ceec890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:39:51.291 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:39:51.286+0000 7f0a3f400640 1 -- 172.21.15.83:0/4106305296 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55654ceea0d0 msgr2=0x55654ceea550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:39:51.291 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:39:51.286+0000 7f0a3f400640 1 --2- 172.21.15.83:0/4106305296 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55654ceea0d0 0x55654ceea550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:39:51.291 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:39:51.286+0000 7f0a3f400640 1 -- 172.21.15.83:0/4106305296 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55654cf61ba0 con 0x55654ced4bf0 2024-08-19T01:39:51.291 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:39:51.286+0000 7f0a2a000640 10 client.0 ms_handle_connect on v2:172.21.15.167:3300/0 2024-08-19T01:39:51.291 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:39:51.287+0000 7f0a3f400640 1 --2- 172.21.15.83:0/4106305296 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55654ced4bf0 0x55654cee9b90 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f0a2c0396f0 tx=0x7f0a2c039480 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d6bfd2ce3c5607f6 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:39:51.292 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:39:51.288+0000 7f0a2a000640 1 -- 172.21.15.83:0/4106305296 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a2c03f070 con 0x55654ced4bf0 2024-08-19T01:39:51.294 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:39:51.288+0000 7f0a2a000640 1 -- 172.21.15.83:0/4106305296 <== mon.1 v2:172.21.15.167:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8269+0+0 (secure 0 0 0) 0x7f0a2c042040 con 0x55654ced4bf0 2024-08-19T01:39:51.295 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:39:51.288+0000 7f0a49e37740 1 -- 172.21.15.83:0/4106305296 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0a3009f900 con 0x55654ced4bf0 2024-08-19T01:39:51.295 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:39:51.288+0000 7f0a2a000640 1 -- 172.21.15.83:0/4106305296 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f0a2c05f050 con 0x55654ced4bf0 2024-08-19T01:39:51.295 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:39:51.288+0000 7f0a49e37740 10 client.25729 fetch_fsmap learned FSMap version 27 2024-08-19T01:39:51.295 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:39:51.288+0000 7f0a49e37740 1 -- 172.21.15.83:0/4106305296 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55654cf62e30 con 0x55654ced4bf0 2024-08-19T01:39:51.295 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:39:51.288+0000 7f0a2a000640 1 -- 172.21.15.83:0/4106305296 <== mon.1 v2:172.21.15.167:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0a2c0035f0 con 0x55654ced4bf0 2024-08-19T01:39:51.295 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:39:51.288+0000 7f0a2a000640 1 -- 172.21.15.83:0/4106305296 <== mon.1 v2:172.21.15.167:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f0a2c033a10 con 0x55654ced4bf0 2024-08-19T01:39:51.296 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:39:51.288+0000 7f0a49e37740 10 client.25729 fetch_fsmap finished waiting for FSMap version 27 2024-08-19T01:39:51.296 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:39:51.288+0000 7f0a49e37740 10 client.25729 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-19T01:39:51.296 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:39:51.288+0000 7f0a49e37740 20 client.25729 populate_metadata read hostname 'smithi083' 2024-08-19T01:39:51.296 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:39:51.288+0000 7f0a49e37740 1 --2- 172.21.15.83:0/4106305296 >> [v2:172.21.15.117:6838/1967595704,v1:172.21.15.117:6839/1967595704] conn(0x55654cf633d0 0x55654cf657f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:39:51.296 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:39:51.288+0000 7f0a49e37740 4 client.25729 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-08-19T01:39:51.296 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:39:51.288+0000 7f0a49e37740 1 -- 172.21.15.83:0/4106305296 --> [v2:172.21.15.117:6838/1967595704,v1:172.21.15.117:6839/1967595704] -- command(tid 0: {"prefix": "scrub status"}) -- 0x7f0a3009f900 con 0x55654cf633d0 2024-08-19T01:39:51.297 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:39:51.288+0000 7f0a3fe00640 1 --2- 172.21.15.83:0/4106305296 >> [v2:172.21.15.117:6838/1967595704,v1:172.21.15.117:6839/1967595704] conn(0x55654cf633d0 0x55654cf657f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:39:51.299 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:39:51.294+0000 7f0a3fe00640 1 --2- 172.21.15.83:0/4106305296 >> [v2:172.21.15.117:6838/1967595704,v1:172.21.15.117:6839/1967595704] conn(0x55654cf633d0 0x55654cf657f0 crc :-1 s=READY pgs=100 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1d18e3de25cd1951 server_cookie=f06a944f08b71494 in_seq=0 out_seq=0 2024-08-19T01:39:51.299 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:39:51.294+0000 7f0a2a000640 10 client.25729 ms_handle_connect on v2:172.21.15.117:6838/1967595704 2024-08-19T01:39:51.302 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:39:51.300+0000 7f0a2a000640 1 -- 172.21.15.83:0/4106305296 <== mds.0 v2:172.21.15.117:6838/1967595704 1 ==== command_reply(tid 0: 0 ) ==== 8+0+268 (crc 0 0 0) 0x7f0a3009f900 con 0x55654cf633d0 2024-08-19T01:39:51.304 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:39:51.300+0000 7f0a2a000640 10 client.25729 handle_command_reply: tid=0 2024-08-19T01:39:51.305 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:39:51.300+0000 7f0a49e37740 1 client.25729 shutdown 2024-08-19T01:39:51.305 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:39:51.300+0000 7f0a49e37740 1 -- 172.21.15.83:0/4106305296 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55654ced4bf0 msgr2=0x55654cee9b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:39:51.305 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:39:51.300+0000 7f0a49e37740 1 --2- 172.21.15.83:0/4106305296 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55654ced4bf0 0x55654cee9b90 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f0a2c0396f0 tx=0x7f0a2c039480 comp rx=0 tx=0).stop 2024-08-19T01:39:51.305 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:39:51.300+0000 7f0a49e37740 1 -- 172.21.15.83:0/4106305296 shutdown_connections 2024-08-19T01:39:51.306 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:39:51.300+0000 7f0a49e37740 1 --2- 172.21.15.83:0/4106305296 >> [v2:172.21.15.117:6838/1967595704,v1:172.21.15.117:6839/1967595704] conn(0x55654cf633d0 0x55654cf657f0 crc :-1 s=READY pgs=100 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:39:51.307 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:39:51.300+0000 7f0a49e37740 1 --2- 172.21.15.83:0/4106305296 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55654ceea0d0 0x55654ceea550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:39:51.307 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:39:51.300+0000 7f0a49e37740 1 --2- 172.21.15.83:0/4106305296 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55654ced4bf0 0x55654cee9b90 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:39:51.307 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:39:51.300+0000 7f0a49e37740 1 --2- 172.21.15.83:0/4106305296 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55654cecfa90 0x55654ceec890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:39:51.307 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:39:51.300+0000 7f0a49e37740 1 -- 172.21.15.83:0/4106305296 >> 172.21.15.83:0/4106305296 conn(0x55654cec8310 msgr2=0x55654cec6bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:39:51.307 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:39:51.300+0000 7f0a2a000640 0 client.25729 ms_handle_reset on v2:172.21.15.117:6838/1967595704 2024-08-19T01:39:51.307 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:39:51.300+0000 7f0a49e37740 1 -- 172.21.15.83:0/4106305296 shutdown_connections 2024-08-19T01:39:51.308 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:39:51.300+0000 7f0a49e37740 1 -- 172.21.15.83:0/4106305296 wait complete. 2024-08-19T01:39:51.308 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:39:51.300+0000 7f0a49e37740 20 client.25729 trim_cache size 0 max 16384 2024-08-19T01:39:51.308 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-08-19T01:39:51.308 INFO:teuthology.orchestra.run.smithi083.stdout: "status": "scrub active (3278 inodes in the stack)", 2024-08-19T01:39:51.309 INFO:teuthology.orchestra.run.smithi083.stdout: "scrubs": { 2024-08-19T01:39:51.309 INFO:teuthology.orchestra.run.smithi083.stdout: "0e436353-7799-40e1-97b3-a89eaf3bf27d": { 2024-08-19T01:39:51.309 INFO:teuthology.orchestra.run.smithi083.stdout: "path": "/", 2024-08-19T01:39:51.309 INFO:teuthology.orchestra.run.smithi083.stdout: "tag": "0e436353-7799-40e1-97b3-a89eaf3bf27d", 2024-08-19T01:39:51.309 INFO:teuthology.orchestra.run.smithi083.stdout: "options": "recursive,force" 2024-08-19T01:39:51.309 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-08-19T01:39:51.309 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-08-19T01:39:51.310 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-08-19T01:39:51.310 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:39:51.300+0000 7f0a45c00640 1 -- 172.21.15.83:0/1022357715 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f0a240a2cd0 msgr2=0x7f0a240a5190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:39:51.311 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:39:51.300+0000 7f0a45c00640 1 --2- 172.21.15.83:0/1022357715 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f0a240a2cd0 0x7f0a240a5190 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f0a380099a0 tx=0x7f0a3800b040 comp rx=0 tx=0).stop 2024-08-19T01:39:51.311 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:39:51.300+0000 7f0a45c00640 1 -- 172.21.15.83:0/1022357715 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0a40102920 msgr2=0x7f0a401004b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:39:51.311 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:39:51.300+0000 7f0a45c00640 1 --2- 172.21.15.83:0/1022357715 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0a40102920 0x7f0a401004b0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f0a3000d950 tx=0x7f0a3000de10 comp rx=0 tx=0).stop 2024-08-19T01:39:51.311 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:39:51.300+0000 7f0a45c00640 1 -- 172.21.15.83:0/1022357715 shutdown_connections 2024-08-19T01:39:51.311 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:39:51.300+0000 7f0a45c00640 1 --2- 172.21.15.83:0/1022357715 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f0a240a2cd0 0x7f0a240a5190 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:39:51.311 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:39:51.300+0000 7f0a45c00640 1 --2- 172.21.15.83:0/1022357715 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0a4010bd00 0x7f0a40100f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:39:51.312 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:39:51.300+0000 7f0a45c00640 1 --2- 172.21.15.83:0/1022357715 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0a40103240 0x7f0a401009f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:39:51.312 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:39:51.300+0000 7f0a45c00640 1 --2- 172.21.15.83:0/1022357715 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0a40102920 0x7f0a401004b0 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:39:51.312 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:39:51.300+0000 7f0a45c00640 1 -- 172.21.15.83:0/1022357715 >> 172.21.15.83:0/1022357715 conn(0x7f0a400fc340 msgr2=0x7f0a400fd020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:39:51.312 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:39:51.306+0000 7f0a45c00640 1 -- 172.21.15.83:0/1022357715 shutdown_connections 2024-08-19T01:39:51.312 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:39:51.306+0000 7f0a45c00640 1 -- 172.21.15.83:0/1022357715 wait complete. 2024-08-19T01:39:51.696 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:39:51 smithi083 ceph-mon[17120]: pgmap v489: 97 pgs: 97 active+clean; 512 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 8.6 MiB/s wr, 860 op/s 2024-08-19T01:39:51.952 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:39:51 smithi117 ceph-mon[22016]: pgmap v489: 97 pgs: 97 active+clean; 512 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 8.6 MiB/s wr, 860 op/s 2024-08-19T01:39:51.965 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:39:51 smithi167 ceph-mon[21875]: pgmap v489: 97 pgs: 97 active+clean; 512 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 8.6 MiB/s wr, 860 op/s 2024-08-19T01:39:52.662 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:39:52.660+0000 7f7853e00640 1 -- 172.21.15.83:0/3297194087 <== mds.0 v2:172.21.15.117:6838/1967595704 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x557d0ed1d310 con 0x557d0ee32450 2024-08-19T01:39:52.662 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:39:52.660+0000 7f7853e00640 10 client.25711 handle_command_reply: tid=0 2024-08-19T01:39:52.663 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:39:52.660+0000 7f787c014740 1 client.25711 shutdown 2024-08-19T01:39:52.663 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:39:52.660+0000 7f787c014740 1 -- 172.21.15.83:0/3297194087 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557d0edb7030 msgr2=0x557d0ee304f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:39:52.663 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:39:52.660+0000 7f787c014740 1 --2- 172.21.15.83:0/3297194087 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557d0edb7030 0x557d0ee304f0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f786c004750 tx=0x7f786c035b50 comp rx=0 tx=0).stop 2024-08-19T01:39:52.663 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:39:52.660+0000 7f787c014740 1 -- 172.21.15.83:0/3297194087 shutdown_connections 2024-08-19T01:39:52.663 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:39:52.660+0000 7f787c014740 1 --2- 172.21.15.83:0/3297194087 >> [v2:172.21.15.117:6838/1967595704,v1:172.21.15.117:6839/1967595704] conn(0x557d0ee32450 0x557d0ee34840 crc :-1 s=READY pgs=98 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:39:52.663 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:39:52.660+0000 7f787c014740 1 --2- 172.21.15.83:0/3297194087 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557d0edb7030 0x557d0ee304f0 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:39:52.663 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:39:52.660+0000 7f787c014740 1 --2- 172.21.15.83:0/3297194087 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557d0edb1fa0 0x557d0edc0450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:39:52.663 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:39:52.660+0000 7f7853e00640 0 client.25711 ms_handle_reset on v2:172.21.15.117:6838/1967595704 2024-08-19T01:39:52.663 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:39:52.660+0000 7f787c014740 1 --2- 172.21.15.83:0/3297194087 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557d0edb1660 0x557d0edbff10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:39:52.663 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:39:52.660+0000 7f787c014740 1 -- 172.21.15.83:0/3297194087 >> 172.21.15.83:0/3297194087 conn(0x557d0edaccd0 msgr2=0x557d0edab720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:39:52.663 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:39:52.660+0000 7f787c014740 1 -- 172.21.15.83:0/3297194087 shutdown_connections 2024-08-19T01:39:52.663 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:39:52.660+0000 7f787c014740 1 -- 172.21.15.83:0/3297194087 wait complete. 2024-08-19T01:39:52.664 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:39:52.660+0000 7f787c014740 20 client.25711 trim_cache size 0 max 16384 2024-08-19T01:39:52.664 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stdout:{ 2024-08-19T01:39:52.664 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stdout: "message": "", 2024-08-19T01:39:52.664 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stdout: "return_code": 0 2024-08-19T01:39:52.664 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stdout:} 2024-08-19T01:39:52.665 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:39:52.660+0000 7f7873200640 1 -- 172.21.15.83:0/1700971206 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f78580a2d90 msgr2=0x7f78580a5250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:39:52.665 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:39:52.660+0000 7f7873200640 1 --2- 172.21.15.83:0/1700971206 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f78580a2d90 0x7f78580a5250 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f7864006fd0 tx=0x7f7864008040 comp rx=0 tx=0).stop 2024-08-19T01:39:52.665 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:39:52.660+0000 7f7873200640 1 -- 172.21.15.83:0/1700971206 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f78740733b0 msgr2=0x7f7874115c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:39:52.665 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:39:52.660+0000 7f7873200640 1 --2- 172.21.15.83:0/1700971206 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f78740733b0 0x7f7874115c20 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f7860009850 tx=0x7f7860032dc0 comp rx=0 tx=0).stop 2024-08-19T01:39:52.665 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:39:52.660+0000 7f7873200640 1 -- 172.21.15.83:0/1700971206 shutdown_connections 2024-08-19T01:39:52.665 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:39:52.660+0000 7f7873200640 1 --2- 172.21.15.83:0/1700971206 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f78580a2d90 0x7f78580a5250 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:39:52.665 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:39:52.660+0000 7f7873200640 1 --2- 172.21.15.83:0/1700971206 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f787410d8e0 0x7f78741b6980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:39:52.665 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:39:52.660+0000 7f7873200640 1 --2- 172.21.15.83:0/1700971206 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f78740733b0 0x7f7874115c20 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:39:52.665 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:39:52.660+0000 7f7873200640 1 --2- 172.21.15.83:0/1700971206 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f78740729e0 0x7f78741156e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:39:52.666 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:39:52.660+0000 7f7873200640 1 -- 172.21.15.83:0/1700971206 >> 172.21.15.83:0/1700971206 conn(0x7f787406e880 msgr2=0x7f7874070660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:39:52.666 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:39:52.660+0000 7f7873200640 1 -- 172.21.15.83:0/1700971206 shutdown_connections 2024-08-19T01:39:52.666 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:39:52.660+0000 7f7873200640 1 -- 172.21.15.83:0/1700971206 wait complete. 2024-08-19T01:39:53.013 INFO:tasks.cephfs.filesystem:scrub status for tag:0e436353-7799-40e1-97b3-a89eaf3bf27d - {'path': '/', 'tag': '0e436353-7799-40e1-97b3-a89eaf3bf27d', 'options': 'recursive,force'} 2024-08-19T01:39:53.656 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:39:53 smithi083 ceph-mon[17120]: pgmap v490: 97 pgs: 97 active+clean; 517 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 9.1 MiB/s wr, 966 op/s 2024-08-19T01:39:53.952 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:39:53 smithi117 ceph-mon[22016]: pgmap v490: 97 pgs: 97 active+clean; 517 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 9.1 MiB/s wr, 966 op/s 2024-08-19T01:39:53.965 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:39:53 smithi167 ceph-mon[21875]: pgmap v490: 97 pgs: 97 active+clean; 517 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 9.1 MiB/s wr, 966 op/s 2024-08-19T01:39:54.952 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:39:54 smithi117 ceph-mon[22016]: scrub summary: idle+waiting paths [/] 2024-08-19T01:39:54.963 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:39:54 smithi083 ceph-mon[17120]: scrub summary: idle+waiting paths [/] 2024-08-19T01:39:54.965 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:39:54 smithi167 ceph-mon[21875]: scrub summary: idle+waiting paths [/] 2024-08-19T01:39:55.904 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:39:55 smithi083 ceph-mon[17120]: pgmap v491: 97 pgs: 97 active+clean; 519 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 10 MiB/s wr, 2.10k op/s 2024-08-19T01:39:55.904 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:39:55 smithi083 ceph-mon[17120]: scrub summary: idle 2024-08-19T01:39:55.952 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:39:55 smithi117 ceph-mon[22016]: pgmap v491: 97 pgs: 97 active+clean; 519 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 10 MiB/s wr, 2.10k op/s 2024-08-19T01:39:55.952 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:39:55 smithi117 ceph-mon[22016]: scrub summary: idle 2024-08-19T01:39:55.965 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:39:55 smithi167 ceph-mon[21875]: pgmap v491: 97 pgs: 97 active+clean; 519 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 10 MiB/s wr, 2.10k op/s 2024-08-19T01:39:55.965 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:39:55 smithi167 ceph-mon[21875]: scrub summary: idle 2024-08-19T01:39:56.952 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:39:56 smithi117 ceph-mon[22016]: scrub summary: idle 2024-08-19T01:39:56.952 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:39:56 smithi117 ceph-mon[22016]: scrub summary: idle 2024-08-19T01:39:56.965 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:39:56 smithi167 ceph-mon[21875]: scrub summary: idle 2024-08-19T01:39:56.965 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:39:56 smithi167 ceph-mon[21875]: scrub summary: idle 2024-08-19T01:39:57.164 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:39:56 smithi083 ceph-mon[17120]: scrub summary: idle 2024-08-19T01:39:57.164 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:39:56 smithi083 ceph-mon[17120]: scrub summary: idle 2024-08-19T01:39:57.952 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:39:57 smithi117 ceph-mon[22016]: pgmap v492: 97 pgs: 97 active+clean; 519 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 9.7 MiB/s wr, 2.12k op/s 2024-08-19T01:39:57.965 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:39:57 smithi167 ceph-mon[21875]: pgmap v492: 97 pgs: 97 active+clean; 519 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 9.7 MiB/s wr, 2.12k op/s 2024-08-19T01:39:58.155 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:39:57 smithi083 ceph-mon[17120]: pgmap v492: 97 pgs: 97 active+clean; 519 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 9.7 MiB/s wr, 2.12k op/s 2024-08-19T01:39:59.952 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:39:59 smithi117 ceph-mon[22016]: pgmap v493: 97 pgs: 97 active+clean; 492 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 9.6 MiB/s wr, 2.35k op/s 2024-08-19T01:39:59.965 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:39:59 smithi167 ceph-mon[21875]: pgmap v493: 97 pgs: 97 active+clean; 492 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 9.6 MiB/s wr, 2.35k op/s 2024-08-19T01:40:00.158 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:39:59 smithi083 ceph-mon[17120]: pgmap v493: 97 pgs: 97 active+clean; 492 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 9.6 MiB/s wr, 2.35k op/s 2024-08-19T01:40:00.643 INFO:tasks.workunit.client.0.smithi083.stdout: 166 492031 452 319639 455 280975 1167 2024-08-19T01:40:00.952 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:40:00 smithi117 ceph-mon[22016]: overall HEALTH_OK 2024-08-19T01:40:00.966 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:40:00 smithi167 ceph-mon[21875]: overall HEALTH_OK 2024-08-19T01:40:01.156 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:40:00 smithi083 ceph-mon[17120]: overall HEALTH_OK 2024-08-19T01:40:01.907 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:40:01 smithi083 ceph-mon[17120]: pgmap v494: 97 pgs: 97 active+clean; 498 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 5.8 MiB/s wr, 2.05k op/s 2024-08-19T01:40:01.952 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:40:01 smithi117 ceph-mon[22016]: pgmap v494: 97 pgs: 97 active+clean; 498 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 5.8 MiB/s wr, 2.05k op/s 2024-08-19T01:40:01.965 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:40:01 smithi167 ceph-mon[21875]: pgmap v494: 97 pgs: 97 active+clean; 498 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 5.8 MiB/s wr, 2.05k op/s 2024-08-19T01:40:02.952 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:40:02 smithi117 ceph-mon[22016]: pgmap v495: 97 pgs: 97 active+clean; 498 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 3.8 MiB/s wr, 1.89k op/s 2024-08-19T01:40:02.965 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:40:02 smithi167 ceph-mon[21875]: pgmap v495: 97 pgs: 97 active+clean; 498 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 3.8 MiB/s wr, 1.89k op/s 2024-08-19T01:40:03.133 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:40:02 smithi083 ceph-mon[17120]: pgmap v495: 97 pgs: 97 active+clean; 498 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 3.8 MiB/s wr, 1.89k op/s 2024-08-19T01:40:05.905 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:40:05 smithi083 ceph-mon[17120]: pgmap v496: 97 pgs: 97 active+clean; 498 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 3.6 MiB/s wr, 1.77k op/s 2024-08-19T01:40:05.952 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:40:05 smithi117 ceph-mon[22016]: pgmap v496: 97 pgs: 97 active+clean; 498 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 3.6 MiB/s wr, 1.77k op/s 2024-08-19T01:40:05.965 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:40:05 smithi167 ceph-mon[21875]: pgmap v496: 97 pgs: 97 active+clean; 498 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 3.6 MiB/s wr, 1.77k op/s 2024-08-19T01:40:07.910 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:40:07 smithi083 ceph-mon[17120]: pgmap v497: 97 pgs: 97 active+clean; 499 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.8 MiB/s wr, 574 op/s 2024-08-19T01:40:07.952 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:40:07 smithi117 ceph-mon[22016]: pgmap v497: 97 pgs: 97 active+clean; 499 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.8 MiB/s wr, 574 op/s 2024-08-19T01:40:07.965 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:40:07 smithi167 ceph-mon[21875]: pgmap v497: 97 pgs: 97 active+clean; 499 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.8 MiB/s wr, 574 op/s 2024-08-19T01:40:08.174 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:40:08.172+0000 7fdfb1c00640 1 -- 172.21.15.83:0/1556116966 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdfac0729e0 msgr2=0x7fdfac072de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:40:08.174 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:40:08.172+0000 7fdfb1c00640 1 --2- 172.21.15.83:0/1556116966 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdfac0729e0 0x7fdfac072de0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fdf9c009890 tx=0x7fdf9c02a3b0 comp rx=0 tx=0).stop 2024-08-19T01:40:08.228 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:40:08.226+0000 7fdfb1c00640 1 -- 172.21.15.83:0/1556116966 shutdown_connections 2024-08-19T01:40:08.228 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:40:08.226+0000 7fdfb1c00640 1 --2- 172.21.15.83:0/1556116966 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdfac10d8e0 0x7fdfac10fda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:40:08.228 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:40:08.226+0000 7fdfb1c00640 1 --2- 172.21.15.83:0/1556116966 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdfac0733b0 0x7fdfac10d220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:40:08.228 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:40:08.226+0000 7fdfb1c00640 1 --2- 172.21.15.83:0/1556116966 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdfac0729e0 0x7fdfac072de0 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:40:08.228 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:40:08.226+0000 7fdfb1c00640 1 -- 172.21.15.83:0/1556116966 >> 172.21.15.83:0/1556116966 conn(0x7fdfac06e880 msgr2=0x7fdfac070cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:40:08.269 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:40:08.255+0000 7fdfb1c00640 1 -- 172.21.15.83:0/1556116966 shutdown_connections 2024-08-19T01:40:08.269 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:40:08.259+0000 7fdfb1c00640 1 -- 172.21.15.83:0/1556116966 wait complete. 2024-08-19T01:40:08.269 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:40:08.260+0000 7fdfb1c00640 1 Processor -- start 2024-08-19T01:40:08.269 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:40:08.267+0000 7fdfb1c00640 1 -- start start 2024-08-19T01:40:08.269 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:40:08.267+0000 7fdfb1c00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdfac0729e0 0x7fdfac19e420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:40:08.269 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:40:08.267+0000 7fdfb1c00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdfac0733b0 0x7fdfac19e960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:40:08.283 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:40:08.278+0000 7fdfb1c00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdfac10d8e0 0x7fdfac07a180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:40:08.283 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:40:08.278+0000 7fdfb1c00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fdfac1132d0 con 0x7fdfac0733b0 2024-08-19T01:40:08.283 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:40:08.278+0000 7fdfb1c00640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fdfac113150 con 0x7fdfac10d8e0 2024-08-19T01:40:08.283 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:40:08.278+0000 7fdfb1c00640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fdfac113450 con 0x7fdfac0729e0 2024-08-19T01:40:08.284 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:40:08.282+0000 7fdfb1200640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdfac10d8e0 0x7fdfac07a180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:40:08.284 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:40:08.282+0000 7fdfb1200640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdfac10d8e0 0x7fdfac07a180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.83:59040/0 (socket says 172.21.15.83:59040) 2024-08-19T01:40:08.284 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:40:08.282+0000 7fdfb1200640 1 -- 172.21.15.83:0/722962834 learned_addr learned my addr 172.21.15.83:0/722962834 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-08-19T01:40:08.284 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:40:08.282+0000 7fdfab400640 1 --2- 172.21.15.83:0/722962834 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdfac0733b0 0x7fdfac19e960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:40:08.290 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:40:08.285+0000 7fdfb1200640 1 -- 172.21.15.83:0/722962834 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdfac0729e0 msgr2=0x7fdfac19e420 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-08-19T01:40:08.293 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:40:08.285+0000 7fdfb1200640 1 --2- 172.21.15.83:0/722962834 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdfac0729e0 0x7fdfac19e420 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:40:08.293 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:40:08.285+0000 7fdfb1200640 1 -- 172.21.15.83:0/722962834 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdfac0733b0 msgr2=0x7fdfac19e960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:40:08.293 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:40:08.285+0000 7fdfb1200640 1 --2- 172.21.15.83:0/722962834 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdfac0733b0 0x7fdfac19e960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:40:08.293 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:40:08.285+0000 7fdfb1200640 1 -- 172.21.15.83:0/722962834 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdfac07a820 con 0x7fdfac10d8e0 2024-08-19T01:40:08.296 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:40:08.293+0000 7fdfb1200640 1 --2- 172.21.15.83:0/722962834 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdfac10d8e0 0x7fdfac07a180 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fdfa4009fd0 tx=0x7fdfa400b260 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:40:08.302 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:40:08.297+0000 7fdfa8c00640 1 -- 172.21.15.83:0/722962834 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfa4017070 con 0x7fdfac10d8e0 2024-08-19T01:40:08.302 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:40:08.297+0000 7fdfa8c00640 1 -- 172.21.15.83:0/722962834 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdfa4003e50 con 0x7fdfac10d8e0 2024-08-19T01:40:08.302 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:40:08.297+0000 7fdfa8c00640 1 -- 172.21.15.83:0/722962834 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfa40106c0 con 0x7fdfac10d8e0 2024-08-19T01:40:08.329 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:40:08.321+0000 7fdfb1c00640 1 -- 172.21.15.83:0/722962834 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdfac07ab10 con 0x7fdfac10d8e0 2024-08-19T01:40:08.329 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:40:08.321+0000 7fdfb1c00640 1 -- 172.21.15.83:0/722962834 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdfac07afe0 con 0x7fdfac10d8e0 2024-08-19T01:40:08.377 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:40:08.375+0000 7fdfb6224740 1 Processor -- start 2024-08-19T01:40:08.445 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:40:08.416+0000 7fdfa8c00640 1 -- 172.21.15.83:0/722962834 <== mon.2 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7fdfa4003970 con 0x7fdfac10d8e0 2024-08-19T01:40:08.445 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:40:08.419+0000 7fdfa8c00640 1 --2- 172.21.15.83:0/722962834 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7fdf900a2d90 0x7fdf900a5250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:40:08.446 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:40:08.419+0000 7fdfa8c00640 1 -- 172.21.15.83:0/722962834 <== mon.2 v2:172.21.15.117:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8269+0+0 (secure 0 0 0) 0x7fdfa40ced80 con 0x7fdfac10d8e0 2024-08-19T01:40:08.446 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:40:08.429+0000 7fdfab400640 1 --2- 172.21.15.83:0/722962834 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7fdf900a2d90 0x7fdf900a5250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:40:08.446 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:40:08.431+0000 7fdfab400640 1 --2- 172.21.15.83:0/722962834 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7fdf900a2d90 0x7fdf900a5250 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fdfac19f550 tx=0x7fdf9801a270 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:40:08.446 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:40:08.433+0000 7fdfb6224740 1 -- start start 2024-08-19T01:40:08.446 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:40:08.433+0000 7fdfb6224740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557061c49680 0x557061c49a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:40:08.446 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:40:08.434+0000 7fdfb6224740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557061c49fc0 0x557061c4ea80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:40:08.446 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:40:08.434+0000 7fdfb6224740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557061c4f050 0x557061c51510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:40:08.446 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:40:08.434+0000 7fdfb6224740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x557061bad890 con 0x557061c4f050 2024-08-19T01:40:08.447 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:40:08.434+0000 7fdfb6224740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x557061b45d90 con 0x557061c49fc0 2024-08-19T01:40:08.447 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:40:08.434+0000 7fdfb6224740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x557061ba9850 con 0x557061c49680 2024-08-19T01:40:08.447 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:40:08.437+0000 7fdfab400640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557061c49fc0 0x557061c4ea80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:40:08.447 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:40:08.437+0000 7fdfab400640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557061c49fc0 0x557061c4ea80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.83:59044/0 (socket says 172.21.15.83:59044) 2024-08-19T01:40:08.447 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:40:08.437+0000 7fdfab400640 1 -- 172.21.15.83:0/1787487376 learned_addr learned my addr 172.21.15.83:0/1787487376 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-08-19T01:40:08.447 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:40:08.437+0000 7fdfab400640 1 -- 172.21.15.83:0/1787487376 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557061c49680 msgr2=0x557061c49a80 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-08-19T01:40:08.447 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:40:08.437+0000 7fdfab400640 1 --2- 172.21.15.83:0/1787487376 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557061c49680 0x557061c49a80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:40:08.447 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:40:08.437+0000 7fdfab400640 1 -- 172.21.15.83:0/1787487376 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557061c4f050 msgr2=0x557061c51510 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-08-19T01:40:08.447 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:40:08.437+0000 7fdfab400640 1 --2- 172.21.15.83:0/1787487376 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557061c4f050 0x557061c51510 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:40:08.447 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:40:08.437+0000 7fdfab400640 1 -- 172.21.15.83:0/1787487376 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5570618c1210 con 0x557061c49fc0 2024-08-19T01:40:08.447 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:40:08.438+0000 7fdfab400640 1 --2- 172.21.15.83:0/1787487376 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557061c49fc0 0x557061c4ea80 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fdf9802d750 tx=0x7fdf98025000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=344ba383528a4774 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:40:08.447 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:40:08.440+0000 7fdfa1600640 1 -- 172.21.15.83:0/1787487376 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf98020670 con 0x557061c49fc0 2024-08-19T01:40:08.447 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:40:08.440+0000 7fdfa1600640 1 -- 172.21.15.83:0/1787487376 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdf98020cf0 con 0x557061c49fc0 2024-08-19T01:40:08.448 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:40:08.440+0000 7fdfa1600640 1 -- 172.21.15.83:0/1787487376 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf9802bdf0 con 0x557061c49fc0 2024-08-19T01:40:08.448 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:40:08.441+0000 7fdfb6224740 1 -- 172.21.15.83:0/1787487376 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557061c49fc0 msgr2=0x557061c4ea80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:40:08.448 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:40:08.441+0000 7fdfb6224740 1 --2- 172.21.15.83:0/1787487376 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557061c49fc0 0x557061c4ea80 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fdf9802d750 tx=0x7fdf98025000 comp rx=0 tx=0).stop 2024-08-19T01:40:08.448 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:40:08.441+0000 7fdfb6224740 1 -- 172.21.15.83:0/1787487376 shutdown_connections 2024-08-19T01:40:08.448 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:40:08.441+0000 7fdfb6224740 1 --2- 172.21.15.83:0/1787487376 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557061c4f050 0x557061c51510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:40:08.448 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:40:08.441+0000 7fdfb6224740 1 --2- 172.21.15.83:0/1787487376 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557061c49fc0 0x557061c4ea80 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:40:08.448 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:40:08.441+0000 7fdfb6224740 1 --2- 172.21.15.83:0/1787487376 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557061c49680 0x557061c49a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:40:08.448 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:40:08.441+0000 7fdfb6224740 1 -- 172.21.15.83:0/1787487376 >> 172.21.15.83:0/1787487376 conn(0x557061c44cf0 msgr2=0x557061c450f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:40:08.468 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:40:08.467+0000 7fdfb6224740 1 -- 172.21.15.83:0/1787487376 shutdown_connections 2024-08-19T01:40:08.469 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:40:08.467+0000 7fdfb6224740 1 -- 172.21.15.83:0/1787487376 wait complete. 2024-08-19T01:40:08.470 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:40:08.468+0000 7fdfb6224740 1 -- start start 2024-08-19T01:40:08.470 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:40:08.468+0000 7fdfb6224740 1 Processor -- start 2024-08-19T01:40:08.470 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:40:08.469+0000 7fdfb6224740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557061c49fc0 0x557061c53a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:40:08.474 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:40:08.472+0000 7fdfb6224740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557061c4f050 0x557061c53fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:40:08.475 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:40:08.472+0000 7fdfab400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557061c4f050 0x557061c53fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:40:08.475 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:40:08.473+0000 7fdfab400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557061c4f050 0x557061c53fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:48856/0 (socket says 172.21.15.83:48856) 2024-08-19T01:40:08.475 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:40:08.473+0000 7fdfab400640 1 -- 172.21.15.83:0/1636180824 learned_addr learned my addr 172.21.15.83:0/1636180824 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-08-19T01:40:08.475 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:40:08.473+0000 7fdfb6224740 1 --2- 172.21.15.83:0/1636180824 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557061c59000 0x557061c5b3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:40:08.475 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:40:08.473+0000 7fdfb6224740 1 -- 172.21.15.83:0/1636180824 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x557061c520c0 con 0x557061c4f050 2024-08-19T01:40:08.475 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:40:08.473+0000 7fdfb6224740 1 -- 172.21.15.83:0/1636180824 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x557061b68f70 con 0x557061c49fc0 2024-08-19T01:40:08.475 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:40:08.473+0000 7fdfb6224740 1 -- 172.21.15.83:0/1636180824 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x557061c52240 con 0x557061c59000 2024-08-19T01:40:08.475 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:40:08.473+0000 7fdfb1200640 1 --2- 172.21.15.83:0/1636180824 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557061c59000 0x557061c5b3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:40:08.476 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:40:08.474+0000 7fdfb6224740 10 client.0 _ll_register_callbacks cb 0x557061ba58d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-19T01:40:08.476 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:40:08.474+0000 7fdfb1200640 1 -- 172.21.15.83:0/1636180824 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557061c49fc0 msgr2=0x557061c53a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:40:08.476 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:40:08.474+0000 7fdfb1200640 1 --2- 172.21.15.83:0/1636180824 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557061c49fc0 0x557061c53a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:40:08.476 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:40:08.474+0000 7fdfb1200640 1 -- 172.21.15.83:0/1636180824 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557061c4f050 msgr2=0x557061c53fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:40:08.476 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:40:08.474+0000 7fdfb1200640 1 --2- 172.21.15.83:0/1636180824 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557061c4f050 0x557061c53fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:40:08.477 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:40:08.474+0000 7fdfb1200640 1 -- 172.21.15.83:0/1636180824 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55706198d620 con 0x557061c59000 2024-08-19T01:40:08.477 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:40:08.474+0000 7fdfb6224740 1 -- 172.21.15.83:0/1636180824 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdf98020cf0 con 0x557061c59000 2024-08-19T01:40:08.477 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:40:08.474+0000 7fdfb1200640 1 --2- 172.21.15.83:0/1636180824 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557061c59000 0x557061c5b3f0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fdfa400b760 tx=0x7fdfa409b470 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4763e3cdc07351ba server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:40:08.477 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:40:08.474+0000 7fdf96000640 10 client.25735 ms_handle_connect on v2:172.21.15.167:3300/0 2024-08-19T01:40:08.477 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:40:08.474+0000 7fdf96000640 1 -- 172.21.15.83:0/1636180824 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfa409b5e0 con 0x557061c59000 2024-08-19T01:40:08.478 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:40:08.475+0000 7fdf96000640 1 -- 172.21.15.83:0/1636180824 <== mon.1 v2:172.21.15.167:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8269+0+0 (secure 0 0 0) 0x7fdfa4097c50 con 0x557061c59000 2024-08-19T01:40:08.478 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:40:08.475+0000 7fdf96000640 1 -- 172.21.15.83:0/1636180824 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fdfa409f990 con 0x557061c59000 2024-08-19T01:40:08.479 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:40:08.475+0000 7fdfab400640 1 --2- 172.21.15.83:0/1636180824 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557061c4f050 0x557061c53fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-19T01:40:08.479 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:40:08.475+0000 7fdfb6224740 10 client.25735 fetch_fsmap learned FSMap version 27 2024-08-19T01:40:08.479 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:40:08.475+0000 7fdfb6224740 1 -- 172.21.15.83:0/1636180824 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({fsmap=27}) -- 0x557061c5e500 con 0x557061c59000 2024-08-19T01:40:08.479 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:40:08.476+0000 7fdf96000640 1 -- 172.21.15.83:0/1636180824 <== mon.1 v2:172.21.15.167:3300/0 4 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7fdfa409aa30 con 0x557061c59000 2024-08-19T01:40:08.479 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:40:08.477+0000 7fdfb6224740 10 client.25735 fetch_fsmap finished waiting for FSMap version 27 2024-08-19T01:40:08.479 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:40:08.477+0000 7fdfb6224740 10 client.25735 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-08-19T01:40:08.479 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:40:08.477+0000 7fdfb6224740 20 client.25735 populate_metadata read hostname 'smithi083' 2024-08-19T01:40:08.479 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:40:08.477+0000 7fdfb6224740 1 --2- 172.21.15.83:0/1636180824 >> [v2:172.21.15.117:6838/1967595704,v1:172.21.15.117:6839/1967595704] conn(0x557061c5fb30 0x557061c61fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:40:08.479 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:40:08.477+0000 7fdfb6224740 4 client.25735 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-19T01:40:08.479 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:40:08.477+0000 7fdfb6224740 1 -- 172.21.15.83:0/1636180824 --> [v2:172.21.15.117:6838/1967595704,v1:172.21.15.117:6839/1967595704] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557061bbafc0 con 0x557061c5fb30 2024-08-19T01:40:08.486 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:40:08.485+0000 7fdfabe00640 1 --2- 172.21.15.83:0/1636180824 >> [v2:172.21.15.117:6838/1967595704,v1:172.21.15.117:6839/1967595704] conn(0x557061c5fb30 0x557061c61fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:40:08.487 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:40:08.486+0000 7fdfabe00640 1 --2- 172.21.15.83:0/1636180824 >> [v2:172.21.15.117:6838/1967595704,v1:172.21.15.117:6839/1967595704] conn(0x557061c5fb30 0x557061c61fc0 crc :-1 s=READY pgs=101 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=814c3affe23c1c86 server_cookie=5ca576217348afd0 in_seq=0 out_seq=0 2024-08-19T01:40:08.487 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:40:08.486+0000 7fdf96000640 10 client.25735 ms_handle_connect on v2:172.21.15.117:6838/1967595704 2024-08-19T01:40:08.488 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:40:08.487+0000 7fdf96000640 1 -- 172.21.15.83:0/1636180824 <== mds.0 v2:172.21.15.117:6838/1967595704 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x557061bbafc0 con 0x557061c5fb30 2024-08-19T01:40:08.488 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:40:08.487+0000 7fdf96000640 10 client.25735 handle_command_reply: tid=0 2024-08-19T01:40:08.489 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:40:08.487+0000 7fdfb6224740 1 client.25735 shutdown 2024-08-19T01:40:08.489 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:40:08.488+0000 7fdfb6224740 1 -- 172.21.15.83:0/1636180824 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557061c59000 msgr2=0x557061c5b3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:40:08.489 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:40:08.488+0000 7fdfb6224740 1 --2- 172.21.15.83:0/1636180824 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557061c59000 0x557061c5b3f0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fdfa400b760 tx=0x7fdfa409b470 comp rx=0 tx=0).stop 2024-08-19T01:40:08.489 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:40:08.488+0000 7fdfb6224740 1 -- 172.21.15.83:0/1636180824 shutdown_connections 2024-08-19T01:40:08.491 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:40:08.488+0000 7fdfb6224740 1 --2- 172.21.15.83:0/1636180824 >> [v2:172.21.15.117:6838/1967595704,v1:172.21.15.117:6839/1967595704] conn(0x557061c5fb30 0x557061c61fc0 crc :-1 s=READY pgs=101 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:40:08.491 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:40:08.488+0000 7fdfb6224740 1 --2- 172.21.15.83:0/1636180824 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557061c59000 0x557061c5b3f0 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:40:08.491 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:40:08.488+0000 7fdfb6224740 1 --2- 172.21.15.83:0/1636180824 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557061c4f050 0x557061c53fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:40:08.491 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:40:08.488+0000 7fdfb6224740 1 --2- 172.21.15.83:0/1636180824 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557061c49fc0 0x557061c53a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:40:08.491 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:40:08.488+0000 7fdfb6224740 1 -- 172.21.15.83:0/1636180824 >> 172.21.15.83:0/1636180824 conn(0x557061c44cf0 msgr2=0x557061c46500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:40:08.491 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:40:08.488+0000 7fdf96000640 0 client.25735 ms_handle_reset on v2:172.21.15.117:6838/1967595704 2024-08-19T01:40:08.491 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:40:08.488+0000 7fdfb6224740 1 -- 172.21.15.83:0/1636180824 shutdown_connections 2024-08-19T01:40:08.491 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:40:08.488+0000 7fdfb6224740 1 -- 172.21.15.83:0/1636180824 wait complete. 2024-08-19T01:40:08.491 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:40:08.489+0000 7fdfb6224740 20 client.25735 trim_cache size 0 max 16384 2024-08-19T01:40:08.663 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:40:08.635+0000 7fdfb6224740 1 Processor -- start 2024-08-19T01:40:08.663 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:40:08.658+0000 7fdfb6224740 1 -- start start 2024-08-19T01:40:08.663 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:40:08.658+0000 7fdfb6224740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557061c49fc0 0x557061c3fe40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:40:08.663 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:40:08.658+0000 7fdfb6224740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557061c4f050 0x557061c53840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:40:08.664 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:40:08.658+0000 7fdfb6224740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557061c59000 0x557061c53d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:40:08.664 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:40:08.658+0000 7fdfb6224740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fdfac1c2b10 con 0x557061c59000 2024-08-19T01:40:08.664 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:40:08.658+0000 7fdfb6224740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fdfac083f70 con 0x557061c4f050 2024-08-19T01:40:08.664 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:40:08.658+0000 7fdfb6224740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fdfac081dc0 con 0x557061c49fc0 2024-08-19T01:40:08.667 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:40:08.662+0000 7fdfb1200640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557061c59000 0x557061c53d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:40:08.667 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:40:08.662+0000 7fdfb1200640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557061c59000 0x557061c53d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:48860/0 (socket says 172.21.15.83:48860) 2024-08-19T01:40:08.667 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:40:08.662+0000 7fdfb1200640 1 -- 172.21.15.83:0/3962988826 learned_addr learned my addr 172.21.15.83:0/3962988826 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-08-19T01:40:08.667 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:40:08.663+0000 7fdfb1200640 1 -- 172.21.15.83:0/3962988826 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557061c49fc0 msgr2=0x557061c3fe40 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-08-19T01:40:08.667 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:40:08.663+0000 7fdfb1200640 1 --2- 172.21.15.83:0/3962988826 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557061c49fc0 0x557061c3fe40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:40:08.668 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:40:08.663+0000 7fdfb1200640 1 -- 172.21.15.83:0/3962988826 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557061c4f050 msgr2=0x557061c53840 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-08-19T01:40:08.668 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:40:08.663+0000 7fdfb1200640 1 --2- 172.21.15.83:0/3962988826 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557061c4f050 0x557061c53840 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:40:08.668 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:40:08.663+0000 7fdfb1200640 1 -- 172.21.15.83:0/3962988826 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557061c49200 con 0x557061c59000 2024-08-19T01:40:08.668 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:40:08.663+0000 7fdfb1200640 1 --2- 172.21.15.83:0/3962988826 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557061c59000 0x557061c53d80 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7fdfa409b2e0 tx=0x7fdfa40ce050 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2e8759a4b5ffca13 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:40:08.668 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:40:08.666+0000 7fdfa0c00640 1 -- 172.21.15.83:0/3962988826 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfa4014030 con 0x557061c59000 2024-08-19T01:40:08.668 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:40:08.666+0000 7fdfa0c00640 1 -- 172.21.15.83:0/3962988826 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdf98020cf0 con 0x557061c59000 2024-08-19T01:40:08.668 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:40:08.666+0000 7fdfa0c00640 1 -- 172.21.15.83:0/3962988826 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfa4097c50 con 0x557061c59000 2024-08-19T01:40:08.677 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:40:08.667+0000 7fdfb6224740 1 -- 172.21.15.83:0/3962988826 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557061c59000 msgr2=0x557061c53d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:40:08.677 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:40:08.667+0000 7fdfb6224740 1 --2- 172.21.15.83:0/3962988826 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557061c59000 0x557061c53d80 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7fdfa409b2e0 tx=0x7fdfa40ce050 comp rx=0 tx=0).stop 2024-08-19T01:40:08.694 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:40:08.689+0000 7fdfb6224740 1 -- 172.21.15.83:0/3962988826 shutdown_connections 2024-08-19T01:40:08.694 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:40:08.689+0000 7fdfb6224740 1 --2- 172.21.15.83:0/3962988826 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557061c59000 0x557061c53d80 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:40:08.694 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:40:08.689+0000 7fdfb6224740 1 --2- 172.21.15.83:0/3962988826 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557061c4f050 0x557061c53840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:40:08.694 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:40:08.689+0000 7fdfb6224740 1 --2- 172.21.15.83:0/3962988826 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557061c49fc0 0x557061c3fe40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:40:08.694 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:40:08.689+0000 7fdfb6224740 1 -- 172.21.15.83:0/3962988826 >> 172.21.15.83:0/3962988826 conn(0x557061c44cf0 msgr2=0x557061c4b200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:40:08.695 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:40:08.693+0000 7fdfb6224740 1 -- 172.21.15.83:0/3962988826 shutdown_connections 2024-08-19T01:40:08.695 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:40:08.693+0000 7fdfb6224740 1 -- 172.21.15.83:0/3962988826 wait complete. 2024-08-19T01:40:08.695 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:40:08.694+0000 7fdfb6224740 1 -- start start 2024-08-19T01:40:08.695 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:40:08.694+0000 7fdfb6224740 1 Processor -- start 2024-08-19T01:40:08.704 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:40:08.701+0000 7fdfb6224740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557061c49fc0 0x557061c70e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:40:08.704 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:40:08.701+0000 7fdfb6224740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557061c4f050 0x557061c713d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:40:08.705 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:40:08.701+0000 7fdfb6224740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557061c59000 0x557061cc8710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:40:08.705 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:40:08.701+0000 7fdfb6224740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fdfac082430 con 0x557061c4f050 2024-08-19T01:40:08.705 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:40:08.701+0000 7fdfb6224740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fdfac0838a0 con 0x557061c49fc0 2024-08-19T01:40:08.705 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:40:08.701+0000 7fdfb6224740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fdfac082b00 con 0x557061c59000 2024-08-19T01:40:08.705 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:40:08.701+0000 7fdfb6224740 10 client.0 _ll_register_callbacks cb 0x7fdfa4097880 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-19T01:40:08.705 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:40:08.702+0000 7fdfb1200640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557061c59000 0x557061cc8710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:40:08.705 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:40:08.702+0000 7fdfb1200640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557061c59000 0x557061cc8710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.83:51952/0 (socket says 172.21.15.83:51952) 2024-08-19T01:40:08.705 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:40:08.702+0000 7fdfb1200640 1 -- 172.21.15.83:0/3083376817 learned_addr learned my addr 172.21.15.83:0/3083376817 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-08-19T01:40:08.705 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:40:08.702+0000 7fdfab400640 1 --2- 172.21.15.83:0/3083376817 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557061c4f050 0x557061c713d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:40:08.705 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:40:08.702+0000 7fdfabe00640 1 --2- 172.21.15.83:0/3083376817 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557061c49fc0 0x557061c70e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:40:08.705 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:40:08.702+0000 7fdfb1200640 1 -- 172.21.15.83:0/3083376817 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557061c49fc0 msgr2=0x557061c70e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:40:08.705 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:40:08.702+0000 7fdfb1200640 1 --2- 172.21.15.83:0/3083376817 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557061c49fc0 0x557061c70e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:40:08.706 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:40:08.702+0000 7fdfb1200640 1 -- 172.21.15.83:0/3083376817 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557061c4f050 msgr2=0x557061c713d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:40:08.706 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:40:08.702+0000 7fdfb1200640 1 --2- 172.21.15.83:0/3083376817 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557061c4f050 0x557061c713d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:40:08.706 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:40:08.702+0000 7fdfb1200640 1 -- 172.21.15.83:0/3083376817 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557061cc8c80 con 0x557061c59000 2024-08-19T01:40:08.706 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:40:08.703+0000 7fdfb1200640 1 --2- 172.21.15.83:0/3083376817 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557061c59000 0x557061cc8710 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fdfa4004450 tx=0x7fdfa40045b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5a2611c50a1a5503 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:40:08.730 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:40:08.727+0000 7fdf96000640 10 client.0 ms_handle_connect on v2:172.21.15.167:3300/0 2024-08-19T01:40:08.730 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:40:08.727+0000 7fdf96000640 1 -- 172.21.15.83:0/3083376817 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfa4014030 con 0x557061c59000 2024-08-19T01:40:08.730 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:40:08.727+0000 7fdf96000640 1 -- 172.21.15.83:0/3083376817 <== mon.1 v2:172.21.15.167:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8269+0+0 (secure 0 0 0) 0x7fdfa40bb540 con 0x557061c59000 2024-08-19T01:40:08.738 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:40:08.737+0000 7fdfb6224740 1 -- 172.21.15.83:0/3083376817 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdf98020cf0 con 0x557061c59000 2024-08-19T01:40:08.750 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:40:08.741+0000 7fdf96000640 1 -- 172.21.15.83:0/3083376817 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fdfa40a4250 con 0x557061c59000 2024-08-19T01:40:08.750 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:40:08.748+0000 7fdfb6224740 10 client.25738 fetch_fsmap learned FSMap version 27 2024-08-19T01:40:08.750 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:40:08.748+0000 7fdfb6224740 1 -- 172.21.15.83:0/3083376817 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x557061cc9f70 con 0x557061c59000 2024-08-19T01:40:08.757 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:40:08.752+0000 7fdf96000640 1 -- 172.21.15.83:0/3083376817 <== mon.1 v2:172.21.15.167:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdf98020cf0 con 0x557061c59000 2024-08-19T01:40:08.757 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:40:08.752+0000 7fdf96000640 1 -- 172.21.15.83:0/3083376817 <== mon.1 v2:172.21.15.167:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7fdfa409a4a0 con 0x557061c59000 2024-08-19T01:40:08.757 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:40:08.752+0000 7fdfb6224740 10 client.25738 fetch_fsmap finished waiting for FSMap version 27 2024-08-19T01:40:08.757 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:40:08.752+0000 7fdfb6224740 10 client.25738 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-08-19T01:40:08.757 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:40:08.752+0000 7fdfb6224740 20 client.25738 populate_metadata read hostname 'smithi083' 2024-08-19T01:40:08.757 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:40:08.752+0000 7fdfb6224740 1 --2- 172.21.15.83:0/3083376817 >> [v2:172.21.15.117:6838/1967595704,v1:172.21.15.117:6839/1967595704] conn(0x557061cca5e0 0x557061ccc9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:40:08.758 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:40:08.752+0000 7fdfb6224740 4 client.25738 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-08-19T01:40:08.758 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:40:08.752+0000 7fdfb6224740 1 -- 172.21.15.83:0/3083376817 --> [v2:172.21.15.117:6838/1967595704,v1:172.21.15.117:6839/1967595704] -- command(tid 0: {"prefix": "flush journal"}) -- 0x557061bb5310 con 0x557061cca5e0 2024-08-19T01:40:08.758 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:40:08.756+0000 7fdfabe00640 1 --2- 172.21.15.83:0/3083376817 >> [v2:172.21.15.117:6838/1967595704,v1:172.21.15.117:6839/1967595704] conn(0x557061cca5e0 0x557061ccc9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:40:08.764 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:40:08.762+0000 7fdfabe00640 1 --2- 172.21.15.83:0/3083376817 >> [v2:172.21.15.117:6838/1967595704,v1:172.21.15.117:6839/1967595704] conn(0x557061cca5e0 0x557061ccc9d0 crc :-1 s=READY pgs=102 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=421bbf61cef58117 server_cookie=cfa0c16d4776579e in_seq=0 out_seq=0 2024-08-19T01:40:08.764 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:40:08.763+0000 7fdf96000640 10 client.25738 ms_handle_connect on v2:172.21.15.117:6838/1967595704 2024-08-19T01:40:09.857 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:40:09 smithi083 ceph-mon[17120]: pgmap v498: 97 pgs: 97 active+clean; 510 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 2.4 MiB/s wr, 429 op/s 2024-08-19T01:40:09.952 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:40:09 smithi117 ceph-mon[22016]: pgmap v498: 97 pgs: 97 active+clean; 510 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 2.4 MiB/s wr, 429 op/s 2024-08-19T01:40:09.965 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:40:09 smithi167 ceph-mon[21875]: pgmap v498: 97 pgs: 97 active+clean; 510 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 2.4 MiB/s wr, 429 op/s 2024-08-19T01:40:10.647 INFO:tasks.workunit.client.0.smithi083.stdout: 176 644531 562 421862 560 370088 1588 2024-08-19T01:40:11.654 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:40:11 smithi083 ceph-mon[17120]: pgmap v499: 97 pgs: 97 active+clean; 514 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.8 MiB/s wr, 214 op/s 2024-08-19T01:40:11.952 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:40:11 smithi117 ceph-mon[22016]: pgmap v499: 97 pgs: 97 active+clean; 514 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.8 MiB/s wr, 214 op/s 2024-08-19T01:40:11.966 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:40:11 smithi167 ceph-mon[21875]: pgmap v499: 97 pgs: 97 active+clean; 514 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.8 MiB/s wr, 214 op/s 2024-08-19T01:40:13.904 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:40:13 smithi083 ceph-mon[17120]: pgmap v500: 97 pgs: 97 active+clean; 516 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.3 MiB/s wr, 195 op/s 2024-08-19T01:40:13.952 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:40:13 smithi117 ceph-mon[22016]: pgmap v500: 97 pgs: 97 active+clean; 516 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.3 MiB/s wr, 195 op/s 2024-08-19T01:40:13.965 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:40:13 smithi167 ceph-mon[21875]: pgmap v500: 97 pgs: 97 active+clean; 516 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.3 MiB/s wr, 195 op/s 2024-08-19T01:40:15.662 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:40:15 smithi083 ceph-mon[17120]: pgmap v501: 97 pgs: 97 active+clean; 536 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 4.3 MiB/s wr, 353 op/s 2024-08-19T01:40:15.952 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:40:15 smithi117 ceph-mon[22016]: pgmap v501: 97 pgs: 97 active+clean; 536 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 4.3 MiB/s wr, 353 op/s 2024-08-19T01:40:15.965 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:40:15 smithi167 ceph-mon[21875]: pgmap v501: 97 pgs: 97 active+clean; 536 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 4.3 MiB/s wr, 353 op/s 2024-08-19T01:40:17.671 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:40:17 smithi083 ceph-mon[17120]: pgmap v502: 97 pgs: 97 active+clean; 547 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 5.0 MiB/s wr, 526 op/s 2024-08-19T01:40:17.952 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:40:17 smithi117 ceph-mon[22016]: pgmap v502: 97 pgs: 97 active+clean; 547 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 5.0 MiB/s wr, 526 op/s 2024-08-19T01:40:17.965 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:40:17 smithi167 ceph-mon[21875]: pgmap v502: 97 pgs: 97 active+clean; 547 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 5.0 MiB/s wr, 526 op/s 2024-08-19T01:40:18.557 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:40:18.551+0000 7fdf96000640 1 -- 172.21.15.83:0/3083376817 <== mds.0 v2:172.21.15.117:6838/1967595704 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x557061bb5310 con 0x557061cca5e0 2024-08-19T01:40:18.557 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:40:18.551+0000 7fdf96000640 10 client.25738 handle_command_reply: tid=0 2024-08-19T01:40:18.557 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:40:18.556+0000 7fdfb6224740 1 client.25738 shutdown 2024-08-19T01:40:18.572 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:40:18.571+0000 7fdfb6224740 1 -- 172.21.15.83:0/3083376817 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557061c59000 msgr2=0x557061cc8710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:40:18.572 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:40:18.571+0000 7fdfb6224740 1 --2- 172.21.15.83:0/3083376817 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557061c59000 0x557061cc8710 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fdfa4004450 tx=0x7fdfa40045b0 comp rx=0 tx=0).stop 2024-08-19T01:40:18.576 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:40:18.575+0000 7fdfb6224740 1 -- 172.21.15.83:0/3083376817 shutdown_connections 2024-08-19T01:40:18.576 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:40:18.575+0000 7fdfb6224740 1 --2- 172.21.15.83:0/3083376817 >> [v2:172.21.15.117:6838/1967595704,v1:172.21.15.117:6839/1967595704] conn(0x557061cca5e0 0x557061ccc9d0 crc :-1 s=READY pgs=102 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:40:18.576 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:40:18.575+0000 7fdfb6224740 1 --2- 172.21.15.83:0/3083376817 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557061c59000 0x557061cc8710 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:40:18.576 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:40:18.575+0000 7fdfb6224740 1 --2- 172.21.15.83:0/3083376817 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557061c4f050 0x557061c713d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:40:18.576 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:40:18.575+0000 7fdfb6224740 1 --2- 172.21.15.83:0/3083376817 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557061c49fc0 0x557061c70e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:40:18.576 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:40:18.575+0000 7fdfb6224740 1 -- 172.21.15.83:0/3083376817 >> 172.21.15.83:0/3083376817 conn(0x557061c44cf0 msgr2=0x557061c53840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:40:18.577 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:40:18.575+0000 7fdf96000640 0 client.25738 ms_handle_reset on v2:172.21.15.117:6838/1967595704 2024-08-19T01:40:18.592 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:40:18.591+0000 7fdfb6224740 1 -- 172.21.15.83:0/3083376817 shutdown_connections 2024-08-19T01:40:18.640 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:40:18.627+0000 7fdfb6224740 1 -- 172.21.15.83:0/3083376817 wait complete. 2024-08-19T01:40:18.652 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:40:18.651+0000 7fdfb6224740 20 client.25738 trim_cache size 0 max 16384 2024-08-19T01:40:18.689 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stdout:{ 2024-08-19T01:40:18.689 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stdout: "message": "", 2024-08-19T01:40:18.689 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stdout: "return_code": 0 2024-08-19T01:40:18.689 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stdout:} 2024-08-19T01:40:18.691 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:40:18.690+0000 7fdfb1c00640 1 -- 172.21.15.83:0/722962834 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7fdf900a2d90 msgr2=0x7fdf900a5250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:40:18.691 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:40:18.690+0000 7fdfb1c00640 1 --2- 172.21.15.83:0/722962834 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7fdf900a2d90 0x7fdf900a5250 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fdfac19f550 tx=0x7fdf9801a270 comp rx=0 tx=0).stop 2024-08-19T01:40:18.692 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:40:18.691+0000 7fdfb1c00640 1 -- 172.21.15.83:0/722962834 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdfac10d8e0 msgr2=0x7fdfac07a180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:40:18.692 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:40:18.691+0000 7fdfb1c00640 1 --2- 172.21.15.83:0/722962834 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdfac10d8e0 0x7fdfac07a180 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fdfa4009fd0 tx=0x7fdfa400b260 comp rx=0 tx=0).stop 2024-08-19T01:40:18.693 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:40:18.691+0000 7fdfb1c00640 1 -- 172.21.15.83:0/722962834 shutdown_connections 2024-08-19T01:40:18.693 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:40:18.691+0000 7fdfb1c00640 1 --2- 172.21.15.83:0/722962834 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7fdf900a2d90 0x7fdf900a5250 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:40:18.693 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:40:18.691+0000 7fdfb1c00640 1 --2- 172.21.15.83:0/722962834 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdfac10d8e0 0x7fdfac07a180 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:40:18.693 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:40:18.691+0000 7fdfb1c00640 1 --2- 172.21.15.83:0/722962834 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdfac0733b0 0x7fdfac19e960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:40:18.693 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:40:18.691+0000 7fdfb1c00640 1 --2- 172.21.15.83:0/722962834 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdfac0729e0 0x7fdfac19e420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:40:18.693 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:40:18.691+0000 7fdfb1c00640 1 -- 172.21.15.83:0/722962834 >> 172.21.15.83:0/722962834 conn(0x7fdfac06e880 msgr2=0x7fdfac070620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:40:18.696 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:40:18.695+0000 7fdfb1c00640 1 -- 172.21.15.83:0/722962834 shutdown_connections 2024-08-19T01:40:18.701 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:40:18.699+0000 7fdfb1c00640 1 -- 172.21.15.83:0/722962834 wait complete. 2024-08-19T01:40:19.614 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:40:19 smithi083 ceph-mon[17120]: pgmap v503: 97 pgs: 97 active+clean; 582 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 8.2 MiB/s wr, 1.31k op/s 2024-08-19T01:40:19.952 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:40:19 smithi117 ceph-mon[22016]: pgmap v503: 97 pgs: 97 active+clean; 582 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 8.2 MiB/s wr, 1.31k op/s 2024-08-19T01:40:19.965 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:40:19 smithi167 ceph-mon[21875]: pgmap v503: 97 pgs: 97 active+clean; 582 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 8.2 MiB/s wr, 1.31k op/s 2024-08-19T01:40:20.660 INFO:tasks.workunit.client.0.smithi083.stdout: 179 487520 373 319550 207 286583 742 2024-08-19T01:40:21.909 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:40:21 smithi083 ceph-mon[17120]: pgmap v504: 97 pgs: 97 active+clean; 576 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 8.1 MiB/s wr, 1.57k op/s 2024-08-19T01:40:21.952 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:40:21 smithi117 ceph-mon[22016]: pgmap v504: 97 pgs: 97 active+clean; 576 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 8.1 MiB/s wr, 1.57k op/s 2024-08-19T01:40:21.965 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:40:21 smithi167 ceph-mon[21875]: pgmap v504: 97 pgs: 97 active+clean; 576 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 8.1 MiB/s wr, 1.57k op/s 2024-08-19T01:40:23.014 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833 shell --fsid bad8780e-5dc9-11ef-bccf-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-08-19T01:40:23.905 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:40:23 smithi083 ceph-mon[17120]: pgmap v505: 97 pgs: 97 active+clean; 575 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 7.6 MiB/s wr, 1.53k op/s 2024-08-19T01:40:23.952 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:40:23 smithi117 ceph-mon[22016]: pgmap v505: 97 pgs: 97 active+clean; 575 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 7.6 MiB/s wr, 1.53k op/s 2024-08-19T01:40:23.965 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:40:23 smithi167 ceph-mon[21875]: pgmap v505: 97 pgs: 97 active+clean; 575 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 7.6 MiB/s wr, 1.53k op/s 2024-08-19T01:40:25.904 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:40:25 smithi083 ceph-mon[17120]: pgmap v506: 97 pgs: 97 active+clean; 556 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 8.3 MiB/s wr, 1.59k op/s 2024-08-19T01:40:25.952 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:40:25 smithi117 ceph-mon[22016]: pgmap v506: 97 pgs: 97 active+clean; 556 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 8.3 MiB/s wr, 1.59k op/s 2024-08-19T01:40:25.965 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:40:25 smithi167 ceph-mon[21875]: pgmap v506: 97 pgs: 97 active+clean; 556 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 8.3 MiB/s wr, 1.59k op/s 2024-08-19T01:40:27.905 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:40:27 smithi083 ceph-mon[17120]: pgmap v507: 97 pgs: 97 active+clean; 554 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 6.3 MiB/s wr, 1.44k op/s 2024-08-19T01:40:27.952 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:40:27 smithi117 ceph-mon[22016]: pgmap v507: 97 pgs: 97 active+clean; 554 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 6.3 MiB/s wr, 1.44k op/s 2024-08-19T01:40:27.965 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:40:27 smithi167 ceph-mon[21875]: pgmap v507: 97 pgs: 97 active+clean; 554 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 6.3 MiB/s wr, 1.44k op/s 2024-08-19T01:40:29.656 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/bad8780e-5dc9-11ef-bccf-c7b262605968/mon.a/config 2024-08-19T01:40:29.904 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:40:29 smithi083 ceph-mon[17120]: pgmap v508: 97 pgs: 97 active+clean; 557 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 5.7 MiB/s wr, 1.28k op/s 2024-08-19T01:40:29.952 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:40:29 smithi117 ceph-mon[22016]: pgmap v508: 97 pgs: 97 active+clean; 557 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 5.7 MiB/s wr, 1.28k op/s 2024-08-19T01:40:29.965 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:40:29 smithi167 ceph-mon[21875]: pgmap v508: 97 pgs: 97 active+clean; 557 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 5.7 MiB/s wr, 1.28k op/s 2024-08-19T01:40:30.666 INFO:tasks.workunit.client.0.smithi083.stdout: 187 656250 668 432542 436 387450 1874 2024-08-19T01:40:30.731 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:40:30.726+0000 7f037d400640 1 -- 172.21.15.83:0/159922106 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f037811ae70 msgr2=0x7f037811d260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:40:30.731 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:40:30.726+0000 7f037d400640 1 --2- 172.21.15.83:0/159922106 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f037811ae70 0x7f037811d260 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f0370009be0 tx=0x7f037002a6e0 comp rx=0 tx=0).stop 2024-08-19T01:40:30.731 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:40:30.726+0000 7f037d400640 1 -- 172.21.15.83:0/159922106 shutdown_connections 2024-08-19T01:40:30.732 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:40:30.726+0000 7f037d400640 1 --2- 172.21.15.83:0/159922106 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f037811ae70 0x7f037811d260 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:40:30.732 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:40:30.726+0000 7f037d400640 1 --2- 172.21.15.83:0/159922106 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f037806e760 0x7f037806eba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:40:30.732 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:40:30.726+0000 7f037d400640 1 --2- 172.21.15.83:0/159922106 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f037810fec0 0x7f037806e220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:40:30.732 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:40:30.726+0000 7f037d400640 1 -- 172.21.15.83:0/159922106 >> 172.21.15.83:0/159922106 conn(0x7f0378072eb0 msgr2=0x7f03780732c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:40:30.732 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:40:30.727+0000 7f037d400640 1 -- 172.21.15.83:0/159922106 shutdown_connections 2024-08-19T01:40:30.732 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:40:30.727+0000 7f037d400640 1 -- 172.21.15.83:0/159922106 wait complete. 2024-08-19T01:40:30.732 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:40:30.728+0000 7f037d400640 1 Processor -- start 2024-08-19T01:40:30.732 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:40:30.729+0000 7f037d400640 1 -- start start 2024-08-19T01:40:30.735 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:40:30.733+0000 7f037d400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f037806e760 0x7f03781a6dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:40:30.736 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:40:30.733+0000 7f037d400640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f037810fec0 0x7f03781a7300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:40:30.736 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:40:30.733+0000 7f037d400640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f037811ae70 0x7f03781ab690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:40:30.737 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:40:30.733+0000 7f037d400640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f0378120900 con 0x7f037806e760 2024-08-19T01:40:30.737 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:40:30.733+0000 7f037d400640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f0378120780 con 0x7f037810fec0 2024-08-19T01:40:30.737 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:40:30.734+0000 7f037d400640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f0378120a80 con 0x7f037811ae70 2024-08-19T01:40:30.737 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:40:30.734+0000 7f037ca00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f037811ae70 0x7f03781ab690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:40:30.737 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:40:30.734+0000 7f037ca00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f037811ae70 0x7f03781ab690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.83:48098/0 (socket says 172.21.15.83:48098) 2024-08-19T01:40:30.737 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:40:30.734+0000 7f037ca00640 1 -- 172.21.15.83:0/3602577395 learned_addr learned my addr 172.21.15.83:0/3602577395 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-08-19T01:40:30.737 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:40:30.735+0000 7f0377e00640 1 --2- 172.21.15.83:0/3602577395 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f037806e760 0x7f03781a6dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:40:30.743 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:40:30.737+0000 7f037ca00640 1 -- 172.21.15.83:0/3602577395 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f037810fec0 msgr2=0x7f03781a7300 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-08-19T01:40:30.743 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:40:30.740+0000 7f0377400640 1 --2- 172.21.15.83:0/3602577395 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f037810fec0 0x7f03781a7300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:40:30.743 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:40:30.740+0000 7f037ca00640 1 --2- 172.21.15.83:0/3602577395 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f037810fec0 0x7f03781a7300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:40:30.744 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:40:30.740+0000 7f037ca00640 1 -- 172.21.15.83:0/3602577395 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f037806e760 msgr2=0x7f03781a6dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:40:30.744 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:40:30.742+0000 7f037ca00640 1 --2- 172.21.15.83:0/3602577395 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f037806e760 0x7f03781a6dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:40:30.744 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:40:30.742+0000 7f037ca00640 1 -- 172.21.15.83:0/3602577395 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f03781abd70 con 0x7f037811ae70 2024-08-19T01:40:30.744 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:40:30.742+0000 7f0377400640 1 --2- 172.21.15.83:0/3602577395 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f037810fec0 0x7f03781a7300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-19T01:40:30.744 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:40:30.742+0000 7f0377e00640 1 --2- 172.21.15.83:0/3602577395 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f037806e760 0x7f03781a6dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-19T01:40:30.744 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:40:30.742+0000 7f037ca00640 1 --2- 172.21.15.83:0/3602577395 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f037811ae70 0x7f03781ab690 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f0370034040 tx=0x7f0370030980 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:40:30.745 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:40:30.743+0000 7f0374c00640 1 -- 172.21.15.83:0/3602577395 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0370041020 con 0x7f037811ae70 2024-08-19T01:40:30.745 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:40:30.743+0000 7f0374c00640 1 -- 172.21.15.83:0/3602577395 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0370030c70 con 0x7f037811ae70 2024-08-19T01:40:30.745 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:40:30.743+0000 7f0374c00640 1 -- 172.21.15.83:0/3602577395 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0370036040 con 0x7f037811ae70 2024-08-19T01:40:30.746 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:40:30.745+0000 7f037d400640 1 -- 172.21.15.83:0/3602577395 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f03781ac000 con 0x7f037811ae70 2024-08-19T01:40:30.746 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:40:30.745+0000 7f037d400640 1 -- 172.21.15.83:0/3602577395 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f03781ac460 con 0x7f037811ae70 2024-08-19T01:40:30.754 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:40:30.752+0000 7f03815fd740 1 Processor -- start 2024-08-19T01:40:30.761 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:40:30.754+0000 7f0374c00640 1 -- 172.21.15.83:0/3602577395 <== mon.1 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7f0370044020 con 0x7f037811ae70 2024-08-19T01:40:30.761 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:40:30.756+0000 7f0374c00640 1 --2- 172.21.15.83:0/3602577395 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f035c0a2c40 0x7f035c0a5100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:40:30.761 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:40:30.756+0000 7f0374c00640 1 -- 172.21.15.83:0/3602577395 <== mon.1 v2:172.21.15.167:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8269+0+0 (secure 0 0 0) 0x7f03700efb30 con 0x7f037811ae70 2024-08-19T01:40:30.761 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:40:30.756+0000 7f0377400640 1 --2- 172.21.15.83:0/3602577395 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f035c0a2c40 0x7f035c0a5100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:40:30.762 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:40:30.757+0000 7f03815fd740 1 -- start start 2024-08-19T01:40:30.762 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:40:30.757+0000 7f03815fd740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559a10f58d80 0x559a10f59160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:40:30.762 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:40:30.757+0000 7f03815fd740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559a10f596a0 0x559a10f63a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:40:30.762 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:40:30.757+0000 7f03815fd740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559a10f63f60 0x559a10f66350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:40:30.762 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:40:30.757+0000 7f03815fd740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x559a10ebebf0 con 0x559a10f58d80 2024-08-19T01:40:30.762 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:40:30.757+0000 7f03815fd740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x559a10e56bd0 con 0x559a10f63f60 2024-08-19T01:40:30.762 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:40:30.757+0000 7f03815fd740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x559a10ebabb0 con 0x559a10f596a0 2024-08-19T01:40:30.762 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:40:30.758+0000 7f0377e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559a10f58d80 0x559a10f59160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:40:30.762 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:40:30.759+0000 7f0377e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559a10f58d80 0x559a10f59160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:46000/0 (socket says 172.21.15.83:46000) 2024-08-19T01:40:30.763 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:40:30.759+0000 7f0377e00640 1 -- 172.21.15.83:0/3011841008 learned_addr learned my addr 172.21.15.83:0/3011841008 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-08-19T01:40:30.763 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:40:30.760+0000 7f0377e00640 1 -- 172.21.15.83:0/3011841008 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559a10f596a0 msgr2=0x559a10f63a20 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-08-19T01:40:30.763 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:40:30.760+0000 7f0377e00640 1 --2- 172.21.15.83:0/3011841008 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559a10f596a0 0x559a10f63a20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:40:30.763 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:40:30.760+0000 7f0377e00640 1 -- 172.21.15.83:0/3011841008 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559a10f63f60 msgr2=0x559a10f66350 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-08-19T01:40:30.763 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:40:30.760+0000 7f0377e00640 1 --2- 172.21.15.83:0/3011841008 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559a10f63f60 0x559a10f66350 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:40:30.763 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:40:30.760+0000 7f0377e00640 1 -- 172.21.15.83:0/3011841008 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559a10e5d8a0 con 0x559a10f58d80 2024-08-19T01:40:30.764 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:40:30.762+0000 7f0377e00640 1 --2- 172.21.15.83:0/3011841008 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559a10f58d80 0x559a10f59160 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f036800f540 tx=0x7f036800fa00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a53459f20b2b8c9a server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:40:30.779 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:40:30.763+0000 7f036d200640 1 -- 172.21.15.83:0/3011841008 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f036801d070 con 0x559a10f58d80 2024-08-19T01:40:30.780 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:40:30.763+0000 7f036d200640 1 -- 172.21.15.83:0/3011841008 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0368018440 con 0x559a10f58d80 2024-08-19T01:40:30.780 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:40:30.763+0000 7f036d200640 1 -- 172.21.15.83:0/3011841008 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0368010690 con 0x559a10f58d80 2024-08-19T01:40:30.780 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:40:30.764+0000 7f03815fd740 1 -- 172.21.15.83:0/3011841008 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559a10f58d80 msgr2=0x559a10f59160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:40:30.780 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:40:30.764+0000 7f03815fd740 1 --2- 172.21.15.83:0/3011841008 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559a10f58d80 0x559a10f59160 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f036800f540 tx=0x7f036800fa00 comp rx=0 tx=0).stop 2024-08-19T01:40:30.780 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:40:30.764+0000 7f0377400640 1 --2- 172.21.15.83:0/3602577395 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f035c0a2c40 0x7f035c0a5100 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f0360006fd0 tx=0x7f0360008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:40:30.780 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:40:30.764+0000 7f03815fd740 1 -- 172.21.15.83:0/3011841008 shutdown_connections 2024-08-19T01:40:30.780 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:40:30.764+0000 7f03815fd740 1 --2- 172.21.15.83:0/3011841008 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559a10f63f60 0x559a10f66350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:40:30.780 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:40:30.764+0000 7f03815fd740 1 --2- 172.21.15.83:0/3011841008 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559a10f596a0 0x559a10f63a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:40:30.780 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:40:30.764+0000 7f03815fd740 1 --2- 172.21.15.83:0/3011841008 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559a10f58d80 0x559a10f59160 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:40:30.780 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:40:30.764+0000 7f03815fd740 1 -- 172.21.15.83:0/3011841008 >> 172.21.15.83:0/3011841008 conn(0x559a10f55310 msgr2=0x559a10f556f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:40:30.780 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:40:30.771+0000 7f03815fd740 1 -- 172.21.15.83:0/3011841008 shutdown_connections 2024-08-19T01:40:30.781 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:40:30.771+0000 7f03815fd740 1 -- 172.21.15.83:0/3011841008 wait complete. 2024-08-19T01:40:30.781 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:40:30.772+0000 7f03815fd740 1 -- start start 2024-08-19T01:40:30.781 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:40:30.772+0000 7f03815fd740 1 Processor -- start 2024-08-19T01:40:30.781 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:40:30.775+0000 7f03815fd740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559a10f58d80 0x559a10f60f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:40:30.781 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:40:30.775+0000 7f03815fd740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559a10f596a0 0x559a10f61470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:40:30.781 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:40:30.775+0000 7f03815fd740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559a10f63f60 0x559a10f619b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:40:30.781 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:40:30.776+0000 7f03815fd740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x559a10f66b30 con 0x559a10f58d80 2024-08-19T01:40:30.781 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:40:30.776+0000 7f037ca00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559a10f63f60 0x559a10f619b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:40:30.781 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:40:30.776+0000 7f037ca00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559a10f63f60 0x559a10f619b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.83:48120/0 (socket says 172.21.15.83:48120) 2024-08-19T01:40:30.781 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:40:30.776+0000 7f037ca00640 1 -- 172.21.15.83:0/4210821144 learned_addr learned my addr 172.21.15.83:0/4210821144 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-08-19T01:40:30.781 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:40:30.776+0000 7f0377400640 1 --2- 172.21.15.83:0/4210821144 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559a10f596a0 0x559a10f61470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:40:30.781 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:40:30.776+0000 7f03815fd740 1 -- 172.21.15.83:0/4210821144 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x559a10e79d40 con 0x559a10f596a0 2024-08-19T01:40:30.781 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:40:30.777+0000 7f03815fd740 1 -- 172.21.15.83:0/4210821144 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x559a10f66cb0 con 0x559a10f63f60 2024-08-19T01:40:30.781 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:40:30.777+0000 7f037ca00640 1 -- 172.21.15.83:0/4210821144 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559a10f596a0 msgr2=0x559a10f61470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:40:30.782 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:40:30.777+0000 7f037ca00640 1 --2- 172.21.15.83:0/4210821144 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559a10f596a0 0x559a10f61470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:40:30.782 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:40:30.777+0000 7f037ca00640 1 -- 172.21.15.83:0/4210821144 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559a10f58d80 msgr2=0x559a10f60f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:40:30.782 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:40:30.777+0000 7f037ca00640 1 --2- 172.21.15.83:0/4210821144 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559a10f58d80 0x559a10f60f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:40:30.782 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:40:30.777+0000 7f037ca00640 1 -- 172.21.15.83:0/4210821144 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559a10c303c0 con 0x559a10f63f60 2024-08-19T01:40:30.782 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:40:30.779+0000 7f037ca00640 1 --2- 172.21.15.83:0/4210821144 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559a10f63f60 0x559a10f619b0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f03700b8d40 tx=0x7f03700bbd90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c06bcb6d4844ea5f server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:40:30.782 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:40:30.779+0000 7f0365600640 10 client.0 ms_handle_connect on v2:172.21.15.167:3300/0 2024-08-19T01:40:30.782 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:40:30.779+0000 7f03815fd740 10 client.0 _ll_register_callbacks cb 0x559a10e42b10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-19T01:40:30.783 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:40:30.779+0000 7f03815fd740 1 -- 172.21.15.83:0/4210821144 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0368018440 con 0x559a10f63f60 2024-08-19T01:40:30.783 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:40:30.779+0000 7f0365600640 1 -- 172.21.15.83:0/4210821144 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03700c0720 con 0x559a10f63f60 2024-08-19T01:40:30.783 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:40:30.779+0000 7f0365600640 1 -- 172.21.15.83:0/4210821144 <== mon.1 v2:172.21.15.167:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8269+0+0 (secure 0 0 0) 0x7f03700d51b0 con 0x559a10f63f60 2024-08-19T01:40:30.783 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:40:30.779+0000 7f0365600640 1 -- 172.21.15.83:0/4210821144 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f03700c4fe0 con 0x559a10f63f60 2024-08-19T01:40:30.783 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:40:30.780+0000 7f03815fd740 10 client.25744 fetch_fsmap learned FSMap version 27 2024-08-19T01:40:30.783 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:40:30.780+0000 7f03815fd740 1 -- 172.21.15.83:0/4210821144 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({fsmap=27}) -- 0x559a10f72370 con 0x559a10f63f60 2024-08-19T01:40:30.783 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:40:30.780+0000 7f0365600640 1 -- 172.21.15.83:0/4210821144 <== mon.1 v2:172.21.15.167:3300/0 4 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f03700c4b50 con 0x559a10f63f60 2024-08-19T01:40:30.789 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:40:30.783+0000 7f03815fd740 10 client.25744 fetch_fsmap finished waiting for FSMap version 27 2024-08-19T01:40:30.789 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:40:30.783+0000 7f03815fd740 10 client.25744 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-19T01:40:30.789 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:40:30.783+0000 7f03815fd740 20 client.25744 populate_metadata read hostname 'smithi083' 2024-08-19T01:40:30.789 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:40:30.783+0000 7f03815fd740 1 --2- 172.21.15.83:0/4210821144 >> [v2:172.21.15.117:6838/1967595704,v1:172.21.15.117:6839/1967595704] conn(0x559a10f73b10 0x559a10f75fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:40:30.789 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:40:30.783+0000 7f03815fd740 4 client.25744 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-19T01:40:30.789 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:40:30.783+0000 7f03815fd740 1 -- 172.21.15.83:0/4210821144 --> [v2:172.21.15.117:6838/1967595704,v1:172.21.15.117:6839/1967595704] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559a10ecc160 con 0x559a10f73b10 2024-08-19T01:40:30.789 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:40:30.784+0000 7f0377e00640 1 --2- 172.21.15.83:0/4210821144 >> [v2:172.21.15.117:6838/1967595704,v1:172.21.15.117:6839/1967595704] conn(0x559a10f73b10 0x559a10f75fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:40:30.794 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:40:30.793+0000 7f0365600640 10 client.25744 ms_handle_connect on v2:172.21.15.117:6838/1967595704 2024-08-19T01:40:30.794 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:40:30.793+0000 7f0377e00640 1 --2- 172.21.15.83:0/4210821144 >> [v2:172.21.15.117:6838/1967595704,v1:172.21.15.117:6839/1967595704] conn(0x559a10f73b10 0x559a10f75fd0 crc :-1 s=READY pgs=103 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=17fdcee5d03e9fc server_cookie=96f8e955835c999e in_seq=0 out_seq=0 2024-08-19T01:40:30.797 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:40:30.794+0000 7f0365600640 1 -- 172.21.15.83:0/4210821144 <== mds.0 v2:172.21.15.117:6838/1967595704 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x559a10ecc160 con 0x559a10f73b10 2024-08-19T01:40:30.797 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:40:30.794+0000 7f0365600640 10 client.25744 handle_command_reply: tid=0 2024-08-19T01:40:30.797 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:40:30.794+0000 7f03815fd740 1 client.25744 shutdown 2024-08-19T01:40:30.797 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:40:30.794+0000 7f03815fd740 1 -- 172.21.15.83:0/4210821144 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559a10f63f60 msgr2=0x559a10f619b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:40:30.797 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:40:30.794+0000 7f03815fd740 1 --2- 172.21.15.83:0/4210821144 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559a10f63f60 0x559a10f619b0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f03700b8d40 tx=0x7f03700bbd90 comp rx=0 tx=0).stop 2024-08-19T01:40:30.797 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:40:30.795+0000 7f03815fd740 1 -- 172.21.15.83:0/4210821144 shutdown_connections 2024-08-19T01:40:30.797 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:40:30.795+0000 7f03815fd740 1 --2- 172.21.15.83:0/4210821144 >> [v2:172.21.15.117:6838/1967595704,v1:172.21.15.117:6839/1967595704] conn(0x559a10f73b10 0x559a10f75fd0 crc :-1 s=READY pgs=103 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:40:30.797 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:40:30.795+0000 7f03815fd740 1 --2- 172.21.15.83:0/4210821144 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559a10f63f60 0x559a10f619b0 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:40:30.797 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:40:30.795+0000 7f03815fd740 1 --2- 172.21.15.83:0/4210821144 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559a10f596a0 0x559a10f61470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:40:30.797 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:40:30.795+0000 7f03815fd740 1 --2- 172.21.15.83:0/4210821144 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559a10f58d80 0x559a10f60f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:40:30.797 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:40:30.795+0000 7f03815fd740 1 -- 172.21.15.83:0/4210821144 >> 172.21.15.83:0/4210821144 conn(0x559a10f55310 msgr2=0x559a10f5b040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:40:30.797 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:40:30.795+0000 7f0365600640 0 client.25744 ms_handle_reset on v2:172.21.15.117:6838/1967595704 2024-08-19T01:40:30.798 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:40:30.795+0000 7f03815fd740 1 -- 172.21.15.83:0/4210821144 shutdown_connections 2024-08-19T01:40:30.798 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:40:30.795+0000 7f03815fd740 1 -- 172.21.15.83:0/4210821144 wait complete. 2024-08-19T01:40:30.798 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:40:30.795+0000 7f03815fd740 20 client.25744 trim_cache size 0 max 16384 2024-08-19T01:40:30.862 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:40:30.857+0000 7f03815fd740 1 Processor -- start 2024-08-19T01:40:30.862 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:40:30.858+0000 7f03815fd740 1 -- start start 2024-08-19T01:40:30.863 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:40:30.858+0000 7f03815fd740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559a10f55310 0x559a10f7b8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:40:30.863 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:40:30.858+0000 7f03815fd740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559a10f58d80 0x559a10f7be20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:40:30.863 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:40:30.859+0000 7f03815fd740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559a10f596a0 0x559a10f5e190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:40:30.863 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:40:30.859+0000 7f03815fd740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f03781be790 con 0x559a10f55310 2024-08-19T01:40:30.864 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:40:30.859+0000 7f03815fd740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f03781bc060 con 0x559a10f596a0 2024-08-19T01:40:30.864 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:40:30.859+0000 7f03815fd740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f03781b9eb0 con 0x559a10f58d80 2024-08-19T01:40:30.864 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:40:30.859+0000 7f0377e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559a10f55310 0x559a10f7b8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:40:30.864 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:40:30.859+0000 7f0377e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559a10f55310 0x559a10f7b8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:42916/0 (socket says 172.21.15.83:42916) 2024-08-19T01:40:30.864 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:40:30.859+0000 7f0377e00640 1 -- 172.21.15.83:0/4141510593 learned_addr learned my addr 172.21.15.83:0/4141510593 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-08-19T01:40:30.864 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:40:30.859+0000 7f037ca00640 1 --2- 172.21.15.83:0/4141510593 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559a10f596a0 0x559a10f5e190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:40:30.864 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:40:30.859+0000 7f0377e00640 1 -- 172.21.15.83:0/4141510593 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559a10f58d80 msgr2=0x559a10f7be20 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-08-19T01:40:30.864 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:40:30.859+0000 7f0377e00640 1 --2- 172.21.15.83:0/4141510593 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559a10f58d80 0x559a10f7be20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:40:30.864 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:40:30.859+0000 7f0377e00640 1 -- 172.21.15.83:0/4141510593 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559a10f596a0 msgr2=0x559a10f5e190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:40:30.864 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:40:30.859+0000 7f0377e00640 1 --2- 172.21.15.83:0/4141510593 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559a10f596a0 0x559a10f5e190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:40:30.864 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:40:30.859+0000 7f0377e00640 1 -- 172.21.15.83:0/4141510593 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559a10f58bb0 con 0x559a10f55310 2024-08-19T01:40:30.864 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:40:30.859+0000 7f037ca00640 1 --2- 172.21.15.83:0/4141510593 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559a10f596a0 0x559a10f5e190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-19T01:40:30.865 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:40:30.861+0000 7f0377e00640 1 --2- 172.21.15.83:0/4141510593 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559a10f55310 0x559a10f7b8e0 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f0368018f10 tx=0x7f0368018f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f889a3a6c68b9459 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:40:30.865 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:40:30.861+0000 7f0367e00640 1 -- 172.21.15.83:0/4141510593 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f036801d040 con 0x559a10f55310 2024-08-19T01:40:30.865 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:40:30.862+0000 7f0367e00640 1 -- 172.21.15.83:0/4141510593 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f036801e040 con 0x559a10f55310 2024-08-19T01:40:30.865 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:40:30.862+0000 7f0367e00640 1 -- 172.21.15.83:0/4141510593 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f036800da30 con 0x559a10f55310 2024-08-19T01:40:30.868 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:40:30.863+0000 7f03815fd740 1 -- 172.21.15.83:0/4141510593 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559a10f55310 msgr2=0x559a10f7b8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:40:30.868 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:40:30.863+0000 7f03815fd740 1 --2- 172.21.15.83:0/4141510593 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559a10f55310 0x559a10f7b8e0 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f0368018f10 tx=0x7f0368018f40 comp rx=0 tx=0).stop 2024-08-19T01:40:30.869 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:40:30.864+0000 7f03815fd740 1 -- 172.21.15.83:0/4141510593 shutdown_connections 2024-08-19T01:40:30.869 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:40:30.864+0000 7f03815fd740 1 --2- 172.21.15.83:0/4141510593 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559a10f596a0 0x559a10f5e190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:40:30.869 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:40:30.864+0000 7f03815fd740 1 --2- 172.21.15.83:0/4141510593 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559a10f58d80 0x559a10f7be20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:40:30.869 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:40:30.864+0000 7f03815fd740 1 --2- 172.21.15.83:0/4141510593 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559a10f55310 0x559a10f7b8e0 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:40:30.869 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:40:30.864+0000 7f03815fd740 1 -- 172.21.15.83:0/4141510593 >> 172.21.15.83:0/4141510593 conn(0x559a10f61b10 msgr2=0x559a10f5f800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:40:30.869 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:40:30.865+0000 7f03815fd740 1 -- 172.21.15.83:0/4141510593 shutdown_connections 2024-08-19T01:40:30.870 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:40:30.865+0000 7f03815fd740 1 -- 172.21.15.83:0/4141510593 wait complete. 2024-08-19T01:40:30.870 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:40:30.866+0000 7f03815fd740 1 -- start start 2024-08-19T01:40:30.870 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:40:30.866+0000 7f03815fd740 1 Processor -- start 2024-08-19T01:40:30.871 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:40:30.869+0000 7f03815fd740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559a10f55310 0x559a10f67eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:40:30.871 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:40:30.870+0000 7f03815fd740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559a10f58d80 0x559a10f683f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:40:30.873 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:40:30.870+0000 7f03815fd740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559a10f596a0 0x559a10f68930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:40:30.873 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:40:30.870+0000 7f03815fd740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f03781ba520 con 0x559a10f55310 2024-08-19T01:40:30.873 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:40:30.870+0000 7f03815fd740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f03781bb990 con 0x559a10f596a0 2024-08-19T01:40:30.873 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:40:30.870+0000 7f03815fd740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f03781babf0 con 0x559a10f58d80 2024-08-19T01:40:30.873 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:40:30.870+0000 7f03815fd740 10 client.0 _ll_register_callbacks cb 0x7f0370005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-19T01:40:30.873 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:40:30.871+0000 7f037ca00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559a10f596a0 0x559a10f68930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:40:30.873 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:40:30.871+0000 7f037ca00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559a10f596a0 0x559a10f68930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.83:51248/0 (socket says 172.21.15.83:51248) 2024-08-19T01:40:30.874 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:40:30.871+0000 7f037ca00640 1 -- 172.21.15.83:0/4146026975 learned_addr learned my addr 172.21.15.83:0/4146026975 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-08-19T01:40:30.874 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:40:30.871+0000 7f0377e00640 1 --2- 172.21.15.83:0/4146026975 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559a10f55310 0x559a10f67eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:40:30.875 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:40:30.872+0000 7f0377400640 1 --2- 172.21.15.83:0/4146026975 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559a10f58d80 0x559a10f683f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:40:30.875 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:40:30.872+0000 7f037ca00640 1 -- 172.21.15.83:0/4146026975 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559a10f58d80 msgr2=0x559a10f683f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:40:30.875 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:40:30.872+0000 7f037ca00640 1 --2- 172.21.15.83:0/4146026975 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559a10f58d80 0x559a10f683f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:40:30.875 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:40:30.872+0000 7f037ca00640 1 -- 172.21.15.83:0/4146026975 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559a10f55310 msgr2=0x559a10f67eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:40:30.875 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:40:30.872+0000 7f037ca00640 1 --2- 172.21.15.83:0/4146026975 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559a10f55310 0x559a10f67eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:40:30.875 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:40:30.872+0000 7f037ca00640 1 -- 172.21.15.83:0/4146026975 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559a10f55fa0 con 0x559a10f596a0 2024-08-19T01:40:30.875 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:40:30.872+0000 7f0377400640 1 --2- 172.21.15.83:0/4146026975 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559a10f58d80 0x559a10f683f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-19T01:40:30.876 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:40:30.873+0000 7f0365600640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-08-19T01:40:30.876 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:40:30.873+0000 7f037ca00640 1 --2- 172.21.15.83:0/4146026975 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559a10f596a0 0x559a10f68930 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f03700ed980 tx=0x7f037002a6e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8643e77e8f18fb55 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:40:30.876 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:40:30.873+0000 7f0365600640 1 -- 172.21.15.83:0/4146026975 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03700b8610 con 0x559a10f596a0 2024-08-19T01:40:30.876 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:40:30.873+0000 7f03815fd740 1 -- 172.21.15.83:0/4146026975 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f036801e040 con 0x559a10f596a0 2024-08-19T01:40:30.876 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:40:30.873+0000 7f0365600640 1 -- 172.21.15.83:0/4146026975 <== mon.2 v2:172.21.15.117:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8269+0+0 (secure 0 0 0) 0x7f03700da070 con 0x559a10f596a0 2024-08-19T01:40:30.876 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:40:30.873+0000 7f0365600640 1 -- 172.21.15.83:0/4146026975 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f03700b8610 con 0x559a10f596a0 2024-08-19T01:40:30.884 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:40:30.874+0000 7f03815fd740 10 client.25811 fetch_fsmap learned FSMap version 27 2024-08-19T01:40:30.884 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:40:30.874+0000 7f03815fd740 1 -- 172.21.15.83:0/4146026975 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x559a10f56480 con 0x559a10f596a0 2024-08-19T01:40:30.884 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:40:30.875+0000 7f0365600640 1 -- 172.21.15.83:0/4146026975 <== mon.2 v2:172.21.15.117:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f036801e040 con 0x559a10f596a0 2024-08-19T01:40:30.885 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:40:30.877+0000 7f0365600640 1 -- 172.21.15.83:0/4146026975 <== mon.2 v2:172.21.15.117:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f03700bb930 con 0x559a10f596a0 2024-08-19T01:40:30.885 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:40:30.877+0000 7f03815fd740 10 client.25811 fetch_fsmap finished waiting for FSMap version 27 2024-08-19T01:40:30.885 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:40:30.877+0000 7f03815fd740 10 client.25811 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-19T01:40:30.885 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:40:30.877+0000 7f03815fd740 20 client.25811 populate_metadata read hostname 'smithi083' 2024-08-19T01:40:30.885 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:40:30.877+0000 7f03815fd740 1 --2- 172.21.15.83:0/4146026975 >> [v2:172.21.15.117:6838/1967595704,v1:172.21.15.117:6839/1967595704] conn(0x559a10f63f60 0x559a10fe0fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:40:30.885 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:40:30.877+0000 7f03815fd740 4 client.25811 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-08-19T01:40:30.885 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:40:30.877+0000 7f03815fd740 1 -- 172.21.15.83:0/4146026975 --> [v2:172.21.15.117:6838/1967595704,v1:172.21.15.117:6839/1967595704] -- command(tid 0: {"prefix": "scrub status"}) -- 0x559a10ec6670 con 0x559a10f63f60 2024-08-19T01:40:30.885 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:40:30.883+0000 7f0377e00640 1 --2- 172.21.15.83:0/4146026975 >> [v2:172.21.15.117:6838/1967595704,v1:172.21.15.117:6839/1967595704] conn(0x559a10f63f60 0x559a10fe0fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:40:30.886 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:40:30.884+0000 7f0377e00640 1 --2- 172.21.15.83:0/4146026975 >> [v2:172.21.15.117:6838/1967595704,v1:172.21.15.117:6839/1967595704] conn(0x559a10f63f60 0x559a10fe0fa0 crc :-1 s=READY pgs=104 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=92bf025d296ea7f3 server_cookie=b7acfddf1ded490e in_seq=0 out_seq=0 2024-08-19T01:40:30.886 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:40:30.884+0000 7f0365600640 10 client.25811 ms_handle_connect on v2:172.21.15.117:6838/1967595704 2024-08-19T01:40:30.891 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:40:30.888+0000 7f0365600640 1 -- 172.21.15.83:0/4146026975 <== mds.0 v2:172.21.15.117:6838/1967595704 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x559a10ec6670 con 0x559a10f63f60 2024-08-19T01:40:30.892 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:40:30.888+0000 7f0365600640 10 client.25811 handle_command_reply: tid=0 2024-08-19T01:40:30.892 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:40:30.888+0000 7f03815fd740 1 client.25811 shutdown 2024-08-19T01:40:30.892 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:40:30.889+0000 7f03815fd740 1 -- 172.21.15.83:0/4146026975 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559a10f596a0 msgr2=0x559a10f68930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:40:30.892 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:40:30.889+0000 7f03815fd740 1 --2- 172.21.15.83:0/4146026975 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559a10f596a0 0x559a10f68930 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f03700ed980 tx=0x7f037002a6e0 comp rx=0 tx=0).stop 2024-08-19T01:40:30.892 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:40:30.889+0000 7f03815fd740 1 -- 172.21.15.83:0/4146026975 shutdown_connections 2024-08-19T01:40:30.892 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:40:30.889+0000 7f03815fd740 1 --2- 172.21.15.83:0/4146026975 >> [v2:172.21.15.117:6838/1967595704,v1:172.21.15.117:6839/1967595704] conn(0x559a10f63f60 0x559a10fe0fa0 crc :-1 s=READY pgs=104 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:40:30.892 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:40:30.889+0000 7f03815fd740 1 --2- 172.21.15.83:0/4146026975 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559a10f596a0 0x559a10f68930 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:40:30.892 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:40:30.889+0000 7f03815fd740 1 --2- 172.21.15.83:0/4146026975 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559a10f58d80 0x559a10f683f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:40:30.892 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:40:30.889+0000 7f03815fd740 1 --2- 172.21.15.83:0/4146026975 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559a10f55310 0x559a10f67eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:40:30.892 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:40:30.889+0000 7f03815fd740 1 -- 172.21.15.83:0/4146026975 >> 172.21.15.83:0/4146026975 conn(0x559a10f61b10 msgr2=0x559a10f5a7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:40:30.892 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:40:30.890+0000 7f0365600640 0 client.25811 ms_handle_reset on v2:172.21.15.117:6838/1967595704 2024-08-19T01:40:30.895 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-08-19T01:40:30.895 INFO:teuthology.orchestra.run.smithi083.stdout: "status": "no active scrubs running", 2024-08-19T01:40:30.895 INFO:teuthology.orchestra.run.smithi083.stdout: "scrubs": {} 2024-08-19T01:40:30.895 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-08-19T01:40:30.895 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:40:30.892+0000 7f03815fd740 1 -- 172.21.15.83:0/4146026975 shutdown_connections 2024-08-19T01:40:30.896 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:40:30.892+0000 7f03815fd740 1 -- 172.21.15.83:0/4146026975 wait complete. 2024-08-19T01:40:30.896 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:40:30.892+0000 7f03815fd740 20 client.25811 trim_cache size 0 max 16384 2024-08-19T01:40:30.898 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:40:30.894+0000 7f037d400640 1 -- 172.21.15.83:0/3602577395 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f035c0a2c40 msgr2=0x7f035c0a5100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:40:30.899 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:40:30.894+0000 7f037d400640 1 --2- 172.21.15.83:0/3602577395 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f035c0a2c40 0x7f035c0a5100 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f0360006fd0 tx=0x7f0360008040 comp rx=0 tx=0).stop 2024-08-19T01:40:30.899 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:40:30.894+0000 7f037d400640 1 -- 172.21.15.83:0/3602577395 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f037811ae70 msgr2=0x7f03781ab690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:40:30.899 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:40:30.894+0000 7f037d400640 1 --2- 172.21.15.83:0/3602577395 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f037811ae70 0x7f03781ab690 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f0370034040 tx=0x7f0370030980 comp rx=0 tx=0).stop 2024-08-19T01:40:30.899 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:40:30.894+0000 7f037d400640 1 -- 172.21.15.83:0/3602577395 shutdown_connections 2024-08-19T01:40:30.899 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:40:30.894+0000 7f037d400640 1 --2- 172.21.15.83:0/3602577395 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f035c0a2c40 0x7f035c0a5100 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:40:30.899 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:40:30.894+0000 7f037d400640 1 --2- 172.21.15.83:0/3602577395 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f037811ae70 0x7f03781ab690 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:40:30.899 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:40:30.894+0000 7f037d400640 1 --2- 172.21.15.83:0/3602577395 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f037810fec0 0x7f03781a7300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:40:30.899 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:40:30.894+0000 7f037d400640 1 --2- 172.21.15.83:0/3602577395 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f037806e760 0x7f03781a6dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:40:30.899 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:40:30.895+0000 7f037d400640 1 -- 172.21.15.83:0/3602577395 >> 172.21.15.83:0/3602577395 conn(0x7f0378072eb0 msgr2=0x7f0378119000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:40:30.899 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:40:30.895+0000 7f037d400640 1 -- 172.21.15.83:0/3602577395 shutdown_connections 2024-08-19T01:40:30.900 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:40:30.895+0000 7f037d400640 1 -- 172.21.15.83:0/3602577395 wait complete. 2024-08-19T01:40:31.919 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:40:31 smithi083 ceph-mon[17120]: pgmap v509: 97 pgs: 97 active+clean; 559 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.7 MiB/s wr, 515 op/s 2024-08-19T01:40:31.919 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:40:31 smithi083 ceph-mon[17120]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-19T01:40:31.952 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:40:31 smithi117 ceph-mon[22016]: pgmap v509: 97 pgs: 97 active+clean; 559 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.7 MiB/s wr, 515 op/s 2024-08-19T01:40:31.952 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:40:31 smithi117 ceph-mon[22016]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-19T01:40:31.965 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:40:31 smithi167 ceph-mon[21875]: pgmap v509: 97 pgs: 97 active+clean; 559 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.7 MiB/s wr, 515 op/s 2024-08-19T01:40:31.966 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:40:31 smithi167 ceph-mon[21875]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-19T01:40:32.519 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-08-19T01:40:32.520 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833 shell --fsid bad8780e-5dc9-11ef-bccf-c7b262605968 -- ceph fs dump --format=json 2024-08-19T01:40:33.652 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:40:33 smithi083 ceph-mon[17120]: pgmap v510: 97 pgs: 97 active+clean; 558 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.7 MiB/s wr, 193 op/s 2024-08-19T01:40:33.952 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:40:33 smithi117 ceph-mon[22016]: pgmap v510: 97 pgs: 97 active+clean; 558 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.7 MiB/s wr, 193 op/s 2024-08-19T01:40:33.965 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:40:33 smithi167 ceph-mon[21875]: pgmap v510: 97 pgs: 97 active+clean; 558 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.7 MiB/s wr, 193 op/s 2024-08-19T01:40:34.673 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:40:34.671+0000 7faa46a00640 1 -- 172.21.15.83:0/1453424744 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7faa4010b210 msgr2=0x7faa4010d680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:40:34.673 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:40:34.672+0000 7faa46a00640 1 --2- 172.21.15.83:0/1453424744 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7faa4010b210 0x7faa4010d680 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7faa38009f00 tx=0x7faa3802ab60 comp rx=0 tx=0).stop 2024-08-19T01:40:34.711 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:40:34.709+0000 7faa46a00640 1 -- 172.21.15.83:0/1453424744 shutdown_connections 2024-08-19T01:40:34.711 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:40:34.709+0000 7faa46a00640 1 --2- 172.21.15.83:0/1453424744 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7faa4010dbc0 0x7faa40110080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:40:34.711 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:40:34.709+0000 7faa46a00640 1 --2- 172.21.15.83:0/1453424744 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7faa4010b210 0x7faa4010d680 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:40:34.711 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:40:34.709+0000 7faa46a00640 1 --2- 172.21.15.83:0/1453424744 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7faa40072ab0 0x7faa40072eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:40:34.711 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:40:34.709+0000 7faa46a00640 1 -- 172.21.15.83:0/1453424744 >> 172.21.15.83:0/1453424744 conn(0x7faa4006e880 msgr2=0x7faa40070cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:40:34.736 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:40:34.735+0000 7faa46a00640 1 -- 172.21.15.83:0/1453424744 shutdown_connections 2024-08-19T01:40:34.739 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:40:34.736+0000 7faa46a00640 1 -- 172.21.15.83:0/1453424744 wait complete. 2024-08-19T01:40:34.739 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:40:34.738+0000 7faa46a00640 1 Processor -- start 2024-08-19T01:40:34.744 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:40:34.738+0000 7faa46a00640 1 -- start start 2024-08-19T01:40:34.754 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:40:34.747+0000 7faa46a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7faa40072ab0 0x7faa40115630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:40:34.754 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:40:34.747+0000 7faa46a00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7faa4010b210 0x7faa40115b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:40:34.754 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:40:34.747+0000 7faa46a00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7faa4010dbc0 0x7faa401b6910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:40:34.754 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:40:34.751+0000 7faa3fe00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7faa40072ab0 0x7faa40115630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:40:34.754 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:40:34.752+0000 7faa3fe00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7faa40072ab0 0x7faa40115630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:42942/0 (socket says 172.21.15.83:42942) 2024-08-19T01:40:34.754 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:40:34.753+0000 7faa3f400640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7faa4010b210 0x7faa40115b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:40:34.754 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:40:34.753+0000 7faa3f400640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7faa4010b210 0x7faa40115b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.83:56722/0 (socket says 172.21.15.83:56722) 2024-08-19T01:40:34.755 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:40:34.753+0000 7faa3fe00640 1 -- 172.21.15.83:0/329605463 learned_addr learned my addr 172.21.15.83:0/329605463 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-08-19T01:40:34.755 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:40:34.753+0000 7faa44a00640 1 --2- 172.21.15.83:0/329605463 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7faa4010dbc0 0x7faa401b6910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:40:34.758 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:40:34.754+0000 7faa46a00640 1 -- 172.21.15.83:0/329605463 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7faa4011c200 con 0x7faa40072ab0 2024-08-19T01:40:34.759 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:40:34.754+0000 7faa46a00640 1 -- 172.21.15.83:0/329605463 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7faa4011c080 con 0x7faa4010dbc0 2024-08-19T01:40:34.759 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:40:34.754+0000 7faa46a00640 1 -- 172.21.15.83:0/329605463 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7faa4011c380 con 0x7faa4010b210 2024-08-19T01:40:34.759 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:40:34.755+0000 7faa44a00640 1 -- 172.21.15.83:0/329605463 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7faa4010b210 msgr2=0x7faa40115b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:40:34.759 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:40:34.755+0000 7faa44a00640 1 --2- 172.21.15.83:0/329605463 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7faa4010b210 0x7faa40115b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:40:34.759 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:40:34.755+0000 7faa44a00640 1 -- 172.21.15.83:0/329605463 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7faa40072ab0 msgr2=0x7faa40115630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:40:34.759 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:40:34.755+0000 7faa44a00640 1 --2- 172.21.15.83:0/329605463 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7faa40072ab0 0x7faa40115630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:40:34.759 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:40:34.755+0000 7faa44a00640 1 -- 172.21.15.83:0/329605463 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faa401b6e50 con 0x7faa4010dbc0 2024-08-19T01:40:34.764 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:40:34.758+0000 7faa44a00640 1 --2- 172.21.15.83:0/329605463 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7faa4010dbc0 0x7faa401b6910 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7faa3000b3e0 tx=0x7faa3000b8a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:40:34.765 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:40:34.759+0000 7faa3cc00640 1 -- 172.21.15.83:0/329605463 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa30015020 con 0x7faa4010dbc0 2024-08-19T01:40:34.765 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:40:34.759+0000 7faa3cc00640 1 -- 172.21.15.83:0/329605463 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7faa30011d10 con 0x7faa4010dbc0 2024-08-19T01:40:34.765 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:40:34.759+0000 7faa3cc00640 1 -- 172.21.15.83:0/329605463 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa3000f2e0 con 0x7faa4010dbc0 2024-08-19T01:40:34.774 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:40:34.772+0000 7faa46a00640 1 -- 172.21.15.83:0/329605463 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faa401b7020 con 0x7faa4010dbc0 2024-08-19T01:40:34.774 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:40:34.773+0000 7faa46a00640 1 -- 172.21.15.83:0/329605463 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faa401b74f0 con 0x7faa4010dbc0 2024-08-19T01:40:34.785 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:40:34.784+0000 7faa49a7c740 1 Processor -- start 2024-08-19T01:40:34.820 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:40:34.804+0000 7faa3cc00640 1 -- 172.21.15.83:0/329605463 <== mon.2 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7faa30018020 con 0x7faa4010dbc0 2024-08-19T01:40:34.820 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:40:34.807+0000 7faa3cc00640 1 --2- 172.21.15.83:0/329605463 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7faa240a2d90 0x7faa240a5250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:40:34.820 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:40:34.807+0000 7faa3cc00640 1 -- 172.21.15.83:0/329605463 <== mon.2 v2:172.21.15.117:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8269+0+0 (secure 0 0 0) 0x7faa300ce680 con 0x7faa4010dbc0 2024-08-19T01:40:34.820 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:40:34.813+0000 7faa3f400640 1 --2- 172.21.15.83:0/329605463 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7faa240a2d90 0x7faa240a5250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:40:34.820 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:40:34.814+0000 7faa3f400640 1 --2- 172.21.15.83:0/329605463 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7faa240a2d90 0x7faa240a5250 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7faa38034a20 tx=0x7faa38000f40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:40:34.832 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:40:34.831+0000 7faa49a7c740 1 -- start start 2024-08-19T01:40:34.833 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:40:34.831+0000 7faa49a7c740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560229ff2660 0x560229ff2a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:40:34.833 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:40:34.831+0000 7faa49a7c740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560229ff2fa0 0x560229ff7a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:40:34.833 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:40:34.831+0000 7faa49a7c740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560229ff8030 0x560229ffa4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:40:34.833 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:40:34.831+0000 7faa49a7c740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x560229f56890 con 0x560229ff2660 2024-08-19T01:40:34.833 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:40:34.831+0000 7faa49a7c740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x560229eeed90 con 0x560229ff8030 2024-08-19T01:40:34.833 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:40:34.831+0000 7faa49a7c740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x560229f52850 con 0x560229ff2fa0 2024-08-19T01:40:34.833 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:40:34.832+0000 7faa3fe00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560229ff2660 0x560229ff2a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:40:34.834 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:40:34.832+0000 7faa3fe00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560229ff2660 0x560229ff2a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:42952/0 (socket says 172.21.15.83:42952) 2024-08-19T01:40:34.834 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:40:34.832+0000 7faa3fe00640 1 -- 172.21.15.83:0/3378932853 learned_addr learned my addr 172.21.15.83:0/3378932853 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-08-19T01:40:34.834 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:40:34.833+0000 7faa3fe00640 1 -- 172.21.15.83:0/3378932853 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560229ff2fa0 msgr2=0x560229ff7a60 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-08-19T01:40:34.835 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:40:34.833+0000 7faa3fe00640 1 --2- 172.21.15.83:0/3378932853 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560229ff2fa0 0x560229ff7a60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:40:34.835 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:40:34.833+0000 7faa3fe00640 1 -- 172.21.15.83:0/3378932853 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560229ff8030 msgr2=0x560229ffa4f0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-08-19T01:40:34.835 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:40:34.833+0000 7faa3fe00640 1 --2- 172.21.15.83:0/3378932853 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560229ff8030 0x560229ffa4f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:40:34.835 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:40:34.833+0000 7faa3fe00640 1 -- 172.21.15.83:0/3378932853 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560229c6a210 con 0x560229ff2660 2024-08-19T01:40:34.846 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:40:34.834+0000 7faa3fe00640 1 --2- 172.21.15.83:0/3378932853 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560229ff2660 0x560229ff2a60 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7faa2c00cef0 tx=0x7faa2c00e720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b51fcc00a098009f server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:40:34.860 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:40:34.858+0000 7faa35200640 1 -- 172.21.15.83:0/3378932853 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa2c01a630 con 0x560229ff2660 2024-08-19T01:40:34.860 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:40:34.858+0000 7faa35200640 1 -- 172.21.15.83:0/3378932853 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7faa2c01acb0 con 0x560229ff2660 2024-08-19T01:40:34.860 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:40:34.858+0000 7faa35200640 1 -- 172.21.15.83:0/3378932853 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa2c010dd0 con 0x560229ff2660 2024-08-19T01:40:34.861 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:40:34.860+0000 7faa49a7c740 1 -- 172.21.15.83:0/3378932853 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560229ff2660 msgr2=0x560229ff2a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:40:34.862 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:40:34.860+0000 7faa49a7c740 1 --2- 172.21.15.83:0/3378932853 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560229ff2660 0x560229ff2a60 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7faa2c00cef0 tx=0x7faa2c00e720 comp rx=0 tx=0).stop 2024-08-19T01:40:34.862 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:40:34.860+0000 7faa49a7c740 1 -- 172.21.15.83:0/3378932853 shutdown_connections 2024-08-19T01:40:34.862 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:40:34.860+0000 7faa49a7c740 1 --2- 172.21.15.83:0/3378932853 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560229ff8030 0x560229ffa4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:40:34.862 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:40:34.860+0000 7faa49a7c740 1 --2- 172.21.15.83:0/3378932853 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560229ff2fa0 0x560229ff7a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:40:34.862 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:40:34.860+0000 7faa49a7c740 1 --2- 172.21.15.83:0/3378932853 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560229ff2660 0x560229ff2a60 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:40:34.862 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:40:34.860+0000 7faa49a7c740 1 -- 172.21.15.83:0/3378932853 >> 172.21.15.83:0/3378932853 conn(0x560229fedcd0 msgr2=0x560229fee0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:40:34.876 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:40:34.865+0000 7faa49a7c740 1 -- 172.21.15.83:0/3378932853 shutdown_connections 2024-08-19T01:40:34.885 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:40:34.883+0000 7faa49a7c740 1 -- 172.21.15.83:0/3378932853 wait complete. 2024-08-19T01:40:34.885 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:40:34.883+0000 7faa49a7c740 1 -- start start 2024-08-19T01:40:34.885 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:40:34.884+0000 7faa49a7c740 1 Processor -- start 2024-08-19T01:40:34.887 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:40:34.884+0000 7faa49a7c740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560229ff2660 0x560229ffcaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:40:34.887 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:40:34.884+0000 7faa49a7c740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560229ff2fa0 0x560229ffcfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:40:34.887 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:40:34.884+0000 7faa49a7c740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560229ff8030 0x56022a003fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:40:34.887 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:40:34.884+0000 7faa49a7c740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x560229ffb0a0 con 0x560229ff8030 2024-08-19T01:40:34.887 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:40:34.884+0000 7faa49a7c740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x560229f11f70 con 0x560229ff2fa0 2024-08-19T01:40:34.888 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:40:34.884+0000 7faa49a7c740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x560229ffb220 con 0x560229ff2660 2024-08-19T01:40:34.888 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:40:34.884+0000 7faa49a7c740 10 client.0 _ll_register_callbacks cb 0x560229f4e8d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-19T01:40:34.888 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:40:34.885+0000 7faa3f400640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560229ff2fa0 0x560229ffcfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:40:34.888 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:40:34.885+0000 7faa3f400640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560229ff2fa0 0x560229ffcfe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.83:51290/0 (socket says 172.21.15.83:51290) 2024-08-19T01:40:34.888 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:40:34.885+0000 7faa3f400640 1 -- 172.21.15.83:0/1923560310 learned_addr learned my addr 172.21.15.83:0/1923560310 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-08-19T01:40:34.888 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:40:34.885+0000 7faa3fe00640 1 --2- 172.21.15.83:0/1923560310 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560229ff2660 0x560229ffcaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:40:34.888 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:40:34.886+0000 7faa3f400640 1 -- 172.21.15.83:0/1923560310 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560229ff2660 msgr2=0x560229ffcaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:40:34.888 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:40:34.886+0000 7faa3f400640 1 --2- 172.21.15.83:0/1923560310 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560229ff2660 0x560229ffcaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:40:34.888 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:40:34.886+0000 7faa3f400640 1 -- 172.21.15.83:0/1923560310 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560229ff8030 msgr2=0x56022a003fb0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-08-19T01:40:34.889 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:40:34.886+0000 7faa3f400640 1 --2- 172.21.15.83:0/1923560310 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560229ff8030 0x56022a003fb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:40:34.889 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:40:34.886+0000 7faa3f400640 1 -- 172.21.15.83:0/1923560310 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560229d36620 con 0x560229ff2fa0 2024-08-19T01:40:34.892 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:40:34.887+0000 7faa3f400640 1 --2- 172.21.15.83:0/1923560310 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560229ff2fa0 0x560229ffcfe0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7faa38009f00 tx=0x7faa380095c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f01236aab740d52c server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:40:34.904 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:40:34.893+0000 7faa29600640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-08-19T01:40:34.904 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:40:34.893+0000 7faa29600640 1 -- 172.21.15.83:0/1923560310 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa38050020 con 0x560229ff2fa0 2024-08-19T01:40:34.904 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:40:34.893+0000 7faa29600640 1 -- 172.21.15.83:0/1923560310 <== mon.2 v2:172.21.15.117:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8269+0+0 (secure 0 0 0) 0x7faa3802cd90 con 0x560229ff2fa0 2024-08-19T01:40:34.914 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:40:34.903+0000 7faa49a7c740 1 -- 172.21.15.83:0/1923560310 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7faa2c01acb0 con 0x560229ff2fa0 2024-08-19T01:40:34.915 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:40:34.914+0000 7faa29600640 1 -- 172.21.15.83:0/1923560310 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7faa3804c480 con 0x560229ff2fa0 2024-08-19T01:40:34.915 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:40:34.914+0000 7faa49a7c740 10 client.25817 fetch_fsmap learned FSMap version 27 2024-08-19T01:40:34.915 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:40:34.914+0000 7faa49a7c740 1 -- 172.21.15.83:0/1923560310 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x56022a0071a0 con 0x560229ff2fa0 2024-08-19T01:40:34.920 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:40:34.919+0000 7faa29600640 1 -- 172.21.15.83:0/1923560310 <== mon.2 v2:172.21.15.117:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7faa2c01acb0 con 0x560229ff2fa0 2024-08-19T01:40:34.920 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:40:34.919+0000 7faa29600640 1 -- 172.21.15.83:0/1923560310 <== mon.2 v2:172.21.15.117:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7faa3805d020 con 0x560229ff2fa0 2024-08-19T01:40:34.920 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:40:34.919+0000 7faa49a7c740 10 client.25817 fetch_fsmap finished waiting for FSMap version 27 2024-08-19T01:40:34.920 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:40:34.919+0000 7faa49a7c740 10 client.25817 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-08-19T01:40:34.921 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:40:34.919+0000 7faa49a7c740 20 client.25817 populate_metadata read hostname 'smithi083' 2024-08-19T01:40:34.921 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:40:34.919+0000 7faa49a7c740 1 --2- 172.21.15.83:0/1923560310 >> [v2:172.21.15.117:6838/1967595704,v1:172.21.15.117:6839/1967595704] conn(0x56022a008950 0x56022a00ad40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:40:34.921 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:40:34.919+0000 7faa49a7c740 4 client.25817 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-19T01:40:34.921 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:40:34.919+0000 7faa49a7c740 1 -- 172.21.15.83:0/1923560310 --> [v2:172.21.15.117:6838/1967595704,v1:172.21.15.117:6839/1967595704] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560229f63fc0 con 0x56022a008950 2024-08-19T01:40:34.922 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:40:34.920+0000 7faa3fe00640 1 --2- 172.21.15.83:0/1923560310 >> [v2:172.21.15.117:6838/1967595704,v1:172.21.15.117:6839/1967595704] conn(0x56022a008950 0x56022a00ad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:40:34.928 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:40:34.926+0000 7faa3fe00640 1 --2- 172.21.15.83:0/1923560310 >> [v2:172.21.15.117:6838/1967595704,v1:172.21.15.117:6839/1967595704] conn(0x56022a008950 0x56022a00ad40 crc :-1 s=READY pgs=105 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4256e035a1d97a9c server_cookie=9446092c32f2d000 in_seq=0 out_seq=0 2024-08-19T01:40:34.941 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:40:34.936+0000 7faa29600640 10 client.25817 ms_handle_connect on v2:172.21.15.117:6838/1967595704 2024-08-19T01:40:34.941 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:40:34.936+0000 7faa29600640 1 -- 172.21.15.83:0/1923560310 <== mds.0 v2:172.21.15.117:6838/1967595704 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x560229f63fc0 con 0x56022a008950 2024-08-19T01:40:34.941 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:40:34.936+0000 7faa29600640 10 client.25817 handle_command_reply: tid=0 2024-08-19T01:40:34.941 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:40:34.940+0000 7faa49a7c740 1 client.25817 shutdown 2024-08-19T01:40:34.957 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:40:34.956+0000 7faa49a7c740 1 -- 172.21.15.83:0/1923560310 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560229ff2fa0 msgr2=0x560229ffcfe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:40:34.958 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:40:34.956+0000 7faa49a7c740 1 --2- 172.21.15.83:0/1923560310 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560229ff2fa0 0x560229ffcfe0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7faa38009f00 tx=0x7faa380095c0 comp rx=0 tx=0).stop 2024-08-19T01:40:34.958 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:40:34.956+0000 7faa49a7c740 1 -- 172.21.15.83:0/1923560310 shutdown_connections 2024-08-19T01:40:34.958 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:40:34.956+0000 7faa49a7c740 1 --2- 172.21.15.83:0/1923560310 >> [v2:172.21.15.117:6838/1967595704,v1:172.21.15.117:6839/1967595704] conn(0x56022a008950 0x56022a00ad40 crc :-1 s=READY pgs=105 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:40:34.958 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:40:34.956+0000 7faa49a7c740 1 --2- 172.21.15.83:0/1923560310 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560229ff8030 0x56022a003fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:40:34.958 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:40:34.956+0000 7faa49a7c740 1 --2- 172.21.15.83:0/1923560310 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560229ff2fa0 0x560229ffcfe0 secure :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7faa38009f00 tx=0x7faa380095c0 comp rx=0 tx=0).stop 2024-08-19T01:40:34.958 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:40:34.956+0000 7faa49a7c740 1 --2- 172.21.15.83:0/1923560310 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560229ff2660 0x560229ffcaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:40:34.958 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:40:34.956+0000 7faa49a7c740 1 -- 172.21.15.83:0/1923560310 >> 172.21.15.83:0/1923560310 conn(0x560229fedcd0 msgr2=0x560229fef4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:40:34.958 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:40:34.956+0000 7faa29600640 0 client.25817 ms_handle_reset on v2:172.21.15.117:6838/1967595704 2024-08-19T01:40:34.959 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:40:34.957+0000 7faa49a7c740 1 -- 172.21.15.83:0/1923560310 shutdown_connections 2024-08-19T01:40:34.969 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:40:34.964+0000 7faa49a7c740 1 -- 172.21.15.83:0/1923560310 wait complete. 2024-08-19T01:40:34.974 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:40:34.973+0000 7faa49a7c740 20 client.25817 trim_cache size 0 max 16384 2024-08-19T01:40:35.314 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:40:35.306+0000 7faa49a7c740 1 Processor -- start 2024-08-19T01:40:35.314 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:40:35.309+0000 7faa49a7c740 1 -- start start 2024-08-19T01:40:35.314 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:40:35.309+0000 7faa49a7c740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560229ff2660 0x560229feeb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:40:35.314 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:40:35.309+0000 7faa49a7c740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560229ff8030 0x56022a003f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:40:35.314 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:40:35.309+0000 7faa49a7c740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560229ff3b50 0x560229fef0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:40:35.314 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:40:35.309+0000 7faa49a7c740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7faa401c2c60 con 0x560229ff2660 2024-08-19T01:40:35.315 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:40:35.309+0000 7faa49a7c740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7faa401c0530 con 0x560229ff3b50 2024-08-19T01:40:35.315 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:40:35.309+0000 7faa49a7c740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7faa401be380 con 0x560229ff8030 2024-08-19T01:40:35.315 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:40:35.309+0000 7faa44a00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560229ff3b50 0x560229fef0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:40:35.315 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:40:35.310+0000 7faa44a00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560229ff3b50 0x560229fef0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.83:51294/0 (socket says 172.21.15.83:51294) 2024-08-19T01:40:35.315 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:40:35.310+0000 7faa44a00640 1 -- 172.21.15.83:0/1143131383 learned_addr learned my addr 172.21.15.83:0/1143131383 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-08-19T01:40:35.315 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:40:35.310+0000 7faa3f400640 1 --2- 172.21.15.83:0/1143131383 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560229ff8030 0x56022a003f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:40:35.315 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:40:35.310+0000 7faa3fe00640 1 --2- 172.21.15.83:0/1143131383 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560229ff2660 0x560229feeb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:40:35.315 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:40:35.311+0000 7faa3fe00640 1 -- 172.21.15.83:0/1143131383 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560229ff8030 msgr2=0x56022a003f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:40:35.315 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:40:35.311+0000 7faa3fe00640 1 --2- 172.21.15.83:0/1143131383 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560229ff8030 0x56022a003f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:40:35.315 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:40:35.311+0000 7faa3fe00640 1 -- 172.21.15.83:0/1143131383 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560229ff3b50 msgr2=0x560229fef0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:40:35.315 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:40:35.311+0000 7faa3fe00640 1 --2- 172.21.15.83:0/1143131383 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560229ff3b50 0x560229fef0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:40:35.315 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:40:35.311+0000 7faa3fe00640 1 -- 172.21.15.83:0/1143131383 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560229fedaf0 con 0x560229ff2660 2024-08-19T01:40:35.315 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:40:35.312+0000 7faa3fe00640 1 --2- 172.21.15.83:0/1143131383 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560229ff2660 0x560229feeb80 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7faa2c00ec20 tx=0x7faa2c00c590 comp rx=0 tx=0).ready entity=mon.0 client_cookie=70a1fcdaf338e415 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:40:35.324 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:40:35.322+0000 7faa2be00640 1 -- 172.21.15.83:0/1143131383 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa2c01ae60 con 0x560229ff2660 2024-08-19T01:40:35.324 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:40:35.322+0000 7faa2be00640 1 -- 172.21.15.83:0/1143131383 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7faa2c005520 con 0x560229ff2660 2024-08-19T01:40:35.324 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:40:35.322+0000 7faa2be00640 1 -- 172.21.15.83:0/1143131383 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa2c004b00 con 0x560229ff2660 2024-08-19T01:40:35.339 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:40:35.338+0000 7faa49a7c740 1 -- 172.21.15.83:0/1143131383 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560229ff2660 msgr2=0x560229feeb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:40:35.339 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:40:35.338+0000 7faa49a7c740 1 --2- 172.21.15.83:0/1143131383 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560229ff2660 0x560229feeb80 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7faa2c00ec20 tx=0x7faa2c00c590 comp rx=0 tx=0).stop 2024-08-19T01:40:35.346 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:40:35.344+0000 7faa49a7c740 1 -- 172.21.15.83:0/1143131383 shutdown_connections 2024-08-19T01:40:35.346 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:40:35.344+0000 7faa49a7c740 1 --2- 172.21.15.83:0/1143131383 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560229ff3b50 0x560229fef0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:40:35.346 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:40:35.344+0000 7faa49a7c740 1 --2- 172.21.15.83:0/1143131383 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560229ff8030 0x56022a003f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:40:35.346 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:40:35.344+0000 7faa49a7c740 1 --2- 172.21.15.83:0/1143131383 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560229ff2660 0x560229feeb80 secure :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7faa2c00ec20 tx=0x7faa2c00c590 comp rx=0 tx=0).stop 2024-08-19T01:40:35.346 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:40:35.344+0000 7faa49a7c740 1 -- 172.21.15.83:0/1143131383 >> 172.21.15.83:0/1143131383 conn(0x560229fedcd0 msgr2=0x560229fecf40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:40:35.387 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:40:35.371+0000 7faa49a7c740 1 -- 172.21.15.83:0/1143131383 shutdown_connections 2024-08-19T01:40:35.387 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:40:35.377+0000 7faa49a7c740 1 -- 172.21.15.83:0/1143131383 wait complete. 2024-08-19T01:40:35.387 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:40:35.378+0000 7faa49a7c740 1 -- start start 2024-08-19T01:40:35.387 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:40:35.378+0000 7faa49a7c740 1 Processor -- start 2024-08-19T01:40:35.391 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:40:35.386+0000 7faa49a7c740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560229ff3b50 0x56022a001630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:40:35.391 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:40:35.386+0000 7faa49a7c740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560229ff8030 0x56022a001b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:40:35.391 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:40:35.386+0000 7faa49a7c740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56022a002b80 0x56022a002fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:40:35.391 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:40:35.386+0000 7faa49a7c740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7faa401be9f0 con 0x560229ff8030 2024-08-19T01:40:35.391 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:40:35.386+0000 7faa49a7c740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7faa401bfe60 con 0x560229ff3b50 2024-08-19T01:40:35.391 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:40:35.386+0000 7faa49a7c740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7faa401bf0c0 con 0x56022a002b80 2024-08-19T01:40:35.391 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:40:35.386+0000 7faa49a7c740 10 client.0 _ll_register_callbacks cb 0x7faa38043070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-19T01:40:35.392 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:40:35.387+0000 7faa3f400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560229ff8030 0x56022a001b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:40:35.392 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:40:35.387+0000 7faa3f400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560229ff8030 0x56022a001b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:42976/0 (socket says 172.21.15.83:42976) 2024-08-19T01:40:35.392 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:40:35.387+0000 7faa3f400640 1 -- 172.21.15.83:0/3594311402 learned_addr learned my addr 172.21.15.83:0/3594311402 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-08-19T01:40:35.392 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:40:35.388+0000 7faa3f400640 1 -- 172.21.15.83:0/3594311402 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56022a002b80 msgr2=0x56022a002fe0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-08-19T01:40:35.392 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:40:35.388+0000 7faa3f400640 1 --2- 172.21.15.83:0/3594311402 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56022a002b80 0x56022a002fe0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:40:35.392 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:40:35.388+0000 7faa3f400640 1 -- 172.21.15.83:0/3594311402 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560229ff3b50 msgr2=0x56022a001630 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-08-19T01:40:35.392 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:40:35.388+0000 7faa3f400640 1 --2- 172.21.15.83:0/3594311402 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560229ff3b50 0x56022a001630 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:40:35.392 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:40:35.388+0000 7faa3f400640 1 -- 172.21.15.83:0/3594311402 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56022a003570 con 0x560229ff8030 2024-08-19T01:40:35.392 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:40:35.388+0000 7faa3f400640 1 --2- 172.21.15.83:0/3594311402 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560229ff8030 0x56022a001b70 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7faa38009e50 tx=0x7faa3802ce20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7779c12cd70da0ce server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:40:35.404 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:40:35.403+0000 7faa29600640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-08-19T01:40:35.404 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:40:35.403+0000 7faa29600640 1 -- 172.21.15.83:0/3594311402 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa38050040 con 0x560229ff8030 2024-08-19T01:40:35.404 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:40:35.403+0000 7faa29600640 1 -- 172.21.15.83:0/3594311402 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8269+0+0 (secure 0 0 0) 0x7faa3804c820 con 0x560229ff8030 2024-08-19T01:40:35.414 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:40:35.413+0000 7faa49a7c740 1 -- 172.21.15.83:0/3594311402 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7faa2c005520 con 0x560229ff8030 2024-08-19T01:40:35.430 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:40:35.429+0000 7faa29600640 1 -- 172.21.15.83:0/3594311402 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7faa3806b050 con 0x560229ff8030 2024-08-19T01:40:35.431 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:40:35.429+0000 7faa49a7c740 10 client.15783 fetch_fsmap learned FSMap version 27 2024-08-19T01:40:35.432 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:40:35.430+0000 7faa49a7c740 1 -- 172.21.15.83:0/3594311402 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x56022a073a80 con 0x560229ff8030 2024-08-19T01:40:35.434 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:40:35.433+0000 7faa29600640 1 -- 172.21.15.83:0/3594311402 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7faa2c005520 con 0x560229ff8030 2024-08-19T01:40:35.434 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:40:35.433+0000 7faa29600640 1 -- 172.21.15.83:0/3594311402 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7faa3805d040 con 0x560229ff8030 2024-08-19T01:40:35.442 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:40:35.433+0000 7faa49a7c740 10 client.15783 fetch_fsmap finished waiting for FSMap version 27 2024-08-19T01:40:35.442 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:40:35.433+0000 7faa49a7c740 10 client.15783 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-08-19T01:40:35.442 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:40:35.433+0000 7faa49a7c740 20 client.15783 populate_metadata read hostname 'smithi083' 2024-08-19T01:40:35.442 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:40:35.433+0000 7faa49a7c740 1 --2- 172.21.15.83:0/3594311402 >> [v2:172.21.15.117:6838/1967595704,v1:172.21.15.117:6839/1967595704] conn(0x56022a074460 0x56022a076880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:40:35.442 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:40:35.433+0000 7faa49a7c740 4 client.15783 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-08-19T01:40:35.442 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:40:35.433+0000 7faa49a7c740 1 -- 172.21.15.83:0/3594311402 --> [v2:172.21.15.117:6838/1967595704,v1:172.21.15.117:6839/1967595704] -- command(tid 0: {"prefix": "flush journal"}) -- 0x560229fe94e0 con 0x56022a074460 2024-08-19T01:40:35.442 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:40:35.437+0000 7faa3fe00640 1 --2- 172.21.15.83:0/3594311402 >> [v2:172.21.15.117:6838/1967595704,v1:172.21.15.117:6839/1967595704] conn(0x56022a074460 0x56022a076880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:40:35.453 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:40:35.449+0000 7faa3fe00640 1 --2- 172.21.15.83:0/3594311402 >> [v2:172.21.15.117:6838/1967595704,v1:172.21.15.117:6839/1967595704] conn(0x56022a074460 0x56022a076880 crc :-1 s=READY pgs=106 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=36a5c71720cd0d00 server_cookie=491a191ffc67f4d5 in_seq=0 out_seq=0 2024-08-19T01:40:35.461 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:40:35.460+0000 7faa29600640 10 client.15783 ms_handle_connect on v2:172.21.15.117:6838/1967595704 2024-08-19T01:40:35.694 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:40:35 smithi083 ceph-mon[17120]: pgmap v511: 97 pgs: 97 active+clean; 567 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 2.7 MiB/s wr, 228 op/s 2024-08-19T01:40:35.952 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:40:35 smithi117 ceph-mon[22016]: pgmap v511: 97 pgs: 97 active+clean; 567 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 2.7 MiB/s wr, 228 op/s 2024-08-19T01:40:35.965 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:40:35 smithi167 ceph-mon[21875]: pgmap v511: 97 pgs: 97 active+clean; 567 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 2.7 MiB/s wr, 228 op/s 2024-08-19T01:40:37.952 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:40:37 smithi117 ceph-mon[22016]: pgmap v512: 97 pgs: 97 active+clean; 566 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.9 MiB/s wr, 167 op/s 2024-08-19T01:40:37.965 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:40:37 smithi167 ceph-mon[21875]: pgmap v512: 97 pgs: 97 active+clean; 566 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.9 MiB/s wr, 167 op/s 2024-08-19T01:40:38.154 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:40:37 smithi083 ceph-mon[17120]: pgmap v512: 97 pgs: 97 active+clean; 566 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.9 MiB/s wr, 167 op/s 2024-08-19T01:40:38.965 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:40:38 smithi167 ceph-mon[21875]: pgmap v513: 97 pgs: 97 active+clean; 564 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 2.1 MiB/s wr, 520 op/s 2024-08-19T01:40:39.147 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:40:38 smithi083 ceph-mon[17120]: pgmap v513: 97 pgs: 97 active+clean; 564 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 2.1 MiB/s wr, 520 op/s 2024-08-19T01:40:39.202 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:40:38 smithi117 ceph-mon[22016]: pgmap v513: 97 pgs: 97 active+clean; 564 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 2.1 MiB/s wr, 520 op/s 2024-08-19T01:40:40.728 INFO:tasks.workunit.client.0.smithi083.stdout: 193 553611 575 359643 447 328544 1337 2024-08-19T01:40:41.904 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:40:41 smithi083 ceph-mon[17120]: pgmap v514: 97 pgs: 97 active+clean; 573 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.6 MiB/s wr, 955 op/s 2024-08-19T01:40:41.952 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:40:41 smithi117 ceph-mon[22016]: pgmap v514: 97 pgs: 97 active+clean; 573 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.6 MiB/s wr, 955 op/s 2024-08-19T01:40:41.965 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:40:41 smithi167 ceph-mon[21875]: pgmap v514: 97 pgs: 97 active+clean; 573 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.6 MiB/s wr, 955 op/s 2024-08-19T01:40:42.413 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/bad8780e-5dc9-11ef-bccf-c7b262605968/mon.a/config 2024-08-19T01:40:43.197 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:40:43.195+0000 7faa29600640 1 -- 172.21.15.83:0/3594311402 <== mds.0 v2:172.21.15.117:6838/1967595704 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x560229fe94e0 con 0x56022a074460 2024-08-19T01:40:43.197 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:40:43.195+0000 7faa29600640 10 client.15783 handle_command_reply: tid=0 2024-08-19T01:40:43.197 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:40:43.196+0000 7faa49a7c740 1 client.15783 shutdown 2024-08-19T01:40:43.198 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:40:43.196+0000 7faa49a7c740 1 -- 172.21.15.83:0/3594311402 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560229ff8030 msgr2=0x56022a001b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:40:43.198 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:40:43.196+0000 7faa49a7c740 1 --2- 172.21.15.83:0/3594311402 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560229ff8030 0x56022a001b70 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7faa38009e50 tx=0x7faa3802ce20 comp rx=0 tx=0).stop 2024-08-19T01:40:43.202 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:40:43.200+0000 7faa49a7c740 1 -- 172.21.15.83:0/3594311402 shutdown_connections 2024-08-19T01:40:43.202 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:40:43.200+0000 7faa49a7c740 1 --2- 172.21.15.83:0/3594311402 >> [v2:172.21.15.117:6838/1967595704,v1:172.21.15.117:6839/1967595704] conn(0x56022a074460 0x56022a076880 crc :-1 s=READY pgs=106 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:40:43.202 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:40:43.200+0000 7faa49a7c740 1 --2- 172.21.15.83:0/3594311402 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56022a002b80 0x56022a002fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:40:43.202 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:40:43.200+0000 7faa49a7c740 1 --2- 172.21.15.83:0/3594311402 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560229ff8030 0x56022a001b70 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:40:43.202 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:40:43.200+0000 7faa49a7c740 1 --2- 172.21.15.83:0/3594311402 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560229ff3b50 0x56022a001630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:40:43.202 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:40:43.201+0000 7faa49a7c740 1 -- 172.21.15.83:0/3594311402 >> 172.21.15.83:0/3594311402 conn(0x560229fedcd0 msgr2=0x560229ffb5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:40:43.203 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:40:43.201+0000 7faa29600640 0 client.15783 ms_handle_reset on v2:172.21.15.117:6838/1967595704 2024-08-19T01:40:43.203 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:40:43.201+0000 7faa49a7c740 1 -- 172.21.15.83:0/3594311402 shutdown_connections 2024-08-19T01:40:43.203 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:40:43.202+0000 7faa49a7c740 1 -- 172.21.15.83:0/3594311402 wait complete. 2024-08-19T01:40:43.203 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:40:43.202+0000 7faa49a7c740 20 client.15783 trim_cache size 0 max 16384 2024-08-19T01:40:43.204 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stdout:{ 2024-08-19T01:40:43.204 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stdout: "message": "", 2024-08-19T01:40:43.204 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stdout: "return_code": 0 2024-08-19T01:40:43.204 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stdout:} 2024-08-19T01:40:43.205 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:40:43.204+0000 7faa46a00640 1 -- 172.21.15.83:0/329605463 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7faa240a2d90 msgr2=0x7faa240a5250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:40:43.205 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:40:43.204+0000 7faa46a00640 1 --2- 172.21.15.83:0/329605463 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7faa240a2d90 0x7faa240a5250 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7faa38034a20 tx=0x7faa38000f40 comp rx=0 tx=0).stop 2024-08-19T01:40:43.206 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:40:43.205+0000 7faa46a00640 1 -- 172.21.15.83:0/329605463 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7faa4010dbc0 msgr2=0x7faa401b6910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:40:43.206 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:40:43.205+0000 7faa46a00640 1 --2- 172.21.15.83:0/329605463 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7faa4010dbc0 0x7faa401b6910 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7faa3000b3e0 tx=0x7faa3000b8a0 comp rx=0 tx=0).stop 2024-08-19T01:40:43.206 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:40:43.205+0000 7faa46a00640 1 -- 172.21.15.83:0/329605463 shutdown_connections 2024-08-19T01:40:43.207 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:40:43.205+0000 7faa46a00640 1 --2- 172.21.15.83:0/329605463 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7faa240a2d90 0x7faa240a5250 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:40:43.207 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:40:43.205+0000 7faa46a00640 1 --2- 172.21.15.83:0/329605463 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7faa4010dbc0 0x7faa401b6910 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:40:43.207 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:40:43.205+0000 7faa46a00640 1 --2- 172.21.15.83:0/329605463 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7faa4010b210 0x7faa40115b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:40:43.207 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:40:43.205+0000 7faa46a00640 1 --2- 172.21.15.83:0/329605463 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7faa40072ab0 0x7faa40115630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:40:43.207 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:40:43.205+0000 7faa46a00640 1 -- 172.21.15.83:0/329605463 >> 172.21.15.83:0/329605463 conn(0x7faa4006e880 msgr2=0x7faa4010ee30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:40:43.211 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:40:43.208+0000 7faa46a00640 1 -- 172.21.15.83:0/329605463 shutdown_connections 2024-08-19T01:40:43.211 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:40:43.208+0000 7faa46a00640 1 -- 172.21.15.83:0/329605463 wait complete. 2024-08-19T01:40:43.906 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:40:43 smithi083 ceph-mon[17120]: pgmap v515: 97 pgs: 97 active+clean; 573 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.3 MiB/s wr, 924 op/s 2024-08-19T01:40:43.952 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:40:43 smithi117 ceph-mon[22016]: pgmap v515: 97 pgs: 97 active+clean; 573 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.3 MiB/s wr, 924 op/s 2024-08-19T01:40:43.965 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:40:43 smithi167 ceph-mon[21875]: pgmap v515: 97 pgs: 97 active+clean; 573 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.3 MiB/s wr, 924 op/s 2024-08-19T01:40:45.012 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:40:45.003+0000 7f02ed800640 1 -- 172.21.15.83:0/2879221888 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f02e8101710 msgr2=0x7f02e8101b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:40:45.012 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:40:45.003+0000 7f02ed800640 1 --2- 172.21.15.83:0/2879221888 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f02e8101710 0x7f02e8101b70 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f02d4009930 tx=0x7f02d402a480 comp rx=0 tx=0).stop 2024-08-19T01:40:45.012 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:40:45.003+0000 7f02ed800640 1 -- 172.21.15.83:0/2879221888 shutdown_connections 2024-08-19T01:40:45.013 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:40:45.003+0000 7f02ed800640 1 --2- 172.21.15.83:0/2879221888 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f02e81020b0 0x7f02e810e6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:40:45.013 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:40:45.003+0000 7f02ed800640 1 --2- 172.21.15.83:0/2879221888 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f02e8101710 0x7f02e8101b70 secure :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f02d4009930 tx=0x7f02d402a480 comp rx=0 tx=0).stop 2024-08-19T01:40:45.013 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:40:45.003+0000 7f02ed800640 1 --2- 172.21.15.83:0/2879221888 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f02e8107820 0x7f02e8107c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:40:45.013 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:40:45.003+0000 7f02ed800640 1 -- 172.21.15.83:0/2879221888 >> 172.21.15.83:0/2879221888 conn(0x7f02e80fd430 msgr2=0x7f02e80ff850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:40:45.013 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:40:45.004+0000 7f02ed800640 1 -- 172.21.15.83:0/2879221888 shutdown_connections 2024-08-19T01:40:45.013 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:40:45.004+0000 7f02ed800640 1 -- 172.21.15.83:0/2879221888 wait complete. 2024-08-19T01:40:45.013 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:40:45.006+0000 7f02ed800640 1 Processor -- start 2024-08-19T01:40:45.013 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:40:45.006+0000 7f02ed800640 1 -- start start 2024-08-19T01:40:45.013 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:40:45.006+0000 7f02ed800640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f02e81020b0 0x7f02e819eea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:40:45.013 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:40:45.006+0000 7f02ed800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f02e8107820 0x7f02e810b860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:40:45.013 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:40:45.006+0000 7f02ed800640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f02e810bda0 0x7f02e810c250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:40:45.013 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:40:45.007+0000 7f02ed800640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f02e8111c70 con 0x7f02e8107820 2024-08-19T01:40:45.014 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:40:45.007+0000 7f02ed800640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f02e8111af0 con 0x7f02e81020b0 2024-08-19T01:40:45.014 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:40:45.007+0000 7f02ed800640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f02e8111df0 con 0x7f02e810bda0 2024-08-19T01:40:45.014 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:40:45.007+0000 7f02e7e00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f02e81020b0 0x7f02e819eea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:40:45.014 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:40:45.007+0000 7f02e7e00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f02e81020b0 0x7f02e819eea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.83:54564/0 (socket says 172.21.15.83:54564) 2024-08-19T01:40:45.014 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:40:45.007+0000 7f02e7e00640 1 -- 172.21.15.83:0/2804550265 learned_addr learned my addr 172.21.15.83:0/2804550265 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-08-19T01:40:45.014 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:40:45.007+0000 7f02ece00640 1 --2- 172.21.15.83:0/2804550265 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f02e810bda0 0x7f02e810c250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:40:45.014 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:40:45.007+0000 7f02ece00640 1 -- 172.21.15.83:0/2804550265 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f02e81020b0 msgr2=0x7f02e819eea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:40:45.014 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:40:45.007+0000 7f02ece00640 1 --2- 172.21.15.83:0/2804550265 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f02e81020b0 0x7f02e819eea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:40:45.014 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:40:45.007+0000 7f02ece00640 1 -- 172.21.15.83:0/2804550265 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f02e8107820 msgr2=0x7f02e810b860 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-08-19T01:40:45.020 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:40:45.011+0000 7f02e7400640 1 --2- 172.21.15.83:0/2804550265 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f02e8107820 0x7f02e810b860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:40:45.020 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:40:45.011+0000 7f02ece00640 1 --2- 172.21.15.83:0/2804550265 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f02e8107820 0x7f02e810b860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:40:45.020 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:40:45.012+0000 7f02ece00640 1 -- 172.21.15.83:0/2804550265 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f02e819a900 con 0x7f02e810bda0 2024-08-19T01:40:45.020 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:40:45.012+0000 7f02e7400640 1 --2- 172.21.15.83:0/2804550265 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f02e8107820 0x7f02e810b860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-19T01:40:45.020 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:40:45.013+0000 7f02ece00640 1 --2- 172.21.15.83:0/2804550265 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f02e810bda0 0x7f02e810c250 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f02e0009fd0 tx=0x7f02e000ef10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:40:45.020 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:40:45.014+0000 7f02e4c00640 1 -- 172.21.15.83:0/2804550265 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02e000ce20 con 0x7f02e810bda0 2024-08-19T01:40:45.020 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:40:45.014+0000 7f02ed800640 1 -- 172.21.15.83:0/2804550265 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f02e819abf0 con 0x7f02e810bda0 2024-08-19T01:40:45.021 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:40:45.014+0000 7f02ed800640 1 -- 172.21.15.83:0/2804550265 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f02e819b180 con 0x7f02e810bda0 2024-08-19T01:40:45.021 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:40:45.015+0000 7f02e4c00640 1 -- 172.21.15.83:0/2804550265 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f02e0005020 con 0x7f02e810bda0 2024-08-19T01:40:45.021 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:40:45.015+0000 7f02e4c00640 1 -- 172.21.15.83:0/2804550265 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02e000ce20 con 0x7f02e810bda0 2024-08-19T01:40:45.024 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:40:45.020+0000 7f02ed800640 1 -- 172.21.15.83:0/2804550265 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f02c8005290 con 0x7f02e810bda0 2024-08-19T01:40:45.031 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:40:45.027+0000 7f02e4c00640 1 -- 172.21.15.83:0/2804550265 <== mon.1 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7f02e0004a40 con 0x7f02e810bda0 2024-08-19T01:40:45.064 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:40:45.033+0000 7f02e4c00640 1 --2- 172.21.15.83:0/2804550265 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f02cc0a2ed0 0x7f02cc0a5390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:40:45.064 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:40:45.034+0000 7f02e4c00640 1 -- 172.21.15.83:0/2804550265 <== mon.1 v2:172.21.15.167:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8269+0+0 (secure 0 0 0) 0x7f02e00cf230 con 0x7f02e810bda0 2024-08-19T01:40:45.064 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:40:45.035+0000 7f02e7e00640 1 --2- 172.21.15.83:0/2804550265 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f02cc0a2ed0 0x7f02cc0a5390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:40:45.064 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:40:45.038+0000 7f02e4c00640 1 -- 172.21.15.83:0/2804550265 <== mon.1 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7f02e009c110 con 0x7f02e810bda0 2024-08-19T01:40:45.064 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:40:45.040+0000 7f02e7e00640 1 --2- 172.21.15.83:0/2804550265 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f02cc0a2ed0 0x7f02cc0a5390 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f02e81976c0 tx=0x7f02d800a4b0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:40:45.487 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:40:45.474+0000 7f02ed800640 1 -- 172.21.15.83:0/2804550265 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f02c8005580 con 0x7f02e810bda0 2024-08-19T01:40:45.488 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:40:45.476+0000 7f02e4c00640 1 -- 172.21.15.83:0/2804550265 <== mon.1 v2:172.21.15.167:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+11923 (secure 0 0 0) 0x7f02e009ba10 con 0x7f02e810bda0 2024-08-19T01:40:45.489 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-08-19T01:40:45.491 INFO:teuthology.orchestra.run.smithi083.stderr:dumped fsmap epoch 27 2024-08-19T01:40:45.492 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":27,"btime":"2024-08-19T01:38:45:565146+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14421,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6835/988556067","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":988556067},{"type":"v1","addr":"172.21.15.83:6835","nonce":988556067}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14433,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6837/869702632","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6836","nonce":869702632},{"type":"v1","addr":"172.21.15.83:6837","nonce":869702632}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14475,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6841/1795975999","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6840","nonce":1795975999},{"type":"v1","addr":"172.21.15.83:6841","nonce":1795975999}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24430,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6833/2505076046","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6832","nonce":2505076046},{"type":"v1","addr":"172.21.15.167:6833","nonce":2505076046}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24449,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6833/3557520478","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":3557520478},{"type":"v1","addr":"172.21.15.117:6833","nonce":3557520478}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24466,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6835/3482045966","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":3482045966},{"type":"v1","addr":"172.21.15.167:6835","nonce":3482045966}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-08-19T01:32:42.054769+0000","modified":"2024-08-19T01:38:45.112142+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24545,"mds_1":24527,"mds_2":24509},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24478":{"gid":24478,"name":"c","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.167:6837/1762130408","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6836","nonce":1762130408},{"type":"v1","addr":"172.21.15.167:6837","nonce":1762130408}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24485":{"gid":24485,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.117:6835/1024213994","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":1024213994},{"type":"v1","addr":"172.21.15.117:6835","nonce":1024213994}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24496":{"gid":24496,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.167:6839/3507387682","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6838","nonce":3507387682},{"type":"v1","addr":"172.21.15.167:6839","nonce":3507387682}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24509":{"gid":24509,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":10,"addr":"172.21.15.117:6837/3185224684","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":3185224684},{"type":"v1","addr":"172.21.15.117:6837","nonce":3185224684}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24527":{"gid":24527,"name":"g","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.83:6839/3239261845","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6838","nonce":3239261845},{"type":"v1","addr":"172.21.15.83:6839","nonce":3239261845}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24545":{"gid":24545,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.117:6839/1967595704","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6838","nonce":1967595704},{"type":"v1","addr":"172.21.15.117:6839","nonce":1967595704}]},"join_fscid":-1,"export_targets":[1,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24545,"qdb_cluster":[24545,24527,24509]},"id":1}]} 2024-08-19T01:40:45.493 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:40:45.483+0000 7f02ed800640 1 -- 172.21.15.83:0/2804550265 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f02cc0a2ed0 msgr2=0x7f02cc0a5390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:40:45.495 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:40:45.483+0000 7f02ed800640 1 --2- 172.21.15.83:0/2804550265 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f02cc0a2ed0 0x7f02cc0a5390 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f02e81976c0 tx=0x7f02d800a4b0 comp rx=0 tx=0).stop 2024-08-19T01:40:45.495 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:40:45.483+0000 7f02ed800640 1 -- 172.21.15.83:0/2804550265 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f02e810bda0 msgr2=0x7f02e810c250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:40:45.495 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:40:45.483+0000 7f02ed800640 1 --2- 172.21.15.83:0/2804550265 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f02e810bda0 0x7f02e810c250 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f02e0009fd0 tx=0x7f02e000ef10 comp rx=0 tx=0).stop 2024-08-19T01:40:45.496 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:40:45.485+0000 7f02ed800640 1 -- 172.21.15.83:0/2804550265 shutdown_connections 2024-08-19T01:40:45.497 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:40:45.485+0000 7f02ed800640 1 --2- 172.21.15.83:0/2804550265 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f02cc0a2ed0 0x7f02cc0a5390 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:40:45.497 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:40:45.485+0000 7f02ed800640 1 --2- 172.21.15.83:0/2804550265 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f02e810bda0 0x7f02e810c250 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:40:45.497 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:40:45.485+0000 7f02ed800640 1 --2- 172.21.15.83:0/2804550265 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f02e8107820 0x7f02e810b860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:40:45.497 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:40:45.485+0000 7f02ed800640 1 --2- 172.21.15.83:0/2804550265 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f02e81020b0 0x7f02e819eea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:40:45.497 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:40:45.485+0000 7f02ed800640 1 -- 172.21.15.83:0/2804550265 >> 172.21.15.83:0/2804550265 conn(0x7f02e80fd430 msgr2=0x7f02e810cb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:40:45.498 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:40:45.485+0000 7f02ed800640 1 -- 172.21.15.83:0/2804550265 shutdown_connections 2024-08-19T01:40:45.498 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:40:45.485+0000 7f02ed800640 1 -- 172.21.15.83:0/2804550265 wait complete. 2024-08-19T01:40:45.734 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:40:45 smithi083 ceph-mon[17120]: pgmap v516: 97 pgs: 97 active+clean; 567 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 3.4 MiB/s wr, 1.35k op/s 2024-08-19T01:40:45.734 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:40:45 smithi083 ceph-mon[17120]: from='client.? 172.21.15.83:0/2804550265' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-19T01:40:45.952 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:40:45 smithi117 ceph-mon[22016]: pgmap v516: 97 pgs: 97 active+clean; 567 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 3.4 MiB/s wr, 1.35k op/s 2024-08-19T01:40:45.952 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:40:45 smithi117 ceph-mon[22016]: from='client.? 172.21.15.83:0/2804550265' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-19T01:40:45.965 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:40:45 smithi167 ceph-mon[21875]: pgmap v516: 97 pgs: 97 active+clean; 567 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 3.4 MiB/s wr, 1.35k op/s 2024-08-19T01:40:45.965 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:40:45 smithi167 ceph-mon[21875]: from='client.? 172.21.15.83:0/2804550265' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-19T01:40:46.752 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:40:46 smithi083 ceph-mon[17120]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-19T01:40:46.752 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:40:46 smithi083 ceph-mon[17120]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-19T01:40:46.752 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:40:46 smithi083 ceph-mon[17120]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' 2024-08-19T01:40:46.952 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:40:46 smithi117 ceph-mon[22016]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-19T01:40:46.952 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:40:46 smithi117 ceph-mon[22016]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-19T01:40:46.952 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:40:46 smithi117 ceph-mon[22016]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' 2024-08-19T01:40:46.965 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:40:46 smithi167 ceph-mon[21875]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-19T01:40:46.966 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:40:46 smithi167 ceph-mon[21875]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-19T01:40:46.966 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:40:46 smithi167 ceph-mon[21875]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' 2024-08-19T01:40:47.547 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833 shell --fsid bad8780e-5dc9-11ef-bccf-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-08-19T01:40:47.908 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:40:47 smithi083 ceph-mon[17120]: pgmap v517: 97 pgs: 97 active+clean; 574 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 3.9 MiB/s wr, 1.50k op/s 2024-08-19T01:40:47.952 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:40:47 smithi117 ceph-mon[22016]: pgmap v517: 97 pgs: 97 active+clean; 574 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 3.9 MiB/s wr, 1.50k op/s 2024-08-19T01:40:47.965 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:40:47 smithi167 ceph-mon[21875]: pgmap v517: 97 pgs: 97 active+clean; 574 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 3.9 MiB/s wr, 1.50k op/s 2024-08-19T01:40:49.328 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/bad8780e-5dc9-11ef-bccf-c7b262605968/mon.a/config 2024-08-19T01:40:49.861 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:40:49 smithi083 ceph-mon[17120]: pgmap v518: 97 pgs: 97 active+clean; 594 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 7.7 MiB/s wr, 1.77k op/s 2024-08-19T01:40:49.952 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:40:49 smithi117 ceph-mon[22016]: pgmap v518: 97 pgs: 97 active+clean; 594 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 7.7 MiB/s wr, 1.77k op/s 2024-08-19T01:40:49.965 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:40:49 smithi167 ceph-mon[21875]: pgmap v518: 97 pgs: 97 active+clean; 594 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 7.7 MiB/s wr, 1.77k op/s 2024-08-19T01:40:50.588 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:40:50.579+0000 7fdeac600640 1 -- 172.21.15.83:0/920112333 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fde9c0050e0 msgr2=0x7fde9c0a2420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:40:50.588 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:40:50.579+0000 7fdeac600640 1 --2- 172.21.15.83:0/920112333 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fde9c0050e0 0x7fde9c0a2420 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7fdea402db70 tx=0x7fdea4097160 comp rx=0 tx=0).stop 2024-08-19T01:40:50.588 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:40:50.583+0000 7fdeac600640 1 -- 172.21.15.83:0/920112333 shutdown_connections 2024-08-19T01:40:50.588 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:40:50.583+0000 7fdeac600640 1 --2- 172.21.15.83:0/920112333 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fde9c0050e0 0x7fde9c0a2420 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:40:50.588 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:40:50.583+0000 7fdeac600640 1 --2- 172.21.15.83:0/920112333 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fde9c004740 0x7fde9c004ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:40:50.588 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:40:50.583+0000 7fdeac600640 1 --2- 172.21.15.83:0/920112333 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fde9c09b600 0x7fde9c09b9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:40:50.588 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:40:50.583+0000 7fdeac600640 1 -- 172.21.15.83:0/920112333 >> 172.21.15.83:0/920112333 conn(0x7fde9c00a1b0 msgr2=0x7fde9c00a5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:40:50.588 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:40:50.583+0000 7fdeac600640 1 -- 172.21.15.83:0/920112333 shutdown_connections 2024-08-19T01:40:50.615 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:40:50.583+0000 7fdeac600640 1 -- 172.21.15.83:0/920112333 wait complete. 2024-08-19T01:40:50.616 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:40:50.585+0000 7fdeac600640 1 Processor -- start 2024-08-19T01:40:50.616 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:40:50.585+0000 7fdeac600640 1 -- start start 2024-08-19T01:40:50.616 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:40:50.585+0000 7fdeac600640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fde9c004740 0x7fde9c0a9b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:40:50.616 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:40:50.586+0000 7fdeac600640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fde9c0050e0 0x7fde9c0aa0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:40:50.616 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:40:50.586+0000 7fdeac600640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fde9c09b600 0x7fde9c0a4ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:40:50.616 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:40:50.586+0000 7fdeaa600640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fde9c09b600 0x7fde9c0a4ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:40:50.616 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:40:50.586+0000 7fdeaa600640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fde9c09b600 0x7fde9c0a4ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:58278/0 (socket says 172.21.15.83:58278) 2024-08-19T01:40:50.616 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:40:50.587+0000 7fdea9c00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fde9c004740 0x7fde9c0a9b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:40:50.617 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:40:50.587+0000 7fdea9200640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fde9c0050e0 0x7fde9c0aa0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:40:50.617 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:40:50.587+0000 7fdea9200640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fde9c0050e0 0x7fde9c0aa0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.83:54576/0 (socket says 172.21.15.83:54576) 2024-08-19T01:40:50.617 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:40:50.587+0000 7fdeaa600640 1 -- 172.21.15.83:0/2836564038 learned_addr learned my addr 172.21.15.83:0/2836564038 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-08-19T01:40:50.617 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:40:50.587+0000 7fdeac600640 1 -- 172.21.15.83:0/2836564038 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fde9c0a3720 con 0x7fde9c09b600 2024-08-19T01:40:50.617 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:40:50.587+0000 7fdeac600640 1 -- 172.21.15.83:0/2836564038 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fde9c0a35a0 con 0x7fde9c0050e0 2024-08-19T01:40:50.617 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:40:50.587+0000 7fdeac600640 1 -- 172.21.15.83:0/2836564038 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fde9c0a38a0 con 0x7fde9c004740 2024-08-19T01:40:50.617 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:40:50.588+0000 7fdea9200640 1 -- 172.21.15.83:0/2836564038 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fde9c004740 msgr2=0x7fde9c0a9b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:40:50.617 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:40:50.588+0000 7fdea9200640 1 --2- 172.21.15.83:0/2836564038 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fde9c004740 0x7fde9c0a9b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:40:50.617 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:40:50.588+0000 7fdea9200640 1 -- 172.21.15.83:0/2836564038 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fde9c09b600 msgr2=0x7fde9c0a4ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:40:50.617 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:40:50.588+0000 7fdea9200640 1 --2- 172.21.15.83:0/2836564038 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fde9c09b600 0x7fde9c0a4ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:40:50.617 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:40:50.588+0000 7fdea9200640 1 -- 172.21.15.83:0/2836564038 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fde9c0a5560 con 0x7fde9c0050e0 2024-08-19T01:40:50.618 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:40:50.588+0000 7fdeaa600640 1 --2- 172.21.15.83:0/2836564038 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fde9c09b600 0x7fde9c0a4ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-19T01:40:50.618 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:40:50.588+0000 7fdea9200640 1 --2- 172.21.15.83:0/2836564038 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fde9c0050e0 0x7fde9c0aa0c0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fde9000c970 tx=0x7fde9000ce30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:40:50.618 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:40:50.589+0000 7fdea2000640 1 -- 172.21.15.83:0/2836564038 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde90007bb0 con 0x7fde9c0050e0 2024-08-19T01:40:50.618 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:40:50.589+0000 7fdeac600640 1 -- 172.21.15.83:0/2836564038 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fde9c0a5850 con 0x7fde9c0050e0 2024-08-19T01:40:50.618 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:40:50.589+0000 7fdeac600640 1 -- 172.21.15.83:0/2836564038 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fde9c0ada60 con 0x7fde9c0050e0 2024-08-19T01:40:50.618 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:40:50.589+0000 7fdea2000640 1 -- 172.21.15.83:0/2836564038 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fde90007d50 con 0x7fde9c0050e0 2024-08-19T01:40:50.618 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:40:50.589+0000 7fdea2000640 1 -- 172.21.15.83:0/2836564038 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde90007bb0 con 0x7fde9c0050e0 2024-08-19T01:40:50.618 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:40:50.598+0000 7fdeaf05c740 1 Processor -- start 2024-08-19T01:40:50.618 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:40:50.599+0000 7fdeaf05c740 1 -- start start 2024-08-19T01:40:50.618 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:40:50.599+0000 7fdeaf05c740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558d636a26f0 0x558d636a2ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:40:50.618 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:40:50.599+0000 7fdeaf05c740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558d636a3010 0x558d636ad390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:40:50.618 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:40:50.599+0000 7fdeaf05c740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558d636ad8d0 0x558d636afcc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:40:50.619 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:40:50.599+0000 7fdeaa600640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558d636a26f0 0x558d636a2ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:40:50.619 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:40:50.599+0000 7fdeaa600640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558d636a26f0 0x558d636a2ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.83:54580/0 (socket says 172.21.15.83:54580) 2024-08-19T01:40:50.619 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:40:50.599+0000 7fdeaa600640 1 -- 172.21.15.83:0/2303177913 learned_addr learned my addr 172.21.15.83:0/2303177913 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-08-19T01:40:50.619 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:40:50.599+0000 7fdeaf05c740 1 -- 172.21.15.83:0/2303177913 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x558d63607bf0 con 0x558d636a3010 2024-08-19T01:40:50.619 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:40:50.599+0000 7fdeaf05c740 1 -- 172.21.15.83:0/2303177913 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x558d6359fb20 con 0x558d636a26f0 2024-08-19T01:40:50.619 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:40:50.599+0000 7fdeaf05c740 1 -- 172.21.15.83:0/2303177913 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x558d63603bb0 con 0x558d636ad8d0 2024-08-19T01:40:50.619 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:40:50.600+0000 7fdeaa600640 1 -- 172.21.15.83:0/2303177913 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558d636ad8d0 msgr2=0x558d636afcc0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-08-19T01:40:50.619 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:40:50.600+0000 7fdeaa600640 1 --2- 172.21.15.83:0/2303177913 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558d636ad8d0 0x558d636afcc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:40:50.619 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:40:50.600+0000 7fdeaa600640 1 -- 172.21.15.83:0/2303177913 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558d636a3010 msgr2=0x558d636ad390 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-08-19T01:40:50.619 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:40:50.600+0000 7fdea2000640 1 -- 172.21.15.83:0/2836564038 <== mon.2 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7fde90004ce0 con 0x7fde9c0050e0 2024-08-19T01:40:50.619 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:40:50.602+0000 7fdea2000640 1 --2- 172.21.15.83:0/2836564038 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7fde8c0a2e50 0x7fde8c0a5310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:40:50.620 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:40:50.603+0000 7fdea9c00640 1 --2- 172.21.15.83:0/2303177913 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558d636a3010 0x558d636ad390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:40:50.620 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:40:50.603+0000 7fdeaa600640 1 --2- 172.21.15.83:0/2303177913 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558d636a3010 0x558d636ad390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:40:50.620 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:40:50.603+0000 7fdea2000640 1 -- 172.21.15.83:0/2836564038 <== mon.2 v2:172.21.15.117:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8269+0+0 (secure 0 0 0) 0x7fde900994b0 con 0x7fde9c0050e0 2024-08-19T01:40:50.620 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:40:50.603+0000 7fdeaa600640 1 -- 172.21.15.83:0/2303177913 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558d635a67f0 con 0x558d636a26f0 2024-08-19T01:40:50.620 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:40:50.603+0000 7fdea9c00640 1 --2- 172.21.15.83:0/2303177913 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558d636a3010 0x558d636ad390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-19T01:40:50.620 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:40:50.603+0000 7fdea9200640 1 --2- 172.21.15.83:0/2836564038 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7fde8c0a2e50 0x7fde8c0a5310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:40:50.620 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:40:50.604+0000 7fdeaa600640 1 --2- 172.21.15.83:0/2303177913 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558d636a26f0 0x558d636a2ad0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fdea40634e0 tx=0x7fdea406bc10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=afb9d11661b7b572 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:40:50.620 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:40:50.604+0000 7fde96a00640 1 -- 172.21.15.83:0/2303177913 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdea40a5070 con 0x558d636a26f0 2024-08-19T01:40:50.620 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:40:50.604+0000 7fde96a00640 1 -- 172.21.15.83:0/2303177913 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdea409b070 con 0x558d636a26f0 2024-08-19T01:40:50.620 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:40:50.604+0000 7fde96a00640 1 -- 172.21.15.83:0/2303177913 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdea409f7f0 con 0x558d636a26f0 2024-08-19T01:40:50.620 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:40:50.607+0000 7fdea9200640 1 --2- 172.21.15.83:0/2836564038 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7fde8c0a2e50 0x7fde8c0a5310 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7fde90098860 tx=0x7fde900a08e0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:40:50.621 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:40:50.607+0000 7fdeaf05c740 1 -- 172.21.15.83:0/2303177913 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558d636a26f0 msgr2=0x558d636a2ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:40:50.621 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:40:50.607+0000 7fdeaf05c740 1 --2- 172.21.15.83:0/2303177913 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558d636a26f0 0x558d636a2ad0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fdea40634e0 tx=0x7fdea406bc10 comp rx=0 tx=0).stop 2024-08-19T01:40:50.621 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:40:50.608+0000 7fdeaf05c740 1 -- 172.21.15.83:0/2303177913 shutdown_connections 2024-08-19T01:40:50.621 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:40:50.608+0000 7fdeaf05c740 1 --2- 172.21.15.83:0/2303177913 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558d636ad8d0 0x558d636afcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:40:50.621 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:40:50.608+0000 7fdeaf05c740 1 --2- 172.21.15.83:0/2303177913 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558d636a3010 0x558d636ad390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:40:50.621 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:40:50.608+0000 7fdeaf05c740 1 --2- 172.21.15.83:0/2303177913 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558d636a26f0 0x558d636a2ad0 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:40:50.621 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:40:50.608+0000 7fdeaf05c740 1 -- 172.21.15.83:0/2303177913 >> 172.21.15.83:0/2303177913 conn(0x558d6369ec80 msgr2=0x558d6369f060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:40:50.621 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:40:50.609+0000 7fdeaf05c740 1 -- 172.21.15.83:0/2303177913 shutdown_connections 2024-08-19T01:40:50.621 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:40:50.609+0000 7fdeaf05c740 1 -- 172.21.15.83:0/2303177913 wait complete. 2024-08-19T01:40:50.621 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:40:50.610+0000 7fdeaf05c740 1 -- start start 2024-08-19T01:40:50.621 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:40:50.610+0000 7fdeaf05c740 1 Processor -- start 2024-08-19T01:40:50.622 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:40:50.615+0000 7fdeaf05c740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558d636a26f0 0x558d636aa7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:40:50.622 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:40:50.615+0000 7fdeaf05c740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558d636a3010 0x558d636aace0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:40:50.622 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:40:50.615+0000 7fdeaa600640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558d636a26f0 0x558d636aa7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:40:50.622 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:40:50.615+0000 7fdeaa600640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558d636a26f0 0x558d636aa7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.83:54594/0 (socket says 172.21.15.83:54594) 2024-08-19T01:40:50.622 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:40:50.617+0000 7fdeaf05c740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558d636ad8d0 0x558d636ab220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:40:50.622 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:40:50.617+0000 7fdeaf05c740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x558d636b04a0 con 0x558d636ad8d0 2024-08-19T01:40:50.623 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:40:50.617+0000 7fdeaf05c740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x558d635c2cf0 con 0x558d636a26f0 2024-08-19T01:40:50.623 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:40:50.617+0000 7fdeaf05c740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x558d636b0620 con 0x558d636a3010 2024-08-19T01:40:50.623 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:40:50.617+0000 7fdea9c00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558d636a3010 0x558d636aace0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:40:50.623 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:40:50.617+0000 7fdea9c00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558d636a3010 0x558d636aace0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.83:58064/0 (socket says 172.21.15.83:58064) 2024-08-19T01:40:50.623 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:40:50.617+0000 7fdea9c00640 1 -- 172.21.15.83:0/3154215371 learned_addr learned my addr 172.21.15.83:0/3154215371 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-08-19T01:40:50.624 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:40:50.617+0000 7fdeaf05c740 10 client.0 _ll_register_callbacks cb 0x558d635aee30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-19T01:40:50.624 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:40:50.618+0000 7fdeaa600640 1 -- 172.21.15.83:0/3154215371 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558d636a3010 msgr2=0x558d636aace0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:40:50.624 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:40:50.618+0000 7fdeaa600640 1 --2- 172.21.15.83:0/3154215371 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558d636a3010 0x558d636aace0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:40:50.624 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:40:50.618+0000 7fdeaa600640 1 -- 172.21.15.83:0/3154215371 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558d636ad8d0 msgr2=0x558d636ab220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:40:50.624 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:40:50.618+0000 7fdeaa600640 1 --2- 172.21.15.83:0/3154215371 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558d636ad8d0 0x558d636ab220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:40:50.624 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:40:50.618+0000 7fdeaa600640 1 -- 172.21.15.83:0/3154215371 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558d633793c0 con 0x558d636a26f0 2024-08-19T01:40:50.625 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:40:50.618+0000 7fdeaa600640 1 --2- 172.21.15.83:0/3154215371 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558d636a26f0 0x558d636aa7a0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fdea40651d0 tx=0x7fdea4067880 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4db72f6f343ad966 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:40:50.625 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:40:50.618+0000 7fde8b400640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-08-19T01:40:50.625 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:40:50.619+0000 7fde8b400640 1 -- 172.21.15.83:0/3154215371 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdea40a5070 con 0x558d636a26f0 2024-08-19T01:40:50.625 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:40:50.619+0000 7fdeaf05c740 1 -- 172.21.15.83:0/3154215371 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdea409b070 con 0x558d636a26f0 2024-08-19T01:40:50.625 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:40:50.619+0000 7fde8b400640 1 -- 172.21.15.83:0/3154215371 <== mon.2 v2:172.21.15.117:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8269+0+0 (secure 0 0 0) 0x7fdea409f570 con 0x558d636a26f0 2024-08-19T01:40:50.625 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:40:50.621+0000 7fde8b400640 1 -- 172.21.15.83:0/3154215371 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fdea409f770 con 0x558d636a26f0 2024-08-19T01:40:50.625 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:40:50.621+0000 7fdeaf05c740 10 client.25835 fetch_fsmap learned FSMap version 27 2024-08-19T01:40:50.625 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:40:50.621+0000 7fdeaf05c740 1 -- 172.21.15.83:0/3154215371 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x558d636b04a0 con 0x558d636a26f0 2024-08-19T01:40:50.625 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:40:50.623+0000 7fde8b400640 1 -- 172.21.15.83:0/3154215371 <== mon.2 v2:172.21.15.117:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdea409b070 con 0x558d636a26f0 2024-08-19T01:40:50.652 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:40:50.623+0000 7fde8b400640 1 -- 172.21.15.83:0/3154215371 <== mon.2 v2:172.21.15.117:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7fdea40b5a80 con 0x558d636a26f0 2024-08-19T01:40:50.653 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:40:50.623+0000 7fdeaf05c740 10 client.25835 fetch_fsmap finished waiting for FSMap version 27 2024-08-19T01:40:50.653 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:40:50.623+0000 7fdeaf05c740 10 client.25835 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-08-19T01:40:50.653 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:40:50.623+0000 7fdeaf05c740 20 client.25835 populate_metadata read hostname 'smithi083' 2024-08-19T01:40:50.653 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:40:50.623+0000 7fdeaf05c740 1 --2- 172.21.15.83:0/3154215371 >> [v2:172.21.15.117:6836/3185224684,v1:172.21.15.117:6837/3185224684] conn(0x558d636bd230 0x558d636bf620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:40:50.653 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:40:50.623+0000 7fdeaf05c740 4 client.25835 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-19T01:40:50.653 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:40:50.623+0000 7fdeaf05c740 1 -- 172.21.15.83:0/3154215371 --> [v2:172.21.15.117:6836/3185224684,v1:172.21.15.117:6837/3185224684] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558d63615160 con 0x558d636bd230 2024-08-19T01:40:50.653 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:40:50.623+0000 7fdea9c00640 1 --2- 172.21.15.83:0/3154215371 >> [v2:172.21.15.117:6836/3185224684,v1:172.21.15.117:6837/3185224684] conn(0x558d636bd230 0x558d636bf620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:40:50.653 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:40:50.625+0000 7fdea9c00640 1 --2- 172.21.15.83:0/3154215371 >> [v2:172.21.15.117:6836/3185224684,v1:172.21.15.117:6837/3185224684] conn(0x558d636bd230 0x558d636bf620 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=b1f334dbe45c9f51 server_cookie=df5218571033455b in_seq=0 out_seq=0 2024-08-19T01:40:50.654 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:40:50.625+0000 7fde8b400640 10 client.25835 ms_handle_connect on v2:172.21.15.117:6836/3185224684 2024-08-19T01:40:50.654 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:40:50.625+0000 7fde8b400640 1 -- 172.21.15.83:0/3154215371 <== mds.2 v2:172.21.15.117:6836/3185224684 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x558d63615160 con 0x558d636bd230 2024-08-19T01:40:50.654 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:40:50.626+0000 7fde8b400640 10 client.25835 handle_command_reply: tid=0 2024-08-19T01:40:50.654 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:40:50.626+0000 7fdeaf05c740 1 client.25835 shutdown 2024-08-19T01:40:50.654 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:40:50.626+0000 7fdeaf05c740 1 -- 172.21.15.83:0/3154215371 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558d636a26f0 msgr2=0x558d636aa7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:40:50.654 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:40:50.626+0000 7fdeaf05c740 1 --2- 172.21.15.83:0/3154215371 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558d636a26f0 0x558d636aa7a0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fdea40651d0 tx=0x7fdea4067880 comp rx=0 tx=0).stop 2024-08-19T01:40:50.654 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:40:50.627+0000 7fdeaf05c740 1 -- 172.21.15.83:0/3154215371 shutdown_connections 2024-08-19T01:40:50.654 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:40:50.627+0000 7fdeaf05c740 1 --2- 172.21.15.83:0/3154215371 >> [v2:172.21.15.117:6836/3185224684,v1:172.21.15.117:6837/3185224684] conn(0x558d636bd230 0x558d636bf620 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:40:50.654 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:40:50.627+0000 7fdeaf05c740 1 --2- 172.21.15.83:0/3154215371 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558d636ad8d0 0x558d636ab220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:40:50.655 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:40:50.627+0000 7fdeaf05c740 1 --2- 172.21.15.83:0/3154215371 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558d636a3010 0x558d636aace0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:40:50.655 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:40:50.627+0000 7fdeaf05c740 1 --2- 172.21.15.83:0/3154215371 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558d636a26f0 0x558d636aa7a0 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:40:50.655 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:40:50.627+0000 7fdeaf05c740 1 -- 172.21.15.83:0/3154215371 >> 172.21.15.83:0/3154215371 conn(0x558d6369ec80 msgr2=0x558d636a49b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:40:50.655 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:40:50.627+0000 7fde8b400640 0 client.25835 ms_handle_reset on v2:172.21.15.117:6836/3185224684 2024-08-19T01:40:50.655 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:40:50.629+0000 7fdeaf05c740 1 -- 172.21.15.83:0/3154215371 shutdown_connections 2024-08-19T01:40:50.655 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:40:50.629+0000 7fdeaf05c740 1 -- 172.21.15.83:0/3154215371 wait complete. 2024-08-19T01:40:50.655 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:40:50.629+0000 7fdeaf05c740 20 client.25835 trim_cache size 0 max 16384 2024-08-19T01:40:50.680 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:40:50.672+0000 7fdeaf05c740 1 Processor -- start 2024-08-19T01:40:50.680 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:40:50.673+0000 7fdeaf05c740 1 -- start start 2024-08-19T01:40:50.680 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:40:50.673+0000 7fdeaf05c740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558d636a26f0 0x558d636a51a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:40:50.680 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:40:50.673+0000 7fdeaf05c740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558d636a3010 0x558d636ac710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:40:50.680 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:40:50.673+0000 7fdeaf05c740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558d636ad8d0 0x558d636c4d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:40:50.680 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:40:50.673+0000 7fdeaf05c740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fde9c1527d0 con 0x558d636a26f0 2024-08-19T01:40:50.681 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:40:50.673+0000 7fdeaf05c740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fde9c1500a0 con 0x558d636a3010 2024-08-19T01:40:50.681 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:40:50.673+0000 7fdeaf05c740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fde9c14def0 con 0x558d636ad8d0 2024-08-19T01:40:50.681 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:40:50.674+0000 7fdea9c00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558d636a3010 0x558d636ac710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:40:50.681 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:40:50.674+0000 7fdea9c00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558d636a3010 0x558d636ac710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.83:54608/0 (socket says 172.21.15.83:54608) 2024-08-19T01:40:50.681 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:40:50.674+0000 7fdea9c00640 1 -- 172.21.15.83:0/1309358154 learned_addr learned my addr 172.21.15.83:0/1309358154 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-08-19T01:40:50.681 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:40:50.676+0000 7fdeaa600640 1 --2- 172.21.15.83:0/1309358154 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558d636a26f0 0x558d636a51a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:40:50.681 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:40:50.676+0000 7fdeaa600640 1 --2- 172.21.15.83:0/1309358154 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558d636ad8d0 0x558d636c4d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:40:50.681 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:40:50.677+0000 7fdeaa600640 1 -- 172.21.15.83:0/1309358154 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558d636ad8d0 msgr2=0x558d636c4d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:40:50.681 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:40:50.677+0000 7fdeaa600640 1 --2- 172.21.15.83:0/1309358154 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558d636ad8d0 0x558d636c4d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:40:50.681 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:40:50.677+0000 7fdeaa600640 1 -- 172.21.15.83:0/1309358154 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558d636a3010 msgr2=0x558d636ac710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:40:50.681 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:40:50.677+0000 7fdeaa600640 1 --2- 172.21.15.83:0/1309358154 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558d636a3010 0x558d636ac710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:40:50.681 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:40:50.677+0000 7fdeaa600640 1 -- 172.21.15.83:0/1309358154 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558d6369d1f0 con 0x558d636a26f0 2024-08-19T01:40:50.682 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:40:50.677+0000 7fdeaa600640 1 --2- 172.21.15.83:0/1309358154 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558d636a26f0 0x558d636a51a0 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7fdea40b5f40 tx=0x7fdea4068140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ebb4e3e232ea493d server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:40:50.682 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:40:50.678+0000 7fde96000640 1 -- 172.21.15.83:0/1309358154 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdea40a5070 con 0x558d636a26f0 2024-08-19T01:40:50.682 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:40:50.678+0000 7fde96000640 1 -- 172.21.15.83:0/1309358154 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdea409b070 con 0x558d636a26f0 2024-08-19T01:40:50.682 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:40:50.678+0000 7fde96000640 1 -- 172.21.15.83:0/1309358154 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdea40bb070 con 0x558d636a26f0 2024-08-19T01:40:50.698 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:40:50.679+0000 7fdeaf05c740 1 -- 172.21.15.83:0/1309358154 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558d636a26f0 msgr2=0x558d636a51a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:40:50.698 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:40:50.679+0000 7fdeaf05c740 1 --2- 172.21.15.83:0/1309358154 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558d636a26f0 0x558d636a51a0 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7fdea40b5f40 tx=0x7fdea4068140 comp rx=0 tx=0).stop 2024-08-19T01:40:50.698 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:40:50.679+0000 7fdeaf05c740 1 -- 172.21.15.83:0/1309358154 shutdown_connections 2024-08-19T01:40:50.698 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:40:50.679+0000 7fdeaf05c740 1 --2- 172.21.15.83:0/1309358154 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558d636ad8d0 0x558d636c4d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:40:50.698 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:40:50.679+0000 7fdeaf05c740 1 --2- 172.21.15.83:0/1309358154 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558d636a3010 0x558d636ac710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:40:50.698 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:40:50.679+0000 7fdeaf05c740 1 --2- 172.21.15.83:0/1309358154 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558d636a26f0 0x558d636a51a0 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:40:50.699 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:40:50.680+0000 7fdeaf05c740 1 -- 172.21.15.83:0/1309358154 >> 172.21.15.83:0/1309358154 conn(0x558d6369ec80 msgr2=0x558d636ac000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:40:50.699 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:40:50.680+0000 7fdeaf05c740 1 -- 172.21.15.83:0/1309358154 shutdown_connections 2024-08-19T01:40:50.699 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:40:50.680+0000 7fdeaf05c740 1 -- 172.21.15.83:0/1309358154 wait complete. 2024-08-19T01:40:50.699 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:40:50.681+0000 7fdeaf05c740 1 -- start start 2024-08-19T01:40:50.699 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:40:50.681+0000 7fdeaf05c740 1 Processor -- start 2024-08-19T01:40:50.699 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:40:50.684+0000 7fdeaf05c740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558d636a26f0 0x558d636b5a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:40:50.700 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:40:50.685+0000 7fdeaa600640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558d636a26f0 0x558d636b5a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:40:50.700 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:40:50.685+0000 7fdeaa600640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558d636a26f0 0x558d636b5a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:58324/0 (socket says 172.21.15.83:58324) 2024-08-19T01:40:50.700 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:40:50.685+0000 7fdeaa600640 1 -- 172.21.15.83:0/2844873045 learned_addr learned my addr 172.21.15.83:0/2844873045 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-08-19T01:40:50.700 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:40:50.685+0000 7fdeaf05c740 1 --2- 172.21.15.83:0/2844873045 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558d636a3010 0x558d636b5fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:40:50.700 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:40:50.685+0000 7fdeaf05c740 1 --2- 172.21.15.83:0/2844873045 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558d636ad8d0 0x558d636b0e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:40:50.700 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:40:50.686+0000 7fdea9c00640 1 --2- 172.21.15.83:0/2844873045 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558d636a3010 0x558d636b5fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:40:50.700 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:40:50.686+0000 7fdeaf05c740 1 -- 172.21.15.83:0/2844873045 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fde9c14e560 con 0x558d636a26f0 2024-08-19T01:40:50.700 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:40:50.689+0000 7fdeaf05c740 1 -- 172.21.15.83:0/2844873045 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fde9c14f9d0 con 0x558d636a3010 2024-08-19T01:40:50.700 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:40:50.689+0000 7fdeaf05c740 1 -- 172.21.15.83:0/2844873045 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fde9c14ec30 con 0x558d636ad8d0 2024-08-19T01:40:50.701 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:40:50.690+0000 7fdeaf05c740 10 client.0 _ll_register_callbacks cb 0x7fdea406ba20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-19T01:40:50.701 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:40:50.690+0000 7fdeaa600640 1 -- 172.21.15.83:0/2844873045 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558d636ad8d0 msgr2=0x558d636b0e60 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-08-19T01:40:50.701 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:40:50.690+0000 7fdeaa600640 1 --2- 172.21.15.83:0/2844873045 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558d636ad8d0 0x558d636b0e60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:40:50.701 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:40:50.690+0000 7fdeaa600640 1 -- 172.21.15.83:0/2844873045 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558d636a3010 msgr2=0x558d636b5fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:40:50.701 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:40:50.690+0000 7fdeaa600640 1 --2- 172.21.15.83:0/2844873045 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558d636a3010 0x558d636b5fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:40:50.701 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:40:50.691+0000 7fdeaa600640 1 -- 172.21.15.83:0/2844873045 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558d636b1660 con 0x558d636a26f0 2024-08-19T01:40:50.702 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:40:50.691+0000 7fdeaf05c740 1 -- 172.21.15.83:0/2844873045 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdea409b070 con 0x558d636a26f0 2024-08-19T01:40:50.702 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:40:50.691+0000 7fdea9c00640 1 --2- 172.21.15.83:0/2844873045 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558d636a3010 0x558d636b5fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-19T01:40:50.702 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:40:50.692+0000 7fdeaa600640 1 --2- 172.21.15.83:0/2844873045 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558d636a26f0 0x558d636b5a70 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7fdea40634e0 tx=0x7fdea409fb20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bc9610524dbdbdfb server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:40:50.702 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:40:50.692+0000 7fde8b400640 10 client.15795 ms_handle_connect on v2:172.21.15.83:3300/0 2024-08-19T01:40:50.702 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:40:50.692+0000 7fde8b400640 1 -- 172.21.15.83:0/2844873045 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdea40bb050 con 0x558d636a26f0 2024-08-19T01:40:50.703 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:40:50.693+0000 7fde8b400640 1 -- 172.21.15.83:0/2844873045 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8269+0+0 (secure 0 0 0) 0x7fdea409fc90 con 0x558d636a26f0 2024-08-19T01:40:50.703 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:40:50.693+0000 7fde8b400640 1 -- 172.21.15.83:0/2844873045 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fdea40bb050 con 0x558d636a26f0 2024-08-19T01:40:50.703 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:40:50.693+0000 7fdeaf05c740 10 client.15795 fetch_fsmap learned FSMap version 27 2024-08-19T01:40:50.703 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:40:50.693+0000 7fdeaf05c740 1 -- 172.21.15.83:0/2844873045 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({fsmap=27}) -- 0x558d63727740 con 0x558d636a26f0 2024-08-19T01:40:50.703 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:40:50.694+0000 7fde8b400640 1 -- 172.21.15.83:0/2844873045 <== mon.0 v2:172.21.15.83:3300/0 4 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7fdea40a98b0 con 0x558d636a26f0 2024-08-19T01:40:50.703 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:40:50.694+0000 7fdeaf05c740 10 client.15795 fetch_fsmap finished waiting for FSMap version 27 2024-08-19T01:40:50.703 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:40:50.694+0000 7fdeaf05c740 10 client.15795 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-08-19T01:40:50.703 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:40:50.694+0000 7fdeaf05c740 20 client.15795 populate_metadata read hostname 'smithi083' 2024-08-19T01:40:50.703 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:40:50.694+0000 7fdeaf05c740 1 --2- 172.21.15.83:0/2844873045 >> [v2:172.21.15.117:6836/3185224684,v1:172.21.15.117:6837/3185224684] conn(0x558d63727a80 0x558d63729e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:40:50.704 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:40:50.694+0000 7fdeaf05c740 4 client.15795 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-08-19T01:40:50.704 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:40:50.694+0000 7fdeaf05c740 1 -- 172.21.15.83:0/2844873045 --> [v2:172.21.15.117:6836/3185224684,v1:172.21.15.117:6837/3185224684] -- command(tid 0: {"prefix": "damage ls"}) -- 0x558d636a63c0 con 0x558d63727a80 2024-08-19T01:40:50.704 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:40:50.695+0000 7fdea9c00640 1 --2- 172.21.15.83:0/2844873045 >> [v2:172.21.15.117:6836/3185224684,v1:172.21.15.117:6837/3185224684] conn(0x558d63727a80 0x558d63729e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:40:50.704 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:40:50.695+0000 7fdea9c00640 1 --2- 172.21.15.83:0/2844873045 >> [v2:172.21.15.117:6836/3185224684,v1:172.21.15.117:6837/3185224684] conn(0x558d63727a80 0x558d63729e70 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=d144138453add8f2 server_cookie=77d2539b4396809a in_seq=0 out_seq=0 2024-08-19T01:40:50.704 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:40:50.695+0000 7fde8b400640 10 client.15795 ms_handle_connect on v2:172.21.15.117:6836/3185224684 2024-08-19T01:40:50.704 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:40:50.696+0000 7fde8b400640 1 -- 172.21.15.83:0/2844873045 <== mds.2 v2:172.21.15.117:6836/3185224684 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x558d636a63c0 con 0x558d63727a80 2024-08-19T01:40:50.704 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:40:50.696+0000 7fde8b400640 10 client.15795 handle_command_reply: tid=0 2024-08-19T01:40:50.704 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:40:50.696+0000 7fdeaf05c740 1 client.15795 shutdown 2024-08-19T01:40:50.705 INFO:teuthology.orchestra.run.smithi083.stdout:[] 2024-08-19T01:40:50.705 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:40:50.696+0000 7fdeaf05c740 1 -- 172.21.15.83:0/2844873045 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558d636a26f0 msgr2=0x558d636b5a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:40:50.705 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:40:50.696+0000 7fdeaf05c740 1 --2- 172.21.15.83:0/2844873045 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558d636a26f0 0x558d636b5a70 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7fdea40634e0 tx=0x7fdea409fb20 comp rx=0 tx=0).stop 2024-08-19T01:40:50.706 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:40:50.696+0000 7fdeaf05c740 1 -- 172.21.15.83:0/2844873045 shutdown_connections 2024-08-19T01:40:50.706 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:40:50.696+0000 7fdeaf05c740 1 --2- 172.21.15.83:0/2844873045 >> [v2:172.21.15.117:6836/3185224684,v1:172.21.15.117:6837/3185224684] conn(0x558d63727a80 0x558d63729e70 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:40:50.706 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:40:50.696+0000 7fdeaf05c740 1 --2- 172.21.15.83:0/2844873045 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558d636ad8d0 0x558d636b0e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:40:50.706 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:40:50.696+0000 7fdeaf05c740 1 --2- 172.21.15.83:0/2844873045 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558d636a3010 0x558d636b5fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:40:50.706 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:40:50.696+0000 7fdeaf05c740 1 --2- 172.21.15.83:0/2844873045 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558d636a26f0 0x558d636b5a70 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:40:50.706 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:40:50.696+0000 7fdeaf05c740 1 -- 172.21.15.83:0/2844873045 >> 172.21.15.83:0/2844873045 conn(0x558d6369ec80 msgr2=0x558d6369d5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:40:50.706 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:40:50.697+0000 7fde8b400640 0 client.15795 ms_handle_reset on v2:172.21.15.117:6836/3185224684 2024-08-19T01:40:50.706 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:40:50.697+0000 7fdeaf05c740 1 -- 172.21.15.83:0/2844873045 shutdown_connections 2024-08-19T01:40:50.706 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:40:50.697+0000 7fdeaf05c740 1 -- 172.21.15.83:0/2844873045 wait complete. 2024-08-19T01:40:50.706 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:40:50.697+0000 7fdeaf05c740 20 client.15795 trim_cache size 0 max 16384 2024-08-19T01:40:50.706 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:40:50.698+0000 7fdeac600640 1 -- 172.21.15.83:0/2836564038 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7fde8c0a2e50 msgr2=0x7fde8c0a5310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:40:50.707 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:40:50.698+0000 7fdeac600640 1 --2- 172.21.15.83:0/2836564038 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7fde8c0a2e50 0x7fde8c0a5310 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7fde90098860 tx=0x7fde900a08e0 comp rx=0 tx=0).stop 2024-08-19T01:40:50.707 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:40:50.698+0000 7fdeac600640 1 -- 172.21.15.83:0/2836564038 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fde9c0050e0 msgr2=0x7fde9c0aa0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:40:50.707 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:40:50.698+0000 7fdeac600640 1 --2- 172.21.15.83:0/2836564038 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fde9c0050e0 0x7fde9c0aa0c0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fde9000c970 tx=0x7fde9000ce30 comp rx=0 tx=0).stop 2024-08-19T01:40:50.707 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:40:50.699+0000 7fdeac600640 1 -- 172.21.15.83:0/2836564038 shutdown_connections 2024-08-19T01:40:50.707 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:40:50.699+0000 7fdeac600640 1 --2- 172.21.15.83:0/2836564038 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7fde8c0a2e50 0x7fde8c0a5310 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:40:50.707 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:40:50.699+0000 7fdeac600640 1 --2- 172.21.15.83:0/2836564038 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fde9c09b600 0x7fde9c0a4ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:40:50.707 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:40:50.699+0000 7fdeac600640 1 --2- 172.21.15.83:0/2836564038 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fde9c0050e0 0x7fde9c0aa0c0 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:40:50.707 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:40:50.699+0000 7fdeac600640 1 --2- 172.21.15.83:0/2836564038 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fde9c004740 0x7fde9c0a9b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:40:50.707 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:40:50.699+0000 7fdeac600640 1 -- 172.21.15.83:0/2836564038 >> 172.21.15.83:0/2836564038 conn(0x7fde9c00a1b0 msgr2=0x7fde9c093580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:40:50.707 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:40:50.699+0000 7fdeac600640 1 -- 172.21.15.83:0/2836564038 shutdown_connections 2024-08-19T01:40:50.707 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:40:50.699+0000 7fdeac600640 1 -- 172.21.15.83:0/2836564038 wait complete. 2024-08-19T01:40:50.731 INFO:tasks.workunit.client.0.smithi083.stdout: 197 462229 311 301194 235 276805 804 2024-08-19T01:40:51.904 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:40:51 smithi083 ceph-mon[17120]: pgmap v519: 97 pgs: 97 active+clean; 622 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 10 MiB/s wr, 1.56k op/s 2024-08-19T01:40:51.952 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:40:51 smithi117 ceph-mon[22016]: pgmap v519: 97 pgs: 97 active+clean; 622 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 10 MiB/s wr, 1.56k op/s 2024-08-19T01:40:51.965 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:40:51 smithi167 ceph-mon[21875]: pgmap v519: 97 pgs: 97 active+clean; 622 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 10 MiB/s wr, 1.56k op/s 2024-08-19T01:40:52.392 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833 shell --fsid bad8780e-5dc9-11ef-bccf-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-08-19T01:40:53.886 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:40:53 smithi083 ceph-mon[17120]: pgmap v520: 97 pgs: 97 active+clean; 627 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 9.5 MiB/s wr, 1.11k op/s 2024-08-19T01:40:53.952 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:40:53 smithi117 ceph-mon[22016]: pgmap v520: 97 pgs: 97 active+clean; 627 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 9.5 MiB/s wr, 1.11k op/s 2024-08-19T01:40:53.965 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:40:53 smithi167 ceph-mon[21875]: pgmap v520: 97 pgs: 97 active+clean; 627 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 9.5 MiB/s wr, 1.11k op/s 2024-08-19T01:40:55.810 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:40:55 smithi083 ceph-mon[17120]: pgmap v521: 97 pgs: 97 active+clean; 635 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 10 MiB/s wr, 1.16k op/s 2024-08-19T01:40:55.952 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:40:55 smithi117 ceph-mon[22016]: pgmap v521: 97 pgs: 97 active+clean; 635 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 10 MiB/s wr, 1.16k op/s 2024-08-19T01:40:55.966 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:40:55 smithi167 ceph-mon[21875]: pgmap v521: 97 pgs: 97 active+clean; 635 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 10 MiB/s wr, 1.16k op/s 2024-08-19T01:40:57.630 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:40:57 smithi083 ceph-mon[17120]: pgmap v522: 97 pgs: 97 active+clean; 633 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 9.3 MiB/s wr, 752 op/s 2024-08-19T01:40:57.952 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:40:57 smithi117 ceph-mon[22016]: pgmap v522: 97 pgs: 97 active+clean; 633 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 9.3 MiB/s wr, 752 op/s 2024-08-19T01:40:57.965 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:40:57 smithi167 ceph-mon[21875]: pgmap v522: 97 pgs: 97 active+clean; 633 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 9.3 MiB/s wr, 752 op/s 2024-08-19T01:40:59.043 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:40:59.042+0000 7fc307a00640 1 -- 172.21.15.83:0/3911145185 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc3000733f0 msgr2=0x7fc300118e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:40:59.043 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:40:59.042+0000 7fc307a00640 1 --2- 172.21.15.83:0/3911145185 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc3000733f0 0x7fc300118e60 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7fc2f8009c30 tx=0x7fc2f802a740 comp rx=0 tx=0).stop 2024-08-19T01:40:59.047 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:40:59.046+0000 7fc307a00640 1 -- 172.21.15.83:0/3911145185 shutdown_connections 2024-08-19T01:40:59.048 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:40:59.046+0000 7fc307a00640 1 --2- 172.21.15.83:0/3911145185 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc3000733f0 0x7fc300118e60 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:40:59.048 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:40:59.046+0000 7fc307a00640 1 --2- 172.21.15.83:0/3911145185 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc300072a50 0x7fc300072eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:40:59.048 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:40:59.046+0000 7fc307a00640 1 --2- 172.21.15.83:0/3911145185 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc300109430 0x7fc300109830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:40:59.048 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:40:59.046+0000 7fc307a00640 1 -- 172.21.15.83:0/3911145185 >> 172.21.15.83:0/3911145185 conn(0x7fc30006ea10 msgr2=0x7fc300070e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:40:59.058 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:40:59.056+0000 7fc307a00640 1 -- 172.21.15.83:0/3911145185 shutdown_connections 2024-08-19T01:40:59.063 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:40:59.059+0000 7fc307a00640 1 -- 172.21.15.83:0/3911145185 wait complete. 2024-08-19T01:40:59.063 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:40:59.062+0000 7fc307a00640 1 Processor -- start 2024-08-19T01:40:59.072 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:40:59.063+0000 7fc307a00640 1 -- start start 2024-08-19T01:40:59.072 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:40:59.068+0000 7fc307a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc300072a50 0x7fc3001a6680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:40:59.072 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:40:59.068+0000 7fc307a00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc3000733f0 0x7fc3001a6bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:40:59.072 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:40:59.069+0000 7fc305000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc300072a50 0x7fc3001a6680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:40:59.072 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:40:59.069+0000 7fc305000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc300072a50 0x7fc3001a6680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:54572/0 (socket says 172.21.15.83:54572) 2024-08-19T01:40:59.076 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:40:59.075+0000 7fc305000640 1 -- 172.21.15.83:0/2999308539 learned_addr learned my addr 172.21.15.83:0/2999308539 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-08-19T01:40:59.076 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:40:59.075+0000 7fc2ffe00640 1 --2- 172.21.15.83:0/2999308539 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc3000733f0 0x7fc3001a6bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:40:59.092 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:40:59.091+0000 7fc307a00640 1 --2- 172.21.15.83:0/2999308539 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc300109430 0x7fc3001adc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:40:59.092 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:40:59.091+0000 7fc307a00640 1 -- 172.21.15.83:0/2999308539 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fc30011c370 con 0x7fc300072a50 2024-08-19T01:40:59.093 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:40:59.091+0000 7fc307a00640 1 -- 172.21.15.83:0/2999308539 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fc30011c1f0 con 0x7fc3000733f0 2024-08-19T01:40:59.093 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:40:59.091+0000 7fc307a00640 1 -- 172.21.15.83:0/2999308539 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fc30011c4f0 con 0x7fc300109430 2024-08-19T01:40:59.093 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:40:59.091+0000 7fc305a00640 1 --2- 172.21.15.83:0/2999308539 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc300109430 0x7fc3001adc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:40:59.093 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:40:59.092+0000 7fc305000640 1 -- 172.21.15.83:0/2999308539 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc300109430 msgr2=0x7fc3001adc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:40:59.093 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:40:59.092+0000 7fc305000640 1 --2- 172.21.15.83:0/2999308539 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc300109430 0x7fc3001adc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:40:59.094 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:40:59.092+0000 7fc305000640 1 -- 172.21.15.83:0/2999308539 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc3000733f0 msgr2=0x7fc3001a6bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:40:59.094 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:40:59.092+0000 7fc305000640 1 --2- 172.21.15.83:0/2999308539 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc3000733f0 0x7fc3001a6bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:40:59.094 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:40:59.092+0000 7fc305000640 1 -- 172.21.15.83:0/2999308539 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc3001ae2a0 con 0x7fc300072a50 2024-08-19T01:40:59.094 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:40:59.092+0000 7fc305000640 1 --2- 172.21.15.83:0/2999308539 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc300072a50 0x7fc3001a6680 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7fc2f000da90 tx=0x7fc2f000df50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:40:59.094 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:40:59.093+0000 7fc2fd600640 1 -- 172.21.15.83:0/2999308539 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2f0017070 con 0x7fc300072a50 2024-08-19T01:40:59.095 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:40:59.093+0000 7fc2fd600640 1 -- 172.21.15.83:0/2999308539 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc2f0009c00 con 0x7fc300072a50 2024-08-19T01:40:59.095 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:40:59.093+0000 7fc2fd600640 1 -- 172.21.15.83:0/2999308539 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2f000f700 con 0x7fc300072a50 2024-08-19T01:40:59.095 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:40:59.094+0000 7fc307a00640 1 -- 172.21.15.83:0/2999308539 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc3001ae590 con 0x7fc300072a50 2024-08-19T01:40:59.095 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:40:59.094+0000 7fc307a00640 1 -- 172.21.15.83:0/2999308539 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc3001aea90 con 0x7fc300072a50 2024-08-19T01:40:59.126 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:40:59.120+0000 7fc30a886740 1 Processor -- start 2024-08-19T01:40:59.126 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:40:59.120+0000 7fc2fd600640 1 -- 172.21.15.83:0/2999308539 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7fc2f000f8a0 con 0x7fc300072a50 2024-08-19T01:40:59.126 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:40:59.120+0000 7fc30a886740 1 -- start start 2024-08-19T01:40:59.127 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:40:59.121+0000 7fc30a886740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556e02995470 0x556e02995870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:40:59.127 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:40:59.121+0000 7fc30a886740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556e02995db0 0x556e0299a870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:40:59.127 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:40:59.121+0000 7fc30a886740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556e0299ae40 0x556e0299d300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:40:59.127 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:40:59.121+0000 7fc30a886740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x556e028f9890 con 0x556e02995db0 2024-08-19T01:40:59.127 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:40:59.121+0000 7fc30a886740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x556e02891d90 con 0x556e02995470 2024-08-19T01:40:59.127 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:40:59.121+0000 7fc30a886740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x556e028f5850 con 0x556e0299ae40 2024-08-19T01:40:59.127 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:40:59.122+0000 7fc305000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556e02995db0 0x556e0299a870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:40:59.127 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:40:59.122+0000 7fc305000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556e02995db0 0x556e0299a870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:54586/0 (socket says 172.21.15.83:54586) 2024-08-19T01:40:59.127 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:40:59.122+0000 7fc305000640 1 -- 172.21.15.83:0/1916759204 learned_addr learned my addr 172.21.15.83:0/1916759204 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-08-19T01:40:59.127 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:40:59.122+0000 7fc305000640 1 -- 172.21.15.83:0/1916759204 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556e0299ae40 msgr2=0x556e0299d300 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-08-19T01:40:59.127 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:40:59.122+0000 7fc305000640 1 --2- 172.21.15.83:0/1916759204 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556e0299ae40 0x556e0299d300 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:40:59.127 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:40:59.122+0000 7fc305000640 1 -- 172.21.15.83:0/1916759204 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556e02995470 msgr2=0x556e02995870 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-08-19T01:40:59.127 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:40:59.122+0000 7fc305000640 1 --2- 172.21.15.83:0/1916759204 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556e02995470 0x556e02995870 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:40:59.128 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:40:59.122+0000 7fc305000640 1 -- 172.21.15.83:0/1916759204 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556e0260d210 con 0x556e02995db0 2024-08-19T01:40:59.128 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:40:59.123+0000 7fc305000640 1 --2- 172.21.15.83:0/1916759204 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556e02995db0 0x556e0299a870 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7fc2f00d2c70 tx=0x7fc2f00d7610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2bf392b0d879aa8c server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:40:59.150 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:40:59.126+0000 7fc2fd600640 1 --2- 172.21.15.83:0/2999308539 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7fc2e40a2e20 0x7fc2e40a52e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:40:59.150 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:40:59.127+0000 7fc2fd600640 1 -- 172.21.15.83:0/2999308539 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8269+0+0 (secure 0 0 0) 0x7fc2f00ceca0 con 0x7fc300072a50 2024-08-19T01:40:59.150 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:40:59.130+0000 7fc2f6000640 1 -- 172.21.15.83:0/1916759204 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2f00df020 con 0x556e02995db0 2024-08-19T01:40:59.150 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:40:59.130+0000 7fc2f6000640 1 -- 172.21.15.83:0/1916759204 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc2f00dbb40 con 0x556e02995db0 2024-08-19T01:40:59.150 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:40:59.130+0000 7fc2f6000640 1 -- 172.21.15.83:0/1916759204 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2f00dab40 con 0x556e02995db0 2024-08-19T01:40:59.150 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:40:59.131+0000 7fc30a886740 1 -- 172.21.15.83:0/1916759204 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556e02995db0 msgr2=0x556e0299a870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:40:59.150 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:40:59.131+0000 7fc30a886740 1 --2- 172.21.15.83:0/1916759204 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556e02995db0 0x556e0299a870 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7fc2f00d2c70 tx=0x7fc2f00d7610 comp rx=0 tx=0).stop 2024-08-19T01:40:59.150 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:40:59.139+0000 7fc30a886740 1 -- 172.21.15.83:0/1916759204 shutdown_connections 2024-08-19T01:40:59.151 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:40:59.139+0000 7fc30a886740 1 --2- 172.21.15.83:0/1916759204 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556e0299ae40 0x556e0299d300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:40:59.151 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:40:59.139+0000 7fc30a886740 1 --2- 172.21.15.83:0/1916759204 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556e02995db0 0x556e0299a870 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:40:59.151 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:40:59.139+0000 7fc30a886740 1 --2- 172.21.15.83:0/1916759204 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556e02995470 0x556e02995870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:40:59.151 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:40:59.139+0000 7fc30a886740 1 -- 172.21.15.83:0/1916759204 >> 172.21.15.83:0/1916759204 conn(0x556e02990ae0 msgr2=0x556e02990ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:40:59.151 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:40:59.140+0000 7fc305a00640 1 --2- 172.21.15.83:0/2999308539 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7fc2e40a2e20 0x7fc2e40a52e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:40:59.151 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:40:59.141+0000 7fc305a00640 1 --2- 172.21.15.83:0/2999308539 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7fc2e40a2e20 0x7fc2e40a52e0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fc2f8009370 tx=0x7fc2f800b040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:40:59.151 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:40:59.144+0000 7fc30a886740 1 -- 172.21.15.83:0/1916759204 shutdown_connections 2024-08-19T01:40:59.151 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:40:59.145+0000 7fc30a886740 1 -- 172.21.15.83:0/1916759204 wait complete. 2024-08-19T01:40:59.151 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:40:59.145+0000 7fc30a886740 1 -- start start 2024-08-19T01:40:59.151 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:40:59.146+0000 7fc30a886740 1 Processor -- start 2024-08-19T01:40:59.151 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:40:59.146+0000 7fc30a886740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556e02995470 0x556e029b0810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:40:59.151 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:40:59.146+0000 7fc30a886740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556e02995db0 0x556e029b0d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:40:59.151 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:40:59.147+0000 7fc305000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556e02995470 0x556e029b0810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:40:59.152 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:40:59.147+0000 7fc305000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556e02995470 0x556e029b0810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:54600/0 (socket says 172.21.15.83:54600) 2024-08-19T01:40:59.152 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:40:59.147+0000 7fc305000640 1 -- 172.21.15.83:0/280652085 learned_addr learned my addr 172.21.15.83:0/280652085 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-08-19T01:40:59.152 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:40:59.147+0000 7fc30a886740 1 --2- 172.21.15.83:0/280652085 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556e0299ae40 0x556e029b7d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:40:59.152 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:40:59.147+0000 7fc2ffe00640 1 --2- 172.21.15.83:0/280652085 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556e02995db0 0x556e029b0d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:40:59.152 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:40:59.147+0000 7fc30a886740 1 -- 172.21.15.83:0/280652085 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x556e029aeec0 con 0x556e02995470 2024-08-19T01:40:59.152 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:40:59.147+0000 7fc30a886740 1 -- 172.21.15.83:0/280652085 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x556e028b4f70 con 0x556e0299ae40 2024-08-19T01:40:59.152 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:40:59.147+0000 7fc30a886740 1 -- 172.21.15.83:0/280652085 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x556e029af040 con 0x556e02995db0 2024-08-19T01:40:59.152 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:40:59.147+0000 7fc305a00640 1 --2- 172.21.15.83:0/280652085 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556e0299ae40 0x556e029b7d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:40:59.152 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:40:59.148+0000 7fc305a00640 1 -- 172.21.15.83:0/280652085 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556e02995db0 msgr2=0x556e029b0d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:40:59.152 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:40:59.148+0000 7fc305a00640 1 --2- 172.21.15.83:0/280652085 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556e02995db0 0x556e029b0d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:40:59.152 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:40:59.148+0000 7fc305a00640 1 -- 172.21.15.83:0/280652085 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556e02995470 msgr2=0x556e029b0810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:40:59.152 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:40:59.148+0000 7fc305a00640 1 --2- 172.21.15.83:0/280652085 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556e02995470 0x556e029b0810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:40:59.152 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:40:59.148+0000 7fc305a00640 1 -- 172.21.15.83:0/280652085 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556e026d9620 con 0x556e0299ae40 2024-08-19T01:40:59.158 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:40:59.148+0000 7fc305000640 1 --2- 172.21.15.83:0/280652085 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556e02995470 0x556e029b0810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-19T01:40:59.158 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:40:59.149+0000 7fc30a886740 10 client.0 _ll_register_callbacks cb 0x556e028f18d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-19T01:40:59.158 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:40:59.149+0000 7fc305a00640 1 --2- 172.21.15.83:0/280652085 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556e0299ae40 0x556e029b7d90 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fc2f8034040 tx=0x7fc2f8005fd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f3520a59d106d7bd server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:40:59.158 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:40:59.149+0000 7fc30a886740 1 -- 172.21.15.83:0/280652085 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc2f00dbb40 con 0x556e0299ae40 2024-08-19T01:40:59.158 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:40:59.150+0000 7fc2eaa00640 10 client.25850 ms_handle_connect on v2:172.21.15.117:3300/0 2024-08-19T01:40:59.158 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:40:59.150+0000 7fc2eaa00640 1 -- 172.21.15.83:0/280652085 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2f8043070 con 0x556e0299ae40 2024-08-19T01:40:59.158 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:40:59.150+0000 7fc2eaa00640 1 -- 172.21.15.83:0/280652085 <== mon.2 v2:172.21.15.117:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8269+0+0 (secure 0 0 0) 0x7fc2f8031850 con 0x556e0299ae40 2024-08-19T01:40:59.162 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:40:59.157+0000 7fc2eaa00640 1 -- 172.21.15.83:0/280652085 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fc2f8036040 con 0x556e0299ae40 2024-08-19T01:40:59.162 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:40:59.159+0000 7fc30a886740 10 client.25850 fetch_fsmap learned FSMap version 27 2024-08-19T01:40:59.165 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:40:59.163+0000 7fc30a886740 1 -- 172.21.15.83:0/280652085 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({fsmap=27}) -- 0x556e029bade0 con 0x556e0299ae40 2024-08-19T01:40:59.169 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:40:59.165+0000 7fc2eaa00640 1 -- 172.21.15.83:0/280652085 <== mon.2 v2:172.21.15.117:3300/0 4 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7fc2f803c040 con 0x556e0299ae40 2024-08-19T01:40:59.169 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:40:59.167+0000 7fc30a886740 10 client.25850 fetch_fsmap finished waiting for FSMap version 27 2024-08-19T01:40:59.169 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:40:59.167+0000 7fc30a886740 10 client.25850 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-08-19T01:40:59.169 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:40:59.168+0000 7fc30a886740 20 client.25850 populate_metadata read hostname 'smithi083' 2024-08-19T01:40:59.169 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:40:59.168+0000 7fc30a886740 1 --2- 172.21.15.83:0/280652085 >> [v2:172.21.15.117:6838/1967595704,v1:172.21.15.117:6839/1967595704] conn(0x556e029bc480 0x556e029be910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:40:59.169 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:40:59.168+0000 7fc30a886740 4 client.25850 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-19T01:40:59.169 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:40:59.168+0000 7fc30a886740 1 -- 172.21.15.83:0/280652085 --> [v2:172.21.15.117:6838/1967595704,v1:172.21.15.117:6839/1967595704] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556e02906fc0 con 0x556e029bc480 2024-08-19T01:40:59.181 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:40:59.180+0000 7fc305000640 1 --2- 172.21.15.83:0/280652085 >> [v2:172.21.15.117:6838/1967595704,v1:172.21.15.117:6839/1967595704] conn(0x556e029bc480 0x556e029be910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:40:59.189 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:40:59.188+0000 7fc305000640 1 --2- 172.21.15.83:0/280652085 >> [v2:172.21.15.117:6838/1967595704,v1:172.21.15.117:6839/1967595704] conn(0x556e029bc480 0x556e029be910 crc :-1 s=READY pgs=107 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b301d1ac2cd46486 server_cookie=ee8c9eab69db66e7 in_seq=0 out_seq=0 2024-08-19T01:40:59.189 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:40:59.188+0000 7fc2eaa00640 10 client.25850 ms_handle_connect on v2:172.21.15.117:6838/1967595704 2024-08-19T01:40:59.196 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:40:59.195+0000 7fc2eaa00640 1 -- 172.21.15.83:0/280652085 <== mds.0 v2:172.21.15.117:6838/1967595704 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x556e02906fc0 con 0x556e029bc480 2024-08-19T01:40:59.197 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:40:59.195+0000 7fc2eaa00640 10 client.25850 handle_command_reply: tid=0 2024-08-19T01:40:59.197 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:40:59.196+0000 7fc30a886740 1 client.25850 shutdown 2024-08-19T01:40:59.197 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:40:59.196+0000 7fc30a886740 1 -- 172.21.15.83:0/280652085 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556e0299ae40 msgr2=0x556e029b7d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:40:59.198 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:40:59.196+0000 7fc30a886740 1 --2- 172.21.15.83:0/280652085 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556e0299ae40 0x556e029b7d90 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fc2f8034040 tx=0x7fc2f8005fd0 comp rx=0 tx=0).stop 2024-08-19T01:40:59.198 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:40:59.196+0000 7fc30a886740 1 -- 172.21.15.83:0/280652085 shutdown_connections 2024-08-19T01:40:59.198 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:40:59.196+0000 7fc30a886740 1 --2- 172.21.15.83:0/280652085 >> [v2:172.21.15.117:6838/1967595704,v1:172.21.15.117:6839/1967595704] conn(0x556e029bc480 0x556e029be910 crc :-1 s=READY pgs=107 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:40:59.198 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:40:59.196+0000 7fc30a886740 1 --2- 172.21.15.83:0/280652085 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556e0299ae40 0x556e029b7d90 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:40:59.198 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:40:59.196+0000 7fc30a886740 1 --2- 172.21.15.83:0/280652085 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556e02995db0 0x556e029b0d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:40:59.198 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:40:59.196+0000 7fc30a886740 1 --2- 172.21.15.83:0/280652085 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556e02995470 0x556e029b0810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:40:59.198 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:40:59.196+0000 7fc30a886740 1 -- 172.21.15.83:0/280652085 >> 172.21.15.83:0/280652085 conn(0x556e02990ae0 msgr2=0x556e029922f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:40:59.198 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:40:59.196+0000 7fc2eaa00640 0 client.25850 ms_handle_reset on v2:172.21.15.117:6838/1967595704 2024-08-19T01:40:59.216 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:40:59.215+0000 7fc30a886740 1 -- 172.21.15.83:0/280652085 shutdown_connections 2024-08-19T01:40:59.216 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:40:59.215+0000 7fc30a886740 1 -- 172.21.15.83:0/280652085 wait complete. 2024-08-19T01:40:59.217 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:40:59.216+0000 7fc30a886740 20 client.25850 trim_cache size 0 max 16384 2024-08-19T01:40:59.500 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:40:59.495+0000 7fc30a886740 1 Processor -- start 2024-08-19T01:40:59.501 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:40:59.495+0000 7fc30a886740 1 -- start start 2024-08-19T01:40:59.501 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:40:59.495+0000 7fc30a886740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556e02995470 0x556e02996fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:40:59.501 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:40:59.495+0000 7fc30a886740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556e02995db0 0x556e029b7d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:40:59.501 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:40:59.495+0000 7fc30a886740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556e0299ae40 0x556e02997510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:40:59.501 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:40:59.495+0000 7fc30a886740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fc2e0010e30 con 0x556e02995db0 2024-08-19T01:40:59.501 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:40:59.495+0000 7fc30a886740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fc2e000e700 con 0x556e0299ae40 2024-08-19T01:40:59.501 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:40:59.495+0000 7fc30a886740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fc2e000c550 con 0x556e02995470 2024-08-19T01:40:59.501 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:40:59.499+0000 7fc305a00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556e0299ae40 0x556e02997510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:40:59.501 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:40:59.499+0000 7fc305a00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556e0299ae40 0x556e02997510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.83:35482/0 (socket says 172.21.15.83:35482) 2024-08-19T01:40:59.501 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:40:59.499+0000 7fc305a00640 1 -- 172.21.15.83:0/3435806793 learned_addr learned my addr 172.21.15.83:0/3435806793 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-08-19T01:40:59.501 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:40:59.499+0000 7fc305a00640 1 -- 172.21.15.83:0/3435806793 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556e02995470 msgr2=0x556e02996fd0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-08-19T01:40:59.502 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:40:59.499+0000 7fc305a00640 1 --2- 172.21.15.83:0/3435806793 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556e02995470 0x556e02996fd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:40:59.502 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:40:59.499+0000 7fc305a00640 1 -- 172.21.15.83:0/3435806793 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556e02995db0 msgr2=0x556e029b7d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:40:59.502 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:40:59.499+0000 7fc2ffe00640 1 --2- 172.21.15.83:0/3435806793 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556e02995db0 0x556e029b7d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:40:59.518 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:40:59.499+0000 7fc305a00640 1 --2- 172.21.15.83:0/3435806793 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556e02995db0 0x556e029b7d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:40:59.518 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:40:59.499+0000 7fc305a00640 1 -- 172.21.15.83:0/3435806793 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556e0299abe0 con 0x556e0299ae40 2024-08-19T01:40:59.518 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:40:59.502+0000 7fc2ffe00640 1 --2- 172.21.15.83:0/3435806793 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556e02995db0 0x556e029b7d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-19T01:40:59.518 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:40:59.502+0000 7fc305a00640 1 --2- 172.21.15.83:0/3435806793 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556e0299ae40 0x556e02997510 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fc2f8005ff0 tx=0x7fc2f8049000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5a71679aaccda72d server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:40:59.518 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:40:59.513+0000 7fc2f5600640 1 -- 172.21.15.83:0/3435806793 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2f8063070 con 0x556e0299ae40 2024-08-19T01:40:59.518 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:40:59.513+0000 7fc2f5600640 1 -- 172.21.15.83:0/3435806793 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc2f00dbb40 con 0x556e0299ae40 2024-08-19T01:40:59.518 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:40:59.513+0000 7fc2f5600640 1 -- 172.21.15.83:0/3435806793 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2f804a6d0 con 0x556e0299ae40 2024-08-19T01:40:59.518 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:40:59.514+0000 7fc30a886740 1 -- 172.21.15.83:0/3435806793 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556e0299ae40 msgr2=0x556e02997510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:40:59.519 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:40:59.514+0000 7fc30a886740 1 --2- 172.21.15.83:0/3435806793 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556e0299ae40 0x556e02997510 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fc2f8005ff0 tx=0x7fc2f8049000 comp rx=0 tx=0).stop 2024-08-19T01:40:59.519 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:40:59.518+0000 7fc30a886740 1 -- 172.21.15.83:0/3435806793 shutdown_connections 2024-08-19T01:40:59.519 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:40:59.518+0000 7fc30a886740 1 --2- 172.21.15.83:0/3435806793 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556e0299ae40 0x556e02997510 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:40:59.519 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:40:59.518+0000 7fc30a886740 1 --2- 172.21.15.83:0/3435806793 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556e02995db0 0x556e029b7d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:40:59.519 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:40:59.518+0000 7fc30a886740 1 --2- 172.21.15.83:0/3435806793 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556e02995470 0x556e02996fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:40:59.519 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:40:59.518+0000 7fc30a886740 1 -- 172.21.15.83:0/3435806793 >> 172.21.15.83:0/3435806793 conn(0x556e02990ae0 msgr2=0x556e02990ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:40:59.522 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:40:59.521+0000 7fc30a886740 1 -- 172.21.15.83:0/3435806793 shutdown_connections 2024-08-19T01:40:59.523 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:40:59.521+0000 7fc30a886740 1 -- 172.21.15.83:0/3435806793 wait complete. 2024-08-19T01:40:59.523 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:40:59.521+0000 7fc30a886740 1 -- start start 2024-08-19T01:40:59.523 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:40:59.522+0000 7fc30a886740 1 Processor -- start 2024-08-19T01:40:59.527 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:40:59.526+0000 7fc30a886740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556e02995470 0x556e029b7ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:40:59.527 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:40:59.526+0000 7fc30a886740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556e02995db0 0x556e029b4820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:40:59.528 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:40:59.527+0000 7fc2ffe00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556e02995db0 0x556e029b4820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:40:59.529 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:40:59.527+0000 7fc2ffe00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556e02995db0 0x556e029b4820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:54612/0 (socket says 172.21.15.83:54612) 2024-08-19T01:40:59.529 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:40:59.527+0000 7fc2ffe00640 1 -- 172.21.15.83:0/3725806129 learned_addr learned my addr 172.21.15.83:0/3725806129 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-08-19T01:40:59.529 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:40:59.527+0000 7fc30a886740 1 --2- 172.21.15.83:0/3725806129 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556e0299ae40 0x556e029b4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:40:59.529 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:40:59.528+0000 7fc30a886740 1 -- 172.21.15.83:0/3725806129 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fc2e000cbc0 con 0x556e02995db0 2024-08-19T01:40:59.531 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:40:59.528+0000 7fc30a886740 1 -- 172.21.15.83:0/3725806129 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fc2e000e030 con 0x556e02995470 2024-08-19T01:40:59.531 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:40:59.528+0000 7fc30a886740 1 -- 172.21.15.83:0/3725806129 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fc2e000d290 con 0x556e0299ae40 2024-08-19T01:40:59.531 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:40:59.528+0000 7fc305a00640 1 --2- 172.21.15.83:0/3725806129 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556e0299ae40 0x556e029b4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:40:59.531 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:40:59.529+0000 7fc2ffe00640 1 -- 172.21.15.83:0/3725806129 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556e0299ae40 msgr2=0x556e029b4dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:40:59.531 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:40:59.529+0000 7fc2ffe00640 1 --2- 172.21.15.83:0/3725806129 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556e0299ae40 0x556e029b4dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:40:59.532 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:40:59.529+0000 7fc2ffe00640 1 -- 172.21.15.83:0/3725806129 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556e02995470 msgr2=0x556e029b7ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:40:59.532 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:40:59.529+0000 7fc2ffe00640 1 --2- 172.21.15.83:0/3725806129 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556e02995470 0x556e029b7ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:40:59.532 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:40:59.529+0000 7fc2ffe00640 1 -- 172.21.15.83:0/3725806129 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556e02a25210 con 0x556e02995db0 2024-08-19T01:40:59.532 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:40:59.529+0000 7fc2ffe00640 1 --2- 172.21.15.83:0/3725806129 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556e02995db0 0x556e029b4820 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7fc3001a7b90 tx=0x7fc2ec0039c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=83492dab658806a3 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:40:59.532 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:40:59.529+0000 7fc30a886740 10 client.0 _ll_register_callbacks cb 0x7fc2f8009a60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-19T01:40:59.532 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:40:59.529+0000 7fc30a886740 1 -- 172.21.15.83:0/3725806129 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc2f00dbb40 con 0x556e02995db0 2024-08-19T01:40:59.532 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:40:59.530+0000 7fc2eaa00640 10 client.15810 ms_handle_connect on v2:172.21.15.83:3300/0 2024-08-19T01:40:59.532 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:40:59.530+0000 7fc2eaa00640 1 -- 172.21.15.83:0/3725806129 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2ec00bdf0 con 0x556e02995db0 2024-08-19T01:40:59.532 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:40:59.530+0000 7fc2eaa00640 1 -- 172.21.15.83:0/3725806129 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8269+0+0 (secure 0 0 0) 0x7fc2ec005020 con 0x556e02995db0 2024-08-19T01:40:59.533 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:40:59.530+0000 7fc2eaa00640 1 -- 172.21.15.83:0/3725806129 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fc2ec005260 con 0x556e02995db0 2024-08-19T01:40:59.538 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:40:59.537+0000 7fc30a886740 10 client.15810 fetch_fsmap learned FSMap version 27 2024-08-19T01:40:59.538 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:40:59.537+0000 7fc30a886740 1 -- 172.21.15.83:0/3725806129 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({fsmap=27}) -- 0x556e02a263c0 con 0x556e02995db0 2024-08-19T01:40:59.539 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:40:59.537+0000 7fc2eaa00640 1 -- 172.21.15.83:0/3725806129 <== mon.0 v2:172.21.15.83:3300/0 4 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7fc2ec005400 con 0x556e02995db0 2024-08-19T01:40:59.539 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:40:59.537+0000 7fc30a886740 10 client.15810 fetch_fsmap finished waiting for FSMap version 27 2024-08-19T01:40:59.539 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:40:59.537+0000 7fc30a886740 10 client.15810 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-08-19T01:40:59.539 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:40:59.537+0000 7fc30a886740 20 client.15810 populate_metadata read hostname 'smithi083' 2024-08-19T01:40:59.539 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:40:59.537+0000 7fc30a886740 1 --2- 172.21.15.83:0/3725806129 >> [v2:172.21.15.117:6838/1967595704,v1:172.21.15.117:6839/1967595704] conn(0x556e02a26cb0 0x556e02a29170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:40:59.539 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:40:59.537+0000 7fc30a886740 4 client.15810 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-08-19T01:40:59.539 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:40:59.537+0000 7fc30a886740 1 -- 172.21.15.83:0/3725806129 --> [v2:172.21.15.117:6838/1967595704,v1:172.21.15.117:6839/1967595704] -- command(tid 0: {"prefix": "flush journal"}) -- 0x556e02901310 con 0x556e02a26cb0 2024-08-19T01:40:59.539 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:40:59.538+0000 7fc305000640 1 --2- 172.21.15.83:0/3725806129 >> [v2:172.21.15.117:6838/1967595704,v1:172.21.15.117:6839/1967595704] conn(0x556e02a26cb0 0x556e02a29170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:40:59.543 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:40:59.538+0000 7fc305000640 1 --2- 172.21.15.83:0/3725806129 >> [v2:172.21.15.117:6838/1967595704,v1:172.21.15.117:6839/1967595704] conn(0x556e02a26cb0 0x556e02a29170 crc :-1 s=READY pgs=108 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=617f39c5c9586fd7 server_cookie=3498363a17c01403 in_seq=0 out_seq=0 2024-08-19T01:40:59.543 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:40:59.538+0000 7fc2eaa00640 10 client.15810 ms_handle_connect on v2:172.21.15.117:6838/1967595704 2024-08-19T01:40:59.644 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:40:59 smithi083 ceph-mon[17120]: pgmap v523: 97 pgs: 97 active+clean; 638 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 8.5 MiB/s wr, 602 op/s 2024-08-19T01:40:59.952 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:40:59 smithi117 ceph-mon[22016]: pgmap v523: 97 pgs: 97 active+clean; 638 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 8.5 MiB/s wr, 602 op/s 2024-08-19T01:40:59.965 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:40:59 smithi167 ceph-mon[21875]: pgmap v523: 97 pgs: 97 active+clean; 638 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 8.5 MiB/s wr, 602 op/s 2024-08-19T01:41:00.274 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/bad8780e-5dc9-11ef-bccf-c7b262605968/mon.a/config 2024-08-19T01:41:00.739 INFO:tasks.workunit.client.0.smithi083.stdout: 207 646051 594 425180 542 380751 1594 2024-08-19T01:41:01.889 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:41:01 smithi083 ceph-mon[17120]: pgmap v524: 97 pgs: 97 active+clean; 640 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 5.0 MiB/s wr, 359 op/s 2024-08-19T01:41:01.952 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:41:01 smithi117 ceph-mon[22016]: pgmap v524: 97 pgs: 97 active+clean; 640 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 5.0 MiB/s wr, 359 op/s 2024-08-19T01:41:01.965 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:41:01 smithi167 ceph-mon[21875]: pgmap v524: 97 pgs: 97 active+clean; 640 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 5.0 MiB/s wr, 359 op/s 2024-08-19T01:41:02.196 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:02.122+0000 7ff4db200640 1 -- 172.21.15.83:0/1607067597 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff4d40a2060 msgr2=0x7ff4d40a4450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:41:02.197 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:02.122+0000 7ff4db200640 1 --2- 172.21.15.83:0/1607067597 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff4d40a2060 0x7ff4d40a4450 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7ff4dc04ffb0 tx=0x7ff4dc096f20 comp rx=0 tx=0).stop 2024-08-19T01:41:02.197 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:02.124+0000 7ff4db200640 1 -- 172.21.15.83:0/1607067597 shutdown_connections 2024-08-19T01:41:02.197 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:02.124+0000 7ff4db200640 1 --2- 172.21.15.83:0/1607067597 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff4d40a2060 0x7ff4d40a4450 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:41:02.197 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:02.124+0000 7ff4db200640 1 --2- 172.21.15.83:0/1607067597 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff4d4007260 0x7ff4d40a1b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:41:02.197 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:02.124+0000 7ff4db200640 1 --2- 172.21.15.83:0/1607067597 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff4d4006940 0x7ff4d4006d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:41:02.197 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:02.124+0000 7ff4db200640 1 -- 172.21.15.83:0/1607067597 >> 172.21.15.83:0/1607067597 conn(0x7ff4d400c9d0 msgr2=0x7ff4d400cde0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:41:02.197 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:02.125+0000 7ff4db200640 1 -- 172.21.15.83:0/1607067597 shutdown_connections 2024-08-19T01:41:02.197 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:02.125+0000 7ff4db200640 1 -- 172.21.15.83:0/1607067597 wait complete. 2024-08-19T01:41:02.197 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:02.126+0000 7ff4db200640 1 Processor -- start 2024-08-19T01:41:02.197 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:02.127+0000 7ff4db200640 1 -- start start 2024-08-19T01:41:02.197 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:02.128+0000 7ff4db200640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff4d4006940 0x7ff4d40a0610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:41:02.197 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:02.128+0000 7ff4db200640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff4d4007260 0x7ff4d40a0b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:41:02.198 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:02.128+0000 7ff4db200640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff4d40a2060 0x7ff4d4148f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:41:02.198 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:02.128+0000 7ff4db200640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7ff4d4096850 con 0x7ff4d40a2060 2024-08-19T01:41:02.198 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:02.128+0000 7ff4db200640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7ff4d40966d0 con 0x7ff4d4007260 2024-08-19T01:41:02.198 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:02.128+0000 7ff4db200640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7ff4d40969d0 con 0x7ff4d4006940 2024-08-19T01:41:02.198 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:02.129+0000 7ff4d9e00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff4d4006940 0x7ff4d40a0610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:41:02.198 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:02.129+0000 7ff4d9400640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff4d4007260 0x7ff4d40a0b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:41:02.198 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:02.129+0000 7ff4d9400640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff4d4007260 0x7ff4d40a0b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.83:42826/0 (socket says 172.21.15.83:42826) 2024-08-19T01:41:02.198 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:02.129+0000 7ff4d9400640 1 -- 172.21.15.83:0/4099916028 learned_addr learned my addr 172.21.15.83:0/4099916028 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-08-19T01:41:02.198 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:02.129+0000 7ff4d9400640 1 -- 172.21.15.83:0/4099916028 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff4d4006940 msgr2=0x7ff4d40a0610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:41:02.198 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:02.131+0000 7ff4d9400640 1 --2- 172.21.15.83:0/4099916028 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff4d4006940 0x7ff4d40a0610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:41:02.198 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:02.131+0000 7ff4d9400640 1 -- 172.21.15.83:0/4099916028 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff4d40a2060 msgr2=0x7ff4d4148f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:41:02.198 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:02.131+0000 7ff4d9400640 1 --2- 172.21.15.83:0/4099916028 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff4d40a2060 0x7ff4d4148f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:41:02.199 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:02.131+0000 7ff4d9400640 1 -- 172.21.15.83:0/4099916028 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff4d4149630 con 0x7ff4d4007260 2024-08-19T01:41:02.199 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:02.131+0000 7ff4d9e00640 1 --2- 172.21.15.83:0/4099916028 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff4d4006940 0x7ff4d40a0610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-19T01:41:02.199 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:02.131+0000 7ff4d9400640 1 --2- 172.21.15.83:0/4099916028 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff4d4007260 0x7ff4d40a0b50 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7ff4cc00d950 tx=0x7ff4cc00de10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:41:02.199 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:02.132+0000 7ff4d2a00640 1 -- 172.21.15.83:0/4099916028 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4cc017070 con 0x7ff4d4007260 2024-08-19T01:41:02.199 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:02.132+0000 7ff4d2a00640 1 -- 172.21.15.83:0/4099916028 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff4cc009c30 con 0x7ff4d4007260 2024-08-19T01:41:02.199 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:02.132+0000 7ff4d2a00640 1 -- 172.21.15.83:0/4099916028 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4cc00f6b0 con 0x7ff4d4007260 2024-08-19T01:41:02.199 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:02.132+0000 7ff4db200640 1 -- 172.21.15.83:0/4099916028 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff4d4149920 con 0x7ff4d4007260 2024-08-19T01:41:02.199 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:02.132+0000 7ff4db200640 1 -- 172.21.15.83:0/4099916028 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff4d414ffb0 con 0x7ff4d4007260 2024-08-19T01:41:02.199 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:02.141+0000 7ff4d2a00640 1 -- 172.21.15.83:0/4099916028 <== mon.2 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7ff4cc00f850 con 0x7ff4d4007260 2024-08-19T01:41:02.199 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:02.152+0000 7ff4d2a00640 1 --2- 172.21.15.83:0/4099916028 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7ff4c00a2c10 0x7ff4c00a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:41:02.199 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:02.152+0000 7ff4d2a00640 1 -- 172.21.15.83:0/4099916028 <== mon.2 v2:172.21.15.117:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8269+0+0 (secure 0 0 0) 0x7ff4cc0cec50 con 0x7ff4d4007260 2024-08-19T01:41:02.199 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:02.153+0000 7ff4e3c37740 1 Processor -- start 2024-08-19T01:41:02.199 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:02.153+0000 7ff4d9e00640 1 --2- 172.21.15.83:0/4099916028 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7ff4c00a2c10 0x7ff4c00a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:41:02.200 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:02.154+0000 7ff4e3c37740 1 -- start start 2024-08-19T01:41:02.200 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:02.155+0000 7ff4e3c37740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d6c882d6e0 0x55d6c882dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:41:02.200 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:02.155+0000 7ff4e3c37740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d6c882e000 0x55d6c8838380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:41:02.200 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:02.155+0000 7ff4e3c37740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d6c88388c0 0x55d6c883acb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:41:02.200 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:02.155+0000 7ff4e3c37740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55d6c8792bf0 con 0x55d6c88388c0 2024-08-19T01:41:02.200 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:02.155+0000 7ff4e3c37740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55d6c872ab20 con 0x55d6c882e000 2024-08-19T01:41:02.200 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:02.155+0000 7ff4e3c37740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55d6c878ebb0 con 0x55d6c882d6e0 2024-08-19T01:41:02.200 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:02.155+0000 7ff4d9e00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d6c882d6e0 0x55d6c882dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:41:02.200 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:02.155+0000 7ff4d9e00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d6c882d6e0 0x55d6c882dac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.83:53136/0 (socket says 172.21.15.83:53136) 2024-08-19T01:41:02.200 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:02.155+0000 7ff4d9e00640 1 -- 172.21.15.83:0/4082570593 learned_addr learned my addr 172.21.15.83:0/4082570593 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-08-19T01:41:02.200 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:02.156+0000 7ff4d9e00640 1 -- 172.21.15.83:0/4082570593 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d6c882e000 msgr2=0x55d6c8838380 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-08-19T01:41:02.200 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:02.156+0000 7ff4d9e00640 1 --2- 172.21.15.83:0/4082570593 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d6c882e000 0x55d6c8838380 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:41:02.201 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:02.156+0000 7ff4d9e00640 1 -- 172.21.15.83:0/4082570593 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d6c88388c0 msgr2=0x55d6c883acb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:41:02.201 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:02.156+0000 7ff4d9e00640 1 --2- 172.21.15.83:0/4082570593 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d6c88388c0 0x55d6c883acb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:41:02.201 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:02.156+0000 7ff4d9e00640 1 -- 172.21.15.83:0/4082570593 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d6c87317f0 con 0x55d6c882d6e0 2024-08-19T01:41:02.201 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:02.157+0000 7ff4d9e00640 1 --2- 172.21.15.83:0/4082570593 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d6c882d6e0 0x55d6c882dac0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7ff4c800ed90 tx=0x7ff4c8016000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9c6a3fbc95568f9f server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:41:02.201 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:02.157+0000 7ff4c7400640 1 -- 172.21.15.83:0/4082570593 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4c8017070 con 0x55d6c882d6e0 2024-08-19T01:41:02.201 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:02.157+0000 7ff4c7400640 1 -- 172.21.15.83:0/4082570593 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff4c8005500 con 0x55d6c882d6e0 2024-08-19T01:41:02.201 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:02.157+0000 7ff4c7400640 1 -- 172.21.15.83:0/4082570593 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4c801d680 con 0x55d6c882d6e0 2024-08-19T01:41:02.201 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:02.159+0000 7ff4e3c37740 1 -- 172.21.15.83:0/4082570593 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d6c882d6e0 msgr2=0x55d6c882dac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:41:02.201 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:02.159+0000 7ff4e3c37740 1 --2- 172.21.15.83:0/4082570593 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d6c882d6e0 0x55d6c882dac0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7ff4c800ed90 tx=0x7ff4c8016000 comp rx=0 tx=0).stop 2024-08-19T01:41:02.201 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:02.159+0000 7ff4e3c37740 1 -- 172.21.15.83:0/4082570593 shutdown_connections 2024-08-19T01:41:02.201 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:02.159+0000 7ff4e3c37740 1 --2- 172.21.15.83:0/4082570593 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d6c88388c0 0x55d6c883acb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:41:02.201 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:02.159+0000 7ff4e3c37740 1 --2- 172.21.15.83:0/4082570593 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d6c882e000 0x55d6c8838380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:41:02.201 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:02.159+0000 7ff4e3c37740 1 --2- 172.21.15.83:0/4082570593 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d6c882d6e0 0x55d6c882dac0 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:41:02.202 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:02.159+0000 7ff4e3c37740 1 -- 172.21.15.83:0/4082570593 >> 172.21.15.83:0/4082570593 conn(0x55d6c8829c70 msgr2=0x55d6c882a050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:41:02.202 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:02.160+0000 7ff4e3c37740 1 -- 172.21.15.83:0/4082570593 shutdown_connections 2024-08-19T01:41:02.202 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:02.160+0000 7ff4e3c37740 1 -- 172.21.15.83:0/4082570593 wait complete. 2024-08-19T01:41:02.202 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:02.161+0000 7ff4e3c37740 1 -- start start 2024-08-19T01:41:02.202 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:02.161+0000 7ff4e3c37740 1 Processor -- start 2024-08-19T01:41:02.202 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:02.165+0000 7ff4d9e00640 1 --2- 172.21.15.83:0/4099916028 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7ff4c00a2c10 0x7ff4c00a50d0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7ff4c8005dc0 tx=0x7ff4c8009570 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:41:02.202 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:02.167+0000 7ff4e3c37740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d6c882d6e0 0x55d6c8835760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:41:02.202 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:02.167+0000 7ff4e3c37740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d6c882e000 0x55d6c8835ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:41:02.202 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:02.167+0000 7ff4e3c37740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d6c88388c0 0x55d6c88361e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:41:02.202 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:02.167+0000 7ff4e3c37740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55d6c883b490 con 0x55d6c882e000 2024-08-19T01:41:02.202 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:02.167+0000 7ff4e3c37740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55d6c874dcf0 con 0x55d6c882d6e0 2024-08-19T01:41:02.202 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:02.167+0000 7ff4e3c37740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55d6c883b610 con 0x55d6c88388c0 2024-08-19T01:41:02.202 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:02.168+0000 7ff4e3c37740 10 client.0 _ll_register_callbacks cb 0x55d6c8739e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-19T01:41:02.203 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:02.168+0000 7ff4da800640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d6c88388c0 0x55d6c88361e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:41:02.203 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:02.168+0000 7ff4da800640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d6c88388c0 0x55d6c88361e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.83:53144/0 (socket says 172.21.15.83:53144) 2024-08-19T01:41:02.203 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:02.168+0000 7ff4da800640 1 -- 172.21.15.83:0/1381692742 learned_addr learned my addr 172.21.15.83:0/1381692742 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-08-19T01:41:02.203 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:02.168+0000 7ff4d9400640 1 --2- 172.21.15.83:0/1381692742 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d6c882e000 0x55d6c8835ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:41:02.203 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:02.168+0000 7ff4da800640 1 -- 172.21.15.83:0/1381692742 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d6c882d6e0 msgr2=0x55d6c8835760 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-08-19T01:41:02.203 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:02.168+0000 7ff4da800640 1 --2- 172.21.15.83:0/1381692742 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d6c882d6e0 0x55d6c8835760 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:41:02.203 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:02.168+0000 7ff4da800640 1 -- 172.21.15.83:0/1381692742 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d6c882e000 msgr2=0x55d6c8835ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:41:02.203 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:02.168+0000 7ff4da800640 1 --2- 172.21.15.83:0/1381692742 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d6c882e000 0x55d6c8835ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:41:02.203 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:02.168+0000 7ff4da800640 1 -- 172.21.15.83:0/1381692742 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d6c85043c0 con 0x55d6c88388c0 2024-08-19T01:41:02.203 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:02.169+0000 7ff4da800640 1 --2- 172.21.15.83:0/1381692742 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d6c88388c0 0x55d6c88361e0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7ff4dc000b60 tx=0x7ff4dc06bb70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=33367e26304ea0d0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:41:02.203 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:02.169+0000 7ff4bbe00640 10 client.0 ms_handle_connect on v2:172.21.15.167:3300/0 2024-08-19T01:41:02.203 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:02.169+0000 7ff4bbe00640 1 -- 172.21.15.83:0/1381692742 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4dc097c40 con 0x55d6c88388c0 2024-08-19T01:41:02.203 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:02.169+0000 7ff4bbe00640 1 -- 172.21.15.83:0/1381692742 <== mon.1 v2:172.21.15.167:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8269+0+0 (secure 0 0 0) 0x7ff4dc097de0 con 0x55d6c88388c0 2024-08-19T01:41:02.204 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:02.170+0000 7ff4e3c37740 1 -- 172.21.15.83:0/1381692742 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff4c8005500 con 0x55d6c88388c0 2024-08-19T01:41:02.204 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:02.170+0000 7ff4bbe00640 1 -- 172.21.15.83:0/1381692742 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7ff4dc099070 con 0x55d6c88388c0 2024-08-19T01:41:02.204 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:02.170+0000 7ff4e3c37740 10 client.25789 fetch_fsmap learned FSMap version 27 2024-08-19T01:41:02.204 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:02.170+0000 7ff4e3c37740 1 -- 172.21.15.83:0/1381692742 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55d6c8847180 con 0x55d6c88388c0 2024-08-19T01:41:02.204 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:02.171+0000 7ff4bbe00640 1 -- 172.21.15.83:0/1381692742 <== mon.1 v2:172.21.15.167:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff4c8005500 con 0x55d6c88388c0 2024-08-19T01:41:02.204 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:02.172+0000 7ff4bbe00640 1 -- 172.21.15.83:0/1381692742 <== mon.1 v2:172.21.15.167:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7ff4dc0a0840 con 0x55d6c88388c0 2024-08-19T01:41:02.204 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:02.172+0000 7ff4e3c37740 10 client.25789 fetch_fsmap finished waiting for FSMap version 27 2024-08-19T01:41:02.204 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:02.172+0000 7ff4e3c37740 10 client.25789 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.g 2024-08-19T01:41:02.204 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:02.172+0000 7ff4e3c37740 20 client.25789 populate_metadata read hostname 'smithi083' 2024-08-19T01:41:02.204 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:02.173+0000 7ff4e3c37740 1 --2- 172.21.15.83:0/1381692742 >> [v2:172.21.15.83:6838/3239261845,v1:172.21.15.83:6839/3239261845] conn(0x55d6c8848a60 0x55d6c884aef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:41:02.204 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:02.173+0000 7ff4e3c37740 4 client.25789 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-19T01:41:02.204 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:02.173+0000 7ff4e3c37740 1 -- 172.21.15.83:0/1381692742 --> [v2:172.21.15.83:6838/3239261845,v1:172.21.15.83:6839/3239261845] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d6c87a0160 con 0x55d6c8848a60 2024-08-19T01:41:02.204 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:02.180+0000 7ff4d9e00640 1 --2- 172.21.15.83:0/1381692742 >> [v2:172.21.15.83:6838/3239261845,v1:172.21.15.83:6839/3239261845] conn(0x55d6c8848a60 0x55d6c884aef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:41:02.205 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:02.185+0000 7ff4d9e00640 1 --2- 172.21.15.83:0/1381692742 >> [v2:172.21.15.83:6838/3239261845,v1:172.21.15.83:6839/3239261845] conn(0x55d6c8848a60 0x55d6c884aef0 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=8ab4ae068d49fb33 server_cookie=461c61d7ca46dd4e in_seq=0 out_seq=0 2024-08-19T01:41:02.205 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:02.186+0000 7ff4bbe00640 10 client.25789 ms_handle_connect on v2:172.21.15.83:6838/3239261845 2024-08-19T01:41:02.205 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:02.190+0000 7ff4bbe00640 1 -- 172.21.15.83:0/1381692742 <== mds.1 v2:172.21.15.83:6838/3239261845 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55d6c87a0160 con 0x55d6c8848a60 2024-08-19T01:41:02.205 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:02.190+0000 7ff4bbe00640 10 client.25789 handle_command_reply: tid=0 2024-08-19T01:41:02.205 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:02.191+0000 7ff4e3c37740 1 client.25789 shutdown 2024-08-19T01:41:02.205 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:02.191+0000 7ff4e3c37740 1 -- 172.21.15.83:0/1381692742 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d6c88388c0 msgr2=0x55d6c88361e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:41:02.205 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:02.191+0000 7ff4e3c37740 1 --2- 172.21.15.83:0/1381692742 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d6c88388c0 0x55d6c88361e0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7ff4dc000b60 tx=0x7ff4dc06bb70 comp rx=0 tx=0).stop 2024-08-19T01:41:02.205 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:02.192+0000 7ff4e3c37740 1 -- 172.21.15.83:0/1381692742 shutdown_connections 2024-08-19T01:41:02.205 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:02.192+0000 7ff4e3c37740 1 --2- 172.21.15.83:0/1381692742 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d6c88388c0 0x55d6c88361e0 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:41:02.205 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:02.192+0000 7ff4e3c37740 1 --2- 172.21.15.83:0/1381692742 >> [v2:172.21.15.83:6838/3239261845,v1:172.21.15.83:6839/3239261845] conn(0x55d6c8848a60 0x55d6c884aef0 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:41:02.205 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:02.192+0000 7ff4e3c37740 1 --2- 172.21.15.83:0/1381692742 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d6c882e000 0x55d6c8835ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:41:02.205 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:02.192+0000 7ff4e3c37740 1 --2- 172.21.15.83:0/1381692742 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d6c882d6e0 0x55d6c8835760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:41:02.205 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:02.192+0000 7ff4e3c37740 1 -- 172.21.15.83:0/1381692742 >> 172.21.15.83:0/1381692742 conn(0x55d6c8829c70 msgr2=0x55d6c882f950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:41:02.206 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:02.192+0000 7ff4bbe00640 0 client.25789 ms_handle_reset on v2:172.21.15.83:6838/3239261845 2024-08-19T01:41:02.206 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:02.193+0000 7ff4e3c37740 1 -- 172.21.15.83:0/1381692742 shutdown_connections 2024-08-19T01:41:02.206 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:02.193+0000 7ff4e3c37740 1 -- 172.21.15.83:0/1381692742 wait complete. 2024-08-19T01:41:02.206 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:02.193+0000 7ff4e3c37740 20 client.25789 trim_cache size 0 max 16384 2024-08-19T01:41:02.223 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:02.221+0000 7ff4e3c37740 1 Processor -- start 2024-08-19T01:41:02.223 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:02.221+0000 7ff4e3c37740 1 -- start start 2024-08-19T01:41:02.224 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:02.222+0000 7ff4e3c37740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d6c882d6e0 0x55d6c8824910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:41:02.224 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:02.222+0000 7ff4e3c37740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d6c882e000 0x55d6c8832a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:41:02.224 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:02.222+0000 7ff4e3c37740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d6c88388c0 0x55d6c8832f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:41:02.224 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:02.222+0000 7ff4e3c37740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7ff4d4174880 con 0x55d6c882d6e0 2024-08-19T01:41:02.224 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:02.222+0000 7ff4e3c37740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7ff4d41721f0 con 0x55d6c882e000 2024-08-19T01:41:02.224 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:02.222+0000 7ff4e3c37740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7ff4d4006d20 con 0x55d6c88388c0 2024-08-19T01:41:02.224 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:02.223+0000 7ff4da800640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d6c88388c0 0x55d6c8832f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:41:02.225 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:02.223+0000 7ff4da800640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d6c88388c0 0x55d6c8832f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.83:53152/0 (socket says 172.21.15.83:53152) 2024-08-19T01:41:02.226 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:02.223+0000 7ff4da800640 1 -- 172.21.15.83:0/4208674931 learned_addr learned my addr 172.21.15.83:0/4208674931 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-08-19T01:41:02.230 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:02.223+0000 7ff4d9e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d6c882d6e0 0x55d6c8824910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:41:02.230 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:02.223+0000 7ff4d9400640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d6c882e000 0x55d6c8832a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:41:02.231 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:02.224+0000 7ff4da800640 1 -- 172.21.15.83:0/4208674931 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d6c882e000 msgr2=0x55d6c8832a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:41:02.231 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:02.224+0000 7ff4da800640 1 --2- 172.21.15.83:0/4208674931 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d6c882e000 0x55d6c8832a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:41:02.231 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:02.224+0000 7ff4da800640 1 -- 172.21.15.83:0/4208674931 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d6c882d6e0 msgr2=0x55d6c8824910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:41:02.232 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:02.224+0000 7ff4da800640 1 --2- 172.21.15.83:0/4208674931 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d6c882d6e0 0x55d6c8824910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:41:02.232 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:02.224+0000 7ff4da800640 1 -- 172.21.15.83:0/4208674931 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d6c883aff0 con 0x55d6c88388c0 2024-08-19T01:41:02.233 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:02.226+0000 7ff4da800640 1 --2- 172.21.15.83:0/4208674931 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d6c88388c0 0x55d6c8832f40 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7ff4dc0515e0 tx=0x7ff4dc0634a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5027c9ef07d704e7 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:41:02.234 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:02.226+0000 7ff4c6a00640 1 -- 172.21.15.83:0/4208674931 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4dc06bbe0 con 0x55d6c88388c0 2024-08-19T01:41:02.235 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:02.226+0000 7ff4c6a00640 1 -- 172.21.15.83:0/4208674931 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff4dc09b070 con 0x55d6c88388c0 2024-08-19T01:41:02.235 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:02.226+0000 7ff4c6a00640 1 -- 172.21.15.83:0/4208674931 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4dc099470 con 0x55d6c88388c0 2024-08-19T01:41:02.235 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:02.227+0000 7ff4e3c37740 1 -- 172.21.15.83:0/4208674931 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d6c88388c0 msgr2=0x55d6c8832f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:41:02.235 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:02.227+0000 7ff4e3c37740 1 --2- 172.21.15.83:0/4208674931 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d6c88388c0 0x55d6c8832f40 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7ff4dc0515e0 tx=0x7ff4dc0634a0 comp rx=0 tx=0).stop 2024-08-19T01:41:02.236 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:02.228+0000 7ff4e3c37740 1 -- 172.21.15.83:0/4208674931 shutdown_connections 2024-08-19T01:41:02.236 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:02.228+0000 7ff4e3c37740 1 --2- 172.21.15.83:0/4208674931 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d6c88388c0 0x55d6c8832f40 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:41:02.236 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:02.228+0000 7ff4e3c37740 1 --2- 172.21.15.83:0/4208674931 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d6c882e000 0x55d6c8832a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:41:02.237 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:02.228+0000 7ff4e3c37740 1 --2- 172.21.15.83:0/4208674931 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d6c882d6e0 0x55d6c8824910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:41:02.238 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:02.228+0000 7ff4e3c37740 1 -- 172.21.15.83:0/4208674931 >> 172.21.15.83:0/4208674931 conn(0x55d6c8829c70 msgr2=0x55d6c8834050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:41:02.238 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:02.228+0000 7ff4e3c37740 1 -- 172.21.15.83:0/4208674931 shutdown_connections 2024-08-19T01:41:02.238 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:02.228+0000 7ff4e3c37740 1 -- 172.21.15.83:0/4208674931 wait complete. 2024-08-19T01:41:02.238 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:02.229+0000 7ff4e3c37740 1 -- start start 2024-08-19T01:41:02.238 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:02.230+0000 7ff4e3c37740 1 Processor -- start 2024-08-19T01:41:02.238 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:02.231+0000 7ff4e3c37740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d6c882d6e0 0x55d6c884d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:41:02.238 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:02.232+0000 7ff4e3c37740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d6c882e000 0x55d6c884da20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:41:02.239 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:02.232+0000 7ff4e3c37740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d6c88388c0 0x55d6c884df60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:41:02.240 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:02.232+0000 7ff4e3c37740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7ff4d40ad580 con 0x55d6c882d6e0 2024-08-19T01:41:02.240 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:02.232+0000 7ff4e3c37740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7ff4d40ae9f0 con 0x55d6c882e000 2024-08-19T01:41:02.241 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:02.232+0000 7ff4e3c37740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7ff4d40adc50 con 0x55d6c88388c0 2024-08-19T01:41:02.241 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:02.232+0000 7ff4e3c37740 10 client.0 _ll_register_callbacks cb 0x7ff4dc099ee0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-19T01:41:02.245 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:02.233+0000 7ff4da800640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d6c88388c0 0x55d6c884df60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:41:02.247 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:02.233+0000 7ff4d9400640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d6c882e000 0x55d6c884da20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:41:02.248 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:02.233+0000 7ff4d9400640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d6c882e000 0x55d6c884da20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.83:42866/0 (socket says 172.21.15.83:42866) 2024-08-19T01:41:02.248 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:02.233+0000 7ff4d9400640 1 -- 172.21.15.83:0/2641389752 learned_addr learned my addr 172.21.15.83:0/2641389752 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-08-19T01:41:02.249 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:02.233+0000 7ff4d9e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d6c882d6e0 0x55d6c884d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:41:02.249 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:02.234+0000 7ff4d9400640 1 -- 172.21.15.83:0/2641389752 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d6c88388c0 msgr2=0x55d6c884df60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:41:02.249 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:02.234+0000 7ff4d9400640 1 --2- 172.21.15.83:0/2641389752 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d6c88388c0 0x55d6c884df60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:41:02.249 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:02.234+0000 7ff4d9400640 1 -- 172.21.15.83:0/2641389752 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d6c882d6e0 msgr2=0x55d6c884d4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:41:02.249 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:02.234+0000 7ff4d9400640 1 --2- 172.21.15.83:0/2641389752 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d6c882d6e0 0x55d6c884d4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:41:02.249 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:02.234+0000 7ff4d9400640 1 -- 172.21.15.83:0/2641389752 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d6c88b1770 con 0x55d6c882e000 2024-08-19T01:41:02.249 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:02.234+0000 7ff4da800640 1 --2- 172.21.15.83:0/2641389752 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d6c88388c0 0x55d6c884df60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-19T01:41:02.249 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:02.235+0000 7ff4d9e00640 1 --2- 172.21.15.83:0/2641389752 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d6c882d6e0 0x55d6c884d4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-19T01:41:02.250 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:02.235+0000 7ff4d9400640 1 --2- 172.21.15.83:0/2641389752 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d6c882e000 0x55d6c884da20 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7ff4cc009610 tx=0x7ff4cc005880 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bac0d60aa9484cef server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:41:02.250 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:02.235+0000 7ff4bbe00640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-08-19T01:41:02.250 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:02.236+0000 7ff4bbe00640 1 -- 172.21.15.83:0/2641389752 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4cc0b3020 con 0x55d6c882e000 2024-08-19T01:41:02.250 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:02.236+0000 7ff4bbe00640 1 -- 172.21.15.83:0/2641389752 <== mon.2 v2:172.21.15.117:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8269+0+0 (secure 0 0 0) 0x7ff4cc0109d0 con 0x55d6c882e000 2024-08-19T01:41:02.250 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:02.236+0000 7ff4e3c37740 1 -- 172.21.15.83:0/2641389752 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff4dc09b070 con 0x55d6c882e000 2024-08-19T01:41:02.250 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:02.237+0000 7ff4bbe00640 1 -- 172.21.15.83:0/2641389752 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7ff4cc020040 con 0x55d6c882e000 2024-08-19T01:41:02.256 INFO:teuthology.orchestra.run.smithi083.stdout:[] 2024-08-19T01:41:02.256 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:02.237+0000 7ff4e3c37740 10 client.25865 fetch_fsmap learned FSMap version 27 2024-08-19T01:41:02.256 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:02.237+0000 7ff4e3c37740 1 -- 172.21.15.83:0/2641389752 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55d6c88b2a50 con 0x55d6c882e000 2024-08-19T01:41:02.256 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:02.238+0000 7ff4bbe00640 1 -- 172.21.15.83:0/2641389752 <== mon.2 v2:172.21.15.117:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff4dc09b070 con 0x55d6c882e000 2024-08-19T01:41:02.256 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:02.238+0000 7ff4bbe00640 1 -- 172.21.15.83:0/2641389752 <== mon.2 v2:172.21.15.117:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7ff4cc09a8e0 con 0x55d6c882e000 2024-08-19T01:41:02.257 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:02.238+0000 7ff4e3c37740 10 client.25865 fetch_fsmap finished waiting for FSMap version 27 2024-08-19T01:41:02.257 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:02.238+0000 7ff4e3c37740 10 client.25865 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.g 2024-08-19T01:41:02.257 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:02.238+0000 7ff4e3c37740 20 client.25865 populate_metadata read hostname 'smithi083' 2024-08-19T01:41:02.257 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:02.238+0000 7ff4e3c37740 1 --2- 172.21.15.83:0/2641389752 >> [v2:172.21.15.83:6838/3239261845,v1:172.21.15.83:6839/3239261845] conn(0x55d6c88b31b0 0x55d6c88b55d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:41:02.257 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:02.238+0000 7ff4e3c37740 4 client.25865 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-08-19T01:41:02.257 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:02.238+0000 7ff4e3c37740 1 -- 172.21.15.83:0/2641389752 --> [v2:172.21.15.83:6838/3239261845,v1:172.21.15.83:6839/3239261845] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55d6c879a670 con 0x55d6c88b31b0 2024-08-19T01:41:02.257 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:02.239+0000 7ff4d9e00640 1 --2- 172.21.15.83:0/2641389752 >> [v2:172.21.15.83:6838/3239261845,v1:172.21.15.83:6839/3239261845] conn(0x55d6c88b31b0 0x55d6c88b55d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:41:02.257 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:02.240+0000 7ff4d9e00640 1 --2- 172.21.15.83:0/2641389752 >> [v2:172.21.15.83:6838/3239261845,v1:172.21.15.83:6839/3239261845] conn(0x55d6c88b31b0 0x55d6c88b55d0 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=ca0aca9a6cceffe5 server_cookie=b74177fc7955ada7 in_seq=0 out_seq=0 2024-08-19T01:41:02.257 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:02.240+0000 7ff4bbe00640 10 client.25865 ms_handle_connect on v2:172.21.15.83:6838/3239261845 2024-08-19T01:41:02.257 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:02.241+0000 7ff4bbe00640 1 -- 172.21.15.83:0/2641389752 <== mds.1 v2:172.21.15.83:6838/3239261845 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55d6c879a670 con 0x55d6c88b31b0 2024-08-19T01:41:02.257 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:02.241+0000 7ff4bbe00640 10 client.25865 handle_command_reply: tid=0 2024-08-19T01:41:02.257 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:02.241+0000 7ff4e3c37740 1 client.25865 shutdown 2024-08-19T01:41:02.257 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:02.241+0000 7ff4e3c37740 1 -- 172.21.15.83:0/2641389752 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d6c882e000 msgr2=0x55d6c884da20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:41:02.258 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:02.242+0000 7ff4e3c37740 1 --2- 172.21.15.83:0/2641389752 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d6c882e000 0x55d6c884da20 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7ff4cc009610 tx=0x7ff4cc005880 comp rx=0 tx=0).stop 2024-08-19T01:41:02.258 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:02.242+0000 7ff4e3c37740 1 -- 172.21.15.83:0/2641389752 shutdown_connections 2024-08-19T01:41:02.258 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:02.242+0000 7ff4e3c37740 1 --2- 172.21.15.83:0/2641389752 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d6c88388c0 0x55d6c884df60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:41:02.258 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:02.242+0000 7ff4e3c37740 1 --2- 172.21.15.83:0/2641389752 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d6c882e000 0x55d6c884da20 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:41:02.258 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:02.242+0000 7ff4e3c37740 1 --2- 172.21.15.83:0/2641389752 >> [v2:172.21.15.83:6838/3239261845,v1:172.21.15.83:6839/3239261845] conn(0x55d6c88b31b0 0x55d6c88b55d0 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:41:02.258 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:02.242+0000 7ff4e3c37740 1 --2- 172.21.15.83:0/2641389752 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d6c882d6e0 0x55d6c884d4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:41:02.258 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:02.242+0000 7ff4e3c37740 1 -- 172.21.15.83:0/2641389752 >> 172.21.15.83:0/2641389752 conn(0x55d6c8829c70 msgr2=0x55d6c88438f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:41:02.258 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:02.242+0000 7ff4bbe00640 0 client.25865 ms_handle_reset on v2:172.21.15.83:6838/3239261845 2024-08-19T01:41:02.258 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:02.242+0000 7ff4e3c37740 1 -- 172.21.15.83:0/2641389752 shutdown_connections 2024-08-19T01:41:02.258 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:02.242+0000 7ff4e3c37740 1 -- 172.21.15.83:0/2641389752 wait complete. 2024-08-19T01:41:02.258 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:02.246+0000 7ff4e3c37740 20 client.25865 trim_cache size 0 max 16384 2024-08-19T01:41:02.258 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:02.247+0000 7ff4db200640 1 -- 172.21.15.83:0/4099916028 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7ff4c00a2c10 msgr2=0x7ff4c00a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:41:02.258 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:02.247+0000 7ff4db200640 1 --2- 172.21.15.83:0/4099916028 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7ff4c00a2c10 0x7ff4c00a50d0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7ff4c8005dc0 tx=0x7ff4c8009570 comp rx=0 tx=0).stop 2024-08-19T01:41:02.258 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:02.247+0000 7ff4db200640 1 -- 172.21.15.83:0/4099916028 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff4d4007260 msgr2=0x7ff4d40a0b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:41:02.259 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:02.247+0000 7ff4db200640 1 --2- 172.21.15.83:0/4099916028 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff4d4007260 0x7ff4d40a0b50 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7ff4cc00d950 tx=0x7ff4cc00de10 comp rx=0 tx=0).stop 2024-08-19T01:41:02.259 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:02.247+0000 7ff4db200640 1 -- 172.21.15.83:0/4099916028 shutdown_connections 2024-08-19T01:41:02.259 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:02.247+0000 7ff4db200640 1 --2- 172.21.15.83:0/4099916028 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7ff4c00a2c10 0x7ff4c00a50d0 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:41:02.259 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:02.247+0000 7ff4db200640 1 --2- 172.21.15.83:0/4099916028 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff4d40a2060 0x7ff4d4148f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:41:02.259 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:02.247+0000 7ff4db200640 1 --2- 172.21.15.83:0/4099916028 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff4d4007260 0x7ff4d40a0b50 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:41:02.259 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:02.247+0000 7ff4db200640 1 --2- 172.21.15.83:0/4099916028 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff4d4006940 0x7ff4d40a0610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:41:02.259 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:02.247+0000 7ff4db200640 1 -- 172.21.15.83:0/4099916028 >> 172.21.15.83:0/4099916028 conn(0x7ff4d400c9d0 msgr2=0x7ff4d4093c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:41:02.259 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:02.247+0000 7ff4db200640 1 -- 172.21.15.83:0/4099916028 shutdown_connections 2024-08-19T01:41:02.259 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:02.248+0000 7ff4db200640 1 -- 172.21.15.83:0/4099916028 wait complete. 2024-08-19T01:41:03.916 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:41:03 smithi083 ceph-mon[17120]: pgmap v525: 97 pgs: 97 active+clean; 639 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.4 MiB/s wr, 224 op/s 2024-08-19T01:41:03.952 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:41:03 smithi117 ceph-mon[22016]: pgmap v525: 97 pgs: 97 active+clean; 639 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.4 MiB/s wr, 224 op/s 2024-08-19T01:41:03.965 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:41:03 smithi167 ceph-mon[21875]: pgmap v525: 97 pgs: 97 active+clean; 639 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.4 MiB/s wr, 224 op/s 2024-08-19T01:41:04.591 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833 shell --fsid bad8780e-5dc9-11ef-bccf-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-08-19T01:41:04.804 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:41:04.801+0000 7fc2eaa00640 1 -- 172.21.15.83:0/3725806129 <== mds.0 v2:172.21.15.117:6838/1967595704 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x556e02901310 con 0x556e02a26cb0 2024-08-19T01:41:04.804 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:41:04.801+0000 7fc2eaa00640 10 client.15810 handle_command_reply: tid=0 2024-08-19T01:41:04.804 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:41:04.801+0000 7fc30a886740 1 client.15810 shutdown 2024-08-19T01:41:04.809 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:41:04.809+0000 7fc30a886740 1 -- 172.21.15.83:0/3725806129 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556e02995db0 msgr2=0x556e029b4820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:41:04.809 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:41:04.809+0000 7fc30a886740 1 --2- 172.21.15.83:0/3725806129 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556e02995db0 0x556e029b4820 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7fc3001a7b90 tx=0x7fc2ec0039c0 comp rx=0 tx=0).stop 2024-08-19T01:41:04.839 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:41:04.839+0000 7fc30a886740 1 -- 172.21.15.83:0/3725806129 shutdown_connections 2024-08-19T01:41:04.840 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:41:04.839+0000 7fc30a886740 1 --2- 172.21.15.83:0/3725806129 >> [v2:172.21.15.117:6838/1967595704,v1:172.21.15.117:6839/1967595704] conn(0x556e02a26cb0 0x556e02a29170 crc :-1 s=READY pgs=108 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:41:04.840 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:41:04.839+0000 7fc30a886740 1 --2- 172.21.15.83:0/3725806129 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556e0299ae40 0x556e029b4dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:41:04.840 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:41:04.839+0000 7fc30a886740 1 --2- 172.21.15.83:0/3725806129 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556e02995db0 0x556e029b4820 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:41:04.840 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:41:04.839+0000 7fc30a886740 1 --2- 172.21.15.83:0/3725806129 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556e02995470 0x556e029b7ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:41:04.840 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:41:04.839+0000 7fc30a886740 1 -- 172.21.15.83:0/3725806129 >> 172.21.15.83:0/3725806129 conn(0x556e02990ae0 msgr2=0x556e02990ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:41:04.840 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:41:04.839+0000 7fc2eaa00640 0 client.15810 ms_handle_reset on v2:172.21.15.117:6838/1967595704 2024-08-19T01:41:04.890 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:41:04.878+0000 7fc30a886740 1 -- 172.21.15.83:0/3725806129 shutdown_connections 2024-08-19T01:41:04.890 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:41:04.890+0000 7fc30a886740 1 -- 172.21.15.83:0/3725806129 wait complete. 2024-08-19T01:41:04.894 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:41:04.890+0000 7fc30a886740 20 client.15810 trim_cache size 0 max 16384 2024-08-19T01:41:04.965 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stdout:{ 2024-08-19T01:41:04.965 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stdout: "message": "", 2024-08-19T01:41:04.965 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stdout: "return_code": 0 2024-08-19T01:41:04.965 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stdout:} 2024-08-19T01:41:04.976 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:41:04.976+0000 7fc307a00640 1 -- 172.21.15.83:0/2999308539 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7fc2e40a2e20 msgr2=0x7fc2e40a52e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:41:04.977 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:41:04.976+0000 7fc307a00640 1 --2- 172.21.15.83:0/2999308539 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7fc2e40a2e20 0x7fc2e40a52e0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fc2f8009370 tx=0x7fc2f800b040 comp rx=0 tx=0).stop 2024-08-19T01:41:04.977 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:41:04.976+0000 7fc307a00640 1 -- 172.21.15.83:0/2999308539 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc300072a50 msgr2=0x7fc3001a6680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:41:04.977 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:41:04.976+0000 7fc307a00640 1 --2- 172.21.15.83:0/2999308539 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc300072a50 0x7fc3001a6680 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7fc2f000da90 tx=0x7fc2f000df50 comp rx=0 tx=0).stop 2024-08-19T01:41:04.979 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:41:04.979+0000 7fc307a00640 1 -- 172.21.15.83:0/2999308539 shutdown_connections 2024-08-19T01:41:04.980 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:41:04.979+0000 7fc307a00640 1 --2- 172.21.15.83:0/2999308539 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7fc2e40a2e20 0x7fc2e40a52e0 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:41:04.980 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:41:04.979+0000 7fc307a00640 1 --2- 172.21.15.83:0/2999308539 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc300109430 0x7fc3001adc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:41:04.980 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:41:04.979+0000 7fc307a00640 1 --2- 172.21.15.83:0/2999308539 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc3000733f0 0x7fc3001a6bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:41:04.980 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:41:04.979+0000 7fc307a00640 1 --2- 172.21.15.83:0/2999308539 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc300072a50 0x7fc3001a6680 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:41:04.980 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:41:04.979+0000 7fc307a00640 1 -- 172.21.15.83:0/2999308539 >> 172.21.15.83:0/2999308539 conn(0x7fc30006ea10 msgr2=0x7fc300071200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:41:04.989 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:41:04.988+0000 7fc307a00640 1 -- 172.21.15.83:0/2999308539 shutdown_connections 2024-08-19T01:41:05.019 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:41:05.018+0000 7fc307a00640 1 -- 172.21.15.83:0/2999308539 wait complete. 2024-08-19T01:41:05.789 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:41:05 smithi083 ceph-mon[17120]: pgmap v526: 97 pgs: 97 active+clean; 642 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 2.7 MiB/s wr, 943 op/s 2024-08-19T01:41:05.952 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:41:05 smithi117 ceph-mon[22016]: pgmap v526: 97 pgs: 97 active+clean; 642 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 2.7 MiB/s wr, 943 op/s 2024-08-19T01:41:05.965 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:41:05 smithi167 ceph-mon[21875]: pgmap v526: 97 pgs: 97 active+clean; 642 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 2.7 MiB/s wr, 943 op/s 2024-08-19T01:41:07.952 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:41:07 smithi117 ceph-mon[22016]: pgmap v527: 97 pgs: 97 active+clean; 633 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.9 MiB/s wr, 1.17k op/s 2024-08-19T01:41:07.966 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:41:07 smithi167 ceph-mon[21875]: pgmap v527: 97 pgs: 97 active+clean; 633 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.9 MiB/s wr, 1.17k op/s 2024-08-19T01:41:07.983 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:41:07 smithi083 ceph-mon[17120]: pgmap v527: 97 pgs: 97 active+clean; 633 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.9 MiB/s wr, 1.17k op/s 2024-08-19T01:41:09.952 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:41:09 smithi117 ceph-mon[22016]: pgmap v528: 97 pgs: 97 active+clean; 610 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.3 MiB/s wr, 1.31k op/s 2024-08-19T01:41:09.965 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:41:09 smithi167 ceph-mon[21875]: pgmap v528: 97 pgs: 97 active+clean; 610 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.3 MiB/s wr, 1.31k op/s 2024-08-19T01:41:10.132 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:41:09 smithi083 ceph-mon[17120]: pgmap v528: 97 pgs: 97 active+clean; 610 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.3 MiB/s wr, 1.31k op/s 2024-08-19T01:41:10.735 INFO:tasks.workunit.client.0.smithi083.stdout: 214 537516 575 360896 425 318881 1303 2024-08-19T01:41:10.874 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/bad8780e-5dc9-11ef-bccf-c7b262605968/mon.a/config 2024-08-19T01:41:11.555 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:11.539+0000 7f08c4a00640 1 -- 172.21.15.83:0/2409102443 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f08c0109900 msgr2=0x7f08c0109ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:41:11.555 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:11.539+0000 7f08c4a00640 1 --2- 172.21.15.83:0/2409102443 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f08c0109900 0x7f08c0109ce0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f08b0009900 tx=0x7f08b002a3f0 comp rx=0 tx=0).stop 2024-08-19T01:41:11.556 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:11.539+0000 7f08c4a00640 1 -- 172.21.15.83:0/2409102443 shutdown_connections 2024-08-19T01:41:11.556 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:11.539+0000 7f08c4a00640 1 --2- 172.21.15.83:0/2409102443 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f08c01042a0 0x7f08c01107d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:41:11.556 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:11.539+0000 7f08c4a00640 1 --2- 172.21.15.83:0/2409102443 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f08c0103900 0x7f08c0103d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:41:11.556 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:11.539+0000 7f08c4a00640 1 --2- 172.21.15.83:0/2409102443 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f08c0109900 0x7f08c0109ce0 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:41:11.556 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:11.539+0000 7f08c4a00640 1 -- 172.21.15.83:0/2409102443 >> 172.21.15.83:0/2409102443 conn(0x7f08c00ff600 msgr2=0x7f08c0101a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:41:11.556 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:11.539+0000 7f08c4a00640 1 -- 172.21.15.83:0/2409102443 shutdown_connections 2024-08-19T01:41:11.556 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:11.539+0000 7f08c4a00640 1 -- 172.21.15.83:0/2409102443 wait complete. 2024-08-19T01:41:11.556 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:11.540+0000 7f08c4a00640 1 Processor -- start 2024-08-19T01:41:11.556 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:11.540+0000 7f08c4a00640 1 -- start start 2024-08-19T01:41:11.556 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:11.541+0000 7f08c4a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f08c0103900 0x7f08c019a200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:41:11.556 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:11.541+0000 7f08c4a00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f08c01042a0 0x7f08c019a740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:41:11.556 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:11.541+0000 7f08c4a00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f08c0109900 0x7f08c019ead0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:41:11.556 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:11.541+0000 7f08c4a00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f08c0113d20 con 0x7f08c0103900 2024-08-19T01:41:11.557 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:11.541+0000 7f08c4a00640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f08c0113ba0 con 0x7f08c0109900 2024-08-19T01:41:11.557 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:11.541+0000 7f08c4a00640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f08c0113ea0 con 0x7f08c01042a0 2024-08-19T01:41:11.557 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:11.541+0000 7f08b7e00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f08c01042a0 0x7f08c019a740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:41:11.557 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:11.541+0000 7f08bf400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f08c0103900 0x7f08c019a200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:41:11.557 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:11.541+0000 7f08bfe00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f08c0109900 0x7f08c019ead0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:41:11.557 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:11.541+0000 7f08b7e00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f08c01042a0 0x7f08c019a740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.83:44798/0 (socket says 172.21.15.83:44798) 2024-08-19T01:41:11.557 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:11.541+0000 7f08b7e00640 1 -- 172.21.15.83:0/188556475 learned_addr learned my addr 172.21.15.83:0/188556475 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-08-19T01:41:11.557 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:11.541+0000 7f08bfe00640 1 -- 172.21.15.83:0/188556475 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f08c01042a0 msgr2=0x7f08c019a740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:41:11.557 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:11.541+0000 7f08bfe00640 1 --2- 172.21.15.83:0/188556475 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f08c01042a0 0x7f08c019a740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:41:11.557 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:11.541+0000 7f08bfe00640 1 -- 172.21.15.83:0/188556475 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f08c0103900 msgr2=0x7f08c019a200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:41:11.557 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:11.541+0000 7f08bfe00640 1 --2- 172.21.15.83:0/188556475 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f08c0103900 0x7f08c019a200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:41:11.557 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:11.541+0000 7f08bfe00640 1 -- 172.21.15.83:0/188556475 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f08c019f1b0 con 0x7f08c0109900 2024-08-19T01:41:11.557 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:11.541+0000 7f08bf400640 1 --2- 172.21.15.83:0/188556475 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f08c0103900 0x7f08c019a200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-19T01:41:11.557 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:11.542+0000 7f08bfe00640 1 --2- 172.21.15.83:0/188556475 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f08c0109900 0x7f08c019ead0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f08b800da10 tx=0x7f08b800ded0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:41:11.558 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:11.542+0000 7f08bcc00640 1 -- 172.21.15.83:0/188556475 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08b800bd70 con 0x7f08c0109900 2024-08-19T01:41:11.558 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:11.542+0000 7f08c4a00640 1 -- 172.21.15.83:0/188556475 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f08c019f4a0 con 0x7f08c0109900 2024-08-19T01:41:11.558 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:11.542+0000 7f08bcc00640 1 -- 172.21.15.83:0/188556475 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f08b8003e50 con 0x7f08c0109900 2024-08-19T01:41:11.558 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:11.542+0000 7f08bcc00640 1 -- 172.21.15.83:0/188556475 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08b80106a0 con 0x7f08c0109900 2024-08-19T01:41:11.558 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:11.542+0000 7f08c4a00640 1 -- 172.21.15.83:0/188556475 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f08c01a6d80 con 0x7f08c0109900 2024-08-19T01:41:11.558 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:11.545+0000 7f08c8c80740 1 Processor -- start 2024-08-19T01:41:11.558 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:11.545+0000 7f08bcc00640 1 -- 172.21.15.83:0/188556475 <== mon.2 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7f08b8003970 con 0x7f08c0109900 2024-08-19T01:41:11.558 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:11.545+0000 7f08c8c80740 1 -- start start 2024-08-19T01:41:11.558 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:11.546+0000 7f08c8c80740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b3b22336e0 0x55b3b2233ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:41:11.558 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:11.546+0000 7f08c8c80740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b3b2234000 0x55b3b223e380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:41:11.558 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:11.546+0000 7f08c8c80740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b3b223e8c0 0x55b3b2240cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:41:11.558 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:11.546+0000 7f08c8c80740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55b3b2198bf0 con 0x55b3b2234000 2024-08-19T01:41:11.558 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:11.546+0000 7f08c8c80740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55b3b2130b20 con 0x55b3b22336e0 2024-08-19T01:41:11.559 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:11.546+0000 7f08c8c80740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55b3b2194bb0 con 0x55b3b223e8c0 2024-08-19T01:41:11.559 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:11.546+0000 7f08bf400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b3b2234000 0x55b3b223e380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:41:11.559 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:11.546+0000 7f08b7e00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b3b223e8c0 0x55b3b2240cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:41:11.559 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:11.546+0000 7f08bf400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b3b2234000 0x55b3b223e380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:57052/0 (socket says 172.21.15.83:57052) 2024-08-19T01:41:11.559 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:11.546+0000 7f08bf400640 1 -- 172.21.15.83:0/3822392296 learned_addr learned my addr 172.21.15.83:0/3822392296 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-08-19T01:41:11.559 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:11.546+0000 7f08b7e00640 1 --2- 172.21.15.83:0/3822392296 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b3b22336e0 0x55b3b2233ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:41:11.559 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:11.546+0000 7f08bf400640 1 -- 172.21.15.83:0/3822392296 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b3b223e8c0 msgr2=0x55b3b2240cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:41:11.559 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:11.547+0000 7f08bf400640 1 --2- 172.21.15.83:0/3822392296 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b3b223e8c0 0x55b3b2240cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:41:11.559 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:11.547+0000 7f08bf400640 1 -- 172.21.15.83:0/3822392296 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b3b22336e0 msgr2=0x55b3b2233ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:41:11.559 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:11.547+0000 7f08bf400640 1 --2- 172.21.15.83:0/3822392296 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b3b22336e0 0x55b3b2233ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:41:11.559 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:11.547+0000 7f08bf400640 1 -- 172.21.15.83:0/3822392296 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b3b21377f0 con 0x55b3b2234000 2024-08-19T01:41:11.559 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:11.547+0000 7f08b7e00640 1 --2- 172.21.15.83:0/3822392296 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b3b223e8c0 0x55b3b2240cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-19T01:41:11.559 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:11.547+0000 7f08bcc00640 1 --2- 172.21.15.83:0/188556475 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f08a40a2b80 0x7f08a40a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:41:11.560 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:11.547+0000 7f08bf400640 1 --2- 172.21.15.83:0/3822392296 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b3b2234000 0x55b3b223e380 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f08b0009900 tx=0x7f08b00044a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=645de42d0ee8ab34 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:41:11.560 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:11.547+0000 7f08bcc00640 1 -- 172.21.15.83:0/188556475 <== mon.2 v2:172.21.15.117:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8269+0+0 (secure 0 0 0) 0x7f08b80cf3a0 con 0x7f08c0109900 2024-08-19T01:41:11.560 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:11.547+0000 7f08b7e00640 1 --2- 172.21.15.83:0/188556475 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f08a40a2b80 0x7f08a40a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:41:11.560 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:11.547+0000 7f08afe00640 1 -- 172.21.15.83:0/3822392296 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08b0035040 con 0x55b3b2234000 2024-08-19T01:41:11.560 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:11.547+0000 7f08afe00640 1 -- 172.21.15.83:0/3822392296 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f08b0004760 con 0x55b3b2234000 2024-08-19T01:41:11.560 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:11.547+0000 7f08afe00640 1 -- 172.21.15.83:0/3822392296 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08b0003ab0 con 0x55b3b2234000 2024-08-19T01:41:11.560 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:11.548+0000 7f08b7e00640 1 --2- 172.21.15.83:0/188556475 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f08a40a2b80 0x7f08a40a5040 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x55b3b2230ff0 tx=0x7f08a8005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:41:11.560 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:11.548+0000 7f08c8c80740 1 -- 172.21.15.83:0/3822392296 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b3b2234000 msgr2=0x55b3b223e380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:41:11.560 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:11.548+0000 7f08c8c80740 1 --2- 172.21.15.83:0/3822392296 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b3b2234000 0x55b3b223e380 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f08b0009900 tx=0x7f08b00044a0 comp rx=0 tx=0).stop 2024-08-19T01:41:11.560 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:11.548+0000 7f08c8c80740 1 -- 172.21.15.83:0/3822392296 shutdown_connections 2024-08-19T01:41:11.560 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:11.548+0000 7f08c8c80740 1 --2- 172.21.15.83:0/3822392296 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b3b223e8c0 0x55b3b2240cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:41:11.560 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:11.548+0000 7f08c8c80740 1 --2- 172.21.15.83:0/3822392296 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b3b2234000 0x55b3b223e380 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:41:11.560 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:11.548+0000 7f08c8c80740 1 --2- 172.21.15.83:0/3822392296 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b3b22336e0 0x55b3b2233ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:41:11.560 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:11.548+0000 7f08c8c80740 1 -- 172.21.15.83:0/3822392296 >> 172.21.15.83:0/3822392296 conn(0x55b3b222fc70 msgr2=0x55b3b2230050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:41:11.561 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:11.548+0000 7f08c8c80740 1 -- 172.21.15.83:0/3822392296 shutdown_connections 2024-08-19T01:41:11.561 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:11.548+0000 7f08c8c80740 1 -- 172.21.15.83:0/3822392296 wait complete. 2024-08-19T01:41:11.561 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:11.549+0000 7f08c8c80740 1 -- start start 2024-08-19T01:41:11.561 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:11.549+0000 7f08c8c80740 1 Processor -- start 2024-08-19T01:41:11.561 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:11.550+0000 7f08c8c80740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b3b22336e0 0x55b3b223b770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:41:11.561 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:11.550+0000 7f08c8c80740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b3b2234000 0x55b3b223bcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:41:11.561 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:11.550+0000 7f08c8c80740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b3b223e8c0 0x55b3b223c1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:41:11.561 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:11.550+0000 7f08c8c80740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55b3b2241490 con 0x55b3b2234000 2024-08-19T01:41:11.561 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:11.550+0000 7f08c8c80740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55b3b2153cf0 con 0x55b3b22336e0 2024-08-19T01:41:11.561 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:11.550+0000 7f08c8c80740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55b3b2241610 con 0x55b3b223e8c0 2024-08-19T01:41:11.561 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:11.550+0000 7f08c8c80740 10 client.0 _ll_register_callbacks cb 0x55b3b213fe30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-19T01:41:11.561 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:11.550+0000 7f08bf400640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b3b22336e0 0x55b3b223b770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:41:11.561 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:11.550+0000 7f08bf400640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b3b22336e0 0x55b3b223b770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.83:41662/0 (socket says 172.21.15.83:41662) 2024-08-19T01:41:11.562 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:11.550+0000 7f08bf400640 1 -- 172.21.15.83:0/2396823573 learned_addr learned my addr 172.21.15.83:0/2396823573 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-08-19T01:41:11.562 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:11.550+0000 7f08bfe00640 1 --2- 172.21.15.83:0/2396823573 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b3b223e8c0 0x55b3b223c1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:41:11.562 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:11.550+0000 7f08b7e00640 1 --2- 172.21.15.83:0/2396823573 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b3b2234000 0x55b3b223bcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:41:11.562 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:11.550+0000 7f08bf400640 1 -- 172.21.15.83:0/2396823573 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b3b223e8c0 msgr2=0x55b3b223c1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:41:11.562 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:11.550+0000 7f08bf400640 1 --2- 172.21.15.83:0/2396823573 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b3b223e8c0 0x55b3b223c1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:41:11.562 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:11.550+0000 7f08bf400640 1 -- 172.21.15.83:0/2396823573 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b3b2234000 msgr2=0x55b3b223bcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:41:11.562 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:11.550+0000 7f08bf400640 1 --2- 172.21.15.83:0/2396823573 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b3b2234000 0x55b3b223bcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:41:11.562 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:11.550+0000 7f08bf400640 1 -- 172.21.15.83:0/2396823573 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b3b1f0a3c0 con 0x55b3b22336e0 2024-08-19T01:41:11.562 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:11.551+0000 7f08bf400640 1 --2- 172.21.15.83:0/2396823573 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b3b22336e0 0x55b3b223b770 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f08b0004f90 tx=0x7f08b0004610 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7ec1a1f639dbbf05 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:41:11.562 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:11.551+0000 7f08acc00640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-08-19T01:41:11.562 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:11.551+0000 7f08acc00640 1 -- 172.21.15.83:0/2396823573 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08b0035030 con 0x55b3b22336e0 2024-08-19T01:41:11.562 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:11.551+0000 7f08c8c80740 1 -- 172.21.15.83:0/2396823573 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f08b0004760 con 0x55b3b22336e0 2024-08-19T01:41:11.562 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:11.551+0000 7f08acc00640 1 -- 172.21.15.83:0/2396823573 <== mon.2 v2:172.21.15.117:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8269+0+0 (secure 0 0 0) 0x7f08b0005310 con 0x55b3b22336e0 2024-08-19T01:41:11.562 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:11.551+0000 7f08acc00640 1 -- 172.21.15.83:0/2396823573 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f08b0054070 con 0x55b3b22336e0 2024-08-19T01:41:11.563 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:11.551+0000 7f08c8c80740 10 client.25877 fetch_fsmap learned FSMap version 27 2024-08-19T01:41:11.563 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:11.551+0000 7f08c8c80740 1 -- 172.21.15.83:0/2396823573 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55b3b224d140 con 0x55b3b22336e0 2024-08-19T01:41:11.563 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:11.551+0000 7f08acc00640 1 -- 172.21.15.83:0/2396823573 <== mon.2 v2:172.21.15.117:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f08b0004760 con 0x55b3b22336e0 2024-08-19T01:41:11.563 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:11.552+0000 7f08acc00640 1 -- 172.21.15.83:0/2396823573 <== mon.2 v2:172.21.15.117:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f08b00316d0 con 0x55b3b22336e0 2024-08-19T01:41:11.563 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:11.552+0000 7f08c8c80740 10 client.25877 fetch_fsmap finished waiting for FSMap version 27 2024-08-19T01:41:11.563 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:11.552+0000 7f08c8c80740 10 client.25877 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-19T01:41:11.563 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:11.552+0000 7f08c8c80740 20 client.25877 populate_metadata read hostname 'smithi083' 2024-08-19T01:41:11.563 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:11.552+0000 7f08c8c80740 1 --2- 172.21.15.83:0/2396823573 >> [v2:172.21.15.117:6838/1967595704,v1:172.21.15.117:6839/1967595704] conn(0x55b3b224e490 0x55b3b2250880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:41:11.563 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:11.552+0000 7f08c8c80740 4 client.25877 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-19T01:41:11.563 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:11.552+0000 7f08c8c80740 1 -- 172.21.15.83:0/2396823573 --> [v2:172.21.15.117:6838/1967595704,v1:172.21.15.117:6839/1967595704] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b3b21a6160 con 0x55b3b224e490 2024-08-19T01:41:11.563 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:11.552+0000 7f08b7e00640 1 --2- 172.21.15.83:0/2396823573 >> [v2:172.21.15.117:6838/1967595704,v1:172.21.15.117:6839/1967595704] conn(0x55b3b224e490 0x55b3b2250880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:41:11.563 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:11.553+0000 7f08b7e00640 1 --2- 172.21.15.83:0/2396823573 >> [v2:172.21.15.117:6838/1967595704,v1:172.21.15.117:6839/1967595704] conn(0x55b3b224e490 0x55b3b2250880 crc :-1 s=READY pgs=109 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=626bad1a0cb604b7 server_cookie=25603d518d3317b2 in_seq=0 out_seq=0 2024-08-19T01:41:11.563 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:11.553+0000 7f08acc00640 10 client.25877 ms_handle_connect on v2:172.21.15.117:6838/1967595704 2024-08-19T01:41:11.563 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:11.554+0000 7f08acc00640 1 -- 172.21.15.83:0/2396823573 <== mds.0 v2:172.21.15.117:6838/1967595704 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55b3b21a6160 con 0x55b3b224e490 2024-08-19T01:41:11.564 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:11.554+0000 7f08acc00640 10 client.25877 handle_command_reply: tid=0 2024-08-19T01:41:11.564 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:11.554+0000 7f08c8c80740 1 client.25877 shutdown 2024-08-19T01:41:11.565 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:11.554+0000 7f08c8c80740 1 -- 172.21.15.83:0/2396823573 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b3b22336e0 msgr2=0x55b3b223b770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:41:11.565 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:11.554+0000 7f08c8c80740 1 --2- 172.21.15.83:0/2396823573 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b3b22336e0 0x55b3b223b770 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f08b0004f90 tx=0x7f08b0004610 comp rx=0 tx=0).stop 2024-08-19T01:41:11.565 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:11.555+0000 7f08c8c80740 1 -- 172.21.15.83:0/2396823573 shutdown_connections 2024-08-19T01:41:11.565 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:11.555+0000 7f08c8c80740 1 --2- 172.21.15.83:0/2396823573 >> [v2:172.21.15.117:6838/1967595704,v1:172.21.15.117:6839/1967595704] conn(0x55b3b224e490 0x55b3b2250880 crc :-1 s=READY pgs=109 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:41:11.565 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:11.555+0000 7f08c8c80740 1 --2- 172.21.15.83:0/2396823573 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b3b223e8c0 0x55b3b223c1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:41:11.565 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:11.555+0000 7f08c8c80740 1 --2- 172.21.15.83:0/2396823573 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b3b2234000 0x55b3b223bcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:41:11.565 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:11.555+0000 7f08acc00640 0 client.25877 ms_handle_reset on v2:172.21.15.117:6838/1967595704 2024-08-19T01:41:11.565 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:11.555+0000 7f08c8c80740 1 --2- 172.21.15.83:0/2396823573 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b3b22336e0 0x55b3b223b770 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:41:11.565 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:11.555+0000 7f08c8c80740 1 -- 172.21.15.83:0/2396823573 >> 172.21.15.83:0/2396823573 conn(0x55b3b222fc70 msgr2=0x55b3b22359a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:41:11.565 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:11.555+0000 7f08c8c80740 1 -- 172.21.15.83:0/2396823573 shutdown_connections 2024-08-19T01:41:11.565 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:11.555+0000 7f08c8c80740 1 -- 172.21.15.83:0/2396823573 wait complete. 2024-08-19T01:41:11.566 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:11.555+0000 7f08c8c80740 20 client.25877 trim_cache size 0 max 16384 2024-08-19T01:41:11.570 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:11.568+0000 7f08c8c80740 1 Processor -- start 2024-08-19T01:41:11.570 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:11.569+0000 7f08c8c80740 1 -- start start 2024-08-19T01:41:11.571 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:11.569+0000 7f08c8c80740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b3b22336e0 0x55b3b222a910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:41:11.571 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:11.569+0000 7f08c8c80740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b3b2234000 0x55b3b22538a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:41:11.571 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:11.569+0000 7f08c8c80740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b3b223e8c0 0x55b3b2253de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:41:11.571 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:11.569+0000 7f08c8c80740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f08c01b1cd0 con 0x55b3b22336e0 2024-08-19T01:41:11.571 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:11.569+0000 7f08c8c80740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f08c01af5a0 con 0x55b3b223e8c0 2024-08-19T01:41:11.571 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:11.569+0000 7f08c8c80740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f08c01ad3f0 con 0x55b3b2234000 2024-08-19T01:41:11.571 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:11.569+0000 7f08bf400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b3b22336e0 0x55b3b222a910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:41:11.571 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:11.569+0000 7f08bf400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b3b22336e0 0x55b3b222a910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:57072/0 (socket says 172.21.15.83:57072) 2024-08-19T01:41:11.572 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:11.569+0000 7f08bf400640 1 -- 172.21.15.83:0/67926178 learned_addr learned my addr 172.21.15.83:0/67926178 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-08-19T01:41:11.572 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:11.569+0000 7f08b7e00640 1 --2- 172.21.15.83:0/67926178 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b3b2234000 0x55b3b22538a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:41:11.572 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:11.570+0000 7f08bf400640 1 -- 172.21.15.83:0/67926178 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b3b2234000 msgr2=0x55b3b22538a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:41:11.572 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:11.570+0000 7f08bf400640 1 --2- 172.21.15.83:0/67926178 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b3b2234000 0x55b3b22538a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:41:11.572 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:11.570+0000 7f08bf400640 1 -- 172.21.15.83:0/67926178 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b3b223e8c0 msgr2=0x55b3b2253de0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-08-19T01:41:11.572 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:11.570+0000 7f08bf400640 1 --2- 172.21.15.83:0/67926178 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b3b223e8c0 0x55b3b2253de0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:41:11.572 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:11.570+0000 7f08bf400640 1 -- 172.21.15.83:0/67926178 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b3b223a720 con 0x55b3b22336e0 2024-08-19T01:41:11.572 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:11.570+0000 7f08bf400640 1 --2- 172.21.15.83:0/67926178 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b3b22336e0 0x55b3b222a910 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f08b0006010 tx=0x7f08b002a3f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b29a5f2d9f56900b server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:41:11.573 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:11.570+0000 7f08af400640 1 -- 172.21.15.83:0/67926178 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08b0035070 con 0x55b3b22336e0 2024-08-19T01:41:11.573 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:11.570+0000 7f08af400640 1 -- 172.21.15.83:0/67926178 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f08b0031d10 con 0x55b3b22336e0 2024-08-19T01:41:11.573 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:11.570+0000 7f08af400640 1 -- 172.21.15.83:0/67926178 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08b0003700 con 0x55b3b22336e0 2024-08-19T01:41:11.573 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:11.571+0000 7f08c8c80740 1 -- 172.21.15.83:0/67926178 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b3b22336e0 msgr2=0x55b3b222a910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:41:11.573 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:11.571+0000 7f08c8c80740 1 --2- 172.21.15.83:0/67926178 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b3b22336e0 0x55b3b222a910 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f08b0006010 tx=0x7f08b002a3f0 comp rx=0 tx=0).stop 2024-08-19T01:41:11.574 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:11.571+0000 7f08c8c80740 1 -- 172.21.15.83:0/67926178 shutdown_connections 2024-08-19T01:41:11.574 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:11.571+0000 7f08c8c80740 1 --2- 172.21.15.83:0/67926178 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b3b223e8c0 0x55b3b2253de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:41:11.574 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:11.571+0000 7f08c8c80740 1 --2- 172.21.15.83:0/67926178 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b3b2234000 0x55b3b22538a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:41:11.574 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:11.571+0000 7f08c8c80740 1 --2- 172.21.15.83:0/67926178 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b3b22336e0 0x55b3b222a910 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:41:11.574 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:11.571+0000 7f08c8c80740 1 -- 172.21.15.83:0/67926178 >> 172.21.15.83:0/67926178 conn(0x55b3b222fc70 msgr2=0x55b3b223d7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:41:11.574 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:11.571+0000 7f08c8c80740 1 -- 172.21.15.83:0/67926178 shutdown_connections 2024-08-19T01:41:11.574 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:11.571+0000 7f08c8c80740 1 -- 172.21.15.83:0/67926178 wait complete. 2024-08-19T01:41:11.574 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:11.572+0000 7f08c8c80740 1 -- start start 2024-08-19T01:41:11.575 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:11.572+0000 7f08c8c80740 1 Processor -- start 2024-08-19T01:41:11.575 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:11.572+0000 7f08c8c80740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b3b22336e0 0x55b3b2254160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:41:11.575 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:11.572+0000 7f08c8c80740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b3b2234000 0x55b3b22514b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:41:11.575 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:11.572+0000 7f08c8c80740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b3b223e8c0 0x55b3b2251c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:41:11.575 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:11.572+0000 7f08c8c80740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f08c01ada60 con 0x55b3b22336e0 2024-08-19T01:41:11.575 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:11.572+0000 7f08c8c80740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f08c01aeed0 con 0x55b3b2234000 2024-08-19T01:41:11.575 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:11.572+0000 7f08c8c80740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f08c01ae130 con 0x55b3b223e8c0 2024-08-19T01:41:11.576 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:11.573+0000 7f08c8c80740 10 client.0 _ll_register_callbacks cb 0x7f08b0004910 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-19T01:41:11.576 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:11.573+0000 7f08b7e00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b3b2234000 0x55b3b22514b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:41:11.576 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:11.573+0000 7f08bf400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b3b22336e0 0x55b3b2254160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:41:11.576 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:11.573+0000 7f08b7e00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b3b2234000 0x55b3b22514b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.83:41680/0 (socket says 172.21.15.83:41680) 2024-08-19T01:41:11.576 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:11.573+0000 7f08b7e00640 1 -- 172.21.15.83:0/3892785013 learned_addr learned my addr 172.21.15.83:0/3892785013 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-08-19T01:41:11.576 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:11.573+0000 7f08bfe00640 1 --2- 172.21.15.83:0/3892785013 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b3b223e8c0 0x55b3b2251c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:41:11.576 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:11.573+0000 7f08bf400640 1 -- 172.21.15.83:0/3892785013 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b3b223e8c0 msgr2=0x55b3b2251c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:41:11.576 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:11.573+0000 7f08bf400640 1 --2- 172.21.15.83:0/3892785013 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b3b223e8c0 0x55b3b2251c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:41:11.576 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:11.573+0000 7f08bf400640 1 -- 172.21.15.83:0/3892785013 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b3b2234000 msgr2=0x55b3b22514b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:41:11.576 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:11.573+0000 7f08bf400640 1 --2- 172.21.15.83:0/3892785013 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b3b2234000 0x55b3b22514b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:41:11.576 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:11.573+0000 7f08bf400640 1 -- 172.21.15.83:0/3892785013 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b3b22b7dd0 con 0x55b3b22336e0 2024-08-19T01:41:11.576 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:11.573+0000 7f08b7e00640 1 --2- 172.21.15.83:0/3892785013 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b3b2234000 0x55b3b22514b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-19T01:41:11.576 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:11.573+0000 7f08bf400640 1 --2- 172.21.15.83:0/3892785013 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b3b22336e0 0x55b3b2254160 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f08b0031b20 tx=0x7f08b0031480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9eedb01191d1d21 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:41:11.577 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:11.573+0000 7f08acc00640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-08-19T01:41:11.578 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:11.574+0000 7f08acc00640 1 -- 172.21.15.83:0/3892785013 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08b0067060 con 0x55b3b22336e0 2024-08-19T01:41:11.578 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:11.574+0000 7f08acc00640 1 -- 172.21.15.83:0/3892785013 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8269+0+0 (secure 0 0 0) 0x7f08b0005310 con 0x55b3b22336e0 2024-08-19T01:41:11.578 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:11.574+0000 7f08c8c80740 1 -- 172.21.15.83:0/3892785013 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f08b0031d10 con 0x55b3b22336e0 2024-08-19T01:41:11.578 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:11.574+0000 7f08acc00640 1 -- 172.21.15.83:0/3892785013 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f08b0035040 con 0x55b3b22336e0 2024-08-19T01:41:11.578 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:11.574+0000 7f08c8c80740 10 client.15834 fetch_fsmap learned FSMap version 27 2024-08-19T01:41:11.578 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:11.574+0000 7f08c8c80740 1 -- 172.21.15.83:0/3892785013 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55b3b22b8f30 con 0x55b3b22336e0 2024-08-19T01:41:11.578 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:11.575+0000 7f08acc00640 1 -- 172.21.15.83:0/3892785013 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f08b0031d10 con 0x55b3b22336e0 2024-08-19T01:41:11.578 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:11.575+0000 7f08acc00640 1 -- 172.21.15.83:0/3892785013 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f08b0003d00 con 0x55b3b22336e0 2024-08-19T01:41:11.578 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:11.575+0000 7f08c8c80740 10 client.15834 fetch_fsmap finished waiting for FSMap version 27 2024-08-19T01:41:11.578 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:11.575+0000 7f08c8c80740 10 client.15834 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-19T01:41:11.578 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:11.575+0000 7f08c8c80740 20 client.15834 populate_metadata read hostname 'smithi083' 2024-08-19T01:41:11.579 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:11.575+0000 7f08c8c80740 1 --2- 172.21.15.83:0/3892785013 >> [v2:172.21.15.117:6838/1967595704,v1:172.21.15.117:6839/1967595704] conn(0x55b3b22b95f0 0x55b3b22bba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:41:11.579 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:11.575+0000 7f08c8c80740 4 client.15834 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-08-19T01:41:11.579 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:11.575+0000 7f08c8c80740 1 -- 172.21.15.83:0/3892785013 --> [v2:172.21.15.117:6838/1967595704,v1:172.21.15.117:6839/1967595704] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55b3b21a0670 con 0x55b3b22b95f0 2024-08-19T01:41:11.579 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:11.575+0000 7f08b7e00640 1 --2- 172.21.15.83:0/3892785013 >> [v2:172.21.15.117:6838/1967595704,v1:172.21.15.117:6839/1967595704] conn(0x55b3b22b95f0 0x55b3b22bba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:41:11.579 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:11.576+0000 7f08b7e00640 1 --2- 172.21.15.83:0/3892785013 >> [v2:172.21.15.117:6838/1967595704,v1:172.21.15.117:6839/1967595704] conn(0x55b3b22b95f0 0x55b3b22bba10 crc :-1 s=READY pgs=110 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=aedf5f015a31fb80 server_cookie=f9feb8f51c4aea09 in_seq=0 out_seq=0 2024-08-19T01:41:11.579 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:11.576+0000 7f08acc00640 10 client.15834 ms_handle_connect on v2:172.21.15.117:6838/1967595704 2024-08-19T01:41:11.580 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:11.578+0000 7f08acc00640 1 -- 172.21.15.83:0/3892785013 <== mds.0 v2:172.21.15.117:6838/1967595704 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55b3b21a0670 con 0x55b3b22b95f0 2024-08-19T01:41:11.580 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:11.578+0000 7f08acc00640 10 client.15834 handle_command_reply: tid=0 2024-08-19T01:41:11.580 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:11.578+0000 7f08c8c80740 1 client.15834 shutdown 2024-08-19T01:41:11.580 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:11.578+0000 7f08c8c80740 1 -- 172.21.15.83:0/3892785013 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b3b22336e0 msgr2=0x55b3b2254160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:41:11.580 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:11.578+0000 7f08c8c80740 1 --2- 172.21.15.83:0/3892785013 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b3b22336e0 0x55b3b2254160 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f08b0031b20 tx=0x7f08b0031480 comp rx=0 tx=0).stop 2024-08-19T01:41:11.580 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:11.578+0000 7f08c8c80740 1 -- 172.21.15.83:0/3892785013 shutdown_connections 2024-08-19T01:41:11.581 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:11.578+0000 7f08c8c80740 1 --2- 172.21.15.83:0/3892785013 >> [v2:172.21.15.117:6838/1967595704,v1:172.21.15.117:6839/1967595704] conn(0x55b3b22b95f0 0x55b3b22bba10 crc :-1 s=READY pgs=110 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:41:11.581 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:11.578+0000 7f08c8c80740 1 --2- 172.21.15.83:0/3892785013 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b3b223e8c0 0x55b3b2251c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:41:11.581 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:11.578+0000 7f08c8c80740 1 --2- 172.21.15.83:0/3892785013 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b3b2234000 0x55b3b22514b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:41:11.581 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:11.578+0000 7f08c8c80740 1 --2- 172.21.15.83:0/3892785013 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b3b22336e0 0x55b3b2254160 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:41:11.581 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:11.578+0000 7f08c8c80740 1 -- 172.21.15.83:0/3892785013 >> 172.21.15.83:0/3892785013 conn(0x55b3b222fc70 msgr2=0x55b3b2249890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:41:11.581 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:11.578+0000 7f08acc00640 0 client.15834 ms_handle_reset on v2:172.21.15.117:6838/1967595704 2024-08-19T01:41:11.581 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:11.578+0000 7f08c8c80740 1 -- 172.21.15.83:0/3892785013 shutdown_connections 2024-08-19T01:41:11.581 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:11.578+0000 7f08c8c80740 1 -- 172.21.15.83:0/3892785013 wait complete. 2024-08-19T01:41:11.581 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:11.578+0000 7f08c8c80740 20 client.15834 trim_cache size 0 max 16384 2024-08-19T01:41:11.581 INFO:teuthology.orchestra.run.smithi083.stdout:[] 2024-08-19T01:41:11.582 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:11.579+0000 7f08c4a00640 1 -- 172.21.15.83:0/188556475 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f08a40a2b80 msgr2=0x7f08a40a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:41:11.582 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:11.579+0000 7f08c4a00640 1 --2- 172.21.15.83:0/188556475 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f08a40a2b80 0x7f08a40a5040 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x55b3b2230ff0 tx=0x7f08a8005f90 comp rx=0 tx=0).stop 2024-08-19T01:41:11.582 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:11.579+0000 7f08c4a00640 1 -- 172.21.15.83:0/188556475 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f08c0109900 msgr2=0x7f08c019ead0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:41:11.582 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:11.579+0000 7f08c4a00640 1 --2- 172.21.15.83:0/188556475 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f08c0109900 0x7f08c019ead0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f08b800da10 tx=0x7f08b800ded0 comp rx=0 tx=0).stop 2024-08-19T01:41:11.582 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:11.579+0000 7f08c4a00640 1 -- 172.21.15.83:0/188556475 shutdown_connections 2024-08-19T01:41:11.583 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:11.579+0000 7f08c4a00640 1 --2- 172.21.15.83:0/188556475 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f08a40a2b80 0x7f08a40a5040 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:41:11.583 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:11.579+0000 7f08c4a00640 1 --2- 172.21.15.83:0/188556475 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f08c0109900 0x7f08c019ead0 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:41:11.583 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:11.579+0000 7f08c4a00640 1 --2- 172.21.15.83:0/188556475 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f08c01042a0 0x7f08c019a740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:41:11.583 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:11.579+0000 7f08c4a00640 1 --2- 172.21.15.83:0/188556475 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f08c0103900 0x7f08c019a200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:41:11.583 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:11.579+0000 7f08c4a00640 1 -- 172.21.15.83:0/188556475 >> 172.21.15.83:0/188556475 conn(0x7f08c00ff600 msgr2=0x7f08c00ffe10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:41:11.583 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:11.579+0000 7f08c4a00640 1 -- 172.21.15.83:0/188556475 shutdown_connections 2024-08-19T01:41:11.583 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:11.580+0000 7f08c4a00640 1 -- 172.21.15.83:0/188556475 wait complete. 2024-08-19T01:41:11.952 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:41:11 smithi117 ceph-mon[22016]: pgmap v529: 97 pgs: 97 active+clean; 601 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.0 MiB/s wr, 1.28k op/s 2024-08-19T01:41:11.965 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:41:11 smithi167 ceph-mon[21875]: pgmap v529: 97 pgs: 97 active+clean; 601 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.0 MiB/s wr, 1.28k op/s 2024-08-19T01:41:12.154 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:41:11 smithi083 ceph-mon[17120]: pgmap v529: 97 pgs: 97 active+clean; 601 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.0 MiB/s wr, 1.28k op/s 2024-08-19T01:41:12.866 INFO:tasks.workunit.client.0.smithi083.stdout: 2024-08-19T01:41:12.866 INFO:tasks.workunit.client.0.smithi083.stdout:Final score for writes: 214 2024-08-19T01:41:12.866 INFO:tasks.workunit.client.0.smithi083.stdout:Final score for reads : 340749 2024-08-19T01:41:12.867 INFO:tasks.workunit.client.0.smithi083.stdout: 2024-08-19T01:41:12.868 INFO:teuthology.orchestra.run:Running command with timeout 3600 2024-08-19T01:41:12.868 DEBUG:teuthology.orchestra.run.smithi083:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2024-08-19T01:41:12.965 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:41:12 smithi167 ceph-mon[21875]: pgmap v530: 97 pgs: 97 active+clean; 609 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.4 MiB/s wr, 1.29k op/s 2024-08-19T01:41:13.154 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:41:12 smithi083 ceph-mon[17120]: pgmap v530: 97 pgs: 97 active+clean; 609 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.4 MiB/s wr, 1.29k op/s 2024-08-19T01:41:13.202 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:41:12 smithi117 ceph-mon[22016]: pgmap v530: 97 pgs: 97 active+clean; 609 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.4 MiB/s wr, 1.29k op/s 2024-08-19T01:41:13.737 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-08-19T01:41:13.738 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833 shell --fsid bad8780e-5dc9-11ef-bccf-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-08-19T01:41:14.164 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/bad8780e-5dc9-11ef-bccf-c7b262605968/mon.a/config 2024-08-19T01:41:14.901 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:14.791+0000 7f1454e00640 1 -- 172.21.15.83:0/1782731509 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f145010dfb0 msgr2=0x7f14501103a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:41:14.901 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:14.791+0000 7f1454e00640 1 --2- 172.21.15.83:0/1782731509 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f145010dfb0 0x7f14501103a0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f1448009be0 tx=0x7f144802a750 comp rx=0 tx=0).stop 2024-08-19T01:41:14.901 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:14.791+0000 7f1454e00640 1 -- 172.21.15.83:0/1782731509 shutdown_connections 2024-08-19T01:41:14.901 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:14.791+0000 7f1454e00640 1 --2- 172.21.15.83:0/1782731509 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f145010dfb0 0x7f14501103a0 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:41:14.901 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:14.791+0000 7f1454e00640 1 --2- 172.21.15.83:0/1782731509 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1450074b60 0x7f145010da70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:41:14.901 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:14.791+0000 7f1454e00640 1 --2- 172.21.15.83:0/1782731509 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1450074240 0x7f1450074620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:41:14.902 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:14.791+0000 7f1454e00640 1 -- 172.21.15.83:0/1782731509 >> 172.21.15.83:0/1782731509 conn(0x7f14500fd470 msgr2=0x7f14500ff890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:41:14.902 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:14.791+0000 7f1454e00640 1 -- 172.21.15.83:0/1782731509 shutdown_connections 2024-08-19T01:41:14.902 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:14.791+0000 7f1454e00640 1 -- 172.21.15.83:0/1782731509 wait complete. 2024-08-19T01:41:14.902 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:14.792+0000 7f1454e00640 1 Processor -- start 2024-08-19T01:41:14.902 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:14.792+0000 7f1454e00640 1 -- start start 2024-08-19T01:41:14.902 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:14.792+0000 7f1454e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1450074240 0x7f145019a1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:41:14.902 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:14.792+0000 7f1454e00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1450074b60 0x7f145019a6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:41:14.902 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:14.792+0000 7f1454e00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f145010dfb0 0x7f145019ea80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:41:14.902 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:14.792+0000 7f1454e00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f1450113c90 con 0x7f1450074240 2024-08-19T01:41:14.902 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:14.792+0000 7f1454e00640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f1450113b10 con 0x7f1450074b60 2024-08-19T01:41:14.902 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:14.792+0000 7f1454e00640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f1450113e10 con 0x7f145010dfb0 2024-08-19T01:41:14.902 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:14.793+0000 7f144ea00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1450074b60 0x7f145019a6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:41:14.903 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:14.793+0000 7f144f400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1450074240 0x7f145019a1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:41:14.903 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:14.793+0000 7f144ea00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1450074b60 0x7f145019a6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.83:41694/0 (socket says 172.21.15.83:41694) 2024-08-19T01:41:14.903 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:14.793+0000 7f144ea00640 1 -- 172.21.15.83:0/955060879 learned_addr learned my addr 172.21.15.83:0/955060879 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-08-19T01:41:14.903 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:14.793+0000 7f144f400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1450074240 0x7f145019a1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:57082/0 (socket says 172.21.15.83:57082) 2024-08-19T01:41:14.903 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:14.793+0000 7f144fe00640 1 --2- 172.21.15.83:0/955060879 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f145010dfb0 0x7f145019ea80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:41:14.903 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:14.793+0000 7f144ea00640 1 -- 172.21.15.83:0/955060879 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f145010dfb0 msgr2=0x7f145019ea80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:41:14.903 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:14.793+0000 7f144ea00640 1 --2- 172.21.15.83:0/955060879 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f145010dfb0 0x7f145019ea80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:41:14.903 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:14.793+0000 7f144ea00640 1 -- 172.21.15.83:0/955060879 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1450074240 msgr2=0x7f145019a1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:41:14.903 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:14.793+0000 7f144ea00640 1 --2- 172.21.15.83:0/955060879 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1450074240 0x7f145019a1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:41:14.903 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:14.793+0000 7f144ea00640 1 -- 172.21.15.83:0/955060879 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f145019f160 con 0x7f1450074b60 2024-08-19T01:41:14.903 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:14.793+0000 7f144f400640 1 --2- 172.21.15.83:0/955060879 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1450074240 0x7f145019a1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-19T01:41:14.903 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:14.793+0000 7f144ea00640 1 --2- 172.21.15.83:0/955060879 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1450074b60 0x7f145019a6f0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f143c00e9f0 tx=0x7f143c00edf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:41:14.903 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:14.793+0000 7f1447e00640 1 -- 172.21.15.83:0/955060879 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f143c00ccc0 con 0x7f1450074b60 2024-08-19T01:41:14.903 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:14.794+0000 7f1447e00640 1 -- 172.21.15.83:0/955060879 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f143c010910 con 0x7f1450074b60 2024-08-19T01:41:14.904 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:14.794+0000 7f1454e00640 1 -- 172.21.15.83:0/955060879 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f145019f450 con 0x7f1450074b60 2024-08-19T01:41:14.904 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:14.794+0000 7f1447e00640 1 -- 172.21.15.83:0/955060879 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f143c0188a0 con 0x7f1450074b60 2024-08-19T01:41:14.904 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:14.794+0000 7f1454e00640 1 -- 172.21.15.83:0/955060879 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f14501a6d30 con 0x7f1450074b60 2024-08-19T01:41:14.904 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:14.797+0000 7f1459074740 1 Processor -- start 2024-08-19T01:41:14.904 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:14.797+0000 7f1459074740 1 -- start start 2024-08-19T01:41:14.904 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:14.797+0000 7f1459074740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b54f662650 0x55b54f662a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:41:14.904 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:14.797+0000 7f1459074740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b54f662f70 0x55b54f66d2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:41:14.904 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:14.797+0000 7f1447e00640 1 -- 172.21.15.83:0/955060879 <== mon.2 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7f143c010ac0 con 0x7f1450074b60 2024-08-19T01:41:14.904 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:14.797+0000 7f1459074740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b54f66d830 0x55b54f66fc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:41:14.904 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:14.797+0000 7f1459074740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55b54f57d7e0 con 0x55b54f66d830 2024-08-19T01:41:14.904 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:14.797+0000 7f1459074740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55b54f5c8ac0 con 0x55b54f662650 2024-08-19T01:41:14.904 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:14.797+0000 7f1459074740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55b54f5608c0 con 0x55b54f662f70 2024-08-19T01:41:14.904 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:14.797+0000 7f144fe00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b54f662650 0x55b54f662a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:41:14.904 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:14.797+0000 7f144fe00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b54f662650 0x55b54f662a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.83:41700/0 (socket says 172.21.15.83:41700) 2024-08-19T01:41:14.905 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:14.797+0000 7f144fe00640 1 -- 172.21.15.83:0/857465308 learned_addr learned my addr 172.21.15.83:0/857465308 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-08-19T01:41:14.905 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:14.798+0000 7f144f400640 1 --2- 172.21.15.83:0/857465308 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b54f662f70 0x55b54f66d2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:41:14.905 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:14.798+0000 7f144ea00640 1 --2- 172.21.15.83:0/857465308 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b54f66d830 0x55b54f66fc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:41:14.905 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:14.798+0000 7f144fe00640 1 -- 172.21.15.83:0/857465308 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b54f662f70 msgr2=0x55b54f66d2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:41:14.905 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:14.798+0000 7f144fe00640 1 --2- 172.21.15.83:0/857465308 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b54f662f70 0x55b54f66d2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:41:14.905 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:14.798+0000 7f144fe00640 1 -- 172.21.15.83:0/857465308 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b54f66d830 msgr2=0x55b54f66fc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:41:14.905 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:14.798+0000 7f144fe00640 1 --2- 172.21.15.83:0/857465308 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b54f66d830 0x55b54f66fc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:41:14.905 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:14.798+0000 7f144fe00640 1 -- 172.21.15.83:0/857465308 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b54f567760 con 0x55b54f662650 2024-08-19T01:41:14.905 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:14.798+0000 7f144fe00640 1 --2- 172.21.15.83:0/857465308 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b54f662650 0x55b54f662a30 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f1450075220 tx=0x7f1448009300 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ad1d71fd9bda6a8a server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:41:14.905 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:14.798+0000 7f1444c00640 1 -- 172.21.15.83:0/857465308 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1448041030 con 0x55b54f662650 2024-08-19T01:41:14.905 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:14.799+0000 7f1444c00640 1 -- 172.21.15.83:0/857465308 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1448037040 con 0x55b54f662650 2024-08-19T01:41:14.905 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:14.799+0000 7f1444c00640 1 -- 172.21.15.83:0/857465308 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1448036040 con 0x55b54f662650 2024-08-19T01:41:14.905 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:14.799+0000 7f1447e00640 1 --2- 172.21.15.83:0/955060879 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f14340a2b80 0x7f14340a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:41:14.906 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:14.799+0000 7f1459074740 1 -- 172.21.15.83:0/857465308 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b54f662650 msgr2=0x55b54f662a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:41:14.906 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:14.799+0000 7f1459074740 1 --2- 172.21.15.83:0/857465308 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b54f662650 0x55b54f662a30 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f1450075220 tx=0x7f1448009300 comp rx=0 tx=0).stop 2024-08-19T01:41:14.906 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:14.799+0000 7f1459074740 1 -- 172.21.15.83:0/857465308 shutdown_connections 2024-08-19T01:41:14.906 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:14.799+0000 7f1459074740 1 --2- 172.21.15.83:0/857465308 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b54f66d830 0x55b54f66fc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:41:14.906 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:14.799+0000 7f1459074740 1 --2- 172.21.15.83:0/857465308 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b54f662f70 0x55b54f66d2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:41:14.906 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:14.799+0000 7f1459074740 1 --2- 172.21.15.83:0/857465308 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b54f662650 0x55b54f662a30 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:41:14.906 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:14.799+0000 7f1459074740 1 -- 172.21.15.83:0/857465308 >> 172.21.15.83:0/857465308 conn(0x55b54f65ebe0 msgr2=0x55b54f65efc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:41:14.906 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:14.799+0000 7f1447e00640 1 -- 172.21.15.83:0/955060879 <== mon.2 v2:172.21.15.117:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8269+0+0 (secure 0 0 0) 0x7f143c0cf550 con 0x7f1450074b60 2024-08-19T01:41:14.906 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:14.799+0000 7f1459074740 1 -- 172.21.15.83:0/857465308 shutdown_connections 2024-08-19T01:41:14.906 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:14.799+0000 7f144f400640 1 --2- 172.21.15.83:0/955060879 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f14340a2b80 0x7f14340a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:41:14.906 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:14.799+0000 7f1459074740 1 -- 172.21.15.83:0/857465308 wait complete. 2024-08-19T01:41:14.906 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:14.800+0000 7f1459074740 1 -- start start 2024-08-19T01:41:14.906 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:14.800+0000 7f1459074740 1 Processor -- start 2024-08-19T01:41:14.906 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:14.800+0000 7f144f400640 1 --2- 172.21.15.83:0/955060879 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f14340a2b80 0x7f14340a5040 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x55b54f65fa40 tx=0x7f1440009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:41:14.907 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:14.801+0000 7f1459074740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b54f662650 0x55b54f66a650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:41:14.907 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:14.801+0000 7f1459074740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b54f662f70 0x55b54f66ab90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:41:14.907 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:14.801+0000 7f1459074740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b54f66d830 0x55b54f66b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:41:14.907 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:14.801+0000 7f1459074740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55b54f583ae0 con 0x55b54f662f70 2024-08-19T01:41:14.907 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:14.801+0000 7f1459074740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55b54f5c4a80 con 0x55b54f66d830 2024-08-19T01:41:14.907 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:14.801+0000 7f1459074740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55b54f6704a0 con 0x55b54f662650 2024-08-19T01:41:14.907 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:14.801+0000 7f1459074740 10 client.0 _ll_register_callbacks cb 0x55b54f585190 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-19T01:41:14.907 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:14.801+0000 7f144fe00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b54f66d830 0x55b54f66b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:41:14.907 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:14.801+0000 7f144ea00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b54f662f70 0x55b54f66ab90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:41:14.907 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:14.801+0000 7f144fe00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b54f66d830 0x55b54f66b0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.83:41706/0 (socket says 172.21.15.83:41706) 2024-08-19T01:41:14.907 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:14.801+0000 7f144fe00640 1 -- 172.21.15.83:0/3793208134 learned_addr learned my addr 172.21.15.83:0/3793208134 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-08-19T01:41:14.907 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:14.801+0000 7f144f400640 1 --2- 172.21.15.83:0/3793208134 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b54f662650 0x55b54f66a650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:41:14.907 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:14.801+0000 7f144fe00640 1 -- 172.21.15.83:0/3793208134 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b54f662650 msgr2=0x55b54f66a650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:41:14.907 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:14.801+0000 7f144fe00640 1 --2- 172.21.15.83:0/3793208134 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b54f662650 0x55b54f66a650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:41:14.908 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:14.801+0000 7f144fe00640 1 -- 172.21.15.83:0/3793208134 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b54f662f70 msgr2=0x55b54f66ab90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:41:14.908 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:14.801+0000 7f144fe00640 1 --2- 172.21.15.83:0/3793208134 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b54f662f70 0x55b54f66ab90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:41:14.908 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:14.801+0000 7f144fe00640 1 -- 172.21.15.83:0/3793208134 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b54f33a6a0 con 0x55b54f66d830 2024-08-19T01:41:14.908 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:14.802+0000 7f144fe00640 1 --2- 172.21.15.83:0/3793208134 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b54f66d830 0x55b54f66b0d0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f1448006010 tx=0x7f1448030650 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e070e622a669e5e server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:41:14.908 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:14.802+0000 7f1439600640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-08-19T01:41:14.908 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:14.802+0000 7f1439600640 1 -- 172.21.15.83:0/3793208134 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f144804a070 con 0x55b54f66d830 2024-08-19T01:41:14.908 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:14.802+0000 7f1439600640 1 -- 172.21.15.83:0/3793208134 <== mon.2 v2:172.21.15.117:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8269+0+0 (secure 0 0 0) 0x7f1448031640 con 0x55b54f66d830 2024-08-19T01:41:14.908 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:14.802+0000 7f1459074740 1 -- 172.21.15.83:0/3793208134 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1448037040 con 0x55b54f66d830 2024-08-19T01:41:14.908 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:14.802+0000 7f1439600640 1 -- 172.21.15.83:0/3793208134 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f1448056050 con 0x55b54f66d830 2024-08-19T01:41:14.908 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:14.802+0000 7f1459074740 10 client.25892 fetch_fsmap learned FSMap version 27 2024-08-19T01:41:14.909 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:14.802+0000 7f1459074740 1 -- 172.21.15.83:0/3793208134 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55b54f684740 con 0x55b54f66d830 2024-08-19T01:41:14.909 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:14.802+0000 7f1439600640 1 -- 172.21.15.83:0/3793208134 <== mon.2 v2:172.21.15.117:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1448037040 con 0x55b54f66d830 2024-08-19T01:41:14.909 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:14.803+0000 7f1439600640 1 -- 172.21.15.83:0/3793208134 <== mon.2 v2:172.21.15.117:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f144803c040 con 0x55b54f66d830 2024-08-19T01:41:14.909 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:14.803+0000 7f1459074740 10 client.25892 fetch_fsmap finished waiting for FSMap version 27 2024-08-19T01:41:14.909 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:14.803+0000 7f1459074740 10 client.25892 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-19T01:41:14.909 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:14.803+0000 7f1459074740 20 client.25892 populate_metadata read hostname 'smithi083' 2024-08-19T01:41:14.909 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:14.803+0000 7f1459074740 1 --2- 172.21.15.83:0/3793208134 >> [v2:172.21.15.117:6838/1967595704,v1:172.21.15.117:6839/1967595704] conn(0x55b54f685a90 0x55b54f687e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:41:14.909 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:14.803+0000 7f1459074740 4 client.25892 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-19T01:41:14.909 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:14.803+0000 7f1459074740 1 -- 172.21.15.83:0/3793208134 --> [v2:172.21.15.117:6838/1967595704,v1:172.21.15.117:6839/1967595704] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b54f5d6030 con 0x55b54f685a90 2024-08-19T01:41:14.909 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:14.803+0000 7f144f400640 1 --2- 172.21.15.83:0/3793208134 >> [v2:172.21.15.117:6838/1967595704,v1:172.21.15.117:6839/1967595704] conn(0x55b54f685a90 0x55b54f687e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:41:14.909 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:14.803+0000 7f1439600640 10 client.25892 ms_handle_connect on v2:172.21.15.117:6838/1967595704 2024-08-19T01:41:14.909 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:14.803+0000 7f144f400640 1 --2- 172.21.15.83:0/3793208134 >> [v2:172.21.15.117:6838/1967595704,v1:172.21.15.117:6839/1967595704] conn(0x55b54f685a90 0x55b54f687e80 crc :-1 s=READY pgs=111 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d50bdaa826549795 server_cookie=25863742071368de in_seq=0 out_seq=0 2024-08-19T01:41:14.909 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:14.805+0000 7f1439600640 1 -- 172.21.15.83:0/3793208134 <== mds.0 v2:172.21.15.117:6838/1967595704 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55b54f5d6030 con 0x55b54f685a90 2024-08-19T01:41:14.909 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:14.805+0000 7f1439600640 10 client.25892 handle_command_reply: tid=0 2024-08-19T01:41:14.910 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:14.805+0000 7f1459074740 1 client.25892 shutdown 2024-08-19T01:41:14.910 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:14.805+0000 7f1459074740 1 -- 172.21.15.83:0/3793208134 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b54f66d830 msgr2=0x55b54f66b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:41:14.910 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:14.805+0000 7f1459074740 1 --2- 172.21.15.83:0/3793208134 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b54f66d830 0x55b54f66b0d0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f1448006010 tx=0x7f1448030650 comp rx=0 tx=0).stop 2024-08-19T01:41:14.910 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:14.805+0000 7f1459074740 1 -- 172.21.15.83:0/3793208134 shutdown_connections 2024-08-19T01:41:14.910 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:14.805+0000 7f1459074740 1 --2- 172.21.15.83:0/3793208134 >> [v2:172.21.15.117:6838/1967595704,v1:172.21.15.117:6839/1967595704] conn(0x55b54f685a90 0x55b54f687e80 crc :-1 s=READY pgs=111 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:41:14.910 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:14.805+0000 7f1459074740 1 --2- 172.21.15.83:0/3793208134 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b54f66d830 0x55b54f66b0d0 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:41:14.910 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:14.805+0000 7f1439600640 0 client.25892 ms_handle_reset on v2:172.21.15.117:6838/1967595704 2024-08-19T01:41:14.910 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:14.805+0000 7f1459074740 1 --2- 172.21.15.83:0/3793208134 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b54f662f70 0x55b54f66ab90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:41:14.910 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:14.805+0000 7f1459074740 1 --2- 172.21.15.83:0/3793208134 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b54f662650 0x55b54f66a650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:41:14.910 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:14.805+0000 7f1459074740 1 -- 172.21.15.83:0/3793208134 >> 172.21.15.83:0/3793208134 conn(0x55b54f65ebe0 msgr2=0x55b54f664960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:41:14.910 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:14.806+0000 7f1459074740 1 -- 172.21.15.83:0/3793208134 shutdown_connections 2024-08-19T01:41:14.910 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:14.806+0000 7f1459074740 1 -- 172.21.15.83:0/3793208134 wait complete. 2024-08-19T01:41:14.910 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:14.806+0000 7f1459074740 20 client.25892 trim_cache size 0 max 16384 2024-08-19T01:41:14.911 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:14.824+0000 7f1459074740 1 Processor -- start 2024-08-19T01:41:14.911 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:14.824+0000 7f1459074740 1 -- start start 2024-08-19T01:41:14.911 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:14.824+0000 7f1459074740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b54f662650 0x55b54f665150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:41:14.911 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:14.824+0000 7f1459074740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b54f662f70 0x55b54f6816c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:41:14.911 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:14.825+0000 7f1459074740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b54f66d830 0x55b54f68aea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:41:14.911 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:14.825+0000 7f1459074740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f14501ad910 con 0x55b54f662f70 2024-08-19T01:41:14.911 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:14.825+0000 7f1459074740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f14501ab1e0 con 0x55b54f662650 2024-08-19T01:41:14.911 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:14.825+0000 7f1459074740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f14501a9030 con 0x55b54f66d830 2024-08-19T01:41:14.911 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:14.825+0000 7f144ea00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b54f662f70 0x55b54f6816c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:41:14.911 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:14.825+0000 7f144f400640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b54f662650 0x55b54f665150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:41:14.911 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:14.825+0000 7f144ea00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b54f662f70 0x55b54f6816c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:57110/0 (socket says 172.21.15.83:57110) 2024-08-19T01:41:14.911 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:14.825+0000 7f144f400640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b54f662650 0x55b54f665150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.83:41722/0 (socket says 172.21.15.83:41722) 2024-08-19T01:41:14.911 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:14.825+0000 7f144ea00640 1 -- 172.21.15.83:0/4197586473 learned_addr learned my addr 172.21.15.83:0/4197586473 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-08-19T01:41:14.911 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:14.825+0000 7f144fe00640 1 --2- 172.21.15.83:0/4197586473 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b54f66d830 0x55b54f68aea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:41:14.911 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:14.825+0000 7f144ea00640 1 -- 172.21.15.83:0/4197586473 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b54f66d830 msgr2=0x55b54f68aea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:41:14.912 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:14.825+0000 7f144ea00640 1 --2- 172.21.15.83:0/4197586473 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b54f66d830 0x55b54f68aea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:41:14.912 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:14.825+0000 7f144ea00640 1 -- 172.21.15.83:0/4197586473 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b54f662650 msgr2=0x55b54f665150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:41:14.912 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:14.825+0000 7f144ea00640 1 --2- 172.21.15.83:0/4197586473 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b54f662650 0x55b54f665150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:41:14.912 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:14.825+0000 7f144ea00640 1 -- 172.21.15.83:0/4197586473 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b54f662da0 con 0x55b54f662f70 2024-08-19T01:41:14.912 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:14.825+0000 7f144f400640 1 --2- 172.21.15.83:0/4197586473 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b54f662650 0x55b54f665150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-19T01:41:14.912 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:14.825+0000 7f144fe00640 1 --2- 172.21.15.83:0/4197586473 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b54f66d830 0x55b54f68aea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-19T01:41:14.912 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:14.826+0000 7f144ea00640 1 --2- 172.21.15.83:0/4197586473 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b54f662f70 0x55b54f6816c0 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f143c098700 tx=0x7f143c009700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6ab9aba85c848ded server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:41:14.912 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:14.826+0000 7f143be00640 1 -- 172.21.15.83:0/4197586473 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f143c09bd30 con 0x55b54f662f70 2024-08-19T01:41:14.912 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:14.826+0000 7f143be00640 1 -- 172.21.15.83:0/4197586473 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f143c0a00e0 con 0x55b54f662f70 2024-08-19T01:41:14.912 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:14.826+0000 7f143be00640 1 -- 172.21.15.83:0/4197586473 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f143c0cf340 con 0x55b54f662f70 2024-08-19T01:41:14.912 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:14.826+0000 7f1459074740 1 -- 172.21.15.83:0/4197586473 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b54f662f70 msgr2=0x55b54f6816c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:41:14.912 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:14.826+0000 7f1459074740 1 --2- 172.21.15.83:0/4197586473 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b54f662f70 0x55b54f6816c0 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f143c098700 tx=0x7f143c009700 comp rx=0 tx=0).stop 2024-08-19T01:41:14.912 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:14.827+0000 7f1459074740 1 -- 172.21.15.83:0/4197586473 shutdown_connections 2024-08-19T01:41:14.912 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:14.827+0000 7f1459074740 1 --2- 172.21.15.83:0/4197586473 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b54f66d830 0x55b54f68aea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:41:14.913 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:14.827+0000 7f1459074740 1 --2- 172.21.15.83:0/4197586473 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b54f662f70 0x55b54f6816c0 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:41:14.913 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:14.827+0000 7f1459074740 1 --2- 172.21.15.83:0/4197586473 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b54f662650 0x55b54f665150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:41:14.913 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:14.827+0000 7f1459074740 1 -- 172.21.15.83:0/4197586473 >> 172.21.15.83:0/4197586473 conn(0x55b54f65ebe0 msgr2=0x55b54f680fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:41:14.913 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:14.827+0000 7f1459074740 1 -- 172.21.15.83:0/4197586473 shutdown_connections 2024-08-19T01:41:14.913 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:14.827+0000 7f1459074740 1 -- 172.21.15.83:0/4197586473 wait complete. 2024-08-19T01:41:14.913 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:14.827+0000 7f1459074740 1 -- start start 2024-08-19T01:41:14.913 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:14.827+0000 7f1459074740 1 Processor -- start 2024-08-19T01:41:14.913 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:14.828+0000 7f1459074740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b54f662650 0x55b54f68b530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:41:14.913 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:14.828+0000 7f1459074740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b54f662f70 0x55b54f688880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:41:14.913 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:14.828+0000 7f1459074740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b54f66d830 0x55b54f688dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:41:14.913 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:14.828+0000 7f1459074740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f14501a96a0 con 0x55b54f662f70 2024-08-19T01:41:14.913 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:14.828+0000 7f1459074740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f14501aab10 con 0x55b54f66d830 2024-08-19T01:41:14.913 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:14.828+0000 7f1459074740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f14501a9d70 con 0x55b54f662650 2024-08-19T01:41:14.913 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:14.828+0000 7f1459074740 10 client.0 _ll_register_callbacks cb 0x7f1440009b70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-19T01:41:14.914 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:14.828+0000 7f144fe00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b54f66d830 0x55b54f688dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:41:14.914 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:14.828+0000 7f144fe00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b54f66d830 0x55b54f688dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.83:41726/0 (socket says 172.21.15.83:41726) 2024-08-19T01:41:14.914 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:14.828+0000 7f144fe00640 1 -- 172.21.15.83:0/131038591 learned_addr learned my addr 172.21.15.83:0/131038591 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-08-19T01:41:14.914 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:14.828+0000 7f144f400640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b54f662650 0x55b54f68b530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:41:14.914 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:14.828+0000 7f144ea00640 1 --2- 172.21.15.83:0/131038591 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b54f662f70 0x55b54f688880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:41:14.914 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:14.828+0000 7f144fe00640 1 -- 172.21.15.83:0/131038591 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b54f662650 msgr2=0x55b54f68b530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:41:14.914 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:14.828+0000 7f144fe00640 1 --2- 172.21.15.83:0/131038591 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b54f662650 0x55b54f68b530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:41:14.914 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:14.828+0000 7f144fe00640 1 -- 172.21.15.83:0/131038591 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b54f662f70 msgr2=0x55b54f688880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:41:14.914 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:14.828+0000 7f144fe00640 1 --2- 172.21.15.83:0/131038591 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b54f662f70 0x55b54f688880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:41:14.914 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:14.828+0000 7f144fe00640 1 -- 172.21.15.83:0/131038591 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b54f6895f0 con 0x55b54f66d830 2024-08-19T01:41:14.914 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:14.829+0000 7f144fe00640 1 --2- 172.21.15.83:0/131038591 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b54f66d830 0x55b54f688dc0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f1448009be0 tx=0x7f1448009480 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a11333e8631f6b79 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:41:14.914 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:14.829+0000 7f1439600640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-08-19T01:41:14.914 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:14.829+0000 7f1439600640 1 -- 172.21.15.83:0/131038591 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1448053020 con 0x55b54f66d830 2024-08-19T01:41:14.915 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:14.829+0000 7f1459074740 1 -- 172.21.15.83:0/131038591 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f143c0a00e0 con 0x55b54f66d830 2024-08-19T01:41:14.915 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:14.829+0000 7f1439600640 1 -- 172.21.15.83:0/131038591 <== mon.2 v2:172.21.15.117:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8269+0+0 (secure 0 0 0) 0x7f1448003b70 con 0x55b54f66d830 2024-08-19T01:41:14.915 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:14.829+0000 7f1439600640 1 -- 172.21.15.83:0/131038591 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f144804a040 con 0x55b54f66d830 2024-08-19T01:41:14.915 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:14.829+0000 7f1459074740 10 client.25898 fetch_fsmap learned FSMap version 27 2024-08-19T01:41:14.915 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:14.829+0000 7f1459074740 1 -- 172.21.15.83:0/131038591 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55b54f6f0350 con 0x55b54f66d830 2024-08-19T01:41:14.915 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:14.829+0000 7f1439600640 1 -- 172.21.15.83:0/131038591 <== mon.2 v2:172.21.15.117:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f143c0a00e0 con 0x55b54f66d830 2024-08-19T01:41:14.915 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:14.829+0000 7f1439600640 1 -- 172.21.15.83:0/131038591 <== mon.2 v2:172.21.15.117:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f144803c050 con 0x55b54f66d830 2024-08-19T01:41:14.915 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:14.830+0000 7f1459074740 10 client.25898 fetch_fsmap finished waiting for FSMap version 27 2024-08-19T01:41:14.915 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:14.830+0000 7f1459074740 10 client.25898 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-19T01:41:14.915 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:14.830+0000 7f1459074740 20 client.25898 populate_metadata read hostname 'smithi083' 2024-08-19T01:41:14.915 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:14.830+0000 7f1459074740 1 --2- 172.21.15.83:0/131038591 >> [v2:172.21.15.117:6838/1967595704,v1:172.21.15.117:6839/1967595704] conn(0x55b54f6f0950 0x55b54f6f2d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:41:14.915 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:14.830+0000 7f1459074740 4 client.25898 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-08-19T01:41:14.915 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:14.830+0000 7f1459074740 1 -- 172.21.15.83:0/131038591 --> [v2:172.21.15.117:6838/1967595704,v1:172.21.15.117:6839/1967595704] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55b54f5d0540 con 0x55b54f6f0950 2024-08-19T01:41:14.915 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:14.830+0000 7f144f400640 1 --2- 172.21.15.83:0/131038591 >> [v2:172.21.15.117:6838/1967595704,v1:172.21.15.117:6839/1967595704] conn(0x55b54f6f0950 0x55b54f6f2d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:41:14.916 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:14.830+0000 7f144f400640 1 --2- 172.21.15.83:0/131038591 >> [v2:172.21.15.117:6838/1967595704,v1:172.21.15.117:6839/1967595704] conn(0x55b54f6f0950 0x55b54f6f2d70 crc :-1 s=READY pgs=112 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4272b471677f466b server_cookie=b9255dc6a3ccbba1 in_seq=0 out_seq=0 2024-08-19T01:41:14.916 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:14.830+0000 7f1439600640 10 client.25898 ms_handle_connect on v2:172.21.15.117:6838/1967595704 2024-08-19T01:41:14.916 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:14.832+0000 7f1439600640 1 -- 172.21.15.83:0/131038591 <== mds.0 v2:172.21.15.117:6838/1967595704 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55b54f5d0540 con 0x55b54f6f0950 2024-08-19T01:41:14.916 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:14.832+0000 7f1439600640 10 client.25898 handle_command_reply: tid=0 2024-08-19T01:41:14.916 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:14.832+0000 7f1459074740 1 client.25898 shutdown 2024-08-19T01:41:14.916 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:14.832+0000 7f1459074740 1 -- 172.21.15.83:0/131038591 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b54f66d830 msgr2=0x55b54f688dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:41:14.916 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:14.832+0000 7f1459074740 1 --2- 172.21.15.83:0/131038591 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b54f66d830 0x55b54f688dc0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f1448009be0 tx=0x7f1448009480 comp rx=0 tx=0).stop 2024-08-19T01:41:14.916 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:14.832+0000 7f1459074740 1 -- 172.21.15.83:0/131038591 shutdown_connections 2024-08-19T01:41:14.916 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:14.832+0000 7f1459074740 1 --2- 172.21.15.83:0/131038591 >> [v2:172.21.15.117:6838/1967595704,v1:172.21.15.117:6839/1967595704] conn(0x55b54f6f0950 0x55b54f6f2d70 crc :-1 s=READY pgs=112 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:41:14.916 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:14.832+0000 7f1459074740 1 --2- 172.21.15.83:0/131038591 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b54f66d830 0x55b54f688dc0 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:41:14.916 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:14.832+0000 7f1459074740 1 --2- 172.21.15.83:0/131038591 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b54f662f70 0x55b54f688880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:41:14.916 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:14.832+0000 7f1459074740 1 --2- 172.21.15.83:0/131038591 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b54f662650 0x55b54f68b530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:41:14.916 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:14.832+0000 7f1459074740 1 -- 172.21.15.83:0/131038591 >> 172.21.15.83:0/131038591 conn(0x55b54f65ebe0 msgr2=0x55b54f6817d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:41:14.916 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:14.833+0000 7f1439600640 0 client.25898 ms_handle_reset on v2:172.21.15.117:6838/1967595704 2024-08-19T01:41:14.916 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:14.833+0000 7f1459074740 1 -- 172.21.15.83:0/131038591 shutdown_connections 2024-08-19T01:41:14.917 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-08-19T01:41:14.917 INFO:teuthology.orchestra.run.smithi083.stdout: "return_code": 0, 2024-08-19T01:41:14.917 INFO:teuthology.orchestra.run.smithi083.stdout: "scrub_tag": "57f5c524-dcae-4283-b318-37fdde10e0c1", 2024-08-19T01:41:14.917 INFO:teuthology.orchestra.run.smithi083.stdout: "mode": "asynchronous" 2024-08-19T01:41:14.917 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-08-19T01:41:14.918 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:14.833+0000 7f1459074740 1 -- 172.21.15.83:0/131038591 wait complete. 2024-08-19T01:41:14.918 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:14.833+0000 7f1459074740 20 client.25898 trim_cache size 0 max 16384 2024-08-19T01:41:14.918 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:14.834+0000 7f1454e00640 1 -- 172.21.15.83:0/955060879 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f14340a2b80 msgr2=0x7f14340a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:41:14.918 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:14.834+0000 7f1454e00640 1 --2- 172.21.15.83:0/955060879 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f14340a2b80 0x7f14340a5040 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x55b54f65fa40 tx=0x7f1440009260 comp rx=0 tx=0).stop 2024-08-19T01:41:14.918 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:14.834+0000 7f1454e00640 1 -- 172.21.15.83:0/955060879 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1450074b60 msgr2=0x7f145019a6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:41:14.918 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:14.834+0000 7f1454e00640 1 --2- 172.21.15.83:0/955060879 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1450074b60 0x7f145019a6f0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f143c00e9f0 tx=0x7f143c00edf0 comp rx=0 tx=0).stop 2024-08-19T01:41:14.918 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:14.834+0000 7f1454e00640 1 -- 172.21.15.83:0/955060879 shutdown_connections 2024-08-19T01:41:14.918 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:14.834+0000 7f1454e00640 1 --2- 172.21.15.83:0/955060879 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f14340a2b80 0x7f14340a5040 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:41:14.918 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:14.834+0000 7f1454e00640 1 --2- 172.21.15.83:0/955060879 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f145010dfb0 0x7f145019ea80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:41:14.918 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:14.834+0000 7f1454e00640 1 --2- 172.21.15.83:0/955060879 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1450074b60 0x7f145019a6f0 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:41:14.918 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:14.834+0000 7f1454e00640 1 --2- 172.21.15.83:0/955060879 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1450074240 0x7f145019a1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:41:14.918 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:14.834+0000 7f1454e00640 1 -- 172.21.15.83:0/955060879 >> 172.21.15.83:0/955060879 conn(0x7f14500fd470 msgr2=0x7f14500fe170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:41:14.919 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:14.834+0000 7f1454e00640 1 -- 172.21.15.83:0/955060879 shutdown_connections 2024-08-19T01:41:14.919 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:14.834+0000 7f1454e00640 1 -- 172.21.15.83:0/955060879 wait complete. 2024-08-19T01:41:15.202 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:41:14 smithi117 ceph-mon[22016]: pgmap v531: 97 pgs: 97 active+clean; 624 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 4.0 MiB/s wr, 1.40k op/s 2024-08-19T01:41:15.215 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:41:14 smithi167 ceph-mon[21875]: pgmap v531: 97 pgs: 97 active+clean; 624 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 4.0 MiB/s wr, 1.40k op/s 2024-08-19T01:41:15.290 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:41:14 smithi083 ceph-mon[17120]: pgmap v531: 97 pgs: 97 active+clean; 624 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 4.0 MiB/s wr, 1.40k op/s 2024-08-19T01:41:15.727 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833 shell --fsid bad8780e-5dc9-11ef-bccf-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-08-19T01:41:15.966 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:41:15 smithi083 ceph-mon[17120]: scrub queued for path: / 2024-08-19T01:41:15.966 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:41:15 smithi083 ceph-mon[17120]: scrub summary: idle+waiting paths [/] 2024-08-19T01:41:15.966 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:41:15 smithi083 ceph-mon[17120]: scrub summary: active paths [/] 2024-08-19T01:41:16.202 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:41:15 smithi117 ceph-mon[22016]: scrub queued for path: / 2024-08-19T01:41:16.202 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:41:15 smithi117 ceph-mon[22016]: scrub summary: idle+waiting paths [/] 2024-08-19T01:41:16.202 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:41:15 smithi117 ceph-mon[22016]: scrub summary: active paths [/] 2024-08-19T01:41:16.215 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:41:15 smithi167 ceph-mon[21875]: scrub queued for path: / 2024-08-19T01:41:16.216 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:41:15 smithi167 ceph-mon[21875]: scrub summary: idle+waiting paths [/] 2024-08-19T01:41:16.216 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:41:15 smithi167 ceph-mon[21875]: scrub summary: active paths [/] 2024-08-19T01:41:16.444 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/bad8780e-5dc9-11ef-bccf-c7b262605968/mon.a/config 2024-08-19T01:41:17.148 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:41:16 smithi083 ceph-mon[17120]: scrub summary: active paths [/] 2024-08-19T01:41:17.148 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:41:16 smithi083 ceph-mon[17120]: scrub summary: idle+waiting paths [/] 2024-08-19T01:41:17.148 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:41:16 smithi083 ceph-mon[17120]: scrub summary: active paths [/] 2024-08-19T01:41:17.148 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:41:16 smithi083 ceph-mon[17120]: scrub summary: idle+waiting paths [/] 2024-08-19T01:41:17.148 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:41:16 smithi083 ceph-mon[17120]: pgmap v532: 97 pgs: 97 active+clean; 631 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 3.8 MiB/s wr, 666 op/s 2024-08-19T01:41:17.202 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:41:16 smithi117 ceph-mon[22016]: scrub summary: active paths [/] 2024-08-19T01:41:17.202 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:41:16 smithi117 ceph-mon[22016]: scrub summary: idle+waiting paths [/] 2024-08-19T01:41:17.202 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:41:16 smithi117 ceph-mon[22016]: scrub summary: active paths [/] 2024-08-19T01:41:17.202 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:41:16 smithi117 ceph-mon[22016]: scrub summary: idle+waiting paths [/] 2024-08-19T01:41:17.202 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:41:16 smithi117 ceph-mon[22016]: pgmap v532: 97 pgs: 97 active+clean; 631 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 3.8 MiB/s wr, 666 op/s 2024-08-19T01:41:17.215 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:41:16 smithi167 ceph-mon[21875]: scrub summary: active paths [/] 2024-08-19T01:41:17.215 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:41:16 smithi167 ceph-mon[21875]: scrub summary: idle+waiting paths [/] 2024-08-19T01:41:17.216 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:41:16 smithi167 ceph-mon[21875]: scrub summary: active paths [/] 2024-08-19T01:41:17.216 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:41:16 smithi167 ceph-mon[21875]: scrub summary: idle+waiting paths [/] 2024-08-19T01:41:17.216 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:41:16 smithi167 ceph-mon[21875]: pgmap v532: 97 pgs: 97 active+clean; 631 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 3.8 MiB/s wr, 666 op/s 2024-08-19T01:41:17.229 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:17.226+0000 7fadc8e00640 1 -- 172.21.15.83:0/4078114888 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fadc4108120 msgr2=0x7fadc4108500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:41:17.229 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:17.226+0000 7fadc8e00640 1 --2- 172.21.15.83:0/4078114888 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fadc4108120 0x7fadc4108500 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fadb0009900 tx=0x7fadb002a420 comp rx=0 tx=0).stop 2024-08-19T01:41:17.229 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:17.227+0000 7fadc8e00640 1 -- 172.21.15.83:0/4078114888 shutdown_connections 2024-08-19T01:41:17.229 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:17.227+0000 7fadc8e00640 1 --2- 172.21.15.83:0/4078114888 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fadc410d250 0x7fadc4114dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:41:17.230 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:17.227+0000 7fadc8e00640 1 --2- 172.21.15.83:0/4078114888 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fadc4108ad0 0x7fadc410cb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:41:17.230 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:17.227+0000 7fadc8e00640 1 --2- 172.21.15.83:0/4078114888 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fadc4108120 0x7fadc4108500 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:41:17.230 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:17.227+0000 7fadc8e00640 1 -- 172.21.15.83:0/4078114888 >> 172.21.15.83:0/4078114888 conn(0x7fadc4077820 msgr2=0x7fadc4079c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:41:17.230 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:17.228+0000 7fadc8e00640 1 -- 172.21.15.83:0/4078114888 shutdown_connections 2024-08-19T01:41:17.231 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:17.228+0000 7fadc8e00640 1 -- 172.21.15.83:0/4078114888 wait complete. 2024-08-19T01:41:17.231 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:17.229+0000 7fadc8e00640 1 Processor -- start 2024-08-19T01:41:17.232 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:17.230+0000 7fadc8e00640 1 -- start start 2024-08-19T01:41:17.233 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:17.232+0000 7fadc8e00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fadc4108120 0x7fadc40824c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:41:17.234 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:17.233+0000 7fadc8e00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fadc4108ad0 0x7fadc4082a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:41:17.235 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:17.233+0000 7fadc8e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fadc410d250 0x7fadc407b730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:41:17.235 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:17.233+0000 7fadc1400640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fadc4108ad0 0x7fadc4082a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:41:17.235 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:17.233+0000 7fadc1400640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fadc4108ad0 0x7fadc4082a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.83:44910/0 (socket says 172.21.15.83:44910) 2024-08-19T01:41:17.235 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:17.234+0000 7fadc1400640 1 -- 172.21.15.83:0/1875140968 learned_addr learned my addr 172.21.15.83:0/1875140968 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-08-19T01:41:17.235 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:17.234+0000 7fadc8e00640 1 -- 172.21.15.83:0/1875140968 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fadc4118340 con 0x7fadc410d250 2024-08-19T01:41:17.236 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:17.234+0000 7fadc2800640 1 --2- 172.21.15.83:0/1875140968 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fadc410d250 0x7fadc407b730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:41:17.236 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:17.234+0000 7fadc8e00640 1 -- 172.21.15.83:0/1875140968 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fadc41181c0 con 0x7fadc4108120 2024-08-19T01:41:17.236 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:17.234+0000 7fadc8e00640 1 -- 172.21.15.83:0/1875140968 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fadc41184c0 con 0x7fadc4108ad0 2024-08-19T01:41:17.237 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:17.235+0000 7fadc1400640 1 -- 172.21.15.83:0/1875140968 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fadc4108120 msgr2=0x7fadc40824c0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-08-19T01:41:17.237 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:17.235+0000 7fadc1400640 1 --2- 172.21.15.83:0/1875140968 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fadc4108120 0x7fadc40824c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:41:17.237 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:17.235+0000 7fadc1400640 1 -- 172.21.15.83:0/1875140968 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fadc410d250 msgr2=0x7fadc407b730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:41:17.238 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:17.236+0000 7fadc1400640 1 --2- 172.21.15.83:0/1875140968 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fadc410d250 0x7fadc407b730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:41:17.238 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:17.236+0000 7fadc1400640 1 -- 172.21.15.83:0/1875140968 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fadc407bff0 con 0x7fadc4108ad0 2024-08-19T01:41:17.238 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:17.237+0000 7fadc1400640 1 --2- 172.21.15.83:0/1875140968 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fadc4108ad0 0x7fadc4082a00 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fadb400b470 tx=0x7fadb400b930 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:41:17.238 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:17.237+0000 7fadbaa00640 1 -- 172.21.15.83:0/1875140968 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fadb4015020 con 0x7fadc4108ad0 2024-08-19T01:41:17.238 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:17.237+0000 7fadbaa00640 1 -- 172.21.15.83:0/1875140968 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fadb4011d70 con 0x7fadc4108ad0 2024-08-19T01:41:17.239 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:17.237+0000 7fadbaa00640 1 -- 172.21.15.83:0/1875140968 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fadb400f340 con 0x7fadc4108ad0 2024-08-19T01:41:17.240 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:17.237+0000 7fadc8e00640 1 -- 172.21.15.83:0/1875140968 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fadc407c2e0 con 0x7fadc4108ad0 2024-08-19T01:41:17.240 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:17.238+0000 7fadc8e00640 1 -- 172.21.15.83:0/1875140968 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fadc4080280 con 0x7fadc4108ad0 2024-08-19T01:41:17.246 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:17.242+0000 7fadcb8a8740 1 Processor -- start 2024-08-19T01:41:17.247 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:17.245+0000 7fadbaa00640 1 -- 172.21.15.83:0/1875140968 <== mon.1 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7fadb4018020 con 0x7fadc4108ad0 2024-08-19T01:41:17.247 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:17.245+0000 7fadcb8a8740 1 -- start start 2024-08-19T01:41:17.248 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:17.246+0000 7fadcb8a8740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559b8d291b90 0x559b8d291f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:41:17.249 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:17.247+0000 7fadc2800640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559b8d291b90 0x559b8d291f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:41:17.249 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:17.247+0000 7fadc2800640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559b8d291b90 0x559b8d291f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.83:41746/0 (socket says 172.21.15.83:41746) 2024-08-19T01:41:17.249 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:17.247+0000 7fadcb8a8740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559b8d2924b0 0x559b8d29c830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:41:17.249 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:17.247+0000 7fadc2800640 1 -- 172.21.15.83:0/3097190860 learned_addr learned my addr 172.21.15.83:0/3097190860 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-08-19T01:41:17.249 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:17.247+0000 7fadbaa00640 1 --2- 172.21.15.83:0/1875140968 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7fada80a2c40 0x7fada80a5100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:41:17.249 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:17.247+0000 7fadcb8a8740 1 --2- 172.21.15.83:0/3097190860 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559b8d29cd70 0x559b8d29f160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:41:17.250 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:17.247+0000 7fadcb8a8740 1 -- 172.21.15.83:0/3097190860 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x559b8d1f7bf0 con 0x559b8d29cd70 2024-08-19T01:41:17.250 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:17.247+0000 7fadcb8a8740 1 -- 172.21.15.83:0/3097190860 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x559b8d18fbd0 con 0x559b8d291b90 2024-08-19T01:41:17.250 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:17.247+0000 7fadcb8a8740 1 -- 172.21.15.83:0/3097190860 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x559b8d1f3bb0 con 0x559b8d2924b0 2024-08-19T01:41:17.250 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:17.248+0000 7fadbaa00640 1 -- 172.21.15.83:0/1875140968 <== mon.1 v2:172.21.15.167:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8269+0+0 (secure 0 0 0) 0x7fadb40ceb50 con 0x7fadc4108ad0 2024-08-19T01:41:17.250 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:17.248+0000 7fadc1e00640 1 --2- 172.21.15.83:0/3097190860 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559b8d2924b0 0x559b8d29c830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:41:17.251 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:17.248+0000 7fadc2800640 1 -- 172.21.15.83:0/3097190860 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559b8d2924b0 msgr2=0x559b8d29c830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:41:17.251 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:17.248+0000 7fadc2800640 1 --2- 172.21.15.83:0/3097190860 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559b8d2924b0 0x559b8d29c830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:41:17.251 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:17.248+0000 7fadc2800640 1 -- 172.21.15.83:0/3097190860 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559b8d29cd70 msgr2=0x559b8d29f160 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-08-19T01:41:17.251 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:17.248+0000 7fadc2800640 1 --2- 172.21.15.83:0/3097190860 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559b8d29cd70 0x559b8d29f160 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:41:17.251 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:17.248+0000 7fadc2800640 1 -- 172.21.15.83:0/3097190860 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559b8d1968a0 con 0x559b8d291b90 2024-08-19T01:41:17.251 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:17.249+0000 7fadc2800640 1 --2- 172.21.15.83:0/1875140968 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7fada80a2c40 0x7fada80a5100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:41:17.254 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:17.253+0000 7fadc2800640 1 --2- 172.21.15.83:0/3097190860 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559b8d291b90 0x559b8d291f70 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fadbc0124d0 tx=0x7fadbc012990 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fe1f995d20a10455 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:41:17.258 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:17.256+0000 7fadaf400640 1 -- 172.21.15.83:0/3097190860 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fadbc00ee20 con 0x559b8d291b90 2024-08-19T01:41:17.258 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:17.256+0000 7fadaf400640 1 -- 172.21.15.83:0/3097190860 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fadbc004480 con 0x559b8d291b90 2024-08-19T01:41:17.259 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:17.257+0000 7fadcb8a8740 1 -- 172.21.15.83:0/3097190860 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559b8d291b90 msgr2=0x559b8d291f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:41:17.259 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:17.257+0000 7fadcb8a8740 1 --2- 172.21.15.83:0/3097190860 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559b8d291b90 0x559b8d291f70 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fadbc0124d0 tx=0x7fadbc012990 comp rx=0 tx=0).stop 2024-08-19T01:41:17.259 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:17.258+0000 7fadc2800640 1 --2- 172.21.15.83:0/1875140968 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7fada80a2c40 0x7fada80a5100 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7fadbc021000 tx=0x7fadbc01e000 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:41:17.260 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:17.258+0000 7fadcb8a8740 1 -- 172.21.15.83:0/3097190860 shutdown_connections 2024-08-19T01:41:17.260 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:17.258+0000 7fadcb8a8740 1 --2- 172.21.15.83:0/3097190860 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559b8d29cd70 0x559b8d29f160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:41:17.260 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:17.258+0000 7fadcb8a8740 1 --2- 172.21.15.83:0/3097190860 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559b8d2924b0 0x559b8d29c830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:41:17.260 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:17.258+0000 7fadcb8a8740 1 --2- 172.21.15.83:0/3097190860 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559b8d291b90 0x559b8d291f70 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:41:17.260 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:17.258+0000 7fadcb8a8740 1 -- 172.21.15.83:0/3097190860 >> 172.21.15.83:0/3097190860 conn(0x559b8d28e120 msgr2=0x559b8d28e500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:41:17.261 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:17.259+0000 7fadcb8a8740 1 -- 172.21.15.83:0/3097190860 shutdown_connections 2024-08-19T01:41:17.261 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:17.259+0000 7fadcb8a8740 1 -- 172.21.15.83:0/3097190860 wait complete. 2024-08-19T01:41:17.261 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:17.260+0000 7fadcb8a8740 1 -- start start 2024-08-19T01:41:17.262 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:17.261+0000 7fadcb8a8740 1 Processor -- start 2024-08-19T01:41:17.263 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:17.262+0000 7fadcb8a8740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559b8d291b90 0x559b8d299b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:41:17.264 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:17.262+0000 7fadcb8a8740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559b8d2924b0 0x559b8d29a0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:41:17.264 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:17.262+0000 7fadcb8a8740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559b8d29cd70 0x559b8d29a610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:41:17.265 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:17.263+0000 7fadcb8a8740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x559b8d29f9e0 con 0x559b8d291b90 2024-08-19T01:41:17.265 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:17.263+0000 7fadcb8a8740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x559b8d1b2d40 con 0x559b8d29cd70 2024-08-19T01:41:17.265 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:17.263+0000 7fadcb8a8740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x559b8d29fb60 con 0x559b8d2924b0 2024-08-19T01:41:17.265 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:17.263+0000 7fadc1e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559b8d291b90 0x559b8d299b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:41:17.265 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:17.263+0000 7fadcb8a8740 10 client.0 _ll_register_callbacks cb 0x559b8d17bb10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-19T01:41:17.265 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:17.263+0000 7fadc1e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559b8d291b90 0x559b8d299b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:57160/0 (socket says 172.21.15.83:57160) 2024-08-19T01:41:17.265 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:17.263+0000 7fadc1e00640 1 -- 172.21.15.83:0/769128805 learned_addr learned my addr 172.21.15.83:0/769128805 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-08-19T01:41:17.266 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:17.264+0000 7fadc1400640 1 --2- 172.21.15.83:0/769128805 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559b8d2924b0 0x559b8d29a0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:41:17.267 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:17.265+0000 7fadc2800640 1 --2- 172.21.15.83:0/769128805 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559b8d29cd70 0x559b8d29a610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:41:17.267 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:17.265+0000 7fadc1e00640 1 -- 172.21.15.83:0/769128805 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559b8d2924b0 msgr2=0x559b8d29a0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:41:17.267 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:17.265+0000 7fadc1e00640 1 --2- 172.21.15.83:0/769128805 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559b8d2924b0 0x559b8d29a0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:41:17.267 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:17.265+0000 7fadc1e00640 1 -- 172.21.15.83:0/769128805 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559b8d29cd70 msgr2=0x559b8d29a610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:41:17.267 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:17.265+0000 7fadc1e00640 1 --2- 172.21.15.83:0/769128805 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559b8d29cd70 0x559b8d29a610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:41:17.268 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:17.265+0000 7fadc1e00640 1 -- 172.21.15.83:0/769128805 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559b8cf693c0 con 0x559b8d291b90 2024-08-19T01:41:17.268 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:17.266+0000 7fadc1e00640 1 --2- 172.21.15.83:0/769128805 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559b8d291b90 0x559b8d299b90 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7fadb002a990 tx=0x7fadb0005270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=75b232c5b57fc035 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:41:17.268 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:17.266+0000 7fada3e00640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-08-19T01:41:17.269 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:17.267+0000 7fada3e00640 1 -- 172.21.15.83:0/769128805 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fadb0040070 con 0x559b8d291b90 2024-08-19T01:41:17.269 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:17.267+0000 7fada3e00640 1 -- 172.21.15.83:0/769128805 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8269+0+0 (secure 0 0 0) 0x7fadb00046c0 con 0x559b8d291b90 2024-08-19T01:41:17.269 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:17.267+0000 7fadcb8a8740 1 -- 172.21.15.83:0/769128805 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fadbc004480 con 0x559b8d291b90 2024-08-19T01:41:17.269 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:17.267+0000 7fada3e00640 1 -- 172.21.15.83:0/769128805 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fadb0035040 con 0x559b8d291b90 2024-08-19T01:41:17.269 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:17.268+0000 7fadcb8a8740 10 client.15861 fetch_fsmap learned FSMap version 27 2024-08-19T01:41:17.269 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:17.268+0000 7fadcb8a8740 1 -- 172.21.15.83:0/769128805 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x559b8d2ab660 con 0x559b8d291b90 2024-08-19T01:41:17.270 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:17.268+0000 7fada3e00640 1 -- 172.21.15.83:0/769128805 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fadb004c070 con 0x559b8d291b90 2024-08-19T01:41:17.270 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:17.269+0000 7fada3e00640 1 -- 172.21.15.83:0/769128805 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7fadb003a040 con 0x559b8d291b90 2024-08-19T01:41:17.270 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:17.269+0000 7fadcb8a8740 10 client.15861 fetch_fsmap finished waiting for FSMap version 27 2024-08-19T01:41:17.270 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:17.269+0000 7fadcb8a8740 10 client.15861 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-19T01:41:17.270 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:17.269+0000 7fadcb8a8740 20 client.15861 populate_metadata read hostname 'smithi083' 2024-08-19T01:41:17.271 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:17.269+0000 7fadcb8a8740 1 --2- 172.21.15.83:0/769128805 >> [v2:172.21.15.117:6838/1967595704,v1:172.21.15.117:6839/1967595704] conn(0x559b8d2acb70 0x559b8d2aefa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:41:17.271 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:17.269+0000 7fadcb8a8740 4 client.15861 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-19T01:41:17.271 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:17.269+0000 7fadcb8a8740 1 -- 172.21.15.83:0/769128805 --> [v2:172.21.15.117:6838/1967595704,v1:172.21.15.117:6839/1967595704] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559b8d205160 con 0x559b8d2acb70 2024-08-19T01:41:17.272 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:17.270+0000 7fadc1400640 1 --2- 172.21.15.83:0/769128805 >> [v2:172.21.15.117:6838/1967595704,v1:172.21.15.117:6839/1967595704] conn(0x559b8d2acb70 0x559b8d2aefa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:41:17.273 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:17.272+0000 7fadc1400640 1 --2- 172.21.15.83:0/769128805 >> [v2:172.21.15.117:6838/1967595704,v1:172.21.15.117:6839/1967595704] conn(0x559b8d2acb70 0x559b8d2aefa0 crc :-1 s=READY pgs=113 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f149fb6b9b0e7936 server_cookie=3d8481a616a67008 in_seq=0 out_seq=0 2024-08-19T01:41:17.274 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:17.272+0000 7fada3e00640 10 client.15861 ms_handle_connect on v2:172.21.15.117:6838/1967595704 2024-08-19T01:41:17.277 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:17.275+0000 7fada3e00640 1 -- 172.21.15.83:0/769128805 <== mds.0 v2:172.21.15.117:6838/1967595704 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x559b8d205160 con 0x559b8d2acb70 2024-08-19T01:41:17.277 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:17.275+0000 7fada3e00640 10 client.15861 handle_command_reply: tid=0 2024-08-19T01:41:17.277 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:17.275+0000 7fadcb8a8740 1 client.15861 shutdown 2024-08-19T01:41:17.278 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:17.276+0000 7fadcb8a8740 1 -- 172.21.15.83:0/769128805 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559b8d291b90 msgr2=0x559b8d299b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:41:17.278 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:17.276+0000 7fadcb8a8740 1 --2- 172.21.15.83:0/769128805 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559b8d291b90 0x559b8d299b90 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7fadb002a990 tx=0x7fadb0005270 comp rx=0 tx=0).stop 2024-08-19T01:41:17.278 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:17.276+0000 7fadcb8a8740 1 -- 172.21.15.83:0/769128805 shutdown_connections 2024-08-19T01:41:17.278 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:17.276+0000 7fadcb8a8740 1 --2- 172.21.15.83:0/769128805 >> [v2:172.21.15.117:6838/1967595704,v1:172.21.15.117:6839/1967595704] conn(0x559b8d2acb70 0x559b8d2aefa0 crc :-1 s=READY pgs=113 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:41:17.278 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:17.276+0000 7fadcb8a8740 1 --2- 172.21.15.83:0/769128805 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559b8d29cd70 0x559b8d29a610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:41:17.278 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:17.276+0000 7fadcb8a8740 1 --2- 172.21.15.83:0/769128805 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559b8d2924b0 0x559b8d29a0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:41:17.278 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:17.276+0000 7fada3e00640 0 client.15861 ms_handle_reset on v2:172.21.15.117:6838/1967595704 2024-08-19T01:41:17.278 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:17.276+0000 7fadcb8a8740 1 --2- 172.21.15.83:0/769128805 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559b8d291b90 0x559b8d299b90 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:41:17.279 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:17.276+0000 7fadcb8a8740 1 -- 172.21.15.83:0/769128805 >> 172.21.15.83:0/769128805 conn(0x559b8d28e120 msgr2=0x559b8d293e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:41:17.279 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:17.276+0000 7fadcb8a8740 1 -- 172.21.15.83:0/769128805 shutdown_connections 2024-08-19T01:41:17.279 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:17.276+0000 7fadcb8a8740 1 -- 172.21.15.83:0/769128805 wait complete. 2024-08-19T01:41:17.279 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:17.276+0000 7fadcb8a8740 20 client.15861 trim_cache size 0 max 16384 2024-08-19T01:41:17.319 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:17.317+0000 7fadcb8a8740 1 Processor -- start 2024-08-19T01:41:17.320 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:17.317+0000 7fadcb8a8740 1 -- start start 2024-08-19T01:41:17.320 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:17.317+0000 7fadcb8a8740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559b8d291b90 0x559b8d293cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:41:17.320 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:17.318+0000 7fadcb8a8740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559b8d2924b0 0x559b8d2b4870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:41:17.320 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:17.318+0000 7fadcb8a8740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559b8d29cd70 0x559b8d2b4db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:41:17.320 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:17.318+0000 7fadcb8a8740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fadc41be9e0 con 0x559b8d29cd70 2024-08-19T01:41:17.320 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:17.318+0000 7fadcb8a8740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fadc41bc2b0 con 0x559b8d291b90 2024-08-19T01:41:17.320 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:17.318+0000 7fadcb8a8740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fadc41ba100 con 0x559b8d2924b0 2024-08-19T01:41:17.320 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:17.318+0000 7fadc1400640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559b8d2924b0 0x559b8d2b4870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:41:17.320 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:17.318+0000 7fadc1400640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559b8d2924b0 0x559b8d2b4870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.83:44946/0 (socket says 172.21.15.83:44946) 2024-08-19T01:41:17.320 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:17.318+0000 7fadc1400640 1 -- 172.21.15.83:0/2576652554 learned_addr learned my addr 172.21.15.83:0/2576652554 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-08-19T01:41:17.320 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:17.318+0000 7fadc1e00640 1 --2- 172.21.15.83:0/2576652554 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559b8d291b90 0x559b8d293cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:41:17.321 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:17.319+0000 7fadc1400640 1 -- 172.21.15.83:0/2576652554 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559b8d291b90 msgr2=0x559b8d293cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:41:17.321 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:17.319+0000 7fadc2800640 1 --2- 172.21.15.83:0/2576652554 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559b8d29cd70 0x559b8d2b4db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:41:17.322 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:17.319+0000 7fadc1400640 1 --2- 172.21.15.83:0/2576652554 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559b8d291b90 0x559b8d293cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:41:17.322 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:17.319+0000 7fadc1400640 1 -- 172.21.15.83:0/2576652554 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559b8d29cd70 msgr2=0x559b8d2b4db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:41:17.322 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:17.319+0000 7fadc1400640 1 --2- 172.21.15.83:0/2576652554 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559b8d29cd70 0x559b8d2b4db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:41:17.322 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:17.319+0000 7fadc1400640 1 -- 172.21.15.83:0/2576652554 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559b8d28f290 con 0x559b8d2924b0 2024-08-19T01:41:17.322 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:17.319+0000 7fadc2800640 1 --2- 172.21.15.83:0/2576652554 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559b8d29cd70 0x559b8d2b4db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-19T01:41:17.322 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:17.320+0000 7fadc1400640 1 --2- 172.21.15.83:0/2576652554 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559b8d2924b0 0x559b8d2b4870 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fadb40977f0 tx=0x7fadb40107f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7765c528975079ec server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:41:17.322 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:17.321+0000 7fadaea00640 1 -- 172.21.15.83:0/2576652554 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fadb4099430 con 0x559b8d2924b0 2024-08-19T01:41:17.323 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:17.321+0000 7fadaea00640 1 -- 172.21.15.83:0/2576652554 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fadb4010550 con 0x559b8d2924b0 2024-08-19T01:41:17.323 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:17.321+0000 7fadaea00640 1 -- 172.21.15.83:0/2576652554 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fadb4017e40 con 0x559b8d2924b0 2024-08-19T01:41:17.324 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:17.323+0000 7fadcb8a8740 1 -- 172.21.15.83:0/2576652554 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559b8d2924b0 msgr2=0x559b8d2b4870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:41:17.324 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:17.323+0000 7fadcb8a8740 1 --2- 172.21.15.83:0/2576652554 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559b8d2924b0 0x559b8d2b4870 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fadb40977f0 tx=0x7fadb40107f0 comp rx=0 tx=0).stop 2024-08-19T01:41:17.324 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:17.323+0000 7fadcb8a8740 1 -- 172.21.15.83:0/2576652554 shutdown_connections 2024-08-19T01:41:17.325 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:17.323+0000 7fadcb8a8740 1 --2- 172.21.15.83:0/2576652554 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559b8d29cd70 0x559b8d2b4db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:41:17.325 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:17.323+0000 7fadcb8a8740 1 --2- 172.21.15.83:0/2576652554 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559b8d2924b0 0x559b8d2b4870 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:41:17.325 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:17.323+0000 7fadcb8a8740 1 --2- 172.21.15.83:0/2576652554 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559b8d291b90 0x559b8d293cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:41:17.325 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:17.323+0000 7fadcb8a8740 1 -- 172.21.15.83:0/2576652554 >> 172.21.15.83:0/2576652554 conn(0x559b8d28e120 msgr2=0x559b8d2935e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:41:17.325 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:17.324+0000 7fadcb8a8740 1 -- 172.21.15.83:0/2576652554 shutdown_connections 2024-08-19T01:41:17.326 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:17.324+0000 7fadcb8a8740 1 -- 172.21.15.83:0/2576652554 wait complete. 2024-08-19T01:41:17.326 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:17.324+0000 7fadcb8a8740 1 -- start start 2024-08-19T01:41:17.326 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:17.324+0000 7fadcb8a8740 1 Processor -- start 2024-08-19T01:41:17.326 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:17.325+0000 7fadcb8a8740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559b8d291b90 0x559b8d314f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:41:17.326 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:17.325+0000 7fadcb8a8740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559b8d2924b0 0x559b8d2a4df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:41:17.326 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:17.325+0000 7fadcb8a8740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559b8d29cd70 0x559b8d2a5330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:41:17.327 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:17.326+0000 7fadc1400640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559b8d2924b0 0x559b8d2a4df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:41:17.327 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:17.326+0000 7fadc1e00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559b8d291b90 0x559b8d314f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:41:17.327 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:17.326+0000 7fadc1400640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559b8d2924b0 0x559b8d2a4df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.83:44960/0 (socket says 172.21.15.83:44960) 2024-08-19T01:41:17.327 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:17.326+0000 7fadc1e00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559b8d291b90 0x559b8d314f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.83:41764/0 (socket says 172.21.15.83:41764) 2024-08-19T01:41:17.328 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:17.326+0000 7fadc1400640 1 -- 172.21.15.83:0/1966548321 learned_addr learned my addr 172.21.15.83:0/1966548321 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-08-19T01:41:17.328 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:17.326+0000 7fadcb8a8740 1 -- 172.21.15.83:0/1966548321 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fadc41ba770 con 0x559b8d29cd70 2024-08-19T01:41:17.329 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:17.326+0000 7fadcb8a8740 1 -- 172.21.15.83:0/1966548321 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fadc41bbbe0 con 0x559b8d291b90 2024-08-19T01:41:17.329 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:17.326+0000 7fadcb8a8740 1 -- 172.21.15.83:0/1966548321 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fadc41bae40 con 0x559b8d2924b0 2024-08-19T01:41:17.329 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:17.326+0000 7fadcb8a8740 10 client.0 _ll_register_callbacks cb 0x7fadbc0093a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-19T01:41:17.329 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:17.326+0000 7fadc2800640 1 --2- 172.21.15.83:0/1966548321 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559b8d29cd70 0x559b8d2a5330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:41:17.329 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:17.327+0000 7fadc1400640 1 -- 172.21.15.83:0/1966548321 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559b8d291b90 msgr2=0x559b8d314f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:41:17.329 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:17.327+0000 7fadc1400640 1 --2- 172.21.15.83:0/1966548321 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559b8d291b90 0x559b8d314f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:41:17.329 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:17.327+0000 7fadc1400640 1 -- 172.21.15.83:0/1966548321 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559b8d29cd70 msgr2=0x559b8d2a5330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:41:17.329 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:17.327+0000 7fadc1400640 1 --2- 172.21.15.83:0/1966548321 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559b8d29cd70 0x559b8d2a5330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:41:17.329 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:17.327+0000 7fadc1400640 1 -- 172.21.15.83:0/1966548321 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559b8d28df50 con 0x559b8d2924b0 2024-08-19T01:41:17.329 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:17.327+0000 7fadc1400640 1 --2- 172.21.15.83:0/1966548321 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559b8d2924b0 0x559b8d2a4df0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fadb409b090 tx=0x7fadb409f8c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1e8f4f91ab251734 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:41:17.330 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:17.327+0000 7fada3e00640 10 client.0 ms_handle_connect on v2:172.21.15.167:3300/0 2024-08-19T01:41:17.330 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:17.327+0000 7fadc2800640 1 --2- 172.21.15.83:0/1966548321 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559b8d29cd70 0x559b8d2a5330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-19T01:41:17.330 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:17.327+0000 7fada3e00640 1 -- 172.21.15.83:0/1966548321 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fadb40b3070 con 0x559b8d2924b0 2024-08-19T01:41:17.330 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:17.328+0000 7fada3e00640 1 -- 172.21.15.83:0/1966548321 <== mon.1 v2:172.21.15.167:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8269+0+0 (secure 0 0 0) 0x7fadb40b4260 con 0x559b8d2924b0 2024-08-19T01:41:17.331 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:17.328+0000 7fadcb8a8740 1 -- 172.21.15.83:0/1966548321 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fadb4010550 con 0x559b8d2924b0 2024-08-19T01:41:17.331 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:17.328+0000 7fada3e00640 1 -- 172.21.15.83:0/1966548321 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fadb409ba90 con 0x559b8d2924b0 2024-08-19T01:41:17.331 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:17.328+0000 7fadcb8a8740 10 client.25846 fetch_fsmap learned FSMap version 27 2024-08-19T01:41:17.331 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:17.328+0000 7fadcb8a8740 1 -- 172.21.15.83:0/1966548321 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x559b8d29e980 con 0x559b8d2924b0 2024-08-19T01:41:17.331 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:17.328+0000 7fada3e00640 1 -- 172.21.15.83:0/1966548321 <== mon.1 v2:172.21.15.167:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fadb4010550 con 0x559b8d2924b0 2024-08-19T01:41:17.331 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:17.329+0000 7fada3e00640 1 -- 172.21.15.83:0/1966548321 <== mon.1 v2:172.21.15.167:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7fadb409a7e0 con 0x559b8d2924b0 2024-08-19T01:41:17.332 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:17.329+0000 7fadcb8a8740 10 client.25846 fetch_fsmap finished waiting for FSMap version 27 2024-08-19T01:41:17.332 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:17.329+0000 7fadcb8a8740 10 client.25846 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-19T01:41:17.332 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:17.329+0000 7fadcb8a8740 20 client.25846 populate_metadata read hostname 'smithi083' 2024-08-19T01:41:17.332 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:17.329+0000 7fadcb8a8740 1 --2- 172.21.15.83:0/1966548321 >> [v2:172.21.15.117:6838/1967595704,v1:172.21.15.117:6839/1967595704] conn(0x559b8d328d50 0x559b8d32b170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:41:17.332 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:17.329+0000 7fadcb8a8740 4 client.25846 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-08-19T01:41:17.332 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:17.329+0000 7fadcb8a8740 1 -- 172.21.15.83:0/1966548321 --> [v2:172.21.15.117:6838/1967595704,v1:172.21.15.117:6839/1967595704] -- command(tid 0: {"prefix": "scrub status"}) -- 0x559b8d1ff670 con 0x559b8d328d50 2024-08-19T01:41:17.332 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:17.330+0000 7fadc1e00640 1 --2- 172.21.15.83:0/1966548321 >> [v2:172.21.15.117:6838/1967595704,v1:172.21.15.117:6839/1967595704] conn(0x559b8d328d50 0x559b8d32b170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:41:17.333 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:17.332+0000 7fadc1e00640 1 --2- 172.21.15.83:0/1966548321 >> [v2:172.21.15.117:6838/1967595704,v1:172.21.15.117:6839/1967595704] conn(0x559b8d328d50 0x559b8d32b170 crc :-1 s=READY pgs=114 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bea98535dc2df45f server_cookie=59a698ba2a50f7f8 in_seq=0 out_seq=0 2024-08-19T01:41:17.333 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:17.332+0000 7fada3e00640 10 client.25846 ms_handle_connect on v2:172.21.15.117:6838/1967595704 2024-08-19T01:41:17.335 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:17.334+0000 7fada3e00640 1 -- 172.21.15.83:0/1966548321 <== mds.0 v2:172.21.15.117:6838/1967595704 1 ==== command_reply(tid 0: 0 ) ==== 8+0+269 (crc 0 0 0) 0x559b8d1ff670 con 0x559b8d328d50 2024-08-19T01:41:17.335 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:17.334+0000 7fada3e00640 10 client.25846 handle_command_reply: tid=0 2024-08-19T01:41:17.335 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:17.334+0000 7fadcb8a8740 1 client.25846 shutdown 2024-08-19T01:41:17.335 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:17.334+0000 7fadcb8a8740 1 -- 172.21.15.83:0/1966548321 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559b8d2924b0 msgr2=0x559b8d2a4df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:41:17.336 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:17.334+0000 7fadcb8a8740 1 --2- 172.21.15.83:0/1966548321 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559b8d2924b0 0x559b8d2a4df0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fadb409b090 tx=0x7fadb409f8c0 comp rx=0 tx=0).stop 2024-08-19T01:41:17.336 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:17.334+0000 7fadcb8a8740 1 -- 172.21.15.83:0/1966548321 shutdown_connections 2024-08-19T01:41:17.336 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:17.334+0000 7fadcb8a8740 1 --2- 172.21.15.83:0/1966548321 >> [v2:172.21.15.117:6838/1967595704,v1:172.21.15.117:6839/1967595704] conn(0x559b8d328d50 0x559b8d32b170 crc :-1 s=READY pgs=114 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:41:17.337 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:17.334+0000 7fadcb8a8740 1 --2- 172.21.15.83:0/1966548321 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559b8d29cd70 0x559b8d2a5330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:41:17.337 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:17.334+0000 7fadcb8a8740 1 --2- 172.21.15.83:0/1966548321 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559b8d2924b0 0x559b8d2a4df0 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:41:17.337 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:17.334+0000 7fadcb8a8740 1 --2- 172.21.15.83:0/1966548321 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559b8d291b90 0x559b8d314f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:41:17.337 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:17.334+0000 7fada3e00640 0 client.25846 ms_handle_reset on v2:172.21.15.117:6838/1967595704 2024-08-19T01:41:17.337 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:17.334+0000 7fadcb8a8740 1 -- 172.21.15.83:0/1966548321 >> 172.21.15.83:0/1966548321 conn(0x559b8d28e120 msgr2=0x559b8d28d280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:41:17.337 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:17.335+0000 7fadcb8a8740 1 -- 172.21.15.83:0/1966548321 shutdown_connections 2024-08-19T01:41:17.337 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:17.335+0000 7fadcb8a8740 1 -- 172.21.15.83:0/1966548321 wait complete. 2024-08-19T01:41:17.338 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:17.337+0000 7fadcb8a8740 20 client.25846 trim_cache size 0 max 16384 2024-08-19T01:41:17.338 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-08-19T01:41:17.339 INFO:teuthology.orchestra.run.smithi083.stdout: "status": "scrub active (20515 inodes in the stack)", 2024-08-19T01:41:17.339 INFO:teuthology.orchestra.run.smithi083.stdout: "scrubs": { 2024-08-19T01:41:17.339 INFO:teuthology.orchestra.run.smithi083.stdout: "57f5c524-dcae-4283-b318-37fdde10e0c1": { 2024-08-19T01:41:17.339 INFO:teuthology.orchestra.run.smithi083.stdout: "path": "/", 2024-08-19T01:41:17.339 INFO:teuthology.orchestra.run.smithi083.stdout: "tag": "57f5c524-dcae-4283-b318-37fdde10e0c1", 2024-08-19T01:41:17.339 INFO:teuthology.orchestra.run.smithi083.stdout: "options": "recursive,force" 2024-08-19T01:41:17.339 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-08-19T01:41:17.339 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-08-19T01:41:17.339 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-08-19T01:41:17.341 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:17.338+0000 7fadc8e00640 1 -- 172.21.15.83:0/1875140968 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7fada80a2c40 msgr2=0x7fada80a5100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:41:17.341 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:17.338+0000 7fadc8e00640 1 --2- 172.21.15.83:0/1875140968 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7fada80a2c40 0x7fada80a5100 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7fadbc021000 tx=0x7fadbc01e000 comp rx=0 tx=0).stop 2024-08-19T01:41:17.341 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:17.338+0000 7fadc8e00640 1 -- 172.21.15.83:0/1875140968 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fadc4108ad0 msgr2=0x7fadc4082a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:41:17.341 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:17.338+0000 7fadc8e00640 1 --2- 172.21.15.83:0/1875140968 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fadc4108ad0 0x7fadc4082a00 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fadb400b470 tx=0x7fadb400b930 comp rx=0 tx=0).stop 2024-08-19T01:41:17.341 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:17.339+0000 7fadc8e00640 1 -- 172.21.15.83:0/1875140968 shutdown_connections 2024-08-19T01:41:17.341 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:17.339+0000 7fadc8e00640 1 --2- 172.21.15.83:0/1875140968 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7fada80a2c40 0x7fada80a5100 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:41:17.341 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:17.339+0000 7fadc8e00640 1 --2- 172.21.15.83:0/1875140968 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fadc410d250 0x7fadc407b730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:41:17.341 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:17.339+0000 7fadc8e00640 1 --2- 172.21.15.83:0/1875140968 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fadc4108ad0 0x7fadc4082a00 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:41:17.341 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:17.339+0000 7fadc8e00640 1 --2- 172.21.15.83:0/1875140968 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fadc4108120 0x7fadc40824c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:41:17.341 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:17.339+0000 7fadc8e00640 1 -- 172.21.15.83:0/1875140968 >> 172.21.15.83:0/1875140968 conn(0x7fadc4077820 msgr2=0x7fadc4078b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:41:17.341 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:17.339+0000 7fadc8e00640 1 -- 172.21.15.83:0/1875140968 shutdown_connections 2024-08-19T01:41:17.341 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:17.339+0000 7fadc8e00640 1 -- 172.21.15.83:0/1875140968 wait complete. 2024-08-19T01:41:18.040 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:41:17 smithi083 ceph-mon[17120]: scrub summary: active paths [/] 2024-08-19T01:41:18.202 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:41:17 smithi117 ceph-mon[22016]: scrub summary: active paths [/] 2024-08-19T01:41:18.215 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:41:17 smithi167 ceph-mon[21875]: scrub summary: active paths [/] 2024-08-19T01:41:18.695 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:41:18.693+0000 7fd595800640 1 -- 172.21.15.83:0/2248081186 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd590104ca0 msgr2=0x7fd590105120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:41:18.695 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:41:18.693+0000 7fd595800640 1 --2- 172.21.15.83:0/2248081186 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd590104ca0 0x7fd590105120 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fd57c009900 tx=0x7fd57c02a3f0 comp rx=0 tx=0).stop 2024-08-19T01:41:18.695 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:41:18.693+0000 7fd595800640 1 -- 172.21.15.83:0/2248081186 shutdown_connections 2024-08-19T01:41:18.695 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:41:18.693+0000 7fd595800640 1 --2- 172.21.15.83:0/2248081186 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd590105660 0x7fd59010bef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:41:18.695 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:41:18.693+0000 7fd595800640 1 --2- 172.21.15.83:0/2248081186 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd590104ca0 0x7fd590105120 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:41:18.695 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:41:18.693+0000 7fd595800640 1 --2- 172.21.15.83:0/2248081186 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd590103ab0 0x7fd590103eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:41:18.695 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:41:18.693+0000 7fd595800640 1 -- 172.21.15.83:0/2248081186 >> 172.21.15.83:0/2248081186 conn(0x7fd5900ff210 msgr2=0x7fd590101650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:41:18.695 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:41:18.693+0000 7fd595800640 1 -- 172.21.15.83:0/2248081186 shutdown_connections 2024-08-19T01:41:18.695 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:41:18.693+0000 7fd595800640 1 -- 172.21.15.83:0/2248081186 wait complete. 2024-08-19T01:41:18.696 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:41:18.694+0000 7fd595800640 1 Processor -- start 2024-08-19T01:41:18.696 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:41:18.694+0000 7fd595800640 1 -- start start 2024-08-19T01:41:18.696 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:41:18.695+0000 7fd595800640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd590103ab0 0x7fd590199d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:41:18.696 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:41:18.695+0000 7fd595800640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd590104ca0 0x7fd59019a290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:41:18.696 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:41:18.695+0000 7fd595800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd590105660 0x7fd5901a1300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:41:18.697 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:41:18.695+0000 7fd595800640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fd59010f760 con 0x7fd590105660 2024-08-19T01:41:18.697 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:41:18.695+0000 7fd595800640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fd59010f5e0 con 0x7fd590103ab0 2024-08-19T01:41:18.697 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:41:18.695+0000 7fd595800640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fd59010f8e0 con 0x7fd590104ca0 2024-08-19T01:41:18.697 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:41:18.695+0000 7fd58ea00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd590103ab0 0x7fd590199d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:41:18.697 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:41:18.695+0000 7fd58ea00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd590103ab0 0x7fd590199d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.83:41776/0 (socket says 172.21.15.83:41776) 2024-08-19T01:41:18.697 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:41:18.695+0000 7fd58ea00640 1 -- 172.21.15.83:0/3469948270 learned_addr learned my addr 172.21.15.83:0/3469948270 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-08-19T01:41:18.697 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:41:18.695+0000 7fd58f400640 1 --2- 172.21.15.83:0/3469948270 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd590105660 0x7fd5901a1300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:41:18.698 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:41:18.695+0000 7fd58ea00640 1 -- 172.21.15.83:0/3469948270 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd590104ca0 msgr2=0x7fd59019a290 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-08-19T01:41:18.698 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:41:18.695+0000 7fd58ea00640 1 --2- 172.21.15.83:0/3469948270 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd590104ca0 0x7fd59019a290 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:41:18.698 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:41:18.695+0000 7fd58ea00640 1 -- 172.21.15.83:0/3469948270 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd590105660 msgr2=0x7fd5901a1300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:41:18.698 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:41:18.695+0000 7fd58ea00640 1 --2- 172.21.15.83:0/3469948270 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd590105660 0x7fd5901a1300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:41:18.698 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:41:18.695+0000 7fd58ea00640 1 -- 172.21.15.83:0/3469948270 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd5901a1970 con 0x7fd590103ab0 2024-08-19T01:41:18.698 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:41:18.696+0000 7fd58ea00640 1 --2- 172.21.15.83:0/3469948270 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd590103ab0 0x7fd590199d50 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fd58000e9c0 tx=0x7fd58000ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:41:18.698 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:41:18.696+0000 7fd587400640 1 -- 172.21.15.83:0/3469948270 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd58000cde0 con 0x7fd590103ab0 2024-08-19T01:41:18.698 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:41:18.696+0000 7fd587400640 1 -- 172.21.15.83:0/3469948270 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd580010910 con 0x7fd590103ab0 2024-08-19T01:41:18.698 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:41:18.696+0000 7fd587400640 1 -- 172.21.15.83:0/3469948270 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd580018a80 con 0x7fd590103ab0 2024-08-19T01:41:18.699 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:41:18.696+0000 7fd595800640 1 -- 172.21.15.83:0/3469948270 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd5901a1c60 con 0x7fd590103ab0 2024-08-19T01:41:18.699 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:41:18.696+0000 7fd595800640 1 -- 172.21.15.83:0/3469948270 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd5901a21b0 con 0x7fd590103ab0 2024-08-19T01:41:18.701 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:41:18.700+0000 7fd587400640 1 -- 172.21.15.83:0/3469948270 <== mon.2 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7fd58001f020 con 0x7fd590103ab0 2024-08-19T01:41:18.702 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:41:18.701+0000 7fd598647740 1 Processor -- start 2024-08-19T01:41:18.702 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:41:18.701+0000 7fd598647740 1 -- start start 2024-08-19T01:41:18.703 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:41:18.701+0000 7fd598647740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55645c86d470 0x55645c86d870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:41:18.703 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:41:18.701+0000 7fd587400640 1 --2- 172.21.15.83:0/3469948270 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7fd5740a2d30 0x7fd5740a51f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:41:18.703 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:41:18.701+0000 7fd598647740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55645c86ddb0 0x55645c872870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:41:18.703 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:41:18.701+0000 7fd598647740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55645c872e40 0x55645c875300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:41:18.703 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:41:18.701+0000 7fd598647740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55645c7d1890 con 0x55645c86ddb0 2024-08-19T01:41:18.703 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:41:18.701+0000 7fd598647740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55645c769d90 con 0x55645c86d470 2024-08-19T01:41:18.703 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:41:18.701+0000 7fd598647740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55645c7cd850 con 0x55645c872e40 2024-08-19T01:41:18.703 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:41:18.701+0000 7fd587400640 1 -- 172.21.15.83:0/3469948270 <== mon.2 v2:172.21.15.117:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8269+0+0 (secure 0 0 0) 0x7fd5800cf3c0 con 0x7fd590103ab0 2024-08-19T01:41:18.704 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:41:18.702+0000 7fd58f400640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55645c872e40 0x55645c875300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:41:18.704 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:41:18.702+0000 7fd58f400640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55645c86d470 0x55645c86d870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:41:18.704 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:41:18.702+0000 7fd58ea00640 1 --2- 172.21.15.83:0/3469948270 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7fd5740a2d30 0x7fd5740a51f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:41:18.704 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:41:18.702+0000 7fd58f400640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55645c86d470 0x55645c86d870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.83:41786/0 (socket says 172.21.15.83:41786) 2024-08-19T01:41:18.705 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:41:18.702+0000 7fd58f400640 1 -- 172.21.15.83:0/1965966198 learned_addr learned my addr 172.21.15.83:0/1965966198 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-08-19T01:41:18.705 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:41:18.702+0000 7fd58e000640 1 --2- 172.21.15.83:0/1965966198 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55645c86ddb0 0x55645c872870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:41:18.705 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:41:18.702+0000 7fd58f400640 1 -- 172.21.15.83:0/1965966198 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55645c872e40 msgr2=0x55645c875300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:41:18.705 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:41:18.702+0000 7fd58f400640 1 --2- 172.21.15.83:0/1965966198 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55645c872e40 0x55645c875300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:41:18.705 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:41:18.702+0000 7fd58f400640 1 -- 172.21.15.83:0/1965966198 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55645c86ddb0 msgr2=0x55645c872870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:41:18.705 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:41:18.702+0000 7fd58f400640 1 --2- 172.21.15.83:0/1965966198 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55645c86ddb0 0x55645c872870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:41:18.705 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:41:18.702+0000 7fd58f400640 1 -- 172.21.15.83:0/1965966198 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55645c4e5210 con 0x55645c86d470 2024-08-19T01:41:18.705 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:41:18.703+0000 7fd58ea00640 1 --2- 172.21.15.83:0/3469948270 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7fd5740a2d30 0x7fd5740a51f0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7fd58009bb30 tx=0x7fd58000c850 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:41:18.705 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:41:18.703+0000 7fd58e000640 1 --2- 172.21.15.83:0/1965966198 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55645c86ddb0 0x55645c872870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-19T01:41:18.705 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:41:18.703+0000 7fd58f400640 1 --2- 172.21.15.83:0/1965966198 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55645c86d470 0x55645c86d870 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fd588009fd0 tx=0x7fd588005bd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b8bf3dd9c3f139b6 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:41:18.706 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:41:18.703+0000 7fd57be00640 1 -- 172.21.15.83:0/1965966198 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd588004d20 con 0x55645c86d470 2024-08-19T01:41:18.706 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:41:18.704+0000 7fd57be00640 1 -- 172.21.15.83:0/1965966198 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd588005500 con 0x55645c86d470 2024-08-19T01:41:18.707 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:41:18.704+0000 7fd57be00640 1 -- 172.21.15.83:0/1965966198 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd58800f810 con 0x55645c86d470 2024-08-19T01:41:18.707 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:41:18.704+0000 7fd598647740 1 -- 172.21.15.83:0/1965966198 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55645c86d470 msgr2=0x55645c86d870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:41:18.707 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:41:18.704+0000 7fd598647740 1 --2- 172.21.15.83:0/1965966198 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55645c86d470 0x55645c86d870 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fd588009fd0 tx=0x7fd588005bd0 comp rx=0 tx=0).stop 2024-08-19T01:41:18.707 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:41:18.705+0000 7fd598647740 1 -- 172.21.15.83:0/1965966198 shutdown_connections 2024-08-19T01:41:18.707 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:41:18.705+0000 7fd598647740 1 --2- 172.21.15.83:0/1965966198 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55645c872e40 0x55645c875300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:41:18.707 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:41:18.705+0000 7fd598647740 1 --2- 172.21.15.83:0/1965966198 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55645c86ddb0 0x55645c872870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:41:18.707 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:41:18.705+0000 7fd598647740 1 --2- 172.21.15.83:0/1965966198 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55645c86d470 0x55645c86d870 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:41:18.707 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:41:18.705+0000 7fd598647740 1 -- 172.21.15.83:0/1965966198 >> 172.21.15.83:0/1965966198 conn(0x55645c868ae0 msgr2=0x55645c868ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:41:18.707 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:41:18.705+0000 7fd598647740 1 -- 172.21.15.83:0/1965966198 shutdown_connections 2024-08-19T01:41:18.707 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:41:18.705+0000 7fd598647740 1 -- 172.21.15.83:0/1965966198 wait complete. 2024-08-19T01:41:18.708 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:41:18.706+0000 7fd598647740 1 -- start start 2024-08-19T01:41:18.708 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:41:18.706+0000 7fd598647740 1 Processor -- start 2024-08-19T01:41:18.708 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:41:18.706+0000 7fd598647740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55645c86d470 0x55645c877760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:41:18.708 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:41:18.707+0000 7fd598647740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55645c86ddb0 0x55645c877ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:41:18.708 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:41:18.707+0000 7fd598647740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55645c872e40 0x55645c87ece0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:41:18.709 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:41:18.707+0000 7fd598647740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55645c875e50 con 0x55645c86ddb0 2024-08-19T01:41:18.709 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:41:18.707+0000 7fd598647740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55645c78cf70 con 0x55645c86d470 2024-08-19T01:41:18.709 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:41:18.707+0000 7fd598647740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55645c875fd0 con 0x55645c872e40 2024-08-19T01:41:18.709 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:41:18.707+0000 7fd598647740 10 client.0 _ll_register_callbacks cb 0x55645c7c98d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-19T01:41:18.709 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:41:18.707+0000 7fd58f400640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55645c86d470 0x55645c877760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:41:18.709 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:41:18.707+0000 7fd58f400640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55645c86d470 0x55645c877760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.83:41800/0 (socket says 172.21.15.83:41800) 2024-08-19T01:41:18.709 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:41:18.707+0000 7fd58f400640 1 -- 172.21.15.83:0/4049853478 learned_addr learned my addr 172.21.15.83:0/4049853478 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-08-19T01:41:18.710 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:41:18.707+0000 7fd58e000640 1 --2- 172.21.15.83:0/4049853478 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55645c86ddb0 0x55645c877ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:41:18.710 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:41:18.707+0000 7fd58f400640 1 --2- 172.21.15.83:0/4049853478 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55645c872e40 0x55645c87ece0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:41:18.710 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:41:18.708+0000 7fd58f400640 1 -- 172.21.15.83:0/4049853478 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55645c872e40 msgr2=0x55645c87ece0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:41:18.710 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:41:18.708+0000 7fd58f400640 1 --2- 172.21.15.83:0/4049853478 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55645c872e40 0x55645c87ece0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:41:18.710 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:41:18.708+0000 7fd58f400640 1 -- 172.21.15.83:0/4049853478 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55645c86ddb0 msgr2=0x55645c877ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:41:18.710 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:41:18.708+0000 7fd58f400640 1 --2- 172.21.15.83:0/4049853478 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55645c86ddb0 0x55645c877ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:41:18.710 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:41:18.708+0000 7fd58f400640 1 -- 172.21.15.83:0/4049853478 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55645c5b1620 con 0x55645c86d470 2024-08-19T01:41:18.711 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:41:18.709+0000 7fd58f400640 1 --2- 172.21.15.83:0/4049853478 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55645c86d470 0x55645c877760 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fd588005cc0 tx=0x7fd588017070 comp rx=0 tx=0).ready entity=mon.2 client_cookie=54365e9a2a21837e server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:41:18.711 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:41:18.709+0000 7fd578c00640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-08-19T01:41:18.711 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:41:18.709+0000 7fd578c00640 1 -- 172.21.15.83:0/4049853478 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd588033060 con 0x55645c86d470 2024-08-19T01:41:18.711 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:41:18.709+0000 7fd598647740 1 -- 172.21.15.83:0/4049853478 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd588005500 con 0x55645c86d470 2024-08-19T01:41:18.711 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:41:18.709+0000 7fd578c00640 1 -- 172.21.15.83:0/4049853478 <== mon.2 v2:172.21.15.117:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8269+0+0 (secure 0 0 0) 0x7fd5880049e0 con 0x55645c86d470 2024-08-19T01:41:18.711 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:41:18.709+0000 7fd578c00640 1 -- 172.21.15.83:0/4049853478 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fd588004be0 con 0x55645c86d470 2024-08-19T01:41:18.711 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:41:18.709+0000 7fd598647740 10 client.25922 fetch_fsmap learned FSMap version 27 2024-08-19T01:41:18.712 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:41:18.709+0000 7fd598647740 1 -- 172.21.15.83:0/4049853478 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55645c881e60 con 0x55645c86d470 2024-08-19T01:41:18.712 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:41:18.710+0000 7fd578c00640 1 -- 172.21.15.83:0/4049853478 <== mon.2 v2:172.21.15.117:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd588005500 con 0x55645c86d470 2024-08-19T01:41:18.712 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:41:18.710+0000 7fd578c00640 1 -- 172.21.15.83:0/4049853478 <== mon.2 v2:172.21.15.117:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7fd58800c070 con 0x55645c86d470 2024-08-19T01:41:18.712 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:41:18.710+0000 7fd598647740 10 client.25922 fetch_fsmap finished waiting for FSMap version 27 2024-08-19T01:41:18.712 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:41:18.710+0000 7fd598647740 10 client.25922 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-08-19T01:41:18.712 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:41:18.710+0000 7fd598647740 20 client.25922 populate_metadata read hostname 'smithi083' 2024-08-19T01:41:18.713 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:41:18.710+0000 7fd598647740 1 --2- 172.21.15.83:0/4049853478 >> [v2:172.21.15.117:6838/1967595704,v1:172.21.15.117:6839/1967595704] conn(0x55645c883270 0x55645c885660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:41:18.713 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:41:18.710+0000 7fd598647740 4 client.25922 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-19T01:41:18.713 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:41:18.710+0000 7fd598647740 1 -- 172.21.15.83:0/4049853478 --> [v2:172.21.15.117:6838/1967595704,v1:172.21.15.117:6839/1967595704] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55645c7defc0 con 0x55645c883270 2024-08-19T01:41:18.713 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:41:18.711+0000 7fd58e000640 1 --2- 172.21.15.83:0/4049853478 >> [v2:172.21.15.117:6838/1967595704,v1:172.21.15.117:6839/1967595704] conn(0x55645c883270 0x55645c885660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:41:18.713 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:41:18.712+0000 7fd58e000640 1 --2- 172.21.15.83:0/4049853478 >> [v2:172.21.15.117:6838/1967595704,v1:172.21.15.117:6839/1967595704] conn(0x55645c883270 0x55645c885660 crc :-1 s=READY pgs=115 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=75383a58281da5eb server_cookie=8f53359002788a2f in_seq=0 out_seq=0 2024-08-19T01:41:18.713 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:41:18.712+0000 7fd578c00640 10 client.25922 ms_handle_connect on v2:172.21.15.117:6838/1967595704 2024-08-19T01:41:18.716 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:41:18.715+0000 7fd578c00640 1 -- 172.21.15.83:0/4049853478 <== mds.0 v2:172.21.15.117:6838/1967595704 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55645c7defc0 con 0x55645c883270 2024-08-19T01:41:18.716 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:41:18.715+0000 7fd578c00640 10 client.25922 handle_command_reply: tid=0 2024-08-19T01:41:18.716 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:41:18.715+0000 7fd598647740 1 client.25922 shutdown 2024-08-19T01:41:18.717 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:41:18.715+0000 7fd598647740 1 -- 172.21.15.83:0/4049853478 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55645c86d470 msgr2=0x55645c877760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:41:18.717 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:41:18.715+0000 7fd598647740 1 --2- 172.21.15.83:0/4049853478 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55645c86d470 0x55645c877760 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fd588005cc0 tx=0x7fd588017070 comp rx=0 tx=0).stop 2024-08-19T01:41:18.717 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:41:18.715+0000 7fd598647740 1 -- 172.21.15.83:0/4049853478 shutdown_connections 2024-08-19T01:41:18.717 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:41:18.715+0000 7fd598647740 1 --2- 172.21.15.83:0/4049853478 >> [v2:172.21.15.117:6838/1967595704,v1:172.21.15.117:6839/1967595704] conn(0x55645c883270 0x55645c885660 crc :-1 s=READY pgs=115 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:41:18.717 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:41:18.716+0000 7fd598647740 1 --2- 172.21.15.83:0/4049853478 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55645c872e40 0x55645c87ece0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:41:18.718 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:41:18.716+0000 7fd598647740 1 --2- 172.21.15.83:0/4049853478 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55645c86ddb0 0x55645c877ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:41:18.718 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:41:18.716+0000 7fd598647740 1 --2- 172.21.15.83:0/4049853478 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55645c86d470 0x55645c877760 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:41:18.718 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:41:18.716+0000 7fd598647740 1 -- 172.21.15.83:0/4049853478 >> 172.21.15.83:0/4049853478 conn(0x55645c868ae0 msgr2=0x55645c86a2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:41:18.718 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:41:18.716+0000 7fd578c00640 0 client.25922 ms_handle_reset on v2:172.21.15.117:6838/1967595704 2024-08-19T01:41:18.718 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:41:18.716+0000 7fd598647740 1 -- 172.21.15.83:0/4049853478 shutdown_connections 2024-08-19T01:41:18.718 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:41:18.716+0000 7fd598647740 1 -- 172.21.15.83:0/4049853478 wait complete. 2024-08-19T01:41:18.718 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:41:18.716+0000 7fd598647740 20 client.25922 trim_cache size 0 max 16384 2024-08-19T01:41:18.719 INFO:tasks.cephfs.filesystem:scrub status for tag:57f5c524-dcae-4283-b318-37fdde10e0c1 - {'path': '/', 'tag': '57f5c524-dcae-4283-b318-37fdde10e0c1', 'options': 'recursive,force'} 2024-08-19T01:41:18.738 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:41:18.736+0000 7fd598647740 1 Processor -- start 2024-08-19T01:41:18.738 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:41:18.736+0000 7fd598647740 1 -- start start 2024-08-19T01:41:18.738 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:41:18.736+0000 7fd598647740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55645c868ae0 0x55645c877d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:41:18.738 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:41:18.736+0000 7fd598647740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55645c86d470 0x55645c87ec00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:41:18.738 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:41:18.736+0000 7fd598647740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55645c86ddb0 0x55645c867be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:41:18.738 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:41:18.736+0000 7fd598647740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fd5901a94f0 con 0x55645c868ae0 2024-08-19T01:41:18.739 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:41:18.736+0000 7fd598647740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fd5901a6dc0 con 0x55645c86d470 2024-08-19T01:41:18.739 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:41:18.736+0000 7fd598647740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fd5901a4c10 con 0x55645c86ddb0 2024-08-19T01:41:18.739 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:41:18.736+0000 7fd58e000640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55645c86d470 0x55645c87ec00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:41:18.739 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:41:18.736+0000 7fd58f400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55645c868ae0 0x55645c877d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:41:18.739 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:41:18.736+0000 7fd58e000640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55645c86d470 0x55645c87ec00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.83:41806/0 (socket says 172.21.15.83:41806) 2024-08-19T01:41:18.739 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:41:18.736+0000 7fd58e000640 1 -- 172.21.15.83:0/4024061034 learned_addr learned my addr 172.21.15.83:0/4024061034 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-08-19T01:41:18.739 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:41:18.736+0000 7fd58f400640 1 --2- 172.21.15.83:0/4024061034 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55645c86ddb0 0x55645c867be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:41:18.739 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:41:18.737+0000 7fd58e000640 1 -- 172.21.15.83:0/4024061034 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55645c86ddb0 msgr2=0x55645c867be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:41:18.739 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:41:18.737+0000 7fd58e000640 1 --2- 172.21.15.83:0/4024061034 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55645c86ddb0 0x55645c867be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:41:18.739 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:41:18.737+0000 7fd58e000640 1 -- 172.21.15.83:0/4024061034 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55645c868ae0 msgr2=0x55645c877d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:41:18.739 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:41:18.737+0000 7fd58e000640 1 --2- 172.21.15.83:0/4024061034 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55645c868ae0 0x55645c877d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:41:18.739 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:41:18.737+0000 7fd58e000640 1 -- 172.21.15.83:0/4024061034 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55645c86e290 con 0x55645c86d470 2024-08-19T01:41:18.739 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:41:18.737+0000 7fd58e000640 1 --2- 172.21.15.83:0/4024061034 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55645c86d470 0x55645c87ec00 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fd57c006010 tx=0x7fd57c031400 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d3683b6ef28e646d server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:41:18.743 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:41:18.737+0000 7fd57b400640 1 -- 172.21.15.83:0/4024061034 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd57c049070 con 0x55645c86d470 2024-08-19T01:41:18.743 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:41:18.737+0000 7fd57b400640 1 -- 172.21.15.83:0/4024061034 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd57c005520 con 0x55645c86d470 2024-08-19T01:41:18.743 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:41:18.737+0000 7fd57b400640 1 -- 172.21.15.83:0/4024061034 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd57c035040 con 0x55645c86d470 2024-08-19T01:41:18.743 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:41:18.738+0000 7fd598647740 1 -- 172.21.15.83:0/4024061034 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55645c86d470 msgr2=0x55645c87ec00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:41:18.743 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:41:18.738+0000 7fd598647740 1 --2- 172.21.15.83:0/4024061034 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55645c86d470 0x55645c87ec00 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fd57c006010 tx=0x7fd57c031400 comp rx=0 tx=0).stop 2024-08-19T01:41:18.743 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:41:18.738+0000 7fd598647740 1 -- 172.21.15.83:0/4024061034 shutdown_connections 2024-08-19T01:41:18.743 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:41:18.738+0000 7fd598647740 1 --2- 172.21.15.83:0/4024061034 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55645c86ddb0 0x55645c867be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:41:18.743 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:41:18.738+0000 7fd598647740 1 --2- 172.21.15.83:0/4024061034 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55645c86d470 0x55645c87ec00 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:41:18.744 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:41:18.738+0000 7fd598647740 1 --2- 172.21.15.83:0/4024061034 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55645c868ae0 0x55645c877d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:41:18.744 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:41:18.738+0000 7fd598647740 1 -- 172.21.15.83:0/4024061034 >> 172.21.15.83:0/4024061034 conn(0x55645c872990 msgr2=0x55645c8674d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:41:18.744 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:41:18.738+0000 7fd598647740 1 -- 172.21.15.83:0/4024061034 shutdown_connections 2024-08-19T01:41:18.744 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:41:18.738+0000 7fd598647740 1 -- 172.21.15.83:0/4024061034 wait complete. 2024-08-19T01:41:18.744 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:41:18.738+0000 7fd598647740 1 -- start start 2024-08-19T01:41:18.744 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:41:18.739+0000 7fd598647740 1 Processor -- start 2024-08-19T01:41:18.744 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:41:18.739+0000 7fd598647740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55645c868ae0 0x55645c87c040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:41:18.744 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:41:18.739+0000 7fd598647740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55645c86d470 0x55645c87c580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:41:18.744 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:41:18.739+0000 7fd598647740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55645c86ddb0 0x55645c87d520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:41:18.744 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:41:18.739+0000 7fd598647740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fd5901a5280 con 0x55645c86ddb0 2024-08-19T01:41:18.744 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:41:18.739+0000 7fd598647740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fd5901a66f0 con 0x55645c86d470 2024-08-19T01:41:18.744 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:41:18.739+0000 7fd598647740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fd5901a5950 con 0x55645c868ae0 2024-08-19T01:41:18.744 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:41:18.739+0000 7fd598647740 10 client.0 _ll_register_callbacks cb 0x7fd58801f9d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-19T01:41:18.745 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:41:18.739+0000 7fd58e000640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55645c86d470 0x55645c87c580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:41:18.745 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:41:18.739+0000 7fd58e000640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55645c86d470 0x55645c87c580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.83:41812/0 (socket says 172.21.15.83:41812) 2024-08-19T01:41:18.745 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:41:18.739+0000 7fd58e000640 1 -- 172.21.15.83:0/1801333557 learned_addr learned my addr 172.21.15.83:0/1801333557 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-08-19T01:41:18.745 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:41:18.739+0000 7fd58f400640 1 --2- 172.21.15.83:0/1801333557 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55645c868ae0 0x55645c87c040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:41:18.745 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:41:18.739+0000 7fd58f400640 1 --2- 172.21.15.83:0/1801333557 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55645c86ddb0 0x55645c87d520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:41:18.745 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:41:18.740+0000 7fd58e000640 1 -- 172.21.15.83:0/1801333557 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55645c868ae0 msgr2=0x55645c87c040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:41:18.745 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:41:18.740+0000 7fd58e000640 1 --2- 172.21.15.83:0/1801333557 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55645c868ae0 0x55645c87c040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:41:18.745 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:41:18.740+0000 7fd58e000640 1 -- 172.21.15.83:0/1801333557 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55645c86ddb0 msgr2=0x55645c87d520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:41:18.745 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:41:18.740+0000 7fd58e000640 1 --2- 172.21.15.83:0/1801333557 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55645c86ddb0 0x55645c87d520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:41:18.745 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:41:18.740+0000 7fd58e000640 1 -- 172.21.15.83:0/1801333557 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55645c87db30 con 0x55645c86d470 2024-08-19T01:41:18.745 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:41:18.740+0000 7fd58f400640 1 --2- 172.21.15.83:0/1801333557 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55645c86ddb0 0x55645c87d520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-19T01:41:18.745 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:41:18.740+0000 7fd58e000640 1 --2- 172.21.15.83:0/1801333557 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55645c86d470 0x55645c87c580 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fd57c02a430 tx=0x7fd57c0056d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a1ca620154a98a6f server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:41:18.745 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:41:18.740+0000 7fd578c00640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-08-19T01:41:18.746 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:41:18.740+0000 7fd578c00640 1 -- 172.21.15.83:0/1801333557 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd57c052060 con 0x55645c86d470 2024-08-19T01:41:18.746 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:41:18.740+0000 7fd598647740 1 -- 172.21.15.83:0/1801333557 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd57c005520 con 0x55645c86d470 2024-08-19T01:41:18.746 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:41:18.740+0000 7fd578c00640 1 -- 172.21.15.83:0/1801333557 <== mon.2 v2:172.21.15.117:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8269+0+0 (secure 0 0 0) 0x7fd57c0046c0 con 0x55645c86d470 2024-08-19T01:41:18.746 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:41:18.740+0000 7fd578c00640 1 -- 172.21.15.83:0/1801333557 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fd57c049040 con 0x55645c86d470 2024-08-19T01:41:18.746 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:41:18.740+0000 7fd598647740 10 client.25928 fetch_fsmap learned FSMap version 27 2024-08-19T01:41:18.746 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:41:18.740+0000 7fd598647740 1 -- 172.21.15.83:0/1801333557 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55645c87ee40 con 0x55645c86d470 2024-08-19T01:41:18.746 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:41:18.741+0000 7fd578c00640 1 -- 172.21.15.83:0/1801333557 <== mon.2 v2:172.21.15.117:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd57c005520 con 0x55645c86d470 2024-08-19T01:41:18.746 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:41:18.741+0000 7fd578c00640 1 -- 172.21.15.83:0/1801333557 <== mon.2 v2:172.21.15.117:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7fd57c03a040 con 0x55645c86d470 2024-08-19T01:41:18.746 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:41:18.741+0000 7fd598647740 10 client.25928 fetch_fsmap finished waiting for FSMap version 27 2024-08-19T01:41:18.746 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:41:18.741+0000 7fd598647740 10 client.25928 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-08-19T01:41:18.746 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:41:18.741+0000 7fd598647740 20 client.25928 populate_metadata read hostname 'smithi083' 2024-08-19T01:41:18.746 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:41:18.741+0000 7fd598647740 1 --2- 172.21.15.83:0/1801333557 >> [v2:172.21.15.117:6838/1967595704,v1:172.21.15.117:6839/1967595704] conn(0x55645c872e40 0x55645c8f04c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:41:18.747 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:41:18.741+0000 7fd598647740 4 client.25928 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-08-19T01:41:18.747 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:41:18.741+0000 7fd598647740 1 -- 172.21.15.83:0/1801333557 --> [v2:172.21.15.117:6838/1967595704,v1:172.21.15.117:6839/1967595704] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55645c7d9310 con 0x55645c872e40 2024-08-19T01:41:18.747 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:41:18.741+0000 7fd58f400640 1 --2- 172.21.15.83:0/1801333557 >> [v2:172.21.15.117:6838/1967595704,v1:172.21.15.117:6839/1967595704] conn(0x55645c872e40 0x55645c8f04c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:41:18.747 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:41:18.742+0000 7fd578c00640 10 client.25928 ms_handle_connect on v2:172.21.15.117:6838/1967595704 2024-08-19T01:41:18.747 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:41:18.742+0000 7fd58f400640 1 --2- 172.21.15.83:0/1801333557 >> [v2:172.21.15.117:6838/1967595704,v1:172.21.15.117:6839/1967595704] conn(0x55645c872e40 0x55645c8f04c0 crc :-1 s=READY pgs=116 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6dd81eef584f38a6 server_cookie=ae4416b3374ee8af in_seq=0 out_seq=0 2024-08-19T01:41:19.103 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:41:18 smithi117 ceph-mon[22016]: scrub summary: active paths [/] 2024-08-19T01:41:19.103 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:41:18 smithi117 ceph-mon[22016]: scrub summary: idle+waiting paths [/] 2024-08-19T01:41:19.103 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:41:18 smithi117 ceph-mon[22016]: scrub summary: idle+waiting paths [/] 2024-08-19T01:41:19.104 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:41:18 smithi117 ceph-mon[22016]: scrub summary: active paths [/] 2024-08-19T01:41:19.104 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:41:18 smithi117 ceph-mon[22016]: scrub summary: idle+waiting paths [/] 2024-08-19T01:41:19.104 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:41:18 smithi117 ceph-mon[22016]: pgmap v533: 97 pgs: 97 active+clean; 671 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 7.9 MiB/s wr, 733 op/s 2024-08-19T01:41:19.104 INFO:journalctl@ceph.mds.b.smithi117.stdout:Aug 19 01:41:18 smithi117 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-mds-b[51905]: 2024-08-19T01:41:18.818+0000 7f79e46bd640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000263c [...2,head] /volumes/qa/sv_1/05b773d9-8346-4397-99ec-d49959a55703/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-42/ auth{2=1} fragtree_t(*^3) v244436 ap=2 RANDEPHEMERALPIN f(v17 m2024-08-19T01:40:36.257782+0000 222=222+0) n(v21 rc2024-08-19T01:40:36.257782+0000 b4293664 223=222+1) (inest sync->mix g=2) caps={25052=pAsLsXsFs/pAsLsXsFsxcral@260} mcw={2=AsLsXsFs} | dirtyscattered=0 request=0 lock=0 dirfrag=8 caps=1 dirtyrstat=0 dirtyparent=0 replicated=1 dirty=0 waiter=0 authpin=1 scrubqueue=0 randepin 0x55b59696b700]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value":"(2)0x1000000263c:[<0x10000000399/blog-42 v74084>,<0x10000000208/blogtest_in v446>,<0x10000000201/src v1710>,<0x10000000006/blogbench-1.0 v1485>,<0x10000000005/tmp v127>,<0x100000001fb/client.0 v113>,<0x100000001fa/05b773d9-8346-4397-99ec-d49959a55703 v116>,<0x10000000001/sv_1 v127>,<0x10000000000/qa v120>,<0x1/volumes v116>]//[]","memoryvalue":"(2)0x1000000263c:[<0x10000000399/blog-42 v244436>,<0x10000000208/blogtest_in v2010>,<0x10000000201/src v2236>,<0x10000000006/blogbench-1.0 v1925>,<0x10000000005/tmp v557>,<0x100000001fb/client.0 v527>,<0x100000001fa/05b773d9-8346-4397-99ec-d49959a55703 v538>,<0x10000000001/sv_1 v538>,<0x10000000000/qa v535>,<0x1/volumes v525>]//[]","error_str":""},"raw_stats":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2024-08-19T01:40:18.306206+0000 221=221+0)","ondisk_value.rstat":"n(v0 rc2024-08-19T01:40:36.257782+0000 b4293664 223=222+1)","memory_value.dirstat":"f(v17 m2024-08-19T01:40:36.257782+0000 222=222+0)","memory_value.rstat":"n(v21 rc2024-08-19T01:40:36.257782+0000 b4293664 223=222+1)","error_str":"freshly-calculated rstats don't match existing ones"},"return_code":0} 2024-08-19T01:41:19.155 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:41:18 smithi083 ceph-mon[17120]: scrub summary: active paths [/] 2024-08-19T01:41:19.155 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:41:18 smithi083 ceph-mon[17120]: scrub summary: idle+waiting paths [/] 2024-08-19T01:41:19.155 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:41:18 smithi083 ceph-mon[17120]: scrub summary: idle+waiting paths [/] 2024-08-19T01:41:19.155 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:41:18 smithi083 ceph-mon[17120]: scrub summary: active paths [/] 2024-08-19T01:41:19.155 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:41:18 smithi083 ceph-mon[17120]: scrub summary: idle+waiting paths [/] 2024-08-19T01:41:19.155 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:41:18 smithi083 ceph-mon[17120]: pgmap v533: 97 pgs: 97 active+clean; 671 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 7.9 MiB/s wr, 733 op/s 2024-08-19T01:41:19.216 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:41:18 smithi167 ceph-mon[21875]: scrub summary: active paths [/] 2024-08-19T01:41:19.216 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:41:18 smithi167 ceph-mon[21875]: scrub summary: idle+waiting paths [/] 2024-08-19T01:41:19.216 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:41:18 smithi167 ceph-mon[21875]: scrub summary: idle+waiting paths [/] 2024-08-19T01:41:19.216 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:41:18 smithi167 ceph-mon[21875]: scrub summary: active paths [/] 2024-08-19T01:41:19.216 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:41:18 smithi167 ceph-mon[21875]: scrub summary: idle+waiting paths [/] 2024-08-19T01:41:19.216 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:41:18 smithi167 ceph-mon[21875]: pgmap v533: 97 pgs: 97 active+clean; 671 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 7.9 MiB/s wr, 733 op/s 2024-08-19T01:41:20.216 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:41:19 smithi167 ceph-mon[21875]: scrub summary: active paths [/] 2024-08-19T01:41:20.216 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:41:19 smithi167 ceph-mon[21875]: scrub summary: idle+waiting paths [/] 2024-08-19T01:41:20.216 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:41:19 smithi167 ceph-mon[21875]: Scrub error on inode 0x1000000263c (/volumes/qa/sv_1/05b773d9-8346-4397-99ec-d49959a55703/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-42) see mds.b log and `damage ls` output for details 2024-08-19T01:41:20.273 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:41:19 smithi083 ceph-mon[17120]: scrub summary: active paths [/] 2024-08-19T01:41:20.273 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:41:19 smithi083 ceph-mon[17120]: scrub summary: idle+waiting paths [/] 2024-08-19T01:41:20.273 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:41:19 smithi083 ceph-mon[17120]: Scrub error on inode 0x1000000263c (/volumes/qa/sv_1/05b773d9-8346-4397-99ec-d49959a55703/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-42) see mds.b log and `damage ls` output for details 2024-08-19T01:41:20.454 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:41:19 smithi117 ceph-mon[22016]: scrub summary: active paths [/] 2024-08-19T01:41:20.454 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:41:19 smithi117 ceph-mon[22016]: scrub summary: idle+waiting paths [/] 2024-08-19T01:41:20.454 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:41:19 smithi117 ceph-mon[22016]: Scrub error on inode 0x1000000263c (/volumes/qa/sv_1/05b773d9-8346-4397-99ec-d49959a55703/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-42) see mds.b log and `damage ls` output for details 2024-08-19T01:41:21.128 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:41:21.127+0000 7fd578c00640 1 -- 172.21.15.83:0/1801333557 <== mds.0 v2:172.21.15.117:6838/1967595704 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55645c7d9310 con 0x55645c872e40 2024-08-19T01:41:21.128 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:41:21.127+0000 7fd578c00640 10 client.25928 handle_command_reply: tid=0 2024-08-19T01:41:21.129 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:41:21.127+0000 7fd598647740 1 client.25928 shutdown 2024-08-19T01:41:21.129 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:41:21.127+0000 7fd598647740 1 -- 172.21.15.83:0/1801333557 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55645c86d470 msgr2=0x55645c87c580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:41:21.130 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:41:21.127+0000 7fd598647740 1 --2- 172.21.15.83:0/1801333557 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55645c86d470 0x55645c87c580 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fd57c02a430 tx=0x7fd57c0056d0 comp rx=0 tx=0).stop 2024-08-19T01:41:21.130 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:41:21.127+0000 7fd598647740 1 -- 172.21.15.83:0/1801333557 shutdown_connections 2024-08-19T01:41:21.130 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:41:21.127+0000 7fd598647740 1 --2- 172.21.15.83:0/1801333557 >> [v2:172.21.15.117:6838/1967595704,v1:172.21.15.117:6839/1967595704] conn(0x55645c872e40 0x55645c8f04c0 crc :-1 s=READY pgs=116 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:41:21.130 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:41:21.127+0000 7fd598647740 1 --2- 172.21.15.83:0/1801333557 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55645c86ddb0 0x55645c87d520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:41:21.130 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:41:21.127+0000 7fd598647740 1 --2- 172.21.15.83:0/1801333557 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55645c86d470 0x55645c87c580 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:41:21.130 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:41:21.127+0000 7fd598647740 1 --2- 172.21.15.83:0/1801333557 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55645c868ae0 0x55645c87c040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:41:21.130 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:41:21.127+0000 7fd578c00640 0 client.25928 ms_handle_reset on v2:172.21.15.117:6838/1967595704 2024-08-19T01:41:21.130 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:41:21.127+0000 7fd598647740 1 -- 172.21.15.83:0/1801333557 >> 172.21.15.83:0/1801333557 conn(0x55645c872990 msgr2=0x55645c867cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:41:21.130 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:41:21.127+0000 7fd598647740 1 -- 172.21.15.83:0/1801333557 shutdown_connections 2024-08-19T01:41:21.130 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:41:21.127+0000 7fd598647740 1 -- 172.21.15.83:0/1801333557 wait complete. 2024-08-19T01:41:21.130 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:41:21.128+0000 7fd598647740 20 client.25928 trim_cache size 0 max 16384 2024-08-19T01:41:21.131 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stdout:{ 2024-08-19T01:41:21.131 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stdout: "message": "", 2024-08-19T01:41:21.131 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stdout: "return_code": 0 2024-08-19T01:41:21.131 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stdout:} 2024-08-19T01:41:21.132 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:41:21.128+0000 7fd595800640 1 -- 172.21.15.83:0/3469948270 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7fd5740a2d30 msgr2=0x7fd5740a51f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:41:21.132 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:41:21.128+0000 7fd595800640 1 --2- 172.21.15.83:0/3469948270 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7fd5740a2d30 0x7fd5740a51f0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7fd58009bb30 tx=0x7fd58000c850 comp rx=0 tx=0).stop 2024-08-19T01:41:21.132 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:41:21.128+0000 7fd595800640 1 -- 172.21.15.83:0/3469948270 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd590103ab0 msgr2=0x7fd590199d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:41:21.132 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:41:21.128+0000 7fd595800640 1 --2- 172.21.15.83:0/3469948270 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd590103ab0 0x7fd590199d50 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fd58000e9c0 tx=0x7fd58000ee80 comp rx=0 tx=0).stop 2024-08-19T01:41:21.132 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:41:21.128+0000 7fd595800640 1 -- 172.21.15.83:0/3469948270 shutdown_connections 2024-08-19T01:41:21.132 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:41:21.128+0000 7fd595800640 1 --2- 172.21.15.83:0/3469948270 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7fd5740a2d30 0x7fd5740a51f0 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:41:21.132 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:41:21.128+0000 7fd595800640 1 --2- 172.21.15.83:0/3469948270 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd590105660 0x7fd5901a1300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:41:21.132 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:41:21.128+0000 7fd595800640 1 --2- 172.21.15.83:0/3469948270 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd590104ca0 0x7fd59019a290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:41:21.132 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:41:21.128+0000 7fd595800640 1 --2- 172.21.15.83:0/3469948270 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd590103ab0 0x7fd590199d50 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:41:21.132 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:41:21.128+0000 7fd595800640 1 -- 172.21.15.83:0/3469948270 >> 172.21.15.83:0/3469948270 conn(0x7fd5900ff210 msgr2=0x7fd590107ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:41:21.132 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:41:21.129+0000 7fd595800640 1 -- 172.21.15.83:0/3469948270 shutdown_connections 2024-08-19T01:41:21.132 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:41:21.129+0000 7fd595800640 1 -- 172.21.15.83:0/3469948270 wait complete. 2024-08-19T01:41:21.216 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:41:20 smithi167 ceph-mon[21875]: pgmap v534: 97 pgs: 97 active+clean; 687 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 9.2 MiB/s wr, 738 op/s 2024-08-19T01:41:21.404 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:41:20 smithi083 ceph-mon[17120]: pgmap v534: 97 pgs: 97 active+clean; 687 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 9.2 MiB/s wr, 738 op/s 2024-08-19T01:41:21.452 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:41:20 smithi117 ceph-mon[22016]: pgmap v534: 97 pgs: 97 active+clean; 687 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 9.2 MiB/s wr, 738 op/s 2024-08-19T01:41:23.904 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:41:23 smithi083 ceph-mon[17120]: pgmap v535: 97 pgs: 97 active+clean; 690 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 9.5 MiB/s wr, 943 op/s 2024-08-19T01:41:23.955 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:41:23 smithi117 ceph-mon[22016]: pgmap v535: 97 pgs: 97 active+clean; 690 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 9.5 MiB/s wr, 943 op/s 2024-08-19T01:41:23.965 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:41:23 smithi167 ceph-mon[21875]: pgmap v535: 97 pgs: 97 active+clean; 690 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 9.5 MiB/s wr, 943 op/s 2024-08-19T01:41:25.904 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:41:25 smithi083 ceph-mon[17120]: pgmap v536: 97 pgs: 97 active+clean; 673 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 9.6 MiB/s wr, 1.66k op/s 2024-08-19T01:41:25.952 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:41:25 smithi117 ceph-mon[22016]: pgmap v536: 97 pgs: 97 active+clean; 673 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 9.6 MiB/s wr, 1.66k op/s 2024-08-19T01:41:25.965 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:41:25 smithi167 ceph-mon[21875]: pgmap v536: 97 pgs: 97 active+clean; 673 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 9.6 MiB/s wr, 1.66k op/s 2024-08-19T01:41:27.904 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:41:27 smithi083 ceph-mon[17120]: pgmap v537: 97 pgs: 97 active+clean; 658 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 8.0 MiB/s wr, 1.76k op/s 2024-08-19T01:41:27.952 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:41:27 smithi117 ceph-mon[22016]: pgmap v537: 97 pgs: 97 active+clean; 658 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 8.0 MiB/s wr, 1.76k op/s 2024-08-19T01:41:27.966 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:41:27 smithi167 ceph-mon[21875]: pgmap v537: 97 pgs: 97 active+clean; 658 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 8.0 MiB/s wr, 1.76k op/s 2024-08-19T01:41:29.904 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:41:29 smithi083 ceph-mon[17120]: pgmap v538: 97 pgs: 97 active+clean; 643 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 7.8 MiB/s wr, 2.27k op/s 2024-08-19T01:41:29.952 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:41:29 smithi117 ceph-mon[22016]: pgmap v538: 97 pgs: 97 active+clean; 643 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 7.8 MiB/s wr, 2.27k op/s 2024-08-19T01:41:29.965 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:41:29 smithi167 ceph-mon[21875]: pgmap v538: 97 pgs: 97 active+clean; 643 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 7.8 MiB/s wr, 2.27k op/s 2024-08-19T01:41:31.904 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:41:31 smithi083 ceph-mon[17120]: pgmap v539: 97 pgs: 97 active+clean; 639 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 4.0 MiB/s wr, 2.14k op/s 2024-08-19T01:41:31.952 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:41:31 smithi117 ceph-mon[22016]: pgmap v539: 97 pgs: 97 active+clean; 639 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 4.0 MiB/s wr, 2.14k op/s 2024-08-19T01:41:31.965 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:41:31 smithi167 ceph-mon[21875]: pgmap v539: 97 pgs: 97 active+clean; 639 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 4.0 MiB/s wr, 2.14k op/s 2024-08-19T01:41:33.904 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:41:33 smithi083 ceph-mon[17120]: pgmap v540: 97 pgs: 97 active+clean; 633 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 2.4 MiB/s wr, 2.29k op/s 2024-08-19T01:41:33.904 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:41:33 smithi083 ceph-mon[17120]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-08-19T01:41:33.952 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:41:33 smithi117 ceph-mon[22016]: pgmap v540: 97 pgs: 97 active+clean; 633 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 2.4 MiB/s wr, 2.29k op/s 2024-08-19T01:41:33.952 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:41:33 smithi117 ceph-mon[22016]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-08-19T01:41:33.966 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:41:33 smithi167 ceph-mon[21875]: pgmap v540: 97 pgs: 97 active+clean; 633 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 2.4 MiB/s wr, 2.29k op/s 2024-08-19T01:41:33.966 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:41:33 smithi167 ceph-mon[21875]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-08-19T01:41:34.263 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:41:34.261+0000 7f0d07e00640 1 -- 172.21.15.83:0/4178052303 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0d08104a90 msgr2=0x7f0d08104f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:41:34.263 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:41:34.261+0000 7f0d07e00640 1 --2- 172.21.15.83:0/4178052303 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0d08104a90 0x7f0d08104f10 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f0cf8009900 tx=0x7f0cf802a3f0 comp rx=0 tx=0).stop 2024-08-19T01:41:34.263 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:41:34.261+0000 7f0d07e00640 1 -- 172.21.15.83:0/4178052303 shutdown_connections 2024-08-19T01:41:34.263 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:41:34.261+0000 7f0d07e00640 1 --2- 172.21.15.83:0/4178052303 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0d08105450 0x7f0d0810bce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:41:34.263 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:41:34.261+0000 7f0d07e00640 1 --2- 172.21.15.83:0/4178052303 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0d08104a90 0x7f0d08104f10 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:41:34.263 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:41:34.261+0000 7f0d07e00640 1 --2- 172.21.15.83:0/4178052303 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0d081038a0 0x7f0d08103ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:41:34.263 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:41:34.261+0000 7f0d07e00640 1 -- 172.21.15.83:0/4178052303 >> 172.21.15.83:0/4178052303 conn(0x7f0d080ff020 msgr2=0x7f0d08101440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:41:34.264 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:41:34.261+0000 7f0d07e00640 1 -- 172.21.15.83:0/4178052303 shutdown_connections 2024-08-19T01:41:34.264 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:41:34.261+0000 7f0d07e00640 1 -- 172.21.15.83:0/4178052303 wait complete. 2024-08-19T01:41:34.264 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:41:34.263+0000 7f0d07e00640 1 Processor -- start 2024-08-19T01:41:34.264 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:41:34.263+0000 7f0d07e00640 1 -- start start 2024-08-19T01:41:34.265 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:41:34.263+0000 7f0d07e00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0d081038a0 0x7f0d08199b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:41:34.265 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:41:34.263+0000 7f0d07e00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0d08104a90 0x7f0d0819a070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:41:34.265 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:41:34.263+0000 7f0d07e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0d08105450 0x7f0d081a10e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:41:34.265 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:41:34.263+0000 7f0d07e00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f0d0810f570 con 0x7f0d08105450 2024-08-19T01:41:34.266 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:41:34.263+0000 7f0d07e00640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f0d0810f3f0 con 0x7f0d08104a90 2024-08-19T01:41:34.266 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:41:34.263+0000 7f0d07e00640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f0d0810f6f0 con 0x7f0d081038a0 2024-08-19T01:41:34.266 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:41:34.264+0000 7f0d06000640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0d08104a90 0x7f0d0819a070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:41:34.266 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:41:34.264+0000 7f0d06000640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0d08104a90 0x7f0d0819a070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.83:53250/0 (socket says 172.21.15.83:53250) 2024-08-19T01:41:34.266 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:41:34.264+0000 7f0d06000640 1 -- 172.21.15.83:0/1145885047 learned_addr learned my addr 172.21.15.83:0/1145885047 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-08-19T01:41:34.266 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:41:34.264+0000 7f0d06a00640 1 --2- 172.21.15.83:0/1145885047 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0d081038a0 0x7f0d08199b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:41:34.266 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:41:34.264+0000 7f0d07400640 1 --2- 172.21.15.83:0/1145885047 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0d08105450 0x7f0d081a10e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:41:34.267 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:41:34.265+0000 7f0d06000640 1 -- 172.21.15.83:0/1145885047 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0d081038a0 msgr2=0x7f0d08199b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:41:34.267 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:41:34.265+0000 7f0d06000640 1 --2- 172.21.15.83:0/1145885047 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0d081038a0 0x7f0d08199b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:41:34.267 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:41:34.265+0000 7f0d06000640 1 -- 172.21.15.83:0/1145885047 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0d08105450 msgr2=0x7f0d081a10e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:41:34.267 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:41:34.265+0000 7f0d06000640 1 --2- 172.21.15.83:0/1145885047 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0d08105450 0x7f0d081a10e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:41:34.267 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:41:34.265+0000 7f0d06000640 1 -- 172.21.15.83:0/1145885047 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0d081a1750 con 0x7f0d08104a90 2024-08-19T01:41:34.267 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:41:34.266+0000 7f0d06000640 1 --2- 172.21.15.83:0/1145885047 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0d08104a90 0x7f0d0819a070 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f0cf8031440 tx=0x7f0cf8032ce0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:41:34.268 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:41:34.266+0000 7f0d0ca00640 1 -- 172.21.15.83:0/1145885047 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0cf803f030 con 0x7f0d08104a90 2024-08-19T01:41:34.268 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:41:34.266+0000 7f0d0ca00640 1 -- 172.21.15.83:0/1145885047 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0cf8003ce0 con 0x7f0d08104a90 2024-08-19T01:41:34.268 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:41:34.266+0000 7f0d0ca00640 1 -- 172.21.15.83:0/1145885047 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0cf8035070 con 0x7f0d08104a90 2024-08-19T01:41:34.270 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:41:34.269+0000 7f0d07e00640 1 -- 172.21.15.83:0/1145885047 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0d081a19e0 con 0x7f0d08104a90 2024-08-19T01:41:34.270 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:41:34.269+0000 7f0d07e00640 1 -- 172.21.15.83:0/1145885047 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0d081a1e80 con 0x7f0d08104a90 2024-08-19T01:41:34.275 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:41:34.273+0000 7f0d10caa740 1 Processor -- start 2024-08-19T01:41:34.275 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:41:34.274+0000 7f0d10caa740 1 -- start start 2024-08-19T01:41:34.275 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:41:34.274+0000 7f0d0ca00640 1 -- 172.21.15.83:0/1145885047 <== mon.2 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7f0cf8042030 con 0x7f0d08104a90 2024-08-19T01:41:34.276 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:41:34.274+0000 7f0d10caa740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5634ba2b5e90 0x5634ba2b6290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:41:34.276 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:41:34.274+0000 7f0d10caa740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5634ba2b67d0 0x5634ba2bb290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:41:34.276 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:41:34.274+0000 7f0d10caa740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5634ba2bb860 0x5634ba2bdd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:41:34.276 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:41:34.274+0000 7f0d10caa740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x5634ba211890 con 0x5634ba2b5e90 2024-08-19T01:41:34.276 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:41:34.274+0000 7f0d10caa740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x5634ba1a9d90 con 0x5634ba2bb860 2024-08-19T01:41:34.276 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:41:34.274+0000 7f0d10caa740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x5634ba20d850 con 0x5634ba2b67d0 2024-08-19T01:41:34.277 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:41:34.275+0000 7f0d07400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5634ba2b5e90 0x5634ba2b6290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:41:34.277 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:41:34.275+0000 7f0d06a00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5634ba2b67d0 0x5634ba2bb290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:41:34.277 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:41:34.275+0000 7f0d06a00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5634ba2b67d0 0x5634ba2bb290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.83:34964/0 (socket says 172.21.15.83:34964) 2024-08-19T01:41:34.277 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:41:34.275+0000 7f0d06a00640 1 -- 172.21.15.83:0/3660321140 learned_addr learned my addr 172.21.15.83:0/3660321140 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-08-19T01:41:34.277 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:41:34.276+0000 7f0d06a00640 1 -- 172.21.15.83:0/3660321140 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5634ba2bb860 msgr2=0x5634ba2bdd20 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-08-19T01:41:34.277 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:41:34.276+0000 7f0d06a00640 1 --2- 172.21.15.83:0/3660321140 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5634ba2bb860 0x5634ba2bdd20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:41:34.278 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:41:34.276+0000 7f0d06a00640 1 -- 172.21.15.83:0/3660321140 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5634ba2b5e90 msgr2=0x5634ba2b6290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:41:34.278 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:41:34.276+0000 7f0d06a00640 1 --2- 172.21.15.83:0/3660321140 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5634ba2b5e90 0x5634ba2b6290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:41:34.278 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:41:34.276+0000 7f0d06a00640 1 -- 172.21.15.83:0/3660321140 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5634b9f25210 con 0x5634ba2b67d0 2024-08-19T01:41:34.278 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:41:34.276+0000 7f0d07400640 1 --2- 172.21.15.83:0/3660321140 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5634ba2b5e90 0x5634ba2b6290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-19T01:41:34.278 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:41:34.277+0000 7f0d06a00640 1 --2- 172.21.15.83:0/3660321140 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5634ba2b67d0 0x5634ba2bb290 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f0cf4009ca0 tx=0x7f0cf4011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a3641ad174aba9d9 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:41:34.279 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:41:34.277+0000 7f0cf3e00640 1 -- 172.21.15.83:0/3660321140 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0cf4004220 con 0x5634ba2b67d0 2024-08-19T01:41:34.279 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:41:34.278+0000 7f0cf3e00640 1 -- 172.21.15.83:0/3660321140 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0cf40043c0 con 0x5634ba2b67d0 2024-08-19T01:41:34.279 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:41:34.278+0000 7f0cf3e00640 1 -- 172.21.15.83:0/3660321140 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0cf4019970 con 0x5634ba2b67d0 2024-08-19T01:41:34.280 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:41:34.278+0000 7f0d0ca00640 1 --2- 172.21.15.83:0/1145885047 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f0cec0a2c80 0x7f0cec0a5140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:41:34.280 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:41:34.278+0000 7f0d0ca00640 1 -- 172.21.15.83:0/1145885047 <== mon.2 v2:172.21.15.117:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8269+0+0 (secure 0 0 0) 0x7f0cf80ef1c0 con 0x7f0d08104a90 2024-08-19T01:41:34.280 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:41:34.278+0000 7f0d07400640 1 --2- 172.21.15.83:0/1145885047 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f0cec0a2c80 0x7f0cec0a5140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:41:34.282 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:41:34.280+0000 7f0d10caa740 1 -- 172.21.15.83:0/3660321140 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5634ba2b67d0 msgr2=0x5634ba2bb290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:41:34.282 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:41:34.280+0000 7f0d10caa740 1 --2- 172.21.15.83:0/3660321140 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5634ba2b67d0 0x5634ba2bb290 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f0cf4009ca0 tx=0x7f0cf4011810 comp rx=0 tx=0).stop 2024-08-19T01:41:34.282 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:41:34.280+0000 7f0d07400640 1 --2- 172.21.15.83:0/1145885047 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f0cec0a2c80 0x7f0cec0a5140 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f0d00007470 tx=0x7f0d00007400 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:41:34.282 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:41:34.280+0000 7f0d10caa740 1 -- 172.21.15.83:0/3660321140 shutdown_connections 2024-08-19T01:41:34.282 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:41:34.280+0000 7f0d10caa740 1 --2- 172.21.15.83:0/3660321140 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5634ba2bb860 0x5634ba2bdd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:41:34.282 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:41:34.280+0000 7f0d10caa740 1 --2- 172.21.15.83:0/3660321140 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5634ba2b67d0 0x5634ba2bb290 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:41:34.282 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:41:34.280+0000 7f0d10caa740 1 --2- 172.21.15.83:0/3660321140 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5634ba2b5e90 0x5634ba2b6290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:41:34.282 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:41:34.280+0000 7f0d10caa740 1 -- 172.21.15.83:0/3660321140 >> 172.21.15.83:0/3660321140 conn(0x5634ba2b1500 msgr2=0x5634ba2b1900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:41:34.282 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:41:34.280+0000 7f0d10caa740 1 -- 172.21.15.83:0/3660321140 shutdown_connections 2024-08-19T01:41:34.282 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:41:34.280+0000 7f0d10caa740 1 -- 172.21.15.83:0/3660321140 wait complete. 2024-08-19T01:41:34.283 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:41:34.281+0000 7f0d10caa740 1 -- start start 2024-08-19T01:41:34.283 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:41:34.282+0000 7f0d10caa740 1 Processor -- start 2024-08-19T01:41:34.285 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:41:34.283+0000 7f0d10caa740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5634ba2b5e90 0x5634ba2c0250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:41:34.285 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:41:34.283+0000 7f0d10caa740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5634ba2b67d0 0x5634ba2c0790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:41:34.285 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:41:34.283+0000 7f0d10caa740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5634ba2bb860 0x5634ba2c7760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:41:34.285 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:41:34.283+0000 7f0d10caa740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x5634ba2be8d0 con 0x5634ba2bb860 2024-08-19T01:41:34.285 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:41:34.283+0000 7f0d10caa740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x5634ba1ccf70 con 0x5634ba2b67d0 2024-08-19T01:41:34.285 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:41:34.283+0000 7f0d10caa740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x5634ba2bea50 con 0x5634ba2b5e90 2024-08-19T01:41:34.285 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:41:34.283+0000 7f0d10caa740 10 client.0 _ll_register_callbacks cb 0x5634ba2098d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-19T01:41:34.286 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:41:34.284+0000 7f0d06a00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5634ba2b5e90 0x5634ba2c0250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:41:34.286 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:41:34.284+0000 7f0d06a00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5634ba2b5e90 0x5634ba2c0250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.83:34968/0 (socket says 172.21.15.83:34968) 2024-08-19T01:41:34.286 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:41:34.284+0000 7f0d06a00640 1 -- 172.21.15.83:0/2132261197 learned_addr learned my addr 172.21.15.83:0/2132261197 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-08-19T01:41:34.286 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:41:34.284+0000 7f0d06000640 1 --2- 172.21.15.83:0/2132261197 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5634ba2b67d0 0x5634ba2c0790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:41:34.286 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:41:34.284+0000 7f0d07400640 1 --2- 172.21.15.83:0/2132261197 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5634ba2bb860 0x5634ba2c7760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:41:34.286 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:41:34.284+0000 7f0d06a00640 1 -- 172.21.15.83:0/2132261197 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5634ba2b67d0 msgr2=0x5634ba2c0790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:41:34.286 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:41:34.284+0000 7f0d06a00640 1 --2- 172.21.15.83:0/2132261197 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5634ba2b67d0 0x5634ba2c0790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:41:34.287 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:41:34.284+0000 7f0d06a00640 1 -- 172.21.15.83:0/2132261197 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5634ba2bb860 msgr2=0x5634ba2c7760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:41:34.287 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:41:34.284+0000 7f0d06a00640 1 --2- 172.21.15.83:0/2132261197 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5634ba2bb860 0x5634ba2c7760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:41:34.287 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:41:34.284+0000 7f0d06a00640 1 -- 172.21.15.83:0/2132261197 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5634b9ff1620 con 0x5634ba2b5e90 2024-08-19T01:41:34.288 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:41:34.286+0000 7f0d06a00640 1 --2- 172.21.15.83:0/2132261197 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5634ba2b5e90 0x5634ba2c0250 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f0cf401b040 tx=0x7f0cf40045e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cb2b412fdcdfc8c4 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:41:34.288 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:41:34.286+0000 7f0cf0c00640 10 client.0 ms_handle_connect on v2:172.21.15.167:3300/0 2024-08-19T01:41:34.288 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:41:34.286+0000 7f0cf0c00640 1 -- 172.21.15.83:0/2132261197 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0cf4028070 con 0x5634ba2b5e90 2024-08-19T01:41:34.288 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:41:34.286+0000 7f0cf0c00640 1 -- 172.21.15.83:0/2132261197 <== mon.1 v2:172.21.15.167:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8269+0+0 (secure 0 0 0) 0x7f0cf4003880 con 0x5634ba2b5e90 2024-08-19T01:41:34.288 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:41:34.286+0000 7f0d10caa740 1 -- 172.21.15.83:0/2132261197 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0cf40043c0 con 0x5634ba2b5e90 2024-08-19T01:41:34.288 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:41:34.286+0000 7f0cf0c00640 1 -- 172.21.15.83:0/2132261197 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f0cf4034050 con 0x5634ba2b5e90 2024-08-19T01:41:34.288 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:41:34.286+0000 7f0d10caa740 10 client.25873 fetch_fsmap learned FSMap version 28 2024-08-19T01:41:34.288 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:41:34.286+0000 7f0d10caa740 1 -- 172.21.15.83:0/2132261197 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5634ba2ca8e0 con 0x5634ba2b5e90 2024-08-19T01:41:34.289 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:41:34.287+0000 7f0cf0c00640 1 -- 172.21.15.83:0/2132261197 <== mon.1 v2:172.21.15.167:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0cf40043c0 con 0x5634ba2b5e90 2024-08-19T01:41:34.289 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:41:34.288+0000 7f0cf0c00640 1 -- 172.21.15.83:0/2132261197 <== mon.1 v2:172.21.15.167:3300/0 5 ==== fsmap(e 28) ==== 8358+0+0 (secure 0 0 0) 0x7f0cf4004ab0 con 0x5634ba2b5e90 2024-08-19T01:41:34.289 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:41:34.288+0000 7f0d10caa740 10 client.25873 fetch_fsmap finished waiting for FSMap version 28 2024-08-19T01:41:34.289 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:41:34.288+0000 7f0d10caa740 10 client.25873 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-08-19T01:41:34.290 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:41:34.288+0000 7f0d10caa740 20 client.25873 populate_metadata read hostname 'smithi083' 2024-08-19T01:41:34.290 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:41:34.288+0000 7f0d10caa740 1 --2- 172.21.15.83:0/2132261197 >> [v2:172.21.15.117:6838/1967595704,v1:172.21.15.117:6839/1967595704] conn(0x5634ba2cbda0 0x5634ba2ce1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:41:34.290 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:41:34.288+0000 7f0d10caa740 4 client.25873 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-19T01:41:34.290 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:41:34.288+0000 7f0d10caa740 1 -- 172.21.15.83:0/2132261197 --> [v2:172.21.15.117:6838/1967595704,v1:172.21.15.117:6839/1967595704] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5634ba21efc0 con 0x5634ba2cbda0 2024-08-19T01:41:34.291 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:41:34.288+0000 7f0d06000640 1 --2- 172.21.15.83:0/2132261197 >> [v2:172.21.15.117:6838/1967595704,v1:172.21.15.117:6839/1967595704] conn(0x5634ba2cbda0 0x5634ba2ce1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:41:34.291 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:41:34.289+0000 7f0d06000640 1 --2- 172.21.15.83:0/2132261197 >> [v2:172.21.15.117:6838/1967595704,v1:172.21.15.117:6839/1967595704] conn(0x5634ba2cbda0 0x5634ba2ce1e0 crc :-1 s=READY pgs=117 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=278e5dd1492cd12 server_cookie=bcaf35e1333ac66b in_seq=0 out_seq=0 2024-08-19T01:41:34.291 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:41:34.289+0000 7f0cf0c00640 10 client.25873 ms_handle_connect on v2:172.21.15.117:6838/1967595704 2024-08-19T01:41:34.293 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:41:34.292+0000 7f0cf0c00640 1 -- 172.21.15.83:0/2132261197 <== mds.0 v2:172.21.15.117:6838/1967595704 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5634ba21efc0 con 0x5634ba2cbda0 2024-08-19T01:41:34.293 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:41:34.292+0000 7f0cf0c00640 10 client.25873 handle_command_reply: tid=0 2024-08-19T01:41:34.293 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:41:34.292+0000 7f0d10caa740 1 client.25873 shutdown 2024-08-19T01:41:34.293 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:41:34.292+0000 7f0d10caa740 1 -- 172.21.15.83:0/2132261197 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5634ba2b5e90 msgr2=0x5634ba2c0250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:41:34.293 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:41:34.292+0000 7f0d10caa740 1 --2- 172.21.15.83:0/2132261197 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5634ba2b5e90 0x5634ba2c0250 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f0cf401b040 tx=0x7f0cf40045e0 comp rx=0 tx=0).stop 2024-08-19T01:41:34.294 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:41:34.292+0000 7f0d10caa740 1 -- 172.21.15.83:0/2132261197 shutdown_connections 2024-08-19T01:41:34.294 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:41:34.292+0000 7f0d10caa740 1 --2- 172.21.15.83:0/2132261197 >> [v2:172.21.15.117:6838/1967595704,v1:172.21.15.117:6839/1967595704] conn(0x5634ba2cbda0 0x5634ba2ce1e0 crc :-1 s=READY pgs=117 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:41:34.294 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:41:34.292+0000 7f0d10caa740 1 --2- 172.21.15.83:0/2132261197 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5634ba2bb860 0x5634ba2c7760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:41:34.294 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:41:34.292+0000 7f0d10caa740 1 --2- 172.21.15.83:0/2132261197 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5634ba2b67d0 0x5634ba2c0790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:41:34.294 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:41:34.292+0000 7f0d10caa740 1 --2- 172.21.15.83:0/2132261197 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5634ba2b5e90 0x5634ba2c0250 secure :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f0cf401b040 tx=0x7f0cf40045e0 comp rx=0 tx=0).stop 2024-08-19T01:41:34.294 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:41:34.292+0000 7f0d10caa740 1 -- 172.21.15.83:0/2132261197 >> 172.21.15.83:0/2132261197 conn(0x5634ba2b1500 msgr2=0x5634ba2a7f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:41:34.295 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:41:34.292+0000 7f0cf0c00640 0 client.25873 ms_handle_reset on v2:172.21.15.117:6838/1967595704 2024-08-19T01:41:34.295 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:41:34.292+0000 7f0d10caa740 1 -- 172.21.15.83:0/2132261197 shutdown_connections 2024-08-19T01:41:34.295 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:41:34.293+0000 7f0d10caa740 1 -- 172.21.15.83:0/2132261197 wait complete. 2024-08-19T01:41:34.295 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:41:34.293+0000 7f0d10caa740 20 client.25873 trim_cache size 0 max 16384 2024-08-19T01:41:34.319 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:41:34.318+0000 7f0d10caa740 1 Processor -- start 2024-08-19T01:41:34.321 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:41:34.318+0000 7f0d10caa740 1 -- start start 2024-08-19T01:41:34.321 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:41:34.318+0000 7f0d10caa740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5634ba2b67d0 0x5634ba2c0710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:41:34.321 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:41:34.318+0000 7f0d10caa740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5634ba2bb860 0x5634ba2c7720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:41:34.321 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:41:34.318+0000 7f0d10caa740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5634ba2b6f20 0x5634ba2a7720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:41:34.321 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:41:34.318+0000 7f0d10caa740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f0d081a92f0 con 0x5634ba2bb860 2024-08-19T01:41:34.321 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:41:34.318+0000 7f0d10caa740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f0d081a6b90 con 0x5634ba2b6f20 2024-08-19T01:41:34.321 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:41:34.318+0000 7f0d10caa740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f0d081a4a00 con 0x5634ba2b67d0 2024-08-19T01:41:34.321 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:41:34.319+0000 7f0d06000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5634ba2bb860 0x5634ba2c7720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:41:34.321 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:41:34.319+0000 7f0d06000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5634ba2bb860 0x5634ba2c7720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:47894/0 (socket says 172.21.15.83:47894) 2024-08-19T01:41:34.321 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:41:34.319+0000 7f0d06000640 1 -- 172.21.15.83:0/492891068 learned_addr learned my addr 172.21.15.83:0/492891068 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-08-19T01:41:34.322 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:41:34.319+0000 7f0d07400640 1 --2- 172.21.15.83:0/492891068 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5634ba2b6f20 0x5634ba2a7720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:41:34.322 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:41:34.319+0000 7f0d06000640 1 -- 172.21.15.83:0/492891068 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5634ba2b67d0 msgr2=0x5634ba2c0710 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-08-19T01:41:34.322 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:41:34.319+0000 7f0d06000640 1 --2- 172.21.15.83:0/492891068 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5634ba2b67d0 0x5634ba2c0710 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:41:34.322 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:41:34.319+0000 7f0d06000640 1 -- 172.21.15.83:0/492891068 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5634ba2b6f20 msgr2=0x5634ba2a7720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:41:34.322 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:41:34.319+0000 7f0d06000640 1 --2- 172.21.15.83:0/492891068 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5634ba2b6f20 0x5634ba2a7720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:41:34.322 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:41:34.319+0000 7f0d06000640 1 -- 172.21.15.83:0/492891068 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5634ba2bdc50 con 0x5634ba2bb860 2024-08-19T01:41:34.322 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:41:34.320+0000 7f0d06000640 1 --2- 172.21.15.83:0/492891068 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5634ba2bb860 0x5634ba2c7720 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f0cf80b7e00 tx=0x7f0cf80ecb10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7dda022ef02477ec server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:41:34.323 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:41:34.320+0000 7f0cf3400640 1 -- 172.21.15.83:0/492891068 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0cf80d3070 con 0x5634ba2bb860 2024-08-19T01:41:34.323 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:41:34.320+0000 7f0cf3400640 1 -- 172.21.15.83:0/492891068 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0cf80bb9a0 con 0x5634ba2bb860 2024-08-19T01:41:34.323 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:41:34.320+0000 7f0cf3400640 1 -- 172.21.15.83:0/492891068 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0cf80b8010 con 0x5634ba2bb860 2024-08-19T01:41:34.323 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:41:34.321+0000 7f0d10caa740 1 -- 172.21.15.83:0/492891068 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5634ba2bb860 msgr2=0x5634ba2c7720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:41:34.323 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:41:34.321+0000 7f0d10caa740 1 --2- 172.21.15.83:0/492891068 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5634ba2bb860 0x5634ba2c7720 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f0cf80b7e00 tx=0x7f0cf80ecb10 comp rx=0 tx=0).stop 2024-08-19T01:41:34.323 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:41:34.321+0000 7f0d10caa740 1 -- 172.21.15.83:0/492891068 shutdown_connections 2024-08-19T01:41:34.323 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:41:34.321+0000 7f0d10caa740 1 --2- 172.21.15.83:0/492891068 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5634ba2b6f20 0x5634ba2a7720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:41:34.324 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:41:34.321+0000 7f0d10caa740 1 --2- 172.21.15.83:0/492891068 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5634ba2bb860 0x5634ba2c7720 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:41:34.324 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:41:34.321+0000 7f0d10caa740 1 --2- 172.21.15.83:0/492891068 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5634ba2b67d0 0x5634ba2c0710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:41:34.324 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:41:34.321+0000 7f0d10caa740 1 -- 172.21.15.83:0/492891068 >> 172.21.15.83:0/492891068 conn(0x5634ba2b1500 msgr2=0x5634ba2c0000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:41:34.324 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:41:34.321+0000 7f0d10caa740 1 -- 172.21.15.83:0/492891068 shutdown_connections 2024-08-19T01:41:34.324 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:41:34.321+0000 7f0d10caa740 1 -- 172.21.15.83:0/492891068 wait complete. 2024-08-19T01:41:34.324 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:41:34.322+0000 7f0d10caa740 1 -- start start 2024-08-19T01:41:34.324 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:41:34.323+0000 7f0d10caa740 1 Processor -- start 2024-08-19T01:41:34.326 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:41:34.324+0000 7f0d10caa740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5634ba2b67d0 0x5634ba2c7600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:41:34.327 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:41:34.324+0000 7f0d10caa740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5634ba2b6f20 0x5634ba2c40f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:41:34.327 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:41:34.324+0000 7f0d10caa740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5634ba2bb860 0x5634ba2c4630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:41:34.327 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:41:34.324+0000 7f0d10caa740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f0d081a5050 con 0x5634ba2b6f20 2024-08-19T01:41:34.327 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:41:34.324+0000 7f0d10caa740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f0d081a64c0 con 0x5634ba2bb860 2024-08-19T01:41:34.327 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:41:34.324+0000 7f0d10caa740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f0d081a5720 con 0x5634ba2b67d0 2024-08-19T01:41:34.327 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:41:34.325+0000 7f0d10caa740 10 client.0 _ll_register_callbacks cb 0x7f0cf4009af0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-19T01:41:34.327 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:41:34.325+0000 7f0d06a00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5634ba2b67d0 0x5634ba2c7600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:41:34.327 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:41:34.325+0000 7f0d06000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5634ba2b6f20 0x5634ba2c40f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:41:34.327 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:41:34.325+0000 7f0d06a00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5634ba2b67d0 0x5634ba2c7600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.83:34986/0 (socket says 172.21.15.83:34986) 2024-08-19T01:41:34.328 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:41:34.325+0000 7f0d06a00640 1 -- 172.21.15.83:0/3050124389 learned_addr learned my addr 172.21.15.83:0/3050124389 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-08-19T01:41:34.328 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:41:34.325+0000 7f0d07400640 1 --2- 172.21.15.83:0/3050124389 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5634ba2bb860 0x5634ba2c4630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:41:34.328 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:41:34.325+0000 7f0d06a00640 1 -- 172.21.15.83:0/3050124389 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5634ba2bb860 msgr2=0x5634ba2c4630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:41:34.328 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:41:34.325+0000 7f0d06a00640 1 --2- 172.21.15.83:0/3050124389 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5634ba2bb860 0x5634ba2c4630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:41:34.328 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:41:34.325+0000 7f0d06a00640 1 -- 172.21.15.83:0/3050124389 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5634ba2b6f20 msgr2=0x5634ba2c40f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:41:34.328 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:41:34.325+0000 7f0d06a00640 1 --2- 172.21.15.83:0/3050124389 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5634ba2b6f20 0x5634ba2c40f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:41:34.328 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:41:34.325+0000 7f0d06a00640 1 -- 172.21.15.83:0/3050124389 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5634ba2ab740 con 0x5634ba2b67d0 2024-08-19T01:41:34.328 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:41:34.325+0000 7f0d06a00640 1 --2- 172.21.15.83:0/3050124389 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5634ba2b67d0 0x5634ba2c7600 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f0cf4004fd0 tx=0x7f0cf40096b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=afe1fdcb7c3540ed server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:41:34.328 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:41:34.325+0000 7f0cf0c00640 10 client.0 ms_handle_connect on v2:172.21.15.167:3300/0 2024-08-19T01:41:34.328 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:41:34.326+0000 7f0cf0c00640 1 -- 172.21.15.83:0/3050124389 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0cf4028070 con 0x5634ba2b67d0 2024-08-19T01:41:34.328 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:41:34.326+0000 7f0cf0c00640 1 -- 172.21.15.83:0/3050124389 <== mon.1 v2:172.21.15.167:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8269+0+0 (secure 0 0 0) 0x7f0cf4003880 con 0x5634ba2b67d0 2024-08-19T01:41:34.328 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:41:34.326+0000 7f0d10caa740 1 -- 172.21.15.83:0/3050124389 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0cf80bb9a0 con 0x5634ba2b67d0 2024-08-19T01:41:34.329 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:41:34.326+0000 7f0cf0c00640 1 -- 172.21.15.83:0/3050124389 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f0cf403e070 con 0x5634ba2b67d0 2024-08-19T01:41:34.329 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:41:34.328+0000 7f0d10caa740 10 client.25876 fetch_fsmap learned FSMap version 28 2024-08-19T01:41:34.330 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:41:34.328+0000 7f0d10caa740 1 -- 172.21.15.83:0/3050124389 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5634ba2aca50 con 0x5634ba2b67d0 2024-08-19T01:41:34.330 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:41:34.328+0000 7f0cf0c00640 1 -- 172.21.15.83:0/3050124389 <== mon.1 v2:172.21.15.167:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0cf80bb9a0 con 0x5634ba2b67d0 2024-08-19T01:41:34.330 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:41:34.329+0000 7f0cf0c00640 1 -- 172.21.15.83:0/3050124389 <== mon.1 v2:172.21.15.167:3300/0 5 ==== fsmap(e 28) ==== 8358+0+0 (secure 0 0 0) 0x7f0cf4004510 con 0x5634ba2b67d0 2024-08-19T01:41:34.331 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:41:34.329+0000 7f0d10caa740 10 client.25876 fetch_fsmap finished waiting for FSMap version 28 2024-08-19T01:41:34.331 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:41:34.329+0000 7f0d10caa740 10 client.25876 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-08-19T01:41:34.332 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:41:34.329+0000 7f0d10caa740 20 client.25876 populate_metadata read hostname 'smithi083' 2024-08-19T01:41:34.332 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:41:34.329+0000 7f0d10caa740 1 --2- 172.21.15.83:0/3050124389 >> [v2:172.21.15.117:6838/1967595704,v1:172.21.15.117:6839/1967595704] conn(0x5634ba2ad330 0x5634ba2af750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:41:34.332 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:41:34.329+0000 7f0d10caa740 4 client.25876 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-08-19T01:41:34.332 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:41:34.329+0000 7f0d10caa740 1 -- 172.21.15.83:0/3050124389 --> [v2:172.21.15.117:6838/1967595704,v1:172.21.15.117:6839/1967595704] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5634ba219310 con 0x5634ba2ad330 2024-08-19T01:41:34.332 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:41:34.330+0000 7f0d06000640 1 --2- 172.21.15.83:0/3050124389 >> [v2:172.21.15.117:6838/1967595704,v1:172.21.15.117:6839/1967595704] conn(0x5634ba2ad330 0x5634ba2af750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:41:34.333 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:41:34.332+0000 7f0d06000640 1 --2- 172.21.15.83:0/3050124389 >> [v2:172.21.15.117:6838/1967595704,v1:172.21.15.117:6839/1967595704] conn(0x5634ba2ad330 0x5634ba2af750 crc :-1 s=READY pgs=118 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4ae74b78827bcdc3 server_cookie=4e5b8b3e7eb8928a in_seq=0 out_seq=0 2024-08-19T01:41:34.333 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:41:34.332+0000 7f0cf0c00640 10 client.25876 ms_handle_connect on v2:172.21.15.117:6838/1967595704 2024-08-19T01:41:34.793 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:41:34.791+0000 7f0cf0c00640 1 -- 172.21.15.83:0/3050124389 <== mds.0 v2:172.21.15.117:6838/1967595704 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5634ba219310 con 0x5634ba2ad330 2024-08-19T01:41:34.794 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:41:34.791+0000 7f0cf0c00640 10 client.25876 handle_command_reply: tid=0 2024-08-19T01:41:34.794 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:41:34.791+0000 7f0d10caa740 1 client.25876 shutdown 2024-08-19T01:41:34.794 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:41:34.792+0000 7f0d10caa740 1 -- 172.21.15.83:0/3050124389 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5634ba2b67d0 msgr2=0x5634ba2c7600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:41:34.794 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:41:34.792+0000 7f0d10caa740 1 --2- 172.21.15.83:0/3050124389 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5634ba2b67d0 0x5634ba2c7600 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f0cf4004fd0 tx=0x7f0cf40096b0 comp rx=0 tx=0).stop 2024-08-19T01:41:34.794 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:41:34.792+0000 7f0d10caa740 1 -- 172.21.15.83:0/3050124389 shutdown_connections 2024-08-19T01:41:34.794 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:41:34.792+0000 7f0d10caa740 1 --2- 172.21.15.83:0/3050124389 >> [v2:172.21.15.117:6838/1967595704,v1:172.21.15.117:6839/1967595704] conn(0x5634ba2ad330 0x5634ba2af750 crc :-1 s=READY pgs=118 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:41:34.794 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:41:34.792+0000 7f0d10caa740 1 --2- 172.21.15.83:0/3050124389 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5634ba2bb860 0x5634ba2c4630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:41:34.794 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:41:34.792+0000 7f0d10caa740 1 --2- 172.21.15.83:0/3050124389 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5634ba2b6f20 0x5634ba2c40f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:41:34.794 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:41:34.792+0000 7f0d10caa740 1 --2- 172.21.15.83:0/3050124389 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5634ba2b67d0 0x5634ba2c7600 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:41:34.794 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:41:34.792+0000 7f0d10caa740 1 -- 172.21.15.83:0/3050124389 >> 172.21.15.83:0/3050124389 conn(0x5634ba2b1500 msgr2=0x5634ba2b2190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:41:34.794 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:41:34.792+0000 7f0cf0c00640 0 client.25876 ms_handle_reset on v2:172.21.15.117:6838/1967595704 2024-08-19T01:41:34.795 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:41:34.792+0000 7f0d10caa740 1 -- 172.21.15.83:0/3050124389 shutdown_connections 2024-08-19T01:41:34.795 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:41:34.792+0000 7f0d10caa740 1 -- 172.21.15.83:0/3050124389 wait complete. 2024-08-19T01:41:34.795 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:41:34.792+0000 7f0d10caa740 20 client.25876 trim_cache size 0 max 16384 2024-08-19T01:41:34.795 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stdout:{ 2024-08-19T01:41:34.795 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stdout: "message": "", 2024-08-19T01:41:34.795 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stdout: "return_code": 0 2024-08-19T01:41:34.795 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stdout:} 2024-08-19T01:41:34.796 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:41:34.793+0000 7f0d07e00640 1 -- 172.21.15.83:0/1145885047 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f0cec0a2c80 msgr2=0x7f0cec0a5140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:41:34.796 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:41:34.793+0000 7f0d07e00640 1 --2- 172.21.15.83:0/1145885047 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f0cec0a2c80 0x7f0cec0a5140 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f0d00007470 tx=0x7f0d00007400 comp rx=0 tx=0).stop 2024-08-19T01:41:34.796 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:41:34.793+0000 7f0d07e00640 1 -- 172.21.15.83:0/1145885047 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0d08104a90 msgr2=0x7f0d0819a070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:41:34.796 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:41:34.793+0000 7f0d07e00640 1 --2- 172.21.15.83:0/1145885047 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0d08104a90 0x7f0d0819a070 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f0cf8031440 tx=0x7f0cf8032ce0 comp rx=0 tx=0).stop 2024-08-19T01:41:34.796 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:41:34.793+0000 7f0d07e00640 1 -- 172.21.15.83:0/1145885047 shutdown_connections 2024-08-19T01:41:34.796 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:41:34.793+0000 7f0d07e00640 1 --2- 172.21.15.83:0/1145885047 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f0cec0a2c80 0x7f0cec0a5140 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:41:34.796 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:41:34.793+0000 7f0d07e00640 1 --2- 172.21.15.83:0/1145885047 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0d08105450 0x7f0d081a10e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:41:34.796 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:41:34.793+0000 7f0d07e00640 1 --2- 172.21.15.83:0/1145885047 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0d08104a90 0x7f0d0819a070 secure :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f0cf8031440 tx=0x7f0cf8032ce0 comp rx=0 tx=0).stop 2024-08-19T01:41:34.796 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:41:34.793+0000 7f0d07e00640 1 --2- 172.21.15.83:0/1145885047 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0d081038a0 0x7f0d08199b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:41:34.797 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:41:34.793+0000 7f0d07e00640 1 -- 172.21.15.83:0/1145885047 >> 172.21.15.83:0/1145885047 conn(0x7f0d080ff020 msgr2=0x7f0d08107d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:41:34.797 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:41:34.793+0000 7f0d07e00640 1 -- 172.21.15.83:0/1145885047 shutdown_connections 2024-08-19T01:41:34.797 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:41:34.793+0000 7f0d07e00640 1 -- 172.21.15.83:0/1145885047 wait complete. 2024-08-19T01:41:35.904 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:41:35 smithi083 ceph-mon[17120]: pgmap v541: 97 pgs: 97 active+clean; 616 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 2.1 MiB/s wr, 2.87k op/s 2024-08-19T01:41:35.952 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:41:35 smithi117 ceph-mon[22016]: pgmap v541: 97 pgs: 97 active+clean; 616 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 2.1 MiB/s wr, 2.87k op/s 2024-08-19T01:41:35.965 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:41:35 smithi167 ceph-mon[21875]: pgmap v541: 97 pgs: 97 active+clean; 616 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 2.1 MiB/s wr, 2.87k op/s 2024-08-19T01:41:37.904 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:41:37 smithi083 ceph-mon[17120]: pgmap v542: 97 pgs: 97 active+clean; 611 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 1.5 MiB/s wr, 2.38k op/s 2024-08-19T01:41:37.952 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:41:37 smithi117 ceph-mon[22016]: pgmap v542: 97 pgs: 97 active+clean; 611 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 1.5 MiB/s wr, 2.38k op/s 2024-08-19T01:41:37.966 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:41:37 smithi167 ceph-mon[21875]: pgmap v542: 97 pgs: 97 active+clean; 611 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 1.5 MiB/s wr, 2.38k op/s 2024-08-19T01:41:39.904 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:41:39 smithi083 ceph-mon[17120]: pgmap v543: 97 pgs: 97 active+clean; 583 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 2.0 MiB/s wr, 2.76k op/s 2024-08-19T01:41:39.904 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:41:39 smithi083 ceph-mon[17120]: scrub summary: idle+waiting paths [/] 2024-08-19T01:41:39.952 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:41:39 smithi117 ceph-mon[22016]: pgmap v543: 97 pgs: 97 active+clean; 583 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 2.0 MiB/s wr, 2.76k op/s 2024-08-19T01:41:39.952 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:41:39 smithi117 ceph-mon[22016]: scrub summary: idle+waiting paths [/] 2024-08-19T01:41:39.965 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:41:39 smithi167 ceph-mon[21875]: pgmap v543: 97 pgs: 97 active+clean; 583 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 2.0 MiB/s wr, 2.76k op/s 2024-08-19T01:41:39.966 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:41:39 smithi167 ceph-mon[21875]: scrub summary: idle+waiting paths [/] 2024-08-19T01:41:41.909 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:41:41 smithi083 ceph-mon[17120]: scrub summary: idle 2024-08-19T01:41:41.909 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:41:41 smithi083 ceph-mon[17120]: scrub summary: idle 2024-08-19T01:41:41.909 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:41:41 smithi083 ceph-mon[17120]: scrub summary: idle 2024-08-19T01:41:41.909 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:41:41 smithi083 ceph-mon[17120]: pgmap v544: 97 pgs: 97 active+clean; 565 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 1.6 MiB/s wr, 2.44k op/s 2024-08-19T01:41:41.952 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:41:41 smithi117 ceph-mon[22016]: scrub summary: idle 2024-08-19T01:41:41.952 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:41:41 smithi117 ceph-mon[22016]: scrub summary: idle 2024-08-19T01:41:41.952 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:41:41 smithi117 ceph-mon[22016]: scrub summary: idle 2024-08-19T01:41:41.952 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:41:41 smithi117 ceph-mon[22016]: pgmap v544: 97 pgs: 97 active+clean; 565 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 1.6 MiB/s wr, 2.44k op/s 2024-08-19T01:41:41.965 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:41:41 smithi167 ceph-mon[21875]: scrub summary: idle 2024-08-19T01:41:41.966 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:41:41 smithi167 ceph-mon[21875]: scrub summary: idle 2024-08-19T01:41:41.966 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:41:41 smithi167 ceph-mon[21875]: scrub summary: idle 2024-08-19T01:41:41.966 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:41:41 smithi167 ceph-mon[21875]: pgmap v544: 97 pgs: 97 active+clean; 565 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 1.6 MiB/s wr, 2.44k op/s 2024-08-19T01:41:43.904 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:41:43 smithi083 ceph-mon[17120]: pgmap v545: 97 pgs: 97 active+clean; 558 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 1.2 MiB/s wr, 2.34k op/s 2024-08-19T01:41:43.952 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:41:43 smithi117 ceph-mon[22016]: pgmap v545: 97 pgs: 97 active+clean; 558 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 1.2 MiB/s wr, 2.34k op/s 2024-08-19T01:41:43.965 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:41:43 smithi167 ceph-mon[21875]: pgmap v545: 97 pgs: 97 active+clean; 558 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 1.2 MiB/s wr, 2.34k op/s 2024-08-19T01:41:45.952 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:41:45 smithi117 ceph-mon[22016]: pgmap v546: 97 pgs: 97 active+clean; 546 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 1.9 MiB/s wr, 2.22k op/s 2024-08-19T01:41:45.965 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:41:45 smithi167 ceph-mon[21875]: pgmap v546: 97 pgs: 97 active+clean; 546 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 1.9 MiB/s wr, 2.22k op/s 2024-08-19T01:41:46.154 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:41:45 smithi083 ceph-mon[17120]: pgmap v546: 97 pgs: 97 active+clean; 546 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 1.9 MiB/s wr, 2.22k op/s 2024-08-19T01:41:47.905 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:41:47 smithi083 ceph-mon[17120]: pgmap v547: 97 pgs: 97 active+clean; 539 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 1.6 MiB/s wr, 1.46k op/s 2024-08-19T01:41:47.905 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:41:47 smithi083 ceph-mon[17120]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-19T01:41:47.918 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:41:47.917+0000 7f2e89800640 1 -- 172.21.15.83:0/3257820376 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2e84104c80 msgr2=0x7f2e84105100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:41:47.918 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:41:47.917+0000 7f2e89800640 1 --2- 172.21.15.83:0/3257820376 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2e84104c80 0x7f2e84105100 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f2e70009900 tx=0x7f2e7002a3f0 comp rx=0 tx=0).stop 2024-08-19T01:41:47.918 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:41:47.917+0000 7f2e89800640 1 -- 172.21.15.83:0/3257820376 shutdown_connections 2024-08-19T01:41:47.918 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:41:47.917+0000 7f2e89800640 1 --2- 172.21.15.83:0/3257820376 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2e84105640 0x7f2e8410bed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:41:47.919 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:41:47.917+0000 7f2e89800640 1 --2- 172.21.15.83:0/3257820376 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2e84104c80 0x7f2e84105100 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:41:47.919 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:41:47.917+0000 7f2e89800640 1 --2- 172.21.15.83:0/3257820376 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2e84103a90 0x7f2e84103e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:41:47.919 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:41:47.917+0000 7f2e89800640 1 -- 172.21.15.83:0/3257820376 >> 172.21.15.83:0/3257820376 conn(0x7f2e840ff210 msgr2=0x7f2e84101630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:41:47.919 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:41:47.918+0000 7f2e89800640 1 -- 172.21.15.83:0/3257820376 shutdown_connections 2024-08-19T01:41:47.919 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:41:47.918+0000 7f2e89800640 1 -- 172.21.15.83:0/3257820376 wait complete. 2024-08-19T01:41:47.920 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:41:47.919+0000 7f2e89800640 1 Processor -- start 2024-08-19T01:41:47.920 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:41:47.919+0000 7f2e89800640 1 -- start start 2024-08-19T01:41:47.921 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:41:47.919+0000 7f2e89800640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2e84103a90 0x7f2e84199d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:41:47.921 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:41:47.919+0000 7f2e89800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2e84104c80 0x7f2e8419a260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:41:47.921 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:41:47.919+0000 7f2e89800640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2e84105640 0x7f2e841a1290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:41:47.921 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:41:47.919+0000 7f2e89800640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f2e8410f760 con 0x7f2e84104c80 2024-08-19T01:41:47.921 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:41:47.919+0000 7f2e89800640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f2e8410f5e0 con 0x7f2e84105640 2024-08-19T01:41:47.921 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:41:47.919+0000 7f2e89800640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f2e8410f8e0 con 0x7f2e84103a90 2024-08-19T01:41:47.921 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:41:47.920+0000 7f2e82a00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2e84103a90 0x7f2e84199d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:41:47.921 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:41:47.920+0000 7f2e83400640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2e84105640 0x7f2e841a1290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:41:47.921 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:41:47.920+0000 7f2e82a00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2e84103a90 0x7f2e84199d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.83:51308/0 (socket says 172.21.15.83:51308) 2024-08-19T01:41:47.921 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:41:47.920+0000 7f2e82a00640 1 -- 172.21.15.83:0/1179753969 learned_addr learned my addr 172.21.15.83:0/1179753969 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-08-19T01:41:47.921 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:41:47.920+0000 7f2e82000640 1 --2- 172.21.15.83:0/1179753969 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2e84104c80 0x7f2e8419a260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:41:47.922 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:41:47.920+0000 7f2e83400640 1 -- 172.21.15.83:0/1179753969 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2e84103a90 msgr2=0x7f2e84199d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:41:47.922 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:41:47.920+0000 7f2e83400640 1 --2- 172.21.15.83:0/1179753969 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2e84103a90 0x7f2e84199d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:41:47.922 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:41:47.920+0000 7f2e83400640 1 -- 172.21.15.83:0/1179753969 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2e84104c80 msgr2=0x7f2e8419a260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:41:47.922 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:41:47.920+0000 7f2e83400640 1 --2- 172.21.15.83:0/1179753969 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2e84104c80 0x7f2e8419a260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:41:47.922 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:41:47.920+0000 7f2e83400640 1 -- 172.21.15.83:0/1179753969 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2e841a1930 con 0x7f2e84105640 2024-08-19T01:41:47.923 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:41:47.920+0000 7f2e83400640 1 --2- 172.21.15.83:0/1179753969 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2e84105640 0x7f2e841a1290 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f2e7c009fd0 tx=0x7f2e7c00ef10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:41:47.923 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:41:47.921+0000 7f2e7b400640 1 -- 172.21.15.83:0/1179753969 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e7c00ce20 con 0x7f2e84105640 2024-08-19T01:41:47.923 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:41:47.921+0000 7f2e7b400640 1 -- 172.21.15.83:0/1179753969 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2e7c005020 con 0x7f2e84105640 2024-08-19T01:41:47.923 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:41:47.921+0000 7f2e7b400640 1 -- 172.21.15.83:0/1179753969 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e7c010720 con 0x7f2e84105640 2024-08-19T01:41:47.923 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:41:47.921+0000 7f2e89800640 1 -- 172.21.15.83:0/1179753969 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2e841a1c20 con 0x7f2e84105640 2024-08-19T01:41:47.923 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:41:47.921+0000 7f2e89800640 1 -- 172.21.15.83:0/1179753969 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2e841a2120 con 0x7f2e84105640 2024-08-19T01:41:47.925 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:41:47.925+0000 7f2e7b400640 1 -- 172.21.15.83:0/1179753969 <== mon.2 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7f2e7c005540 con 0x7f2e84105640 2024-08-19T01:41:47.926 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:41:47.926+0000 7f2e8c5e3740 1 Processor -- start 2024-08-19T01:41:47.926 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:41:47.926+0000 7f2e8c5e3740 1 -- start start 2024-08-19T01:41:47.933 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:41:47.926+0000 7f2e8c5e3740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e5ee32ec80 0x55e5ee32f080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:41:47.933 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:41:47.926+0000 7f2e8c5e3740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e5ee32f5c0 0x55e5ee334080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:41:47.933 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:41:47.926+0000 7f2e8c5e3740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e5ee334650 0x55e5ee336b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:41:47.933 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:41:47.926+0000 7f2e8c5e3740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55e5ee28a890 con 0x55e5ee334650 2024-08-19T01:41:47.934 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:41:47.926+0000 7f2e8c5e3740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55e5ee222d90 con 0x55e5ee32ec80 2024-08-19T01:41:47.934 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:41:47.926+0000 7f2e8c5e3740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55e5ee286850 con 0x55e5ee32f5c0 2024-08-19T01:41:47.934 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:41:47.927+0000 7f2e7b400640 1 --2- 172.21.15.83:0/1179753969 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f2e680a2cb0 0x7f2e680a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:41:47.934 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:41:47.927+0000 7f2e82000640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e5ee32ec80 0x55e5ee32f080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:41:47.934 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:41:47.927+0000 7f2e82000640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e5ee32ec80 0x55e5ee32f080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.83:33048/0 (socket says 172.21.15.83:33048) 2024-08-19T01:41:47.934 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:41:47.927+0000 7f2e82000640 1 -- 172.21.15.83:0/1856069577 learned_addr learned my addr 172.21.15.83:0/1856069577 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-08-19T01:41:47.934 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:41:47.927+0000 7f2e82a00640 1 --2- 172.21.15.83:0/1856069577 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e5ee32f5c0 0x55e5ee334080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:41:47.934 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:41:47.927+0000 7f2e82000640 1 --2- 172.21.15.83:0/1856069577 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e5ee334650 0x55e5ee336b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:41:47.934 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:41:47.927+0000 7f2e7b400640 1 -- 172.21.15.83:0/1179753969 <== mon.2 v2:172.21.15.117:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8269+0+0 (secure 0 0 0) 0x7f2e7c0cf770 con 0x7f2e84105640 2024-08-19T01:41:47.934 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:41:47.927+0000 7f2e83400640 1 --2- 172.21.15.83:0/1179753969 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f2e680a2cb0 0x7f2e680a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:41:47.934 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:41:47.927+0000 7f2e82a00640 1 -- 172.21.15.83:0/1856069577 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e5ee32ec80 msgr2=0x55e5ee32f080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:41:47.934 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:41:47.927+0000 7f2e82a00640 1 --2- 172.21.15.83:0/1856069577 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e5ee32ec80 0x55e5ee32f080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:41:47.935 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:41:47.927+0000 7f2e82a00640 1 -- 172.21.15.83:0/1856069577 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e5ee334650 msgr2=0x55e5ee336b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:41:47.935 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:41:47.927+0000 7f2e82a00640 1 --2- 172.21.15.83:0/1856069577 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e5ee334650 0x55e5ee336b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:41:47.935 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:41:47.927+0000 7f2e82a00640 1 -- 172.21.15.83:0/1856069577 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e5edf9e210 con 0x55e5ee32f5c0 2024-08-19T01:41:47.935 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:41:47.927+0000 7f2e82000640 1 --2- 172.21.15.83:0/1856069577 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e5ee32ec80 0x55e5ee32f080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-19T01:41:47.935 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:41:47.928+0000 7f2e82a00640 1 --2- 172.21.15.83:0/1856069577 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e5ee32f5c0 0x55e5ee334080 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f2e8419b1f0 tx=0x7f2e74011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a413f100a219a123 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:41:47.935 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:41:47.928+0000 7f2e6fe00640 1 -- 172.21.15.83:0/1856069577 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e74004230 con 0x55e5ee32f5c0 2024-08-19T01:41:47.935 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:41:47.928+0000 7f2e6fe00640 1 -- 172.21.15.83:0/1856069577 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2e740043d0 con 0x55e5ee32f5c0 2024-08-19T01:41:47.935 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:41:47.928+0000 7f2e6fe00640 1 -- 172.21.15.83:0/1856069577 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e74004e00 con 0x55e5ee32f5c0 2024-08-19T01:41:47.935 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:41:47.929+0000 7f2e8c5e3740 1 -- 172.21.15.83:0/1856069577 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e5ee32f5c0 msgr2=0x55e5ee334080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:41:47.935 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:41:47.929+0000 7f2e83400640 1 --2- 172.21.15.83:0/1179753969 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f2e680a2cb0 0x7f2e680a5170 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f2e7c098380 tx=0x7f2e7c005940 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:41:47.935 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:41:47.929+0000 7f2e8c5e3740 1 --2- 172.21.15.83:0/1856069577 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e5ee32f5c0 0x55e5ee334080 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f2e8419b1f0 tx=0x7f2e74011810 comp rx=0 tx=0).stop 2024-08-19T01:41:47.935 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:41:47.929+0000 7f2e8c5e3740 1 -- 172.21.15.83:0/1856069577 shutdown_connections 2024-08-19T01:41:47.935 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:41:47.930+0000 7f2e8c5e3740 1 --2- 172.21.15.83:0/1856069577 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e5ee334650 0x55e5ee336b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:41:47.935 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:41:47.930+0000 7f2e8c5e3740 1 --2- 172.21.15.83:0/1856069577 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e5ee32f5c0 0x55e5ee334080 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:41:47.936 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:41:47.930+0000 7f2e8c5e3740 1 --2- 172.21.15.83:0/1856069577 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e5ee32ec80 0x55e5ee32f080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:41:47.936 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:41:47.930+0000 7f2e8c5e3740 1 -- 172.21.15.83:0/1856069577 >> 172.21.15.83:0/1856069577 conn(0x55e5ee321ae0 msgr2=0x55e5ee321ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:41:47.936 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:41:47.930+0000 7f2e8c5e3740 1 -- 172.21.15.83:0/1856069577 shutdown_connections 2024-08-19T01:41:47.936 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:41:47.930+0000 7f2e8c5e3740 1 -- 172.21.15.83:0/1856069577 wait complete. 2024-08-19T01:41:47.936 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:41:47.930+0000 7f2e8c5e3740 1 -- start start 2024-08-19T01:41:47.936 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:41:47.931+0000 7f2e8c5e3740 1 Processor -- start 2024-08-19T01:41:47.936 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:41:47.931+0000 7f2e8c5e3740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e5ee32ec80 0x55e5ee338f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:41:47.936 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:41:47.931+0000 7f2e8c5e3740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e5ee32f5c0 0x55e5ee339480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:41:47.936 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:41:47.931+0000 7f2e8c5e3740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e5ee334650 0x55e5ee3404c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:41:47.936 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:41:47.931+0000 7f2e8c5e3740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55e5ee3376c0 con 0x55e5ee32ec80 2024-08-19T01:41:47.936 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:41:47.931+0000 7f2e8c5e3740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55e5ee245f70 con 0x55e5ee32f5c0 2024-08-19T01:41:47.936 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:41:47.931+0000 7f2e8c5e3740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55e5ee337840 con 0x55e5ee334650 2024-08-19T01:41:47.937 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:41:47.932+0000 7f2e82000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e5ee32ec80 0x55e5ee338f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:41:47.937 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:41:47.932+0000 7f2e82000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e5ee32ec80 0x55e5ee338f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:48572/0 (socket says 172.21.15.83:48572) 2024-08-19T01:41:47.937 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:41:47.932+0000 7f2e82000640 1 -- 172.21.15.83:0/358650228 learned_addr learned my addr 172.21.15.83:0/358650228 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-08-19T01:41:47.937 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:41:47.932+0000 7f2e82000640 1 --2- 172.21.15.83:0/358650228 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e5ee334650 0x55e5ee3404c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:41:47.937 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:41:47.932+0000 7f2e8c5e3740 10 client.0 _ll_register_callbacks cb 0x55e5ee2828d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-19T01:41:47.937 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:41:47.932+0000 7f2e82000640 1 -- 172.21.15.83:0/358650228 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e5ee32f5c0 msgr2=0x55e5ee339480 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-08-19T01:41:47.937 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:41:47.932+0000 7f2e82a00640 1 --2- 172.21.15.83:0/358650228 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e5ee32f5c0 0x55e5ee339480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:41:47.937 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:41:47.932+0000 7f2e82000640 1 --2- 172.21.15.83:0/358650228 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e5ee32f5c0 0x55e5ee339480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:41:47.937 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:41:47.932+0000 7f2e82000640 1 -- 172.21.15.83:0/358650228 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e5ee32ec80 msgr2=0x55e5ee338f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:41:47.937 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:41:47.932+0000 7f2e82000640 1 --2- 172.21.15.83:0/358650228 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e5ee32ec80 0x55e5ee338f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:41:47.937 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:41:47.932+0000 7f2e82000640 1 -- 172.21.15.83:0/358650228 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e5ee06a620 con 0x55e5ee334650 2024-08-19T01:41:47.937 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:41:47.933+0000 7f2e82a00640 1 --2- 172.21.15.83:0/358650228 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e5ee32f5c0 0x55e5ee339480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-19T01:41:47.937 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:41:47.933+0000 7f2e82000640 1 --2- 172.21.15.83:0/358650228 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e5ee334650 0x55e5ee3404c0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x55e5ee338470 tx=0x7f2e70009260 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bfaaa3d8b92e3985 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:41:47.938 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:41:47.933+0000 7f2e6cc00640 10 client.0 ms_handle_connect on v2:172.21.15.167:3300/0 2024-08-19T01:41:47.940 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:41:47.933+0000 7f2e6cc00640 1 -- 172.21.15.83:0/358650228 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e70035030 con 0x55e5ee334650 2024-08-19T01:41:47.940 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:41:47.933+0000 7f2e6cc00640 1 -- 172.21.15.83:0/358650228 <== mon.1 v2:172.21.15.167:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8269+0+0 (secure 0 0 0) 0x7f2e70031a20 con 0x55e5ee334650 2024-08-19T01:41:47.940 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:41:47.934+0000 7f2e8c5e3740 1 -- 172.21.15.83:0/358650228 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2e740043d0 con 0x55e5ee334650 2024-08-19T01:41:47.940 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:41:47.934+0000 7f2e6cc00640 1 -- 172.21.15.83:0/358650228 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f2e70031c20 con 0x55e5ee334650 2024-08-19T01:41:47.940 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:41:47.934+0000 7f2e8c5e3740 10 client.25888 fetch_fsmap learned FSMap version 28 2024-08-19T01:41:47.940 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:41:47.934+0000 7f2e8c5e3740 1 -- 172.21.15.83:0/358650228 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55e5ee3436e0 con 0x55e5ee334650 2024-08-19T01:41:47.940 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:41:47.935+0000 7f2e6cc00640 1 -- 172.21.15.83:0/358650228 <== mon.1 v2:172.21.15.167:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2e740043d0 con 0x55e5ee334650 2024-08-19T01:41:47.940 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:41:47.935+0000 7f2e6cc00640 1 -- 172.21.15.83:0/358650228 <== mon.1 v2:172.21.15.167:3300/0 5 ==== fsmap(e 28) ==== 8358+0+0 (secure 0 0 0) 0x7f2e70049020 con 0x55e5ee334650 2024-08-19T01:41:47.940 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:41:47.935+0000 7f2e8c5e3740 10 client.25888 fetch_fsmap finished waiting for FSMap version 28 2024-08-19T01:41:47.940 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:41:47.935+0000 7f2e8c5e3740 10 client.25888 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-08-19T01:41:47.940 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:41:47.935+0000 7f2e8c5e3740 20 client.25888 populate_metadata read hostname 'smithi083' 2024-08-19T01:41:47.941 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:41:47.935+0000 7f2e8c5e3740 1 --2- 172.21.15.83:0/358650228 >> [v2:172.21.15.117:6838/1967595704,v1:172.21.15.117:6839/1967595704] conn(0x55e5ee344de0 0x55e5ee347280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:41:47.941 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:41:47.935+0000 7f2e8c5e3740 4 client.25888 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-19T01:41:47.941 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:41:47.935+0000 7f2e8c5e3740 1 -- 172.21.15.83:0/358650228 --> [v2:172.21.15.117:6838/1967595704,v1:172.21.15.117:6839/1967595704] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e5ee297fc0 con 0x55e5ee344de0 2024-08-19T01:41:47.941 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:41:47.936+0000 7f2e82a00640 1 --2- 172.21.15.83:0/358650228 >> [v2:172.21.15.117:6838/1967595704,v1:172.21.15.117:6839/1967595704] conn(0x55e5ee344de0 0x55e5ee347280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:41:47.941 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:41:47.936+0000 7f2e82a00640 1 --2- 172.21.15.83:0/358650228 >> [v2:172.21.15.117:6838/1967595704,v1:172.21.15.117:6839/1967595704] conn(0x55e5ee344de0 0x55e5ee347280 crc :-1 s=READY pgs=119 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=710839a4e1e1d4e2 server_cookie=ae1f6397e50c51bc in_seq=0 out_seq=0 2024-08-19T01:41:47.941 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:41:47.936+0000 7f2e6cc00640 10 client.25888 ms_handle_connect on v2:172.21.15.117:6838/1967595704 2024-08-19T01:41:47.941 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:41:47.937+0000 7f2e6cc00640 1 -- 172.21.15.83:0/358650228 <== mds.0 v2:172.21.15.117:6838/1967595704 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55e5ee297fc0 con 0x55e5ee344de0 2024-08-19T01:41:47.941 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:41:47.937+0000 7f2e6cc00640 10 client.25888 handle_command_reply: tid=0 2024-08-19T01:41:47.941 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:41:47.937+0000 7f2e8c5e3740 1 client.25888 shutdown 2024-08-19T01:41:47.941 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:41:47.937+0000 7f2e8c5e3740 1 -- 172.21.15.83:0/358650228 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e5ee334650 msgr2=0x55e5ee3404c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:41:47.941 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:41:47.937+0000 7f2e8c5e3740 1 --2- 172.21.15.83:0/358650228 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e5ee334650 0x55e5ee3404c0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x55e5ee338470 tx=0x7f2e70009260 comp rx=0 tx=0).stop 2024-08-19T01:41:47.941 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:41:47.938+0000 7f2e8c5e3740 1 -- 172.21.15.83:0/358650228 shutdown_connections 2024-08-19T01:41:47.941 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:41:47.938+0000 7f2e8c5e3740 1 --2- 172.21.15.83:0/358650228 >> [v2:172.21.15.117:6838/1967595704,v1:172.21.15.117:6839/1967595704] conn(0x55e5ee344de0 0x55e5ee347280 crc :-1 s=READY pgs=119 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:41:47.942 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:41:47.938+0000 7f2e8c5e3740 1 --2- 172.21.15.83:0/358650228 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e5ee334650 0x55e5ee3404c0 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:41:47.942 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:41:47.938+0000 7f2e8c5e3740 1 --2- 172.21.15.83:0/358650228 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e5ee32f5c0 0x55e5ee339480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:41:47.942 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:41:47.938+0000 7f2e6cc00640 0 client.25888 ms_handle_reset on v2:172.21.15.117:6838/1967595704 2024-08-19T01:41:47.942 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:41:47.938+0000 7f2e8c5e3740 1 --2- 172.21.15.83:0/358650228 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e5ee32ec80 0x55e5ee338f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:41:47.942 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:41:47.938+0000 7f2e8c5e3740 1 -- 172.21.15.83:0/358650228 >> 172.21.15.83:0/358650228 conn(0x55e5ee321ae0 msgr2=0x55e5ee3232f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:41:47.942 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:41:47.938+0000 7f2e8c5e3740 1 -- 172.21.15.83:0/358650228 shutdown_connections 2024-08-19T01:41:47.942 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:41:47.938+0000 7f2e8c5e3740 1 -- 172.21.15.83:0/358650228 wait complete. 2024-08-19T01:41:47.942 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:41:47.938+0000 7f2e8c5e3740 20 client.25888 trim_cache size 0 max 16384 2024-08-19T01:41:47.952 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:41:47 smithi117 ceph-mon[22016]: pgmap v547: 97 pgs: 97 active+clean; 539 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 1.6 MiB/s wr, 1.46k op/s 2024-08-19T01:41:47.952 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:41:47 smithi117 ceph-mon[22016]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-19T01:41:47.953 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:41:47.953+0000 7f2e8c5e3740 1 Processor -- start 2024-08-19T01:41:47.954 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:41:47.953+0000 7f2e8c5e3740 1 -- start start 2024-08-19T01:41:47.954 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:41:47.953+0000 7f2e8c5e3740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e5ee32ec80 0x55e5ee337c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:41:47.954 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:41:47.953+0000 7f2e8c5e3740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e5ee32f5c0 0x55e5ee340480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:41:47.954 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:41:47.953+0000 7f2e8c5e3740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e5ee334650 0x55e5ee322990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:41:47.954 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:41:47.953+0000 7f2e8c5e3740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f2e841a93c0 con 0x55e5ee32f5c0 2024-08-19T01:41:47.954 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:41:47.953+0000 7f2e8c5e3740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f2e841a6c60 con 0x55e5ee32ec80 2024-08-19T01:41:47.954 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:41:47.953+0000 7f2e8c5e3740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f2e841a4ad0 con 0x55e5ee334650 2024-08-19T01:41:47.956 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:41:47.954+0000 7f2e82000640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e5ee32ec80 0x55e5ee337c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:41:47.956 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:41:47.954+0000 7f2e82000640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e5ee32ec80 0x55e5ee337c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.83:33064/0 (socket says 172.21.15.83:33064) 2024-08-19T01:41:47.956 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:41:47.954+0000 7f2e82000640 1 -- 172.21.15.83:0/2754822582 learned_addr learned my addr 172.21.15.83:0/2754822582 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-08-19T01:41:47.957 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:41:47.954+0000 7f2e82a00640 1 --2- 172.21.15.83:0/2754822582 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e5ee32f5c0 0x55e5ee340480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:41:47.957 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:41:47.954+0000 7f2e82000640 1 --2- 172.21.15.83:0/2754822582 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e5ee334650 0x55e5ee322990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:41:47.957 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:41:47.954+0000 7f2e82a00640 1 -- 172.21.15.83:0/2754822582 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e5ee334650 msgr2=0x55e5ee322990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:41:47.957 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:41:47.954+0000 7f2e82a00640 1 --2- 172.21.15.83:0/2754822582 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e5ee334650 0x55e5ee322990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:41:47.957 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:41:47.954+0000 7f2e82a00640 1 -- 172.21.15.83:0/2754822582 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e5ee32ec80 msgr2=0x55e5ee337c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:41:47.957 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:41:47.954+0000 7f2e82a00640 1 --2- 172.21.15.83:0/2754822582 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e5ee32ec80 0x55e5ee337c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:41:47.957 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:41:47.954+0000 7f2e82a00640 1 -- 172.21.15.83:0/2754822582 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e5ee347900 con 0x55e5ee32f5c0 2024-08-19T01:41:47.957 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:41:47.954+0000 7f2e82000640 1 --2- 172.21.15.83:0/2754822582 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e5ee32ec80 0x55e5ee337c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-19T01:41:47.957 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:41:47.954+0000 7f2e82a00640 1 --2- 172.21.15.83:0/2754822582 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e5ee32f5c0 0x55e5ee340480 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x55e5ee33a420 tx=0x7f2e740045c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=14c15ff77401bafc server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:41:47.957 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:41:47.955+0000 7f2e6f400640 1 -- 172.21.15.83:0/2754822582 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e74029070 con 0x55e5ee32f5c0 2024-08-19T01:41:47.957 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:41:47.955+0000 7f2e6f400640 1 -- 172.21.15.83:0/2754822582 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2e74019990 con 0x55e5ee32f5c0 2024-08-19T01:41:47.958 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:41:47.955+0000 7f2e6f400640 1 -- 172.21.15.83:0/2754822582 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e74025690 con 0x55e5ee32f5c0 2024-08-19T01:41:47.958 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:41:47.955+0000 7f2e8c5e3740 1 -- 172.21.15.83:0/2754822582 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e5ee32f5c0 msgr2=0x55e5ee340480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:41:47.958 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:41:47.955+0000 7f2e8c5e3740 1 --2- 172.21.15.83:0/2754822582 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e5ee32f5c0 0x55e5ee340480 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x55e5ee33a420 tx=0x7f2e740045c0 comp rx=0 tx=0).stop 2024-08-19T01:41:47.958 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:41:47.955+0000 7f2e8c5e3740 1 -- 172.21.15.83:0/2754822582 shutdown_connections 2024-08-19T01:41:47.958 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:41:47.956+0000 7f2e8c5e3740 1 --2- 172.21.15.83:0/2754822582 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e5ee334650 0x55e5ee322990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:41:47.958 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:41:47.956+0000 7f2e8c5e3740 1 --2- 172.21.15.83:0/2754822582 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e5ee32f5c0 0x55e5ee340480 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:41:47.958 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:41:47.956+0000 7f2e8c5e3740 1 --2- 172.21.15.83:0/2754822582 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e5ee32ec80 0x55e5ee337c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:41:47.958 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:41:47.956+0000 7f2e8c5e3740 1 -- 172.21.15.83:0/2754822582 >> 172.21.15.83:0/2754822582 conn(0x55e5ee321ae0 msgr2=0x55e5ee330760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:41:47.958 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:41:47.956+0000 7f2e8c5e3740 1 -- 172.21.15.83:0/2754822582 shutdown_connections 2024-08-19T01:41:47.958 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:41:47.956+0000 7f2e8c5e3740 1 -- 172.21.15.83:0/2754822582 wait complete. 2024-08-19T01:41:47.958 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:41:47.956+0000 7f2e8c5e3740 1 -- start start 2024-08-19T01:41:47.961 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:41:47.956+0000 7f2e8c5e3740 1 Processor -- start 2024-08-19T01:41:47.961 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:41:47.957+0000 7f2e8c5e3740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e5ee32ec80 0x55e5ee33d8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:41:47.961 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:41:47.957+0000 7f2e8c5e3740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e5ee32f5c0 0x55e5ee33dde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:41:47.961 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:41:47.957+0000 7f2e8c5e3740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e5ee334650 0x55e5ee33edf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:41:47.961 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:41:47.957+0000 7f2e8c5e3740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f2e841a5120 con 0x55e5ee334650 2024-08-19T01:41:47.961 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:41:47.957+0000 7f2e8c5e3740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f2e841a6590 con 0x55e5ee32ec80 2024-08-19T01:41:47.961 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:41:47.957+0000 7f2e8c5e3740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f2e841a57f0 con 0x55e5ee32f5c0 2024-08-19T01:41:47.961 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:41:47.957+0000 7f2e8c5e3740 10 client.0 _ll_register_callbacks cb 0x7f2e70009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-19T01:41:47.961 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:41:47.957+0000 7f2e82000640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e5ee32ec80 0x55e5ee33d8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:41:47.961 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:41:47.957+0000 7f2e82000640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e5ee32ec80 0x55e5ee33d8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.83:33078/0 (socket says 172.21.15.83:33078) 2024-08-19T01:41:47.962 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:41:47.957+0000 7f2e82000640 1 -- 172.21.15.83:0/2955357518 learned_addr learned my addr 172.21.15.83:0/2955357518 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-08-19T01:41:47.962 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:41:47.957+0000 7f2e82a00640 1 --2- 172.21.15.83:0/2955357518 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e5ee32f5c0 0x55e5ee33dde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:41:47.962 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:41:47.957+0000 7f2e82000640 1 --2- 172.21.15.83:0/2955357518 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e5ee334650 0x55e5ee33edf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:41:47.962 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:41:47.958+0000 7f2e82a00640 1 -- 172.21.15.83:0/2955357518 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e5ee32ec80 msgr2=0x55e5ee33d8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:41:47.962 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:41:47.958+0000 7f2e82a00640 1 --2- 172.21.15.83:0/2955357518 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e5ee32ec80 0x55e5ee33d8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:41:47.962 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:41:47.958+0000 7f2e82a00640 1 -- 172.21.15.83:0/2955357518 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e5ee334650 msgr2=0x55e5ee33edf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:41:47.962 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:41:47.958+0000 7f2e82a00640 1 --2- 172.21.15.83:0/2955357518 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e5ee334650 0x55e5ee33edf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:41:47.962 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:41:47.958+0000 7f2e82a00640 1 -- 172.21.15.83:0/2955357518 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e5ee33f400 con 0x55e5ee32f5c0 2024-08-19T01:41:47.962 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:41:47.958+0000 7f2e82000640 1 --2- 172.21.15.83:0/2955357518 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e5ee32ec80 0x55e5ee33d8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-19T01:41:47.962 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:41:47.958+0000 7f2e82a00640 1 --2- 172.21.15.83:0/2955357518 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e5ee32f5c0 0x55e5ee33dde0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f2e740058c0 tx=0x7f2e74011da0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4727934651ceb3d1 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:41:47.962 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:41:47.958+0000 7f2e6cc00640 10 client.0 ms_handle_connect on v2:172.21.15.167:3300/0 2024-08-19T01:41:47.962 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:41:47.958+0000 7f2e6cc00640 1 -- 172.21.15.83:0/2955357518 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e74029070 con 0x55e5ee32f5c0 2024-08-19T01:41:47.962 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:41:47.958+0000 7f2e8c5e3740 1 -- 172.21.15.83:0/2955357518 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2e74019990 con 0x55e5ee32f5c0 2024-08-19T01:41:47.963 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:41:47.958+0000 7f2e6cc00640 1 -- 172.21.15.83:0/2955357518 <== mon.1 v2:172.21.15.167:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8269+0+0 (secure 0 0 0) 0x7f2e740194b0 con 0x55e5ee32f5c0 2024-08-19T01:41:47.963 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:41:47.958+0000 7f2e6cc00640 1 -- 172.21.15.83:0/2955357518 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f2e74019e40 con 0x55e5ee32f5c0 2024-08-19T01:41:47.963 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:41:47.958+0000 7f2e8c5e3740 10 client.25894 fetch_fsmap learned FSMap version 28 2024-08-19T01:41:47.963 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:41:47.958+0000 7f2e8c5e3740 1 -- 172.21.15.83:0/2955357518 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55e5ee340710 con 0x55e5ee32f5c0 2024-08-19T01:41:47.963 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:41:47.959+0000 7f2e6cc00640 1 -- 172.21.15.83:0/2955357518 <== mon.1 v2:172.21.15.167:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2e74019990 con 0x55e5ee32f5c0 2024-08-19T01:41:47.963 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:41:47.959+0000 7f2e6cc00640 1 -- 172.21.15.83:0/2955357518 <== mon.1 v2:172.21.15.167:3300/0 5 ==== fsmap(e 28) ==== 8358+0+0 (secure 0 0 0) 0x7f2e74025430 con 0x55e5ee32f5c0 2024-08-19T01:41:47.963 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:41:47.959+0000 7f2e8c5e3740 10 client.25894 fetch_fsmap finished waiting for FSMap version 28 2024-08-19T01:41:47.963 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:41:47.959+0000 7f2e8c5e3740 10 client.25894 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-08-19T01:41:47.963 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:41:47.959+0000 7f2e8c5e3740 20 client.25894 populate_metadata read hostname 'smithi083' 2024-08-19T01:41:47.963 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:41:47.959+0000 7f2e8c5e3740 1 --2- 172.21.15.83:0/2955357518 >> [v2:172.21.15.117:6838/1967595704,v1:172.21.15.117:6839/1967595704] conn(0x55e5ee3b0080 0x55e5ee3b24a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:41:47.963 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:41:47.959+0000 7f2e8c5e3740 4 client.25894 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-08-19T01:41:47.963 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:41:47.959+0000 7f2e8c5e3740 1 -- 172.21.15.83:0/2955357518 --> [v2:172.21.15.117:6838/1967595704,v1:172.21.15.117:6839/1967595704] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55e5ee292310 con 0x55e5ee3b0080 2024-08-19T01:41:47.963 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:41:47.959+0000 7f2e82000640 1 --2- 172.21.15.83:0/2955357518 >> [v2:172.21.15.117:6838/1967595704,v1:172.21.15.117:6839/1967595704] conn(0x55e5ee3b0080 0x55e5ee3b24a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:41:47.963 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:41:47.960+0000 7f2e6cc00640 10 client.25894 ms_handle_connect on v2:172.21.15.117:6838/1967595704 2024-08-19T01:41:47.964 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:41:47.960+0000 7f2e82000640 1 --2- 172.21.15.83:0/2955357518 >> [v2:172.21.15.117:6838/1967595704,v1:172.21.15.117:6839/1967595704] conn(0x55e5ee3b0080 0x55e5ee3b24a0 crc :-1 s=READY pgs=120 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e60b0ee827d6c52a server_cookie=a79da5ee9f7dbb90 in_seq=0 out_seq=0 2024-08-19T01:41:47.965 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:41:47 smithi167 ceph-mon[21875]: pgmap v547: 97 pgs: 97 active+clean; 539 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 1.6 MiB/s wr, 1.46k op/s 2024-08-19T01:41:47.966 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:41:47 smithi167 ceph-mon[21875]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-19T01:41:48.720 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833 shell --fsid bad8780e-5dc9-11ef-bccf-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-08-19T01:41:48.906 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:41:48.905+0000 7f2e6cc00640 1 -- 172.21.15.83:0/2955357518 <== mds.0 v2:172.21.15.117:6838/1967595704 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55e5ee292310 con 0x55e5ee3b0080 2024-08-19T01:41:48.906 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:41:48.905+0000 7f2e6cc00640 10 client.25894 handle_command_reply: tid=0 2024-08-19T01:41:48.907 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:41:48.905+0000 7f2e8c5e3740 1 client.25894 shutdown 2024-08-19T01:41:48.907 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:41:48.905+0000 7f2e8c5e3740 1 -- 172.21.15.83:0/2955357518 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e5ee32f5c0 msgr2=0x55e5ee33dde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:41:48.907 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:41:48.905+0000 7f2e8c5e3740 1 --2- 172.21.15.83:0/2955357518 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e5ee32f5c0 0x55e5ee33dde0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f2e740058c0 tx=0x7f2e74011da0 comp rx=0 tx=0).stop 2024-08-19T01:41:48.907 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:41:48.905+0000 7f2e8c5e3740 1 -- 172.21.15.83:0/2955357518 shutdown_connections 2024-08-19T01:41:48.907 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:41:48.905+0000 7f2e8c5e3740 1 --2- 172.21.15.83:0/2955357518 >> [v2:172.21.15.117:6838/1967595704,v1:172.21.15.117:6839/1967595704] conn(0x55e5ee3b0080 0x55e5ee3b24a0 crc :-1 s=READY pgs=120 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:41:48.907 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:41:48.906+0000 7f2e8c5e3740 1 --2- 172.21.15.83:0/2955357518 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e5ee334650 0x55e5ee33edf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:41:48.907 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:41:48.906+0000 7f2e8c5e3740 1 --2- 172.21.15.83:0/2955357518 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e5ee32f5c0 0x55e5ee33dde0 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:41:48.907 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:41:48.906+0000 7f2e8c5e3740 1 --2- 172.21.15.83:0/2955357518 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e5ee32ec80 0x55e5ee33d8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:41:48.907 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:41:48.906+0000 7f2e8c5e3740 1 -- 172.21.15.83:0/2955357518 >> 172.21.15.83:0/2955357518 conn(0x55e5ee321ae0 msgr2=0x55e5ee3204d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:41:48.907 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:41:48.906+0000 7f2e6cc00640 0 client.25894 ms_handle_reset on v2:172.21.15.117:6838/1967595704 2024-08-19T01:41:48.907 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:41:48.906+0000 7f2e8c5e3740 1 -- 172.21.15.83:0/2955357518 shutdown_connections 2024-08-19T01:41:48.907 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:41:48.906+0000 7f2e8c5e3740 1 -- 172.21.15.83:0/2955357518 wait complete. 2024-08-19T01:41:48.908 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:41:48.906+0000 7f2e8c5e3740 20 client.25894 trim_cache size 0 max 16384 2024-08-19T01:41:48.908 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stdout:{ 2024-08-19T01:41:48.908 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stdout: "message": "", 2024-08-19T01:41:48.908 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stdout: "return_code": 0 2024-08-19T01:41:48.908 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stdout:} 2024-08-19T01:41:48.909 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:41:48.907+0000 7f2e89800640 1 -- 172.21.15.83:0/1179753969 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f2e680a2cb0 msgr2=0x7f2e680a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:41:48.909 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:41:48.907+0000 7f2e89800640 1 --2- 172.21.15.83:0/1179753969 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f2e680a2cb0 0x7f2e680a5170 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f2e7c098380 tx=0x7f2e7c005940 comp rx=0 tx=0).stop 2024-08-19T01:41:48.909 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:41:48.907+0000 7f2e89800640 1 -- 172.21.15.83:0/1179753969 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2e84105640 msgr2=0x7f2e841a1290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:41:48.909 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:41:48.907+0000 7f2e89800640 1 --2- 172.21.15.83:0/1179753969 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2e84105640 0x7f2e841a1290 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f2e7c009fd0 tx=0x7f2e7c00ef10 comp rx=0 tx=0).stop 2024-08-19T01:41:48.909 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:41:48.907+0000 7f2e89800640 1 -- 172.21.15.83:0/1179753969 shutdown_connections 2024-08-19T01:41:48.909 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:41:48.908+0000 7f2e89800640 1 --2- 172.21.15.83:0/1179753969 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f2e680a2cb0 0x7f2e680a5170 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:41:48.909 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:41:48.908+0000 7f2e89800640 1 --2- 172.21.15.83:0/1179753969 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2e84105640 0x7f2e841a1290 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:41:48.909 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:41:48.908+0000 7f2e89800640 1 --2- 172.21.15.83:0/1179753969 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2e84104c80 0x7f2e8419a260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:41:48.909 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:41:48.908+0000 7f2e89800640 1 --2- 172.21.15.83:0/1179753969 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2e84103a90 0x7f2e84199d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:41:48.909 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:41:48.908+0000 7f2e89800640 1 -- 172.21.15.83:0/1179753969 >> 172.21.15.83:0/1179753969 conn(0x7f2e840ff210 msgr2=0x7f2e84107f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:41:48.910 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:41:48.908+0000 7f2e89800640 1 -- 172.21.15.83:0/1179753969 shutdown_connections 2024-08-19T01:41:48.910 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:41:48.908+0000 7f2e89800640 1 -- 172.21.15.83:0/1179753969 wait complete. 2024-08-19T01:41:48.952 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:41:48 smithi117 ceph-mon[22016]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-08-19T01:41:48.952 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:41:48 smithi117 ceph-mon[22016]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-08-19T01:41:48.953 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:41:48 smithi117 ceph-mon[22016]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-08-19T01:41:48.953 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:41:48 smithi117 ceph-mon[22016]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-08-19T01:41:48.953 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:41:48 smithi117 ceph-mon[22016]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-08-19T01:41:48.953 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:41:48 smithi117 ceph-mon[22016]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-08-19T01:41:48.953 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:41:48 smithi117 ceph-mon[22016]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-08-19T01:41:48.953 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:41:48 smithi117 ceph-mon[22016]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-08-19T01:41:48.953 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:41:48 smithi117 ceph-mon[22016]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-08-19T01:41:48.953 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:41:48 smithi117 ceph-mon[22016]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-08-19T01:41:48.953 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:41:48 smithi117 ceph-mon[22016]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-08-19T01:41:48.953 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:41:48 smithi117 ceph-mon[22016]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-08-19T01:41:48.953 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:41:48 smithi117 ceph-mon[22016]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-19T01:41:48.953 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:41:48 smithi117 ceph-mon[22016]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-19T01:41:48.953 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:41:48 smithi117 ceph-mon[22016]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' 2024-08-19T01:41:48.966 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:41:48 smithi167 ceph-mon[21875]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-08-19T01:41:48.966 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:41:48 smithi167 ceph-mon[21875]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-08-19T01:41:48.966 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:41:48 smithi167 ceph-mon[21875]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-08-19T01:41:48.966 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:41:48 smithi167 ceph-mon[21875]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-08-19T01:41:48.966 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:41:48 smithi167 ceph-mon[21875]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-08-19T01:41:48.966 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:41:48 smithi167 ceph-mon[21875]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-08-19T01:41:48.966 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:41:48 smithi167 ceph-mon[21875]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-08-19T01:41:48.966 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:41:48 smithi167 ceph-mon[21875]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-08-19T01:41:48.966 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:41:48 smithi167 ceph-mon[21875]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-08-19T01:41:48.967 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:41:48 smithi167 ceph-mon[21875]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-08-19T01:41:48.967 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:41:48 smithi167 ceph-mon[21875]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-08-19T01:41:48.967 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:41:48 smithi167 ceph-mon[21875]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-08-19T01:41:48.967 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:41:48 smithi167 ceph-mon[21875]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-19T01:41:48.967 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:41:48 smithi167 ceph-mon[21875]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-19T01:41:48.967 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:41:48 smithi167 ceph-mon[21875]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' 2024-08-19T01:41:49.154 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:41:48 smithi083 ceph-mon[17120]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-08-19T01:41:49.154 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:41:48 smithi083 ceph-mon[17120]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-08-19T01:41:49.155 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:41:48 smithi083 ceph-mon[17120]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-08-19T01:41:49.155 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:41:48 smithi083 ceph-mon[17120]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-08-19T01:41:49.155 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:41:48 smithi083 ceph-mon[17120]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-08-19T01:41:49.155 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:41:48 smithi083 ceph-mon[17120]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-08-19T01:41:49.155 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:41:48 smithi083 ceph-mon[17120]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-08-19T01:41:49.155 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:41:48 smithi083 ceph-mon[17120]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-08-19T01:41:49.155 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:41:48 smithi083 ceph-mon[17120]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-08-19T01:41:49.155 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:41:48 smithi083 ceph-mon[17120]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-08-19T01:41:49.155 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:41:48 smithi083 ceph-mon[17120]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-08-19T01:41:49.155 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:41:48 smithi083 ceph-mon[17120]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-08-19T01:41:49.155 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:41:48 smithi083 ceph-mon[17120]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-19T01:41:49.155 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:41:48 smithi083 ceph-mon[17120]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-19T01:41:49.155 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:41:48 smithi083 ceph-mon[17120]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' 2024-08-19T01:41:49.158 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/bad8780e-5dc9-11ef-bccf-c7b262605968/mon.a/config 2024-08-19T01:41:49.860 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:49.858+0000 7f95eb200640 1 -- 172.21.15.83:0/3284930889 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f95e41042a0 msgr2=0x7f95e41107d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:41:49.860 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:49.858+0000 7f95eb200640 1 --2- 172.21.15.83:0/3284930889 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f95e41042a0 0x7f95e41107d0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f95e0009ca0 tx=0x7f95e002a8c0 comp rx=0 tx=0).stop 2024-08-19T01:41:49.860 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:49.858+0000 7f95eb200640 1 -- 172.21.15.83:0/3284930889 shutdown_connections 2024-08-19T01:41:49.860 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:49.858+0000 7f95eb200640 1 --2- 172.21.15.83:0/3284930889 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f95e41042a0 0x7f95e41107d0 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:41:49.860 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:49.858+0000 7f95eb200640 1 --2- 172.21.15.83:0/3284930889 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f95e4103900 0x7f95e4103d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:41:49.861 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:49.858+0000 7f95eb200640 1 --2- 172.21.15.83:0/3284930889 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f95e4109900 0x7f95e4109ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:41:49.861 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:49.858+0000 7f95eb200640 1 -- 172.21.15.83:0/3284930889 >> 172.21.15.83:0/3284930889 conn(0x7f95e40ff600 msgr2=0x7f95e4101a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:41:49.861 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:49.859+0000 7f95eb200640 1 -- 172.21.15.83:0/3284930889 shutdown_connections 2024-08-19T01:41:49.861 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:49.859+0000 7f95eb200640 1 -- 172.21.15.83:0/3284930889 wait complete. 2024-08-19T01:41:49.861 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:41:49 smithi083 ceph-mon[17120]: Adjusting osd_memory_target on smithi167 to 176.0M 2024-08-19T01:41:49.861 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:41:49 smithi083 ceph-mon[17120]: Unable to set osd_memory_target on smithi167 to 184597504: error parsing value: Value '184597504' is below minimum 939524096 2024-08-19T01:41:49.861 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:41:49 smithi083 ceph-mon[17120]: Adjusting osd_memory_target on smithi083 to 176.4M 2024-08-19T01:41:49.861 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:41:49 smithi083 ceph-mon[17120]: Unable to set osd_memory_target on smithi083 to 185035468: error parsing value: Value '185035468' is below minimum 939524096 2024-08-19T01:41:49.861 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:41:49 smithi083 ceph-mon[17120]: Adjusting osd_memory_target on smithi117 to 176.0M 2024-08-19T01:41:49.862 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:41:49 smithi083 ceph-mon[17120]: Unable to set osd_memory_target on smithi117 to 184592486: error parsing value: Value '184592486' is below minimum 939524096 2024-08-19T01:41:49.862 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:41:49 smithi083 ceph-mon[17120]: pgmap v548: 97 pgs: 97 active+clean; 528 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 2.1 MiB/s wr, 1.31k op/s 2024-08-19T01:41:49.864 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:49.860+0000 7f95eb200640 1 Processor -- start 2024-08-19T01:41:49.864 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:49.860+0000 7f95eb200640 1 -- start start 2024-08-19T01:41:49.864 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:49.860+0000 7f95eb200640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f95e4103900 0x7f95e4073640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:41:49.864 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:49.860+0000 7f95eb200640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f95e41042a0 0x7f95e4073b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:41:49.864 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:49.860+0000 7f95eb200640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f95e4109900 0x7f95e406c900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:41:49.864 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:49.860+0000 7f95eb200640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f95e4113d50 con 0x7f95e41042a0 2024-08-19T01:41:49.865 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:49.860+0000 7f95eb200640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f95e4113bd0 con 0x7f95e4109900 2024-08-19T01:41:49.865 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:49.860+0000 7f95eb200640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f95e4113ed0 con 0x7f95e4103900 2024-08-19T01:41:49.865 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:49.860+0000 7f95ea800640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f95e4109900 0x7f95e406c900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:41:49.865 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:49.860+0000 7f95ea800640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f95e4109900 0x7f95e406c900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.83:33104/0 (socket says 172.21.15.83:33104) 2024-08-19T01:41:49.865 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:49.860+0000 7f95ea800640 1 -- 172.21.15.83:0/1011797458 learned_addr learned my addr 172.21.15.83:0/1011797458 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-08-19T01:41:49.865 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:49.860+0000 7f95e9400640 1 --2- 172.21.15.83:0/1011797458 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f95e41042a0 0x7f95e4073b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:41:49.865 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:49.861+0000 7f95e9e00640 1 --2- 172.21.15.83:0/1011797458 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f95e4103900 0x7f95e4073640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:41:49.865 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:49.861+0000 7f95ea800640 1 -- 172.21.15.83:0/1011797458 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f95e4103900 msgr2=0x7f95e4073640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:41:49.865 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:49.861+0000 7f95ea800640 1 --2- 172.21.15.83:0/1011797458 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f95e4103900 0x7f95e4073640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:41:49.866 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:49.861+0000 7f95ea800640 1 -- 172.21.15.83:0/1011797458 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f95e41042a0 msgr2=0x7f95e4073b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:41:49.866 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:49.861+0000 7f95ea800640 1 --2- 172.21.15.83:0/1011797458 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f95e41042a0 0x7f95e4073b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:41:49.866 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:49.861+0000 7f95ea800640 1 -- 172.21.15.83:0/1011797458 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f95e406d130 con 0x7f95e4109900 2024-08-19T01:41:49.866 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:49.861+0000 7f95e9400640 1 --2- 172.21.15.83:0/1011797458 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f95e41042a0 0x7f95e4073b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-19T01:41:49.866 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:49.862+0000 7f95ea800640 1 --2- 172.21.15.83:0/1011797458 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f95e4109900 0x7f95e406c900 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f95e0003ce0 tx=0x7f95e0003d10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:41:49.866 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:49.862+0000 7f95dea00640 1 -- 172.21.15.83:0/1011797458 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95e0031650 con 0x7f95e4109900 2024-08-19T01:41:49.866 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:49.862+0000 7f95dea00640 1 -- 172.21.15.83:0/1011797458 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f95e00317f0 con 0x7f95e4109900 2024-08-19T01:41:49.866 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:49.862+0000 7f95eb200640 1 -- 172.21.15.83:0/1011797458 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f95e406d3c0 con 0x7f95e4109900 2024-08-19T01:41:49.867 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:49.862+0000 7f95dea00640 1 -- 172.21.15.83:0/1011797458 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95e002f9f0 con 0x7f95e4109900 2024-08-19T01:41:49.867 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:49.862+0000 7f95eb200640 1 -- 172.21.15.83:0/1011797458 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f95e4071390 con 0x7f95e4109900 2024-08-19T01:41:49.869 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:49.865+0000 7f95dea00640 1 -- 172.21.15.83:0/1011797458 <== mon.2 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7f95e002fc10 con 0x7f95e4109900 2024-08-19T01:41:49.869 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:49.865+0000 7f95ef3bc740 1 Processor -- start 2024-08-19T01:41:49.869 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:49.866+0000 7f95ef3bc740 1 -- start start 2024-08-19T01:41:49.869 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:49.866+0000 7f95ef3bc740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561c834e6d80 0x561c834e7160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:41:49.870 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:49.866+0000 7f95ef3bc740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561c834e76a0 0x561c834f1a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:41:49.870 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:49.866+0000 7f95ef3bc740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561c834f1f60 0x561c834f4350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:41:49.870 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:49.866+0000 7f95ef3bc740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x561c8344cbf0 con 0x561c834f1f60 2024-08-19T01:41:49.870 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:49.866+0000 7f95ef3bc740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x561c833e4bd0 con 0x561c834e76a0 2024-08-19T01:41:49.870 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:49.866+0000 7f95ef3bc740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x561c83448bb0 con 0x561c834e6d80 2024-08-19T01:41:49.870 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:49.866+0000 7f95e9e00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561c834e76a0 0x561c834f1a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:41:49.870 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:49.866+0000 7f95e9e00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561c834e76a0 0x561c834f1a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.83:33120/0 (socket says 172.21.15.83:33120) 2024-08-19T01:41:49.870 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:49.866+0000 7f95e9e00640 1 -- 172.21.15.83:0/4165676197 learned_addr learned my addr 172.21.15.83:0/4165676197 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-08-19T01:41:49.870 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:49.866+0000 7f95e9400640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561c834e6d80 0x561c834e7160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:41:49.870 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:49.866+0000 7f95e9400640 1 --2- 172.21.15.83:0/4165676197 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561c834f1f60 0x561c834f4350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:41:49.870 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:49.867+0000 7f95e9e00640 1 -- 172.21.15.83:0/4165676197 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561c834e6d80 msgr2=0x561c834e7160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:41:49.870 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:49.867+0000 7f95e9e00640 1 --2- 172.21.15.83:0/4165676197 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561c834e6d80 0x561c834e7160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:41:49.871 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:49.867+0000 7f95e9e00640 1 -- 172.21.15.83:0/4165676197 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561c834f1f60 msgr2=0x561c834f4350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:41:49.871 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:49.867+0000 7f95e9e00640 1 --2- 172.21.15.83:0/4165676197 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561c834f1f60 0x561c834f4350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:41:49.871 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:49.867+0000 7f95e9e00640 1 -- 172.21.15.83:0/4165676197 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561c833eb8a0 con 0x561c834e76a0 2024-08-19T01:41:49.871 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:49.867+0000 7f95e9e00640 1 --2- 172.21.15.83:0/4165676197 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561c834e76a0 0x561c834f1a20 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f95d8009c00 tx=0x7f95d80117b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=29791de0774d323c server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:41:49.871 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:49.867+0000 7f95dea00640 1 --2- 172.21.15.83:0/1011797458 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f95cc0a2c00 0x7f95cc0a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:41:49.871 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:49.867+0000 7f95d3000640 1 -- 172.21.15.83:0/4165676197 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95d8012450 con 0x561c834e76a0 2024-08-19T01:41:49.871 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:49.867+0000 7f95d3000640 1 -- 172.21.15.83:0/4165676197 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f95d8012ad0 con 0x561c834e76a0 2024-08-19T01:41:49.871 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:49.867+0000 7f95dea00640 1 -- 172.21.15.83:0/1011797458 <== mon.2 v2:172.21.15.117:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8269+0+0 (secure 0 0 0) 0x7f95e00f0540 con 0x7f95e4109900 2024-08-19T01:41:49.872 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:49.867+0000 7f95e9400640 1 --2- 172.21.15.83:0/1011797458 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f95cc0a2c00 0x7f95cc0a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:41:49.872 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:49.868+0000 7f95ef3bc740 1 -- 172.21.15.83:0/4165676197 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561c834e76a0 msgr2=0x561c834f1a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:41:49.872 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:49.868+0000 7f95ef3bc740 1 --2- 172.21.15.83:0/4165676197 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561c834e76a0 0x561c834f1a20 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f95d8009c00 tx=0x7f95d80117b0 comp rx=0 tx=0).stop 2024-08-19T01:41:49.872 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:49.868+0000 7f95e9400640 1 --2- 172.21.15.83:0/1011797458 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f95cc0a2c00 0x7f95cc0a50c0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x561c834e4690 tx=0x7f95d4004730 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:41:49.872 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:49.868+0000 7f95ef3bc740 1 -- 172.21.15.83:0/4165676197 shutdown_connections 2024-08-19T01:41:49.872 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:49.868+0000 7f95ef3bc740 1 --2- 172.21.15.83:0/4165676197 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561c834f1f60 0x561c834f4350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:41:49.872 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:49.868+0000 7f95ef3bc740 1 --2- 172.21.15.83:0/4165676197 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561c834e76a0 0x561c834f1a20 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:41:49.872 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:49.868+0000 7f95ef3bc740 1 --2- 172.21.15.83:0/4165676197 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561c834e6d80 0x561c834e7160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:41:49.872 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:49.868+0000 7f95ef3bc740 1 -- 172.21.15.83:0/4165676197 >> 172.21.15.83:0/4165676197 conn(0x561c834e3310 msgr2=0x561c834e36f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:41:49.872 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:49.868+0000 7f95ef3bc740 1 -- 172.21.15.83:0/4165676197 shutdown_connections 2024-08-19T01:41:49.872 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:49.868+0000 7f95ef3bc740 1 -- 172.21.15.83:0/4165676197 wait complete. 2024-08-19T01:41:49.872 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:49.869+0000 7f95ef3bc740 1 -- start start 2024-08-19T01:41:49.872 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:49.869+0000 7f95ef3bc740 1 Processor -- start 2024-08-19T01:41:49.875 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:49.869+0000 7f95ef3bc740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561c834e6d80 0x561c834f19c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:41:49.876 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:49.869+0000 7f95ef3bc740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561c834e76a0 0x561c834eec70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:41:49.876 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:49.869+0000 7f95ef3bc740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561c834f1f60 0x561c834ef400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:41:49.876 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:49.869+0000 7f95ef3bc740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x561c834f4b80 con 0x561c834e76a0 2024-08-19T01:41:49.876 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:49.869+0000 7f95ef3bc740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x561c83407d40 con 0x561c834f1f60 2024-08-19T01:41:49.876 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:49.869+0000 7f95ef3bc740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x561c834f4d00 con 0x561c834e6d80 2024-08-19T01:41:49.876 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:49.870+0000 7f95ef3bc740 10 client.0 _ll_register_callbacks cb 0x561c833d0b10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-19T01:41:49.876 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:49.870+0000 7f95e9e00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561c834e6d80 0x561c834f19c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:41:49.876 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:49.870+0000 7f95ea800640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561c834f1f60 0x561c834ef400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:41:49.876 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:49.870+0000 7f95ea800640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561c834f1f60 0x561c834ef400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.83:33130/0 (socket says 172.21.15.83:33130) 2024-08-19T01:41:49.877 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:49.870+0000 7f95e9e00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561c834e6d80 0x561c834f19c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.83:51396/0 (socket says 172.21.15.83:51396) 2024-08-19T01:41:49.877 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:49.870+0000 7f95ea800640 1 -- 172.21.15.83:0/1466102469 learned_addr learned my addr 172.21.15.83:0/1466102469 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-08-19T01:41:49.877 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:49.870+0000 7f95e9400640 1 --2- 172.21.15.83:0/1466102469 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561c834e76a0 0x561c834eec70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:41:49.877 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:49.870+0000 7f95ea800640 1 -- 172.21.15.83:0/1466102469 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561c834e6d80 msgr2=0x561c834f19c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:41:49.877 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:49.870+0000 7f95ea800640 1 --2- 172.21.15.83:0/1466102469 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561c834e6d80 0x561c834f19c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:41:49.877 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:49.870+0000 7f95ea800640 1 -- 172.21.15.83:0/1466102469 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561c834e76a0 msgr2=0x561c834eec70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:41:49.877 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:49.870+0000 7f95ea800640 1 --2- 172.21.15.83:0/1466102469 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561c834e76a0 0x561c834eec70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:41:49.877 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:49.870+0000 7f95ea800640 1 -- 172.21.15.83:0/1466102469 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561c831be3c0 con 0x561c834f1f60 2024-08-19T01:41:49.877 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:49.870+0000 7f95ea800640 1 --2- 172.21.15.83:0/1466102469 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561c834f1f60 0x561c834ef400 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f95e00b9240 tx=0x7f95e00bc920 comp rx=0 tx=0).ready entity=mon.2 client_cookie=da16b91f2b5011cd server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:41:49.877 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:49.870+0000 7f95c7400640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-08-19T01:41:49.878 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:49.871+0000 7f95c7400640 1 -- 172.21.15.83:0/1466102469 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95e00bcde0 con 0x561c834f1f60 2024-08-19T01:41:49.878 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:49.871+0000 7f95c7400640 1 -- 172.21.15.83:0/1466102469 <== mon.2 v2:172.21.15.117:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8269+0+0 (secure 0 0 0) 0x7f95e00b9450 con 0x561c834f1f60 2024-08-19T01:41:49.878 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:49.871+0000 7f95ef3bc740 1 -- 172.21.15.83:0/1466102469 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f95d8012ad0 con 0x561c834f1f60 2024-08-19T01:41:49.878 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:49.871+0000 7f95c7400640 1 -- 172.21.15.83:0/1466102469 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f95e00c1190 con 0x561c834f1f60 2024-08-19T01:41:49.878 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:49.871+0000 7f95ef3bc740 10 client.25970 fetch_fsmap learned FSMap version 28 2024-08-19T01:41:49.878 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:49.871+0000 7f95ef3bc740 1 -- 172.21.15.83:0/1466102469 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x561c834f4b80 con 0x561c834f1f60 2024-08-19T01:41:49.878 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:49.871+0000 7f95c7400640 1 -- 172.21.15.83:0/1466102469 <== mon.2 v2:172.21.15.117:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f95d8012ad0 con 0x561c834f1f60 2024-08-19T01:41:49.878 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:49.871+0000 7f95c7400640 1 -- 172.21.15.83:0/1466102469 <== mon.2 v2:172.21.15.117:3300/0 5 ==== fsmap(e 28) ==== 8358+0+0 (secure 0 0 0) 0x7f95e00bd140 con 0x561c834f1f60 2024-08-19T01:41:49.878 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:49.871+0000 7f95ef3bc740 10 client.25970 fetch_fsmap finished waiting for FSMap version 28 2024-08-19T01:41:49.878 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:49.871+0000 7f95ef3bc740 10 client.25970 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-19T01:41:49.878 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:49.871+0000 7f95ef3bc740 20 client.25970 populate_metadata read hostname 'smithi083' 2024-08-19T01:41:49.879 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:49.871+0000 7f95ef3bc740 1 --2- 172.21.15.83:0/1466102469 >> [v2:172.21.15.117:6838/1967595704,v1:172.21.15.117:6839/1967595704] conn(0x561c83501be0 0x561c83504060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:41:49.879 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:49.871+0000 7f95ef3bc740 4 client.25970 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-19T01:41:49.879 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:49.871+0000 7f95ef3bc740 1 -- 172.21.15.83:0/1466102469 --> [v2:172.21.15.117:6838/1967595704,v1:172.21.15.117:6839/1967595704] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561c8345a160 con 0x561c83501be0 2024-08-19T01:41:49.879 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:49.872+0000 7f95e9e00640 1 --2- 172.21.15.83:0/1466102469 >> [v2:172.21.15.117:6838/1967595704,v1:172.21.15.117:6839/1967595704] conn(0x561c83501be0 0x561c83504060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:41:49.879 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:49.872+0000 7f95e9e00640 1 --2- 172.21.15.83:0/1466102469 >> [v2:172.21.15.117:6838/1967595704,v1:172.21.15.117:6839/1967595704] conn(0x561c83501be0 0x561c83504060 crc :-1 s=READY pgs=121 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=706f16425882324c server_cookie=13afb299192e4112 in_seq=0 out_seq=0 2024-08-19T01:41:49.879 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:49.872+0000 7f95c7400640 10 client.25970 ms_handle_connect on v2:172.21.15.117:6838/1967595704 2024-08-19T01:41:49.879 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:49.874+0000 7f95c7400640 1 -- 172.21.15.83:0/1466102469 <== mds.0 v2:172.21.15.117:6838/1967595704 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x561c8345a160 con 0x561c83501be0 2024-08-19T01:41:49.879 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:49.874+0000 7f95c7400640 10 client.25970 handle_command_reply: tid=0 2024-08-19T01:41:49.880 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:49.874+0000 7f95ef3bc740 1 client.25970 shutdown 2024-08-19T01:41:49.880 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:49.874+0000 7f95ef3bc740 1 -- 172.21.15.83:0/1466102469 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561c834f1f60 msgr2=0x561c834ef400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:41:49.880 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:49.874+0000 7f95ef3bc740 1 --2- 172.21.15.83:0/1466102469 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561c834f1f60 0x561c834ef400 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f95e00b9240 tx=0x7f95e00bc920 comp rx=0 tx=0).stop 2024-08-19T01:41:49.880 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:49.874+0000 7f95ef3bc740 1 -- 172.21.15.83:0/1466102469 shutdown_connections 2024-08-19T01:41:49.880 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:49.874+0000 7f95ef3bc740 1 --2- 172.21.15.83:0/1466102469 >> [v2:172.21.15.117:6838/1967595704,v1:172.21.15.117:6839/1967595704] conn(0x561c83501be0 0x561c83504060 crc :-1 s=READY pgs=121 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:41:49.880 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:49.874+0000 7f95ef3bc740 1 --2- 172.21.15.83:0/1466102469 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561c834f1f60 0x561c834ef400 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:41:49.880 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:49.874+0000 7f95ef3bc740 1 --2- 172.21.15.83:0/1466102469 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561c834e76a0 0x561c834eec70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:41:49.880 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:49.874+0000 7f95ef3bc740 1 --2- 172.21.15.83:0/1466102469 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561c834e6d80 0x561c834f19c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:41:49.880 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:49.874+0000 7f95c7400640 0 client.25970 ms_handle_reset on v2:172.21.15.117:6838/1967595704 2024-08-19T01:41:49.880 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:49.874+0000 7f95ef3bc740 1 -- 172.21.15.83:0/1466102469 >> 172.21.15.83:0/1466102469 conn(0x561c834e3310 msgr2=0x561c834e8ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:41:49.881 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:49.874+0000 7f95ef3bc740 1 -- 172.21.15.83:0/1466102469 shutdown_connections 2024-08-19T01:41:49.881 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:49.874+0000 7f95ef3bc740 1 -- 172.21.15.83:0/1466102469 wait complete. 2024-08-19T01:41:49.881 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:49.875+0000 7f95ef3bc740 20 client.25970 trim_cache size 0 max 16384 2024-08-19T01:41:49.896 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:49.894+0000 7f95ef3bc740 1 Processor -- start 2024-08-19T01:41:49.896 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:49.894+0000 7f95ef3bc740 1 -- start start 2024-08-19T01:41:49.896 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:49.894+0000 7f95ef3bc740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561c834e6d80 0x561c834ed540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:41:49.896 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:49.894+0000 7f95ef3bc740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561c834e76a0 0x561c834eda80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:41:49.896 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:49.894+0000 7f95ef3bc740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561c834f1f60 0x561c83509970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:41:49.896 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:49.894+0000 7f95ef3bc740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f95e41ba370 con 0x561c834f1f60 2024-08-19T01:41:49.896 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:49.894+0000 7f95ef3bc740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f95e41b7c10 con 0x561c834e76a0 2024-08-19T01:41:49.896 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:49.894+0000 7f95ef3bc740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f95e41b5a60 con 0x561c834e6d80 2024-08-19T01:41:49.897 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:49.895+0000 7f95e9400640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561c834e76a0 0x561c834eda80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:41:49.897 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:49.895+0000 7f95e9e00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561c834e6d80 0x561c834ed540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:41:49.897 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:49.895+0000 7f95ea800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561c834f1f60 0x561c83509970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:41:49.897 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:49.895+0000 7f95e9400640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561c834e76a0 0x561c834eda80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.83:33142/0 (socket says 172.21.15.83:33142) 2024-08-19T01:41:49.897 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:49.895+0000 7f95e9400640 1 -- 172.21.15.83:0/414169606 learned_addr learned my addr 172.21.15.83:0/414169606 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-08-19T01:41:49.897 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:49.895+0000 7f95e9e00640 1 -- 172.21.15.83:0/414169606 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561c834e76a0 msgr2=0x561c834eda80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:41:49.897 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:49.895+0000 7f95e9e00640 1 --2- 172.21.15.83:0/414169606 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561c834e76a0 0x561c834eda80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:41:49.897 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:49.895+0000 7f95e9e00640 1 -- 172.21.15.83:0/414169606 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561c834f1f60 msgr2=0x561c83509970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:41:49.897 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:49.895+0000 7f95e9e00640 1 --2- 172.21.15.83:0/414169606 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561c834f1f60 0x561c83509970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:41:49.897 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:49.895+0000 7f95e9e00640 1 -- 172.21.15.83:0/414169606 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561c834e3140 con 0x561c834e6d80 2024-08-19T01:41:49.897 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:49.895+0000 7f95e9e00640 1 --2- 172.21.15.83:0/414169606 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561c834e6d80 0x561c834ed540 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f95d800f480 tx=0x7f95d800f680 comp rx=0 tx=0).ready entity=mon.1 client_cookie=59fc7a26357e6588 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:41:49.900 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:49.896+0000 7f95d2600640 1 -- 172.21.15.83:0/414169606 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95d8030020 con 0x561c834e6d80 2024-08-19T01:41:49.900 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:49.896+0000 7f95d2600640 1 -- 172.21.15.83:0/414169606 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f95d8005590 con 0x561c834e6d80 2024-08-19T01:41:49.900 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:49.896+0000 7f95d2600640 1 -- 172.21.15.83:0/414169606 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95d8003970 con 0x561c834e6d80 2024-08-19T01:41:49.900 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:49.896+0000 7f95ef3bc740 1 -- 172.21.15.83:0/414169606 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561c834e6d80 msgr2=0x561c834ed540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:41:49.900 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:49.896+0000 7f95ef3bc740 1 --2- 172.21.15.83:0/414169606 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561c834e6d80 0x561c834ed540 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f95d800f480 tx=0x7f95d800f680 comp rx=0 tx=0).stop 2024-08-19T01:41:49.900 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:49.896+0000 7f95ef3bc740 1 -- 172.21.15.83:0/414169606 shutdown_connections 2024-08-19T01:41:49.900 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:49.896+0000 7f95ef3bc740 1 --2- 172.21.15.83:0/414169606 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561c834f1f60 0x561c83509970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:41:49.900 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:49.896+0000 7f95ef3bc740 1 --2- 172.21.15.83:0/414169606 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561c834e76a0 0x561c834eda80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:41:49.900 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:49.896+0000 7f95ef3bc740 1 --2- 172.21.15.83:0/414169606 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561c834e6d80 0x561c834ed540 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:41:49.901 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:49.896+0000 7f95ef3bc740 1 -- 172.21.15.83:0/414169606 >> 172.21.15.83:0/414169606 conn(0x561c834e3310 msgr2=0x561c834e9fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:41:49.901 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:49.897+0000 7f95ef3bc740 1 -- 172.21.15.83:0/414169606 shutdown_connections 2024-08-19T01:41:49.901 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:49.897+0000 7f95ef3bc740 1 -- 172.21.15.83:0/414169606 wait complete. 2024-08-19T01:41:49.901 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:49.897+0000 7f95ef3bc740 1 -- start start 2024-08-19T01:41:49.901 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:49.897+0000 7f95ef3bc740 1 Processor -- start 2024-08-19T01:41:49.901 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:49.898+0000 7f95ef3bc740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561c834e6d80 0x561c834fa270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:41:49.901 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:49.898+0000 7f95ef3bc740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561c834e76a0 0x561c834fa7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:41:49.901 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:49.898+0000 7f95ef3bc740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561c834f1f60 0x561c834f28e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:41:49.901 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:49.898+0000 7f95ef3bc740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f95e41b60d0 con 0x561c834e76a0 2024-08-19T01:41:49.901 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:49.898+0000 7f95ef3bc740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f95e41b7540 con 0x561c834f1f60 2024-08-19T01:41:49.901 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:49.898+0000 7f95ef3bc740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f95e41b67a0 con 0x561c834e6d80 2024-08-19T01:41:49.901 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:49.898+0000 7f95ef3bc740 10 client.0 _ll_register_callbacks cb 0x7f95e00319a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-19T01:41:49.901 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:49.898+0000 7f95e9e00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561c834e6d80 0x561c834fa270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:41:49.902 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:49.898+0000 7f95e9e00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561c834e6d80 0x561c834fa270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.83:51418/0 (socket says 172.21.15.83:51418) 2024-08-19T01:41:49.902 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:49.898+0000 7f95e9e00640 1 -- 172.21.15.83:0/1033013116 learned_addr learned my addr 172.21.15.83:0/1033013116 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-08-19T01:41:49.902 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:49.898+0000 7f95e9400640 1 --2- 172.21.15.83:0/1033013116 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561c834e76a0 0x561c834fa7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:41:49.902 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:49.898+0000 7f95ea800640 1 --2- 172.21.15.83:0/1033013116 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561c834f1f60 0x561c834f28e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:41:49.902 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:49.898+0000 7f95e9e00640 1 -- 172.21.15.83:0/1033013116 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561c834f1f60 msgr2=0x561c834f28e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:41:49.902 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:49.898+0000 7f95e9e00640 1 --2- 172.21.15.83:0/1033013116 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561c834f1f60 0x561c834f28e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:41:49.902 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:49.898+0000 7f95e9e00640 1 -- 172.21.15.83:0/1033013116 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561c834e76a0 msgr2=0x561c834fa7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:41:49.902 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:49.898+0000 7f95e9e00640 1 --2- 172.21.15.83:0/1033013116 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561c834e76a0 0x561c834fa7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:41:49.902 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:49.899+0000 7f95e9e00640 1 -- 172.21.15.83:0/1033013116 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561c834f3110 con 0x561c834e6d80 2024-08-19T01:41:49.902 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:49.899+0000 7f95e9e00640 1 --2- 172.21.15.83:0/1033013116 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561c834e6d80 0x561c834fa270 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f95d801c040 tx=0x7f95d8000bc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2becf8fff5ebaa5b server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:41:49.903 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:49.899+0000 7f95c7400640 10 client.0 ms_handle_connect on v2:172.21.15.167:3300/0 2024-08-19T01:41:49.903 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:49.899+0000 7f95c7400640 1 -- 172.21.15.83:0/1033013116 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95d8030040 con 0x561c834e6d80 2024-08-19T01:41:49.903 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:49.899+0000 7f95ef3bc740 1 -- 172.21.15.83:0/1033013116 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f95d8005590 con 0x561c834e6d80 2024-08-19T01:41:49.903 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:49.899+0000 7f95c7400640 1 -- 172.21.15.83:0/1033013116 <== mon.1 v2:172.21.15.167:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8269+0+0 (secure 0 0 0) 0x7f95d8009630 con 0x561c834e6d80 2024-08-19T01:41:49.903 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:49.899+0000 7f95c7400640 1 -- 172.21.15.83:0/1033013116 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f95d80050b0 con 0x561c834e6d80 2024-08-19T01:41:49.903 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:49.899+0000 7f95ef3bc740 10 client.25912 fetch_fsmap learned FSMap version 28 2024-08-19T01:41:49.903 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:49.899+0000 7f95ef3bc740 1 -- 172.21.15.83:0/1033013116 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x561c834f9b20 con 0x561c834e6d80 2024-08-19T01:41:49.903 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:49.900+0000 7f95c7400640 1 -- 172.21.15.83:0/1033013116 <== mon.1 v2:172.21.15.167:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f95d8005590 con 0x561c834e6d80 2024-08-19T01:41:49.904 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:49.900+0000 7f95c7400640 1 -- 172.21.15.83:0/1033013116 <== mon.1 v2:172.21.15.167:3300/0 5 ==== fsmap(e 28) ==== 8358+0+0 (secure 0 0 0) 0x7f95d8034420 con 0x561c834e6d80 2024-08-19T01:41:49.905 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:49.900+0000 7f95ef3bc740 10 client.25912 fetch_fsmap finished waiting for FSMap version 28 2024-08-19T01:41:49.905 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:49.900+0000 7f95ef3bc740 10 client.25912 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-19T01:41:49.905 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:49.900+0000 7f95ef3bc740 20 client.25912 populate_metadata read hostname 'smithi083' 2024-08-19T01:41:49.905 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:49.900+0000 7f95ef3bc740 1 --2- 172.21.15.83:0/1033013116 >> [v2:172.21.15.117:6838/1967595704,v1:172.21.15.117:6839/1967595704] conn(0x561c83575300 0x561c83577720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:41:49.905 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:49.900+0000 7f95ef3bc740 4 client.25912 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-08-19T01:41:49.905 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:49.900+0000 7f95ef3bc740 1 -- 172.21.15.83:0/1033013116 --> [v2:172.21.15.117:6838/1967595704,v1:172.21.15.117:6839/1967595704] -- command(tid 0: {"prefix": "scrub status"}) -- 0x561c83454670 con 0x561c83575300 2024-08-19T01:41:49.905 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:49.900+0000 7f95e9400640 1 --2- 172.21.15.83:0/1033013116 >> [v2:172.21.15.117:6838/1967595704,v1:172.21.15.117:6839/1967595704] conn(0x561c83575300 0x561c83577720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:41:49.905 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:49.901+0000 7f95e9400640 1 --2- 172.21.15.83:0/1033013116 >> [v2:172.21.15.117:6838/1967595704,v1:172.21.15.117:6839/1967595704] conn(0x561c83575300 0x561c83577720 crc :-1 s=READY pgs=122 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5fd3368447f4f8b1 server_cookie=57c2e8cd9b79d485 in_seq=0 out_seq=0 2024-08-19T01:41:49.905 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:49.901+0000 7f95c7400640 10 client.25912 ms_handle_connect on v2:172.21.15.117:6838/1967595704 2024-08-19T01:41:49.905 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:49.902+0000 7f95c7400640 1 -- 172.21.15.83:0/1033013116 <== mds.0 v2:172.21.15.117:6838/1967595704 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x561c83454670 con 0x561c83575300 2024-08-19T01:41:49.905 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:49.902+0000 7f95c7400640 10 client.25912 handle_command_reply: tid=0 2024-08-19T01:41:49.905 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:49.902+0000 7f95ef3bc740 1 client.25912 shutdown 2024-08-19T01:41:49.905 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:49.902+0000 7f95ef3bc740 1 -- 172.21.15.83:0/1033013116 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561c834e6d80 msgr2=0x561c834fa270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:41:49.905 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:49.902+0000 7f95ef3bc740 1 --2- 172.21.15.83:0/1033013116 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561c834e6d80 0x561c834fa270 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f95d801c040 tx=0x7f95d8000bc0 comp rx=0 tx=0).stop 2024-08-19T01:41:49.906 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:49.902+0000 7f95ef3bc740 1 -- 172.21.15.83:0/1033013116 shutdown_connections 2024-08-19T01:41:49.906 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:49.902+0000 7f95ef3bc740 1 --2- 172.21.15.83:0/1033013116 >> [v2:172.21.15.117:6838/1967595704,v1:172.21.15.117:6839/1967595704] conn(0x561c83575300 0x561c83577720 crc :-1 s=READY pgs=122 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:41:49.906 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:49.902+0000 7f95ef3bc740 1 --2- 172.21.15.83:0/1033013116 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561c834f1f60 0x561c834f28e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:41:49.906 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:49.902+0000 7f95ef3bc740 1 --2- 172.21.15.83:0/1033013116 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561c834e76a0 0x561c834fa7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:41:49.906 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:49.902+0000 7f95c7400640 0 client.25912 ms_handle_reset on v2:172.21.15.117:6838/1967595704 2024-08-19T01:41:49.906 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:49.902+0000 7f95ef3bc740 1 --2- 172.21.15.83:0/1033013116 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561c834e6d80 0x561c834fa270 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:41:49.906 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:49.902+0000 7f95ef3bc740 1 -- 172.21.15.83:0/1033013116 >> 172.21.15.83:0/1033013116 conn(0x561c834e3310 msgr2=0x561c834f04d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:41:49.906 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:49.902+0000 7f95ef3bc740 1 -- 172.21.15.83:0/1033013116 shutdown_connections 2024-08-19T01:41:49.906 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:49.902+0000 7f95ef3bc740 1 -- 172.21.15.83:0/1033013116 wait complete. 2024-08-19T01:41:49.906 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:49.903+0000 7f95ef3bc740 20 client.25912 trim_cache size 0 max 16384 2024-08-19T01:41:49.907 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-08-19T01:41:49.907 INFO:teuthology.orchestra.run.smithi083.stdout: "status": "no active scrubs running", 2024-08-19T01:41:49.907 INFO:teuthology.orchestra.run.smithi083.stdout: "scrubs": {} 2024-08-19T01:41:49.907 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-08-19T01:41:49.908 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:49.903+0000 7f95eb200640 1 -- 172.21.15.83:0/1011797458 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f95cc0a2c00 msgr2=0x7f95cc0a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:41:49.908 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:49.903+0000 7f95eb200640 1 --2- 172.21.15.83:0/1011797458 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f95cc0a2c00 0x7f95cc0a50c0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x561c834e4690 tx=0x7f95d4004730 comp rx=0 tx=0).stop 2024-08-19T01:41:49.908 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:49.903+0000 7f95eb200640 1 -- 172.21.15.83:0/1011797458 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f95e4109900 msgr2=0x7f95e406c900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:41:49.908 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:49.903+0000 7f95eb200640 1 --2- 172.21.15.83:0/1011797458 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f95e4109900 0x7f95e406c900 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f95e0003ce0 tx=0x7f95e0003d10 comp rx=0 tx=0).stop 2024-08-19T01:41:49.909 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:49.903+0000 7f95eb200640 1 -- 172.21.15.83:0/1011797458 shutdown_connections 2024-08-19T01:41:49.909 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:49.903+0000 7f95eb200640 1 --2- 172.21.15.83:0/1011797458 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f95cc0a2c00 0x7f95cc0a50c0 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:41:49.909 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:49.903+0000 7f95eb200640 1 --2- 172.21.15.83:0/1011797458 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f95e4109900 0x7f95e406c900 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:41:49.909 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:49.903+0000 7f95eb200640 1 --2- 172.21.15.83:0/1011797458 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f95e41042a0 0x7f95e4073b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:41:49.909 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:49.903+0000 7f95eb200640 1 --2- 172.21.15.83:0/1011797458 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f95e4103900 0x7f95e4073640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:41:49.909 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:49.903+0000 7f95eb200640 1 -- 172.21.15.83:0/1011797458 >> 172.21.15.83:0/1011797458 conn(0x7f95e40ff600 msgr2=0x7f95e40ffd70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:41:49.910 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:49.904+0000 7f95eb200640 1 -- 172.21.15.83:0/1011797458 shutdown_connections 2024-08-19T01:41:49.910 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:49.904+0000 7f95eb200640 1 -- 172.21.15.83:0/1011797458 wait complete. 2024-08-19T01:41:49.952 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:41:49 smithi117 ceph-mon[22016]: Adjusting osd_memory_target on smithi167 to 176.0M 2024-08-19T01:41:49.952 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:41:49 smithi117 ceph-mon[22016]: Unable to set osd_memory_target on smithi167 to 184597504: error parsing value: Value '184597504' is below minimum 939524096 2024-08-19T01:41:49.952 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:41:49 smithi117 ceph-mon[22016]: Adjusting osd_memory_target on smithi083 to 176.4M 2024-08-19T01:41:49.952 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:41:49 smithi117 ceph-mon[22016]: Unable to set osd_memory_target on smithi083 to 185035468: error parsing value: Value '185035468' is below minimum 939524096 2024-08-19T01:41:49.952 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:41:49 smithi117 ceph-mon[22016]: Adjusting osd_memory_target on smithi117 to 176.0M 2024-08-19T01:41:49.953 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:41:49 smithi117 ceph-mon[22016]: Unable to set osd_memory_target on smithi117 to 184592486: error parsing value: Value '184592486' is below minimum 939524096 2024-08-19T01:41:49.953 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:41:49 smithi117 ceph-mon[22016]: pgmap v548: 97 pgs: 97 active+clean; 528 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 2.1 MiB/s wr, 1.31k op/s 2024-08-19T01:41:49.966 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:41:49 smithi167 ceph-mon[21875]: Adjusting osd_memory_target on smithi167 to 176.0M 2024-08-19T01:41:49.966 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:41:49 smithi167 ceph-mon[21875]: Unable to set osd_memory_target on smithi167 to 184597504: error parsing value: Value '184597504' is below minimum 939524096 2024-08-19T01:41:49.966 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:41:49 smithi167 ceph-mon[21875]: Adjusting osd_memory_target on smithi083 to 176.4M 2024-08-19T01:41:49.966 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:41:49 smithi167 ceph-mon[21875]: Unable to set osd_memory_target on smithi083 to 185035468: error parsing value: Value '185035468' is below minimum 939524096 2024-08-19T01:41:49.966 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:41:49 smithi167 ceph-mon[21875]: Adjusting osd_memory_target on smithi117 to 176.0M 2024-08-19T01:41:49.966 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:41:49 smithi167 ceph-mon[21875]: Unable to set osd_memory_target on smithi117 to 184592486: error parsing value: Value '184592486' is below minimum 939524096 2024-08-19T01:41:49.966 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:41:49 smithi167 ceph-mon[21875]: pgmap v548: 97 pgs: 97 active+clean; 528 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 2.1 MiB/s wr, 1.31k op/s 2024-08-19T01:41:50.852 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-08-19T01:41:50.852 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833 shell --fsid bad8780e-5dc9-11ef-bccf-c7b262605968 -- ceph fs dump --format=json 2024-08-19T01:41:51.252 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/bad8780e-5dc9-11ef-bccf-c7b262605968/mon.a/config 2024-08-19T01:41:51.874 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:51.872+0000 7f86f5200640 1 -- 172.21.15.83:0/4006531182 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f86f0107780 msgr2=0x7f86f0107b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:41:51.874 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:51.872+0000 7f86f5200640 1 --2- 172.21.15.83:0/4006531182 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f86f0107780 0x7f86f0107b60 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f86e0009930 tx=0x7f86e002a420 comp rx=0 tx=0).stop 2024-08-19T01:41:51.874 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:51.872+0000 7f86f5200640 1 -- 172.21.15.83:0/4006531182 shutdown_connections 2024-08-19T01:41:51.874 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:51.872+0000 7f86f5200640 1 --2- 172.21.15.83:0/4006531182 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f86f0100610 0x7f86f010e650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:41:51.874 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:51.872+0000 7f86f5200640 1 --2- 172.21.15.83:0/4006531182 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f86f00ffc70 0x7f86f01000d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:41:51.874 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:51.872+0000 7f86f5200640 1 --2- 172.21.15.83:0/4006531182 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f86f0107780 0x7f86f0107b60 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:41:51.874 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:51.872+0000 7f86f5200640 1 -- 172.21.15.83:0/4006531182 >> 172.21.15.83:0/4006531182 conn(0x7f86f0076da0 msgr2=0x7f86f00fe1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:41:51.875 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:51.872+0000 7f86f5200640 1 -- 172.21.15.83:0/4006531182 shutdown_connections 2024-08-19T01:41:51.875 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:51.872+0000 7f86f5200640 1 -- 172.21.15.83:0/4006531182 wait complete. 2024-08-19T01:41:51.876 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:51.873+0000 7f86f5200640 1 Processor -- start 2024-08-19T01:41:51.876 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:51.874+0000 7f86f5200640 1 -- start start 2024-08-19T01:41:51.877 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:51.874+0000 7f86f5200640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f86f00ffc70 0x7f86f0100e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:41:51.877 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:51.874+0000 7f86f5200640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f86f0100610 0x7f86f0101390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:41:51.877 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:51.874+0000 7f86f5200640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f86f0107780 0x7f86f01018d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:41:51.877 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:51.874+0000 7f86f5200640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f86f0111bd0 con 0x7f86f0107780 2024-08-19T01:41:51.877 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:51.874+0000 7f86f5200640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f86f0111a50 con 0x7f86f00ffc70 2024-08-19T01:41:51.877 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:51.874+0000 7f86f5200640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f86f0111d50 con 0x7f86f0100610 2024-08-19T01:41:51.878 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:51.874+0000 7f86ef400640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f86f00ffc70 0x7f86f0100e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:41:51.878 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:51.874+0000 7f86ef400640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f86f00ffc70 0x7f86f0100e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.83:33080/0 (socket says 172.21.15.83:33080) 2024-08-19T01:41:51.878 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:51.874+0000 7f86e6a00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f86f0100610 0x7f86f0101390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:41:51.878 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:51.874+0000 7f86ef400640 1 -- 172.21.15.83:0/1320089841 learned_addr learned my addr 172.21.15.83:0/1320089841 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-08-19T01:41:51.878 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:51.874+0000 7f86efe00640 1 --2- 172.21.15.83:0/1320089841 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f86f0107780 0x7f86f01018d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:41:51.881 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:51.875+0000 7f86ef400640 1 -- 172.21.15.83:0/1320089841 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f86f0100610 msgr2=0x7f86f0101390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:41:51.881 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:51.875+0000 7f86ef400640 1 --2- 172.21.15.83:0/1320089841 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f86f0100610 0x7f86f0101390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:41:51.881 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:51.875+0000 7f86ef400640 1 -- 172.21.15.83:0/1320089841 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f86f0107780 msgr2=0x7f86f01018d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:41:51.881 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:51.875+0000 7f86ef400640 1 --2- 172.21.15.83:0/1320089841 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f86f0107780 0x7f86f01018d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:41:51.881 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:51.875+0000 7f86ef400640 1 -- 172.21.15.83:0/1320089841 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f86f01a65c0 con 0x7f86f00ffc70 2024-08-19T01:41:51.881 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:51.875+0000 7f86e6a00640 1 --2- 172.21.15.83:0/1320089841 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f86f0100610 0x7f86f0101390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-19T01:41:51.881 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:51.875+0000 7f86ef400640 1 --2- 172.21.15.83:0/1320089841 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f86f00ffc70 0x7f86f0100e50 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f86e002a940 tx=0x7f86e0003b20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:41:51.881 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:51.875+0000 7f86ecc00640 1 -- 172.21.15.83:0/1320089841 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86e003f070 con 0x7f86f00ffc70 2024-08-19T01:41:51.881 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:51.875+0000 7f86ecc00640 1 -- 172.21.15.83:0/1320089841 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f86e0031c90 con 0x7f86f00ffc70 2024-08-19T01:41:51.881 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:51.875+0000 7f86f5200640 1 -- 172.21.15.83:0/1320089841 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f86f01a6850 con 0x7f86f00ffc70 2024-08-19T01:41:51.882 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:51.875+0000 7f86ecc00640 1 -- 172.21.15.83:0/1320089841 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86e0035030 con 0x7f86f00ffc70 2024-08-19T01:41:51.882 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:51.876+0000 7f86f5200640 1 -- 172.21.15.83:0/1320089841 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f86f01a6d30 con 0x7f86f00ffc70 2024-08-19T01:41:51.882 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:51.877+0000 7f86f5200640 1 -- 172.21.15.83:0/1320089841 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f86f01000d0 con 0x7f86f00ffc70 2024-08-19T01:41:51.882 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:51.879+0000 7f86ecc00640 1 -- 172.21.15.83:0/1320089841 <== mon.2 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7f86e0042070 con 0x7f86f00ffc70 2024-08-19T01:41:51.895 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:51.880+0000 7f86ecc00640 1 --2- 172.21.15.83:0/1320089841 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f86d40a2b80 0x7f86d40a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:41:51.895 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:51.880+0000 7f86ecc00640 1 -- 172.21.15.83:0/1320089841 <== mon.2 v2:172.21.15.117:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8269+0+0 (secure 0 0 0) 0x7f86e00ef6c0 con 0x7f86f00ffc70 2024-08-19T01:41:51.896 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:51.881+0000 7f86e6a00640 1 --2- 172.21.15.83:0/1320089841 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f86d40a2b80 0x7f86d40a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:41:51.896 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:51.881+0000 7f86e6a00640 1 --2- 172.21.15.83:0/1320089841 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f86d40a2b80 0x7f86d40a5040 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f86f00ff3c0 tx=0x7f86d8005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:41:51.901 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:51.887+0000 7f86ecc00640 1 -- 172.21.15.83:0/1320089841 <== mon.2 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7f86e00bbf50 con 0x7f86f00ffc70 2024-08-19T01:41:51.952 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:41:51 smithi117 ceph-mon[22016]: pgmap v549: 97 pgs: 97 active+clean; 524 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 2.5 MiB/s wr, 749 op/s 2024-08-19T01:41:51.966 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:41:51 smithi167 ceph-mon[21875]: pgmap v549: 97 pgs: 97 active+clean; 524 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 2.5 MiB/s wr, 749 op/s 2024-08-19T01:41:52.112 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:52.110+0000 7f86f5200640 1 -- 172.21.15.83:0/1320089841 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f86f0109cf0 con 0x7f86f00ffc70 2024-08-19T01:41:52.112 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:41:51 smithi083 ceph-mon[17120]: pgmap v549: 97 pgs: 97 active+clean; 524 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 2.5 MiB/s wr, 749 op/s 2024-08-19T01:41:52.113 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:52.111+0000 7f86ecc00640 1 -- 172.21.15.83:0/1320089841 <== mon.2 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+11924 (secure 0 0 0) 0x7f86e00c0250 con 0x7f86f00ffc70 2024-08-19T01:41:52.113 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-08-19T01:41:52.113 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":28,"btime":"2024-08-19T01:41:32:625368+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14421,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6835/988556067","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":988556067},{"type":"v1","addr":"172.21.15.83:6835","nonce":988556067}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14433,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6837/869702632","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6836","nonce":869702632},{"type":"v1","addr":"172.21.15.83:6837","nonce":869702632}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14475,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6841/1795975999","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6840","nonce":1795975999},{"type":"v1","addr":"172.21.15.83:6841","nonce":1795975999}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24430,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6833/2505076046","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6832","nonce":2505076046},{"type":"v1","addr":"172.21.15.167:6833","nonce":2505076046}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24449,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6833/3557520478","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":3557520478},{"type":"v1","addr":"172.21.15.117:6833","nonce":3557520478}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24466,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6835/3482045966","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":3482045966},{"type":"v1","addr":"172.21.15.167:6835","nonce":3482045966}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7}],"filesystems":[{"mdsmap":{"epoch":28,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-08-19T01:32:42.054769+0000","modified":"2024-08-19T01:41:31.877975+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24545,"mds_1":24527,"mds_2":24509},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24478":{"gid":24478,"name":"c","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.167:6837/1762130408","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6836","nonce":1762130408},{"type":"v1","addr":"172.21.15.167:6837","nonce":1762130408}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24485":{"gid":24485,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.117:6835/1024213994","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":1024213994},{"type":"v1","addr":"172.21.15.117:6835","nonce":1024213994}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24496":{"gid":24496,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.167:6839/3507387682","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6838","nonce":3507387682},{"type":"v1","addr":"172.21.15.167:6839","nonce":3507387682}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24509":{"gid":24509,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":10,"addr":"172.21.15.117:6837/3185224684","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":3185224684},{"type":"v1","addr":"172.21.15.117:6837","nonce":3185224684}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24527":{"gid":24527,"name":"g","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.83:6839/3239261845","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6838","nonce":3239261845},{"type":"v1","addr":"172.21.15.83:6839","nonce":3239261845}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24545":{"gid":24545,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.117:6839/1967595704","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6838","nonce":1967595704},{"type":"v1","addr":"172.21.15.117:6839","nonce":1967595704}]},"join_fscid":-1,"export_targets":[1,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24545,"qdb_cluster":[24509,24527,24545]},"id":1}]} 2024-08-19T01:41:52.114 INFO:teuthology.orchestra.run.smithi083.stderr:dumped fsmap epoch 28 2024-08-19T01:41:52.115 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:52.114+0000 7f86f5200640 1 -- 172.21.15.83:0/1320089841 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f86d40a2b80 msgr2=0x7f86d40a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:41:52.116 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:52.114+0000 7f86f5200640 1 --2- 172.21.15.83:0/1320089841 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f86d40a2b80 0x7f86d40a5040 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f86f00ff3c0 tx=0x7f86d8005f90 comp rx=0 tx=0).stop 2024-08-19T01:41:52.116 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:52.114+0000 7f86f5200640 1 -- 172.21.15.83:0/1320089841 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f86f00ffc70 msgr2=0x7f86f0100e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:41:52.116 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:52.114+0000 7f86f5200640 1 --2- 172.21.15.83:0/1320089841 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f86f00ffc70 0x7f86f0100e50 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f86e002a940 tx=0x7f86e0003b20 comp rx=0 tx=0).stop 2024-08-19T01:41:52.116 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:52.114+0000 7f86f5200640 1 -- 172.21.15.83:0/1320089841 shutdown_connections 2024-08-19T01:41:52.116 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:52.114+0000 7f86f5200640 1 --2- 172.21.15.83:0/1320089841 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f86d40a2b80 0x7f86d40a5040 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:41:52.116 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:52.114+0000 7f86f5200640 1 --2- 172.21.15.83:0/1320089841 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f86f0107780 0x7f86f01018d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:41:52.116 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:52.114+0000 7f86f5200640 1 --2- 172.21.15.83:0/1320089841 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f86f0100610 0x7f86f0101390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:41:52.116 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:52.114+0000 7f86f5200640 1 --2- 172.21.15.83:0/1320089841 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f86f00ffc70 0x7f86f0100e50 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:41:52.116 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:52.114+0000 7f86f5200640 1 -- 172.21.15.83:0/1320089841 >> 172.21.15.83:0/1320089841 conn(0x7f86f0076da0 msgr2=0x7f86f01045c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:41:52.116 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:52.114+0000 7f86f5200640 1 -- 172.21.15.83:0/1320089841 shutdown_connections 2024-08-19T01:41:52.116 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:52.114+0000 7f86f5200640 1 -- 172.21.15.83:0/1320089841 wait complete. 2024-08-19T01:41:52.941 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:41:52 smithi083 ceph-mon[17120]: from='client.? 172.21.15.83:0/1320089841' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-19T01:41:52.944 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833 shell --fsid bad8780e-5dc9-11ef-bccf-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-08-19T01:41:52.952 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:41:52 smithi117 ceph-mon[22016]: from='client.? 172.21.15.83:0/1320089841' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-19T01:41:52.965 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:41:52 smithi167 ceph-mon[21875]: from='client.? 172.21.15.83:0/1320089841' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-19T01:41:53.355 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/bad8780e-5dc9-11ef-bccf-c7b262605968/mon.a/config 2024-08-19T01:41:53.904 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:41:53 smithi083 ceph-mon[17120]: pgmap v550: 97 pgs: 97 active+clean; 517 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.5 MiB/s wr, 581 op/s 2024-08-19T01:41:53.952 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:41:53 smithi117 ceph-mon[22016]: pgmap v550: 97 pgs: 97 active+clean; 517 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.5 MiB/s wr, 581 op/s 2024-08-19T01:41:53.966 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:41:53 smithi167 ceph-mon[21875]: pgmap v550: 97 pgs: 97 active+clean; 517 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.5 MiB/s wr, 581 op/s 2024-08-19T01:41:54.160 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:54.158+0000 7f0d54a00640 1 -- 172.21.15.83:0/1689983453 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0d501016f0 msgr2=0x7f0d50101b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:41:54.160 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:54.158+0000 7f0d54a00640 1 --2- 172.21.15.83:0/1689983453 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0d501016f0 0x7f0d50101b50 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f0d3c009900 tx=0x7f0d3c02a410 comp rx=0 tx=0).stop 2024-08-19T01:41:54.160 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:54.158+0000 7f0d54a00640 1 -- 172.21.15.83:0/1689983453 shutdown_connections 2024-08-19T01:41:54.160 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:54.158+0000 7f0d54a00640 1 --2- 172.21.15.83:0/1689983453 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0d50102090 0x7f0d5010e5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:41:54.160 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:54.158+0000 7f0d54a00640 1 --2- 172.21.15.83:0/1689983453 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0d501016f0 0x7f0d50101b50 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:41:54.160 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:54.158+0000 7f0d54a00640 1 --2- 172.21.15.83:0/1689983453 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0d501076f0 0x7f0d50107ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:41:54.160 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:54.158+0000 7f0d54a00640 1 -- 172.21.15.83:0/1689983453 >> 172.21.15.83:0/1689983453 conn(0x7f0d500fd470 msgr2=0x7f0d500ff890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:41:54.160 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:54.158+0000 7f0d54a00640 1 -- 172.21.15.83:0/1689983453 shutdown_connections 2024-08-19T01:41:54.160 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:54.158+0000 7f0d54a00640 1 -- 172.21.15.83:0/1689983453 wait complete. 2024-08-19T01:41:54.161 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:54.159+0000 7f0d54a00640 1 Processor -- start 2024-08-19T01:41:54.161 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:54.159+0000 7f0d54a00640 1 -- start start 2024-08-19T01:41:54.161 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:54.160+0000 7f0d54a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0d501016f0 0x7f0d50074220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:41:54.161 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:54.160+0000 7f0d54a00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0d50102090 0x7f0d50074760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:41:54.162 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:54.160+0000 7f0d54a00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0d501076f0 0x7f0d50074ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:41:54.162 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:54.160+0000 7f0d54a00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f0d50111b40 con 0x7f0d501016f0 2024-08-19T01:41:54.162 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:54.160+0000 7f0d54a00640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f0d501119c0 con 0x7f0d501076f0 2024-08-19T01:41:54.162 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:54.160+0000 7f0d54a00640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f0d50111cc0 con 0x7f0d50102090 2024-08-19T01:41:54.162 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:54.160+0000 7f0d4f400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0d501016f0 0x7f0d50074220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:41:54.162 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:54.160+0000 7f0d4fe00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0d501076f0 0x7f0d50074ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:41:54.162 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:54.160+0000 7f0d4f400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0d501016f0 0x7f0d50074220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:33324/0 (socket says 172.21.15.83:33324) 2024-08-19T01:41:54.162 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:54.160+0000 7f0d4f400640 1 -- 172.21.15.83:0/2093169199 learned_addr learned my addr 172.21.15.83:0/2093169199 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-08-19T01:41:54.162 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:54.160+0000 7f0d4ea00640 1 --2- 172.21.15.83:0/2093169199 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0d50102090 0x7f0d50074760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:41:54.163 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:54.160+0000 7f0d4fe00640 1 -- 172.21.15.83:0/2093169199 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0d50102090 msgr2=0x7f0d50074760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:41:54.163 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:54.160+0000 7f0d4fe00640 1 --2- 172.21.15.83:0/2093169199 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0d50102090 0x7f0d50074760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:41:54.163 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:54.160+0000 7f0d4fe00640 1 -- 172.21.15.83:0/2093169199 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0d501016f0 msgr2=0x7f0d50074220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:41:54.163 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:54.160+0000 7f0d4fe00640 1 --2- 172.21.15.83:0/2093169199 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0d501016f0 0x7f0d50074220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:41:54.163 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:54.160+0000 7f0d4fe00640 1 -- 172.21.15.83:0/2093169199 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0d501a6470 con 0x7f0d501076f0 2024-08-19T01:41:54.164 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:54.161+0000 7f0d4fe00640 1 --2- 172.21.15.83:0/2093169199 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0d501076f0 0x7f0d50074ca0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f0d48009fd0 tx=0x7f0d4800ef10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:41:54.164 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:54.161+0000 7f0d47e00640 1 -- 172.21.15.83:0/2093169199 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d4800ce20 con 0x7f0d501076f0 2024-08-19T01:41:54.164 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:54.161+0000 7f0d47e00640 1 -- 172.21.15.83:0/2093169199 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0d48005020 con 0x7f0d501076f0 2024-08-19T01:41:54.164 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:54.161+0000 7f0d54a00640 1 -- 172.21.15.83:0/2093169199 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0d501a6760 con 0x7f0d501076f0 2024-08-19T01:41:54.164 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:54.161+0000 7f0d47e00640 1 -- 172.21.15.83:0/2093169199 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d48010720 con 0x7f0d501076f0 2024-08-19T01:41:54.164 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:54.161+0000 7f0d54a00640 1 -- 172.21.15.83:0/2093169199 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0d501a6c70 con 0x7f0d501076f0 2024-08-19T01:41:54.165 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:54.164+0000 7f0d47e00640 1 -- 172.21.15.83:0/2093169199 <== mon.2 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7f0d48005540 con 0x7f0d501076f0 2024-08-19T01:41:54.166 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:54.164+0000 7f0d58bf6740 1 Processor -- start 2024-08-19T01:41:54.166 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:54.164+0000 7f0d58bf6740 1 -- start start 2024-08-19T01:41:54.166 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:54.164+0000 7f0d58bf6740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5647df2916e0 0x5647df291ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:41:54.166 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:54.165+0000 7f0d58bf6740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5647df292000 0x5647df29c380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:41:54.166 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:54.165+0000 7f0d58bf6740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5647df29c8c0 0x5647df29ecb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:41:54.166 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:54.165+0000 7f0d58bf6740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x5647df1f6bf0 con 0x5647df29c8c0 2024-08-19T01:41:54.166 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:54.165+0000 7f0d58bf6740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x5647df18eb20 con 0x5647df2916e0 2024-08-19T01:41:54.166 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:54.165+0000 7f0d58bf6740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x5647df1f2bb0 con 0x5647df292000 2024-08-19T01:41:54.167 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:54.165+0000 7f0d4ea00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5647df2916e0 0x5647df291ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:41:54.168 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:54.165+0000 7f0d4ea00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5647df2916e0 0x5647df291ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.83:33114/0 (socket says 172.21.15.83:33114) 2024-08-19T01:41:54.168 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:54.165+0000 7f0d4ea00640 1 -- 172.21.15.83:0/2174376446 learned_addr learned my addr 172.21.15.83:0/2174376446 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-08-19T01:41:54.168 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:54.165+0000 7f0d4f400640 1 --2- 172.21.15.83:0/2174376446 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5647df292000 0x5647df29c380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:41:54.168 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:54.165+0000 7f0d4ea00640 1 --2- 172.21.15.83:0/2174376446 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5647df29c8c0 0x5647df29ecb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:41:54.168 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:54.165+0000 7f0d4f400640 1 -- 172.21.15.83:0/2174376446 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5647df2916e0 msgr2=0x5647df291ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:41:54.168 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:54.165+0000 7f0d4f400640 1 --2- 172.21.15.83:0/2174376446 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5647df2916e0 0x5647df291ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:41:54.168 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:54.165+0000 7f0d4f400640 1 -- 172.21.15.83:0/2174376446 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5647df29c8c0 msgr2=0x5647df29ecb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:41:54.168 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:54.165+0000 7f0d4f400640 1 --2- 172.21.15.83:0/2174376446 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5647df29c8c0 0x5647df29ecb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:41:54.168 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:54.165+0000 7f0d4f400640 1 -- 172.21.15.83:0/2174376446 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5647df1957f0 con 0x5647df292000 2024-08-19T01:41:54.168 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:54.166+0000 7f0d4f400640 1 --2- 172.21.15.83:0/2174376446 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5647df292000 0x5647df29c380 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f0d40005fd0 tx=0x7f0d4000bbb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d70af51f449d66ad server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:41:54.168 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:54.166+0000 7f0d47e00640 1 --2- 172.21.15.83:0/2093169199 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f0d340a2c00 0x7f0d340a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:41:54.168 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:54.166+0000 7f0d47e00640 1 -- 172.21.15.83:0/2093169199 <== mon.2 v2:172.21.15.117:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8269+0+0 (secure 0 0 0) 0x7f0d480cf740 con 0x7f0d501076f0 2024-08-19T01:41:54.169 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:54.166+0000 7f0d4ea00640 1 --2- 172.21.15.83:0/2093169199 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f0d340a2c00 0x7f0d340a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:41:54.169 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:54.166+0000 7f0d3be00640 1 -- 172.21.15.83:0/2174376446 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d40003b50 con 0x5647df292000 2024-08-19T01:41:54.169 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:54.166+0000 7f0d3be00640 1 -- 172.21.15.83:0/2174376446 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0d40015070 con 0x5647df292000 2024-08-19T01:41:54.169 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:54.166+0000 7f0d3be00640 1 -- 172.21.15.83:0/2174376446 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d40004dd0 con 0x5647df292000 2024-08-19T01:41:54.170 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:54.167+0000 7f0d58bf6740 1 -- 172.21.15.83:0/2174376446 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5647df292000 msgr2=0x5647df29c380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:41:54.170 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:54.167+0000 7f0d4ea00640 1 --2- 172.21.15.83:0/2093169199 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f0d340a2c00 0x7f0d340a50c0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x5647df28eff0 tx=0x7f0d3c02a410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:41:54.170 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:54.167+0000 7f0d58bf6740 1 --2- 172.21.15.83:0/2174376446 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5647df292000 0x5647df29c380 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f0d40005fd0 tx=0x7f0d4000bbb0 comp rx=0 tx=0).stop 2024-08-19T01:41:54.170 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:54.167+0000 7f0d58bf6740 1 -- 172.21.15.83:0/2174376446 shutdown_connections 2024-08-19T01:41:54.170 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:54.167+0000 7f0d58bf6740 1 --2- 172.21.15.83:0/2174376446 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5647df29c8c0 0x5647df29ecb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:41:54.170 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:54.167+0000 7f0d58bf6740 1 --2- 172.21.15.83:0/2174376446 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5647df292000 0x5647df29c380 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:41:54.170 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:54.167+0000 7f0d58bf6740 1 --2- 172.21.15.83:0/2174376446 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5647df2916e0 0x5647df291ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:41:54.170 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:54.167+0000 7f0d58bf6740 1 -- 172.21.15.83:0/2174376446 >> 172.21.15.83:0/2174376446 conn(0x5647df28dc70 msgr2=0x5647df28e050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:41:54.170 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:54.167+0000 7f0d58bf6740 1 -- 172.21.15.83:0/2174376446 shutdown_connections 2024-08-19T01:41:54.170 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:54.167+0000 7f0d58bf6740 1 -- 172.21.15.83:0/2174376446 wait complete. 2024-08-19T01:41:54.171 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:54.168+0000 7f0d58bf6740 1 -- start start 2024-08-19T01:41:54.171 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:54.168+0000 7f0d58bf6740 1 Processor -- start 2024-08-19T01:41:54.171 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:54.168+0000 7f0d58bf6740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5647df2916e0 0x5647df2997c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:41:54.171 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:54.168+0000 7f0d58bf6740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5647df292000 0x5647df299d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:41:54.171 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:54.168+0000 7f0d58bf6740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5647df29c8c0 0x5647df29a240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:41:54.171 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:54.169+0000 7f0d58bf6740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x5647df29f490 con 0x5647df292000 2024-08-19T01:41:54.171 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:54.169+0000 7f0d58bf6740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x5647df1b1cf0 con 0x5647df2916e0 2024-08-19T01:41:54.171 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:54.169+0000 7f0d58bf6740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x5647df29f610 con 0x5647df29c8c0 2024-08-19T01:41:54.171 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:54.169+0000 7f0d58bf6740 10 client.0 _ll_register_callbacks cb 0x5647df19de30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-19T01:41:54.173 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:54.169+0000 7f0d4f400640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5647df2916e0 0x5647df2997c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:41:54.173 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:54.169+0000 7f0d4f400640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5647df2916e0 0x5647df2997c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.83:33122/0 (socket says 172.21.15.83:33122) 2024-08-19T01:41:54.173 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:54.169+0000 7f0d4ea00640 1 --2- 172.21.15.83:0/3739762722 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5647df292000 0x5647df299d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:41:54.173 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:54.169+0000 7f0d4f400640 1 -- 172.21.15.83:0/3739762722 learned_addr learned my addr 172.21.15.83:0/3739762722 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-08-19T01:41:54.173 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:54.169+0000 7f0d4fe00640 1 --2- 172.21.15.83:0/3739762722 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5647df29c8c0 0x5647df29a240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:41:54.173 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:54.169+0000 7f0d4f400640 1 -- 172.21.15.83:0/3739762722 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5647df29c8c0 msgr2=0x5647df29a240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:41:54.173 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:54.169+0000 7f0d4f400640 1 --2- 172.21.15.83:0/3739762722 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5647df29c8c0 0x5647df29a240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:41:54.173 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:54.169+0000 7f0d4f400640 1 -- 172.21.15.83:0/3739762722 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5647df292000 msgr2=0x5647df299d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:41:54.173 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:54.169+0000 7f0d4f400640 1 --2- 172.21.15.83:0/3739762722 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5647df292000 0x5647df299d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:41:54.173 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:54.169+0000 7f0d4f400640 1 -- 172.21.15.83:0/3739762722 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5647def683c0 con 0x5647df2916e0 2024-08-19T01:41:54.173 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:54.169+0000 7f0d4f400640 1 --2- 172.21.15.83:0/3739762722 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5647df2916e0 0x5647df2997c0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f0d40000c00 tx=0x7f0d400047e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=66020b24884c96ee server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:41:54.173 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:54.169+0000 7f0d38c00640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-08-19T01:41:54.173 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:54.170+0000 7f0d38c00640 1 -- 172.21.15.83:0/3739762722 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d40029060 con 0x5647df2916e0 2024-08-19T01:41:54.174 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:54.170+0000 7f0d38c00640 1 -- 172.21.15.83:0/3739762722 <== mon.2 v2:172.21.15.117:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8269+0+0 (secure 0 0 0) 0x7f0d40011640 con 0x5647df2916e0 2024-08-19T01:41:54.174 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:54.170+0000 7f0d58bf6740 1 -- 172.21.15.83:0/3739762722 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0d40015070 con 0x5647df2916e0 2024-08-19T01:41:54.174 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:54.170+0000 7f0d38c00640 1 -- 172.21.15.83:0/3739762722 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f0d40011840 con 0x5647df2916e0 2024-08-19T01:41:54.174 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:54.170+0000 7f0d58bf6740 10 client.25994 fetch_fsmap learned FSMap version 28 2024-08-19T01:41:54.174 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:54.170+0000 7f0d58bf6740 1 -- 172.21.15.83:0/3739762722 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5647df2ab150 con 0x5647df2916e0 2024-08-19T01:41:54.174 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:54.170+0000 7f0d38c00640 1 -- 172.21.15.83:0/3739762722 <== mon.2 v2:172.21.15.117:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0d40015070 con 0x5647df2916e0 2024-08-19T01:41:54.174 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:54.170+0000 7f0d38c00640 1 -- 172.21.15.83:0/3739762722 <== mon.2 v2:172.21.15.117:3300/0 5 ==== fsmap(e 28) ==== 8358+0+0 (secure 0 0 0) 0x7f0d40022070 con 0x5647df2916e0 2024-08-19T01:41:54.174 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:54.171+0000 7f0d58bf6740 10 client.25994 fetch_fsmap finished waiting for FSMap version 28 2024-08-19T01:41:54.174 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:54.171+0000 7f0d58bf6740 10 client.25994 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-08-19T01:41:54.174 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:54.171+0000 7f0d58bf6740 20 client.25994 populate_metadata read hostname 'smithi083' 2024-08-19T01:41:54.174 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:54.171+0000 7f0d58bf6740 1 --2- 172.21.15.83:0/3739762722 >> [v2:172.21.15.117:6836/3185224684,v1:172.21.15.117:6837/3185224684] conn(0x5647df2ac560 0x5647df2ae950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:41:54.174 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:54.171+0000 7f0d58bf6740 4 client.25994 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-19T01:41:54.174 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:54.171+0000 7f0d58bf6740 1 -- 172.21.15.83:0/3739762722 --> [v2:172.21.15.117:6836/3185224684,v1:172.21.15.117:6837/3185224684] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5647df204160 con 0x5647df2ac560 2024-08-19T01:41:54.174 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:54.171+0000 7f0d4ea00640 1 --2- 172.21.15.83:0/3739762722 >> [v2:172.21.15.117:6836/3185224684,v1:172.21.15.117:6837/3185224684] conn(0x5647df2ac560 0x5647df2ae950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:41:54.175 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:54.171+0000 7f0d4ea00640 1 --2- 172.21.15.83:0/3739762722 >> [v2:172.21.15.117:6836/3185224684,v1:172.21.15.117:6837/3185224684] conn(0x5647df2ac560 0x5647df2ae950 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=fe5f610a563e4398 server_cookie=1f2b8264464f0a2 in_seq=0 out_seq=0 2024-08-19T01:41:54.175 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:54.171+0000 7f0d38c00640 10 client.25994 ms_handle_connect on v2:172.21.15.117:6836/3185224684 2024-08-19T01:41:54.175 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:54.173+0000 7f0d38c00640 1 -- 172.21.15.83:0/3739762722 <== mds.2 v2:172.21.15.117:6836/3185224684 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5647df204160 con 0x5647df2ac560 2024-08-19T01:41:54.175 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:54.173+0000 7f0d38c00640 10 client.25994 handle_command_reply: tid=0 2024-08-19T01:41:54.176 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:54.173+0000 7f0d58bf6740 1 client.25994 shutdown 2024-08-19T01:41:54.176 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:54.173+0000 7f0d58bf6740 1 -- 172.21.15.83:0/3739762722 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5647df2916e0 msgr2=0x5647df2997c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:41:54.176 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:54.173+0000 7f0d58bf6740 1 --2- 172.21.15.83:0/3739762722 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5647df2916e0 0x5647df2997c0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f0d40000c00 tx=0x7f0d400047e0 comp rx=0 tx=0).stop 2024-08-19T01:41:54.176 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:54.173+0000 7f0d58bf6740 1 -- 172.21.15.83:0/3739762722 shutdown_connections 2024-08-19T01:41:54.176 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:54.173+0000 7f0d58bf6740 1 --2- 172.21.15.83:0/3739762722 >> [v2:172.21.15.117:6836/3185224684,v1:172.21.15.117:6837/3185224684] conn(0x5647df2ac560 0x5647df2ae950 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:41:54.176 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:54.173+0000 7f0d58bf6740 1 --2- 172.21.15.83:0/3739762722 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5647df29c8c0 0x5647df29a240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:41:54.176 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:54.173+0000 7f0d58bf6740 1 --2- 172.21.15.83:0/3739762722 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5647df292000 0x5647df299d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:41:54.176 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:54.173+0000 7f0d38c00640 0 client.25994 ms_handle_reset on v2:172.21.15.117:6836/3185224684 2024-08-19T01:41:54.176 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:54.173+0000 7f0d58bf6740 1 --2- 172.21.15.83:0/3739762722 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5647df2916e0 0x5647df2997c0 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:41:54.176 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:54.173+0000 7f0d58bf6740 1 -- 172.21.15.83:0/3739762722 >> 172.21.15.83:0/3739762722 conn(0x5647df28dc70 msgr2=0x5647df293950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:41:54.176 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:54.173+0000 7f0d58bf6740 1 -- 172.21.15.83:0/3739762722 shutdown_connections 2024-08-19T01:41:54.176 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:54.173+0000 7f0d58bf6740 1 -- 172.21.15.83:0/3739762722 wait complete. 2024-08-19T01:41:54.176 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:54.173+0000 7f0d58bf6740 20 client.25994 trim_cache size 0 max 16384 2024-08-19T01:41:54.189 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:54.187+0000 7f0d58bf6740 1 Processor -- start 2024-08-19T01:41:54.189 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:54.187+0000 7f0d58bf6740 1 -- start start 2024-08-19T01:41:54.189 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:54.187+0000 7f0d58bf6740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5647df2916e0 0x5647df28cf50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:41:54.190 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:54.187+0000 7f0d58bf6740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5647df292000 0x5647df298f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:41:54.190 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:54.187+0000 7f0d58bf6740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5647df29c8c0 0x5647df2b3f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:41:54.190 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:54.187+0000 7f0d58bf6740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f0d501b1c30 con 0x5647df29c8c0 2024-08-19T01:41:54.190 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:54.187+0000 7f0d58bf6740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f0d501af4d0 con 0x5647df2916e0 2024-08-19T01:41:54.190 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:54.187+0000 7f0d58bf6740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f0d501ad320 con 0x5647df292000 2024-08-19T01:41:54.191 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:54.187+0000 7f0d4f400640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5647df2916e0 0x5647df28cf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:41:54.191 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:54.187+0000 7f0d4f400640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5647df2916e0 0x5647df28cf50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.83:33138/0 (socket says 172.21.15.83:33138) 2024-08-19T01:41:54.191 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:54.188+0000 7f0d4f400640 1 -- 172.21.15.83:0/4248954957 learned_addr learned my addr 172.21.15.83:0/4248954957 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-08-19T01:41:54.191 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:54.188+0000 7f0d4ea00640 1 --2- 172.21.15.83:0/4248954957 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5647df292000 0x5647df298f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:41:54.191 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:54.188+0000 7f0d4fe00640 1 --2- 172.21.15.83:0/4248954957 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5647df29c8c0 0x5647df2b3f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:41:54.191 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:54.188+0000 7f0d4f400640 1 -- 172.21.15.83:0/4248954957 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5647df292000 msgr2=0x5647df298f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:41:54.191 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:54.188+0000 7f0d4f400640 1 --2- 172.21.15.83:0/4248954957 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5647df292000 0x5647df298f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:41:54.191 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:54.188+0000 7f0d4f400640 1 -- 172.21.15.83:0/4248954957 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5647df29c8c0 msgr2=0x5647df2b3f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:41:54.191 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:54.188+0000 7f0d4f400640 1 --2- 172.21.15.83:0/4248954957 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5647df29c8c0 0x5647df2b3f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:41:54.191 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:54.188+0000 7f0d4f400640 1 -- 172.21.15.83:0/4248954957 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5647df29c310 con 0x5647df2916e0 2024-08-19T01:41:54.191 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:54.188+0000 7f0d4f400640 1 --2- 172.21.15.83:0/4248954957 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5647df2916e0 0x5647df28cf50 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f0d40006010 tx=0x7f0d40005d30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a18e56ca0991274c server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:41:54.192 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:54.188+0000 7f0d3b400640 1 -- 172.21.15.83:0/4248954957 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d40019d10 con 0x5647df2916e0 2024-08-19T01:41:54.192 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:54.188+0000 7f0d3b400640 1 -- 172.21.15.83:0/4248954957 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0d40015040 con 0x5647df2916e0 2024-08-19T01:41:54.192 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:54.188+0000 7f0d3b400640 1 -- 172.21.15.83:0/4248954957 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d400198a0 con 0x5647df2916e0 2024-08-19T01:41:54.194 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:54.189+0000 7f0d58bf6740 1 -- 172.21.15.83:0/4248954957 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5647df2916e0 msgr2=0x5647df28cf50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:41:54.195 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:54.189+0000 7f0d58bf6740 1 --2- 172.21.15.83:0/4248954957 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5647df2916e0 0x5647df28cf50 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f0d40006010 tx=0x7f0d40005d30 comp rx=0 tx=0).stop 2024-08-19T01:41:54.195 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:54.189+0000 7f0d58bf6740 1 -- 172.21.15.83:0/4248954957 shutdown_connections 2024-08-19T01:41:54.195 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:54.189+0000 7f0d58bf6740 1 --2- 172.21.15.83:0/4248954957 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5647df29c8c0 0x5647df2b3f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:41:54.195 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:54.189+0000 7f0d58bf6740 1 --2- 172.21.15.83:0/4248954957 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5647df292000 0x5647df298f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:41:54.195 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:54.189+0000 7f0d58bf6740 1 --2- 172.21.15.83:0/4248954957 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5647df2916e0 0x5647df28cf50 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:41:54.195 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:54.189+0000 7f0d58bf6740 1 -- 172.21.15.83:0/4248954957 >> 172.21.15.83:0/4248954957 conn(0x5647df28dc70 msgr2=0x5647df29b840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:41:54.195 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:54.189+0000 7f0d58bf6740 1 -- 172.21.15.83:0/4248954957 shutdown_connections 2024-08-19T01:41:54.195 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:54.189+0000 7f0d58bf6740 1 -- 172.21.15.83:0/4248954957 wait complete. 2024-08-19T01:41:54.195 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:54.190+0000 7f0d58bf6740 1 -- start start 2024-08-19T01:41:54.195 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:54.190+0000 7f0d58bf6740 1 Processor -- start 2024-08-19T01:41:54.195 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:54.190+0000 7f0d58bf6740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5647df2916e0 0x5647df2a49a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:41:54.196 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:54.190+0000 7f0d58bf6740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5647df292000 0x5647df2a4ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:41:54.196 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:54.190+0000 7f0d58bf6740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5647df29c8c0 0x5647df29fdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:41:54.196 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:54.191+0000 7f0d58bf6740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f0d501ad990 con 0x5647df2916e0 2024-08-19T01:41:54.196 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:54.191+0000 7f0d58bf6740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f0d501aee00 con 0x5647df292000 2024-08-19T01:41:54.196 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:54.191+0000 7f0d58bf6740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f0d501ae060 con 0x5647df29c8c0 2024-08-19T01:41:54.196 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:54.191+0000 7f0d58bf6740 10 client.0 _ll_register_callbacks cb 0x7f0d400043c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-19T01:41:54.196 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:54.191+0000 7f0d4ea00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5647df292000 0x5647df2a4ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:41:54.196 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:54.191+0000 7f0d4f400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5647df2916e0 0x5647df2a49a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:41:54.196 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:54.191+0000 7f0d4ea00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5647df292000 0x5647df2a4ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.83:33148/0 (socket says 172.21.15.83:33148) 2024-08-19T01:41:54.196 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:54.191+0000 7f0d4ea00640 1 -- 172.21.15.83:0/2644252407 learned_addr learned my addr 172.21.15.83:0/2644252407 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-08-19T01:41:54.196 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:54.191+0000 7f0d4fe00640 1 --2- 172.21.15.83:0/2644252407 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5647df29c8c0 0x5647df29fdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:41:54.196 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:54.191+0000 7f0d4f400640 1 -- 172.21.15.83:0/2644252407 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5647df29c8c0 msgr2=0x5647df29fdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:41:54.196 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:54.191+0000 7f0d4f400640 1 --2- 172.21.15.83:0/2644252407 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5647df29c8c0 0x5647df29fdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:41:54.196 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:54.191+0000 7f0d4f400640 1 -- 172.21.15.83:0/2644252407 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5647df292000 msgr2=0x5647df2a4ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:41:54.197 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:54.191+0000 7f0d4f400640 1 --2- 172.21.15.83:0/2644252407 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5647df292000 0x5647df2a4ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:41:54.197 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:54.191+0000 7f0d4f400640 1 -- 172.21.15.83:0/2644252407 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5647df2a05e0 con 0x5647df2916e0 2024-08-19T01:41:54.197 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:54.191+0000 7f0d4ea00640 1 --2- 172.21.15.83:0/2644252407 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5647df292000 0x5647df2a4ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-19T01:41:54.197 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:54.192+0000 7f0d4f400640 1 --2- 172.21.15.83:0/2644252407 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5647df2916e0 0x5647df2a49a0 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f0d40004510 tx=0x7f0d40005dc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=543a15f89209e053 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:41:54.197 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:54.192+0000 7f0d38c00640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-08-19T01:41:54.197 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:54.192+0000 7f0d38c00640 1 -- 172.21.15.83:0/2644252407 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d40011be0 con 0x5647df2916e0 2024-08-19T01:41:54.197 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:54.192+0000 7f0d38c00640 1 -- 172.21.15.83:0/2644252407 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8269+0+0 (secure 0 0 0) 0x7f0d40011d80 con 0x5647df2916e0 2024-08-19T01:41:54.197 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:54.192+0000 7f0d58bf6740 1 -- 172.21.15.83:0/2644252407 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0d40015040 con 0x5647df2916e0 2024-08-19T01:41:54.197 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:54.192+0000 7f0d38c00640 1 -- 172.21.15.83:0/2644252407 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f0d400115d0 con 0x5647df2916e0 2024-08-19T01:41:54.197 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:54.192+0000 7f0d58bf6740 10 client.15954 fetch_fsmap learned FSMap version 28 2024-08-19T01:41:54.197 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:54.192+0000 7f0d58bf6740 1 -- 172.21.15.83:0/2644252407 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5647df3169c0 con 0x5647df2916e0 2024-08-19T01:41:54.197 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:54.193+0000 7f0d38c00640 1 -- 172.21.15.83:0/2644252407 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0d40015040 con 0x5647df2916e0 2024-08-19T01:41:54.197 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:54.193+0000 7f0d38c00640 1 -- 172.21.15.83:0/2644252407 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 28) ==== 8358+0+0 (secure 0 0 0) 0x7f0d40022030 con 0x5647df2916e0 2024-08-19T01:41:54.197 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:54.193+0000 7f0d58bf6740 10 client.15954 fetch_fsmap finished waiting for FSMap version 28 2024-08-19T01:41:54.198 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:54.193+0000 7f0d58bf6740 10 client.15954 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-08-19T01:41:54.198 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:54.193+0000 7f0d58bf6740 20 client.15954 populate_metadata read hostname 'smithi083' 2024-08-19T01:41:54.198 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:54.193+0000 7f0d58bf6740 1 --2- 172.21.15.83:0/2644252407 >> [v2:172.21.15.117:6836/3185224684,v1:172.21.15.117:6837/3185224684] conn(0x5647df316fc0 0x5647df3193d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:41:54.198 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:54.193+0000 7f0d58bf6740 4 client.15954 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-08-19T01:41:54.198 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:54.193+0000 7f0d58bf6740 1 -- 172.21.15.83:0/2644252407 --> [v2:172.21.15.117:6836/3185224684,v1:172.21.15.117:6837/3185224684] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5647df1fe670 con 0x5647df316fc0 2024-08-19T01:41:54.199 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:54.193+0000 7f0d4ea00640 1 --2- 172.21.15.83:0/2644252407 >> [v2:172.21.15.117:6836/3185224684,v1:172.21.15.117:6837/3185224684] conn(0x5647df316fc0 0x5647df3193d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:41:54.199 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:54.194+0000 7f0d4ea00640 1 --2- 172.21.15.83:0/2644252407 >> [v2:172.21.15.117:6836/3185224684,v1:172.21.15.117:6837/3185224684] conn(0x5647df316fc0 0x5647df3193d0 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=33308c4ba3d3e821 server_cookie=cf764c5d814950d2 in_seq=0 out_seq=0 2024-08-19T01:41:54.199 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:54.194+0000 7f0d38c00640 10 client.15954 ms_handle_connect on v2:172.21.15.117:6836/3185224684 2024-08-19T01:41:54.199 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:54.194+0000 7f0d38c00640 1 -- 172.21.15.83:0/2644252407 <== mds.2 v2:172.21.15.117:6836/3185224684 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5647df1fe670 con 0x5647df316fc0 2024-08-19T01:41:54.199 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:54.194+0000 7f0d38c00640 10 client.15954 handle_command_reply: tid=0 2024-08-19T01:41:54.199 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:54.194+0000 7f0d58bf6740 1 client.15954 shutdown 2024-08-19T01:41:54.199 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:54.194+0000 7f0d58bf6740 1 -- 172.21.15.83:0/2644252407 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5647df2916e0 msgr2=0x5647df2a49a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:41:54.199 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:54.194+0000 7f0d58bf6740 1 --2- 172.21.15.83:0/2644252407 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5647df2916e0 0x5647df2a49a0 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f0d40004510 tx=0x7f0d40005dc0 comp rx=0 tx=0).stop 2024-08-19T01:41:54.199 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:54.194+0000 7f0d58bf6740 1 -- 172.21.15.83:0/2644252407 shutdown_connections 2024-08-19T01:41:54.200 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:54.194+0000 7f0d58bf6740 1 --2- 172.21.15.83:0/2644252407 >> [v2:172.21.15.117:6836/3185224684,v1:172.21.15.117:6837/3185224684] conn(0x5647df316fc0 0x5647df3193d0 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:41:54.200 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:54.194+0000 7f0d58bf6740 1 --2- 172.21.15.83:0/2644252407 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5647df29c8c0 0x5647df29fdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:41:54.200 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:54.194+0000 7f0d58bf6740 1 --2- 172.21.15.83:0/2644252407 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5647df292000 0x5647df2a4ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:41:54.200 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:54.194+0000 7f0d58bf6740 1 --2- 172.21.15.83:0/2644252407 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5647df2916e0 0x5647df2a49a0 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:41:54.200 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:54.194+0000 7f0d58bf6740 1 -- 172.21.15.83:0/2644252407 >> 172.21.15.83:0/2644252407 conn(0x5647df28dc70 msgr2=0x5647df293130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:41:54.200 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:54.194+0000 7f0d38c00640 0 client.15954 ms_handle_reset on v2:172.21.15.117:6836/3185224684 2024-08-19T01:41:54.200 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:54.194+0000 7f0d58bf6740 1 -- 172.21.15.83:0/2644252407 shutdown_connections 2024-08-19T01:41:54.200 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:54.194+0000 7f0d58bf6740 1 -- 172.21.15.83:0/2644252407 wait complete. 2024-08-19T01:41:54.200 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:54.195+0000 7f0d58bf6740 20 client.15954 trim_cache size 0 max 16384 2024-08-19T01:41:54.200 INFO:teuthology.orchestra.run.smithi083.stdout:[] 2024-08-19T01:41:54.201 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:54.195+0000 7f0d54a00640 1 -- 172.21.15.83:0/2093169199 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f0d340a2c00 msgr2=0x7f0d340a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:41:54.201 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:54.195+0000 7f0d54a00640 1 --2- 172.21.15.83:0/2093169199 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f0d340a2c00 0x7f0d340a50c0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x5647df28eff0 tx=0x7f0d3c02a410 comp rx=0 tx=0).stop 2024-08-19T01:41:54.201 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:54.195+0000 7f0d54a00640 1 -- 172.21.15.83:0/2093169199 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0d501076f0 msgr2=0x7f0d50074ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:41:54.201 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:54.195+0000 7f0d54a00640 1 --2- 172.21.15.83:0/2093169199 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0d501076f0 0x7f0d50074ca0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f0d48009fd0 tx=0x7f0d4800ef10 comp rx=0 tx=0).stop 2024-08-19T01:41:54.201 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:54.195+0000 7f0d54a00640 1 -- 172.21.15.83:0/2093169199 shutdown_connections 2024-08-19T01:41:54.201 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:54.195+0000 7f0d54a00640 1 --2- 172.21.15.83:0/2093169199 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f0d340a2c00 0x7f0d340a50c0 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:41:54.202 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:54.195+0000 7f0d54a00640 1 --2- 172.21.15.83:0/2093169199 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0d501076f0 0x7f0d50074ca0 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:41:54.202 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:54.195+0000 7f0d54a00640 1 --2- 172.21.15.83:0/2093169199 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0d50102090 0x7f0d50074760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:41:54.202 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:54.195+0000 7f0d54a00640 1 --2- 172.21.15.83:0/2093169199 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0d501016f0 0x7f0d50074220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:41:54.202 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:54.195+0000 7f0d54a00640 1 -- 172.21.15.83:0/2093169199 >> 172.21.15.83:0/2093169199 conn(0x7f0d500fd470 msgr2=0x7f0d500fdc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:41:54.202 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:54.196+0000 7f0d54a00640 1 -- 172.21.15.83:0/2093169199 shutdown_connections 2024-08-19T01:41:54.202 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:54.196+0000 7f0d54a00640 1 -- 172.21.15.83:0/2093169199 wait complete. 2024-08-19T01:41:55.381 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833 shell --fsid bad8780e-5dc9-11ef-bccf-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-08-19T01:41:55.820 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/bad8780e-5dc9-11ef-bccf-c7b262605968/mon.a/config 2024-08-19T01:41:55.952 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:41:55 smithi117 ceph-mon[22016]: pgmap v551: 97 pgs: 97 active+clean; 489 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 3.1 MiB/s wr, 608 op/s 2024-08-19T01:41:55.963 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:41:55 smithi083 ceph-mon[17120]: pgmap v551: 97 pgs: 97 active+clean; 489 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 3.1 MiB/s wr, 608 op/s 2024-08-19T01:41:55.965 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:41:55 smithi167 ceph-mon[21875]: pgmap v551: 97 pgs: 97 active+clean; 489 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 3.1 MiB/s wr, 608 op/s 2024-08-19T01:41:56.408 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:56.406+0000 7ff913e00640 1 -- 172.21.15.83:0/459442273 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff914103900 msgr2=0x7ff914103d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:41:56.409 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:56.406+0000 7ff913e00640 1 --2- 172.21.15.83:0/459442273 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff914103900 0x7ff914103d60 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7ff900009890 tx=0x7ff90002a380 comp rx=0 tx=0).stop 2024-08-19T01:41:56.409 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:56.406+0000 7ff913e00640 1 -- 172.21.15.83:0/459442273 shutdown_connections 2024-08-19T01:41:56.409 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:56.406+0000 7ff913e00640 1 --2- 172.21.15.83:0/459442273 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff9141042a0 0x7ff9141107d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:41:56.409 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:56.406+0000 7ff913e00640 1 --2- 172.21.15.83:0/459442273 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff914103900 0x7ff914103d60 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:41:56.409 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:56.406+0000 7ff913e00640 1 --2- 172.21.15.83:0/459442273 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff914109900 0x7ff914109ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:41:56.409 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:56.406+0000 7ff913e00640 1 -- 172.21.15.83:0/459442273 >> 172.21.15.83:0/459442273 conn(0x7ff9140ff640 msgr2=0x7ff914101a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:41:56.409 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:56.406+0000 7ff913e00640 1 -- 172.21.15.83:0/459442273 shutdown_connections 2024-08-19T01:41:56.409 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:56.406+0000 7ff913e00640 1 -- 172.21.15.83:0/459442273 wait complete. 2024-08-19T01:41:56.411 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:56.408+0000 7ff913e00640 1 Processor -- start 2024-08-19T01:41:56.411 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:56.408+0000 7ff913e00640 1 -- start start 2024-08-19T01:41:56.411 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:56.408+0000 7ff913e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff914103900 0x7ff914076430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:41:56.411 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:56.408+0000 7ff913e00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff9141042a0 0x7ff914076970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:41:56.411 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:56.408+0000 7ff913e00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff914109900 0x7ff914076eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:41:56.411 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:56.408+0000 7ff913e00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7ff914113d00 con 0x7ff914103900 2024-08-19T01:41:56.411 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:56.408+0000 7ff913e00640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7ff914113b80 con 0x7ff9141042a0 2024-08-19T01:41:56.411 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:56.408+0000 7ff913e00640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7ff914113e80 con 0x7ff914109900 2024-08-19T01:41:56.412 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:56.409+0000 7ff912000640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff9141042a0 0x7ff914076970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:41:56.412 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:56.409+0000 7ff912a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff914103900 0x7ff914076430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:41:56.412 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:56.409+0000 7ff913400640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff914109900 0x7ff914076eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:41:56.412 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:56.410+0000 7ff912000640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff9141042a0 0x7ff914076970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.83:33176/0 (socket says 172.21.15.83:33176) 2024-08-19T01:41:56.412 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:56.410+0000 7ff912a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff914103900 0x7ff914076430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:33388/0 (socket says 172.21.15.83:33388) 2024-08-19T01:41:56.412 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:56.410+0000 7ff912000640 1 -- 172.21.15.83:0/1926967642 learned_addr learned my addr 172.21.15.83:0/1926967642 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-08-19T01:41:56.412 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:56.410+0000 7ff913400640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff914109900 0x7ff914076eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.83:45924/0 (socket says 172.21.15.83:45924) 2024-08-19T01:41:56.413 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:56.410+0000 7ff913400640 1 -- 172.21.15.83:0/1926967642 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff9141042a0 msgr2=0x7ff914076970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:41:56.413 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:56.411+0000 7ff913400640 1 --2- 172.21.15.83:0/1926967642 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff9141042a0 0x7ff914076970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:41:56.413 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:56.411+0000 7ff913400640 1 -- 172.21.15.83:0/1926967642 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff914103900 msgr2=0x7ff914076430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:41:56.413 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:56.411+0000 7ff913400640 1 --2- 172.21.15.83:0/1926967642 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff914103900 0x7ff914076430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:41:56.413 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:56.411+0000 7ff913400640 1 -- 172.21.15.83:0/1926967642 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff914072850 con 0x7ff914109900 2024-08-19T01:41:56.413 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:56.411+0000 7ff912a00640 1 --2- 172.21.15.83:0/1926967642 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff914103900 0x7ff914076430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-19T01:41:56.413 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:56.412+0000 7ff913400640 1 --2- 172.21.15.83:0/1926967642 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff914109900 0x7ff914076eb0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7ff90c00e710 tx=0x7ff90c00ebd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:41:56.414 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:56.412+0000 7ff90b400640 1 -- 172.21.15.83:0/1926967642 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff90c004220 con 0x7ff914109900 2024-08-19T01:41:56.414 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:56.412+0000 7ff90b400640 1 -- 172.21.15.83:0/1926967642 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff90c0043c0 con 0x7ff914109900 2024-08-19T01:41:56.414 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:56.412+0000 7ff90b400640 1 -- 172.21.15.83:0/1926967642 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff90c004a00 con 0x7ff914109900 2024-08-19T01:41:56.414 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:56.412+0000 7ff913e00640 1 -- 172.21.15.83:0/1926967642 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff914072b40 con 0x7ff914109900 2024-08-19T01:41:56.414 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:56.413+0000 7ff913e00640 1 -- 172.21.15.83:0/1926967642 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff914073030 con 0x7ff914109900 2024-08-19T01:41:56.419 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:56.418+0000 7ff91c25e740 1 Processor -- start 2024-08-19T01:41:56.420 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:56.419+0000 7ff90b400640 1 -- 172.21.15.83:0/1926967642 <== mon.1 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7ff90c004ba0 con 0x7ff914109900 2024-08-19T01:41:56.420 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:56.419+0000 7ff91c25e740 1 -- start start 2024-08-19T01:41:56.421 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:56.419+0000 7ff91c25e740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559431e5f6e0 0x559431e5fac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:41:56.421 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:56.419+0000 7ff91c25e740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559431e60000 0x559431e6a380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:41:56.421 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:56.420+0000 7ff91c25e740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559431e6a8c0 0x559431e6ccb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:41:56.421 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:56.420+0000 7ff91c25e740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x559431dc4bf0 con 0x559431e60000 2024-08-19T01:41:56.421 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:56.420+0000 7ff91c25e740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x559431d5cb20 con 0x559431e6a8c0 2024-08-19T01:41:56.421 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:56.420+0000 7ff91c25e740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x559431dc0bb0 con 0x559431e5f6e0 2024-08-19T01:41:56.422 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:56.420+0000 7ff912a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559431e60000 0x559431e6a380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:41:56.423 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:56.420+0000 7ff912a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559431e60000 0x559431e6a380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:33404/0 (socket says 172.21.15.83:33404) 2024-08-19T01:41:56.423 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:56.420+0000 7ff912a00640 1 -- 172.21.15.83:0/1449813513 learned_addr learned my addr 172.21.15.83:0/1449813513 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-08-19T01:41:56.423 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:56.420+0000 7ff912000640 1 --2- 172.21.15.83:0/1449813513 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559431e5f6e0 0x559431e5fac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:41:56.423 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:56.421+0000 7ff912a00640 1 -- 172.21.15.83:0/1449813513 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559431e5f6e0 msgr2=0x559431e5fac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:41:56.423 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:56.421+0000 7ff912a00640 1 --2- 172.21.15.83:0/1449813513 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559431e5f6e0 0x559431e5fac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:41:56.423 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:56.421+0000 7ff912a00640 1 -- 172.21.15.83:0/1449813513 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559431e6a8c0 msgr2=0x559431e6ccb0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-08-19T01:41:56.423 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:56.421+0000 7ff912a00640 1 --2- 172.21.15.83:0/1449813513 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559431e6a8c0 0x559431e6ccb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:41:56.423 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:56.421+0000 7ff912a00640 1 -- 172.21.15.83:0/1449813513 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559431d637f0 con 0x559431e60000 2024-08-19T01:41:56.423 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:56.421+0000 7ff90b400640 1 --2- 172.21.15.83:0/1926967642 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7ff8f80a2c00 0x7ff8f80a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:41:56.423 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:56.421+0000 7ff90b400640 1 -- 172.21.15.83:0/1926967642 <== mon.1 v2:172.21.15.167:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8269+0+0 (secure 0 0 0) 0x7ff90c0cee70 con 0x7ff914109900 2024-08-19T01:41:56.423 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:56.421+0000 7ff912a00640 1 --2- 172.21.15.83:0/1449813513 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559431e60000 0x559431e6a380 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7ff904011810 tx=0x7ff904011cd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ba709da6853c920f server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:41:56.424 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:56.422+0000 7ff912000640 1 --2- 172.21.15.83:0/1449813513 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559431e5f6e0 0x559431e5fac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-19T01:41:56.424 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:56.422+0000 7ff8ff400640 1 -- 172.21.15.83:0/1449813513 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff904003c30 con 0x559431e60000 2024-08-19T01:41:56.424 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:56.422+0000 7ff8ff400640 1 -- 172.21.15.83:0/1449813513 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff90400fe80 con 0x559431e60000 2024-08-19T01:41:56.424 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:56.422+0000 7ff8ff400640 1 -- 172.21.15.83:0/1449813513 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9040193e0 con 0x559431e60000 2024-08-19T01:41:56.425 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:56.423+0000 7ff912000640 1 --2- 172.21.15.83:0/1926967642 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7ff8f80a2c00 0x7ff8f80a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:41:56.425 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:56.423+0000 7ff91c25e740 1 -- 172.21.15.83:0/1449813513 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559431e60000 msgr2=0x559431e6a380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:41:56.425 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:56.423+0000 7ff91c25e740 1 --2- 172.21.15.83:0/1449813513 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559431e60000 0x559431e6a380 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7ff904011810 tx=0x7ff904011cd0 comp rx=0 tx=0).stop 2024-08-19T01:41:56.425 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:56.424+0000 7ff91c25e740 1 -- 172.21.15.83:0/1449813513 shutdown_connections 2024-08-19T01:41:56.426 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:56.424+0000 7ff91c25e740 1 --2- 172.21.15.83:0/1449813513 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559431e6a8c0 0x559431e6ccb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:41:56.426 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:56.424+0000 7ff91c25e740 1 --2- 172.21.15.83:0/1449813513 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559431e60000 0x559431e6a380 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:41:56.426 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:56.424+0000 7ff91c25e740 1 --2- 172.21.15.83:0/1449813513 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559431e5f6e0 0x559431e5fac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:41:56.426 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:56.424+0000 7ff91c25e740 1 -- 172.21.15.83:0/1449813513 >> 172.21.15.83:0/1449813513 conn(0x559431e5bc70 msgr2=0x559431e5c050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:41:56.426 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:56.425+0000 7ff912000640 1 --2- 172.21.15.83:0/1926967642 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7ff8f80a2c00 0x7ff8f80a50c0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7ff900005e00 tx=0x7ff90002c040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:41:56.426 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:56.425+0000 7ff91c25e740 1 -- 172.21.15.83:0/1449813513 shutdown_connections 2024-08-19T01:41:56.427 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:56.425+0000 7ff91c25e740 1 -- 172.21.15.83:0/1449813513 wait complete. 2024-08-19T01:41:56.427 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:56.425+0000 7ff91c25e740 1 -- start start 2024-08-19T01:41:56.427 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:56.425+0000 7ff91c25e740 1 Processor -- start 2024-08-19T01:41:56.427 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:56.426+0000 7ff91c25e740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559431e5f6e0 0x559431e67740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:41:56.428 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:56.426+0000 7ff91c25e740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559431e60000 0x559431e67c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:41:56.428 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:56.426+0000 7ff91c25e740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559431e6a8c0 0x559431e681c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:41:56.428 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:56.426+0000 7ff91c25e740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x559431e6d4e0 con 0x559431e5f6e0 2024-08-19T01:41:56.428 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:56.426+0000 7ff91c25e740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x559431d7fcf0 con 0x559431e60000 2024-08-19T01:41:56.428 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:56.426+0000 7ff91c25e740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x559431e6d660 con 0x559431e6a8c0 2024-08-19T01:41:56.428 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:56.426+0000 7ff91c25e740 10 client.0 _ll_register_callbacks cb 0x559431d6be30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-19T01:41:56.429 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:56.427+0000 7ff913400640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559431e6a8c0 0x559431e681c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:41:56.429 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:56.427+0000 7ff913400640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559431e6a8c0 0x559431e681c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.83:45950/0 (socket says 172.21.15.83:45950) 2024-08-19T01:41:56.429 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:56.427+0000 7ff913400640 1 -- 172.21.15.83:0/2366242131 learned_addr learned my addr 172.21.15.83:0/2366242131 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-08-19T01:41:56.429 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:56.428+0000 7ff913400640 1 -- 172.21.15.83:0/2366242131 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559431e60000 msgr2=0x559431e67c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:41:56.429 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:56.428+0000 7ff913400640 1 --2- 172.21.15.83:0/2366242131 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559431e60000 0x559431e67c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:41:56.430 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:56.428+0000 7ff913400640 1 -- 172.21.15.83:0/2366242131 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559431e5f6e0 msgr2=0x559431e67740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:41:56.430 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:56.428+0000 7ff913400640 1 --2- 172.21.15.83:0/2366242131 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559431e5f6e0 0x559431e67740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:41:56.430 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:56.428+0000 7ff913400640 1 -- 172.21.15.83:0/2366242131 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559431b363c0 con 0x559431e6a8c0 2024-08-19T01:41:56.430 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:56.428+0000 7ff913400640 1 --2- 172.21.15.83:0/2366242131 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559431e6a8c0 0x559431e681c0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7ff90c098020 tx=0x7ff90c09afb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=86af2f7813f07662 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:41:56.430 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:56.428+0000 7ff8f3e00640 10 client.0 ms_handle_connect on v2:172.21.15.167:3300/0 2024-08-19T01:41:56.431 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:56.428+0000 7ff8f3e00640 1 -- 172.21.15.83:0/2366242131 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff90c0b4060 con 0x559431e6a8c0 2024-08-19T01:41:56.431 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:56.428+0000 7ff91c25e740 1 -- 172.21.15.83:0/2366242131 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x559431dd2160 con 0x559431e6a8c0 2024-08-19T01:41:56.431 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:56.428+0000 7ff8f3e00640 1 -- 172.21.15.83:0/2366242131 <== mon.1 v2:172.21.15.167:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8269+0+0 (secure 0 0 0) 0x7ff90c084be0 con 0x559431e6a8c0 2024-08-19T01:41:56.431 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:56.429+0000 7ff8f3e00640 1 -- 172.21.15.83:0/2366242131 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7ff90c01c040 con 0x559431e6a8c0 2024-08-19T01:41:56.431 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:56.429+0000 7ff91c25e740 10 client.25948 fetch_fsmap learned FSMap version 28 2024-08-19T01:41:56.431 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:56.429+0000 7ff91c25e740 1 -- 172.21.15.83:0/2366242131 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x559431e79220 con 0x559431e6a8c0 2024-08-19T01:41:56.431 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:56.429+0000 7ff8f3e00640 1 -- 172.21.15.83:0/2366242131 <== mon.1 v2:172.21.15.167:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x559431dd2160 con 0x559431e6a8c0 2024-08-19T01:41:56.431 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:56.429+0000 7ff8f3e00640 1 -- 172.21.15.83:0/2366242131 <== mon.1 v2:172.21.15.167:3300/0 5 ==== fsmap(e 28) ==== 8358+0+0 (secure 0 0 0) 0x7ff90c0a3e30 con 0x559431e6a8c0 2024-08-19T01:41:56.432 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:56.429+0000 7ff91c25e740 10 client.25948 fetch_fsmap finished waiting for FSMap version 28 2024-08-19T01:41:56.432 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:56.429+0000 7ff91c25e740 10 client.25948 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.g 2024-08-19T01:41:56.432 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:56.429+0000 7ff91c25e740 20 client.25948 populate_metadata read hostname 'smithi083' 2024-08-19T01:41:56.432 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:56.429+0000 7ff91c25e740 1 --2- 172.21.15.83:0/2366242131 >> [v2:172.21.15.83:6838/3239261845,v1:172.21.15.83:6839/3239261845] conn(0x559431e7a630 0x559431e7ca80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:41:56.432 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:56.430+0000 7ff91c25e740 4 client.25948 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-19T01:41:56.432 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:56.430+0000 7ff91c25e740 1 -- 172.21.15.83:0/2366242131 --> [v2:172.21.15.83:6838/3239261845,v1:172.21.15.83:6839/3239261845] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559431dcc670 con 0x559431e7a630 2024-08-19T01:41:56.432 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:56.430+0000 7ff912a00640 1 --2- 172.21.15.83:0/2366242131 >> [v2:172.21.15.83:6838/3239261845,v1:172.21.15.83:6839/3239261845] conn(0x559431e7a630 0x559431e7ca80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:41:56.432 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:56.430+0000 7ff912a00640 1 --2- 172.21.15.83:0/2366242131 >> [v2:172.21.15.83:6838/3239261845,v1:172.21.15.83:6839/3239261845] conn(0x559431e7a630 0x559431e7ca80 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=6525b28907436699 server_cookie=15f29f7d7fbfaaf1 in_seq=0 out_seq=0 2024-08-19T01:41:56.432 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:56.431+0000 7ff8f3e00640 10 client.25948 ms_handle_connect on v2:172.21.15.83:6838/3239261845 2024-08-19T01:41:56.433 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:56.432+0000 7ff8f3e00640 1 -- 172.21.15.83:0/2366242131 <== mds.1 v2:172.21.15.83:6838/3239261845 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x559431dcc670 con 0x559431e7a630 2024-08-19T01:41:56.433 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:56.432+0000 7ff8f3e00640 10 client.25948 handle_command_reply: tid=0 2024-08-19T01:41:56.433 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:56.432+0000 7ff91c25e740 1 client.25948 shutdown 2024-08-19T01:41:56.434 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:56.432+0000 7ff91c25e740 1 -- 172.21.15.83:0/2366242131 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559431e6a8c0 msgr2=0x559431e681c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:41:56.434 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:56.432+0000 7ff91c25e740 1 --2- 172.21.15.83:0/2366242131 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559431e6a8c0 0x559431e681c0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7ff90c098020 tx=0x7ff90c09afb0 comp rx=0 tx=0).stop 2024-08-19T01:41:56.435 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:56.433+0000 7ff91c25e740 1 -- 172.21.15.83:0/2366242131 shutdown_connections 2024-08-19T01:41:56.435 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:56.433+0000 7ff91c25e740 1 --2- 172.21.15.83:0/2366242131 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559431e6a8c0 0x559431e681c0 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:41:56.435 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:56.433+0000 7ff91c25e740 1 --2- 172.21.15.83:0/2366242131 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559431e60000 0x559431e67c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:41:56.435 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:56.433+0000 7ff91c25e740 1 --2- 172.21.15.83:0/2366242131 >> [v2:172.21.15.83:6838/3239261845,v1:172.21.15.83:6839/3239261845] conn(0x559431e7a630 0x559431e7ca80 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:41:56.435 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:56.433+0000 7ff91c25e740 1 --2- 172.21.15.83:0/2366242131 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559431e5f6e0 0x559431e67740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:41:56.435 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:56.433+0000 7ff91c25e740 1 -- 172.21.15.83:0/2366242131 >> 172.21.15.83:0/2366242131 conn(0x559431e5bc70 msgr2=0x559431e619a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:41:56.435 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:56.433+0000 7ff8f3e00640 0 client.25948 ms_handle_reset on v2:172.21.15.83:6838/3239261845 2024-08-19T01:41:56.435 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:56.433+0000 7ff91c25e740 1 -- 172.21.15.83:0/2366242131 shutdown_connections 2024-08-19T01:41:56.436 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:56.433+0000 7ff91c25e740 1 -- 172.21.15.83:0/2366242131 wait complete. 2024-08-19T01:41:56.436 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:56.434+0000 7ff91c25e740 20 client.25948 trim_cache size 0 max 16384 2024-08-19T01:41:56.461 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:56.459+0000 7ff91c25e740 1 Processor -- start 2024-08-19T01:41:56.461 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:56.459+0000 7ff91c25e740 1 -- start start 2024-08-19T01:41:56.461 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:56.460+0000 7ff91c25e740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559431e5f6e0 0x559431e62140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:41:56.461 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:56.460+0000 7ff91c25e740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559431e60000 0x559431e67000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:41:56.461 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:56.460+0000 7ff91c25e740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559431e6a8c0 0x559431e5ab90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:41:56.462 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:56.460+0000 7ff91c25e740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7ff9141c7180 con 0x559431e6a8c0 2024-08-19T01:41:56.462 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:56.460+0000 7ff91c25e740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7ff9141b1a00 con 0x559431e5f6e0 2024-08-19T01:41:56.462 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:56.460+0000 7ff91c25e740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7ff9141af850 con 0x559431e60000 2024-08-19T01:41:56.462 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:56.460+0000 7ff912a00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559431e5f6e0 0x559431e62140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:41:56.462 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:56.460+0000 7ff912a00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559431e5f6e0 0x559431e62140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.83:33192/0 (socket says 172.21.15.83:33192) 2024-08-19T01:41:56.462 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:56.460+0000 7ff912a00640 1 -- 172.21.15.83:0/3642200704 learned_addr learned my addr 172.21.15.83:0/3642200704 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-08-19T01:41:56.462 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:56.461+0000 7ff912000640 1 --2- 172.21.15.83:0/3642200704 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559431e60000 0x559431e67000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:41:56.463 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:56.461+0000 7ff912a00640 1 -- 172.21.15.83:0/3642200704 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559431e60000 msgr2=0x559431e67000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:41:56.463 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:56.461+0000 7ff912a00640 1 --2- 172.21.15.83:0/3642200704 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559431e60000 0x559431e67000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:41:56.463 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:56.461+0000 7ff912a00640 1 -- 172.21.15.83:0/3642200704 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559431e6a8c0 msgr2=0x559431e5ab90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:41:56.463 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:56.461+0000 7ff912a00640 1 --2- 172.21.15.83:0/3642200704 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559431e6a8c0 0x559431e5ab90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:41:56.463 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:56.461+0000 7ff912a00640 1 -- 172.21.15.83:0/3642200704 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559431e666f0 con 0x559431e5f6e0 2024-08-19T01:41:56.463 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:56.461+0000 7ff912000640 1 --2- 172.21.15.83:0/3642200704 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559431e60000 0x559431e67000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-19T01:41:56.464 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:56.463+0000 7ff912a00640 1 --2- 172.21.15.83:0/3642200704 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559431e5f6e0 0x559431e62140 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7ff9040117e0 tx=0x7ff9040044a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=26e8f1aff3039816 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:41:56.465 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:56.463+0000 7ff8fea00640 1 -- 172.21.15.83:0/3642200704 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff90401f040 con 0x559431e5f6e0 2024-08-19T01:41:56.465 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:56.463+0000 7ff8fea00640 1 -- 172.21.15.83:0/3642200704 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff90402a030 con 0x559431e5f6e0 2024-08-19T01:41:56.465 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:56.463+0000 7ff8fea00640 1 -- 172.21.15.83:0/3642200704 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff904004df0 con 0x559431e5f6e0 2024-08-19T01:41:56.466 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:56.464+0000 7ff91c25e740 1 -- 172.21.15.83:0/3642200704 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559431e5f6e0 msgr2=0x559431e62140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:41:56.466 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:56.464+0000 7ff91c25e740 1 --2- 172.21.15.83:0/3642200704 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559431e5f6e0 0x559431e62140 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7ff9040117e0 tx=0x7ff9040044a0 comp rx=0 tx=0).stop 2024-08-19T01:41:56.466 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:56.465+0000 7ff91c25e740 1 -- 172.21.15.83:0/3642200704 shutdown_connections 2024-08-19T01:41:56.466 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:56.465+0000 7ff91c25e740 1 --2- 172.21.15.83:0/3642200704 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559431e6a8c0 0x559431e5ab90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:41:56.467 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:56.465+0000 7ff91c25e740 1 --2- 172.21.15.83:0/3642200704 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559431e60000 0x559431e67000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:41:56.467 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:56.465+0000 7ff91c25e740 1 --2- 172.21.15.83:0/3642200704 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559431e5f6e0 0x559431e62140 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:41:56.467 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:56.465+0000 7ff91c25e740 1 -- 172.21.15.83:0/3642200704 >> 172.21.15.83:0/3642200704 conn(0x559431e5bc70 msgr2=0x559431e5a480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:41:56.467 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:56.465+0000 7ff91c25e740 1 -- 172.21.15.83:0/3642200704 shutdown_connections 2024-08-19T01:41:56.467 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:56.465+0000 7ff91c25e740 1 -- 172.21.15.83:0/3642200704 wait complete. 2024-08-19T01:41:56.467 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:56.466+0000 7ff91c25e740 1 -- start start 2024-08-19T01:41:56.467 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:56.466+0000 7ff91c25e740 1 Processor -- start 2024-08-19T01:41:56.468 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:56.466+0000 7ff91c25e740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559431e5f6e0 0x559431e72b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:41:56.468 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:56.466+0000 7ff91c25e740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559431e60000 0x559431e730d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:41:56.468 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:56.466+0000 7ff91c25e740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559431e6a8c0 0x559431e64090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:41:56.468 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:56.466+0000 7ff91c25e740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7ff9141afec0 con 0x559431e5f6e0 2024-08-19T01:41:56.469 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:56.466+0000 7ff91c25e740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7ff9141b1330 con 0x559431e6a8c0 2024-08-19T01:41:56.469 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:56.466+0000 7ff91c25e740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7ff9141b0590 con 0x559431e60000 2024-08-19T01:41:56.469 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:56.466+0000 7ff91c25e740 10 client.0 _ll_register_callbacks cb 0x7ff90c09b420 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-19T01:41:56.469 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:56.467+0000 7ff912000640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559431e60000 0x559431e730d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:41:56.469 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:56.467+0000 7ff912000640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559431e60000 0x559431e730d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.83:45970/0 (socket says 172.21.15.83:45970) 2024-08-19T01:41:56.470 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:56.467+0000 7ff912000640 1 -- 172.21.15.83:0/2481760082 learned_addr learned my addr 172.21.15.83:0/2481760082 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-08-19T01:41:56.470 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:56.467+0000 7ff913400640 1 --2- 172.21.15.83:0/2481760082 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559431e6a8c0 0x559431e64090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:41:56.470 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:56.467+0000 7ff912000640 1 -- 172.21.15.83:0/2481760082 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559431e6a8c0 msgr2=0x559431e64090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:41:56.470 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:56.467+0000 7ff912000640 1 --2- 172.21.15.83:0/2481760082 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559431e6a8c0 0x559431e64090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:41:56.470 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:56.468+0000 7ff912a00640 1 --2- 172.21.15.83:0/2481760082 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559431e5f6e0 0x559431e72b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:41:56.470 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:56.468+0000 7ff912000640 1 -- 172.21.15.83:0/2481760082 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559431e5f6e0 msgr2=0x559431e72b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:41:56.470 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:56.468+0000 7ff912000640 1 --2- 172.21.15.83:0/2481760082 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559431e5f6e0 0x559431e72b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:41:56.470 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:56.468+0000 7ff912000640 1 -- 172.21.15.83:0/2481760082 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559431e648c0 con 0x559431e60000 2024-08-19T01:41:56.470 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:56.468+0000 7ff912a00640 1 --2- 172.21.15.83:0/2481760082 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559431e5f6e0 0x559431e72b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-19T01:41:56.471 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:56.469+0000 7ff912000640 1 --2- 172.21.15.83:0/2481760082 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559431e60000 0x559431e730d0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7ff900009230 tx=0x7ff900030fc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f3bd401a4c85e350 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:41:56.471 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:56.469+0000 7ff8f3e00640 10 client.0 ms_handle_connect on v2:172.21.15.167:3300/0 2024-08-19T01:41:56.471 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:56.470+0000 7ff8f3e00640 1 -- 172.21.15.83:0/2481760082 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff900037040 con 0x559431e60000 2024-08-19T01:41:56.471 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:56.470+0000 7ff8f3e00640 1 -- 172.21.15.83:0/2481760082 <== mon.1 v2:172.21.15.167:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8269+0+0 (secure 0 0 0) 0x7ff900030960 con 0x559431e60000 2024-08-19T01:41:56.471 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:56.470+0000 7ff91c25e740 1 -- 172.21.15.83:0/2481760082 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff90402a030 con 0x559431e60000 2024-08-19T01:41:56.472 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:56.471+0000 7ff8f3e00640 1 -- 172.21.15.83:0/2481760082 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7ff90005a050 con 0x559431e60000 2024-08-19T01:41:56.472 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:56.471+0000 7ff91c25e740 10 client.25951 fetch_fsmap learned FSMap version 28 2024-08-19T01:41:56.472 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:56.471+0000 7ff91c25e740 1 -- 172.21.15.83:0/2481760082 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x559431ee4e20 con 0x559431e60000 2024-08-19T01:41:56.473 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:56.471+0000 7ff8f3e00640 1 -- 172.21.15.83:0/2481760082 <== mon.1 v2:172.21.15.167:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff90402a030 con 0x559431e60000 2024-08-19T01:41:56.473 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:56.471+0000 7ff8f3e00640 1 -- 172.21.15.83:0/2481760082 <== mon.1 v2:172.21.15.167:3300/0 5 ==== fsmap(e 28) ==== 8358+0+0 (secure 0 0 0) 0x7ff90003e730 con 0x559431e60000 2024-08-19T01:41:56.473 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:56.471+0000 7ff91c25e740 10 client.25951 fetch_fsmap finished waiting for FSMap version 28 2024-08-19T01:41:56.473 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:56.471+0000 7ff91c25e740 10 client.25951 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.g 2024-08-19T01:41:56.473 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:56.471+0000 7ff91c25e740 20 client.25951 populate_metadata read hostname 'smithi083' 2024-08-19T01:41:56.473 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:56.472+0000 7ff91c25e740 1 --2- 172.21.15.83:0/2481760082 >> [v2:172.21.15.83:6838/3239261845,v1:172.21.15.83:6839/3239261845] conn(0x559431ee55d0 0x559431ee79f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:41:56.473 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:56.472+0000 7ff91c25e740 4 client.25951 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-08-19T01:41:56.473 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:56.472+0000 7ff91c25e740 1 -- 172.21.15.83:0/2481760082 --> [v2:172.21.15.83:6838/3239261845,v1:172.21.15.83:6839/3239261845] -- command(tid 0: {"prefix": "damage ls"}) -- 0x559431d49010 con 0x559431ee55d0 2024-08-19T01:41:56.474 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:56.472+0000 7ff912a00640 1 --2- 172.21.15.83:0/2481760082 >> [v2:172.21.15.83:6838/3239261845,v1:172.21.15.83:6839/3239261845] conn(0x559431ee55d0 0x559431ee79f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:41:56.476 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:56.474+0000 7ff912a00640 1 --2- 172.21.15.83:0/2481760082 >> [v2:172.21.15.83:6838/3239261845,v1:172.21.15.83:6839/3239261845] conn(0x559431ee55d0 0x559431ee79f0 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=c555e233fdbbc5 server_cookie=71794fec60f8dc81 in_seq=0 out_seq=0 2024-08-19T01:41:56.476 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:56.474+0000 7ff8f3e00640 10 client.25951 ms_handle_connect on v2:172.21.15.83:6838/3239261845 2024-08-19T01:41:56.476 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:56.475+0000 7ff8f3e00640 1 -- 172.21.15.83:0/2481760082 <== mds.1 v2:172.21.15.83:6838/3239261845 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x559431d49010 con 0x559431ee55d0 2024-08-19T01:41:56.477 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:56.475+0000 7ff8f3e00640 10 client.25951 handle_command_reply: tid=0 2024-08-19T01:41:56.477 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:56.475+0000 7ff91c25e740 1 client.25951 shutdown 2024-08-19T01:41:56.478 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:56.476+0000 7ff91c25e740 1 -- 172.21.15.83:0/2481760082 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559431e60000 msgr2=0x559431e730d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:41:56.478 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:56.476+0000 7ff91c25e740 1 --2- 172.21.15.83:0/2481760082 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559431e60000 0x559431e730d0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7ff900009230 tx=0x7ff900030fc0 comp rx=0 tx=0).stop 2024-08-19T01:41:56.478 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:56.476+0000 7ff91c25e740 1 -- 172.21.15.83:0/2481760082 shutdown_connections 2024-08-19T01:41:56.478 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:56.476+0000 7ff91c25e740 1 --2- 172.21.15.83:0/2481760082 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559431e6a8c0 0x559431e64090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:41:56.478 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:56.476+0000 7ff91c25e740 1 --2- 172.21.15.83:0/2481760082 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559431e60000 0x559431e730d0 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:41:56.478 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:56.476+0000 7ff91c25e740 1 --2- 172.21.15.83:0/2481760082 >> [v2:172.21.15.83:6838/3239261845,v1:172.21.15.83:6839/3239261845] conn(0x559431ee55d0 0x559431ee79f0 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:41:56.478 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:56.476+0000 7ff91c25e740 1 --2- 172.21.15.83:0/2481760082 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559431e5f6e0 0x559431e72b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:41:56.478 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:56.476+0000 7ff91c25e740 1 -- 172.21.15.83:0/2481760082 >> 172.21.15.83:0/2481760082 conn(0x559431e5bc70 msgr2=0x559431e5aca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:41:56.479 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:56.476+0000 7ff8f3e00640 0 client.25951 ms_handle_reset on v2:172.21.15.83:6838/3239261845 2024-08-19T01:41:56.479 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:56.476+0000 7ff91c25e740 1 -- 172.21.15.83:0/2481760082 shutdown_connections 2024-08-19T01:41:56.479 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:56.477+0000 7ff91c25e740 1 -- 172.21.15.83:0/2481760082 wait complete. 2024-08-19T01:41:56.479 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:56.477+0000 7ff91c25e740 20 client.25951 trim_cache size 0 max 16384 2024-08-19T01:41:56.479 INFO:teuthology.orchestra.run.smithi083.stdout:[] 2024-08-19T01:41:56.480 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:56.478+0000 7ff913e00640 1 -- 172.21.15.83:0/1926967642 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7ff8f80a2c00 msgr2=0x7ff8f80a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:41:56.480 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:56.478+0000 7ff913e00640 1 --2- 172.21.15.83:0/1926967642 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7ff8f80a2c00 0x7ff8f80a50c0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7ff900005e00 tx=0x7ff90002c040 comp rx=0 tx=0).stop 2024-08-19T01:41:56.480 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:56.478+0000 7ff913e00640 1 -- 172.21.15.83:0/1926967642 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff914109900 msgr2=0x7ff914076eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:41:56.480 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:56.478+0000 7ff913e00640 1 --2- 172.21.15.83:0/1926967642 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff914109900 0x7ff914076eb0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7ff90c00e710 tx=0x7ff90c00ebd0 comp rx=0 tx=0).stop 2024-08-19T01:41:56.480 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:56.478+0000 7ff913e00640 1 -- 172.21.15.83:0/1926967642 shutdown_connections 2024-08-19T01:41:56.480 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:56.478+0000 7ff913e00640 1 --2- 172.21.15.83:0/1926967642 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7ff8f80a2c00 0x7ff8f80a50c0 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:41:56.480 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:56.478+0000 7ff913e00640 1 --2- 172.21.15.83:0/1926967642 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff914109900 0x7ff914076eb0 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:41:56.481 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:56.478+0000 7ff913e00640 1 --2- 172.21.15.83:0/1926967642 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff9141042a0 0x7ff914076970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:41:56.481 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:56.479+0000 7ff913e00640 1 --2- 172.21.15.83:0/1926967642 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff914103900 0x7ff914076430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:41:56.481 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:56.479+0000 7ff913e00640 1 -- 172.21.15.83:0/1926967642 >> 172.21.15.83:0/1926967642 conn(0x7ff9140ff640 msgr2=0x7ff9140ffec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:41:56.481 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:56.479+0000 7ff913e00640 1 -- 172.21.15.83:0/1926967642 shutdown_connections 2024-08-19T01:41:56.481 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:56.479+0000 7ff913e00640 1 -- 172.21.15.83:0/1926967642 wait complete. 2024-08-19T01:41:57.248 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833 shell --fsid bad8780e-5dc9-11ef-bccf-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-08-19T01:41:57.670 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/bad8780e-5dc9-11ef-bccf-c7b262605968/mon.a/config 2024-08-19T01:41:57.955 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:41:57 smithi083 ceph-mon[17120]: pgmap v552: 97 pgs: 97 active+clean; 481 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.5 MiB/s wr, 459 op/s 2024-08-19T01:41:57.955 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:41:57 smithi117 ceph-mon[22016]: pgmap v552: 97 pgs: 97 active+clean; 481 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.5 MiB/s wr, 459 op/s 2024-08-19T01:41:57.965 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:41:57 smithi167 ceph-mon[21875]: pgmap v552: 97 pgs: 97 active+clean; 481 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.5 MiB/s wr, 459 op/s 2024-08-19T01:41:58.255 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:58.253+0000 7fca6da00640 1 -- 172.21.15.83:0/4067567720 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fca680742c0 msgr2=0x7fca680746a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:41:58.255 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:58.253+0000 7fca6da00640 1 --2- 172.21.15.83:0/4067567720 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fca680742c0 0x7fca680746a0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fca58009860 tx=0x7fca5802a3d0 comp rx=0 tx=0).stop 2024-08-19T01:41:58.255 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:58.253+0000 7fca6da00640 1 -- 172.21.15.83:0/4067567720 shutdown_connections 2024-08-19T01:41:58.255 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:58.253+0000 7fca6da00640 1 --2- 172.21.15.83:0/4067567720 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fca681128c0 0x7fca68114cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:41:58.255 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:58.253+0000 7fca6da00640 1 --2- 172.21.15.83:0/4067567720 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fca68074be0 0x7fca68112380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:41:58.256 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:58.253+0000 7fca6da00640 1 --2- 172.21.15.83:0/4067567720 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fca680742c0 0x7fca680746a0 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:41:58.256 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:58.253+0000 7fca6da00640 1 -- 172.21.15.83:0/4067567720 >> 172.21.15.83:0/4067567720 conn(0x7fca680fd5f0 msgr2=0x7fca680ffa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:41:58.256 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:58.253+0000 7fca6da00640 1 -- 172.21.15.83:0/4067567720 shutdown_connections 2024-08-19T01:41:58.256 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:58.253+0000 7fca6da00640 1 -- 172.21.15.83:0/4067567720 wait complete. 2024-08-19T01:41:58.257 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:58.254+0000 7fca6da00640 1 Processor -- start 2024-08-19T01:41:58.257 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:58.255+0000 7fca6da00640 1 -- start start 2024-08-19T01:41:58.257 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:58.255+0000 7fca6da00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fca680742c0 0x7fca681a45b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:41:58.257 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:58.255+0000 7fca6da00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fca68074be0 0x7fca681a4af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:41:58.257 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:58.255+0000 7fca6da00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fca681128c0 0x7fca6819e680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:41:58.257 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:58.255+0000 7fca6da00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fca681182d0 con 0x7fca680742c0 2024-08-19T01:41:58.258 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:58.255+0000 7fca6da00640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fca68118150 con 0x7fca68074be0 2024-08-19T01:41:58.258 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:58.255+0000 7fca6da00640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fca68118450 con 0x7fca681128c0 2024-08-19T01:41:58.258 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:58.255+0000 7fca5e000640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fca68074be0 0x7fca681a4af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:41:58.258 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:58.255+0000 7fca5e000640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fca68074be0 0x7fca681a4af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.83:33220/0 (socket says 172.21.15.83:33220) 2024-08-19T01:41:58.258 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:58.255+0000 7fca66a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fca680742c0 0x7fca681a45b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:41:58.258 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:58.255+0000 7fca5e000640 1 -- 172.21.15.83:0/2286347413 learned_addr learned my addr 172.21.15.83:0/2286347413 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-08-19T01:41:58.258 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:58.255+0000 7fca67400640 1 --2- 172.21.15.83:0/2286347413 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fca681128c0 0x7fca6819e680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:41:58.258 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:58.256+0000 7fca5e000640 1 -- 172.21.15.83:0/2286347413 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fca681128c0 msgr2=0x7fca6819e680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:41:58.258 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:58.256+0000 7fca5e000640 1 --2- 172.21.15.83:0/2286347413 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fca681128c0 0x7fca6819e680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:41:58.258 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:58.256+0000 7fca5e000640 1 -- 172.21.15.83:0/2286347413 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fca680742c0 msgr2=0x7fca681a45b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:41:58.258 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:58.256+0000 7fca5e000640 1 --2- 172.21.15.83:0/2286347413 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fca680742c0 0x7fca681a45b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:41:58.258 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:58.256+0000 7fca5e000640 1 -- 172.21.15.83:0/2286347413 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fca6819eef0 con 0x7fca68074be0 2024-08-19T01:41:58.258 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:58.256+0000 7fca67400640 1 --2- 172.21.15.83:0/2286347413 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fca681128c0 0x7fca6819e680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-19T01:41:58.259 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:58.256+0000 7fca5e000640 1 --2- 172.21.15.83:0/2286347413 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fca68074be0 0x7fca681a4af0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fca5000d950 tx=0x7fca5000de10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:41:58.259 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:58.256+0000 7fca5fe00640 1 -- 172.21.15.83:0/2286347413 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca50017070 con 0x7fca68074be0 2024-08-19T01:41:58.259 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:58.256+0000 7fca5fe00640 1 -- 172.21.15.83:0/2286347413 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fca50009c30 con 0x7fca68074be0 2024-08-19T01:41:58.260 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:58.256+0000 7fca5fe00640 1 -- 172.21.15.83:0/2286347413 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca5000f6b0 con 0x7fca68074be0 2024-08-19T01:41:58.260 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:58.256+0000 7fca6da00640 1 -- 172.21.15.83:0/2286347413 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fca6819f1e0 con 0x7fca68074be0 2024-08-19T01:41:58.260 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:58.256+0000 7fca6da00640 1 -- 172.21.15.83:0/2286347413 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fca681ab360 con 0x7fca68074be0 2024-08-19T01:41:58.261 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:58.259+0000 7fca705c1740 1 Processor -- start 2024-08-19T01:41:58.261 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:58.259+0000 7fca5fe00640 1 -- 172.21.15.83:0/2286347413 <== mon.2 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7fca500109d0 con 0x7fca68074be0 2024-08-19T01:41:58.261 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:58.259+0000 7fca705c1740 1 -- start start 2024-08-19T01:41:58.261 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:58.260+0000 7fca705c1740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555887e624f0 0x555887e628d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:41:58.261 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:58.260+0000 7fca705c1740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x555887e62e10 0x555887e6d190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:41:58.261 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:58.260+0000 7fca705c1740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555887e6d6d0 0x555887e6fac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:41:58.261 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:58.260+0000 7fca705c1740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x555887dc7bf0 con 0x555887e624f0 2024-08-19T01:41:58.262 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:58.260+0000 7fca705c1740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x555887d5fb20 con 0x555887e6d6d0 2024-08-19T01:41:58.262 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:58.260+0000 7fca705c1740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x555887dc3bb0 con 0x555887e62e10 2024-08-19T01:41:58.263 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:58.260+0000 7fca5e000640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555887e6d6d0 0x555887e6fac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:41:58.263 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:58.260+0000 7fca67400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555887e624f0 0x555887e628d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:41:58.263 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:58.260+0000 7fca5e000640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555887e6d6d0 0x555887e6fac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.83:33228/0 (socket says 172.21.15.83:33228) 2024-08-19T01:41:58.263 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:58.260+0000 7fca5e000640 1 -- 172.21.15.83:0/1311066042 learned_addr learned my addr 172.21.15.83:0/1311066042 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-08-19T01:41:58.263 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:58.260+0000 7fca66a00640 1 --2- 172.21.15.83:0/1311066042 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x555887e62e10 0x555887e6d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:41:58.263 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:58.261+0000 7fca5e000640 1 -- 172.21.15.83:0/1311066042 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x555887e62e10 msgr2=0x555887e6d190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:41:58.263 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:58.261+0000 7fca5e000640 1 --2- 172.21.15.83:0/1311066042 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x555887e62e10 0x555887e6d190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:41:58.264 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:58.261+0000 7fca5e000640 1 -- 172.21.15.83:0/1311066042 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555887e624f0 msgr2=0x555887e628d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:41:58.264 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:58.261+0000 7fca5e000640 1 --2- 172.21.15.83:0/1311066042 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555887e624f0 0x555887e628d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:41:58.264 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:58.261+0000 7fca5e000640 1 -- 172.21.15.83:0/1311066042 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555887d667f0 con 0x555887e6d6d0 2024-08-19T01:41:58.264 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:58.261+0000 7fca5e000640 1 --2- 172.21.15.83:0/1311066042 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555887e6d6d0 0x555887e6fac0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fca500d2de0 tx=0x7fca500d7700 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1753137640fa20b server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:41:58.264 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:58.261+0000 7fca57e00640 1 -- 172.21.15.83:0/1311066042 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca500d9820 con 0x555887e6d6d0 2024-08-19T01:41:58.264 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:58.261+0000 7fca57e00640 1 -- 172.21.15.83:0/1311066042 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fca500e1400 con 0x555887e6d6d0 2024-08-19T01:41:58.264 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:58.261+0000 7fca57e00640 1 -- 172.21.15.83:0/1311066042 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca500d5610 con 0x555887e6d6d0 2024-08-19T01:41:58.264 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:58.261+0000 7fca5fe00640 1 --2- 172.21.15.83:0/2286347413 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7fca4c0a2c00 0x7fca4c0a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:41:58.264 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:58.262+0000 7fca5fe00640 1 -- 172.21.15.83:0/2286347413 <== mon.2 v2:172.21.15.117:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8269+0+0 (secure 0 0 0) 0x7fca500cecb0 con 0x7fca68074be0 2024-08-19T01:41:58.264 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:58.262+0000 7fca67400640 1 --2- 172.21.15.83:0/2286347413 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7fca4c0a2c00 0x7fca4c0a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:41:58.266 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:58.262+0000 7fca705c1740 1 -- 172.21.15.83:0/1311066042 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555887e6d6d0 msgr2=0x555887e6fac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:41:58.266 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:58.262+0000 7fca705c1740 1 --2- 172.21.15.83:0/1311066042 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555887e6d6d0 0x555887e6fac0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fca500d2de0 tx=0x7fca500d7700 comp rx=0 tx=0).stop 2024-08-19T01:41:58.266 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:58.262+0000 7fca705c1740 1 -- 172.21.15.83:0/1311066042 shutdown_connections 2024-08-19T01:41:58.266 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:58.262+0000 7fca705c1740 1 --2- 172.21.15.83:0/1311066042 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555887e6d6d0 0x555887e6fac0 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:41:58.266 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:58.262+0000 7fca705c1740 1 --2- 172.21.15.83:0/1311066042 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x555887e62e10 0x555887e6d190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:41:58.266 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:58.262+0000 7fca705c1740 1 --2- 172.21.15.83:0/1311066042 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555887e624f0 0x555887e628d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:41:58.266 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:58.262+0000 7fca705c1740 1 -- 172.21.15.83:0/1311066042 >> 172.21.15.83:0/1311066042 conn(0x555887e5ea80 msgr2=0x555887e5ee60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:41:58.267 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:58.263+0000 7fca705c1740 1 -- 172.21.15.83:0/1311066042 shutdown_connections 2024-08-19T01:41:58.267 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:58.263+0000 7fca67400640 1 --2- 172.21.15.83:0/2286347413 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7fca4c0a2c00 0x7fca4c0a50c0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7fca68076330 tx=0x7fca60011040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:41:58.267 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:58.263+0000 7fca705c1740 1 -- 172.21.15.83:0/1311066042 wait complete. 2024-08-19T01:41:58.267 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:58.263+0000 7fca705c1740 1 -- start start 2024-08-19T01:41:58.267 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:58.263+0000 7fca705c1740 1 Processor -- start 2024-08-19T01:41:58.267 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:58.264+0000 7fca705c1740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555887e624f0 0x555887e6a550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:41:58.267 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:58.264+0000 7fca705c1740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555887e62e10 0x555887e6aa90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:41:58.267 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:58.264+0000 7fca705c1740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x555887e6d6d0 0x555887e6afd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:41:58.267 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:58.264+0000 7fca705c1740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x555887e702a0 con 0x555887e624f0 2024-08-19T01:41:58.267 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:58.264+0000 7fca705c1740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x555887d82cf0 con 0x555887e62e10 2024-08-19T01:41:58.267 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:58.264+0000 7fca705c1740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x555887e70420 con 0x555887e6d6d0 2024-08-19T01:41:58.267 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:58.264+0000 7fca66a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555887e624f0 0x555887e6a550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:41:58.267 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:58.264+0000 7fca66a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555887e624f0 0x555887e6a550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:33456/0 (socket says 172.21.15.83:33456) 2024-08-19T01:41:58.268 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:58.264+0000 7fca5e000640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555887e62e10 0x555887e6aa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:41:58.268 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:58.264+0000 7fca66a00640 1 -- 172.21.15.83:0/3330199601 learned_addr learned my addr 172.21.15.83:0/3330199601 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-08-19T01:41:58.268 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:58.264+0000 7fca705c1740 10 client.0 _ll_register_callbacks cb 0x555887d6ee30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-19T01:41:58.268 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:58.264+0000 7fca67400640 1 --2- 172.21.15.83:0/3330199601 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x555887e6d6d0 0x555887e6afd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:41:58.268 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:58.264+0000 7fca66a00640 1 -- 172.21.15.83:0/3330199601 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x555887e6d6d0 msgr2=0x555887e6afd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:41:58.268 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:58.265+0000 7fca66a00640 1 --2- 172.21.15.83:0/3330199601 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x555887e6d6d0 0x555887e6afd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:41:58.268 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:58.265+0000 7fca66a00640 1 -- 172.21.15.83:0/3330199601 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555887e62e10 msgr2=0x555887e6aa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:41:58.268 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:58.265+0000 7fca66a00640 1 --2- 172.21.15.83:0/3330199601 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555887e62e10 0x555887e6aa90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:41:58.268 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:58.265+0000 7fca66a00640 1 -- 172.21.15.83:0/3330199601 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555887b393c0 con 0x555887e624f0 2024-08-19T01:41:58.268 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:58.265+0000 7fca66a00640 1 --2- 172.21.15.83:0/3330199601 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555887e624f0 0x555887e6a550 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7fca58034000 tx=0x7fca580305d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=502adc059afbc2eb server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:41:58.268 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:58.265+0000 7fca54c00640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-08-19T01:41:58.270 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:58.265+0000 7fca54c00640 1 -- 172.21.15.83:0/3330199601 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca58004050 con 0x555887e624f0 2024-08-19T01:41:58.270 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:58.265+0000 7fca54c00640 1 -- 172.21.15.83:0/3330199601 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8269+0+0 (secure 0 0 0) 0x7fca580041f0 con 0x555887e624f0 2024-08-19T01:41:58.270 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:58.265+0000 7fca705c1740 1 -- 172.21.15.83:0/3330199601 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fca500e1400 con 0x555887e624f0 2024-08-19T01:41:58.270 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:58.265+0000 7fca54c00640 1 -- 172.21.15.83:0/3330199601 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fca5804e050 con 0x555887e624f0 2024-08-19T01:41:58.270 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:58.265+0000 7fca705c1740 10 client.15975 fetch_fsmap learned FSMap version 28 2024-08-19T01:41:58.270 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:58.265+0000 7fca705c1740 1 -- 172.21.15.83:0/3330199601 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x555887e7bf40 con 0x555887e624f0 2024-08-19T01:41:58.271 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:58.266+0000 7fca54c00640 1 -- 172.21.15.83:0/3330199601 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fca500e1400 con 0x555887e624f0 2024-08-19T01:41:58.271 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:58.266+0000 7fca54c00640 1 -- 172.21.15.83:0/3330199601 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 28) ==== 8358+0+0 (secure 0 0 0) 0x7fca58004a00 con 0x555887e624f0 2024-08-19T01:41:58.271 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:58.266+0000 7fca705c1740 10 client.15975 fetch_fsmap finished waiting for FSMap version 28 2024-08-19T01:41:58.271 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:58.266+0000 7fca705c1740 10 client.15975 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-19T01:41:58.271 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:58.266+0000 7fca705c1740 20 client.15975 populate_metadata read hostname 'smithi083' 2024-08-19T01:41:58.271 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:58.266+0000 7fca705c1740 1 --2- 172.21.15.83:0/3330199601 >> [v2:172.21.15.117:6838/1967595704,v1:172.21.15.117:6839/1967595704] conn(0x555887e7d410 0x555887e7f8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:41:58.271 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:58.266+0000 7fca705c1740 4 client.15975 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-19T01:41:58.271 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:58.266+0000 7fca705c1740 1 -- 172.21.15.83:0/3330199601 --> [v2:172.21.15.117:6838/1967595704,v1:172.21.15.117:6839/1967595704] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555887dd5160 con 0x555887e7d410 2024-08-19T01:41:58.271 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:58.266+0000 7fca5e000640 1 --2- 172.21.15.83:0/3330199601 >> [v2:172.21.15.117:6838/1967595704,v1:172.21.15.117:6839/1967595704] conn(0x555887e7d410 0x555887e7f8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:41:58.271 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:58.267+0000 7fca5e000640 1 --2- 172.21.15.83:0/3330199601 >> [v2:172.21.15.117:6838/1967595704,v1:172.21.15.117:6839/1967595704] conn(0x555887e7d410 0x555887e7f8d0 crc :-1 s=READY pgs=123 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e9166d155a51084b server_cookie=d7475fa44096405d in_seq=0 out_seq=0 2024-08-19T01:41:58.271 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:58.267+0000 7fca54c00640 10 client.15975 ms_handle_connect on v2:172.21.15.117:6838/1967595704 2024-08-19T01:41:58.271 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:58.268+0000 7fca54c00640 1 -- 172.21.15.83:0/3330199601 <== mds.0 v2:172.21.15.117:6838/1967595704 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x555887dd5160 con 0x555887e7d410 2024-08-19T01:41:58.271 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:58.268+0000 7fca54c00640 10 client.15975 handle_command_reply: tid=0 2024-08-19T01:41:58.271 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:58.268+0000 7fca705c1740 1 client.15975 shutdown 2024-08-19T01:41:58.272 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:58.268+0000 7fca705c1740 1 -- 172.21.15.83:0/3330199601 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555887e624f0 msgr2=0x555887e6a550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:41:58.272 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:58.268+0000 7fca705c1740 1 --2- 172.21.15.83:0/3330199601 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555887e624f0 0x555887e6a550 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7fca58034000 tx=0x7fca580305d0 comp rx=0 tx=0).stop 2024-08-19T01:41:58.272 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:58.268+0000 7fca705c1740 1 -- 172.21.15.83:0/3330199601 shutdown_connections 2024-08-19T01:41:58.272 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:58.268+0000 7fca705c1740 1 --2- 172.21.15.83:0/3330199601 >> [v2:172.21.15.117:6838/1967595704,v1:172.21.15.117:6839/1967595704] conn(0x555887e7d410 0x555887e7f8d0 crc :-1 s=READY pgs=123 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:41:58.272 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:58.268+0000 7fca705c1740 1 --2- 172.21.15.83:0/3330199601 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x555887e6d6d0 0x555887e6afd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:41:58.272 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:58.268+0000 7fca705c1740 1 --2- 172.21.15.83:0/3330199601 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555887e62e10 0x555887e6aa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:41:58.272 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:58.268+0000 7fca705c1740 1 --2- 172.21.15.83:0/3330199601 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555887e624f0 0x555887e6a550 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:41:58.272 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:58.268+0000 7fca705c1740 1 -- 172.21.15.83:0/3330199601 >> 172.21.15.83:0/3330199601 conn(0x555887e5ea80 msgr2=0x555887e647b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:41:58.272 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:58.268+0000 7fca54c00640 0 client.15975 ms_handle_reset on v2:172.21.15.117:6838/1967595704 2024-08-19T01:41:58.272 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:58.268+0000 7fca705c1740 1 -- 172.21.15.83:0/3330199601 shutdown_connections 2024-08-19T01:41:58.272 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:58.268+0000 7fca705c1740 1 -- 172.21.15.83:0/3330199601 wait complete. 2024-08-19T01:41:58.272 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:58.269+0000 7fca705c1740 20 client.15975 trim_cache size 0 max 16384 2024-08-19T01:41:58.284 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:58.282+0000 7fca705c1740 1 Processor -- start 2024-08-19T01:41:58.284 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:58.282+0000 7fca705c1740 1 -- start start 2024-08-19T01:41:58.285 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:58.283+0000 7fca705c1740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555887e624f0 0x555887e6c4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:41:58.285 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:58.283+0000 7fca705c1740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555887e62e10 0x555887e851a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:41:58.285 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:58.283+0000 7fca705c1740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x555887e6d6d0 0x555887e856e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:41:58.285 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:58.283+0000 7fca705c1740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fca681b6340 con 0x555887e624f0 2024-08-19T01:41:58.285 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:58.283+0000 7fca705c1740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fca681b3be0 con 0x555887e62e10 2024-08-19T01:41:58.285 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:58.283+0000 7fca705c1740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fca681b1a30 con 0x555887e6d6d0 2024-08-19T01:41:58.285 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:58.283+0000 7fca66a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555887e624f0 0x555887e6c4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:41:58.285 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:58.283+0000 7fca5e000640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555887e62e10 0x555887e851a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:41:58.285 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:58.283+0000 7fca5e000640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555887e62e10 0x555887e851a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.83:33240/0 (socket says 172.21.15.83:33240) 2024-08-19T01:41:58.286 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:58.283+0000 7fca5e000640 1 -- 172.21.15.83:0/998482857 learned_addr learned my addr 172.21.15.83:0/998482857 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-08-19T01:41:58.286 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:58.283+0000 7fca67400640 1 --2- 172.21.15.83:0/998482857 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x555887e6d6d0 0x555887e856e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:41:58.286 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:58.283+0000 7fca5e000640 1 -- 172.21.15.83:0/998482857 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x555887e6d6d0 msgr2=0x555887e856e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:41:58.286 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:58.283+0000 7fca5e000640 1 --2- 172.21.15.83:0/998482857 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x555887e6d6d0 0x555887e856e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:41:58.286 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:58.283+0000 7fca5e000640 1 -- 172.21.15.83:0/998482857 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555887e624f0 msgr2=0x555887e6c4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:41:58.286 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:58.283+0000 7fca5e000640 1 --2- 172.21.15.83:0/998482857 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555887e624f0 0x555887e6c4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:41:58.286 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:58.283+0000 7fca5e000640 1 -- 172.21.15.83:0/998482857 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555887e5e8b0 con 0x555887e62e10 2024-08-19T01:41:58.286 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:58.283+0000 7fca67400640 1 --2- 172.21.15.83:0/998482857 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x555887e6d6d0 0x555887e856e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-19T01:41:58.286 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:58.284+0000 7fca5e000640 1 --2- 172.21.15.83:0/998482857 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555887e62e10 0x555887e851a0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fca500d7c00 tx=0x7fca500e1ba0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ec06d46438ac5525 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:41:58.289 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:58.284+0000 7fca57400640 1 -- 172.21.15.83:0/998482857 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca50020040 con 0x555887e62e10 2024-08-19T01:41:58.289 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:58.284+0000 7fca57400640 1 -- 172.21.15.83:0/998482857 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fca500e1c10 con 0x555887e62e10 2024-08-19T01:41:58.289 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:58.284+0000 7fca57400640 1 -- 172.21.15.83:0/998482857 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca5009adf0 con 0x555887e62e10 2024-08-19T01:41:58.289 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:58.285+0000 7fca705c1740 1 -- 172.21.15.83:0/998482857 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555887e62e10 msgr2=0x555887e851a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:41:58.289 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:58.285+0000 7fca705c1740 1 --2- 172.21.15.83:0/998482857 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555887e62e10 0x555887e851a0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fca500d7c00 tx=0x7fca500e1ba0 comp rx=0 tx=0).stop 2024-08-19T01:41:58.289 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:58.285+0000 7fca705c1740 1 -- 172.21.15.83:0/998482857 shutdown_connections 2024-08-19T01:41:58.289 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:58.285+0000 7fca705c1740 1 --2- 172.21.15.83:0/998482857 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x555887e6d6d0 0x555887e856e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:41:58.289 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:58.285+0000 7fca705c1740 1 --2- 172.21.15.83:0/998482857 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555887e62e10 0x555887e851a0 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:41:58.289 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:58.285+0000 7fca705c1740 1 --2- 172.21.15.83:0/998482857 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555887e624f0 0x555887e6c4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:41:58.289 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:58.285+0000 7fca705c1740 1 -- 172.21.15.83:0/998482857 >> 172.21.15.83:0/998482857 conn(0x555887e5ea80 msgr2=0x555887e6bdb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:41:58.289 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:58.285+0000 7fca705c1740 1 -- 172.21.15.83:0/998482857 shutdown_connections 2024-08-19T01:41:58.290 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:58.285+0000 7fca705c1740 1 -- 172.21.15.83:0/998482857 wait complete. 2024-08-19T01:41:58.290 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:58.285+0000 7fca705c1740 1 -- start start 2024-08-19T01:41:58.290 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:58.286+0000 7fca705c1740 1 Processor -- start 2024-08-19T01:41:58.290 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:58.286+0000 7fca705c1740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555887e624f0 0x555887e756d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:41:58.290 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:58.286+0000 7fca705c1740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555887e62e10 0x555887e75c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:41:58.290 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:58.286+0000 7fca705c1740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x555887e6d6d0 0x555887e66ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:41:58.290 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:58.286+0000 7fca705c1740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fca681b20a0 con 0x555887e62e10 2024-08-19T01:41:58.290 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:58.286+0000 7fca705c1740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fca681b3510 con 0x555887e624f0 2024-08-19T01:41:58.290 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:58.286+0000 7fca705c1740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fca681b2770 con 0x555887e6d6d0 2024-08-19T01:41:58.290 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:58.286+0000 7fca705c1740 10 client.0 _ll_register_callbacks cb 0x7fca58009450 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-19T01:41:58.290 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:58.286+0000 7fca66a00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555887e624f0 0x555887e756d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:41:58.290 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:58.286+0000 7fca66a00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555887e624f0 0x555887e756d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.83:33250/0 (socket says 172.21.15.83:33250) 2024-08-19T01:41:58.290 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:58.286+0000 7fca66a00640 1 -- 172.21.15.83:0/148722667 learned_addr learned my addr 172.21.15.83:0/148722667 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-08-19T01:41:58.291 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:58.286+0000 7fca5e000640 1 --2- 172.21.15.83:0/148722667 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555887e62e10 0x555887e75c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:41:58.291 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:58.287+0000 7fca67400640 1 --2- 172.21.15.83:0/148722667 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x555887e6d6d0 0x555887e66ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:41:58.291 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:58.287+0000 7fca66a00640 1 -- 172.21.15.83:0/148722667 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x555887e6d6d0 msgr2=0x555887e66ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:41:58.291 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:58.287+0000 7fca66a00640 1 --2- 172.21.15.83:0/148722667 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x555887e6d6d0 0x555887e66ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:41:58.291 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:58.287+0000 7fca66a00640 1 -- 172.21.15.83:0/148722667 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555887e62e10 msgr2=0x555887e75c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:41:58.291 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:58.287+0000 7fca66a00640 1 --2- 172.21.15.83:0/148722667 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555887e62e10 0x555887e75c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:41:58.291 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:58.287+0000 7fca66a00640 1 -- 172.21.15.83:0/148722667 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555887e670e0 con 0x555887e624f0 2024-08-19T01:41:58.291 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:58.287+0000 7fca66a00640 1 --2- 172.21.15.83:0/148722667 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555887e624f0 0x555887e756d0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fca5802a9c0 tx=0x7fca580048f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b030913dc85f77d7 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:41:58.291 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:58.287+0000 7fca54c00640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-08-19T01:41:58.291 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:58.287+0000 7fca54c00640 1 -- 172.21.15.83:0/148722667 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca58056020 con 0x555887e624f0 2024-08-19T01:41:58.291 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:58.287+0000 7fca54c00640 1 -- 172.21.15.83:0/148722667 <== mon.2 v2:172.21.15.117:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8269+0+0 (secure 0 0 0) 0x7fca58004f70 con 0x555887e624f0 2024-08-19T01:41:58.291 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:58.287+0000 7fca705c1740 1 -- 172.21.15.83:0/148722667 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fca500e1c10 con 0x555887e624f0 2024-08-19T01:41:58.293 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:58.288+0000 7fca54c00640 1 -- 172.21.15.83:0/148722667 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fca58061050 con 0x555887e624f0 2024-08-19T01:41:58.293 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:58.288+0000 7fca705c1740 10 client.26030 fetch_fsmap learned FSMap version 28 2024-08-19T01:41:58.293 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:58.288+0000 7fca705c1740 1 -- 172.21.15.83:0/148722667 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x555887ee78e0 con 0x555887e624f0 2024-08-19T01:41:58.293 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:58.288+0000 7fca54c00640 1 -- 172.21.15.83:0/148722667 <== mon.2 v2:172.21.15.117:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fca500e1c10 con 0x555887e624f0 2024-08-19T01:41:58.293 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:58.288+0000 7fca54c00640 1 -- 172.21.15.83:0/148722667 <== mon.2 v2:172.21.15.117:3300/0 5 ==== fsmap(e 28) ==== 8358+0+0 (secure 0 0 0) 0x7fca5803d430 con 0x555887e624f0 2024-08-19T01:41:58.293 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:58.288+0000 7fca705c1740 10 client.26030 fetch_fsmap finished waiting for FSMap version 28 2024-08-19T01:41:58.293 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:58.288+0000 7fca705c1740 10 client.26030 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-19T01:41:58.293 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:58.288+0000 7fca705c1740 20 client.26030 populate_metadata read hostname 'smithi083' 2024-08-19T01:41:58.294 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:58.288+0000 7fca705c1740 1 --2- 172.21.15.83:0/148722667 >> [v2:172.21.15.117:6838/1967595704,v1:172.21.15.117:6839/1967595704] conn(0x555887ee8140 0x555887eea560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:41:58.294 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:58.288+0000 7fca705c1740 4 client.26030 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-08-19T01:41:58.294 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:58.288+0000 7fca705c1740 1 -- 172.21.15.83:0/148722667 --> [v2:172.21.15.117:6838/1967595704,v1:172.21.15.117:6839/1967595704] -- command(tid 0: {"prefix": "damage ls"}) -- 0x555887e59ff0 con 0x555887ee8140 2024-08-19T01:41:58.294 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:58.288+0000 7fca5e000640 1 --2- 172.21.15.83:0/148722667 >> [v2:172.21.15.117:6838/1967595704,v1:172.21.15.117:6839/1967595704] conn(0x555887ee8140 0x555887eea560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:41:58.294 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:58.289+0000 7fca5e000640 1 --2- 172.21.15.83:0/148722667 >> [v2:172.21.15.117:6838/1967595704,v1:172.21.15.117:6839/1967595704] conn(0x555887ee8140 0x555887eea560 crc :-1 s=READY pgs=124 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ee5eeb9a7a03f0ce server_cookie=49aeb2d1b204862 in_seq=0 out_seq=0 2024-08-19T01:41:58.294 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:58.289+0000 7fca54c00640 10 client.26030 ms_handle_connect on v2:172.21.15.117:6838/1967595704 2024-08-19T01:41:58.294 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:58.290+0000 7fca54c00640 1 -- 172.21.15.83:0/148722667 <== mds.0 v2:172.21.15.117:6838/1967595704 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x555887e59ff0 con 0x555887ee8140 2024-08-19T01:41:58.294 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:58.290+0000 7fca54c00640 10 client.26030 handle_command_reply: tid=0 2024-08-19T01:41:58.294 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:58.290+0000 7fca705c1740 1 client.26030 shutdown 2024-08-19T01:41:58.294 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:58.290+0000 7fca705c1740 1 -- 172.21.15.83:0/148722667 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555887e624f0 msgr2=0x555887e756d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:41:58.294 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:58.290+0000 7fca705c1740 1 --2- 172.21.15.83:0/148722667 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555887e624f0 0x555887e756d0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fca5802a9c0 tx=0x7fca580048f0 comp rx=0 tx=0).stop 2024-08-19T01:41:58.294 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:58.290+0000 7fca705c1740 1 -- 172.21.15.83:0/148722667 shutdown_connections 2024-08-19T01:41:58.294 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:58.290+0000 7fca705c1740 1 --2- 172.21.15.83:0/148722667 >> [v2:172.21.15.117:6838/1967595704,v1:172.21.15.117:6839/1967595704] conn(0x555887ee8140 0x555887eea560 crc :-1 s=READY pgs=124 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:41:58.294 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:58.290+0000 7fca705c1740 1 --2- 172.21.15.83:0/148722667 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x555887e6d6d0 0x555887e66ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:41:58.295 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:58.290+0000 7fca705c1740 1 --2- 172.21.15.83:0/148722667 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555887e62e10 0x555887e75c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:41:58.295 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:58.290+0000 7fca705c1740 1 --2- 172.21.15.83:0/148722667 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555887e624f0 0x555887e756d0 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:41:58.295 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:58.290+0000 7fca54c00640 0 client.26030 ms_handle_reset on v2:172.21.15.117:6838/1967595704 2024-08-19T01:41:58.295 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:58.290+0000 7fca705c1740 1 -- 172.21.15.83:0/148722667 >> 172.21.15.83:0/148722667 conn(0x555887e5ea80 msgr2=0x555887e64760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:41:58.295 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:58.291+0000 7fca705c1740 1 -- 172.21.15.83:0/148722667 shutdown_connections 2024-08-19T01:41:58.295 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:58.291+0000 7fca705c1740 1 -- 172.21.15.83:0/148722667 wait complete. 2024-08-19T01:41:58.295 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:58.291+0000 7fca705c1740 20 client.26030 trim_cache size 0 max 16384 2024-08-19T01:41:58.295 INFO:teuthology.orchestra.run.smithi083.stdout:[] 2024-08-19T01:41:58.296 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:58.291+0000 7fca6da00640 1 -- 172.21.15.83:0/2286347413 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7fca4c0a2c00 msgr2=0x7fca4c0a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:41:58.296 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:58.291+0000 7fca6da00640 1 --2- 172.21.15.83:0/2286347413 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7fca4c0a2c00 0x7fca4c0a50c0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7fca68076330 tx=0x7fca60011040 comp rx=0 tx=0).stop 2024-08-19T01:41:58.296 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:58.291+0000 7fca6da00640 1 -- 172.21.15.83:0/2286347413 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fca68074be0 msgr2=0x7fca681a4af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:41:58.296 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:58.291+0000 7fca6da00640 1 --2- 172.21.15.83:0/2286347413 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fca68074be0 0x7fca681a4af0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fca5000d950 tx=0x7fca5000de10 comp rx=0 tx=0).stop 2024-08-19T01:41:58.296 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:58.291+0000 7fca6da00640 1 -- 172.21.15.83:0/2286347413 shutdown_connections 2024-08-19T01:41:58.296 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:58.291+0000 7fca6da00640 1 --2- 172.21.15.83:0/2286347413 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7fca4c0a2c00 0x7fca4c0a50c0 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:41:58.296 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:58.291+0000 7fca6da00640 1 --2- 172.21.15.83:0/2286347413 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fca681128c0 0x7fca6819e680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:41:58.296 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:58.291+0000 7fca6da00640 1 --2- 172.21.15.83:0/2286347413 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fca68074be0 0x7fca681a4af0 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:41:58.296 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:58.291+0000 7fca6da00640 1 --2- 172.21.15.83:0/2286347413 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fca680742c0 0x7fca681a45b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:41:58.297 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:58.291+0000 7fca6da00640 1 -- 172.21.15.83:0/2286347413 >> 172.21.15.83:0/2286347413 conn(0x7fca680fd5f0 msgr2=0x7fca680fed30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:41:58.297 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:58.292+0000 7fca6da00640 1 -- 172.21.15.83:0/2286347413 shutdown_connections 2024-08-19T01:41:58.297 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:41:58.292+0000 7fca6da00640 1 -- 172.21.15.83:0/2286347413 wait complete. 2024-08-19T01:41:59.952 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:41:59 smithi117 ceph-mon[22016]: pgmap v553: 97 pgs: 97 active+clean; 460 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.6 MiB/s wr, 531 op/s 2024-08-19T01:41:59.965 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:41:59 smithi167 ceph-mon[21875]: pgmap v553: 97 pgs: 97 active+clean; 460 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.6 MiB/s wr, 531 op/s 2024-08-19T01:42:00.154 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:41:59 smithi083 ceph-mon[17120]: pgmap v553: 97 pgs: 97 active+clean; 460 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.6 MiB/s wr, 531 op/s 2024-08-19T01:42:00.399 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-08-19T01:42:00.399 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833 shell --fsid bad8780e-5dc9-11ef-bccf-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-08-19T01:42:00.894 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/bad8780e-5dc9-11ef-bccf-c7b262605968/mon.a/config 2024-08-19T01:42:01.534 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:01.532+0000 7fa50a800640 1 -- 172.21.15.83:0/3964373368 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa5041068b0 msgr2=0x7fa504106c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:42:01.534 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:01.533+0000 7fa50a800640 1 --2- 172.21.15.83:0/3964373368 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa5041068b0 0x7fa504106c90 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7fa4f4009900 tx=0x7fa4f402a3f0 comp rx=0 tx=0).stop 2024-08-19T01:42:01.535 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:01.533+0000 7fa50a800640 1 -- 172.21.15.83:0/3964373368 shutdown_connections 2024-08-19T01:42:01.535 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:01.533+0000 7fa50a800640 1 --2- 172.21.15.83:0/3964373368 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa5041002d0 0x7fa50410e750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:01.535 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:01.533+0000 7fa50a800640 1 --2- 172.21.15.83:0/3964373368 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa5040ff930 0x7fa5040ffd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:01.535 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:01.533+0000 7fa50a800640 1 --2- 172.21.15.83:0/3964373368 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa5041068b0 0x7fa504106c90 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:01.536 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:01.534+0000 7fa50a800640 1 -- 172.21.15.83:0/3964373368 >> 172.21.15.83:0/3964373368 conn(0x7fa5040fb7c0 msgr2=0x7fa5040fdbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:42:01.536 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:01.534+0000 7fa50a800640 1 -- 172.21.15.83:0/3964373368 shutdown_connections 2024-08-19T01:42:01.536 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:01.534+0000 7fa50a800640 1 -- 172.21.15.83:0/3964373368 wait complete. 2024-08-19T01:42:01.536 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:01.535+0000 7fa50a800640 1 Processor -- start 2024-08-19T01:42:01.537 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:01.535+0000 7fa50a800640 1 -- start start 2024-08-19T01:42:01.537 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:01.535+0000 7fa50a800640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa5040ff930 0x7fa5041087d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:42:01.537 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:01.535+0000 7fa50a800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa5041002d0 0x7fa504108d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:42:01.537 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:01.535+0000 7fa50a800640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa5041068b0 0x7fa504109250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:42:01.537 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:01.535+0000 7fa50a800640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fa504111cf0 con 0x7fa5041002d0 2024-08-19T01:42:01.537 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:01.535+0000 7fa50a800640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fa504111b70 con 0x7fa5041068b0 2024-08-19T01:42:01.537 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:01.535+0000 7fa50a800640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fa504111e70 con 0x7fa5040ff930 2024-08-19T01:42:01.538 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:01.536+0000 7fa503400640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa5040ff930 0x7fa5041087d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:42:01.538 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:01.536+0000 7fa503400640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa5040ff930 0x7fa5041087d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.83:38566/0 (socket says 172.21.15.83:38566) 2024-08-19T01:42:01.538 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:01.536+0000 7fa503400640 1 -- 172.21.15.83:0/3176766740 learned_addr learned my addr 172.21.15.83:0/3176766740 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-08-19T01:42:01.538 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:01.536+0000 7fa502a00640 1 --2- 172.21.15.83:0/3176766740 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa5041002d0 0x7fa504108d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:42:01.539 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:01.536+0000 7fa503e00640 1 --2- 172.21.15.83:0/3176766740 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa5041068b0 0x7fa504109250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:42:01.539 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:01.536+0000 7fa503400640 1 -- 172.21.15.83:0/3176766740 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa5041068b0 msgr2=0x7fa504109250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:42:01.539 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:01.536+0000 7fa503400640 1 --2- 172.21.15.83:0/3176766740 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa5041068b0 0x7fa504109250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:01.539 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:01.536+0000 7fa503400640 1 -- 172.21.15.83:0/3176766740 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa5041002d0 msgr2=0x7fa504108d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:42:01.539 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:01.537+0000 7fa503400640 1 --2- 172.21.15.83:0/3176766740 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa5041002d0 0x7fa504108d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:01.539 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:01.537+0000 7fa503400640 1 -- 172.21.15.83:0/3176766740 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa5041aa940 con 0x7fa5040ff930 2024-08-19T01:42:01.539 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:01.537+0000 7fa502a00640 1 --2- 172.21.15.83:0/3176766740 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa5041002d0 0x7fa504108d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-19T01:42:01.540 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:01.538+0000 7fa503400640 1 --2- 172.21.15.83:0/3176766740 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa5040ff930 0x7fa5041087d0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fa4f4006010 tx=0x7fa4f4003c10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:42:01.540 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:01.538+0000 7fa4fbe00640 1 -- 172.21.15.83:0/3176766740 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4f403f070 con 0x7fa5040ff930 2024-08-19T01:42:01.540 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:01.538+0000 7fa4fbe00640 1 -- 172.21.15.83:0/3176766740 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa4f4031c20 con 0x7fa5040ff930 2024-08-19T01:42:01.540 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:01.538+0000 7fa4fbe00640 1 -- 172.21.15.83:0/3176766740 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4f4035030 con 0x7fa5040ff930 2024-08-19T01:42:01.540 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:01.538+0000 7fa50a800640 1 -- 172.21.15.83:0/3176766740 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa5041aabd0 con 0x7fa5040ff930 2024-08-19T01:42:01.540 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:01.539+0000 7fa50a800640 1 -- 172.21.15.83:0/3176766740 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa5041ab060 con 0x7fa5040ff930 2024-08-19T01:42:01.543 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:01.542+0000 7fa4fbe00640 1 -- 172.21.15.83:0/3176766740 <== mon.1 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7fa4f4042070 con 0x7fa5040ff930 2024-08-19T01:42:01.543 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:01.542+0000 7fa50d409740 1 Processor -- start 2024-08-19T01:42:01.544 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:01.542+0000 7fa50d409740 1 -- start start 2024-08-19T01:42:01.544 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:01.542+0000 7fa50d409740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564ff442a480 0x564ff442a860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:42:01.544 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:01.542+0000 7fa50d409740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564ff442ada0 0x564ff4435120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:42:01.545 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:01.542+0000 7fa50d409740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564ff4435660 0x564ff4437a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:42:01.545 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:01.542+0000 7fa50d409740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x564ff43457e0 con 0x564ff442ada0 2024-08-19T01:42:01.545 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:01.542+0000 7fa50d409740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x564ff4390ac0 con 0x564ff442a480 2024-08-19T01:42:01.545 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:01.542+0000 7fa50d409740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x564ff43288c0 con 0x564ff4435660 2024-08-19T01:42:01.545 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:01.543+0000 7fa503400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564ff442ada0 0x564ff4435120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:42:01.545 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:01.543+0000 7fa503e00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564ff442a480 0x564ff442a860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:42:01.545 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:01.543+0000 7fa503e00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564ff442a480 0x564ff442a860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.83:54862/0 (socket says 172.21.15.83:54862) 2024-08-19T01:42:01.545 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:01.543+0000 7fa503e00640 1 -- 172.21.15.83:0/1000461265 learned_addr learned my addr 172.21.15.83:0/1000461265 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-08-19T01:42:01.545 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:01.543+0000 7fa502a00640 1 --2- 172.21.15.83:0/1000461265 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564ff4435660 0x564ff4437a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:42:01.546 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:01.543+0000 7fa503400640 1 -- 172.21.15.83:0/1000461265 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564ff4435660 msgr2=0x564ff4437a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:42:01.546 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:01.543+0000 7fa503400640 1 --2- 172.21.15.83:0/1000461265 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564ff4435660 0x564ff4437a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:01.546 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:01.543+0000 7fa503400640 1 -- 172.21.15.83:0/1000461265 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564ff442a480 msgr2=0x564ff442a860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:42:01.546 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:01.543+0000 7fa503400640 1 --2- 172.21.15.83:0/1000461265 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564ff442a480 0x564ff442a860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:01.547 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:01.544+0000 7fa503400640 1 -- 172.21.15.83:0/1000461265 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564ff432f760 con 0x564ff442ada0 2024-08-19T01:42:01.547 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:01.544+0000 7fa503400640 1 --2- 172.21.15.83:0/1000461265 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564ff442ada0 0x564ff4435120 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7fa4f40ed1e0 tx=0x7fa4f40f5cb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=edd0edfe28318b7f server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:42:01.547 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:01.544+0000 7fa4fbe00640 1 --2- 172.21.15.83:0/3176766740 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7fa4e80a2c00 0x7fa4e80a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:42:01.547 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:01.544+0000 7fa4fbe00640 1 -- 172.21.15.83:0/3176766740 <== mon.1 v2:172.21.15.167:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8269+0+0 (secure 0 0 0) 0x7fa4f40ef7c0 con 0x7fa5040ff930 2024-08-19T01:42:01.547 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:01.544+0000 7fa4f8c00640 1 -- 172.21.15.83:0/1000461265 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4f40fe070 con 0x564ff442ada0 2024-08-19T01:42:01.547 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:01.544+0000 7fa4f8c00640 1 -- 172.21.15.83:0/1000461265 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa4f40bc6a0 con 0x564ff442ada0 2024-08-19T01:42:01.547 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:01.544+0000 7fa4f8c00640 1 -- 172.21.15.83:0/1000461265 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4f40bbd70 con 0x564ff442ada0 2024-08-19T01:42:01.547 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:01.545+0000 7fa503e00640 1 --2- 172.21.15.83:0/3176766740 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7fa4e80a2c00 0x7fa4e80a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:42:01.547 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:01.545+0000 7fa50d409740 1 -- 172.21.15.83:0/1000461265 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564ff442ada0 msgr2=0x564ff4435120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:42:01.547 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:01.545+0000 7fa50d409740 1 --2- 172.21.15.83:0/1000461265 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564ff442ada0 0x564ff4435120 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7fa4f40ed1e0 tx=0x7fa4f40f5cb0 comp rx=0 tx=0).stop 2024-08-19T01:42:01.548 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:01.545+0000 7fa50d409740 1 -- 172.21.15.83:0/1000461265 shutdown_connections 2024-08-19T01:42:01.548 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:01.545+0000 7fa50d409740 1 --2- 172.21.15.83:0/1000461265 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564ff4435660 0x564ff4437a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:01.548 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:01.545+0000 7fa50d409740 1 --2- 172.21.15.83:0/1000461265 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564ff442ada0 0x564ff4435120 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:01.548 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:01.545+0000 7fa50d409740 1 --2- 172.21.15.83:0/1000461265 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564ff442a480 0x564ff442a860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:01.548 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:01.545+0000 7fa50d409740 1 -- 172.21.15.83:0/1000461265 >> 172.21.15.83:0/1000461265 conn(0x564ff4426a10 msgr2=0x564ff4426df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:42:01.548 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:01.545+0000 7fa50d409740 1 -- 172.21.15.83:0/1000461265 shutdown_connections 2024-08-19T01:42:01.548 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:01.546+0000 7fa503e00640 1 --2- 172.21.15.83:0/3176766740 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7fa4e80a2c00 0x7fa4e80a50c0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fa4fc009fd0 tx=0x7fa4fc009500 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:42:01.549 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:01.546+0000 7fa50d409740 1 -- 172.21.15.83:0/1000461265 wait complete. 2024-08-19T01:42:01.549 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:01.546+0000 7fa50d409740 1 -- start start 2024-08-19T01:42:01.549 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:01.546+0000 7fa50d409740 1 Processor -- start 2024-08-19T01:42:01.549 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:01.547+0000 7fa50d409740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564ff442a480 0x564ff4432600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:42:01.550 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:01.547+0000 7fa50d409740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564ff442ada0 0x564ff4432b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:42:01.550 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:01.547+0000 7fa50d409740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564ff4435660 0x564ff4433080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:42:01.550 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:01.547+0000 7fa50d409740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x564ff434bae0 con 0x564ff442a480 2024-08-19T01:42:01.551 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:01.547+0000 7fa50d409740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x564ff438ca80 con 0x564ff4435660 2024-08-19T01:42:01.551 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:01.547+0000 7fa50d409740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x564ff4437ff0 con 0x564ff442ada0 2024-08-19T01:42:01.551 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:01.547+0000 7fa50d409740 10 client.0 _ll_register_callbacks cb 0x564ff434d190 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-19T01:42:01.551 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:01.547+0000 7fa503e00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564ff4435660 0x564ff4433080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:42:01.551 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:01.547+0000 7fa503400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564ff442a480 0x564ff4432600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:42:01.551 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:01.547+0000 7fa503e00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564ff4435660 0x564ff4433080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.83:54866/0 (socket says 172.21.15.83:54866) 2024-08-19T01:42:01.551 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:01.547+0000 7fa503e00640 1 -- 172.21.15.83:0/2565680898 learned_addr learned my addr 172.21.15.83:0/2565680898 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-08-19T01:42:01.551 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:01.547+0000 7fa502a00640 1 --2- 172.21.15.83:0/2565680898 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564ff442ada0 0x564ff4432b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:42:01.551 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:01.547+0000 7fa503e00640 1 -- 172.21.15.83:0/2565680898 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564ff442ada0 msgr2=0x564ff4432b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:42:01.551 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:01.547+0000 7fa503e00640 1 --2- 172.21.15.83:0/2565680898 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564ff442ada0 0x564ff4432b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:01.551 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:01.547+0000 7fa503e00640 1 -- 172.21.15.83:0/2565680898 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564ff442a480 msgr2=0x564ff4432600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:42:01.551 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:01.547+0000 7fa503e00640 1 --2- 172.21.15.83:0/2565680898 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564ff442a480 0x564ff4432600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:01.551 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:01.547+0000 7fa503e00640 1 -- 172.21.15.83:0/2565680898 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564ff41026a0 con 0x564ff4435660 2024-08-19T01:42:01.551 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:01.548+0000 7fa503e00640 1 --2- 172.21.15.83:0/2565680898 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564ff4435660 0x564ff4433080 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fa4fc01f9e0 tx=0x7fa4fc004ab0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6642b2ebb0d7ebb6 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:42:01.551 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:01.548+0000 7fa4ed600640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-08-19T01:42:01.552 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:01.548+0000 7fa4ed600640 1 -- 172.21.15.83:0/2565680898 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4fc02c060 con 0x564ff4435660 2024-08-19T01:42:01.552 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:01.548+0000 7fa50d409740 1 -- 172.21.15.83:0/2565680898 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa4f40bc6a0 con 0x564ff4435660 2024-08-19T01:42:01.552 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:01.548+0000 7fa4ed600640 1 -- 172.21.15.83:0/2565680898 <== mon.2 v2:172.21.15.117:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8269+0+0 (secure 0 0 0) 0x7fa4fc017030 con 0x564ff4435660 2024-08-19T01:42:01.552 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:01.548+0000 7fa4ed600640 1 -- 172.21.15.83:0/2565680898 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fa4fc004740 con 0x564ff4435660 2024-08-19T01:42:01.552 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:01.548+0000 7fa50d409740 10 client.26042 fetch_fsmap learned FSMap version 28 2024-08-19T01:42:01.552 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:01.548+0000 7fa50d409740 1 -- 172.21.15.83:0/2565680898 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x564ff4443b60 con 0x564ff4435660 2024-08-19T01:42:01.552 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:01.548+0000 7fa4ed600640 1 -- 172.21.15.83:0/2565680898 <== mon.2 v2:172.21.15.117:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa4f40bc6a0 con 0x564ff4435660 2024-08-19T01:42:01.552 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:01.549+0000 7fa4ed600640 1 -- 172.21.15.83:0/2565680898 <== mon.2 v2:172.21.15.117:3300/0 5 ==== fsmap(e 28) ==== 8358+0+0 (secure 0 0 0) 0x7fa4fc032400 con 0x564ff4435660 2024-08-19T01:42:01.552 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:01.549+0000 7fa50d409740 10 client.26042 fetch_fsmap finished waiting for FSMap version 28 2024-08-19T01:42:01.552 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:01.549+0000 7fa50d409740 10 client.26042 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-19T01:42:01.552 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:01.549+0000 7fa50d409740 20 client.26042 populate_metadata read hostname 'smithi083' 2024-08-19T01:42:01.552 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:01.549+0000 7fa50d409740 1 --2- 172.21.15.83:0/2565680898 >> [v2:172.21.15.117:6838/1967595704,v1:172.21.15.117:6839/1967595704] conn(0x564ff4444fc0 0x564ff4447410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:42:01.552 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:01.549+0000 7fa50d409740 4 client.26042 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-19T01:42:01.552 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:01.549+0000 7fa50d409740 1 -- 172.21.15.83:0/2565680898 --> [v2:172.21.15.117:6838/1967595704,v1:172.21.15.117:6839/1967595704] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564ff439e030 con 0x564ff4444fc0 2024-08-19T01:42:01.552 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:01.549+0000 7fa503400640 1 --2- 172.21.15.83:0/2565680898 >> [v2:172.21.15.117:6838/1967595704,v1:172.21.15.117:6839/1967595704] conn(0x564ff4444fc0 0x564ff4447410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:42:01.553 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:01.549+0000 7fa503400640 1 --2- 172.21.15.83:0/2565680898 >> [v2:172.21.15.117:6838/1967595704,v1:172.21.15.117:6839/1967595704] conn(0x564ff4444fc0 0x564ff4447410 crc :-1 s=READY pgs=125 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a0105e0ea4df4d11 server_cookie=ae5537a2801cfc7c in_seq=0 out_seq=0 2024-08-19T01:42:01.553 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:01.549+0000 7fa4ed600640 10 client.26042 ms_handle_connect on v2:172.21.15.117:6838/1967595704 2024-08-19T01:42:01.553 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:01.552+0000 7fa4ed600640 1 -- 172.21.15.83:0/2565680898 <== mds.0 v2:172.21.15.117:6838/1967595704 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x564ff439e030 con 0x564ff4444fc0 2024-08-19T01:42:01.553 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:01.552+0000 7fa4ed600640 10 client.26042 handle_command_reply: tid=0 2024-08-19T01:42:01.553 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:01.552+0000 7fa50d409740 1 client.26042 shutdown 2024-08-19T01:42:01.554 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:01.552+0000 7fa50d409740 1 -- 172.21.15.83:0/2565680898 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564ff4435660 msgr2=0x564ff4433080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:42:01.554 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:01.552+0000 7fa50d409740 1 --2- 172.21.15.83:0/2565680898 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564ff4435660 0x564ff4433080 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fa4fc01f9e0 tx=0x7fa4fc004ab0 comp rx=0 tx=0).stop 2024-08-19T01:42:01.554 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:01.552+0000 7fa50d409740 1 -- 172.21.15.83:0/2565680898 shutdown_connections 2024-08-19T01:42:01.554 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:01.552+0000 7fa50d409740 1 --2- 172.21.15.83:0/2565680898 >> [v2:172.21.15.117:6838/1967595704,v1:172.21.15.117:6839/1967595704] conn(0x564ff4444fc0 0x564ff4447410 crc :-1 s=READY pgs=125 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:01.554 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:01.552+0000 7fa50d409740 1 --2- 172.21.15.83:0/2565680898 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564ff4435660 0x564ff4433080 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:01.554 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:01.552+0000 7fa50d409740 1 --2- 172.21.15.83:0/2565680898 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564ff442ada0 0x564ff4432b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:01.555 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:01.552+0000 7fa50d409740 1 --2- 172.21.15.83:0/2565680898 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564ff442a480 0x564ff4432600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:01.555 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:01.552+0000 7fa50d409740 1 -- 172.21.15.83:0/2565680898 >> 172.21.15.83:0/2565680898 conn(0x564ff4426a10 msgr2=0x564ff442c740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:42:01.555 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:01.552+0000 7fa4ed600640 0 client.26042 ms_handle_reset on v2:172.21.15.117:6838/1967595704 2024-08-19T01:42:01.555 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:01.552+0000 7fa50d409740 1 -- 172.21.15.83:0/2565680898 shutdown_connections 2024-08-19T01:42:01.555 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:01.553+0000 7fa50d409740 1 -- 172.21.15.83:0/2565680898 wait complete. 2024-08-19T01:42:01.555 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:01.553+0000 7fa50d409740 20 client.26042 trim_cache size 0 max 16384 2024-08-19T01:42:01.574 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:01.573+0000 7fa50d409740 1 Processor -- start 2024-08-19T01:42:01.574 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:01.573+0000 7fa50d409740 1 -- start start 2024-08-19T01:42:01.575 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:01.573+0000 7fa50d409740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564ff442a480 0x564ff444cd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:42:01.575 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:01.573+0000 7fa50d409740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564ff442ada0 0x564ff444d260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:42:01.575 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:01.573+0000 7fa50d409740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564ff4435660 0x564ff4425350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:42:01.575 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:01.573+0000 7fa50d409740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fa5041b6270 con 0x564ff442a480 2024-08-19T01:42:01.575 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:01.573+0000 7fa50d409740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fa5041b3b10 con 0x564ff4435660 2024-08-19T01:42:01.575 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:01.573+0000 7fa50d409740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fa5041b1960 con 0x564ff442ada0 2024-08-19T01:42:01.576 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:01.573+0000 7fa503e00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564ff4435660 0x564ff4425350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:42:01.576 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:01.573+0000 7fa503400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564ff442a480 0x564ff444cd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:42:01.576 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:01.573+0000 7fa503e00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564ff4435660 0x564ff4425350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.83:54882/0 (socket says 172.21.15.83:54882) 2024-08-19T01:42:01.576 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:01.574+0000 7fa503e00640 1 -- 172.21.15.83:0/2157056436 learned_addr learned my addr 172.21.15.83:0/2157056436 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-08-19T01:42:01.576 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:01.574+0000 7fa502a00640 1 --2- 172.21.15.83:0/2157056436 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564ff442ada0 0x564ff444d260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:42:01.576 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:01.574+0000 7fa503e00640 1 -- 172.21.15.83:0/2157056436 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564ff442ada0 msgr2=0x564ff444d260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:42:01.576 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:01.574+0000 7fa503e00640 1 --2- 172.21.15.83:0/2157056436 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564ff442ada0 0x564ff444d260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:01.576 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:01.574+0000 7fa503e00640 1 -- 172.21.15.83:0/2157056436 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564ff442a480 msgr2=0x564ff444cd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:42:01.577 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:01.574+0000 7fa503e00640 1 --2- 172.21.15.83:0/2157056436 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564ff442a480 0x564ff444cd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:01.577 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:01.574+0000 7fa503e00640 1 -- 172.21.15.83:0/2157056436 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564ff442a2b0 con 0x564ff4435660 2024-08-19T01:42:01.577 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:01.574+0000 7fa503400640 1 --2- 172.21.15.83:0/2157056436 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564ff442a480 0x564ff444cd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-19T01:42:01.577 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:01.574+0000 7fa503e00640 1 --2- 172.21.15.83:0/2157056436 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564ff4435660 0x564ff4425350 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fa4fc0044d0 tx=0x7fa4fc009b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7286fdbc2effe3 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:42:01.577 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:01.574+0000 7fa4efe00640 1 -- 172.21.15.83:0/2157056436 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4fc032600 con 0x564ff4435660 2024-08-19T01:42:01.577 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:01.574+0000 7fa4efe00640 1 -- 172.21.15.83:0/2157056436 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa4fc032a60 con 0x564ff4435660 2024-08-19T01:42:01.577 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:01.574+0000 7fa4efe00640 1 -- 172.21.15.83:0/2157056436 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4fc027c50 con 0x564ff4435660 2024-08-19T01:42:01.578 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:01.575+0000 7fa50d409740 1 -- 172.21.15.83:0/2157056436 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564ff4435660 msgr2=0x564ff4425350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:42:01.578 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:01.575+0000 7fa50d409740 1 --2- 172.21.15.83:0/2157056436 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564ff4435660 0x564ff4425350 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fa4fc0044d0 tx=0x7fa4fc009b50 comp rx=0 tx=0).stop 2024-08-19T01:42:01.578 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:01.575+0000 7fa50d409740 1 -- 172.21.15.83:0/2157056436 shutdown_connections 2024-08-19T01:42:01.578 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:01.575+0000 7fa50d409740 1 --2- 172.21.15.83:0/2157056436 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564ff4435660 0x564ff4425350 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:01.578 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:01.575+0000 7fa50d409740 1 --2- 172.21.15.83:0/2157056436 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564ff442ada0 0x564ff444d260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:01.578 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:01.575+0000 7fa50d409740 1 --2- 172.21.15.83:0/2157056436 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564ff442a480 0x564ff444cd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:01.578 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:01.575+0000 7fa50d409740 1 -- 172.21.15.83:0/2157056436 >> 172.21.15.83:0/2157056436 conn(0x564ff4426a10 msgr2=0x564ff4430ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:42:01.578 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:01.575+0000 7fa50d409740 1 -- 172.21.15.83:0/2157056436 shutdown_connections 2024-08-19T01:42:01.578 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:01.575+0000 7fa50d409740 1 -- 172.21.15.83:0/2157056436 wait complete. 2024-08-19T01:42:01.578 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:01.576+0000 7fa50d409740 1 -- start start 2024-08-19T01:42:01.578 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:01.576+0000 7fa50d409740 1 Processor -- start 2024-08-19T01:42:01.580 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:01.576+0000 7fa50d409740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564ff442a480 0x564ff443d580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:42:01.580 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:01.576+0000 7fa50d409740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564ff442ada0 0x564ff4438900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:42:01.580 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:01.576+0000 7fa50d409740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564ff4435660 0x564ff4438e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:42:01.580 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:01.576+0000 7fa50d409740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fa5041b1fd0 con 0x564ff4435660 2024-08-19T01:42:01.580 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:01.576+0000 7fa50d409740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fa5041b3440 con 0x564ff442ada0 2024-08-19T01:42:01.581 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:01.576+0000 7fa50d409740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fa5041b26a0 con 0x564ff442a480 2024-08-19T01:42:01.581 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:01.577+0000 7fa50d409740 10 client.0 _ll_register_callbacks cb 0x7fa4fc01f8a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-19T01:42:01.581 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:01.577+0000 7fa502a00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564ff442ada0 0x564ff4438900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:42:01.581 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:01.577+0000 7fa502a00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564ff442ada0 0x564ff4438900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.83:54896/0 (socket says 172.21.15.83:54896) 2024-08-19T01:42:01.581 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:01.577+0000 7fa502a00640 1 -- 172.21.15.83:0/1585219953 learned_addr learned my addr 172.21.15.83:0/1585219953 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-08-19T01:42:01.581 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:01.577+0000 7fa503400640 1 --2- 172.21.15.83:0/1585219953 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564ff442a480 0x564ff443d580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:42:01.581 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:01.577+0000 7fa503e00640 1 --2- 172.21.15.83:0/1585219953 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564ff4435660 0x564ff4438e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:42:01.581 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:01.577+0000 7fa502a00640 1 -- 172.21.15.83:0/1585219953 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564ff442a480 msgr2=0x564ff443d580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:42:01.581 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:01.577+0000 7fa502a00640 1 --2- 172.21.15.83:0/1585219953 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564ff442a480 0x564ff443d580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:01.581 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:01.577+0000 7fa502a00640 1 -- 172.21.15.83:0/1585219953 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564ff4435660 msgr2=0x564ff4438e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:42:01.581 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:01.577+0000 7fa502a00640 1 --2- 172.21.15.83:0/1585219953 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564ff4435660 0x564ff4438e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:01.581 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:01.577+0000 7fa502a00640 1 -- 172.21.15.83:0/1585219953 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564ff4439380 con 0x564ff442ada0 2024-08-19T01:42:01.581 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:01.577+0000 7fa503e00640 1 --2- 172.21.15.83:0/1585219953 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564ff4435660 0x564ff4438e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-19T01:42:01.582 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:01.577+0000 7fa502a00640 1 --2- 172.21.15.83:0/1585219953 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564ff442ada0 0x564ff4438900 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7fa504109f90 tx=0x7fa4f000fe80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b309aaf3b348c9b9 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:42:01.582 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:01.577+0000 7fa4ed600640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-08-19T01:42:01.582 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:01.578+0000 7fa4ed600640 1 -- 172.21.15.83:0/1585219953 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4f000ed90 con 0x564ff442ada0 2024-08-19T01:42:01.582 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:01.578+0000 7fa4ed600640 1 -- 172.21.15.83:0/1585219953 <== mon.2 v2:172.21.15.117:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8269+0+0 (secure 0 0 0) 0x7fa4f0005020 con 0x564ff442ada0 2024-08-19T01:42:01.582 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:01.578+0000 7fa50d409740 1 -- 172.21.15.83:0/1585219953 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa4fc032a60 con 0x564ff442ada0 2024-08-19T01:42:01.582 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:01.578+0000 7fa4ed600640 1 -- 172.21.15.83:0/1585219953 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fa4f0005220 con 0x564ff442ada0 2024-08-19T01:42:01.582 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:01.578+0000 7fa50d409740 10 client.26048 fetch_fsmap learned FSMap version 28 2024-08-19T01:42:01.582 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:01.578+0000 7fa50d409740 1 -- 172.21.15.83:0/1585219953 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x564ff44afad0 con 0x564ff442ada0 2024-08-19T01:42:01.582 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:01.578+0000 7fa4ed600640 1 -- 172.21.15.83:0/1585219953 <== mon.2 v2:172.21.15.117:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa4fc032a60 con 0x564ff442ada0 2024-08-19T01:42:01.582 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:01.578+0000 7fa4ed600640 1 -- 172.21.15.83:0/1585219953 <== mon.2 v2:172.21.15.117:3300/0 5 ==== fsmap(e 28) ==== 8358+0+0 (secure 0 0 0) 0x7fa4f001a070 con 0x564ff442ada0 2024-08-19T01:42:01.582 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:01.578+0000 7fa50d409740 10 client.26048 fetch_fsmap finished waiting for FSMap version 28 2024-08-19T01:42:01.582 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:01.578+0000 7fa50d409740 10 client.26048 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-19T01:42:01.582 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:01.578+0000 7fa50d409740 20 client.26048 populate_metadata read hostname 'smithi083' 2024-08-19T01:42:01.582 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:01.578+0000 7fa50d409740 1 --2- 172.21.15.83:0/1585219953 >> [v2:172.21.15.117:6838/1967595704,v1:172.21.15.117:6839/1967595704] conn(0x564ff44b0360 0x564ff44b2780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:42:01.583 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:01.578+0000 7fa50d409740 4 client.26048 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-08-19T01:42:01.583 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:01.578+0000 7fa50d409740 1 -- 172.21.15.83:0/1585219953 --> [v2:172.21.15.117:6838/1967595704,v1:172.21.15.117:6839/1967595704] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x564ff4398540 con 0x564ff44b0360 2024-08-19T01:42:01.583 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:01.579+0000 7fa503400640 1 --2- 172.21.15.83:0/1585219953 >> [v2:172.21.15.117:6838/1967595704,v1:172.21.15.117:6839/1967595704] conn(0x564ff44b0360 0x564ff44b2780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:42:01.583 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:01.579+0000 7fa503400640 1 --2- 172.21.15.83:0/1585219953 >> [v2:172.21.15.117:6838/1967595704,v1:172.21.15.117:6839/1967595704] conn(0x564ff44b0360 0x564ff44b2780 crc :-1 s=READY pgs=126 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=edc1e5cff2a9a424 server_cookie=c35f9d19e70acec3 in_seq=0 out_seq=0 2024-08-19T01:42:01.583 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:01.579+0000 7fa4ed600640 10 client.26048 ms_handle_connect on v2:172.21.15.117:6838/1967595704 2024-08-19T01:42:01.584 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:01.580+0000 7fa4ed600640 1 -- 172.21.15.83:0/1585219953 <== mds.0 v2:172.21.15.117:6838/1967595704 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x564ff4398540 con 0x564ff44b0360 2024-08-19T01:42:01.584 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:01.580+0000 7fa4ed600640 10 client.26048 handle_command_reply: tid=0 2024-08-19T01:42:01.584 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:01.580+0000 7fa50d409740 1 client.26048 shutdown 2024-08-19T01:42:01.584 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:01.580+0000 7fa50d409740 1 -- 172.21.15.83:0/1585219953 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564ff442ada0 msgr2=0x564ff4438900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:42:01.584 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:01.580+0000 7fa50d409740 1 --2- 172.21.15.83:0/1585219953 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564ff442ada0 0x564ff4438900 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7fa504109f90 tx=0x7fa4f000fe80 comp rx=0 tx=0).stop 2024-08-19T01:42:01.584 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:01.580+0000 7fa50d409740 1 -- 172.21.15.83:0/1585219953 shutdown_connections 2024-08-19T01:42:01.584 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:01.580+0000 7fa50d409740 1 --2- 172.21.15.83:0/1585219953 >> [v2:172.21.15.117:6838/1967595704,v1:172.21.15.117:6839/1967595704] conn(0x564ff44b0360 0x564ff44b2780 crc :-1 s=READY pgs=126 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:01.584 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:01.580+0000 7fa50d409740 1 --2- 172.21.15.83:0/1585219953 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564ff4435660 0x564ff4438e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:01.584 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:01.580+0000 7fa50d409740 1 --2- 172.21.15.83:0/1585219953 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564ff442ada0 0x564ff4438900 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:01.585 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:01.580+0000 7fa50d409740 1 --2- 172.21.15.83:0/1585219953 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564ff442a480 0x564ff443d580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:01.585 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:01.580+0000 7fa50d409740 1 -- 172.21.15.83:0/1585219953 >> 172.21.15.83:0/1585219953 conn(0x564ff4426a10 msgr2=0x564ff442c740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:42:01.585 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:01.580+0000 7fa4ed600640 0 client.26048 ms_handle_reset on v2:172.21.15.117:6838/1967595704 2024-08-19T01:42:01.585 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:01.580+0000 7fa50d409740 1 -- 172.21.15.83:0/1585219953 shutdown_connections 2024-08-19T01:42:01.585 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:01.580+0000 7fa50d409740 1 -- 172.21.15.83:0/1585219953 wait complete. 2024-08-19T01:42:01.585 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:01.581+0000 7fa50d409740 20 client.26048 trim_cache size 0 max 16384 2024-08-19T01:42:01.585 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-08-19T01:42:01.585 INFO:teuthology.orchestra.run.smithi083.stdout: "return_code": 0, 2024-08-19T01:42:01.585 INFO:teuthology.orchestra.run.smithi083.stdout: "scrub_tag": "f1269cca-7069-4417-baf3-393540b92a77", 2024-08-19T01:42:01.585 INFO:teuthology.orchestra.run.smithi083.stdout: "mode": "asynchronous" 2024-08-19T01:42:01.586 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-08-19T01:42:01.586 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:01.581+0000 7fa50a800640 1 -- 172.21.15.83:0/3176766740 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7fa4e80a2c00 msgr2=0x7fa4e80a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:42:01.586 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:01.581+0000 7fa50a800640 1 --2- 172.21.15.83:0/3176766740 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7fa4e80a2c00 0x7fa4e80a50c0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fa4fc009fd0 tx=0x7fa4fc009500 comp rx=0 tx=0).stop 2024-08-19T01:42:01.586 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:01.581+0000 7fa50a800640 1 -- 172.21.15.83:0/3176766740 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa5040ff930 msgr2=0x7fa5041087d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:42:01.586 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:01.581+0000 7fa50a800640 1 --2- 172.21.15.83:0/3176766740 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa5040ff930 0x7fa5041087d0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fa4f4006010 tx=0x7fa4f4003c10 comp rx=0 tx=0).stop 2024-08-19T01:42:01.586 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:01.581+0000 7fa50a800640 1 -- 172.21.15.83:0/3176766740 shutdown_connections 2024-08-19T01:42:01.586 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:01.581+0000 7fa50a800640 1 --2- 172.21.15.83:0/3176766740 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7fa4e80a2c00 0x7fa4e80a50c0 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:01.586 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:01.581+0000 7fa50a800640 1 --2- 172.21.15.83:0/3176766740 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa5041068b0 0x7fa504109250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:01.587 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:01.581+0000 7fa50a800640 1 --2- 172.21.15.83:0/3176766740 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa5041002d0 0x7fa504108d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:01.587 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:01.581+0000 7fa50a800640 1 --2- 172.21.15.83:0/3176766740 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa5040ff930 0x7fa5041087d0 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:01.587 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:01.581+0000 7fa50a800640 1 -- 172.21.15.83:0/3176766740 >> 172.21.15.83:0/3176766740 conn(0x7fa5040fb7c0 msgr2=0x7fa50410cbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:42:01.587 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:01.582+0000 7fa50a800640 1 -- 172.21.15.83:0/3176766740 shutdown_connections 2024-08-19T01:42:01.587 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:01.582+0000 7fa50a800640 1 -- 172.21.15.83:0/3176766740 wait complete. 2024-08-19T01:42:01.952 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:42:01 smithi117 ceph-mon[22016]: pgmap v554: 97 pgs: 97 active+clean; 456 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.2 MiB/s wr, 449 op/s 2024-08-19T01:42:01.965 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:42:01 smithi167 ceph-mon[21875]: pgmap v554: 97 pgs: 97 active+clean; 456 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.2 MiB/s wr, 449 op/s 2024-08-19T01:42:02.029 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:02.027+0000 7f062fa00640 1 -- 172.21.15.83:0/3324983214 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0628103a90 msgr2=0x7f0628103e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:42:02.029 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:02.027+0000 7f062fa00640 1 --2- 172.21.15.83:0/3324983214 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0628103a90 0x7f0628103e90 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f061c009900 tx=0x7f061c02a410 comp rx=0 tx=0).stop 2024-08-19T01:42:02.029 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:02.027+0000 7f062fa00640 1 -- 172.21.15.83:0/3324983214 shutdown_connections 2024-08-19T01:42:02.029 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:02.027+0000 7f062fa00640 1 --2- 172.21.15.83:0/3324983214 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0628105640 0x7f062810bed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:02.029 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:02.027+0000 7f062fa00640 1 --2- 172.21.15.83:0/3324983214 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0628104c80 0x7f0628105100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:02.029 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:02.027+0000 7f062fa00640 1 --2- 172.21.15.83:0/3324983214 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0628103a90 0x7f0628103e90 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:02.029 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:02.027+0000 7f062fa00640 1 -- 172.21.15.83:0/3324983214 >> 172.21.15.83:0/3324983214 conn(0x7f06280ff210 msgr2=0x7f0628101630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:42:02.029 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:02.027+0000 7f062fa00640 1 -- 172.21.15.83:0/3324983214 shutdown_connections 2024-08-19T01:42:02.029 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:02.027+0000 7f062fa00640 1 -- 172.21.15.83:0/3324983214 wait complete. 2024-08-19T01:42:02.031 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:02.028+0000 7f062fa00640 1 Processor -- start 2024-08-19T01:42:02.031 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:02.028+0000 7f062fa00640 1 -- start start 2024-08-19T01:42:02.031 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:02.029+0000 7f062fa00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0628103a90 0x7f0628199d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:42:02.031 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:02.029+0000 7f062fa00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0628104c80 0x7f062819a260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:42:02.031 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:02.029+0000 7f062fa00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0628105640 0x7f06281a1290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:42:02.031 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:02.029+0000 7f062fa00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f062810f740 con 0x7f0628105640 2024-08-19T01:42:02.032 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:02.029+0000 7f062fa00640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f062810f5c0 con 0x7f0628104c80 2024-08-19T01:42:02.032 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:02.029+0000 7f062fa00640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f062810f8c0 con 0x7f0628103a90 2024-08-19T01:42:02.032 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:02.029+0000 7f0627e00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0628104c80 0x7f062819a260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:42:02.032 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:02.029+0000 7f0627e00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0628104c80 0x7f062819a260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.83:54906/0 (socket says 172.21.15.83:54906) 2024-08-19T01:42:02.032 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:02.029+0000 7f0627e00640 1 -- 172.21.15.83:0/2949498541 learned_addr learned my addr 172.21.15.83:0/2949498541 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-08-19T01:42:02.032 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:02.029+0000 7f062d000640 1 --2- 172.21.15.83:0/2949498541 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0628103a90 0x7f0628199d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:42:02.032 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:02.029+0000 7f062da00640 1 --2- 172.21.15.83:0/2949498541 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0628105640 0x7f06281a1290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:42:02.032 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:02.029+0000 7f0627e00640 1 -- 172.21.15.83:0/2949498541 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0628103a90 msgr2=0x7f0628199d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:42:02.032 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:02.029+0000 7f0627e00640 1 --2- 172.21.15.83:0/2949498541 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0628103a90 0x7f0628199d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:02.032 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:02.029+0000 7f0627e00640 1 -- 172.21.15.83:0/2949498541 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0628105640 msgr2=0x7f06281a1290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:42:02.032 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:02.029+0000 7f0627e00640 1 --2- 172.21.15.83:0/2949498541 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0628105640 0x7f06281a1290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:02.032 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:02.029+0000 7f0627e00640 1 -- 172.21.15.83:0/2949498541 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f06281a1930 con 0x7f0628104c80 2024-08-19T01:42:02.033 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:02.030+0000 7f0627e00640 1 --2- 172.21.15.83:0/2949498541 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0628104c80 0x7f062819a260 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f061400d950 tx=0x7f061400de10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:42:02.033 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:02.030+0000 7f0625600640 1 -- 172.21.15.83:0/2949498541 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0614017030 con 0x7f0628104c80 2024-08-19T01:42:02.033 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:02.030+0000 7f0625600640 1 -- 172.21.15.83:0/2949498541 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0614004020 con 0x7f0628104c80 2024-08-19T01:42:02.033 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:02.030+0000 7f062fa00640 1 -- 172.21.15.83:0/2949498541 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f06281a1c20 con 0x7f0628104c80 2024-08-19T01:42:02.034 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:02.030+0000 7f0625600640 1 -- 172.21.15.83:0/2949498541 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06140047a0 con 0x7f0628104c80 2024-08-19T01:42:02.034 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:02.030+0000 7f062fa00640 1 -- 172.21.15.83:0/2949498541 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f06281a2190 con 0x7f0628104c80 2024-08-19T01:42:02.036 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:02.034+0000 7f06329c2740 1 Processor -- start 2024-08-19T01:42:02.036 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:02.034+0000 7f06329c2740 1 -- start start 2024-08-19T01:42:02.037 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:02.035+0000 7f06329c2740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55eb38487470 0x55eb38487870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:42:02.037 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:02.035+0000 7f06329c2740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55eb38487db0 0x55eb3848c870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:42:02.037 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:02.035+0000 7f06329c2740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55eb3848ce40 0x55eb3848f300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:42:02.037 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:02.035+0000 7f06329c2740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55eb383eb890 con 0x55eb38487470 2024-08-19T01:42:02.037 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:02.035+0000 7f06329c2740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55eb38383d90 con 0x55eb3848ce40 2024-08-19T01:42:02.037 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:02.035+0000 7f06329c2740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55eb383e7850 con 0x55eb38487db0 2024-08-19T01:42:02.037 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:02.035+0000 7f062d000640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55eb38487db0 0x55eb3848c870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:42:02.038 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:02.035+0000 7f062da00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55eb38487470 0x55eb38487870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:42:02.038 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:02.035+0000 7f062d000640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55eb38487db0 0x55eb3848c870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.83:38640/0 (socket says 172.21.15.83:38640) 2024-08-19T01:42:02.038 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:02.035+0000 7f062d000640 1 -- 172.21.15.83:0/4064197535 learned_addr learned my addr 172.21.15.83:0/4064197535 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-08-19T01:42:02.038 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:02.035+0000 7f0625600640 1 -- 172.21.15.83:0/2949498541 <== mon.2 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7f06140041d0 con 0x7f0628104c80 2024-08-19T01:42:02.038 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:02.036+0000 7f062d000640 1 -- 172.21.15.83:0/4064197535 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55eb3848ce40 msgr2=0x55eb3848f300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:42:02.038 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:02.036+0000 7f062d000640 1 --2- 172.21.15.83:0/4064197535 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55eb3848ce40 0x55eb3848f300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:02.038 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:02.036+0000 7f062d000640 1 -- 172.21.15.83:0/4064197535 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55eb38487470 msgr2=0x55eb38487870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:42:02.038 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:02.036+0000 7f062d000640 1 --2- 172.21.15.83:0/4064197535 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55eb38487470 0x55eb38487870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:02.038 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:02.036+0000 7f062d000640 1 -- 172.21.15.83:0/4064197535 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55eb380ff210 con 0x55eb38487db0 2024-08-19T01:42:02.043 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:02.036+0000 7f062d000640 1 --2- 172.21.15.83:0/4064197535 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55eb38487db0 0x55eb3848c870 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f061c031440 tx=0x7f061c004170 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1dd1850dd5a70049 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:42:02.043 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:02.036+0000 7f061a000640 1 -- 172.21.15.83:0/4064197535 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f061c035040 con 0x55eb38487db0 2024-08-19T01:42:02.043 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:02.036+0000 7f061a000640 1 -- 172.21.15.83:0/4064197535 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f061c005520 con 0x55eb38487db0 2024-08-19T01:42:02.043 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:02.036+0000 7f061a000640 1 -- 172.21.15.83:0/4064197535 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f061c031910 con 0x55eb38487db0 2024-08-19T01:42:02.043 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:02.037+0000 7f0625600640 1 --2- 172.21.15.83:0/2949498541 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f06100a2cb0 0x7f06100a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:42:02.043 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:02.037+0000 7f0625600640 1 -- 172.21.15.83:0/2949498541 <== mon.2 v2:172.21.15.117:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8269+0+0 (secure 0 0 0) 0x7f06140ceaa0 con 0x7f0628104c80 2024-08-19T01:42:02.043 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:02.037+0000 7f06329c2740 1 -- 172.21.15.83:0/4064197535 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55eb38487db0 msgr2=0x55eb3848c870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:42:02.043 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:02.037+0000 7f06329c2740 1 --2- 172.21.15.83:0/4064197535 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55eb38487db0 0x55eb3848c870 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f061c031440 tx=0x7f061c004170 comp rx=0 tx=0).stop 2024-08-19T01:42:02.044 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:02.037+0000 7f062da00640 1 --2- 172.21.15.83:0/2949498541 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f06100a2cb0 0x7f06100a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:42:02.044 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:02.037+0000 7f06329c2740 1 -- 172.21.15.83:0/4064197535 shutdown_connections 2024-08-19T01:42:02.044 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:02.037+0000 7f06329c2740 1 --2- 172.21.15.83:0/4064197535 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55eb3848ce40 0x55eb3848f300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:02.044 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:02.037+0000 7f06329c2740 1 --2- 172.21.15.83:0/4064197535 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55eb38487db0 0x55eb3848c870 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:02.044 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:02.037+0000 7f06329c2740 1 --2- 172.21.15.83:0/4064197535 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55eb38487470 0x55eb38487870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:02.044 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:02.037+0000 7f06329c2740 1 -- 172.21.15.83:0/4064197535 >> 172.21.15.83:0/4064197535 conn(0x55eb38482ae0 msgr2=0x55eb38482ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:42:02.044 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:02.037+0000 7f06329c2740 1 -- 172.21.15.83:0/4064197535 shutdown_connections 2024-08-19T01:42:02.044 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:02.037+0000 7f06329c2740 1 -- 172.21.15.83:0/4064197535 wait complete. 2024-08-19T01:42:02.044 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:02.038+0000 7f062da00640 1 --2- 172.21.15.83:0/2949498541 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f06100a2cb0 0x7f06100a5170 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f0620005fd0 tx=0x7f0620009500 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:42:02.044 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:02.038+0000 7f06329c2740 1 -- start start 2024-08-19T01:42:02.044 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:02.038+0000 7f06329c2740 1 Processor -- start 2024-08-19T01:42:02.045 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:02.038+0000 7f06329c2740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55eb38487470 0x55eb38491730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:42:02.045 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:02.038+0000 7f06329c2740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55eb38487db0 0x55eb38491c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:42:02.045 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:02.038+0000 7f06329c2740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55eb3848ce40 0x55eb38498cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:42:02.045 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:02.038+0000 7f06329c2740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55eb3848fe50 con 0x55eb38487470 2024-08-19T01:42:02.045 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:02.038+0000 7f06329c2740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55eb383a6f70 con 0x55eb3848ce40 2024-08-19T01:42:02.045 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:02.038+0000 7f06329c2740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55eb3848ffd0 con 0x55eb38487db0 2024-08-19T01:42:02.045 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:02.039+0000 7f06329c2740 10 client.0 _ll_register_callbacks cb 0x55eb383e38d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-19T01:42:02.045 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:02.039+0000 7f062d000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55eb38487470 0x55eb38491730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:42:02.045 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:02.039+0000 7f062da00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55eb3848ce40 0x55eb38498cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:42:02.045 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:02.039+0000 7f062da00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55eb3848ce40 0x55eb38498cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.83:54922/0 (socket says 172.21.15.83:54922) 2024-08-19T01:42:02.045 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:02.039+0000 7f062da00640 1 -- 172.21.15.83:0/3454651753 learned_addr learned my addr 172.21.15.83:0/3454651753 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-08-19T01:42:02.045 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:02.039+0000 7f0627e00640 1 --2- 172.21.15.83:0/3454651753 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55eb38487db0 0x55eb38491c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:42:02.045 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:02.039+0000 7f062d000640 1 -- 172.21.15.83:0/3454651753 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55eb38487db0 msgr2=0x55eb38491c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:42:02.045 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:02.039+0000 7f062d000640 1 --2- 172.21.15.83:0/3454651753 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55eb38487db0 0x55eb38491c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:02.046 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:02.039+0000 7f062d000640 1 -- 172.21.15.83:0/3454651753 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55eb3848ce40 msgr2=0x55eb38498cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:42:02.046 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:02.039+0000 7f062d000640 1 --2- 172.21.15.83:0/3454651753 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55eb3848ce40 0x55eb38498cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:02.046 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:02.039+0000 7f062d000640 1 -- 172.21.15.83:0/3454651753 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55eb381cb620 con 0x55eb38487470 2024-08-19T01:42:02.046 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:02.040+0000 7f062d000640 1 --2- 172.21.15.83:0/3454651753 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55eb38487470 0x55eb38491730 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f061c02aa00 tx=0x7f061c004590 comp rx=0 tx=0).ready entity=mon.0 client_cookie=31dcba375d18f0fd server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:42:02.046 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:02.040+0000 7f060ea00640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-08-19T01:42:02.046 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:02.040+0000 7f060ea00640 1 -- 172.21.15.83:0/3454651753 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f061c053060 con 0x55eb38487470 2024-08-19T01:42:02.046 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:02.040+0000 7f060ea00640 1 -- 172.21.15.83:0/3454651753 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8269+0+0 (secure 0 0 0) 0x7f061c004980 con 0x55eb38487470 2024-08-19T01:42:02.046 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:02.040+0000 7f06329c2740 1 -- 172.21.15.83:0/3454651753 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f061c005520 con 0x55eb38487470 2024-08-19T01:42:02.046 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:02.040+0000 7f060ea00640 1 -- 172.21.15.83:0/3454651753 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f061c035050 con 0x55eb38487470 2024-08-19T01:42:02.046 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:02.040+0000 7f06329c2740 10 client.16008 fetch_fsmap learned FSMap version 28 2024-08-19T01:42:02.046 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:02.040+0000 7f06329c2740 1 -- 172.21.15.83:0/3454651753 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55eb3849be30 con 0x55eb38487470 2024-08-19T01:42:02.046 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:02.040+0000 7f060ea00640 1 -- 172.21.15.83:0/3454651753 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f061c005520 con 0x55eb38487470 2024-08-19T01:42:02.047 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:02.041+0000 7f060ea00640 1 -- 172.21.15.83:0/3454651753 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 28) ==== 8358+0+0 (secure 0 0 0) 0x7f061c031c30 con 0x55eb38487470 2024-08-19T01:42:02.047 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:02.041+0000 7f06329c2740 10 client.16008 fetch_fsmap finished waiting for FSMap version 28 2024-08-19T01:42:02.047 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:02.041+0000 7f06329c2740 10 client.16008 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-08-19T01:42:02.047 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:02.041+0000 7f06329c2740 20 client.16008 populate_metadata read hostname 'smithi083' 2024-08-19T01:42:02.047 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:02.041+0000 7f06329c2740 1 --2- 172.21.15.83:0/3454651753 >> [v2:172.21.15.117:6838/1967595704,v1:172.21.15.117:6839/1967595704] conn(0x55eb3849d180 0x55eb3849f570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:42:02.047 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:02.041+0000 7f06329c2740 4 client.16008 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-19T01:42:02.047 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:02.041+0000 7f06329c2740 1 -- 172.21.15.83:0/3454651753 --> [v2:172.21.15.117:6838/1967595704,v1:172.21.15.117:6839/1967595704] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55eb383f8fc0 con 0x55eb3849d180 2024-08-19T01:42:02.047 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:02.041+0000 7f0627e00640 1 --2- 172.21.15.83:0/3454651753 >> [v2:172.21.15.117:6838/1967595704,v1:172.21.15.117:6839/1967595704] conn(0x55eb3849d180 0x55eb3849f570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:42:02.047 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:02.041+0000 7f0627e00640 1 --2- 172.21.15.83:0/3454651753 >> [v2:172.21.15.117:6838/1967595704,v1:172.21.15.117:6839/1967595704] conn(0x55eb3849d180 0x55eb3849f570 crc :-1 s=READY pgs=127 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=71c3722c65ed834d server_cookie=ee0a3d318c49cb1d in_seq=0 out_seq=0 2024-08-19T01:42:02.047 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:02.041+0000 7f060ea00640 10 client.16008 ms_handle_connect on v2:172.21.15.117:6838/1967595704 2024-08-19T01:42:02.063 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:02.061+0000 7f060ea00640 1 -- 172.21.15.83:0/3454651753 <== mds.0 v2:172.21.15.117:6838/1967595704 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55eb383f8fc0 con 0x55eb3849d180 2024-08-19T01:42:02.063 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:02.061+0000 7f060ea00640 10 client.16008 handle_command_reply: tid=0 2024-08-19T01:42:02.063 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:02.061+0000 7f06329c2740 1 client.16008 shutdown 2024-08-19T01:42:02.063 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:02.061+0000 7f06329c2740 1 -- 172.21.15.83:0/3454651753 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55eb38487470 msgr2=0x55eb38491730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:42:02.063 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:02.061+0000 7f06329c2740 1 --2- 172.21.15.83:0/3454651753 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55eb38487470 0x55eb38491730 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f061c02aa00 tx=0x7f061c004590 comp rx=0 tx=0).stop 2024-08-19T01:42:02.064 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:02.062+0000 7f06329c2740 1 -- 172.21.15.83:0/3454651753 shutdown_connections 2024-08-19T01:42:02.064 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:02.062+0000 7f06329c2740 1 --2- 172.21.15.83:0/3454651753 >> [v2:172.21.15.117:6838/1967595704,v1:172.21.15.117:6839/1967595704] conn(0x55eb3849d180 0x55eb3849f570 crc :-1 s=READY pgs=127 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:02.064 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:02.062+0000 7f06329c2740 1 --2- 172.21.15.83:0/3454651753 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55eb3848ce40 0x55eb38498cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:02.064 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:02.062+0000 7f06329c2740 1 --2- 172.21.15.83:0/3454651753 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55eb38487db0 0x55eb38491c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:02.064 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:02.062+0000 7f060ea00640 0 client.16008 ms_handle_reset on v2:172.21.15.117:6838/1967595704 2024-08-19T01:42:02.064 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:02.062+0000 7f06329c2740 1 --2- 172.21.15.83:0/3454651753 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55eb38487470 0x55eb38491730 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:02.064 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:02.062+0000 7f06329c2740 1 -- 172.21.15.83:0/3454651753 >> 172.21.15.83:0/3454651753 conn(0x55eb38482ae0 msgr2=0x55eb384842f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:42:02.064 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:02.062+0000 7f06329c2740 1 -- 172.21.15.83:0/3454651753 shutdown_connections 2024-08-19T01:42:02.064 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:02.062+0000 7f06329c2740 1 -- 172.21.15.83:0/3454651753 wait complete. 2024-08-19T01:42:02.064 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:02.062+0000 7f06329c2740 20 client.16008 trim_cache size 0 max 16384 2024-08-19T01:42:02.079 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:02.077+0000 7f06329c2740 1 Processor -- start 2024-08-19T01:42:02.079 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:02.077+0000 7f06329c2740 1 -- start start 2024-08-19T01:42:02.079 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:02.077+0000 7f06329c2740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55eb38487470 0x55eb38491bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:42:02.079 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:02.077+0000 7f06329c2740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55eb38487db0 0x55eb38498c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:42:02.079 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:02.077+0000 7f06329c2740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55eb3848ce40 0x55eb38482ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:42:02.079 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:02.077+0000 7f06329c2740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f06281ad850 con 0x55eb3848ce40 2024-08-19T01:42:02.079 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:02.077+0000 7f06329c2740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f06281ab0f0 con 0x55eb38487db0 2024-08-19T01:42:02.079 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:02.077+0000 7f06329c2740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f06281a8f40 con 0x55eb38487470 2024-08-19T01:42:02.079 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:02.077+0000 7f062d000640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55eb38487470 0x55eb38491bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:42:02.079 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:02.077+0000 7f0627e00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55eb38487db0 0x55eb38498c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:42:02.080 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:02.077+0000 7f062da00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55eb3848ce40 0x55eb38482ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:42:02.080 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:02.077+0000 7f062d000640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55eb38487470 0x55eb38491bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.83:38650/0 (socket says 172.21.15.83:38650) 2024-08-19T01:42:02.080 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:02.077+0000 7f062d000640 1 -- 172.21.15.83:0/3881611465 learned_addr learned my addr 172.21.15.83:0/3881611465 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-08-19T01:42:02.080 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:02.078+0000 7f062d000640 1 -- 172.21.15.83:0/3881611465 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55eb38487db0 msgr2=0x55eb38498c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:42:02.080 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:02.078+0000 7f062d000640 1 --2- 172.21.15.83:0/3881611465 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55eb38487db0 0x55eb38498c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:02.080 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:02.078+0000 7f062d000640 1 -- 172.21.15.83:0/3881611465 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55eb3848ce40 msgr2=0x55eb38482ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:42:02.080 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:02.078+0000 7f062d000640 1 --2- 172.21.15.83:0/3881611465 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55eb3848ce40 0x55eb38482ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:02.080 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:02.078+0000 7f062d000640 1 -- 172.21.15.83:0/3881611465 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55eb38490700 con 0x55eb38487470 2024-08-19T01:42:02.083 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:02.078+0000 7f062d000640 1 --2- 172.21.15.83:0/3881611465 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55eb38487470 0x55eb38491bf0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f061c02a9d0 tx=0x7f061c004170 comp rx=0 tx=0).ready entity=mon.1 client_cookie=81438d6eeee8bd93 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:42:02.084 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:02.078+0000 7f0619600640 1 -- 172.21.15.83:0/3881611465 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f061c05d070 con 0x55eb38487470 2024-08-19T01:42:02.084 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:02.078+0000 7f0619600640 1 -- 172.21.15.83:0/3881611465 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f061c003930 con 0x55eb38487470 2024-08-19T01:42:02.084 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:02.078+0000 7f0619600640 1 -- 172.21.15.83:0/3881611465 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f061c04e6d0 con 0x55eb38487470 2024-08-19T01:42:02.084 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:02.079+0000 7f06329c2740 1 -- 172.21.15.83:0/3881611465 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55eb38487470 msgr2=0x55eb38491bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:42:02.084 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:02.079+0000 7f06329c2740 1 --2- 172.21.15.83:0/3881611465 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55eb38487470 0x55eb38491bf0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f061c02a9d0 tx=0x7f061c004170 comp rx=0 tx=0).stop 2024-08-19T01:42:02.084 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:02.079+0000 7f06329c2740 1 -- 172.21.15.83:0/3881611465 shutdown_connections 2024-08-19T01:42:02.084 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:02.079+0000 7f06329c2740 1 --2- 172.21.15.83:0/3881611465 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55eb3848ce40 0x55eb38482ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:02.084 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:02.079+0000 7f06329c2740 1 --2- 172.21.15.83:0/3881611465 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55eb38487db0 0x55eb38498c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:02.084 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:02.079+0000 7f06329c2740 1 --2- 172.21.15.83:0/3881611465 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55eb38487470 0x55eb38491bf0 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:02.084 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:02.079+0000 7f06329c2740 1 -- 172.21.15.83:0/3881611465 >> 172.21.15.83:0/3881611465 conn(0x55eb38482ae0 msgr2=0x55eb384914e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:42:02.084 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:02.079+0000 7f06329c2740 1 -- 172.21.15.83:0/3881611465 shutdown_connections 2024-08-19T01:42:02.084 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:02.079+0000 7f06329c2740 1 -- 172.21.15.83:0/3881611465 wait complete. 2024-08-19T01:42:02.084 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:02.079+0000 7f06329c2740 1 -- start start 2024-08-19T01:42:02.085 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:02.080+0000 7f06329c2740 1 Processor -- start 2024-08-19T01:42:02.085 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:02.080+0000 7f06329c2740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55eb38487470 0x55eb384963c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:42:02.085 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:02.080+0000 7f06329c2740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55eb38487db0 0x55eb38496900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:42:02.085 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:02.080+0000 7f06329c2740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55eb3848ce40 0x55eb38497910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:42:02.085 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:02.080+0000 7f06329c2740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f06281a95b0 con 0x55eb38487470 2024-08-19T01:42:02.085 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:02.080+0000 7f06329c2740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f06281aaa20 con 0x55eb3848ce40 2024-08-19T01:42:02.085 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:02.080+0000 7f06329c2740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f06281a9c80 con 0x55eb38487db0 2024-08-19T01:42:02.085 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:02.080+0000 7f06329c2740 10 client.0 _ll_register_callbacks cb 0x7f0620009b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-19T01:42:02.085 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:02.080+0000 7f062d000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55eb38487470 0x55eb384963c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:42:02.085 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:02.080+0000 7f062d000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55eb38487470 0x55eb384963c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:39566/0 (socket says 172.21.15.83:39566) 2024-08-19T01:42:02.085 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:02.080+0000 7f062d000640 1 -- 172.21.15.83:0/2964480875 learned_addr learned my addr 172.21.15.83:0/2964480875 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-08-19T01:42:02.085 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:02.080+0000 7f062da00640 1 --2- 172.21.15.83:0/2964480875 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55eb3848ce40 0x55eb38497910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:42:02.085 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:02.080+0000 7f0627e00640 1 --2- 172.21.15.83:0/2964480875 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55eb38487db0 0x55eb38496900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:42:02.086 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:02.080+0000 7f062d000640 1 -- 172.21.15.83:0/2964480875 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55eb38487db0 msgr2=0x55eb38496900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:42:02.086 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:02.080+0000 7f062d000640 1 --2- 172.21.15.83:0/2964480875 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55eb38487db0 0x55eb38496900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:02.086 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:02.081+0000 7f062d000640 1 -- 172.21.15.83:0/2964480875 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55eb3848ce40 msgr2=0x55eb38497910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:42:02.086 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:02.081+0000 7f062d000640 1 --2- 172.21.15.83:0/2964480875 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55eb3848ce40 0x55eb38497910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:02.086 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:02.081+0000 7f062d000640 1 -- 172.21.15.83:0/2964480875 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55eb38497f20 con 0x55eb38487470 2024-08-19T01:42:02.086 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:02.081+0000 7f062d000640 1 --2- 172.21.15.83:0/2964480875 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55eb38487470 0x55eb384963c0 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f061c02a930 tx=0x7f061c003d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=68334819dcb996ff server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:42:02.086 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:02.081+0000 7f060ea00640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-08-19T01:42:02.086 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:02.081+0000 7f060ea00640 1 -- 172.21.15.83:0/2964480875 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f061c05d040 con 0x55eb38487470 2024-08-19T01:42:02.086 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:02.081+0000 7f060ea00640 1 -- 172.21.15.83:0/2964480875 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8269+0+0 (secure 0 0 0) 0x7f061c04e680 con 0x55eb38487470 2024-08-19T01:42:02.086 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:02.081+0000 7f06329c2740 1 -- 172.21.15.83:0/2964480875 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f061c003930 con 0x55eb38487470 2024-08-19T01:42:02.086 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:02.081+0000 7f060ea00640 1 -- 172.21.15.83:0/2964480875 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f061c073050 con 0x55eb38487470 2024-08-19T01:42:02.086 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:02.081+0000 7f06329c2740 10 client.16014 fetch_fsmap learned FSMap version 28 2024-08-19T01:42:02.086 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:02.081+0000 7f06329c2740 1 -- 172.21.15.83:0/2964480875 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55eb38507e40 con 0x55eb38487470 2024-08-19T01:42:02.087 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:02.082+0000 7f060ea00640 1 -- 172.21.15.83:0/2964480875 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f061c003930 con 0x55eb38487470 2024-08-19T01:42:02.087 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:02.082+0000 7f060ea00640 1 -- 172.21.15.83:0/2964480875 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 28) ==== 8358+0+0 (secure 0 0 0) 0x7f061c062570 con 0x55eb38487470 2024-08-19T01:42:02.087 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:02.082+0000 7f06329c2740 10 client.16014 fetch_fsmap finished waiting for FSMap version 28 2024-08-19T01:42:02.087 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:02.082+0000 7f06329c2740 10 client.16014 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-08-19T01:42:02.087 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:02.082+0000 7f06329c2740 20 client.16014 populate_metadata read hostname 'smithi083' 2024-08-19T01:42:02.087 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:02.082+0000 7f06329c2740 1 --2- 172.21.15.83:0/2964480875 >> [v2:172.21.15.117:6838/1967595704,v1:172.21.15.117:6839/1967595704] conn(0x55eb385087c0 0x55eb3850abe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:42:02.087 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:02.082+0000 7f06329c2740 4 client.16014 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-08-19T01:42:02.087 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:02.082+0000 7f06329c2740 1 -- 172.21.15.83:0/2964480875 --> [v2:172.21.15.117:6838/1967595704,v1:172.21.15.117:6839/1967595704] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55eb383f3310 con 0x55eb385087c0 2024-08-19T01:42:02.087 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:02.082+0000 7f0627e00640 1 --2- 172.21.15.83:0/2964480875 >> [v2:172.21.15.117:6838/1967595704,v1:172.21.15.117:6839/1967595704] conn(0x55eb385087c0 0x55eb3850abe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:42:02.088 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:02.083+0000 7f0627e00640 1 --2- 172.21.15.83:0/2964480875 >> [v2:172.21.15.117:6838/1967595704,v1:172.21.15.117:6839/1967595704] conn(0x55eb385087c0 0x55eb3850abe0 crc :-1 s=READY pgs=128 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d2829aceaefbd39c server_cookie=bdfd1998ef45814c in_seq=0 out_seq=0 2024-08-19T01:42:02.088 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:02.083+0000 7f060ea00640 10 client.16014 ms_handle_connect on v2:172.21.15.117:6838/1967595704 2024-08-19T01:42:02.154 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:42:01 smithi083 ceph-mon[17120]: pgmap v554: 97 pgs: 97 active+clean; 456 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.2 MiB/s wr, 449 op/s 2024-08-19T01:42:02.528 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:02.525+0000 7f060ea00640 1 -- 172.21.15.83:0/2964480875 <== mds.0 v2:172.21.15.117:6838/1967595704 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55eb383f3310 con 0x55eb385087c0 2024-08-19T01:42:02.528 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:02.525+0000 7f060ea00640 10 client.16014 handle_command_reply: tid=0 2024-08-19T01:42:02.528 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:02.525+0000 7f06329c2740 1 client.16014 shutdown 2024-08-19T01:42:02.528 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:02.526+0000 7f06329c2740 1 -- 172.21.15.83:0/2964480875 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55eb38487470 msgr2=0x55eb384963c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:42:02.528 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:02.526+0000 7f06329c2740 1 --2- 172.21.15.83:0/2964480875 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55eb38487470 0x55eb384963c0 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f061c02a930 tx=0x7f061c003d10 comp rx=0 tx=0).stop 2024-08-19T01:42:02.528 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:02.526+0000 7f06329c2740 1 -- 172.21.15.83:0/2964480875 shutdown_connections 2024-08-19T01:42:02.529 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:02.526+0000 7f06329c2740 1 --2- 172.21.15.83:0/2964480875 >> [v2:172.21.15.117:6838/1967595704,v1:172.21.15.117:6839/1967595704] conn(0x55eb385087c0 0x55eb3850abe0 crc :-1 s=READY pgs=128 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:02.529 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:02.526+0000 7f06329c2740 1 --2- 172.21.15.83:0/2964480875 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55eb3848ce40 0x55eb38497910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:02.529 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:02.526+0000 7f06329c2740 1 --2- 172.21.15.83:0/2964480875 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55eb38487db0 0x55eb38496900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:02.529 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:02.526+0000 7f06329c2740 1 --2- 172.21.15.83:0/2964480875 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55eb38487470 0x55eb384963c0 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:02.529 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:02.526+0000 7f06329c2740 1 -- 172.21.15.83:0/2964480875 >> 172.21.15.83:0/2964480875 conn(0x55eb38482ae0 msgr2=0x55eb384814d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:42:02.529 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:02.526+0000 7f060ea00640 0 client.16014 ms_handle_reset on v2:172.21.15.117:6838/1967595704 2024-08-19T01:42:02.529 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:02.526+0000 7f06329c2740 1 -- 172.21.15.83:0/2964480875 shutdown_connections 2024-08-19T01:42:02.529 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:02.526+0000 7f06329c2740 1 -- 172.21.15.83:0/2964480875 wait complete. 2024-08-19T01:42:02.529 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:02.526+0000 7f06329c2740 20 client.16014 trim_cache size 0 max 16384 2024-08-19T01:42:02.529 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stdout:{ 2024-08-19T01:42:02.530 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stdout: "message": "", 2024-08-19T01:42:02.530 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stdout: "return_code": 0 2024-08-19T01:42:02.530 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stdout:} 2024-08-19T01:42:02.530 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:02.528+0000 7f062fa00640 1 -- 172.21.15.83:0/2949498541 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f06100a2cb0 msgr2=0x7f06100a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:42:02.530 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:02.528+0000 7f062fa00640 1 --2- 172.21.15.83:0/2949498541 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f06100a2cb0 0x7f06100a5170 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f0620005fd0 tx=0x7f0620009500 comp rx=0 tx=0).stop 2024-08-19T01:42:02.530 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:02.528+0000 7f062fa00640 1 -- 172.21.15.83:0/2949498541 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0628104c80 msgr2=0x7f062819a260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:42:02.531 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:02.528+0000 7f062fa00640 1 --2- 172.21.15.83:0/2949498541 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0628104c80 0x7f062819a260 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f061400d950 tx=0x7f061400de10 comp rx=0 tx=0).stop 2024-08-19T01:42:02.531 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:02.528+0000 7f062fa00640 1 -- 172.21.15.83:0/2949498541 shutdown_connections 2024-08-19T01:42:02.531 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:02.528+0000 7f062fa00640 1 --2- 172.21.15.83:0/2949498541 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f06100a2cb0 0x7f06100a5170 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:02.531 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:02.528+0000 7f062fa00640 1 --2- 172.21.15.83:0/2949498541 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0628105640 0x7f06281a1290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:02.531 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:02.528+0000 7f062fa00640 1 --2- 172.21.15.83:0/2949498541 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0628104c80 0x7f062819a260 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:02.531 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:02.528+0000 7f062fa00640 1 --2- 172.21.15.83:0/2949498541 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0628103a90 0x7f0628199d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:02.531 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:02.528+0000 7f062fa00640 1 -- 172.21.15.83:0/2949498541 >> 172.21.15.83:0/2949498541 conn(0x7f06280ff210 msgr2=0x7f0628107f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:42:02.531 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:02.528+0000 7f062fa00640 1 -- 172.21.15.83:0/2949498541 shutdown_connections 2024-08-19T01:42:02.531 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:02.528+0000 7f062fa00640 1 -- 172.21.15.83:0/2949498541 wait complete. 2024-08-19T01:42:02.773 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833 shell --fsid bad8780e-5dc9-11ef-bccf-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-08-19T01:42:02.952 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:42:02 smithi117 ceph-mon[22016]: scrub queued for path: / 2024-08-19T01:42:02.952 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:42:02 smithi117 ceph-mon[22016]: scrub summary: idle+waiting paths [/] 2024-08-19T01:42:02.952 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:42:02 smithi117 ceph-mon[22016]: scrub summary: active paths [/] 2024-08-19T01:42:02.952 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:42:02 smithi117 ceph-mon[22016]: scrub summary: active paths [/] 2024-08-19T01:42:02.953 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:42:02 smithi117 ceph-mon[22016]: scrub summary: idle+waiting paths [/] 2024-08-19T01:42:02.953 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:42:02 smithi117 ceph-mon[22016]: scrub summary: active paths [/] 2024-08-19T01:42:02.953 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:42:02 smithi117 ceph-mon[22016]: scrub summary: active paths [/] 2024-08-19T01:42:02.953 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:42:02 smithi117 ceph-mon[22016]: scrub summary: idle+waiting paths [/] 2024-08-19T01:42:02.953 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:42:02 smithi117 ceph-mon[22016]: scrub summary: idle+waiting paths [/] 2024-08-19T01:42:02.953 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:42:02 smithi117 ceph-mon[22016]: scrub summary: active paths [/] 2024-08-19T01:42:02.953 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:42:02 smithi117 ceph-mon[22016]: scrub summary: idle+waiting paths [/] 2024-08-19T01:42:02.966 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:42:02 smithi167 ceph-mon[21875]: scrub queued for path: / 2024-08-19T01:42:02.966 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:42:02 smithi167 ceph-mon[21875]: scrub summary: idle+waiting paths [/] 2024-08-19T01:42:02.966 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:42:02 smithi167 ceph-mon[21875]: scrub summary: active paths [/] 2024-08-19T01:42:02.966 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:42:02 smithi167 ceph-mon[21875]: scrub summary: active paths [/] 2024-08-19T01:42:02.966 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:42:02 smithi167 ceph-mon[21875]: scrub summary: idle+waiting paths [/] 2024-08-19T01:42:02.966 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:42:02 smithi167 ceph-mon[21875]: scrub summary: active paths [/] 2024-08-19T01:42:02.966 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:42:02 smithi167 ceph-mon[21875]: scrub summary: active paths [/] 2024-08-19T01:42:02.966 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:42:02 smithi167 ceph-mon[21875]: scrub summary: idle+waiting paths [/] 2024-08-19T01:42:02.966 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:42:02 smithi167 ceph-mon[21875]: scrub summary: idle+waiting paths [/] 2024-08-19T01:42:02.967 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:42:02 smithi167 ceph-mon[21875]: scrub summary: active paths [/] 2024-08-19T01:42:02.967 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:42:02 smithi167 ceph-mon[21875]: scrub summary: idle+waiting paths [/] 2024-08-19T01:42:03.154 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:42:02 smithi083 ceph-mon[17120]: scrub queued for path: / 2024-08-19T01:42:03.155 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:42:02 smithi083 ceph-mon[17120]: scrub summary: idle+waiting paths [/] 2024-08-19T01:42:03.155 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:42:02 smithi083 ceph-mon[17120]: scrub summary: active paths [/] 2024-08-19T01:42:03.155 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:42:02 smithi083 ceph-mon[17120]: scrub summary: active paths [/] 2024-08-19T01:42:03.155 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:42:02 smithi083 ceph-mon[17120]: scrub summary: idle+waiting paths [/] 2024-08-19T01:42:03.155 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:42:02 smithi083 ceph-mon[17120]: scrub summary: active paths [/] 2024-08-19T01:42:03.155 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:42:02 smithi083 ceph-mon[17120]: scrub summary: active paths [/] 2024-08-19T01:42:03.155 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:42:02 smithi083 ceph-mon[17120]: scrub summary: idle+waiting paths [/] 2024-08-19T01:42:03.155 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:42:02 smithi083 ceph-mon[17120]: scrub summary: idle+waiting paths [/] 2024-08-19T01:42:03.155 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:42:02 smithi083 ceph-mon[17120]: scrub summary: active paths [/] 2024-08-19T01:42:03.155 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:42:02 smithi083 ceph-mon[17120]: scrub summary: idle+waiting paths [/] 2024-08-19T01:42:03.230 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/bad8780e-5dc9-11ef-bccf-c7b262605968/mon.a/config 2024-08-19T01:42:03.900 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:03.898+0000 7f5cb7e00640 1 -- 172.21.15.83:0/769072658 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5cb0116bd0 msgr2=0x7f5cb0118fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:42:03.901 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:03.898+0000 7f5cb7e00640 1 --2- 172.21.15.83:0/769072658 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5cb0116bd0 0x7f5cb0118fc0 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f5ca8009be0 tx=0x7f5ca802a6c0 comp rx=0 tx=0).stop 2024-08-19T01:42:03.901 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:03.898+0000 7f5cb7e00640 1 -- 172.21.15.83:0/769072658 shutdown_connections 2024-08-19T01:42:03.901 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:03.898+0000 7f5cb7e00640 1 --2- 172.21.15.83:0/769072658 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5cb0116bd0 0x7f5cb0118fc0 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:03.901 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:03.898+0000 7f5cb7e00640 1 --2- 172.21.15.83:0/769072658 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5cb0109840 0x7f5cb0116690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:03.901 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:03.898+0000 7f5cb7e00640 1 --2- 172.21.15.83:0/769072658 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5cb0108f20 0x7f5cb0109300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:03.901 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:03.899+0000 7f5cb7e00640 1 -- 172.21.15.83:0/769072658 >> 172.21.15.83:0/769072658 conn(0x7f5cb006e000 msgr2=0x7f5cb006e410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:42:03.901 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:03.899+0000 7f5cb7e00640 1 -- 172.21.15.83:0/769072658 shutdown_connections 2024-08-19T01:42:03.902 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:03.899+0000 7f5cb7e00640 1 -- 172.21.15.83:0/769072658 wait complete. 2024-08-19T01:42:03.902 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:03.901+0000 7f5cb7e00640 1 Processor -- start 2024-08-19T01:42:03.903 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:03.902+0000 7f5cb7e00640 1 -- start start 2024-08-19T01:42:03.904 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:03.902+0000 7f5cb7e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5cb0108f20 0x7f5cb01164f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:42:03.904 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:03.902+0000 7f5cb7e00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5cb0109840 0x7f5cb01135a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:42:03.904 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:03.903+0000 7f5cb7e00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5cb0116bd0 0x7f5cb0113ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:42:03.905 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:03.903+0000 7f5cb7e00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f5cb011c740 con 0x7f5cb0108f20 2024-08-19T01:42:03.905 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:03.903+0000 7f5cb7e00640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f5cb011c5c0 con 0x7f5cb0109840 2024-08-19T01:42:03.905 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:03.903+0000 7f5cb5400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5cb0108f20 0x7f5cb01164f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:42:03.905 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:03.903+0000 7f5cb7e00640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f5cb011c8c0 con 0x7f5cb0116bd0 2024-08-19T01:42:03.905 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:03.903+0000 7f5cb5400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5cb0108f20 0x7f5cb01164f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:39582/0 (socket says 172.21.15.83:39582) 2024-08-19T01:42:03.905 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:03.903+0000 7f5cb5400640 1 -- 172.21.15.83:0/2208192926 learned_addr learned my addr 172.21.15.83:0/2208192926 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-08-19T01:42:03.905 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:03.904+0000 7f5cb5e00640 1 --2- 172.21.15.83:0/2208192926 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5cb0116bd0 0x7f5cb0113ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:42:03.906 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:03.904+0000 7f5cb4a00640 1 --2- 172.21.15.83:0/2208192926 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5cb0109840 0x7f5cb01135a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:42:03.906 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:03.904+0000 7f5cb5400640 1 -- 172.21.15.83:0/2208192926 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5cb0116bd0 msgr2=0x7f5cb0113ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:42:03.906 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:03.904+0000 7f5cb5400640 1 --2- 172.21.15.83:0/2208192926 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5cb0116bd0 0x7f5cb0113ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:03.906 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:03.904+0000 7f5cb5400640 1 -- 172.21.15.83:0/2208192926 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5cb0109840 msgr2=0x7f5cb01135a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:42:03.906 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:03.904+0000 7f5cb5400640 1 --2- 172.21.15.83:0/2208192926 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5cb0109840 0x7f5cb01135a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:03.906 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:03.904+0000 7f5cb5400640 1 -- 172.21.15.83:0/2208192926 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5cb0114370 con 0x7f5cb0108f20 2024-08-19T01:42:03.906 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:03.905+0000 7f5cb5400640 1 --2- 172.21.15.83:0/2208192926 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5cb0108f20 0x7f5cb01164f0 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f5ca400e9c0 tx=0x7f5ca400ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:42:03.907 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:03.905+0000 7f5cb4a00640 1 --2- 172.21.15.83:0/2208192926 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5cb0109840 0x7f5cb01135a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-19T01:42:03.907 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:03.905+0000 7f5cae000640 1 -- 172.21.15.83:0/2208192926 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ca400cde0 con 0x7f5cb0108f20 2024-08-19T01:42:03.907 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:03.905+0000 7f5cb7e00640 1 -- 172.21.15.83:0/2208192926 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5cb01b3540 con 0x7f5cb0108f20 2024-08-19T01:42:03.908 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:03.905+0000 7f5cb7e00640 1 -- 172.21.15.83:0/2208192926 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5cb01b3aa0 con 0x7f5cb0108f20 2024-08-19T01:42:03.908 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:03.906+0000 7f5cae000640 1 -- 172.21.15.83:0/2208192926 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5ca4009520 con 0x7f5cb0108f20 2024-08-19T01:42:03.908 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:03.906+0000 7f5cae000640 1 -- 172.21.15.83:0/2208192926 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ca4017790 con 0x7f5cb0108f20 2024-08-19T01:42:03.912 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:03.910+0000 7f5cae000640 1 -- 172.21.15.83:0/2208192926 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7f5ca4017950 con 0x7f5cb0108f20 2024-08-19T01:42:03.912 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:03.910+0000 7f5cba8bb740 1 Processor -- start 2024-08-19T01:42:03.912 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:03.910+0000 7f5cba8bb740 1 -- start start 2024-08-19T01:42:03.913 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:03.911+0000 7f5cba8bb740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5654f5a19b90 0x5654f5a19f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:42:03.913 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:03.911+0000 7f5cba8bb740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5654f5a1a4b0 0x5654f5a24830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:42:03.913 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:03.911+0000 7f5cba8bb740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5654f5a24d70 0x5654f5a27160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:42:03.913 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:03.911+0000 7f5cba8bb740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x5654f597fbf0 con 0x5654f5a1a4b0 2024-08-19T01:42:03.913 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:03.911+0000 7f5cba8bb740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x5654f5917bd0 con 0x5654f5a24d70 2024-08-19T01:42:03.913 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:03.911+0000 7f5cba8bb740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x5654f597bbb0 con 0x5654f5a19b90 2024-08-19T01:42:03.914 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:03.913+0000 7f5cb5e00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5654f5a19b90 0x5654f5a19f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:42:03.914 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:03.913+0000 7f5cb4a00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5654f5a24d70 0x5654f5a27160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:42:03.914 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:03.913+0000 7f5cae000640 1 --2- 172.21.15.83:0/2208192926 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f5c980a2c80 0x7f5c980a5140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:42:03.915 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:03.913+0000 7f5cb5e00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5654f5a19b90 0x5654f5a19f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.83:38676/0 (socket says 172.21.15.83:38676) 2024-08-19T01:42:03.915 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:03.913+0000 7f5cae000640 1 -- 172.21.15.83:0/2208192926 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8269+0+0 (secure 0 0 0) 0x7f5ca40cf360 con 0x7f5cb0108f20 2024-08-19T01:42:03.916 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:03.913+0000 7f5cb5e00640 1 -- 172.21.15.83:0/1307947504 learned_addr learned my addr 172.21.15.83:0/1307947504 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-08-19T01:42:03.916 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:03.914+0000 7f5cb5e00640 1 -- 172.21.15.83:0/1307947504 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5654f5a24d70 msgr2=0x5654f5a27160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:42:03.916 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:03.914+0000 7f5cb5e00640 1 --2- 172.21.15.83:0/1307947504 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5654f5a24d70 0x5654f5a27160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:03.916 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:03.914+0000 7f5cb5e00640 1 -- 172.21.15.83:0/1307947504 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5654f5a1a4b0 msgr2=0x5654f5a24830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:42:03.916 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:03.914+0000 7f5cb5e00640 1 --2- 172.21.15.83:0/1307947504 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5654f5a1a4b0 0x5654f5a24830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:03.916 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:03.914+0000 7f5cb5e00640 1 -- 172.21.15.83:0/1307947504 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5654f591e8a0 con 0x5654f5a19b90 2024-08-19T01:42:03.917 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:03.914+0000 7f5cb5e00640 1 --2- 172.21.15.83:0/1307947504 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5654f5a19b90 0x5654f5a19f70 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f5ca8006010 tx=0x7f5ca8030ca0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3a9edfbebe6c29a4 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:42:03.917 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:03.914+0000 7f5cb5e00640 1 --2- 172.21.15.83:0/2208192926 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f5c980a2c80 0x7f5c980a5140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:42:03.917 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:03.915+0000 7f5ca2600640 1 -- 172.21.15.83:0/1307947504 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ca8042030 con 0x5654f5a19b90 2024-08-19T01:42:03.917 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:03.915+0000 7f5ca2600640 1 -- 172.21.15.83:0/1307947504 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5ca8030e90 con 0x5654f5a19b90 2024-08-19T01:42:03.917 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:03.915+0000 7f5ca2600640 1 -- 172.21.15.83:0/1307947504 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ca8036040 con 0x5654f5a19b90 2024-08-19T01:42:03.918 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:03.915+0000 7f5cba8bb740 1 -- 172.21.15.83:0/1307947504 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5654f5a19b90 msgr2=0x5654f5a19f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:42:03.918 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:03.915+0000 7f5cba8bb740 1 --2- 172.21.15.83:0/1307947504 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5654f5a19b90 0x5654f5a19f70 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f5ca8006010 tx=0x7f5ca8030ca0 comp rx=0 tx=0).stop 2024-08-19T01:42:03.918 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:03.916+0000 7f5cba8bb740 1 -- 172.21.15.83:0/1307947504 shutdown_connections 2024-08-19T01:42:03.918 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:03.916+0000 7f5cba8bb740 1 --2- 172.21.15.83:0/1307947504 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5654f5a24d70 0x5654f5a27160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:03.918 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:03.916+0000 7f5cba8bb740 1 --2- 172.21.15.83:0/1307947504 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5654f5a1a4b0 0x5654f5a24830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:03.918 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:03.916+0000 7f5cba8bb740 1 --2- 172.21.15.83:0/1307947504 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5654f5a19b90 0x5654f5a19f70 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:03.918 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:03.916+0000 7f5cba8bb740 1 -- 172.21.15.83:0/1307947504 >> 172.21.15.83:0/1307947504 conn(0x5654f5a16120 msgr2=0x5654f5a16500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:42:03.919 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:03.916+0000 7f5cb5e00640 1 --2- 172.21.15.83:0/2208192926 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f5c980a2c80 0x7f5c980a5140 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f5ca8005f00 tx=0x7f5ca8049040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:42:03.919 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:03.916+0000 7f5cba8bb740 1 -- 172.21.15.83:0/1307947504 shutdown_connections 2024-08-19T01:42:03.919 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:03.916+0000 7f5cba8bb740 1 -- 172.21.15.83:0/1307947504 wait complete. 2024-08-19T01:42:03.919 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:03.917+0000 7f5cba8bb740 1 -- start start 2024-08-19T01:42:03.919 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:03.918+0000 7f5cba8bb740 1 Processor -- start 2024-08-19T01:42:03.920 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:03.918+0000 7f5cba8bb740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5654f5a19b90 0x5654f5a21ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:42:03.920 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:03.919+0000 7f5cba8bb740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5654f5a1a4b0 0x5654f5a220e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:42:03.921 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:03.919+0000 7f5cba8bb740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5654f5a24d70 0x5654f5a22620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:42:03.921 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:03.919+0000 7f5cba8bb740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x5654f5a27940 con 0x5654f5a24d70 2024-08-19T01:42:03.921 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:03.919+0000 7f5cba8bb740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x5654f593ad40 con 0x5654f5a19b90 2024-08-19T01:42:03.921 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:03.919+0000 7f5cba8bb740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x5654f5a27ac0 con 0x5654f5a1a4b0 2024-08-19T01:42:03.921 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:03.919+0000 7f5cba8bb740 10 client.0 _ll_register_callbacks cb 0x5654f5903b10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-19T01:42:03.921 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:03.919+0000 7f5cb4a00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5654f5a1a4b0 0x5654f5a220e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:42:03.921 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:03.919+0000 7f5cb4a00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5654f5a1a4b0 0x5654f5a220e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.83:38678/0 (socket says 172.21.15.83:38678) 2024-08-19T01:42:03.921 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:03.919+0000 7f5cb4a00640 1 -- 172.21.15.83:0/1269843633 learned_addr learned my addr 172.21.15.83:0/1269843633 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-08-19T01:42:03.921 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:03.919+0000 7f5cb5400640 1 --2- 172.21.15.83:0/1269843633 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5654f5a19b90 0x5654f5a21ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:42:03.921 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:03.919+0000 7f5cb4a00640 1 -- 172.21.15.83:0/1269843633 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5654f5a19b90 msgr2=0x5654f5a21ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:42:03.922 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:03.920+0000 7f5cb4a00640 1 --2- 172.21.15.83:0/1269843633 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5654f5a19b90 0x5654f5a21ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:03.922 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:03.920+0000 7f5cb4a00640 1 -- 172.21.15.83:0/1269843633 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5654f5a24d70 msgr2=0x5654f5a22620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:42:03.922 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:03.920+0000 7f5cb4a00640 1 --2- 172.21.15.83:0/1269843633 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5654f5a24d70 0x5654f5a22620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:03.922 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:03.920+0000 7f5cb4a00640 1 -- 172.21.15.83:0/1269843633 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5654f56f13c0 con 0x5654f5a1a4b0 2024-08-19T01:42:03.923 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:03.920+0000 7f5cb4a00640 1 --2- 172.21.15.83:0/1269843633 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5654f5a1a4b0 0x5654f5a220e0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f5c9c00e300 tx=0x7f5c9c00e700 comp rx=0 tx=0).ready entity=mon.1 client_cookie=46097eec1ab2c2a8 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:42:03.923 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:03.920+0000 7f5c96a00640 10 client.0 ms_handle_connect on v2:172.21.15.167:3300/0 2024-08-19T01:42:03.923 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:03.922+0000 7f5c96a00640 1 -- 172.21.15.83:0/1269843633 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c9c018030 con 0x5654f5a1a4b0 2024-08-19T01:42:03.923 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:03.922+0000 7f5c96a00640 1 -- 172.21.15.83:0/1269843633 <== mon.1 v2:172.21.15.167:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8269+0+0 (secure 0 0 0) 0x7f5c9c00f040 con 0x5654f5a1a4b0 2024-08-19T01:42:03.924 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:03.922+0000 7f5cba8bb740 1 -- 172.21.15.83:0/1269843633 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5ca8030e90 con 0x5654f5a1a4b0 2024-08-19T01:42:03.924 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:03.922+0000 7f5c96a00640 1 -- 172.21.15.83:0/1269843633 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f5c9c013660 con 0x5654f5a1a4b0 2024-08-19T01:42:03.924 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:03.922+0000 7f5cba8bb740 10 client.26008 fetch_fsmap learned FSMap version 28 2024-08-19T01:42:03.924 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:03.922+0000 7f5cba8bb740 1 -- 172.21.15.83:0/1269843633 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5654f5a33660 con 0x5654f5a1a4b0 2024-08-19T01:42:03.924 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:03.923+0000 7f5c96a00640 1 -- 172.21.15.83:0/1269843633 <== mon.1 v2:172.21.15.167:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5ca8030e90 con 0x5654f5a1a4b0 2024-08-19T01:42:03.925 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:03.923+0000 7f5c96a00640 1 -- 172.21.15.83:0/1269843633 <== mon.1 v2:172.21.15.167:3300/0 5 ==== fsmap(e 28) ==== 8358+0+0 (secure 0 0 0) 0x7f5c9c01e970 con 0x5654f5a1a4b0 2024-08-19T01:42:03.925 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:03.923+0000 7f5cba8bb740 10 client.26008 fetch_fsmap finished waiting for FSMap version 28 2024-08-19T01:42:03.925 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:03.923+0000 7f5cba8bb740 10 client.26008 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-19T01:42:03.925 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:03.923+0000 7f5cba8bb740 20 client.26008 populate_metadata read hostname 'smithi083' 2024-08-19T01:42:03.925 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:03.923+0000 7f5cba8bb740 1 --2- 172.21.15.83:0/1269843633 >> [v2:172.21.15.117:6838/1967595704,v1:172.21.15.117:6839/1967595704] conn(0x5654f5a34bf0 0x5654f5a37010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:42:03.927 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:03.924+0000 7f5cba8bb740 4 client.26008 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-19T01:42:03.927 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:03.924+0000 7f5cba8bb740 1 -- 172.21.15.83:0/1269843633 --> [v2:172.21.15.117:6838/1967595704,v1:172.21.15.117:6839/1967595704] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5654f598d160 con 0x5654f5a34bf0 2024-08-19T01:42:03.927 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:03.926+0000 7f5cb5400640 1 --2- 172.21.15.83:0/1269843633 >> [v2:172.21.15.117:6838/1967595704,v1:172.21.15.117:6839/1967595704] conn(0x5654f5a34bf0 0x5654f5a37010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:42:03.928 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:03.926+0000 7f5cb5400640 1 --2- 172.21.15.83:0/1269843633 >> [v2:172.21.15.117:6838/1967595704,v1:172.21.15.117:6839/1967595704] conn(0x5654f5a34bf0 0x5654f5a37010 crc :-1 s=READY pgs=129 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6ee41b84c806bd70 server_cookie=1789a5b3bd955bd4 in_seq=0 out_seq=0 2024-08-19T01:42:03.928 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:03.926+0000 7f5c96a00640 10 client.26008 ms_handle_connect on v2:172.21.15.117:6838/1967595704 2024-08-19T01:42:03.930 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:03.928+0000 7f5c96a00640 1 -- 172.21.15.83:0/1269843633 <== mds.0 v2:172.21.15.117:6838/1967595704 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5654f598d160 con 0x5654f5a34bf0 2024-08-19T01:42:03.930 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:03.928+0000 7f5c96a00640 10 client.26008 handle_command_reply: tid=0 2024-08-19T01:42:03.930 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:03.928+0000 7f5cba8bb740 1 client.26008 shutdown 2024-08-19T01:42:03.931 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:03.928+0000 7f5cba8bb740 1 -- 172.21.15.83:0/1269843633 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5654f5a1a4b0 msgr2=0x5654f5a220e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:42:03.931 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:03.928+0000 7f5cba8bb740 1 --2- 172.21.15.83:0/1269843633 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5654f5a1a4b0 0x5654f5a220e0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f5c9c00e300 tx=0x7f5c9c00e700 comp rx=0 tx=0).stop 2024-08-19T01:42:03.931 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:03.929+0000 7f5cba8bb740 1 -- 172.21.15.83:0/1269843633 shutdown_connections 2024-08-19T01:42:03.931 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:03.929+0000 7f5cba8bb740 1 --2- 172.21.15.83:0/1269843633 >> [v2:172.21.15.117:6838/1967595704,v1:172.21.15.117:6839/1967595704] conn(0x5654f5a34bf0 0x5654f5a37010 crc :-1 s=READY pgs=129 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:03.931 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:03.929+0000 7f5cba8bb740 1 --2- 172.21.15.83:0/1269843633 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5654f5a24d70 0x5654f5a22620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:03.931 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:03.929+0000 7f5cba8bb740 1 --2- 172.21.15.83:0/1269843633 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5654f5a1a4b0 0x5654f5a220e0 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:03.931 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:03.929+0000 7f5cba8bb740 1 --2- 172.21.15.83:0/1269843633 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5654f5a19b90 0x5654f5a21ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:03.931 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:03.929+0000 7f5cba8bb740 1 -- 172.21.15.83:0/1269843633 >> 172.21.15.83:0/1269843633 conn(0x5654f5a16120 msgr2=0x5654f5a1be50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:42:03.931 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:03.929+0000 7f5c96a00640 0 client.26008 ms_handle_reset on v2:172.21.15.117:6838/1967595704 2024-08-19T01:42:03.932 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:03.929+0000 7f5cba8bb740 1 -- 172.21.15.83:0/1269843633 shutdown_connections 2024-08-19T01:42:03.932 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:03.929+0000 7f5cba8bb740 1 -- 172.21.15.83:0/1269843633 wait complete. 2024-08-19T01:42:03.932 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:03.930+0000 7f5cba8bb740 20 client.26008 trim_cache size 0 max 16384 2024-08-19T01:42:03.952 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:42:03 smithi117 ceph-mon[22016]: scrub summary: active paths [/] 2024-08-19T01:42:03.952 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:42:03 smithi117 ceph-mon[22016]: scrub summary: idle+waiting paths [/] 2024-08-19T01:42:03.952 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:42:03 smithi117 ceph-mon[22016]: scrub summary: active paths [/] 2024-08-19T01:42:03.952 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:42:03 smithi117 ceph-mon[22016]: scrub summary: idle+waiting paths [/] 2024-08-19T01:42:03.952 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:42:03 smithi117 ceph-mon[22016]: pgmap v555: 97 pgs: 97 active+clean; 448 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.3 MiB/s wr, 428 op/s 2024-08-19T01:42:03.965 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:42:03 smithi167 ceph-mon[21875]: scrub summary: active paths [/] 2024-08-19T01:42:03.965 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:42:03 smithi167 ceph-mon[21875]: scrub summary: idle+waiting paths [/] 2024-08-19T01:42:03.966 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:42:03 smithi167 ceph-mon[21875]: scrub summary: active paths [/] 2024-08-19T01:42:03.966 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:42:03 smithi167 ceph-mon[21875]: scrub summary: idle+waiting paths [/] 2024-08-19T01:42:03.966 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:42:03 smithi167 ceph-mon[21875]: pgmap v555: 97 pgs: 97 active+clean; 448 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.3 MiB/s wr, 428 op/s 2024-08-19T01:42:03.969 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:03.966+0000 7f5cba8bb740 1 Processor -- start 2024-08-19T01:42:03.969 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:03.967+0000 7f5cba8bb740 1 -- start start 2024-08-19T01:42:03.969 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:03.967+0000 7f5cba8bb740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5654f5a19b90 0x5654f5a23b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:42:03.969 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:03.967+0000 7f5cba8bb740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5654f5a1a4b0 0x5654f5a3c8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:42:03.970 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:03.967+0000 7f5cba8bb740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5654f5a24d70 0x5654f5a3cde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:42:03.970 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:03.967+0000 7f5cba8bb740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f5c90010eb0 con 0x5654f5a1a4b0 2024-08-19T01:42:03.970 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:03.967+0000 7f5cba8bb740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f5c9000e750 con 0x5654f5a19b90 2024-08-19T01:42:03.970 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:03.967+0000 7f5cba8bb740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f5c9000c5a0 con 0x5654f5a24d70 2024-08-19T01:42:03.970 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:03.968+0000 7f5cb5400640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5654f5a19b90 0x5654f5a23b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:42:03.970 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:03.968+0000 7f5cb5400640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5654f5a19b90 0x5654f5a23b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.83:55000/0 (socket says 172.21.15.83:55000) 2024-08-19T01:42:03.970 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:03.968+0000 7f5cb5400640 1 -- 172.21.15.83:0/1804103508 learned_addr learned my addr 172.21.15.83:0/1804103508 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-08-19T01:42:03.970 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:03.968+0000 7f5cb5e00640 1 --2- 172.21.15.83:0/1804103508 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5654f5a24d70 0x5654f5a3cde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:42:03.971 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:03.968+0000 7f5cb4a00640 1 --2- 172.21.15.83:0/1804103508 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5654f5a1a4b0 0x5654f5a3c8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:42:03.971 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:42:03 smithi083 ceph-mon[17120]: scrub summary: active paths [/] 2024-08-19T01:42:03.971 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:42:03 smithi083 ceph-mon[17120]: scrub summary: idle+waiting paths [/] 2024-08-19T01:42:03.971 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:42:03 smithi083 ceph-mon[17120]: scrub summary: active paths [/] 2024-08-19T01:42:03.971 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:42:03 smithi083 ceph-mon[17120]: scrub summary: idle+waiting paths [/] 2024-08-19T01:42:03.971 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:42:03 smithi083 ceph-mon[17120]: pgmap v555: 97 pgs: 97 active+clean; 448 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.3 MiB/s wr, 428 op/s 2024-08-19T01:42:03.972 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:03.969+0000 7f5cb5400640 1 -- 172.21.15.83:0/1804103508 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5654f5a24d70 msgr2=0x5654f5a3cde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:42:03.972 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:03.969+0000 7f5cb5400640 1 --2- 172.21.15.83:0/1804103508 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5654f5a24d70 0x5654f5a3cde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:03.972 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:03.969+0000 7f5cb5400640 1 -- 172.21.15.83:0/1804103508 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5654f5a1a4b0 msgr2=0x5654f5a3c8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:42:03.972 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:03.970+0000 7f5cb5400640 1 --2- 172.21.15.83:0/1804103508 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5654f5a1a4b0 0x5654f5a3c8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:03.972 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:03.970+0000 7f5cb5400640 1 -- 172.21.15.83:0/1804103508 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5654f5a1a2e0 con 0x5654f5a19b90 2024-08-19T01:42:03.972 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:03.970+0000 7f5cb5400640 1 --2- 172.21.15.83:0/1804103508 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5654f5a19b90 0x5654f5a23b10 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f5ca4099680 tx=0x7f5ca4099080 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b11bbb1762c3de27 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:42:03.972 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:03.970+0000 7f5ca1c00640 1 -- 172.21.15.83:0/1804103508 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ca40b9070 con 0x5654f5a19b90 2024-08-19T01:42:03.973 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:03.970+0000 7f5ca1c00640 1 -- 172.21.15.83:0/1804103508 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5ca40ccf50 con 0x5654f5a19b90 2024-08-19T01:42:03.973 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:03.970+0000 7f5ca1c00640 1 -- 172.21.15.83:0/1804103508 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ca40a56f0 con 0x5654f5a19b90 2024-08-19T01:42:03.973 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:03.972+0000 7f5cba8bb740 1 -- 172.21.15.83:0/1804103508 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5654f5a19b90 msgr2=0x5654f5a23b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:42:03.973 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:03.972+0000 7f5cba8bb740 1 --2- 172.21.15.83:0/1804103508 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5654f5a19b90 0x5654f5a23b10 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f5ca4099680 tx=0x7f5ca4099080 comp rx=0 tx=0).stop 2024-08-19T01:42:03.974 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:03.972+0000 7f5cba8bb740 1 -- 172.21.15.83:0/1804103508 shutdown_connections 2024-08-19T01:42:03.974 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:03.972+0000 7f5cba8bb740 1 --2- 172.21.15.83:0/1804103508 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5654f5a24d70 0x5654f5a3cde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:03.974 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:03.972+0000 7f5cba8bb740 1 --2- 172.21.15.83:0/1804103508 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5654f5a1a4b0 0x5654f5a3c8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:03.974 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:03.972+0000 7f5cba8bb740 1 --2- 172.21.15.83:0/1804103508 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5654f5a19b90 0x5654f5a23b10 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:03.974 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:03.972+0000 7f5cba8bb740 1 -- 172.21.15.83:0/1804103508 >> 172.21.15.83:0/1804103508 conn(0x5654f5a16120 msgr2=0x5654f5a23400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:42:03.974 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:03.973+0000 7f5cba8bb740 1 -- 172.21.15.83:0/1804103508 shutdown_connections 2024-08-19T01:42:03.975 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:03.973+0000 7f5cba8bb740 1 -- 172.21.15.83:0/1804103508 wait complete. 2024-08-19T01:42:03.975 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:03.973+0000 7f5cba8bb740 1 -- start start 2024-08-19T01:42:03.975 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:03.973+0000 7f5cba8bb740 1 Processor -- start 2024-08-19T01:42:03.976 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:03.975+0000 7f5cba8bb740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5654f5a19b90 0x5654f5a2d360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:42:03.977 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:03.975+0000 7f5cba8bb740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5654f5a1a4b0 0x5654f5a1e750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:42:03.977 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:03.976+0000 7f5cba8bb740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5654f5a24d70 0x5654f5a1ec90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:42:03.978 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:03.976+0000 7f5cba8bb740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f5c9000cc10 con 0x5654f5a19b90 2024-08-19T01:42:03.978 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:03.976+0000 7f5cba8bb740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f5c9000e080 con 0x5654f5a24d70 2024-08-19T01:42:03.978 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:03.976+0000 7f5cba8bb740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f5c9000d2e0 con 0x5654f5a1a4b0 2024-08-19T01:42:03.978 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:03.976+0000 7f5cba8bb740 10 client.0 _ll_register_callbacks cb 0x7f5c9c00c190 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-19T01:42:03.978 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:03.976+0000 7f5cb4a00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5654f5a1a4b0 0x5654f5a1e750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:42:03.978 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:03.977+0000 7f5cb4a00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5654f5a1a4b0 0x5654f5a1e750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.83:38704/0 (socket says 172.21.15.83:38704) 2024-08-19T01:42:03.978 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:03.977+0000 7f5cb4a00640 1 -- 172.21.15.83:0/1888661372 learned_addr learned my addr 172.21.15.83:0/1888661372 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-08-19T01:42:03.978 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:03.977+0000 7f5cb5e00640 1 --2- 172.21.15.83:0/1888661372 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5654f5a24d70 0x5654f5a1ec90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:42:03.978 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:03.976+0000 7f5cb5400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5654f5a19b90 0x5654f5a2d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:42:03.979 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:03.977+0000 7f5cb4a00640 1 -- 172.21.15.83:0/1888661372 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5654f5a24d70 msgr2=0x5654f5a1ec90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:42:03.979 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:03.977+0000 7f5cb4a00640 1 --2- 172.21.15.83:0/1888661372 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5654f5a24d70 0x5654f5a1ec90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:03.979 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:03.977+0000 7f5cb4a00640 1 -- 172.21.15.83:0/1888661372 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5654f5a19b90 msgr2=0x5654f5a2d360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:42:03.979 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:03.977+0000 7f5cb4a00640 1 --2- 172.21.15.83:0/1888661372 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5654f5a19b90 0x5654f5a2d360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:03.979 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:03.977+0000 7f5cb4a00640 1 -- 172.21.15.83:0/1888661372 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5654f5a1f300 con 0x5654f5a1a4b0 2024-08-19T01:42:03.980 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:03.978+0000 7f5cb4a00640 1 --2- 172.21.15.83:0/1888661372 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5654f5a1a4b0 0x5654f5a1e750 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f5c9c00e2d0 tx=0x7f5c9c00e0b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1e6ba9dae30a8b29 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:42:03.980 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:03.978+0000 7f5c96a00640 10 client.0 ms_handle_connect on v2:172.21.15.167:3300/0 2024-08-19T01:42:03.980 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:03.978+0000 7f5c96a00640 1 -- 172.21.15.83:0/1888661372 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c9c018040 con 0x5654f5a1a4b0 2024-08-19T01:42:03.980 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:03.978+0000 7f5c96a00640 1 -- 172.21.15.83:0/1888661372 <== mon.1 v2:172.21.15.167:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8269+0+0 (secure 0 0 0) 0x7f5c9c01e400 con 0x5654f5a1a4b0 2024-08-19T01:42:03.980 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:03.979+0000 7f5cba8bb740 1 -- 172.21.15.83:0/1888661372 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5ca40ccf50 con 0x5654f5a1a4b0 2024-08-19T01:42:03.981 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:03.979+0000 7f5c96a00640 1 -- 172.21.15.83:0/1888661372 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f5c9c0059c0 con 0x5654f5a1a4b0 2024-08-19T01:42:03.981 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:03.980+0000 7f5cba8bb740 10 client.26014 fetch_fsmap learned FSMap version 28 2024-08-19T01:42:03.982 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:03.980+0000 7f5cba8bb740 1 -- 172.21.15.83:0/1888661372 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5654f5a9f6f0 con 0x5654f5a1a4b0 2024-08-19T01:42:03.982 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:03.980+0000 7f5c96a00640 1 -- 172.21.15.83:0/1888661372 <== mon.1 v2:172.21.15.167:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5ca40ccf50 con 0x5654f5a1a4b0 2024-08-19T01:42:03.982 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:03.980+0000 7f5c96a00640 1 -- 172.21.15.83:0/1888661372 <== mon.1 v2:172.21.15.167:3300/0 5 ==== fsmap(e 28) ==== 8358+0+0 (secure 0 0 0) 0x7f5c9c027c60 con 0x5654f5a1a4b0 2024-08-19T01:42:03.982 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:03.981+0000 7f5cba8bb740 10 client.26014 fetch_fsmap finished waiting for FSMap version 28 2024-08-19T01:42:03.982 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:03.981+0000 7f5cba8bb740 10 client.26014 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-19T01:42:03.983 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:03.981+0000 7f5cba8bb740 20 client.26014 populate_metadata read hostname 'smithi083' 2024-08-19T01:42:03.983 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:03.981+0000 7f5cba8bb740 1 --2- 172.21.15.83:0/1888661372 >> [v2:172.21.15.117:6838/1967595704,v1:172.21.15.117:6839/1967595704] conn(0x5654f5a9fed0 0x5654f5aa22f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:42:03.983 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:03.981+0000 7f5cba8bb740 4 client.26014 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-08-19T01:42:03.983 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:03.981+0000 7f5cba8bb740 1 -- 172.21.15.83:0/1888661372 --> [v2:172.21.15.117:6838/1967595704,v1:172.21.15.117:6839/1967595704] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5654f5987670 con 0x5654f5a9fed0 2024-08-19T01:42:03.983 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:03.981+0000 7f5cb5400640 1 --2- 172.21.15.83:0/1888661372 >> [v2:172.21.15.117:6838/1967595704,v1:172.21.15.117:6839/1967595704] conn(0x5654f5a9fed0 0x5654f5aa22f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:42:03.984 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:03.982+0000 7f5cb5400640 1 --2- 172.21.15.83:0/1888661372 >> [v2:172.21.15.117:6838/1967595704,v1:172.21.15.117:6839/1967595704] conn(0x5654f5a9fed0 0x5654f5aa22f0 crc :-1 s=READY pgs=130 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ff74b25d8f058158 server_cookie=e4d33ed4be3aa3c7 in_seq=0 out_seq=0 2024-08-19T01:42:03.984 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:03.982+0000 7f5c96a00640 10 client.26014 ms_handle_connect on v2:172.21.15.117:6838/1967595704 2024-08-19T01:42:03.985 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:03.984+0000 7f5c96a00640 1 -- 172.21.15.83:0/1888661372 <== mds.0 v2:172.21.15.117:6838/1967595704 1 ==== command_reply(tid 0: 0 ) ==== 8+0+269 (crc 0 0 0) 0x5654f5987670 con 0x5654f5a9fed0 2024-08-19T01:42:03.985 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:03.984+0000 7f5c96a00640 10 client.26014 handle_command_reply: tid=0 2024-08-19T01:42:03.986 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:03.984+0000 7f5cba8bb740 1 client.26014 shutdown 2024-08-19T01:42:03.986 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:03.985+0000 7f5cba8bb740 1 -- 172.21.15.83:0/1888661372 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5654f5a1a4b0 msgr2=0x5654f5a1e750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:42:03.986 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:03.985+0000 7f5cba8bb740 1 --2- 172.21.15.83:0/1888661372 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5654f5a1a4b0 0x5654f5a1e750 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f5c9c00e2d0 tx=0x7f5c9c00e0b0 comp rx=0 tx=0).stop 2024-08-19T01:42:03.987 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:03.985+0000 7f5cba8bb740 1 -- 172.21.15.83:0/1888661372 shutdown_connections 2024-08-19T01:42:03.987 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:03.985+0000 7f5cba8bb740 1 --2- 172.21.15.83:0/1888661372 >> [v2:172.21.15.117:6838/1967595704,v1:172.21.15.117:6839/1967595704] conn(0x5654f5a9fed0 0x5654f5aa22f0 crc :-1 s=READY pgs=130 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:03.987 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:03.985+0000 7f5cba8bb740 1 --2- 172.21.15.83:0/1888661372 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5654f5a24d70 0x5654f5a1ec90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:03.987 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:03.985+0000 7f5cba8bb740 1 --2- 172.21.15.83:0/1888661372 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5654f5a1a4b0 0x5654f5a1e750 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:03.987 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:03.985+0000 7f5c96a00640 0 client.26014 ms_handle_reset on v2:172.21.15.117:6838/1967595704 2024-08-19T01:42:03.987 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:03.985+0000 7f5cba8bb740 1 --2- 172.21.15.83:0/1888661372 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5654f5a19b90 0x5654f5a2d360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:03.987 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:03.985+0000 7f5cba8bb740 1 -- 172.21.15.83:0/1888661372 >> 172.21.15.83:0/1888661372 conn(0x5654f5a16120 msgr2=0x5654f5a15280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:42:03.987 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:03.986+0000 7f5cba8bb740 1 -- 172.21.15.83:0/1888661372 shutdown_connections 2024-08-19T01:42:03.988 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:03.986+0000 7f5cba8bb740 1 -- 172.21.15.83:0/1888661372 wait complete. 2024-08-19T01:42:03.988 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:03.987+0000 7f5cba8bb740 20 client.26014 trim_cache size 0 max 16384 2024-08-19T01:42:03.989 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-08-19T01:42:03.989 INFO:teuthology.orchestra.run.smithi083.stdout: "status": "scrub active (17100 inodes in the stack)", 2024-08-19T01:42:03.989 INFO:teuthology.orchestra.run.smithi083.stdout: "scrubs": { 2024-08-19T01:42:03.989 INFO:teuthology.orchestra.run.smithi083.stdout: "f1269cca-7069-4417-baf3-393540b92a77": { 2024-08-19T01:42:03.989 INFO:teuthology.orchestra.run.smithi083.stdout: "path": "/", 2024-08-19T01:42:03.989 INFO:teuthology.orchestra.run.smithi083.stdout: "tag": "f1269cca-7069-4417-baf3-393540b92a77", 2024-08-19T01:42:03.989 INFO:teuthology.orchestra.run.smithi083.stdout: "options": "recursive,force" 2024-08-19T01:42:03.989 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-08-19T01:42:03.989 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-08-19T01:42:03.989 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-08-19T01:42:03.991 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:03.989+0000 7f5cb7e00640 1 -- 172.21.15.83:0/2208192926 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f5c980a2c80 msgr2=0x7f5c980a5140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:42:03.991 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:03.989+0000 7f5cb7e00640 1 --2- 172.21.15.83:0/2208192926 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f5c980a2c80 0x7f5c980a5140 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f5ca8005f00 tx=0x7f5ca8049040 comp rx=0 tx=0).stop 2024-08-19T01:42:03.991 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:03.989+0000 7f5cb7e00640 1 -- 172.21.15.83:0/2208192926 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5cb0108f20 msgr2=0x7f5cb01164f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:42:03.991 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:03.989+0000 7f5cb7e00640 1 --2- 172.21.15.83:0/2208192926 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5cb0108f20 0x7f5cb01164f0 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f5ca400e9c0 tx=0x7f5ca400ee80 comp rx=0 tx=0).stop 2024-08-19T01:42:03.991 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:03.989+0000 7f5cb7e00640 1 -- 172.21.15.83:0/2208192926 shutdown_connections 2024-08-19T01:42:03.991 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:03.989+0000 7f5cb7e00640 1 --2- 172.21.15.83:0/2208192926 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f5c980a2c80 0x7f5c980a5140 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:03.991 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:03.989+0000 7f5cb7e00640 1 --2- 172.21.15.83:0/2208192926 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5cb0116bd0 0x7f5cb0113ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:03.991 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:03.989+0000 7f5cb7e00640 1 --2- 172.21.15.83:0/2208192926 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5cb0109840 0x7f5cb01135a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:03.991 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:03.989+0000 7f5cb7e00640 1 --2- 172.21.15.83:0/2208192926 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5cb0108f20 0x7f5cb01164f0 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:03.992 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:03.989+0000 7f5cb7e00640 1 -- 172.21.15.83:0/2208192926 >> 172.21.15.83:0/2208192926 conn(0x7f5cb006e000 msgr2=0x7f5cb0117070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:42:03.992 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:03.989+0000 7f5cb7e00640 1 -- 172.21.15.83:0/2208192926 shutdown_connections 2024-08-19T01:42:03.992 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:03.990+0000 7f5cb7e00640 1 -- 172.21.15.83:0/2208192926 wait complete. 2024-08-19T01:42:04.851 INFO:tasks.cephfs.filesystem:scrub status for tag:f1269cca-7069-4417-baf3-393540b92a77 - {'path': '/', 'tag': 'f1269cca-7069-4417-baf3-393540b92a77', 'options': 'recursive,force'} 2024-08-19T01:42:05.965 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:42:05 smithi167 ceph-mon[21875]: pgmap v556: 97 pgs: 97 active+clean; 426 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.9 MiB/s wr, 608 op/s 2024-08-19T01:42:06.154 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:42:05 smithi083 ceph-mon[17120]: pgmap v556: 97 pgs: 97 active+clean; 426 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.9 MiB/s wr, 608 op/s 2024-08-19T01:42:06.202 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:42:05 smithi117 ceph-mon[22016]: pgmap v556: 97 pgs: 97 active+clean; 426 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.9 MiB/s wr, 608 op/s 2024-08-19T01:42:07.966 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:42:07 smithi167 ceph-mon[21875]: pgmap v557: 97 pgs: 97 active+clean; 412 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.2 MiB/s wr, 643 op/s 2024-08-19T01:42:08.154 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:42:07 smithi083 ceph-mon[17120]: pgmap v557: 97 pgs: 97 active+clean; 412 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.2 MiB/s wr, 643 op/s 2024-08-19T01:42:08.202 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:42:07 smithi117 ceph-mon[22016]: pgmap v557: 97 pgs: 97 active+clean; 412 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.2 MiB/s wr, 643 op/s 2024-08-19T01:42:09.154 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:42:08 smithi083 ceph-mon[17120]: pgmap v558: 97 pgs: 97 active+clean; 393 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 1.8 MiB/s wr, 1.16k op/s 2024-08-19T01:42:09.202 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:42:08 smithi117 ceph-mon[22016]: pgmap v558: 97 pgs: 97 active+clean; 393 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 1.8 MiB/s wr, 1.16k op/s 2024-08-19T01:42:09.215 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:42:08 smithi167 ceph-mon[21875]: pgmap v558: 97 pgs: 97 active+clean; 393 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 1.8 MiB/s wr, 1.16k op/s 2024-08-19T01:42:11.904 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:42:11 smithi083 ceph-mon[17120]: pgmap v559: 97 pgs: 97 active+clean; 385 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 1.6 MiB/s wr, 1.29k op/s 2024-08-19T01:42:11.952 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:42:11 smithi117 ceph-mon[22016]: pgmap v559: 97 pgs: 97 active+clean; 385 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 1.6 MiB/s wr, 1.29k op/s 2024-08-19T01:42:11.965 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:42:11 smithi167 ceph-mon[21875]: pgmap v559: 97 pgs: 97 active+clean; 385 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 1.6 MiB/s wr, 1.29k op/s 2024-08-19T01:42:12.904 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:42:12 smithi083 ceph-mon[17120]: scrub summary: idle+waiting paths [/] 2024-08-19T01:42:12.952 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:42:12 smithi117 ceph-mon[22016]: scrub summary: idle+waiting paths [/] 2024-08-19T01:42:12.965 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:42:12 smithi167 ceph-mon[21875]: scrub summary: idle+waiting paths [/] 2024-08-19T01:42:13.904 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:42:13 smithi083 ceph-mon[17120]: pgmap v560: 97 pgs: 97 active+clean; 374 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 1.3 MiB/s wr, 1.53k op/s 2024-08-19T01:42:13.952 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:42:13 smithi117 ceph-mon[22016]: pgmap v560: 97 pgs: 97 active+clean; 374 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 1.3 MiB/s wr, 1.53k op/s 2024-08-19T01:42:13.965 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:42:13 smithi167 ceph-mon[21875]: pgmap v560: 97 pgs: 97 active+clean; 374 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 1.3 MiB/s wr, 1.53k op/s 2024-08-19T01:42:15.677 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:15.675+0000 7f192ae00640 1 -- 172.21.15.83:0/3693953522 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1924104ca0 msgr2=0x7f1924105120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:42:15.677 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:15.675+0000 7f192ae00640 1 --2- 172.21.15.83:0/3693953522 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1924104ca0 0x7f1924105120 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f1910009900 tx=0x7f191002a420 comp rx=0 tx=0).stop 2024-08-19T01:42:15.677 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:15.675+0000 7f192ae00640 1 -- 172.21.15.83:0/3693953522 shutdown_connections 2024-08-19T01:42:15.677 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:15.675+0000 7f192ae00640 1 --2- 172.21.15.83:0/3693953522 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1924105660 0x7f192410bef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:15.677 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:15.675+0000 7f192ae00640 1 --2- 172.21.15.83:0/3693953522 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1924104ca0 0x7f1924105120 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:15.677 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:15.675+0000 7f192ae00640 1 --2- 172.21.15.83:0/3693953522 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1924103ab0 0x7f1924103eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:15.677 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:15.675+0000 7f192ae00640 1 -- 172.21.15.83:0/3693953522 >> 172.21.15.83:0/3693953522 conn(0x7f19240ff210 msgr2=0x7f1924101650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:42:15.677 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:15.675+0000 7f192ae00640 1 -- 172.21.15.83:0/3693953522 shutdown_connections 2024-08-19T01:42:15.679 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:15.676+0000 7f192ae00640 1 -- 172.21.15.83:0/3693953522 wait complete. 2024-08-19T01:42:15.679 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:15.677+0000 7f192ae00640 1 Processor -- start 2024-08-19T01:42:15.680 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:15.677+0000 7f192ae00640 1 -- start start 2024-08-19T01:42:15.680 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:15.677+0000 7f192ae00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1924103ab0 0x7f1924070810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:42:15.680 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:15.677+0000 7f192ae00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1924104ca0 0x7f1924070d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:42:15.680 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:15.677+0000 7f192ae00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1924105660 0x7f1924198f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:42:15.680 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:15.677+0000 7f192ae00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f192410f800 con 0x7f1924103ab0 2024-08-19T01:42:15.680 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:15.677+0000 7f192ae00640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f192410f680 con 0x7f1924105660 2024-08-19T01:42:15.680 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:15.677+0000 7f192ae00640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f192410f980 con 0x7f1924104ca0 2024-08-19T01:42:15.680 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:15.678+0000 7f1923400640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1924104ca0 0x7f1924070d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:42:15.680 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:15.678+0000 7f1923400640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1924104ca0 0x7f1924070d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.83:55690/0 (socket says 172.21.15.83:55690) 2024-08-19T01:42:15.680 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:15.678+0000 7f1923400640 1 -- 172.21.15.83:0/2452139578 learned_addr learned my addr 172.21.15.83:0/2452139578 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-08-19T01:42:15.680 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:15.678+0000 7f1923e00640 1 --2- 172.21.15.83:0/2452139578 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1924103ab0 0x7f1924070810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:42:15.681 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:15.678+0000 7f1928e00640 1 --2- 172.21.15.83:0/2452139578 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1924105660 0x7f1924198f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:42:15.681 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:15.679+0000 7f1923400640 1 -- 172.21.15.83:0/2452139578 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1924105660 msgr2=0x7f1924198f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:42:15.681 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:15.679+0000 7f1923400640 1 --2- 172.21.15.83:0/2452139578 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1924105660 0x7f1924198f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:15.681 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:15.679+0000 7f1923400640 1 -- 172.21.15.83:0/2452139578 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1924103ab0 msgr2=0x7f1924070810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:42:15.681 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:15.679+0000 7f1923400640 1 --2- 172.21.15.83:0/2452139578 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1924103ab0 0x7f1924070810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:15.682 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:15.679+0000 7f1923400640 1 -- 172.21.15.83:0/2452139578 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1924199440 con 0x7f1924104ca0 2024-08-19T01:42:15.682 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:15.679+0000 7f1928e00640 1 --2- 172.21.15.83:0/2452139578 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1924105660 0x7f1924198f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-19T01:42:15.682 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:15.679+0000 7f1923e00640 1 --2- 172.21.15.83:0/2452139578 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1924103ab0 0x7f1924070810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-19T01:42:15.682 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:15.679+0000 7f1923400640 1 --2- 172.21.15.83:0/2452139578 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1924104ca0 0x7f1924070d50 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f1910031610 tx=0x7f1910032c80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:42:15.682 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:15.680+0000 7f1920c00640 1 -- 172.21.15.83:0/2452139578 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f191003f030 con 0x7f1924104ca0 2024-08-19T01:42:15.682 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:15.680+0000 7f1920c00640 1 -- 172.21.15.83:0/2452139578 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1910003ad0 con 0x7f1924104ca0 2024-08-19T01:42:15.682 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:15.680+0000 7f192ae00640 1 -- 172.21.15.83:0/2452139578 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f19241996d0 con 0x7f1924104ca0 2024-08-19T01:42:15.683 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:15.681+0000 7f192ae00640 1 -- 172.21.15.83:0/2452139578 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1924199c40 con 0x7f1924104ca0 2024-08-19T01:42:15.683 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:15.681+0000 7f1920c00640 1 -- 172.21.15.83:0/2452139578 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1910035070 con 0x7f1924104ca0 2024-08-19T01:42:15.686 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:15.685+0000 7f192ddfd740 1 Processor -- start 2024-08-19T01:42:15.686 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:15.685+0000 7f1920c00640 1 -- 172.21.15.83:0/2452139578 <== mon.1 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7f1910042030 con 0x7f1924104ca0 2024-08-19T01:42:15.687 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:15.685+0000 7f192ddfd740 1 -- start start 2024-08-19T01:42:15.687 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:15.686+0000 7f192ddfd740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559596f91470 0x559596f91870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:42:15.687 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:15.686+0000 7f192ddfd740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559596f91db0 0x559596f96870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:42:15.688 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:15.686+0000 7f192ddfd740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559596f96e40 0x559596f99300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:42:15.688 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:15.686+0000 7f192ddfd740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x559596ef5890 con 0x559596f91470 2024-08-19T01:42:15.688 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:15.686+0000 7f192ddfd740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x559596e8dd90 con 0x559596f91db0 2024-08-19T01:42:15.688 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:15.686+0000 7f192ddfd740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x559596ef1850 con 0x559596f96e40 2024-08-19T01:42:15.688 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:15.686+0000 7f1923e00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559596f91db0 0x559596f96870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:42:15.688 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:15.686+0000 7f1923e00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559596f91db0 0x559596f96870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.83:46816/0 (socket says 172.21.15.83:46816) 2024-08-19T01:42:15.688 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:15.686+0000 7f1923e00640 1 -- 172.21.15.83:0/3504588369 learned_addr learned my addr 172.21.15.83:0/3504588369 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-08-19T01:42:15.688 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:15.686+0000 7f1923400640 1 --2- 172.21.15.83:0/3504588369 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559596f96e40 0x559596f99300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:42:15.688 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:15.687+0000 7f1923e00640 1 -- 172.21.15.83:0/3504588369 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559596f96e40 msgr2=0x559596f99300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:42:15.689 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:15.687+0000 7f1923e00640 1 --2- 172.21.15.83:0/3504588369 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559596f96e40 0x559596f99300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:15.689 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:15.687+0000 7f1923e00640 1 -- 172.21.15.83:0/3504588369 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559596f91470 msgr2=0x559596f91870 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-08-19T01:42:15.689 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:15.687+0000 7f1923e00640 1 --2- 172.21.15.83:0/3504588369 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559596f91470 0x559596f91870 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:15.689 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:15.687+0000 7f1923e00640 1 -- 172.21.15.83:0/3504588369 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559596c09210 con 0x559596f91db0 2024-08-19T01:42:15.689 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:15.687+0000 7f1923400640 1 --2- 172.21.15.83:0/3504588369 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559596f96e40 0x559596f99300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-19T01:42:15.690 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:15.688+0000 7f1923e00640 1 --2- 172.21.15.83:0/3504588369 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559596f91db0 0x559596f96870 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f1914009840 tx=0x7f1914009d00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9262f526f41033f9 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:42:15.690 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:15.688+0000 7f1920c00640 1 --2- 172.21.15.83:0/2452139578 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f19080a2df0 0x7f19080a52b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:42:15.690 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:15.688+0000 7f1920c00640 1 -- 172.21.15.83:0/2452139578 <== mon.1 v2:172.21.15.167:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8269+0+0 (secure 0 0 0) 0x7f19100ef550 con 0x7f1924104ca0 2024-08-19T01:42:15.690 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:15.689+0000 7f1928e00640 1 --2- 172.21.15.83:0/2452139578 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f19080a2df0 0x7f19080a52b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:42:15.691 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:15.689+0000 7f1919600640 1 -- 172.21.15.83:0/3504588369 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f191400d950 con 0x559596f91db0 2024-08-19T01:42:15.691 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:15.689+0000 7f1919600640 1 -- 172.21.15.83:0/3504588369 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1914019070 con 0x559596f91db0 2024-08-19T01:42:15.691 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:15.689+0000 7f1919600640 1 -- 172.21.15.83:0/3504588369 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19140123e0 con 0x559596f91db0 2024-08-19T01:42:15.692 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:15.691+0000 7f1928e00640 1 --2- 172.21.15.83:0/2452139578 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f19080a2df0 0x7f19080a52b0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f191c005fd0 tx=0x7f191c009500 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:42:15.692 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:15.691+0000 7f192ddfd740 1 -- 172.21.15.83:0/3504588369 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559596f91db0 msgr2=0x559596f96870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:42:15.692 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:15.691+0000 7f192ddfd740 1 --2- 172.21.15.83:0/3504588369 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559596f91db0 0x559596f96870 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f1914009840 tx=0x7f1914009d00 comp rx=0 tx=0).stop 2024-08-19T01:42:15.693 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:15.691+0000 7f192ddfd740 1 -- 172.21.15.83:0/3504588369 shutdown_connections 2024-08-19T01:42:15.693 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:15.691+0000 7f192ddfd740 1 --2- 172.21.15.83:0/3504588369 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559596f96e40 0x559596f99300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:15.693 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:15.691+0000 7f192ddfd740 1 --2- 172.21.15.83:0/3504588369 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559596f91db0 0x559596f96870 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:15.693 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:15.691+0000 7f192ddfd740 1 --2- 172.21.15.83:0/3504588369 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559596f91470 0x559596f91870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:15.693 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:15.691+0000 7f192ddfd740 1 -- 172.21.15.83:0/3504588369 >> 172.21.15.83:0/3504588369 conn(0x559596f8cae0 msgr2=0x559596f8cee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:42:15.693 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:15.692+0000 7f192ddfd740 1 -- 172.21.15.83:0/3504588369 shutdown_connections 2024-08-19T01:42:15.693 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:15.692+0000 7f192ddfd740 1 -- 172.21.15.83:0/3504588369 wait complete. 2024-08-19T01:42:15.694 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:15.693+0000 7f192ddfd740 1 -- start start 2024-08-19T01:42:15.694 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:15.693+0000 7f192ddfd740 1 Processor -- start 2024-08-19T01:42:15.695 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:15.694+0000 7f192ddfd740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559596f91470 0x559596fa3f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:42:15.696 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:15.694+0000 7f192ddfd740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559596f91db0 0x559596fa44b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:42:15.696 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:15.694+0000 7f192ddfd740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559596f96e40 0x559596fab4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:42:15.696 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:15.694+0000 7f192ddfd740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x559596f99f00 con 0x559596f96e40 2024-08-19T01:42:15.697 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:15.694+0000 7f192ddfd740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x559596eb0f70 con 0x559596f91470 2024-08-19T01:42:15.697 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:15.694+0000 7f192ddfd740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x559596f9a080 con 0x559596f91db0 2024-08-19T01:42:15.697 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:15.694+0000 7f1923400640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559596f91db0 0x559596fa44b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:42:15.697 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:15.694+0000 7f192ddfd740 10 client.0 _ll_register_callbacks cb 0x559596eed8d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-19T01:42:15.697 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:15.694+0000 7f1923400640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559596f91db0 0x559596fa44b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.83:55706/0 (socket says 172.21.15.83:55706) 2024-08-19T01:42:15.697 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:15.694+0000 7f1923400640 1 -- 172.21.15.83:0/3547529452 learned_addr learned my addr 172.21.15.83:0/3547529452 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-08-19T01:42:15.697 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:15.694+0000 7f1928e00640 1 --2- 172.21.15.83:0/3547529452 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559596f96e40 0x559596fab4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:42:15.697 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:15.695+0000 7f1923e00640 1 --2- 172.21.15.83:0/3547529452 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559596f91470 0x559596fa3f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:42:15.697 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:15.695+0000 7f1923400640 1 -- 172.21.15.83:0/3547529452 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559596f91470 msgr2=0x559596fa3f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:42:15.697 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:15.695+0000 7f1923400640 1 --2- 172.21.15.83:0/3547529452 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559596f91470 0x559596fa3f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:15.697 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:15.695+0000 7f1923400640 1 -- 172.21.15.83:0/3547529452 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559596f96e40 msgr2=0x559596fab4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:42:15.698 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:15.695+0000 7f1923400640 1 --2- 172.21.15.83:0/3547529452 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559596f96e40 0x559596fab4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:15.698 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:15.695+0000 7f1923400640 1 -- 172.21.15.83:0/3547529452 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559596cd5620 con 0x559596f91db0 2024-08-19T01:42:15.698 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:15.695+0000 7f1928e00640 1 --2- 172.21.15.83:0/3547529452 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559596f96e40 0x559596fab4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-19T01:42:15.698 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:15.697+0000 7f1923400640 1 --2- 172.21.15.83:0/3547529452 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559596f91db0 0x559596fa44b0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f191002aa10 tx=0x7f19100bb990 comp rx=0 tx=0).ready entity=mon.1 client_cookie=847237febbadbe77 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:42:15.698 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:15.697+0000 7f190e000640 10 client.0 ms_handle_connect on v2:172.21.15.167:3300/0 2024-08-19T01:42:15.700 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:15.697+0000 7f190e000640 1 -- 172.21.15.83:0/3547529452 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1910049040 con 0x559596f91db0 2024-08-19T01:42:15.700 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:15.697+0000 7f190e000640 1 -- 172.21.15.83:0/3547529452 <== mon.1 v2:172.21.15.167:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8269+0+0 (secure 0 0 0) 0x7f1910003c80 con 0x559596f91db0 2024-08-19T01:42:15.700 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:15.697+0000 7f192ddfd740 1 -- 172.21.15.83:0/3547529452 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1914019070 con 0x559596f91db0 2024-08-19T01:42:15.700 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:15.698+0000 7f190e000640 1 -- 172.21.15.83:0/3547529452 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f19100bbd90 con 0x559596f91db0 2024-08-19T01:42:15.700 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:15.698+0000 7f192ddfd740 10 client.26020 fetch_fsmap learned FSMap version 28 2024-08-19T01:42:15.700 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:15.698+0000 7f192ddfd740 1 -- 172.21.15.83:0/3547529452 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x559596f99f70 con 0x559596f91db0 2024-08-19T01:42:15.700 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:15.698+0000 7f190e000640 1 -- 172.21.15.83:0/3547529452 <== mon.1 v2:172.21.15.167:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1914019070 con 0x559596f91db0 2024-08-19T01:42:15.700 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:15.698+0000 7f190e000640 1 -- 172.21.15.83:0/3547529452 <== mon.1 v2:172.21.15.167:3300/0 5 ==== fsmap(e 28) ==== 8358+0+0 (secure 0 0 0) 0x7f191003b4f0 con 0x559596f91db0 2024-08-19T01:42:15.700 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:15.698+0000 7f192ddfd740 10 client.26020 fetch_fsmap finished waiting for FSMap version 28 2024-08-19T01:42:15.700 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:15.698+0000 7f192ddfd740 10 client.26020 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-08-19T01:42:15.700 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:15.698+0000 7f192ddfd740 20 client.26020 populate_metadata read hostname 'smithi083' 2024-08-19T01:42:15.700 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:15.698+0000 7f192ddfd740 1 --2- 172.21.15.83:0/3547529452 >> [v2:172.21.15.117:6838/1967595704,v1:172.21.15.117:6839/1967595704] conn(0x559596f9cb40 0x559596f9efa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:42:15.701 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:15.698+0000 7f192ddfd740 4 client.26020 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-19T01:42:15.701 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:15.698+0000 7f192ddfd740 1 -- 172.21.15.83:0/3547529452 --> [v2:172.21.15.117:6838/1967595704,v1:172.21.15.117:6839/1967595704] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559596f02fc0 con 0x559596f9cb40 2024-08-19T01:42:15.701 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:15.698+0000 7f1923e00640 1 --2- 172.21.15.83:0/3547529452 >> [v2:172.21.15.117:6838/1967595704,v1:172.21.15.117:6839/1967595704] conn(0x559596f9cb40 0x559596f9efa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:42:15.701 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:15.699+0000 7f1923e00640 1 --2- 172.21.15.83:0/3547529452 >> [v2:172.21.15.117:6838/1967595704,v1:172.21.15.117:6839/1967595704] conn(0x559596f9cb40 0x559596f9efa0 crc :-1 s=READY pgs=131 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f29826077b749fa5 server_cookie=400788f05d41ada6 in_seq=0 out_seq=0 2024-08-19T01:42:15.701 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:15.699+0000 7f190e000640 10 client.26020 ms_handle_connect on v2:172.21.15.117:6838/1967595704 2024-08-19T01:42:15.702 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:15.701+0000 7f190e000640 1 -- 172.21.15.83:0/3547529452 <== mds.0 v2:172.21.15.117:6838/1967595704 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x559596f02fc0 con 0x559596f9cb40 2024-08-19T01:42:15.703 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:15.701+0000 7f190e000640 10 client.26020 handle_command_reply: tid=0 2024-08-19T01:42:15.703 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:15.701+0000 7f192ddfd740 1 client.26020 shutdown 2024-08-19T01:42:15.703 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:15.702+0000 7f192ddfd740 1 -- 172.21.15.83:0/3547529452 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559596f91db0 msgr2=0x559596fa44b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:42:15.703 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:15.702+0000 7f192ddfd740 1 --2- 172.21.15.83:0/3547529452 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559596f91db0 0x559596fa44b0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f191002aa10 tx=0x7f19100bb990 comp rx=0 tx=0).stop 2024-08-19T01:42:15.704 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:15.702+0000 7f192ddfd740 1 -- 172.21.15.83:0/3547529452 shutdown_connections 2024-08-19T01:42:15.704 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:15.702+0000 7f192ddfd740 1 --2- 172.21.15.83:0/3547529452 >> [v2:172.21.15.117:6838/1967595704,v1:172.21.15.117:6839/1967595704] conn(0x559596f9cb40 0x559596f9efa0 crc :-1 s=READY pgs=131 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:15.704 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:15.702+0000 7f192ddfd740 1 --2- 172.21.15.83:0/3547529452 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559596f96e40 0x559596fab4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:15.704 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:15.702+0000 7f192ddfd740 1 --2- 172.21.15.83:0/3547529452 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559596f91db0 0x559596fa44b0 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:15.704 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:15.702+0000 7f192ddfd740 1 --2- 172.21.15.83:0/3547529452 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559596f91470 0x559596fa3f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:15.704 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:15.702+0000 7f192ddfd740 1 -- 172.21.15.83:0/3547529452 >> 172.21.15.83:0/3547529452 conn(0x559596f8cae0 msgr2=0x559596f8e2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:42:15.704 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:15.702+0000 7f190e000640 0 client.26020 ms_handle_reset on v2:172.21.15.117:6838/1967595704 2024-08-19T01:42:15.704 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:15.702+0000 7f192ddfd740 1 -- 172.21.15.83:0/3547529452 shutdown_connections 2024-08-19T01:42:15.705 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:15.702+0000 7f192ddfd740 1 -- 172.21.15.83:0/3547529452 wait complete. 2024-08-19T01:42:15.705 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:15.703+0000 7f192ddfd740 20 client.26020 trim_cache size 0 max 16384 2024-08-19T01:42:15.732 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:15.730+0000 7f192ddfd740 1 Processor -- start 2024-08-19T01:42:15.732 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:15.730+0000 7f192ddfd740 1 -- start start 2024-08-19T01:42:15.732 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:15.731+0000 7f192ddfd740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559596f91470 0x559596f92ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:42:15.733 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:15.731+0000 7f192ddfd740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559596f91db0 0x559596f93430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:42:15.733 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:15.731+0000 7f192ddfd740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559596f96e40 0x559596f8d990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:42:15.733 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:15.731+0000 7f192ddfd740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f1904010d80 con 0x559596f91470 2024-08-19T01:42:15.733 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:15.731+0000 7f192ddfd740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f190400e620 con 0x559596f91db0 2024-08-19T01:42:15.733 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:15.731+0000 7f192ddfd740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f190400c470 con 0x559596f96e40 2024-08-19T01:42:15.733 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:15.731+0000 7f1928e00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559596f96e40 0x559596f8d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:42:15.733 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:15.731+0000 7f1928e00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559596f96e40 0x559596f8d990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.83:55714/0 (socket says 172.21.15.83:55714) 2024-08-19T01:42:15.733 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:15.731+0000 7f1928e00640 1 -- 172.21.15.83:0/2284104453 learned_addr learned my addr 172.21.15.83:0/2284104453 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-08-19T01:42:15.733 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:15.731+0000 7f1923400640 1 --2- 172.21.15.83:0/2284104453 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559596f91db0 0x559596f93430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:42:15.734 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:15.732+0000 7f1923e00640 1 --2- 172.21.15.83:0/2284104453 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559596f91470 0x559596f92ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:42:15.734 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:15.732+0000 7f1928e00640 1 -- 172.21.15.83:0/2284104453 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559596f91db0 msgr2=0x559596f93430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:42:15.734 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:15.732+0000 7f1928e00640 1 --2- 172.21.15.83:0/2284104453 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559596f91db0 0x559596f93430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:15.735 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:15.732+0000 7f1928e00640 1 -- 172.21.15.83:0/2284104453 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559596f91470 msgr2=0x559596f92ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:42:15.735 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:15.732+0000 7f1928e00640 1 --2- 172.21.15.83:0/2284104453 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559596f91470 0x559596f92ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:15.735 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:15.732+0000 7f1928e00640 1 -- 172.21.15.83:0/2284104453 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559596f9f620 con 0x559596f96e40 2024-08-19T01:42:15.735 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:15.732+0000 7f1923e00640 1 --2- 172.21.15.83:0/2284104453 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559596f91470 0x559596f92ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-19T01:42:15.735 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:15.733+0000 7f1928e00640 1 --2- 172.21.15.83:0/2284104453 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559596f96e40 0x559596f8d990 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f191c011760 tx=0x7f191c012000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=56ffdf7995532575 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:42:15.735 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:15.733+0000 7f1918c00640 1 -- 172.21.15.83:0/2284104453 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f191c004d20 con 0x559596f96e40 2024-08-19T01:42:15.735 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:15.733+0000 7f1918c00640 1 -- 172.21.15.83:0/2284104453 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f191c02fa70 con 0x559596f96e40 2024-08-19T01:42:15.735 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:15.733+0000 7f1918c00640 1 -- 172.21.15.83:0/2284104453 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f191c0057a0 con 0x559596f96e40 2024-08-19T01:42:15.735 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:15.734+0000 7f192ddfd740 1 -- 172.21.15.83:0/2284104453 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559596f96e40 msgr2=0x559596f8d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:42:15.736 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:15.734+0000 7f192ddfd740 1 --2- 172.21.15.83:0/2284104453 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559596f96e40 0x559596f8d990 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f191c011760 tx=0x7f191c012000 comp rx=0 tx=0).stop 2024-08-19T01:42:15.736 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:15.735+0000 7f192ddfd740 1 -- 172.21.15.83:0/2284104453 shutdown_connections 2024-08-19T01:42:15.736 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:15.735+0000 7f192ddfd740 1 --2- 172.21.15.83:0/2284104453 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559596f96e40 0x559596f8d990 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:15.736 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:15.735+0000 7f192ddfd740 1 --2- 172.21.15.83:0/2284104453 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559596f91db0 0x559596f93430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:15.736 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:15.735+0000 7f192ddfd740 1 --2- 172.21.15.83:0/2284104453 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559596f91470 0x559596f92ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:15.736 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:15.735+0000 7f192ddfd740 1 -- 172.21.15.83:0/2284104453 >> 172.21.15.83:0/2284104453 conn(0x559596f8cae0 msgr2=0x559596fa3f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:42:15.736 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:15.735+0000 7f192ddfd740 1 -- 172.21.15.83:0/2284104453 shutdown_connections 2024-08-19T01:42:15.737 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:15.736+0000 7f192ddfd740 1 -- 172.21.15.83:0/2284104453 wait complete. 2024-08-19T01:42:15.738 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:15.736+0000 7f192ddfd740 1 -- start start 2024-08-19T01:42:15.738 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:15.737+0000 7f192ddfd740 1 Processor -- start 2024-08-19T01:42:15.739 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:15.737+0000 7f192ddfd740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559596f91470 0x559596fb2a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:42:15.739 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:15.737+0000 7f192ddfd740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559596f91db0 0x559596fb2fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:42:15.739 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:15.737+0000 7f192ddfd740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559596f96e40 0x5595970105b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:42:15.739 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:15.737+0000 7f192ddfd740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f190400cae0 con 0x559596f91470 2024-08-19T01:42:15.740 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:15.737+0000 7f192ddfd740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f190400df50 con 0x559596f91db0 2024-08-19T01:42:15.740 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:15.737+0000 7f192ddfd740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f190400d1b0 con 0x559596f96e40 2024-08-19T01:42:15.740 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:15.737+0000 7f192ddfd740 10 client.0 _ll_register_callbacks cb 0x7f1910009260 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-19T01:42:15.740 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:15.738+0000 7f1928e00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559596f96e40 0x5595970105b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:42:15.740 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:15.738+0000 7f1923e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559596f91470 0x559596fb2a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:42:15.740 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:15.738+0000 7f1923e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559596f91470 0x559596fb2a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:53080/0 (socket says 172.21.15.83:53080) 2024-08-19T01:42:15.740 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:15.738+0000 7f1923e00640 1 -- 172.21.15.83:0/3966268887 learned_addr learned my addr 172.21.15.83:0/3966268887 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-08-19T01:42:15.741 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:15.738+0000 7f1923e00640 1 -- 172.21.15.83:0/3966268887 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559596f96e40 msgr2=0x5595970105b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:42:15.741 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:15.738+0000 7f1923e00640 1 --2- 172.21.15.83:0/3966268887 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559596f96e40 0x5595970105b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:15.741 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:15.738+0000 7f1923400640 1 --2- 172.21.15.83:0/3966268887 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559596f91db0 0x559596fb2fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:42:15.741 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:15.738+0000 7f1923e00640 1 -- 172.21.15.83:0/3966268887 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559596f91db0 msgr2=0x559596fb2fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:42:15.741 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:15.738+0000 7f1923e00640 1 --2- 172.21.15.83:0/3966268887 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559596f91db0 0x559596fb2fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:15.741 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:15.738+0000 7f1923e00640 1 -- 172.21.15.83:0/3966268887 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559597010af0 con 0x559596f91470 2024-08-19T01:42:15.741 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:15.738+0000 7f1923400640 1 --2- 172.21.15.83:0/3966268887 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559596f91db0 0x559596fb2fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-19T01:42:15.742 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:15.739+0000 7f1923e00640 1 --2- 172.21.15.83:0/3966268887 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559596f91470 0x559596fb2a70 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f1914006de0 tx=0x7f1914006d70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dcb1123c6787adda server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:42:15.742 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:15.739+0000 7f190e000640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-08-19T01:42:15.742 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:15.739+0000 7f190e000640 1 -- 172.21.15.83:0/3966268887 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f191402a070 con 0x559596f91470 2024-08-19T01:42:15.742 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:15.739+0000 7f190e000640 1 -- 172.21.15.83:0/3966268887 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8269+0+0 (secure 0 0 0) 0x7f191400d950 con 0x559596f91470 2024-08-19T01:42:15.742 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:15.739+0000 7f192ddfd740 1 -- 172.21.15.83:0/3966268887 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f191c02fa70 con 0x559596f91470 2024-08-19T01:42:15.742 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:15.740+0000 7f190e000640 1 -- 172.21.15.83:0/3966268887 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f191400db70 con 0x559596f91470 2024-08-19T01:42:15.742 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:15.740+0000 7f192ddfd740 10 client.16041 fetch_fsmap learned FSMap version 28 2024-08-19T01:42:15.742 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:15.740+0000 7f192ddfd740 1 -- 172.21.15.83:0/3966268887 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x559597011d50 con 0x559596f91470 2024-08-19T01:42:15.743 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:15.741+0000 7f190e000640 1 -- 172.21.15.83:0/3966268887 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f191c02fa70 con 0x559596f91470 2024-08-19T01:42:15.743 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:15.741+0000 7f190e000640 1 -- 172.21.15.83:0/3966268887 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 28) ==== 8358+0+0 (secure 0 0 0) 0x7f1914013070 con 0x559596f91470 2024-08-19T01:42:15.743 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:15.742+0000 7f192ddfd740 10 client.16041 fetch_fsmap finished waiting for FSMap version 28 2024-08-19T01:42:15.743 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:15.742+0000 7f192ddfd740 10 client.16041 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-08-19T01:42:15.743 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:15.742+0000 7f192ddfd740 20 client.16041 populate_metadata read hostname 'smithi083' 2024-08-19T01:42:15.743 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:15.742+0000 7f192ddfd740 1 --2- 172.21.15.83:0/3966268887 >> [v2:172.21.15.117:6838/1967595704,v1:172.21.15.117:6839/1967595704] conn(0x559597012420 0x559597014810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:42:15.744 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:15.742+0000 7f1923400640 1 --2- 172.21.15.83:0/3966268887 >> [v2:172.21.15.117:6838/1967595704,v1:172.21.15.117:6839/1967595704] conn(0x559597012420 0x559597014810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:42:15.744 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:15.742+0000 7f192ddfd740 4 client.16041 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-08-19T01:42:15.744 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:15.742+0000 7f192ddfd740 1 -- 172.21.15.83:0/3966268887 --> [v2:172.21.15.117:6838/1967595704,v1:172.21.15.117:6839/1967595704] -- command(tid 0: {"prefix": "flush journal"}) -- 0x559596efd310 con 0x559597012420 2024-08-19T01:42:15.744 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:15.743+0000 7f1923400640 1 --2- 172.21.15.83:0/3966268887 >> [v2:172.21.15.117:6838/1967595704,v1:172.21.15.117:6839/1967595704] conn(0x559597012420 0x559597014810 crc :-1 s=READY pgs=132 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e1f989858384cb05 server_cookie=abc5350f112ac0b8 in_seq=0 out_seq=0 2024-08-19T01:42:15.744 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:15.743+0000 7f190e000640 10 client.16041 ms_handle_connect on v2:172.21.15.117:6838/1967595704 2024-08-19T01:42:15.904 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:42:15 smithi083 ceph-mon[17120]: pgmap v561: 97 pgs: 97 active+clean; 359 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 2.0 MiB/s wr, 2.13k op/s 2024-08-19T01:42:15.905 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:42:15 smithi083 ceph-mon[17120]: scrub summary: idle 2024-08-19T01:42:15.952 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:42:15 smithi117 ceph-mon[22016]: pgmap v561: 97 pgs: 97 active+clean; 359 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 2.0 MiB/s wr, 2.13k op/s 2024-08-19T01:42:15.952 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:42:15 smithi117 ceph-mon[22016]: scrub summary: idle 2024-08-19T01:42:15.961 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:15.958+0000 7f190e000640 1 -- 172.21.15.83:0/3966268887 <== mds.0 v2:172.21.15.117:6838/1967595704 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x559596efd310 con 0x559597012420 2024-08-19T01:42:15.961 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:15.958+0000 7f190e000640 10 client.16041 handle_command_reply: tid=0 2024-08-19T01:42:15.961 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:15.959+0000 7f192ddfd740 1 client.16041 shutdown 2024-08-19T01:42:15.961 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:15.959+0000 7f192ddfd740 1 -- 172.21.15.83:0/3966268887 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559596f91470 msgr2=0x559596fb2a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:42:15.961 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:15.959+0000 7f192ddfd740 1 --2- 172.21.15.83:0/3966268887 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559596f91470 0x559596fb2a70 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f1914006de0 tx=0x7f1914006d70 comp rx=0 tx=0).stop 2024-08-19T01:42:15.961 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:15.959+0000 7f192ddfd740 1 -- 172.21.15.83:0/3966268887 shutdown_connections 2024-08-19T01:42:15.961 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:15.959+0000 7f192ddfd740 1 --2- 172.21.15.83:0/3966268887 >> [v2:172.21.15.117:6838/1967595704,v1:172.21.15.117:6839/1967595704] conn(0x559597012420 0x559597014810 crc :-1 s=READY pgs=132 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:15.961 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:15.959+0000 7f192ddfd740 1 --2- 172.21.15.83:0/3966268887 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559596f96e40 0x5595970105b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:15.962 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:15.959+0000 7f192ddfd740 1 --2- 172.21.15.83:0/3966268887 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559596f91db0 0x559596fb2fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:15.962 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:15.959+0000 7f192ddfd740 1 --2- 172.21.15.83:0/3966268887 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559596f91470 0x559596fb2a70 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:15.962 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:15.959+0000 7f192ddfd740 1 -- 172.21.15.83:0/3966268887 >> 172.21.15.83:0/3966268887 conn(0x559596f8cae0 msgr2=0x559596f8cec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:42:15.962 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:15.959+0000 7f190e000640 0 client.16041 ms_handle_reset on v2:172.21.15.117:6838/1967595704 2024-08-19T01:42:15.962 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:15.960+0000 7f192ddfd740 1 -- 172.21.15.83:0/3966268887 shutdown_connections 2024-08-19T01:42:15.963 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:15.960+0000 7f192ddfd740 1 -- 172.21.15.83:0/3966268887 wait complete. 2024-08-19T01:42:15.963 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:15.960+0000 7f192ddfd740 20 client.16041 trim_cache size 0 max 16384 2024-08-19T01:42:15.963 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stdout:{ 2024-08-19T01:42:15.963 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stdout: "message": "", 2024-08-19T01:42:15.963 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stdout: "return_code": 0 2024-08-19T01:42:15.963 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stdout:} 2024-08-19T01:42:15.964 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:15.961+0000 7f192ae00640 1 -- 172.21.15.83:0/2452139578 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f19080a2df0 msgr2=0x7f19080a52b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:42:15.964 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:15.961+0000 7f192ae00640 1 --2- 172.21.15.83:0/2452139578 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f19080a2df0 0x7f19080a52b0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f191c005fd0 tx=0x7f191c009500 comp rx=0 tx=0).stop 2024-08-19T01:42:15.964 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:15.961+0000 7f192ae00640 1 -- 172.21.15.83:0/2452139578 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1924104ca0 msgr2=0x7f1924070d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:42:15.964 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:15.961+0000 7f192ae00640 1 --2- 172.21.15.83:0/2452139578 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1924104ca0 0x7f1924070d50 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f1910031610 tx=0x7f1910032c80 comp rx=0 tx=0).stop 2024-08-19T01:42:15.964 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:15.961+0000 7f192ae00640 1 -- 172.21.15.83:0/2452139578 shutdown_connections 2024-08-19T01:42:15.964 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:15.961+0000 7f192ae00640 1 --2- 172.21.15.83:0/2452139578 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f19080a2df0 0x7f19080a52b0 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:15.964 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:15.961+0000 7f192ae00640 1 --2- 172.21.15.83:0/2452139578 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1924105660 0x7f1924198f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:15.964 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:15.961+0000 7f192ae00640 1 --2- 172.21.15.83:0/2452139578 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1924104ca0 0x7f1924070d50 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:15.965 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:15.961+0000 7f192ae00640 1 --2- 172.21.15.83:0/2452139578 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1924103ab0 0x7f1924070810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:15.965 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:15.961+0000 7f192ae00640 1 -- 172.21.15.83:0/2452139578 >> 172.21.15.83:0/2452139578 conn(0x7f19240ff210 msgr2=0x7f1924108010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:42:15.965 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:15.961+0000 7f192ae00640 1 -- 172.21.15.83:0/2452139578 shutdown_connections 2024-08-19T01:42:15.965 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:15.961+0000 7f192ae00640 1 -- 172.21.15.83:0/2452139578 wait complete. 2024-08-19T01:42:15.965 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:42:15 smithi167 ceph-mon[21875]: pgmap v561: 97 pgs: 97 active+clean; 359 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 2.0 MiB/s wr, 2.13k op/s 2024-08-19T01:42:15.965 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:42:15 smithi167 ceph-mon[21875]: scrub summary: idle 2024-08-19T01:42:16.904 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:42:16 smithi083 ceph-mon[17120]: scrub summary: idle 2024-08-19T01:42:16.904 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:42:16 smithi083 ceph-mon[17120]: scrub summary: idle 2024-08-19T01:42:16.952 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:42:16 smithi117 ceph-mon[22016]: scrub summary: idle 2024-08-19T01:42:16.952 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:42:16 smithi117 ceph-mon[22016]: scrub summary: idle 2024-08-19T01:42:16.965 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:42:16 smithi167 ceph-mon[21875]: scrub summary: idle 2024-08-19T01:42:16.965 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:42:16 smithi167 ceph-mon[21875]: scrub summary: idle 2024-08-19T01:42:17.904 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:42:17 smithi083 ceph-mon[17120]: pgmap v562: 97 pgs: 97 active+clean; 350 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 1.4 MiB/s wr, 2.00k op/s 2024-08-19T01:42:17.952 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:42:17 smithi117 ceph-mon[22016]: pgmap v562: 97 pgs: 97 active+clean; 350 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 1.4 MiB/s wr, 2.00k op/s 2024-08-19T01:42:17.965 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:42:17 smithi167 ceph-mon[21875]: pgmap v562: 97 pgs: 97 active+clean; 350 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 1.4 MiB/s wr, 2.00k op/s 2024-08-19T01:42:19.904 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:42:19 smithi083 ceph-mon[17120]: pgmap v563: 97 pgs: 97 active+clean; 312 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 1.7 MiB/s wr, 1.94k op/s 2024-08-19T01:42:19.952 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:42:19 smithi117 ceph-mon[22016]: pgmap v563: 97 pgs: 97 active+clean; 312 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 1.7 MiB/s wr, 1.94k op/s 2024-08-19T01:42:19.965 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:42:19 smithi167 ceph-mon[21875]: pgmap v563: 97 pgs: 97 active+clean; 312 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 1.7 MiB/s wr, 1.94k op/s 2024-08-19T01:42:21.904 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:42:21 smithi083 ceph-mon[17120]: pgmap v564: 97 pgs: 97 active+clean; 320 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 2.1 MiB/s wr, 1.41k op/s 2024-08-19T01:42:21.952 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:42:21 smithi117 ceph-mon[22016]: pgmap v564: 97 pgs: 97 active+clean; 320 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 2.1 MiB/s wr, 1.41k op/s 2024-08-19T01:42:21.965 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:42:21 smithi167 ceph-mon[21875]: pgmap v564: 97 pgs: 97 active+clean; 320 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 2.1 MiB/s wr, 1.41k op/s 2024-08-19T01:42:23.904 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:42:23 smithi083 ceph-mon[17120]: pgmap v565: 97 pgs: 97 active+clean; 319 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 2.1 MiB/s wr, 1.17k op/s 2024-08-19T01:42:23.952 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:42:23 smithi117 ceph-mon[22016]: pgmap v565: 97 pgs: 97 active+clean; 319 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 2.1 MiB/s wr, 1.17k op/s 2024-08-19T01:42:23.965 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:42:23 smithi167 ceph-mon[21875]: pgmap v565: 97 pgs: 97 active+clean; 319 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 2.1 MiB/s wr, 1.17k op/s 2024-08-19T01:42:25.952 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:42:25 smithi117 ceph-mon[22016]: pgmap v566: 97 pgs: 97 active+clean; 325 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 3.0 MiB/s wr, 912 op/s 2024-08-19T01:42:25.966 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:42:25 smithi167 ceph-mon[21875]: pgmap v566: 97 pgs: 97 active+clean; 325 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 3.0 MiB/s wr, 912 op/s 2024-08-19T01:42:26.154 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:42:25 smithi083 ceph-mon[17120]: pgmap v566: 97 pgs: 97 active+clean; 325 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 3.0 MiB/s wr, 912 op/s 2024-08-19T01:42:27.952 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:42:27 smithi117 ceph-mon[22016]: pgmap v567: 97 pgs: 97 active+clean; 320 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 2.3 MiB/s wr, 301 op/s 2024-08-19T01:42:27.966 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:42:27 smithi167 ceph-mon[21875]: pgmap v567: 97 pgs: 97 active+clean; 320 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 2.3 MiB/s wr, 301 op/s 2024-08-19T01:42:28.154 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:42:27 smithi083 ceph-mon[17120]: pgmap v567: 97 pgs: 97 active+clean; 320 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 2.3 MiB/s wr, 301 op/s 2024-08-19T01:42:29.100 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:29.095+0000 7f8560400640 1 -- 172.21.15.83:0/2587112176 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8558103a90 msgr2=0x7f8558103e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:42:29.100 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:29.095+0000 7f8560400640 1 --2- 172.21.15.83:0/2587112176 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8558103a90 0x7f8558103e90 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f854c009ca0 tx=0x7f854c02a8c0 comp rx=0 tx=0).stop 2024-08-19T01:42:29.100 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:29.096+0000 7f8560400640 1 -- 172.21.15.83:0/2587112176 shutdown_connections 2024-08-19T01:42:29.100 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:29.096+0000 7f8560400640 1 --2- 172.21.15.83:0/2587112176 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8558105640 0x7f855810bed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:29.100 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:29.096+0000 7f8560400640 1 --2- 172.21.15.83:0/2587112176 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8558104c80 0x7f8558105100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:29.100 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:29.096+0000 7f8560400640 1 --2- 172.21.15.83:0/2587112176 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8558103a90 0x7f8558103e90 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:29.101 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:29.096+0000 7f8560400640 1 -- 172.21.15.83:0/2587112176 >> 172.21.15.83:0/2587112176 conn(0x7f85580ff210 msgr2=0x7f8558101630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:42:29.101 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:29.096+0000 7f8560400640 1 -- 172.21.15.83:0/2587112176 shutdown_connections 2024-08-19T01:42:29.101 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:29.096+0000 7f8560400640 1 -- 172.21.15.83:0/2587112176 wait complete. 2024-08-19T01:42:29.101 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:29.097+0000 7f8560400640 1 Processor -- start 2024-08-19T01:42:29.101 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:29.097+0000 7f8560400640 1 -- start start 2024-08-19T01:42:29.101 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:29.097+0000 7f8560400640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8558103a90 0x7f8558199d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:42:29.101 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:29.097+0000 7f8560400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8558104c80 0x7f855819a270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:42:29.102 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:29.097+0000 7f8560400640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8558105640 0x7f85581a12a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:42:29.102 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:29.097+0000 7f8560400640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f855810f740 con 0x7f8558104c80 2024-08-19T01:42:29.102 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:29.097+0000 7f8560400640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f855810f5c0 con 0x7f8558105640 2024-08-19T01:42:29.102 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:29.097+0000 7f8560400640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f855810f8c0 con 0x7f8558103a90 2024-08-19T01:42:29.102 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:29.097+0000 7f855d000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8558104c80 0x7f855819a270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:42:29.102 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:29.097+0000 7f855e400640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8558105640 0x7f85581a12a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:42:29.103 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:29.097+0000 7f855da00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8558103a90 0x7f8558199d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:42:29.103 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:29.097+0000 7f855d000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8558104c80 0x7f855819a270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:36848/0 (socket says 172.21.15.83:36848) 2024-08-19T01:42:29.103 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:29.097+0000 7f855d000640 1 -- 172.21.15.83:0/1600133633 learned_addr learned my addr 172.21.15.83:0/1600133633 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-08-19T01:42:29.103 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:29.098+0000 7f855e400640 1 -- 172.21.15.83:0/1600133633 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8558103a90 msgr2=0x7f8558199d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:42:29.104 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:29.098+0000 7f855e400640 1 --2- 172.21.15.83:0/1600133633 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8558103a90 0x7f8558199d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:29.104 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:29.098+0000 7f855e400640 1 -- 172.21.15.83:0/1600133633 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8558104c80 msgr2=0x7f855819a270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:42:29.104 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:29.098+0000 7f855e400640 1 --2- 172.21.15.83:0/1600133633 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8558104c80 0x7f855819a270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:29.104 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:29.098+0000 7f855e400640 1 -- 172.21.15.83:0/1600133633 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f85581a1940 con 0x7f8558105640 2024-08-19T01:42:29.104 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:29.098+0000 7f855da00640 1 --2- 172.21.15.83:0/1600133633 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8558103a90 0x7f8558199d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-19T01:42:29.104 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:29.098+0000 7f855e400640 1 --2- 172.21.15.83:0/1600133633 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8558105640 0x7f85581a12a0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f855000e970 tx=0x7f855000ee30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:42:29.105 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:29.098+0000 7f8556000640 1 -- 172.21.15.83:0/1600133633 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f855000ccc0 con 0x7f8558105640 2024-08-19T01:42:29.105 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:29.098+0000 7f8556000640 1 -- 172.21.15.83:0/1600133633 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f85500035f0 con 0x7f8558105640 2024-08-19T01:42:29.105 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:29.098+0000 7f8560400640 1 -- 172.21.15.83:0/1600133633 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f85581a1c30 con 0x7f8558105640 2024-08-19T01:42:29.105 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:29.098+0000 7f8556000640 1 -- 172.21.15.83:0/1600133633 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f855000ce60 con 0x7f8558105640 2024-08-19T01:42:29.105 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:29.098+0000 7f8560400640 1 -- 172.21.15.83:0/1600133633 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f85581a2100 con 0x7f8558105640 2024-08-19T01:42:29.115 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:29.101+0000 7f85631d3740 1 Processor -- start 2024-08-19T01:42:29.115 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:29.101+0000 7f8556000640 1 -- 172.21.15.83:0/1600133633 <== mon.2 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7f85500037a0 con 0x7f8558105640 2024-08-19T01:42:29.115 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:29.101+0000 7f85631d3740 1 -- start start 2024-08-19T01:42:29.116 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:29.102+0000 7f85631d3740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560919acc470 0x560919acc870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:42:29.116 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:29.102+0000 7f85631d3740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560919accdb0 0x560919ad1870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:42:29.116 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:29.102+0000 7f85631d3740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560919ad1e40 0x560919ad4300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:42:29.116 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:29.102+0000 7f85631d3740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x560919a30890 con 0x560919accdb0 2024-08-19T01:42:29.116 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:29.102+0000 7f85631d3740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x5609199c8d90 con 0x560919ad1e40 2024-08-19T01:42:29.116 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:29.102+0000 7f85631d3740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x560919a2c850 con 0x560919acc470 2024-08-19T01:42:29.117 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:29.102+0000 7f855d000640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560919acc470 0x560919acc870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:42:29.117 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:29.102+0000 7f855d000640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560919acc470 0x560919acc870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.83:34666/0 (socket says 172.21.15.83:34666) 2024-08-19T01:42:29.117 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:29.102+0000 7f855d000640 1 -- 172.21.15.83:0/2298676001 learned_addr learned my addr 172.21.15.83:0/2298676001 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-08-19T01:42:29.117 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:29.102+0000 7f855da00640 1 --2- 172.21.15.83:0/2298676001 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560919accdb0 0x560919ad1870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:42:29.117 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:29.102+0000 7f855d000640 1 --2- 172.21.15.83:0/2298676001 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560919ad1e40 0x560919ad4300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:42:29.118 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:29.102+0000 7f855d000640 1 -- 172.21.15.83:0/2298676001 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560919ad1e40 msgr2=0x560919ad4300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:42:29.118 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:29.102+0000 7f855d000640 1 --2- 172.21.15.83:0/2298676001 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560919ad1e40 0x560919ad4300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:29.118 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:29.102+0000 7f855d000640 1 -- 172.21.15.83:0/2298676001 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560919accdb0 msgr2=0x560919ad1870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:42:29.118 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:29.102+0000 7f855d000640 1 --2- 172.21.15.83:0/2298676001 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560919accdb0 0x560919ad1870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:29.118 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:29.102+0000 7f855d000640 1 -- 172.21.15.83:0/2298676001 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560919744210 con 0x560919acc470 2024-08-19T01:42:29.118 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:29.103+0000 7f855d000640 1 --2- 172.21.15.83:0/2298676001 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560919acc470 0x560919acc870 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f85440097f0 tx=0x7f85440098b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e369d19262d8059b server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:42:29.119 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:29.103+0000 7f854aa00640 1 -- 172.21.15.83:0/2298676001 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8544005020 con 0x560919acc470 2024-08-19T01:42:29.119 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:29.103+0000 7f8556000640 1 --2- 172.21.15.83:0/1600133633 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f85400a2c30 0x7f85400a50f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:42:29.119 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:29.103+0000 7f854aa00640 1 -- 172.21.15.83:0/2298676001 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f85440047a0 con 0x560919acc470 2024-08-19T01:42:29.119 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:29.103+0000 7f854aa00640 1 -- 172.21.15.83:0/2298676001 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f854400f9e0 con 0x560919acc470 2024-08-19T01:42:29.119 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:29.103+0000 7f8556000640 1 -- 172.21.15.83:0/1600133633 <== mon.2 v2:172.21.15.117:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8269+0+0 (secure 0 0 0) 0x7f85500cf370 con 0x7f8558105640 2024-08-19T01:42:29.119 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:29.103+0000 7f855da00640 1 --2- 172.21.15.83:0/1600133633 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f85400a2c30 0x7f85400a50f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:42:29.119 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:29.104+0000 7f85631d3740 1 -- 172.21.15.83:0/2298676001 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560919acc470 msgr2=0x560919acc870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:42:29.119 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:29.104+0000 7f85631d3740 1 --2- 172.21.15.83:0/2298676001 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560919acc470 0x560919acc870 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f85440097f0 tx=0x7f85440098b0 comp rx=0 tx=0).stop 2024-08-19T01:42:29.119 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:29.104+0000 7f85631d3740 1 -- 172.21.15.83:0/2298676001 shutdown_connections 2024-08-19T01:42:29.119 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:29.104+0000 7f85631d3740 1 --2- 172.21.15.83:0/2298676001 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560919ad1e40 0x560919ad4300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:29.119 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:29.104+0000 7f85631d3740 1 --2- 172.21.15.83:0/2298676001 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560919accdb0 0x560919ad1870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:29.119 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:29.104+0000 7f85631d3740 1 --2- 172.21.15.83:0/2298676001 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560919acc470 0x560919acc870 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:29.120 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:29.104+0000 7f85631d3740 1 -- 172.21.15.83:0/2298676001 >> 172.21.15.83:0/2298676001 conn(0x560919ac7ae0 msgr2=0x560919ac7ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:42:29.120 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:29.104+0000 7f85631d3740 1 -- 172.21.15.83:0/2298676001 shutdown_connections 2024-08-19T01:42:29.120 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:29.104+0000 7f855da00640 1 --2- 172.21.15.83:0/1600133633 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f85400a2c30 0x7f85400a50f0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f854c000f80 tx=0x7f854c004a40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:42:29.120 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:29.104+0000 7f85631d3740 1 -- 172.21.15.83:0/2298676001 wait complete. 2024-08-19T01:42:29.120 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:29.104+0000 7f85631d3740 1 -- start start 2024-08-19T01:42:29.120 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:29.105+0000 7f85631d3740 1 Processor -- start 2024-08-19T01:42:29.120 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:29.105+0000 7f85631d3740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560919acc470 0x560919ad67d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:42:29.120 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:29.105+0000 7f85631d3740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560919accdb0 0x560919ad6d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:42:29.120 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:29.105+0000 7f85631d3740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560919ad1e40 0x560919addd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:42:29.120 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:29.105+0000 7f85631d3740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x560919ad4eb0 con 0x560919accdb0 2024-08-19T01:42:29.120 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:29.105+0000 7f85631d3740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x5609199ebf70 con 0x560919ad1e40 2024-08-19T01:42:29.121 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:29.105+0000 7f85631d3740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x560919ad5030 con 0x560919acc470 2024-08-19T01:42:29.121 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:29.105+0000 7f85631d3740 10 client.0 _ll_register_callbacks cb 0x560919a288d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-19T01:42:29.121 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:29.105+0000 7f855da00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560919acc470 0x560919ad67d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:42:29.121 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:29.105+0000 7f855d000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560919accdb0 0x560919ad6d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:42:29.121 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:29.105+0000 7f855e400640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560919ad1e40 0x560919addd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:42:29.121 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:29.105+0000 7f855e400640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560919ad1e40 0x560919addd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.83:42584/0 (socket says 172.21.15.83:42584) 2024-08-19T01:42:29.121 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:29.105+0000 7f855da00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560919acc470 0x560919ad67d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.83:34670/0 (socket says 172.21.15.83:34670) 2024-08-19T01:42:29.121 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:29.105+0000 7f855e400640 1 -- 172.21.15.83:0/2342431270 learned_addr learned my addr 172.21.15.83:0/2342431270 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-08-19T01:42:29.121 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:29.106+0000 7f855e400640 1 -- 172.21.15.83:0/2342431270 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560919acc470 msgr2=0x560919ad67d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:42:29.121 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:29.106+0000 7f855e400640 1 --2- 172.21.15.83:0/2342431270 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560919acc470 0x560919ad67d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:29.121 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:29.106+0000 7f855e400640 1 -- 172.21.15.83:0/2342431270 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560919accdb0 msgr2=0x560919ad6d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:42:29.122 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:29.106+0000 7f855e400640 1 --2- 172.21.15.83:0/2342431270 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560919accdb0 0x560919ad6d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:29.122 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:29.106+0000 7f855e400640 1 -- 172.21.15.83:0/2342431270 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560919810620 con 0x560919ad1e40 2024-08-19T01:42:29.122 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:29.106+0000 7f855e400640 1 --2- 172.21.15.83:0/2342431270 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560919ad1e40 0x560919addd50 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f8550098490 tx=0x7f8550099d80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=82de7a462ce37c04 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:42:29.122 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:29.106+0000 7f853f400640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-08-19T01:42:29.122 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:29.106+0000 7f853f400640 1 -- 172.21.15.83:0/2342431270 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85500b3070 con 0x560919ad1e40 2024-08-19T01:42:29.122 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:29.106+0000 7f853f400640 1 -- 172.21.15.83:0/2342431270 <== mon.2 v2:172.21.15.117:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8269+0+0 (secure 0 0 0) 0x7f855009b1a0 con 0x560919ad1e40 2024-08-19T01:42:29.122 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:29.106+0000 7f85631d3740 1 -- 172.21.15.83:0/2342431270 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f85440047a0 con 0x560919ad1e40 2024-08-19T01:42:29.122 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:29.106+0000 7f853f400640 1 -- 172.21.15.83:0/2342431270 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f855009ff00 con 0x560919ad1e40 2024-08-19T01:42:29.122 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:29.106+0000 7f85631d3740 10 client.26102 fetch_fsmap learned FSMap version 28 2024-08-19T01:42:29.122 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:29.106+0000 7f85631d3740 1 -- 172.21.15.83:0/2342431270 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x560919ae0f90 con 0x560919ad1e40 2024-08-19T01:42:29.122 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:29.107+0000 7f853f400640 1 -- 172.21.15.83:0/2342431270 <== mon.2 v2:172.21.15.117:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f85440047a0 con 0x560919ad1e40 2024-08-19T01:42:29.123 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:29.107+0000 7f853f400640 1 -- 172.21.15.83:0/2342431270 <== mon.2 v2:172.21.15.117:3300/0 5 ==== fsmap(e 28) ==== 8358+0+0 (secure 0 0 0) 0x7f85500978a0 con 0x560919ad1e40 2024-08-19T01:42:29.123 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:29.107+0000 7f85631d3740 10 client.26102 fetch_fsmap finished waiting for FSMap version 28 2024-08-19T01:42:29.123 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:29.107+0000 7f85631d3740 10 client.26102 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-08-19T01:42:29.123 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:29.107+0000 7f85631d3740 20 client.26102 populate_metadata read hostname 'smithi083' 2024-08-19T01:42:29.123 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:29.107+0000 7f85631d3740 1 --2- 172.21.15.83:0/2342431270 >> [v2:172.21.15.117:6838/1967595704,v1:172.21.15.117:6839/1967595704] conn(0x560919ae2680 0x560919ae4a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:42:29.123 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:29.107+0000 7f85631d3740 4 client.26102 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-19T01:42:29.123 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:29.107+0000 7f85631d3740 1 -- 172.21.15.83:0/2342431270 --> [v2:172.21.15.117:6838/1967595704,v1:172.21.15.117:6839/1967595704] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560919a3dfc0 con 0x560919ae2680 2024-08-19T01:42:29.123 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:29.107+0000 7f855da00640 1 --2- 172.21.15.83:0/2342431270 >> [v2:172.21.15.117:6838/1967595704,v1:172.21.15.117:6839/1967595704] conn(0x560919ae2680 0x560919ae4a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:42:29.123 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:29.108+0000 7f855da00640 1 --2- 172.21.15.83:0/2342431270 >> [v2:172.21.15.117:6838/1967595704,v1:172.21.15.117:6839/1967595704] conn(0x560919ae2680 0x560919ae4a70 crc :-1 s=READY pgs=133 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cc65becfa610b8a1 server_cookie=2c3224ceef554178 in_seq=0 out_seq=0 2024-08-19T01:42:29.123 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:29.108+0000 7f853f400640 10 client.26102 ms_handle_connect on v2:172.21.15.117:6838/1967595704 2024-08-19T01:42:29.123 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:29.109+0000 7f853f400640 1 -- 172.21.15.83:0/2342431270 <== mds.0 v2:172.21.15.117:6838/1967595704 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x560919a3dfc0 con 0x560919ae2680 2024-08-19T01:42:29.123 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:29.109+0000 7f853f400640 10 client.26102 handle_command_reply: tid=0 2024-08-19T01:42:29.123 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:29.109+0000 7f85631d3740 1 client.26102 shutdown 2024-08-19T01:42:29.124 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:29.110+0000 7f85631d3740 1 -- 172.21.15.83:0/2342431270 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560919ad1e40 msgr2=0x560919addd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:42:29.124 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:29.110+0000 7f85631d3740 1 --2- 172.21.15.83:0/2342431270 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560919ad1e40 0x560919addd50 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f8550098490 tx=0x7f8550099d80 comp rx=0 tx=0).stop 2024-08-19T01:42:29.124 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:29.110+0000 7f85631d3740 1 -- 172.21.15.83:0/2342431270 shutdown_connections 2024-08-19T01:42:29.124 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:29.110+0000 7f85631d3740 1 --2- 172.21.15.83:0/2342431270 >> [v2:172.21.15.117:6838/1967595704,v1:172.21.15.117:6839/1967595704] conn(0x560919ae2680 0x560919ae4a70 crc :-1 s=READY pgs=133 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:29.124 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:29.110+0000 7f85631d3740 1 --2- 172.21.15.83:0/2342431270 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560919ad1e40 0x560919addd50 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:29.124 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:29.110+0000 7f85631d3740 1 --2- 172.21.15.83:0/2342431270 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560919accdb0 0x560919ad6d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:29.124 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:29.110+0000 7f85631d3740 1 --2- 172.21.15.83:0/2342431270 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560919acc470 0x560919ad67d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:29.124 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:29.110+0000 7f853f400640 0 client.26102 ms_handle_reset on v2:172.21.15.117:6838/1967595704 2024-08-19T01:42:29.124 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:29.110+0000 7f85631d3740 1 -- 172.21.15.83:0/2342431270 >> 172.21.15.83:0/2342431270 conn(0x560919ac7ae0 msgr2=0x560919ac92f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:42:29.124 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:29.110+0000 7f85631d3740 1 -- 172.21.15.83:0/2342431270 shutdown_connections 2024-08-19T01:42:29.124 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:29.110+0000 7f85631d3740 1 -- 172.21.15.83:0/2342431270 wait complete. 2024-08-19T01:42:29.124 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:29.110+0000 7f85631d3740 20 client.26102 trim_cache size 0 max 16384 2024-08-19T01:42:29.134 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:29.126+0000 7f85631d3740 1 Processor -- start 2024-08-19T01:42:29.134 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:29.126+0000 7f85631d3740 1 -- start start 2024-08-19T01:42:29.135 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:29.127+0000 7f85631d3740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560919acc470 0x560919ac7ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:42:29.135 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:29.127+0000 7f85631d3740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560919accdb0 0x560919addd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:42:29.135 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:29.127+0000 7f85631d3740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560919ad1e40 0x560919ac8400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:42:29.135 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:29.127+0000 7f85631d3740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f85581ad7a0 con 0x560919accdb0 2024-08-19T01:42:29.135 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:29.127+0000 7f85631d3740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f85581ab040 con 0x560919ad1e40 2024-08-19T01:42:29.135 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:29.127+0000 7f85631d3740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f85581a8e90 con 0x560919acc470 2024-08-19T01:42:29.136 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:29.127+0000 7f855e400640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560919ad1e40 0x560919ac8400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:42:29.136 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:29.127+0000 7f855da00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560919acc470 0x560919ac7ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:42:29.136 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:29.127+0000 7f855e400640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560919ad1e40 0x560919ac8400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.83:42590/0 (socket says 172.21.15.83:42590) 2024-08-19T01:42:29.136 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:29.127+0000 7f855e400640 1 -- 172.21.15.83:0/3337770483 learned_addr learned my addr 172.21.15.83:0/3337770483 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-08-19T01:42:29.136 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:29.127+0000 7f855d000640 1 --2- 172.21.15.83:0/3337770483 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560919accdb0 0x560919addd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:42:29.136 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:29.127+0000 7f855e400640 1 -- 172.21.15.83:0/3337770483 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560919acc470 msgr2=0x560919ac7ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:42:29.137 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:29.127+0000 7f855e400640 1 --2- 172.21.15.83:0/3337770483 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560919acc470 0x560919ac7ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:29.137 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:29.127+0000 7f855e400640 1 -- 172.21.15.83:0/3337770483 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560919accdb0 msgr2=0x560919addd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:42:29.137 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:29.127+0000 7f855e400640 1 --2- 172.21.15.83:0/3337770483 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560919accdb0 0x560919addd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:29.137 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:29.127+0000 7f855e400640 1 -- 172.21.15.83:0/3337770483 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560919ac7900 con 0x560919ad1e40 2024-08-19T01:42:29.137 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:29.127+0000 7f855da00640 1 --2- 172.21.15.83:0/3337770483 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560919acc470 0x560919ac7ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-19T01:42:29.137 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:29.128+0000 7f855e400640 1 --2- 172.21.15.83:0/3337770483 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560919ad1e40 0x560919ac8400 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f85500a00c0 tx=0x7f855009bcd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4b1b90a950ff9f7d server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:42:29.138 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:29.128+0000 7f854a000640 1 -- 172.21.15.83:0/3337770483 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8550081030 con 0x560919ad1e40 2024-08-19T01:42:29.138 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:29.128+0000 7f854a000640 1 -- 172.21.15.83:0/3337770483 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f855009c250 con 0x560919ad1e40 2024-08-19T01:42:29.138 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:29.128+0000 7f854a000640 1 -- 172.21.15.83:0/3337770483 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85500b3030 con 0x560919ad1e40 2024-08-19T01:42:29.138 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:29.128+0000 7f85631d3740 1 -- 172.21.15.83:0/3337770483 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560919ad1e40 msgr2=0x560919ac8400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:42:29.138 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:29.128+0000 7f85631d3740 1 --2- 172.21.15.83:0/3337770483 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560919ad1e40 0x560919ac8400 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f85500a00c0 tx=0x7f855009bcd0 comp rx=0 tx=0).stop 2024-08-19T01:42:29.138 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:29.128+0000 7f85631d3740 1 -- 172.21.15.83:0/3337770483 shutdown_connections 2024-08-19T01:42:29.139 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:29.128+0000 7f85631d3740 1 --2- 172.21.15.83:0/3337770483 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560919ad1e40 0x560919ac8400 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:29.139 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:29.128+0000 7f85631d3740 1 --2- 172.21.15.83:0/3337770483 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560919accdb0 0x560919addd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:29.139 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:29.128+0000 7f85631d3740 1 --2- 172.21.15.83:0/3337770483 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560919acc470 0x560919ac7ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:29.139 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:29.128+0000 7f85631d3740 1 -- 172.21.15.83:0/3337770483 >> 172.21.15.83:0/3337770483 conn(0x560919ac7ae0 msgr2=0x560919ad67d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:42:29.139 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:29.128+0000 7f85631d3740 1 -- 172.21.15.83:0/3337770483 shutdown_connections 2024-08-19T01:42:29.139 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:29.129+0000 7f85631d3740 1 -- 172.21.15.83:0/3337770483 wait complete. 2024-08-19T01:42:29.140 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:29.129+0000 7f85631d3740 1 -- start start 2024-08-19T01:42:29.140 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:29.129+0000 7f85631d3740 1 Processor -- start 2024-08-19T01:42:29.140 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:29.129+0000 7f85631d3740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560919acc470 0x560919adaf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:42:29.140 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:29.129+0000 7f85631d3740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560919accdb0 0x560919adb470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:42:29.140 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:29.129+0000 7f85631d3740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560919ad1e40 0x560919adc480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:42:29.140 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:29.129+0000 7f85631d3740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f85581a9500 con 0x560919accdb0 2024-08-19T01:42:29.140 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:29.129+0000 7f85631d3740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f85581aa970 con 0x560919acc470 2024-08-19T01:42:29.141 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:29.129+0000 7f85631d3740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f85581a9bd0 con 0x560919ad1e40 2024-08-19T01:42:29.141 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:29.129+0000 7f85631d3740 10 client.0 _ll_register_callbacks cb 0x7f8550097d60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-19T01:42:29.141 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:29.130+0000 7f855da00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560919acc470 0x560919adaf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:42:29.141 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:29.130+0000 7f855da00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560919acc470 0x560919adaf30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.83:42598/0 (socket says 172.21.15.83:42598) 2024-08-19T01:42:29.141 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:29.130+0000 7f855da00640 1 -- 172.21.15.83:0/1522729187 learned_addr learned my addr 172.21.15.83:0/1522729187 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-08-19T01:42:29.141 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:29.130+0000 7f855e400640 1 --2- 172.21.15.83:0/1522729187 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560919ad1e40 0x560919adc480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:42:29.141 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:29.130+0000 7f855d000640 1 --2- 172.21.15.83:0/1522729187 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560919accdb0 0x560919adb470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:42:29.142 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:29.130+0000 7f855da00640 1 -- 172.21.15.83:0/1522729187 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560919ad1e40 msgr2=0x560919adc480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:42:29.142 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:29.130+0000 7f855da00640 1 --2- 172.21.15.83:0/1522729187 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560919ad1e40 0x560919adc480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:29.142 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:29.130+0000 7f855da00640 1 -- 172.21.15.83:0/1522729187 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560919accdb0 msgr2=0x560919adb470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:42:29.142 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:29.130+0000 7f855da00640 1 --2- 172.21.15.83:0/1522729187 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560919accdb0 0x560919adb470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:29.142 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:29.130+0000 7f855da00640 1 -- 172.21.15.83:0/1522729187 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560919adca90 con 0x560919acc470 2024-08-19T01:42:29.142 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:29.130+0000 7f855d000640 1 --2- 172.21.15.83:0/1522729187 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560919accdb0 0x560919adb470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-19T01:42:29.143 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:29.130+0000 7f855da00640 1 --2- 172.21.15.83:0/1522729187 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560919acc470 0x560919adaf30 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f854c031660 tx=0x7f854c02a8c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3e3825f437a7d94c server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:42:29.143 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:29.130+0000 7f853f400640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-08-19T01:42:29.143 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:29.130+0000 7f853f400640 1 -- 172.21.15.83:0/1522729187 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f854c02f610 con 0x560919acc470 2024-08-19T01:42:29.143 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:29.130+0000 7f85631d3740 1 -- 172.21.15.83:0/1522729187 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f855009c250 con 0x560919acc470 2024-08-19T01:42:29.143 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:29.130+0000 7f853f400640 1 -- 172.21.15.83:0/1522729187 <== mon.2 v2:172.21.15.117:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8269+0+0 (secure 0 0 0) 0x7f854c02f7b0 con 0x560919acc470 2024-08-19T01:42:29.143 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:29.131+0000 7f853f400640 1 -- 172.21.15.83:0/1522729187 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f854c02f9b0 con 0x560919acc470 2024-08-19T01:42:29.144 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:29.131+0000 7f85631d3740 10 client.26108 fetch_fsmap learned FSMap version 28 2024-08-19T01:42:29.144 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:29.131+0000 7f85631d3740 1 -- 172.21.15.83:0/1522729187 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x560919adde10 con 0x560919acc470 2024-08-19T01:42:29.144 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:29.131+0000 7f853f400640 1 -- 172.21.15.83:0/1522729187 <== mon.2 v2:172.21.15.117:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f855009c250 con 0x560919acc470 2024-08-19T01:42:29.144 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:29.131+0000 7f853f400640 1 -- 172.21.15.83:0/1522729187 <== mon.2 v2:172.21.15.117:3300/0 5 ==== fsmap(e 28) ==== 8358+0+0 (secure 0 0 0) 0x7f854c0305c0 con 0x560919acc470 2024-08-19T01:42:29.144 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:29.131+0000 7f85631d3740 10 client.26108 fetch_fsmap finished waiting for FSMap version 28 2024-08-19T01:42:29.145 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:29.131+0000 7f85631d3740 10 client.26108 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-08-19T01:42:29.145 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:29.131+0000 7f85631d3740 20 client.26108 populate_metadata read hostname 'smithi083' 2024-08-19T01:42:29.145 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:29.131+0000 7f85631d3740 1 --2- 172.21.15.83:0/1522729187 >> [v2:172.21.15.117:6838/1967595704,v1:172.21.15.117:6839/1967595704] conn(0x560919b4d950 0x560919b4fd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:42:29.145 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:29.131+0000 7f85631d3740 4 client.26108 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-08-19T01:42:29.145 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:29.131+0000 7f85631d3740 1 -- 172.21.15.83:0/1522729187 --> [v2:172.21.15.117:6838/1967595704,v1:172.21.15.117:6839/1967595704] -- command(tid 0: {"prefix": "flush journal"}) -- 0x560919a38310 con 0x560919b4d950 2024-08-19T01:42:29.145 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:29.131+0000 7f855d000640 1 --2- 172.21.15.83:0/1522729187 >> [v2:172.21.15.117:6838/1967595704,v1:172.21.15.117:6839/1967595704] conn(0x560919b4d950 0x560919b4fd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:42:29.145 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:29.132+0000 7f855d000640 1 --2- 172.21.15.83:0/1522729187 >> [v2:172.21.15.117:6838/1967595704,v1:172.21.15.117:6839/1967595704] conn(0x560919b4d950 0x560919b4fd70 crc :-1 s=READY pgs=134 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=75020d78df7e903e server_cookie=1951fb4a454a1ea9 in_seq=0 out_seq=0 2024-08-19T01:42:29.146 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:29.132+0000 7f853f400640 10 client.26108 ms_handle_connect on v2:172.21.15.117:6838/1967595704 2024-08-19T01:42:29.952 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:42:29 smithi117 ceph-mon[22016]: pgmap v568: 97 pgs: 97 active+clean; 306 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 3.0 MiB/s wr, 309 op/s 2024-08-19T01:42:29.965 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:42:29 smithi167 ceph-mon[21875]: pgmap v568: 97 pgs: 97 active+clean; 306 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 3.0 MiB/s wr, 309 op/s 2024-08-19T01:42:29.975 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:29.974+0000 7f853f400640 1 -- 172.21.15.83:0/1522729187 <== mds.0 v2:172.21.15.117:6838/1967595704 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x560919a38310 con 0x560919b4d950 2024-08-19T01:42:29.975 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:29.974+0000 7f853f400640 10 client.26108 handle_command_reply: tid=0 2024-08-19T01:42:29.975 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:29.974+0000 7f85631d3740 1 client.26108 shutdown 2024-08-19T01:42:29.975 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:29.975+0000 7f85631d3740 1 -- 172.21.15.83:0/1522729187 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560919acc470 msgr2=0x560919adaf30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:42:29.975 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:29.975+0000 7f85631d3740 1 --2- 172.21.15.83:0/1522729187 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560919acc470 0x560919adaf30 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f854c031660 tx=0x7f854c02a8c0 comp rx=0 tx=0).stop 2024-08-19T01:42:29.975 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:29.975+0000 7f85631d3740 1 -- 172.21.15.83:0/1522729187 shutdown_connections 2024-08-19T01:42:29.976 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:29.975+0000 7f85631d3740 1 --2- 172.21.15.83:0/1522729187 >> [v2:172.21.15.117:6838/1967595704,v1:172.21.15.117:6839/1967595704] conn(0x560919b4d950 0x560919b4fd70 crc :-1 s=READY pgs=134 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:29.976 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:29.975+0000 7f85631d3740 1 --2- 172.21.15.83:0/1522729187 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560919ad1e40 0x560919adc480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:29.976 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:29.975+0000 7f85631d3740 1 --2- 172.21.15.83:0/1522729187 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560919accdb0 0x560919adb470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:29.976 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:29.975+0000 7f85631d3740 1 --2- 172.21.15.83:0/1522729187 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560919acc470 0x560919adaf30 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:29.976 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:29.975+0000 7f853f400640 0 client.26108 ms_handle_reset on v2:172.21.15.117:6838/1967595704 2024-08-19T01:42:29.977 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:29.975+0000 7f85631d3740 1 -- 172.21.15.83:0/1522729187 >> 172.21.15.83:0/1522729187 conn(0x560919ac7ae0 msgr2=0x560919ac64d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:42:29.977 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:29.975+0000 7f85631d3740 1 -- 172.21.15.83:0/1522729187 shutdown_connections 2024-08-19T01:42:29.977 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:29.976+0000 7f85631d3740 1 -- 172.21.15.83:0/1522729187 wait complete. 2024-08-19T01:42:29.977 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:29.976+0000 7f85631d3740 20 client.26108 trim_cache size 0 max 16384 2024-08-19T01:42:29.977 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stdout:{ 2024-08-19T01:42:29.977 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stdout: "message": "", 2024-08-19T01:42:29.977 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stdout: "return_code": 0 2024-08-19T01:42:29.977 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stdout:} 2024-08-19T01:42:29.978 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:29.977+0000 7f8560400640 1 -- 172.21.15.83:0/1600133633 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f85400a2c30 msgr2=0x7f85400a50f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:42:29.978 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:29.977+0000 7f8560400640 1 --2- 172.21.15.83:0/1600133633 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f85400a2c30 0x7f85400a50f0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f854c000f80 tx=0x7f854c004a40 comp rx=0 tx=0).stop 2024-08-19T01:42:29.978 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:29.977+0000 7f8560400640 1 -- 172.21.15.83:0/1600133633 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8558105640 msgr2=0x7f85581a12a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:42:29.978 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:29.977+0000 7f8560400640 1 --2- 172.21.15.83:0/1600133633 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8558105640 0x7f85581a12a0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f855000e970 tx=0x7f855000ee30 comp rx=0 tx=0).stop 2024-08-19T01:42:29.978 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:29.978+0000 7f8560400640 1 -- 172.21.15.83:0/1600133633 shutdown_connections 2024-08-19T01:42:29.978 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:29.978+0000 7f8560400640 1 --2- 172.21.15.83:0/1600133633 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f85400a2c30 0x7f85400a50f0 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:29.979 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:29.978+0000 7f8560400640 1 --2- 172.21.15.83:0/1600133633 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8558105640 0x7f85581a12a0 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:29.979 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:29.978+0000 7f8560400640 1 --2- 172.21.15.83:0/1600133633 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8558104c80 0x7f855819a270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:29.979 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:29.978+0000 7f8560400640 1 --2- 172.21.15.83:0/1600133633 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8558103a90 0x7f8558199d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:29.979 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:29.978+0000 7f8560400640 1 -- 172.21.15.83:0/1600133633 >> 172.21.15.83:0/1600133633 conn(0x7f85580ff210 msgr2=0x7f8558107f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:42:29.979 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:29.978+0000 7f8560400640 1 -- 172.21.15.83:0/1600133633 shutdown_connections 2024-08-19T01:42:29.979 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:29.978+0000 7f8560400640 1 -- 172.21.15.83:0/1600133633 wait complete. 2024-08-19T01:42:30.154 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:42:29 smithi083 ceph-mon[17120]: pgmap v568: 97 pgs: 97 active+clean; 306 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 3.0 MiB/s wr, 309 op/s 2024-08-19T01:42:30.952 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:42:30 smithi117 ceph-mon[22016]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-08-19T01:42:30.965 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:42:30 smithi167 ceph-mon[21875]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-08-19T01:42:31.154 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:42:30 smithi083 ceph-mon[17120]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-08-19T01:42:31.952 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:42:31 smithi117 ceph-mon[22016]: pgmap v569: 97 pgs: 97 active+clean; 295 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 2.7 MiB/s wr, 271 op/s 2024-08-19T01:42:31.965 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:42:31 smithi167 ceph-mon[21875]: pgmap v569: 97 pgs: 97 active+clean; 295 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 2.7 MiB/s wr, 271 op/s 2024-08-19T01:42:32.154 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:42:31 smithi083 ceph-mon[17120]: pgmap v569: 97 pgs: 97 active+clean; 295 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 2.7 MiB/s wr, 271 op/s 2024-08-19T01:42:33.952 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:42:33 smithi117 ceph-mon[22016]: pgmap v570: 97 pgs: 97 active+clean; 286 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.7 MiB/s wr, 302 op/s 2024-08-19T01:42:33.965 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:42:33 smithi167 ceph-mon[21875]: pgmap v570: 97 pgs: 97 active+clean; 286 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.7 MiB/s wr, 302 op/s 2024-08-19T01:42:34.154 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:42:33 smithi083 ceph-mon[17120]: pgmap v570: 97 pgs: 97 active+clean; 286 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.7 MiB/s wr, 302 op/s 2024-08-19T01:42:34.852 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833 shell --fsid bad8780e-5dc9-11ef-bccf-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-08-19T01:42:35.280 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/bad8780e-5dc9-11ef-bccf-c7b262605968/mon.a/config 2024-08-19T01:42:35.932 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:35.929+0000 7fe688a00640 1 -- 172.21.15.83:0/1084186987 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe6841004b0 msgr2=0x7fe684100890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:42:35.932 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:35.929+0000 7fe688a00640 1 --2- 172.21.15.83:0/1084186987 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe6841004b0 0x7fe684100890 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7fe674009860 tx=0x7fe67402a340 comp rx=0 tx=0).stop 2024-08-19T01:42:35.932 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:35.929+0000 7fe688a00640 1 -- 172.21.15.83:0/1084186987 shutdown_connections 2024-08-19T01:42:35.932 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:35.929+0000 7fe688a00640 1 --2- 172.21.15.83:0/1084186987 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe684106680 0x7fe684114860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:35.932 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:35.929+0000 7fe688a00640 1 --2- 172.21.15.83:0/1084186987 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe684100e60 0x7fe684112310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:35.932 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:35.929+0000 7fe688a00640 1 --2- 172.21.15.83:0/1084186987 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe6841004b0 0x7fe684100890 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:35.933 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:35.929+0000 7fe688a00640 1 -- 172.21.15.83:0/1084186987 >> 172.21.15.83:0/1084186987 conn(0x7fe6840fc340 msgr2=0x7fe6840fe760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:42:35.933 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:35.930+0000 7fe688a00640 1 -- 172.21.15.83:0/1084186987 shutdown_connections 2024-08-19T01:42:35.933 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:35.930+0000 7fe688a00640 1 -- 172.21.15.83:0/1084186987 wait complete. 2024-08-19T01:42:35.933 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:35.931+0000 7fe688a00640 1 Processor -- start 2024-08-19T01:42:35.933 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:35.931+0000 7fe688a00640 1 -- start start 2024-08-19T01:42:35.934 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:42:35 smithi083 ceph-mon[17120]: pgmap v571: 97 pgs: 97 active+clean; 251 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 2.8 MiB/s wr, 550 op/s 2024-08-19T01:42:35.934 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:35.932+0000 7fe688a00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe684100e60 0x7fe684114620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:42:35.934 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:35.932+0000 7fe688a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe684106680 0x7fe68410d6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:42:35.934 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:35.932+0000 7fe688a00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe68410dbf0 0x7fe68410e0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:42:35.935 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:35.932+0000 7fe683400640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe684100e60 0x7fe684114620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:42:35.935 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:35.932+0000 7fe683400640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe684100e60 0x7fe684114620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.83:55604/0 (socket says 172.21.15.83:55604) 2024-08-19T01:42:35.935 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:35.933+0000 7fe688a00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fe684118110 con 0x7fe684106680 2024-08-19T01:42:35.935 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:35.933+0000 7fe688a00640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fe684117f90 con 0x7fe68410dbf0 2024-08-19T01:42:35.935 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:35.933+0000 7fe688a00640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fe684118290 con 0x7fe684100e60 2024-08-19T01:42:35.935 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:35.933+0000 7fe683400640 1 -- 172.21.15.83:0/735642103 learned_addr learned my addr 172.21.15.83:0/735642103 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-08-19T01:42:35.935 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:35.933+0000 7fe682a00640 1 --2- 172.21.15.83:0/735642103 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe684106680 0x7fe68410d6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:42:35.935 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:35.934+0000 7fe682a00640 1 -- 172.21.15.83:0/735642103 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe684100e60 msgr2=0x7fe684114620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:42:35.935 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:35.934+0000 7fe682a00640 1 --2- 172.21.15.83:0/735642103 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe684100e60 0x7fe684114620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:35.935 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:35.934+0000 7fe682a00640 1 -- 172.21.15.83:0/735642103 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe68410dbf0 msgr2=0x7fe68410e0a0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-08-19T01:42:35.936 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:35.934+0000 7fe682a00640 1 --2- 172.21.15.83:0/735642103 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe68410dbf0 0x7fe68410e0a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:35.936 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:35.934+0000 7fe682a00640 1 -- 172.21.15.83:0/735642103 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe684112340 con 0x7fe684106680 2024-08-19T01:42:35.936 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:35.934+0000 7fe682a00640 1 --2- 172.21.15.83:0/735642103 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe684106680 0x7fe68410d6b0 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7fe67000e9f0 tx=0x7fe67000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:42:35.936 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:35.935+0000 7fe67be00640 1 -- 172.21.15.83:0/735642103 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe67000ccc0 con 0x7fe684106680 2024-08-19T01:42:35.937 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:35.935+0000 7fe67be00640 1 -- 172.21.15.83:0/735642103 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe670010910 con 0x7fe684106680 2024-08-19T01:42:35.937 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:35.936+0000 7fe688a00640 1 -- 172.21.15.83:0/735642103 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe6841125d0 con 0x7fe684106680 2024-08-19T01:42:35.937 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:35.936+0000 7fe67be00640 1 -- 172.21.15.83:0/735642103 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe670018980 con 0x7fe684106680 2024-08-19T01:42:35.938 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:35.936+0000 7fe688a00640 1 -- 172.21.15.83:0/735642103 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe684112b60 con 0x7fe684106680 2024-08-19T01:42:35.942 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:35.940+0000 7fe68ca5c740 1 Processor -- start 2024-08-19T01:42:35.942 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:35.941+0000 7fe68ca5c740 1 -- start start 2024-08-19T01:42:35.944 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:35.943+0000 7fe68ca5c740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562c49f50d80 0x562c49f51160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:42:35.945 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:35.943+0000 7fe68ca5c740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562c49f516a0 0x562c49f5ba20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:42:35.945 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:35.943+0000 7fe68ca5c740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562c49f5bf60 0x562c49f5e350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:42:35.945 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:35.943+0000 7fe68ca5c740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x562c49eb6bf0 con 0x562c49f516a0 2024-08-19T01:42:35.945 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:35.943+0000 7fe68ca5c740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x562c49e4ebd0 con 0x562c49f50d80 2024-08-19T01:42:35.945 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:35.943+0000 7fe68ca5c740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x562c49eb2bb0 con 0x562c49f5bf60 2024-08-19T01:42:35.945 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:35.943+0000 7fe683e00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562c49f50d80 0x562c49f51160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:42:35.945 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:35.944+0000 7fe67be00640 1 -- 172.21.15.83:0/735642103 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7fe670018b20 con 0x7fe684106680 2024-08-19T01:42:35.946 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:35.944+0000 7fe683e00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562c49f50d80 0x562c49f51160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.83:54930/0 (socket says 172.21.15.83:54930) 2024-08-19T01:42:35.946 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:35.944+0000 7fe683e00640 1 -- 172.21.15.83:0/1369385222 learned_addr learned my addr 172.21.15.83:0/1369385222 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-08-19T01:42:35.946 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:35.944+0000 7fe682a00640 1 --2- 172.21.15.83:0/1369385222 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562c49f5bf60 0x562c49f5e350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:42:35.946 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:35.944+0000 7fe683400640 1 --2- 172.21.15.83:0/1369385222 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562c49f516a0 0x562c49f5ba20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:42:35.947 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:35.946+0000 7fe67be00640 1 --2- 172.21.15.83:0/735642103 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7fe6680a2cc0 0x7fe6680a5180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:42:35.948 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:35.946+0000 7fe67be00640 1 -- 172.21.15.83:0/735642103 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8269+0+0 (secure 0 0 0) 0x7fe6700cf350 con 0x7fe684106680 2024-08-19T01:42:35.949 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:35.947+0000 7fe683400640 1 -- 172.21.15.83:0/1369385222 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562c49f5bf60 msgr2=0x562c49f5e350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:42:35.949 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:35.947+0000 7fe683400640 1 --2- 172.21.15.83:0/1369385222 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562c49f5bf60 0x562c49f5e350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:35.949 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:35.947+0000 7fe683400640 1 -- 172.21.15.83:0/1369385222 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562c49f50d80 msgr2=0x562c49f51160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:42:35.949 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:35.947+0000 7fe683400640 1 --2- 172.21.15.83:0/1369385222 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562c49f50d80 0x562c49f51160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:35.949 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:35.947+0000 7fe683400640 1 -- 172.21.15.83:0/1369385222 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562c49e558a0 con 0x562c49f516a0 2024-08-19T01:42:35.949 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:35.947+0000 7fe682a00640 1 --2- 172.21.15.83:0/1369385222 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562c49f5bf60 0x562c49f5e350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-19T01:42:35.949 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:35.948+0000 7fe683e00640 1 --2- 172.21.15.83:0/735642103 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7fe6680a2cc0 0x7fe6680a5180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:42:35.950 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:35.948+0000 7fe683400640 1 --2- 172.21.15.83:0/1369385222 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562c49f516a0 0x562c49f5ba20 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7fe674009830 tx=0x7fe674030340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=41c16d53fcdce054 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:42:35.950 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:35.949+0000 7fe66fe00640 1 -- 172.21.15.83:0/1369385222 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe674042070 con 0x562c49f516a0 2024-08-19T01:42:35.950 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:35.949+0000 7fe66fe00640 1 -- 172.21.15.83:0/1369385222 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe674030550 con 0x562c49f516a0 2024-08-19T01:42:35.951 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:35.949+0000 7fe66fe00640 1 -- 172.21.15.83:0/1369385222 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe674037040 con 0x562c49f516a0 2024-08-19T01:42:35.951 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:35.950+0000 7fe68ca5c740 1 -- 172.21.15.83:0/1369385222 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562c49f516a0 msgr2=0x562c49f5ba20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:42:35.951 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:35.950+0000 7fe68ca5c740 1 --2- 172.21.15.83:0/1369385222 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562c49f516a0 0x562c49f5ba20 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7fe674009830 tx=0x7fe674030340 comp rx=0 tx=0).stop 2024-08-19T01:42:35.952 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:35.950+0000 7fe683e00640 1 --2- 172.21.15.83:0/735642103 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7fe6680a2cc0 0x7fe6680a5180 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7fe67c009fd0 tx=0x7fe67c004a40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:42:35.952 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:42:35 smithi117 ceph-mon[22016]: pgmap v571: 97 pgs: 97 active+clean; 251 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 2.8 MiB/s wr, 550 op/s 2024-08-19T01:42:35.952 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:35.950+0000 7fe68ca5c740 1 -- 172.21.15.83:0/1369385222 shutdown_connections 2024-08-19T01:42:35.952 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:35.950+0000 7fe68ca5c740 1 --2- 172.21.15.83:0/1369385222 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562c49f5bf60 0x562c49f5e350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:35.953 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:35.950+0000 7fe68ca5c740 1 --2- 172.21.15.83:0/1369385222 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562c49f516a0 0x562c49f5ba20 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:35.953 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:35.950+0000 7fe68ca5c740 1 --2- 172.21.15.83:0/1369385222 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562c49f50d80 0x562c49f51160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:35.953 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:35.950+0000 7fe68ca5c740 1 -- 172.21.15.83:0/1369385222 >> 172.21.15.83:0/1369385222 conn(0x562c49f4d310 msgr2=0x562c49f4d6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:42:35.953 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:35.950+0000 7fe68ca5c740 1 -- 172.21.15.83:0/1369385222 shutdown_connections 2024-08-19T01:42:35.953 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:35.951+0000 7fe68ca5c740 1 -- 172.21.15.83:0/1369385222 wait complete. 2024-08-19T01:42:35.954 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:35.951+0000 7fe68ca5c740 1 -- start start 2024-08-19T01:42:35.954 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:35.951+0000 7fe68ca5c740 1 Processor -- start 2024-08-19T01:42:35.954 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:35.952+0000 7fe68ca5c740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562c49f50d80 0x562c49f58d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:42:35.954 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:35.952+0000 7fe68ca5c740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562c49f516a0 0x562c49f592c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:42:35.954 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:35.952+0000 7fe68ca5c740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562c49f5bf60 0x562c49f59800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:42:35.954 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:35.952+0000 7fe68ca5c740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x562c49f5eb30 con 0x562c49f5bf60 2024-08-19T01:42:35.954 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:35.952+0000 7fe68ca5c740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x562c49e71d40 con 0x562c49f50d80 2024-08-19T01:42:35.955 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:35.952+0000 7fe68ca5c740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x562c49f5ecb0 con 0x562c49f516a0 2024-08-19T01:42:35.955 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:35.952+0000 7fe68ca5c740 10 client.0 _ll_register_callbacks cb 0x562c49e3ab10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-19T01:42:35.955 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:35.953+0000 7fe683e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562c49f5bf60 0x562c49f59800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:42:35.955 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:35.953+0000 7fe683400640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562c49f50d80 0x562c49f58d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:42:35.956 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:35.953+0000 7fe683400640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562c49f50d80 0x562c49f58d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.83:54944/0 (socket says 172.21.15.83:54944) 2024-08-19T01:42:35.956 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:35.953+0000 7fe683400640 1 -- 172.21.15.83:0/2791626940 learned_addr learned my addr 172.21.15.83:0/2791626940 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-08-19T01:42:35.956 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:35.954+0000 7fe682a00640 1 --2- 172.21.15.83:0/2791626940 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562c49f516a0 0x562c49f592c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:42:35.956 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:35.954+0000 7fe683e00640 1 -- 172.21.15.83:0/2791626940 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562c49f516a0 msgr2=0x562c49f592c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:42:35.957 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:35.955+0000 7fe683e00640 1 --2- 172.21.15.83:0/2791626940 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562c49f516a0 0x562c49f592c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:35.957 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:35.955+0000 7fe683e00640 1 -- 172.21.15.83:0/2791626940 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562c49f50d80 msgr2=0x562c49f58d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:42:35.957 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:35.955+0000 7fe683e00640 1 --2- 172.21.15.83:0/2791626940 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562c49f50d80 0x562c49f58d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:35.957 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:35.955+0000 7fe683e00640 1 -- 172.21.15.83:0/2791626940 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562c49c283c0 con 0x562c49f5bf60 2024-08-19T01:42:35.957 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:35.955+0000 7fe683400640 1 --2- 172.21.15.83:0/2791626940 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562c49f50d80 0x562c49f58d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-19T01:42:35.958 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:35.955+0000 7fe682a00640 1 --2- 172.21.15.83:0/2791626940 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562c49f516a0 0x562c49f592c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-19T01:42:35.958 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:35.956+0000 7fe683e00640 1 --2- 172.21.15.83:0/2791626940 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562c49f5bf60 0x562c49f59800 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7fe67c009580 tx=0x7fe67c004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=139eb0025ff07e00 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:42:35.958 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:35.956+0000 7fe66cc00640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-08-19T01:42:35.959 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:35.957+0000 7fe66cc00640 1 -- 172.21.15.83:0/2791626940 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe67c017040 con 0x562c49f5bf60 2024-08-19T01:42:35.959 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:35.957+0000 7fe66cc00640 1 -- 172.21.15.83:0/2791626940 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8269+0+0 (secure 0 0 0) 0x7fe67c004b20 con 0x562c49f5bf60 2024-08-19T01:42:35.959 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:35.957+0000 7fe68ca5c740 1 -- 172.21.15.83:0/2791626940 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe674030550 con 0x562c49f5bf60 2024-08-19T01:42:35.960 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:35.958+0000 7fe66cc00640 1 -- 172.21.15.83:0/2791626940 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fe67c004d20 con 0x562c49f5bf60 2024-08-19T01:42:35.960 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:35.959+0000 7fe68ca5c740 10 client.16071 fetch_fsmap learned FSMap version 29 2024-08-19T01:42:35.960 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:35.959+0000 7fe68ca5c740 1 -- 172.21.15.83:0/2791626940 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x562c49f6a7d0 con 0x562c49f5bf60 2024-08-19T01:42:35.961 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:35.959+0000 7fe66cc00640 1 -- 172.21.15.83:0/2791626940 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe674030550 con 0x562c49f5bf60 2024-08-19T01:42:35.961 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:35.960+0000 7fe66cc00640 1 -- 172.21.15.83:0/2791626940 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7fe67c00b040 con 0x562c49f5bf60 2024-08-19T01:42:35.961 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:35.960+0000 7fe68ca5c740 10 client.16071 fetch_fsmap finished waiting for FSMap version 29 2024-08-19T01:42:35.961 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:35.960+0000 7fe68ca5c740 10 client.16071 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-19T01:42:35.961 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:35.960+0000 7fe68ca5c740 20 client.16071 populate_metadata read hostname 'smithi083' 2024-08-19T01:42:35.962 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:35.960+0000 7fe68ca5c740 1 --2- 172.21.15.83:0/2791626940 >> [v2:172.21.15.117:6838/1967595704,v1:172.21.15.117:6839/1967595704] conn(0x562c49f6bd30 0x562c49f6e1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:42:35.962 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:35.960+0000 7fe68ca5c740 4 client.16071 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-19T01:42:35.962 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:35.960+0000 7fe68ca5c740 1 -- 172.21.15.83:0/2791626940 --> [v2:172.21.15.117:6838/1967595704,v1:172.21.15.117:6839/1967595704] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562c49ec4160 con 0x562c49f6bd30 2024-08-19T01:42:35.963 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:35.962+0000 7fe683400640 1 --2- 172.21.15.83:0/2791626940 >> [v2:172.21.15.117:6838/1967595704,v1:172.21.15.117:6839/1967595704] conn(0x562c49f6bd30 0x562c49f6e1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:42:35.965 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:35.963+0000 7fe683400640 1 --2- 172.21.15.83:0/2791626940 >> [v2:172.21.15.117:6838/1967595704,v1:172.21.15.117:6839/1967595704] conn(0x562c49f6bd30 0x562c49f6e1b0 crc :-1 s=READY pgs=135 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5ca27c8d6cb5cf05 server_cookie=e113386ae570874a in_seq=0 out_seq=0 2024-08-19T01:42:35.965 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:35.963+0000 7fe66cc00640 10 client.16071 ms_handle_connect on v2:172.21.15.117:6838/1967595704 2024-08-19T01:42:35.965 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:42:35 smithi167 ceph-mon[21875]: pgmap v571: 97 pgs: 97 active+clean; 251 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 2.8 MiB/s wr, 550 op/s 2024-08-19T01:42:35.970 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:35.968+0000 7fe66cc00640 1 -- 172.21.15.83:0/2791626940 <== mds.0 v2:172.21.15.117:6838/1967595704 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x562c49ec4160 con 0x562c49f6bd30 2024-08-19T01:42:35.970 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:35.968+0000 7fe66cc00640 10 client.16071 handle_command_reply: tid=0 2024-08-19T01:42:35.970 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:35.968+0000 7fe68ca5c740 1 client.16071 shutdown 2024-08-19T01:42:35.970 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:35.969+0000 7fe68ca5c740 1 -- 172.21.15.83:0/2791626940 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562c49f5bf60 msgr2=0x562c49f59800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:42:35.970 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:35.969+0000 7fe68ca5c740 1 --2- 172.21.15.83:0/2791626940 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562c49f5bf60 0x562c49f59800 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7fe67c009580 tx=0x7fe67c004ab0 comp rx=0 tx=0).stop 2024-08-19T01:42:35.971 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:35.969+0000 7fe68ca5c740 1 -- 172.21.15.83:0/2791626940 shutdown_connections 2024-08-19T01:42:35.971 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:35.969+0000 7fe68ca5c740 1 --2- 172.21.15.83:0/2791626940 >> [v2:172.21.15.117:6838/1967595704,v1:172.21.15.117:6839/1967595704] conn(0x562c49f6bd30 0x562c49f6e1b0 crc :-1 s=READY pgs=135 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:35.971 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:35.969+0000 7fe68ca5c740 1 --2- 172.21.15.83:0/2791626940 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562c49f5bf60 0x562c49f59800 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:35.971 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:35.969+0000 7fe68ca5c740 1 --2- 172.21.15.83:0/2791626940 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562c49f516a0 0x562c49f592c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:35.971 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:35.969+0000 7fe68ca5c740 1 --2- 172.21.15.83:0/2791626940 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562c49f50d80 0x562c49f58d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:35.971 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:35.969+0000 7fe68ca5c740 1 -- 172.21.15.83:0/2791626940 >> 172.21.15.83:0/2791626940 conn(0x562c49f4d310 msgr2=0x562c49f52ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:42:35.972 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:35.970+0000 7fe66cc00640 0 client.16071 ms_handle_reset on v2:172.21.15.117:6838/1967595704 2024-08-19T01:42:35.972 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:35.970+0000 7fe68ca5c740 1 -- 172.21.15.83:0/2791626940 shutdown_connections 2024-08-19T01:42:35.972 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:35.970+0000 7fe68ca5c740 1 -- 172.21.15.83:0/2791626940 wait complete. 2024-08-19T01:42:35.972 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:35.970+0000 7fe68ca5c740 20 client.16071 trim_cache size 0 max 16384 2024-08-19T01:42:36.003 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:36.001+0000 7fe68ca5c740 1 Processor -- start 2024-08-19T01:42:36.003 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:36.001+0000 7fe68ca5c740 1 -- start start 2024-08-19T01:42:36.003 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:36.002+0000 7fe68ca5c740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562c49f50d80 0x562c49f57c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:42:36.003 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:36.002+0000 7fe68ca5c740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562c49f516a0 0x562c49f73ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:42:36.003 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:36.002+0000 7fe68ca5c740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562c49f5bf60 0x562c49f74000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:42:36.004 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:36.002+0000 7fe68ca5c740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fe664010e60 con 0x562c49f516a0 2024-08-19T01:42:36.004 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:36.002+0000 7fe68ca5c740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fe66400e6d0 con 0x562c49f5bf60 2024-08-19T01:42:36.004 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:36.002+0000 7fe68ca5c740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fe66400c4e0 con 0x562c49f50d80 2024-08-19T01:42:36.004 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:36.002+0000 7fe683400640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562c49f50d80 0x562c49f57c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:42:36.004 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:36.002+0000 7fe683400640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562c49f50d80 0x562c49f57c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.83:55624/0 (socket says 172.21.15.83:55624) 2024-08-19T01:42:36.004 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:36.002+0000 7fe683400640 1 -- 172.21.15.83:0/4109001141 learned_addr learned my addr 172.21.15.83:0/4109001141 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-08-19T01:42:36.004 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:36.002+0000 7fe683e00640 1 --2- 172.21.15.83:0/4109001141 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562c49f5bf60 0x562c49f74000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:42:36.005 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:36.003+0000 7fe682a00640 1 --2- 172.21.15.83:0/4109001141 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562c49f516a0 0x562c49f73ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:42:36.006 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:36.004+0000 7fe683400640 1 -- 172.21.15.83:0/4109001141 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562c49f5bf60 msgr2=0x562c49f74000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:42:36.006 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:36.004+0000 7fe683400640 1 --2- 172.21.15.83:0/4109001141 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562c49f5bf60 0x562c49f74000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:36.006 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:36.004+0000 7fe683400640 1 -- 172.21.15.83:0/4109001141 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562c49f516a0 msgr2=0x562c49f73ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:42:36.006 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:36.004+0000 7fe683400640 1 --2- 172.21.15.83:0/4109001141 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562c49f516a0 0x562c49f73ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:36.007 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:36.005+0000 7fe683400640 1 -- 172.21.15.83:0/4109001141 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562c49f4d140 con 0x562c49f50d80 2024-08-19T01:42:36.007 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:36.005+0000 7fe682a00640 1 --2- 172.21.15.83:0/4109001141 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562c49f516a0 0x562c49f73ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-19T01:42:36.007 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:36.005+0000 7fe683400640 1 --2- 172.21.15.83:0/4109001141 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562c49f50d80 0x562c49f57c30 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fe67402a930 tx=0x7fe67402a340 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6ea9044b7a0f9a14 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:42:36.007 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:36.006+0000 7fe66f400640 1 -- 172.21.15.83:0/4109001141 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe674042070 con 0x562c49f50d80 2024-08-19T01:42:36.009 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:36.006+0000 7fe66f400640 1 -- 172.21.15.83:0/4109001141 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe674043040 con 0x562c49f50d80 2024-08-19T01:42:36.009 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:36.006+0000 7fe66f400640 1 -- 172.21.15.83:0/4109001141 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe674037040 con 0x562c49f50d80 2024-08-19T01:42:36.009 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:36.007+0000 7fe68ca5c740 1 -- 172.21.15.83:0/4109001141 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562c49f50d80 msgr2=0x562c49f57c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:42:36.009 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:36.007+0000 7fe68ca5c740 1 --2- 172.21.15.83:0/4109001141 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562c49f50d80 0x562c49f57c30 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fe67402a930 tx=0x7fe67402a340 comp rx=0 tx=0).stop 2024-08-19T01:42:36.010 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:36.008+0000 7fe68ca5c740 1 -- 172.21.15.83:0/4109001141 shutdown_connections 2024-08-19T01:42:36.010 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:36.008+0000 7fe68ca5c740 1 --2- 172.21.15.83:0/4109001141 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562c49f5bf60 0x562c49f74000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:36.010 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:36.008+0000 7fe68ca5c740 1 --2- 172.21.15.83:0/4109001141 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562c49f516a0 0x562c49f73ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:36.010 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:36.008+0000 7fe68ca5c740 1 --2- 172.21.15.83:0/4109001141 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562c49f50d80 0x562c49f57c30 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:36.010 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:36.008+0000 7fe68ca5c740 1 -- 172.21.15.83:0/4109001141 >> 172.21.15.83:0/4109001141 conn(0x562c49f4d310 msgr2=0x562c49f57520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:42:36.010 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:36.008+0000 7fe68ca5c740 1 -- 172.21.15.83:0/4109001141 shutdown_connections 2024-08-19T01:42:36.010 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:36.008+0000 7fe68ca5c740 1 -- 172.21.15.83:0/4109001141 wait complete. 2024-08-19T01:42:36.011 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:36.009+0000 7fe68ca5c740 1 -- start start 2024-08-19T01:42:36.011 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:36.009+0000 7fe68ca5c740 1 Processor -- start 2024-08-19T01:42:36.011 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:36.010+0000 7fe68ca5c740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562c49f50d80 0x562c49fd3a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:42:36.011 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:36.010+0000 7fe68ca5c740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562c49f516a0 0x562c49fd3fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:42:36.012 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:36.010+0000 7fe68ca5c740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562c49f5bf60 0x562c49f63f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:42:36.012 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:36.010+0000 7fe68ca5c740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fe66400cb90 con 0x562c49f516a0 2024-08-19T01:42:36.012 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:36.010+0000 7fe68ca5c740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fe66400e000 con 0x562c49f5bf60 2024-08-19T01:42:36.012 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:36.010+0000 7fe68ca5c740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fe66400d260 con 0x562c49f50d80 2024-08-19T01:42:36.012 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:36.010+0000 7fe68ca5c740 10 client.0 _ll_register_callbacks cb 0x7fe67c00e4f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-19T01:42:36.013 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:36.011+0000 7fe683e00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562c49f5bf60 0x562c49f63f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:42:36.013 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:36.011+0000 7fe682a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562c49f516a0 0x562c49fd3fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:42:36.013 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:36.011+0000 7fe683400640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562c49f50d80 0x562c49fd3a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:42:36.013 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:36.011+0000 7fe683e00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562c49f5bf60 0x562c49f63f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.83:54972/0 (socket says 172.21.15.83:54972) 2024-08-19T01:42:36.013 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:36.011+0000 7fe683e00640 1 -- 172.21.15.83:0/2147959484 learned_addr learned my addr 172.21.15.83:0/2147959484 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-08-19T01:42:36.014 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:36.012+0000 7fe682a00640 1 -- 172.21.15.83:0/2147959484 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562c49f50d80 msgr2=0x562c49fd3a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:42:36.014 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:36.012+0000 7fe682a00640 1 --2- 172.21.15.83:0/2147959484 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562c49f50d80 0x562c49fd3a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:36.014 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:36.012+0000 7fe682a00640 1 -- 172.21.15.83:0/2147959484 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562c49f5bf60 msgr2=0x562c49f63f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:42:36.014 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:36.012+0000 7fe682a00640 1 --2- 172.21.15.83:0/2147959484 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562c49f5bf60 0x562c49f63f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:36.014 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:36.012+0000 7fe682a00640 1 -- 172.21.15.83:0/2147959484 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562c49f64490 con 0x562c49f516a0 2024-08-19T01:42:36.015 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:36.013+0000 7fe682a00640 1 --2- 172.21.15.83:0/2147959484 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562c49f516a0 0x562c49fd3fd0 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7fe6700a4270 tx=0x7fe670010d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f64abbbf6fb29c9 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:42:36.015 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:36.013+0000 7fe66cc00640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-08-19T01:42:36.016 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:36.014+0000 7fe66cc00640 1 -- 172.21.15.83:0/2147959484 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe67009ce60 con 0x562c49f516a0 2024-08-19T01:42:36.016 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:36.014+0000 7fe66cc00640 1 -- 172.21.15.83:0/2147959484 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8269+0+0 (secure 0 0 0) 0x7fe670010ac0 con 0x562c49f516a0 2024-08-19T01:42:36.016 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:36.014+0000 7fe68ca5c740 1 -- 172.21.15.83:0/2147959484 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe674043040 con 0x562c49f516a0 2024-08-19T01:42:36.016 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:36.015+0000 7fe66cc00640 1 -- 172.21.15.83:0/2147959484 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fe67009fee0 con 0x562c49f516a0 2024-08-19T01:42:36.016 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:36.015+0000 7fe68ca5c740 10 client.16077 fetch_fsmap learned FSMap version 29 2024-08-19T01:42:36.016 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:36.015+0000 7fe68ca5c740 1 -- 172.21.15.83:0/2147959484 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x562c49f5d160 con 0x562c49f516a0 2024-08-19T01:42:36.017 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:36.016+0000 7fe66cc00640 1 -- 172.21.15.83:0/2147959484 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe674043040 con 0x562c49f516a0 2024-08-19T01:42:36.017 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:36.016+0000 7fe66cc00640 1 -- 172.21.15.83:0/2147959484 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7fe6700ba290 con 0x562c49f516a0 2024-08-19T01:42:36.017 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:36.016+0000 7fe68ca5c740 10 client.16077 fetch_fsmap finished waiting for FSMap version 29 2024-08-19T01:42:36.018 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:36.016+0000 7fe68ca5c740 10 client.16077 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-19T01:42:36.019 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:36.016+0000 7fe68ca5c740 20 client.16077 populate_metadata read hostname 'smithi083' 2024-08-19T01:42:36.019 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:36.016+0000 7fe68ca5c740 1 --2- 172.21.15.83:0/2147959484 >> [v2:172.21.15.117:6838/1967595704,v1:172.21.15.117:6839/1967595704] conn(0x562c49fe7b60 0x562c49fe9f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:42:36.019 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:36.016+0000 7fe68ca5c740 4 client.16077 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-08-19T01:42:36.019 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:36.016+0000 7fe68ca5c740 1 -- 172.21.15.83:0/2147959484 --> [v2:172.21.15.117:6838/1967595704,v1:172.21.15.117:6839/1967595704] -- command(tid 0: {"prefix": "scrub status"}) -- 0x562c49ebe670 con 0x562c49fe7b60 2024-08-19T01:42:36.019 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:36.016+0000 7fe683400640 1 --2- 172.21.15.83:0/2147959484 >> [v2:172.21.15.117:6838/1967595704,v1:172.21.15.117:6839/1967595704] conn(0x562c49fe7b60 0x562c49fe9f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:42:36.019 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:36.016+0000 7fe683400640 1 --2- 172.21.15.83:0/2147959484 >> [v2:172.21.15.117:6838/1967595704,v1:172.21.15.117:6839/1967595704] conn(0x562c49fe7b60 0x562c49fe9f80 crc :-1 s=READY pgs=136 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=584e71b9aae06ef2 server_cookie=35577aadaf113a50 in_seq=0 out_seq=0 2024-08-19T01:42:36.019 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:36.017+0000 7fe66cc00640 10 client.16077 ms_handle_connect on v2:172.21.15.117:6838/1967595704 2024-08-19T01:42:36.019 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:36.017+0000 7fe66cc00640 1 -- 172.21.15.83:0/2147959484 <== mds.0 v2:172.21.15.117:6838/1967595704 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x562c49ebe670 con 0x562c49fe7b60 2024-08-19T01:42:36.019 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:36.017+0000 7fe66cc00640 10 client.16077 handle_command_reply: tid=0 2024-08-19T01:42:36.019 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:36.017+0000 7fe68ca5c740 1 client.16077 shutdown 2024-08-19T01:42:36.019 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:36.017+0000 7fe68ca5c740 1 -- 172.21.15.83:0/2147959484 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562c49f516a0 msgr2=0x562c49fd3fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:42:36.019 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:36.017+0000 7fe68ca5c740 1 --2- 172.21.15.83:0/2147959484 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562c49f516a0 0x562c49fd3fd0 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7fe6700a4270 tx=0x7fe670010d80 comp rx=0 tx=0).stop 2024-08-19T01:42:36.019 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:36.017+0000 7fe68ca5c740 1 -- 172.21.15.83:0/2147959484 shutdown_connections 2024-08-19T01:42:36.020 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:36.017+0000 7fe68ca5c740 1 --2- 172.21.15.83:0/2147959484 >> [v2:172.21.15.117:6838/1967595704,v1:172.21.15.117:6839/1967595704] conn(0x562c49fe7b60 0x562c49fe9f80 crc :-1 s=READY pgs=136 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:36.020 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:36.017+0000 7fe68ca5c740 1 --2- 172.21.15.83:0/2147959484 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562c49f5bf60 0x562c49f63f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:36.020 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:36.017+0000 7fe68ca5c740 1 --2- 172.21.15.83:0/2147959484 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562c49f516a0 0x562c49fd3fd0 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:36.020 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:36.017+0000 7fe68ca5c740 1 --2- 172.21.15.83:0/2147959484 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562c49f50d80 0x562c49fd3a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:36.020 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:36.017+0000 7fe68ca5c740 1 -- 172.21.15.83:0/2147959484 >> 172.21.15.83:0/2147959484 conn(0x562c49f4d310 msgr2=0x562c49f59560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:42:36.020 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:36.017+0000 7fe66cc00640 0 client.16077 ms_handle_reset on v2:172.21.15.117:6838/1967595704 2024-08-19T01:42:36.020 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:36.018+0000 7fe68ca5c740 1 -- 172.21.15.83:0/2147959484 shutdown_connections 2024-08-19T01:42:36.021 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:36.018+0000 7fe68ca5c740 1 -- 172.21.15.83:0/2147959484 wait complete. 2024-08-19T01:42:36.021 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:36.018+0000 7fe68ca5c740 20 client.16077 trim_cache size 0 max 16384 2024-08-19T01:42:36.021 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-08-19T01:42:36.021 INFO:teuthology.orchestra.run.smithi083.stdout: "status": "no active scrubs running", 2024-08-19T01:42:36.021 INFO:teuthology.orchestra.run.smithi083.stdout: "scrubs": {} 2024-08-19T01:42:36.021 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-08-19T01:42:36.022 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:36.019+0000 7fe688a00640 1 -- 172.21.15.83:0/735642103 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7fe6680a2cc0 msgr2=0x7fe6680a5180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:42:36.022 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:36.019+0000 7fe688a00640 1 --2- 172.21.15.83:0/735642103 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7fe6680a2cc0 0x7fe6680a5180 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7fe67c009fd0 tx=0x7fe67c004a40 comp rx=0 tx=0).stop 2024-08-19T01:42:36.022 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:36.019+0000 7fe688a00640 1 -- 172.21.15.83:0/735642103 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe684106680 msgr2=0x7fe68410d6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:42:36.022 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:36.019+0000 7fe688a00640 1 --2- 172.21.15.83:0/735642103 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe684106680 0x7fe68410d6b0 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7fe67000e9f0 tx=0x7fe67000eeb0 comp rx=0 tx=0).stop 2024-08-19T01:42:36.022 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:36.019+0000 7fe688a00640 1 -- 172.21.15.83:0/735642103 shutdown_connections 2024-08-19T01:42:36.022 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:36.019+0000 7fe688a00640 1 --2- 172.21.15.83:0/735642103 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7fe6680a2cc0 0x7fe6680a5180 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:36.022 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:36.019+0000 7fe688a00640 1 --2- 172.21.15.83:0/735642103 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe68410dbf0 0x7fe68410e0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:36.022 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:36.019+0000 7fe688a00640 1 --2- 172.21.15.83:0/735642103 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe684106680 0x7fe68410d6b0 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:36.022 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:36.019+0000 7fe688a00640 1 --2- 172.21.15.83:0/735642103 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe684100e60 0x7fe684114620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:36.022 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:36.019+0000 7fe688a00640 1 -- 172.21.15.83:0/735642103 >> 172.21.15.83:0/735642103 conn(0x7fe6840fc340 msgr2=0x7fe6840fd740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:42:36.022 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:36.019+0000 7fe688a00640 1 -- 172.21.15.83:0/735642103 shutdown_connections 2024-08-19T01:42:36.022 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:36.019+0000 7fe688a00640 1 -- 172.21.15.83:0/735642103 wait complete. 2024-08-19T01:42:36.917 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-08-19T01:42:36.917 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833 shell --fsid bad8780e-5dc9-11ef-bccf-c7b262605968 -- ceph fs dump --format=json 2024-08-19T01:42:37.312 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/bad8780e-5dc9-11ef-bccf-c7b262605968/mon.a/config 2024-08-19T01:42:37.798 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:42:37 smithi083 ceph-mon[17120]: pgmap v572: 97 pgs: 97 active+clean; 238 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.9 MiB/s wr, 567 op/s 2024-08-19T01:42:37.952 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:42:37 smithi117 ceph-mon[22016]: pgmap v572: 97 pgs: 97 active+clean; 238 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.9 MiB/s wr, 567 op/s 2024-08-19T01:42:37.965 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:42:37 smithi167 ceph-mon[21875]: pgmap v572: 97 pgs: 97 active+clean; 238 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.9 MiB/s wr, 567 op/s 2024-08-19T01:42:37.988 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:37.986+0000 7f659de00640 1 -- 172.21.15.83:0/3803654555 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f659810a140 msgr2=0x7f659810a580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:42:37.989 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:37.986+0000 7f659de00640 1 --2- 172.21.15.83:0/3803654555 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f659810a140 0x7f659810a580 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f6588009900 tx=0x7f658802a490 comp rx=0 tx=0).stop 2024-08-19T01:42:37.989 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:37.987+0000 7f659de00640 1 -- 172.21.15.83:0/3803654555 shutdown_connections 2024-08-19T01:42:37.989 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:37.987+0000 7f659de00640 1 --2- 172.21.15.83:0/3803654555 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f659810aac0 0x7f6598116f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:37.989 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:37.987+0000 7f659de00640 1 --2- 172.21.15.83:0/3803654555 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f659810a140 0x7f659810a580 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:37.989 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:37.987+0000 7f659de00640 1 --2- 172.21.15.83:0/3803654555 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f65981101a0 0x7f6598110580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:37.989 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:37.987+0000 7f659de00640 1 -- 172.21.15.83:0/3803654555 >> 172.21.15.83:0/3803654555 conn(0x7f6598071360 msgr2=0x7f6598073780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:42:37.989 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:37.987+0000 7f659de00640 1 -- 172.21.15.83:0/3803654555 shutdown_connections 2024-08-19T01:42:37.989 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:37.987+0000 7f659de00640 1 -- 172.21.15.83:0/3803654555 wait complete. 2024-08-19T01:42:37.990 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:37.988+0000 7f659de00640 1 Processor -- start 2024-08-19T01:42:37.990 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:37.988+0000 7f659de00640 1 -- start start 2024-08-19T01:42:37.990 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:37.989+0000 7f659de00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f659810a140 0x7f6598117560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:42:37.991 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:37.989+0000 7f659de00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f659810aac0 0x7f6598117aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:42:37.991 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:37.989+0000 7f659de00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f65981101a0 0x7f6598117fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:42:37.991 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:37.989+0000 7f659de00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f659811c6f0 con 0x7f65981101a0 2024-08-19T01:42:37.991 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:37.989+0000 7f659de00640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f659811c570 con 0x7f659810aac0 2024-08-19T01:42:37.991 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:37.989+0000 7f659de00640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f659811c870 con 0x7f659810a140 2024-08-19T01:42:37.991 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:37.989+0000 7f6596a00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f659810a140 0x7f6598117560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:42:37.991 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:37.989+0000 7f6596a00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f659810a140 0x7f6598117560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.83:55658/0 (socket says 172.21.15.83:55658) 2024-08-19T01:42:37.991 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:37.989+0000 7f6596a00640 1 -- 172.21.15.83:0/1083487449 learned_addr learned my addr 172.21.15.83:0/1083487449 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-08-19T01:42:37.991 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:37.989+0000 7f6596000640 1 --2- 172.21.15.83:0/1083487449 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f659810aac0 0x7f6598117aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:42:37.991 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:37.989+0000 7f6597400640 1 --2- 172.21.15.83:0/1083487449 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f65981101a0 0x7f6598117fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:42:37.992 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:37.990+0000 7f6596000640 1 -- 172.21.15.83:0/1083487449 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f659810a140 msgr2=0x7f6598117560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:42:37.992 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:37.990+0000 7f6596000640 1 --2- 172.21.15.83:0/1083487449 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f659810a140 0x7f6598117560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:37.992 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:37.990+0000 7f6596000640 1 -- 172.21.15.83:0/1083487449 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f65981101a0 msgr2=0x7f6598117fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:42:37.992 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:37.990+0000 7f6596000640 1 --2- 172.21.15.83:0/1083487449 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f65981101a0 0x7f6598117fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:37.992 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:37.990+0000 7f6596000640 1 -- 172.21.15.83:0/1083487449 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f65981b3130 con 0x7f659810aac0 2024-08-19T01:42:37.992 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:37.990+0000 7f6596000640 1 --2- 172.21.15.83:0/1083487449 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f659810aac0 0x7f6598117aa0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f6588009900 tx=0x7f6588003b20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:42:37.993 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:37.990+0000 7f659ca00640 1 -- 172.21.15.83:0/1083487449 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f658803f070 con 0x7f659810aac0 2024-08-19T01:42:37.993 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:37.990+0000 7f659de00640 1 -- 172.21.15.83:0/1083487449 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f65981b33c0 con 0x7f659810aac0 2024-08-19T01:42:37.993 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:37.990+0000 7f659ca00640 1 -- 172.21.15.83:0/1083487449 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6588031c90 con 0x7f659810aac0 2024-08-19T01:42:37.993 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:37.990+0000 7f659ca00640 1 -- 172.21.15.83:0/1083487449 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6588035030 con 0x7f659810aac0 2024-08-19T01:42:37.993 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:37.990+0000 7f659de00640 1 -- 172.21.15.83:0/1083487449 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f65981b38f0 con 0x7f659810aac0 2024-08-19T01:42:37.993 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:37.992+0000 7f659de00640 1 -- 172.21.15.83:0/1083487449 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f659810b7e0 con 0x7f659810aac0 2024-08-19T01:42:37.994 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:37.993+0000 7f659ca00640 1 -- 172.21.15.83:0/1083487449 <== mon.2 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7f6588042070 con 0x7f659810aac0 2024-08-19T01:42:37.996 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:37.994+0000 7f659ca00640 1 --2- 172.21.15.83:0/1083487449 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f657c0a2c00 0x7f657c0a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:42:37.996 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:37.995+0000 7f659ca00640 1 -- 172.21.15.83:0/1083487449 <== mon.2 v2:172.21.15.117:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8269+0+0 (secure 0 0 0) 0x7f65880ef8f0 con 0x7f659810aac0 2024-08-19T01:42:37.996 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:37.995+0000 7f6596a00640 1 --2- 172.21.15.83:0/1083487449 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f657c0a2c00 0x7f657c0a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:42:37.997 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:37.995+0000 7f6596a00640 1 --2- 172.21.15.83:0/1083487449 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f657c0a2c00 0x7f657c0a50c0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f6584009c90 tx=0x7f6584005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:42:38.000 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:37.999+0000 7f659ca00640 1 -- 172.21.15.83:0/1083487449 <== mon.2 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7f65880bc7d0 con 0x7f659810aac0 2024-08-19T01:42:38.220 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:38.219+0000 7f659de00640 1 -- 172.21.15.83:0/1083487449 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f65981b3bb0 con 0x7f659810aac0 2024-08-19T01:42:38.221 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:38.219+0000 7f659ca00640 1 -- 172.21.15.83:0/1083487449 <== mon.2 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) ==== 94+0+11925 (secure 0 0 0) 0x7f65880bc0d0 con 0x7f659810aac0 2024-08-19T01:42:38.221 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-08-19T01:42:38.221 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":29,"btime":"2024-08-19T01:42:29:660493+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14421,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6835/988556067","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":988556067},{"type":"v1","addr":"172.21.15.83:6835","nonce":988556067}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14433,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6837/869702632","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6836","nonce":869702632},{"type":"v1","addr":"172.21.15.83:6837","nonce":869702632}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14475,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6841/1795975999","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6840","nonce":1795975999},{"type":"v1","addr":"172.21.15.83:6841","nonce":1795975999}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24430,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6833/2505076046","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6832","nonce":2505076046},{"type":"v1","addr":"172.21.15.167:6833","nonce":2505076046}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24449,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6833/3557520478","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":3557520478},{"type":"v1","addr":"172.21.15.117:6833","nonce":3557520478}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24466,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6835/3482045966","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":3482045966},{"type":"v1","addr":"172.21.15.167:6835","nonce":3482045966}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7}],"filesystems":[{"mdsmap":{"epoch":29,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-08-19T01:32:42.054769+0000","modified":"2024-08-19T01:42:28.830648+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24545,"mds_1":24527,"mds_2":24509},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24478":{"gid":24478,"name":"c","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.167:6837/1762130408","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6836","nonce":1762130408},{"type":"v1","addr":"172.21.15.167:6837","nonce":1762130408}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24485":{"gid":24485,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.117:6835/1024213994","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":1024213994},{"type":"v1","addr":"172.21.15.117:6835","nonce":1024213994}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24496":{"gid":24496,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.167:6839/3507387682","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6838","nonce":3507387682},{"type":"v1","addr":"172.21.15.167:6839","nonce":3507387682}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24509":{"gid":24509,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":10,"addr":"172.21.15.117:6837/3185224684","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":3185224684},{"type":"v1","addr":"172.21.15.117:6837","nonce":3185224684}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24527":{"gid":24527,"name":"g","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.83:6839/3239261845","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6838","nonce":3239261845},{"type":"v1","addr":"172.21.15.83:6839","nonce":3239261845}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24545":{"gid":24545,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.117:6839/1967595704","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6838","nonce":1967595704},{"type":"v1","addr":"172.21.15.117:6839","nonce":1967595704}]},"join_fscid":-1,"export_targets":[1,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24545,"qdb_cluster":[24545,24527,24509]},"id":1}]} 2024-08-19T01:42:38.222 INFO:teuthology.orchestra.run.smithi083.stderr:dumped fsmap epoch 29 2024-08-19T01:42:38.223 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:38.222+0000 7f659de00640 1 -- 172.21.15.83:0/1083487449 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f657c0a2c00 msgr2=0x7f657c0a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:42:38.224 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:38.222+0000 7f659de00640 1 --2- 172.21.15.83:0/1083487449 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f657c0a2c00 0x7f657c0a50c0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f6584009c90 tx=0x7f6584005f90 comp rx=0 tx=0).stop 2024-08-19T01:42:38.224 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:38.222+0000 7f659de00640 1 -- 172.21.15.83:0/1083487449 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f659810aac0 msgr2=0x7f6598117aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:42:38.224 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:38.222+0000 7f659de00640 1 --2- 172.21.15.83:0/1083487449 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f659810aac0 0x7f6598117aa0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f6588009900 tx=0x7f6588003b20 comp rx=0 tx=0).stop 2024-08-19T01:42:38.224 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:38.222+0000 7f659de00640 1 -- 172.21.15.83:0/1083487449 shutdown_connections 2024-08-19T01:42:38.224 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:38.222+0000 7f659de00640 1 --2- 172.21.15.83:0/1083487449 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f657c0a2c00 0x7f657c0a50c0 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:38.224 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:38.222+0000 7f659de00640 1 --2- 172.21.15.83:0/1083487449 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f65981101a0 0x7f6598117fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:38.224 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:38.222+0000 7f659de00640 1 --2- 172.21.15.83:0/1083487449 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f659810aac0 0x7f6598117aa0 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:38.224 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:38.222+0000 7f659de00640 1 --2- 172.21.15.83:0/1083487449 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f659810a140 0x7f6598117560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:38.224 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:38.222+0000 7f659de00640 1 -- 172.21.15.83:0/1083487449 >> 172.21.15.83:0/1083487449 conn(0x7f6598071360 msgr2=0x7f65980729d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:42:38.224 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:38.222+0000 7f659de00640 1 -- 172.21.15.83:0/1083487449 shutdown_connections 2024-08-19T01:42:38.224 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:38.222+0000 7f659de00640 1 -- 172.21.15.83:0/1083487449 wait complete. 2024-08-19T01:42:38.904 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:42:38 smithi083 ceph-mon[17120]: from='client.? 172.21.15.83:0/1083487449' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-19T01:42:38.952 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:42:38 smithi117 ceph-mon[22016]: from='client.? 172.21.15.83:0/1083487449' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-19T01:42:38.965 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:42:38 smithi167 ceph-mon[21875]: from='client.? 172.21.15.83:0/1083487449' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-19T01:42:39.047 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833 shell --fsid bad8780e-5dc9-11ef-bccf-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-08-19T01:42:39.458 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/bad8780e-5dc9-11ef-bccf-c7b262605968/mon.a/config 2024-08-19T01:42:39.904 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:42:39 smithi083 ceph-mon[17120]: pgmap v573: 97 pgs: 97 active+clean; 221 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.3 MiB/s wr, 648 op/s 2024-08-19T01:42:39.952 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:42:39 smithi117 ceph-mon[22016]: pgmap v573: 97 pgs: 97 active+clean; 221 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.3 MiB/s wr, 648 op/s 2024-08-19T01:42:39.965 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:42:39 smithi167 ceph-mon[21875]: pgmap v573: 97 pgs: 97 active+clean; 221 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.3 MiB/s wr, 648 op/s 2024-08-19T01:42:40.104 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:40.102+0000 7fc89c800640 1 -- 172.21.15.83:0/2225755250 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc894109b70 msgr2=0x7fc894109f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:42:40.104 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:40.102+0000 7fc89c800640 1 --2- 172.21.15.83:0/2225755250 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc894109b70 0x7fc894109f50 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fc888009900 tx=0x7fc88802a3f0 comp rx=0 tx=0).stop 2024-08-19T01:42:40.104 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:40.102+0000 7fc89c800640 1 -- 172.21.15.83:0/2225755250 shutdown_connections 2024-08-19T01:42:40.104 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:40.102+0000 7fc89c800640 1 --2- 172.21.15.83:0/2225755250 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc894104480 0x7fc8941109b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:40.104 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:40.102+0000 7fc89c800640 1 --2- 172.21.15.83:0/2225755250 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc894103ae0 0x7fc894103f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:40.104 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:40.102+0000 7fc89c800640 1 --2- 172.21.15.83:0/2225755250 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc894109b70 0x7fc894109f50 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:40.105 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:40.102+0000 7fc89c800640 1 -- 172.21.15.83:0/2225755250 >> 172.21.15.83:0/2225755250 conn(0x7fc8940ff780 msgr2=0x7fc894101ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:42:40.105 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:40.102+0000 7fc89c800640 1 -- 172.21.15.83:0/2225755250 shutdown_connections 2024-08-19T01:42:40.105 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:40.102+0000 7fc89c800640 1 -- 172.21.15.83:0/2225755250 wait complete. 2024-08-19T01:42:40.106 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:40.103+0000 7fc89c800640 1 Processor -- start 2024-08-19T01:42:40.106 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:40.103+0000 7fc89c800640 1 -- start start 2024-08-19T01:42:40.106 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:40.104+0000 7fc89c800640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc894103ae0 0x7fc89419a390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:42:40.106 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:40.104+0000 7fc89c800640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc894104480 0x7fc89419a8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:42:40.106 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:40.104+0000 7fc89c800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc894109b70 0x7fc89419ec60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:42:40.106 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:40.104+0000 7fc89c800640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fc894113f30 con 0x7fc894109b70 2024-08-19T01:42:40.106 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:40.104+0000 7fc89c800640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fc894113db0 con 0x7fc894103ae0 2024-08-19T01:42:40.106 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:40.104+0000 7fc89c800640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fc8941140b0 con 0x7fc894104480 2024-08-19T01:42:40.107 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:40.104+0000 7fc899e00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc894103ae0 0x7fc89419a390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:42:40.107 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:40.104+0000 7fc899e00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc894103ae0 0x7fc89419a390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.83:55012/0 (socket says 172.21.15.83:55012) 2024-08-19T01:42:40.107 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:40.104+0000 7fc899e00640 1 -- 172.21.15.83:0/1077024538 learned_addr learned my addr 172.21.15.83:0/1077024538 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-08-19T01:42:40.107 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:40.104+0000 7fc89a800640 1 --2- 172.21.15.83:0/1077024538 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc894109b70 0x7fc89419ec60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:42:40.107 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:40.104+0000 7fc899400640 1 --2- 172.21.15.83:0/1077024538 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc894104480 0x7fc89419a8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:42:40.107 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:40.104+0000 7fc899e00640 1 -- 172.21.15.83:0/1077024538 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc894104480 msgr2=0x7fc89419a8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:42:40.107 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:40.104+0000 7fc899e00640 1 --2- 172.21.15.83:0/1077024538 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc894104480 0x7fc89419a8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:40.107 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:40.104+0000 7fc899e00640 1 -- 172.21.15.83:0/1077024538 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc894109b70 msgr2=0x7fc89419ec60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:42:40.107 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:40.104+0000 7fc899e00640 1 --2- 172.21.15.83:0/1077024538 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc894109b70 0x7fc89419ec60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:40.107 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:40.104+0000 7fc899e00640 1 -- 172.21.15.83:0/1077024538 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc89419f340 con 0x7fc894103ae0 2024-08-19T01:42:40.107 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:40.105+0000 7fc899e00640 1 --2- 172.21.15.83:0/1077024538 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc894103ae0 0x7fc89419a390 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fc88802a9e0 tx=0x7fc888003c10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:42:40.108 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:40.105+0000 7fc88ea00640 1 -- 172.21.15.83:0/1077024538 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc88803f070 con 0x7fc894103ae0 2024-08-19T01:42:40.108 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:40.105+0000 7fc88ea00640 1 -- 172.21.15.83:0/1077024538 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc888031c30 con 0x7fc894103ae0 2024-08-19T01:42:40.108 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:40.105+0000 7fc89c800640 1 -- 172.21.15.83:0/1077024538 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc89419f5d0 con 0x7fc894103ae0 2024-08-19T01:42:40.108 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:40.105+0000 7fc88ea00640 1 -- 172.21.15.83:0/1077024538 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc888035030 con 0x7fc894103ae0 2024-08-19T01:42:40.108 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:40.105+0000 7fc89c800640 1 -- 172.21.15.83:0/1077024538 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc8941a6f10 con 0x7fc894103ae0 2024-08-19T01:42:40.110 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:40.108+0000 7fc88ea00640 1 -- 172.21.15.83:0/1077024538 <== mon.2 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7fc888042070 con 0x7fc894103ae0 2024-08-19T01:42:40.110 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:40.108+0000 7fc89f2c8740 1 Processor -- start 2024-08-19T01:42:40.110 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:40.108+0000 7fc89f2c8740 1 -- start start 2024-08-19T01:42:40.111 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:40.109+0000 7fc89f2c8740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562530b944f0 0x562530b948d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:42:40.111 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:40.109+0000 7fc89f2c8740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562530b94e10 0x562530b9f190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:42:40.111 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:40.109+0000 7fc89f2c8740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562530b9f6d0 0x562530ba1ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:42:40.111 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:40.109+0000 7fc89f2c8740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x562530af9bf0 con 0x562530b94e10 2024-08-19T01:42:40.111 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:40.109+0000 7fc89f2c8740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x562530a91b20 con 0x562530b944f0 2024-08-19T01:42:40.111 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:40.109+0000 7fc89f2c8740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x562530af5bb0 con 0x562530b9f6d0 2024-08-19T01:42:40.111 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:40.109+0000 7fc89a800640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562530b944f0 0x562530b948d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:42:40.111 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:40.109+0000 7fc89a800640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562530b944f0 0x562530b948d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.83:55018/0 (socket says 172.21.15.83:55018) 2024-08-19T01:42:40.111 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:40.109+0000 7fc89a800640 1 -- 172.21.15.83:0/1621476587 learned_addr learned my addr 172.21.15.83:0/1621476587 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-08-19T01:42:40.111 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:40.109+0000 7fc899e00640 1 --2- 172.21.15.83:0/1621476587 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562530b94e10 0x562530b9f190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:42:40.111 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:40.109+0000 7fc899400640 1 --2- 172.21.15.83:0/1621476587 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562530b9f6d0 0x562530ba1ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:42:40.113 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:40.110+0000 7fc89a800640 1 -- 172.21.15.83:0/1621476587 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562530b9f6d0 msgr2=0x562530ba1ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:42:40.113 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:40.110+0000 7fc89a800640 1 --2- 172.21.15.83:0/1621476587 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562530b9f6d0 0x562530ba1ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:40.113 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:40.110+0000 7fc89a800640 1 -- 172.21.15.83:0/1621476587 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562530b94e10 msgr2=0x562530b9f190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:42:40.113 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:40.110+0000 7fc89a800640 1 --2- 172.21.15.83:0/1621476587 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562530b94e10 0x562530b9f190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:40.113 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:40.110+0000 7fc89a800640 1 -- 172.21.15.83:0/1621476587 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562530a987f0 con 0x562530b944f0 2024-08-19T01:42:40.113 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:40.110+0000 7fc899400640 1 --2- 172.21.15.83:0/1621476587 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562530b9f6d0 0x562530ba1ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-19T01:42:40.113 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:40.110+0000 7fc89a800640 1 --2- 172.21.15.83:0/1621476587 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562530b944f0 0x562530b948d0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fc890009f50 tx=0x7fc8900118d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=207ee0bc51adcef9 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:42:40.113 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:40.110+0000 7fc88ea00640 1 --2- 172.21.15.83:0/1077024538 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7fc87c0a2b80 0x7fc87c0a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:42:40.113 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:40.110+0000 7fc883000640 1 -- 172.21.15.83:0/1621476587 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc890004b20 con 0x562530b944f0 2024-08-19T01:42:40.114 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:40.110+0000 7fc883000640 1 -- 172.21.15.83:0/1621476587 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc890004cc0 con 0x562530b944f0 2024-08-19T01:42:40.114 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:40.110+0000 7fc899e00640 1 --2- 172.21.15.83:0/1077024538 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7fc87c0a2b80 0x7fc87c0a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:42:40.114 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:40.110+0000 7fc883000640 1 -- 172.21.15.83:0/1621476587 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc890005020 con 0x562530b944f0 2024-08-19T01:42:40.114 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:40.110+0000 7fc88ea00640 1 -- 172.21.15.83:0/1077024538 <== mon.2 v2:172.21.15.117:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8269+0+0 (secure 0 0 0) 0x7fc8880ef670 con 0x7fc894103ae0 2024-08-19T01:42:40.114 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:40.111+0000 7fc89f2c8740 1 -- 172.21.15.83:0/1621476587 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562530b944f0 msgr2=0x562530b948d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:42:40.114 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:40.111+0000 7fc89f2c8740 1 --2- 172.21.15.83:0/1621476587 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562530b944f0 0x562530b948d0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fc890009f50 tx=0x7fc8900118d0 comp rx=0 tx=0).stop 2024-08-19T01:42:40.114 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:40.111+0000 7fc899e00640 1 --2- 172.21.15.83:0/1077024538 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7fc87c0a2b80 0x7fc87c0a5040 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7fc8880bbc00 tx=0x7fc88803b750 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:42:40.114 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:40.111+0000 7fc89f2c8740 1 -- 172.21.15.83:0/1621476587 shutdown_connections 2024-08-19T01:42:40.114 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:40.111+0000 7fc89f2c8740 1 --2- 172.21.15.83:0/1621476587 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562530b9f6d0 0x562530ba1ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:40.114 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:40.111+0000 7fc89f2c8740 1 --2- 172.21.15.83:0/1621476587 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562530b94e10 0x562530b9f190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:40.114 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:40.111+0000 7fc89f2c8740 1 --2- 172.21.15.83:0/1621476587 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562530b944f0 0x562530b948d0 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:40.114 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:40.111+0000 7fc89f2c8740 1 -- 172.21.15.83:0/1621476587 >> 172.21.15.83:0/1621476587 conn(0x562530b90a80 msgr2=0x562530b90e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:42:40.115 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:40.111+0000 7fc89f2c8740 1 -- 172.21.15.83:0/1621476587 shutdown_connections 2024-08-19T01:42:40.115 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:40.111+0000 7fc89f2c8740 1 -- 172.21.15.83:0/1621476587 wait complete. 2024-08-19T01:42:40.115 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:40.112+0000 7fc89f2c8740 1 -- start start 2024-08-19T01:42:40.117 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:40.112+0000 7fc89f2c8740 1 Processor -- start 2024-08-19T01:42:40.117 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:40.112+0000 7fc89f2c8740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562530b944f0 0x562530b9c600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:42:40.117 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:40.112+0000 7fc89f2c8740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562530b94e10 0x562530b9cb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:42:40.117 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:40.112+0000 7fc89f2c8740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562530b9f6d0 0x562530b9d080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:42:40.117 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:40.112+0000 7fc89f2c8740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x562530ba22a0 con 0x562530b94e10 2024-08-19T01:42:40.117 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:40.112+0000 7fc89f2c8740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x562530ab4cf0 con 0x562530b944f0 2024-08-19T01:42:40.118 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:40.112+0000 7fc89f2c8740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x562530ba2420 con 0x562530b9f6d0 2024-08-19T01:42:40.118 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:40.112+0000 7fc89f2c8740 10 client.0 _ll_register_callbacks cb 0x562530aa0e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-19T01:42:40.118 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:40.113+0000 7fc89a800640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562530b944f0 0x562530b9c600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:42:40.118 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:40.113+0000 7fc89a800640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562530b944f0 0x562530b9c600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.83:55030/0 (socket says 172.21.15.83:55030) 2024-08-19T01:42:40.118 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:40.113+0000 7fc89a800640 1 -- 172.21.15.83:0/2949275267 learned_addr learned my addr 172.21.15.83:0/2949275267 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-08-19T01:42:40.118 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:40.113+0000 7fc899400640 1 --2- 172.21.15.83:0/2949275267 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562530b94e10 0x562530b9cb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:42:40.118 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:40.113+0000 7fc89a800640 1 --2- 172.21.15.83:0/2949275267 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562530b9f6d0 0x562530b9d080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:42:40.118 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:40.113+0000 7fc89a800640 1 -- 172.21.15.83:0/2949275267 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562530b9f6d0 msgr2=0x562530b9d080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:42:40.118 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:40.113+0000 7fc89a800640 1 --2- 172.21.15.83:0/2949275267 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562530b9f6d0 0x562530b9d080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:40.118 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:40.113+0000 7fc89a800640 1 -- 172.21.15.83:0/2949275267 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562530b94e10 msgr2=0x562530b9cb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:42:40.118 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:40.113+0000 7fc89a800640 1 --2- 172.21.15.83:0/2949275267 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562530b94e10 0x562530b9cb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:40.118 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:40.113+0000 7fc89a800640 1 -- 172.21.15.83:0/2949275267 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56253086b3c0 con 0x562530b944f0 2024-08-19T01:42:40.119 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:40.113+0000 7fc89a800640 1 --2- 172.21.15.83:0/2949275267 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562530b944f0 0x562530b9c600 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fc890004f90 tx=0x7fc890004540 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f2d9e45632d7ce19 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:42:40.119 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:40.113+0000 7fc877400640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-08-19T01:42:40.119 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:40.114+0000 7fc877400640 1 -- 172.21.15.83:0/2949275267 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc89002c020 con 0x562530b944f0 2024-08-19T01:42:40.119 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:40.114+0000 7fc877400640 1 -- 172.21.15.83:0/2949275267 <== mon.2 v2:172.21.15.117:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8269+0+0 (secure 0 0 0) 0x7fc890016040 con 0x562530b944f0 2024-08-19T01:42:40.119 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:40.114+0000 7fc89f2c8740 1 -- 172.21.15.83:0/2949275267 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc890004cc0 con 0x562530b944f0 2024-08-19T01:42:40.119 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:40.114+0000 7fc877400640 1 -- 172.21.15.83:0/2949275267 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fc890037050 con 0x562530b944f0 2024-08-19T01:42:40.119 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:40.114+0000 7fc89f2c8740 10 client.26141 fetch_fsmap learned FSMap version 29 2024-08-19T01:42:40.119 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:40.114+0000 7fc89f2c8740 1 -- 172.21.15.83:0/2949275267 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x562530badfa0 con 0x562530b944f0 2024-08-19T01:42:40.119 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:40.114+0000 7fc877400640 1 -- 172.21.15.83:0/2949275267 <== mon.2 v2:172.21.15.117:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc890004cc0 con 0x562530b944f0 2024-08-19T01:42:40.119 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:40.114+0000 7fc877400640 1 -- 172.21.15.83:0/2949275267 <== mon.2 v2:172.21.15.117:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7fc89000f480 con 0x562530b944f0 2024-08-19T01:42:40.119 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:40.114+0000 7fc89f2c8740 10 client.26141 fetch_fsmap finished waiting for FSMap version 29 2024-08-19T01:42:40.119 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:40.114+0000 7fc89f2c8740 10 client.26141 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-08-19T01:42:40.119 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:40.114+0000 7fc89f2c8740 20 client.26141 populate_metadata read hostname 'smithi083' 2024-08-19T01:42:40.119 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:40.114+0000 7fc89f2c8740 1 --2- 172.21.15.83:0/2949275267 >> [v2:172.21.15.117:6836/3185224684,v1:172.21.15.117:6837/3185224684] conn(0x562530baf400 0x562530bb17f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:42:40.120 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:40.114+0000 7fc89f2c8740 4 client.26141 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-19T01:42:40.120 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:40.115+0000 7fc89f2c8740 1 -- 172.21.15.83:0/2949275267 --> [v2:172.21.15.117:6836/3185224684,v1:172.21.15.117:6837/3185224684] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562530b07160 con 0x562530baf400 2024-08-19T01:42:40.120 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:40.115+0000 7fc899400640 1 --2- 172.21.15.83:0/2949275267 >> [v2:172.21.15.117:6836/3185224684,v1:172.21.15.117:6837/3185224684] conn(0x562530baf400 0x562530bb17f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:42:40.120 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:40.115+0000 7fc877400640 10 client.26141 ms_handle_connect on v2:172.21.15.117:6836/3185224684 2024-08-19T01:42:40.120 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:40.115+0000 7fc899400640 1 --2- 172.21.15.83:0/2949275267 >> [v2:172.21.15.117:6836/3185224684,v1:172.21.15.117:6837/3185224684] conn(0x562530baf400 0x562530bb17f0 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=be459df41ff6968a server_cookie=220f20d0162a925c in_seq=0 out_seq=0 2024-08-19T01:42:40.121 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:40.116+0000 7fc877400640 1 -- 172.21.15.83:0/2949275267 <== mds.2 v2:172.21.15.117:6836/3185224684 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x562530b07160 con 0x562530baf400 2024-08-19T01:42:40.121 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:40.116+0000 7fc877400640 10 client.26141 handle_command_reply: tid=0 2024-08-19T01:42:40.121 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:40.116+0000 7fc89f2c8740 1 client.26141 shutdown 2024-08-19T01:42:40.121 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:40.116+0000 7fc89f2c8740 1 -- 172.21.15.83:0/2949275267 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562530b944f0 msgr2=0x562530b9c600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:42:40.121 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:40.116+0000 7fc89f2c8740 1 --2- 172.21.15.83:0/2949275267 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562530b944f0 0x562530b9c600 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fc890004f90 tx=0x7fc890004540 comp rx=0 tx=0).stop 2024-08-19T01:42:40.121 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:40.117+0000 7fc89f2c8740 1 -- 172.21.15.83:0/2949275267 shutdown_connections 2024-08-19T01:42:40.121 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:40.117+0000 7fc89f2c8740 1 --2- 172.21.15.83:0/2949275267 >> [v2:172.21.15.117:6836/3185224684,v1:172.21.15.117:6837/3185224684] conn(0x562530baf400 0x562530bb17f0 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:40.122 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:40.117+0000 7fc89f2c8740 1 --2- 172.21.15.83:0/2949275267 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562530b9f6d0 0x562530b9d080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:40.122 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:40.117+0000 7fc89f2c8740 1 --2- 172.21.15.83:0/2949275267 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562530b94e10 0x562530b9cb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:40.122 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:40.117+0000 7fc89f2c8740 1 --2- 172.21.15.83:0/2949275267 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562530b944f0 0x562530b9c600 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:40.122 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:40.117+0000 7fc89f2c8740 1 -- 172.21.15.83:0/2949275267 >> 172.21.15.83:0/2949275267 conn(0x562530b90a80 msgr2=0x562530b967b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:42:40.122 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:40.117+0000 7fc877400640 0 client.26141 ms_handle_reset on v2:172.21.15.117:6836/3185224684 2024-08-19T01:42:40.122 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:40.117+0000 7fc89f2c8740 1 -- 172.21.15.83:0/2949275267 shutdown_connections 2024-08-19T01:42:40.122 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:40.117+0000 7fc89f2c8740 1 -- 172.21.15.83:0/2949275267 wait complete. 2024-08-19T01:42:40.122 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:40.117+0000 7fc89f2c8740 20 client.26141 trim_cache size 0 max 16384 2024-08-19T01:42:40.133 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:40.131+0000 7fc89f2c8740 1 Processor -- start 2024-08-19T01:42:40.133 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:40.131+0000 7fc89f2c8740 1 -- start start 2024-08-19T01:42:40.133 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:40.132+0000 7fc89f2c8740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562530b944f0 0x562530b9e570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:42:40.133 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:40.132+0000 7fc89f2c8740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562530b94e10 0x562530bb6fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:42:40.133 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:40.132+0000 7fc89f2c8740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562530b9f6d0 0x562530bb7500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:42:40.133 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:40.132+0000 7fc89f2c8740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fc8941b1ee0 con 0x562530b94e10 2024-08-19T01:42:40.133 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:40.132+0000 7fc89f2c8740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fc8941af750 con 0x562530b944f0 2024-08-19T01:42:40.133 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:40.132+0000 7fc89f2c8740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fc8941ad5a0 con 0x562530b9f6d0 2024-08-19T01:42:40.134 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:40.132+0000 7fc899400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562530b94e10 0x562530bb6fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:42:40.135 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:40.132+0000 7fc89a800640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562530b944f0 0x562530b9e570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:42:40.135 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:40.132+0000 7fc899400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562530b94e10 0x562530bb6fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:47366/0 (socket says 172.21.15.83:47366) 2024-08-19T01:42:40.135 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:40.132+0000 7fc899400640 1 -- 172.21.15.83:0/2306698941 learned_addr learned my addr 172.21.15.83:0/2306698941 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-08-19T01:42:40.135 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:40.132+0000 7fc89a800640 1 --2- 172.21.15.83:0/2306698941 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562530b9f6d0 0x562530bb7500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:42:40.135 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:40.132+0000 7fc899400640 1 -- 172.21.15.83:0/2306698941 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562530b9f6d0 msgr2=0x562530bb7500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:42:40.135 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:40.132+0000 7fc899400640 1 --2- 172.21.15.83:0/2306698941 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562530b9f6d0 0x562530bb7500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:40.135 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:40.132+0000 7fc899400640 1 -- 172.21.15.83:0/2306698941 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562530b944f0 msgr2=0x562530b9e570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:42:40.135 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:40.132+0000 7fc899400640 1 --2- 172.21.15.83:0/2306698941 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562530b944f0 0x562530b9e570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:40.135 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:40.132+0000 7fc899400640 1 -- 172.21.15.83:0/2306698941 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562530b91710 con 0x562530b94e10 2024-08-19T01:42:40.135 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:40.133+0000 7fc89a800640 1 --2- 172.21.15.83:0/2306698941 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562530b9f6d0 0x562530bb7500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-19T01:42:40.135 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:40.133+0000 7fc899400640 1 --2- 172.21.15.83:0/2306698941 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562530b94e10 0x562530bb6fc0 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7fc884004700 tx=0x7fc884005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3a65722c4ac08590 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:42:40.135 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:40.133+0000 7fc882600640 1 -- 172.21.15.83:0/2306698941 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc884003de0 con 0x562530b94e10 2024-08-19T01:42:40.135 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:40.133+0000 7fc882600640 1 -- 172.21.15.83:0/2306698941 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc884010910 con 0x562530b94e10 2024-08-19T01:42:40.135 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:40.133+0000 7fc882600640 1 -- 172.21.15.83:0/2306698941 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc88401f970 con 0x562530b94e10 2024-08-19T01:42:40.136 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:40.134+0000 7fc89f2c8740 1 -- 172.21.15.83:0/2306698941 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562530b94e10 msgr2=0x562530bb6fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:42:40.136 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:40.134+0000 7fc89f2c8740 1 --2- 172.21.15.83:0/2306698941 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562530b94e10 0x562530bb6fc0 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7fc884004700 tx=0x7fc884005f90 comp rx=0 tx=0).stop 2024-08-19T01:42:40.136 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:40.134+0000 7fc89f2c8740 1 -- 172.21.15.83:0/2306698941 shutdown_connections 2024-08-19T01:42:40.137 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:40.134+0000 7fc89f2c8740 1 --2- 172.21.15.83:0/2306698941 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562530b9f6d0 0x562530bb7500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:40.137 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:40.134+0000 7fc89f2c8740 1 --2- 172.21.15.83:0/2306698941 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562530b94e10 0x562530bb6fc0 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:40.137 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:40.134+0000 7fc89f2c8740 1 --2- 172.21.15.83:0/2306698941 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562530b944f0 0x562530b9e570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:40.137 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:40.134+0000 7fc89f2c8740 1 -- 172.21.15.83:0/2306698941 >> 172.21.15.83:0/2306698941 conn(0x562530b90a80 msgr2=0x562530b9de60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:42:40.137 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:40.134+0000 7fc89f2c8740 1 -- 172.21.15.83:0/2306698941 shutdown_connections 2024-08-19T01:42:40.137 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:40.134+0000 7fc89f2c8740 1 -- 172.21.15.83:0/2306698941 wait complete. 2024-08-19T01:42:40.137 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:40.135+0000 7fc89f2c8740 1 -- start start 2024-08-19T01:42:40.137 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:40.135+0000 7fc89f2c8740 1 Processor -- start 2024-08-19T01:42:40.139 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:40.136+0000 7fc89f2c8740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562530b944f0 0x562530ba7670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:42:40.139 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:40.136+0000 7fc89f2c8740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562530b94e10 0x562530ba7bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:42:40.139 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:40.136+0000 7fc89f2c8740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562530b9f6d0 0x562530b988d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:42:40.139 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:40.136+0000 7fc89f2c8740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fc8941adc10 con 0x562530b94e10 2024-08-19T01:42:40.139 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:40.136+0000 7fc89f2c8740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fc8941af080 con 0x562530b944f0 2024-08-19T01:42:40.139 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:40.136+0000 7fc89f2c8740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fc8941ae2e0 con 0x562530b9f6d0 2024-08-19T01:42:40.139 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:40.136+0000 7fc89f2c8740 10 client.0 _ll_register_callbacks cb 0x7fc890004a40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-19T01:42:40.139 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:40.136+0000 7fc89a800640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562530b944f0 0x562530ba7670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:42:40.139 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:40.136+0000 7fc89a800640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562530b944f0 0x562530ba7670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.83:55042/0 (socket says 172.21.15.83:55042) 2024-08-19T01:42:40.139 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:40.136+0000 7fc89a800640 1 -- 172.21.15.83:0/2549421961 learned_addr learned my addr 172.21.15.83:0/2549421961 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-08-19T01:42:40.139 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:40.136+0000 7fc899400640 1 --2- 172.21.15.83:0/2549421961 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562530b94e10 0x562530ba7bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:42:40.139 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:40.136+0000 7fc89a800640 1 --2- 172.21.15.83:0/2549421961 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562530b9f6d0 0x562530b988d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:42:40.140 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:40.136+0000 7fc89a800640 1 -- 172.21.15.83:0/2549421961 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562530b9f6d0 msgr2=0x562530b988d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:42:40.140 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:40.136+0000 7fc89a800640 1 --2- 172.21.15.83:0/2549421961 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562530b9f6d0 0x562530b988d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:40.140 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:40.136+0000 7fc89a800640 1 -- 172.21.15.83:0/2549421961 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562530b94e10 msgr2=0x562530ba7bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:42:40.140 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:40.136+0000 7fc89a800640 1 --2- 172.21.15.83:0/2549421961 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562530b94e10 0x562530ba7bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:40.140 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:40.136+0000 7fc89a800640 1 -- 172.21.15.83:0/2549421961 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562530b9a580 con 0x562530b944f0 2024-08-19T01:42:40.140 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:40.137+0000 7fc89a800640 1 --2- 172.21.15.83:0/2549421961 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562530b944f0 0x562530ba7670 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7fc890000f80 tx=0x7fc890014040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d17532af84d3dc92 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:42:40.140 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:40.137+0000 7fc877400640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-08-19T01:42:40.140 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:40.137+0000 7fc877400640 1 -- 172.21.15.83:0/2549421961 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc89002c040 con 0x562530b944f0 2024-08-19T01:42:40.140 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:40.137+0000 7fc89f2c8740 1 -- 172.21.15.83:0/2549421961 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc884010910 con 0x562530b944f0 2024-08-19T01:42:40.140 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:40.137+0000 7fc877400640 1 -- 172.21.15.83:0/2549421961 <== mon.2 v2:172.21.15.117:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8269+0+0 (secure 0 0 0) 0x7fc89000fa70 con 0x562530b944f0 2024-08-19T01:42:40.140 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:40.137+0000 7fc877400640 1 -- 172.21.15.83:0/2549421961 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fc890043070 con 0x562530b944f0 2024-08-19T01:42:40.140 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:40.137+0000 7fc89f2c8740 10 client.26147 fetch_fsmap learned FSMap version 29 2024-08-19T01:42:40.140 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:40.137+0000 7fc89f2c8740 1 -- 172.21.15.83:0/2549421961 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x562530c19470 con 0x562530b944f0 2024-08-19T01:42:40.140 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:40.137+0000 7fc877400640 1 -- 172.21.15.83:0/2549421961 <== mon.2 v2:172.21.15.117:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc884010910 con 0x562530b944f0 2024-08-19T01:42:40.142 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:40.138+0000 7fc877400640 1 -- 172.21.15.83:0/2549421961 <== mon.2 v2:172.21.15.117:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7fc890028930 con 0x562530b944f0 2024-08-19T01:42:40.142 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:40.138+0000 7fc89f2c8740 10 client.26147 fetch_fsmap finished waiting for FSMap version 29 2024-08-19T01:42:40.142 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:40.138+0000 7fc89f2c8740 10 client.26147 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-08-19T01:42:40.142 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:40.138+0000 7fc89f2c8740 20 client.26147 populate_metadata read hostname 'smithi083' 2024-08-19T01:42:40.142 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:40.138+0000 7fc89f2c8740 1 --2- 172.21.15.83:0/2549421961 >> [v2:172.21.15.117:6836/3185224684,v1:172.21.15.117:6837/3185224684] conn(0x562530c19a70 0x562530c1be90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:42:40.142 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:40.138+0000 7fc89f2c8740 4 client.26147 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-08-19T01:42:40.142 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:40.138+0000 7fc89f2c8740 1 -- 172.21.15.83:0/2549421961 --> [v2:172.21.15.117:6836/3185224684,v1:172.21.15.117:6837/3185224684] -- command(tid 0: {"prefix": "damage ls"}) -- 0x562530b01670 con 0x562530c19a70 2024-08-19T01:42:40.142 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:40.138+0000 7fc899400640 1 --2- 172.21.15.83:0/2549421961 >> [v2:172.21.15.117:6836/3185224684,v1:172.21.15.117:6837/3185224684] conn(0x562530c19a70 0x562530c1be90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:42:40.142 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:40.139+0000 7fc899400640 1 --2- 172.21.15.83:0/2549421961 >> [v2:172.21.15.117:6836/3185224684,v1:172.21.15.117:6837/3185224684] conn(0x562530c19a70 0x562530c1be90 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=55030966a0b188ed server_cookie=3802756372c89603 in_seq=0 out_seq=0 2024-08-19T01:42:40.143 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:40.139+0000 7fc877400640 10 client.26147 ms_handle_connect on v2:172.21.15.117:6836/3185224684 2024-08-19T01:42:40.143 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:40.139+0000 7fc877400640 1 -- 172.21.15.83:0/2549421961 <== mds.2 v2:172.21.15.117:6836/3185224684 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x562530b01670 con 0x562530c19a70 2024-08-19T01:42:40.143 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:40.139+0000 7fc877400640 10 client.26147 handle_command_reply: tid=0 2024-08-19T01:42:40.143 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:40.139+0000 7fc89f2c8740 1 client.26147 shutdown 2024-08-19T01:42:40.143 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:40.139+0000 7fc89f2c8740 1 -- 172.21.15.83:0/2549421961 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562530b944f0 msgr2=0x562530ba7670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:42:40.143 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:40.139+0000 7fc89f2c8740 1 --2- 172.21.15.83:0/2549421961 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562530b944f0 0x562530ba7670 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7fc890000f80 tx=0x7fc890014040 comp rx=0 tx=0).stop 2024-08-19T01:42:40.143 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:40.139+0000 7fc89f2c8740 1 -- 172.21.15.83:0/2549421961 shutdown_connections 2024-08-19T01:42:40.143 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:40.139+0000 7fc89f2c8740 1 --2- 172.21.15.83:0/2549421961 >> [v2:172.21.15.117:6836/3185224684,v1:172.21.15.117:6837/3185224684] conn(0x562530c19a70 0x562530c1be90 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:40.143 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:40.139+0000 7fc89f2c8740 1 --2- 172.21.15.83:0/2549421961 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562530b9f6d0 0x562530b988d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:40.143 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:40.139+0000 7fc89f2c8740 1 --2- 172.21.15.83:0/2549421961 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562530b94e10 0x562530ba7bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:40.143 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:40.139+0000 7fc89f2c8740 1 --2- 172.21.15.83:0/2549421961 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562530b944f0 0x562530ba7670 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:40.143 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:40.139+0000 7fc877400640 0 client.26147 ms_handle_reset on v2:172.21.15.117:6836/3185224684 2024-08-19T01:42:40.143 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:40.139+0000 7fc89f2c8740 1 -- 172.21.15.83:0/2549421961 >> 172.21.15.83:0/2549421961 conn(0x562530b90a80 msgr2=0x562530b9cde0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:42:40.143 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:40.139+0000 7fc89f2c8740 1 -- 172.21.15.83:0/2549421961 shutdown_connections 2024-08-19T01:42:40.144 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:40.139+0000 7fc89f2c8740 1 -- 172.21.15.83:0/2549421961 wait complete. 2024-08-19T01:42:40.144 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:40.139+0000 7fc89f2c8740 20 client.26147 trim_cache size 0 max 16384 2024-08-19T01:42:40.144 INFO:teuthology.orchestra.run.smithi083.stdout:[] 2024-08-19T01:42:40.145 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:40.140+0000 7fc89c800640 1 -- 172.21.15.83:0/1077024538 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7fc87c0a2b80 msgr2=0x7fc87c0a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:42:40.145 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:40.140+0000 7fc89c800640 1 --2- 172.21.15.83:0/1077024538 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7fc87c0a2b80 0x7fc87c0a5040 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7fc8880bbc00 tx=0x7fc88803b750 comp rx=0 tx=0).stop 2024-08-19T01:42:40.145 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:40.140+0000 7fc89c800640 1 -- 172.21.15.83:0/1077024538 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc894103ae0 msgr2=0x7fc89419a390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:42:40.145 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:40.140+0000 7fc89c800640 1 --2- 172.21.15.83:0/1077024538 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc894103ae0 0x7fc89419a390 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fc88802a9e0 tx=0x7fc888003c10 comp rx=0 tx=0).stop 2024-08-19T01:42:40.145 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:40.140+0000 7fc89c800640 1 -- 172.21.15.83:0/1077024538 shutdown_connections 2024-08-19T01:42:40.145 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:40.140+0000 7fc89c800640 1 --2- 172.21.15.83:0/1077024538 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7fc87c0a2b80 0x7fc87c0a5040 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:40.145 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:40.140+0000 7fc89c800640 1 --2- 172.21.15.83:0/1077024538 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc894109b70 0x7fc89419ec60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:40.145 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:40.140+0000 7fc89c800640 1 --2- 172.21.15.83:0/1077024538 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc894104480 0x7fc89419a8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:40.145 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:40.140+0000 7fc89c800640 1 --2- 172.21.15.83:0/1077024538 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc894103ae0 0x7fc89419a390 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:40.145 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:40.140+0000 7fc89c800640 1 -- 172.21.15.83:0/1077024538 >> 172.21.15.83:0/1077024538 conn(0x7fc8940ff780 msgr2=0x7fc8940fff90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:42:40.145 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:40.140+0000 7fc89c800640 1 -- 172.21.15.83:0/1077024538 shutdown_connections 2024-08-19T01:42:40.146 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:40.141+0000 7fc89c800640 1 -- 172.21.15.83:0/1077024538 wait complete. 2024-08-19T01:42:41.179 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833 shell --fsid bad8780e-5dc9-11ef-bccf-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-08-19T01:42:41.583 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/bad8780e-5dc9-11ef-bccf-c7b262605968/mon.a/config 2024-08-19T01:42:41.948 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:42:41 smithi083 ceph-mon[17120]: pgmap v574: 97 pgs: 97 active+clean; 219 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 2.3 MiB/s wr, 589 op/s 2024-08-19T01:42:41.952 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:42:41 smithi117 ceph-mon[22016]: pgmap v574: 97 pgs: 97 active+clean; 219 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 2.3 MiB/s wr, 589 op/s 2024-08-19T01:42:41.965 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:42:41 smithi167 ceph-mon[21875]: pgmap v574: 97 pgs: 97 active+clean; 219 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 2.3 MiB/s wr, 589 op/s 2024-08-19T01:42:42.218 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:42.216+0000 7f1e16e00640 1 -- 172.21.15.83:0/973481263 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1e10104460 msgr2=0x7f1e10110990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:42:42.218 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:42.216+0000 7f1e16e00640 1 --2- 172.21.15.83:0/973481263 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1e10104460 0x7f1e10110990 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f1e08009890 tx=0x7f1e0802a360 comp rx=0 tx=0).stop 2024-08-19T01:42:42.218 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:42.216+0000 7f1e16e00640 1 -- 172.21.15.83:0/973481263 shutdown_connections 2024-08-19T01:42:42.218 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:42.216+0000 7f1e16e00640 1 --2- 172.21.15.83:0/973481263 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1e10104460 0x7f1e10110990 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:42.219 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:42.216+0000 7f1e16e00640 1 --2- 172.21.15.83:0/973481263 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1e10103ac0 0x7f1e10103f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:42.219 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:42.216+0000 7f1e16e00640 1 --2- 172.21.15.83:0/973481263 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1e10109ac0 0x7f1e10109ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:42.219 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:42.216+0000 7f1e16e00640 1 -- 172.21.15.83:0/973481263 >> 172.21.15.83:0/973481263 conn(0x7f1e100ff7c0 msgr2=0x7f1e10101be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:42:42.219 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:42.216+0000 7f1e16e00640 1 -- 172.21.15.83:0/973481263 shutdown_connections 2024-08-19T01:42:42.219 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:42.216+0000 7f1e16e00640 1 -- 172.21.15.83:0/973481263 wait complete. 2024-08-19T01:42:42.220 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:42.217+0000 7f1e16e00640 1 Processor -- start 2024-08-19T01:42:42.220 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:42.218+0000 7f1e16e00640 1 -- start start 2024-08-19T01:42:42.220 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:42.218+0000 7f1e16e00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1e10103ac0 0x7f1e1019a380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:42:42.220 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:42.218+0000 7f1e16e00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1e10104460 0x7f1e1019a8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:42:42.221 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:42.218+0000 7f1e16e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1e10109ac0 0x7f1e1019ec50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:42:42.221 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:42.218+0000 7f1e16e00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f1e10113ec0 con 0x7f1e10109ac0 2024-08-19T01:42:42.221 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:42.218+0000 7f1e16e00640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f1e10113d40 con 0x7f1e10104460 2024-08-19T01:42:42.221 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:42.218+0000 7f1e16e00640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f1e10114040 con 0x7f1e10103ac0 2024-08-19T01:42:42.221 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:42.218+0000 7f1e0f400640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1e10104460 0x7f1e1019a8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:42:42.221 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:42.218+0000 7f1e0f400640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1e10104460 0x7f1e1019a8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.83:40256/0 (socket says 172.21.15.83:40256) 2024-08-19T01:42:42.221 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:42.218+0000 7f1e0f400640 1 -- 172.21.15.83:0/3440089004 learned_addr learned my addr 172.21.15.83:0/3440089004 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-08-19T01:42:42.221 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:42.218+0000 7f1e0fe00640 1 --2- 172.21.15.83:0/3440089004 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1e10103ac0 0x7f1e1019a380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:42:42.221 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:42.219+0000 7f1e0f400640 1 -- 172.21.15.83:0/3440089004 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1e10103ac0 msgr2=0x7f1e1019a380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:42:42.221 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:42.219+0000 7f1e0f400640 1 --2- 172.21.15.83:0/3440089004 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1e10103ac0 0x7f1e1019a380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:42.221 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:42.219+0000 7f1e14e00640 1 --2- 172.21.15.83:0/3440089004 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1e10109ac0 0x7f1e1019ec50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:42:42.221 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:42.219+0000 7f1e0f400640 1 -- 172.21.15.83:0/3440089004 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1e10109ac0 msgr2=0x7f1e1019ec50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:42:42.221 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:42.219+0000 7f1e0f400640 1 --2- 172.21.15.83:0/3440089004 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1e10109ac0 0x7f1e1019ec50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:42.222 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:42.219+0000 7f1e0f400640 1 -- 172.21.15.83:0/3440089004 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1e1019f330 con 0x7f1e10104460 2024-08-19T01:42:42.222 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:42.219+0000 7f1e14e00640 1 --2- 172.21.15.83:0/3440089004 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1e10109ac0 0x7f1e1019ec50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-19T01:42:42.222 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:42.219+0000 7f1e0f400640 1 --2- 172.21.15.83:0/3440089004 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1e10104460 0x7f1e1019a8c0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f1dfc00e9f0 tx=0x7f1dfc00edf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:42:42.222 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:42.220+0000 7f1e0cc00640 1 -- 172.21.15.83:0/3440089004 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1dfc00cd80 con 0x7f1e10104460 2024-08-19T01:42:42.222 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:42.220+0000 7f1e0cc00640 1 -- 172.21.15.83:0/3440089004 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1dfc005500 con 0x7f1e10104460 2024-08-19T01:42:42.223 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:42.220+0000 7f1e0cc00640 1 -- 172.21.15.83:0/3440089004 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1dfc010690 con 0x7f1e10104460 2024-08-19T01:42:42.223 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:42.220+0000 7f1e16e00640 1 -- 172.21.15.83:0/3440089004 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1e1019f620 con 0x7f1e10104460 2024-08-19T01:42:42.223 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:42.220+0000 7f1e16e00640 1 -- 172.21.15.83:0/3440089004 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1e101a6f00 con 0x7f1e10104460 2024-08-19T01:42:42.224 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:42.223+0000 7f1e19817740 1 Processor -- start 2024-08-19T01:42:42.224 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:42.223+0000 7f1e19817740 1 -- start start 2024-08-19T01:42:42.225 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:42.223+0000 7f1e19817740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d7d70354f0 0x55d7d70358d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:42:42.225 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:42.223+0000 7f1e19817740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d7d7035e10 0x55d7d7040190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:42:42.225 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:42.223+0000 7f1e19817740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d7d70406d0 0x55d7d7042ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:42:42.225 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:42.223+0000 7f1e19817740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55d7d6f9abf0 con 0x55d7d70406d0 2024-08-19T01:42:42.225 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:42.223+0000 7f1e19817740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55d7d6f32b20 con 0x55d7d70354f0 2024-08-19T01:42:42.225 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:42.223+0000 7f1e19817740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55d7d6f96bb0 con 0x55d7d7035e10 2024-08-19T01:42:42.226 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:42.224+0000 7f1e14e00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d7d70354f0 0x55d7d70358d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:42:42.226 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:42.224+0000 7f1e14e00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d7d70354f0 0x55d7d70358d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.83:40258/0 (socket says 172.21.15.83:40258) 2024-08-19T01:42:42.226 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:42.224+0000 7f1e14e00640 1 -- 172.21.15.83:0/1185723589 learned_addr learned my addr 172.21.15.83:0/1185723589 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-08-19T01:42:42.226 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:42.224+0000 7f1e0fe00640 1 --2- 172.21.15.83:0/1185723589 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d7d7035e10 0x55d7d7040190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:42:42.227 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:42.224+0000 7f1e0cc00640 1 -- 172.21.15.83:0/3440089004 <== mon.2 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7f1dfc0046c0 con 0x7f1e10104460 2024-08-19T01:42:42.227 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:42.224+0000 7f1e14e00640 1 -- 172.21.15.83:0/1185723589 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d7d7035e10 msgr2=0x55d7d7040190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:42:42.227 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:42.224+0000 7f1e14e00640 1 --2- 172.21.15.83:0/1185723589 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d7d7035e10 0x55d7d7040190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:42.227 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:42.224+0000 7f1e14e00640 1 -- 172.21.15.83:0/1185723589 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d7d70406d0 msgr2=0x55d7d7042ac0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-08-19T01:42:42.227 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:42.224+0000 7f1e14e00640 1 --2- 172.21.15.83:0/1185723589 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d7d70406d0 0x55d7d7042ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:42.227 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:42.224+0000 7f1e14e00640 1 -- 172.21.15.83:0/1185723589 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d7d6f397f0 con 0x55d7d70354f0 2024-08-19T01:42:42.227 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:42.224+0000 7f1e14e00640 1 --2- 172.21.15.83:0/1185723589 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d7d70354f0 0x55d7d70358d0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f1e08005b60 tx=0x7f1e08030340 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4202ffde1dcdda7b server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:42:42.227 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:42.225+0000 7f1e05600640 1 -- 172.21.15.83:0/1185723589 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e08040020 con 0x55d7d70354f0 2024-08-19T01:42:42.227 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:42.225+0000 7f1e05600640 1 -- 172.21.15.83:0/1185723589 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1e08038040 con 0x55d7d70354f0 2024-08-19T01:42:42.227 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:42.225+0000 7f1e05600640 1 -- 172.21.15.83:0/1185723589 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e08037070 con 0x55d7d70354f0 2024-08-19T01:42:42.228 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:42.225+0000 7f1e19817740 1 -- 172.21.15.83:0/1185723589 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d7d70354f0 msgr2=0x55d7d70358d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:42:42.228 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:42.226+0000 7f1e19817740 1 --2- 172.21.15.83:0/1185723589 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d7d70354f0 0x55d7d70358d0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f1e08005b60 tx=0x7f1e08030340 comp rx=0 tx=0).stop 2024-08-19T01:42:42.228 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:42.226+0000 7f1e0cc00640 1 --2- 172.21.15.83:0/3440089004 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f1df40a2c00 0x7f1df40a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:42:42.228 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:42.226+0000 7f1e19817740 1 -- 172.21.15.83:0/1185723589 shutdown_connections 2024-08-19T01:42:42.228 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:42.226+0000 7f1e19817740 1 --2- 172.21.15.83:0/1185723589 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d7d70406d0 0x55d7d7042ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:42.229 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:42.226+0000 7f1e19817740 1 --2- 172.21.15.83:0/1185723589 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d7d7035e10 0x55d7d7040190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:42.229 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:42.226+0000 7f1e19817740 1 --2- 172.21.15.83:0/1185723589 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d7d70354f0 0x55d7d70358d0 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:42.229 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:42.226+0000 7f1e19817740 1 -- 172.21.15.83:0/1185723589 >> 172.21.15.83:0/1185723589 conn(0x55d7d7031a80 msgr2=0x55d7d7031e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:42:42.229 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:42.226+0000 7f1e0cc00640 1 -- 172.21.15.83:0/3440089004 <== mon.2 v2:172.21.15.117:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8269+0+0 (secure 0 0 0) 0x7f1dfc0cf540 con 0x7f1e10104460 2024-08-19T01:42:42.229 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:42.226+0000 7f1e14e00640 1 --2- 172.21.15.83:0/3440089004 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f1df40a2c00 0x7f1df40a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:42:42.229 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:42.226+0000 7f1e19817740 1 -- 172.21.15.83:0/1185723589 shutdown_connections 2024-08-19T01:42:42.229 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:42.226+0000 7f1e19817740 1 -- 172.21.15.83:0/1185723589 wait complete. 2024-08-19T01:42:42.229 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:42.227+0000 7f1e19817740 1 -- start start 2024-08-19T01:42:42.229 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:42.227+0000 7f1e14e00640 1 --2- 172.21.15.83:0/3440089004 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f1df40a2c00 0x7f1df40a50c0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f1e080306a0 tx=0x7f1e0802a360 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:42:42.229 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:42.227+0000 7f1e19817740 1 Processor -- start 2024-08-19T01:42:42.231 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:42.227+0000 7f1e19817740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d7d70354f0 0x55d7d703d590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:42:42.231 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:42.227+0000 7f1e19817740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d7d7035e10 0x55d7d703dad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:42:42.231 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:42.227+0000 7f1e19817740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d7d70406d0 0x55d7d703e010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:42:42.231 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:42.227+0000 7f1e19817740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55d7d70432a0 con 0x55d7d70406d0 2024-08-19T01:42:42.231 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:42.227+0000 7f1e19817740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55d7d6f55cf0 con 0x55d7d70354f0 2024-08-19T01:42:42.231 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:42.227+0000 7f1e19817740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55d7d7043420 con 0x55d7d7035e10 2024-08-19T01:42:42.232 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:42.227+0000 7f1e19817740 10 client.0 _ll_register_callbacks cb 0x55d7d6f41e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-19T01:42:42.232 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:42.228+0000 7f1e0fe00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d7d70354f0 0x55d7d703d590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:42:42.232 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:42.228+0000 7f1e0fe00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d7d70354f0 0x55d7d703d590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.83:40260/0 (socket says 172.21.15.83:40260) 2024-08-19T01:42:42.232 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:42.228+0000 7f1e0fe00640 1 -- 172.21.15.83:0/3699189945 learned_addr learned my addr 172.21.15.83:0/3699189945 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-08-19T01:42:42.232 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:42.228+0000 7f1e14e00640 1 --2- 172.21.15.83:0/3699189945 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d7d70406d0 0x55d7d703e010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:42:42.232 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:42.228+0000 7f1e0f400640 1 --2- 172.21.15.83:0/3699189945 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d7d7035e10 0x55d7d703dad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:42:42.232 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:42.228+0000 7f1e0fe00640 1 -- 172.21.15.83:0/3699189945 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d7d7035e10 msgr2=0x55d7d703dad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:42:42.232 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:42.228+0000 7f1e0fe00640 1 --2- 172.21.15.83:0/3699189945 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d7d7035e10 0x55d7d703dad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:42.232 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:42.228+0000 7f1e0fe00640 1 -- 172.21.15.83:0/3699189945 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d7d70406d0 msgr2=0x55d7d703e010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:42:42.232 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:42.228+0000 7f1e0fe00640 1 --2- 172.21.15.83:0/3699189945 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d7d70406d0 0x55d7d703e010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:42.232 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:42.228+0000 7f1e0fe00640 1 -- 172.21.15.83:0/3699189945 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d7d6d0c3c0 con 0x55d7d70354f0 2024-08-19T01:42:42.232 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:42.228+0000 7f1e14e00640 1 --2- 172.21.15.83:0/3699189945 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d7d70406d0 0x55d7d703e010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-19T01:42:42.232 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:42.228+0000 7f1e0f400640 1 --2- 172.21.15.83:0/3699189945 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d7d7035e10 0x55d7d703dad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-19T01:42:42.233 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:42.228+0000 7f1e0fe00640 1 --2- 172.21.15.83:0/3699189945 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d7d70354f0 0x55d7d703d590 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f1e0000eef0 tx=0x7f1e0000d7b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5a76190f94f7905e server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:42:42.233 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:42.228+0000 7f1dfa000640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-08-19T01:42:42.233 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:42.229+0000 7f1dfa000640 1 -- 172.21.15.83:0/3699189945 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e0001c030 con 0x55d7d70354f0 2024-08-19T01:42:42.233 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:42.229+0000 7f1e19817740 1 -- 172.21.15.83:0/3699189945 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1e08038040 con 0x55d7d70354f0 2024-08-19T01:42:42.233 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:42.229+0000 7f1dfa000640 1 -- 172.21.15.83:0/3699189945 <== mon.2 v2:172.21.15.117:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8269+0+0 (secure 0 0 0) 0x7f1e00004b20 con 0x55d7d70354f0 2024-08-19T01:42:42.233 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:42.229+0000 7f1dfa000640 1 -- 172.21.15.83:0/3699189945 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f1e00004d20 con 0x55d7d70354f0 2024-08-19T01:42:42.233 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:42.229+0000 7f1e19817740 10 client.26159 fetch_fsmap learned FSMap version 29 2024-08-19T01:42:42.233 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:42.229+0000 7f1e19817740 1 -- 172.21.15.83:0/3699189945 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55d7d70577f0 con 0x55d7d70354f0 2024-08-19T01:42:42.233 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:42.229+0000 7f1dfa000640 1 -- 172.21.15.83:0/3699189945 <== mon.2 v2:172.21.15.117:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1e08038040 con 0x55d7d70354f0 2024-08-19T01:42:42.233 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:42.229+0000 7f1dfa000640 1 -- 172.21.15.83:0/3699189945 <== mon.2 v2:172.21.15.117:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f1e0000b040 con 0x55d7d70354f0 2024-08-19T01:42:42.233 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:42.230+0000 7f1e19817740 10 client.26159 fetch_fsmap finished waiting for FSMap version 29 2024-08-19T01:42:42.233 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:42.230+0000 7f1e19817740 10 client.26159 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.g 2024-08-19T01:42:42.233 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:42.230+0000 7f1e19817740 20 client.26159 populate_metadata read hostname 'smithi083' 2024-08-19T01:42:42.233 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:42.230+0000 7f1e19817740 1 --2- 172.21.15.83:0/3699189945 >> [v2:172.21.15.83:6838/3239261845,v1:172.21.15.83:6839/3239261845] conn(0x55d7d7058cb0 0x55d7d705b0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:42:42.234 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:42.230+0000 7f1e19817740 4 client.26159 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-19T01:42:42.234 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:42.230+0000 7f1e19817740 1 -- 172.21.15.83:0/3699189945 --> [v2:172.21.15.83:6838/3239261845,v1:172.21.15.83:6839/3239261845] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d7d6fa8160 con 0x55d7d7058cb0 2024-08-19T01:42:42.234 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:42.230+0000 7f1e0f400640 1 --2- 172.21.15.83:0/3699189945 >> [v2:172.21.15.83:6838/3239261845,v1:172.21.15.83:6839/3239261845] conn(0x55d7d7058cb0 0x55d7d705b0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:42:42.235 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:42.230+0000 7f1e0f400640 1 --2- 172.21.15.83:0/3699189945 >> [v2:172.21.15.83:6838/3239261845,v1:172.21.15.83:6839/3239261845] conn(0x55d7d7058cb0 0x55d7d705b0e0 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=9d3d0dcfb1b9daf5 server_cookie=8b2026322ad8fab4 in_seq=0 out_seq=0 2024-08-19T01:42:42.235 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:42.230+0000 7f1dfa000640 10 client.26159 ms_handle_connect on v2:172.21.15.83:6838/3239261845 2024-08-19T01:42:42.235 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:42.231+0000 7f1dfa000640 1 -- 172.21.15.83:0/3699189945 <== mds.1 v2:172.21.15.83:6838/3239261845 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55d7d6fa8160 con 0x55d7d7058cb0 2024-08-19T01:42:42.235 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:42.231+0000 7f1dfa000640 10 client.26159 handle_command_reply: tid=0 2024-08-19T01:42:42.235 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:42.231+0000 7f1e19817740 1 client.26159 shutdown 2024-08-19T01:42:42.235 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:42.231+0000 7f1e19817740 1 -- 172.21.15.83:0/3699189945 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d7d70354f0 msgr2=0x55d7d703d590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:42:42.235 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:42.231+0000 7f1e19817740 1 --2- 172.21.15.83:0/3699189945 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d7d70354f0 0x55d7d703d590 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f1e0000eef0 tx=0x7f1e0000d7b0 comp rx=0 tx=0).stop 2024-08-19T01:42:42.235 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:42.231+0000 7f1e19817740 1 -- 172.21.15.83:0/3699189945 shutdown_connections 2024-08-19T01:42:42.235 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:42.231+0000 7f1e19817740 1 --2- 172.21.15.83:0/3699189945 >> [v2:172.21.15.83:6838/3239261845,v1:172.21.15.83:6839/3239261845] conn(0x55d7d7058cb0 0x55d7d705b0e0 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:42.235 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:42.231+0000 7f1e19817740 1 --2- 172.21.15.83:0/3699189945 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d7d70406d0 0x55d7d703e010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:42.236 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:42.231+0000 7f1e19817740 1 --2- 172.21.15.83:0/3699189945 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d7d7035e10 0x55d7d703dad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:42.236 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:42.231+0000 7f1e19817740 1 --2- 172.21.15.83:0/3699189945 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d7d70354f0 0x55d7d703d590 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:42.236 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:42.231+0000 7f1dfa000640 0 client.26159 ms_handle_reset on v2:172.21.15.83:6838/3239261845 2024-08-19T01:42:42.236 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:42.231+0000 7f1e19817740 1 -- 172.21.15.83:0/3699189945 >> 172.21.15.83:0/3699189945 conn(0x55d7d7031a80 msgr2=0x55d7d7037760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:42:42.236 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:42.232+0000 7f1e19817740 1 -- 172.21.15.83:0/3699189945 shutdown_connections 2024-08-19T01:42:42.236 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:42.232+0000 7f1e19817740 1 -- 172.21.15.83:0/3699189945 wait complete. 2024-08-19T01:42:42.236 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:42.232+0000 7f1e19817740 20 client.26159 trim_cache size 0 max 16384 2024-08-19T01:42:42.247 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:42.246+0000 7f1e19817740 1 Processor -- start 2024-08-19T01:42:42.247 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:42.246+0000 7f1e19817740 1 -- start start 2024-08-19T01:42:42.248 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:42.246+0000 7f1e19817740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d7d70354f0 0x55d7d702cb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:42:42.249 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:42.246+0000 7f1e19817740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d7d7035e10 0x55d7d70609b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:42:42.249 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:42.246+0000 7f1e19817740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d7d70406d0 0x55d7d7060ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:42:42.249 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:42.246+0000 7f1e19817740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f1e101ada70 con 0x55d7d7035e10 2024-08-19T01:42:42.249 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:42.246+0000 7f1e19817740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f1e101ab2e0 con 0x55d7d70354f0 2024-08-19T01:42:42.249 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:42.246+0000 7f1e19817740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f1e101a9150 con 0x55d7d70406d0 2024-08-19T01:42:42.249 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:42.246+0000 7f1e0fe00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d7d70354f0 0x55d7d702cb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:42:42.249 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:42.246+0000 7f1e0fe00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d7d70354f0 0x55d7d702cb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.83:40266/0 (socket says 172.21.15.83:40266) 2024-08-19T01:42:42.249 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:42.246+0000 7f1e0fe00640 1 -- 172.21.15.83:0/2496808566 learned_addr learned my addr 172.21.15.83:0/2496808566 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-08-19T01:42:42.249 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:42.246+0000 7f1e0f400640 1 --2- 172.21.15.83:0/2496808566 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d7d7035e10 0x55d7d70609b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:42:42.249 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:42.246+0000 7f1e14e00640 1 --2- 172.21.15.83:0/2496808566 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d7d70406d0 0x55d7d7060ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:42:42.249 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:42.247+0000 7f1e0fe00640 1 -- 172.21.15.83:0/2496808566 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d7d70406d0 msgr2=0x55d7d7060ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:42:42.249 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:42.247+0000 7f1e0fe00640 1 --2- 172.21.15.83:0/2496808566 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d7d70406d0 0x55d7d7060ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:42.249 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:42.247+0000 7f1e0fe00640 1 -- 172.21.15.83:0/2496808566 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d7d7035e10 msgr2=0x55d7d70609b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:42:42.249 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:42.247+0000 7f1e0fe00640 1 --2- 172.21.15.83:0/2496808566 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d7d7035e10 0x55d7d70609b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:42.250 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:42.247+0000 7f1e0fe00640 1 -- 172.21.15.83:0/2496808566 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d7d7054df0 con 0x55d7d70354f0 2024-08-19T01:42:42.250 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:42.247+0000 7f1e0fe00640 1 --2- 172.21.15.83:0/2496808566 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d7d70354f0 0x55d7d702cb40 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f1e0000e870 tx=0x7f1e00000f40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=331f340c3ea828da server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:42:42.250 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:42.247+0000 7f1e04c00640 1 -- 172.21.15.83:0/2496808566 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e0001c070 con 0x55d7d70354f0 2024-08-19T01:42:42.250 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:42.247+0000 7f1e04c00640 1 -- 172.21.15.83:0/2496808566 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1e0001d040 con 0x55d7d70354f0 2024-08-19T01:42:42.250 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:42.247+0000 7f1e04c00640 1 -- 172.21.15.83:0/2496808566 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e00018550 con 0x55d7d70354f0 2024-08-19T01:42:42.252 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:42.248+0000 7f1e19817740 1 -- 172.21.15.83:0/2496808566 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d7d70354f0 msgr2=0x55d7d702cb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:42:42.252 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:42.248+0000 7f1e19817740 1 --2- 172.21.15.83:0/2496808566 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d7d70354f0 0x55d7d702cb40 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f1e0000e870 tx=0x7f1e00000f40 comp rx=0 tx=0).stop 2024-08-19T01:42:42.252 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:42.248+0000 7f1e19817740 1 -- 172.21.15.83:0/2496808566 shutdown_connections 2024-08-19T01:42:42.252 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:42.248+0000 7f1e19817740 1 --2- 172.21.15.83:0/2496808566 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d7d70406d0 0x55d7d7060ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:42.253 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:42.248+0000 7f1e19817740 1 --2- 172.21.15.83:0/2496808566 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d7d7035e10 0x55d7d70609b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:42.253 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:42.248+0000 7f1e19817740 1 --2- 172.21.15.83:0/2496808566 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d7d70354f0 0x55d7d702cb40 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:42.253 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:42.248+0000 7f1e19817740 1 -- 172.21.15.83:0/2496808566 >> 172.21.15.83:0/2496808566 conn(0x55d7d7031a80 msgr2=0x55d7d703c680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:42:42.253 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:42.248+0000 7f1e19817740 1 -- 172.21.15.83:0/2496808566 shutdown_connections 2024-08-19T01:42:42.253 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:42.248+0000 7f1e19817740 1 -- 172.21.15.83:0/2496808566 wait complete. 2024-08-19T01:42:42.253 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:42.249+0000 7f1e19817740 1 -- start start 2024-08-19T01:42:42.253 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:42.249+0000 7f1e19817740 1 Processor -- start 2024-08-19T01:42:42.253 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:42.249+0000 7f1e19817740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d7d70354f0 0x55d7d7050f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:42:42.253 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:42.249+0000 7f1e19817740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d7d7035e10 0x55d7d70514a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:42:42.253 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:42.249+0000 7f1e19817740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d7d70406d0 0x55d7d7039bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:42:42.253 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:42.249+0000 7f1e19817740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f1e101a97a0 con 0x55d7d7035e10 2024-08-19T01:42:42.253 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:42.249+0000 7f1e19817740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f1e101aac10 con 0x55d7d70406d0 2024-08-19T01:42:42.253 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:42.249+0000 7f1e19817740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f1e101a9e70 con 0x55d7d70354f0 2024-08-19T01:42:42.253 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:42.249+0000 7f1e19817740 10 client.0 _ll_register_callbacks cb 0x7f1e0000ead0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-19T01:42:42.254 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:42.250+0000 7f1e14e00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d7d70406d0 0x55d7d7039bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:42:42.254 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:42.250+0000 7f1e0f400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d7d7035e10 0x55d7d70514a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:42:42.254 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:42.250+0000 7f1e14e00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d7d70406d0 0x55d7d7039bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.83:40272/0 (socket says 172.21.15.83:40272) 2024-08-19T01:42:42.254 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:42.250+0000 7f1e14e00640 1 -- 172.21.15.83:0/4137999786 learned_addr learned my addr 172.21.15.83:0/4137999786 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-08-19T01:42:42.254 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:42.250+0000 7f1e0f400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d7d7035e10 0x55d7d70514a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:34892/0 (socket says 172.21.15.83:34892) 2024-08-19T01:42:42.254 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:42.250+0000 7f1e0fe00640 1 --2- 172.21.15.83:0/4137999786 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d7d70354f0 0x55d7d7050f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:42:42.254 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:42.250+0000 7f1e14e00640 1 -- 172.21.15.83:0/4137999786 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d7d70354f0 msgr2=0x55d7d7050f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:42:42.254 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:42.250+0000 7f1e14e00640 1 --2- 172.21.15.83:0/4137999786 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d7d70354f0 0x55d7d7050f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:42.254 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:42.250+0000 7f1e14e00640 1 -- 172.21.15.83:0/4137999786 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d7d7035e10 msgr2=0x55d7d70514a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:42:42.254 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:42.250+0000 7f1e14e00640 1 --2- 172.21.15.83:0/4137999786 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d7d7035e10 0x55d7d70514a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:42.254 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:42.250+0000 7f1e14e00640 1 -- 172.21.15.83:0/4137999786 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d7d703a130 con 0x55d7d70406d0 2024-08-19T01:42:42.254 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:42.250+0000 7f1e0f400640 1 --2- 172.21.15.83:0/4137999786 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d7d7035e10 0x55d7d70514a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-19T01:42:42.254 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:42.250+0000 7f1e14e00640 1 --2- 172.21.15.83:0/4137999786 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d7d70406d0 0x55d7d7039bc0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f1e08030a20 tx=0x7f1e08030a50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2df0d7f0061eedcc server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:42:42.255 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:42.250+0000 7f1dfa000640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-08-19T01:42:42.255 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:42.251+0000 7f1dfa000640 1 -- 172.21.15.83:0/4137999786 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e08040050 con 0x55d7d70406d0 2024-08-19T01:42:42.255 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:42.251+0000 7f1e19817740 1 -- 172.21.15.83:0/4137999786 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1e0001d040 con 0x55d7d70406d0 2024-08-19T01:42:42.255 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:42.251+0000 7f1dfa000640 1 -- 172.21.15.83:0/4137999786 <== mon.2 v2:172.21.15.117:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8269+0+0 (secure 0 0 0) 0x7f1e080037f0 con 0x55d7d70406d0 2024-08-19T01:42:42.255 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:42.251+0000 7f1dfa000640 1 -- 172.21.15.83:0/4137999786 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f1e08062050 con 0x55d7d70406d0 2024-08-19T01:42:42.255 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:42.251+0000 7f1e19817740 10 client.26165 fetch_fsmap learned FSMap version 29 2024-08-19T01:42:42.255 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:42.251+0000 7f1e19817740 1 -- 172.21.15.83:0/4137999786 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55d7d70c31e0 con 0x55d7d70406d0 2024-08-19T01:42:42.257 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:42.251+0000 7f1dfa000640 1 -- 172.21.15.83:0/4137999786 <== mon.2 v2:172.21.15.117:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1e0001d040 con 0x55d7d70406d0 2024-08-19T01:42:42.257 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:42.251+0000 7f1dfa000640 1 -- 172.21.15.83:0/4137999786 <== mon.2 v2:172.21.15.117:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f1e080329a0 con 0x55d7d70406d0 2024-08-19T01:42:42.257 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:42.252+0000 7f1e19817740 10 client.26165 fetch_fsmap finished waiting for FSMap version 29 2024-08-19T01:42:42.257 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:42.252+0000 7f1e19817740 10 client.26165 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.g 2024-08-19T01:42:42.257 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:42.252+0000 7f1e19817740 20 client.26165 populate_metadata read hostname 'smithi083' 2024-08-19T01:42:42.257 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:42.252+0000 7f1e19817740 1 --2- 172.21.15.83:0/4137999786 >> [v2:172.21.15.83:6838/3239261845,v1:172.21.15.83:6839/3239261845] conn(0x55d7d70c35e0 0x55d7d70c59d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:42:42.257 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:42.252+0000 7f1e19817740 4 client.26165 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-08-19T01:42:42.257 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:42.252+0000 7f1e19817740 1 -- 172.21.15.83:0/4137999786 --> [v2:172.21.15.83:6838/3239261845,v1:172.21.15.83:6839/3239261845] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55d7d6fa2670 con 0x55d7d70c35e0 2024-08-19T01:42:42.257 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:42.252+0000 7f1e0fe00640 1 --2- 172.21.15.83:0/4137999786 >> [v2:172.21.15.83:6838/3239261845,v1:172.21.15.83:6839/3239261845] conn(0x55d7d70c35e0 0x55d7d70c59d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:42:42.257 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:42.252+0000 7f1e0fe00640 1 --2- 172.21.15.83:0/4137999786 >> [v2:172.21.15.83:6838/3239261845,v1:172.21.15.83:6839/3239261845] conn(0x55d7d70c35e0 0x55d7d70c59d0 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=c036b9c010e24727 server_cookie=914a95d3aa3cea36 in_seq=0 out_seq=0 2024-08-19T01:42:42.257 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:42.252+0000 7f1dfa000640 10 client.26165 ms_handle_connect on v2:172.21.15.83:6838/3239261845 2024-08-19T01:42:42.257 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:42.253+0000 7f1dfa000640 1 -- 172.21.15.83:0/4137999786 <== mds.1 v2:172.21.15.83:6838/3239261845 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55d7d6fa2670 con 0x55d7d70c35e0 2024-08-19T01:42:42.257 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:42.253+0000 7f1dfa000640 10 client.26165 handle_command_reply: tid=0 2024-08-19T01:42:42.258 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:42.253+0000 7f1e19817740 1 client.26165 shutdown 2024-08-19T01:42:42.258 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:42.253+0000 7f1e19817740 1 -- 172.21.15.83:0/4137999786 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d7d70406d0 msgr2=0x55d7d7039bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:42:42.258 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:42.253+0000 7f1e19817740 1 --2- 172.21.15.83:0/4137999786 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d7d70406d0 0x55d7d7039bc0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f1e08030a20 tx=0x7f1e08030a50 comp rx=0 tx=0).stop 2024-08-19T01:42:42.258 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:42.253+0000 7f1e19817740 1 -- 172.21.15.83:0/4137999786 shutdown_connections 2024-08-19T01:42:42.258 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:42.253+0000 7f1e19817740 1 --2- 172.21.15.83:0/4137999786 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d7d70406d0 0x55d7d7039bc0 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:42.258 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:42.253+0000 7f1e19817740 1 --2- 172.21.15.83:0/4137999786 >> [v2:172.21.15.83:6838/3239261845,v1:172.21.15.83:6839/3239261845] conn(0x55d7d70c35e0 0x55d7d70c59d0 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:42.258 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:42.253+0000 7f1e19817740 1 --2- 172.21.15.83:0/4137999786 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d7d7035e10 0x55d7d70514a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:42.258 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:42.253+0000 7f1e19817740 1 --2- 172.21.15.83:0/4137999786 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d7d70354f0 0x55d7d7050f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:42.258 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:42.253+0000 7f1dfa000640 0 client.26165 ms_handle_reset on v2:172.21.15.83:6838/3239261845 2024-08-19T01:42:42.258 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:42.253+0000 7f1e19817740 1 -- 172.21.15.83:0/4137999786 >> 172.21.15.83:0/4137999786 conn(0x55d7d7031a80 msgr2=0x55d7d7036f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:42:42.258 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:42.253+0000 7f1e19817740 1 -- 172.21.15.83:0/4137999786 shutdown_connections 2024-08-19T01:42:42.258 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:42.253+0000 7f1e19817740 1 -- 172.21.15.83:0/4137999786 wait complete. 2024-08-19T01:42:42.258 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:42.253+0000 7f1e19817740 20 client.26165 trim_cache size 0 max 16384 2024-08-19T01:42:42.259 INFO:teuthology.orchestra.run.smithi083.stdout:[] 2024-08-19T01:42:42.259 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:42.254+0000 7f1e16e00640 1 -- 172.21.15.83:0/3440089004 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f1df40a2c00 msgr2=0x7f1df40a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:42:42.260 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:42.254+0000 7f1e16e00640 1 --2- 172.21.15.83:0/3440089004 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f1df40a2c00 0x7f1df40a50c0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f1e080306a0 tx=0x7f1e0802a360 comp rx=0 tx=0).stop 2024-08-19T01:42:42.260 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:42.254+0000 7f1e16e00640 1 -- 172.21.15.83:0/3440089004 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1e10104460 msgr2=0x7f1e1019a8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:42:42.260 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:42.254+0000 7f1e16e00640 1 --2- 172.21.15.83:0/3440089004 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1e10104460 0x7f1e1019a8c0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f1dfc00e9f0 tx=0x7f1dfc00edf0 comp rx=0 tx=0).stop 2024-08-19T01:42:42.260 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:42.254+0000 7f1e16e00640 1 -- 172.21.15.83:0/3440089004 shutdown_connections 2024-08-19T01:42:42.260 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:42.254+0000 7f1e16e00640 1 --2- 172.21.15.83:0/3440089004 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f1df40a2c00 0x7f1df40a50c0 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:42.260 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:42.254+0000 7f1e16e00640 1 --2- 172.21.15.83:0/3440089004 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1e10109ac0 0x7f1e1019ec50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:42.260 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:42.254+0000 7f1e16e00640 1 --2- 172.21.15.83:0/3440089004 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1e10104460 0x7f1e1019a8c0 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:42.260 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:42.254+0000 7f1e16e00640 1 --2- 172.21.15.83:0/3440089004 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1e10103ac0 0x7f1e1019a380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:42.260 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:42.254+0000 7f1e16e00640 1 -- 172.21.15.83:0/3440089004 >> 172.21.15.83:0/3440089004 conn(0x7f1e100ff7c0 msgr2=0x7f1e100fff40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:42:42.260 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:42.254+0000 7f1e16e00640 1 -- 172.21.15.83:0/3440089004 shutdown_connections 2024-08-19T01:42:42.260 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:42.254+0000 7f1e16e00640 1 -- 172.21.15.83:0/3440089004 wait complete. 2024-08-19T01:42:43.104 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:43.102+0000 7f093f200640 1 -- 172.21.15.83:0/4085853364 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f09401020f0 msgr2=0x7f09401044e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:42:43.104 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:43.102+0000 7f093f200640 1 --2- 172.21.15.83:0/4085853364 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f09401020f0 0x7f09401044e0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f0930009860 tx=0x7f093002a340 comp rx=0 tx=0).stop 2024-08-19T01:42:43.104 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:43.102+0000 7f093f200640 1 -- 172.21.15.83:0/4085853364 shutdown_connections 2024-08-19T01:42:43.104 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:43.102+0000 7f093f200640 1 --2- 172.21.15.83:0/4085853364 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0940107350 0x7f09401097c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:43.104 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:43.102+0000 7f093f200640 1 --2- 172.21.15.83:0/4085853364 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0940104a20 0x7f0940106e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:43.104 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:43.102+0000 7f093f200640 1 --2- 172.21.15.83:0/4085853364 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f09401020f0 0x7f09401044e0 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:43.104 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:43.102+0000 7f093f200640 1 -- 172.21.15.83:0/4085853364 >> 172.21.15.83:0/4085853364 conn(0x7f09400fbcd0 msgr2=0x7f09400fe110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:42:43.104 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:43.102+0000 7f093f200640 1 -- 172.21.15.83:0/4085853364 shutdown_connections 2024-08-19T01:42:43.104 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:43.102+0000 7f093f200640 1 -- 172.21.15.83:0/4085853364 wait complete. 2024-08-19T01:42:43.106 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:43.103+0000 7f093f200640 1 Processor -- start 2024-08-19T01:42:43.106 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:43.103+0000 7f093f200640 1 -- start start 2024-08-19T01:42:43.106 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:43.103+0000 7f093f200640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f09401020f0 0x7f0940101a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:42:43.106 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:43.103+0000 7f093f200640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0940104a20 0x7f09401000a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:42:43.106 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:43.103+0000 7f093f200640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0940107350 0x7f09401005e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:42:43.107 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:43.103+0000 7f093f200640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f094010d340 con 0x7f0940104a20 2024-08-19T01:42:43.107 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:43.103+0000 7f093f200640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f094010d1c0 con 0x7f09401020f0 2024-08-19T01:42:43.107 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:43.103+0000 7f093f200640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f094010d4c0 con 0x7f0940107350 2024-08-19T01:42:43.107 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:43.103+0000 7f093de00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f09401020f0 0x7f0940101a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:42:43.107 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:43.103+0000 7f093d400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0940104a20 0x7f09401000a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:42:43.107 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:43.103+0000 7f093de00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f09401020f0 0x7f0940101a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.83:40298/0 (socket says 172.21.15.83:40298) 2024-08-19T01:42:43.107 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:43.103+0000 7f093de00640 1 -- 172.21.15.83:0/456985686 learned_addr learned my addr 172.21.15.83:0/456985686 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-08-19T01:42:43.107 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:43.104+0000 7f093e800640 1 --2- 172.21.15.83:0/456985686 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0940107350 0x7f09401005e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:42:43.107 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:43.104+0000 7f093de00640 1 -- 172.21.15.83:0/456985686 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0940107350 msgr2=0x7f09401005e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:42:43.107 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:43.104+0000 7f093de00640 1 --2- 172.21.15.83:0/456985686 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0940107350 0x7f09401005e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:43.107 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:43.104+0000 7f093de00640 1 -- 172.21.15.83:0/456985686 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0940104a20 msgr2=0x7f09401000a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:42:43.107 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:43.104+0000 7f093de00640 1 --2- 172.21.15.83:0/456985686 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0940104a20 0x7f09401000a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:43.107 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:43.104+0000 7f093de00640 1 -- 172.21.15.83:0/456985686 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0940100e70 con 0x7f09401020f0 2024-08-19T01:42:43.108 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:43.104+0000 7f093de00640 1 --2- 172.21.15.83:0/456985686 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f09401020f0 0x7f0940101a60 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f0930035040 tx=0x7f0930033d90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:42:43.108 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:43.104+0000 7f0936a00640 1 -- 172.21.15.83:0/456985686 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0930037040 con 0x7f09401020f0 2024-08-19T01:42:43.108 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:43.104+0000 7f0936a00640 1 -- 172.21.15.83:0/456985686 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0930030850 con 0x7f09401020f0 2024-08-19T01:42:43.108 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:43.104+0000 7f093f200640 1 -- 172.21.15.83:0/456985686 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f09401a1b40 con 0x7f09401020f0 2024-08-19T01:42:43.108 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:43.104+0000 7f0936a00640 1 -- 172.21.15.83:0/456985686 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0930004920 con 0x7f09401020f0 2024-08-19T01:42:43.108 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:43.104+0000 7f093f200640 1 -- 172.21.15.83:0/456985686 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f09401a2090 con 0x7f09401020f0 2024-08-19T01:42:43.108 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:43.107+0000 7f09480d0740 1 Processor -- start 2024-08-19T01:42:43.108 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:43.107+0000 7f09480d0740 1 -- start start 2024-08-19T01:42:43.109 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:43.107+0000 7f0936a00640 1 -- 172.21.15.83:0/456985686 <== mon.2 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7f0930004050 con 0x7f09401020f0 2024-08-19T01:42:43.110 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:43.108+0000 7f09480d0740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556df454d680 0x556df454da80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:42:43.110 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:43.108+0000 7f09480d0740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556df454dfc0 0x556df4552a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:42:43.110 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:43.108+0000 7f09480d0740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556df4553050 0x556df4555510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:42:43.110 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:43.108+0000 7f09480d0740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x556df44b1890 con 0x556df454d680 2024-08-19T01:42:43.110 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:43.108+0000 7f09480d0740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x556df4449d90 con 0x556df454dfc0 2024-08-19T01:42:43.111 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:43.108+0000 7f09480d0740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x556df44ad850 con 0x556df4553050 2024-08-19T01:42:43.111 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:43.108+0000 7f093de00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556df454dfc0 0x556df4552a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:42:43.111 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:43.108+0000 7f093e800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556df454d680 0x556df454da80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:42:43.111 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:43.108+0000 7f093de00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556df454dfc0 0x556df4552a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.83:40300/0 (socket says 172.21.15.83:40300) 2024-08-19T01:42:43.111 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:43.108+0000 7f093de00640 1 -- 172.21.15.83:0/2846364845 learned_addr learned my addr 172.21.15.83:0/2846364845 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-08-19T01:42:43.111 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:43.108+0000 7f093d400640 1 --2- 172.21.15.83:0/2846364845 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556df4553050 0x556df4555510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:42:43.111 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:43.108+0000 7f093de00640 1 -- 172.21.15.83:0/2846364845 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556df4553050 msgr2=0x556df4555510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:42:43.111 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:43.108+0000 7f093de00640 1 --2- 172.21.15.83:0/2846364845 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556df4553050 0x556df4555510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:43.111 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:43.108+0000 7f093de00640 1 -- 172.21.15.83:0/2846364845 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556df454d680 msgr2=0x556df454da80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:42:43.111 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:43.108+0000 7f093de00640 1 --2- 172.21.15.83:0/2846364845 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556df454d680 0x556df454da80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:43.111 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:43.108+0000 7f093de00640 1 -- 172.21.15.83:0/2846364845 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556df41c5210 con 0x556df454dfc0 2024-08-19T01:42:43.111 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:43.108+0000 7f093e800640 1 --2- 172.21.15.83:0/2846364845 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556df454d680 0x556df454da80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-19T01:42:43.111 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:43.108+0000 7f093d400640 1 --2- 172.21.15.83:0/2846364845 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556df4553050 0x556df4555510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-19T01:42:43.112 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:43.109+0000 7f093de00640 1 --2- 172.21.15.83:0/2846364845 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556df454dfc0 0x556df4552a80 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f09300f2fc0 tx=0x7f09300f71a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=10e5ccc47a361149 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:42:43.112 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:43.109+0000 7f092b400640 1 -- 172.21.15.83:0/2846364845 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09300f7bd0 con 0x556df454dfc0 2024-08-19T01:42:43.112 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:43.109+0000 7f092b400640 1 -- 172.21.15.83:0/2846364845 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f09300fc030 con 0x556df454dfc0 2024-08-19T01:42:43.112 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:43.109+0000 7f092b400640 1 -- 172.21.15.83:0/2846364845 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0930100630 con 0x556df454dfc0 2024-08-19T01:42:43.112 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:43.109+0000 7f0936a00640 1 --2- 172.21.15.83:0/456985686 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f09240a2b80 0x7f09240a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:42:43.116 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:43.109+0000 7f0936a00640 1 -- 172.21.15.83:0/456985686 <== mon.2 v2:172.21.15.117:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8269+0+0 (secure 0 0 0) 0x7f09300ef0b0 con 0x7f09401020f0 2024-08-19T01:42:43.116 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:43.109+0000 7f093e800640 1 --2- 172.21.15.83:0/456985686 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f09240a2b80 0x7f09240a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:42:43.116 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:43.110+0000 7f09480d0740 1 -- 172.21.15.83:0/2846364845 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556df454dfc0 msgr2=0x556df4552a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:42:43.116 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:43.110+0000 7f09480d0740 1 --2- 172.21.15.83:0/2846364845 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556df454dfc0 0x556df4552a80 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f09300f2fc0 tx=0x7f09300f71a0 comp rx=0 tx=0).stop 2024-08-19T01:42:43.116 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:43.110+0000 7f09480d0740 1 -- 172.21.15.83:0/2846364845 shutdown_connections 2024-08-19T01:42:43.116 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:43.110+0000 7f09480d0740 1 --2- 172.21.15.83:0/2846364845 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556df4553050 0x556df4555510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:43.116 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:43.110+0000 7f09480d0740 1 --2- 172.21.15.83:0/2846364845 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556df454dfc0 0x556df4552a80 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:43.116 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:43.110+0000 7f09480d0740 1 --2- 172.21.15.83:0/2846364845 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556df454d680 0x556df454da80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:43.116 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:43.110+0000 7f09480d0740 1 -- 172.21.15.83:0/2846364845 >> 172.21.15.83:0/2846364845 conn(0x556df4548cf0 msgr2=0x556df45490f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:42:43.116 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:43.110+0000 7f093e800640 1 --2- 172.21.15.83:0/456985686 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f09240a2b80 0x7f09240a5040 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f094006a6b0 tx=0x7f0938011040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:42:43.117 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:43.110+0000 7f09480d0740 1 -- 172.21.15.83:0/2846364845 shutdown_connections 2024-08-19T01:42:43.117 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:43.110+0000 7f09480d0740 1 -- 172.21.15.83:0/2846364845 wait complete. 2024-08-19T01:42:43.117 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:43.110+0000 7f09480d0740 1 -- start start 2024-08-19T01:42:43.117 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:43.111+0000 7f09480d0740 1 Processor -- start 2024-08-19T01:42:43.117 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:43.111+0000 7f09480d0740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556df454d680 0x556df45576c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:42:43.117 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:43.111+0000 7f09480d0740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556df454dfc0 0x556df4557c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:42:43.117 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:43.111+0000 7f09480d0740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556df4553050 0x556df455ec40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:42:43.117 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:43.111+0000 7f09480d0740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x556df4555de0 con 0x556df454dfc0 2024-08-19T01:42:43.117 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:43.111+0000 7f09480d0740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x556df446cf70 con 0x556df4553050 2024-08-19T01:42:43.117 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:43.111+0000 7f09480d0740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x556df4555f60 con 0x556df454d680 2024-08-19T01:42:43.117 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:43.111+0000 7f09480d0740 10 client.0 _ll_register_callbacks cb 0x556df44a98d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-19T01:42:43.117 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:43.111+0000 7f093e800640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556df4553050 0x556df455ec40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:42:43.117 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:43.111+0000 7f093e800640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556df4553050 0x556df455ec40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.83:40304/0 (socket says 172.21.15.83:40304) 2024-08-19T01:42:43.118 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:43.111+0000 7f093d400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556df454dfc0 0x556df4557c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:42:43.118 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:43.111+0000 7f093e800640 1 -- 172.21.15.83:0/3176519744 learned_addr learned my addr 172.21.15.83:0/3176519744 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-08-19T01:42:43.118 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:43.111+0000 7f093de00640 1 --2- 172.21.15.83:0/3176519744 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556df454d680 0x556df45576c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:42:43.118 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:43.112+0000 7f093e800640 1 -- 172.21.15.83:0/3176519744 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556df454d680 msgr2=0x556df45576c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:42:43.118 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:43.112+0000 7f093e800640 1 --2- 172.21.15.83:0/3176519744 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556df454d680 0x556df45576c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:43.118 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:43.112+0000 7f093e800640 1 -- 172.21.15.83:0/3176519744 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556df454dfc0 msgr2=0x556df4557c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:42:43.118 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:43.112+0000 7f093e800640 1 --2- 172.21.15.83:0/3176519744 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556df454dfc0 0x556df4557c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:43.118 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:43.112+0000 7f093e800640 1 -- 172.21.15.83:0/3176519744 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556df4291620 con 0x556df4553050 2024-08-19T01:42:43.118 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:43.112+0000 7f093d400640 1 --2- 172.21.15.83:0/3176519744 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556df454dfc0 0x556df4557c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-19T01:42:43.118 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:43.112+0000 7f093e800640 1 --2- 172.21.15.83:0/3176519744 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556df4553050 0x556df455ec40 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f093800bab0 tx=0x7f0938004ab0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ab86cabf0eb91558 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:42:43.118 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:43.112+0000 7f091fe00640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-08-19T01:42:43.118 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:43.112+0000 7f091fe00640 1 -- 172.21.15.83:0/3176519744 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0938004b20 con 0x556df4553050 2024-08-19T01:42:43.118 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:43.112+0000 7f091fe00640 1 -- 172.21.15.83:0/3176519744 <== mon.2 v2:172.21.15.117:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8269+0+0 (secure 0 0 0) 0x7f0938004cc0 con 0x556df4553050 2024-08-19T01:42:43.119 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:43.112+0000 7f09480d0740 1 -- 172.21.15.83:0/3176519744 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f09300fc030 con 0x556df4553050 2024-08-19T01:42:43.119 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:43.113+0000 7f091fe00640 1 -- 172.21.15.83:0/3176519744 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f093801d930 con 0x556df4553050 2024-08-19T01:42:43.119 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:43.113+0000 7f09480d0740 10 client.26177 fetch_fsmap learned FSMap version 29 2024-08-19T01:42:43.119 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:43.113+0000 7f09480d0740 1 -- 172.21.15.83:0/3176519744 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x556df4561e30 con 0x556df4553050 2024-08-19T01:42:43.119 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:43.113+0000 7f091fe00640 1 -- 172.21.15.83:0/3176519744 <== mon.2 v2:172.21.15.117:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f09300fc030 con 0x556df4553050 2024-08-19T01:42:43.119 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:43.113+0000 7f091fe00640 1 -- 172.21.15.83:0/3176519744 <== mon.2 v2:172.21.15.117:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f093800b040 con 0x556df4553050 2024-08-19T01:42:43.119 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:43.113+0000 7f09480d0740 10 client.26177 fetch_fsmap finished waiting for FSMap version 29 2024-08-19T01:42:43.119 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:43.113+0000 7f09480d0740 10 client.26177 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-08-19T01:42:43.119 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:43.113+0000 7f09480d0740 20 client.26177 populate_metadata read hostname 'smithi083' 2024-08-19T01:42:43.119 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:43.113+0000 7f09480d0740 1 --2- 172.21.15.83:0/3176519744 >> [v2:172.21.15.117:6838/1967595704,v1:172.21.15.117:6839/1967595704] conn(0x556df45635e0 0x556df45659d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:42:43.119 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:43.113+0000 7f09480d0740 4 client.26177 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-19T01:42:43.119 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:43.113+0000 7f09480d0740 1 -- 172.21.15.83:0/3176519744 --> [v2:172.21.15.117:6838/1967595704,v1:172.21.15.117:6839/1967595704] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556df44befc0 con 0x556df45635e0 2024-08-19T01:42:43.119 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:43.113+0000 7f093de00640 1 --2- 172.21.15.83:0/3176519744 >> [v2:172.21.15.117:6838/1967595704,v1:172.21.15.117:6839/1967595704] conn(0x556df45635e0 0x556df45659d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:42:43.120 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:43.114+0000 7f093de00640 1 --2- 172.21.15.83:0/3176519744 >> [v2:172.21.15.117:6838/1967595704,v1:172.21.15.117:6839/1967595704] conn(0x556df45635e0 0x556df45659d0 crc :-1 s=READY pgs=137 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7a7e0d906962a329 server_cookie=78fbb97048a9f05d in_seq=0 out_seq=0 2024-08-19T01:42:43.120 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:43.114+0000 7f091fe00640 10 client.26177 ms_handle_connect on v2:172.21.15.117:6838/1967595704 2024-08-19T01:42:43.121 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:43.115+0000 7f091fe00640 1 -- 172.21.15.83:0/3176519744 <== mds.0 v2:172.21.15.117:6838/1967595704 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x556df44befc0 con 0x556df45635e0 2024-08-19T01:42:43.121 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:43.115+0000 7f091fe00640 10 client.26177 handle_command_reply: tid=0 2024-08-19T01:42:43.121 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:43.116+0000 7f09480d0740 1 client.26177 shutdown 2024-08-19T01:42:43.121 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:43.116+0000 7f09480d0740 1 -- 172.21.15.83:0/3176519744 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556df4553050 msgr2=0x556df455ec40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:42:43.121 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:43.116+0000 7f09480d0740 1 --2- 172.21.15.83:0/3176519744 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556df4553050 0x556df455ec40 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f093800bab0 tx=0x7f0938004ab0 comp rx=0 tx=0).stop 2024-08-19T01:42:43.121 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:43.116+0000 7f09480d0740 1 -- 172.21.15.83:0/3176519744 shutdown_connections 2024-08-19T01:42:43.121 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:43.116+0000 7f09480d0740 1 --2- 172.21.15.83:0/3176519744 >> [v2:172.21.15.117:6838/1967595704,v1:172.21.15.117:6839/1967595704] conn(0x556df45635e0 0x556df45659d0 crc :-1 s=READY pgs=137 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:43.121 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:43.116+0000 7f09480d0740 1 --2- 172.21.15.83:0/3176519744 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556df4553050 0x556df455ec40 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:43.121 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:43.116+0000 7f09480d0740 1 --2- 172.21.15.83:0/3176519744 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556df454dfc0 0x556df4557c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:43.121 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:43.116+0000 7f09480d0740 1 --2- 172.21.15.83:0/3176519744 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556df454d680 0x556df45576c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:43.121 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:43.116+0000 7f091fe00640 0 client.26177 ms_handle_reset on v2:172.21.15.117:6838/1967595704 2024-08-19T01:42:43.121 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:43.116+0000 7f09480d0740 1 -- 172.21.15.83:0/3176519744 >> 172.21.15.83:0/3176519744 conn(0x556df4548cf0 msgr2=0x556df454a500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:42:43.122 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:43.116+0000 7f09480d0740 1 -- 172.21.15.83:0/3176519744 shutdown_connections 2024-08-19T01:42:43.122 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:43.116+0000 7f09480d0740 1 -- 172.21.15.83:0/3176519744 wait complete. 2024-08-19T01:42:43.122 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:43.116+0000 7f09480d0740 20 client.26177 trim_cache size 0 max 16384 2024-08-19T01:42:43.134 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:43.131+0000 7f09480d0740 1 Processor -- start 2024-08-19T01:42:43.134 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:43.132+0000 7f09480d0740 1 -- start start 2024-08-19T01:42:43.134 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:43.132+0000 7f09480d0740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556df454d680 0x556df45576c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:42:43.134 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:43.132+0000 7f09480d0740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556df454dfc0 0x556df455ec00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:42:43.134 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:43.132+0000 7f09480d0740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556df4553050 0x556df4557c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:42:43.134 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:43.132+0000 7f09480d0740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f09401ad840 con 0x556df454dfc0 2024-08-19T01:42:43.134 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:43.132+0000 7f09480d0740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f09401ab0b0 con 0x556df4553050 2024-08-19T01:42:43.134 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:43.132+0000 7f09480d0740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f09401a8f00 con 0x556df454d680 2024-08-19T01:42:43.134 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:43.132+0000 7f093d400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556df454dfc0 0x556df455ec00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:42:43.134 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:43.132+0000 7f093e800640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556df4553050 0x556df4557c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:42:43.134 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:43.132+0000 7f093d400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556df454dfc0 0x556df455ec00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:34948/0 (socket says 172.21.15.83:34948) 2024-08-19T01:42:43.134 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:43.132+0000 7f093d400640 1 -- 172.21.15.83:0/97774846 learned_addr learned my addr 172.21.15.83:0/97774846 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-08-19T01:42:43.135 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:43.132+0000 7f093de00640 1 --2- 172.21.15.83:0/97774846 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556df454d680 0x556df45576c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:42:43.135 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:43.132+0000 7f093d400640 1 -- 172.21.15.83:0/97774846 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556df454d680 msgr2=0x556df45576c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:42:43.135 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:43.132+0000 7f093d400640 1 --2- 172.21.15.83:0/97774846 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556df454d680 0x556df45576c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:43.135 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:43.132+0000 7f093d400640 1 -- 172.21.15.83:0/97774846 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556df4553050 msgr2=0x556df4557c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:42:43.135 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:43.132+0000 7f093d400640 1 --2- 172.21.15.83:0/97774846 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556df4553050 0x556df4557c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:43.135 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:43.132+0000 7f093d400640 1 -- 172.21.15.83:0/97774846 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556df454e4a0 con 0x556df454dfc0 2024-08-19T01:42:43.138 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:43.133+0000 7f093d400640 1 --2- 172.21.15.83:0/97774846 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556df454dfc0 0x556df455ec00 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f092c0125f0 tx=0x7f092c005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4333aa4add14a490 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:42:43.138 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:43.133+0000 7f092aa00640 1 -- 172.21.15.83:0/97774846 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f092c00f5c0 con 0x556df454dfc0 2024-08-19T01:42:43.138 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:43.133+0000 7f092aa00640 1 -- 172.21.15.83:0/97774846 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f092c021070 con 0x556df454dfc0 2024-08-19T01:42:43.138 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:43.133+0000 7f092aa00640 1 -- 172.21.15.83:0/97774846 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f092c010b30 con 0x556df454dfc0 2024-08-19T01:42:43.138 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:43.133+0000 7f09480d0740 1 -- 172.21.15.83:0/97774846 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556df454dfc0 msgr2=0x556df455ec00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:42:43.138 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:43.133+0000 7f09480d0740 1 --2- 172.21.15.83:0/97774846 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556df454dfc0 0x556df455ec00 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f092c0125f0 tx=0x7f092c005f90 comp rx=0 tx=0).stop 2024-08-19T01:42:43.139 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:43.134+0000 7f09480d0740 1 -- 172.21.15.83:0/97774846 shutdown_connections 2024-08-19T01:42:43.139 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:43.134+0000 7f09480d0740 1 --2- 172.21.15.83:0/97774846 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556df4553050 0x556df4557c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:43.139 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:43.134+0000 7f09480d0740 1 --2- 172.21.15.83:0/97774846 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556df454dfc0 0x556df455ec00 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:43.139 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:43.134+0000 7f09480d0740 1 --2- 172.21.15.83:0/97774846 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556df454d680 0x556df45576c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:43.139 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:43.134+0000 7f09480d0740 1 -- 172.21.15.83:0/97774846 >> 172.21.15.83:0/97774846 conn(0x556df4548cf0 msgr2=0x556df454f1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:42:43.139 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:43.134+0000 7f09480d0740 1 -- 172.21.15.83:0/97774846 shutdown_connections 2024-08-19T01:42:43.139 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:43.134+0000 7f09480d0740 1 -- 172.21.15.83:0/97774846 wait complete. 2024-08-19T01:42:43.139 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:43.134+0000 7f09480d0740 1 -- start start 2024-08-19T01:42:43.139 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:43.134+0000 7f09480d0740 1 Processor -- start 2024-08-19T01:42:43.139 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:43.135+0000 7f09480d0740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556df454d680 0x556df455ba40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:42:43.139 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:43.135+0000 7f09480d0740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556df454dfc0 0x556df455bf80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:42:43.139 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:43.135+0000 7f09480d0740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556df4553050 0x556df45d4a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:42:43.139 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:43.135+0000 7f09480d0740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f09401a9570 con 0x556df4553050 2024-08-19T01:42:43.139 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:43.135+0000 7f09480d0740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f09401aa9e0 con 0x556df454dfc0 2024-08-19T01:42:43.140 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:43.135+0000 7f09480d0740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f09401a9c40 con 0x556df454d680 2024-08-19T01:42:43.140 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:43.135+0000 7f09480d0740 10 client.0 _ll_register_callbacks cb 0x7f0938009b50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-19T01:42:43.140 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:43.135+0000 7f093d400640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556df454dfc0 0x556df455bf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:42:43.140 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:43.135+0000 7f093de00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556df454d680 0x556df455ba40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:42:43.140 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:43.135+0000 7f093e800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556df4553050 0x556df45d4a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:42:43.140 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:43.135+0000 7f093de00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556df454d680 0x556df455ba40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.83:49914/0 (socket says 172.21.15.83:49914) 2024-08-19T01:42:43.140 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:43.135+0000 7f093d400640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556df454dfc0 0x556df455bf80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.83:40324/0 (socket says 172.21.15.83:40324) 2024-08-19T01:42:43.140 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:43.135+0000 7f093de00640 1 -- 172.21.15.83:0/3083625989 learned_addr learned my addr 172.21.15.83:0/3083625989 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-08-19T01:42:43.140 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:43.135+0000 7f093de00640 1 -- 172.21.15.83:0/3083625989 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556df454dfc0 msgr2=0x556df455bf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:42:43.140 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:43.135+0000 7f093de00640 1 --2- 172.21.15.83:0/3083625989 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556df454dfc0 0x556df455bf80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:43.140 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:43.135+0000 7f093de00640 1 -- 172.21.15.83:0/3083625989 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556df4553050 msgr2=0x556df45d4a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:42:43.140 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:43.135+0000 7f093de00640 1 --2- 172.21.15.83:0/3083625989 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556df4553050 0x556df45d4a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:43.140 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:43.135+0000 7f093de00640 1 -- 172.21.15.83:0/3083625989 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556df45d4fd0 con 0x556df454d680 2024-08-19T01:42:43.141 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:43.135+0000 7f093de00640 1 --2- 172.21.15.83:0/3083625989 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556df454d680 0x556df455ba40 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f09300b7cf0 tx=0x7f093002a340 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f83382ba1acd0a3 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:42:43.141 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:43.135+0000 7f091fe00640 10 client.0 ms_handle_connect on v2:172.21.15.167:3300/0 2024-08-19T01:42:43.141 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:43.136+0000 7f091fe00640 1 -- 172.21.15.83:0/3083625989 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09300b7b30 con 0x556df454d680 2024-08-19T01:42:43.141 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:43.136+0000 7f091fe00640 1 -- 172.21.15.83:0/3083625989 <== mon.1 v2:172.21.15.167:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8269+0+0 (secure 0 0 0) 0x7f09300b9930 con 0x556df454d680 2024-08-19T01:42:43.141 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:43.136+0000 7f09480d0740 1 -- 172.21.15.83:0/3083625989 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f092c021070 con 0x556df454d680 2024-08-19T01:42:43.141 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:43.136+0000 7f091fe00640 1 -- 172.21.15.83:0/3083625989 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f09300bb1f0 con 0x556df454d680 2024-08-19T01:42:43.141 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:43.136+0000 7f09480d0740 10 client.26119 fetch_fsmap learned FSMap version 29 2024-08-19T01:42:43.141 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:43.136+0000 7f09480d0740 1 -- 172.21.15.83:0/3083625989 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x556df45d6250 con 0x556df454d680 2024-08-19T01:42:43.141 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:43.136+0000 7f091fe00640 1 -- 172.21.15.83:0/3083625989 <== mon.1 v2:172.21.15.167:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f092c021070 con 0x556df454d680 2024-08-19T01:42:43.141 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:43.136+0000 7f091fe00640 1 -- 172.21.15.83:0/3083625989 <== mon.1 v2:172.21.15.167:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f093003a070 con 0x556df454d680 2024-08-19T01:42:43.141 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:43.136+0000 7f09480d0740 10 client.26119 fetch_fsmap finished waiting for FSMap version 29 2024-08-19T01:42:43.141 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:43.136+0000 7f09480d0740 10 client.26119 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-08-19T01:42:43.141 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:43.136+0000 7f09480d0740 20 client.26119 populate_metadata read hostname 'smithi083' 2024-08-19T01:42:43.141 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:43.136+0000 7f09480d0740 1 --2- 172.21.15.83:0/3083625989 >> [v2:172.21.15.117:6838/1967595704,v1:172.21.15.117:6839/1967595704] conn(0x556df45d6880 0x556df45d8ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:42:43.142 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:43.136+0000 7f09480d0740 4 client.26119 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-08-19T01:42:43.142 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:43.136+0000 7f09480d0740 1 -- 172.21.15.83:0/3083625989 --> [v2:172.21.15.117:6838/1967595704,v1:172.21.15.117:6839/1967595704] -- command(tid 0: {"prefix": "flush journal"}) -- 0x556df44b9310 con 0x556df45d6880 2024-08-19T01:42:43.142 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:43.137+0000 7f093d400640 1 --2- 172.21.15.83:0/3083625989 >> [v2:172.21.15.117:6838/1967595704,v1:172.21.15.117:6839/1967595704] conn(0x556df45d6880 0x556df45d8ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:42:43.142 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:43.137+0000 7f093d400640 1 --2- 172.21.15.83:0/3083625989 >> [v2:172.21.15.117:6838/1967595704,v1:172.21.15.117:6839/1967595704] conn(0x556df45d6880 0x556df45d8ca0 crc :-1 s=READY pgs=138 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a9e41748d3db8fe9 server_cookie=fa98dccca9308d72 in_seq=0 out_seq=0 2024-08-19T01:42:43.142 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:43.137+0000 7f091fe00640 10 client.26119 ms_handle_connect on v2:172.21.15.117:6838/1967595704 2024-08-19T01:42:43.559 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833 shell --fsid bad8780e-5dc9-11ef-bccf-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-08-19T01:42:43.603 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:43.601+0000 7f091fe00640 1 -- 172.21.15.83:0/3083625989 <== mds.0 v2:172.21.15.117:6838/1967595704 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x556df44b9310 con 0x556df45d6880 2024-08-19T01:42:43.603 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:43.601+0000 7f091fe00640 10 client.26119 handle_command_reply: tid=0 2024-08-19T01:42:43.603 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:43.601+0000 7f09480d0740 1 client.26119 shutdown 2024-08-19T01:42:43.603 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:43.602+0000 7f09480d0740 1 -- 172.21.15.83:0/3083625989 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556df454d680 msgr2=0x556df455ba40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:42:43.604 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:43.602+0000 7f09480d0740 1 --2- 172.21.15.83:0/3083625989 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556df454d680 0x556df455ba40 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f09300b7cf0 tx=0x7f093002a340 comp rx=0 tx=0).stop 2024-08-19T01:42:43.604 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:43.602+0000 7f09480d0740 1 -- 172.21.15.83:0/3083625989 shutdown_connections 2024-08-19T01:42:43.604 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:43.602+0000 7f09480d0740 1 --2- 172.21.15.83:0/3083625989 >> [v2:172.21.15.117:6838/1967595704,v1:172.21.15.117:6839/1967595704] conn(0x556df45d6880 0x556df45d8ca0 crc :-1 s=READY pgs=138 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:43.605 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:43.602+0000 7f09480d0740 1 --2- 172.21.15.83:0/3083625989 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556df4553050 0x556df45d4a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:43.605 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:43.602+0000 7f09480d0740 1 --2- 172.21.15.83:0/3083625989 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556df454dfc0 0x556df455bf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:43.605 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:43.602+0000 7f09480d0740 1 --2- 172.21.15.83:0/3083625989 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556df454d680 0x556df455ba40 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:43.605 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:43.602+0000 7f09480d0740 1 -- 172.21.15.83:0/3083625989 >> 172.21.15.83:0/3083625989 conn(0x556df4548cf0 msgr2=0x556df45576c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:42:43.605 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:43.602+0000 7f091fe00640 0 client.26119 ms_handle_reset on v2:172.21.15.117:6838/1967595704 2024-08-19T01:42:43.605 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:43.602+0000 7f09480d0740 1 -- 172.21.15.83:0/3083625989 shutdown_connections 2024-08-19T01:42:43.605 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:43.603+0000 7f09480d0740 1 -- 172.21.15.83:0/3083625989 wait complete. 2024-08-19T01:42:43.605 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:43.603+0000 7f09480d0740 20 client.26119 trim_cache size 0 max 16384 2024-08-19T01:42:43.605 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stdout:{ 2024-08-19T01:42:43.605 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stdout: "message": "", 2024-08-19T01:42:43.606 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stdout: "return_code": 0 2024-08-19T01:42:43.606 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stdout:} 2024-08-19T01:42:43.606 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:43.604+0000 7f093f200640 1 -- 172.21.15.83:0/456985686 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f09240a2b80 msgr2=0x7f09240a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:42:43.607 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:43.604+0000 7f093f200640 1 --2- 172.21.15.83:0/456985686 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f09240a2b80 0x7f09240a5040 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f094006a6b0 tx=0x7f0938011040 comp rx=0 tx=0).stop 2024-08-19T01:42:43.607 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:43.604+0000 7f093f200640 1 -- 172.21.15.83:0/456985686 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f09401020f0 msgr2=0x7f0940101a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:42:43.607 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:43.604+0000 7f093f200640 1 --2- 172.21.15.83:0/456985686 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f09401020f0 0x7f0940101a60 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f0930035040 tx=0x7f0930033d90 comp rx=0 tx=0).stop 2024-08-19T01:42:43.607 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:43.604+0000 7f093f200640 1 -- 172.21.15.83:0/456985686 shutdown_connections 2024-08-19T01:42:43.607 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:43.604+0000 7f093f200640 1 --2- 172.21.15.83:0/456985686 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f09240a2b80 0x7f09240a5040 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:43.607 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:43.604+0000 7f093f200640 1 --2- 172.21.15.83:0/456985686 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0940107350 0x7f09401005e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:43.607 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:43.604+0000 7f093f200640 1 --2- 172.21.15.83:0/456985686 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0940104a20 0x7f09401000a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:43.607 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:43.604+0000 7f093f200640 1 --2- 172.21.15.83:0/456985686 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f09401020f0 0x7f0940101a60 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:43.607 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:43.604+0000 7f093f200640 1 -- 172.21.15.83:0/456985686 >> 172.21.15.83:0/456985686 conn(0x7f09400fbcd0 msgr2=0x7f0940102a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:42:43.607 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:43.604+0000 7f093f200640 1 -- 172.21.15.83:0/456985686 shutdown_connections 2024-08-19T01:42:43.607 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:43.604+0000 7f093f200640 1 -- 172.21.15.83:0/456985686 wait complete. 2024-08-19T01:42:43.904 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:42:43 smithi083 ceph-mon[17120]: pgmap v575: 97 pgs: 97 active+clean; 210 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.2 MiB/s wr, 565 op/s 2024-08-19T01:42:43.952 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:42:43 smithi117 ceph-mon[22016]: pgmap v575: 97 pgs: 97 active+clean; 210 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.2 MiB/s wr, 565 op/s 2024-08-19T01:42:43.965 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:42:43 smithi167 ceph-mon[21875]: pgmap v575: 97 pgs: 97 active+clean; 210 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.2 MiB/s wr, 565 op/s 2024-08-19T01:42:44.037 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/bad8780e-5dc9-11ef-bccf-c7b262605968/mon.a/config 2024-08-19T01:42:44.659 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:44.657+0000 7f7f8fe00640 1 -- 172.21.15.83:0/1230909498 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7f8810e0f0 msgr2=0x7f7f88110540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:42:44.659 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:44.657+0000 7f7f8fe00640 1 --2- 172.21.15.83:0/1230909498 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7f8810e0f0 0x7f7f88110540 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f7f80009ca0 tx=0x7f7f8002a950 comp rx=0 tx=0).stop 2024-08-19T01:42:44.659 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:44.657+0000 7f7f8fe00640 1 -- 172.21.15.83:0/1230909498 shutdown_connections 2024-08-19T01:42:44.659 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:44.657+0000 7f7f8fe00640 1 --2- 172.21.15.83:0/1230909498 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7f8810e0f0 0x7f7f88110540 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:44.659 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:44.657+0000 7f7f8fe00640 1 --2- 172.21.15.83:0/1230909498 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7f88074c70 0x7f7f8810dbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:44.659 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:44.657+0000 7f7f8fe00640 1 --2- 172.21.15.83:0/1230909498 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7f88074350 0x7f7f88074730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:44.659 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:44.657+0000 7f7f8fe00640 1 -- 172.21.15.83:0/1230909498 >> 172.21.15.83:0/1230909498 conn(0x7f7f880fd5f0 msgr2=0x7f7f880ffa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:42:44.660 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:44.657+0000 7f7f8fe00640 1 -- 172.21.15.83:0/1230909498 shutdown_connections 2024-08-19T01:42:44.660 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:44.657+0000 7f7f8fe00640 1 -- 172.21.15.83:0/1230909498 wait complete. 2024-08-19T01:42:44.662 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:44.658+0000 7f7f8fe00640 1 Processor -- start 2024-08-19T01:42:44.662 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:44.658+0000 7f7f8fe00640 1 -- start start 2024-08-19T01:42:44.662 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:44.658+0000 7f7f8fe00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7f88074350 0x7f7f8819a340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:42:44.662 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:44.659+0000 7f7f8fe00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7f88074c70 0x7f7f8819a880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:42:44.662 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:44.659+0000 7f7f8fe00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7f8810e0f0 0x7f7f8819ec10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:42:44.662 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:44.659+0000 7f7f8fe00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f7f88113ed0 con 0x7f7f8810e0f0 2024-08-19T01:42:44.662 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:44.659+0000 7f7f8fe00640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f7f88113d50 con 0x7f7f88074c70 2024-08-19T01:42:44.662 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:44.659+0000 7f7f8fe00640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f7f88114050 con 0x7f7f88074350 2024-08-19T01:42:44.662 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:44.659+0000 7f7f8ca00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7f88074c70 0x7f7f8819a880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:42:44.662 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:44.659+0000 7f7f8ca00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7f88074c70 0x7f7f8819a880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.83:40336/0 (socket says 172.21.15.83:40336) 2024-08-19T01:42:44.663 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:44.659+0000 7f7f8d400640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7f88074350 0x7f7f8819a340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:42:44.663 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:44.659+0000 7f7f8ca00640 1 -- 172.21.15.83:0/1063298140 learned_addr learned my addr 172.21.15.83:0/1063298140 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-08-19T01:42:44.663 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:44.659+0000 7f7f8de00640 1 --2- 172.21.15.83:0/1063298140 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7f8810e0f0 0x7f7f8819ec10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:42:44.663 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:44.659+0000 7f7f8ca00640 1 -- 172.21.15.83:0/1063298140 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7f88074350 msgr2=0x7f7f8819a340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:42:44.663 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:44.659+0000 7f7f8ca00640 1 --2- 172.21.15.83:0/1063298140 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7f88074350 0x7f7f8819a340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:44.663 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:44.659+0000 7f7f8ca00640 1 -- 172.21.15.83:0/1063298140 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7f8810e0f0 msgr2=0x7f7f8819ec10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:42:44.663 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:44.659+0000 7f7f8ca00640 1 --2- 172.21.15.83:0/1063298140 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7f8810e0f0 0x7f7f8819ec10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:44.663 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:44.659+0000 7f7f8ca00640 1 -- 172.21.15.83:0/1063298140 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7f8819f2f0 con 0x7f7f88074c70 2024-08-19T01:42:44.663 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:44.659+0000 7f7f8d400640 1 --2- 172.21.15.83:0/1063298140 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7f88074350 0x7f7f8819a340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-19T01:42:44.663 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:44.660+0000 7f7f8ca00640 1 --2- 172.21.15.83:0/1063298140 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7f88074c70 0x7f7f8819a880 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f7f7400e9f0 tx=0x7f7f7400edf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:42:44.663 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:44.660+0000 7f7f86000640 1 -- 172.21.15.83:0/1063298140 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f7400cd80 con 0x7f7f88074c70 2024-08-19T01:42:44.663 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:44.660+0000 7f7f86000640 1 -- 172.21.15.83:0/1063298140 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7f74005500 con 0x7f7f88074c70 2024-08-19T01:42:44.663 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:44.660+0000 7f7f86000640 1 -- 172.21.15.83:0/1063298140 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f74010690 con 0x7f7f88074c70 2024-08-19T01:42:44.663 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:44.660+0000 7f7f8fe00640 1 -- 172.21.15.83:0/1063298140 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7f8819f5e0 con 0x7f7f88074c70 2024-08-19T01:42:44.664 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:44.660+0000 7f7f8fe00640 1 -- 172.21.15.83:0/1063298140 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7f881a6ec0 con 0x7f7f88074c70 2024-08-19T01:42:44.664 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:44.663+0000 7f7f928d5740 1 Processor -- start 2024-08-19T01:42:44.664 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:44.663+0000 7f7f928d5740 1 -- start start 2024-08-19T01:42:44.665 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:44.663+0000 7f7f928d5740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5614560b74f0 0x5614560b78d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:42:44.665 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:44.663+0000 7f7f928d5740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5614560b7e10 0x5614560c2190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:42:44.665 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:44.663+0000 7f7f928d5740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5614560c26d0 0x5614560c4ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:42:44.665 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:44.664+0000 7f7f928d5740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x56145601cbf0 con 0x5614560b7e10 2024-08-19T01:42:44.665 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:44.664+0000 7f7f928d5740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x561455fb4b20 con 0x5614560c26d0 2024-08-19T01:42:44.666 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:44.664+0000 7f7f928d5740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x561456018bb0 con 0x5614560b74f0 2024-08-19T01:42:44.666 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:44.664+0000 7f7f8d400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5614560b7e10 0x5614560c2190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:42:44.666 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:44.664+0000 7f7f86000640 1 -- 172.21.15.83:0/1063298140 <== mon.2 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7f7f740046c0 con 0x7f7f88074c70 2024-08-19T01:42:44.666 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:44.664+0000 7f7f8d400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5614560b7e10 0x5614560c2190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:34998/0 (socket says 172.21.15.83:34998) 2024-08-19T01:42:44.666 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:44.664+0000 7f7f8d400640 1 -- 172.21.15.83:0/3466475801 learned_addr learned my addr 172.21.15.83:0/3466475801 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-08-19T01:42:44.666 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:44.664+0000 7f7f8de00640 1 --2- 172.21.15.83:0/3466475801 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5614560b74f0 0x5614560b78d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:42:44.667 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:44.664+0000 7f7f8ca00640 1 --2- 172.21.15.83:0/3466475801 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5614560c26d0 0x5614560c4ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:42:44.667 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:44.664+0000 7f7f8d400640 1 -- 172.21.15.83:0/3466475801 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5614560b74f0 msgr2=0x5614560b78d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:42:44.668 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:44.664+0000 7f7f8d400640 1 --2- 172.21.15.83:0/3466475801 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5614560b74f0 0x5614560b78d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:44.668 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:44.664+0000 7f7f8d400640 1 -- 172.21.15.83:0/3466475801 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5614560c26d0 msgr2=0x5614560c4ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:42:44.668 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:44.664+0000 7f7f8d400640 1 --2- 172.21.15.83:0/3466475801 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5614560c26d0 0x5614560c4ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:44.668 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:44.664+0000 7f7f8d400640 1 -- 172.21.15.83:0/3466475801 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561455fbb7f0 con 0x5614560b7e10 2024-08-19T01:42:44.668 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:44.664+0000 7f7f8de00640 1 --2- 172.21.15.83:0/3466475801 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5614560b74f0 0x5614560b78d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-19T01:42:44.668 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:44.665+0000 7f7f8d400640 1 --2- 172.21.15.83:0/3466475801 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5614560b7e10 0x5614560c2190 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f7f7c0115f0 tx=0x7f7f7c011ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5d50ea90e16d673b server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:42:44.668 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:44.665+0000 7f7f7a600640 1 -- 172.21.15.83:0/3466475801 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f7c01e070 con 0x5614560b7e10 2024-08-19T01:42:44.668 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:44.665+0000 7f7f7a600640 1 -- 172.21.15.83:0/3466475801 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7f7c00fb20 con 0x5614560b7e10 2024-08-19T01:42:44.668 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:44.665+0000 7f7f7a600640 1 -- 172.21.15.83:0/3466475801 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f7c019be0 con 0x5614560b7e10 2024-08-19T01:42:44.668 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:44.665+0000 7f7f86000640 1 --2- 172.21.15.83:0/1063298140 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f7f700a2b80 0x7f7f700a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:42:44.668 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:44.666+0000 7f7f86000640 1 -- 172.21.15.83:0/1063298140 <== mon.2 v2:172.21.15.117:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8269+0+0 (secure 0 0 0) 0x7f7f740cf460 con 0x7f7f88074c70 2024-08-19T01:42:44.668 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:44.666+0000 7f7f928d5740 1 -- 172.21.15.83:0/3466475801 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5614560b7e10 msgr2=0x5614560c2190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:42:44.668 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:44.666+0000 7f7f928d5740 1 --2- 172.21.15.83:0/3466475801 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5614560b7e10 0x5614560c2190 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f7f7c0115f0 tx=0x7f7f7c011ab0 comp rx=0 tx=0).stop 2024-08-19T01:42:44.669 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:44.666+0000 7f7f8de00640 1 --2- 172.21.15.83:0/1063298140 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f7f700a2b80 0x7f7f700a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:42:44.669 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:44.666+0000 7f7f928d5740 1 -- 172.21.15.83:0/3466475801 shutdown_connections 2024-08-19T01:42:44.669 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:44.666+0000 7f7f928d5740 1 --2- 172.21.15.83:0/3466475801 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5614560c26d0 0x5614560c4ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:44.669 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:44.666+0000 7f7f928d5740 1 --2- 172.21.15.83:0/3466475801 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5614560b7e10 0x5614560c2190 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:44.669 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:44.666+0000 7f7f928d5740 1 --2- 172.21.15.83:0/3466475801 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5614560b74f0 0x5614560b78d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:44.669 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:44.666+0000 7f7f928d5740 1 -- 172.21.15.83:0/3466475801 >> 172.21.15.83:0/3466475801 conn(0x5614560b3a80 msgr2=0x5614560b3e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:42:44.669 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:44.666+0000 7f7f928d5740 1 -- 172.21.15.83:0/3466475801 shutdown_connections 2024-08-19T01:42:44.669 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:44.666+0000 7f7f928d5740 1 -- 172.21.15.83:0/3466475801 wait complete. 2024-08-19T01:42:44.672 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:44.667+0000 7f7f8de00640 1 --2- 172.21.15.83:0/1063298140 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f7f700a2b80 0x7f7f700a5040 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f7f80006010 tx=0x7f7f80005d80 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:42:44.672 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:44.667+0000 7f7f928d5740 1 -- start start 2024-08-19T01:42:44.672 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:44.667+0000 7f7f928d5740 1 Processor -- start 2024-08-19T01:42:44.672 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:44.667+0000 7f7f928d5740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5614560b74f0 0x5614560bf600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:42:44.672 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:44.667+0000 7f7f928d5740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5614560b7e10 0x5614560bfb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:42:44.672 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:44.667+0000 7f7f928d5740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5614560c26d0 0x5614560c0080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:42:44.672 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:44.667+0000 7f7f928d5740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x5614560c52a0 con 0x5614560b74f0 2024-08-19T01:42:44.673 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:44.667+0000 7f7f928d5740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x561455fd7cf0 con 0x5614560b7e10 2024-08-19T01:42:44.673 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:44.667+0000 7f7f928d5740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x5614560c5420 con 0x5614560c26d0 2024-08-19T01:42:44.673 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:44.667+0000 7f7f928d5740 10 client.0 _ll_register_callbacks cb 0x561455fc3e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-19T01:42:44.673 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:44.668+0000 7f7f8d400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5614560b74f0 0x5614560bf600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:42:44.673 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:44.668+0000 7f7f8d400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5614560b74f0 0x5614560bf600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:35004/0 (socket says 172.21.15.83:35004) 2024-08-19T01:42:44.673 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:44.668+0000 7f7f8d400640 1 -- 172.21.15.83:0/1016768223 learned_addr learned my addr 172.21.15.83:0/1016768223 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-08-19T01:42:44.673 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:44.668+0000 7f7f8ca00640 1 --2- 172.21.15.83:0/1016768223 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5614560b7e10 0x5614560bfb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:42:44.673 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:44.668+0000 7f7f8de00640 1 --2- 172.21.15.83:0/1016768223 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5614560c26d0 0x5614560c0080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:42:44.673 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:44.668+0000 7f7f8d400640 1 -- 172.21.15.83:0/1016768223 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5614560c26d0 msgr2=0x5614560c0080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:42:44.673 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:44.668+0000 7f7f8d400640 1 --2- 172.21.15.83:0/1016768223 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5614560c26d0 0x5614560c0080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:44.673 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:44.668+0000 7f7f8d400640 1 -- 172.21.15.83:0/1016768223 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5614560b7e10 msgr2=0x5614560bfb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:42:44.673 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:44.668+0000 7f7f8d400640 1 --2- 172.21.15.83:0/1016768223 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5614560b7e10 0x5614560bfb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:44.673 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:44.668+0000 7f7f8d400640 1 -- 172.21.15.83:0/1016768223 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561455d8e3c0 con 0x5614560b74f0 2024-08-19T01:42:44.674 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:44.668+0000 7f7f8ca00640 1 --2- 172.21.15.83:0/1016768223 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5614560b7e10 0x5614560bfb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-19T01:42:44.674 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:44.668+0000 7f7f8d400640 1 --2- 172.21.15.83:0/1016768223 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5614560b74f0 0x5614560bf600 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f7f7c00f920 tx=0x7f7f7c00f950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b6d9236d77e80ee6 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:42:44.674 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:44.668+0000 7f7f6ea00640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-08-19T01:42:44.674 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:44.669+0000 7f7f6ea00640 1 -- 172.21.15.83:0/1016768223 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f7c01e040 con 0x5614560b74f0 2024-08-19T01:42:44.674 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:44.669+0000 7f7f928d5740 1 -- 172.21.15.83:0/1016768223 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7f7c00fb20 con 0x5614560b74f0 2024-08-19T01:42:44.674 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:44.669+0000 7f7f6ea00640 1 -- 172.21.15.83:0/1016768223 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8269+0+0 (secure 0 0 0) 0x7f7f7c0199c0 con 0x5614560b74f0 2024-08-19T01:42:44.674 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:44.669+0000 7f7f6ea00640 1 -- 172.21.15.83:0/1016768223 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f7f7c019410 con 0x5614560b74f0 2024-08-19T01:42:44.674 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:44.669+0000 7f7f928d5740 10 client.16143 fetch_fsmap learned FSMap version 29 2024-08-19T01:42:44.674 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:44.669+0000 7f7f928d5740 1 -- 172.21.15.83:0/1016768223 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5614560d0fa0 con 0x5614560b74f0 2024-08-19T01:42:44.674 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:44.669+0000 7f7f6ea00640 1 -- 172.21.15.83:0/1016768223 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7f7c00fb20 con 0x5614560b74f0 2024-08-19T01:42:44.674 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:44.669+0000 7f7f6ea00640 1 -- 172.21.15.83:0/1016768223 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f7f7c021070 con 0x5614560b74f0 2024-08-19T01:42:44.674 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:44.670+0000 7f7f928d5740 10 client.16143 fetch_fsmap finished waiting for FSMap version 29 2024-08-19T01:42:44.674 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:44.670+0000 7f7f928d5740 10 client.16143 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-19T01:42:44.675 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:44.670+0000 7f7f928d5740 20 client.16143 populate_metadata read hostname 'smithi083' 2024-08-19T01:42:44.675 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:44.670+0000 7f7f928d5740 1 --2- 172.21.15.83:0/1016768223 >> [v2:172.21.15.117:6838/1967595704,v1:172.21.15.117:6839/1967595704] conn(0x5614560d23b0 0x5614560d47a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:42:44.675 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:44.670+0000 7f7f928d5740 4 client.16143 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-19T01:42:44.675 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:44.670+0000 7f7f928d5740 1 -- 172.21.15.83:0/1016768223 --> [v2:172.21.15.117:6838/1967595704,v1:172.21.15.117:6839/1967595704] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56145602a160 con 0x5614560d23b0 2024-08-19T01:42:44.675 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:44.670+0000 7f7f8ca00640 1 --2- 172.21.15.83:0/1016768223 >> [v2:172.21.15.117:6838/1967595704,v1:172.21.15.117:6839/1967595704] conn(0x5614560d23b0 0x5614560d47a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:42:44.675 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:44.670+0000 7f7f8ca00640 1 --2- 172.21.15.83:0/1016768223 >> [v2:172.21.15.117:6838/1967595704,v1:172.21.15.117:6839/1967595704] conn(0x5614560d23b0 0x5614560d47a0 crc :-1 s=READY pgs=139 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6b8384a960e8cc06 server_cookie=e7c45af879533534 in_seq=0 out_seq=0 2024-08-19T01:42:44.675 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:44.670+0000 7f7f6ea00640 10 client.16143 ms_handle_connect on v2:172.21.15.117:6838/1967595704 2024-08-19T01:42:44.676 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:44.671+0000 7f7f6ea00640 1 -- 172.21.15.83:0/1016768223 <== mds.0 v2:172.21.15.117:6838/1967595704 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x56145602a160 con 0x5614560d23b0 2024-08-19T01:42:44.676 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:44.671+0000 7f7f6ea00640 10 client.16143 handle_command_reply: tid=0 2024-08-19T01:42:44.676 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:44.671+0000 7f7f928d5740 1 client.16143 shutdown 2024-08-19T01:42:44.676 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:44.672+0000 7f7f928d5740 1 -- 172.21.15.83:0/1016768223 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5614560b74f0 msgr2=0x5614560bf600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:42:44.676 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:44.672+0000 7f7f928d5740 1 --2- 172.21.15.83:0/1016768223 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5614560b74f0 0x5614560bf600 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f7f7c00f920 tx=0x7f7f7c00f950 comp rx=0 tx=0).stop 2024-08-19T01:42:44.677 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:44.672+0000 7f7f928d5740 1 -- 172.21.15.83:0/1016768223 shutdown_connections 2024-08-19T01:42:44.677 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:44.672+0000 7f7f928d5740 1 --2- 172.21.15.83:0/1016768223 >> [v2:172.21.15.117:6838/1967595704,v1:172.21.15.117:6839/1967595704] conn(0x5614560d23b0 0x5614560d47a0 crc :-1 s=READY pgs=139 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:44.677 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:44.672+0000 7f7f928d5740 1 --2- 172.21.15.83:0/1016768223 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5614560c26d0 0x5614560c0080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:44.677 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:44.672+0000 7f7f928d5740 1 --2- 172.21.15.83:0/1016768223 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5614560b7e10 0x5614560bfb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:44.677 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:44.672+0000 7f7f928d5740 1 --2- 172.21.15.83:0/1016768223 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5614560b74f0 0x5614560bf600 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:44.677 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:44.672+0000 7f7f928d5740 1 -- 172.21.15.83:0/1016768223 >> 172.21.15.83:0/1016768223 conn(0x5614560b3a80 msgr2=0x5614560b2c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:42:44.677 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:44.672+0000 7f7f6ea00640 0 client.16143 ms_handle_reset on v2:172.21.15.117:6838/1967595704 2024-08-19T01:42:44.677 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:44.672+0000 7f7f928d5740 1 -- 172.21.15.83:0/1016768223 shutdown_connections 2024-08-19T01:42:44.677 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:44.672+0000 7f7f928d5740 1 -- 172.21.15.83:0/1016768223 wait complete. 2024-08-19T01:42:44.677 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:44.672+0000 7f7f928d5740 20 client.16143 trim_cache size 0 max 16384 2024-08-19T01:42:44.689 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:44.687+0000 7f7f928d5740 1 Processor -- start 2024-08-19T01:42:44.689 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:44.687+0000 7f7f928d5740 1 -- start start 2024-08-19T01:42:44.689 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:44.687+0000 7f7f928d5740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5614560b74f0 0x5614560c1570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:42:44.689 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:44.687+0000 7f7f928d5740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5614560b7e10 0x5614560da000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:42:44.689 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:44.687+0000 7f7f928d5740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5614560c26d0 0x5614560da540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:42:44.690 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:44.687+0000 7f7f928d5740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f7f881b1f10 con 0x5614560c26d0 2024-08-19T01:42:44.690 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:44.687+0000 7f7f928d5740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f7f881af780 con 0x5614560b74f0 2024-08-19T01:42:44.690 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:44.687+0000 7f7f928d5740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f7f881ad5d0 con 0x5614560b7e10 2024-08-19T01:42:44.690 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:44.687+0000 7f7f8d400640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5614560b74f0 0x5614560c1570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:42:44.690 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:44.687+0000 7f7f8de00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5614560c26d0 0x5614560da540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:42:44.690 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:44.687+0000 7f7f8d400640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5614560b74f0 0x5614560c1570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.83:40358/0 (socket says 172.21.15.83:40358) 2024-08-19T01:42:44.690 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:44.687+0000 7f7f8d400640 1 -- 172.21.15.83:0/2012489597 learned_addr learned my addr 172.21.15.83:0/2012489597 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-08-19T01:42:44.690 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:44.687+0000 7f7f8ca00640 1 --2- 172.21.15.83:0/2012489597 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5614560b7e10 0x5614560da000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:42:44.690 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:44.688+0000 7f7f8d400640 1 -- 172.21.15.83:0/2012489597 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5614560b7e10 msgr2=0x5614560da000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:42:44.690 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:44.688+0000 7f7f8d400640 1 --2- 172.21.15.83:0/2012489597 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5614560b7e10 0x5614560da000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:44.690 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:44.688+0000 7f7f8d400640 1 -- 172.21.15.83:0/2012489597 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5614560c26d0 msgr2=0x5614560da540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:42:44.691 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:44.688+0000 7f7f8d400640 1 --2- 172.21.15.83:0/2012489597 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5614560c26d0 0x5614560da540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:44.691 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:44.688+0000 7f7f8d400640 1 -- 172.21.15.83:0/2012489597 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5614560d4b10 con 0x5614560b74f0 2024-08-19T01:42:44.691 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:44.688+0000 7f7f8ca00640 1 --2- 172.21.15.83:0/2012489597 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5614560b7e10 0x5614560da000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-19T01:42:44.691 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:44.688+0000 7f7f8d400640 1 --2- 172.21.15.83:0/2012489597 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5614560b74f0 0x5614560c1570 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f7f7c009bd0 tx=0x7f7f7c0039c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a884a21d154e6d6e server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:42:44.693 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:44.688+0000 7f7f79c00640 1 -- 172.21.15.83:0/2012489597 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f7c01e040 con 0x5614560b74f0 2024-08-19T01:42:44.694 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:44.688+0000 7f7f79c00640 1 -- 172.21.15.83:0/2012489597 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7f7c015030 con 0x5614560b74f0 2024-08-19T01:42:44.694 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:44.688+0000 7f7f79c00640 1 -- 172.21.15.83:0/2012489597 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f7c019a50 con 0x5614560b74f0 2024-08-19T01:42:44.694 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:44.689+0000 7f7f928d5740 1 -- 172.21.15.83:0/2012489597 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5614560b74f0 msgr2=0x5614560c1570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:42:44.694 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:44.689+0000 7f7f928d5740 1 --2- 172.21.15.83:0/2012489597 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5614560b74f0 0x5614560c1570 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f7f7c009bd0 tx=0x7f7f7c0039c0 comp rx=0 tx=0).stop 2024-08-19T01:42:44.694 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:44.689+0000 7f7f928d5740 1 -- 172.21.15.83:0/2012489597 shutdown_connections 2024-08-19T01:42:44.694 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:44.689+0000 7f7f928d5740 1 --2- 172.21.15.83:0/2012489597 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5614560c26d0 0x5614560da540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:44.694 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:44.689+0000 7f7f928d5740 1 --2- 172.21.15.83:0/2012489597 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5614560b7e10 0x5614560da000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:44.694 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:44.689+0000 7f7f928d5740 1 --2- 172.21.15.83:0/2012489597 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5614560b74f0 0x5614560c1570 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:44.694 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:44.689+0000 7f7f928d5740 1 -- 172.21.15.83:0/2012489597 >> 172.21.15.83:0/2012489597 conn(0x5614560b3a80 msgr2=0x5614560c0e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:42:44.694 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:44.689+0000 7f7f928d5740 1 -- 172.21.15.83:0/2012489597 shutdown_connections 2024-08-19T01:42:44.695 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:44.689+0000 7f7f928d5740 1 -- 172.21.15.83:0/2012489597 wait complete. 2024-08-19T01:42:44.695 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:44.689+0000 7f7f928d5740 1 -- start start 2024-08-19T01:42:44.695 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:44.690+0000 7f7f928d5740 1 Processor -- start 2024-08-19T01:42:44.695 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:44.690+0000 7f7f928d5740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5614560b74f0 0x5614560cac70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:42:44.695 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:44.690+0000 7f7f928d5740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5614560b7e10 0x5614560c5600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:42:44.695 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:44.690+0000 7f7f928d5740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5614560c26d0 0x5614560c5b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:42:44.695 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:44.690+0000 7f7f928d5740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f7f881adc40 con 0x5614560c26d0 2024-08-19T01:42:44.695 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:44.690+0000 7f7f928d5740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f7f881af0b0 con 0x5614560b7e10 2024-08-19T01:42:44.695 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:44.690+0000 7f7f928d5740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f7f881ae310 con 0x5614560b74f0 2024-08-19T01:42:44.695 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:44.690+0000 7f7f928d5740 10 client.0 _ll_register_callbacks cb 0x7f7f7c011440 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-19T01:42:44.695 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:44.690+0000 7f7f8ca00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5614560b7e10 0x5614560c5600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:42:44.695 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:44.690+0000 7f7f8ca00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5614560b7e10 0x5614560c5600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.83:40362/0 (socket says 172.21.15.83:40362) 2024-08-19T01:42:44.696 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:44.690+0000 7f7f8ca00640 1 -- 172.21.15.83:0/3912083086 learned_addr learned my addr 172.21.15.83:0/3912083086 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-08-19T01:42:44.696 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:44.690+0000 7f7f8de00640 1 --2- 172.21.15.83:0/3912083086 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5614560c26d0 0x5614560c5b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:42:44.696 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:44.690+0000 7f7f8d400640 1 --2- 172.21.15.83:0/3912083086 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5614560b74f0 0x5614560cac70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:42:44.696 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:44.691+0000 7f7f8ca00640 1 -- 172.21.15.83:0/3912083086 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5614560b74f0 msgr2=0x5614560cac70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:42:44.696 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:44.691+0000 7f7f8ca00640 1 --2- 172.21.15.83:0/3912083086 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5614560b74f0 0x5614560cac70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:44.696 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:44.691+0000 7f7f8ca00640 1 -- 172.21.15.83:0/3912083086 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5614560c26d0 msgr2=0x5614560c5b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:42:44.696 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:44.691+0000 7f7f8ca00640 1 --2- 172.21.15.83:0/3912083086 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5614560c26d0 0x5614560c5b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:44.696 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:44.691+0000 7f7f8ca00640 1 -- 172.21.15.83:0/3912083086 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5614560c6200 con 0x5614560b7e10 2024-08-19T01:42:44.696 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:44.691+0000 7f7f8ca00640 1 --2- 172.21.15.83:0/3912083086 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5614560b7e10 0x5614560c5600 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f7f740980a0 tx=0x7f7f7409b660 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4b006d7966262527 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:42:44.696 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:44.691+0000 7f7f6ea00640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-08-19T01:42:44.696 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:44.691+0000 7f7f6ea00640 1 -- 172.21.15.83:0/3912083086 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f7409fff0 con 0x5614560b7e10 2024-08-19T01:42:44.696 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:44.691+0000 7f7f6ea00640 1 -- 172.21.15.83:0/3912083086 <== mon.2 v2:172.21.15.117:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8269+0+0 (secure 0 0 0) 0x7f7f740ccff0 con 0x5614560b7e10 2024-08-19T01:42:44.696 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:44.691+0000 7f7f928d5740 1 -- 172.21.15.83:0/3912083086 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7f7c015030 con 0x5614560b7e10 2024-08-19T01:42:44.696 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:44.691+0000 7f7f6ea00640 1 -- 172.21.15.83:0/3912083086 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f7f740ba070 con 0x5614560b7e10 2024-08-19T01:42:44.697 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:44.691+0000 7f7f928d5740 10 client.26201 fetch_fsmap learned FSMap version 29 2024-08-19T01:42:44.697 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:44.692+0000 7f7f928d5740 1 -- 172.21.15.83:0/3912083086 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x56145613beb0 con 0x5614560b7e10 2024-08-19T01:42:44.697 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:44.692+0000 7f7f6ea00640 1 -- 172.21.15.83:0/3912083086 <== mon.2 v2:172.21.15.117:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7f7c015030 con 0x5614560b7e10 2024-08-19T01:42:44.697 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:44.692+0000 7f7f6ea00640 1 -- 172.21.15.83:0/3912083086 <== mon.2 v2:172.21.15.117:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f7f7409bea0 con 0x5614560b7e10 2024-08-19T01:42:44.697 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:44.692+0000 7f7f928d5740 10 client.26201 fetch_fsmap finished waiting for FSMap version 29 2024-08-19T01:42:44.697 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:44.692+0000 7f7f928d5740 10 client.26201 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-19T01:42:44.697 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:44.692+0000 7f7f928d5740 20 client.26201 populate_metadata read hostname 'smithi083' 2024-08-19T01:42:44.697 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:44.692+0000 7f7f928d5740 1 --2- 172.21.15.83:0/3912083086 >> [v2:172.21.15.117:6838/1967595704,v1:172.21.15.117:6839/1967595704] conn(0x56145613c640 0x56145613ea60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:42:44.697 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:44.692+0000 7f7f928d5740 4 client.26201 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-08-19T01:42:44.697 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:44.692+0000 7f7f928d5740 1 -- 172.21.15.83:0/3912083086 --> [v2:172.21.15.117:6838/1967595704,v1:172.21.15.117:6839/1967595704] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5614560aeff0 con 0x56145613c640 2024-08-19T01:42:44.697 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:44.692+0000 7f7f8d400640 1 --2- 172.21.15.83:0/3912083086 >> [v2:172.21.15.117:6838/1967595704,v1:172.21.15.117:6839/1967595704] conn(0x56145613c640 0x56145613ea60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:42:44.698 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:44.693+0000 7f7f8d400640 1 --2- 172.21.15.83:0/3912083086 >> [v2:172.21.15.117:6838/1967595704,v1:172.21.15.117:6839/1967595704] conn(0x56145613c640 0x56145613ea60 crc :-1 s=READY pgs=140 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=275acd7f57680f60 server_cookie=2cda1cc393c50c51 in_seq=0 out_seq=0 2024-08-19T01:42:44.699 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:44.693+0000 7f7f6ea00640 10 client.26201 ms_handle_connect on v2:172.21.15.117:6838/1967595704 2024-08-19T01:42:44.699 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:44.693+0000 7f7f6ea00640 1 -- 172.21.15.83:0/3912083086 <== mds.0 v2:172.21.15.117:6838/1967595704 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5614560aeff0 con 0x56145613c640 2024-08-19T01:42:44.699 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:44.693+0000 7f7f6ea00640 10 client.26201 handle_command_reply: tid=0 2024-08-19T01:42:44.699 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:44.693+0000 7f7f928d5740 1 client.26201 shutdown 2024-08-19T01:42:44.699 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:44.693+0000 7f7f928d5740 1 -- 172.21.15.83:0/3912083086 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5614560b7e10 msgr2=0x5614560c5600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:42:44.699 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:44.693+0000 7f7f928d5740 1 --2- 172.21.15.83:0/3912083086 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5614560b7e10 0x5614560c5600 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f7f740980a0 tx=0x7f7f7409b660 comp rx=0 tx=0).stop 2024-08-19T01:42:44.699 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:44.693+0000 7f7f928d5740 1 -- 172.21.15.83:0/3912083086 shutdown_connections 2024-08-19T01:42:44.699 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:44.693+0000 7f7f928d5740 1 --2- 172.21.15.83:0/3912083086 >> [v2:172.21.15.117:6838/1967595704,v1:172.21.15.117:6839/1967595704] conn(0x56145613c640 0x56145613ea60 crc :-1 s=READY pgs=140 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:44.699 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:44.693+0000 7f7f928d5740 1 --2- 172.21.15.83:0/3912083086 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5614560c26d0 0x5614560c5b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:44.699 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:44.693+0000 7f7f928d5740 1 --2- 172.21.15.83:0/3912083086 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5614560b7e10 0x5614560c5600 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:44.699 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:44.693+0000 7f7f928d5740 1 --2- 172.21.15.83:0/3912083086 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5614560b74f0 0x5614560cac70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:44.699 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:44.693+0000 7f7f928d5740 1 -- 172.21.15.83:0/3912083086 >> 172.21.15.83:0/3912083086 conn(0x5614560b3a80 msgr2=0x5614560b2c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:42:44.700 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:44.693+0000 7f7f6ea00640 0 client.26201 ms_handle_reset on v2:172.21.15.117:6838/1967595704 2024-08-19T01:42:44.700 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:44.693+0000 7f7f928d5740 1 -- 172.21.15.83:0/3912083086 shutdown_connections 2024-08-19T01:42:44.700 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:44.693+0000 7f7f928d5740 1 -- 172.21.15.83:0/3912083086 wait complete. 2024-08-19T01:42:44.700 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:44.694+0000 7f7f928d5740 20 client.26201 trim_cache size 0 max 16384 2024-08-19T01:42:44.700 INFO:teuthology.orchestra.run.smithi083.stdout:[] 2024-08-19T01:42:44.701 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:44.694+0000 7f7f8fe00640 1 -- 172.21.15.83:0/1063298140 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f7f700a2b80 msgr2=0x7f7f700a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:42:44.701 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:44.694+0000 7f7f8fe00640 1 --2- 172.21.15.83:0/1063298140 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f7f700a2b80 0x7f7f700a5040 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f7f80006010 tx=0x7f7f80005d80 comp rx=0 tx=0).stop 2024-08-19T01:42:44.701 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:44.694+0000 7f7f8fe00640 1 -- 172.21.15.83:0/1063298140 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7f88074c70 msgr2=0x7f7f8819a880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:42:44.701 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:44.694+0000 7f7f8fe00640 1 --2- 172.21.15.83:0/1063298140 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7f88074c70 0x7f7f8819a880 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f7f7400e9f0 tx=0x7f7f7400edf0 comp rx=0 tx=0).stop 2024-08-19T01:42:44.701 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:44.694+0000 7f7f8fe00640 1 -- 172.21.15.83:0/1063298140 shutdown_connections 2024-08-19T01:42:44.701 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:44.694+0000 7f7f8fe00640 1 --2- 172.21.15.83:0/1063298140 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f7f700a2b80 0x7f7f700a5040 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:44.701 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:44.694+0000 7f7f8fe00640 1 --2- 172.21.15.83:0/1063298140 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7f8810e0f0 0x7f7f8819ec10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:44.701 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:44.694+0000 7f7f8fe00640 1 --2- 172.21.15.83:0/1063298140 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7f88074c70 0x7f7f8819a880 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:44.701 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:44.694+0000 7f7f8fe00640 1 --2- 172.21.15.83:0/1063298140 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7f88074350 0x7f7f8819a340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:44.701 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:44.694+0000 7f7f8fe00640 1 -- 172.21.15.83:0/1063298140 >> 172.21.15.83:0/1063298140 conn(0x7f7f880fd5f0 msgr2=0x7f7f880fec70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:42:44.701 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:44.694+0000 7f7f8fe00640 1 -- 172.21.15.83:0/1063298140 shutdown_connections 2024-08-19T01:42:44.702 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:44.695+0000 7f7f8fe00640 1 -- 172.21.15.83:0/1063298140 wait complete. 2024-08-19T01:42:45.904 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:42:45 smithi083 ceph-mon[17120]: pgmap v576: 97 pgs: 97 active+clean; 185 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 2.8 MiB/s wr, 620 op/s 2024-08-19T01:42:45.952 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:42:45 smithi117 ceph-mon[22016]: pgmap v576: 97 pgs: 97 active+clean; 185 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 2.8 MiB/s wr, 620 op/s 2024-08-19T01:42:45.965 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:42:45 smithi167 ceph-mon[21875]: pgmap v576: 97 pgs: 97 active+clean; 185 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 2.8 MiB/s wr, 620 op/s 2024-08-19T01:42:46.563 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-08-19T01:42:46.563 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833 shell --fsid bad8780e-5dc9-11ef-bccf-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-08-19T01:42:47.012 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/bad8780e-5dc9-11ef-bccf-c7b262605968/mon.a/config 2024-08-19T01:42:47.578 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:47.576+0000 7f1b76a00640 1 -- 172.21.15.83:0/1368805488 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1b701042a0 msgr2=0x7f1b701107d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:42:47.579 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:47.576+0000 7f1b76a00640 1 --2- 172.21.15.83:0/1368805488 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1b701042a0 0x7f1b701107d0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f1b68009ca0 tx=0x7f1b6802a8c0 comp rx=0 tx=0).stop 2024-08-19T01:42:47.579 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:47.576+0000 7f1b76a00640 1 -- 172.21.15.83:0/1368805488 shutdown_connections 2024-08-19T01:42:47.579 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:47.576+0000 7f1b76a00640 1 --2- 172.21.15.83:0/1368805488 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1b701042a0 0x7f1b701107d0 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:47.579 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:47.576+0000 7f1b76a00640 1 --2- 172.21.15.83:0/1368805488 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1b70103900 0x7f1b70103d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:47.579 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:47.576+0000 7f1b76a00640 1 --2- 172.21.15.83:0/1368805488 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1b70109900 0x7f1b70109ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:47.579 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:47.576+0000 7f1b76a00640 1 -- 172.21.15.83:0/1368805488 >> 172.21.15.83:0/1368805488 conn(0x7f1b700ff640 msgr2=0x7f1b70101a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:42:47.579 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:47.577+0000 7f1b76a00640 1 -- 172.21.15.83:0/1368805488 shutdown_connections 2024-08-19T01:42:47.579 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:47.577+0000 7f1b76a00640 1 -- 172.21.15.83:0/1368805488 wait complete. 2024-08-19T01:42:47.581 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:47.577+0000 7f1b76a00640 1 Processor -- start 2024-08-19T01:42:47.581 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:47.578+0000 7f1b76a00640 1 -- start start 2024-08-19T01:42:47.581 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:47.578+0000 7f1b76a00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1b70103900 0x7f1b7019a170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:42:47.581 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:47.578+0000 7f1b76a00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1b701042a0 0x7f1b7019a6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:42:47.581 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:47.578+0000 7f1b76a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1b70109900 0x7f1b7019ea40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:42:47.581 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:47.578+0000 7f1b76a00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f1b70113d50 con 0x7f1b70109900 2024-08-19T01:42:47.581 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:47.578+0000 7f1b76a00640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f1b70113bd0 con 0x7f1b70103900 2024-08-19T01:42:47.581 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:47.578+0000 7f1b76a00640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f1b70113ed0 con 0x7f1b701042a0 2024-08-19T01:42:47.581 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:47.578+0000 7f1b75600640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1b70103900 0x7f1b7019a170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:42:47.582 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:47.578+0000 7f1b75600640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1b70103900 0x7f1b7019a170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.83:40374/0 (socket says 172.21.15.83:40374) 2024-08-19T01:42:47.582 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:47.578+0000 7f1b75600640 1 -- 172.21.15.83:0/2468689558 learned_addr learned my addr 172.21.15.83:0/2468689558 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-08-19T01:42:47.582 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:47.578+0000 7f1b76000640 1 --2- 172.21.15.83:0/2468689558 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1b70109900 0x7f1b7019ea40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:42:47.582 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:47.578+0000 7f1b74c00640 1 --2- 172.21.15.83:0/2468689558 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1b701042a0 0x7f1b7019a6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:42:47.582 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:47.579+0000 7f1b75600640 1 -- 172.21.15.83:0/2468689558 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1b701042a0 msgr2=0x7f1b7019a6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:42:47.582 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:47.579+0000 7f1b75600640 1 --2- 172.21.15.83:0/2468689558 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1b701042a0 0x7f1b7019a6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:47.582 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:47.579+0000 7f1b75600640 1 -- 172.21.15.83:0/2468689558 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1b70109900 msgr2=0x7f1b7019ea40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:42:47.582 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:47.579+0000 7f1b75600640 1 --2- 172.21.15.83:0/2468689558 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1b70109900 0x7f1b7019ea40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:47.582 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:47.579+0000 7f1b75600640 1 -- 172.21.15.83:0/2468689558 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1b7019f120 con 0x7f1b70103900 2024-08-19T01:42:47.582 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:47.579+0000 7f1b76000640 1 --2- 172.21.15.83:0/2468689558 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1b70109900 0x7f1b7019ea40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-19T01:42:47.583 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:47.579+0000 7f1b75600640 1 --2- 172.21.15.83:0/2468689558 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1b70103900 0x7f1b7019a170 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f1b6400e9f0 tx=0x7f1b6400edf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:42:47.583 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:47.579+0000 7f1b6e000640 1 -- 172.21.15.83:0/2468689558 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b6400cd80 con 0x7f1b70103900 2024-08-19T01:42:47.583 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:47.579+0000 7f1b76a00640 1 -- 172.21.15.83:0/2468689558 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1b7019f410 con 0x7f1b70103900 2024-08-19T01:42:47.583 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:47.579+0000 7f1b6e000640 1 -- 172.21.15.83:0/2468689558 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1b64005500 con 0x7f1b70103900 2024-08-19T01:42:47.583 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:47.579+0000 7f1b6e000640 1 -- 172.21.15.83:0/2468689558 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b64010690 con 0x7f1b70103900 2024-08-19T01:42:47.583 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:47.579+0000 7f1b76a00640 1 -- 172.21.15.83:0/2468689558 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1b701a6cf0 con 0x7f1b70103900 2024-08-19T01:42:47.584 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:47.582+0000 7f1b7aadb740 1 Processor -- start 2024-08-19T01:42:47.584 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:47.582+0000 7f1b6e000640 1 -- 172.21.15.83:0/2468689558 <== mon.2 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7f1b640046c0 con 0x7f1b70103900 2024-08-19T01:42:47.584 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:47.583+0000 7f1b7aadb740 1 -- start start 2024-08-19T01:42:47.585 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:47.583+0000 7f1b7aadb740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cefaae8570 0x55cefaae8950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:42:47.585 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:47.583+0000 7f1b7aadb740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55cefaae8e90 0x55cefaafbb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:42:47.585 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:47.583+0000 7f1b7aadb740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55cefaafc040 0x55cefaafe430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:42:47.585 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:47.583+0000 7f1b7aadb740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55cefaa037e0 con 0x55cefaae8570 2024-08-19T01:42:47.585 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:47.583+0000 7f1b7aadb740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55cefaa4eac0 con 0x55cefaae8e90 2024-08-19T01:42:47.585 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:47.583+0000 7f1b7aadb740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55cefa9e68c0 con 0x55cefaafc040 2024-08-19T01:42:47.586 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:47.583+0000 7f1b75600640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55cefaae8e90 0x55cefaafbb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:42:47.586 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:47.583+0000 7f1b76000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cefaae8570 0x55cefaae8950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:42:47.586 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:47.583+0000 7f1b75600640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55cefaae8e90 0x55cefaafbb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.83:40376/0 (socket says 172.21.15.83:40376) 2024-08-19T01:42:47.587 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:47.583+0000 7f1b75600640 1 -- 172.21.15.83:0/959654886 learned_addr learned my addr 172.21.15.83:0/959654886 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-08-19T01:42:47.587 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:47.584+0000 7f1b75600640 1 -- 172.21.15.83:0/959654886 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55cefaafc040 msgr2=0x55cefaafe430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:42:47.587 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:47.584+0000 7f1b75600640 1 --2- 172.21.15.83:0/959654886 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55cefaafc040 0x55cefaafe430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:47.587 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:47.584+0000 7f1b75600640 1 -- 172.21.15.83:0/959654886 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cefaae8570 msgr2=0x55cefaae8950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:42:47.587 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:47.584+0000 7f1b75600640 1 --2- 172.21.15.83:0/959654886 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cefaae8570 0x55cefaae8950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:47.587 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:47.584+0000 7f1b75600640 1 -- 172.21.15.83:0/959654886 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cefa9ed760 con 0x55cefaae8e90 2024-08-19T01:42:47.587 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:47.584+0000 7f1b6e000640 1 --2- 172.21.15.83:0/2468689558 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f1b580a2b80 0x7f1b580a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:42:47.587 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:47.585+0000 7f1b6e000640 1 -- 172.21.15.83:0/2468689558 <== mon.2 v2:172.21.15.117:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8269+0+0 (secure 0 0 0) 0x7f1b640cf5c0 con 0x7f1b70103900 2024-08-19T01:42:47.587 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:47.585+0000 7f1b75600640 1 --2- 172.21.15.83:0/959654886 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55cefaae8e90 0x55cefaafbb00 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f1b640d5530 tx=0x7f1b640d5620 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c515f20b3336f2e server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:42:47.587 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:47.585+0000 7f1b76000640 1 --2- 172.21.15.83:0/2468689558 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f1b580a2b80 0x7f1b580a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:42:47.587 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:47.585+0000 7f1b62600640 1 -- 172.21.15.83:0/959654886 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b64098040 con 0x55cefaae8e90 2024-08-19T01:42:47.587 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:47.585+0000 7f1b62600640 1 -- 172.21.15.83:0/959654886 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1b640d5e50 con 0x55cefaae8e90 2024-08-19T01:42:47.587 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:47.585+0000 7f1b62600640 1 -- 172.21.15.83:0/959654886 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b6409b700 con 0x55cefaae8e90 2024-08-19T01:42:47.589 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:47.586+0000 7f1b7aadb740 1 -- 172.21.15.83:0/959654886 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55cefaae8e90 msgr2=0x55cefaafbb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:42:47.589 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:47.586+0000 7f1b76000640 1 --2- 172.21.15.83:0/2468689558 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f1b580a2b80 0x7f1b580a5040 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f1b68009430 tx=0x7f1b68004a40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:42:47.589 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:47.586+0000 7f1b7aadb740 1 --2- 172.21.15.83:0/959654886 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55cefaae8e90 0x55cefaafbb00 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f1b640d5530 tx=0x7f1b640d5620 comp rx=0 tx=0).stop 2024-08-19T01:42:47.589 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:47.586+0000 7f1b7aadb740 1 -- 172.21.15.83:0/959654886 shutdown_connections 2024-08-19T01:42:47.590 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:47.586+0000 7f1b7aadb740 1 --2- 172.21.15.83:0/959654886 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55cefaafc040 0x55cefaafe430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:47.590 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:47.586+0000 7f1b7aadb740 1 --2- 172.21.15.83:0/959654886 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55cefaae8e90 0x55cefaafbb00 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:47.590 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:47.586+0000 7f1b7aadb740 1 --2- 172.21.15.83:0/959654886 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cefaae8570 0x55cefaae8950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:47.590 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:47.586+0000 7f1b7aadb740 1 -- 172.21.15.83:0/959654886 >> 172.21.15.83:0/959654886 conn(0x55cefaae4be0 msgr2=0x55cefaae4fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:42:47.590 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:47.586+0000 7f1b7aadb740 1 -- 172.21.15.83:0/959654886 shutdown_connections 2024-08-19T01:42:47.590 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:47.586+0000 7f1b7aadb740 1 -- 172.21.15.83:0/959654886 wait complete. 2024-08-19T01:42:47.590 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:47.586+0000 7f1b7aadb740 1 -- start start 2024-08-19T01:42:47.590 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:47.587+0000 7f1b7aadb740 1 Processor -- start 2024-08-19T01:42:47.590 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:47.587+0000 7f1b7aadb740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55cefaae8570 0x55cefaaf8f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:42:47.590 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:47.587+0000 7f1b7aadb740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cefaae8e90 0x55cefaaf94b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:42:47.591 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:47.587+0000 7f1b7aadb740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55cefaafc040 0x55cefaaf99f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:42:47.591 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:47.587+0000 7f1b7aadb740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55cefaa09ae0 con 0x55cefaae8e90 2024-08-19T01:42:47.591 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:47.587+0000 7f1b7aadb740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55cefaa4aa80 con 0x55cefaae8570 2024-08-19T01:42:47.591 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:47.587+0000 7f1b7aadb740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55cefaafec50 con 0x55cefaafc040 2024-08-19T01:42:47.591 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:47.587+0000 7f1b7aadb740 10 client.0 _ll_register_callbacks cb 0x55cefaa0b190 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-19T01:42:47.591 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:47.587+0000 7f1b75600640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55cefaae8570 0x55cefaaf8f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:42:47.591 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:47.587+0000 7f1b75600640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55cefaae8570 0x55cefaaf8f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.83:40388/0 (socket says 172.21.15.83:40388) 2024-08-19T01:42:47.591 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:47.587+0000 7f1b75600640 1 -- 172.21.15.83:0/2143163016 learned_addr learned my addr 172.21.15.83:0/2143163016 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-08-19T01:42:47.591 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:47.587+0000 7f1b74c00640 1 --2- 172.21.15.83:0/2143163016 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cefaae8e90 0x55cefaaf94b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:42:47.591 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:47.588+0000 7f1b76000640 1 --2- 172.21.15.83:0/2143163016 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55cefaafc040 0x55cefaaf99f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:42:47.591 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:47.588+0000 7f1b75600640 1 -- 172.21.15.83:0/2143163016 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55cefaafc040 msgr2=0x55cefaaf99f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:42:47.591 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:47.588+0000 7f1b75600640 1 --2- 172.21.15.83:0/2143163016 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55cefaafc040 0x55cefaaf99f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:47.591 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:47.588+0000 7f1b75600640 1 -- 172.21.15.83:0/2143163016 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cefaae8e90 msgr2=0x55cefaaf94b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:42:47.592 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:47.588+0000 7f1b75600640 1 --2- 172.21.15.83:0/2143163016 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cefaae8e90 0x55cefaaf94b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:47.592 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:47.588+0000 7f1b75600640 1 -- 172.21.15.83:0/2143163016 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cefa7c06a0 con 0x55cefaae8570 2024-08-19T01:42:47.592 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:47.588+0000 7f1b76000640 1 --2- 172.21.15.83:0/2143163016 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55cefaafc040 0x55cefaaf99f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-19T01:42:47.592 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:47.588+0000 7f1b74c00640 1 --2- 172.21.15.83:0/2143163016 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cefaae8e90 0x55cefaaf94b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-19T01:42:47.592 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:47.588+0000 7f1b75600640 1 --2- 172.21.15.83:0/2143163016 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55cefaae8570 0x55cefaaf8f70 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f1b640d5500 tx=0x7f1b640d2b30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c40ce16f431850ac server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:42:47.592 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:47.588+0000 7f1b56a00640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-08-19T01:42:47.594 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:47.588+0000 7f1b56a00640 1 -- 172.21.15.83:0/2143163016 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b6409f030 con 0x55cefaae8570 2024-08-19T01:42:47.594 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:47.588+0000 7f1b7aadb740 1 -- 172.21.15.83:0/2143163016 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1b640d5e50 con 0x55cefaae8570 2024-08-19T01:42:47.594 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:47.588+0000 7f1b56a00640 1 -- 172.21.15.83:0/2143163016 <== mon.2 v2:172.21.15.117:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8269+0+0 (secure 0 0 0) 0x7f1b6409c000 con 0x55cefaae8570 2024-08-19T01:42:47.594 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:47.589+0000 7f1b56a00640 1 -- 172.21.15.83:0/2143163016 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f1b6409b8e0 con 0x55cefaae8570 2024-08-19T01:42:47.595 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:47.589+0000 7f1b7aadb740 10 client.26213 fetch_fsmap learned FSMap version 29 2024-08-19T01:42:47.595 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:47.589+0000 7f1b7aadb740 1 -- 172.21.15.83:0/2143163016 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55cefab0a820 con 0x55cefaae8570 2024-08-19T01:42:47.595 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:47.589+0000 7f1b56a00640 1 -- 172.21.15.83:0/2143163016 <== mon.2 v2:172.21.15.117:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1b640d5e50 con 0x55cefaae8570 2024-08-19T01:42:47.595 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:47.589+0000 7f1b56a00640 1 -- 172.21.15.83:0/2143163016 <== mon.2 v2:172.21.15.117:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f1b640a4580 con 0x55cefaae8570 2024-08-19T01:42:47.595 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:47.589+0000 7f1b7aadb740 10 client.26213 fetch_fsmap finished waiting for FSMap version 29 2024-08-19T01:42:47.595 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:47.589+0000 7f1b7aadb740 10 client.26213 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-19T01:42:47.595 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:47.589+0000 7f1b7aadb740 20 client.26213 populate_metadata read hostname 'smithi083' 2024-08-19T01:42:47.595 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:47.589+0000 7f1b7aadb740 1 --2- 172.21.15.83:0/2143163016 >> [v2:172.21.15.117:6838/1967595704,v1:172.21.15.117:6839/1967595704] conn(0x55cefab0bb70 0x55cefab0df60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:42:47.595 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:47.589+0000 7f1b7aadb740 4 client.26213 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-19T01:42:47.595 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:47.589+0000 7f1b7aadb740 1 -- 172.21.15.83:0/2143163016 --> [v2:172.21.15.117:6838/1967595704,v1:172.21.15.117:6839/1967595704] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55cefaa5c030 con 0x55cefab0bb70 2024-08-19T01:42:47.595 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:47.590+0000 7f1b74c00640 1 --2- 172.21.15.83:0/2143163016 >> [v2:172.21.15.117:6838/1967595704,v1:172.21.15.117:6839/1967595704] conn(0x55cefab0bb70 0x55cefab0df60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:42:47.595 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:47.590+0000 7f1b74c00640 1 --2- 172.21.15.83:0/2143163016 >> [v2:172.21.15.117:6838/1967595704,v1:172.21.15.117:6839/1967595704] conn(0x55cefab0bb70 0x55cefab0df60 crc :-1 s=READY pgs=141 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b16b904353ed0131 server_cookie=3cd4de5c83525def in_seq=0 out_seq=0 2024-08-19T01:42:47.595 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:47.590+0000 7f1b56a00640 10 client.26213 ms_handle_connect on v2:172.21.15.117:6838/1967595704 2024-08-19T01:42:47.596 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:47.592+0000 7f1b56a00640 1 -- 172.21.15.83:0/2143163016 <== mds.0 v2:172.21.15.117:6838/1967595704 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55cefaa5c030 con 0x55cefab0bb70 2024-08-19T01:42:47.596 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:47.592+0000 7f1b56a00640 10 client.26213 handle_command_reply: tid=0 2024-08-19T01:42:47.596 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:47.592+0000 7f1b7aadb740 1 client.26213 shutdown 2024-08-19T01:42:47.596 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:47.592+0000 7f1b7aadb740 1 -- 172.21.15.83:0/2143163016 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55cefaae8570 msgr2=0x55cefaaf8f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:42:47.596 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:47.592+0000 7f1b7aadb740 1 --2- 172.21.15.83:0/2143163016 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55cefaae8570 0x55cefaaf8f70 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f1b640d5500 tx=0x7f1b640d2b30 comp rx=0 tx=0).stop 2024-08-19T01:42:47.596 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:47.592+0000 7f1b7aadb740 1 -- 172.21.15.83:0/2143163016 shutdown_connections 2024-08-19T01:42:47.596 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:47.592+0000 7f1b7aadb740 1 --2- 172.21.15.83:0/2143163016 >> [v2:172.21.15.117:6838/1967595704,v1:172.21.15.117:6839/1967595704] conn(0x55cefab0bb70 0x55cefab0df60 crc :-1 s=READY pgs=141 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:47.596 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:47.592+0000 7f1b7aadb740 1 --2- 172.21.15.83:0/2143163016 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55cefaafc040 0x55cefaaf99f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:47.596 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:47.592+0000 7f1b7aadb740 1 --2- 172.21.15.83:0/2143163016 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cefaae8e90 0x55cefaaf94b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:47.596 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:47.592+0000 7f1b7aadb740 1 --2- 172.21.15.83:0/2143163016 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55cefaae8570 0x55cefaaf8f70 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:47.596 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:47.592+0000 7f1b7aadb740 1 -- 172.21.15.83:0/2143163016 >> 172.21.15.83:0/2143163016 conn(0x55cefaae4be0 msgr2=0x55cefaae3d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:42:47.596 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:47.592+0000 7f1b56a00640 0 client.26213 ms_handle_reset on v2:172.21.15.117:6838/1967595704 2024-08-19T01:42:47.596 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:47.592+0000 7f1b7aadb740 1 -- 172.21.15.83:0/2143163016 shutdown_connections 2024-08-19T01:42:47.597 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:47.592+0000 7f1b7aadb740 1 -- 172.21.15.83:0/2143163016 wait complete. 2024-08-19T01:42:47.597 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:47.593+0000 7f1b7aadb740 20 client.26213 trim_cache size 0 max 16384 2024-08-19T01:42:47.612 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:47.610+0000 7f1b7aadb740 1 Processor -- start 2024-08-19T01:42:47.612 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:47.611+0000 7f1b7aadb740 1 -- start start 2024-08-19T01:42:47.612 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:47.611+0000 7f1b7aadb740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cefaae8570 0x55cefaae3c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:42:47.613 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:47.611+0000 7f1b7aadb740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55cefaae8e90 0x55cefab10ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:42:47.613 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:47.611+0000 7f1b7aadb740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55cefaafc040 0x55cefab110e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:42:47.613 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:47.611+0000 7f1b7aadb740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f1b701ad920 con 0x55cefaae8570 2024-08-19T01:42:47.613 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:47.611+0000 7f1b7aadb740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f1b701ab190 con 0x55cefaafc040 2024-08-19T01:42:47.614 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:47.611+0000 7f1b7aadb740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f1b701a9080 con 0x55cefaae8e90 2024-08-19T01:42:47.614 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:47.611+0000 7f1b75600640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cefaae8570 0x55cefaae3c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:42:47.614 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:47.611+0000 7f1b76000640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55cefaafc040 0x55cefab110e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:42:47.614 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:47.611+0000 7f1b75600640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cefaae8570 0x55cefaae3c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:35088/0 (socket says 172.21.15.83:35088) 2024-08-19T01:42:47.614 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:47.611+0000 7f1b75600640 1 -- 172.21.15.83:0/628607441 learned_addr learned my addr 172.21.15.83:0/628607441 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-08-19T01:42:47.614 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:47.611+0000 7f1b74c00640 1 --2- 172.21.15.83:0/628607441 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55cefaae8e90 0x55cefab10ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:42:47.614 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:47.612+0000 7f1b75600640 1 -- 172.21.15.83:0/628607441 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55cefaae8e90 msgr2=0x55cefab10ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:42:47.614 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:47.612+0000 7f1b75600640 1 --2- 172.21.15.83:0/628607441 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55cefaae8e90 0x55cefab10ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:47.614 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:47.612+0000 7f1b75600640 1 -- 172.21.15.83:0/628607441 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55cefaafc040 msgr2=0x55cefab110e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:42:47.614 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:47.612+0000 7f1b75600640 1 --2- 172.21.15.83:0/628607441 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55cefaafc040 0x55cefab110e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:47.614 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:47.612+0000 7f1b75600640 1 -- 172.21.15.83:0/628607441 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cefaaf7f20 con 0x55cefaae8570 2024-08-19T01:42:47.614 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:47.612+0000 7f1b75600640 1 --2- 172.21.15.83:0/628607441 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cefaae8570 0x55cefaae3c30 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f1b64098200 tx=0x7f1b640d2c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=85b23ba05a0a70ee server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:42:47.615 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:47.612+0000 7f1b61c00640 1 -- 172.21.15.83:0/628607441 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b64066070 con 0x55cefaae8570 2024-08-19T01:42:47.615 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:47.612+0000 7f1b61c00640 1 -- 172.21.15.83:0/628607441 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1b640d5e50 con 0x55cefaae8570 2024-08-19T01:42:47.616 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:47.612+0000 7f1b61c00640 1 -- 172.21.15.83:0/628607441 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b6409f070 con 0x55cefaae8570 2024-08-19T01:42:47.616 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:47.613+0000 7f1b7aadb740 1 -- 172.21.15.83:0/628607441 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cefaae8570 msgr2=0x55cefaae3c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:42:47.616 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:47.613+0000 7f1b7aadb740 1 --2- 172.21.15.83:0/628607441 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cefaae8570 0x55cefaae3c30 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f1b64098200 tx=0x7f1b640d2c20 comp rx=0 tx=0).stop 2024-08-19T01:42:47.616 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:47.613+0000 7f1b7aadb740 1 -- 172.21.15.83:0/628607441 shutdown_connections 2024-08-19T01:42:47.616 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:47.613+0000 7f1b7aadb740 1 --2- 172.21.15.83:0/628607441 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55cefaafc040 0x55cefab110e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:47.616 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:47.613+0000 7f1b7aadb740 1 --2- 172.21.15.83:0/628607441 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55cefaae8e90 0x55cefab10ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:47.616 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:47.613+0000 7f1b7aadb740 1 --2- 172.21.15.83:0/628607441 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cefaae8570 0x55cefaae3c30 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:47.616 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:47.613+0000 7f1b7aadb740 1 -- 172.21.15.83:0/628607441 >> 172.21.15.83:0/628607441 conn(0x55cefaae4be0 msgr2=0x55cefaae3520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:42:47.616 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:47.613+0000 7f1b7aadb740 1 -- 172.21.15.83:0/628607441 shutdown_connections 2024-08-19T01:42:47.616 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:47.614+0000 7f1b7aadb740 1 -- 172.21.15.83:0/628607441 wait complete. 2024-08-19T01:42:47.616 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:47.614+0000 7f1b7aadb740 1 -- start start 2024-08-19T01:42:47.616 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:47.614+0000 7f1b7aadb740 1 Processor -- start 2024-08-19T01:42:47.620 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:47.615+0000 7f1b7aadb740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cefaae8570 0x55cefab0e1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:42:47.621 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:47.615+0000 7f1b7aadb740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55cefaae8e90 0x55cefab0e730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:42:47.621 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:47.615+0000 7f1b7aadb740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55cefaafc040 0x55cefab6e4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:42:47.621 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:47.615+0000 7f1b7aadb740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f1b701a96d0 con 0x55cefaae8570 2024-08-19T01:42:47.621 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:47.615+0000 7f1b7aadb740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f1b701aaac0 con 0x55cefaafc040 2024-08-19T01:42:47.621 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:47.615+0000 7f1b7aadb740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f1b701a9d20 con 0x55cefaae8e90 2024-08-19T01:42:47.622 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:47.615+0000 7f1b7aadb740 10 client.0 _ll_register_callbacks cb 0x7f1b5c005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-19T01:42:47.622 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:47.615+0000 7f1b74c00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55cefaae8e90 0x55cefab0e730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:42:47.622 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:47.615+0000 7f1b75600640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cefaae8570 0x55cefab0e1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:42:47.622 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:47.615+0000 7f1b74c00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55cefaae8e90 0x55cefab0e730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.83:50032/0 (socket says 172.21.15.83:50032) 2024-08-19T01:42:47.622 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:47.615+0000 7f1b74c00640 1 -- 172.21.15.83:0/3654508411 learned_addr learned my addr 172.21.15.83:0/3654508411 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-08-19T01:42:47.622 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:47.615+0000 7f1b76000640 1 --2- 172.21.15.83:0/3654508411 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55cefaafc040 0x55cefab6e4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:42:47.623 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:47.615+0000 7f1b75600640 1 -- 172.21.15.83:0/3654508411 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55cefaae8e90 msgr2=0x55cefab0e730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:42:47.623 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:47.615+0000 7f1b75600640 1 --2- 172.21.15.83:0/3654508411 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55cefaae8e90 0x55cefab0e730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:47.623 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:47.615+0000 7f1b75600640 1 -- 172.21.15.83:0/3654508411 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55cefaafc040 msgr2=0x55cefab6e4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:42:47.623 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:47.615+0000 7f1b75600640 1 --2- 172.21.15.83:0/3654508411 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55cefaafc040 0x55cefab6e4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:47.623 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:47.616+0000 7f1b75600640 1 -- 172.21.15.83:0/3654508411 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cefab6eb50 con 0x55cefaae8570 2024-08-19T01:42:47.623 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:47.616+0000 7f1b74c00640 1 --2- 172.21.15.83:0/3654508411 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55cefaae8e90 0x55cefab0e730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-19T01:42:47.624 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:47.616+0000 7f1b75600640 1 --2- 172.21.15.83:0/3654508411 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cefaae8570 0x55cefab0e1f0 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f1b6400e9c0 tx=0x7f1b64017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=df22f8a065597544 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:42:47.624 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:47.616+0000 7f1b56a00640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-08-19T01:42:47.624 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:47.616+0000 7f1b56a00640 1 -- 172.21.15.83:0/3654508411 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b64066070 con 0x55cefaae8570 2024-08-19T01:42:47.624 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:47.616+0000 7f1b56a00640 1 -- 172.21.15.83:0/3654508411 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8269+0+0 (secure 0 0 0) 0x7f1b640cf320 con 0x55cefaae8570 2024-08-19T01:42:47.624 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:47.616+0000 7f1b7aadb740 1 -- 172.21.15.83:0/3654508411 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1b640d5e50 con 0x55cefaae8570 2024-08-19T01:42:47.624 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:47.616+0000 7f1b56a00640 1 -- 172.21.15.83:0/3654508411 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f1b6409f070 con 0x55cefaae8570 2024-08-19T01:42:47.625 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:47.617+0000 7f1b7aadb740 10 client.16167 fetch_fsmap learned FSMap version 29 2024-08-19T01:42:47.625 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:47.617+0000 7f1b7aadb740 1 -- 172.21.15.83:0/3654508411 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55cefab6eb50 con 0x55cefaae8570 2024-08-19T01:42:47.625 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:47.617+0000 7f1b56a00640 1 -- 172.21.15.83:0/3654508411 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1b640d5e50 con 0x55cefaae8570 2024-08-19T01:42:47.625 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:47.617+0000 7f1b56a00640 1 -- 172.21.15.83:0/3654508411 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f1b6409b1f0 con 0x55cefaae8570 2024-08-19T01:42:47.625 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:47.617+0000 7f1b7aadb740 10 client.16167 fetch_fsmap finished waiting for FSMap version 29 2024-08-19T01:42:47.625 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:47.617+0000 7f1b7aadb740 10 client.16167 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-19T01:42:47.626 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:47.617+0000 7f1b7aadb740 20 client.16167 populate_metadata read hostname 'smithi083' 2024-08-19T01:42:47.626 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:47.617+0000 7f1b7aadb740 1 --2- 172.21.15.83:0/3654508411 >> [v2:172.21.15.117:6838/1967595704,v1:172.21.15.117:6839/1967595704] conn(0x55cefab761c0 0x55cefab785e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:42:47.626 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:47.617+0000 7f1b7aadb740 4 client.16167 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-08-19T01:42:47.626 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:47.617+0000 7f1b7aadb740 1 -- 172.21.15.83:0/3654508411 --> [v2:172.21.15.117:6838/1967595704,v1:172.21.15.117:6839/1967595704] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55cefaa56540 con 0x55cefab761c0 2024-08-19T01:42:47.626 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:47.617+0000 7f1b74c00640 1 --2- 172.21.15.83:0/3654508411 >> [v2:172.21.15.117:6838/1967595704,v1:172.21.15.117:6839/1967595704] conn(0x55cefab761c0 0x55cefab785e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:42:47.627 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:47.618+0000 7f1b74c00640 1 --2- 172.21.15.83:0/3654508411 >> [v2:172.21.15.117:6838/1967595704,v1:172.21.15.117:6839/1967595704] conn(0x55cefab761c0 0x55cefab785e0 crc :-1 s=READY pgs=142 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8f379daa5bc4313c server_cookie=8a4c082ce22b7bd in_seq=0 out_seq=0 2024-08-19T01:42:47.627 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:47.618+0000 7f1b56a00640 10 client.16167 ms_handle_connect on v2:172.21.15.117:6838/1967595704 2024-08-19T01:42:47.629 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:47.620+0000 7f1b56a00640 1 -- 172.21.15.83:0/3654508411 <== mds.0 v2:172.21.15.117:6838/1967595704 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55cefaa56540 con 0x55cefab761c0 2024-08-19T01:42:47.629 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:47.620+0000 7f1b56a00640 10 client.16167 handle_command_reply: tid=0 2024-08-19T01:42:47.629 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:47.620+0000 7f1b7aadb740 1 client.16167 shutdown 2024-08-19T01:42:47.630 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:47.620+0000 7f1b7aadb740 1 -- 172.21.15.83:0/3654508411 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cefaae8570 msgr2=0x55cefab0e1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:42:47.630 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:47.620+0000 7f1b7aadb740 1 --2- 172.21.15.83:0/3654508411 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cefaae8570 0x55cefab0e1f0 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f1b6400e9c0 tx=0x7f1b64017000 comp rx=0 tx=0).stop 2024-08-19T01:42:47.630 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:47.620+0000 7f1b7aadb740 1 -- 172.21.15.83:0/3654508411 shutdown_connections 2024-08-19T01:42:47.630 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:47.620+0000 7f1b7aadb740 1 --2- 172.21.15.83:0/3654508411 >> [v2:172.21.15.117:6838/1967595704,v1:172.21.15.117:6839/1967595704] conn(0x55cefab761c0 0x55cefab785e0 crc :-1 s=READY pgs=142 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:47.630 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:47.620+0000 7f1b7aadb740 1 --2- 172.21.15.83:0/3654508411 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55cefaafc040 0x55cefab6e4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:47.630 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:47.620+0000 7f1b7aadb740 1 --2- 172.21.15.83:0/3654508411 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55cefaae8e90 0x55cefab0e730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:47.631 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:47.620+0000 7f1b7aadb740 1 --2- 172.21.15.83:0/3654508411 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cefaae8570 0x55cefab0e1f0 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:47.631 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:47.620+0000 7f1b7aadb740 1 -- 172.21.15.83:0/3654508411 >> 172.21.15.83:0/3654508411 conn(0x55cefaae4be0 msgr2=0x55cefab07730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:42:47.631 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:47.620+0000 7f1b56a00640 0 client.16167 ms_handle_reset on v2:172.21.15.117:6838/1967595704 2024-08-19T01:42:47.631 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:47.620+0000 7f1b7aadb740 1 -- 172.21.15.83:0/3654508411 shutdown_connections 2024-08-19T01:42:47.631 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:47.620+0000 7f1b7aadb740 1 -- 172.21.15.83:0/3654508411 wait complete. 2024-08-19T01:42:47.632 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:47.621+0000 7f1b7aadb740 20 client.16167 trim_cache size 0 max 16384 2024-08-19T01:42:47.632 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-08-19T01:42:47.632 INFO:teuthology.orchestra.run.smithi083.stdout: "return_code": 0, 2024-08-19T01:42:47.632 INFO:teuthology.orchestra.run.smithi083.stdout: "scrub_tag": "de233f4c-1ddb-4f44-89e8-d482c9c1ff55", 2024-08-19T01:42:47.633 INFO:teuthology.orchestra.run.smithi083.stdout: "mode": "asynchronous" 2024-08-19T01:42:47.633 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-08-19T01:42:47.634 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:47.621+0000 7f1b76a00640 1 -- 172.21.15.83:0/2468689558 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f1b580a2b80 msgr2=0x7f1b580a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:42:47.634 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:47.621+0000 7f1b76a00640 1 --2- 172.21.15.83:0/2468689558 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f1b580a2b80 0x7f1b580a5040 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f1b68009430 tx=0x7f1b68004a40 comp rx=0 tx=0).stop 2024-08-19T01:42:47.634 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:47.621+0000 7f1b76a00640 1 -- 172.21.15.83:0/2468689558 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1b70103900 msgr2=0x7f1b7019a170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:42:47.634 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:47.621+0000 7f1b76a00640 1 --2- 172.21.15.83:0/2468689558 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1b70103900 0x7f1b7019a170 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f1b6400e9f0 tx=0x7f1b6400edf0 comp rx=0 tx=0).stop 2024-08-19T01:42:47.635 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:47.622+0000 7f1b76a00640 1 -- 172.21.15.83:0/2468689558 shutdown_connections 2024-08-19T01:42:47.635 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:47.622+0000 7f1b76a00640 1 --2- 172.21.15.83:0/2468689558 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f1b580a2b80 0x7f1b580a5040 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:47.635 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:47.622+0000 7f1b76a00640 1 --2- 172.21.15.83:0/2468689558 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1b70109900 0x7f1b7019ea40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:47.635 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:47.622+0000 7f1b76a00640 1 --2- 172.21.15.83:0/2468689558 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1b701042a0 0x7f1b7019a6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:47.635 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:47.622+0000 7f1b76a00640 1 --2- 172.21.15.83:0/2468689558 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1b70103900 0x7f1b7019a170 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:47.635 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:47.622+0000 7f1b76a00640 1 -- 172.21.15.83:0/2468689558 >> 172.21.15.83:0/2468689558 conn(0x7f1b700ff640 msgr2=0x7f1b70100c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:42:47.636 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:47.622+0000 7f1b76a00640 1 -- 172.21.15.83:0/2468689558 shutdown_connections 2024-08-19T01:42:47.636 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:47.622+0000 7f1b76a00640 1 -- 172.21.15.83:0/2468689558 wait complete. 2024-08-19T01:42:47.817 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:42:47 smithi083 ceph-mon[17120]: pgmap v577: 97 pgs: 97 active+clean; 175 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.7 MiB/s wr, 437 op/s 2024-08-19T01:42:47.965 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:42:47 smithi167 ceph-mon[21875]: pgmap v577: 97 pgs: 97 active+clean; 175 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.7 MiB/s wr, 437 op/s 2024-08-19T01:42:48.202 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:42:47 smithi117 ceph-mon[22016]: pgmap v577: 97 pgs: 97 active+clean; 175 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.7 MiB/s wr, 437 op/s 2024-08-19T01:42:48.203 INFO:journalctl@ceph.mds.b.smithi117.stdout:Aug 19 01:42:47 smithi117 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-mds-b[51905]: 2024-08-19T01:42:47.835+0000 7f79e46bd640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000263c [...2,head] /volumes/qa/sv_1/05b773d9-8346-4397-99ec-d49959a55703/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-42/ auth{2=1} fragtree_t(*^3) v246680 ap=2 RANDEPHEMERALPIN f(v17 m2024-08-19T01:40:36.257782+0000 222=222+0) n(v22 rc2024-08-19T01:40:36.257782+0000 b4293664 223=222+1) (inest sync->mix g=2) caps={25052=pAsLsXsFs/pAsLsXsFsxcral@260} mcw={2=AsLsXsFs} | dirtyscattered=0 request=0 lock=0 dirfrag=8 caps=1 dirtyrstat=0 dirtyparent=0 replicated=1 dirty=0 waiter=0 authpin=1 scrubqueue=0 randepin 0x55b59696b700]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value":"(2)0x1000000263c:[<0x10000000399/blog-42 v74084>,<0x10000000208/blogtest_in v446>,<0x10000000201/src v1710>,<0x10000000006/blogbench-1.0 v1485>,<0x10000000005/tmp v127>,<0x100000001fb/client.0 v113>,<0x100000001fa/05b773d9-8346-4397-99ec-d49959a55703 v116>,<0x10000000001/sv_1 v127>,<0x10000000000/qa v120>,<0x1/volumes v116>]//[]","memoryvalue":"(2)0x1000000263c:[<0x10000000399/blog-42 v246680>,<0x10000000208/blogtest_in v2482>,<0x10000000201/src v2446>,<0x10000000006/blogbench-1.0 v2124>,<0x10000000005/tmp v740>,<0x100000001fb/client.0 v701>,<0x100000001fa/05b773d9-8346-4397-99ec-d49959a55703 v705>,<0x10000000001/sv_1 v708>,<0x10000000000/qa v698>,<0x1/volumes v686>]//[]","error_str":""},"raw_stats":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2024-08-19T01:40:18.306206+0000 221=221+0)","ondisk_value.rstat":"n(v0 rc2024-08-19T01:40:36.257782+0000 b4293664 223=222+1)","memory_value.dirstat":"f(v17 m2024-08-19T01:40:36.257782+0000 222=222+0)","memory_value.rstat":"n(v22 rc2024-08-19T01:40:36.257782+0000 b4293664 223=222+1)","error_str":"freshly-calculated rstats don't match existing ones"},"return_code":0} 2024-08-19T01:42:48.476 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833 shell --fsid bad8780e-5dc9-11ef-bccf-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-08-19T01:42:48.928 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/bad8780e-5dc9-11ef-bccf-c7b262605968/mon.a/config 2024-08-19T01:42:48.966 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:42:48 smithi167 ceph-mon[21875]: scrub queued for path: / 2024-08-19T01:42:48.966 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:42:48 smithi167 ceph-mon[21875]: scrub summary: idle+waiting paths [/] 2024-08-19T01:42:48.966 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:42:48 smithi167 ceph-mon[21875]: scrub summary: active paths [/] 2024-08-19T01:42:48.966 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:42:48 smithi167 ceph-mon[21875]: scrub summary: active paths [/] 2024-08-19T01:42:48.966 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:42:48 smithi167 ceph-mon[21875]: Scrub error on inode 0x1000000263c (/volumes/qa/sv_1/05b773d9-8346-4397-99ec-d49959a55703/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-42) see mds.b log and `damage ls` output for details 2024-08-19T01:42:48.966 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:42:48 smithi167 ceph-mon[21875]: scrub summary: idle+waiting paths [/] 2024-08-19T01:42:48.966 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:42:48 smithi167 ceph-mon[21875]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-19T01:42:49.101 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:42:48 smithi083 ceph-mon[17120]: scrub queued for path: / 2024-08-19T01:42:49.101 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:42:48 smithi083 ceph-mon[17120]: scrub summary: idle+waiting paths [/] 2024-08-19T01:42:49.101 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:42:48 smithi083 ceph-mon[17120]: scrub summary: active paths [/] 2024-08-19T01:42:49.101 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:42:48 smithi083 ceph-mon[17120]: scrub summary: active paths [/] 2024-08-19T01:42:49.101 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:42:48 smithi083 ceph-mon[17120]: Scrub error on inode 0x1000000263c (/volumes/qa/sv_1/05b773d9-8346-4397-99ec-d49959a55703/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-42) see mds.b log and `damage ls` output for details 2024-08-19T01:42:49.101 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:42:48 smithi083 ceph-mon[17120]: scrub summary: idle+waiting paths [/] 2024-08-19T01:42:49.102 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:42:48 smithi083 ceph-mon[17120]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-19T01:42:49.103 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:42:48 smithi117 ceph-mon[22016]: scrub queued for path: / 2024-08-19T01:42:49.103 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:42:48 smithi117 ceph-mon[22016]: scrub summary: idle+waiting paths [/] 2024-08-19T01:42:49.103 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:42:48 smithi117 ceph-mon[22016]: scrub summary: active paths [/] 2024-08-19T01:42:49.103 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:42:48 smithi117 ceph-mon[22016]: scrub summary: active paths [/] 2024-08-19T01:42:49.103 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:42:48 smithi117 ceph-mon[22016]: Scrub error on inode 0x1000000263c (/volumes/qa/sv_1/05b773d9-8346-4397-99ec-d49959a55703/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-42) see mds.b log and `damage ls` output for details 2024-08-19T01:42:49.103 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:42:48 smithi117 ceph-mon[22016]: scrub summary: idle+waiting paths [/] 2024-08-19T01:42:49.103 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:42:48 smithi117 ceph-mon[22016]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-19T01:42:49.723 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:49.721+0000 7fbbaa000640 1 -- 172.21.15.83:0/4206571239 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbba4102250 msgr2=0x7fbba410e780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:42:49.723 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:49.721+0000 7fbbaa000640 1 --2- 172.21.15.83:0/4206571239 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbba4102250 0x7fbba410e780 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7fbb9c009930 tx=0x7fbb9c02a420 comp rx=0 tx=0).stop 2024-08-19T01:42:49.723 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:49.721+0000 7fbbaa000640 1 -- 172.21.15.83:0/4206571239 shutdown_connections 2024-08-19T01:42:49.724 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:49.721+0000 7fbbaa000640 1 --2- 172.21.15.83:0/4206571239 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbba4102250 0x7fbba410e780 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:49.724 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:49.721+0000 7fbbaa000640 1 --2- 172.21.15.83:0/4206571239 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbba41018b0 0x7fbba4101d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:49.724 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:49.721+0000 7fbbaa000640 1 --2- 172.21.15.83:0/4206571239 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbba41078b0 0x7fbba4107c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:49.724 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:49.721+0000 7fbbaa000640 1 -- 172.21.15.83:0/4206571239 >> 172.21.15.83:0/4206571239 conn(0x7fbba40fd5f0 msgr2=0x7fbba40ffa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:42:49.724 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:49.721+0000 7fbbaa000640 1 -- 172.21.15.83:0/4206571239 shutdown_connections 2024-08-19T01:42:49.724 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:49.721+0000 7fbbaa000640 1 -- 172.21.15.83:0/4206571239 wait complete. 2024-08-19T01:42:49.725 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:49.722+0000 7fbbaa000640 1 Processor -- start 2024-08-19T01:42:49.726 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:49.723+0000 7fbbaa000640 1 -- start start 2024-08-19T01:42:49.726 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:49.723+0000 7fbbaa000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbba41018b0 0x7fbba419a370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:42:49.726 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:49.723+0000 7fbbaa000640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbba4102250 0x7fbba419a8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:42:49.726 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:49.723+0000 7fbbaa000640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbba41078b0 0x7fbba419ec40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:42:49.726 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:49.723+0000 7fbbaa000640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fbba4111d00 con 0x7fbba41018b0 2024-08-19T01:42:49.726 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:49.723+0000 7fbbaa000640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fbba4111b80 con 0x7fbba41078b0 2024-08-19T01:42:49.726 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:49.723+0000 7fbbaa000640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fbba4111e80 con 0x7fbba4102250 2024-08-19T01:42:49.726 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:49.723+0000 7fbba2a00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbba4102250 0x7fbba419a8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:42:49.726 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:49.723+0000 7fbba3e00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbba41078b0 0x7fbba419ec40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:42:49.726 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:49.723+0000 7fbba2a00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbba4102250 0x7fbba419a8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.83:50040/0 (socket says 172.21.15.83:50040) 2024-08-19T01:42:49.726 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:49.723+0000 7fbba2a00640 1 -- 172.21.15.83:0/2821130495 learned_addr learned my addr 172.21.15.83:0/2821130495 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-08-19T01:42:49.726 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:49.723+0000 7fbba3400640 1 --2- 172.21.15.83:0/2821130495 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbba41018b0 0x7fbba419a370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:42:49.727 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:49.724+0000 7fbba2a00640 1 -- 172.21.15.83:0/2821130495 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbba41078b0 msgr2=0x7fbba419ec40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:42:49.727 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:49.724+0000 7fbba2a00640 1 --2- 172.21.15.83:0/2821130495 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbba41078b0 0x7fbba419ec40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:49.727 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:49.724+0000 7fbba2a00640 1 -- 172.21.15.83:0/2821130495 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbba41018b0 msgr2=0x7fbba419a370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:42:49.727 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:49.724+0000 7fbba2a00640 1 --2- 172.21.15.83:0/2821130495 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbba41018b0 0x7fbba419a370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:49.727 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:49.724+0000 7fbba2a00640 1 -- 172.21.15.83:0/2821130495 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbba419f320 con 0x7fbba4102250 2024-08-19T01:42:49.727 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:49.724+0000 7fbba2a00640 1 --2- 172.21.15.83:0/2821130495 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbba4102250 0x7fbba419a8b0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fbb8c00ea60 tx=0x7fbb8c00ef20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:42:49.727 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:49.724+0000 7fbb9be00640 1 -- 172.21.15.83:0/2821130495 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb8c00cde0 con 0x7fbba4102250 2024-08-19T01:42:49.727 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:49.724+0000 7fbb9be00640 1 -- 172.21.15.83:0/2821130495 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbb8c0035f0 con 0x7fbba4102250 2024-08-19T01:42:49.727 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:49.724+0000 7fbbaa000640 1 -- 172.21.15.83:0/2821130495 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbba419f610 con 0x7fbba4102250 2024-08-19T01:42:49.727 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:49.724+0000 7fbb9be00640 1 -- 172.21.15.83:0/2821130495 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb8c010670 con 0x7fbba4102250 2024-08-19T01:42:49.727 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:49.724+0000 7fbbaa000640 1 -- 172.21.15.83:0/2821130495 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbba41a6ef0 con 0x7fbba4102250 2024-08-19T01:42:49.728 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:49.727+0000 7fbb9be00640 1 -- 172.21.15.83:0/2821130495 <== mon.1 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7fbb8c0037a0 con 0x7fbba4102250 2024-08-19T01:42:49.729 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:49.727+0000 7fbbacc01740 1 Processor -- start 2024-08-19T01:42:49.729 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:49.728+0000 7fbbacc01740 1 -- start start 2024-08-19T01:42:49.729 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:49.728+0000 7fbbacc01740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c36d4d2b90 0x55c36d4d2f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:42:49.730 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:49.728+0000 7fbbacc01740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c36d4d34b0 0x55c36d4dd830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:42:49.730 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:49.728+0000 7fbbacc01740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c36d4ddd70 0x55c36d4e0160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:42:49.730 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:49.728+0000 7fbbacc01740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55c36d438bf0 con 0x55c36d4ddd70 2024-08-19T01:42:49.730 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:49.728+0000 7fbbacc01740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55c36d3d0bd0 con 0x55c36d4d34b0 2024-08-19T01:42:49.730 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:49.728+0000 7fbbacc01740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55c36d434bb0 con 0x55c36d4d2b90 2024-08-19T01:42:49.731 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:49.728+0000 7fbba3400640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c36d4d34b0 0x55c36d4dd830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:42:49.731 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:49.728+0000 7fbba3400640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c36d4d34b0 0x55c36d4dd830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.83:40430/0 (socket says 172.21.15.83:40430) 2024-08-19T01:42:49.731 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:49.728+0000 7fbba3400640 1 -- 172.21.15.83:0/3392374301 learned_addr learned my addr 172.21.15.83:0/3392374301 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-08-19T01:42:49.731 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:49.729+0000 7fbba3e00640 1 --2- 172.21.15.83:0/3392374301 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c36d4d2b90 0x55c36d4d2f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:42:49.731 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:49.729+0000 7fbba2a00640 1 --2- 172.21.15.83:0/3392374301 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c36d4ddd70 0x55c36d4e0160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:42:49.731 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:49.729+0000 7fbb9be00640 1 --2- 172.21.15.83:0/2821130495 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7fbb880a2b80 0x7fbb880a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:42:49.731 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:49.729+0000 7fbba3400640 1 -- 172.21.15.83:0/3392374301 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c36d4d2b90 msgr2=0x55c36d4d2f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:42:49.732 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:49.729+0000 7fbba3400640 1 --2- 172.21.15.83:0/3392374301 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c36d4d2b90 0x55c36d4d2f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:49.732 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:49.729+0000 7fbba3400640 1 -- 172.21.15.83:0/3392374301 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c36d4ddd70 msgr2=0x55c36d4e0160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:42:49.732 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:49.729+0000 7fbba3400640 1 --2- 172.21.15.83:0/3392374301 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c36d4ddd70 0x55c36d4e0160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:49.732 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:49.729+0000 7fbba3400640 1 -- 172.21.15.83:0/3392374301 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c36d3d78a0 con 0x55c36d4d34b0 2024-08-19T01:42:49.732 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:49.729+0000 7fbb9be00640 1 -- 172.21.15.83:0/2821130495 <== mon.1 v2:172.21.15.167:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8269+0+0 (secure 0 0 0) 0x7fbb8c0cf420 con 0x7fbba4102250 2024-08-19T01:42:49.732 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:49.729+0000 7fbba3400640 1 --2- 172.21.15.83:0/3392374301 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c36d4d34b0 0x55c36d4dd830 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7fbb94009ee0 tx=0x7fbb94010740 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6d2e3712b58c1b15 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:42:49.732 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:49.729+0000 7fbba3e00640 1 --2- 172.21.15.83:0/2821130495 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7fbb880a2b80 0x7fbb880a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:42:49.732 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:49.730+0000 7fbb93e00640 1 -- 172.21.15.83:0/3392374301 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb94016070 con 0x55c36d4d34b0 2024-08-19T01:42:49.732 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:49.730+0000 7fbb93e00640 1 -- 172.21.15.83:0/3392374301 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbb94004d20 con 0x55c36d4d34b0 2024-08-19T01:42:49.732 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:49.730+0000 7fbb93e00640 1 -- 172.21.15.83:0/3392374301 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb94012840 con 0x55c36d4d34b0 2024-08-19T01:42:49.734 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:49.730+0000 7fbbacc01740 1 -- 172.21.15.83:0/3392374301 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c36d4d34b0 msgr2=0x55c36d4dd830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:42:49.734 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:49.730+0000 7fbba3e00640 1 --2- 172.21.15.83:0/2821130495 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7fbb880a2b80 0x7fbb880a5040 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7fbb9c004770 tx=0x7fbb9c009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:42:49.734 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:49.730+0000 7fbbacc01740 1 --2- 172.21.15.83:0/3392374301 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c36d4d34b0 0x55c36d4dd830 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7fbb94009ee0 tx=0x7fbb94010740 comp rx=0 tx=0).stop 2024-08-19T01:42:49.734 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:49.730+0000 7fbbacc01740 1 -- 172.21.15.83:0/3392374301 shutdown_connections 2024-08-19T01:42:49.734 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:49.730+0000 7fbbacc01740 1 --2- 172.21.15.83:0/3392374301 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c36d4ddd70 0x55c36d4e0160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:49.734 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:49.730+0000 7fbbacc01740 1 --2- 172.21.15.83:0/3392374301 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c36d4d34b0 0x55c36d4dd830 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:49.734 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:49.730+0000 7fbbacc01740 1 --2- 172.21.15.83:0/3392374301 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c36d4d2b90 0x55c36d4d2f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:49.734 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:49.730+0000 7fbbacc01740 1 -- 172.21.15.83:0/3392374301 >> 172.21.15.83:0/3392374301 conn(0x55c36d4cf120 msgr2=0x55c36d4cf500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:42:49.735 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:49.731+0000 7fbbacc01740 1 -- 172.21.15.83:0/3392374301 shutdown_connections 2024-08-19T01:42:49.735 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:49.731+0000 7fbbacc01740 1 -- 172.21.15.83:0/3392374301 wait complete. 2024-08-19T01:42:49.735 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:49.731+0000 7fbbacc01740 1 -- start start 2024-08-19T01:42:49.735 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:49.731+0000 7fbbacc01740 1 Processor -- start 2024-08-19T01:42:49.735 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:49.732+0000 7fbbacc01740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c36d4d2b90 0x55c36d4dac70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:42:49.735 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:49.732+0000 7fbbacc01740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c36d4d34b0 0x55c36d4db1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:42:49.735 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:49.732+0000 7fbbacc01740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c36d4ddd70 0x55c36d4db6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:42:49.735 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:49.732+0000 7fbbacc01740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55c36d4e0940 con 0x55c36d4d2b90 2024-08-19T01:42:49.735 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:49.732+0000 7fbbacc01740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55c36d3f3d40 con 0x55c36d4ddd70 2024-08-19T01:42:49.735 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:49.732+0000 7fbbacc01740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55c36d4e0ac0 con 0x55c36d4d34b0 2024-08-19T01:42:49.735 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:49.732+0000 7fbbacc01740 10 client.0 _ll_register_callbacks cb 0x55c36d3bcb10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-19T01:42:49.735 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:49.732+0000 7fbba3e00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c36d4ddd70 0x55c36d4db6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:42:49.735 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:49.732+0000 7fbba2a00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c36d4d34b0 0x55c36d4db1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:42:49.735 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:49.732+0000 7fbba3400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c36d4d2b90 0x55c36d4dac70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:42:49.735 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:49.732+0000 7fbba2a00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c36d4d34b0 0x55c36d4db1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.83:50044/0 (socket says 172.21.15.83:50044) 2024-08-19T01:42:49.735 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:49.732+0000 7fbba3e00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c36d4ddd70 0x55c36d4db6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.83:40440/0 (socket says 172.21.15.83:40440) 2024-08-19T01:42:49.736 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:49.732+0000 7fbba2a00640 1 -- 172.21.15.83:0/1895021063 learned_addr learned my addr 172.21.15.83:0/1895021063 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-08-19T01:42:49.736 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:49.733+0000 7fbba2a00640 1 -- 172.21.15.83:0/1895021063 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c36d4ddd70 msgr2=0x55c36d4db6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:42:49.736 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:49.733+0000 7fbba2a00640 1 --2- 172.21.15.83:0/1895021063 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c36d4ddd70 0x55c36d4db6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:49.736 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:49.733+0000 7fbba2a00640 1 -- 172.21.15.83:0/1895021063 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c36d4d2b90 msgr2=0x55c36d4dac70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:42:49.736 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:49.733+0000 7fbba2a00640 1 --2- 172.21.15.83:0/1895021063 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c36d4d2b90 0x55c36d4dac70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:49.736 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:49.733+0000 7fbba2a00640 1 -- 172.21.15.83:0/1895021063 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c36d1aa3c0 con 0x55c36d4d34b0 2024-08-19T01:42:49.737 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:49.733+0000 7fbba2a00640 1 --2- 172.21.15.83:0/1895021063 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c36d4d34b0 0x55c36d4db1b0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fbb8c0a4340 tx=0x7fbb8c009580 comp rx=0 tx=0).ready entity=mon.1 client_cookie=40943a7404c38a5f server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:42:49.737 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:49.733+0000 7fbb90c00640 10 client.0 ms_handle_connect on v2:172.21.15.167:3300/0 2024-08-19T01:42:49.738 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:49.733+0000 7fbb90c00640 1 -- 172.21.15.83:0/1895021063 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb8c0b2060 con 0x55c36d4d34b0 2024-08-19T01:42:49.738 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:49.733+0000 7fbbacc01740 1 -- 172.21.15.83:0/1895021063 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbb94004d20 con 0x55c36d4d34b0 2024-08-19T01:42:49.738 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:49.733+0000 7fbb90c00640 1 -- 172.21.15.83:0/1895021063 <== mon.1 v2:172.21.15.167:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8269+0+0 (secure 0 0 0) 0x7fbb8c098270 con 0x55c36d4d34b0 2024-08-19T01:42:49.738 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:49.733+0000 7fbb90c00640 1 -- 172.21.15.83:0/1895021063 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fbb8c09c300 con 0x55c36d4d34b0 2024-08-19T01:42:49.738 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:49.733+0000 7fbbacc01740 10 client.26161 fetch_fsmap learned FSMap version 29 2024-08-19T01:42:49.738 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:49.733+0000 7fbbacc01740 1 -- 172.21.15.83:0/1895021063 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55c36d4ec660 con 0x55c36d4d34b0 2024-08-19T01:42:49.738 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:49.734+0000 7fbb90c00640 1 -- 172.21.15.83:0/1895021063 <== mon.1 v2:172.21.15.167:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbb94004d20 con 0x55c36d4d34b0 2024-08-19T01:42:49.738 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:49.734+0000 7fbb90c00640 1 -- 172.21.15.83:0/1895021063 <== mon.1 v2:172.21.15.167:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7fbb8c0d2b50 con 0x55c36d4d34b0 2024-08-19T01:42:49.738 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:49.734+0000 7fbbacc01740 10 client.26161 fetch_fsmap finished waiting for FSMap version 29 2024-08-19T01:42:49.738 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:49.734+0000 7fbbacc01740 10 client.26161 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-19T01:42:49.738 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:49.734+0000 7fbbacc01740 20 client.26161 populate_metadata read hostname 'smithi083' 2024-08-19T01:42:49.738 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:49.734+0000 7fbbacc01740 1 --2- 172.21.15.83:0/1895021063 >> [v2:172.21.15.117:6838/1967595704,v1:172.21.15.117:6839/1967595704] conn(0x55c36d4edb20 0x55c36d4eff10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:42:49.738 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:49.734+0000 7fbbacc01740 4 client.26161 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-19T01:42:49.738 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:49.734+0000 7fbbacc01740 1 -- 172.21.15.83:0/1895021063 --> [v2:172.21.15.117:6838/1967595704,v1:172.21.15.117:6839/1967595704] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c36d446160 con 0x55c36d4edb20 2024-08-19T01:42:49.739 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:49.734+0000 7fbba3400640 1 --2- 172.21.15.83:0/1895021063 >> [v2:172.21.15.117:6838/1967595704,v1:172.21.15.117:6839/1967595704] conn(0x55c36d4edb20 0x55c36d4eff10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:42:49.739 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:49.735+0000 7fbb90c00640 10 client.26161 ms_handle_connect on v2:172.21.15.117:6838/1967595704 2024-08-19T01:42:49.739 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:49.735+0000 7fbba3400640 1 --2- 172.21.15.83:0/1895021063 >> [v2:172.21.15.117:6838/1967595704,v1:172.21.15.117:6839/1967595704] conn(0x55c36d4edb20 0x55c36d4eff10 crc :-1 s=READY pgs=143 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5568df39da8c92f9 server_cookie=492c2aa240968a7b in_seq=0 out_seq=0 2024-08-19T01:42:49.739 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:49.736+0000 7fbb90c00640 1 -- 172.21.15.83:0/1895021063 <== mds.0 v2:172.21.15.117:6838/1967595704 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55c36d446160 con 0x55c36d4edb20 2024-08-19T01:42:49.739 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:49.736+0000 7fbb90c00640 10 client.26161 handle_command_reply: tid=0 2024-08-19T01:42:49.739 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:49.736+0000 7fbbacc01740 1 client.26161 shutdown 2024-08-19T01:42:49.740 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:49.736+0000 7fbbacc01740 1 -- 172.21.15.83:0/1895021063 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c36d4d34b0 msgr2=0x55c36d4db1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:42:49.740 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:49.736+0000 7fbbacc01740 1 --2- 172.21.15.83:0/1895021063 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c36d4d34b0 0x55c36d4db1b0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fbb8c0a4340 tx=0x7fbb8c009580 comp rx=0 tx=0).stop 2024-08-19T01:42:49.740 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:49.736+0000 7fbbacc01740 1 -- 172.21.15.83:0/1895021063 shutdown_connections 2024-08-19T01:42:49.740 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:49.736+0000 7fbbacc01740 1 --2- 172.21.15.83:0/1895021063 >> [v2:172.21.15.117:6838/1967595704,v1:172.21.15.117:6839/1967595704] conn(0x55c36d4edb20 0x55c36d4eff10 crc :-1 s=READY pgs=143 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:49.740 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:49.736+0000 7fbbacc01740 1 --2- 172.21.15.83:0/1895021063 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c36d4ddd70 0x55c36d4db6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:49.740 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:49.736+0000 7fbb90c00640 0 client.26161 ms_handle_reset on v2:172.21.15.117:6838/1967595704 2024-08-19T01:42:49.740 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:49.736+0000 7fbbacc01740 1 --2- 172.21.15.83:0/1895021063 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c36d4d34b0 0x55c36d4db1b0 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:49.740 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:49.736+0000 7fbbacc01740 1 --2- 172.21.15.83:0/1895021063 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c36d4d2b90 0x55c36d4dac70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:49.740 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:49.736+0000 7fbbacc01740 1 -- 172.21.15.83:0/1895021063 >> 172.21.15.83:0/1895021063 conn(0x55c36d4cf120 msgr2=0x55c36d4d4e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:42:49.740 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:49.737+0000 7fbbacc01740 1 -- 172.21.15.83:0/1895021063 shutdown_connections 2024-08-19T01:42:49.740 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:49.737+0000 7fbbacc01740 1 -- 172.21.15.83:0/1895021063 wait complete. 2024-08-19T01:42:49.740 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:49.737+0000 7fbbacc01740 20 client.26161 trim_cache size 0 max 16384 2024-08-19T01:42:49.757 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:49.756+0000 7fbbacc01740 1 Processor -- start 2024-08-19T01:42:49.757 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:49.756+0000 7fbbacc01740 1 -- start start 2024-08-19T01:42:49.759 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:49.756+0000 7fbbacc01740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c36d4d2b90 0x55c36d4dcbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:42:49.759 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:49.756+0000 7fbbacc01740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c36d4d34b0 0x55c36d4f57d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:42:49.759 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:49.756+0000 7fbbacc01740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c36d4ddd70 0x55c36d4f5d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:42:49.759 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:49.756+0000 7fbbacc01740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fbba41b1e50 con 0x55c36d4ddd70 2024-08-19T01:42:49.759 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:49.756+0000 7fbbacc01740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fbba41af6c0 con 0x55c36d4d34b0 2024-08-19T01:42:49.759 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:49.756+0000 7fbbacc01740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fbba41ad510 con 0x55c36d4d2b90 2024-08-19T01:42:49.759 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:49.757+0000 7fbba3400640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c36d4d2b90 0x55c36d4dcbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:42:49.759 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:49.757+0000 7fbba3400640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c36d4d2b90 0x55c36d4dcbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.83:50052/0 (socket says 172.21.15.83:50052) 2024-08-19T01:42:49.759 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:49.757+0000 7fbba3400640 1 -- 172.21.15.83:0/3452452332 learned_addr learned my addr 172.21.15.83:0/3452452332 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-08-19T01:42:49.759 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:49.757+0000 7fbba2a00640 1 --2- 172.21.15.83:0/3452452332 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c36d4d34b0 0x55c36d4f57d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:42:49.759 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:49.757+0000 7fbba3e00640 1 --2- 172.21.15.83:0/3452452332 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c36d4ddd70 0x55c36d4f5d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:42:49.759 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:49.757+0000 7fbba3400640 1 -- 172.21.15.83:0/3452452332 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c36d4d34b0 msgr2=0x55c36d4f57d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:42:49.760 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:49.757+0000 7fbba3400640 1 --2- 172.21.15.83:0/3452452332 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c36d4d34b0 0x55c36d4f57d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:49.760 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:49.757+0000 7fbba3400640 1 -- 172.21.15.83:0/3452452332 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c36d4ddd70 msgr2=0x55c36d4f5d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:42:49.760 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:49.757+0000 7fbba3400640 1 --2- 172.21.15.83:0/3452452332 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c36d4ddd70 0x55c36d4f5d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:49.760 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:49.757+0000 7fbba3400640 1 -- 172.21.15.83:0/3452452332 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c36d4f0280 con 0x55c36d4d2b90 2024-08-19T01:42:49.760 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:49.757+0000 7fbba3e00640 1 --2- 172.21.15.83:0/3452452332 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c36d4ddd70 0x55c36d4f5d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-19T01:42:49.760 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:49.757+0000 7fbba3400640 1 --2- 172.21.15.83:0/3452452332 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c36d4d2b90 0x55c36d4dcbe0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x55c36d4cff80 tx=0x7fbb940097e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e623029470747aa0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:42:49.761 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:49.758+0000 7fbb93400640 1 -- 172.21.15.83:0/3452452332 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb94016040 con 0x55c36d4d2b90 2024-08-19T01:42:49.761 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:49.758+0000 7fbb93400640 1 -- 172.21.15.83:0/3452452332 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbb94012ba0 con 0x55c36d4d2b90 2024-08-19T01:42:49.761 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:49.758+0000 7fbb93400640 1 -- 172.21.15.83:0/3452452332 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb94025bc0 con 0x55c36d4d2b90 2024-08-19T01:42:49.762 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:49.758+0000 7fbbacc01740 1 -- 172.21.15.83:0/3452452332 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c36d4d2b90 msgr2=0x55c36d4dcbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:42:49.762 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:49.758+0000 7fbbacc01740 1 --2- 172.21.15.83:0/3452452332 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c36d4d2b90 0x55c36d4dcbe0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x55c36d4cff80 tx=0x7fbb940097e0 comp rx=0 tx=0).stop 2024-08-19T01:42:49.762 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:49.758+0000 7fbbacc01740 1 -- 172.21.15.83:0/3452452332 shutdown_connections 2024-08-19T01:42:49.762 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:49.758+0000 7fbbacc01740 1 --2- 172.21.15.83:0/3452452332 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c36d4ddd70 0x55c36d4f5d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:49.762 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:49.758+0000 7fbbacc01740 1 --2- 172.21.15.83:0/3452452332 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c36d4d34b0 0x55c36d4f57d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:49.762 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:49.758+0000 7fbbacc01740 1 --2- 172.21.15.83:0/3452452332 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c36d4d2b90 0x55c36d4dcbe0 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:49.762 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:49.758+0000 7fbbacc01740 1 -- 172.21.15.83:0/3452452332 >> 172.21.15.83:0/3452452332 conn(0x55c36d4cf120 msgr2=0x55c36d4dc4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:42:49.762 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:49.759+0000 7fbbacc01740 1 -- 172.21.15.83:0/3452452332 shutdown_connections 2024-08-19T01:42:49.762 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:49.759+0000 7fbbacc01740 1 -- 172.21.15.83:0/3452452332 wait complete. 2024-08-19T01:42:49.762 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:49.759+0000 7fbbacc01740 1 -- start start 2024-08-19T01:42:49.762 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:49.759+0000 7fbbacc01740 1 Processor -- start 2024-08-19T01:42:49.762 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:49.760+0000 7fbbacc01740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c36d4d2b90 0x55c36d4e6470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:42:49.762 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:49.760+0000 7fbbacc01740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c36d4d34b0 0x55c36d4e17f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:42:49.763 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:49.760+0000 7fbbacc01740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c36d4ddd70 0x55c36d4e1d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:42:49.763 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:49.760+0000 7fbbacc01740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fbba41adb80 con 0x55c36d4d2b90 2024-08-19T01:42:49.763 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:49.760+0000 7fbbacc01740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fbba41aeff0 con 0x55c36d4ddd70 2024-08-19T01:42:49.763 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:49.760+0000 7fbbacc01740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fbba41ae250 con 0x55c36d4d34b0 2024-08-19T01:42:49.763 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:49.760+0000 7fbbacc01740 10 client.0 _ll_register_callbacks cb 0x7fbb8c003e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-19T01:42:49.763 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:49.760+0000 7fbba3e00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c36d4ddd70 0x55c36d4e1d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:42:49.763 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:49.760+0000 7fbba2a00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c36d4d34b0 0x55c36d4e17f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:42:49.763 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:49.760+0000 7fbba3e00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c36d4ddd70 0x55c36d4e1d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.83:40456/0 (socket says 172.21.15.83:40456) 2024-08-19T01:42:49.763 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:49.760+0000 7fbba3e00640 1 -- 172.21.15.83:0/799283278 learned_addr learned my addr 172.21.15.83:0/799283278 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-08-19T01:42:49.763 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:49.760+0000 7fbba3400640 1 --2- 172.21.15.83:0/799283278 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c36d4d2b90 0x55c36d4e6470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:42:49.765 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:49.760+0000 7fbba2a00640 1 -- 172.21.15.83:0/799283278 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c36d4ddd70 msgr2=0x55c36d4e1d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:42:49.765 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:49.760+0000 7fbba2a00640 1 --2- 172.21.15.83:0/799283278 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c36d4ddd70 0x55c36d4e1d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:49.765 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:49.760+0000 7fbba2a00640 1 -- 172.21.15.83:0/799283278 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c36d4d2b90 msgr2=0x55c36d4e6470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:42:49.765 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:49.760+0000 7fbba2a00640 1 --2- 172.21.15.83:0/799283278 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c36d4d2b90 0x55c36d4e6470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:49.765 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:49.760+0000 7fbba2a00640 1 -- 172.21.15.83:0/799283278 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c36d4e2510 con 0x55c36d4d34b0 2024-08-19T01:42:49.765 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:49.761+0000 7fbba2a00640 1 --2- 172.21.15.83:0/799283278 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c36d4d34b0 0x55c36d4e17f0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fbb8c098060 tx=0x7fbb8c003d60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ab673f63614afa45 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:42:49.765 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:49.761+0000 7fbb90c00640 10 client.0 ms_handle_connect on v2:172.21.15.167:3300/0 2024-08-19T01:42:49.765 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:49.761+0000 7fbb90c00640 1 -- 172.21.15.83:0/799283278 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb8c09b740 con 0x55c36d4d34b0 2024-08-19T01:42:49.765 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:49.761+0000 7fbb90c00640 1 -- 172.21.15.83:0/799283278 <== mon.1 v2:172.21.15.167:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8269+0+0 (secure 0 0 0) 0x7fbb8c0ba2f0 con 0x55c36d4d34b0 2024-08-19T01:42:49.765 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:49.761+0000 7fbbacc01740 1 -- 172.21.15.83:0/799283278 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbb94012ba0 con 0x55c36d4d34b0 2024-08-19T01:42:49.765 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:49.761+0000 7fbb90c00640 1 -- 172.21.15.83:0/799283278 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fbb8c0a4870 con 0x55c36d4d34b0 2024-08-19T01:42:49.765 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:49.761+0000 7fbbacc01740 10 client.26167 fetch_fsmap learned FSMap version 29 2024-08-19T01:42:49.766 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:49.761+0000 7fbbacc01740 1 -- 172.21.15.83:0/799283278 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55c36d4e2510 con 0x55c36d4d34b0 2024-08-19T01:42:49.766 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:49.761+0000 7fbb90c00640 1 -- 172.21.15.83:0/799283278 <== mon.1 v2:172.21.15.167:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbb94012ba0 con 0x55c36d4d34b0 2024-08-19T01:42:49.766 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:49.762+0000 7fbb90c00640 1 -- 172.21.15.83:0/799283278 <== mon.1 v2:172.21.15.167:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7fbb8c097950 con 0x55c36d4d34b0 2024-08-19T01:42:49.766 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:49.762+0000 7fbbacc01740 10 client.26167 fetch_fsmap finished waiting for FSMap version 29 2024-08-19T01:42:49.766 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:49.762+0000 7fbbacc01740 10 client.26167 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-19T01:42:49.766 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:49.762+0000 7fbbacc01740 20 client.26167 populate_metadata read hostname 'smithi083' 2024-08-19T01:42:49.766 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:49.762+0000 7fbbacc01740 1 --2- 172.21.15.83:0/799283278 >> [v2:172.21.15.117:6838/1967595704,v1:172.21.15.117:6839/1967595704] conn(0x55c36d5614c0 0x55c36d5638e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:42:49.766 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:49.762+0000 7fbbacc01740 4 client.26167 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-08-19T01:42:49.766 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:49.762+0000 7fbbacc01740 1 -- 172.21.15.83:0/799283278 --> [v2:172.21.15.117:6838/1967595704,v1:172.21.15.117:6839/1967595704] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55c36d440670 con 0x55c36d5614c0 2024-08-19T01:42:49.766 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:49.762+0000 7fbba3400640 1 --2- 172.21.15.83:0/799283278 >> [v2:172.21.15.117:6838/1967595704,v1:172.21.15.117:6839/1967595704] conn(0x55c36d5614c0 0x55c36d5638e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:42:49.766 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:49.762+0000 7fbba3400640 1 --2- 172.21.15.83:0/799283278 >> [v2:172.21.15.117:6838/1967595704,v1:172.21.15.117:6839/1967595704] conn(0x55c36d5614c0 0x55c36d5638e0 crc :-1 s=READY pgs=144 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=23672b5086fd5e62 server_cookie=659c74f06f128f7e in_seq=0 out_seq=0 2024-08-19T01:42:49.766 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:49.762+0000 7fbb90c00640 10 client.26167 ms_handle_connect on v2:172.21.15.117:6838/1967595704 2024-08-19T01:42:49.767 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:49.764+0000 7fbb90c00640 1 -- 172.21.15.83:0/799283278 <== mds.0 v2:172.21.15.117:6838/1967595704 1 ==== command_reply(tid 0: 0 ) ==== 8+0+268 (crc 0 0 0) 0x55c36d440670 con 0x55c36d5614c0 2024-08-19T01:42:49.767 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:49.764+0000 7fbb90c00640 10 client.26167 handle_command_reply: tid=0 2024-08-19T01:42:49.768 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:49.764+0000 7fbbacc01740 1 client.26167 shutdown 2024-08-19T01:42:49.768 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:49.764+0000 7fbbacc01740 1 -- 172.21.15.83:0/799283278 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c36d4d34b0 msgr2=0x55c36d4e17f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:42:49.768 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:49.764+0000 7fbbacc01740 1 --2- 172.21.15.83:0/799283278 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c36d4d34b0 0x55c36d4e17f0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fbb8c098060 tx=0x7fbb8c003d60 comp rx=0 tx=0).stop 2024-08-19T01:42:49.768 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:49.764+0000 7fbbacc01740 1 -- 172.21.15.83:0/799283278 shutdown_connections 2024-08-19T01:42:49.768 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:49.764+0000 7fbbacc01740 1 --2- 172.21.15.83:0/799283278 >> [v2:172.21.15.117:6838/1967595704,v1:172.21.15.117:6839/1967595704] conn(0x55c36d5614c0 0x55c36d5638e0 crc :-1 s=READY pgs=144 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:49.768 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:49.764+0000 7fbbacc01740 1 --2- 172.21.15.83:0/799283278 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c36d4ddd70 0x55c36d4e1d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:49.768 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:49.764+0000 7fbb90c00640 0 client.26167 ms_handle_reset on v2:172.21.15.117:6838/1967595704 2024-08-19T01:42:49.768 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:49.764+0000 7fbbacc01740 1 --2- 172.21.15.83:0/799283278 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c36d4d34b0 0x55c36d4e17f0 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:49.768 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:49.764+0000 7fbbacc01740 1 --2- 172.21.15.83:0/799283278 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c36d4d2b90 0x55c36d4e6470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:49.768 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:49.764+0000 7fbbacc01740 1 -- 172.21.15.83:0/799283278 >> 172.21.15.83:0/799283278 conn(0x55c36d4cf120 msgr2=0x55c36d4ce280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:42:49.768 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:49.765+0000 7fbbacc01740 1 -- 172.21.15.83:0/799283278 shutdown_connections 2024-08-19T01:42:49.768 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:49.765+0000 7fbbacc01740 1 -- 172.21.15.83:0/799283278 wait complete. 2024-08-19T01:42:49.768 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:49.765+0000 7fbbacc01740 20 client.26167 trim_cache size 0 max 16384 2024-08-19T01:42:49.769 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-08-19T01:42:49.769 INFO:teuthology.orchestra.run.smithi083.stdout: "status": "scrub active (1307 inodes in the stack)", 2024-08-19T01:42:49.769 INFO:teuthology.orchestra.run.smithi083.stdout: "scrubs": { 2024-08-19T01:42:49.769 INFO:teuthology.orchestra.run.smithi083.stdout: "de233f4c-1ddb-4f44-89e8-d482c9c1ff55": { 2024-08-19T01:42:49.769 INFO:teuthology.orchestra.run.smithi083.stdout: "path": "/", 2024-08-19T01:42:49.769 INFO:teuthology.orchestra.run.smithi083.stdout: "tag": "de233f4c-1ddb-4f44-89e8-d482c9c1ff55", 2024-08-19T01:42:49.769 INFO:teuthology.orchestra.run.smithi083.stdout: "options": "recursive,force" 2024-08-19T01:42:49.769 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-08-19T01:42:49.769 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-08-19T01:42:49.769 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-08-19T01:42:49.770 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:49.765+0000 7fbbaa000640 1 -- 172.21.15.83:0/2821130495 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7fbb880a2b80 msgr2=0x7fbb880a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:42:49.770 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:49.765+0000 7fbbaa000640 1 --2- 172.21.15.83:0/2821130495 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7fbb880a2b80 0x7fbb880a5040 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7fbb9c004770 tx=0x7fbb9c009260 comp rx=0 tx=0).stop 2024-08-19T01:42:49.770 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:49.765+0000 7fbbaa000640 1 -- 172.21.15.83:0/2821130495 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbba4102250 msgr2=0x7fbba419a8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:42:49.770 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:49.765+0000 7fbbaa000640 1 --2- 172.21.15.83:0/2821130495 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbba4102250 0x7fbba419a8b0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fbb8c00ea60 tx=0x7fbb8c00ef20 comp rx=0 tx=0).stop 2024-08-19T01:42:49.770 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:49.766+0000 7fbbaa000640 1 -- 172.21.15.83:0/2821130495 shutdown_connections 2024-08-19T01:42:49.771 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:49.766+0000 7fbbaa000640 1 --2- 172.21.15.83:0/2821130495 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7fbb880a2b80 0x7fbb880a5040 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:49.771 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:49.766+0000 7fbbaa000640 1 --2- 172.21.15.83:0/2821130495 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbba41078b0 0x7fbba419ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:49.771 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:49.766+0000 7fbbaa000640 1 --2- 172.21.15.83:0/2821130495 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbba4102250 0x7fbba419a8b0 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:49.771 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:49.766+0000 7fbbaa000640 1 --2- 172.21.15.83:0/2821130495 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbba41018b0 0x7fbba419a370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:49.771 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:49.766+0000 7fbbaa000640 1 -- 172.21.15.83:0/2821130495 >> 172.21.15.83:0/2821130495 conn(0x7fbba40fd5f0 msgr2=0x7fbba410ce60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:42:49.771 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:49.766+0000 7fbbaa000640 1 -- 172.21.15.83:0/2821130495 shutdown_connections 2024-08-19T01:42:49.771 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:42:49.766+0000 7fbbaa000640 1 -- 172.21.15.83:0/2821130495 wait complete. 2024-08-19T01:42:49.966 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:42:49 smithi167 ceph-mon[21875]: pgmap v578: 97 pgs: 97 active+clean; 143 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 2.2 MiB/s wr, 554 op/s 2024-08-19T01:42:49.966 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:42:49 smithi167 ceph-mon[21875]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-19T01:42:49.966 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:42:49 smithi167 ceph-mon[21875]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-19T01:42:49.966 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:42:49 smithi167 ceph-mon[21875]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' 2024-08-19T01:42:50.155 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:42:49 smithi083 ceph-mon[17120]: pgmap v578: 97 pgs: 97 active+clean; 143 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 2.2 MiB/s wr, 554 op/s 2024-08-19T01:42:50.155 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:42:49 smithi083 ceph-mon[17120]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-19T01:42:50.155 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:42:49 smithi083 ceph-mon[17120]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-19T01:42:50.155 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:42:49 smithi083 ceph-mon[17120]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' 2024-08-19T01:42:50.202 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:42:49 smithi117 ceph-mon[22016]: pgmap v578: 97 pgs: 97 active+clean; 143 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 2.2 MiB/s wr, 554 op/s 2024-08-19T01:42:50.202 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:42:49 smithi117 ceph-mon[22016]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-19T01:42:50.202 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:42:49 smithi117 ceph-mon[22016]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-19T01:42:50.202 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:42:49 smithi117 ceph-mon[22016]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' 2024-08-19T01:42:50.767 INFO:tasks.cephfs.filesystem:scrub status for tag:de233f4c-1ddb-4f44-89e8-d482c9c1ff55 - {'path': '/', 'tag': 'de233f4c-1ddb-4f44-89e8-d482c9c1ff55', 'options': 'recursive,force'} 2024-08-19T01:42:51.966 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:42:51 smithi167 ceph-mon[21875]: pgmap v579: 97 pgs: 97 active+clean; 139 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 2.2 MiB/s wr, 547 op/s 2024-08-19T01:42:51.966 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:42:51 smithi167 ceph-mon[21875]: scrub summary: idle+waiting paths [/] 2024-08-19T01:42:52.155 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:42:51 smithi083 ceph-mon[17120]: pgmap v579: 97 pgs: 97 active+clean; 139 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 2.2 MiB/s wr, 547 op/s 2024-08-19T01:42:52.155 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:42:51 smithi083 ceph-mon[17120]: scrub summary: idle+waiting paths [/] 2024-08-19T01:42:52.202 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:42:51 smithi117 ceph-mon[22016]: pgmap v579: 97 pgs: 97 active+clean; 139 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 2.2 MiB/s wr, 547 op/s 2024-08-19T01:42:52.202 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:42:51 smithi117 ceph-mon[22016]: scrub summary: idle+waiting paths [/] 2024-08-19T01:42:54.154 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:42:53 smithi083 ceph-mon[17120]: pgmap v580: 97 pgs: 97 active+clean; 131 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 1.6 MiB/s wr, 643 op/s 2024-08-19T01:42:54.202 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:42:53 smithi117 ceph-mon[22016]: pgmap v580: 97 pgs: 97 active+clean; 131 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 1.6 MiB/s wr, 643 op/s 2024-08-19T01:42:54.216 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:42:53 smithi167 ceph-mon[21875]: pgmap v580: 97 pgs: 97 active+clean; 131 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 1.6 MiB/s wr, 643 op/s 2024-08-19T01:42:54.904 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:42:54 smithi083 ceph-mon[17120]: pgmap v581: 97 pgs: 97 active+clean; 121 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 2.4 MiB/s wr, 948 op/s 2024-08-19T01:42:55.202 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:42:54 smithi117 ceph-mon[22016]: pgmap v581: 97 pgs: 97 active+clean; 121 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 2.4 MiB/s wr, 948 op/s 2024-08-19T01:42:55.216 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:42:54 smithi167 ceph-mon[21875]: pgmap v581: 97 pgs: 97 active+clean; 121 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 2.4 MiB/s wr, 948 op/s 2024-08-19T01:42:55.904 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:42:55 smithi083 ceph-mon[17120]: scrub summary: idle 2024-08-19T01:42:56.202 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:42:55 smithi117 ceph-mon[22016]: scrub summary: idle 2024-08-19T01:42:56.215 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:42:55 smithi167 ceph-mon[21875]: scrub summary: idle 2024-08-19T01:42:56.739 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:56.734+0000 7fd200e00640 1 -- 172.21.15.83:0/2028105949 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd1fc100880 msgr2=0x7fd1fc100ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:42:56.739 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:56.734+0000 7fd200e00640 1 --2- 172.21.15.83:0/2028105949 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd1fc100880 0x7fd1fc100ce0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7fd1e4009900 tx=0x7fd1e402a3f0 comp rx=0 tx=0).stop 2024-08-19T01:42:56.739 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:56.734+0000 7fd200e00640 1 -- 172.21.15.83:0/2028105949 shutdown_connections 2024-08-19T01:42:56.739 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:56.734+0000 7fd200e00640 1 --2- 172.21.15.83:0/2028105949 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd1fc101220 0x7fd1fc1016d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:56.739 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:56.734+0000 7fd200e00640 1 --2- 172.21.15.83:0/2028105949 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd1fc100880 0x7fd1fc100ce0 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:56.739 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:56.734+0000 7fd200e00640 1 --2- 172.21.15.83:0/2028105949 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd1fc0fff40 0x7fd1fc100340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:56.739 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:56.734+0000 7fd200e00640 1 -- 172.21.15.83:0/2028105949 >> 172.21.15.83:0/2028105949 conn(0x7fd1fc0fbcd0 msgr2=0x7fd1fc0fe110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:42:56.739 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:56.734+0000 7fd200e00640 1 -- 172.21.15.83:0/2028105949 shutdown_connections 2024-08-19T01:42:56.739 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:56.735+0000 7fd200e00640 1 -- 172.21.15.83:0/2028105949 wait complete. 2024-08-19T01:42:56.739 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:56.735+0000 7fd200e00640 1 Processor -- start 2024-08-19T01:42:56.739 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:56.736+0000 7fd200e00640 1 -- start start 2024-08-19T01:42:56.740 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:56.736+0000 7fd200e00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd1fc0fff40 0x7fd1fc199b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:42:56.740 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:56.736+0000 7fd200e00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd1fc100880 0x7fd1fc19a0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:42:56.740 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:56.736+0000 7fd200e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd1fc101220 0x7fd1fc1a1110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:42:56.740 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:56.736+0000 7fd200e00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fd1fc10f4d0 con 0x7fd1fc101220 2024-08-19T01:42:56.740 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:56.736+0000 7fd200e00640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fd1fc10f350 con 0x7fd1fc100880 2024-08-19T01:42:56.740 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:56.736+0000 7fd200e00640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fd1fc10f650 con 0x7fd1fc0fff40 2024-08-19T01:42:56.740 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:56.736+0000 7fd1f2a00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd1fc100880 0x7fd1fc19a0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:42:56.740 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:56.736+0000 7fd1f2a00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd1fc100880 0x7fd1fc19a0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.83:46782/0 (socket says 172.21.15.83:46782) 2024-08-19T01:42:56.740 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:56.736+0000 7fd1f2a00640 1 -- 172.21.15.83:0/3282639185 learned_addr learned my addr 172.21.15.83:0/3282639185 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-08-19T01:42:56.740 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:56.736+0000 7fd1fbe00640 1 --2- 172.21.15.83:0/3282639185 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd1fc101220 0x7fd1fc1a1110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:42:56.740 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:56.736+0000 7fd1fb400640 1 --2- 172.21.15.83:0/3282639185 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd1fc0fff40 0x7fd1fc199b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:42:56.740 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:56.736+0000 7fd1f2a00640 1 -- 172.21.15.83:0/3282639185 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd1fc0fff40 msgr2=0x7fd1fc199b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:42:56.740 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:56.736+0000 7fd1f2a00640 1 --2- 172.21.15.83:0/3282639185 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd1fc0fff40 0x7fd1fc199b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:56.741 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:56.736+0000 7fd1f2a00640 1 -- 172.21.15.83:0/3282639185 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd1fc101220 msgr2=0x7fd1fc1a1110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:42:56.741 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:56.736+0000 7fd1f2a00640 1 --2- 172.21.15.83:0/3282639185 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd1fc101220 0x7fd1fc1a1110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:56.741 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:56.736+0000 7fd1f2a00640 1 -- 172.21.15.83:0/3282639185 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd1fc1a1780 con 0x7fd1fc100880 2024-08-19T01:42:56.741 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:56.737+0000 7fd1f2a00640 1 --2- 172.21.15.83:0/3282639185 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd1fc100880 0x7fd1fc19a0a0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7fd1e4031440 tx=0x7fd1e4032ce0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:42:56.741 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:56.737+0000 7fd1f8c00640 1 -- 172.21.15.83:0/3282639185 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1e403f030 con 0x7fd1fc100880 2024-08-19T01:42:56.741 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:56.737+0000 7fd1f8c00640 1 -- 172.21.15.83:0/3282639185 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd1e4003ce0 con 0x7fd1fc100880 2024-08-19T01:42:56.741 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:56.737+0000 7fd200e00640 1 -- 172.21.15.83:0/3282639185 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd1fc1a1a10 con 0x7fd1fc100880 2024-08-19T01:42:56.741 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:56.737+0000 7fd1f8c00640 1 -- 172.21.15.83:0/3282639185 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1e4035070 con 0x7fd1fc100880 2024-08-19T01:42:56.741 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:56.737+0000 7fd200e00640 1 -- 172.21.15.83:0/3282639185 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd1fc1a1fb0 con 0x7fd1fc100880 2024-08-19T01:42:56.743 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:56.740+0000 7fd1f8c00640 1 -- 172.21.15.83:0/3282639185 <== mon.2 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7fd1e4042030 con 0x7fd1fc100880 2024-08-19T01:42:56.743 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:56.740+0000 7fd20542e740 1 Processor -- start 2024-08-19T01:42:56.743 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:56.740+0000 7fd20542e740 1 -- start start 2024-08-19T01:42:56.744 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:56.740+0000 7fd20542e740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5563d598b680 0x5563d598ba80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:42:56.744 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:56.740+0000 7fd20542e740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5563d598bfc0 0x5563d5990a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:42:56.744 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:56.740+0000 7fd20542e740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5563d5991050 0x5563d5993510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:42:56.744 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:56.740+0000 7fd20542e740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x5563d58ef890 con 0x5563d598b680 2024-08-19T01:42:56.744 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:56.740+0000 7fd20542e740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x5563d5887d90 con 0x5563d598bfc0 2024-08-19T01:42:56.744 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:56.740+0000 7fd20542e740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x5563d58eb850 con 0x5563d5991050 2024-08-19T01:42:56.744 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:56.741+0000 7fd1fbe00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5563d598b680 0x5563d598ba80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:42:56.744 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:56.741+0000 7fd1fb400640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5563d598bfc0 0x5563d5990a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:42:56.744 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:56.741+0000 7fd1fbe00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5563d598b680 0x5563d598ba80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:47768/0 (socket says 172.21.15.83:47768) 2024-08-19T01:42:56.744 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:56.741+0000 7fd1fbe00640 1 -- 172.21.15.83:0/250974485 learned_addr learned my addr 172.21.15.83:0/250974485 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-08-19T01:42:56.744 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:56.741+0000 7fd1f2a00640 1 --2- 172.21.15.83:0/250974485 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5563d5991050 0x5563d5993510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:42:56.744 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:56.741+0000 7fd1fb400640 1 -- 172.21.15.83:0/250974485 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5563d5991050 msgr2=0x5563d5993510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:42:56.744 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:56.741+0000 7fd1fb400640 1 --2- 172.21.15.83:0/250974485 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5563d5991050 0x5563d5993510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:56.745 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:56.741+0000 7fd1fb400640 1 -- 172.21.15.83:0/250974485 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5563d598b680 msgr2=0x5563d598ba80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:42:56.745 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:56.741+0000 7fd1fb400640 1 --2- 172.21.15.83:0/250974485 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5563d598b680 0x5563d598ba80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:56.745 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:56.741+0000 7fd1fb400640 1 -- 172.21.15.83:0/250974485 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5563d5603210 con 0x5563d598bfc0 2024-08-19T01:42:56.745 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:56.741+0000 7fd1fbe00640 1 --2- 172.21.15.83:0/250974485 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5563d598b680 0x5563d598ba80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-19T01:42:56.745 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:56.741+0000 7fd1fb400640 1 --2- 172.21.15.83:0/250974485 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5563d598bfc0 0x5563d5990a80 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7fd1ec009f50 tx=0x7fd1ec010860 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a82bca0f05135f9 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:42:56.745 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:56.742+0000 7fd1f0c00640 1 -- 172.21.15.83:0/250974485 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1ec003b70 con 0x5563d598bfc0 2024-08-19T01:42:56.745 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:56.742+0000 7fd1f0c00640 1 -- 172.21.15.83:0/250974485 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd1ec015070 con 0x5563d598bfc0 2024-08-19T01:42:56.745 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:56.742+0000 7fd1f0c00640 1 -- 172.21.15.83:0/250974485 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1ec011930 con 0x5563d598bfc0 2024-08-19T01:42:56.745 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:56.742+0000 7fd1f8c00640 1 --2- 172.21.15.83:0/3282639185 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7fd1e00a2c00 0x7fd1e00a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:42:56.745 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:56.742+0000 7fd1f8c00640 1 -- 172.21.15.83:0/3282639185 <== mon.2 v2:172.21.15.117:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8269+0+0 (secure 0 0 0) 0x7fd1e40ef910 con 0x7fd1fc100880 2024-08-19T01:42:56.745 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:56.742+0000 7fd1fbe00640 1 --2- 172.21.15.83:0/3282639185 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7fd1e00a2c00 0x7fd1e00a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:42:56.745 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:56.742+0000 7fd20542e740 1 -- 172.21.15.83:0/250974485 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5563d598bfc0 msgr2=0x5563d5990a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:42:56.745 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:56.742+0000 7fd20542e740 1 --2- 172.21.15.83:0/250974485 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5563d598bfc0 0x5563d5990a80 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7fd1ec009f50 tx=0x7fd1ec010860 comp rx=0 tx=0).stop 2024-08-19T01:42:56.749 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:56.742+0000 7fd20542e740 1 -- 172.21.15.83:0/250974485 shutdown_connections 2024-08-19T01:42:56.749 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:56.742+0000 7fd20542e740 1 --2- 172.21.15.83:0/250974485 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5563d5991050 0x5563d5993510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:56.749 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:56.742+0000 7fd20542e740 1 --2- 172.21.15.83:0/250974485 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5563d598bfc0 0x5563d5990a80 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:56.749 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:56.742+0000 7fd20542e740 1 --2- 172.21.15.83:0/250974485 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5563d598b680 0x5563d598ba80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:56.749 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:56.742+0000 7fd20542e740 1 -- 172.21.15.83:0/250974485 >> 172.21.15.83:0/250974485 conn(0x5563d5986cf0 msgr2=0x5563d59870f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:42:56.749 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:56.743+0000 7fd1fbe00640 1 --2- 172.21.15.83:0/3282639185 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7fd1e00a2c00 0x7fd1e00a50c0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7fd1f4012560 tx=0x7fd1f4005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:42:56.749 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:56.743+0000 7fd20542e740 1 -- 172.21.15.83:0/250974485 shutdown_connections 2024-08-19T01:42:56.749 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:56.743+0000 7fd20542e740 1 -- 172.21.15.83:0/250974485 wait complete. 2024-08-19T01:42:56.749 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:56.743+0000 7fd20542e740 1 -- start start 2024-08-19T01:42:56.750 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:56.743+0000 7fd20542e740 1 Processor -- start 2024-08-19T01:42:56.750 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:56.744+0000 7fd20542e740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5563d598b680 0x5563d599e180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:42:56.750 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:56.744+0000 7fd20542e740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5563d598bfc0 0x5563d599e6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:42:56.750 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:56.744+0000 7fd20542e740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5563d5991050 0x5563d59a5700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:42:56.750 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:56.744+0000 7fd20542e740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x5563d59940c0 con 0x5563d5991050 2024-08-19T01:42:56.750 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:56.744+0000 7fd20542e740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x5563d58aaf70 con 0x5563d598b680 2024-08-19T01:42:56.750 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:56.744+0000 7fd20542e740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x5563d5994240 con 0x5563d598bfc0 2024-08-19T01:42:56.750 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:56.744+0000 7fd20542e740 10 client.0 _ll_register_callbacks cb 0x5563d58e78d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-19T01:42:56.750 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:56.744+0000 7fd1fb400640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5563d598b680 0x5563d599e180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:42:56.750 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:56.744+0000 7fd1fbe00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5563d5991050 0x5563d59a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:42:56.750 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:56.744+0000 7fd1fb400640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5563d598b680 0x5563d599e180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.83:46800/0 (socket says 172.21.15.83:46800) 2024-08-19T01:42:56.750 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:56.744+0000 7fd1f2a00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5563d598bfc0 0x5563d599e6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:42:56.750 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:56.744+0000 7fd1fbe00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5563d5991050 0x5563d59a5700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:47772/0 (socket says 172.21.15.83:47772) 2024-08-19T01:42:56.751 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:56.744+0000 7fd1fb400640 1 -- 172.21.15.83:0/1236200207 learned_addr learned my addr 172.21.15.83:0/1236200207 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-08-19T01:42:56.751 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:56.744+0000 7fd1f2a00640 1 -- 172.21.15.83:0/1236200207 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5563d598b680 msgr2=0x5563d599e180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:42:56.751 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:56.744+0000 7fd1f2a00640 1 --2- 172.21.15.83:0/1236200207 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5563d598b680 0x5563d599e180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:56.751 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:56.744+0000 7fd1f2a00640 1 -- 172.21.15.83:0/1236200207 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5563d5991050 msgr2=0x5563d59a5700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:42:56.751 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:56.744+0000 7fd1f2a00640 1 --2- 172.21.15.83:0/1236200207 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5563d5991050 0x5563d59a5700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:56.751 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:56.744+0000 7fd1f2a00640 1 -- 172.21.15.83:0/1236200207 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5563d56cf620 con 0x5563d598bfc0 2024-08-19T01:42:56.751 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:56.744+0000 7fd1fb400640 1 --2- 172.21.15.83:0/1236200207 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5563d598b680 0x5563d599e180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-19T01:42:56.751 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:56.745+0000 7fd1f2a00640 1 --2- 172.21.15.83:0/1236200207 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5563d598bfc0 0x5563d599e6c0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fd1e40ed390 tx=0x7fd1e4009620 comp rx=0 tx=0).ready entity=mon.1 client_cookie=da8b91e7980555ba server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:42:56.751 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:56.745+0000 7fd1e9600640 10 client.0 ms_handle_connect on v2:172.21.15.167:3300/0 2024-08-19T01:42:56.751 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:56.745+0000 7fd1e9600640 1 -- 172.21.15.83:0/1236200207 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1e4049040 con 0x5563d598bfc0 2024-08-19T01:42:56.751 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:56.745+0000 7fd20542e740 1 -- 172.21.15.83:0/1236200207 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd1ec015070 con 0x5563d598bfc0 2024-08-19T01:42:56.751 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:56.745+0000 7fd1e9600640 1 -- 172.21.15.83:0/1236200207 <== mon.1 v2:172.21.15.167:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8269+0+0 (secure 0 0 0) 0x7fd1e40dc070 con 0x5563d598bfc0 2024-08-19T01:42:56.751 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:56.745+0000 7fd1e9600640 1 -- 172.21.15.83:0/1236200207 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fd1e40bba50 con 0x5563d598bfc0 2024-08-19T01:42:56.751 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:56.745+0000 7fd20542e740 10 client.26179 fetch_fsmap learned FSMap version 29 2024-08-19T01:42:56.752 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:56.745+0000 7fd20542e740 1 -- 172.21.15.83:0/1236200207 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5563d59940c0 con 0x5563d598bfc0 2024-08-19T01:42:56.752 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:56.745+0000 7fd1e9600640 1 -- 172.21.15.83:0/1236200207 <== mon.1 v2:172.21.15.167:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd1ec015070 con 0x5563d598bfc0 2024-08-19T01:42:56.752 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:56.746+0000 7fd1e9600640 1 -- 172.21.15.83:0/1236200207 <== mon.1 v2:172.21.15.167:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7fd1e40d6080 con 0x5563d598bfc0 2024-08-19T01:42:56.752 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:56.746+0000 7fd20542e740 10 client.26179 fetch_fsmap finished waiting for FSMap version 29 2024-08-19T01:42:56.752 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:56.746+0000 7fd20542e740 10 client.26179 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-08-19T01:42:56.752 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:56.746+0000 7fd20542e740 20 client.26179 populate_metadata read hostname 'smithi083' 2024-08-19T01:42:56.752 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:56.746+0000 7fd20542e740 1 --2- 172.21.15.83:0/1236200207 >> [v2:172.21.15.117:6838/1967595704,v1:172.21.15.117:6839/1967595704] conn(0x5563d59aa0a0 0x5563d59ac4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:42:56.752 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:56.746+0000 7fd20542e740 4 client.26179 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-19T01:42:56.752 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:56.746+0000 7fd20542e740 1 -- 172.21.15.83:0/1236200207 --> [v2:172.21.15.117:6838/1967595704,v1:172.21.15.117:6839/1967595704] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5563d58fcfc0 con 0x5563d59aa0a0 2024-08-19T01:42:56.752 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:56.746+0000 7fd1fb400640 1 --2- 172.21.15.83:0/1236200207 >> [v2:172.21.15.117:6838/1967595704,v1:172.21.15.117:6839/1967595704] conn(0x5563d59aa0a0 0x5563d59ac4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:42:56.752 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:56.746+0000 7fd1fb400640 1 --2- 172.21.15.83:0/1236200207 >> [v2:172.21.15.117:6838/1967595704,v1:172.21.15.117:6839/1967595704] conn(0x5563d59aa0a0 0x5563d59ac4e0 crc :-1 s=READY pgs=145 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=62db0e842bc16bcd server_cookie=86ae8c934066312f in_seq=0 out_seq=0 2024-08-19T01:42:56.752 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:56.746+0000 7fd1e9600640 10 client.26179 ms_handle_connect on v2:172.21.15.117:6838/1967595704 2024-08-19T01:42:56.753 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:56.748+0000 7fd1e9600640 1 -- 172.21.15.83:0/1236200207 <== mds.0 v2:172.21.15.117:6838/1967595704 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5563d58fcfc0 con 0x5563d59aa0a0 2024-08-19T01:42:56.753 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:56.748+0000 7fd1e9600640 10 client.26179 handle_command_reply: tid=0 2024-08-19T01:42:56.754 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:56.748+0000 7fd20542e740 1 client.26179 shutdown 2024-08-19T01:42:56.754 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:56.748+0000 7fd20542e740 1 -- 172.21.15.83:0/1236200207 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5563d598bfc0 msgr2=0x5563d599e6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:42:56.754 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:56.748+0000 7fd20542e740 1 --2- 172.21.15.83:0/1236200207 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5563d598bfc0 0x5563d599e6c0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fd1e40ed390 tx=0x7fd1e4009620 comp rx=0 tx=0).stop 2024-08-19T01:42:56.754 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:56.748+0000 7fd20542e740 1 -- 172.21.15.83:0/1236200207 shutdown_connections 2024-08-19T01:42:56.754 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:56.748+0000 7fd20542e740 1 --2- 172.21.15.83:0/1236200207 >> [v2:172.21.15.117:6838/1967595704,v1:172.21.15.117:6839/1967595704] conn(0x5563d59aa0a0 0x5563d59ac4e0 crc :-1 s=READY pgs=145 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:56.754 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:56.748+0000 7fd20542e740 1 --2- 172.21.15.83:0/1236200207 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5563d5991050 0x5563d59a5700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:56.754 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:56.748+0000 7fd20542e740 1 --2- 172.21.15.83:0/1236200207 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5563d598bfc0 0x5563d599e6c0 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:56.754 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:56.748+0000 7fd20542e740 1 --2- 172.21.15.83:0/1236200207 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5563d598b680 0x5563d599e180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:56.754 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:56.748+0000 7fd1e9600640 0 client.26179 ms_handle_reset on v2:172.21.15.117:6838/1967595704 2024-08-19T01:42:56.754 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:56.748+0000 7fd20542e740 1 -- 172.21.15.83:0/1236200207 >> 172.21.15.83:0/1236200207 conn(0x5563d5986cf0 msgr2=0x5563d5988500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:42:56.754 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:56.749+0000 7fd20542e740 1 -- 172.21.15.83:0/1236200207 shutdown_connections 2024-08-19T01:42:56.754 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:56.749+0000 7fd20542e740 1 -- 172.21.15.83:0/1236200207 wait complete. 2024-08-19T01:42:56.755 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:56.749+0000 7fd20542e740 20 client.26179 trim_cache size 0 max 16384 2024-08-19T01:42:56.767 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:56.765+0000 7fd20542e740 1 Processor -- start 2024-08-19T01:42:56.767 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:56.765+0000 7fd20542e740 1 -- start start 2024-08-19T01:42:56.767 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:56.766+0000 7fd20542e740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5563d5986cf0 0x5563d59907e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:42:56.768 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:56.766+0000 7fd20542e740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5563d598b680 0x5563d59a56c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:42:56.768 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:56.766+0000 7fd20542e740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5563d598bfc0 0x5563d599e180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:42:56.768 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:56.766+0000 7fd20542e740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fd1fc1a92c0 con 0x5563d598b680 2024-08-19T01:42:56.768 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:56.766+0000 7fd20542e740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fd1fc1a6b30 con 0x5563d5986cf0 2024-08-19T01:42:56.768 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:56.766+0000 7fd20542e740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fd1fc1a4980 con 0x5563d598bfc0 2024-08-19T01:42:56.768 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:56.766+0000 7fd1fb400640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5563d5986cf0 0x5563d59907e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:42:56.768 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:56.766+0000 7fd1f2a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5563d598b680 0x5563d59a56c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:42:56.768 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:56.766+0000 7fd1fbe00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5563d598bfc0 0x5563d599e180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:42:56.768 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:56.766+0000 7fd1fb400640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5563d5986cf0 0x5563d59907e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.83:46810/0 (socket says 172.21.15.83:46810) 2024-08-19T01:42:56.768 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:56.766+0000 7fd1fbe00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5563d598bfc0 0x5563d599e180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.83:43100/0 (socket says 172.21.15.83:43100) 2024-08-19T01:42:56.768 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:56.766+0000 7fd1fb400640 1 -- 172.21.15.83:0/422176237 learned_addr learned my addr 172.21.15.83:0/422176237 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-08-19T01:42:56.772 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:56.766+0000 7fd1fb400640 1 -- 172.21.15.83:0/422176237 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5563d598bfc0 msgr2=0x5563d599e180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:42:56.772 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:56.766+0000 7fd1fb400640 1 --2- 172.21.15.83:0/422176237 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5563d598bfc0 0x5563d599e180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:56.772 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:56.766+0000 7fd1fb400640 1 -- 172.21.15.83:0/422176237 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5563d598b680 msgr2=0x5563d59a56c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:42:56.772 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:56.766+0000 7fd1fb400640 1 --2- 172.21.15.83:0/422176237 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5563d598b680 0x5563d59a56c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:56.772 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:56.766+0000 7fd1fb400640 1 -- 172.21.15.83:0/422176237 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5563d5982700 con 0x5563d5986cf0 2024-08-19T01:42:56.772 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:56.767+0000 7fd1fb400640 1 --2- 172.21.15.83:0/422176237 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5563d5986cf0 0x5563d59907e0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x5563d5994ea0 tx=0x7fd1ec013040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=284e187c9d1b7621 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:42:56.772 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:56.767+0000 7fd1ebe00640 1 -- 172.21.15.83:0/422176237 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1ec02c020 con 0x5563d5986cf0 2024-08-19T01:42:56.772 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:56.767+0000 7fd1ebe00640 1 -- 172.21.15.83:0/422176237 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd1ec015070 con 0x5563d5986cf0 2024-08-19T01:42:56.772 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:56.767+0000 7fd1ebe00640 1 -- 172.21.15.83:0/422176237 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1ec011a30 con 0x5563d5986cf0 2024-08-19T01:42:56.773 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:56.767+0000 7fd20542e740 1 -- 172.21.15.83:0/422176237 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5563d5986cf0 msgr2=0x5563d59907e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:42:56.773 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:56.767+0000 7fd20542e740 1 --2- 172.21.15.83:0/422176237 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5563d5986cf0 0x5563d59907e0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x5563d5994ea0 tx=0x7fd1ec013040 comp rx=0 tx=0).stop 2024-08-19T01:42:56.773 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:56.767+0000 7fd20542e740 1 -- 172.21.15.83:0/422176237 shutdown_connections 2024-08-19T01:42:56.773 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:56.767+0000 7fd20542e740 1 --2- 172.21.15.83:0/422176237 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5563d598bfc0 0x5563d599e180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:56.773 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:56.767+0000 7fd20542e740 1 --2- 172.21.15.83:0/422176237 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5563d598b680 0x5563d59a56c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:56.773 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:56.767+0000 7fd20542e740 1 --2- 172.21.15.83:0/422176237 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5563d5986cf0 0x5563d59907e0 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:56.773 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:56.767+0000 7fd20542e740 1 -- 172.21.15.83:0/422176237 >> 172.21.15.83:0/422176237 conn(0x5563d598ca80 msgr2=0x5563d598d1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:42:56.773 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:56.768+0000 7fd20542e740 1 -- 172.21.15.83:0/422176237 shutdown_connections 2024-08-19T01:42:56.773 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:56.768+0000 7fd20542e740 1 -- 172.21.15.83:0/422176237 wait complete. 2024-08-19T01:42:56.773 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:56.768+0000 7fd20542e740 1 -- start start 2024-08-19T01:42:56.773 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:56.768+0000 7fd20542e740 1 Processor -- start 2024-08-19T01:42:56.773 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:56.768+0000 7fd20542e740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5563d5986cf0 0x5563d59a2300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:42:56.773 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:56.768+0000 7fd20542e740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5563d598b680 0x5563d59a2840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:42:56.774 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:56.768+0000 7fd20542e740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5563d598bfc0 0x5563d5a12d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:42:56.774 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:56.768+0000 7fd20542e740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fd1fc1a4ff0 con 0x5563d5986cf0 2024-08-19T01:42:56.774 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:56.768+0000 7fd20542e740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fd1fc1a6460 con 0x5563d598bfc0 2024-08-19T01:42:56.774 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:56.768+0000 7fd20542e740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fd1fc1a56c0 con 0x5563d598b680 2024-08-19T01:42:56.774 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:56.768+0000 7fd20542e740 10 client.0 _ll_register_callbacks cb 0x7fd1e4005b50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-19T01:42:56.774 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:56.769+0000 7fd1fbe00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5563d598bfc0 0x5563d5a12d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:42:56.774 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:56.769+0000 7fd1fb400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5563d5986cf0 0x5563d59a2300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:42:56.774 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:56.769+0000 7fd1fbe00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5563d598bfc0 0x5563d5a12d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.83:46822/0 (socket says 172.21.15.83:46822) 2024-08-19T01:42:56.774 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:56.769+0000 7fd1fbe00640 1 -- 172.21.15.83:0/54585792 learned_addr learned my addr 172.21.15.83:0/54585792 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-08-19T01:42:56.774 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:56.769+0000 7fd1f2a00640 1 --2- 172.21.15.83:0/54585792 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5563d598b680 0x5563d59a2840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:42:56.774 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:56.769+0000 7fd1fbe00640 1 -- 172.21.15.83:0/54585792 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5563d598b680 msgr2=0x5563d59a2840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:42:56.774 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:56.769+0000 7fd1fbe00640 1 --2- 172.21.15.83:0/54585792 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5563d598b680 0x5563d59a2840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:56.774 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:56.769+0000 7fd1fbe00640 1 -- 172.21.15.83:0/54585792 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5563d5986cf0 msgr2=0x5563d59a2300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:42:56.775 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:56.769+0000 7fd1fbe00640 1 --2- 172.21.15.83:0/54585792 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5563d5986cf0 0x5563d59a2300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:56.775 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:56.769+0000 7fd1fbe00640 1 -- 172.21.15.83:0/54585792 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5563d59a5930 con 0x5563d598bfc0 2024-08-19T01:42:56.775 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:56.769+0000 7fd1fb400640 1 --2- 172.21.15.83:0/54585792 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5563d5986cf0 0x5563d59a2300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-19T01:42:56.775 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:56.769+0000 7fd1fbe00640 1 --2- 172.21.15.83:0/54585792 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5563d598bfc0 0x5563d5a12d70 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7fd1f400fa50 tx=0x7fd1f4004730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=aa9d05332b0f55f0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:42:56.775 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:56.769+0000 7fd1e9600640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-08-19T01:42:56.775 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:56.770+0000 7fd1e9600640 1 -- 172.21.15.83:0/54585792 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1f4004020 con 0x5563d598bfc0 2024-08-19T01:42:56.775 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:56.770+0000 7fd20542e740 1 -- 172.21.15.83:0/54585792 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd1ec015070 con 0x5563d598bfc0 2024-08-19T01:42:56.775 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:56.770+0000 7fd1e9600640 1 -- 172.21.15.83:0/54585792 <== mon.2 v2:172.21.15.117:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8269+0+0 (secure 0 0 0) 0x7fd1f40041c0 con 0x5563d598bfc0 2024-08-19T01:42:56.775 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:56.770+0000 7fd1e9600640 1 -- 172.21.15.83:0/54585792 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fd1f40043c0 con 0x5563d598bfc0 2024-08-19T01:42:56.775 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:56.770+0000 7fd20542e740 10 client.26255 fetch_fsmap learned FSMap version 29 2024-08-19T01:42:56.775 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:56.770+0000 7fd20542e740 1 -- 172.21.15.83:0/54585792 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5563d5a14410 con 0x5563d598bfc0 2024-08-19T01:42:56.775 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:56.770+0000 7fd1e9600640 1 -- 172.21.15.83:0/54585792 <== mon.2 v2:172.21.15.117:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd1ec015070 con 0x5563d598bfc0 2024-08-19T01:42:56.775 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:56.770+0000 7fd1e9600640 1 -- 172.21.15.83:0/54585792 <== mon.2 v2:172.21.15.117:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7fd1f401d7a0 con 0x5563d598bfc0 2024-08-19T01:42:56.775 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:56.770+0000 7fd20542e740 10 client.26255 fetch_fsmap finished waiting for FSMap version 29 2024-08-19T01:42:56.776 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:56.770+0000 7fd20542e740 10 client.26255 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-08-19T01:42:56.776 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:56.770+0000 7fd20542e740 20 client.26255 populate_metadata read hostname 'smithi083' 2024-08-19T01:42:56.776 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:56.770+0000 7fd20542e740 1 --2- 172.21.15.83:0/54585792 >> [v2:172.21.15.117:6838/1967595704,v1:172.21.15.117:6839/1967595704] conn(0x5563d5991050 0x5563d5a16e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:42:56.776 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:56.770+0000 7fd20542e740 4 client.26255 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-08-19T01:42:56.776 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:56.770+0000 7fd20542e740 1 -- 172.21.15.83:0/54585792 --> [v2:172.21.15.117:6838/1967595704,v1:172.21.15.117:6839/1967595704] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5563d58f7310 con 0x5563d5991050 2024-08-19T01:42:56.776 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:56.771+0000 7fd1fb400640 1 --2- 172.21.15.83:0/54585792 >> [v2:172.21.15.117:6838/1967595704,v1:172.21.15.117:6839/1967595704] conn(0x5563d5991050 0x5563d5a16e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:42:56.776 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:56.771+0000 7fd1fb400640 1 --2- 172.21.15.83:0/54585792 >> [v2:172.21.15.117:6838/1967595704,v1:172.21.15.117:6839/1967595704] conn(0x5563d5991050 0x5563d5a16e30 crc :-1 s=READY pgs=146 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=28f8cd55a8fec88a server_cookie=45c3f66d71c09250 in_seq=0 out_seq=0 2024-08-19T01:42:56.776 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:56.771+0000 7fd1e9600640 10 client.26255 ms_handle_connect on v2:172.21.15.117:6838/1967595704 2024-08-19T01:42:56.904 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:42:56 smithi083 ceph-mon[17120]: scrub summary: idle 2024-08-19T01:42:56.904 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:42:56 smithi083 ceph-mon[17120]: pgmap v582: 97 pgs: 97 active+clean; 112 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 1.8 MiB/s wr, 885 op/s 2024-08-19T01:42:57.147 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:57.145+0000 7fd1e9600640 1 -- 172.21.15.83:0/54585792 <== mds.0 v2:172.21.15.117:6838/1967595704 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5563d58f7310 con 0x5563d5991050 2024-08-19T01:42:57.147 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:57.145+0000 7fd1e9600640 10 client.26255 handle_command_reply: tid=0 2024-08-19T01:42:57.147 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:57.145+0000 7fd20542e740 1 client.26255 shutdown 2024-08-19T01:42:57.148 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:57.145+0000 7fd20542e740 1 -- 172.21.15.83:0/54585792 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5563d598bfc0 msgr2=0x5563d5a12d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:42:57.148 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:57.145+0000 7fd20542e740 1 --2- 172.21.15.83:0/54585792 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5563d598bfc0 0x5563d5a12d70 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7fd1f400fa50 tx=0x7fd1f4004730 comp rx=0 tx=0).stop 2024-08-19T01:42:57.148 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:57.145+0000 7fd20542e740 1 -- 172.21.15.83:0/54585792 shutdown_connections 2024-08-19T01:42:57.148 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:57.145+0000 7fd20542e740 1 --2- 172.21.15.83:0/54585792 >> [v2:172.21.15.117:6838/1967595704,v1:172.21.15.117:6839/1967595704] conn(0x5563d5991050 0x5563d5a16e30 crc :-1 s=READY pgs=146 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:57.148 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:57.145+0000 7fd20542e740 1 --2- 172.21.15.83:0/54585792 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5563d598bfc0 0x5563d5a12d70 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:57.148 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:57.145+0000 7fd20542e740 1 --2- 172.21.15.83:0/54585792 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5563d598b680 0x5563d59a2840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:57.148 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:57.145+0000 7fd20542e740 1 --2- 172.21.15.83:0/54585792 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5563d5986cf0 0x5563d59a2300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:57.148 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:57.145+0000 7fd20542e740 1 -- 172.21.15.83:0/54585792 >> 172.21.15.83:0/54585792 conn(0x5563d598ca80 msgr2=0x5563d598d1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:42:57.148 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:57.146+0000 7fd1e9600640 0 client.26255 ms_handle_reset on v2:172.21.15.117:6838/1967595704 2024-08-19T01:42:57.148 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:57.146+0000 7fd20542e740 1 -- 172.21.15.83:0/54585792 shutdown_connections 2024-08-19T01:42:57.148 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:57.146+0000 7fd20542e740 1 -- 172.21.15.83:0/54585792 wait complete. 2024-08-19T01:42:57.148 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:57.146+0000 7fd20542e740 20 client.26255 trim_cache size 0 max 16384 2024-08-19T01:42:57.149 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stdout:{ 2024-08-19T01:42:57.149 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stdout: "message": "", 2024-08-19T01:42:57.149 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stdout: "return_code": 0 2024-08-19T01:42:57.149 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stdout:} 2024-08-19T01:42:57.150 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:57.147+0000 7fd200e00640 1 -- 172.21.15.83:0/3282639185 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7fd1e00a2c00 msgr2=0x7fd1e00a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:42:57.150 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:57.147+0000 7fd200e00640 1 --2- 172.21.15.83:0/3282639185 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7fd1e00a2c00 0x7fd1e00a50c0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7fd1f4012560 tx=0x7fd1f4005f90 comp rx=0 tx=0).stop 2024-08-19T01:42:57.150 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:57.147+0000 7fd200e00640 1 -- 172.21.15.83:0/3282639185 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd1fc100880 msgr2=0x7fd1fc19a0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:42:57.150 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:57.147+0000 7fd200e00640 1 --2- 172.21.15.83:0/3282639185 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd1fc100880 0x7fd1fc19a0a0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7fd1e4031440 tx=0x7fd1e4032ce0 comp rx=0 tx=0).stop 2024-08-19T01:42:57.150 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:57.147+0000 7fd200e00640 1 -- 172.21.15.83:0/3282639185 shutdown_connections 2024-08-19T01:42:57.150 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:57.147+0000 7fd200e00640 1 --2- 172.21.15.83:0/3282639185 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7fd1e00a2c00 0x7fd1e00a50c0 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:57.150 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:57.147+0000 7fd200e00640 1 --2- 172.21.15.83:0/3282639185 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd1fc101220 0x7fd1fc1a1110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:57.150 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:57.147+0000 7fd200e00640 1 --2- 172.21.15.83:0/3282639185 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd1fc100880 0x7fd1fc19a0a0 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:57.150 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:57.147+0000 7fd200e00640 1 --2- 172.21.15.83:0/3282639185 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd1fc0fff40 0x7fd1fc199b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:57.151 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:57.147+0000 7fd200e00640 1 -- 172.21.15.83:0/3282639185 >> 172.21.15.83:0/3282639185 conn(0x7fd1fc0fbcd0 msgr2=0x7fd1fc0fd4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:42:57.151 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:57.147+0000 7fd200e00640 1 -- 172.21.15.83:0/3282639185 shutdown_connections 2024-08-19T01:42:57.151 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-08-19T01:42:57.147+0000 7fd200e00640 1 -- 172.21.15.83:0/3282639185 wait complete. 2024-08-19T01:42:57.202 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:42:56 smithi117 ceph-mon[22016]: scrub summary: idle 2024-08-19T01:42:57.202 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:42:56 smithi117 ceph-mon[22016]: pgmap v582: 97 pgs: 97 active+clean; 112 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 1.8 MiB/s wr, 885 op/s 2024-08-19T01:42:57.215 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:42:56 smithi167 ceph-mon[21875]: scrub summary: idle 2024-08-19T01:42:57.215 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:42:56 smithi167 ceph-mon[21875]: pgmap v582: 97 pgs: 97 active+clean; 112 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 1.8 MiB/s wr, 885 op/s 2024-08-19T01:42:59.904 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:42:59 smithi083 ceph-mon[17120]: pgmap v583: 97 pgs: 97 active+clean; 83 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 1.8 MiB/s wr, 936 op/s 2024-08-19T01:42:59.952 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:42:59 smithi117 ceph-mon[22016]: pgmap v583: 97 pgs: 97 active+clean; 83 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 1.8 MiB/s wr, 936 op/s 2024-08-19T01:42:59.966 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:42:59 smithi167 ceph-mon[21875]: pgmap v583: 97 pgs: 97 active+clean; 83 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 1.8 MiB/s wr, 936 op/s 2024-08-19T01:43:01.904 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:43:01 smithi083 ceph-mon[17120]: pgmap v584: 97 pgs: 97 active+clean; 66 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 1.9 MiB/s wr, 838 op/s 2024-08-19T01:43:01.952 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:43:01 smithi117 ceph-mon[22016]: pgmap v584: 97 pgs: 97 active+clean; 66 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 1.9 MiB/s wr, 838 op/s 2024-08-19T01:43:01.965 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:43:01 smithi167 ceph-mon[21875]: pgmap v584: 97 pgs: 97 active+clean; 66 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 1.9 MiB/s wr, 838 op/s 2024-08-19T01:43:02.903 INFO:tasks.workunit:Stopping ['suites/blogbench.sh'] on client.0... 2024-08-19T01:43:02.903 DEBUG:teuthology.orchestra.run.smithi083:> sudo rm -rf -- /home/ubuntu/cephtest/workunits.list.client.0 /home/ubuntu/cephtest/clone.client.0 2024-08-19T01:43:03.904 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:43:03 smithi083 ceph-mon[17120]: pgmap v585: 97 pgs: 97 active+clean; 58 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.4 MiB/s wr, 779 op/s 2024-08-19T01:43:03.952 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:43:03 smithi117 ceph-mon[22016]: pgmap v585: 97 pgs: 97 active+clean; 58 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.4 MiB/s wr, 779 op/s 2024-08-19T01:43:03.966 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:43:03 smithi167 ceph-mon[21875]: pgmap v585: 97 pgs: 97 active+clean; 58 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.4 MiB/s wr, 779 op/s 2024-08-19T01:43:04.060 DEBUG:teuthology.parallel:result is None 2024-08-19T01:43:04.060 DEBUG:teuthology.orchestra.run.smithi083:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0 2024-08-19T01:43:04.130 INFO:tasks.workunit:Deleted dir /home/ubuntu/cephtest/mnt.0/client.0 2024-08-19T01:43:04.130 DEBUG:teuthology.run_tasks:Unwinding manager background_exec 2024-08-19T01:43:04.140 INFO:teuthology.task.background_exec:Stopping background command on ubuntu@smithi083.front.sepia.ceph.com 2024-08-19T01:43:04.152 DEBUG:teuthology.run_tasks:Unwinding manager fwd_scrub 2024-08-19T01:43:04.161 INFO:tasks.fwd_scrub:joining ForwardScrubbers 2024-08-19T01:43:05.904 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:43:05 smithi083 ceph-mon[17120]: pgmap v586: 97 pgs: 97 active+clean; 43 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 2.1 MiB/s wr, 757 op/s 2024-08-19T01:43:05.952 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:43:05 smithi117 ceph-mon[22016]: pgmap v586: 97 pgs: 97 active+clean; 43 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 2.1 MiB/s wr, 757 op/s 2024-08-19T01:43:05.966 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:43:05 smithi167 ceph-mon[21875]: pgmap v586: 97 pgs: 97 active+clean; 43 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 2.1 MiB/s wr, 757 op/s 2024-08-19T01:43:07.904 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:43:07 smithi083 ceph-mon[17120]: pgmap v587: 97 pgs: 97 active+clean; 40 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.4 MiB/s wr, 439 op/s 2024-08-19T01:43:07.952 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:43:07 smithi117 ceph-mon[22016]: pgmap v587: 97 pgs: 97 active+clean; 40 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.4 MiB/s wr, 439 op/s 2024-08-19T01:43:07.965 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:43:07 smithi167 ceph-mon[21875]: pgmap v587: 97 pgs: 97 active+clean; 40 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.4 MiB/s wr, 439 op/s 2024-08-19T01:43:09.904 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:43:09 smithi083 ceph-mon[17120]: pgmap v588: 97 pgs: 97 active+clean; 35 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.4 MiB/s wr, 405 op/s 2024-08-19T01:43:09.952 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:43:09 smithi117 ceph-mon[22016]: pgmap v588: 97 pgs: 97 active+clean; 35 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.4 MiB/s wr, 405 op/s 2024-08-19T01:43:09.965 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:43:09 smithi167 ceph-mon[21875]: pgmap v588: 97 pgs: 97 active+clean; 35 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.4 MiB/s wr, 405 op/s 2024-08-19T01:43:11.952 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:43:11 smithi117 ceph-mon[22016]: pgmap v589: 97 pgs: 97 active+clean; 35 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.4 MiB/s wr, 292 op/s 2024-08-19T01:43:11.952 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:43:11 smithi117 ceph-mon[22016]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-08-19T01:43:11.966 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:43:11 smithi167 ceph-mon[21875]: pgmap v589: 97 pgs: 97 active+clean; 35 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.4 MiB/s wr, 292 op/s 2024-08-19T01:43:11.966 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:43:11 smithi167 ceph-mon[21875]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-08-19T01:43:12.154 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:43:11 smithi083 ceph-mon[17120]: pgmap v589: 97 pgs: 97 active+clean; 35 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.4 MiB/s wr, 292 op/s 2024-08-19T01:43:12.154 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:43:11 smithi083 ceph-mon[17120]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-08-19T01:43:13.952 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:43:13 smithi117 ceph-mon[22016]: pgmap v590: 97 pgs: 97 active+clean; 35 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 781 KiB/s rd, 781 KiB/s wr, 226 op/s 2024-08-19T01:43:13.965 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:43:13 smithi167 ceph-mon[21875]: pgmap v590: 97 pgs: 97 active+clean; 35 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 781 KiB/s rd, 781 KiB/s wr, 226 op/s 2024-08-19T01:43:14.154 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:43:13 smithi083 ceph-mon[17120]: pgmap v590: 97 pgs: 97 active+clean; 35 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 781 KiB/s rd, 781 KiB/s wr, 226 op/s 2024-08-19T01:43:15.952 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:43:15 smithi117 ceph-mon[22016]: pgmap v591: 97 pgs: 97 active+clean; 35 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 782 KiB/s rd, 781 KiB/s wr, 182 op/s 2024-08-19T01:43:15.966 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:43:15 smithi167 ceph-mon[21875]: pgmap v591: 97 pgs: 97 active+clean; 35 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 782 KiB/s rd, 781 KiB/s wr, 182 op/s 2024-08-19T01:43:16.154 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:43:15 smithi083 ceph-mon[17120]: pgmap v591: 97 pgs: 97 active+clean; 35 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 782 KiB/s rd, 781 KiB/s wr, 182 op/s 2024-08-19T01:43:16.952 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:43:16 smithi117 ceph-mon[22016]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-08-19T01:43:16.965 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:43:16 smithi167 ceph-mon[21875]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-08-19T01:43:17.056 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:43:16 smithi083 ceph-mon[17120]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-08-19T01:43:17.952 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:43:17 smithi117 ceph-mon[22016]: pgmap v592: 97 pgs: 97 active+clean; 35 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 245 KiB/s rd, 102 KiB/s wr, 62 op/s 2024-08-19T01:43:17.965 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:43:17 smithi167 ceph-mon[21875]: pgmap v592: 97 pgs: 97 active+clean; 35 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 245 KiB/s rd, 102 KiB/s wr, 62 op/s 2024-08-19T01:43:18.081 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:43:17 smithi083 ceph-mon[17120]: pgmap v592: 97 pgs: 97 active+clean; 35 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 245 KiB/s rd, 102 KiB/s wr, 62 op/s 2024-08-19T01:43:19.904 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:43:19 smithi083 ceph-mon[17120]: pgmap v593: 97 pgs: 97 active+clean; 35 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 152 KiB/s rd, 8.1 KiB/s wr, 34 op/s 2024-08-19T01:43:19.952 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:43:19 smithi117 ceph-mon[22016]: pgmap v593: 97 pgs: 97 active+clean; 35 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 152 KiB/s rd, 8.1 KiB/s wr, 34 op/s 2024-08-19T01:43:19.966 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:43:19 smithi167 ceph-mon[21875]: pgmap v593: 97 pgs: 97 active+clean; 35 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 152 KiB/s rd, 8.1 KiB/s wr, 34 op/s 2024-08-19T01:43:20.769 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833 shell --fsid bad8780e-5dc9-11ef-bccf-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-08-19T01:43:21.157 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/bad8780e-5dc9-11ef-bccf-c7b262605968/mon.a/config 2024-08-19T01:43:21.759 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:21.754+0000 7f0afaa00640 1 -- 172.21.15.83:0/3018366667 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0af41078b0 msgr2=0x7f0af4107c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:43:21.759 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:21.754+0000 7f0afaa00640 1 --2- 172.21.15.83:0/3018366667 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0af41078b0 0x7f0af4107c90 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f0ae4009930 tx=0x7f0ae402a420 comp rx=0 tx=0).stop 2024-08-19T01:43:21.759 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:21.754+0000 7f0afaa00640 1 -- 172.21.15.83:0/3018366667 shutdown_connections 2024-08-19T01:43:21.759 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:21.754+0000 7f0afaa00640 1 --2- 172.21.15.83:0/3018366667 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0af4102250 0x7f0af410e780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:43:21.759 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:21.754+0000 7f0afaa00640 1 --2- 172.21.15.83:0/3018366667 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0af41018b0 0x7f0af4101d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:43:21.760 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:21.754+0000 7f0afaa00640 1 --2- 172.21.15.83:0/3018366667 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0af41078b0 0x7f0af4107c90 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:43:21.760 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:21.754+0000 7f0afaa00640 1 -- 172.21.15.83:0/3018366667 >> 172.21.15.83:0/3018366667 conn(0x7f0af40fd5f0 msgr2=0x7f0af40ffa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:43:21.760 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:21.754+0000 7f0afaa00640 1 -- 172.21.15.83:0/3018366667 shutdown_connections 2024-08-19T01:43:21.760 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:21.754+0000 7f0afaa00640 1 -- 172.21.15.83:0/3018366667 wait complete. 2024-08-19T01:43:21.760 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:21.755+0000 7f0afaa00640 1 Processor -- start 2024-08-19T01:43:21.760 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:21.755+0000 7f0afaa00640 1 -- start start 2024-08-19T01:43:21.760 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:21.756+0000 7f0afaa00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0af41018b0 0x7f0af419a380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:43:21.760 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:21.756+0000 7f0afaa00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0af4102250 0x7f0af419a8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:43:21.760 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:21.756+0000 7f0afaa00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0af41078b0 0x7f0af419ec50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:43:21.760 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:21.756+0000 7f0afaa00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f0af4111cc0 con 0x7f0af41018b0 2024-08-19T01:43:21.760 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:21.756+0000 7f0afaa00640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f0af4111b40 con 0x7f0af4102250 2024-08-19T01:43:21.761 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:21.756+0000 7f0afaa00640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f0af4111e40 con 0x7f0af41078b0 2024-08-19T01:43:21.761 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:21.756+0000 7f0af3e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0af41018b0 0x7f0af419a380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:43:21.761 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:21.756+0000 7f0af3e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0af41018b0 0x7f0af419a380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:50746/0 (socket says 172.21.15.83:50746) 2024-08-19T01:43:21.761 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:21.756+0000 7f0af3e00640 1 -- 172.21.15.83:0/3028108555 learned_addr learned my addr 172.21.15.83:0/3028108555 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-08-19T01:43:21.761 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:21.756+0000 7f0aebe00640 1 --2- 172.21.15.83:0/3028108555 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0af4102250 0x7f0af419a8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:43:21.761 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:21.756+0000 7f0af8a00640 1 --2- 172.21.15.83:0/3028108555 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0af41078b0 0x7f0af419ec50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:43:21.761 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:21.756+0000 7f0af3e00640 1 -- 172.21.15.83:0/3028108555 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0af41078b0 msgr2=0x7f0af419ec50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:43:21.761 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:21.756+0000 7f0af3e00640 1 --2- 172.21.15.83:0/3028108555 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0af41078b0 0x7f0af419ec50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:43:21.761 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:21.756+0000 7f0af3e00640 1 -- 172.21.15.83:0/3028108555 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0af4102250 msgr2=0x7f0af419a8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:43:21.761 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:21.756+0000 7f0af3e00640 1 --2- 172.21.15.83:0/3028108555 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0af4102250 0x7f0af419a8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:43:21.761 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:21.756+0000 7f0af3e00640 1 -- 172.21.15.83:0/3028108555 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0af419f330 con 0x7f0af41018b0 2024-08-19T01:43:21.761 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:21.757+0000 7f0af3e00640 1 --2- 172.21.15.83:0/3028108555 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0af41018b0 0x7f0af419a380 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f0ae402a940 tx=0x7f0ae4003b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:43:21.761 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:21.757+0000 7f0af1600640 1 -- 172.21.15.83:0/3028108555 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ae403f070 con 0x7f0af41018b0 2024-08-19T01:43:21.761 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:21.757+0000 7f0af1600640 1 -- 172.21.15.83:0/3028108555 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0ae4031c90 con 0x7f0af41018b0 2024-08-19T01:43:21.762 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:21.757+0000 7f0af1600640 1 -- 172.21.15.83:0/3028108555 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ae4035030 con 0x7f0af41018b0 2024-08-19T01:43:21.762 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:21.757+0000 7f0afaa00640 1 -- 172.21.15.83:0/3028108555 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0af419f5c0 con 0x7f0af41018b0 2024-08-19T01:43:21.762 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:21.757+0000 7f0afaa00640 1 -- 172.21.15.83:0/3028108555 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0af419fa50 con 0x7f0af41018b0 2024-08-19T01:43:21.762 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:21.761+0000 7f0af1600640 1 -- 172.21.15.83:0/3028108555 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7f0ae4042070 con 0x7f0af41018b0 2024-08-19T01:43:21.762 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:21.761+0000 7f0afd41d740 1 Processor -- start 2024-08-19T01:43:21.763 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:21.761+0000 7f0afd41d740 1 -- start start 2024-08-19T01:43:21.763 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:21.761+0000 7f0afd41d740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5570e068fb90 0x5570e068ff70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:43:21.763 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:21.761+0000 7f0afd41d740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5570e06904b0 0x5570e069a830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:43:21.763 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:21.761+0000 7f0afd41d740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5570e069ad70 0x5570e069d160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:43:21.763 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:21.761+0000 7f0afd41d740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x5570e05f5bf0 con 0x5570e069ad70 2024-08-19T01:43:21.763 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:21.761+0000 7f0afd41d740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x5570e058dbd0 con 0x5570e068fb90 2024-08-19T01:43:21.763 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:21.761+0000 7f0afd41d740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x5570e05f1bb0 con 0x5570e06904b0 2024-08-19T01:43:21.764 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:21.762+0000 7f0af8a00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5570e068fb90 0x5570e068ff70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:43:21.764 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:21.762+0000 7f0af8a00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5570e068fb90 0x5570e068ff70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.83:50180/0 (socket says 172.21.15.83:50180) 2024-08-19T01:43:21.764 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:21.762+0000 7f0af8a00640 1 -- 172.21.15.83:0/1043393446 learned_addr learned my addr 172.21.15.83:0/1043393446 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-08-19T01:43:21.764 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:21.762+0000 7f0aebe00640 1 --2- 172.21.15.83:0/1043393446 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5570e069ad70 0x5570e069d160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:43:21.764 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:21.762+0000 7f0af3e00640 1 --2- 172.21.15.83:0/1043393446 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5570e06904b0 0x5570e069a830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:43:21.764 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:21.762+0000 7f0af8a00640 1 -- 172.21.15.83:0/1043393446 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5570e06904b0 msgr2=0x5570e069a830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:43:21.765 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:21.762+0000 7f0af8a00640 1 --2- 172.21.15.83:0/1043393446 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5570e06904b0 0x5570e069a830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:43:21.765 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:21.762+0000 7f0af8a00640 1 -- 172.21.15.83:0/1043393446 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5570e069ad70 msgr2=0x5570e069d160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:43:21.765 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:21.762+0000 7f0af8a00640 1 --2- 172.21.15.83:0/1043393446 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5570e069ad70 0x5570e069d160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:43:21.765 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:21.762+0000 7f0af8a00640 1 -- 172.21.15.83:0/1043393446 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5570e05948a0 con 0x5570e068fb90 2024-08-19T01:43:21.765 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:21.762+0000 7f0af3e00640 1 --2- 172.21.15.83:0/1043393446 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5570e06904b0 0x5570e069a830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-19T01:43:21.765 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:21.762+0000 7f0af1600640 1 --2- 172.21.15.83:0/3028108555 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f0ad80a2c00 0x7f0ad80a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:43:21.765 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:21.762+0000 7f0af8a00640 1 --2- 172.21.15.83:0/1043393446 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5570e068fb90 0x5570e068ff70 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f0aec009f40 tx=0x7f0aec010860 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4f938b9b811845ac server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:43:21.765 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:21.763+0000 7f0af1600640 1 -- 172.21.15.83:0/3028108555 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8269+0+0 (secure 0 0 0) 0x7f0ae40ef6f0 con 0x7f0af41018b0 2024-08-19T01:43:21.765 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:21.763+0000 7f0ae9200640 1 -- 172.21.15.83:0/1043393446 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0aec003b70 con 0x5570e068fb90 2024-08-19T01:43:21.765 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:21.763+0000 7f0af3e00640 1 --2- 172.21.15.83:0/3028108555 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f0ad80a2c00 0x7f0ad80a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:43:21.765 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:21.763+0000 7f0ae9200640 1 -- 172.21.15.83:0/1043393446 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0aec003d10 con 0x5570e068fb90 2024-08-19T01:43:21.765 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:21.763+0000 7f0ae9200640 1 -- 172.21.15.83:0/1043393446 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0aec011a40 con 0x5570e068fb90 2024-08-19T01:43:21.767 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:21.763+0000 7f0afd41d740 1 -- 172.21.15.83:0/1043393446 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5570e068fb90 msgr2=0x5570e068ff70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:43:21.767 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:21.763+0000 7f0afd41d740 1 --2- 172.21.15.83:0/1043393446 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5570e068fb90 0x5570e068ff70 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f0aec009f40 tx=0x7f0aec010860 comp rx=0 tx=0).stop 2024-08-19T01:43:21.768 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:21.763+0000 7f0af3e00640 1 --2- 172.21.15.83:0/3028108555 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f0ad80a2c00 0x7f0ad80a50c0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f0ae40b8330 tx=0x7f0ae4031490 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:43:21.768 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:21.764+0000 7f0afd41d740 1 -- 172.21.15.83:0/1043393446 shutdown_connections 2024-08-19T01:43:21.768 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:21.764+0000 7f0afd41d740 1 --2- 172.21.15.83:0/1043393446 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5570e069ad70 0x5570e069d160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:43:21.768 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:21.764+0000 7f0afd41d740 1 --2- 172.21.15.83:0/1043393446 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5570e06904b0 0x5570e069a830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:43:21.768 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:21.764+0000 7f0afd41d740 1 --2- 172.21.15.83:0/1043393446 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5570e068fb90 0x5570e068ff70 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:43:21.768 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:21.764+0000 7f0afd41d740 1 -- 172.21.15.83:0/1043393446 >> 172.21.15.83:0/1043393446 conn(0x5570e068c120 msgr2=0x5570e068c500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:43:21.768 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:21.764+0000 7f0afd41d740 1 -- 172.21.15.83:0/1043393446 shutdown_connections 2024-08-19T01:43:21.768 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:21.764+0000 7f0afd41d740 1 -- 172.21.15.83:0/1043393446 wait complete. 2024-08-19T01:43:21.768 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:21.764+0000 7f0afd41d740 1 -- start start 2024-08-19T01:43:21.768 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:21.764+0000 7f0afd41d740 1 Processor -- start 2024-08-19T01:43:21.768 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:21.765+0000 7f0afd41d740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5570e068fb90 0x5570e0697c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:43:21.768 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:21.765+0000 7f0afd41d740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5570e06904b0 0x5570e06981b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:43:21.768 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:21.765+0000 7f0afd41d740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5570e069ad70 0x5570e06986f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:43:21.768 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:21.765+0000 7f0afd41d740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x5570e069d940 con 0x5570e069ad70 2024-08-19T01:43:21.769 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:21.765+0000 7f0afd41d740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x5570e05b0d40 con 0x5570e06904b0 2024-08-19T01:43:21.769 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:21.765+0000 7f0afd41d740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x5570e069dac0 con 0x5570e068fb90 2024-08-19T01:43:21.769 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:21.765+0000 7f0afd41d740 10 client.0 _ll_register_callbacks cb 0x5570e0579b10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-19T01:43:21.769 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:21.765+0000 7f0af8a00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5570e068fb90 0x5570e0697c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:43:21.769 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:21.765+0000 7f0af8a00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5570e068fb90 0x5570e0697c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.83:51106/0 (socket says 172.21.15.83:51106) 2024-08-19T01:43:21.769 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:21.765+0000 7f0af8a00640 1 -- 172.21.15.83:0/486654443 learned_addr learned my addr 172.21.15.83:0/486654443 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-08-19T01:43:21.769 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:21.765+0000 7f0aebe00640 1 --2- 172.21.15.83:0/486654443 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5570e06904b0 0x5570e06981b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:43:21.769 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:21.765+0000 7f0af8a00640 1 --2- 172.21.15.83:0/486654443 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5570e069ad70 0x5570e06986f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:43:21.769 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:21.766+0000 7f0aebe00640 1 -- 172.21.15.83:0/486654443 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5570e068fb90 msgr2=0x5570e0697c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:43:21.769 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:21.766+0000 7f0aebe00640 1 --2- 172.21.15.83:0/486654443 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5570e068fb90 0x5570e0697c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:43:21.769 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:21.766+0000 7f0aebe00640 1 -- 172.21.15.83:0/486654443 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5570e069ad70 msgr2=0x5570e06986f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:43:21.769 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:21.766+0000 7f0aebe00640 1 --2- 172.21.15.83:0/486654443 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5570e069ad70 0x5570e06986f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:43:21.769 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:21.766+0000 7f0aebe00640 1 -- 172.21.15.83:0/486654443 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5570e03673c0 con 0x5570e06904b0 2024-08-19T01:43:21.769 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:21.766+0000 7f0af8a00640 1 --2- 172.21.15.83:0/486654443 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5570e068fb90 0x5570e0697c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-19T01:43:21.770 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:21.766+0000 7f0aebe00640 1 --2- 172.21.15.83:0/486654443 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5570e06904b0 0x5570e06981b0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f0adc0126d0 tx=0x7f0adc012b90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=89a752206966b7a3 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:43:21.770 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:21.766+0000 7f0ae1600640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-08-19T01:43:21.772 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:21.767+0000 7f0ae1600640 1 -- 172.21.15.83:0/486654443 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0adc004050 con 0x5570e06904b0 2024-08-19T01:43:21.772 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:21.767+0000 7f0ae1600640 1 -- 172.21.15.83:0/486654443 <== mon.2 v2:172.21.15.117:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8269+0+0 (secure 0 0 0) 0x7f0adc0041f0 con 0x5570e06904b0 2024-08-19T01:43:21.772 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:21.767+0000 7f0afd41d740 1 -- 172.21.15.83:0/486654443 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0aec003d10 con 0x5570e06904b0 2024-08-19T01:43:21.772 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:21.767+0000 7f0ae1600640 1 -- 172.21.15.83:0/486654443 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f0adc0043f0 con 0x5570e06904b0 2024-08-19T01:43:21.772 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:21.767+0000 7f0afd41d740 10 client.26267 fetch_fsmap learned FSMap version 31 2024-08-19T01:43:21.772 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:21.767+0000 7f0afd41d740 1 -- 172.21.15.83:0/486654443 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x5570e06a9660 con 0x5570e06904b0 2024-08-19T01:43:21.772 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:21.767+0000 7f0ae1600640 1 -- 172.21.15.83:0/486654443 <== mon.2 v2:172.21.15.117:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0aec003d10 con 0x5570e06904b0 2024-08-19T01:43:21.772 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:21.767+0000 7f0ae1600640 1 -- 172.21.15.83:0/486654443 <== mon.2 v2:172.21.15.117:3300/0 5 ==== fsmap(e 31) ==== 8354+0+0 (secure 0 0 0) 0x7f0adc003bf0 con 0x5570e06904b0 2024-08-19T01:43:21.772 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:21.768+0000 7f0afd41d740 10 client.26267 fetch_fsmap finished waiting for FSMap version 31 2024-08-19T01:43:21.772 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:21.768+0000 7f0afd41d740 10 client.26267 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-19T01:43:21.772 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:21.768+0000 7f0afd41d740 20 client.26267 populate_metadata read hostname 'smithi083' 2024-08-19T01:43:21.772 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:21.768+0000 7f0afd41d740 1 --2- 172.21.15.83:0/486654443 >> [v2:172.21.15.117:6838/1967595704,v1:172.21.15.117:6839/1967595704] conn(0x5570e06aab70 0x5570e06acf90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:43:21.772 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:21.768+0000 7f0afd41d740 4 client.26267 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-19T01:43:21.773 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:21.768+0000 7f0afd41d740 1 -- 172.21.15.83:0/486654443 --> [v2:172.21.15.117:6838/1967595704,v1:172.21.15.117:6839/1967595704] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5570e0603160 con 0x5570e06aab70 2024-08-19T01:43:21.773 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:21.768+0000 7f0af8a00640 1 --2- 172.21.15.83:0/486654443 >> [v2:172.21.15.117:6838/1967595704,v1:172.21.15.117:6839/1967595704] conn(0x5570e06aab70 0x5570e06acf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:43:21.773 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:21.768+0000 7f0af8a00640 1 --2- 172.21.15.83:0/486654443 >> [v2:172.21.15.117:6838/1967595704,v1:172.21.15.117:6839/1967595704] conn(0x5570e06aab70 0x5570e06acf90 crc :-1 s=READY pgs=147 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cfce45fd1f7417c7 server_cookie=7b1a63b396a7b48d in_seq=0 out_seq=0 2024-08-19T01:43:21.773 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:21.768+0000 7f0ae1600640 10 client.26267 ms_handle_connect on v2:172.21.15.117:6838/1967595704 2024-08-19T01:43:21.773 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:21.769+0000 7f0ae1600640 1 -- 172.21.15.83:0/486654443 <== mds.0 v2:172.21.15.117:6838/1967595704 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5570e0603160 con 0x5570e06aab70 2024-08-19T01:43:21.773 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:21.769+0000 7f0ae1600640 10 client.26267 handle_command_reply: tid=0 2024-08-19T01:43:21.773 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:21.769+0000 7f0afd41d740 1 client.26267 shutdown 2024-08-19T01:43:21.773 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:21.769+0000 7f0afd41d740 1 -- 172.21.15.83:0/486654443 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5570e06904b0 msgr2=0x5570e06981b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:43:21.773 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:21.769+0000 7f0afd41d740 1 --2- 172.21.15.83:0/486654443 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5570e06904b0 0x5570e06981b0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f0adc0126d0 tx=0x7f0adc012b90 comp rx=0 tx=0).stop 2024-08-19T01:43:21.773 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:21.770+0000 7f0afd41d740 1 -- 172.21.15.83:0/486654443 shutdown_connections 2024-08-19T01:43:21.773 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:21.770+0000 7f0afd41d740 1 --2- 172.21.15.83:0/486654443 >> [v2:172.21.15.117:6838/1967595704,v1:172.21.15.117:6839/1967595704] conn(0x5570e06aab70 0x5570e06acf90 crc :-1 s=READY pgs=147 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:43:21.773 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:21.770+0000 7f0afd41d740 1 --2- 172.21.15.83:0/486654443 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5570e069ad70 0x5570e06986f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:43:21.773 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:21.770+0000 7f0afd41d740 1 --2- 172.21.15.83:0/486654443 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5570e06904b0 0x5570e06981b0 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:43:21.773 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:21.770+0000 7f0afd41d740 1 --2- 172.21.15.83:0/486654443 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5570e068fb90 0x5570e0697c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:43:21.774 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:21.770+0000 7f0ae1600640 0 client.26267 ms_handle_reset on v2:172.21.15.117:6838/1967595704 2024-08-19T01:43:21.774 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:21.770+0000 7f0afd41d740 1 -- 172.21.15.83:0/486654443 >> 172.21.15.83:0/486654443 conn(0x5570e068c120 msgr2=0x5570e0691e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:43:21.774 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:21.770+0000 7f0afd41d740 1 -- 172.21.15.83:0/486654443 shutdown_connections 2024-08-19T01:43:21.774 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:21.770+0000 7f0afd41d740 1 -- 172.21.15.83:0/486654443 wait complete. 2024-08-19T01:43:21.774 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:21.770+0000 7f0afd41d740 20 client.26267 trim_cache size 0 max 16384 2024-08-19T01:43:21.789 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:21.787+0000 7f0afd41d740 1 Processor -- start 2024-08-19T01:43:21.789 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:21.787+0000 7f0afd41d740 1 -- start start 2024-08-19T01:43:21.789 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:21.787+0000 7f0afd41d740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5570e068fb90 0x5570e06b2860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:43:21.789 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:21.788+0000 7f0afd41d740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5570e06904b0 0x5570e06b2da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:43:21.789 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:21.788+0000 7f0afd41d740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5570e069ad70 0x5570e0694f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:43:21.789 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:21.788+0000 7f0afd41d740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f0af41b1ec0 con 0x5570e06904b0 2024-08-19T01:43:21.789 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:21.788+0000 7f0afd41d740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f0af41af730 con 0x5570e068fb90 2024-08-19T01:43:21.789 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:21.788+0000 7f0afd41d740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f0af41ad580 con 0x5570e069ad70 2024-08-19T01:43:21.790 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:21.788+0000 7f0aebe00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5570e06904b0 0x5570e06b2da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:43:21.791 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:21.788+0000 7f0aebe00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5570e06904b0 0x5570e06b2da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:50768/0 (socket says 172.21.15.83:50768) 2024-08-19T01:43:21.791 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:21.788+0000 7f0aebe00640 1 -- 172.21.15.83:0/3483615600 learned_addr learned my addr 172.21.15.83:0/3483615600 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-08-19T01:43:21.791 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:21.788+0000 7f0af8a00640 1 --2- 172.21.15.83:0/3483615600 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5570e068fb90 0x5570e06b2860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:43:21.791 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:21.788+0000 7f0af8a00640 1 --2- 172.21.15.83:0/3483615600 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5570e069ad70 0x5570e0694f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:43:21.791 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:21.788+0000 7f0aebe00640 1 -- 172.21.15.83:0/3483615600 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5570e069ad70 msgr2=0x5570e0694f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:43:21.791 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:21.788+0000 7f0aebe00640 1 --2- 172.21.15.83:0/3483615600 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5570e069ad70 0x5570e0694f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:43:21.791 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:21.788+0000 7f0aebe00640 1 -- 172.21.15.83:0/3483615600 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5570e068fb90 msgr2=0x5570e06b2860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:43:21.791 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:21.788+0000 7f0aebe00640 1 --2- 172.21.15.83:0/3483615600 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5570e068fb90 0x5570e06b2860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:43:21.791 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:21.788+0000 7f0aebe00640 1 -- 172.21.15.83:0/3483615600 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5570e069d4d0 con 0x5570e06904b0 2024-08-19T01:43:21.791 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:21.789+0000 7f0aebe00640 1 --2- 172.21.15.83:0/3483615600 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5570e06904b0 0x5570e06b2da0 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f0adc010430 tx=0x7f0adc00f790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e58d7ef9c2e41e38 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:43:21.791 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:21.789+0000 7f0ae3e00640 1 -- 172.21.15.83:0/3483615600 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0adc0106a0 con 0x5570e06904b0 2024-08-19T01:43:21.791 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:21.789+0000 7f0ae3e00640 1 -- 172.21.15.83:0/3483615600 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0adc010840 con 0x5570e06904b0 2024-08-19T01:43:21.792 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:21.789+0000 7f0ae3e00640 1 -- 172.21.15.83:0/3483615600 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0adc00fca0 con 0x5570e06904b0 2024-08-19T01:43:21.793 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:21.790+0000 7f0afd41d740 1 -- 172.21.15.83:0/3483615600 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5570e06904b0 msgr2=0x5570e06b2da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:43:21.794 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:21.790+0000 7f0afd41d740 1 --2- 172.21.15.83:0/3483615600 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5570e06904b0 0x5570e06b2da0 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f0adc010430 tx=0x7f0adc00f790 comp rx=0 tx=0).stop 2024-08-19T01:43:21.794 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:21.790+0000 7f0afd41d740 1 -- 172.21.15.83:0/3483615600 shutdown_connections 2024-08-19T01:43:21.794 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:21.790+0000 7f0afd41d740 1 --2- 172.21.15.83:0/3483615600 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5570e069ad70 0x5570e0694f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:43:21.794 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:21.790+0000 7f0afd41d740 1 --2- 172.21.15.83:0/3483615600 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5570e06904b0 0x5570e06b2da0 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:43:21.794 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:21.790+0000 7f0afd41d740 1 --2- 172.21.15.83:0/3483615600 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5570e068fb90 0x5570e06b2860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:43:21.794 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:21.790+0000 7f0afd41d740 1 -- 172.21.15.83:0/3483615600 >> 172.21.15.83:0/3483615600 conn(0x5570e068c120 msgr2=0x5570e06915e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:43:21.794 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:21.790+0000 7f0afd41d740 1 -- 172.21.15.83:0/3483615600 shutdown_connections 2024-08-19T01:43:21.794 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:21.790+0000 7f0afd41d740 1 -- 172.21.15.83:0/3483615600 wait complete. 2024-08-19T01:43:21.794 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:21.790+0000 7f0afd41d740 1 -- start start 2024-08-19T01:43:21.794 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:21.791+0000 7f0afd41d740 1 Processor -- start 2024-08-19T01:43:21.794 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:21.791+0000 7f0afd41d740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5570e068fb90 0x5570e06a2ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:43:21.794 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:21.791+0000 7f0afd41d740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5570e06904b0 0x5570e06a3430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:43:21.795 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:21.791+0000 7f0afd41d740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5570e069ad70 0x5570e06944c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:43:21.795 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:21.791+0000 7f0afd41d740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f0af41adbf0 con 0x5570e068fb90 2024-08-19T01:43:21.795 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:21.791+0000 7f0afd41d740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f0af41af060 con 0x5570e069ad70 2024-08-19T01:43:21.795 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:21.791+0000 7f0afd41d740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f0af41ae2c0 con 0x5570e06904b0 2024-08-19T01:43:21.795 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:21.791+0000 7f0afd41d740 10 client.0 _ll_register_callbacks cb 0x7f0adc0097a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-19T01:43:21.795 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:21.791+0000 7f0af8a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5570e068fb90 0x5570e06a2ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:43:21.795 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:21.791+0000 7f0af8a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5570e068fb90 0x5570e06a2ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:50776/0 (socket says 172.21.15.83:50776) 2024-08-19T01:43:21.795 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:21.791+0000 7f0af8a00640 1 -- 172.21.15.83:0/3858601668 learned_addr learned my addr 172.21.15.83:0/3858601668 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-08-19T01:43:21.795 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:21.791+0000 7f0aebe00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5570e06904b0 0x5570e06a3430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:43:21.795 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:21.792+0000 7f0af8a00640 1 --2- 172.21.15.83:0/3858601668 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5570e069ad70 0x5570e06944c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:43:21.795 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:21.792+0000 7f0aebe00640 1 -- 172.21.15.83:0/3858601668 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5570e069ad70 msgr2=0x5570e06944c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:43:21.795 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:21.792+0000 7f0aebe00640 1 --2- 172.21.15.83:0/3858601668 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5570e069ad70 0x5570e06944c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:43:21.795 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:21.792+0000 7f0aebe00640 1 -- 172.21.15.83:0/3858601668 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5570e068fb90 msgr2=0x5570e06a2ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:43:21.796 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:21.792+0000 7f0aebe00640 1 --2- 172.21.15.83:0/3858601668 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5570e068fb90 0x5570e06a2ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:43:21.796 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:21.792+0000 7f0aebe00640 1 -- 172.21.15.83:0/3858601668 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5570e0694b50 con 0x5570e06904b0 2024-08-19T01:43:21.796 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:21.792+0000 7f0af8a00640 1 --2- 172.21.15.83:0/3858601668 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5570e068fb90 0x5570e06a2ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-19T01:43:21.796 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:21.792+0000 7f0aebe00640 1 --2- 172.21.15.83:0/3858601668 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5570e06904b0 0x5570e06a3430 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x5570e068aa00 tx=0x7f0adc004210 comp rx=0 tx=0).ready entity=mon.1 client_cookie=dbcbca60517b5357 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:43:21.796 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:21.792+0000 7f0ae1600640 10 client.0 ms_handle_connect on v2:172.21.15.167:3300/0 2024-08-19T01:43:21.796 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:21.792+0000 7f0ae1600640 1 -- 172.21.15.83:0/3858601668 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0adc03d060 con 0x5570e06904b0 2024-08-19T01:43:21.796 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:21.792+0000 7f0ae1600640 1 -- 172.21.15.83:0/3858601668 <== mon.1 v2:172.21.15.167:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8269+0+0 (secure 0 0 0) 0x7f0adc00fd30 con 0x5570e06904b0 2024-08-19T01:43:21.796 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:21.792+0000 7f0afd41d740 1 -- 172.21.15.83:0/3858601668 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0adc010840 con 0x5570e06904b0 2024-08-19T01:43:21.798 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:21.793+0000 7f0ae1600640 1 -- 172.21.15.83:0/3858601668 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f0adc011eb0 con 0x5570e06904b0 2024-08-19T01:43:21.798 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:21.793+0000 7f0afd41d740 10 client.26203 fetch_fsmap learned FSMap version 31 2024-08-19T01:43:21.798 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:21.793+0000 7f0afd41d740 1 -- 172.21.15.83:0/3858601668 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x5570e071d9d0 con 0x5570e06904b0 2024-08-19T01:43:21.798 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:21.793+0000 7f0ae1600640 1 -- 172.21.15.83:0/3858601668 <== mon.1 v2:172.21.15.167:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0adc010840 con 0x5570e06904b0 2024-08-19T01:43:21.798 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:21.793+0000 7f0ae1600640 1 -- 172.21.15.83:0/3858601668 <== mon.1 v2:172.21.15.167:3300/0 5 ==== fsmap(e 31) ==== 8354+0+0 (secure 0 0 0) 0x7f0adc0037f0 con 0x5570e06904b0 2024-08-19T01:43:21.798 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:21.793+0000 7f0afd41d740 10 client.26203 fetch_fsmap finished waiting for FSMap version 31 2024-08-19T01:43:21.798 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:21.793+0000 7f0afd41d740 10 client.26203 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-19T01:43:21.798 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:21.793+0000 7f0afd41d740 20 client.26203 populate_metadata read hostname 'smithi083' 2024-08-19T01:43:21.798 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:21.793+0000 7f0afd41d740 1 --2- 172.21.15.83:0/3858601668 >> [v2:172.21.15.117:6838/1967595704,v1:172.21.15.117:6839/1967595704] conn(0x5570e071e030 0x5570e0720450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:43:21.798 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:21.793+0000 7f0afd41d740 4 client.26203 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-08-19T01:43:21.799 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:21.793+0000 7f0afd41d740 1 -- 172.21.15.83:0/3858601668 --> [v2:172.21.15.117:6838/1967595704,v1:172.21.15.117:6839/1967595704] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5570e05fd670 con 0x5570e071e030 2024-08-19T01:43:21.799 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:21.794+0000 7f0af8a00640 1 --2- 172.21.15.83:0/3858601668 >> [v2:172.21.15.117:6838/1967595704,v1:172.21.15.117:6839/1967595704] conn(0x5570e071e030 0x5570e0720450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:43:21.799 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:21.794+0000 7f0af8a00640 1 --2- 172.21.15.83:0/3858601668 >> [v2:172.21.15.117:6838/1967595704,v1:172.21.15.117:6839/1967595704] conn(0x5570e071e030 0x5570e0720450 crc :-1 s=READY pgs=148 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=70cdc486e70d9ffa server_cookie=e8d7146b5da2aa8c in_seq=0 out_seq=0 2024-08-19T01:43:21.799 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:21.794+0000 7f0ae1600640 10 client.26203 ms_handle_connect on v2:172.21.15.117:6838/1967595704 2024-08-19T01:43:21.799 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:21.794+0000 7f0ae1600640 1 -- 172.21.15.83:0/3858601668 <== mds.0 v2:172.21.15.117:6838/1967595704 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x5570e05fd670 con 0x5570e071e030 2024-08-19T01:43:21.799 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:21.794+0000 7f0ae1600640 10 client.26203 handle_command_reply: tid=0 2024-08-19T01:43:21.799 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:21.794+0000 7f0afd41d740 1 client.26203 shutdown 2024-08-19T01:43:21.799 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:21.794+0000 7f0afd41d740 1 -- 172.21.15.83:0/3858601668 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5570e06904b0 msgr2=0x5570e06a3430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:43:21.799 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:21.794+0000 7f0afd41d740 1 --2- 172.21.15.83:0/3858601668 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5570e06904b0 0x5570e06a3430 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x5570e068aa00 tx=0x7f0adc004210 comp rx=0 tx=0).stop 2024-08-19T01:43:21.799 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:21.795+0000 7f0afd41d740 1 -- 172.21.15.83:0/3858601668 shutdown_connections 2024-08-19T01:43:21.799 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:21.795+0000 7f0afd41d740 1 --2- 172.21.15.83:0/3858601668 >> [v2:172.21.15.117:6838/1967595704,v1:172.21.15.117:6839/1967595704] conn(0x5570e071e030 0x5570e0720450 crc :-1 s=READY pgs=148 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:43:21.799 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:21.795+0000 7f0afd41d740 1 --2- 172.21.15.83:0/3858601668 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5570e069ad70 0x5570e06944c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:43:21.799 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:21.795+0000 7f0afd41d740 1 --2- 172.21.15.83:0/3858601668 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5570e06904b0 0x5570e06a3430 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:43:21.799 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:21.795+0000 7f0ae1600640 0 client.26203 ms_handle_reset on v2:172.21.15.117:6838/1967595704 2024-08-19T01:43:21.800 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:21.795+0000 7f0afd41d740 1 --2- 172.21.15.83:0/3858601668 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5570e068fb90 0x5570e06a2ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:43:21.800 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:21.795+0000 7f0afd41d740 1 -- 172.21.15.83:0/3858601668 >> 172.21.15.83:0/3858601668 conn(0x5570e068c120 msgr2=0x5570e0696560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:43:21.800 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:21.795+0000 7f0afd41d740 1 -- 172.21.15.83:0/3858601668 shutdown_connections 2024-08-19T01:43:21.800 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:21.795+0000 7f0afd41d740 1 -- 172.21.15.83:0/3858601668 wait complete. 2024-08-19T01:43:21.800 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:21.795+0000 7f0afd41d740 20 client.26203 trim_cache size 0 max 16384 2024-08-19T01:43:21.800 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-08-19T01:43:21.800 INFO:teuthology.orchestra.run.smithi083.stdout: "status": "no active scrubs running", 2024-08-19T01:43:21.800 INFO:teuthology.orchestra.run.smithi083.stdout: "scrubs": {} 2024-08-19T01:43:21.800 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-08-19T01:43:21.801 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:21.795+0000 7f0afaa00640 1 -- 172.21.15.83:0/3028108555 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f0ad80a2c00 msgr2=0x7f0ad80a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:43:21.801 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:21.795+0000 7f0afaa00640 1 --2- 172.21.15.83:0/3028108555 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f0ad80a2c00 0x7f0ad80a50c0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f0ae40b8330 tx=0x7f0ae4031490 comp rx=0 tx=0).stop 2024-08-19T01:43:21.801 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:21.795+0000 7f0afaa00640 1 -- 172.21.15.83:0/3028108555 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0af41018b0 msgr2=0x7f0af419a380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:43:21.801 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:21.795+0000 7f0afaa00640 1 --2- 172.21.15.83:0/3028108555 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0af41018b0 0x7f0af419a380 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f0ae402a940 tx=0x7f0ae4003b20 comp rx=0 tx=0).stop 2024-08-19T01:43:21.801 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:21.795+0000 7f0afaa00640 1 -- 172.21.15.83:0/3028108555 shutdown_connections 2024-08-19T01:43:21.801 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:21.795+0000 7f0afaa00640 1 --2- 172.21.15.83:0/3028108555 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f0ad80a2c00 0x7f0ad80a50c0 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:43:21.801 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:21.795+0000 7f0afaa00640 1 --2- 172.21.15.83:0/3028108555 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0af41078b0 0x7f0af419ec50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:43:21.801 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:21.796+0000 7f0afaa00640 1 --2- 172.21.15.83:0/3028108555 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0af4102250 0x7f0af419a8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:43:21.802 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:21.796+0000 7f0afaa00640 1 --2- 172.21.15.83:0/3028108555 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0af41018b0 0x7f0af419a380 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:43:21.802 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:21.796+0000 7f0afaa00640 1 -- 172.21.15.83:0/3028108555 >> 172.21.15.83:0/3028108555 conn(0x7f0af40fd5f0 msgr2=0x7f0af40fdc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:43:21.802 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:21.796+0000 7f0afaa00640 1 -- 172.21.15.83:0/3028108555 shutdown_connections 2024-08-19T01:43:21.802 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:21.796+0000 7f0afaa00640 1 -- 172.21.15.83:0/3028108555 wait complete. 2024-08-19T01:43:21.904 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:43:21 smithi083 ceph-mon[17120]: pgmap v594: 97 pgs: 97 active+clean; 35 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 8.1 KiB/s wr, 13 op/s 2024-08-19T01:43:21.952 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:43:21 smithi117 ceph-mon[22016]: pgmap v594: 97 pgs: 97 active+clean; 35 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 8.1 KiB/s wr, 13 op/s 2024-08-19T01:43:21.966 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:43:21 smithi167 ceph-mon[21875]: pgmap v594: 97 pgs: 97 active+clean; 35 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 8.1 KiB/s wr, 13 op/s 2024-08-19T01:43:22.764 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-08-19T01:43:22.764 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833 shell --fsid bad8780e-5dc9-11ef-bccf-c7b262605968 -- ceph fs dump --format=json 2024-08-19T01:43:23.140 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/bad8780e-5dc9-11ef-bccf-c7b262605968/mon.a/config 2024-08-19T01:43:23.688 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:23.685+0000 7f57c6400640 1 -- 172.21.15.83:0/2747057131 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f57c0112760 msgr2=0x7f57c0114b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:43:23.688 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:23.685+0000 7f57c6400640 1 --2- 172.21.15.83:0/2747057131 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f57c0112760 0x7f57c0114b50 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f57b8009ca0 tx=0x7f57b802a8c0 comp rx=0 tx=0).stop 2024-08-19T01:43:23.688 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:23.686+0000 7f57c6400640 1 -- 172.21.15.83:0/2747057131 shutdown_connections 2024-08-19T01:43:23.688 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:23.686+0000 7f57c6400640 1 --2- 172.21.15.83:0/2747057131 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f57c0112760 0x7f57c0114b50 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:43:23.688 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:23.686+0000 7f57c6400640 1 --2- 172.21.15.83:0/2747057131 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f57c0076d70 0x7f57c0112220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:43:23.688 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:23.686+0000 7f57c6400640 1 --2- 172.21.15.83:0/2747057131 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f57c0076450 0x7f57c0076830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:43:23.688 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:23.686+0000 7f57c6400640 1 -- 172.21.15.83:0/2747057131 >> 172.21.15.83:0/2747057131 conn(0x7f57c00ff7c0 msgr2=0x7f57c0101be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:43:23.688 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:23.686+0000 7f57c6400640 1 -- 172.21.15.83:0/2747057131 shutdown_connections 2024-08-19T01:43:23.688 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:23.686+0000 7f57c6400640 1 -- 172.21.15.83:0/2747057131 wait complete. 2024-08-19T01:43:23.689 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:23.687+0000 7f57c6400640 1 Processor -- start 2024-08-19T01:43:23.689 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:23.687+0000 7f57c6400640 1 -- start start 2024-08-19T01:43:23.689 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:23.688+0000 7f57c6400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f57c0076450 0x7f57c019e740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:43:23.689 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:23.688+0000 7f57c6400640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f57c0076d70 0x7f57c019ec80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:43:23.690 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:23.688+0000 7f57c6400640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f57c0112760 0x7f57c01a3010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:43:23.690 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:23.688+0000 7f57c6400640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f57c0118300 con 0x7f57c0076450 2024-08-19T01:43:23.690 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:23.688+0000 7f57c6400640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f57c0118180 con 0x7f57c0076d70 2024-08-19T01:43:23.690 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:23.688+0000 7f57c6400640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f57c0118480 con 0x7f57c0112760 2024-08-19T01:43:23.690 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:23.688+0000 7f57bf400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f57c0076450 0x7f57c019e740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:43:23.690 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:23.688+0000 7f57bf400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f57c0076450 0x7f57c019e740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:50788/0 (socket says 172.21.15.83:50788) 2024-08-19T01:43:23.690 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:23.688+0000 7f57bf400640 1 -- 172.21.15.83:0/2246714481 learned_addr learned my addr 172.21.15.83:0/2246714481 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-08-19T01:43:23.690 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:23.688+0000 7f57bfe00640 1 --2- 172.21.15.83:0/2246714481 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f57c0112760 0x7f57c01a3010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:43:23.690 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:23.688+0000 7f57bea00640 1 --2- 172.21.15.83:0/2246714481 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f57c0076d70 0x7f57c019ec80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:43:23.691 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:23.688+0000 7f57bf400640 1 -- 172.21.15.83:0/2246714481 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f57c0112760 msgr2=0x7f57c01a3010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:43:23.691 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:23.688+0000 7f57bf400640 1 --2- 172.21.15.83:0/2246714481 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f57c0112760 0x7f57c01a3010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:43:23.691 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:23.688+0000 7f57bf400640 1 -- 172.21.15.83:0/2246714481 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f57c0076d70 msgr2=0x7f57c019ec80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:43:23.691 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:23.688+0000 7f57bf400640 1 --2- 172.21.15.83:0/2246714481 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f57c0076d70 0x7f57c019ec80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:43:23.691 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:23.688+0000 7f57bf400640 1 -- 172.21.15.83:0/2246714481 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f57c01a36f0 con 0x7f57c0076450 2024-08-19T01:43:23.691 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:23.689+0000 7f57bf400640 1 --2- 172.21.15.83:0/2246714481 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f57c0076450 0x7f57c019e740 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f57b000e9f0 tx=0x7f57b000edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:43:23.692 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:23.690+0000 7f57b7e00640 1 -- 172.21.15.83:0/2246714481 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57b000cd80 con 0x7f57c0076450 2024-08-19T01:43:23.692 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:23.690+0000 7f57b7e00640 1 -- 172.21.15.83:0/2246714481 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f57b0005500 con 0x7f57c0076450 2024-08-19T01:43:23.692 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:23.690+0000 7f57b7e00640 1 -- 172.21.15.83:0/2246714481 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57b0010690 con 0x7f57c0076450 2024-08-19T01:43:23.692 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:23.690+0000 7f57c6400640 1 -- 172.21.15.83:0/2246714481 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f57c01a39e0 con 0x7f57c0076450 2024-08-19T01:43:23.692 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:23.690+0000 7f57c6400640 1 -- 172.21.15.83:0/2246714481 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f57c01ab2c0 con 0x7f57c0076450 2024-08-19T01:43:23.693 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:23.691+0000 7f57c6400640 1 -- 172.21.15.83:0/2246714481 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f57c0076830 con 0x7f57c0076450 2024-08-19T01:43:23.694 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:23.693+0000 7f57b7e00640 1 -- 172.21.15.83:0/2246714481 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7f57b00046c0 con 0x7f57c0076450 2024-08-19T01:43:23.696 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:23.694+0000 7f57b7e00640 1 --2- 172.21.15.83:0/2246714481 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f57a40a2c00 0x7f57a40a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:43:23.696 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:23.695+0000 7f57b7e00640 1 -- 172.21.15.83:0/2246714481 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8269+0+0 (secure 0 0 0) 0x7f57b00cf440 con 0x7f57c0076450 2024-08-19T01:43:23.696 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:23.695+0000 7f57bea00640 1 --2- 172.21.15.83:0/2246714481 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f57a40a2c00 0x7f57a40a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:43:23.697 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:23.695+0000 7f57bea00640 1 --2- 172.21.15.83:0/2246714481 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f57a40a2c00 0x7f57a40a50c0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f57c0078f50 tx=0x7f57ac005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:43:23.700 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:23.698+0000 7f57b7e00640 1 -- 172.21.15.83:0/2246714481 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7f57b009c320 con 0x7f57c0076450 2024-08-19T01:43:23.904 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:43:23 smithi083 ceph-mon[17120]: pgmap v595: 97 pgs: 97 active+clean; 35 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 7.8 KiB/s rd, 5.5 KiB/s wr, 7 op/s 2024-08-19T01:43:23.912 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:23.911+0000 7f57c6400640 1 -- 172.21.15.83:0/2246714481 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f57c019faa0 con 0x7f57c0076450 2024-08-19T01:43:23.913 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:23.911+0000 7f57b7e00640 1 -- 172.21.15.83:0/2246714481 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 31 v31) ==== 94+0+11922 (secure 0 0 0) 0x7f57b0086070 con 0x7f57c0076450 2024-08-19T01:43:23.913 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-08-19T01:43:23.913 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":31,"btime":"2024-08-19T01:43:15:661376+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14421,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6835/988556067","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":988556067},{"type":"v1","addr":"172.21.15.83:6835","nonce":988556067}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14433,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6837/869702632","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6836","nonce":869702632},{"type":"v1","addr":"172.21.15.83:6837","nonce":869702632}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14475,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6841/1795975999","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6840","nonce":1795975999},{"type":"v1","addr":"172.21.15.83:6841","nonce":1795975999}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24430,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6833/2505076046","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6832","nonce":2505076046},{"type":"v1","addr":"172.21.15.167:6833","nonce":2505076046}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24449,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6833/3557520478","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":3557520478},{"type":"v1","addr":"172.21.15.117:6833","nonce":3557520478}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24466,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6835/3482045966","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":3482045966},{"type":"v1","addr":"172.21.15.167:6835","nonce":3482045966}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7}],"filesystems":[{"mdsmap":{"epoch":31,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-08-19T01:32:42.054769+0000","modified":"2024-08-19T01:43:15.162835+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24545,"mds_1":24527,"mds_2":24509},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24478":{"gid":24478,"name":"c","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.167:6837/1762130408","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6836","nonce":1762130408},{"type":"v1","addr":"172.21.15.167:6837","nonce":1762130408}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24485":{"gid":24485,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.117:6835/1024213994","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":1024213994},{"type":"v1","addr":"172.21.15.117:6835","nonce":1024213994}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24496":{"gid":24496,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.167:6839/3507387682","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6838","nonce":3507387682},{"type":"v1","addr":"172.21.15.167:6839","nonce":3507387682}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24509":{"gid":24509,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":10,"addr":"172.21.15.117:6837/3185224684","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":3185224684},{"type":"v1","addr":"172.21.15.117:6837","nonce":3185224684}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24527":{"gid":24527,"name":"g","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.83:6839/3239261845","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6838","nonce":3239261845},{"type":"v1","addr":"172.21.15.83:6839","nonce":3239261845}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24545":{"gid":24545,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.117:6839/1967595704","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6838","nonce":1967595704},{"type":"v1","addr":"172.21.15.117:6839","nonce":1967595704}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24545,"qdb_cluster":[24545,24527,24509]},"id":1}]} 2024-08-19T01:43:23.914 INFO:teuthology.orchestra.run.smithi083.stderr:dumped fsmap epoch 31 2024-08-19T01:43:23.916 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:23.914+0000 7f57c6400640 1 -- 172.21.15.83:0/2246714481 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f57a40a2c00 msgr2=0x7f57a40a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:43:23.916 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:23.914+0000 7f57c6400640 1 --2- 172.21.15.83:0/2246714481 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f57a40a2c00 0x7f57a40a50c0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f57c0078f50 tx=0x7f57ac005f90 comp rx=0 tx=0).stop 2024-08-19T01:43:23.916 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:23.914+0000 7f57c6400640 1 -- 172.21.15.83:0/2246714481 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f57c0076450 msgr2=0x7f57c019e740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:43:23.916 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:23.914+0000 7f57c6400640 1 --2- 172.21.15.83:0/2246714481 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f57c0076450 0x7f57c019e740 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f57b000e9f0 tx=0x7f57b000edf0 comp rx=0 tx=0).stop 2024-08-19T01:43:23.916 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:23.914+0000 7f57c6400640 1 -- 172.21.15.83:0/2246714481 shutdown_connections 2024-08-19T01:43:23.916 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:23.914+0000 7f57c6400640 1 --2- 172.21.15.83:0/2246714481 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f57a40a2c00 0x7f57a40a50c0 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:43:23.916 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:23.914+0000 7f57c6400640 1 --2- 172.21.15.83:0/2246714481 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f57c0112760 0x7f57c01a3010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:43:23.916 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:23.914+0000 7f57c6400640 1 --2- 172.21.15.83:0/2246714481 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f57c0076d70 0x7f57c019ec80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:43:23.916 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:23.914+0000 7f57c6400640 1 --2- 172.21.15.83:0/2246714481 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f57c0076450 0x7f57c019e740 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:43:23.916 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:23.914+0000 7f57c6400640 1 -- 172.21.15.83:0/2246714481 >> 172.21.15.83:0/2246714481 conn(0x7f57c00ff7c0 msgr2=0x7f57c0101bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:43:23.916 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:23.914+0000 7f57c6400640 1 -- 172.21.15.83:0/2246714481 shutdown_connections 2024-08-19T01:43:23.916 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:23.914+0000 7f57c6400640 1 -- 172.21.15.83:0/2246714481 wait complete. 2024-08-19T01:43:23.952 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:43:23 smithi117 ceph-mon[22016]: pgmap v595: 97 pgs: 97 active+clean; 35 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 7.8 KiB/s rd, 5.5 KiB/s wr, 7 op/s 2024-08-19T01:43:23.965 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:43:23 smithi167 ceph-mon[21875]: pgmap v595: 97 pgs: 97 active+clean; 35 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 7.8 KiB/s rd, 5.5 KiB/s wr, 7 op/s 2024-08-19T01:43:24.614 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833 shell --fsid bad8780e-5dc9-11ef-bccf-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-08-19T01:43:24.904 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:43:24 smithi083 ceph-mon[17120]: from='client.? 172.21.15.83:0/2246714481' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-19T01:43:24.952 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:43:24 smithi117 ceph-mon[22016]: from='client.? 172.21.15.83:0/2246714481' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-19T01:43:24.965 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:43:24 smithi167 ceph-mon[21875]: from='client.? 172.21.15.83:0/2246714481' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-19T01:43:24.990 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/bad8780e-5dc9-11ef-bccf-c7b262605968/mon.a/config 2024-08-19T01:43:25.607 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:25.606+0000 7f200da00640 1 -- 172.21.15.83:0/2716575738 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2008102250 msgr2=0x7f200810e780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:43:25.608 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:25.606+0000 7f200da00640 1 --2- 172.21.15.83:0/2716575738 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2008102250 0x7f200810e780 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f2000009cd0 tx=0x7f200002a8f0 comp rx=0 tx=0).stop 2024-08-19T01:43:25.608 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:25.606+0000 7f200da00640 1 -- 172.21.15.83:0/2716575738 shutdown_connections 2024-08-19T01:43:25.608 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:25.606+0000 7f200da00640 1 --2- 172.21.15.83:0/2716575738 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2008102250 0x7f200810e780 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:43:25.608 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:25.606+0000 7f200da00640 1 --2- 172.21.15.83:0/2716575738 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f20081018b0 0x7f2008101d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:43:25.608 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:25.606+0000 7f200da00640 1 --2- 172.21.15.83:0/2716575738 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f20081078b0 0x7f2008107c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:43:25.608 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:25.606+0000 7f200da00640 1 -- 172.21.15.83:0/2716575738 >> 172.21.15.83:0/2716575738 conn(0x7f20080fd5f0 msgr2=0x7f20080ffa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:43:25.608 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:25.606+0000 7f200da00640 1 -- 172.21.15.83:0/2716575738 shutdown_connections 2024-08-19T01:43:25.608 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:25.606+0000 7f200da00640 1 -- 172.21.15.83:0/2716575738 wait complete. 2024-08-19T01:43:25.609 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:25.607+0000 7f200da00640 1 Processor -- start 2024-08-19T01:43:25.610 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:25.607+0000 7f200da00640 1 -- start start 2024-08-19T01:43:25.610 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:25.607+0000 7f200da00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f20081018b0 0x7f200819a340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:43:25.610 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:25.607+0000 7f200da00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2008102250 0x7f200819a880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:43:25.610 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:25.607+0000 7f200da00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f20081078b0 0x7f200819ec10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:43:25.610 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:25.607+0000 7f200da00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f2008111d00 con 0x7f2008102250 2024-08-19T01:43:25.610 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:25.607+0000 7f200da00640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f2008111b80 con 0x7f20081018b0 2024-08-19T01:43:25.610 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:25.607+0000 7f200da00640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f2008111e80 con 0x7f20081078b0 2024-08-19T01:43:25.610 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:25.607+0000 7f2006a00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f20081018b0 0x7f200819a340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:43:25.610 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:25.607+0000 7f2006000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2008102250 0x7f200819a880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:43:25.610 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:25.607+0000 7f2006a00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f20081018b0 0x7f200819a340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.83:50238/0 (socket says 172.21.15.83:50238) 2024-08-19T01:43:25.610 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:25.607+0000 7f2006a00640 1 -- 172.21.15.83:0/1195897354 learned_addr learned my addr 172.21.15.83:0/1195897354 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-08-19T01:43:25.610 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:25.608+0000 7f2007400640 1 --2- 172.21.15.83:0/1195897354 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f20081078b0 0x7f200819ec10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:43:25.611 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:25.608+0000 7f2006a00640 1 -- 172.21.15.83:0/1195897354 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f20081078b0 msgr2=0x7f200819ec10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:43:25.611 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:25.608+0000 7f2006a00640 1 --2- 172.21.15.83:0/1195897354 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f20081078b0 0x7f200819ec10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:43:25.611 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:25.608+0000 7f2006a00640 1 -- 172.21.15.83:0/1195897354 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2008102250 msgr2=0x7f200819a880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:43:25.611 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:25.608+0000 7f2006a00640 1 --2- 172.21.15.83:0/1195897354 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2008102250 0x7f200819a880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:43:25.611 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:25.608+0000 7f2006a00640 1 -- 172.21.15.83:0/1195897354 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f200819f2f0 con 0x7f20081018b0 2024-08-19T01:43:25.611 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:25.608+0000 7f2006000640 1 --2- 172.21.15.83:0/1195897354 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2008102250 0x7f200819a880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-19T01:43:25.611 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:25.608+0000 7f2006a00640 1 --2- 172.21.15.83:0/1195897354 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f20081018b0 0x7f200819a340 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f1ff800e9f0 tx=0x7f1ff800eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:43:25.612 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:25.608+0000 7f1fff400640 1 -- 172.21.15.83:0/1195897354 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ff800cd80 con 0x7f20081018b0 2024-08-19T01:43:25.612 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:25.609+0000 7f1fff400640 1 -- 172.21.15.83:0/1195897354 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1ff8010910 con 0x7f20081018b0 2024-08-19T01:43:25.612 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:25.609+0000 7f1fff400640 1 -- 172.21.15.83:0/1195897354 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ff8018990 con 0x7f20081018b0 2024-08-19T01:43:25.612 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:25.609+0000 7f200da00640 1 -- 172.21.15.83:0/1195897354 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f200819f5e0 con 0x7f20081018b0 2024-08-19T01:43:25.612 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:25.609+0000 7f200da00640 1 -- 172.21.15.83:0/1195897354 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f20081a6ec0 con 0x7f20081018b0 2024-08-19T01:43:25.613 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:25.612+0000 7f201048f740 1 Processor -- start 2024-08-19T01:43:25.613 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:25.612+0000 7f201048f740 1 -- start start 2024-08-19T01:43:25.614 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:25.612+0000 7f1fff400640 1 -- 172.21.15.83:0/1195897354 <== mon.2 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7f1ff8010ac0 con 0x7f20081018b0 2024-08-19T01:43:25.614 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:25.612+0000 7f201048f740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55fe803de4f0 0x55fe803de8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:43:25.614 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:25.612+0000 7f201048f740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55fe803dee10 0x55fe803e9190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:43:25.614 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:25.612+0000 7f201048f740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fe803e96d0 0x55fe803ebac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:43:25.614 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:25.612+0000 7f201048f740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55fe80343bf0 con 0x55fe803e96d0 2024-08-19T01:43:25.615 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:25.612+0000 7f201048f740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55fe802dbb20 con 0x55fe803de4f0 2024-08-19T01:43:25.615 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:25.612+0000 7f201048f740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55fe8033fbb0 con 0x55fe803dee10 2024-08-19T01:43:25.615 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:25.612+0000 7f2006000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fe803e96d0 0x55fe803ebac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:43:25.615 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:25.612+0000 7f2006000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fe803e96d0 0x55fe803ebac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:50816/0 (socket says 172.21.15.83:50816) 2024-08-19T01:43:25.615 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:25.612+0000 7f2006000640 1 -- 172.21.15.83:0/1530545750 learned_addr learned my addr 172.21.15.83:0/1530545750 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-08-19T01:43:25.615 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:25.612+0000 7f2007400640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55fe803de4f0 0x55fe803de8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:43:25.615 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:25.612+0000 7f2006a00640 1 --2- 172.21.15.83:0/1530545750 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55fe803dee10 0x55fe803e9190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:43:25.615 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:25.613+0000 7f2006000640 1 -- 172.21.15.83:0/1530545750 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55fe803dee10 msgr2=0x55fe803e9190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:43:25.615 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:25.613+0000 7f2006000640 1 --2- 172.21.15.83:0/1530545750 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55fe803dee10 0x55fe803e9190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:43:25.615 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:25.613+0000 7f2006000640 1 -- 172.21.15.83:0/1530545750 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55fe803de4f0 msgr2=0x55fe803de8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:43:25.615 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:25.613+0000 7f2006000640 1 --2- 172.21.15.83:0/1530545750 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55fe803de4f0 0x55fe803de8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:43:25.615 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:25.613+0000 7f2006000640 1 -- 172.21.15.83:0/1530545750 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fe802e27f0 con 0x55fe803e96d0 2024-08-19T01:43:25.617 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:25.613+0000 7f2006000640 1 --2- 172.21.15.83:0/1530545750 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fe803e96d0 0x55fe803ebac0 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f1ff40114f0 tx=0x7f1ff40119b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b548649d726c752b server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:43:25.617 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:25.613+0000 7f1ff3e00640 1 -- 172.21.15.83:0/1530545750 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ff4004020 con 0x55fe803e96d0 2024-08-19T01:43:25.617 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:25.614+0000 7f1ff3e00640 1 -- 172.21.15.83:0/1530545750 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1ff40041c0 con 0x55fe803e96d0 2024-08-19T01:43:25.618 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:25.614+0000 7f1ff3e00640 1 -- 172.21.15.83:0/1530545750 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ff400f960 con 0x55fe803e96d0 2024-08-19T01:43:25.618 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:25.614+0000 7f1fff400640 1 --2- 172.21.15.83:0/1195897354 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f1fec0a2c00 0x7f1fec0a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:43:25.618 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:25.614+0000 7f1fff400640 1 -- 172.21.15.83:0/1195897354 <== mon.2 v2:172.21.15.117:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8269+0+0 (secure 0 0 0) 0x7f1ff80cf230 con 0x7f20081018b0 2024-08-19T01:43:25.618 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:25.614+0000 7f2007400640 1 --2- 172.21.15.83:0/1195897354 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f1fec0a2c00 0x7f1fec0a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:43:25.618 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:25.614+0000 7f201048f740 1 -- 172.21.15.83:0/1530545750 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fe803e96d0 msgr2=0x55fe803ebac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:43:25.618 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:25.614+0000 7f201048f740 1 --2- 172.21.15.83:0/1530545750 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fe803e96d0 0x55fe803ebac0 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f1ff40114f0 tx=0x7f1ff40119b0 comp rx=0 tx=0).stop 2024-08-19T01:43:25.618 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:25.614+0000 7f201048f740 1 -- 172.21.15.83:0/1530545750 shutdown_connections 2024-08-19T01:43:25.618 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:25.614+0000 7f201048f740 1 --2- 172.21.15.83:0/1530545750 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fe803e96d0 0x55fe803ebac0 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:43:25.618 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:25.614+0000 7f201048f740 1 --2- 172.21.15.83:0/1530545750 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55fe803dee10 0x55fe803e9190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:43:25.618 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:25.614+0000 7f201048f740 1 --2- 172.21.15.83:0/1530545750 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55fe803de4f0 0x55fe803de8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:43:25.618 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:25.614+0000 7f201048f740 1 -- 172.21.15.83:0/1530545750 >> 172.21.15.83:0/1530545750 conn(0x55fe803daa80 msgr2=0x55fe803dae60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:43:25.619 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:25.615+0000 7f201048f740 1 -- 172.21.15.83:0/1530545750 shutdown_connections 2024-08-19T01:43:25.619 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:25.615+0000 7f201048f740 1 -- 172.21.15.83:0/1530545750 wait complete. 2024-08-19T01:43:25.619 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:25.615+0000 7f2007400640 1 --2- 172.21.15.83:0/1195897354 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f1fec0a2c00 0x7f1fec0a50c0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f2000004af0 tx=0x7f2000005d80 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:43:25.619 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:25.615+0000 7f201048f740 1 -- start start 2024-08-19T01:43:25.619 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:25.615+0000 7f201048f740 1 Processor -- start 2024-08-19T01:43:25.619 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:25.616+0000 7f201048f740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55fe803de4f0 0x55fe803e6590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:43:25.619 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:25.616+0000 7f201048f740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fe803dee10 0x55fe803e6ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:43:25.619 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:25.616+0000 7f201048f740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55fe803e96d0 0x55fe803e7010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:43:25.619 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:25.616+0000 7f201048f740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55fe803ec2f0 con 0x55fe803dee10 2024-08-19T01:43:25.619 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:25.616+0000 7f201048f740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55fe802fecf0 con 0x55fe803e96d0 2024-08-19T01:43:25.619 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:25.616+0000 7f201048f740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55fe803ec470 con 0x55fe803de4f0 2024-08-19T01:43:25.619 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:25.616+0000 7f201048f740 10 client.0 _ll_register_callbacks cb 0x55fe802eae30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-19T01:43:25.619 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:25.616+0000 7f2006000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fe803dee10 0x55fe803e6ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:43:25.619 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:25.616+0000 7f2006000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fe803dee10 0x55fe803e6ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:50826/0 (socket says 172.21.15.83:50826) 2024-08-19T01:43:25.620 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:25.616+0000 7f2006000640 1 -- 172.21.15.83:0/491578224 learned_addr learned my addr 172.21.15.83:0/491578224 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-08-19T01:43:25.620 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:25.616+0000 7f2006a00640 1 --2- 172.21.15.83:0/491578224 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55fe803de4f0 0x55fe803e6590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:43:25.620 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:25.616+0000 7f2007400640 1 --2- 172.21.15.83:0/491578224 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55fe803e96d0 0x55fe803e7010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:43:25.622 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:25.616+0000 7f2006000640 1 -- 172.21.15.83:0/491578224 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55fe803de4f0 msgr2=0x55fe803e6590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:43:25.622 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:25.616+0000 7f2006000640 1 --2- 172.21.15.83:0/491578224 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55fe803de4f0 0x55fe803e6590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:43:25.622 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:25.617+0000 7f2006000640 1 -- 172.21.15.83:0/491578224 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55fe803e96d0 msgr2=0x55fe803e7010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:43:25.622 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:25.617+0000 7f2006000640 1 --2- 172.21.15.83:0/491578224 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55fe803e96d0 0x55fe803e7010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:43:25.622 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:25.617+0000 7f2006000640 1 -- 172.21.15.83:0/491578224 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fe800b53c0 con 0x55fe803dee10 2024-08-19T01:43:25.623 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:25.617+0000 7f2006a00640 1 --2- 172.21.15.83:0/491578224 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55fe803de4f0 0x55fe803e6590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-19T01:43:25.623 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:25.617+0000 7f2006000640 1 --2- 172.21.15.83:0/491578224 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fe803dee10 0x55fe803e6ad0 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f1ff401b040 tx=0x7f1ff4005740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=73bab06b7c3fc087 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:43:25.623 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:25.617+0000 7f1ff0c00640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-08-19T01:43:25.623 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:25.617+0000 7f1ff0c00640 1 -- 172.21.15.83:0/491578224 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ff4028020 con 0x55fe803dee10 2024-08-19T01:43:25.623 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:25.617+0000 7f1ff0c00640 1 -- 172.21.15.83:0/491578224 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8269+0+0 (secure 0 0 0) 0x7f1ff4016040 con 0x55fe803dee10 2024-08-19T01:43:25.623 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:25.617+0000 7f201048f740 1 -- 172.21.15.83:0/491578224 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1ff40041c0 con 0x55fe803dee10 2024-08-19T01:43:25.623 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:25.618+0000 7f1ff0c00640 1 -- 172.21.15.83:0/491578224 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f1ff4033050 con 0x55fe803dee10 2024-08-19T01:43:25.623 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:25.618+0000 7f201048f740 10 client.16239 fetch_fsmap learned FSMap version 31 2024-08-19T01:43:25.623 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:25.618+0000 7f201048f740 1 -- 172.21.15.83:0/491578224 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55fe803f7fa0 con 0x55fe803dee10 2024-08-19T01:43:25.623 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:25.618+0000 7f1ff0c00640 1 -- 172.21.15.83:0/491578224 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1ff40041c0 con 0x55fe803dee10 2024-08-19T01:43:25.623 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:25.618+0000 7f1ff0c00640 1 -- 172.21.15.83:0/491578224 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 31) ==== 8354+0+0 (secure 0 0 0) 0x7f1ff4004a50 con 0x55fe803dee10 2024-08-19T01:43:25.623 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:25.618+0000 7f201048f740 10 client.16239 fetch_fsmap finished waiting for FSMap version 31 2024-08-19T01:43:25.623 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:25.618+0000 7f201048f740 10 client.16239 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-08-19T01:43:25.623 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:25.618+0000 7f201048f740 20 client.16239 populate_metadata read hostname 'smithi083' 2024-08-19T01:43:25.624 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:25.618+0000 7f201048f740 1 --2- 172.21.15.83:0/491578224 >> [v2:172.21.15.117:6836/3185224684,v1:172.21.15.117:6837/3185224684] conn(0x55fe803f9400 0x55fe803fb7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:43:25.624 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:25.618+0000 7f201048f740 4 client.16239 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-19T01:43:25.624 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:25.618+0000 7f201048f740 1 -- 172.21.15.83:0/491578224 --> [v2:172.21.15.117:6836/3185224684,v1:172.21.15.117:6837/3185224684] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55fe80351160 con 0x55fe803f9400 2024-08-19T01:43:25.624 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:25.619+0000 7f2006a00640 1 --2- 172.21.15.83:0/491578224 >> [v2:172.21.15.117:6836/3185224684,v1:172.21.15.117:6837/3185224684] conn(0x55fe803f9400 0x55fe803fb7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:43:25.624 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:25.619+0000 7f2006a00640 1 --2- 172.21.15.83:0/491578224 >> [v2:172.21.15.117:6836/3185224684,v1:172.21.15.117:6837/3185224684] conn(0x55fe803f9400 0x55fe803fb7f0 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=4533bb5879d3beaa server_cookie=fb3e041c5c0b9996 in_seq=0 out_seq=0 2024-08-19T01:43:25.624 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:25.619+0000 7f1ff0c00640 10 client.16239 ms_handle_connect on v2:172.21.15.117:6836/3185224684 2024-08-19T01:43:25.624 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:25.620+0000 7f1ff0c00640 1 -- 172.21.15.83:0/491578224 <== mds.2 v2:172.21.15.117:6836/3185224684 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55fe80351160 con 0x55fe803f9400 2024-08-19T01:43:25.624 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:25.620+0000 7f1ff0c00640 10 client.16239 handle_command_reply: tid=0 2024-08-19T01:43:25.624 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:25.620+0000 7f201048f740 1 client.16239 shutdown 2024-08-19T01:43:25.624 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:25.620+0000 7f201048f740 1 -- 172.21.15.83:0/491578224 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fe803dee10 msgr2=0x55fe803e6ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:43:25.624 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:25.620+0000 7f201048f740 1 --2- 172.21.15.83:0/491578224 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fe803dee10 0x55fe803e6ad0 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f1ff401b040 tx=0x7f1ff4005740 comp rx=0 tx=0).stop 2024-08-19T01:43:25.624 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:25.620+0000 7f201048f740 1 -- 172.21.15.83:0/491578224 shutdown_connections 2024-08-19T01:43:25.624 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:25.620+0000 7f201048f740 1 --2- 172.21.15.83:0/491578224 >> [v2:172.21.15.117:6836/3185224684,v1:172.21.15.117:6837/3185224684] conn(0x55fe803f9400 0x55fe803fb7f0 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:43:25.624 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:25.620+0000 7f201048f740 1 --2- 172.21.15.83:0/491578224 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55fe803e96d0 0x55fe803e7010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:43:25.625 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:25.620+0000 7f201048f740 1 --2- 172.21.15.83:0/491578224 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fe803dee10 0x55fe803e6ad0 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:43:25.625 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:25.620+0000 7f201048f740 1 --2- 172.21.15.83:0/491578224 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55fe803de4f0 0x55fe803e6590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:43:25.625 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:25.620+0000 7f201048f740 1 -- 172.21.15.83:0/491578224 >> 172.21.15.83:0/491578224 conn(0x55fe803daa80 msgr2=0x55fe803d9c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:43:25.625 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:25.620+0000 7f1ff0c00640 0 client.16239 ms_handle_reset on v2:172.21.15.117:6836/3185224684 2024-08-19T01:43:25.625 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:25.620+0000 7f201048f740 1 -- 172.21.15.83:0/491578224 shutdown_connections 2024-08-19T01:43:25.625 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:25.620+0000 7f201048f740 1 -- 172.21.15.83:0/491578224 wait complete. 2024-08-19T01:43:25.625 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:25.621+0000 7f201048f740 20 client.16239 trim_cache size 0 max 16384 2024-08-19T01:43:25.636 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:25.634+0000 7f201048f740 1 Processor -- start 2024-08-19T01:43:25.636 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:25.634+0000 7f201048f740 1 -- start start 2024-08-19T01:43:25.636 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:25.634+0000 7f201048f740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55fe803de4f0 0x55fe803e5e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:43:25.636 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:25.634+0000 7f201048f740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55fe803dee10 0x55fe80401040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:43:25.636 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:25.634+0000 7f201048f740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fe803e96d0 0x55fe80401580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:43:25.636 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:25.634+0000 7f201048f740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f20081b1ec0 con 0x55fe803e96d0 2024-08-19T01:43:25.636 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:25.634+0000 7f201048f740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f20081af730 con 0x55fe803de4f0 2024-08-19T01:43:25.636 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:25.634+0000 7f201048f740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f20081ad580 con 0x55fe803dee10 2024-08-19T01:43:25.637 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:25.635+0000 7f2007400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fe803e96d0 0x55fe80401580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:43:25.638 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:25.635+0000 7f2007400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fe803e96d0 0x55fe80401580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:50834/0 (socket says 172.21.15.83:50834) 2024-08-19T01:43:25.638 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:25.635+0000 7f2007400640 1 -- 172.21.15.83:0/1661140871 learned_addr learned my addr 172.21.15.83:0/1661140871 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-08-19T01:43:25.638 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:25.635+0000 7f2006000640 1 --2- 172.21.15.83:0/1661140871 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55fe803dee10 0x55fe80401040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:43:25.638 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:25.635+0000 7f2006a00640 1 --2- 172.21.15.83:0/1661140871 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55fe803de4f0 0x55fe803e5e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:43:25.638 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:25.635+0000 7f2007400640 1 -- 172.21.15.83:0/1661140871 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55fe803dee10 msgr2=0x55fe80401040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:43:25.638 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:25.635+0000 7f2007400640 1 --2- 172.21.15.83:0/1661140871 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55fe803dee10 0x55fe80401040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:43:25.638 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:25.635+0000 7f2007400640 1 -- 172.21.15.83:0/1661140871 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55fe803de4f0 msgr2=0x55fe803e5e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:43:25.638 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:25.635+0000 7f2007400640 1 --2- 172.21.15.83:0/1661140871 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55fe803de4f0 0x55fe803e5e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:43:25.638 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:25.635+0000 7f2007400640 1 -- 172.21.15.83:0/1661140871 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fe803e1360 con 0x55fe803e96d0 2024-08-19T01:43:25.638 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:25.635+0000 7f2006a00640 1 --2- 172.21.15.83:0/1661140871 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55fe803de4f0 0x55fe803e5e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-19T01:43:25.638 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:25.636+0000 7f2007400640 1 --2- 172.21.15.83:0/1661140871 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fe803e96d0 0x55fe80401580 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f2000035000 tx=0x7f2000041000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cd3bd61271dabbdb server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:43:25.639 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:25.636+0000 7f1ff3400640 1 -- 172.21.15.83:0/1661140871 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2000003d30 con 0x55fe803e96d0 2024-08-19T01:43:25.639 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:25.636+0000 7f1ff3400640 1 -- 172.21.15.83:0/1661140871 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2000036050 con 0x55fe803e96d0 2024-08-19T01:43:25.639 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:25.636+0000 7f1ff3400640 1 -- 172.21.15.83:0/1661140871 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2000004740 con 0x55fe803e96d0 2024-08-19T01:43:25.641 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:25.637+0000 7f201048f740 1 -- 172.21.15.83:0/1661140871 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fe803e96d0 msgr2=0x55fe80401580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:43:25.641 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:25.637+0000 7f201048f740 1 --2- 172.21.15.83:0/1661140871 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fe803e96d0 0x55fe80401580 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f2000035000 tx=0x7f2000041000 comp rx=0 tx=0).stop 2024-08-19T01:43:25.641 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:25.637+0000 7f201048f740 1 -- 172.21.15.83:0/1661140871 shutdown_connections 2024-08-19T01:43:25.641 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:25.637+0000 7f201048f740 1 --2- 172.21.15.83:0/1661140871 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fe803e96d0 0x55fe80401580 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:43:25.641 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:25.637+0000 7f201048f740 1 --2- 172.21.15.83:0/1661140871 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55fe803dee10 0x55fe80401040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:43:25.641 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:25.637+0000 7f201048f740 1 --2- 172.21.15.83:0/1661140871 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55fe803de4f0 0x55fe803e5e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:43:25.641 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:25.637+0000 7f201048f740 1 -- 172.21.15.83:0/1661140871 >> 172.21.15.83:0/1661140871 conn(0x55fe803daa80 msgr2=0x55fe803fb410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:43:25.641 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:25.637+0000 7f201048f740 1 -- 172.21.15.83:0/1661140871 shutdown_connections 2024-08-19T01:43:25.641 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:25.637+0000 7f201048f740 1 -- 172.21.15.83:0/1661140871 wait complete. 2024-08-19T01:43:25.641 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:25.637+0000 7f201048f740 1 -- start start 2024-08-19T01:43:25.641 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:25.637+0000 7f201048f740 1 Processor -- start 2024-08-19T01:43:25.641 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:25.638+0000 7f201048f740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55fe803de4f0 0x55fe803ecc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:43:25.642 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:25.638+0000 7f201048f740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55fe803dee10 0x55fe803ed1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:43:25.642 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:25.638+0000 7f201048f740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fe803e96d0 0x55fe8046a810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:43:25.642 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:25.638+0000 7f201048f740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f20081adbf0 con 0x55fe803e96d0 2024-08-19T01:43:25.642 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:25.638+0000 7f201048f740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f20081af060 con 0x55fe803de4f0 2024-08-19T01:43:25.642 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:25.638+0000 7f201048f740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f20081ae2c0 con 0x55fe803dee10 2024-08-19T01:43:25.642 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:25.638+0000 7f201048f740 10 client.0 _ll_register_callbacks cb 0x7f1ff4009ee0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-19T01:43:25.642 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:25.638+0000 7f2007400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fe803e96d0 0x55fe8046a810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:43:25.642 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:25.638+0000 7f2007400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fe803e96d0 0x55fe8046a810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:50842/0 (socket says 172.21.15.83:50842) 2024-08-19T01:43:25.642 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:25.638+0000 7f2007400640 1 -- 172.21.15.83:0/141183962 learned_addr learned my addr 172.21.15.83:0/141183962 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-08-19T01:43:25.642 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:25.638+0000 7f2006000640 1 --2- 172.21.15.83:0/141183962 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55fe803dee10 0x55fe803ed1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:43:25.642 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:25.638+0000 7f2006a00640 1 --2- 172.21.15.83:0/141183962 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55fe803de4f0 0x55fe803ecc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:43:25.642 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:25.638+0000 7f2007400640 1 -- 172.21.15.83:0/141183962 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55fe803dee10 msgr2=0x55fe803ed1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:43:25.642 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:25.638+0000 7f2007400640 1 --2- 172.21.15.83:0/141183962 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55fe803dee10 0x55fe803ed1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:43:25.643 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:25.638+0000 7f2007400640 1 -- 172.21.15.83:0/141183962 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55fe803de4f0 msgr2=0x55fe803ecc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:43:25.643 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:25.638+0000 7f2007400640 1 --2- 172.21.15.83:0/141183962 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55fe803de4f0 0x55fe803ecc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:43:25.643 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:25.638+0000 7f2007400640 1 -- 172.21.15.83:0/141183962 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fe8046add0 con 0x55fe803e96d0 2024-08-19T01:43:25.643 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:25.639+0000 7f2006000640 1 --2- 172.21.15.83:0/141183962 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55fe803dee10 0x55fe803ed1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-19T01:43:25.643 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:25.639+0000 7f2007400640 1 --2- 172.21.15.83:0/141183962 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fe803e96d0 0x55fe8046a810 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f2000055040 tx=0x7f200002a8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8e8c59c58684615a server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:43:25.643 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:25.639+0000 7f1ff0c00640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-08-19T01:43:25.643 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:25.639+0000 7f1ff0c00640 1 -- 172.21.15.83:0/141183962 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2000061020 con 0x55fe803e96d0 2024-08-19T01:43:25.643 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:25.639+0000 7f1ff0c00640 1 -- 172.21.15.83:0/141183962 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8269+0+0 (secure 0 0 0) 0x7f2000030930 con 0x55fe803e96d0 2024-08-19T01:43:25.643 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:25.639+0000 7f201048f740 1 -- 172.21.15.83:0/141183962 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2000036050 con 0x55fe803e96d0 2024-08-19T01:43:25.643 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:25.639+0000 7f1ff0c00640 1 -- 172.21.15.83:0/141183962 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f2000004b40 con 0x55fe803e96d0 2024-08-19T01:43:25.643 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:25.639+0000 7f201048f740 10 client.16245 fetch_fsmap learned FSMap version 31 2024-08-19T01:43:25.643 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:25.639+0000 7f201048f740 1 -- 172.21.15.83:0/141183962 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55fe8046c0b0 con 0x55fe803e96d0 2024-08-19T01:43:25.645 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:25.640+0000 7f1ff0c00640 1 -- 172.21.15.83:0/141183962 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2000036050 con 0x55fe803e96d0 2024-08-19T01:43:25.645 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:25.640+0000 7f1ff0c00640 1 -- 172.21.15.83:0/141183962 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 31) ==== 8354+0+0 (secure 0 0 0) 0x7f200005c440 con 0x55fe803e96d0 2024-08-19T01:43:25.645 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:25.640+0000 7f201048f740 10 client.16245 fetch_fsmap finished waiting for FSMap version 31 2024-08-19T01:43:25.645 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:25.640+0000 7f201048f740 10 client.16245 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-08-19T01:43:25.645 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:25.640+0000 7f201048f740 20 client.16245 populate_metadata read hostname 'smithi083' 2024-08-19T01:43:25.645 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:25.640+0000 7f201048f740 1 --2- 172.21.15.83:0/141183962 >> [v2:172.21.15.117:6836/3185224684,v1:172.21.15.117:6837/3185224684] conn(0x55fe8046c7f0 0x55fe8046ec10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:43:25.645 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:25.640+0000 7f201048f740 4 client.16245 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-08-19T01:43:25.645 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:25.640+0000 7f201048f740 1 -- 172.21.15.83:0/141183962 --> [v2:172.21.15.117:6836/3185224684,v1:172.21.15.117:6837/3185224684] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55fe803d5ff0 con 0x55fe8046c7f0 2024-08-19T01:43:25.646 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:25.640+0000 7f2006a00640 1 --2- 172.21.15.83:0/141183962 >> [v2:172.21.15.117:6836/3185224684,v1:172.21.15.117:6837/3185224684] conn(0x55fe8046c7f0 0x55fe8046ec10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:43:25.646 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:25.641+0000 7f2006a00640 1 --2- 172.21.15.83:0/141183962 >> [v2:172.21.15.117:6836/3185224684,v1:172.21.15.117:6837/3185224684] conn(0x55fe8046c7f0 0x55fe8046ec10 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=c8d961cd4be58ac4 server_cookie=147d61db9551f29e in_seq=0 out_seq=0 2024-08-19T01:43:25.646 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:25.641+0000 7f1ff0c00640 10 client.16245 ms_handle_connect on v2:172.21.15.117:6836/3185224684 2024-08-19T01:43:25.646 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:25.641+0000 7f1ff0c00640 1 -- 172.21.15.83:0/141183962 <== mds.2 v2:172.21.15.117:6836/3185224684 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55fe803d5ff0 con 0x55fe8046c7f0 2024-08-19T01:43:25.646 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:25.641+0000 7f1ff0c00640 10 client.16245 handle_command_reply: tid=0 2024-08-19T01:43:25.646 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:25.641+0000 7f201048f740 1 client.16245 shutdown 2024-08-19T01:43:25.646 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:25.641+0000 7f201048f740 1 -- 172.21.15.83:0/141183962 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fe803e96d0 msgr2=0x55fe8046a810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:43:25.646 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:25.641+0000 7f201048f740 1 --2- 172.21.15.83:0/141183962 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fe803e96d0 0x55fe8046a810 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f2000055040 tx=0x7f200002a8f0 comp rx=0 tx=0).stop 2024-08-19T01:43:25.646 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:25.641+0000 7f201048f740 1 -- 172.21.15.83:0/141183962 shutdown_connections 2024-08-19T01:43:25.646 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:25.641+0000 7f201048f740 1 --2- 172.21.15.83:0/141183962 >> [v2:172.21.15.117:6836/3185224684,v1:172.21.15.117:6837/3185224684] conn(0x55fe8046c7f0 0x55fe8046ec10 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:43:25.646 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:25.641+0000 7f201048f740 1 --2- 172.21.15.83:0/141183962 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fe803e96d0 0x55fe8046a810 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:43:25.646 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:25.641+0000 7f201048f740 1 --2- 172.21.15.83:0/141183962 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55fe803dee10 0x55fe803ed1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:43:25.647 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:25.641+0000 7f1ff0c00640 0 client.16245 ms_handle_reset on v2:172.21.15.117:6836/3185224684 2024-08-19T01:43:25.647 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:25.641+0000 7f201048f740 1 --2- 172.21.15.83:0/141183962 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55fe803de4f0 0x55fe803ecc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:43:25.647 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:25.641+0000 7f201048f740 1 -- 172.21.15.83:0/141183962 >> 172.21.15.83:0/141183962 conn(0x55fe803daa80 msgr2=0x55fe803dff40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:43:25.647 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:25.641+0000 7f201048f740 1 -- 172.21.15.83:0/141183962 shutdown_connections 2024-08-19T01:43:25.647 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:25.642+0000 7f201048f740 1 -- 172.21.15.83:0/141183962 wait complete. 2024-08-19T01:43:25.647 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:25.642+0000 7f201048f740 20 client.16245 trim_cache size 0 max 16384 2024-08-19T01:43:25.647 INFO:teuthology.orchestra.run.smithi083.stdout:[] 2024-08-19T01:43:25.648 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:25.642+0000 7f200da00640 1 -- 172.21.15.83:0/1195897354 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f1fec0a2c00 msgr2=0x7f1fec0a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:43:25.648 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:25.642+0000 7f200da00640 1 --2- 172.21.15.83:0/1195897354 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f1fec0a2c00 0x7f1fec0a50c0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f2000004af0 tx=0x7f2000005d80 comp rx=0 tx=0).stop 2024-08-19T01:43:25.648 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:25.642+0000 7f200da00640 1 -- 172.21.15.83:0/1195897354 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f20081018b0 msgr2=0x7f200819a340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:43:25.648 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:25.642+0000 7f200da00640 1 --2- 172.21.15.83:0/1195897354 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f20081018b0 0x7f200819a340 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f1ff800e9f0 tx=0x7f1ff800eeb0 comp rx=0 tx=0).stop 2024-08-19T01:43:25.648 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:25.642+0000 7f200da00640 1 -- 172.21.15.83:0/1195897354 shutdown_connections 2024-08-19T01:43:25.648 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:25.642+0000 7f200da00640 1 --2- 172.21.15.83:0/1195897354 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f1fec0a2c00 0x7f1fec0a50c0 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:43:25.648 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:25.642+0000 7f200da00640 1 --2- 172.21.15.83:0/1195897354 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f20081078b0 0x7f200819ec10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:43:25.648 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:25.642+0000 7f200da00640 1 --2- 172.21.15.83:0/1195897354 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2008102250 0x7f200819a880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:43:25.648 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:25.642+0000 7f200da00640 1 --2- 172.21.15.83:0/1195897354 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f20081018b0 0x7f200819a340 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:43:25.648 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:25.642+0000 7f200da00640 1 -- 172.21.15.83:0/1195897354 >> 172.21.15.83:0/1195897354 conn(0x7f20080fd5f0 msgr2=0x7f200810ce60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:43:25.649 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:25.643+0000 7f200da00640 1 -- 172.21.15.83:0/1195897354 shutdown_connections 2024-08-19T01:43:25.649 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:25.643+0000 7f200da00640 1 -- 172.21.15.83:0/1195897354 wait complete. 2024-08-19T01:43:25.905 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:43:25 smithi083 ceph-mon[17120]: pgmap v596: 97 pgs: 97 active+clean; 35 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 5.5 KiB/s wr, 8 op/s 2024-08-19T01:43:25.952 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:43:25 smithi117 ceph-mon[22016]: pgmap v596: 97 pgs: 97 active+clean; 35 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 5.5 KiB/s wr, 8 op/s 2024-08-19T01:43:25.965 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:43:25 smithi167 ceph-mon[21875]: pgmap v596: 97 pgs: 97 active+clean; 35 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 5.5 KiB/s wr, 8 op/s 2024-08-19T01:43:26.423 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833 shell --fsid bad8780e-5dc9-11ef-bccf-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-08-19T01:43:26.800 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/bad8780e-5dc9-11ef-bccf-c7b262605968/mon.a/config 2024-08-19T01:43:27.386 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:27.384+0000 7f971e000640 1 -- 172.21.15.83:0/544560195 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f971810e0f0 msgr2=0x7f9718110540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:43:27.386 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:27.384+0000 7f971e000640 1 --2- 172.21.15.83:0/544560195 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f971810e0f0 0x7f9718110540 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f9710009900 tx=0x7f971002a480 comp rx=0 tx=0).stop 2024-08-19T01:43:27.386 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:27.384+0000 7f971e000640 1 -- 172.21.15.83:0/544560195 shutdown_connections 2024-08-19T01:43:27.387 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:27.384+0000 7f971e000640 1 --2- 172.21.15.83:0/544560195 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f971810e0f0 0x7f9718110540 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:43:27.387 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:27.384+0000 7f971e000640 1 --2- 172.21.15.83:0/544560195 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9718074c70 0x7f971810dbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:43:27.387 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:27.384+0000 7f971e000640 1 --2- 172.21.15.83:0/544560195 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9718074350 0x7f9718074730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:43:27.387 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:27.384+0000 7f971e000640 1 -- 172.21.15.83:0/544560195 >> 172.21.15.83:0/544560195 conn(0x7f97180fd5f0 msgr2=0x7f97180ffa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:43:27.387 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:27.384+0000 7f971e000640 1 -- 172.21.15.83:0/544560195 shutdown_connections 2024-08-19T01:43:27.387 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:27.384+0000 7f971e000640 1 -- 172.21.15.83:0/544560195 wait complete. 2024-08-19T01:43:27.387 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:27.385+0000 7f971e000640 1 Processor -- start 2024-08-19T01:43:27.387 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:27.385+0000 7f971e000640 1 -- start start 2024-08-19T01:43:27.389 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:27.385+0000 7f971e000640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9718074350 0x7f971819a3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:43:27.389 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:27.385+0000 7f971e000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9718074c70 0x7f971819a8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:43:27.389 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:27.385+0000 7f971e000640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f971810e0f0 0x7f971819ec80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:43:27.389 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:27.385+0000 7f971e000640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f9718113ef0 con 0x7f9718074c70 2024-08-19T01:43:27.389 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:27.385+0000 7f971e000640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f9718113d70 con 0x7f9718074350 2024-08-19T01:43:27.389 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:27.385+0000 7f971e000640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f9718114070 con 0x7f971810e0f0 2024-08-19T01:43:27.390 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:27.385+0000 7f970ea00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9718074c70 0x7f971819a8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:43:27.390 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:27.385+0000 7f9717400640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9718074350 0x7f971819a3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:43:27.390 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:27.386+0000 7f9717400640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9718074350 0x7f971819a3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.83:50306/0 (socket says 172.21.15.83:50306) 2024-08-19T01:43:27.390 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:27.386+0000 7f9717400640 1 -- 172.21.15.83:0/40996781 learned_addr learned my addr 172.21.15.83:0/40996781 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-08-19T01:43:27.390 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:27.386+0000 7f9717e00640 1 --2- 172.21.15.83:0/40996781 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f971810e0f0 0x7f971819ec80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:43:27.390 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:27.386+0000 7f9717400640 1 -- 172.21.15.83:0/40996781 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f971810e0f0 msgr2=0x7f971819ec80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:43:27.390 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:27.386+0000 7f9717400640 1 --2- 172.21.15.83:0/40996781 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f971810e0f0 0x7f971819ec80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:43:27.390 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:27.386+0000 7f9717400640 1 -- 172.21.15.83:0/40996781 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9718074c70 msgr2=0x7f971819a8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:43:27.390 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:27.386+0000 7f9717400640 1 --2- 172.21.15.83:0/40996781 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9718074c70 0x7f971819a8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:43:27.390 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:27.386+0000 7f9717400640 1 -- 172.21.15.83:0/40996781 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f971819f360 con 0x7f9718074350 2024-08-19T01:43:27.390 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:27.386+0000 7f9717e00640 1 --2- 172.21.15.83:0/40996781 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f971810e0f0 0x7f971819ec80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-19T01:43:27.390 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:27.386+0000 7f970ea00640 1 --2- 172.21.15.83:0/40996781 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9718074c70 0x7f971819a8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-19T01:43:27.391 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:27.386+0000 7f9717400640 1 --2- 172.21.15.83:0/40996781 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9718074350 0x7f971819a3b0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f970800e9f0 tx=0x7f970800edf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:43:27.391 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:27.387+0000 7f9714c00640 1 -- 172.21.15.83:0/40996781 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f970800cd80 con 0x7f9718074350 2024-08-19T01:43:27.391 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:27.387+0000 7f9714c00640 1 -- 172.21.15.83:0/40996781 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9708005500 con 0x7f9718074350 2024-08-19T01:43:27.391 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:27.387+0000 7f971e000640 1 -- 172.21.15.83:0/40996781 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f971819f650 con 0x7f9718074350 2024-08-19T01:43:27.391 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:27.387+0000 7f9714c00640 1 -- 172.21.15.83:0/40996781 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9708010690 con 0x7f9718074350 2024-08-19T01:43:27.391 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:27.387+0000 7f971e000640 1 -- 172.21.15.83:0/40996781 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f97181a6f30 con 0x7f9718074350 2024-08-19T01:43:27.392 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:27.390+0000 7f9720a2b740 1 Processor -- start 2024-08-19T01:43:27.392 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:27.390+0000 7f9720a2b740 1 -- start start 2024-08-19T01:43:27.392 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:27.390+0000 7f9714c00640 1 -- 172.21.15.83:0/40996781 <== mon.2 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7f97080046c0 con 0x7f9718074350 2024-08-19T01:43:27.392 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:27.390+0000 7f9720a2b740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56086d0a54f0 0x56086d0a58d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:43:27.392 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:27.390+0000 7f9720a2b740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56086d0a5e10 0x56086d0b0190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:43:27.393 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:27.390+0000 7f9720a2b740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56086d0b06d0 0x56086d0b2ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:43:27.393 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:27.390+0000 7f9720a2b740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x56086d00abf0 con 0x56086d0a5e10 2024-08-19T01:43:27.393 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:27.390+0000 7f9720a2b740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x56086cfa2b20 con 0x56086d0a54f0 2024-08-19T01:43:27.393 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:27.390+0000 7f9720a2b740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x56086d006bb0 con 0x56086d0b06d0 2024-08-19T01:43:27.393 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:27.391+0000 7f9717400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56086d0a5e10 0x56086d0b0190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:43:27.393 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:27.391+0000 7f9717400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56086d0a5e10 0x56086d0b0190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:50864/0 (socket says 172.21.15.83:50864) 2024-08-19T01:43:27.393 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:27.391+0000 7f9717400640 1 -- 172.21.15.83:0/3005348524 learned_addr learned my addr 172.21.15.83:0/3005348524 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-08-19T01:43:27.393 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:27.391+0000 7f9717e00640 1 --2- 172.21.15.83:0/3005348524 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56086d0a54f0 0x56086d0a58d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:43:27.393 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:27.391+0000 7f970ea00640 1 --2- 172.21.15.83:0/3005348524 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56086d0b06d0 0x56086d0b2ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:43:27.395 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:27.391+0000 7f9717400640 1 -- 172.21.15.83:0/3005348524 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56086d0b06d0 msgr2=0x56086d0b2ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:43:27.395 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:27.391+0000 7f9717400640 1 --2- 172.21.15.83:0/3005348524 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56086d0b06d0 0x56086d0b2ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:43:27.395 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:27.391+0000 7f9717400640 1 -- 172.21.15.83:0/3005348524 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56086d0a54f0 msgr2=0x56086d0a58d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:43:27.395 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:27.391+0000 7f9717400640 1 --2- 172.21.15.83:0/3005348524 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56086d0a54f0 0x56086d0a58d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:43:27.395 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:27.391+0000 7f9717400640 1 -- 172.21.15.83:0/3005348524 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56086cfa97f0 con 0x56086d0a5e10 2024-08-19T01:43:27.395 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:27.391+0000 7f9717e00640 1 --2- 172.21.15.83:0/3005348524 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56086d0a54f0 0x56086d0a58d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-19T01:43:27.395 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:27.391+0000 7f9717400640 1 --2- 172.21.15.83:0/3005348524 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56086d0a5e10 0x56086d0b0190 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f97080d5460 tx=0x7f97080d5550 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bad7a7ea5d471b2 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:43:27.395 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:27.392+0000 7f9707e00640 1 -- 172.21.15.83:0/3005348524 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97080e3070 con 0x56086d0a5e10 2024-08-19T01:43:27.395 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:27.392+0000 7f9714c00640 1 --2- 172.21.15.83:0/40996781 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f96fc0a2c00 0x7f96fc0a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:43:27.395 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:27.392+0000 7f9707e00640 1 -- 172.21.15.83:0/3005348524 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9708098340 con 0x56086d0a5e10 2024-08-19T01:43:27.395 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:27.392+0000 7f9707e00640 1 -- 172.21.15.83:0/3005348524 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f970809baa0 con 0x56086d0a5e10 2024-08-19T01:43:27.395 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:27.392+0000 7f9714c00640 1 -- 172.21.15.83:0/40996781 <== mon.2 v2:172.21.15.117:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8269+0+0 (secure 0 0 0) 0x7f97080cf4f0 con 0x7f9718074350 2024-08-19T01:43:27.396 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:27.392+0000 7f9717e00640 1 --2- 172.21.15.83:0/40996781 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f96fc0a2c00 0x7f96fc0a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:43:27.396 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:27.393+0000 7f9720a2b740 1 -- 172.21.15.83:0/3005348524 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56086d0a5e10 msgr2=0x56086d0b0190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:43:27.396 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:27.393+0000 7f9720a2b740 1 --2- 172.21.15.83:0/3005348524 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56086d0a5e10 0x56086d0b0190 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f97080d5460 tx=0x7f97080d5550 comp rx=0 tx=0).stop 2024-08-19T01:43:27.396 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:27.393+0000 7f9720a2b740 1 -- 172.21.15.83:0/3005348524 shutdown_connections 2024-08-19T01:43:27.396 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:27.393+0000 7f9720a2b740 1 --2- 172.21.15.83:0/3005348524 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56086d0b06d0 0x56086d0b2ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:43:27.396 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:27.393+0000 7f9720a2b740 1 --2- 172.21.15.83:0/3005348524 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56086d0a5e10 0x56086d0b0190 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:43:27.396 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:27.393+0000 7f9720a2b740 1 --2- 172.21.15.83:0/3005348524 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56086d0a54f0 0x56086d0a58d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:43:27.396 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:27.393+0000 7f9720a2b740 1 -- 172.21.15.83:0/3005348524 >> 172.21.15.83:0/3005348524 conn(0x56086d0a1a80 msgr2=0x56086d0a1e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:43:27.396 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:27.393+0000 7f9720a2b740 1 -- 172.21.15.83:0/3005348524 shutdown_connections 2024-08-19T01:43:27.396 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:27.393+0000 7f9717e00640 1 --2- 172.21.15.83:0/40996781 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f96fc0a2c00 0x7f96fc0a50c0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f9710005c40 tx=0x7f9710005bb0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:43:27.396 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:27.393+0000 7f9720a2b740 1 -- 172.21.15.83:0/3005348524 wait complete. 2024-08-19T01:43:27.396 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:27.393+0000 7f9720a2b740 1 -- start start 2024-08-19T01:43:27.402 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:27.394+0000 7f9720a2b740 1 Processor -- start 2024-08-19T01:43:27.402 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:27.394+0000 7f9720a2b740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56086d0a54f0 0x56086d0ad5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:43:27.402 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:27.394+0000 7f9720a2b740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56086d0a5e10 0x56086d0adb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:43:27.402 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:27.394+0000 7f9720a2b740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56086d0b06d0 0x56086d0ae060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:43:27.402 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:27.394+0000 7f9720a2b740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x56086d0b32a0 con 0x56086d0a5e10 2024-08-19T01:43:27.402 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:27.394+0000 7f9720a2b740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x56086cfc5cf0 con 0x56086d0a54f0 2024-08-19T01:43:27.402 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:27.394+0000 7f9720a2b740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x56086d0b3420 con 0x56086d0b06d0 2024-08-19T01:43:27.402 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:27.394+0000 7f9720a2b740 10 client.0 _ll_register_callbacks cb 0x56086cfb1e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-19T01:43:27.402 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:27.394+0000 7f970ea00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56086d0a5e10 0x56086d0adb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:43:27.402 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:27.394+0000 7f970ea00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56086d0a5e10 0x56086d0adb20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:50880/0 (socket says 172.21.15.83:50880) 2024-08-19T01:43:27.403 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:27.394+0000 7f970ea00640 1 -- 172.21.15.83:0/1494009522 learned_addr learned my addr 172.21.15.83:0/1494009522 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-08-19T01:43:27.403 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:27.394+0000 7f9717400640 1 --2- 172.21.15.83:0/1494009522 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56086d0a54f0 0x56086d0ad5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:43:27.403 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:27.395+0000 7f9717e00640 1 --2- 172.21.15.83:0/1494009522 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56086d0b06d0 0x56086d0ae060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:43:27.403 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:27.395+0000 7f9717400640 1 -- 172.21.15.83:0/1494009522 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56086d0b06d0 msgr2=0x56086d0ae060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:43:27.403 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:27.395+0000 7f9717400640 1 --2- 172.21.15.83:0/1494009522 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56086d0b06d0 0x56086d0ae060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:43:27.403 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:27.395+0000 7f9717400640 1 -- 172.21.15.83:0/1494009522 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56086d0a5e10 msgr2=0x56086d0adb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:43:27.403 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:27.395+0000 7f9717400640 1 --2- 172.21.15.83:0/1494009522 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56086d0a5e10 0x56086d0adb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:43:27.403 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:27.395+0000 7f9717400640 1 -- 172.21.15.83:0/1494009522 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56086cd7c3c0 con 0x56086d0a54f0 2024-08-19T01:43:27.403 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:27.395+0000 7f970ea00640 1 --2- 172.21.15.83:0/1494009522 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56086d0a5e10 0x56086d0adb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-19T01:43:27.403 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:27.395+0000 7f9717400640 1 --2- 172.21.15.83:0/1494009522 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56086d0a54f0 0x56086d0ad5e0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x56086d0a28e0 tx=0x7f97080cf770 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9f606c1669313107 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:43:27.403 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:27.395+0000 7f9704c00640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-08-19T01:43:27.404 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:27.395+0000 7f9704c00640 1 -- 172.21.15.83:0/1494009522 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f970809a480 con 0x56086d0a54f0 2024-08-19T01:43:27.404 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:27.395+0000 7f9720a2b740 1 -- 172.21.15.83:0/1494009522 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9708098340 con 0x56086d0a54f0 2024-08-19T01:43:27.404 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:27.395+0000 7f9704c00640 1 -- 172.21.15.83:0/1494009522 <== mon.2 v2:172.21.15.117:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8269+0+0 (secure 0 0 0) 0x7f97080cd080 con 0x56086d0a54f0 2024-08-19T01:43:27.404 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:27.396+0000 7f9704c00640 1 -- 172.21.15.83:0/1494009522 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f970809a800 con 0x56086d0a54f0 2024-08-19T01:43:27.404 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:27.396+0000 7f9720a2b740 10 client.26309 fetch_fsmap learned FSMap version 31 2024-08-19T01:43:27.404 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:27.396+0000 7f9720a2b740 1 -- 172.21.15.83:0/1494009522 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x56086d0b32a0 con 0x56086d0a54f0 2024-08-19T01:43:27.404 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:27.396+0000 7f9704c00640 1 -- 172.21.15.83:0/1494009522 <== mon.2 v2:172.21.15.117:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9708098340 con 0x56086d0a54f0 2024-08-19T01:43:27.404 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:27.396+0000 7f9704c00640 1 -- 172.21.15.83:0/1494009522 <== mon.2 v2:172.21.15.117:3300/0 5 ==== fsmap(e 31) ==== 8354+0+0 (secure 0 0 0) 0x7f97080a3ed0 con 0x56086d0a54f0 2024-08-19T01:43:27.404 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:27.396+0000 7f9720a2b740 10 client.26309 fetch_fsmap finished waiting for FSMap version 31 2024-08-19T01:43:27.404 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:27.396+0000 7f9720a2b740 10 client.26309 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.g 2024-08-19T01:43:27.404 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:27.396+0000 7f9720a2b740 20 client.26309 populate_metadata read hostname 'smithi083' 2024-08-19T01:43:27.404 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:27.396+0000 7f9720a2b740 1 --2- 172.21.15.83:0/1494009522 >> [v2:172.21.15.83:6838/3239261845,v1:172.21.15.83:6839/3239261845] conn(0x56086d0c0100 0x56086d0c24f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:43:27.404 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:27.396+0000 7f9720a2b740 4 client.26309 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-19T01:43:27.404 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:27.396+0000 7f9720a2b740 1 -- 172.21.15.83:0/1494009522 --> [v2:172.21.15.83:6838/3239261845,v1:172.21.15.83:6839/3239261845] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56086d018160 con 0x56086d0c0100 2024-08-19T01:43:27.405 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:27.397+0000 7f970ea00640 1 --2- 172.21.15.83:0/1494009522 >> [v2:172.21.15.83:6838/3239261845,v1:172.21.15.83:6839/3239261845] conn(0x56086d0c0100 0x56086d0c24f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:43:27.405 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:27.398+0000 7f9704c00640 10 client.26309 ms_handle_connect on v2:172.21.15.83:6838/3239261845 2024-08-19T01:43:27.405 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:27.397+0000 7f970ea00640 1 --2- 172.21.15.83:0/1494009522 >> [v2:172.21.15.83:6838/3239261845,v1:172.21.15.83:6839/3239261845] conn(0x56086d0c0100 0x56086d0c24f0 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=d8b7740c056d1e41 server_cookie=ef5ee46ec0565b1c in_seq=0 out_seq=0 2024-08-19T01:43:27.405 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:27.398+0000 7f9704c00640 1 -- 172.21.15.83:0/1494009522 <== mds.1 v2:172.21.15.83:6838/3239261845 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x56086d018160 con 0x56086d0c0100 2024-08-19T01:43:27.405 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:27.398+0000 7f9704c00640 10 client.26309 handle_command_reply: tid=0 2024-08-19T01:43:27.405 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:27.398+0000 7f9720a2b740 1 client.26309 shutdown 2024-08-19T01:43:27.405 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:27.398+0000 7f9720a2b740 1 -- 172.21.15.83:0/1494009522 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56086d0a54f0 msgr2=0x56086d0ad5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:43:27.405 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:27.398+0000 7f9720a2b740 1 --2- 172.21.15.83:0/1494009522 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56086d0a54f0 0x56086d0ad5e0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x56086d0a28e0 tx=0x7f97080cf770 comp rx=0 tx=0).stop 2024-08-19T01:43:27.405 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:27.399+0000 7f9720a2b740 1 -- 172.21.15.83:0/1494009522 shutdown_connections 2024-08-19T01:43:27.405 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:27.399+0000 7f9720a2b740 1 --2- 172.21.15.83:0/1494009522 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56086d0b06d0 0x56086d0ae060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:43:27.405 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:27.399+0000 7f9720a2b740 1 --2- 172.21.15.83:0/1494009522 >> [v2:172.21.15.83:6838/3239261845,v1:172.21.15.83:6839/3239261845] conn(0x56086d0c0100 0x56086d0c24f0 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:43:27.405 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:27.399+0000 7f9720a2b740 1 --2- 172.21.15.83:0/1494009522 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56086d0a5e10 0x56086d0adb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:43:27.405 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:27.399+0000 7f9720a2b740 1 --2- 172.21.15.83:0/1494009522 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56086d0a54f0 0x56086d0ad5e0 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:43:27.405 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:27.399+0000 7f9704c00640 0 client.26309 ms_handle_reset on v2:172.21.15.83:6838/3239261845 2024-08-19T01:43:27.406 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:27.399+0000 7f9720a2b740 1 -- 172.21.15.83:0/1494009522 >> 172.21.15.83:0/1494009522 conn(0x56086d0a1a80 msgr2=0x56086d0a0c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:43:27.406 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:27.399+0000 7f9720a2b740 1 -- 172.21.15.83:0/1494009522 shutdown_connections 2024-08-19T01:43:27.406 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:27.399+0000 7f9720a2b740 1 -- 172.21.15.83:0/1494009522 wait complete. 2024-08-19T01:43:27.406 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:27.399+0000 7f9720a2b740 20 client.26309 trim_cache size 0 max 16384 2024-08-19T01:43:27.417 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:27.412+0000 7f9720a2b740 1 Processor -- start 2024-08-19T01:43:27.417 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:27.412+0000 7f9720a2b740 1 -- start start 2024-08-19T01:43:27.417 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:27.413+0000 7f9720a2b740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56086d0a54f0 0x56086d0c7d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:43:27.418 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:27.413+0000 7f9720a2b740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56086d0a5e10 0x56086d0c8260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:43:27.418 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:27.413+0000 7f9720a2b740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56086d0b06d0 0x56086d0a03e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:43:27.418 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:27.413+0000 7f9720a2b740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f97181b1ec0 con 0x56086d0a5e10 2024-08-19T01:43:27.418 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:27.413+0000 7f9720a2b740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f97181af730 con 0x56086d0a54f0 2024-08-19T01:43:27.418 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:27.413+0000 7f9720a2b740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f97181ad580 con 0x56086d0b06d0 2024-08-19T01:43:27.418 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:27.413+0000 7f970ea00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56086d0a5e10 0x56086d0c8260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:43:27.418 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:27.413+0000 7f970ea00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56086d0a5e10 0x56086d0c8260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:50894/0 (socket says 172.21.15.83:50894) 2024-08-19T01:43:27.418 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:27.413+0000 7f970ea00640 1 -- 172.21.15.83:0/4174751870 learned_addr learned my addr 172.21.15.83:0/4174751870 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-08-19T01:43:27.418 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:27.413+0000 7f9717e00640 1 --2- 172.21.15.83:0/4174751870 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56086d0b06d0 0x56086d0a03e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:43:27.418 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:27.413+0000 7f9717400640 1 --2- 172.21.15.83:0/4174751870 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56086d0a54f0 0x56086d0c7d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:43:27.419 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:27.413+0000 7f970ea00640 1 -- 172.21.15.83:0/4174751870 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56086d0b06d0 msgr2=0x56086d0a03e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:43:27.419 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:27.414+0000 7f970ea00640 1 --2- 172.21.15.83:0/4174751870 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56086d0b06d0 0x56086d0a03e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:43:27.419 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:27.414+0000 7f970ea00640 1 -- 172.21.15.83:0/4174751870 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56086d0a54f0 msgr2=0x56086d0c7d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:43:27.419 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:27.414+0000 7f970ea00640 1 --2- 172.21.15.83:0/4174751870 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56086d0a54f0 0x56086d0c7d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:43:27.419 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:27.414+0000 7f970ea00640 1 -- 172.21.15.83:0/4174751870 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56086d0a1520 con 0x56086d0a5e10 2024-08-19T01:43:27.419 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:27.414+0000 7f970ea00640 1 --2- 172.21.15.83:0/4174751870 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56086d0a5e10 0x56086d0c8260 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f9700009fd0 tx=0x7f9700009500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8c02e3abf3592ee1 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:43:27.419 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:27.414+0000 7f9707400640 1 -- 172.21.15.83:0/4174751870 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9700004d20 con 0x56086d0a5e10 2024-08-19T01:43:27.419 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:27.414+0000 7f9707400640 1 -- 172.21.15.83:0/4174751870 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9700003e50 con 0x56086d0a5e10 2024-08-19T01:43:27.419 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:27.414+0000 7f9707400640 1 -- 172.21.15.83:0/4174751870 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9700005020 con 0x56086d0a5e10 2024-08-19T01:43:27.419 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:27.415+0000 7f9720a2b740 1 -- 172.21.15.83:0/4174751870 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56086d0a5e10 msgr2=0x56086d0c8260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:43:27.419 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:27.415+0000 7f9720a2b740 1 --2- 172.21.15.83:0/4174751870 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56086d0a5e10 0x56086d0c8260 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f9700009fd0 tx=0x7f9700009500 comp rx=0 tx=0).stop 2024-08-19T01:43:27.419 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:27.415+0000 7f9720a2b740 1 -- 172.21.15.83:0/4174751870 shutdown_connections 2024-08-19T01:43:27.420 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:27.415+0000 7f9720a2b740 1 --2- 172.21.15.83:0/4174751870 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56086d0b06d0 0x56086d0a03e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:43:27.420 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:27.415+0000 7f9720a2b740 1 --2- 172.21.15.83:0/4174751870 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56086d0a5e10 0x56086d0c8260 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:43:27.420 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:27.415+0000 7f9720a2b740 1 --2- 172.21.15.83:0/4174751870 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56086d0a54f0 0x56086d0c7d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:43:27.420 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:27.415+0000 7f9720a2b740 1 -- 172.21.15.83:0/4174751870 >> 172.21.15.83:0/4174751870 conn(0x56086d0a1a80 msgr2=0x56086d0a8910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:43:27.420 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:27.415+0000 7f9720a2b740 1 -- 172.21.15.83:0/4174751870 shutdown_connections 2024-08-19T01:43:27.420 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:27.415+0000 7f9720a2b740 1 -- 172.21.15.83:0/4174751870 wait complete. 2024-08-19T01:43:27.420 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:27.415+0000 7f9720a2b740 1 -- start start 2024-08-19T01:43:27.420 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:27.416+0000 7f9720a2b740 1 Processor -- start 2024-08-19T01:43:27.423 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:27.416+0000 7f9720a2b740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56086d0a54f0 0x56086d0b3df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:43:27.423 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:27.416+0000 7f9720a2b740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56086d0a5e10 0x56086d0b4330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:43:27.423 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:27.416+0000 7f9720a2b740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56086d0b06d0 0x56086d0b8420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:43:27.424 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:27.416+0000 7f9720a2b740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f97181adbf0 con 0x56086d0a54f0 2024-08-19T01:43:27.424 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:27.416+0000 7f9720a2b740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f97181af060 con 0x56086d0b06d0 2024-08-19T01:43:27.424 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:27.416+0000 7f9720a2b740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f97181ae2c0 con 0x56086d0a5e10 2024-08-19T01:43:27.424 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:27.416+0000 7f9720a2b740 10 client.0 _ll_register_callbacks cb 0x7f9700009f00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-19T01:43:27.424 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:27.416+0000 7f9717400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56086d0a54f0 0x56086d0b3df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:43:27.424 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:27.416+0000 7f9717400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56086d0a54f0 0x56086d0b3df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:50910/0 (socket says 172.21.15.83:50910) 2024-08-19T01:43:27.424 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:27.416+0000 7f9717400640 1 -- 172.21.15.83:0/1655333857 learned_addr learned my addr 172.21.15.83:0/1655333857 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-08-19T01:43:27.424 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:27.416+0000 7f970ea00640 1 --2- 172.21.15.83:0/1655333857 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56086d0a5e10 0x56086d0b4330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:43:27.424 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:27.417+0000 7f9717e00640 1 --2- 172.21.15.83:0/1655333857 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56086d0b06d0 0x56086d0b8420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:43:27.424 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:27.417+0000 7f9717400640 1 -- 172.21.15.83:0/1655333857 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56086d0a5e10 msgr2=0x56086d0b4330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:43:27.424 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:27.417+0000 7f9717400640 1 --2- 172.21.15.83:0/1655333857 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56086d0a5e10 0x56086d0b4330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:43:27.425 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:27.417+0000 7f9717400640 1 -- 172.21.15.83:0/1655333857 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56086d0b06d0 msgr2=0x56086d0b8420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:43:27.425 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:27.417+0000 7f9717400640 1 --2- 172.21.15.83:0/1655333857 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56086d0b06d0 0x56086d0b8420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:43:27.425 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:27.417+0000 7f9717400640 1 -- 172.21.15.83:0/1655333857 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56086d0b8a90 con 0x56086d0a54f0 2024-08-19T01:43:27.425 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:27.417+0000 7f9717400640 1 --2- 172.21.15.83:0/1655333857 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56086d0a54f0 0x56086d0b3df0 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f970809bc60 tx=0x7f9708005b30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fe36aedd774dbb6c server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:43:27.425 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:27.417+0000 7f9704c00640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-08-19T01:43:27.425 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:27.417+0000 7f9704c00640 1 -- 172.21.15.83:0/1655333857 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f970809a9d0 con 0x56086d0a54f0 2024-08-19T01:43:27.425 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:27.417+0000 7f9704c00640 1 -- 172.21.15.83:0/1655333857 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8269+0+0 (secure 0 0 0) 0x7f9708099d70 con 0x56086d0a54f0 2024-08-19T01:43:27.425 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:27.417+0000 7f9720a2b740 1 -- 172.21.15.83:0/1655333857 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9700003e50 con 0x56086d0a54f0 2024-08-19T01:43:27.425 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:27.418+0000 7f9704c00640 1 -- 172.21.15.83:0/1655333857 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f970809b120 con 0x56086d0a54f0 2024-08-19T01:43:27.425 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:27.418+0000 7f9720a2b740 10 client.16263 fetch_fsmap learned FSMap version 31 2024-08-19T01:43:27.425 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:27.418+0000 7f9720a2b740 1 -- 172.21.15.83:0/1655333857 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x56086d132fb0 con 0x56086d0a54f0 2024-08-19T01:43:27.425 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:27.418+0000 7f9704c00640 1 -- 172.21.15.83:0/1655333857 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9700003e50 con 0x56086d0a54f0 2024-08-19T01:43:27.425 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:27.418+0000 7f9704c00640 1 -- 172.21.15.83:0/1655333857 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 31) ==== 8354+0+0 (secure 0 0 0) 0x7f97080ccff0 con 0x56086d0a54f0 2024-08-19T01:43:27.425 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:27.418+0000 7f9720a2b740 10 client.16263 fetch_fsmap finished waiting for FSMap version 31 2024-08-19T01:43:27.426 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:27.418+0000 7f9720a2b740 10 client.16263 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.g 2024-08-19T01:43:27.426 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:27.418+0000 7f9720a2b740 20 client.16263 populate_metadata read hostname 'smithi083' 2024-08-19T01:43:27.426 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:27.418+0000 7f9720a2b740 1 --2- 172.21.15.83:0/1655333857 >> [v2:172.21.15.83:6838/3239261845,v1:172.21.15.83:6839/3239261845] conn(0x56086d1334e0 0x56086d1358d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:43:27.426 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:27.418+0000 7f9720a2b740 4 client.16263 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-08-19T01:43:27.426 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:27.418+0000 7f9720a2b740 1 -- 172.21.15.83:0/1655333857 --> [v2:172.21.15.83:6838/3239261845,v1:172.21.15.83:6839/3239261845] -- command(tid 0: {"prefix": "damage ls"}) -- 0x56086d012670 con 0x56086d1334e0 2024-08-19T01:43:27.426 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:27.419+0000 7f970ea00640 1 --2- 172.21.15.83:0/1655333857 >> [v2:172.21.15.83:6838/3239261845,v1:172.21.15.83:6839/3239261845] conn(0x56086d1334e0 0x56086d1358d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:43:27.426 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:27.419+0000 7f970ea00640 1 --2- 172.21.15.83:0/1655333857 >> [v2:172.21.15.83:6838/3239261845,v1:172.21.15.83:6839/3239261845] conn(0x56086d1334e0 0x56086d1358d0 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=feea755529d482d5 server_cookie=ecb83e8d354800b5 in_seq=0 out_seq=0 2024-08-19T01:43:27.426 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:27.419+0000 7f9704c00640 10 client.16263 ms_handle_connect on v2:172.21.15.83:6838/3239261845 2024-08-19T01:43:27.426 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:27.419+0000 7f9704c00640 1 -- 172.21.15.83:0/1655333857 <== mds.1 v2:172.21.15.83:6838/3239261845 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x56086d012670 con 0x56086d1334e0 2024-08-19T01:43:27.426 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:27.419+0000 7f9704c00640 10 client.16263 handle_command_reply: tid=0 2024-08-19T01:43:27.426 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:27.419+0000 7f9720a2b740 1 client.16263 shutdown 2024-08-19T01:43:27.426 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:27.420+0000 7f9720a2b740 1 -- 172.21.15.83:0/1655333857 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56086d0a54f0 msgr2=0x56086d0b3df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:43:27.426 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:27.420+0000 7f9720a2b740 1 --2- 172.21.15.83:0/1655333857 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56086d0a54f0 0x56086d0b3df0 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f970809bc60 tx=0x7f9708005b30 comp rx=0 tx=0).stop 2024-08-19T01:43:27.427 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:27.420+0000 7f9720a2b740 1 -- 172.21.15.83:0/1655333857 shutdown_connections 2024-08-19T01:43:27.427 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:27.420+0000 7f9720a2b740 1 --2- 172.21.15.83:0/1655333857 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56086d0b06d0 0x56086d0b8420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:43:27.427 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:27.420+0000 7f9720a2b740 1 --2- 172.21.15.83:0/1655333857 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56086d0a5e10 0x56086d0b4330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:43:27.427 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:27.420+0000 7f9720a2b740 1 --2- 172.21.15.83:0/1655333857 >> [v2:172.21.15.83:6838/3239261845,v1:172.21.15.83:6839/3239261845] conn(0x56086d1334e0 0x56086d1358d0 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:43:27.427 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:27.420+0000 7f9720a2b740 1 --2- 172.21.15.83:0/1655333857 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56086d0a54f0 0x56086d0b3df0 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:43:27.427 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:27.420+0000 7f9704c00640 0 client.16263 ms_handle_reset on v2:172.21.15.83:6838/3239261845 2024-08-19T01:43:27.427 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:27.420+0000 7f9720a2b740 1 -- 172.21.15.83:0/1655333857 >> 172.21.15.83:0/1655333857 conn(0x56086d0a1a80 msgr2=0x56086d0aee40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:43:27.427 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:27.420+0000 7f9720a2b740 1 -- 172.21.15.83:0/1655333857 shutdown_connections 2024-08-19T01:43:27.427 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:27.420+0000 7f9720a2b740 1 -- 172.21.15.83:0/1655333857 wait complete. 2024-08-19T01:43:27.427 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:27.420+0000 7f9720a2b740 20 client.16263 trim_cache size 0 max 16384 2024-08-19T01:43:27.427 INFO:teuthology.orchestra.run.smithi083.stdout:[] 2024-08-19T01:43:27.428 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:27.421+0000 7f971e000640 1 -- 172.21.15.83:0/40996781 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f96fc0a2c00 msgr2=0x7f96fc0a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:43:27.428 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:27.421+0000 7f971e000640 1 --2- 172.21.15.83:0/40996781 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f96fc0a2c00 0x7f96fc0a50c0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f9710005c40 tx=0x7f9710005bb0 comp rx=0 tx=0).stop 2024-08-19T01:43:27.428 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:27.421+0000 7f971e000640 1 -- 172.21.15.83:0/40996781 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9718074350 msgr2=0x7f971819a3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:43:27.428 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:27.421+0000 7f971e000640 1 --2- 172.21.15.83:0/40996781 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9718074350 0x7f971819a3b0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f970800e9f0 tx=0x7f970800edf0 comp rx=0 tx=0).stop 2024-08-19T01:43:27.428 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:27.421+0000 7f971e000640 1 -- 172.21.15.83:0/40996781 shutdown_connections 2024-08-19T01:43:27.428 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:27.421+0000 7f971e000640 1 --2- 172.21.15.83:0/40996781 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f96fc0a2c00 0x7f96fc0a50c0 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:43:27.429 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:27.421+0000 7f971e000640 1 --2- 172.21.15.83:0/40996781 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f971810e0f0 0x7f971819ec80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:43:27.429 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:27.421+0000 7f971e000640 1 --2- 172.21.15.83:0/40996781 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9718074c70 0x7f971819a8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:43:27.429 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:27.421+0000 7f971e000640 1 --2- 172.21.15.83:0/40996781 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9718074350 0x7f971819a3b0 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:43:27.429 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:27.421+0000 7f971e000640 1 -- 172.21.15.83:0/40996781 >> 172.21.15.83:0/40996781 conn(0x7f97180fd5f0 msgr2=0x7f97180fe290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:43:27.429 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:27.421+0000 7f971e000640 1 -- 172.21.15.83:0/40996781 shutdown_connections 2024-08-19T01:43:27.429 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:27.421+0000 7f971e000640 1 -- 172.21.15.83:0/40996781 wait complete. 2024-08-19T01:43:27.952 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:43:27 smithi117 ceph-mon[22016]: pgmap v597: 97 pgs: 97 active+clean; 35 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 8.6 KiB/s rd, 5.5 KiB/s wr, 7 op/s 2024-08-19T01:43:27.958 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:43:27 smithi083 ceph-mon[17120]: pgmap v597: 97 pgs: 97 active+clean; 35 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 8.6 KiB/s rd, 5.5 KiB/s wr, 7 op/s 2024-08-19T01:43:27.965 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:43:27 smithi167 ceph-mon[21875]: pgmap v597: 97 pgs: 97 active+clean; 35 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 8.6 KiB/s rd, 5.5 KiB/s wr, 7 op/s 2024-08-19T01:43:28.207 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833 shell --fsid bad8780e-5dc9-11ef-bccf-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-08-19T01:43:28.579 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/bad8780e-5dc9-11ef-bccf-c7b262605968/mon.a/config 2024-08-19T01:43:29.137 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:29.135+0000 7fea99400640 1 -- 172.21.15.83:0/1967675627 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fea94074be0 msgr2=0x7fea94112390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:43:29.137 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:29.135+0000 7fea99400640 1 --2- 172.21.15.83:0/1967675627 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fea94074be0 0x7fea94112390 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7fea80009900 tx=0x7fea8002a480 comp rx=0 tx=0).stop 2024-08-19T01:43:29.137 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:29.135+0000 7fea99400640 1 -- 172.21.15.83:0/1967675627 shutdown_connections 2024-08-19T01:43:29.138 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:29.135+0000 7fea99400640 1 --2- 172.21.15.83:0/1967675627 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fea941128d0 0x7fea94114cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:43:29.138 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:29.135+0000 7fea99400640 1 --2- 172.21.15.83:0/1967675627 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fea94074be0 0x7fea94112390 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:43:29.138 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:29.135+0000 7fea99400640 1 --2- 172.21.15.83:0/1967675627 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fea940742c0 0x7fea940746a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:43:29.138 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:29.135+0000 7fea99400640 1 -- 172.21.15.83:0/1967675627 >> 172.21.15.83:0/1967675627 conn(0x7fea940fd5e0 msgr2=0x7fea940ffa00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:43:29.138 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:29.135+0000 7fea99400640 1 -- 172.21.15.83:0/1967675627 shutdown_connections 2024-08-19T01:43:29.138 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:29.135+0000 7fea99400640 1 -- 172.21.15.83:0/1967675627 wait complete. 2024-08-19T01:43:29.138 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:29.136+0000 7fea99400640 1 Processor -- start 2024-08-19T01:43:29.138 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:29.136+0000 7fea99400640 1 -- start start 2024-08-19T01:43:29.140 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:29.136+0000 7fea99400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fea940742c0 0x7fea941a4440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:43:29.140 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:29.136+0000 7fea99400640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fea94074be0 0x7fea941a4980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:43:29.140 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:29.136+0000 7fea99400640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fea941128d0 0x7fea9419e600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:43:29.140 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:29.136+0000 7fea99400640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fea941182f0 con 0x7fea940742c0 2024-08-19T01:43:29.140 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:29.136+0000 7fea99400640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fea94118170 con 0x7fea94074be0 2024-08-19T01:43:29.140 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:29.136+0000 7fea99400640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fea94118470 con 0x7fea941128d0 2024-08-19T01:43:29.140 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:29.137+0000 7fea92800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fea940742c0 0x7fea941a4440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:43:29.140 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:29.137+0000 7fea92800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fea940742c0 0x7fea941a4440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:50922/0 (socket says 172.21.15.83:50922) 2024-08-19T01:43:29.140 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:29.137+0000 7fea92800640 1 -- 172.21.15.83:0/2822225432 learned_addr learned my addr 172.21.15.83:0/2822225432 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-08-19T01:43:29.140 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:29.137+0000 7fea91e00640 1 --2- 172.21.15.83:0/2822225432 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fea94074be0 0x7fea941a4980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:43:29.140 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:29.137+0000 7fea93200640 1 --2- 172.21.15.83:0/2822225432 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fea941128d0 0x7fea9419e600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:43:29.141 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:29.137+0000 7fea92800640 1 -- 172.21.15.83:0/2822225432 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fea941128d0 msgr2=0x7fea9419e600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:43:29.141 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:29.137+0000 7fea92800640 1 --2- 172.21.15.83:0/2822225432 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fea941128d0 0x7fea9419e600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:43:29.141 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:29.137+0000 7fea92800640 1 -- 172.21.15.83:0/2822225432 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fea94074be0 msgr2=0x7fea941a4980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:43:29.141 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:29.137+0000 7fea92800640 1 --2- 172.21.15.83:0/2822225432 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fea94074be0 0x7fea941a4980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:43:29.141 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:29.137+0000 7fea92800640 1 -- 172.21.15.83:0/2822225432 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fea9419ed30 con 0x7fea940742c0 2024-08-19T01:43:29.141 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:29.137+0000 7fea92800640 1 --2- 172.21.15.83:0/2822225432 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fea940742c0 0x7fea941a4440 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7fea84009d50 tx=0x7fea8400c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:43:29.141 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:29.138+0000 7fea8b400640 1 -- 172.21.15.83:0/2822225432 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea84003b70 con 0x7fea940742c0 2024-08-19T01:43:29.141 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:29.138+0000 7fea99400640 1 -- 172.21.15.83:0/2822225432 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fea9419f020 con 0x7fea940742c0 2024-08-19T01:43:29.141 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:29.138+0000 7fea8b400640 1 -- 172.21.15.83:0/2822225432 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fea84003d10 con 0x7fea940742c0 2024-08-19T01:43:29.141 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:29.138+0000 7fea8b400640 1 -- 172.21.15.83:0/2822225432 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea84010a20 con 0x7fea940742c0 2024-08-19T01:43:29.141 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:29.138+0000 7fea99400640 1 -- 172.21.15.83:0/2822225432 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fea941ab310 con 0x7fea940742c0 2024-08-19T01:43:29.142 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:29.141+0000 7fea8b400640 1 -- 172.21.15.83:0/2822225432 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7fea84004000 con 0x7fea940742c0 2024-08-19T01:43:29.143 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:29.142+0000 7fea9bfe9740 1 Processor -- start 2024-08-19T01:43:29.143 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:29.142+0000 7fea9bfe9740 1 -- start start 2024-08-19T01:43:29.144 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:29.142+0000 7fea9bfe9740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55729e24b4f0 0x55729e24b8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:43:29.144 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:29.142+0000 7fea9bfe9740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55729e24be10 0x55729e256190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:43:29.144 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:29.142+0000 7fea9bfe9740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55729e2566d0 0x55729e258ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:43:29.144 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:29.142+0000 7fea9bfe9740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55729e1b0bf0 con 0x55729e24b4f0 2024-08-19T01:43:29.144 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:29.142+0000 7fea9bfe9740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55729e148b20 con 0x55729e24be10 2024-08-19T01:43:29.144 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:29.142+0000 7fea9bfe9740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55729e1acbb0 con 0x55729e2566d0 2024-08-19T01:43:29.144 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:29.142+0000 7fea93200640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55729e24b4f0 0x55729e24b8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:43:29.145 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:29.143+0000 7fea92800640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55729e24be10 0x55729e256190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:43:29.145 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:29.143+0000 7fea93200640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55729e24b4f0 0x55729e24b8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:50930/0 (socket says 172.21.15.83:50930) 2024-08-19T01:43:29.145 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:29.143+0000 7fea93200640 1 -- 172.21.15.83:0/1869431037 learned_addr learned my addr 172.21.15.83:0/1869431037 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-08-19T01:43:29.145 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:29.143+0000 7fea91e00640 1 --2- 172.21.15.83:0/1869431037 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55729e2566d0 0x55729e258ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:43:29.145 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:29.143+0000 7fea8b400640 1 --2- 172.21.15.83:0/2822225432 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7fea780a2c00 0x7fea780a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:43:29.146 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:29.143+0000 7fea8b400640 1 -- 172.21.15.83:0/2822225432 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8269+0+0 (secure 0 0 0) 0x7fea840cf700 con 0x7fea940742c0 2024-08-19T01:43:29.146 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:29.143+0000 7fea92800640 1 -- 172.21.15.83:0/1869431037 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55729e2566d0 msgr2=0x55729e258ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:43:29.146 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:29.143+0000 7fea92800640 1 --2- 172.21.15.83:0/1869431037 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55729e2566d0 0x55729e258ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:43:29.146 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:29.143+0000 7fea92800640 1 -- 172.21.15.83:0/1869431037 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55729e24b4f0 msgr2=0x55729e24b8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:43:29.146 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:29.143+0000 7fea92800640 1 --2- 172.21.15.83:0/1869431037 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55729e24b4f0 0x55729e24b8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:43:29.146 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:29.143+0000 7fea92800640 1 -- 172.21.15.83:0/1869431037 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55729e14f7f0 con 0x55729e24be10 2024-08-19T01:43:29.146 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:29.143+0000 7fea91e00640 1 --2- 172.21.15.83:0/1869431037 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55729e2566d0 0x55729e258ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-19T01:43:29.146 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:29.143+0000 7fea93200640 1 --2- 172.21.15.83:0/2822225432 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7fea780a2c00 0x7fea780a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:43:29.146 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:29.143+0000 7fea92800640 1 --2- 172.21.15.83:0/1869431037 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55729e24be10 0x55729e256190 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7fea840cd120 tx=0x7fea84010430 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ce6bf168e7bfe794 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:43:29.146 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:29.144+0000 7fea7f400640 1 -- 172.21.15.83:0/1869431037 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea8409bcb0 con 0x55729e24be10 2024-08-19T01:43:29.146 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:29.144+0000 7fea7f400640 1 -- 172.21.15.83:0/1869431037 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fea84009820 con 0x55729e24be10 2024-08-19T01:43:29.146 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:29.144+0000 7fea7f400640 1 -- 172.21.15.83:0/1869431037 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea8409b840 con 0x55729e24be10 2024-08-19T01:43:29.148 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:29.145+0000 7fea9bfe9740 1 -- 172.21.15.83:0/1869431037 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55729e24be10 msgr2=0x55729e256190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:43:29.149 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:29.145+0000 7fea9bfe9740 1 --2- 172.21.15.83:0/1869431037 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55729e24be10 0x55729e256190 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7fea840cd120 tx=0x7fea84010430 comp rx=0 tx=0).stop 2024-08-19T01:43:29.149 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:29.145+0000 7fea93200640 1 --2- 172.21.15.83:0/2822225432 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7fea780a2c00 0x7fea780a50c0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7fea8c0045e0 tx=0x7fea8c0046c0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:43:29.149 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:29.145+0000 7fea9bfe9740 1 -- 172.21.15.83:0/1869431037 shutdown_connections 2024-08-19T01:43:29.149 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:29.145+0000 7fea9bfe9740 1 --2- 172.21.15.83:0/1869431037 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55729e2566d0 0x55729e258ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:43:29.149 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:29.145+0000 7fea9bfe9740 1 --2- 172.21.15.83:0/1869431037 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55729e24be10 0x55729e256190 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:43:29.149 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:29.145+0000 7fea9bfe9740 1 --2- 172.21.15.83:0/1869431037 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55729e24b4f0 0x55729e24b8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:43:29.149 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:29.145+0000 7fea9bfe9740 1 -- 172.21.15.83:0/1869431037 >> 172.21.15.83:0/1869431037 conn(0x55729e247a80 msgr2=0x55729e247e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:43:29.149 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:29.145+0000 7fea9bfe9740 1 -- 172.21.15.83:0/1869431037 shutdown_connections 2024-08-19T01:43:29.149 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:29.145+0000 7fea9bfe9740 1 -- 172.21.15.83:0/1869431037 wait complete. 2024-08-19T01:43:29.149 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:29.146+0000 7fea9bfe9740 1 -- start start 2024-08-19T01:43:29.149 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:29.146+0000 7fea9bfe9740 1 Processor -- start 2024-08-19T01:43:29.149 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:29.146+0000 7fea9bfe9740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55729e24b4f0 0x55729e253580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:43:29.149 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:29.146+0000 7fea9bfe9740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55729e24be10 0x55729e253ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:43:29.150 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:29.146+0000 7fea9bfe9740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55729e2566d0 0x55729e254000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:43:29.150 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:29.146+0000 7fea9bfe9740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55729e2592a0 con 0x55729e24b4f0 2024-08-19T01:43:29.150 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:29.146+0000 7fea9bfe9740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55729e16bcf0 con 0x55729e2566d0 2024-08-19T01:43:29.150 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:29.146+0000 7fea9bfe9740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55729e259420 con 0x55729e24be10 2024-08-19T01:43:29.150 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:29.146+0000 7fea9bfe9740 10 client.0 _ll_register_callbacks cb 0x55729e157e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-19T01:43:29.150 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:29.147+0000 7fea92800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55729e24b4f0 0x55729e253580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:43:29.150 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:29.147+0000 7fea92800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55729e24b4f0 0x55729e253580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:50940/0 (socket says 172.21.15.83:50940) 2024-08-19T01:43:29.150 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:29.147+0000 7fea92800640 1 -- 172.21.15.83:0/3125244923 learned_addr learned my addr 172.21.15.83:0/3125244923 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-08-19T01:43:29.150 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:29.147+0000 7fea91e00640 1 --2- 172.21.15.83:0/3125244923 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55729e24be10 0x55729e253ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:43:29.151 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:29.147+0000 7fea93200640 1 --2- 172.21.15.83:0/3125244923 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55729e2566d0 0x55729e254000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:43:29.151 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:29.147+0000 7fea92800640 1 -- 172.21.15.83:0/3125244923 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55729e24be10 msgr2=0x55729e253ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:43:29.151 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:29.147+0000 7fea92800640 1 --2- 172.21.15.83:0/3125244923 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55729e24be10 0x55729e253ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:43:29.151 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:29.147+0000 7fea92800640 1 -- 172.21.15.83:0/3125244923 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55729e2566d0 msgr2=0x55729e254000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:43:29.151 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:29.147+0000 7fea92800640 1 --2- 172.21.15.83:0/3125244923 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55729e2566d0 0x55729e254000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:43:29.151 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:29.147+0000 7fea92800640 1 -- 172.21.15.83:0/3125244923 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55729df223c0 con 0x55729e24b4f0 2024-08-19T01:43:29.151 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:29.147+0000 7fea92800640 1 --2- 172.21.15.83:0/3125244923 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55729e24b4f0 0x55729e253580 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7fea8409a190 tx=0x7fea84097ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4db2f49f88e9aae7 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:43:29.151 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:29.147+0000 7fea73e00640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-08-19T01:43:29.153 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:29.147+0000 7fea73e00640 1 -- 172.21.15.83:0/3125244923 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea8409ba20 con 0x55729e24b4f0 2024-08-19T01:43:29.153 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:29.148+0000 7fea73e00640 1 -- 172.21.15.83:0/3125244923 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8269+0+0 (secure 0 0 0) 0x7fea84098550 con 0x55729e24b4f0 2024-08-19T01:43:29.153 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:29.148+0000 7fea9bfe9740 1 -- 172.21.15.83:0/3125244923 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fea84009820 con 0x55729e24b4f0 2024-08-19T01:43:29.153 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:29.148+0000 7fea73e00640 1 -- 172.21.15.83:0/3125244923 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7fea8409bcb0 con 0x55729e24b4f0 2024-08-19T01:43:29.154 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:29.148+0000 7fea9bfe9740 10 client.16275 fetch_fsmap learned FSMap version 31 2024-08-19T01:43:29.154 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:29.148+0000 7fea9bfe9740 1 -- 172.21.15.83:0/3125244923 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55729e264f20 con 0x55729e24b4f0 2024-08-19T01:43:29.154 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:29.148+0000 7fea73e00640 1 -- 172.21.15.83:0/3125244923 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fea84009820 con 0x55729e24b4f0 2024-08-19T01:43:29.154 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:29.148+0000 7fea73e00640 1 -- 172.21.15.83:0/3125244923 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 31) ==== 8354+0+0 (secure 0 0 0) 0x7fea84010bc0 con 0x55729e24b4f0 2024-08-19T01:43:29.154 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:29.148+0000 7fea9bfe9740 10 client.16275 fetch_fsmap finished waiting for FSMap version 31 2024-08-19T01:43:29.154 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:29.148+0000 7fea9bfe9740 10 client.16275 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-19T01:43:29.154 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:29.148+0000 7fea9bfe9740 20 client.16275 populate_metadata read hostname 'smithi083' 2024-08-19T01:43:29.154 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:29.148+0000 7fea9bfe9740 1 --2- 172.21.15.83:0/3125244923 >> [v2:172.21.15.117:6838/1967595704,v1:172.21.15.117:6839/1967595704] conn(0x55729e266270 0x55729e268660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:43:29.154 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:29.149+0000 7fea9bfe9740 4 client.16275 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-19T01:43:29.154 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:29.149+0000 7fea9bfe9740 1 -- 172.21.15.83:0/3125244923 --> [v2:172.21.15.117:6838/1967595704,v1:172.21.15.117:6839/1967595704] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55729e1be160 con 0x55729e266270 2024-08-19T01:43:29.154 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:29.149+0000 7fea91e00640 1 --2- 172.21.15.83:0/3125244923 >> [v2:172.21.15.117:6838/1967595704,v1:172.21.15.117:6839/1967595704] conn(0x55729e266270 0x55729e268660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:43:29.154 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:29.149+0000 7fea91e00640 1 --2- 172.21.15.83:0/3125244923 >> [v2:172.21.15.117:6838/1967595704,v1:172.21.15.117:6839/1967595704] conn(0x55729e266270 0x55729e268660 crc :-1 s=READY pgs=149 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6a8662928f4b2b5c server_cookie=ccc2ad034ed23009 in_seq=0 out_seq=0 2024-08-19T01:43:29.154 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:29.149+0000 7fea73e00640 10 client.16275 ms_handle_connect on v2:172.21.15.117:6838/1967595704 2024-08-19T01:43:29.155 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:29.150+0000 7fea73e00640 1 -- 172.21.15.83:0/3125244923 <== mds.0 v2:172.21.15.117:6838/1967595704 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55729e1be160 con 0x55729e266270 2024-08-19T01:43:29.155 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:29.150+0000 7fea73e00640 10 client.16275 handle_command_reply: tid=0 2024-08-19T01:43:29.155 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:29.150+0000 7fea9bfe9740 1 client.16275 shutdown 2024-08-19T01:43:29.155 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:29.150+0000 7fea9bfe9740 1 -- 172.21.15.83:0/3125244923 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55729e24b4f0 msgr2=0x55729e253580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:43:29.155 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:29.150+0000 7fea9bfe9740 1 --2- 172.21.15.83:0/3125244923 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55729e24b4f0 0x55729e253580 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7fea8409a190 tx=0x7fea84097ad0 comp rx=0 tx=0).stop 2024-08-19T01:43:29.155 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:29.151+0000 7fea9bfe9740 1 -- 172.21.15.83:0/3125244923 shutdown_connections 2024-08-19T01:43:29.155 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:29.151+0000 7fea9bfe9740 1 --2- 172.21.15.83:0/3125244923 >> [v2:172.21.15.117:6838/1967595704,v1:172.21.15.117:6839/1967595704] conn(0x55729e266270 0x55729e268660 crc :-1 s=READY pgs=149 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:43:29.155 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:29.151+0000 7fea9bfe9740 1 --2- 172.21.15.83:0/3125244923 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55729e2566d0 0x55729e254000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:43:29.155 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:29.151+0000 7fea9bfe9740 1 --2- 172.21.15.83:0/3125244923 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55729e24be10 0x55729e253ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:43:29.155 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:29.151+0000 7fea9bfe9740 1 --2- 172.21.15.83:0/3125244923 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55729e24b4f0 0x55729e253580 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:43:29.155 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:29.151+0000 7fea9bfe9740 1 -- 172.21.15.83:0/3125244923 >> 172.21.15.83:0/3125244923 conn(0x55729e247a80 msgr2=0x55729e24d7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:43:29.155 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:29.151+0000 7fea73e00640 0 client.16275 ms_handle_reset on v2:172.21.15.117:6838/1967595704 2024-08-19T01:43:29.155 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:29.151+0000 7fea9bfe9740 1 -- 172.21.15.83:0/3125244923 shutdown_connections 2024-08-19T01:43:29.156 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:29.151+0000 7fea9bfe9740 1 -- 172.21.15.83:0/3125244923 wait complete. 2024-08-19T01:43:29.156 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:29.151+0000 7fea9bfe9740 20 client.16275 trim_cache size 0 max 16384 2024-08-19T01:43:29.166 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:29.164+0000 7fea9bfe9740 1 Processor -- start 2024-08-19T01:43:29.166 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:29.164+0000 7fea9bfe9740 1 -- start start 2024-08-19T01:43:29.167 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:29.165+0000 7fea9bfe9740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55729e24b4f0 0x55729e247e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:43:29.167 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:29.165+0000 7fea9bfe9740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55729e24be10 0x55729e253e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:43:29.167 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:29.165+0000 7fea9bfe9740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55729e2566d0 0x55729e261d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:43:29.167 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:29.165+0000 7fea9bfe9740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fea941b6340 con 0x55729e2566d0 2024-08-19T01:43:29.167 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:29.165+0000 7fea9bfe9740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fea941b3bb0 con 0x55729e24be10 2024-08-19T01:43:29.167 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:29.165+0000 7fea9bfe9740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fea941b1a00 con 0x55729e24b4f0 2024-08-19T01:43:29.167 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:29.165+0000 7fea91e00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55729e24be10 0x55729e253e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:43:29.167 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:29.165+0000 7fea92800640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55729e24b4f0 0x55729e247e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:43:29.167 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:29.165+0000 7fea91e00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55729e24be10 0x55729e253e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.83:50388/0 (socket says 172.21.15.83:50388) 2024-08-19T01:43:29.168 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:29.165+0000 7fea91e00640 1 -- 172.21.15.83:0/3634765450 learned_addr learned my addr 172.21.15.83:0/3634765450 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-08-19T01:43:29.168 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:29.165+0000 7fea93200640 1 --2- 172.21.15.83:0/3634765450 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55729e2566d0 0x55729e261d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:43:29.168 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:29.165+0000 7fea91e00640 1 -- 172.21.15.83:0/3634765450 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55729e24b4f0 msgr2=0x55729e247e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:43:29.168 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:29.165+0000 7fea91e00640 1 --2- 172.21.15.83:0/3634765450 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55729e24b4f0 0x55729e247e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:43:29.168 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:29.165+0000 7fea91e00640 1 -- 172.21.15.83:0/3634765450 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55729e2566d0 msgr2=0x55729e261d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:43:29.168 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:29.165+0000 7fea91e00640 1 --2- 172.21.15.83:0/3634765450 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55729e2566d0 0x55729e261d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:43:29.168 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:29.165+0000 7fea91e00640 1 -- 172.21.15.83:0/3634765450 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55729e252530 con 0x55729e24be10 2024-08-19T01:43:29.168 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:29.165+0000 7fea93200640 1 --2- 172.21.15.83:0/3634765450 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55729e2566d0 0x55729e261d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-19T01:43:29.168 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:29.165+0000 7fea92800640 1 --2- 172.21.15.83:0/3634765450 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55729e24b4f0 0x55729e247e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-19T01:43:29.168 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:29.166+0000 7fea91e00640 1 --2- 172.21.15.83:0/3634765450 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55729e24be10 0x55729e253e50 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7fea80033000 tx=0x7fea80005b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5bc10055c8f77f93 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:43:29.171 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:29.166+0000 7fea7ea00640 1 -- 172.21.15.83:0/3634765450 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea80035040 con 0x55729e24be10 2024-08-19T01:43:29.171 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:29.166+0000 7fea7ea00640 1 -- 172.21.15.83:0/3634765450 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fea8004b070 con 0x55729e24be10 2024-08-19T01:43:29.171 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:29.166+0000 7fea7ea00640 1 -- 172.21.15.83:0/3634765450 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea80031810 con 0x55729e24be10 2024-08-19T01:43:29.171 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:29.167+0000 7fea9bfe9740 1 -- 172.21.15.83:0/3634765450 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55729e24be10 msgr2=0x55729e253e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:43:29.172 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:29.167+0000 7fea9bfe9740 1 --2- 172.21.15.83:0/3634765450 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55729e24be10 0x55729e253e50 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7fea80033000 tx=0x7fea80005b50 comp rx=0 tx=0).stop 2024-08-19T01:43:29.172 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:29.167+0000 7fea9bfe9740 1 -- 172.21.15.83:0/3634765450 shutdown_connections 2024-08-19T01:43:29.172 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:29.167+0000 7fea9bfe9740 1 --2- 172.21.15.83:0/3634765450 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55729e2566d0 0x55729e261d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:43:29.172 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:29.167+0000 7fea9bfe9740 1 --2- 172.21.15.83:0/3634765450 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55729e24be10 0x55729e253e50 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:43:29.172 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:29.167+0000 7fea9bfe9740 1 --2- 172.21.15.83:0/3634765450 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55729e24b4f0 0x55729e247e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:43:29.172 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:29.167+0000 7fea9bfe9740 1 -- 172.21.15.83:0/3634765450 >> 172.21.15.83:0/3634765450 conn(0x55729e247a80 msgr2=0x55729e261670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:43:29.172 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:29.167+0000 7fea9bfe9740 1 -- 172.21.15.83:0/3634765450 shutdown_connections 2024-08-19T01:43:29.172 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:29.167+0000 7fea9bfe9740 1 -- 172.21.15.83:0/3634765450 wait complete. 2024-08-19T01:43:29.172 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:29.167+0000 7fea9bfe9740 1 -- start start 2024-08-19T01:43:29.172 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:29.167+0000 7fea9bfe9740 1 Processor -- start 2024-08-19T01:43:29.172 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:29.168+0000 7fea9bfe9740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55729e24b4f0 0x55729e2683d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:43:29.172 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:29.168+0000 7fea9bfe9740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55729e24be10 0x55729e268910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:43:29.172 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:29.168+0000 7fea9bfe9740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55729e2566d0 0x55729e2ceba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:43:29.173 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:29.168+0000 7fea9bfe9740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fea941b2070 con 0x55729e2566d0 2024-08-19T01:43:29.173 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:29.168+0000 7fea9bfe9740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fea941b34e0 con 0x55729e24be10 2024-08-19T01:43:29.173 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:29.168+0000 7fea9bfe9740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fea941b2740 con 0x55729e24b4f0 2024-08-19T01:43:29.173 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:29.168+0000 7fea9bfe9740 10 client.0 _ll_register_callbacks cb 0x7fea840048f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-19T01:43:29.173 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:29.168+0000 7fea92800640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55729e24b4f0 0x55729e2683d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:43:29.173 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:29.168+0000 7fea93200640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55729e2566d0 0x55729e2ceba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:43:29.173 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:29.168+0000 7fea92800640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55729e24b4f0 0x55729e2683d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.83:51338/0 (socket says 172.21.15.83:51338) 2024-08-19T01:43:29.173 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:29.168+0000 7fea92800640 1 -- 172.21.15.83:0/1897003466 learned_addr learned my addr 172.21.15.83:0/1897003466 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-08-19T01:43:29.173 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:29.168+0000 7fea91e00640 1 --2- 172.21.15.83:0/1897003466 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55729e24be10 0x55729e268910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:43:29.173 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:29.169+0000 7fea92800640 1 -- 172.21.15.83:0/1897003466 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55729e24be10 msgr2=0x55729e268910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:43:29.173 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:29.169+0000 7fea92800640 1 --2- 172.21.15.83:0/1897003466 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55729e24be10 0x55729e268910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:43:29.173 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:29.169+0000 7fea92800640 1 -- 172.21.15.83:0/1897003466 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55729e2566d0 msgr2=0x55729e2ceba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:43:29.173 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:29.169+0000 7fea92800640 1 --2- 172.21.15.83:0/1897003466 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55729e2566d0 0x55729e2ceba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:43:29.173 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:29.169+0000 7fea92800640 1 -- 172.21.15.83:0/1897003466 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55729e2cf0e0 con 0x55729e24b4f0 2024-08-19T01:43:29.174 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:29.169+0000 7fea92800640 1 --2- 172.21.15.83:0/1897003466 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55729e24b4f0 0x55729e2683d0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fea8401f000 tx=0x7fea840cd290 comp rx=0 tx=0).ready entity=mon.1 client_cookie=dbd407878b27f5a5 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:43:29.174 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:29.169+0000 7fea73e00640 10 client.0 ms_handle_connect on v2:172.21.15.167:3300/0 2024-08-19T01:43:29.174 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:29.169+0000 7fea73e00640 1 -- 172.21.15.83:0/1897003466 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea84009820 con 0x55729e24b4f0 2024-08-19T01:43:29.174 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:29.169+0000 7fea73e00640 1 -- 172.21.15.83:0/1897003466 <== mon.1 v2:172.21.15.167:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8269+0+0 (secure 0 0 0) 0x7fea8409b380 con 0x55729e24b4f0 2024-08-19T01:43:29.174 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:29.169+0000 7fea9bfe9740 1 -- 172.21.15.83:0/1897003466 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fea8004b070 con 0x55729e24b4f0 2024-08-19T01:43:29.174 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:29.169+0000 7fea73e00640 1 -- 172.21.15.83:0/1897003466 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7fea8409c5e0 con 0x55729e24b4f0 2024-08-19T01:43:29.174 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:29.169+0000 7fea9bfe9740 10 client.26263 fetch_fsmap learned FSMap version 31 2024-08-19T01:43:29.174 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:29.169+0000 7fea9bfe9740 1 -- 172.21.15.83:0/1897003466 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55729e2d01a0 con 0x55729e24b4f0 2024-08-19T01:43:29.174 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:29.170+0000 7fea73e00640 1 -- 172.21.15.83:0/1897003466 <== mon.1 v2:172.21.15.167:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fea8004b070 con 0x55729e24b4f0 2024-08-19T01:43:29.174 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:29.170+0000 7fea73e00640 1 -- 172.21.15.83:0/1897003466 <== mon.1 v2:172.21.15.167:3300/0 5 ==== fsmap(e 31) ==== 8354+0+0 (secure 0 0 0) 0x7fea84097d50 con 0x55729e24b4f0 2024-08-19T01:43:29.174 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:29.170+0000 7fea9bfe9740 10 client.26263 fetch_fsmap finished waiting for FSMap version 31 2024-08-19T01:43:29.174 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:29.170+0000 7fea9bfe9740 10 client.26263 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-19T01:43:29.175 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:29.170+0000 7fea9bfe9740 20 client.26263 populate_metadata read hostname 'smithi083' 2024-08-19T01:43:29.175 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:29.170+0000 7fea9bfe9740 1 --2- 172.21.15.83:0/1897003466 >> [v2:172.21.15.117:6838/1967595704,v1:172.21.15.117:6839/1967595704] conn(0x55729e2d06d0 0x55729e2d2ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:43:29.175 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:29.170+0000 7fea9bfe9740 4 client.26263 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-08-19T01:43:29.175 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:29.170+0000 7fea9bfe9740 1 -- 172.21.15.83:0/1897003466 --> [v2:172.21.15.117:6838/1967595704,v1:172.21.15.117:6839/1967595704] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55729e1b8670 con 0x55729e2d06d0 2024-08-19T01:43:29.176 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:29.170+0000 7fea91e00640 1 --2- 172.21.15.83:0/1897003466 >> [v2:172.21.15.117:6838/1967595704,v1:172.21.15.117:6839/1967595704] conn(0x55729e2d06d0 0x55729e2d2ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:43:29.176 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:29.171+0000 7fea91e00640 1 --2- 172.21.15.83:0/1897003466 >> [v2:172.21.15.117:6838/1967595704,v1:172.21.15.117:6839/1967595704] conn(0x55729e2d06d0 0x55729e2d2ac0 crc :-1 s=READY pgs=150 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ff3a56aba13ae9a8 server_cookie=d911b6c94fab61e5 in_seq=0 out_seq=0 2024-08-19T01:43:29.176 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:29.171+0000 7fea73e00640 10 client.26263 ms_handle_connect on v2:172.21.15.117:6838/1967595704 2024-08-19T01:43:29.176 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:29.171+0000 7fea73e00640 1 -- 172.21.15.83:0/1897003466 <== mds.0 v2:172.21.15.117:6838/1967595704 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55729e1b8670 con 0x55729e2d06d0 2024-08-19T01:43:29.176 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:29.171+0000 7fea73e00640 10 client.26263 handle_command_reply: tid=0 2024-08-19T01:43:29.176 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:29.171+0000 7fea9bfe9740 1 client.26263 shutdown 2024-08-19T01:43:29.177 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:29.171+0000 7fea9bfe9740 1 -- 172.21.15.83:0/1897003466 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55729e24b4f0 msgr2=0x55729e2683d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:43:29.177 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:29.171+0000 7fea9bfe9740 1 --2- 172.21.15.83:0/1897003466 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55729e24b4f0 0x55729e2683d0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fea8401f000 tx=0x7fea840cd290 comp rx=0 tx=0).stop 2024-08-19T01:43:29.177 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:29.171+0000 7fea9bfe9740 1 -- 172.21.15.83:0/1897003466 shutdown_connections 2024-08-19T01:43:29.177 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:29.171+0000 7fea9bfe9740 1 --2- 172.21.15.83:0/1897003466 >> [v2:172.21.15.117:6838/1967595704,v1:172.21.15.117:6839/1967595704] conn(0x55729e2d06d0 0x55729e2d2ac0 crc :-1 s=READY pgs=150 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:43:29.177 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:29.171+0000 7fea9bfe9740 1 --2- 172.21.15.83:0/1897003466 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55729e2566d0 0x55729e2ceba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:43:29.177 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:29.171+0000 7fea9bfe9740 1 --2- 172.21.15.83:0/1897003466 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55729e24be10 0x55729e268910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:43:29.177 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:29.171+0000 7fea9bfe9740 1 --2- 172.21.15.83:0/1897003466 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55729e24b4f0 0x55729e2683d0 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:43:29.177 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:29.171+0000 7fea73e00640 0 client.26263 ms_handle_reset on v2:172.21.15.117:6838/1967595704 2024-08-19T01:43:29.177 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:29.171+0000 7fea9bfe9740 1 -- 172.21.15.83:0/1897003466 >> 172.21.15.83:0/1897003466 conn(0x55729e247a80 msgr2=0x55729e246c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:43:29.177 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:29.171+0000 7fea9bfe9740 1 -- 172.21.15.83:0/1897003466 shutdown_connections 2024-08-19T01:43:29.177 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:29.171+0000 7fea9bfe9740 1 -- 172.21.15.83:0/1897003466 wait complete. 2024-08-19T01:43:29.177 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:29.172+0000 7fea9bfe9740 20 client.26263 trim_cache size 0 max 16384 2024-08-19T01:43:29.178 INFO:teuthology.orchestra.run.smithi083.stdout:[] 2024-08-19T01:43:29.178 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:29.172+0000 7fea99400640 1 -- 172.21.15.83:0/2822225432 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7fea780a2c00 msgr2=0x7fea780a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:43:29.178 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:29.172+0000 7fea99400640 1 --2- 172.21.15.83:0/2822225432 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7fea780a2c00 0x7fea780a50c0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7fea8c0045e0 tx=0x7fea8c0046c0 comp rx=0 tx=0).stop 2024-08-19T01:43:29.178 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:29.172+0000 7fea99400640 1 -- 172.21.15.83:0/2822225432 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fea940742c0 msgr2=0x7fea941a4440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:43:29.179 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:29.172+0000 7fea99400640 1 --2- 172.21.15.83:0/2822225432 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fea940742c0 0x7fea941a4440 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7fea84009d50 tx=0x7fea8400c6a0 comp rx=0 tx=0).stop 2024-08-19T01:43:29.179 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:29.172+0000 7fea99400640 1 -- 172.21.15.83:0/2822225432 shutdown_connections 2024-08-19T01:43:29.179 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:29.172+0000 7fea99400640 1 --2- 172.21.15.83:0/2822225432 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7fea780a2c00 0x7fea780a50c0 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:43:29.179 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:29.172+0000 7fea99400640 1 --2- 172.21.15.83:0/2822225432 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fea941128d0 0x7fea9419e600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:43:29.179 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:29.172+0000 7fea99400640 1 --2- 172.21.15.83:0/2822225432 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fea94074be0 0x7fea941a4980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:43:29.179 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:29.172+0000 7fea99400640 1 --2- 172.21.15.83:0/2822225432 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fea940742c0 0x7fea941a4440 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:43:29.179 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:29.172+0000 7fea99400640 1 -- 172.21.15.83:0/2822225432 >> 172.21.15.83:0/2822225432 conn(0x7fea940fd5e0 msgr2=0x7fea940fed30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:43:29.179 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:29.172+0000 7fea99400640 1 -- 172.21.15.83:0/2822225432 shutdown_connections 2024-08-19T01:43:29.179 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:29.172+0000 7fea99400640 1 -- 172.21.15.83:0/2822225432 wait complete. 2024-08-19T01:43:29.904 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:43:29 smithi083 ceph-mon[17120]: pgmap v598: 97 pgs: 97 active+clean; 35 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 255 B/s wr, 6 op/s 2024-08-19T01:43:29.952 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:43:29 smithi117 ceph-mon[22016]: pgmap v598: 97 pgs: 97 active+clean; 35 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 255 B/s wr, 6 op/s 2024-08-19T01:43:29.966 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:43:29 smithi167 ceph-mon[21875]: pgmap v598: 97 pgs: 97 active+clean; 35 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 255 B/s wr, 6 op/s 2024-08-19T01:43:30.053 ERROR:tasks.fwd_scrub.fs.[cephfs]:exception: Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_ceph_78215f5a1cf5eaf321d506a66539942e12851f34/qa/tasks/fwd_scrub.py", line 35, in _run self.do_scrub() File "/home/teuthworker/src/git.ceph.com_ceph_78215f5a1cf5eaf321d506a66539942e12851f34/qa/tasks/fwd_scrub.py", line 49, in do_scrub self.sleep_unless_stopped(self.sleep_between_iterations) File "/home/teuthworker/src/git.ceph.com_ceph_78215f5a1cf5eaf321d506a66539942e12851f34/qa/tasks/thrasher.py", line 47, in sleep_unless_stopped raise self.Stopped() tasks.thrasher.ThrasherGreenlet.Stopped 2024-08-19T01:43:30.054 INFO:tasks.fwd_scrub:done joining 2024-08-19T01:43:30.054 DEBUG:teuthology.run_tasks:Unwinding manager check-counter 2024-08-19T01:43:30.065 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-08-19T01:43:30.221 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:30.219+0000 7efee4e00640 1 -- 172.21.15.83:0/2778589231 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7efee0104a90 msgr2=0x7efee0104f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:43:30.221 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:30.219+0000 7efee4e00640 1 --2- 172.21.15.83:0/2778589231 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7efee0104a90 0x7efee0104f10 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7efecc009900 tx=0x7efecc02a410 comp rx=0 tx=0).stop 2024-08-19T01:43:30.221 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:30.219+0000 7efee4e00640 1 -- 172.21.15.83:0/2778589231 shutdown_connections 2024-08-19T01:43:30.221 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:30.219+0000 7efee4e00640 1 --2- 172.21.15.83:0/2778589231 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7efee0105450 0x7efee010bce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:43:30.221 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:30.219+0000 7efee4e00640 1 --2- 172.21.15.83:0/2778589231 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7efee0104a90 0x7efee0104f10 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:43:30.221 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:30.219+0000 7efee4e00640 1 --2- 172.21.15.83:0/2778589231 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7efee01038a0 0x7efee0103ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:43:30.221 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:30.219+0000 7efee4e00640 1 -- 172.21.15.83:0/2778589231 >> 172.21.15.83:0/2778589231 conn(0x7efee00ff020 msgr2=0x7efee0101440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:43:30.221 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:30.219+0000 7efee4e00640 1 -- 172.21.15.83:0/2778589231 shutdown_connections 2024-08-19T01:43:30.221 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:30.219+0000 7efee4e00640 1 -- 172.21.15.83:0/2778589231 wait complete. 2024-08-19T01:43:30.223 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:30.220+0000 7efee4e00640 1 Processor -- start 2024-08-19T01:43:30.223 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:30.220+0000 7efee4e00640 1 -- start start 2024-08-19T01:43:30.224 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:30.220+0000 7efee4e00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7efee01038a0 0x7efee0199bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:43:30.224 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:30.220+0000 7efee4e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7efee0104a90 0x7efee019a130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:43:30.224 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:30.220+0000 7efee4e00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7efee0105450 0x7efee01a11a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:43:30.224 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:30.220+0000 7efee4e00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7efee010f570 con 0x7efee0104a90 2024-08-19T01:43:30.224 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:30.220+0000 7efee4e00640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7efee010f3f0 con 0x7efee0105450 2024-08-19T01:43:30.224 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:30.220+0000 7efee4e00640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7efee010f6f0 con 0x7efee01038a0 2024-08-19T01:43:30.224 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:30.221+0000 7efedea00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7efee0104a90 0x7efee019a130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:43:30.224 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:30.221+0000 7efedea00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7efee0104a90 0x7efee019a130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:50968/0 (socket says 172.21.15.83:50968) 2024-08-19T01:43:30.224 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:30.221+0000 7efedea00640 1 -- 172.21.15.83:0/1725126825 learned_addr learned my addr 172.21.15.83:0/1725126825 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-08-19T01:43:30.224 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:30.221+0000 7efedfe00640 1 --2- 172.21.15.83:0/1725126825 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7efee0105450 0x7efee01a11a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:43:30.224 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:30.221+0000 7efedf400640 1 --2- 172.21.15.83:0/1725126825 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7efee01038a0 0x7efee0199bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:43:30.224 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:30.221+0000 7efedea00640 1 -- 172.21.15.83:0/1725126825 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7efee01038a0 msgr2=0x7efee0199bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:43:30.224 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:30.221+0000 7efedea00640 1 --2- 172.21.15.83:0/1725126825 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7efee01038a0 0x7efee0199bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:43:30.224 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:30.221+0000 7efedea00640 1 -- 172.21.15.83:0/1725126825 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7efee0105450 msgr2=0x7efee01a11a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:43:30.224 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:30.221+0000 7efedea00640 1 --2- 172.21.15.83:0/1725126825 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7efee0105450 0x7efee01a11a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:43:30.225 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:30.221+0000 7efedea00640 1 -- 172.21.15.83:0/1725126825 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efee01a1810 con 0x7efee0104a90 2024-08-19T01:43:30.225 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:30.221+0000 7efedea00640 1 --2- 172.21.15.83:0/1725126825 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7efee0104a90 0x7efee019a130 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7efecc031440 tx=0x7efecc032ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:43:30.225 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:30.222+0000 7efed7e00640 1 -- 172.21.15.83:0/1725126825 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efecc03f030 con 0x7efee0104a90 2024-08-19T01:43:30.225 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:30.222+0000 7efed7e00640 1 -- 172.21.15.83:0/1725126825 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efecc003ce0 con 0x7efee0104a90 2024-08-19T01:43:30.225 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:30.222+0000 7efee4e00640 1 -- 172.21.15.83:0/1725126825 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efee01a1aa0 con 0x7efee0104a90 2024-08-19T01:43:30.225 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:30.222+0000 7efed7e00640 1 -- 172.21.15.83:0/1725126825 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efecc035070 con 0x7efee0104a90 2024-08-19T01:43:30.225 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:30.222+0000 7efee4e00640 1 -- 172.21.15.83:0/1725126825 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efee01a1f90 con 0x7efee0104a90 2024-08-19T01:43:30.225 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:30.223+0000 7efee4e00640 1 -- 172.21.15.83:0/1725126825 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efee010ac60 con 0x7efee0104a90 2024-08-19T01:43:30.226 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:30.225+0000 7efed7e00640 1 -- 172.21.15.83:0/1725126825 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7efecc042030 con 0x7efee0104a90 2024-08-19T01:43:30.227 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:30.226+0000 7efed7e00640 1 --2- 172.21.15.83:0/1725126825 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7efec40a2b90 0x7efec40a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:43:30.227 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:30.226+0000 7efed7e00640 1 -- 172.21.15.83:0/1725126825 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8269+0+0 (secure 0 0 0) 0x7efecc0ef980 con 0x7efee0104a90 2024-08-19T01:43:30.227 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:30.226+0000 7efedf400640 1 --2- 172.21.15.83:0/1725126825 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7efec40a2b90 0x7efec40a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:43:30.228 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:30.227+0000 7efedf400640 1 --2- 172.21.15.83:0/1725126825 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7efec40a2b90 0x7efec40a5050 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7efee01048f0 tx=0x7efed0005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:43:30.230 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:30.228+0000 7efed7e00640 1 -- 172.21.15.83:0/1725126825 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7efecc0bc210 con 0x7efee0104a90 2024-08-19T01:43:30.406 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:30.404+0000 7efee4e00640 1 -- 172.21.15.83:0/1725126825 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7efee002d830 con 0x7efee0104a90 2024-08-19T01:43:30.407 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:30.406+0000 7efed7e00640 1 -- 172.21.15.83:0/1725126825 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v86) ==== 74+0+22722 (secure 0 0 0) 0x7efecc0c0510 con 0x7efee0104a90 2024-08-19T01:43:30.408 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-08-19T01:43:30.408 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":86,"fsid":"bad8780e-5dc9-11ef-bccf-c7b262605968","created":"2024-08-19T01:25:12.626841+0000","modified":"2024-08-19T01:33:05.461998+0000","last_up_change":"2024-08-19T01:31:47.254559+0000","last_in_change":"2024-08-19T01:31:27.652847+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-08-19T01:28:16.862007+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-08-19T01:32:37.455535+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-08-19T01:32:40.201935+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ffa74041-f9d2-4555-bb43-508b8c581487","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6802","nonce":1082526824},{"type":"v1","addr":"172.21.15.83:6803","nonce":1082526824}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6804","nonce":1082526824},{"type":"v1","addr":"172.21.15.83:6805","nonce":1082526824}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6808","nonce":1082526824},{"type":"v1","addr":"172.21.15.83:6809","nonce":1082526824}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6806","nonce":1082526824},{"type":"v1","addr":"172.21.15.83:6807","nonce":1082526824}]},"public_addr":"172.21.15.83:6803/1082526824","cluster_addr":"172.21.15.83:6805/1082526824","heartbeat_back_addr":"172.21.15.83:6809/1082526824","heartbeat_front_addr":"172.21.15.83:6807/1082526824","state":["exists","up"]},{"osd":1,"uuid":"1977570b-b4f5-44fd-bb1e-3a31318be649","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6800","nonce":1925419802},{"type":"v1","addr":"172.21.15.117:6801","nonce":1925419802}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6802","nonce":1925419802},{"type":"v1","addr":"172.21.15.117:6803","nonce":1925419802}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6806","nonce":1925419802},{"type":"v1","addr":"172.21.15.117:6807","nonce":1925419802}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6804","nonce":1925419802},{"type":"v1","addr":"172.21.15.117:6805","nonce":1925419802}]},"public_addr":"172.21.15.117:6801/1925419802","cluster_addr":"172.21.15.117:6803/1925419802","heartbeat_back_addr":"172.21.15.117:6807/1925419802","heartbeat_front_addr":"172.21.15.117:6805/1925419802","state":["exists","up"]},{"osd":2,"uuid":"a75278d1-161f-41d5-a92f-058d1077bf2a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6800","nonce":551625328},{"type":"v1","addr":"172.21.15.167:6801","nonce":551625328}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6802","nonce":551625328},{"type":"v1","addr":"172.21.15.167:6803","nonce":551625328}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6806","nonce":551625328},{"type":"v1","addr":"172.21.15.167:6807","nonce":551625328}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6804","nonce":551625328},{"type":"v1","addr":"172.21.15.167:6805","nonce":551625328}]},"public_addr":"172.21.15.167:6801/551625328","cluster_addr":"172.21.15.167:6803/551625328","heartbeat_back_addr":"172.21.15.167:6807/551625328","heartbeat_front_addr":"172.21.15.167:6805/551625328","state":["exists","up"]},{"osd":3,"uuid":"d1fb16f9-910e-46fa-b1e3-8e5bef4d346d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6810","nonce":3138942062},{"type":"v1","addr":"172.21.15.83:6811","nonce":3138942062}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6812","nonce":3138942062},{"type":"v1","addr":"172.21.15.83:6813","nonce":3138942062}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6816","nonce":3138942062},{"type":"v1","addr":"172.21.15.83:6817","nonce":3138942062}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6814","nonce":3138942062},{"type":"v1","addr":"172.21.15.83:6815","nonce":3138942062}]},"public_addr":"172.21.15.83:6811/3138942062","cluster_addr":"172.21.15.83:6813/3138942062","heartbeat_back_addr":"172.21.15.83:6817/3138942062","heartbeat_front_addr":"172.21.15.83:6815/3138942062","state":["exists","up"]},{"osd":4,"uuid":"8176ed18-1fe2-446d-8994-8c60f6e08f38","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6808","nonce":3978863827},{"type":"v1","addr":"172.21.15.117:6809","nonce":3978863827}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6810","nonce":3978863827},{"type":"v1","addr":"172.21.15.117:6811","nonce":3978863827}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6814","nonce":3978863827},{"type":"v1","addr":"172.21.15.117:6815","nonce":3978863827}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6812","nonce":3978863827},{"type":"v1","addr":"172.21.15.117:6813","nonce":3978863827}]},"public_addr":"172.21.15.117:6809/3978863827","cluster_addr":"172.21.15.117:6811/3978863827","heartbeat_back_addr":"172.21.15.117:6815/3978863827","heartbeat_front_addr":"172.21.15.117:6813/3978863827","state":["exists","up"]},{"osd":5,"uuid":"e01e3f15-25c8-4d2c-be6f-a170f036064c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6808","nonce":2761858035},{"type":"v1","addr":"172.21.15.167:6809","nonce":2761858035}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6810","nonce":2761858035},{"type":"v1","addr":"172.21.15.167:6811","nonce":2761858035}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6814","nonce":2761858035},{"type":"v1","addr":"172.21.15.167:6815","nonce":2761858035}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6812","nonce":2761858035},{"type":"v1","addr":"172.21.15.167:6813","nonce":2761858035}]},"public_addr":"172.21.15.167:6809/2761858035","cluster_addr":"172.21.15.167:6811/2761858035","heartbeat_back_addr":"172.21.15.167:6815/2761858035","heartbeat_front_addr":"172.21.15.167:6813/2761858035","state":["exists","up"]},{"osd":6,"uuid":"78cf0a9b-9165-444f-b073-ebe13016de9d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6818","nonce":260644552},{"type":"v1","addr":"172.21.15.83:6819","nonce":260644552}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6820","nonce":260644552},{"type":"v1","addr":"172.21.15.83:6821","nonce":260644552}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6824","nonce":260644552},{"type":"v1","addr":"172.21.15.83:6825","nonce":260644552}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6822","nonce":260644552},{"type":"v1","addr":"172.21.15.83:6823","nonce":260644552}]},"public_addr":"172.21.15.83:6819/260644552","cluster_addr":"172.21.15.83:6821/260644552","heartbeat_back_addr":"172.21.15.83:6825/260644552","heartbeat_front_addr":"172.21.15.83:6823/260644552","state":["exists","up"]},{"osd":7,"uuid":"ee524f7f-672e-430e-9f58-1a9117513466","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6816","nonce":104113252},{"type":"v1","addr":"172.21.15.117:6817","nonce":104113252}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6818","nonce":104113252},{"type":"v1","addr":"172.21.15.117:6819","nonce":104113252}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6822","nonce":104113252},{"type":"v1","addr":"172.21.15.117:6823","nonce":104113252}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6820","nonce":104113252},{"type":"v1","addr":"172.21.15.117:6821","nonce":104113252}]},"public_addr":"172.21.15.117:6817/104113252","cluster_addr":"172.21.15.117:6819/104113252","heartbeat_back_addr":"172.21.15.117:6823/104113252","heartbeat_front_addr":"172.21.15.117:6821/104113252","state":["exists","up"]},{"osd":8,"uuid":"76d5317d-4845-4138-b210-e7bd2d80472a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":58,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6816","nonce":2196898857},{"type":"v1","addr":"172.21.15.167:6817","nonce":2196898857}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6818","nonce":2196898857},{"type":"v1","addr":"172.21.15.167:6819","nonce":2196898857}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6822","nonce":2196898857},{"type":"v1","addr":"172.21.15.167:6823","nonce":2196898857}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6820","nonce":2196898857},{"type":"v1","addr":"172.21.15.167:6821","nonce":2196898857}]},"public_addr":"172.21.15.167:6817/2196898857","cluster_addr":"172.21.15.167:6819/2196898857","heartbeat_back_addr":"172.21.15.167:6823/2196898857","heartbeat_front_addr":"172.21.15.167:6821/2196898857","state":["exists","up"]},{"osd":9,"uuid":"0151bc2a-3ad2-4db3-8505-5855792cc9bb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6826","nonce":903463450},{"type":"v1","addr":"172.21.15.83:6827","nonce":903463450}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6828","nonce":903463450},{"type":"v1","addr":"172.21.15.83:6829","nonce":903463450}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6832","nonce":903463450},{"type":"v1","addr":"172.21.15.83:6833","nonce":903463450}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6830","nonce":903463450},{"type":"v1","addr":"172.21.15.83:6831","nonce":903463450}]},"public_addr":"172.21.15.83:6827/903463450","cluster_addr":"172.21.15.83:6829/903463450","heartbeat_back_addr":"172.21.15.83:6833/903463450","heartbeat_front_addr":"172.21.15.83:6831/903463450","state":["exists","up"]},{"osd":10,"uuid":"437ed848-0da4-4cd4-8716-0c55bb99b676","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6824","nonce":1183570837},{"type":"v1","addr":"172.21.15.117:6825","nonce":1183570837}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6826","nonce":1183570837},{"type":"v1","addr":"172.21.15.117:6827","nonce":1183570837}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6830","nonce":1183570837},{"type":"v1","addr":"172.21.15.117:6831","nonce":1183570837}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6828","nonce":1183570837},{"type":"v1","addr":"172.21.15.117:6829","nonce":1183570837}]},"public_addr":"172.21.15.117:6825/1183570837","cluster_addr":"172.21.15.117:6827/1183570837","heartbeat_back_addr":"172.21.15.117:6831/1183570837","heartbeat_front_addr":"172.21.15.117:6829/1183570837","state":["exists","up"]},{"osd":11,"uuid":"13345f05-cdc0-4dae-a58b-551abbec1915","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":78,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6824","nonce":187027350},{"type":"v1","addr":"172.21.15.167:6825","nonce":187027350}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6826","nonce":187027350},{"type":"v1","addr":"172.21.15.167:6827","nonce":187027350}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6830","nonce":187027350},{"type":"v1","addr":"172.21.15.167:6831","nonce":187027350}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6828","nonce":187027350},{"type":"v1","addr":"172.21.15.167:6829","nonce":187027350}]},"public_addr":"172.21.15.167:6825/187027350","cluster_addr":"172.21.15.167:6827/187027350","heartbeat_back_addr":"172.21.15.167:6831/187027350","heartbeat_front_addr":"172.21.15.167:6829/187027350","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-19T01:27:28.760517+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-19T01:27:50.493028+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-19T01:28:11.997674+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-19T01:28:38.109934+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-19T01:29:00.173065+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-19T01:29:21.336085+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-19T01:29:47.871498+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-19T01:30:10.720498+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-19T01:30:32.425438+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-19T01:30:59.524989+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-19T01:31:22.806162+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-19T01:31:45.043246+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.1a","mappings":[{"from":8,"to":1}]},{"pgid":"2.1b","mappings":[{"from":2,"to":4}]},{"pgid":"2.1d","mappings":[{"from":8,"to":11}]},{"pgid":"3.6","mappings":[{"from":4,"to":6}]},{"pgid":"3.11","mappings":[{"from":4,"to":8}]},{"pgid":"3.1b","mappings":[{"from":5,"to":9}]},{"pgid":"3.20","mappings":[{"from":5,"to":9}]},{"pgid":"3.31","mappings":[{"from":4,"to":7}]},{"pgid":"3.3b","mappings":[{"from":3,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.83:0/3138095466":"2024-08-20T01:26:04.277716+0000","172.21.15.83:0/3835005198":"2024-08-20T01:26:04.277716+0000","172.21.15.83:0/1790374088":"2024-08-20T01:25:37.228583+0000","172.21.15.83:6801/573907319":"2024-08-20T01:26:04.277716+0000","172.21.15.83:6800/573907319":"2024-08-20T01:26:04.277716+0000","172.21.15.83:0/664676002":"2024-08-20T01:25:37.228583+0000","172.21.15.83:0/2144611449":"2024-08-20T01:25:37.228583+0000","172.21.15.83:6801/3261152818":"2024-08-20T01:25:37.228583+0000","172.21.15.83:0/1609552666":"2024-08-20T01:26:04.277716+0000","172.21.15.83:6800/3261152818":"2024-08-20T01:25:37.228583+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-08-19T01:43:30.410 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:30.408+0000 7efee4e00640 1 -- 172.21.15.83:0/1725126825 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7efec40a2b90 msgr2=0x7efec40a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:43:30.410 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:30.408+0000 7efee4e00640 1 --2- 172.21.15.83:0/1725126825 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7efec40a2b90 0x7efec40a5050 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7efee01048f0 tx=0x7efed0005f90 comp rx=0 tx=0).stop 2024-08-19T01:43:30.410 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:30.408+0000 7efee4e00640 1 -- 172.21.15.83:0/1725126825 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7efee0104a90 msgr2=0x7efee019a130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:43:30.410 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:30.408+0000 7efee4e00640 1 --2- 172.21.15.83:0/1725126825 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7efee0104a90 0x7efee019a130 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7efecc031440 tx=0x7efecc032ce0 comp rx=0 tx=0).stop 2024-08-19T01:43:30.410 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:30.408+0000 7efee4e00640 1 -- 172.21.15.83:0/1725126825 shutdown_connections 2024-08-19T01:43:30.410 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:30.408+0000 7efee4e00640 1 --2- 172.21.15.83:0/1725126825 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7efec40a2b90 0x7efec40a5050 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:43:30.410 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:30.408+0000 7efee4e00640 1 --2- 172.21.15.83:0/1725126825 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7efee0105450 0x7efee01a11a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:43:30.410 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:30.408+0000 7efee4e00640 1 --2- 172.21.15.83:0/1725126825 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7efee0104a90 0x7efee019a130 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:43:30.411 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:30.408+0000 7efee4e00640 1 --2- 172.21.15.83:0/1725126825 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7efee01038a0 0x7efee0199bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:43:30.411 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:30.408+0000 7efee4e00640 1 -- 172.21.15.83:0/1725126825 >> 172.21.15.83:0/1725126825 conn(0x7efee00ff020 msgr2=0x7efee0107e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:43:30.411 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:30.408+0000 7efee4e00640 1 -- 172.21.15.83:0/1725126825 shutdown_connections 2024-08-19T01:43:30.411 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:30.408+0000 7efee4e00640 1 -- 172.21.15.83:0/1725126825 wait complete. 2024-08-19T01:43:30.426 INFO:tasks.check_counter.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-08-19T01:28:16.862007+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-08-19T01:32:37.455535+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '84', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-08-19T01:32:40.201935+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '84', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-08-19T01:43:30.426 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-08-19T01:43:30.585 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:30.583+0000 7f1d48c00640 1 -- 172.21.15.83:0/2167701407 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1d44077b20 msgr2=0x7f1d44077f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:43:30.585 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:30.583+0000 7f1d48c00640 1 --2- 172.21.15.83:0/2167701407 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1d44077b20 0x7f1d44077f80 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f1d30009900 tx=0x7f1d3002a410 comp rx=0 tx=0).stop 2024-08-19T01:43:30.585 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:30.583+0000 7f1d48c00640 1 -- 172.21.15.83:0/2167701407 shutdown_connections 2024-08-19T01:43:30.585 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:30.583+0000 7f1d48c00640 1 --2- 172.21.15.83:0/2167701407 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1d440784c0 0x7f1d44078d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:43:30.586 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:30.583+0000 7f1d48c00640 1 --2- 172.21.15.83:0/2167701407 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1d44077b20 0x7f1d44077f80 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:43:30.586 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:30.583+0000 7f1d48c00640 1 --2- 172.21.15.83:0/2167701407 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1d440768e0 0x7f1d44076ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:43:30.586 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:30.583+0000 7f1d48c00640 1 -- 172.21.15.83:0/2167701407 >> 172.21.15.83:0/2167701407 conn(0x7f1d440ff1d0 msgr2=0x7f1d441015f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:43:30.586 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:30.584+0000 7f1d48c00640 1 -- 172.21.15.83:0/2167701407 shutdown_connections 2024-08-19T01:43:30.586 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:30.584+0000 7f1d48c00640 1 -- 172.21.15.83:0/2167701407 wait complete. 2024-08-19T01:43:30.588 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:30.584+0000 7f1d48c00640 1 Processor -- start 2024-08-19T01:43:30.588 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:30.585+0000 7f1d48c00640 1 -- start start 2024-08-19T01:43:30.588 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:30.585+0000 7f1d48c00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1d440768e0 0x7f1d4419e1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:43:30.588 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:30.585+0000 7f1d48c00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1d44077b20 0x7f1d4419e700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:43:30.588 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:30.585+0000 7f1d48c00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1d440784c0 0x7f1d441a5770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:43:30.588 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:30.585+0000 7f1d48c00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f1d44113b30 con 0x7f1d440768e0 2024-08-19T01:43:30.588 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:30.585+0000 7f1d48c00640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f1d441139b0 con 0x7f1d44077b20 2024-08-19T01:43:30.588 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:30.585+0000 7f1d48c00640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f1d44113cb0 con 0x7f1d440784c0 2024-08-19T01:43:30.588 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:30.585+0000 7f1d41e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1d440768e0 0x7f1d4419e1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:43:30.589 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:30.585+0000 7f1d41e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1d440768e0 0x7f1d4419e1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:50990/0 (socket says 172.21.15.83:50990) 2024-08-19T01:43:30.589 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:30.585+0000 7f1d41e00640 1 -- 172.21.15.83:0/3275406329 learned_addr learned my addr 172.21.15.83:0/3275406329 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-08-19T01:43:30.589 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:30.585+0000 7f1d42800640 1 --2- 172.21.15.83:0/3275406329 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1d440784c0 0x7f1d441a5770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:43:30.589 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:30.585+0000 7f1d41400640 1 --2- 172.21.15.83:0/3275406329 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1d44077b20 0x7f1d4419e700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:43:30.589 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:30.586+0000 7f1d41e00640 1 -- 172.21.15.83:0/3275406329 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1d440784c0 msgr2=0x7f1d441a5770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:43:30.589 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:30.586+0000 7f1d41e00640 1 --2- 172.21.15.83:0/3275406329 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1d440784c0 0x7f1d441a5770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:43:30.589 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:30.586+0000 7f1d41e00640 1 -- 172.21.15.83:0/3275406329 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1d44077b20 msgr2=0x7f1d4419e700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:43:30.589 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:30.586+0000 7f1d41e00640 1 --2- 172.21.15.83:0/3275406329 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1d44077b20 0x7f1d4419e700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:43:30.589 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:30.586+0000 7f1d41e00640 1 -- 172.21.15.83:0/3275406329 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1d441a5de0 con 0x7f1d440768e0 2024-08-19T01:43:30.589 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:30.586+0000 7f1d41400640 1 --2- 172.21.15.83:0/3275406329 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1d44077b20 0x7f1d4419e700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-19T01:43:30.589 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:30.586+0000 7f1d42800640 1 --2- 172.21.15.83:0/3275406329 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1d440784c0 0x7f1d441a5770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-19T01:43:30.589 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:30.586+0000 7f1d41e00640 1 --2- 172.21.15.83:0/3275406329 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1d440768e0 0x7f1d4419e1c0 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f1d3400e9f0 tx=0x7f1d3400edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:43:30.589 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:30.586+0000 7f1d3aa00640 1 -- 172.21.15.83:0/3275406329 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d3400cd80 con 0x7f1d440768e0 2024-08-19T01:43:30.589 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:30.586+0000 7f1d3aa00640 1 -- 172.21.15.83:0/3275406329 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1d34005500 con 0x7f1d440768e0 2024-08-19T01:43:30.590 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:30.586+0000 7f1d48c00640 1 -- 172.21.15.83:0/3275406329 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1d441a60d0 con 0x7f1d440768e0 2024-08-19T01:43:30.590 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:30.586+0000 7f1d3aa00640 1 -- 172.21.15.83:0/3275406329 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d34010690 con 0x7f1d440768e0 2024-08-19T01:43:30.590 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:30.586+0000 7f1d48c00640 1 -- 172.21.15.83:0/3275406329 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1d441a65f0 con 0x7f1d440768e0 2024-08-19T01:43:30.590 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:30.587+0000 7f1d48c00640 1 -- 172.21.15.83:0/3275406329 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1d441105a0 con 0x7f1d440768e0 2024-08-19T01:43:30.590 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:30.589+0000 7f1d3aa00640 1 -- 172.21.15.83:0/3275406329 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7f1d340046c0 con 0x7f1d440768e0 2024-08-19T01:43:30.592 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:30.591+0000 7f1d3aa00640 1 --2- 172.21.15.83:0/3275406329 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f1d280a2c10 0x7f1d280a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:43:30.593 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:30.592+0000 7f1d3aa00640 1 -- 172.21.15.83:0/3275406329 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8269+0+0 (secure 0 0 0) 0x7f1d340cf4f0 con 0x7f1d440768e0 2024-08-19T01:43:30.593 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:30.592+0000 7f1d41400640 1 --2- 172.21.15.83:0/3275406329 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f1d280a2c10 0x7f1d280a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:43:30.593 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:30.592+0000 7f1d41400640 1 --2- 172.21.15.83:0/3275406329 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f1d280a2c10 0x7f1d280a50d0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f1d30006010 tx=0x7f1d30000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:43:30.597 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:30.596+0000 7f1d3aa00640 1 -- 172.21.15.83:0/3275406329 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7f1d3409c3d0 con 0x7f1d440768e0 2024-08-19T01:43:30.774 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:30.773+0000 7f1d48c00640 1 -- 172.21.15.83:0/3275406329 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f1d44062b20 con 0x7f1d440768e0 2024-08-19T01:43:30.775 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:30.773+0000 7f1d3aa00640 1 -- 172.21.15.83:0/3275406329 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v86) ==== 93+0+10 (secure 0 0 0) 0x7f1d3409bcd0 con 0x7f1d440768e0 2024-08-19T01:43:30.775 INFO:teuthology.orchestra.run.smithi083.stdout:pg_num: 1 2024-08-19T01:43:30.777 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:30.775+0000 7f1d48c00640 1 -- 172.21.15.83:0/3275406329 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f1d280a2c10 msgr2=0x7f1d280a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:43:30.777 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:30.775+0000 7f1d48c00640 1 --2- 172.21.15.83:0/3275406329 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f1d280a2c10 0x7f1d280a50d0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f1d30006010 tx=0x7f1d30000bc0 comp rx=0 tx=0).stop 2024-08-19T01:43:30.777 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:30.775+0000 7f1d48c00640 1 -- 172.21.15.83:0/3275406329 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1d440768e0 msgr2=0x7f1d4419e1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:43:30.778 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:30.775+0000 7f1d48c00640 1 --2- 172.21.15.83:0/3275406329 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1d440768e0 0x7f1d4419e1c0 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f1d3400e9f0 tx=0x7f1d3400edf0 comp rx=0 tx=0).stop 2024-08-19T01:43:30.778 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:30.776+0000 7f1d48c00640 1 -- 172.21.15.83:0/3275406329 shutdown_connections 2024-08-19T01:43:30.778 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:30.776+0000 7f1d48c00640 1 --2- 172.21.15.83:0/3275406329 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f1d280a2c10 0x7f1d280a50d0 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:43:30.778 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:30.776+0000 7f1d48c00640 1 --2- 172.21.15.83:0/3275406329 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1d440784c0 0x7f1d441a5770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:43:30.778 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:30.776+0000 7f1d48c00640 1 --2- 172.21.15.83:0/3275406329 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1d44077b20 0x7f1d4419e700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:43:30.778 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:30.776+0000 7f1d48c00640 1 --2- 172.21.15.83:0/3275406329 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1d440768e0 0x7f1d4419e1c0 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:43:30.778 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:30.776+0000 7f1d48c00640 1 -- 172.21.15.83:0/3275406329 >> 172.21.15.83:0/3275406329 conn(0x7f1d440ff1d0 msgr2=0x7f1d44100970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:43:30.778 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:30.776+0000 7f1d48c00640 1 -- 172.21.15.83:0/3275406329 shutdown_connections 2024-08-19T01:43:30.778 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:30.776+0000 7f1d48c00640 1 -- 172.21.15.83:0/3275406329 wait complete. 2024-08-19T01:43:30.793 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-08-19T01:43:30.951 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:30.949+0000 7f7a48600640 1 -- 172.21.15.83:0/3517293194 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7a40105600 msgr2=0x7f7a4010be90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:43:30.952 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:30.949+0000 7f7a48600640 1 --2- 172.21.15.83:0/3517293194 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7a40105600 0x7f7a4010be90 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f7a38009ca0 tx=0x7f7a3802a8c0 comp rx=0 tx=0).stop 2024-08-19T01:43:30.952 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:30.950+0000 7f7a48600640 1 -- 172.21.15.83:0/3517293194 shutdown_connections 2024-08-19T01:43:30.952 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:30.950+0000 7f7a48600640 1 --2- 172.21.15.83:0/3517293194 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7a40105600 0x7f7a4010be90 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:43:30.952 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:30.950+0000 7f7a48600640 1 --2- 172.21.15.83:0/3517293194 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7a40104c40 0x7f7a401050c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:43:30.952 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:30.950+0000 7f7a48600640 1 --2- 172.21.15.83:0/3517293194 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7a40103a50 0x7f7a40103e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:43:30.952 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:30.950+0000 7f7a48600640 1 -- 172.21.15.83:0/3517293194 >> 172.21.15.83:0/3517293194 conn(0x7f7a400ff1d0 msgr2=0x7f7a401015f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:43:30.952 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:30.950+0000 7f7a48600640 1 -- 172.21.15.83:0/3517293194 shutdown_connections 2024-08-19T01:43:30.952 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:30.950+0000 7f7a48600640 1 -- 172.21.15.83:0/3517293194 wait complete. 2024-08-19T01:43:30.954 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:30.951+0000 7f7a48600640 1 Processor -- start 2024-08-19T01:43:30.954 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:30.951+0000 7f7a48600640 1 -- start start 2024-08-19T01:43:30.955 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:30.951+0000 7f7a48600640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7a40103a50 0x7f7a40199d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:43:30.955 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:30.951+0000 7f7a48600640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7a40104c40 0x7f7a4019a270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:43:30.955 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:30.951+0000 7f7a48600640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7a40105600 0x7f7a401a12a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:43:30.955 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:30.951+0000 7f7a48600640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f7a4010f720 con 0x7f7a40105600 2024-08-19T01:43:30.955 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:30.951+0000 7f7a48600640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f7a4010f5a0 con 0x7f7a40104c40 2024-08-19T01:43:30.955 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:30.951+0000 7f7a48600640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f7a4010f8a0 con 0x7f7a40103a50 2024-08-19T01:43:30.955 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:30.951+0000 7f7a46600640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7a40105600 0x7f7a401a12a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:43:30.955 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:30.951+0000 7f7a45c00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7a40103a50 0x7f7a40199d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:43:30.955 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:30.951+0000 7f7a46600640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7a40105600 0x7f7a401a12a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:34740/0 (socket says 172.21.15.83:34740) 2024-08-19T01:43:30.955 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:30.951+0000 7f7a46600640 1 -- 172.21.15.83:0/3951265864 learned_addr learned my addr 172.21.15.83:0/3951265864 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-08-19T01:43:30.955 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:30.951+0000 7f7a45200640 1 --2- 172.21.15.83:0/3951265864 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7a40104c40 0x7f7a4019a270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:43:30.955 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:30.952+0000 7f7a45c00640 1 -- 172.21.15.83:0/3951265864 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7a40104c40 msgr2=0x7f7a4019a270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:43:30.955 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:30.952+0000 7f7a45c00640 1 --2- 172.21.15.83:0/3951265864 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7a40104c40 0x7f7a4019a270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:43:30.956 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:30.952+0000 7f7a45c00640 1 -- 172.21.15.83:0/3951265864 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7a40105600 msgr2=0x7f7a401a12a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:43:30.956 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:30.952+0000 7f7a45c00640 1 --2- 172.21.15.83:0/3951265864 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7a40105600 0x7f7a401a12a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:43:30.956 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:30.952+0000 7f7a45c00640 1 -- 172.21.15.83:0/3951265864 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7a401a1940 con 0x7f7a40103a50 2024-08-19T01:43:30.956 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:30.952+0000 7f7a45c00640 1 --2- 172.21.15.83:0/3951265864 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7a40103a50 0x7f7a40199d30 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f7a3400e9c0 tx=0x7f7a3400ee80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:43:30.956 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:30.952+0000 7f7a3e000640 1 -- 172.21.15.83:0/3951265864 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a3400ccc0 con 0x7f7a40103a50 2024-08-19T01:43:30.956 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:30.952+0000 7f7a3e000640 1 -- 172.21.15.83:0/3951265864 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7a340035f0 con 0x7f7a40103a50 2024-08-19T01:43:30.956 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:30.952+0000 7f7a48600640 1 -- 172.21.15.83:0/3951265864 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7a401a1c30 con 0x7f7a40103a50 2024-08-19T01:43:30.956 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:30.952+0000 7f7a3e000640 1 -- 172.21.15.83:0/3951265864 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a3400ce60 con 0x7f7a40103a50 2024-08-19T01:43:30.956 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:30.952+0000 7f7a48600640 1 -- 172.21.15.83:0/3951265864 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7a401a2130 con 0x7f7a40103a50 2024-08-19T01:43:30.957 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:30.953+0000 7f7a48600640 1 -- 172.21.15.83:0/3951265864 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7a4010adf0 con 0x7f7a40103a50 2024-08-19T01:43:30.957 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:30.955+0000 7f7a3e000640 1 -- 172.21.15.83:0/3951265864 <== mon.1 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7f7a340037a0 con 0x7f7a40103a50 2024-08-19T01:43:30.957 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:30.956+0000 7f7a3e000640 1 --2- 172.21.15.83:0/3951265864 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f7a280a2b90 0x7f7a280a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:43:30.958 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:30.957+0000 7f7a3e000640 1 -- 172.21.15.83:0/3951265864 <== mon.1 v2:172.21.15.167:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8269+0+0 (secure 0 0 0) 0x7f7a340cf650 con 0x7f7a40103a50 2024-08-19T01:43:30.958 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:30.957+0000 7f7a45200640 1 --2- 172.21.15.83:0/3951265864 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f7a280a2b90 0x7f7a280a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:43:30.958 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:30.957+0000 7f7a45200640 1 --2- 172.21.15.83:0/3951265864 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f7a280a2b90 0x7f7a280a5050 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f7a2c009bd0 tx=0x7f7a2c005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:43:30.962 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:30.960+0000 7f7a3e000640 1 -- 172.21.15.83:0/3951265864 <== mon.1 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7f7a3409c530 con 0x7f7a40103a50 2024-08-19T01:43:30.965 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:43:30 smithi167 ceph-mon[21875]: from='client.? 172.21.15.83:0/1725126825' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-08-19T01:43:31.136 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:31.135+0000 7f7a48600640 1 -- 172.21.15.83:0/3951265864 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f7a40062bc0 con 0x7f7a40103a50 2024-08-19T01:43:31.137 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:31.136+0000 7f7a3e000640 1 -- 172.21.15.83:0/3951265864 <== mon.1 v2:172.21.15.167:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v86) ==== 104+0+11 (secure 0 0 0) 0x7f7a3409be30 con 0x7f7a40103a50 2024-08-19T01:43:31.137 INFO:teuthology.orchestra.run.smithi083.stdout:pg_num: 32 2024-08-19T01:43:31.139 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:31.138+0000 7f7a48600640 1 -- 172.21.15.83:0/3951265864 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f7a280a2b90 msgr2=0x7f7a280a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:43:31.140 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:31.138+0000 7f7a48600640 1 --2- 172.21.15.83:0/3951265864 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f7a280a2b90 0x7f7a280a5050 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f7a2c009bd0 tx=0x7f7a2c005f90 comp rx=0 tx=0).stop 2024-08-19T01:43:31.140 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:31.138+0000 7f7a48600640 1 -- 172.21.15.83:0/3951265864 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7a40103a50 msgr2=0x7f7a40199d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:43:31.140 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:31.138+0000 7f7a48600640 1 --2- 172.21.15.83:0/3951265864 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7a40103a50 0x7f7a40199d30 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f7a3400e9c0 tx=0x7f7a3400ee80 comp rx=0 tx=0).stop 2024-08-19T01:43:31.140 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:31.138+0000 7f7a48600640 1 -- 172.21.15.83:0/3951265864 shutdown_connections 2024-08-19T01:43:31.140 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:31.138+0000 7f7a48600640 1 --2- 172.21.15.83:0/3951265864 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f7a280a2b90 0x7f7a280a5050 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:43:31.140 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:31.138+0000 7f7a48600640 1 --2- 172.21.15.83:0/3951265864 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7a40105600 0x7f7a401a12a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:43:31.140 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:31.138+0000 7f7a48600640 1 --2- 172.21.15.83:0/3951265864 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7a40104c40 0x7f7a4019a270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:43:31.140 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:31.138+0000 7f7a48600640 1 --2- 172.21.15.83:0/3951265864 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7a40103a50 0x7f7a40199d30 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:43:31.140 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:31.138+0000 7f7a48600640 1 -- 172.21.15.83:0/3951265864 >> 172.21.15.83:0/3951265864 conn(0x7f7a400ff1d0 msgr2=0x7f7a40107fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:43:31.140 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:31.138+0000 7f7a48600640 1 -- 172.21.15.83:0/3951265864 shutdown_connections 2024-08-19T01:43:31.140 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:31.138+0000 7f7a48600640 1 -- 172.21.15.83:0/3951265864 wait complete. 2024-08-19T01:43:31.153 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:43:30 smithi083 ceph-mon[17120]: from='client.? 172.21.15.83:0/1725126825' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-08-19T01:43:31.155 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-08-19T01:43:31.202 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:43:30 smithi117 ceph-mon[22016]: from='client.? 172.21.15.83:0/1725126825' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-08-19T01:43:31.314 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:31.312+0000 7ffb5f200640 1 -- 172.21.15.83:0/188527297 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ffb60103940 msgr2=0x7ffb60103d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:43:31.314 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:31.312+0000 7ffb5f200640 1 --2- 172.21.15.83:0/188527297 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ffb60103940 0x7ffb60103d40 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7ffb50009900 tx=0x7ffb5002a3f0 comp rx=0 tx=0).stop 2024-08-19T01:43:31.314 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:31.312+0000 7ffb5f200640 1 -- 172.21.15.83:0/188527297 shutdown_connections 2024-08-19T01:43:31.315 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:31.312+0000 7ffb5f200640 1 --2- 172.21.15.83:0/188527297 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ffb601054f0 0x7ffb6010bd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:43:31.315 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:31.312+0000 7ffb5f200640 1 --2- 172.21.15.83:0/188527297 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ffb60104b30 0x7ffb60104fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:43:31.315 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:31.312+0000 7ffb5f200640 1 --2- 172.21.15.83:0/188527297 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ffb60103940 0x7ffb60103d40 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:43:31.315 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:31.312+0000 7ffb5f200640 1 -- 172.21.15.83:0/188527297 >> 172.21.15.83:0/188527297 conn(0x7ffb600ff0c0 msgr2=0x7ffb601014e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:43:31.315 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:31.312+0000 7ffb5f200640 1 -- 172.21.15.83:0/188527297 shutdown_connections 2024-08-19T01:43:31.315 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:31.312+0000 7ffb5f200640 1 -- 172.21.15.83:0/188527297 wait complete. 2024-08-19T01:43:31.317 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:31.313+0000 7ffb5f200640 1 Processor -- start 2024-08-19T01:43:31.317 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:31.313+0000 7ffb5f200640 1 -- start start 2024-08-19T01:43:31.317 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:31.314+0000 7ffb5f200640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ffb60103940 0x7ffb60199c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:43:31.318 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:31.314+0000 7ffb5f200640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ffb60104b30 0x7ffb6019a1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:43:31.318 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:31.314+0000 7ffb5f200640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ffb601054f0 0x7ffb601a1230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:43:31.318 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:31.314+0000 7ffb5f200640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7ffb6010f610 con 0x7ffb60103940 2024-08-19T01:43:31.318 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:31.314+0000 7ffb5f200640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7ffb6010f490 con 0x7ffb601054f0 2024-08-19T01:43:31.318 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:31.314+0000 7ffb5f200640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7ffb6010f790 con 0x7ffb60104b30 2024-08-19T01:43:31.318 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:31.314+0000 7ffb5de00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ffb60103940 0x7ffb60199c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:43:31.318 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:31.314+0000 7ffb5e800640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ffb601054f0 0x7ffb601a1230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:43:31.318 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:31.314+0000 7ffb5e800640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ffb601054f0 0x7ffb601a1230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.83:59002/0 (socket says 172.21.15.83:59002) 2024-08-19T01:43:31.318 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:31.314+0000 7ffb5e800640 1 -- 172.21.15.83:0/1590710327 learned_addr learned my addr 172.21.15.83:0/1590710327 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-08-19T01:43:31.318 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:31.314+0000 7ffb5d400640 1 --2- 172.21.15.83:0/1590710327 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ffb60104b30 0x7ffb6019a1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:43:31.318 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:31.314+0000 7ffb5e800640 1 -- 172.21.15.83:0/1590710327 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ffb60104b30 msgr2=0x7ffb6019a1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:43:31.318 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:31.314+0000 7ffb5e800640 1 --2- 172.21.15.83:0/1590710327 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ffb60104b30 0x7ffb6019a1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:43:31.319 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:31.314+0000 7ffb5e800640 1 -- 172.21.15.83:0/1590710327 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ffb60103940 msgr2=0x7ffb60199c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:43:31.319 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:31.314+0000 7ffb5e800640 1 --2- 172.21.15.83:0/1590710327 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ffb60103940 0x7ffb60199c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:43:31.319 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:31.314+0000 7ffb5e800640 1 -- 172.21.15.83:0/1590710327 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffb601a18d0 con 0x7ffb601054f0 2024-08-19T01:43:31.319 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:31.314+0000 7ffb5de00640 1 --2- 172.21.15.83:0/1590710327 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ffb60103940 0x7ffb60199c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-19T01:43:31.319 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:31.315+0000 7ffb5e800640 1 --2- 172.21.15.83:0/1590710327 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ffb601054f0 0x7ffb601a1230 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7ffb5800eb10 tx=0x7ffb5800c2d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:43:31.319 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:31.315+0000 7ffb56a00640 1 -- 172.21.15.83:0/1590710327 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb58004b20 con 0x7ffb601054f0 2024-08-19T01:43:31.319 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:31.315+0000 7ffb5f200640 1 -- 172.21.15.83:0/1590710327 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffb601a1bc0 con 0x7ffb601054f0 2024-08-19T01:43:31.319 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:31.315+0000 7ffb56a00640 1 -- 172.21.15.83:0/1590710327 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ffb58004cc0 con 0x7ffb601054f0 2024-08-19T01:43:31.319 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:31.315+0000 7ffb56a00640 1 -- 172.21.15.83:0/1590710327 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb58010760 con 0x7ffb601054f0 2024-08-19T01:43:31.319 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:31.315+0000 7ffb5f200640 1 -- 172.21.15.83:0/1590710327 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffb601a20e0 con 0x7ffb601054f0 2024-08-19T01:43:31.319 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:31.316+0000 7ffb5f200640 1 -- 172.21.15.83:0/1590710327 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffb6010ad30 con 0x7ffb601054f0 2024-08-19T01:43:31.320 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:31.318+0000 7ffb56a00640 1 -- 172.21.15.83:0/1590710327 <== mon.2 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7ffb58010900 con 0x7ffb601054f0 2024-08-19T01:43:31.321 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:31.319+0000 7ffb56a00640 1 --2- 172.21.15.83:0/1590710327 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7ffb440a2b90 0x7ffb440a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:43:31.321 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:31.320+0000 7ffb56a00640 1 -- 172.21.15.83:0/1590710327 <== mon.2 v2:172.21.15.117:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8269+0+0 (secure 0 0 0) 0x7ffb580cf840 con 0x7ffb601054f0 2024-08-19T01:43:31.321 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:31.320+0000 7ffb5de00640 1 --2- 172.21.15.83:0/1590710327 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7ffb440a2b90 0x7ffb440a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:43:31.321 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:31.320+0000 7ffb5de00640 1 --2- 172.21.15.83:0/1590710327 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7ffb440a2b90 0x7ffb440a5050 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7ffb5002a9e0 tx=0x7ffb500315b0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:43:31.322 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:31.321+0000 7ffb56a00640 1 -- 172.21.15.83:0/1590710327 <== mon.2 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7ffb58053df0 con 0x7ffb601054f0 2024-08-19T01:43:31.499 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:31.498+0000 7ffb5f200640 1 -- 172.21.15.83:0/1590710327 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7ffb6002d7e0 con 0x7ffb601054f0 2024-08-19T01:43:31.500 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:31.499+0000 7ffb56a00640 1 -- 172.21.15.83:0/1590710327 <== mon.2 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v86) ==== 100+0+11 (secure 0 0 0) 0x7ffb5809c720 con 0x7ffb601054f0 2024-08-19T01:43:31.501 INFO:teuthology.orchestra.run.smithi083.stdout:pg_num: 64 2024-08-19T01:43:31.503 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:31.501+0000 7ffb5f200640 1 -- 172.21.15.83:0/1590710327 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7ffb440a2b90 msgr2=0x7ffb440a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:43:31.503 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:31.501+0000 7ffb5f200640 1 --2- 172.21.15.83:0/1590710327 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7ffb440a2b90 0x7ffb440a5050 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7ffb5002a9e0 tx=0x7ffb500315b0 comp rx=0 tx=0).stop 2024-08-19T01:43:31.503 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:31.501+0000 7ffb5f200640 1 -- 172.21.15.83:0/1590710327 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ffb601054f0 msgr2=0x7ffb601a1230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:43:31.503 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:31.501+0000 7ffb5f200640 1 --2- 172.21.15.83:0/1590710327 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ffb601054f0 0x7ffb601a1230 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7ffb5800eb10 tx=0x7ffb5800c2d0 comp rx=0 tx=0).stop 2024-08-19T01:43:31.503 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:31.501+0000 7ffb5f200640 1 -- 172.21.15.83:0/1590710327 shutdown_connections 2024-08-19T01:43:31.503 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:31.501+0000 7ffb5f200640 1 --2- 172.21.15.83:0/1590710327 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7ffb440a2b90 0x7ffb440a5050 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:43:31.504 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:31.501+0000 7ffb5f200640 1 --2- 172.21.15.83:0/1590710327 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ffb601054f0 0x7ffb601a1230 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:43:31.504 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:31.501+0000 7ffb5f200640 1 --2- 172.21.15.83:0/1590710327 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ffb60104b30 0x7ffb6019a1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:43:31.504 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:31.501+0000 7ffb5f200640 1 --2- 172.21.15.83:0/1590710327 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ffb60103940 0x7ffb60199c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:43:31.504 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:31.501+0000 7ffb5f200640 1 -- 172.21.15.83:0/1590710327 >> 172.21.15.83:0/1590710327 conn(0x7ffb600ff0c0 msgr2=0x7ffb60107f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:43:31.504 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:31.501+0000 7ffb5f200640 1 -- 172.21.15.83:0/1590710327 shutdown_connections 2024-08-19T01:43:31.504 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:31.501+0000 7ffb5f200640 1 -- 172.21.15.83:0/1590710327 wait complete. 2024-08-19T01:43:31.519 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json-pretty 2024-08-19T01:43:31.675 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:31.674+0000 7f733b200640 1 -- 172.21.15.83:0/2917473052 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f733c0fff80 msgr2=0x7f733c100380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:43:31.676 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:31.674+0000 7f733b200640 1 --2- 172.21.15.83:0/2917473052 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f733c0fff80 0x7f733c100380 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f732c009900 tx=0x7f732c02a480 comp rx=0 tx=0).stop 2024-08-19T01:43:31.676 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:31.674+0000 7f733b200640 1 -- 172.21.15.83:0/2917473052 shutdown_connections 2024-08-19T01:43:31.676 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:31.674+0000 7f733b200640 1 --2- 172.21.15.83:0/2917473052 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f733c101260 0x7f733c101710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:43:31.676 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:31.674+0000 7f733b200640 1 --2- 172.21.15.83:0/2917473052 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f733c1008c0 0x7f733c100d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:43:31.676 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:31.674+0000 7f733b200640 1 --2- 172.21.15.83:0/2917473052 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f733c0fff80 0x7f733c100380 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:43:31.676 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:31.674+0000 7f733b200640 1 -- 172.21.15.83:0/2917473052 >> 172.21.15.83:0/2917473052 conn(0x7f733c0fbd30 msgr2=0x7f733c0fe150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:43:31.676 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:31.674+0000 7f733b200640 1 -- 172.21.15.83:0/2917473052 shutdown_connections 2024-08-19T01:43:31.676 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:31.674+0000 7f733b200640 1 -- 172.21.15.83:0/2917473052 wait complete. 2024-08-19T01:43:31.678 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:31.675+0000 7f733b200640 1 Processor -- start 2024-08-19T01:43:31.678 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:31.675+0000 7f733b200640 1 -- start start 2024-08-19T01:43:31.678 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:31.675+0000 7f733b200640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f733c0fff80 0x7f733c199be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:43:31.679 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:31.675+0000 7f733b200640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f733c1008c0 0x7f733c19a120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:43:31.679 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:31.675+0000 7f733b200640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f733c101260 0x7f733c1a1190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:43:31.679 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:31.675+0000 7f733b200640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f733c10f560 con 0x7f733c101260 2024-08-19T01:43:31.679 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:31.675+0000 7f733b200640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f733c10f3e0 con 0x7f733c0fff80 2024-08-19T01:43:31.679 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:31.675+0000 7f733b200640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f733c10f6e0 con 0x7f733c1008c0 2024-08-19T01:43:31.679 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:31.676+0000 7f733a800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f733c101260 0x7f733c1a1190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:43:31.679 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:31.676+0000 7f7339e00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f733c0fff80 0x7f733c199be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:43:31.679 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:31.676+0000 7f7339e00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f733c0fff80 0x7f733c199be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.83:59008/0 (socket says 172.21.15.83:59008) 2024-08-19T01:43:31.679 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:31.676+0000 7f7339e00640 1 -- 172.21.15.83:0/2973240649 learned_addr learned my addr 172.21.15.83:0/2973240649 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-08-19T01:43:31.679 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:31.676+0000 7f7339400640 1 --2- 172.21.15.83:0/2973240649 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f733c1008c0 0x7f733c19a120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:43:31.679 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:31.676+0000 7f733a800640 1 -- 172.21.15.83:0/2973240649 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f733c1008c0 msgr2=0x7f733c19a120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:43:31.679 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:31.676+0000 7f733a800640 1 --2- 172.21.15.83:0/2973240649 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f733c1008c0 0x7f733c19a120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:43:31.680 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:31.676+0000 7f733a800640 1 -- 172.21.15.83:0/2973240649 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f733c0fff80 msgr2=0x7f733c199be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:43:31.680 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:31.676+0000 7f733a800640 1 --2- 172.21.15.83:0/2973240649 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f733c0fff80 0x7f733c199be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:43:31.680 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:31.676+0000 7f733a800640 1 -- 172.21.15.83:0/2973240649 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f733c1a1800 con 0x7f733c101260 2024-08-19T01:43:31.680 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:31.676+0000 7f7339e00640 1 --2- 172.21.15.83:0/2973240649 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f733c0fff80 0x7f733c199be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-19T01:43:31.680 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:31.676+0000 7f733a800640 1 --2- 172.21.15.83:0/2973240649 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f733c101260 0x7f733c1a1190 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f7334009fd0 tx=0x7f733400ef10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:43:31.680 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:31.676+0000 7f7332a00640 1 -- 172.21.15.83:0/2973240649 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f733400ce20 con 0x7f733c101260 2024-08-19T01:43:31.680 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:31.677+0000 7f733b200640 1 -- 172.21.15.83:0/2973240649 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f733c1a1af0 con 0x7f733c101260 2024-08-19T01:43:31.680 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:31.677+0000 7f7332a00640 1 -- 172.21.15.83:0/2973240649 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7334005020 con 0x7f733c101260 2024-08-19T01:43:31.680 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:31.677+0000 7f7332a00640 1 -- 172.21.15.83:0/2973240649 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7334010720 con 0x7f733c101260 2024-08-19T01:43:31.680 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:31.677+0000 7f733b200640 1 -- 172.21.15.83:0/2973240649 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f733c1a2040 con 0x7f733c101260 2024-08-19T01:43:31.681 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:31.678+0000 7f733b200640 1 -- 172.21.15.83:0/2973240649 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f733c10acd0 con 0x7f733c101260 2024-08-19T01:43:31.681 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:31.680+0000 7f7332a00640 1 -- 172.21.15.83:0/2973240649 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7f7334005540 con 0x7f733c101260 2024-08-19T01:43:31.682 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:31.681+0000 7f7332a00640 1 --2- 172.21.15.83:0/2973240649 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f73200a2b90 0x7f73200a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:43:31.682 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:31.681+0000 7f7332a00640 1 -- 172.21.15.83:0/2973240649 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8269+0+0 (secure 0 0 0) 0x7f73340cf6b0 con 0x7f733c101260 2024-08-19T01:43:31.682 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:31.681+0000 7f7339e00640 1 --2- 172.21.15.83:0/2973240649 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f73200a2b90 0x7f73200a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:43:31.683 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:31.682+0000 7f7339e00640 1 --2- 172.21.15.83:0/2973240649 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f73200a2b90 0x7f73200a5050 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f732c0096c0 tx=0x7f732c0315b0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:43:31.686 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:31.685+0000 7f7332a00640 1 -- 172.21.15.83:0/2973240649 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7f733409c590 con 0x7f733c101260 2024-08-19T01:43:31.904 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:43:31 smithi083 ceph-mon[17120]: pgmap v599: 97 pgs: 97 active+clean; 35 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 255 B/s wr, 6 op/s 2024-08-19T01:43:31.904 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:43:31 smithi083 ceph-mon[17120]: from='client.? 172.21.15.83:0/3275406329' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-08-19T01:43:31.905 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:43:31 smithi083 ceph-mon[17120]: from='client.? 172.21.15.83:0/3951265864' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-08-19T01:43:31.905 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:43:31 smithi083 ceph-mon[17120]: from='client.? 172.21.15.83:0/1590710327' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-08-19T01:43:31.907 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:31.905+0000 7f733b200640 1 -- 172.21.15.83:0/2973240649 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json-pretty"} v 0) -- 0x7f733c062bc0 con 0x7f733c101260 2024-08-19T01:43:31.916 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:31.915+0000 7f7332a00640 1 -- 172.21.15.83:0/2973240649 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json-pretty"}]=0 v20) ==== 81+0+740368 (secure 0 0 0) 0x7f733409be90 con 0x7f733c101260 2024-08-19T01:43:31.927 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-08-19T01:43:31.927 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-08-19T01:43:31.927 INFO:teuthology.orchestra.run.smithi083.stdout: "epoch": 20, 2024-08-19T01:43:31.927 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-08-19T01:43:31.928 INFO:teuthology.orchestra.run.smithi083.stdout: "active_gid": 14150, 2024-08-19T01:43:31.928 INFO:teuthology.orchestra.run.smithi083.stdout: "active_name": "x", 2024-08-19T01:43:31.928 INFO:teuthology.orchestra.run.smithi083.stdout: "active_addrs": { 2024-08-19T01:43:31.928 INFO:teuthology.orchestra.run.smithi083.stdout: "addrvec": [ 2024-08-19T01:43:31.928 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-08-19T01:43:31.928 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "v2", 2024-08-19T01:43:31.928 INFO:teuthology.orchestra.run.smithi083.stdout: "addr": "172.21.15.83:6800", 2024-08-19T01:43:31.928 INFO:teuthology.orchestra.run.smithi083.stdout: "nonce": 683740916 2024-08-19T01:43:31.928 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:31.928 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-08-19T01:43:31.928 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "v1", 2024-08-19T01:43:31.928 INFO:teuthology.orchestra.run.smithi083.stdout: "addr": "172.21.15.83:6801", 2024-08-19T01:43:31.929 INFO:teuthology.orchestra.run.smithi083.stdout: "nonce": 683740916 2024-08-19T01:43:31.929 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-08-19T01:43:31.929 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-08-19T01:43:31.929 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:31.929 INFO:teuthology.orchestra.run.smithi083.stdout: "active_addr": "172.21.15.83:6801/683740916", 2024-08-19T01:43:31.929 INFO:teuthology.orchestra.run.smithi083.stdout: "active_change": "2024-08-19T01:26:04.277823+0000", 2024-08-19T01:43:31.929 INFO:teuthology.orchestra.run.smithi083.stdout: "active_mgr_features": 4540701547738038271, 2024-08-19T01:43:31.929 INFO:teuthology.orchestra.run.smithi083.stdout: "available": true, 2024-08-19T01:43:31.929 INFO:teuthology.orchestra.run.smithi083.stdout: "standbys": [ 2024-08-19T01:43:31.929 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-08-19T01:43:31.929 INFO:teuthology.orchestra.run.smithi083.stdout: "gid": 24109, 2024-08-19T01:43:31.929 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "z", 2024-08-19T01:43:31.929 INFO:teuthology.orchestra.run.smithi083.stdout: "mgr_features": 4540701547738038271, 2024-08-19T01:43:31.929 INFO:teuthology.orchestra.run.smithi083.stdout: "available_modules": [ 2024-08-19T01:43:31.930 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-08-19T01:43:31.930 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "alerts", 2024-08-19T01:43:31.930 INFO:teuthology.orchestra.run.smithi083.stdout: "can_run": true, 2024-08-19T01:43:31.930 INFO:teuthology.orchestra.run.smithi083.stdout: "error_string": "", 2024-08-19T01:43:31.930 INFO:teuthology.orchestra.run.smithi083.stdout: "module_options": { 2024-08-19T01:43:31.930 INFO:teuthology.orchestra.run.smithi083.stdout: "interval": { 2024-08-19T01:43:31.930 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "interval", 2024-08-19T01:43:31.930 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "secs", 2024-08-19T01:43:31.930 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:31.930 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-08-19T01:43:31.930 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "60", 2024-08-19T01:43:31.930 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:31.930 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:31.931 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:31.931 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "How frequently to reexamine health status", 2024-08-19T01:43:31.931 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:31.931 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:31.931 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:31.931 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:31.931 INFO:teuthology.orchestra.run.smithi083.stdout: "log_level": { 2024-08-19T01:43:31.931 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_level", 2024-08-19T01:43:31.931 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-08-19T01:43:31.931 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:31.931 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-08-19T01:43:31.931 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-08-19T01:43:31.931 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:31.932 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:31.932 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-08-19T01:43:31.932 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-08-19T01:43:31.932 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-08-19T01:43:31.932 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-08-19T01:43:31.932 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-08-19T01:43:31.932 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-08-19T01:43:31.932 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-08-19T01:43:31.932 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-08-19T01:43:31.932 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-08-19T01:43:31.932 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:31.932 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:31.932 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:31.933 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:31.933 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster": { 2024-08-19T01:43:31.933 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster", 2024-08-19T01:43:31.933 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-08-19T01:43:31.933 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:31.933 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-08-19T01:43:31.933 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-08-19T01:43:31.933 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:31.933 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:31.933 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:31.933 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-08-19T01:43:31.933 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:31.933 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:31.933 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:31.934 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:31.934 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster_level": { 2024-08-19T01:43:31.934 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster_level", 2024-08-19T01:43:31.934 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-08-19T01:43:31.934 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:31.934 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-08-19T01:43:31.934 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "info", 2024-08-19T01:43:31.934 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:31.934 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:31.934 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-08-19T01:43:31.934 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-08-19T01:43:31.935 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-08-19T01:43:31.935 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-08-19T01:43:31.935 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-08-19T01:43:31.935 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-08-19T01:43:31.935 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-08-19T01:43:31.935 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-08-19T01:43:31.935 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-08-19T01:43:31.935 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:31.935 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:31.935 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:31.935 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:31.935 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_file": { 2024-08-19T01:43:31.935 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_file", 2024-08-19T01:43:31.936 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-08-19T01:43:31.936 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:31.936 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-08-19T01:43:31.936 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-08-19T01:43:31.936 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:31.936 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:31.936 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:31.936 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-08-19T01:43:31.936 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:31.936 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:31.936 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:31.936 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:31.936 INFO:teuthology.orchestra.run.smithi083.stdout: "smtp_destination": { 2024-08-19T01:43:31.937 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "smtp_destination", 2024-08-19T01:43:31.937 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-08-19T01:43:31.937 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:31.937 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-08-19T01:43:31.937 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-08-19T01:43:31.937 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:31.937 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:31.937 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:31.937 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Email address to send alerts to", 2024-08-19T01:43:31.937 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:31.937 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:31.937 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:31.938 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:31.938 INFO:teuthology.orchestra.run.smithi083.stdout: "smtp_from_name": { 2024-08-19T01:43:31.938 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "smtp_from_name", 2024-08-19T01:43:31.938 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-08-19T01:43:31.938 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:31.938 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-08-19T01:43:31.938 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "Ceph", 2024-08-19T01:43:31.938 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:31.938 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:31.938 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:31.938 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Email From: name", 2024-08-19T01:43:31.938 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:31.938 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:31.939 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:31.939 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:31.939 INFO:teuthology.orchestra.run.smithi083.stdout: "smtp_host": { 2024-08-19T01:43:31.939 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "smtp_host", 2024-08-19T01:43:31.939 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-08-19T01:43:31.939 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:31.939 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-08-19T01:43:31.939 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-08-19T01:43:31.939 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:31.939 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:31.939 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:31.939 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "SMTP server", 2024-08-19T01:43:31.939 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:31.940 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:31.940 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:31.940 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:31.940 INFO:teuthology.orchestra.run.smithi083.stdout: "smtp_password": { 2024-08-19T01:43:31.940 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "smtp_password", 2024-08-19T01:43:31.940 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-08-19T01:43:31.940 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:31.940 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-08-19T01:43:31.940 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-08-19T01:43:31.940 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:31.940 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:31.940 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:31.940 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Password to authenticate with", 2024-08-19T01:43:31.941 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:31.941 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:31.941 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:31.941 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:31.941 INFO:teuthology.orchestra.run.smithi083.stdout: "smtp_port": { 2024-08-19T01:43:31.941 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "smtp_port", 2024-08-19T01:43:31.941 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2024-08-19T01:43:31.941 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:31.941 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-08-19T01:43:31.941 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "465", 2024-08-19T01:43:31.941 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:31.941 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:31.942 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:31.942 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "SMTP port", 2024-08-19T01:43:31.942 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:31.942 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:31.942 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:31.942 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:31.942 INFO:teuthology.orchestra.run.smithi083.stdout: "smtp_sender": { 2024-08-19T01:43:31.942 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "smtp_sender", 2024-08-19T01:43:31.942 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-08-19T01:43:31.942 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:31.942 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-08-19T01:43:31.942 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-08-19T01:43:31.942 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:31.943 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:31.943 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:31.943 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "SMTP envelope sender", 2024-08-19T01:43:31.943 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:31.943 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:31.943 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:31.943 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:31.943 INFO:teuthology.orchestra.run.smithi083.stdout: "smtp_ssl": { 2024-08-19T01:43:31.943 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "smtp_ssl", 2024-08-19T01:43:31.943 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-08-19T01:43:31.943 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:31.943 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-08-19T01:43:31.944 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "True", 2024-08-19T01:43:31.944 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:31.944 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:31.944 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:31.944 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Use SSL to connect to SMTP server", 2024-08-19T01:43:31.944 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:31.944 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:31.944 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:31.944 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:31.944 INFO:teuthology.orchestra.run.smithi083.stdout: "smtp_user": { 2024-08-19T01:43:31.944 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "smtp_user", 2024-08-19T01:43:31.944 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-08-19T01:43:31.945 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:31.945 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-08-19T01:43:31.945 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-08-19T01:43:31.945 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:31.945 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:31.945 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:31.945 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "User to authenticate as", 2024-08-19T01:43:31.945 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:31.945 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:31.945 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:31.945 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-08-19T01:43:31.945 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-08-19T01:43:31.946 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:31.946 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-08-19T01:43:31.946 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "balancer", 2024-08-19T01:43:31.946 INFO:teuthology.orchestra.run.smithi083.stdout: "can_run": true, 2024-08-19T01:43:31.946 INFO:teuthology.orchestra.run.smithi083.stdout: "error_string": "", 2024-08-19T01:43:31.946 INFO:teuthology.orchestra.run.smithi083.stdout: "module_options": { 2024-08-19T01:43:31.946 INFO:teuthology.orchestra.run.smithi083.stdout: "active": { 2024-08-19T01:43:31.946 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "active", 2024-08-19T01:43:31.946 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-08-19T01:43:31.946 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:31.946 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-08-19T01:43:31.946 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "True", 2024-08-19T01:43:31.947 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:31.947 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:31.947 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:31.947 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "automatically balance PGs across cluster", 2024-08-19T01:43:31.947 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:31.947 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:31.947 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:31.947 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:31.947 INFO:teuthology.orchestra.run.smithi083.stdout: "begin_time": { 2024-08-19T01:43:31.947 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "begin_time", 2024-08-19T01:43:31.947 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-08-19T01:43:31.947 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:31.947 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-08-19T01:43:31.948 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "0000", 2024-08-19T01:43:31.948 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:31.948 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:31.948 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:31.948 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "beginning time of day to automatically balance", 2024-08-19T01:43:31.948 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-08-19T01:43:31.948 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:31.948 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:31.948 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:31.948 INFO:teuthology.orchestra.run.smithi083.stdout: "begin_weekday": { 2024-08-19T01:43:31.948 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "begin_weekday", 2024-08-19T01:43:31.948 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "uint", 2024-08-19T01:43:31.949 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:31.949 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-08-19T01:43:31.949 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "0", 2024-08-19T01:43:31.949 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "0", 2024-08-19T01:43:31.949 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "6", 2024-08-19T01:43:31.949 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:31.949 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-08-19T01:43:31.949 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-08-19T01:43:31.949 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:31.949 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:31.949 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:31.949 INFO:teuthology.orchestra.run.smithi083.stdout: "crush_compat_max_iterations": { 2024-08-19T01:43:31.950 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "crush_compat_max_iterations", 2024-08-19T01:43:31.950 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "uint", 2024-08-19T01:43:31.950 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:31.950 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-08-19T01:43:31.950 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "25", 2024-08-19T01:43:31.950 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "1", 2024-08-19T01:43:31.950 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "250", 2024-08-19T01:43:31.950 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:31.950 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-08-19T01:43:31.950 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:31.950 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:31.950 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:31.951 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:31.951 INFO:teuthology.orchestra.run.smithi083.stdout: "crush_compat_metrics": { 2024-08-19T01:43:31.951 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "crush_compat_metrics", 2024-08-19T01:43:31.951 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-08-19T01:43:31.951 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:31.951 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-08-19T01:43:31.951 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "pgs,objects,bytes", 2024-08-19T01:43:31.951 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:31.951 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:31.951 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:31.951 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-08-19T01:43:31.951 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-08-19T01:43:31.952 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:31.952 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:31.952 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:31.952 INFO:teuthology.orchestra.run.smithi083.stdout: "crush_compat_step": { 2024-08-19T01:43:31.952 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "crush_compat_step", 2024-08-19T01:43:31.952 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "float", 2024-08-19T01:43:31.952 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:31.952 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-08-19T01:43:31.952 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "0.5", 2024-08-19T01:43:31.952 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "0.001", 2024-08-19T01:43:31.952 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "0.999", 2024-08-19T01:43:31.952 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:31.953 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "aggressiveness of optimization", 2024-08-19T01:43:31.953 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-08-19T01:43:31.953 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:31.953 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:31.953 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:31.953 INFO:teuthology.orchestra.run.smithi083.stdout: "end_time": { 2024-08-19T01:43:31.953 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "end_time", 2024-08-19T01:43:31.953 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-08-19T01:43:31.953 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:31.953 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-08-19T01:43:31.953 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "2359", 2024-08-19T01:43:31.953 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:31.953 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:31.954 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:31.954 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "ending time of day to automatically balance", 2024-08-19T01:43:31.954 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-08-19T01:43:31.954 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:31.954 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:31.954 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:31.954 INFO:teuthology.orchestra.run.smithi083.stdout: "end_weekday": { 2024-08-19T01:43:31.954 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "end_weekday", 2024-08-19T01:43:31.954 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "uint", 2024-08-19T01:43:31.954 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:31.954 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-08-19T01:43:31.954 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "0", 2024-08-19T01:43:31.955 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "0", 2024-08-19T01:43:31.955 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "6", 2024-08-19T01:43:31.955 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:31.955 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-08-19T01:43:31.955 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-08-19T01:43:31.955 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:31.955 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:31.955 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:31.955 INFO:teuthology.orchestra.run.smithi083.stdout: "log_level": { 2024-08-19T01:43:31.955 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_level", 2024-08-19T01:43:31.955 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-08-19T01:43:31.955 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:31.955 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-08-19T01:43:31.955 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-08-19T01:43:31.955 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:31.956 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:31.956 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-08-19T01:43:31.956 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-08-19T01:43:31.956 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-08-19T01:43:31.956 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-08-19T01:43:31.956 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-08-19T01:43:31.956 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-08-19T01:43:31.956 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-08-19T01:43:31.956 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-08-19T01:43:31.956 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-08-19T01:43:31.956 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:31.956 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:31.957 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:31.957 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:31.957 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster": { 2024-08-19T01:43:31.957 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster", 2024-08-19T01:43:31.957 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-08-19T01:43:31.957 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:31.957 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-08-19T01:43:31.957 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-08-19T01:43:31.957 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:31.957 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:31.957 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:31.958 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-08-19T01:43:31.958 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:31.958 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:31.958 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:31.958 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:31.958 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster_level": { 2024-08-19T01:43:31.958 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster_level", 2024-08-19T01:43:31.958 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-08-19T01:43:31.958 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:31.958 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-08-19T01:43:31.958 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "info", 2024-08-19T01:43:31.958 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:31.958 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:31.959 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-08-19T01:43:31.959 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-08-19T01:43:31.959 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-08-19T01:43:31.959 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-08-19T01:43:31.959 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-08-19T01:43:31.959 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-08-19T01:43:31.959 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-08-19T01:43:31.959 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-08-19T01:43:31.959 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-08-19T01:43:31.959 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:31.959 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:31.959 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:31.959 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:31.960 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_file": { 2024-08-19T01:43:31.960 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_file", 2024-08-19T01:43:31.960 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-08-19T01:43:31.960 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:31.960 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-08-19T01:43:31.960 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-08-19T01:43:31.960 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:31.960 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:31.960 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:31.961 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-08-19T01:43:31.961 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:31.961 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:31.961 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:31.961 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:31.961 INFO:teuthology.orchestra.run.smithi083.stdout: "min_score": { 2024-08-19T01:43:31.961 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "min_score", 2024-08-19T01:43:31.961 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "float", 2024-08-19T01:43:31.961 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:31.962 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-08-19T01:43:31.962 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "0", 2024-08-19T01:43:31.962 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:31.962 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:31.962 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:31.962 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-08-19T01:43:31.962 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:31.962 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:31.962 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:31.962 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:31.962 INFO:teuthology.orchestra.run.smithi083.stdout: "mode": { 2024-08-19T01:43:31.963 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "mode", 2024-08-19T01:43:31.963 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-08-19T01:43:31.963 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:31.963 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-08-19T01:43:31.963 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "upmap", 2024-08-19T01:43:31.963 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:31.963 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:31.963 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-08-19T01:43:31.963 INFO:teuthology.orchestra.run.smithi083.stdout: "crush-compat", 2024-08-19T01:43:31.963 INFO:teuthology.orchestra.run.smithi083.stdout: "none", 2024-08-19T01:43:31.963 INFO:teuthology.orchestra.run.smithi083.stdout: "read", 2024-08-19T01:43:31.963 INFO:teuthology.orchestra.run.smithi083.stdout: "upmap", 2024-08-19T01:43:31.964 INFO:teuthology.orchestra.run.smithi083.stdout: "upmap-read" 2024-08-19T01:43:31.964 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-08-19T01:43:31.964 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Balancer mode", 2024-08-19T01:43:31.964 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:31.964 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:31.964 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:31.964 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:31.964 INFO:teuthology.orchestra.run.smithi083.stdout: "pool_ids": { 2024-08-19T01:43:31.964 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "pool_ids", 2024-08-19T01:43:31.964 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-08-19T01:43:31.964 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:31.964 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-08-19T01:43:31.965 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-08-19T01:43:31.965 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:31.965 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:31.965 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:31.965 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-08-19T01:43:31.965 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:31.965 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:31.965 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:31.965 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:31.965 INFO:teuthology.orchestra.run.smithi083.stdout: "sleep_interval": { 2024-08-19T01:43:31.965 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "sleep_interval", 2024-08-19T01:43:31.965 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "secs", 2024-08-19T01:43:31.965 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:31.966 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-08-19T01:43:31.966 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "60", 2024-08-19T01:43:31.966 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:31.966 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:31.966 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:31.966 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-08-19T01:43:31.966 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:31.966 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:31.966 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:31.966 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:31.966 INFO:teuthology.orchestra.run.smithi083.stdout: "upmap_max_deviation": { 2024-08-19T01:43:31.967 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "upmap_max_deviation", 2024-08-19T01:43:31.967 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2024-08-19T01:43:31.967 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:31.967 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-08-19T01:43:31.967 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "5", 2024-08-19T01:43:31.967 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "1", 2024-08-19T01:43:31.967 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:31.967 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:31.967 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "deviation below which no optimization is attempted", 2024-08-19T01:43:31.967 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-08-19T01:43:31.967 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:31.967 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:31.967 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:31.967 INFO:teuthology.orchestra.run.smithi083.stdout: "upmap_max_optimizations": { 2024-08-19T01:43:31.968 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "upmap_max_optimizations", 2024-08-19T01:43:31.968 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "uint", 2024-08-19T01:43:31.968 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:31.968 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-08-19T01:43:31.968 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "10", 2024-08-19T01:43:31.968 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:31.968 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:31.968 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:31.968 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-08-19T01:43:31.968 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:31.968 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:31.968 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:31.969 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-08-19T01:43:31.969 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-08-19T01:43:31.969 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:31.969 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-08-19T01:43:31.969 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "cephadm", 2024-08-19T01:43:31.969 INFO:teuthology.orchestra.run.smithi083.stdout: "can_run": true, 2024-08-19T01:43:31.969 INFO:teuthology.orchestra.run.smithi083.stdout: "error_string": "", 2024-08-19T01:43:31.969 INFO:teuthology.orchestra.run.smithi083.stdout: "module_options": { 2024-08-19T01:43:31.969 INFO:teuthology.orchestra.run.smithi083.stdout: "agent_down_multiplier": { 2024-08-19T01:43:31.969 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "agent_down_multiplier", 2024-08-19T01:43:31.969 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "float", 2024-08-19T01:43:31.969 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:31.969 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-08-19T01:43:31.969 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "3.0", 2024-08-19T01:43:31.969 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:31.970 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:31.970 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:31.970 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-08-19T01:43:31.970 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:31.970 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:31.970 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:31.970 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:31.970 INFO:teuthology.orchestra.run.smithi083.stdout: "agent_refresh_rate": { 2024-08-19T01:43:31.970 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "agent_refresh_rate", 2024-08-19T01:43:31.970 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "secs", 2024-08-19T01:43:31.970 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:31.970 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-08-19T01:43:31.970 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "20", 2024-08-19T01:43:31.970 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:31.971 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:31.971 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:31.971 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-08-19T01:43:31.971 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:31.971 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:31.971 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:31.971 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:31.971 INFO:teuthology.orchestra.run.smithi083.stdout: "agent_starting_port": { 2024-08-19T01:43:31.971 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "agent_starting_port", 2024-08-19T01:43:31.971 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2024-08-19T01:43:31.971 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:31.971 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-08-19T01:43:31.971 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "4721", 2024-08-19T01:43:31.971 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:31.971 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:31.972 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:31.972 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-08-19T01:43:31.972 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:31.972 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:31.972 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:31.972 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:31.972 INFO:teuthology.orchestra.run.smithi083.stdout: "allow_ptrace": { 2024-08-19T01:43:31.972 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "allow_ptrace", 2024-08-19T01:43:31.972 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-08-19T01:43:31.972 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:31.972 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-08-19T01:43:31.972 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-08-19T01:43:31.972 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:31.973 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:31.973 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:31.973 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-08-19T01:43:31.973 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-08-19T01:43:31.973 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:31.973 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:31.973 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:31.973 INFO:teuthology.orchestra.run.smithi083.stdout: "autotune_interval": { 2024-08-19T01:43:31.973 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "autotune_interval", 2024-08-19T01:43:31.973 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "secs", 2024-08-19T01:43:31.973 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:31.973 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-08-19T01:43:31.973 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "600", 2024-08-19T01:43:31.974 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:31.974 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:31.974 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:31.974 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "how frequently to autotune daemon memory", 2024-08-19T01:43:31.974 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:31.974 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:31.974 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:31.974 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:31.974 INFO:teuthology.orchestra.run.smithi083.stdout: "autotune_memory_target_ratio": { 2024-08-19T01:43:31.974 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "autotune_memory_target_ratio", 2024-08-19T01:43:31.974 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "float", 2024-08-19T01:43:31.974 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:31.974 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-08-19T01:43:31.975 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "0.7", 2024-08-19T01:43:31.975 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:31.975 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:31.975 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:31.975 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-08-19T01:43:31.975 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:31.975 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:31.975 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:31.975 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:31.975 INFO:teuthology.orchestra.run.smithi083.stdout: "cephadm_log_destination": { 2024-08-19T01:43:31.975 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "cephadm_log_destination", 2024-08-19T01:43:31.975 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-08-19T01:43:31.975 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:31.976 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-08-19T01:43:31.976 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-08-19T01:43:31.976 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:31.976 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:31.976 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-08-19T01:43:31.976 INFO:teuthology.orchestra.run.smithi083.stdout: "file", 2024-08-19T01:43:31.976 INFO:teuthology.orchestra.run.smithi083.stdout: "file,syslog", 2024-08-19T01:43:31.976 INFO:teuthology.orchestra.run.smithi083.stdout: "syslog" 2024-08-19T01:43:31.976 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-08-19T01:43:31.976 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-08-19T01:43:31.976 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:31.976 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:31.977 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:31.977 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:31.977 INFO:teuthology.orchestra.run.smithi083.stdout: "cgroups_split": { 2024-08-19T01:43:31.977 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "cgroups_split", 2024-08-19T01:43:31.977 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-08-19T01:43:31.977 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:31.977 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-08-19T01:43:31.977 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "True", 2024-08-19T01:43:31.977 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:31.977 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:31.977 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:31.977 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-08-19T01:43:31.978 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:31.978 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:31.978 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:31.978 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:31.978 INFO:teuthology.orchestra.run.smithi083.stdout: "config_checks_enabled": { 2024-08-19T01:43:31.978 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "config_checks_enabled", 2024-08-19T01:43:31.978 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-08-19T01:43:31.978 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:31.978 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-08-19T01:43:31.978 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-08-19T01:43:31.978 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:31.978 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:31.978 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:31.979 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-08-19T01:43:31.979 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:31.979 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:31.979 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:31.979 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:31.979 INFO:teuthology.orchestra.run.smithi083.stdout: "config_dashboard": { 2024-08-19T01:43:31.979 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "config_dashboard", 2024-08-19T01:43:31.979 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-08-19T01:43:31.979 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:31.979 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-08-19T01:43:31.979 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "True", 2024-08-19T01:43:31.979 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:31.979 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:31.980 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:31.980 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-08-19T01:43:31.980 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:31.980 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:31.980 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:31.980 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:31.980 INFO:teuthology.orchestra.run.smithi083.stdout: "container_image_alertmanager": { 2024-08-19T01:43:31.980 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "container_image_alertmanager", 2024-08-19T01:43:31.980 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-08-19T01:43:31.980 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:31.980 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-08-19T01:43:31.980 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-08-19T01:43:31.980 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:31.981 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:31.981 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:31.981 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Prometheus container image", 2024-08-19T01:43:31.981 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:31.981 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:31.981 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:31.981 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:31.981 INFO:teuthology.orchestra.run.smithi083.stdout: "container_image_base": { 2024-08-19T01:43:31.981 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "container_image_base", 2024-08-19T01:43:31.981 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-08-19T01:43:31.981 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:31.981 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-08-19T01:43:31.981 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "quay.io/ceph/ceph", 2024-08-19T01:43:31.982 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:31.982 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:31.982 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:31.982 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Container image name, without the tag", 2024-08-19T01:43:31.982 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:31.982 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:31.982 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:31.982 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:31.982 INFO:teuthology.orchestra.run.smithi083.stdout: "container_image_elasticsearch": { 2024-08-19T01:43:31.982 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "container_image_elasticsearch", 2024-08-19T01:43:31.982 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-08-19T01:43:31.982 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:31.983 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-08-19T01:43:31.983 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-08-19T01:43:31.983 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:31.983 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:31.983 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:31.983 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "elasticsearch container image", 2024-08-19T01:43:31.983 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:31.983 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:31.983 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:31.983 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:31.983 INFO:teuthology.orchestra.run.smithi083.stdout: "container_image_grafana": { 2024-08-19T01:43:31.983 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "container_image_grafana", 2024-08-19T01:43:31.983 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-08-19T01:43:31.983 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:31.984 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-08-19T01:43:31.984 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "quay.io/ceph/grafana:9.4.12", 2024-08-19T01:43:31.984 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:31.984 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:31.984 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:31.984 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Prometheus container image", 2024-08-19T01:43:31.984 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:31.984 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:31.984 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:31.984 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:31.984 INFO:teuthology.orchestra.run.smithi083.stdout: "container_image_haproxy": { 2024-08-19T01:43:31.984 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "container_image_haproxy", 2024-08-19T01:43:31.984 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-08-19T01:43:31.985 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:31.985 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-08-19T01:43:31.985 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-08-19T01:43:31.985 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:31.985 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:31.985 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:31.985 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "HAproxy container image", 2024-08-19T01:43:31.985 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:31.985 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:31.985 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:31.985 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:31.985 INFO:teuthology.orchestra.run.smithi083.stdout: "container_image_jaeger_agent": { 2024-08-19T01:43:31.986 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "container_image_jaeger_agent", 2024-08-19T01:43:31.986 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-08-19T01:43:31.986 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:31.986 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-08-19T01:43:31.986 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-08-19T01:43:31.986 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:31.986 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:31.986 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:31.986 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Jaeger agent container image", 2024-08-19T01:43:31.986 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:31.986 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:31.986 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:31.986 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:31.987 INFO:teuthology.orchestra.run.smithi083.stdout: "container_image_jaeger_collector": { 2024-08-19T01:43:31.987 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "container_image_jaeger_collector", 2024-08-19T01:43:31.987 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-08-19T01:43:31.987 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:31.987 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-08-19T01:43:31.987 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-08-19T01:43:31.987 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:31.987 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:31.987 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:31.987 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Jaeger collector container image", 2024-08-19T01:43:31.987 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:31.987 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:31.987 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:31.987 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:31.988 INFO:teuthology.orchestra.run.smithi083.stdout: "container_image_jaeger_query": { 2024-08-19T01:43:31.988 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "container_image_jaeger_query", 2024-08-19T01:43:31.988 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-08-19T01:43:31.988 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:31.988 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-08-19T01:43:31.988 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-08-19T01:43:31.988 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:31.988 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:31.988 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:31.988 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Jaeger query container image", 2024-08-19T01:43:31.988 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:31.988 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:31.989 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:31.989 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:31.989 INFO:teuthology.orchestra.run.smithi083.stdout: "container_image_keepalived": { 2024-08-19T01:43:31.989 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "container_image_keepalived", 2024-08-19T01:43:31.989 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-08-19T01:43:31.989 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:31.989 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-08-19T01:43:31.989 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-08-19T01:43:31.989 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:31.989 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:31.989 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:31.989 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Keepalived container image", 2024-08-19T01:43:31.989 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:31.989 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:31.990 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:31.990 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:31.990 INFO:teuthology.orchestra.run.smithi083.stdout: "container_image_loki": { 2024-08-19T01:43:31.990 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "container_image_loki", 2024-08-19T01:43:31.990 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-08-19T01:43:31.990 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:31.990 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-08-19T01:43:31.990 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "docker.io/grafana/loki:3.0.0", 2024-08-19T01:43:31.990 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:31.990 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:31.990 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:31.990 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Loki container image", 2024-08-19T01:43:31.991 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:31.991 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:31.991 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:31.991 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:31.991 INFO:teuthology.orchestra.run.smithi083.stdout: "container_image_node_exporter": { 2024-08-19T01:43:31.991 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "container_image_node_exporter", 2024-08-19T01:43:31.991 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-08-19T01:43:31.991 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:31.991 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-08-19T01:43:31.991 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-08-19T01:43:31.991 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:31.991 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:31.991 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:31.991 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Prometheus container image", 2024-08-19T01:43:31.992 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:31.992 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:31.992 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:31.992 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:31.992 INFO:teuthology.orchestra.run.smithi083.stdout: "container_image_nvmeof": { 2024-08-19T01:43:31.992 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "container_image_nvmeof", 2024-08-19T01:43:31.992 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-08-19T01:43:31.992 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:31.992 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-08-19T01:43:31.992 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "quay.io/ceph/nvmeof:1.2.5", 2024-08-19T01:43:31.992 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:31.992 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:31.992 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:31.993 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Nvme-of container image", 2024-08-19T01:43:31.993 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:31.993 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:31.993 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:31.993 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:31.993 INFO:teuthology.orchestra.run.smithi083.stdout: "container_image_prometheus": { 2024-08-19T01:43:31.993 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "container_image_prometheus", 2024-08-19T01:43:31.993 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-08-19T01:43:31.993 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:31.993 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-08-19T01:43:31.993 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-08-19T01:43:31.993 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:31.994 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:31.994 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:31.994 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Prometheus container image", 2024-08-19T01:43:31.994 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:31.994 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:31.994 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:31.994 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:31.994 INFO:teuthology.orchestra.run.smithi083.stdout: "container_image_promtail": { 2024-08-19T01:43:31.994 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "container_image_promtail", 2024-08-19T01:43:31.994 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-08-19T01:43:31.994 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:31.994 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-08-19T01:43:31.995 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "docker.io/grafana/promtail:3.0.0", 2024-08-19T01:43:31.995 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:31.995 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:31.995 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:31.995 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Promtail container image", 2024-08-19T01:43:31.995 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:31.995 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:31.995 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:31.995 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:31.995 INFO:teuthology.orchestra.run.smithi083.stdout: "container_image_samba": { 2024-08-19T01:43:31.995 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "container_image_samba", 2024-08-19T01:43:31.995 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-08-19T01:43:31.996 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:31.996 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-08-19T01:43:31.996 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "quay.io/samba.org/samba-server:devbuilds-centos-amd64", 2024-08-19T01:43:31.996 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:31.996 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:31.996 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:31.996 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Samba/SMB container image", 2024-08-19T01:43:31.996 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:31.996 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:31.996 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:31.996 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:31.996 INFO:teuthology.orchestra.run.smithi083.stdout: "container_image_snmp_gateway": { 2024-08-19T01:43:31.996 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "container_image_snmp_gateway", 2024-08-19T01:43:31.997 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-08-19T01:43:31.997 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:31.997 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-08-19T01:43:31.997 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-08-19T01:43:31.997 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:31.997 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:31.997 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:31.997 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "SNMP Gateway container image", 2024-08-19T01:43:31.997 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:31.997 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:31.997 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:31.997 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:31.997 INFO:teuthology.orchestra.run.smithi083.stdout: "container_init": { 2024-08-19T01:43:31.998 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "container_init", 2024-08-19T01:43:31.998 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-08-19T01:43:31.998 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:31.998 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-08-19T01:43:31.998 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "True", 2024-08-19T01:43:31.998 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:31.998 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:31.998 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:31.998 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Run podman/docker with `--init`", 2024-08-19T01:43:31.998 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:31.998 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:31.999 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:31.999 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:31.999 INFO:teuthology.orchestra.run.smithi083.stdout: "daemon_cache_timeout": { 2024-08-19T01:43:31.999 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "daemon_cache_timeout", 2024-08-19T01:43:31.999 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "secs", 2024-08-19T01:43:31.999 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:31.999 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-08-19T01:43:31.999 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "600", 2024-08-19T01:43:31.999 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:31.999 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:31.999 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:31.999 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-08-19T01:43:31.999 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:31.999 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:31.999 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.000 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.000 INFO:teuthology.orchestra.run.smithi083.stdout: "default_cephadm_command_timeout": { 2024-08-19T01:43:32.000 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "default_cephadm_command_timeout", 2024-08-19T01:43:32.000 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2024-08-19T01:43:32.000 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.000 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-08-19T01:43:32.000 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "900", 2024-08-19T01:43:32.000 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.000 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.000 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:32.000 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-08-19T01:43:32.000 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.000 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.001 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.001 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.001 INFO:teuthology.orchestra.run.smithi083.stdout: "default_registry": { 2024-08-19T01:43:32.001 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "default_registry", 2024-08-19T01:43:32.001 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-08-19T01:43:32.001 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.001 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-08-19T01:43:32.001 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "docker.io", 2024-08-19T01:43:32.001 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.001 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.001 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:32.001 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-08-19T01:43:32.001 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.002 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.002 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.002 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.002 INFO:teuthology.orchestra.run.smithi083.stdout: "device_cache_timeout": { 2024-08-19T01:43:32.002 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "device_cache_timeout", 2024-08-19T01:43:32.002 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "secs", 2024-08-19T01:43:32.002 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.002 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-08-19T01:43:32.002 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "1800", 2024-08-19T01:43:32.002 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.002 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.002 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:32.002 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "seconds to cache device inventory", 2024-08-19T01:43:32.003 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.003 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.003 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.003 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.003 INFO:teuthology.orchestra.run.smithi083.stdout: "device_enhanced_scan": { 2024-08-19T01:43:32.003 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "device_enhanced_scan", 2024-08-19T01:43:32.003 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-08-19T01:43:32.003 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.003 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-08-19T01:43:32.003 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-08-19T01:43:32.003 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.003 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.003 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:32.003 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Use libstoragemgmt during device scans", 2024-08-19T01:43:32.004 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.004 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.004 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.004 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.004 INFO:teuthology.orchestra.run.smithi083.stdout: "facts_cache_timeout": { 2024-08-19T01:43:32.004 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "facts_cache_timeout", 2024-08-19T01:43:32.004 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "secs", 2024-08-19T01:43:32.004 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.004 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-08-19T01:43:32.004 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "60", 2024-08-19T01:43:32.004 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.004 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.004 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:32.004 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "seconds to cache host facts data", 2024-08-19T01:43:32.004 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.005 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.005 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.005 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.005 INFO:teuthology.orchestra.run.smithi083.stdout: "grafana_dashboards_path": { 2024-08-19T01:43:32.005 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "grafana_dashboards_path", 2024-08-19T01:43:32.005 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-08-19T01:43:32.005 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.005 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-08-19T01:43:32.005 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "/etc/grafana/dashboards/ceph-dashboard/", 2024-08-19T01:43:32.005 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.005 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.005 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:32.005 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "location of dashboards to include in grafana deployments", 2024-08-19T01:43:32.005 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.006 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.006 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.006 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.006 INFO:teuthology.orchestra.run.smithi083.stdout: "host_check_interval": { 2024-08-19T01:43:32.006 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "host_check_interval", 2024-08-19T01:43:32.006 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "secs", 2024-08-19T01:43:32.006 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.006 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-08-19T01:43:32.006 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "600", 2024-08-19T01:43:32.006 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.006 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.006 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:32.006 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "how frequently to perform a host check", 2024-08-19T01:43:32.006 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.007 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.007 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.007 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.007 INFO:teuthology.orchestra.run.smithi083.stdout: "hw_monitoring": { 2024-08-19T01:43:32.007 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "hw_monitoring", 2024-08-19T01:43:32.007 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-08-19T01:43:32.007 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.007 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-08-19T01:43:32.007 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-08-19T01:43:32.007 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.007 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.007 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:32.007 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-08-19T01:43:32.007 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.008 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.008 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.008 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.008 INFO:teuthology.orchestra.run.smithi083.stdout: "inventory_list_all": { 2024-08-19T01:43:32.008 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "inventory_list_all", 2024-08-19T01:43:32.008 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-08-19T01:43:32.008 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.008 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-08-19T01:43:32.008 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-08-19T01:43:32.008 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.008 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.008 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:32.009 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-08-19T01:43:32.009 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.009 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.009 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.009 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.009 INFO:teuthology.orchestra.run.smithi083.stdout: "log_level": { 2024-08-19T01:43:32.009 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_level", 2024-08-19T01:43:32.009 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-08-19T01:43:32.009 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.009 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-08-19T01:43:32.009 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-08-19T01:43:32.010 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.010 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.010 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-08-19T01:43:32.010 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-08-19T01:43:32.010 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-08-19T01:43:32.010 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-08-19T01:43:32.010 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-08-19T01:43:32.010 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-08-19T01:43:32.010 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-08-19T01:43:32.010 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-08-19T01:43:32.010 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-08-19T01:43:32.010 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.010 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.011 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.011 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.011 INFO:teuthology.orchestra.run.smithi083.stdout: "log_refresh_metadata": { 2024-08-19T01:43:32.011 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_refresh_metadata", 2024-08-19T01:43:32.011 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-08-19T01:43:32.011 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.011 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-08-19T01:43:32.011 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-08-19T01:43:32.011 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.011 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.011 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:32.012 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-08-19T01:43:32.012 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.012 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.012 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.012 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.012 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster": { 2024-08-19T01:43:32.012 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster", 2024-08-19T01:43:32.012 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-08-19T01:43:32.012 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.012 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-08-19T01:43:32.012 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "True", 2024-08-19T01:43:32.012 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.012 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.012 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:32.013 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-08-19T01:43:32.013 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.013 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.013 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.013 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.013 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster_level": { 2024-08-19T01:43:32.013 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster_level", 2024-08-19T01:43:32.013 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-08-19T01:43:32.013 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.013 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-08-19T01:43:32.013 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "info", 2024-08-19T01:43:32.013 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.013 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.013 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-08-19T01:43:32.014 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-08-19T01:43:32.014 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-08-19T01:43:32.014 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-08-19T01:43:32.014 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-08-19T01:43:32.014 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-08-19T01:43:32.014 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-08-19T01:43:32.014 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-08-19T01:43:32.014 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-08-19T01:43:32.014 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.014 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.014 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.014 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.014 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_file": { 2024-08-19T01:43:32.014 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_file", 2024-08-19T01:43:32.014 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-08-19T01:43:32.015 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.015 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-08-19T01:43:32.015 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-08-19T01:43:32.015 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.015 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.015 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:32.015 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-08-19T01:43:32.015 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.015 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.015 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.015 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.015 INFO:teuthology.orchestra.run.smithi083.stdout: "manage_etc_ceph_ceph_conf": { 2024-08-19T01:43:32.015 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-08-19T01:43:32.015 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-08-19T01:43:32.016 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.016 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-08-19T01:43:32.016 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-08-19T01:43:32.016 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.016 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.016 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:32.016 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-08-19T01:43:32.016 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.016 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.016 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.016 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.016 INFO:teuthology.orchestra.run.smithi083.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-08-19T01:43:32.016 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-08-19T01:43:32.016 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-08-19T01:43:32.016 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.017 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-08-19T01:43:32.017 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "*", 2024-08-19T01:43:32.017 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.017 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.017 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:32.017 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-08-19T01:43:32.017 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.017 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.017 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.017 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.017 INFO:teuthology.orchestra.run.smithi083.stdout: "max_count_per_host": { 2024-08-19T01:43:32.017 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "max_count_per_host", 2024-08-19T01:43:32.017 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2024-08-19T01:43:32.017 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.017 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-08-19T01:43:32.018 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "10", 2024-08-19T01:43:32.018 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.018 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.018 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:32.018 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "max number of daemons per service per host", 2024-08-19T01:43:32.018 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.018 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.018 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.018 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.018 INFO:teuthology.orchestra.run.smithi083.stdout: "max_osd_draining_count": { 2024-08-19T01:43:32.018 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "max_osd_draining_count", 2024-08-19T01:43:32.018 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2024-08-19T01:43:32.018 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.018 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-08-19T01:43:32.019 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "10", 2024-08-19T01:43:32.019 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.019 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.019 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:32.019 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-08-19T01:43:32.019 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.019 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.019 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.019 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.019 INFO:teuthology.orchestra.run.smithi083.stdout: "migration_current": { 2024-08-19T01:43:32.019 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "migration_current", 2024-08-19T01:43:32.019 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2024-08-19T01:43:32.019 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.019 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-08-19T01:43:32.019 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-08-19T01:43:32.020 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.020 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.020 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:32.020 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "internal - do not modify", 2024-08-19T01:43:32.020 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.020 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.020 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.020 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.020 INFO:teuthology.orchestra.run.smithi083.stdout: "mode": { 2024-08-19T01:43:32.020 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "mode", 2024-08-19T01:43:32.020 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-08-19T01:43:32.020 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.020 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-08-19T01:43:32.020 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "root", 2024-08-19T01:43:32.021 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.021 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.021 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-08-19T01:43:32.021 INFO:teuthology.orchestra.run.smithi083.stdout: "cephadm-package", 2024-08-19T01:43:32.021 INFO:teuthology.orchestra.run.smithi083.stdout: "root" 2024-08-19T01:43:32.021 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-08-19T01:43:32.021 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "mode for remote execution of cephadm", 2024-08-19T01:43:32.021 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.021 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.021 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.021 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.021 INFO:teuthology.orchestra.run.smithi083.stdout: "oob_default_addr": { 2024-08-19T01:43:32.021 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "oob_default_addr", 2024-08-19T01:43:32.021 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-08-19T01:43:32.022 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.022 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-08-19T01:43:32.022 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "169.254.1.1", 2024-08-19T01:43:32.022 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.022 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.022 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:32.022 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Default address for RedFish API (oob management).", 2024-08-19T01:43:32.022 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.022 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.022 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.022 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.022 INFO:teuthology.orchestra.run.smithi083.stdout: "prometheus_alerts_path": { 2024-08-19T01:43:32.022 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "prometheus_alerts_path", 2024-08-19T01:43:32.023 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-08-19T01:43:32.023 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.023 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-08-19T01:43:32.023 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-08-19T01:43:32.023 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.023 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.023 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:32.023 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-08-19T01:43:32.023 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.023 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.023 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.023 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.023 INFO:teuthology.orchestra.run.smithi083.stdout: "registry_insecure": { 2024-08-19T01:43:32.023 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "registry_insecure", 2024-08-19T01:43:32.024 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-08-19T01:43:32.024 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.024 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-08-19T01:43:32.024 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-08-19T01:43:32.024 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.024 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.024 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:32.024 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-08-19T01:43:32.024 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.024 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.024 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.024 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.024 INFO:teuthology.orchestra.run.smithi083.stdout: "registry_password": { 2024-08-19T01:43:32.025 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "registry_password", 2024-08-19T01:43:32.025 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-08-19T01:43:32.025 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.025 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-08-19T01:43:32.025 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-08-19T01:43:32.025 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.025 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.025 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:32.025 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-08-19T01:43:32.025 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.025 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.025 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.025 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.025 INFO:teuthology.orchestra.run.smithi083.stdout: "registry_url": { 2024-08-19T01:43:32.026 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "registry_url", 2024-08-19T01:43:32.026 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-08-19T01:43:32.026 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.026 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-08-19T01:43:32.026 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-08-19T01:43:32.026 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.026 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.026 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:32.026 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-08-19T01:43:32.026 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.026 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.026 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.026 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.027 INFO:teuthology.orchestra.run.smithi083.stdout: "registry_username": { 2024-08-19T01:43:32.027 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "registry_username", 2024-08-19T01:43:32.027 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-08-19T01:43:32.027 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.027 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-08-19T01:43:32.027 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-08-19T01:43:32.027 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.027 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.027 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:32.027 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-08-19T01:43:32.027 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.027 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.027 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.028 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.028 INFO:teuthology.orchestra.run.smithi083.stdout: "secure_monitoring_stack": { 2024-08-19T01:43:32.028 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "secure_monitoring_stack", 2024-08-19T01:43:32.028 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-08-19T01:43:32.028 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.028 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-08-19T01:43:32.028 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-08-19T01:43:32.028 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.028 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.028 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:32.028 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-08-19T01:43:32.028 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.028 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.029 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.029 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.029 INFO:teuthology.orchestra.run.smithi083.stdout: "service_discovery_port": { 2024-08-19T01:43:32.029 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "service_discovery_port", 2024-08-19T01:43:32.029 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2024-08-19T01:43:32.029 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.029 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-08-19T01:43:32.029 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "8765", 2024-08-19T01:43:32.029 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.029 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.029 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:32.030 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "cephadm service discovery port", 2024-08-19T01:43:32.030 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.030 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.030 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.030 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.030 INFO:teuthology.orchestra.run.smithi083.stdout: "ssh_config_file": { 2024-08-19T01:43:32.030 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "ssh_config_file", 2024-08-19T01:43:32.030 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-08-19T01:43:32.030 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.030 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-08-19T01:43:32.030 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-08-19T01:43:32.030 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.030 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.030 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:32.030 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-08-19T01:43:32.031 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.031 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.031 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.031 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.031 INFO:teuthology.orchestra.run.smithi083.stdout: "use_agent": { 2024-08-19T01:43:32.031 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "use_agent", 2024-08-19T01:43:32.031 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-08-19T01:43:32.031 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.031 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-08-19T01:43:32.031 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-08-19T01:43:32.031 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.031 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.031 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:32.032 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-08-19T01:43:32.032 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.032 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.032 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.032 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.032 INFO:teuthology.orchestra.run.smithi083.stdout: "use_repo_digest": { 2024-08-19T01:43:32.032 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "use_repo_digest", 2024-08-19T01:43:32.032 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-08-19T01:43:32.032 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.032 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-08-19T01:43:32.032 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "True", 2024-08-19T01:43:32.032 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.032 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.032 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:32.032 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-08-19T01:43:32.033 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.033 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.033 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.033 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.033 INFO:teuthology.orchestra.run.smithi083.stdout: "warn_on_failed_host_check": { 2024-08-19T01:43:32.033 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "warn_on_failed_host_check", 2024-08-19T01:43:32.033 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-08-19T01:43:32.033 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.033 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-08-19T01:43:32.033 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "True", 2024-08-19T01:43:32.033 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.033 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.033 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:32.033 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "raise a health warning if the host check fails", 2024-08-19T01:43:32.033 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.034 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.034 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.034 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.034 INFO:teuthology.orchestra.run.smithi083.stdout: "warn_on_stray_daemons": { 2024-08-19T01:43:32.034 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "warn_on_stray_daemons", 2024-08-19T01:43:32.034 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-08-19T01:43:32.034 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.034 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-08-19T01:43:32.034 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "True", 2024-08-19T01:43:32.034 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.034 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.034 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:32.034 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-08-19T01:43:32.035 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.035 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.035 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.035 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.035 INFO:teuthology.orchestra.run.smithi083.stdout: "warn_on_stray_hosts": { 2024-08-19T01:43:32.035 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "warn_on_stray_hosts", 2024-08-19T01:43:32.035 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-08-19T01:43:32.035 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.035 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-08-19T01:43:32.035 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "True", 2024-08-19T01:43:32.035 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.035 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.035 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:32.036 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-08-19T01:43:32.036 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.036 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.036 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.036 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-08-19T01:43:32.036 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-08-19T01:43:32.036 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.036 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-08-19T01:43:32.036 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "crash", 2024-08-19T01:43:32.036 INFO:teuthology.orchestra.run.smithi083.stdout: "can_run": true, 2024-08-19T01:43:32.036 INFO:teuthology.orchestra.run.smithi083.stdout: "error_string": "", 2024-08-19T01:43:32.036 INFO:teuthology.orchestra.run.smithi083.stdout: "module_options": { 2024-08-19T01:43:32.036 INFO:teuthology.orchestra.run.smithi083.stdout: "log_level": { 2024-08-19T01:43:32.037 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_level", 2024-08-19T01:43:32.037 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-08-19T01:43:32.037 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.037 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-08-19T01:43:32.037 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-08-19T01:43:32.037 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.037 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.037 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-08-19T01:43:32.037 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-08-19T01:43:32.037 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-08-19T01:43:32.037 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-08-19T01:43:32.037 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-08-19T01:43:32.038 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-08-19T01:43:32.038 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-08-19T01:43:32.038 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-08-19T01:43:32.038 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-08-19T01:43:32.038 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.038 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.038 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.038 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.038 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster": { 2024-08-19T01:43:32.038 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster", 2024-08-19T01:43:32.038 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-08-19T01:43:32.038 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.039 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-08-19T01:43:32.039 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-08-19T01:43:32.039 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.039 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.039 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:32.039 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-08-19T01:43:32.039 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.039 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.039 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.039 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.039 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster_level": { 2024-08-19T01:43:32.039 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster_level", 2024-08-19T01:43:32.039 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-08-19T01:43:32.040 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.040 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-08-19T01:43:32.040 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "info", 2024-08-19T01:43:32.040 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.040 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.040 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-08-19T01:43:32.040 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-08-19T01:43:32.040 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-08-19T01:43:32.040 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-08-19T01:43:32.040 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-08-19T01:43:32.040 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-08-19T01:43:32.040 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-08-19T01:43:32.040 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-08-19T01:43:32.040 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-08-19T01:43:32.041 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.041 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.041 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.041 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.041 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_file": { 2024-08-19T01:43:32.041 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_file", 2024-08-19T01:43:32.041 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-08-19T01:43:32.041 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.041 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-08-19T01:43:32.041 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-08-19T01:43:32.041 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.041 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.041 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:32.042 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-08-19T01:43:32.042 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.042 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.042 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.042 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.042 INFO:teuthology.orchestra.run.smithi083.stdout: "retain_interval": { 2024-08-19T01:43:32.042 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "retain_interval", 2024-08-19T01:43:32.042 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "secs", 2024-08-19T01:43:32.042 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.042 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-08-19T01:43:32.042 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "31536000", 2024-08-19T01:43:32.042 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.042 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.043 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:32.043 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "how long to retain crashes before pruning them", 2024-08-19T01:43:32.043 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.043 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.043 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.043 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.043 INFO:teuthology.orchestra.run.smithi083.stdout: "warn_recent_interval": { 2024-08-19T01:43:32.043 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "warn_recent_interval", 2024-08-19T01:43:32.043 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "secs", 2024-08-19T01:43:32.043 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.043 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-08-19T01:43:32.043 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "1209600", 2024-08-19T01:43:32.043 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.044 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.044 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:32.044 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "time interval in which to warn about recent crashes", 2024-08-19T01:43:32.044 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.044 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.044 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.044 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-08-19T01:43:32.044 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-08-19T01:43:32.044 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.044 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-08-19T01:43:32.044 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "dashboard", 2024-08-19T01:43:32.044 INFO:teuthology.orchestra.run.smithi083.stdout: "can_run": true, 2024-08-19T01:43:32.044 INFO:teuthology.orchestra.run.smithi083.stdout: "error_string": "", 2024-08-19T01:43:32.044 INFO:teuthology.orchestra.run.smithi083.stdout: "module_options": { 2024-08-19T01:43:32.045 INFO:teuthology.orchestra.run.smithi083.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-08-19T01:43:32.045 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-08-19T01:43:32.045 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2024-08-19T01:43:32.045 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.045 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-08-19T01:43:32.045 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "10", 2024-08-19T01:43:32.045 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.045 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.045 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:32.045 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-08-19T01:43:32.045 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.045 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.045 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.046 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.046 INFO:teuthology.orchestra.run.smithi083.stdout: "ALERTMANAGER_API_HOST": { 2024-08-19T01:43:32.046 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "ALERTMANAGER_API_HOST", 2024-08-19T01:43:32.046 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-08-19T01:43:32.046 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.046 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-08-19T01:43:32.046 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-08-19T01:43:32.046 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.046 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.046 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:32.046 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-08-19T01:43:32.046 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.046 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.047 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.047 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.047 INFO:teuthology.orchestra.run.smithi083.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-08-19T01:43:32.047 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-08-19T01:43:32.047 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-08-19T01:43:32.047 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.047 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-08-19T01:43:32.047 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "True", 2024-08-19T01:43:32.047 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.047 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.047 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:32.047 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-08-19T01:43:32.047 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.048 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.048 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.048 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.048 INFO:teuthology.orchestra.run.smithi083.stdout: "AUDIT_API_ENABLED": { 2024-08-19T01:43:32.048 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "AUDIT_API_ENABLED", 2024-08-19T01:43:32.048 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-08-19T01:43:32.048 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.048 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-08-19T01:43:32.048 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-08-19T01:43:32.048 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.048 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.048 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:32.048 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-08-19T01:43:32.049 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.049 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.049 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.049 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.049 INFO:teuthology.orchestra.run.smithi083.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-08-19T01:43:32.049 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-08-19T01:43:32.049 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-08-19T01:43:32.049 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.049 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-08-19T01:43:32.049 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "True", 2024-08-19T01:43:32.049 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.049 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.049 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:32.049 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-08-19T01:43:32.050 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.050 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.050 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.050 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.050 INFO:teuthology.orchestra.run.smithi083.stdout: "ENABLE_BROWSABLE_API": { 2024-08-19T01:43:32.050 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "ENABLE_BROWSABLE_API", 2024-08-19T01:43:32.050 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-08-19T01:43:32.050 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.050 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-08-19T01:43:32.050 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "True", 2024-08-19T01:43:32.050 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.050 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.051 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:32.051 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-08-19T01:43:32.051 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.051 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.051 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.051 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.051 INFO:teuthology.orchestra.run.smithi083.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-08-19T01:43:32.051 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-08-19T01:43:32.051 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-08-19T01:43:32.051 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.051 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-08-19T01:43:32.051 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "True", 2024-08-19T01:43:32.052 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.052 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.052 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:32.052 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-08-19T01:43:32.052 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.052 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.052 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.052 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.052 INFO:teuthology.orchestra.run.smithi083.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-08-19T01:43:32.052 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-08-19T01:43:32.052 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-08-19T01:43:32.052 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.052 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-08-19T01:43:32.053 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "True", 2024-08-19T01:43:32.053 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.053 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.053 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:32.053 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-08-19T01:43:32.053 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.053 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.053 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.053 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.053 INFO:teuthology.orchestra.run.smithi083.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-08-19T01:43:32.053 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-08-19T01:43:32.053 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-08-19T01:43:32.053 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.053 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-08-19T01:43:32.054 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "True", 2024-08-19T01:43:32.054 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.054 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.054 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:32.054 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-08-19T01:43:32.054 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.054 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.054 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.054 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.054 INFO:teuthology.orchestra.run.smithi083.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-08-19T01:43:32.054 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-08-19T01:43:32.054 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-08-19T01:43:32.054 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.055 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-08-19T01:43:32.055 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "True", 2024-08-19T01:43:32.055 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.055 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.055 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:32.055 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-08-19T01:43:32.055 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.055 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.055 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.055 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.055 INFO:teuthology.orchestra.run.smithi083.stdout: "FEATURE_TOGGLE_NFS": { 2024-08-19T01:43:32.055 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-08-19T01:43:32.055 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-08-19T01:43:32.056 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.056 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-08-19T01:43:32.056 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "True", 2024-08-19T01:43:32.056 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.056 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.056 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:32.056 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-08-19T01:43:32.056 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.056 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.056 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.056 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.056 INFO:teuthology.orchestra.run.smithi083.stdout: "FEATURE_TOGGLE_RBD": { 2024-08-19T01:43:32.056 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-08-19T01:43:32.057 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-08-19T01:43:32.057 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.057 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-08-19T01:43:32.057 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "True", 2024-08-19T01:43:32.057 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.057 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.057 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:32.057 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-08-19T01:43:32.057 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.057 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.057 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.057 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.057 INFO:teuthology.orchestra.run.smithi083.stdout: "FEATURE_TOGGLE_RGW": { 2024-08-19T01:43:32.058 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-08-19T01:43:32.058 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-08-19T01:43:32.058 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.058 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-08-19T01:43:32.058 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "True", 2024-08-19T01:43:32.058 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.058 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.058 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:32.058 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-08-19T01:43:32.058 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.058 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.058 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.058 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.059 INFO:teuthology.orchestra.run.smithi083.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-08-19T01:43:32.059 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-08-19T01:43:32.059 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-08-19T01:43:32.059 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.059 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-08-19T01:43:32.059 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-08-19T01:43:32.059 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.059 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.059 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:32.059 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-08-19T01:43:32.059 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.059 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.059 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.059 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.060 INFO:teuthology.orchestra.run.smithi083.stdout: "GRAFANA_API_PASSWORD": { 2024-08-19T01:43:32.060 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "GRAFANA_API_PASSWORD", 2024-08-19T01:43:32.060 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-08-19T01:43:32.060 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.060 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-08-19T01:43:32.060 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "admin", 2024-08-19T01:43:32.060 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.060 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.060 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:32.060 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-08-19T01:43:32.060 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.060 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.060 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.061 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.061 INFO:teuthology.orchestra.run.smithi083.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-08-19T01:43:32.061 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-08-19T01:43:32.061 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-08-19T01:43:32.061 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.061 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-08-19T01:43:32.061 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "True", 2024-08-19T01:43:32.061 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.061 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.062 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:32.062 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-08-19T01:43:32.062 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.062 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.062 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.062 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.062 INFO:teuthology.orchestra.run.smithi083.stdout: "GRAFANA_API_URL": { 2024-08-19T01:43:32.062 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "GRAFANA_API_URL", 2024-08-19T01:43:32.062 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-08-19T01:43:32.062 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.062 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-08-19T01:43:32.062 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-08-19T01:43:32.063 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.063 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.063 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:32.063 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-08-19T01:43:32.063 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.063 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.063 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.063 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.063 INFO:teuthology.orchestra.run.smithi083.stdout: "GRAFANA_API_USERNAME": { 2024-08-19T01:43:32.063 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "GRAFANA_API_USERNAME", 2024-08-19T01:43:32.063 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-08-19T01:43:32.063 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.063 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-08-19T01:43:32.063 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "admin", 2024-08-19T01:43:32.064 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.064 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.064 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:32.064 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-08-19T01:43:32.064 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.064 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.064 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.064 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.064 INFO:teuthology.orchestra.run.smithi083.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-08-19T01:43:32.064 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-08-19T01:43:32.064 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-08-19T01:43:32.064 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.064 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-08-19T01:43:32.064 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-08-19T01:43:32.065 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.065 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.065 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:32.065 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-08-19T01:43:32.065 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.065 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.065 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.065 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.065 INFO:teuthology.orchestra.run.smithi083.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-08-19T01:43:32.065 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-08-19T01:43:32.065 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-08-19T01:43:32.065 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.066 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-08-19T01:43:32.066 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-08-19T01:43:32.066 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.066 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.066 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:32.066 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-08-19T01:43:32.066 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.066 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.066 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.066 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.066 INFO:teuthology.orchestra.run.smithi083.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-08-19T01:43:32.067 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-08-19T01:43:32.067 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-08-19T01:43:32.067 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.067 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-08-19T01:43:32.067 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "True", 2024-08-19T01:43:32.067 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.067 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.067 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:32.067 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-08-19T01:43:32.067 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.067 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.067 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.067 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.068 INFO:teuthology.orchestra.run.smithi083.stdout: "ISSUE_TRACKER_API_KEY": { 2024-08-19T01:43:32.068 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-08-19T01:43:32.068 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-08-19T01:43:32.068 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.068 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-08-19T01:43:32.068 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-08-19T01:43:32.068 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.068 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.068 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:32.068 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-08-19T01:43:32.068 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.068 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.068 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.068 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.069 INFO:teuthology.orchestra.run.smithi083.stdout: "PROMETHEUS_API_HOST": { 2024-08-19T01:43:32.069 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "PROMETHEUS_API_HOST", 2024-08-19T01:43:32.069 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-08-19T01:43:32.069 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.069 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-08-19T01:43:32.069 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-08-19T01:43:32.069 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.069 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.069 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:32.069 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-08-19T01:43:32.069 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.069 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.069 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.069 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.069 INFO:teuthology.orchestra.run.smithi083.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-08-19T01:43:32.070 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-08-19T01:43:32.070 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-08-19T01:43:32.070 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.070 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-08-19T01:43:32.070 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "True", 2024-08-19T01:43:32.070 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.070 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.070 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:32.070 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-08-19T01:43:32.070 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.070 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.070 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.070 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.071 INFO:teuthology.orchestra.run.smithi083.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-08-19T01:43:32.071 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-08-19T01:43:32.071 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-08-19T01:43:32.071 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.071 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-08-19T01:43:32.071 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-08-19T01:43:32.071 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.071 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.071 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:32.071 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-08-19T01:43:32.071 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.071 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.071 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.072 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.072 INFO:teuthology.orchestra.run.smithi083.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-08-19T01:43:32.072 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-08-19T01:43:32.072 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-08-19T01:43:32.072 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.072 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-08-19T01:43:32.072 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-08-19T01:43:32.072 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.072 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.072 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:32.072 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-08-19T01:43:32.072 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.072 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.072 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.073 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.073 INFO:teuthology.orchestra.run.smithi083.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-08-19T01:43:32.073 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-08-19T01:43:32.073 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-08-19T01:43:32.073 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.073 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-08-19T01:43:32.073 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "True", 2024-08-19T01:43:32.073 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.073 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.073 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:32.073 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-08-19T01:43:32.073 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.073 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.073 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.074 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.074 INFO:teuthology.orchestra.run.smithi083.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-08-19T01:43:32.074 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-08-19T01:43:32.074 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-08-19T01:43:32.074 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.074 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-08-19T01:43:32.074 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "True", 2024-08-19T01:43:32.074 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.074 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.074 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:32.074 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-08-19T01:43:32.074 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.075 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.075 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.075 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.075 INFO:teuthology.orchestra.run.smithi083.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-08-19T01:43:32.075 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-08-19T01:43:32.075 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-08-19T01:43:32.075 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.075 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-08-19T01:43:32.075 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-08-19T01:43:32.075 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.075 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.075 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:32.075 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-08-19T01:43:32.075 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.076 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.076 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.076 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.076 INFO:teuthology.orchestra.run.smithi083.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-08-19T01:43:32.076 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-08-19T01:43:32.076 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-08-19T01:43:32.076 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.076 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-08-19T01:43:32.076 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-08-19T01:43:32.076 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.076 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.076 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:32.077 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-08-19T01:43:32.077 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.077 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.077 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.077 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.077 INFO:teuthology.orchestra.run.smithi083.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-08-19T01:43:32.077 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-08-19T01:43:32.077 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-08-19T01:43:32.077 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.077 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-08-19T01:43:32.077 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-08-19T01:43:32.077 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.077 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.078 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:32.078 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-08-19T01:43:32.078 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.078 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.078 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.078 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.078 INFO:teuthology.orchestra.run.smithi083.stdout: "PWD_POLICY_ENABLED": { 2024-08-19T01:43:32.078 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "PWD_POLICY_ENABLED", 2024-08-19T01:43:32.078 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-08-19T01:43:32.078 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.078 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-08-19T01:43:32.078 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "True", 2024-08-19T01:43:32.078 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.078 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.079 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:32.079 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-08-19T01:43:32.079 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.079 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.079 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.079 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.079 INFO:teuthology.orchestra.run.smithi083.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-08-19T01:43:32.079 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-08-19T01:43:32.079 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-08-19T01:43:32.079 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.079 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-08-19T01:43:32.079 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-08-19T01:43:32.080 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.080 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.080 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:32.080 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-08-19T01:43:32.080 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.080 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.080 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.080 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.080 INFO:teuthology.orchestra.run.smithi083.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-08-19T01:43:32.080 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-08-19T01:43:32.080 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2024-08-19T01:43:32.080 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.080 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-08-19T01:43:32.081 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "10", 2024-08-19T01:43:32.081 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.081 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.081 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:32.081 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-08-19T01:43:32.081 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.081 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.081 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.081 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.081 INFO:teuthology.orchestra.run.smithi083.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-08-19T01:43:32.081 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-08-19T01:43:32.081 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2024-08-19T01:43:32.082 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.082 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-08-19T01:43:32.082 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "8", 2024-08-19T01:43:32.082 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.082 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.082 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:32.082 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-08-19T01:43:32.082 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.082 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.082 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.082 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.082 INFO:teuthology.orchestra.run.smithi083.stdout: "REST_REQUESTS_TIMEOUT": { 2024-08-19T01:43:32.082 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-08-19T01:43:32.083 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2024-08-19T01:43:32.083 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.083 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-08-19T01:43:32.083 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "45", 2024-08-19T01:43:32.083 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.083 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.083 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:32.083 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-08-19T01:43:32.083 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.083 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.083 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.083 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.083 INFO:teuthology.orchestra.run.smithi083.stdout: "RGW_API_ACCESS_KEY": { 2024-08-19T01:43:32.084 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "RGW_API_ACCESS_KEY", 2024-08-19T01:43:32.084 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-08-19T01:43:32.084 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.084 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-08-19T01:43:32.084 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-08-19T01:43:32.084 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.084 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.084 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:32.084 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-08-19T01:43:32.084 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.084 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.084 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.084 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.084 INFO:teuthology.orchestra.run.smithi083.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-08-19T01:43:32.085 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-08-19T01:43:32.085 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-08-19T01:43:32.085 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.085 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-08-19T01:43:32.085 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "admin", 2024-08-19T01:43:32.085 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.085 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.085 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:32.085 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-08-19T01:43:32.085 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.085 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.085 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.085 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.086 INFO:teuthology.orchestra.run.smithi083.stdout: "RGW_API_SECRET_KEY": { 2024-08-19T01:43:32.086 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "RGW_API_SECRET_KEY", 2024-08-19T01:43:32.086 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-08-19T01:43:32.086 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.086 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-08-19T01:43:32.086 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-08-19T01:43:32.086 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.086 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.086 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:32.086 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-08-19T01:43:32.086 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.086 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.086 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.087 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.087 INFO:teuthology.orchestra.run.smithi083.stdout: "RGW_API_SSL_VERIFY": { 2024-08-19T01:43:32.087 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "RGW_API_SSL_VERIFY", 2024-08-19T01:43:32.087 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-08-19T01:43:32.087 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.087 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-08-19T01:43:32.087 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "True", 2024-08-19T01:43:32.087 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.087 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.087 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:32.087 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-08-19T01:43:32.087 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.087 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.088 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.088 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.088 INFO:teuthology.orchestra.run.smithi083.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-08-19T01:43:32.088 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-08-19T01:43:32.088 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2024-08-19T01:43:32.088 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.088 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-08-19T01:43:32.088 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "0", 2024-08-19T01:43:32.088 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.088 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.088 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:32.088 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-08-19T01:43:32.088 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.088 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.089 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.089 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.089 INFO:teuthology.orchestra.run.smithi083.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-08-19T01:43:32.089 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-08-19T01:43:32.089 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2024-08-19T01:43:32.089 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.089 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-08-19T01:43:32.089 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "10", 2024-08-19T01:43:32.089 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.089 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.089 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:32.089 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-08-19T01:43:32.089 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.089 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.090 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.090 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.090 INFO:teuthology.orchestra.run.smithi083.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-08-19T01:43:32.090 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-08-19T01:43:32.090 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2024-08-19T01:43:32.090 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.090 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-08-19T01:43:32.090 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "5", 2024-08-19T01:43:32.090 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.090 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.090 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:32.090 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-08-19T01:43:32.091 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.091 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.091 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.091 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.091 INFO:teuthology.orchestra.run.smithi083.stdout: "cross_origin_url": { 2024-08-19T01:43:32.091 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "cross_origin_url", 2024-08-19T01:43:32.091 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-08-19T01:43:32.091 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.091 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-08-19T01:43:32.091 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-08-19T01:43:32.091 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.091 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.091 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:32.091 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-08-19T01:43:32.092 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.092 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.092 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.092 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.092 INFO:teuthology.orchestra.run.smithi083.stdout: "crt_file": { 2024-08-19T01:43:32.092 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "crt_file", 2024-08-19T01:43:32.092 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-08-19T01:43:32.092 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.092 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-08-19T01:43:32.092 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-08-19T01:43:32.092 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.092 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.092 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:32.093 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-08-19T01:43:32.093 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.093 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.093 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.093 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.093 INFO:teuthology.orchestra.run.smithi083.stdout: "debug": { 2024-08-19T01:43:32.093 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "debug", 2024-08-19T01:43:32.093 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-08-19T01:43:32.093 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.093 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-08-19T01:43:32.093 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-08-19T01:43:32.093 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.093 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.094 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:32.094 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Enable/disable debug options", 2024-08-19T01:43:32.094 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.094 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.094 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.094 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.094 INFO:teuthology.orchestra.run.smithi083.stdout: "jwt_token_ttl": { 2024-08-19T01:43:32.094 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "jwt_token_ttl", 2024-08-19T01:43:32.094 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2024-08-19T01:43:32.094 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.094 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-08-19T01:43:32.094 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "28800", 2024-08-19T01:43:32.095 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.095 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.095 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:32.095 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-08-19T01:43:32.095 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.095 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.095 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.095 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.095 INFO:teuthology.orchestra.run.smithi083.stdout: "key_file": { 2024-08-19T01:43:32.095 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "key_file", 2024-08-19T01:43:32.095 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-08-19T01:43:32.095 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.095 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-08-19T01:43:32.095 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-08-19T01:43:32.096 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.096 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.096 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:32.096 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-08-19T01:43:32.096 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.096 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.096 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.096 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.096 INFO:teuthology.orchestra.run.smithi083.stdout: "log_level": { 2024-08-19T01:43:32.096 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_level", 2024-08-19T01:43:32.096 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-08-19T01:43:32.096 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.096 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-08-19T01:43:32.096 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-08-19T01:43:32.097 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.097 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.097 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-08-19T01:43:32.097 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-08-19T01:43:32.097 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-08-19T01:43:32.097 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-08-19T01:43:32.097 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-08-19T01:43:32.097 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-08-19T01:43:32.097 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-08-19T01:43:32.097 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-08-19T01:43:32.097 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-08-19T01:43:32.097 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.097 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.097 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.098 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.098 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster": { 2024-08-19T01:43:32.098 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster", 2024-08-19T01:43:32.098 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-08-19T01:43:32.098 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.098 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-08-19T01:43:32.098 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-08-19T01:43:32.098 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.098 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.098 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:32.098 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-08-19T01:43:32.098 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.098 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.099 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.099 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.099 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster_level": { 2024-08-19T01:43:32.099 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster_level", 2024-08-19T01:43:32.099 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-08-19T01:43:32.099 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.099 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-08-19T01:43:32.099 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "info", 2024-08-19T01:43:32.099 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.099 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.099 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-08-19T01:43:32.099 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-08-19T01:43:32.099 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-08-19T01:43:32.099 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-08-19T01:43:32.099 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-08-19T01:43:32.100 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-08-19T01:43:32.100 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-08-19T01:43:32.100 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-08-19T01:43:32.100 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-08-19T01:43:32.100 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.100 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.100 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.100 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.100 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_file": { 2024-08-19T01:43:32.100 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_file", 2024-08-19T01:43:32.100 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-08-19T01:43:32.100 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.100 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-08-19T01:43:32.101 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-08-19T01:43:32.101 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.101 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.101 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:32.101 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-08-19T01:43:32.101 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.101 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.101 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.101 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.101 INFO:teuthology.orchestra.run.smithi083.stdout: "motd": { 2024-08-19T01:43:32.101 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "motd", 2024-08-19T01:43:32.101 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-08-19T01:43:32.101 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.101 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-08-19T01:43:32.102 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-08-19T01:43:32.102 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.102 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.102 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:32.102 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "The message of the day", 2024-08-19T01:43:32.102 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.102 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.102 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.102 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.102 INFO:teuthology.orchestra.run.smithi083.stdout: "redirect_resolve_ip_addr": { 2024-08-19T01:43:32.102 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "redirect_resolve_ip_addr", 2024-08-19T01:43:32.102 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-08-19T01:43:32.102 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.103 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-08-19T01:43:32.103 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-08-19T01:43:32.103 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.103 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.103 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:32.103 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-08-19T01:43:32.103 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.103 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.103 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.103 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.103 INFO:teuthology.orchestra.run.smithi083.stdout: "server_addr": { 2024-08-19T01:43:32.103 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "server_addr", 2024-08-19T01:43:32.103 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-08-19T01:43:32.103 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.103 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-08-19T01:43:32.104 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "::", 2024-08-19T01:43:32.104 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.104 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.104 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:32.104 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-08-19T01:43:32.104 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.104 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.104 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.104 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.104 INFO:teuthology.orchestra.run.smithi083.stdout: "server_port": { 2024-08-19T01:43:32.104 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "server_port", 2024-08-19T01:43:32.104 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2024-08-19T01:43:32.104 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.104 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-08-19T01:43:32.104 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "8080", 2024-08-19T01:43:32.105 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.105 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.105 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:32.105 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-08-19T01:43:32.105 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.105 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.105 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.105 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.105 INFO:teuthology.orchestra.run.smithi083.stdout: "ssl": { 2024-08-19T01:43:32.105 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "ssl", 2024-08-19T01:43:32.105 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-08-19T01:43:32.105 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.105 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-08-19T01:43:32.105 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "True", 2024-08-19T01:43:32.105 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.105 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.106 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:32.106 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-08-19T01:43:32.106 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.106 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.106 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.106 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.106 INFO:teuthology.orchestra.run.smithi083.stdout: "ssl_server_port": { 2024-08-19T01:43:32.106 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "ssl_server_port", 2024-08-19T01:43:32.106 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2024-08-19T01:43:32.106 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.106 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-08-19T01:43:32.106 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "8443", 2024-08-19T01:43:32.106 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.107 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.107 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:32.107 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-08-19T01:43:32.107 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.107 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.107 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.107 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.107 INFO:teuthology.orchestra.run.smithi083.stdout: "standby_behaviour": { 2024-08-19T01:43:32.107 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "standby_behaviour", 2024-08-19T01:43:32.107 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-08-19T01:43:32.107 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.107 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-08-19T01:43:32.107 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "redirect", 2024-08-19T01:43:32.108 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.108 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.108 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-08-19T01:43:32.108 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-08-19T01:43:32.108 INFO:teuthology.orchestra.run.smithi083.stdout: "redirect" 2024-08-19T01:43:32.108 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-08-19T01:43:32.108 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-08-19T01:43:32.108 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.108 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.108 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.108 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.108 INFO:teuthology.orchestra.run.smithi083.stdout: "standby_error_status_code": { 2024-08-19T01:43:32.108 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "standby_error_status_code", 2024-08-19T01:43:32.108 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2024-08-19T01:43:32.109 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.109 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-08-19T01:43:32.109 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "500", 2024-08-19T01:43:32.109 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "400", 2024-08-19T01:43:32.109 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "599", 2024-08-19T01:43:32.109 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:32.109 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-08-19T01:43:32.109 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.109 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.109 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.109 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.109 INFO:teuthology.orchestra.run.smithi083.stdout: "url_prefix": { 2024-08-19T01:43:32.109 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "url_prefix", 2024-08-19T01:43:32.109 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-08-19T01:43:32.110 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.110 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-08-19T01:43:32.110 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-08-19T01:43:32.110 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.110 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.110 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:32.110 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-08-19T01:43:32.110 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.110 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.110 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.110 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-08-19T01:43:32.110 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-08-19T01:43:32.110 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.111 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-08-19T01:43:32.111 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "devicehealth", 2024-08-19T01:43:32.111 INFO:teuthology.orchestra.run.smithi083.stdout: "can_run": true, 2024-08-19T01:43:32.111 INFO:teuthology.orchestra.run.smithi083.stdout: "error_string": "", 2024-08-19T01:43:32.111 INFO:teuthology.orchestra.run.smithi083.stdout: "module_options": { 2024-08-19T01:43:32.111 INFO:teuthology.orchestra.run.smithi083.stdout: "enable_monitoring": { 2024-08-19T01:43:32.111 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "enable_monitoring", 2024-08-19T01:43:32.111 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-08-19T01:43:32.111 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.111 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-08-19T01:43:32.111 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "True", 2024-08-19T01:43:32.111 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.111 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.112 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:32.112 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "monitor device health metrics", 2024-08-19T01:43:32.112 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.112 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.112 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.112 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.112 INFO:teuthology.orchestra.run.smithi083.stdout: "log_level": { 2024-08-19T01:43:32.112 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_level", 2024-08-19T01:43:32.112 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-08-19T01:43:32.112 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.112 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-08-19T01:43:32.112 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-08-19T01:43:32.112 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.112 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.112 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-08-19T01:43:32.113 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-08-19T01:43:32.113 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-08-19T01:43:32.113 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-08-19T01:43:32.113 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-08-19T01:43:32.113 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-08-19T01:43:32.113 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-08-19T01:43:32.113 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-08-19T01:43:32.113 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-08-19T01:43:32.113 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.113 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.113 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.113 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.113 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster": { 2024-08-19T01:43:32.113 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster", 2024-08-19T01:43:32.113 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-08-19T01:43:32.114 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.114 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-08-19T01:43:32.114 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-08-19T01:43:32.114 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.114 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.114 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:32.114 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-08-19T01:43:32.114 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.114 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.114 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.114 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.114 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster_level": { 2024-08-19T01:43:32.114 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster_level", 2024-08-19T01:43:32.115 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-08-19T01:43:32.115 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.115 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-08-19T01:43:32.115 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "info", 2024-08-19T01:43:32.115 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.115 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.115 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-08-19T01:43:32.115 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-08-19T01:43:32.115 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-08-19T01:43:32.115 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-08-19T01:43:32.115 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-08-19T01:43:32.115 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-08-19T01:43:32.115 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-08-19T01:43:32.115 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-08-19T01:43:32.115 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-08-19T01:43:32.116 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.116 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.116 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.116 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.116 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_file": { 2024-08-19T01:43:32.116 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_file", 2024-08-19T01:43:32.116 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-08-19T01:43:32.116 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.116 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-08-19T01:43:32.116 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-08-19T01:43:32.116 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.116 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.116 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:32.116 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-08-19T01:43:32.116 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.117 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.117 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.117 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.117 INFO:teuthology.orchestra.run.smithi083.stdout: "mark_out_threshold": { 2024-08-19T01:43:32.117 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "mark_out_threshold", 2024-08-19T01:43:32.117 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "secs", 2024-08-19T01:43:32.117 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.117 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-08-19T01:43:32.117 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "2419200", 2024-08-19T01:43:32.117 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.117 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.117 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:32.117 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-08-19T01:43:32.117 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.118 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.118 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.118 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.118 INFO:teuthology.orchestra.run.smithi083.stdout: "pool_name": { 2024-08-19T01:43:32.118 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "pool_name", 2024-08-19T01:43:32.118 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-08-19T01:43:32.118 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.118 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-08-19T01:43:32.118 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "device_health_metrics", 2024-08-19T01:43:32.118 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.118 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.118 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:32.118 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "name of pool in which to store device health metrics", 2024-08-19T01:43:32.119 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.119 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.119 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.119 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.119 INFO:teuthology.orchestra.run.smithi083.stdout: "retention_period": { 2024-08-19T01:43:32.119 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "retention_period", 2024-08-19T01:43:32.119 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "secs", 2024-08-19T01:43:32.119 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.119 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-08-19T01:43:32.119 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "15552000", 2024-08-19T01:43:32.119 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.119 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.119 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:32.120 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "how long to retain device health metrics", 2024-08-19T01:43:32.120 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.120 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.120 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.120 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.120 INFO:teuthology.orchestra.run.smithi083.stdout: "scrape_frequency": { 2024-08-19T01:43:32.120 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "scrape_frequency", 2024-08-19T01:43:32.120 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "secs", 2024-08-19T01:43:32.120 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.120 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-08-19T01:43:32.120 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "86400", 2024-08-19T01:43:32.120 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.121 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.121 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:32.121 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "how frequently to scrape device health metrics", 2024-08-19T01:43:32.121 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.121 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.121 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.121 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.121 INFO:teuthology.orchestra.run.smithi083.stdout: "self_heal": { 2024-08-19T01:43:32.121 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "self_heal", 2024-08-19T01:43:32.121 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-08-19T01:43:32.121 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.121 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-08-19T01:43:32.121 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "True", 2024-08-19T01:43:32.121 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.122 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.122 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:32.122 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-08-19T01:43:32.122 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.122 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.122 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.122 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.122 INFO:teuthology.orchestra.run.smithi083.stdout: "sleep_interval": { 2024-08-19T01:43:32.122 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "sleep_interval", 2024-08-19T01:43:32.122 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "secs", 2024-08-19T01:43:32.122 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.122 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-08-19T01:43:32.122 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "600", 2024-08-19T01:43:32.122 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.123 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.123 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:32.123 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "how frequently to wake up and check device health", 2024-08-19T01:43:32.123 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.123 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.123 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.123 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.123 INFO:teuthology.orchestra.run.smithi083.stdout: "warn_threshold": { 2024-08-19T01:43:32.123 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "warn_threshold", 2024-08-19T01:43:32.123 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "secs", 2024-08-19T01:43:32.123 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.123 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-08-19T01:43:32.123 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "7257600", 2024-08-19T01:43:32.123 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.124 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.124 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:32.124 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-08-19T01:43:32.124 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.124 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.124 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.124 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-08-19T01:43:32.124 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-08-19T01:43:32.124 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.124 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-08-19T01:43:32.124 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "diskprediction_local", 2024-08-19T01:43:32.124 INFO:teuthology.orchestra.run.smithi083.stdout: "can_run": true, 2024-08-19T01:43:32.124 INFO:teuthology.orchestra.run.smithi083.stdout: "error_string": "", 2024-08-19T01:43:32.124 INFO:teuthology.orchestra.run.smithi083.stdout: "module_options": { 2024-08-19T01:43:32.124 INFO:teuthology.orchestra.run.smithi083.stdout: "log_level": { 2024-08-19T01:43:32.125 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_level", 2024-08-19T01:43:32.125 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-08-19T01:43:32.125 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.125 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-08-19T01:43:32.125 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-08-19T01:43:32.125 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.125 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.125 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-08-19T01:43:32.125 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-08-19T01:43:32.125 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-08-19T01:43:32.125 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-08-19T01:43:32.125 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-08-19T01:43:32.125 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-08-19T01:43:32.125 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-08-19T01:43:32.126 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-08-19T01:43:32.126 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-08-19T01:43:32.126 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.126 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.126 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.126 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.126 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster": { 2024-08-19T01:43:32.126 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster", 2024-08-19T01:43:32.126 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-08-19T01:43:32.126 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.126 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-08-19T01:43:32.126 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-08-19T01:43:32.126 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.126 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.126 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:32.127 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-08-19T01:43:32.127 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.127 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.127 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.127 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.127 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster_level": { 2024-08-19T01:43:32.127 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster_level", 2024-08-19T01:43:32.127 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-08-19T01:43:32.127 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.127 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-08-19T01:43:32.127 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "info", 2024-08-19T01:43:32.127 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.127 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.127 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-08-19T01:43:32.127 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-08-19T01:43:32.128 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-08-19T01:43:32.128 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-08-19T01:43:32.128 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-08-19T01:43:32.128 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-08-19T01:43:32.128 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-08-19T01:43:32.128 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-08-19T01:43:32.128 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-08-19T01:43:32.128 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.128 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.128 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.128 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.128 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_file": { 2024-08-19T01:43:32.128 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_file", 2024-08-19T01:43:32.128 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-08-19T01:43:32.129 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.129 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-08-19T01:43:32.129 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-08-19T01:43:32.129 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.129 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.129 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:32.129 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-08-19T01:43:32.129 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.129 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.129 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.129 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.129 INFO:teuthology.orchestra.run.smithi083.stdout: "predict_interval": { 2024-08-19T01:43:32.129 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "predict_interval", 2024-08-19T01:43:32.129 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-08-19T01:43:32.130 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.130 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-08-19T01:43:32.130 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "86400", 2024-08-19T01:43:32.130 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.130 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.130 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:32.130 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-08-19T01:43:32.130 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.130 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.130 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.130 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.130 INFO:teuthology.orchestra.run.smithi083.stdout: "predictor_model": { 2024-08-19T01:43:32.130 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "predictor_model", 2024-08-19T01:43:32.131 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-08-19T01:43:32.131 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.131 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-08-19T01:43:32.131 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "prophetstor", 2024-08-19T01:43:32.131 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.131 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.131 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:32.131 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-08-19T01:43:32.131 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.131 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.131 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.131 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.132 INFO:teuthology.orchestra.run.smithi083.stdout: "sleep_interval": { 2024-08-19T01:43:32.132 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "sleep_interval", 2024-08-19T01:43:32.132 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-08-19T01:43:32.132 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.132 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-08-19T01:43:32.132 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "600", 2024-08-19T01:43:32.132 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.132 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.132 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:32.132 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-08-19T01:43:32.132 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.132 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.132 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.132 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-08-19T01:43:32.133 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-08-19T01:43:32.133 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.133 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-08-19T01:43:32.133 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "influx", 2024-08-19T01:43:32.133 INFO:teuthology.orchestra.run.smithi083.stdout: "can_run": false, 2024-08-19T01:43:32.133 INFO:teuthology.orchestra.run.smithi083.stdout: "error_string": "influxdb python module not found", 2024-08-19T01:43:32.133 INFO:teuthology.orchestra.run.smithi083.stdout: "module_options": { 2024-08-19T01:43:32.133 INFO:teuthology.orchestra.run.smithi083.stdout: "batch_size": { 2024-08-19T01:43:32.133 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "batch_size", 2024-08-19T01:43:32.133 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2024-08-19T01:43:32.133 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.133 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-08-19T01:43:32.133 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "5000", 2024-08-19T01:43:32.134 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.134 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.134 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:32.134 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-08-19T01:43:32.134 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.134 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.134 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.134 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.134 INFO:teuthology.orchestra.run.smithi083.stdout: "database": { 2024-08-19T01:43:32.134 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "database", 2024-08-19T01:43:32.134 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-08-19T01:43:32.134 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.135 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-08-19T01:43:32.135 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "ceph", 2024-08-19T01:43:32.135 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.135 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.135 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:32.135 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-08-19T01:43:32.135 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.135 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.135 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.135 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.135 INFO:teuthology.orchestra.run.smithi083.stdout: "hostname": { 2024-08-19T01:43:32.135 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "hostname", 2024-08-19T01:43:32.135 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-08-19T01:43:32.135 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.136 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-08-19T01:43:32.136 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-08-19T01:43:32.136 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.136 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.136 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:32.136 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "InfluxDB server hostname", 2024-08-19T01:43:32.136 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.136 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.136 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.136 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.136 INFO:teuthology.orchestra.run.smithi083.stdout: "interval": { 2024-08-19T01:43:32.136 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "interval", 2024-08-19T01:43:32.136 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "secs", 2024-08-19T01:43:32.136 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.137 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-08-19T01:43:32.137 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "30", 2024-08-19T01:43:32.137 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "5", 2024-08-19T01:43:32.137 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.137 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:32.137 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-08-19T01:43:32.137 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.137 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.137 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.137 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.137 INFO:teuthology.orchestra.run.smithi083.stdout: "log_level": { 2024-08-19T01:43:32.137 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_level", 2024-08-19T01:43:32.137 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-08-19T01:43:32.138 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.138 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-08-19T01:43:32.138 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-08-19T01:43:32.138 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.138 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.138 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-08-19T01:43:32.138 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-08-19T01:43:32.138 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-08-19T01:43:32.138 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-08-19T01:43:32.138 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-08-19T01:43:32.138 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-08-19T01:43:32.138 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-08-19T01:43:32.138 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-08-19T01:43:32.138 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-08-19T01:43:32.139 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.139 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.139 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.139 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.139 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster": { 2024-08-19T01:43:32.139 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster", 2024-08-19T01:43:32.139 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-08-19T01:43:32.139 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.139 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-08-19T01:43:32.139 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-08-19T01:43:32.139 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.139 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.139 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:32.140 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-08-19T01:43:32.140 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.140 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.140 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.140 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.140 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster_level": { 2024-08-19T01:43:32.140 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster_level", 2024-08-19T01:43:32.140 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-08-19T01:43:32.140 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.140 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-08-19T01:43:32.140 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "info", 2024-08-19T01:43:32.140 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.140 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.141 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-08-19T01:43:32.141 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-08-19T01:43:32.141 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-08-19T01:43:32.141 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-08-19T01:43:32.141 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-08-19T01:43:32.141 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-08-19T01:43:32.141 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-08-19T01:43:32.141 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-08-19T01:43:32.141 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-08-19T01:43:32.141 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.141 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.141 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.141 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.141 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_file": { 2024-08-19T01:43:32.142 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_file", 2024-08-19T01:43:32.142 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-08-19T01:43:32.142 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.142 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-08-19T01:43:32.142 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-08-19T01:43:32.142 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.142 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.142 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:32.142 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-08-19T01:43:32.142 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.142 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.142 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.142 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.143 INFO:teuthology.orchestra.run.smithi083.stdout: "password": { 2024-08-19T01:43:32.143 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "password", 2024-08-19T01:43:32.143 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-08-19T01:43:32.143 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.143 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-08-19T01:43:32.143 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-08-19T01:43:32.143 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.143 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.143 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:32.143 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "password of InfluxDB server user", 2024-08-19T01:43:32.143 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.143 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.143 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.144 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.144 INFO:teuthology.orchestra.run.smithi083.stdout: "port": { 2024-08-19T01:43:32.144 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "port", 2024-08-19T01:43:32.144 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2024-08-19T01:43:32.144 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.144 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-08-19T01:43:32.144 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "8086", 2024-08-19T01:43:32.144 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.144 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.144 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:32.144 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "InfluxDB server port", 2024-08-19T01:43:32.144 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.144 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.145 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.145 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.145 INFO:teuthology.orchestra.run.smithi083.stdout: "ssl": { 2024-08-19T01:43:32.145 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "ssl", 2024-08-19T01:43:32.145 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-08-19T01:43:32.145 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.145 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-08-19T01:43:32.145 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "false", 2024-08-19T01:43:32.145 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.145 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.145 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:32.145 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-08-19T01:43:32.145 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.145 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.146 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.146 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.146 INFO:teuthology.orchestra.run.smithi083.stdout: "threads": { 2024-08-19T01:43:32.146 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "threads", 2024-08-19T01:43:32.146 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2024-08-19T01:43:32.146 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.146 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-08-19T01:43:32.146 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "5", 2024-08-19T01:43:32.146 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "1", 2024-08-19T01:43:32.146 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "32", 2024-08-19T01:43:32.146 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:32.147 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-08-19T01:43:32.147 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.147 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.147 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.147 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.147 INFO:teuthology.orchestra.run.smithi083.stdout: "username": { 2024-08-19T01:43:32.147 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "username", 2024-08-19T01:43:32.147 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-08-19T01:43:32.147 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.147 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-08-19T01:43:32.147 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-08-19T01:43:32.147 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.147 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.147 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:32.148 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "username of InfluxDB server user", 2024-08-19T01:43:32.148 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.148 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.148 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.148 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.148 INFO:teuthology.orchestra.run.smithi083.stdout: "verify_ssl": { 2024-08-19T01:43:32.148 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "verify_ssl", 2024-08-19T01:43:32.148 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-08-19T01:43:32.148 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.148 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-08-19T01:43:32.148 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "true", 2024-08-19T01:43:32.148 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.149 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.149 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:32.149 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-08-19T01:43:32.149 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.149 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.149 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.149 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-08-19T01:43:32.149 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-08-19T01:43:32.149 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.149 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-08-19T01:43:32.149 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "insights", 2024-08-19T01:43:32.149 INFO:teuthology.orchestra.run.smithi083.stdout: "can_run": true, 2024-08-19T01:43:32.149 INFO:teuthology.orchestra.run.smithi083.stdout: "error_string": "", 2024-08-19T01:43:32.150 INFO:teuthology.orchestra.run.smithi083.stdout: "module_options": { 2024-08-19T01:43:32.150 INFO:teuthology.orchestra.run.smithi083.stdout: "log_level": { 2024-08-19T01:43:32.150 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_level", 2024-08-19T01:43:32.150 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-08-19T01:43:32.150 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.150 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-08-19T01:43:32.150 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-08-19T01:43:32.150 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.150 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.150 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-08-19T01:43:32.150 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-08-19T01:43:32.150 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-08-19T01:43:32.150 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-08-19T01:43:32.151 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-08-19T01:43:32.151 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-08-19T01:43:32.151 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-08-19T01:43:32.151 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-08-19T01:43:32.151 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-08-19T01:43:32.151 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.151 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.151 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.151 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.151 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster": { 2024-08-19T01:43:32.151 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster", 2024-08-19T01:43:32.152 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-08-19T01:43:32.152 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.152 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-08-19T01:43:32.152 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-08-19T01:43:32.152 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.152 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.152 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:32.152 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-08-19T01:43:32.152 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.152 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.152 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.152 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.152 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster_level": { 2024-08-19T01:43:32.152 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster_level", 2024-08-19T01:43:32.153 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-08-19T01:43:32.153 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.153 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-08-19T01:43:32.153 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "info", 2024-08-19T01:43:32.153 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.153 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.153 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-08-19T01:43:32.153 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-08-19T01:43:32.153 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-08-19T01:43:32.153 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-08-19T01:43:32.153 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-08-19T01:43:32.153 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-08-19T01:43:32.153 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-08-19T01:43:32.153 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-08-19T01:43:32.153 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-08-19T01:43:32.154 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.154 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.154 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.154 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.154 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_file": { 2024-08-19T01:43:32.154 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_file", 2024-08-19T01:43:32.154 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-08-19T01:43:32.154 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.154 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-08-19T01:43:32.154 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-08-19T01:43:32.154 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.154 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.155 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:32.155 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-08-19T01:43:32.155 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.155 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.155 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.155 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-08-19T01:43:32.155 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-08-19T01:43:32.155 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.155 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-08-19T01:43:32.155 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "iostat", 2024-08-19T01:43:32.155 INFO:teuthology.orchestra.run.smithi083.stdout: "can_run": true, 2024-08-19T01:43:32.155 INFO:teuthology.orchestra.run.smithi083.stdout: "error_string": "", 2024-08-19T01:43:32.155 INFO:teuthology.orchestra.run.smithi083.stdout: "module_options": { 2024-08-19T01:43:32.155 INFO:teuthology.orchestra.run.smithi083.stdout: "log_level": { 2024-08-19T01:43:32.155 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_level", 2024-08-19T01:43:32.156 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-08-19T01:43:32.156 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.156 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-08-19T01:43:32.156 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-08-19T01:43:32.156 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.156 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.156 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-08-19T01:43:32.156 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-08-19T01:43:32.156 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-08-19T01:43:32.156 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-08-19T01:43:32.156 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-08-19T01:43:32.156 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-08-19T01:43:32.156 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-08-19T01:43:32.156 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-08-19T01:43:32.156 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-08-19T01:43:32.157 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.157 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.157 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.157 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.157 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster": { 2024-08-19T01:43:32.157 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster", 2024-08-19T01:43:32.157 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-08-19T01:43:32.157 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.157 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-08-19T01:43:32.157 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-08-19T01:43:32.157 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.157 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.157 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:32.157 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-08-19T01:43:32.158 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.158 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.158 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.158 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.158 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster_level": { 2024-08-19T01:43:32.158 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster_level", 2024-08-19T01:43:32.158 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-08-19T01:43:32.158 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.158 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-08-19T01:43:32.158 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "info", 2024-08-19T01:43:32.158 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.158 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.159 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-08-19T01:43:32.159 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-08-19T01:43:32.159 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-08-19T01:43:32.159 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-08-19T01:43:32.159 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-08-19T01:43:32.159 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-08-19T01:43:32.159 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-08-19T01:43:32.159 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-08-19T01:43:32.159 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-08-19T01:43:32.159 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.159 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.159 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.160 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.160 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_file": { 2024-08-19T01:43:32.160 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_file", 2024-08-19T01:43:32.160 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-08-19T01:43:32.160 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.160 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-08-19T01:43:32.160 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-08-19T01:43:32.160 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.160 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.160 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:32.160 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-08-19T01:43:32.160 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.160 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.161 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.161 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-08-19T01:43:32.161 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-08-19T01:43:32.161 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.161 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-08-19T01:43:32.161 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "k8sevents", 2024-08-19T01:43:32.161 INFO:teuthology.orchestra.run.smithi083.stdout: "can_run": true, 2024-08-19T01:43:32.161 INFO:teuthology.orchestra.run.smithi083.stdout: "error_string": "", 2024-08-19T01:43:32.161 INFO:teuthology.orchestra.run.smithi083.stdout: "module_options": { 2024-08-19T01:43:32.161 INFO:teuthology.orchestra.run.smithi083.stdout: "ceph_event_retention_days": { 2024-08-19T01:43:32.162 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "ceph_event_retention_days", 2024-08-19T01:43:32.162 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2024-08-19T01:43:32.162 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.162 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-08-19T01:43:32.162 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "7", 2024-08-19T01:43:32.162 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.162 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.162 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:32.162 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Days to hold ceph event information within local cache", 2024-08-19T01:43:32.162 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.162 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.162 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.163 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.163 INFO:teuthology.orchestra.run.smithi083.stdout: "config_check_secs": { 2024-08-19T01:43:32.163 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "config_check_secs", 2024-08-19T01:43:32.163 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2024-08-19T01:43:32.163 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.163 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-08-19T01:43:32.163 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "10", 2024-08-19T01:43:32.163 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "10", 2024-08-19T01:43:32.163 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.163 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:32.163 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-08-19T01:43:32.163 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.164 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.164 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.164 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.164 INFO:teuthology.orchestra.run.smithi083.stdout: "log_level": { 2024-08-19T01:43:32.164 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_level", 2024-08-19T01:43:32.164 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-08-19T01:43:32.164 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.164 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-08-19T01:43:32.164 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-08-19T01:43:32.164 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.164 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.164 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-08-19T01:43:32.165 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-08-19T01:43:32.165 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-08-19T01:43:32.165 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-08-19T01:43:32.165 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-08-19T01:43:32.165 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-08-19T01:43:32.165 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-08-19T01:43:32.165 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-08-19T01:43:32.165 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-08-19T01:43:32.165 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.165 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.165 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.165 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.166 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster": { 2024-08-19T01:43:32.166 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster", 2024-08-19T01:43:32.166 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-08-19T01:43:32.166 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.166 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-08-19T01:43:32.166 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-08-19T01:43:32.166 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.166 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.166 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:32.166 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-08-19T01:43:32.166 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.167 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.167 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.167 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.167 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster_level": { 2024-08-19T01:43:32.167 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster_level", 2024-08-19T01:43:32.167 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-08-19T01:43:32.167 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.167 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-08-19T01:43:32.167 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "info", 2024-08-19T01:43:32.167 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.167 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.167 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-08-19T01:43:32.168 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-08-19T01:43:32.168 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-08-19T01:43:32.168 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-08-19T01:43:32.168 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-08-19T01:43:32.168 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-08-19T01:43:32.168 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-08-19T01:43:32.168 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-08-19T01:43:32.168 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-08-19T01:43:32.168 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.168 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.168 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.168 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.168 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_file": { 2024-08-19T01:43:32.168 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_file", 2024-08-19T01:43:32.169 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-08-19T01:43:32.169 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.169 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-08-19T01:43:32.169 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-08-19T01:43:32.169 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.169 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.169 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:32.169 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-08-19T01:43:32.169 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.169 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.169 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.170 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-08-19T01:43:32.170 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-08-19T01:43:32.170 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.170 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-08-19T01:43:32.170 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "localpool", 2024-08-19T01:43:32.170 INFO:teuthology.orchestra.run.smithi083.stdout: "can_run": true, 2024-08-19T01:43:32.170 INFO:teuthology.orchestra.run.smithi083.stdout: "error_string": "", 2024-08-19T01:43:32.170 INFO:teuthology.orchestra.run.smithi083.stdout: "module_options": { 2024-08-19T01:43:32.170 INFO:teuthology.orchestra.run.smithi083.stdout: "failure_domain": { 2024-08-19T01:43:32.170 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "failure_domain", 2024-08-19T01:43:32.170 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-08-19T01:43:32.170 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.170 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-08-19T01:43:32.171 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "host", 2024-08-19T01:43:32.171 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.171 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.171 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:32.171 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "failure domain for any created local pool", 2024-08-19T01:43:32.171 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-08-19T01:43:32.171 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.171 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.171 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.171 INFO:teuthology.orchestra.run.smithi083.stdout: "log_level": { 2024-08-19T01:43:32.171 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_level", 2024-08-19T01:43:32.171 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-08-19T01:43:32.171 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.171 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-08-19T01:43:32.171 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-08-19T01:43:32.172 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.172 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.172 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-08-19T01:43:32.172 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-08-19T01:43:32.172 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-08-19T01:43:32.172 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-08-19T01:43:32.172 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-08-19T01:43:32.172 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-08-19T01:43:32.172 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-08-19T01:43:32.172 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-08-19T01:43:32.172 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-08-19T01:43:32.172 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.172 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.172 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.173 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.173 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster": { 2024-08-19T01:43:32.173 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster", 2024-08-19T01:43:32.173 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-08-19T01:43:32.173 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.173 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-08-19T01:43:32.173 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-08-19T01:43:32.173 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.173 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.173 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:32.173 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-08-19T01:43:32.173 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.173 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.173 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.174 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.174 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster_level": { 2024-08-19T01:43:32.174 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster_level", 2024-08-19T01:43:32.174 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-08-19T01:43:32.174 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.174 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-08-19T01:43:32.174 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "info", 2024-08-19T01:43:32.174 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.174 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.174 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-08-19T01:43:32.174 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-08-19T01:43:32.174 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-08-19T01:43:32.174 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-08-19T01:43:32.175 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-08-19T01:43:32.175 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-08-19T01:43:32.175 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-08-19T01:43:32.175 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-08-19T01:43:32.175 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-08-19T01:43:32.175 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.175 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.175 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.175 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.175 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_file": { 2024-08-19T01:43:32.175 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_file", 2024-08-19T01:43:32.175 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-08-19T01:43:32.175 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.175 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-08-19T01:43:32.176 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-08-19T01:43:32.176 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.176 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.176 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:32.176 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-08-19T01:43:32.176 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.176 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.176 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.176 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.176 INFO:teuthology.orchestra.run.smithi083.stdout: "min_size": { 2024-08-19T01:43:32.176 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "min_size", 2024-08-19T01:43:32.176 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2024-08-19T01:43:32.176 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.176 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-08-19T01:43:32.177 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-08-19T01:43:32.177 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.177 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.177 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:32.177 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "default min_size for any created local pool", 2024-08-19T01:43:32.177 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-08-19T01:43:32.177 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.177 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.177 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.177 INFO:teuthology.orchestra.run.smithi083.stdout: "num_rep": { 2024-08-19T01:43:32.177 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "num_rep", 2024-08-19T01:43:32.177 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2024-08-19T01:43:32.177 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.177 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-08-19T01:43:32.177 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "3", 2024-08-19T01:43:32.177 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.178 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.178 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:32.178 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "default replica count for any created local pool", 2024-08-19T01:43:32.178 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.178 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.178 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.178 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.178 INFO:teuthology.orchestra.run.smithi083.stdout: "pg_num": { 2024-08-19T01:43:32.178 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "pg_num", 2024-08-19T01:43:32.178 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2024-08-19T01:43:32.178 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.178 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-08-19T01:43:32.178 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "128", 2024-08-19T01:43:32.179 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.179 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.179 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:32.179 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "default pg_num for any created local pool", 2024-08-19T01:43:32.179 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.179 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.179 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.179 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.179 INFO:teuthology.orchestra.run.smithi083.stdout: "prefix": { 2024-08-19T01:43:32.179 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "prefix", 2024-08-19T01:43:32.179 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-08-19T01:43:32.179 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.179 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-08-19T01:43:32.179 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-08-19T01:43:32.180 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.180 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.180 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:32.180 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "name prefix for any created local pool", 2024-08-19T01:43:32.180 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.180 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.180 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.180 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.180 INFO:teuthology.orchestra.run.smithi083.stdout: "subtree": { 2024-08-19T01:43:32.180 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "subtree", 2024-08-19T01:43:32.180 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-08-19T01:43:32.180 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.180 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-08-19T01:43:32.180 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "rack", 2024-08-19T01:43:32.180 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.181 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.181 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:32.181 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "CRUSH level for which to create a local pool", 2024-08-19T01:43:32.181 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-08-19T01:43:32.181 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.181 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.181 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-08-19T01:43:32.181 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-08-19T01:43:32.181 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.181 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-08-19T01:43:32.181 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "mds_autoscaler", 2024-08-19T01:43:32.181 INFO:teuthology.orchestra.run.smithi083.stdout: "can_run": true, 2024-08-19T01:43:32.181 INFO:teuthology.orchestra.run.smithi083.stdout: "error_string": "", 2024-08-19T01:43:32.181 INFO:teuthology.orchestra.run.smithi083.stdout: "module_options": { 2024-08-19T01:43:32.181 INFO:teuthology.orchestra.run.smithi083.stdout: "log_level": { 2024-08-19T01:43:32.182 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_level", 2024-08-19T01:43:32.182 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-08-19T01:43:32.182 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.182 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-08-19T01:43:32.182 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-08-19T01:43:32.182 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.182 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.182 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-08-19T01:43:32.182 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-08-19T01:43:32.182 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-08-19T01:43:32.182 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-08-19T01:43:32.182 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-08-19T01:43:32.183 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-08-19T01:43:32.183 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-08-19T01:43:32.183 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-08-19T01:43:32.183 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-08-19T01:43:32.183 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.183 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.183 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.183 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.183 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster": { 2024-08-19T01:43:32.183 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster", 2024-08-19T01:43:32.183 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-08-19T01:43:32.183 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.183 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-08-19T01:43:32.184 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-08-19T01:43:32.184 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.184 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.184 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:32.184 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-08-19T01:43:32.184 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.184 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.184 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.184 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.184 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster_level": { 2024-08-19T01:43:32.184 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster_level", 2024-08-19T01:43:32.184 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-08-19T01:43:32.184 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.184 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-08-19T01:43:32.185 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "info", 2024-08-19T01:43:32.185 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.185 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.185 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-08-19T01:43:32.185 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-08-19T01:43:32.185 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-08-19T01:43:32.185 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-08-19T01:43:32.185 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-08-19T01:43:32.185 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-08-19T01:43:32.185 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-08-19T01:43:32.185 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-08-19T01:43:32.185 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-08-19T01:43:32.185 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.185 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.186 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.186 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.186 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_file": { 2024-08-19T01:43:32.186 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_file", 2024-08-19T01:43:32.186 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-08-19T01:43:32.186 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.186 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-08-19T01:43:32.186 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-08-19T01:43:32.186 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.186 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.186 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:32.186 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-08-19T01:43:32.186 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.187 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.187 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.187 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-08-19T01:43:32.187 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-08-19T01:43:32.187 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.187 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-08-19T01:43:32.187 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "mirroring", 2024-08-19T01:43:32.187 INFO:teuthology.orchestra.run.smithi083.stdout: "can_run": true, 2024-08-19T01:43:32.187 INFO:teuthology.orchestra.run.smithi083.stdout: "error_string": "", 2024-08-19T01:43:32.187 INFO:teuthology.orchestra.run.smithi083.stdout: "module_options": { 2024-08-19T01:43:32.187 INFO:teuthology.orchestra.run.smithi083.stdout: "log_level": { 2024-08-19T01:43:32.187 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_level", 2024-08-19T01:43:32.187 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-08-19T01:43:32.187 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.188 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-08-19T01:43:32.188 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-08-19T01:43:32.188 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.188 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.188 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-08-19T01:43:32.188 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-08-19T01:43:32.188 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-08-19T01:43:32.188 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-08-19T01:43:32.188 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-08-19T01:43:32.188 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-08-19T01:43:32.188 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-08-19T01:43:32.188 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-08-19T01:43:32.188 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-08-19T01:43:32.188 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.188 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.189 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.189 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.189 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster": { 2024-08-19T01:43:32.189 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster", 2024-08-19T01:43:32.189 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-08-19T01:43:32.189 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.189 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-08-19T01:43:32.189 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-08-19T01:43:32.189 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.189 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.189 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:32.189 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-08-19T01:43:32.190 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.190 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.190 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.190 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.190 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster_level": { 2024-08-19T01:43:32.190 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster_level", 2024-08-19T01:43:32.190 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-08-19T01:43:32.190 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.190 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-08-19T01:43:32.190 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "info", 2024-08-19T01:43:32.190 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.190 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.190 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-08-19T01:43:32.191 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-08-19T01:43:32.191 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-08-19T01:43:32.191 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-08-19T01:43:32.191 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-08-19T01:43:32.191 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-08-19T01:43:32.191 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-08-19T01:43:32.191 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-08-19T01:43:32.191 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-08-19T01:43:32.191 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.191 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.191 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.191 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.191 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_file": { 2024-08-19T01:43:32.192 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_file", 2024-08-19T01:43:32.192 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-08-19T01:43:32.192 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.192 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-08-19T01:43:32.192 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-08-19T01:43:32.192 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.192 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.192 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:32.192 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-08-19T01:43:32.192 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.192 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.192 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.192 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-08-19T01:43:32.192 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-08-19T01:43:32.193 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.193 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-08-19T01:43:32.193 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "nfs", 2024-08-19T01:43:32.193 INFO:teuthology.orchestra.run.smithi083.stdout: "can_run": true, 2024-08-19T01:43:32.193 INFO:teuthology.orchestra.run.smithi083.stdout: "error_string": "", 2024-08-19T01:43:32.193 INFO:teuthology.orchestra.run.smithi083.stdout: "module_options": { 2024-08-19T01:43:32.193 INFO:teuthology.orchestra.run.smithi083.stdout: "log_level": { 2024-08-19T01:43:32.193 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_level", 2024-08-19T01:43:32.193 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-08-19T01:43:32.193 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.193 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-08-19T01:43:32.193 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-08-19T01:43:32.193 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.193 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.194 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-08-19T01:43:32.194 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-08-19T01:43:32.194 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-08-19T01:43:32.194 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-08-19T01:43:32.194 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-08-19T01:43:32.194 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-08-19T01:43:32.194 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-08-19T01:43:32.194 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-08-19T01:43:32.194 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-08-19T01:43:32.194 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.194 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.194 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.194 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.194 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster": { 2024-08-19T01:43:32.195 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster", 2024-08-19T01:43:32.195 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-08-19T01:43:32.195 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.195 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-08-19T01:43:32.195 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-08-19T01:43:32.195 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.195 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.195 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:32.195 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-08-19T01:43:32.195 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.195 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.195 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.195 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.195 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster_level": { 2024-08-19T01:43:32.196 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster_level", 2024-08-19T01:43:32.196 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-08-19T01:43:32.196 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.196 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-08-19T01:43:32.196 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "info", 2024-08-19T01:43:32.196 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.196 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.196 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-08-19T01:43:32.196 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-08-19T01:43:32.196 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-08-19T01:43:32.196 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-08-19T01:43:32.196 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-08-19T01:43:32.196 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-08-19T01:43:32.196 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-08-19T01:43:32.196 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-08-19T01:43:32.197 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-08-19T01:43:32.197 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.197 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.197 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.197 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.197 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_file": { 2024-08-19T01:43:32.197 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_file", 2024-08-19T01:43:32.197 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-08-19T01:43:32.197 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.197 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-08-19T01:43:32.197 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-08-19T01:43:32.197 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.197 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.197 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:32.198 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-08-19T01:43:32.198 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.198 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.198 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.198 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-08-19T01:43:32.198 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-08-19T01:43:32.198 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.198 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-08-19T01:43:32.198 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "orchestrator", 2024-08-19T01:43:32.198 INFO:teuthology.orchestra.run.smithi083.stdout: "can_run": true, 2024-08-19T01:43:32.198 INFO:teuthology.orchestra.run.smithi083.stdout: "error_string": "", 2024-08-19T01:43:32.198 INFO:teuthology.orchestra.run.smithi083.stdout: "module_options": { 2024-08-19T01:43:32.198 INFO:teuthology.orchestra.run.smithi083.stdout: "fail_fs": { 2024-08-19T01:43:32.199 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "fail_fs", 2024-08-19T01:43:32.199 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-08-19T01:43:32.199 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.199 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-08-19T01:43:32.199 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-08-19T01:43:32.199 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.199 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.199 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:32.199 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-08-19T01:43:32.199 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.199 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.199 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.199 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.199 INFO:teuthology.orchestra.run.smithi083.stdout: "log_level": { 2024-08-19T01:43:32.200 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_level", 2024-08-19T01:43:32.200 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-08-19T01:43:32.200 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.200 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-08-19T01:43:32.200 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-08-19T01:43:32.200 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.200 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.200 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-08-19T01:43:32.200 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-08-19T01:43:32.200 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-08-19T01:43:32.200 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-08-19T01:43:32.200 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-08-19T01:43:32.200 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-08-19T01:43:32.200 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-08-19T01:43:32.200 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-08-19T01:43:32.201 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-08-19T01:43:32.201 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.201 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.201 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.201 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.201 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster": { 2024-08-19T01:43:32.201 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster", 2024-08-19T01:43:32.201 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-08-19T01:43:32.201 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.201 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-08-19T01:43:32.201 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-08-19T01:43:32.201 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.201 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.201 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:32.202 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-08-19T01:43:32.202 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.202 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.202 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.202 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.202 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster_level": { 2024-08-19T01:43:32.202 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster_level", 2024-08-19T01:43:32.202 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-08-19T01:43:32.202 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.202 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-08-19T01:43:32.202 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "info", 2024-08-19T01:43:32.202 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.202 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.203 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-08-19T01:43:32.203 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-08-19T01:43:32.203 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-08-19T01:43:32.203 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-08-19T01:43:32.203 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-08-19T01:43:32.203 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-08-19T01:43:32.203 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-08-19T01:43:32.203 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-08-19T01:43:32.203 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-08-19T01:43:32.203 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.203 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.203 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.203 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.203 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_file": { 2024-08-19T01:43:32.203 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_file", 2024-08-19T01:43:32.204 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-08-19T01:43:32.204 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.204 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-08-19T01:43:32.204 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-08-19T01:43:32.204 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.204 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.204 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:32.204 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-08-19T01:43:32.204 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.204 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.204 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.204 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.204 INFO:teuthology.orchestra.run.smithi083.stdout: "orchestrator": { 2024-08-19T01:43:32.204 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "orchestrator", 2024-08-19T01:43:32.204 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-08-19T01:43:32.204 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.205 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-08-19T01:43:32.205 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-08-19T01:43:32.205 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.205 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.205 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-08-19T01:43:32.205 INFO:teuthology.orchestra.run.smithi083.stdout: "cephadm", 2024-08-19T01:43:32.205 INFO:teuthology.orchestra.run.smithi083.stdout: "rook", 2024-08-19T01:43:32.205 INFO:teuthology.orchestra.run.smithi083.stdout: "test_orchestrator" 2024-08-19T01:43:32.205 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-08-19T01:43:32.205 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Orchestrator backend", 2024-08-19T01:43:32.205 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.205 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.206 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.206 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-08-19T01:43:32.206 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-08-19T01:43:32.206 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.206 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-08-19T01:43:32.206 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "osd_perf_query", 2024-08-19T01:43:32.206 INFO:teuthology.orchestra.run.smithi083.stdout: "can_run": true, 2024-08-19T01:43:32.206 INFO:teuthology.orchestra.run.smithi083.stdout: "error_string": "", 2024-08-19T01:43:32.206 INFO:teuthology.orchestra.run.smithi083.stdout: "module_options": { 2024-08-19T01:43:32.206 INFO:teuthology.orchestra.run.smithi083.stdout: "log_level": { 2024-08-19T01:43:32.206 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_level", 2024-08-19T01:43:32.206 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-08-19T01:43:32.207 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.207 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-08-19T01:43:32.207 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-08-19T01:43:32.207 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.207 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.207 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-08-19T01:43:32.207 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-08-19T01:43:32.207 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-08-19T01:43:32.207 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-08-19T01:43:32.207 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-08-19T01:43:32.207 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-08-19T01:43:32.207 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-08-19T01:43:32.207 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-08-19T01:43:32.208 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-08-19T01:43:32.208 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.208 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.208 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.208 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.208 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster": { 2024-08-19T01:43:32.208 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster", 2024-08-19T01:43:32.208 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-08-19T01:43:32.208 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.208 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-08-19T01:43:32.208 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-08-19T01:43:32.208 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.208 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.208 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:32.209 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-08-19T01:43:32.209 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.209 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.209 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.209 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.209 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster_level": { 2024-08-19T01:43:32.209 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster_level", 2024-08-19T01:43:32.209 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-08-19T01:43:32.209 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.209 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-08-19T01:43:32.209 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "info", 2024-08-19T01:43:32.209 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.209 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.209 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-08-19T01:43:32.210 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-08-19T01:43:32.210 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-08-19T01:43:32.210 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-08-19T01:43:32.210 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-08-19T01:43:32.210 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-08-19T01:43:32.210 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-08-19T01:43:32.210 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-08-19T01:43:32.210 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-08-19T01:43:32.210 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.210 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.210 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.210 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.211 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_file": { 2024-08-19T01:43:32.211 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_file", 2024-08-19T01:43:32.211 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-08-19T01:43:32.211 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.211 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-08-19T01:43:32.211 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-08-19T01:43:32.211 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.211 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.211 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:32.211 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-08-19T01:43:32.211 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.211 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.211 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.212 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-08-19T01:43:32.212 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-08-19T01:43:32.212 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.212 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-08-19T01:43:32.212 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "osd_support", 2024-08-19T01:43:32.212 INFO:teuthology.orchestra.run.smithi083.stdout: "can_run": true, 2024-08-19T01:43:32.212 INFO:teuthology.orchestra.run.smithi083.stdout: "error_string": "", 2024-08-19T01:43:32.212 INFO:teuthology.orchestra.run.smithi083.stdout: "module_options": { 2024-08-19T01:43:32.212 INFO:teuthology.orchestra.run.smithi083.stdout: "log_level": { 2024-08-19T01:43:32.212 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_level", 2024-08-19T01:43:32.212 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-08-19T01:43:32.212 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.212 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-08-19T01:43:32.213 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-08-19T01:43:32.213 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.213 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.213 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-08-19T01:43:32.213 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-08-19T01:43:32.213 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-08-19T01:43:32.213 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-08-19T01:43:32.213 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-08-19T01:43:32.213 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-08-19T01:43:32.213 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-08-19T01:43:32.213 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-08-19T01:43:32.213 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-08-19T01:43:32.213 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.213 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.214 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.214 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.214 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster": { 2024-08-19T01:43:32.214 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster", 2024-08-19T01:43:32.214 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-08-19T01:43:32.214 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.214 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-08-19T01:43:32.214 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-08-19T01:43:32.214 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.214 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.214 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:32.214 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-08-19T01:43:32.214 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.214 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.215 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.215 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.215 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster_level": { 2024-08-19T01:43:32.215 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster_level", 2024-08-19T01:43:32.215 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-08-19T01:43:32.215 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.215 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-08-19T01:43:32.215 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "info", 2024-08-19T01:43:32.215 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.215 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.215 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-08-19T01:43:32.215 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-08-19T01:43:32.215 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-08-19T01:43:32.215 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-08-19T01:43:32.215 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-08-19T01:43:32.216 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-08-19T01:43:32.216 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-08-19T01:43:32.216 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-08-19T01:43:32.216 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-08-19T01:43:32.216 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.216 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.216 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.216 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.216 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_file": { 2024-08-19T01:43:32.216 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_file", 2024-08-19T01:43:32.216 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-08-19T01:43:32.216 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.216 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-08-19T01:43:32.216 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-08-19T01:43:32.216 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.217 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.217 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:32.217 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-08-19T01:43:32.217 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.217 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.217 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.217 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-08-19T01:43:32.217 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-08-19T01:43:32.217 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.217 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-08-19T01:43:32.217 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "pg_autoscaler", 2024-08-19T01:43:32.217 INFO:teuthology.orchestra.run.smithi083.stdout: "can_run": true, 2024-08-19T01:43:32.217 INFO:teuthology.orchestra.run.smithi083.stdout: "error_string": "", 2024-08-19T01:43:32.217 INFO:teuthology.orchestra.run.smithi083.stdout: "module_options": { 2024-08-19T01:43:32.217 INFO:teuthology.orchestra.run.smithi083.stdout: "log_level": { 2024-08-19T01:43:32.217 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_level", 2024-08-19T01:43:32.218 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-08-19T01:43:32.218 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.218 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-08-19T01:43:32.218 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-08-19T01:43:32.218 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.218 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.218 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-08-19T01:43:32.218 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-08-19T01:43:32.218 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-08-19T01:43:32.218 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-08-19T01:43:32.218 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-08-19T01:43:32.218 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-08-19T01:43:32.218 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-08-19T01:43:32.218 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-08-19T01:43:32.218 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-08-19T01:43:32.218 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.219 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.219 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.219 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.219 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster": { 2024-08-19T01:43:32.219 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster", 2024-08-19T01:43:32.219 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-08-19T01:43:32.219 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.219 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-08-19T01:43:32.219 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-08-19T01:43:32.219 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.219 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.219 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:32.219 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-08-19T01:43:32.219 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.219 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.219 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.220 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.220 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster_level": { 2024-08-19T01:43:32.220 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster_level", 2024-08-19T01:43:32.220 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-08-19T01:43:32.220 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.220 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-08-19T01:43:32.220 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "info", 2024-08-19T01:43:32.220 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.220 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.220 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-08-19T01:43:32.220 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-08-19T01:43:32.220 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-08-19T01:43:32.220 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-08-19T01:43:32.220 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-08-19T01:43:32.221 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-08-19T01:43:32.221 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-08-19T01:43:32.221 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-08-19T01:43:32.221 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-08-19T01:43:32.221 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.221 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.221 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.221 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.221 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_file": { 2024-08-19T01:43:32.221 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_file", 2024-08-19T01:43:32.221 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-08-19T01:43:32.221 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.221 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-08-19T01:43:32.221 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-08-19T01:43:32.221 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.222 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.222 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:32.222 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-08-19T01:43:32.222 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.222 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.222 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.222 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.222 INFO:teuthology.orchestra.run.smithi083.stdout: "sleep_interval": { 2024-08-19T01:43:32.222 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "sleep_interval", 2024-08-19T01:43:32.222 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "secs", 2024-08-19T01:43:32.222 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.222 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-08-19T01:43:32.222 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "60", 2024-08-19T01:43:32.222 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.223 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.223 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:32.223 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-08-19T01:43:32.223 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.223 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.223 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.223 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.223 INFO:teuthology.orchestra.run.smithi083.stdout: "threshold": { 2024-08-19T01:43:32.223 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "threshold", 2024-08-19T01:43:32.223 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "float", 2024-08-19T01:43:32.223 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.223 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-08-19T01:43:32.223 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "3.0", 2024-08-19T01:43:32.223 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "1.0", 2024-08-19T01:43:32.223 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.224 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:32.224 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "scaling threshold", 2024-08-19T01:43:32.224 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-08-19T01:43:32.224 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.224 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.224 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-08-19T01:43:32.224 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-08-19T01:43:32.224 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.224 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-08-19T01:43:32.224 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "progress", 2024-08-19T01:43:32.224 INFO:teuthology.orchestra.run.smithi083.stdout: "can_run": true, 2024-08-19T01:43:32.224 INFO:teuthology.orchestra.run.smithi083.stdout: "error_string": "", 2024-08-19T01:43:32.224 INFO:teuthology.orchestra.run.smithi083.stdout: "module_options": { 2024-08-19T01:43:32.224 INFO:teuthology.orchestra.run.smithi083.stdout: "allow_pg_recovery_event": { 2024-08-19T01:43:32.224 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "allow_pg_recovery_event", 2024-08-19T01:43:32.225 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-08-19T01:43:32.225 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.225 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-08-19T01:43:32.225 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-08-19T01:43:32.225 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.225 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.225 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:32.225 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "allow the module to show pg recovery progress", 2024-08-19T01:43:32.225 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.225 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.225 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.225 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.225 INFO:teuthology.orchestra.run.smithi083.stdout: "enabled": { 2024-08-19T01:43:32.225 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "enabled", 2024-08-19T01:43:32.225 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-08-19T01:43:32.226 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.226 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-08-19T01:43:32.226 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "True", 2024-08-19T01:43:32.226 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.226 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.226 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:32.226 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-08-19T01:43:32.226 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.226 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.226 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.226 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.226 INFO:teuthology.orchestra.run.smithi083.stdout: "log_level": { 2024-08-19T01:43:32.226 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_level", 2024-08-19T01:43:32.226 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-08-19T01:43:32.227 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.227 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-08-19T01:43:32.227 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-08-19T01:43:32.227 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.227 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.227 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-08-19T01:43:32.227 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-08-19T01:43:32.227 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-08-19T01:43:32.227 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-08-19T01:43:32.227 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-08-19T01:43:32.227 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-08-19T01:43:32.227 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-08-19T01:43:32.227 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-08-19T01:43:32.227 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-08-19T01:43:32.228 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.228 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.228 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.228 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.228 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster": { 2024-08-19T01:43:32.228 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster", 2024-08-19T01:43:32.228 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-08-19T01:43:32.228 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.228 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-08-19T01:43:32.228 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-08-19T01:43:32.228 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.228 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.228 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:32.228 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-08-19T01:43:32.229 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.229 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.229 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.229 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.229 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster_level": { 2024-08-19T01:43:32.229 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster_level", 2024-08-19T01:43:32.229 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-08-19T01:43:32.229 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.229 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-08-19T01:43:32.229 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "info", 2024-08-19T01:43:32.229 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.229 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.229 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-08-19T01:43:32.229 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-08-19T01:43:32.229 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-08-19T01:43:32.230 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-08-19T01:43:32.230 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-08-19T01:43:32.230 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-08-19T01:43:32.230 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-08-19T01:43:32.230 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-08-19T01:43:32.230 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-08-19T01:43:32.230 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.230 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.230 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.230 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.230 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_file": { 2024-08-19T01:43:32.230 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_file", 2024-08-19T01:43:32.230 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-08-19T01:43:32.230 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.230 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-08-19T01:43:32.231 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-08-19T01:43:32.231 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.231 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.231 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:32.231 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-08-19T01:43:32.231 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.231 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.231 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.231 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.231 INFO:teuthology.orchestra.run.smithi083.stdout: "max_completed_events": { 2024-08-19T01:43:32.231 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "max_completed_events", 2024-08-19T01:43:32.231 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2024-08-19T01:43:32.231 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.231 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-08-19T01:43:32.232 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "50", 2024-08-19T01:43:32.232 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.232 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.232 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:32.232 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "number of past completed events to remember", 2024-08-19T01:43:32.232 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.232 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.232 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.232 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.232 INFO:teuthology.orchestra.run.smithi083.stdout: "sleep_interval": { 2024-08-19T01:43:32.232 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "sleep_interval", 2024-08-19T01:43:32.232 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "secs", 2024-08-19T01:43:32.232 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.232 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-08-19T01:43:32.232 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "5", 2024-08-19T01:43:32.233 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.233 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.233 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:32.233 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "how long the module is going to sleep", 2024-08-19T01:43:32.233 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.233 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.233 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.233 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-08-19T01:43:32.233 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-08-19T01:43:32.233 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.233 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-08-19T01:43:32.233 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "prometheus", 2024-08-19T01:43:32.233 INFO:teuthology.orchestra.run.smithi083.stdout: "can_run": true, 2024-08-19T01:43:32.233 INFO:teuthology.orchestra.run.smithi083.stdout: "error_string": "", 2024-08-19T01:43:32.234 INFO:teuthology.orchestra.run.smithi083.stdout: "module_options": { 2024-08-19T01:43:32.234 INFO:teuthology.orchestra.run.smithi083.stdout: "cache": { 2024-08-19T01:43:32.234 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "cache", 2024-08-19T01:43:32.234 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-08-19T01:43:32.234 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.234 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-08-19T01:43:32.234 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "True", 2024-08-19T01:43:32.234 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.234 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.234 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:32.234 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-08-19T01:43:32.234 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.234 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.235 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.235 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.235 INFO:teuthology.orchestra.run.smithi083.stdout: "exclude_perf_counters": { 2024-08-19T01:43:32.235 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "exclude_perf_counters", 2024-08-19T01:43:32.235 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-08-19T01:43:32.235 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.235 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-08-19T01:43:32.235 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "True", 2024-08-19T01:43:32.235 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.235 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.235 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:32.235 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-08-19T01:43:32.235 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-08-19T01:43:32.236 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.236 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.236 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.236 INFO:teuthology.orchestra.run.smithi083.stdout: "log_level": { 2024-08-19T01:43:32.236 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_level", 2024-08-19T01:43:32.236 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-08-19T01:43:32.236 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.236 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-08-19T01:43:32.236 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-08-19T01:43:32.236 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.236 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.236 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-08-19T01:43:32.236 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-08-19T01:43:32.236 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-08-19T01:43:32.237 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-08-19T01:43:32.237 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-08-19T01:43:32.237 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-08-19T01:43:32.237 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-08-19T01:43:32.237 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-08-19T01:43:32.237 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-08-19T01:43:32.237 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.237 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.237 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.237 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.237 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster": { 2024-08-19T01:43:32.237 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster", 2024-08-19T01:43:32.237 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-08-19T01:43:32.237 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.237 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-08-19T01:43:32.238 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-08-19T01:43:32.238 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.238 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.238 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:32.238 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-08-19T01:43:32.238 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.238 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.238 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.238 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.238 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster_level": { 2024-08-19T01:43:32.238 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster_level", 2024-08-19T01:43:32.238 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-08-19T01:43:32.238 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.239 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-08-19T01:43:32.239 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "info", 2024-08-19T01:43:32.239 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.239 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.239 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-08-19T01:43:32.239 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-08-19T01:43:32.239 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-08-19T01:43:32.239 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-08-19T01:43:32.239 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-08-19T01:43:32.239 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-08-19T01:43:32.239 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-08-19T01:43:32.239 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-08-19T01:43:32.239 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-08-19T01:43:32.239 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.240 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.240 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.240 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.240 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_file": { 2024-08-19T01:43:32.240 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_file", 2024-08-19T01:43:32.240 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-08-19T01:43:32.240 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.240 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-08-19T01:43:32.240 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-08-19T01:43:32.240 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.240 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.240 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:32.240 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-08-19T01:43:32.240 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.241 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.241 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.241 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.241 INFO:teuthology.orchestra.run.smithi083.stdout: "rbd_stats_pools": { 2024-08-19T01:43:32.241 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "rbd_stats_pools", 2024-08-19T01:43:32.241 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-08-19T01:43:32.241 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.241 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-08-19T01:43:32.241 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-08-19T01:43:32.241 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.241 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.241 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:32.241 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-08-19T01:43:32.242 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.242 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.242 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.242 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.242 INFO:teuthology.orchestra.run.smithi083.stdout: "rbd_stats_pools_refresh_interval": { 2024-08-19T01:43:32.242 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-08-19T01:43:32.242 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2024-08-19T01:43:32.242 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.242 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-08-19T01:43:32.242 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "300", 2024-08-19T01:43:32.242 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.242 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.242 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:32.243 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-08-19T01:43:32.243 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.243 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.243 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.243 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.243 INFO:teuthology.orchestra.run.smithi083.stdout: "scrape_interval": { 2024-08-19T01:43:32.243 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "scrape_interval", 2024-08-19T01:43:32.243 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "float", 2024-08-19T01:43:32.243 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.243 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-08-19T01:43:32.243 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "15.0", 2024-08-19T01:43:32.243 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.243 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.244 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:32.244 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-08-19T01:43:32.244 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.244 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.244 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.244 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.244 INFO:teuthology.orchestra.run.smithi083.stdout: "server_addr": { 2024-08-19T01:43:32.244 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "server_addr", 2024-08-19T01:43:32.244 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-08-19T01:43:32.244 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.244 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-08-19T01:43:32.244 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "::", 2024-08-19T01:43:32.244 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.244 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.245 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:32.245 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-08-19T01:43:32.245 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.245 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.245 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.245 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.245 INFO:teuthology.orchestra.run.smithi083.stdout: "server_port": { 2024-08-19T01:43:32.245 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "server_port", 2024-08-19T01:43:32.245 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2024-08-19T01:43:32.245 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.245 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-08-19T01:43:32.245 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "9283", 2024-08-19T01:43:32.245 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.245 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.246 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:32.246 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-08-19T01:43:32.246 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.246 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.246 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.246 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.246 INFO:teuthology.orchestra.run.smithi083.stdout: "stale_cache_strategy": { 2024-08-19T01:43:32.246 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "stale_cache_strategy", 2024-08-19T01:43:32.246 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-08-19T01:43:32.246 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.246 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-08-19T01:43:32.246 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "log", 2024-08-19T01:43:32.247 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.247 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.247 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:32.247 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-08-19T01:43:32.247 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.247 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.247 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.247 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.247 INFO:teuthology.orchestra.run.smithi083.stdout: "standby_behaviour": { 2024-08-19T01:43:32.247 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "standby_behaviour", 2024-08-19T01:43:32.247 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-08-19T01:43:32.247 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.247 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-08-19T01:43:32.248 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "default", 2024-08-19T01:43:32.248 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.248 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.248 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-08-19T01:43:32.248 INFO:teuthology.orchestra.run.smithi083.stdout: "default", 2024-08-19T01:43:32.248 INFO:teuthology.orchestra.run.smithi083.stdout: "error" 2024-08-19T01:43:32.248 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-08-19T01:43:32.248 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-08-19T01:43:32.248 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.248 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.248 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.248 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.248 INFO:teuthology.orchestra.run.smithi083.stdout: "standby_error_status_code": { 2024-08-19T01:43:32.248 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "standby_error_status_code", 2024-08-19T01:43:32.249 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2024-08-19T01:43:32.249 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.249 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-08-19T01:43:32.249 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "500", 2024-08-19T01:43:32.249 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "400", 2024-08-19T01:43:32.249 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "599", 2024-08-19T01:43:32.249 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:32.249 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-08-19T01:43:32.249 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.249 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.249 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.249 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-08-19T01:43:32.249 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-08-19T01:43:32.249 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.249 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-08-19T01:43:32.250 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "rbd_support", 2024-08-19T01:43:32.250 INFO:teuthology.orchestra.run.smithi083.stdout: "can_run": true, 2024-08-19T01:43:32.250 INFO:teuthology.orchestra.run.smithi083.stdout: "error_string": "", 2024-08-19T01:43:32.250 INFO:teuthology.orchestra.run.smithi083.stdout: "module_options": { 2024-08-19T01:43:32.250 INFO:teuthology.orchestra.run.smithi083.stdout: "log_level": { 2024-08-19T01:43:32.250 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_level", 2024-08-19T01:43:32.250 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-08-19T01:43:32.250 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.250 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-08-19T01:43:32.250 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-08-19T01:43:32.250 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.250 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.251 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-08-19T01:43:32.251 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-08-19T01:43:32.251 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-08-19T01:43:32.251 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-08-19T01:43:32.251 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-08-19T01:43:32.251 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-08-19T01:43:32.251 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-08-19T01:43:32.251 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-08-19T01:43:32.251 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-08-19T01:43:32.251 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.251 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.251 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.251 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.251 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster": { 2024-08-19T01:43:32.252 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster", 2024-08-19T01:43:32.252 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-08-19T01:43:32.252 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.252 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-08-19T01:43:32.252 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-08-19T01:43:32.252 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.252 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.252 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:32.252 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-08-19T01:43:32.252 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.252 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.252 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.252 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.253 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster_level": { 2024-08-19T01:43:32.253 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster_level", 2024-08-19T01:43:32.253 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-08-19T01:43:32.253 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.253 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-08-19T01:43:32.253 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "info", 2024-08-19T01:43:32.253 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.253 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.253 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-08-19T01:43:32.253 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-08-19T01:43:32.253 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-08-19T01:43:32.253 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-08-19T01:43:32.254 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-08-19T01:43:32.254 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-08-19T01:43:32.254 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-08-19T01:43:32.254 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-08-19T01:43:32.254 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-08-19T01:43:32.254 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.254 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.254 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.254 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.254 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_file": { 2024-08-19T01:43:32.254 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_file", 2024-08-19T01:43:32.254 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-08-19T01:43:32.255 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.255 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-08-19T01:43:32.255 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-08-19T01:43:32.255 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.255 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.255 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:32.255 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-08-19T01:43:32.255 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.255 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.255 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.255 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.255 INFO:teuthology.orchestra.run.smithi083.stdout: "max_concurrent_snap_create": { 2024-08-19T01:43:32.255 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "max_concurrent_snap_create", 2024-08-19T01:43:32.255 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2024-08-19T01:43:32.256 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.256 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-08-19T01:43:32.256 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "10", 2024-08-19T01:43:32.256 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.256 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.256 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:32.256 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-08-19T01:43:32.256 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.256 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.256 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.256 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.256 INFO:teuthology.orchestra.run.smithi083.stdout: "mirror_snapshot_schedule": { 2024-08-19T01:43:32.256 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "mirror_snapshot_schedule", 2024-08-19T01:43:32.256 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-08-19T01:43:32.257 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.257 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-08-19T01:43:32.257 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-08-19T01:43:32.257 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.257 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.257 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:32.257 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-08-19T01:43:32.257 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.257 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.257 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.257 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.257 INFO:teuthology.orchestra.run.smithi083.stdout: "trash_purge_schedule": { 2024-08-19T01:43:32.257 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "trash_purge_schedule", 2024-08-19T01:43:32.257 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-08-19T01:43:32.258 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.258 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-08-19T01:43:32.258 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-08-19T01:43:32.258 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.258 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.258 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:32.258 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-08-19T01:43:32.258 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.258 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.258 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.258 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-08-19T01:43:32.258 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-08-19T01:43:32.258 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.259 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-08-19T01:43:32.259 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "restful", 2024-08-19T01:43:32.259 INFO:teuthology.orchestra.run.smithi083.stdout: "can_run": true, 2024-08-19T01:43:32.259 INFO:teuthology.orchestra.run.smithi083.stdout: "error_string": "", 2024-08-19T01:43:32.259 INFO:teuthology.orchestra.run.smithi083.stdout: "module_options": { 2024-08-19T01:43:32.259 INFO:teuthology.orchestra.run.smithi083.stdout: "enable_auth": { 2024-08-19T01:43:32.259 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "enable_auth", 2024-08-19T01:43:32.259 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-08-19T01:43:32.259 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.259 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-08-19T01:43:32.259 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "True", 2024-08-19T01:43:32.259 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.259 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.260 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:32.260 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-08-19T01:43:32.260 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.260 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.260 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.260 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.260 INFO:teuthology.orchestra.run.smithi083.stdout: "key_file": { 2024-08-19T01:43:32.260 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "key_file", 2024-08-19T01:43:32.260 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-08-19T01:43:32.260 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.260 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-08-19T01:43:32.260 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-08-19T01:43:32.260 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.260 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.261 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:32.261 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-08-19T01:43:32.261 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.261 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.261 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.261 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.261 INFO:teuthology.orchestra.run.smithi083.stdout: "log_level": { 2024-08-19T01:43:32.261 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_level", 2024-08-19T01:43:32.261 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-08-19T01:43:32.261 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.262 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-08-19T01:43:32.262 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-08-19T01:43:32.262 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.262 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.262 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-08-19T01:43:32.262 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-08-19T01:43:32.262 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-08-19T01:43:32.262 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-08-19T01:43:32.263 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-08-19T01:43:32.263 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-08-19T01:43:32.263 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-08-19T01:43:32.263 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-08-19T01:43:32.263 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-08-19T01:43:32.263 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.263 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.263 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.263 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.263 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster": { 2024-08-19T01:43:32.263 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster", 2024-08-19T01:43:32.263 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-08-19T01:43:32.263 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.263 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-08-19T01:43:32.264 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-08-19T01:43:32.264 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.264 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.264 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:32.264 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-08-19T01:43:32.264 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.264 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.264 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.264 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.264 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster_level": { 2024-08-19T01:43:32.264 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster_level", 2024-08-19T01:43:32.264 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-08-19T01:43:32.264 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.264 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-08-19T01:43:32.264 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "info", 2024-08-19T01:43:32.265 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.265 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.265 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-08-19T01:43:32.265 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-08-19T01:43:32.265 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-08-19T01:43:32.265 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-08-19T01:43:32.265 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-08-19T01:43:32.265 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-08-19T01:43:32.265 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-08-19T01:43:32.265 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-08-19T01:43:32.265 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-08-19T01:43:32.265 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.265 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.265 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.265 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.266 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_file": { 2024-08-19T01:43:32.266 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_file", 2024-08-19T01:43:32.266 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-08-19T01:43:32.266 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.266 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-08-19T01:43:32.266 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-08-19T01:43:32.266 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.266 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.266 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:32.266 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-08-19T01:43:32.266 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.266 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.266 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.267 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.267 INFO:teuthology.orchestra.run.smithi083.stdout: "server_addr": { 2024-08-19T01:43:32.267 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "server_addr", 2024-08-19T01:43:32.267 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-08-19T01:43:32.267 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.267 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-08-19T01:43:32.267 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-08-19T01:43:32.267 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.267 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.267 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:32.267 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-08-19T01:43:32.267 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.267 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.267 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.267 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.268 INFO:teuthology.orchestra.run.smithi083.stdout: "server_port": { 2024-08-19T01:43:32.268 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "server_port", 2024-08-19T01:43:32.268 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-08-19T01:43:32.268 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.268 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-08-19T01:43:32.268 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-08-19T01:43:32.268 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.268 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.268 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:32.268 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-08-19T01:43:32.268 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.268 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.268 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.268 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-08-19T01:43:32.268 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-08-19T01:43:32.269 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.269 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-08-19T01:43:32.269 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "rgw", 2024-08-19T01:43:32.269 INFO:teuthology.orchestra.run.smithi083.stdout: "can_run": true, 2024-08-19T01:43:32.269 INFO:teuthology.orchestra.run.smithi083.stdout: "error_string": "", 2024-08-19T01:43:32.269 INFO:teuthology.orchestra.run.smithi083.stdout: "module_options": { 2024-08-19T01:43:32.269 INFO:teuthology.orchestra.run.smithi083.stdout: "log_level": { 2024-08-19T01:43:32.269 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_level", 2024-08-19T01:43:32.269 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-08-19T01:43:32.269 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.269 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-08-19T01:43:32.269 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-08-19T01:43:32.269 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.269 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.269 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-08-19T01:43:32.270 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-08-19T01:43:32.270 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-08-19T01:43:32.270 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-08-19T01:43:32.270 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-08-19T01:43:32.270 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-08-19T01:43:32.270 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-08-19T01:43:32.270 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-08-19T01:43:32.270 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-08-19T01:43:32.270 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.270 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.270 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.270 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.270 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster": { 2024-08-19T01:43:32.271 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster", 2024-08-19T01:43:32.271 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-08-19T01:43:32.271 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.271 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-08-19T01:43:32.271 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-08-19T01:43:32.271 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.271 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.271 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:32.271 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-08-19T01:43:32.271 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.271 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.271 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.271 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.271 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster_level": { 2024-08-19T01:43:32.271 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster_level", 2024-08-19T01:43:32.272 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-08-19T01:43:32.272 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.272 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-08-19T01:43:32.272 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "info", 2024-08-19T01:43:32.272 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.272 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.272 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-08-19T01:43:32.272 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-08-19T01:43:32.272 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-08-19T01:43:32.272 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-08-19T01:43:32.272 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-08-19T01:43:32.272 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-08-19T01:43:32.272 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-08-19T01:43:32.272 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-08-19T01:43:32.272 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-08-19T01:43:32.273 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.273 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.273 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.273 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.273 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_file": { 2024-08-19T01:43:32.273 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_file", 2024-08-19T01:43:32.273 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-08-19T01:43:32.273 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.273 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-08-19T01:43:32.273 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-08-19T01:43:32.273 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.273 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.273 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:32.274 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-08-19T01:43:32.274 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.274 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.274 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.274 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-08-19T01:43:32.274 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-08-19T01:43:32.274 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.274 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-08-19T01:43:32.274 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "rook", 2024-08-19T01:43:32.274 INFO:teuthology.orchestra.run.smithi083.stdout: "can_run": true, 2024-08-19T01:43:32.274 INFO:teuthology.orchestra.run.smithi083.stdout: "error_string": "", 2024-08-19T01:43:32.274 INFO:teuthology.orchestra.run.smithi083.stdout: "module_options": { 2024-08-19T01:43:32.274 INFO:teuthology.orchestra.run.smithi083.stdout: "log_level": { 2024-08-19T01:43:32.274 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_level", 2024-08-19T01:43:32.274 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-08-19T01:43:32.275 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.275 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-08-19T01:43:32.275 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-08-19T01:43:32.275 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.275 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.275 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-08-19T01:43:32.275 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-08-19T01:43:32.275 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-08-19T01:43:32.275 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-08-19T01:43:32.275 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-08-19T01:43:32.275 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-08-19T01:43:32.275 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-08-19T01:43:32.275 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-08-19T01:43:32.276 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-08-19T01:43:32.276 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.276 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.276 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.276 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.276 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster": { 2024-08-19T01:43:32.276 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster", 2024-08-19T01:43:32.276 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-08-19T01:43:32.276 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.276 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-08-19T01:43:32.276 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-08-19T01:43:32.276 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.276 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.277 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:32.277 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-08-19T01:43:32.277 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.277 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.277 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.277 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.277 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster_level": { 2024-08-19T01:43:32.277 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster_level", 2024-08-19T01:43:32.277 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-08-19T01:43:32.277 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.277 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-08-19T01:43:32.277 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "info", 2024-08-19T01:43:32.277 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.277 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.278 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-08-19T01:43:32.278 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-08-19T01:43:32.278 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-08-19T01:43:32.278 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-08-19T01:43:32.278 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-08-19T01:43:32.278 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-08-19T01:43:32.278 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-08-19T01:43:32.278 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-08-19T01:43:32.278 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-08-19T01:43:32.278 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.278 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.278 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.278 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.279 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_file": { 2024-08-19T01:43:32.279 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_file", 2024-08-19T01:43:32.279 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-08-19T01:43:32.279 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.279 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-08-19T01:43:32.279 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-08-19T01:43:32.279 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.279 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.279 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:32.279 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-08-19T01:43:32.279 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.279 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.279 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.279 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.280 INFO:teuthology.orchestra.run.smithi083.stdout: "storage_class": { 2024-08-19T01:43:32.280 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "storage_class", 2024-08-19T01:43:32.280 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-08-19T01:43:32.280 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.280 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-08-19T01:43:32.280 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "local", 2024-08-19T01:43:32.280 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.280 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.280 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:32.280 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-08-19T01:43:32.280 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.280 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.280 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.280 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-08-19T01:43:32.281 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-08-19T01:43:32.281 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.281 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-08-19T01:43:32.281 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "selftest", 2024-08-19T01:43:32.281 INFO:teuthology.orchestra.run.smithi083.stdout: "can_run": true, 2024-08-19T01:43:32.281 INFO:teuthology.orchestra.run.smithi083.stdout: "error_string": "", 2024-08-19T01:43:32.281 INFO:teuthology.orchestra.run.smithi083.stdout: "module_options": { 2024-08-19T01:43:32.281 INFO:teuthology.orchestra.run.smithi083.stdout: "log_level": { 2024-08-19T01:43:32.281 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_level", 2024-08-19T01:43:32.281 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-08-19T01:43:32.281 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.281 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-08-19T01:43:32.282 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-08-19T01:43:32.282 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.282 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.282 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-08-19T01:43:32.282 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-08-19T01:43:32.282 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-08-19T01:43:32.282 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-08-19T01:43:32.282 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-08-19T01:43:32.282 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-08-19T01:43:32.282 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-08-19T01:43:32.282 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-08-19T01:43:32.282 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-08-19T01:43:32.282 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.283 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.283 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.283 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.283 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster": { 2024-08-19T01:43:32.283 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster", 2024-08-19T01:43:32.283 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-08-19T01:43:32.283 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.283 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-08-19T01:43:32.283 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-08-19T01:43:32.283 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.283 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.283 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:32.283 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-08-19T01:43:32.284 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.284 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.284 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.284 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.284 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster_level": { 2024-08-19T01:43:32.284 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster_level", 2024-08-19T01:43:32.284 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-08-19T01:43:32.284 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.284 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-08-19T01:43:32.284 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "info", 2024-08-19T01:43:32.284 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.284 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.284 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-08-19T01:43:32.284 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-08-19T01:43:32.285 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-08-19T01:43:32.285 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-08-19T01:43:32.285 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-08-19T01:43:32.285 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-08-19T01:43:32.285 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-08-19T01:43:32.285 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-08-19T01:43:32.285 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-08-19T01:43:32.285 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.285 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.285 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.285 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.285 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_file": { 2024-08-19T01:43:32.285 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_file", 2024-08-19T01:43:32.286 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-08-19T01:43:32.286 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.286 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-08-19T01:43:32.286 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-08-19T01:43:32.286 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.286 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.286 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:32.286 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-08-19T01:43:32.286 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.286 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.286 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.286 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.286 INFO:teuthology.orchestra.run.smithi083.stdout: "roption1": { 2024-08-19T01:43:32.287 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "roption1", 2024-08-19T01:43:32.287 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-08-19T01:43:32.287 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.287 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-08-19T01:43:32.287 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-08-19T01:43:32.287 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.287 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.287 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:32.287 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-08-19T01:43:32.287 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.287 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.287 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.288 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.288 INFO:teuthology.orchestra.run.smithi083.stdout: "roption2": { 2024-08-19T01:43:32.288 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "roption2", 2024-08-19T01:43:32.288 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-08-19T01:43:32.288 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.288 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-08-19T01:43:32.288 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "xyz", 2024-08-19T01:43:32.288 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.288 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.288 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:32.288 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-08-19T01:43:32.288 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.288 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.288 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.288 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.289 INFO:teuthology.orchestra.run.smithi083.stdout: "rwoption1": { 2024-08-19T01:43:32.289 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "rwoption1", 2024-08-19T01:43:32.289 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-08-19T01:43:32.289 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.289 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-08-19T01:43:32.289 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-08-19T01:43:32.289 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.289 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.289 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:32.289 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-08-19T01:43:32.289 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.289 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.289 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.289 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.289 INFO:teuthology.orchestra.run.smithi083.stdout: "rwoption2": { 2024-08-19T01:43:32.290 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "rwoption2", 2024-08-19T01:43:32.290 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2024-08-19T01:43:32.290 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.290 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-08-19T01:43:32.290 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-08-19T01:43:32.290 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.290 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.290 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:32.290 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-08-19T01:43:32.290 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.290 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.290 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.291 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.291 INFO:teuthology.orchestra.run.smithi083.stdout: "rwoption3": { 2024-08-19T01:43:32.291 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "rwoption3", 2024-08-19T01:43:32.291 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "float", 2024-08-19T01:43:32.291 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.291 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-08-19T01:43:32.291 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-08-19T01:43:32.291 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.291 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.291 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:32.291 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-08-19T01:43:32.291 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.291 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.292 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.292 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.292 INFO:teuthology.orchestra.run.smithi083.stdout: "rwoption4": { 2024-08-19T01:43:32.292 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "rwoption4", 2024-08-19T01:43:32.292 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-08-19T01:43:32.292 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.292 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-08-19T01:43:32.292 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-08-19T01:43:32.292 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.292 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.292 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:32.292 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-08-19T01:43:32.292 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.292 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.293 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.293 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.293 INFO:teuthology.orchestra.run.smithi083.stdout: "rwoption5": { 2024-08-19T01:43:32.293 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "rwoption5", 2024-08-19T01:43:32.293 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-08-19T01:43:32.293 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.293 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-08-19T01:43:32.293 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-08-19T01:43:32.293 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.293 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.293 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:32.293 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-08-19T01:43:32.293 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.293 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.293 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.294 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.294 INFO:teuthology.orchestra.run.smithi083.stdout: "rwoption6": { 2024-08-19T01:43:32.294 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "rwoption6", 2024-08-19T01:43:32.294 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-08-19T01:43:32.294 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.294 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-08-19T01:43:32.294 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "True", 2024-08-19T01:43:32.294 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.294 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.294 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:32.294 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-08-19T01:43:32.294 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.294 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.294 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.295 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.295 INFO:teuthology.orchestra.run.smithi083.stdout: "rwoption7": { 2024-08-19T01:43:32.295 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "rwoption7", 2024-08-19T01:43:32.295 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2024-08-19T01:43:32.295 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.295 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-08-19T01:43:32.295 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-08-19T01:43:32.295 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "1", 2024-08-19T01:43:32.295 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "42", 2024-08-19T01:43:32.295 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:32.295 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-08-19T01:43:32.295 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.295 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.295 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.296 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.296 INFO:teuthology.orchestra.run.smithi083.stdout: "testkey": { 2024-08-19T01:43:32.296 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "testkey", 2024-08-19T01:43:32.296 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-08-19T01:43:32.296 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.296 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-08-19T01:43:32.296 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-08-19T01:43:32.296 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.296 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.296 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:32.296 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-08-19T01:43:32.296 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.296 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.297 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.297 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.297 INFO:teuthology.orchestra.run.smithi083.stdout: "testlkey": { 2024-08-19T01:43:32.297 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "testlkey", 2024-08-19T01:43:32.297 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-08-19T01:43:32.297 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.297 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-08-19T01:43:32.297 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-08-19T01:43:32.297 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.297 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.297 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:32.297 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-08-19T01:43:32.297 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.297 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.297 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.298 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.298 INFO:teuthology.orchestra.run.smithi083.stdout: "testnewline": { 2024-08-19T01:43:32.298 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "testnewline", 2024-08-19T01:43:32.298 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-08-19T01:43:32.298 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.298 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-08-19T01:43:32.298 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-08-19T01:43:32.298 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.298 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.298 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:32.298 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-08-19T01:43:32.298 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.298 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.298 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.299 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-08-19T01:43:32.299 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-08-19T01:43:32.299 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.299 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-08-19T01:43:32.299 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "snap_schedule", 2024-08-19T01:43:32.299 INFO:teuthology.orchestra.run.smithi083.stdout: "can_run": true, 2024-08-19T01:43:32.299 INFO:teuthology.orchestra.run.smithi083.stdout: "error_string": "", 2024-08-19T01:43:32.299 INFO:teuthology.orchestra.run.smithi083.stdout: "module_options": { 2024-08-19T01:43:32.299 INFO:teuthology.orchestra.run.smithi083.stdout: "allow_m_granularity": { 2024-08-19T01:43:32.299 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "allow_m_granularity", 2024-08-19T01:43:32.299 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-08-19T01:43:32.299 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.299 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-08-19T01:43:32.299 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-08-19T01:43:32.299 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.300 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.300 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:32.300 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "allow minute scheduled snapshots", 2024-08-19T01:43:32.300 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.300 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.300 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.300 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.300 INFO:teuthology.orchestra.run.smithi083.stdout: "dump_on_update": { 2024-08-19T01:43:32.300 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "dump_on_update", 2024-08-19T01:43:32.300 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-08-19T01:43:32.300 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.300 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-08-19T01:43:32.300 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-08-19T01:43:32.300 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.301 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.301 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:32.301 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "dump database to debug log on update", 2024-08-19T01:43:32.301 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.301 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.301 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.301 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.301 INFO:teuthology.orchestra.run.smithi083.stdout: "log_level": { 2024-08-19T01:43:32.301 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_level", 2024-08-19T01:43:32.301 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-08-19T01:43:32.301 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.301 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-08-19T01:43:32.301 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-08-19T01:43:32.301 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.301 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.302 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-08-19T01:43:32.302 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-08-19T01:43:32.302 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-08-19T01:43:32.302 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-08-19T01:43:32.302 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-08-19T01:43:32.302 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-08-19T01:43:32.302 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-08-19T01:43:32.302 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-08-19T01:43:32.302 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-08-19T01:43:32.302 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.302 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.302 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.303 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.303 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster": { 2024-08-19T01:43:32.303 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster", 2024-08-19T01:43:32.303 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-08-19T01:43:32.303 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.303 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-08-19T01:43:32.303 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-08-19T01:43:32.303 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.303 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.303 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:32.303 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-08-19T01:43:32.303 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.303 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.303 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.304 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.304 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster_level": { 2024-08-19T01:43:32.304 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster_level", 2024-08-19T01:43:32.304 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-08-19T01:43:32.304 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.304 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-08-19T01:43:32.304 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "info", 2024-08-19T01:43:32.304 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.304 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.304 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-08-19T01:43:32.304 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-08-19T01:43:32.304 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-08-19T01:43:32.304 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-08-19T01:43:32.305 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-08-19T01:43:32.305 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-08-19T01:43:32.305 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-08-19T01:43:32.305 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-08-19T01:43:32.305 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-08-19T01:43:32.305 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.305 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.305 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.305 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.305 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_file": { 2024-08-19T01:43:32.305 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_file", 2024-08-19T01:43:32.305 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-08-19T01:43:32.305 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.305 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-08-19T01:43:32.306 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-08-19T01:43:32.306 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.306 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.306 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:32.306 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-08-19T01:43:32.306 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.306 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.306 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.306 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-08-19T01:43:32.306 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-08-19T01:43:32.306 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.306 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-08-19T01:43:32.306 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "stats", 2024-08-19T01:43:32.306 INFO:teuthology.orchestra.run.smithi083.stdout: "can_run": true, 2024-08-19T01:43:32.307 INFO:teuthology.orchestra.run.smithi083.stdout: "error_string": "", 2024-08-19T01:43:32.307 INFO:teuthology.orchestra.run.smithi083.stdout: "module_options": { 2024-08-19T01:43:32.307 INFO:teuthology.orchestra.run.smithi083.stdout: "log_level": { 2024-08-19T01:43:32.307 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_level", 2024-08-19T01:43:32.307 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-08-19T01:43:32.307 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.307 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-08-19T01:43:32.307 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-08-19T01:43:32.307 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.307 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.307 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-08-19T01:43:32.307 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-08-19T01:43:32.307 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-08-19T01:43:32.307 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-08-19T01:43:32.308 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-08-19T01:43:32.308 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-08-19T01:43:32.308 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-08-19T01:43:32.308 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-08-19T01:43:32.308 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-08-19T01:43:32.308 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.308 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.308 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.308 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.308 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster": { 2024-08-19T01:43:32.308 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster", 2024-08-19T01:43:32.308 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-08-19T01:43:32.308 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.308 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-08-19T01:43:32.309 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-08-19T01:43:32.309 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.309 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.309 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:32.309 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-08-19T01:43:32.309 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.309 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.309 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.309 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.309 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster_level": { 2024-08-19T01:43:32.309 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster_level", 2024-08-19T01:43:32.309 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-08-19T01:43:32.309 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.309 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-08-19T01:43:32.309 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "info", 2024-08-19T01:43:32.310 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.310 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.310 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-08-19T01:43:32.310 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-08-19T01:43:32.310 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-08-19T01:43:32.310 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-08-19T01:43:32.310 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-08-19T01:43:32.310 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-08-19T01:43:32.310 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-08-19T01:43:32.310 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-08-19T01:43:32.310 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-08-19T01:43:32.310 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.310 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.310 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.311 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.311 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_file": { 2024-08-19T01:43:32.311 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_file", 2024-08-19T01:43:32.311 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-08-19T01:43:32.311 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.311 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-08-19T01:43:32.311 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-08-19T01:43:32.311 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.311 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.311 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:32.311 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-08-19T01:43:32.311 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.311 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.311 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.311 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-08-19T01:43:32.312 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-08-19T01:43:32.312 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.312 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-08-19T01:43:32.312 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "status", 2024-08-19T01:43:32.312 INFO:teuthology.orchestra.run.smithi083.stdout: "can_run": true, 2024-08-19T01:43:32.312 INFO:teuthology.orchestra.run.smithi083.stdout: "error_string": "", 2024-08-19T01:43:32.312 INFO:teuthology.orchestra.run.smithi083.stdout: "module_options": { 2024-08-19T01:43:32.312 INFO:teuthology.orchestra.run.smithi083.stdout: "log_level": { 2024-08-19T01:43:32.312 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_level", 2024-08-19T01:43:32.312 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-08-19T01:43:32.312 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.312 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-08-19T01:43:32.312 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-08-19T01:43:32.312 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.313 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.313 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-08-19T01:43:32.313 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-08-19T01:43:32.313 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-08-19T01:43:32.313 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-08-19T01:43:32.313 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-08-19T01:43:32.313 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-08-19T01:43:32.313 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-08-19T01:43:32.313 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-08-19T01:43:32.313 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-08-19T01:43:32.313 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.313 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.313 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.314 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.314 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster": { 2024-08-19T01:43:32.314 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster", 2024-08-19T01:43:32.314 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-08-19T01:43:32.314 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.314 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-08-19T01:43:32.314 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-08-19T01:43:32.314 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.314 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.314 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:32.314 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-08-19T01:43:32.314 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.314 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.314 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.315 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.315 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster_level": { 2024-08-19T01:43:32.315 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster_level", 2024-08-19T01:43:32.315 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-08-19T01:43:32.315 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.315 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-08-19T01:43:32.315 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "info", 2024-08-19T01:43:32.315 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.315 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.315 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-08-19T01:43:32.315 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-08-19T01:43:32.315 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-08-19T01:43:32.315 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-08-19T01:43:32.315 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-08-19T01:43:32.316 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-08-19T01:43:32.316 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-08-19T01:43:32.316 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-08-19T01:43:32.316 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-08-19T01:43:32.316 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.316 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.316 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.316 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.316 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_file": { 2024-08-19T01:43:32.316 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_file", 2024-08-19T01:43:32.316 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-08-19T01:43:32.316 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.316 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-08-19T01:43:32.316 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-08-19T01:43:32.316 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.317 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.317 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:32.317 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-08-19T01:43:32.317 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.317 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.317 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.317 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-08-19T01:43:32.317 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-08-19T01:43:32.317 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.317 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-08-19T01:43:32.317 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "telegraf", 2024-08-19T01:43:32.317 INFO:teuthology.orchestra.run.smithi083.stdout: "can_run": true, 2024-08-19T01:43:32.317 INFO:teuthology.orchestra.run.smithi083.stdout: "error_string": "", 2024-08-19T01:43:32.317 INFO:teuthology.orchestra.run.smithi083.stdout: "module_options": { 2024-08-19T01:43:32.317 INFO:teuthology.orchestra.run.smithi083.stdout: "address": { 2024-08-19T01:43:32.318 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "address", 2024-08-19T01:43:32.318 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-08-19T01:43:32.318 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.318 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-08-19T01:43:32.318 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-08-19T01:43:32.318 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.318 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.318 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:32.318 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-08-19T01:43:32.318 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.318 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.318 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.318 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.319 INFO:teuthology.orchestra.run.smithi083.stdout: "interval": { 2024-08-19T01:43:32.319 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "interval", 2024-08-19T01:43:32.319 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "secs", 2024-08-19T01:43:32.319 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.319 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-08-19T01:43:32.319 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "15", 2024-08-19T01:43:32.319 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.319 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.319 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:32.319 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-08-19T01:43:32.319 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.319 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.319 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.319 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.319 INFO:teuthology.orchestra.run.smithi083.stdout: "log_level": { 2024-08-19T01:43:32.320 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_level", 2024-08-19T01:43:32.320 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-08-19T01:43:32.320 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.320 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-08-19T01:43:32.320 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-08-19T01:43:32.320 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.320 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.320 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-08-19T01:43:32.320 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-08-19T01:43:32.320 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-08-19T01:43:32.320 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-08-19T01:43:32.320 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-08-19T01:43:32.320 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-08-19T01:43:32.320 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-08-19T01:43:32.320 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-08-19T01:43:32.321 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-08-19T01:43:32.321 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.321 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.321 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.321 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.321 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster": { 2024-08-19T01:43:32.321 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster", 2024-08-19T01:43:32.321 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-08-19T01:43:32.321 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.321 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-08-19T01:43:32.321 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-08-19T01:43:32.321 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.321 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.321 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:32.321 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-08-19T01:43:32.322 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.322 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.322 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.322 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.322 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster_level": { 2024-08-19T01:43:32.322 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster_level", 2024-08-19T01:43:32.322 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-08-19T01:43:32.322 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.322 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-08-19T01:43:32.322 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "info", 2024-08-19T01:43:32.322 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.322 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.322 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-08-19T01:43:32.323 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-08-19T01:43:32.323 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-08-19T01:43:32.323 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-08-19T01:43:32.323 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-08-19T01:43:32.323 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-08-19T01:43:32.323 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-08-19T01:43:32.323 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-08-19T01:43:32.323 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-08-19T01:43:32.323 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.323 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.323 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.323 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.323 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_file": { 2024-08-19T01:43:32.323 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_file", 2024-08-19T01:43:32.323 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-08-19T01:43:32.324 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.324 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-08-19T01:43:32.324 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-08-19T01:43:32.324 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.324 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.324 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:32.324 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-08-19T01:43:32.324 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.324 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.324 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.324 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-08-19T01:43:32.324 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-08-19T01:43:32.324 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.324 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-08-19T01:43:32.324 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "telemetry", 2024-08-19T01:43:32.324 INFO:teuthology.orchestra.run.smithi083.stdout: "can_run": true, 2024-08-19T01:43:32.325 INFO:teuthology.orchestra.run.smithi083.stdout: "error_string": "", 2024-08-19T01:43:32.325 INFO:teuthology.orchestra.run.smithi083.stdout: "module_options": { 2024-08-19T01:43:32.325 INFO:teuthology.orchestra.run.smithi083.stdout: "channel_basic": { 2024-08-19T01:43:32.325 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "channel_basic", 2024-08-19T01:43:32.325 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-08-19T01:43:32.325 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.325 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-08-19T01:43:32.325 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "True", 2024-08-19T01:43:32.325 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.325 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.325 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:32.325 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Share basic cluster information (size, version)", 2024-08-19T01:43:32.325 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.325 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.325 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.326 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.326 INFO:teuthology.orchestra.run.smithi083.stdout: "channel_crash": { 2024-08-19T01:43:32.326 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "channel_crash", 2024-08-19T01:43:32.326 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-08-19T01:43:32.326 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.326 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-08-19T01:43:32.326 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "True", 2024-08-19T01:43:32.326 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.326 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.326 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:32.326 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-08-19T01:43:32.326 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.326 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.326 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.326 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.327 INFO:teuthology.orchestra.run.smithi083.stdout: "channel_device": { 2024-08-19T01:43:32.327 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "channel_device", 2024-08-19T01:43:32.327 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-08-19T01:43:32.327 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.327 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-08-19T01:43:32.327 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "True", 2024-08-19T01:43:32.327 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.327 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.327 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:32.327 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-08-19T01:43:32.327 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.327 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.327 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.327 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.327 INFO:teuthology.orchestra.run.smithi083.stdout: "channel_ident": { 2024-08-19T01:43:32.328 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "channel_ident", 2024-08-19T01:43:32.328 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-08-19T01:43:32.328 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.328 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-08-19T01:43:32.328 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-08-19T01:43:32.328 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.328 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.328 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:32.328 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-08-19T01:43:32.328 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.328 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.328 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.328 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.328 INFO:teuthology.orchestra.run.smithi083.stdout: "channel_perf": { 2024-08-19T01:43:32.328 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "channel_perf", 2024-08-19T01:43:32.329 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-08-19T01:43:32.329 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.329 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-08-19T01:43:32.329 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-08-19T01:43:32.329 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.329 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.329 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:32.329 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Share various performance metrics of a cluster", 2024-08-19T01:43:32.329 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.329 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.329 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.329 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.329 INFO:teuthology.orchestra.run.smithi083.stdout: "contact": { 2024-08-19T01:43:32.329 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "contact", 2024-08-19T01:43:32.329 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-08-19T01:43:32.329 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.330 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-08-19T01:43:32.330 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-08-19T01:43:32.330 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.330 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.330 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:32.330 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-08-19T01:43:32.330 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.330 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.330 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.330 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.330 INFO:teuthology.orchestra.run.smithi083.stdout: "description": { 2024-08-19T01:43:32.330 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "description", 2024-08-19T01:43:32.330 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-08-19T01:43:32.331 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.331 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-08-19T01:43:32.331 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-08-19T01:43:32.331 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.331 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.331 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:32.331 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-08-19T01:43:32.331 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.331 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.331 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.331 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.331 INFO:teuthology.orchestra.run.smithi083.stdout: "device_url": { 2024-08-19T01:43:32.331 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "device_url", 2024-08-19T01:43:32.331 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-08-19T01:43:32.332 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.332 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-08-19T01:43:32.332 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-08-19T01:43:32.332 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.332 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.332 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:32.332 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-08-19T01:43:32.332 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.332 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.332 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.332 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.332 INFO:teuthology.orchestra.run.smithi083.stdout: "enabled": { 2024-08-19T01:43:32.332 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "enabled", 2024-08-19T01:43:32.332 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-08-19T01:43:32.333 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.333 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-08-19T01:43:32.333 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-08-19T01:43:32.333 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.333 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.333 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:32.333 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-08-19T01:43:32.333 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.333 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.333 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.333 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.333 INFO:teuthology.orchestra.run.smithi083.stdout: "interval": { 2024-08-19T01:43:32.333 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "interval", 2024-08-19T01:43:32.333 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2024-08-19T01:43:32.333 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.334 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-08-19T01:43:32.334 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "24", 2024-08-19T01:43:32.334 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "8", 2024-08-19T01:43:32.334 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.334 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:32.334 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-08-19T01:43:32.334 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.334 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.334 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.334 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.334 INFO:teuthology.orchestra.run.smithi083.stdout: "last_opt_revision": { 2024-08-19T01:43:32.334 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "last_opt_revision", 2024-08-19T01:43:32.334 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2024-08-19T01:43:32.335 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.335 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-08-19T01:43:32.335 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "1", 2024-08-19T01:43:32.335 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.335 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.335 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:32.335 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-08-19T01:43:32.335 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.335 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.335 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.335 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.335 INFO:teuthology.orchestra.run.smithi083.stdout: "leaderboard": { 2024-08-19T01:43:32.336 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "leaderboard", 2024-08-19T01:43:32.336 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-08-19T01:43:32.336 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.336 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-08-19T01:43:32.336 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-08-19T01:43:32.336 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.336 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.336 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:32.336 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-08-19T01:43:32.336 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.336 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.336 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.336 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.337 INFO:teuthology.orchestra.run.smithi083.stdout: "leaderboard_description": { 2024-08-19T01:43:32.337 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "leaderboard_description", 2024-08-19T01:43:32.337 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-08-19T01:43:32.337 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.337 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-08-19T01:43:32.337 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-08-19T01:43:32.337 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.337 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.337 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:32.337 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-08-19T01:43:32.337 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.337 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.337 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.337 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.338 INFO:teuthology.orchestra.run.smithi083.stdout: "log_level": { 2024-08-19T01:43:32.338 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_level", 2024-08-19T01:43:32.338 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-08-19T01:43:32.338 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.338 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-08-19T01:43:32.338 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-08-19T01:43:32.338 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.338 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.338 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-08-19T01:43:32.338 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-08-19T01:43:32.338 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-08-19T01:43:32.338 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-08-19T01:43:32.338 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-08-19T01:43:32.338 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-08-19T01:43:32.339 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-08-19T01:43:32.339 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-08-19T01:43:32.339 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-08-19T01:43:32.339 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.339 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.339 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.339 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.339 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster": { 2024-08-19T01:43:32.339 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster", 2024-08-19T01:43:32.339 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-08-19T01:43:32.339 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.339 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-08-19T01:43:32.339 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-08-19T01:43:32.339 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.340 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.340 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:32.340 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-08-19T01:43:32.340 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.340 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.340 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.340 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.340 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster_level": { 2024-08-19T01:43:32.340 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster_level", 2024-08-19T01:43:32.340 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-08-19T01:43:32.340 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.340 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-08-19T01:43:32.340 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "info", 2024-08-19T01:43:32.341 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.341 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.341 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-08-19T01:43:32.341 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-08-19T01:43:32.341 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-08-19T01:43:32.341 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-08-19T01:43:32.341 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-08-19T01:43:32.341 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-08-19T01:43:32.341 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-08-19T01:43:32.341 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-08-19T01:43:32.341 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-08-19T01:43:32.341 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.341 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.342 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.342 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.342 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_file": { 2024-08-19T01:43:32.342 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_file", 2024-08-19T01:43:32.342 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-08-19T01:43:32.342 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.342 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-08-19T01:43:32.342 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-08-19T01:43:32.342 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.342 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.342 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:32.342 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-08-19T01:43:32.343 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.343 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.343 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.343 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.343 INFO:teuthology.orchestra.run.smithi083.stdout: "organization": { 2024-08-19T01:43:32.343 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "organization", 2024-08-19T01:43:32.343 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-08-19T01:43:32.343 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.343 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-08-19T01:43:32.343 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-08-19T01:43:32.343 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.343 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.343 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:32.344 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-08-19T01:43:32.344 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.344 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.344 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.344 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.344 INFO:teuthology.orchestra.run.smithi083.stdout: "proxy": { 2024-08-19T01:43:32.344 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "proxy", 2024-08-19T01:43:32.344 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-08-19T01:43:32.344 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.344 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-08-19T01:43:32.344 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-08-19T01:43:32.344 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.344 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.344 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:32.344 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-08-19T01:43:32.345 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.345 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.345 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.345 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.345 INFO:teuthology.orchestra.run.smithi083.stdout: "url": { 2024-08-19T01:43:32.345 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "url", 2024-08-19T01:43:32.345 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-08-19T01:43:32.345 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.345 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-08-19T01:43:32.345 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-08-19T01:43:32.345 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.345 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.346 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:32.346 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-08-19T01:43:32.346 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.346 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.346 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.346 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-08-19T01:43:32.346 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-08-19T01:43:32.346 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.346 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-08-19T01:43:32.346 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "test_orchestrator", 2024-08-19T01:43:32.346 INFO:teuthology.orchestra.run.smithi083.stdout: "can_run": true, 2024-08-19T01:43:32.346 INFO:teuthology.orchestra.run.smithi083.stdout: "error_string": "", 2024-08-19T01:43:32.346 INFO:teuthology.orchestra.run.smithi083.stdout: "module_options": { 2024-08-19T01:43:32.347 INFO:teuthology.orchestra.run.smithi083.stdout: "log_level": { 2024-08-19T01:43:32.347 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_level", 2024-08-19T01:43:32.347 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-08-19T01:43:32.347 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.347 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-08-19T01:43:32.347 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-08-19T01:43:32.347 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.347 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.347 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-08-19T01:43:32.347 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-08-19T01:43:32.347 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-08-19T01:43:32.347 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-08-19T01:43:32.347 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-08-19T01:43:32.347 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-08-19T01:43:32.348 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-08-19T01:43:32.348 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-08-19T01:43:32.348 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-08-19T01:43:32.348 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.348 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.348 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.348 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.348 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster": { 2024-08-19T01:43:32.348 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster", 2024-08-19T01:43:32.348 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-08-19T01:43:32.348 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.348 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-08-19T01:43:32.348 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-08-19T01:43:32.348 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.349 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.349 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:32.349 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-08-19T01:43:32.349 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.349 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.349 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.349 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.349 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster_level": { 2024-08-19T01:43:32.349 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster_level", 2024-08-19T01:43:32.349 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-08-19T01:43:32.349 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.349 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-08-19T01:43:32.349 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "info", 2024-08-19T01:43:32.349 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.349 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.349 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-08-19T01:43:32.350 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-08-19T01:43:32.350 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-08-19T01:43:32.350 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-08-19T01:43:32.350 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-08-19T01:43:32.350 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-08-19T01:43:32.350 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-08-19T01:43:32.350 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-08-19T01:43:32.350 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-08-19T01:43:32.350 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.350 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.350 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.350 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.350 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_file": { 2024-08-19T01:43:32.351 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_file", 2024-08-19T01:43:32.351 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-08-19T01:43:32.351 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.351 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-08-19T01:43:32.351 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-08-19T01:43:32.351 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.351 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.351 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:32.351 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-08-19T01:43:32.351 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.351 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.351 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.351 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-08-19T01:43:32.351 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-08-19T01:43:32.352 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.352 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-08-19T01:43:32.352 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "volumes", 2024-08-19T01:43:32.352 INFO:teuthology.orchestra.run.smithi083.stdout: "can_run": true, 2024-08-19T01:43:32.352 INFO:teuthology.orchestra.run.smithi083.stdout: "error_string": "", 2024-08-19T01:43:32.352 INFO:teuthology.orchestra.run.smithi083.stdout: "module_options": { 2024-08-19T01:43:32.352 INFO:teuthology.orchestra.run.smithi083.stdout: "log_level": { 2024-08-19T01:43:32.352 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_level", 2024-08-19T01:43:32.352 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-08-19T01:43:32.352 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.352 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-08-19T01:43:32.352 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-08-19T01:43:32.352 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.353 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.353 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-08-19T01:43:32.353 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-08-19T01:43:32.353 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-08-19T01:43:32.353 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-08-19T01:43:32.353 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-08-19T01:43:32.353 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-08-19T01:43:32.353 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-08-19T01:43:32.353 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-08-19T01:43:32.353 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-08-19T01:43:32.353 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.353 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.353 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.353 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.353 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster": { 2024-08-19T01:43:32.354 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster", 2024-08-19T01:43:32.354 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-08-19T01:43:32.354 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.354 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-08-19T01:43:32.354 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-08-19T01:43:32.354 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.354 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.354 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:32.354 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-08-19T01:43:32.354 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.354 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.354 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.354 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.354 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster_level": { 2024-08-19T01:43:32.354 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster_level", 2024-08-19T01:43:32.355 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-08-19T01:43:32.355 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.355 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-08-19T01:43:32.355 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "info", 2024-08-19T01:43:32.355 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.355 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.355 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-08-19T01:43:32.355 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-08-19T01:43:32.355 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-08-19T01:43:32.355 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-08-19T01:43:32.355 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-08-19T01:43:32.355 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-08-19T01:43:32.355 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-08-19T01:43:32.355 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-08-19T01:43:32.355 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-08-19T01:43:32.356 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.356 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.356 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.356 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.356 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_file": { 2024-08-19T01:43:32.356 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_file", 2024-08-19T01:43:32.356 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-08-19T01:43:32.356 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.356 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-08-19T01:43:32.356 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-08-19T01:43:32.356 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.356 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.356 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:32.356 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-08-19T01:43:32.356 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.357 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.357 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.357 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.357 INFO:teuthology.orchestra.run.smithi083.stdout: "max_concurrent_clones": { 2024-08-19T01:43:32.357 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "max_concurrent_clones", 2024-08-19T01:43:32.357 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2024-08-19T01:43:32.357 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.357 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-08-19T01:43:32.357 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "4", 2024-08-19T01:43:32.357 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.357 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.357 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:32.357 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Number of asynchronous cloner threads", 2024-08-19T01:43:32.357 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.357 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.357 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.358 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.358 INFO:teuthology.orchestra.run.smithi083.stdout: "periodic_async_work": { 2024-08-19T01:43:32.358 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "periodic_async_work", 2024-08-19T01:43:32.358 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-08-19T01:43:32.358 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.358 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-08-19T01:43:32.358 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-08-19T01:43:32.358 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.358 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.358 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:32.358 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Periodically check for async work", 2024-08-19T01:43:32.358 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.358 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.358 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.359 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.359 INFO:teuthology.orchestra.run.smithi083.stdout: "snapshot_clone_delay": { 2024-08-19T01:43:32.359 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "snapshot_clone_delay", 2024-08-19T01:43:32.359 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2024-08-19T01:43:32.359 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.359 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-08-19T01:43:32.359 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "0", 2024-08-19T01:43:32.359 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.359 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.359 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:32.359 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-08-19T01:43:32.359 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.359 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.359 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.359 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.360 INFO:teuthology.orchestra.run.smithi083.stdout: "snapshot_clone_no_wait": { 2024-08-19T01:43:32.360 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "snapshot_clone_no_wait", 2024-08-19T01:43:32.360 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-08-19T01:43:32.360 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.360 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-08-19T01:43:32.360 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "True", 2024-08-19T01:43:32.360 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.360 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.360 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:32.360 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-08-19T01:43:32.360 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.360 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.360 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.360 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-08-19T01:43:32.360 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-08-19T01:43:32.361 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.361 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-08-19T01:43:32.361 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "zabbix", 2024-08-19T01:43:32.361 INFO:teuthology.orchestra.run.smithi083.stdout: "can_run": true, 2024-08-19T01:43:32.361 INFO:teuthology.orchestra.run.smithi083.stdout: "error_string": "", 2024-08-19T01:43:32.361 INFO:teuthology.orchestra.run.smithi083.stdout: "module_options": { 2024-08-19T01:43:32.361 INFO:teuthology.orchestra.run.smithi083.stdout: "discovery_interval": { 2024-08-19T01:43:32.361 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "discovery_interval", 2024-08-19T01:43:32.361 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "uint", 2024-08-19T01:43:32.361 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.361 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-08-19T01:43:32.361 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "100", 2024-08-19T01:43:32.361 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.361 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.361 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:32.362 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-08-19T01:43:32.362 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.362 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.362 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.362 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.362 INFO:teuthology.orchestra.run.smithi083.stdout: "identifier": { 2024-08-19T01:43:32.362 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "identifier", 2024-08-19T01:43:32.362 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-08-19T01:43:32.362 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.362 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-08-19T01:43:32.363 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-08-19T01:43:32.363 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.363 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.363 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:32.363 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-08-19T01:43:32.363 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.363 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.363 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.363 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.363 INFO:teuthology.orchestra.run.smithi083.stdout: "interval": { 2024-08-19T01:43:32.363 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "interval", 2024-08-19T01:43:32.363 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "secs", 2024-08-19T01:43:32.363 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.364 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-08-19T01:43:32.364 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "60", 2024-08-19T01:43:32.364 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.364 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.364 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:32.364 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-08-19T01:43:32.364 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.364 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.364 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.364 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.364 INFO:teuthology.orchestra.run.smithi083.stdout: "log_level": { 2024-08-19T01:43:32.364 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_level", 2024-08-19T01:43:32.365 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-08-19T01:43:32.365 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.365 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-08-19T01:43:32.365 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-08-19T01:43:32.365 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.365 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.365 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-08-19T01:43:32.365 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-08-19T01:43:32.365 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-08-19T01:43:32.365 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-08-19T01:43:32.365 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-08-19T01:43:32.365 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-08-19T01:43:32.365 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-08-19T01:43:32.366 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-08-19T01:43:32.366 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-08-19T01:43:32.366 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.366 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.366 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.366 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.366 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster": { 2024-08-19T01:43:32.366 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster", 2024-08-19T01:43:32.366 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-08-19T01:43:32.366 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.366 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-08-19T01:43:32.366 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-08-19T01:43:32.367 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.367 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.367 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:32.367 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-08-19T01:43:32.367 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.367 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.367 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.367 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.367 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster_level": { 2024-08-19T01:43:32.367 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster_level", 2024-08-19T01:43:32.367 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-08-19T01:43:32.367 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.367 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-08-19T01:43:32.367 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "info", 2024-08-19T01:43:32.368 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.368 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.368 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-08-19T01:43:32.368 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-08-19T01:43:32.368 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-08-19T01:43:32.368 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-08-19T01:43:32.368 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-08-19T01:43:32.368 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-08-19T01:43:32.368 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-08-19T01:43:32.368 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-08-19T01:43:32.368 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-08-19T01:43:32.368 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.368 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.369 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.369 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.369 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_file": { 2024-08-19T01:43:32.369 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_file", 2024-08-19T01:43:32.369 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-08-19T01:43:32.369 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.369 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-08-19T01:43:32.369 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-08-19T01:43:32.369 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.369 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.369 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:32.369 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-08-19T01:43:32.369 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.370 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.370 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.370 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.370 INFO:teuthology.orchestra.run.smithi083.stdout: "zabbix_host": { 2024-08-19T01:43:32.370 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "zabbix_host", 2024-08-19T01:43:32.370 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-08-19T01:43:32.370 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.370 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-08-19T01:43:32.370 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-08-19T01:43:32.370 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.370 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.370 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:32.371 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-08-19T01:43:32.371 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.371 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.371 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.371 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.371 INFO:teuthology.orchestra.run.smithi083.stdout: "zabbix_port": { 2024-08-19T01:43:32.371 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "zabbix_port", 2024-08-19T01:43:32.371 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2024-08-19T01:43:32.371 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.371 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-08-19T01:43:32.371 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "10051", 2024-08-19T01:43:32.371 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.371 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.371 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:32.372 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-08-19T01:43:32.372 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.372 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.372 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.372 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.372 INFO:teuthology.orchestra.run.smithi083.stdout: "zabbix_sender": { 2024-08-19T01:43:32.372 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "zabbix_sender", 2024-08-19T01:43:32.372 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-08-19T01:43:32.372 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.372 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-08-19T01:43:32.372 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-08-19T01:43:32.372 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.372 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.373 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:32.373 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-08-19T01:43:32.373 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.373 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.373 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.373 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-08-19T01:43:32.373 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-08-19T01:43:32.373 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-08-19T01:43:32.373 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-08-19T01:43:32.373 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.373 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-08-19T01:43:32.373 INFO:teuthology.orchestra.run.smithi083.stdout: "gid": 24122, 2024-08-19T01:43:32.373 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "y", 2024-08-19T01:43:32.374 INFO:teuthology.orchestra.run.smithi083.stdout: "mgr_features": 4540701547738038271, 2024-08-19T01:43:32.374 INFO:teuthology.orchestra.run.smithi083.stdout: "available_modules": [ 2024-08-19T01:43:32.374 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-08-19T01:43:32.374 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "alerts", 2024-08-19T01:43:32.374 INFO:teuthology.orchestra.run.smithi083.stdout: "can_run": true, 2024-08-19T01:43:32.374 INFO:teuthology.orchestra.run.smithi083.stdout: "error_string": "", 2024-08-19T01:43:32.374 INFO:teuthology.orchestra.run.smithi083.stdout: "module_options": { 2024-08-19T01:43:32.374 INFO:teuthology.orchestra.run.smithi083.stdout: "interval": { 2024-08-19T01:43:32.374 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "interval", 2024-08-19T01:43:32.374 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "secs", 2024-08-19T01:43:32.374 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.374 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-08-19T01:43:32.375 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "60", 2024-08-19T01:43:32.375 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.375 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.375 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:32.375 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "How frequently to reexamine health status", 2024-08-19T01:43:32.375 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.375 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.375 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.375 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.375 INFO:teuthology.orchestra.run.smithi083.stdout: "log_level": { 2024-08-19T01:43:32.375 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_level", 2024-08-19T01:43:32.375 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-08-19T01:43:32.375 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.376 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-08-19T01:43:32.376 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-08-19T01:43:32.376 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.376 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.376 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-08-19T01:43:32.376 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-08-19T01:43:32.376 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-08-19T01:43:32.376 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-08-19T01:43:32.376 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-08-19T01:43:32.376 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-08-19T01:43:32.376 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-08-19T01:43:32.376 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-08-19T01:43:32.376 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-08-19T01:43:32.376 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.377 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.377 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.377 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.377 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster": { 2024-08-19T01:43:32.377 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster", 2024-08-19T01:43:32.377 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-08-19T01:43:32.377 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.377 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-08-19T01:43:32.377 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-08-19T01:43:32.377 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.377 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.377 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:32.377 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-08-19T01:43:32.378 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.378 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.378 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.378 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.378 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster_level": { 2024-08-19T01:43:32.378 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster_level", 2024-08-19T01:43:32.378 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-08-19T01:43:32.378 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.378 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-08-19T01:43:32.378 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "info", 2024-08-19T01:43:32.378 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.378 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.378 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-08-19T01:43:32.379 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-08-19T01:43:32.379 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-08-19T01:43:32.379 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-08-19T01:43:32.379 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-08-19T01:43:32.379 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-08-19T01:43:32.379 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-08-19T01:43:32.379 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-08-19T01:43:32.379 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-08-19T01:43:32.379 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.379 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.379 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.379 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.379 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_file": { 2024-08-19T01:43:32.379 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_file", 2024-08-19T01:43:32.380 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-08-19T01:43:32.380 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.380 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-08-19T01:43:32.380 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-08-19T01:43:32.380 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.380 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.380 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:32.380 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-08-19T01:43:32.380 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.380 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.380 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.380 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.380 INFO:teuthology.orchestra.run.smithi083.stdout: "smtp_destination": { 2024-08-19T01:43:32.381 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "smtp_destination", 2024-08-19T01:43:32.381 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-08-19T01:43:32.381 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.381 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-08-19T01:43:32.381 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-08-19T01:43:32.381 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.381 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.381 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:32.381 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Email address to send alerts to", 2024-08-19T01:43:32.381 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.381 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.382 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.382 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.382 INFO:teuthology.orchestra.run.smithi083.stdout: "smtp_from_name": { 2024-08-19T01:43:32.382 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "smtp_from_name", 2024-08-19T01:43:32.382 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-08-19T01:43:32.382 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.382 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-08-19T01:43:32.382 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "Ceph", 2024-08-19T01:43:32.382 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.382 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.382 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:32.382 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Email From: name", 2024-08-19T01:43:32.382 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.383 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.383 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.383 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.383 INFO:teuthology.orchestra.run.smithi083.stdout: "smtp_host": { 2024-08-19T01:43:32.383 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "smtp_host", 2024-08-19T01:43:32.383 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-08-19T01:43:32.383 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.383 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-08-19T01:43:32.383 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-08-19T01:43:32.383 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.383 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.383 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:32.383 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "SMTP server", 2024-08-19T01:43:32.384 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.384 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.384 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.384 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.384 INFO:teuthology.orchestra.run.smithi083.stdout: "smtp_password": { 2024-08-19T01:43:32.384 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "smtp_password", 2024-08-19T01:43:32.384 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-08-19T01:43:32.384 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.384 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-08-19T01:43:32.384 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-08-19T01:43:32.384 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.384 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.384 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:32.384 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Password to authenticate with", 2024-08-19T01:43:32.384 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.385 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.385 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.385 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.385 INFO:teuthology.orchestra.run.smithi083.stdout: "smtp_port": { 2024-08-19T01:43:32.385 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "smtp_port", 2024-08-19T01:43:32.385 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2024-08-19T01:43:32.385 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.385 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-08-19T01:43:32.385 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "465", 2024-08-19T01:43:32.385 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.385 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.385 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:32.385 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "SMTP port", 2024-08-19T01:43:32.386 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.386 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.386 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.386 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.386 INFO:teuthology.orchestra.run.smithi083.stdout: "smtp_sender": { 2024-08-19T01:43:32.386 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "smtp_sender", 2024-08-19T01:43:32.386 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-08-19T01:43:32.386 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.386 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-08-19T01:43:32.386 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-08-19T01:43:32.386 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.386 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.386 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:32.387 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "SMTP envelope sender", 2024-08-19T01:43:32.387 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.387 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.387 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.387 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.387 INFO:teuthology.orchestra.run.smithi083.stdout: "smtp_ssl": { 2024-08-19T01:43:32.387 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "smtp_ssl", 2024-08-19T01:43:32.387 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-08-19T01:43:32.387 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.387 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-08-19T01:43:32.387 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "True", 2024-08-19T01:43:32.387 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.387 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.387 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:32.387 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Use SSL to connect to SMTP server", 2024-08-19T01:43:32.388 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.388 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.388 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.388 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.388 INFO:teuthology.orchestra.run.smithi083.stdout: "smtp_user": { 2024-08-19T01:43:32.388 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "smtp_user", 2024-08-19T01:43:32.388 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-08-19T01:43:32.388 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.388 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-08-19T01:43:32.388 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-08-19T01:43:32.388 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.388 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.389 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:32.389 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "User to authenticate as", 2024-08-19T01:43:32.389 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.389 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.389 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.389 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-08-19T01:43:32.389 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-08-19T01:43:32.389 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.389 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-08-19T01:43:32.389 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "balancer", 2024-08-19T01:43:32.389 INFO:teuthology.orchestra.run.smithi083.stdout: "can_run": true, 2024-08-19T01:43:32.389 INFO:teuthology.orchestra.run.smithi083.stdout: "error_string": "", 2024-08-19T01:43:32.389 INFO:teuthology.orchestra.run.smithi083.stdout: "module_options": { 2024-08-19T01:43:32.389 INFO:teuthology.orchestra.run.smithi083.stdout: "active": { 2024-08-19T01:43:32.390 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "active", 2024-08-19T01:43:32.390 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-08-19T01:43:32.390 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.390 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-08-19T01:43:32.390 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "True", 2024-08-19T01:43:32.390 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.390 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.390 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:32.390 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "automatically balance PGs across cluster", 2024-08-19T01:43:32.390 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.390 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.390 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.390 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.391 INFO:teuthology.orchestra.run.smithi083.stdout: "begin_time": { 2024-08-19T01:43:32.391 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "begin_time", 2024-08-19T01:43:32.391 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-08-19T01:43:32.391 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.391 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-08-19T01:43:32.391 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "0000", 2024-08-19T01:43:32.391 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.391 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.391 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:32.391 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "beginning time of day to automatically balance", 2024-08-19T01:43:32.391 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-08-19T01:43:32.391 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.391 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.392 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.392 INFO:teuthology.orchestra.run.smithi083.stdout: "begin_weekday": { 2024-08-19T01:43:32.392 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "begin_weekday", 2024-08-19T01:43:32.392 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "uint", 2024-08-19T01:43:32.392 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.392 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-08-19T01:43:32.392 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "0", 2024-08-19T01:43:32.392 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "0", 2024-08-19T01:43:32.392 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "6", 2024-08-19T01:43:32.392 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:32.392 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-08-19T01:43:32.392 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-08-19T01:43:32.392 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.392 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.392 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.393 INFO:teuthology.orchestra.run.smithi083.stdout: "crush_compat_max_iterations": { 2024-08-19T01:43:32.393 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "crush_compat_max_iterations", 2024-08-19T01:43:32.393 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "uint", 2024-08-19T01:43:32.393 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.393 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-08-19T01:43:32.393 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "25", 2024-08-19T01:43:32.393 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "1", 2024-08-19T01:43:32.393 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "250", 2024-08-19T01:43:32.393 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:32.393 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-08-19T01:43:32.393 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.393 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.393 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.393 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.393 INFO:teuthology.orchestra.run.smithi083.stdout: "crush_compat_metrics": { 2024-08-19T01:43:32.394 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "crush_compat_metrics", 2024-08-19T01:43:32.394 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-08-19T01:43:32.394 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.394 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-08-19T01:43:32.394 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "pgs,objects,bytes", 2024-08-19T01:43:32.394 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.394 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.394 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:32.394 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-08-19T01:43:32.394 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-08-19T01:43:32.394 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.394 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.394 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.395 INFO:teuthology.orchestra.run.smithi083.stdout: "crush_compat_step": { 2024-08-19T01:43:32.395 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "crush_compat_step", 2024-08-19T01:43:32.395 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "float", 2024-08-19T01:43:32.395 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.395 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-08-19T01:43:32.395 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "0.5", 2024-08-19T01:43:32.395 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "0.001", 2024-08-19T01:43:32.395 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "0.999", 2024-08-19T01:43:32.395 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:32.395 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "aggressiveness of optimization", 2024-08-19T01:43:32.395 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-08-19T01:43:32.395 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.395 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.395 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.395 INFO:teuthology.orchestra.run.smithi083.stdout: "end_time": { 2024-08-19T01:43:32.396 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "end_time", 2024-08-19T01:43:32.396 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-08-19T01:43:32.396 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.396 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-08-19T01:43:32.396 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "2359", 2024-08-19T01:43:32.396 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.396 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.396 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:32.396 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "ending time of day to automatically balance", 2024-08-19T01:43:32.396 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-08-19T01:43:32.396 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.396 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.396 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.396 INFO:teuthology.orchestra.run.smithi083.stdout: "end_weekday": { 2024-08-19T01:43:32.396 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "end_weekday", 2024-08-19T01:43:32.397 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "uint", 2024-08-19T01:43:32.397 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.397 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-08-19T01:43:32.397 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "0", 2024-08-19T01:43:32.397 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "0", 2024-08-19T01:43:32.397 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "6", 2024-08-19T01:43:32.397 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:32.397 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-08-19T01:43:32.397 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-08-19T01:43:32.397 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.397 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.397 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.397 INFO:teuthology.orchestra.run.smithi083.stdout: "log_level": { 2024-08-19T01:43:32.397 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_level", 2024-08-19T01:43:32.397 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-08-19T01:43:32.398 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.398 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-08-19T01:43:32.398 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-08-19T01:43:32.398 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.398 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.398 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-08-19T01:43:32.398 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-08-19T01:43:32.398 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-08-19T01:43:32.398 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-08-19T01:43:32.398 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-08-19T01:43:32.398 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-08-19T01:43:32.398 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-08-19T01:43:32.398 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-08-19T01:43:32.398 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-08-19T01:43:32.398 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.399 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.399 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.399 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.399 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster": { 2024-08-19T01:43:32.399 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster", 2024-08-19T01:43:32.399 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-08-19T01:43:32.399 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.399 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-08-19T01:43:32.399 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-08-19T01:43:32.399 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.399 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.399 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:32.399 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-08-19T01:43:32.399 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.399 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.400 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.400 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.400 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster_level": { 2024-08-19T01:43:32.400 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster_level", 2024-08-19T01:43:32.400 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-08-19T01:43:32.400 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.400 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-08-19T01:43:32.400 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "info", 2024-08-19T01:43:32.400 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.400 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.400 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-08-19T01:43:32.400 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-08-19T01:43:32.400 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-08-19T01:43:32.400 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-08-19T01:43:32.401 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-08-19T01:43:32.401 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-08-19T01:43:32.401 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-08-19T01:43:32.401 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-08-19T01:43:32.401 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-08-19T01:43:32.401 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.401 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.401 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.401 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.401 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_file": { 2024-08-19T01:43:32.401 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_file", 2024-08-19T01:43:32.401 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-08-19T01:43:32.402 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.402 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-08-19T01:43:32.402 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-08-19T01:43:32.402 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.402 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.402 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:32.402 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-08-19T01:43:32.402 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.402 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.402 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.402 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.402 INFO:teuthology.orchestra.run.smithi083.stdout: "min_score": { 2024-08-19T01:43:32.403 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "min_score", 2024-08-19T01:43:32.403 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "float", 2024-08-19T01:43:32.403 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.403 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-08-19T01:43:32.403 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "0", 2024-08-19T01:43:32.403 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.403 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.403 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:32.403 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-08-19T01:43:32.403 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.403 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.403 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.403 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.403 INFO:teuthology.orchestra.run.smithi083.stdout: "mode": { 2024-08-19T01:43:32.404 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "mode", 2024-08-19T01:43:32.404 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-08-19T01:43:32.404 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.404 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-08-19T01:43:32.404 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "upmap", 2024-08-19T01:43:32.404 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.404 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.404 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-08-19T01:43:32.404 INFO:teuthology.orchestra.run.smithi083.stdout: "crush-compat", 2024-08-19T01:43:32.404 INFO:teuthology.orchestra.run.smithi083.stdout: "none", 2024-08-19T01:43:32.404 INFO:teuthology.orchestra.run.smithi083.stdout: "read", 2024-08-19T01:43:32.404 INFO:teuthology.orchestra.run.smithi083.stdout: "upmap", 2024-08-19T01:43:32.404 INFO:teuthology.orchestra.run.smithi083.stdout: "upmap-read" 2024-08-19T01:43:32.405 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-08-19T01:43:32.405 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Balancer mode", 2024-08-19T01:43:32.405 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.405 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.405 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.405 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.405 INFO:teuthology.orchestra.run.smithi083.stdout: "pool_ids": { 2024-08-19T01:43:32.405 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "pool_ids", 2024-08-19T01:43:32.405 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-08-19T01:43:32.405 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.405 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-08-19T01:43:32.405 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-08-19T01:43:32.405 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.405 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.405 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:32.406 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-08-19T01:43:32.406 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.406 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.406 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.406 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.406 INFO:teuthology.orchestra.run.smithi083.stdout: "sleep_interval": { 2024-08-19T01:43:32.406 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "sleep_interval", 2024-08-19T01:43:32.406 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "secs", 2024-08-19T01:43:32.406 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.406 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-08-19T01:43:32.406 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "60", 2024-08-19T01:43:32.406 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.406 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.407 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:32.407 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-08-19T01:43:32.407 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.407 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.407 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.407 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.407 INFO:teuthology.orchestra.run.smithi083.stdout: "upmap_max_deviation": { 2024-08-19T01:43:32.407 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "upmap_max_deviation", 2024-08-19T01:43:32.407 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2024-08-19T01:43:32.407 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.407 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-08-19T01:43:32.407 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "5", 2024-08-19T01:43:32.407 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "1", 2024-08-19T01:43:32.408 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.408 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:32.408 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "deviation below which no optimization is attempted", 2024-08-19T01:43:32.408 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-08-19T01:43:32.408 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.408 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.408 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.408 INFO:teuthology.orchestra.run.smithi083.stdout: "upmap_max_optimizations": { 2024-08-19T01:43:32.408 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "upmap_max_optimizations", 2024-08-19T01:43:32.408 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "uint", 2024-08-19T01:43:32.408 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.408 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-08-19T01:43:32.408 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "10", 2024-08-19T01:43:32.408 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.409 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.409 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:32.409 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-08-19T01:43:32.409 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.409 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.409 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.409 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-08-19T01:43:32.409 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-08-19T01:43:32.409 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.409 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-08-19T01:43:32.409 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "cephadm", 2024-08-19T01:43:32.409 INFO:teuthology.orchestra.run.smithi083.stdout: "can_run": true, 2024-08-19T01:43:32.409 INFO:teuthology.orchestra.run.smithi083.stdout: "error_string": "", 2024-08-19T01:43:32.409 INFO:teuthology.orchestra.run.smithi083.stdout: "module_options": { 2024-08-19T01:43:32.409 INFO:teuthology.orchestra.run.smithi083.stdout: "agent_down_multiplier": { 2024-08-19T01:43:32.409 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "agent_down_multiplier", 2024-08-19T01:43:32.410 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "float", 2024-08-19T01:43:32.410 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.410 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-08-19T01:43:32.410 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "3.0", 2024-08-19T01:43:32.410 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.410 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.410 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:32.410 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-08-19T01:43:32.410 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.410 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.410 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.410 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.410 INFO:teuthology.orchestra.run.smithi083.stdout: "agent_refresh_rate": { 2024-08-19T01:43:32.410 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "agent_refresh_rate", 2024-08-19T01:43:32.411 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "secs", 2024-08-19T01:43:32.411 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.411 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-08-19T01:43:32.411 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "20", 2024-08-19T01:43:32.411 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.411 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.411 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:32.411 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-08-19T01:43:32.411 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.411 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.411 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.411 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.411 INFO:teuthology.orchestra.run.smithi083.stdout: "agent_starting_port": { 2024-08-19T01:43:32.411 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "agent_starting_port", 2024-08-19T01:43:32.412 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2024-08-19T01:43:32.412 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.412 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-08-19T01:43:32.412 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "4721", 2024-08-19T01:43:32.412 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.412 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.412 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:32.412 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-08-19T01:43:32.412 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.412 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.412 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.412 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.412 INFO:teuthology.orchestra.run.smithi083.stdout: "allow_ptrace": { 2024-08-19T01:43:32.413 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "allow_ptrace", 2024-08-19T01:43:32.413 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-08-19T01:43:32.413 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.413 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-08-19T01:43:32.413 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-08-19T01:43:32.413 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.413 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.413 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:32.413 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-08-19T01:43:32.413 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-08-19T01:43:32.413 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.413 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.413 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.414 INFO:teuthology.orchestra.run.smithi083.stdout: "autotune_interval": { 2024-08-19T01:43:32.414 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "autotune_interval", 2024-08-19T01:43:32.414 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "secs", 2024-08-19T01:43:32.414 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.414 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-08-19T01:43:32.414 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "600", 2024-08-19T01:43:32.414 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.414 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.414 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:32.414 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "how frequently to autotune daemon memory", 2024-08-19T01:43:32.414 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.414 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.415 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.415 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.415 INFO:teuthology.orchestra.run.smithi083.stdout: "autotune_memory_target_ratio": { 2024-08-19T01:43:32.415 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "autotune_memory_target_ratio", 2024-08-19T01:43:32.415 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "float", 2024-08-19T01:43:32.415 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.415 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-08-19T01:43:32.415 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "0.7", 2024-08-19T01:43:32.415 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.415 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.415 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:32.415 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-08-19T01:43:32.415 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.416 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.416 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.416 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.416 INFO:teuthology.orchestra.run.smithi083.stdout: "cephadm_log_destination": { 2024-08-19T01:43:32.416 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "cephadm_log_destination", 2024-08-19T01:43:32.416 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-08-19T01:43:32.416 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.416 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-08-19T01:43:32.416 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-08-19T01:43:32.416 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.416 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.416 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-08-19T01:43:32.416 INFO:teuthology.orchestra.run.smithi083.stdout: "file", 2024-08-19T01:43:32.416 INFO:teuthology.orchestra.run.smithi083.stdout: "file,syslog", 2024-08-19T01:43:32.417 INFO:teuthology.orchestra.run.smithi083.stdout: "syslog" 2024-08-19T01:43:32.417 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-08-19T01:43:32.417 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-08-19T01:43:32.417 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.417 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.417 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.417 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.417 INFO:teuthology.orchestra.run.smithi083.stdout: "cgroups_split": { 2024-08-19T01:43:32.417 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "cgroups_split", 2024-08-19T01:43:32.417 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-08-19T01:43:32.417 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.417 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-08-19T01:43:32.417 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "True", 2024-08-19T01:43:32.417 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.417 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.418 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:32.418 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-08-19T01:43:32.418 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.418 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.418 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.418 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.418 INFO:teuthology.orchestra.run.smithi083.stdout: "config_checks_enabled": { 2024-08-19T01:43:32.418 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "config_checks_enabled", 2024-08-19T01:43:32.418 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-08-19T01:43:32.418 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.418 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-08-19T01:43:32.418 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-08-19T01:43:32.418 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.419 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.419 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:32.419 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-08-19T01:43:32.419 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.419 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.419 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.419 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.419 INFO:teuthology.orchestra.run.smithi083.stdout: "config_dashboard": { 2024-08-19T01:43:32.419 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "config_dashboard", 2024-08-19T01:43:32.419 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-08-19T01:43:32.419 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.419 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-08-19T01:43:32.419 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "True", 2024-08-19T01:43:32.419 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.419 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.420 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:32.420 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-08-19T01:43:32.420 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.420 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.420 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.420 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.420 INFO:teuthology.orchestra.run.smithi083.stdout: "container_image_alertmanager": { 2024-08-19T01:43:32.420 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "container_image_alertmanager", 2024-08-19T01:43:32.420 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-08-19T01:43:32.420 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.420 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-08-19T01:43:32.420 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-08-19T01:43:32.420 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.421 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.421 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:32.421 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Prometheus container image", 2024-08-19T01:43:32.421 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.421 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.421 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.421 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.421 INFO:teuthology.orchestra.run.smithi083.stdout: "container_image_base": { 2024-08-19T01:43:32.421 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "container_image_base", 2024-08-19T01:43:32.421 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-08-19T01:43:32.421 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.421 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-08-19T01:43:32.421 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "quay.io/ceph/ceph", 2024-08-19T01:43:32.422 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.422 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.422 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:32.422 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Container image name, without the tag", 2024-08-19T01:43:32.422 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.422 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.422 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.422 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.422 INFO:teuthology.orchestra.run.smithi083.stdout: "container_image_elasticsearch": { 2024-08-19T01:43:32.422 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "container_image_elasticsearch", 2024-08-19T01:43:32.422 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-08-19T01:43:32.422 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.422 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-08-19T01:43:32.423 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-08-19T01:43:32.423 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.423 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.423 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:32.423 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "elasticsearch container image", 2024-08-19T01:43:32.423 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.423 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.423 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.423 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.423 INFO:teuthology.orchestra.run.smithi083.stdout: "container_image_grafana": { 2024-08-19T01:43:32.423 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "container_image_grafana", 2024-08-19T01:43:32.423 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-08-19T01:43:32.423 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.423 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-08-19T01:43:32.424 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "quay.io/ceph/grafana:9.4.12", 2024-08-19T01:43:32.424 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.424 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.424 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:32.424 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Prometheus container image", 2024-08-19T01:43:32.424 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.424 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.424 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.424 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.424 INFO:teuthology.orchestra.run.smithi083.stdout: "container_image_haproxy": { 2024-08-19T01:43:32.424 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "container_image_haproxy", 2024-08-19T01:43:32.424 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-08-19T01:43:32.424 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.424 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-08-19T01:43:32.424 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-08-19T01:43:32.424 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.425 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.425 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:32.425 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "HAproxy container image", 2024-08-19T01:43:32.425 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.425 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.425 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.425 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.425 INFO:teuthology.orchestra.run.smithi083.stdout: "container_image_jaeger_agent": { 2024-08-19T01:43:32.425 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "container_image_jaeger_agent", 2024-08-19T01:43:32.425 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-08-19T01:43:32.425 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.425 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-08-19T01:43:32.425 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-08-19T01:43:32.426 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.426 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.426 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:32.426 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Jaeger agent container image", 2024-08-19T01:43:32.426 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.426 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.426 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.426 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.426 INFO:teuthology.orchestra.run.smithi083.stdout: "container_image_jaeger_collector": { 2024-08-19T01:43:32.426 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "container_image_jaeger_collector", 2024-08-19T01:43:32.426 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-08-19T01:43:32.426 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.426 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-08-19T01:43:32.427 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-08-19T01:43:32.427 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.427 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.427 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:32.427 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Jaeger collector container image", 2024-08-19T01:43:32.427 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.427 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.427 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.427 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.427 INFO:teuthology.orchestra.run.smithi083.stdout: "container_image_jaeger_query": { 2024-08-19T01:43:32.427 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "container_image_jaeger_query", 2024-08-19T01:43:32.427 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-08-19T01:43:32.427 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.427 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-08-19T01:43:32.428 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-08-19T01:43:32.428 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.428 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.428 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:32.428 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Jaeger query container image", 2024-08-19T01:43:32.428 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.428 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.428 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.428 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.428 INFO:teuthology.orchestra.run.smithi083.stdout: "container_image_keepalived": { 2024-08-19T01:43:32.428 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "container_image_keepalived", 2024-08-19T01:43:32.428 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-08-19T01:43:32.428 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.428 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-08-19T01:43:32.428 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-08-19T01:43:32.429 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.429 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.429 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:32.429 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Keepalived container image", 2024-08-19T01:43:32.429 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.429 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.429 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.429 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.429 INFO:teuthology.orchestra.run.smithi083.stdout: "container_image_loki": { 2024-08-19T01:43:32.429 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "container_image_loki", 2024-08-19T01:43:32.429 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-08-19T01:43:32.429 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.429 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-08-19T01:43:32.429 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "docker.io/grafana/loki:3.0.0", 2024-08-19T01:43:32.430 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.430 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.430 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:32.430 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Loki container image", 2024-08-19T01:43:32.430 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.430 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.430 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.430 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.430 INFO:teuthology.orchestra.run.smithi083.stdout: "container_image_node_exporter": { 2024-08-19T01:43:32.430 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "container_image_node_exporter", 2024-08-19T01:43:32.430 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-08-19T01:43:32.430 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.430 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-08-19T01:43:32.430 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-08-19T01:43:32.431 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.431 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.431 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:32.431 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Prometheus container image", 2024-08-19T01:43:32.431 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.431 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.431 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.431 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.431 INFO:teuthology.orchestra.run.smithi083.stdout: "container_image_nvmeof": { 2024-08-19T01:43:32.431 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "container_image_nvmeof", 2024-08-19T01:43:32.431 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-08-19T01:43:32.431 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.432 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-08-19T01:43:32.432 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "quay.io/ceph/nvmeof:1.2.5", 2024-08-19T01:43:32.432 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.432 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.432 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:32.432 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Nvme-of container image", 2024-08-19T01:43:32.432 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.432 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.432 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.432 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.432 INFO:teuthology.orchestra.run.smithi083.stdout: "container_image_prometheus": { 2024-08-19T01:43:32.432 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "container_image_prometheus", 2024-08-19T01:43:32.432 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-08-19T01:43:32.432 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.433 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-08-19T01:43:32.433 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-08-19T01:43:32.433 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.433 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.433 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:32.433 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Prometheus container image", 2024-08-19T01:43:32.433 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.433 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.433 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.433 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.433 INFO:teuthology.orchestra.run.smithi083.stdout: "container_image_promtail": { 2024-08-19T01:43:32.433 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "container_image_promtail", 2024-08-19T01:43:32.433 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-08-19T01:43:32.433 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.433 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-08-19T01:43:32.434 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "docker.io/grafana/promtail:3.0.0", 2024-08-19T01:43:32.434 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.434 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.434 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:32.434 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Promtail container image", 2024-08-19T01:43:32.434 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.434 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.434 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.434 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.434 INFO:teuthology.orchestra.run.smithi083.stdout: "container_image_samba": { 2024-08-19T01:43:32.434 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "container_image_samba", 2024-08-19T01:43:32.434 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-08-19T01:43:32.434 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.434 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-08-19T01:43:32.434 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "quay.io/samba.org/samba-server:devbuilds-centos-amd64", 2024-08-19T01:43:32.435 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.435 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.435 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:32.435 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Samba/SMB container image", 2024-08-19T01:43:32.435 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.435 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.435 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.435 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.435 INFO:teuthology.orchestra.run.smithi083.stdout: "container_image_snmp_gateway": { 2024-08-19T01:43:32.435 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "container_image_snmp_gateway", 2024-08-19T01:43:32.435 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-08-19T01:43:32.435 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.435 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-08-19T01:43:32.435 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-08-19T01:43:32.435 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.436 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.436 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:32.436 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "SNMP Gateway container image", 2024-08-19T01:43:32.436 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.436 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.436 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.436 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.436 INFO:teuthology.orchestra.run.smithi083.stdout: "container_init": { 2024-08-19T01:43:32.436 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "container_init", 2024-08-19T01:43:32.436 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-08-19T01:43:32.436 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.436 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-08-19T01:43:32.436 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "True", 2024-08-19T01:43:32.436 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.436 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.437 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:32.437 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Run podman/docker with `--init`", 2024-08-19T01:43:32.437 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.437 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.437 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.437 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.437 INFO:teuthology.orchestra.run.smithi083.stdout: "daemon_cache_timeout": { 2024-08-19T01:43:32.437 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "daemon_cache_timeout", 2024-08-19T01:43:32.437 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "secs", 2024-08-19T01:43:32.437 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.437 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-08-19T01:43:32.437 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "600", 2024-08-19T01:43:32.438 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.438 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.438 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:32.438 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-08-19T01:43:32.438 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.438 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.438 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.438 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.438 INFO:teuthology.orchestra.run.smithi083.stdout: "default_cephadm_command_timeout": { 2024-08-19T01:43:32.438 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "default_cephadm_command_timeout", 2024-08-19T01:43:32.438 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2024-08-19T01:43:32.439 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.439 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-08-19T01:43:32.439 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "900", 2024-08-19T01:43:32.439 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.439 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.439 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:32.439 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-08-19T01:43:32.439 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.439 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.439 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.439 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.439 INFO:teuthology.orchestra.run.smithi083.stdout: "default_registry": { 2024-08-19T01:43:32.439 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "default_registry", 2024-08-19T01:43:32.439 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-08-19T01:43:32.440 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.440 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-08-19T01:43:32.440 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "docker.io", 2024-08-19T01:43:32.440 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.440 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.440 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:32.440 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-08-19T01:43:32.440 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.440 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.440 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.440 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.440 INFO:teuthology.orchestra.run.smithi083.stdout: "device_cache_timeout": { 2024-08-19T01:43:32.441 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "device_cache_timeout", 2024-08-19T01:43:32.441 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "secs", 2024-08-19T01:43:32.441 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.441 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-08-19T01:43:32.441 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "1800", 2024-08-19T01:43:32.441 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.441 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.441 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:32.441 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "seconds to cache device inventory", 2024-08-19T01:43:32.441 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.441 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.441 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.442 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.442 INFO:teuthology.orchestra.run.smithi083.stdout: "device_enhanced_scan": { 2024-08-19T01:43:32.442 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "device_enhanced_scan", 2024-08-19T01:43:32.442 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-08-19T01:43:32.442 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.442 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-08-19T01:43:32.442 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-08-19T01:43:32.442 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.442 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.442 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:32.442 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Use libstoragemgmt during device scans", 2024-08-19T01:43:32.442 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.442 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.443 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.443 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.443 INFO:teuthology.orchestra.run.smithi083.stdout: "facts_cache_timeout": { 2024-08-19T01:43:32.443 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "facts_cache_timeout", 2024-08-19T01:43:32.443 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "secs", 2024-08-19T01:43:32.443 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.443 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-08-19T01:43:32.443 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "60", 2024-08-19T01:43:32.443 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.443 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.443 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:32.443 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "seconds to cache host facts data", 2024-08-19T01:43:32.443 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.443 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.444 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.444 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.444 INFO:teuthology.orchestra.run.smithi083.stdout: "grafana_dashboards_path": { 2024-08-19T01:43:32.444 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "grafana_dashboards_path", 2024-08-19T01:43:32.444 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-08-19T01:43:32.444 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.444 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-08-19T01:43:32.444 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "/etc/grafana/dashboards/ceph-dashboard/", 2024-08-19T01:43:32.444 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.444 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.444 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:32.444 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "location of dashboards to include in grafana deployments", 2024-08-19T01:43:32.444 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.445 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.445 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.445 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.445 INFO:teuthology.orchestra.run.smithi083.stdout: "host_check_interval": { 2024-08-19T01:43:32.445 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "host_check_interval", 2024-08-19T01:43:32.445 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "secs", 2024-08-19T01:43:32.445 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.445 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-08-19T01:43:32.445 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "600", 2024-08-19T01:43:32.445 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.445 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.445 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:32.445 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "how frequently to perform a host check", 2024-08-19T01:43:32.445 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.445 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.446 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.446 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.446 INFO:teuthology.orchestra.run.smithi083.stdout: "hw_monitoring": { 2024-08-19T01:43:32.446 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "hw_monitoring", 2024-08-19T01:43:32.446 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-08-19T01:43:32.446 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.446 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-08-19T01:43:32.446 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-08-19T01:43:32.446 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.446 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.446 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:32.447 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-08-19T01:43:32.447 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.447 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.447 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.447 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.447 INFO:teuthology.orchestra.run.smithi083.stdout: "inventory_list_all": { 2024-08-19T01:43:32.447 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "inventory_list_all", 2024-08-19T01:43:32.447 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-08-19T01:43:32.447 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.447 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-08-19T01:43:32.447 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-08-19T01:43:32.447 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.447 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.447 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:32.447 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-08-19T01:43:32.448 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.448 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.448 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.448 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.448 INFO:teuthology.orchestra.run.smithi083.stdout: "log_level": { 2024-08-19T01:43:32.448 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_level", 2024-08-19T01:43:32.448 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-08-19T01:43:32.448 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.448 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-08-19T01:43:32.448 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-08-19T01:43:32.448 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.448 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.448 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-08-19T01:43:32.448 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-08-19T01:43:32.448 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-08-19T01:43:32.448 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-08-19T01:43:32.449 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-08-19T01:43:32.449 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-08-19T01:43:32.449 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-08-19T01:43:32.449 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-08-19T01:43:32.449 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-08-19T01:43:32.449 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.449 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.449 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.449 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.449 INFO:teuthology.orchestra.run.smithi083.stdout: "log_refresh_metadata": { 2024-08-19T01:43:32.449 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_refresh_metadata", 2024-08-19T01:43:32.449 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-08-19T01:43:32.449 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.449 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-08-19T01:43:32.450 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-08-19T01:43:32.450 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.450 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.450 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:32.450 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-08-19T01:43:32.450 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.450 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.450 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.450 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.450 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster": { 2024-08-19T01:43:32.450 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster", 2024-08-19T01:43:32.450 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-08-19T01:43:32.450 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.451 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-08-19T01:43:32.451 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "True", 2024-08-19T01:43:32.451 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.451 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.451 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:32.451 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-08-19T01:43:32.451 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.451 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.451 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.451 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.451 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster_level": { 2024-08-19T01:43:32.451 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster_level", 2024-08-19T01:43:32.451 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-08-19T01:43:32.452 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.452 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-08-19T01:43:32.452 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "info", 2024-08-19T01:43:32.452 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.452 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.452 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-08-19T01:43:32.452 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-08-19T01:43:32.452 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-08-19T01:43:32.452 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-08-19T01:43:32.452 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-08-19T01:43:32.452 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-08-19T01:43:32.452 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-08-19T01:43:32.452 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-08-19T01:43:32.452 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-08-19T01:43:32.453 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.453 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.453 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.453 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.453 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_file": { 2024-08-19T01:43:32.453 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_file", 2024-08-19T01:43:32.453 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-08-19T01:43:32.453 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.453 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-08-19T01:43:32.453 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-08-19T01:43:32.453 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.453 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.453 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:32.453 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-08-19T01:43:32.453 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.454 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.454 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.454 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.454 INFO:teuthology.orchestra.run.smithi083.stdout: "manage_etc_ceph_ceph_conf": { 2024-08-19T01:43:32.454 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-08-19T01:43:32.454 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-08-19T01:43:32.454 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.454 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-08-19T01:43:32.454 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-08-19T01:43:32.454 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.454 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.454 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:32.454 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-08-19T01:43:32.454 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.455 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.455 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.455 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.455 INFO:teuthology.orchestra.run.smithi083.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-08-19T01:43:32.455 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-08-19T01:43:32.455 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-08-19T01:43:32.455 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.455 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-08-19T01:43:32.455 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "*", 2024-08-19T01:43:32.455 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.455 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.455 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:32.455 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-08-19T01:43:32.455 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.456 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.456 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.456 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.456 INFO:teuthology.orchestra.run.smithi083.stdout: "max_count_per_host": { 2024-08-19T01:43:32.456 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "max_count_per_host", 2024-08-19T01:43:32.456 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2024-08-19T01:43:32.456 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.456 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-08-19T01:43:32.456 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "10", 2024-08-19T01:43:32.456 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.456 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.456 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:32.456 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "max number of daemons per service per host", 2024-08-19T01:43:32.456 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.456 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.457 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.457 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.457 INFO:teuthology.orchestra.run.smithi083.stdout: "max_osd_draining_count": { 2024-08-19T01:43:32.457 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "max_osd_draining_count", 2024-08-19T01:43:32.457 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2024-08-19T01:43:32.457 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.457 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-08-19T01:43:32.457 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "10", 2024-08-19T01:43:32.457 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.457 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.457 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:32.457 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-08-19T01:43:32.457 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.457 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.458 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.458 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.458 INFO:teuthology.orchestra.run.smithi083.stdout: "migration_current": { 2024-08-19T01:43:32.458 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "migration_current", 2024-08-19T01:43:32.458 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2024-08-19T01:43:32.458 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.458 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-08-19T01:43:32.458 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-08-19T01:43:32.458 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.458 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.458 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:32.458 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "internal - do not modify", 2024-08-19T01:43:32.458 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.459 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.459 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.459 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.459 INFO:teuthology.orchestra.run.smithi083.stdout: "mode": { 2024-08-19T01:43:32.459 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "mode", 2024-08-19T01:43:32.459 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-08-19T01:43:32.459 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.459 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-08-19T01:43:32.459 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "root", 2024-08-19T01:43:32.459 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.459 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.459 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-08-19T01:43:32.459 INFO:teuthology.orchestra.run.smithi083.stdout: "cephadm-package", 2024-08-19T01:43:32.459 INFO:teuthology.orchestra.run.smithi083.stdout: "root" 2024-08-19T01:43:32.459 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-08-19T01:43:32.460 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "mode for remote execution of cephadm", 2024-08-19T01:43:32.460 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.460 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.460 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.460 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.460 INFO:teuthology.orchestra.run.smithi083.stdout: "oob_default_addr": { 2024-08-19T01:43:32.460 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "oob_default_addr", 2024-08-19T01:43:32.460 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-08-19T01:43:32.460 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.460 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-08-19T01:43:32.460 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "169.254.1.1", 2024-08-19T01:43:32.460 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.460 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.460 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:32.460 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Default address for RedFish API (oob management).", 2024-08-19T01:43:32.461 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.461 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.461 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.461 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.461 INFO:teuthology.orchestra.run.smithi083.stdout: "prometheus_alerts_path": { 2024-08-19T01:43:32.461 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "prometheus_alerts_path", 2024-08-19T01:43:32.461 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-08-19T01:43:32.461 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.461 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-08-19T01:43:32.461 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-08-19T01:43:32.461 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.461 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.461 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:32.461 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-08-19T01:43:32.461 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.462 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.462 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.462 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.462 INFO:teuthology.orchestra.run.smithi083.stdout: "registry_insecure": { 2024-08-19T01:43:32.462 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "registry_insecure", 2024-08-19T01:43:32.462 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-08-19T01:43:32.462 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.462 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-08-19T01:43:32.462 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-08-19T01:43:32.463 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.463 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.463 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:32.463 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-08-19T01:43:32.463 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.463 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.463 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.463 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.463 INFO:teuthology.orchestra.run.smithi083.stdout: "registry_password": { 2024-08-19T01:43:32.463 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "registry_password", 2024-08-19T01:43:32.463 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-08-19T01:43:32.463 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.463 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-08-19T01:43:32.463 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-08-19T01:43:32.464 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.464 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.464 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:32.464 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-08-19T01:43:32.464 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.464 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.464 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.464 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.464 INFO:teuthology.orchestra.run.smithi083.stdout: "registry_url": { 2024-08-19T01:43:32.464 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "registry_url", 2024-08-19T01:43:32.464 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-08-19T01:43:32.464 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.464 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-08-19T01:43:32.464 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-08-19T01:43:32.464 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.465 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.465 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:32.465 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-08-19T01:43:32.465 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.465 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.465 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.465 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.465 INFO:teuthology.orchestra.run.smithi083.stdout: "registry_username": { 2024-08-19T01:43:32.465 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "registry_username", 2024-08-19T01:43:32.465 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-08-19T01:43:32.465 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.465 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-08-19T01:43:32.465 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-08-19T01:43:32.465 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.466 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.466 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:32.466 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-08-19T01:43:32.466 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.466 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.466 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.466 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.466 INFO:teuthology.orchestra.run.smithi083.stdout: "secure_monitoring_stack": { 2024-08-19T01:43:32.466 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "secure_monitoring_stack", 2024-08-19T01:43:32.466 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-08-19T01:43:32.466 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.467 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-08-19T01:43:32.467 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-08-19T01:43:32.467 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.467 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.467 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:32.467 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-08-19T01:43:32.467 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.467 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.467 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.467 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.467 INFO:teuthology.orchestra.run.smithi083.stdout: "service_discovery_port": { 2024-08-19T01:43:32.467 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "service_discovery_port", 2024-08-19T01:43:32.467 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2024-08-19T01:43:32.468 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.468 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-08-19T01:43:32.468 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "8765", 2024-08-19T01:43:32.468 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.468 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.468 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:32.468 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "cephadm service discovery port", 2024-08-19T01:43:32.468 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.468 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.468 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.468 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.468 INFO:teuthology.orchestra.run.smithi083.stdout: "ssh_config_file": { 2024-08-19T01:43:32.468 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "ssh_config_file", 2024-08-19T01:43:32.468 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-08-19T01:43:32.469 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.469 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-08-19T01:43:32.469 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-08-19T01:43:32.469 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.469 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.469 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:32.469 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-08-19T01:43:32.469 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.469 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.469 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.469 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.469 INFO:teuthology.orchestra.run.smithi083.stdout: "use_agent": { 2024-08-19T01:43:32.469 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "use_agent", 2024-08-19T01:43:32.470 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-08-19T01:43:32.470 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.470 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-08-19T01:43:32.470 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-08-19T01:43:32.470 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.470 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.470 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:32.470 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-08-19T01:43:32.470 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.470 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.470 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.470 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.471 INFO:teuthology.orchestra.run.smithi083.stdout: "use_repo_digest": { 2024-08-19T01:43:32.471 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "use_repo_digest", 2024-08-19T01:43:32.471 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-08-19T01:43:32.471 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.471 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-08-19T01:43:32.471 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "True", 2024-08-19T01:43:32.471 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.471 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.471 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:32.471 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-08-19T01:43:32.471 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.471 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.471 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.472 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.472 INFO:teuthology.orchestra.run.smithi083.stdout: "warn_on_failed_host_check": { 2024-08-19T01:43:32.472 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "warn_on_failed_host_check", 2024-08-19T01:43:32.472 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-08-19T01:43:32.472 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.472 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-08-19T01:43:32.472 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "True", 2024-08-19T01:43:32.472 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.472 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.472 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:32.472 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "raise a health warning if the host check fails", 2024-08-19T01:43:32.472 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.472 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.473 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.473 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.473 INFO:teuthology.orchestra.run.smithi083.stdout: "warn_on_stray_daemons": { 2024-08-19T01:43:32.473 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "warn_on_stray_daemons", 2024-08-19T01:43:32.473 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-08-19T01:43:32.473 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.473 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-08-19T01:43:32.473 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "True", 2024-08-19T01:43:32.473 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.473 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.473 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:32.473 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-08-19T01:43:32.473 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.473 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.474 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.474 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.474 INFO:teuthology.orchestra.run.smithi083.stdout: "warn_on_stray_hosts": { 2024-08-19T01:43:32.474 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "warn_on_stray_hosts", 2024-08-19T01:43:32.474 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-08-19T01:43:32.474 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.474 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-08-19T01:43:32.474 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "True", 2024-08-19T01:43:32.474 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.474 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.474 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:32.474 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-08-19T01:43:32.474 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.475 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.475 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.475 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-08-19T01:43:32.475 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-08-19T01:43:32.475 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.475 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-08-19T01:43:32.475 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "crash", 2024-08-19T01:43:32.475 INFO:teuthology.orchestra.run.smithi083.stdout: "can_run": true, 2024-08-19T01:43:32.475 INFO:teuthology.orchestra.run.smithi083.stdout: "error_string": "", 2024-08-19T01:43:32.475 INFO:teuthology.orchestra.run.smithi083.stdout: "module_options": { 2024-08-19T01:43:32.475 INFO:teuthology.orchestra.run.smithi083.stdout: "log_level": { 2024-08-19T01:43:32.475 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_level", 2024-08-19T01:43:32.475 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-08-19T01:43:32.475 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.476 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-08-19T01:43:32.476 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-08-19T01:43:32.476 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.476 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.476 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-08-19T01:43:32.476 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-08-19T01:43:32.476 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-08-19T01:43:32.476 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-08-19T01:43:32.476 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-08-19T01:43:32.476 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-08-19T01:43:32.476 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-08-19T01:43:32.476 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-08-19T01:43:32.477 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-08-19T01:43:32.477 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.477 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.477 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.477 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.477 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster": { 2024-08-19T01:43:32.477 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster", 2024-08-19T01:43:32.477 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-08-19T01:43:32.477 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.477 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-08-19T01:43:32.477 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-08-19T01:43:32.477 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.477 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.477 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:32.478 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-08-19T01:43:32.478 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.478 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.478 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.478 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.478 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster_level": { 2024-08-19T01:43:32.478 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster_level", 2024-08-19T01:43:32.478 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-08-19T01:43:32.478 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.478 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-08-19T01:43:32.478 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "info", 2024-08-19T01:43:32.479 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.479 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.479 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-08-19T01:43:32.479 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-08-19T01:43:32.479 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-08-19T01:43:32.479 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-08-19T01:43:32.479 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-08-19T01:43:32.479 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-08-19T01:43:32.479 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-08-19T01:43:32.479 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-08-19T01:43:32.479 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-08-19T01:43:32.479 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.479 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.480 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.480 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.480 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_file": { 2024-08-19T01:43:32.480 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_file", 2024-08-19T01:43:32.480 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-08-19T01:43:32.480 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.480 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-08-19T01:43:32.480 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-08-19T01:43:32.480 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.480 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.480 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:32.480 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-08-19T01:43:32.480 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.480 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.481 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.481 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.481 INFO:teuthology.orchestra.run.smithi083.stdout: "retain_interval": { 2024-08-19T01:43:32.481 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "retain_interval", 2024-08-19T01:43:32.481 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "secs", 2024-08-19T01:43:32.481 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.481 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-08-19T01:43:32.481 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "31536000", 2024-08-19T01:43:32.481 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.481 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.481 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:32.481 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "how long to retain crashes before pruning them", 2024-08-19T01:43:32.481 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.482 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.482 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.482 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.482 INFO:teuthology.orchestra.run.smithi083.stdout: "warn_recent_interval": { 2024-08-19T01:43:32.482 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "warn_recent_interval", 2024-08-19T01:43:32.482 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "secs", 2024-08-19T01:43:32.482 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.482 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-08-19T01:43:32.482 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "1209600", 2024-08-19T01:43:32.482 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.482 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.483 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:32.483 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "time interval in which to warn about recent crashes", 2024-08-19T01:43:32.483 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.483 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.483 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.483 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-08-19T01:43:32.483 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-08-19T01:43:32.483 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.483 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-08-19T01:43:32.483 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "dashboard", 2024-08-19T01:43:32.483 INFO:teuthology.orchestra.run.smithi083.stdout: "can_run": true, 2024-08-19T01:43:32.483 INFO:teuthology.orchestra.run.smithi083.stdout: "error_string": "", 2024-08-19T01:43:32.483 INFO:teuthology.orchestra.run.smithi083.stdout: "module_options": { 2024-08-19T01:43:32.484 INFO:teuthology.orchestra.run.smithi083.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-08-19T01:43:32.484 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-08-19T01:43:32.484 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2024-08-19T01:43:32.484 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.484 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-08-19T01:43:32.484 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "10", 2024-08-19T01:43:32.484 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.484 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.484 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:32.484 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-08-19T01:43:32.484 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.484 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.484 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.485 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.485 INFO:teuthology.orchestra.run.smithi083.stdout: "ALERTMANAGER_API_HOST": { 2024-08-19T01:43:32.485 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "ALERTMANAGER_API_HOST", 2024-08-19T01:43:32.485 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-08-19T01:43:32.485 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.485 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-08-19T01:43:32.485 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-08-19T01:43:32.485 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.485 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.485 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:32.485 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-08-19T01:43:32.485 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.485 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.486 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.486 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.486 INFO:teuthology.orchestra.run.smithi083.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-08-19T01:43:32.486 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-08-19T01:43:32.486 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-08-19T01:43:32.486 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.486 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-08-19T01:43:32.486 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "True", 2024-08-19T01:43:32.486 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.486 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.486 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:32.486 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-08-19T01:43:32.486 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.487 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.487 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.487 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.487 INFO:teuthology.orchestra.run.smithi083.stdout: "AUDIT_API_ENABLED": { 2024-08-19T01:43:32.487 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "AUDIT_API_ENABLED", 2024-08-19T01:43:32.487 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-08-19T01:43:32.487 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.487 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-08-19T01:43:32.487 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-08-19T01:43:32.487 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.487 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.487 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:32.487 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-08-19T01:43:32.487 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.488 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.488 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.488 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.488 INFO:teuthology.orchestra.run.smithi083.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-08-19T01:43:32.488 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-08-19T01:43:32.488 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-08-19T01:43:32.488 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.488 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-08-19T01:43:32.488 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "True", 2024-08-19T01:43:32.488 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.488 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.488 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:32.488 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-08-19T01:43:32.488 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.489 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.489 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.489 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.489 INFO:teuthology.orchestra.run.smithi083.stdout: "ENABLE_BROWSABLE_API": { 2024-08-19T01:43:32.489 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "ENABLE_BROWSABLE_API", 2024-08-19T01:43:32.489 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-08-19T01:43:32.489 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.489 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-08-19T01:43:32.489 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "True", 2024-08-19T01:43:32.489 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.489 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.489 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:32.489 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-08-19T01:43:32.489 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.490 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.490 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.490 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.490 INFO:teuthology.orchestra.run.smithi083.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-08-19T01:43:32.490 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-08-19T01:43:32.490 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-08-19T01:43:32.490 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.490 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-08-19T01:43:32.490 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "True", 2024-08-19T01:43:32.490 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.490 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.490 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:32.491 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-08-19T01:43:32.491 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.491 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.491 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.491 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.491 INFO:teuthology.orchestra.run.smithi083.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-08-19T01:43:32.491 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-08-19T01:43:32.491 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-08-19T01:43:32.491 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.491 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-08-19T01:43:32.491 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "True", 2024-08-19T01:43:32.491 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.492 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.492 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:32.492 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-08-19T01:43:32.492 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.492 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.492 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.492 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.492 INFO:teuthology.orchestra.run.smithi083.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-08-19T01:43:32.492 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-08-19T01:43:32.492 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-08-19T01:43:32.492 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.492 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-08-19T01:43:32.492 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "True", 2024-08-19T01:43:32.492 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.492 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.493 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:32.493 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-08-19T01:43:32.493 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.493 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.493 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.493 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.493 INFO:teuthology.orchestra.run.smithi083.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-08-19T01:43:32.493 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-08-19T01:43:32.493 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-08-19T01:43:32.493 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.493 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-08-19T01:43:32.493 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "True", 2024-08-19T01:43:32.493 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.493 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.494 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:32.494 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-08-19T01:43:32.494 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.494 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.494 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.494 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.494 INFO:teuthology.orchestra.run.smithi083.stdout: "FEATURE_TOGGLE_NFS": { 2024-08-19T01:43:32.494 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-08-19T01:43:32.494 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-08-19T01:43:32.494 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.494 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-08-19T01:43:32.494 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "True", 2024-08-19T01:43:32.494 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.495 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.495 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:32.495 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-08-19T01:43:32.495 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.495 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.495 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.495 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.495 INFO:teuthology.orchestra.run.smithi083.stdout: "FEATURE_TOGGLE_RBD": { 2024-08-19T01:43:32.495 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-08-19T01:43:32.495 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-08-19T01:43:32.495 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.495 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-08-19T01:43:32.495 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "True", 2024-08-19T01:43:32.495 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.496 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.496 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:32.496 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-08-19T01:43:32.496 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.496 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.496 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.496 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.496 INFO:teuthology.orchestra.run.smithi083.stdout: "FEATURE_TOGGLE_RGW": { 2024-08-19T01:43:32.496 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-08-19T01:43:32.496 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-08-19T01:43:32.496 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.496 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-08-19T01:43:32.496 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "True", 2024-08-19T01:43:32.496 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.497 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.497 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:32.497 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-08-19T01:43:32.497 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.497 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.497 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.497 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.497 INFO:teuthology.orchestra.run.smithi083.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-08-19T01:43:32.497 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-08-19T01:43:32.497 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-08-19T01:43:32.497 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.497 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-08-19T01:43:32.497 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-08-19T01:43:32.498 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.498 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.498 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:32.498 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-08-19T01:43:32.498 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.498 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.498 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.498 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.498 INFO:teuthology.orchestra.run.smithi083.stdout: "GRAFANA_API_PASSWORD": { 2024-08-19T01:43:32.498 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "GRAFANA_API_PASSWORD", 2024-08-19T01:43:32.498 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-08-19T01:43:32.498 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.498 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-08-19T01:43:32.499 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "admin", 2024-08-19T01:43:32.499 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.499 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.499 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:32.499 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-08-19T01:43:32.499 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.499 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.499 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.499 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.499 INFO:teuthology.orchestra.run.smithi083.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-08-19T01:43:32.499 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-08-19T01:43:32.499 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-08-19T01:43:32.499 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.500 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-08-19T01:43:32.500 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "True", 2024-08-19T01:43:32.500 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.500 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.500 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:32.500 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-08-19T01:43:32.500 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.500 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.500 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.500 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.500 INFO:teuthology.orchestra.run.smithi083.stdout: "GRAFANA_API_URL": { 2024-08-19T01:43:32.500 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "GRAFANA_API_URL", 2024-08-19T01:43:32.500 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-08-19T01:43:32.500 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.501 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-08-19T01:43:32.501 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-08-19T01:43:32.501 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.501 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.501 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:32.501 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-08-19T01:43:32.501 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.501 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.501 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.501 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.501 INFO:teuthology.orchestra.run.smithi083.stdout: "GRAFANA_API_USERNAME": { 2024-08-19T01:43:32.501 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "GRAFANA_API_USERNAME", 2024-08-19T01:43:32.501 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-08-19T01:43:32.502 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.502 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-08-19T01:43:32.502 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "admin", 2024-08-19T01:43:32.502 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.502 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.502 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:32.502 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-08-19T01:43:32.502 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.502 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.502 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.502 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.502 INFO:teuthology.orchestra.run.smithi083.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-08-19T01:43:32.502 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-08-19T01:43:32.503 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-08-19T01:43:32.503 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.503 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-08-19T01:43:32.503 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-08-19T01:43:32.503 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.503 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.503 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:32.503 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-08-19T01:43:32.503 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.503 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.503 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.503 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.504 INFO:teuthology.orchestra.run.smithi083.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-08-19T01:43:32.504 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-08-19T01:43:32.504 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-08-19T01:43:32.504 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.504 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-08-19T01:43:32.504 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-08-19T01:43:32.504 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.504 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.504 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:32.504 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-08-19T01:43:32.504 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.504 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.504 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.505 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.505 INFO:teuthology.orchestra.run.smithi083.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-08-19T01:43:32.505 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-08-19T01:43:32.505 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-08-19T01:43:32.505 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.505 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-08-19T01:43:32.505 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "True", 2024-08-19T01:43:32.505 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.505 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.505 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:32.505 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-08-19T01:43:32.505 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.505 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.506 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.506 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.506 INFO:teuthology.orchestra.run.smithi083.stdout: "ISSUE_TRACKER_API_KEY": { 2024-08-19T01:43:32.506 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-08-19T01:43:32.506 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-08-19T01:43:32.506 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.506 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-08-19T01:43:32.506 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-08-19T01:43:32.506 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.506 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.506 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:32.506 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-08-19T01:43:32.506 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.507 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.507 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.507 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.507 INFO:teuthology.orchestra.run.smithi083.stdout: "PROMETHEUS_API_HOST": { 2024-08-19T01:43:32.507 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "PROMETHEUS_API_HOST", 2024-08-19T01:43:32.507 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-08-19T01:43:32.507 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.507 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-08-19T01:43:32.507 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-08-19T01:43:32.507 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.507 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.507 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:32.507 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-08-19T01:43:32.507 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.508 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.508 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.508 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.508 INFO:teuthology.orchestra.run.smithi083.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-08-19T01:43:32.508 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-08-19T01:43:32.508 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-08-19T01:43:32.508 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.508 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-08-19T01:43:32.508 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "True", 2024-08-19T01:43:32.508 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.508 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.508 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:32.508 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-08-19T01:43:32.508 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.509 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.509 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.509 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.509 INFO:teuthology.orchestra.run.smithi083.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-08-19T01:43:32.509 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-08-19T01:43:32.509 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-08-19T01:43:32.509 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.509 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-08-19T01:43:32.509 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-08-19T01:43:32.509 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.509 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.509 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:32.509 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-08-19T01:43:32.509 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.510 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.510 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.510 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.510 INFO:teuthology.orchestra.run.smithi083.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-08-19T01:43:32.510 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-08-19T01:43:32.510 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-08-19T01:43:32.510 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.510 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-08-19T01:43:32.510 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-08-19T01:43:32.510 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.510 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.510 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:32.510 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-08-19T01:43:32.511 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.511 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.511 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.511 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.511 INFO:teuthology.orchestra.run.smithi083.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-08-19T01:43:32.511 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-08-19T01:43:32.511 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-08-19T01:43:32.511 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.511 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-08-19T01:43:32.511 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "True", 2024-08-19T01:43:32.511 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.511 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.512 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:32.512 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-08-19T01:43:32.512 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.512 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.512 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.512 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.512 INFO:teuthology.orchestra.run.smithi083.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-08-19T01:43:32.512 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-08-19T01:43:32.512 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-08-19T01:43:32.512 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.512 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-08-19T01:43:32.512 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "True", 2024-08-19T01:43:32.512 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.512 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.513 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:32.513 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-08-19T01:43:32.513 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.513 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.513 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.513 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.513 INFO:teuthology.orchestra.run.smithi083.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-08-19T01:43:32.513 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-08-19T01:43:32.513 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-08-19T01:43:32.513 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.513 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-08-19T01:43:32.513 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-08-19T01:43:32.513 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.513 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.513 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:32.514 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-08-19T01:43:32.514 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.514 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.514 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.514 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.514 INFO:teuthology.orchestra.run.smithi083.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-08-19T01:43:32.514 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-08-19T01:43:32.514 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-08-19T01:43:32.514 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.514 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-08-19T01:43:32.514 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-08-19T01:43:32.514 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.514 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.515 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:32.515 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-08-19T01:43:32.515 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.515 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.515 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.515 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.515 INFO:teuthology.orchestra.run.smithi083.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-08-19T01:43:32.515 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-08-19T01:43:32.515 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-08-19T01:43:32.515 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.515 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-08-19T01:43:32.515 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-08-19T01:43:32.515 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.515 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.515 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:32.516 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-08-19T01:43:32.516 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.516 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.516 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.516 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.516 INFO:teuthology.orchestra.run.smithi083.stdout: "PWD_POLICY_ENABLED": { 2024-08-19T01:43:32.516 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "PWD_POLICY_ENABLED", 2024-08-19T01:43:32.516 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-08-19T01:43:32.516 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.516 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-08-19T01:43:32.516 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "True", 2024-08-19T01:43:32.516 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.516 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.516 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:32.516 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-08-19T01:43:32.517 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.517 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.517 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.517 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.517 INFO:teuthology.orchestra.run.smithi083.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-08-19T01:43:32.517 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-08-19T01:43:32.517 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-08-19T01:43:32.517 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.517 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-08-19T01:43:32.517 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-08-19T01:43:32.517 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.517 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.517 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:32.517 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-08-19T01:43:32.518 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.518 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.518 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.518 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.518 INFO:teuthology.orchestra.run.smithi083.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-08-19T01:43:32.518 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-08-19T01:43:32.518 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2024-08-19T01:43:32.518 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.518 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-08-19T01:43:32.518 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "10", 2024-08-19T01:43:32.518 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.518 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.518 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:32.518 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-08-19T01:43:32.519 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.519 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.519 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.519 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.519 INFO:teuthology.orchestra.run.smithi083.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-08-19T01:43:32.519 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-08-19T01:43:32.519 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2024-08-19T01:43:32.519 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.519 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-08-19T01:43:32.519 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "8", 2024-08-19T01:43:32.519 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.519 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.519 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:32.519 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-08-19T01:43:32.519 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.520 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.520 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.520 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.520 INFO:teuthology.orchestra.run.smithi083.stdout: "REST_REQUESTS_TIMEOUT": { 2024-08-19T01:43:32.520 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-08-19T01:43:32.520 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2024-08-19T01:43:32.520 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.520 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-08-19T01:43:32.520 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "45", 2024-08-19T01:43:32.520 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.520 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.520 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:32.520 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-08-19T01:43:32.520 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.520 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.521 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.521 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.521 INFO:teuthology.orchestra.run.smithi083.stdout: "RGW_API_ACCESS_KEY": { 2024-08-19T01:43:32.521 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "RGW_API_ACCESS_KEY", 2024-08-19T01:43:32.521 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-08-19T01:43:32.521 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.521 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-08-19T01:43:32.521 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-08-19T01:43:32.521 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.521 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.521 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:32.521 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-08-19T01:43:32.521 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.521 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.522 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.522 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.522 INFO:teuthology.orchestra.run.smithi083.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-08-19T01:43:32.522 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-08-19T01:43:32.522 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-08-19T01:43:32.522 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.522 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-08-19T01:43:32.522 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "admin", 2024-08-19T01:43:32.522 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.522 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.522 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:32.522 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-08-19T01:43:32.522 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.522 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.523 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.523 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.523 INFO:teuthology.orchestra.run.smithi083.stdout: "RGW_API_SECRET_KEY": { 2024-08-19T01:43:32.523 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "RGW_API_SECRET_KEY", 2024-08-19T01:43:32.523 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-08-19T01:43:32.523 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.523 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-08-19T01:43:32.523 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-08-19T01:43:32.523 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.523 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.523 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:32.523 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-08-19T01:43:32.523 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.524 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.524 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.524 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.524 INFO:teuthology.orchestra.run.smithi083.stdout: "RGW_API_SSL_VERIFY": { 2024-08-19T01:43:32.524 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "RGW_API_SSL_VERIFY", 2024-08-19T01:43:32.524 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-08-19T01:43:32.524 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.524 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-08-19T01:43:32.524 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "True", 2024-08-19T01:43:32.524 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.524 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.524 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:32.524 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-08-19T01:43:32.525 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.525 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.525 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.525 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.525 INFO:teuthology.orchestra.run.smithi083.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-08-19T01:43:32.525 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-08-19T01:43:32.525 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2024-08-19T01:43:32.525 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.525 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-08-19T01:43:32.525 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "0", 2024-08-19T01:43:32.525 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.525 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.525 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:32.525 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-08-19T01:43:32.525 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.526 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.526 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.526 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.526 INFO:teuthology.orchestra.run.smithi083.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-08-19T01:43:32.526 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-08-19T01:43:32.526 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2024-08-19T01:43:32.526 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.526 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-08-19T01:43:32.526 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "10", 2024-08-19T01:43:32.526 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.526 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.526 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:32.526 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-08-19T01:43:32.526 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.527 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.527 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.527 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.527 INFO:teuthology.orchestra.run.smithi083.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-08-19T01:43:32.527 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-08-19T01:43:32.527 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2024-08-19T01:43:32.527 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.527 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-08-19T01:43:32.527 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "5", 2024-08-19T01:43:32.527 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.527 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.527 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:32.527 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-08-19T01:43:32.527 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.528 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.528 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.528 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.528 INFO:teuthology.orchestra.run.smithi083.stdout: "cross_origin_url": { 2024-08-19T01:43:32.528 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "cross_origin_url", 2024-08-19T01:43:32.528 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-08-19T01:43:32.528 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.528 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-08-19T01:43:32.528 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-08-19T01:43:32.528 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.528 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.528 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:32.528 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-08-19T01:43:32.528 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.529 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.529 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.529 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.529 INFO:teuthology.orchestra.run.smithi083.stdout: "crt_file": { 2024-08-19T01:43:32.529 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "crt_file", 2024-08-19T01:43:32.529 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-08-19T01:43:32.529 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.529 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-08-19T01:43:32.529 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-08-19T01:43:32.529 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.529 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.529 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:32.529 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-08-19T01:43:32.529 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.530 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.530 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.530 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.530 INFO:teuthology.orchestra.run.smithi083.stdout: "debug": { 2024-08-19T01:43:32.530 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "debug", 2024-08-19T01:43:32.530 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-08-19T01:43:32.530 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.530 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-08-19T01:43:32.530 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-08-19T01:43:32.530 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.530 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.530 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:32.530 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Enable/disable debug options", 2024-08-19T01:43:32.530 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.530 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.531 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.531 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.531 INFO:teuthology.orchestra.run.smithi083.stdout: "jwt_token_ttl": { 2024-08-19T01:43:32.531 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "jwt_token_ttl", 2024-08-19T01:43:32.531 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2024-08-19T01:43:32.531 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.531 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-08-19T01:43:32.531 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "28800", 2024-08-19T01:43:32.531 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.531 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.531 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:32.531 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-08-19T01:43:32.531 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.532 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.532 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.532 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.532 INFO:teuthology.orchestra.run.smithi083.stdout: "key_file": { 2024-08-19T01:43:32.532 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "key_file", 2024-08-19T01:43:32.532 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-08-19T01:43:32.532 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.532 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-08-19T01:43:32.532 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-08-19T01:43:32.532 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.532 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.532 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:32.532 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-08-19T01:43:32.532 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.533 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.533 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.533 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.533 INFO:teuthology.orchestra.run.smithi083.stdout: "log_level": { 2024-08-19T01:43:32.533 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_level", 2024-08-19T01:43:32.533 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-08-19T01:43:32.533 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.533 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-08-19T01:43:32.533 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-08-19T01:43:32.533 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.533 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.533 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-08-19T01:43:32.533 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-08-19T01:43:32.533 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-08-19T01:43:32.534 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-08-19T01:43:32.534 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-08-19T01:43:32.534 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-08-19T01:43:32.534 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-08-19T01:43:32.534 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-08-19T01:43:32.534 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-08-19T01:43:32.534 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.534 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.534 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.534 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.534 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster": { 2024-08-19T01:43:32.534 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster", 2024-08-19T01:43:32.534 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-08-19T01:43:32.535 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.535 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-08-19T01:43:32.535 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-08-19T01:43:32.535 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.535 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.535 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:32.535 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-08-19T01:43:32.535 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.535 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.535 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.535 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.535 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster_level": { 2024-08-19T01:43:32.535 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster_level", 2024-08-19T01:43:32.535 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-08-19T01:43:32.536 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.536 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-08-19T01:43:32.536 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "info", 2024-08-19T01:43:32.536 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.536 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.536 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-08-19T01:43:32.536 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-08-19T01:43:32.536 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-08-19T01:43:32.536 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-08-19T01:43:32.536 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-08-19T01:43:32.536 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-08-19T01:43:32.536 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-08-19T01:43:32.536 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-08-19T01:43:32.536 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-08-19T01:43:32.537 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.537 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.537 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.537 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.537 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_file": { 2024-08-19T01:43:32.537 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_file", 2024-08-19T01:43:32.537 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-08-19T01:43:32.537 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.537 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-08-19T01:43:32.537 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-08-19T01:43:32.537 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.537 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.537 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:32.537 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-08-19T01:43:32.537 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.538 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.538 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.538 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.538 INFO:teuthology.orchestra.run.smithi083.stdout: "motd": { 2024-08-19T01:43:32.538 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "motd", 2024-08-19T01:43:32.538 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-08-19T01:43:32.538 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.538 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-08-19T01:43:32.538 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-08-19T01:43:32.538 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.538 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.538 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:32.538 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "The message of the day", 2024-08-19T01:43:32.539 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.539 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.539 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.539 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.539 INFO:teuthology.orchestra.run.smithi083.stdout: "redirect_resolve_ip_addr": { 2024-08-19T01:43:32.539 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "redirect_resolve_ip_addr", 2024-08-19T01:43:32.539 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-08-19T01:43:32.539 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.539 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-08-19T01:43:32.539 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-08-19T01:43:32.539 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.539 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.539 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:32.539 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-08-19T01:43:32.540 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.540 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.540 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.540 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.540 INFO:teuthology.orchestra.run.smithi083.stdout: "server_addr": { 2024-08-19T01:43:32.540 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "server_addr", 2024-08-19T01:43:32.540 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-08-19T01:43:32.540 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.540 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-08-19T01:43:32.540 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "::", 2024-08-19T01:43:32.540 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.540 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.540 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:32.540 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-08-19T01:43:32.540 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.541 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.541 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.541 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.541 INFO:teuthology.orchestra.run.smithi083.stdout: "server_port": { 2024-08-19T01:43:32.541 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "server_port", 2024-08-19T01:43:32.541 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2024-08-19T01:43:32.541 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.541 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-08-19T01:43:32.541 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "8080", 2024-08-19T01:43:32.541 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.541 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.541 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:32.541 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-08-19T01:43:32.541 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.542 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.542 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.542 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.542 INFO:teuthology.orchestra.run.smithi083.stdout: "ssl": { 2024-08-19T01:43:32.542 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "ssl", 2024-08-19T01:43:32.542 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-08-19T01:43:32.542 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.542 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-08-19T01:43:32.542 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "True", 2024-08-19T01:43:32.542 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.542 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.542 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:32.542 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-08-19T01:43:32.543 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.543 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.543 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.543 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.543 INFO:teuthology.orchestra.run.smithi083.stdout: "ssl_server_port": { 2024-08-19T01:43:32.543 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "ssl_server_port", 2024-08-19T01:43:32.543 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2024-08-19T01:43:32.543 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.543 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-08-19T01:43:32.543 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "8443", 2024-08-19T01:43:32.543 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.543 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.543 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:32.544 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-08-19T01:43:32.544 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.544 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.544 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.544 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.544 INFO:teuthology.orchestra.run.smithi083.stdout: "standby_behaviour": { 2024-08-19T01:43:32.544 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "standby_behaviour", 2024-08-19T01:43:32.544 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-08-19T01:43:32.544 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.544 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-08-19T01:43:32.544 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "redirect", 2024-08-19T01:43:32.544 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.544 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.544 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-08-19T01:43:32.545 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-08-19T01:43:32.545 INFO:teuthology.orchestra.run.smithi083.stdout: "redirect" 2024-08-19T01:43:32.545 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-08-19T01:43:32.545 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-08-19T01:43:32.545 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.545 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.545 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.545 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.545 INFO:teuthology.orchestra.run.smithi083.stdout: "standby_error_status_code": { 2024-08-19T01:43:32.545 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "standby_error_status_code", 2024-08-19T01:43:32.545 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2024-08-19T01:43:32.545 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.545 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-08-19T01:43:32.546 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "500", 2024-08-19T01:43:32.546 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "400", 2024-08-19T01:43:32.546 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "599", 2024-08-19T01:43:32.546 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:32.546 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-08-19T01:43:32.546 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.546 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.546 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.546 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.546 INFO:teuthology.orchestra.run.smithi083.stdout: "url_prefix": { 2024-08-19T01:43:32.546 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "url_prefix", 2024-08-19T01:43:32.546 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-08-19T01:43:32.546 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.547 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-08-19T01:43:32.547 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-08-19T01:43:32.547 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.547 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.547 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:32.547 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-08-19T01:43:32.547 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.547 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.547 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.547 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-08-19T01:43:32.547 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-08-19T01:43:32.547 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.547 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-08-19T01:43:32.547 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "devicehealth", 2024-08-19T01:43:32.548 INFO:teuthology.orchestra.run.smithi083.stdout: "can_run": true, 2024-08-19T01:43:32.548 INFO:teuthology.orchestra.run.smithi083.stdout: "error_string": "", 2024-08-19T01:43:32.548 INFO:teuthology.orchestra.run.smithi083.stdout: "module_options": { 2024-08-19T01:43:32.548 INFO:teuthology.orchestra.run.smithi083.stdout: "enable_monitoring": { 2024-08-19T01:43:32.548 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "enable_monitoring", 2024-08-19T01:43:32.548 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-08-19T01:43:32.548 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.548 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-08-19T01:43:32.548 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "True", 2024-08-19T01:43:32.548 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.548 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.548 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:32.549 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "monitor device health metrics", 2024-08-19T01:43:32.549 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.549 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.549 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.549 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.549 INFO:teuthology.orchestra.run.smithi083.stdout: "log_level": { 2024-08-19T01:43:32.549 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_level", 2024-08-19T01:43:32.549 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-08-19T01:43:32.549 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.549 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-08-19T01:43:32.549 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-08-19T01:43:32.549 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.549 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.549 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-08-19T01:43:32.550 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-08-19T01:43:32.550 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-08-19T01:43:32.550 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-08-19T01:43:32.550 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-08-19T01:43:32.550 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-08-19T01:43:32.550 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-08-19T01:43:32.550 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-08-19T01:43:32.550 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-08-19T01:43:32.550 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.550 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.550 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.550 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.551 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster": { 2024-08-19T01:43:32.551 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster", 2024-08-19T01:43:32.551 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-08-19T01:43:32.551 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.551 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-08-19T01:43:32.551 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-08-19T01:43:32.551 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.551 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.551 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:32.551 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-08-19T01:43:32.551 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.551 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.551 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.552 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.552 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster_level": { 2024-08-19T01:43:32.552 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster_level", 2024-08-19T01:43:32.552 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-08-19T01:43:32.552 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.552 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-08-19T01:43:32.552 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "info", 2024-08-19T01:43:32.552 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.552 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.552 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-08-19T01:43:32.552 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-08-19T01:43:32.552 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-08-19T01:43:32.552 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-08-19T01:43:32.553 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-08-19T01:43:32.553 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-08-19T01:43:32.553 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-08-19T01:43:32.553 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-08-19T01:43:32.553 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-08-19T01:43:32.553 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.553 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.553 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.553 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.553 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_file": { 2024-08-19T01:43:32.553 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_file", 2024-08-19T01:43:32.553 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-08-19T01:43:32.553 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.553 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-08-19T01:43:32.554 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-08-19T01:43:32.554 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.554 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.554 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:32.554 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-08-19T01:43:32.554 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.554 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.554 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.554 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.554 INFO:teuthology.orchestra.run.smithi083.stdout: "mark_out_threshold": { 2024-08-19T01:43:32.554 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "mark_out_threshold", 2024-08-19T01:43:32.554 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "secs", 2024-08-19T01:43:32.554 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.554 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-08-19T01:43:32.555 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "2419200", 2024-08-19T01:43:32.555 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.555 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.555 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:32.555 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-08-19T01:43:32.555 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.555 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.555 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.555 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.555 INFO:teuthology.orchestra.run.smithi083.stdout: "pool_name": { 2024-08-19T01:43:32.555 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "pool_name", 2024-08-19T01:43:32.555 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-08-19T01:43:32.555 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.555 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-08-19T01:43:32.555 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "device_health_metrics", 2024-08-19T01:43:32.556 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.556 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.556 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:32.556 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "name of pool in which to store device health metrics", 2024-08-19T01:43:32.556 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.556 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.556 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.556 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.556 INFO:teuthology.orchestra.run.smithi083.stdout: "retention_period": { 2024-08-19T01:43:32.556 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "retention_period", 2024-08-19T01:43:32.556 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "secs", 2024-08-19T01:43:32.556 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.556 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-08-19T01:43:32.557 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "15552000", 2024-08-19T01:43:32.557 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.557 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.557 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:32.557 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "how long to retain device health metrics", 2024-08-19T01:43:32.557 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.557 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.557 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.557 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.557 INFO:teuthology.orchestra.run.smithi083.stdout: "scrape_frequency": { 2024-08-19T01:43:32.557 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "scrape_frequency", 2024-08-19T01:43:32.557 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "secs", 2024-08-19T01:43:32.557 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.557 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-08-19T01:43:32.557 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "86400", 2024-08-19T01:43:32.558 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.558 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.558 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:32.558 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "how frequently to scrape device health metrics", 2024-08-19T01:43:32.558 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.558 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.558 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.558 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.558 INFO:teuthology.orchestra.run.smithi083.stdout: "self_heal": { 2024-08-19T01:43:32.558 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "self_heal", 2024-08-19T01:43:32.558 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-08-19T01:43:32.558 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.558 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-08-19T01:43:32.559 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "True", 2024-08-19T01:43:32.559 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.559 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.559 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:32.559 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-08-19T01:43:32.559 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.559 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.559 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.559 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.559 INFO:teuthology.orchestra.run.smithi083.stdout: "sleep_interval": { 2024-08-19T01:43:32.559 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "sleep_interval", 2024-08-19T01:43:32.559 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "secs", 2024-08-19T01:43:32.559 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.559 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-08-19T01:43:32.560 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "600", 2024-08-19T01:43:32.560 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.560 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.560 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:32.560 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "how frequently to wake up and check device health", 2024-08-19T01:43:32.560 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.560 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.560 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.560 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.560 INFO:teuthology.orchestra.run.smithi083.stdout: "warn_threshold": { 2024-08-19T01:43:32.560 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "warn_threshold", 2024-08-19T01:43:32.560 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "secs", 2024-08-19T01:43:32.560 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.560 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-08-19T01:43:32.561 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "7257600", 2024-08-19T01:43:32.561 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.561 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.561 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:32.561 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-08-19T01:43:32.561 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.561 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.561 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.561 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-08-19T01:43:32.561 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-08-19T01:43:32.561 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.561 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-08-19T01:43:32.561 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "diskprediction_local", 2024-08-19T01:43:32.561 INFO:teuthology.orchestra.run.smithi083.stdout: "can_run": true, 2024-08-19T01:43:32.562 INFO:teuthology.orchestra.run.smithi083.stdout: "error_string": "", 2024-08-19T01:43:32.562 INFO:teuthology.orchestra.run.smithi083.stdout: "module_options": { 2024-08-19T01:43:32.562 INFO:teuthology.orchestra.run.smithi083.stdout: "log_level": { 2024-08-19T01:43:32.562 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_level", 2024-08-19T01:43:32.562 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-08-19T01:43:32.562 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.562 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-08-19T01:43:32.562 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-08-19T01:43:32.562 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.562 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.562 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-08-19T01:43:32.562 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-08-19T01:43:32.562 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-08-19T01:43:32.563 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-08-19T01:43:32.563 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-08-19T01:43:32.563 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-08-19T01:43:32.563 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-08-19T01:43:32.563 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-08-19T01:43:32.563 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-08-19T01:43:32.563 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.563 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.563 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.563 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.563 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster": { 2024-08-19T01:43:32.563 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster", 2024-08-19T01:43:32.563 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-08-19T01:43:32.564 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.564 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-08-19T01:43:32.564 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-08-19T01:43:32.564 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.564 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.564 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:32.564 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-08-19T01:43:32.564 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.564 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.564 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.564 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.564 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster_level": { 2024-08-19T01:43:32.564 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster_level", 2024-08-19T01:43:32.565 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-08-19T01:43:32.565 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.565 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-08-19T01:43:32.565 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "info", 2024-08-19T01:43:32.565 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.565 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.565 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-08-19T01:43:32.565 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-08-19T01:43:32.565 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-08-19T01:43:32.565 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-08-19T01:43:32.565 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-08-19T01:43:32.565 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-08-19T01:43:32.565 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-08-19T01:43:32.565 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-08-19T01:43:32.566 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-08-19T01:43:32.566 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.566 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.566 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.566 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.566 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_file": { 2024-08-19T01:43:32.566 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_file", 2024-08-19T01:43:32.566 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-08-19T01:43:32.566 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.566 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-08-19T01:43:32.566 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-08-19T01:43:32.566 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.566 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.567 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:32.567 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-08-19T01:43:32.567 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.567 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.567 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.567 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.567 INFO:teuthology.orchestra.run.smithi083.stdout: "predict_interval": { 2024-08-19T01:43:32.567 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "predict_interval", 2024-08-19T01:43:32.567 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-08-19T01:43:32.567 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.567 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-08-19T01:43:32.567 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "86400", 2024-08-19T01:43:32.567 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.567 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.568 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:32.568 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-08-19T01:43:32.568 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.568 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.568 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.568 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.568 INFO:teuthology.orchestra.run.smithi083.stdout: "predictor_model": { 2024-08-19T01:43:32.568 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "predictor_model", 2024-08-19T01:43:32.568 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-08-19T01:43:32.568 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.568 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-08-19T01:43:32.568 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "prophetstor", 2024-08-19T01:43:32.568 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.569 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.569 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:32.569 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-08-19T01:43:32.569 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.569 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.569 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.569 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.569 INFO:teuthology.orchestra.run.smithi083.stdout: "sleep_interval": { 2024-08-19T01:43:32.569 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "sleep_interval", 2024-08-19T01:43:32.569 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-08-19T01:43:32.569 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.569 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-08-19T01:43:32.569 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "600", 2024-08-19T01:43:32.570 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.570 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.570 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:32.570 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-08-19T01:43:32.570 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.570 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.570 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.570 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-08-19T01:43:32.570 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-08-19T01:43:32.570 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.570 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-08-19T01:43:32.570 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "influx", 2024-08-19T01:43:32.571 INFO:teuthology.orchestra.run.smithi083.stdout: "can_run": false, 2024-08-19T01:43:32.571 INFO:teuthology.orchestra.run.smithi083.stdout: "error_string": "influxdb python module not found", 2024-08-19T01:43:32.571 INFO:teuthology.orchestra.run.smithi083.stdout: "module_options": { 2024-08-19T01:43:32.571 INFO:teuthology.orchestra.run.smithi083.stdout: "batch_size": { 2024-08-19T01:43:32.571 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "batch_size", 2024-08-19T01:43:32.571 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2024-08-19T01:43:32.571 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.571 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-08-19T01:43:32.571 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "5000", 2024-08-19T01:43:32.571 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.571 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.571 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:32.571 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-08-19T01:43:32.572 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.572 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.572 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.572 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.572 INFO:teuthology.orchestra.run.smithi083.stdout: "database": { 2024-08-19T01:43:32.572 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "database", 2024-08-19T01:43:32.572 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-08-19T01:43:32.572 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.572 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-08-19T01:43:32.572 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "ceph", 2024-08-19T01:43:32.572 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.572 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.572 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:32.573 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-08-19T01:43:32.573 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.573 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.573 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.573 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.573 INFO:teuthology.orchestra.run.smithi083.stdout: "hostname": { 2024-08-19T01:43:32.573 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "hostname", 2024-08-19T01:43:32.573 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-08-19T01:43:32.573 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.573 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-08-19T01:43:32.573 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-08-19T01:43:32.573 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.573 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.573 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:32.574 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "InfluxDB server hostname", 2024-08-19T01:43:32.574 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.574 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.574 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.574 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.574 INFO:teuthology.orchestra.run.smithi083.stdout: "interval": { 2024-08-19T01:43:32.574 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "interval", 2024-08-19T01:43:32.574 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "secs", 2024-08-19T01:43:32.574 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.574 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-08-19T01:43:32.574 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "30", 2024-08-19T01:43:32.574 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "5", 2024-08-19T01:43:32.574 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.575 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:32.575 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-08-19T01:43:32.575 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.575 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.575 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.575 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.575 INFO:teuthology.orchestra.run.smithi083.stdout: "log_level": { 2024-08-19T01:43:32.575 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_level", 2024-08-19T01:43:32.575 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-08-19T01:43:32.575 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.575 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-08-19T01:43:32.575 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-08-19T01:43:32.575 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.575 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.575 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-08-19T01:43:32.576 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-08-19T01:43:32.576 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-08-19T01:43:32.576 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-08-19T01:43:32.576 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-08-19T01:43:32.576 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-08-19T01:43:32.576 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-08-19T01:43:32.576 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-08-19T01:43:32.576 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-08-19T01:43:32.576 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.576 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.576 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.576 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.576 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster": { 2024-08-19T01:43:32.576 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster", 2024-08-19T01:43:32.576 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-08-19T01:43:32.576 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.577 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-08-19T01:43:32.577 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-08-19T01:43:32.577 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.577 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.577 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:32.577 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-08-19T01:43:32.577 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.577 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.577 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.577 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.577 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster_level": { 2024-08-19T01:43:32.577 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster_level", 2024-08-19T01:43:32.577 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-08-19T01:43:32.578 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.578 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-08-19T01:43:32.578 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "info", 2024-08-19T01:43:32.578 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.578 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.578 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-08-19T01:43:32.578 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-08-19T01:43:32.578 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-08-19T01:43:32.578 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-08-19T01:43:32.578 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-08-19T01:43:32.578 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-08-19T01:43:32.578 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-08-19T01:43:32.579 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-08-19T01:43:32.579 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-08-19T01:43:32.579 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.579 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.579 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.579 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.579 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_file": { 2024-08-19T01:43:32.579 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_file", 2024-08-19T01:43:32.579 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-08-19T01:43:32.579 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.579 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-08-19T01:43:32.579 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-08-19T01:43:32.579 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.580 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.580 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:32.580 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-08-19T01:43:32.580 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.580 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.580 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.580 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.580 INFO:teuthology.orchestra.run.smithi083.stdout: "password": { 2024-08-19T01:43:32.580 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "password", 2024-08-19T01:43:32.580 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-08-19T01:43:32.580 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.580 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-08-19T01:43:32.580 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-08-19T01:43:32.581 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.581 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.581 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:32.581 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "password of InfluxDB server user", 2024-08-19T01:43:32.581 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.581 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.581 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.581 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.581 INFO:teuthology.orchestra.run.smithi083.stdout: "port": { 2024-08-19T01:43:32.581 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "port", 2024-08-19T01:43:32.581 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2024-08-19T01:43:32.581 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.581 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-08-19T01:43:32.582 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "8086", 2024-08-19T01:43:32.582 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.582 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.582 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:32.582 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "InfluxDB server port", 2024-08-19T01:43:32.582 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.582 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.582 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.582 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.582 INFO:teuthology.orchestra.run.smithi083.stdout: "ssl": { 2024-08-19T01:43:32.582 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "ssl", 2024-08-19T01:43:32.582 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-08-19T01:43:32.583 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.583 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-08-19T01:43:32.583 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "false", 2024-08-19T01:43:32.583 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.583 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.583 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:32.583 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-08-19T01:43:32.583 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.583 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.583 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.583 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.583 INFO:teuthology.orchestra.run.smithi083.stdout: "threads": { 2024-08-19T01:43:32.583 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "threads", 2024-08-19T01:43:32.584 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2024-08-19T01:43:32.584 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.584 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-08-19T01:43:32.584 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "5", 2024-08-19T01:43:32.584 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "1", 2024-08-19T01:43:32.584 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "32", 2024-08-19T01:43:32.584 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:32.584 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-08-19T01:43:32.584 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.584 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.584 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.584 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.584 INFO:teuthology.orchestra.run.smithi083.stdout: "username": { 2024-08-19T01:43:32.584 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "username", 2024-08-19T01:43:32.585 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-08-19T01:43:32.585 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.585 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-08-19T01:43:32.585 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-08-19T01:43:32.585 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.585 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.585 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:32.585 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "username of InfluxDB server user", 2024-08-19T01:43:32.585 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.585 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.585 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.585 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.585 INFO:teuthology.orchestra.run.smithi083.stdout: "verify_ssl": { 2024-08-19T01:43:32.585 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "verify_ssl", 2024-08-19T01:43:32.586 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-08-19T01:43:32.586 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.586 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-08-19T01:43:32.586 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "true", 2024-08-19T01:43:32.586 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.586 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.586 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:32.586 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-08-19T01:43:32.586 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.586 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.586 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.587 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-08-19T01:43:32.587 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-08-19T01:43:32.587 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.587 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-08-19T01:43:32.587 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "insights", 2024-08-19T01:43:32.587 INFO:teuthology.orchestra.run.smithi083.stdout: "can_run": true, 2024-08-19T01:43:32.587 INFO:teuthology.orchestra.run.smithi083.stdout: "error_string": "", 2024-08-19T01:43:32.587 INFO:teuthology.orchestra.run.smithi083.stdout: "module_options": { 2024-08-19T01:43:32.587 INFO:teuthology.orchestra.run.smithi083.stdout: "log_level": { 2024-08-19T01:43:32.587 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_level", 2024-08-19T01:43:32.587 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-08-19T01:43:32.587 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.587 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-08-19T01:43:32.588 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-08-19T01:43:32.588 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.588 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.588 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-08-19T01:43:32.588 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-08-19T01:43:32.588 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-08-19T01:43:32.588 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-08-19T01:43:32.588 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-08-19T01:43:32.588 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-08-19T01:43:32.588 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-08-19T01:43:32.588 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-08-19T01:43:32.588 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-08-19T01:43:32.588 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.589 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.589 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.589 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.589 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster": { 2024-08-19T01:43:32.589 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster", 2024-08-19T01:43:32.589 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-08-19T01:43:32.589 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.589 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-08-19T01:43:32.589 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-08-19T01:43:32.589 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.589 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.589 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:32.589 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-08-19T01:43:32.590 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.590 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.590 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.590 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.590 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster_level": { 2024-08-19T01:43:32.590 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster_level", 2024-08-19T01:43:32.590 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-08-19T01:43:32.590 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.590 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-08-19T01:43:32.590 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "info", 2024-08-19T01:43:32.590 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.590 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.590 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-08-19T01:43:32.591 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-08-19T01:43:32.591 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-08-19T01:43:32.591 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-08-19T01:43:32.591 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-08-19T01:43:32.591 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-08-19T01:43:32.591 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-08-19T01:43:32.591 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-08-19T01:43:32.591 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-08-19T01:43:32.591 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.591 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.591 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.591 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.591 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_file": { 2024-08-19T01:43:32.591 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_file", 2024-08-19T01:43:32.592 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-08-19T01:43:32.592 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.592 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-08-19T01:43:32.592 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-08-19T01:43:32.592 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.592 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.592 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:32.592 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-08-19T01:43:32.592 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.592 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.592 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.592 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-08-19T01:43:32.593 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-08-19T01:43:32.593 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.593 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-08-19T01:43:32.593 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "iostat", 2024-08-19T01:43:32.593 INFO:teuthology.orchestra.run.smithi083.stdout: "can_run": true, 2024-08-19T01:43:32.593 INFO:teuthology.orchestra.run.smithi083.stdout: "error_string": "", 2024-08-19T01:43:32.593 INFO:teuthology.orchestra.run.smithi083.stdout: "module_options": { 2024-08-19T01:43:32.593 INFO:teuthology.orchestra.run.smithi083.stdout: "log_level": { 2024-08-19T01:43:32.593 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_level", 2024-08-19T01:43:32.593 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-08-19T01:43:32.593 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.593 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-08-19T01:43:32.593 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-08-19T01:43:32.594 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.594 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.594 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-08-19T01:43:32.594 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-08-19T01:43:32.594 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-08-19T01:43:32.594 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-08-19T01:43:32.594 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-08-19T01:43:32.594 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-08-19T01:43:32.594 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-08-19T01:43:32.594 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-08-19T01:43:32.594 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-08-19T01:43:32.594 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.594 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.595 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.595 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.595 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster": { 2024-08-19T01:43:32.595 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster", 2024-08-19T01:43:32.595 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-08-19T01:43:32.595 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.595 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-08-19T01:43:32.595 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-08-19T01:43:32.595 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.595 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.595 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:32.595 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-08-19T01:43:32.596 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.596 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.596 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.596 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.596 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster_level": { 2024-08-19T01:43:32.596 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster_level", 2024-08-19T01:43:32.596 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-08-19T01:43:32.596 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.596 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-08-19T01:43:32.596 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "info", 2024-08-19T01:43:32.596 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.596 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.596 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-08-19T01:43:32.597 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-08-19T01:43:32.597 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-08-19T01:43:32.597 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-08-19T01:43:32.597 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-08-19T01:43:32.597 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-08-19T01:43:32.597 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-08-19T01:43:32.597 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-08-19T01:43:32.597 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-08-19T01:43:32.597 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.597 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.597 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.597 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.597 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_file": { 2024-08-19T01:43:32.598 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_file", 2024-08-19T01:43:32.598 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-08-19T01:43:32.598 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.598 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-08-19T01:43:32.598 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-08-19T01:43:32.598 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.598 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.598 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:32.598 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-08-19T01:43:32.598 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.598 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.598 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.599 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-08-19T01:43:32.599 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-08-19T01:43:32.599 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.599 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-08-19T01:43:32.599 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "k8sevents", 2024-08-19T01:43:32.599 INFO:teuthology.orchestra.run.smithi083.stdout: "can_run": true, 2024-08-19T01:43:32.599 INFO:teuthology.orchestra.run.smithi083.stdout: "error_string": "", 2024-08-19T01:43:32.599 INFO:teuthology.orchestra.run.smithi083.stdout: "module_options": { 2024-08-19T01:43:32.599 INFO:teuthology.orchestra.run.smithi083.stdout: "ceph_event_retention_days": { 2024-08-19T01:43:32.599 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "ceph_event_retention_days", 2024-08-19T01:43:32.599 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2024-08-19T01:43:32.599 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.600 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-08-19T01:43:32.600 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "7", 2024-08-19T01:43:32.600 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.600 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.600 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:32.600 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Days to hold ceph event information within local cache", 2024-08-19T01:43:32.600 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.600 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.600 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.600 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.600 INFO:teuthology.orchestra.run.smithi083.stdout: "config_check_secs": { 2024-08-19T01:43:32.600 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "config_check_secs", 2024-08-19T01:43:32.600 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2024-08-19T01:43:32.600 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.601 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-08-19T01:43:32.601 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "10", 2024-08-19T01:43:32.601 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "10", 2024-08-19T01:43:32.601 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.601 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:32.601 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-08-19T01:43:32.601 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.601 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.601 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.601 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.601 INFO:teuthology.orchestra.run.smithi083.stdout: "log_level": { 2024-08-19T01:43:32.601 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_level", 2024-08-19T01:43:32.601 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-08-19T01:43:32.602 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.602 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-08-19T01:43:32.602 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-08-19T01:43:32.602 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.602 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.602 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-08-19T01:43:32.602 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-08-19T01:43:32.602 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-08-19T01:43:32.602 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-08-19T01:43:32.602 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-08-19T01:43:32.602 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-08-19T01:43:32.602 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-08-19T01:43:32.602 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-08-19T01:43:32.602 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-08-19T01:43:32.603 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.603 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.603 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.603 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.603 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster": { 2024-08-19T01:43:32.603 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster", 2024-08-19T01:43:32.603 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-08-19T01:43:32.603 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.603 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-08-19T01:43:32.603 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-08-19T01:43:32.603 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.603 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.603 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:32.603 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-08-19T01:43:32.604 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.604 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.604 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.604 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.604 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster_level": { 2024-08-19T01:43:32.604 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster_level", 2024-08-19T01:43:32.604 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-08-19T01:43:32.604 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.604 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-08-19T01:43:32.604 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "info", 2024-08-19T01:43:32.604 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.604 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.604 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-08-19T01:43:32.604 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-08-19T01:43:32.604 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-08-19T01:43:32.605 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-08-19T01:43:32.605 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-08-19T01:43:32.605 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-08-19T01:43:32.605 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-08-19T01:43:32.605 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-08-19T01:43:32.605 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-08-19T01:43:32.605 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.605 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.605 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.605 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.605 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_file": { 2024-08-19T01:43:32.605 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_file", 2024-08-19T01:43:32.605 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-08-19T01:43:32.605 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.605 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-08-19T01:43:32.605 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-08-19T01:43:32.606 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.606 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.606 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:32.606 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-08-19T01:43:32.606 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.606 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.606 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.606 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-08-19T01:43:32.606 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-08-19T01:43:32.606 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.606 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-08-19T01:43:32.606 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "localpool", 2024-08-19T01:43:32.606 INFO:teuthology.orchestra.run.smithi083.stdout: "can_run": true, 2024-08-19T01:43:32.607 INFO:teuthology.orchestra.run.smithi083.stdout: "error_string": "", 2024-08-19T01:43:32.607 INFO:teuthology.orchestra.run.smithi083.stdout: "module_options": { 2024-08-19T01:43:32.607 INFO:teuthology.orchestra.run.smithi083.stdout: "failure_domain": { 2024-08-19T01:43:32.607 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "failure_domain", 2024-08-19T01:43:32.607 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-08-19T01:43:32.607 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.607 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-08-19T01:43:32.607 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "host", 2024-08-19T01:43:32.607 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.607 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.607 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:32.607 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "failure domain for any created local pool", 2024-08-19T01:43:32.607 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-08-19T01:43:32.607 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.607 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.608 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.608 INFO:teuthology.orchestra.run.smithi083.stdout: "log_level": { 2024-08-19T01:43:32.608 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_level", 2024-08-19T01:43:32.608 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-08-19T01:43:32.608 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.608 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-08-19T01:43:32.608 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-08-19T01:43:32.608 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.608 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.608 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-08-19T01:43:32.608 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-08-19T01:43:32.608 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-08-19T01:43:32.608 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-08-19T01:43:32.608 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-08-19T01:43:32.609 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-08-19T01:43:32.609 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-08-19T01:43:32.609 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-08-19T01:43:32.609 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-08-19T01:43:32.609 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.609 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.609 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.609 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.609 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster": { 2024-08-19T01:43:32.609 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster", 2024-08-19T01:43:32.609 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-08-19T01:43:32.609 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.610 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-08-19T01:43:32.610 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-08-19T01:43:32.610 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.610 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.610 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:32.610 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-08-19T01:43:32.610 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.610 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.610 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.610 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.610 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster_level": { 2024-08-19T01:43:32.610 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster_level", 2024-08-19T01:43:32.610 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-08-19T01:43:32.610 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.611 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-08-19T01:43:32.611 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "info", 2024-08-19T01:43:32.611 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.611 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.611 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-08-19T01:43:32.611 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-08-19T01:43:32.611 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-08-19T01:43:32.611 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-08-19T01:43:32.611 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-08-19T01:43:32.611 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-08-19T01:43:32.611 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-08-19T01:43:32.612 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-08-19T01:43:32.612 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-08-19T01:43:32.612 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.612 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.612 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.612 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.612 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_file": { 2024-08-19T01:43:32.612 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_file", 2024-08-19T01:43:32.612 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-08-19T01:43:32.612 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.612 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-08-19T01:43:32.612 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-08-19T01:43:32.612 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.612 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.613 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:32.613 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-08-19T01:43:32.613 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.613 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.613 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.613 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.613 INFO:teuthology.orchestra.run.smithi083.stdout: "min_size": { 2024-08-19T01:43:32.613 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "min_size", 2024-08-19T01:43:32.613 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2024-08-19T01:43:32.613 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.613 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-08-19T01:43:32.613 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-08-19T01:43:32.613 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.613 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.613 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:32.614 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "default min_size for any created local pool", 2024-08-19T01:43:32.614 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-08-19T01:43:32.614 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.614 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.614 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.614 INFO:teuthology.orchestra.run.smithi083.stdout: "num_rep": { 2024-08-19T01:43:32.614 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "num_rep", 2024-08-19T01:43:32.614 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2024-08-19T01:43:32.614 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.614 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-08-19T01:43:32.614 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "3", 2024-08-19T01:43:32.614 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.614 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.614 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:32.614 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "default replica count for any created local pool", 2024-08-19T01:43:32.615 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.615 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.615 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.615 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.615 INFO:teuthology.orchestra.run.smithi083.stdout: "pg_num": { 2024-08-19T01:43:32.615 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "pg_num", 2024-08-19T01:43:32.615 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2024-08-19T01:43:32.615 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.615 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-08-19T01:43:32.615 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "128", 2024-08-19T01:43:32.615 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.616 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.616 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:32.616 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "default pg_num for any created local pool", 2024-08-19T01:43:32.616 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.616 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.616 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.616 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.616 INFO:teuthology.orchestra.run.smithi083.stdout: "prefix": { 2024-08-19T01:43:32.616 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "prefix", 2024-08-19T01:43:32.616 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-08-19T01:43:32.616 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.616 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-08-19T01:43:32.616 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-08-19T01:43:32.616 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.616 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.617 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:32.617 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "name prefix for any created local pool", 2024-08-19T01:43:32.617 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.617 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.617 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.617 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.617 INFO:teuthology.orchestra.run.smithi083.stdout: "subtree": { 2024-08-19T01:43:32.617 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "subtree", 2024-08-19T01:43:32.617 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-08-19T01:43:32.617 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.617 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-08-19T01:43:32.617 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "rack", 2024-08-19T01:43:32.617 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.617 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.617 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:32.618 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "CRUSH level for which to create a local pool", 2024-08-19T01:43:32.618 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-08-19T01:43:32.618 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.618 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.618 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-08-19T01:43:32.618 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-08-19T01:43:32.618 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.618 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-08-19T01:43:32.618 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "mds_autoscaler", 2024-08-19T01:43:32.618 INFO:teuthology.orchestra.run.smithi083.stdout: "can_run": true, 2024-08-19T01:43:32.618 INFO:teuthology.orchestra.run.smithi083.stdout: "error_string": "", 2024-08-19T01:43:32.618 INFO:teuthology.orchestra.run.smithi083.stdout: "module_options": { 2024-08-19T01:43:32.618 INFO:teuthology.orchestra.run.smithi083.stdout: "log_level": { 2024-08-19T01:43:32.618 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_level", 2024-08-19T01:43:32.619 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-08-19T01:43:32.619 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.619 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-08-19T01:43:32.619 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-08-19T01:43:32.619 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.619 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.619 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-08-19T01:43:32.619 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-08-19T01:43:32.619 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-08-19T01:43:32.619 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-08-19T01:43:32.619 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-08-19T01:43:32.619 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-08-19T01:43:32.619 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-08-19T01:43:32.619 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-08-19T01:43:32.620 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-08-19T01:43:32.620 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.620 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.620 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.620 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.620 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster": { 2024-08-19T01:43:32.620 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster", 2024-08-19T01:43:32.620 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-08-19T01:43:32.620 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.620 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-08-19T01:43:32.620 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-08-19T01:43:32.620 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.620 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.620 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:32.621 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-08-19T01:43:32.621 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.621 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.621 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.621 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.621 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster_level": { 2024-08-19T01:43:32.621 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster_level", 2024-08-19T01:43:32.621 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-08-19T01:43:32.621 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.621 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-08-19T01:43:32.621 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "info", 2024-08-19T01:43:32.621 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.621 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.621 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-08-19T01:43:32.622 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-08-19T01:43:32.622 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-08-19T01:43:32.622 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-08-19T01:43:32.622 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-08-19T01:43:32.622 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-08-19T01:43:32.622 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-08-19T01:43:32.622 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-08-19T01:43:32.622 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-08-19T01:43:32.622 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.622 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.622 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.622 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.622 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_file": { 2024-08-19T01:43:32.623 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_file", 2024-08-19T01:43:32.623 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-08-19T01:43:32.623 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.623 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-08-19T01:43:32.623 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-08-19T01:43:32.623 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.623 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.623 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:32.623 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-08-19T01:43:32.623 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.623 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.623 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.623 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-08-19T01:43:32.624 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-08-19T01:43:32.624 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.624 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-08-19T01:43:32.624 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "mirroring", 2024-08-19T01:43:32.624 INFO:teuthology.orchestra.run.smithi083.stdout: "can_run": true, 2024-08-19T01:43:32.624 INFO:teuthology.orchestra.run.smithi083.stdout: "error_string": "", 2024-08-19T01:43:32.624 INFO:teuthology.orchestra.run.smithi083.stdout: "module_options": { 2024-08-19T01:43:32.624 INFO:teuthology.orchestra.run.smithi083.stdout: "log_level": { 2024-08-19T01:43:32.624 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_level", 2024-08-19T01:43:32.624 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-08-19T01:43:32.624 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.624 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-08-19T01:43:32.624 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-08-19T01:43:32.624 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.625 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.625 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-08-19T01:43:32.625 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-08-19T01:43:32.625 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-08-19T01:43:32.625 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-08-19T01:43:32.625 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-08-19T01:43:32.625 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-08-19T01:43:32.625 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-08-19T01:43:32.625 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-08-19T01:43:32.625 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-08-19T01:43:32.625 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.625 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.625 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.625 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.625 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster": { 2024-08-19T01:43:32.626 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster", 2024-08-19T01:43:32.626 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-08-19T01:43:32.626 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.626 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-08-19T01:43:32.626 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-08-19T01:43:32.626 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.626 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.626 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:32.626 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-08-19T01:43:32.626 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.626 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.626 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.626 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.627 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster_level": { 2024-08-19T01:43:32.627 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster_level", 2024-08-19T01:43:32.627 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-08-19T01:43:32.627 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.627 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-08-19T01:43:32.627 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "info", 2024-08-19T01:43:32.627 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.627 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.627 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-08-19T01:43:32.627 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-08-19T01:43:32.627 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-08-19T01:43:32.627 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-08-19T01:43:32.627 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-08-19T01:43:32.627 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-08-19T01:43:32.627 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-08-19T01:43:32.628 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-08-19T01:43:32.628 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-08-19T01:43:32.628 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.628 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.628 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.628 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.628 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_file": { 2024-08-19T01:43:32.628 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_file", 2024-08-19T01:43:32.628 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-08-19T01:43:32.628 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.628 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-08-19T01:43:32.628 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-08-19T01:43:32.628 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.628 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.628 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:32.628 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-08-19T01:43:32.629 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.629 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.629 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.629 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-08-19T01:43:32.629 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-08-19T01:43:32.629 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.629 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-08-19T01:43:32.629 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "nfs", 2024-08-19T01:43:32.629 INFO:teuthology.orchestra.run.smithi083.stdout: "can_run": true, 2024-08-19T01:43:32.629 INFO:teuthology.orchestra.run.smithi083.stdout: "error_string": "", 2024-08-19T01:43:32.629 INFO:teuthology.orchestra.run.smithi083.stdout: "module_options": { 2024-08-19T01:43:32.629 INFO:teuthology.orchestra.run.smithi083.stdout: "log_level": { 2024-08-19T01:43:32.629 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_level", 2024-08-19T01:43:32.629 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-08-19T01:43:32.630 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.630 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-08-19T01:43:32.630 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-08-19T01:43:32.630 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.630 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.630 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-08-19T01:43:32.630 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-08-19T01:43:32.630 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-08-19T01:43:32.630 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-08-19T01:43:32.630 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-08-19T01:43:32.630 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-08-19T01:43:32.630 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-08-19T01:43:32.630 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-08-19T01:43:32.630 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-08-19T01:43:32.630 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.631 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.631 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.631 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.631 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster": { 2024-08-19T01:43:32.631 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster", 2024-08-19T01:43:32.631 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-08-19T01:43:32.631 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.631 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-08-19T01:43:32.631 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-08-19T01:43:32.631 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.631 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.631 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:32.631 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-08-19T01:43:32.632 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.632 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.632 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.632 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.632 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster_level": { 2024-08-19T01:43:32.632 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster_level", 2024-08-19T01:43:32.632 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-08-19T01:43:32.632 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.632 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-08-19T01:43:32.632 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "info", 2024-08-19T01:43:32.632 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.632 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.632 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-08-19T01:43:32.633 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-08-19T01:43:32.633 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-08-19T01:43:32.633 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-08-19T01:43:32.633 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-08-19T01:43:32.633 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-08-19T01:43:32.633 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-08-19T01:43:32.633 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-08-19T01:43:32.633 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-08-19T01:43:32.633 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.633 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.633 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.633 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.633 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_file": { 2024-08-19T01:43:32.634 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_file", 2024-08-19T01:43:32.634 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-08-19T01:43:32.634 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.634 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-08-19T01:43:32.634 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-08-19T01:43:32.634 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.634 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.634 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:32.634 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-08-19T01:43:32.634 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.634 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.634 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.635 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-08-19T01:43:32.635 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-08-19T01:43:32.635 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.635 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-08-19T01:43:32.635 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "orchestrator", 2024-08-19T01:43:32.635 INFO:teuthology.orchestra.run.smithi083.stdout: "can_run": true, 2024-08-19T01:43:32.635 INFO:teuthology.orchestra.run.smithi083.stdout: "error_string": "", 2024-08-19T01:43:32.635 INFO:teuthology.orchestra.run.smithi083.stdout: "module_options": { 2024-08-19T01:43:32.635 INFO:teuthology.orchestra.run.smithi083.stdout: "fail_fs": { 2024-08-19T01:43:32.635 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "fail_fs", 2024-08-19T01:43:32.635 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-08-19T01:43:32.635 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.635 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-08-19T01:43:32.636 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-08-19T01:43:32.636 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.636 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.636 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:32.636 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-08-19T01:43:32.636 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.636 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.636 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.636 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.636 INFO:teuthology.orchestra.run.smithi083.stdout: "log_level": { 2024-08-19T01:43:32.636 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_level", 2024-08-19T01:43:32.636 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-08-19T01:43:32.636 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.637 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-08-19T01:43:32.637 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-08-19T01:43:32.637 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.637 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.637 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-08-19T01:43:32.637 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-08-19T01:43:32.637 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-08-19T01:43:32.637 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-08-19T01:43:32.637 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-08-19T01:43:32.637 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-08-19T01:43:32.637 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-08-19T01:43:32.637 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-08-19T01:43:32.637 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-08-19T01:43:32.637 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.637 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.638 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.638 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.638 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster": { 2024-08-19T01:43:32.638 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster", 2024-08-19T01:43:32.638 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-08-19T01:43:32.638 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.638 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-08-19T01:43:32.638 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-08-19T01:43:32.638 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.638 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.638 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:32.638 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-08-19T01:43:32.638 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.638 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.638 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.639 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.639 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster_level": { 2024-08-19T01:43:32.639 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster_level", 2024-08-19T01:43:32.639 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-08-19T01:43:32.639 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.639 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-08-19T01:43:32.639 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "info", 2024-08-19T01:43:32.639 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.639 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.639 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-08-19T01:43:32.639 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-08-19T01:43:32.639 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-08-19T01:43:32.639 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-08-19T01:43:32.640 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-08-19T01:43:32.640 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-08-19T01:43:32.640 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-08-19T01:43:32.640 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-08-19T01:43:32.640 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-08-19T01:43:32.640 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.640 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.640 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.640 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.640 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_file": { 2024-08-19T01:43:32.640 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_file", 2024-08-19T01:43:32.640 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-08-19T01:43:32.640 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.640 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-08-19T01:43:32.641 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-08-19T01:43:32.641 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.641 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.641 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:32.641 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-08-19T01:43:32.641 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.641 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.641 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.641 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.641 INFO:teuthology.orchestra.run.smithi083.stdout: "orchestrator": { 2024-08-19T01:43:32.641 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "orchestrator", 2024-08-19T01:43:32.641 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-08-19T01:43:32.641 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.641 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-08-19T01:43:32.642 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-08-19T01:43:32.642 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.642 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.642 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-08-19T01:43:32.642 INFO:teuthology.orchestra.run.smithi083.stdout: "cephadm", 2024-08-19T01:43:32.642 INFO:teuthology.orchestra.run.smithi083.stdout: "rook", 2024-08-19T01:43:32.642 INFO:teuthology.orchestra.run.smithi083.stdout: "test_orchestrator" 2024-08-19T01:43:32.642 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-08-19T01:43:32.642 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Orchestrator backend", 2024-08-19T01:43:32.642 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.642 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.642 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.642 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-08-19T01:43:32.642 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-08-19T01:43:32.643 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.643 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-08-19T01:43:32.643 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "osd_perf_query", 2024-08-19T01:43:32.643 INFO:teuthology.orchestra.run.smithi083.stdout: "can_run": true, 2024-08-19T01:43:32.643 INFO:teuthology.orchestra.run.smithi083.stdout: "error_string": "", 2024-08-19T01:43:32.643 INFO:teuthology.orchestra.run.smithi083.stdout: "module_options": { 2024-08-19T01:43:32.643 INFO:teuthology.orchestra.run.smithi083.stdout: "log_level": { 2024-08-19T01:43:32.643 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_level", 2024-08-19T01:43:32.643 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-08-19T01:43:32.643 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.643 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-08-19T01:43:32.643 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-08-19T01:43:32.643 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.643 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.644 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-08-19T01:43:32.644 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-08-19T01:43:32.644 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-08-19T01:43:32.644 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-08-19T01:43:32.644 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-08-19T01:43:32.644 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-08-19T01:43:32.644 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-08-19T01:43:32.644 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-08-19T01:43:32.644 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-08-19T01:43:32.644 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.644 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.644 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.644 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.645 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster": { 2024-08-19T01:43:32.645 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster", 2024-08-19T01:43:32.645 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-08-19T01:43:32.645 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.645 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-08-19T01:43:32.645 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-08-19T01:43:32.645 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.645 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.645 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:32.645 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-08-19T01:43:32.645 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.645 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.645 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.645 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.646 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster_level": { 2024-08-19T01:43:32.646 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster_level", 2024-08-19T01:43:32.646 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-08-19T01:43:32.646 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.646 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-08-19T01:43:32.646 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "info", 2024-08-19T01:43:32.646 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.646 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.646 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-08-19T01:43:32.646 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-08-19T01:43:32.646 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-08-19T01:43:32.647 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-08-19T01:43:32.647 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-08-19T01:43:32.647 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-08-19T01:43:32.647 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-08-19T01:43:32.647 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-08-19T01:43:32.647 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-08-19T01:43:32.647 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.647 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.647 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.647 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.647 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_file": { 2024-08-19T01:43:32.647 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_file", 2024-08-19T01:43:32.647 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-08-19T01:43:32.648 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.648 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-08-19T01:43:32.648 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-08-19T01:43:32.648 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.648 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.648 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:32.648 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-08-19T01:43:32.648 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.648 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.648 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.648 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-08-19T01:43:32.648 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-08-19T01:43:32.648 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.649 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-08-19T01:43:32.649 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "osd_support", 2024-08-19T01:43:32.649 INFO:teuthology.orchestra.run.smithi083.stdout: "can_run": true, 2024-08-19T01:43:32.649 INFO:teuthology.orchestra.run.smithi083.stdout: "error_string": "", 2024-08-19T01:43:32.649 INFO:teuthology.orchestra.run.smithi083.stdout: "module_options": { 2024-08-19T01:43:32.649 INFO:teuthology.orchestra.run.smithi083.stdout: "log_level": { 2024-08-19T01:43:32.649 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_level", 2024-08-19T01:43:32.649 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-08-19T01:43:32.649 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.649 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-08-19T01:43:32.649 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-08-19T01:43:32.649 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.649 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.649 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-08-19T01:43:32.650 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-08-19T01:43:32.650 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-08-19T01:43:32.650 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-08-19T01:43:32.650 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-08-19T01:43:32.650 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-08-19T01:43:32.650 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-08-19T01:43:32.650 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-08-19T01:43:32.650 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-08-19T01:43:32.650 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.650 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.650 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.650 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.651 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster": { 2024-08-19T01:43:32.651 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster", 2024-08-19T01:43:32.651 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-08-19T01:43:32.651 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.651 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-08-19T01:43:32.651 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-08-19T01:43:32.651 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.651 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.651 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:32.651 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-08-19T01:43:32.651 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.651 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.651 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.652 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.652 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster_level": { 2024-08-19T01:43:32.652 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster_level", 2024-08-19T01:43:32.652 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-08-19T01:43:32.652 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.652 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-08-19T01:43:32.652 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "info", 2024-08-19T01:43:32.652 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.652 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.652 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-08-19T01:43:32.652 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-08-19T01:43:32.653 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-08-19T01:43:32.653 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-08-19T01:43:32.653 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-08-19T01:43:32.653 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-08-19T01:43:32.653 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-08-19T01:43:32.653 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-08-19T01:43:32.653 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-08-19T01:43:32.653 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.653 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.653 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.653 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.653 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_file": { 2024-08-19T01:43:32.653 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_file", 2024-08-19T01:43:32.653 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-08-19T01:43:32.653 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.654 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-08-19T01:43:32.654 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-08-19T01:43:32.654 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.654 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.654 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:32.654 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-08-19T01:43:32.654 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.654 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.654 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.654 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-08-19T01:43:32.654 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-08-19T01:43:32.654 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.654 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-08-19T01:43:32.655 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "pg_autoscaler", 2024-08-19T01:43:32.655 INFO:teuthology.orchestra.run.smithi083.stdout: "can_run": true, 2024-08-19T01:43:32.655 INFO:teuthology.orchestra.run.smithi083.stdout: "error_string": "", 2024-08-19T01:43:32.655 INFO:teuthology.orchestra.run.smithi083.stdout: "module_options": { 2024-08-19T01:43:32.655 INFO:teuthology.orchestra.run.smithi083.stdout: "log_level": { 2024-08-19T01:43:32.655 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_level", 2024-08-19T01:43:32.655 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-08-19T01:43:32.655 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.655 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-08-19T01:43:32.655 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-08-19T01:43:32.655 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.655 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.655 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-08-19T01:43:32.655 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-08-19T01:43:32.655 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-08-19T01:43:32.656 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-08-19T01:43:32.656 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-08-19T01:43:32.656 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-08-19T01:43:32.656 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-08-19T01:43:32.656 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-08-19T01:43:32.656 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-08-19T01:43:32.656 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.656 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.656 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.656 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.656 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster": { 2024-08-19T01:43:32.656 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster", 2024-08-19T01:43:32.656 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-08-19T01:43:32.656 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.656 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-08-19T01:43:32.657 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-08-19T01:43:32.657 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.657 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.657 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:32.657 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-08-19T01:43:32.657 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.657 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.657 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.657 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.657 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster_level": { 2024-08-19T01:43:32.657 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster_level", 2024-08-19T01:43:32.658 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-08-19T01:43:32.658 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.658 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-08-19T01:43:32.658 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "info", 2024-08-19T01:43:32.658 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.658 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.658 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-08-19T01:43:32.658 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-08-19T01:43:32.658 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-08-19T01:43:32.658 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-08-19T01:43:32.658 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-08-19T01:43:32.658 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-08-19T01:43:32.659 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-08-19T01:43:32.659 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-08-19T01:43:32.659 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-08-19T01:43:32.659 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.659 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.659 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.659 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.659 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_file": { 2024-08-19T01:43:32.659 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_file", 2024-08-19T01:43:32.659 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-08-19T01:43:32.660 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.660 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-08-19T01:43:32.660 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-08-19T01:43:32.660 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.660 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.660 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:32.660 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-08-19T01:43:32.660 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.660 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.660 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.660 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.660 INFO:teuthology.orchestra.run.smithi083.stdout: "sleep_interval": { 2024-08-19T01:43:32.660 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "sleep_interval", 2024-08-19T01:43:32.660 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "secs", 2024-08-19T01:43:32.661 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.661 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-08-19T01:43:32.661 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "60", 2024-08-19T01:43:32.661 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.661 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.661 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:32.661 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-08-19T01:43:32.661 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.661 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.661 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.661 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.661 INFO:teuthology.orchestra.run.smithi083.stdout: "threshold": { 2024-08-19T01:43:32.661 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "threshold", 2024-08-19T01:43:32.662 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "float", 2024-08-19T01:43:32.662 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.662 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-08-19T01:43:32.662 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "3.0", 2024-08-19T01:43:32.662 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "1.0", 2024-08-19T01:43:32.662 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.662 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:32.662 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "scaling threshold", 2024-08-19T01:43:32.662 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-08-19T01:43:32.662 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.662 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.662 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-08-19T01:43:32.662 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-08-19T01:43:32.663 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.663 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-08-19T01:43:32.663 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "progress", 2024-08-19T01:43:32.663 INFO:teuthology.orchestra.run.smithi083.stdout: "can_run": true, 2024-08-19T01:43:32.663 INFO:teuthology.orchestra.run.smithi083.stdout: "error_string": "", 2024-08-19T01:43:32.663 INFO:teuthology.orchestra.run.smithi083.stdout: "module_options": { 2024-08-19T01:43:32.663 INFO:teuthology.orchestra.run.smithi083.stdout: "allow_pg_recovery_event": { 2024-08-19T01:43:32.663 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "allow_pg_recovery_event", 2024-08-19T01:43:32.663 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-08-19T01:43:32.663 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.663 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-08-19T01:43:32.663 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-08-19T01:43:32.663 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.664 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.664 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:32.664 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "allow the module to show pg recovery progress", 2024-08-19T01:43:32.664 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.664 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.664 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.664 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.664 INFO:teuthology.orchestra.run.smithi083.stdout: "enabled": { 2024-08-19T01:43:32.664 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "enabled", 2024-08-19T01:43:32.664 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-08-19T01:43:32.664 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.664 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-08-19T01:43:32.664 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "True", 2024-08-19T01:43:32.665 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.665 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.665 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:32.665 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-08-19T01:43:32.665 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.665 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.665 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.665 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.665 INFO:teuthology.orchestra.run.smithi083.stdout: "log_level": { 2024-08-19T01:43:32.665 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_level", 2024-08-19T01:43:32.665 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-08-19T01:43:32.665 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.665 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-08-19T01:43:32.665 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-08-19T01:43:32.666 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.666 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.666 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-08-19T01:43:32.666 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-08-19T01:43:32.666 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-08-19T01:43:32.666 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-08-19T01:43:32.666 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-08-19T01:43:32.666 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-08-19T01:43:32.666 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-08-19T01:43:32.666 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-08-19T01:43:32.666 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-08-19T01:43:32.666 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.666 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.667 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.667 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.667 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster": { 2024-08-19T01:43:32.667 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster", 2024-08-19T01:43:32.667 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-08-19T01:43:32.667 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.667 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-08-19T01:43:32.667 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-08-19T01:43:32.667 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.667 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.667 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:32.667 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-08-19T01:43:32.667 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.668 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.668 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.668 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.668 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster_level": { 2024-08-19T01:43:32.668 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster_level", 2024-08-19T01:43:32.668 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-08-19T01:43:32.668 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.668 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-08-19T01:43:32.668 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "info", 2024-08-19T01:43:32.668 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.668 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.668 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-08-19T01:43:32.668 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-08-19T01:43:32.669 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-08-19T01:43:32.669 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-08-19T01:43:32.669 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-08-19T01:43:32.669 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-08-19T01:43:32.669 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-08-19T01:43:32.669 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-08-19T01:43:32.669 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-08-19T01:43:32.669 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.669 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.669 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.669 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.669 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_file": { 2024-08-19T01:43:32.669 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_file", 2024-08-19T01:43:32.669 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-08-19T01:43:32.670 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.670 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-08-19T01:43:32.670 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-08-19T01:43:32.670 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.670 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.670 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:32.670 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-08-19T01:43:32.670 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.670 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.670 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.670 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.670 INFO:teuthology.orchestra.run.smithi083.stdout: "max_completed_events": { 2024-08-19T01:43:32.670 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "max_completed_events", 2024-08-19T01:43:32.671 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2024-08-19T01:43:32.671 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.671 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-08-19T01:43:32.671 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "50", 2024-08-19T01:43:32.671 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.671 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.671 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:32.671 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "number of past completed events to remember", 2024-08-19T01:43:32.671 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.671 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.671 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.671 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.672 INFO:teuthology.orchestra.run.smithi083.stdout: "sleep_interval": { 2024-08-19T01:43:32.672 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "sleep_interval", 2024-08-19T01:43:32.672 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "secs", 2024-08-19T01:43:32.672 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.672 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-08-19T01:43:32.672 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "5", 2024-08-19T01:43:32.672 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.672 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.672 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:32.672 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "how long the module is going to sleep", 2024-08-19T01:43:32.672 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.672 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.672 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.672 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-08-19T01:43:32.673 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-08-19T01:43:32.673 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.673 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-08-19T01:43:32.673 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "prometheus", 2024-08-19T01:43:32.673 INFO:teuthology.orchestra.run.smithi083.stdout: "can_run": true, 2024-08-19T01:43:32.673 INFO:teuthology.orchestra.run.smithi083.stdout: "error_string": "", 2024-08-19T01:43:32.673 INFO:teuthology.orchestra.run.smithi083.stdout: "module_options": { 2024-08-19T01:43:32.673 INFO:teuthology.orchestra.run.smithi083.stdout: "cache": { 2024-08-19T01:43:32.673 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "cache", 2024-08-19T01:43:32.673 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-08-19T01:43:32.673 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.673 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-08-19T01:43:32.673 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "True", 2024-08-19T01:43:32.673 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.674 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.674 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:32.674 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-08-19T01:43:32.674 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.674 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.674 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.674 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.674 INFO:teuthology.orchestra.run.smithi083.stdout: "exclude_perf_counters": { 2024-08-19T01:43:32.674 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "exclude_perf_counters", 2024-08-19T01:43:32.674 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-08-19T01:43:32.674 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.674 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-08-19T01:43:32.674 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "True", 2024-08-19T01:43:32.675 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.675 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.675 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:32.675 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-08-19T01:43:32.675 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-08-19T01:43:32.675 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.675 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.675 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.675 INFO:teuthology.orchestra.run.smithi083.stdout: "log_level": { 2024-08-19T01:43:32.675 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_level", 2024-08-19T01:43:32.675 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-08-19T01:43:32.675 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.675 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-08-19T01:43:32.676 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-08-19T01:43:32.676 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.676 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.676 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-08-19T01:43:32.676 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-08-19T01:43:32.676 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-08-19T01:43:32.676 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-08-19T01:43:32.676 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-08-19T01:43:32.676 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-08-19T01:43:32.676 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-08-19T01:43:32.676 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-08-19T01:43:32.676 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-08-19T01:43:32.676 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.677 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.677 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.677 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.677 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster": { 2024-08-19T01:43:32.677 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster", 2024-08-19T01:43:32.677 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-08-19T01:43:32.677 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.677 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-08-19T01:43:32.677 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-08-19T01:43:32.677 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.677 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.677 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:32.677 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-08-19T01:43:32.677 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.678 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.678 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.678 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.678 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster_level": { 2024-08-19T01:43:32.678 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster_level", 2024-08-19T01:43:32.678 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-08-19T01:43:32.678 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.678 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-08-19T01:43:32.678 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "info", 2024-08-19T01:43:32.678 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.678 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.679 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-08-19T01:43:32.679 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-08-19T01:43:32.679 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-08-19T01:43:32.679 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-08-19T01:43:32.679 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-08-19T01:43:32.679 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-08-19T01:43:32.679 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-08-19T01:43:32.679 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-08-19T01:43:32.679 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-08-19T01:43:32.679 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.679 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.679 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.679 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.680 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_file": { 2024-08-19T01:43:32.680 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_file", 2024-08-19T01:43:32.680 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-08-19T01:43:32.680 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.680 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-08-19T01:43:32.680 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-08-19T01:43:32.680 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.680 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.680 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:32.680 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-08-19T01:43:32.680 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.680 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.680 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.681 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.681 INFO:teuthology.orchestra.run.smithi083.stdout: "rbd_stats_pools": { 2024-08-19T01:43:32.681 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "rbd_stats_pools", 2024-08-19T01:43:32.681 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-08-19T01:43:32.681 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.681 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-08-19T01:43:32.681 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-08-19T01:43:32.681 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.681 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.681 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:32.681 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-08-19T01:43:32.681 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.681 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.682 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.682 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.682 INFO:teuthology.orchestra.run.smithi083.stdout: "rbd_stats_pools_refresh_interval": { 2024-08-19T01:43:32.682 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-08-19T01:43:32.682 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2024-08-19T01:43:32.682 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.682 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-08-19T01:43:32.682 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "300", 2024-08-19T01:43:32.682 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.682 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.682 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:32.683 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-08-19T01:43:32.683 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.683 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.683 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.683 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.683 INFO:teuthology.orchestra.run.smithi083.stdout: "scrape_interval": { 2024-08-19T01:43:32.683 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "scrape_interval", 2024-08-19T01:43:32.683 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "float", 2024-08-19T01:43:32.683 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.683 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-08-19T01:43:32.683 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "15.0", 2024-08-19T01:43:32.683 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.683 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.683 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:32.684 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-08-19T01:43:32.684 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.684 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.684 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.684 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.684 INFO:teuthology.orchestra.run.smithi083.stdout: "server_addr": { 2024-08-19T01:43:32.684 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "server_addr", 2024-08-19T01:43:32.684 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-08-19T01:43:32.684 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.684 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-08-19T01:43:32.684 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "::", 2024-08-19T01:43:32.684 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.684 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.685 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:32.685 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-08-19T01:43:32.685 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.685 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.685 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.685 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.685 INFO:teuthology.orchestra.run.smithi083.stdout: "server_port": { 2024-08-19T01:43:32.685 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "server_port", 2024-08-19T01:43:32.685 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2024-08-19T01:43:32.685 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.685 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-08-19T01:43:32.685 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "9283", 2024-08-19T01:43:32.686 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.686 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.686 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:32.686 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-08-19T01:43:32.686 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.686 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.686 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.686 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.686 INFO:teuthology.orchestra.run.smithi083.stdout: "stale_cache_strategy": { 2024-08-19T01:43:32.686 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "stale_cache_strategy", 2024-08-19T01:43:32.686 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-08-19T01:43:32.686 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.687 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-08-19T01:43:32.687 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "log", 2024-08-19T01:43:32.687 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.687 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.687 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:32.687 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-08-19T01:43:32.687 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.687 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.687 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.687 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.687 INFO:teuthology.orchestra.run.smithi083.stdout: "standby_behaviour": { 2024-08-19T01:43:32.687 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "standby_behaviour", 2024-08-19T01:43:32.688 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-08-19T01:43:32.688 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.688 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-08-19T01:43:32.688 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "default", 2024-08-19T01:43:32.688 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.688 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.688 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-08-19T01:43:32.688 INFO:teuthology.orchestra.run.smithi083.stdout: "default", 2024-08-19T01:43:32.688 INFO:teuthology.orchestra.run.smithi083.stdout: "error" 2024-08-19T01:43:32.688 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-08-19T01:43:32.688 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-08-19T01:43:32.688 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.688 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.689 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.689 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.689 INFO:teuthology.orchestra.run.smithi083.stdout: "standby_error_status_code": { 2024-08-19T01:43:32.689 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "standby_error_status_code", 2024-08-19T01:43:32.689 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2024-08-19T01:43:32.689 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.689 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-08-19T01:43:32.689 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "500", 2024-08-19T01:43:32.689 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "400", 2024-08-19T01:43:32.689 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "599", 2024-08-19T01:43:32.689 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:32.689 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-08-19T01:43:32.690 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.690 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.690 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.690 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-08-19T01:43:32.690 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-08-19T01:43:32.690 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.690 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-08-19T01:43:32.690 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "rbd_support", 2024-08-19T01:43:32.690 INFO:teuthology.orchestra.run.smithi083.stdout: "can_run": true, 2024-08-19T01:43:32.690 INFO:teuthology.orchestra.run.smithi083.stdout: "error_string": "", 2024-08-19T01:43:32.690 INFO:teuthology.orchestra.run.smithi083.stdout: "module_options": { 2024-08-19T01:43:32.691 INFO:teuthology.orchestra.run.smithi083.stdout: "log_level": { 2024-08-19T01:43:32.691 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_level", 2024-08-19T01:43:32.691 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-08-19T01:43:32.691 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.691 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-08-19T01:43:32.691 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-08-19T01:43:32.691 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.691 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.691 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-08-19T01:43:32.691 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-08-19T01:43:32.691 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-08-19T01:43:32.691 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-08-19T01:43:32.691 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-08-19T01:43:32.692 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-08-19T01:43:32.692 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-08-19T01:43:32.692 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-08-19T01:43:32.692 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-08-19T01:43:32.692 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.692 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.692 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.692 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.692 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster": { 2024-08-19T01:43:32.692 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster", 2024-08-19T01:43:32.692 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-08-19T01:43:32.692 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.692 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-08-19T01:43:32.693 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-08-19T01:43:32.693 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.693 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.693 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:32.693 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-08-19T01:43:32.693 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.693 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.693 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.693 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.693 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster_level": { 2024-08-19T01:43:32.693 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster_level", 2024-08-19T01:43:32.694 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-08-19T01:43:32.694 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.694 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-08-19T01:43:32.694 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "info", 2024-08-19T01:43:32.694 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.694 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.694 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-08-19T01:43:32.694 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-08-19T01:43:32.694 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-08-19T01:43:32.694 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-08-19T01:43:32.694 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-08-19T01:43:32.694 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-08-19T01:43:32.694 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-08-19T01:43:32.695 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-08-19T01:43:32.695 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-08-19T01:43:32.695 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.695 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.695 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.695 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.695 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_file": { 2024-08-19T01:43:32.695 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_file", 2024-08-19T01:43:32.695 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-08-19T01:43:32.695 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.695 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-08-19T01:43:32.695 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-08-19T01:43:32.695 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.695 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.696 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:32.696 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-08-19T01:43:32.696 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.696 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.696 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.696 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.696 INFO:teuthology.orchestra.run.smithi083.stdout: "max_concurrent_snap_create": { 2024-08-19T01:43:32.696 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "max_concurrent_snap_create", 2024-08-19T01:43:32.696 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2024-08-19T01:43:32.696 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.696 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-08-19T01:43:32.696 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "10", 2024-08-19T01:43:32.696 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.697 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.697 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:32.697 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-08-19T01:43:32.697 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.697 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.697 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.697 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.697 INFO:teuthology.orchestra.run.smithi083.stdout: "mirror_snapshot_schedule": { 2024-08-19T01:43:32.697 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "mirror_snapshot_schedule", 2024-08-19T01:43:32.697 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-08-19T01:43:32.697 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.697 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-08-19T01:43:32.697 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-08-19T01:43:32.698 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.698 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.698 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:32.698 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-08-19T01:43:32.698 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.698 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.698 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.698 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.698 INFO:teuthology.orchestra.run.smithi083.stdout: "trash_purge_schedule": { 2024-08-19T01:43:32.698 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "trash_purge_schedule", 2024-08-19T01:43:32.698 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-08-19T01:43:32.698 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.698 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-08-19T01:43:32.699 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-08-19T01:43:32.699 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.699 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.699 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:32.699 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-08-19T01:43:32.699 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.699 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.699 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.699 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-08-19T01:43:32.699 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-08-19T01:43:32.699 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.699 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-08-19T01:43:32.699 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "restful", 2024-08-19T01:43:32.699 INFO:teuthology.orchestra.run.smithi083.stdout: "can_run": true, 2024-08-19T01:43:32.700 INFO:teuthology.orchestra.run.smithi083.stdout: "error_string": "", 2024-08-19T01:43:32.700 INFO:teuthology.orchestra.run.smithi083.stdout: "module_options": { 2024-08-19T01:43:32.700 INFO:teuthology.orchestra.run.smithi083.stdout: "enable_auth": { 2024-08-19T01:43:32.700 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "enable_auth", 2024-08-19T01:43:32.700 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-08-19T01:43:32.700 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.700 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-08-19T01:43:32.700 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "True", 2024-08-19T01:43:32.700 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.700 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.700 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:32.700 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-08-19T01:43:32.700 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.700 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.700 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.701 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.701 INFO:teuthology.orchestra.run.smithi083.stdout: "key_file": { 2024-08-19T01:43:32.701 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "key_file", 2024-08-19T01:43:32.701 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-08-19T01:43:32.701 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.701 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-08-19T01:43:32.701 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-08-19T01:43:32.701 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.701 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.701 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:32.701 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-08-19T01:43:32.701 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.702 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.702 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.702 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.702 INFO:teuthology.orchestra.run.smithi083.stdout: "log_level": { 2024-08-19T01:43:32.702 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_level", 2024-08-19T01:43:32.702 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-08-19T01:43:32.702 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.702 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-08-19T01:43:32.702 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-08-19T01:43:32.702 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.702 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.702 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-08-19T01:43:32.703 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-08-19T01:43:32.703 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-08-19T01:43:32.703 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-08-19T01:43:32.703 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-08-19T01:43:32.703 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-08-19T01:43:32.703 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-08-19T01:43:32.703 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-08-19T01:43:32.703 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-08-19T01:43:32.703 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.703 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.703 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.704 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.704 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster": { 2024-08-19T01:43:32.704 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster", 2024-08-19T01:43:32.704 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-08-19T01:43:32.704 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.704 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-08-19T01:43:32.704 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-08-19T01:43:32.704 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.704 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.704 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:32.704 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-08-19T01:43:32.704 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.704 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.705 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.705 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.705 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster_level": { 2024-08-19T01:43:32.705 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster_level", 2024-08-19T01:43:32.705 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-08-19T01:43:32.705 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.705 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-08-19T01:43:32.705 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "info", 2024-08-19T01:43:32.705 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.705 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.705 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-08-19T01:43:32.705 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-08-19T01:43:32.705 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-08-19T01:43:32.705 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-08-19T01:43:32.705 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-08-19T01:43:32.706 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-08-19T01:43:32.706 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-08-19T01:43:32.706 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-08-19T01:43:32.706 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-08-19T01:43:32.706 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.706 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.706 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.706 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.706 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_file": { 2024-08-19T01:43:32.706 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_file", 2024-08-19T01:43:32.706 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-08-19T01:43:32.706 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.706 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-08-19T01:43:32.707 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-08-19T01:43:32.707 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.707 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.707 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:32.707 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-08-19T01:43:32.707 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.707 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.707 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.707 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.707 INFO:teuthology.orchestra.run.smithi083.stdout: "server_addr": { 2024-08-19T01:43:32.707 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "server_addr", 2024-08-19T01:43:32.707 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-08-19T01:43:32.707 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.707 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-08-19T01:43:32.707 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-08-19T01:43:32.708 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.708 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.708 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:32.708 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-08-19T01:43:32.708 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.708 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.708 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.708 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.708 INFO:teuthology.orchestra.run.smithi083.stdout: "server_port": { 2024-08-19T01:43:32.708 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "server_port", 2024-08-19T01:43:32.708 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-08-19T01:43:32.708 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.708 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-08-19T01:43:32.708 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-08-19T01:43:32.709 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.709 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.709 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:32.709 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-08-19T01:43:32.709 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.709 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.709 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.709 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-08-19T01:43:32.709 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-08-19T01:43:32.709 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.709 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-08-19T01:43:32.709 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "rgw", 2024-08-19T01:43:32.709 INFO:teuthology.orchestra.run.smithi083.stdout: "can_run": true, 2024-08-19T01:43:32.709 INFO:teuthology.orchestra.run.smithi083.stdout: "error_string": "", 2024-08-19T01:43:32.710 INFO:teuthology.orchestra.run.smithi083.stdout: "module_options": { 2024-08-19T01:43:32.710 INFO:teuthology.orchestra.run.smithi083.stdout: "log_level": { 2024-08-19T01:43:32.710 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_level", 2024-08-19T01:43:32.710 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-08-19T01:43:32.710 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.710 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-08-19T01:43:32.710 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-08-19T01:43:32.710 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.710 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.710 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-08-19T01:43:32.710 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-08-19T01:43:32.710 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-08-19T01:43:32.711 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-08-19T01:43:32.711 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-08-19T01:43:32.711 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-08-19T01:43:32.711 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-08-19T01:43:32.711 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-08-19T01:43:32.711 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-08-19T01:43:32.711 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.711 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.711 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.711 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.711 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster": { 2024-08-19T01:43:32.711 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster", 2024-08-19T01:43:32.712 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-08-19T01:43:32.712 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.712 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-08-19T01:43:32.712 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-08-19T01:43:32.712 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.712 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.712 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:32.712 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-08-19T01:43:32.712 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.712 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.712 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.712 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.712 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster_level": { 2024-08-19T01:43:32.713 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster_level", 2024-08-19T01:43:32.713 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-08-19T01:43:32.713 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.713 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-08-19T01:43:32.713 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "info", 2024-08-19T01:43:32.713 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.713 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.713 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-08-19T01:43:32.713 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-08-19T01:43:32.713 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-08-19T01:43:32.713 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-08-19T01:43:32.713 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-08-19T01:43:32.713 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-08-19T01:43:32.714 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-08-19T01:43:32.714 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-08-19T01:43:32.714 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-08-19T01:43:32.714 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.714 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.714 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.714 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.714 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_file": { 2024-08-19T01:43:32.714 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_file", 2024-08-19T01:43:32.714 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-08-19T01:43:32.714 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.714 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-08-19T01:43:32.715 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-08-19T01:43:32.715 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.715 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.715 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:32.715 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-08-19T01:43:32.715 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.715 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.715 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.715 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-08-19T01:43:32.715 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-08-19T01:43:32.715 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.715 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-08-19T01:43:32.715 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "rook", 2024-08-19T01:43:32.715 INFO:teuthology.orchestra.run.smithi083.stdout: "can_run": true, 2024-08-19T01:43:32.716 INFO:teuthology.orchestra.run.smithi083.stdout: "error_string": "", 2024-08-19T01:43:32.716 INFO:teuthology.orchestra.run.smithi083.stdout: "module_options": { 2024-08-19T01:43:32.716 INFO:teuthology.orchestra.run.smithi083.stdout: "log_level": { 2024-08-19T01:43:32.716 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_level", 2024-08-19T01:43:32.716 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-08-19T01:43:32.716 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.716 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-08-19T01:43:32.716 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-08-19T01:43:32.716 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.716 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.716 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-08-19T01:43:32.716 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-08-19T01:43:32.716 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-08-19T01:43:32.716 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-08-19T01:43:32.717 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-08-19T01:43:32.717 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-08-19T01:43:32.717 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-08-19T01:43:32.717 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-08-19T01:43:32.717 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-08-19T01:43:32.717 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.717 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.717 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.717 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.717 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster": { 2024-08-19T01:43:32.717 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster", 2024-08-19T01:43:32.717 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-08-19T01:43:32.717 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.718 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-08-19T01:43:32.718 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-08-19T01:43:32.718 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.718 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.718 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:32.718 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-08-19T01:43:32.718 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.718 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.718 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.718 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.718 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster_level": { 2024-08-19T01:43:32.718 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster_level", 2024-08-19T01:43:32.718 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-08-19T01:43:32.719 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.719 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-08-19T01:43:32.719 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "info", 2024-08-19T01:43:32.719 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.719 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.719 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-08-19T01:43:32.719 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-08-19T01:43:32.719 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-08-19T01:43:32.719 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-08-19T01:43:32.719 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-08-19T01:43:32.719 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-08-19T01:43:32.719 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-08-19T01:43:32.719 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-08-19T01:43:32.720 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-08-19T01:43:32.720 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.720 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.720 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.720 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.720 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_file": { 2024-08-19T01:43:32.720 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_file", 2024-08-19T01:43:32.720 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-08-19T01:43:32.720 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.720 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-08-19T01:43:32.720 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-08-19T01:43:32.720 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.720 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.721 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:32.721 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-08-19T01:43:32.721 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.721 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.721 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.721 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.721 INFO:teuthology.orchestra.run.smithi083.stdout: "storage_class": { 2024-08-19T01:43:32.721 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "storage_class", 2024-08-19T01:43:32.721 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-08-19T01:43:32.721 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.721 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-08-19T01:43:32.721 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "local", 2024-08-19T01:43:32.721 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.722 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.722 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:32.722 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-08-19T01:43:32.722 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.722 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.722 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.722 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-08-19T01:43:32.722 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-08-19T01:43:32.722 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.722 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-08-19T01:43:32.722 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "selftest", 2024-08-19T01:43:32.722 INFO:teuthology.orchestra.run.smithi083.stdout: "can_run": true, 2024-08-19T01:43:32.722 INFO:teuthology.orchestra.run.smithi083.stdout: "error_string": "", 2024-08-19T01:43:32.723 INFO:teuthology.orchestra.run.smithi083.stdout: "module_options": { 2024-08-19T01:43:32.723 INFO:teuthology.orchestra.run.smithi083.stdout: "log_level": { 2024-08-19T01:43:32.723 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_level", 2024-08-19T01:43:32.723 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-08-19T01:43:32.723 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.723 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-08-19T01:43:32.723 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-08-19T01:43:32.723 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.723 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.723 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-08-19T01:43:32.723 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-08-19T01:43:32.723 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-08-19T01:43:32.723 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-08-19T01:43:32.724 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-08-19T01:43:32.724 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-08-19T01:43:32.724 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-08-19T01:43:32.724 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-08-19T01:43:32.724 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-08-19T01:43:32.724 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.724 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.724 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.724 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.724 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster": { 2024-08-19T01:43:32.724 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster", 2024-08-19T01:43:32.724 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-08-19T01:43:32.724 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.725 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-08-19T01:43:32.725 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-08-19T01:43:32.725 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.725 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.725 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:32.725 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-08-19T01:43:32.725 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.725 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.725 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.725 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.725 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster_level": { 2024-08-19T01:43:32.725 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster_level", 2024-08-19T01:43:32.725 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-08-19T01:43:32.725 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.726 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-08-19T01:43:32.726 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "info", 2024-08-19T01:43:32.726 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.726 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.726 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-08-19T01:43:32.726 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-08-19T01:43:32.726 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-08-19T01:43:32.726 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-08-19T01:43:32.726 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-08-19T01:43:32.726 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-08-19T01:43:32.726 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-08-19T01:43:32.726 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-08-19T01:43:32.727 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-08-19T01:43:32.727 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.727 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.727 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.727 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.727 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_file": { 2024-08-19T01:43:32.727 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_file", 2024-08-19T01:43:32.727 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-08-19T01:43:32.727 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.727 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-08-19T01:43:32.727 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-08-19T01:43:32.727 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.727 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.727 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:32.727 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-08-19T01:43:32.728 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.728 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.728 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.728 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.728 INFO:teuthology.orchestra.run.smithi083.stdout: "roption1": { 2024-08-19T01:43:32.728 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "roption1", 2024-08-19T01:43:32.728 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-08-19T01:43:32.728 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.728 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-08-19T01:43:32.728 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-08-19T01:43:32.728 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.728 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.728 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:32.728 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-08-19T01:43:32.728 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.729 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.729 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.729 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.729 INFO:teuthology.orchestra.run.smithi083.stdout: "roption2": { 2024-08-19T01:43:32.729 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "roption2", 2024-08-19T01:43:32.729 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-08-19T01:43:32.729 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.729 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-08-19T01:43:32.729 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "xyz", 2024-08-19T01:43:32.729 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.729 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.729 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:32.729 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-08-19T01:43:32.729 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.730 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.730 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.730 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.730 INFO:teuthology.orchestra.run.smithi083.stdout: "rwoption1": { 2024-08-19T01:43:32.730 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "rwoption1", 2024-08-19T01:43:32.730 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-08-19T01:43:32.730 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.730 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-08-19T01:43:32.730 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-08-19T01:43:32.730 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.730 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.730 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:32.730 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-08-19T01:43:32.730 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.730 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.731 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.731 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.731 INFO:teuthology.orchestra.run.smithi083.stdout: "rwoption2": { 2024-08-19T01:43:32.731 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "rwoption2", 2024-08-19T01:43:32.731 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2024-08-19T01:43:32.731 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.731 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-08-19T01:43:32.731 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-08-19T01:43:32.731 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.731 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.731 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:32.731 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-08-19T01:43:32.731 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.732 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.732 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.732 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.732 INFO:teuthology.orchestra.run.smithi083.stdout: "rwoption3": { 2024-08-19T01:43:32.732 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "rwoption3", 2024-08-19T01:43:32.732 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "float", 2024-08-19T01:43:32.732 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.732 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-08-19T01:43:32.732 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-08-19T01:43:32.732 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.732 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.732 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:32.732 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-08-19T01:43:32.732 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.733 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.733 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.733 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.733 INFO:teuthology.orchestra.run.smithi083.stdout: "rwoption4": { 2024-08-19T01:43:32.733 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "rwoption4", 2024-08-19T01:43:32.733 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-08-19T01:43:32.733 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.733 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-08-19T01:43:32.733 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-08-19T01:43:32.733 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.733 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.733 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:32.733 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-08-19T01:43:32.734 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.734 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.734 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.734 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.734 INFO:teuthology.orchestra.run.smithi083.stdout: "rwoption5": { 2024-08-19T01:43:32.734 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "rwoption5", 2024-08-19T01:43:32.734 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-08-19T01:43:32.734 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.734 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-08-19T01:43:32.734 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-08-19T01:43:32.734 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.734 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.734 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:32.735 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-08-19T01:43:32.735 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.735 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.735 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.735 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.735 INFO:teuthology.orchestra.run.smithi083.stdout: "rwoption6": { 2024-08-19T01:43:32.735 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "rwoption6", 2024-08-19T01:43:32.735 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-08-19T01:43:32.735 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.735 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-08-19T01:43:32.735 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "True", 2024-08-19T01:43:32.735 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.735 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.735 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:32.736 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-08-19T01:43:32.736 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.736 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.736 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.736 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.736 INFO:teuthology.orchestra.run.smithi083.stdout: "rwoption7": { 2024-08-19T01:43:32.736 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "rwoption7", 2024-08-19T01:43:32.736 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2024-08-19T01:43:32.736 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.736 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-08-19T01:43:32.736 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-08-19T01:43:32.736 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "1", 2024-08-19T01:43:32.736 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "42", 2024-08-19T01:43:32.736 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:32.737 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-08-19T01:43:32.737 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.737 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.737 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.737 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.737 INFO:teuthology.orchestra.run.smithi083.stdout: "testkey": { 2024-08-19T01:43:32.737 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "testkey", 2024-08-19T01:43:32.737 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-08-19T01:43:32.737 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.737 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-08-19T01:43:32.737 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-08-19T01:43:32.737 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.737 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.737 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:32.738 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-08-19T01:43:32.738 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.738 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.738 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.738 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.738 INFO:teuthology.orchestra.run.smithi083.stdout: "testlkey": { 2024-08-19T01:43:32.738 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "testlkey", 2024-08-19T01:43:32.738 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-08-19T01:43:32.738 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.738 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-08-19T01:43:32.738 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-08-19T01:43:32.738 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.738 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.739 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:32.739 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-08-19T01:43:32.739 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.739 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.739 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.739 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.739 INFO:teuthology.orchestra.run.smithi083.stdout: "testnewline": { 2024-08-19T01:43:32.739 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "testnewline", 2024-08-19T01:43:32.739 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-08-19T01:43:32.739 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.739 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-08-19T01:43:32.739 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-08-19T01:43:32.739 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.739 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.739 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:32.740 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-08-19T01:43:32.740 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.740 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.740 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.740 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-08-19T01:43:32.740 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-08-19T01:43:32.740 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.740 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-08-19T01:43:32.740 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "snap_schedule", 2024-08-19T01:43:32.740 INFO:teuthology.orchestra.run.smithi083.stdout: "can_run": true, 2024-08-19T01:43:32.740 INFO:teuthology.orchestra.run.smithi083.stdout: "error_string": "", 2024-08-19T01:43:32.740 INFO:teuthology.orchestra.run.smithi083.stdout: "module_options": { 2024-08-19T01:43:32.740 INFO:teuthology.orchestra.run.smithi083.stdout: "allow_m_granularity": { 2024-08-19T01:43:32.741 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "allow_m_granularity", 2024-08-19T01:43:32.741 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-08-19T01:43:32.741 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.741 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-08-19T01:43:32.741 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-08-19T01:43:32.741 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.741 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.741 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:32.741 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "allow minute scheduled snapshots", 2024-08-19T01:43:32.741 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.741 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.741 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.741 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.742 INFO:teuthology.orchestra.run.smithi083.stdout: "dump_on_update": { 2024-08-19T01:43:32.742 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "dump_on_update", 2024-08-19T01:43:32.742 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-08-19T01:43:32.742 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.742 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-08-19T01:43:32.742 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-08-19T01:43:32.742 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.742 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.742 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:32.742 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "dump database to debug log on update", 2024-08-19T01:43:32.742 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.742 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.742 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.742 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.743 INFO:teuthology.orchestra.run.smithi083.stdout: "log_level": { 2024-08-19T01:43:32.743 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_level", 2024-08-19T01:43:32.743 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-08-19T01:43:32.743 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.743 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-08-19T01:43:32.743 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-08-19T01:43:32.743 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.743 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.743 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-08-19T01:43:32.743 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-08-19T01:43:32.743 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-08-19T01:43:32.743 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-08-19T01:43:32.743 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-08-19T01:43:32.744 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-08-19T01:43:32.744 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-08-19T01:43:32.744 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-08-19T01:43:32.744 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-08-19T01:43:32.744 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.744 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.744 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.744 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.744 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster": { 2024-08-19T01:43:32.744 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster", 2024-08-19T01:43:32.744 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-08-19T01:43:32.744 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.744 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-08-19T01:43:32.744 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-08-19T01:43:32.745 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.745 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.745 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:32.745 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-08-19T01:43:32.745 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.745 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.745 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.745 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.745 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster_level": { 2024-08-19T01:43:32.745 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster_level", 2024-08-19T01:43:32.745 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-08-19T01:43:32.745 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.745 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-08-19T01:43:32.745 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "info", 2024-08-19T01:43:32.746 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.746 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.746 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-08-19T01:43:32.746 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-08-19T01:43:32.746 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-08-19T01:43:32.746 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-08-19T01:43:32.746 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-08-19T01:43:32.746 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-08-19T01:43:32.746 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-08-19T01:43:32.746 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-08-19T01:43:32.746 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-08-19T01:43:32.746 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.747 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.747 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.747 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.747 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_file": { 2024-08-19T01:43:32.747 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_file", 2024-08-19T01:43:32.747 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-08-19T01:43:32.747 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.747 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-08-19T01:43:32.747 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-08-19T01:43:32.747 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.747 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.747 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:32.747 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-08-19T01:43:32.747 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.747 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.748 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.748 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-08-19T01:43:32.748 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-08-19T01:43:32.748 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.748 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-08-19T01:43:32.748 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "stats", 2024-08-19T01:43:32.748 INFO:teuthology.orchestra.run.smithi083.stdout: "can_run": true, 2024-08-19T01:43:32.748 INFO:teuthology.orchestra.run.smithi083.stdout: "error_string": "", 2024-08-19T01:43:32.748 INFO:teuthology.orchestra.run.smithi083.stdout: "module_options": { 2024-08-19T01:43:32.748 INFO:teuthology.orchestra.run.smithi083.stdout: "log_level": { 2024-08-19T01:43:32.748 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_level", 2024-08-19T01:43:32.748 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-08-19T01:43:32.748 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.748 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-08-19T01:43:32.749 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-08-19T01:43:32.749 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.749 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.749 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-08-19T01:43:32.749 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-08-19T01:43:32.749 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-08-19T01:43:32.749 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-08-19T01:43:32.749 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-08-19T01:43:32.749 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-08-19T01:43:32.749 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-08-19T01:43:32.749 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-08-19T01:43:32.749 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-08-19T01:43:32.749 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.749 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.749 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.749 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.750 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster": { 2024-08-19T01:43:32.750 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster", 2024-08-19T01:43:32.750 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-08-19T01:43:32.750 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.750 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-08-19T01:43:32.750 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-08-19T01:43:32.750 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.750 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.750 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:32.750 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-08-19T01:43:32.750 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.750 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.751 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.751 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.751 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster_level": { 2024-08-19T01:43:32.751 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster_level", 2024-08-19T01:43:32.751 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-08-19T01:43:32.751 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.751 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-08-19T01:43:32.751 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "info", 2024-08-19T01:43:32.751 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.751 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.751 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-08-19T01:43:32.751 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-08-19T01:43:32.751 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-08-19T01:43:32.752 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-08-19T01:43:32.752 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-08-19T01:43:32.752 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-08-19T01:43:32.752 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-08-19T01:43:32.752 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-08-19T01:43:32.752 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-08-19T01:43:32.752 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.752 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.752 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.752 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.752 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_file": { 2024-08-19T01:43:32.752 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_file", 2024-08-19T01:43:32.752 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-08-19T01:43:32.753 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.753 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-08-19T01:43:32.753 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-08-19T01:43:32.753 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.753 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.753 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:32.753 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-08-19T01:43:32.753 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.753 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.753 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.753 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-08-19T01:43:32.753 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-08-19T01:43:32.753 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.754 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-08-19T01:43:32.754 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "status", 2024-08-19T01:43:32.754 INFO:teuthology.orchestra.run.smithi083.stdout: "can_run": true, 2024-08-19T01:43:32.754 INFO:teuthology.orchestra.run.smithi083.stdout: "error_string": "", 2024-08-19T01:43:32.754 INFO:teuthology.orchestra.run.smithi083.stdout: "module_options": { 2024-08-19T01:43:32.754 INFO:teuthology.orchestra.run.smithi083.stdout: "log_level": { 2024-08-19T01:43:32.754 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_level", 2024-08-19T01:43:32.754 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-08-19T01:43:32.754 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.754 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-08-19T01:43:32.754 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-08-19T01:43:32.754 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.754 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.755 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-08-19T01:43:32.755 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-08-19T01:43:32.755 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-08-19T01:43:32.755 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-08-19T01:43:32.755 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-08-19T01:43:32.755 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-08-19T01:43:32.755 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-08-19T01:43:32.755 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-08-19T01:43:32.755 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-08-19T01:43:32.755 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.755 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.755 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.755 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.756 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster": { 2024-08-19T01:43:32.756 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster", 2024-08-19T01:43:32.756 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-08-19T01:43:32.756 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.756 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-08-19T01:43:32.756 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-08-19T01:43:32.756 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.756 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.756 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:32.756 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-08-19T01:43:32.756 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.756 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.756 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.757 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.757 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster_level": { 2024-08-19T01:43:32.757 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster_level", 2024-08-19T01:43:32.757 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-08-19T01:43:32.757 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.757 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-08-19T01:43:32.757 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "info", 2024-08-19T01:43:32.757 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.757 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.757 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-08-19T01:43:32.757 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-08-19T01:43:32.757 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-08-19T01:43:32.757 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-08-19T01:43:32.758 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-08-19T01:43:32.758 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-08-19T01:43:32.758 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-08-19T01:43:32.758 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-08-19T01:43:32.758 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-08-19T01:43:32.758 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.758 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.758 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.758 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.758 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_file": { 2024-08-19T01:43:32.758 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_file", 2024-08-19T01:43:32.758 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-08-19T01:43:32.759 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.759 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-08-19T01:43:32.759 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-08-19T01:43:32.759 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.759 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.759 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:32.759 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-08-19T01:43:32.759 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.759 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.759 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.759 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-08-19T01:43:32.759 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-08-19T01:43:32.759 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.759 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-08-19T01:43:32.760 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "telegraf", 2024-08-19T01:43:32.760 INFO:teuthology.orchestra.run.smithi083.stdout: "can_run": true, 2024-08-19T01:43:32.760 INFO:teuthology.orchestra.run.smithi083.stdout: "error_string": "", 2024-08-19T01:43:32.760 INFO:teuthology.orchestra.run.smithi083.stdout: "module_options": { 2024-08-19T01:43:32.760 INFO:teuthology.orchestra.run.smithi083.stdout: "address": { 2024-08-19T01:43:32.760 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "address", 2024-08-19T01:43:32.760 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-08-19T01:43:32.760 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.760 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-08-19T01:43:32.760 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-08-19T01:43:32.760 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.760 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.760 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:32.761 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-08-19T01:43:32.761 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.761 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.761 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.761 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.761 INFO:teuthology.orchestra.run.smithi083.stdout: "interval": { 2024-08-19T01:43:32.761 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "interval", 2024-08-19T01:43:32.761 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "secs", 2024-08-19T01:43:32.761 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.761 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-08-19T01:43:32.761 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "15", 2024-08-19T01:43:32.761 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.762 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.762 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:32.762 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-08-19T01:43:32.762 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.762 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.762 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.762 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.762 INFO:teuthology.orchestra.run.smithi083.stdout: "log_level": { 2024-08-19T01:43:32.762 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_level", 2024-08-19T01:43:32.762 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-08-19T01:43:32.762 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.762 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-08-19T01:43:32.763 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-08-19T01:43:32.763 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.763 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.763 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-08-19T01:43:32.763 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-08-19T01:43:32.763 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-08-19T01:43:32.763 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-08-19T01:43:32.763 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-08-19T01:43:32.763 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-08-19T01:43:32.763 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-08-19T01:43:32.763 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-08-19T01:43:32.763 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-08-19T01:43:32.763 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.763 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.764 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.764 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.764 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster": { 2024-08-19T01:43:32.764 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster", 2024-08-19T01:43:32.764 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-08-19T01:43:32.764 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.764 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-08-19T01:43:32.764 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-08-19T01:43:32.764 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.764 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.764 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:32.764 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-08-19T01:43:32.764 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.764 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.765 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.765 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.765 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster_level": { 2024-08-19T01:43:32.765 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster_level", 2024-08-19T01:43:32.765 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-08-19T01:43:32.765 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.765 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-08-19T01:43:32.765 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "info", 2024-08-19T01:43:32.765 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.765 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.766 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-08-19T01:43:32.766 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-08-19T01:43:32.766 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-08-19T01:43:32.766 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-08-19T01:43:32.766 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-08-19T01:43:32.766 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-08-19T01:43:32.766 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-08-19T01:43:32.766 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-08-19T01:43:32.766 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-08-19T01:43:32.766 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.766 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.766 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.766 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.767 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_file": { 2024-08-19T01:43:32.767 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_file", 2024-08-19T01:43:32.767 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-08-19T01:43:32.767 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.767 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-08-19T01:43:32.767 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-08-19T01:43:32.767 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.767 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.767 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:32.767 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-08-19T01:43:32.767 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.767 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.767 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.768 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-08-19T01:43:32.768 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-08-19T01:43:32.768 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.768 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-08-19T01:43:32.768 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "telemetry", 2024-08-19T01:43:32.768 INFO:teuthology.orchestra.run.smithi083.stdout: "can_run": true, 2024-08-19T01:43:32.768 INFO:teuthology.orchestra.run.smithi083.stdout: "error_string": "", 2024-08-19T01:43:32.768 INFO:teuthology.orchestra.run.smithi083.stdout: "module_options": { 2024-08-19T01:43:32.768 INFO:teuthology.orchestra.run.smithi083.stdout: "channel_basic": { 2024-08-19T01:43:32.768 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "channel_basic", 2024-08-19T01:43:32.768 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-08-19T01:43:32.768 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.768 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-08-19T01:43:32.768 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "True", 2024-08-19T01:43:32.769 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.769 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.769 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:32.769 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Share basic cluster information (size, version)", 2024-08-19T01:43:32.769 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.769 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.769 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.769 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.769 INFO:teuthology.orchestra.run.smithi083.stdout: "channel_crash": { 2024-08-19T01:43:32.769 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "channel_crash", 2024-08-19T01:43:32.769 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-08-19T01:43:32.769 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.770 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-08-19T01:43:32.770 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "True", 2024-08-19T01:43:32.770 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.770 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.770 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:32.770 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-08-19T01:43:32.770 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.770 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.770 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.770 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.770 INFO:teuthology.orchestra.run.smithi083.stdout: "channel_device": { 2024-08-19T01:43:32.770 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "channel_device", 2024-08-19T01:43:32.771 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-08-19T01:43:32.771 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.771 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-08-19T01:43:32.771 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "True", 2024-08-19T01:43:32.771 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.771 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.771 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:32.771 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-08-19T01:43:32.771 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.771 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.771 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.771 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.772 INFO:teuthology.orchestra.run.smithi083.stdout: "channel_ident": { 2024-08-19T01:43:32.772 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "channel_ident", 2024-08-19T01:43:32.772 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-08-19T01:43:32.772 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.772 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-08-19T01:43:32.772 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-08-19T01:43:32.772 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.772 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.772 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:32.772 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-08-19T01:43:32.772 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.772 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.772 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.773 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.773 INFO:teuthology.orchestra.run.smithi083.stdout: "channel_perf": { 2024-08-19T01:43:32.773 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "channel_perf", 2024-08-19T01:43:32.773 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-08-19T01:43:32.773 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.773 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-08-19T01:43:32.773 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-08-19T01:43:32.773 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.773 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.773 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:32.773 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Share various performance metrics of a cluster", 2024-08-19T01:43:32.773 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.774 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.774 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.774 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.774 INFO:teuthology.orchestra.run.smithi083.stdout: "contact": { 2024-08-19T01:43:32.774 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "contact", 2024-08-19T01:43:32.774 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-08-19T01:43:32.774 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.774 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-08-19T01:43:32.774 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-08-19T01:43:32.774 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.774 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.774 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:32.774 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-08-19T01:43:32.775 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.775 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.775 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.775 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.775 INFO:teuthology.orchestra.run.smithi083.stdout: "description": { 2024-08-19T01:43:32.775 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "description", 2024-08-19T01:43:32.775 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-08-19T01:43:32.775 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.775 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-08-19T01:43:32.775 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-08-19T01:43:32.775 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.775 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.775 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:32.775 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-08-19T01:43:32.776 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.776 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.776 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.776 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.776 INFO:teuthology.orchestra.run.smithi083.stdout: "device_url": { 2024-08-19T01:43:32.776 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "device_url", 2024-08-19T01:43:32.776 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-08-19T01:43:32.776 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.776 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-08-19T01:43:32.776 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-08-19T01:43:32.776 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.776 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.776 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:32.776 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-08-19T01:43:32.777 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.777 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.777 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.777 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.777 INFO:teuthology.orchestra.run.smithi083.stdout: "enabled": { 2024-08-19T01:43:32.777 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "enabled", 2024-08-19T01:43:32.777 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-08-19T01:43:32.777 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.777 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-08-19T01:43:32.777 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-08-19T01:43:32.777 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.777 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.777 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:32.777 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-08-19T01:43:32.777 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.777 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.778 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.778 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.778 INFO:teuthology.orchestra.run.smithi083.stdout: "interval": { 2024-08-19T01:43:32.778 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "interval", 2024-08-19T01:43:32.778 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2024-08-19T01:43:32.778 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.778 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-08-19T01:43:32.778 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "24", 2024-08-19T01:43:32.778 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "8", 2024-08-19T01:43:32.778 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.778 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:32.778 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-08-19T01:43:32.778 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.779 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.779 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.779 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.779 INFO:teuthology.orchestra.run.smithi083.stdout: "last_opt_revision": { 2024-08-19T01:43:32.779 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "last_opt_revision", 2024-08-19T01:43:32.779 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2024-08-19T01:43:32.779 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.779 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-08-19T01:43:32.779 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "1", 2024-08-19T01:43:32.779 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.779 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.779 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:32.779 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-08-19T01:43:32.780 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.780 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.780 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.780 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.780 INFO:teuthology.orchestra.run.smithi083.stdout: "leaderboard": { 2024-08-19T01:43:32.780 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "leaderboard", 2024-08-19T01:43:32.780 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-08-19T01:43:32.780 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.780 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-08-19T01:43:32.780 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-08-19T01:43:32.780 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.780 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.780 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:32.780 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-08-19T01:43:32.781 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.781 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.781 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.781 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.781 INFO:teuthology.orchestra.run.smithi083.stdout: "leaderboard_description": { 2024-08-19T01:43:32.781 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "leaderboard_description", 2024-08-19T01:43:32.781 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-08-19T01:43:32.781 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.781 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-08-19T01:43:32.781 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-08-19T01:43:32.781 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.781 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.782 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:32.782 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-08-19T01:43:32.782 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.782 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.782 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.782 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.782 INFO:teuthology.orchestra.run.smithi083.stdout: "log_level": { 2024-08-19T01:43:32.782 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_level", 2024-08-19T01:43:32.782 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-08-19T01:43:32.782 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.782 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-08-19T01:43:32.782 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-08-19T01:43:32.783 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.783 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.783 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-08-19T01:43:32.783 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-08-19T01:43:32.783 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-08-19T01:43:32.783 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-08-19T01:43:32.783 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-08-19T01:43:32.783 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-08-19T01:43:32.783 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-08-19T01:43:32.783 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-08-19T01:43:32.783 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-08-19T01:43:32.783 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.784 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.784 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.784 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.784 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster": { 2024-08-19T01:43:32.784 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster", 2024-08-19T01:43:32.784 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-08-19T01:43:32.784 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.784 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-08-19T01:43:32.784 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-08-19T01:43:32.784 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.784 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.784 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:32.784 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-08-19T01:43:32.785 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.785 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.785 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.785 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.785 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster_level": { 2024-08-19T01:43:32.785 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster_level", 2024-08-19T01:43:32.785 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-08-19T01:43:32.785 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.785 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-08-19T01:43:32.785 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "info", 2024-08-19T01:43:32.785 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.785 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.785 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-08-19T01:43:32.786 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-08-19T01:43:32.786 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-08-19T01:43:32.786 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-08-19T01:43:32.786 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-08-19T01:43:32.786 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-08-19T01:43:32.786 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-08-19T01:43:32.786 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-08-19T01:43:32.786 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-08-19T01:43:32.786 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.786 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.786 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.786 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.786 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_file": { 2024-08-19T01:43:32.786 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_file", 2024-08-19T01:43:32.787 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-08-19T01:43:32.787 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.787 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-08-19T01:43:32.787 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-08-19T01:43:32.787 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.787 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.787 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:32.787 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-08-19T01:43:32.787 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.787 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.787 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.787 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.787 INFO:teuthology.orchestra.run.smithi083.stdout: "organization": { 2024-08-19T01:43:32.788 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "organization", 2024-08-19T01:43:32.788 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-08-19T01:43:32.788 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.788 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-08-19T01:43:32.788 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-08-19T01:43:32.788 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.788 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.788 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:32.788 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-08-19T01:43:32.788 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.788 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.788 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.788 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.788 INFO:teuthology.orchestra.run.smithi083.stdout: "proxy": { 2024-08-19T01:43:32.789 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "proxy", 2024-08-19T01:43:32.789 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-08-19T01:43:32.789 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.789 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-08-19T01:43:32.789 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-08-19T01:43:32.789 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.789 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.789 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:32.789 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-08-19T01:43:32.789 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.789 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.789 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.789 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.790 INFO:teuthology.orchestra.run.smithi083.stdout: "url": { 2024-08-19T01:43:32.790 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "url", 2024-08-19T01:43:32.790 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-08-19T01:43:32.790 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.790 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-08-19T01:43:32.790 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-08-19T01:43:32.790 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.790 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.790 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:32.790 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-08-19T01:43:32.790 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.790 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.790 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.791 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-08-19T01:43:32.791 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-08-19T01:43:32.791 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.791 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-08-19T01:43:32.791 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "test_orchestrator", 2024-08-19T01:43:32.791 INFO:teuthology.orchestra.run.smithi083.stdout: "can_run": true, 2024-08-19T01:43:32.791 INFO:teuthology.orchestra.run.smithi083.stdout: "error_string": "", 2024-08-19T01:43:32.791 INFO:teuthology.orchestra.run.smithi083.stdout: "module_options": { 2024-08-19T01:43:32.791 INFO:teuthology.orchestra.run.smithi083.stdout: "log_level": { 2024-08-19T01:43:32.791 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_level", 2024-08-19T01:43:32.791 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-08-19T01:43:32.791 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.791 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-08-19T01:43:32.792 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-08-19T01:43:32.792 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.792 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.792 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-08-19T01:43:32.792 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-08-19T01:43:32.792 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-08-19T01:43:32.792 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-08-19T01:43:32.792 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-08-19T01:43:32.792 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-08-19T01:43:32.792 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-08-19T01:43:32.792 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-08-19T01:43:32.792 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-08-19T01:43:32.792 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.793 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.793 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.793 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.793 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster": { 2024-08-19T01:43:32.793 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster", 2024-08-19T01:43:32.793 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-08-19T01:43:32.793 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.793 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-08-19T01:43:32.793 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-08-19T01:43:32.793 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.793 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.793 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:32.793 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-08-19T01:43:32.794 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.794 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.794 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.794 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.794 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster_level": { 2024-08-19T01:43:32.794 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster_level", 2024-08-19T01:43:32.794 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-08-19T01:43:32.794 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.794 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-08-19T01:43:32.794 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "info", 2024-08-19T01:43:32.794 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.794 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.794 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-08-19T01:43:32.795 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-08-19T01:43:32.795 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-08-19T01:43:32.795 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-08-19T01:43:32.795 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-08-19T01:43:32.795 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-08-19T01:43:32.795 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-08-19T01:43:32.795 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-08-19T01:43:32.795 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-08-19T01:43:32.795 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.795 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.795 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.795 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.795 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_file": { 2024-08-19T01:43:32.796 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_file", 2024-08-19T01:43:32.796 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-08-19T01:43:32.796 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.796 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-08-19T01:43:32.796 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-08-19T01:43:32.796 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.796 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.796 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:32.796 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-08-19T01:43:32.796 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.796 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.796 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.796 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-08-19T01:43:32.796 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-08-19T01:43:32.797 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.797 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-08-19T01:43:32.797 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "volumes", 2024-08-19T01:43:32.797 INFO:teuthology.orchestra.run.smithi083.stdout: "can_run": true, 2024-08-19T01:43:32.797 INFO:teuthology.orchestra.run.smithi083.stdout: "error_string": "", 2024-08-19T01:43:32.797 INFO:teuthology.orchestra.run.smithi083.stdout: "module_options": { 2024-08-19T01:43:32.797 INFO:teuthology.orchestra.run.smithi083.stdout: "log_level": { 2024-08-19T01:43:32.797 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_level", 2024-08-19T01:43:32.797 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-08-19T01:43:32.797 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.797 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-08-19T01:43:32.797 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-08-19T01:43:32.797 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.798 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.798 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-08-19T01:43:32.798 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-08-19T01:43:32.798 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-08-19T01:43:32.798 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-08-19T01:43:32.798 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-08-19T01:43:32.798 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-08-19T01:43:32.798 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-08-19T01:43:32.798 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-08-19T01:43:32.798 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-08-19T01:43:32.798 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.798 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.798 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.799 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.799 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster": { 2024-08-19T01:43:32.799 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster", 2024-08-19T01:43:32.799 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-08-19T01:43:32.799 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.799 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-08-19T01:43:32.799 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-08-19T01:43:32.799 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.799 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.799 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:32.799 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-08-19T01:43:32.799 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.799 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.799 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.800 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.800 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster_level": { 2024-08-19T01:43:32.800 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster_level", 2024-08-19T01:43:32.800 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-08-19T01:43:32.800 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.800 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-08-19T01:43:32.800 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "info", 2024-08-19T01:43:32.800 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.800 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.800 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-08-19T01:43:32.800 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-08-19T01:43:32.800 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-08-19T01:43:32.800 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-08-19T01:43:32.801 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-08-19T01:43:32.801 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-08-19T01:43:32.801 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-08-19T01:43:32.801 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-08-19T01:43:32.801 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-08-19T01:43:32.801 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.801 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.801 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.801 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.801 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_file": { 2024-08-19T01:43:32.801 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_file", 2024-08-19T01:43:32.801 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-08-19T01:43:32.801 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.802 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-08-19T01:43:32.802 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-08-19T01:43:32.802 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.802 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.802 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:32.802 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-08-19T01:43:32.802 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.802 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.802 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.802 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.802 INFO:teuthology.orchestra.run.smithi083.stdout: "max_concurrent_clones": { 2024-08-19T01:43:32.802 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "max_concurrent_clones", 2024-08-19T01:43:32.802 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2024-08-19T01:43:32.802 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.803 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-08-19T01:43:32.803 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "4", 2024-08-19T01:43:32.803 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.803 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.803 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:32.803 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Number of asynchronous cloner threads", 2024-08-19T01:43:32.803 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.803 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.803 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.803 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.803 INFO:teuthology.orchestra.run.smithi083.stdout: "periodic_async_work": { 2024-08-19T01:43:32.803 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "periodic_async_work", 2024-08-19T01:43:32.803 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-08-19T01:43:32.804 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.804 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-08-19T01:43:32.804 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-08-19T01:43:32.804 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.804 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.804 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:32.804 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Periodically check for async work", 2024-08-19T01:43:32.804 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.804 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.804 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.804 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.804 INFO:teuthology.orchestra.run.smithi083.stdout: "snapshot_clone_delay": { 2024-08-19T01:43:32.804 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "snapshot_clone_delay", 2024-08-19T01:43:32.805 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2024-08-19T01:43:32.805 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.805 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-08-19T01:43:32.805 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "0", 2024-08-19T01:43:32.805 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.805 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.805 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:32.805 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-08-19T01:43:32.805 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.805 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.805 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.805 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.806 INFO:teuthology.orchestra.run.smithi083.stdout: "snapshot_clone_no_wait": { 2024-08-19T01:43:32.806 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "snapshot_clone_no_wait", 2024-08-19T01:43:32.806 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-08-19T01:43:32.806 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.806 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-08-19T01:43:32.806 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "True", 2024-08-19T01:43:32.806 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.806 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.806 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:32.806 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-08-19T01:43:32.806 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.806 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.806 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.807 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-08-19T01:43:32.807 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-08-19T01:43:32.807 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.807 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-08-19T01:43:32.807 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "zabbix", 2024-08-19T01:43:32.807 INFO:teuthology.orchestra.run.smithi083.stdout: "can_run": true, 2024-08-19T01:43:32.807 INFO:teuthology.orchestra.run.smithi083.stdout: "error_string": "", 2024-08-19T01:43:32.807 INFO:teuthology.orchestra.run.smithi083.stdout: "module_options": { 2024-08-19T01:43:32.807 INFO:teuthology.orchestra.run.smithi083.stdout: "discovery_interval": { 2024-08-19T01:43:32.807 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "discovery_interval", 2024-08-19T01:43:32.807 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "uint", 2024-08-19T01:43:32.807 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.807 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-08-19T01:43:32.808 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "100", 2024-08-19T01:43:32.808 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.808 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.808 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:32.808 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-08-19T01:43:32.808 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.808 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.808 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.808 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.808 INFO:teuthology.orchestra.run.smithi083.stdout: "identifier": { 2024-08-19T01:43:32.808 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "identifier", 2024-08-19T01:43:32.808 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-08-19T01:43:32.808 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.809 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-08-19T01:43:32.809 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-08-19T01:43:32.809 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.809 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.809 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:32.809 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-08-19T01:43:32.809 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.809 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.809 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.809 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.809 INFO:teuthology.orchestra.run.smithi083.stdout: "interval": { 2024-08-19T01:43:32.809 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "interval", 2024-08-19T01:43:32.810 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "secs", 2024-08-19T01:43:32.810 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.810 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-08-19T01:43:32.810 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "60", 2024-08-19T01:43:32.810 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.810 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.810 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:32.810 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-08-19T01:43:32.810 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.810 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.810 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.810 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.810 INFO:teuthology.orchestra.run.smithi083.stdout: "log_level": { 2024-08-19T01:43:32.811 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_level", 2024-08-19T01:43:32.811 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-08-19T01:43:32.811 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.811 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-08-19T01:43:32.811 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-08-19T01:43:32.811 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.811 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.811 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-08-19T01:43:32.811 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-08-19T01:43:32.811 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-08-19T01:43:32.811 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-08-19T01:43:32.811 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-08-19T01:43:32.811 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-08-19T01:43:32.811 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-08-19T01:43:32.812 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-08-19T01:43:32.812 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-08-19T01:43:32.812 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.812 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.812 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.812 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.812 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster": { 2024-08-19T01:43:32.812 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster", 2024-08-19T01:43:32.812 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-08-19T01:43:32.812 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.812 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-08-19T01:43:32.812 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-08-19T01:43:32.812 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.812 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.813 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:32.813 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-08-19T01:43:32.813 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.813 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.813 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.813 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.813 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster_level": { 2024-08-19T01:43:32.813 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster_level", 2024-08-19T01:43:32.813 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-08-19T01:43:32.813 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.813 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-08-19T01:43:32.813 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "info", 2024-08-19T01:43:32.813 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.814 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.814 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-08-19T01:43:32.814 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-08-19T01:43:32.814 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-08-19T01:43:32.814 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-08-19T01:43:32.814 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-08-19T01:43:32.814 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-08-19T01:43:32.814 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-08-19T01:43:32.814 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-08-19T01:43:32.814 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-08-19T01:43:32.814 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.814 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.814 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.815 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.815 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_file": { 2024-08-19T01:43:32.815 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_file", 2024-08-19T01:43:32.815 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-08-19T01:43:32.815 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.815 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-08-19T01:43:32.815 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-08-19T01:43:32.815 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.815 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.815 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:32.815 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-08-19T01:43:32.815 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.815 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.816 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.816 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.816 INFO:teuthology.orchestra.run.smithi083.stdout: "zabbix_host": { 2024-08-19T01:43:32.816 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "zabbix_host", 2024-08-19T01:43:32.816 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-08-19T01:43:32.816 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.816 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-08-19T01:43:32.816 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-08-19T01:43:32.816 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.816 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.816 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:32.816 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-08-19T01:43:32.816 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.817 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.817 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.817 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.817 INFO:teuthology.orchestra.run.smithi083.stdout: "zabbix_port": { 2024-08-19T01:43:32.817 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "zabbix_port", 2024-08-19T01:43:32.817 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2024-08-19T01:43:32.817 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.817 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-08-19T01:43:32.817 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "10051", 2024-08-19T01:43:32.817 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.817 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.817 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:32.817 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-08-19T01:43:32.817 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.818 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.818 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.818 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.818 INFO:teuthology.orchestra.run.smithi083.stdout: "zabbix_sender": { 2024-08-19T01:43:32.818 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "zabbix_sender", 2024-08-19T01:43:32.818 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-08-19T01:43:32.818 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.818 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-08-19T01:43:32.818 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-08-19T01:43:32.818 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.818 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.818 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:32.818 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-08-19T01:43:32.819 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.819 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.819 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.819 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-08-19T01:43:32.819 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-08-19T01:43:32.819 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-08-19T01:43:32.819 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-08-19T01:43:32.819 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-08-19T01:43:32.819 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-08-19T01:43:32.819 INFO:teuthology.orchestra.run.smithi083.stdout: "modules": [ 2024-08-19T01:43:32.819 INFO:teuthology.orchestra.run.smithi083.stdout: "cephadm", 2024-08-19T01:43:32.819 INFO:teuthology.orchestra.run.smithi083.stdout: "dashboard", 2024-08-19T01:43:32.819 INFO:teuthology.orchestra.run.smithi083.stdout: "iostat", 2024-08-19T01:43:32.820 INFO:teuthology.orchestra.run.smithi083.stdout: "nfs", 2024-08-19T01:43:32.820 INFO:teuthology.orchestra.run.smithi083.stdout: "restful" 2024-08-19T01:43:32.820 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-08-19T01:43:32.820 INFO:teuthology.orchestra.run.smithi083.stdout: "available_modules": [ 2024-08-19T01:43:32.820 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-08-19T01:43:32.820 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "alerts", 2024-08-19T01:43:32.820 INFO:teuthology.orchestra.run.smithi083.stdout: "can_run": true, 2024-08-19T01:43:32.820 INFO:teuthology.orchestra.run.smithi083.stdout: "error_string": "", 2024-08-19T01:43:32.820 INFO:teuthology.orchestra.run.smithi083.stdout: "module_options": { 2024-08-19T01:43:32.820 INFO:teuthology.orchestra.run.smithi083.stdout: "interval": { 2024-08-19T01:43:32.820 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "interval", 2024-08-19T01:43:32.820 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "secs", 2024-08-19T01:43:32.820 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.820 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-08-19T01:43:32.821 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "60", 2024-08-19T01:43:32.821 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.821 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.821 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:32.821 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "How frequently to reexamine health status", 2024-08-19T01:43:32.821 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.821 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.821 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.821 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.821 INFO:teuthology.orchestra.run.smithi083.stdout: "log_level": { 2024-08-19T01:43:32.821 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_level", 2024-08-19T01:43:32.821 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-08-19T01:43:32.821 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.822 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-08-19T01:43:32.822 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-08-19T01:43:32.822 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.822 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.822 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-08-19T01:43:32.822 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-08-19T01:43:32.822 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-08-19T01:43:32.822 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-08-19T01:43:32.822 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-08-19T01:43:32.822 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-08-19T01:43:32.822 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-08-19T01:43:32.822 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-08-19T01:43:32.822 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-08-19T01:43:32.823 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.823 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.823 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.823 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.823 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster": { 2024-08-19T01:43:32.823 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster", 2024-08-19T01:43:32.823 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-08-19T01:43:32.823 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.823 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-08-19T01:43:32.823 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-08-19T01:43:32.823 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.823 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.823 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:32.823 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-08-19T01:43:32.824 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.824 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.824 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.824 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.824 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster_level": { 2024-08-19T01:43:32.824 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster_level", 2024-08-19T01:43:32.824 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-08-19T01:43:32.824 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.824 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-08-19T01:43:32.824 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "info", 2024-08-19T01:43:32.824 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.824 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.824 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-08-19T01:43:32.824 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-08-19T01:43:32.825 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-08-19T01:43:32.825 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-08-19T01:43:32.825 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-08-19T01:43:32.825 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-08-19T01:43:32.825 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-08-19T01:43:32.825 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-08-19T01:43:32.825 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-08-19T01:43:32.825 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.825 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.825 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.825 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.825 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_file": { 2024-08-19T01:43:32.825 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_file", 2024-08-19T01:43:32.825 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-08-19T01:43:32.826 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.826 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-08-19T01:43:32.826 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-08-19T01:43:32.826 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.826 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.826 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:32.826 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-08-19T01:43:32.826 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.826 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.826 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.826 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.826 INFO:teuthology.orchestra.run.smithi083.stdout: "smtp_destination": { 2024-08-19T01:43:32.827 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "smtp_destination", 2024-08-19T01:43:32.827 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-08-19T01:43:32.827 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.827 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-08-19T01:43:32.827 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-08-19T01:43:32.827 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.827 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.827 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:32.827 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Email address to send alerts to", 2024-08-19T01:43:32.827 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.827 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.827 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.827 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.827 INFO:teuthology.orchestra.run.smithi083.stdout: "smtp_from_name": { 2024-08-19T01:43:32.828 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "smtp_from_name", 2024-08-19T01:43:32.828 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-08-19T01:43:32.828 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.828 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-08-19T01:43:32.828 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "Ceph", 2024-08-19T01:43:32.828 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.828 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.828 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:32.828 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Email From: name", 2024-08-19T01:43:32.828 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.828 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.828 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.828 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.829 INFO:teuthology.orchestra.run.smithi083.stdout: "smtp_host": { 2024-08-19T01:43:32.829 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "smtp_host", 2024-08-19T01:43:32.829 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-08-19T01:43:32.829 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.829 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-08-19T01:43:32.829 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-08-19T01:43:32.829 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.829 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.829 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:32.829 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "SMTP server", 2024-08-19T01:43:32.829 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.829 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.829 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.830 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.830 INFO:teuthology.orchestra.run.smithi083.stdout: "smtp_password": { 2024-08-19T01:43:32.830 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "smtp_password", 2024-08-19T01:43:32.830 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-08-19T01:43:32.830 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.830 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-08-19T01:43:32.830 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-08-19T01:43:32.830 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.830 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.830 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:32.830 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Password to authenticate with", 2024-08-19T01:43:32.830 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.830 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.831 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.831 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.831 INFO:teuthology.orchestra.run.smithi083.stdout: "smtp_port": { 2024-08-19T01:43:32.831 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "smtp_port", 2024-08-19T01:43:32.831 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2024-08-19T01:43:32.831 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.831 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-08-19T01:43:32.831 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "465", 2024-08-19T01:43:32.831 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.831 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.831 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:32.831 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "SMTP port", 2024-08-19T01:43:32.832 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.832 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.832 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.832 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.832 INFO:teuthology.orchestra.run.smithi083.stdout: "smtp_sender": { 2024-08-19T01:43:32.832 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "smtp_sender", 2024-08-19T01:43:32.832 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-08-19T01:43:32.832 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.832 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-08-19T01:43:32.832 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-08-19T01:43:32.832 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.832 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.832 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:32.832 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "SMTP envelope sender", 2024-08-19T01:43:32.832 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.833 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.833 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.833 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.833 INFO:teuthology.orchestra.run.smithi083.stdout: "smtp_ssl": { 2024-08-19T01:43:32.833 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "smtp_ssl", 2024-08-19T01:43:32.833 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-08-19T01:43:32.833 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.833 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-08-19T01:43:32.833 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "True", 2024-08-19T01:43:32.833 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.833 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.833 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:32.833 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Use SSL to connect to SMTP server", 2024-08-19T01:43:32.833 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.833 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.834 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.834 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.834 INFO:teuthology.orchestra.run.smithi083.stdout: "smtp_user": { 2024-08-19T01:43:32.834 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "smtp_user", 2024-08-19T01:43:32.834 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-08-19T01:43:32.834 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.834 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-08-19T01:43:32.834 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-08-19T01:43:32.834 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.834 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.834 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:32.834 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "User to authenticate as", 2024-08-19T01:43:32.834 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.835 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.835 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.835 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-08-19T01:43:32.835 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-08-19T01:43:32.835 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.835 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-08-19T01:43:32.835 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "balancer", 2024-08-19T01:43:32.835 INFO:teuthology.orchestra.run.smithi083.stdout: "can_run": true, 2024-08-19T01:43:32.835 INFO:teuthology.orchestra.run.smithi083.stdout: "error_string": "", 2024-08-19T01:43:32.835 INFO:teuthology.orchestra.run.smithi083.stdout: "module_options": { 2024-08-19T01:43:32.835 INFO:teuthology.orchestra.run.smithi083.stdout: "active": { 2024-08-19T01:43:32.835 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "active", 2024-08-19T01:43:32.835 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-08-19T01:43:32.835 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.836 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-08-19T01:43:32.836 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "True", 2024-08-19T01:43:32.836 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.836 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.836 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:32.836 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "automatically balance PGs across cluster", 2024-08-19T01:43:32.836 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.836 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.836 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.836 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.836 INFO:teuthology.orchestra.run.smithi083.stdout: "begin_time": { 2024-08-19T01:43:32.836 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "begin_time", 2024-08-19T01:43:32.836 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-08-19T01:43:32.837 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.837 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-08-19T01:43:32.837 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "0000", 2024-08-19T01:43:32.837 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.837 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.837 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:32.837 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "beginning time of day to automatically balance", 2024-08-19T01:43:32.837 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-08-19T01:43:32.837 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.837 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.837 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.837 INFO:teuthology.orchestra.run.smithi083.stdout: "begin_weekday": { 2024-08-19T01:43:32.837 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "begin_weekday", 2024-08-19T01:43:32.837 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "uint", 2024-08-19T01:43:32.838 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.838 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-08-19T01:43:32.838 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "0", 2024-08-19T01:43:32.838 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "0", 2024-08-19T01:43:32.838 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "6", 2024-08-19T01:43:32.838 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:32.838 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-08-19T01:43:32.838 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-08-19T01:43:32.838 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.838 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.838 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.838 INFO:teuthology.orchestra.run.smithi083.stdout: "crush_compat_max_iterations": { 2024-08-19T01:43:32.838 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "crush_compat_max_iterations", 2024-08-19T01:43:32.839 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "uint", 2024-08-19T01:43:32.839 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.839 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-08-19T01:43:32.839 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "25", 2024-08-19T01:43:32.839 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "1", 2024-08-19T01:43:32.839 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "250", 2024-08-19T01:43:32.839 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:32.839 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-08-19T01:43:32.839 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.839 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.839 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.839 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.839 INFO:teuthology.orchestra.run.smithi083.stdout: "crush_compat_metrics": { 2024-08-19T01:43:32.839 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "crush_compat_metrics", 2024-08-19T01:43:32.840 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-08-19T01:43:32.840 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.840 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-08-19T01:43:32.840 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "pgs,objects,bytes", 2024-08-19T01:43:32.840 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.840 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.840 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:32.840 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-08-19T01:43:32.840 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-08-19T01:43:32.840 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.840 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.840 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.840 INFO:teuthology.orchestra.run.smithi083.stdout: "crush_compat_step": { 2024-08-19T01:43:32.841 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "crush_compat_step", 2024-08-19T01:43:32.841 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "float", 2024-08-19T01:43:32.841 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.841 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-08-19T01:43:32.841 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "0.5", 2024-08-19T01:43:32.841 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "0.001", 2024-08-19T01:43:32.841 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "0.999", 2024-08-19T01:43:32.841 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:32.841 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "aggressiveness of optimization", 2024-08-19T01:43:32.841 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-08-19T01:43:32.841 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.841 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.841 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.841 INFO:teuthology.orchestra.run.smithi083.stdout: "end_time": { 2024-08-19T01:43:32.842 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "end_time", 2024-08-19T01:43:32.842 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-08-19T01:43:32.842 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.842 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-08-19T01:43:32.842 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "2359", 2024-08-19T01:43:32.842 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.842 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.842 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:32.842 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "ending time of day to automatically balance", 2024-08-19T01:43:32.842 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-08-19T01:43:32.842 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.842 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.842 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.843 INFO:teuthology.orchestra.run.smithi083.stdout: "end_weekday": { 2024-08-19T01:43:32.843 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "end_weekday", 2024-08-19T01:43:32.843 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "uint", 2024-08-19T01:43:32.843 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.843 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-08-19T01:43:32.843 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "0", 2024-08-19T01:43:32.843 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "0", 2024-08-19T01:43:32.843 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "6", 2024-08-19T01:43:32.843 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:32.843 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-08-19T01:43:32.843 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-08-19T01:43:32.843 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.843 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.844 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.844 INFO:teuthology.orchestra.run.smithi083.stdout: "log_level": { 2024-08-19T01:43:32.844 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_level", 2024-08-19T01:43:32.844 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-08-19T01:43:32.844 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.844 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-08-19T01:43:32.844 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-08-19T01:43:32.844 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.844 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.844 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-08-19T01:43:32.844 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-08-19T01:43:32.844 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-08-19T01:43:32.845 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-08-19T01:43:32.845 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-08-19T01:43:32.845 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-08-19T01:43:32.845 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-08-19T01:43:32.845 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-08-19T01:43:32.845 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-08-19T01:43:32.845 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.845 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.845 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.845 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.845 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster": { 2024-08-19T01:43:32.845 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster", 2024-08-19T01:43:32.845 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-08-19T01:43:32.846 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.846 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-08-19T01:43:32.846 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-08-19T01:43:32.846 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.846 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.846 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:32.846 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-08-19T01:43:32.846 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.846 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.846 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.846 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.846 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster_level": { 2024-08-19T01:43:32.847 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster_level", 2024-08-19T01:43:32.847 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-08-19T01:43:32.847 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.847 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-08-19T01:43:32.847 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "info", 2024-08-19T01:43:32.847 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.847 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.847 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-08-19T01:43:32.847 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-08-19T01:43:32.847 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-08-19T01:43:32.847 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-08-19T01:43:32.847 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-08-19T01:43:32.847 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-08-19T01:43:32.847 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-08-19T01:43:32.848 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-08-19T01:43:32.848 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-08-19T01:43:32.848 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.848 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.848 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.848 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.848 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_file": { 2024-08-19T01:43:32.848 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_file", 2024-08-19T01:43:32.848 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-08-19T01:43:32.848 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.848 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-08-19T01:43:32.848 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-08-19T01:43:32.848 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.849 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.849 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:32.849 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-08-19T01:43:32.849 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.849 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.849 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.849 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.849 INFO:teuthology.orchestra.run.smithi083.stdout: "min_score": { 2024-08-19T01:43:32.849 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "min_score", 2024-08-19T01:43:32.849 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "float", 2024-08-19T01:43:32.849 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.849 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-08-19T01:43:32.849 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "0", 2024-08-19T01:43:32.850 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.850 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.850 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:32.850 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-08-19T01:43:32.850 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.850 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.850 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.850 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.850 INFO:teuthology.orchestra.run.smithi083.stdout: "mode": { 2024-08-19T01:43:32.850 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "mode", 2024-08-19T01:43:32.850 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-08-19T01:43:32.850 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.850 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-08-19T01:43:32.850 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "upmap", 2024-08-19T01:43:32.851 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.851 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.851 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-08-19T01:43:32.851 INFO:teuthology.orchestra.run.smithi083.stdout: "crush-compat", 2024-08-19T01:43:32.851 INFO:teuthology.orchestra.run.smithi083.stdout: "none", 2024-08-19T01:43:32.851 INFO:teuthology.orchestra.run.smithi083.stdout: "read", 2024-08-19T01:43:32.851 INFO:teuthology.orchestra.run.smithi083.stdout: "upmap", 2024-08-19T01:43:32.851 INFO:teuthology.orchestra.run.smithi083.stdout: "upmap-read" 2024-08-19T01:43:32.851 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-08-19T01:43:32.851 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Balancer mode", 2024-08-19T01:43:32.851 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.851 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.851 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.852 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.852 INFO:teuthology.orchestra.run.smithi083.stdout: "pool_ids": { 2024-08-19T01:43:32.852 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "pool_ids", 2024-08-19T01:43:32.852 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-08-19T01:43:32.852 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.852 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-08-19T01:43:32.852 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-08-19T01:43:32.852 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.852 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.852 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:32.852 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-08-19T01:43:32.852 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.852 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.853 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.853 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.853 INFO:teuthology.orchestra.run.smithi083.stdout: "sleep_interval": { 2024-08-19T01:43:32.853 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "sleep_interval", 2024-08-19T01:43:32.853 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "secs", 2024-08-19T01:43:32.853 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.853 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-08-19T01:43:32.853 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "60", 2024-08-19T01:43:32.853 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.853 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.853 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:32.853 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-08-19T01:43:32.853 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.853 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.854 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.854 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.854 INFO:teuthology.orchestra.run.smithi083.stdout: "upmap_max_deviation": { 2024-08-19T01:43:32.854 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "upmap_max_deviation", 2024-08-19T01:43:32.854 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2024-08-19T01:43:32.854 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.854 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-08-19T01:43:32.854 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "5", 2024-08-19T01:43:32.854 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "1", 2024-08-19T01:43:32.854 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.854 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:32.854 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "deviation below which no optimization is attempted", 2024-08-19T01:43:32.854 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-08-19T01:43:32.855 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.855 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.855 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.855 INFO:teuthology.orchestra.run.smithi083.stdout: "upmap_max_optimizations": { 2024-08-19T01:43:32.855 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "upmap_max_optimizations", 2024-08-19T01:43:32.855 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "uint", 2024-08-19T01:43:32.855 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.855 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-08-19T01:43:32.855 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "10", 2024-08-19T01:43:32.855 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.855 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.855 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:32.855 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-08-19T01:43:32.856 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.856 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.856 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.856 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-08-19T01:43:32.856 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-08-19T01:43:32.856 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.856 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-08-19T01:43:32.856 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "cephadm", 2024-08-19T01:43:32.856 INFO:teuthology.orchestra.run.smithi083.stdout: "can_run": true, 2024-08-19T01:43:32.856 INFO:teuthology.orchestra.run.smithi083.stdout: "error_string": "", 2024-08-19T01:43:32.856 INFO:teuthology.orchestra.run.smithi083.stdout: "module_options": { 2024-08-19T01:43:32.856 INFO:teuthology.orchestra.run.smithi083.stdout: "agent_down_multiplier": { 2024-08-19T01:43:32.856 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "agent_down_multiplier", 2024-08-19T01:43:32.856 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "float", 2024-08-19T01:43:32.857 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.857 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-08-19T01:43:32.857 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "3.0", 2024-08-19T01:43:32.857 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.857 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.857 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:32.857 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-08-19T01:43:32.857 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.857 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.857 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.857 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.857 INFO:teuthology.orchestra.run.smithi083.stdout: "agent_refresh_rate": { 2024-08-19T01:43:32.857 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "agent_refresh_rate", 2024-08-19T01:43:32.858 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "secs", 2024-08-19T01:43:32.858 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.858 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-08-19T01:43:32.858 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "20", 2024-08-19T01:43:32.858 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.858 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.858 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:32.858 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-08-19T01:43:32.858 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.858 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.858 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.858 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.858 INFO:teuthology.orchestra.run.smithi083.stdout: "agent_starting_port": { 2024-08-19T01:43:32.859 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "agent_starting_port", 2024-08-19T01:43:32.859 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2024-08-19T01:43:32.859 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.859 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-08-19T01:43:32.859 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "4721", 2024-08-19T01:43:32.859 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.859 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.859 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:32.859 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-08-19T01:43:32.859 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.859 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.859 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.859 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.859 INFO:teuthology.orchestra.run.smithi083.stdout: "allow_ptrace": { 2024-08-19T01:43:32.860 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "allow_ptrace", 2024-08-19T01:43:32.860 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-08-19T01:43:32.860 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.860 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-08-19T01:43:32.860 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-08-19T01:43:32.860 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.860 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.860 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:32.860 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-08-19T01:43:32.860 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-08-19T01:43:32.860 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.860 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.860 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.860 INFO:teuthology.orchestra.run.smithi083.stdout: "autotune_interval": { 2024-08-19T01:43:32.861 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "autotune_interval", 2024-08-19T01:43:32.861 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "secs", 2024-08-19T01:43:32.861 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.861 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-08-19T01:43:32.861 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "600", 2024-08-19T01:43:32.861 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.861 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.861 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:32.861 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "how frequently to autotune daemon memory", 2024-08-19T01:43:32.861 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.861 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.861 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.861 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.862 INFO:teuthology.orchestra.run.smithi083.stdout: "autotune_memory_target_ratio": { 2024-08-19T01:43:32.862 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "autotune_memory_target_ratio", 2024-08-19T01:43:32.862 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "float", 2024-08-19T01:43:32.862 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.862 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-08-19T01:43:32.862 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "0.7", 2024-08-19T01:43:32.862 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.862 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.862 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:32.862 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-08-19T01:43:32.862 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.862 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.862 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.863 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.863 INFO:teuthology.orchestra.run.smithi083.stdout: "cephadm_log_destination": { 2024-08-19T01:43:32.863 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "cephadm_log_destination", 2024-08-19T01:43:32.863 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-08-19T01:43:32.863 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.863 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-08-19T01:43:32.863 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-08-19T01:43:32.863 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.863 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.863 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-08-19T01:43:32.863 INFO:teuthology.orchestra.run.smithi083.stdout: "file", 2024-08-19T01:43:32.863 INFO:teuthology.orchestra.run.smithi083.stdout: "file,syslog", 2024-08-19T01:43:32.863 INFO:teuthology.orchestra.run.smithi083.stdout: "syslog" 2024-08-19T01:43:32.864 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-08-19T01:43:32.864 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-08-19T01:43:32.864 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.864 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.864 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.864 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.864 INFO:teuthology.orchestra.run.smithi083.stdout: "cgroups_split": { 2024-08-19T01:43:32.864 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "cgroups_split", 2024-08-19T01:43:32.864 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-08-19T01:43:32.864 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.864 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-08-19T01:43:32.864 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "True", 2024-08-19T01:43:32.864 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.865 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.865 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:32.865 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-08-19T01:43:32.865 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.865 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.865 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.865 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.865 INFO:teuthology.orchestra.run.smithi083.stdout: "config_checks_enabled": { 2024-08-19T01:43:32.865 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "config_checks_enabled", 2024-08-19T01:43:32.865 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-08-19T01:43:32.865 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.865 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-08-19T01:43:32.865 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-08-19T01:43:32.865 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.866 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.866 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:32.866 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-08-19T01:43:32.866 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.866 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.866 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.866 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.866 INFO:teuthology.orchestra.run.smithi083.stdout: "config_dashboard": { 2024-08-19T01:43:32.866 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "config_dashboard", 2024-08-19T01:43:32.866 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-08-19T01:43:32.866 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.867 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-08-19T01:43:32.867 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "True", 2024-08-19T01:43:32.867 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.867 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.867 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:32.867 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-08-19T01:43:32.867 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.867 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.867 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.867 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.867 INFO:teuthology.orchestra.run.smithi083.stdout: "container_image_alertmanager": { 2024-08-19T01:43:32.867 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "container_image_alertmanager", 2024-08-19T01:43:32.867 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-08-19T01:43:32.867 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.867 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-08-19T01:43:32.868 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-08-19T01:43:32.868 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.868 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.868 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:32.868 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Prometheus container image", 2024-08-19T01:43:32.868 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.868 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.868 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.868 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.868 INFO:teuthology.orchestra.run.smithi083.stdout: "container_image_base": { 2024-08-19T01:43:32.868 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "container_image_base", 2024-08-19T01:43:32.868 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-08-19T01:43:32.868 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.868 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-08-19T01:43:32.869 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "quay.io/ceph/ceph", 2024-08-19T01:43:32.869 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.869 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.869 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:32.869 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Container image name, without the tag", 2024-08-19T01:43:32.869 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.869 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.869 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.869 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.869 INFO:teuthology.orchestra.run.smithi083.stdout: "container_image_elasticsearch": { 2024-08-19T01:43:32.869 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "container_image_elasticsearch", 2024-08-19T01:43:32.869 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-08-19T01:43:32.869 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.869 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-08-19T01:43:32.870 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-08-19T01:43:32.870 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.870 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.870 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:32.870 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "elasticsearch container image", 2024-08-19T01:43:32.870 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.870 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.870 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.870 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.870 INFO:teuthology.orchestra.run.smithi083.stdout: "container_image_grafana": { 2024-08-19T01:43:32.870 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "container_image_grafana", 2024-08-19T01:43:32.870 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-08-19T01:43:32.870 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.870 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-08-19T01:43:32.871 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "quay.io/ceph/grafana:9.4.12", 2024-08-19T01:43:32.871 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.871 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.871 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:32.871 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Prometheus container image", 2024-08-19T01:43:32.871 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.871 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.871 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.871 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.871 INFO:teuthology.orchestra.run.smithi083.stdout: "container_image_haproxy": { 2024-08-19T01:43:32.871 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "container_image_haproxy", 2024-08-19T01:43:32.871 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-08-19T01:43:32.871 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.871 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-08-19T01:43:32.872 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-08-19T01:43:32.872 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.872 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.872 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:32.873 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "HAproxy container image", 2024-08-19T01:43:32.873 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.873 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.873 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.873 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.873 INFO:teuthology.orchestra.run.smithi083.stdout: "container_image_jaeger_agent": { 2024-08-19T01:43:32.873 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "container_image_jaeger_agent", 2024-08-19T01:43:32.874 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-08-19T01:43:32.874 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.874 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-08-19T01:43:32.874 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-08-19T01:43:32.874 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.874 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.874 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:32.874 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Jaeger agent container image", 2024-08-19T01:43:32.874 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.874 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.874 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.874 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.874 INFO:teuthology.orchestra.run.smithi083.stdout: "container_image_jaeger_collector": { 2024-08-19T01:43:32.875 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "container_image_jaeger_collector", 2024-08-19T01:43:32.875 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-08-19T01:43:32.875 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.875 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-08-19T01:43:32.875 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-08-19T01:43:32.875 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.875 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.875 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:32.875 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Jaeger collector container image", 2024-08-19T01:43:32.875 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.875 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.875 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.875 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.875 INFO:teuthology.orchestra.run.smithi083.stdout: "container_image_jaeger_query": { 2024-08-19T01:43:32.875 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "container_image_jaeger_query", 2024-08-19T01:43:32.876 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-08-19T01:43:32.876 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.876 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-08-19T01:43:32.876 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-08-19T01:43:32.876 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.876 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.876 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:32.876 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Jaeger query container image", 2024-08-19T01:43:32.876 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.876 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.876 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.876 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.876 INFO:teuthology.orchestra.run.smithi083.stdout: "container_image_keepalived": { 2024-08-19T01:43:32.876 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "container_image_keepalived", 2024-08-19T01:43:32.876 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-08-19T01:43:32.876 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.877 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-08-19T01:43:32.877 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-08-19T01:43:32.877 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.877 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.877 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:32.877 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Keepalived container image", 2024-08-19T01:43:32.877 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.877 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.877 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.877 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.877 INFO:teuthology.orchestra.run.smithi083.stdout: "container_image_loki": { 2024-08-19T01:43:32.877 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "container_image_loki", 2024-08-19T01:43:32.877 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-08-19T01:43:32.877 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.878 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-08-19T01:43:32.878 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "docker.io/grafana/loki:3.0.0", 2024-08-19T01:43:32.878 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.878 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.878 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:32.878 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Loki container image", 2024-08-19T01:43:32.878 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.878 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.878 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.878 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.878 INFO:teuthology.orchestra.run.smithi083.stdout: "container_image_node_exporter": { 2024-08-19T01:43:32.878 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "container_image_node_exporter", 2024-08-19T01:43:32.878 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-08-19T01:43:32.878 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.879 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-08-19T01:43:32.879 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-08-19T01:43:32.879 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.879 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.879 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:32.879 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Prometheus container image", 2024-08-19T01:43:32.879 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.879 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.879 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.879 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.879 INFO:teuthology.orchestra.run.smithi083.stdout: "container_image_nvmeof": { 2024-08-19T01:43:32.879 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "container_image_nvmeof", 2024-08-19T01:43:32.879 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-08-19T01:43:32.879 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.880 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-08-19T01:43:32.880 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "quay.io/ceph/nvmeof:1.2.5", 2024-08-19T01:43:32.880 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.880 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.880 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:32.880 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Nvme-of container image", 2024-08-19T01:43:32.880 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.880 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.880 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.880 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.880 INFO:teuthology.orchestra.run.smithi083.stdout: "container_image_prometheus": { 2024-08-19T01:43:32.880 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "container_image_prometheus", 2024-08-19T01:43:32.881 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-08-19T01:43:32.881 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.881 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-08-19T01:43:32.881 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-08-19T01:43:32.881 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.881 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.881 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:32.881 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Prometheus container image", 2024-08-19T01:43:32.881 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.881 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.881 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.881 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.881 INFO:teuthology.orchestra.run.smithi083.stdout: "container_image_promtail": { 2024-08-19T01:43:32.881 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "container_image_promtail", 2024-08-19T01:43:32.881 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-08-19T01:43:32.882 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.882 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-08-19T01:43:32.882 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "docker.io/grafana/promtail:3.0.0", 2024-08-19T01:43:32.882 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.882 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.882 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:32.882 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Promtail container image", 2024-08-19T01:43:32.882 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.882 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.882 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.882 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.882 INFO:teuthology.orchestra.run.smithi083.stdout: "container_image_samba": { 2024-08-19T01:43:32.882 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "container_image_samba", 2024-08-19T01:43:32.882 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-08-19T01:43:32.883 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.883 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-08-19T01:43:32.883 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "quay.io/samba.org/samba-server:devbuilds-centos-amd64", 2024-08-19T01:43:32.883 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.883 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.883 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:32.883 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Samba/SMB container image", 2024-08-19T01:43:32.883 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.883 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.883 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.883 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.883 INFO:teuthology.orchestra.run.smithi083.stdout: "container_image_snmp_gateway": { 2024-08-19T01:43:32.883 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "container_image_snmp_gateway", 2024-08-19T01:43:32.884 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-08-19T01:43:32.884 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.884 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-08-19T01:43:32.884 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-08-19T01:43:32.884 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.884 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.884 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:32.884 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "SNMP Gateway container image", 2024-08-19T01:43:32.884 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.884 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.884 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.884 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.884 INFO:teuthology.orchestra.run.smithi083.stdout: "container_init": { 2024-08-19T01:43:32.885 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "container_init", 2024-08-19T01:43:32.885 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-08-19T01:43:32.885 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.885 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-08-19T01:43:32.885 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "True", 2024-08-19T01:43:32.885 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.885 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.885 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:32.885 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Run podman/docker with `--init`", 2024-08-19T01:43:32.885 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.885 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.885 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.885 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.886 INFO:teuthology.orchestra.run.smithi083.stdout: "daemon_cache_timeout": { 2024-08-19T01:43:32.886 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "daemon_cache_timeout", 2024-08-19T01:43:32.886 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "secs", 2024-08-19T01:43:32.886 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.886 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-08-19T01:43:32.886 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "600", 2024-08-19T01:43:32.886 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.886 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.886 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:32.886 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-08-19T01:43:32.886 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.886 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.886 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.886 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.887 INFO:teuthology.orchestra.run.smithi083.stdout: "default_cephadm_command_timeout": { 2024-08-19T01:43:32.887 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "default_cephadm_command_timeout", 2024-08-19T01:43:32.887 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2024-08-19T01:43:32.887 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.887 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-08-19T01:43:32.887 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "900", 2024-08-19T01:43:32.887 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.887 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.887 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:32.887 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-08-19T01:43:32.887 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.887 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.887 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.888 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.888 INFO:teuthology.orchestra.run.smithi083.stdout: "default_registry": { 2024-08-19T01:43:32.888 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "default_registry", 2024-08-19T01:43:32.888 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-08-19T01:43:32.888 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.888 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-08-19T01:43:32.888 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "docker.io", 2024-08-19T01:43:32.888 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.888 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.888 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:32.888 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-08-19T01:43:32.888 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.888 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.889 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.889 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.889 INFO:teuthology.orchestra.run.smithi083.stdout: "device_cache_timeout": { 2024-08-19T01:43:32.889 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "device_cache_timeout", 2024-08-19T01:43:32.889 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "secs", 2024-08-19T01:43:32.889 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.889 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-08-19T01:43:32.889 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "1800", 2024-08-19T01:43:32.889 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.889 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.889 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:32.889 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "seconds to cache device inventory", 2024-08-19T01:43:32.889 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.890 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.890 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.890 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.890 INFO:teuthology.orchestra.run.smithi083.stdout: "device_enhanced_scan": { 2024-08-19T01:43:32.890 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "device_enhanced_scan", 2024-08-19T01:43:32.890 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-08-19T01:43:32.890 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.890 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-08-19T01:43:32.890 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-08-19T01:43:32.890 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.890 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.890 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:32.890 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Use libstoragemgmt during device scans", 2024-08-19T01:43:32.890 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.891 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.891 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.891 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.891 INFO:teuthology.orchestra.run.smithi083.stdout: "facts_cache_timeout": { 2024-08-19T01:43:32.891 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "facts_cache_timeout", 2024-08-19T01:43:32.891 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "secs", 2024-08-19T01:43:32.891 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.891 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-08-19T01:43:32.891 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "60", 2024-08-19T01:43:32.891 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.891 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.891 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:32.891 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "seconds to cache host facts data", 2024-08-19T01:43:32.892 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.892 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.892 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.892 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.892 INFO:teuthology.orchestra.run.smithi083.stdout: "grafana_dashboards_path": { 2024-08-19T01:43:32.892 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "grafana_dashboards_path", 2024-08-19T01:43:32.892 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-08-19T01:43:32.892 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.892 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-08-19T01:43:32.892 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "/etc/grafana/dashboards/ceph-dashboard/", 2024-08-19T01:43:32.892 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.892 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.893 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:32.893 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "location of dashboards to include in grafana deployments", 2024-08-19T01:43:32.893 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.893 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.893 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.893 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.893 INFO:teuthology.orchestra.run.smithi083.stdout: "host_check_interval": { 2024-08-19T01:43:32.893 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "host_check_interval", 2024-08-19T01:43:32.893 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "secs", 2024-08-19T01:43:32.893 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.893 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-08-19T01:43:32.893 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "600", 2024-08-19T01:43:32.893 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.894 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.894 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:32.894 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "how frequently to perform a host check", 2024-08-19T01:43:32.894 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.894 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.894 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.894 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.894 INFO:teuthology.orchestra.run.smithi083.stdout: "hw_monitoring": { 2024-08-19T01:43:32.894 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "hw_monitoring", 2024-08-19T01:43:32.894 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-08-19T01:43:32.894 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.894 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-08-19T01:43:32.894 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-08-19T01:43:32.895 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.895 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.895 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:32.895 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-08-19T01:43:32.895 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.895 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.895 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.895 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.895 INFO:teuthology.orchestra.run.smithi083.stdout: "inventory_list_all": { 2024-08-19T01:43:32.895 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "inventory_list_all", 2024-08-19T01:43:32.895 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-08-19T01:43:32.895 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.895 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-08-19T01:43:32.895 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-08-19T01:43:32.896 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.896 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.896 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:32.896 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-08-19T01:43:32.896 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.896 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.896 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.896 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.896 INFO:teuthology.orchestra.run.smithi083.stdout: "log_level": { 2024-08-19T01:43:32.896 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_level", 2024-08-19T01:43:32.896 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-08-19T01:43:32.896 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.896 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-08-19T01:43:32.896 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-08-19T01:43:32.896 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.897 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.897 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-08-19T01:43:32.897 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-08-19T01:43:32.897 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-08-19T01:43:32.897 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-08-19T01:43:32.897 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-08-19T01:43:32.897 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-08-19T01:43:32.897 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-08-19T01:43:32.897 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-08-19T01:43:32.897 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-08-19T01:43:32.897 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.897 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.897 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.897 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.897 INFO:teuthology.orchestra.run.smithi083.stdout: "log_refresh_metadata": { 2024-08-19T01:43:32.898 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_refresh_metadata", 2024-08-19T01:43:32.898 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-08-19T01:43:32.898 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.898 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-08-19T01:43:32.898 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-08-19T01:43:32.898 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.898 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.898 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:32.898 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-08-19T01:43:32.898 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.898 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.898 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.898 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.899 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster": { 2024-08-19T01:43:32.899 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster", 2024-08-19T01:43:32.899 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-08-19T01:43:32.899 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.899 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-08-19T01:43:32.899 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "True", 2024-08-19T01:43:32.899 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.899 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.899 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:32.899 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-08-19T01:43:32.899 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.899 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.899 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.899 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.899 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster_level": { 2024-08-19T01:43:32.900 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster_level", 2024-08-19T01:43:32.900 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-08-19T01:43:32.900 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.900 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-08-19T01:43:32.900 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "info", 2024-08-19T01:43:32.900 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.900 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.900 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-08-19T01:43:32.900 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-08-19T01:43:32.900 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-08-19T01:43:32.900 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-08-19T01:43:32.900 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-08-19T01:43:32.900 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-08-19T01:43:32.901 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-08-19T01:43:32.901 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-08-19T01:43:32.901 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-08-19T01:43:32.901 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.901 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.901 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.901 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.901 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_file": { 2024-08-19T01:43:32.901 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_file", 2024-08-19T01:43:32.901 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-08-19T01:43:32.901 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.901 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-08-19T01:43:32.901 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-08-19T01:43:32.902 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.902 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.902 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:32.902 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-08-19T01:43:32.902 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.902 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.902 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.902 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.902 INFO:teuthology.orchestra.run.smithi083.stdout: "manage_etc_ceph_ceph_conf": { 2024-08-19T01:43:32.902 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-08-19T01:43:32.902 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-08-19T01:43:32.902 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.902 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-08-19T01:43:32.903 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-08-19T01:43:32.903 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.903 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.903 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:32.903 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-08-19T01:43:32.903 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.903 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.903 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.903 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.903 INFO:teuthology.orchestra.run.smithi083.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-08-19T01:43:32.903 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-08-19T01:43:32.903 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-08-19T01:43:32.903 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.904 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-08-19T01:43:32.904 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "*", 2024-08-19T01:43:32.904 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.904 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.904 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:32.904 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-08-19T01:43:32.904 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.904 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.904 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.904 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.904 INFO:teuthology.orchestra.run.smithi083.stdout: "max_count_per_host": { 2024-08-19T01:43:32.904 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "max_count_per_host", 2024-08-19T01:43:32.904 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2024-08-19T01:43:32.905 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.905 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-08-19T01:43:32.905 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "10", 2024-08-19T01:43:32.905 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.905 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.905 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:32.905 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "max number of daemons per service per host", 2024-08-19T01:43:32.905 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.905 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.905 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.905 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.905 INFO:teuthology.orchestra.run.smithi083.stdout: "max_osd_draining_count": { 2024-08-19T01:43:32.905 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "max_osd_draining_count", 2024-08-19T01:43:32.906 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2024-08-19T01:43:32.906 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.906 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-08-19T01:43:32.906 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "10", 2024-08-19T01:43:32.906 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.906 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.906 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:32.906 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-08-19T01:43:32.906 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.906 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.906 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.906 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.907 INFO:teuthology.orchestra.run.smithi083.stdout: "migration_current": { 2024-08-19T01:43:32.907 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "migration_current", 2024-08-19T01:43:32.907 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2024-08-19T01:43:32.907 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.907 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-08-19T01:43:32.907 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-08-19T01:43:32.907 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.907 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.907 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:32.907 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "internal - do not modify", 2024-08-19T01:43:32.907 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.907 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.907 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.908 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.908 INFO:teuthology.orchestra.run.smithi083.stdout: "mode": { 2024-08-19T01:43:32.908 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "mode", 2024-08-19T01:43:32.908 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-08-19T01:43:32.908 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.908 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-08-19T01:43:32.908 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "root", 2024-08-19T01:43:32.908 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.908 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.908 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-08-19T01:43:32.908 INFO:teuthology.orchestra.run.smithi083.stdout: "cephadm-package", 2024-08-19T01:43:32.908 INFO:teuthology.orchestra.run.smithi083.stdout: "root" 2024-08-19T01:43:32.908 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-08-19T01:43:32.909 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "mode for remote execution of cephadm", 2024-08-19T01:43:32.909 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.909 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.909 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.909 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.909 INFO:teuthology.orchestra.run.smithi083.stdout: "oob_default_addr": { 2024-08-19T01:43:32.909 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "oob_default_addr", 2024-08-19T01:43:32.909 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-08-19T01:43:32.909 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.909 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-08-19T01:43:32.909 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "169.254.1.1", 2024-08-19T01:43:32.909 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.909 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.909 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:32.910 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Default address for RedFish API (oob management).", 2024-08-19T01:43:32.910 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.910 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.910 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.910 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.910 INFO:teuthology.orchestra.run.smithi083.stdout: "prometheus_alerts_path": { 2024-08-19T01:43:32.910 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "prometheus_alerts_path", 2024-08-19T01:43:32.910 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-08-19T01:43:32.910 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.910 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-08-19T01:43:32.910 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-08-19T01:43:32.910 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.910 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.910 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:32.911 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-08-19T01:43:32.911 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.911 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.911 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.911 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.911 INFO:teuthology.orchestra.run.smithi083.stdout: "registry_insecure": { 2024-08-19T01:43:32.911 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "registry_insecure", 2024-08-19T01:43:32.911 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-08-19T01:43:32.911 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.911 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-08-19T01:43:32.911 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-08-19T01:43:32.911 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.911 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.912 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:32.912 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-08-19T01:43:32.912 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.912 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.912 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.912 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.912 INFO:teuthology.orchestra.run.smithi083.stdout: "registry_password": { 2024-08-19T01:43:32.912 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "registry_password", 2024-08-19T01:43:32.912 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-08-19T01:43:32.912 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.912 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-08-19T01:43:32.912 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-08-19T01:43:32.912 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.912 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.913 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:32.913 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-08-19T01:43:32.913 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.913 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.913 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.913 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.913 INFO:teuthology.orchestra.run.smithi083.stdout: "registry_url": { 2024-08-19T01:43:32.913 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "registry_url", 2024-08-19T01:43:32.913 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-08-19T01:43:32.913 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.913 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-08-19T01:43:32.913 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-08-19T01:43:32.913 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.914 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.914 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:32.914 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-08-19T01:43:32.914 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.914 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.914 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.914 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.914 INFO:teuthology.orchestra.run.smithi083.stdout: "registry_username": { 2024-08-19T01:43:32.914 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "registry_username", 2024-08-19T01:43:32.914 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-08-19T01:43:32.914 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.914 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-08-19T01:43:32.914 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-08-19T01:43:32.914 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.914 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.915 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:32.915 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-08-19T01:43:32.915 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.915 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.915 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.915 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.915 INFO:teuthology.orchestra.run.smithi083.stdout: "secure_monitoring_stack": { 2024-08-19T01:43:32.915 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "secure_monitoring_stack", 2024-08-19T01:43:32.915 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-08-19T01:43:32.915 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.915 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-08-19T01:43:32.915 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-08-19T01:43:32.915 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.915 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.915 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:32.916 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-08-19T01:43:32.916 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.916 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.916 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.916 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.916 INFO:teuthology.orchestra.run.smithi083.stdout: "service_discovery_port": { 2024-08-19T01:43:32.916 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "service_discovery_port", 2024-08-19T01:43:32.916 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2024-08-19T01:43:32.916 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.916 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-08-19T01:43:32.916 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "8765", 2024-08-19T01:43:32.916 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.916 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.916 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:32.916 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "cephadm service discovery port", 2024-08-19T01:43:32.917 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.917 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.917 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.917 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.917 INFO:teuthology.orchestra.run.smithi083.stdout: "ssh_config_file": { 2024-08-19T01:43:32.917 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "ssh_config_file", 2024-08-19T01:43:32.917 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-08-19T01:43:32.917 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.917 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-08-19T01:43:32.917 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-08-19T01:43:32.917 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.917 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.917 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:32.917 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-08-19T01:43:32.917 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.918 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.918 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.918 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.918 INFO:teuthology.orchestra.run.smithi083.stdout: "use_agent": { 2024-08-19T01:43:32.918 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "use_agent", 2024-08-19T01:43:32.918 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-08-19T01:43:32.918 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.918 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-08-19T01:43:32.918 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-08-19T01:43:32.918 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.918 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.918 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:32.918 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-08-19T01:43:32.918 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.919 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.919 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.919 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.919 INFO:teuthology.orchestra.run.smithi083.stdout: "use_repo_digest": { 2024-08-19T01:43:32.919 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "use_repo_digest", 2024-08-19T01:43:32.919 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-08-19T01:43:32.919 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.919 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-08-19T01:43:32.919 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "True", 2024-08-19T01:43:32.919 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.919 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.919 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:32.919 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-08-19T01:43:32.919 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.919 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.920 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.920 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.920 INFO:teuthology.orchestra.run.smithi083.stdout: "warn_on_failed_host_check": { 2024-08-19T01:43:32.920 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "warn_on_failed_host_check", 2024-08-19T01:43:32.920 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-08-19T01:43:32.920 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.920 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-08-19T01:43:32.920 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "True", 2024-08-19T01:43:32.920 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.920 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.920 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:32.920 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "raise a health warning if the host check fails", 2024-08-19T01:43:32.920 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.920 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.921 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.921 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.921 INFO:teuthology.orchestra.run.smithi083.stdout: "warn_on_stray_daemons": { 2024-08-19T01:43:32.921 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "warn_on_stray_daemons", 2024-08-19T01:43:32.921 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-08-19T01:43:32.921 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.921 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-08-19T01:43:32.921 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "True", 2024-08-19T01:43:32.921 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.921 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.921 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:32.921 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-08-19T01:43:32.921 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.922 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.922 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.922 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.922 INFO:teuthology.orchestra.run.smithi083.stdout: "warn_on_stray_hosts": { 2024-08-19T01:43:32.922 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "warn_on_stray_hosts", 2024-08-19T01:43:32.922 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-08-19T01:43:32.922 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.922 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-08-19T01:43:32.922 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "True", 2024-08-19T01:43:32.922 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.922 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.922 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:32.922 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-08-19T01:43:32.923 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.923 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.923 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.923 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-08-19T01:43:32.923 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-08-19T01:43:32.923 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.923 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-08-19T01:43:32.923 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "crash", 2024-08-19T01:43:32.923 INFO:teuthology.orchestra.run.smithi083.stdout: "can_run": true, 2024-08-19T01:43:32.923 INFO:teuthology.orchestra.run.smithi083.stdout: "error_string": "", 2024-08-19T01:43:32.923 INFO:teuthology.orchestra.run.smithi083.stdout: "module_options": { 2024-08-19T01:43:32.923 INFO:teuthology.orchestra.run.smithi083.stdout: "log_level": { 2024-08-19T01:43:32.923 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_level", 2024-08-19T01:43:32.923 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-08-19T01:43:32.924 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.924 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-08-19T01:43:32.924 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-08-19T01:43:32.924 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.924 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.924 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-08-19T01:43:32.924 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-08-19T01:43:32.924 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-08-19T01:43:32.924 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-08-19T01:43:32.924 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-08-19T01:43:32.924 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-08-19T01:43:32.924 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-08-19T01:43:32.924 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-08-19T01:43:32.925 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-08-19T01:43:32.925 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.925 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.925 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.925 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.925 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster": { 2024-08-19T01:43:32.925 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster", 2024-08-19T01:43:32.925 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-08-19T01:43:32.925 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.925 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-08-19T01:43:32.925 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-08-19T01:43:32.925 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.925 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.925 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:32.925 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-08-19T01:43:32.926 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.926 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.926 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.926 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.926 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster_level": { 2024-08-19T01:43:32.926 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster_level", 2024-08-19T01:43:32.926 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-08-19T01:43:32.926 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.926 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-08-19T01:43:32.926 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "info", 2024-08-19T01:43:32.926 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.926 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.926 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-08-19T01:43:32.926 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-08-19T01:43:32.927 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-08-19T01:43:32.927 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-08-19T01:43:32.927 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-08-19T01:43:32.927 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-08-19T01:43:32.927 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-08-19T01:43:32.927 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-08-19T01:43:32.927 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-08-19T01:43:32.927 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.927 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.927 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.927 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.927 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_file": { 2024-08-19T01:43:32.927 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_file", 2024-08-19T01:43:32.927 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-08-19T01:43:32.928 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.928 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-08-19T01:43:32.928 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-08-19T01:43:32.928 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.928 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.928 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:32.928 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-08-19T01:43:32.928 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.928 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.928 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.928 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.928 INFO:teuthology.orchestra.run.smithi083.stdout: "retain_interval": { 2024-08-19T01:43:32.928 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "retain_interval", 2024-08-19T01:43:32.928 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "secs", 2024-08-19T01:43:32.928 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.929 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-08-19T01:43:32.929 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "31536000", 2024-08-19T01:43:32.929 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.929 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.929 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:32.929 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "how long to retain crashes before pruning them", 2024-08-19T01:43:32.929 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.929 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.929 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.929 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.929 INFO:teuthology.orchestra.run.smithi083.stdout: "warn_recent_interval": { 2024-08-19T01:43:32.929 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "warn_recent_interval", 2024-08-19T01:43:32.929 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "secs", 2024-08-19T01:43:32.929 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.929 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-08-19T01:43:32.929 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "1209600", 2024-08-19T01:43:32.930 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.930 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.930 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:32.930 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "time interval in which to warn about recent crashes", 2024-08-19T01:43:32.930 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.930 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.930 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.930 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-08-19T01:43:32.930 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-08-19T01:43:32.930 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.930 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-08-19T01:43:32.930 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "dashboard", 2024-08-19T01:43:32.930 INFO:teuthology.orchestra.run.smithi083.stdout: "can_run": true, 2024-08-19T01:43:32.930 INFO:teuthology.orchestra.run.smithi083.stdout: "error_string": "", 2024-08-19T01:43:32.930 INFO:teuthology.orchestra.run.smithi083.stdout: "module_options": { 2024-08-19T01:43:32.931 INFO:teuthology.orchestra.run.smithi083.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-08-19T01:43:32.931 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-08-19T01:43:32.931 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2024-08-19T01:43:32.931 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.931 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-08-19T01:43:32.931 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "10", 2024-08-19T01:43:32.931 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.931 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.931 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:32.931 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-08-19T01:43:32.931 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.931 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.931 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.931 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.932 INFO:teuthology.orchestra.run.smithi083.stdout: "ALERTMANAGER_API_HOST": { 2024-08-19T01:43:32.932 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "ALERTMANAGER_API_HOST", 2024-08-19T01:43:32.932 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-08-19T01:43:32.932 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.932 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-08-19T01:43:32.932 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-08-19T01:43:32.932 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.932 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.932 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:32.932 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-08-19T01:43:32.932 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.932 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.932 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.933 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.933 INFO:teuthology.orchestra.run.smithi083.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-08-19T01:43:32.933 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-08-19T01:43:32.933 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-08-19T01:43:32.933 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.933 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-08-19T01:43:32.933 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "True", 2024-08-19T01:43:32.933 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.933 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.933 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:32.933 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-08-19T01:43:32.933 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.933 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.933 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.933 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.933 INFO:teuthology.orchestra.run.smithi083.stdout: "AUDIT_API_ENABLED": { 2024-08-19T01:43:32.934 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "AUDIT_API_ENABLED", 2024-08-19T01:43:32.934 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-08-19T01:43:32.934 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.934 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-08-19T01:43:32.934 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-08-19T01:43:32.934 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.934 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.934 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:32.934 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-08-19T01:43:32.934 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.934 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.934 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.935 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.935 INFO:teuthology.orchestra.run.smithi083.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-08-19T01:43:32.935 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-08-19T01:43:32.935 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-08-19T01:43:32.935 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.935 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-08-19T01:43:32.935 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "True", 2024-08-19T01:43:32.935 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.935 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.935 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:32.935 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-08-19T01:43:32.935 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.935 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.935 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.935 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.936 INFO:teuthology.orchestra.run.smithi083.stdout: "ENABLE_BROWSABLE_API": { 2024-08-19T01:43:32.936 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "ENABLE_BROWSABLE_API", 2024-08-19T01:43:32.936 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-08-19T01:43:32.936 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.936 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-08-19T01:43:32.936 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "True", 2024-08-19T01:43:32.936 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.936 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.936 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:32.936 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-08-19T01:43:32.936 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.936 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.936 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.937 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.937 INFO:teuthology.orchestra.run.smithi083.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-08-19T01:43:32.937 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-08-19T01:43:32.937 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-08-19T01:43:32.937 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.937 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-08-19T01:43:32.937 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "True", 2024-08-19T01:43:32.937 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.937 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.937 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:32.937 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-08-19T01:43:32.937 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.937 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.937 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.938 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.938 INFO:teuthology.orchestra.run.smithi083.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-08-19T01:43:32.938 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-08-19T01:43:32.938 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-08-19T01:43:32.938 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.938 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-08-19T01:43:32.938 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "True", 2024-08-19T01:43:32.938 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.938 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.938 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:32.938 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-08-19T01:43:32.938 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.938 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.939 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.939 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.939 INFO:teuthology.orchestra.run.smithi083.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-08-19T01:43:32.939 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-08-19T01:43:32.939 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-08-19T01:43:32.939 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.939 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-08-19T01:43:32.939 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "True", 2024-08-19T01:43:32.939 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.939 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.939 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:32.939 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-08-19T01:43:32.939 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.940 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.940 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.940 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.940 INFO:teuthology.orchestra.run.smithi083.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-08-19T01:43:32.940 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-08-19T01:43:32.940 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-08-19T01:43:32.940 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.940 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-08-19T01:43:32.940 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "True", 2024-08-19T01:43:32.940 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.940 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.940 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:32.940 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-08-19T01:43:32.941 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.941 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.941 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.941 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.941 INFO:teuthology.orchestra.run.smithi083.stdout: "FEATURE_TOGGLE_NFS": { 2024-08-19T01:43:32.941 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-08-19T01:43:32.941 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-08-19T01:43:32.941 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.941 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-08-19T01:43:32.941 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "True", 2024-08-19T01:43:32.941 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.941 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.941 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:32.942 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-08-19T01:43:32.942 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.942 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.942 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.942 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.942 INFO:teuthology.orchestra.run.smithi083.stdout: "FEATURE_TOGGLE_RBD": { 2024-08-19T01:43:32.942 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-08-19T01:43:32.942 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-08-19T01:43:32.942 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.942 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-08-19T01:43:32.942 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "True", 2024-08-19T01:43:32.942 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.942 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.943 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:32.943 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-08-19T01:43:32.943 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.943 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.943 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.943 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.943 INFO:teuthology.orchestra.run.smithi083.stdout: "FEATURE_TOGGLE_RGW": { 2024-08-19T01:43:32.943 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-08-19T01:43:32.943 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-08-19T01:43:32.943 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.943 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-08-19T01:43:32.943 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "True", 2024-08-19T01:43:32.943 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.944 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.944 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:32.944 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-08-19T01:43:32.944 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.944 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.944 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.944 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.944 INFO:teuthology.orchestra.run.smithi083.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-08-19T01:43:32.944 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-08-19T01:43:32.944 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-08-19T01:43:32.944 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.944 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-08-19T01:43:32.944 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-08-19T01:43:32.945 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.945 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.945 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:32.945 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-08-19T01:43:32.945 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.945 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.945 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.945 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.945 INFO:teuthology.orchestra.run.smithi083.stdout: "GRAFANA_API_PASSWORD": { 2024-08-19T01:43:32.945 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "GRAFANA_API_PASSWORD", 2024-08-19T01:43:32.945 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-08-19T01:43:32.945 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.945 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-08-19T01:43:32.945 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "admin", 2024-08-19T01:43:32.946 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.946 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.946 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:32.946 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-08-19T01:43:32.946 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.946 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.946 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.946 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.946 INFO:teuthology.orchestra.run.smithi083.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-08-19T01:43:32.946 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-08-19T01:43:32.946 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-08-19T01:43:32.946 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.947 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-08-19T01:43:32.947 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "True", 2024-08-19T01:43:32.947 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.947 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.947 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:32.947 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-08-19T01:43:32.947 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.947 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.947 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.947 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.947 INFO:teuthology.orchestra.run.smithi083.stdout: "GRAFANA_API_URL": { 2024-08-19T01:43:32.947 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "GRAFANA_API_URL", 2024-08-19T01:43:32.947 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-08-19T01:43:32.947 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.948 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-08-19T01:43:32.948 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-08-19T01:43:32.948 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.948 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.948 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:32.948 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-08-19T01:43:32.948 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.948 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.948 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.948 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.948 INFO:teuthology.orchestra.run.smithi083.stdout: "GRAFANA_API_USERNAME": { 2024-08-19T01:43:32.948 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "GRAFANA_API_USERNAME", 2024-08-19T01:43:32.948 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-08-19T01:43:32.949 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.949 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-08-19T01:43:32.949 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "admin", 2024-08-19T01:43:32.949 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.949 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.949 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:32.949 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-08-19T01:43:32.949 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.949 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.949 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.949 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.949 INFO:teuthology.orchestra.run.smithi083.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-08-19T01:43:32.949 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-08-19T01:43:32.949 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-08-19T01:43:32.950 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.950 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-08-19T01:43:32.950 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-08-19T01:43:32.950 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.950 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.950 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:32.950 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-08-19T01:43:32.950 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.950 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.950 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.950 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.950 INFO:teuthology.orchestra.run.smithi083.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-08-19T01:43:32.951 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-08-19T01:43:32.951 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-08-19T01:43:32.951 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.951 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-08-19T01:43:32.951 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-08-19T01:43:32.951 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.951 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.951 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:32.951 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-08-19T01:43:32.951 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.951 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.951 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.951 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.951 INFO:teuthology.orchestra.run.smithi083.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-08-19T01:43:32.952 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-08-19T01:43:32.952 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-08-19T01:43:32.952 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.952 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-08-19T01:43:32.952 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "True", 2024-08-19T01:43:32.952 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.952 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.952 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:32.952 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-08-19T01:43:32.952 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.952 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.952 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.952 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.952 INFO:teuthology.orchestra.run.smithi083.stdout: "ISSUE_TRACKER_API_KEY": { 2024-08-19T01:43:32.952 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-08-19T01:43:32.952 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-08-19T01:43:32.953 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.953 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-08-19T01:43:32.953 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-08-19T01:43:32.953 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.953 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.953 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:32.953 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-08-19T01:43:32.953 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.953 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.953 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.953 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.953 INFO:teuthology.orchestra.run.smithi083.stdout: "PROMETHEUS_API_HOST": { 2024-08-19T01:43:32.953 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "PROMETHEUS_API_HOST", 2024-08-19T01:43:32.954 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-08-19T01:43:32.954 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.954 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-08-19T01:43:32.954 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-08-19T01:43:32.954 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.954 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.954 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:32.954 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-08-19T01:43:32.954 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.954 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.954 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.954 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.954 INFO:teuthology.orchestra.run.smithi083.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-08-19T01:43:32.955 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-08-19T01:43:32.955 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-08-19T01:43:32.955 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.955 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-08-19T01:43:32.955 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "True", 2024-08-19T01:43:32.955 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.955 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.955 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:32.955 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-08-19T01:43:32.955 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.955 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.955 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.955 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.955 INFO:teuthology.orchestra.run.smithi083.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-08-19T01:43:32.955 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-08-19T01:43:32.956 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-08-19T01:43:32.956 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.956 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-08-19T01:43:32.956 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-08-19T01:43:32.956 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.956 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.956 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:32.956 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-08-19T01:43:32.956 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.956 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.956 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.956 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.956 INFO:teuthology.orchestra.run.smithi083.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-08-19T01:43:32.956 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-08-19T01:43:32.956 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-08-19T01:43:32.957 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.957 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-08-19T01:43:32.957 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-08-19T01:43:32.957 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.957 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.957 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:32.957 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-08-19T01:43:32.957 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.957 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.957 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.957 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.957 INFO:teuthology.orchestra.run.smithi083.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-08-19T01:43:32.957 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-08-19T01:43:32.957 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-08-19T01:43:32.957 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.958 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-08-19T01:43:32.958 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "True", 2024-08-19T01:43:32.958 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.958 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.958 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:32.958 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-08-19T01:43:32.958 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.958 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.958 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.958 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.958 INFO:teuthology.orchestra.run.smithi083.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-08-19T01:43:32.958 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-08-19T01:43:32.958 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-08-19T01:43:32.958 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.959 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-08-19T01:43:32.959 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "True", 2024-08-19T01:43:32.959 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.959 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.959 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:32.959 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-08-19T01:43:32.959 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.959 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.959 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.959 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.959 INFO:teuthology.orchestra.run.smithi083.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-08-19T01:43:32.959 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-08-19T01:43:32.959 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-08-19T01:43:32.959 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.959 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-08-19T01:43:32.960 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-08-19T01:43:32.960 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.960 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.960 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:32.960 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-08-19T01:43:32.960 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.960 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.960 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.960 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.960 INFO:teuthology.orchestra.run.smithi083.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-08-19T01:43:32.960 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-08-19T01:43:32.960 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-08-19T01:43:32.960 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.960 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-08-19T01:43:32.960 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-08-19T01:43:32.960 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.961 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.961 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:32.961 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-08-19T01:43:32.961 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.961 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.961 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.961 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.961 INFO:teuthology.orchestra.run.smithi083.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-08-19T01:43:32.961 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-08-19T01:43:32.961 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-08-19T01:43:32.961 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.961 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-08-19T01:43:32.961 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-08-19T01:43:32.962 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.962 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.962 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:32.962 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-08-19T01:43:32.962 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.962 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.962 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.962 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.962 INFO:teuthology.orchestra.run.smithi083.stdout: "PWD_POLICY_ENABLED": { 2024-08-19T01:43:32.962 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "PWD_POLICY_ENABLED", 2024-08-19T01:43:32.962 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-08-19T01:43:32.962 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.962 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-08-19T01:43:32.962 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "True", 2024-08-19T01:43:32.963 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.963 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.963 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:32.963 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-08-19T01:43:32.963 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.963 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.963 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.963 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.963 INFO:teuthology.orchestra.run.smithi083.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-08-19T01:43:32.963 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-08-19T01:43:32.963 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-08-19T01:43:32.963 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.963 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-08-19T01:43:32.963 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-08-19T01:43:32.963 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.963 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.964 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:32.964 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-08-19T01:43:32.964 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.964 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.964 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.964 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.964 INFO:teuthology.orchestra.run.smithi083.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-08-19T01:43:32.964 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-08-19T01:43:32.964 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2024-08-19T01:43:32.964 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.964 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-08-19T01:43:32.964 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "10", 2024-08-19T01:43:32.964 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.964 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.965 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:32.965 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-08-19T01:43:32.965 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.965 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.965 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.965 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.965 INFO:teuthology.orchestra.run.smithi083.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-08-19T01:43:32.965 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-08-19T01:43:32.965 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2024-08-19T01:43:32.965 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.965 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-08-19T01:43:32.965 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "8", 2024-08-19T01:43:32.965 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.966 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.966 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:32.966 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-08-19T01:43:32.966 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.966 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.966 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.966 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.966 INFO:teuthology.orchestra.run.smithi083.stdout: "REST_REQUESTS_TIMEOUT": { 2024-08-19T01:43:32.966 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-08-19T01:43:32.966 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2024-08-19T01:43:32.966 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.966 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-08-19T01:43:32.966 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "45", 2024-08-19T01:43:32.966 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.966 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.967 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:32.967 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-08-19T01:43:32.967 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.967 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.967 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.967 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.967 INFO:teuthology.orchestra.run.smithi083.stdout: "RGW_API_ACCESS_KEY": { 2024-08-19T01:43:32.967 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "RGW_API_ACCESS_KEY", 2024-08-19T01:43:32.967 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-08-19T01:43:32.967 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.967 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-08-19T01:43:32.967 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-08-19T01:43:32.967 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.967 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.968 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:32.968 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-08-19T01:43:32.968 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.968 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.968 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.968 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.968 INFO:teuthology.orchestra.run.smithi083.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-08-19T01:43:32.968 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-08-19T01:43:32.968 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-08-19T01:43:32.968 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.968 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-08-19T01:43:32.968 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "admin", 2024-08-19T01:43:32.968 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.968 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.968 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:32.969 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-08-19T01:43:32.969 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.969 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.969 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.969 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.969 INFO:teuthology.orchestra.run.smithi083.stdout: "RGW_API_SECRET_KEY": { 2024-08-19T01:43:32.969 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "RGW_API_SECRET_KEY", 2024-08-19T01:43:32.969 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-08-19T01:43:32.969 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.969 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-08-19T01:43:32.969 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-08-19T01:43:32.969 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.969 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.969 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:32.970 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-08-19T01:43:32.970 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.970 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.970 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.970 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.970 INFO:teuthology.orchestra.run.smithi083.stdout: "RGW_API_SSL_VERIFY": { 2024-08-19T01:43:32.970 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "RGW_API_SSL_VERIFY", 2024-08-19T01:43:32.970 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-08-19T01:43:32.970 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.970 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-08-19T01:43:32.970 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "True", 2024-08-19T01:43:32.970 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.970 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.970 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:32.971 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-08-19T01:43:32.971 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.971 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.971 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.971 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.971 INFO:teuthology.orchestra.run.smithi083.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-08-19T01:43:32.971 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-08-19T01:43:32.971 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2024-08-19T01:43:32.971 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.971 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-08-19T01:43:32.971 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "0", 2024-08-19T01:43:32.971 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.971 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.972 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:32.972 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-08-19T01:43:32.972 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.972 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.972 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.972 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.972 INFO:teuthology.orchestra.run.smithi083.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-08-19T01:43:32.972 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-08-19T01:43:32.972 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2024-08-19T01:43:32.972 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.972 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-08-19T01:43:32.972 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "10", 2024-08-19T01:43:32.972 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.973 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.973 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:32.973 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-08-19T01:43:32.973 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.973 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.973 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.973 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.973 INFO:teuthology.orchestra.run.smithi083.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-08-19T01:43:32.973 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-08-19T01:43:32.973 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2024-08-19T01:43:32.973 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.973 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-08-19T01:43:32.973 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "5", 2024-08-19T01:43:32.974 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.974 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.974 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:32.974 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-08-19T01:43:32.974 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.974 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.974 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.974 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.974 INFO:teuthology.orchestra.run.smithi083.stdout: "cross_origin_url": { 2024-08-19T01:43:32.974 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "cross_origin_url", 2024-08-19T01:43:32.974 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-08-19T01:43:32.974 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.974 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-08-19T01:43:32.975 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-08-19T01:43:32.975 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.975 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.975 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:32.975 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-08-19T01:43:32.975 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.975 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.975 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.975 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.975 INFO:teuthology.orchestra.run.smithi083.stdout: "crt_file": { 2024-08-19T01:43:32.975 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "crt_file", 2024-08-19T01:43:32.975 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-08-19T01:43:32.975 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.975 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-08-19T01:43:32.976 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-08-19T01:43:32.976 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.976 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.976 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:32.976 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-08-19T01:43:32.976 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.976 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.976 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.976 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.976 INFO:teuthology.orchestra.run.smithi083.stdout: "debug": { 2024-08-19T01:43:32.976 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "debug", 2024-08-19T01:43:32.976 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-08-19T01:43:32.976 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.976 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-08-19T01:43:32.976 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-08-19T01:43:32.977 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.977 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.977 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:32.977 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Enable/disable debug options", 2024-08-19T01:43:32.977 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.977 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.977 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.977 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.977 INFO:teuthology.orchestra.run.smithi083.stdout: "jwt_token_ttl": { 2024-08-19T01:43:32.977 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "jwt_token_ttl", 2024-08-19T01:43:32.977 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2024-08-19T01:43:32.977 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.977 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-08-19T01:43:32.977 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "28800", 2024-08-19T01:43:32.978 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.978 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.978 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:32.978 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-08-19T01:43:32.978 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.978 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.978 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.978 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.978 INFO:teuthology.orchestra.run.smithi083.stdout: "key_file": { 2024-08-19T01:43:32.978 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "key_file", 2024-08-19T01:43:32.978 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-08-19T01:43:32.978 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.978 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-08-19T01:43:32.978 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-08-19T01:43:32.979 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.979 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.979 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:32.979 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-08-19T01:43:32.979 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.979 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.979 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.979 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.979 INFO:teuthology.orchestra.run.smithi083.stdout: "log_level": { 2024-08-19T01:43:32.979 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_level", 2024-08-19T01:43:32.979 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-08-19T01:43:32.979 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.979 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-08-19T01:43:32.979 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-08-19T01:43:32.979 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.980 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.980 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-08-19T01:43:32.980 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-08-19T01:43:32.980 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-08-19T01:43:32.980 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-08-19T01:43:32.980 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-08-19T01:43:32.980 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-08-19T01:43:32.980 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-08-19T01:43:32.980 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-08-19T01:43:32.980 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-08-19T01:43:32.980 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.980 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.980 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.980 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.981 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster": { 2024-08-19T01:43:32.981 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster", 2024-08-19T01:43:32.981 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-08-19T01:43:32.981 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.981 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-08-19T01:43:32.981 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-08-19T01:43:32.981 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.981 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.981 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:32.981 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-08-19T01:43:32.981 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.981 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.981 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.981 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.981 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster_level": { 2024-08-19T01:43:32.982 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster_level", 2024-08-19T01:43:32.982 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-08-19T01:43:32.982 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.982 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-08-19T01:43:32.982 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "info", 2024-08-19T01:43:32.982 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.982 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.982 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-08-19T01:43:32.982 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-08-19T01:43:32.982 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-08-19T01:43:32.982 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-08-19T01:43:32.982 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-08-19T01:43:32.983 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-08-19T01:43:32.983 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-08-19T01:43:32.983 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-08-19T01:43:32.983 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-08-19T01:43:32.983 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.983 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.983 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.983 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.983 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_file": { 2024-08-19T01:43:32.983 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_file", 2024-08-19T01:43:32.983 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-08-19T01:43:32.983 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.983 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-08-19T01:43:32.984 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-08-19T01:43:32.984 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.984 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.984 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:32.984 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-08-19T01:43:32.984 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.984 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.984 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.984 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.984 INFO:teuthology.orchestra.run.smithi083.stdout: "motd": { 2024-08-19T01:43:32.984 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "motd", 2024-08-19T01:43:32.984 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-08-19T01:43:32.984 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.985 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-08-19T01:43:32.985 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-08-19T01:43:32.985 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.985 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.985 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:32.985 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "The message of the day", 2024-08-19T01:43:32.985 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.985 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.985 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.985 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.985 INFO:teuthology.orchestra.run.smithi083.stdout: "redirect_resolve_ip_addr": { 2024-08-19T01:43:32.985 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "redirect_resolve_ip_addr", 2024-08-19T01:43:32.985 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-08-19T01:43:32.985 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.986 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-08-19T01:43:32.986 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-08-19T01:43:32.986 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.986 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.986 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:32.986 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-08-19T01:43:32.986 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.986 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.986 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.986 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.986 INFO:teuthology.orchestra.run.smithi083.stdout: "server_addr": { 2024-08-19T01:43:32.986 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "server_addr", 2024-08-19T01:43:32.986 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-08-19T01:43:32.987 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.987 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-08-19T01:43:32.987 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "::", 2024-08-19T01:43:32.987 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.987 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.987 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:32.987 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-08-19T01:43:32.987 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.987 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.987 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.987 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.987 INFO:teuthology.orchestra.run.smithi083.stdout: "server_port": { 2024-08-19T01:43:32.987 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "server_port", 2024-08-19T01:43:32.987 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2024-08-19T01:43:32.987 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.987 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-08-19T01:43:32.988 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "8080", 2024-08-19T01:43:32.988 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.988 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.988 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:32.988 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-08-19T01:43:32.988 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.988 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.988 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.988 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.988 INFO:teuthology.orchestra.run.smithi083.stdout: "ssl": { 2024-08-19T01:43:32.988 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "ssl", 2024-08-19T01:43:32.988 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-08-19T01:43:32.988 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.988 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-08-19T01:43:32.988 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "True", 2024-08-19T01:43:32.989 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.989 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.989 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:32.989 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-08-19T01:43:32.989 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.989 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.989 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.989 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.989 INFO:teuthology.orchestra.run.smithi083.stdout: "ssl_server_port": { 2024-08-19T01:43:32.989 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "ssl_server_port", 2024-08-19T01:43:32.989 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2024-08-19T01:43:32.989 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.989 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-08-19T01:43:32.989 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "8443", 2024-08-19T01:43:32.989 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.990 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.990 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:32.990 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-08-19T01:43:32.990 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.990 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.990 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.990 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.990 INFO:teuthology.orchestra.run.smithi083.stdout: "standby_behaviour": { 2024-08-19T01:43:32.990 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "standby_behaviour", 2024-08-19T01:43:32.990 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-08-19T01:43:32.990 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.990 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-08-19T01:43:32.991 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "redirect", 2024-08-19T01:43:32.991 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.991 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.991 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-08-19T01:43:32.991 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-08-19T01:43:32.991 INFO:teuthology.orchestra.run.smithi083.stdout: "redirect" 2024-08-19T01:43:32.991 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-08-19T01:43:32.991 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-08-19T01:43:32.991 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.991 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.991 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.991 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.991 INFO:teuthology.orchestra.run.smithi083.stdout: "standby_error_status_code": { 2024-08-19T01:43:32.992 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "standby_error_status_code", 2024-08-19T01:43:32.992 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2024-08-19T01:43:32.992 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.992 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-08-19T01:43:32.992 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "500", 2024-08-19T01:43:32.992 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "400", 2024-08-19T01:43:32.992 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "599", 2024-08-19T01:43:32.992 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:32.992 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-08-19T01:43:32.992 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.992 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.992 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.992 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.993 INFO:teuthology.orchestra.run.smithi083.stdout: "url_prefix": { 2024-08-19T01:43:32.993 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "url_prefix", 2024-08-19T01:43:32.993 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-08-19T01:43:32.993 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.993 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-08-19T01:43:32.993 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-08-19T01:43:32.993 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.993 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.993 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:32.993 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-08-19T01:43:32.993 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.993 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.993 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.993 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-08-19T01:43:32.993 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-08-19T01:43:32.994 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.994 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-08-19T01:43:32.994 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "devicehealth", 2024-08-19T01:43:32.994 INFO:teuthology.orchestra.run.smithi083.stdout: "can_run": true, 2024-08-19T01:43:32.994 INFO:teuthology.orchestra.run.smithi083.stdout: "error_string": "", 2024-08-19T01:43:32.994 INFO:teuthology.orchestra.run.smithi083.stdout: "module_options": { 2024-08-19T01:43:32.994 INFO:teuthology.orchestra.run.smithi083.stdout: "enable_monitoring": { 2024-08-19T01:43:32.994 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "enable_monitoring", 2024-08-19T01:43:32.994 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-08-19T01:43:32.994 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.994 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-08-19T01:43:32.994 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "True", 2024-08-19T01:43:32.994 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.994 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.995 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:32.995 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "monitor device health metrics", 2024-08-19T01:43:32.995 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.995 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.995 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.995 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.995 INFO:teuthology.orchestra.run.smithi083.stdout: "log_level": { 2024-08-19T01:43:32.995 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_level", 2024-08-19T01:43:32.995 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-08-19T01:43:32.995 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.995 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-08-19T01:43:32.995 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-08-19T01:43:32.995 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.995 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.995 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-08-19T01:43:32.995 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-08-19T01:43:32.996 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-08-19T01:43:32.996 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-08-19T01:43:32.996 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-08-19T01:43:32.996 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-08-19T01:43:32.996 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-08-19T01:43:32.996 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-08-19T01:43:32.996 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-08-19T01:43:32.996 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.996 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.996 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.996 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.996 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster": { 2024-08-19T01:43:32.996 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster", 2024-08-19T01:43:32.996 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-08-19T01:43:32.996 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.997 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-08-19T01:43:32.997 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-08-19T01:43:32.997 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.997 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.997 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:32.997 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-08-19T01:43:32.997 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.997 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.997 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.997 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.997 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster_level": { 2024-08-19T01:43:32.997 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster_level", 2024-08-19T01:43:32.997 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-08-19T01:43:32.997 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.997 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-08-19T01:43:32.998 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "info", 2024-08-19T01:43:32.998 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.998 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.998 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-08-19T01:43:32.998 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-08-19T01:43:32.998 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-08-19T01:43:32.998 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-08-19T01:43:32.998 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-08-19T01:43:32.998 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-08-19T01:43:32.998 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-08-19T01:43:32.998 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-08-19T01:43:32.998 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-08-19T01:43:32.998 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.998 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.999 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.999 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:32.999 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_file": { 2024-08-19T01:43:32.999 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_file", 2024-08-19T01:43:32.999 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-08-19T01:43:32.999 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:32.999 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-08-19T01:43:32.999 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-08-19T01:43:32.999 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:32.999 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:32.999 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:32.999 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-08-19T01:43:32.999 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:32.999 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:32.999 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:32.999 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:33.000 INFO:teuthology.orchestra.run.smithi083.stdout: "mark_out_threshold": { 2024-08-19T01:43:33.000 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "mark_out_threshold", 2024-08-19T01:43:33.000 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "secs", 2024-08-19T01:43:33.000 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:33.000 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-08-19T01:43:33.000 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "2419200", 2024-08-19T01:43:33.000 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:33.000 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:33.000 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:33.000 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-08-19T01:43:33.000 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:33.000 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:33.000 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:33.000 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:33.001 INFO:teuthology.orchestra.run.smithi083.stdout: "pool_name": { 2024-08-19T01:43:33.001 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "pool_name", 2024-08-19T01:43:33.001 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-08-19T01:43:33.001 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:33.001 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-08-19T01:43:33.001 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "device_health_metrics", 2024-08-19T01:43:33.001 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:33.001 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:33.001 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:33.001 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "name of pool in which to store device health metrics", 2024-08-19T01:43:33.001 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:33.001 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:33.001 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:33.001 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:33.001 INFO:teuthology.orchestra.run.smithi083.stdout: "retention_period": { 2024-08-19T01:43:33.002 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "retention_period", 2024-08-19T01:43:33.002 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "secs", 2024-08-19T01:43:33.002 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:33.002 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-08-19T01:43:33.002 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "15552000", 2024-08-19T01:43:33.002 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:33.002 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:33.002 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:33.002 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "how long to retain device health metrics", 2024-08-19T01:43:33.002 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:33.002 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:33.002 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:33.002 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:33.002 INFO:teuthology.orchestra.run.smithi083.stdout: "scrape_frequency": { 2024-08-19T01:43:33.003 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "scrape_frequency", 2024-08-19T01:43:33.003 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "secs", 2024-08-19T01:43:33.003 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:33.003 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-08-19T01:43:33.003 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "86400", 2024-08-19T01:43:33.003 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:33.003 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:33.003 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:33.003 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "how frequently to scrape device health metrics", 2024-08-19T01:43:33.003 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:33.003 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:33.003 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:33.003 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:33.003 INFO:teuthology.orchestra.run.smithi083.stdout: "self_heal": { 2024-08-19T01:43:33.003 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "self_heal", 2024-08-19T01:43:33.003 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-08-19T01:43:33.004 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:33.004 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-08-19T01:43:33.004 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "True", 2024-08-19T01:43:33.004 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:33.004 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:33.004 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:33.004 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-08-19T01:43:33.004 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:33.004 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:33.004 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:33.004 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:33.004 INFO:teuthology.orchestra.run.smithi083.stdout: "sleep_interval": { 2024-08-19T01:43:33.004 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "sleep_interval", 2024-08-19T01:43:33.005 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "secs", 2024-08-19T01:43:33.005 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:33.005 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-08-19T01:43:33.005 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "600", 2024-08-19T01:43:33.005 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:33.005 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:33.005 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:33.005 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "how frequently to wake up and check device health", 2024-08-19T01:43:33.005 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:33.005 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:33.005 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:33.005 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:33.005 INFO:teuthology.orchestra.run.smithi083.stdout: "warn_threshold": { 2024-08-19T01:43:33.005 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "warn_threshold", 2024-08-19T01:43:33.006 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "secs", 2024-08-19T01:43:33.006 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:33.006 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-08-19T01:43:33.006 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "7257600", 2024-08-19T01:43:33.006 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:33.006 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:33.006 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:33.006 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-08-19T01:43:33.006 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:33.006 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:33.006 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:33.006 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-08-19T01:43:33.006 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-08-19T01:43:33.007 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:33.007 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-08-19T01:43:33.007 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "diskprediction_local", 2024-08-19T01:43:33.007 INFO:teuthology.orchestra.run.smithi083.stdout: "can_run": true, 2024-08-19T01:43:33.007 INFO:teuthology.orchestra.run.smithi083.stdout: "error_string": "", 2024-08-19T01:43:33.007 INFO:teuthology.orchestra.run.smithi083.stdout: "module_options": { 2024-08-19T01:43:33.007 INFO:teuthology.orchestra.run.smithi083.stdout: "log_level": { 2024-08-19T01:43:33.007 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_level", 2024-08-19T01:43:33.007 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-08-19T01:43:33.007 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:33.007 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-08-19T01:43:33.007 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-08-19T01:43:33.007 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:33.007 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:33.008 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-08-19T01:43:33.008 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-08-19T01:43:33.008 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-08-19T01:43:33.008 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-08-19T01:43:33.008 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-08-19T01:43:33.008 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-08-19T01:43:33.008 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-08-19T01:43:33.008 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-08-19T01:43:33.008 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-08-19T01:43:33.008 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:33.008 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:33.008 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:33.008 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:33.008 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster": { 2024-08-19T01:43:33.008 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster", 2024-08-19T01:43:33.009 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-08-19T01:43:33.009 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:33.009 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-08-19T01:43:33.009 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-08-19T01:43:33.009 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:33.009 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:33.009 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:33.009 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-08-19T01:43:33.009 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:33.009 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:33.009 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:33.009 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:33.009 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster_level": { 2024-08-19T01:43:33.009 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster_level", 2024-08-19T01:43:33.010 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-08-19T01:43:33.010 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:33.010 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-08-19T01:43:33.010 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "info", 2024-08-19T01:43:33.010 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:33.010 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:33.010 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-08-19T01:43:33.010 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-08-19T01:43:33.010 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-08-19T01:43:33.010 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-08-19T01:43:33.010 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-08-19T01:43:33.010 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-08-19T01:43:33.010 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-08-19T01:43:33.011 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-08-19T01:43:33.011 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-08-19T01:43:33.011 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:33.011 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:33.011 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:33.011 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:33.011 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_file": { 2024-08-19T01:43:33.011 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_file", 2024-08-19T01:43:33.011 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-08-19T01:43:33.011 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:33.011 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-08-19T01:43:33.011 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-08-19T01:43:33.011 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:33.011 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:33.012 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:33.012 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-08-19T01:43:33.012 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:33.012 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:33.012 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:33.012 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:33.012 INFO:teuthology.orchestra.run.smithi083.stdout: "predict_interval": { 2024-08-19T01:43:33.012 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "predict_interval", 2024-08-19T01:43:33.012 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-08-19T01:43:33.012 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:33.012 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-08-19T01:43:33.013 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "86400", 2024-08-19T01:43:33.013 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:33.013 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:33.013 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:33.013 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-08-19T01:43:33.013 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:33.013 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:33.013 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:33.013 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:33.013 INFO:teuthology.orchestra.run.smithi083.stdout: "predictor_model": { 2024-08-19T01:43:33.013 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "predictor_model", 2024-08-19T01:43:33.013 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-08-19T01:43:33.013 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:33.013 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-08-19T01:43:33.014 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "prophetstor", 2024-08-19T01:43:33.014 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:33.014 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:33.014 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:33.014 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-08-19T01:43:33.014 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:33.014 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:33.014 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:33.014 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:33.014 INFO:teuthology.orchestra.run.smithi083.stdout: "sleep_interval": { 2024-08-19T01:43:33.014 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "sleep_interval", 2024-08-19T01:43:33.014 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-08-19T01:43:33.014 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:33.015 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-08-19T01:43:33.015 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "600", 2024-08-19T01:43:33.015 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:33.015 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:33.015 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:33.015 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-08-19T01:43:33.015 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:33.015 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:33.015 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:33.015 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-08-19T01:43:33.015 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-08-19T01:43:33.015 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:33.016 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-08-19T01:43:33.016 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "influx", 2024-08-19T01:43:33.016 INFO:teuthology.orchestra.run.smithi083.stdout: "can_run": false, 2024-08-19T01:43:33.016 INFO:teuthology.orchestra.run.smithi083.stdout: "error_string": "influxdb python module not found", 2024-08-19T01:43:33.016 INFO:teuthology.orchestra.run.smithi083.stdout: "module_options": { 2024-08-19T01:43:33.016 INFO:teuthology.orchestra.run.smithi083.stdout: "batch_size": { 2024-08-19T01:43:33.016 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "batch_size", 2024-08-19T01:43:33.016 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2024-08-19T01:43:33.016 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:33.016 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-08-19T01:43:33.016 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "5000", 2024-08-19T01:43:33.016 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:33.016 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:33.017 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:33.017 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-08-19T01:43:33.017 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:33.017 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:33.017 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:33.017 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:33.017 INFO:teuthology.orchestra.run.smithi083.stdout: "database": { 2024-08-19T01:43:33.017 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "database", 2024-08-19T01:43:33.017 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-08-19T01:43:33.017 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:33.017 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-08-19T01:43:33.017 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "ceph", 2024-08-19T01:43:33.017 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:33.017 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:33.017 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:33.018 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-08-19T01:43:33.018 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:33.018 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:33.018 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:33.018 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:33.018 INFO:teuthology.orchestra.run.smithi083.stdout: "hostname": { 2024-08-19T01:43:33.018 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "hostname", 2024-08-19T01:43:33.018 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-08-19T01:43:33.018 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:33.018 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-08-19T01:43:33.018 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-08-19T01:43:33.018 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:33.018 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:33.018 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:33.019 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "InfluxDB server hostname", 2024-08-19T01:43:33.019 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:33.019 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:33.019 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:33.019 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:33.019 INFO:teuthology.orchestra.run.smithi083.stdout: "interval": { 2024-08-19T01:43:33.019 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "interval", 2024-08-19T01:43:33.019 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "secs", 2024-08-19T01:43:33.019 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:33.019 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-08-19T01:43:33.019 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "30", 2024-08-19T01:43:33.019 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "5", 2024-08-19T01:43:33.019 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:33.019 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:33.020 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-08-19T01:43:33.020 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:33.020 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:33.020 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:33.020 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:33.020 INFO:teuthology.orchestra.run.smithi083.stdout: "log_level": { 2024-08-19T01:43:33.020 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_level", 2024-08-19T01:43:33.020 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-08-19T01:43:33.020 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:33.020 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-08-19T01:43:33.020 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-08-19T01:43:33.020 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:33.020 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:33.020 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-08-19T01:43:33.020 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-08-19T01:43:33.021 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-08-19T01:43:33.021 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-08-19T01:43:33.021 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-08-19T01:43:33.021 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-08-19T01:43:33.021 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-08-19T01:43:33.021 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-08-19T01:43:33.021 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-08-19T01:43:33.021 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:33.021 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:33.021 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:33.021 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:33.021 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster": { 2024-08-19T01:43:33.021 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster", 2024-08-19T01:43:33.021 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-08-19T01:43:33.022 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:33.022 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-08-19T01:43:33.022 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-08-19T01:43:33.022 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:33.022 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:33.022 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:33.022 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-08-19T01:43:33.022 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:33.022 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:33.022 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:33.022 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:33.022 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster_level": { 2024-08-19T01:43:33.022 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster_level", 2024-08-19T01:43:33.022 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-08-19T01:43:33.023 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:33.023 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-08-19T01:43:33.023 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "info", 2024-08-19T01:43:33.023 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:33.023 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:33.023 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-08-19T01:43:33.023 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-08-19T01:43:33.023 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-08-19T01:43:33.023 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-08-19T01:43:33.023 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-08-19T01:43:33.023 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-08-19T01:43:33.023 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-08-19T01:43:33.023 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-08-19T01:43:33.023 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-08-19T01:43:33.023 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:33.024 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:33.024 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:33.024 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:33.024 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_file": { 2024-08-19T01:43:33.024 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_file", 2024-08-19T01:43:33.024 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-08-19T01:43:33.024 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:33.024 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-08-19T01:43:33.024 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-08-19T01:43:33.024 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:33.024 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:33.024 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:33.024 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-08-19T01:43:33.024 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:33.024 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:33.025 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:33.025 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:33.025 INFO:teuthology.orchestra.run.smithi083.stdout: "password": { 2024-08-19T01:43:33.025 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "password", 2024-08-19T01:43:33.025 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-08-19T01:43:33.025 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:33.025 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-08-19T01:43:33.025 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-08-19T01:43:33.025 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:33.025 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:33.025 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:33.025 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "password of InfluxDB server user", 2024-08-19T01:43:33.025 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:33.025 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:33.025 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:33.026 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:33.026 INFO:teuthology.orchestra.run.smithi083.stdout: "port": { 2024-08-19T01:43:33.026 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "port", 2024-08-19T01:43:33.026 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2024-08-19T01:43:33.026 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:33.026 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-08-19T01:43:33.026 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "8086", 2024-08-19T01:43:33.026 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:33.026 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:33.026 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:33.026 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "InfluxDB server port", 2024-08-19T01:43:33.026 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:33.026 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:33.026 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:33.027 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:33.027 INFO:teuthology.orchestra.run.smithi083.stdout: "ssl": { 2024-08-19T01:43:33.027 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "ssl", 2024-08-19T01:43:33.027 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-08-19T01:43:33.027 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:33.027 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-08-19T01:43:33.027 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "false", 2024-08-19T01:43:33.027 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:33.027 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:33.027 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:33.027 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-08-19T01:43:33.027 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:33.027 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:33.027 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:33.027 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:33.028 INFO:teuthology.orchestra.run.smithi083.stdout: "threads": { 2024-08-19T01:43:33.028 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "threads", 2024-08-19T01:43:33.028 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2024-08-19T01:43:33.028 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:33.028 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-08-19T01:43:33.028 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "5", 2024-08-19T01:43:33.028 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "1", 2024-08-19T01:43:33.028 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "32", 2024-08-19T01:43:33.028 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:33.028 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-08-19T01:43:33.028 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:33.028 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:33.028 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:33.028 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:33.028 INFO:teuthology.orchestra.run.smithi083.stdout: "username": { 2024-08-19T01:43:33.029 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "username", 2024-08-19T01:43:33.029 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-08-19T01:43:33.029 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:33.029 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-08-19T01:43:33.029 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-08-19T01:43:33.029 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:33.029 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:33.029 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:33.029 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "username of InfluxDB server user", 2024-08-19T01:43:33.029 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:33.029 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:33.029 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:33.029 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:33.029 INFO:teuthology.orchestra.run.smithi083.stdout: "verify_ssl": { 2024-08-19T01:43:33.029 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "verify_ssl", 2024-08-19T01:43:33.030 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-08-19T01:43:33.030 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:33.030 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-08-19T01:43:33.030 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "true", 2024-08-19T01:43:33.030 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:33.030 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:33.030 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:33.030 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-08-19T01:43:33.030 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:33.030 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:33.030 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:33.030 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-08-19T01:43:33.030 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-08-19T01:43:33.030 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:33.030 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-08-19T01:43:33.031 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "insights", 2024-08-19T01:43:33.031 INFO:teuthology.orchestra.run.smithi083.stdout: "can_run": true, 2024-08-19T01:43:33.031 INFO:teuthology.orchestra.run.smithi083.stdout: "error_string": "", 2024-08-19T01:43:33.031 INFO:teuthology.orchestra.run.smithi083.stdout: "module_options": { 2024-08-19T01:43:33.031 INFO:teuthology.orchestra.run.smithi083.stdout: "log_level": { 2024-08-19T01:43:33.031 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_level", 2024-08-19T01:43:33.031 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-08-19T01:43:33.031 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:33.031 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-08-19T01:43:33.031 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-08-19T01:43:33.031 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:33.031 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:33.031 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-08-19T01:43:33.031 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-08-19T01:43:33.032 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-08-19T01:43:33.032 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-08-19T01:43:33.032 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-08-19T01:43:33.032 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-08-19T01:43:33.032 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-08-19T01:43:33.032 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-08-19T01:43:33.032 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-08-19T01:43:33.032 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:33.032 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:33.032 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:33.032 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:33.032 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster": { 2024-08-19T01:43:33.033 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster", 2024-08-19T01:43:33.033 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-08-19T01:43:33.033 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:33.033 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-08-19T01:43:33.033 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-08-19T01:43:33.033 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:33.033 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:33.033 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:33.033 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-08-19T01:43:33.033 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:33.033 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:33.033 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:33.033 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:33.033 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster_level": { 2024-08-19T01:43:33.033 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster_level", 2024-08-19T01:43:33.034 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-08-19T01:43:33.034 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:33.034 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-08-19T01:43:33.034 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "info", 2024-08-19T01:43:33.034 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:33.034 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:33.034 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-08-19T01:43:33.034 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-08-19T01:43:33.034 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-08-19T01:43:33.034 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-08-19T01:43:33.034 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-08-19T01:43:33.034 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-08-19T01:43:33.034 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-08-19T01:43:33.035 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-08-19T01:43:33.035 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-08-19T01:43:33.035 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:33.035 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:33.035 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:33.035 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:33.035 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_file": { 2024-08-19T01:43:33.035 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_file", 2024-08-19T01:43:33.035 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-08-19T01:43:33.035 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:33.035 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-08-19T01:43:33.035 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-08-19T01:43:33.035 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:33.035 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:33.036 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:33.036 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-08-19T01:43:33.036 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:33.036 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:33.036 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:33.036 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-08-19T01:43:33.036 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-08-19T01:43:33.036 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:33.036 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-08-19T01:43:33.036 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "iostat", 2024-08-19T01:43:33.036 INFO:teuthology.orchestra.run.smithi083.stdout: "can_run": true, 2024-08-19T01:43:33.036 INFO:teuthology.orchestra.run.smithi083.stdout: "error_string": "", 2024-08-19T01:43:33.036 INFO:teuthology.orchestra.run.smithi083.stdout: "module_options": { 2024-08-19T01:43:33.036 INFO:teuthology.orchestra.run.smithi083.stdout: "log_level": { 2024-08-19T01:43:33.036 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_level", 2024-08-19T01:43:33.037 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-08-19T01:43:33.037 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:33.037 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-08-19T01:43:33.037 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-08-19T01:43:33.037 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:33.037 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:33.037 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-08-19T01:43:33.037 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-08-19T01:43:33.037 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-08-19T01:43:33.037 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-08-19T01:43:33.037 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-08-19T01:43:33.037 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-08-19T01:43:33.037 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-08-19T01:43:33.037 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-08-19T01:43:33.037 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-08-19T01:43:33.038 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:33.038 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:33.038 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:33.038 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:33.038 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster": { 2024-08-19T01:43:33.038 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster", 2024-08-19T01:43:33.038 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-08-19T01:43:33.038 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:33.038 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-08-19T01:43:33.038 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-08-19T01:43:33.038 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:33.038 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:33.038 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:33.038 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-08-19T01:43:33.039 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:33.039 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:33.039 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:33.039 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:33.039 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster_level": { 2024-08-19T01:43:33.039 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster_level", 2024-08-19T01:43:33.039 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-08-19T01:43:33.039 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:33.039 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-08-19T01:43:33.039 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "info", 2024-08-19T01:43:33.039 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:33.039 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:33.039 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-08-19T01:43:33.039 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-08-19T01:43:33.039 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-08-19T01:43:33.040 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-08-19T01:43:33.040 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-08-19T01:43:33.040 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-08-19T01:43:33.040 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-08-19T01:43:33.040 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-08-19T01:43:33.040 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-08-19T01:43:33.040 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:33.040 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:33.040 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:33.040 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:33.040 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_file": { 2024-08-19T01:43:33.040 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_file", 2024-08-19T01:43:33.040 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-08-19T01:43:33.040 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:33.040 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-08-19T01:43:33.040 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-08-19T01:43:33.041 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:33.041 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:33.041 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:33.041 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-08-19T01:43:33.041 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:33.041 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:33.041 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:33.041 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-08-19T01:43:33.041 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-08-19T01:43:33.041 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:33.041 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-08-19T01:43:33.041 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "k8sevents", 2024-08-19T01:43:33.041 INFO:teuthology.orchestra.run.smithi083.stdout: "can_run": true, 2024-08-19T01:43:33.041 INFO:teuthology.orchestra.run.smithi083.stdout: "error_string": "", 2024-08-19T01:43:33.042 INFO:teuthology.orchestra.run.smithi083.stdout: "module_options": { 2024-08-19T01:43:33.042 INFO:teuthology.orchestra.run.smithi083.stdout: "ceph_event_retention_days": { 2024-08-19T01:43:33.042 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "ceph_event_retention_days", 2024-08-19T01:43:33.042 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2024-08-19T01:43:33.042 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:33.042 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-08-19T01:43:33.042 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "7", 2024-08-19T01:43:33.042 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:33.042 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:33.042 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:33.042 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Days to hold ceph event information within local cache", 2024-08-19T01:43:33.042 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:33.042 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:33.042 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:33.043 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:33.043 INFO:teuthology.orchestra.run.smithi083.stdout: "config_check_secs": { 2024-08-19T01:43:33.043 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "config_check_secs", 2024-08-19T01:43:33.043 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2024-08-19T01:43:33.043 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:33.043 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-08-19T01:43:33.043 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "10", 2024-08-19T01:43:33.043 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "10", 2024-08-19T01:43:33.043 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:33.043 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:33.043 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-08-19T01:43:33.043 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:33.043 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:33.043 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:33.043 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:33.043 INFO:teuthology.orchestra.run.smithi083.stdout: "log_level": { 2024-08-19T01:43:33.044 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_level", 2024-08-19T01:43:33.044 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-08-19T01:43:33.044 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:33.044 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-08-19T01:43:33.044 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-08-19T01:43:33.044 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:33.044 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:33.044 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-08-19T01:43:33.044 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-08-19T01:43:33.044 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-08-19T01:43:33.044 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-08-19T01:43:33.044 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-08-19T01:43:33.044 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-08-19T01:43:33.044 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-08-19T01:43:33.044 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-08-19T01:43:33.045 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-08-19T01:43:33.045 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:33.045 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:33.045 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:33.045 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:33.045 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster": { 2024-08-19T01:43:33.045 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster", 2024-08-19T01:43:33.045 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-08-19T01:43:33.045 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:33.045 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-08-19T01:43:33.045 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-08-19T01:43:33.045 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:33.045 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:33.045 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:33.045 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-08-19T01:43:33.046 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:33.046 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:33.046 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:33.046 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:33.046 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster_level": { 2024-08-19T01:43:33.046 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster_level", 2024-08-19T01:43:33.046 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-08-19T01:43:33.046 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:33.046 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-08-19T01:43:33.046 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "info", 2024-08-19T01:43:33.046 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:33.046 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:33.046 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-08-19T01:43:33.046 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-08-19T01:43:33.047 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-08-19T01:43:33.047 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-08-19T01:43:33.047 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-08-19T01:43:33.047 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-08-19T01:43:33.047 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-08-19T01:43:33.047 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-08-19T01:43:33.047 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-08-19T01:43:33.047 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:33.047 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:33.047 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:33.047 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:33.047 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_file": { 2024-08-19T01:43:33.047 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_file", 2024-08-19T01:43:33.048 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-08-19T01:43:33.048 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:33.048 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-08-19T01:43:33.048 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-08-19T01:43:33.048 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:33.048 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:33.048 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:33.048 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-08-19T01:43:33.048 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:33.048 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:33.048 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:33.048 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-08-19T01:43:33.048 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-08-19T01:43:33.048 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:33.049 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-08-19T01:43:33.049 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "localpool", 2024-08-19T01:43:33.049 INFO:teuthology.orchestra.run.smithi083.stdout: "can_run": true, 2024-08-19T01:43:33.049 INFO:teuthology.orchestra.run.smithi083.stdout: "error_string": "", 2024-08-19T01:43:33.049 INFO:teuthology.orchestra.run.smithi083.stdout: "module_options": { 2024-08-19T01:43:33.049 INFO:teuthology.orchestra.run.smithi083.stdout: "failure_domain": { 2024-08-19T01:43:33.049 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "failure_domain", 2024-08-19T01:43:33.049 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-08-19T01:43:33.049 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:33.049 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-08-19T01:43:33.049 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "host", 2024-08-19T01:43:33.049 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:33.049 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:33.049 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:33.050 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "failure domain for any created local pool", 2024-08-19T01:43:33.050 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-08-19T01:43:33.050 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:33.050 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:33.050 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:33.050 INFO:teuthology.orchestra.run.smithi083.stdout: "log_level": { 2024-08-19T01:43:33.050 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_level", 2024-08-19T01:43:33.050 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-08-19T01:43:33.050 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:33.050 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-08-19T01:43:33.050 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-08-19T01:43:33.050 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:33.050 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:33.050 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-08-19T01:43:33.051 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-08-19T01:43:33.051 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-08-19T01:43:33.051 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-08-19T01:43:33.051 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-08-19T01:43:33.051 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-08-19T01:43:33.051 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-08-19T01:43:33.051 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-08-19T01:43:33.051 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-08-19T01:43:33.051 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:33.051 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:33.051 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:33.051 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:33.052 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster": { 2024-08-19T01:43:33.052 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster", 2024-08-19T01:43:33.052 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-08-19T01:43:33.052 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:33.052 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-08-19T01:43:33.052 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-08-19T01:43:33.052 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:33.052 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:33.052 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:33.052 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-08-19T01:43:33.052 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:33.052 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:33.052 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:33.053 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:33.053 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster_level": { 2024-08-19T01:43:33.053 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster_level", 2024-08-19T01:43:33.053 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-08-19T01:43:33.053 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:33.053 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-08-19T01:43:33.053 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "info", 2024-08-19T01:43:33.053 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:33.053 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:33.053 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-08-19T01:43:33.053 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-08-19T01:43:33.053 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-08-19T01:43:33.053 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-08-19T01:43:33.053 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-08-19T01:43:33.054 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-08-19T01:43:33.054 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-08-19T01:43:33.054 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-08-19T01:43:33.054 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-08-19T01:43:33.054 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:33.054 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:33.054 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:33.054 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:33.054 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_file": { 2024-08-19T01:43:33.054 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_file", 2024-08-19T01:43:33.054 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-08-19T01:43:33.054 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:33.054 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-08-19T01:43:33.055 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-08-19T01:43:33.055 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:33.055 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:33.055 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:33.055 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-08-19T01:43:33.055 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:33.055 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:33.055 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:33.055 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:33.055 INFO:teuthology.orchestra.run.smithi083.stdout: "min_size": { 2024-08-19T01:43:33.055 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "min_size", 2024-08-19T01:43:33.055 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2024-08-19T01:43:33.055 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:33.056 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-08-19T01:43:33.056 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-08-19T01:43:33.056 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:33.056 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:33.056 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:33.056 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "default min_size for any created local pool", 2024-08-19T01:43:33.056 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-08-19T01:43:33.056 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:33.056 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:33.056 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:33.056 INFO:teuthology.orchestra.run.smithi083.stdout: "num_rep": { 2024-08-19T01:43:33.056 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "num_rep", 2024-08-19T01:43:33.056 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2024-08-19T01:43:33.056 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:33.057 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-08-19T01:43:33.057 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "3", 2024-08-19T01:43:33.057 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:33.057 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:33.057 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:33.057 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "default replica count for any created local pool", 2024-08-19T01:43:33.057 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:33.057 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:33.057 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:33.057 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:33.057 INFO:teuthology.orchestra.run.smithi083.stdout: "pg_num": { 2024-08-19T01:43:33.057 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "pg_num", 2024-08-19T01:43:33.057 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2024-08-19T01:43:33.057 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:33.058 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-08-19T01:43:33.058 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "128", 2024-08-19T01:43:33.058 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:33.058 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:33.058 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:33.058 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "default pg_num for any created local pool", 2024-08-19T01:43:33.058 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:33.058 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:33.058 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:33.058 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:33.058 INFO:teuthology.orchestra.run.smithi083.stdout: "prefix": { 2024-08-19T01:43:33.058 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "prefix", 2024-08-19T01:43:33.059 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-08-19T01:43:33.059 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:33.059 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-08-19T01:43:33.059 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-08-19T01:43:33.059 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:33.059 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:33.059 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:33.059 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "name prefix for any created local pool", 2024-08-19T01:43:33.059 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:33.059 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:33.059 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:33.059 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:33.059 INFO:teuthology.orchestra.run.smithi083.stdout: "subtree": { 2024-08-19T01:43:33.059 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "subtree", 2024-08-19T01:43:33.059 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-08-19T01:43:33.060 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:33.060 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-08-19T01:43:33.060 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "rack", 2024-08-19T01:43:33.060 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:33.060 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:33.060 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:33.060 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "CRUSH level for which to create a local pool", 2024-08-19T01:43:33.060 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-08-19T01:43:33.060 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:33.060 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:33.060 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-08-19T01:43:33.060 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-08-19T01:43:33.061 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:33.061 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-08-19T01:43:33.061 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "mds_autoscaler", 2024-08-19T01:43:33.061 INFO:teuthology.orchestra.run.smithi083.stdout: "can_run": true, 2024-08-19T01:43:33.061 INFO:teuthology.orchestra.run.smithi083.stdout: "error_string": "", 2024-08-19T01:43:33.061 INFO:teuthology.orchestra.run.smithi083.stdout: "module_options": { 2024-08-19T01:43:33.061 INFO:teuthology.orchestra.run.smithi083.stdout: "log_level": { 2024-08-19T01:43:33.061 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_level", 2024-08-19T01:43:33.061 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-08-19T01:43:33.061 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:33.061 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-08-19T01:43:33.061 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-08-19T01:43:33.061 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:33.061 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:33.062 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-08-19T01:43:33.062 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-08-19T01:43:33.062 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-08-19T01:43:33.062 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-08-19T01:43:33.062 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-08-19T01:43:33.062 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-08-19T01:43:33.062 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-08-19T01:43:33.062 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-08-19T01:43:33.062 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-08-19T01:43:33.062 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:33.062 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:33.063 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:33.063 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:33.063 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster": { 2024-08-19T01:43:33.063 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster", 2024-08-19T01:43:33.063 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-08-19T01:43:33.063 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:33.063 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-08-19T01:43:33.063 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-08-19T01:43:33.063 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:33.063 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:33.063 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:33.063 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-08-19T01:43:33.063 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:33.063 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:33.063 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:33.064 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:33.064 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster_level": { 2024-08-19T01:43:33.064 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster_level", 2024-08-19T01:43:33.064 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-08-19T01:43:33.064 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:33.064 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-08-19T01:43:33.064 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "info", 2024-08-19T01:43:33.064 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:33.064 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:33.064 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-08-19T01:43:33.064 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-08-19T01:43:33.064 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-08-19T01:43:33.064 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-08-19T01:43:33.064 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-08-19T01:43:33.065 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-08-19T01:43:33.065 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-08-19T01:43:33.065 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-08-19T01:43:33.065 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-08-19T01:43:33.065 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:33.065 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:33.065 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:33.065 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:33.065 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_file": { 2024-08-19T01:43:33.065 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_file", 2024-08-19T01:43:33.065 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-08-19T01:43:33.065 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:33.065 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-08-19T01:43:33.065 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-08-19T01:43:33.066 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:33.066 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:33.066 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:33.066 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-08-19T01:43:33.066 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:33.066 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:33.066 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:33.066 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-08-19T01:43:33.066 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-08-19T01:43:33.066 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:33.066 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-08-19T01:43:33.066 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "mirroring", 2024-08-19T01:43:33.066 INFO:teuthology.orchestra.run.smithi083.stdout: "can_run": true, 2024-08-19T01:43:33.066 INFO:teuthology.orchestra.run.smithi083.stdout: "error_string": "", 2024-08-19T01:43:33.067 INFO:teuthology.orchestra.run.smithi083.stdout: "module_options": { 2024-08-19T01:43:33.067 INFO:teuthology.orchestra.run.smithi083.stdout: "log_level": { 2024-08-19T01:43:33.067 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_level", 2024-08-19T01:43:33.067 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-08-19T01:43:33.067 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:33.067 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-08-19T01:43:33.067 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-08-19T01:43:33.067 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:33.067 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:33.067 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-08-19T01:43:33.067 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-08-19T01:43:33.067 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-08-19T01:43:33.067 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-08-19T01:43:33.067 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-08-19T01:43:33.068 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-08-19T01:43:33.068 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-08-19T01:43:33.068 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-08-19T01:43:33.068 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-08-19T01:43:33.068 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:33.068 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:33.068 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:33.068 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:33.068 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster": { 2024-08-19T01:43:33.068 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster", 2024-08-19T01:43:33.068 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-08-19T01:43:33.068 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:33.068 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-08-19T01:43:33.068 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-08-19T01:43:33.069 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:33.069 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:33.069 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:33.069 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-08-19T01:43:33.069 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:33.069 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:33.069 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:33.069 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:33.069 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster_level": { 2024-08-19T01:43:33.069 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster_level", 2024-08-19T01:43:33.069 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-08-19T01:43:33.069 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:33.069 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-08-19T01:43:33.069 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "info", 2024-08-19T01:43:33.070 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:33.070 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:33.070 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-08-19T01:43:33.070 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-08-19T01:43:33.070 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-08-19T01:43:33.070 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-08-19T01:43:33.070 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-08-19T01:43:33.070 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-08-19T01:43:33.070 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-08-19T01:43:33.070 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-08-19T01:43:33.070 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-08-19T01:43:33.070 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:33.070 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:33.070 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:33.071 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:33.071 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_file": { 2024-08-19T01:43:33.071 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_file", 2024-08-19T01:43:33.071 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-08-19T01:43:33.071 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:33.071 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-08-19T01:43:33.071 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-08-19T01:43:33.071 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:33.071 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:33.071 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:33.071 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-08-19T01:43:33.071 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:33.071 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:33.071 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:33.072 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-08-19T01:43:33.072 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-08-19T01:43:33.072 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:33.072 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-08-19T01:43:33.072 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "nfs", 2024-08-19T01:43:33.072 INFO:teuthology.orchestra.run.smithi083.stdout: "can_run": true, 2024-08-19T01:43:33.072 INFO:teuthology.orchestra.run.smithi083.stdout: "error_string": "", 2024-08-19T01:43:33.072 INFO:teuthology.orchestra.run.smithi083.stdout: "module_options": { 2024-08-19T01:43:33.072 INFO:teuthology.orchestra.run.smithi083.stdout: "log_level": { 2024-08-19T01:43:33.072 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_level", 2024-08-19T01:43:33.072 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-08-19T01:43:33.072 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:33.072 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-08-19T01:43:33.072 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-08-19T01:43:33.072 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:33.073 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:33.073 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-08-19T01:43:33.073 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-08-19T01:43:33.073 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-08-19T01:43:33.073 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-08-19T01:43:33.073 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-08-19T01:43:33.073 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-08-19T01:43:33.073 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-08-19T01:43:33.073 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-08-19T01:43:33.073 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-08-19T01:43:33.073 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:33.073 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:33.073 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:33.073 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:33.073 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster": { 2024-08-19T01:43:33.074 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster", 2024-08-19T01:43:33.074 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-08-19T01:43:33.074 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:33.074 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-08-19T01:43:33.074 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-08-19T01:43:33.074 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:33.074 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:33.074 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:33.074 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-08-19T01:43:33.074 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:33.074 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:33.074 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:33.074 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:33.075 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster_level": { 2024-08-19T01:43:33.075 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster_level", 2024-08-19T01:43:33.075 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-08-19T01:43:33.075 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:33.075 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-08-19T01:43:33.075 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "info", 2024-08-19T01:43:33.075 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:33.075 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:33.075 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-08-19T01:43:33.075 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-08-19T01:43:33.075 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-08-19T01:43:33.075 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-08-19T01:43:33.075 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-08-19T01:43:33.075 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-08-19T01:43:33.075 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-08-19T01:43:33.077 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-08-19T01:43:33.077 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-08-19T01:43:33.077 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:33.077 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:33.077 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:33.078 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:33.078 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_file": { 2024-08-19T01:43:33.078 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_file", 2024-08-19T01:43:33.078 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-08-19T01:43:33.078 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:33.078 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-08-19T01:43:33.078 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-08-19T01:43:33.078 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:33.078 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:33.078 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:33.078 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-08-19T01:43:33.078 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:33.078 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:33.078 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:33.078 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-08-19T01:43:33.079 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-08-19T01:43:33.079 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:33.079 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-08-19T01:43:33.079 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "orchestrator", 2024-08-19T01:43:33.079 INFO:teuthology.orchestra.run.smithi083.stdout: "can_run": true, 2024-08-19T01:43:33.079 INFO:teuthology.orchestra.run.smithi083.stdout: "error_string": "", 2024-08-19T01:43:33.079 INFO:teuthology.orchestra.run.smithi083.stdout: "module_options": { 2024-08-19T01:43:33.079 INFO:teuthology.orchestra.run.smithi083.stdout: "fail_fs": { 2024-08-19T01:43:33.079 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "fail_fs", 2024-08-19T01:43:33.079 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-08-19T01:43:33.079 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:33.079 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-08-19T01:43:33.079 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-08-19T01:43:33.079 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:33.079 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:33.080 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:33.080 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-08-19T01:43:33.080 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:33.080 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:33.080 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:33.080 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:33.080 INFO:teuthology.orchestra.run.smithi083.stdout: "log_level": { 2024-08-19T01:43:33.080 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_level", 2024-08-19T01:43:33.080 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-08-19T01:43:33.080 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:33.080 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-08-19T01:43:33.080 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-08-19T01:43:33.080 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:33.080 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:33.080 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-08-19T01:43:33.080 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-08-19T01:43:33.081 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-08-19T01:43:33.081 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-08-19T01:43:33.081 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-08-19T01:43:33.081 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-08-19T01:43:33.081 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-08-19T01:43:33.081 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-08-19T01:43:33.081 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-08-19T01:43:33.081 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:33.081 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:33.081 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:33.081 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:33.081 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster": { 2024-08-19T01:43:33.081 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster", 2024-08-19T01:43:33.081 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-08-19T01:43:33.082 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:33.082 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-08-19T01:43:33.082 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-08-19T01:43:33.082 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:33.082 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:33.082 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:33.082 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-08-19T01:43:33.082 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:33.082 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:33.082 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:33.082 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:33.082 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster_level": { 2024-08-19T01:43:33.082 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster_level", 2024-08-19T01:43:33.083 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-08-19T01:43:33.083 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:33.083 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-08-19T01:43:33.083 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "info", 2024-08-19T01:43:33.083 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:33.083 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:33.083 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-08-19T01:43:33.083 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-08-19T01:43:33.083 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-08-19T01:43:33.083 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-08-19T01:43:33.083 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-08-19T01:43:33.083 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-08-19T01:43:33.083 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-08-19T01:43:33.083 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-08-19T01:43:33.084 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-08-19T01:43:33.084 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:33.084 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:33.084 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:33.084 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:33.084 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_file": { 2024-08-19T01:43:33.084 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_file", 2024-08-19T01:43:33.084 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-08-19T01:43:33.084 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:33.084 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-08-19T01:43:33.084 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-08-19T01:43:33.084 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:33.084 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:33.085 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:33.085 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-08-19T01:43:33.085 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:33.085 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:33.085 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:33.085 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:33.085 INFO:teuthology.orchestra.run.smithi083.stdout: "orchestrator": { 2024-08-19T01:43:33.085 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "orchestrator", 2024-08-19T01:43:33.085 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-08-19T01:43:33.085 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:33.085 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-08-19T01:43:33.085 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-08-19T01:43:33.085 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:33.086 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:33.086 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-08-19T01:43:33.086 INFO:teuthology.orchestra.run.smithi083.stdout: "cephadm", 2024-08-19T01:43:33.086 INFO:teuthology.orchestra.run.smithi083.stdout: "rook", 2024-08-19T01:43:33.086 INFO:teuthology.orchestra.run.smithi083.stdout: "test_orchestrator" 2024-08-19T01:43:33.086 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-08-19T01:43:33.086 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Orchestrator backend", 2024-08-19T01:43:33.086 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:33.086 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:33.086 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:33.086 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-08-19T01:43:33.086 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-08-19T01:43:33.086 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:33.087 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-08-19T01:43:33.087 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "osd_perf_query", 2024-08-19T01:43:33.087 INFO:teuthology.orchestra.run.smithi083.stdout: "can_run": true, 2024-08-19T01:43:33.087 INFO:teuthology.orchestra.run.smithi083.stdout: "error_string": "", 2024-08-19T01:43:33.087 INFO:teuthology.orchestra.run.smithi083.stdout: "module_options": { 2024-08-19T01:43:33.087 INFO:teuthology.orchestra.run.smithi083.stdout: "log_level": { 2024-08-19T01:43:33.087 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_level", 2024-08-19T01:43:33.087 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-08-19T01:43:33.087 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:33.087 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-08-19T01:43:33.087 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-08-19T01:43:33.087 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:33.088 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:33.088 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-08-19T01:43:33.088 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-08-19T01:43:33.088 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-08-19T01:43:33.088 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-08-19T01:43:33.088 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-08-19T01:43:33.088 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-08-19T01:43:33.088 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-08-19T01:43:33.088 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-08-19T01:43:33.088 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-08-19T01:43:33.088 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:33.088 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:33.088 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:33.089 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:33.089 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster": { 2024-08-19T01:43:33.089 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster", 2024-08-19T01:43:33.089 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-08-19T01:43:33.089 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:33.089 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-08-19T01:43:33.089 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-08-19T01:43:33.089 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:33.089 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:33.089 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:33.089 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-08-19T01:43:33.089 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:33.089 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:33.089 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:33.090 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:33.090 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster_level": { 2024-08-19T01:43:33.090 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster_level", 2024-08-19T01:43:33.090 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-08-19T01:43:33.090 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:33.090 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-08-19T01:43:33.090 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "info", 2024-08-19T01:43:33.090 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:33.090 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:33.090 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-08-19T01:43:33.090 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-08-19T01:43:33.090 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-08-19T01:43:33.091 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-08-19T01:43:33.091 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-08-19T01:43:33.091 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-08-19T01:43:33.091 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-08-19T01:43:33.091 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-08-19T01:43:33.091 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-08-19T01:43:33.091 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:33.091 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:33.091 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:33.091 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:33.091 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_file": { 2024-08-19T01:43:33.091 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_file", 2024-08-19T01:43:33.091 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-08-19T01:43:33.092 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:33.092 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-08-19T01:43:33.092 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-08-19T01:43:33.092 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:33.092 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:33.092 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:33.092 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-08-19T01:43:33.092 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:33.092 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:33.092 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:33.092 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-08-19T01:43:33.092 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-08-19T01:43:33.092 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:33.092 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-08-19T01:43:33.093 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "osd_support", 2024-08-19T01:43:33.093 INFO:teuthology.orchestra.run.smithi083.stdout: "can_run": true, 2024-08-19T01:43:33.093 INFO:teuthology.orchestra.run.smithi083.stdout: "error_string": "", 2024-08-19T01:43:33.093 INFO:teuthology.orchestra.run.smithi083.stdout: "module_options": { 2024-08-19T01:43:33.093 INFO:teuthology.orchestra.run.smithi083.stdout: "log_level": { 2024-08-19T01:43:33.093 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_level", 2024-08-19T01:43:33.093 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-08-19T01:43:33.093 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:33.093 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-08-19T01:43:33.093 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-08-19T01:43:33.093 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:33.093 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:33.093 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-08-19T01:43:33.093 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-08-19T01:43:33.094 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-08-19T01:43:33.094 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-08-19T01:43:33.094 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-08-19T01:43:33.094 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-08-19T01:43:33.094 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-08-19T01:43:33.094 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-08-19T01:43:33.094 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-08-19T01:43:33.094 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:33.094 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:33.094 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:33.094 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:33.094 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster": { 2024-08-19T01:43:33.094 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster", 2024-08-19T01:43:33.094 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-08-19T01:43:33.095 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:33.095 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-08-19T01:43:33.095 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-08-19T01:43:33.095 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:33.095 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:33.095 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:33.095 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-08-19T01:43:33.095 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:33.095 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:33.095 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:33.095 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:33.095 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster_level": { 2024-08-19T01:43:33.095 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster_level", 2024-08-19T01:43:33.095 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-08-19T01:43:33.095 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:33.096 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-08-19T01:43:33.096 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "info", 2024-08-19T01:43:33.096 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:33.096 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:33.096 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-08-19T01:43:33.096 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-08-19T01:43:33.096 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-08-19T01:43:33.096 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-08-19T01:43:33.096 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-08-19T01:43:33.096 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-08-19T01:43:33.096 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-08-19T01:43:33.096 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-08-19T01:43:33.096 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-08-19T01:43:33.096 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:33.097 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:33.097 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:33.097 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:33.097 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_file": { 2024-08-19T01:43:33.097 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_file", 2024-08-19T01:43:33.097 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-08-19T01:43:33.097 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:33.097 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-08-19T01:43:33.097 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-08-19T01:43:33.097 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:33.097 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:33.097 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:33.097 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-08-19T01:43:33.097 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:33.097 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:33.098 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:33.098 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-08-19T01:43:33.098 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-08-19T01:43:33.098 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:33.098 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-08-19T01:43:33.098 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "pg_autoscaler", 2024-08-19T01:43:33.098 INFO:teuthology.orchestra.run.smithi083.stdout: "can_run": true, 2024-08-19T01:43:33.098 INFO:teuthology.orchestra.run.smithi083.stdout: "error_string": "", 2024-08-19T01:43:33.098 INFO:teuthology.orchestra.run.smithi083.stdout: "module_options": { 2024-08-19T01:43:33.098 INFO:teuthology.orchestra.run.smithi083.stdout: "log_level": { 2024-08-19T01:43:33.098 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_level", 2024-08-19T01:43:33.098 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-08-19T01:43:33.098 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:33.099 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-08-19T01:43:33.099 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-08-19T01:43:33.099 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:33.099 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:33.099 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-08-19T01:43:33.099 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-08-19T01:43:33.099 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-08-19T01:43:33.099 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-08-19T01:43:33.099 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-08-19T01:43:33.099 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-08-19T01:43:33.099 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-08-19T01:43:33.099 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-08-19T01:43:33.099 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-08-19T01:43:33.099 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:33.099 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:33.100 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:33.100 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:33.100 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster": { 2024-08-19T01:43:33.100 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster", 2024-08-19T01:43:33.100 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-08-19T01:43:33.100 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:33.100 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-08-19T01:43:33.100 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-08-19T01:43:33.100 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:33.100 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:33.100 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:33.100 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-08-19T01:43:33.100 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:33.100 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:33.100 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:33.101 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:33.101 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster_level": { 2024-08-19T01:43:33.101 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster_level", 2024-08-19T01:43:33.101 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-08-19T01:43:33.101 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:33.101 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-08-19T01:43:33.101 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "info", 2024-08-19T01:43:33.101 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:33.101 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:33.101 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-08-19T01:43:33.101 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-08-19T01:43:33.101 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-08-19T01:43:33.101 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-08-19T01:43:33.102 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-08-19T01:43:33.102 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-08-19T01:43:33.102 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-08-19T01:43:33.102 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-08-19T01:43:33.102 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-08-19T01:43:33.102 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:33.102 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:33.102 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:33.102 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:33.102 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_file": { 2024-08-19T01:43:33.102 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_file", 2024-08-19T01:43:33.102 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-08-19T01:43:33.102 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:33.102 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-08-19T01:43:33.103 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-08-19T01:43:33.103 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:33.103 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:33.103 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:33.103 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-08-19T01:43:33.103 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:33.103 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:33.103 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:33.103 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:33.103 INFO:teuthology.orchestra.run.smithi083.stdout: "sleep_interval": { 2024-08-19T01:43:33.103 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "sleep_interval", 2024-08-19T01:43:33.103 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "secs", 2024-08-19T01:43:33.103 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:33.103 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-08-19T01:43:33.103 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "60", 2024-08-19T01:43:33.104 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:33.104 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:33.104 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:33.104 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-08-19T01:43:33.104 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:33.104 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:33.104 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:33.104 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:33.104 INFO:teuthology.orchestra.run.smithi083.stdout: "threshold": { 2024-08-19T01:43:33.104 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "threshold", 2024-08-19T01:43:33.104 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "float", 2024-08-19T01:43:33.104 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:33.104 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-08-19T01:43:33.104 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "3.0", 2024-08-19T01:43:33.104 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "1.0", 2024-08-19T01:43:33.105 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:33.105 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:33.105 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "scaling threshold", 2024-08-19T01:43:33.105 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-08-19T01:43:33.105 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:33.105 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:33.105 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-08-19T01:43:33.105 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-08-19T01:43:33.105 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:33.105 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-08-19T01:43:33.105 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "progress", 2024-08-19T01:43:33.105 INFO:teuthology.orchestra.run.smithi083.stdout: "can_run": true, 2024-08-19T01:43:33.105 INFO:teuthology.orchestra.run.smithi083.stdout: "error_string": "", 2024-08-19T01:43:33.105 INFO:teuthology.orchestra.run.smithi083.stdout: "module_options": { 2024-08-19T01:43:33.106 INFO:teuthology.orchestra.run.smithi083.stdout: "allow_pg_recovery_event": { 2024-08-19T01:43:33.106 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "allow_pg_recovery_event", 2024-08-19T01:43:33.106 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-08-19T01:43:33.106 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:33.106 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-08-19T01:43:33.106 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-08-19T01:43:33.106 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:33.106 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:33.106 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:33.106 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "allow the module to show pg recovery progress", 2024-08-19T01:43:33.106 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:33.106 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:33.107 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:33.107 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:33.107 INFO:teuthology.orchestra.run.smithi083.stdout: "enabled": { 2024-08-19T01:43:33.107 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "enabled", 2024-08-19T01:43:33.107 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-08-19T01:43:33.107 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:33.107 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-08-19T01:43:33.107 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "True", 2024-08-19T01:43:33.107 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:33.107 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:33.107 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:33.107 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-08-19T01:43:33.107 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:33.108 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:33.108 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:33.108 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:33.108 INFO:teuthology.orchestra.run.smithi083.stdout: "log_level": { 2024-08-19T01:43:33.108 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_level", 2024-08-19T01:43:33.108 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-08-19T01:43:33.108 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:33.108 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-08-19T01:43:33.108 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-08-19T01:43:33.108 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:33.108 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:33.108 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-08-19T01:43:33.108 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-08-19T01:43:33.108 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-08-19T01:43:33.109 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-08-19T01:43:33.109 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-08-19T01:43:33.109 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-08-19T01:43:33.109 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-08-19T01:43:33.109 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-08-19T01:43:33.109 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-08-19T01:43:33.109 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:33.109 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:33.109 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:33.109 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:33.109 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster": { 2024-08-19T01:43:33.109 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster", 2024-08-19T01:43:33.109 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-08-19T01:43:33.109 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:33.109 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-08-19T01:43:33.109 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-08-19T01:43:33.110 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:33.110 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:33.110 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:33.110 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-08-19T01:43:33.110 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:33.110 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:33.110 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:33.110 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:33.110 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster_level": { 2024-08-19T01:43:33.110 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster_level", 2024-08-19T01:43:33.110 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-08-19T01:43:33.110 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:33.110 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-08-19T01:43:33.110 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "info", 2024-08-19T01:43:33.111 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:33.111 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:33.111 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-08-19T01:43:33.111 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-08-19T01:43:33.111 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-08-19T01:43:33.111 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-08-19T01:43:33.111 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-08-19T01:43:33.111 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-08-19T01:43:33.111 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-08-19T01:43:33.111 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-08-19T01:43:33.111 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-08-19T01:43:33.111 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:33.111 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:33.111 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:33.112 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:33.112 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_file": { 2024-08-19T01:43:33.112 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_file", 2024-08-19T01:43:33.112 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-08-19T01:43:33.112 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:33.112 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-08-19T01:43:33.112 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-08-19T01:43:33.112 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:33.112 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:33.112 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:33.112 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-08-19T01:43:33.112 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:33.112 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:33.112 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:33.113 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:33.113 INFO:teuthology.orchestra.run.smithi083.stdout: "max_completed_events": { 2024-08-19T01:43:33.113 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "max_completed_events", 2024-08-19T01:43:33.113 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2024-08-19T01:43:33.113 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:33.113 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-08-19T01:43:33.113 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "50", 2024-08-19T01:43:33.113 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:33.113 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:33.113 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:33.113 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "number of past completed events to remember", 2024-08-19T01:43:33.113 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:33.113 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:33.113 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:33.114 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:33.114 INFO:teuthology.orchestra.run.smithi083.stdout: "sleep_interval": { 2024-08-19T01:43:33.114 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "sleep_interval", 2024-08-19T01:43:33.114 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "secs", 2024-08-19T01:43:33.114 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:33.114 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-08-19T01:43:33.114 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "5", 2024-08-19T01:43:33.114 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:33.114 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:33.114 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:33.114 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "how long the module is going to sleep", 2024-08-19T01:43:33.114 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:33.114 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:33.115 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:33.115 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-08-19T01:43:33.115 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-08-19T01:43:33.115 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:33.115 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-08-19T01:43:33.115 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "prometheus", 2024-08-19T01:43:33.115 INFO:teuthology.orchestra.run.smithi083.stdout: "can_run": true, 2024-08-19T01:43:33.115 INFO:teuthology.orchestra.run.smithi083.stdout: "error_string": "", 2024-08-19T01:43:33.115 INFO:teuthology.orchestra.run.smithi083.stdout: "module_options": { 2024-08-19T01:43:33.115 INFO:teuthology.orchestra.run.smithi083.stdout: "cache": { 2024-08-19T01:43:33.115 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "cache", 2024-08-19T01:43:33.115 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-08-19T01:43:33.115 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:33.115 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-08-19T01:43:33.115 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "True", 2024-08-19T01:43:33.116 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:33.116 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:33.116 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:33.116 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-08-19T01:43:33.116 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:33.116 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:33.116 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:33.116 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:33.116 INFO:teuthology.orchestra.run.smithi083.stdout: "exclude_perf_counters": { 2024-08-19T01:43:33.116 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "exclude_perf_counters", 2024-08-19T01:43:33.116 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-08-19T01:43:33.116 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:33.116 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-08-19T01:43:33.116 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "True", 2024-08-19T01:43:33.116 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:33.117 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:33.117 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:33.117 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-08-19T01:43:33.117 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-08-19T01:43:33.117 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:33.117 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:33.117 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:33.117 INFO:teuthology.orchestra.run.smithi083.stdout: "log_level": { 2024-08-19T01:43:33.117 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_level", 2024-08-19T01:43:33.117 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-08-19T01:43:33.117 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:33.117 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-08-19T01:43:33.117 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-08-19T01:43:33.117 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:33.117 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:33.118 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-08-19T01:43:33.118 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-08-19T01:43:33.118 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-08-19T01:43:33.118 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-08-19T01:43:33.118 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-08-19T01:43:33.118 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-08-19T01:43:33.118 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-08-19T01:43:33.118 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-08-19T01:43:33.118 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-08-19T01:43:33.118 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:33.118 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:33.118 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:33.118 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:33.118 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster": { 2024-08-19T01:43:33.118 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster", 2024-08-19T01:43:33.119 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-08-19T01:43:33.119 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:33.119 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-08-19T01:43:33.119 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-08-19T01:43:33.119 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:33.119 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:33.119 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:33.119 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-08-19T01:43:33.119 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:33.119 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:33.119 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:33.119 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:33.119 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster_level": { 2024-08-19T01:43:33.119 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster_level", 2024-08-19T01:43:33.119 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-08-19T01:43:33.120 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:33.120 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-08-19T01:43:33.120 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "info", 2024-08-19T01:43:33.120 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:33.120 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:33.120 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-08-19T01:43:33.120 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-08-19T01:43:33.120 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-08-19T01:43:33.120 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-08-19T01:43:33.120 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-08-19T01:43:33.120 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-08-19T01:43:33.120 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-08-19T01:43:33.120 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-08-19T01:43:33.120 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-08-19T01:43:33.120 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:33.121 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:33.121 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:33.121 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:33.121 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_file": { 2024-08-19T01:43:33.121 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_file", 2024-08-19T01:43:33.121 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-08-19T01:43:33.121 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:33.121 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-08-19T01:43:33.121 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-08-19T01:43:33.121 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:33.121 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:33.121 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:33.121 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-08-19T01:43:33.121 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:33.121 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:33.121 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:33.122 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:33.122 INFO:teuthology.orchestra.run.smithi083.stdout: "rbd_stats_pools": { 2024-08-19T01:43:33.122 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "rbd_stats_pools", 2024-08-19T01:43:33.122 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-08-19T01:43:33.122 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:33.122 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-08-19T01:43:33.122 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-08-19T01:43:33.122 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:33.122 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:33.122 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:33.122 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-08-19T01:43:33.122 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:33.122 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:33.122 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:33.123 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:33.123 INFO:teuthology.orchestra.run.smithi083.stdout: "rbd_stats_pools_refresh_interval": { 2024-08-19T01:43:33.123 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-08-19T01:43:33.123 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2024-08-19T01:43:33.123 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:33.123 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-08-19T01:43:33.123 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "300", 2024-08-19T01:43:33.123 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:33.123 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:33.123 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:33.123 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-08-19T01:43:33.123 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:33.123 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:33.123 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:33.123 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:33.124 INFO:teuthology.orchestra.run.smithi083.stdout: "scrape_interval": { 2024-08-19T01:43:33.124 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "scrape_interval", 2024-08-19T01:43:33.124 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "float", 2024-08-19T01:43:33.124 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:33.124 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-08-19T01:43:33.124 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "15.0", 2024-08-19T01:43:33.124 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:33.124 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:33.124 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:33.124 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-08-19T01:43:33.124 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:33.124 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:33.124 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:33.124 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:33.124 INFO:teuthology.orchestra.run.smithi083.stdout: "server_addr": { 2024-08-19T01:43:33.125 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "server_addr", 2024-08-19T01:43:33.125 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-08-19T01:43:33.125 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:33.125 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-08-19T01:43:33.125 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "::", 2024-08-19T01:43:33.125 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:33.125 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:33.125 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:33.125 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-08-19T01:43:33.125 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:33.125 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:33.125 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:33.125 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:33.126 INFO:teuthology.orchestra.run.smithi083.stdout: "server_port": { 2024-08-19T01:43:33.126 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "server_port", 2024-08-19T01:43:33.126 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2024-08-19T01:43:33.126 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:33.126 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-08-19T01:43:33.126 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "9283", 2024-08-19T01:43:33.126 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:33.126 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:33.126 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:33.126 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-08-19T01:43:33.126 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:33.126 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:33.127 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:33.127 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:33.127 INFO:teuthology.orchestra.run.smithi083.stdout: "stale_cache_strategy": { 2024-08-19T01:43:33.127 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "stale_cache_strategy", 2024-08-19T01:43:33.127 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-08-19T01:43:33.127 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:33.127 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-08-19T01:43:33.127 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "log", 2024-08-19T01:43:33.127 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:33.127 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:33.127 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:33.127 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-08-19T01:43:33.127 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:33.127 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:33.128 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:33.128 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:33.128 INFO:teuthology.orchestra.run.smithi083.stdout: "standby_behaviour": { 2024-08-19T01:43:33.128 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "standby_behaviour", 2024-08-19T01:43:33.128 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-08-19T01:43:33.128 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:33.128 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-08-19T01:43:33.128 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "default", 2024-08-19T01:43:33.128 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:33.128 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:33.128 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-08-19T01:43:33.128 INFO:teuthology.orchestra.run.smithi083.stdout: "default", 2024-08-19T01:43:33.128 INFO:teuthology.orchestra.run.smithi083.stdout: "error" 2024-08-19T01:43:33.128 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-08-19T01:43:33.129 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-08-19T01:43:33.129 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:33.129 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:33.129 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:33.129 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:33.129 INFO:teuthology.orchestra.run.smithi083.stdout: "standby_error_status_code": { 2024-08-19T01:43:33.129 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "standby_error_status_code", 2024-08-19T01:43:33.129 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2024-08-19T01:43:33.129 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:33.129 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-08-19T01:43:33.129 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "500", 2024-08-19T01:43:33.129 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "400", 2024-08-19T01:43:33.129 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "599", 2024-08-19T01:43:33.129 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:33.129 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-08-19T01:43:33.130 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:33.130 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:33.130 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:33.130 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-08-19T01:43:33.130 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-08-19T01:43:33.130 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:33.130 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-08-19T01:43:33.130 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "rbd_support", 2024-08-19T01:43:33.130 INFO:teuthology.orchestra.run.smithi083.stdout: "can_run": true, 2024-08-19T01:43:33.130 INFO:teuthology.orchestra.run.smithi083.stdout: "error_string": "", 2024-08-19T01:43:33.130 INFO:teuthology.orchestra.run.smithi083.stdout: "module_options": { 2024-08-19T01:43:33.130 INFO:teuthology.orchestra.run.smithi083.stdout: "log_level": { 2024-08-19T01:43:33.130 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_level", 2024-08-19T01:43:33.130 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-08-19T01:43:33.130 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:33.131 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-08-19T01:43:33.131 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-08-19T01:43:33.131 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:33.131 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:33.131 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-08-19T01:43:33.131 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-08-19T01:43:33.131 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-08-19T01:43:33.131 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-08-19T01:43:33.131 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-08-19T01:43:33.131 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-08-19T01:43:33.131 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-08-19T01:43:33.131 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-08-19T01:43:33.131 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-08-19T01:43:33.131 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:33.132 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:33.132 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:33.132 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:33.132 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster": { 2024-08-19T01:43:33.132 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster", 2024-08-19T01:43:33.132 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-08-19T01:43:33.132 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:33.132 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-08-19T01:43:33.132 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-08-19T01:43:33.132 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:33.132 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:33.132 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:33.132 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-08-19T01:43:33.132 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:33.133 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:33.133 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:33.133 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:33.133 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster_level": { 2024-08-19T01:43:33.133 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster_level", 2024-08-19T01:43:33.133 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-08-19T01:43:33.133 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:33.133 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-08-19T01:43:33.133 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "info", 2024-08-19T01:43:33.133 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:33.133 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:33.133 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-08-19T01:43:33.133 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-08-19T01:43:33.134 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-08-19T01:43:33.134 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-08-19T01:43:33.134 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-08-19T01:43:33.134 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-08-19T01:43:33.134 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-08-19T01:43:33.134 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-08-19T01:43:33.134 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-08-19T01:43:33.134 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:33.134 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:33.134 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:33.134 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:33.134 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_file": { 2024-08-19T01:43:33.134 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_file", 2024-08-19T01:43:33.135 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-08-19T01:43:33.135 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:33.135 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-08-19T01:43:33.135 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-08-19T01:43:33.135 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:33.135 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:33.135 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:33.135 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-08-19T01:43:33.135 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:33.135 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:33.135 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:33.135 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:33.135 INFO:teuthology.orchestra.run.smithi083.stdout: "max_concurrent_snap_create": { 2024-08-19T01:43:33.136 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "max_concurrent_snap_create", 2024-08-19T01:43:33.136 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2024-08-19T01:43:33.136 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:33.136 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-08-19T01:43:33.136 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "10", 2024-08-19T01:43:33.136 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:33.136 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:33.136 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:33.136 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-08-19T01:43:33.136 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:33.136 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:33.136 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:33.136 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:33.137 INFO:teuthology.orchestra.run.smithi083.stdout: "mirror_snapshot_schedule": { 2024-08-19T01:43:33.137 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "mirror_snapshot_schedule", 2024-08-19T01:43:33.137 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-08-19T01:43:33.137 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:33.137 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-08-19T01:43:33.137 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-08-19T01:43:33.137 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:33.137 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:33.137 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:33.137 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-08-19T01:43:33.137 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:33.137 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:33.137 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:33.138 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:33.138 INFO:teuthology.orchestra.run.smithi083.stdout: "trash_purge_schedule": { 2024-08-19T01:43:33.138 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "trash_purge_schedule", 2024-08-19T01:43:33.138 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-08-19T01:43:33.138 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:33.138 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-08-19T01:43:33.138 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-08-19T01:43:33.138 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:33.138 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:33.138 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:33.138 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-08-19T01:43:33.138 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:33.139 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:33.139 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:33.139 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-08-19T01:43:33.139 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-08-19T01:43:33.139 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:33.139 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-08-19T01:43:33.139 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "restful", 2024-08-19T01:43:33.139 INFO:teuthology.orchestra.run.smithi083.stdout: "can_run": true, 2024-08-19T01:43:33.139 INFO:teuthology.orchestra.run.smithi083.stdout: "error_string": "", 2024-08-19T01:43:33.139 INFO:teuthology.orchestra.run.smithi083.stdout: "module_options": { 2024-08-19T01:43:33.139 INFO:teuthology.orchestra.run.smithi083.stdout: "enable_auth": { 2024-08-19T01:43:33.139 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "enable_auth", 2024-08-19T01:43:33.139 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-08-19T01:43:33.139 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:33.140 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-08-19T01:43:33.140 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "True", 2024-08-19T01:43:33.140 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:33.140 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:33.140 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:33.140 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-08-19T01:43:33.140 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:33.140 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:33.140 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:33.140 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:33.140 INFO:teuthology.orchestra.run.smithi083.stdout: "key_file": { 2024-08-19T01:43:33.140 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "key_file", 2024-08-19T01:43:33.140 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-08-19T01:43:33.141 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:33.141 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-08-19T01:43:33.141 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-08-19T01:43:33.141 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:33.141 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:33.141 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:33.141 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-08-19T01:43:33.141 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:33.141 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:33.141 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:33.141 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:33.141 INFO:teuthology.orchestra.run.smithi083.stdout: "log_level": { 2024-08-19T01:43:33.141 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_level", 2024-08-19T01:43:33.142 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-08-19T01:43:33.142 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:33.142 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-08-19T01:43:33.142 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-08-19T01:43:33.142 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:33.142 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:33.142 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-08-19T01:43:33.142 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-08-19T01:43:33.142 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-08-19T01:43:33.142 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-08-19T01:43:33.142 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-08-19T01:43:33.142 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-08-19T01:43:33.143 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-08-19T01:43:33.143 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-08-19T01:43:33.143 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-08-19T01:43:33.143 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:33.143 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:33.143 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:33.143 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:33.143 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster": { 2024-08-19T01:43:33.143 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster", 2024-08-19T01:43:33.143 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-08-19T01:43:33.143 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:33.143 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-08-19T01:43:33.143 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-08-19T01:43:33.144 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:33.144 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:33.144 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:33.144 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-08-19T01:43:33.144 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:33.144 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:33.144 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:33.144 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:33.144 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster_level": { 2024-08-19T01:43:33.144 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster_level", 2024-08-19T01:43:33.144 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-08-19T01:43:33.144 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:33.144 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-08-19T01:43:33.145 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "info", 2024-08-19T01:43:33.145 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:33.145 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:33.145 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-08-19T01:43:33.145 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-08-19T01:43:33.145 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-08-19T01:43:33.145 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-08-19T01:43:33.145 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-08-19T01:43:33.145 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-08-19T01:43:33.145 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-08-19T01:43:33.145 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-08-19T01:43:33.145 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-08-19T01:43:33.145 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:33.145 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:33.146 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:33.146 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:33.146 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_file": { 2024-08-19T01:43:33.146 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_file", 2024-08-19T01:43:33.146 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-08-19T01:43:33.146 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:33.146 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-08-19T01:43:33.146 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-08-19T01:43:33.146 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:33.146 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:33.146 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:33.146 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-08-19T01:43:33.146 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:33.146 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:33.147 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:33.147 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:33.147 INFO:teuthology.orchestra.run.smithi083.stdout: "server_addr": { 2024-08-19T01:43:33.147 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "server_addr", 2024-08-19T01:43:33.147 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-08-19T01:43:33.147 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:33.147 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-08-19T01:43:33.147 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-08-19T01:43:33.147 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:33.147 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:33.147 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:33.147 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-08-19T01:43:33.147 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:33.148 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:33.148 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:33.148 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:33.148 INFO:teuthology.orchestra.run.smithi083.stdout: "server_port": { 2024-08-19T01:43:33.148 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "server_port", 2024-08-19T01:43:33.148 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-08-19T01:43:33.148 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:33.148 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-08-19T01:43:33.148 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-08-19T01:43:33.148 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:33.148 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:33.148 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:33.148 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-08-19T01:43:33.148 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:33.149 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:33.149 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:33.149 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-08-19T01:43:33.149 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-08-19T01:43:33.149 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:33.149 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-08-19T01:43:33.149 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "rgw", 2024-08-19T01:43:33.149 INFO:teuthology.orchestra.run.smithi083.stdout: "can_run": true, 2024-08-19T01:43:33.149 INFO:teuthology.orchestra.run.smithi083.stdout: "error_string": "", 2024-08-19T01:43:33.149 INFO:teuthology.orchestra.run.smithi083.stdout: "module_options": { 2024-08-19T01:43:33.149 INFO:teuthology.orchestra.run.smithi083.stdout: "log_level": { 2024-08-19T01:43:33.149 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_level", 2024-08-19T01:43:33.149 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-08-19T01:43:33.150 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:33.150 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-08-19T01:43:33.150 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-08-19T01:43:33.150 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:33.150 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:33.150 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-08-19T01:43:33.150 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-08-19T01:43:33.150 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-08-19T01:43:33.150 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-08-19T01:43:33.150 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-08-19T01:43:33.150 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-08-19T01:43:33.150 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-08-19T01:43:33.151 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-08-19T01:43:33.151 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-08-19T01:43:33.151 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:33.151 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:33.151 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:33.151 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:33.151 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster": { 2024-08-19T01:43:33.151 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster", 2024-08-19T01:43:33.151 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-08-19T01:43:33.151 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:33.151 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-08-19T01:43:33.151 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-08-19T01:43:33.151 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:33.151 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:33.152 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:33.152 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-08-19T01:43:33.152 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:33.152 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:33.152 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:33.152 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:33.152 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster_level": { 2024-08-19T01:43:33.152 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster_level", 2024-08-19T01:43:33.152 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-08-19T01:43:33.152 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:33.152 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-08-19T01:43:33.152 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "info", 2024-08-19T01:43:33.152 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:33.152 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:33.152 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-08-19T01:43:33.152 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-08-19T01:43:33.153 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-08-19T01:43:33.153 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-08-19T01:43:33.153 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-08-19T01:43:33.153 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-08-19T01:43:33.153 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-08-19T01:43:33.153 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-08-19T01:43:33.153 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-08-19T01:43:33.153 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:33.153 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:33.153 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:33.153 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:33.153 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_file": { 2024-08-19T01:43:33.153 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_file", 2024-08-19T01:43:33.153 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-08-19T01:43:33.154 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:33.154 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-08-19T01:43:33.154 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-08-19T01:43:33.154 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:33.154 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:33.154 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:33.154 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-08-19T01:43:33.154 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:33.154 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:33.154 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:33.154 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-08-19T01:43:33.154 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-08-19T01:43:33.154 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:33.155 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-08-19T01:43:33.155 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "rook", 2024-08-19T01:43:33.155 INFO:teuthology.orchestra.run.smithi083.stdout: "can_run": true, 2024-08-19T01:43:33.155 INFO:teuthology.orchestra.run.smithi083.stdout: "error_string": "", 2024-08-19T01:43:33.155 INFO:teuthology.orchestra.run.smithi083.stdout: "module_options": { 2024-08-19T01:43:33.155 INFO:teuthology.orchestra.run.smithi083.stdout: "log_level": { 2024-08-19T01:43:33.155 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_level", 2024-08-19T01:43:33.155 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-08-19T01:43:33.155 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:33.155 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-08-19T01:43:33.155 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-08-19T01:43:33.155 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:33.155 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:33.155 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-08-19T01:43:33.156 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-08-19T01:43:33.156 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-08-19T01:43:33.156 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-08-19T01:43:33.156 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-08-19T01:43:33.156 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-08-19T01:43:33.156 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-08-19T01:43:33.156 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-08-19T01:43:33.156 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-08-19T01:43:33.156 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:33.156 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:33.156 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:33.156 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:33.156 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster": { 2024-08-19T01:43:33.157 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster", 2024-08-19T01:43:33.157 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-08-19T01:43:33.157 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:33.157 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-08-19T01:43:33.157 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-08-19T01:43:33.157 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:33.157 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:33.157 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:33.157 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-08-19T01:43:33.157 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:33.157 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:33.157 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:33.157 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:33.157 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster_level": { 2024-08-19T01:43:33.157 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster_level", 2024-08-19T01:43:33.158 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-08-19T01:43:33.158 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:33.158 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-08-19T01:43:33.158 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "info", 2024-08-19T01:43:33.158 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:33.158 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:33.158 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-08-19T01:43:33.158 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-08-19T01:43:33.158 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-08-19T01:43:33.158 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-08-19T01:43:33.158 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-08-19T01:43:33.158 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-08-19T01:43:33.158 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-08-19T01:43:33.158 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-08-19T01:43:33.159 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-08-19T01:43:33.159 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:33.159 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:33.159 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:33.159 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:33.159 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_file": { 2024-08-19T01:43:33.159 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_file", 2024-08-19T01:43:33.159 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-08-19T01:43:33.159 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:33.159 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-08-19T01:43:33.159 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-08-19T01:43:33.159 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:33.159 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:33.159 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:33.159 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-08-19T01:43:33.159 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:33.160 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:33.160 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:33.160 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:33.160 INFO:teuthology.orchestra.run.smithi083.stdout: "storage_class": { 2024-08-19T01:43:33.160 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "storage_class", 2024-08-19T01:43:33.160 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-08-19T01:43:33.160 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:33.160 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-08-19T01:43:33.160 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "local", 2024-08-19T01:43:33.160 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:33.160 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:33.160 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:33.160 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-08-19T01:43:33.160 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:33.160 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:33.161 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:33.161 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-08-19T01:43:33.161 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-08-19T01:43:33.161 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:33.161 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-08-19T01:43:33.161 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "selftest", 2024-08-19T01:43:33.161 INFO:teuthology.orchestra.run.smithi083.stdout: "can_run": true, 2024-08-19T01:43:33.161 INFO:teuthology.orchestra.run.smithi083.stdout: "error_string": "", 2024-08-19T01:43:33.161 INFO:teuthology.orchestra.run.smithi083.stdout: "module_options": { 2024-08-19T01:43:33.161 INFO:teuthology.orchestra.run.smithi083.stdout: "log_level": { 2024-08-19T01:43:33.161 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_level", 2024-08-19T01:43:33.161 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-08-19T01:43:33.161 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:33.161 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-08-19T01:43:33.161 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-08-19T01:43:33.161 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:33.162 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:33.162 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-08-19T01:43:33.162 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-08-19T01:43:33.162 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-08-19T01:43:33.162 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-08-19T01:43:33.162 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-08-19T01:43:33.162 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-08-19T01:43:33.162 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-08-19T01:43:33.162 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-08-19T01:43:33.162 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-08-19T01:43:33.162 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:33.162 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:33.162 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:33.162 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:33.162 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster": { 2024-08-19T01:43:33.163 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster", 2024-08-19T01:43:33.163 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-08-19T01:43:33.163 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:33.163 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-08-19T01:43:33.163 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-08-19T01:43:33.163 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:33.163 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:33.163 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:33.163 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-08-19T01:43:33.163 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:33.163 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:33.163 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:33.163 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:33.163 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster_level": { 2024-08-19T01:43:33.163 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster_level", 2024-08-19T01:43:33.164 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-08-19T01:43:33.164 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:33.164 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-08-19T01:43:33.164 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "info", 2024-08-19T01:43:33.164 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:33.164 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:33.164 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-08-19T01:43:33.164 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-08-19T01:43:33.164 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-08-19T01:43:33.164 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-08-19T01:43:33.164 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-08-19T01:43:33.164 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-08-19T01:43:33.164 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-08-19T01:43:33.164 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-08-19T01:43:33.164 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-08-19T01:43:33.165 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:33.165 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:33.165 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:33.165 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:33.165 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_file": { 2024-08-19T01:43:33.165 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_file", 2024-08-19T01:43:33.165 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-08-19T01:43:33.165 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:33.165 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-08-19T01:43:33.165 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-08-19T01:43:33.165 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:33.165 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:33.165 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:33.165 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-08-19T01:43:33.166 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:33.166 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:33.166 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:33.166 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:33.166 INFO:teuthology.orchestra.run.smithi083.stdout: "roption1": { 2024-08-19T01:43:33.166 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "roption1", 2024-08-19T01:43:33.166 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-08-19T01:43:33.166 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:33.166 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-08-19T01:43:33.166 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-08-19T01:43:33.166 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:33.166 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:33.166 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:33.166 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-08-19T01:43:33.167 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:33.167 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:33.167 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:33.167 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:33.167 INFO:teuthology.orchestra.run.smithi083.stdout: "roption2": { 2024-08-19T01:43:33.167 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "roption2", 2024-08-19T01:43:33.167 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-08-19T01:43:33.167 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:33.167 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-08-19T01:43:33.167 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "xyz", 2024-08-19T01:43:33.167 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:33.167 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:33.167 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:33.167 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-08-19T01:43:33.167 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:33.167 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:33.168 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:33.168 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:33.168 INFO:teuthology.orchestra.run.smithi083.stdout: "rwoption1": { 2024-08-19T01:43:33.168 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "rwoption1", 2024-08-19T01:43:33.168 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-08-19T01:43:33.168 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:33.168 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-08-19T01:43:33.168 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-08-19T01:43:33.168 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:33.168 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:33.168 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:33.168 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-08-19T01:43:33.168 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:33.168 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:33.168 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:33.169 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:33.169 INFO:teuthology.orchestra.run.smithi083.stdout: "rwoption2": { 2024-08-19T01:43:33.169 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "rwoption2", 2024-08-19T01:43:33.169 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2024-08-19T01:43:33.169 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:33.169 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-08-19T01:43:33.169 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-08-19T01:43:33.169 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:33.169 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:33.169 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:33.169 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-08-19T01:43:33.169 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:33.169 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:33.169 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:33.169 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:33.170 INFO:teuthology.orchestra.run.smithi083.stdout: "rwoption3": { 2024-08-19T01:43:33.170 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "rwoption3", 2024-08-19T01:43:33.170 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "float", 2024-08-19T01:43:33.170 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:33.170 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-08-19T01:43:33.170 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-08-19T01:43:33.170 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:33.170 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:33.170 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:33.170 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-08-19T01:43:33.170 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:33.170 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:33.170 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:33.170 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:33.171 INFO:teuthology.orchestra.run.smithi083.stdout: "rwoption4": { 2024-08-19T01:43:33.171 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "rwoption4", 2024-08-19T01:43:33.171 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-08-19T01:43:33.171 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:33.171 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-08-19T01:43:33.171 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-08-19T01:43:33.171 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:33.171 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:33.171 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:33.171 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-08-19T01:43:33.171 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:33.171 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:33.171 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:33.171 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:33.171 INFO:teuthology.orchestra.run.smithi083.stdout: "rwoption5": { 2024-08-19T01:43:33.171 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "rwoption5", 2024-08-19T01:43:33.172 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-08-19T01:43:33.172 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:33.172 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-08-19T01:43:33.172 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-08-19T01:43:33.172 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:33.172 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:33.172 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:33.172 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-08-19T01:43:33.172 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:33.172 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:33.172 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:33.172 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:33.172 INFO:teuthology.orchestra.run.smithi083.stdout: "rwoption6": { 2024-08-19T01:43:33.173 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "rwoption6", 2024-08-19T01:43:33.173 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-08-19T01:43:33.173 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:33.173 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-08-19T01:43:33.173 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "True", 2024-08-19T01:43:33.173 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:33.173 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:33.173 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:33.173 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-08-19T01:43:33.173 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:33.173 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:33.173 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:33.173 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:33.173 INFO:teuthology.orchestra.run.smithi083.stdout: "rwoption7": { 2024-08-19T01:43:33.174 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "rwoption7", 2024-08-19T01:43:33.174 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2024-08-19T01:43:33.174 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:33.174 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-08-19T01:43:33.174 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-08-19T01:43:33.174 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "1", 2024-08-19T01:43:33.174 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "42", 2024-08-19T01:43:33.174 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:33.174 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-08-19T01:43:33.174 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:33.174 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:33.174 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:33.174 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:33.175 INFO:teuthology.orchestra.run.smithi083.stdout: "testkey": { 2024-08-19T01:43:33.175 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "testkey", 2024-08-19T01:43:33.175 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-08-19T01:43:33.175 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:33.175 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-08-19T01:43:33.175 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-08-19T01:43:33.175 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:33.175 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:33.175 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:33.175 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-08-19T01:43:33.175 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:33.175 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:33.175 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:33.176 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:33.176 INFO:teuthology.orchestra.run.smithi083.stdout: "testlkey": { 2024-08-19T01:43:33.176 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "testlkey", 2024-08-19T01:43:33.176 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-08-19T01:43:33.176 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:33.176 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-08-19T01:43:33.176 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-08-19T01:43:33.176 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:33.176 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:33.176 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:33.176 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-08-19T01:43:33.176 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:33.176 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:33.176 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:33.177 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:33.177 INFO:teuthology.orchestra.run.smithi083.stdout: "testnewline": { 2024-08-19T01:43:33.177 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "testnewline", 2024-08-19T01:43:33.177 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-08-19T01:43:33.177 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:33.177 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-08-19T01:43:33.177 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-08-19T01:43:33.177 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:33.177 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:33.177 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:33.177 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-08-19T01:43:33.177 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:33.177 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:33.178 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:33.178 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-08-19T01:43:33.178 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-08-19T01:43:33.178 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:33.178 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-08-19T01:43:33.178 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "snap_schedule", 2024-08-19T01:43:33.178 INFO:teuthology.orchestra.run.smithi083.stdout: "can_run": true, 2024-08-19T01:43:33.178 INFO:teuthology.orchestra.run.smithi083.stdout: "error_string": "", 2024-08-19T01:43:33.178 INFO:teuthology.orchestra.run.smithi083.stdout: "module_options": { 2024-08-19T01:43:33.178 INFO:teuthology.orchestra.run.smithi083.stdout: "allow_m_granularity": { 2024-08-19T01:43:33.178 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "allow_m_granularity", 2024-08-19T01:43:33.178 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-08-19T01:43:33.178 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:33.179 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-08-19T01:43:33.179 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-08-19T01:43:33.179 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:33.179 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:33.179 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:33.179 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "allow minute scheduled snapshots", 2024-08-19T01:43:33.179 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:33.179 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:33.179 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:33.179 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:33.179 INFO:teuthology.orchestra.run.smithi083.stdout: "dump_on_update": { 2024-08-19T01:43:33.179 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "dump_on_update", 2024-08-19T01:43:33.179 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-08-19T01:43:33.179 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:33.180 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-08-19T01:43:33.180 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-08-19T01:43:33.180 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:33.180 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:33.180 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:33.180 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "dump database to debug log on update", 2024-08-19T01:43:33.180 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:33.180 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:33.180 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:33.180 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:33.180 INFO:teuthology.orchestra.run.smithi083.stdout: "log_level": { 2024-08-19T01:43:33.180 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_level", 2024-08-19T01:43:33.180 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-08-19T01:43:33.181 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:33.181 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-08-19T01:43:33.181 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-08-19T01:43:33.181 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:33.181 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:33.181 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-08-19T01:43:33.181 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-08-19T01:43:33.181 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-08-19T01:43:33.181 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-08-19T01:43:33.181 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-08-19T01:43:33.181 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-08-19T01:43:33.181 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-08-19T01:43:33.181 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-08-19T01:43:33.181 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-08-19T01:43:33.181 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:33.182 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:33.182 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:33.182 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:33.182 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster": { 2024-08-19T01:43:33.182 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster", 2024-08-19T01:43:33.182 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-08-19T01:43:33.182 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:33.182 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-08-19T01:43:33.182 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-08-19T01:43:33.182 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:33.182 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:33.182 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:33.182 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-08-19T01:43:33.183 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:33.183 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:33.183 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:33.183 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:33.183 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster_level": { 2024-08-19T01:43:33.183 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster_level", 2024-08-19T01:43:33.183 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-08-19T01:43:33.183 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:33.183 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-08-19T01:43:33.183 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "info", 2024-08-19T01:43:33.183 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:33.183 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:33.183 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-08-19T01:43:33.183 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-08-19T01:43:33.184 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-08-19T01:43:33.184 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-08-19T01:43:33.184 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-08-19T01:43:33.184 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-08-19T01:43:33.184 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-08-19T01:43:33.184 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-08-19T01:43:33.184 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-08-19T01:43:33.184 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:33.184 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:33.184 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:33.184 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:33.184 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_file": { 2024-08-19T01:43:33.184 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_file", 2024-08-19T01:43:33.184 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-08-19T01:43:33.185 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:33.185 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-08-19T01:43:33.185 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-08-19T01:43:33.185 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:33.185 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:33.185 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:33.185 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-08-19T01:43:33.185 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:33.185 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:33.185 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:33.185 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-08-19T01:43:33.185 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-08-19T01:43:33.185 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:33.185 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-08-19T01:43:33.185 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "stats", 2024-08-19T01:43:33.185 INFO:teuthology.orchestra.run.smithi083.stdout: "can_run": true, 2024-08-19T01:43:33.186 INFO:teuthology.orchestra.run.smithi083.stdout: "error_string": "", 2024-08-19T01:43:33.186 INFO:teuthology.orchestra.run.smithi083.stdout: "module_options": { 2024-08-19T01:43:33.186 INFO:teuthology.orchestra.run.smithi083.stdout: "log_level": { 2024-08-19T01:43:33.186 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_level", 2024-08-19T01:43:33.186 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-08-19T01:43:33.186 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:33.186 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-08-19T01:43:33.186 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-08-19T01:43:33.186 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:33.186 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:33.186 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-08-19T01:43:33.186 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-08-19T01:43:33.186 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-08-19T01:43:33.186 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-08-19T01:43:33.186 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-08-19T01:43:33.187 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-08-19T01:43:33.187 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-08-19T01:43:33.187 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-08-19T01:43:33.187 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-08-19T01:43:33.187 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:33.187 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:33.187 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:33.187 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:33.187 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster": { 2024-08-19T01:43:33.187 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster", 2024-08-19T01:43:33.187 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-08-19T01:43:33.187 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:33.187 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-08-19T01:43:33.187 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-08-19T01:43:33.188 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:33.188 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:33.188 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:33.188 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-08-19T01:43:33.188 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:33.188 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:33.188 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:33.188 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:33.188 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster_level": { 2024-08-19T01:43:33.188 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster_level", 2024-08-19T01:43:33.188 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-08-19T01:43:33.188 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:33.188 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-08-19T01:43:33.188 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "info", 2024-08-19T01:43:33.189 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:33.189 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:33.189 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-08-19T01:43:33.189 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-08-19T01:43:33.189 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-08-19T01:43:33.189 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-08-19T01:43:33.189 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-08-19T01:43:33.189 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-08-19T01:43:33.189 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-08-19T01:43:33.189 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-08-19T01:43:33.189 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-08-19T01:43:33.189 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:33.189 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:33.189 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:33.190 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:33.190 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_file": { 2024-08-19T01:43:33.190 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_file", 2024-08-19T01:43:33.190 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-08-19T01:43:33.190 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:33.190 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-08-19T01:43:33.190 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-08-19T01:43:33.190 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:33.190 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:33.190 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:33.190 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-08-19T01:43:33.190 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:33.191 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:33.191 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:33.191 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-08-19T01:43:33.191 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-08-19T01:43:33.191 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:33.191 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-08-19T01:43:33.191 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "status", 2024-08-19T01:43:33.191 INFO:teuthology.orchestra.run.smithi083.stdout: "can_run": true, 2024-08-19T01:43:33.191 INFO:teuthology.orchestra.run.smithi083.stdout: "error_string": "", 2024-08-19T01:43:33.191 INFO:teuthology.orchestra.run.smithi083.stdout: "module_options": { 2024-08-19T01:43:33.191 INFO:teuthology.orchestra.run.smithi083.stdout: "log_level": { 2024-08-19T01:43:33.191 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_level", 2024-08-19T01:43:33.191 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-08-19T01:43:33.192 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:33.192 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-08-19T01:43:33.192 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-08-19T01:43:33.192 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:33.192 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:33.192 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-08-19T01:43:33.192 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-08-19T01:43:33.192 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-08-19T01:43:33.192 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-08-19T01:43:33.192 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-08-19T01:43:33.192 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-08-19T01:43:33.192 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-08-19T01:43:33.192 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-08-19T01:43:33.192 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-08-19T01:43:33.192 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:33.193 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:33.193 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:33.193 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:33.193 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster": { 2024-08-19T01:43:33.193 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster", 2024-08-19T01:43:33.193 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-08-19T01:43:33.193 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:33.193 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-08-19T01:43:33.193 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-08-19T01:43:33.193 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:33.193 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:33.193 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:33.193 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-08-19T01:43:33.193 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:33.193 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:33.194 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:33.194 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:33.194 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster_level": { 2024-08-19T01:43:33.194 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster_level", 2024-08-19T01:43:33.194 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-08-19T01:43:33.194 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:33.194 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-08-19T01:43:33.194 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "info", 2024-08-19T01:43:33.194 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:33.194 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:33.194 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-08-19T01:43:33.194 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-08-19T01:43:33.194 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-08-19T01:43:33.194 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-08-19T01:43:33.195 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-08-19T01:43:33.195 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-08-19T01:43:33.195 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-08-19T01:43:33.195 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-08-19T01:43:33.195 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-08-19T01:43:33.195 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:33.195 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:33.195 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:33.195 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:33.195 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_file": { 2024-08-19T01:43:33.195 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_file", 2024-08-19T01:43:33.195 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-08-19T01:43:33.195 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:33.195 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-08-19T01:43:33.195 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-08-19T01:43:33.196 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:33.196 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:33.196 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:33.196 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-08-19T01:43:33.196 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:33.196 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:33.196 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:33.196 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-08-19T01:43:33.196 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-08-19T01:43:33.196 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:33.196 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-08-19T01:43:33.196 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "telegraf", 2024-08-19T01:43:33.196 INFO:teuthology.orchestra.run.smithi083.stdout: "can_run": true, 2024-08-19T01:43:33.196 INFO:teuthology.orchestra.run.smithi083.stdout: "error_string": "", 2024-08-19T01:43:33.197 INFO:teuthology.orchestra.run.smithi083.stdout: "module_options": { 2024-08-19T01:43:33.197 INFO:teuthology.orchestra.run.smithi083.stdout: "address": { 2024-08-19T01:43:33.197 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "address", 2024-08-19T01:43:33.197 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-08-19T01:43:33.197 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:33.197 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-08-19T01:43:33.197 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-08-19T01:43:33.197 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:33.197 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:33.197 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:33.197 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-08-19T01:43:33.197 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:33.197 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:33.197 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:33.198 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:33.198 INFO:teuthology.orchestra.run.smithi083.stdout: "interval": { 2024-08-19T01:43:33.198 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "interval", 2024-08-19T01:43:33.198 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "secs", 2024-08-19T01:43:33.198 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:33.198 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-08-19T01:43:33.198 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "15", 2024-08-19T01:43:33.198 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:33.198 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:33.198 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:33.198 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-08-19T01:43:33.198 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:33.198 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:33.198 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:33.199 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:33.199 INFO:teuthology.orchestra.run.smithi083.stdout: "log_level": { 2024-08-19T01:43:33.199 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_level", 2024-08-19T01:43:33.199 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-08-19T01:43:33.199 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:33.199 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-08-19T01:43:33.199 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-08-19T01:43:33.199 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:33.199 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:33.199 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-08-19T01:43:33.199 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-08-19T01:43:33.199 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-08-19T01:43:33.199 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-08-19T01:43:33.199 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-08-19T01:43:33.199 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-08-19T01:43:33.200 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-08-19T01:43:33.200 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-08-19T01:43:33.200 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-08-19T01:43:33.200 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:33.200 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:33.200 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:33.200 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:33.200 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster": { 2024-08-19T01:43:33.200 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster", 2024-08-19T01:43:33.200 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-08-19T01:43:33.200 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:33.200 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-08-19T01:43:33.200 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-08-19T01:43:33.200 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:33.201 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:33.201 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:33.201 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-08-19T01:43:33.201 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:33.201 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:33.201 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:33.201 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:33.201 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster_level": { 2024-08-19T01:43:33.201 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster_level", 2024-08-19T01:43:33.201 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-08-19T01:43:33.201 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:33.201 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-08-19T01:43:33.201 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "info", 2024-08-19T01:43:33.201 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:33.202 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:33.202 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-08-19T01:43:33.202 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-08-19T01:43:33.202 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-08-19T01:43:33.202 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-08-19T01:43:33.202 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-08-19T01:43:33.202 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-08-19T01:43:33.202 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-08-19T01:43:33.202 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-08-19T01:43:33.202 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-08-19T01:43:33.202 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:33.202 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:33.202 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:33.203 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:33.203 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_file": { 2024-08-19T01:43:33.203 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_file", 2024-08-19T01:43:33.203 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-08-19T01:43:33.203 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:33.203 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-08-19T01:43:33.203 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-08-19T01:43:33.203 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:33.203 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:33.203 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:33.203 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-08-19T01:43:33.203 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:33.203 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:33.204 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:33.204 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-08-19T01:43:33.204 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-08-19T01:43:33.204 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:33.204 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-08-19T01:43:33.204 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "telemetry", 2024-08-19T01:43:33.204 INFO:teuthology.orchestra.run.smithi083.stdout: "can_run": true, 2024-08-19T01:43:33.204 INFO:teuthology.orchestra.run.smithi083.stdout: "error_string": "", 2024-08-19T01:43:33.204 INFO:teuthology.orchestra.run.smithi083.stdout: "module_options": { 2024-08-19T01:43:33.204 INFO:teuthology.orchestra.run.smithi083.stdout: "channel_basic": { 2024-08-19T01:43:33.204 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "channel_basic", 2024-08-19T01:43:33.204 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-08-19T01:43:33.204 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:33.205 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-08-19T01:43:33.205 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "True", 2024-08-19T01:43:33.205 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:33.205 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:33.205 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:33.205 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Share basic cluster information (size, version)", 2024-08-19T01:43:33.205 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:33.205 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:33.205 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:33.205 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:33.205 INFO:teuthology.orchestra.run.smithi083.stdout: "channel_crash": { 2024-08-19T01:43:33.205 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "channel_crash", 2024-08-19T01:43:33.205 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-08-19T01:43:33.206 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:33.206 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-08-19T01:43:33.206 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "True", 2024-08-19T01:43:33.206 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:33.206 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:33.206 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:33.206 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-08-19T01:43:33.206 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:33.206 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:33.206 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:33.206 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:33.206 INFO:teuthology.orchestra.run.smithi083.stdout: "channel_device": { 2024-08-19T01:43:33.206 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "channel_device", 2024-08-19T01:43:33.206 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-08-19T01:43:33.207 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:33.207 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-08-19T01:43:33.207 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "True", 2024-08-19T01:43:33.207 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:33.207 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:33.207 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:33.207 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-08-19T01:43:33.207 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:33.207 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:33.207 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:33.207 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:33.207 INFO:teuthology.orchestra.run.smithi083.stdout: "channel_ident": { 2024-08-19T01:43:33.207 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "channel_ident", 2024-08-19T01:43:33.207 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-08-19T01:43:33.207 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:33.208 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-08-19T01:43:33.208 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-08-19T01:43:33.208 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:33.208 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:33.208 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:33.208 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-08-19T01:43:33.208 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:33.208 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:33.208 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:33.208 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:33.208 INFO:teuthology.orchestra.run.smithi083.stdout: "channel_perf": { 2024-08-19T01:43:33.208 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "channel_perf", 2024-08-19T01:43:33.208 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-08-19T01:43:33.208 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:33.208 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-08-19T01:43:33.208 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-08-19T01:43:33.209 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:33.209 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:33.209 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:33.209 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Share various performance metrics of a cluster", 2024-08-19T01:43:33.209 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:33.209 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:33.209 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:33.209 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:33.209 INFO:teuthology.orchestra.run.smithi083.stdout: "contact": { 2024-08-19T01:43:33.209 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "contact", 2024-08-19T01:43:33.209 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-08-19T01:43:33.209 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:33.209 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-08-19T01:43:33.209 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-08-19T01:43:33.210 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:33.210 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:33.210 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:33.210 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-08-19T01:43:33.210 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:33.210 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:33.210 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:33.210 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:33.210 INFO:teuthology.orchestra.run.smithi083.stdout: "description": { 2024-08-19T01:43:33.210 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "description", 2024-08-19T01:43:33.210 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-08-19T01:43:33.210 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:33.210 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-08-19T01:43:33.211 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-08-19T01:43:33.211 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:33.211 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:33.211 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:33.211 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-08-19T01:43:33.211 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:33.211 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:33.211 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:33.211 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:33.211 INFO:teuthology.orchestra.run.smithi083.stdout: "device_url": { 2024-08-19T01:43:33.211 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "device_url", 2024-08-19T01:43:33.211 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-08-19T01:43:33.211 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:33.211 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-08-19T01:43:33.211 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-08-19T01:43:33.212 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:33.212 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:33.212 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:33.212 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-08-19T01:43:33.212 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:33.212 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:33.212 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:33.212 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:33.212 INFO:teuthology.orchestra.run.smithi083.stdout: "enabled": { 2024-08-19T01:43:33.212 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "enabled", 2024-08-19T01:43:33.212 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-08-19T01:43:33.212 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:33.212 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-08-19T01:43:33.213 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-08-19T01:43:33.213 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:33.213 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:33.213 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:33.213 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-08-19T01:43:33.213 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:33.213 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:33.213 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:33.213 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:33.213 INFO:teuthology.orchestra.run.smithi083.stdout: "interval": { 2024-08-19T01:43:33.213 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "interval", 2024-08-19T01:43:33.213 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2024-08-19T01:43:33.213 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:33.214 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-08-19T01:43:33.214 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "24", 2024-08-19T01:43:33.214 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "8", 2024-08-19T01:43:33.214 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:33.214 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:33.214 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-08-19T01:43:33.214 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:33.214 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:33.214 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:33.214 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:33.214 INFO:teuthology.orchestra.run.smithi083.stdout: "last_opt_revision": { 2024-08-19T01:43:33.214 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "last_opt_revision", 2024-08-19T01:43:33.214 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2024-08-19T01:43:33.215 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:33.215 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-08-19T01:43:33.215 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "1", 2024-08-19T01:43:33.215 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:33.215 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:33.215 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:33.215 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-08-19T01:43:33.215 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:33.215 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:33.215 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:33.215 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:33.215 INFO:teuthology.orchestra.run.smithi083.stdout: "leaderboard": { 2024-08-19T01:43:33.215 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "leaderboard", 2024-08-19T01:43:33.215 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-08-19T01:43:33.215 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:33.215 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-08-19T01:43:33.216 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-08-19T01:43:33.216 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:33.216 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:33.216 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:33.216 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-08-19T01:43:33.216 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:33.216 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:33.216 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:33.216 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:33.216 INFO:teuthology.orchestra.run.smithi083.stdout: "leaderboard_description": { 2024-08-19T01:43:33.216 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "leaderboard_description", 2024-08-19T01:43:33.216 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-08-19T01:43:33.216 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:33.216 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-08-19T01:43:33.217 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-08-19T01:43:33.217 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:33.217 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:33.217 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:33.217 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-08-19T01:43:33.217 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:33.217 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:33.217 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:33.217 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:33.217 INFO:teuthology.orchestra.run.smithi083.stdout: "log_level": { 2024-08-19T01:43:33.217 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_level", 2024-08-19T01:43:33.217 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-08-19T01:43:33.217 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:33.217 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-08-19T01:43:33.218 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-08-19T01:43:33.218 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:33.218 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:33.218 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-08-19T01:43:33.218 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-08-19T01:43:33.218 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-08-19T01:43:33.218 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-08-19T01:43:33.218 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-08-19T01:43:33.218 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-08-19T01:43:33.218 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-08-19T01:43:33.218 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-08-19T01:43:33.218 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-08-19T01:43:33.218 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:33.218 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:33.218 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:33.219 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:33.219 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster": { 2024-08-19T01:43:33.219 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster", 2024-08-19T01:43:33.219 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-08-19T01:43:33.219 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:33.219 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-08-19T01:43:33.219 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-08-19T01:43:33.219 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:33.219 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:33.219 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:33.219 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-08-19T01:43:33.219 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:33.219 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:33.219 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:33.220 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:33.220 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster_level": { 2024-08-19T01:43:33.220 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster_level", 2024-08-19T01:43:33.220 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-08-19T01:43:33.220 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:33.220 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-08-19T01:43:33.220 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "info", 2024-08-19T01:43:33.220 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:33.220 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:33.220 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-08-19T01:43:33.220 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-08-19T01:43:33.220 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-08-19T01:43:33.220 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-08-19T01:43:33.220 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-08-19T01:43:33.221 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-08-19T01:43:33.221 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-08-19T01:43:33.221 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-08-19T01:43:33.221 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-08-19T01:43:33.221 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:33.221 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:33.221 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:33.221 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:33.221 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_file": { 2024-08-19T01:43:33.221 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_file", 2024-08-19T01:43:33.221 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-08-19T01:43:33.221 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:33.221 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-08-19T01:43:33.221 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-08-19T01:43:33.222 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:33.222 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:33.222 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:33.222 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-08-19T01:43:33.222 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:33.222 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:33.222 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:33.222 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:33.222 INFO:teuthology.orchestra.run.smithi083.stdout: "organization": { 2024-08-19T01:43:33.222 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "organization", 2024-08-19T01:43:33.222 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-08-19T01:43:33.222 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:33.223 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-08-19T01:43:33.223 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-08-19T01:43:33.223 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:33.223 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:33.223 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:33.223 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-08-19T01:43:33.223 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:33.223 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:33.223 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:33.223 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:33.223 INFO:teuthology.orchestra.run.smithi083.stdout: "proxy": { 2024-08-19T01:43:33.223 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "proxy", 2024-08-19T01:43:33.223 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-08-19T01:43:33.224 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:33.224 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-08-19T01:43:33.224 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-08-19T01:43:33.224 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:33.224 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:33.224 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:33.224 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-08-19T01:43:33.224 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:33.224 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:33.224 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:33.224 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:33.224 INFO:teuthology.orchestra.run.smithi083.stdout: "url": { 2024-08-19T01:43:33.224 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "url", 2024-08-19T01:43:33.224 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-08-19T01:43:33.225 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:33.225 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-08-19T01:43:33.225 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-08-19T01:43:33.225 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:33.225 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:33.225 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:33.225 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-08-19T01:43:33.225 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:33.225 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:33.225 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:33.225 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-08-19T01:43:33.225 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-08-19T01:43:33.225 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:33.225 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-08-19T01:43:33.226 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "test_orchestrator", 2024-08-19T01:43:33.226 INFO:teuthology.orchestra.run.smithi083.stdout: "can_run": true, 2024-08-19T01:43:33.226 INFO:teuthology.orchestra.run.smithi083.stdout: "error_string": "", 2024-08-19T01:43:33.226 INFO:teuthology.orchestra.run.smithi083.stdout: "module_options": { 2024-08-19T01:43:33.226 INFO:teuthology.orchestra.run.smithi083.stdout: "log_level": { 2024-08-19T01:43:33.226 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_level", 2024-08-19T01:43:33.226 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-08-19T01:43:33.226 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:33.226 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-08-19T01:43:33.226 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-08-19T01:43:33.226 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:33.226 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:33.226 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-08-19T01:43:33.226 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-08-19T01:43:33.227 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-08-19T01:43:33.227 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-08-19T01:43:33.227 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-08-19T01:43:33.227 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-08-19T01:43:33.227 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-08-19T01:43:33.227 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-08-19T01:43:33.227 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-08-19T01:43:33.227 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:33.227 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:33.227 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:33.227 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:33.227 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster": { 2024-08-19T01:43:33.227 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster", 2024-08-19T01:43:33.228 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-08-19T01:43:33.228 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:33.228 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-08-19T01:43:33.228 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-08-19T01:43:33.228 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:33.228 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:33.228 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:33.228 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-08-19T01:43:33.228 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:33.228 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:33.228 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:33.228 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:33.228 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster_level": { 2024-08-19T01:43:33.228 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster_level", 2024-08-19T01:43:33.229 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-08-19T01:43:33.229 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:33.229 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-08-19T01:43:33.229 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "info", 2024-08-19T01:43:33.229 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:33.229 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:33.229 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-08-19T01:43:33.229 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-08-19T01:43:33.229 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-08-19T01:43:33.229 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-08-19T01:43:33.229 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-08-19T01:43:33.229 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-08-19T01:43:33.229 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-08-19T01:43:33.229 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-08-19T01:43:33.230 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-08-19T01:43:33.230 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:33.230 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:33.230 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:33.230 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:33.230 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_file": { 2024-08-19T01:43:33.230 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_file", 2024-08-19T01:43:33.230 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-08-19T01:43:33.230 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:33.230 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-08-19T01:43:33.230 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-08-19T01:43:33.230 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:33.230 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:33.230 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:33.231 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-08-19T01:43:33.231 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:33.231 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:33.231 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:33.231 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-08-19T01:43:33.231 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-08-19T01:43:33.231 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:33.231 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-08-19T01:43:33.231 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "volumes", 2024-08-19T01:43:33.231 INFO:teuthology.orchestra.run.smithi083.stdout: "can_run": true, 2024-08-19T01:43:33.231 INFO:teuthology.orchestra.run.smithi083.stdout: "error_string": "", 2024-08-19T01:43:33.231 INFO:teuthology.orchestra.run.smithi083.stdout: "module_options": { 2024-08-19T01:43:33.231 INFO:teuthology.orchestra.run.smithi083.stdout: "log_level": { 2024-08-19T01:43:33.231 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_level", 2024-08-19T01:43:33.232 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-08-19T01:43:33.232 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:33.232 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-08-19T01:43:33.232 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-08-19T01:43:33.232 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:33.232 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:33.232 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-08-19T01:43:33.232 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-08-19T01:43:33.232 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-08-19T01:43:33.232 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-08-19T01:43:33.232 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-08-19T01:43:33.232 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-08-19T01:43:33.232 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-08-19T01:43:33.233 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-08-19T01:43:33.233 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-08-19T01:43:33.233 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:33.233 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:33.233 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:33.233 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:33.233 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster": { 2024-08-19T01:43:33.233 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster", 2024-08-19T01:43:33.233 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-08-19T01:43:33.233 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:33.233 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-08-19T01:43:33.233 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-08-19T01:43:33.233 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:33.233 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:33.233 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:33.233 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-08-19T01:43:33.234 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:33.234 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:33.234 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:33.234 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:33.234 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster_level": { 2024-08-19T01:43:33.234 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster_level", 2024-08-19T01:43:33.234 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-08-19T01:43:33.234 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:33.234 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-08-19T01:43:33.234 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "info", 2024-08-19T01:43:33.234 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:33.234 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:33.234 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-08-19T01:43:33.234 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-08-19T01:43:33.234 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-08-19T01:43:33.235 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-08-19T01:43:33.235 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-08-19T01:43:33.235 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-08-19T01:43:33.235 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-08-19T01:43:33.235 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-08-19T01:43:33.235 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-08-19T01:43:33.235 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:33.235 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:33.235 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:33.235 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:33.235 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_file": { 2024-08-19T01:43:33.235 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_file", 2024-08-19T01:43:33.235 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-08-19T01:43:33.235 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:33.236 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-08-19T01:43:33.236 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-08-19T01:43:33.236 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:33.236 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:33.236 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:33.236 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-08-19T01:43:33.236 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:33.236 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:33.236 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:33.236 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:33.236 INFO:teuthology.orchestra.run.smithi083.stdout: "max_concurrent_clones": { 2024-08-19T01:43:33.236 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "max_concurrent_clones", 2024-08-19T01:43:33.236 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2024-08-19T01:43:33.236 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:33.236 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-08-19T01:43:33.236 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "4", 2024-08-19T01:43:33.237 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:33.237 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:33.237 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:33.237 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Number of asynchronous cloner threads", 2024-08-19T01:43:33.237 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:33.237 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:33.237 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:33.237 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:33.237 INFO:teuthology.orchestra.run.smithi083.stdout: "periodic_async_work": { 2024-08-19T01:43:33.237 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "periodic_async_work", 2024-08-19T01:43:33.237 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-08-19T01:43:33.237 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:33.237 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-08-19T01:43:33.237 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-08-19T01:43:33.238 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:33.238 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:33.238 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:33.238 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Periodically check for async work", 2024-08-19T01:43:33.238 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:33.238 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:33.238 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:33.238 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:33.238 INFO:teuthology.orchestra.run.smithi083.stdout: "snapshot_clone_delay": { 2024-08-19T01:43:33.238 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "snapshot_clone_delay", 2024-08-19T01:43:33.238 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2024-08-19T01:43:33.238 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:33.238 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-08-19T01:43:33.238 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "0", 2024-08-19T01:43:33.239 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:33.239 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:33.239 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:33.239 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-08-19T01:43:33.239 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:33.239 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:33.239 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:33.239 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:33.239 INFO:teuthology.orchestra.run.smithi083.stdout: "snapshot_clone_no_wait": { 2024-08-19T01:43:33.239 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "snapshot_clone_no_wait", 2024-08-19T01:43:33.239 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-08-19T01:43:33.239 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:33.239 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-08-19T01:43:33.239 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "True", 2024-08-19T01:43:33.239 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:33.240 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:33.240 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:33.240 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-08-19T01:43:33.240 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:33.240 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:33.240 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:33.240 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-08-19T01:43:33.240 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-08-19T01:43:33.240 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:33.240 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-08-19T01:43:33.240 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "zabbix", 2024-08-19T01:43:33.240 INFO:teuthology.orchestra.run.smithi083.stdout: "can_run": true, 2024-08-19T01:43:33.240 INFO:teuthology.orchestra.run.smithi083.stdout: "error_string": "", 2024-08-19T01:43:33.240 INFO:teuthology.orchestra.run.smithi083.stdout: "module_options": { 2024-08-19T01:43:33.240 INFO:teuthology.orchestra.run.smithi083.stdout: "discovery_interval": { 2024-08-19T01:43:33.241 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "discovery_interval", 2024-08-19T01:43:33.241 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "uint", 2024-08-19T01:43:33.241 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:33.241 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-08-19T01:43:33.241 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "100", 2024-08-19T01:43:33.241 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:33.241 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:33.241 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:33.241 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-08-19T01:43:33.241 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:33.241 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:33.241 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:33.241 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:33.241 INFO:teuthology.orchestra.run.smithi083.stdout: "identifier": { 2024-08-19T01:43:33.242 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "identifier", 2024-08-19T01:43:33.242 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-08-19T01:43:33.242 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:33.242 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-08-19T01:43:33.242 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-08-19T01:43:33.242 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:33.242 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:33.242 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:33.242 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-08-19T01:43:33.242 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:33.242 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:33.242 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:33.242 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:33.243 INFO:teuthology.orchestra.run.smithi083.stdout: "interval": { 2024-08-19T01:43:33.243 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "interval", 2024-08-19T01:43:33.243 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "secs", 2024-08-19T01:43:33.243 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:33.243 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-08-19T01:43:33.243 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "60", 2024-08-19T01:43:33.243 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:33.243 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:33.243 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:33.243 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-08-19T01:43:33.243 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:33.243 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:33.243 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:33.243 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:33.243 INFO:teuthology.orchestra.run.smithi083.stdout: "log_level": { 2024-08-19T01:43:33.244 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_level", 2024-08-19T01:43:33.244 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-08-19T01:43:33.244 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:33.244 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-08-19T01:43:33.244 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-08-19T01:43:33.244 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:33.244 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:33.244 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-08-19T01:43:33.244 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-08-19T01:43:33.244 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-08-19T01:43:33.244 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-08-19T01:43:33.244 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-08-19T01:43:33.244 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-08-19T01:43:33.244 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-08-19T01:43:33.244 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-08-19T01:43:33.245 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-08-19T01:43:33.245 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:33.245 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:33.245 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:33.245 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:33.245 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster": { 2024-08-19T01:43:33.245 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster", 2024-08-19T01:43:33.245 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-08-19T01:43:33.245 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:33.245 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-08-19T01:43:33.245 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-08-19T01:43:33.245 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:33.245 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:33.245 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:33.245 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-08-19T01:43:33.246 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:33.246 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:33.246 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:33.246 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:33.246 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster_level": { 2024-08-19T01:43:33.246 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster_level", 2024-08-19T01:43:33.246 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-08-19T01:43:33.246 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:33.246 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-08-19T01:43:33.246 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "info", 2024-08-19T01:43:33.246 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:33.246 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:33.246 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-08-19T01:43:33.246 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-08-19T01:43:33.247 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-08-19T01:43:33.247 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-08-19T01:43:33.247 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-08-19T01:43:33.247 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-08-19T01:43:33.247 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-08-19T01:43:33.247 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-08-19T01:43:33.247 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-08-19T01:43:33.247 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:33.247 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:33.247 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:33.247 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:33.247 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_file": { 2024-08-19T01:43:33.247 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_file", 2024-08-19T01:43:33.248 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-08-19T01:43:33.248 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:33.248 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-08-19T01:43:33.248 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-08-19T01:43:33.248 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:33.248 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:33.248 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:33.248 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-08-19T01:43:33.248 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:33.248 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:33.248 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:33.248 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:33.248 INFO:teuthology.orchestra.run.smithi083.stdout: "zabbix_host": { 2024-08-19T01:43:33.249 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "zabbix_host", 2024-08-19T01:43:33.249 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-08-19T01:43:33.249 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:33.249 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-08-19T01:43:33.249 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-08-19T01:43:33.249 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:33.249 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:33.249 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:33.249 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-08-19T01:43:33.249 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:33.249 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:33.249 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:33.249 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:33.249 INFO:teuthology.orchestra.run.smithi083.stdout: "zabbix_port": { 2024-08-19T01:43:33.249 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "zabbix_port", 2024-08-19T01:43:33.249 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2024-08-19T01:43:33.250 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:33.250 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-08-19T01:43:33.250 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "10051", 2024-08-19T01:43:33.250 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:33.250 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:33.250 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:33.250 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-08-19T01:43:33.250 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:33.250 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:33.250 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:33.250 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:33.250 INFO:teuthology.orchestra.run.smithi083.stdout: "zabbix_sender": { 2024-08-19T01:43:33.251 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "zabbix_sender", 2024-08-19T01:43:33.251 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-08-19T01:43:33.251 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-08-19T01:43:33.251 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-08-19T01:43:33.251 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-08-19T01:43:33.251 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-08-19T01:43:33.251 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-08-19T01:43:33.251 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-08-19T01:43:33.251 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-08-19T01:43:33.251 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-08-19T01:43:33.251 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-08-19T01:43:33.251 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-08-19T01:43:33.251 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-08-19T01:43:33.252 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-08-19T01:43:33.252 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-08-19T01:43:33.252 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-08-19T01:43:33.252 INFO:teuthology.orchestra.run.smithi083.stdout: "services": { 2024-08-19T01:43:33.252 INFO:teuthology.orchestra.run.smithi083.stdout: "dashboard": "https://172.21.15.83:8443/" 2024-08-19T01:43:33.252 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:33.252 INFO:teuthology.orchestra.run.smithi083.stdout: "always_on_modules": { 2024-08-19T01:43:33.252 INFO:teuthology.orchestra.run.smithi083.stdout: "octopus": [ 2024-08-19T01:43:33.252 INFO:teuthology.orchestra.run.smithi083.stdout: "balancer", 2024-08-19T01:43:33.252 INFO:teuthology.orchestra.run.smithi083.stdout: "crash", 2024-08-19T01:43:33.252 INFO:teuthology.orchestra.run.smithi083.stdout: "devicehealth", 2024-08-19T01:43:33.252 INFO:teuthology.orchestra.run.smithi083.stdout: "orchestrator", 2024-08-19T01:43:33.252 INFO:teuthology.orchestra.run.smithi083.stdout: "pg_autoscaler", 2024-08-19T01:43:33.252 INFO:teuthology.orchestra.run.smithi083.stdout: "progress", 2024-08-19T01:43:33.253 INFO:teuthology.orchestra.run.smithi083.stdout: "rbd_support", 2024-08-19T01:43:33.253 INFO:teuthology.orchestra.run.smithi083.stdout: "status", 2024-08-19T01:43:33.253 INFO:teuthology.orchestra.run.smithi083.stdout: "telemetry", 2024-08-19T01:43:33.253 INFO:teuthology.orchestra.run.smithi083.stdout: "volumes" 2024-08-19T01:43:33.253 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-08-19T01:43:33.253 INFO:teuthology.orchestra.run.smithi083.stdout: "pacific": [ 2024-08-19T01:43:33.253 INFO:teuthology.orchestra.run.smithi083.stdout: "balancer", 2024-08-19T01:43:33.253 INFO:teuthology.orchestra.run.smithi083.stdout: "crash", 2024-08-19T01:43:33.253 INFO:teuthology.orchestra.run.smithi083.stdout: "devicehealth", 2024-08-19T01:43:33.253 INFO:teuthology.orchestra.run.smithi083.stdout: "orchestrator", 2024-08-19T01:43:33.253 INFO:teuthology.orchestra.run.smithi083.stdout: "pg_autoscaler", 2024-08-19T01:43:33.253 INFO:teuthology.orchestra.run.smithi083.stdout: "progress", 2024-08-19T01:43:33.253 INFO:teuthology.orchestra.run.smithi083.stdout: "rbd_support", 2024-08-19T01:43:33.253 INFO:teuthology.orchestra.run.smithi083.stdout: "status", 2024-08-19T01:43:33.253 INFO:teuthology.orchestra.run.smithi083.stdout: "telemetry", 2024-08-19T01:43:33.254 INFO:teuthology.orchestra.run.smithi083.stdout: "volumes" 2024-08-19T01:43:33.254 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-08-19T01:43:33.254 INFO:teuthology.orchestra.run.smithi083.stdout: "quincy": [ 2024-08-19T01:43:33.254 INFO:teuthology.orchestra.run.smithi083.stdout: "balancer", 2024-08-19T01:43:33.254 INFO:teuthology.orchestra.run.smithi083.stdout: "crash", 2024-08-19T01:43:33.254 INFO:teuthology.orchestra.run.smithi083.stdout: "devicehealth", 2024-08-19T01:43:33.254 INFO:teuthology.orchestra.run.smithi083.stdout: "orchestrator", 2024-08-19T01:43:33.254 INFO:teuthology.orchestra.run.smithi083.stdout: "pg_autoscaler", 2024-08-19T01:43:33.254 INFO:teuthology.orchestra.run.smithi083.stdout: "progress", 2024-08-19T01:43:33.254 INFO:teuthology.orchestra.run.smithi083.stdout: "rbd_support", 2024-08-19T01:43:33.254 INFO:teuthology.orchestra.run.smithi083.stdout: "status", 2024-08-19T01:43:33.254 INFO:teuthology.orchestra.run.smithi083.stdout: "telemetry", 2024-08-19T01:43:33.254 INFO:teuthology.orchestra.run.smithi083.stdout: "volumes" 2024-08-19T01:43:33.254 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-08-19T01:43:33.255 INFO:teuthology.orchestra.run.smithi083.stdout: "reef": [ 2024-08-19T01:43:33.255 INFO:teuthology.orchestra.run.smithi083.stdout: "balancer", 2024-08-19T01:43:33.255 INFO:teuthology.orchestra.run.smithi083.stdout: "crash", 2024-08-19T01:43:33.255 INFO:teuthology.orchestra.run.smithi083.stdout: "devicehealth", 2024-08-19T01:43:33.255 INFO:teuthology.orchestra.run.smithi083.stdout: "orchestrator", 2024-08-19T01:43:33.255 INFO:teuthology.orchestra.run.smithi083.stdout: "pg_autoscaler", 2024-08-19T01:43:33.255 INFO:teuthology.orchestra.run.smithi083.stdout: "progress", 2024-08-19T01:43:33.255 INFO:teuthology.orchestra.run.smithi083.stdout: "rbd_support", 2024-08-19T01:43:33.255 INFO:teuthology.orchestra.run.smithi083.stdout: "status", 2024-08-19T01:43:33.255 INFO:teuthology.orchestra.run.smithi083.stdout: "telemetry", 2024-08-19T01:43:33.255 INFO:teuthology.orchestra.run.smithi083.stdout: "volumes" 2024-08-19T01:43:33.255 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-08-19T01:43:33.255 INFO:teuthology.orchestra.run.smithi083.stdout: "squid": [ 2024-08-19T01:43:33.255 INFO:teuthology.orchestra.run.smithi083.stdout: "balancer", 2024-08-19T01:43:33.256 INFO:teuthology.orchestra.run.smithi083.stdout: "crash", 2024-08-19T01:43:33.256 INFO:teuthology.orchestra.run.smithi083.stdout: "devicehealth", 2024-08-19T01:43:33.256 INFO:teuthology.orchestra.run.smithi083.stdout: "orchestrator", 2024-08-19T01:43:33.256 INFO:teuthology.orchestra.run.smithi083.stdout: "pg_autoscaler", 2024-08-19T01:43:33.256 INFO:teuthology.orchestra.run.smithi083.stdout: "progress", 2024-08-19T01:43:33.256 INFO:teuthology.orchestra.run.smithi083.stdout: "rbd_support", 2024-08-19T01:43:33.256 INFO:teuthology.orchestra.run.smithi083.stdout: "status", 2024-08-19T01:43:33.256 INFO:teuthology.orchestra.run.smithi083.stdout: "telemetry", 2024-08-19T01:43:33.256 INFO:teuthology.orchestra.run.smithi083.stdout: "volumes" 2024-08-19T01:43:33.256 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-08-19T01:43:33.256 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:33.256 INFO:teuthology.orchestra.run.smithi083.stdout: "last_failure_osd_epoch": 3, 2024-08-19T01:43:33.256 INFO:teuthology.orchestra.run.smithi083.stdout: "active_clients": [ 2024-08-19T01:43:33.256 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-08-19T01:43:33.257 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "devicehealth", 2024-08-19T01:43:33.257 INFO:teuthology.orchestra.run.smithi083.stdout: "addrvec": [ 2024-08-19T01:43:33.257 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-08-19T01:43:33.257 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "v2", 2024-08-19T01:43:33.257 INFO:teuthology.orchestra.run.smithi083.stdout: "addr": "172.21.15.83:0", 2024-08-19T01:43:33.257 INFO:teuthology.orchestra.run.smithi083.stdout: "nonce": 2373054747 2024-08-19T01:43:33.257 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-08-19T01:43:33.257 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-08-19T01:43:33.257 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:33.257 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-08-19T01:43:33.257 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "libcephsqlite", 2024-08-19T01:43:33.257 INFO:teuthology.orchestra.run.smithi083.stdout: "addrvec": [ 2024-08-19T01:43:33.257 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-08-19T01:43:33.257 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "v2", 2024-08-19T01:43:33.257 INFO:teuthology.orchestra.run.smithi083.stdout: "addr": "172.21.15.83:0", 2024-08-19T01:43:33.258 INFO:teuthology.orchestra.run.smithi083.stdout: "nonce": 3822765168 2024-08-19T01:43:33.258 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-08-19T01:43:33.258 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-08-19T01:43:33.258 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:33.258 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-08-19T01:43:33.258 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "rbd_support", 2024-08-19T01:43:33.258 INFO:teuthology.orchestra.run.smithi083.stdout: "addrvec": [ 2024-08-19T01:43:33.258 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-08-19T01:43:33.258 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "v2", 2024-08-19T01:43:33.258 INFO:teuthology.orchestra.run.smithi083.stdout: "addr": "172.21.15.83:0", 2024-08-19T01:43:33.258 INFO:teuthology.orchestra.run.smithi083.stdout: "nonce": 2728865953 2024-08-19T01:43:33.258 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-08-19T01:43:33.258 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-08-19T01:43:33.259 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:33.259 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-08-19T01:43:33.259 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "volumes", 2024-08-19T01:43:33.259 INFO:teuthology.orchestra.run.smithi083.stdout: "addrvec": [ 2024-08-19T01:43:33.259 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-08-19T01:43:33.259 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "v2", 2024-08-19T01:43:33.259 INFO:teuthology.orchestra.run.smithi083.stdout: "addr": "172.21.15.83:0", 2024-08-19T01:43:33.259 INFO:teuthology.orchestra.run.smithi083.stdout: "nonce": 3847157033 2024-08-19T01:43:33.259 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-08-19T01:43:33.259 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-08-19T01:43:33.259 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-08-19T01:43:33.259 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-08-19T01:43:33.259 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-08-19T01:43:33.261 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:43:31 smithi117 ceph-mon[22016]: pgmap v599: 97 pgs: 97 active+clean; 35 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 255 B/s wr, 6 op/s 2024-08-19T01:43:33.261 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:43:31 smithi117 ceph-mon[22016]: from='client.? 172.21.15.83:0/3275406329' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-08-19T01:43:33.261 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:43:31 smithi117 ceph-mon[22016]: from='client.? 172.21.15.83:0/3951265864' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-08-19T01:43:33.261 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:43:31 smithi117 ceph-mon[22016]: from='client.? 172.21.15.83:0/1590710327' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-08-19T01:43:33.261 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:43:32 smithi117 ceph-mon[22016]: from='client.? 172.21.15.83:0/2973240649' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-08-19T01:43:33.261 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:43:31 smithi167 ceph-mon[21875]: pgmap v599: 97 pgs: 97 active+clean; 35 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 255 B/s wr, 6 op/s 2024-08-19T01:43:33.262 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:43:31 smithi167 ceph-mon[21875]: from='client.? 172.21.15.83:0/3275406329' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-08-19T01:43:33.262 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:43:31 smithi167 ceph-mon[21875]: from='client.? 172.21.15.83:0/3951265864' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-08-19T01:43:33.262 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:43:31 smithi167 ceph-mon[21875]: from='client.? 172.21.15.83:0/1590710327' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-08-19T01:43:33.262 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:43:32 smithi167 ceph-mon[21875]: from='client.? 172.21.15.83:0/2973240649' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-08-19T01:43:33.262 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:31.922+0000 7f733b200640 1 -- 172.21.15.83:0/2973240649 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f73200a2b90 msgr2=0x7f73200a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:43:33.262 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:31.922+0000 7f733b200640 1 --2- 172.21.15.83:0/2973240649 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f73200a2b90 0x7f73200a5050 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f732c0096c0 tx=0x7f732c0315b0 comp rx=0 tx=0).stop 2024-08-19T01:43:33.262 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:31.922+0000 7f733b200640 1 -- 172.21.15.83:0/2973240649 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f733c101260 msgr2=0x7f733c1a1190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:43:33.263 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:31.922+0000 7f733b200640 1 --2- 172.21.15.83:0/2973240649 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f733c101260 0x7f733c1a1190 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f7334009fd0 tx=0x7f733400ef10 comp rx=0 tx=0).stop 2024-08-19T01:43:33.263 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:31.922+0000 7f733b200640 1 -- 172.21.15.83:0/2973240649 shutdown_connections 2024-08-19T01:43:33.263 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:31.922+0000 7f733b200640 1 --2- 172.21.15.83:0/2973240649 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f73200a2b90 0x7f73200a5050 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:43:33.263 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:31.922+0000 7f733b200640 1 --2- 172.21.15.83:0/2973240649 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f733c101260 0x7f733c1a1190 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:43:33.263 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:31.922+0000 7f733b200640 1 --2- 172.21.15.83:0/2973240649 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f733c1008c0 0x7f733c19a120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:43:33.263 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:31.922+0000 7f733b200640 1 --2- 172.21.15.83:0/2973240649 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f733c0fff80 0x7f733c199be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:43:33.263 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:31.922+0000 7f733b200640 1 -- 172.21.15.83:0/2973240649 >> 172.21.15.83:0/2973240649 conn(0x7f733c0fbd30 msgr2=0x7f733c0fd7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:43:33.263 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:31.922+0000 7f733b200640 1 -- 172.21.15.83:0/2973240649 shutdown_connections 2024-08-19T01:43:33.263 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:31.922+0000 7f733b200640 1 -- 172.21.15.83:0/2973240649 wait complete. 2024-08-19T01:43:33.269 DEBUG:tasks.check_counter:Getting stats from a 2024-08-19T01:43:33.269 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833 shell --fsid bad8780e-5dc9-11ef-bccf-c7b262605968 -- ceph daemon mds.a perf dump 2024-08-19T01:43:33.269 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:43:32 smithi083 ceph-mon[17120]: from='client.? 172.21.15.83:0/2973240649' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-08-19T01:43:33.652 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/bad8780e-5dc9-11ef-bccf-c7b262605968/mon.a/config 2024-08-19T01:43:33.998 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:43:33 smithi083 ceph-mon[17120]: pgmap v600: 97 pgs: 97 active+clean; 35 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2024-08-19T01:43:34.202 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:43:33 smithi117 ceph-mon[22016]: pgmap v600: 97 pgs: 97 active+clean; 35 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2024-08-19T01:43:34.216 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:43:33 smithi167 ceph-mon[21875]: pgmap v600: 97 pgs: 97 active+clean; 35 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2024-08-19T01:43:34.284 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-08-19T01:43:34.284 INFO:teuthology.orchestra.run.smithi083.stdout: "AsyncMessenger::Worker-0": { 2024-08-19T01:43:34.285 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_recv_messages": 4, 2024-08-19T01:43:34.285 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_send_messages": 4, 2024-08-19T01:43:34.285 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_recv_bytes": 3024, 2024-08-19T01:43:34.285 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_send_bytes": 448, 2024-08-19T01:43:34.285 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_created_connections": 4, 2024-08-19T01:43:34.285 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_active_connections": 0, 2024-08-19T01:43:34.285 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_running_total_time": 0.006519520, 2024-08-19T01:43:34.285 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_running_send_time": 0.000062318, 2024-08-19T01:43:34.285 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_running_recv_time": 0.003499116, 2024-08-19T01:43:34.285 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-08-19T01:43:34.285 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_send_messages_queue_lat": { 2024-08-19T01:43:34.285 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 3, 2024-08-19T01:43:34.285 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.006080477, 2024-08-19T01:43:34.285 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.002026825 2024-08-19T01:43:34.286 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:34.286 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_handle_ack_lat": { 2024-08-19T01:43:34.286 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-08-19T01:43:34.286 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-08-19T01:43:34.286 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-08-19T01:43:34.286 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:34.286 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_recv_encrypted_bytes": 3024, 2024-08-19T01:43:34.286 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_send_encrypted_bytes": 448 2024-08-19T01:43:34.286 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:34.286 INFO:teuthology.orchestra.run.smithi083.stdout: "AsyncMessenger::Worker-1": { 2024-08-19T01:43:34.286 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_recv_messages": 1, 2024-08-19T01:43:34.286 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_send_messages": 137, 2024-08-19T01:43:34.286 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_recv_bytes": 160, 2024-08-19T01:43:34.286 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_send_bytes": 102336, 2024-08-19T01:43:34.287 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_created_connections": 3, 2024-08-19T01:43:34.287 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_active_connections": 2, 2024-08-19T01:43:34.287 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_running_total_time": 0.016749752, 2024-08-19T01:43:34.287 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_running_send_time": 0.013682492, 2024-08-19T01:43:34.287 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_running_recv_time": 0.001429060, 2024-08-19T01:43:34.287 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-08-19T01:43:34.287 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_send_messages_queue_lat": { 2024-08-19T01:43:34.287 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 135, 2024-08-19T01:43:34.287 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.016198189, 2024-08-19T01:43:34.287 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000119986 2024-08-19T01:43:34.287 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:34.287 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_handle_ack_lat": { 2024-08-19T01:43:34.287 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-08-19T01:43:34.288 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-08-19T01:43:34.288 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-08-19T01:43:34.288 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:34.288 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_recv_encrypted_bytes": 160, 2024-08-19T01:43:34.288 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_send_encrypted_bytes": 102336 2024-08-19T01:43:34.288 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:34.288 INFO:teuthology.orchestra.run.smithi083.stdout: "AsyncMessenger::Worker-2": { 2024-08-19T01:43:34.288 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_recv_messages": 185, 2024-08-19T01:43:34.288 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_send_messages": 180, 2024-08-19T01:43:34.288 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_recv_bytes": 743776, 2024-08-19T01:43:34.288 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_send_bytes": 110288, 2024-08-19T01:43:34.288 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_created_connections": 2, 2024-08-19T01:43:34.288 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_active_connections": 1, 2024-08-19T01:43:34.288 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_running_total_time": 0.041852068, 2024-08-19T01:43:34.288 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_running_send_time": 0.018764853, 2024-08-19T01:43:34.289 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_running_recv_time": 0.030188418, 2024-08-19T01:43:34.289 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-08-19T01:43:34.289 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_send_messages_queue_lat": { 2024-08-19T01:43:34.289 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 179, 2024-08-19T01:43:34.289 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.034923152, 2024-08-19T01:43:34.289 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000195101 2024-08-19T01:43:34.289 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:34.289 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_handle_ack_lat": { 2024-08-19T01:43:34.289 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-08-19T01:43:34.289 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-08-19T01:43:34.289 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-08-19T01:43:34.289 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:34.289 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_recv_encrypted_bytes": 743776, 2024-08-19T01:43:34.289 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_send_encrypted_bytes": 110288 2024-08-19T01:43:34.289 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:34.290 INFO:teuthology.orchestra.run.smithi083.stdout: "cct": { 2024-08-19T01:43:34.290 INFO:teuthology.orchestra.run.smithi083.stdout: "total_workers": 0, 2024-08-19T01:43:34.290 INFO:teuthology.orchestra.run.smithi083.stdout: "unhealthy_workers": 0 2024-08-19T01:43:34.290 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:34.290 INFO:teuthology.orchestra.run.smithi083.stdout: "mempool": { 2024-08-19T01:43:34.290 INFO:teuthology.orchestra.run.smithi083.stdout: "bloom_filter_bytes": 0, 2024-08-19T01:43:34.290 INFO:teuthology.orchestra.run.smithi083.stdout: "bloom_filter_items": 0, 2024-08-19T01:43:34.290 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_alloc_bytes": 0, 2024-08-19T01:43:34.290 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_alloc_items": 0, 2024-08-19T01:43:34.290 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_cache_data_bytes": 0, 2024-08-19T01:43:34.290 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_cache_data_items": 0, 2024-08-19T01:43:34.290 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_cache_onode_bytes": 0, 2024-08-19T01:43:34.290 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_cache_onode_items": 0, 2024-08-19T01:43:34.290 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_cache_meta_bytes": 0, 2024-08-19T01:43:34.291 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_cache_meta_items": 0, 2024-08-19T01:43:34.291 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_cache_other_bytes": 0, 2024-08-19T01:43:34.291 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_cache_other_items": 0, 2024-08-19T01:43:34.291 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_cache_buffer_bytes": 0, 2024-08-19T01:43:34.291 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_cache_buffer_items": 0, 2024-08-19T01:43:34.291 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_extent_bytes": 0, 2024-08-19T01:43:34.291 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_extent_items": 0, 2024-08-19T01:43:34.291 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_blob_bytes": 0, 2024-08-19T01:43:34.291 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_blob_items": 0, 2024-08-19T01:43:34.291 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_shared_blob_bytes": 0, 2024-08-19T01:43:34.291 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_shared_blob_items": 0, 2024-08-19T01:43:34.291 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_inline_bl_bytes": 0, 2024-08-19T01:43:34.291 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_inline_bl_items": 0, 2024-08-19T01:43:34.292 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_fsck_bytes": 0, 2024-08-19T01:43:34.292 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_fsck_items": 0, 2024-08-19T01:43:34.292 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_txc_bytes": 0, 2024-08-19T01:43:34.292 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_txc_items": 0, 2024-08-19T01:43:34.292 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_writing_deferred_bytes": 0, 2024-08-19T01:43:34.292 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_writing_deferred_items": 0, 2024-08-19T01:43:34.292 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_writing_bytes": 0, 2024-08-19T01:43:34.292 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_writing_items": 0, 2024-08-19T01:43:34.292 INFO:teuthology.orchestra.run.smithi083.stdout: "bluefs_bytes": 0, 2024-08-19T01:43:34.292 INFO:teuthology.orchestra.run.smithi083.stdout: "bluefs_items": 0, 2024-08-19T01:43:34.292 INFO:teuthology.orchestra.run.smithi083.stdout: "bluefs_file_reader_bytes": 0, 2024-08-19T01:43:34.292 INFO:teuthology.orchestra.run.smithi083.stdout: "bluefs_file_reader_items": 0, 2024-08-19T01:43:34.292 INFO:teuthology.orchestra.run.smithi083.stdout: "bluefs_file_writer_bytes": 0, 2024-08-19T01:43:34.293 INFO:teuthology.orchestra.run.smithi083.stdout: "bluefs_file_writer_items": 0, 2024-08-19T01:43:34.293 INFO:teuthology.orchestra.run.smithi083.stdout: "buffer_anon_bytes": 148984, 2024-08-19T01:43:34.293 INFO:teuthology.orchestra.run.smithi083.stdout: "buffer_anon_items": 30, 2024-08-19T01:43:34.293 INFO:teuthology.orchestra.run.smithi083.stdout: "buffer_meta_bytes": 0, 2024-08-19T01:43:34.293 INFO:teuthology.orchestra.run.smithi083.stdout: "buffer_meta_items": 0, 2024-08-19T01:43:34.293 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_bytes": 0, 2024-08-19T01:43:34.293 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_items": 0, 2024-08-19T01:43:34.293 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_mapbl_bytes": 0, 2024-08-19T01:43:34.293 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_mapbl_items": 0, 2024-08-19T01:43:34.293 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_pglog_bytes": 0, 2024-08-19T01:43:34.293 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_pglog_items": 0, 2024-08-19T01:43:34.293 INFO:teuthology.orchestra.run.smithi083.stdout: "osdmap_bytes": 0, 2024-08-19T01:43:34.293 INFO:teuthology.orchestra.run.smithi083.stdout: "osdmap_items": 0, 2024-08-19T01:43:34.293 INFO:teuthology.orchestra.run.smithi083.stdout: "osdmap_mapping_bytes": 0, 2024-08-19T01:43:34.294 INFO:teuthology.orchestra.run.smithi083.stdout: "osdmap_mapping_items": 0, 2024-08-19T01:43:34.294 INFO:teuthology.orchestra.run.smithi083.stdout: "pgmap_bytes": 0, 2024-08-19T01:43:34.294 INFO:teuthology.orchestra.run.smithi083.stdout: "pgmap_items": 0, 2024-08-19T01:43:34.294 INFO:teuthology.orchestra.run.smithi083.stdout: "mds_co_bytes": 928, 2024-08-19T01:43:34.294 INFO:teuthology.orchestra.run.smithi083.stdout: "mds_co_items": 2, 2024-08-19T01:43:34.294 INFO:teuthology.orchestra.run.smithi083.stdout: "unittest_1_bytes": 0, 2024-08-19T01:43:34.294 INFO:teuthology.orchestra.run.smithi083.stdout: "unittest_1_items": 0, 2024-08-19T01:43:34.294 INFO:teuthology.orchestra.run.smithi083.stdout: "unittest_2_bytes": 0, 2024-08-19T01:43:34.294 INFO:teuthology.orchestra.run.smithi083.stdout: "unittest_2_items": 0 2024-08-19T01:43:34.294 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:34.294 INFO:teuthology.orchestra.run.smithi083.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-08-19T01:43:34.294 INFO:teuthology.orchestra.run.smithi083.stdout: "val": 0, 2024-08-19T01:43:34.294 INFO:teuthology.orchestra.run.smithi083.stdout: "max": 104857600, 2024-08-19T01:43:34.294 INFO:teuthology.orchestra.run.smithi083.stdout: "get_started": 0, 2024-08-19T01:43:34.295 INFO:teuthology.orchestra.run.smithi083.stdout: "get": 186, 2024-08-19T01:43:34.295 INFO:teuthology.orchestra.run.smithi083.stdout: "get_sum": 717671, 2024-08-19T01:43:34.295 INFO:teuthology.orchestra.run.smithi083.stdout: "get_or_fail_fail": 0, 2024-08-19T01:43:34.295 INFO:teuthology.orchestra.run.smithi083.stdout: "get_or_fail_success": 186, 2024-08-19T01:43:34.295 INFO:teuthology.orchestra.run.smithi083.stdout: "take": 0, 2024-08-19T01:43:34.295 INFO:teuthology.orchestra.run.smithi083.stdout: "take_sum": 0, 2024-08-19T01:43:34.295 INFO:teuthology.orchestra.run.smithi083.stdout: "put": 186, 2024-08-19T01:43:34.295 INFO:teuthology.orchestra.run.smithi083.stdout: "put_sum": 717671, 2024-08-19T01:43:34.295 INFO:teuthology.orchestra.run.smithi083.stdout: "wait": { 2024-08-19T01:43:34.295 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-08-19T01:43:34.295 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-08-19T01:43:34.295 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-08-19T01:43:34.295 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-08-19T01:43:34.296 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-08-19T01:43:34.296 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-08-19T01:43:34.989 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.a 2024-08-19T01:43:34.989 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.a 2024-08-19T01:43:34.989 WARNING:tasks.check_counter:Counter 'mds.dir_split' not found on daemon mds.a 2024-08-19T01:43:34.989 DEBUG:tasks.check_counter:Getting stats from d 2024-08-19T01:43:34.990 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833 shell --fsid bad8780e-5dc9-11ef-bccf-c7b262605968 -- ceph daemon mds.d perf dump 2024-08-19T01:43:35.375 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/bad8780e-5dc9-11ef-bccf-c7b262605968/mon.a/config 2024-08-19T01:43:35.977 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:43:35 smithi083 ceph-mon[17120]: pgmap v601: 97 pgs: 97 active+clean; 35 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 5 op/s 2024-08-19T01:43:35.993 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-08-19T01:43:35.994 INFO:teuthology.orchestra.run.smithi083.stdout: "AsyncMessenger::Worker-0": { 2024-08-19T01:43:35.994 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_recv_messages": 4, 2024-08-19T01:43:35.994 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_send_messages": 4, 2024-08-19T01:43:35.994 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_recv_bytes": 3024, 2024-08-19T01:43:35.994 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_send_bytes": 448, 2024-08-19T01:43:35.994 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_created_connections": 4, 2024-08-19T01:43:35.994 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_active_connections": 0, 2024-08-19T01:43:35.994 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_running_total_time": 0.006097309, 2024-08-19T01:43:35.994 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_running_send_time": 0.000076572, 2024-08-19T01:43:35.994 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_running_recv_time": 0.003422926, 2024-08-19T01:43:35.994 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-08-19T01:43:35.994 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_send_messages_queue_lat": { 2024-08-19T01:43:35.995 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 3, 2024-08-19T01:43:35.995 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.006114543, 2024-08-19T01:43:35.995 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.002038181 2024-08-19T01:43:35.995 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:35.995 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_handle_ack_lat": { 2024-08-19T01:43:35.995 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-08-19T01:43:35.995 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-08-19T01:43:35.995 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-08-19T01:43:35.995 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:35.995 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_recv_encrypted_bytes": 3024, 2024-08-19T01:43:35.995 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_send_encrypted_bytes": 448 2024-08-19T01:43:35.995 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:35.995 INFO:teuthology.orchestra.run.smithi083.stdout: "AsyncMessenger::Worker-1": { 2024-08-19T01:43:35.995 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_recv_messages": 191, 2024-08-19T01:43:35.996 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_send_messages": 186, 2024-08-19T01:43:35.996 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_recv_bytes": 745408, 2024-08-19T01:43:35.996 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_send_bytes": 114128, 2024-08-19T01:43:35.996 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_created_connections": 2, 2024-08-19T01:43:35.996 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_active_connections": 1, 2024-08-19T01:43:35.996 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_running_total_time": 0.045550993, 2024-08-19T01:43:35.996 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_running_send_time": 0.021785681, 2024-08-19T01:43:35.996 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_running_recv_time": 0.029837628, 2024-08-19T01:43:35.996 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-08-19T01:43:35.996 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_send_messages_queue_lat": { 2024-08-19T01:43:35.996 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 185, 2024-08-19T01:43:35.996 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.030676081, 2024-08-19T01:43:35.996 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000165816 2024-08-19T01:43:35.996 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:35.997 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_handle_ack_lat": { 2024-08-19T01:43:35.997 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-08-19T01:43:35.997 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-08-19T01:43:35.997 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-08-19T01:43:35.997 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:35.997 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_recv_encrypted_bytes": 745408, 2024-08-19T01:43:35.997 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_send_encrypted_bytes": 114128 2024-08-19T01:43:35.997 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:35.997 INFO:teuthology.orchestra.run.smithi083.stdout: "AsyncMessenger::Worker-2": { 2024-08-19T01:43:35.997 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_recv_messages": 1, 2024-08-19T01:43:35.997 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_send_messages": 141, 2024-08-19T01:43:35.997 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_recv_bytes": 160, 2024-08-19T01:43:35.997 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_send_bytes": 103040, 2024-08-19T01:43:35.997 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_created_connections": 3, 2024-08-19T01:43:35.998 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_active_connections": 2, 2024-08-19T01:43:35.998 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_running_total_time": 0.016113424, 2024-08-19T01:43:35.998 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_running_send_time": 0.012478347, 2024-08-19T01:43:35.998 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_running_recv_time": 0.001819262, 2024-08-19T01:43:35.998 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-08-19T01:43:35.998 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_send_messages_queue_lat": { 2024-08-19T01:43:35.998 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 139, 2024-08-19T01:43:35.998 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.018183910, 2024-08-19T01:43:35.998 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000130819 2024-08-19T01:43:35.998 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:35.998 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_handle_ack_lat": { 2024-08-19T01:43:35.998 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-08-19T01:43:35.998 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-08-19T01:43:35.999 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-08-19T01:43:35.999 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:35.999 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_recv_encrypted_bytes": 160, 2024-08-19T01:43:35.999 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_send_encrypted_bytes": 103040 2024-08-19T01:43:35.999 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:35.999 INFO:teuthology.orchestra.run.smithi083.stdout: "cct": { 2024-08-19T01:43:35.999 INFO:teuthology.orchestra.run.smithi083.stdout: "total_workers": 0, 2024-08-19T01:43:35.999 INFO:teuthology.orchestra.run.smithi083.stdout: "unhealthy_workers": 0 2024-08-19T01:43:35.999 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:35.999 INFO:teuthology.orchestra.run.smithi083.stdout: "mempool": { 2024-08-19T01:43:35.999 INFO:teuthology.orchestra.run.smithi083.stdout: "bloom_filter_bytes": 0, 2024-08-19T01:43:35.999 INFO:teuthology.orchestra.run.smithi083.stdout: "bloom_filter_items": 0, 2024-08-19T01:43:35.999 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_alloc_bytes": 0, 2024-08-19T01:43:35.999 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_alloc_items": 0, 2024-08-19T01:43:36.000 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_cache_data_bytes": 0, 2024-08-19T01:43:36.000 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_cache_data_items": 0, 2024-08-19T01:43:36.000 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_cache_onode_bytes": 0, 2024-08-19T01:43:36.000 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_cache_onode_items": 0, 2024-08-19T01:43:36.000 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_cache_meta_bytes": 0, 2024-08-19T01:43:36.000 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_cache_meta_items": 0, 2024-08-19T01:43:36.000 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_cache_other_bytes": 0, 2024-08-19T01:43:36.000 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_cache_other_items": 0, 2024-08-19T01:43:36.000 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_cache_buffer_bytes": 0, 2024-08-19T01:43:36.000 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_cache_buffer_items": 0, 2024-08-19T01:43:36.000 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_extent_bytes": 0, 2024-08-19T01:43:36.000 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_extent_items": 0, 2024-08-19T01:43:36.000 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_blob_bytes": 0, 2024-08-19T01:43:36.000 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_blob_items": 0, 2024-08-19T01:43:36.001 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_shared_blob_bytes": 0, 2024-08-19T01:43:36.001 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_shared_blob_items": 0, 2024-08-19T01:43:36.001 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_inline_bl_bytes": 0, 2024-08-19T01:43:36.001 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_inline_bl_items": 0, 2024-08-19T01:43:36.001 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_fsck_bytes": 0, 2024-08-19T01:43:36.001 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_fsck_items": 0, 2024-08-19T01:43:36.001 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_txc_bytes": 0, 2024-08-19T01:43:36.001 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_txc_items": 0, 2024-08-19T01:43:36.001 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_writing_deferred_bytes": 0, 2024-08-19T01:43:36.001 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_writing_deferred_items": 0, 2024-08-19T01:43:36.001 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_writing_bytes": 0, 2024-08-19T01:43:36.001 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_writing_items": 0, 2024-08-19T01:43:36.001 INFO:teuthology.orchestra.run.smithi083.stdout: "bluefs_bytes": 0, 2024-08-19T01:43:36.001 INFO:teuthology.orchestra.run.smithi083.stdout: "bluefs_items": 0, 2024-08-19T01:43:36.002 INFO:teuthology.orchestra.run.smithi083.stdout: "bluefs_file_reader_bytes": 0, 2024-08-19T01:43:36.002 INFO:teuthology.orchestra.run.smithi083.stdout: "bluefs_file_reader_items": 0, 2024-08-19T01:43:36.002 INFO:teuthology.orchestra.run.smithi083.stdout: "bluefs_file_writer_bytes": 0, 2024-08-19T01:43:36.002 INFO:teuthology.orchestra.run.smithi083.stdout: "bluefs_file_writer_items": 0, 2024-08-19T01:43:36.002 INFO:teuthology.orchestra.run.smithi083.stdout: "buffer_anon_bytes": 148984, 2024-08-19T01:43:36.002 INFO:teuthology.orchestra.run.smithi083.stdout: "buffer_anon_items": 30, 2024-08-19T01:43:36.002 INFO:teuthology.orchestra.run.smithi083.stdout: "buffer_meta_bytes": 0, 2024-08-19T01:43:36.002 INFO:teuthology.orchestra.run.smithi083.stdout: "buffer_meta_items": 0, 2024-08-19T01:43:36.002 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_bytes": 0, 2024-08-19T01:43:36.002 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_items": 0, 2024-08-19T01:43:36.002 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_mapbl_bytes": 0, 2024-08-19T01:43:36.002 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_mapbl_items": 0, 2024-08-19T01:43:36.002 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_pglog_bytes": 0, 2024-08-19T01:43:36.002 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_pglog_items": 0, 2024-08-19T01:43:36.003 INFO:teuthology.orchestra.run.smithi083.stdout: "osdmap_bytes": 0, 2024-08-19T01:43:36.003 INFO:teuthology.orchestra.run.smithi083.stdout: "osdmap_items": 0, 2024-08-19T01:43:36.003 INFO:teuthology.orchestra.run.smithi083.stdout: "osdmap_mapping_bytes": 0, 2024-08-19T01:43:36.003 INFO:teuthology.orchestra.run.smithi083.stdout: "osdmap_mapping_items": 0, 2024-08-19T01:43:36.003 INFO:teuthology.orchestra.run.smithi083.stdout: "pgmap_bytes": 0, 2024-08-19T01:43:36.003 INFO:teuthology.orchestra.run.smithi083.stdout: "pgmap_items": 0, 2024-08-19T01:43:36.003 INFO:teuthology.orchestra.run.smithi083.stdout: "mds_co_bytes": 928, 2024-08-19T01:43:36.003 INFO:teuthology.orchestra.run.smithi083.stdout: "mds_co_items": 2, 2024-08-19T01:43:36.003 INFO:teuthology.orchestra.run.smithi083.stdout: "unittest_1_bytes": 0, 2024-08-19T01:43:36.003 INFO:teuthology.orchestra.run.smithi083.stdout: "unittest_1_items": 0, 2024-08-19T01:43:36.003 INFO:teuthology.orchestra.run.smithi083.stdout: "unittest_2_bytes": 0, 2024-08-19T01:43:36.003 INFO:teuthology.orchestra.run.smithi083.stdout: "unittest_2_items": 0 2024-08-19T01:43:36.003 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:36.003 INFO:teuthology.orchestra.run.smithi083.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-08-19T01:43:36.003 INFO:teuthology.orchestra.run.smithi083.stdout: "val": 0, 2024-08-19T01:43:36.003 INFO:teuthology.orchestra.run.smithi083.stdout: "max": 104857600, 2024-08-19T01:43:36.004 INFO:teuthology.orchestra.run.smithi083.stdout: "get_started": 0, 2024-08-19T01:43:36.004 INFO:teuthology.orchestra.run.smithi083.stdout: "get": 192, 2024-08-19T01:43:36.004 INFO:teuthology.orchestra.run.smithi083.stdout: "get_sum": 718451, 2024-08-19T01:43:36.004 INFO:teuthology.orchestra.run.smithi083.stdout: "get_or_fail_fail": 0, 2024-08-19T01:43:36.004 INFO:teuthology.orchestra.run.smithi083.stdout: "get_or_fail_success": 192, 2024-08-19T01:43:36.004 INFO:teuthology.orchestra.run.smithi083.stdout: "take": 0, 2024-08-19T01:43:36.004 INFO:teuthology.orchestra.run.smithi083.stdout: "take_sum": 0, 2024-08-19T01:43:36.004 INFO:teuthology.orchestra.run.smithi083.stdout: "put": 192, 2024-08-19T01:43:36.004 INFO:teuthology.orchestra.run.smithi083.stdout: "put_sum": 718451, 2024-08-19T01:43:36.004 INFO:teuthology.orchestra.run.smithi083.stdout: "wait": { 2024-08-19T01:43:36.004 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-08-19T01:43:36.004 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-08-19T01:43:36.004 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-08-19T01:43:36.004 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-08-19T01:43:36.004 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-08-19T01:43:36.004 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-08-19T01:43:36.202 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:43:35 smithi117 ceph-mon[22016]: pgmap v601: 97 pgs: 97 active+clean; 35 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 5 op/s 2024-08-19T01:43:36.215 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:43:35 smithi167 ceph-mon[21875]: pgmap v601: 97 pgs: 97 active+clean; 35 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 5 op/s 2024-08-19T01:43:36.769 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.d 2024-08-19T01:43:36.769 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.d 2024-08-19T01:43:36.769 WARNING:tasks.check_counter:Counter 'mds.dir_split' not found on daemon mds.d 2024-08-19T01:43:36.769 DEBUG:tasks.check_counter:Getting stats from g 2024-08-19T01:43:36.770 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833 shell --fsid bad8780e-5dc9-11ef-bccf-c7b262605968 -- ceph daemon mds.g perf dump 2024-08-19T01:43:37.146 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/bad8780e-5dc9-11ef-bccf-c7b262605968/mon.a/config 2024-08-19T01:43:37.722 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-08-19T01:43:37.722 INFO:teuthology.orchestra.run.smithi083.stdout: "AsyncMessenger::Worker-0": { 2024-08-19T01:43:37.722 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_recv_messages": 19052, 2024-08-19T01:43:37.722 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_send_messages": 21325, 2024-08-19T01:43:37.722 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_recv_bytes": 6709453, 2024-08-19T01:43:37.722 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_send_bytes": 16038995, 2024-08-19T01:43:37.723 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_created_connections": 8, 2024-08-19T01:43:37.723 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_active_connections": 4, 2024-08-19T01:43:37.723 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_running_total_time": 3.311908833, 2024-08-19T01:43:37.723 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_running_send_time": 2.387842217, 2024-08-19T01:43:37.723 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_running_recv_time": 1.306332662, 2024-08-19T01:43:37.723 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_running_fast_dispatch_time": 0.030673469, 2024-08-19T01:43:37.723 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_send_messages_queue_lat": { 2024-08-19T01:43:37.723 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 21325, 2024-08-19T01:43:37.723 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 6.475207209, 2024-08-19T01:43:37.723 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000303643 2024-08-19T01:43:37.723 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:37.724 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_handle_ack_lat": { 2024-08-19T01:43:37.724 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 21267, 2024-08-19T01:43:37.724 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.012280188, 2024-08-19T01:43:37.724 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000577 2024-08-19T01:43:37.724 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:37.724 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_recv_encrypted_bytes": 811392, 2024-08-19T01:43:37.724 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_send_encrypted_bytes": 121824 2024-08-19T01:43:37.724 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:37.724 INFO:teuthology.orchestra.run.smithi083.stdout: "AsyncMessenger::Worker-1": { 2024-08-19T01:43:37.724 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_recv_messages": 682, 2024-08-19T01:43:37.724 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_send_messages": 819, 2024-08-19T01:43:37.724 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_recv_bytes": 172278, 2024-08-19T01:43:37.724 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_send_bytes": 753042, 2024-08-19T01:43:37.725 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_created_connections": 23, 2024-08-19T01:43:37.725 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_active_connections": 5, 2024-08-19T01:43:37.725 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_running_total_time": 0.102753214, 2024-08-19T01:43:37.725 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_running_send_time": 0.039895555, 2024-08-19T01:43:37.725 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_running_recv_time": 0.032391297, 2024-08-19T01:43:37.725 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_running_fast_dispatch_time": 0.022817528, 2024-08-19T01:43:37.725 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_send_messages_queue_lat": { 2024-08-19T01:43:37.725 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 817, 2024-08-19T01:43:37.725 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.126233726, 2024-08-19T01:43:37.725 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000154508 2024-08-19T01:43:37.725 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:37.725 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_handle_ack_lat": { 2024-08-19T01:43:37.725 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 34, 2024-08-19T01:43:37.725 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000005307, 2024-08-19T01:43:37.726 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000156 2024-08-19T01:43:37.726 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:37.726 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_recv_encrypted_bytes": 3264, 2024-08-19T01:43:37.726 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_send_encrypted_bytes": 324928 2024-08-19T01:43:37.726 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:37.726 INFO:teuthology.orchestra.run.smithi083.stdout: "AsyncMessenger::Worker-2": { 2024-08-19T01:43:37.726 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_recv_messages": 653, 2024-08-19T01:43:37.726 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_send_messages": 655, 2024-08-19T01:43:37.726 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_recv_bytes": 213051, 2024-08-19T01:43:37.726 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_send_bytes": 463959, 2024-08-19T01:43:37.726 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_created_connections": 15, 2024-08-19T01:43:37.726 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_active_connections": 5, 2024-08-19T01:43:37.726 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_running_total_time": 0.094720577, 2024-08-19T01:43:37.726 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_running_send_time": 0.029722188, 2024-08-19T01:43:37.727 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_running_recv_time": 0.025204752, 2024-08-19T01:43:37.727 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_running_fast_dispatch_time": 0.031426431, 2024-08-19T01:43:37.727 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_send_messages_queue_lat": { 2024-08-19T01:43:37.727 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 653, 2024-08-19T01:43:37.727 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.130949691, 2024-08-19T01:43:37.727 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000200535 2024-08-19T01:43:37.727 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:37.727 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_handle_ack_lat": { 2024-08-19T01:43:37.727 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 8, 2024-08-19T01:43:37.727 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000001022, 2024-08-19T01:43:37.727 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000127 2024-08-19T01:43:37.727 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:37.728 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_recv_encrypted_bytes": 0, 2024-08-19T01:43:37.728 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_send_encrypted_bytes": 0 2024-08-19T01:43:37.728 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:37.728 INFO:teuthology.orchestra.run.smithi083.stdout: "cct": { 2024-08-19T01:43:37.728 INFO:teuthology.orchestra.run.smithi083.stdout: "total_workers": 1, 2024-08-19T01:43:37.728 INFO:teuthology.orchestra.run.smithi083.stdout: "unhealthy_workers": 0 2024-08-19T01:43:37.728 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:37.728 INFO:teuthology.orchestra.run.smithi083.stdout: "finisher-MDSRank": { 2024-08-19T01:43:37.728 INFO:teuthology.orchestra.run.smithi083.stdout: "queue_len": 0, 2024-08-19T01:43:37.728 INFO:teuthology.orchestra.run.smithi083.stdout: "complete_latency": { 2024-08-19T01:43:37.728 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 1038, 2024-08-19T01:43:37.728 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 1.748444533, 2024-08-19T01:43:37.728 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.001684435 2024-08-19T01:43:37.728 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-08-19T01:43:37.729 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:37.729 INFO:teuthology.orchestra.run.smithi083.stdout: "finisher-PurgeQueue": { 2024-08-19T01:43:37.729 INFO:teuthology.orchestra.run.smithi083.stdout: "queue_len": 0, 2024-08-19T01:43:37.729 INFO:teuthology.orchestra.run.smithi083.stdout: "complete_latency": { 2024-08-19T01:43:37.729 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 806, 2024-08-19T01:43:37.729 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.133108225, 2024-08-19T01:43:37.729 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000165146 2024-08-19T01:43:37.729 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-08-19T01:43:37.729 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:37.729 INFO:teuthology.orchestra.run.smithi083.stdout: "mds": { 2024-08-19T01:43:37.729 INFO:teuthology.orchestra.run.smithi083.stdout: "request": 11933, 2024-08-19T01:43:37.729 INFO:teuthology.orchestra.run.smithi083.stdout: "reply": 11930, 2024-08-19T01:43:37.729 INFO:teuthology.orchestra.run.smithi083.stdout: "reply_latency": { 2024-08-19T01:43:37.729 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 11930, 2024-08-19T01:43:37.730 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 360.487457285, 2024-08-19T01:43:37.730 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.030216886 2024-08-19T01:43:37.730 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:37.730 INFO:teuthology.orchestra.run.smithi083.stdout: "slow_reply": 0, 2024-08-19T01:43:37.730 INFO:teuthology.orchestra.run.smithi083.stdout: "forward": 3, 2024-08-19T01:43:37.730 INFO:teuthology.orchestra.run.smithi083.stdout: "dir_fetch_complete": 0, 2024-08-19T01:43:37.730 INFO:teuthology.orchestra.run.smithi083.stdout: "dir_fetch_keys": 0, 2024-08-19T01:43:37.730 INFO:teuthology.orchestra.run.smithi083.stdout: "dir_commit": 20, 2024-08-19T01:43:37.730 INFO:teuthology.orchestra.run.smithi083.stdout: "dir_split": 1, 2024-08-19T01:43:37.730 INFO:teuthology.orchestra.run.smithi083.stdout: "dir_merge": 0, 2024-08-19T01:43:37.730 INFO:teuthology.orchestra.run.smithi083.stdout: "inodes": 16, 2024-08-19T01:43:37.730 INFO:teuthology.orchestra.run.smithi083.stdout: "inodes_top": 2, 2024-08-19T01:43:37.730 INFO:teuthology.orchestra.run.smithi083.stdout: "inodes_bottom": 1, 2024-08-19T01:43:37.731 INFO:teuthology.orchestra.run.smithi083.stdout: "inodes_pin_tail": 13, 2024-08-19T01:43:37.731 INFO:teuthology.orchestra.run.smithi083.stdout: "inodes_pinned": 13, 2024-08-19T01:43:37.731 INFO:teuthology.orchestra.run.smithi083.stdout: "inodes_expired": 57, 2024-08-19T01:43:37.731 INFO:teuthology.orchestra.run.smithi083.stdout: "inodes_with_caps": 1, 2024-08-19T01:43:37.731 INFO:teuthology.orchestra.run.smithi083.stdout: "caps": 1, 2024-08-19T01:43:37.731 INFO:teuthology.orchestra.run.smithi083.stdout: "subtrees": 3, 2024-08-19T01:43:37.731 INFO:teuthology.orchestra.run.smithi083.stdout: "traverse": 24205, 2024-08-19T01:43:37.731 INFO:teuthology.orchestra.run.smithi083.stdout: "traverse_hit": 17184, 2024-08-19T01:43:37.731 INFO:teuthology.orchestra.run.smithi083.stdout: "traverse_forward": 1, 2024-08-19T01:43:37.731 INFO:teuthology.orchestra.run.smithi083.stdout: "traverse_discover": 5, 2024-08-19T01:43:37.731 INFO:teuthology.orchestra.run.smithi083.stdout: "traverse_dir_fetch": 0, 2024-08-19T01:43:37.731 INFO:teuthology.orchestra.run.smithi083.stdout: "traverse_remote_ino": 0, 2024-08-19T01:43:37.731 INFO:teuthology.orchestra.run.smithi083.stdout: "traverse_lock": 41, 2024-08-19T01:43:37.731 INFO:teuthology.orchestra.run.smithi083.stdout: "load_cent": 0, 2024-08-19T01:43:37.731 INFO:teuthology.orchestra.run.smithi083.stdout: "q": 0, 2024-08-19T01:43:37.732 INFO:teuthology.orchestra.run.smithi083.stdout: "exported": 10, 2024-08-19T01:43:37.732 INFO:teuthology.orchestra.run.smithi083.stdout: "exported_inodes": 28, 2024-08-19T01:43:37.732 INFO:teuthology.orchestra.run.smithi083.stdout: "imported": 3, 2024-08-19T01:43:37.732 INFO:teuthology.orchestra.run.smithi083.stdout: "imported_inodes": 465, 2024-08-19T01:43:37.732 INFO:teuthology.orchestra.run.smithi083.stdout: "openino_dir_fetch": 0, 2024-08-19T01:43:37.732 INFO:teuthology.orchestra.run.smithi083.stdout: "openino_backtrace_fetch": 845, 2024-08-19T01:43:37.732 INFO:teuthology.orchestra.run.smithi083.stdout: "openino_peer_discover": 1, 2024-08-19T01:43:37.732 INFO:teuthology.orchestra.run.smithi083.stdout: "root_rfiles": 4, 2024-08-19T01:43:37.732 INFO:teuthology.orchestra.run.smithi083.stdout: "root_rbytes": 234, 2024-08-19T01:43:37.732 INFO:teuthology.orchestra.run.smithi083.stdout: "root_rsnaps": 0, 2024-08-19T01:43:37.732 INFO:teuthology.orchestra.run.smithi083.stdout: "scrub_backtrace_fetch": 845, 2024-08-19T01:43:37.732 INFO:teuthology.orchestra.run.smithi083.stdout: "scrub_set_tag": 0, 2024-08-19T01:43:37.732 INFO:teuthology.orchestra.run.smithi083.stdout: "scrub_backtrace_repaired": 0, 2024-08-19T01:43:37.732 INFO:teuthology.orchestra.run.smithi083.stdout: "scrub_inotable_repaired": 0, 2024-08-19T01:43:37.733 INFO:teuthology.orchestra.run.smithi083.stdout: "scrub_dir_inodes": 0, 2024-08-19T01:43:37.733 INFO:teuthology.orchestra.run.smithi083.stdout: "scrub_dir_base_inodes": 0, 2024-08-19T01:43:37.733 INFO:teuthology.orchestra.run.smithi083.stdout: "scrub_dirfrag_rstats": 0, 2024-08-19T01:43:37.733 INFO:teuthology.orchestra.run.smithi083.stdout: "scrub_file_inodes": 845, 2024-08-19T01:43:37.733 INFO:teuthology.orchestra.run.smithi083.stdout: "handle_inode_file_caps": 45, 2024-08-19T01:43:37.733 INFO:teuthology.orchestra.run.smithi083.stdout: "ceph_cap_op_revoke": 480, 2024-08-19T01:43:37.733 INFO:teuthology.orchestra.run.smithi083.stdout: "ceph_cap_op_grant": 839, 2024-08-19T01:43:37.733 INFO:teuthology.orchestra.run.smithi083.stdout: "ceph_cap_op_trunc": 0, 2024-08-19T01:43:37.733 INFO:teuthology.orchestra.run.smithi083.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-08-19T01:43:37.733 INFO:teuthology.orchestra.run.smithi083.stdout: "ceph_cap_op_flush_ack": 0, 2024-08-19T01:43:37.733 INFO:teuthology.orchestra.run.smithi083.stdout: "handle_client_caps": 1053, 2024-08-19T01:43:37.733 INFO:teuthology.orchestra.run.smithi083.stdout: "handle_client_caps_dirty": 448, 2024-08-19T01:43:37.733 INFO:teuthology.orchestra.run.smithi083.stdout: "handle_client_cap_release": 33, 2024-08-19T01:43:37.733 INFO:teuthology.orchestra.run.smithi083.stdout: "process_request_cap_release": 1546 2024-08-19T01:43:37.734 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:37.734 INFO:teuthology.orchestra.run.smithi083.stdout: "mds_cache": { 2024-08-19T01:43:37.734 INFO:teuthology.orchestra.run.smithi083.stdout: "dir_update": 0, 2024-08-19T01:43:37.734 INFO:teuthology.orchestra.run.smithi083.stdout: "dir_update_receipt": 0, 2024-08-19T01:43:37.734 INFO:teuthology.orchestra.run.smithi083.stdout: "dir_try_discover": 0, 2024-08-19T01:43:37.734 INFO:teuthology.orchestra.run.smithi083.stdout: "dir_send_discover": 6, 2024-08-19T01:43:37.734 INFO:teuthology.orchestra.run.smithi083.stdout: "dir_handle_discover": 11, 2024-08-19T01:43:37.734 INFO:teuthology.orchestra.run.smithi083.stdout: "num_strays": 0, 2024-08-19T01:43:37.734 INFO:teuthology.orchestra.run.smithi083.stdout: "num_strays_delayed": 0, 2024-08-19T01:43:37.734 INFO:teuthology.orchestra.run.smithi083.stdout: "num_strays_enqueuing": 0, 2024-08-19T01:43:37.734 INFO:teuthology.orchestra.run.smithi083.stdout: "strays_created": 448, 2024-08-19T01:43:37.734 INFO:teuthology.orchestra.run.smithi083.stdout: "strays_enqueued": 448, 2024-08-19T01:43:37.734 INFO:teuthology.orchestra.run.smithi083.stdout: "strays_reintegrated": 0, 2024-08-19T01:43:37.735 INFO:teuthology.orchestra.run.smithi083.stdout: "strays_migrated": 0, 2024-08-19T01:43:37.735 INFO:teuthology.orchestra.run.smithi083.stdout: "num_recovering_processing": 0, 2024-08-19T01:43:37.735 INFO:teuthology.orchestra.run.smithi083.stdout: "num_recovering_enqueued": 0, 2024-08-19T01:43:37.735 INFO:teuthology.orchestra.run.smithi083.stdout: "num_recovering_prioritized": 0, 2024-08-19T01:43:37.735 INFO:teuthology.orchestra.run.smithi083.stdout: "recovery_started": 0, 2024-08-19T01:43:37.735 INFO:teuthology.orchestra.run.smithi083.stdout: "recovery_completed": 0, 2024-08-19T01:43:37.735 INFO:teuthology.orchestra.run.smithi083.stdout: "ireq_quiesce_path": 0, 2024-08-19T01:43:37.735 INFO:teuthology.orchestra.run.smithi083.stdout: "ireq_quiesce_inode": 0, 2024-08-19T01:43:37.735 INFO:teuthology.orchestra.run.smithi083.stdout: "ireq_enqueue_scrub": 0, 2024-08-19T01:43:37.735 INFO:teuthology.orchestra.run.smithi083.stdout: "ireq_exportdir": 10, 2024-08-19T01:43:37.735 INFO:teuthology.orchestra.run.smithi083.stdout: "ireq_flush": 0, 2024-08-19T01:43:37.735 INFO:teuthology.orchestra.run.smithi083.stdout: "ireq_fragmentdir": 1, 2024-08-19T01:43:37.735 INFO:teuthology.orchestra.run.smithi083.stdout: "ireq_fragstats": 0, 2024-08-19T01:43:37.735 INFO:teuthology.orchestra.run.smithi083.stdout: "ireq_inodestats": 0 2024-08-19T01:43:37.735 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:37.736 INFO:teuthology.orchestra.run.smithi083.stdout: "mds_log": { 2024-08-19T01:43:37.736 INFO:teuthology.orchestra.run.smithi083.stdout: "evlrg": 0, 2024-08-19T01:43:37.736 INFO:teuthology.orchestra.run.smithi083.stdout: "evadd": 2278, 2024-08-19T01:43:37.736 INFO:teuthology.orchestra.run.smithi083.stdout: "evex": 0, 2024-08-19T01:43:37.736 INFO:teuthology.orchestra.run.smithi083.stdout: "evtrm": 0, 2024-08-19T01:43:37.736 INFO:teuthology.orchestra.run.smithi083.stdout: "ev": 2278, 2024-08-19T01:43:37.736 INFO:teuthology.orchestra.run.smithi083.stdout: "evexg": 0, 2024-08-19T01:43:37.736 INFO:teuthology.orchestra.run.smithi083.stdout: "evexd": 0, 2024-08-19T01:43:37.736 INFO:teuthology.orchestra.run.smithi083.stdout: "segadd": 4, 2024-08-19T01:43:37.736 INFO:teuthology.orchestra.run.smithi083.stdout: "segex": 0, 2024-08-19T01:43:37.736 INFO:teuthology.orchestra.run.smithi083.stdout: "segtrm": 0, 2024-08-19T01:43:37.736 INFO:teuthology.orchestra.run.smithi083.stdout: "seg": 4, 2024-08-19T01:43:37.736 INFO:teuthology.orchestra.run.smithi083.stdout: "segmjr": 2, 2024-08-19T01:43:37.736 INFO:teuthology.orchestra.run.smithi083.stdout: "segexg": 0, 2024-08-19T01:43:37.737 INFO:teuthology.orchestra.run.smithi083.stdout: "segexd": 0, 2024-08-19T01:43:37.737 INFO:teuthology.orchestra.run.smithi083.stdout: "expos": 4194304, 2024-08-19T01:43:37.737 INFO:teuthology.orchestra.run.smithi083.stdout: "wrpos": 7552280, 2024-08-19T01:43:37.737 INFO:teuthology.orchestra.run.smithi083.stdout: "rdpos": 0, 2024-08-19T01:43:37.737 INFO:teuthology.orchestra.run.smithi083.stdout: "jlat": { 2024-08-19T01:43:37.737 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 127, 2024-08-19T01:43:37.737 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.689681036, 2024-08-19T01:43:37.737 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.005430559 2024-08-19T01:43:37.737 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:37.737 INFO:teuthology.orchestra.run.smithi083.stdout: "replayed": 0 2024-08-19T01:43:37.737 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:37.737 INFO:teuthology.orchestra.run.smithi083.stdout: "mds_mem": { 2024-08-19T01:43:37.737 INFO:teuthology.orchestra.run.smithi083.stdout: "ino": 20, 2024-08-19T01:43:37.737 INFO:teuthology.orchestra.run.smithi083.stdout: "ino+": 477, 2024-08-19T01:43:37.738 INFO:teuthology.orchestra.run.smithi083.stdout: "ino-": 457, 2024-08-19T01:43:37.738 INFO:teuthology.orchestra.run.smithi083.stdout: "dir": 19, 2024-08-19T01:43:37.738 INFO:teuthology.orchestra.run.smithi083.stdout: "dir+": 38, 2024-08-19T01:43:37.738 INFO:teuthology.orchestra.run.smithi083.stdout: "dir-": 19, 2024-08-19T01:43:37.738 INFO:teuthology.orchestra.run.smithi083.stdout: "dn": 16, 2024-08-19T01:43:37.738 INFO:teuthology.orchestra.run.smithi083.stdout: "dn+": 1214, 2024-08-19T01:43:37.738 INFO:teuthology.orchestra.run.smithi083.stdout: "dn-": 1198, 2024-08-19T01:43:37.738 INFO:teuthology.orchestra.run.smithi083.stdout: "cap": 1, 2024-08-19T01:43:37.738 INFO:teuthology.orchestra.run.smithi083.stdout: "cap+": 800, 2024-08-19T01:43:37.738 INFO:teuthology.orchestra.run.smithi083.stdout: "cap-": 799, 2024-08-19T01:43:37.738 INFO:teuthology.orchestra.run.smithi083.stdout: "rss": 59488, 2024-08-19T01:43:37.738 INFO:teuthology.orchestra.run.smithi083.stdout: "heap": 223520 2024-08-19T01:43:37.738 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:37.738 INFO:teuthology.orchestra.run.smithi083.stdout: "mds_server": { 2024-08-19T01:43:37.739 INFO:teuthology.orchestra.run.smithi083.stdout: "dispatch_client_request": 12998, 2024-08-19T01:43:37.739 INFO:teuthology.orchestra.run.smithi083.stdout: "dispatch_server_request": 125, 2024-08-19T01:43:37.739 INFO:teuthology.orchestra.run.smithi083.stdout: "handle_client_request": 11933, 2024-08-19T01:43:37.739 INFO:teuthology.orchestra.run.smithi083.stdout: "handle_client_session": 15, 2024-08-19T01:43:37.739 INFO:teuthology.orchestra.run.smithi083.stdout: "handle_peer_request": 1131, 2024-08-19T01:43:37.739 INFO:teuthology.orchestra.run.smithi083.stdout: "req_create_latency": { 2024-08-19T01:43:37.739 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 251, 2024-08-19T01:43:37.739 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 2.157526666, 2024-08-19T01:43:37.739 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.008595723 2024-08-19T01:43:37.739 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:37.739 INFO:teuthology.orchestra.run.smithi083.stdout: "req_getattr_latency": { 2024-08-19T01:43:37.739 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 1, 2024-08-19T01:43:37.739 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.001224815, 2024-08-19T01:43:37.739 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.001224815 2024-08-19T01:43:37.739 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:37.740 INFO:teuthology.orchestra.run.smithi083.stdout: "req_getfilelock_latency": { 2024-08-19T01:43:37.740 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-08-19T01:43:37.740 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-08-19T01:43:37.740 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-08-19T01:43:37.740 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:37.740 INFO:teuthology.orchestra.run.smithi083.stdout: "req_link_latency": { 2024-08-19T01:43:37.740 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-08-19T01:43:37.740 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-08-19T01:43:37.740 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-08-19T01:43:37.740 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:37.740 INFO:teuthology.orchestra.run.smithi083.stdout: "req_lookup_latency": { 2024-08-19T01:43:37.740 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 10844, 2024-08-19T01:43:37.740 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 354.644144751, 2024-08-19T01:43:37.740 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.032704181 2024-08-19T01:43:37.741 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:37.741 INFO:teuthology.orchestra.run.smithi083.stdout: "req_lookuphash_latency": { 2024-08-19T01:43:37.741 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-08-19T01:43:37.741 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-08-19T01:43:37.741 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-08-19T01:43:37.741 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:37.741 INFO:teuthology.orchestra.run.smithi083.stdout: "req_lookupino_latency": { 2024-08-19T01:43:37.741 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 1, 2024-08-19T01:43:37.741 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000102288, 2024-08-19T01:43:37.741 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000102288 2024-08-19T01:43:37.741 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:37.741 INFO:teuthology.orchestra.run.smithi083.stdout: "req_lookupname_latency": { 2024-08-19T01:43:37.741 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-08-19T01:43:37.741 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-08-19T01:43:37.741 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-08-19T01:43:37.742 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:37.742 INFO:teuthology.orchestra.run.smithi083.stdout: "req_lookupparent_latency": { 2024-08-19T01:43:37.742 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-08-19T01:43:37.742 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-08-19T01:43:37.742 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-08-19T01:43:37.742 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:37.742 INFO:teuthology.orchestra.run.smithi083.stdout: "req_lookupsnap_latency": { 2024-08-19T01:43:37.742 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-08-19T01:43:37.742 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-08-19T01:43:37.742 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-08-19T01:43:37.742 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:37.742 INFO:teuthology.orchestra.run.smithi083.stdout: "req_lssnap_latency": { 2024-08-19T01:43:37.742 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-08-19T01:43:37.742 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-08-19T01:43:37.743 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-08-19T01:43:37.743 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:37.743 INFO:teuthology.orchestra.run.smithi083.stdout: "req_mkdir_latency": { 2024-08-19T01:43:37.743 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-08-19T01:43:37.743 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-08-19T01:43:37.744 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-08-19T01:43:37.744 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:37.744 INFO:teuthology.orchestra.run.smithi083.stdout: "req_mknod_latency": { 2024-08-19T01:43:37.744 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-08-19T01:43:37.744 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-08-19T01:43:37.744 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-08-19T01:43:37.744 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:37.744 INFO:teuthology.orchestra.run.smithi083.stdout: "req_mksnap_latency": { 2024-08-19T01:43:37.744 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-08-19T01:43:37.744 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-08-19T01:43:37.744 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-08-19T01:43:37.744 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:37.745 INFO:teuthology.orchestra.run.smithi083.stdout: "req_open_latency": { 2024-08-19T01:43:37.745 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 227, 2024-08-19T01:43:37.745 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.668939430, 2024-08-19T01:43:37.745 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.002946869 2024-08-19T01:43:37.745 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:37.745 INFO:teuthology.orchestra.run.smithi083.stdout: "req_readdir_latency": { 2024-08-19T01:43:37.745 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 10, 2024-08-19T01:43:37.745 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.008682666, 2024-08-19T01:43:37.745 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000868266 2024-08-19T01:43:37.745 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:37.745 INFO:teuthology.orchestra.run.smithi083.stdout: "req_rename_latency": { 2024-08-19T01:43:37.746 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 251, 2024-08-19T01:43:37.746 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 2.429111726, 2024-08-19T01:43:37.746 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.009677735 2024-08-19T01:43:37.746 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:37.746 INFO:teuthology.orchestra.run.smithi083.stdout: "req_renamesnap_latency": { 2024-08-19T01:43:37.746 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-08-19T01:43:37.746 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-08-19T01:43:37.746 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-08-19T01:43:37.746 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:37.746 INFO:teuthology.orchestra.run.smithi083.stdout: "req_snapdiff_latency": { 2024-08-19T01:43:37.746 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-08-19T01:43:37.746 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-08-19T01:43:37.746 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-08-19T01:43:37.747 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:37.747 INFO:teuthology.orchestra.run.smithi083.stdout: "req_rmdir_latency": { 2024-08-19T01:43:37.747 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-08-19T01:43:37.747 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-08-19T01:43:37.747 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-08-19T01:43:37.747 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:37.747 INFO:teuthology.orchestra.run.smithi083.stdout: "req_rmsnap_latency": { 2024-08-19T01:43:37.747 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-08-19T01:43:37.747 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-08-19T01:43:37.747 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-08-19T01:43:37.747 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:37.747 INFO:teuthology.orchestra.run.smithi083.stdout: "req_rmxattr_latency": { 2024-08-19T01:43:37.747 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-08-19T01:43:37.748 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-08-19T01:43:37.748 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-08-19T01:43:37.748 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:37.748 INFO:teuthology.orchestra.run.smithi083.stdout: "req_setattr_latency": { 2024-08-19T01:43:37.748 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-08-19T01:43:37.748 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-08-19T01:43:37.748 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-08-19T01:43:37.748 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:37.748 INFO:teuthology.orchestra.run.smithi083.stdout: "req_setdirlayout_latency": { 2024-08-19T01:43:37.748 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-08-19T01:43:37.748 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-08-19T01:43:37.748 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-08-19T01:43:37.748 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:37.748 INFO:teuthology.orchestra.run.smithi083.stdout: "req_setfilelock_latency": { 2024-08-19T01:43:37.749 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-08-19T01:43:37.749 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-08-19T01:43:37.749 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-08-19T01:43:37.749 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:37.749 INFO:teuthology.orchestra.run.smithi083.stdout: "req_setlayout_latency": { 2024-08-19T01:43:37.749 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-08-19T01:43:37.749 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-08-19T01:43:37.749 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-08-19T01:43:37.749 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:37.749 INFO:teuthology.orchestra.run.smithi083.stdout: "req_setxattr_latency": { 2024-08-19T01:43:37.749 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-08-19T01:43:37.749 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-08-19T01:43:37.749 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-08-19T01:43:37.749 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:37.750 INFO:teuthology.orchestra.run.smithi083.stdout: "req_symlink_latency": { 2024-08-19T01:43:37.750 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-08-19T01:43:37.750 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-08-19T01:43:37.750 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-08-19T01:43:37.750 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:37.750 INFO:teuthology.orchestra.run.smithi083.stdout: "req_unlink_latency": { 2024-08-19T01:43:37.750 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 345, 2024-08-19T01:43:37.750 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.577724943, 2024-08-19T01:43:37.750 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.001674565 2024-08-19T01:43:37.750 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:37.750 INFO:teuthology.orchestra.run.smithi083.stdout: "cap_revoke_eviction": 0, 2024-08-19T01:43:37.750 INFO:teuthology.orchestra.run.smithi083.stdout: "cap_acquisition_throttle": 0, 2024-08-19T01:43:37.750 INFO:teuthology.orchestra.run.smithi083.stdout: "req_getvxattr_latency": { 2024-08-19T01:43:37.751 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-08-19T01:43:37.751 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-08-19T01:43:37.751 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-08-19T01:43:37.751 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-08-19T01:43:37.751 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:37.751 INFO:teuthology.orchestra.run.smithi083.stdout: "mds_sessions": { 2024-08-19T01:43:37.751 INFO:teuthology.orchestra.run.smithi083.stdout: "session_count": 1, 2024-08-19T01:43:37.751 INFO:teuthology.orchestra.run.smithi083.stdout: "session_add": 3, 2024-08-19T01:43:37.751 INFO:teuthology.orchestra.run.smithi083.stdout: "session_remove": 2, 2024-08-19T01:43:37.751 INFO:teuthology.orchestra.run.smithi083.stdout: "sessions_open": 1, 2024-08-19T01:43:37.751 INFO:teuthology.orchestra.run.smithi083.stdout: "sessions_stale": 0, 2024-08-19T01:43:37.751 INFO:teuthology.orchestra.run.smithi083.stdout: "total_load": 1655, 2024-08-19T01:43:37.751 INFO:teuthology.orchestra.run.smithi083.stdout: "average_load": 1655, 2024-08-19T01:43:37.751 INFO:teuthology.orchestra.run.smithi083.stdout: "avg_session_uptime": 522, 2024-08-19T01:43:37.751 INFO:teuthology.orchestra.run.smithi083.stdout: "mdthresh_evicted": 0 2024-08-19T01:43:37.752 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:37.752 INFO:teuthology.orchestra.run.smithi083.stdout: "mempool": { 2024-08-19T01:43:37.752 INFO:teuthology.orchestra.run.smithi083.stdout: "bloom_filter_bytes": 0, 2024-08-19T01:43:37.752 INFO:teuthology.orchestra.run.smithi083.stdout: "bloom_filter_items": 0, 2024-08-19T01:43:37.752 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_alloc_bytes": 0, 2024-08-19T01:43:37.752 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_alloc_items": 0, 2024-08-19T01:43:37.752 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_cache_data_bytes": 0, 2024-08-19T01:43:37.752 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_cache_data_items": 0, 2024-08-19T01:43:37.752 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_cache_onode_bytes": 0, 2024-08-19T01:43:37.752 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_cache_onode_items": 0, 2024-08-19T01:43:37.752 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_cache_meta_bytes": 0, 2024-08-19T01:43:37.752 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_cache_meta_items": 0, 2024-08-19T01:43:37.752 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_cache_other_bytes": 0, 2024-08-19T01:43:37.752 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_cache_other_items": 0, 2024-08-19T01:43:37.752 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_cache_buffer_bytes": 0, 2024-08-19T01:43:37.753 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_cache_buffer_items": 0, 2024-08-19T01:43:37.753 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_extent_bytes": 0, 2024-08-19T01:43:37.753 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_extent_items": 0, 2024-08-19T01:43:37.753 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_blob_bytes": 0, 2024-08-19T01:43:37.753 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_blob_items": 0, 2024-08-19T01:43:37.753 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_shared_blob_bytes": 0, 2024-08-19T01:43:37.753 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_shared_blob_items": 0, 2024-08-19T01:43:37.753 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_inline_bl_bytes": 0, 2024-08-19T01:43:37.753 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_inline_bl_items": 0, 2024-08-19T01:43:37.753 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_fsck_bytes": 0, 2024-08-19T01:43:37.753 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_fsck_items": 0, 2024-08-19T01:43:37.753 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_txc_bytes": 0, 2024-08-19T01:43:37.753 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_txc_items": 0, 2024-08-19T01:43:37.754 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_writing_deferred_bytes": 0, 2024-08-19T01:43:37.754 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_writing_deferred_items": 0, 2024-08-19T01:43:37.754 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_writing_bytes": 0, 2024-08-19T01:43:37.754 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_writing_items": 0, 2024-08-19T01:43:37.754 INFO:teuthology.orchestra.run.smithi083.stdout: "bluefs_bytes": 0, 2024-08-19T01:43:37.754 INFO:teuthology.orchestra.run.smithi083.stdout: "bluefs_items": 0, 2024-08-19T01:43:37.754 INFO:teuthology.orchestra.run.smithi083.stdout: "bluefs_file_reader_bytes": 0, 2024-08-19T01:43:37.754 INFO:teuthology.orchestra.run.smithi083.stdout: "bluefs_file_reader_items": 0, 2024-08-19T01:43:37.754 INFO:teuthology.orchestra.run.smithi083.stdout: "bluefs_file_writer_bytes": 0, 2024-08-19T01:43:37.754 INFO:teuthology.orchestra.run.smithi083.stdout: "bluefs_file_writer_items": 0, 2024-08-19T01:43:37.754 INFO:teuthology.orchestra.run.smithi083.stdout: "buffer_anon_bytes": 293519, 2024-08-19T01:43:37.754 INFO:teuthology.orchestra.run.smithi083.stdout: "buffer_anon_items": 123, 2024-08-19T01:43:37.754 INFO:teuthology.orchestra.run.smithi083.stdout: "buffer_meta_bytes": 880, 2024-08-19T01:43:37.755 INFO:teuthology.orchestra.run.smithi083.stdout: "buffer_meta_items": 10, 2024-08-19T01:43:37.755 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_bytes": 0, 2024-08-19T01:43:37.755 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_items": 0, 2024-08-19T01:43:37.755 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_mapbl_bytes": 0, 2024-08-19T01:43:37.755 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_mapbl_items": 0, 2024-08-19T01:43:37.755 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_pglog_bytes": 0, 2024-08-19T01:43:37.755 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_pglog_items": 0, 2024-08-19T01:43:37.755 INFO:teuthology.orchestra.run.smithi083.stdout: "osdmap_bytes": 7688, 2024-08-19T01:43:37.755 INFO:teuthology.orchestra.run.smithi083.stdout: "osdmap_items": 158, 2024-08-19T01:43:37.755 INFO:teuthology.orchestra.run.smithi083.stdout: "osdmap_mapping_bytes": 0, 2024-08-19T01:43:37.756 INFO:teuthology.orchestra.run.smithi083.stdout: "osdmap_mapping_items": 0, 2024-08-19T01:43:37.756 INFO:teuthology.orchestra.run.smithi083.stdout: "pgmap_bytes": 0, 2024-08-19T01:43:37.756 INFO:teuthology.orchestra.run.smithi083.stdout: "pgmap_items": 0, 2024-08-19T01:43:37.756 INFO:teuthology.orchestra.run.smithi083.stdout: "mds_co_bytes": 82109, 2024-08-19T01:43:37.756 INFO:teuthology.orchestra.run.smithi083.stdout: "mds_co_items": 382, 2024-08-19T01:43:37.756 INFO:teuthology.orchestra.run.smithi083.stdout: "unittest_1_bytes": 0, 2024-08-19T01:43:37.756 INFO:teuthology.orchestra.run.smithi083.stdout: "unittest_1_items": 0, 2024-08-19T01:43:37.756 INFO:teuthology.orchestra.run.smithi083.stdout: "unittest_2_bytes": 0, 2024-08-19T01:43:37.756 INFO:teuthology.orchestra.run.smithi083.stdout: "unittest_2_items": 0 2024-08-19T01:43:37.756 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:37.756 INFO:teuthology.orchestra.run.smithi083.stdout: "objecter": { 2024-08-19T01:43:37.756 INFO:teuthology.orchestra.run.smithi083.stdout: "op_active": 0, 2024-08-19T01:43:37.756 INFO:teuthology.orchestra.run.smithi083.stdout: "op_laggy": 0, 2024-08-19T01:43:37.756 INFO:teuthology.orchestra.run.smithi083.stdout: "op_send": 1651, 2024-08-19T01:43:37.757 INFO:teuthology.orchestra.run.smithi083.stdout: "op_send_bytes": 3657828, 2024-08-19T01:43:37.757 INFO:teuthology.orchestra.run.smithi083.stdout: "op_resend": 33, 2024-08-19T01:43:37.757 INFO:teuthology.orchestra.run.smithi083.stdout: "op_reply": 1651, 2024-08-19T01:43:37.757 INFO:teuthology.orchestra.run.smithi083.stdout: "op_latency": { 2024-08-19T01:43:37.757 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 1651, 2024-08-19T01:43:37.757 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 15.430635573, 2024-08-19T01:43:37.757 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.009346235 2024-08-19T01:43:37.757 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:37.757 INFO:teuthology.orchestra.run.smithi083.stdout: "op_inflight": 0, 2024-08-19T01:43:37.757 INFO:teuthology.orchestra.run.smithi083.stdout: "oplen_avg": { 2024-08-19T01:43:37.757 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 1651, 2024-08-19T01:43:37.757 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 1763 2024-08-19T01:43:37.757 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:37.757 INFO:teuthology.orchestra.run.smithi083.stdout: "op": 1651, 2024-08-19T01:43:37.758 INFO:teuthology.orchestra.run.smithi083.stdout: "op_r": 880, 2024-08-19T01:43:37.758 INFO:teuthology.orchestra.run.smithi083.stdout: "op_w": 771, 2024-08-19T01:43:37.758 INFO:teuthology.orchestra.run.smithi083.stdout: "op_rmw": 0, 2024-08-19T01:43:37.758 INFO:teuthology.orchestra.run.smithi083.stdout: "op_pg": 0, 2024-08-19T01:43:37.758 INFO:teuthology.orchestra.run.smithi083.stdout: "osdop_stat": 0, 2024-08-19T01:43:37.758 INFO:teuthology.orchestra.run.smithi083.stdout: "osdop_create": 10, 2024-08-19T01:43:37.758 INFO:teuthology.orchestra.run.smithi083.stdout: "osdop_read": 35, 2024-08-19T01:43:37.758 INFO:teuthology.orchestra.run.smithi083.stdout: "osdop_write": 162, 2024-08-19T01:43:37.758 INFO:teuthology.orchestra.run.smithi083.stdout: "osdop_writefull": 46, 2024-08-19T01:43:37.758 INFO:teuthology.orchestra.run.smithi083.stdout: "osdop_writesame": 0, 2024-08-19T01:43:37.758 INFO:teuthology.orchestra.run.smithi083.stdout: "osdop_append": 0, 2024-08-19T01:43:37.758 INFO:teuthology.orchestra.run.smithi083.stdout: "osdop_zero": 0, 2024-08-19T01:43:37.758 INFO:teuthology.orchestra.run.smithi083.stdout: "osdop_truncate": 1, 2024-08-19T01:43:37.759 INFO:teuthology.orchestra.run.smithi083.stdout: "osdop_delete": 460, 2024-08-19T01:43:37.759 INFO:teuthology.orchestra.run.smithi083.stdout: "osdop_mapext": 0, 2024-08-19T01:43:37.759 INFO:teuthology.orchestra.run.smithi083.stdout: "osdop_sparse_read": 0, 2024-08-19T01:43:37.759 INFO:teuthology.orchestra.run.smithi083.stdout: "osdop_clonerange": 0, 2024-08-19T01:43:37.759 INFO:teuthology.orchestra.run.smithi083.stdout: "osdop_getxattr": 845, 2024-08-19T01:43:37.759 INFO:teuthology.orchestra.run.smithi083.stdout: "osdop_setxattr": 20, 2024-08-19T01:43:37.759 INFO:teuthology.orchestra.run.smithi083.stdout: "osdop_cmpxattr": 0, 2024-08-19T01:43:37.759 INFO:teuthology.orchestra.run.smithi083.stdout: "osdop_rmxattr": 0, 2024-08-19T01:43:37.759 INFO:teuthology.orchestra.run.smithi083.stdout: "osdop_resetxattrs": 0, 2024-08-19T01:43:37.759 INFO:teuthology.orchestra.run.smithi083.stdout: "osdop_call": 0, 2024-08-19T01:43:37.759 INFO:teuthology.orchestra.run.smithi083.stdout: "osdop_watch": 0, 2024-08-19T01:43:37.759 INFO:teuthology.orchestra.run.smithi083.stdout: "osdop_notify": 0, 2024-08-19T01:43:37.760 INFO:teuthology.orchestra.run.smithi083.stdout: "osdop_src_cmpxattr": 0, 2024-08-19T01:43:37.760 INFO:teuthology.orchestra.run.smithi083.stdout: "osdop_pgls": 0, 2024-08-19T01:43:37.760 INFO:teuthology.orchestra.run.smithi083.stdout: "osdop_pgls_filter": 0, 2024-08-19T01:43:37.760 INFO:teuthology.orchestra.run.smithi083.stdout: "osdop_other": 0, 2024-08-19T01:43:37.760 INFO:teuthology.orchestra.run.smithi083.stdout: "linger_active": 0, 2024-08-19T01:43:37.760 INFO:teuthology.orchestra.run.smithi083.stdout: "linger_send": 0, 2024-08-19T01:43:37.760 INFO:teuthology.orchestra.run.smithi083.stdout: "linger_resend": 0, 2024-08-19T01:43:37.760 INFO:teuthology.orchestra.run.smithi083.stdout: "linger_ping": 0, 2024-08-19T01:43:37.760 INFO:teuthology.orchestra.run.smithi083.stdout: "poolop_active": 0, 2024-08-19T01:43:37.760 INFO:teuthology.orchestra.run.smithi083.stdout: "poolop_send": 0, 2024-08-19T01:43:37.760 INFO:teuthology.orchestra.run.smithi083.stdout: "poolop_resend": 0, 2024-08-19T01:43:37.760 INFO:teuthology.orchestra.run.smithi083.stdout: "poolstat_active": 0, 2024-08-19T01:43:37.760 INFO:teuthology.orchestra.run.smithi083.stdout: "poolstat_send": 0, 2024-08-19T01:43:37.761 INFO:teuthology.orchestra.run.smithi083.stdout: "poolstat_resend": 0, 2024-08-19T01:43:37.761 INFO:teuthology.orchestra.run.smithi083.stdout: "statfs_active": 0, 2024-08-19T01:43:37.761 INFO:teuthology.orchestra.run.smithi083.stdout: "statfs_send": 0, 2024-08-19T01:43:37.761 INFO:teuthology.orchestra.run.smithi083.stdout: "statfs_resend": 0, 2024-08-19T01:43:37.761 INFO:teuthology.orchestra.run.smithi083.stdout: "command_active": 0, 2024-08-19T01:43:37.761 INFO:teuthology.orchestra.run.smithi083.stdout: "command_send": 0, 2024-08-19T01:43:37.761 INFO:teuthology.orchestra.run.smithi083.stdout: "command_resend": 0, 2024-08-19T01:43:37.761 INFO:teuthology.orchestra.run.smithi083.stdout: "map_epoch": 86, 2024-08-19T01:43:37.761 INFO:teuthology.orchestra.run.smithi083.stdout: "map_full": 0, 2024-08-19T01:43:37.761 INFO:teuthology.orchestra.run.smithi083.stdout: "map_inc": 2, 2024-08-19T01:43:37.761 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_sessions": 12, 2024-08-19T01:43:37.761 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_session_open": 12, 2024-08-19T01:43:37.761 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_session_close": 0, 2024-08-19T01:43:37.762 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_laggy": 0, 2024-08-19T01:43:37.762 INFO:teuthology.orchestra.run.smithi083.stdout: "omap_wr": 149, 2024-08-19T01:43:37.762 INFO:teuthology.orchestra.run.smithi083.stdout: "omap_rd": 0, 2024-08-19T01:43:37.762 INFO:teuthology.orchestra.run.smithi083.stdout: "omap_del": 35 2024-08-19T01:43:37.762 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:37.762 INFO:teuthology.orchestra.run.smithi083.stdout: "oft": { 2024-08-19T01:43:37.762 INFO:teuthology.orchestra.run.smithi083.stdout: "omap_total_objs": 1, 2024-08-19T01:43:37.762 INFO:teuthology.orchestra.run.smithi083.stdout: "omap_total_kv_pairs": 0, 2024-08-19T01:43:37.762 INFO:teuthology.orchestra.run.smithi083.stdout: "omap_total_updates": 0, 2024-08-19T01:43:37.762 INFO:teuthology.orchestra.run.smithi083.stdout: "omap_total_removes": 0 2024-08-19T01:43:37.762 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:37.762 INFO:teuthology.orchestra.run.smithi083.stdout: "osd-slow-ops": { 2024-08-19T01:43:37.762 INFO:teuthology.orchestra.run.smithi083.stdout: "slow_ops_count": 0 2024-08-19T01:43:37.762 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:37.763 INFO:teuthology.orchestra.run.smithi083.stdout: "purge_queue": { 2024-08-19T01:43:37.763 INFO:teuthology.orchestra.run.smithi083.stdout: "pq_executing_ops": 0, 2024-08-19T01:43:37.763 INFO:teuthology.orchestra.run.smithi083.stdout: "pq_executing_ops_high_water": 10, 2024-08-19T01:43:37.763 INFO:teuthology.orchestra.run.smithi083.stdout: "pq_executing": 0, 2024-08-19T01:43:37.763 INFO:teuthology.orchestra.run.smithi083.stdout: "pq_executing_high_water": 10, 2024-08-19T01:43:37.763 INFO:teuthology.orchestra.run.smithi083.stdout: "pq_executed_ops": 448, 2024-08-19T01:43:37.763 INFO:teuthology.orchestra.run.smithi083.stdout: "pq_executed": 448, 2024-08-19T01:43:37.763 INFO:teuthology.orchestra.run.smithi083.stdout: "pq_item_in_journal": 0 2024-08-19T01:43:37.763 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:37.763 INFO:teuthology.orchestra.run.smithi083.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-08-19T01:43:37.763 INFO:teuthology.orchestra.run.smithi083.stdout: "val": 0, 2024-08-19T01:43:37.763 INFO:teuthology.orchestra.run.smithi083.stdout: "max": 104857600, 2024-08-19T01:43:37.763 INFO:teuthology.orchestra.run.smithi083.stdout: "get_started": 0, 2024-08-19T01:43:37.763 INFO:teuthology.orchestra.run.smithi083.stdout: "get": 20383, 2024-08-19T01:43:37.764 INFO:teuthology.orchestra.run.smithi083.stdout: "get_sum": 5488580, 2024-08-19T01:43:37.764 INFO:teuthology.orchestra.run.smithi083.stdout: "get_or_fail_fail": 0, 2024-08-19T01:43:37.764 INFO:teuthology.orchestra.run.smithi083.stdout: "get_or_fail_success": 20383, 2024-08-19T01:43:37.764 INFO:teuthology.orchestra.run.smithi083.stdout: "take": 0, 2024-08-19T01:43:37.764 INFO:teuthology.orchestra.run.smithi083.stdout: "take_sum": 0, 2024-08-19T01:43:37.764 INFO:teuthology.orchestra.run.smithi083.stdout: "put": 20383, 2024-08-19T01:43:37.764 INFO:teuthology.orchestra.run.smithi083.stdout: "put_sum": 5488580, 2024-08-19T01:43:37.764 INFO:teuthology.orchestra.run.smithi083.stdout: "wait": { 2024-08-19T01:43:37.764 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-08-19T01:43:37.764 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-08-19T01:43:37.764 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-08-19T01:43:37.764 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-08-19T01:43:37.764 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:37.764 INFO:teuthology.orchestra.run.smithi083.stdout: "throttle-objecter_bytes": { 2024-08-19T01:43:37.765 INFO:teuthology.orchestra.run.smithi083.stdout: "val": 0, 2024-08-19T01:43:37.765 INFO:teuthology.orchestra.run.smithi083.stdout: "max": 104857600, 2024-08-19T01:43:37.765 INFO:teuthology.orchestra.run.smithi083.stdout: "get_started": 0, 2024-08-19T01:43:37.765 INFO:teuthology.orchestra.run.smithi083.stdout: "get": 0, 2024-08-19T01:43:37.765 INFO:teuthology.orchestra.run.smithi083.stdout: "get_sum": 0, 2024-08-19T01:43:37.765 INFO:teuthology.orchestra.run.smithi083.stdout: "get_or_fail_fail": 0, 2024-08-19T01:43:37.765 INFO:teuthology.orchestra.run.smithi083.stdout: "get_or_fail_success": 0, 2024-08-19T01:43:37.765 INFO:teuthology.orchestra.run.smithi083.stdout: "take": 1651, 2024-08-19T01:43:37.765 INFO:teuthology.orchestra.run.smithi083.stdout: "take_sum": 3703076, 2024-08-19T01:43:37.765 INFO:teuthology.orchestra.run.smithi083.stdout: "put": 1651, 2024-08-19T01:43:37.765 INFO:teuthology.orchestra.run.smithi083.stdout: "put_sum": 3703076, 2024-08-19T01:43:37.765 INFO:teuthology.orchestra.run.smithi083.stdout: "wait": { 2024-08-19T01:43:37.765 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-08-19T01:43:37.765 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-08-19T01:43:37.766 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-08-19T01:43:37.766 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-08-19T01:43:37.766 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:37.766 INFO:teuthology.orchestra.run.smithi083.stdout: "throttle-objecter_ops": { 2024-08-19T01:43:37.766 INFO:teuthology.orchestra.run.smithi083.stdout: "val": 0, 2024-08-19T01:43:37.766 INFO:teuthology.orchestra.run.smithi083.stdout: "max": 1024, 2024-08-19T01:43:37.766 INFO:teuthology.orchestra.run.smithi083.stdout: "get_started": 0, 2024-08-19T01:43:37.766 INFO:teuthology.orchestra.run.smithi083.stdout: "get": 0, 2024-08-19T01:43:37.766 INFO:teuthology.orchestra.run.smithi083.stdout: "get_sum": 0, 2024-08-19T01:43:37.766 INFO:teuthology.orchestra.run.smithi083.stdout: "get_or_fail_fail": 0, 2024-08-19T01:43:37.766 INFO:teuthology.orchestra.run.smithi083.stdout: "get_or_fail_success": 0, 2024-08-19T01:43:37.766 INFO:teuthology.orchestra.run.smithi083.stdout: "take": 1651, 2024-08-19T01:43:37.767 INFO:teuthology.orchestra.run.smithi083.stdout: "take_sum": 1651, 2024-08-19T01:43:37.767 INFO:teuthology.orchestra.run.smithi083.stdout: "put": 1651, 2024-08-19T01:43:37.767 INFO:teuthology.orchestra.run.smithi083.stdout: "put_sum": 1651, 2024-08-19T01:43:37.767 INFO:teuthology.orchestra.run.smithi083.stdout: "wait": { 2024-08-19T01:43:37.767 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-08-19T01:43:37.767 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-08-19T01:43:37.767 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-08-19T01:43:37.767 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-08-19T01:43:37.767 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:37.767 INFO:teuthology.orchestra.run.smithi083.stdout: "throttle-write_buf_throttle": { 2024-08-19T01:43:37.767 INFO:teuthology.orchestra.run.smithi083.stdout: "val": 0, 2024-08-19T01:43:37.767 INFO:teuthology.orchestra.run.smithi083.stdout: "max": 3758096384, 2024-08-19T01:43:37.767 INFO:teuthology.orchestra.run.smithi083.stdout: "get_started": 0, 2024-08-19T01:43:37.767 INFO:teuthology.orchestra.run.smithi083.stdout: "get": 448, 2024-08-19T01:43:37.768 INFO:teuthology.orchestra.run.smithi083.stdout: "get_sum": 45248, 2024-08-19T01:43:37.768 INFO:teuthology.orchestra.run.smithi083.stdout: "get_or_fail_fail": 0, 2024-08-19T01:43:37.768 INFO:teuthology.orchestra.run.smithi083.stdout: "get_or_fail_success": 448, 2024-08-19T01:43:37.768 INFO:teuthology.orchestra.run.smithi083.stdout: "take": 0, 2024-08-19T01:43:37.768 INFO:teuthology.orchestra.run.smithi083.stdout: "take_sum": 0, 2024-08-19T01:43:37.768 INFO:teuthology.orchestra.run.smithi083.stdout: "put": 35, 2024-08-19T01:43:37.768 INFO:teuthology.orchestra.run.smithi083.stdout: "put_sum": 45248, 2024-08-19T01:43:37.768 INFO:teuthology.orchestra.run.smithi083.stdout: "wait": { 2024-08-19T01:43:37.768 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-08-19T01:43:37.768 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-08-19T01:43:37.768 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-08-19T01:43:37.768 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-08-19T01:43:37.768 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:37.768 INFO:teuthology.orchestra.run.smithi083.stdout: "throttle-write_buf_throttle-0x55a1e692e960": { 2024-08-19T01:43:37.769 INFO:teuthology.orchestra.run.smithi083.stdout: "val": 0, 2024-08-19T01:43:37.769 INFO:teuthology.orchestra.run.smithi083.stdout: "max": 3758096384, 2024-08-19T01:43:37.769 INFO:teuthology.orchestra.run.smithi083.stdout: "get_started": 0, 2024-08-19T01:43:37.769 INFO:teuthology.orchestra.run.smithi083.stdout: "get": 2278, 2024-08-19T01:43:37.769 INFO:teuthology.orchestra.run.smithi083.stdout: "get_sum": 3357976, 2024-08-19T01:43:37.769 INFO:teuthology.orchestra.run.smithi083.stdout: "get_or_fail_fail": 0, 2024-08-19T01:43:37.769 INFO:teuthology.orchestra.run.smithi083.stdout: "get_or_fail_success": 2278, 2024-08-19T01:43:37.769 INFO:teuthology.orchestra.run.smithi083.stdout: "take": 0, 2024-08-19T01:43:37.769 INFO:teuthology.orchestra.run.smithi083.stdout: "take_sum": 0, 2024-08-19T01:43:37.769 INFO:teuthology.orchestra.run.smithi083.stdout: "put": 127, 2024-08-19T01:43:37.769 INFO:teuthology.orchestra.run.smithi083.stdout: "put_sum": 3357976, 2024-08-19T01:43:37.769 INFO:teuthology.orchestra.run.smithi083.stdout: "wait": { 2024-08-19T01:43:37.769 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-08-19T01:43:37.769 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-08-19T01:43:37.770 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-08-19T01:43:37.770 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-08-19T01:43:37.770 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-08-19T01:43:37.770 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-08-19T01:43:37.904 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:43:37 smithi083 ceph-mon[17120]: pgmap v602: 97 pgs: 97 active+clean; 35 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-08-19T01:43:38.202 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:43:37 smithi117 ceph-mon[22016]: pgmap v602: 97 pgs: 97 active+clean; 35 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-08-19T01:43:38.216 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:43:37 smithi167 ceph-mon[21875]: pgmap v602: 97 pgs: 97 active+clean; 35 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-08-19T01:43:38.622 INFO:tasks.check_counter:Daemon mds.g mds.exported=10 2024-08-19T01:43:38.623 INFO:tasks.check_counter:Daemon mds.g mds.imported=3 2024-08-19T01:43:38.623 INFO:tasks.check_counter:Daemon mds.g mds.dir_split=1 2024-08-19T01:43:38.623 DEBUG:tasks.check_counter:Getting stats from j 2024-08-19T01:43:38.623 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833 shell --fsid bad8780e-5dc9-11ef-bccf-c7b262605968 -- ceph daemon mds.j perf dump 2024-08-19T01:43:39.000 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/bad8780e-5dc9-11ef-bccf-c7b262605968/mon.a/config 2024-08-19T01:43:39.612 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-08-19T01:43:39.612 INFO:teuthology.orchestra.run.smithi083.stdout: "AsyncMessenger::Worker-0": { 2024-08-19T01:43:39.613 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_recv_messages": 0, 2024-08-19T01:43:39.613 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_send_messages": 2, 2024-08-19T01:43:39.613 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_recv_bytes": 0, 2024-08-19T01:43:39.613 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_send_bytes": 0, 2024-08-19T01:43:39.614 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_created_connections": 4, 2024-08-19T01:43:39.614 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_active_connections": 0, 2024-08-19T01:43:39.614 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_running_total_time": 0.003043714, 2024-08-19T01:43:39.614 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_running_send_time": 0.000000000, 2024-08-19T01:43:39.614 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_running_recv_time": 0.000521479, 2024-08-19T01:43:39.614 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-08-19T01:43:39.614 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_send_messages_queue_lat": { 2024-08-19T01:43:39.614 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-08-19T01:43:39.614 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-08-19T01:43:39.614 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-08-19T01:43:39.614 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:39.614 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_handle_ack_lat": { 2024-08-19T01:43:39.614 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-08-19T01:43:39.614 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-08-19T01:43:39.615 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-08-19T01:43:39.615 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:39.615 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_recv_encrypted_bytes": 0, 2024-08-19T01:43:39.615 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_send_encrypted_bytes": 0 2024-08-19T01:43:39.615 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:39.615 INFO:teuthology.orchestra.run.smithi083.stdout: "AsyncMessenger::Worker-1": { 2024-08-19T01:43:39.615 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_recv_messages": 5, 2024-08-19T01:43:39.615 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_send_messages": 145, 2024-08-19T01:43:39.615 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_recv_bytes": 3184, 2024-08-19T01:43:39.615 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_send_bytes": 103840, 2024-08-19T01:43:39.615 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_created_connections": 3, 2024-08-19T01:43:39.615 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_active_connections": 2, 2024-08-19T01:43:39.615 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_running_total_time": 0.022937196, 2024-08-19T01:43:39.615 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_running_send_time": 0.016983572, 2024-08-19T01:43:39.615 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_running_recv_time": 0.004472910, 2024-08-19T01:43:39.616 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-08-19T01:43:39.616 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_send_messages_queue_lat": { 2024-08-19T01:43:39.616 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 144, 2024-08-19T01:43:39.616 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.022966150, 2024-08-19T01:43:39.616 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000159487 2024-08-19T01:43:39.616 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:39.616 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_handle_ack_lat": { 2024-08-19T01:43:39.616 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-08-19T01:43:39.616 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-08-19T01:43:39.616 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-08-19T01:43:39.616 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:39.616 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_recv_encrypted_bytes": 3184, 2024-08-19T01:43:39.616 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_send_encrypted_bytes": 103840 2024-08-19T01:43:39.617 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:39.617 INFO:teuthology.orchestra.run.smithi083.stdout: "AsyncMessenger::Worker-2": { 2024-08-19T01:43:39.617 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_recv_messages": 193, 2024-08-19T01:43:39.617 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_send_messages": 188, 2024-08-19T01:43:39.617 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_recv_bytes": 745952, 2024-08-19T01:43:39.617 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_send_bytes": 115408, 2024-08-19T01:43:39.617 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_created_connections": 2, 2024-08-19T01:43:39.617 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_active_connections": 1, 2024-08-19T01:43:39.617 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_running_total_time": 0.061049072, 2024-08-19T01:43:39.617 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_running_send_time": 0.031102400, 2024-08-19T01:43:39.617 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_running_recv_time": 0.036842223, 2024-08-19T01:43:39.617 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-08-19T01:43:39.618 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_send_messages_queue_lat": { 2024-08-19T01:43:39.618 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 187, 2024-08-19T01:43:39.618 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.024245135, 2024-08-19T01:43:39.618 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000129653 2024-08-19T01:43:39.618 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:39.618 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_handle_ack_lat": { 2024-08-19T01:43:39.618 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-08-19T01:43:39.618 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-08-19T01:43:39.618 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-08-19T01:43:39.618 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:39.618 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_recv_encrypted_bytes": 745952, 2024-08-19T01:43:39.618 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_send_encrypted_bytes": 115408 2024-08-19T01:43:39.619 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:39.619 INFO:teuthology.orchestra.run.smithi083.stdout: "cct": { 2024-08-19T01:43:39.619 INFO:teuthology.orchestra.run.smithi083.stdout: "total_workers": 0, 2024-08-19T01:43:39.619 INFO:teuthology.orchestra.run.smithi083.stdout: "unhealthy_workers": 0 2024-08-19T01:43:39.619 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:39.619 INFO:teuthology.orchestra.run.smithi083.stdout: "mempool": { 2024-08-19T01:43:39.619 INFO:teuthology.orchestra.run.smithi083.stdout: "bloom_filter_bytes": 0, 2024-08-19T01:43:39.619 INFO:teuthology.orchestra.run.smithi083.stdout: "bloom_filter_items": 0, 2024-08-19T01:43:39.619 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_alloc_bytes": 0, 2024-08-19T01:43:39.619 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_alloc_items": 0, 2024-08-19T01:43:39.619 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_cache_data_bytes": 0, 2024-08-19T01:43:39.619 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_cache_data_items": 0, 2024-08-19T01:43:39.619 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_cache_onode_bytes": 0, 2024-08-19T01:43:39.620 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_cache_onode_items": 0, 2024-08-19T01:43:39.620 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_cache_meta_bytes": 0, 2024-08-19T01:43:39.620 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_cache_meta_items": 0, 2024-08-19T01:43:39.620 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_cache_other_bytes": 0, 2024-08-19T01:43:39.620 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_cache_other_items": 0, 2024-08-19T01:43:39.620 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_cache_buffer_bytes": 0, 2024-08-19T01:43:39.620 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_cache_buffer_items": 0, 2024-08-19T01:43:39.620 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_extent_bytes": 0, 2024-08-19T01:43:39.620 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_extent_items": 0, 2024-08-19T01:43:39.620 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_blob_bytes": 0, 2024-08-19T01:43:39.620 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_blob_items": 0, 2024-08-19T01:43:39.620 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_shared_blob_bytes": 0, 2024-08-19T01:43:39.620 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_shared_blob_items": 0, 2024-08-19T01:43:39.621 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_inline_bl_bytes": 0, 2024-08-19T01:43:39.621 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_inline_bl_items": 0, 2024-08-19T01:43:39.621 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_fsck_bytes": 0, 2024-08-19T01:43:39.621 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_fsck_items": 0, 2024-08-19T01:43:39.621 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_txc_bytes": 0, 2024-08-19T01:43:39.621 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_txc_items": 0, 2024-08-19T01:43:39.621 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_writing_deferred_bytes": 0, 2024-08-19T01:43:39.621 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_writing_deferred_items": 0, 2024-08-19T01:43:39.621 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_writing_bytes": 0, 2024-08-19T01:43:39.621 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_writing_items": 0, 2024-08-19T01:43:39.621 INFO:teuthology.orchestra.run.smithi083.stdout: "bluefs_bytes": 0, 2024-08-19T01:43:39.621 INFO:teuthology.orchestra.run.smithi083.stdout: "bluefs_items": 0, 2024-08-19T01:43:39.621 INFO:teuthology.orchestra.run.smithi083.stdout: "bluefs_file_reader_bytes": 0, 2024-08-19T01:43:39.621 INFO:teuthology.orchestra.run.smithi083.stdout: "bluefs_file_reader_items": 0, 2024-08-19T01:43:39.622 INFO:teuthology.orchestra.run.smithi083.stdout: "bluefs_file_writer_bytes": 0, 2024-08-19T01:43:39.622 INFO:teuthology.orchestra.run.smithi083.stdout: "bluefs_file_writer_items": 0, 2024-08-19T01:43:39.622 INFO:teuthology.orchestra.run.smithi083.stdout: "buffer_anon_bytes": 148984, 2024-08-19T01:43:39.622 INFO:teuthology.orchestra.run.smithi083.stdout: "buffer_anon_items": 30, 2024-08-19T01:43:39.622 INFO:teuthology.orchestra.run.smithi083.stdout: "buffer_meta_bytes": 0, 2024-08-19T01:43:39.622 INFO:teuthology.orchestra.run.smithi083.stdout: "buffer_meta_items": 0, 2024-08-19T01:43:39.622 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_bytes": 0, 2024-08-19T01:43:39.622 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_items": 0, 2024-08-19T01:43:39.622 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_mapbl_bytes": 0, 2024-08-19T01:43:39.622 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_mapbl_items": 0, 2024-08-19T01:43:39.622 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_pglog_bytes": 0, 2024-08-19T01:43:39.622 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_pglog_items": 0, 2024-08-19T01:43:39.623 INFO:teuthology.orchestra.run.smithi083.stdout: "osdmap_bytes": 0, 2024-08-19T01:43:39.623 INFO:teuthology.orchestra.run.smithi083.stdout: "osdmap_items": 0, 2024-08-19T01:43:39.623 INFO:teuthology.orchestra.run.smithi083.stdout: "osdmap_mapping_bytes": 0, 2024-08-19T01:43:39.623 INFO:teuthology.orchestra.run.smithi083.stdout: "osdmap_mapping_items": 0, 2024-08-19T01:43:39.623 INFO:teuthology.orchestra.run.smithi083.stdout: "pgmap_bytes": 0, 2024-08-19T01:43:39.623 INFO:teuthology.orchestra.run.smithi083.stdout: "pgmap_items": 0, 2024-08-19T01:43:39.623 INFO:teuthology.orchestra.run.smithi083.stdout: "mds_co_bytes": 928, 2024-08-19T01:43:39.623 INFO:teuthology.orchestra.run.smithi083.stdout: "mds_co_items": 2, 2024-08-19T01:43:39.623 INFO:teuthology.orchestra.run.smithi083.stdout: "unittest_1_bytes": 0, 2024-08-19T01:43:39.623 INFO:teuthology.orchestra.run.smithi083.stdout: "unittest_1_items": 0, 2024-08-19T01:43:39.623 INFO:teuthology.orchestra.run.smithi083.stdout: "unittest_2_bytes": 0, 2024-08-19T01:43:39.623 INFO:teuthology.orchestra.run.smithi083.stdout: "unittest_2_items": 0 2024-08-19T01:43:39.623 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-19T01:43:39.624 INFO:teuthology.orchestra.run.smithi083.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-08-19T01:43:39.624 INFO:teuthology.orchestra.run.smithi083.stdout: "val": 0, 2024-08-19T01:43:39.624 INFO:teuthology.orchestra.run.smithi083.stdout: "max": 104857600, 2024-08-19T01:43:39.624 INFO:teuthology.orchestra.run.smithi083.stdout: "get_started": 0, 2024-08-19T01:43:39.624 INFO:teuthology.orchestra.run.smithi083.stdout: "get": 194, 2024-08-19T01:43:39.624 INFO:teuthology.orchestra.run.smithi083.stdout: "get_sum": 718711, 2024-08-19T01:43:39.624 INFO:teuthology.orchestra.run.smithi083.stdout: "get_or_fail_fail": 0, 2024-08-19T01:43:39.624 INFO:teuthology.orchestra.run.smithi083.stdout: "get_or_fail_success": 194, 2024-08-19T01:43:39.624 INFO:teuthology.orchestra.run.smithi083.stdout: "take": 0, 2024-08-19T01:43:39.624 INFO:teuthology.orchestra.run.smithi083.stdout: "take_sum": 0, 2024-08-19T01:43:39.624 INFO:teuthology.orchestra.run.smithi083.stdout: "put": 194, 2024-08-19T01:43:39.624 INFO:teuthology.orchestra.run.smithi083.stdout: "put_sum": 718711, 2024-08-19T01:43:39.624 INFO:teuthology.orchestra.run.smithi083.stdout: "wait": { 2024-08-19T01:43:39.624 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-08-19T01:43:39.625 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-08-19T01:43:39.625 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-08-19T01:43:39.625 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-08-19T01:43:39.625 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-08-19T01:43:39.625 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-08-19T01:43:40.032 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:43:39 smithi083 ceph-mon[17120]: pgmap v603: 97 pgs: 97 active+clean; 35 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-08-19T01:43:40.202 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:43:39 smithi117 ceph-mon[22016]: pgmap v603: 97 pgs: 97 active+clean; 35 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-08-19T01:43:40.215 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:43:39 smithi167 ceph-mon[21875]: pgmap v603: 97 pgs: 97 active+clean; 35 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-08-19T01:43:40.375 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.j 2024-08-19T01:43:40.375 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.j 2024-08-19T01:43:40.375 WARNING:tasks.check_counter:Counter 'mds.dir_split' not found on daemon mds.j 2024-08-19T01:43:40.375 DEBUG:tasks.check_counter:Getting stats from b 2024-08-19T01:43:40.376 DEBUG:teuthology.orchestra.run.smithi117:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833 shell --fsid bad8780e-5dc9-11ef-bccf-c7b262605968 -- ceph daemon mds.b perf dump 2024-08-19T01:43:40.755 INFO:teuthology.orchestra.run.smithi117.stderr:Inferring config /var/lib/ceph/bad8780e-5dc9-11ef-bccf-c7b262605968/mon.b/config 2024-08-19T01:43:41.304 INFO:teuthology.orchestra.run.smithi117.stdout:{ 2024-08-19T01:43:41.304 INFO:teuthology.orchestra.run.smithi117.stdout: "AsyncMessenger::Worker-0": { 2024-08-19T01:43:41.304 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_recv_messages": 81147, 2024-08-19T01:43:41.305 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_send_messages": 80498, 2024-08-19T01:43:41.305 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_recv_bytes": 36860950, 2024-08-19T01:43:41.305 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_send_bytes": 426777040, 2024-08-19T01:43:41.305 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_created_connections": 106, 2024-08-19T01:43:41.305 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_active_connections": 7, 2024-08-19T01:43:41.305 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_running_total_time": 7.004148096, 2024-08-19T01:43:41.305 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_running_send_time": 3.063761228, 2024-08-19T01:43:41.305 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_running_recv_time": 1.243401867, 2024-08-19T01:43:41.305 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_running_fast_dispatch_time": 2.359912445, 2024-08-19T01:43:41.305 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_send_messages_queue_lat": { 2024-08-19T01:43:41.305 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 80499, 2024-08-19T01:43:41.305 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 45.178952759, 2024-08-19T01:43:41.305 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000561236 2024-08-19T01:43:41.305 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-08-19T01:43:41.305 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_handle_ack_lat": { 2024-08-19T01:43:41.306 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 6777, 2024-08-19T01:43:41.306 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.001143368, 2024-08-19T01:43:41.306 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000168 2024-08-19T01:43:41.306 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-08-19T01:43:41.306 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_recv_encrypted_bytes": 39312, 2024-08-19T01:43:41.306 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_send_encrypted_bytes": 1456 2024-08-19T01:43:41.306 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-08-19T01:43:41.306 INFO:teuthology.orchestra.run.smithi117.stdout: "AsyncMessenger::Worker-1": { 2024-08-19T01:43:41.306 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_recv_messages": 439726, 2024-08-19T01:43:41.306 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_send_messages": 702543, 2024-08-19T01:43:41.306 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_recv_bytes": 164165349, 2024-08-19T01:43:41.306 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_send_bytes": 567713776, 2024-08-19T01:43:41.306 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_created_connections": 42, 2024-08-19T01:43:41.306 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_active_connections": 5, 2024-08-19T01:43:41.307 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_running_total_time": 20.676906242, 2024-08-19T01:43:41.307 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_running_send_time": 9.797446031, 2024-08-19T01:43:41.307 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_running_recv_time": 28.449260590, 2024-08-19T01:43:41.307 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_running_fast_dispatch_time": 2.809607390, 2024-08-19T01:43:41.307 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_send_messages_queue_lat": { 2024-08-19T01:43:41.307 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 702542, 2024-08-19T01:43:41.307 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 91.945089166, 2024-08-19T01:43:41.307 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000130874 2024-08-19T01:43:41.307 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-08-19T01:43:41.307 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_handle_ack_lat": { 2024-08-19T01:43:41.307 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 394615, 2024-08-19T01:43:41.307 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.095700772, 2024-08-19T01:43:41.307 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000242 2024-08-19T01:43:41.307 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-08-19T01:43:41.307 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_recv_encrypted_bytes": 51136, 2024-08-19T01:43:41.308 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_send_encrypted_bytes": 359008 2024-08-19T01:43:41.308 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-08-19T01:43:41.308 INFO:teuthology.orchestra.run.smithi117.stdout: "AsyncMessenger::Worker-2": { 2024-08-19T01:43:41.308 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_recv_messages": 84580, 2024-08-19T01:43:41.308 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_send_messages": 80120, 2024-08-19T01:43:41.308 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_recv_bytes": 35449999, 2024-08-19T01:43:41.308 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_send_bytes": 230482652, 2024-08-19T01:43:41.308 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_created_connections": 8, 2024-08-19T01:43:41.308 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_active_connections": 7, 2024-08-19T01:43:41.308 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_running_total_time": 6.613074941, 2024-08-19T01:43:41.308 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_running_send_time": 2.706630672, 2024-08-19T01:43:41.308 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_running_recv_time": 1.623864668, 2024-08-19T01:43:41.308 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_running_fast_dispatch_time": 2.140320483, 2024-08-19T01:43:41.308 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_send_messages_queue_lat": { 2024-08-19T01:43:41.309 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 80116, 2024-08-19T01:43:41.309 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 41.944919521, 2024-08-19T01:43:41.309 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000523552 2024-08-19T01:43:41.309 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-08-19T01:43:41.309 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_handle_ack_lat": { 2024-08-19T01:43:41.309 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 29031, 2024-08-19T01:43:41.309 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.006490185, 2024-08-19T01:43:41.309 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000223 2024-08-19T01:43:41.309 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-08-19T01:43:41.309 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_recv_encrypted_bytes": 817216, 2024-08-19T01:43:41.309 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_send_encrypted_bytes": 130368 2024-08-19T01:43:41.309 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-08-19T01:43:41.310 INFO:teuthology.orchestra.run.smithi117.stdout: "cct": { 2024-08-19T01:43:41.310 INFO:teuthology.orchestra.run.smithi117.stdout: "total_workers": 1, 2024-08-19T01:43:41.310 INFO:teuthology.orchestra.run.smithi117.stdout: "unhealthy_workers": 0 2024-08-19T01:43:41.310 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-08-19T01:43:41.310 INFO:teuthology.orchestra.run.smithi117.stdout: "finisher-MDSRank": { 2024-08-19T01:43:41.310 INFO:teuthology.orchestra.run.smithi117.stdout: "queue_len": 0, 2024-08-19T01:43:41.310 INFO:teuthology.orchestra.run.smithi117.stdout: "complete_latency": { 2024-08-19T01:43:41.310 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 46872, 2024-08-19T01:43:41.310 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 301.827315498, 2024-08-19T01:43:41.310 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.006439394 2024-08-19T01:43:41.310 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-08-19T01:43:41.310 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-08-19T01:43:41.310 INFO:teuthology.orchestra.run.smithi117.stdout: "finisher-PurgeQueue": { 2024-08-19T01:43:41.310 INFO:teuthology.orchestra.run.smithi117.stdout: "queue_len": 0, 2024-08-19T01:43:41.310 INFO:teuthology.orchestra.run.smithi117.stdout: "complete_latency": { 2024-08-19T01:43:41.311 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 103448, 2024-08-19T01:43:41.311 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 36.574287823, 2024-08-19T01:43:41.311 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000353552 2024-08-19T01:43:41.311 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-08-19T01:43:41.311 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-08-19T01:43:41.311 INFO:teuthology.orchestra.run.smithi117.stdout: "mds": { 2024-08-19T01:43:41.311 INFO:teuthology.orchestra.run.smithi117.stdout: "request": 194767, 2024-08-19T01:43:41.311 INFO:teuthology.orchestra.run.smithi117.stdout: "reply": 191848, 2024-08-19T01:43:41.311 INFO:teuthology.orchestra.run.smithi117.stdout: "reply_latency": { 2024-08-19T01:43:41.311 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 191848, 2024-08-19T01:43:41.311 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 3614.323135407, 2024-08-19T01:43:41.311 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.018839514 2024-08-19T01:43:41.311 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-08-19T01:43:41.311 INFO:teuthology.orchestra.run.smithi117.stdout: "slow_reply": 0, 2024-08-19T01:43:41.311 INFO:teuthology.orchestra.run.smithi117.stdout: "forward": 2919, 2024-08-19T01:43:41.312 INFO:teuthology.orchestra.run.smithi117.stdout: "dir_fetch_complete": 10, 2024-08-19T01:43:41.312 INFO:teuthology.orchestra.run.smithi117.stdout: "dir_fetch_keys": 2, 2024-08-19T01:43:41.312 INFO:teuthology.orchestra.run.smithi117.stdout: "dir_commit": 13214, 2024-08-19T01:43:41.312 INFO:teuthology.orchestra.run.smithi117.stdout: "dir_split": 202, 2024-08-19T01:43:41.312 INFO:teuthology.orchestra.run.smithi117.stdout: "dir_merge": 35, 2024-08-19T01:43:41.312 INFO:teuthology.orchestra.run.smithi117.stdout: "inodes": 781, 2024-08-19T01:43:41.312 INFO:teuthology.orchestra.run.smithi117.stdout: "inodes_top": 256, 2024-08-19T01:43:41.312 INFO:teuthology.orchestra.run.smithi117.stdout: "inodes_bottom": 511, 2024-08-19T01:43:41.312 INFO:teuthology.orchestra.run.smithi117.stdout: "inodes_pin_tail": 14, 2024-08-19T01:43:41.312 INFO:teuthology.orchestra.run.smithi117.stdout: "inodes_pinned": 345, 2024-08-19T01:43:41.312 INFO:teuthology.orchestra.run.smithi117.stdout: "inodes_expired": 2517, 2024-08-19T01:43:41.312 INFO:teuthology.orchestra.run.smithi117.stdout: "inodes_with_caps": 2, 2024-08-19T01:43:41.312 INFO:teuthology.orchestra.run.smithi117.stdout: "caps": 2, 2024-08-19T01:43:41.313 INFO:teuthology.orchestra.run.smithi117.stdout: "subtrees": 4, 2024-08-19T01:43:41.313 INFO:teuthology.orchestra.run.smithi117.stdout: "traverse": 276966, 2024-08-19T01:43:41.313 INFO:teuthology.orchestra.run.smithi117.stdout: "traverse_hit": 260123, 2024-08-19T01:43:41.313 INFO:teuthology.orchestra.run.smithi117.stdout: "traverse_forward": 2913, 2024-08-19T01:43:41.313 INFO:teuthology.orchestra.run.smithi117.stdout: "traverse_discover": 53, 2024-08-19T01:43:41.313 INFO:teuthology.orchestra.run.smithi117.stdout: "traverse_dir_fetch": 0, 2024-08-19T01:43:41.313 INFO:teuthology.orchestra.run.smithi117.stdout: "traverse_remote_ino": 0, 2024-08-19T01:43:41.313 INFO:teuthology.orchestra.run.smithi117.stdout: "traverse_lock": 948, 2024-08-19T01:43:41.313 INFO:teuthology.orchestra.run.smithi117.stdout: "load_cent": 0, 2024-08-19T01:43:41.313 INFO:teuthology.orchestra.run.smithi117.stdout: "q": 0, 2024-08-19T01:43:41.313 INFO:teuthology.orchestra.run.smithi117.stdout: "exported": 10, 2024-08-19T01:43:41.313 INFO:teuthology.orchestra.run.smithi117.stdout: "exported_inodes": 1469, 2024-08-19T01:43:41.313 INFO:teuthology.orchestra.run.smithi117.stdout: "imported": 66, 2024-08-19T01:43:41.313 INFO:teuthology.orchestra.run.smithi117.stdout: "imported_inodes": 331, 2024-08-19T01:43:41.313 INFO:teuthology.orchestra.run.smithi117.stdout: "openino_dir_fetch": 0, 2024-08-19T01:43:41.313 INFO:teuthology.orchestra.run.smithi117.stdout: "openino_backtrace_fetch": 92398, 2024-08-19T01:43:41.314 INFO:teuthology.orchestra.run.smithi117.stdout: "openino_peer_discover": 0, 2024-08-19T01:43:41.314 INFO:teuthology.orchestra.run.smithi117.stdout: "root_rfiles": 4, 2024-08-19T01:43:41.314 INFO:teuthology.orchestra.run.smithi117.stdout: "root_rbytes": 234, 2024-08-19T01:43:41.314 INFO:teuthology.orchestra.run.smithi117.stdout: "root_rsnaps": 0, 2024-08-19T01:43:41.314 INFO:teuthology.orchestra.run.smithi117.stdout: "scrub_backtrace_fetch": 92398, 2024-08-19T01:43:41.314 INFO:teuthology.orchestra.run.smithi117.stdout: "scrub_set_tag": 0, 2024-08-19T01:43:41.314 INFO:teuthology.orchestra.run.smithi117.stdout: "scrub_backtrace_repaired": 0, 2024-08-19T01:43:41.314 INFO:teuthology.orchestra.run.smithi117.stdout: "scrub_inotable_repaired": 0, 2024-08-19T01:43:41.314 INFO:teuthology.orchestra.run.smithi117.stdout: "scrub_dir_inodes": 690, 2024-08-19T01:43:41.314 INFO:teuthology.orchestra.run.smithi117.stdout: "scrub_dir_base_inodes": 8, 2024-08-19T01:43:41.314 INFO:teuthology.orchestra.run.smithi117.stdout: "scrub_dirfrag_rstats": 682, 2024-08-19T01:43:41.314 INFO:teuthology.orchestra.run.smithi117.stdout: "scrub_file_inodes": 91708, 2024-08-19T01:43:41.314 INFO:teuthology.orchestra.run.smithi117.stdout: "handle_inode_file_caps": 21, 2024-08-19T01:43:41.315 INFO:teuthology.orchestra.run.smithi117.stdout: "ceph_cap_op_revoke": 50512, 2024-08-19T01:43:41.315 INFO:teuthology.orchestra.run.smithi117.stdout: "ceph_cap_op_grant": 130397, 2024-08-19T01:43:41.315 INFO:teuthology.orchestra.run.smithi117.stdout: "ceph_cap_op_trunc": 5, 2024-08-19T01:43:41.315 INFO:teuthology.orchestra.run.smithi117.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-08-19T01:43:41.315 INFO:teuthology.orchestra.run.smithi117.stdout: "ceph_cap_op_flush_ack": 0, 2024-08-19T01:43:41.315 INFO:teuthology.orchestra.run.smithi117.stdout: "handle_client_caps": 133385, 2024-08-19T01:43:41.315 INFO:teuthology.orchestra.run.smithi117.stdout: "handle_client_caps_dirty": 61550, 2024-08-19T01:43:41.315 INFO:teuthology.orchestra.run.smithi117.stdout: "handle_client_cap_release": 4961, 2024-08-19T01:43:41.315 INFO:teuthology.orchestra.run.smithi117.stdout: "process_request_cap_release": 127194 2024-08-19T01:43:41.315 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-08-19T01:43:41.315 INFO:teuthology.orchestra.run.smithi117.stdout: "mds_cache": { 2024-08-19T01:43:41.315 INFO:teuthology.orchestra.run.smithi117.stdout: "dir_update": 0, 2024-08-19T01:43:41.315 INFO:teuthology.orchestra.run.smithi117.stdout: "dir_update_receipt": 0, 2024-08-19T01:43:41.315 INFO:teuthology.orchestra.run.smithi117.stdout: "dir_try_discover": 0, 2024-08-19T01:43:41.315 INFO:teuthology.orchestra.run.smithi117.stdout: "dir_send_discover": 53, 2024-08-19T01:43:41.316 INFO:teuthology.orchestra.run.smithi117.stdout: "dir_handle_discover": 16, 2024-08-19T01:43:41.316 INFO:teuthology.orchestra.run.smithi117.stdout: "num_strays": 3, 2024-08-19T01:43:41.316 INFO:teuthology.orchestra.run.smithi117.stdout: "num_strays_delayed": 0, 2024-08-19T01:43:41.316 INFO:teuthology.orchestra.run.smithi117.stdout: "num_strays_enqueuing": 0, 2024-08-19T01:43:41.316 INFO:teuthology.orchestra.run.smithi117.stdout: "strays_created": 61098, 2024-08-19T01:43:41.316 INFO:teuthology.orchestra.run.smithi117.stdout: "strays_enqueued": 61095, 2024-08-19T01:43:41.316 INFO:teuthology.orchestra.run.smithi117.stdout: "strays_reintegrated": 0, 2024-08-19T01:43:41.316 INFO:teuthology.orchestra.run.smithi117.stdout: "strays_migrated": 0, 2024-08-19T01:43:41.316 INFO:teuthology.orchestra.run.smithi117.stdout: "num_recovering_processing": 0, 2024-08-19T01:43:41.316 INFO:teuthology.orchestra.run.smithi117.stdout: "num_recovering_enqueued": 0, 2024-08-19T01:43:41.317 INFO:teuthology.orchestra.run.smithi117.stdout: "num_recovering_prioritized": 0, 2024-08-19T01:43:41.317 INFO:teuthology.orchestra.run.smithi117.stdout: "recovery_started": 0, 2024-08-19T01:43:41.317 INFO:teuthology.orchestra.run.smithi117.stdout: "recovery_completed": 0, 2024-08-19T01:43:41.317 INFO:teuthology.orchestra.run.smithi117.stdout: "ireq_quiesce_path": 0, 2024-08-19T01:43:41.317 INFO:teuthology.orchestra.run.smithi117.stdout: "ireq_quiesce_inode": 0, 2024-08-19T01:43:41.317 INFO:teuthology.orchestra.run.smithi117.stdout: "ireq_enqueue_scrub": 8, 2024-08-19T01:43:41.317 INFO:teuthology.orchestra.run.smithi117.stdout: "ireq_exportdir": 10, 2024-08-19T01:43:41.317 INFO:teuthology.orchestra.run.smithi117.stdout: "ireq_flush": 0, 2024-08-19T01:43:41.317 INFO:teuthology.orchestra.run.smithi117.stdout: "ireq_fragmentdir": 237, 2024-08-19T01:43:41.317 INFO:teuthology.orchestra.run.smithi117.stdout: "ireq_fragstats": 0, 2024-08-19T01:43:41.317 INFO:teuthology.orchestra.run.smithi117.stdout: "ireq_inodestats": 0 2024-08-19T01:43:41.317 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-08-19T01:43:41.317 INFO:teuthology.orchestra.run.smithi117.stdout: "mds_log": { 2024-08-19T01:43:41.317 INFO:teuthology.orchestra.run.smithi117.stdout: "evlrg": 0, 2024-08-19T01:43:41.318 INFO:teuthology.orchestra.run.smithi117.stdout: "evadd": 323890, 2024-08-19T01:43:41.318 INFO:teuthology.orchestra.run.smithi117.stdout: "evex": 316643, 2024-08-19T01:43:41.318 INFO:teuthology.orchestra.run.smithi117.stdout: "evtrm": 316643, 2024-08-19T01:43:41.318 INFO:teuthology.orchestra.run.smithi117.stdout: "ev": 7247, 2024-08-19T01:43:41.318 INFO:teuthology.orchestra.run.smithi117.stdout: "evexg": 0, 2024-08-19T01:43:41.318 INFO:teuthology.orchestra.run.smithi117.stdout: "evexd": 13558, 2024-08-19T01:43:41.318 INFO:teuthology.orchestra.run.smithi117.stdout: "segadd": 427, 2024-08-19T01:43:41.318 INFO:teuthology.orchestra.run.smithi117.stdout: "segex": 418, 2024-08-19T01:43:41.318 INFO:teuthology.orchestra.run.smithi117.stdout: "segtrm": 418, 2024-08-19T01:43:41.318 INFO:teuthology.orchestra.run.smithi117.stdout: "seg": 9, 2024-08-19T01:43:41.318 INFO:teuthology.orchestra.run.smithi117.stdout: "segmjr": 1, 2024-08-19T01:43:41.318 INFO:teuthology.orchestra.run.smithi117.stdout: "segexg": 0, 2024-08-19T01:43:41.318 INFO:teuthology.orchestra.run.smithi117.stdout: "segexd": 17, 2024-08-19T01:43:41.319 INFO:teuthology.orchestra.run.smithi117.stdout: "expos": 722879719, 2024-08-19T01:43:41.319 INFO:teuthology.orchestra.run.smithi117.stdout: "wrpos": 735098979, 2024-08-19T01:43:41.319 INFO:teuthology.orchestra.run.smithi117.stdout: "rdpos": 0, 2024-08-19T01:43:41.319 INFO:teuthology.orchestra.run.smithi117.stdout: "jlat": { 2024-08-19T01:43:41.319 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 3237, 2024-08-19T01:43:41.319 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 789.142941411, 2024-08-19T01:43:41.319 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.243788366 2024-08-19T01:43:41.319 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-08-19T01:43:41.319 INFO:teuthology.orchestra.run.smithi117.stdout: "replayed": 0 2024-08-19T01:43:41.319 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-08-19T01:43:41.319 INFO:teuthology.orchestra.run.smithi117.stdout: "mds_mem": { 2024-08-19T01:43:41.319 INFO:teuthology.orchestra.run.smithi117.stdout: "ino": 40, 2024-08-19T01:43:41.319 INFO:teuthology.orchestra.run.smithi117.stdout: "ino+": 61896, 2024-08-19T01:43:41.319 INFO:teuthology.orchestra.run.smithi117.stdout: "ino-": 61856, 2024-08-19T01:43:41.319 INFO:teuthology.orchestra.run.smithi117.stdout: "dir": 42, 2024-08-19T01:43:41.320 INFO:teuthology.orchestra.run.smithi117.stdout: "dir+": 1978, 2024-08-19T01:43:41.320 INFO:teuthology.orchestra.run.smithi117.stdout: "dir-": 1936, 2024-08-19T01:43:41.320 INFO:teuthology.orchestra.run.smithi117.stdout: "dn": 781, 2024-08-19T01:43:41.320 INFO:teuthology.orchestra.run.smithi117.stdout: "dn+": 137295, 2024-08-19T01:43:41.320 INFO:teuthology.orchestra.run.smithi117.stdout: "dn-": 136514, 2024-08-19T01:43:41.320 INFO:teuthology.orchestra.run.smithi117.stdout: "cap": 2, 2024-08-19T01:43:41.320 INFO:teuthology.orchestra.run.smithi117.stdout: "cap+": 62864, 2024-08-19T01:43:41.320 INFO:teuthology.orchestra.run.smithi117.stdout: "cap-": 62862, 2024-08-19T01:43:41.320 INFO:teuthology.orchestra.run.smithi117.stdout: "rss": 175740, 2024-08-19T01:43:41.320 INFO:teuthology.orchestra.run.smithi117.stdout: "heap": 231712 2024-08-19T01:43:41.320 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-08-19T01:43:41.320 INFO:teuthology.orchestra.run.smithi117.stdout: "mds_server": { 2024-08-19T01:43:41.320 INFO:teuthology.orchestra.run.smithi117.stdout: "dispatch_client_request": 204650, 2024-08-19T01:43:41.320 INFO:teuthology.orchestra.run.smithi117.stdout: "dispatch_server_request": 5374, 2024-08-19T01:43:41.320 INFO:teuthology.orchestra.run.smithi117.stdout: "handle_client_request": 194767, 2024-08-19T01:43:41.321 INFO:teuthology.orchestra.run.smithi117.stdout: "handle_client_session": 28, 2024-08-19T01:43:41.321 INFO:teuthology.orchestra.run.smithi117.stdout: "handle_peer_request": 11596, 2024-08-19T01:43:41.321 INFO:teuthology.orchestra.run.smithi117.stdout: "req_create_latency": { 2024-08-19T01:43:41.321 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 61562, 2024-08-19T01:43:41.321 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 768.664317232, 2024-08-19T01:43:41.321 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.012486019 2024-08-19T01:43:41.321 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-08-19T01:43:41.321 INFO:teuthology.orchestra.run.smithi117.stdout: "req_getattr_latency": { 2024-08-19T01:43:41.321 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 3765, 2024-08-19T01:43:41.321 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 821.503639724, 2024-08-19T01:43:41.321 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.218194857 2024-08-19T01:43:41.321 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-08-19T01:43:41.321 INFO:teuthology.orchestra.run.smithi117.stdout: "req_getfilelock_latency": { 2024-08-19T01:43:41.321 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2024-08-19T01:43:41.322 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2024-08-19T01:43:41.322 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2024-08-19T01:43:41.322 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-08-19T01:43:41.322 INFO:teuthology.orchestra.run.smithi117.stdout: "req_link_latency": { 2024-08-19T01:43:41.322 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2024-08-19T01:43:41.322 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2024-08-19T01:43:41.322 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2024-08-19T01:43:41.322 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-08-19T01:43:41.322 INFO:teuthology.orchestra.run.smithi117.stdout: "req_lookup_latency": { 2024-08-19T01:43:41.322 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 15972, 2024-08-19T01:43:41.322 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 777.573587750, 2024-08-19T01:43:41.322 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.048683545 2024-08-19T01:43:41.323 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-08-19T01:43:41.323 INFO:teuthology.orchestra.run.smithi117.stdout: "req_lookuphash_latency": { 2024-08-19T01:43:41.323 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2024-08-19T01:43:41.323 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2024-08-19T01:43:41.323 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2024-08-19T01:43:41.323 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-08-19T01:43:41.323 INFO:teuthology.orchestra.run.smithi117.stdout: "req_lookupino_latency": { 2024-08-19T01:43:41.323 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 1, 2024-08-19T01:43:41.323 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000151211, 2024-08-19T01:43:41.323 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000151211 2024-08-19T01:43:41.323 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-08-19T01:43:41.323 INFO:teuthology.orchestra.run.smithi117.stdout: "req_lookupname_latency": { 2024-08-19T01:43:41.323 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2024-08-19T01:43:41.323 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2024-08-19T01:43:41.324 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2024-08-19T01:43:41.324 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-08-19T01:43:41.324 INFO:teuthology.orchestra.run.smithi117.stdout: "req_lookupparent_latency": { 2024-08-19T01:43:41.324 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2024-08-19T01:43:41.324 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2024-08-19T01:43:41.324 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2024-08-19T01:43:41.324 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-08-19T01:43:41.324 INFO:teuthology.orchestra.run.smithi117.stdout: "req_lookupsnap_latency": { 2024-08-19T01:43:41.324 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2024-08-19T01:43:41.324 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2024-08-19T01:43:41.324 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2024-08-19T01:43:41.324 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-08-19T01:43:41.324 INFO:teuthology.orchestra.run.smithi117.stdout: "req_lssnap_latency": { 2024-08-19T01:43:41.324 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2024-08-19T01:43:41.325 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2024-08-19T01:43:41.325 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2024-08-19T01:43:41.325 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-08-19T01:43:41.325 INFO:teuthology.orchestra.run.smithi117.stdout: "req_mkdir_latency": { 2024-08-19T01:43:41.325 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 235, 2024-08-19T01:43:41.325 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 6.415722859, 2024-08-19T01:43:41.325 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.027300948 2024-08-19T01:43:41.325 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-08-19T01:43:41.325 INFO:teuthology.orchestra.run.smithi117.stdout: "req_mknod_latency": { 2024-08-19T01:43:41.325 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2024-08-19T01:43:41.325 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2024-08-19T01:43:41.325 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2024-08-19T01:43:41.325 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-08-19T01:43:41.325 INFO:teuthology.orchestra.run.smithi117.stdout: "req_mksnap_latency": { 2024-08-19T01:43:41.326 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2024-08-19T01:43:41.326 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2024-08-19T01:43:41.326 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2024-08-19T01:43:41.326 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-08-19T01:43:41.326 INFO:teuthology.orchestra.run.smithi117.stdout: "req_open_latency": { 2024-08-19T01:43:41.326 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 12095, 2024-08-19T01:43:41.326 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 205.175938378, 2024-08-19T01:43:41.326 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.016963698 2024-08-19T01:43:41.326 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-08-19T01:43:41.326 INFO:teuthology.orchestra.run.smithi117.stdout: "req_readdir_latency": { 2024-08-19T01:43:41.326 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 1572, 2024-08-19T01:43:41.326 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 2.150291644, 2024-08-19T01:43:41.326 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.001367870 2024-08-19T01:43:41.326 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-08-19T01:43:41.327 INFO:teuthology.orchestra.run.smithi117.stdout: "req_rename_latency": { 2024-08-19T01:43:41.327 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 61180, 2024-08-19T01:43:41.327 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 947.731797175, 2024-08-19T01:43:41.327 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.015490876 2024-08-19T01:43:41.327 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-08-19T01:43:41.327 INFO:teuthology.orchestra.run.smithi117.stdout: "req_renamesnap_latency": { 2024-08-19T01:43:41.327 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2024-08-19T01:43:41.327 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2024-08-19T01:43:41.327 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2024-08-19T01:43:41.327 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-08-19T01:43:41.327 INFO:teuthology.orchestra.run.smithi117.stdout: "req_snapdiff_latency": { 2024-08-19T01:43:41.327 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2024-08-19T01:43:41.327 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2024-08-19T01:43:41.327 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2024-08-19T01:43:41.327 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-08-19T01:43:41.327 INFO:teuthology.orchestra.run.smithi117.stdout: "req_rmdir_latency": { 2024-08-19T01:43:41.328 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 229, 2024-08-19T01:43:41.328 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 27.119108649, 2024-08-19T01:43:41.328 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.118424055 2024-08-19T01:43:41.328 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-08-19T01:43:41.328 INFO:teuthology.orchestra.run.smithi117.stdout: "req_rmsnap_latency": { 2024-08-19T01:43:41.328 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2024-08-19T01:43:41.328 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2024-08-19T01:43:41.328 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2024-08-19T01:43:41.328 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-08-19T01:43:41.328 INFO:teuthology.orchestra.run.smithi117.stdout: "req_rmxattr_latency": { 2024-08-19T01:43:41.328 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2024-08-19T01:43:41.328 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2024-08-19T01:43:41.328 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2024-08-19T01:43:41.328 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-08-19T01:43:41.329 INFO:teuthology.orchestra.run.smithi117.stdout: "req_setattr_latency": { 2024-08-19T01:43:41.329 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 6, 2024-08-19T01:43:41.329 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.028346146, 2024-08-19T01:43:41.329 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.004724357 2024-08-19T01:43:41.329 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-08-19T01:43:41.329 INFO:teuthology.orchestra.run.smithi117.stdout: "req_setdirlayout_latency": { 2024-08-19T01:43:41.329 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2024-08-19T01:43:41.329 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2024-08-19T01:43:41.329 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2024-08-19T01:43:41.329 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-08-19T01:43:41.329 INFO:teuthology.orchestra.run.smithi117.stdout: "req_setfilelock_latency": { 2024-08-19T01:43:41.329 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2024-08-19T01:43:41.329 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2024-08-19T01:43:41.329 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2024-08-19T01:43:41.329 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-08-19T01:43:41.330 INFO:teuthology.orchestra.run.smithi117.stdout: "req_setlayout_latency": { 2024-08-19T01:43:41.330 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2024-08-19T01:43:41.330 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2024-08-19T01:43:41.330 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2024-08-19T01:43:41.330 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-08-19T01:43:41.330 INFO:teuthology.orchestra.run.smithi117.stdout: "req_setxattr_latency": { 2024-08-19T01:43:41.330 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 8, 2024-08-19T01:43:41.330 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.021689720, 2024-08-19T01:43:41.330 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.002711215 2024-08-19T01:43:41.330 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-08-19T01:43:41.330 INFO:teuthology.orchestra.run.smithi117.stdout: "req_symlink_latency": { 2024-08-19T01:43:41.330 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 3, 2024-08-19T01:43:41.330 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.001641765, 2024-08-19T01:43:41.330 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000547255 2024-08-19T01:43:41.330 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-08-19T01:43:41.330 INFO:teuthology.orchestra.run.smithi117.stdout: "req_unlink_latency": { 2024-08-19T01:43:41.331 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 35220, 2024-08-19T01:43:41.331 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 57.936903154, 2024-08-19T01:43:41.331 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.001645000 2024-08-19T01:43:41.331 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-08-19T01:43:41.331 INFO:teuthology.orchestra.run.smithi117.stdout: "cap_revoke_eviction": 0, 2024-08-19T01:43:41.331 INFO:teuthology.orchestra.run.smithi117.stdout: "cap_acquisition_throttle": 0, 2024-08-19T01:43:41.331 INFO:teuthology.orchestra.run.smithi117.stdout: "req_getvxattr_latency": { 2024-08-19T01:43:41.331 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2024-08-19T01:43:41.331 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2024-08-19T01:43:41.331 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2024-08-19T01:43:41.331 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-08-19T01:43:41.331 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-08-19T01:43:41.331 INFO:teuthology.orchestra.run.smithi117.stdout: "mds_sessions": { 2024-08-19T01:43:41.331 INFO:teuthology.orchestra.run.smithi117.stdout: "session_count": 1, 2024-08-19T01:43:41.331 INFO:teuthology.orchestra.run.smithi117.stdout: "session_add": 3, 2024-08-19T01:43:41.332 INFO:teuthology.orchestra.run.smithi117.stdout: "session_remove": 2, 2024-08-19T01:43:41.332 INFO:teuthology.orchestra.run.smithi117.stdout: "sessions_open": 1, 2024-08-19T01:43:41.332 INFO:teuthology.orchestra.run.smithi117.stdout: "sessions_stale": 0, 2024-08-19T01:43:41.332 INFO:teuthology.orchestra.run.smithi117.stdout: "total_load": 35553, 2024-08-19T01:43:41.332 INFO:teuthology.orchestra.run.smithi117.stdout: "average_load": 35553, 2024-08-19T01:43:41.332 INFO:teuthology.orchestra.run.smithi117.stdout: "avg_session_uptime": 540, 2024-08-19T01:43:41.332 INFO:teuthology.orchestra.run.smithi117.stdout: "mdthresh_evicted": 0 2024-08-19T01:43:41.332 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-08-19T01:43:41.332 INFO:teuthology.orchestra.run.smithi117.stdout: "mempool": { 2024-08-19T01:43:41.332 INFO:teuthology.orchestra.run.smithi117.stdout: "bloom_filter_bytes": 0, 2024-08-19T01:43:41.332 INFO:teuthology.orchestra.run.smithi117.stdout: "bloom_filter_items": 0, 2024-08-19T01:43:41.332 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_alloc_bytes": 0, 2024-08-19T01:43:41.332 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_alloc_items": 0, 2024-08-19T01:43:41.332 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_cache_data_bytes": 0, 2024-08-19T01:43:41.332 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_cache_data_items": 0, 2024-08-19T01:43:41.332 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_cache_onode_bytes": 0, 2024-08-19T01:43:41.333 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_cache_onode_items": 0, 2024-08-19T01:43:41.333 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_cache_meta_bytes": 0, 2024-08-19T01:43:41.333 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_cache_meta_items": 0, 2024-08-19T01:43:41.333 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_cache_other_bytes": 0, 2024-08-19T01:43:41.333 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_cache_other_items": 0, 2024-08-19T01:43:41.333 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_cache_buffer_bytes": 0, 2024-08-19T01:43:41.333 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_cache_buffer_items": 0, 2024-08-19T01:43:41.333 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_extent_bytes": 0, 2024-08-19T01:43:41.333 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_extent_items": 0, 2024-08-19T01:43:41.333 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_blob_bytes": 0, 2024-08-19T01:43:41.333 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_blob_items": 0, 2024-08-19T01:43:41.333 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_shared_blob_bytes": 0, 2024-08-19T01:43:41.333 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_shared_blob_items": 0, 2024-08-19T01:43:41.333 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_inline_bl_bytes": 0, 2024-08-19T01:43:41.333 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_inline_bl_items": 0, 2024-08-19T01:43:41.333 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_fsck_bytes": 0, 2024-08-19T01:43:41.334 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_fsck_items": 0, 2024-08-19T01:43:41.334 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_txc_bytes": 0, 2024-08-19T01:43:41.334 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_txc_items": 0, 2024-08-19T01:43:41.334 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_writing_deferred_bytes": 0, 2024-08-19T01:43:41.334 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_writing_deferred_items": 0, 2024-08-19T01:43:41.334 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_writing_bytes": 0, 2024-08-19T01:43:41.334 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_writing_items": 0, 2024-08-19T01:43:41.334 INFO:teuthology.orchestra.run.smithi117.stdout: "bluefs_bytes": 0, 2024-08-19T01:43:41.334 INFO:teuthology.orchestra.run.smithi117.stdout: "bluefs_items": 0, 2024-08-19T01:43:41.334 INFO:teuthology.orchestra.run.smithi117.stdout: "bluefs_file_reader_bytes": 0, 2024-08-19T01:43:41.334 INFO:teuthology.orchestra.run.smithi117.stdout: "bluefs_file_reader_items": 0, 2024-08-19T01:43:41.334 INFO:teuthology.orchestra.run.smithi117.stdout: "bluefs_file_writer_bytes": 0, 2024-08-19T01:43:41.334 INFO:teuthology.orchestra.run.smithi117.stdout: "bluefs_file_writer_items": 0, 2024-08-19T01:43:41.335 INFO:teuthology.orchestra.run.smithi117.stdout: "buffer_anon_bytes": 298056, 2024-08-19T01:43:41.335 INFO:teuthology.orchestra.run.smithi117.stdout: "buffer_anon_items": 117, 2024-08-19T01:43:41.335 INFO:teuthology.orchestra.run.smithi117.stdout: "buffer_meta_bytes": 0, 2024-08-19T01:43:41.335 INFO:teuthology.orchestra.run.smithi117.stdout: "buffer_meta_items": 0, 2024-08-19T01:43:41.335 INFO:teuthology.orchestra.run.smithi117.stdout: "osd_bytes": 0, 2024-08-19T01:43:41.335 INFO:teuthology.orchestra.run.smithi117.stdout: "osd_items": 0, 2024-08-19T01:43:41.335 INFO:teuthology.orchestra.run.smithi117.stdout: "osd_mapbl_bytes": 0, 2024-08-19T01:43:41.335 INFO:teuthology.orchestra.run.smithi117.stdout: "osd_mapbl_items": 0, 2024-08-19T01:43:41.335 INFO:teuthology.orchestra.run.smithi117.stdout: "osd_pglog_bytes": 0, 2024-08-19T01:43:41.335 INFO:teuthology.orchestra.run.smithi117.stdout: "osd_pglog_items": 0, 2024-08-19T01:43:41.335 INFO:teuthology.orchestra.run.smithi117.stdout: "osdmap_bytes": 7688, 2024-08-19T01:43:41.335 INFO:teuthology.orchestra.run.smithi117.stdout: "osdmap_items": 158, 2024-08-19T01:43:41.335 INFO:teuthology.orchestra.run.smithi117.stdout: "osdmap_mapping_bytes": 0, 2024-08-19T01:43:41.335 INFO:teuthology.orchestra.run.smithi117.stdout: "osdmap_mapping_items": 0, 2024-08-19T01:43:41.335 INFO:teuthology.orchestra.run.smithi117.stdout: "pgmap_bytes": 0, 2024-08-19T01:43:41.336 INFO:teuthology.orchestra.run.smithi117.stdout: "pgmap_items": 0, 2024-08-19T01:43:41.336 INFO:teuthology.orchestra.run.smithi117.stdout: "mds_co_bytes": 729300, 2024-08-19T01:43:41.336 INFO:teuthology.orchestra.run.smithi117.stdout: "mds_co_items": 8033, 2024-08-19T01:43:41.336 INFO:teuthology.orchestra.run.smithi117.stdout: "unittest_1_bytes": 0, 2024-08-19T01:43:41.336 INFO:teuthology.orchestra.run.smithi117.stdout: "unittest_1_items": 0, 2024-08-19T01:43:41.336 INFO:teuthology.orchestra.run.smithi117.stdout: "unittest_2_bytes": 0, 2024-08-19T01:43:41.336 INFO:teuthology.orchestra.run.smithi117.stdout: "unittest_2_items": 0 2024-08-19T01:43:41.336 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-08-19T01:43:41.336 INFO:teuthology.orchestra.run.smithi117.stdout: "objecter": { 2024-08-19T01:43:41.336 INFO:teuthology.orchestra.run.smithi117.stdout: "op_active": 0, 2024-08-19T01:43:41.336 INFO:teuthology.orchestra.run.smithi117.stdout: "op_laggy": 0, 2024-08-19T01:43:41.336 INFO:teuthology.orchestra.run.smithi117.stdout: "op_send": 240643, 2024-08-19T01:43:41.336 INFO:teuthology.orchestra.run.smithi117.stdout: "op_send_bytes": 869550413, 2024-08-19T01:43:41.336 INFO:teuthology.orchestra.run.smithi117.stdout: "op_resend": 37, 2024-08-19T01:43:41.336 INFO:teuthology.orchestra.run.smithi117.stdout: "op_reply": 240643, 2024-08-19T01:43:41.336 INFO:teuthology.orchestra.run.smithi117.stdout: "op_latency": { 2024-08-19T01:43:41.337 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 240643, 2024-08-19T01:43:41.337 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 46810.469906592, 2024-08-19T01:43:41.337 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.194522466 2024-08-19T01:43:41.337 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-08-19T01:43:41.337 INFO:teuthology.orchestra.run.smithi117.stdout: "op_inflight": 0, 2024-08-19T01:43:41.337 INFO:teuthology.orchestra.run.smithi117.stdout: "oplen_avg": { 2024-08-19T01:43:41.337 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 240643, 2024-08-19T01:43:41.337 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 402407 2024-08-19T01:43:41.337 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-08-19T01:43:41.337 INFO:teuthology.orchestra.run.smithi117.stdout: "op": 240643, 2024-08-19T01:43:41.337 INFO:teuthology.orchestra.run.smithi117.stdout: "op_r": 93319, 2024-08-19T01:43:41.337 INFO:teuthology.orchestra.run.smithi117.stdout: "op_w": 147324, 2024-08-19T01:43:41.337 INFO:teuthology.orchestra.run.smithi117.stdout: "op_rmw": 0, 2024-08-19T01:43:41.337 INFO:teuthology.orchestra.run.smithi117.stdout: "op_pg": 0, 2024-08-19T01:43:41.337 INFO:teuthology.orchestra.run.smithi117.stdout: "osdop_stat": 11327, 2024-08-19T01:43:41.337 INFO:teuthology.orchestra.run.smithi117.stdout: "osdop_create": 65167, 2024-08-19T01:43:41.338 INFO:teuthology.orchestra.run.smithi117.stdout: "osdop_read": 891, 2024-08-19T01:43:41.338 INFO:teuthology.orchestra.run.smithi117.stdout: "osdop_write": 4120, 2024-08-19T01:43:41.338 INFO:teuthology.orchestra.run.smithi117.stdout: "osdop_writefull": 506, 2024-08-19T01:43:41.338 INFO:teuthology.orchestra.run.smithi117.stdout: "osdop_writesame": 0, 2024-08-19T01:43:41.338 INFO:teuthology.orchestra.run.smithi117.stdout: "osdop_append": 0, 2024-08-19T01:43:41.338 INFO:teuthology.orchestra.run.smithi117.stdout: "osdop_zero": 0, 2024-08-19T01:43:41.338 INFO:teuthology.orchestra.run.smithi117.stdout: "osdop_truncate": 202, 2024-08-19T01:43:41.338 INFO:teuthology.orchestra.run.smithi117.stdout: "osdop_delete": 63642, 2024-08-19T01:43:41.338 INFO:teuthology.orchestra.run.smithi117.stdout: "osdop_mapext": 0, 2024-08-19T01:43:41.338 INFO:teuthology.orchestra.run.smithi117.stdout: "osdop_sparse_read": 0, 2024-08-19T01:43:41.338 INFO:teuthology.orchestra.run.smithi117.stdout: "osdop_clonerange": 0, 2024-08-19T01:43:41.338 INFO:teuthology.orchestra.run.smithi117.stdout: "osdop_getxattr": 92413, 2024-08-19T01:43:41.338 INFO:teuthology.orchestra.run.smithi117.stdout: "osdop_setxattr": 130334, 2024-08-19T01:43:41.338 INFO:teuthology.orchestra.run.smithi117.stdout: "osdop_cmpxattr": 0, 2024-08-19T01:43:41.338 INFO:teuthology.orchestra.run.smithi117.stdout: "osdop_rmxattr": 0, 2024-08-19T01:43:41.338 INFO:teuthology.orchestra.run.smithi117.stdout: "osdop_resetxattrs": 0, 2024-08-19T01:43:41.339 INFO:teuthology.orchestra.run.smithi117.stdout: "osdop_call": 0, 2024-08-19T01:43:41.339 INFO:teuthology.orchestra.run.smithi117.stdout: "osdop_watch": 0, 2024-08-19T01:43:41.339 INFO:teuthology.orchestra.run.smithi117.stdout: "osdop_notify": 0, 2024-08-19T01:43:41.339 INFO:teuthology.orchestra.run.smithi117.stdout: "osdop_src_cmpxattr": 0, 2024-08-19T01:43:41.339 INFO:teuthology.orchestra.run.smithi117.stdout: "osdop_pgls": 0, 2024-08-19T01:43:41.339 INFO:teuthology.orchestra.run.smithi117.stdout: "osdop_pgls_filter": 0, 2024-08-19T01:43:41.339 INFO:teuthology.orchestra.run.smithi117.stdout: "osdop_other": 5, 2024-08-19T01:43:41.339 INFO:teuthology.orchestra.run.smithi117.stdout: "linger_active": 0, 2024-08-19T01:43:41.339 INFO:teuthology.orchestra.run.smithi117.stdout: "linger_send": 0, 2024-08-19T01:43:41.339 INFO:teuthology.orchestra.run.smithi117.stdout: "linger_resend": 0, 2024-08-19T01:43:41.339 INFO:teuthology.orchestra.run.smithi117.stdout: "linger_ping": 0, 2024-08-19T01:43:41.339 INFO:teuthology.orchestra.run.smithi117.stdout: "poolop_active": 0, 2024-08-19T01:43:41.339 INFO:teuthology.orchestra.run.smithi117.stdout: "poolop_send": 0, 2024-08-19T01:43:41.339 INFO:teuthology.orchestra.run.smithi117.stdout: "poolop_resend": 0, 2024-08-19T01:43:41.339 INFO:teuthology.orchestra.run.smithi117.stdout: "poolstat_active": 0, 2024-08-19T01:43:41.340 INFO:teuthology.orchestra.run.smithi117.stdout: "poolstat_send": 0, 2024-08-19T01:43:41.340 INFO:teuthology.orchestra.run.smithi117.stdout: "poolstat_resend": 0, 2024-08-19T01:43:41.340 INFO:teuthology.orchestra.run.smithi117.stdout: "statfs_active": 0, 2024-08-19T01:43:41.340 INFO:teuthology.orchestra.run.smithi117.stdout: "statfs_send": 0, 2024-08-19T01:43:41.340 INFO:teuthology.orchestra.run.smithi117.stdout: "statfs_resend": 0, 2024-08-19T01:43:41.340 INFO:teuthology.orchestra.run.smithi117.stdout: "command_active": 0, 2024-08-19T01:43:41.340 INFO:teuthology.orchestra.run.smithi117.stdout: "command_send": 0, 2024-08-19T01:43:41.340 INFO:teuthology.orchestra.run.smithi117.stdout: "command_resend": 0, 2024-08-19T01:43:41.340 INFO:teuthology.orchestra.run.smithi117.stdout: "map_epoch": 86, 2024-08-19T01:43:41.340 INFO:teuthology.orchestra.run.smithi117.stdout: "map_full": 0, 2024-08-19T01:43:41.340 INFO:teuthology.orchestra.run.smithi117.stdout: "map_inc": 2, 2024-08-19T01:43:41.340 INFO:teuthology.orchestra.run.smithi117.stdout: "osd_sessions": 12, 2024-08-19T01:43:41.340 INFO:teuthology.orchestra.run.smithi117.stdout: "osd_session_open": 12, 2024-08-19T01:43:41.340 INFO:teuthology.orchestra.run.smithi117.stdout: "osd_session_close": 0, 2024-08-19T01:43:41.340 INFO:teuthology.orchestra.run.smithi117.stdout: "osd_laggy": 0, 2024-08-19T01:43:41.340 INFO:teuthology.orchestra.run.smithi117.stdout: "omap_wr": 26450, 2024-08-19T01:43:41.340 INFO:teuthology.orchestra.run.smithi117.stdout: "omap_rd": 34, 2024-08-19T01:43:41.341 INFO:teuthology.orchestra.run.smithi117.stdout: "omap_del": 7316 2024-08-19T01:43:41.341 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-08-19T01:43:41.341 INFO:teuthology.orchestra.run.smithi117.stdout: "oft": { 2024-08-19T01:43:41.341 INFO:teuthology.orchestra.run.smithi117.stdout: "omap_total_objs": 1, 2024-08-19T01:43:41.341 INFO:teuthology.orchestra.run.smithi117.stdout: "omap_total_kv_pairs": 0, 2024-08-19T01:43:41.341 INFO:teuthology.orchestra.run.smithi117.stdout: "omap_total_updates": 0, 2024-08-19T01:43:41.341 INFO:teuthology.orchestra.run.smithi117.stdout: "omap_total_removes": 0 2024-08-19T01:43:41.341 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-08-19T01:43:41.341 INFO:teuthology.orchestra.run.smithi117.stdout: "osd-slow-ops": { 2024-08-19T01:43:41.341 INFO:teuthology.orchestra.run.smithi117.stdout: "slow_ops_count": 0 2024-08-19T01:43:41.341 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-08-19T01:43:41.341 INFO:teuthology.orchestra.run.smithi117.stdout: "purge_queue": { 2024-08-19T01:43:41.341 INFO:teuthology.orchestra.run.smithi117.stdout: "pq_executing_ops": 0, 2024-08-19T01:43:41.341 INFO:teuthology.orchestra.run.smithi117.stdout: "pq_executing_ops_high_water": 18, 2024-08-19T01:43:41.342 INFO:teuthology.orchestra.run.smithi117.stdout: "pq_executing": 0, 2024-08-19T01:43:41.342 INFO:teuthology.orchestra.run.smithi117.stdout: "pq_executing_high_water": 10, 2024-08-19T01:43:41.342 INFO:teuthology.orchestra.run.smithi117.stdout: "pq_executed_ops": 62612, 2024-08-19T01:43:41.342 INFO:teuthology.orchestra.run.smithi117.stdout: "pq_executed": 61095, 2024-08-19T01:43:41.342 INFO:teuthology.orchestra.run.smithi117.stdout: "pq_item_in_journal": 0 2024-08-19T01:43:41.342 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-08-19T01:43:41.342 INFO:teuthology.orchestra.run.smithi117.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-08-19T01:43:41.342 INFO:teuthology.orchestra.run.smithi117.stdout: "val": 0, 2024-08-19T01:43:41.342 INFO:teuthology.orchestra.run.smithi117.stdout: "max": 104857600, 2024-08-19T01:43:41.342 INFO:teuthology.orchestra.run.smithi117.stdout: "get_started": 0, 2024-08-19T01:43:41.342 INFO:teuthology.orchestra.run.smithi117.stdout: "get": 605449, 2024-08-19T01:43:41.342 INFO:teuthology.orchestra.run.smithi117.stdout: "get_sum": 187455197, 2024-08-19T01:43:41.342 INFO:teuthology.orchestra.run.smithi117.stdout: "get_or_fail_fail": 0, 2024-08-19T01:43:41.342 INFO:teuthology.orchestra.run.smithi117.stdout: "get_or_fail_success": 605449, 2024-08-19T01:43:41.343 INFO:teuthology.orchestra.run.smithi117.stdout: "take": 0, 2024-08-19T01:43:41.343 INFO:teuthology.orchestra.run.smithi117.stdout: "take_sum": 0, 2024-08-19T01:43:41.343 INFO:teuthology.orchestra.run.smithi117.stdout: "put": 605449, 2024-08-19T01:43:41.343 INFO:teuthology.orchestra.run.smithi117.stdout: "put_sum": 187455197, 2024-08-19T01:43:41.343 INFO:teuthology.orchestra.run.smithi117.stdout: "wait": { 2024-08-19T01:43:41.343 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2024-08-19T01:43:41.343 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2024-08-19T01:43:41.343 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2024-08-19T01:43:41.343 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-08-19T01:43:41.343 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-08-19T01:43:41.343 INFO:teuthology.orchestra.run.smithi117.stdout: "throttle-objecter_bytes": { 2024-08-19T01:43:41.343 INFO:teuthology.orchestra.run.smithi117.stdout: "val": 0, 2024-08-19T01:43:41.343 INFO:teuthology.orchestra.run.smithi117.stdout: "max": 104857600, 2024-08-19T01:43:41.343 INFO:teuthology.orchestra.run.smithi117.stdout: "get_started": 0, 2024-08-19T01:43:41.343 INFO:teuthology.orchestra.run.smithi117.stdout: "get": 0, 2024-08-19T01:43:41.343 INFO:teuthology.orchestra.run.smithi117.stdout: "get_sum": 0, 2024-08-19T01:43:41.344 INFO:teuthology.orchestra.run.smithi117.stdout: "get_or_fail_fail": 0, 2024-08-19T01:43:41.344 INFO:teuthology.orchestra.run.smithi117.stdout: "get_or_fail_success": 0, 2024-08-19T01:43:41.344 INFO:teuthology.orchestra.run.smithi117.stdout: "take": 240643, 2024-08-19T01:43:41.344 INFO:teuthology.orchestra.run.smithi117.stdout: "take_sum": 875722018, 2024-08-19T01:43:41.344 INFO:teuthology.orchestra.run.smithi117.stdout: "put": 240643, 2024-08-19T01:43:41.344 INFO:teuthology.orchestra.run.smithi117.stdout: "put_sum": 875722018, 2024-08-19T01:43:41.344 INFO:teuthology.orchestra.run.smithi117.stdout: "wait": { 2024-08-19T01:43:41.344 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2024-08-19T01:43:41.344 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2024-08-19T01:43:41.344 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2024-08-19T01:43:41.344 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-08-19T01:43:41.344 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-08-19T01:43:41.344 INFO:teuthology.orchestra.run.smithi117.stdout: "throttle-objecter_ops": { 2024-08-19T01:43:41.344 INFO:teuthology.orchestra.run.smithi117.stdout: "val": 0, 2024-08-19T01:43:41.344 INFO:teuthology.orchestra.run.smithi117.stdout: "max": 1024, 2024-08-19T01:43:41.344 INFO:teuthology.orchestra.run.smithi117.stdout: "get_started": 0, 2024-08-19T01:43:41.344 INFO:teuthology.orchestra.run.smithi117.stdout: "get": 0, 2024-08-19T01:43:41.345 INFO:teuthology.orchestra.run.smithi117.stdout: "get_sum": 0, 2024-08-19T01:43:41.345 INFO:teuthology.orchestra.run.smithi117.stdout: "get_or_fail_fail": 0, 2024-08-19T01:43:41.345 INFO:teuthology.orchestra.run.smithi117.stdout: "get_or_fail_success": 0, 2024-08-19T01:43:41.345 INFO:teuthology.orchestra.run.smithi117.stdout: "take": 240643, 2024-08-19T01:43:41.345 INFO:teuthology.orchestra.run.smithi117.stdout: "take_sum": 240643, 2024-08-19T01:43:41.345 INFO:teuthology.orchestra.run.smithi117.stdout: "put": 240643, 2024-08-19T01:43:41.345 INFO:teuthology.orchestra.run.smithi117.stdout: "put_sum": 240643, 2024-08-19T01:43:41.345 INFO:teuthology.orchestra.run.smithi117.stdout: "wait": { 2024-08-19T01:43:41.345 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2024-08-19T01:43:41.345 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2024-08-19T01:43:41.345 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2024-08-19T01:43:41.345 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-08-19T01:43:41.345 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-08-19T01:43:41.345 INFO:teuthology.orchestra.run.smithi117.stdout: "throttle-write_buf_throttle": { 2024-08-19T01:43:41.345 INFO:teuthology.orchestra.run.smithi117.stdout: "val": 0, 2024-08-19T01:43:41.345 INFO:teuthology.orchestra.run.smithi117.stdout: "max": 3758096384, 2024-08-19T01:43:41.346 INFO:teuthology.orchestra.run.smithi117.stdout: "get_started": 0, 2024-08-19T01:43:41.346 INFO:teuthology.orchestra.run.smithi117.stdout: "get": 61095, 2024-08-19T01:43:41.346 INFO:teuthology.orchestra.run.smithi117.stdout: "get_sum": 6172123, 2024-08-19T01:43:41.346 INFO:teuthology.orchestra.run.smithi117.stdout: "get_or_fail_fail": 0, 2024-08-19T01:43:41.346 INFO:teuthology.orchestra.run.smithi117.stdout: "get_or_fail_success": 61095, 2024-08-19T01:43:41.346 INFO:teuthology.orchestra.run.smithi117.stdout: "take": 0, 2024-08-19T01:43:41.346 INFO:teuthology.orchestra.run.smithi117.stdout: "take_sum": 0, 2024-08-19T01:43:41.346 INFO:teuthology.orchestra.run.smithi117.stdout: "put": 883, 2024-08-19T01:43:41.346 INFO:teuthology.orchestra.run.smithi117.stdout: "put_sum": 6172123, 2024-08-19T01:43:41.346 INFO:teuthology.orchestra.run.smithi117.stdout: "wait": { 2024-08-19T01:43:41.346 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2024-08-19T01:43:41.346 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2024-08-19T01:43:41.346 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2024-08-19T01:43:41.346 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-08-19T01:43:41.347 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-08-19T01:43:41.347 INFO:teuthology.orchestra.run.smithi117.stdout: "throttle-write_buf_throttle-0x55b595156960": { 2024-08-19T01:43:41.347 INFO:teuthology.orchestra.run.smithi117.stdout: "val": 0, 2024-08-19T01:43:41.347 INFO:teuthology.orchestra.run.smithi117.stdout: "max": 3758096384, 2024-08-19T01:43:41.347 INFO:teuthology.orchestra.run.smithi117.stdout: "get_started": 0, 2024-08-19T01:43:41.347 INFO:teuthology.orchestra.run.smithi117.stdout: "get": 323890, 2024-08-19T01:43:41.347 INFO:teuthology.orchestra.run.smithi117.stdout: "get_sum": 730904675, 2024-08-19T01:43:41.347 INFO:teuthology.orchestra.run.smithi117.stdout: "get_or_fail_fail": 0, 2024-08-19T01:43:41.347 INFO:teuthology.orchestra.run.smithi117.stdout: "get_or_fail_success": 323890, 2024-08-19T01:43:41.347 INFO:teuthology.orchestra.run.smithi117.stdout: "take": 0, 2024-08-19T01:43:41.347 INFO:teuthology.orchestra.run.smithi117.stdout: "take_sum": 0, 2024-08-19T01:43:41.347 INFO:teuthology.orchestra.run.smithi117.stdout: "put": 3237, 2024-08-19T01:43:41.347 INFO:teuthology.orchestra.run.smithi117.stdout: "put_sum": 730904675, 2024-08-19T01:43:41.347 INFO:teuthology.orchestra.run.smithi117.stdout: "wait": { 2024-08-19T01:43:41.347 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2024-08-19T01:43:41.347 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2024-08-19T01:43:41.348 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2024-08-19T01:43:41.348 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-08-19T01:43:41.348 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-08-19T01:43:41.348 INFO:teuthology.orchestra.run.smithi117.stdout:} 2024-08-19T01:43:42.154 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:43:41 smithi083 ceph-mon[17120]: pgmap v604: 97 pgs: 97 active+clean; 35 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-08-19T01:43:42.202 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:43:41 smithi117 ceph-mon[22016]: pgmap v604: 97 pgs: 97 active+clean; 35 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-08-19T01:43:42.215 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:43:41 smithi167 ceph-mon[21875]: pgmap v604: 97 pgs: 97 active+clean; 35 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-08-19T01:43:42.453 INFO:tasks.check_counter:Daemon mds.b mds.exported=10 2024-08-19T01:43:42.454 INFO:tasks.check_counter:Daemon mds.b mds.imported=66 2024-08-19T01:43:42.454 INFO:tasks.check_counter:Daemon mds.b mds.dir_split=202 2024-08-19T01:43:42.454 DEBUG:tasks.check_counter:Getting stats from e 2024-08-19T01:43:42.454 DEBUG:teuthology.orchestra.run.smithi117:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833 shell --fsid bad8780e-5dc9-11ef-bccf-c7b262605968 -- ceph daemon mds.e perf dump 2024-08-19T01:43:42.718 INFO:teuthology.orchestra.run.smithi117.stderr:Inferring config /var/lib/ceph/bad8780e-5dc9-11ef-bccf-c7b262605968/mon.b/config 2024-08-19T01:43:43.009 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:43:42 smithi117 ceph-mon[22016]: pgmap v605: 97 pgs: 97 active+clean; 35 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-08-19T01:43:43.154 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:43:42 smithi083 ceph-mon[17120]: pgmap v605: 97 pgs: 97 active+clean; 35 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-08-19T01:43:43.216 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:43:42 smithi167 ceph-mon[21875]: pgmap v605: 97 pgs: 97 active+clean; 35 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-08-19T01:43:43.348 INFO:teuthology.orchestra.run.smithi117.stdout:{ 2024-08-19T01:43:43.348 INFO:teuthology.orchestra.run.smithi117.stdout: "AsyncMessenger::Worker-0": { 2024-08-19T01:43:43.348 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_recv_messages": 1946, 2024-08-19T01:43:43.348 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_send_messages": 1947, 2024-08-19T01:43:43.348 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_recv_bytes": 3906395, 2024-08-19T01:43:43.348 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_send_bytes": 618517, 2024-08-19T01:43:43.348 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_created_connections": 6, 2024-08-19T01:43:43.348 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_active_connections": 2, 2024-08-19T01:43:43.348 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_running_total_time": 0.128385502, 2024-08-19T01:43:43.348 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_running_send_time": 0.038404246, 2024-08-19T01:43:43.349 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_running_recv_time": 0.035604708, 2024-08-19T01:43:43.349 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_running_fast_dispatch_time": 0.044667180, 2024-08-19T01:43:43.349 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_send_messages_queue_lat": { 2024-08-19T01:43:43.349 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 1945, 2024-08-19T01:43:43.349 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.050027270, 2024-08-19T01:43:43.349 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000025720 2024-08-19T01:43:43.349 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-08-19T01:43:43.349 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_handle_ack_lat": { 2024-08-19T01:43:43.349 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2024-08-19T01:43:43.349 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2024-08-19T01:43:43.349 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2024-08-19T01:43:43.349 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-08-19T01:43:43.349 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_recv_encrypted_bytes": 0, 2024-08-19T01:43:43.350 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_send_encrypted_bytes": 0 2024-08-19T01:43:43.350 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-08-19T01:43:43.350 INFO:teuthology.orchestra.run.smithi117.stdout: "AsyncMessenger::Worker-1": { 2024-08-19T01:43:43.350 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_recv_messages": 3, 2024-08-19T01:43:43.350 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_send_messages": 144, 2024-08-19T01:43:43.350 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_recv_bytes": 716, 2024-08-19T01:43:43.350 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_send_bytes": 323274, 2024-08-19T01:43:43.350 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_created_connections": 4, 2024-08-19T01:43:43.350 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_active_connections": 3, 2024-08-19T01:43:43.350 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_running_total_time": 0.007748030, 2024-08-19T01:43:43.350 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_running_send_time": 0.005342414, 2024-08-19T01:43:43.351 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_running_recv_time": 0.001217568, 2024-08-19T01:43:43.351 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_running_fast_dispatch_time": 0.000065969, 2024-08-19T01:43:43.351 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_send_messages_queue_lat": { 2024-08-19T01:43:43.351 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 142, 2024-08-19T01:43:43.351 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.006443072, 2024-08-19T01:43:43.351 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000045373 2024-08-19T01:43:43.351 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-08-19T01:43:43.351 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_handle_ack_lat": { 2024-08-19T01:43:43.351 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2024-08-19T01:43:43.351 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2024-08-19T01:43:43.351 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2024-08-19T01:43:43.351 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-08-19T01:43:43.351 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_recv_encrypted_bytes": 160, 2024-08-19T01:43:43.352 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_send_encrypted_bytes": 322576 2024-08-19T01:43:43.352 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-08-19T01:43:43.352 INFO:teuthology.orchestra.run.smithi117.stdout: "AsyncMessenger::Worker-2": { 2024-08-19T01:43:43.352 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_recv_messages": 212, 2024-08-19T01:43:43.352 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_send_messages": 197, 2024-08-19T01:43:43.352 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_recv_bytes": 797932, 2024-08-19T01:43:43.352 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_send_bytes": 116718, 2024-08-19T01:43:43.352 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_created_connections": 3, 2024-08-19T01:43:43.352 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_active_connections": 2, 2024-08-19T01:43:43.352 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_running_total_time": 0.028868820, 2024-08-19T01:43:43.352 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_running_send_time": 0.010602884, 2024-08-19T01:43:43.352 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_running_recv_time": 0.025129447, 2024-08-19T01:43:43.353 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_running_fast_dispatch_time": 0.000011160, 2024-08-19T01:43:43.353 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_send_messages_queue_lat": { 2024-08-19T01:43:43.353 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 197, 2024-08-19T01:43:43.353 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.012847158, 2024-08-19T01:43:43.353 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000065214 2024-08-19T01:43:43.353 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-08-19T01:43:43.353 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_handle_ack_lat": { 2024-08-19T01:43:43.353 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2024-08-19T01:43:43.353 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2024-08-19T01:43:43.353 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2024-08-19T01:43:43.353 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-08-19T01:43:43.353 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_recv_encrypted_bytes": 797664, 2024-08-19T01:43:43.354 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_send_encrypted_bytes": 116400 2024-08-19T01:43:43.354 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-08-19T01:43:43.354 INFO:teuthology.orchestra.run.smithi117.stdout: "cct": { 2024-08-19T01:43:43.354 INFO:teuthology.orchestra.run.smithi117.stdout: "total_workers": 1, 2024-08-19T01:43:43.354 INFO:teuthology.orchestra.run.smithi117.stdout: "unhealthy_workers": 0 2024-08-19T01:43:43.354 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-08-19T01:43:43.354 INFO:teuthology.orchestra.run.smithi117.stdout: "finisher-MDSRank": { 2024-08-19T01:43:43.354 INFO:teuthology.orchestra.run.smithi117.stdout: "queue_len": 0, 2024-08-19T01:43:43.354 INFO:teuthology.orchestra.run.smithi117.stdout: "complete_latency": { 2024-08-19T01:43:43.354 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 3302, 2024-08-19T01:43:43.354 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.072442617, 2024-08-19T01:43:43.354 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000021939 2024-08-19T01:43:43.355 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-08-19T01:43:43.355 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-08-19T01:43:43.355 INFO:teuthology.orchestra.run.smithi117.stdout: "finisher-PurgeQueue": { 2024-08-19T01:43:43.355 INFO:teuthology.orchestra.run.smithi117.stdout: "queue_len": 0, 2024-08-19T01:43:43.355 INFO:teuthology.orchestra.run.smithi117.stdout: "complete_latency": { 2024-08-19T01:43:43.355 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2024-08-19T01:43:43.355 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2024-08-19T01:43:43.355 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2024-08-19T01:43:43.355 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-08-19T01:43:43.355 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-08-19T01:43:43.355 INFO:teuthology.orchestra.run.smithi117.stdout: "mds": { 2024-08-19T01:43:43.355 INFO:teuthology.orchestra.run.smithi117.stdout: "request": 0, 2024-08-19T01:43:43.355 INFO:teuthology.orchestra.run.smithi117.stdout: "reply": 0, 2024-08-19T01:43:43.356 INFO:teuthology.orchestra.run.smithi117.stdout: "reply_latency": { 2024-08-19T01:43:43.356 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2024-08-19T01:43:43.356 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2024-08-19T01:43:43.356 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2024-08-19T01:43:43.356 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-08-19T01:43:43.356 INFO:teuthology.orchestra.run.smithi117.stdout: "slow_reply": 0, 2024-08-19T01:43:43.356 INFO:teuthology.orchestra.run.smithi117.stdout: "forward": 0, 2024-08-19T01:43:43.356 INFO:teuthology.orchestra.run.smithi117.stdout: "dir_fetch_complete": 0, 2024-08-19T01:43:43.356 INFO:teuthology.orchestra.run.smithi117.stdout: "dir_fetch_keys": 0, 2024-08-19T01:43:43.356 INFO:teuthology.orchestra.run.smithi117.stdout: "dir_commit": 0, 2024-08-19T01:43:43.356 INFO:teuthology.orchestra.run.smithi117.stdout: "dir_split": 0, 2024-08-19T01:43:43.356 INFO:teuthology.orchestra.run.smithi117.stdout: "dir_merge": 0, 2024-08-19T01:43:43.356 INFO:teuthology.orchestra.run.smithi117.stdout: "inodes": 451, 2024-08-19T01:43:43.357 INFO:teuthology.orchestra.run.smithi117.stdout: "inodes_top": 0, 2024-08-19T01:43:43.357 INFO:teuthology.orchestra.run.smithi117.stdout: "inodes_bottom": 451, 2024-08-19T01:43:43.357 INFO:teuthology.orchestra.run.smithi117.stdout: "inodes_pin_tail": 0, 2024-08-19T01:43:43.357 INFO:teuthology.orchestra.run.smithi117.stdout: "inodes_pinned": 451, 2024-08-19T01:43:43.357 INFO:teuthology.orchestra.run.smithi117.stdout: "inodes_expired": 0, 2024-08-19T01:43:43.357 INFO:teuthology.orchestra.run.smithi117.stdout: "inodes_with_caps": 0, 2024-08-19T01:43:43.357 INFO:teuthology.orchestra.run.smithi117.stdout: "caps": 0, 2024-08-19T01:43:43.357 INFO:teuthology.orchestra.run.smithi117.stdout: "subtrees": 1, 2024-08-19T01:43:43.357 INFO:teuthology.orchestra.run.smithi117.stdout: "traverse": 0, 2024-08-19T01:43:43.357 INFO:teuthology.orchestra.run.smithi117.stdout: "traverse_hit": 0, 2024-08-19T01:43:43.357 INFO:teuthology.orchestra.run.smithi117.stdout: "traverse_forward": 0, 2024-08-19T01:43:43.357 INFO:teuthology.orchestra.run.smithi117.stdout: "traverse_discover": 0, 2024-08-19T01:43:43.357 INFO:teuthology.orchestra.run.smithi117.stdout: "traverse_dir_fetch": 0, 2024-08-19T01:43:43.358 INFO:teuthology.orchestra.run.smithi117.stdout: "traverse_remote_ino": 0, 2024-08-19T01:43:43.358 INFO:teuthology.orchestra.run.smithi117.stdout: "traverse_lock": 0, 2024-08-19T01:43:43.358 INFO:teuthology.orchestra.run.smithi117.stdout: "load_cent": 0, 2024-08-19T01:43:43.358 INFO:teuthology.orchestra.run.smithi117.stdout: "q": 0, 2024-08-19T01:43:43.358 INFO:teuthology.orchestra.run.smithi117.stdout: "exported": 0, 2024-08-19T01:43:43.358 INFO:teuthology.orchestra.run.smithi117.stdout: "exported_inodes": 0, 2024-08-19T01:43:43.358 INFO:teuthology.orchestra.run.smithi117.stdout: "imported": 0, 2024-08-19T01:43:43.358 INFO:teuthology.orchestra.run.smithi117.stdout: "imported_inodes": 0, 2024-08-19T01:43:43.358 INFO:teuthology.orchestra.run.smithi117.stdout: "openino_dir_fetch": 0, 2024-08-19T01:43:43.358 INFO:teuthology.orchestra.run.smithi117.stdout: "openino_backtrace_fetch": 0, 2024-08-19T01:43:43.358 INFO:teuthology.orchestra.run.smithi117.stdout: "openino_peer_discover": 0, 2024-08-19T01:43:43.359 INFO:teuthology.orchestra.run.smithi117.stdout: "root_rfiles": 0, 2024-08-19T01:43:43.359 INFO:teuthology.orchestra.run.smithi117.stdout: "root_rbytes": 0, 2024-08-19T01:43:43.359 INFO:teuthology.orchestra.run.smithi117.stdout: "root_rsnaps": 0, 2024-08-19T01:43:43.359 INFO:teuthology.orchestra.run.smithi117.stdout: "scrub_backtrace_fetch": 0, 2024-08-19T01:43:43.359 INFO:teuthology.orchestra.run.smithi117.stdout: "scrub_set_tag": 0, 2024-08-19T01:43:43.359 INFO:teuthology.orchestra.run.smithi117.stdout: "scrub_backtrace_repaired": 0, 2024-08-19T01:43:43.359 INFO:teuthology.orchestra.run.smithi117.stdout: "scrub_inotable_repaired": 0, 2024-08-19T01:43:43.359 INFO:teuthology.orchestra.run.smithi117.stdout: "scrub_dir_inodes": 0, 2024-08-19T01:43:43.359 INFO:teuthology.orchestra.run.smithi117.stdout: "scrub_dir_base_inodes": 0, 2024-08-19T01:43:43.359 INFO:teuthology.orchestra.run.smithi117.stdout: "scrub_dirfrag_rstats": 0, 2024-08-19T01:43:43.359 INFO:teuthology.orchestra.run.smithi117.stdout: "scrub_file_inodes": 0, 2024-08-19T01:43:43.359 INFO:teuthology.orchestra.run.smithi117.stdout: "handle_inode_file_caps": 0, 2024-08-19T01:43:43.359 INFO:teuthology.orchestra.run.smithi117.stdout: "ceph_cap_op_revoke": 0, 2024-08-19T01:43:43.360 INFO:teuthology.orchestra.run.smithi117.stdout: "ceph_cap_op_grant": 0, 2024-08-19T01:43:43.360 INFO:teuthology.orchestra.run.smithi117.stdout: "ceph_cap_op_trunc": 0, 2024-08-19T01:43:43.360 INFO:teuthology.orchestra.run.smithi117.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-08-19T01:43:43.360 INFO:teuthology.orchestra.run.smithi117.stdout: "ceph_cap_op_flush_ack": 0, 2024-08-19T01:43:43.360 INFO:teuthology.orchestra.run.smithi117.stdout: "handle_client_caps": 0, 2024-08-19T01:43:43.361 INFO:teuthology.orchestra.run.smithi117.stdout: "handle_client_caps_dirty": 0, 2024-08-19T01:43:43.361 INFO:teuthology.orchestra.run.smithi117.stdout: "handle_client_cap_release": 0, 2024-08-19T01:43:43.361 INFO:teuthology.orchestra.run.smithi117.stdout: "process_request_cap_release": 0 2024-08-19T01:43:43.361 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-08-19T01:43:43.361 INFO:teuthology.orchestra.run.smithi117.stdout: "mds_cache": { 2024-08-19T01:43:43.361 INFO:teuthology.orchestra.run.smithi117.stdout: "dir_update": 0, 2024-08-19T01:43:43.361 INFO:teuthology.orchestra.run.smithi117.stdout: "dir_update_receipt": 0, 2024-08-19T01:43:43.361 INFO:teuthology.orchestra.run.smithi117.stdout: "dir_try_discover": 0, 2024-08-19T01:43:43.361 INFO:teuthology.orchestra.run.smithi117.stdout: "dir_send_discover": 0, 2024-08-19T01:43:43.361 INFO:teuthology.orchestra.run.smithi117.stdout: "dir_handle_discover": 0, 2024-08-19T01:43:43.361 INFO:teuthology.orchestra.run.smithi117.stdout: "num_strays": 0, 2024-08-19T01:43:43.361 INFO:teuthology.orchestra.run.smithi117.stdout: "num_strays_delayed": 0, 2024-08-19T01:43:43.362 INFO:teuthology.orchestra.run.smithi117.stdout: "num_strays_enqueuing": 0, 2024-08-19T01:43:43.362 INFO:teuthology.orchestra.run.smithi117.stdout: "strays_created": 0, 2024-08-19T01:43:43.362 INFO:teuthology.orchestra.run.smithi117.stdout: "strays_enqueued": 0, 2024-08-19T01:43:43.362 INFO:teuthology.orchestra.run.smithi117.stdout: "strays_reintegrated": 0, 2024-08-19T01:43:43.362 INFO:teuthology.orchestra.run.smithi117.stdout: "strays_migrated": 0, 2024-08-19T01:43:43.362 INFO:teuthology.orchestra.run.smithi117.stdout: "num_recovering_processing": 0, 2024-08-19T01:43:43.362 INFO:teuthology.orchestra.run.smithi117.stdout: "num_recovering_enqueued": 0, 2024-08-19T01:43:43.362 INFO:teuthology.orchestra.run.smithi117.stdout: "num_recovering_prioritized": 0, 2024-08-19T01:43:43.362 INFO:teuthology.orchestra.run.smithi117.stdout: "recovery_started": 0, 2024-08-19T01:43:43.362 INFO:teuthology.orchestra.run.smithi117.stdout: "recovery_completed": 0, 2024-08-19T01:43:43.362 INFO:teuthology.orchestra.run.smithi117.stdout: "ireq_quiesce_path": 0, 2024-08-19T01:43:43.363 INFO:teuthology.orchestra.run.smithi117.stdout: "ireq_quiesce_inode": 0, 2024-08-19T01:43:43.363 INFO:teuthology.orchestra.run.smithi117.stdout: "ireq_enqueue_scrub": 0, 2024-08-19T01:43:43.363 INFO:teuthology.orchestra.run.smithi117.stdout: "ireq_exportdir": 0, 2024-08-19T01:43:43.363 INFO:teuthology.orchestra.run.smithi117.stdout: "ireq_flush": 0, 2024-08-19T01:43:43.363 INFO:teuthology.orchestra.run.smithi117.stdout: "ireq_fragmentdir": 0, 2024-08-19T01:43:43.363 INFO:teuthology.orchestra.run.smithi117.stdout: "ireq_fragstats": 0, 2024-08-19T01:43:43.363 INFO:teuthology.orchestra.run.smithi117.stdout: "ireq_inodestats": 0 2024-08-19T01:43:43.363 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-08-19T01:43:43.363 INFO:teuthology.orchestra.run.smithi117.stdout: "mds_log": { 2024-08-19T01:43:43.363 INFO:teuthology.orchestra.run.smithi117.stdout: "evlrg": 0, 2024-08-19T01:43:43.363 INFO:teuthology.orchestra.run.smithi117.stdout: "evadd": 0, 2024-08-19T01:43:43.363 INFO:teuthology.orchestra.run.smithi117.stdout: "evex": 0, 2024-08-19T01:43:43.363 INFO:teuthology.orchestra.run.smithi117.stdout: "evtrm": 0, 2024-08-19T01:43:43.364 INFO:teuthology.orchestra.run.smithi117.stdout: "ev": 2278, 2024-08-19T01:43:43.364 INFO:teuthology.orchestra.run.smithi117.stdout: "evexg": 0, 2024-08-19T01:43:43.364 INFO:teuthology.orchestra.run.smithi117.stdout: "evexd": 0, 2024-08-19T01:43:43.364 INFO:teuthology.orchestra.run.smithi117.stdout: "segadd": 0, 2024-08-19T01:43:43.364 INFO:teuthology.orchestra.run.smithi117.stdout: "segex": 0, 2024-08-19T01:43:43.364 INFO:teuthology.orchestra.run.smithi117.stdout: "segtrm": 0, 2024-08-19T01:43:43.364 INFO:teuthology.orchestra.run.smithi117.stdout: "seg": 4, 2024-08-19T01:43:43.364 INFO:teuthology.orchestra.run.smithi117.stdout: "segmjr": 2, 2024-08-19T01:43:43.364 INFO:teuthology.orchestra.run.smithi117.stdout: "segexg": 0, 2024-08-19T01:43:43.364 INFO:teuthology.orchestra.run.smithi117.stdout: "segexd": 0, 2024-08-19T01:43:43.364 INFO:teuthology.orchestra.run.smithi117.stdout: "expos": 4194304, 2024-08-19T01:43:43.364 INFO:teuthology.orchestra.run.smithi117.stdout: "wrpos": 7552280, 2024-08-19T01:43:43.364 INFO:teuthology.orchestra.run.smithi117.stdout: "rdpos": 7551142, 2024-08-19T01:43:43.365 INFO:teuthology.orchestra.run.smithi117.stdout: "jlat": { 2024-08-19T01:43:43.365 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2024-08-19T01:43:43.365 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2024-08-19T01:43:43.365 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2024-08-19T01:43:43.365 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-08-19T01:43:43.365 INFO:teuthology.orchestra.run.smithi117.stdout: "replayed": 2278 2024-08-19T01:43:43.365 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-08-19T01:43:43.365 INFO:teuthology.orchestra.run.smithi117.stdout: "mds_mem": { 2024-08-19T01:43:43.365 INFO:teuthology.orchestra.run.smithi117.stdout: "ino": 6, 2024-08-19T01:43:43.365 INFO:teuthology.orchestra.run.smithi117.stdout: "ino+": 471, 2024-08-19T01:43:43.365 INFO:teuthology.orchestra.run.smithi117.stdout: "ino-": 465, 2024-08-19T01:43:43.365 INFO:teuthology.orchestra.run.smithi117.stdout: "dir": 4, 2024-08-19T01:43:43.366 INFO:teuthology.orchestra.run.smithi117.stdout: "dir+": 31, 2024-08-19T01:43:43.366 INFO:teuthology.orchestra.run.smithi117.stdout: "dir-": 27, 2024-08-19T01:43:43.366 INFO:teuthology.orchestra.run.smithi117.stdout: "dn": 451, 2024-08-19T01:43:43.366 INFO:teuthology.orchestra.run.smithi117.stdout: "dn+": 1199, 2024-08-19T01:43:43.366 INFO:teuthology.orchestra.run.smithi117.stdout: "dn-": 748, 2024-08-19T01:43:43.366 INFO:teuthology.orchestra.run.smithi117.stdout: "cap": 0, 2024-08-19T01:43:43.366 INFO:teuthology.orchestra.run.smithi117.stdout: "cap+": 0, 2024-08-19T01:43:43.366 INFO:teuthology.orchestra.run.smithi117.stdout: "cap-": 0, 2024-08-19T01:43:43.366 INFO:teuthology.orchestra.run.smithi117.stdout: "rss": 51284, 2024-08-19T01:43:43.366 INFO:teuthology.orchestra.run.smithi117.stdout: "heap": 215328 2024-08-19T01:43:43.366 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-08-19T01:43:43.367 INFO:teuthology.orchestra.run.smithi117.stdout: "mds_server": { 2024-08-19T01:43:43.367 INFO:teuthology.orchestra.run.smithi117.stdout: "dispatch_client_request": 0, 2024-08-19T01:43:43.367 INFO:teuthology.orchestra.run.smithi117.stdout: "dispatch_server_request": 0, 2024-08-19T01:43:43.367 INFO:teuthology.orchestra.run.smithi117.stdout: "handle_client_request": 0, 2024-08-19T01:43:43.367 INFO:teuthology.orchestra.run.smithi117.stdout: "handle_client_session": 0, 2024-08-19T01:43:43.367 INFO:teuthology.orchestra.run.smithi117.stdout: "handle_peer_request": 0, 2024-08-19T01:43:43.367 INFO:teuthology.orchestra.run.smithi117.stdout: "req_create_latency": { 2024-08-19T01:43:43.367 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2024-08-19T01:43:43.367 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2024-08-19T01:43:43.367 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2024-08-19T01:43:43.367 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-08-19T01:43:43.367 INFO:teuthology.orchestra.run.smithi117.stdout: "req_getattr_latency": { 2024-08-19T01:43:43.367 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2024-08-19T01:43:43.367 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2024-08-19T01:43:43.368 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2024-08-19T01:43:43.368 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-08-19T01:43:43.368 INFO:teuthology.orchestra.run.smithi117.stdout: "req_getfilelock_latency": { 2024-08-19T01:43:43.368 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2024-08-19T01:43:43.368 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2024-08-19T01:43:43.368 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2024-08-19T01:43:43.368 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-08-19T01:43:43.368 INFO:teuthology.orchestra.run.smithi117.stdout: "req_link_latency": { 2024-08-19T01:43:43.368 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2024-08-19T01:43:43.368 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2024-08-19T01:43:43.368 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2024-08-19T01:43:43.368 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-08-19T01:43:43.368 INFO:teuthology.orchestra.run.smithi117.stdout: "req_lookup_latency": { 2024-08-19T01:43:43.369 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2024-08-19T01:43:43.369 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2024-08-19T01:43:43.369 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2024-08-19T01:43:43.369 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-08-19T01:43:43.369 INFO:teuthology.orchestra.run.smithi117.stdout: "req_lookuphash_latency": { 2024-08-19T01:43:43.369 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2024-08-19T01:43:43.369 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2024-08-19T01:43:43.369 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2024-08-19T01:43:43.369 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-08-19T01:43:43.369 INFO:teuthology.orchestra.run.smithi117.stdout: "req_lookupino_latency": { 2024-08-19T01:43:43.369 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2024-08-19T01:43:43.369 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2024-08-19T01:43:43.369 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2024-08-19T01:43:43.369 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-08-19T01:43:43.370 INFO:teuthology.orchestra.run.smithi117.stdout: "req_lookupname_latency": { 2024-08-19T01:43:43.370 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2024-08-19T01:43:43.370 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2024-08-19T01:43:43.370 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2024-08-19T01:43:43.370 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-08-19T01:43:43.370 INFO:teuthology.orchestra.run.smithi117.stdout: "req_lookupparent_latency": { 2024-08-19T01:43:43.370 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2024-08-19T01:43:43.370 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2024-08-19T01:43:43.370 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2024-08-19T01:43:43.370 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-08-19T01:43:43.370 INFO:teuthology.orchestra.run.smithi117.stdout: "req_lookupsnap_latency": { 2024-08-19T01:43:43.370 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2024-08-19T01:43:43.371 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2024-08-19T01:43:43.371 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2024-08-19T01:43:43.371 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-08-19T01:43:43.371 INFO:teuthology.orchestra.run.smithi117.stdout: "req_lssnap_latency": { 2024-08-19T01:43:43.371 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2024-08-19T01:43:43.371 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2024-08-19T01:43:43.371 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2024-08-19T01:43:43.371 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-08-19T01:43:43.371 INFO:teuthology.orchestra.run.smithi117.stdout: "req_mkdir_latency": { 2024-08-19T01:43:43.371 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2024-08-19T01:43:43.371 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2024-08-19T01:43:43.371 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2024-08-19T01:43:43.372 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-08-19T01:43:43.372 INFO:teuthology.orchestra.run.smithi117.stdout: "req_mknod_latency": { 2024-08-19T01:43:43.372 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2024-08-19T01:43:43.372 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2024-08-19T01:43:43.372 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2024-08-19T01:43:43.372 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-08-19T01:43:43.372 INFO:teuthology.orchestra.run.smithi117.stdout: "req_mksnap_latency": { 2024-08-19T01:43:43.372 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2024-08-19T01:43:43.372 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2024-08-19T01:43:43.372 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2024-08-19T01:43:43.372 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-08-19T01:43:43.372 INFO:teuthology.orchestra.run.smithi117.stdout: "req_open_latency": { 2024-08-19T01:43:43.372 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2024-08-19T01:43:43.373 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2024-08-19T01:43:43.373 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2024-08-19T01:43:43.373 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-08-19T01:43:43.373 INFO:teuthology.orchestra.run.smithi117.stdout: "req_readdir_latency": { 2024-08-19T01:43:43.373 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2024-08-19T01:43:43.373 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2024-08-19T01:43:43.373 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2024-08-19T01:43:43.373 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-08-19T01:43:43.373 INFO:teuthology.orchestra.run.smithi117.stdout: "req_rename_latency": { 2024-08-19T01:43:43.373 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2024-08-19T01:43:43.373 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2024-08-19T01:43:43.373 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2024-08-19T01:43:43.374 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-08-19T01:43:43.374 INFO:teuthology.orchestra.run.smithi117.stdout: "req_renamesnap_latency": { 2024-08-19T01:43:43.374 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2024-08-19T01:43:43.374 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2024-08-19T01:43:43.374 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2024-08-19T01:43:43.374 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-08-19T01:43:43.374 INFO:teuthology.orchestra.run.smithi117.stdout: "req_snapdiff_latency": { 2024-08-19T01:43:43.374 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2024-08-19T01:43:43.374 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2024-08-19T01:43:43.374 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2024-08-19T01:43:43.374 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-08-19T01:43:43.375 INFO:teuthology.orchestra.run.smithi117.stdout: "req_rmdir_latency": { 2024-08-19T01:43:43.375 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2024-08-19T01:43:43.375 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2024-08-19T01:43:43.375 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2024-08-19T01:43:43.375 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-08-19T01:43:43.375 INFO:teuthology.orchestra.run.smithi117.stdout: "req_rmsnap_latency": { 2024-08-19T01:43:43.375 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2024-08-19T01:43:43.375 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2024-08-19T01:43:43.375 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2024-08-19T01:43:43.375 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-08-19T01:43:43.375 INFO:teuthology.orchestra.run.smithi117.stdout: "req_rmxattr_latency": { 2024-08-19T01:43:43.375 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2024-08-19T01:43:43.376 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2024-08-19T01:43:43.376 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2024-08-19T01:43:43.376 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-08-19T01:43:43.376 INFO:teuthology.orchestra.run.smithi117.stdout: "req_setattr_latency": { 2024-08-19T01:43:43.376 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2024-08-19T01:43:43.376 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2024-08-19T01:43:43.376 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2024-08-19T01:43:43.376 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-08-19T01:43:43.376 INFO:teuthology.orchestra.run.smithi117.stdout: "req_setdirlayout_latency": { 2024-08-19T01:43:43.376 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2024-08-19T01:43:43.376 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2024-08-19T01:43:43.376 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2024-08-19T01:43:43.376 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-08-19T01:43:43.377 INFO:teuthology.orchestra.run.smithi117.stdout: "req_setfilelock_latency": { 2024-08-19T01:43:43.377 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2024-08-19T01:43:43.377 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2024-08-19T01:43:43.377 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2024-08-19T01:43:43.377 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-08-19T01:43:43.377 INFO:teuthology.orchestra.run.smithi117.stdout: "req_setlayout_latency": { 2024-08-19T01:43:43.377 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2024-08-19T01:43:43.377 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2024-08-19T01:43:43.377 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2024-08-19T01:43:43.377 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-08-19T01:43:43.377 INFO:teuthology.orchestra.run.smithi117.stdout: "req_setxattr_latency": { 2024-08-19T01:43:43.377 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2024-08-19T01:43:43.378 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2024-08-19T01:43:43.378 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2024-08-19T01:43:43.378 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-08-19T01:43:43.378 INFO:teuthology.orchestra.run.smithi117.stdout: "req_symlink_latency": { 2024-08-19T01:43:43.378 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2024-08-19T01:43:43.378 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2024-08-19T01:43:43.378 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2024-08-19T01:43:43.378 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-08-19T01:43:43.378 INFO:teuthology.orchestra.run.smithi117.stdout: "req_unlink_latency": { 2024-08-19T01:43:43.378 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2024-08-19T01:43:43.378 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2024-08-19T01:43:43.378 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2024-08-19T01:43:43.379 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-08-19T01:43:43.379 INFO:teuthology.orchestra.run.smithi117.stdout: "cap_revoke_eviction": 0, 2024-08-19T01:43:43.379 INFO:teuthology.orchestra.run.smithi117.stdout: "cap_acquisition_throttle": 0, 2024-08-19T01:43:43.379 INFO:teuthology.orchestra.run.smithi117.stdout: "req_getvxattr_latency": { 2024-08-19T01:43:43.379 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2024-08-19T01:43:43.379 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2024-08-19T01:43:43.379 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2024-08-19T01:43:43.379 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-08-19T01:43:43.379 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-08-19T01:43:43.379 INFO:teuthology.orchestra.run.smithi117.stdout: "mds_sessions": { 2024-08-19T01:43:43.379 INFO:teuthology.orchestra.run.smithi117.stdout: "session_count": 1, 2024-08-19T01:43:43.379 INFO:teuthology.orchestra.run.smithi117.stdout: "session_add": 3, 2024-08-19T01:43:43.379 INFO:teuthology.orchestra.run.smithi117.stdout: "session_remove": 2, 2024-08-19T01:43:43.380 INFO:teuthology.orchestra.run.smithi117.stdout: "sessions_open": 2, 2024-08-19T01:43:43.380 INFO:teuthology.orchestra.run.smithi117.stdout: "sessions_stale": 0, 2024-08-19T01:43:43.380 INFO:teuthology.orchestra.run.smithi117.stdout: "total_load": 0, 2024-08-19T01:43:43.380 INFO:teuthology.orchestra.run.smithi117.stdout: "average_load": 0, 2024-08-19T01:43:43.380 INFO:teuthology.orchestra.run.smithi117.stdout: "avg_session_uptime": 3007, 2024-08-19T01:43:43.380 INFO:teuthology.orchestra.run.smithi117.stdout: "mdthresh_evicted": 0 2024-08-19T01:43:43.380 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-08-19T01:43:43.380 INFO:teuthology.orchestra.run.smithi117.stdout: "mempool": { 2024-08-19T01:43:43.380 INFO:teuthology.orchestra.run.smithi117.stdout: "bloom_filter_bytes": 0, 2024-08-19T01:43:43.380 INFO:teuthology.orchestra.run.smithi117.stdout: "bloom_filter_items": 0, 2024-08-19T01:43:43.380 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_alloc_bytes": 0, 2024-08-19T01:43:43.380 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_alloc_items": 0, 2024-08-19T01:43:43.381 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_cache_data_bytes": 0, 2024-08-19T01:43:43.381 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_cache_data_items": 0, 2024-08-19T01:43:43.381 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_cache_onode_bytes": 0, 2024-08-19T01:43:43.381 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_cache_onode_items": 0, 2024-08-19T01:43:43.381 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_cache_meta_bytes": 0, 2024-08-19T01:43:43.381 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_cache_meta_items": 0, 2024-08-19T01:43:43.381 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_cache_other_bytes": 0, 2024-08-19T01:43:43.381 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_cache_other_items": 0, 2024-08-19T01:43:43.381 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_cache_buffer_bytes": 0, 2024-08-19T01:43:43.381 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_cache_buffer_items": 0, 2024-08-19T01:43:43.381 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_extent_bytes": 0, 2024-08-19T01:43:43.381 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_extent_items": 0, 2024-08-19T01:43:43.381 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_blob_bytes": 0, 2024-08-19T01:43:43.382 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_blob_items": 0, 2024-08-19T01:43:43.382 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_shared_blob_bytes": 0, 2024-08-19T01:43:43.382 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_shared_blob_items": 0, 2024-08-19T01:43:43.382 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_inline_bl_bytes": 0, 2024-08-19T01:43:43.382 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_inline_bl_items": 0, 2024-08-19T01:43:43.382 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_fsck_bytes": 0, 2024-08-19T01:43:43.382 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_fsck_items": 0, 2024-08-19T01:43:43.382 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_txc_bytes": 0, 2024-08-19T01:43:43.382 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_txc_items": 0, 2024-08-19T01:43:43.382 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_writing_deferred_bytes": 0, 2024-08-19T01:43:43.383 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_writing_deferred_items": 0, 2024-08-19T01:43:43.383 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_writing_bytes": 0, 2024-08-19T01:43:43.383 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_writing_items": 0, 2024-08-19T01:43:43.383 INFO:teuthology.orchestra.run.smithi117.stdout: "bluefs_bytes": 0, 2024-08-19T01:43:43.383 INFO:teuthology.orchestra.run.smithi117.stdout: "bluefs_items": 0, 2024-08-19T01:43:43.383 INFO:teuthology.orchestra.run.smithi117.stdout: "bluefs_file_reader_bytes": 0, 2024-08-19T01:43:43.383 INFO:teuthology.orchestra.run.smithi117.stdout: "bluefs_file_reader_items": 0, 2024-08-19T01:43:43.383 INFO:teuthology.orchestra.run.smithi117.stdout: "bluefs_file_writer_bytes": 0, 2024-08-19T01:43:43.383 INFO:teuthology.orchestra.run.smithi117.stdout: "bluefs_file_writer_items": 0, 2024-08-19T01:43:43.383 INFO:teuthology.orchestra.run.smithi117.stdout: "buffer_anon_bytes": 181321, 2024-08-19T01:43:43.383 INFO:teuthology.orchestra.run.smithi117.stdout: "buffer_anon_items": 47, 2024-08-19T01:43:43.384 INFO:teuthology.orchestra.run.smithi117.stdout: "buffer_meta_bytes": 0, 2024-08-19T01:43:43.384 INFO:teuthology.orchestra.run.smithi117.stdout: "buffer_meta_items": 0, 2024-08-19T01:43:43.384 INFO:teuthology.orchestra.run.smithi117.stdout: "osd_bytes": 0, 2024-08-19T01:43:43.384 INFO:teuthology.orchestra.run.smithi117.stdout: "osd_items": 0, 2024-08-19T01:43:43.384 INFO:teuthology.orchestra.run.smithi117.stdout: "osd_mapbl_bytes": 0, 2024-08-19T01:43:43.384 INFO:teuthology.orchestra.run.smithi117.stdout: "osd_mapbl_items": 0, 2024-08-19T01:43:43.384 INFO:teuthology.orchestra.run.smithi117.stdout: "osd_pglog_bytes": 0, 2024-08-19T01:43:43.384 INFO:teuthology.orchestra.run.smithi117.stdout: "osd_pglog_items": 0, 2024-08-19T01:43:43.384 INFO:teuthology.orchestra.run.smithi117.stdout: "osdmap_bytes": 7688, 2024-08-19T01:43:43.384 INFO:teuthology.orchestra.run.smithi117.stdout: "osdmap_items": 158, 2024-08-19T01:43:43.384 INFO:teuthology.orchestra.run.smithi117.stdout: "osdmap_mapping_bytes": 0, 2024-08-19T01:43:43.384 INFO:teuthology.orchestra.run.smithi117.stdout: "osdmap_mapping_items": 0, 2024-08-19T01:43:43.385 INFO:teuthology.orchestra.run.smithi117.stdout: "pgmap_bytes": 0, 2024-08-19T01:43:43.385 INFO:teuthology.orchestra.run.smithi117.stdout: "pgmap_items": 0, 2024-08-19T01:43:43.385 INFO:teuthology.orchestra.run.smithi117.stdout: "mds_co_bytes": 336568, 2024-08-19T01:43:43.385 INFO:teuthology.orchestra.run.smithi117.stdout: "mds_co_items": 1855, 2024-08-19T01:43:43.385 INFO:teuthology.orchestra.run.smithi117.stdout: "unittest_1_bytes": 0, 2024-08-19T01:43:43.385 INFO:teuthology.orchestra.run.smithi117.stdout: "unittest_1_items": 0, 2024-08-19T01:43:43.385 INFO:teuthology.orchestra.run.smithi117.stdout: "unittest_2_bytes": 0, 2024-08-19T01:43:43.385 INFO:teuthology.orchestra.run.smithi117.stdout: "unittest_2_items": 0 2024-08-19T01:43:43.385 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-08-19T01:43:43.385 INFO:teuthology.orchestra.run.smithi117.stdout: "objecter": { 2024-08-19T01:43:43.385 INFO:teuthology.orchestra.run.smithi117.stdout: "op_active": 0, 2024-08-19T01:43:43.385 INFO:teuthology.orchestra.run.smithi117.stdout: "op_laggy": 0, 2024-08-19T01:43:43.385 INFO:teuthology.orchestra.run.smithi117.stdout: "op_send": 1948, 2024-08-19T01:43:43.386 INFO:teuthology.orchestra.run.smithi117.stdout: "op_send_bytes": 21, 2024-08-19T01:43:43.386 INFO:teuthology.orchestra.run.smithi117.stdout: "op_resend": 3, 2024-08-19T01:43:43.386 INFO:teuthology.orchestra.run.smithi117.stdout: "op_reply": 1948, 2024-08-19T01:43:43.386 INFO:teuthology.orchestra.run.smithi117.stdout: "op_latency": { 2024-08-19T01:43:43.386 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 1948, 2024-08-19T01:43:43.386 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 1.801964359, 2024-08-19T01:43:43.386 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000925033 2024-08-19T01:43:43.386 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-08-19T01:43:43.386 INFO:teuthology.orchestra.run.smithi117.stdout: "op_inflight": 0, 2024-08-19T01:43:43.386 INFO:teuthology.orchestra.run.smithi117.stdout: "oplen_avg": { 2024-08-19T01:43:43.386 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 1948, 2024-08-19T01:43:43.386 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 1949 2024-08-19T01:43:43.387 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-08-19T01:43:43.387 INFO:teuthology.orchestra.run.smithi117.stdout: "op": 1948, 2024-08-19T01:43:43.387 INFO:teuthology.orchestra.run.smithi117.stdout: "op_r": 1948, 2024-08-19T01:43:43.387 INFO:teuthology.orchestra.run.smithi117.stdout: "op_w": 0, 2024-08-19T01:43:43.387 INFO:teuthology.orchestra.run.smithi117.stdout: "op_rmw": 0, 2024-08-19T01:43:43.387 INFO:teuthology.orchestra.run.smithi117.stdout: "op_pg": 0, 2024-08-19T01:43:43.387 INFO:teuthology.orchestra.run.smithi117.stdout: "osdop_stat": 1228, 2024-08-19T01:43:43.387 INFO:teuthology.orchestra.run.smithi117.stdout: "osdop_create": 0, 2024-08-19T01:43:43.387 INFO:teuthology.orchestra.run.smithi117.stdout: "osdop_read": 718, 2024-08-19T01:43:43.387 INFO:teuthology.orchestra.run.smithi117.stdout: "osdop_write": 0, 2024-08-19T01:43:43.387 INFO:teuthology.orchestra.run.smithi117.stdout: "osdop_writefull": 0, 2024-08-19T01:43:43.387 INFO:teuthology.orchestra.run.smithi117.stdout: "osdop_writesame": 0, 2024-08-19T01:43:43.387 INFO:teuthology.orchestra.run.smithi117.stdout: "osdop_append": 0, 2024-08-19T01:43:43.387 INFO:teuthology.orchestra.run.smithi117.stdout: "osdop_zero": 0, 2024-08-19T01:43:43.388 INFO:teuthology.orchestra.run.smithi117.stdout: "osdop_truncate": 0, 2024-08-19T01:43:43.388 INFO:teuthology.orchestra.run.smithi117.stdout: "osdop_delete": 0, 2024-08-19T01:43:43.388 INFO:teuthology.orchestra.run.smithi117.stdout: "osdop_mapext": 0, 2024-08-19T01:43:43.388 INFO:teuthology.orchestra.run.smithi117.stdout: "osdop_sparse_read": 0, 2024-08-19T01:43:43.388 INFO:teuthology.orchestra.run.smithi117.stdout: "osdop_clonerange": 0, 2024-08-19T01:43:43.388 INFO:teuthology.orchestra.run.smithi117.stdout: "osdop_getxattr": 1, 2024-08-19T01:43:43.388 INFO:teuthology.orchestra.run.smithi117.stdout: "osdop_setxattr": 0, 2024-08-19T01:43:43.388 INFO:teuthology.orchestra.run.smithi117.stdout: "osdop_cmpxattr": 0, 2024-08-19T01:43:43.388 INFO:teuthology.orchestra.run.smithi117.stdout: "osdop_rmxattr": 0, 2024-08-19T01:43:43.388 INFO:teuthology.orchestra.run.smithi117.stdout: "osdop_resetxattrs": 0, 2024-08-19T01:43:43.388 INFO:teuthology.orchestra.run.smithi117.stdout: "osdop_call": 0, 2024-08-19T01:43:43.388 INFO:teuthology.orchestra.run.smithi117.stdout: "osdop_watch": 0, 2024-08-19T01:43:43.388 INFO:teuthology.orchestra.run.smithi117.stdout: "osdop_notify": 0, 2024-08-19T01:43:43.389 INFO:teuthology.orchestra.run.smithi117.stdout: "osdop_src_cmpxattr": 0, 2024-08-19T01:43:43.389 INFO:teuthology.orchestra.run.smithi117.stdout: "osdop_pgls": 0, 2024-08-19T01:43:43.389 INFO:teuthology.orchestra.run.smithi117.stdout: "osdop_pgls_filter": 0, 2024-08-19T01:43:43.389 INFO:teuthology.orchestra.run.smithi117.stdout: "osdop_other": 0, 2024-08-19T01:43:43.389 INFO:teuthology.orchestra.run.smithi117.stdout: "linger_active": 0, 2024-08-19T01:43:43.389 INFO:teuthology.orchestra.run.smithi117.stdout: "linger_send": 0, 2024-08-19T01:43:43.389 INFO:teuthology.orchestra.run.smithi117.stdout: "linger_resend": 0, 2024-08-19T01:43:43.389 INFO:teuthology.orchestra.run.smithi117.stdout: "linger_ping": 0, 2024-08-19T01:43:43.389 INFO:teuthology.orchestra.run.smithi117.stdout: "poolop_active": 0, 2024-08-19T01:43:43.389 INFO:teuthology.orchestra.run.smithi117.stdout: "poolop_send": 0, 2024-08-19T01:43:43.389 INFO:teuthology.orchestra.run.smithi117.stdout: "poolop_resend": 0, 2024-08-19T01:43:43.389 INFO:teuthology.orchestra.run.smithi117.stdout: "poolstat_active": 0, 2024-08-19T01:43:43.390 INFO:teuthology.orchestra.run.smithi117.stdout: "poolstat_send": 0, 2024-08-19T01:43:43.390 INFO:teuthology.orchestra.run.smithi117.stdout: "poolstat_resend": 0, 2024-08-19T01:43:43.390 INFO:teuthology.orchestra.run.smithi117.stdout: "statfs_active": 0, 2024-08-19T01:43:43.390 INFO:teuthology.orchestra.run.smithi117.stdout: "statfs_send": 0, 2024-08-19T01:43:43.390 INFO:teuthology.orchestra.run.smithi117.stdout: "statfs_resend": 0, 2024-08-19T01:43:43.390 INFO:teuthology.orchestra.run.smithi117.stdout: "command_active": 0, 2024-08-19T01:43:43.390 INFO:teuthology.orchestra.run.smithi117.stdout: "command_send": 0, 2024-08-19T01:43:43.390 INFO:teuthology.orchestra.run.smithi117.stdout: "command_resend": 0, 2024-08-19T01:43:43.390 INFO:teuthology.orchestra.run.smithi117.stdout: "map_epoch": 86, 2024-08-19T01:43:43.390 INFO:teuthology.orchestra.run.smithi117.stdout: "map_full": 0, 2024-08-19T01:43:43.390 INFO:teuthology.orchestra.run.smithi117.stdout: "map_inc": 2, 2024-08-19T01:43:43.390 INFO:teuthology.orchestra.run.smithi117.stdout: "osd_sessions": 4, 2024-08-19T01:43:43.390 INFO:teuthology.orchestra.run.smithi117.stdout: "osd_session_open": 4, 2024-08-19T01:43:43.391 INFO:teuthology.orchestra.run.smithi117.stdout: "osd_session_close": 0, 2024-08-19T01:43:43.391 INFO:teuthology.orchestra.run.smithi117.stdout: "osd_laggy": 0, 2024-08-19T01:43:43.391 INFO:teuthology.orchestra.run.smithi117.stdout: "omap_wr": 0, 2024-08-19T01:43:43.391 INFO:teuthology.orchestra.run.smithi117.stdout: "omap_rd": 2, 2024-08-19T01:43:43.391 INFO:teuthology.orchestra.run.smithi117.stdout: "omap_del": 0 2024-08-19T01:43:43.391 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-08-19T01:43:43.391 INFO:teuthology.orchestra.run.smithi117.stdout: "oft": { 2024-08-19T01:43:43.391 INFO:teuthology.orchestra.run.smithi117.stdout: "omap_total_objs": 0, 2024-08-19T01:43:43.391 INFO:teuthology.orchestra.run.smithi117.stdout: "omap_total_kv_pairs": 0, 2024-08-19T01:43:43.391 INFO:teuthology.orchestra.run.smithi117.stdout: "omap_total_updates": 0, 2024-08-19T01:43:43.391 INFO:teuthology.orchestra.run.smithi117.stdout: "omap_total_removes": 0 2024-08-19T01:43:43.392 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-08-19T01:43:43.392 INFO:teuthology.orchestra.run.smithi117.stdout: "osd-slow-ops": { 2024-08-19T01:43:43.392 INFO:teuthology.orchestra.run.smithi117.stdout: "slow_ops_count": 0 2024-08-19T01:43:43.392 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-08-19T01:43:43.392 INFO:teuthology.orchestra.run.smithi117.stdout: "purge_queue": { 2024-08-19T01:43:43.392 INFO:teuthology.orchestra.run.smithi117.stdout: "pq_executing_ops": 0, 2024-08-19T01:43:43.392 INFO:teuthology.orchestra.run.smithi117.stdout: "pq_executing_ops_high_water": 0, 2024-08-19T01:43:43.392 INFO:teuthology.orchestra.run.smithi117.stdout: "pq_executing": 0, 2024-08-19T01:43:43.392 INFO:teuthology.orchestra.run.smithi117.stdout: "pq_executing_high_water": 0, 2024-08-19T01:43:43.392 INFO:teuthology.orchestra.run.smithi117.stdout: "pq_executed_ops": 0, 2024-08-19T01:43:43.392 INFO:teuthology.orchestra.run.smithi117.stdout: "pq_executed": 0, 2024-08-19T01:43:43.392 INFO:teuthology.orchestra.run.smithi117.stdout: "pq_item_in_journal": 0 2024-08-19T01:43:43.392 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-08-19T01:43:43.392 INFO:teuthology.orchestra.run.smithi117.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-08-19T01:43:43.393 INFO:teuthology.orchestra.run.smithi117.stdout: "val": 0, 2024-08-19T01:43:43.393 INFO:teuthology.orchestra.run.smithi117.stdout: "max": 104857600, 2024-08-19T01:43:43.393 INFO:teuthology.orchestra.run.smithi117.stdout: "get_started": 0, 2024-08-19T01:43:43.393 INFO:teuthology.orchestra.run.smithi117.stdout: "get": 2157, 2024-08-19T01:43:43.393 INFO:teuthology.orchestra.run.smithi117.stdout: "get_sum": 4497296, 2024-08-19T01:43:43.393 INFO:teuthology.orchestra.run.smithi117.stdout: "get_or_fail_fail": 0, 2024-08-19T01:43:43.393 INFO:teuthology.orchestra.run.smithi117.stdout: "get_or_fail_success": 2157, 2024-08-19T01:43:43.393 INFO:teuthology.orchestra.run.smithi117.stdout: "take": 0, 2024-08-19T01:43:43.393 INFO:teuthology.orchestra.run.smithi117.stdout: "take_sum": 0, 2024-08-19T01:43:43.393 INFO:teuthology.orchestra.run.smithi117.stdout: "put": 2157, 2024-08-19T01:43:43.393 INFO:teuthology.orchestra.run.smithi117.stdout: "put_sum": 4497296, 2024-08-19T01:43:43.393 INFO:teuthology.orchestra.run.smithi117.stdout: "wait": { 2024-08-19T01:43:43.394 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2024-08-19T01:43:43.394 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2024-08-19T01:43:43.394 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2024-08-19T01:43:43.394 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-08-19T01:43:43.394 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-08-19T01:43:43.394 INFO:teuthology.orchestra.run.smithi117.stdout: "throttle-objecter_bytes": { 2024-08-19T01:43:43.394 INFO:teuthology.orchestra.run.smithi117.stdout: "val": 0, 2024-08-19T01:43:43.394 INFO:teuthology.orchestra.run.smithi117.stdout: "max": 104857600, 2024-08-19T01:43:43.394 INFO:teuthology.orchestra.run.smithi117.stdout: "get_started": 0, 2024-08-19T01:43:43.394 INFO:teuthology.orchestra.run.smithi117.stdout: "get": 0, 2024-08-19T01:43:43.394 INFO:teuthology.orchestra.run.smithi117.stdout: "get_sum": 0, 2024-08-19T01:43:43.394 INFO:teuthology.orchestra.run.smithi117.stdout: "get_or_fail_fail": 0, 2024-08-19T01:43:43.395 INFO:teuthology.orchestra.run.smithi117.stdout: "get_or_fail_success": 0, 2024-08-19T01:43:43.395 INFO:teuthology.orchestra.run.smithi117.stdout: "take": 1948, 2024-08-19T01:43:43.395 INFO:teuthology.orchestra.run.smithi117.stdout: "take_sum": 3357981, 2024-08-19T01:43:43.395 INFO:teuthology.orchestra.run.smithi117.stdout: "put": 1948, 2024-08-19T01:43:43.395 INFO:teuthology.orchestra.run.smithi117.stdout: "put_sum": 3357981, 2024-08-19T01:43:43.395 INFO:teuthology.orchestra.run.smithi117.stdout: "wait": { 2024-08-19T01:43:43.395 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2024-08-19T01:43:43.395 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2024-08-19T01:43:43.395 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2024-08-19T01:43:43.395 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-08-19T01:43:43.395 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-08-19T01:43:43.395 INFO:teuthology.orchestra.run.smithi117.stdout: "throttle-objecter_ops": { 2024-08-19T01:43:43.395 INFO:teuthology.orchestra.run.smithi117.stdout: "val": 0, 2024-08-19T01:43:43.396 INFO:teuthology.orchestra.run.smithi117.stdout: "max": 1024, 2024-08-19T01:43:43.396 INFO:teuthology.orchestra.run.smithi117.stdout: "get_started": 0, 2024-08-19T01:43:43.396 INFO:teuthology.orchestra.run.smithi117.stdout: "get": 0, 2024-08-19T01:43:43.396 INFO:teuthology.orchestra.run.smithi117.stdout: "get_sum": 0, 2024-08-19T01:43:43.396 INFO:teuthology.orchestra.run.smithi117.stdout: "get_or_fail_fail": 0, 2024-08-19T01:43:43.396 INFO:teuthology.orchestra.run.smithi117.stdout: "get_or_fail_success": 0, 2024-08-19T01:43:43.396 INFO:teuthology.orchestra.run.smithi117.stdout: "take": 1948, 2024-08-19T01:43:43.396 INFO:teuthology.orchestra.run.smithi117.stdout: "take_sum": 1948, 2024-08-19T01:43:43.396 INFO:teuthology.orchestra.run.smithi117.stdout: "put": 1948, 2024-08-19T01:43:43.396 INFO:teuthology.orchestra.run.smithi117.stdout: "put_sum": 1948, 2024-08-19T01:43:43.396 INFO:teuthology.orchestra.run.smithi117.stdout: "wait": { 2024-08-19T01:43:43.396 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2024-08-19T01:43:43.396 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2024-08-19T01:43:43.396 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2024-08-19T01:43:43.397 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-08-19T01:43:43.397 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-08-19T01:43:43.397 INFO:teuthology.orchestra.run.smithi117.stdout: "throttle-write_buf_throttle": { 2024-08-19T01:43:43.397 INFO:teuthology.orchestra.run.smithi117.stdout: "val": 0, 2024-08-19T01:43:43.397 INFO:teuthology.orchestra.run.smithi117.stdout: "max": 3758096384, 2024-08-19T01:43:43.397 INFO:teuthology.orchestra.run.smithi117.stdout: "get_started": 0, 2024-08-19T01:43:43.397 INFO:teuthology.orchestra.run.smithi117.stdout: "get": 0, 2024-08-19T01:43:43.397 INFO:teuthology.orchestra.run.smithi117.stdout: "get_sum": 0, 2024-08-19T01:43:43.397 INFO:teuthology.orchestra.run.smithi117.stdout: "get_or_fail_fail": 0, 2024-08-19T01:43:43.397 INFO:teuthology.orchestra.run.smithi117.stdout: "get_or_fail_success": 0, 2024-08-19T01:43:43.397 INFO:teuthology.orchestra.run.smithi117.stdout: "take": 0, 2024-08-19T01:43:43.397 INFO:teuthology.orchestra.run.smithi117.stdout: "take_sum": 0, 2024-08-19T01:43:43.397 INFO:teuthology.orchestra.run.smithi117.stdout: "put": 0, 2024-08-19T01:43:43.398 INFO:teuthology.orchestra.run.smithi117.stdout: "put_sum": 0, 2024-08-19T01:43:43.398 INFO:teuthology.orchestra.run.smithi117.stdout: "wait": { 2024-08-19T01:43:43.398 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2024-08-19T01:43:43.398 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2024-08-19T01:43:43.398 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2024-08-19T01:43:43.398 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-08-19T01:43:43.398 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-08-19T01:43:43.398 INFO:teuthology.orchestra.run.smithi117.stdout: "throttle-write_buf_throttle-0x562f0701ab40": { 2024-08-19T01:43:43.398 INFO:teuthology.orchestra.run.smithi117.stdout: "val": 0, 2024-08-19T01:43:43.398 INFO:teuthology.orchestra.run.smithi117.stdout: "max": 3758096384, 2024-08-19T01:43:43.398 INFO:teuthology.orchestra.run.smithi117.stdout: "get_started": 0, 2024-08-19T01:43:43.398 INFO:teuthology.orchestra.run.smithi117.stdout: "get": 0, 2024-08-19T01:43:43.399 INFO:teuthology.orchestra.run.smithi117.stdout: "get_sum": 0, 2024-08-19T01:43:43.399 INFO:teuthology.orchestra.run.smithi117.stdout: "get_or_fail_fail": 0, 2024-08-19T01:43:43.399 INFO:teuthology.orchestra.run.smithi117.stdout: "get_or_fail_success": 0, 2024-08-19T01:43:43.399 INFO:teuthology.orchestra.run.smithi117.stdout: "take": 0, 2024-08-19T01:43:43.399 INFO:teuthology.orchestra.run.smithi117.stdout: "take_sum": 0, 2024-08-19T01:43:43.399 INFO:teuthology.orchestra.run.smithi117.stdout: "put": 0, 2024-08-19T01:43:43.399 INFO:teuthology.orchestra.run.smithi117.stdout: "put_sum": 0, 2024-08-19T01:43:43.399 INFO:teuthology.orchestra.run.smithi117.stdout: "wait": { 2024-08-19T01:43:43.399 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2024-08-19T01:43:43.399 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2024-08-19T01:43:43.399 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2024-08-19T01:43:43.399 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-08-19T01:43:43.400 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-08-19T01:43:43.400 INFO:teuthology.orchestra.run.smithi117.stdout:} 2024-08-19T01:43:44.466 INFO:tasks.check_counter:Daemon mds.e mds.exported=0 2024-08-19T01:43:44.466 INFO:tasks.check_counter:Daemon mds.e mds.imported=0 2024-08-19T01:43:44.466 INFO:tasks.check_counter:Daemon mds.e mds.dir_split=0 2024-08-19T01:43:44.466 DEBUG:tasks.check_counter:Getting stats from h 2024-08-19T01:43:44.466 DEBUG:teuthology.orchestra.run.smithi117:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833 shell --fsid bad8780e-5dc9-11ef-bccf-c7b262605968 -- ceph daemon mds.h perf dump 2024-08-19T01:43:44.727 INFO:teuthology.orchestra.run.smithi117.stderr:Inferring config /var/lib/ceph/bad8780e-5dc9-11ef-bccf-c7b262605968/mon.b/config 2024-08-19T01:43:45.283 INFO:teuthology.orchestra.run.smithi117.stdout:{ 2024-08-19T01:43:45.283 INFO:teuthology.orchestra.run.smithi117.stdout: "AsyncMessenger::Worker-0": { 2024-08-19T01:43:45.283 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_recv_messages": 14353, 2024-08-19T01:43:45.283 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_send_messages": 18800, 2024-08-19T01:43:45.283 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_recv_bytes": 4365046, 2024-08-19T01:43:45.284 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_send_bytes": 9893111, 2024-08-19T01:43:45.284 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_created_connections": 11, 2024-08-19T01:43:45.284 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_active_connections": 3, 2024-08-19T01:43:45.284 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_running_total_time": 0.999883464, 2024-08-19T01:43:45.284 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_running_send_time": 0.469167636, 2024-08-19T01:43:45.284 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_running_recv_time": 0.582246906, 2024-08-19T01:43:45.284 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_running_fast_dispatch_time": 0.044146354, 2024-08-19T01:43:45.284 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_send_messages_queue_lat": { 2024-08-19T01:43:45.284 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 18800, 2024-08-19T01:43:45.284 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.435549799, 2024-08-19T01:43:45.284 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000023167 2024-08-19T01:43:45.284 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-08-19T01:43:45.284 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_handle_ack_lat": { 2024-08-19T01:43:45.284 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 28922, 2024-08-19T01:43:45.285 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.005863636, 2024-08-19T01:43:45.285 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000202 2024-08-19T01:43:45.285 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-08-19T01:43:45.285 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_recv_encrypted_bytes": 0, 2024-08-19T01:43:45.285 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_send_encrypted_bytes": 0 2024-08-19T01:43:45.285 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-08-19T01:43:45.285 INFO:teuthology.orchestra.run.smithi117.stdout: "AsyncMessenger::Worker-1": { 2024-08-19T01:43:45.285 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_recv_messages": 3410, 2024-08-19T01:43:45.285 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_send_messages": 3394, 2024-08-19T01:43:45.285 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_recv_bytes": 1711734, 2024-08-19T01:43:45.285 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_send_bytes": 9043865, 2024-08-19T01:43:45.285 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_created_connections": 10, 2024-08-19T01:43:45.285 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_active_connections": 6, 2024-08-19T01:43:45.286 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_running_total_time": 0.232723457, 2024-08-19T01:43:45.286 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_running_send_time": 0.080326351, 2024-08-19T01:43:45.286 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_running_recv_time": 0.074339272, 2024-08-19T01:43:45.286 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_running_fast_dispatch_time": 0.070199368, 2024-08-19T01:43:45.286 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_send_messages_queue_lat": { 2024-08-19T01:43:45.286 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 3394, 2024-08-19T01:43:45.286 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.144745996, 2024-08-19T01:43:45.286 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000042647 2024-08-19T01:43:45.286 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-08-19T01:43:45.286 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_handle_ack_lat": { 2024-08-19T01:43:45.286 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 9, 2024-08-19T01:43:45.286 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000001605, 2024-08-19T01:43:45.286 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000178 2024-08-19T01:43:45.287 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-08-19T01:43:45.287 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_recv_encrypted_bytes": 810400, 2024-08-19T01:43:45.287 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_send_encrypted_bytes": 129520 2024-08-19T01:43:45.287 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-08-19T01:43:45.287 INFO:teuthology.orchestra.run.smithi117.stdout: "AsyncMessenger::Worker-2": { 2024-08-19T01:43:45.287 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_recv_messages": 28692, 2024-08-19T01:43:45.287 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_send_messages": 36969, 2024-08-19T01:43:45.287 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_recv_bytes": 10013263, 2024-08-19T01:43:45.287 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_send_bytes": 21473550, 2024-08-19T01:43:45.287 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_created_connections": 25, 2024-08-19T01:43:45.287 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_active_connections": 6, 2024-08-19T01:43:45.287 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_running_total_time": 1.202049267, 2024-08-19T01:43:45.287 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_running_send_time": 0.579874562, 2024-08-19T01:43:45.287 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_running_recv_time": 0.746231013, 2024-08-19T01:43:45.288 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_running_fast_dispatch_time": 0.052058520, 2024-08-19T01:43:45.288 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_send_messages_queue_lat": { 2024-08-19T01:43:45.288 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 36965, 2024-08-19T01:43:45.288 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 1.042870467, 2024-08-19T01:43:45.288 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000028212 2024-08-19T01:43:45.288 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-08-19T01:43:45.288 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_handle_ack_lat": { 2024-08-19T01:43:45.288 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 23746, 2024-08-19T01:43:45.288 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.006711854, 2024-08-19T01:43:45.288 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000282 2024-08-19T01:43:45.288 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-08-19T01:43:45.288 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_recv_encrypted_bytes": 1744, 2024-08-19T01:43:45.288 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_send_encrypted_bytes": 326160 2024-08-19T01:43:45.288 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-08-19T01:43:45.288 INFO:teuthology.orchestra.run.smithi117.stdout: "cct": { 2024-08-19T01:43:45.289 INFO:teuthology.orchestra.run.smithi117.stdout: "total_workers": 1, 2024-08-19T01:43:45.289 INFO:teuthology.orchestra.run.smithi117.stdout: "unhealthy_workers": 0 2024-08-19T01:43:45.289 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-08-19T01:43:45.289 INFO:teuthology.orchestra.run.smithi117.stdout: "finisher-MDSRank": { 2024-08-19T01:43:45.289 INFO:teuthology.orchestra.run.smithi117.stdout: "queue_len": 0, 2024-08-19T01:43:45.289 INFO:teuthology.orchestra.run.smithi117.stdout: "complete_latency": { 2024-08-19T01:43:45.289 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 4389, 2024-08-19T01:43:45.289 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 4.439035988, 2024-08-19T01:43:45.289 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.001011400 2024-08-19T01:43:45.289 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-08-19T01:43:45.289 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-08-19T01:43:45.289 INFO:teuthology.orchestra.run.smithi117.stdout: "finisher-PurgeQueue": { 2024-08-19T01:43:45.289 INFO:teuthology.orchestra.run.smithi117.stdout: "queue_len": 0, 2024-08-19T01:43:45.290 INFO:teuthology.orchestra.run.smithi117.stdout: "complete_latency": { 2024-08-19T01:43:45.290 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 3884, 2024-08-19T01:43:45.290 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.330659728, 2024-08-19T01:43:45.290 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000085133 2024-08-19T01:43:45.290 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-08-19T01:43:45.290 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-08-19T01:43:45.290 INFO:teuthology.orchestra.run.smithi117.stdout: "mds": { 2024-08-19T01:43:45.290 INFO:teuthology.orchestra.run.smithi117.stdout: "request": 11037, 2024-08-19T01:43:45.290 INFO:teuthology.orchestra.run.smithi117.stdout: "reply": 11033, 2024-08-19T01:43:45.290 INFO:teuthology.orchestra.run.smithi117.stdout: "reply_latency": { 2024-08-19T01:43:45.290 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 11033, 2024-08-19T01:43:45.290 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 74.624470361, 2024-08-19T01:43:45.290 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.006763751 2024-08-19T01:43:45.291 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-08-19T01:43:45.291 INFO:teuthology.orchestra.run.smithi117.stdout: "slow_reply": 0, 2024-08-19T01:43:45.291 INFO:teuthology.orchestra.run.smithi117.stdout: "forward": 4, 2024-08-19T01:43:45.291 INFO:teuthology.orchestra.run.smithi117.stdout: "dir_fetch_complete": 5, 2024-08-19T01:43:45.291 INFO:teuthology.orchestra.run.smithi117.stdout: "dir_fetch_keys": 0, 2024-08-19T01:43:45.291 INFO:teuthology.orchestra.run.smithi117.stdout: "dir_commit": 71, 2024-08-19T01:43:45.291 INFO:teuthology.orchestra.run.smithi117.stdout: "dir_split": 7, 2024-08-19T01:43:45.291 INFO:teuthology.orchestra.run.smithi117.stdout: "dir_merge": 0, 2024-08-19T01:43:45.291 INFO:teuthology.orchestra.run.smithi117.stdout: "inodes": 15, 2024-08-19T01:43:45.291 INFO:teuthology.orchestra.run.smithi117.stdout: "inodes_top": 0, 2024-08-19T01:43:45.291 INFO:teuthology.orchestra.run.smithi117.stdout: "inodes_bottom": 1, 2024-08-19T01:43:45.291 INFO:teuthology.orchestra.run.smithi117.stdout: "inodes_pin_tail": 14, 2024-08-19T01:43:45.291 INFO:teuthology.orchestra.run.smithi117.stdout: "inodes_pinned": 14, 2024-08-19T01:43:45.291 INFO:teuthology.orchestra.run.smithi117.stdout: "inodes_expired": 388, 2024-08-19T01:43:45.292 INFO:teuthology.orchestra.run.smithi117.stdout: "inodes_with_caps": 1, 2024-08-19T01:43:45.292 INFO:teuthology.orchestra.run.smithi117.stdout: "caps": 1, 2024-08-19T01:43:45.292 INFO:teuthology.orchestra.run.smithi117.stdout: "subtrees": 3, 2024-08-19T01:43:45.292 INFO:teuthology.orchestra.run.smithi117.stdout: "traverse": 24333, 2024-08-19T01:43:45.292 INFO:teuthology.orchestra.run.smithi117.stdout: "traverse_hit": 19230, 2024-08-19T01:43:45.292 INFO:teuthology.orchestra.run.smithi117.stdout: "traverse_forward": 2, 2024-08-19T01:43:45.292 INFO:teuthology.orchestra.run.smithi117.stdout: "traverse_discover": 9, 2024-08-19T01:43:45.292 INFO:teuthology.orchestra.run.smithi117.stdout: "traverse_dir_fetch": 0, 2024-08-19T01:43:45.292 INFO:teuthology.orchestra.run.smithi117.stdout: "traverse_remote_ino": 0, 2024-08-19T01:43:45.292 INFO:teuthology.orchestra.run.smithi117.stdout: "traverse_lock": 205, 2024-08-19T01:43:45.292 INFO:teuthology.orchestra.run.smithi117.stdout: "load_cent": 0, 2024-08-19T01:43:45.292 INFO:teuthology.orchestra.run.smithi117.stdout: "q": 0, 2024-08-19T01:43:45.292 INFO:teuthology.orchestra.run.smithi117.stdout: "exported": 56, 2024-08-19T01:43:45.292 INFO:teuthology.orchestra.run.smithi117.stdout: "exported_inodes": 303, 2024-08-19T01:43:45.293 INFO:teuthology.orchestra.run.smithi117.stdout: "imported": 7, 2024-08-19T01:43:45.293 INFO:teuthology.orchestra.run.smithi117.stdout: "imported_inodes": 1004, 2024-08-19T01:43:45.293 INFO:teuthology.orchestra.run.smithi117.stdout: "openino_dir_fetch": 0, 2024-08-19T01:43:45.293 INFO:teuthology.orchestra.run.smithi117.stdout: "openino_backtrace_fetch": 3865, 2024-08-19T01:43:45.293 INFO:teuthology.orchestra.run.smithi117.stdout: "openino_peer_discover": 0, 2024-08-19T01:43:45.293 INFO:teuthology.orchestra.run.smithi117.stdout: "root_rfiles": 4, 2024-08-19T01:43:45.293 INFO:teuthology.orchestra.run.smithi117.stdout: "root_rbytes": 234, 2024-08-19T01:43:45.293 INFO:teuthology.orchestra.run.smithi117.stdout: "root_rsnaps": 0, 2024-08-19T01:43:45.293 INFO:teuthology.orchestra.run.smithi117.stdout: "scrub_backtrace_fetch": 3865, 2024-08-19T01:43:45.293 INFO:teuthology.orchestra.run.smithi117.stdout: "scrub_set_tag": 0, 2024-08-19T01:43:45.293 INFO:teuthology.orchestra.run.smithi117.stdout: "scrub_backtrace_repaired": 0, 2024-08-19T01:43:45.293 INFO:teuthology.orchestra.run.smithi117.stdout: "scrub_inotable_repaired": 0, 2024-08-19T01:43:45.293 INFO:teuthology.orchestra.run.smithi117.stdout: "scrub_dir_inodes": 0, 2024-08-19T01:43:45.294 INFO:teuthology.orchestra.run.smithi117.stdout: "scrub_dir_base_inodes": 0, 2024-08-19T01:43:45.294 INFO:teuthology.orchestra.run.smithi117.stdout: "scrub_dirfrag_rstats": 0, 2024-08-19T01:43:45.294 INFO:teuthology.orchestra.run.smithi117.stdout: "scrub_file_inodes": 3865, 2024-08-19T01:43:45.294 INFO:teuthology.orchestra.run.smithi117.stdout: "handle_inode_file_caps": 228, 2024-08-19T01:43:45.294 INFO:teuthology.orchestra.run.smithi117.stdout: "ceph_cap_op_revoke": 2542, 2024-08-19T01:43:45.294 INFO:teuthology.orchestra.run.smithi117.stdout: "ceph_cap_op_grant": 4813, 2024-08-19T01:43:45.294 INFO:teuthology.orchestra.run.smithi117.stdout: "ceph_cap_op_trunc": 0, 2024-08-19T01:43:45.294 INFO:teuthology.orchestra.run.smithi117.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-08-19T01:43:45.294 INFO:teuthology.orchestra.run.smithi117.stdout: "ceph_cap_op_flush_ack": 0, 2024-08-19T01:43:45.294 INFO:teuthology.orchestra.run.smithi117.stdout: "handle_client_caps": 5832, 2024-08-19T01:43:45.294 INFO:teuthology.orchestra.run.smithi117.stdout: "handle_client_caps_dirty": 2079, 2024-08-19T01:43:45.294 INFO:teuthology.orchestra.run.smithi117.stdout: "handle_client_cap_release": 329, 2024-08-19T01:43:45.294 INFO:teuthology.orchestra.run.smithi117.stdout: "process_request_cap_release": 8047 2024-08-19T01:43:45.294 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-08-19T01:43:45.295 INFO:teuthology.orchestra.run.smithi117.stdout: "mds_cache": { 2024-08-19T01:43:45.295 INFO:teuthology.orchestra.run.smithi117.stdout: "dir_update": 0, 2024-08-19T01:43:45.295 INFO:teuthology.orchestra.run.smithi117.stdout: "dir_update_receipt": 0, 2024-08-19T01:43:45.295 INFO:teuthology.orchestra.run.smithi117.stdout: "dir_try_discover": 0, 2024-08-19T01:43:45.295 INFO:teuthology.orchestra.run.smithi117.stdout: "dir_send_discover": 10, 2024-08-19T01:43:45.295 INFO:teuthology.orchestra.run.smithi117.stdout: "dir_handle_discover": 42, 2024-08-19T01:43:45.295 INFO:teuthology.orchestra.run.smithi117.stdout: "num_strays": 0, 2024-08-19T01:43:45.295 INFO:teuthology.orchestra.run.smithi117.stdout: "num_strays_delayed": 0, 2024-08-19T01:43:45.295 INFO:teuthology.orchestra.run.smithi117.stdout: "num_strays_enqueuing": 0, 2024-08-19T01:43:45.295 INFO:teuthology.orchestra.run.smithi117.stdout: "strays_created": 2078, 2024-08-19T01:43:45.295 INFO:teuthology.orchestra.run.smithi117.stdout: "strays_enqueued": 2078, 2024-08-19T01:43:45.295 INFO:teuthology.orchestra.run.smithi117.stdout: "strays_reintegrated": 0, 2024-08-19T01:43:45.296 INFO:teuthology.orchestra.run.smithi117.stdout: "strays_migrated": 0, 2024-08-19T01:43:45.296 INFO:teuthology.orchestra.run.smithi117.stdout: "num_recovering_processing": 0, 2024-08-19T01:43:45.296 INFO:teuthology.orchestra.run.smithi117.stdout: "num_recovering_enqueued": 0, 2024-08-19T01:43:45.296 INFO:teuthology.orchestra.run.smithi117.stdout: "num_recovering_prioritized": 0, 2024-08-19T01:43:45.296 INFO:teuthology.orchestra.run.smithi117.stdout: "recovery_started": 0, 2024-08-19T01:43:45.296 INFO:teuthology.orchestra.run.smithi117.stdout: "recovery_completed": 0, 2024-08-19T01:43:45.296 INFO:teuthology.orchestra.run.smithi117.stdout: "ireq_quiesce_path": 0, 2024-08-19T01:43:45.296 INFO:teuthology.orchestra.run.smithi117.stdout: "ireq_quiesce_inode": 0, 2024-08-19T01:43:45.296 INFO:teuthology.orchestra.run.smithi117.stdout: "ireq_enqueue_scrub": 0, 2024-08-19T01:43:45.296 INFO:teuthology.orchestra.run.smithi117.stdout: "ireq_exportdir": 56, 2024-08-19T01:43:45.296 INFO:teuthology.orchestra.run.smithi117.stdout: "ireq_flush": 0, 2024-08-19T01:43:45.296 INFO:teuthology.orchestra.run.smithi117.stdout: "ireq_fragmentdir": 7, 2024-08-19T01:43:45.296 INFO:teuthology.orchestra.run.smithi117.stdout: "ireq_fragstats": 0, 2024-08-19T01:43:45.296 INFO:teuthology.orchestra.run.smithi117.stdout: "ireq_inodestats": 0 2024-08-19T01:43:45.296 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-08-19T01:43:45.297 INFO:teuthology.orchestra.run.smithi117.stdout: "mds_log": { 2024-08-19T01:43:45.297 INFO:teuthology.orchestra.run.smithi117.stdout: "evlrg": 0, 2024-08-19T01:43:45.297 INFO:teuthology.orchestra.run.smithi117.stdout: "evadd": 11484, 2024-08-19T01:43:45.297 INFO:teuthology.orchestra.run.smithi117.stdout: "evex": 0, 2024-08-19T01:43:45.297 INFO:teuthology.orchestra.run.smithi117.stdout: "evtrm": 0, 2024-08-19T01:43:45.297 INFO:teuthology.orchestra.run.smithi117.stdout: "ev": 11484, 2024-08-19T01:43:45.297 INFO:teuthology.orchestra.run.smithi117.stdout: "evexg": 0, 2024-08-19T01:43:45.297 INFO:teuthology.orchestra.run.smithi117.stdout: "evexd": 0, 2024-08-19T01:43:45.297 INFO:teuthology.orchestra.run.smithi117.stdout: "segadd": 13, 2024-08-19T01:43:45.297 INFO:teuthology.orchestra.run.smithi117.stdout: "segex": 0, 2024-08-19T01:43:45.297 INFO:teuthology.orchestra.run.smithi117.stdout: "segtrm": 0, 2024-08-19T01:43:45.297 INFO:teuthology.orchestra.run.smithi117.stdout: "seg": 13, 2024-08-19T01:43:45.297 INFO:teuthology.orchestra.run.smithi117.stdout: "segmjr": 2, 2024-08-19T01:43:45.297 INFO:teuthology.orchestra.run.smithi117.stdout: "segexg": 0, 2024-08-19T01:43:45.297 INFO:teuthology.orchestra.run.smithi117.stdout: "segexd": 0, 2024-08-19T01:43:45.298 INFO:teuthology.orchestra.run.smithi117.stdout: "expos": 4194304, 2024-08-19T01:43:45.298 INFO:teuthology.orchestra.run.smithi117.stdout: "wrpos": 20838457, 2024-08-19T01:43:45.298 INFO:teuthology.orchestra.run.smithi117.stdout: "rdpos": 0, 2024-08-19T01:43:45.298 INFO:teuthology.orchestra.run.smithi117.stdout: "jlat": { 2024-08-19T01:43:45.298 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 498, 2024-08-19T01:43:45.298 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 12.935860248, 2024-08-19T01:43:45.298 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.025975622 2024-08-19T01:43:45.298 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-08-19T01:43:45.298 INFO:teuthology.orchestra.run.smithi117.stdout: "replayed": 0 2024-08-19T01:43:45.298 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-08-19T01:43:45.298 INFO:teuthology.orchestra.run.smithi117.stdout: "mds_mem": { 2024-08-19T01:43:45.298 INFO:teuthology.orchestra.run.smithi117.stdout: "ino": 19, 2024-08-19T01:43:45.298 INFO:teuthology.orchestra.run.smithi117.stdout: "ino+": 2132, 2024-08-19T01:43:45.299 INFO:teuthology.orchestra.run.smithi117.stdout: "ino-": 2113, 2024-08-19T01:43:45.299 INFO:teuthology.orchestra.run.smithi117.stdout: "dir": 18, 2024-08-19T01:43:45.299 INFO:teuthology.orchestra.run.smithi117.stdout: "dir+": 103, 2024-08-19T01:43:45.299 INFO:teuthology.orchestra.run.smithi117.stdout: "dir-": 85, 2024-08-19T01:43:45.299 INFO:teuthology.orchestra.run.smithi117.stdout: "dn": 15, 2024-08-19T01:43:45.299 INFO:teuthology.orchestra.run.smithi117.stdout: "dn+": 4597, 2024-08-19T01:43:45.299 INFO:teuthology.orchestra.run.smithi117.stdout: "dn-": 4582, 2024-08-19T01:43:45.299 INFO:teuthology.orchestra.run.smithi117.stdout: "cap": 1, 2024-08-19T01:43:45.299 INFO:teuthology.orchestra.run.smithi117.stdout: "cap+": 3132, 2024-08-19T01:43:45.299 INFO:teuthology.orchestra.run.smithi117.stdout: "cap-": 3131, 2024-08-19T01:43:45.299 INFO:teuthology.orchestra.run.smithi117.stdout: "rss": 70104, 2024-08-19T01:43:45.299 INFO:teuthology.orchestra.run.smithi117.stdout: "heap": 223520 2024-08-19T01:43:45.299 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-08-19T01:43:45.299 INFO:teuthology.orchestra.run.smithi117.stdout: "mds_server": { 2024-08-19T01:43:45.300 INFO:teuthology.orchestra.run.smithi117.stdout: "dispatch_client_request": 16503, 2024-08-19T01:43:45.300 INFO:teuthology.orchestra.run.smithi117.stdout: "dispatch_server_request": 874, 2024-08-19T01:43:45.300 INFO:teuthology.orchestra.run.smithi117.stdout: "handle_client_request": 11037, 2024-08-19T01:43:45.300 INFO:teuthology.orchestra.run.smithi117.stdout: "handle_client_session": 16, 2024-08-19T01:43:45.300 INFO:teuthology.orchestra.run.smithi117.stdout: "handle_peer_request": 6233, 2024-08-19T01:43:45.300 INFO:teuthology.orchestra.run.smithi117.stdout: "req_create_latency": { 2024-08-19T01:43:45.300 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 1583, 2024-08-19T01:43:45.300 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 11.574318585, 2024-08-19T01:43:45.300 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.007311635 2024-08-19T01:43:45.300 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-08-19T01:43:45.300 INFO:teuthology.orchestra.run.smithi117.stdout: "req_getattr_latency": { 2024-08-19T01:43:45.300 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 2, 2024-08-19T01:43:45.300 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000949544, 2024-08-19T01:43:45.300 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000474772 2024-08-19T01:43:45.301 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-08-19T01:43:45.301 INFO:teuthology.orchestra.run.smithi117.stdout: "req_getfilelock_latency": { 2024-08-19T01:43:45.301 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2024-08-19T01:43:45.301 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2024-08-19T01:43:45.301 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2024-08-19T01:43:45.301 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-08-19T01:43:45.301 INFO:teuthology.orchestra.run.smithi117.stdout: "req_link_latency": { 2024-08-19T01:43:45.301 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2024-08-19T01:43:45.301 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2024-08-19T01:43:45.301 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2024-08-19T01:43:45.301 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-08-19T01:43:45.301 INFO:teuthology.orchestra.run.smithi117.stdout: "req_lookup_latency": { 2024-08-19T01:43:45.301 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 5147, 2024-08-19T01:43:45.301 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 46.083237837, 2024-08-19T01:43:45.302 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.008953417 2024-08-19T01:43:45.302 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-08-19T01:43:45.302 INFO:teuthology.orchestra.run.smithi117.stdout: "req_lookuphash_latency": { 2024-08-19T01:43:45.302 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2024-08-19T01:43:45.302 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2024-08-19T01:43:45.302 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2024-08-19T01:43:45.302 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-08-19T01:43:45.302 INFO:teuthology.orchestra.run.smithi117.stdout: "req_lookupino_latency": { 2024-08-19T01:43:45.302 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2024-08-19T01:43:45.302 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2024-08-19T01:43:45.303 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2024-08-19T01:43:45.303 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-08-19T01:43:45.303 INFO:teuthology.orchestra.run.smithi117.stdout: "req_lookupname_latency": { 2024-08-19T01:43:45.303 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2024-08-19T01:43:45.303 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2024-08-19T01:43:45.303 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2024-08-19T01:43:45.303 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-08-19T01:43:45.303 INFO:teuthology.orchestra.run.smithi117.stdout: "req_lookupparent_latency": { 2024-08-19T01:43:45.303 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2024-08-19T01:43:45.303 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2024-08-19T01:43:45.303 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2024-08-19T01:43:45.303 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-08-19T01:43:45.303 INFO:teuthology.orchestra.run.smithi117.stdout: "req_lookupsnap_latency": { 2024-08-19T01:43:45.303 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2024-08-19T01:43:45.304 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2024-08-19T01:43:45.304 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2024-08-19T01:43:45.304 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-08-19T01:43:45.304 INFO:teuthology.orchestra.run.smithi117.stdout: "req_lssnap_latency": { 2024-08-19T01:43:45.304 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2024-08-19T01:43:45.304 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2024-08-19T01:43:45.304 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2024-08-19T01:43:45.304 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-08-19T01:43:45.304 INFO:teuthology.orchestra.run.smithi117.stdout: "req_mkdir_latency": { 2024-08-19T01:43:45.304 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2024-08-19T01:43:45.304 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2024-08-19T01:43:45.304 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2024-08-19T01:43:45.304 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-08-19T01:43:45.304 INFO:teuthology.orchestra.run.smithi117.stdout: "req_mknod_latency": { 2024-08-19T01:43:45.304 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2024-08-19T01:43:45.305 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2024-08-19T01:43:45.305 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2024-08-19T01:43:45.305 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-08-19T01:43:45.305 INFO:teuthology.orchestra.run.smithi117.stdout: "req_mksnap_latency": { 2024-08-19T01:43:45.305 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2024-08-19T01:43:45.305 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2024-08-19T01:43:45.305 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2024-08-19T01:43:45.305 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-08-19T01:43:45.305 INFO:teuthology.orchestra.run.smithi117.stdout: "req_open_latency": { 2024-08-19T01:43:45.305 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 1467, 2024-08-19T01:43:45.305 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.831127023, 2024-08-19T01:43:45.305 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000566548 2024-08-19T01:43:45.305 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-08-19T01:43:45.305 INFO:teuthology.orchestra.run.smithi117.stdout: "req_readdir_latency": { 2024-08-19T01:43:45.305 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 64, 2024-08-19T01:43:45.306 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.049212917, 2024-08-19T01:43:45.306 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000768951 2024-08-19T01:43:45.306 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-08-19T01:43:45.306 INFO:teuthology.orchestra.run.smithi117.stdout: "req_rename_latency": { 2024-08-19T01:43:45.306 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 1587, 2024-08-19T01:43:45.306 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 14.038567842, 2024-08-19T01:43:45.306 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.008845978 2024-08-19T01:43:45.306 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-08-19T01:43:45.306 INFO:teuthology.orchestra.run.smithi117.stdout: "req_renamesnap_latency": { 2024-08-19T01:43:45.306 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2024-08-19T01:43:45.306 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2024-08-19T01:43:45.306 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2024-08-19T01:43:45.306 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-08-19T01:43:45.307 INFO:teuthology.orchestra.run.smithi117.stdout: "req_snapdiff_latency": { 2024-08-19T01:43:45.307 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2024-08-19T01:43:45.307 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2024-08-19T01:43:45.307 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2024-08-19T01:43:45.307 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-08-19T01:43:45.307 INFO:teuthology.orchestra.run.smithi117.stdout: "req_rmdir_latency": { 2024-08-19T01:43:45.307 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2024-08-19T01:43:45.307 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2024-08-19T01:43:45.307 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2024-08-19T01:43:45.307 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-08-19T01:43:45.307 INFO:teuthology.orchestra.run.smithi117.stdout: "req_rmsnap_latency": { 2024-08-19T01:43:45.307 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2024-08-19T01:43:45.308 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2024-08-19T01:43:45.308 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2024-08-19T01:43:45.308 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-08-19T01:43:45.308 INFO:teuthology.orchestra.run.smithi117.stdout: "req_rmxattr_latency": { 2024-08-19T01:43:45.308 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2024-08-19T01:43:45.308 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2024-08-19T01:43:45.308 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2024-08-19T01:43:45.308 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-08-19T01:43:45.308 INFO:teuthology.orchestra.run.smithi117.stdout: "req_setattr_latency": { 2024-08-19T01:43:45.308 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2024-08-19T01:43:45.308 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2024-08-19T01:43:45.308 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2024-08-19T01:43:45.308 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-08-19T01:43:45.309 INFO:teuthology.orchestra.run.smithi117.stdout: "req_setdirlayout_latency": { 2024-08-19T01:43:45.309 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2024-08-19T01:43:45.309 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2024-08-19T01:43:45.309 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2024-08-19T01:43:45.309 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-08-19T01:43:45.309 INFO:teuthology.orchestra.run.smithi117.stdout: "req_setfilelock_latency": { 2024-08-19T01:43:45.309 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2024-08-19T01:43:45.309 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2024-08-19T01:43:45.309 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2024-08-19T01:43:45.309 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-08-19T01:43:45.309 INFO:teuthology.orchestra.run.smithi117.stdout: "req_setlayout_latency": { 2024-08-19T01:43:45.309 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2024-08-19T01:43:45.309 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2024-08-19T01:43:45.309 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2024-08-19T01:43:45.309 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-08-19T01:43:45.310 INFO:teuthology.orchestra.run.smithi117.stdout: "req_setxattr_latency": { 2024-08-19T01:43:45.310 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2024-08-19T01:43:45.310 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2024-08-19T01:43:45.310 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2024-08-19T01:43:45.310 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-08-19T01:43:45.310 INFO:teuthology.orchestra.run.smithi117.stdout: "req_symlink_latency": { 2024-08-19T01:43:45.310 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2024-08-19T01:43:45.310 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2024-08-19T01:43:45.310 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2024-08-19T01:43:45.310 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-08-19T01:43:45.310 INFO:teuthology.orchestra.run.smithi117.stdout: "req_unlink_latency": { 2024-08-19T01:43:45.310 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 1183, 2024-08-19T01:43:45.310 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 2.047056613, 2024-08-19T01:43:45.311 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.001730394 2024-08-19T01:43:45.311 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-08-19T01:43:45.311 INFO:teuthology.orchestra.run.smithi117.stdout: "cap_revoke_eviction": 0, 2024-08-19T01:43:45.311 INFO:teuthology.orchestra.run.smithi117.stdout: "cap_acquisition_throttle": 0, 2024-08-19T01:43:45.311 INFO:teuthology.orchestra.run.smithi117.stdout: "req_getvxattr_latency": { 2024-08-19T01:43:45.311 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2024-08-19T01:43:45.311 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2024-08-19T01:43:45.311 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2024-08-19T01:43:45.311 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-08-19T01:43:45.311 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-08-19T01:43:45.311 INFO:teuthology.orchestra.run.smithi117.stdout: "mds_sessions": { 2024-08-19T01:43:45.311 INFO:teuthology.orchestra.run.smithi117.stdout: "session_count": 1, 2024-08-19T01:43:45.311 INFO:teuthology.orchestra.run.smithi117.stdout: "session_add": 3, 2024-08-19T01:43:45.311 INFO:teuthology.orchestra.run.smithi117.stdout: "session_remove": 2, 2024-08-19T01:43:45.312 INFO:teuthology.orchestra.run.smithi117.stdout: "sessions_open": 1, 2024-08-19T01:43:45.312 INFO:teuthology.orchestra.run.smithi117.stdout: "sessions_stale": 0, 2024-08-19T01:43:45.312 INFO:teuthology.orchestra.run.smithi117.stdout: "total_load": 2455, 2024-08-19T01:43:45.312 INFO:teuthology.orchestra.run.smithi117.stdout: "average_load": 2455, 2024-08-19T01:43:45.312 INFO:teuthology.orchestra.run.smithi117.stdout: "avg_session_uptime": 542, 2024-08-19T01:43:45.312 INFO:teuthology.orchestra.run.smithi117.stdout: "mdthresh_evicted": 0 2024-08-19T01:43:45.312 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-08-19T01:43:45.312 INFO:teuthology.orchestra.run.smithi117.stdout: "mempool": { 2024-08-19T01:43:45.312 INFO:teuthology.orchestra.run.smithi117.stdout: "bloom_filter_bytes": 0, 2024-08-19T01:43:45.312 INFO:teuthology.orchestra.run.smithi117.stdout: "bloom_filter_items": 0, 2024-08-19T01:43:45.312 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_alloc_bytes": 0, 2024-08-19T01:43:45.312 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_alloc_items": 0, 2024-08-19T01:43:45.312 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_cache_data_bytes": 0, 2024-08-19T01:43:45.312 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_cache_data_items": 0, 2024-08-19T01:43:45.313 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_cache_onode_bytes": 0, 2024-08-19T01:43:45.313 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_cache_onode_items": 0, 2024-08-19T01:43:45.313 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_cache_meta_bytes": 0, 2024-08-19T01:43:45.313 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_cache_meta_items": 0, 2024-08-19T01:43:45.313 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_cache_other_bytes": 0, 2024-08-19T01:43:45.313 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_cache_other_items": 0, 2024-08-19T01:43:45.313 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_cache_buffer_bytes": 0, 2024-08-19T01:43:45.313 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_cache_buffer_items": 0, 2024-08-19T01:43:45.313 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_extent_bytes": 0, 2024-08-19T01:43:45.313 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_extent_items": 0, 2024-08-19T01:43:45.313 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_blob_bytes": 0, 2024-08-19T01:43:45.313 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_blob_items": 0, 2024-08-19T01:43:45.313 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_shared_blob_bytes": 0, 2024-08-19T01:43:45.313 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_shared_blob_items": 0, 2024-08-19T01:43:45.314 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_inline_bl_bytes": 0, 2024-08-19T01:43:45.314 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_inline_bl_items": 0, 2024-08-19T01:43:45.314 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_fsck_bytes": 0, 2024-08-19T01:43:45.314 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_fsck_items": 0, 2024-08-19T01:43:45.314 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_txc_bytes": 0, 2024-08-19T01:43:45.314 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_txc_items": 0, 2024-08-19T01:43:45.314 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_writing_deferred_bytes": 0, 2024-08-19T01:43:45.314 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_writing_deferred_items": 0, 2024-08-19T01:43:45.314 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_writing_bytes": 0, 2024-08-19T01:43:45.314 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_writing_items": 0, 2024-08-19T01:43:45.314 INFO:teuthology.orchestra.run.smithi117.stdout: "bluefs_bytes": 0, 2024-08-19T01:43:45.314 INFO:teuthology.orchestra.run.smithi117.stdout: "bluefs_items": 0, 2024-08-19T01:43:45.314 INFO:teuthology.orchestra.run.smithi117.stdout: "bluefs_file_reader_bytes": 0, 2024-08-19T01:43:45.314 INFO:teuthology.orchestra.run.smithi117.stdout: "bluefs_file_reader_items": 0, 2024-08-19T01:43:45.315 INFO:teuthology.orchestra.run.smithi117.stdout: "bluefs_file_writer_bytes": 0, 2024-08-19T01:43:45.315 INFO:teuthology.orchestra.run.smithi117.stdout: "bluefs_file_writer_items": 0, 2024-08-19T01:43:45.315 INFO:teuthology.orchestra.run.smithi117.stdout: "buffer_anon_bytes": 285944, 2024-08-19T01:43:45.315 INFO:teuthology.orchestra.run.smithi117.stdout: "buffer_anon_items": 120, 2024-08-19T01:43:45.315 INFO:teuthology.orchestra.run.smithi117.stdout: "buffer_meta_bytes": 880, 2024-08-19T01:43:45.315 INFO:teuthology.orchestra.run.smithi117.stdout: "buffer_meta_items": 10, 2024-08-19T01:43:45.315 INFO:teuthology.orchestra.run.smithi117.stdout: "osd_bytes": 0, 2024-08-19T01:43:45.315 INFO:teuthology.orchestra.run.smithi117.stdout: "osd_items": 0, 2024-08-19T01:43:45.315 INFO:teuthology.orchestra.run.smithi117.stdout: "osd_mapbl_bytes": 0, 2024-08-19T01:43:45.315 INFO:teuthology.orchestra.run.smithi117.stdout: "osd_mapbl_items": 0, 2024-08-19T01:43:45.315 INFO:teuthology.orchestra.run.smithi117.stdout: "osd_pglog_bytes": 0, 2024-08-19T01:43:45.315 INFO:teuthology.orchestra.run.smithi117.stdout: "osd_pglog_items": 0, 2024-08-19T01:43:45.315 INFO:teuthology.orchestra.run.smithi117.stdout: "osdmap_bytes": 7688, 2024-08-19T01:43:45.315 INFO:teuthology.orchestra.run.smithi117.stdout: "osdmap_items": 158, 2024-08-19T01:43:45.315 INFO:teuthology.orchestra.run.smithi117.stdout: "osdmap_mapping_bytes": 0, 2024-08-19T01:43:45.316 INFO:teuthology.orchestra.run.smithi117.stdout: "osdmap_mapping_items": 0, 2024-08-19T01:43:45.316 INFO:teuthology.orchestra.run.smithi117.stdout: "pgmap_bytes": 0, 2024-08-19T01:43:45.316 INFO:teuthology.orchestra.run.smithi117.stdout: "pgmap_items": 0, 2024-08-19T01:43:45.316 INFO:teuthology.orchestra.run.smithi117.stdout: "mds_co_bytes": 79061, 2024-08-19T01:43:45.316 INFO:teuthology.orchestra.run.smithi117.stdout: "mds_co_items": 440, 2024-08-19T01:43:45.316 INFO:teuthology.orchestra.run.smithi117.stdout: "unittest_1_bytes": 0, 2024-08-19T01:43:45.316 INFO:teuthology.orchestra.run.smithi117.stdout: "unittest_1_items": 0, 2024-08-19T01:43:45.316 INFO:teuthology.orchestra.run.smithi117.stdout: "unittest_2_bytes": 0, 2024-08-19T01:43:45.316 INFO:teuthology.orchestra.run.smithi117.stdout: "unittest_2_items": 0 2024-08-19T01:43:45.316 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-08-19T01:43:45.316 INFO:teuthology.orchestra.run.smithi117.stdout: "objecter": { 2024-08-19T01:43:45.316 INFO:teuthology.orchestra.run.smithi117.stdout: "op_active": 0, 2024-08-19T01:43:45.316 INFO:teuthology.orchestra.run.smithi117.stdout: "op_laggy": 0, 2024-08-19T01:43:45.316 INFO:teuthology.orchestra.run.smithi117.stdout: "op_send": 7116, 2024-08-19T01:43:45.316 INFO:teuthology.orchestra.run.smithi117.stdout: "op_send_bytes": 18045379, 2024-08-19T01:43:45.317 INFO:teuthology.orchestra.run.smithi117.stdout: "op_resend": 29, 2024-08-19T01:43:45.317 INFO:teuthology.orchestra.run.smithi117.stdout: "op_reply": 7116, 2024-08-19T01:43:45.317 INFO:teuthology.orchestra.run.smithi117.stdout: "op_latency": { 2024-08-19T01:43:45.317 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 7116, 2024-08-19T01:43:45.317 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 92.974157497, 2024-08-19T01:43:45.317 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.013065508 2024-08-19T01:43:45.317 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-08-19T01:43:45.317 INFO:teuthology.orchestra.run.smithi117.stdout: "op_inflight": 0, 2024-08-19T01:43:45.317 INFO:teuthology.orchestra.run.smithi117.stdout: "oplen_avg": { 2024-08-19T01:43:45.317 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 7116, 2024-08-19T01:43:45.317 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 7473 2024-08-19T01:43:45.317 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-08-19T01:43:45.317 INFO:teuthology.orchestra.run.smithi117.stdout: "op": 7116, 2024-08-19T01:43:45.318 INFO:teuthology.orchestra.run.smithi117.stdout: "op_r": 4000, 2024-08-19T01:43:45.318 INFO:teuthology.orchestra.run.smithi117.stdout: "op_w": 3116, 2024-08-19T01:43:45.318 INFO:teuthology.orchestra.run.smithi117.stdout: "op_rmw": 0, 2024-08-19T01:43:45.318 INFO:teuthology.orchestra.run.smithi117.stdout: "op_pg": 0, 2024-08-19T01:43:45.318 INFO:teuthology.orchestra.run.smithi117.stdout: "osdop_stat": 0, 2024-08-19T01:43:45.318 INFO:teuthology.orchestra.run.smithi117.stdout: "osdop_create": 10, 2024-08-19T01:43:45.318 INFO:teuthology.orchestra.run.smithi117.stdout: "osdop_read": 122, 2024-08-19T01:43:45.318 INFO:teuthology.orchestra.run.smithi117.stdout: "osdop_write": 620, 2024-08-19T01:43:45.318 INFO:teuthology.orchestra.run.smithi117.stdout: "osdop_writefull": 98, 2024-08-19T01:43:45.318 INFO:teuthology.orchestra.run.smithi117.stdout: "osdop_writesame": 0, 2024-08-19T01:43:45.318 INFO:teuthology.orchestra.run.smithi117.stdout: "osdop_append": 0, 2024-08-19T01:43:45.318 INFO:teuthology.orchestra.run.smithi117.stdout: "osdop_zero": 0, 2024-08-19T01:43:45.318 INFO:teuthology.orchestra.run.smithi117.stdout: "osdop_truncate": 7, 2024-08-19T01:43:45.319 INFO:teuthology.orchestra.run.smithi117.stdout: "osdop_delete": 2093, 2024-08-19T01:43:45.319 INFO:teuthology.orchestra.run.smithi117.stdout: "osdop_mapext": 0, 2024-08-19T01:43:45.319 INFO:teuthology.orchestra.run.smithi117.stdout: "osdop_sparse_read": 0, 2024-08-19T01:43:45.319 INFO:teuthology.orchestra.run.smithi117.stdout: "osdop_clonerange": 0, 2024-08-19T01:43:45.319 INFO:teuthology.orchestra.run.smithi117.stdout: "osdop_getxattr": 3865, 2024-08-19T01:43:45.319 INFO:teuthology.orchestra.run.smithi117.stdout: "osdop_setxattr": 20, 2024-08-19T01:43:45.319 INFO:teuthology.orchestra.run.smithi117.stdout: "osdop_cmpxattr": 0, 2024-08-19T01:43:45.319 INFO:teuthology.orchestra.run.smithi117.stdout: "osdop_rmxattr": 0, 2024-08-19T01:43:45.319 INFO:teuthology.orchestra.run.smithi117.stdout: "osdop_resetxattrs": 0, 2024-08-19T01:43:45.319 INFO:teuthology.orchestra.run.smithi117.stdout: "osdop_call": 0, 2024-08-19T01:43:45.319 INFO:teuthology.orchestra.run.smithi117.stdout: "osdop_watch": 0, 2024-08-19T01:43:45.319 INFO:teuthology.orchestra.run.smithi117.stdout: "osdop_notify": 0, 2024-08-19T01:43:45.319 INFO:teuthology.orchestra.run.smithi117.stdout: "osdop_src_cmpxattr": 0, 2024-08-19T01:43:45.320 INFO:teuthology.orchestra.run.smithi117.stdout: "osdop_pgls": 0, 2024-08-19T01:43:45.320 INFO:teuthology.orchestra.run.smithi117.stdout: "osdop_pgls_filter": 0, 2024-08-19T01:43:45.320 INFO:teuthology.orchestra.run.smithi117.stdout: "osdop_other": 0, 2024-08-19T01:43:45.320 INFO:teuthology.orchestra.run.smithi117.stdout: "linger_active": 0, 2024-08-19T01:43:45.320 INFO:teuthology.orchestra.run.smithi117.stdout: "linger_send": 0, 2024-08-19T01:43:45.320 INFO:teuthology.orchestra.run.smithi117.stdout: "linger_resend": 0, 2024-08-19T01:43:45.320 INFO:teuthology.orchestra.run.smithi117.stdout: "linger_ping": 0, 2024-08-19T01:43:45.320 INFO:teuthology.orchestra.run.smithi117.stdout: "poolop_active": 0, 2024-08-19T01:43:45.320 INFO:teuthology.orchestra.run.smithi117.stdout: "poolop_send": 0, 2024-08-19T01:43:45.320 INFO:teuthology.orchestra.run.smithi117.stdout: "poolop_resend": 0, 2024-08-19T01:43:45.320 INFO:teuthology.orchestra.run.smithi117.stdout: "poolstat_active": 0, 2024-08-19T01:43:45.320 INFO:teuthology.orchestra.run.smithi117.stdout: "poolstat_send": 0, 2024-08-19T01:43:45.320 INFO:teuthology.orchestra.run.smithi117.stdout: "poolstat_resend": 0, 2024-08-19T01:43:45.320 INFO:teuthology.orchestra.run.smithi117.stdout: "statfs_active": 0, 2024-08-19T01:43:45.321 INFO:teuthology.orchestra.run.smithi117.stdout: "statfs_send": 0, 2024-08-19T01:43:45.321 INFO:teuthology.orchestra.run.smithi117.stdout: "statfs_resend": 0, 2024-08-19T01:43:45.321 INFO:teuthology.orchestra.run.smithi117.stdout: "command_active": 0, 2024-08-19T01:43:45.321 INFO:teuthology.orchestra.run.smithi117.stdout: "command_send": 0, 2024-08-19T01:43:45.321 INFO:teuthology.orchestra.run.smithi117.stdout: "command_resend": 0, 2024-08-19T01:43:45.321 INFO:teuthology.orchestra.run.smithi117.stdout: "map_epoch": 86, 2024-08-19T01:43:45.321 INFO:teuthology.orchestra.run.smithi117.stdout: "map_full": 0, 2024-08-19T01:43:45.321 INFO:teuthology.orchestra.run.smithi117.stdout: "map_inc": 2, 2024-08-19T01:43:45.321 INFO:teuthology.orchestra.run.smithi117.stdout: "osd_sessions": 12, 2024-08-19T01:43:45.321 INFO:teuthology.orchestra.run.smithi117.stdout: "osd_session_open": 12, 2024-08-19T01:43:45.321 INFO:teuthology.orchestra.run.smithi117.stdout: "osd_session_close": 0, 2024-08-19T01:43:45.321 INFO:teuthology.orchestra.run.smithi117.stdout: "osd_laggy": 0, 2024-08-19T01:43:45.321 INFO:teuthology.orchestra.run.smithi117.stdout: "omap_wr": 522, 2024-08-19T01:43:45.321 INFO:teuthology.orchestra.run.smithi117.stdout: "omap_rd": 18, 2024-08-19T01:43:45.321 INFO:teuthology.orchestra.run.smithi117.stdout: "omap_del": 98 2024-08-19T01:43:45.321 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-08-19T01:43:45.322 INFO:teuthology.orchestra.run.smithi117.stdout: "oft": { 2024-08-19T01:43:45.322 INFO:teuthology.orchestra.run.smithi117.stdout: "omap_total_objs": 1, 2024-08-19T01:43:45.322 INFO:teuthology.orchestra.run.smithi117.stdout: "omap_total_kv_pairs": 0, 2024-08-19T01:43:45.322 INFO:teuthology.orchestra.run.smithi117.stdout: "omap_total_updates": 0, 2024-08-19T01:43:45.322 INFO:teuthology.orchestra.run.smithi117.stdout: "omap_total_removes": 0 2024-08-19T01:43:45.322 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-08-19T01:43:45.322 INFO:teuthology.orchestra.run.smithi117.stdout: "osd-slow-ops": { 2024-08-19T01:43:45.322 INFO:teuthology.orchestra.run.smithi117.stdout: "slow_ops_count": 0 2024-08-19T01:43:45.322 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-08-19T01:43:45.322 INFO:teuthology.orchestra.run.smithi117.stdout: "purge_queue": { 2024-08-19T01:43:45.322 INFO:teuthology.orchestra.run.smithi117.stdout: "pq_executing_ops": 0, 2024-08-19T01:43:45.322 INFO:teuthology.orchestra.run.smithi117.stdout: "pq_executing_ops_high_water": 10, 2024-08-19T01:43:45.322 INFO:teuthology.orchestra.run.smithi117.stdout: "pq_executing": 0, 2024-08-19T01:43:45.323 INFO:teuthology.orchestra.run.smithi117.stdout: "pq_executing_high_water": 10, 2024-08-19T01:43:45.323 INFO:teuthology.orchestra.run.smithi117.stdout: "pq_executed_ops": 2078, 2024-08-19T01:43:45.323 INFO:teuthology.orchestra.run.smithi117.stdout: "pq_executed": 2078, 2024-08-19T01:43:45.323 INFO:teuthology.orchestra.run.smithi117.stdout: "pq_item_in_journal": 0 2024-08-19T01:43:45.323 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-08-19T01:43:45.323 INFO:teuthology.orchestra.run.smithi117.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-08-19T01:43:45.323 INFO:teuthology.orchestra.run.smithi117.stdout: "val": 0, 2024-08-19T01:43:45.323 INFO:teuthology.orchestra.run.smithi117.stdout: "max": 104857600, 2024-08-19T01:43:45.323 INFO:teuthology.orchestra.run.smithi117.stdout: "get_started": 0, 2024-08-19T01:43:45.323 INFO:teuthology.orchestra.run.smithi117.stdout: "get": 46451, 2024-08-19T01:43:45.323 INFO:teuthology.orchestra.run.smithi117.stdout: "get_sum": 12317511, 2024-08-19T01:43:45.323 INFO:teuthology.orchestra.run.smithi117.stdout: "get_or_fail_fail": 0, 2024-08-19T01:43:45.323 INFO:teuthology.orchestra.run.smithi117.stdout: "get_or_fail_success": 46451, 2024-08-19T01:43:45.323 INFO:teuthology.orchestra.run.smithi117.stdout: "take": 0, 2024-08-19T01:43:45.323 INFO:teuthology.orchestra.run.smithi117.stdout: "take_sum": 0, 2024-08-19T01:43:45.324 INFO:teuthology.orchestra.run.smithi117.stdout: "put": 46451, 2024-08-19T01:43:45.324 INFO:teuthology.orchestra.run.smithi117.stdout: "put_sum": 12317511, 2024-08-19T01:43:45.324 INFO:teuthology.orchestra.run.smithi117.stdout: "wait": { 2024-08-19T01:43:45.324 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2024-08-19T01:43:45.324 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2024-08-19T01:43:45.324 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2024-08-19T01:43:45.324 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-08-19T01:43:45.324 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-08-19T01:43:45.324 INFO:teuthology.orchestra.run.smithi117.stdout: "throttle-objecter_bytes": { 2024-08-19T01:43:45.324 INFO:teuthology.orchestra.run.smithi117.stdout: "val": 0, 2024-08-19T01:43:45.324 INFO:teuthology.orchestra.run.smithi117.stdout: "max": 104857600, 2024-08-19T01:43:45.324 INFO:teuthology.orchestra.run.smithi117.stdout: "get_started": 0, 2024-08-19T01:43:45.324 INFO:teuthology.orchestra.run.smithi117.stdout: "get": 0, 2024-08-19T01:43:45.324 INFO:teuthology.orchestra.run.smithi117.stdout: "get_sum": 0, 2024-08-19T01:43:45.324 INFO:teuthology.orchestra.run.smithi117.stdout: "get_or_fail_fail": 0, 2024-08-19T01:43:45.325 INFO:teuthology.orchestra.run.smithi117.stdout: "get_or_fail_success": 0, 2024-08-19T01:43:45.325 INFO:teuthology.orchestra.run.smithi117.stdout: "take": 7116, 2024-08-19T01:43:45.325 INFO:teuthology.orchestra.run.smithi117.stdout: "take_sum": 18254898, 2024-08-19T01:43:45.325 INFO:teuthology.orchestra.run.smithi117.stdout: "put": 7116, 2024-08-19T01:43:45.325 INFO:teuthology.orchestra.run.smithi117.stdout: "put_sum": 18254898, 2024-08-19T01:43:45.325 INFO:teuthology.orchestra.run.smithi117.stdout: "wait": { 2024-08-19T01:43:45.325 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2024-08-19T01:43:45.325 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2024-08-19T01:43:45.325 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2024-08-19T01:43:45.325 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-08-19T01:43:45.325 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-08-19T01:43:45.325 INFO:teuthology.orchestra.run.smithi117.stdout: "throttle-objecter_ops": { 2024-08-19T01:43:45.325 INFO:teuthology.orchestra.run.smithi117.stdout: "val": 0, 2024-08-19T01:43:45.326 INFO:teuthology.orchestra.run.smithi117.stdout: "max": 1024, 2024-08-19T01:43:45.326 INFO:teuthology.orchestra.run.smithi117.stdout: "get_started": 0, 2024-08-19T01:43:45.326 INFO:teuthology.orchestra.run.smithi117.stdout: "get": 0, 2024-08-19T01:43:45.326 INFO:teuthology.orchestra.run.smithi117.stdout: "get_sum": 0, 2024-08-19T01:43:45.326 INFO:teuthology.orchestra.run.smithi117.stdout: "get_or_fail_fail": 0, 2024-08-19T01:43:45.326 INFO:teuthology.orchestra.run.smithi117.stdout: "get_or_fail_success": 0, 2024-08-19T01:43:45.326 INFO:teuthology.orchestra.run.smithi117.stdout: "take": 7116, 2024-08-19T01:43:45.326 INFO:teuthology.orchestra.run.smithi117.stdout: "take_sum": 7116, 2024-08-19T01:43:45.326 INFO:teuthology.orchestra.run.smithi117.stdout: "put": 7116, 2024-08-19T01:43:45.326 INFO:teuthology.orchestra.run.smithi117.stdout: "put_sum": 7116, 2024-08-19T01:43:45.326 INFO:teuthology.orchestra.run.smithi117.stdout: "wait": { 2024-08-19T01:43:45.326 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2024-08-19T01:43:45.326 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2024-08-19T01:43:45.327 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2024-08-19T01:43:45.327 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-08-19T01:43:45.327 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-08-19T01:43:45.327 INFO:teuthology.orchestra.run.smithi117.stdout: "throttle-write_buf_throttle": { 2024-08-19T01:43:45.327 INFO:teuthology.orchestra.run.smithi117.stdout: "val": 0, 2024-08-19T01:43:45.327 INFO:teuthology.orchestra.run.smithi117.stdout: "max": 3758096384, 2024-08-19T01:43:45.327 INFO:teuthology.orchestra.run.smithi117.stdout: "get_started": 0, 2024-08-19T01:43:45.327 INFO:teuthology.orchestra.run.smithi117.stdout: "get": 2078, 2024-08-19T01:43:45.327 INFO:teuthology.orchestra.run.smithi117.stdout: "get_sum": 209878, 2024-08-19T01:43:45.327 INFO:teuthology.orchestra.run.smithi117.stdout: "get_or_fail_fail": 0, 2024-08-19T01:43:45.327 INFO:teuthology.orchestra.run.smithi117.stdout: "get_or_fail_success": 2078, 2024-08-19T01:43:45.327 INFO:teuthology.orchestra.run.smithi117.stdout: "take": 0, 2024-08-19T01:43:45.327 INFO:teuthology.orchestra.run.smithi117.stdout: "take_sum": 0, 2024-08-19T01:43:45.328 INFO:teuthology.orchestra.run.smithi117.stdout: "put": 122, 2024-08-19T01:43:45.328 INFO:teuthology.orchestra.run.smithi117.stdout: "put_sum": 209878, 2024-08-19T01:43:45.328 INFO:teuthology.orchestra.run.smithi117.stdout: "wait": { 2024-08-19T01:43:45.328 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2024-08-19T01:43:45.328 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2024-08-19T01:43:45.328 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2024-08-19T01:43:45.328 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-08-19T01:43:45.328 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-08-19T01:43:45.328 INFO:teuthology.orchestra.run.smithi117.stdout: "throttle-write_buf_throttle-0x564ffeba2960": { 2024-08-19T01:43:45.328 INFO:teuthology.orchestra.run.smithi117.stdout: "val": 0, 2024-08-19T01:43:45.328 INFO:teuthology.orchestra.run.smithi117.stdout: "max": 3758096384, 2024-08-19T01:43:45.328 INFO:teuthology.orchestra.run.smithi117.stdout: "get_started": 0, 2024-08-19T01:43:45.328 INFO:teuthology.orchestra.run.smithi117.stdout: "get": 11484, 2024-08-19T01:43:45.328 INFO:teuthology.orchestra.run.smithi117.stdout: "get_sum": 16644153, 2024-08-19T01:43:45.329 INFO:teuthology.orchestra.run.smithi117.stdout: "get_or_fail_fail": 0, 2024-08-19T01:43:45.329 INFO:teuthology.orchestra.run.smithi117.stdout: "get_or_fail_success": 11484, 2024-08-19T01:43:45.329 INFO:teuthology.orchestra.run.smithi117.stdout: "take": 0, 2024-08-19T01:43:45.329 INFO:teuthology.orchestra.run.smithi117.stdout: "take_sum": 0, 2024-08-19T01:43:45.329 INFO:teuthology.orchestra.run.smithi117.stdout: "put": 498, 2024-08-19T01:43:45.329 INFO:teuthology.orchestra.run.smithi117.stdout: "put_sum": 16644153, 2024-08-19T01:43:45.329 INFO:teuthology.orchestra.run.smithi117.stdout: "wait": { 2024-08-19T01:43:45.329 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2024-08-19T01:43:45.329 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2024-08-19T01:43:45.329 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2024-08-19T01:43:45.329 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-08-19T01:43:45.329 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-08-19T01:43:45.329 INFO:teuthology.orchestra.run.smithi117.stdout:} 2024-08-19T01:43:45.904 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:43:45 smithi083 ceph-mon[17120]: pgmap v606: 97 pgs: 97 active+clean; 35 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-08-19T01:43:45.952 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:43:45 smithi117 ceph-mon[22016]: pgmap v606: 97 pgs: 97 active+clean; 35 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-08-19T01:43:45.965 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:43:45 smithi167 ceph-mon[21875]: pgmap v606: 97 pgs: 97 active+clean; 35 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-08-19T01:43:46.034 INFO:tasks.check_counter:Daemon mds.h mds.exported=56 2024-08-19T01:43:46.034 INFO:tasks.check_counter:Daemon mds.h mds.imported=7 2024-08-19T01:43:46.034 INFO:tasks.check_counter:Daemon mds.h mds.dir_split=7 2024-08-19T01:43:46.035 DEBUG:tasks.check_counter:Getting stats from k 2024-08-19T01:43:46.035 DEBUG:teuthology.orchestra.run.smithi117:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833 shell --fsid bad8780e-5dc9-11ef-bccf-c7b262605968 -- ceph daemon mds.k perf dump 2024-08-19T01:43:46.289 INFO:teuthology.orchestra.run.smithi117.stderr:Inferring config /var/lib/ceph/bad8780e-5dc9-11ef-bccf-c7b262605968/mon.b/config 2024-08-19T01:43:46.783 INFO:teuthology.orchestra.run.smithi117.stdout:{ 2024-08-19T01:43:46.784 INFO:teuthology.orchestra.run.smithi117.stdout: "AsyncMessenger::Worker-0": { 2024-08-19T01:43:46.784 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_recv_messages": 0, 2024-08-19T01:43:46.784 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_send_messages": 2, 2024-08-19T01:43:46.784 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_recv_bytes": 0, 2024-08-19T01:43:46.784 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_send_bytes": 0, 2024-08-19T01:43:46.784 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_created_connections": 4, 2024-08-19T01:43:46.784 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_active_connections": 0, 2024-08-19T01:43:46.784 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_running_total_time": 0.001054273, 2024-08-19T01:43:46.784 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_running_send_time": 0.000000000, 2024-08-19T01:43:46.784 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_running_recv_time": 0.000198458, 2024-08-19T01:43:46.784 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-08-19T01:43:46.784 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_send_messages_queue_lat": { 2024-08-19T01:43:46.784 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2024-08-19T01:43:46.784 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2024-08-19T01:43:46.784 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2024-08-19T01:43:46.785 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-08-19T01:43:46.785 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_handle_ack_lat": { 2024-08-19T01:43:46.785 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2024-08-19T01:43:46.785 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2024-08-19T01:43:46.785 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2024-08-19T01:43:46.785 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-08-19T01:43:46.785 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_recv_encrypted_bytes": 0, 2024-08-19T01:43:46.785 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_send_encrypted_bytes": 0 2024-08-19T01:43:46.785 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-08-19T01:43:46.785 INFO:teuthology.orchestra.run.smithi117.stdout: "AsyncMessenger::Worker-1": { 2024-08-19T01:43:46.785 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_recv_messages": 196, 2024-08-19T01:43:46.785 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_send_messages": 191, 2024-08-19T01:43:46.785 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_recv_bytes": 746768, 2024-08-19T01:43:46.786 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_send_bytes": 117328, 2024-08-19T01:43:46.786 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_created_connections": 2, 2024-08-19T01:43:46.786 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_active_connections": 1, 2024-08-19T01:43:46.786 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_running_total_time": 0.027895128, 2024-08-19T01:43:46.786 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_running_send_time": 0.010894388, 2024-08-19T01:43:46.786 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_running_recv_time": 0.023367209, 2024-08-19T01:43:46.786 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-08-19T01:43:46.786 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_send_messages_queue_lat": { 2024-08-19T01:43:46.786 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 190, 2024-08-19T01:43:46.786 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.009671067, 2024-08-19T01:43:46.786 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000050900 2024-08-19T01:43:46.786 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-08-19T01:43:46.787 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_handle_ack_lat": { 2024-08-19T01:43:46.787 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2024-08-19T01:43:46.787 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2024-08-19T01:43:46.787 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2024-08-19T01:43:46.787 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-08-19T01:43:46.787 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_recv_encrypted_bytes": 746768, 2024-08-19T01:43:46.787 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_send_encrypted_bytes": 117328 2024-08-19T01:43:46.787 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-08-19T01:43:46.787 INFO:teuthology.orchestra.run.smithi117.stdout: "AsyncMessenger::Worker-2": { 2024-08-19T01:43:46.787 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_recv_messages": 5, 2024-08-19T01:43:46.788 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_send_messages": 147, 2024-08-19T01:43:46.788 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_recv_bytes": 3184, 2024-08-19T01:43:46.788 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_send_bytes": 104192, 2024-08-19T01:43:46.788 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_created_connections": 3, 2024-08-19T01:43:46.788 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_active_connections": 2, 2024-08-19T01:43:46.788 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_running_total_time": 0.009963540, 2024-08-19T01:43:46.788 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_running_send_time": 0.005953123, 2024-08-19T01:43:46.788 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_running_recv_time": 0.002679337, 2024-08-19T01:43:46.788 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-08-19T01:43:46.788 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_send_messages_queue_lat": { 2024-08-19T01:43:46.788 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 146, 2024-08-19T01:43:46.788 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.010482210, 2024-08-19T01:43:46.789 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000071795 2024-08-19T01:43:46.789 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-08-19T01:43:46.789 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_handle_ack_lat": { 2024-08-19T01:43:46.789 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2024-08-19T01:43:46.789 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2024-08-19T01:43:46.789 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2024-08-19T01:43:46.789 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-08-19T01:43:46.789 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_recv_encrypted_bytes": 3184, 2024-08-19T01:43:46.789 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_send_encrypted_bytes": 104192 2024-08-19T01:43:46.789 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-08-19T01:43:46.789 INFO:teuthology.orchestra.run.smithi117.stdout: "cct": { 2024-08-19T01:43:46.789 INFO:teuthology.orchestra.run.smithi117.stdout: "total_workers": 0, 2024-08-19T01:43:46.789 INFO:teuthology.orchestra.run.smithi117.stdout: "unhealthy_workers": 0 2024-08-19T01:43:46.789 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-08-19T01:43:46.790 INFO:teuthology.orchestra.run.smithi117.stdout: "mempool": { 2024-08-19T01:43:46.790 INFO:teuthology.orchestra.run.smithi117.stdout: "bloom_filter_bytes": 0, 2024-08-19T01:43:46.790 INFO:teuthology.orchestra.run.smithi117.stdout: "bloom_filter_items": 0, 2024-08-19T01:43:46.790 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_alloc_bytes": 0, 2024-08-19T01:43:46.790 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_alloc_items": 0, 2024-08-19T01:43:46.790 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_cache_data_bytes": 0, 2024-08-19T01:43:46.790 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_cache_data_items": 0, 2024-08-19T01:43:46.790 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_cache_onode_bytes": 0, 2024-08-19T01:43:46.790 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_cache_onode_items": 0, 2024-08-19T01:43:46.790 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_cache_meta_bytes": 0, 2024-08-19T01:43:46.790 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_cache_meta_items": 0, 2024-08-19T01:43:46.790 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_cache_other_bytes": 0, 2024-08-19T01:43:46.790 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_cache_other_items": 0, 2024-08-19T01:43:46.790 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_cache_buffer_bytes": 0, 2024-08-19T01:43:46.791 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_cache_buffer_items": 0, 2024-08-19T01:43:46.791 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_extent_bytes": 0, 2024-08-19T01:43:46.791 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_extent_items": 0, 2024-08-19T01:43:46.791 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_blob_bytes": 0, 2024-08-19T01:43:46.791 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_blob_items": 0, 2024-08-19T01:43:46.791 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_shared_blob_bytes": 0, 2024-08-19T01:43:46.791 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_shared_blob_items": 0, 2024-08-19T01:43:46.791 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_inline_bl_bytes": 0, 2024-08-19T01:43:46.791 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_inline_bl_items": 0, 2024-08-19T01:43:46.791 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_fsck_bytes": 0, 2024-08-19T01:43:46.791 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_fsck_items": 0, 2024-08-19T01:43:46.791 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_txc_bytes": 0, 2024-08-19T01:43:46.791 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_txc_items": 0, 2024-08-19T01:43:46.791 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_writing_deferred_bytes": 0, 2024-08-19T01:43:46.791 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_writing_deferred_items": 0, 2024-08-19T01:43:46.791 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_writing_bytes": 0, 2024-08-19T01:43:46.792 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_writing_items": 0, 2024-08-19T01:43:46.792 INFO:teuthology.orchestra.run.smithi117.stdout: "bluefs_bytes": 0, 2024-08-19T01:43:46.792 INFO:teuthology.orchestra.run.smithi117.stdout: "bluefs_items": 0, 2024-08-19T01:43:46.792 INFO:teuthology.orchestra.run.smithi117.stdout: "bluefs_file_reader_bytes": 0, 2024-08-19T01:43:46.792 INFO:teuthology.orchestra.run.smithi117.stdout: "bluefs_file_reader_items": 0, 2024-08-19T01:43:46.792 INFO:teuthology.orchestra.run.smithi117.stdout: "bluefs_file_writer_bytes": 0, 2024-08-19T01:43:46.792 INFO:teuthology.orchestra.run.smithi117.stdout: "bluefs_file_writer_items": 0, 2024-08-19T01:43:46.792 INFO:teuthology.orchestra.run.smithi117.stdout: "buffer_anon_bytes": 149007, 2024-08-19T01:43:46.792 INFO:teuthology.orchestra.run.smithi117.stdout: "buffer_anon_items": 30, 2024-08-19T01:43:46.792 INFO:teuthology.orchestra.run.smithi117.stdout: "buffer_meta_bytes": 0, 2024-08-19T01:43:46.792 INFO:teuthology.orchestra.run.smithi117.stdout: "buffer_meta_items": 0, 2024-08-19T01:43:46.792 INFO:teuthology.orchestra.run.smithi117.stdout: "osd_bytes": 0, 2024-08-19T01:43:46.792 INFO:teuthology.orchestra.run.smithi117.stdout: "osd_items": 0, 2024-08-19T01:43:46.792 INFO:teuthology.orchestra.run.smithi117.stdout: "osd_mapbl_bytes": 0, 2024-08-19T01:43:46.792 INFO:teuthology.orchestra.run.smithi117.stdout: "osd_mapbl_items": 0, 2024-08-19T01:43:46.792 INFO:teuthology.orchestra.run.smithi117.stdout: "osd_pglog_bytes": 0, 2024-08-19T01:43:46.793 INFO:teuthology.orchestra.run.smithi117.stdout: "osd_pglog_items": 0, 2024-08-19T01:43:46.793 INFO:teuthology.orchestra.run.smithi117.stdout: "osdmap_bytes": 0, 2024-08-19T01:43:46.793 INFO:teuthology.orchestra.run.smithi117.stdout: "osdmap_items": 0, 2024-08-19T01:43:46.793 INFO:teuthology.orchestra.run.smithi117.stdout: "osdmap_mapping_bytes": 0, 2024-08-19T01:43:46.793 INFO:teuthology.orchestra.run.smithi117.stdout: "osdmap_mapping_items": 0, 2024-08-19T01:43:46.793 INFO:teuthology.orchestra.run.smithi117.stdout: "pgmap_bytes": 0, 2024-08-19T01:43:46.793 INFO:teuthology.orchestra.run.smithi117.stdout: "pgmap_items": 0, 2024-08-19T01:43:46.793 INFO:teuthology.orchestra.run.smithi117.stdout: "mds_co_bytes": 928, 2024-08-19T01:43:46.793 INFO:teuthology.orchestra.run.smithi117.stdout: "mds_co_items": 2, 2024-08-19T01:43:46.793 INFO:teuthology.orchestra.run.smithi117.stdout: "unittest_1_bytes": 0, 2024-08-19T01:43:46.793 INFO:teuthology.orchestra.run.smithi117.stdout: "unittest_1_items": 0, 2024-08-19T01:43:46.793 INFO:teuthology.orchestra.run.smithi117.stdout: "unittest_2_bytes": 0, 2024-08-19T01:43:46.793 INFO:teuthology.orchestra.run.smithi117.stdout: "unittest_2_items": 0 2024-08-19T01:43:46.793 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-08-19T01:43:46.793 INFO:teuthology.orchestra.run.smithi117.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-08-19T01:43:46.794 INFO:teuthology.orchestra.run.smithi117.stdout: "val": 0, 2024-08-19T01:43:46.794 INFO:teuthology.orchestra.run.smithi117.stdout: "max": 104857600, 2024-08-19T01:43:46.794 INFO:teuthology.orchestra.run.smithi117.stdout: "get_started": 0, 2024-08-19T01:43:46.794 INFO:teuthology.orchestra.run.smithi117.stdout: "get": 197, 2024-08-19T01:43:46.794 INFO:teuthology.orchestra.run.smithi117.stdout: "get_sum": 719101, 2024-08-19T01:43:46.794 INFO:teuthology.orchestra.run.smithi117.stdout: "get_or_fail_fail": 0, 2024-08-19T01:43:46.794 INFO:teuthology.orchestra.run.smithi117.stdout: "get_or_fail_success": 197, 2024-08-19T01:43:46.794 INFO:teuthology.orchestra.run.smithi117.stdout: "take": 0, 2024-08-19T01:43:46.794 INFO:teuthology.orchestra.run.smithi117.stdout: "take_sum": 0, 2024-08-19T01:43:46.794 INFO:teuthology.orchestra.run.smithi117.stdout: "put": 197, 2024-08-19T01:43:46.794 INFO:teuthology.orchestra.run.smithi117.stdout: "put_sum": 719101, 2024-08-19T01:43:46.794 INFO:teuthology.orchestra.run.smithi117.stdout: "wait": { 2024-08-19T01:43:46.794 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2024-08-19T01:43:46.794 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2024-08-19T01:43:46.794 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2024-08-19T01:43:46.795 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-08-19T01:43:46.795 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-08-19T01:43:46.795 INFO:teuthology.orchestra.run.smithi117.stdout:} 2024-08-19T01:43:47.432 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.k 2024-08-19T01:43:47.432 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.k 2024-08-19T01:43:47.432 WARNING:tasks.check_counter:Counter 'mds.dir_split' not found on daemon mds.k 2024-08-19T01:43:47.432 DEBUG:tasks.check_counter:Getting stats from c 2024-08-19T01:43:47.432 DEBUG:teuthology.orchestra.run.smithi167:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833 shell --fsid bad8780e-5dc9-11ef-bccf-c7b262605968 -- ceph daemon mds.c perf dump 2024-08-19T01:43:47.698 INFO:teuthology.orchestra.run.smithi167.stderr:Inferring config /var/lib/ceph/bad8780e-5dc9-11ef-bccf-c7b262605968/mon.c/config 2024-08-19T01:43:47.702 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:43:47 smithi117 ceph-mon[22016]: pgmap v607: 97 pgs: 97 active+clean; 35 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-08-19T01:43:47.716 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:43:47 smithi167 ceph-mon[21875]: pgmap v607: 97 pgs: 97 active+clean; 35 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-08-19T01:43:47.906 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:43:47 smithi083 ceph-mon[17120]: pgmap v607: 97 pgs: 97 active+clean; 35 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-08-19T01:43:48.183 INFO:teuthology.orchestra.run.smithi167.stdout:{ 2024-08-19T01:43:48.184 INFO:teuthology.orchestra.run.smithi167.stdout: "AsyncMessenger::Worker-0": { 2024-08-19T01:43:48.184 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_recv_messages": 502, 2024-08-19T01:43:48.184 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_send_messages": 504, 2024-08-19T01:43:48.184 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_recv_bytes": 4323875, 2024-08-19T01:43:48.184 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_send_bytes": 159700, 2024-08-19T01:43:48.184 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_created_connections": 7, 2024-08-19T01:43:48.184 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_active_connections": 3, 2024-08-19T01:43:48.184 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_running_total_time": 0.051006436, 2024-08-19T01:43:48.184 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_running_send_time": 0.013848741, 2024-08-19T01:43:48.184 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_running_recv_time": 0.017141800, 2024-08-19T01:43:48.184 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_running_fast_dispatch_time": 0.014687187, 2024-08-19T01:43:48.184 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_send_messages_queue_lat": { 2024-08-19T01:43:48.185 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 502, 2024-08-19T01:43:48.185 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.022276126, 2024-08-19T01:43:48.185 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000044374 2024-08-19T01:43:48.185 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-08-19T01:43:48.185 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_handle_ack_lat": { 2024-08-19T01:43:48.185 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-08-19T01:43:48.185 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-08-19T01:43:48.185 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-08-19T01:43:48.185 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-08-19T01:43:48.185 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_recv_encrypted_bytes": 0, 2024-08-19T01:43:48.185 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_send_encrypted_bytes": 0 2024-08-19T01:43:48.185 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-08-19T01:43:48.185 INFO:teuthology.orchestra.run.smithi167.stdout: "AsyncMessenger::Worker-1": { 2024-08-19T01:43:48.186 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_recv_messages": 643, 2024-08-19T01:43:48.186 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_send_messages": 784, 2024-08-19T01:43:48.186 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_recv_bytes": 8417727, 2024-08-19T01:43:48.186 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_send_bytes": 528353, 2024-08-19T01:43:48.186 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_created_connections": 5, 2024-08-19T01:43:48.186 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_active_connections": 4, 2024-08-19T01:43:48.186 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_running_total_time": 0.072895547, 2024-08-19T01:43:48.186 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_running_send_time": 0.023772223, 2024-08-19T01:43:48.186 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_running_recv_time": 0.021778800, 2024-08-19T01:43:48.186 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_running_fast_dispatch_time": 0.020787980, 2024-08-19T01:43:48.186 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_send_messages_queue_lat": { 2024-08-19T01:43:48.186 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 782, 2024-08-19T01:43:48.186 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.030920070, 2024-08-19T01:43:48.187 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000039539 2024-08-19T01:43:48.187 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-08-19T01:43:48.187 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_handle_ack_lat": { 2024-08-19T01:43:48.187 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-08-19T01:43:48.187 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-08-19T01:43:48.187 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-08-19T01:43:48.187 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-08-19T01:43:48.187 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_recv_encrypted_bytes": 160, 2024-08-19T01:43:48.187 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_send_encrypted_bytes": 324192 2024-08-19T01:43:48.187 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-08-19T01:43:48.187 INFO:teuthology.orchestra.run.smithi167.stdout: "AsyncMessenger::Worker-2": { 2024-08-19T01:43:48.187 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_recv_messages": 1103, 2024-08-19T01:43:48.187 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_send_messages": 1088, 2024-08-19T01:43:48.188 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_recv_bytes": 5272156, 2024-08-19T01:43:48.188 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_send_bytes": 400056, 2024-08-19T01:43:48.188 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_created_connections": 4, 2024-08-19T01:43:48.188 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_active_connections": 3, 2024-08-19T01:43:48.188 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_running_total_time": 0.125167076, 2024-08-19T01:43:48.188 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_running_send_time": 0.040001177, 2024-08-19T01:43:48.188 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_running_recv_time": 0.054920589, 2024-08-19T01:43:48.188 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_running_fast_dispatch_time": 0.030116190, 2024-08-19T01:43:48.188 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_send_messages_queue_lat": { 2024-08-19T01:43:48.188 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 1088, 2024-08-19T01:43:48.188 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.050901330, 2024-08-19T01:43:48.188 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000046784 2024-08-19T01:43:48.189 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-08-19T01:43:48.189 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_handle_ack_lat": { 2024-08-19T01:43:48.189 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-08-19T01:43:48.189 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-08-19T01:43:48.189 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-08-19T01:43:48.189 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-08-19T01:43:48.189 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_recv_encrypted_bytes": 797664, 2024-08-19T01:43:48.189 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_send_encrypted_bytes": 116400 2024-08-19T01:43:48.189 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-08-19T01:43:48.189 INFO:teuthology.orchestra.run.smithi167.stdout: "cct": { 2024-08-19T01:43:48.189 INFO:teuthology.orchestra.run.smithi167.stdout: "total_workers": 1, 2024-08-19T01:43:48.189 INFO:teuthology.orchestra.run.smithi167.stdout: "unhealthy_workers": 0 2024-08-19T01:43:48.189 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-08-19T01:43:48.190 INFO:teuthology.orchestra.run.smithi167.stdout: "finisher-MDSRank": { 2024-08-19T01:43:48.190 INFO:teuthology.orchestra.run.smithi167.stdout: "queue_len": 0, 2024-08-19T01:43:48.190 INFO:teuthology.orchestra.run.smithi167.stdout: "complete_latency": { 2024-08-19T01:43:48.190 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 3434, 2024-08-19T01:43:48.190 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.095917478, 2024-08-19T01:43:48.190 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000027931 2024-08-19T01:43:48.190 INFO:teuthology.orchestra.run.smithi167.stdout: } 2024-08-19T01:43:48.190 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-08-19T01:43:48.190 INFO:teuthology.orchestra.run.smithi167.stdout: "finisher-PurgeQueue": { 2024-08-19T01:43:48.190 INFO:teuthology.orchestra.run.smithi167.stdout: "queue_len": 0, 2024-08-19T01:43:48.190 INFO:teuthology.orchestra.run.smithi167.stdout: "complete_latency": { 2024-08-19T01:43:48.190 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-08-19T01:43:48.190 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-08-19T01:43:48.190 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-08-19T01:43:48.191 INFO:teuthology.orchestra.run.smithi167.stdout: } 2024-08-19T01:43:48.191 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-08-19T01:43:48.191 INFO:teuthology.orchestra.run.smithi167.stdout: "mds": { 2024-08-19T01:43:48.191 INFO:teuthology.orchestra.run.smithi167.stdout: "request": 0, 2024-08-19T01:43:48.191 INFO:teuthology.orchestra.run.smithi167.stdout: "reply": 0, 2024-08-19T01:43:48.191 INFO:teuthology.orchestra.run.smithi167.stdout: "reply_latency": { 2024-08-19T01:43:48.191 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-08-19T01:43:48.191 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-08-19T01:43:48.191 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-08-19T01:43:48.191 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-08-19T01:43:48.191 INFO:teuthology.orchestra.run.smithi167.stdout: "slow_reply": 0, 2024-08-19T01:43:48.191 INFO:teuthology.orchestra.run.smithi167.stdout: "forward": 0, 2024-08-19T01:43:48.191 INFO:teuthology.orchestra.run.smithi167.stdout: "dir_fetch_complete": 0, 2024-08-19T01:43:48.191 INFO:teuthology.orchestra.run.smithi167.stdout: "dir_fetch_keys": 0, 2024-08-19T01:43:48.192 INFO:teuthology.orchestra.run.smithi167.stdout: "dir_commit": 0, 2024-08-19T01:43:48.192 INFO:teuthology.orchestra.run.smithi167.stdout: "dir_split": 0, 2024-08-19T01:43:48.192 INFO:teuthology.orchestra.run.smithi167.stdout: "dir_merge": 0, 2024-08-19T01:43:48.192 INFO:teuthology.orchestra.run.smithi167.stdout: "inodes": 2088, 2024-08-19T01:43:48.192 INFO:teuthology.orchestra.run.smithi167.stdout: "inodes_top": 0, 2024-08-19T01:43:48.192 INFO:teuthology.orchestra.run.smithi167.stdout: "inodes_bottom": 2088, 2024-08-19T01:43:48.192 INFO:teuthology.orchestra.run.smithi167.stdout: "inodes_pin_tail": 0, 2024-08-19T01:43:48.192 INFO:teuthology.orchestra.run.smithi167.stdout: "inodes_pinned": 2088, 2024-08-19T01:43:48.192 INFO:teuthology.orchestra.run.smithi167.stdout: "inodes_expired": 0, 2024-08-19T01:43:48.192 INFO:teuthology.orchestra.run.smithi167.stdout: "inodes_with_caps": 0, 2024-08-19T01:43:48.192 INFO:teuthology.orchestra.run.smithi167.stdout: "caps": 0, 2024-08-19T01:43:48.192 INFO:teuthology.orchestra.run.smithi167.stdout: "subtrees": 1, 2024-08-19T01:43:48.192 INFO:teuthology.orchestra.run.smithi167.stdout: "traverse": 0, 2024-08-19T01:43:48.193 INFO:teuthology.orchestra.run.smithi167.stdout: "traverse_hit": 0, 2024-08-19T01:43:48.193 INFO:teuthology.orchestra.run.smithi167.stdout: "traverse_forward": 0, 2024-08-19T01:43:48.193 INFO:teuthology.orchestra.run.smithi167.stdout: "traverse_discover": 0, 2024-08-19T01:43:48.193 INFO:teuthology.orchestra.run.smithi167.stdout: "traverse_dir_fetch": 0, 2024-08-19T01:43:48.193 INFO:teuthology.orchestra.run.smithi167.stdout: "traverse_remote_ino": 0, 2024-08-19T01:43:48.193 INFO:teuthology.orchestra.run.smithi167.stdout: "traverse_lock": 0, 2024-08-19T01:43:48.193 INFO:teuthology.orchestra.run.smithi167.stdout: "load_cent": 0, 2024-08-19T01:43:48.193 INFO:teuthology.orchestra.run.smithi167.stdout: "q": 0, 2024-08-19T01:43:48.193 INFO:teuthology.orchestra.run.smithi167.stdout: "exported": 0, 2024-08-19T01:43:48.193 INFO:teuthology.orchestra.run.smithi167.stdout: "exported_inodes": 0, 2024-08-19T01:43:48.193 INFO:teuthology.orchestra.run.smithi167.stdout: "imported": 0, 2024-08-19T01:43:48.193 INFO:teuthology.orchestra.run.smithi167.stdout: "imported_inodes": 0, 2024-08-19T01:43:48.194 INFO:teuthology.orchestra.run.smithi167.stdout: "openino_dir_fetch": 0, 2024-08-19T01:43:48.194 INFO:teuthology.orchestra.run.smithi167.stdout: "openino_backtrace_fetch": 0, 2024-08-19T01:43:48.194 INFO:teuthology.orchestra.run.smithi167.stdout: "openino_peer_discover": 0, 2024-08-19T01:43:48.194 INFO:teuthology.orchestra.run.smithi167.stdout: "root_rfiles": 0, 2024-08-19T01:43:48.194 INFO:teuthology.orchestra.run.smithi167.stdout: "root_rbytes": 0, 2024-08-19T01:43:48.194 INFO:teuthology.orchestra.run.smithi167.stdout: "root_rsnaps": 0, 2024-08-19T01:43:48.194 INFO:teuthology.orchestra.run.smithi167.stdout: "scrub_backtrace_fetch": 0, 2024-08-19T01:43:48.194 INFO:teuthology.orchestra.run.smithi167.stdout: "scrub_set_tag": 0, 2024-08-19T01:43:48.194 INFO:teuthology.orchestra.run.smithi167.stdout: "scrub_backtrace_repaired": 0, 2024-08-19T01:43:48.194 INFO:teuthology.orchestra.run.smithi167.stdout: "scrub_inotable_repaired": 0, 2024-08-19T01:43:48.194 INFO:teuthology.orchestra.run.smithi167.stdout: "scrub_dir_inodes": 0, 2024-08-19T01:43:48.194 INFO:teuthology.orchestra.run.smithi167.stdout: "scrub_dir_base_inodes": 0, 2024-08-19T01:43:48.194 INFO:teuthology.orchestra.run.smithi167.stdout: "scrub_dirfrag_rstats": 0, 2024-08-19T01:43:48.195 INFO:teuthology.orchestra.run.smithi167.stdout: "scrub_file_inodes": 0, 2024-08-19T01:43:48.195 INFO:teuthology.orchestra.run.smithi167.stdout: "handle_inode_file_caps": 0, 2024-08-19T01:43:48.195 INFO:teuthology.orchestra.run.smithi167.stdout: "ceph_cap_op_revoke": 0, 2024-08-19T01:43:48.195 INFO:teuthology.orchestra.run.smithi167.stdout: "ceph_cap_op_grant": 0, 2024-08-19T01:43:48.195 INFO:teuthology.orchestra.run.smithi167.stdout: "ceph_cap_op_trunc": 0, 2024-08-19T01:43:48.195 INFO:teuthology.orchestra.run.smithi167.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-08-19T01:43:48.195 INFO:teuthology.orchestra.run.smithi167.stdout: "ceph_cap_op_flush_ack": 0, 2024-08-19T01:43:48.195 INFO:teuthology.orchestra.run.smithi167.stdout: "handle_client_caps": 0, 2024-08-19T01:43:48.195 INFO:teuthology.orchestra.run.smithi167.stdout: "handle_client_caps_dirty": 0, 2024-08-19T01:43:48.195 INFO:teuthology.orchestra.run.smithi167.stdout: "handle_client_cap_release": 0, 2024-08-19T01:43:48.195 INFO:teuthology.orchestra.run.smithi167.stdout: "process_request_cap_release": 0 2024-08-19T01:43:48.195 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-08-19T01:43:48.195 INFO:teuthology.orchestra.run.smithi167.stdout: "mds_cache": { 2024-08-19T01:43:48.195 INFO:teuthology.orchestra.run.smithi167.stdout: "dir_update": 0, 2024-08-19T01:43:48.196 INFO:teuthology.orchestra.run.smithi167.stdout: "dir_update_receipt": 0, 2024-08-19T01:43:48.196 INFO:teuthology.orchestra.run.smithi167.stdout: "dir_try_discover": 0, 2024-08-19T01:43:48.196 INFO:teuthology.orchestra.run.smithi167.stdout: "dir_send_discover": 0, 2024-08-19T01:43:48.196 INFO:teuthology.orchestra.run.smithi167.stdout: "dir_handle_discover": 0, 2024-08-19T01:43:48.196 INFO:teuthology.orchestra.run.smithi167.stdout: "num_strays": 0, 2024-08-19T01:43:48.196 INFO:teuthology.orchestra.run.smithi167.stdout: "num_strays_delayed": 0, 2024-08-19T01:43:48.196 INFO:teuthology.orchestra.run.smithi167.stdout: "num_strays_enqueuing": 0, 2024-08-19T01:43:48.196 INFO:teuthology.orchestra.run.smithi167.stdout: "strays_created": 0, 2024-08-19T01:43:48.196 INFO:teuthology.orchestra.run.smithi167.stdout: "strays_enqueued": 0, 2024-08-19T01:43:48.196 INFO:teuthology.orchestra.run.smithi167.stdout: "strays_reintegrated": 0, 2024-08-19T01:43:48.196 INFO:teuthology.orchestra.run.smithi167.stdout: "strays_migrated": 0, 2024-08-19T01:43:48.196 INFO:teuthology.orchestra.run.smithi167.stdout: "num_recovering_processing": 0, 2024-08-19T01:43:48.196 INFO:teuthology.orchestra.run.smithi167.stdout: "num_recovering_enqueued": 0, 2024-08-19T01:43:48.196 INFO:teuthology.orchestra.run.smithi167.stdout: "num_recovering_prioritized": 0, 2024-08-19T01:43:48.197 INFO:teuthology.orchestra.run.smithi167.stdout: "recovery_started": 0, 2024-08-19T01:43:48.197 INFO:teuthology.orchestra.run.smithi167.stdout: "recovery_completed": 0, 2024-08-19T01:43:48.197 INFO:teuthology.orchestra.run.smithi167.stdout: "ireq_quiesce_path": 0, 2024-08-19T01:43:48.197 INFO:teuthology.orchestra.run.smithi167.stdout: "ireq_quiesce_inode": 0, 2024-08-19T01:43:48.197 INFO:teuthology.orchestra.run.smithi167.stdout: "ireq_enqueue_scrub": 0, 2024-08-19T01:43:48.197 INFO:teuthology.orchestra.run.smithi167.stdout: "ireq_exportdir": 0, 2024-08-19T01:43:48.197 INFO:teuthology.orchestra.run.smithi167.stdout: "ireq_flush": 0, 2024-08-19T01:43:48.197 INFO:teuthology.orchestra.run.smithi167.stdout: "ireq_fragmentdir": 0, 2024-08-19T01:43:48.197 INFO:teuthology.orchestra.run.smithi167.stdout: "ireq_fragstats": 0, 2024-08-19T01:43:48.197 INFO:teuthology.orchestra.run.smithi167.stdout: "ireq_inodestats": 0 2024-08-19T01:43:48.197 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-08-19T01:43:48.197 INFO:teuthology.orchestra.run.smithi167.stdout: "mds_log": { 2024-08-19T01:43:48.197 INFO:teuthology.orchestra.run.smithi167.stdout: "evlrg": 0, 2024-08-19T01:43:48.198 INFO:teuthology.orchestra.run.smithi167.stdout: "evadd": 0, 2024-08-19T01:43:48.198 INFO:teuthology.orchestra.run.smithi167.stdout: "evex": 0, 2024-08-19T01:43:48.198 INFO:teuthology.orchestra.run.smithi167.stdout: "evtrm": 0, 2024-08-19T01:43:48.198 INFO:teuthology.orchestra.run.smithi167.stdout: "ev": 11484, 2024-08-19T01:43:48.198 INFO:teuthology.orchestra.run.smithi167.stdout: "evexg": 0, 2024-08-19T01:43:48.198 INFO:teuthology.orchestra.run.smithi167.stdout: "evexd": 0, 2024-08-19T01:43:48.198 INFO:teuthology.orchestra.run.smithi167.stdout: "segadd": 0, 2024-08-19T01:43:48.198 INFO:teuthology.orchestra.run.smithi167.stdout: "segex": 0, 2024-08-19T01:43:48.198 INFO:teuthology.orchestra.run.smithi167.stdout: "segtrm": 0, 2024-08-19T01:43:48.198 INFO:teuthology.orchestra.run.smithi167.stdout: "seg": 13, 2024-08-19T01:43:48.198 INFO:teuthology.orchestra.run.smithi167.stdout: "segmjr": 2, 2024-08-19T01:43:48.198 INFO:teuthology.orchestra.run.smithi167.stdout: "segexg": 0, 2024-08-19T01:43:48.198 INFO:teuthology.orchestra.run.smithi167.stdout: "segexd": 0, 2024-08-19T01:43:48.198 INFO:teuthology.orchestra.run.smithi167.stdout: "expos": 4194304, 2024-08-19T01:43:48.199 INFO:teuthology.orchestra.run.smithi167.stdout: "wrpos": 20838457, 2024-08-19T01:43:48.199 INFO:teuthology.orchestra.run.smithi167.stdout: "rdpos": 20837319, 2024-08-19T01:43:48.199 INFO:teuthology.orchestra.run.smithi167.stdout: "jlat": { 2024-08-19T01:43:48.199 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-08-19T01:43:48.199 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-08-19T01:43:48.199 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-08-19T01:43:48.199 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-08-19T01:43:48.199 INFO:teuthology.orchestra.run.smithi167.stdout: "replayed": 11484 2024-08-19T01:43:48.199 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-08-19T01:43:48.199 INFO:teuthology.orchestra.run.smithi167.stdout: "mds_mem": { 2024-08-19T01:43:48.199 INFO:teuthology.orchestra.run.smithi167.stdout: "ino": 13, 2024-08-19T01:43:48.199 INFO:teuthology.orchestra.run.smithi167.stdout: "ino+": 2121, 2024-08-19T01:43:48.199 INFO:teuthology.orchestra.run.smithi167.stdout: "ino-": 2108, 2024-08-19T01:43:48.199 INFO:teuthology.orchestra.run.smithi167.stdout: "dir": 11, 2024-08-19T01:43:48.200 INFO:teuthology.orchestra.run.smithi167.stdout: "dir+": 108, 2024-08-19T01:43:48.200 INFO:teuthology.orchestra.run.smithi167.stdout: "dir-": 97, 2024-08-19T01:43:48.200 INFO:teuthology.orchestra.run.smithi167.stdout: "dn": 2088, 2024-08-19T01:43:48.200 INFO:teuthology.orchestra.run.smithi167.stdout: "dn+": 4490, 2024-08-19T01:43:48.200 INFO:teuthology.orchestra.run.smithi167.stdout: "dn-": 2402, 2024-08-19T01:43:48.200 INFO:teuthology.orchestra.run.smithi167.stdout: "cap": 0, 2024-08-19T01:43:48.200 INFO:teuthology.orchestra.run.smithi167.stdout: "cap+": 0, 2024-08-19T01:43:48.200 INFO:teuthology.orchestra.run.smithi167.stdout: "cap-": 0, 2024-08-19T01:43:48.200 INFO:teuthology.orchestra.run.smithi167.stdout: "rss": 54404, 2024-08-19T01:43:48.200 INFO:teuthology.orchestra.run.smithi167.stdout: "heap": 215328 2024-08-19T01:43:48.200 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-08-19T01:43:48.200 INFO:teuthology.orchestra.run.smithi167.stdout: "mds_server": { 2024-08-19T01:43:48.200 INFO:teuthology.orchestra.run.smithi167.stdout: "dispatch_client_request": 0, 2024-08-19T01:43:48.200 INFO:teuthology.orchestra.run.smithi167.stdout: "dispatch_server_request": 0, 2024-08-19T01:43:48.201 INFO:teuthology.orchestra.run.smithi167.stdout: "handle_client_request": 0, 2024-08-19T01:43:48.201 INFO:teuthology.orchestra.run.smithi167.stdout: "handle_client_session": 0, 2024-08-19T01:43:48.201 INFO:teuthology.orchestra.run.smithi167.stdout: "handle_peer_request": 0, 2024-08-19T01:43:48.201 INFO:teuthology.orchestra.run.smithi167.stdout: "req_create_latency": { 2024-08-19T01:43:48.201 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-08-19T01:43:48.201 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-08-19T01:43:48.201 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-08-19T01:43:48.201 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-08-19T01:43:48.201 INFO:teuthology.orchestra.run.smithi167.stdout: "req_getattr_latency": { 2024-08-19T01:43:48.201 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-08-19T01:43:48.201 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-08-19T01:43:48.201 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-08-19T01:43:48.201 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-08-19T01:43:48.201 INFO:teuthology.orchestra.run.smithi167.stdout: "req_getfilelock_latency": { 2024-08-19T01:43:48.202 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-08-19T01:43:48.202 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-08-19T01:43:48.202 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-08-19T01:43:48.202 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-08-19T01:43:48.202 INFO:teuthology.orchestra.run.smithi167.stdout: "req_link_latency": { 2024-08-19T01:43:48.202 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-08-19T01:43:48.202 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-08-19T01:43:48.202 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-08-19T01:43:48.202 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-08-19T01:43:48.202 INFO:teuthology.orchestra.run.smithi167.stdout: "req_lookup_latency": { 2024-08-19T01:43:48.202 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-08-19T01:43:48.202 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-08-19T01:43:48.202 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-08-19T01:43:48.203 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-08-19T01:43:48.203 INFO:teuthology.orchestra.run.smithi167.stdout: "req_lookuphash_latency": { 2024-08-19T01:43:48.203 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-08-19T01:43:48.203 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-08-19T01:43:48.203 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-08-19T01:43:48.203 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-08-19T01:43:48.203 INFO:teuthology.orchestra.run.smithi167.stdout: "req_lookupino_latency": { 2024-08-19T01:43:48.203 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-08-19T01:43:48.203 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-08-19T01:43:48.203 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-08-19T01:43:48.203 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-08-19T01:43:48.203 INFO:teuthology.orchestra.run.smithi167.stdout: "req_lookupname_latency": { 2024-08-19T01:43:48.203 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-08-19T01:43:48.203 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-08-19T01:43:48.204 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-08-19T01:43:48.204 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-08-19T01:43:48.204 INFO:teuthology.orchestra.run.smithi167.stdout: "req_lookupparent_latency": { 2024-08-19T01:43:48.204 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-08-19T01:43:48.204 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-08-19T01:43:48.204 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-08-19T01:43:48.204 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-08-19T01:43:48.204 INFO:teuthology.orchestra.run.smithi167.stdout: "req_lookupsnap_latency": { 2024-08-19T01:43:48.204 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-08-19T01:43:48.204 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-08-19T01:43:48.204 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-08-19T01:43:48.204 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-08-19T01:43:48.204 INFO:teuthology.orchestra.run.smithi167.stdout: "req_lssnap_latency": { 2024-08-19T01:43:48.204 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-08-19T01:43:48.205 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-08-19T01:43:48.205 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-08-19T01:43:48.205 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-08-19T01:43:48.205 INFO:teuthology.orchestra.run.smithi167.stdout: "req_mkdir_latency": { 2024-08-19T01:43:48.205 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-08-19T01:43:48.205 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-08-19T01:43:48.205 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-08-19T01:43:48.205 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-08-19T01:43:48.205 INFO:teuthology.orchestra.run.smithi167.stdout: "req_mknod_latency": { 2024-08-19T01:43:48.205 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-08-19T01:43:48.205 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-08-19T01:43:48.205 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-08-19T01:43:48.206 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-08-19T01:43:48.206 INFO:teuthology.orchestra.run.smithi167.stdout: "req_mksnap_latency": { 2024-08-19T01:43:48.206 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-08-19T01:43:48.206 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-08-19T01:43:48.206 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-08-19T01:43:48.206 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-08-19T01:43:48.206 INFO:teuthology.orchestra.run.smithi167.stdout: "req_open_latency": { 2024-08-19T01:43:48.206 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-08-19T01:43:48.206 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-08-19T01:43:48.206 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-08-19T01:43:48.206 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-08-19T01:43:48.206 INFO:teuthology.orchestra.run.smithi167.stdout: "req_readdir_latency": { 2024-08-19T01:43:48.206 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-08-19T01:43:48.207 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-08-19T01:43:48.207 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-08-19T01:43:48.207 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-08-19T01:43:48.207 INFO:teuthology.orchestra.run.smithi167.stdout: "req_rename_latency": { 2024-08-19T01:43:48.207 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-08-19T01:43:48.207 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-08-19T01:43:48.207 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-08-19T01:43:48.207 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-08-19T01:43:48.207 INFO:teuthology.orchestra.run.smithi167.stdout: "req_renamesnap_latency": { 2024-08-19T01:43:48.207 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-08-19T01:43:48.207 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-08-19T01:43:48.207 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-08-19T01:43:48.207 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-08-19T01:43:48.207 INFO:teuthology.orchestra.run.smithi167.stdout: "req_snapdiff_latency": { 2024-08-19T01:43:48.208 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-08-19T01:43:48.208 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-08-19T01:43:48.208 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-08-19T01:43:48.208 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-08-19T01:43:48.208 INFO:teuthology.orchestra.run.smithi167.stdout: "req_rmdir_latency": { 2024-08-19T01:43:48.208 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-08-19T01:43:48.208 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-08-19T01:43:48.208 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-08-19T01:43:48.208 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-08-19T01:43:48.208 INFO:teuthology.orchestra.run.smithi167.stdout: "req_rmsnap_latency": { 2024-08-19T01:43:48.208 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-08-19T01:43:48.208 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-08-19T01:43:48.208 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-08-19T01:43:48.209 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-08-19T01:43:48.209 INFO:teuthology.orchestra.run.smithi167.stdout: "req_rmxattr_latency": { 2024-08-19T01:43:48.209 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-08-19T01:43:48.209 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-08-19T01:43:48.209 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-08-19T01:43:48.209 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-08-19T01:43:48.209 INFO:teuthology.orchestra.run.smithi167.stdout: "req_setattr_latency": { 2024-08-19T01:43:48.209 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-08-19T01:43:48.209 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-08-19T01:43:48.209 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-08-19T01:43:48.209 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-08-19T01:43:48.209 INFO:teuthology.orchestra.run.smithi167.stdout: "req_setdirlayout_latency": { 2024-08-19T01:43:48.209 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-08-19T01:43:48.209 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-08-19T01:43:48.210 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-08-19T01:43:48.210 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-08-19T01:43:48.210 INFO:teuthology.orchestra.run.smithi167.stdout: "req_setfilelock_latency": { 2024-08-19T01:43:48.210 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-08-19T01:43:48.210 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-08-19T01:43:48.210 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-08-19T01:43:48.210 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-08-19T01:43:48.210 INFO:teuthology.orchestra.run.smithi167.stdout: "req_setlayout_latency": { 2024-08-19T01:43:48.210 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-08-19T01:43:48.210 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-08-19T01:43:48.210 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-08-19T01:43:48.210 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-08-19T01:43:48.210 INFO:teuthology.orchestra.run.smithi167.stdout: "req_setxattr_latency": { 2024-08-19T01:43:48.211 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-08-19T01:43:48.211 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-08-19T01:43:48.211 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-08-19T01:43:48.211 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-08-19T01:43:48.211 INFO:teuthology.orchestra.run.smithi167.stdout: "req_symlink_latency": { 2024-08-19T01:43:48.211 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-08-19T01:43:48.211 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-08-19T01:43:48.211 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-08-19T01:43:48.211 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-08-19T01:43:48.211 INFO:teuthology.orchestra.run.smithi167.stdout: "req_unlink_latency": { 2024-08-19T01:43:48.211 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-08-19T01:43:48.211 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-08-19T01:43:48.211 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-08-19T01:43:48.211 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-08-19T01:43:48.211 INFO:teuthology.orchestra.run.smithi167.stdout: "cap_revoke_eviction": 0, 2024-08-19T01:43:48.212 INFO:teuthology.orchestra.run.smithi167.stdout: "cap_acquisition_throttle": 0, 2024-08-19T01:43:48.212 INFO:teuthology.orchestra.run.smithi167.stdout: "req_getvxattr_latency": { 2024-08-19T01:43:48.212 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-08-19T01:43:48.212 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-08-19T01:43:48.212 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-08-19T01:43:48.212 INFO:teuthology.orchestra.run.smithi167.stdout: } 2024-08-19T01:43:48.212 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-08-19T01:43:48.212 INFO:teuthology.orchestra.run.smithi167.stdout: "mds_sessions": { 2024-08-19T01:43:48.213 INFO:teuthology.orchestra.run.smithi167.stdout: "session_count": 1, 2024-08-19T01:43:48.213 INFO:teuthology.orchestra.run.smithi167.stdout: "session_add": 3, 2024-08-19T01:43:48.213 INFO:teuthology.orchestra.run.smithi167.stdout: "session_remove": 2, 2024-08-19T01:43:48.213 INFO:teuthology.orchestra.run.smithi167.stdout: "sessions_open": 2, 2024-08-19T01:43:48.213 INFO:teuthology.orchestra.run.smithi167.stdout: "sessions_stale": 0, 2024-08-19T01:43:48.213 INFO:teuthology.orchestra.run.smithi167.stdout: "total_load": 0, 2024-08-19T01:43:48.213 INFO:teuthology.orchestra.run.smithi167.stdout: "average_load": 0, 2024-08-19T01:43:48.213 INFO:teuthology.orchestra.run.smithi167.stdout: "avg_session_uptime": 2976, 2024-08-19T01:43:48.213 INFO:teuthology.orchestra.run.smithi167.stdout: "mdthresh_evicted": 0 2024-08-19T01:43:48.213 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-08-19T01:43:48.213 INFO:teuthology.orchestra.run.smithi167.stdout: "mempool": { 2024-08-19T01:43:48.213 INFO:teuthology.orchestra.run.smithi167.stdout: "bloom_filter_bytes": 0, 2024-08-19T01:43:48.213 INFO:teuthology.orchestra.run.smithi167.stdout: "bloom_filter_items": 0, 2024-08-19T01:43:48.214 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_alloc_bytes": 0, 2024-08-19T01:43:48.214 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_alloc_items": 0, 2024-08-19T01:43:48.214 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_cache_data_bytes": 0, 2024-08-19T01:43:48.214 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_cache_data_items": 0, 2024-08-19T01:43:48.214 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_cache_onode_bytes": 0, 2024-08-19T01:43:48.214 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_cache_onode_items": 0, 2024-08-19T01:43:48.214 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_cache_meta_bytes": 0, 2024-08-19T01:43:48.214 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_cache_meta_items": 0, 2024-08-19T01:43:48.214 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_cache_other_bytes": 0, 2024-08-19T01:43:48.214 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_cache_other_items": 0, 2024-08-19T01:43:48.214 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_cache_buffer_bytes": 0, 2024-08-19T01:43:48.214 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_cache_buffer_items": 0, 2024-08-19T01:43:48.215 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_extent_bytes": 0, 2024-08-19T01:43:48.215 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_extent_items": 0, 2024-08-19T01:43:48.215 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_blob_bytes": 0, 2024-08-19T01:43:48.215 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_blob_items": 0, 2024-08-19T01:43:48.215 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_shared_blob_bytes": 0, 2024-08-19T01:43:48.215 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_shared_blob_items": 0, 2024-08-19T01:43:48.215 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_inline_bl_bytes": 0, 2024-08-19T01:43:48.215 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_inline_bl_items": 0, 2024-08-19T01:43:48.215 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_fsck_bytes": 0, 2024-08-19T01:43:48.215 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_fsck_items": 0, 2024-08-19T01:43:48.215 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_txc_bytes": 0, 2024-08-19T01:43:48.215 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_txc_items": 0, 2024-08-19T01:43:48.215 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_writing_deferred_bytes": 0, 2024-08-19T01:43:48.215 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_writing_deferred_items": 0, 2024-08-19T01:43:48.215 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_writing_bytes": 0, 2024-08-19T01:43:48.216 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_writing_items": 0, 2024-08-19T01:43:48.216 INFO:teuthology.orchestra.run.smithi167.stdout: "bluefs_bytes": 0, 2024-08-19T01:43:48.216 INFO:teuthology.orchestra.run.smithi167.stdout: "bluefs_items": 0, 2024-08-19T01:43:48.216 INFO:teuthology.orchestra.run.smithi167.stdout: "bluefs_file_reader_bytes": 0, 2024-08-19T01:43:48.216 INFO:teuthology.orchestra.run.smithi167.stdout: "bluefs_file_reader_items": 0, 2024-08-19T01:43:48.216 INFO:teuthology.orchestra.run.smithi167.stdout: "bluefs_file_writer_bytes": 0, 2024-08-19T01:43:48.216 INFO:teuthology.orchestra.run.smithi167.stdout: "bluefs_file_writer_items": 0, 2024-08-19T01:43:48.216 INFO:teuthology.orchestra.run.smithi167.stdout: "buffer_anon_bytes": 206327, 2024-08-19T01:43:48.216 INFO:teuthology.orchestra.run.smithi167.stdout: "buffer_anon_items": 60, 2024-08-19T01:43:48.216 INFO:teuthology.orchestra.run.smithi167.stdout: "buffer_meta_bytes": 0, 2024-08-19T01:43:48.216 INFO:teuthology.orchestra.run.smithi167.stdout: "buffer_meta_items": 0, 2024-08-19T01:43:48.216 INFO:teuthology.orchestra.run.smithi167.stdout: "osd_bytes": 0, 2024-08-19T01:43:48.216 INFO:teuthology.orchestra.run.smithi167.stdout: "osd_items": 0, 2024-08-19T01:43:48.217 INFO:teuthology.orchestra.run.smithi167.stdout: "osd_mapbl_bytes": 0, 2024-08-19T01:43:48.217 INFO:teuthology.orchestra.run.smithi167.stdout: "osd_mapbl_items": 0, 2024-08-19T01:43:48.217 INFO:teuthology.orchestra.run.smithi167.stdout: "osd_pglog_bytes": 0, 2024-08-19T01:43:48.217 INFO:teuthology.orchestra.run.smithi167.stdout: "osd_pglog_items": 0, 2024-08-19T01:43:48.217 INFO:teuthology.orchestra.run.smithi167.stdout: "osdmap_bytes": 7688, 2024-08-19T01:43:48.217 INFO:teuthology.orchestra.run.smithi167.stdout: "osdmap_items": 158, 2024-08-19T01:43:48.217 INFO:teuthology.orchestra.run.smithi167.stdout: "osdmap_mapping_bytes": 0, 2024-08-19T01:43:48.217 INFO:teuthology.orchestra.run.smithi167.stdout: "osdmap_mapping_items": 0, 2024-08-19T01:43:48.217 INFO:teuthology.orchestra.run.smithi167.stdout: "pgmap_bytes": 0, 2024-08-19T01:43:48.217 INFO:teuthology.orchestra.run.smithi167.stdout: "pgmap_items": 0, 2024-08-19T01:43:48.217 INFO:teuthology.orchestra.run.smithi167.stdout: "mds_co_bytes": 1513544, 2024-08-19T01:43:48.217 INFO:teuthology.orchestra.run.smithi167.stdout: "mds_co_items": 8480, 2024-08-19T01:43:48.217 INFO:teuthology.orchestra.run.smithi167.stdout: "unittest_1_bytes": 0, 2024-08-19T01:43:48.217 INFO:teuthology.orchestra.run.smithi167.stdout: "unittest_1_items": 0, 2024-08-19T01:43:48.218 INFO:teuthology.orchestra.run.smithi167.stdout: "unittest_2_bytes": 0, 2024-08-19T01:43:48.218 INFO:teuthology.orchestra.run.smithi167.stdout: "unittest_2_items": 0 2024-08-19T01:43:48.218 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-08-19T01:43:48.218 INFO:teuthology.orchestra.run.smithi167.stdout: "objecter": { 2024-08-19T01:43:48.218 INFO:teuthology.orchestra.run.smithi167.stdout: "op_active": 0, 2024-08-19T01:43:48.218 INFO:teuthology.orchestra.run.smithi167.stdout: "op_laggy": 0, 2024-08-19T01:43:48.218 INFO:teuthology.orchestra.run.smithi167.stdout: "op_send": 2036, 2024-08-19T01:43:48.218 INFO:teuthology.orchestra.run.smithi167.stdout: "op_send_bytes": 21, 2024-08-19T01:43:48.218 INFO:teuthology.orchestra.run.smithi167.stdout: "op_resend": 3, 2024-08-19T01:43:48.218 INFO:teuthology.orchestra.run.smithi167.stdout: "op_reply": 2036, 2024-08-19T01:43:48.218 INFO:teuthology.orchestra.run.smithi167.stdout: "op_latency": { 2024-08-19T01:43:48.218 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 2036, 2024-08-19T01:43:48.218 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 10.177820986, 2024-08-19T01:43:48.219 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.004998929 2024-08-19T01:43:48.219 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-08-19T01:43:48.219 INFO:teuthology.orchestra.run.smithi167.stdout: "op_inflight": 0, 2024-08-19T01:43:48.219 INFO:teuthology.orchestra.run.smithi167.stdout: "oplen_avg": { 2024-08-19T01:43:48.219 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 2036, 2024-08-19T01:43:48.219 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 2037 2024-08-19T01:43:48.219 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-08-19T01:43:48.219 INFO:teuthology.orchestra.run.smithi167.stdout: "op": 2036, 2024-08-19T01:43:48.219 INFO:teuthology.orchestra.run.smithi167.stdout: "op_r": 2036, 2024-08-19T01:43:48.219 INFO:teuthology.orchestra.run.smithi167.stdout: "op_w": 0, 2024-08-19T01:43:48.219 INFO:teuthology.orchestra.run.smithi167.stdout: "op_rmw": 0, 2024-08-19T01:43:48.219 INFO:teuthology.orchestra.run.smithi167.stdout: "op_pg": 0, 2024-08-19T01:43:48.219 INFO:teuthology.orchestra.run.smithi167.stdout: "osdop_stat": 1220, 2024-08-19T01:43:48.219 INFO:teuthology.orchestra.run.smithi167.stdout: "osdop_create": 0, 2024-08-19T01:43:48.219 INFO:teuthology.orchestra.run.smithi167.stdout: "osdop_read": 814, 2024-08-19T01:43:48.220 INFO:teuthology.orchestra.run.smithi167.stdout: "osdop_write": 0, 2024-08-19T01:43:48.220 INFO:teuthology.orchestra.run.smithi167.stdout: "osdop_writefull": 0, 2024-08-19T01:43:48.220 INFO:teuthology.orchestra.run.smithi167.stdout: "osdop_writesame": 0, 2024-08-19T01:43:48.220 INFO:teuthology.orchestra.run.smithi167.stdout: "osdop_append": 0, 2024-08-19T01:43:48.220 INFO:teuthology.orchestra.run.smithi167.stdout: "osdop_zero": 0, 2024-08-19T01:43:48.220 INFO:teuthology.orchestra.run.smithi167.stdout: "osdop_truncate": 0, 2024-08-19T01:43:48.220 INFO:teuthology.orchestra.run.smithi167.stdout: "osdop_delete": 0, 2024-08-19T01:43:48.220 INFO:teuthology.orchestra.run.smithi167.stdout: "osdop_mapext": 0, 2024-08-19T01:43:48.220 INFO:teuthology.orchestra.run.smithi167.stdout: "osdop_sparse_read": 0, 2024-08-19T01:43:48.220 INFO:teuthology.orchestra.run.smithi167.stdout: "osdop_clonerange": 0, 2024-08-19T01:43:48.220 INFO:teuthology.orchestra.run.smithi167.stdout: "osdop_getxattr": 1, 2024-08-19T01:43:48.220 INFO:teuthology.orchestra.run.smithi167.stdout: "osdop_setxattr": 0, 2024-08-19T01:43:48.220 INFO:teuthology.orchestra.run.smithi167.stdout: "osdop_cmpxattr": 0, 2024-08-19T01:43:48.220 INFO:teuthology.orchestra.run.smithi167.stdout: "osdop_rmxattr": 0, 2024-08-19T01:43:48.220 INFO:teuthology.orchestra.run.smithi167.stdout: "osdop_resetxattrs": 0, 2024-08-19T01:43:48.221 INFO:teuthology.orchestra.run.smithi167.stdout: "osdop_call": 0, 2024-08-19T01:43:48.221 INFO:teuthology.orchestra.run.smithi167.stdout: "osdop_watch": 0, 2024-08-19T01:43:48.221 INFO:teuthology.orchestra.run.smithi167.stdout: "osdop_notify": 0, 2024-08-19T01:43:48.221 INFO:teuthology.orchestra.run.smithi167.stdout: "osdop_src_cmpxattr": 0, 2024-08-19T01:43:48.221 INFO:teuthology.orchestra.run.smithi167.stdout: "osdop_pgls": 0, 2024-08-19T01:43:48.221 INFO:teuthology.orchestra.run.smithi167.stdout: "osdop_pgls_filter": 0, 2024-08-19T01:43:48.221 INFO:teuthology.orchestra.run.smithi167.stdout: "osdop_other": 0, 2024-08-19T01:43:48.221 INFO:teuthology.orchestra.run.smithi167.stdout: "linger_active": 0, 2024-08-19T01:43:48.221 INFO:teuthology.orchestra.run.smithi167.stdout: "linger_send": 0, 2024-08-19T01:43:48.221 INFO:teuthology.orchestra.run.smithi167.stdout: "linger_resend": 0, 2024-08-19T01:43:48.221 INFO:teuthology.orchestra.run.smithi167.stdout: "linger_ping": 0, 2024-08-19T01:43:48.221 INFO:teuthology.orchestra.run.smithi167.stdout: "poolop_active": 0, 2024-08-19T01:43:48.221 INFO:teuthology.orchestra.run.smithi167.stdout: "poolop_send": 0, 2024-08-19T01:43:48.221 INFO:teuthology.orchestra.run.smithi167.stdout: "poolop_resend": 0, 2024-08-19T01:43:48.222 INFO:teuthology.orchestra.run.smithi167.stdout: "poolstat_active": 0, 2024-08-19T01:43:48.222 INFO:teuthology.orchestra.run.smithi167.stdout: "poolstat_send": 0, 2024-08-19T01:43:48.222 INFO:teuthology.orchestra.run.smithi167.stdout: "poolstat_resend": 0, 2024-08-19T01:43:48.222 INFO:teuthology.orchestra.run.smithi167.stdout: "statfs_active": 0, 2024-08-19T01:43:48.222 INFO:teuthology.orchestra.run.smithi167.stdout: "statfs_send": 0, 2024-08-19T01:43:48.222 INFO:teuthology.orchestra.run.smithi167.stdout: "statfs_resend": 0, 2024-08-19T01:43:48.222 INFO:teuthology.orchestra.run.smithi167.stdout: "command_active": 0, 2024-08-19T01:43:48.223 INFO:teuthology.orchestra.run.smithi167.stdout: "command_send": 0, 2024-08-19T01:43:48.223 INFO:teuthology.orchestra.run.smithi167.stdout: "command_resend": 0, 2024-08-19T01:43:48.223 INFO:teuthology.orchestra.run.smithi167.stdout: "map_epoch": 86, 2024-08-19T01:43:48.223 INFO:teuthology.orchestra.run.smithi167.stdout: "map_full": 0, 2024-08-19T01:43:48.223 INFO:teuthology.orchestra.run.smithi167.stdout: "map_inc": 2, 2024-08-19T01:43:48.223 INFO:teuthology.orchestra.run.smithi167.stdout: "osd_sessions": 7, 2024-08-19T01:43:48.223 INFO:teuthology.orchestra.run.smithi167.stdout: "osd_session_open": 7, 2024-08-19T01:43:48.223 INFO:teuthology.orchestra.run.smithi167.stdout: "osd_session_close": 0, 2024-08-19T01:43:48.223 INFO:teuthology.orchestra.run.smithi167.stdout: "osd_laggy": 0, 2024-08-19T01:43:48.223 INFO:teuthology.orchestra.run.smithi167.stdout: "omap_wr": 0, 2024-08-19T01:43:48.223 INFO:teuthology.orchestra.run.smithi167.stdout: "omap_rd": 2, 2024-08-19T01:43:48.223 INFO:teuthology.orchestra.run.smithi167.stdout: "omap_del": 0 2024-08-19T01:43:48.223 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-08-19T01:43:48.223 INFO:teuthology.orchestra.run.smithi167.stdout: "oft": { 2024-08-19T01:43:48.224 INFO:teuthology.orchestra.run.smithi167.stdout: "omap_total_objs": 0, 2024-08-19T01:43:48.224 INFO:teuthology.orchestra.run.smithi167.stdout: "omap_total_kv_pairs": 0, 2024-08-19T01:43:48.224 INFO:teuthology.orchestra.run.smithi167.stdout: "omap_total_updates": 0, 2024-08-19T01:43:48.224 INFO:teuthology.orchestra.run.smithi167.stdout: "omap_total_removes": 0 2024-08-19T01:43:48.224 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-08-19T01:43:48.224 INFO:teuthology.orchestra.run.smithi167.stdout: "osd-slow-ops": { 2024-08-19T01:43:48.224 INFO:teuthology.orchestra.run.smithi167.stdout: "slow_ops_count": 0 2024-08-19T01:43:48.224 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-08-19T01:43:48.224 INFO:teuthology.orchestra.run.smithi167.stdout: "purge_queue": { 2024-08-19T01:43:48.224 INFO:teuthology.orchestra.run.smithi167.stdout: "pq_executing_ops": 0, 2024-08-19T01:43:48.224 INFO:teuthology.orchestra.run.smithi167.stdout: "pq_executing_ops_high_water": 0, 2024-08-19T01:43:48.224 INFO:teuthology.orchestra.run.smithi167.stdout: "pq_executing": 0, 2024-08-19T01:43:48.224 INFO:teuthology.orchestra.run.smithi167.stdout: "pq_executing_high_water": 0, 2024-08-19T01:43:48.224 INFO:teuthology.orchestra.run.smithi167.stdout: "pq_executed_ops": 0, 2024-08-19T01:43:48.225 INFO:teuthology.orchestra.run.smithi167.stdout: "pq_executed": 0, 2024-08-19T01:43:48.225 INFO:teuthology.orchestra.run.smithi167.stdout: "pq_item_in_journal": 0 2024-08-19T01:43:48.225 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-08-19T01:43:48.225 INFO:teuthology.orchestra.run.smithi167.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-08-19T01:43:48.225 INFO:teuthology.orchestra.run.smithi167.stdout: "val": 0, 2024-08-19T01:43:48.225 INFO:teuthology.orchestra.run.smithi167.stdout: "max": 104857600, 2024-08-19T01:43:48.225 INFO:teuthology.orchestra.run.smithi167.stdout: "get_started": 0, 2024-08-19T01:43:48.225 INFO:teuthology.orchestra.run.smithi167.stdout: "get": 2244, 2024-08-19T01:43:48.225 INFO:teuthology.orchestra.run.smithi167.stdout: "get_sum": 17798181, 2024-08-19T01:43:48.225 INFO:teuthology.orchestra.run.smithi167.stdout: "get_or_fail_fail": 0, 2024-08-19T01:43:48.225 INFO:teuthology.orchestra.run.smithi167.stdout: "get_or_fail_success": 2244, 2024-08-19T01:43:48.225 INFO:teuthology.orchestra.run.smithi167.stdout: "take": 0, 2024-08-19T01:43:48.225 INFO:teuthology.orchestra.run.smithi167.stdout: "take_sum": 0, 2024-08-19T01:43:48.225 INFO:teuthology.orchestra.run.smithi167.stdout: "put": 2244, 2024-08-19T01:43:48.226 INFO:teuthology.orchestra.run.smithi167.stdout: "put_sum": 17798181, 2024-08-19T01:43:48.226 INFO:teuthology.orchestra.run.smithi167.stdout: "wait": { 2024-08-19T01:43:48.226 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-08-19T01:43:48.226 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-08-19T01:43:48.226 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-08-19T01:43:48.226 INFO:teuthology.orchestra.run.smithi167.stdout: } 2024-08-19T01:43:48.226 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-08-19T01:43:48.226 INFO:teuthology.orchestra.run.smithi167.stdout: "throttle-objecter_bytes": { 2024-08-19T01:43:48.226 INFO:teuthology.orchestra.run.smithi167.stdout: "val": 0, 2024-08-19T01:43:48.226 INFO:teuthology.orchestra.run.smithi167.stdout: "max": 104857600, 2024-08-19T01:43:48.226 INFO:teuthology.orchestra.run.smithi167.stdout: "get_started": 0, 2024-08-19T01:43:48.226 INFO:teuthology.orchestra.run.smithi167.stdout: "get": 0, 2024-08-19T01:43:48.226 INFO:teuthology.orchestra.run.smithi167.stdout: "get_sum": 0, 2024-08-19T01:43:48.227 INFO:teuthology.orchestra.run.smithi167.stdout: "get_or_fail_fail": 0, 2024-08-19T01:43:48.227 INFO:teuthology.orchestra.run.smithi167.stdout: "get_or_fail_success": 0, 2024-08-19T01:43:48.227 INFO:teuthology.orchestra.run.smithi167.stdout: "take": 2036, 2024-08-19T01:43:48.227 INFO:teuthology.orchestra.run.smithi167.stdout: "take_sum": 16645498, 2024-08-19T01:43:48.227 INFO:teuthology.orchestra.run.smithi167.stdout: "put": 2036, 2024-08-19T01:43:48.227 INFO:teuthology.orchestra.run.smithi167.stdout: "put_sum": 16645498, 2024-08-19T01:43:48.227 INFO:teuthology.orchestra.run.smithi167.stdout: "wait": { 2024-08-19T01:43:48.227 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-08-19T01:43:48.227 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-08-19T01:43:48.227 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-08-19T01:43:48.227 INFO:teuthology.orchestra.run.smithi167.stdout: } 2024-08-19T01:43:48.227 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-08-19T01:43:48.227 INFO:teuthology.orchestra.run.smithi167.stdout: "throttle-objecter_ops": { 2024-08-19T01:43:48.228 INFO:teuthology.orchestra.run.smithi167.stdout: "val": 0, 2024-08-19T01:43:48.228 INFO:teuthology.orchestra.run.smithi167.stdout: "max": 1024, 2024-08-19T01:43:48.228 INFO:teuthology.orchestra.run.smithi167.stdout: "get_started": 0, 2024-08-19T01:43:48.228 INFO:teuthology.orchestra.run.smithi167.stdout: "get": 0, 2024-08-19T01:43:48.228 INFO:teuthology.orchestra.run.smithi167.stdout: "get_sum": 0, 2024-08-19T01:43:48.228 INFO:teuthology.orchestra.run.smithi167.stdout: "get_or_fail_fail": 0, 2024-08-19T01:43:48.228 INFO:teuthology.orchestra.run.smithi167.stdout: "get_or_fail_success": 0, 2024-08-19T01:43:48.228 INFO:teuthology.orchestra.run.smithi167.stdout: "take": 2036, 2024-08-19T01:43:48.228 INFO:teuthology.orchestra.run.smithi167.stdout: "take_sum": 2036, 2024-08-19T01:43:48.228 INFO:teuthology.orchestra.run.smithi167.stdout: "put": 2036, 2024-08-19T01:43:48.228 INFO:teuthology.orchestra.run.smithi167.stdout: "put_sum": 2036, 2024-08-19T01:43:48.228 INFO:teuthology.orchestra.run.smithi167.stdout: "wait": { 2024-08-19T01:43:48.228 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-08-19T01:43:48.228 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-08-19T01:43:48.228 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-08-19T01:43:48.228 INFO:teuthology.orchestra.run.smithi167.stdout: } 2024-08-19T01:43:48.229 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-08-19T01:43:48.229 INFO:teuthology.orchestra.run.smithi167.stdout: "throttle-write_buf_throttle": { 2024-08-19T01:43:48.229 INFO:teuthology.orchestra.run.smithi167.stdout: "val": 0, 2024-08-19T01:43:48.229 INFO:teuthology.orchestra.run.smithi167.stdout: "max": 3758096384, 2024-08-19T01:43:48.229 INFO:teuthology.orchestra.run.smithi167.stdout: "get_started": 0, 2024-08-19T01:43:48.229 INFO:teuthology.orchestra.run.smithi167.stdout: "get": 0, 2024-08-19T01:43:48.229 INFO:teuthology.orchestra.run.smithi167.stdout: "get_sum": 0, 2024-08-19T01:43:48.229 INFO:teuthology.orchestra.run.smithi167.stdout: "get_or_fail_fail": 0, 2024-08-19T01:43:48.229 INFO:teuthology.orchestra.run.smithi167.stdout: "get_or_fail_success": 0, 2024-08-19T01:43:48.229 INFO:teuthology.orchestra.run.smithi167.stdout: "take": 0, 2024-08-19T01:43:48.229 INFO:teuthology.orchestra.run.smithi167.stdout: "take_sum": 0, 2024-08-19T01:43:48.229 INFO:teuthology.orchestra.run.smithi167.stdout: "put": 0, 2024-08-19T01:43:48.229 INFO:teuthology.orchestra.run.smithi167.stdout: "put_sum": 0, 2024-08-19T01:43:48.230 INFO:teuthology.orchestra.run.smithi167.stdout: "wait": { 2024-08-19T01:43:48.230 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-08-19T01:43:48.230 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-08-19T01:43:48.230 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-08-19T01:43:48.230 INFO:teuthology.orchestra.run.smithi167.stdout: } 2024-08-19T01:43:48.230 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-08-19T01:43:48.230 INFO:teuthology.orchestra.run.smithi167.stdout: "throttle-write_buf_throttle-0x55e079f34b40": { 2024-08-19T01:43:48.230 INFO:teuthology.orchestra.run.smithi167.stdout: "val": 0, 2024-08-19T01:43:48.230 INFO:teuthology.orchestra.run.smithi167.stdout: "max": 3758096384, 2024-08-19T01:43:48.230 INFO:teuthology.orchestra.run.smithi167.stdout: "get_started": 0, 2024-08-19T01:43:48.230 INFO:teuthology.orchestra.run.smithi167.stdout: "get": 0, 2024-08-19T01:43:48.230 INFO:teuthology.orchestra.run.smithi167.stdout: "get_sum": 0, 2024-08-19T01:43:48.231 INFO:teuthology.orchestra.run.smithi167.stdout: "get_or_fail_fail": 0, 2024-08-19T01:43:48.231 INFO:teuthology.orchestra.run.smithi167.stdout: "get_or_fail_success": 0, 2024-08-19T01:43:48.231 INFO:teuthology.orchestra.run.smithi167.stdout: "take": 0, 2024-08-19T01:43:48.231 INFO:teuthology.orchestra.run.smithi167.stdout: "take_sum": 0, 2024-08-19T01:43:48.231 INFO:teuthology.orchestra.run.smithi167.stdout: "put": 0, 2024-08-19T01:43:48.231 INFO:teuthology.orchestra.run.smithi167.stdout: "put_sum": 0, 2024-08-19T01:43:48.231 INFO:teuthology.orchestra.run.smithi167.stdout: "wait": { 2024-08-19T01:43:48.231 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-08-19T01:43:48.231 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-08-19T01:43:48.231 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-08-19T01:43:48.231 INFO:teuthology.orchestra.run.smithi167.stdout: } 2024-08-19T01:43:48.231 INFO:teuthology.orchestra.run.smithi167.stdout: } 2024-08-19T01:43:48.231 INFO:teuthology.orchestra.run.smithi167.stdout:} 2024-08-19T01:43:49.160 INFO:tasks.check_counter:Daemon mds.c mds.exported=0 2024-08-19T01:43:49.160 INFO:tasks.check_counter:Daemon mds.c mds.imported=0 2024-08-19T01:43:49.160 INFO:tasks.check_counter:Daemon mds.c mds.dir_split=0 2024-08-19T01:43:49.161 DEBUG:tasks.check_counter:Getting stats from f 2024-08-19T01:43:49.161 DEBUG:teuthology.orchestra.run.smithi167:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833 shell --fsid bad8780e-5dc9-11ef-bccf-c7b262605968 -- ceph daemon mds.f perf dump 2024-08-19T01:43:49.418 INFO:teuthology.orchestra.run.smithi167.stderr:Inferring config /var/lib/ceph/bad8780e-5dc9-11ef-bccf-c7b262605968/mon.c/config 2024-08-19T01:43:49.868 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:43:49 smithi167 ceph-mon[21875]: pgmap v608: 97 pgs: 97 active+clean; 35 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-08-19T01:43:49.904 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:43:49 smithi083 ceph-mon[17120]: pgmap v608: 97 pgs: 97 active+clean; 35 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-08-19T01:43:49.952 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:43:49 smithi117 ceph-mon[22016]: pgmap v608: 97 pgs: 97 active+clean; 35 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-08-19T01:43:49.976 INFO:teuthology.orchestra.run.smithi167.stdout:{ 2024-08-19T01:43:49.976 INFO:teuthology.orchestra.run.smithi167.stdout: "AsyncMessenger::Worker-0": { 2024-08-19T01:43:49.976 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_recv_messages": 0, 2024-08-19T01:43:49.976 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_send_messages": 2, 2024-08-19T01:43:49.976 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_recv_bytes": 0, 2024-08-19T01:43:49.976 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_send_bytes": 0, 2024-08-19T01:43:49.976 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_created_connections": 4, 2024-08-19T01:43:49.976 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_active_connections": 0, 2024-08-19T01:43:49.976 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_running_total_time": 0.001037677, 2024-08-19T01:43:49.976 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_running_send_time": 0.000000000, 2024-08-19T01:43:49.976 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_running_recv_time": 0.000174277, 2024-08-19T01:43:49.976 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-08-19T01:43:49.977 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_send_messages_queue_lat": { 2024-08-19T01:43:49.977 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-08-19T01:43:49.977 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-08-19T01:43:49.977 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-08-19T01:43:49.977 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-08-19T01:43:49.977 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_handle_ack_lat": { 2024-08-19T01:43:49.977 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-08-19T01:43:49.977 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-08-19T01:43:49.977 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-08-19T01:43:49.977 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-08-19T01:43:49.977 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_recv_encrypted_bytes": 0, 2024-08-19T01:43:49.977 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_send_encrypted_bytes": 0 2024-08-19T01:43:49.977 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-08-19T01:43:49.977 INFO:teuthology.orchestra.run.smithi167.stdout: "AsyncMessenger::Worker-1": { 2024-08-19T01:43:49.977 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_recv_messages": 5, 2024-08-19T01:43:49.977 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_send_messages": 145, 2024-08-19T01:43:49.978 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_recv_bytes": 3184, 2024-08-19T01:43:49.978 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_send_bytes": 103840, 2024-08-19T01:43:49.978 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_created_connections": 3, 2024-08-19T01:43:49.978 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_active_connections": 2, 2024-08-19T01:43:49.978 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_running_total_time": 0.009824424, 2024-08-19T01:43:49.978 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_running_send_time": 0.006001854, 2024-08-19T01:43:49.978 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_running_recv_time": 0.002425442, 2024-08-19T01:43:49.978 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-08-19T01:43:49.978 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_send_messages_queue_lat": { 2024-08-19T01:43:49.978 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 144, 2024-08-19T01:43:49.978 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.012255538, 2024-08-19T01:43:49.978 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000085107 2024-08-19T01:43:49.978 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-08-19T01:43:49.978 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_handle_ack_lat": { 2024-08-19T01:43:49.979 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-08-19T01:43:49.979 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-08-19T01:43:49.979 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-08-19T01:43:49.979 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-08-19T01:43:49.979 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_recv_encrypted_bytes": 3184, 2024-08-19T01:43:49.979 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_send_encrypted_bytes": 103840 2024-08-19T01:43:49.979 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-08-19T01:43:49.979 INFO:teuthology.orchestra.run.smithi167.stdout: "AsyncMessenger::Worker-2": { 2024-08-19T01:43:49.979 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_recv_messages": 193, 2024-08-19T01:43:49.979 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_send_messages": 188, 2024-08-19T01:43:49.979 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_recv_bytes": 745952, 2024-08-19T01:43:49.979 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_send_bytes": 115408, 2024-08-19T01:43:49.979 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_created_connections": 2, 2024-08-19T01:43:49.979 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_active_connections": 1, 2024-08-19T01:43:49.979 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_running_total_time": 0.029149396, 2024-08-19T01:43:49.980 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_running_send_time": 0.012641118, 2024-08-19T01:43:49.980 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_running_recv_time": 0.022289820, 2024-08-19T01:43:49.980 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-08-19T01:43:49.980 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_send_messages_queue_lat": { 2024-08-19T01:43:49.980 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 187, 2024-08-19T01:43:49.980 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.010054819, 2024-08-19T01:43:49.980 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000053769 2024-08-19T01:43:49.980 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-08-19T01:43:49.980 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_handle_ack_lat": { 2024-08-19T01:43:49.980 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-08-19T01:43:49.980 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-08-19T01:43:49.980 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-08-19T01:43:49.980 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-08-19T01:43:49.980 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_recv_encrypted_bytes": 745952, 2024-08-19T01:43:49.980 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_send_encrypted_bytes": 115408 2024-08-19T01:43:49.980 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-08-19T01:43:49.981 INFO:teuthology.orchestra.run.smithi167.stdout: "cct": { 2024-08-19T01:43:49.981 INFO:teuthology.orchestra.run.smithi167.stdout: "total_workers": 0, 2024-08-19T01:43:49.981 INFO:teuthology.orchestra.run.smithi167.stdout: "unhealthy_workers": 0 2024-08-19T01:43:49.981 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-08-19T01:43:49.981 INFO:teuthology.orchestra.run.smithi167.stdout: "mempool": { 2024-08-19T01:43:49.981 INFO:teuthology.orchestra.run.smithi167.stdout: "bloom_filter_bytes": 0, 2024-08-19T01:43:49.981 INFO:teuthology.orchestra.run.smithi167.stdout: "bloom_filter_items": 0, 2024-08-19T01:43:49.981 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_alloc_bytes": 0, 2024-08-19T01:43:49.981 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_alloc_items": 0, 2024-08-19T01:43:49.981 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_cache_data_bytes": 0, 2024-08-19T01:43:49.981 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_cache_data_items": 0, 2024-08-19T01:43:49.981 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_cache_onode_bytes": 0, 2024-08-19T01:43:49.981 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_cache_onode_items": 0, 2024-08-19T01:43:49.981 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_cache_meta_bytes": 0, 2024-08-19T01:43:49.981 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_cache_meta_items": 0, 2024-08-19T01:43:49.981 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_cache_other_bytes": 0, 2024-08-19T01:43:49.982 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_cache_other_items": 0, 2024-08-19T01:43:49.982 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_cache_buffer_bytes": 0, 2024-08-19T01:43:49.982 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_cache_buffer_items": 0, 2024-08-19T01:43:49.982 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_extent_bytes": 0, 2024-08-19T01:43:49.982 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_extent_items": 0, 2024-08-19T01:43:49.982 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_blob_bytes": 0, 2024-08-19T01:43:49.982 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_blob_items": 0, 2024-08-19T01:43:49.982 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_shared_blob_bytes": 0, 2024-08-19T01:43:49.982 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_shared_blob_items": 0, 2024-08-19T01:43:49.982 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_inline_bl_bytes": 0, 2024-08-19T01:43:49.982 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_inline_bl_items": 0, 2024-08-19T01:43:49.982 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_fsck_bytes": 0, 2024-08-19T01:43:49.982 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_fsck_items": 0, 2024-08-19T01:43:49.983 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_txc_bytes": 0, 2024-08-19T01:43:49.983 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_txc_items": 0, 2024-08-19T01:43:49.983 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_writing_deferred_bytes": 0, 2024-08-19T01:43:49.983 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_writing_deferred_items": 0, 2024-08-19T01:43:49.983 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_writing_bytes": 0, 2024-08-19T01:43:49.983 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_writing_items": 0, 2024-08-19T01:43:49.983 INFO:teuthology.orchestra.run.smithi167.stdout: "bluefs_bytes": 0, 2024-08-19T01:43:49.983 INFO:teuthology.orchestra.run.smithi167.stdout: "bluefs_items": 0, 2024-08-19T01:43:49.983 INFO:teuthology.orchestra.run.smithi167.stdout: "bluefs_file_reader_bytes": 0, 2024-08-19T01:43:49.983 INFO:teuthology.orchestra.run.smithi167.stdout: "bluefs_file_reader_items": 0, 2024-08-19T01:43:49.983 INFO:teuthology.orchestra.run.smithi167.stdout: "bluefs_file_writer_bytes": 0, 2024-08-19T01:43:49.983 INFO:teuthology.orchestra.run.smithi167.stdout: "bluefs_file_writer_items": 0, 2024-08-19T01:43:49.983 INFO:teuthology.orchestra.run.smithi167.stdout: "buffer_anon_bytes": 148967, 2024-08-19T01:43:49.983 INFO:teuthology.orchestra.run.smithi167.stdout: "buffer_anon_items": 29, 2024-08-19T01:43:49.984 INFO:teuthology.orchestra.run.smithi167.stdout: "buffer_meta_bytes": 0, 2024-08-19T01:43:49.984 INFO:teuthology.orchestra.run.smithi167.stdout: "buffer_meta_items": 0, 2024-08-19T01:43:49.984 INFO:teuthology.orchestra.run.smithi167.stdout: "osd_bytes": 0, 2024-08-19T01:43:49.984 INFO:teuthology.orchestra.run.smithi167.stdout: "osd_items": 0, 2024-08-19T01:43:49.984 INFO:teuthology.orchestra.run.smithi167.stdout: "osd_mapbl_bytes": 0, 2024-08-19T01:43:49.984 INFO:teuthology.orchestra.run.smithi167.stdout: "osd_mapbl_items": 0, 2024-08-19T01:43:49.984 INFO:teuthology.orchestra.run.smithi167.stdout: "osd_pglog_bytes": 0, 2024-08-19T01:43:49.984 INFO:teuthology.orchestra.run.smithi167.stdout: "osd_pglog_items": 0, 2024-08-19T01:43:49.984 INFO:teuthology.orchestra.run.smithi167.stdout: "osdmap_bytes": 0, 2024-08-19T01:43:49.984 INFO:teuthology.orchestra.run.smithi167.stdout: "osdmap_items": 0, 2024-08-19T01:43:49.984 INFO:teuthology.orchestra.run.smithi167.stdout: "osdmap_mapping_bytes": 0, 2024-08-19T01:43:49.984 INFO:teuthology.orchestra.run.smithi167.stdout: "osdmap_mapping_items": 0, 2024-08-19T01:43:49.984 INFO:teuthology.orchestra.run.smithi167.stdout: "pgmap_bytes": 0, 2024-08-19T01:43:49.984 INFO:teuthology.orchestra.run.smithi167.stdout: "pgmap_items": 0, 2024-08-19T01:43:49.985 INFO:teuthology.orchestra.run.smithi167.stdout: "mds_co_bytes": 928, 2024-08-19T01:43:49.985 INFO:teuthology.orchestra.run.smithi167.stdout: "mds_co_items": 2, 2024-08-19T01:43:49.985 INFO:teuthology.orchestra.run.smithi167.stdout: "unittest_1_bytes": 0, 2024-08-19T01:43:49.985 INFO:teuthology.orchestra.run.smithi167.stdout: "unittest_1_items": 0, 2024-08-19T01:43:49.985 INFO:teuthology.orchestra.run.smithi167.stdout: "unittest_2_bytes": 0, 2024-08-19T01:43:49.985 INFO:teuthology.orchestra.run.smithi167.stdout: "unittest_2_items": 0 2024-08-19T01:43:49.985 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-08-19T01:43:49.985 INFO:teuthology.orchestra.run.smithi167.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-08-19T01:43:49.985 INFO:teuthology.orchestra.run.smithi167.stdout: "val": 0, 2024-08-19T01:43:49.985 INFO:teuthology.orchestra.run.smithi167.stdout: "max": 104857600, 2024-08-19T01:43:49.985 INFO:teuthology.orchestra.run.smithi167.stdout: "get_started": 0, 2024-08-19T01:43:49.985 INFO:teuthology.orchestra.run.smithi167.stdout: "get": 194, 2024-08-19T01:43:49.985 INFO:teuthology.orchestra.run.smithi167.stdout: "get_sum": 718711, 2024-08-19T01:43:49.985 INFO:teuthology.orchestra.run.smithi167.stdout: "get_or_fail_fail": 0, 2024-08-19T01:43:49.986 INFO:teuthology.orchestra.run.smithi167.stdout: "get_or_fail_success": 194, 2024-08-19T01:43:49.986 INFO:teuthology.orchestra.run.smithi167.stdout: "take": 0, 2024-08-19T01:43:49.986 INFO:teuthology.orchestra.run.smithi167.stdout: "take_sum": 0, 2024-08-19T01:43:49.986 INFO:teuthology.orchestra.run.smithi167.stdout: "put": 194, 2024-08-19T01:43:49.986 INFO:teuthology.orchestra.run.smithi167.stdout: "put_sum": 718711, 2024-08-19T01:43:49.986 INFO:teuthology.orchestra.run.smithi167.stdout: "wait": { 2024-08-19T01:43:49.986 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-08-19T01:43:49.986 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-08-19T01:43:49.986 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-08-19T01:43:49.986 INFO:teuthology.orchestra.run.smithi167.stdout: } 2024-08-19T01:43:49.986 INFO:teuthology.orchestra.run.smithi167.stdout: } 2024-08-19T01:43:49.986 INFO:teuthology.orchestra.run.smithi167.stdout:} 2024-08-19T01:43:50.690 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.f 2024-08-19T01:43:50.690 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.f 2024-08-19T01:43:50.690 WARNING:tasks.check_counter:Counter 'mds.dir_split' not found on daemon mds.f 2024-08-19T01:43:50.690 DEBUG:tasks.check_counter:Getting stats from i 2024-08-19T01:43:50.691 DEBUG:teuthology.orchestra.run.smithi167:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833 shell --fsid bad8780e-5dc9-11ef-bccf-c7b262605968 -- ceph daemon mds.i perf dump 2024-08-19T01:43:50.941 INFO:teuthology.orchestra.run.smithi167.stderr:Inferring config /var/lib/ceph/bad8780e-5dc9-11ef-bccf-c7b262605968/mon.c/config 2024-08-19T01:43:50.952 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:43:50 smithi117 ceph-mon[22016]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-19T01:43:50.952 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:43:50 smithi117 ceph-mon[22016]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-19T01:43:50.952 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:43:50 smithi117 ceph-mon[22016]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-19T01:43:50.952 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:43:50 smithi117 ceph-mon[22016]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' 2024-08-19T01:43:50.965 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:43:50 smithi167 ceph-mon[21875]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-19T01:43:50.965 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:43:50 smithi167 ceph-mon[21875]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-19T01:43:50.966 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:43:50 smithi167 ceph-mon[21875]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-19T01:43:50.966 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:43:50 smithi167 ceph-mon[21875]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' 2024-08-19T01:43:51.154 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:43:50 smithi083 ceph-mon[17120]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-19T01:43:51.155 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:43:50 smithi083 ceph-mon[17120]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-19T01:43:51.155 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:43:50 smithi083 ceph-mon[17120]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-19T01:43:51.155 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:43:50 smithi083 ceph-mon[17120]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' 2024-08-19T01:43:51.429 INFO:teuthology.orchestra.run.smithi167.stdout:{ 2024-08-19T01:43:51.430 INFO:teuthology.orchestra.run.smithi167.stdout: "AsyncMessenger::Worker-0": { 2024-08-19T01:43:51.430 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_recv_messages": 1657, 2024-08-19T01:43:51.430 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_send_messages": 1659, 2024-08-19T01:43:51.430 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_recv_bytes": 265839105, 2024-08-19T01:43:51.430 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_send_bytes": 526936, 2024-08-19T01:43:51.430 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_created_connections": 8, 2024-08-19T01:43:51.430 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_active_connections": 4, 2024-08-19T01:43:51.430 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_running_total_time": 0.273213873, 2024-08-19T01:43:51.430 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_running_send_time": 0.045116486, 2024-08-19T01:43:51.430 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_running_recv_time": 0.168840387, 2024-08-19T01:43:51.430 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_running_fast_dispatch_time": 0.045262452, 2024-08-19T01:43:51.430 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_send_messages_queue_lat": { 2024-08-19T01:43:51.431 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 1657, 2024-08-19T01:43:51.431 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.059980664, 2024-08-19T01:43:51.431 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000036198 2024-08-19T01:43:51.431 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-08-19T01:43:51.431 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_handle_ack_lat": { 2024-08-19T01:43:51.431 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-08-19T01:43:51.431 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-08-19T01:43:51.431 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-08-19T01:43:51.431 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-08-19T01:43:51.431 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_recv_encrypted_bytes": 0, 2024-08-19T01:43:51.431 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_send_encrypted_bytes": 0 2024-08-19T01:43:51.431 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-08-19T01:43:51.431 INFO:teuthology.orchestra.run.smithi167.stdout: "AsyncMessenger::Worker-1": { 2024-08-19T01:43:51.432 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_recv_messages": 879, 2024-08-19T01:43:51.432 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_send_messages": 1018, 2024-08-19T01:43:51.432 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_recv_bytes": 293849743, 2024-08-19T01:43:51.432 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_send_bytes": 604254, 2024-08-19T01:43:51.432 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_created_connections": 7, 2024-08-19T01:43:51.432 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_active_connections": 6, 2024-08-19T01:43:51.432 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_running_total_time": 0.211258425, 2024-08-19T01:43:51.432 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_running_send_time": 0.024009049, 2024-08-19T01:43:51.432 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_running_recv_time": 0.160100462, 2024-08-19T01:43:51.432 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_running_fast_dispatch_time": 0.019897012, 2024-08-19T01:43:51.432 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_send_messages_queue_lat": { 2024-08-19T01:43:51.432 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 1017, 2024-08-19T01:43:51.432 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.037014403, 2024-08-19T01:43:51.432 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000036395 2024-08-19T01:43:51.433 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-08-19T01:43:51.433 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_handle_ack_lat": { 2024-08-19T01:43:51.433 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-08-19T01:43:51.433 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-08-19T01:43:51.433 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-08-19T01:43:51.433 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-08-19T01:43:51.433 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_recv_encrypted_bytes": 3184, 2024-08-19T01:43:51.433 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_send_encrypted_bytes": 326256 2024-08-19T01:43:51.433 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-08-19T01:43:51.433 INFO:teuthology.orchestra.run.smithi167.stdout: "AsyncMessenger::Worker-2": { 2024-08-19T01:43:51.433 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_recv_messages": 915, 2024-08-19T01:43:51.434 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_send_messages": 901, 2024-08-19T01:43:51.434 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_recv_bytes": 172969673, 2024-08-19T01:43:51.434 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_send_bytes": 340299, 2024-08-19T01:43:51.434 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_created_connections": 5, 2024-08-19T01:43:51.434 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_active_connections": 4, 2024-08-19T01:43:51.434 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_running_total_time": 0.162984700, 2024-08-19T01:43:51.434 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_running_send_time": 0.031941520, 2024-08-19T01:43:51.434 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_running_recv_time": 0.114889628, 2024-08-19T01:43:51.434 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_running_fast_dispatch_time": 0.017180156, 2024-08-19T01:43:51.434 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_send_messages_queue_lat": { 2024-08-19T01:43:51.434 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 900, 2024-08-19T01:43:51.435 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.032431664, 2024-08-19T01:43:51.435 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000036035 2024-08-19T01:43:51.435 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-08-19T01:43:51.435 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_handle_ack_lat": { 2024-08-19T01:43:51.435 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-08-19T01:43:51.435 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-08-19T01:43:51.435 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-08-19T01:43:51.435 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-08-19T01:43:51.435 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_recv_encrypted_bytes": 794528, 2024-08-19T01:43:51.435 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_send_encrypted_bytes": 115472 2024-08-19T01:43:51.435 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-08-19T01:43:51.435 INFO:teuthology.orchestra.run.smithi167.stdout: "cct": { 2024-08-19T01:43:51.435 INFO:teuthology.orchestra.run.smithi167.stdout: "total_workers": 1, 2024-08-19T01:43:51.436 INFO:teuthology.orchestra.run.smithi167.stdout: "unhealthy_workers": 0 2024-08-19T01:43:51.436 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-08-19T01:43:51.436 INFO:teuthology.orchestra.run.smithi167.stdout: "finisher-MDSRank": { 2024-08-19T01:43:51.436 INFO:teuthology.orchestra.run.smithi167.stdout: "queue_len": 0, 2024-08-19T01:43:51.436 INFO:teuthology.orchestra.run.smithi167.stdout: "complete_latency": { 2024-08-19T01:43:51.436 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 4832, 2024-08-19T01:43:51.436 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.980030218, 2024-08-19T01:43:51.436 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000202820 2024-08-19T01:43:51.436 INFO:teuthology.orchestra.run.smithi167.stdout: } 2024-08-19T01:43:51.436 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-08-19T01:43:51.436 INFO:teuthology.orchestra.run.smithi167.stdout: "finisher-PurgeQueue": { 2024-08-19T01:43:51.436 INFO:teuthology.orchestra.run.smithi167.stdout: "queue_len": 0, 2024-08-19T01:43:51.436 INFO:teuthology.orchestra.run.smithi167.stdout: "complete_latency": { 2024-08-19T01:43:51.436 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-08-19T01:43:51.437 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-08-19T01:43:51.437 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-08-19T01:43:51.437 INFO:teuthology.orchestra.run.smithi167.stdout: } 2024-08-19T01:43:51.437 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-08-19T01:43:51.437 INFO:teuthology.orchestra.run.smithi167.stdout: "mds": { 2024-08-19T01:43:51.437 INFO:teuthology.orchestra.run.smithi167.stdout: "request": 0, 2024-08-19T01:43:51.437 INFO:teuthology.orchestra.run.smithi167.stdout: "reply": 0, 2024-08-19T01:43:51.437 INFO:teuthology.orchestra.run.smithi167.stdout: "reply_latency": { 2024-08-19T01:43:51.437 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-08-19T01:43:51.437 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-08-19T01:43:51.437 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-08-19T01:43:51.437 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-08-19T01:43:51.437 INFO:teuthology.orchestra.run.smithi167.stdout: "slow_reply": 0, 2024-08-19T01:43:51.437 INFO:teuthology.orchestra.run.smithi167.stdout: "forward": 0, 2024-08-19T01:43:51.438 INFO:teuthology.orchestra.run.smithi167.stdout: "dir_fetch_complete": 0, 2024-08-19T01:43:51.438 INFO:teuthology.orchestra.run.smithi167.stdout: "dir_fetch_keys": 0, 2024-08-19T01:43:51.438 INFO:teuthology.orchestra.run.smithi167.stdout: "dir_commit": 0, 2024-08-19T01:43:51.438 INFO:teuthology.orchestra.run.smithi167.stdout: "dir_split": 0, 2024-08-19T01:43:51.438 INFO:teuthology.orchestra.run.smithi167.stdout: "dir_merge": 0, 2024-08-19T01:43:51.438 INFO:teuthology.orchestra.run.smithi167.stdout: "inodes": 61453, 2024-08-19T01:43:51.438 INFO:teuthology.orchestra.run.smithi167.stdout: "inodes_top": 41092, 2024-08-19T01:43:51.438 INFO:teuthology.orchestra.run.smithi167.stdout: "inodes_bottom": 20361, 2024-08-19T01:43:51.438 INFO:teuthology.orchestra.run.smithi167.stdout: "inodes_pin_tail": 0, 2024-08-19T01:43:51.438 INFO:teuthology.orchestra.run.smithi167.stdout: "inodes_pinned": 2749, 2024-08-19T01:43:51.438 INFO:teuthology.orchestra.run.smithi167.stdout: "inodes_expired": 0, 2024-08-19T01:43:51.438 INFO:teuthology.orchestra.run.smithi167.stdout: "inodes_with_caps": 0, 2024-08-19T01:43:51.438 INFO:teuthology.orchestra.run.smithi167.stdout: "caps": 0, 2024-08-19T01:43:51.439 INFO:teuthology.orchestra.run.smithi167.stdout: "subtrees": 2, 2024-08-19T01:43:51.439 INFO:teuthology.orchestra.run.smithi167.stdout: "traverse": 0, 2024-08-19T01:43:51.439 INFO:teuthology.orchestra.run.smithi167.stdout: "traverse_hit": 0, 2024-08-19T01:43:51.439 INFO:teuthology.orchestra.run.smithi167.stdout: "traverse_forward": 0, 2024-08-19T01:43:51.439 INFO:teuthology.orchestra.run.smithi167.stdout: "traverse_discover": 0, 2024-08-19T01:43:51.439 INFO:teuthology.orchestra.run.smithi167.stdout: "traverse_dir_fetch": 0, 2024-08-19T01:43:51.439 INFO:teuthology.orchestra.run.smithi167.stdout: "traverse_remote_ino": 0, 2024-08-19T01:43:51.439 INFO:teuthology.orchestra.run.smithi167.stdout: "traverse_lock": 0, 2024-08-19T01:43:51.439 INFO:teuthology.orchestra.run.smithi167.stdout: "load_cent": 0, 2024-08-19T01:43:51.439 INFO:teuthology.orchestra.run.smithi167.stdout: "q": 0, 2024-08-19T01:43:51.439 INFO:teuthology.orchestra.run.smithi167.stdout: "exported": 0, 2024-08-19T01:43:51.439 INFO:teuthology.orchestra.run.smithi167.stdout: "exported_inodes": 0, 2024-08-19T01:43:51.439 INFO:teuthology.orchestra.run.smithi167.stdout: "imported": 0, 2024-08-19T01:43:51.439 INFO:teuthology.orchestra.run.smithi167.stdout: "imported_inodes": 0, 2024-08-19T01:43:51.440 INFO:teuthology.orchestra.run.smithi167.stdout: "openino_dir_fetch": 0, 2024-08-19T01:43:51.440 INFO:teuthology.orchestra.run.smithi167.stdout: "openino_backtrace_fetch": 0, 2024-08-19T01:43:51.440 INFO:teuthology.orchestra.run.smithi167.stdout: "openino_peer_discover": 0, 2024-08-19T01:43:51.440 INFO:teuthology.orchestra.run.smithi167.stdout: "root_rfiles": 4, 2024-08-19T01:43:51.440 INFO:teuthology.orchestra.run.smithi167.stdout: "root_rbytes": 234, 2024-08-19T01:43:51.440 INFO:teuthology.orchestra.run.smithi167.stdout: "root_rsnaps": 0, 2024-08-19T01:43:51.440 INFO:teuthology.orchestra.run.smithi167.stdout: "scrub_backtrace_fetch": 0, 2024-08-19T01:43:51.440 INFO:teuthology.orchestra.run.smithi167.stdout: "scrub_set_tag": 0, 2024-08-19T01:43:51.440 INFO:teuthology.orchestra.run.smithi167.stdout: "scrub_backtrace_repaired": 0, 2024-08-19T01:43:51.440 INFO:teuthology.orchestra.run.smithi167.stdout: "scrub_inotable_repaired": 0, 2024-08-19T01:43:51.440 INFO:teuthology.orchestra.run.smithi167.stdout: "scrub_dir_inodes": 0, 2024-08-19T01:43:51.440 INFO:teuthology.orchestra.run.smithi167.stdout: "scrub_dir_base_inodes": 0, 2024-08-19T01:43:51.440 INFO:teuthology.orchestra.run.smithi167.stdout: "scrub_dirfrag_rstats": 0, 2024-08-19T01:43:51.441 INFO:teuthology.orchestra.run.smithi167.stdout: "scrub_file_inodes": 0, 2024-08-19T01:43:51.441 INFO:teuthology.orchestra.run.smithi167.stdout: "handle_inode_file_caps": 0, 2024-08-19T01:43:51.441 INFO:teuthology.orchestra.run.smithi167.stdout: "ceph_cap_op_revoke": 0, 2024-08-19T01:43:51.441 INFO:teuthology.orchestra.run.smithi167.stdout: "ceph_cap_op_grant": 0, 2024-08-19T01:43:51.441 INFO:teuthology.orchestra.run.smithi167.stdout: "ceph_cap_op_trunc": 0, 2024-08-19T01:43:51.441 INFO:teuthology.orchestra.run.smithi167.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-08-19T01:43:51.441 INFO:teuthology.orchestra.run.smithi167.stdout: "ceph_cap_op_flush_ack": 0, 2024-08-19T01:43:51.441 INFO:teuthology.orchestra.run.smithi167.stdout: "handle_client_caps": 0, 2024-08-19T01:43:51.441 INFO:teuthology.orchestra.run.smithi167.stdout: "handle_client_caps_dirty": 0, 2024-08-19T01:43:51.441 INFO:teuthology.orchestra.run.smithi167.stdout: "handle_client_cap_release": 0, 2024-08-19T01:43:51.441 INFO:teuthology.orchestra.run.smithi167.stdout: "process_request_cap_release": 0 2024-08-19T01:43:51.441 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-08-19T01:43:51.441 INFO:teuthology.orchestra.run.smithi167.stdout: "mds_cache": { 2024-08-19T01:43:51.441 INFO:teuthology.orchestra.run.smithi167.stdout: "dir_update": 0, 2024-08-19T01:43:51.442 INFO:teuthology.orchestra.run.smithi167.stdout: "dir_update_receipt": 0, 2024-08-19T01:43:51.442 INFO:teuthology.orchestra.run.smithi167.stdout: "dir_try_discover": 0, 2024-08-19T01:43:51.442 INFO:teuthology.orchestra.run.smithi167.stdout: "dir_send_discover": 0, 2024-08-19T01:43:51.442 INFO:teuthology.orchestra.run.smithi167.stdout: "dir_handle_discover": 0, 2024-08-19T01:43:51.442 INFO:teuthology.orchestra.run.smithi167.stdout: "num_strays": 0, 2024-08-19T01:43:51.442 INFO:teuthology.orchestra.run.smithi167.stdout: "num_strays_delayed": 0, 2024-08-19T01:43:51.442 INFO:teuthology.orchestra.run.smithi167.stdout: "num_strays_enqueuing": 0, 2024-08-19T01:43:51.442 INFO:teuthology.orchestra.run.smithi167.stdout: "strays_created": 0, 2024-08-19T01:43:51.442 INFO:teuthology.orchestra.run.smithi167.stdout: "strays_enqueued": 0, 2024-08-19T01:43:51.442 INFO:teuthology.orchestra.run.smithi167.stdout: "strays_reintegrated": 0, 2024-08-19T01:43:51.442 INFO:teuthology.orchestra.run.smithi167.stdout: "strays_migrated": 0, 2024-08-19T01:43:51.442 INFO:teuthology.orchestra.run.smithi167.stdout: "num_recovering_processing": 0, 2024-08-19T01:43:51.442 INFO:teuthology.orchestra.run.smithi167.stdout: "num_recovering_enqueued": 0, 2024-08-19T01:43:51.443 INFO:teuthology.orchestra.run.smithi167.stdout: "num_recovering_prioritized": 0, 2024-08-19T01:43:51.443 INFO:teuthology.orchestra.run.smithi167.stdout: "recovery_started": 0, 2024-08-19T01:43:51.443 INFO:teuthology.orchestra.run.smithi167.stdout: "recovery_completed": 0, 2024-08-19T01:43:51.443 INFO:teuthology.orchestra.run.smithi167.stdout: "ireq_quiesce_path": 0, 2024-08-19T01:43:51.443 INFO:teuthology.orchestra.run.smithi167.stdout: "ireq_quiesce_inode": 0, 2024-08-19T01:43:51.443 INFO:teuthology.orchestra.run.smithi167.stdout: "ireq_enqueue_scrub": 0, 2024-08-19T01:43:51.443 INFO:teuthology.orchestra.run.smithi167.stdout: "ireq_exportdir": 0, 2024-08-19T01:43:51.443 INFO:teuthology.orchestra.run.smithi167.stdout: "ireq_flush": 0, 2024-08-19T01:43:51.443 INFO:teuthology.orchestra.run.smithi167.stdout: "ireq_fragmentdir": 0, 2024-08-19T01:43:51.443 INFO:teuthology.orchestra.run.smithi167.stdout: "ireq_fragstats": 0, 2024-08-19T01:43:51.443 INFO:teuthology.orchestra.run.smithi167.stdout: "ireq_inodestats": 0 2024-08-19T01:43:51.443 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-08-19T01:43:51.443 INFO:teuthology.orchestra.run.smithi167.stdout: "mds_log": { 2024-08-19T01:43:51.443 INFO:teuthology.orchestra.run.smithi167.stdout: "evlrg": 0, 2024-08-19T01:43:51.444 INFO:teuthology.orchestra.run.smithi167.stdout: "evadd": 0, 2024-08-19T01:43:51.444 INFO:teuthology.orchestra.run.smithi167.stdout: "evex": 0, 2024-08-19T01:43:51.444 INFO:teuthology.orchestra.run.smithi167.stdout: "evtrm": 0, 2024-08-19T01:43:51.444 INFO:teuthology.orchestra.run.smithi167.stdout: "ev": 323890, 2024-08-19T01:43:51.444 INFO:teuthology.orchestra.run.smithi167.stdout: "evexg": 0, 2024-08-19T01:43:51.444 INFO:teuthology.orchestra.run.smithi167.stdout: "evexd": 0, 2024-08-19T01:43:51.444 INFO:teuthology.orchestra.run.smithi167.stdout: "segadd": 0, 2024-08-19T01:43:51.444 INFO:teuthology.orchestra.run.smithi167.stdout: "segex": 0, 2024-08-19T01:43:51.444 INFO:teuthology.orchestra.run.smithi167.stdout: "segtrm": 0, 2024-08-19T01:43:51.444 INFO:teuthology.orchestra.run.smithi167.stdout: "seg": 9, 2024-08-19T01:43:51.444 INFO:teuthology.orchestra.run.smithi167.stdout: "segmjr": 46, 2024-08-19T01:43:51.444 INFO:teuthology.orchestra.run.smithi167.stdout: "segexg": 0, 2024-08-19T01:43:51.444 INFO:teuthology.orchestra.run.smithi167.stdout: "segexd": 0, 2024-08-19T01:43:51.444 INFO:teuthology.orchestra.run.smithi167.stdout: "expos": 722879719, 2024-08-19T01:43:51.444 INFO:teuthology.orchestra.run.smithi167.stdout: "wrpos": 735098979, 2024-08-19T01:43:51.445 INFO:teuthology.orchestra.run.smithi167.stdout: "rdpos": 735097328, 2024-08-19T01:43:51.445 INFO:teuthology.orchestra.run.smithi167.stdout: "jlat": { 2024-08-19T01:43:51.445 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-08-19T01:43:51.445 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-08-19T01:43:51.445 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-08-19T01:43:51.445 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-08-19T01:43:51.445 INFO:teuthology.orchestra.run.smithi167.stdout: "replayed": 323890 2024-08-19T01:43:51.445 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-08-19T01:43:51.445 INFO:teuthology.orchestra.run.smithi167.stdout: "mds_mem": { 2024-08-19T01:43:51.445 INFO:teuthology.orchestra.run.smithi167.stdout: "ino": 26, 2024-08-19T01:43:51.445 INFO:teuthology.orchestra.run.smithi167.stdout: "ino+": 61813, 2024-08-19T01:43:51.445 INFO:teuthology.orchestra.run.smithi167.stdout: "ino-": 61787, 2024-08-19T01:43:51.445 INFO:teuthology.orchestra.run.smithi167.stdout: "dir": 28, 2024-08-19T01:43:51.446 INFO:teuthology.orchestra.run.smithi167.stdout: "dir+": 1962, 2024-08-19T01:43:51.446 INFO:teuthology.orchestra.run.smithi167.stdout: "dir-": 1934, 2024-08-19T01:43:51.446 INFO:teuthology.orchestra.run.smithi167.stdout: "dn": 61453, 2024-08-19T01:43:51.446 INFO:teuthology.orchestra.run.smithi167.stdout: "dn+": 132847, 2024-08-19T01:43:51.446 INFO:teuthology.orchestra.run.smithi167.stdout: "dn-": 71394, 2024-08-19T01:43:51.446 INFO:teuthology.orchestra.run.smithi167.stdout: "cap": 0, 2024-08-19T01:43:51.446 INFO:teuthology.orchestra.run.smithi167.stdout: "cap+": 0, 2024-08-19T01:43:51.446 INFO:teuthology.orchestra.run.smithi167.stdout: "cap-": 0, 2024-08-19T01:43:51.446 INFO:teuthology.orchestra.run.smithi167.stdout: "rss": 161568, 2024-08-19T01:43:51.446 INFO:teuthology.orchestra.run.smithi167.stdout: "heap": 215328 2024-08-19T01:43:51.446 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-08-19T01:43:51.446 INFO:teuthology.orchestra.run.smithi167.stdout: "mds_server": { 2024-08-19T01:43:51.446 INFO:teuthology.orchestra.run.smithi167.stdout: "dispatch_client_request": 0, 2024-08-19T01:43:51.446 INFO:teuthology.orchestra.run.smithi167.stdout: "dispatch_server_request": 0, 2024-08-19T01:43:51.447 INFO:teuthology.orchestra.run.smithi167.stdout: "handle_client_request": 0, 2024-08-19T01:43:51.447 INFO:teuthology.orchestra.run.smithi167.stdout: "handle_client_session": 0, 2024-08-19T01:43:51.447 INFO:teuthology.orchestra.run.smithi167.stdout: "handle_peer_request": 0, 2024-08-19T01:43:51.447 INFO:teuthology.orchestra.run.smithi167.stdout: "req_create_latency": { 2024-08-19T01:43:51.447 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-08-19T01:43:51.447 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-08-19T01:43:51.447 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-08-19T01:43:51.447 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-08-19T01:43:51.447 INFO:teuthology.orchestra.run.smithi167.stdout: "req_getattr_latency": { 2024-08-19T01:43:51.447 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-08-19T01:43:51.447 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-08-19T01:43:51.447 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-08-19T01:43:51.447 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-08-19T01:43:51.447 INFO:teuthology.orchestra.run.smithi167.stdout: "req_getfilelock_latency": { 2024-08-19T01:43:51.448 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-08-19T01:43:51.448 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-08-19T01:43:51.448 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-08-19T01:43:51.448 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-08-19T01:43:51.448 INFO:teuthology.orchestra.run.smithi167.stdout: "req_link_latency": { 2024-08-19T01:43:51.448 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-08-19T01:43:51.448 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-08-19T01:43:51.448 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-08-19T01:43:51.448 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-08-19T01:43:51.448 INFO:teuthology.orchestra.run.smithi167.stdout: "req_lookup_latency": { 2024-08-19T01:43:51.448 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-08-19T01:43:51.448 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-08-19T01:43:51.448 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-08-19T01:43:51.449 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-08-19T01:43:51.449 INFO:teuthology.orchestra.run.smithi167.stdout: "req_lookuphash_latency": { 2024-08-19T01:43:51.449 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-08-19T01:43:51.449 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-08-19T01:43:51.449 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-08-19T01:43:51.449 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-08-19T01:43:51.449 INFO:teuthology.orchestra.run.smithi167.stdout: "req_lookupino_latency": { 2024-08-19T01:43:51.449 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-08-19T01:43:51.449 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-08-19T01:43:51.449 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-08-19T01:43:51.449 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-08-19T01:43:51.449 INFO:teuthology.orchestra.run.smithi167.stdout: "req_lookupname_latency": { 2024-08-19T01:43:51.449 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-08-19T01:43:51.449 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-08-19T01:43:51.450 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-08-19T01:43:51.450 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-08-19T01:43:51.450 INFO:teuthology.orchestra.run.smithi167.stdout: "req_lookupparent_latency": { 2024-08-19T01:43:51.450 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-08-19T01:43:51.450 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-08-19T01:43:51.450 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-08-19T01:43:51.450 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-08-19T01:43:51.450 INFO:teuthology.orchestra.run.smithi167.stdout: "req_lookupsnap_latency": { 2024-08-19T01:43:51.450 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-08-19T01:43:51.450 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-08-19T01:43:51.450 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-08-19T01:43:51.450 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-08-19T01:43:51.450 INFO:teuthology.orchestra.run.smithi167.stdout: "req_lssnap_latency": { 2024-08-19T01:43:51.451 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-08-19T01:43:51.451 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-08-19T01:43:51.451 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-08-19T01:43:51.451 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-08-19T01:43:51.451 INFO:teuthology.orchestra.run.smithi167.stdout: "req_mkdir_latency": { 2024-08-19T01:43:51.451 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-08-19T01:43:51.451 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-08-19T01:43:51.451 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-08-19T01:43:51.451 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-08-19T01:43:51.451 INFO:teuthology.orchestra.run.smithi167.stdout: "req_mknod_latency": { 2024-08-19T01:43:51.451 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-08-19T01:43:51.451 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-08-19T01:43:51.451 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-08-19T01:43:51.451 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-08-19T01:43:51.452 INFO:teuthology.orchestra.run.smithi167.stdout: "req_mksnap_latency": { 2024-08-19T01:43:51.452 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-08-19T01:43:51.452 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-08-19T01:43:51.452 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-08-19T01:43:51.452 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-08-19T01:43:51.452 INFO:teuthology.orchestra.run.smithi167.stdout: "req_open_latency": { 2024-08-19T01:43:51.452 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-08-19T01:43:51.452 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-08-19T01:43:51.452 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-08-19T01:43:51.452 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-08-19T01:43:51.452 INFO:teuthology.orchestra.run.smithi167.stdout: "req_readdir_latency": { 2024-08-19T01:43:51.452 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-08-19T01:43:51.452 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-08-19T01:43:51.452 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-08-19T01:43:51.453 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-08-19T01:43:51.453 INFO:teuthology.orchestra.run.smithi167.stdout: "req_rename_latency": { 2024-08-19T01:43:51.453 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-08-19T01:43:51.453 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-08-19T01:43:51.453 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-08-19T01:43:51.453 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-08-19T01:43:51.453 INFO:teuthology.orchestra.run.smithi167.stdout: "req_renamesnap_latency": { 2024-08-19T01:43:51.453 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-08-19T01:43:51.453 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-08-19T01:43:51.453 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-08-19T01:43:51.453 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-08-19T01:43:51.453 INFO:teuthology.orchestra.run.smithi167.stdout: "req_snapdiff_latency": { 2024-08-19T01:43:51.453 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-08-19T01:43:51.454 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-08-19T01:43:51.454 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-08-19T01:43:51.454 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-08-19T01:43:51.454 INFO:teuthology.orchestra.run.smithi167.stdout: "req_rmdir_latency": { 2024-08-19T01:43:51.454 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-08-19T01:43:51.454 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-08-19T01:43:51.454 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-08-19T01:43:51.454 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-08-19T01:43:51.454 INFO:teuthology.orchestra.run.smithi167.stdout: "req_rmsnap_latency": { 2024-08-19T01:43:51.454 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-08-19T01:43:51.454 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-08-19T01:43:51.454 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-08-19T01:43:51.454 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-08-19T01:43:51.455 INFO:teuthology.orchestra.run.smithi167.stdout: "req_rmxattr_latency": { 2024-08-19T01:43:51.455 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-08-19T01:43:51.455 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-08-19T01:43:51.455 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-08-19T01:43:51.455 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-08-19T01:43:51.455 INFO:teuthology.orchestra.run.smithi167.stdout: "req_setattr_latency": { 2024-08-19T01:43:51.455 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-08-19T01:43:51.455 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-08-19T01:43:51.455 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-08-19T01:43:51.455 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-08-19T01:43:51.455 INFO:teuthology.orchestra.run.smithi167.stdout: "req_setdirlayout_latency": { 2024-08-19T01:43:51.455 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-08-19T01:43:51.455 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-08-19T01:43:51.456 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-08-19T01:43:51.456 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-08-19T01:43:51.456 INFO:teuthology.orchestra.run.smithi167.stdout: "req_setfilelock_latency": { 2024-08-19T01:43:51.456 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-08-19T01:43:51.456 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-08-19T01:43:51.456 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-08-19T01:43:51.456 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-08-19T01:43:51.456 INFO:teuthology.orchestra.run.smithi167.stdout: "req_setlayout_latency": { 2024-08-19T01:43:51.456 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-08-19T01:43:51.456 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-08-19T01:43:51.456 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-08-19T01:43:51.456 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-08-19T01:43:51.456 INFO:teuthology.orchestra.run.smithi167.stdout: "req_setxattr_latency": { 2024-08-19T01:43:51.457 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-08-19T01:43:51.457 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-08-19T01:43:51.457 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-08-19T01:43:51.457 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-08-19T01:43:51.457 INFO:teuthology.orchestra.run.smithi167.stdout: "req_symlink_latency": { 2024-08-19T01:43:51.457 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-08-19T01:43:51.457 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-08-19T01:43:51.457 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-08-19T01:43:51.457 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-08-19T01:43:51.457 INFO:teuthology.orchestra.run.smithi167.stdout: "req_unlink_latency": { 2024-08-19T01:43:51.457 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-08-19T01:43:51.457 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-08-19T01:43:51.457 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-08-19T01:43:51.457 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-08-19T01:43:51.458 INFO:teuthology.orchestra.run.smithi167.stdout: "cap_revoke_eviction": 0, 2024-08-19T01:43:51.458 INFO:teuthology.orchestra.run.smithi167.stdout: "cap_acquisition_throttle": 0, 2024-08-19T01:43:51.458 INFO:teuthology.orchestra.run.smithi167.stdout: "req_getvxattr_latency": { 2024-08-19T01:43:51.458 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-08-19T01:43:51.458 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-08-19T01:43:51.458 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-08-19T01:43:51.458 INFO:teuthology.orchestra.run.smithi167.stdout: } 2024-08-19T01:43:51.458 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-08-19T01:43:51.458 INFO:teuthology.orchestra.run.smithi167.stdout: "mds_sessions": { 2024-08-19T01:43:51.458 INFO:teuthology.orchestra.run.smithi167.stdout: "session_count": 1, 2024-08-19T01:43:51.458 INFO:teuthology.orchestra.run.smithi167.stdout: "session_add": 3, 2024-08-19T01:43:51.458 INFO:teuthology.orchestra.run.smithi167.stdout: "session_remove": 2, 2024-08-19T01:43:51.459 INFO:teuthology.orchestra.run.smithi167.stdout: "sessions_open": 2, 2024-08-19T01:43:51.459 INFO:teuthology.orchestra.run.smithi167.stdout: "sessions_stale": 0, 2024-08-19T01:43:51.459 INFO:teuthology.orchestra.run.smithi167.stdout: "total_load": 0, 2024-08-19T01:43:51.459 INFO:teuthology.orchestra.run.smithi167.stdout: "average_load": 0, 2024-08-19T01:43:51.459 INFO:teuthology.orchestra.run.smithi167.stdout: "avg_session_uptime": 2982, 2024-08-19T01:43:51.459 INFO:teuthology.orchestra.run.smithi167.stdout: "mdthresh_evicted": 0 2024-08-19T01:43:51.459 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-08-19T01:43:51.459 INFO:teuthology.orchestra.run.smithi167.stdout: "mempool": { 2024-08-19T01:43:51.459 INFO:teuthology.orchestra.run.smithi167.stdout: "bloom_filter_bytes": 0, 2024-08-19T01:43:51.459 INFO:teuthology.orchestra.run.smithi167.stdout: "bloom_filter_items": 0, 2024-08-19T01:43:51.459 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_alloc_bytes": 0, 2024-08-19T01:43:51.459 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_alloc_items": 0, 2024-08-19T01:43:51.459 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_cache_data_bytes": 0, 2024-08-19T01:43:51.459 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_cache_data_items": 0, 2024-08-19T01:43:51.460 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_cache_onode_bytes": 0, 2024-08-19T01:43:51.460 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_cache_onode_items": 0, 2024-08-19T01:43:51.460 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_cache_meta_bytes": 0, 2024-08-19T01:43:51.460 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_cache_meta_items": 0, 2024-08-19T01:43:51.460 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_cache_other_bytes": 0, 2024-08-19T01:43:51.460 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_cache_other_items": 0, 2024-08-19T01:43:51.460 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_cache_buffer_bytes": 0, 2024-08-19T01:43:51.460 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_cache_buffer_items": 0, 2024-08-19T01:43:51.460 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_extent_bytes": 0, 2024-08-19T01:43:51.460 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_extent_items": 0, 2024-08-19T01:43:51.460 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_blob_bytes": 0, 2024-08-19T01:43:51.460 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_blob_items": 0, 2024-08-19T01:43:51.460 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_shared_blob_bytes": 0, 2024-08-19T01:43:51.460 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_shared_blob_items": 0, 2024-08-19T01:43:51.460 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_inline_bl_bytes": 0, 2024-08-19T01:43:51.461 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_inline_bl_items": 0, 2024-08-19T01:43:51.461 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_fsck_bytes": 0, 2024-08-19T01:43:51.461 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_fsck_items": 0, 2024-08-19T01:43:51.461 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_txc_bytes": 0, 2024-08-19T01:43:51.461 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_txc_items": 0, 2024-08-19T01:43:51.461 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_writing_deferred_bytes": 0, 2024-08-19T01:43:51.461 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_writing_deferred_items": 0, 2024-08-19T01:43:51.461 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_writing_bytes": 0, 2024-08-19T01:43:51.461 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_writing_items": 0, 2024-08-19T01:43:51.461 INFO:teuthology.orchestra.run.smithi167.stdout: "bluefs_bytes": 0, 2024-08-19T01:43:51.461 INFO:teuthology.orchestra.run.smithi167.stdout: "bluefs_items": 0, 2024-08-19T01:43:51.461 INFO:teuthology.orchestra.run.smithi167.stdout: "bluefs_file_reader_bytes": 0, 2024-08-19T01:43:51.461 INFO:teuthology.orchestra.run.smithi167.stdout: "bluefs_file_reader_items": 0, 2024-08-19T01:43:51.461 INFO:teuthology.orchestra.run.smithi167.stdout: "bluefs_file_writer_bytes": 0, 2024-08-19T01:43:51.462 INFO:teuthology.orchestra.run.smithi167.stdout: "bluefs_file_writer_items": 0, 2024-08-19T01:43:51.462 INFO:teuthology.orchestra.run.smithi167.stdout: "buffer_anon_bytes": 238798, 2024-08-19T01:43:51.462 INFO:teuthology.orchestra.run.smithi167.stdout: "buffer_anon_items": 80, 2024-08-19T01:43:51.462 INFO:teuthology.orchestra.run.smithi167.stdout: "buffer_meta_bytes": 0, 2024-08-19T01:43:51.462 INFO:teuthology.orchestra.run.smithi167.stdout: "buffer_meta_items": 0, 2024-08-19T01:43:51.462 INFO:teuthology.orchestra.run.smithi167.stdout: "osd_bytes": 0, 2024-08-19T01:43:51.462 INFO:teuthology.orchestra.run.smithi167.stdout: "osd_items": 0, 2024-08-19T01:43:51.462 INFO:teuthology.orchestra.run.smithi167.stdout: "osd_mapbl_bytes": 0, 2024-08-19T01:43:51.462 INFO:teuthology.orchestra.run.smithi167.stdout: "osd_mapbl_items": 0, 2024-08-19T01:43:51.462 INFO:teuthology.orchestra.run.smithi167.stdout: "osd_pglog_bytes": 0, 2024-08-19T01:43:51.462 INFO:teuthology.orchestra.run.smithi167.stdout: "osd_pglog_items": 0, 2024-08-19T01:43:51.462 INFO:teuthology.orchestra.run.smithi167.stdout: "osdmap_bytes": 7688, 2024-08-19T01:43:51.463 INFO:teuthology.orchestra.run.smithi167.stdout: "osdmap_items": 158, 2024-08-19T01:43:51.463 INFO:teuthology.orchestra.run.smithi167.stdout: "osdmap_mapping_bytes": 0, 2024-08-19T01:43:51.463 INFO:teuthology.orchestra.run.smithi167.stdout: "osdmap_mapping_items": 0, 2024-08-19T01:43:51.463 INFO:teuthology.orchestra.run.smithi167.stdout: "pgmap_bytes": 0, 2024-08-19T01:43:51.463 INFO:teuthology.orchestra.run.smithi167.stdout: "pgmap_items": 0, 2024-08-19T01:43:51.463 INFO:teuthology.orchestra.run.smithi167.stdout: "mds_co_bytes": 43358068, 2024-08-19T01:43:51.463 INFO:teuthology.orchestra.run.smithi167.stdout: "mds_co_items": 247596, 2024-08-19T01:43:51.463 INFO:teuthology.orchestra.run.smithi167.stdout: "unittest_1_bytes": 0, 2024-08-19T01:43:51.463 INFO:teuthology.orchestra.run.smithi167.stdout: "unittest_1_items": 0, 2024-08-19T01:43:51.463 INFO:teuthology.orchestra.run.smithi167.stdout: "unittest_2_bytes": 0, 2024-08-19T01:43:51.463 INFO:teuthology.orchestra.run.smithi167.stdout: "unittest_2_items": 0 2024-08-19T01:43:51.463 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-08-19T01:43:51.463 INFO:teuthology.orchestra.run.smithi167.stdout: "objecter": { 2024-08-19T01:43:51.463 INFO:teuthology.orchestra.run.smithi167.stdout: "op_active": 0, 2024-08-19T01:43:51.463 INFO:teuthology.orchestra.run.smithi167.stdout: "op_laggy": 0, 2024-08-19T01:43:51.464 INFO:teuthology.orchestra.run.smithi167.stdout: "op_send": 3238, 2024-08-19T01:43:51.464 INFO:teuthology.orchestra.run.smithi167.stdout: "op_send_bytes": 26, 2024-08-19T01:43:51.464 INFO:teuthology.orchestra.run.smithi167.stdout: "op_resend": 4, 2024-08-19T01:43:51.464 INFO:teuthology.orchestra.run.smithi167.stdout: "op_reply": 3238, 2024-08-19T01:43:51.464 INFO:teuthology.orchestra.run.smithi167.stdout: "op_latency": { 2024-08-19T01:43:51.464 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 3238, 2024-08-19T01:43:51.464 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 12.372782454, 2024-08-19T01:43:51.464 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.003821118 2024-08-19T01:43:51.464 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-08-19T01:43:51.464 INFO:teuthology.orchestra.run.smithi167.stdout: "op_inflight": 0, 2024-08-19T01:43:51.464 INFO:teuthology.orchestra.run.smithi167.stdout: "oplen_avg": { 2024-08-19T01:43:51.464 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 3238, 2024-08-19T01:43:51.464 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 3239 2024-08-19T01:43:51.465 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-08-19T01:43:51.465 INFO:teuthology.orchestra.run.smithi167.stdout: "op": 3238, 2024-08-19T01:43:51.465 INFO:teuthology.orchestra.run.smithi167.stdout: "op_r": 3238, 2024-08-19T01:43:51.465 INFO:teuthology.orchestra.run.smithi167.stdout: "op_w": 0, 2024-08-19T01:43:51.465 INFO:teuthology.orchestra.run.smithi167.stdout: "op_rmw": 0, 2024-08-19T01:43:51.465 INFO:teuthology.orchestra.run.smithi167.stdout: "op_pg": 0, 2024-08-19T01:43:51.465 INFO:teuthology.orchestra.run.smithi167.stdout: "osdop_stat": 2032, 2024-08-19T01:43:51.465 INFO:teuthology.orchestra.run.smithi167.stdout: "osdop_create": 0, 2024-08-19T01:43:51.465 INFO:teuthology.orchestra.run.smithi167.stdout: "osdop_read": 1203, 2024-08-19T01:43:51.465 INFO:teuthology.orchestra.run.smithi167.stdout: "osdop_write": 0, 2024-08-19T01:43:51.465 INFO:teuthology.orchestra.run.smithi167.stdout: "osdop_writefull": 0, 2024-08-19T01:43:51.465 INFO:teuthology.orchestra.run.smithi167.stdout: "osdop_writesame": 0, 2024-08-19T01:43:51.465 INFO:teuthology.orchestra.run.smithi167.stdout: "osdop_append": 0, 2024-08-19T01:43:51.465 INFO:teuthology.orchestra.run.smithi167.stdout: "osdop_zero": 0, 2024-08-19T01:43:51.465 INFO:teuthology.orchestra.run.smithi167.stdout: "osdop_truncate": 0, 2024-08-19T01:43:51.466 INFO:teuthology.orchestra.run.smithi167.stdout: "osdop_delete": 0, 2024-08-19T01:43:51.466 INFO:teuthology.orchestra.run.smithi167.stdout: "osdop_mapext": 0, 2024-08-19T01:43:51.466 INFO:teuthology.orchestra.run.smithi167.stdout: "osdop_sparse_read": 0, 2024-08-19T01:43:51.466 INFO:teuthology.orchestra.run.smithi167.stdout: "osdop_clonerange": 0, 2024-08-19T01:43:51.466 INFO:teuthology.orchestra.run.smithi167.stdout: "osdop_getxattr": 2, 2024-08-19T01:43:51.466 INFO:teuthology.orchestra.run.smithi167.stdout: "osdop_setxattr": 0, 2024-08-19T01:43:51.466 INFO:teuthology.orchestra.run.smithi167.stdout: "osdop_cmpxattr": 0, 2024-08-19T01:43:51.466 INFO:teuthology.orchestra.run.smithi167.stdout: "osdop_rmxattr": 0, 2024-08-19T01:43:51.466 INFO:teuthology.orchestra.run.smithi167.stdout: "osdop_resetxattrs": 0, 2024-08-19T01:43:51.466 INFO:teuthology.orchestra.run.smithi167.stdout: "osdop_call": 0, 2024-08-19T01:43:51.466 INFO:teuthology.orchestra.run.smithi167.stdout: "osdop_watch": 0, 2024-08-19T01:43:51.466 INFO:teuthology.orchestra.run.smithi167.stdout: "osdop_notify": 0, 2024-08-19T01:43:51.466 INFO:teuthology.orchestra.run.smithi167.stdout: "osdop_src_cmpxattr": 0, 2024-08-19T01:43:51.466 INFO:teuthology.orchestra.run.smithi167.stdout: "osdop_pgls": 0, 2024-08-19T01:43:51.467 INFO:teuthology.orchestra.run.smithi167.stdout: "osdop_pgls_filter": 0, 2024-08-19T01:43:51.467 INFO:teuthology.orchestra.run.smithi167.stdout: "osdop_other": 0, 2024-08-19T01:43:51.467 INFO:teuthology.orchestra.run.smithi167.stdout: "linger_active": 0, 2024-08-19T01:43:51.467 INFO:teuthology.orchestra.run.smithi167.stdout: "linger_send": 0, 2024-08-19T01:43:51.467 INFO:teuthology.orchestra.run.smithi167.stdout: "linger_resend": 0, 2024-08-19T01:43:51.467 INFO:teuthology.orchestra.run.smithi167.stdout: "linger_ping": 0, 2024-08-19T01:43:51.467 INFO:teuthology.orchestra.run.smithi167.stdout: "poolop_active": 0, 2024-08-19T01:43:51.467 INFO:teuthology.orchestra.run.smithi167.stdout: "poolop_send": 0, 2024-08-19T01:43:51.467 INFO:teuthology.orchestra.run.smithi167.stdout: "poolop_resend": 0, 2024-08-19T01:43:51.467 INFO:teuthology.orchestra.run.smithi167.stdout: "poolstat_active": 0, 2024-08-19T01:43:51.467 INFO:teuthology.orchestra.run.smithi167.stdout: "poolstat_send": 0, 2024-08-19T01:43:51.467 INFO:teuthology.orchestra.run.smithi167.stdout: "poolstat_resend": 0, 2024-08-19T01:43:51.467 INFO:teuthology.orchestra.run.smithi167.stdout: "statfs_active": 0, 2024-08-19T01:43:51.468 INFO:teuthology.orchestra.run.smithi167.stdout: "statfs_send": 0, 2024-08-19T01:43:51.468 INFO:teuthology.orchestra.run.smithi167.stdout: "statfs_resend": 0, 2024-08-19T01:43:51.468 INFO:teuthology.orchestra.run.smithi167.stdout: "command_active": 0, 2024-08-19T01:43:51.468 INFO:teuthology.orchestra.run.smithi167.stdout: "command_send": 0, 2024-08-19T01:43:51.468 INFO:teuthology.orchestra.run.smithi167.stdout: "command_resend": 0, 2024-08-19T01:43:51.468 INFO:teuthology.orchestra.run.smithi167.stdout: "map_epoch": 86, 2024-08-19T01:43:51.468 INFO:teuthology.orchestra.run.smithi167.stdout: "map_full": 0, 2024-08-19T01:43:51.468 INFO:teuthology.orchestra.run.smithi167.stdout: "map_inc": 2, 2024-08-19T01:43:51.468 INFO:teuthology.orchestra.run.smithi167.stdout: "osd_sessions": 11, 2024-08-19T01:43:51.468 INFO:teuthology.orchestra.run.smithi167.stdout: "osd_session_open": 11, 2024-08-19T01:43:51.468 INFO:teuthology.orchestra.run.smithi167.stdout: "osd_session_close": 0, 2024-08-19T01:43:51.468 INFO:teuthology.orchestra.run.smithi167.stdout: "osd_laggy": 0, 2024-08-19T01:43:51.468 INFO:teuthology.orchestra.run.smithi167.stdout: "omap_wr": 0, 2024-08-19T01:43:51.468 INFO:teuthology.orchestra.run.smithi167.stdout: "omap_rd": 2, 2024-08-19T01:43:51.469 INFO:teuthology.orchestra.run.smithi167.stdout: "omap_del": 0 2024-08-19T01:43:51.469 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-08-19T01:43:51.469 INFO:teuthology.orchestra.run.smithi167.stdout: "oft": { 2024-08-19T01:43:51.469 INFO:teuthology.orchestra.run.smithi167.stdout: "omap_total_objs": 0, 2024-08-19T01:43:51.469 INFO:teuthology.orchestra.run.smithi167.stdout: "omap_total_kv_pairs": 0, 2024-08-19T01:43:51.469 INFO:teuthology.orchestra.run.smithi167.stdout: "omap_total_updates": 0, 2024-08-19T01:43:51.469 INFO:teuthology.orchestra.run.smithi167.stdout: "omap_total_removes": 0 2024-08-19T01:43:51.469 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-08-19T01:43:51.469 INFO:teuthology.orchestra.run.smithi167.stdout: "osd-slow-ops": { 2024-08-19T01:43:51.469 INFO:teuthology.orchestra.run.smithi167.stdout: "slow_ops_count": 0 2024-08-19T01:43:51.469 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-08-19T01:43:51.469 INFO:teuthology.orchestra.run.smithi167.stdout: "purge_queue": { 2024-08-19T01:43:51.469 INFO:teuthology.orchestra.run.smithi167.stdout: "pq_executing_ops": 0, 2024-08-19T01:43:51.469 INFO:teuthology.orchestra.run.smithi167.stdout: "pq_executing_ops_high_water": 0, 2024-08-19T01:43:51.469 INFO:teuthology.orchestra.run.smithi167.stdout: "pq_executing": 0, 2024-08-19T01:43:51.470 INFO:teuthology.orchestra.run.smithi167.stdout: "pq_executing_high_water": 0, 2024-08-19T01:43:51.470 INFO:teuthology.orchestra.run.smithi167.stdout: "pq_executed_ops": 0, 2024-08-19T01:43:51.470 INFO:teuthology.orchestra.run.smithi167.stdout: "pq_executed": 0, 2024-08-19T01:43:51.470 INFO:teuthology.orchestra.run.smithi167.stdout: "pq_item_in_journal": 0 2024-08-19T01:43:51.470 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-08-19T01:43:51.470 INFO:teuthology.orchestra.run.smithi167.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-08-19T01:43:51.470 INFO:teuthology.orchestra.run.smithi167.stdout: "val": 0, 2024-08-19T01:43:51.470 INFO:teuthology.orchestra.run.smithi167.stdout: "max": 104857600, 2024-08-19T01:43:51.470 INFO:teuthology.orchestra.run.smithi167.stdout: "get_started": 0, 2024-08-19T01:43:51.470 INFO:teuthology.orchestra.run.smithi167.stdout: "get": 3447, 2024-08-19T01:43:51.470 INFO:teuthology.orchestra.run.smithi167.stdout: "get_sum": 732334680, 2024-08-19T01:43:51.470 INFO:teuthology.orchestra.run.smithi167.stdout: "get_or_fail_fail": 0, 2024-08-19T01:43:51.470 INFO:teuthology.orchestra.run.smithi167.stdout: "get_or_fail_success": 3447, 2024-08-19T01:43:51.470 INFO:teuthology.orchestra.run.smithi167.stdout: "take": 0, 2024-08-19T01:43:51.470 INFO:teuthology.orchestra.run.smithi167.stdout: "take_sum": 0, 2024-08-19T01:43:51.471 INFO:teuthology.orchestra.run.smithi167.stdout: "put": 3447, 2024-08-19T01:43:51.471 INFO:teuthology.orchestra.run.smithi167.stdout: "put_sum": 732334680, 2024-08-19T01:43:51.471 INFO:teuthology.orchestra.run.smithi167.stdout: "wait": { 2024-08-19T01:43:51.471 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-08-19T01:43:51.471 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-08-19T01:43:51.471 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-08-19T01:43:51.471 INFO:teuthology.orchestra.run.smithi167.stdout: } 2024-08-19T01:43:51.471 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-08-19T01:43:51.471 INFO:teuthology.orchestra.run.smithi167.stdout: "throttle-objecter_bytes": { 2024-08-19T01:43:51.471 INFO:teuthology.orchestra.run.smithi167.stdout: "val": 0, 2024-08-19T01:43:51.471 INFO:teuthology.orchestra.run.smithi167.stdout: "max": 104857600, 2024-08-19T01:43:51.471 INFO:teuthology.orchestra.run.smithi167.stdout: "get_started": 0, 2024-08-19T01:43:51.471 INFO:teuthology.orchestra.run.smithi167.stdout: "get": 0, 2024-08-19T01:43:51.471 INFO:teuthology.orchestra.run.smithi167.stdout: "get_sum": 0, 2024-08-19T01:43:51.471 INFO:teuthology.orchestra.run.smithi167.stdout: "get_or_fail_fail": 0, 2024-08-19T01:43:51.472 INFO:teuthology.orchestra.run.smithi167.stdout: "get_or_fail_success": 0, 2024-08-19T01:43:51.472 INFO:teuthology.orchestra.run.smithi167.stdout: "take": 3238, 2024-08-19T01:43:51.472 INFO:teuthology.orchestra.run.smithi167.stdout: "take_sum": 730979145, 2024-08-19T01:43:51.472 INFO:teuthology.orchestra.run.smithi167.stdout: "put": 3238, 2024-08-19T01:43:51.472 INFO:teuthology.orchestra.run.smithi167.stdout: "put_sum": 730979145, 2024-08-19T01:43:51.472 INFO:teuthology.orchestra.run.smithi167.stdout: "wait": { 2024-08-19T01:43:51.472 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-08-19T01:43:51.472 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-08-19T01:43:51.472 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-08-19T01:43:51.472 INFO:teuthology.orchestra.run.smithi167.stdout: } 2024-08-19T01:43:51.472 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-08-19T01:43:51.472 INFO:teuthology.orchestra.run.smithi167.stdout: "throttle-objecter_ops": { 2024-08-19T01:43:51.472 INFO:teuthology.orchestra.run.smithi167.stdout: "val": 0, 2024-08-19T01:43:51.472 INFO:teuthology.orchestra.run.smithi167.stdout: "max": 1024, 2024-08-19T01:43:51.473 INFO:teuthology.orchestra.run.smithi167.stdout: "get_started": 0, 2024-08-19T01:43:51.473 INFO:teuthology.orchestra.run.smithi167.stdout: "get": 0, 2024-08-19T01:43:51.473 INFO:teuthology.orchestra.run.smithi167.stdout: "get_sum": 0, 2024-08-19T01:43:51.473 INFO:teuthology.orchestra.run.smithi167.stdout: "get_or_fail_fail": 0, 2024-08-19T01:43:51.473 INFO:teuthology.orchestra.run.smithi167.stdout: "get_or_fail_success": 0, 2024-08-19T01:43:51.473 INFO:teuthology.orchestra.run.smithi167.stdout: "take": 3238, 2024-08-19T01:43:51.473 INFO:teuthology.orchestra.run.smithi167.stdout: "take_sum": 3238, 2024-08-19T01:43:51.473 INFO:teuthology.orchestra.run.smithi167.stdout: "put": 3238, 2024-08-19T01:43:51.473 INFO:teuthology.orchestra.run.smithi167.stdout: "put_sum": 3238, 2024-08-19T01:43:51.473 INFO:teuthology.orchestra.run.smithi167.stdout: "wait": { 2024-08-19T01:43:51.473 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-08-19T01:43:51.473 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-08-19T01:43:51.473 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-08-19T01:43:51.473 INFO:teuthology.orchestra.run.smithi167.stdout: } 2024-08-19T01:43:51.473 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-08-19T01:43:51.474 INFO:teuthology.orchestra.run.smithi167.stdout: "throttle-write_buf_throttle": { 2024-08-19T01:43:51.474 INFO:teuthology.orchestra.run.smithi167.stdout: "val": 0, 2024-08-19T01:43:51.474 INFO:teuthology.orchestra.run.smithi167.stdout: "max": 3758096384, 2024-08-19T01:43:51.474 INFO:teuthology.orchestra.run.smithi167.stdout: "get_started": 0, 2024-08-19T01:43:51.474 INFO:teuthology.orchestra.run.smithi167.stdout: "get": 0, 2024-08-19T01:43:51.474 INFO:teuthology.orchestra.run.smithi167.stdout: "get_sum": 0, 2024-08-19T01:43:51.474 INFO:teuthology.orchestra.run.smithi167.stdout: "get_or_fail_fail": 0, 2024-08-19T01:43:51.474 INFO:teuthology.orchestra.run.smithi167.stdout: "get_or_fail_success": 0, 2024-08-19T01:43:51.474 INFO:teuthology.orchestra.run.smithi167.stdout: "take": 0, 2024-08-19T01:43:51.474 INFO:teuthology.orchestra.run.smithi167.stdout: "take_sum": 0, 2024-08-19T01:43:51.474 INFO:teuthology.orchestra.run.smithi167.stdout: "put": 0, 2024-08-19T01:43:51.474 INFO:teuthology.orchestra.run.smithi167.stdout: "put_sum": 0, 2024-08-19T01:43:51.474 INFO:teuthology.orchestra.run.smithi167.stdout: "wait": { 2024-08-19T01:43:51.474 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-08-19T01:43:51.474 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-08-19T01:43:51.475 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-08-19T01:43:51.475 INFO:teuthology.orchestra.run.smithi167.stdout: } 2024-08-19T01:43:51.475 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-08-19T01:43:51.475 INFO:teuthology.orchestra.run.smithi167.stdout: "throttle-write_buf_throttle-0x55bb0792cb40": { 2024-08-19T01:43:51.475 INFO:teuthology.orchestra.run.smithi167.stdout: "val": 0, 2024-08-19T01:43:51.475 INFO:teuthology.orchestra.run.smithi167.stdout: "max": 3758096384, 2024-08-19T01:43:51.475 INFO:teuthology.orchestra.run.smithi167.stdout: "get_started": 0, 2024-08-19T01:43:51.475 INFO:teuthology.orchestra.run.smithi167.stdout: "get": 0, 2024-08-19T01:43:51.475 INFO:teuthology.orchestra.run.smithi167.stdout: "get_sum": 0, 2024-08-19T01:43:51.475 INFO:teuthology.orchestra.run.smithi167.stdout: "get_or_fail_fail": 0, 2024-08-19T01:43:51.475 INFO:teuthology.orchestra.run.smithi167.stdout: "get_or_fail_success": 0, 2024-08-19T01:43:51.475 INFO:teuthology.orchestra.run.smithi167.stdout: "take": 0, 2024-08-19T01:43:51.475 INFO:teuthology.orchestra.run.smithi167.stdout: "take_sum": 0, 2024-08-19T01:43:51.475 INFO:teuthology.orchestra.run.smithi167.stdout: "put": 0, 2024-08-19T01:43:51.475 INFO:teuthology.orchestra.run.smithi167.stdout: "put_sum": 0, 2024-08-19T01:43:51.476 INFO:teuthology.orchestra.run.smithi167.stdout: "wait": { 2024-08-19T01:43:51.476 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-08-19T01:43:51.476 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-08-19T01:43:51.476 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-08-19T01:43:51.476 INFO:teuthology.orchestra.run.smithi167.stdout: } 2024-08-19T01:43:51.476 INFO:teuthology.orchestra.run.smithi167.stdout: } 2024-08-19T01:43:51.476 INFO:teuthology.orchestra.run.smithi167.stdout:} 2024-08-19T01:43:51.952 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:43:51 smithi117 ceph-mon[22016]: pgmap v609: 97 pgs: 97 active+clean; 35 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-08-19T01:43:51.965 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:43:51 smithi167 ceph-mon[21875]: pgmap v609: 97 pgs: 97 active+clean; 35 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-08-19T01:43:52.154 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:43:51 smithi083 ceph-mon[17120]: pgmap v609: 97 pgs: 97 active+clean; 35 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-08-19T01:43:52.201 INFO:tasks.check_counter:Daemon mds.i mds.exported=0 2024-08-19T01:43:52.201 INFO:tasks.check_counter:Daemon mds.i mds.imported=0 2024-08-19T01:43:52.201 INFO:tasks.check_counter:Daemon mds.i mds.dir_split=0 2024-08-19T01:43:52.201 DEBUG:tasks.check_counter:Getting stats from l 2024-08-19T01:43:52.201 DEBUG:teuthology.orchestra.run.smithi167:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833 shell --fsid bad8780e-5dc9-11ef-bccf-c7b262605968 -- ceph daemon mds.l perf dump 2024-08-19T01:43:52.450 INFO:teuthology.orchestra.run.smithi167.stderr:Inferring config /var/lib/ceph/bad8780e-5dc9-11ef-bccf-c7b262605968/mon.c/config 2024-08-19T01:43:52.945 INFO:teuthology.orchestra.run.smithi167.stdout:{ 2024-08-19T01:43:52.945 INFO:teuthology.orchestra.run.smithi167.stdout: "AsyncMessenger::Worker-0": { 2024-08-19T01:43:52.945 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_recv_messages": 0, 2024-08-19T01:43:52.945 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_send_messages": 2, 2024-08-19T01:43:52.945 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_recv_bytes": 0, 2024-08-19T01:43:52.946 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_send_bytes": 0, 2024-08-19T01:43:52.946 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_created_connections": 4, 2024-08-19T01:43:52.946 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_active_connections": 0, 2024-08-19T01:43:52.946 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_running_total_time": 0.001122494, 2024-08-19T01:43:52.946 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_running_send_time": 0.000000000, 2024-08-19T01:43:52.946 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_running_recv_time": 0.000228745, 2024-08-19T01:43:52.946 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-08-19T01:43:52.946 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_send_messages_queue_lat": { 2024-08-19T01:43:52.946 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-08-19T01:43:52.946 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-08-19T01:43:52.946 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-08-19T01:43:52.946 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-08-19T01:43:52.946 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_handle_ack_lat": { 2024-08-19T01:43:52.947 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-08-19T01:43:52.947 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-08-19T01:43:52.947 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-08-19T01:43:52.947 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-08-19T01:43:52.947 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_recv_encrypted_bytes": 0, 2024-08-19T01:43:52.947 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_send_encrypted_bytes": 0 2024-08-19T01:43:52.947 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-08-19T01:43:52.947 INFO:teuthology.orchestra.run.smithi167.stdout: "AsyncMessenger::Worker-1": { 2024-08-19T01:43:52.947 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_recv_messages": 202, 2024-08-19T01:43:52.947 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_send_messages": 195, 2024-08-19T01:43:52.947 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_recv_bytes": 750336, 2024-08-19T01:43:52.947 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_send_bytes": 119056, 2024-08-19T01:43:52.947 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_created_connections": 2, 2024-08-19T01:43:52.947 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_active_connections": 1, 2024-08-19T01:43:52.948 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_running_total_time": 0.031945295, 2024-08-19T01:43:52.948 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_running_send_time": 0.012736132, 2024-08-19T01:43:52.948 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_running_recv_time": 0.026001597, 2024-08-19T01:43:52.948 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-08-19T01:43:52.948 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_send_messages_queue_lat": { 2024-08-19T01:43:52.948 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 195, 2024-08-19T01:43:52.948 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.014631712, 2024-08-19T01:43:52.948 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000075034 2024-08-19T01:43:52.948 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-08-19T01:43:52.948 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_handle_ack_lat": { 2024-08-19T01:43:52.948 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-08-19T01:43:52.948 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-08-19T01:43:52.948 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-08-19T01:43:52.948 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-08-19T01:43:52.948 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_recv_encrypted_bytes": 750336, 2024-08-19T01:43:52.949 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_send_encrypted_bytes": 119056 2024-08-19T01:43:52.949 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-08-19T01:43:52.949 INFO:teuthology.orchestra.run.smithi167.stdout: "AsyncMessenger::Worker-2": { 2024-08-19T01:43:52.949 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_recv_messages": 1, 2024-08-19T01:43:52.949 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_send_messages": 147, 2024-08-19T01:43:52.949 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_recv_bytes": 160, 2024-08-19T01:43:52.949 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_send_bytes": 104096, 2024-08-19T01:43:52.949 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_created_connections": 3, 2024-08-19T01:43:52.949 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_active_connections": 2, 2024-08-19T01:43:52.949 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_running_total_time": 0.008364114, 2024-08-19T01:43:52.949 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_running_send_time": 0.006250976, 2024-08-19T01:43:52.949 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_running_recv_time": 0.000527581, 2024-08-19T01:43:52.949 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-08-19T01:43:52.949 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_send_messages_queue_lat": { 2024-08-19T01:43:52.949 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 145, 2024-08-19T01:43:52.950 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.008726284, 2024-08-19T01:43:52.950 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000060181 2024-08-19T01:43:52.950 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-08-19T01:43:52.950 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_handle_ack_lat": { 2024-08-19T01:43:52.950 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-08-19T01:43:52.950 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-08-19T01:43:52.950 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-08-19T01:43:52.950 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-08-19T01:43:52.950 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_recv_encrypted_bytes": 160, 2024-08-19T01:43:52.950 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_send_encrypted_bytes": 104096 2024-08-19T01:43:52.951 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-08-19T01:43:52.951 INFO:teuthology.orchestra.run.smithi167.stdout: "cct": { 2024-08-19T01:43:52.951 INFO:teuthology.orchestra.run.smithi167.stdout: "total_workers": 0, 2024-08-19T01:43:52.951 INFO:teuthology.orchestra.run.smithi167.stdout: "unhealthy_workers": 0 2024-08-19T01:43:52.951 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-08-19T01:43:52.951 INFO:teuthology.orchestra.run.smithi167.stdout: "mempool": { 2024-08-19T01:43:52.951 INFO:teuthology.orchestra.run.smithi167.stdout: "bloom_filter_bytes": 0, 2024-08-19T01:43:52.951 INFO:teuthology.orchestra.run.smithi167.stdout: "bloom_filter_items": 0, 2024-08-19T01:43:52.951 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_alloc_bytes": 0, 2024-08-19T01:43:52.951 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_alloc_items": 0, 2024-08-19T01:43:52.951 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_cache_data_bytes": 0, 2024-08-19T01:43:52.951 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_cache_data_items": 0, 2024-08-19T01:43:52.951 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_cache_onode_bytes": 0, 2024-08-19T01:43:52.951 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_cache_onode_items": 0, 2024-08-19T01:43:52.951 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_cache_meta_bytes": 0, 2024-08-19T01:43:52.952 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_cache_meta_items": 0, 2024-08-19T01:43:52.952 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_cache_other_bytes": 0, 2024-08-19T01:43:52.952 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_cache_other_items": 0, 2024-08-19T01:43:52.952 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_cache_buffer_bytes": 0, 2024-08-19T01:43:52.952 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_cache_buffer_items": 0, 2024-08-19T01:43:52.952 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_extent_bytes": 0, 2024-08-19T01:43:52.952 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_extent_items": 0, 2024-08-19T01:43:52.952 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_blob_bytes": 0, 2024-08-19T01:43:52.952 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_blob_items": 0, 2024-08-19T01:43:52.952 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_shared_blob_bytes": 0, 2024-08-19T01:43:52.952 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_shared_blob_items": 0, 2024-08-19T01:43:52.952 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_inline_bl_bytes": 0, 2024-08-19T01:43:52.952 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_inline_bl_items": 0, 2024-08-19T01:43:52.952 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_fsck_bytes": 0, 2024-08-19T01:43:52.953 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_fsck_items": 0, 2024-08-19T01:43:52.953 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_txc_bytes": 0, 2024-08-19T01:43:52.953 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_txc_items": 0, 2024-08-19T01:43:52.953 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_writing_deferred_bytes": 0, 2024-08-19T01:43:52.953 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_writing_deferred_items": 0, 2024-08-19T01:43:52.953 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_writing_bytes": 0, 2024-08-19T01:43:52.953 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_writing_items": 0, 2024-08-19T01:43:52.953 INFO:teuthology.orchestra.run.smithi167.stdout: "bluefs_bytes": 0, 2024-08-19T01:43:52.953 INFO:teuthology.orchestra.run.smithi167.stdout: "bluefs_items": 0, 2024-08-19T01:43:52.953 INFO:teuthology.orchestra.run.smithi167.stdout: "bluefs_file_reader_bytes": 0, 2024-08-19T01:43:52.953 INFO:teuthology.orchestra.run.smithi167.stdout: "bluefs_file_reader_items": 0, 2024-08-19T01:43:52.953 INFO:teuthology.orchestra.run.smithi167.stdout: "bluefs_file_writer_bytes": 0, 2024-08-19T01:43:52.953 INFO:teuthology.orchestra.run.smithi167.stdout: "bluefs_file_writer_items": 0, 2024-08-19T01:43:52.954 INFO:teuthology.orchestra.run.smithi167.stdout: "buffer_anon_bytes": 148985, 2024-08-19T01:43:52.954 INFO:teuthology.orchestra.run.smithi167.stdout: "buffer_anon_items": 30, 2024-08-19T01:43:52.954 INFO:teuthology.orchestra.run.smithi167.stdout: "buffer_meta_bytes": 0, 2024-08-19T01:43:52.954 INFO:teuthology.orchestra.run.smithi167.stdout: "buffer_meta_items": 0, 2024-08-19T01:43:52.954 INFO:teuthology.orchestra.run.smithi167.stdout: "osd_bytes": 0, 2024-08-19T01:43:52.954 INFO:teuthology.orchestra.run.smithi167.stdout: "osd_items": 0, 2024-08-19T01:43:52.954 INFO:teuthology.orchestra.run.smithi167.stdout: "osd_mapbl_bytes": 0, 2024-08-19T01:43:52.954 INFO:teuthology.orchestra.run.smithi167.stdout: "osd_mapbl_items": 0, 2024-08-19T01:43:52.954 INFO:teuthology.orchestra.run.smithi167.stdout: "osd_pglog_bytes": 0, 2024-08-19T01:43:52.954 INFO:teuthology.orchestra.run.smithi167.stdout: "osd_pglog_items": 0, 2024-08-19T01:43:52.954 INFO:teuthology.orchestra.run.smithi167.stdout: "osdmap_bytes": 0, 2024-08-19T01:43:52.954 INFO:teuthology.orchestra.run.smithi167.stdout: "osdmap_items": 0, 2024-08-19T01:43:52.954 INFO:teuthology.orchestra.run.smithi167.stdout: "osdmap_mapping_bytes": 0, 2024-08-19T01:43:52.954 INFO:teuthology.orchestra.run.smithi167.stdout: "osdmap_mapping_items": 0, 2024-08-19T01:43:52.954 INFO:teuthology.orchestra.run.smithi167.stdout: "pgmap_bytes": 0, 2024-08-19T01:43:52.955 INFO:teuthology.orchestra.run.smithi167.stdout: "pgmap_items": 0, 2024-08-19T01:43:52.955 INFO:teuthology.orchestra.run.smithi167.stdout: "mds_co_bytes": 928, 2024-08-19T01:43:52.955 INFO:teuthology.orchestra.run.smithi167.stdout: "mds_co_items": 2, 2024-08-19T01:43:52.955 INFO:teuthology.orchestra.run.smithi167.stdout: "unittest_1_bytes": 0, 2024-08-19T01:43:52.955 INFO:teuthology.orchestra.run.smithi167.stdout: "unittest_1_items": 0, 2024-08-19T01:43:52.955 INFO:teuthology.orchestra.run.smithi167.stdout: "unittest_2_bytes": 0, 2024-08-19T01:43:52.955 INFO:teuthology.orchestra.run.smithi167.stdout: "unittest_2_items": 0 2024-08-19T01:43:52.955 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-08-19T01:43:52.955 INFO:teuthology.orchestra.run.smithi167.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-08-19T01:43:52.955 INFO:teuthology.orchestra.run.smithi167.stdout: "val": 0, 2024-08-19T01:43:52.955 INFO:teuthology.orchestra.run.smithi167.stdout: "max": 104857600, 2024-08-19T01:43:52.955 INFO:teuthology.orchestra.run.smithi167.stdout: "get_started": 0, 2024-08-19T01:43:52.955 INFO:teuthology.orchestra.run.smithi167.stdout: "get": 199, 2024-08-19T01:43:52.955 INFO:teuthology.orchestra.run.smithi167.stdout: "get_sum": 719361, 2024-08-19T01:43:52.955 INFO:teuthology.orchestra.run.smithi167.stdout: "get_or_fail_fail": 0, 2024-08-19T01:43:52.956 INFO:teuthology.orchestra.run.smithi167.stdout: "get_or_fail_success": 199, 2024-08-19T01:43:52.956 INFO:teuthology.orchestra.run.smithi167.stdout: "take": 0, 2024-08-19T01:43:52.956 INFO:teuthology.orchestra.run.smithi167.stdout: "take_sum": 0, 2024-08-19T01:43:52.956 INFO:teuthology.orchestra.run.smithi167.stdout: "put": 199, 2024-08-19T01:43:52.956 INFO:teuthology.orchestra.run.smithi167.stdout: "put_sum": 719361, 2024-08-19T01:43:52.956 INFO:teuthology.orchestra.run.smithi167.stdout: "wait": { 2024-08-19T01:43:52.956 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-08-19T01:43:52.956 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-08-19T01:43:52.956 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-08-19T01:43:52.956 INFO:teuthology.orchestra.run.smithi167.stdout: } 2024-08-19T01:43:52.956 INFO:teuthology.orchestra.run.smithi167.stdout: } 2024-08-19T01:43:52.956 INFO:teuthology.orchestra.run.smithi167.stdout:} 2024-08-19T01:43:53.676 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.l 2024-08-19T01:43:53.676 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.l 2024-08-19T01:43:53.676 WARNING:tasks.check_counter:Counter 'mds.dir_split' not found on daemon mds.l 2024-08-19T01:43:53.676 DEBUG:teuthology.run_tasks:Unwinding manager kclient 2024-08-19T01:43:53.686 INFO:tasks.kclient:Unmounting kernel clients... 2024-08-19T01:43:53.686 DEBUG:teuthology.orchestra.run.smithi083:> set -ex 2024-08-19T01:43:53.687 DEBUG:teuthology.orchestra.run.smithi083:> dd if=/proc/self/mounts of=/dev/stdout 2024-08-19T01:43:53.729 DEBUG:teuthology.orchestra.run.smithi083:> set -ex 2024-08-19T01:43:53.729 DEBUG:teuthology.orchestra.run.smithi083:> dd if=/proc/self/mounts of=/dev/stdout 2024-08-19T01:43:53.808 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833 shell --fsid bad8780e-5dc9-11ef-bccf-c7b262605968 -- ceph osd blocklist ls 2024-08-19T01:43:53.952 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:43:53 smithi117 ceph-mon[22016]: pgmap v610: 97 pgs: 97 active+clean; 35 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-08-19T01:43:53.965 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:43:53 smithi167 ceph-mon[21875]: pgmap v610: 97 pgs: 97 active+clean; 35 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-08-19T01:43:54.154 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:43:53 smithi083 ceph-mon[17120]: pgmap v610: 97 pgs: 97 active+clean; 35 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-08-19T01:43:54.227 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/bad8780e-5dc9-11ef-bccf-c7b262605968/mon.a/config 2024-08-19T01:43:54.914 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:54.913+0000 7f614f200640 1 -- 172.21.15.83:0/299880834 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6150103900 msgr2=0x7f6150103d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:43:54.914 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:54.913+0000 7f614f200640 1 --2- 172.21.15.83:0/299880834 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6150103900 0x7f6150103d60 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f613c009900 tx=0x7f613c02a3f0 comp rx=0 tx=0).stop 2024-08-19T01:43:54.915 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:54.913+0000 7f614f200640 1 -- 172.21.15.83:0/299880834 shutdown_connections 2024-08-19T01:43:54.915 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:54.913+0000 7f614f200640 1 --2- 172.21.15.83:0/299880834 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f61501042a0 0x7f61501107d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:43:54.915 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:54.913+0000 7f614f200640 1 --2- 172.21.15.83:0/299880834 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6150103900 0x7f6150103d60 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:43:54.915 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:54.913+0000 7f614f200640 1 --2- 172.21.15.83:0/299880834 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6150109900 0x7f6150109ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:43:54.915 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:54.913+0000 7f614f200640 1 -- 172.21.15.83:0/299880834 >> 172.21.15.83:0/299880834 conn(0x7f61500ff640 msgr2=0x7f6150101a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:43:54.915 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:54.913+0000 7f614f200640 1 -- 172.21.15.83:0/299880834 shutdown_connections 2024-08-19T01:43:54.915 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:54.913+0000 7f614f200640 1 -- 172.21.15.83:0/299880834 wait complete. 2024-08-19T01:43:54.915 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:54.914+0000 7f614f200640 1 Processor -- start 2024-08-19T01:43:54.917 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:54.914+0000 7f614f200640 1 -- start start 2024-08-19T01:43:54.917 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:54.915+0000 7f614f200640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6150103900 0x7f615019a230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:43:54.917 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:54.915+0000 7f614f200640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f61501042a0 0x7f615019a770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:43:54.917 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:54.915+0000 7f614f200640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6150109900 0x7f615019eb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:43:54.917 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:54.915+0000 7f614f200640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f6150113d50 con 0x7f6150109900 2024-08-19T01:43:54.917 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:54.915+0000 7f614f200640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f6150113bd0 con 0x7f6150103900 2024-08-19T01:43:54.917 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:54.915+0000 7f614f200640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f6150113ed0 con 0x7f61501042a0 2024-08-19T01:43:54.917 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:54.915+0000 7f614e800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6150109900 0x7f615019eb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:43:54.918 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:54.915+0000 7f614de00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6150103900 0x7f615019a230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:43:54.918 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:54.915+0000 7f614de00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6150103900 0x7f615019a230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.83:43886/0 (socket says 172.21.15.83:43886) 2024-08-19T01:43:54.918 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:54.915+0000 7f614de00640 1 -- 172.21.15.83:0/3567084892 learned_addr learned my addr 172.21.15.83:0/3567084892 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-08-19T01:43:54.918 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:54.915+0000 7f614e800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6150109900 0x7f615019eb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:34006/0 (socket says 172.21.15.83:34006) 2024-08-19T01:43:54.918 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:54.915+0000 7f614d400640 1 --2- 172.21.15.83:0/3567084892 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f61501042a0 0x7f615019a770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:43:54.918 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:54.916+0000 7f614de00640 1 -- 172.21.15.83:0/3567084892 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f61501042a0 msgr2=0x7f615019a770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:43:54.918 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:54.916+0000 7f614de00640 1 --2- 172.21.15.83:0/3567084892 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f61501042a0 0x7f615019a770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:43:54.918 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:54.916+0000 7f614de00640 1 -- 172.21.15.83:0/3567084892 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6150109900 msgr2=0x7f615019eb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:43:54.918 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:54.916+0000 7f614de00640 1 --2- 172.21.15.83:0/3567084892 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6150109900 0x7f615019eb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:43:54.918 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:54.916+0000 7f614de00640 1 -- 172.21.15.83:0/3567084892 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f615019f1e0 con 0x7f6150103900 2024-08-19T01:43:54.918 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:54.916+0000 7f614de00640 1 --2- 172.21.15.83:0/3567084892 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6150103900 0x7f615019a230 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f614000e9f0 tx=0x7f614000edf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:43:54.918 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:54.916+0000 7f6146a00640 1 -- 172.21.15.83:0/3567084892 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f614000ccc0 con 0x7f6150103900 2024-08-19T01:43:54.919 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:54.916+0000 7f614f200640 1 -- 172.21.15.83:0/3567084892 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f615019f4d0 con 0x7f6150103900 2024-08-19T01:43:54.919 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:54.916+0000 7f614f200640 1 -- 172.21.15.83:0/3567084892 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f61501a6db0 con 0x7f6150103900 2024-08-19T01:43:54.919 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:54.916+0000 7f6146a00640 1 -- 172.21.15.83:0/3567084892 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f614000ce60 con 0x7f6150103900 2024-08-19T01:43:54.919 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:54.916+0000 7f6146a00640 1 -- 172.21.15.83:0/3567084892 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6140010430 con 0x7f6150103900 2024-08-19T01:43:54.919 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:54.917+0000 7f614f200640 1 -- 172.21.15.83:0/3567084892 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6130005290 con 0x7f6150103900 2024-08-19T01:43:54.919 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:54.919+0000 7f6146a00640 1 -- 172.21.15.83:0/3567084892 <== mon.2 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7f61400046c0 con 0x7f6150103900 2024-08-19T01:43:54.921 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:54.920+0000 7f6146a00640 1 --2- 172.21.15.83:0/3567084892 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f61340a2c00 0x7f61340a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:43:54.921 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:54.921+0000 7f6146a00640 1 -- 172.21.15.83:0/3567084892 <== mon.2 v2:172.21.15.117:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8269+0+0 (secure 0 0 0) 0x7f61400cf2b0 con 0x7f6150103900 2024-08-19T01:43:54.921 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:54.921+0000 7f614d400640 1 --2- 172.21.15.83:0/3567084892 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f61340a2c00 0x7f61340a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:43:54.921 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:54.921+0000 7f614d400640 1 --2- 172.21.15.83:0/3567084892 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f61340a2c00 0x7f61340a50c0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f613c000c00 tx=0x7f613c005b50 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:43:54.925 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:54.925+0000 7f6146a00640 1 -- 172.21.15.83:0/3567084892 <== mon.2 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7f614009c190 con 0x7f6150103900 2024-08-19T01:43:55.111 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:55.111+0000 7f614f200640 1 -- 172.21.15.83:0/3567084892 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f6130005580 con 0x7f6150103900 2024-08-19T01:43:55.112 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:55.111+0000 7f6146a00640 1 -- 172.21.15.83:0/3567084892 <== mon.2 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 10 entries v86) ==== 81+0+589 (secure 0 0 0) 0x7f614009ba90 con 0x7f6150103900 2024-08-19T01:43:55.112 INFO:teuthology.orchestra.run.smithi083.stdout:172.21.15.83:0/3138095466 2024-08-20T01:26:04.277716+0000 2024-08-19T01:43:55.112 INFO:teuthology.orchestra.run.smithi083.stdout:172.21.15.83:0/3835005198 2024-08-20T01:26:04.277716+0000 2024-08-19T01:43:55.112 INFO:teuthology.orchestra.run.smithi083.stdout:172.21.15.83:0/1790374088 2024-08-20T01:25:37.228583+0000 2024-08-19T01:43:55.112 INFO:teuthology.orchestra.run.smithi083.stdout:172.21.15.83:6801/573907319 2024-08-20T01:26:04.277716+0000 2024-08-19T01:43:55.112 INFO:teuthology.orchestra.run.smithi083.stdout:172.21.15.83:6800/573907319 2024-08-20T01:26:04.277716+0000 2024-08-19T01:43:55.112 INFO:teuthology.orchestra.run.smithi083.stdout:172.21.15.83:0/664676002 2024-08-20T01:25:37.228583+0000 2024-08-19T01:43:55.112 INFO:teuthology.orchestra.run.smithi083.stdout:172.21.15.83:0/2144611449 2024-08-20T01:25:37.228583+0000 2024-08-19T01:43:55.113 INFO:teuthology.orchestra.run.smithi083.stdout:172.21.15.83:6801/3261152818 2024-08-20T01:25:37.228583+0000 2024-08-19T01:43:55.113 INFO:teuthology.orchestra.run.smithi083.stdout:172.21.15.83:0/1609552666 2024-08-20T01:26:04.277716+0000 2024-08-19T01:43:55.113 INFO:teuthology.orchestra.run.smithi083.stdout:172.21.15.83:6800/3261152818 2024-08-20T01:25:37.228583+0000 2024-08-19T01:43:55.113 INFO:teuthology.orchestra.run.smithi083.stderr:listed 10 entries 2024-08-19T01:43:55.114 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:55.113+0000 7f614f200640 1 -- 172.21.15.83:0/3567084892 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f61340a2c00 msgr2=0x7f61340a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:43:55.114 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:55.113+0000 7f614f200640 1 --2- 172.21.15.83:0/3567084892 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f61340a2c00 0x7f61340a50c0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f613c000c00 tx=0x7f613c005b50 comp rx=0 tx=0).stop 2024-08-19T01:43:55.114 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:55.113+0000 7f614f200640 1 -- 172.21.15.83:0/3567084892 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6150103900 msgr2=0x7f615019a230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:43:55.114 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:55.113+0000 7f614f200640 1 --2- 172.21.15.83:0/3567084892 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6150103900 0x7f615019a230 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f614000e9f0 tx=0x7f614000edf0 comp rx=0 tx=0).stop 2024-08-19T01:43:55.114 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:55.113+0000 7f614f200640 1 -- 172.21.15.83:0/3567084892 shutdown_connections 2024-08-19T01:43:55.115 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:55.114+0000 7f614f200640 1 --2- 172.21.15.83:0/3567084892 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f61340a2c00 0x7f61340a50c0 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:43:55.115 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:55.114+0000 7f614f200640 1 --2- 172.21.15.83:0/3567084892 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6150109900 0x7f615019eb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:43:55.115 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:55.114+0000 7f614f200640 1 --2- 172.21.15.83:0/3567084892 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f61501042a0 0x7f615019a770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:43:55.115 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:55.114+0000 7f614f200640 1 --2- 172.21.15.83:0/3567084892 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6150103900 0x7f615019a230 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:43:55.115 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:55.114+0000 7f614f200640 1 -- 172.21.15.83:0/3567084892 >> 172.21.15.83:0/3567084892 conn(0x7f61500ff640 msgr2=0x7f61500ffe60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:43:55.115 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:55.114+0000 7f614f200640 1 -- 172.21.15.83:0/3567084892 shutdown_connections 2024-08-19T01:43:55.115 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:55.114+0000 7f614f200640 1 -- 172.21.15.83:0/3567084892 wait complete. 2024-08-19T01:43:55.904 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:43:55 smithi083 ceph-mon[17120]: pgmap v611: 97 pgs: 97 active+clean; 35 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-08-19T01:43:55.904 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:43:55 smithi083 ceph-mon[17120]: from='client.? 172.21.15.83:0/3567084892' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-08-19T01:43:55.952 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:43:55 smithi117 ceph-mon[22016]: pgmap v611: 97 pgs: 97 active+clean; 35 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-08-19T01:43:55.952 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:43:55 smithi117 ceph-mon[22016]: from='client.? 172.21.15.83:0/3567084892' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-08-19T01:43:55.966 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:43:55 smithi167 ceph-mon[21875]: pgmap v611: 97 pgs: 97 active+clean; 35 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-08-19T01:43:55.966 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:43:55 smithi167 ceph-mon[21875]: from='client.? 172.21.15.83:0/3567084892' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-08-19T01:43:55.971 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-08-19T01:43:55.971 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-08-19T01:43:55.971 DEBUG:teuthology.orchestra.run.smithi083:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-08-19T01:43:56.060 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi083.front.sepia.ceph.com 2024-08-19T01:43:56.060 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-08-19T01:43:56.060 DEBUG:teuthology.orchestra.run.smithi083:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-08-19T01:43:56.142 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-08-19T01:43:56.142 DEBUG:teuthology.orchestra.run.smithi083:> ip netns list 2024-08-19T01:43:56.221 INFO:teuthology.orchestra.run.smithi083.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-08-19T01:43:56.221 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-08-19T01:43:56.221 DEBUG:teuthology.orchestra.run.smithi083:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.0 2024-08-19T01:43:56.350 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-08-19T01:43:56.350 DEBUG:teuthology.orchestra.run.smithi083:> sudo ip link delete ceph-brx 2024-08-19T01:43:56.666 DEBUG:teuthology.run_tasks:Unwinding manager cephadm 2024-08-19T01:43:56.676 INFO:tasks.cephadm:Teardown begin 2024-08-19T01:43:56.676 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-08-19T01:43:56.677 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833 shell --fsid bad8780e-5dc9-11ef-bccf-c7b262605968 -- ceph fs dump --format=json 2024-08-19T01:43:57.053 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/bad8780e-5dc9-11ef-bccf-c7b262605968/mon.a/config 2024-08-19T01:43:57.593 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:57.592+0000 7f3e3ec00640 1 -- 172.21.15.83:0/902185999 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3e38104480 msgr2=0x7f3e381109b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:43:57.593 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:57.592+0000 7f3e3ec00640 1 --2- 172.21.15.83:0/902185999 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3e38104480 0x7f3e381109b0 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f3e30009cd0 tx=0x7f3e3002a8e0 comp rx=0 tx=0).stop 2024-08-19T01:43:57.593 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:57.592+0000 7f3e3ec00640 1 -- 172.21.15.83:0/902185999 shutdown_connections 2024-08-19T01:43:57.594 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:57.592+0000 7f3e3ec00640 1 --2- 172.21.15.83:0/902185999 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3e38104480 0x7f3e381109b0 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:43:57.594 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:57.592+0000 7f3e3ec00640 1 --2- 172.21.15.83:0/902185999 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3e38103ae0 0x7f3e38103f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:43:57.594 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:57.592+0000 7f3e3ec00640 1 --2- 172.21.15.83:0/902185999 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3e38109b70 0x7f3e38109f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:43:57.594 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:57.592+0000 7f3e3ec00640 1 -- 172.21.15.83:0/902185999 >> 172.21.15.83:0/902185999 conn(0x7f3e380ff780 msgr2=0x7f3e38101ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:43:57.594 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:57.592+0000 7f3e3ec00640 1 -- 172.21.15.83:0/902185999 shutdown_connections 2024-08-19T01:43:57.594 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:57.592+0000 7f3e3ec00640 1 -- 172.21.15.83:0/902185999 wait complete. 2024-08-19T01:43:57.598 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:57.593+0000 7f3e3ec00640 1 Processor -- start 2024-08-19T01:43:57.598 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:57.593+0000 7f3e3ec00640 1 -- start start 2024-08-19T01:43:57.598 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:57.594+0000 7f3e3ec00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3e38103ae0 0x7f3e3819a350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:43:57.598 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:57.594+0000 7f3e3ec00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3e38104480 0x7f3e3819a890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:43:57.598 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:57.594+0000 7f3e3ec00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3e38109b70 0x7f3e3819ec20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:43:57.599 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:57.594+0000 7f3e3ec00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f3e38113f30 con 0x7f3e38109b70 2024-08-19T01:43:57.599 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:57.594+0000 7f3e3ec00640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f3e38113db0 con 0x7f3e38104480 2024-08-19T01:43:57.599 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:57.594+0000 7f3e3ec00640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f3e381140b0 con 0x7f3e38103ae0 2024-08-19T01:43:57.599 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:57.594+0000 7f3e37e00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3e38103ae0 0x7f3e3819a350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:43:57.599 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:57.594+0000 7f3e37e00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3e38103ae0 0x7f3e3819a350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.83:40968/0 (socket says 172.21.15.83:40968) 2024-08-19T01:43:57.599 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:57.594+0000 7f3e37e00640 1 -- 172.21.15.83:0/2410689215 learned_addr learned my addr 172.21.15.83:0/2410689215 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-08-19T01:43:57.599 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:57.594+0000 7f3e3cc00640 1 --2- 172.21.15.83:0/2410689215 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3e38109b70 0x7f3e3819ec20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:43:57.599 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:57.594+0000 7f3e2f400640 1 --2- 172.21.15.83:0/2410689215 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3e38104480 0x7f3e3819a890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:43:57.599 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:57.595+0000 7f3e37e00640 1 -- 172.21.15.83:0/2410689215 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3e38104480 msgr2=0x7f3e3819a890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:43:57.599 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:57.595+0000 7f3e37e00640 1 --2- 172.21.15.83:0/2410689215 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3e38104480 0x7f3e3819a890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:43:57.599 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:57.595+0000 7f3e37e00640 1 -- 172.21.15.83:0/2410689215 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3e38109b70 msgr2=0x7f3e3819ec20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:43:57.599 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:57.595+0000 7f3e37e00640 1 --2- 172.21.15.83:0/2410689215 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3e38109b70 0x7f3e3819ec20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:43:57.599 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:57.595+0000 7f3e37e00640 1 -- 172.21.15.83:0/2410689215 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3e3819f300 con 0x7f3e38103ae0 2024-08-19T01:43:57.599 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:57.595+0000 7f3e3cc00640 1 --2- 172.21.15.83:0/2410689215 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3e38109b70 0x7f3e3819ec20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-19T01:43:57.600 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:57.595+0000 7f3e37e00640 1 --2- 172.21.15.83:0/2410689215 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3e38103ae0 0x7f3e3819a350 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f3e2800e9f0 tx=0x7f3e2800eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:43:57.600 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:57.595+0000 7f3e35600640 1 -- 172.21.15.83:0/2410689215 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e2800cd80 con 0x7f3e38103ae0 2024-08-19T01:43:57.600 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:57.595+0000 7f3e3ec00640 1 -- 172.21.15.83:0/2410689215 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3e3819f5f0 con 0x7f3e38103ae0 2024-08-19T01:43:57.600 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:57.595+0000 7f3e35600640 1 -- 172.21.15.83:0/2410689215 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3e28010910 con 0x7f3e38103ae0 2024-08-19T01:43:57.600 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:57.595+0000 7f3e35600640 1 -- 172.21.15.83:0/2410689215 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e280188f0 con 0x7f3e38103ae0 2024-08-19T01:43:57.600 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:57.596+0000 7f3e3ec00640 1 -- 172.21.15.83:0/2410689215 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3e381a6ed0 con 0x7f3e38103ae0 2024-08-19T01:43:57.600 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:57.597+0000 7f3e3ec00640 1 -- 172.21.15.83:0/2410689215 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3e38103f40 con 0x7f3e38103ae0 2024-08-19T01:43:57.601 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:57.599+0000 7f3e35600640 1 -- 172.21.15.83:0/2410689215 <== mon.1 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7f3e28010ac0 con 0x7f3e38103ae0 2024-08-19T01:43:57.601 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:57.600+0000 7f3e35600640 1 --2- 172.21.15.83:0/2410689215 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f3e1c0a2b80 0x7f3e1c0a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:43:57.601 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:57.600+0000 7f3e35600640 1 -- 172.21.15.83:0/2410689215 <== mon.1 v2:172.21.15.167:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8269+0+0 (secure 0 0 0) 0x7f3e280cf7d0 con 0x7f3e38103ae0 2024-08-19T01:43:57.601 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:57.600+0000 7f3e2f400640 1 --2- 172.21.15.83:0/2410689215 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f3e1c0a2b80 0x7f3e1c0a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:43:57.601 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:57.601+0000 7f3e2f400640 1 --2- 172.21.15.83:0/2410689215 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f3e1c0a2b80 0x7f3e1c0a5040 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f3e381091a0 tx=0x7f3e2000b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:43:57.603 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:57.602+0000 7f3e35600640 1 -- 172.21.15.83:0/2410689215 <== mon.1 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7f3e2809c060 con 0x7f3e38103ae0 2024-08-19T01:43:57.813 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:43:57 smithi083 ceph-mon[17120]: pgmap v612: 97 pgs: 97 active+clean; 35 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-08-19T01:43:57.813 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:57.812+0000 7f3e3ec00640 1 -- 172.21.15.83:0/2410689215 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f3e3819b6b0 con 0x7f3e38103ae0 2024-08-19T01:43:57.814 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:57.813+0000 7f3e35600640 1 -- 172.21.15.83:0/2410689215 <== mon.1 v2:172.21.15.167:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 31 v31) ==== 94+0+11922 (secure 0 0 0) 0x7f3e280a0360 con 0x7f3e38103ae0 2024-08-19T01:43:57.814 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-08-19T01:43:57.814 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":31,"btime":"2024-08-19T01:43:15:661376+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14421,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6835/988556067","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":988556067},{"type":"v1","addr":"172.21.15.83:6835","nonce":988556067}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14433,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6837/869702632","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6836","nonce":869702632},{"type":"v1","addr":"172.21.15.83:6837","nonce":869702632}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14475,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6841/1795975999","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6840","nonce":1795975999},{"type":"v1","addr":"172.21.15.83:6841","nonce":1795975999}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24430,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6833/2505076046","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6832","nonce":2505076046},{"type":"v1","addr":"172.21.15.167:6833","nonce":2505076046}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24449,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6833/3557520478","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":3557520478},{"type":"v1","addr":"172.21.15.117:6833","nonce":3557520478}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24466,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6835/3482045966","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":3482045966},{"type":"v1","addr":"172.21.15.167:6835","nonce":3482045966}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7}],"filesystems":[{"mdsmap":{"epoch":31,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-08-19T01:32:42.054769+0000","modified":"2024-08-19T01:43:15.162835+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24545,"mds_1":24527,"mds_2":24509},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24478":{"gid":24478,"name":"c","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.167:6837/1762130408","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6836","nonce":1762130408},{"type":"v1","addr":"172.21.15.167:6837","nonce":1762130408}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24485":{"gid":24485,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.117:6835/1024213994","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":1024213994},{"type":"v1","addr":"172.21.15.117:6835","nonce":1024213994}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24496":{"gid":24496,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.167:6839/3507387682","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6838","nonce":3507387682},{"type":"v1","addr":"172.21.15.167:6839","nonce":3507387682}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24509":{"gid":24509,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":10,"addr":"172.21.15.117:6837/3185224684","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":3185224684},{"type":"v1","addr":"172.21.15.117:6837","nonce":3185224684}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24527":{"gid":24527,"name":"g","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.83:6839/3239261845","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6838","nonce":3239261845},{"type":"v1","addr":"172.21.15.83:6839","nonce":3239261845}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24545":{"gid":24545,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.117:6839/1967595704","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6838","nonce":1967595704},{"type":"v1","addr":"172.21.15.117:6839","nonce":1967595704}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24545,"qdb_cluster":[24545,24527,24509]},"id":1}]} 2024-08-19T01:43:57.814 INFO:teuthology.orchestra.run.smithi083.stderr:dumped fsmap epoch 31 2024-08-19T01:43:57.816 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:57.815+0000 7f3e3ec00640 1 -- 172.21.15.83:0/2410689215 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f3e1c0a2b80 msgr2=0x7f3e1c0a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:43:57.816 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:57.815+0000 7f3e3ec00640 1 --2- 172.21.15.83:0/2410689215 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f3e1c0a2b80 0x7f3e1c0a5040 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f3e381091a0 tx=0x7f3e2000b410 comp rx=0 tx=0).stop 2024-08-19T01:43:57.816 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:57.815+0000 7f3e3ec00640 1 -- 172.21.15.83:0/2410689215 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3e38103ae0 msgr2=0x7f3e3819a350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:43:57.816 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:57.815+0000 7f3e3ec00640 1 --2- 172.21.15.83:0/2410689215 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3e38103ae0 0x7f3e3819a350 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f3e2800e9f0 tx=0x7f3e2800eeb0 comp rx=0 tx=0).stop 2024-08-19T01:43:57.816 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:57.815+0000 7f3e3ec00640 1 -- 172.21.15.83:0/2410689215 shutdown_connections 2024-08-19T01:43:57.816 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:57.815+0000 7f3e3ec00640 1 --2- 172.21.15.83:0/2410689215 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f3e1c0a2b80 0x7f3e1c0a5040 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:43:57.816 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:57.815+0000 7f3e3ec00640 1 --2- 172.21.15.83:0/2410689215 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3e38109b70 0x7f3e3819ec20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:43:57.816 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:57.815+0000 7f3e3ec00640 1 --2- 172.21.15.83:0/2410689215 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3e38104480 0x7f3e3819a890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:43:57.816 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:57.815+0000 7f3e3ec00640 1 --2- 172.21.15.83:0/2410689215 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3e38103ae0 0x7f3e3819a350 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:43:57.817 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:57.815+0000 7f3e3ec00640 1 -- 172.21.15.83:0/2410689215 >> 172.21.15.83:0/2410689215 conn(0x7f3e380ff780 msgr2=0x7f3e38100ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:43:57.817 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:57.815+0000 7f3e3ec00640 1 -- 172.21.15.83:0/2410689215 shutdown_connections 2024-08-19T01:43:57.817 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:57.816+0000 7f3e3ec00640 1 -- 172.21.15.83:0/2410689215 wait complete. 2024-08-19T01:43:57.952 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:43:57 smithi117 ceph-mon[22016]: pgmap v612: 97 pgs: 97 active+clean; 35 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-08-19T01:43:57.966 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:43:57 smithi167 ceph-mon[21875]: pgmap v612: 97 pgs: 97 active+clean; 35 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-08-19T01:43:58.634 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833 shell --fsid bad8780e-5dc9-11ef-bccf-c7b262605968 -- ceph fs dump --format=json 2024-08-19T01:43:58.904 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:43:58 smithi083 ceph-mon[17120]: from='client.? 172.21.15.83:0/2410689215' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-19T01:43:58.952 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:43:58 smithi117 ceph-mon[22016]: from='client.? 172.21.15.83:0/2410689215' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-19T01:43:58.965 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:43:58 smithi167 ceph-mon[21875]: from='client.? 172.21.15.83:0/2410689215' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-19T01:43:59.013 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/bad8780e-5dc9-11ef-bccf-c7b262605968/mon.a/config 2024-08-19T01:43:59.618 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:59.615+0000 7f4ee1400640 1 -- 172.21.15.83:0/1522864750 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4edc109ac0 msgr2=0x7f4edc109ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:43:59.618 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:59.615+0000 7f4ee1400640 1 --2- 172.21.15.83:0/1522864750 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4edc109ac0 0x7f4edc109ea0 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f4ecc009cd0 tx=0x7f4ecc02a8e0 comp rx=0 tx=0).stop 2024-08-19T01:43:59.618 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:59.616+0000 7f4ee1400640 1 -- 172.21.15.83:0/1522864750 shutdown_connections 2024-08-19T01:43:59.618 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:59.616+0000 7f4ee1400640 1 --2- 172.21.15.83:0/1522864750 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4edc104460 0x7f4edc110990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:43:59.618 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:59.616+0000 7f4ee1400640 1 --2- 172.21.15.83:0/1522864750 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4edc103ac0 0x7f4edc103f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:43:59.618 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:59.616+0000 7f4ee1400640 1 --2- 172.21.15.83:0/1522864750 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4edc109ac0 0x7f4edc109ea0 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:43:59.619 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:59.616+0000 7f4ee1400640 1 -- 172.21.15.83:0/1522864750 >> 172.21.15.83:0/1522864750 conn(0x7f4edc0ff7c0 msgr2=0x7f4edc101be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:43:59.619 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:59.616+0000 7f4ee1400640 1 -- 172.21.15.83:0/1522864750 shutdown_connections 2024-08-19T01:43:59.619 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:59.616+0000 7f4ee1400640 1 -- 172.21.15.83:0/1522864750 wait complete. 2024-08-19T01:43:59.620 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:59.617+0000 7f4ee1400640 1 Processor -- start 2024-08-19T01:43:59.621 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:59.617+0000 7f4ee1400640 1 -- start start 2024-08-19T01:43:59.621 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:59.617+0000 7f4ee1400640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4edc103ac0 0x7f4edc19a390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:43:59.621 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:59.617+0000 7f4ee1400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4edc104460 0x7f4edc19a8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:43:59.621 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:59.617+0000 7f4ee1400640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4edc109ac0 0x7f4edc19ec60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:43:59.621 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:59.617+0000 7f4ee1400640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f4edc113f10 con 0x7f4edc104460 2024-08-19T01:43:59.621 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:59.617+0000 7f4ee1400640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f4edc113d90 con 0x7f4edc109ac0 2024-08-19T01:43:59.621 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:59.617+0000 7f4ee1400640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f4edc114090 con 0x7f4edc103ac0 2024-08-19T01:43:59.621 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:59.618+0000 7f4ed9e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4edc104460 0x7f4edc19a8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:43:59.621 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:59.618+0000 7f4eda800640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4edc103ac0 0x7f4edc19a390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:43:59.621 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:59.618+0000 7f4edb200640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4edc109ac0 0x7f4edc19ec60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:43:59.621 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:59.618+0000 7f4eda800640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4edc103ac0 0x7f4edc19a390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.83:40990/0 (socket says 172.21.15.83:40990) 2024-08-19T01:43:59.622 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:59.618+0000 7f4ed9e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4edc104460 0x7f4edc19a8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:34044/0 (socket says 172.21.15.83:34044) 2024-08-19T01:43:59.622 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:59.618+0000 7f4eda800640 1 -- 172.21.15.83:0/3327690839 learned_addr learned my addr 172.21.15.83:0/3327690839 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-08-19T01:43:59.622 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:59.618+0000 7f4edb200640 1 -- 172.21.15.83:0/3327690839 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4edc103ac0 msgr2=0x7f4edc19a390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:43:59.622 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:59.618+0000 7f4edb200640 1 --2- 172.21.15.83:0/3327690839 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4edc103ac0 0x7f4edc19a390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:43:59.622 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:59.618+0000 7f4edb200640 1 -- 172.21.15.83:0/3327690839 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4edc104460 msgr2=0x7f4edc19a8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:43:59.622 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:59.618+0000 7f4edb200640 1 --2- 172.21.15.83:0/3327690839 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4edc104460 0x7f4edc19a8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:43:59.622 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:59.618+0000 7f4edb200640 1 -- 172.21.15.83:0/3327690839 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4edc19f340 con 0x7f4edc109ac0 2024-08-19T01:43:59.622 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:59.618+0000 7f4ed9e00640 1 --2- 172.21.15.83:0/3327690839 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4edc104460 0x7f4edc19a8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-19T01:43:59.622 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:59.619+0000 7f4edb200640 1 --2- 172.21.15.83:0/3327690839 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4edc109ac0 0x7f4edc19ec60 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f4ed400d680 tx=0x7f4ed400db40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:43:59.622 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:59.619+0000 7f4ed3400640 1 -- 172.21.15.83:0/3327690839 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ed4017070 con 0x7f4edc109ac0 2024-08-19T01:43:59.623 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:59.619+0000 7f4ee1400640 1 -- 172.21.15.83:0/3327690839 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4edc19f630 con 0x7f4edc109ac0 2024-08-19T01:43:59.623 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:59.619+0000 7f4ed3400640 1 -- 172.21.15.83:0/3327690839 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4ed4009820 con 0x7f4edc109ac0 2024-08-19T01:43:59.623 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:59.619+0000 7f4ed3400640 1 -- 172.21.15.83:0/3327690839 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ed4004e00 con 0x7f4edc109ac0 2024-08-19T01:43:59.623 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:59.619+0000 7f4ee1400640 1 -- 172.21.15.83:0/3327690839 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4edc1a6f10 con 0x7f4edc109ac0 2024-08-19T01:43:59.623 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:59.620+0000 7f4ee1400640 1 -- 172.21.15.83:0/3327690839 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4edc103f20 con 0x7f4edc109ac0 2024-08-19T01:43:59.623 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:59.622+0000 7f4ed3400640 1 -- 172.21.15.83:0/3327690839 <== mon.2 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7f4ed40099d0 con 0x7f4edc109ac0 2024-08-19T01:43:59.625 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:59.624+0000 7f4ed3400640 1 --2- 172.21.15.83:0/3327690839 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f4ec00a2b80 0x7f4ec00a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:43:59.625 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:59.624+0000 7f4ed3400640 1 -- 172.21.15.83:0/3327690839 <== mon.2 v2:172.21.15.117:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8269+0+0 (secure 0 0 0) 0x7f4ed40ce840 con 0x7f4edc109ac0 2024-08-19T01:43:59.625 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:59.624+0000 7f4eda800640 1 --2- 172.21.15.83:0/3327690839 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f4ec00a2b80 0x7f4ec00a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:43:59.626 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:59.625+0000 7f4eda800640 1 --2- 172.21.15.83:0/3327690839 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f4ec00a2b80 0x7f4ec00a5040 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f4ecc000f80 tx=0x7f4ecc032fd0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:43:59.627 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:59.626+0000 7f4ed3400640 1 -- 172.21.15.83:0/3327690839 <== mon.2 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7f4ed409b0d0 con 0x7f4edc109ac0 2024-08-19T01:43:59.836 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:59.834+0000 7f4ee1400640 1 -- 172.21.15.83:0/3327690839 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f4edc10c030 con 0x7f4edc109ac0 2024-08-19T01:43:59.837 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:43:59 smithi083 ceph-mon[17120]: pgmap v613: 97 pgs: 97 active+clean; 35 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 255 B/s wr, 6 op/s 2024-08-19T01:43:59.837 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:59.835+0000 7f4ed3400640 1 -- 172.21.15.83:0/3327690839 <== mon.2 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 31 v31) ==== 94+0+11922 (secure 0 0 0) 0x7f4ed409f3d0 con 0x7f4edc109ac0 2024-08-19T01:43:59.838 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-08-19T01:43:59.838 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":31,"btime":"2024-08-19T01:43:15:661376+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14421,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6835/988556067","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":988556067},{"type":"v1","addr":"172.21.15.83:6835","nonce":988556067}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14433,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6837/869702632","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6836","nonce":869702632},{"type":"v1","addr":"172.21.15.83:6837","nonce":869702632}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14475,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6841/1795975999","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6840","nonce":1795975999},{"type":"v1","addr":"172.21.15.83:6841","nonce":1795975999}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24430,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6833/2505076046","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6832","nonce":2505076046},{"type":"v1","addr":"172.21.15.167:6833","nonce":2505076046}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24449,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6833/3557520478","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":3557520478},{"type":"v1","addr":"172.21.15.117:6833","nonce":3557520478}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24466,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6835/3482045966","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":3482045966},{"type":"v1","addr":"172.21.15.167:6835","nonce":3482045966}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7}],"filesystems":[{"mdsmap":{"epoch":31,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-08-19T01:32:42.054769+0000","modified":"2024-08-19T01:43:15.162835+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24545,"mds_1":24527,"mds_2":24509},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24478":{"gid":24478,"name":"c","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.167:6837/1762130408","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6836","nonce":1762130408},{"type":"v1","addr":"172.21.15.167:6837","nonce":1762130408}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24485":{"gid":24485,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.117:6835/1024213994","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":1024213994},{"type":"v1","addr":"172.21.15.117:6835","nonce":1024213994}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24496":{"gid":24496,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.167:6839/3507387682","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6838","nonce":3507387682},{"type":"v1","addr":"172.21.15.167:6839","nonce":3507387682}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24509":{"gid":24509,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":10,"addr":"172.21.15.117:6837/3185224684","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":3185224684},{"type":"v1","addr":"172.21.15.117:6837","nonce":3185224684}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24527":{"gid":24527,"name":"g","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.83:6839/3239261845","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6838","nonce":3239261845},{"type":"v1","addr":"172.21.15.83:6839","nonce":3239261845}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24545":{"gid":24545,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.117:6839/1967595704","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6838","nonce":1967595704},{"type":"v1","addr":"172.21.15.117:6839","nonce":1967595704}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24545,"qdb_cluster":[24545,24527,24509]},"id":1}]} 2024-08-19T01:43:59.838 INFO:teuthology.orchestra.run.smithi083.stderr:dumped fsmap epoch 31 2024-08-19T01:43:59.839 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:59.838+0000 7f4ee1400640 1 -- 172.21.15.83:0/3327690839 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f4ec00a2b80 msgr2=0x7f4ec00a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:43:59.839 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:59.838+0000 7f4ee1400640 1 --2- 172.21.15.83:0/3327690839 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f4ec00a2b80 0x7f4ec00a5040 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f4ecc000f80 tx=0x7f4ecc032fd0 comp rx=0 tx=0).stop 2024-08-19T01:43:59.839 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:59.838+0000 7f4ee1400640 1 -- 172.21.15.83:0/3327690839 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4edc109ac0 msgr2=0x7f4edc19ec60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:43:59.840 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:59.838+0000 7f4ee1400640 1 --2- 172.21.15.83:0/3327690839 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4edc109ac0 0x7f4edc19ec60 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f4ed400d680 tx=0x7f4ed400db40 comp rx=0 tx=0).stop 2024-08-19T01:43:59.840 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:59.838+0000 7f4ee1400640 1 -- 172.21.15.83:0/3327690839 shutdown_connections 2024-08-19T01:43:59.840 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:59.838+0000 7f4ee1400640 1 --2- 172.21.15.83:0/3327690839 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f4ec00a2b80 0x7f4ec00a5040 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:43:59.840 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:59.838+0000 7f4ee1400640 1 --2- 172.21.15.83:0/3327690839 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4edc109ac0 0x7f4edc19ec60 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:43:59.840 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:59.838+0000 7f4ee1400640 1 --2- 172.21.15.83:0/3327690839 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4edc104460 0x7f4edc19a8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:43:59.840 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:59.838+0000 7f4ee1400640 1 --2- 172.21.15.83:0/3327690839 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4edc103ac0 0x7f4edc19a390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:43:59.840 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:59.838+0000 7f4ee1400640 1 -- 172.21.15.83:0/3327690839 >> 172.21.15.83:0/3327690839 conn(0x7f4edc0ff7c0 msgr2=0x7f4edc0fffb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:43:59.840 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:59.838+0000 7f4ee1400640 1 -- 172.21.15.83:0/3327690839 shutdown_connections 2024-08-19T01:43:59.840 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:43:59.838+0000 7f4ee1400640 1 -- 172.21.15.83:0/3327690839 wait complete. 2024-08-19T01:43:59.952 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:43:59 smithi117 ceph-mon[22016]: pgmap v613: 97 pgs: 97 active+clean; 35 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 255 B/s wr, 6 op/s 2024-08-19T01:43:59.965 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:43:59 smithi167 ceph-mon[21875]: pgmap v613: 97 pgs: 97 active+clean; 35 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 255 B/s wr, 6 op/s 2024-08-19T01:44:00.720 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833 shell --fsid bad8780e-5dc9-11ef-bccf-c7b262605968 -- ceph osd dump --format=json 2024-08-19T01:44:00.952 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:44:00 smithi117 ceph-mon[22016]: from='client.? 172.21.15.83:0/3327690839' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-19T01:44:00.965 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:44:00 smithi167 ceph-mon[21875]: from='client.? 172.21.15.83:0/3327690839' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-19T01:44:01.093 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/bad8780e-5dc9-11ef-bccf-c7b262605968/mon.a/config 2024-08-19T01:44:01.154 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:44:00 smithi083 ceph-mon[17120]: from='client.? 172.21.15.83:0/3327690839' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-19T01:44:01.691 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:44:01.673+0000 7fce1a400640 1 -- 172.21.15.83:0/3325502990 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fce14103900 msgr2=0x7fce14103d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:44:01.691 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:44:01.673+0000 7fce1a400640 1 --2- 172.21.15.83:0/3325502990 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fce14103900 0x7fce14103d60 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7fce00009930 tx=0x7fce0002a420 comp rx=0 tx=0).stop 2024-08-19T01:44:01.692 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:44:01.673+0000 7fce1a400640 1 -- 172.21.15.83:0/3325502990 shutdown_connections 2024-08-19T01:44:01.692 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:44:01.673+0000 7fce1a400640 1 --2- 172.21.15.83:0/3325502990 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fce141042a0 0x7fce141107d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:44:01.692 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:44:01.673+0000 7fce1a400640 1 --2- 172.21.15.83:0/3325502990 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fce14103900 0x7fce14103d60 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:44:01.692 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:44:01.673+0000 7fce1a400640 1 --2- 172.21.15.83:0/3325502990 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fce14109900 0x7fce14109ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:44:01.692 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:44:01.673+0000 7fce1a400640 1 -- 172.21.15.83:0/3325502990 >> 172.21.15.83:0/3325502990 conn(0x7fce140ff600 msgr2=0x7fce14101a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:44:01.692 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:44:01.673+0000 7fce1a400640 1 -- 172.21.15.83:0/3325502990 shutdown_connections 2024-08-19T01:44:01.692 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:44:01.673+0000 7fce1a400640 1 -- 172.21.15.83:0/3325502990 wait complete. 2024-08-19T01:44:01.692 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:44:01.674+0000 7fce1a400640 1 Processor -- start 2024-08-19T01:44:01.692 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:44:01.674+0000 7fce1a400640 1 -- start start 2024-08-19T01:44:01.692 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:44:01.675+0000 7fce1a400640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fce14103900 0x7fce1419a250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:44:01.692 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:44:01.675+0000 7fce1a400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fce141042a0 0x7fce1419a790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:44:01.692 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:44:01.675+0000 7fce1a400640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fce14109900 0x7fce1419eb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:44:01.692 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:44:01.675+0000 7fce1a400640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fce14113d50 con 0x7fce141042a0 2024-08-19T01:44:01.693 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:44:01.675+0000 7fce1a400640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fce14113bd0 con 0x7fce14109900 2024-08-19T01:44:01.693 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:44:01.675+0000 7fce1a400640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fce14113ed0 con 0x7fce14103900 2024-08-19T01:44:01.693 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:44:01.675+0000 7fce13e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fce141042a0 0x7fce1419a790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:44:01.693 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:44:01.675+0000 7fce13e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fce141042a0 0x7fce1419a790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:59468/0 (socket says 172.21.15.83:59468) 2024-08-19T01:44:01.693 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:44:01.675+0000 7fce13e00640 1 -- 172.21.15.83:0/1521032002 learned_addr learned my addr 172.21.15.83:0/1521032002 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-08-19T01:44:01.693 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:44:01.675+0000 7fce19a00640 1 --2- 172.21.15.83:0/1521032002 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fce14109900 0x7fce1419eb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:44:01.693 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:44:01.675+0000 7fce19000640 1 --2- 172.21.15.83:0/1521032002 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fce14103900 0x7fce1419a250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:44:01.693 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:44:01.675+0000 7fce13e00640 1 -- 172.21.15.83:0/1521032002 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fce14103900 msgr2=0x7fce1419a250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:44:01.693 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:44:01.675+0000 7fce13e00640 1 --2- 172.21.15.83:0/1521032002 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fce14103900 0x7fce1419a250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:44:01.693 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:44:01.675+0000 7fce13e00640 1 -- 172.21.15.83:0/1521032002 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fce14109900 msgr2=0x7fce1419eb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:44:01.693 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:44:01.675+0000 7fce13e00640 1 --2- 172.21.15.83:0/1521032002 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fce14109900 0x7fce1419eb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:44:01.693 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:44:01.675+0000 7fce13e00640 1 -- 172.21.15.83:0/1521032002 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fce1419f200 con 0x7fce141042a0 2024-08-19T01:44:01.693 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:44:01.676+0000 7fce13e00640 1 --2- 172.21.15.83:0/1521032002 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fce141042a0 0x7fce1419a790 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7fce0002a940 tx=0x7fce00003b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:44:01.694 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:44:01.676+0000 7fce11600640 1 -- 172.21.15.83:0/1521032002 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce0003f070 con 0x7fce141042a0 2024-08-19T01:44:01.694 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:44:01.676+0000 7fce11600640 1 -- 172.21.15.83:0/1521032002 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fce00031c90 con 0x7fce141042a0 2024-08-19T01:44:01.694 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:44:01.676+0000 7fce11600640 1 -- 172.21.15.83:0/1521032002 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce00035030 con 0x7fce141042a0 2024-08-19T01:44:01.694 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:44:01.676+0000 7fce1a400640 1 -- 172.21.15.83:0/1521032002 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fce1419f490 con 0x7fce141042a0 2024-08-19T01:44:01.694 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:44:01.676+0000 7fce1a400640 1 -- 172.21.15.83:0/1521032002 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fce141a6dd0 con 0x7fce141042a0 2024-08-19T01:44:01.694 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:44:01.677+0000 7fce1a400640 1 -- 172.21.15.83:0/1521032002 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fce14103d60 con 0x7fce141042a0 2024-08-19T01:44:01.694 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:44:01.679+0000 7fce11600640 1 -- 172.21.15.83:0/1521032002 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7fce00042070 con 0x7fce141042a0 2024-08-19T01:44:01.694 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:44:01.680+0000 7fce11600640 1 --2- 172.21.15.83:0/1521032002 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7fcdf80a2c00 0x7fcdf80a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:44:01.694 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:44:01.680+0000 7fce11600640 1 -- 172.21.15.83:0/1521032002 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8269+0+0 (secure 0 0 0) 0x7fce000ef9a0 con 0x7fce141042a0 2024-08-19T01:44:01.694 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:44:01.681+0000 7fce19000640 1 --2- 172.21.15.83:0/1521032002 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7fcdf80a2c00 0x7fcdf80a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:44:01.694 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:44:01.681+0000 7fce19000640 1 --2- 172.21.15.83:0/1521032002 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7fcdf80a2c00 0x7fcdf80a50c0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7fce04009b50 tx=0x7fce04005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:44:01.694 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:44:01.684+0000 7fce11600640 1 -- 172.21.15.83:0/1521032002 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7fce000bc880 con 0x7fce141042a0 2024-08-19T01:44:01.863 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:44:01.862+0000 7fce1a400640 1 -- 172.21.15.83:0/1521032002 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fce1410bea0 con 0x7fce141042a0 2024-08-19T01:44:01.863 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:44:01 smithi083 ceph-mon[17120]: pgmap v614: 97 pgs: 97 active+clean; 35 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 767 B/s wr, 8 op/s 2024-08-19T01:44:01.865 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:44:01.863+0000 7fce11600640 1 -- 172.21.15.83:0/1521032002 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v86) ==== 74+0+22722 (secure 0 0 0) 0x7fce000bc180 con 0x7fce141042a0 2024-08-19T01:44:01.865 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-08-19T01:44:01.866 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":86,"fsid":"bad8780e-5dc9-11ef-bccf-c7b262605968","created":"2024-08-19T01:25:12.626841+0000","modified":"2024-08-19T01:33:05.461998+0000","last_up_change":"2024-08-19T01:31:47.254559+0000","last_in_change":"2024-08-19T01:31:27.652847+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-08-19T01:28:16.862007+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-08-19T01:32:37.455535+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-08-19T01:32:40.201935+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ffa74041-f9d2-4555-bb43-508b8c581487","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6802","nonce":1082526824},{"type":"v1","addr":"172.21.15.83:6803","nonce":1082526824}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6804","nonce":1082526824},{"type":"v1","addr":"172.21.15.83:6805","nonce":1082526824}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6808","nonce":1082526824},{"type":"v1","addr":"172.21.15.83:6809","nonce":1082526824}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6806","nonce":1082526824},{"type":"v1","addr":"172.21.15.83:6807","nonce":1082526824}]},"public_addr":"172.21.15.83:6803/1082526824","cluster_addr":"172.21.15.83:6805/1082526824","heartbeat_back_addr":"172.21.15.83:6809/1082526824","heartbeat_front_addr":"172.21.15.83:6807/1082526824","state":["exists","up"]},{"osd":1,"uuid":"1977570b-b4f5-44fd-bb1e-3a31318be649","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6800","nonce":1925419802},{"type":"v1","addr":"172.21.15.117:6801","nonce":1925419802}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6802","nonce":1925419802},{"type":"v1","addr":"172.21.15.117:6803","nonce":1925419802}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6806","nonce":1925419802},{"type":"v1","addr":"172.21.15.117:6807","nonce":1925419802}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6804","nonce":1925419802},{"type":"v1","addr":"172.21.15.117:6805","nonce":1925419802}]},"public_addr":"172.21.15.117:6801/1925419802","cluster_addr":"172.21.15.117:6803/1925419802","heartbeat_back_addr":"172.21.15.117:6807/1925419802","heartbeat_front_addr":"172.21.15.117:6805/1925419802","state":["exists","up"]},{"osd":2,"uuid":"a75278d1-161f-41d5-a92f-058d1077bf2a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6800","nonce":551625328},{"type":"v1","addr":"172.21.15.167:6801","nonce":551625328}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6802","nonce":551625328},{"type":"v1","addr":"172.21.15.167:6803","nonce":551625328}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6806","nonce":551625328},{"type":"v1","addr":"172.21.15.167:6807","nonce":551625328}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6804","nonce":551625328},{"type":"v1","addr":"172.21.15.167:6805","nonce":551625328}]},"public_addr":"172.21.15.167:6801/551625328","cluster_addr":"172.21.15.167:6803/551625328","heartbeat_back_addr":"172.21.15.167:6807/551625328","heartbeat_front_addr":"172.21.15.167:6805/551625328","state":["exists","up"]},{"osd":3,"uuid":"d1fb16f9-910e-46fa-b1e3-8e5bef4d346d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6810","nonce":3138942062},{"type":"v1","addr":"172.21.15.83:6811","nonce":3138942062}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6812","nonce":3138942062},{"type":"v1","addr":"172.21.15.83:6813","nonce":3138942062}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6816","nonce":3138942062},{"type":"v1","addr":"172.21.15.83:6817","nonce":3138942062}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6814","nonce":3138942062},{"type":"v1","addr":"172.21.15.83:6815","nonce":3138942062}]},"public_addr":"172.21.15.83:6811/3138942062","cluster_addr":"172.21.15.83:6813/3138942062","heartbeat_back_addr":"172.21.15.83:6817/3138942062","heartbeat_front_addr":"172.21.15.83:6815/3138942062","state":["exists","up"]},{"osd":4,"uuid":"8176ed18-1fe2-446d-8994-8c60f6e08f38","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6808","nonce":3978863827},{"type":"v1","addr":"172.21.15.117:6809","nonce":3978863827}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6810","nonce":3978863827},{"type":"v1","addr":"172.21.15.117:6811","nonce":3978863827}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6814","nonce":3978863827},{"type":"v1","addr":"172.21.15.117:6815","nonce":3978863827}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6812","nonce":3978863827},{"type":"v1","addr":"172.21.15.117:6813","nonce":3978863827}]},"public_addr":"172.21.15.117:6809/3978863827","cluster_addr":"172.21.15.117:6811/3978863827","heartbeat_back_addr":"172.21.15.117:6815/3978863827","heartbeat_front_addr":"172.21.15.117:6813/3978863827","state":["exists","up"]},{"osd":5,"uuid":"e01e3f15-25c8-4d2c-be6f-a170f036064c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6808","nonce":2761858035},{"type":"v1","addr":"172.21.15.167:6809","nonce":2761858035}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6810","nonce":2761858035},{"type":"v1","addr":"172.21.15.167:6811","nonce":2761858035}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6814","nonce":2761858035},{"type":"v1","addr":"172.21.15.167:6815","nonce":2761858035}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6812","nonce":2761858035},{"type":"v1","addr":"172.21.15.167:6813","nonce":2761858035}]},"public_addr":"172.21.15.167:6809/2761858035","cluster_addr":"172.21.15.167:6811/2761858035","heartbeat_back_addr":"172.21.15.167:6815/2761858035","heartbeat_front_addr":"172.21.15.167:6813/2761858035","state":["exists","up"]},{"osd":6,"uuid":"78cf0a9b-9165-444f-b073-ebe13016de9d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6818","nonce":260644552},{"type":"v1","addr":"172.21.15.83:6819","nonce":260644552}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6820","nonce":260644552},{"type":"v1","addr":"172.21.15.83:6821","nonce":260644552}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6824","nonce":260644552},{"type":"v1","addr":"172.21.15.83:6825","nonce":260644552}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6822","nonce":260644552},{"type":"v1","addr":"172.21.15.83:6823","nonce":260644552}]},"public_addr":"172.21.15.83:6819/260644552","cluster_addr":"172.21.15.83:6821/260644552","heartbeat_back_addr":"172.21.15.83:6825/260644552","heartbeat_front_addr":"172.21.15.83:6823/260644552","state":["exists","up"]},{"osd":7,"uuid":"ee524f7f-672e-430e-9f58-1a9117513466","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6816","nonce":104113252},{"type":"v1","addr":"172.21.15.117:6817","nonce":104113252}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6818","nonce":104113252},{"type":"v1","addr":"172.21.15.117:6819","nonce":104113252}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6822","nonce":104113252},{"type":"v1","addr":"172.21.15.117:6823","nonce":104113252}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6820","nonce":104113252},{"type":"v1","addr":"172.21.15.117:6821","nonce":104113252}]},"public_addr":"172.21.15.117:6817/104113252","cluster_addr":"172.21.15.117:6819/104113252","heartbeat_back_addr":"172.21.15.117:6823/104113252","heartbeat_front_addr":"172.21.15.117:6821/104113252","state":["exists","up"]},{"osd":8,"uuid":"76d5317d-4845-4138-b210-e7bd2d80472a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":58,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6816","nonce":2196898857},{"type":"v1","addr":"172.21.15.167:6817","nonce":2196898857}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6818","nonce":2196898857},{"type":"v1","addr":"172.21.15.167:6819","nonce":2196898857}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6822","nonce":2196898857},{"type":"v1","addr":"172.21.15.167:6823","nonce":2196898857}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6820","nonce":2196898857},{"type":"v1","addr":"172.21.15.167:6821","nonce":2196898857}]},"public_addr":"172.21.15.167:6817/2196898857","cluster_addr":"172.21.15.167:6819/2196898857","heartbeat_back_addr":"172.21.15.167:6823/2196898857","heartbeat_front_addr":"172.21.15.167:6821/2196898857","state":["exists","up"]},{"osd":9,"uuid":"0151bc2a-3ad2-4db3-8505-5855792cc9bb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6826","nonce":903463450},{"type":"v1","addr":"172.21.15.83:6827","nonce":903463450}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6828","nonce":903463450},{"type":"v1","addr":"172.21.15.83:6829","nonce":903463450}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6832","nonce":903463450},{"type":"v1","addr":"172.21.15.83:6833","nonce":903463450}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6830","nonce":903463450},{"type":"v1","addr":"172.21.15.83:6831","nonce":903463450}]},"public_addr":"172.21.15.83:6827/903463450","cluster_addr":"172.21.15.83:6829/903463450","heartbeat_back_addr":"172.21.15.83:6833/903463450","heartbeat_front_addr":"172.21.15.83:6831/903463450","state":["exists","up"]},{"osd":10,"uuid":"437ed848-0da4-4cd4-8716-0c55bb99b676","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6824","nonce":1183570837},{"type":"v1","addr":"172.21.15.117:6825","nonce":1183570837}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6826","nonce":1183570837},{"type":"v1","addr":"172.21.15.117:6827","nonce":1183570837}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6830","nonce":1183570837},{"type":"v1","addr":"172.21.15.117:6831","nonce":1183570837}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6828","nonce":1183570837},{"type":"v1","addr":"172.21.15.117:6829","nonce":1183570837}]},"public_addr":"172.21.15.117:6825/1183570837","cluster_addr":"172.21.15.117:6827/1183570837","heartbeat_back_addr":"172.21.15.117:6831/1183570837","heartbeat_front_addr":"172.21.15.117:6829/1183570837","state":["exists","up"]},{"osd":11,"uuid":"13345f05-cdc0-4dae-a58b-551abbec1915","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":78,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6824","nonce":187027350},{"type":"v1","addr":"172.21.15.167:6825","nonce":187027350}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6826","nonce":187027350},{"type":"v1","addr":"172.21.15.167:6827","nonce":187027350}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6830","nonce":187027350},{"type":"v1","addr":"172.21.15.167:6831","nonce":187027350}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6828","nonce":187027350},{"type":"v1","addr":"172.21.15.167:6829","nonce":187027350}]},"public_addr":"172.21.15.167:6825/187027350","cluster_addr":"172.21.15.167:6827/187027350","heartbeat_back_addr":"172.21.15.167:6831/187027350","heartbeat_front_addr":"172.21.15.167:6829/187027350","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-19T01:27:28.760517+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-19T01:27:50.493028+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-19T01:28:11.997674+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-19T01:28:38.109934+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-19T01:29:00.173065+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-19T01:29:21.336085+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-19T01:29:47.871498+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-19T01:30:10.720498+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-19T01:30:32.425438+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-19T01:30:59.524989+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-19T01:31:22.806162+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-19T01:31:45.043246+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.1a","mappings":[{"from":8,"to":1}]},{"pgid":"2.1b","mappings":[{"from":2,"to":4}]},{"pgid":"2.1d","mappings":[{"from":8,"to":11}]},{"pgid":"3.6","mappings":[{"from":4,"to":6}]},{"pgid":"3.11","mappings":[{"from":4,"to":8}]},{"pgid":"3.1b","mappings":[{"from":5,"to":9}]},{"pgid":"3.20","mappings":[{"from":5,"to":9}]},{"pgid":"3.31","mappings":[{"from":4,"to":7}]},{"pgid":"3.3b","mappings":[{"from":3,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.83:0/3138095466":"2024-08-20T01:26:04.277716+0000","172.21.15.83:0/3835005198":"2024-08-20T01:26:04.277716+0000","172.21.15.83:0/1790374088":"2024-08-20T01:25:37.228583+0000","172.21.15.83:6801/573907319":"2024-08-20T01:26:04.277716+0000","172.21.15.83:6800/573907319":"2024-08-20T01:26:04.277716+0000","172.21.15.83:0/664676002":"2024-08-20T01:25:37.228583+0000","172.21.15.83:0/2144611449":"2024-08-20T01:25:37.228583+0000","172.21.15.83:6801/3261152818":"2024-08-20T01:25:37.228583+0000","172.21.15.83:0/1609552666":"2024-08-20T01:26:04.277716+0000","172.21.15.83:6800/3261152818":"2024-08-20T01:25:37.228583+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-08-19T01:44:01.868 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:44:01.866+0000 7fce1a400640 1 -- 172.21.15.83:0/1521032002 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7fcdf80a2c00 msgr2=0x7fcdf80a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:44:01.868 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:44:01.866+0000 7fce1a400640 1 --2- 172.21.15.83:0/1521032002 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7fcdf80a2c00 0x7fcdf80a50c0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7fce04009b50 tx=0x7fce04005f90 comp rx=0 tx=0).stop 2024-08-19T01:44:01.868 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:44:01.866+0000 7fce1a400640 1 -- 172.21.15.83:0/1521032002 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fce141042a0 msgr2=0x7fce1419a790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:44:01.868 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:44:01.866+0000 7fce1a400640 1 --2- 172.21.15.83:0/1521032002 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fce141042a0 0x7fce1419a790 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7fce0002a940 tx=0x7fce00003b20 comp rx=0 tx=0).stop 2024-08-19T01:44:01.868 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:44:01.866+0000 7fce1a400640 1 -- 172.21.15.83:0/1521032002 shutdown_connections 2024-08-19T01:44:01.868 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:44:01.866+0000 7fce1a400640 1 --2- 172.21.15.83:0/1521032002 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7fcdf80a2c00 0x7fcdf80a50c0 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:44:01.868 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:44:01.866+0000 7fce1a400640 1 --2- 172.21.15.83:0/1521032002 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fce14109900 0x7fce1419eb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:44:01.868 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:44:01.866+0000 7fce1a400640 1 --2- 172.21.15.83:0/1521032002 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fce141042a0 0x7fce1419a790 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:44:01.868 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:44:01.866+0000 7fce1a400640 1 --2- 172.21.15.83:0/1521032002 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fce14103900 0x7fce1419a250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:44:01.868 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:44:01.866+0000 7fce1a400640 1 -- 172.21.15.83:0/1521032002 >> 172.21.15.83:0/1521032002 conn(0x7fce140ff600 msgr2=0x7fce140ffe40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:44:01.868 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:44:01.866+0000 7fce1a400640 1 -- 172.21.15.83:0/1521032002 shutdown_connections 2024-08-19T01:44:01.869 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:44:01.866+0000 7fce1a400640 1 -- 172.21.15.83:0/1521032002 wait complete. 2024-08-19T01:44:01.952 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:44:01 smithi117 ceph-mon[22016]: pgmap v614: 97 pgs: 97 active+clean; 35 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 767 B/s wr, 8 op/s 2024-08-19T01:44:01.965 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:44:01 smithi167 ceph-mon[21875]: pgmap v614: 97 pgs: 97 active+clean; 35 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 767 B/s wr, 8 op/s 2024-08-19T01:44:02.601 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833 shell --fsid bad8780e-5dc9-11ef-bccf-c7b262605968 -- ceph fs fail cephfs 2024-08-19T01:44:02.904 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:44:02 smithi083 ceph-mon[17120]: from='client.? 172.21.15.83:0/1521032002' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-08-19T01:44:02.952 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:44:02 smithi117 ceph-mon[22016]: from='client.? 172.21.15.83:0/1521032002' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-08-19T01:44:02.965 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:44:02 smithi167 ceph-mon[21875]: from='client.? 172.21.15.83:0/1521032002' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-08-19T01:44:02.980 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/bad8780e-5dc9-11ef-bccf-c7b262605968/mon.a/config 2024-08-19T01:44:03.535 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:44:03.533+0000 7fe102000640 1 -- 172.21.15.83:0/3899172331 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe0fc109a30 msgr2=0x7fe0fc109e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:44:03.535 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:44:03.533+0000 7fe102000640 1 --2- 172.21.15.83:0/3899172331 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe0fc109a30 0x7fe0fc109e10 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fe0ec009890 tx=0x7fe0ec02a360 comp rx=0 tx=0).stop 2024-08-19T01:44:03.535 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:44:03.533+0000 7fe102000640 1 -- 172.21.15.83:0/3899172331 shutdown_connections 2024-08-19T01:44:03.535 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:44:03.533+0000 7fe102000640 1 --2- 172.21.15.83:0/3899172331 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe0fc1042c0 0x7fe0fc110900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:44:03.535 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:44:03.533+0000 7fe102000640 1 --2- 172.21.15.83:0/3899172331 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe0fc103920 0x7fe0fc103d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:44:03.535 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:44:03.533+0000 7fe102000640 1 --2- 172.21.15.83:0/3899172331 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe0fc109a30 0x7fe0fc109e10 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:44:03.535 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:44:03.533+0000 7fe102000640 1 -- 172.21.15.83:0/3899172331 >> 172.21.15.83:0/3899172331 conn(0x7fe0fc0ff5e0 msgr2=0x7fe0fc101a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:44:03.536 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:44:03.533+0000 7fe102000640 1 -- 172.21.15.83:0/3899172331 shutdown_connections 2024-08-19T01:44:03.536 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:44:03.533+0000 7fe102000640 1 -- 172.21.15.83:0/3899172331 wait complete. 2024-08-19T01:44:03.536 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:44:03.534+0000 7fe102000640 1 Processor -- start 2024-08-19T01:44:03.537 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:44:03.534+0000 7fe102000640 1 -- start start 2024-08-19T01:44:03.537 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:44:03.535+0000 7fe102000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe0fc103920 0x7fe0fc19a330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:44:03.537 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:44:03.535+0000 7fe102000640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe0fc1042c0 0x7fe0fc19a870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:44:03.537 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:44:03.535+0000 7fe102000640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe0fc109a30 0x7fe0fc19ec00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:44:03.537 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:44:03.535+0000 7fe102000640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fe0fc113e30 con 0x7fe0fc103920 2024-08-19T01:44:03.537 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:44:03.535+0000 7fe102000640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fe0fc113cb0 con 0x7fe0fc1042c0 2024-08-19T01:44:03.537 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:44:03.535+0000 7fe102000640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fe0fc113fb0 con 0x7fe0fc109a30 2024-08-19T01:44:03.537 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:44:03.535+0000 7fe0fbe00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe0fc1042c0 0x7fe0fc19a870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:44:03.537 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:44:03.535+0000 7fe101600640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe0fc109a30 0x7fe0fc19ec00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:44:03.537 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:44:03.535+0000 7fe100c00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe0fc103920 0x7fe0fc19a330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:44:03.537 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:44:03.535+0000 7fe0fbe00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe0fc1042c0 0x7fe0fc19a870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.83:54166/0 (socket says 172.21.15.83:54166) 2024-08-19T01:44:03.537 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:44:03.535+0000 7fe101600640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe0fc109a30 0x7fe0fc19ec00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.83:47048/0 (socket says 172.21.15.83:47048) 2024-08-19T01:44:03.537 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:44:03.535+0000 7fe0fbe00640 1 -- 172.21.15.83:0/3085488315 learned_addr learned my addr 172.21.15.83:0/3085488315 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-08-19T01:44:03.538 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:44:03.535+0000 7fe0fbe00640 1 -- 172.21.15.83:0/3085488315 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe0fc109a30 msgr2=0x7fe0fc19ec00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:44:03.539 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:44:03.535+0000 7fe0fbe00640 1 --2- 172.21.15.83:0/3085488315 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe0fc109a30 0x7fe0fc19ec00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:44:03.539 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:44:03.535+0000 7fe0fbe00640 1 -- 172.21.15.83:0/3085488315 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe0fc103920 msgr2=0x7fe0fc19a330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:44:03.539 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:44:03.535+0000 7fe0fbe00640 1 --2- 172.21.15.83:0/3085488315 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe0fc103920 0x7fe0fc19a330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:44:03.539 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:44:03.536+0000 7fe0fbe00640 1 -- 172.21.15.83:0/3085488315 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe0fc19f2e0 con 0x7fe0fc1042c0 2024-08-19T01:44:03.539 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:44:03.536+0000 7fe101600640 1 --2- 172.21.15.83:0/3085488315 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe0fc109a30 0x7fe0fc19ec00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-19T01:44:03.539 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:44:03.536+0000 7fe100c00640 1 --2- 172.21.15.83:0/3085488315 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe0fc103920 0x7fe0fc19a330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-19T01:44:03.539 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:44:03.536+0000 7fe0fbe00640 1 --2- 172.21.15.83:0/3085488315 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe0fc1042c0 0x7fe0fc19a870 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7fe0e800e9c0 tx=0x7fe0e800ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:44:03.539 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:44:03.536+0000 7fe0f9600640 1 -- 172.21.15.83:0/3085488315 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0e800ccc0 con 0x7fe0fc1042c0 2024-08-19T01:44:03.539 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:44:03.536+0000 7fe102000640 1 -- 172.21.15.83:0/3085488315 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe0fc19f5d0 con 0x7fe0fc1042c0 2024-08-19T01:44:03.539 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:44:03.536+0000 7fe0f9600640 1 -- 172.21.15.83:0/3085488315 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe0e8005500 con 0x7fe0fc1042c0 2024-08-19T01:44:03.540 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:44:03.536+0000 7fe0f9600640 1 -- 172.21.15.83:0/3085488315 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0e8010430 con 0x7fe0fc1042c0 2024-08-19T01:44:03.540 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:44:03.536+0000 7fe102000640 1 -- 172.21.15.83:0/3085488315 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe0fc1a6eb0 con 0x7fe0fc1042c0 2024-08-19T01:44:03.540 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:44:03.537+0000 7fe102000640 1 -- 172.21.15.83:0/3085488315 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe0fc103d80 con 0x7fe0fc1042c0 2024-08-19T01:44:03.541 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:44:03.539+0000 7fe0f9600640 1 -- 172.21.15.83:0/3085488315 <== mon.2 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7fe0e80046c0 con 0x7fe0fc1042c0 2024-08-19T01:44:03.542 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:44:03.541+0000 7fe0f9600640 1 --2- 172.21.15.83:0/3085488315 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7fe0e00a2b80 0x7fe0e00a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:44:03.542 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:44:03.541+0000 7fe0f9600640 1 -- 172.21.15.83:0/3085488315 <== mon.2 v2:172.21.15.117:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8269+0+0 (secure 0 0 0) 0x7fe0e80cf5b0 con 0x7fe0fc1042c0 2024-08-19T01:44:03.543 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:44:03.541+0000 7fe100c00640 1 --2- 172.21.15.83:0/3085488315 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7fe0e00a2b80 0x7fe0e00a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:44:03.543 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:44:03.542+0000 7fe100c00640 1 --2- 172.21.15.83:0/3085488315 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7fe0e00a2b80 0x7fe0e00a5040 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7fe0ec02a950 tx=0x7fe0ec0056d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:44:03.544 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:44:03.542+0000 7fe0f9600640 1 -- 172.21.15.83:0/3085488315 <== mon.2 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7fe0e8053df0 con 0x7fe0fc1042c0 2024-08-19T01:44:03.754 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:44:03.753+0000 7fe102000640 1 -- 172.21.15.83:0/3085488315 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7fe0fc105130 con 0x7fe0fc1042c0 2024-08-19T01:44:03.763 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:44:03.762+0000 7fe0f9600640 1 -- 172.21.15.83:0/3085488315 <== mon.2 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v32) ==== 161+0+0 (secure 0 0 0) 0x7fe0e809c490 con 0x7fe0fc1042c0 2024-08-19T01:44:03.764 INFO:teuthology.orchestra.run.smithi083.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-08-19T01:44:03.766 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:44:03.764+0000 7fe102000640 1 -- 172.21.15.83:0/3085488315 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7fe0e00a2b80 msgr2=0x7fe0e00a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:44:03.766 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:44:03.764+0000 7fe102000640 1 --2- 172.21.15.83:0/3085488315 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7fe0e00a2b80 0x7fe0e00a5040 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7fe0ec02a950 tx=0x7fe0ec0056d0 comp rx=0 tx=0).stop 2024-08-19T01:44:03.766 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:44:03.764+0000 7fe102000640 1 -- 172.21.15.83:0/3085488315 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe0fc1042c0 msgr2=0x7fe0fc19a870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:44:03.766 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:44:03.764+0000 7fe102000640 1 --2- 172.21.15.83:0/3085488315 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe0fc1042c0 0x7fe0fc19a870 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7fe0e800e9c0 tx=0x7fe0e800ee80 comp rx=0 tx=0).stop 2024-08-19T01:44:03.766 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:44:03.764+0000 7fe102000640 1 -- 172.21.15.83:0/3085488315 shutdown_connections 2024-08-19T01:44:03.767 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:44:03.764+0000 7fe102000640 1 --2- 172.21.15.83:0/3085488315 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7fe0e00a2b80 0x7fe0e00a5040 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:44:03.767 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:44:03.764+0000 7fe102000640 1 --2- 172.21.15.83:0/3085488315 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe0fc109a30 0x7fe0fc19ec00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:44:03.767 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:44:03.764+0000 7fe102000640 1 --2- 172.21.15.83:0/3085488315 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe0fc1042c0 0x7fe0fc19a870 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:44:03.767 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:44:03.764+0000 7fe102000640 1 --2- 172.21.15.83:0/3085488315 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe0fc103920 0x7fe0fc19a330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:44:03.767 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:44:03.764+0000 7fe102000640 1 -- 172.21.15.83:0/3085488315 >> 172.21.15.83:0/3085488315 conn(0x7fe0fc0ff5e0 msgr2=0x7fe0fc0ffe70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:44:03.767 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:44:03.765+0000 7fe102000640 1 -- 172.21.15.83:0/3085488315 shutdown_connections 2024-08-19T01:44:03.767 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:44:03.765+0000 7fe102000640 1 -- 172.21.15.83:0/3085488315 wait complete. 2024-08-19T01:44:03.952 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:44:03 smithi117 ceph-mon[22016]: pgmap v615: 97 pgs: 97 active+clean; 35 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 767 B/s wr, 6 op/s 2024-08-19T01:44:03.952 INFO:journalctl@ceph.mds.b.smithi117.stdout:Aug 19 01:44:03 smithi117 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-mds-b[51905]: ignoring --setuser ceph since I am not root 2024-08-19T01:44:03.952 INFO:journalctl@ceph.mds.b.smithi117.stdout:Aug 19 01:44:03 smithi117 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-mds-b[51905]: ignoring --setgroup ceph since I am not root 2024-08-19T01:44:03.953 INFO:journalctl@ceph.mds.b.smithi117.stdout:Aug 19 01:44:03 smithi117 ceph-mds[51909]: -- 172.21.15.117:0/608966937 <== mon.1 v2:172.21.15.167:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55b8e9c4d860 con 0x55b8e9c2a800 2024-08-19T01:44:03.953 INFO:journalctl@ceph.mds.b.smithi117.stdout:Aug 19 01:44:03 smithi117 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-mds-b[51905]: starting mds.b at 2024-08-19T01:44:03.953 INFO:journalctl@ceph.mds.e.smithi117.stdout:Aug 19 01:44:03 smithi117 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-mds-e[51290]: ignoring --setuser ceph since I am not root 2024-08-19T01:44:03.953 INFO:journalctl@ceph.mds.e.smithi117.stdout:Aug 19 01:44:03 smithi117 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-mds-e[51290]: ignoring --setgroup ceph since I am not root 2024-08-19T01:44:03.953 INFO:journalctl@ceph.mds.e.smithi117.stdout:Aug 19 01:44:03 smithi117 ceph-mds[51294]: -- 172.21.15.117:0/825714711 <== mon.2 v2:172.21.15.117:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x557a6e2e5860 con 0x557a6e2c2800 2024-08-19T01:44:03.953 INFO:journalctl@ceph.mds.e.smithi117.stdout:Aug 19 01:44:03 smithi117 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-mds-e[51290]: starting mds.e at 2024-08-19T01:44:03.953 INFO:journalctl@ceph.mds.h.smithi117.stdout:Aug 19 01:44:03 smithi117 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-mds-h[51591]: ignoring --setuser ceph since I am not root 2024-08-19T01:44:03.953 INFO:journalctl@ceph.mds.h.smithi117.stdout:Aug 19 01:44:03 smithi117 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-mds-h[51591]: ignoring --setgroup ceph since I am not root 2024-08-19T01:44:03.954 INFO:journalctl@ceph.mds.h.smithi117.stdout:Aug 19 01:44:03 smithi117 ceph-mds[51595]: -- 172.21.15.117:0/464856469 <== mon.1 v2:172.21.15.167:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5565b5f69860 con 0x5565b5f46800 2024-08-19T01:44:03.954 INFO:journalctl@ceph.mds.h.smithi117.stdout:Aug 19 01:44:03 smithi117 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-mds-h[51591]: starting mds.h at 2024-08-19T01:44:03.966 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:44:03 smithi167 ceph-mon[21875]: pgmap v615: 97 pgs: 97 active+clean; 35 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 767 B/s wr, 6 op/s 2024-08-19T01:44:03.966 INFO:journalctl@ceph.mds.i.smithi167.stdout:Aug 19 01:44:03 smithi167 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-mds-i[52219]: ignoring --setuser ceph since I am not root 2024-08-19T01:44:03.966 INFO:journalctl@ceph.mds.i.smithi167.stdout:Aug 19 01:44:03 smithi167 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-mds-i[52219]: ignoring --setgroup ceph since I am not root 2024-08-19T01:44:03.966 INFO:journalctl@ceph.mds.i.smithi167.stdout:Aug 19 01:44:03 smithi167 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-mds-i[52219]: starting mds.i at 2024-08-19T01:44:03.966 INFO:journalctl@ceph.mds.c.smithi167.stdout:Aug 19 01:44:03 smithi167 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-mds-c[51904]: ignoring --setuser ceph since I am not root 2024-08-19T01:44:03.966 INFO:journalctl@ceph.mds.c.smithi167.stdout:Aug 19 01:44:03 smithi167 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-mds-c[51904]: ignoring --setgroup ceph since I am not root 2024-08-19T01:44:03.967 INFO:journalctl@ceph.mds.c.smithi167.stdout:Aug 19 01:44:03 smithi167 ceph-mds[51908]: -- 172.21.15.167:0/3461595174 <== mon.1 v2:172.21.15.167:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x556bfa163860 con 0x556bfa141400 2024-08-19T01:44:03.967 INFO:journalctl@ceph.mds.c.smithi167.stdout:Aug 19 01:44:03 smithi167 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-mds-c[51904]: starting mds.c at 2024-08-19T01:44:04.140 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:44:03 smithi083 ceph-mon[17120]: pgmap v615: 97 pgs: 97 active+clean; 35 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 767 B/s wr, 6 op/s 2024-08-19T01:44:04.140 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:44:03 smithi083 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-mon-a[17116]: 2024-08-19T01:44:03.754+0000 7f95f4400640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-08-19T01:44:04.140 INFO:journalctl@ceph.mds.g.smithi083.stdout:Aug 19 01:44:03 smithi083 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-mds-g[57656]: ignoring --setuser ceph since I am not root 2024-08-19T01:44:04.140 INFO:journalctl@ceph.mds.g.smithi083.stdout:Aug 19 01:44:03 smithi083 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-mds-g[57656]: ignoring --setgroup ceph since I am not root 2024-08-19T01:44:04.141 INFO:journalctl@ceph.mds.g.smithi083.stdout:Aug 19 01:44:03 smithi083 ceph-mds[57660]: -- 172.21.15.83:0/3322191701 <== mon.1 v2:172.21.15.167:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5583f06e9860 con 0x5583f06c6800 2024-08-19T01:44:04.141 INFO:journalctl@ceph.mds.g.smithi083.stdout:Aug 19 01:44:03 smithi083 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-mds-g[57656]: starting mds.g at 2024-08-19T01:44:04.442 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833 shell --fsid bad8780e-5dc9-11ef-bccf-c7b262605968 -- ceph fs rm cephfs --yes-i-really-mean-it 2024-08-19T01:44:04.698 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:44:04 smithi083 ceph-mon[17120]: from='client.? 172.21.15.83:0/3085488315' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-08-19T01:44:04.698 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:44:04 smithi083 ceph-mon[17120]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-08-19T01:44:04.698 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:44:04 smithi083 ceph-mon[17120]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-08-19T01:44:04.698 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:44:04 smithi083 ceph-mon[17120]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-08-19T01:44:04.698 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:44:04 smithi083 ceph-mon[17120]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-08-19T01:44:04.699 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:44:04 smithi083 ceph-mon[17120]: osdmap e87: 12 total, 12 up, 12 in 2024-08-19T01:44:04.699 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:44:04 smithi083 ceph-mon[17120]: fsmap cephfs:0/3 6 up:standby, 3 failed 2024-08-19T01:44:04.822 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/bad8780e-5dc9-11ef-bccf-c7b262605968/mon.a/config 2024-08-19T01:44:04.952 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:44:04 smithi117 ceph-mon[22016]: from='client.? 172.21.15.83:0/3085488315' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-08-19T01:44:04.952 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:44:04 smithi117 ceph-mon[22016]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-08-19T01:44:04.952 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:44:04 smithi117 ceph-mon[22016]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-08-19T01:44:04.953 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:44:04 smithi117 ceph-mon[22016]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-08-19T01:44:04.953 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:44:04 smithi117 ceph-mon[22016]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-08-19T01:44:04.953 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:44:04 smithi117 ceph-mon[22016]: osdmap e87: 12 total, 12 up, 12 in 2024-08-19T01:44:04.953 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:44:04 smithi117 ceph-mon[22016]: fsmap cephfs:0/3 6 up:standby, 3 failed 2024-08-19T01:44:04.966 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:44:04 smithi167 ceph-mon[21875]: from='client.? 172.21.15.83:0/3085488315' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-08-19T01:44:04.966 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:44:04 smithi167 ceph-mon[21875]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-08-19T01:44:04.966 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:44:04 smithi167 ceph-mon[21875]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-08-19T01:44:04.966 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:44:04 smithi167 ceph-mon[21875]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-08-19T01:44:04.966 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:44:04 smithi167 ceph-mon[21875]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-08-19T01:44:04.966 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:44:04 smithi167 ceph-mon[21875]: osdmap e87: 12 total, 12 up, 12 in 2024-08-19T01:44:04.966 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:44:04 smithi167 ceph-mon[21875]: fsmap cephfs:0/3 6 up:standby, 3 failed 2024-08-19T01:44:05.459 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:44:05.326+0000 7fd684800640 1 -- 172.21.15.83:0/3791587663 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd67c1018b0 msgr2=0x7fd67c101d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:44:05.459 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:44:05.326+0000 7fd684800640 1 --2- 172.21.15.83:0/3791587663 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd67c1018b0 0x7fd67c101d10 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7fd66c009930 tx=0x7fd66c02a420 comp rx=0 tx=0).stop 2024-08-19T01:44:05.459 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:44:05.326+0000 7fd684800640 1 -- 172.21.15.83:0/3791587663 shutdown_connections 2024-08-19T01:44:05.460 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:44:05.326+0000 7fd684800640 1 --2- 172.21.15.83:0/3791587663 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd67c102250 0x7fd67c10e780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:44:05.460 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:44:05.327+0000 7fd684800640 1 --2- 172.21.15.83:0/3791587663 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd67c1018b0 0x7fd67c101d10 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:44:05.460 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:44:05.327+0000 7fd684800640 1 --2- 172.21.15.83:0/3791587663 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd67c1078b0 0x7fd67c107c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:44:05.460 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:44:05.327+0000 7fd684800640 1 -- 172.21.15.83:0/3791587663 >> 172.21.15.83:0/3791587663 conn(0x7fd67c0fd5f0 msgr2=0x7fd67c0ffa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:44:05.460 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:44:05.327+0000 7fd684800640 1 -- 172.21.15.83:0/3791587663 shutdown_connections 2024-08-19T01:44:05.460 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:44:05.327+0000 7fd684800640 1 -- 172.21.15.83:0/3791587663 wait complete. 2024-08-19T01:44:05.460 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:44:05.328+0000 7fd684800640 1 Processor -- start 2024-08-19T01:44:05.460 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:44:05.328+0000 7fd684800640 1 -- start start 2024-08-19T01:44:05.460 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:44:05.328+0000 7fd684800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd67c1018b0 0x7fd67c074220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:44:05.460 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:44:05.328+0000 7fd684800640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd67c102250 0x7fd67c074760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:44:05.460 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:44:05.328+0000 7fd684800640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd67c1078b0 0x7fd67c074ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:44:05.460 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:44:05.328+0000 7fd684800640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fd67c111d00 con 0x7fd67c1018b0 2024-08-19T01:44:05.460 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:44:05.328+0000 7fd684800640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fd67c111b80 con 0x7fd67c102250 2024-08-19T01:44:05.461 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:44:05.328+0000 7fd684800640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fd67c111e80 con 0x7fd67c1078b0 2024-08-19T01:44:05.461 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:44:05.329+0000 7fd681e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd67c1018b0 0x7fd67c074220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:44:05.461 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:44:05.329+0000 7fd681e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd67c1018b0 0x7fd67c074220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:59526/0 (socket says 172.21.15.83:59526) 2024-08-19T01:44:05.461 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:44:05.329+0000 7fd681e00640 1 -- 172.21.15.83:0/950456530 learned_addr learned my addr 172.21.15.83:0/950456530 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-08-19T01:44:05.461 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:44:05.329+0000 7fd681400640 1 --2- 172.21.15.83:0/950456530 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd67c102250 0x7fd67c074760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:44:05.461 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:44:05.329+0000 7fd682800640 1 --2- 172.21.15.83:0/950456530 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd67c1078b0 0x7fd67c074ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:44:05.461 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:44:05.329+0000 7fd681e00640 1 -- 172.21.15.83:0/950456530 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd67c1078b0 msgr2=0x7fd67c074ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:44:05.461 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:44:05.329+0000 7fd681e00640 1 --2- 172.21.15.83:0/950456530 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd67c1078b0 0x7fd67c074ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:44:05.461 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:44:05.329+0000 7fd681e00640 1 -- 172.21.15.83:0/950456530 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd67c102250 msgr2=0x7fd67c074760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:44:05.461 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:44:05.329+0000 7fd681e00640 1 --2- 172.21.15.83:0/950456530 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd67c102250 0x7fd67c074760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:44:05.461 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:44:05.329+0000 7fd681e00640 1 -- 172.21.15.83:0/950456530 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd67c1aab70 con 0x7fd67c1018b0 2024-08-19T01:44:05.461 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:44:05.329+0000 7fd681400640 1 --2- 172.21.15.83:0/950456530 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd67c102250 0x7fd67c074760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-19T01:44:05.461 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:44:05.329+0000 7fd681e00640 1 --2- 172.21.15.83:0/950456530 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd67c1018b0 0x7fd67c074220 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7fd67000e9f0 tx=0x7fd67000edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:44:05.462 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:44:05.330+0000 7fd676a00640 1 -- 172.21.15.83:0/950456530 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd67000cd80 con 0x7fd67c1018b0 2024-08-19T01:44:05.462 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:44:05.330+0000 7fd676a00640 1 -- 172.21.15.83:0/950456530 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd670005500 con 0x7fd67c1018b0 2024-08-19T01:44:05.462 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:44:05.330+0000 7fd684800640 1 -- 172.21.15.83:0/950456530 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd67c1aada0 con 0x7fd67c1018b0 2024-08-19T01:44:05.462 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:44:05.330+0000 7fd676a00640 1 -- 172.21.15.83:0/950456530 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd670010690 con 0x7fd67c1018b0 2024-08-19T01:44:05.462 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:44:05.330+0000 7fd684800640 1 -- 172.21.15.83:0/950456530 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd67c1ab1a0 con 0x7fd67c1018b0 2024-08-19T01:44:05.462 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:44:05.331+0000 7fd684800640 1 -- 172.21.15.83:0/950456530 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd67c102ff0 con 0x7fd67c1018b0 2024-08-19T01:44:05.462 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:44:05.333+0000 7fd676a00640 1 -- 172.21.15.83:0/950456530 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7fd6700046c0 con 0x7fd67c1018b0 2024-08-19T01:44:05.462 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:44:05.334+0000 7fd676a00640 1 --2- 172.21.15.83:0/950456530 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7fd6640a2b80 0x7fd6640a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:44:05.462 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:44:05.334+0000 7fd676a00640 1 -- 172.21.15.83:0/950456530 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8527+0+0 (secure 0 0 0) 0x7fd6700cf590 con 0x7fd67c1018b0 2024-08-19T01:44:05.462 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:44:05.334+0000 7fd681400640 1 --2- 172.21.15.83:0/950456530 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7fd6640a2b80 0x7fd6640a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:44:05.462 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:44:05.335+0000 7fd681400640 1 --2- 172.21.15.83:0/950456530 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7fd6640a2b80 0x7fd6640a5040 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7fd67c19c300 tx=0x7fd66c031510 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:44:05.462 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:44:05.338+0000 7fd676a00640 1 -- 172.21.15.83:0/950456530 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7fd67009c370 con 0x7fd67c1018b0 2024-08-19T01:44:05.551 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:44:05.550+0000 7fd684800640 1 -- 172.21.15.83:0/950456530 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7fd67c109e20 con 0x7fd67c1018b0 2024-08-19T01:44:05.767 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:44:05.762+0000 7fd676a00640 1 -- 172.21.15.83:0/950456530 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v34) ==== 104+0+0 (secure 0 0 0) 0x7fd67009bc70 con 0x7fd67c1018b0 2024-08-19T01:44:05.767 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:44:05.765+0000 7fd684800640 1 -- 172.21.15.83:0/950456530 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7fd6640a2b80 msgr2=0x7fd6640a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:44:05.767 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:44:05.765+0000 7fd684800640 1 --2- 172.21.15.83:0/950456530 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7fd6640a2b80 0x7fd6640a5040 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7fd67c19c300 tx=0x7fd66c031510 comp rx=0 tx=0).stop 2024-08-19T01:44:05.767 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:44:05.765+0000 7fd684800640 1 -- 172.21.15.83:0/950456530 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd67c1018b0 msgr2=0x7fd67c074220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:44:05.767 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:44:05.765+0000 7fd684800640 1 --2- 172.21.15.83:0/950456530 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd67c1018b0 0x7fd67c074220 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7fd67000e9f0 tx=0x7fd67000edf0 comp rx=0 tx=0).stop 2024-08-19T01:44:05.767 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:44:05.765+0000 7fd684800640 1 -- 172.21.15.83:0/950456530 shutdown_connections 2024-08-19T01:44:05.767 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:44:05.765+0000 7fd684800640 1 --2- 172.21.15.83:0/950456530 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7fd6640a2b80 0x7fd6640a5040 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:44:05.767 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:44:05.765+0000 7fd684800640 1 --2- 172.21.15.83:0/950456530 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd67c1078b0 0x7fd67c074ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:44:05.767 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:44:05.765+0000 7fd684800640 1 --2- 172.21.15.83:0/950456530 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd67c102250 0x7fd67c074760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:44:05.768 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:44:05.765+0000 7fd684800640 1 --2- 172.21.15.83:0/950456530 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd67c1018b0 0x7fd67c074220 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:44:05.768 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:44:05.765+0000 7fd684800640 1 -- 172.21.15.83:0/950456530 >> 172.21.15.83:0/950456530 conn(0x7fd67c0fd5f0 msgr2=0x7fd67c0fde00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:44:05.768 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:44:05.765+0000 7fd684800640 1 -- 172.21.15.83:0/950456530 shutdown_connections 2024-08-19T01:44:05.768 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:44:05.765+0000 7fd684800640 1 -- 172.21.15.83:0/950456530 wait complete. 2024-08-19T01:44:05.952 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:44:05 smithi117 ceph-mon[22016]: pgmap v617: 97 pgs: 97 active+clean; 35 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 921 B/s wr, 7 op/s 2024-08-19T01:44:05.953 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:44:05 smithi117 ceph-mon[22016]: mds.? [v2:172.21.15.167:6836/3204531242,v1:172.21.15.167:6837/3204531242] up:boot 2024-08-19T01:44:05.953 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:44:05 smithi117 ceph-mon[22016]: mds.? [v2:172.21.15.167:6838/1181098572,v1:172.21.15.167:6839/1181098572] up:boot 2024-08-19T01:44:05.953 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:44:05 smithi117 ceph-mon[22016]: mds.? [v2:172.21.15.117:6834/3677999144,v1:172.21.15.117:6835/3677999144] up:boot 2024-08-19T01:44:05.953 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:44:05 smithi117 ceph-mon[22016]: mds.? [v2:172.21.15.117:6836/175995488,v1:172.21.15.117:6837/175995488] up:boot 2024-08-19T01:44:05.953 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:44:05 smithi117 ceph-mon[22016]: mds.? [v2:172.21.15.117:6838/2743593230,v1:172.21.15.117:6839/2743593230] up:boot 2024-08-19T01:44:05.953 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:44:05 smithi117 ceph-mon[22016]: mds.? [v2:172.21.15.83:6838/1071168887,v1:172.21.15.83:6839/1071168887] up:boot 2024-08-19T01:44:05.953 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:44:05 smithi117 ceph-mon[22016]: fsmap cephfs:0/3 12 up:standby, 3 failed 2024-08-19T01:44:05.953 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:44:05 smithi117 ceph-mon[22016]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-08-19T01:44:05.953 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:44:05 smithi117 ceph-mon[22016]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-08-19T01:44:05.953 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:44:05 smithi117 ceph-mon[22016]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-08-19T01:44:05.953 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:44:05 smithi117 ceph-mon[22016]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-08-19T01:44:05.953 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:44:05 smithi117 ceph-mon[22016]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-08-19T01:44:05.953 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:44:05 smithi117 ceph-mon[22016]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-08-19T01:44:05.953 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:44:05 smithi117 ceph-mon[22016]: from='client.? 172.21.15.83:0/950456530' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-08-19T01:44:05.966 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:44:05 smithi167 ceph-mon[21875]: pgmap v617: 97 pgs: 97 active+clean; 35 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 921 B/s wr, 7 op/s 2024-08-19T01:44:05.966 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:44:05 smithi167 ceph-mon[21875]: mds.? [v2:172.21.15.167:6836/3204531242,v1:172.21.15.167:6837/3204531242] up:boot 2024-08-19T01:44:05.966 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:44:05 smithi167 ceph-mon[21875]: mds.? [v2:172.21.15.167:6838/1181098572,v1:172.21.15.167:6839/1181098572] up:boot 2024-08-19T01:44:05.966 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:44:05 smithi167 ceph-mon[21875]: mds.? [v2:172.21.15.117:6834/3677999144,v1:172.21.15.117:6835/3677999144] up:boot 2024-08-19T01:44:05.966 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:44:05 smithi167 ceph-mon[21875]: mds.? [v2:172.21.15.117:6836/175995488,v1:172.21.15.117:6837/175995488] up:boot 2024-08-19T01:44:05.966 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:44:05 smithi167 ceph-mon[21875]: mds.? [v2:172.21.15.117:6838/2743593230,v1:172.21.15.117:6839/2743593230] up:boot 2024-08-19T01:44:05.966 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:44:05 smithi167 ceph-mon[21875]: mds.? [v2:172.21.15.83:6838/1071168887,v1:172.21.15.83:6839/1071168887] up:boot 2024-08-19T01:44:05.966 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:44:05 smithi167 ceph-mon[21875]: fsmap cephfs:0/3 12 up:standby, 3 failed 2024-08-19T01:44:05.967 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:44:05 smithi167 ceph-mon[21875]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-08-19T01:44:05.967 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:44:05 smithi167 ceph-mon[21875]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-08-19T01:44:05.967 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:44:05 smithi167 ceph-mon[21875]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-08-19T01:44:05.967 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:44:05 smithi167 ceph-mon[21875]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-08-19T01:44:05.967 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:44:05 smithi167 ceph-mon[21875]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-08-19T01:44:05.967 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:44:05 smithi167 ceph-mon[21875]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-08-19T01:44:05.967 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:44:05 smithi167 ceph-mon[21875]: from='client.? 172.21.15.83:0/950456530' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-08-19T01:44:06.038 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:44:05 smithi083 ceph-mon[17120]: pgmap v617: 97 pgs: 97 active+clean; 35 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 921 B/s wr, 7 op/s 2024-08-19T01:44:06.038 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:44:05 smithi083 ceph-mon[17120]: mds.? [v2:172.21.15.167:6836/3204531242,v1:172.21.15.167:6837/3204531242] up:boot 2024-08-19T01:44:06.038 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:44:05 smithi083 ceph-mon[17120]: mds.? [v2:172.21.15.167:6838/1181098572,v1:172.21.15.167:6839/1181098572] up:boot 2024-08-19T01:44:06.038 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:44:05 smithi083 ceph-mon[17120]: mds.? [v2:172.21.15.117:6834/3677999144,v1:172.21.15.117:6835/3677999144] up:boot 2024-08-19T01:44:06.039 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:44:05 smithi083 ceph-mon[17120]: mds.? [v2:172.21.15.117:6836/175995488,v1:172.21.15.117:6837/175995488] up:boot 2024-08-19T01:44:06.039 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:44:05 smithi083 ceph-mon[17120]: mds.? [v2:172.21.15.117:6838/2743593230,v1:172.21.15.117:6839/2743593230] up:boot 2024-08-19T01:44:06.039 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:44:05 smithi083 ceph-mon[17120]: mds.? [v2:172.21.15.83:6838/1071168887,v1:172.21.15.83:6839/1071168887] up:boot 2024-08-19T01:44:06.039 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:44:05 smithi083 ceph-mon[17120]: fsmap cephfs:0/3 12 up:standby, 3 failed 2024-08-19T01:44:06.039 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:44:05 smithi083 ceph-mon[17120]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-08-19T01:44:06.039 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:44:05 smithi083 ceph-mon[17120]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-08-19T01:44:06.039 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:44:05 smithi083 ceph-mon[17120]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-08-19T01:44:06.039 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:44:05 smithi083 ceph-mon[17120]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-08-19T01:44:06.039 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:44:05 smithi083 ceph-mon[17120]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-08-19T01:44:06.039 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:44:05 smithi083 ceph-mon[17120]: from='mgr.14150 172.21.15.83:0/686875149' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-08-19T01:44:06.039 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:44:05 smithi083 ceph-mon[17120]: from='client.? 172.21.15.83:0/950456530' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-08-19T01:44:06.395 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833 shell --fsid bad8780e-5dc9-11ef-bccf-c7b262605968 -- ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-08-19T01:44:06.769 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/bad8780e-5dc9-11ef-bccf-c7b262605968/mon.a/config 2024-08-19T01:44:06.952 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:44:06 smithi117 ceph-mon[22016]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-08-19T01:44:06.952 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:44:06 smithi117 ceph-mon[22016]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-08-19T01:44:06.952 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:44:06 smithi117 ceph-mon[22016]: Cluster is now healthy 2024-08-19T01:44:06.952 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:44:06 smithi117 ceph-mon[22016]: from='client.? 172.21.15.83:0/950456530' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-08-19T01:44:06.952 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:44:06 smithi117 ceph-mon[22016]: fsmap 12 up:standby 2024-08-19T01:44:06.956 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:44:06 smithi083 ceph-mon[17120]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-08-19T01:44:06.956 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:44:06 smithi083 ceph-mon[17120]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-08-19T01:44:06.956 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:44:06 smithi083 ceph-mon[17120]: Cluster is now healthy 2024-08-19T01:44:06.956 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:44:06 smithi083 ceph-mon[17120]: from='client.? 172.21.15.83:0/950456530' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-08-19T01:44:06.956 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:44:06 smithi083 ceph-mon[17120]: fsmap 12 up:standby 2024-08-19T01:44:06.965 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:44:06 smithi167 ceph-mon[21875]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-08-19T01:44:06.965 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:44:06 smithi167 ceph-mon[21875]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-08-19T01:44:06.966 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:44:06 smithi167 ceph-mon[21875]: Cluster is now healthy 2024-08-19T01:44:06.966 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:44:06 smithi167 ceph-mon[21875]: from='client.? 172.21.15.83:0/950456530' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-08-19T01:44:06.966 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:44:06 smithi167 ceph-mon[21875]: fsmap 12 up:standby 2024-08-19T01:44:07.343 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:44:07.338+0000 7f5605600640 1 -- 172.21.15.83:0/34393841 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f56001125a0 msgr2=0x7f5600114990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:44:07.343 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:44:07.338+0000 7f5605600640 1 --2- 172.21.15.83:0/34393841 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f56001125a0 0x7f5600114990 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f55f8009ca0 tx=0x7f55f802a8c0 comp rx=0 tx=0).stop 2024-08-19T01:44:07.343 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:44:07.338+0000 7f5605600640 1 -- 172.21.15.83:0/34393841 shutdown_connections 2024-08-19T01:44:07.343 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:44:07.338+0000 7f5605600640 1 --2- 172.21.15.83:0/34393841 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f56001125a0 0x7f5600114990 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:44:07.344 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:44:07.338+0000 7f5605600640 1 --2- 172.21.15.83:0/34393841 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5600076d70 0x7f5600112060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:44:07.344 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:44:07.338+0000 7f5605600640 1 --2- 172.21.15.83:0/34393841 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5600076450 0x7f5600076830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:44:07.344 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:44:07.338+0000 7f5605600640 1 -- 172.21.15.83:0/34393841 >> 172.21.15.83:0/34393841 conn(0x7f56000ff600 msgr2=0x7f5600101a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:44:07.344 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:44:07.338+0000 7f5605600640 1 -- 172.21.15.83:0/34393841 shutdown_connections 2024-08-19T01:44:07.344 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:44:07.338+0000 7f5605600640 1 -- 172.21.15.83:0/34393841 wait complete. 2024-08-19T01:44:07.344 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:44:07.339+0000 7f5605600640 1 Processor -- start 2024-08-19T01:44:07.344 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:44:07.340+0000 7f5605600640 1 -- start start 2024-08-19T01:44:07.344 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:44:07.340+0000 7f5605600640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5600076450 0x7f560019e560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:44:07.344 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:44:07.340+0000 7f5605600640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5600076d70 0x7f560019eaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:44:07.344 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:44:07.340+0000 7f5605600640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f56001125a0 0x7f56001a2e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:44:07.344 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:44:07.340+0000 7f5605600640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f56001180c0 con 0x7f56001125a0 2024-08-19T01:44:07.344 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:44:07.340+0000 7f5605600640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f5600117f40 con 0x7f5600076d70 2024-08-19T01:44:07.344 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:44:07.340+0000 7f5605600640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f5600118240 con 0x7f5600076450 2024-08-19T01:44:07.345 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:44:07.340+0000 7f5604c00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f56001125a0 0x7f56001a2e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:44:07.345 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:44:07.340+0000 7f55ffe00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5600076450 0x7f560019e560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:44:07.345 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:44:07.340+0000 7f55ffe00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5600076450 0x7f560019e560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.83:47108/0 (socket says 172.21.15.83:47108) 2024-08-19T01:44:07.345 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:44:07.340+0000 7f55ffe00640 1 -- 172.21.15.83:0/3719130795 learned_addr learned my addr 172.21.15.83:0/3719130795 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-08-19T01:44:07.345 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:44:07.340+0000 7f55f7400640 1 --2- 172.21.15.83:0/3719130795 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5600076d70 0x7f560019eaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:44:07.345 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:44:07.341+0000 7f55ffe00640 1 -- 172.21.15.83:0/3719130795 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5600076d70 msgr2=0x7f560019eaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:44:07.345 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:44:07.341+0000 7f55ffe00640 1 --2- 172.21.15.83:0/3719130795 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5600076d70 0x7f560019eaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:44:07.345 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:44:07.341+0000 7f55ffe00640 1 -- 172.21.15.83:0/3719130795 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f56001125a0 msgr2=0x7f56001a2e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:44:07.345 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:44:07.341+0000 7f55ffe00640 1 --2- 172.21.15.83:0/3719130795 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f56001125a0 0x7f56001a2e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:44:07.345 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:44:07.341+0000 7f55ffe00640 1 -- 172.21.15.83:0/3719130795 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f56001a3510 con 0x7f5600076450 2024-08-19T01:44:07.345 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:44:07.341+0000 7f5604c00640 1 --2- 172.21.15.83:0/3719130795 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f56001125a0 0x7f56001a2e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-19T01:44:07.345 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:44:07.341+0000 7f55ffe00640 1 --2- 172.21.15.83:0/3719130795 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5600076450 0x7f560019e560 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f55f000d950 tx=0x7f55f000de10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:44:07.345 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:44:07.341+0000 7f55fd600640 1 -- 172.21.15.83:0/3719130795 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55f0017030 con 0x7f5600076450 2024-08-19T01:44:07.345 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:44:07.341+0000 7f55fd600640 1 -- 172.21.15.83:0/3719130795 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f55f0004020 con 0x7f5600076450 2024-08-19T01:44:07.345 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:44:07.341+0000 7f55fd600640 1 -- 172.21.15.83:0/3719130795 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55f00047a0 con 0x7f5600076450 2024-08-19T01:44:07.346 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:44:07.342+0000 7f5605600640 1 -- 172.21.15.83:0/3719130795 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f56001a3800 con 0x7f5600076450 2024-08-19T01:44:07.346 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:44:07.342+0000 7f5605600640 1 -- 172.21.15.83:0/3719130795 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f56001ab0e0 con 0x7f5600076450 2024-08-19T01:44:07.346 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:44:07.343+0000 7f5605600640 1 -- 172.21.15.83:0/3719130795 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5600076830 con 0x7f5600076450 2024-08-19T01:44:07.346 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:44:07.345+0000 7f55fd600640 1 -- 172.21.15.83:0/3719130795 <== mon.1 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7f55f00041d0 con 0x7f5600076450 2024-08-19T01:44:07.347 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:44:07.346+0000 7f55fd600640 1 --2- 172.21.15.83:0/3719130795 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f55e40a2b80 0x7f55e40a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:44:07.348 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:44:07.346+0000 7f55fd600640 1 -- 172.21.15.83:0/3719130795 <== mon.1 v2:172.21.15.167:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8527+0+0 (secure 0 0 0) 0x7f55f00cf160 con 0x7f5600076450 2024-08-19T01:44:07.348 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:44:07.346+0000 7f55f7400640 1 --2- 172.21.15.83:0/3719130795 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f55e40a2b80 0x7f55e40a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:44:07.348 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:44:07.347+0000 7f55f7400640 1 --2- 172.21.15.83:0/3719130795 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f55e40a2b80 0x7f55e40a5040 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f5600079030 tx=0x7f55e800b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:44:07.350 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:44:07.348+0000 7f55fd600640 1 -- 172.21.15.83:0/3719130795 <== mon.1 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7f55f009bf40 con 0x7f5600076450 2024-08-19T01:44:07.532 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:44:07.531+0000 7f5605600640 1 -- 172.21.15.83:0/3719130795 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7f560019f8c0 con 0x7f5600076450 2024-08-19T01:44:07.771 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:44:07.769+0000 7f55fd600640 1 -- 172.21.15.83:0/3719130795 <== mon.1 v2:172.21.15.167:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v88) ==== 181+0+0 (secure 0 0 0) 0x7f55f009b840 con 0x7f5600076450 2024-08-19T01:44:07.771 INFO:teuthology.orchestra.run.smithi083.stderr:pool 'cephfs_metadata' removed 2024-08-19T01:44:07.776 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:44:07.774+0000 7f5605600640 1 -- 172.21.15.83:0/3719130795 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f55e40a2b80 msgr2=0x7f55e40a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:44:07.776 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:44:07.774+0000 7f5605600640 1 --2- 172.21.15.83:0/3719130795 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f55e40a2b80 0x7f55e40a5040 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f5600079030 tx=0x7f55e800b410 comp rx=0 tx=0).stop 2024-08-19T01:44:07.776 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:44:07.774+0000 7f5605600640 1 -- 172.21.15.83:0/3719130795 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5600076450 msgr2=0x7f560019e560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:44:07.776 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:44:07.774+0000 7f5605600640 1 --2- 172.21.15.83:0/3719130795 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5600076450 0x7f560019e560 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f55f000d950 tx=0x7f55f000de10 comp rx=0 tx=0).stop 2024-08-19T01:44:07.776 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:44:07.774+0000 7f5605600640 1 -- 172.21.15.83:0/3719130795 shutdown_connections 2024-08-19T01:44:07.776 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:44:07.774+0000 7f5605600640 1 --2- 172.21.15.83:0/3719130795 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f55e40a2b80 0x7f55e40a5040 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:44:07.777 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:44:07.774+0000 7f5605600640 1 --2- 172.21.15.83:0/3719130795 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f56001125a0 0x7f56001a2e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:44:07.777 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:44:07.774+0000 7f5605600640 1 --2- 172.21.15.83:0/3719130795 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5600076d70 0x7f560019eaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:44:07.777 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:44:07.774+0000 7f5605600640 1 --2- 172.21.15.83:0/3719130795 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5600076450 0x7f560019e560 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:44:07.777 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:44:07.774+0000 7f5605600640 1 -- 172.21.15.83:0/3719130795 >> 172.21.15.83:0/3719130795 conn(0x7f56000ff600 msgr2=0x7f56001019f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:44:07.777 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:44:07.774+0000 7f5605600640 1 -- 172.21.15.83:0/3719130795 shutdown_connections 2024-08-19T01:44:07.777 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:44:07.775+0000 7f5605600640 1 -- 172.21.15.83:0/3719130795 wait complete. 2024-08-19T01:44:07.795 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:44:07 smithi083 ceph-mon[17120]: pgmap v618: 97 pgs: 97 active+clean; 35 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 921 B/s wr, 7 op/s 2024-08-19T01:44:07.795 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:44:07 smithi083 ceph-mon[17120]: from='client.? 172.21.15.83:0/3719130795' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-08-19T01:44:07.795 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:44:07 smithi083 ceph-mon[17120]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-08-19T01:44:07.952 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:44:07 smithi117 ceph-mon[22016]: pgmap v618: 97 pgs: 97 active+clean; 35 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 921 B/s wr, 7 op/s 2024-08-19T01:44:07.952 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:44:07 smithi117 ceph-mon[22016]: from='client.? 172.21.15.83:0/3719130795' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-08-19T01:44:07.952 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:44:07 smithi117 ceph-mon[22016]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-08-19T01:44:07.966 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:44:07 smithi167 ceph-mon[21875]: pgmap v618: 97 pgs: 97 active+clean; 35 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 921 B/s wr, 7 op/s 2024-08-19T01:44:07.966 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:44:07 smithi167 ceph-mon[21875]: from='client.? 172.21.15.83:0/3719130795' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-08-19T01:44:07.966 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:44:07 smithi167 ceph-mon[21875]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-08-19T01:44:08.511 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833 shell --fsid bad8780e-5dc9-11ef-bccf-c7b262605968 -- ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-08-19T01:44:08.894 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/bad8780e-5dc9-11ef-bccf-c7b262605968/mon.a/config 2024-08-19T01:44:09.036 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:44:08 smithi083 ceph-mon[17120]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-08-19T01:44:09.036 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:44:08 smithi083 ceph-mon[17120]: osdmap e88: 12 total, 12 up, 12 in 2024-08-19T01:44:09.036 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:44:08 smithi083 ceph-mon[17120]: pgmap v620: 65 pgs: 65 active+clean; 577 KiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail 2024-08-19T01:44:09.202 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:44:08 smithi117 ceph-mon[22016]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-08-19T01:44:09.202 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:44:08 smithi117 ceph-mon[22016]: osdmap e88: 12 total, 12 up, 12 in 2024-08-19T01:44:09.202 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:44:08 smithi117 ceph-mon[22016]: pgmap v620: 65 pgs: 65 active+clean; 577 KiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail 2024-08-19T01:44:09.215 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:44:08 smithi167 ceph-mon[21875]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-08-19T01:44:09.216 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:44:08 smithi167 ceph-mon[21875]: osdmap e88: 12 total, 12 up, 12 in 2024-08-19T01:44:09.216 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:44:08 smithi167 ceph-mon[21875]: pgmap v620: 65 pgs: 65 active+clean; 577 KiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail 2024-08-19T01:44:09.427 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:44:09.399+0000 7f0a95400640 1 -- 172.21.15.83:0/289139582 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0a90103ae0 msgr2=0x7f0a90103f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:44:09.427 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:44:09.399+0000 7f0a95400640 1 --2- 172.21.15.83:0/289139582 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0a90103ae0 0x7f0a90103f40 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f0a7c009930 tx=0x7f0a7c02a420 comp rx=0 tx=0).stop 2024-08-19T01:44:09.427 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:44:09.399+0000 7f0a95400640 1 -- 172.21.15.83:0/289139582 shutdown_connections 2024-08-19T01:44:09.428 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:44:09.399+0000 7f0a95400640 1 --2- 172.21.15.83:0/289139582 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0a90104480 0x7f0a901109b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:44:09.428 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:44:09.399+0000 7f0a95400640 1 --2- 172.21.15.83:0/289139582 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0a90103ae0 0x7f0a90103f40 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:44:09.428 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:44:09.399+0000 7f0a95400640 1 --2- 172.21.15.83:0/289139582 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0a90109b70 0x7f0a90109f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:44:09.428 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:44:09.399+0000 7f0a95400640 1 -- 172.21.15.83:0/289139582 >> 172.21.15.83:0/289139582 conn(0x7f0a900ff780 msgr2=0x7f0a90101ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:44:09.428 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:44:09.399+0000 7f0a95400640 1 -- 172.21.15.83:0/289139582 shutdown_connections 2024-08-19T01:44:09.428 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:44:09.399+0000 7f0a95400640 1 -- 172.21.15.83:0/289139582 wait complete. 2024-08-19T01:44:09.428 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:44:09.400+0000 7f0a95400640 1 Processor -- start 2024-08-19T01:44:09.428 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:44:09.400+0000 7f0a95400640 1 -- start start 2024-08-19T01:44:09.428 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:44:09.400+0000 7f0a95400640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0a90103ae0 0x7f0a9019a3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:44:09.428 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:44:09.400+0000 7f0a95400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0a90104480 0x7f0a9019a8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:44:09.428 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:44:09.400+0000 7f0a95400640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0a90109b70 0x7f0a9019ec80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:44:09.428 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:44:09.400+0000 7f0a95400640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f0a90113f30 con 0x7f0a90104480 2024-08-19T01:44:09.428 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:44:09.400+0000 7f0a95400640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f0a90113db0 con 0x7f0a90109b70 2024-08-19T01:44:09.428 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:44:09.400+0000 7f0a95400640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f0a901140b0 con 0x7f0a90103ae0 2024-08-19T01:44:09.429 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:44:09.401+0000 7f0a8de00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0a90104480 0x7f0a9019a8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:44:09.429 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:44:09.401+0000 7f0a8f200640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0a90109b70 0x7f0a9019ec80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:44:09.429 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:44:09.401+0000 7f0a8e800640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0a90103ae0 0x7f0a9019a3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:44:09.429 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:44:09.401+0000 7f0a8f200640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0a90109b70 0x7f0a9019ec80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.83:54220/0 (socket says 172.21.15.83:54220) 2024-08-19T01:44:09.429 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:44:09.401+0000 7f0a8f200640 1 -- 172.21.15.83:0/463855040 learned_addr learned my addr 172.21.15.83:0/463855040 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-08-19T01:44:09.429 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:44:09.401+0000 7f0a8f200640 1 -- 172.21.15.83:0/463855040 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0a90103ae0 msgr2=0x7f0a9019a3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:44:09.429 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:44:09.401+0000 7f0a8f200640 1 --2- 172.21.15.83:0/463855040 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0a90103ae0 0x7f0a9019a3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:44:09.429 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:44:09.401+0000 7f0a8f200640 1 -- 172.21.15.83:0/463855040 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0a90104480 msgr2=0x7f0a9019a8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:44:09.429 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:44:09.401+0000 7f0a8f200640 1 --2- 172.21.15.83:0/463855040 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0a90104480 0x7f0a9019a8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:44:09.429 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:44:09.401+0000 7f0a8f200640 1 -- 172.21.15.83:0/463855040 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0a9019f360 con 0x7f0a90109b70 2024-08-19T01:44:09.429 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:44:09.401+0000 7f0a8f200640 1 --2- 172.21.15.83:0/463855040 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0a90109b70 0x7f0a9019ec80 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f0a8800eaa0 tx=0x7f0a8800ef60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:44:09.429 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:44:09.402+0000 7f0a87400640 1 -- 172.21.15.83:0/463855040 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a8800cdb0 con 0x7f0a90109b70 2024-08-19T01:44:09.429 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:44:09.402+0000 7f0a95400640 1 -- 172.21.15.83:0/463855040 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0a9019f650 con 0x7f0a90109b70 2024-08-19T01:44:09.430 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:44:09.402+0000 7f0a87400640 1 -- 172.21.15.83:0/463855040 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0a88005020 con 0x7f0a90109b70 2024-08-19T01:44:09.430 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:44:09.402+0000 7f0a87400640 1 -- 172.21.15.83:0/463855040 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a880116d0 con 0x7f0a90109b70 2024-08-19T01:44:09.430 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:44:09.402+0000 7f0a95400640 1 -- 172.21.15.83:0/463855040 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0a901a6f30 con 0x7f0a90109b70 2024-08-19T01:44:09.430 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:44:09.403+0000 7f0a95400640 1 -- 172.21.15.83:0/463855040 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0a90103f40 con 0x7f0a90109b70 2024-08-19T01:44:09.430 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:44:09.405+0000 7f0a87400640 1 -- 172.21.15.83:0/463855040 <== mon.2 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7f0a880054d0 con 0x7f0a90109b70 2024-08-19T01:44:09.430 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:44:09.406+0000 7f0a87400640 1 --2- 172.21.15.83:0/463855040 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f0a740a2b80 0x7f0a740a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:44:09.430 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:44:09.406+0000 7f0a87400640 1 -- 172.21.15.83:0/463855040 <== mon.2 v2:172.21.15.117:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 7973+0+0 (secure 0 0 0) 0x7f0a880cf4b0 con 0x7f0a90109b70 2024-08-19T01:44:09.430 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:44:09.406+0000 7f0a8e800640 1 --2- 172.21.15.83:0/463855040 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f0a740a2b80 0x7f0a740a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:44:09.430 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:44:09.407+0000 7f0a8e800640 1 --2- 172.21.15.83:0/463855040 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f0a740a2b80 0x7f0a740a5040 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f0a80005d00 tx=0x7f0a8000b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:44:09.430 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:44:09.408+0000 7f0a87400640 1 -- 172.21.15.83:0/463855040 <== mon.2 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7f0a88016030 con 0x7f0a90109b70 2024-08-19T01:44:09.588 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:44:09.586+0000 7f0a95400640 1 -- 172.21.15.83:0/463855040 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7f0a9002d5e0 con 0x7f0a90109b70 2024-08-19T01:44:09.787 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:44:09.785+0000 7f0a87400640 1 -- 172.21.15.83:0/463855040 <== mon.2 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v89) ==== 169+0+0 (secure 0 0 0) 0x7f0a8809be70 con 0x7f0a90109b70 2024-08-19T01:44:09.787 INFO:teuthology.orchestra.run.smithi083.stderr:pool 'cephfs_data' removed 2024-08-19T01:44:09.791 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:44:09.788+0000 7f0a95400640 1 -- 172.21.15.83:0/463855040 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f0a740a2b80 msgr2=0x7f0a740a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:44:09.791 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:44:09.789+0000 7f0a95400640 1 --2- 172.21.15.83:0/463855040 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f0a740a2b80 0x7f0a740a5040 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f0a80005d00 tx=0x7f0a8000b410 comp rx=0 tx=0).stop 2024-08-19T01:44:09.791 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:44:09.789+0000 7f0a95400640 1 -- 172.21.15.83:0/463855040 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0a90109b70 msgr2=0x7f0a9019ec80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:44:09.791 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:44:09.789+0000 7f0a95400640 1 --2- 172.21.15.83:0/463855040 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0a90109b70 0x7f0a9019ec80 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f0a8800eaa0 tx=0x7f0a8800ef60 comp rx=0 tx=0).stop 2024-08-19T01:44:09.791 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:44:09.789+0000 7f0a95400640 1 -- 172.21.15.83:0/463855040 shutdown_connections 2024-08-19T01:44:09.791 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:44:09.789+0000 7f0a95400640 1 --2- 172.21.15.83:0/463855040 >> [v2:172.21.15.83:6800/683740916,v1:172.21.15.83:6801/683740916] conn(0x7f0a740a2b80 0x7f0a740a5040 secure :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f0a80005d00 tx=0x7f0a8000b410 comp rx=0 tx=0).stop 2024-08-19T01:44:09.791 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:44:09.789+0000 7f0a95400640 1 --2- 172.21.15.83:0/463855040 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0a90109b70 0x7f0a9019ec80 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:44:09.792 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:44:09.789+0000 7f0a95400640 1 --2- 172.21.15.83:0/463855040 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0a90104480 0x7f0a9019a8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:44:09.792 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:44:09.789+0000 7f0a95400640 1 --2- 172.21.15.83:0/463855040 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0a90103ae0 0x7f0a9019a3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:44:09.792 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:44:09.789+0000 7f0a95400640 1 -- 172.21.15.83:0/463855040 >> 172.21.15.83:0/463855040 conn(0x7f0a900ff780 msgr2=0x7f0a900fff30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:44:09.793 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:44:09.791+0000 7f0a95400640 1 -- 172.21.15.83:0/463855040 shutdown_connections 2024-08-19T01:44:09.794 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-19T01:44:09.793+0000 7f0a95400640 1 -- 172.21.15.83:0/463855040 wait complete. 2024-08-19T01:44:09.904 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:44:09 smithi083 ceph-mon[17120]: from='client.? 172.21.15.83:0/463855040' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-08-19T01:44:09.904 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:44:09 smithi083 ceph-mon[17120]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-08-19T01:44:10.202 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:44:09 smithi117 ceph-mon[22016]: from='client.? 172.21.15.83:0/463855040' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-08-19T01:44:10.202 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:44:09 smithi117 ceph-mon[22016]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-08-19T01:44:10.215 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:44:09 smithi167 ceph-mon[21875]: from='client.? 172.21.15.83:0/463855040' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-08-19T01:44:10.216 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:44:09 smithi167 ceph-mon[21875]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-08-19T01:44:10.588 DEBUG:teuthology.orchestra.run.smithi083:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-08-19T01:44:10.646 DEBUG:teuthology.orchestra.run.smithi117:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-08-19T01:44:10.688 DEBUG:teuthology.orchestra.run.smithi167:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-08-19T01:44:10.717 INFO:tasks.cephadm:Cleaning up testdir ceph.* files... 2024-08-19T01:44:10.717 DEBUG:teuthology.orchestra.run.smithi083:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-08-19T01:44:10.752 DEBUG:teuthology.orchestra.run.smithi117:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-08-19T01:44:10.768 DEBUG:teuthology.orchestra.run.smithi167:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-08-19T01:44:10.783 INFO:tasks.cephadm:Stopping all daemons... 2024-08-19T01:44:10.784 INFO:tasks.cephadm.mon.a:Stopping mon.a... 2024-08-19T01:44:10.784 DEBUG:teuthology.orchestra.run.smithi083:> sudo systemctl stop ceph-bad8780e-5dc9-11ef-bccf-c7b262605968@mon.a 2024-08-19T01:44:10.840 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:44:10 smithi083 ceph-mon[17120]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]': finished 2024-08-19T01:44:10.840 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:44:10 smithi083 ceph-mon[17120]: osdmap e89: 12 total, 12 up, 12 in 2024-08-19T01:44:10.840 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:44:10 smithi083 ceph-mon[17120]: pgmap v622: 1 pgs: 1 active+clean; 577 KiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail 2024-08-19T01:44:10.952 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:44:10 smithi117 ceph-mon[22016]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]': finished 2024-08-19T01:44:10.952 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:44:10 smithi117 ceph-mon[22016]: osdmap e89: 12 total, 12 up, 12 in 2024-08-19T01:44:10.952 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:44:10 smithi117 ceph-mon[22016]: pgmap v622: 1 pgs: 1 active+clean; 577 KiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail 2024-08-19T01:44:10.965 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:44:10 smithi167 ceph-mon[21875]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]': finished 2024-08-19T01:44:10.966 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:44:10 smithi167 ceph-mon[21875]: osdmap e89: 12 total, 12 up, 12 in 2024-08-19T01:44:10.966 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:44:10 smithi167 ceph-mon[21875]: pgmap v622: 1 pgs: 1 active+clean; 577 KiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail 2024-08-19T01:44:11.154 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:44:10 smithi083 systemd[1]: Stopping Ceph mon.a for bad8780e-5dc9-11ef-bccf-c7b262605968... 2024-08-19T01:44:11.539 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:44:11 smithi083 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-mon-a[17116]: 2024-08-19T01:44:11.213+0000 7f95fb200640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-08-19T01:44:11.539 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:44:11 smithi083 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-mon-a[17116]: 2024-08-19T01:44:11.213+0000 7f95fb200640 -1 mon.a@0(leader) e3 *** Got Signal Terminated *** 2024-08-19T01:44:11.539 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:44:11 smithi083 podman[84108]: 2024-08-19 01:44:11.244797064 +0000 UTC m=+0.161367322 container died dbe1fc960772ed81dc7b03ac39c0b11bfb4e77ec275e14044b305ec1335132dd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833, name=ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-mon-a, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, ceph=True, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, RELEASE=squid-de09dd0, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, io.buildah.version=1.36.0, maintainer=Guillaume Abrioux ) 2024-08-19T01:44:11.904 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:44:11 smithi083 podman[84108]: 2024-08-19 01:44:11.702759711 +0000 UTC m=+0.619329969 container cleanup dbe1fc960772ed81dc7b03ac39c0b11bfb4e77ec275e14044b305ec1335132dd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833, name=ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-mon-a, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, GIT_CLEAN=True, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, RELEASE=squid-de09dd0, io.buildah.version=1.36.0) 2024-08-19T01:44:11.904 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:44:11 smithi083 bash[84108]: ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-mon-a 2024-08-19T01:44:12.235 INFO:journalctl@ceph.mon.a.smithi083.stdout:Aug 19 01:44:11 smithi083 podman[84121]: 2024-08-19 01:44:11.954735789 +0000 UTC m=+0.702789498 container remove dbe1fc960772ed81dc7b03ac39c0b11bfb4e77ec275e14044b305ec1335132dd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833, name=ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-mon-a, GIT_BRANCH=HEAD, RELEASE=squid-de09dd0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_CLEAN=True, io.buildah.version=1.36.0, maintainer=Guillaume Abrioux , ceph=True, CEPH_POINT_RELEASE=) 2024-08-19T01:44:12.261 DEBUG:teuthology.orchestra.run.smithi083:> sudo pkill -f 'journalctl -f -n 0 -u ceph-bad8780e-5dc9-11ef-bccf-c7b262605968@mon.a.service' 2024-08-19T01:44:12.350 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-08-19T01:44:12.350 INFO:tasks.cephadm.mon.a:Stopped mon.a 2024-08-19T01:44:12.350 INFO:tasks.cephadm.mon.c:Stopping mon.b... 2024-08-19T01:44:12.350 DEBUG:teuthology.orchestra.run.smithi117:> sudo systemctl stop ceph-bad8780e-5dc9-11ef-bccf-c7b262605968@mon.b 2024-08-19T01:44:12.702 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:44:12 smithi117 systemd[1]: Stopping Ceph mon.b for bad8780e-5dc9-11ef-bccf-c7b262605968... 2024-08-19T01:44:13.105 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:44:12 smithi117 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-mon-b[22012]: 2024-08-19T01:44:12.720+0000 7f3d1daaa640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-08-19T01:44:13.105 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:44:12 smithi117 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-mon-b[22012]: 2024-08-19T01:44:12.720+0000 7f3d1daaa640 -1 mon.b@2(peon) e3 *** Got Signal Terminated *** 2024-08-19T01:44:13.105 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:44:12 smithi117 podman[55530]: 2024-08-19 01:44:12.83277022 +0000 UTC m=+0.235498562 container died b45592b97ea9960eeaa366eedfbc8ed4b3bfe292a097169552f7f2c0974bfbe6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c9dd28080a631be25c77c36c63b17a3ef340ffb2e21c87d4d7b56b6398436d9, name=ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-mon-b, RELEASE=squid-de09dd0, GIT_CLEAN=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, io.buildah.version=1.36.0, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, ceph=True, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image) 2024-08-19T01:44:13.452 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:44:13 smithi117 podman[55530]: 2024-08-19 01:44:13.20392136 +0000 UTC m=+0.606649700 container cleanup b45592b97ea9960eeaa366eedfbc8ed4b3bfe292a097169552f7f2c0974bfbe6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c9dd28080a631be25c77c36c63b17a3ef340ffb2e21c87d4d7b56b6398436d9, name=ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-mon-b, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.36.0, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, ceph=True, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, RELEASE=squid-de09dd0, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD) 2024-08-19T01:44:13.452 INFO:journalctl@ceph.mon.b.smithi117.stdout:Aug 19 01:44:13 smithi117 bash[55530]: ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-mon-b 2024-08-19T01:44:13.603 DEBUG:teuthology.orchestra.run.smithi117:> sudo pkill -f 'journalctl -f -n 0 -u ceph-bad8780e-5dc9-11ef-bccf-c7b262605968@mon.b.service' 2024-08-19T01:44:13.653 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-08-19T01:44:13.653 INFO:tasks.cephadm.mon.c:Stopped mon.b 2024-08-19T01:44:13.653 INFO:tasks.cephadm.mon.c:Stopping mon.c... 2024-08-19T01:44:13.654 DEBUG:teuthology.orchestra.run.smithi167:> sudo systemctl stop ceph-bad8780e-5dc9-11ef-bccf-c7b262605968@mon.c 2024-08-19T01:44:13.966 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:44:13 smithi167 systemd[1]: Stopping Ceph mon.c for bad8780e-5dc9-11ef-bccf-c7b262605968... 2024-08-19T01:44:14.436 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:44:14 smithi167 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-mon-c[21871]: 2024-08-19T01:44:14.041+0000 7f51bfa16640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-08-19T01:44:14.436 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:44:14 smithi167 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-mon-c[21871]: 2024-08-19T01:44:14.042+0000 7f51bfa16640 -1 mon.c@1(peon) e3 *** Got Signal Terminated *** 2024-08-19T01:44:14.436 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:44:14 smithi167 podman[56307]: 2024-08-19 01:44:14.135216069 +0000 UTC m=+0.268017599 container died 16aa4382925d30ce36a900a6bcb7e804f4f669245485ef4ad47d1184f14dcdf6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c9dd28080a631be25c77c36c63b17a3ef340ffb2e21c87d4d7b56b6398436d9, name=ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-mon-c, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, io.buildah.version=1.36.0, ceph=True, CEPH_POINT_RELEASE=, RELEASE=squid-de09dd0, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-08-19T01:44:14.716 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:44:14 smithi167 podman[56307]: 2024-08-19 01:44:14.543342795 +0000 UTC m=+0.676144330 container cleanup 16aa4382925d30ce36a900a6bcb7e804f4f669245485ef4ad47d1184f14dcdf6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c9dd28080a631be25c77c36c63b17a3ef340ffb2e21c87d4d7b56b6398436d9, name=ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-mon-c, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, RELEASE=squid-de09dd0, io.buildah.version=1.36.0, GIT_CLEAN=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, ceph=True, maintainer=Guillaume Abrioux ) 2024-08-19T01:44:14.716 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 19 01:44:14 smithi167 bash[56307]: ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-mon-c 2024-08-19T01:44:14.895 DEBUG:teuthology.orchestra.run.smithi167:> sudo pkill -f 'journalctl -f -n 0 -u ceph-bad8780e-5dc9-11ef-bccf-c7b262605968@mon.c.service' 2024-08-19T01:44:14.944 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-08-19T01:44:14.944 INFO:tasks.cephadm.mon.c:Stopped mon.c 2024-08-19T01:44:14.944 INFO:tasks.cephadm.mgr.x:Stopping mgr.x... 2024-08-19T01:44:14.944 DEBUG:teuthology.orchestra.run.smithi083:> sudo systemctl stop ceph-bad8780e-5dc9-11ef-bccf-c7b262605968@mgr.x 2024-08-19T01:44:15.404 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Aug 19 01:44:15 smithi083 systemd[1]: Stopping Ceph mgr.x for bad8780e-5dc9-11ef-bccf-c7b262605968... 2024-08-19T01:44:15.654 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Aug 19 01:44:15 smithi083 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-mgr-x[17344]: 2024-08-19T01:44:15.401+0000 7f030ea00640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mgr -n mgr.x -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-08-19T01:44:15.654 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Aug 19 01:44:15 smithi083 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-mgr-x[17344]: 2024-08-19T01:44:15.401+0000 7f030ea00640 -1 mgr handle_mgr_signal *** Got signal Terminated *** 2024-08-19T01:44:15.655 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Aug 19 01:44:15 smithi083 podman[84228]: 2024-08-19 01:44:15.485201492 +0000 UTC m=+0.246229431 container died c168c67385d5dd2105e374e7a8d10bbed67b5ffd3f14a963f3deaea98ea71195 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833, name=ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-mgr-x, GIT_BRANCH=HEAD, io.buildah.version=1.36.0, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, RELEASE=squid-de09dd0, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=) 2024-08-19T01:44:16.154 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Aug 19 01:44:16 smithi083 podman[84228]: 2024-08-19 01:44:16.100693409 +0000 UTC m=+0.861721336 container cleanup c168c67385d5dd2105e374e7a8d10bbed67b5ffd3f14a963f3deaea98ea71195 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833, name=ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-mgr-x, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.build-date=20240731, GIT_CLEAN=True, RELEASE=squid-de09dd0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.36.0) 2024-08-19T01:44:16.154 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Aug 19 01:44:16 smithi083 bash[84228]: ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-mgr-x 2024-08-19T01:44:16.538 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Aug 19 01:44:16 smithi083 podman[84242]: 2024-08-19 01:44:16.285647875 +0000 UTC m=+0.790244050 container remove c168c67385d5dd2105e374e7a8d10bbed67b5ffd3f14a963f3deaea98ea71195 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833, name=ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-mgr-x, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, ceph=True, maintainer=Guillaume Abrioux , io.buildah.version=1.36.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, RELEASE=squid-de09dd0, org.label-schema.vendor=CentOS) 2024-08-19T01:44:16.561 DEBUG:teuthology.orchestra.run.smithi083:> sudo pkill -f 'journalctl -f -n 0 -u ceph-bad8780e-5dc9-11ef-bccf-c7b262605968@mgr.x.service' 2024-08-19T01:44:16.635 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-08-19T01:44:16.635 INFO:tasks.cephadm.mgr.x:Stopped mgr.x 2024-08-19T01:44:16.635 INFO:tasks.cephadm.mgr.y:Stopping mgr.y... 2024-08-19T01:44:16.635 DEBUG:teuthology.orchestra.run.smithi117:> sudo systemctl stop ceph-bad8780e-5dc9-11ef-bccf-c7b262605968@mgr.y 2024-08-19T01:44:16.952 INFO:journalctl@ceph.mgr.y.smithi117.stdout:Aug 19 01:44:16 smithi117 systemd[1]: Stopping Ceph mgr.y for bad8780e-5dc9-11ef-bccf-c7b262605968... 2024-08-19T01:44:17.259 INFO:journalctl@ceph.mgr.y.smithi117.stdout:Aug 19 01:44:16 smithi117 podman[55652]: 2024-08-19 01:44:16.970928488 +0000 UTC m=+0.144632252 container died 01f29b2e497cc95d0e2773dee9b554f625c33b929a05fa79cea6abd9f3bf2878 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c9dd28080a631be25c77c36c63b17a3ef340ffb2e21c87d4d7b56b6398436d9, name=ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-mgr-y, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.36.0, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, RELEASE=squid-de09dd0, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, ceph=True, org.label-schema.build-date=20240731) 2024-08-19T01:44:17.602 INFO:journalctl@ceph.mgr.y.smithi117.stdout:Aug 19 01:44:17 smithi117 podman[55652]: 2024-08-19 01:44:17.366857558 +0000 UTC m=+0.540561321 container cleanup 01f29b2e497cc95d0e2773dee9b554f625c33b929a05fa79cea6abd9f3bf2878 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c9dd28080a631be25c77c36c63b17a3ef340ffb2e21c87d4d7b56b6398436d9, name=ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-mgr-y, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, RELEASE=squid-de09dd0, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, ceph=True, io.buildah.version=1.36.0, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image) 2024-08-19T01:44:17.602 INFO:journalctl@ceph.mgr.y.smithi117.stdout:Aug 19 01:44:17 smithi117 bash[55652]: ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-mgr-y 2024-08-19T01:44:17.729 DEBUG:teuthology.orchestra.run.smithi117:> sudo pkill -f 'journalctl -f -n 0 -u ceph-bad8780e-5dc9-11ef-bccf-c7b262605968@mgr.y.service' 2024-08-19T01:44:17.763 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-08-19T01:44:17.764 INFO:tasks.cephadm.mgr.y:Stopped mgr.y 2024-08-19T01:44:17.764 INFO:tasks.cephadm.mgr.z:Stopping mgr.z... 2024-08-19T01:44:17.764 DEBUG:teuthology.orchestra.run.smithi167:> sudo systemctl stop ceph-bad8780e-5dc9-11ef-bccf-c7b262605968@mgr.z 2024-08-19T01:44:18.078 INFO:journalctl@ceph.mgr.z.smithi167.stdout:Aug 19 01:44:17 smithi167 systemd[1]: Stopping Ceph mgr.z for bad8780e-5dc9-11ef-bccf-c7b262605968... 2024-08-19T01:44:18.349 INFO:journalctl@ceph.mgr.z.smithi167.stdout:Aug 19 01:44:18 smithi167 podman[56432]: 2024-08-19 01:44:18.077285497 +0000 UTC m=+0.118640215 container died e7079f51b08daa4e8ee256732662fb1a4d1b90173a6a67f33ef8a6315c1b26a7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c9dd28080a631be25c77c36c63b17a3ef340ffb2e21c87d4d7b56b6398436d9, name=ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-mgr-z, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, org.label-schema.license=GPLv2, RELEASE=squid-de09dd0, io.buildah.version=1.36.0, GIT_BRANCH=HEAD) 2024-08-19T01:44:18.716 INFO:journalctl@ceph.mgr.z.smithi167.stdout:Aug 19 01:44:18 smithi167 podman[56432]: 2024-08-19 01:44:18.422861022 +0000 UTC m=+0.464215742 container cleanup e7079f51b08daa4e8ee256732662fb1a4d1b90173a6a67f33ef8a6315c1b26a7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c9dd28080a631be25c77c36c63b17a3ef340ffb2e21c87d4d7b56b6398436d9, name=ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-mgr-z, io.buildah.version=1.36.0, ceph=True, RELEASE=squid-de09dd0, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS) 2024-08-19T01:44:18.716 INFO:journalctl@ceph.mgr.z.smithi167.stdout:Aug 19 01:44:18 smithi167 bash[56432]: ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-mgr-z 2024-08-19T01:44:18.716 INFO:journalctl@ceph.mgr.z.smithi167.stdout:Aug 19 01:44:18 smithi167 podman[56444]: 2024-08-19 01:44:18.582803419 +0000 UTC m=+0.500555104 container remove e7079f51b08daa4e8ee256732662fb1a4d1b90173a6a67f33ef8a6315c1b26a7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c9dd28080a631be25c77c36c63b17a3ef340ffb2e21c87d4d7b56b6398436d9, name=ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-mgr-z, io.buildah.version=1.36.0, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=squid-de09dd0, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS) 2024-08-19T01:44:18.716 INFO:journalctl@ceph.mgr.z.smithi167.stdout:Aug 19 01:44:18 smithi167 systemd[1]: ceph-bad8780e-5dc9-11ef-bccf-c7b262605968@mgr.z.service: Main process exited, code=exited, status=143/n/a 2024-08-19T01:44:18.809 DEBUG:teuthology.orchestra.run.smithi167:> sudo pkill -f 'journalctl -f -n 0 -u ceph-bad8780e-5dc9-11ef-bccf-c7b262605968@mgr.z.service' 2024-08-19T01:44:18.844 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-08-19T01:44:18.844 INFO:tasks.cephadm.mgr.z:Stopped mgr.z 2024-08-19T01:44:18.844 INFO:tasks.cephadm.osd.0:Stopping osd.0... 2024-08-19T01:44:18.845 DEBUG:teuthology.orchestra.run.smithi083:> sudo systemctl stop ceph-bad8780e-5dc9-11ef-bccf-c7b262605968@osd.0 2024-08-19T01:44:19.154 INFO:journalctl@ceph.osd.0.smithi083.stdout:Aug 19 01:44:18 smithi083 systemd[1]: Stopping Ceph osd.0 for bad8780e-5dc9-11ef-bccf-c7b262605968... 2024-08-19T01:44:19.472 INFO:journalctl@ceph.osd.0.smithi083.stdout:Aug 19 01:44:19 smithi083 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-0[31265]: 2024-08-19T01:44:19.274+0000 7fb3bce00640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.0 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-08-19T01:44:19.473 INFO:journalctl@ceph.osd.0.smithi083.stdout:Aug 19 01:44:19 smithi083 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-0[31265]: 2024-08-19T01:44:19.274+0000 7fb3bce00640 -1 osd.0 89 *** Got signal Terminated *** 2024-08-19T01:44:19.473 INFO:journalctl@ceph.osd.0.smithi083.stdout:Aug 19 01:44:19 smithi083 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-0[31265]: 2024-08-19T01:44:19.274+0000 7fb3bce00640 -1 osd.0 89 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-08-19T01:44:25.100 INFO:journalctl@ceph.osd.0.smithi083.stdout:Aug 19 01:44:24 smithi083 podman[84348]: 2024-08-19 01:44:24.799347258 +0000 UTC m=+5.658051870 container died 2b55f821138b638a9861e0b46ca236adfff8ed7c97a227d67b676f65c9d63b38 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c9dd28080a631be25c77c36c63b17a3ef340ffb2e21c87d4d7b56b6398436d9, name=ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-0, ceph=True, CEPH_POINT_RELEASE=, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, io.buildah.version=1.36.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, RELEASE=squid-de09dd0) 2024-08-19T01:44:25.404 INFO:journalctl@ceph.osd.0.smithi083.stdout:Aug 19 01:44:25 smithi083 podman[84348]: 2024-08-19 01:44:25.197132826 +0000 UTC m=+6.055837430 container cleanup 2b55f821138b638a9861e0b46ca236adfff8ed7c97a227d67b676f65c9d63b38 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c9dd28080a631be25c77c36c63b17a3ef340ffb2e21c87d4d7b56b6398436d9, name=ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-0, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=squid-de09dd0, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, io.buildah.version=1.36.0, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-08-19T01:44:25.404 INFO:journalctl@ceph.osd.0.smithi083.stdout:Aug 19 01:44:25 smithi083 bash[84348]: ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-0 2024-08-19T01:44:25.904 INFO:journalctl@ceph.osd.0.smithi083.stdout:Aug 19 01:44:25 smithi083 podman[84376]: 2024-08-19 01:44:25.540726182 +0000 UTC m=+0.734610948 container remove 2b55f821138b638a9861e0b46ca236adfff8ed7c97a227d67b676f65c9d63b38 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c9dd28080a631be25c77c36c63b17a3ef340ffb2e21c87d4d7b56b6398436d9, name=ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-0, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, ceph=True, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.36.0, RELEASE=squid-de09dd0, org.label-schema.name=CentOS Stream 9 Base Image) 2024-08-19T01:44:26.263 INFO:journalctl@ceph.osd.0.smithi083.stdout:Aug 19 01:44:25 smithi083 podman[84469]: 2024-08-19 01:44:25.903212062 +0000 UTC m=+0.028789165 image pull ea7a5189f30bad3aca9524788e2965d2368d7ce786df09d4edcd8c3edf9cefba quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c9dd28080a631be25c77c36c63b17a3ef340ffb2e21c87d4d7b56b6398436d9 2024-08-19T01:44:26.263 INFO:journalctl@ceph.osd.0.smithi083.stdout:Aug 19 01:44:26 smithi083 podman[84469]: 2024-08-19 01:44:26.017083226 +0000 UTC m=+0.142660299 container create 5886438a7e4f7cf864c61b06bff3f57faa0e6dc6c8a218987673aa698f13d8b6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c9dd28080a631be25c77c36c63b17a3ef340ffb2e21c87d4d7b56b6398436d9, name=ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-0-deactivate, io.buildah.version=1.36.0, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=squid-de09dd0, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.build-date=20240731) 2024-08-19T01:44:26.524 INFO:journalctl@ceph.osd.0.smithi083.stdout:Aug 19 01:44:26 smithi083 podman[84469]: 2024-08-19 01:44:26.26208719 +0000 UTC m=+0.387664274 container init 5886438a7e4f7cf864c61b06bff3f57faa0e6dc6c8a218987673aa698f13d8b6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c9dd28080a631be25c77c36c63b17a3ef340ffb2e21c87d4d7b56b6398436d9, name=ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-0-deactivate, CEPH_POINT_RELEASE=, RELEASE=squid-de09dd0, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, io.buildah.version=1.36.0, org.label-schema.vendor=CentOS) 2024-08-19T01:44:26.524 INFO:journalctl@ceph.osd.0.smithi083.stdout:Aug 19 01:44:26 smithi083 podman[84469]: 2024-08-19 01:44:26.2693499 +0000 UTC m=+0.394926982 container start 5886438a7e4f7cf864c61b06bff3f57faa0e6dc6c8a218987673aa698f13d8b6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c9dd28080a631be25c77c36c63b17a3ef340ffb2e21c87d4d7b56b6398436d9, name=ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-0-deactivate, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, RELEASE=squid-de09dd0, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.36.0, GIT_BRANCH=HEAD) 2024-08-19T01:44:26.524 INFO:journalctl@ceph.osd.0.smithi083.stdout:Aug 19 01:44:26 smithi083 podman[84469]: 2024-08-19 01:44:26.353019857 +0000 UTC m=+0.478596962 container attach 5886438a7e4f7cf864c61b06bff3f57faa0e6dc6c8a218987673aa698f13d8b6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c9dd28080a631be25c77c36c63b17a3ef340ffb2e21c87d4d7b56b6398436d9, name=ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-0-deactivate, GIT_CLEAN=True, CEPH_POINT_RELEASE=, ceph=True, io.buildah.version=1.36.0, RELEASE=squid-de09dd0, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS) 2024-08-19T01:44:26.897 INFO:journalctl@ceph.osd.0.smithi083.stdout:Aug 19 01:44:26 smithi083 podman[84493]: 2024-08-19 01:44:26.565163513 +0000 UTC m=+0.028420487 container died 5886438a7e4f7cf864c61b06bff3f57faa0e6dc6c8a218987673aa698f13d8b6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c9dd28080a631be25c77c36c63b17a3ef340ffb2e21c87d4d7b56b6398436d9, name=ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-0-deactivate, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, RELEASE=squid-de09dd0, io.buildah.version=1.36.0, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, maintainer=Guillaume Abrioux ) 2024-08-19T01:44:27.263 DEBUG:teuthology.orchestra.run.smithi083:> sudo pkill -f 'journalctl -f -n 0 -u ceph-bad8780e-5dc9-11ef-bccf-c7b262605968@osd.0.service' 2024-08-19T01:44:27.338 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-08-19T01:44:27.339 INFO:tasks.cephadm.osd.0:Stopped osd.0 2024-08-19T01:44:27.339 INFO:tasks.cephadm.osd.1:Stopping osd.1... 2024-08-19T01:44:27.339 DEBUG:teuthology.orchestra.run.smithi117:> sudo systemctl stop ceph-bad8780e-5dc9-11ef-bccf-c7b262605968@osd.1 2024-08-19T01:44:27.646 INFO:journalctl@ceph.osd.1.smithi117.stdout:Aug 19 01:44:27 smithi117 systemd[1]: Stopping Ceph osd.1 for bad8780e-5dc9-11ef-bccf-c7b262605968... 2024-08-19T01:44:27.952 INFO:journalctl@ceph.osd.1.smithi117.stdout:Aug 19 01:44:27 smithi117 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-1[26443]: 2024-08-19T01:44:27.643+0000 7f6d790af640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.1 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-08-19T01:44:27.952 INFO:journalctl@ceph.osd.1.smithi117.stdout:Aug 19 01:44:27 smithi117 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-1[26443]: 2024-08-19T01:44:27.643+0000 7f6d790af640 -1 osd.1 89 *** Got signal Terminated *** 2024-08-19T01:44:27.952 INFO:journalctl@ceph.osd.1.smithi117.stdout:Aug 19 01:44:27 smithi117 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-1[26443]: 2024-08-19T01:44:27.643+0000 7f6d790af640 -1 osd.1 89 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-08-19T01:44:33.452 INFO:journalctl@ceph.osd.1.smithi117.stdout:Aug 19 01:44:33 smithi117 podman[55781]: 2024-08-19 01:44:33.199456356 +0000 UTC m=+5.654203258 container died a14802c52aa693674c175119f115030b79e2184fa3b9ed730562e61362f5286d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c9dd28080a631be25c77c36c63b17a3ef340ffb2e21c87d4d7b56b6398436d9, name=ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-1, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , io.buildah.version=1.36.0, ceph=True, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-de09dd0) 2024-08-19T01:44:33.871 INFO:journalctl@ceph.osd.1.smithi117.stdout:Aug 19 01:44:33 smithi117 podman[55781]: 2024-08-19 01:44:33.564582862 +0000 UTC m=+6.019329768 container cleanup a14802c52aa693674c175119f115030b79e2184fa3b9ed730562e61362f5286d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c9dd28080a631be25c77c36c63b17a3ef340ffb2e21c87d4d7b56b6398436d9, name=ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-1, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.36.0, CEPH_POINT_RELEASE=, RELEASE=squid-de09dd0, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2) 2024-08-19T01:44:33.871 INFO:journalctl@ceph.osd.1.smithi117.stdout:Aug 19 01:44:33 smithi117 bash[55781]: ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-1 2024-08-19T01:44:34.195 INFO:journalctl@ceph.osd.1.smithi117.stdout:Aug 19 01:44:33 smithi117 podman[55806]: 2024-08-19 01:44:33.870924534 +0000 UTC m=+0.674296859 container remove a14802c52aa693674c175119f115030b79e2184fa3b9ed730562e61362f5286d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c9dd28080a631be25c77c36c63b17a3ef340ffb2e21c87d4d7b56b6398436d9, name=ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-1, RELEASE=squid-de09dd0, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.build-date=20240731, io.buildah.version=1.36.0, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-08-19T01:44:34.452 INFO:journalctl@ceph.osd.1.smithi117.stdout:Aug 19 01:44:34 smithi117 podman[55896]: 2024-08-19 01:44:34.097669142 +0000 UTC m=+0.017624846 image pull ea7a5189f30bad3aca9524788e2965d2368d7ce786df09d4edcd8c3edf9cefba quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c9dd28080a631be25c77c36c63b17a3ef340ffb2e21c87d4d7b56b6398436d9 2024-08-19T01:44:34.452 INFO:journalctl@ceph.osd.1.smithi117.stdout:Aug 19 01:44:34 smithi117 podman[55896]: 2024-08-19 01:44:34.20528854 +0000 UTC m=+0.125244280 container create cae96cbb50e97025d7692ea73b0d6a5ed852027bc90b79c6acf1a3765654c064 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c9dd28080a631be25c77c36c63b17a3ef340ffb2e21c87d4d7b56b6398436d9, name=ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-1-deactivate, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, io.buildah.version=1.36.0, maintainer=Guillaume Abrioux , GIT_CLEAN=True, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, RELEASE=squid-de09dd0, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731) 2024-08-19T01:44:34.453 INFO:journalctl@ceph.osd.1.smithi117.stdout:Aug 19 01:44:34 smithi117 podman[55896]: 2024-08-19 01:44:34.41605461 +0000 UTC m=+0.336010311 container init cae96cbb50e97025d7692ea73b0d6a5ed852027bc90b79c6acf1a3765654c064 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c9dd28080a631be25c77c36c63b17a3ef340ffb2e21c87d4d7b56b6398436d9, name=ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-1-deactivate, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.36.0, RELEASE=squid-de09dd0, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, ceph=True, GIT_BRANCH=HEAD) 2024-08-19T01:44:34.453 INFO:journalctl@ceph.osd.1.smithi117.stdout:Aug 19 01:44:34 smithi117 podman[55896]: 2024-08-19 01:44:34.419565311 +0000 UTC m=+0.339521019 container start cae96cbb50e97025d7692ea73b0d6a5ed852027bc90b79c6acf1a3765654c064 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c9dd28080a631be25c77c36c63b17a3ef340ffb2e21c87d4d7b56b6398436d9, name=ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-1-deactivate, ceph=True, org.label-schema.vendor=CentOS, io.buildah.version=1.36.0, CEPH_POINT_RELEASE=, GIT_CLEAN=True, RELEASE=squid-de09dd0, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD) 2024-08-19T01:44:34.902 INFO:journalctl@ceph.osd.1.smithi117.stdout:Aug 19 01:44:34 smithi117 podman[55896]: 2024-08-19 01:44:34.507648537 +0000 UTC m=+0.427604235 container attach cae96cbb50e97025d7692ea73b0d6a5ed852027bc90b79c6acf1a3765654c064 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c9dd28080a631be25c77c36c63b17a3ef340ffb2e21c87d4d7b56b6398436d9, name=ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-1-deactivate, org.label-schema.schema-version=1.0, io.buildah.version=1.36.0, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, RELEASE=squid-de09dd0, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image) 2024-08-19T01:44:34.903 INFO:journalctl@ceph.osd.1.smithi117.stdout:Aug 19 01:44:34 smithi117 podman[55896]: 2024-08-19 01:44:34.635999804 +0000 UTC m=+0.555955505 container died cae96cbb50e97025d7692ea73b0d6a5ed852027bc90b79c6acf1a3765654c064 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c9dd28080a631be25c77c36c63b17a3ef340ffb2e21c87d4d7b56b6398436d9, name=ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-1-deactivate, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-de09dd0, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , io.buildah.version=1.36.0) 2024-08-19T01:44:35.189 DEBUG:teuthology.orchestra.run.smithi117:> sudo pkill -f 'journalctl -f -n 0 -u ceph-bad8780e-5dc9-11ef-bccf-c7b262605968@osd.1.service' 2024-08-19T01:44:35.228 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-08-19T01:44:35.228 INFO:tasks.cephadm.osd.1:Stopped osd.1 2024-08-19T01:44:35.228 INFO:tasks.cephadm.osd.2:Stopping osd.2... 2024-08-19T01:44:35.229 DEBUG:teuthology.orchestra.run.smithi167:> sudo systemctl stop ceph-bad8780e-5dc9-11ef-bccf-c7b262605968@osd.2 2024-08-19T01:44:35.537 INFO:journalctl@ceph.osd.2.smithi167.stdout:Aug 19 01:44:35 smithi167 systemd[1]: Stopping Ceph osd.2 for bad8780e-5dc9-11ef-bccf-c7b262605968... 2024-08-19T01:44:35.965 INFO:journalctl@ceph.osd.2.smithi167.stdout:Aug 19 01:44:35 smithi167 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-2[26831]: 2024-08-19T01:44:35.535+0000 7fa6ea61c640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.2 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-08-19T01:44:35.966 INFO:journalctl@ceph.osd.2.smithi167.stdout:Aug 19 01:44:35 smithi167 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-2[26831]: 2024-08-19T01:44:35.535+0000 7fa6ea61c640 -1 osd.2 89 *** Got signal Terminated *** 2024-08-19T01:44:35.966 INFO:journalctl@ceph.osd.2.smithi167.stdout:Aug 19 01:44:35 smithi167 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-2[26831]: 2024-08-19T01:44:35.535+0000 7fa6ea61c640 -1 osd.2 89 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-08-19T01:44:41.216 INFO:journalctl@ceph.osd.2.smithi167.stdout:Aug 19 01:44:40 smithi167 podman[56568]: 2024-08-19 01:44:40.936927421 +0000 UTC m=+5.504320531 container died 415027713ec592d7432fe76b3837a712118eb2526c97b19f4729c6d6d993c31a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c9dd28080a631be25c77c36c63b17a3ef340ffb2e21c87d4d7b56b6398436d9, name=ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-2, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_CLEAN=True, io.buildah.version=1.36.0, RELEASE=squid-de09dd0) 2024-08-19T01:44:41.586 INFO:journalctl@ceph.osd.2.smithi167.stdout:Aug 19 01:44:41 smithi167 podman[56568]: 2024-08-19 01:44:41.343908914 +0000 UTC m=+5.911302028 container cleanup 415027713ec592d7432fe76b3837a712118eb2526c97b19f4729c6d6d993c31a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c9dd28080a631be25c77c36c63b17a3ef340ffb2e21c87d4d7b56b6398436d9, name=ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, CEPH_POINT_RELEASE=, io.buildah.version=1.36.0, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, RELEASE=squid-de09dd0) 2024-08-19T01:44:41.586 INFO:journalctl@ceph.osd.2.smithi167.stdout:Aug 19 01:44:41 smithi167 bash[56568]: ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-2 2024-08-19T01:44:41.904 INFO:journalctl@ceph.osd.2.smithi167.stdout:Aug 19 01:44:41 smithi167 podman[56583]: 2024-08-19 01:44:41.585537778 +0000 UTC m=+0.643971593 container remove 415027713ec592d7432fe76b3837a712118eb2526c97b19f4729c6d6d993c31a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c9dd28080a631be25c77c36c63b17a3ef340ffb2e21c87d4d7b56b6398436d9, name=ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-2, ceph=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, GIT_CLEAN=True, RELEASE=squid-de09dd0, io.buildah.version=1.36.0) 2024-08-19T01:44:42.199 INFO:journalctl@ceph.osd.2.smithi167.stdout:Aug 19 01:44:41 smithi167 podman[56672]: 2024-08-19 01:44:41.904242529 +0000 UTC m=+0.111005954 container create 3de11d6c33d4f3b15a41f98ee83f615dd4bbf228b695d371c26be2114e03a8cd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c9dd28080a631be25c77c36c63b17a3ef340ffb2e21c87d4d7b56b6398436d9, name=ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-2-deactivate, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, io.buildah.version=1.36.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=squid-de09dd0, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, ceph=True, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image) 2024-08-19T01:44:42.199 INFO:journalctl@ceph.osd.2.smithi167.stdout:Aug 19 01:44:41 smithi167 podman[56672]: 2024-08-19 01:44:41.81031007 +0000 UTC m=+0.017073493 image pull ea7a5189f30bad3aca9524788e2965d2368d7ce786df09d4edcd8c3edf9cefba quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c9dd28080a631be25c77c36c63b17a3ef340ffb2e21c87d4d7b56b6398436d9 2024-08-19T01:44:42.199 INFO:journalctl@ceph.osd.2.smithi167.stdout:Aug 19 01:44:42 smithi167 podman[56672]: 2024-08-19 01:44:42.123676138 +0000 UTC m=+0.330439575 container init 3de11d6c33d4f3b15a41f98ee83f615dd4bbf228b695d371c26be2114e03a8cd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c9dd28080a631be25c77c36c63b17a3ef340ffb2e21c87d4d7b56b6398436d9, name=ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-2-deactivate, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, io.buildah.version=1.36.0, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=squid-de09dd0, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.build-date=20240731) 2024-08-19T01:44:42.199 INFO:journalctl@ceph.osd.2.smithi167.stdout:Aug 19 01:44:42 smithi167 podman[56672]: 2024-08-19 01:44:42.127056693 +0000 UTC m=+0.333820136 container start 3de11d6c33d4f3b15a41f98ee83f615dd4bbf228b695d371c26be2114e03a8cd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c9dd28080a631be25c77c36c63b17a3ef340ffb2e21c87d4d7b56b6398436d9, name=ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-2-deactivate, RELEASE=squid-de09dd0, io.buildah.version=1.36.0, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2) 2024-08-19T01:44:42.466 INFO:journalctl@ceph.osd.2.smithi167.stdout:Aug 19 01:44:42 smithi167 podman[56672]: 2024-08-19 01:44:42.198623065 +0000 UTC m=+0.405386505 container attach 3de11d6c33d4f3b15a41f98ee83f615dd4bbf228b695d371c26be2114e03a8cd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c9dd28080a631be25c77c36c63b17a3ef340ffb2e21c87d4d7b56b6398436d9, name=ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-2-deactivate, io.buildah.version=1.36.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.schema-version=1.0, GIT_CLEAN=True, RELEASE=squid-de09dd0, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2) 2024-08-19T01:44:42.466 INFO:journalctl@ceph.osd.2.smithi167.stdout:Aug 19 01:44:42 smithi167 podman[56672]: 2024-08-19 01:44:42.315122957 +0000 UTC m=+0.521886381 container died 3de11d6c33d4f3b15a41f98ee83f615dd4bbf228b695d371c26be2114e03a8cd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c9dd28080a631be25c77c36c63b17a3ef340ffb2e21c87d4d7b56b6398436d9, name=ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-2-deactivate, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=squid-de09dd0, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_CLEAN=True, io.buildah.version=1.36.0, ceph=True, org.label-schema.license=GPLv2) 2024-08-19T01:44:42.916 DEBUG:teuthology.orchestra.run.smithi167:> sudo pkill -f 'journalctl -f -n 0 -u ceph-bad8780e-5dc9-11ef-bccf-c7b262605968@osd.2.service' 2024-08-19T01:44:42.955 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-08-19T01:44:42.955 INFO:tasks.cephadm.osd.2:Stopped osd.2 2024-08-19T01:44:42.955 INFO:tasks.cephadm.osd.3:Stopping osd.3... 2024-08-19T01:44:42.955 DEBUG:teuthology.orchestra.run.smithi083:> sudo systemctl stop ceph-bad8780e-5dc9-11ef-bccf-c7b262605968@osd.3 2024-08-19T01:44:43.347 INFO:journalctl@ceph.osd.3.smithi083.stdout:Aug 19 01:44:43 smithi083 systemd[1]: Stopping Ceph osd.3 for bad8780e-5dc9-11ef-bccf-c7b262605968... 2024-08-19T01:44:43.654 INFO:journalctl@ceph.osd.3.smithi083.stdout:Aug 19 01:44:43 smithi083 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-3[38049]: 2024-08-19T01:44:43.345+0000 7f49a0200640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.3 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-08-19T01:44:43.654 INFO:journalctl@ceph.osd.3.smithi083.stdout:Aug 19 01:44:43 smithi083 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-3[38049]: 2024-08-19T01:44:43.345+0000 7f49a0200640 -1 osd.3 89 *** Got signal Terminated *** 2024-08-19T01:44:43.654 INFO:journalctl@ceph.osd.3.smithi083.stdout:Aug 19 01:44:43 smithi083 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-3[38049]: 2024-08-19T01:44:43.345+0000 7f49a0200640 -1 osd.3 89 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-08-19T01:44:46.966 INFO:journalctl@ceph.osd.11.smithi167.stdout:Aug 19 01:44:46 smithi167 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-11[46643]: 2024-08-19T01:44:46.699+0000 7f447f624640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-08-19T01:44:22.880210+0000 front 2024-08-19T01:44:22.880404+0000 (oldest deadline 2024-08-19T01:44:45.780352+0000) 2024-08-19T01:44:47.466 INFO:journalctl@ceph.osd.5.smithi167.stdout:Aug 19 01:44:47 smithi167 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-5[33276]: 2024-08-19T01:44:47.163+0000 7f8191d31640 -1 osd.5 89 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-08-19T01:44:22.949456+0000 front 2024-08-19T01:44:22.949396+0000 (oldest deadline 2024-08-19T01:44:46.449550+0000) 2024-08-19T01:44:47.904 INFO:journalctl@ceph.osd.9.smithi083.stdout:Aug 19 01:44:47 smithi083 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-9[51472]: 2024-08-19T01:44:47.592+0000 7f54e8c00640 -1 osd.9 89 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-08-19T01:44:20.873282+0000 front 2024-08-19T01:44:20.873259+0000 (oldest deadline 2024-08-19T01:44:46.773413+0000) 2024-08-19T01:44:47.952 INFO:journalctl@ceph.osd.10.smithi117.stdout:Aug 19 01:44:47 smithi117 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-10[46616]: 2024-08-19T01:44:47.614+0000 7f34b3bf7640 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-08-19T01:44:24.060751+0000 front 2024-08-19T01:44:24.060716+0000 (oldest deadline 2024-08-19T01:44:47.560677+0000) 2024-08-19T01:44:47.965 INFO:journalctl@ceph.osd.11.smithi167.stdout:Aug 19 01:44:47 smithi167 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-11[46643]: 2024-08-19T01:44:47.659+0000 7f447f624640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-08-19T01:44:22.880210+0000 front 2024-08-19T01:44:22.880404+0000 (oldest deadline 2024-08-19T01:44:45.780352+0000) 2024-08-19T01:44:48.466 INFO:journalctl@ceph.osd.8.smithi167.stdout:Aug 19 01:44:48 smithi167 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-8[39928]: 2024-08-19T01:44:48.152+0000 7f17fe820640 -1 osd.8 89 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-08-19T01:44:22.747507+0000 front 2024-08-19T01:44:22.747516+0000 (oldest deadline 2024-08-19T01:44:48.047324+0000) 2024-08-19T01:44:48.466 INFO:journalctl@ceph.osd.5.smithi167.stdout:Aug 19 01:44:48 smithi167 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-5[33276]: 2024-08-19T01:44:48.127+0000 7f8191d31640 -1 osd.5 89 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-08-19T01:44:22.949456+0000 front 2024-08-19T01:44:22.949396+0000 (oldest deadline 2024-08-19T01:44:46.449550+0000) 2024-08-19T01:44:48.571 INFO:journalctl@ceph.osd.3.smithi083.stdout:Aug 19 01:44:48 smithi083 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-3[38049]: 2024-08-19T01:44:48.248+0000 7f499c600640 -1 osd.3 89 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-08-19T01:44:23.243929+0000 front 2024-08-19T01:44:23.243960+0000 (oldest deadline 2024-08-19T01:44:47.344331+0000) 2024-08-19T01:44:48.904 INFO:journalctl@ceph.osd.9.smithi083.stdout:Aug 19 01:44:48 smithi083 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-9[51472]: 2024-08-19T01:44:48.569+0000 7f54e8c00640 -1 osd.9 89 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-08-19T01:44:20.873282+0000 front 2024-08-19T01:44:20.873259+0000 (oldest deadline 2024-08-19T01:44:46.773413+0000) 2024-08-19T01:44:48.905 INFO:journalctl@ceph.osd.3.smithi083.stdout:Aug 19 01:44:48 smithi083 podman[84574]: 2024-08-19 01:44:48.774448034 +0000 UTC m=+5.521308534 container died ee290eb0504a16b56e543d2a6cc7f0021245722f91cd1423a65fc71d0a30fd78 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c9dd28080a631be25c77c36c63b17a3ef340ffb2e21c87d4d7b56b6398436d9, name=ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-3, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, RELEASE=squid-de09dd0, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, io.buildah.version=1.36.0, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.license=GPLv2) 2024-08-19T01:44:48.905 INFO:journalctl@ceph.osd.6.smithi083.stdout:Aug 19 01:44:48 smithi083 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-6[44809]: 2024-08-19T01:44:48.624+0000 7f3cc6e00640 -1 osd.6 89 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-08-19T01:44:21.901735+0000 front 2024-08-19T01:44:21.901860+0000 (oldest deadline 2024-08-19T01:44:47.801737+0000) 2024-08-19T01:44:48.952 INFO:journalctl@ceph.osd.10.smithi117.stdout:Aug 19 01:44:48 smithi117 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-10[46616]: 2024-08-19T01:44:48.640+0000 7f34b3bf7640 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-08-19T01:44:24.060751+0000 front 2024-08-19T01:44:24.060716+0000 (oldest deadline 2024-08-19T01:44:47.560677+0000) 2024-08-19T01:44:48.965 INFO:journalctl@ceph.osd.11.smithi167.stdout:Aug 19 01:44:48 smithi167 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-11[46643]: 2024-08-19T01:44:48.653+0000 7f447f624640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-08-19T01:44:22.880210+0000 front 2024-08-19T01:44:22.880404+0000 (oldest deadline 2024-08-19T01:44:45.780352+0000) 2024-08-19T01:44:49.404 INFO:journalctl@ceph.osd.3.smithi083.stdout:Aug 19 01:44:49 smithi083 podman[84574]: 2024-08-19 01:44:49.209318986 +0000 UTC m=+5.956179484 container cleanup ee290eb0504a16b56e543d2a6cc7f0021245722f91cd1423a65fc71d0a30fd78 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c9dd28080a631be25c77c36c63b17a3ef340ffb2e21c87d4d7b56b6398436d9, name=ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-3, io.buildah.version=1.36.0, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, RELEASE=squid-de09dd0, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image) 2024-08-19T01:44:49.404 INFO:journalctl@ceph.osd.3.smithi083.stdout:Aug 19 01:44:49 smithi083 bash[84574]: ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-3 2024-08-19T01:44:49.466 INFO:journalctl@ceph.osd.5.smithi167.stdout:Aug 19 01:44:49 smithi167 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-5[33276]: 2024-08-19T01:44:49.117+0000 7f8191d31640 -1 osd.5 89 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-08-19T01:44:22.949456+0000 front 2024-08-19T01:44:22.949396+0000 (oldest deadline 2024-08-19T01:44:46.449550+0000) 2024-08-19T01:44:49.466 INFO:journalctl@ceph.osd.8.smithi167.stdout:Aug 19 01:44:49 smithi167 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-8[39928]: 2024-08-19T01:44:49.110+0000 7f17fe820640 -1 osd.8 89 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-08-19T01:44:22.747507+0000 front 2024-08-19T01:44:22.747516+0000 (oldest deadline 2024-08-19T01:44:48.047324+0000) 2024-08-19T01:44:49.904 INFO:journalctl@ceph.osd.6.smithi083.stdout:Aug 19 01:44:49 smithi083 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-6[44809]: 2024-08-19T01:44:49.585+0000 7f3cc6e00640 -1 osd.6 89 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-08-19T01:44:21.901735+0000 front 2024-08-19T01:44:21.901860+0000 (oldest deadline 2024-08-19T01:44:47.801737+0000) 2024-08-19T01:44:49.905 INFO:journalctl@ceph.osd.9.smithi083.stdout:Aug 19 01:44:49 smithi083 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-9[51472]: 2024-08-19T01:44:49.559+0000 7f54e8c00640 -1 osd.9 89 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-08-19T01:44:20.873282+0000 front 2024-08-19T01:44:20.873259+0000 (oldest deadline 2024-08-19T01:44:46.773413+0000) 2024-08-19T01:44:49.905 INFO:journalctl@ceph.osd.3.smithi083.stdout:Aug 19 01:44:49 smithi083 podman[84591]: 2024-08-19 01:44:49.444055298 +0000 UTC m=+0.656153472 container remove ee290eb0504a16b56e543d2a6cc7f0021245722f91cd1423a65fc71d0a30fd78 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c9dd28080a631be25c77c36c63b17a3ef340ffb2e21c87d4d7b56b6398436d9, name=ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-3, io.buildah.version=1.36.0, RELEASE=squid-de09dd0, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, ceph=True, maintainer=Guillaume Abrioux ) 2024-08-19T01:44:49.954 INFO:journalctl@ceph.osd.10.smithi117.stdout:Aug 19 01:44:49 smithi117 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-10[46616]: 2024-08-19T01:44:49.599+0000 7f34b3bf7640 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-08-19T01:44:24.060751+0000 front 2024-08-19T01:44:24.060716+0000 (oldest deadline 2024-08-19T01:44:47.560677+0000) 2024-08-19T01:44:49.965 INFO:journalctl@ceph.osd.11.smithi167.stdout:Aug 19 01:44:49 smithi167 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-11[46643]: 2024-08-19T01:44:49.609+0000 7f447f624640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-08-19T01:44:22.880210+0000 front 2024-08-19T01:44:22.880404+0000 (oldest deadline 2024-08-19T01:44:45.780352+0000) 2024-08-19T01:44:50.338 INFO:journalctl@ceph.osd.3.smithi083.stdout:Aug 19 01:44:49 smithi083 podman[84677]: 2024-08-19 01:44:49.898331524 +0000 UTC m=+0.029522936 image pull ea7a5189f30bad3aca9524788e2965d2368d7ce786df09d4edcd8c3edf9cefba quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c9dd28080a631be25c77c36c63b17a3ef340ffb2e21c87d4d7b56b6398436d9 2024-08-19T01:44:50.338 INFO:journalctl@ceph.osd.3.smithi083.stdout:Aug 19 01:44:50 smithi083 podman[84677]: 2024-08-19 01:44:50.053526137 +0000 UTC m=+0.184717548 container create 378fbc3e520b56305d7a47e6fb8ccb765884355616d1081ea06094c09b46faa8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c9dd28080a631be25c77c36c63b17a3ef340ffb2e21c87d4d7b56b6398436d9, name=ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-3-deactivate, RELEASE=squid-de09dd0, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, ceph=True, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.36.0, CEPH_POINT_RELEASE=) 2024-08-19T01:44:50.466 INFO:journalctl@ceph.osd.5.smithi167.stdout:Aug 19 01:44:50 smithi167 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-5[33276]: 2024-08-19T01:44:50.161+0000 7f8191d31640 -1 osd.5 89 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-08-19T01:44:22.949456+0000 front 2024-08-19T01:44:22.949396+0000 (oldest deadline 2024-08-19T01:44:46.449550+0000) 2024-08-19T01:44:50.466 INFO:journalctl@ceph.osd.8.smithi167.stdout:Aug 19 01:44:50 smithi167 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-8[39928]: 2024-08-19T01:44:50.128+0000 7f17fe820640 -1 osd.8 89 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-08-19T01:44:22.747507+0000 front 2024-08-19T01:44:22.747516+0000 (oldest deadline 2024-08-19T01:44:48.047324+0000) 2024-08-19T01:44:50.611 INFO:journalctl@ceph.osd.9.smithi083.stdout:Aug 19 01:44:50 smithi083 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-9[51472]: 2024-08-19T01:44:50.560+0000 7f54e8c00640 -1 osd.9 89 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-08-19T01:44:20.873282+0000 front 2024-08-19T01:44:20.873259+0000 (oldest deadline 2024-08-19T01:44:46.773413+0000) 2024-08-19T01:44:50.612 INFO:journalctl@ceph.osd.3.smithi083.stdout:Aug 19 01:44:50 smithi083 podman[84677]: 2024-08-19 01:44:50.337660143 +0000 UTC m=+0.468851555 container init 378fbc3e520b56305d7a47e6fb8ccb765884355616d1081ea06094c09b46faa8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c9dd28080a631be25c77c36c63b17a3ef340ffb2e21c87d4d7b56b6398436d9, name=ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-3-deactivate, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, RELEASE=squid-de09dd0, GIT_CLEAN=True, io.buildah.version=1.36.0, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, ceph=True) 2024-08-19T01:44:50.612 INFO:journalctl@ceph.osd.3.smithi083.stdout:Aug 19 01:44:50 smithi083 podman[84677]: 2024-08-19 01:44:50.344924072 +0000 UTC m=+0.476115484 container start 378fbc3e520b56305d7a47e6fb8ccb765884355616d1081ea06094c09b46faa8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c9dd28080a631be25c77c36c63b17a3ef340ffb2e21c87d4d7b56b6398436d9, name=ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-3-deactivate, RELEASE=squid-de09dd0, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, io.buildah.version=1.36.0, ceph=True) 2024-08-19T01:44:50.612 INFO:journalctl@ceph.osd.3.smithi083.stdout:Aug 19 01:44:50 smithi083 podman[84677]: 2024-08-19 01:44:50.448038277 +0000 UTC m=+0.579229689 container attach 378fbc3e520b56305d7a47e6fb8ccb765884355616d1081ea06094c09b46faa8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c9dd28080a631be25c77c36c63b17a3ef340ffb2e21c87d4d7b56b6398436d9, name=ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-3-deactivate, CEPH_POINT_RELEASE=, RELEASE=squid-de09dd0, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, ceph=True, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, io.buildah.version=1.36.0, org.label-schema.license=GPLv2) 2024-08-19T01:44:50.904 INFO:journalctl@ceph.osd.6.smithi083.stdout:Aug 19 01:44:50 smithi083 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-6[44809]: 2024-08-19T01:44:50.609+0000 7f3cc6e00640 -1 osd.6 89 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-08-19T01:44:21.901735+0000 front 2024-08-19T01:44:21.901860+0000 (oldest deadline 2024-08-19T01:44:47.801737+0000) 2024-08-19T01:44:50.904 INFO:journalctl@ceph.osd.3.smithi083.stdout:Aug 19 01:44:50 smithi083 podman[84703]: 2024-08-19 01:44:50.665885156 +0000 UTC m=+0.029570651 container died 378fbc3e520b56305d7a47e6fb8ccb765884355616d1081ea06094c09b46faa8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c9dd28080a631be25c77c36c63b17a3ef340ffb2e21c87d4d7b56b6398436d9, name=ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-3-deactivate, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, RELEASE=squid-de09dd0, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.36.0, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-08-19T01:44:50.952 INFO:journalctl@ceph.osd.10.smithi117.stdout:Aug 19 01:44:50 smithi117 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-10[46616]: 2024-08-19T01:44:50.606+0000 7f34b3bf7640 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-08-19T01:44:24.060751+0000 front 2024-08-19T01:44:24.060716+0000 (oldest deadline 2024-08-19T01:44:47.560677+0000) 2024-08-19T01:44:50.965 INFO:journalctl@ceph.osd.11.smithi167.stdout:Aug 19 01:44:50 smithi167 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-11[46643]: 2024-08-19T01:44:50.617+0000 7f447f624640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-08-19T01:44:22.880210+0000 front 2024-08-19T01:44:22.880404+0000 (oldest deadline 2024-08-19T01:44:45.780352+0000) 2024-08-19T01:44:51.362 DEBUG:teuthology.orchestra.run.smithi083:> sudo pkill -f 'journalctl -f -n 0 -u ceph-bad8780e-5dc9-11ef-bccf-c7b262605968@osd.3.service' 2024-08-19T01:44:51.440 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-08-19T01:44:51.440 INFO:tasks.cephadm.osd.3:Stopped osd.3 2024-08-19T01:44:51.440 INFO:tasks.cephadm.osd.4:Stopping osd.4... 2024-08-19T01:44:51.440 DEBUG:teuthology.orchestra.run.smithi117:> sudo systemctl stop ceph-bad8780e-5dc9-11ef-bccf-c7b262605968@osd.4 2024-08-19T01:44:51.465 INFO:journalctl@ceph.osd.5.smithi167.stdout:Aug 19 01:44:51 smithi167 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-5[33276]: 2024-08-19T01:44:51.136+0000 7f8191d31640 -1 osd.5 89 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-08-19T01:44:22.949456+0000 front 2024-08-19T01:44:22.949396+0000 (oldest deadline 2024-08-19T01:44:46.449550+0000) 2024-08-19T01:44:51.466 INFO:journalctl@ceph.osd.8.smithi167.stdout:Aug 19 01:44:51 smithi167 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-8[39928]: 2024-08-19T01:44:51.092+0000 7f17fe820640 -1 osd.8 89 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-08-19T01:44:22.747507+0000 front 2024-08-19T01:44:22.747516+0000 (oldest deadline 2024-08-19T01:44:48.047324+0000) 2024-08-19T01:44:51.738 INFO:journalctl@ceph.osd.4.smithi117.stdout:Aug 19 01:44:51 smithi117 systemd[1]: Stopping Ceph osd.4 for bad8780e-5dc9-11ef-bccf-c7b262605968... 2024-08-19T01:44:51.739 INFO:journalctl@ceph.osd.10.smithi117.stdout:Aug 19 01:44:51 smithi117 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-10[46616]: 2024-08-19T01:44:51.593+0000 7f34b3bf7640 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-08-19T01:44:24.060751+0000 front 2024-08-19T01:44:24.060716+0000 (oldest deadline 2024-08-19T01:44:47.560677+0000) 2024-08-19T01:44:51.904 INFO:journalctl@ceph.osd.6.smithi083.stdout:Aug 19 01:44:51 smithi083 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-6[44809]: 2024-08-19T01:44:51.635+0000 7f3cc6e00640 -1 osd.6 89 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-08-19T01:44:21.901735+0000 front 2024-08-19T01:44:21.901860+0000 (oldest deadline 2024-08-19T01:44:47.801737+0000) 2024-08-19T01:44:51.904 INFO:journalctl@ceph.osd.9.smithi083.stdout:Aug 19 01:44:51 smithi083 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-9[51472]: 2024-08-19T01:44:51.593+0000 7f54e8c00640 -1 osd.9 89 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-08-19T01:44:20.873282+0000 front 2024-08-19T01:44:20.873259+0000 (oldest deadline 2024-08-19T01:44:46.773413+0000) 2024-08-19T01:44:51.966 INFO:journalctl@ceph.osd.11.smithi167.stdout:Aug 19 01:44:51 smithi167 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-11[46643]: 2024-08-19T01:44:51.653+0000 7f447f624640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-08-19T01:44:22.880210+0000 front 2024-08-19T01:44:22.880404+0000 (oldest deadline 2024-08-19T01:44:45.780352+0000) 2024-08-19T01:44:52.202 INFO:journalctl@ceph.osd.4.smithi117.stdout:Aug 19 01:44:51 smithi117 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-4[32888]: 2024-08-19T01:44:51.737+0000 7fa6fc5bd640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.4 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-08-19T01:44:52.202 INFO:journalctl@ceph.osd.4.smithi117.stdout:Aug 19 01:44:51 smithi117 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-4[32888]: 2024-08-19T01:44:51.737+0000 7fa6fc5bd640 -1 osd.4 89 *** Got signal Terminated *** 2024-08-19T01:44:52.202 INFO:journalctl@ceph.osd.4.smithi117.stdout:Aug 19 01:44:51 smithi117 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-4[32888]: 2024-08-19T01:44:51.737+0000 7fa6fc5bd640 -1 osd.4 89 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-08-19T01:44:52.466 INFO:journalctl@ceph.osd.5.smithi167.stdout:Aug 19 01:44:52 smithi167 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-5[33276]: 2024-08-19T01:44:52.101+0000 7f8191d31640 -1 osd.5 89 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-08-19T01:44:22.949456+0000 front 2024-08-19T01:44:22.949396+0000 (oldest deadline 2024-08-19T01:44:46.449550+0000) 2024-08-19T01:44:52.466 INFO:journalctl@ceph.osd.8.smithi167.stdout:Aug 19 01:44:52 smithi167 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-8[39928]: 2024-08-19T01:44:52.083+0000 7f17fe820640 -1 osd.8 89 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-08-19T01:44:22.747507+0000 front 2024-08-19T01:44:22.747516+0000 (oldest deadline 2024-08-19T01:44:48.047324+0000) 2024-08-19T01:44:52.904 INFO:journalctl@ceph.osd.6.smithi083.stdout:Aug 19 01:44:52 smithi083 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-6[44809]: 2024-08-19T01:44:52.626+0000 7f3cc6e00640 -1 osd.6 89 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-08-19T01:44:21.901735+0000 front 2024-08-19T01:44:21.901860+0000 (oldest deadline 2024-08-19T01:44:47.801737+0000) 2024-08-19T01:44:52.904 INFO:journalctl@ceph.osd.9.smithi083.stdout:Aug 19 01:44:52 smithi083 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-9[51472]: 2024-08-19T01:44:52.570+0000 7f54e8c00640 -1 osd.9 89 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-08-19T01:44:20.873282+0000 front 2024-08-19T01:44:20.873259+0000 (oldest deadline 2024-08-19T01:44:46.773413+0000) 2024-08-19T01:44:52.952 INFO:journalctl@ceph.osd.10.smithi117.stdout:Aug 19 01:44:52 smithi117 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-10[46616]: 2024-08-19T01:44:52.558+0000 7f34b3bf7640 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-08-19T01:44:24.060751+0000 front 2024-08-19T01:44:24.060716+0000 (oldest deadline 2024-08-19T01:44:47.560677+0000) 2024-08-19T01:44:52.966 INFO:journalctl@ceph.osd.11.smithi167.stdout:Aug 19 01:44:52 smithi167 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-11[46643]: 2024-08-19T01:44:52.669+0000 7f447f624640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-08-19T01:44:22.880210+0000 front 2024-08-19T01:44:22.880404+0000 (oldest deadline 2024-08-19T01:44:45.780352+0000) 2024-08-19T01:44:53.465 INFO:journalctl@ceph.osd.5.smithi167.stdout:Aug 19 01:44:53 smithi167 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-5[33276]: 2024-08-19T01:44:53.149+0000 7f8191d31640 -1 osd.5 89 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-08-19T01:44:22.949456+0000 front 2024-08-19T01:44:22.949396+0000 (oldest deadline 2024-08-19T01:44:46.449550+0000) 2024-08-19T01:44:53.466 INFO:journalctl@ceph.osd.8.smithi167.stdout:Aug 19 01:44:53 smithi167 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-8[39928]: 2024-08-19T01:44:53.074+0000 7f17fe820640 -1 osd.8 89 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-08-19T01:44:22.747507+0000 front 2024-08-19T01:44:22.747516+0000 (oldest deadline 2024-08-19T01:44:48.047324+0000) 2024-08-19T01:44:53.904 INFO:journalctl@ceph.osd.6.smithi083.stdout:Aug 19 01:44:53 smithi083 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-6[44809]: 2024-08-19T01:44:53.650+0000 7f3cc6e00640 -1 osd.6 89 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-08-19T01:44:21.901735+0000 front 2024-08-19T01:44:21.901860+0000 (oldest deadline 2024-08-19T01:44:47.801737+0000) 2024-08-19T01:44:53.904 INFO:journalctl@ceph.osd.9.smithi083.stdout:Aug 19 01:44:53 smithi083 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-9[51472]: 2024-08-19T01:44:53.537+0000 7f54e8c00640 -1 osd.9 89 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-08-19T01:44:20.873282+0000 front 2024-08-19T01:44:20.873259+0000 (oldest deadline 2024-08-19T01:44:46.773413+0000) 2024-08-19T01:44:53.952 INFO:journalctl@ceph.osd.10.smithi117.stdout:Aug 19 01:44:53 smithi117 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-10[46616]: 2024-08-19T01:44:53.529+0000 7f34b3bf7640 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-08-19T01:44:24.060751+0000 front 2024-08-19T01:44:24.060716+0000 (oldest deadline 2024-08-19T01:44:47.560677+0000) 2024-08-19T01:44:53.965 INFO:journalctl@ceph.osd.11.smithi167.stdout:Aug 19 01:44:53 smithi167 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-11[46643]: 2024-08-19T01:44:53.633+0000 7f447f624640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-08-19T01:44:22.880210+0000 front 2024-08-19T01:44:22.880404+0000 (oldest deadline 2024-08-19T01:44:45.780352+0000) 2024-08-19T01:44:54.452 INFO:journalctl@ceph.osd.7.smithi117.stdout:Aug 19 01:44:54 smithi117 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-7[39667]: 2024-08-19T01:44:54.091+0000 7f66eaeae640 -1 osd.7 89 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-08-19T01:44:31.724600+0000 front 2024-08-19T01:44:31.724641+0000 (oldest deadline 2024-08-19T01:44:54.024721+0000) 2024-08-19T01:44:54.466 INFO:journalctl@ceph.osd.8.smithi167.stdout:Aug 19 01:44:54 smithi167 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-8[39928]: 2024-08-19T01:44:54.122+0000 7f17fe820640 -1 osd.8 89 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-08-19T01:44:22.747507+0000 front 2024-08-19T01:44:22.747516+0000 (oldest deadline 2024-08-19T01:44:48.047324+0000) 2024-08-19T01:44:54.466 INFO:journalctl@ceph.osd.8.smithi167.stdout:Aug 19 01:44:54 smithi167 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-8[39928]: 2024-08-19T01:44:54.122+0000 7f17fe820640 -1 osd.8 89 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-08-19T01:44:32.049249+0000 front 2024-08-19T01:44:32.049153+0000 (oldest deadline 2024-08-19T01:44:53.749126+0000) 2024-08-19T01:44:54.466 INFO:journalctl@ceph.osd.5.smithi167.stdout:Aug 19 01:44:54 smithi167 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-5[33276]: 2024-08-19T01:44:54.121+0000 7f8191d31640 -1 osd.5 89 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-08-19T01:44:22.949456+0000 front 2024-08-19T01:44:22.949396+0000 (oldest deadline 2024-08-19T01:44:46.449550+0000) 2024-08-19T01:44:54.466 INFO:journalctl@ceph.osd.5.smithi167.stdout:Aug 19 01:44:54 smithi167 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-5[33276]: 2024-08-19T01:44:54.121+0000 7f8191d31640 -1 osd.5 89 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-08-19T01:44:31.051151+0000 front 2024-08-19T01:44:31.051292+0000 (oldest deadline 2024-08-19T01:44:53.351090+0000) 2024-08-19T01:44:54.904 INFO:journalctl@ceph.osd.6.smithi083.stdout:Aug 19 01:44:54 smithi083 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-6[44809]: 2024-08-19T01:44:54.631+0000 7f3cc6e00640 -1 osd.6 89 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-08-19T01:44:21.901735+0000 front 2024-08-19T01:44:21.901860+0000 (oldest deadline 2024-08-19T01:44:47.801737+0000) 2024-08-19T01:44:54.904 INFO:journalctl@ceph.osd.9.smithi083.stdout:Aug 19 01:44:54 smithi083 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-9[51472]: 2024-08-19T01:44:54.503+0000 7f54e8c00640 -1 osd.9 89 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-08-19T01:44:20.873282+0000 front 2024-08-19T01:44:20.873259+0000 (oldest deadline 2024-08-19T01:44:46.773413+0000) 2024-08-19T01:44:54.905 INFO:journalctl@ceph.osd.9.smithi083.stdout:Aug 19 01:44:54 smithi083 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-9[51472]: 2024-08-19T01:44:54.503+0000 7f54e8c00640 -1 osd.9 89 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-08-19T01:44:26.774104+0000 front 2024-08-19T01:44:26.774036+0000 (oldest deadline 2024-08-19T01:44:53.774823+0000) 2024-08-19T01:44:54.952 INFO:journalctl@ceph.osd.10.smithi117.stdout:Aug 19 01:44:54 smithi117 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-10[46616]: 2024-08-19T01:44:54.552+0000 7f34b3bf7640 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-08-19T01:44:24.060751+0000 front 2024-08-19T01:44:24.060716+0000 (oldest deadline 2024-08-19T01:44:47.560677+0000) 2024-08-19T01:44:54.966 INFO:journalctl@ceph.osd.11.smithi167.stdout:Aug 19 01:44:54 smithi167 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-11[46643]: 2024-08-19T01:44:54.676+0000 7f447f624640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-08-19T01:44:22.880210+0000 front 2024-08-19T01:44:22.880404+0000 (oldest deadline 2024-08-19T01:44:45.780352+0000) 2024-08-19T01:44:54.966 INFO:journalctl@ceph.osd.11.smithi167.stdout:Aug 19 01:44:54 smithi167 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-11[46643]: 2024-08-19T01:44:54.676+0000 7f447f624640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-08-19T01:44:31.081546+0000 front 2024-08-19T01:44:31.081654+0000 (oldest deadline 2024-08-19T01:44:53.981595+0000) 2024-08-19T01:44:55.452 INFO:journalctl@ceph.osd.7.smithi117.stdout:Aug 19 01:44:55 smithi117 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-7[39667]: 2024-08-19T01:44:55.060+0000 7f66eaeae640 -1 osd.7 89 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-08-19T01:44:31.724600+0000 front 2024-08-19T01:44:31.724641+0000 (oldest deadline 2024-08-19T01:44:54.024721+0000) 2024-08-19T01:44:55.466 INFO:journalctl@ceph.osd.8.smithi167.stdout:Aug 19 01:44:55 smithi167 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-8[39928]: 2024-08-19T01:44:55.077+0000 7f17fe820640 -1 osd.8 89 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-08-19T01:44:22.747507+0000 front 2024-08-19T01:44:22.747516+0000 (oldest deadline 2024-08-19T01:44:48.047324+0000) 2024-08-19T01:44:55.466 INFO:journalctl@ceph.osd.8.smithi167.stdout:Aug 19 01:44:55 smithi167 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-8[39928]: 2024-08-19T01:44:55.077+0000 7f17fe820640 -1 osd.8 89 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-08-19T01:44:32.049249+0000 front 2024-08-19T01:44:32.049153+0000 (oldest deadline 2024-08-19T01:44:53.749126+0000) 2024-08-19T01:44:55.466 INFO:journalctl@ceph.osd.5.smithi167.stdout:Aug 19 01:44:55 smithi167 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-5[33276]: 2024-08-19T01:44:55.123+0000 7f8191d31640 -1 osd.5 89 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-08-19T01:44:22.949456+0000 front 2024-08-19T01:44:22.949396+0000 (oldest deadline 2024-08-19T01:44:46.449550+0000) 2024-08-19T01:44:55.466 INFO:journalctl@ceph.osd.5.smithi167.stdout:Aug 19 01:44:55 smithi167 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-5[33276]: 2024-08-19T01:44:55.123+0000 7f8191d31640 -1 osd.5 89 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-08-19T01:44:31.051151+0000 front 2024-08-19T01:44:31.051292+0000 (oldest deadline 2024-08-19T01:44:53.351090+0000) 2024-08-19T01:44:55.904 INFO:journalctl@ceph.osd.6.smithi083.stdout:Aug 19 01:44:55 smithi083 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-6[44809]: 2024-08-19T01:44:55.681+0000 7f3cc6e00640 -1 osd.6 89 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-08-19T01:44:21.901735+0000 front 2024-08-19T01:44:21.901860+0000 (oldest deadline 2024-08-19T01:44:47.801737+0000) 2024-08-19T01:44:55.905 INFO:journalctl@ceph.osd.9.smithi083.stdout:Aug 19 01:44:55 smithi083 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-9[51472]: 2024-08-19T01:44:55.516+0000 7f54e8c00640 -1 osd.9 89 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-08-19T01:44:20.873282+0000 front 2024-08-19T01:44:20.873259+0000 (oldest deadline 2024-08-19T01:44:46.773413+0000) 2024-08-19T01:44:55.905 INFO:journalctl@ceph.osd.9.smithi083.stdout:Aug 19 01:44:55 smithi083 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-9[51472]: 2024-08-19T01:44:55.516+0000 7f54e8c00640 -1 osd.9 89 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-08-19T01:44:26.774104+0000 front 2024-08-19T01:44:26.774036+0000 (oldest deadline 2024-08-19T01:44:53.774823+0000) 2024-08-19T01:44:55.952 INFO:journalctl@ceph.osd.10.smithi117.stdout:Aug 19 01:44:55 smithi117 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-10[46616]: 2024-08-19T01:44:55.507+0000 7f34b3bf7640 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-08-19T01:44:24.060751+0000 front 2024-08-19T01:44:24.060716+0000 (oldest deadline 2024-08-19T01:44:47.560677+0000) 2024-08-19T01:44:55.965 INFO:journalctl@ceph.osd.11.smithi167.stdout:Aug 19 01:44:55 smithi167 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-11[46643]: 2024-08-19T01:44:55.648+0000 7f447f624640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-08-19T01:44:22.880210+0000 front 2024-08-19T01:44:22.880404+0000 (oldest deadline 2024-08-19T01:44:45.780352+0000) 2024-08-19T01:44:55.966 INFO:journalctl@ceph.osd.11.smithi167.stdout:Aug 19 01:44:55 smithi167 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-11[46643]: 2024-08-19T01:44:55.648+0000 7f447f624640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-08-19T01:44:31.081546+0000 front 2024-08-19T01:44:31.081654+0000 (oldest deadline 2024-08-19T01:44:53.981595+0000) 2024-08-19T01:44:56.452 INFO:journalctl@ceph.osd.7.smithi117.stdout:Aug 19 01:44:56 smithi117 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-7[39667]: 2024-08-19T01:44:56.038+0000 7f66eaeae640 -1 osd.7 89 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-08-19T01:44:31.724600+0000 front 2024-08-19T01:44:31.724641+0000 (oldest deadline 2024-08-19T01:44:54.024721+0000) 2024-08-19T01:44:56.466 INFO:journalctl@ceph.osd.8.smithi167.stdout:Aug 19 01:44:56 smithi167 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-8[39928]: 2024-08-19T01:44:56.118+0000 7f17fe820640 -1 osd.8 89 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-08-19T01:44:22.747507+0000 front 2024-08-19T01:44:22.747516+0000 (oldest deadline 2024-08-19T01:44:48.047324+0000) 2024-08-19T01:44:56.466 INFO:journalctl@ceph.osd.8.smithi167.stdout:Aug 19 01:44:56 smithi167 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-8[39928]: 2024-08-19T01:44:56.118+0000 7f17fe820640 -1 osd.8 89 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-08-19T01:44:32.049249+0000 front 2024-08-19T01:44:32.049153+0000 (oldest deadline 2024-08-19T01:44:53.749126+0000) 2024-08-19T01:44:56.466 INFO:journalctl@ceph.osd.5.smithi167.stdout:Aug 19 01:44:56 smithi167 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-5[33276]: 2024-08-19T01:44:56.139+0000 7f8191d31640 -1 osd.5 89 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-08-19T01:44:22.949456+0000 front 2024-08-19T01:44:22.949396+0000 (oldest deadline 2024-08-19T01:44:46.449550+0000) 2024-08-19T01:44:56.466 INFO:journalctl@ceph.osd.5.smithi167.stdout:Aug 19 01:44:56 smithi167 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-5[33276]: 2024-08-19T01:44:56.139+0000 7f8191d31640 -1 osd.5 89 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-08-19T01:44:31.051151+0000 front 2024-08-19T01:44:31.051292+0000 (oldest deadline 2024-08-19T01:44:53.351090+0000) 2024-08-19T01:44:56.904 INFO:journalctl@ceph.osd.6.smithi083.stdout:Aug 19 01:44:56 smithi083 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-6[44809]: 2024-08-19T01:44:56.709+0000 7f3cc6e00640 -1 osd.6 89 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-08-19T01:44:21.901735+0000 front 2024-08-19T01:44:21.901860+0000 (oldest deadline 2024-08-19T01:44:47.801737+0000) 2024-08-19T01:44:56.904 INFO:journalctl@ceph.osd.6.smithi083.stdout:Aug 19 01:44:56 smithi083 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-6[44809]: 2024-08-19T01:44:56.709+0000 7f3cc6e00640 -1 osd.6 89 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-08-19T01:44:28.303418+0000 front 2024-08-19T01:44:28.303477+0000 (oldest deadline 2024-08-19T01:44:55.904426+0000) 2024-08-19T01:44:56.905 INFO:journalctl@ceph.osd.9.smithi083.stdout:Aug 19 01:44:56 smithi083 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-9[51472]: 2024-08-19T01:44:56.486+0000 7f54e8c00640 -1 osd.9 89 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-08-19T01:44:20.873282+0000 front 2024-08-19T01:44:20.873259+0000 (oldest deadline 2024-08-19T01:44:46.773413+0000) 2024-08-19T01:44:56.905 INFO:journalctl@ceph.osd.9.smithi083.stdout:Aug 19 01:44:56 smithi083 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-9[51472]: 2024-08-19T01:44:56.486+0000 7f54e8c00640 -1 osd.9 89 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-08-19T01:44:26.774104+0000 front 2024-08-19T01:44:26.774036+0000 (oldest deadline 2024-08-19T01:44:53.774823+0000) 2024-08-19T01:44:56.952 INFO:journalctl@ceph.osd.10.smithi117.stdout:Aug 19 01:44:56 smithi117 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-10[46616]: 2024-08-19T01:44:56.479+0000 7f34b3bf7640 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-08-19T01:44:24.060751+0000 front 2024-08-19T01:44:24.060716+0000 (oldest deadline 2024-08-19T01:44:47.560677+0000) 2024-08-19T01:44:56.952 INFO:journalctl@ceph.osd.10.smithi117.stdout:Aug 19 01:44:56 smithi117 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-10[46616]: 2024-08-19T01:44:56.479+0000 7f34b3bf7640 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-08-19T01:44:28.661503+0000 front 2024-08-19T01:44:28.661522+0000 (oldest deadline 2024-08-19T01:44:56.262298+0000) 2024-08-19T01:44:56.966 INFO:journalctl@ceph.osd.11.smithi167.stdout:Aug 19 01:44:56 smithi167 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-11[46643]: 2024-08-19T01:44:56.693+0000 7f447f624640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-08-19T01:44:22.880210+0000 front 2024-08-19T01:44:22.880404+0000 (oldest deadline 2024-08-19T01:44:45.780352+0000) 2024-08-19T01:44:56.966 INFO:journalctl@ceph.osd.11.smithi167.stdout:Aug 19 01:44:56 smithi167 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-11[46643]: 2024-08-19T01:44:56.693+0000 7f447f624640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-08-19T01:44:31.081546+0000 front 2024-08-19T01:44:31.081654+0000 (oldest deadline 2024-08-19T01:44:53.981595+0000) 2024-08-19T01:44:57.372 INFO:journalctl@ceph.osd.7.smithi117.stdout:Aug 19 01:44:57 smithi117 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-7[39667]: 2024-08-19T01:44:57.004+0000 7f66eaeae640 -1 osd.7 89 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-08-19T01:44:31.724600+0000 front 2024-08-19T01:44:31.724641+0000 (oldest deadline 2024-08-19T01:44:54.024721+0000) 2024-08-19T01:44:57.372 INFO:journalctl@ceph.osd.4.smithi117.stdout:Aug 19 01:44:57 smithi117 podman[56000]: 2024-08-19 01:44:57.127567143 +0000 UTC m=+5.489811080 container died b3cffe3e90969b5399f4dec5790c0a8591308d98c4c87199c110894716aec9ab (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c9dd28080a631be25c77c36c63b17a3ef340ffb2e21c87d4d7b56b6398436d9, name=ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-4, GIT_BRANCH=HEAD, io.buildah.version=1.36.0, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, RELEASE=squid-de09dd0, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-08-19T01:44:57.466 INFO:journalctl@ceph.osd.8.smithi167.stdout:Aug 19 01:44:57 smithi167 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-8[39928]: 2024-08-19T01:44:57.088+0000 7f17fe820640 -1 osd.8 89 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-08-19T01:44:22.747507+0000 front 2024-08-19T01:44:22.747516+0000 (oldest deadline 2024-08-19T01:44:48.047324+0000) 2024-08-19T01:44:57.466 INFO:journalctl@ceph.osd.8.smithi167.stdout:Aug 19 01:44:57 smithi167 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-8[39928]: 2024-08-19T01:44:57.088+0000 7f17fe820640 -1 osd.8 89 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-08-19T01:44:32.049249+0000 front 2024-08-19T01:44:32.049153+0000 (oldest deadline 2024-08-19T01:44:53.749126+0000) 2024-08-19T01:44:57.466 INFO:journalctl@ceph.osd.5.smithi167.stdout:Aug 19 01:44:57 smithi167 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-5[33276]: 2024-08-19T01:44:57.149+0000 7f8191d31640 -1 osd.5 89 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-08-19T01:44:22.949456+0000 front 2024-08-19T01:44:22.949396+0000 (oldest deadline 2024-08-19T01:44:46.449550+0000) 2024-08-19T01:44:57.466 INFO:journalctl@ceph.osd.5.smithi167.stdout:Aug 19 01:44:57 smithi167 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-5[33276]: 2024-08-19T01:44:57.149+0000 7f8191d31640 -1 osd.5 89 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-08-19T01:44:31.051151+0000 front 2024-08-19T01:44:31.051292+0000 (oldest deadline 2024-08-19T01:44:53.351090+0000) 2024-08-19T01:44:57.656 INFO:journalctl@ceph.osd.4.smithi117.stdout:Aug 19 01:44:57 smithi117 podman[56000]: 2024-08-19 01:44:57.470467383 +0000 UTC m=+5.832711318 container cleanup b3cffe3e90969b5399f4dec5790c0a8591308d98c4c87199c110894716aec9ab (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c9dd28080a631be25c77c36c63b17a3ef340ffb2e21c87d4d7b56b6398436d9, name=ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-4, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , io.buildah.version=1.36.0, GIT_CLEAN=True, RELEASE=squid-de09dd0, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, ceph=True) 2024-08-19T01:44:57.656 INFO:journalctl@ceph.osd.4.smithi117.stdout:Aug 19 01:44:57 smithi117 bash[56000]: ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-4 2024-08-19T01:44:57.656 INFO:journalctl@ceph.osd.10.smithi117.stdout:Aug 19 01:44:57 smithi117 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-10[46616]: 2024-08-19T01:44:57.522+0000 7f34b3bf7640 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-08-19T01:44:24.060751+0000 front 2024-08-19T01:44:24.060716+0000 (oldest deadline 2024-08-19T01:44:47.560677+0000) 2024-08-19T01:44:57.656 INFO:journalctl@ceph.osd.10.smithi117.stdout:Aug 19 01:44:57 smithi117 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-10[46616]: 2024-08-19T01:44:57.522+0000 7f34b3bf7640 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-08-19T01:44:28.661503+0000 front 2024-08-19T01:44:28.661522+0000 (oldest deadline 2024-08-19T01:44:56.262298+0000) 2024-08-19T01:44:57.754 INFO:journalctl@ceph.osd.9.smithi083.stdout:Aug 19 01:44:57 smithi083 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-9[51472]: 2024-08-19T01:44:57.478+0000 7f54e8c00640 -1 osd.9 89 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-08-19T01:44:20.873282+0000 front 2024-08-19T01:44:20.873259+0000 (oldest deadline 2024-08-19T01:44:46.773413+0000) 2024-08-19T01:44:57.755 INFO:journalctl@ceph.osd.9.smithi083.stdout:Aug 19 01:44:57 smithi083 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-9[51472]: 2024-08-19T01:44:57.478+0000 7f54e8c00640 -1 osd.9 89 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-08-19T01:44:26.774104+0000 front 2024-08-19T01:44:26.774036+0000 (oldest deadline 2024-08-19T01:44:53.774823+0000) 2024-08-19T01:44:57.952 INFO:journalctl@ceph.osd.4.smithi117.stdout:Aug 19 01:44:57 smithi117 podman[56015]: 2024-08-19 01:44:57.655202887 +0000 UTC m=+0.525539374 container remove b3cffe3e90969b5399f4dec5790c0a8591308d98c4c87199c110894716aec9ab (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c9dd28080a631be25c77c36c63b17a3ef340ffb2e21c87d4d7b56b6398436d9, name=ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-4, maintainer=Guillaume Abrioux , io.buildah.version=1.36.0, org.label-schema.license=GPLv2, ceph=True, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, RELEASE=squid-de09dd0, org.label-schema.build-date=20240731) 2024-08-19T01:44:58.060 INFO:journalctl@ceph.osd.11.smithi167.stdout:Aug 19 01:44:57 smithi167 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-11[46643]: 2024-08-19T01:44:57.743+0000 7f447f624640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-08-19T01:44:22.880210+0000 front 2024-08-19T01:44:22.880404+0000 (oldest deadline 2024-08-19T01:44:45.780352+0000) 2024-08-19T01:44:58.060 INFO:journalctl@ceph.osd.11.smithi167.stdout:Aug 19 01:44:57 smithi167 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-11[46643]: 2024-08-19T01:44:57.743+0000 7f447f624640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-08-19T01:44:31.081546+0000 front 2024-08-19T01:44:31.081654+0000 (oldest deadline 2024-08-19T01:44:53.981595+0000) 2024-08-19T01:44:58.154 INFO:journalctl@ceph.osd.6.smithi083.stdout:Aug 19 01:44:57 smithi083 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-6[44809]: 2024-08-19T01:44:57.753+0000 7f3cc6e00640 -1 osd.6 89 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-08-19T01:44:21.901735+0000 front 2024-08-19T01:44:21.901860+0000 (oldest deadline 2024-08-19T01:44:47.801737+0000) 2024-08-19T01:44:58.155 INFO:journalctl@ceph.osd.6.smithi083.stdout:Aug 19 01:44:57 smithi083 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-6[44809]: 2024-08-19T01:44:57.753+0000 7f3cc6e00640 -1 osd.6 89 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-08-19T01:44:28.303418+0000 front 2024-08-19T01:44:28.303477+0000 (oldest deadline 2024-08-19T01:44:55.904426+0000) 2024-08-19T01:44:58.334 INFO:journalctl@ceph.osd.7.smithi117.stdout:Aug 19 01:44:58 smithi117 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-7[39667]: 2024-08-19T01:44:58.042+0000 7f66eaeae640 -1 osd.7 89 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-08-19T01:44:31.724600+0000 front 2024-08-19T01:44:31.724641+0000 (oldest deadline 2024-08-19T01:44:54.024721+0000) 2024-08-19T01:44:58.334 INFO:journalctl@ceph.osd.4.smithi117.stdout:Aug 19 01:44:58 smithi117 podman[56106]: 2024-08-19 01:44:58.097607097 +0000 UTC m=+0.109945719 container create 06191c68c42d5bd8c787be0bca60459ce34e2725bd2096efcb9b01d553314df0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c9dd28080a631be25c77c36c63b17a3ef340ffb2e21c87d4d7b56b6398436d9, name=ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-4-deactivate, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_CLEAN=True, io.buildah.version=1.36.0, RELEASE=squid-de09dd0, org.label-schema.license=GPLv2, ceph=True) 2024-08-19T01:44:58.334 INFO:journalctl@ceph.osd.4.smithi117.stdout:Aug 19 01:44:58 smithi117 podman[56106]: 2024-08-19 01:44:58.004963396 +0000 UTC m=+0.017301986 image pull ea7a5189f30bad3aca9524788e2965d2368d7ce786df09d4edcd8c3edf9cefba quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c9dd28080a631be25c77c36c63b17a3ef340ffb2e21c87d4d7b56b6398436d9 2024-08-19T01:44:58.335 INFO:journalctl@ceph.osd.4.smithi117.stdout:Aug 19 01:44:58 smithi117 podman[56106]: 2024-08-19 01:44:58.333289691 +0000 UTC m=+0.345628268 container init 06191c68c42d5bd8c787be0bca60459ce34e2725bd2096efcb9b01d553314df0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c9dd28080a631be25c77c36c63b17a3ef340ffb2e21c87d4d7b56b6398436d9, name=ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-4-deactivate, GIT_CLEAN=True, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , ceph=True, CEPH_POINT_RELEASE=, RELEASE=squid-de09dd0, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, io.buildah.version=1.36.0, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-08-19T01:44:58.466 INFO:journalctl@ceph.osd.8.smithi167.stdout:Aug 19 01:44:58 smithi167 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-8[39928]: 2024-08-19T01:44:58.058+0000 7f17fe820640 -1 osd.8 89 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-08-19T01:44:22.747507+0000 front 2024-08-19T01:44:22.747516+0000 (oldest deadline 2024-08-19T01:44:48.047324+0000) 2024-08-19T01:44:58.466 INFO:journalctl@ceph.osd.8.smithi167.stdout:Aug 19 01:44:58 smithi167 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-8[39928]: 2024-08-19T01:44:58.058+0000 7f17fe820640 -1 osd.8 89 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-08-19T01:44:32.049249+0000 front 2024-08-19T01:44:32.049153+0000 (oldest deadline 2024-08-19T01:44:53.749126+0000) 2024-08-19T01:44:58.466 INFO:journalctl@ceph.osd.5.smithi167.stdout:Aug 19 01:44:58 smithi167 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-5[33276]: 2024-08-19T01:44:58.142+0000 7f8191d31640 -1 osd.5 89 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-08-19T01:44:22.949456+0000 front 2024-08-19T01:44:22.949396+0000 (oldest deadline 2024-08-19T01:44:46.449550+0000) 2024-08-19T01:44:58.466 INFO:journalctl@ceph.osd.5.smithi167.stdout:Aug 19 01:44:58 smithi167 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-5[33276]: 2024-08-19T01:44:58.142+0000 7f8191d31640 -1 osd.5 89 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-08-19T01:44:31.051151+0000 front 2024-08-19T01:44:31.051292+0000 (oldest deadline 2024-08-19T01:44:53.351090+0000) 2024-08-19T01:44:58.702 INFO:journalctl@ceph.osd.10.smithi117.stdout:Aug 19 01:44:58 smithi117 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-10[46616]: 2024-08-19T01:44:58.566+0000 7f34b3bf7640 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-08-19T01:44:24.060751+0000 front 2024-08-19T01:44:24.060716+0000 (oldest deadline 2024-08-19T01:44:47.560677+0000) 2024-08-19T01:44:58.703 INFO:journalctl@ceph.osd.10.smithi117.stdout:Aug 19 01:44:58 smithi117 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-10[46616]: 2024-08-19T01:44:58.566+0000 7f34b3bf7640 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-08-19T01:44:28.661503+0000 front 2024-08-19T01:44:28.661522+0000 (oldest deadline 2024-08-19T01:44:56.262298+0000) 2024-08-19T01:44:58.703 INFO:journalctl@ceph.osd.4.smithi117.stdout:Aug 19 01:44:58 smithi117 podman[56106]: 2024-08-19 01:44:58.337074534 +0000 UTC m=+0.349413112 container start 06191c68c42d5bd8c787be0bca60459ce34e2725bd2096efcb9b01d553314df0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c9dd28080a631be25c77c36c63b17a3ef340ffb2e21c87d4d7b56b6398436d9, name=ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-4-deactivate, ceph=True, GIT_CLEAN=True, RELEASE=squid-de09dd0, io.buildah.version=1.36.0, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-08-19T01:44:58.703 INFO:journalctl@ceph.osd.4.smithi117.stdout:Aug 19 01:44:58 smithi117 podman[56106]: 2024-08-19 01:44:58.416619488 +0000 UTC m=+0.428958068 container attach 06191c68c42d5bd8c787be0bca60459ce34e2725bd2096efcb9b01d553314df0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c9dd28080a631be25c77c36c63b17a3ef340ffb2e21c87d4d7b56b6398436d9, name=ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-4-deactivate, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, ceph=True, GIT_CLEAN=True, CEPH_POINT_RELEASE=, io.buildah.version=1.36.0, RELEASE=squid-de09dd0, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image) 2024-08-19T01:44:58.703 INFO:journalctl@ceph.osd.4.smithi117.stdout:Aug 19 01:44:58 smithi117 podman[56106]: 2024-08-19 01:44:58.522455099 +0000 UTC m=+0.534793687 container died 06191c68c42d5bd8c787be0bca60459ce34e2725bd2096efcb9b01d553314df0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c9dd28080a631be25c77c36c63b17a3ef340ffb2e21c87d4d7b56b6398436d9, name=ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-4-deactivate, org.label-schema.build-date=20240731, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, RELEASE=squid-de09dd0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, io.buildah.version=1.36.0, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux ) 2024-08-19T01:44:58.782 INFO:journalctl@ceph.osd.9.smithi083.stdout:Aug 19 01:44:58 smithi083 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-9[51472]: 2024-08-19T01:44:58.514+0000 7f54e8c00640 -1 osd.9 89 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-08-19T01:44:20.873282+0000 front 2024-08-19T01:44:20.873259+0000 (oldest deadline 2024-08-19T01:44:46.773413+0000) 2024-08-19T01:44:58.783 INFO:journalctl@ceph.osd.9.smithi083.stdout:Aug 19 01:44:58 smithi083 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-9[51472]: 2024-08-19T01:44:58.514+0000 7f54e8c00640 -1 osd.9 89 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-08-19T01:44:26.774104+0000 front 2024-08-19T01:44:26.774036+0000 (oldest deadline 2024-08-19T01:44:53.774823+0000) 2024-08-19T01:44:59.084 INFO:journalctl@ceph.osd.11.smithi167.stdout:Aug 19 01:44:58 smithi167 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-11[46643]: 2024-08-19T01:44:58.789+0000 7f447f624640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-08-19T01:44:22.880210+0000 front 2024-08-19T01:44:22.880404+0000 (oldest deadline 2024-08-19T01:44:45.780352+0000) 2024-08-19T01:44:59.084 INFO:journalctl@ceph.osd.11.smithi167.stdout:Aug 19 01:44:58 smithi167 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-11[46643]: 2024-08-19T01:44:58.789+0000 7f447f624640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-08-19T01:44:31.081546+0000 front 2024-08-19T01:44:31.081654+0000 (oldest deadline 2024-08-19T01:44:53.981595+0000) 2024-08-19T01:44:59.100 DEBUG:teuthology.orchestra.run.smithi117:> sudo pkill -f 'journalctl -f -n 0 -u ceph-bad8780e-5dc9-11ef-bccf-c7b262605968@osd.4.service' 2024-08-19T01:44:59.123 INFO:journalctl@ceph.osd.7.smithi117.stdout:Aug 19 01:44:59 smithi117 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-7[39667]: 2024-08-19T01:44:59.033+0000 7f66eaeae640 -1 osd.7 89 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-08-19T01:44:31.724600+0000 front 2024-08-19T01:44:31.724641+0000 (oldest deadline 2024-08-19T01:44:54.024721+0000) 2024-08-19T01:44:59.123 INFO:journalctl@ceph.osd.4.smithi117.stdout:Aug 19 01:44:59 smithi117 podman[56129]: 2024-08-19 01:44:59.078367708 +0000 UTC m=+0.550007938 container remove 06191c68c42d5bd8c787be0bca60459ce34e2725bd2096efcb9b01d553314df0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c9dd28080a631be25c77c36c63b17a3ef340ffb2e21c87d4d7b56b6398436d9, name=ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-4-deactivate, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, io.buildah.version=1.36.0, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, RELEASE=squid-de09dd0, maintainer=Guillaume Abrioux ) 2024-08-19T01:44:59.123 INFO:journalctl@ceph.osd.4.smithi117.stdout:Aug 19 01:44:59 smithi117 systemd[1]: ceph-bad8780e-5dc9-11ef-bccf-c7b262605968@osd.4.service: Deactivated successfully. 2024-08-19T01:44:59.123 INFO:journalctl@ceph.osd.4.smithi117.stdout:Aug 19 01:44:59 smithi117 systemd[1]: Stopped Ceph osd.4 for bad8780e-5dc9-11ef-bccf-c7b262605968. 2024-08-19T01:44:59.123 INFO:journalctl@ceph.osd.4.smithi117.stdout:Aug 19 01:44:59 smithi117 systemd[1]: ceph-bad8780e-5dc9-11ef-bccf-c7b262605968@osd.4.service: Consumed 1min 25.200s CPU time. 2024-08-19T01:44:59.137 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-08-19T01:44:59.138 INFO:tasks.cephadm.osd.4:Stopped osd.4 2024-08-19T01:44:59.138 INFO:tasks.cephadm.osd.5:Stopping osd.5... 2024-08-19T01:44:59.138 DEBUG:teuthology.orchestra.run.smithi167:> sudo systemctl stop ceph-bad8780e-5dc9-11ef-bccf-c7b262605968@osd.5 2024-08-19T01:44:59.154 INFO:journalctl@ceph.osd.6.smithi083.stdout:Aug 19 01:44:58 smithi083 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-6[44809]: 2024-08-19T01:44:58.781+0000 7f3cc6e00640 -1 osd.6 89 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-08-19T01:44:21.901735+0000 front 2024-08-19T01:44:21.901860+0000 (oldest deadline 2024-08-19T01:44:47.801737+0000) 2024-08-19T01:44:59.154 INFO:journalctl@ceph.osd.6.smithi083.stdout:Aug 19 01:44:58 smithi083 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-6[44809]: 2024-08-19T01:44:58.781+0000 7f3cc6e00640 -1 osd.6 89 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-08-19T01:44:28.303418+0000 front 2024-08-19T01:44:28.303477+0000 (oldest deadline 2024-08-19T01:44:55.904426+0000) 2024-08-19T01:44:59.439 INFO:journalctl@ceph.osd.8.smithi167.stdout:Aug 19 01:44:59 smithi167 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-8[39928]: 2024-08-19T01:44:59.083+0000 7f17fe820640 -1 osd.8 89 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-08-19T01:44:22.747507+0000 front 2024-08-19T01:44:22.747516+0000 (oldest deadline 2024-08-19T01:44:48.047324+0000) 2024-08-19T01:44:59.439 INFO:journalctl@ceph.osd.8.smithi167.stdout:Aug 19 01:44:59 smithi167 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-8[39928]: 2024-08-19T01:44:59.083+0000 7f17fe820640 -1 osd.8 89 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-08-19T01:44:32.049249+0000 front 2024-08-19T01:44:32.049153+0000 (oldest deadline 2024-08-19T01:44:53.749126+0000) 2024-08-19T01:44:59.439 INFO:journalctl@ceph.osd.5.smithi167.stdout:Aug 19 01:44:59 smithi167 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-5[33276]: 2024-08-19T01:44:59.183+0000 7f8191d31640 -1 osd.5 89 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-08-19T01:44:22.949456+0000 front 2024-08-19T01:44:22.949396+0000 (oldest deadline 2024-08-19T01:44:46.449550+0000) 2024-08-19T01:44:59.439 INFO:journalctl@ceph.osd.5.smithi167.stdout:Aug 19 01:44:59 smithi167 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-5[33276]: 2024-08-19T01:44:59.183+0000 7f8191d31640 -1 osd.5 89 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-08-19T01:44:31.051151+0000 front 2024-08-19T01:44:31.051292+0000 (oldest deadline 2024-08-19T01:44:53.351090+0000) 2024-08-19T01:44:59.439 INFO:journalctl@ceph.osd.5.smithi167.stdout:Aug 19 01:44:59 smithi167 systemd[1]: Stopping Ceph osd.5 for bad8780e-5dc9-11ef-bccf-c7b262605968... 2024-08-19T01:44:59.439 INFO:journalctl@ceph.osd.5.smithi167.stdout:Aug 19 01:44:59 smithi167 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-5[33276]: 2024-08-19T01:44:59.437+0000 7f8194f2b640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.5 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-08-19T01:44:59.440 INFO:journalctl@ceph.osd.5.smithi167.stdout:Aug 19 01:44:59 smithi167 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-5[33276]: 2024-08-19T01:44:59.437+0000 7f8194f2b640 -1 osd.5 89 *** Got signal Terminated *** 2024-08-19T01:44:59.716 INFO:journalctl@ceph.osd.5.smithi167.stdout:Aug 19 01:44:59 smithi167 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-5[33276]: 2024-08-19T01:44:59.437+0000 7f8194f2b640 -1 osd.5 89 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-08-19T01:44:59.831 INFO:journalctl@ceph.osd.9.smithi083.stdout:Aug 19 01:44:59 smithi083 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-9[51472]: 2024-08-19T01:44:59.540+0000 7f54e8c00640 -1 osd.9 89 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-08-19T01:44:20.873282+0000 front 2024-08-19T01:44:20.873259+0000 (oldest deadline 2024-08-19T01:44:46.773413+0000) 2024-08-19T01:44:59.832 INFO:journalctl@ceph.osd.9.smithi083.stdout:Aug 19 01:44:59 smithi083 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-9[51472]: 2024-08-19T01:44:59.540+0000 7f54e8c00640 -1 osd.9 89 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-08-19T01:44:26.774104+0000 front 2024-08-19T01:44:26.774036+0000 (oldest deadline 2024-08-19T01:44:53.774823+0000) 2024-08-19T01:44:59.952 INFO:journalctl@ceph.osd.10.smithi117.stdout:Aug 19 01:44:59 smithi117 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-10[46616]: 2024-08-19T01:44:59.610+0000 7f34b3bf7640 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-08-19T01:44:24.060751+0000 front 2024-08-19T01:44:24.060716+0000 (oldest deadline 2024-08-19T01:44:47.560677+0000) 2024-08-19T01:44:59.952 INFO:journalctl@ceph.osd.10.smithi117.stdout:Aug 19 01:44:59 smithi117 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-10[46616]: 2024-08-19T01:44:59.610+0000 7f34b3bf7640 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-08-19T01:44:28.661503+0000 front 2024-08-19T01:44:28.661522+0000 (oldest deadline 2024-08-19T01:44:56.262298+0000) 2024-08-19T01:45:00.082 INFO:journalctl@ceph.osd.11.smithi167.stdout:Aug 19 01:44:59 smithi167 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-11[46643]: 2024-08-19T01:44:59.750+0000 7f447f624640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-08-19T01:44:22.880210+0000 front 2024-08-19T01:44:22.880404+0000 (oldest deadline 2024-08-19T01:44:45.780352+0000) 2024-08-19T01:45:00.082 INFO:journalctl@ceph.osd.11.smithi167.stdout:Aug 19 01:44:59 smithi167 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-11[46643]: 2024-08-19T01:44:59.750+0000 7f447f624640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-08-19T01:44:31.081546+0000 front 2024-08-19T01:44:31.081654+0000 (oldest deadline 2024-08-19T01:44:53.981595+0000) 2024-08-19T01:45:00.154 INFO:journalctl@ceph.osd.6.smithi083.stdout:Aug 19 01:44:59 smithi083 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-6[44809]: 2024-08-19T01:44:59.830+0000 7f3cc6e00640 -1 osd.6 89 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-08-19T01:44:21.901735+0000 front 2024-08-19T01:44:21.901860+0000 (oldest deadline 2024-08-19T01:44:47.801737+0000) 2024-08-19T01:45:00.154 INFO:journalctl@ceph.osd.6.smithi083.stdout:Aug 19 01:44:59 smithi083 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-6[44809]: 2024-08-19T01:44:59.830+0000 7f3cc6e00640 -1 osd.6 89 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-08-19T01:44:28.303418+0000 front 2024-08-19T01:44:28.303477+0000 (oldest deadline 2024-08-19T01:44:55.904426+0000) 2024-08-19T01:45:00.452 INFO:journalctl@ceph.osd.7.smithi117.stdout:Aug 19 01:45:00 smithi117 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-7[39667]: 2024-08-19T01:45:00.043+0000 7f66eaeae640 -1 osd.7 89 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-08-19T01:44:31.724600+0000 front 2024-08-19T01:44:31.724641+0000 (oldest deadline 2024-08-19T01:44:54.024721+0000) 2024-08-19T01:45:00.465 INFO:journalctl@ceph.osd.5.smithi167.stdout:Aug 19 01:45:00 smithi167 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-5[33276]: 2024-08-19T01:45:00.208+0000 7f8191d31640 -1 osd.5 89 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-08-19T01:44:22.949456+0000 front 2024-08-19T01:44:22.949396+0000 (oldest deadline 2024-08-19T01:44:46.449550+0000) 2024-08-19T01:45:00.465 INFO:journalctl@ceph.osd.5.smithi167.stdout:Aug 19 01:45:00 smithi167 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-5[33276]: 2024-08-19T01:45:00.208+0000 7f8191d31640 -1 osd.5 89 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-08-19T01:44:31.051151+0000 front 2024-08-19T01:44:31.051292+0000 (oldest deadline 2024-08-19T01:44:53.351090+0000) 2024-08-19T01:45:00.466 INFO:journalctl@ceph.osd.8.smithi167.stdout:Aug 19 01:45:00 smithi167 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-8[39928]: 2024-08-19T01:45:00.080+0000 7f17fe820640 -1 osd.8 89 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-08-19T01:44:22.747507+0000 front 2024-08-19T01:44:22.747516+0000 (oldest deadline 2024-08-19T01:44:48.047324+0000) 2024-08-19T01:45:00.466 INFO:journalctl@ceph.osd.8.smithi167.stdout:Aug 19 01:45:00 smithi167 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-8[39928]: 2024-08-19T01:45:00.080+0000 7f17fe820640 -1 osd.8 89 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-08-19T01:44:32.049249+0000 front 2024-08-19T01:44:32.049153+0000 (oldest deadline 2024-08-19T01:44:53.749126+0000) 2024-08-19T01:45:00.877 INFO:journalctl@ceph.osd.9.smithi083.stdout:Aug 19 01:45:00 smithi083 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-9[51472]: 2024-08-19T01:45:00.492+0000 7f54e8c00640 -1 osd.9 89 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-08-19T01:44:20.873282+0000 front 2024-08-19T01:44:20.873259+0000 (oldest deadline 2024-08-19T01:44:46.773413+0000) 2024-08-19T01:45:00.877 INFO:journalctl@ceph.osd.9.smithi083.stdout:Aug 19 01:45:00 smithi083 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-9[51472]: 2024-08-19T01:45:00.492+0000 7f54e8c00640 -1 osd.9 89 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-08-19T01:44:26.774104+0000 front 2024-08-19T01:44:26.774036+0000 (oldest deadline 2024-08-19T01:44:53.774823+0000) 2024-08-19T01:45:00.952 INFO:journalctl@ceph.osd.10.smithi117.stdout:Aug 19 01:45:00 smithi117 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-10[46616]: 2024-08-19T01:45:00.603+0000 7f34b3bf7640 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-08-19T01:44:24.060751+0000 front 2024-08-19T01:44:24.060716+0000 (oldest deadline 2024-08-19T01:44:47.560677+0000) 2024-08-19T01:45:00.952 INFO:journalctl@ceph.osd.10.smithi117.stdout:Aug 19 01:45:00 smithi117 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-10[46616]: 2024-08-19T01:45:00.603+0000 7f34b3bf7640 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-08-19T01:44:28.661503+0000 front 2024-08-19T01:44:28.661522+0000 (oldest deadline 2024-08-19T01:44:56.262298+0000) 2024-08-19T01:45:00.966 INFO:journalctl@ceph.osd.11.smithi167.stdout:Aug 19 01:45:00 smithi167 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-11[46643]: 2024-08-19T01:45:00.706+0000 7f447f624640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-08-19T01:44:22.880210+0000 front 2024-08-19T01:44:22.880404+0000 (oldest deadline 2024-08-19T01:44:45.780352+0000) 2024-08-19T01:45:00.966 INFO:journalctl@ceph.osd.11.smithi167.stdout:Aug 19 01:45:00 smithi167 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-11[46643]: 2024-08-19T01:45:00.706+0000 7f447f624640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-08-19T01:44:31.081546+0000 front 2024-08-19T01:44:31.081654+0000 (oldest deadline 2024-08-19T01:44:53.981595+0000) 2024-08-19T01:45:01.154 INFO:journalctl@ceph.osd.6.smithi083.stdout:Aug 19 01:45:00 smithi083 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-6[44809]: 2024-08-19T01:45:00.876+0000 7f3cc6e00640 -1 osd.6 89 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-08-19T01:44:21.901735+0000 front 2024-08-19T01:44:21.901860+0000 (oldest deadline 2024-08-19T01:44:47.801737+0000) 2024-08-19T01:45:01.154 INFO:journalctl@ceph.osd.6.smithi083.stdout:Aug 19 01:45:00 smithi083 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-6[44809]: 2024-08-19T01:45:00.876+0000 7f3cc6e00640 -1 osd.6 89 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-08-19T01:44:28.303418+0000 front 2024-08-19T01:44:28.303477+0000 (oldest deadline 2024-08-19T01:44:55.904426+0000) 2024-08-19T01:45:01.452 INFO:journalctl@ceph.osd.7.smithi117.stdout:Aug 19 01:45:01 smithi117 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-7[39667]: 2024-08-19T01:45:01.029+0000 7f66eaeae640 -1 osd.7 89 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-08-19T01:44:31.724600+0000 front 2024-08-19T01:44:31.724641+0000 (oldest deadline 2024-08-19T01:44:54.024721+0000) 2024-08-19T01:45:01.465 INFO:journalctl@ceph.osd.5.smithi167.stdout:Aug 19 01:45:01 smithi167 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-5[33276]: 2024-08-19T01:45:01.212+0000 7f8191d31640 -1 osd.5 89 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-08-19T01:44:22.949456+0000 front 2024-08-19T01:44:22.949396+0000 (oldest deadline 2024-08-19T01:44:46.449550+0000) 2024-08-19T01:45:01.466 INFO:journalctl@ceph.osd.5.smithi167.stdout:Aug 19 01:45:01 smithi167 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-5[33276]: 2024-08-19T01:45:01.212+0000 7f8191d31640 -1 osd.5 89 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-08-19T01:44:31.051151+0000 front 2024-08-19T01:44:31.051292+0000 (oldest deadline 2024-08-19T01:44:53.351090+0000) 2024-08-19T01:45:01.466 INFO:journalctl@ceph.osd.8.smithi167.stdout:Aug 19 01:45:01 smithi167 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-8[39928]: 2024-08-19T01:45:01.065+0000 7f17fe820640 -1 osd.8 89 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-08-19T01:44:22.747507+0000 front 2024-08-19T01:44:22.747516+0000 (oldest deadline 2024-08-19T01:44:48.047324+0000) 2024-08-19T01:45:01.466 INFO:journalctl@ceph.osd.8.smithi167.stdout:Aug 19 01:45:01 smithi167 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-8[39928]: 2024-08-19T01:45:01.065+0000 7f17fe820640 -1 osd.8 89 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-08-19T01:44:32.049249+0000 front 2024-08-19T01:44:32.049153+0000 (oldest deadline 2024-08-19T01:44:53.749126+0000) 2024-08-19T01:45:01.850 INFO:journalctl@ceph.osd.9.smithi083.stdout:Aug 19 01:45:01 smithi083 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-9[51472]: 2024-08-19T01:45:01.477+0000 7f54e8c00640 -1 osd.9 89 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-08-19T01:44:20.873282+0000 front 2024-08-19T01:44:20.873259+0000 (oldest deadline 2024-08-19T01:44:46.773413+0000) 2024-08-19T01:45:01.851 INFO:journalctl@ceph.osd.9.smithi083.stdout:Aug 19 01:45:01 smithi083 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-9[51472]: 2024-08-19T01:45:01.477+0000 7f54e8c00640 -1 osd.9 89 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-08-19T01:44:26.774104+0000 front 2024-08-19T01:44:26.774036+0000 (oldest deadline 2024-08-19T01:44:53.774823+0000) 2024-08-19T01:45:01.952 INFO:journalctl@ceph.osd.10.smithi117.stdout:Aug 19 01:45:01 smithi117 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-10[46616]: 2024-08-19T01:45:01.643+0000 7f34b3bf7640 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-08-19T01:44:24.060751+0000 front 2024-08-19T01:44:24.060716+0000 (oldest deadline 2024-08-19T01:44:47.560677+0000) 2024-08-19T01:45:01.952 INFO:journalctl@ceph.osd.10.smithi117.stdout:Aug 19 01:45:01 smithi117 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-10[46616]: 2024-08-19T01:45:01.643+0000 7f34b3bf7640 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-08-19T01:44:28.661503+0000 front 2024-08-19T01:44:28.661522+0000 (oldest deadline 2024-08-19T01:44:56.262298+0000) 2024-08-19T01:45:02.076 INFO:journalctl@ceph.osd.11.smithi167.stdout:Aug 19 01:45:01 smithi167 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-11[46643]: 2024-08-19T01:45:01.752+0000 7f447f624640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-08-19T01:44:22.880210+0000 front 2024-08-19T01:44:22.880404+0000 (oldest deadline 2024-08-19T01:44:45.780352+0000) 2024-08-19T01:45:02.077 INFO:journalctl@ceph.osd.11.smithi167.stdout:Aug 19 01:45:01 smithi167 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-11[46643]: 2024-08-19T01:45:01.752+0000 7f447f624640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-08-19T01:44:31.081546+0000 front 2024-08-19T01:44:31.081654+0000 (oldest deadline 2024-08-19T01:44:53.981595+0000) 2024-08-19T01:45:02.154 INFO:journalctl@ceph.osd.6.smithi083.stdout:Aug 19 01:45:01 smithi083 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-6[44809]: 2024-08-19T01:45:01.849+0000 7f3cc6e00640 -1 osd.6 89 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-08-19T01:44:21.901735+0000 front 2024-08-19T01:44:21.901860+0000 (oldest deadline 2024-08-19T01:44:47.801737+0000) 2024-08-19T01:45:02.154 INFO:journalctl@ceph.osd.6.smithi083.stdout:Aug 19 01:45:01 smithi083 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-6[44809]: 2024-08-19T01:45:01.849+0000 7f3cc6e00640 -1 osd.6 89 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-08-19T01:44:28.303418+0000 front 2024-08-19T01:44:28.303477+0000 (oldest deadline 2024-08-19T01:44:55.904426+0000) 2024-08-19T01:45:02.154 INFO:journalctl@ceph.osd.6.smithi083.stdout:Aug 19 01:45:01 smithi083 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-6[44809]: 2024-08-19T01:45:01.849+0000 7f3cc6e00640 -1 osd.6 89 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-08-19T01:44:37.506923+0000 front 2024-08-19T01:44:37.507030+0000 (oldest deadline 2024-08-19T01:45:01.606759+0000) 2024-08-19T01:45:02.452 INFO:journalctl@ceph.osd.7.smithi117.stdout:Aug 19 01:45:02 smithi117 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-7[39667]: 2024-08-19T01:45:02.078+0000 7f66eaeae640 -1 osd.7 89 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-08-19T01:44:31.724600+0000 front 2024-08-19T01:44:31.724641+0000 (oldest deadline 2024-08-19T01:44:54.024721+0000) 2024-08-19T01:45:02.452 INFO:journalctl@ceph.osd.7.smithi117.stdout:Aug 19 01:45:02 smithi117 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-7[39667]: 2024-08-19T01:45:02.078+0000 7f66eaeae640 -1 osd.7 89 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-08-19T01:44:40.426018+0000 front 2024-08-19T01:44:40.426002+0000 (oldest deadline 2024-08-19T01:45:01.526013+0000) 2024-08-19T01:45:02.466 INFO:journalctl@ceph.osd.5.smithi167.stdout:Aug 19 01:45:02 smithi167 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-5[33276]: 2024-08-19T01:45:02.238+0000 7f8191d31640 -1 osd.5 89 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-08-19T01:44:22.949456+0000 front 2024-08-19T01:44:22.949396+0000 (oldest deadline 2024-08-19T01:44:46.449550+0000) 2024-08-19T01:45:02.466 INFO:journalctl@ceph.osd.5.smithi167.stdout:Aug 19 01:45:02 smithi167 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-5[33276]: 2024-08-19T01:45:02.238+0000 7f8191d31640 -1 osd.5 89 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-08-19T01:44:31.051151+0000 front 2024-08-19T01:44:31.051292+0000 (oldest deadline 2024-08-19T01:44:53.351090+0000) 2024-08-19T01:45:02.466 INFO:journalctl@ceph.osd.8.smithi167.stdout:Aug 19 01:45:02 smithi167 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-8[39928]: 2024-08-19T01:45:02.075+0000 7f17fe820640 -1 osd.8 89 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-08-19T01:44:22.747507+0000 front 2024-08-19T01:44:22.747516+0000 (oldest deadline 2024-08-19T01:44:48.047324+0000) 2024-08-19T01:45:02.466 INFO:journalctl@ceph.osd.8.smithi167.stdout:Aug 19 01:45:02 smithi167 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-8[39928]: 2024-08-19T01:45:02.075+0000 7f17fe820640 -1 osd.8 89 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-08-19T01:44:32.049249+0000 front 2024-08-19T01:44:32.049153+0000 (oldest deadline 2024-08-19T01:44:53.749126+0000) 2024-08-19T01:45:02.893 INFO:journalctl@ceph.osd.9.smithi083.stdout:Aug 19 01:45:02 smithi083 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-9[51472]: 2024-08-19T01:45:02.484+0000 7f54e8c00640 -1 osd.9 89 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-08-19T01:44:20.873282+0000 front 2024-08-19T01:44:20.873259+0000 (oldest deadline 2024-08-19T01:44:46.773413+0000) 2024-08-19T01:45:02.893 INFO:journalctl@ceph.osd.9.smithi083.stdout:Aug 19 01:45:02 smithi083 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-9[51472]: 2024-08-19T01:45:02.484+0000 7f54e8c00640 -1 osd.9 89 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-08-19T01:44:26.774104+0000 front 2024-08-19T01:44:26.774036+0000 (oldest deadline 2024-08-19T01:44:53.774823+0000) 2024-08-19T01:45:02.952 INFO:journalctl@ceph.osd.10.smithi117.stdout:Aug 19 01:45:02 smithi117 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-10[46616]: 2024-08-19T01:45:02.626+0000 7f34b3bf7640 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-08-19T01:44:24.060751+0000 front 2024-08-19T01:44:24.060716+0000 (oldest deadline 2024-08-19T01:44:47.560677+0000) 2024-08-19T01:45:02.952 INFO:journalctl@ceph.osd.10.smithi117.stdout:Aug 19 01:45:02 smithi117 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-10[46616]: 2024-08-19T01:45:02.626+0000 7f34b3bf7640 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-08-19T01:44:28.661503+0000 front 2024-08-19T01:44:28.661522+0000 (oldest deadline 2024-08-19T01:44:56.262298+0000) 2024-08-19T01:45:02.952 INFO:journalctl@ceph.osd.10.smithi117.stdout:Aug 19 01:45:02 smithi117 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-10[46616]: 2024-08-19T01:45:02.626+0000 7f34b3bf7640 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-08-19T01:44:38.563674+0000 front 2024-08-19T01:44:38.563520+0000 (oldest deadline 2024-08-19T01:45:02.063517+0000) 2024-08-19T01:45:03.053 INFO:journalctl@ceph.osd.11.smithi167.stdout:Aug 19 01:45:02 smithi167 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-11[46643]: 2024-08-19T01:45:02.786+0000 7f447f624640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-08-19T01:44:22.880210+0000 front 2024-08-19T01:44:22.880404+0000 (oldest deadline 2024-08-19T01:44:45.780352+0000) 2024-08-19T01:45:03.054 INFO:journalctl@ceph.osd.11.smithi167.stdout:Aug 19 01:45:02 smithi167 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-11[46643]: 2024-08-19T01:45:02.786+0000 7f447f624640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-08-19T01:44:31.081546+0000 front 2024-08-19T01:44:31.081654+0000 (oldest deadline 2024-08-19T01:44:53.981595+0000) 2024-08-19T01:45:03.154 INFO:journalctl@ceph.osd.6.smithi083.stdout:Aug 19 01:45:02 smithi083 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-6[44809]: 2024-08-19T01:45:02.892+0000 7f3cc6e00640 -1 osd.6 89 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-08-19T01:44:21.901735+0000 front 2024-08-19T01:44:21.901860+0000 (oldest deadline 2024-08-19T01:44:47.801737+0000) 2024-08-19T01:45:03.154 INFO:journalctl@ceph.osd.6.smithi083.stdout:Aug 19 01:45:02 smithi083 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-6[44809]: 2024-08-19T01:45:02.892+0000 7f3cc6e00640 -1 osd.6 89 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-08-19T01:44:28.303418+0000 front 2024-08-19T01:44:28.303477+0000 (oldest deadline 2024-08-19T01:44:55.904426+0000) 2024-08-19T01:45:03.154 INFO:journalctl@ceph.osd.6.smithi083.stdout:Aug 19 01:45:02 smithi083 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-6[44809]: 2024-08-19T01:45:02.892+0000 7f3cc6e00640 -1 osd.6 89 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-08-19T01:44:37.506923+0000 front 2024-08-19T01:44:37.507030+0000 (oldest deadline 2024-08-19T01:45:01.606759+0000) 2024-08-19T01:45:03.452 INFO:journalctl@ceph.osd.7.smithi117.stdout:Aug 19 01:45:03 smithi117 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-7[39667]: 2024-08-19T01:45:03.127+0000 7f66eaeae640 -1 osd.7 89 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-08-19T01:44:31.724600+0000 front 2024-08-19T01:44:31.724641+0000 (oldest deadline 2024-08-19T01:44:54.024721+0000) 2024-08-19T01:45:03.452 INFO:journalctl@ceph.osd.7.smithi117.stdout:Aug 19 01:45:03 smithi117 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-7[39667]: 2024-08-19T01:45:03.127+0000 7f66eaeae640 -1 osd.7 89 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-08-19T01:44:40.426018+0000 front 2024-08-19T01:44:40.426002+0000 (oldest deadline 2024-08-19T01:45:01.526013+0000) 2024-08-19T01:45:03.465 INFO:journalctl@ceph.osd.8.smithi167.stdout:Aug 19 01:45:03 smithi167 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-8[39928]: 2024-08-19T01:45:03.051+0000 7f17fe820640 -1 osd.8 89 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-08-19T01:44:22.747507+0000 front 2024-08-19T01:44:22.747516+0000 (oldest deadline 2024-08-19T01:44:48.047324+0000) 2024-08-19T01:45:03.466 INFO:journalctl@ceph.osd.8.smithi167.stdout:Aug 19 01:45:03 smithi167 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-8[39928]: 2024-08-19T01:45:03.051+0000 7f17fe820640 -1 osd.8 89 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-08-19T01:44:32.049249+0000 front 2024-08-19T01:44:32.049153+0000 (oldest deadline 2024-08-19T01:44:53.749126+0000) 2024-08-19T01:45:03.466 INFO:journalctl@ceph.osd.5.smithi167.stdout:Aug 19 01:45:03 smithi167 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-5[33276]: 2024-08-19T01:45:03.245+0000 7f8191d31640 -1 osd.5 89 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-08-19T01:44:22.949456+0000 front 2024-08-19T01:44:22.949396+0000 (oldest deadline 2024-08-19T01:44:46.449550+0000) 2024-08-19T01:45:03.466 INFO:journalctl@ceph.osd.5.smithi167.stdout:Aug 19 01:45:03 smithi167 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-5[33276]: 2024-08-19T01:45:03.245+0000 7f8191d31640 -1 osd.5 89 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-08-19T01:44:31.051151+0000 front 2024-08-19T01:44:31.051292+0000 (oldest deadline 2024-08-19T01:44:53.351090+0000) 2024-08-19T01:45:03.882 INFO:journalctl@ceph.osd.9.smithi083.stdout:Aug 19 01:45:03 smithi083 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-9[51472]: 2024-08-19T01:45:03.443+0000 7f54e8c00640 -1 osd.9 89 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-08-19T01:44:20.873282+0000 front 2024-08-19T01:44:20.873259+0000 (oldest deadline 2024-08-19T01:44:46.773413+0000) 2024-08-19T01:45:03.882 INFO:journalctl@ceph.osd.9.smithi083.stdout:Aug 19 01:45:03 smithi083 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-9[51472]: 2024-08-19T01:45:03.443+0000 7f54e8c00640 -1 osd.9 89 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-08-19T01:44:26.774104+0000 front 2024-08-19T01:44:26.774036+0000 (oldest deadline 2024-08-19T01:44:53.774823+0000) 2024-08-19T01:45:03.952 INFO:journalctl@ceph.osd.10.smithi117.stdout:Aug 19 01:45:03 smithi117 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-10[46616]: 2024-08-19T01:45:03.599+0000 7f34b3bf7640 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-08-19T01:44:24.060751+0000 front 2024-08-19T01:44:24.060716+0000 (oldest deadline 2024-08-19T01:44:47.560677+0000) 2024-08-19T01:45:03.953 INFO:journalctl@ceph.osd.10.smithi117.stdout:Aug 19 01:45:03 smithi117 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-10[46616]: 2024-08-19T01:45:03.599+0000 7f34b3bf7640 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-08-19T01:44:28.661503+0000 front 2024-08-19T01:44:28.661522+0000 (oldest deadline 2024-08-19T01:44:56.262298+0000) 2024-08-19T01:45:03.953 INFO:journalctl@ceph.osd.10.smithi117.stdout:Aug 19 01:45:03 smithi117 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-10[46616]: 2024-08-19T01:45:03.599+0000 7f34b3bf7640 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-08-19T01:44:38.563674+0000 front 2024-08-19T01:44:38.563520+0000 (oldest deadline 2024-08-19T01:45:02.063517+0000) 2024-08-19T01:45:04.082 INFO:journalctl@ceph.osd.11.smithi167.stdout:Aug 19 01:45:03 smithi167 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-11[46643]: 2024-08-19T01:45:03.756+0000 7f447f624640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-08-19T01:44:22.880210+0000 front 2024-08-19T01:44:22.880404+0000 (oldest deadline 2024-08-19T01:44:45.780352+0000) 2024-08-19T01:45:04.082 INFO:journalctl@ceph.osd.11.smithi167.stdout:Aug 19 01:45:03 smithi167 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-11[46643]: 2024-08-19T01:45:03.756+0000 7f447f624640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-08-19T01:44:31.081546+0000 front 2024-08-19T01:44:31.081654+0000 (oldest deadline 2024-08-19T01:44:53.981595+0000) 2024-08-19T01:45:04.154 INFO:journalctl@ceph.osd.6.smithi083.stdout:Aug 19 01:45:03 smithi083 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-6[44809]: 2024-08-19T01:45:03.880+0000 7f3cc6e00640 -1 osd.6 89 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-08-19T01:44:21.901735+0000 front 2024-08-19T01:44:21.901860+0000 (oldest deadline 2024-08-19T01:44:47.801737+0000) 2024-08-19T01:45:04.154 INFO:journalctl@ceph.osd.6.smithi083.stdout:Aug 19 01:45:03 smithi083 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-6[44809]: 2024-08-19T01:45:03.880+0000 7f3cc6e00640 -1 osd.6 89 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-08-19T01:44:28.303418+0000 front 2024-08-19T01:44:28.303477+0000 (oldest deadline 2024-08-19T01:44:55.904426+0000) 2024-08-19T01:45:04.154 INFO:journalctl@ceph.osd.6.smithi083.stdout:Aug 19 01:45:03 smithi083 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-6[44809]: 2024-08-19T01:45:03.880+0000 7f3cc6e00640 -1 osd.6 89 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-08-19T01:44:37.506923+0000 front 2024-08-19T01:44:37.507030+0000 (oldest deadline 2024-08-19T01:45:01.606759+0000) 2024-08-19T01:45:04.452 INFO:journalctl@ceph.osd.7.smithi117.stdout:Aug 19 01:45:04 smithi117 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-7[39667]: 2024-08-19T01:45:04.100+0000 7f66eaeae640 -1 osd.7 89 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-08-19T01:44:31.724600+0000 front 2024-08-19T01:44:31.724641+0000 (oldest deadline 2024-08-19T01:44:54.024721+0000) 2024-08-19T01:45:04.452 INFO:journalctl@ceph.osd.7.smithi117.stdout:Aug 19 01:45:04 smithi117 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-7[39667]: 2024-08-19T01:45:04.100+0000 7f66eaeae640 -1 osd.7 89 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-08-19T01:44:40.426018+0000 front 2024-08-19T01:44:40.426002+0000 (oldest deadline 2024-08-19T01:45:01.526013+0000) 2024-08-19T01:45:04.466 INFO:journalctl@ceph.osd.5.smithi167.stdout:Aug 19 01:45:04 smithi167 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-5[33276]: 2024-08-19T01:45:04.284+0000 7f8191d31640 -1 osd.5 89 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-08-19T01:44:22.949456+0000 front 2024-08-19T01:44:22.949396+0000 (oldest deadline 2024-08-19T01:44:46.449550+0000) 2024-08-19T01:45:04.466 INFO:journalctl@ceph.osd.5.smithi167.stdout:Aug 19 01:45:04 smithi167 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-5[33276]: 2024-08-19T01:45:04.284+0000 7f8191d31640 -1 osd.5 89 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-08-19T01:44:31.051151+0000 front 2024-08-19T01:44:31.051292+0000 (oldest deadline 2024-08-19T01:44:53.351090+0000) 2024-08-19T01:45:04.466 INFO:journalctl@ceph.osd.8.smithi167.stdout:Aug 19 01:45:04 smithi167 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-8[39928]: 2024-08-19T01:45:04.080+0000 7f17fe820640 -1 osd.8 89 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-08-19T01:44:22.747507+0000 front 2024-08-19T01:44:22.747516+0000 (oldest deadline 2024-08-19T01:44:48.047324+0000) 2024-08-19T01:45:04.466 INFO:journalctl@ceph.osd.8.smithi167.stdout:Aug 19 01:45:04 smithi167 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-8[39928]: 2024-08-19T01:45:04.080+0000 7f17fe820640 -1 osd.8 89 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-08-19T01:44:32.049249+0000 front 2024-08-19T01:44:32.049153+0000 (oldest deadline 2024-08-19T01:44:53.749126+0000) 2024-08-19T01:45:04.843 INFO:journalctl@ceph.osd.9.smithi083.stdout:Aug 19 01:45:04 smithi083 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-9[51472]: 2024-08-19T01:45:04.429+0000 7f54e8c00640 -1 osd.9 89 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-08-19T01:44:20.873282+0000 front 2024-08-19T01:44:20.873259+0000 (oldest deadline 2024-08-19T01:44:46.773413+0000) 2024-08-19T01:45:04.843 INFO:journalctl@ceph.osd.9.smithi083.stdout:Aug 19 01:45:04 smithi083 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-9[51472]: 2024-08-19T01:45:04.429+0000 7f54e8c00640 -1 osd.9 89 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-08-19T01:44:26.774104+0000 front 2024-08-19T01:44:26.774036+0000 (oldest deadline 2024-08-19T01:44:53.774823+0000) 2024-08-19T01:45:04.843 INFO:journalctl@ceph.osd.9.smithi083.stdout:Aug 19 01:45:04 smithi083 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-9[51472]: 2024-08-19T01:45:04.429+0000 7f54e8c00640 -1 osd.9 89 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-08-19T01:44:39.676617+0000 front 2024-08-19T01:44:39.676511+0000 (oldest deadline 2024-08-19T01:45:03.776684+0000) 2024-08-19T01:45:04.952 INFO:journalctl@ceph.osd.10.smithi117.stdout:Aug 19 01:45:04 smithi117 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-10[46616]: 2024-08-19T01:45:04.557+0000 7f34b3bf7640 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-08-19T01:44:24.060751+0000 front 2024-08-19T01:44:24.060716+0000 (oldest deadline 2024-08-19T01:44:47.560677+0000) 2024-08-19T01:45:04.952 INFO:journalctl@ceph.osd.10.smithi117.stdout:Aug 19 01:45:04 smithi117 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-10[46616]: 2024-08-19T01:45:04.557+0000 7f34b3bf7640 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-08-19T01:44:28.661503+0000 front 2024-08-19T01:44:28.661522+0000 (oldest deadline 2024-08-19T01:44:56.262298+0000) 2024-08-19T01:45:04.952 INFO:journalctl@ceph.osd.10.smithi117.stdout:Aug 19 01:45:04 smithi117 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-10[46616]: 2024-08-19T01:45:04.557+0000 7f34b3bf7640 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-08-19T01:44:38.563674+0000 front 2024-08-19T01:44:38.563520+0000 (oldest deadline 2024-08-19T01:45:02.063517+0000) 2024-08-19T01:45:05.060 INFO:journalctl@ceph.osd.5.smithi167.stdout:Aug 19 01:45:04 smithi167 podman[56789]: 2024-08-19 01:45:04.874734271 +0000 UTC m=+5.534890404 container died 75eb620dc5c589bd13f6487d4ad7ff09f80293fb5f3999c810e63ae3bf3f4e53 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c9dd28080a631be25c77c36c63b17a3ef340ffb2e21c87d4d7b56b6398436d9, name=ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-5, io.buildah.version=1.36.0, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, GIT_CLEAN=True, RELEASE=squid-de09dd0, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, ceph=True) 2024-08-19T01:45:05.061 INFO:journalctl@ceph.osd.8.smithi167.stdout:Aug 19 01:45:05 smithi167 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-8[39928]: 2024-08-19T01:45:05.058+0000 7f17fe820640 -1 osd.8 89 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-08-19T01:44:22.747507+0000 front 2024-08-19T01:44:22.747516+0000 (oldest deadline 2024-08-19T01:44:48.047324+0000) 2024-08-19T01:45:05.061 INFO:journalctl@ceph.osd.8.smithi167.stdout:Aug 19 01:45:05 smithi167 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-8[39928]: 2024-08-19T01:45:05.058+0000 7f17fe820640 -1 osd.8 89 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-08-19T01:44:32.049249+0000 front 2024-08-19T01:44:32.049153+0000 (oldest deadline 2024-08-19T01:44:53.749126+0000) 2024-08-19T01:45:05.061 INFO:journalctl@ceph.osd.11.smithi167.stdout:Aug 19 01:45:04 smithi167 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-11[46643]: 2024-08-19T01:45:04.736+0000 7f447f624640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-08-19T01:44:22.880210+0000 front 2024-08-19T01:44:22.880404+0000 (oldest deadline 2024-08-19T01:44:45.780352+0000) 2024-08-19T01:45:05.061 INFO:journalctl@ceph.osd.11.smithi167.stdout:Aug 19 01:45:04 smithi167 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-11[46643]: 2024-08-19T01:45:04.736+0000 7f447f624640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-08-19T01:44:31.081546+0000 front 2024-08-19T01:44:31.081654+0000 (oldest deadline 2024-08-19T01:44:53.981595+0000) 2024-08-19T01:45:05.061 INFO:journalctl@ceph.osd.11.smithi167.stdout:Aug 19 01:45:04 smithi167 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-11[46643]: 2024-08-19T01:45:04.736+0000 7f447f624640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-08-19T01:44:39.282638+0000 front 2024-08-19T01:44:39.282723+0000 (oldest deadline 2024-08-19T01:45:04.582709+0000) 2024-08-19T01:45:05.154 INFO:journalctl@ceph.osd.6.smithi083.stdout:Aug 19 01:45:04 smithi083 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-6[44809]: 2024-08-19T01:45:04.841+0000 7f3cc6e00640 -1 osd.6 89 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-08-19T01:44:21.901735+0000 front 2024-08-19T01:44:21.901860+0000 (oldest deadline 2024-08-19T01:44:47.801737+0000) 2024-08-19T01:45:05.154 INFO:journalctl@ceph.osd.6.smithi083.stdout:Aug 19 01:45:04 smithi083 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-6[44809]: 2024-08-19T01:45:04.841+0000 7f3cc6e00640 -1 osd.6 89 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-08-19T01:44:28.303418+0000 front 2024-08-19T01:44:28.303477+0000 (oldest deadline 2024-08-19T01:44:55.904426+0000) 2024-08-19T01:45:05.155 INFO:journalctl@ceph.osd.6.smithi083.stdout:Aug 19 01:45:04 smithi083 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-6[44809]: 2024-08-19T01:45:04.841+0000 7f3cc6e00640 -1 osd.6 89 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-08-19T01:44:37.506923+0000 front 2024-08-19T01:44:37.507030+0000 (oldest deadline 2024-08-19T01:45:01.606759+0000) 2024-08-19T01:45:05.432 INFO:journalctl@ceph.osd.5.smithi167.stdout:Aug 19 01:45:05 smithi167 podman[56789]: 2024-08-19 01:45:05.188221501 +0000 UTC m=+5.848377633 container cleanup 75eb620dc5c589bd13f6487d4ad7ff09f80293fb5f3999c810e63ae3bf3f4e53 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c9dd28080a631be25c77c36c63b17a3ef340ffb2e21c87d4d7b56b6398436d9, name=ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-5, CEPH_POINT_RELEASE=, ceph=True, GIT_CLEAN=True, GIT_BRANCH=HEAD, io.buildah.version=1.36.0, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=squid-de09dd0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image) 2024-08-19T01:45:05.432 INFO:journalctl@ceph.osd.5.smithi167.stdout:Aug 19 01:45:05 smithi167 bash[56789]: ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-5 2024-08-19T01:45:05.452 INFO:journalctl@ceph.osd.7.smithi117.stdout:Aug 19 01:45:05 smithi117 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-7[39667]: 2024-08-19T01:45:05.122+0000 7f66eaeae640 -1 osd.7 89 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-08-19T01:44:31.724600+0000 front 2024-08-19T01:44:31.724641+0000 (oldest deadline 2024-08-19T01:44:54.024721+0000) 2024-08-19T01:45:05.452 INFO:journalctl@ceph.osd.7.smithi117.stdout:Aug 19 01:45:05 smithi117 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-7[39667]: 2024-08-19T01:45:05.122+0000 7f66eaeae640 -1 osd.7 89 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-08-19T01:44:40.426018+0000 front 2024-08-19T01:44:40.426002+0000 (oldest deadline 2024-08-19T01:45:01.526013+0000) 2024-08-19T01:45:05.654 INFO:journalctl@ceph.osd.9.smithi083.stdout:Aug 19 01:45:05 smithi083 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-9[51472]: 2024-08-19T01:45:05.398+0000 7f54e8c00640 -1 osd.9 89 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-08-19T01:44:20.873282+0000 front 2024-08-19T01:44:20.873259+0000 (oldest deadline 2024-08-19T01:44:46.773413+0000) 2024-08-19T01:45:05.654 INFO:journalctl@ceph.osd.9.smithi083.stdout:Aug 19 01:45:05 smithi083 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-9[51472]: 2024-08-19T01:45:05.398+0000 7f54e8c00640 -1 osd.9 89 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-08-19T01:44:26.774104+0000 front 2024-08-19T01:44:26.774036+0000 (oldest deadline 2024-08-19T01:44:53.774823+0000) 2024-08-19T01:45:05.654 INFO:journalctl@ceph.osd.9.smithi083.stdout:Aug 19 01:45:05 smithi083 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-9[51472]: 2024-08-19T01:45:05.398+0000 7f54e8c00640 -1 osd.9 89 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-08-19T01:44:39.676617+0000 front 2024-08-19T01:44:39.676511+0000 (oldest deadline 2024-08-19T01:45:03.776684+0000) 2024-08-19T01:45:05.716 INFO:journalctl@ceph.osd.5.smithi167.stdout:Aug 19 01:45:05 smithi167 podman[56804]: 2024-08-19 01:45:05.431202958 +0000 UTC m=+0.551874754 container remove 75eb620dc5c589bd13f6487d4ad7ff09f80293fb5f3999c810e63ae3bf3f4e53 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c9dd28080a631be25c77c36c63b17a3ef340ffb2e21c87d4d7b56b6398436d9, name=ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, RELEASE=squid-de09dd0, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, io.buildah.version=1.36.0, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0) 2024-08-19T01:45:05.952 INFO:journalctl@ceph.osd.10.smithi117.stdout:Aug 19 01:45:05 smithi117 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-10[46616]: 2024-08-19T01:45:05.534+0000 7f34b3bf7640 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-08-19T01:44:24.060751+0000 front 2024-08-19T01:44:24.060716+0000 (oldest deadline 2024-08-19T01:44:47.560677+0000) 2024-08-19T01:45:05.952 INFO:journalctl@ceph.osd.10.smithi117.stdout:Aug 19 01:45:05 smithi117 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-10[46616]: 2024-08-19T01:45:05.534+0000 7f34b3bf7640 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-08-19T01:44:28.661503+0000 front 2024-08-19T01:44:28.661522+0000 (oldest deadline 2024-08-19T01:44:56.262298+0000) 2024-08-19T01:45:05.952 INFO:journalctl@ceph.osd.10.smithi117.stdout:Aug 19 01:45:05 smithi117 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-10[46616]: 2024-08-19T01:45:05.534+0000 7f34b3bf7640 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-08-19T01:44:38.563674+0000 front 2024-08-19T01:44:38.563520+0000 (oldest deadline 2024-08-19T01:45:02.063517+0000) 2024-08-19T01:45:06.077 INFO:journalctl@ceph.osd.5.smithi167.stdout:Aug 19 01:45:05 smithi167 podman[56894]: 2024-08-19 01:45:05.666833271 +0000 UTC m=+0.017284977 image pull ea7a5189f30bad3aca9524788e2965d2368d7ce786df09d4edcd8c3edf9cefba quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c9dd28080a631be25c77c36c63b17a3ef340ffb2e21c87d4d7b56b6398436d9 2024-08-19T01:45:06.077 INFO:journalctl@ceph.osd.5.smithi167.stdout:Aug 19 01:45:05 smithi167 podman[56894]: 2024-08-19 01:45:05.765321013 +0000 UTC m=+0.115772708 container create cbe1272b90fe1c0ccd8c18378223a46e0bca8757d155af2bfe1a58caa6893146 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c9dd28080a631be25c77c36c63b17a3ef340ffb2e21c87d4d7b56b6398436d9, name=ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-5-deactivate, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, io.buildah.version=1.36.0, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-de09dd0, ceph=True) 2024-08-19T01:45:06.077 INFO:journalctl@ceph.osd.5.smithi167.stdout:Aug 19 01:45:06 smithi167 podman[56894]: 2024-08-19 01:45:06.001790083 +0000 UTC m=+0.352241797 container init cbe1272b90fe1c0ccd8c18378223a46e0bca8757d155af2bfe1a58caa6893146 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c9dd28080a631be25c77c36c63b17a3ef340ffb2e21c87d4d7b56b6398436d9, name=ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-5-deactivate, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, RELEASE=squid-de09dd0, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.36.0, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , GIT_CLEAN=True) 2024-08-19T01:45:06.077 INFO:journalctl@ceph.osd.5.smithi167.stdout:Aug 19 01:45:06 smithi167 podman[56894]: 2024-08-19 01:45:06.005499762 +0000 UTC m=+0.355951458 container start cbe1272b90fe1c0ccd8c18378223a46e0bca8757d155af2bfe1a58caa6893146 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c9dd28080a631be25c77c36c63b17a3ef340ffb2e21c87d4d7b56b6398436d9, name=ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-5-deactivate, GIT_BRANCH=HEAD, RELEASE=squid-de09dd0, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, GIT_CLEAN=True, io.buildah.version=1.36.0, CEPH_POINT_RELEASE=) 2024-08-19T01:45:06.077 INFO:journalctl@ceph.osd.5.smithi167.stdout:Aug 19 01:45:06 smithi167 podman[56894]: 2024-08-19 01:45:06.076134593 +0000 UTC m=+0.426586320 container attach cbe1272b90fe1c0ccd8c18378223a46e0bca8757d155af2bfe1a58caa6893146 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c9dd28080a631be25c77c36c63b17a3ef340ffb2e21c87d4d7b56b6398436d9, name=ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-5-deactivate, io.buildah.version=1.36.0, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-de09dd0, ceph=True, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_CLEAN=True) 2024-08-19T01:45:06.077 INFO:journalctl@ceph.osd.11.smithi167.stdout:Aug 19 01:45:05 smithi167 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-11[46643]: 2024-08-19T01:45:05.782+0000 7f447f624640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-08-19T01:44:22.880210+0000 front 2024-08-19T01:44:22.880404+0000 (oldest deadline 2024-08-19T01:44:45.780352+0000) 2024-08-19T01:45:06.077 INFO:journalctl@ceph.osd.11.smithi167.stdout:Aug 19 01:45:05 smithi167 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-11[46643]: 2024-08-19T01:45:05.782+0000 7f447f624640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-08-19T01:44:31.081546+0000 front 2024-08-19T01:44:31.081654+0000 (oldest deadline 2024-08-19T01:44:53.981595+0000) 2024-08-19T01:45:06.078 INFO:journalctl@ceph.osd.11.smithi167.stdout:Aug 19 01:45:05 smithi167 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-11[46643]: 2024-08-19T01:45:05.782+0000 7f447f624640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-08-19T01:44:39.282638+0000 front 2024-08-19T01:44:39.282723+0000 (oldest deadline 2024-08-19T01:45:04.582709+0000) 2024-08-19T01:45:06.154 INFO:journalctl@ceph.osd.6.smithi083.stdout:Aug 19 01:45:05 smithi083 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-6[44809]: 2024-08-19T01:45:05.890+0000 7f3cc6e00640 -1 osd.6 89 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-08-19T01:44:21.901735+0000 front 2024-08-19T01:44:21.901860+0000 (oldest deadline 2024-08-19T01:44:47.801737+0000) 2024-08-19T01:45:06.154 INFO:journalctl@ceph.osd.6.smithi083.stdout:Aug 19 01:45:05 smithi083 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-6[44809]: 2024-08-19T01:45:05.890+0000 7f3cc6e00640 -1 osd.6 89 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-08-19T01:44:28.303418+0000 front 2024-08-19T01:44:28.303477+0000 (oldest deadline 2024-08-19T01:44:55.904426+0000) 2024-08-19T01:45:06.154 INFO:journalctl@ceph.osd.6.smithi083.stdout:Aug 19 01:45:05 smithi083 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-6[44809]: 2024-08-19T01:45:05.890+0000 7f3cc6e00640 -1 osd.6 89 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-08-19T01:44:37.506923+0000 front 2024-08-19T01:44:37.507030+0000 (oldest deadline 2024-08-19T01:45:01.606759+0000) 2024-08-19T01:45:06.452 INFO:journalctl@ceph.osd.7.smithi117.stdout:Aug 19 01:45:06 smithi117 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-7[39667]: 2024-08-19T01:45:06.155+0000 7f66eaeae640 -1 osd.7 89 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-08-19T01:44:31.724600+0000 front 2024-08-19T01:44:31.724641+0000 (oldest deadline 2024-08-19T01:44:54.024721+0000) 2024-08-19T01:45:06.452 INFO:journalctl@ceph.osd.7.smithi117.stdout:Aug 19 01:45:06 smithi117 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-7[39667]: 2024-08-19T01:45:06.155+0000 7f66eaeae640 -1 osd.7 89 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-08-19T01:44:40.426018+0000 front 2024-08-19T01:44:40.426002+0000 (oldest deadline 2024-08-19T01:45:01.526013+0000) 2024-08-19T01:45:06.466 INFO:journalctl@ceph.osd.5.smithi167.stdout:Aug 19 01:45:06 smithi167 podman[56894]: 2024-08-19 01:45:06.189792324 +0000 UTC m=+0.540244024 container died cbe1272b90fe1c0ccd8c18378223a46e0bca8757d155af2bfe1a58caa6893146 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c9dd28080a631be25c77c36c63b17a3ef340ffb2e21c87d4d7b56b6398436d9, name=ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-5-deactivate, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, io.buildah.version=1.36.0, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, RELEASE=squid-de09dd0) 2024-08-19T01:45:06.466 INFO:journalctl@ceph.osd.8.smithi167.stdout:Aug 19 01:45:06 smithi167 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-8[39928]: 2024-08-19T01:45:06.107+0000 7f17fe820640 -1 osd.8 89 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-08-19T01:44:22.747507+0000 front 2024-08-19T01:44:22.747516+0000 (oldest deadline 2024-08-19T01:44:48.047324+0000) 2024-08-19T01:45:06.466 INFO:journalctl@ceph.osd.8.smithi167.stdout:Aug 19 01:45:06 smithi167 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-8[39928]: 2024-08-19T01:45:06.107+0000 7f17fe820640 -1 osd.8 89 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-08-19T01:44:32.049249+0000 front 2024-08-19T01:44:32.049153+0000 (oldest deadline 2024-08-19T01:44:53.749126+0000) 2024-08-19T01:45:06.466 INFO:journalctl@ceph.osd.8.smithi167.stdout:Aug 19 01:45:06 smithi167 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-8[39928]: 2024-08-19T01:45:06.107+0000 7f17fe820640 -1 osd.8 89 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-08-19T01:44:36.050034+0000 front 2024-08-19T01:44:36.049973+0000 (oldest deadline 2024-08-19T01:45:06.050849+0000) 2024-08-19T01:45:06.654 INFO:journalctl@ceph.osd.9.smithi083.stdout:Aug 19 01:45:06 smithi083 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-9[51472]: 2024-08-19T01:45:06.376+0000 7f54e8c00640 -1 osd.9 89 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-08-19T01:44:20.873282+0000 front 2024-08-19T01:44:20.873259+0000 (oldest deadline 2024-08-19T01:44:46.773413+0000) 2024-08-19T01:45:06.654 INFO:journalctl@ceph.osd.9.smithi083.stdout:Aug 19 01:45:06 smithi083 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-9[51472]: 2024-08-19T01:45:06.376+0000 7f54e8c00640 -1 osd.9 89 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-08-19T01:44:26.774104+0000 front 2024-08-19T01:44:26.774036+0000 (oldest deadline 2024-08-19T01:44:53.774823+0000) 2024-08-19T01:45:06.654 INFO:journalctl@ceph.osd.9.smithi083.stdout:Aug 19 01:45:06 smithi083 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-9[51472]: 2024-08-19T01:45:06.376+0000 7f54e8c00640 -1 osd.9 89 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-08-19T01:44:39.676617+0000 front 2024-08-19T01:44:39.676511+0000 (oldest deadline 2024-08-19T01:45:03.776684+0000) 2024-08-19T01:45:06.787 DEBUG:teuthology.orchestra.run.smithi167:> sudo pkill -f 'journalctl -f -n 0 -u ceph-bad8780e-5dc9-11ef-bccf-c7b262605968@osd.5.service' 2024-08-19T01:45:06.828 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-08-19T01:45:06.828 INFO:tasks.cephadm.osd.5:Stopped osd.5 2024-08-19T01:45:06.828 INFO:tasks.cephadm.osd.6:Stopping osd.6... 2024-08-19T01:45:06.828 DEBUG:teuthology.orchestra.run.smithi083:> sudo systemctl stop ceph-bad8780e-5dc9-11ef-bccf-c7b262605968@osd.6 2024-08-19T01:45:06.952 INFO:journalctl@ceph.osd.10.smithi117.stdout:Aug 19 01:45:06 smithi117 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-10[46616]: 2024-08-19T01:45:06.536+0000 7f34b3bf7640 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-08-19T01:44:24.060751+0000 front 2024-08-19T01:44:24.060716+0000 (oldest deadline 2024-08-19T01:44:47.560677+0000) 2024-08-19T01:45:06.952 INFO:journalctl@ceph.osd.10.smithi117.stdout:Aug 19 01:45:06 smithi117 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-10[46616]: 2024-08-19T01:45:06.536+0000 7f34b3bf7640 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-08-19T01:44:28.661503+0000 front 2024-08-19T01:44:28.661522+0000 (oldest deadline 2024-08-19T01:44:56.262298+0000) 2024-08-19T01:45:06.952 INFO:journalctl@ceph.osd.10.smithi117.stdout:Aug 19 01:45:06 smithi117 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-10[46616]: 2024-08-19T01:45:06.536+0000 7f34b3bf7640 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-08-19T01:44:38.563674+0000 front 2024-08-19T01:44:38.563520+0000 (oldest deadline 2024-08-19T01:45:02.063517+0000) 2024-08-19T01:45:07.145 INFO:journalctl@ceph.osd.11.smithi167.stdout:Aug 19 01:45:06 smithi167 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-11[46643]: 2024-08-19T01:45:06.831+0000 7f447f624640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-08-19T01:44:22.880210+0000 front 2024-08-19T01:44:22.880404+0000 (oldest deadline 2024-08-19T01:44:45.780352+0000) 2024-08-19T01:45:07.145 INFO:journalctl@ceph.osd.11.smithi167.stdout:Aug 19 01:45:06 smithi167 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-11[46643]: 2024-08-19T01:45:06.831+0000 7f447f624640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-08-19T01:44:31.081546+0000 front 2024-08-19T01:44:31.081654+0000 (oldest deadline 2024-08-19T01:44:53.981595+0000) 2024-08-19T01:45:07.145 INFO:journalctl@ceph.osd.11.smithi167.stdout:Aug 19 01:45:06 smithi167 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-11[46643]: 2024-08-19T01:45:06.831+0000 7f447f624640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-08-19T01:44:39.282638+0000 front 2024-08-19T01:44:39.282723+0000 (oldest deadline 2024-08-19T01:45:04.582709+0000) 2024-08-19T01:45:07.154 INFO:journalctl@ceph.osd.6.smithi083.stdout:Aug 19 01:45:06 smithi083 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-6[44809]: 2024-08-19T01:45:06.880+0000 7f3cc6e00640 -1 osd.6 89 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-08-19T01:44:21.901735+0000 front 2024-08-19T01:44:21.901860+0000 (oldest deadline 2024-08-19T01:44:47.801737+0000) 2024-08-19T01:45:07.154 INFO:journalctl@ceph.osd.6.smithi083.stdout:Aug 19 01:45:06 smithi083 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-6[44809]: 2024-08-19T01:45:06.880+0000 7f3cc6e00640 -1 osd.6 89 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-08-19T01:44:28.303418+0000 front 2024-08-19T01:44:28.303477+0000 (oldest deadline 2024-08-19T01:44:55.904426+0000) 2024-08-19T01:45:07.154 INFO:journalctl@ceph.osd.6.smithi083.stdout:Aug 19 01:45:06 smithi083 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-6[44809]: 2024-08-19T01:45:06.880+0000 7f3cc6e00640 -1 osd.6 89 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-08-19T01:44:37.506923+0000 front 2024-08-19T01:44:37.507030+0000 (oldest deadline 2024-08-19T01:45:01.606759+0000) 2024-08-19T01:45:07.154 INFO:journalctl@ceph.osd.6.smithi083.stdout:Aug 19 01:45:06 smithi083 systemd[1]: Stopping Ceph osd.6 for bad8780e-5dc9-11ef-bccf-c7b262605968... 2024-08-19T01:45:07.452 INFO:journalctl@ceph.osd.7.smithi117.stdout:Aug 19 01:45:07 smithi117 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-7[39667]: 2024-08-19T01:45:07.135+0000 7f66eaeae640 -1 osd.7 89 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-08-19T01:44:31.724600+0000 front 2024-08-19T01:44:31.724641+0000 (oldest deadline 2024-08-19T01:44:54.024721+0000) 2024-08-19T01:45:07.452 INFO:journalctl@ceph.osd.7.smithi117.stdout:Aug 19 01:45:07 smithi117 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-7[39667]: 2024-08-19T01:45:07.135+0000 7f66eaeae640 -1 osd.7 89 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-08-19T01:44:40.426018+0000 front 2024-08-19T01:44:40.426002+0000 (oldest deadline 2024-08-19T01:45:01.526013+0000) 2024-08-19T01:45:07.465 INFO:journalctl@ceph.osd.8.smithi167.stdout:Aug 19 01:45:07 smithi167 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-8[39928]: 2024-08-19T01:45:07.144+0000 7f17fe820640 -1 osd.8 89 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-08-19T01:44:22.747507+0000 front 2024-08-19T01:44:22.747516+0000 (oldest deadline 2024-08-19T01:44:48.047324+0000) 2024-08-19T01:45:07.466 INFO:journalctl@ceph.osd.8.smithi167.stdout:Aug 19 01:45:07 smithi167 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-8[39928]: 2024-08-19T01:45:07.144+0000 7f17fe820640 -1 osd.8 89 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-08-19T01:44:32.049249+0000 front 2024-08-19T01:44:32.049153+0000 (oldest deadline 2024-08-19T01:44:53.749126+0000) 2024-08-19T01:45:07.466 INFO:journalctl@ceph.osd.8.smithi167.stdout:Aug 19 01:45:07 smithi167 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-8[39928]: 2024-08-19T01:45:07.144+0000 7f17fe820640 -1 osd.8 89 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-08-19T01:44:36.050034+0000 front 2024-08-19T01:44:36.049973+0000 (oldest deadline 2024-08-19T01:45:06.050849+0000) 2024-08-19T01:45:07.654 INFO:journalctl@ceph.osd.9.smithi083.stdout:Aug 19 01:45:07 smithi083 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-9[51472]: 2024-08-19T01:45:07.389+0000 7f54e8c00640 -1 osd.9 89 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-08-19T01:44:20.873282+0000 front 2024-08-19T01:44:20.873259+0000 (oldest deadline 2024-08-19T01:44:46.773413+0000) 2024-08-19T01:45:07.655 INFO:journalctl@ceph.osd.9.smithi083.stdout:Aug 19 01:45:07 smithi083 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-9[51472]: 2024-08-19T01:45:07.389+0000 7f54e8c00640 -1 osd.9 89 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-08-19T01:44:26.774104+0000 front 2024-08-19T01:44:26.774036+0000 (oldest deadline 2024-08-19T01:44:53.774823+0000) 2024-08-19T01:45:07.655 INFO:journalctl@ceph.osd.9.smithi083.stdout:Aug 19 01:45:07 smithi083 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-9[51472]: 2024-08-19T01:45:07.389+0000 7f54e8c00640 -1 osd.9 89 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-08-19T01:44:39.676617+0000 front 2024-08-19T01:44:39.676511+0000 (oldest deadline 2024-08-19T01:45:03.776684+0000) 2024-08-19T01:45:07.655 INFO:journalctl@ceph.osd.6.smithi083.stdout:Aug 19 01:45:07 smithi083 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-6[44809]: 2024-08-19T01:45:07.299+0000 7f3ccaa00640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.6 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-08-19T01:45:07.655 INFO:journalctl@ceph.osd.6.smithi083.stdout:Aug 19 01:45:07 smithi083 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-6[44809]: 2024-08-19T01:45:07.299+0000 7f3ccaa00640 -1 osd.6 89 *** Got signal Terminated *** 2024-08-19T01:45:07.655 INFO:journalctl@ceph.osd.6.smithi083.stdout:Aug 19 01:45:07 smithi083 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-6[44809]: 2024-08-19T01:45:07.299+0000 7f3ccaa00640 -1 osd.6 89 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-08-19T01:45:07.952 INFO:journalctl@ceph.osd.10.smithi117.stdout:Aug 19 01:45:07 smithi117 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-10[46616]: 2024-08-19T01:45:07.508+0000 7f34b3bf7640 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-08-19T01:44:24.060751+0000 front 2024-08-19T01:44:24.060716+0000 (oldest deadline 2024-08-19T01:44:47.560677+0000) 2024-08-19T01:45:07.952 INFO:journalctl@ceph.osd.10.smithi117.stdout:Aug 19 01:45:07 smithi117 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-10[46616]: 2024-08-19T01:45:07.508+0000 7f34b3bf7640 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-08-19T01:44:28.661503+0000 front 2024-08-19T01:44:28.661522+0000 (oldest deadline 2024-08-19T01:44:56.262298+0000) 2024-08-19T01:45:07.953 INFO:journalctl@ceph.osd.10.smithi117.stdout:Aug 19 01:45:07 smithi117 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-10[46616]: 2024-08-19T01:45:07.508+0000 7f34b3bf7640 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-08-19T01:44:38.563674+0000 front 2024-08-19T01:44:38.563520+0000 (oldest deadline 2024-08-19T01:45:02.063517+0000) 2024-08-19T01:45:08.154 INFO:journalctl@ceph.osd.6.smithi083.stdout:Aug 19 01:45:07 smithi083 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-6[44809]: 2024-08-19T01:45:07.842+0000 7f3cc6e00640 -1 osd.6 89 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-08-19T01:44:21.901735+0000 front 2024-08-19T01:44:21.901860+0000 (oldest deadline 2024-08-19T01:44:47.801737+0000) 2024-08-19T01:45:08.154 INFO:journalctl@ceph.osd.6.smithi083.stdout:Aug 19 01:45:07 smithi083 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-6[44809]: 2024-08-19T01:45:07.842+0000 7f3cc6e00640 -1 osd.6 89 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-08-19T01:44:28.303418+0000 front 2024-08-19T01:44:28.303477+0000 (oldest deadline 2024-08-19T01:44:55.904426+0000) 2024-08-19T01:45:08.155 INFO:journalctl@ceph.osd.6.smithi083.stdout:Aug 19 01:45:07 smithi083 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-6[44809]: 2024-08-19T01:45:07.842+0000 7f3cc6e00640 -1 osd.6 89 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-08-19T01:44:37.506923+0000 front 2024-08-19T01:44:37.507030+0000 (oldest deadline 2024-08-19T01:45:01.606759+0000) 2024-08-19T01:45:08.216 INFO:journalctl@ceph.osd.8.smithi167.stdout:Aug 19 01:45:08 smithi167 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-8[39928]: 2024-08-19T01:45:08.121+0000 7f17fe820640 -1 osd.8 89 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-08-19T01:44:22.747507+0000 front 2024-08-19T01:44:22.747516+0000 (oldest deadline 2024-08-19T01:44:48.047324+0000) 2024-08-19T01:45:08.216 INFO:journalctl@ceph.osd.8.smithi167.stdout:Aug 19 01:45:08 smithi167 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-8[39928]: 2024-08-19T01:45:08.121+0000 7f17fe820640 -1 osd.8 89 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-08-19T01:44:32.049249+0000 front 2024-08-19T01:44:32.049153+0000 (oldest deadline 2024-08-19T01:44:53.749126+0000) 2024-08-19T01:45:08.216 INFO:journalctl@ceph.osd.8.smithi167.stdout:Aug 19 01:45:08 smithi167 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-8[39928]: 2024-08-19T01:45:08.121+0000 7f17fe820640 -1 osd.8 89 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-08-19T01:44:36.050034+0000 front 2024-08-19T01:44:36.049973+0000 (oldest deadline 2024-08-19T01:45:06.050849+0000) 2024-08-19T01:45:08.216 INFO:journalctl@ceph.osd.11.smithi167.stdout:Aug 19 01:45:07 smithi167 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-11[46643]: 2024-08-19T01:45:07.875+0000 7f447f624640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-08-19T01:44:22.880210+0000 front 2024-08-19T01:44:22.880404+0000 (oldest deadline 2024-08-19T01:44:45.780352+0000) 2024-08-19T01:45:08.216 INFO:journalctl@ceph.osd.11.smithi167.stdout:Aug 19 01:45:07 smithi167 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-11[46643]: 2024-08-19T01:45:07.875+0000 7f447f624640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-08-19T01:44:31.081546+0000 front 2024-08-19T01:44:31.081654+0000 (oldest deadline 2024-08-19T01:44:53.981595+0000) 2024-08-19T01:45:08.217 INFO:journalctl@ceph.osd.11.smithi167.stdout:Aug 19 01:45:07 smithi167 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-11[46643]: 2024-08-19T01:45:07.875+0000 7f447f624640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-08-19T01:44:39.282638+0000 front 2024-08-19T01:44:39.282723+0000 (oldest deadline 2024-08-19T01:45:04.582709+0000) 2024-08-19T01:45:08.452 INFO:journalctl@ceph.osd.7.smithi117.stdout:Aug 19 01:45:08 smithi117 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-7[39667]: 2024-08-19T01:45:08.121+0000 7f66eaeae640 -1 osd.7 89 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-08-19T01:44:31.724600+0000 front 2024-08-19T01:44:31.724641+0000 (oldest deadline 2024-08-19T01:44:54.024721+0000) 2024-08-19T01:45:08.452 INFO:journalctl@ceph.osd.7.smithi117.stdout:Aug 19 01:45:08 smithi117 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-7[39667]: 2024-08-19T01:45:08.121+0000 7f66eaeae640 -1 osd.7 89 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-08-19T01:44:40.426018+0000 front 2024-08-19T01:44:40.426002+0000 (oldest deadline 2024-08-19T01:45:01.526013+0000) 2024-08-19T01:45:08.888 INFO:journalctl@ceph.osd.9.smithi083.stdout:Aug 19 01:45:08 smithi083 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-9[51472]: 2024-08-19T01:45:08.407+0000 7f54e8c00640 -1 osd.9 89 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-08-19T01:44:20.873282+0000 front 2024-08-19T01:44:20.873259+0000 (oldest deadline 2024-08-19T01:44:46.773413+0000) 2024-08-19T01:45:08.888 INFO:journalctl@ceph.osd.9.smithi083.stdout:Aug 19 01:45:08 smithi083 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-9[51472]: 2024-08-19T01:45:08.407+0000 7f54e8c00640 -1 osd.9 89 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-08-19T01:44:26.774104+0000 front 2024-08-19T01:44:26.774036+0000 (oldest deadline 2024-08-19T01:44:53.774823+0000) 2024-08-19T01:45:08.888 INFO:journalctl@ceph.osd.9.smithi083.stdout:Aug 19 01:45:08 smithi083 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-9[51472]: 2024-08-19T01:45:08.407+0000 7f54e8c00640 -1 osd.9 89 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-08-19T01:44:39.676617+0000 front 2024-08-19T01:44:39.676511+0000 (oldest deadline 2024-08-19T01:45:03.776684+0000) 2024-08-19T01:45:08.952 INFO:journalctl@ceph.osd.10.smithi117.stdout:Aug 19 01:45:08 smithi117 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-10[46616]: 2024-08-19T01:45:08.533+0000 7f34b3bf7640 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-08-19T01:44:24.060751+0000 front 2024-08-19T01:44:24.060716+0000 (oldest deadline 2024-08-19T01:44:47.560677+0000) 2024-08-19T01:45:08.952 INFO:journalctl@ceph.osd.10.smithi117.stdout:Aug 19 01:45:08 smithi117 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-10[46616]: 2024-08-19T01:45:08.533+0000 7f34b3bf7640 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-08-19T01:44:28.661503+0000 front 2024-08-19T01:44:28.661522+0000 (oldest deadline 2024-08-19T01:44:56.262298+0000) 2024-08-19T01:45:08.952 INFO:journalctl@ceph.osd.10.smithi117.stdout:Aug 19 01:45:08 smithi117 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-10[46616]: 2024-08-19T01:45:08.533+0000 7f34b3bf7640 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-08-19T01:44:38.563674+0000 front 2024-08-19T01:44:38.563520+0000 (oldest deadline 2024-08-19T01:45:02.063517+0000) 2024-08-19T01:45:09.154 INFO:journalctl@ceph.osd.6.smithi083.stdout:Aug 19 01:45:08 smithi083 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-6[44809]: 2024-08-19T01:45:08.887+0000 7f3cc6e00640 -1 osd.6 89 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-08-19T01:44:21.901735+0000 front 2024-08-19T01:44:21.901860+0000 (oldest deadline 2024-08-19T01:44:47.801737+0000) 2024-08-19T01:45:09.154 INFO:journalctl@ceph.osd.6.smithi083.stdout:Aug 19 01:45:08 smithi083 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-6[44809]: 2024-08-19T01:45:08.887+0000 7f3cc6e00640 -1 osd.6 89 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-08-19T01:44:28.303418+0000 front 2024-08-19T01:44:28.303477+0000 (oldest deadline 2024-08-19T01:44:55.904426+0000) 2024-08-19T01:45:09.155 INFO:journalctl@ceph.osd.6.smithi083.stdout:Aug 19 01:45:08 smithi083 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-6[44809]: 2024-08-19T01:45:08.887+0000 7f3cc6e00640 -1 osd.6 89 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-08-19T01:44:37.506923+0000 front 2024-08-19T01:44:37.507030+0000 (oldest deadline 2024-08-19T01:45:01.606759+0000) 2024-08-19T01:45:09.216 INFO:journalctl@ceph.osd.8.smithi167.stdout:Aug 19 01:45:09 smithi167 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-8[39928]: 2024-08-19T01:45:09.092+0000 7f17fe820640 -1 osd.8 89 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-08-19T01:44:22.747507+0000 front 2024-08-19T01:44:22.747516+0000 (oldest deadline 2024-08-19T01:44:48.047324+0000) 2024-08-19T01:45:09.216 INFO:journalctl@ceph.osd.8.smithi167.stdout:Aug 19 01:45:09 smithi167 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-8[39928]: 2024-08-19T01:45:09.092+0000 7f17fe820640 -1 osd.8 89 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-08-19T01:44:32.049249+0000 front 2024-08-19T01:44:32.049153+0000 (oldest deadline 2024-08-19T01:44:53.749126+0000) 2024-08-19T01:45:09.216 INFO:journalctl@ceph.osd.8.smithi167.stdout:Aug 19 01:45:09 smithi167 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-8[39928]: 2024-08-19T01:45:09.092+0000 7f17fe820640 -1 osd.8 89 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-08-19T01:44:36.050034+0000 front 2024-08-19T01:44:36.049973+0000 (oldest deadline 2024-08-19T01:45:06.050849+0000) 2024-08-19T01:45:09.216 INFO:journalctl@ceph.osd.11.smithi167.stdout:Aug 19 01:45:08 smithi167 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-11[46643]: 2024-08-19T01:45:08.913+0000 7f447f624640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-08-19T01:44:22.880210+0000 front 2024-08-19T01:44:22.880404+0000 (oldest deadline 2024-08-19T01:44:45.780352+0000) 2024-08-19T01:45:09.216 INFO:journalctl@ceph.osd.11.smithi167.stdout:Aug 19 01:45:08 smithi167 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-11[46643]: 2024-08-19T01:45:08.913+0000 7f447f624640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-08-19T01:44:31.081546+0000 front 2024-08-19T01:44:31.081654+0000 (oldest deadline 2024-08-19T01:44:53.981595+0000) 2024-08-19T01:45:09.216 INFO:journalctl@ceph.osd.11.smithi167.stdout:Aug 19 01:45:08 smithi167 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-11[46643]: 2024-08-19T01:45:08.913+0000 7f447f624640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-08-19T01:44:39.282638+0000 front 2024-08-19T01:44:39.282723+0000 (oldest deadline 2024-08-19T01:45:04.582709+0000) 2024-08-19T01:45:09.452 INFO:journalctl@ceph.osd.7.smithi117.stdout:Aug 19 01:45:09 smithi117 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-7[39667]: 2024-08-19T01:45:09.121+0000 7f66eaeae640 -1 osd.7 89 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-08-19T01:44:31.724600+0000 front 2024-08-19T01:44:31.724641+0000 (oldest deadline 2024-08-19T01:44:54.024721+0000) 2024-08-19T01:45:09.452 INFO:journalctl@ceph.osd.7.smithi117.stdout:Aug 19 01:45:09 smithi117 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-7[39667]: 2024-08-19T01:45:09.121+0000 7f66eaeae640 -1 osd.7 89 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-08-19T01:44:40.426018+0000 front 2024-08-19T01:44:40.426002+0000 (oldest deadline 2024-08-19T01:45:01.526013+0000) 2024-08-19T01:45:09.654 INFO:journalctl@ceph.osd.9.smithi083.stdout:Aug 19 01:45:09 smithi083 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-9[51472]: 2024-08-19T01:45:09.399+0000 7f54e8c00640 -1 osd.9 89 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-08-19T01:44:20.873282+0000 front 2024-08-19T01:44:20.873259+0000 (oldest deadline 2024-08-19T01:44:46.773413+0000) 2024-08-19T01:45:09.654 INFO:journalctl@ceph.osd.9.smithi083.stdout:Aug 19 01:45:09 smithi083 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-9[51472]: 2024-08-19T01:45:09.399+0000 7f54e8c00640 -1 osd.9 89 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-08-19T01:44:26.774104+0000 front 2024-08-19T01:44:26.774036+0000 (oldest deadline 2024-08-19T01:44:53.774823+0000) 2024-08-19T01:45:09.654 INFO:journalctl@ceph.osd.9.smithi083.stdout:Aug 19 01:45:09 smithi083 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-9[51472]: 2024-08-19T01:45:09.399+0000 7f54e8c00640 -1 osd.9 89 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-08-19T01:44:39.676617+0000 front 2024-08-19T01:44:39.676511+0000 (oldest deadline 2024-08-19T01:45:03.776684+0000) 2024-08-19T01:45:09.952 INFO:journalctl@ceph.osd.10.smithi117.stdout:Aug 19 01:45:09 smithi117 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-10[46616]: 2024-08-19T01:45:09.512+0000 7f34b3bf7640 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-08-19T01:44:24.060751+0000 front 2024-08-19T01:44:24.060716+0000 (oldest deadline 2024-08-19T01:44:47.560677+0000) 2024-08-19T01:45:09.952 INFO:journalctl@ceph.osd.10.smithi117.stdout:Aug 19 01:45:09 smithi117 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-10[46616]: 2024-08-19T01:45:09.512+0000 7f34b3bf7640 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-08-19T01:44:28.661503+0000 front 2024-08-19T01:44:28.661522+0000 (oldest deadline 2024-08-19T01:44:56.262298+0000) 2024-08-19T01:45:09.952 INFO:journalctl@ceph.osd.10.smithi117.stdout:Aug 19 01:45:09 smithi117 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-10[46616]: 2024-08-19T01:45:09.512+0000 7f34b3bf7640 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-08-19T01:44:38.563674+0000 front 2024-08-19T01:44:38.563520+0000 (oldest deadline 2024-08-19T01:45:02.063517+0000) 2024-08-19T01:45:10.154 INFO:journalctl@ceph.osd.6.smithi083.stdout:Aug 19 01:45:09 smithi083 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-6[44809]: 2024-08-19T01:45:09.888+0000 7f3cc6e00640 -1 osd.6 89 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-08-19T01:44:21.901735+0000 front 2024-08-19T01:44:21.901860+0000 (oldest deadline 2024-08-19T01:44:47.801737+0000) 2024-08-19T01:45:10.154 INFO:journalctl@ceph.osd.6.smithi083.stdout:Aug 19 01:45:09 smithi083 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-6[44809]: 2024-08-19T01:45:09.888+0000 7f3cc6e00640 -1 osd.6 89 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-08-19T01:44:28.303418+0000 front 2024-08-19T01:44:28.303477+0000 (oldest deadline 2024-08-19T01:44:55.904426+0000) 2024-08-19T01:45:10.155 INFO:journalctl@ceph.osd.6.smithi083.stdout:Aug 19 01:45:09 smithi083 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-6[44809]: 2024-08-19T01:45:09.888+0000 7f3cc6e00640 -1 osd.6 89 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-08-19T01:44:37.506923+0000 front 2024-08-19T01:44:37.507030+0000 (oldest deadline 2024-08-19T01:45:01.606759+0000) 2024-08-19T01:45:10.155 INFO:journalctl@ceph.osd.6.smithi083.stdout:Aug 19 01:45:09 smithi083 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-6[44809]: 2024-08-19T01:45:09.888+0000 7f3cc6e00640 -1 osd.6 89 heartbeat_check: no reply from 172.21.15.83:6814 osd.3 since back 2024-08-19T01:44:44.308698+0000 front 2024-08-19T01:44:44.308741+0000 (oldest deadline 2024-08-19T01:45:08.909353+0000) 2024-08-19T01:45:10.216 INFO:journalctl@ceph.osd.8.smithi167.stdout:Aug 19 01:45:10 smithi167 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-8[39928]: 2024-08-19T01:45:10.104+0000 7f17fe820640 -1 osd.8 89 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-08-19T01:44:22.747507+0000 front 2024-08-19T01:44:22.747516+0000 (oldest deadline 2024-08-19T01:44:48.047324+0000) 2024-08-19T01:45:10.216 INFO:journalctl@ceph.osd.8.smithi167.stdout:Aug 19 01:45:10 smithi167 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-8[39928]: 2024-08-19T01:45:10.104+0000 7f17fe820640 -1 osd.8 89 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-08-19T01:44:32.049249+0000 front 2024-08-19T01:44:32.049153+0000 (oldest deadline 2024-08-19T01:44:53.749126+0000) 2024-08-19T01:45:10.216 INFO:journalctl@ceph.osd.8.smithi167.stdout:Aug 19 01:45:10 smithi167 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-8[39928]: 2024-08-19T01:45:10.104+0000 7f17fe820640 -1 osd.8 89 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-08-19T01:44:36.050034+0000 front 2024-08-19T01:44:36.049973+0000 (oldest deadline 2024-08-19T01:45:06.050849+0000) 2024-08-19T01:45:10.216 INFO:journalctl@ceph.osd.11.smithi167.stdout:Aug 19 01:45:09 smithi167 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-11[46643]: 2024-08-19T01:45:09.927+0000 7f447f624640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-08-19T01:44:22.880210+0000 front 2024-08-19T01:44:22.880404+0000 (oldest deadline 2024-08-19T01:44:45.780352+0000) 2024-08-19T01:45:10.216 INFO:journalctl@ceph.osd.11.smithi167.stdout:Aug 19 01:45:09 smithi167 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-11[46643]: 2024-08-19T01:45:09.927+0000 7f447f624640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-08-19T01:44:31.081546+0000 front 2024-08-19T01:44:31.081654+0000 (oldest deadline 2024-08-19T01:44:53.981595+0000) 2024-08-19T01:45:10.217 INFO:journalctl@ceph.osd.11.smithi167.stdout:Aug 19 01:45:09 smithi167 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-11[46643]: 2024-08-19T01:45:09.927+0000 7f447f624640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-08-19T01:44:39.282638+0000 front 2024-08-19T01:44:39.282723+0000 (oldest deadline 2024-08-19T01:45:04.582709+0000) 2024-08-19T01:45:10.452 INFO:journalctl@ceph.osd.7.smithi117.stdout:Aug 19 01:45:10 smithi117 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-7[39667]: 2024-08-19T01:45:10.095+0000 7f66eaeae640 -1 osd.7 89 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-08-19T01:44:31.724600+0000 front 2024-08-19T01:44:31.724641+0000 (oldest deadline 2024-08-19T01:44:54.024721+0000) 2024-08-19T01:45:10.452 INFO:journalctl@ceph.osd.7.smithi117.stdout:Aug 19 01:45:10 smithi117 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-7[39667]: 2024-08-19T01:45:10.095+0000 7f66eaeae640 -1 osd.7 89 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-08-19T01:44:40.426018+0000 front 2024-08-19T01:44:40.426002+0000 (oldest deadline 2024-08-19T01:45:01.526013+0000) 2024-08-19T01:45:10.452 INFO:journalctl@ceph.osd.7.smithi117.stdout:Aug 19 01:45:10 smithi117 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-7[39667]: 2024-08-19T01:45:10.095+0000 7f66eaeae640 -1 osd.7 89 heartbeat_check: no reply from 172.21.15.83:6814 osd.3 since back 2024-08-19T01:44:45.627330+0000 front 2024-08-19T01:44:45.627160+0000 (oldest deadline 2024-08-19T01:45:09.727143+0000) 2024-08-19T01:45:10.655 INFO:journalctl@ceph.osd.9.smithi083.stdout:Aug 19 01:45:10 smithi083 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-9[51472]: 2024-08-19T01:45:10.363+0000 7f54e8c00640 -1 osd.9 89 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-08-19T01:44:20.873282+0000 front 2024-08-19T01:44:20.873259+0000 (oldest deadline 2024-08-19T01:44:46.773413+0000) 2024-08-19T01:45:10.655 INFO:journalctl@ceph.osd.9.smithi083.stdout:Aug 19 01:45:10 smithi083 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-9[51472]: 2024-08-19T01:45:10.363+0000 7f54e8c00640 -1 osd.9 89 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-08-19T01:44:26.774104+0000 front 2024-08-19T01:44:26.774036+0000 (oldest deadline 2024-08-19T01:44:53.774823+0000) 2024-08-19T01:45:10.655 INFO:journalctl@ceph.osd.9.smithi083.stdout:Aug 19 01:45:10 smithi083 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-9[51472]: 2024-08-19T01:45:10.363+0000 7f54e8c00640 -1 osd.9 89 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-08-19T01:44:39.676617+0000 front 2024-08-19T01:44:39.676511+0000 (oldest deadline 2024-08-19T01:45:03.776684+0000) 2024-08-19T01:45:10.952 INFO:journalctl@ceph.osd.10.smithi117.stdout:Aug 19 01:45:10 smithi117 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-10[46616]: 2024-08-19T01:45:10.559+0000 7f34b3bf7640 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-08-19T01:44:24.060751+0000 front 2024-08-19T01:44:24.060716+0000 (oldest deadline 2024-08-19T01:44:47.560677+0000) 2024-08-19T01:45:10.952 INFO:journalctl@ceph.osd.10.smithi117.stdout:Aug 19 01:45:10 smithi117 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-10[46616]: 2024-08-19T01:45:10.559+0000 7f34b3bf7640 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-08-19T01:44:28.661503+0000 front 2024-08-19T01:44:28.661522+0000 (oldest deadline 2024-08-19T01:44:56.262298+0000) 2024-08-19T01:45:10.952 INFO:journalctl@ceph.osd.10.smithi117.stdout:Aug 19 01:45:10 smithi117 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-10[46616]: 2024-08-19T01:45:10.559+0000 7f34b3bf7640 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-08-19T01:44:38.563674+0000 front 2024-08-19T01:44:38.563520+0000 (oldest deadline 2024-08-19T01:45:02.063517+0000) 2024-08-19T01:45:10.952 INFO:journalctl@ceph.osd.10.smithi117.stdout:Aug 19 01:45:10 smithi117 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-10[46616]: 2024-08-19T01:45:10.559+0000 7f34b3bf7640 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.83:6814 osd.3 since back 2024-08-19T01:44:47.265176+0000 front 2024-08-19T01:44:47.265102+0000 (oldest deadline 2024-08-19T01:45:09.564999+0000) 2024-08-19T01:45:11.154 INFO:journalctl@ceph.osd.6.smithi083.stdout:Aug 19 01:45:10 smithi083 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-6[44809]: 2024-08-19T01:45:10.858+0000 7f3cc6e00640 -1 osd.6 89 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-08-19T01:44:21.901735+0000 front 2024-08-19T01:44:21.901860+0000 (oldest deadline 2024-08-19T01:44:47.801737+0000) 2024-08-19T01:45:11.154 INFO:journalctl@ceph.osd.6.smithi083.stdout:Aug 19 01:45:10 smithi083 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-6[44809]: 2024-08-19T01:45:10.858+0000 7f3cc6e00640 -1 osd.6 89 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-08-19T01:44:28.303418+0000 front 2024-08-19T01:44:28.303477+0000 (oldest deadline 2024-08-19T01:44:55.904426+0000) 2024-08-19T01:45:11.154 INFO:journalctl@ceph.osd.6.smithi083.stdout:Aug 19 01:45:10 smithi083 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-6[44809]: 2024-08-19T01:45:10.858+0000 7f3cc6e00640 -1 osd.6 89 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-08-19T01:44:37.506923+0000 front 2024-08-19T01:44:37.507030+0000 (oldest deadline 2024-08-19T01:45:01.606759+0000) 2024-08-19T01:45:11.154 INFO:journalctl@ceph.osd.6.smithi083.stdout:Aug 19 01:45:10 smithi083 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-6[44809]: 2024-08-19T01:45:10.858+0000 7f3cc6e00640 -1 osd.6 89 heartbeat_check: no reply from 172.21.15.83:6814 osd.3 since back 2024-08-19T01:44:44.308698+0000 front 2024-08-19T01:44:44.308741+0000 (oldest deadline 2024-08-19T01:45:08.909353+0000) 2024-08-19T01:45:11.216 INFO:journalctl@ceph.osd.8.smithi167.stdout:Aug 19 01:45:11 smithi167 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-8[39928]: 2024-08-19T01:45:11.134+0000 7f17fe820640 -1 osd.8 89 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-08-19T01:44:22.747507+0000 front 2024-08-19T01:44:22.747516+0000 (oldest deadline 2024-08-19T01:44:48.047324+0000) 2024-08-19T01:45:11.216 INFO:journalctl@ceph.osd.8.smithi167.stdout:Aug 19 01:45:11 smithi167 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-8[39928]: 2024-08-19T01:45:11.134+0000 7f17fe820640 -1 osd.8 89 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-08-19T01:44:32.049249+0000 front 2024-08-19T01:44:32.049153+0000 (oldest deadline 2024-08-19T01:44:53.749126+0000) 2024-08-19T01:45:11.216 INFO:journalctl@ceph.osd.8.smithi167.stdout:Aug 19 01:45:11 smithi167 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-8[39928]: 2024-08-19T01:45:11.134+0000 7f17fe820640 -1 osd.8 89 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-08-19T01:44:36.050034+0000 front 2024-08-19T01:44:36.049973+0000 (oldest deadline 2024-08-19T01:45:06.050849+0000) 2024-08-19T01:45:11.216 INFO:journalctl@ceph.osd.11.smithi167.stdout:Aug 19 01:45:10 smithi167 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-11[46643]: 2024-08-19T01:45:10.915+0000 7f447f624640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-08-19T01:44:22.880210+0000 front 2024-08-19T01:44:22.880404+0000 (oldest deadline 2024-08-19T01:44:45.780352+0000) 2024-08-19T01:45:11.216 INFO:journalctl@ceph.osd.11.smithi167.stdout:Aug 19 01:45:10 smithi167 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-11[46643]: 2024-08-19T01:45:10.915+0000 7f447f624640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-08-19T01:44:31.081546+0000 front 2024-08-19T01:44:31.081654+0000 (oldest deadline 2024-08-19T01:44:53.981595+0000) 2024-08-19T01:45:11.216 INFO:journalctl@ceph.osd.11.smithi167.stdout:Aug 19 01:45:10 smithi167 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-11[46643]: 2024-08-19T01:45:10.915+0000 7f447f624640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-08-19T01:44:39.282638+0000 front 2024-08-19T01:44:39.282723+0000 (oldest deadline 2024-08-19T01:45:04.582709+0000) 2024-08-19T01:45:11.216 INFO:journalctl@ceph.osd.11.smithi167.stdout:Aug 19 01:45:10 smithi167 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-11[46643]: 2024-08-19T01:45:10.915+0000 7f447f624640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.83:6814 osd.3 since back 2024-08-19T01:44:45.083762+0000 front 2024-08-19T01:44:45.083777+0000 (oldest deadline 2024-08-19T01:45:10.383738+0000) 2024-08-19T01:45:11.452 INFO:journalctl@ceph.osd.7.smithi117.stdout:Aug 19 01:45:11 smithi117 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-7[39667]: 2024-08-19T01:45:11.125+0000 7f66eaeae640 -1 osd.7 89 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-08-19T01:44:31.724600+0000 front 2024-08-19T01:44:31.724641+0000 (oldest deadline 2024-08-19T01:44:54.024721+0000) 2024-08-19T01:45:11.452 INFO:journalctl@ceph.osd.7.smithi117.stdout:Aug 19 01:45:11 smithi117 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-7[39667]: 2024-08-19T01:45:11.125+0000 7f66eaeae640 -1 osd.7 89 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-08-19T01:44:40.426018+0000 front 2024-08-19T01:44:40.426002+0000 (oldest deadline 2024-08-19T01:45:01.526013+0000) 2024-08-19T01:45:11.452 INFO:journalctl@ceph.osd.7.smithi117.stdout:Aug 19 01:45:11 smithi117 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-7[39667]: 2024-08-19T01:45:11.125+0000 7f66eaeae640 -1 osd.7 89 heartbeat_check: no reply from 172.21.15.83:6814 osd.3 since back 2024-08-19T01:44:45.627330+0000 front 2024-08-19T01:44:45.627160+0000 (oldest deadline 2024-08-19T01:45:09.727143+0000) 2024-08-19T01:45:11.654 INFO:journalctl@ceph.osd.9.smithi083.stdout:Aug 19 01:45:11 smithi083 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-9[51472]: 2024-08-19T01:45:11.333+0000 7f54e8c00640 -1 osd.9 89 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-08-19T01:44:20.873282+0000 front 2024-08-19T01:44:20.873259+0000 (oldest deadline 2024-08-19T01:44:46.773413+0000) 2024-08-19T01:45:11.654 INFO:journalctl@ceph.osd.9.smithi083.stdout:Aug 19 01:45:11 smithi083 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-9[51472]: 2024-08-19T01:45:11.333+0000 7f54e8c00640 -1 osd.9 89 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-08-19T01:44:26.774104+0000 front 2024-08-19T01:44:26.774036+0000 (oldest deadline 2024-08-19T01:44:53.774823+0000) 2024-08-19T01:45:11.654 INFO:journalctl@ceph.osd.9.smithi083.stdout:Aug 19 01:45:11 smithi083 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-9[51472]: 2024-08-19T01:45:11.333+0000 7f54e8c00640 -1 osd.9 89 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-08-19T01:44:39.676617+0000 front 2024-08-19T01:44:39.676511+0000 (oldest deadline 2024-08-19T01:45:03.776684+0000) 2024-08-19T01:45:11.952 INFO:journalctl@ceph.osd.10.smithi117.stdout:Aug 19 01:45:11 smithi117 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-10[46616]: 2024-08-19T01:45:11.603+0000 7f34b3bf7640 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-08-19T01:44:24.060751+0000 front 2024-08-19T01:44:24.060716+0000 (oldest deadline 2024-08-19T01:44:47.560677+0000) 2024-08-19T01:45:11.952 INFO:journalctl@ceph.osd.10.smithi117.stdout:Aug 19 01:45:11 smithi117 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-10[46616]: 2024-08-19T01:45:11.603+0000 7f34b3bf7640 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-08-19T01:44:28.661503+0000 front 2024-08-19T01:44:28.661522+0000 (oldest deadline 2024-08-19T01:44:56.262298+0000) 2024-08-19T01:45:11.952 INFO:journalctl@ceph.osd.10.smithi117.stdout:Aug 19 01:45:11 smithi117 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-10[46616]: 2024-08-19T01:45:11.603+0000 7f34b3bf7640 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-08-19T01:44:38.563674+0000 front 2024-08-19T01:44:38.563520+0000 (oldest deadline 2024-08-19T01:45:02.063517+0000) 2024-08-19T01:45:11.952 INFO:journalctl@ceph.osd.10.smithi117.stdout:Aug 19 01:45:11 smithi117 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-10[46616]: 2024-08-19T01:45:11.603+0000 7f34b3bf7640 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.83:6814 osd.3 since back 2024-08-19T01:44:47.265176+0000 front 2024-08-19T01:44:47.265102+0000 (oldest deadline 2024-08-19T01:45:09.564999+0000) 2024-08-19T01:45:12.154 INFO:journalctl@ceph.osd.6.smithi083.stdout:Aug 19 01:45:11 smithi083 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-6[44809]: 2024-08-19T01:45:11.886+0000 7f3cc6e00640 -1 osd.6 89 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-08-19T01:44:21.901735+0000 front 2024-08-19T01:44:21.901860+0000 (oldest deadline 2024-08-19T01:44:47.801737+0000) 2024-08-19T01:45:12.154 INFO:journalctl@ceph.osd.6.smithi083.stdout:Aug 19 01:45:11 smithi083 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-6[44809]: 2024-08-19T01:45:11.886+0000 7f3cc6e00640 -1 osd.6 89 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-08-19T01:44:28.303418+0000 front 2024-08-19T01:44:28.303477+0000 (oldest deadline 2024-08-19T01:44:55.904426+0000) 2024-08-19T01:45:12.155 INFO:journalctl@ceph.osd.6.smithi083.stdout:Aug 19 01:45:11 smithi083 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-6[44809]: 2024-08-19T01:45:11.886+0000 7f3cc6e00640 -1 osd.6 89 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-08-19T01:44:37.506923+0000 front 2024-08-19T01:44:37.507030+0000 (oldest deadline 2024-08-19T01:45:01.606759+0000) 2024-08-19T01:45:12.155 INFO:journalctl@ceph.osd.6.smithi083.stdout:Aug 19 01:45:11 smithi083 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-6[44809]: 2024-08-19T01:45:11.886+0000 7f3cc6e00640 -1 osd.6 89 heartbeat_check: no reply from 172.21.15.83:6814 osd.3 since back 2024-08-19T01:44:44.308698+0000 front 2024-08-19T01:44:44.308741+0000 (oldest deadline 2024-08-19T01:45:08.909353+0000) 2024-08-19T01:45:12.216 INFO:journalctl@ceph.osd.8.smithi167.stdout:Aug 19 01:45:12 smithi167 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-8[39928]: 2024-08-19T01:45:12.085+0000 7f17fe820640 -1 osd.8 89 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-08-19T01:44:22.747507+0000 front 2024-08-19T01:44:22.747516+0000 (oldest deadline 2024-08-19T01:44:48.047324+0000) 2024-08-19T01:45:12.216 INFO:journalctl@ceph.osd.8.smithi167.stdout:Aug 19 01:45:12 smithi167 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-8[39928]: 2024-08-19T01:45:12.085+0000 7f17fe820640 -1 osd.8 89 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-08-19T01:44:32.049249+0000 front 2024-08-19T01:44:32.049153+0000 (oldest deadline 2024-08-19T01:44:53.749126+0000) 2024-08-19T01:45:12.216 INFO:journalctl@ceph.osd.8.smithi167.stdout:Aug 19 01:45:12 smithi167 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-8[39928]: 2024-08-19T01:45:12.085+0000 7f17fe820640 -1 osd.8 89 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-08-19T01:44:36.050034+0000 front 2024-08-19T01:44:36.049973+0000 (oldest deadline 2024-08-19T01:45:06.050849+0000) 2024-08-19T01:45:12.216 INFO:journalctl@ceph.osd.11.smithi167.stdout:Aug 19 01:45:11 smithi167 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-11[46643]: 2024-08-19T01:45:11.876+0000 7f447f624640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-08-19T01:44:22.880210+0000 front 2024-08-19T01:44:22.880404+0000 (oldest deadline 2024-08-19T01:44:45.780352+0000) 2024-08-19T01:45:12.216 INFO:journalctl@ceph.osd.11.smithi167.stdout:Aug 19 01:45:11 smithi167 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-11[46643]: 2024-08-19T01:45:11.876+0000 7f447f624640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-08-19T01:44:31.081546+0000 front 2024-08-19T01:44:31.081654+0000 (oldest deadline 2024-08-19T01:44:53.981595+0000) 2024-08-19T01:45:12.216 INFO:journalctl@ceph.osd.11.smithi167.stdout:Aug 19 01:45:11 smithi167 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-11[46643]: 2024-08-19T01:45:11.876+0000 7f447f624640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-08-19T01:44:39.282638+0000 front 2024-08-19T01:44:39.282723+0000 (oldest deadline 2024-08-19T01:45:04.582709+0000) 2024-08-19T01:45:12.216 INFO:journalctl@ceph.osd.11.smithi167.stdout:Aug 19 01:45:11 smithi167 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-11[46643]: 2024-08-19T01:45:11.876+0000 7f447f624640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.83:6814 osd.3 since back 2024-08-19T01:44:45.083762+0000 front 2024-08-19T01:44:45.083777+0000 (oldest deadline 2024-08-19T01:45:10.383738+0000) 2024-08-19T01:45:12.452 INFO:journalctl@ceph.osd.7.smithi117.stdout:Aug 19 01:45:12 smithi117 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-7[39667]: 2024-08-19T01:45:12.144+0000 7f66eaeae640 -1 osd.7 89 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-08-19T01:44:31.724600+0000 front 2024-08-19T01:44:31.724641+0000 (oldest deadline 2024-08-19T01:44:54.024721+0000) 2024-08-19T01:45:12.505 INFO:journalctl@ceph.osd.7.smithi117.stdout:Aug 19 01:45:12 smithi117 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-7[39667]: 2024-08-19T01:45:12.144+0000 7f66eaeae640 -1 osd.7 89 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-08-19T01:44:40.426018+0000 front 2024-08-19T01:44:40.426002+0000 (oldest deadline 2024-08-19T01:45:01.526013+0000) 2024-08-19T01:45:12.506 INFO:journalctl@ceph.osd.7.smithi117.stdout:Aug 19 01:45:12 smithi117 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-7[39667]: 2024-08-19T01:45:12.144+0000 7f66eaeae640 -1 osd.7 89 heartbeat_check: no reply from 172.21.15.83:6814 osd.3 since back 2024-08-19T01:44:45.627330+0000 front 2024-08-19T01:44:45.627160+0000 (oldest deadline 2024-08-19T01:45:09.727143+0000) 2024-08-19T01:45:12.654 INFO:journalctl@ceph.osd.9.smithi083.stdout:Aug 19 01:45:12 smithi083 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-9[51472]: 2024-08-19T01:45:12.331+0000 7f54e8c00640 -1 osd.9 89 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-08-19T01:44:20.873282+0000 front 2024-08-19T01:44:20.873259+0000 (oldest deadline 2024-08-19T01:44:46.773413+0000) 2024-08-19T01:45:12.655 INFO:journalctl@ceph.osd.9.smithi083.stdout:Aug 19 01:45:12 smithi083 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-9[51472]: 2024-08-19T01:45:12.331+0000 7f54e8c00640 -1 osd.9 89 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-08-19T01:44:26.774104+0000 front 2024-08-19T01:44:26.774036+0000 (oldest deadline 2024-08-19T01:44:53.774823+0000) 2024-08-19T01:45:12.655 INFO:journalctl@ceph.osd.9.smithi083.stdout:Aug 19 01:45:12 smithi083 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-9[51472]: 2024-08-19T01:45:12.331+0000 7f54e8c00640 -1 osd.9 89 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-08-19T01:44:39.676617+0000 front 2024-08-19T01:44:39.676511+0000 (oldest deadline 2024-08-19T01:45:03.776684+0000) 2024-08-19T01:45:12.655 INFO:journalctl@ceph.osd.9.smithi083.stdout:Aug 19 01:45:12 smithi083 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-9[51472]: 2024-08-19T01:45:12.331+0000 7f54e8c00640 -1 osd.9 89 heartbeat_check: no reply from 172.21.15.83:6814 osd.3 since back 2024-08-19T01:44:46.678051+0000 front 2024-08-19T01:44:46.678168+0000 (oldest deadline 2024-08-19T01:45:11.378113+0000) 2024-08-19T01:45:12.952 INFO:journalctl@ceph.osd.10.smithi117.stdout:Aug 19 01:45:12 smithi117 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-10[46616]: 2024-08-19T01:45:12.559+0000 7f34b3bf7640 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-08-19T01:44:24.060751+0000 front 2024-08-19T01:44:24.060716+0000 (oldest deadline 2024-08-19T01:44:47.560677+0000) 2024-08-19T01:45:12.952 INFO:journalctl@ceph.osd.10.smithi117.stdout:Aug 19 01:45:12 smithi117 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-10[46616]: 2024-08-19T01:45:12.559+0000 7f34b3bf7640 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-08-19T01:44:28.661503+0000 front 2024-08-19T01:44:28.661522+0000 (oldest deadline 2024-08-19T01:44:56.262298+0000) 2024-08-19T01:45:12.953 INFO:journalctl@ceph.osd.10.smithi117.stdout:Aug 19 01:45:12 smithi117 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-10[46616]: 2024-08-19T01:45:12.559+0000 7f34b3bf7640 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-08-19T01:44:38.563674+0000 front 2024-08-19T01:44:38.563520+0000 (oldest deadline 2024-08-19T01:45:02.063517+0000) 2024-08-19T01:45:12.953 INFO:journalctl@ceph.osd.10.smithi117.stdout:Aug 19 01:45:12 smithi117 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-10[46616]: 2024-08-19T01:45:12.559+0000 7f34b3bf7640 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.83:6814 osd.3 since back 2024-08-19T01:44:47.265176+0000 front 2024-08-19T01:44:47.265102+0000 (oldest deadline 2024-08-19T01:45:09.564999+0000) 2024-08-19T01:45:13.107 INFO:journalctl@ceph.osd.6.smithi083.stdout:Aug 19 01:45:12 smithi083 podman[84791]: 2024-08-19 01:45:12.815897602 +0000 UTC m=+5.695395568 container died 1a4817db6ba32959430364cef20be89602cc04665cfa5721f952adfa5853e6d7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c9dd28080a631be25c77c36c63b17a3ef340ffb2e21c87d4d7b56b6398436d9, name=ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-6, ceph=True, RELEASE=squid-de09dd0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, io.buildah.version=1.36.0, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=) 2024-08-19T01:45:13.132 INFO:journalctl@ceph.osd.11.smithi167.stdout:Aug 19 01:45:12 smithi167 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-11[46643]: 2024-08-19T01:45:12.856+0000 7f447f624640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-08-19T01:44:22.880210+0000 front 2024-08-19T01:44:22.880404+0000 (oldest deadline 2024-08-19T01:44:45.780352+0000) 2024-08-19T01:45:13.132 INFO:journalctl@ceph.osd.11.smithi167.stdout:Aug 19 01:45:12 smithi167 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-11[46643]: 2024-08-19T01:45:12.856+0000 7f447f624640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-08-19T01:44:31.081546+0000 front 2024-08-19T01:44:31.081654+0000 (oldest deadline 2024-08-19T01:44:53.981595+0000) 2024-08-19T01:45:13.132 INFO:journalctl@ceph.osd.11.smithi167.stdout:Aug 19 01:45:12 smithi167 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-11[46643]: 2024-08-19T01:45:12.856+0000 7f447f624640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-08-19T01:44:39.282638+0000 front 2024-08-19T01:44:39.282723+0000 (oldest deadline 2024-08-19T01:45:04.582709+0000) 2024-08-19T01:45:13.132 INFO:journalctl@ceph.osd.11.smithi167.stdout:Aug 19 01:45:12 smithi167 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-11[46643]: 2024-08-19T01:45:12.856+0000 7f447f624640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.83:6814 osd.3 since back 2024-08-19T01:44:45.083762+0000 front 2024-08-19T01:44:45.083777+0000 (oldest deadline 2024-08-19T01:45:10.383738+0000) 2024-08-19T01:45:13.404 INFO:journalctl@ceph.osd.6.smithi083.stdout:Aug 19 01:45:13 smithi083 podman[84791]: 2024-08-19 01:45:13.255062605 +0000 UTC m=+6.134560569 container cleanup 1a4817db6ba32959430364cef20be89602cc04665cfa5721f952adfa5853e6d7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c9dd28080a631be25c77c36c63b17a3ef340ffb2e21c87d4d7b56b6398436d9, name=ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-6, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, ceph=True, io.buildah.version=1.36.0, RELEASE=squid-de09dd0, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0) 2024-08-19T01:45:13.404 INFO:journalctl@ceph.osd.6.smithi083.stdout:Aug 19 01:45:13 smithi083 bash[84791]: ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-6 2024-08-19T01:45:13.405 INFO:journalctl@ceph.osd.9.smithi083.stdout:Aug 19 01:45:13 smithi083 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-9[51472]: 2024-08-19T01:45:13.314+0000 7f54e8c00640 -1 osd.9 89 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-08-19T01:44:20.873282+0000 front 2024-08-19T01:44:20.873259+0000 (oldest deadline 2024-08-19T01:44:46.773413+0000) 2024-08-19T01:45:13.405 INFO:journalctl@ceph.osd.9.smithi083.stdout:Aug 19 01:45:13 smithi083 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-9[51472]: 2024-08-19T01:45:13.314+0000 7f54e8c00640 -1 osd.9 89 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-08-19T01:44:26.774104+0000 front 2024-08-19T01:44:26.774036+0000 (oldest deadline 2024-08-19T01:44:53.774823+0000) 2024-08-19T01:45:13.405 INFO:journalctl@ceph.osd.9.smithi083.stdout:Aug 19 01:45:13 smithi083 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-9[51472]: 2024-08-19T01:45:13.314+0000 7f54e8c00640 -1 osd.9 89 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-08-19T01:44:39.676617+0000 front 2024-08-19T01:44:39.676511+0000 (oldest deadline 2024-08-19T01:45:03.776684+0000) 2024-08-19T01:45:13.405 INFO:journalctl@ceph.osd.9.smithi083.stdout:Aug 19 01:45:13 smithi083 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-9[51472]: 2024-08-19T01:45:13.314+0000 7f54e8c00640 -1 osd.9 89 heartbeat_check: no reply from 172.21.15.83:6814 osd.3 since back 2024-08-19T01:44:46.678051+0000 front 2024-08-19T01:44:46.678168+0000 (oldest deadline 2024-08-19T01:45:11.378113+0000) 2024-08-19T01:45:13.452 INFO:journalctl@ceph.osd.7.smithi117.stdout:Aug 19 01:45:13 smithi117 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-7[39667]: 2024-08-19T01:45:13.116+0000 7f66eaeae640 -1 osd.7 89 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-08-19T01:44:31.724600+0000 front 2024-08-19T01:44:31.724641+0000 (oldest deadline 2024-08-19T01:44:54.024721+0000) 2024-08-19T01:45:13.452 INFO:journalctl@ceph.osd.7.smithi117.stdout:Aug 19 01:45:13 smithi117 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-7[39667]: 2024-08-19T01:45:13.116+0000 7f66eaeae640 -1 osd.7 89 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-08-19T01:44:40.426018+0000 front 2024-08-19T01:44:40.426002+0000 (oldest deadline 2024-08-19T01:45:01.526013+0000) 2024-08-19T01:45:13.452 INFO:journalctl@ceph.osd.7.smithi117.stdout:Aug 19 01:45:13 smithi117 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-7[39667]: 2024-08-19T01:45:13.116+0000 7f66eaeae640 -1 osd.7 89 heartbeat_check: no reply from 172.21.15.83:6814 osd.3 since back 2024-08-19T01:44:45.627330+0000 front 2024-08-19T01:44:45.627160+0000 (oldest deadline 2024-08-19T01:45:09.727143+0000) 2024-08-19T01:45:13.466 INFO:journalctl@ceph.osd.8.smithi167.stdout:Aug 19 01:45:13 smithi167 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-8[39928]: 2024-08-19T01:45:13.131+0000 7f17fe820640 -1 osd.8 89 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-08-19T01:44:22.747507+0000 front 2024-08-19T01:44:22.747516+0000 (oldest deadline 2024-08-19T01:44:48.047324+0000) 2024-08-19T01:45:13.466 INFO:journalctl@ceph.osd.8.smithi167.stdout:Aug 19 01:45:13 smithi167 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-8[39928]: 2024-08-19T01:45:13.131+0000 7f17fe820640 -1 osd.8 89 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-08-19T01:44:32.049249+0000 front 2024-08-19T01:44:32.049153+0000 (oldest deadline 2024-08-19T01:44:53.749126+0000) 2024-08-19T01:45:13.466 INFO:journalctl@ceph.osd.8.smithi167.stdout:Aug 19 01:45:13 smithi167 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-8[39928]: 2024-08-19T01:45:13.131+0000 7f17fe820640 -1 osd.8 89 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-08-19T01:44:36.050034+0000 front 2024-08-19T01:44:36.049973+0000 (oldest deadline 2024-08-19T01:45:06.050849+0000) 2024-08-19T01:45:13.904 INFO:journalctl@ceph.osd.6.smithi083.stdout:Aug 19 01:45:13 smithi083 podman[84808]: 2024-08-19 01:45:13.464863618 +0000 UTC m=+0.638169400 container remove 1a4817db6ba32959430364cef20be89602cc04665cfa5721f952adfa5853e6d7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c9dd28080a631be25c77c36c63b17a3ef340ffb2e21c87d4d7b56b6398436d9, name=ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-6, GIT_BRANCH=HEAD, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.36.0, RELEASE=squid-de09dd0, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731) 2024-08-19T01:45:13.952 INFO:journalctl@ceph.osd.10.smithi117.stdout:Aug 19 01:45:13 smithi117 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-10[46616]: 2024-08-19T01:45:13.513+0000 7f34b3bf7640 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-08-19T01:44:24.060751+0000 front 2024-08-19T01:44:24.060716+0000 (oldest deadline 2024-08-19T01:44:47.560677+0000) 2024-08-19T01:45:13.952 INFO:journalctl@ceph.osd.10.smithi117.stdout:Aug 19 01:45:13 smithi117 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-10[46616]: 2024-08-19T01:45:13.513+0000 7f34b3bf7640 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-08-19T01:44:28.661503+0000 front 2024-08-19T01:44:28.661522+0000 (oldest deadline 2024-08-19T01:44:56.262298+0000) 2024-08-19T01:45:13.952 INFO:journalctl@ceph.osd.10.smithi117.stdout:Aug 19 01:45:13 smithi117 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-10[46616]: 2024-08-19T01:45:13.513+0000 7f34b3bf7640 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-08-19T01:44:38.563674+0000 front 2024-08-19T01:44:38.563520+0000 (oldest deadline 2024-08-19T01:45:02.063517+0000) 2024-08-19T01:45:13.952 INFO:journalctl@ceph.osd.10.smithi117.stdout:Aug 19 01:45:13 smithi117 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-10[46616]: 2024-08-19T01:45:13.513+0000 7f34b3bf7640 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.83:6814 osd.3 since back 2024-08-19T01:44:47.265176+0000 front 2024-08-19T01:44:47.265102+0000 (oldest deadline 2024-08-19T01:45:09.564999+0000) 2024-08-19T01:45:14.174 INFO:journalctl@ceph.osd.11.smithi167.stdout:Aug 19 01:45:13 smithi167 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-11[46643]: 2024-08-19T01:45:13.881+0000 7f447f624640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-08-19T01:44:22.880210+0000 front 2024-08-19T01:44:22.880404+0000 (oldest deadline 2024-08-19T01:44:45.780352+0000) 2024-08-19T01:45:14.174 INFO:journalctl@ceph.osd.11.smithi167.stdout:Aug 19 01:45:13 smithi167 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-11[46643]: 2024-08-19T01:45:13.881+0000 7f447f624640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-08-19T01:44:31.081546+0000 front 2024-08-19T01:44:31.081654+0000 (oldest deadline 2024-08-19T01:44:53.981595+0000) 2024-08-19T01:45:14.174 INFO:journalctl@ceph.osd.11.smithi167.stdout:Aug 19 01:45:13 smithi167 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-11[46643]: 2024-08-19T01:45:13.881+0000 7f447f624640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-08-19T01:44:39.282638+0000 front 2024-08-19T01:44:39.282723+0000 (oldest deadline 2024-08-19T01:45:04.582709+0000) 2024-08-19T01:45:14.174 INFO:journalctl@ceph.osd.11.smithi167.stdout:Aug 19 01:45:13 smithi167 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-11[46643]: 2024-08-19T01:45:13.881+0000 7f447f624640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.83:6814 osd.3 since back 2024-08-19T01:44:45.083762+0000 front 2024-08-19T01:44:45.083777+0000 (oldest deadline 2024-08-19T01:45:10.383738+0000) 2024-08-19T01:45:14.294 INFO:journalctl@ceph.osd.6.smithi083.stdout:Aug 19 01:45:14 smithi083 podman[84897]: 2024-08-19 01:45:13.920216424 +0000 UTC m=+0.029531134 image pull ea7a5189f30bad3aca9524788e2965d2368d7ce786df09d4edcd8c3edf9cefba quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c9dd28080a631be25c77c36c63b17a3ef340ffb2e21c87d4d7b56b6398436d9 2024-08-19T01:45:14.295 INFO:journalctl@ceph.osd.6.smithi083.stdout:Aug 19 01:45:14 smithi083 podman[84897]: 2024-08-19 01:45:14.091145701 +0000 UTC m=+0.200460395 container create 449ef5231fbebecdfba5722ea75b8e25e487159ee659f2183cb211f50520ec26 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c9dd28080a631be25c77c36c63b17a3ef340ffb2e21c87d4d7b56b6398436d9, name=ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-6-deactivate, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.36.0, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-de09dd0, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.license=GPLv2) 2024-08-19T01:45:14.452 INFO:journalctl@ceph.osd.7.smithi117.stdout:Aug 19 01:45:14 smithi117 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-7[39667]: 2024-08-19T01:45:14.128+0000 7f66eaeae640 -1 osd.7 89 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-08-19T01:44:31.724600+0000 front 2024-08-19T01:44:31.724641+0000 (oldest deadline 2024-08-19T01:44:54.024721+0000) 2024-08-19T01:45:14.452 INFO:journalctl@ceph.osd.7.smithi117.stdout:Aug 19 01:45:14 smithi117 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-7[39667]: 2024-08-19T01:45:14.128+0000 7f66eaeae640 -1 osd.7 89 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-08-19T01:44:40.426018+0000 front 2024-08-19T01:44:40.426002+0000 (oldest deadline 2024-08-19T01:45:01.526013+0000) 2024-08-19T01:45:14.452 INFO:journalctl@ceph.osd.7.smithi117.stdout:Aug 19 01:45:14 smithi117 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-7[39667]: 2024-08-19T01:45:14.128+0000 7f66eaeae640 -1 osd.7 89 heartbeat_check: no reply from 172.21.15.83:6814 osd.3 since back 2024-08-19T01:44:45.627330+0000 front 2024-08-19T01:44:45.627160+0000 (oldest deadline 2024-08-19T01:45:09.727143+0000) 2024-08-19T01:45:14.465 INFO:journalctl@ceph.osd.8.smithi167.stdout:Aug 19 01:45:14 smithi167 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-8[39928]: 2024-08-19T01:45:14.173+0000 7f17fe820640 -1 osd.8 89 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-08-19T01:44:22.747507+0000 front 2024-08-19T01:44:22.747516+0000 (oldest deadline 2024-08-19T01:44:48.047324+0000) 2024-08-19T01:45:14.466 INFO:journalctl@ceph.osd.8.smithi167.stdout:Aug 19 01:45:14 smithi167 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-8[39928]: 2024-08-19T01:45:14.173+0000 7f17fe820640 -1 osd.8 89 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-08-19T01:44:32.049249+0000 front 2024-08-19T01:44:32.049153+0000 (oldest deadline 2024-08-19T01:44:53.749126+0000) 2024-08-19T01:45:14.466 INFO:journalctl@ceph.osd.8.smithi167.stdout:Aug 19 01:45:14 smithi167 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-8[39928]: 2024-08-19T01:45:14.173+0000 7f17fe820640 -1 osd.8 89 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-08-19T01:44:36.050034+0000 front 2024-08-19T01:44:36.049973+0000 (oldest deadline 2024-08-19T01:45:06.050849+0000) 2024-08-19T01:45:14.552 INFO:journalctl@ceph.osd.9.smithi083.stdout:Aug 19 01:45:14 smithi083 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-9[51472]: 2024-08-19T01:45:14.339+0000 7f54e8c00640 -1 osd.9 89 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-08-19T01:44:20.873282+0000 front 2024-08-19T01:44:20.873259+0000 (oldest deadline 2024-08-19T01:44:46.773413+0000) 2024-08-19T01:45:14.552 INFO:journalctl@ceph.osd.9.smithi083.stdout:Aug 19 01:45:14 smithi083 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-9[51472]: 2024-08-19T01:45:14.339+0000 7f54e8c00640 -1 osd.9 89 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-08-19T01:44:26.774104+0000 front 2024-08-19T01:44:26.774036+0000 (oldest deadline 2024-08-19T01:44:53.774823+0000) 2024-08-19T01:45:14.552 INFO:journalctl@ceph.osd.9.smithi083.stdout:Aug 19 01:45:14 smithi083 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-9[51472]: 2024-08-19T01:45:14.339+0000 7f54e8c00640 -1 osd.9 89 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-08-19T01:44:39.676617+0000 front 2024-08-19T01:44:39.676511+0000 (oldest deadline 2024-08-19T01:45:03.776684+0000) 2024-08-19T01:45:14.552 INFO:journalctl@ceph.osd.9.smithi083.stdout:Aug 19 01:45:14 smithi083 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-9[51472]: 2024-08-19T01:45:14.339+0000 7f54e8c00640 -1 osd.9 89 heartbeat_check: no reply from 172.21.15.83:6814 osd.3 since back 2024-08-19T01:44:46.678051+0000 front 2024-08-19T01:44:46.678168+0000 (oldest deadline 2024-08-19T01:45:11.378113+0000) 2024-08-19T01:45:14.552 INFO:journalctl@ceph.osd.6.smithi083.stdout:Aug 19 01:45:14 smithi083 podman[84897]: 2024-08-19 01:45:14.294387493 +0000 UTC m=+0.403702193 container init 449ef5231fbebecdfba5722ea75b8e25e487159ee659f2183cb211f50520ec26 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c9dd28080a631be25c77c36c63b17a3ef340ffb2e21c87d4d7b56b6398436d9, name=ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-6-deactivate, GIT_BRANCH=HEAD, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, RELEASE=squid-de09dd0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, io.buildah.version=1.36.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, ceph=True) 2024-08-19T01:45:14.553 INFO:journalctl@ceph.osd.6.smithi083.stdout:Aug 19 01:45:14 smithi083 podman[84897]: 2024-08-19 01:45:14.301150378 +0000 UTC m=+0.410465077 container start 449ef5231fbebecdfba5722ea75b8e25e487159ee659f2183cb211f50520ec26 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c9dd28080a631be25c77c36c63b17a3ef340ffb2e21c87d4d7b56b6398436d9, name=ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-6-deactivate, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, RELEASE=squid-de09dd0, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, ceph=True, maintainer=Guillaume Abrioux , io.buildah.version=1.36.0, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2) 2024-08-19T01:45:14.553 INFO:journalctl@ceph.osd.6.smithi083.stdout:Aug 19 01:45:14 smithi083 podman[84897]: 2024-08-19 01:45:14.404630854 +0000 UTC m=+0.513945554 container attach 449ef5231fbebecdfba5722ea75b8e25e487159ee659f2183cb211f50520ec26 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c9dd28080a631be25c77c36c63b17a3ef340ffb2e21c87d4d7b56b6398436d9, name=ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-6-deactivate, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, ceph=True, RELEASE=squid-de09dd0, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.36.0, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2) 2024-08-19T01:45:14.553 INFO:journalctl@ceph.osd.6.smithi083.stdout:Aug 19 01:45:14 smithi083 podman[84897]: 2024-08-19 01:45:14.551856324 +0000 UTC m=+0.661171032 container died 449ef5231fbebecdfba5722ea75b8e25e487159ee659f2183cb211f50520ec26 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c9dd28080a631be25c77c36c63b17a3ef340ffb2e21c87d4d7b56b6398436d9, name=ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-6-deactivate, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_CLEAN=True, RELEASE=squid-de09dd0, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, ceph=True, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, io.buildah.version=1.36.0) 2024-08-19T01:45:14.952 INFO:journalctl@ceph.osd.10.smithi117.stdout:Aug 19 01:45:14 smithi117 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-10[46616]: 2024-08-19T01:45:14.498+0000 7f34b3bf7640 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-08-19T01:44:24.060751+0000 front 2024-08-19T01:44:24.060716+0000 (oldest deadline 2024-08-19T01:44:47.560677+0000) 2024-08-19T01:45:14.953 INFO:journalctl@ceph.osd.10.smithi117.stdout:Aug 19 01:45:14 smithi117 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-10[46616]: 2024-08-19T01:45:14.498+0000 7f34b3bf7640 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-08-19T01:44:28.661503+0000 front 2024-08-19T01:44:28.661522+0000 (oldest deadline 2024-08-19T01:44:56.262298+0000) 2024-08-19T01:45:14.953 INFO:journalctl@ceph.osd.10.smithi117.stdout:Aug 19 01:45:14 smithi117 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-10[46616]: 2024-08-19T01:45:14.498+0000 7f34b3bf7640 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-08-19T01:44:38.563674+0000 front 2024-08-19T01:44:38.563520+0000 (oldest deadline 2024-08-19T01:45:02.063517+0000) 2024-08-19T01:45:14.953 INFO:journalctl@ceph.osd.10.smithi117.stdout:Aug 19 01:45:14 smithi117 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-10[46616]: 2024-08-19T01:45:14.498+0000 7f34b3bf7640 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.83:6814 osd.3 since back 2024-08-19T01:44:47.265176+0000 front 2024-08-19T01:44:47.265102+0000 (oldest deadline 2024-08-19T01:45:09.564999+0000) 2024-08-19T01:45:15.165 INFO:journalctl@ceph.osd.11.smithi167.stdout:Aug 19 01:45:14 smithi167 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-11[46643]: 2024-08-19T01:45:14.875+0000 7f447f624640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-08-19T01:44:22.880210+0000 front 2024-08-19T01:44:22.880404+0000 (oldest deadline 2024-08-19T01:44:45.780352+0000) 2024-08-19T01:45:15.165 INFO:journalctl@ceph.osd.11.smithi167.stdout:Aug 19 01:45:14 smithi167 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-11[46643]: 2024-08-19T01:45:14.875+0000 7f447f624640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-08-19T01:44:31.081546+0000 front 2024-08-19T01:44:31.081654+0000 (oldest deadline 2024-08-19T01:44:53.981595+0000) 2024-08-19T01:45:15.165 INFO:journalctl@ceph.osd.11.smithi167.stdout:Aug 19 01:45:14 smithi167 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-11[46643]: 2024-08-19T01:45:14.875+0000 7f447f624640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-08-19T01:44:39.282638+0000 front 2024-08-19T01:44:39.282723+0000 (oldest deadline 2024-08-19T01:45:04.582709+0000) 2024-08-19T01:45:15.165 INFO:journalctl@ceph.osd.11.smithi167.stdout:Aug 19 01:45:14 smithi167 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-11[46643]: 2024-08-19T01:45:14.875+0000 7f447f624640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.83:6814 osd.3 since back 2024-08-19T01:44:45.083762+0000 front 2024-08-19T01:44:45.083777+0000 (oldest deadline 2024-08-19T01:45:10.383738+0000) 2024-08-19T01:45:15.266 DEBUG:teuthology.orchestra.run.smithi083:> sudo pkill -f 'journalctl -f -n 0 -u ceph-bad8780e-5dc9-11ef-bccf-c7b262605968@osd.6.service' 2024-08-19T01:45:15.343 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-08-19T01:45:15.343 INFO:tasks.cephadm.osd.6:Stopped osd.6 2024-08-19T01:45:15.343 INFO:tasks.cephadm.osd.7:Stopping osd.7... 2024-08-19T01:45:15.344 DEBUG:teuthology.orchestra.run.smithi117:> sudo systemctl stop ceph-bad8780e-5dc9-11ef-bccf-c7b262605968@osd.7 2024-08-19T01:45:15.371 INFO:journalctl@ceph.osd.7.smithi117.stdout:Aug 19 01:45:15 smithi117 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-7[39667]: 2024-08-19T01:45:15.106+0000 7f66eaeae640 -1 osd.7 89 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-08-19T01:44:31.724600+0000 front 2024-08-19T01:44:31.724641+0000 (oldest deadline 2024-08-19T01:44:54.024721+0000) 2024-08-19T01:45:15.371 INFO:journalctl@ceph.osd.7.smithi117.stdout:Aug 19 01:45:15 smithi117 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-7[39667]: 2024-08-19T01:45:15.106+0000 7f66eaeae640 -1 osd.7 89 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-08-19T01:44:40.426018+0000 front 2024-08-19T01:44:40.426002+0000 (oldest deadline 2024-08-19T01:45:01.526013+0000) 2024-08-19T01:45:15.371 INFO:journalctl@ceph.osd.7.smithi117.stdout:Aug 19 01:45:15 smithi117 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-7[39667]: 2024-08-19T01:45:15.106+0000 7f66eaeae640 -1 osd.7 89 heartbeat_check: no reply from 172.21.15.83:6814 osd.3 since back 2024-08-19T01:44:45.627330+0000 front 2024-08-19T01:44:45.627160+0000 (oldest deadline 2024-08-19T01:45:09.727143+0000) 2024-08-19T01:45:15.466 INFO:journalctl@ceph.osd.8.smithi167.stdout:Aug 19 01:45:15 smithi167 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-8[39928]: 2024-08-19T01:45:15.164+0000 7f17fe820640 -1 osd.8 89 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-08-19T01:44:22.747507+0000 front 2024-08-19T01:44:22.747516+0000 (oldest deadline 2024-08-19T01:44:48.047324+0000) 2024-08-19T01:45:15.466 INFO:journalctl@ceph.osd.8.smithi167.stdout:Aug 19 01:45:15 smithi167 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-8[39928]: 2024-08-19T01:45:15.164+0000 7f17fe820640 -1 osd.8 89 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-08-19T01:44:32.049249+0000 front 2024-08-19T01:44:32.049153+0000 (oldest deadline 2024-08-19T01:44:53.749126+0000) 2024-08-19T01:45:15.466 INFO:journalctl@ceph.osd.8.smithi167.stdout:Aug 19 01:45:15 smithi167 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-8[39928]: 2024-08-19T01:45:15.164+0000 7f17fe820640 -1 osd.8 89 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-08-19T01:44:36.050034+0000 front 2024-08-19T01:44:36.049973+0000 (oldest deadline 2024-08-19T01:45:06.050849+0000) 2024-08-19T01:45:15.568 INFO:journalctl@ceph.osd.9.smithi083.stdout:Aug 19 01:45:15 smithi083 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-9[51472]: 2024-08-19T01:45:15.302+0000 7f54e8c00640 -1 osd.9 89 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-08-19T01:44:20.873282+0000 front 2024-08-19T01:44:20.873259+0000 (oldest deadline 2024-08-19T01:44:46.773413+0000) 2024-08-19T01:45:15.568 INFO:journalctl@ceph.osd.9.smithi083.stdout:Aug 19 01:45:15 smithi083 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-9[51472]: 2024-08-19T01:45:15.302+0000 7f54e8c00640 -1 osd.9 89 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-08-19T01:44:26.774104+0000 front 2024-08-19T01:44:26.774036+0000 (oldest deadline 2024-08-19T01:44:53.774823+0000) 2024-08-19T01:45:15.568 INFO:journalctl@ceph.osd.9.smithi083.stdout:Aug 19 01:45:15 smithi083 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-9[51472]: 2024-08-19T01:45:15.302+0000 7f54e8c00640 -1 osd.9 89 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-08-19T01:44:39.676617+0000 front 2024-08-19T01:44:39.676511+0000 (oldest deadline 2024-08-19T01:45:03.776684+0000) 2024-08-19T01:45:15.568 INFO:journalctl@ceph.osd.9.smithi083.stdout:Aug 19 01:45:15 smithi083 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-9[51472]: 2024-08-19T01:45:15.302+0000 7f54e8c00640 -1 osd.9 89 heartbeat_check: no reply from 172.21.15.83:6814 osd.3 since back 2024-08-19T01:44:46.678051+0000 front 2024-08-19T01:44:46.678168+0000 (oldest deadline 2024-08-19T01:45:11.378113+0000) 2024-08-19T01:45:15.647 INFO:journalctl@ceph.osd.7.smithi117.stdout:Aug 19 01:45:15 smithi117 systemd[1]: Stopping Ceph osd.7 for bad8780e-5dc9-11ef-bccf-c7b262605968... 2024-08-19T01:45:15.647 INFO:journalctl@ceph.osd.10.smithi117.stdout:Aug 19 01:45:15 smithi117 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-10[46616]: 2024-08-19T01:45:15.530+0000 7f34b3bf7640 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-08-19T01:44:24.060751+0000 front 2024-08-19T01:44:24.060716+0000 (oldest deadline 2024-08-19T01:44:47.560677+0000) 2024-08-19T01:45:15.647 INFO:journalctl@ceph.osd.10.smithi117.stdout:Aug 19 01:45:15 smithi117 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-10[46616]: 2024-08-19T01:45:15.530+0000 7f34b3bf7640 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-08-19T01:44:28.661503+0000 front 2024-08-19T01:44:28.661522+0000 (oldest deadline 2024-08-19T01:44:56.262298+0000) 2024-08-19T01:45:15.647 INFO:journalctl@ceph.osd.10.smithi117.stdout:Aug 19 01:45:15 smithi117 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-10[46616]: 2024-08-19T01:45:15.530+0000 7f34b3bf7640 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-08-19T01:44:38.563674+0000 front 2024-08-19T01:44:38.563520+0000 (oldest deadline 2024-08-19T01:45:02.063517+0000) 2024-08-19T01:45:15.647 INFO:journalctl@ceph.osd.10.smithi117.stdout:Aug 19 01:45:15 smithi117 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-10[46616]: 2024-08-19T01:45:15.530+0000 7f34b3bf7640 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.83:6814 osd.3 since back 2024-08-19T01:44:47.265176+0000 front 2024-08-19T01:44:47.265102+0000 (oldest deadline 2024-08-19T01:45:09.564999+0000) 2024-08-19T01:45:15.952 INFO:journalctl@ceph.osd.7.smithi117.stdout:Aug 19 01:45:15 smithi117 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-7[39667]: 2024-08-19T01:45:15.644+0000 7f66f20b0640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.7 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-08-19T01:45:15.953 INFO:journalctl@ceph.osd.7.smithi117.stdout:Aug 19 01:45:15 smithi117 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-7[39667]: 2024-08-19T01:45:15.644+0000 7f66f20b0640 -1 osd.7 89 *** Got signal Terminated *** 2024-08-19T01:45:15.953 INFO:journalctl@ceph.osd.7.smithi117.stdout:Aug 19 01:45:15 smithi117 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-7[39667]: 2024-08-19T01:45:15.644+0000 7f66f20b0640 -1 osd.7 89 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-08-19T01:45:16.198 INFO:journalctl@ceph.osd.11.smithi167.stdout:Aug 19 01:45:15 smithi167 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-11[46643]: 2024-08-19T01:45:15.830+0000 7f447f624640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-08-19T01:44:22.880210+0000 front 2024-08-19T01:44:22.880404+0000 (oldest deadline 2024-08-19T01:44:45.780352+0000) 2024-08-19T01:45:16.198 INFO:journalctl@ceph.osd.11.smithi167.stdout:Aug 19 01:45:15 smithi167 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-11[46643]: 2024-08-19T01:45:15.830+0000 7f447f624640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-08-19T01:44:31.081546+0000 front 2024-08-19T01:44:31.081654+0000 (oldest deadline 2024-08-19T01:44:53.981595+0000) 2024-08-19T01:45:16.198 INFO:journalctl@ceph.osd.11.smithi167.stdout:Aug 19 01:45:15 smithi167 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-11[46643]: 2024-08-19T01:45:15.830+0000 7f447f624640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-08-19T01:44:39.282638+0000 front 2024-08-19T01:44:39.282723+0000 (oldest deadline 2024-08-19T01:45:04.582709+0000) 2024-08-19T01:45:16.198 INFO:journalctl@ceph.osd.11.smithi167.stdout:Aug 19 01:45:15 smithi167 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-11[46643]: 2024-08-19T01:45:15.830+0000 7f447f624640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.83:6814 osd.3 since back 2024-08-19T01:44:45.083762+0000 front 2024-08-19T01:44:45.083777+0000 (oldest deadline 2024-08-19T01:45:10.383738+0000) 2024-08-19T01:45:16.452 INFO:journalctl@ceph.osd.7.smithi117.stdout:Aug 19 01:45:16 smithi117 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-7[39667]: 2024-08-19T01:45:16.099+0000 7f66eaeae640 -1 osd.7 89 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-08-19T01:44:31.724600+0000 front 2024-08-19T01:44:31.724641+0000 (oldest deadline 2024-08-19T01:44:54.024721+0000) 2024-08-19T01:45:16.452 INFO:journalctl@ceph.osd.7.smithi117.stdout:Aug 19 01:45:16 smithi117 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-7[39667]: 2024-08-19T01:45:16.099+0000 7f66eaeae640 -1 osd.7 89 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-08-19T01:44:40.426018+0000 front 2024-08-19T01:44:40.426002+0000 (oldest deadline 2024-08-19T01:45:01.526013+0000) 2024-08-19T01:45:16.452 INFO:journalctl@ceph.osd.7.smithi117.stdout:Aug 19 01:45:16 smithi117 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-7[39667]: 2024-08-19T01:45:16.099+0000 7f66eaeae640 -1 osd.7 89 heartbeat_check: no reply from 172.21.15.83:6814 osd.3 since back 2024-08-19T01:44:45.627330+0000 front 2024-08-19T01:44:45.627160+0000 (oldest deadline 2024-08-19T01:45:09.727143+0000) 2024-08-19T01:45:16.465 INFO:journalctl@ceph.osd.8.smithi167.stdout:Aug 19 01:45:16 smithi167 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-8[39928]: 2024-08-19T01:45:16.197+0000 7f17fe820640 -1 osd.8 89 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-08-19T01:44:22.747507+0000 front 2024-08-19T01:44:22.747516+0000 (oldest deadline 2024-08-19T01:44:48.047324+0000) 2024-08-19T01:45:16.465 INFO:journalctl@ceph.osd.8.smithi167.stdout:Aug 19 01:45:16 smithi167 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-8[39928]: 2024-08-19T01:45:16.197+0000 7f17fe820640 -1 osd.8 89 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-08-19T01:44:32.049249+0000 front 2024-08-19T01:44:32.049153+0000 (oldest deadline 2024-08-19T01:44:53.749126+0000) 2024-08-19T01:45:16.466 INFO:journalctl@ceph.osd.8.smithi167.stdout:Aug 19 01:45:16 smithi167 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-8[39928]: 2024-08-19T01:45:16.197+0000 7f17fe820640 -1 osd.8 89 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-08-19T01:44:36.050034+0000 front 2024-08-19T01:44:36.049973+0000 (oldest deadline 2024-08-19T01:45:06.050849+0000) 2024-08-19T01:45:16.648 INFO:journalctl@ceph.osd.9.smithi083.stdout:Aug 19 01:45:16 smithi083 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-9[51472]: 2024-08-19T01:45:16.327+0000 7f54e8c00640 -1 osd.9 89 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-08-19T01:44:20.873282+0000 front 2024-08-19T01:44:20.873259+0000 (oldest deadline 2024-08-19T01:44:46.773413+0000) 2024-08-19T01:45:16.649 INFO:journalctl@ceph.osd.9.smithi083.stdout:Aug 19 01:45:16 smithi083 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-9[51472]: 2024-08-19T01:45:16.327+0000 7f54e8c00640 -1 osd.9 89 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-08-19T01:44:26.774104+0000 front 2024-08-19T01:44:26.774036+0000 (oldest deadline 2024-08-19T01:44:53.774823+0000) 2024-08-19T01:45:16.649 INFO:journalctl@ceph.osd.9.smithi083.stdout:Aug 19 01:45:16 smithi083 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-9[51472]: 2024-08-19T01:45:16.327+0000 7f54e8c00640 -1 osd.9 89 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-08-19T01:44:39.676617+0000 front 2024-08-19T01:44:39.676511+0000 (oldest deadline 2024-08-19T01:45:03.776684+0000) 2024-08-19T01:45:16.649 INFO:journalctl@ceph.osd.9.smithi083.stdout:Aug 19 01:45:16 smithi083 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-9[51472]: 2024-08-19T01:45:16.327+0000 7f54e8c00640 -1 osd.9 89 heartbeat_check: no reply from 172.21.15.83:6814 osd.3 since back 2024-08-19T01:44:46.678051+0000 front 2024-08-19T01:44:46.678168+0000 (oldest deadline 2024-08-19T01:45:11.378113+0000) 2024-08-19T01:45:16.952 INFO:journalctl@ceph.osd.10.smithi117.stdout:Aug 19 01:45:16 smithi117 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-10[46616]: 2024-08-19T01:45:16.559+0000 7f34b3bf7640 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-08-19T01:44:24.060751+0000 front 2024-08-19T01:44:24.060716+0000 (oldest deadline 2024-08-19T01:44:47.560677+0000) 2024-08-19T01:45:16.952 INFO:journalctl@ceph.osd.10.smithi117.stdout:Aug 19 01:45:16 smithi117 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-10[46616]: 2024-08-19T01:45:16.559+0000 7f34b3bf7640 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-08-19T01:44:28.661503+0000 front 2024-08-19T01:44:28.661522+0000 (oldest deadline 2024-08-19T01:44:56.262298+0000) 2024-08-19T01:45:16.952 INFO:journalctl@ceph.osd.10.smithi117.stdout:Aug 19 01:45:16 smithi117 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-10[46616]: 2024-08-19T01:45:16.559+0000 7f34b3bf7640 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-08-19T01:44:38.563674+0000 front 2024-08-19T01:44:38.563520+0000 (oldest deadline 2024-08-19T01:45:02.063517+0000) 2024-08-19T01:45:16.952 INFO:journalctl@ceph.osd.10.smithi117.stdout:Aug 19 01:45:16 smithi117 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-10[46616]: 2024-08-19T01:45:16.559+0000 7f34b3bf7640 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.83:6814 osd.3 since back 2024-08-19T01:44:47.265176+0000 front 2024-08-19T01:44:47.265102+0000 (oldest deadline 2024-08-19T01:45:09.564999+0000) 2024-08-19T01:45:17.192 INFO:journalctl@ceph.osd.11.smithi167.stdout:Aug 19 01:45:16 smithi167 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-11[46643]: 2024-08-19T01:45:16.793+0000 7f447f624640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-08-19T01:44:22.880210+0000 front 2024-08-19T01:44:22.880404+0000 (oldest deadline 2024-08-19T01:44:45.780352+0000) 2024-08-19T01:45:17.192 INFO:journalctl@ceph.osd.11.smithi167.stdout:Aug 19 01:45:16 smithi167 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-11[46643]: 2024-08-19T01:45:16.793+0000 7f447f624640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-08-19T01:44:31.081546+0000 front 2024-08-19T01:44:31.081654+0000 (oldest deadline 2024-08-19T01:44:53.981595+0000) 2024-08-19T01:45:17.192 INFO:journalctl@ceph.osd.11.smithi167.stdout:Aug 19 01:45:16 smithi167 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-11[46643]: 2024-08-19T01:45:16.793+0000 7f447f624640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-08-19T01:44:39.282638+0000 front 2024-08-19T01:44:39.282723+0000 (oldest deadline 2024-08-19T01:45:04.582709+0000) 2024-08-19T01:45:17.192 INFO:journalctl@ceph.osd.11.smithi167.stdout:Aug 19 01:45:16 smithi167 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-11[46643]: 2024-08-19T01:45:16.793+0000 7f447f624640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.83:6814 osd.3 since back 2024-08-19T01:44:45.083762+0000 front 2024-08-19T01:44:45.083777+0000 (oldest deadline 2024-08-19T01:45:10.383738+0000) 2024-08-19T01:45:17.452 INFO:journalctl@ceph.osd.7.smithi117.stdout:Aug 19 01:45:17 smithi117 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-7[39667]: 2024-08-19T01:45:17.098+0000 7f66eaeae640 -1 osd.7 89 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-08-19T01:44:31.724600+0000 front 2024-08-19T01:44:31.724641+0000 (oldest deadline 2024-08-19T01:44:54.024721+0000) 2024-08-19T01:45:17.452 INFO:journalctl@ceph.osd.7.smithi117.stdout:Aug 19 01:45:17 smithi117 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-7[39667]: 2024-08-19T01:45:17.098+0000 7f66eaeae640 -1 osd.7 89 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-08-19T01:44:40.426018+0000 front 2024-08-19T01:44:40.426002+0000 (oldest deadline 2024-08-19T01:45:01.526013+0000) 2024-08-19T01:45:17.452 INFO:journalctl@ceph.osd.7.smithi117.stdout:Aug 19 01:45:17 smithi117 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-7[39667]: 2024-08-19T01:45:17.098+0000 7f66eaeae640 -1 osd.7 89 heartbeat_check: no reply from 172.21.15.83:6814 osd.3 since back 2024-08-19T01:44:45.627330+0000 front 2024-08-19T01:44:45.627160+0000 (oldest deadline 2024-08-19T01:45:09.727143+0000) 2024-08-19T01:45:17.466 INFO:journalctl@ceph.osd.8.smithi167.stdout:Aug 19 01:45:17 smithi167 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-8[39928]: 2024-08-19T01:45:17.191+0000 7f17fe820640 -1 osd.8 89 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-08-19T01:44:22.747507+0000 front 2024-08-19T01:44:22.747516+0000 (oldest deadline 2024-08-19T01:44:48.047324+0000) 2024-08-19T01:45:17.466 INFO:journalctl@ceph.osd.8.smithi167.stdout:Aug 19 01:45:17 smithi167 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-8[39928]: 2024-08-19T01:45:17.191+0000 7f17fe820640 -1 osd.8 89 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-08-19T01:44:32.049249+0000 front 2024-08-19T01:44:32.049153+0000 (oldest deadline 2024-08-19T01:44:53.749126+0000) 2024-08-19T01:45:17.466 INFO:journalctl@ceph.osd.8.smithi167.stdout:Aug 19 01:45:17 smithi167 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-8[39928]: 2024-08-19T01:45:17.191+0000 7f17fe820640 -1 osd.8 89 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-08-19T01:44:36.050034+0000 front 2024-08-19T01:44:36.049973+0000 (oldest deadline 2024-08-19T01:45:06.050849+0000) 2024-08-19T01:45:17.654 INFO:journalctl@ceph.osd.9.smithi083.stdout:Aug 19 01:45:17 smithi083 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-9[51472]: 2024-08-19T01:45:17.349+0000 7f54e8c00640 -1 osd.9 89 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-08-19T01:44:20.873282+0000 front 2024-08-19T01:44:20.873259+0000 (oldest deadline 2024-08-19T01:44:46.773413+0000) 2024-08-19T01:45:17.654 INFO:journalctl@ceph.osd.9.smithi083.stdout:Aug 19 01:45:17 smithi083 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-9[51472]: 2024-08-19T01:45:17.349+0000 7f54e8c00640 -1 osd.9 89 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-08-19T01:44:26.774104+0000 front 2024-08-19T01:44:26.774036+0000 (oldest deadline 2024-08-19T01:44:53.774823+0000) 2024-08-19T01:45:17.655 INFO:journalctl@ceph.osd.9.smithi083.stdout:Aug 19 01:45:17 smithi083 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-9[51472]: 2024-08-19T01:45:17.349+0000 7f54e8c00640 -1 osd.9 89 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-08-19T01:44:39.676617+0000 front 2024-08-19T01:44:39.676511+0000 (oldest deadline 2024-08-19T01:45:03.776684+0000) 2024-08-19T01:45:17.655 INFO:journalctl@ceph.osd.9.smithi083.stdout:Aug 19 01:45:17 smithi083 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-9[51472]: 2024-08-19T01:45:17.349+0000 7f54e8c00640 -1 osd.9 89 heartbeat_check: no reply from 172.21.15.83:6814 osd.3 since back 2024-08-19T01:44:46.678051+0000 front 2024-08-19T01:44:46.678168+0000 (oldest deadline 2024-08-19T01:45:11.378113+0000) 2024-08-19T01:45:17.952 INFO:journalctl@ceph.osd.10.smithi117.stdout:Aug 19 01:45:17 smithi117 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-10[46616]: 2024-08-19T01:45:17.546+0000 7f34b3bf7640 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-08-19T01:44:24.060751+0000 front 2024-08-19T01:44:24.060716+0000 (oldest deadline 2024-08-19T01:44:47.560677+0000) 2024-08-19T01:45:17.952 INFO:journalctl@ceph.osd.10.smithi117.stdout:Aug 19 01:45:17 smithi117 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-10[46616]: 2024-08-19T01:45:17.546+0000 7f34b3bf7640 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-08-19T01:44:28.661503+0000 front 2024-08-19T01:44:28.661522+0000 (oldest deadline 2024-08-19T01:44:56.262298+0000) 2024-08-19T01:45:17.952 INFO:journalctl@ceph.osd.10.smithi117.stdout:Aug 19 01:45:17 smithi117 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-10[46616]: 2024-08-19T01:45:17.546+0000 7f34b3bf7640 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-08-19T01:44:38.563674+0000 front 2024-08-19T01:44:38.563520+0000 (oldest deadline 2024-08-19T01:45:02.063517+0000) 2024-08-19T01:45:17.952 INFO:journalctl@ceph.osd.10.smithi117.stdout:Aug 19 01:45:17 smithi117 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-10[46616]: 2024-08-19T01:45:17.546+0000 7f34b3bf7640 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.83:6814 osd.3 since back 2024-08-19T01:44:47.265176+0000 front 2024-08-19T01:44:47.265102+0000 (oldest deadline 2024-08-19T01:45:09.564999+0000) 2024-08-19T01:45:18.192 INFO:journalctl@ceph.osd.11.smithi167.stdout:Aug 19 01:45:17 smithi167 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-11[46643]: 2024-08-19T01:45:17.799+0000 7f447f624640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-08-19T01:44:22.880210+0000 front 2024-08-19T01:44:22.880404+0000 (oldest deadline 2024-08-19T01:44:45.780352+0000) 2024-08-19T01:45:18.192 INFO:journalctl@ceph.osd.11.smithi167.stdout:Aug 19 01:45:17 smithi167 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-11[46643]: 2024-08-19T01:45:17.799+0000 7f447f624640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-08-19T01:44:31.081546+0000 front 2024-08-19T01:44:31.081654+0000 (oldest deadline 2024-08-19T01:44:53.981595+0000) 2024-08-19T01:45:18.192 INFO:journalctl@ceph.osd.11.smithi167.stdout:Aug 19 01:45:17 smithi167 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-11[46643]: 2024-08-19T01:45:17.799+0000 7f447f624640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-08-19T01:44:39.282638+0000 front 2024-08-19T01:44:39.282723+0000 (oldest deadline 2024-08-19T01:45:04.582709+0000) 2024-08-19T01:45:18.192 INFO:journalctl@ceph.osd.11.smithi167.stdout:Aug 19 01:45:17 smithi167 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-11[46643]: 2024-08-19T01:45:17.799+0000 7f447f624640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.83:6814 osd.3 since back 2024-08-19T01:44:45.083762+0000 front 2024-08-19T01:44:45.083777+0000 (oldest deadline 2024-08-19T01:45:10.383738+0000) 2024-08-19T01:45:18.452 INFO:journalctl@ceph.osd.7.smithi117.stdout:Aug 19 01:45:18 smithi117 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-7[39667]: 2024-08-19T01:45:18.140+0000 7f66eaeae640 -1 osd.7 89 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-08-19T01:44:31.724600+0000 front 2024-08-19T01:44:31.724641+0000 (oldest deadline 2024-08-19T01:44:54.024721+0000) 2024-08-19T01:45:18.452 INFO:journalctl@ceph.osd.7.smithi117.stdout:Aug 19 01:45:18 smithi117 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-7[39667]: 2024-08-19T01:45:18.140+0000 7f66eaeae640 -1 osd.7 89 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-08-19T01:44:40.426018+0000 front 2024-08-19T01:44:40.426002+0000 (oldest deadline 2024-08-19T01:45:01.526013+0000) 2024-08-19T01:45:18.452 INFO:journalctl@ceph.osd.7.smithi117.stdout:Aug 19 01:45:18 smithi117 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-7[39667]: 2024-08-19T01:45:18.140+0000 7f66eaeae640 -1 osd.7 89 heartbeat_check: no reply from 172.21.15.83:6814 osd.3 since back 2024-08-19T01:44:45.627330+0000 front 2024-08-19T01:44:45.627160+0000 (oldest deadline 2024-08-19T01:45:09.727143+0000) 2024-08-19T01:45:18.465 INFO:journalctl@ceph.osd.8.smithi167.stdout:Aug 19 01:45:18 smithi167 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-8[39928]: 2024-08-19T01:45:18.191+0000 7f17fe820640 -1 osd.8 89 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-08-19T01:44:22.747507+0000 front 2024-08-19T01:44:22.747516+0000 (oldest deadline 2024-08-19T01:44:48.047324+0000) 2024-08-19T01:45:18.466 INFO:journalctl@ceph.osd.8.smithi167.stdout:Aug 19 01:45:18 smithi167 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-8[39928]: 2024-08-19T01:45:18.191+0000 7f17fe820640 -1 osd.8 89 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-08-19T01:44:32.049249+0000 front 2024-08-19T01:44:32.049153+0000 (oldest deadline 2024-08-19T01:44:53.749126+0000) 2024-08-19T01:45:18.466 INFO:journalctl@ceph.osd.8.smithi167.stdout:Aug 19 01:45:18 smithi167 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-8[39928]: 2024-08-19T01:45:18.191+0000 7f17fe820640 -1 osd.8 89 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-08-19T01:44:36.050034+0000 front 2024-08-19T01:44:36.049973+0000 (oldest deadline 2024-08-19T01:45:06.050849+0000) 2024-08-19T01:45:18.654 INFO:journalctl@ceph.osd.9.smithi083.stdout:Aug 19 01:45:18 smithi083 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-9[51472]: 2024-08-19T01:45:18.393+0000 7f54e8c00640 -1 osd.9 89 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-08-19T01:44:20.873282+0000 front 2024-08-19T01:44:20.873259+0000 (oldest deadline 2024-08-19T01:44:46.773413+0000) 2024-08-19T01:45:18.655 INFO:journalctl@ceph.osd.9.smithi083.stdout:Aug 19 01:45:18 smithi083 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-9[51472]: 2024-08-19T01:45:18.393+0000 7f54e8c00640 -1 osd.9 89 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-08-19T01:44:26.774104+0000 front 2024-08-19T01:44:26.774036+0000 (oldest deadline 2024-08-19T01:44:53.774823+0000) 2024-08-19T01:45:18.655 INFO:journalctl@ceph.osd.9.smithi083.stdout:Aug 19 01:45:18 smithi083 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-9[51472]: 2024-08-19T01:45:18.393+0000 7f54e8c00640 -1 osd.9 89 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-08-19T01:44:39.676617+0000 front 2024-08-19T01:44:39.676511+0000 (oldest deadline 2024-08-19T01:45:03.776684+0000) 2024-08-19T01:45:18.655 INFO:journalctl@ceph.osd.9.smithi083.stdout:Aug 19 01:45:18 smithi083 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-9[51472]: 2024-08-19T01:45:18.393+0000 7f54e8c00640 -1 osd.9 89 heartbeat_check: no reply from 172.21.15.83:6814 osd.3 since back 2024-08-19T01:44:46.678051+0000 front 2024-08-19T01:44:46.678168+0000 (oldest deadline 2024-08-19T01:45:11.378113+0000) 2024-08-19T01:45:18.655 INFO:journalctl@ceph.osd.9.smithi083.stdout:Aug 19 01:45:18 smithi083 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-9[51472]: 2024-08-19T01:45:18.393+0000 7f54e8c00640 -1 osd.9 89 heartbeat_check: no reply from 172.21.15.117:6812 osd.4 since back 2024-08-19T01:44:54.279278+0000 front 2024-08-19T01:44:54.279303+0000 (oldest deadline 2024-08-19T01:45:18.379436+0000) 2024-08-19T01:45:18.952 INFO:journalctl@ceph.osd.10.smithi117.stdout:Aug 19 01:45:18 smithi117 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-10[46616]: 2024-08-19T01:45:18.526+0000 7f34b3bf7640 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-08-19T01:44:24.060751+0000 front 2024-08-19T01:44:24.060716+0000 (oldest deadline 2024-08-19T01:44:47.560677+0000) 2024-08-19T01:45:18.952 INFO:journalctl@ceph.osd.10.smithi117.stdout:Aug 19 01:45:18 smithi117 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-10[46616]: 2024-08-19T01:45:18.526+0000 7f34b3bf7640 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-08-19T01:44:28.661503+0000 front 2024-08-19T01:44:28.661522+0000 (oldest deadline 2024-08-19T01:44:56.262298+0000) 2024-08-19T01:45:18.952 INFO:journalctl@ceph.osd.10.smithi117.stdout:Aug 19 01:45:18 smithi117 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-10[46616]: 2024-08-19T01:45:18.526+0000 7f34b3bf7640 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-08-19T01:44:38.563674+0000 front 2024-08-19T01:44:38.563520+0000 (oldest deadline 2024-08-19T01:45:02.063517+0000) 2024-08-19T01:45:18.952 INFO:journalctl@ceph.osd.10.smithi117.stdout:Aug 19 01:45:18 smithi117 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-10[46616]: 2024-08-19T01:45:18.526+0000 7f34b3bf7640 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.83:6814 osd.3 since back 2024-08-19T01:44:47.265176+0000 front 2024-08-19T01:44:47.265102+0000 (oldest deadline 2024-08-19T01:45:09.564999+0000) 2024-08-19T01:45:19.207 INFO:journalctl@ceph.osd.11.smithi167.stdout:Aug 19 01:45:18 smithi167 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-11[46643]: 2024-08-19T01:45:18.751+0000 7f447f624640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-08-19T01:44:22.880210+0000 front 2024-08-19T01:44:22.880404+0000 (oldest deadline 2024-08-19T01:44:45.780352+0000) 2024-08-19T01:45:19.207 INFO:journalctl@ceph.osd.11.smithi167.stdout:Aug 19 01:45:18 smithi167 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-11[46643]: 2024-08-19T01:45:18.751+0000 7f447f624640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-08-19T01:44:31.081546+0000 front 2024-08-19T01:44:31.081654+0000 (oldest deadline 2024-08-19T01:44:53.981595+0000) 2024-08-19T01:45:19.208 INFO:journalctl@ceph.osd.11.smithi167.stdout:Aug 19 01:45:18 smithi167 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-11[46643]: 2024-08-19T01:45:18.751+0000 7f447f624640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-08-19T01:44:39.282638+0000 front 2024-08-19T01:44:39.282723+0000 (oldest deadline 2024-08-19T01:45:04.582709+0000) 2024-08-19T01:45:19.208 INFO:journalctl@ceph.osd.11.smithi167.stdout:Aug 19 01:45:18 smithi167 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-11[46643]: 2024-08-19T01:45:18.751+0000 7f447f624640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.83:6814 osd.3 since back 2024-08-19T01:44:45.083762+0000 front 2024-08-19T01:44:45.083777+0000 (oldest deadline 2024-08-19T01:45:10.383738+0000) 2024-08-19T01:45:19.452 INFO:journalctl@ceph.osd.7.smithi117.stdout:Aug 19 01:45:19 smithi117 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-7[39667]: 2024-08-19T01:45:19.147+0000 7f66eaeae640 -1 osd.7 89 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-08-19T01:44:31.724600+0000 front 2024-08-19T01:44:31.724641+0000 (oldest deadline 2024-08-19T01:44:54.024721+0000) 2024-08-19T01:45:19.452 INFO:journalctl@ceph.osd.7.smithi117.stdout:Aug 19 01:45:19 smithi117 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-7[39667]: 2024-08-19T01:45:19.147+0000 7f66eaeae640 -1 osd.7 89 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-08-19T01:44:40.426018+0000 front 2024-08-19T01:44:40.426002+0000 (oldest deadline 2024-08-19T01:45:01.526013+0000) 2024-08-19T01:45:19.453 INFO:journalctl@ceph.osd.7.smithi117.stdout:Aug 19 01:45:19 smithi117 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-7[39667]: 2024-08-19T01:45:19.147+0000 7f66eaeae640 -1 osd.7 89 heartbeat_check: no reply from 172.21.15.83:6814 osd.3 since back 2024-08-19T01:44:45.627330+0000 front 2024-08-19T01:44:45.627160+0000 (oldest deadline 2024-08-19T01:45:09.727143+0000) 2024-08-19T01:45:19.453 INFO:journalctl@ceph.osd.7.smithi117.stdout:Aug 19 01:45:19 smithi117 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-7[39667]: 2024-08-19T01:45:19.147+0000 7f66eaeae640 -1 osd.7 89 heartbeat_check: no reply from 172.21.15.117:6812 osd.4 since back 2024-08-19T01:44:55.628183+0000 front 2024-08-19T01:44:55.628220+0000 (oldest deadline 2024-08-19T01:45:18.528311+0000) 2024-08-19T01:45:19.466 INFO:journalctl@ceph.osd.8.smithi167.stdout:Aug 19 01:45:19 smithi167 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-8[39928]: 2024-08-19T01:45:19.206+0000 7f17fe820640 -1 osd.8 89 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-08-19T01:44:22.747507+0000 front 2024-08-19T01:44:22.747516+0000 (oldest deadline 2024-08-19T01:44:48.047324+0000) 2024-08-19T01:45:19.466 INFO:journalctl@ceph.osd.8.smithi167.stdout:Aug 19 01:45:19 smithi167 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-8[39928]: 2024-08-19T01:45:19.206+0000 7f17fe820640 -1 osd.8 89 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-08-19T01:44:32.049249+0000 front 2024-08-19T01:44:32.049153+0000 (oldest deadline 2024-08-19T01:44:53.749126+0000) 2024-08-19T01:45:19.466 INFO:journalctl@ceph.osd.8.smithi167.stdout:Aug 19 01:45:19 smithi167 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-8[39928]: 2024-08-19T01:45:19.206+0000 7f17fe820640 -1 osd.8 89 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-08-19T01:44:36.050034+0000 front 2024-08-19T01:44:36.049973+0000 (oldest deadline 2024-08-19T01:45:06.050849+0000) 2024-08-19T01:45:19.654 INFO:journalctl@ceph.osd.9.smithi083.stdout:Aug 19 01:45:19 smithi083 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-9[51472]: 2024-08-19T01:45:19.367+0000 7f54e8c00640 -1 osd.9 89 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-08-19T01:44:20.873282+0000 front 2024-08-19T01:44:20.873259+0000 (oldest deadline 2024-08-19T01:44:46.773413+0000) 2024-08-19T01:45:19.655 INFO:journalctl@ceph.osd.9.smithi083.stdout:Aug 19 01:45:19 smithi083 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-9[51472]: 2024-08-19T01:45:19.367+0000 7f54e8c00640 -1 osd.9 89 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-08-19T01:44:26.774104+0000 front 2024-08-19T01:44:26.774036+0000 (oldest deadline 2024-08-19T01:44:53.774823+0000) 2024-08-19T01:45:19.655 INFO:journalctl@ceph.osd.9.smithi083.stdout:Aug 19 01:45:19 smithi083 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-9[51472]: 2024-08-19T01:45:19.367+0000 7f54e8c00640 -1 osd.9 89 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-08-19T01:44:39.676617+0000 front 2024-08-19T01:44:39.676511+0000 (oldest deadline 2024-08-19T01:45:03.776684+0000) 2024-08-19T01:45:19.655 INFO:journalctl@ceph.osd.9.smithi083.stdout:Aug 19 01:45:19 smithi083 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-9[51472]: 2024-08-19T01:45:19.367+0000 7f54e8c00640 -1 osd.9 89 heartbeat_check: no reply from 172.21.15.83:6814 osd.3 since back 2024-08-19T01:44:46.678051+0000 front 2024-08-19T01:44:46.678168+0000 (oldest deadline 2024-08-19T01:45:11.378113+0000) 2024-08-19T01:45:19.655 INFO:journalctl@ceph.osd.9.smithi083.stdout:Aug 19 01:45:19 smithi083 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-9[51472]: 2024-08-19T01:45:19.367+0000 7f54e8c00640 -1 osd.9 89 heartbeat_check: no reply from 172.21.15.117:6812 osd.4 since back 2024-08-19T01:44:54.279278+0000 front 2024-08-19T01:44:54.279303+0000 (oldest deadline 2024-08-19T01:45:18.379436+0000) 2024-08-19T01:45:19.953 INFO:journalctl@ceph.osd.10.smithi117.stdout:Aug 19 01:45:19 smithi117 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-10[46616]: 2024-08-19T01:45:19.504+0000 7f34b3bf7640 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-08-19T01:44:24.060751+0000 front 2024-08-19T01:44:24.060716+0000 (oldest deadline 2024-08-19T01:44:47.560677+0000) 2024-08-19T01:45:19.953 INFO:journalctl@ceph.osd.10.smithi117.stdout:Aug 19 01:45:19 smithi117 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-10[46616]: 2024-08-19T01:45:19.504+0000 7f34b3bf7640 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-08-19T01:44:28.661503+0000 front 2024-08-19T01:44:28.661522+0000 (oldest deadline 2024-08-19T01:44:56.262298+0000) 2024-08-19T01:45:19.953 INFO:journalctl@ceph.osd.10.smithi117.stdout:Aug 19 01:45:19 smithi117 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-10[46616]: 2024-08-19T01:45:19.504+0000 7f34b3bf7640 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-08-19T01:44:38.563674+0000 front 2024-08-19T01:44:38.563520+0000 (oldest deadline 2024-08-19T01:45:02.063517+0000) 2024-08-19T01:45:19.953 INFO:journalctl@ceph.osd.10.smithi117.stdout:Aug 19 01:45:19 smithi117 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-10[46616]: 2024-08-19T01:45:19.504+0000 7f34b3bf7640 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.83:6814 osd.3 since back 2024-08-19T01:44:47.265176+0000 front 2024-08-19T01:44:47.265102+0000 (oldest deadline 2024-08-19T01:45:09.564999+0000) 2024-08-19T01:45:20.216 INFO:journalctl@ceph.osd.11.smithi167.stdout:Aug 19 01:45:19 smithi167 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-11[46643]: 2024-08-19T01:45:19.734+0000 7f447f624640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-08-19T01:44:22.880210+0000 front 2024-08-19T01:44:22.880404+0000 (oldest deadline 2024-08-19T01:44:45.780352+0000) 2024-08-19T01:45:20.216 INFO:journalctl@ceph.osd.11.smithi167.stdout:Aug 19 01:45:19 smithi167 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-11[46643]: 2024-08-19T01:45:19.734+0000 7f447f624640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-08-19T01:44:31.081546+0000 front 2024-08-19T01:44:31.081654+0000 (oldest deadline 2024-08-19T01:44:53.981595+0000) 2024-08-19T01:45:20.216 INFO:journalctl@ceph.osd.11.smithi167.stdout:Aug 19 01:45:19 smithi167 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-11[46643]: 2024-08-19T01:45:19.734+0000 7f447f624640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-08-19T01:44:39.282638+0000 front 2024-08-19T01:44:39.282723+0000 (oldest deadline 2024-08-19T01:45:04.582709+0000) 2024-08-19T01:45:20.216 INFO:journalctl@ceph.osd.11.smithi167.stdout:Aug 19 01:45:19 smithi167 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-11[46643]: 2024-08-19T01:45:19.734+0000 7f447f624640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.83:6814 osd.3 since back 2024-08-19T01:44:45.083762+0000 front 2024-08-19T01:44:45.083777+0000 (oldest deadline 2024-08-19T01:45:10.383738+0000) 2024-08-19T01:45:20.452 INFO:journalctl@ceph.osd.7.smithi117.stdout:Aug 19 01:45:20 smithi117 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-7[39667]: 2024-08-19T01:45:20.187+0000 7f66eaeae640 -1 osd.7 89 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-08-19T01:44:31.724600+0000 front 2024-08-19T01:44:31.724641+0000 (oldest deadline 2024-08-19T01:44:54.024721+0000) 2024-08-19T01:45:20.452 INFO:journalctl@ceph.osd.7.smithi117.stdout:Aug 19 01:45:20 smithi117 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-7[39667]: 2024-08-19T01:45:20.187+0000 7f66eaeae640 -1 osd.7 89 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-08-19T01:44:40.426018+0000 front 2024-08-19T01:44:40.426002+0000 (oldest deadline 2024-08-19T01:45:01.526013+0000) 2024-08-19T01:45:20.452 INFO:journalctl@ceph.osd.7.smithi117.stdout:Aug 19 01:45:20 smithi117 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-7[39667]: 2024-08-19T01:45:20.187+0000 7f66eaeae640 -1 osd.7 89 heartbeat_check: no reply from 172.21.15.83:6814 osd.3 since back 2024-08-19T01:44:45.627330+0000 front 2024-08-19T01:44:45.627160+0000 (oldest deadline 2024-08-19T01:45:09.727143+0000) 2024-08-19T01:45:20.452 INFO:journalctl@ceph.osd.7.smithi117.stdout:Aug 19 01:45:20 smithi117 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-7[39667]: 2024-08-19T01:45:20.187+0000 7f66eaeae640 -1 osd.7 89 heartbeat_check: no reply from 172.21.15.117:6812 osd.4 since back 2024-08-19T01:44:55.628183+0000 front 2024-08-19T01:44:55.628220+0000 (oldest deadline 2024-08-19T01:45:18.528311+0000) 2024-08-19T01:45:20.481 INFO:journalctl@ceph.osd.8.smithi167.stdout:Aug 19 01:45:20 smithi167 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-8[39928]: 2024-08-19T01:45:20.240+0000 7f17fe820640 -1 osd.8 89 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-08-19T01:44:22.747507+0000 front 2024-08-19T01:44:22.747516+0000 (oldest deadline 2024-08-19T01:44:48.047324+0000) 2024-08-19T01:45:20.482 INFO:journalctl@ceph.osd.8.smithi167.stdout:Aug 19 01:45:20 smithi167 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-8[39928]: 2024-08-19T01:45:20.240+0000 7f17fe820640 -1 osd.8 89 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-08-19T01:44:32.049249+0000 front 2024-08-19T01:44:32.049153+0000 (oldest deadline 2024-08-19T01:44:53.749126+0000) 2024-08-19T01:45:20.482 INFO:journalctl@ceph.osd.8.smithi167.stdout:Aug 19 01:45:20 smithi167 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-8[39928]: 2024-08-19T01:45:20.240+0000 7f17fe820640 -1 osd.8 89 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-08-19T01:44:36.050034+0000 front 2024-08-19T01:44:36.049973+0000 (oldest deadline 2024-08-19T01:45:06.050849+0000) 2024-08-19T01:45:20.654 INFO:journalctl@ceph.osd.9.smithi083.stdout:Aug 19 01:45:20 smithi083 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-9[51472]: 2024-08-19T01:45:20.331+0000 7f54e8c00640 -1 osd.9 89 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-08-19T01:44:20.873282+0000 front 2024-08-19T01:44:20.873259+0000 (oldest deadline 2024-08-19T01:44:46.773413+0000) 2024-08-19T01:45:20.655 INFO:journalctl@ceph.osd.9.smithi083.stdout:Aug 19 01:45:20 smithi083 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-9[51472]: 2024-08-19T01:45:20.331+0000 7f54e8c00640 -1 osd.9 89 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-08-19T01:44:26.774104+0000 front 2024-08-19T01:44:26.774036+0000 (oldest deadline 2024-08-19T01:44:53.774823+0000) 2024-08-19T01:45:20.655 INFO:journalctl@ceph.osd.9.smithi083.stdout:Aug 19 01:45:20 smithi083 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-9[51472]: 2024-08-19T01:45:20.331+0000 7f54e8c00640 -1 osd.9 89 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-08-19T01:44:39.676617+0000 front 2024-08-19T01:44:39.676511+0000 (oldest deadline 2024-08-19T01:45:03.776684+0000) 2024-08-19T01:45:20.655 INFO:journalctl@ceph.osd.9.smithi083.stdout:Aug 19 01:45:20 smithi083 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-9[51472]: 2024-08-19T01:45:20.331+0000 7f54e8c00640 -1 osd.9 89 heartbeat_check: no reply from 172.21.15.83:6814 osd.3 since back 2024-08-19T01:44:46.678051+0000 front 2024-08-19T01:44:46.678168+0000 (oldest deadline 2024-08-19T01:45:11.378113+0000) 2024-08-19T01:45:20.655 INFO:journalctl@ceph.osd.9.smithi083.stdout:Aug 19 01:45:20 smithi083 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-9[51472]: 2024-08-19T01:45:20.331+0000 7f54e8c00640 -1 osd.9 89 heartbeat_check: no reply from 172.21.15.117:6812 osd.4 since back 2024-08-19T01:44:54.279278+0000 front 2024-08-19T01:44:54.279303+0000 (oldest deadline 2024-08-19T01:45:18.379436+0000) 2024-08-19T01:45:20.952 INFO:journalctl@ceph.osd.10.smithi117.stdout:Aug 19 01:45:20 smithi117 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-10[46616]: 2024-08-19T01:45:20.463+0000 7f34b3bf7640 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-08-19T01:44:24.060751+0000 front 2024-08-19T01:44:24.060716+0000 (oldest deadline 2024-08-19T01:44:47.560677+0000) 2024-08-19T01:45:20.952 INFO:journalctl@ceph.osd.10.smithi117.stdout:Aug 19 01:45:20 smithi117 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-10[46616]: 2024-08-19T01:45:20.463+0000 7f34b3bf7640 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-08-19T01:44:28.661503+0000 front 2024-08-19T01:44:28.661522+0000 (oldest deadline 2024-08-19T01:44:56.262298+0000) 2024-08-19T01:45:20.953 INFO:journalctl@ceph.osd.10.smithi117.stdout:Aug 19 01:45:20 smithi117 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-10[46616]: 2024-08-19T01:45:20.463+0000 7f34b3bf7640 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-08-19T01:44:38.563674+0000 front 2024-08-19T01:44:38.563520+0000 (oldest deadline 2024-08-19T01:45:02.063517+0000) 2024-08-19T01:45:20.953 INFO:journalctl@ceph.osd.10.smithi117.stdout:Aug 19 01:45:20 smithi117 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-10[46616]: 2024-08-19T01:45:20.463+0000 7f34b3bf7640 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.83:6814 osd.3 since back 2024-08-19T01:44:47.265176+0000 front 2024-08-19T01:44:47.265102+0000 (oldest deadline 2024-08-19T01:45:09.564999+0000) 2024-08-19T01:45:20.953 INFO:journalctl@ceph.osd.10.smithi117.stdout:Aug 19 01:45:20 smithi117 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-10[46616]: 2024-08-19T01:45:20.463+0000 7f34b3bf7640 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.117:6812 osd.4 since back 2024-08-19T01:44:54.865666+0000 front 2024-08-19T01:44:54.865695+0000 (oldest deadline 2024-08-19T01:45:19.565735+0000) 2024-08-19T01:45:21.216 INFO:journalctl@ceph.osd.11.smithi167.stdout:Aug 19 01:45:20 smithi167 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-11[46643]: 2024-08-19T01:45:20.750+0000 7f447f624640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-08-19T01:44:22.880210+0000 front 2024-08-19T01:44:22.880404+0000 (oldest deadline 2024-08-19T01:44:45.780352+0000) 2024-08-19T01:45:21.216 INFO:journalctl@ceph.osd.11.smithi167.stdout:Aug 19 01:45:20 smithi167 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-11[46643]: 2024-08-19T01:45:20.750+0000 7f447f624640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-08-19T01:44:31.081546+0000 front 2024-08-19T01:44:31.081654+0000 (oldest deadline 2024-08-19T01:44:53.981595+0000) 2024-08-19T01:45:21.216 INFO:journalctl@ceph.osd.11.smithi167.stdout:Aug 19 01:45:20 smithi167 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-11[46643]: 2024-08-19T01:45:20.750+0000 7f447f624640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-08-19T01:44:39.282638+0000 front 2024-08-19T01:44:39.282723+0000 (oldest deadline 2024-08-19T01:45:04.582709+0000) 2024-08-19T01:45:21.216 INFO:journalctl@ceph.osd.11.smithi167.stdout:Aug 19 01:45:20 smithi167 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-11[46643]: 2024-08-19T01:45:20.750+0000 7f447f624640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.83:6814 osd.3 since back 2024-08-19T01:44:45.083762+0000 front 2024-08-19T01:44:45.083777+0000 (oldest deadline 2024-08-19T01:45:10.383738+0000) 2024-08-19T01:45:21.447 INFO:journalctl@ceph.osd.7.smithi117.stdout:Aug 19 01:45:21 smithi117 podman[56209]: 2024-08-19 01:45:21.168111841 +0000 UTC m=+5.620636742 container died 98bb53d2daa42a014a940d260b2fd135a415881731d8537d7a98798685cb3de0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c9dd28080a631be25c77c36c63b17a3ef340ffb2e21c87d4d7b56b6398436d9, name=ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-7, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-de09dd0, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, io.buildah.version=1.36.0, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, ceph=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS) 2024-08-19T01:45:21.654 INFO:journalctl@ceph.osd.9.smithi083.stdout:Aug 19 01:45:21 smithi083 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-9[51472]: 2024-08-19T01:45:21.316+0000 7f54e8c00640 -1 osd.9 89 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-08-19T01:44:20.873282+0000 front 2024-08-19T01:44:20.873259+0000 (oldest deadline 2024-08-19T01:44:46.773413+0000) 2024-08-19T01:45:21.655 INFO:journalctl@ceph.osd.9.smithi083.stdout:Aug 19 01:45:21 smithi083 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-9[51472]: 2024-08-19T01:45:21.316+0000 7f54e8c00640 -1 osd.9 89 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-08-19T01:44:26.774104+0000 front 2024-08-19T01:44:26.774036+0000 (oldest deadline 2024-08-19T01:44:53.774823+0000) 2024-08-19T01:45:21.655 INFO:journalctl@ceph.osd.9.smithi083.stdout:Aug 19 01:45:21 smithi083 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-9[51472]: 2024-08-19T01:45:21.316+0000 7f54e8c00640 -1 osd.9 89 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-08-19T01:44:39.676617+0000 front 2024-08-19T01:44:39.676511+0000 (oldest deadline 2024-08-19T01:45:03.776684+0000) 2024-08-19T01:45:21.655 INFO:journalctl@ceph.osd.9.smithi083.stdout:Aug 19 01:45:21 smithi083 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-9[51472]: 2024-08-19T01:45:21.316+0000 7f54e8c00640 -1 osd.9 89 heartbeat_check: no reply from 172.21.15.83:6814 osd.3 since back 2024-08-19T01:44:46.678051+0000 front 2024-08-19T01:44:46.678168+0000 (oldest deadline 2024-08-19T01:45:11.378113+0000) 2024-08-19T01:45:21.655 INFO:journalctl@ceph.osd.9.smithi083.stdout:Aug 19 01:45:21 smithi083 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-9[51472]: 2024-08-19T01:45:21.316+0000 7f54e8c00640 -1 osd.9 89 heartbeat_check: no reply from 172.21.15.117:6812 osd.4 since back 2024-08-19T01:44:54.279278+0000 front 2024-08-19T01:44:54.279303+0000 (oldest deadline 2024-08-19T01:45:18.379436+0000) 2024-08-19T01:45:21.702 INFO:journalctl@ceph.osd.7.smithi117.stdout:Aug 19 01:45:21 smithi117 podman[56209]: 2024-08-19 01:45:21.545284919 +0000 UTC m=+5.997809817 container cleanup 98bb53d2daa42a014a940d260b2fd135a415881731d8537d7a98798685cb3de0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c9dd28080a631be25c77c36c63b17a3ef340ffb2e21c87d4d7b56b6398436d9, name=ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-7, RELEASE=squid-de09dd0, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, io.buildah.version=1.36.0, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD) 2024-08-19T01:45:21.702 INFO:journalctl@ceph.osd.7.smithi117.stdout:Aug 19 01:45:21 smithi117 bash[56209]: ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-7 2024-08-19T01:45:21.703 INFO:journalctl@ceph.osd.10.smithi117.stdout:Aug 19 01:45:21 smithi117 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-10[46616]: 2024-08-19T01:45:21.512+0000 7f34b3bf7640 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-08-19T01:44:24.060751+0000 front 2024-08-19T01:44:24.060716+0000 (oldest deadline 2024-08-19T01:44:47.560677+0000) 2024-08-19T01:45:21.703 INFO:journalctl@ceph.osd.10.smithi117.stdout:Aug 19 01:45:21 smithi117 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-10[46616]: 2024-08-19T01:45:21.512+0000 7f34b3bf7640 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-08-19T01:44:28.661503+0000 front 2024-08-19T01:44:28.661522+0000 (oldest deadline 2024-08-19T01:44:56.262298+0000) 2024-08-19T01:45:21.703 INFO:journalctl@ceph.osd.10.smithi117.stdout:Aug 19 01:45:21 smithi117 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-10[46616]: 2024-08-19T01:45:21.512+0000 7f34b3bf7640 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-08-19T01:44:38.563674+0000 front 2024-08-19T01:44:38.563520+0000 (oldest deadline 2024-08-19T01:45:02.063517+0000) 2024-08-19T01:45:21.703 INFO:journalctl@ceph.osd.10.smithi117.stdout:Aug 19 01:45:21 smithi117 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-10[46616]: 2024-08-19T01:45:21.512+0000 7f34b3bf7640 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.83:6814 osd.3 since back 2024-08-19T01:44:47.265176+0000 front 2024-08-19T01:44:47.265102+0000 (oldest deadline 2024-08-19T01:45:09.564999+0000) 2024-08-19T01:45:21.703 INFO:journalctl@ceph.osd.10.smithi117.stdout:Aug 19 01:45:21 smithi117 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-10[46616]: 2024-08-19T01:45:21.512+0000 7f34b3bf7640 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.117:6812 osd.4 since back 2024-08-19T01:44:54.865666+0000 front 2024-08-19T01:44:54.865695+0000 (oldest deadline 2024-08-19T01:45:19.565735+0000) 2024-08-19T01:45:21.715 INFO:journalctl@ceph.osd.8.smithi167.stdout:Aug 19 01:45:21 smithi167 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-8[39928]: 2024-08-19T01:45:21.255+0000 7f17fe820640 -1 osd.8 89 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-08-19T01:44:22.747507+0000 front 2024-08-19T01:44:22.747516+0000 (oldest deadline 2024-08-19T01:44:48.047324+0000) 2024-08-19T01:45:21.716 INFO:journalctl@ceph.osd.8.smithi167.stdout:Aug 19 01:45:21 smithi167 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-8[39928]: 2024-08-19T01:45:21.255+0000 7f17fe820640 -1 osd.8 89 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-08-19T01:44:32.049249+0000 front 2024-08-19T01:44:32.049153+0000 (oldest deadline 2024-08-19T01:44:53.749126+0000) 2024-08-19T01:45:21.716 INFO:journalctl@ceph.osd.8.smithi167.stdout:Aug 19 01:45:21 smithi167 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-8[39928]: 2024-08-19T01:45:21.255+0000 7f17fe820640 -1 osd.8 89 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-08-19T01:44:36.050034+0000 front 2024-08-19T01:44:36.049973+0000 (oldest deadline 2024-08-19T01:45:06.050849+0000) 2024-08-19T01:45:21.716 INFO:journalctl@ceph.osd.8.smithi167.stdout:Aug 19 01:45:21 smithi167 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-8[39928]: 2024-08-19T01:45:21.255+0000 7f17fe820640 -1 osd.8 89 heartbeat_check: no reply from 172.21.15.117:6812 osd.4 since back 2024-08-19T01:44:56.552909+0000 front 2024-08-19T01:44:56.552897+0000 (oldest deadline 2024-08-19T01:45:20.652757+0000) 2024-08-19T01:45:22.106 INFO:journalctl@ceph.osd.7.smithi117.stdout:Aug 19 01:45:21 smithi117 podman[56224]: 2024-08-19 01:45:21.804902675 +0000 UTC m=+0.632102333 container remove 98bb53d2daa42a014a940d260b2fd135a415881731d8537d7a98798685cb3de0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c9dd28080a631be25c77c36c63b17a3ef340ffb2e21c87d4d7b56b6398436d9, name=ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-7, org.label-schema.vendor=CentOS, GIT_CLEAN=True, RELEASE=squid-de09dd0, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.36.0, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.license=GPLv2) 2024-08-19T01:45:22.216 INFO:journalctl@ceph.osd.11.smithi167.stdout:Aug 19 01:45:21 smithi167 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-11[46643]: 2024-08-19T01:45:21.729+0000 7f447f624640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-08-19T01:44:22.880210+0000 front 2024-08-19T01:44:22.880404+0000 (oldest deadline 2024-08-19T01:44:45.780352+0000) 2024-08-19T01:45:22.216 INFO:journalctl@ceph.osd.11.smithi167.stdout:Aug 19 01:45:21 smithi167 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-11[46643]: 2024-08-19T01:45:21.729+0000 7f447f624640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-08-19T01:44:31.081546+0000 front 2024-08-19T01:44:31.081654+0000 (oldest deadline 2024-08-19T01:44:53.981595+0000) 2024-08-19T01:45:22.216 INFO:journalctl@ceph.osd.11.smithi167.stdout:Aug 19 01:45:21 smithi167 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-11[46643]: 2024-08-19T01:45:21.729+0000 7f447f624640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-08-19T01:44:39.282638+0000 front 2024-08-19T01:44:39.282723+0000 (oldest deadline 2024-08-19T01:45:04.582709+0000) 2024-08-19T01:45:22.217 INFO:journalctl@ceph.osd.11.smithi167.stdout:Aug 19 01:45:21 smithi167 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-11[46643]: 2024-08-19T01:45:21.729+0000 7f447f624640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.83:6814 osd.3 since back 2024-08-19T01:44:45.083762+0000 front 2024-08-19T01:44:45.083777+0000 (oldest deadline 2024-08-19T01:45:10.383738+0000) 2024-08-19T01:45:22.217 INFO:journalctl@ceph.osd.11.smithi167.stdout:Aug 19 01:45:21 smithi167 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-11[46643]: 2024-08-19T01:45:21.729+0000 7f447f624640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.117:6812 osd.4 since back 2024-08-19T01:44:55.585644+0000 front 2024-08-19T01:44:55.585746+0000 (oldest deadline 2024-08-19T01:45:20.885631+0000) 2024-08-19T01:45:22.392 INFO:journalctl@ceph.osd.7.smithi117.stdout:Aug 19 01:45:22 smithi117 podman[56314]: 2024-08-19 01:45:22.105601125 +0000 UTC m=+0.095330410 container create 24ff36cf6535e300e2b1d9d97b37700d25e5d06587de194fc51f89305cb5d2f0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c9dd28080a631be25c77c36c63b17a3ef340ffb2e21c87d4d7b56b6398436d9, name=ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-7-deactivate, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , RELEASE=squid-de09dd0, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, io.buildah.version=1.36.0, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_CLEAN=True) 2024-08-19T01:45:22.392 INFO:journalctl@ceph.osd.7.smithi117.stdout:Aug 19 01:45:22 smithi117 podman[56314]: 2024-08-19 01:45:22.027279338 +0000 UTC m=+0.017008641 image pull ea7a5189f30bad3aca9524788e2965d2368d7ce786df09d4edcd8c3edf9cefba quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c9dd28080a631be25c77c36c63b17a3ef340ffb2e21c87d4d7b56b6398436d9 2024-08-19T01:45:22.392 INFO:journalctl@ceph.osd.7.smithi117.stdout:Aug 19 01:45:22 smithi117 podman[56314]: 2024-08-19 01:45:22.316225034 +0000 UTC m=+0.305954335 container init 24ff36cf6535e300e2b1d9d97b37700d25e5d06587de194fc51f89305cb5d2f0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c9dd28080a631be25c77c36c63b17a3ef340ffb2e21c87d4d7b56b6398436d9, name=ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-7-deactivate, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.36.0, GIT_CLEAN=True, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, RELEASE=squid-de09dd0, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS) 2024-08-19T01:45:22.392 INFO:journalctl@ceph.osd.7.smithi117.stdout:Aug 19 01:45:22 smithi117 podman[56314]: 2024-08-19 01:45:22.319471541 +0000 UTC m=+0.309200827 container start 24ff36cf6535e300e2b1d9d97b37700d25e5d06587de194fc51f89305cb5d2f0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c9dd28080a631be25c77c36c63b17a3ef340ffb2e21c87d4d7b56b6398436d9, name=ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-7-deactivate, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, RELEASE=squid-de09dd0, org.label-schema.license=GPLv2, io.buildah.version=1.36.0) 2024-08-19T01:45:22.392 INFO:journalctl@ceph.osd.7.smithi117.stdout:Aug 19 01:45:22 smithi117 podman[56314]: 2024-08-19 01:45:22.391120353 +0000 UTC m=+0.380849651 container attach 24ff36cf6535e300e2b1d9d97b37700d25e5d06587de194fc51f89305cb5d2f0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c9dd28080a631be25c77c36c63b17a3ef340ffb2e21c87d4d7b56b6398436d9, name=ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-7-deactivate, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.36.0, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, ceph=True, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-de09dd0, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731) 2024-08-19T01:45:22.654 INFO:journalctl@ceph.osd.9.smithi083.stdout:Aug 19 01:45:22 smithi083 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-9[51472]: 2024-08-19T01:45:22.347+0000 7f54e8c00640 -1 osd.9 89 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-08-19T01:44:20.873282+0000 front 2024-08-19T01:44:20.873259+0000 (oldest deadline 2024-08-19T01:44:46.773413+0000) 2024-08-19T01:45:22.655 INFO:journalctl@ceph.osd.9.smithi083.stdout:Aug 19 01:45:22 smithi083 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-9[51472]: 2024-08-19T01:45:22.347+0000 7f54e8c00640 -1 osd.9 89 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-08-19T01:44:26.774104+0000 front 2024-08-19T01:44:26.774036+0000 (oldest deadline 2024-08-19T01:44:53.774823+0000) 2024-08-19T01:45:22.655 INFO:journalctl@ceph.osd.9.smithi083.stdout:Aug 19 01:45:22 smithi083 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-9[51472]: 2024-08-19T01:45:22.347+0000 7f54e8c00640 -1 osd.9 89 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-08-19T01:44:39.676617+0000 front 2024-08-19T01:44:39.676511+0000 (oldest deadline 2024-08-19T01:45:03.776684+0000) 2024-08-19T01:45:22.655 INFO:journalctl@ceph.osd.9.smithi083.stdout:Aug 19 01:45:22 smithi083 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-9[51472]: 2024-08-19T01:45:22.347+0000 7f54e8c00640 -1 osd.9 89 heartbeat_check: no reply from 172.21.15.83:6814 osd.3 since back 2024-08-19T01:44:46.678051+0000 front 2024-08-19T01:44:46.678168+0000 (oldest deadline 2024-08-19T01:45:11.378113+0000) 2024-08-19T01:45:22.655 INFO:journalctl@ceph.osd.9.smithi083.stdout:Aug 19 01:45:22 smithi083 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-9[51472]: 2024-08-19T01:45:22.347+0000 7f54e8c00640 -1 osd.9 89 heartbeat_check: no reply from 172.21.15.117:6812 osd.4 since back 2024-08-19T01:44:54.279278+0000 front 2024-08-19T01:44:54.279303+0000 (oldest deadline 2024-08-19T01:45:18.379436+0000) 2024-08-19T01:45:22.703 INFO:journalctl@ceph.osd.7.smithi117.stdout:Aug 19 01:45:22 smithi117 podman[56314]: 2024-08-19 01:45:22.502704344 +0000 UTC m=+0.492433639 container died 24ff36cf6535e300e2b1d9d97b37700d25e5d06587de194fc51f89305cb5d2f0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c9dd28080a631be25c77c36c63b17a3ef340ffb2e21c87d4d7b56b6398436d9, name=ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-7-deactivate, org.label-schema.license=GPLv2, io.buildah.version=1.36.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, RELEASE=squid-de09dd0, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True) 2024-08-19T01:45:22.703 INFO:journalctl@ceph.osd.10.smithi117.stdout:Aug 19 01:45:22 smithi117 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-10[46616]: 2024-08-19T01:45:22.485+0000 7f34b3bf7640 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-08-19T01:44:24.060751+0000 front 2024-08-19T01:44:24.060716+0000 (oldest deadline 2024-08-19T01:44:47.560677+0000) 2024-08-19T01:45:22.703 INFO:journalctl@ceph.osd.10.smithi117.stdout:Aug 19 01:45:22 smithi117 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-10[46616]: 2024-08-19T01:45:22.485+0000 7f34b3bf7640 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-08-19T01:44:28.661503+0000 front 2024-08-19T01:44:28.661522+0000 (oldest deadline 2024-08-19T01:44:56.262298+0000) 2024-08-19T01:45:22.703 INFO:journalctl@ceph.osd.10.smithi117.stdout:Aug 19 01:45:22 smithi117 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-10[46616]: 2024-08-19T01:45:22.485+0000 7f34b3bf7640 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-08-19T01:44:38.563674+0000 front 2024-08-19T01:44:38.563520+0000 (oldest deadline 2024-08-19T01:45:02.063517+0000) 2024-08-19T01:45:22.703 INFO:journalctl@ceph.osd.10.smithi117.stdout:Aug 19 01:45:22 smithi117 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-10[46616]: 2024-08-19T01:45:22.485+0000 7f34b3bf7640 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.83:6814 osd.3 since back 2024-08-19T01:44:47.265176+0000 front 2024-08-19T01:44:47.265102+0000 (oldest deadline 2024-08-19T01:45:09.564999+0000) 2024-08-19T01:45:22.703 INFO:journalctl@ceph.osd.10.smithi117.stdout:Aug 19 01:45:22 smithi117 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-10[46616]: 2024-08-19T01:45:22.485+0000 7f34b3bf7640 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.117:6812 osd.4 since back 2024-08-19T01:44:54.865666+0000 front 2024-08-19T01:44:54.865695+0000 (oldest deadline 2024-08-19T01:45:19.565735+0000) 2024-08-19T01:45:22.716 INFO:journalctl@ceph.osd.8.smithi167.stdout:Aug 19 01:45:22 smithi167 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-8[39928]: 2024-08-19T01:45:22.219+0000 7f17fe820640 -1 osd.8 89 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-08-19T01:44:22.747507+0000 front 2024-08-19T01:44:22.747516+0000 (oldest deadline 2024-08-19T01:44:48.047324+0000) 2024-08-19T01:45:22.716 INFO:journalctl@ceph.osd.8.smithi167.stdout:Aug 19 01:45:22 smithi167 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-8[39928]: 2024-08-19T01:45:22.219+0000 7f17fe820640 -1 osd.8 89 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-08-19T01:44:32.049249+0000 front 2024-08-19T01:44:32.049153+0000 (oldest deadline 2024-08-19T01:44:53.749126+0000) 2024-08-19T01:45:22.716 INFO:journalctl@ceph.osd.8.smithi167.stdout:Aug 19 01:45:22 smithi167 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-8[39928]: 2024-08-19T01:45:22.219+0000 7f17fe820640 -1 osd.8 89 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-08-19T01:44:36.050034+0000 front 2024-08-19T01:44:36.049973+0000 (oldest deadline 2024-08-19T01:45:06.050849+0000) 2024-08-19T01:45:22.716 INFO:journalctl@ceph.osd.8.smithi167.stdout:Aug 19 01:45:22 smithi167 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-8[39928]: 2024-08-19T01:45:22.219+0000 7f17fe820640 -1 osd.8 89 heartbeat_check: no reply from 172.21.15.117:6812 osd.4 since back 2024-08-19T01:44:56.552909+0000 front 2024-08-19T01:44:56.552897+0000 (oldest deadline 2024-08-19T01:45:20.652757+0000) 2024-08-19T01:45:23.090 DEBUG:teuthology.orchestra.run.smithi117:> sudo pkill -f 'journalctl -f -n 0 -u ceph-bad8780e-5dc9-11ef-bccf-c7b262605968@osd.7.service' 2024-08-19T01:45:23.168 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-08-19T01:45:23.168 INFO:tasks.cephadm.osd.7:Stopped osd.7 2024-08-19T01:45:23.168 INFO:tasks.cephadm.osd.8:Stopping osd.8... 2024-08-19T01:45:23.169 DEBUG:teuthology.orchestra.run.smithi167:> sudo systemctl stop ceph-bad8780e-5dc9-11ef-bccf-c7b262605968@osd.8 2024-08-19T01:45:23.198 INFO:journalctl@ceph.osd.11.smithi167.stdout:Aug 19 01:45:22 smithi167 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-11[46643]: 2024-08-19T01:45:22.717+0000 7f447f624640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-08-19T01:44:22.880210+0000 front 2024-08-19T01:44:22.880404+0000 (oldest deadline 2024-08-19T01:44:45.780352+0000) 2024-08-19T01:45:23.199 INFO:journalctl@ceph.osd.11.smithi167.stdout:Aug 19 01:45:22 smithi167 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-11[46643]: 2024-08-19T01:45:22.717+0000 7f447f624640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-08-19T01:44:31.081546+0000 front 2024-08-19T01:44:31.081654+0000 (oldest deadline 2024-08-19T01:44:53.981595+0000) 2024-08-19T01:45:23.199 INFO:journalctl@ceph.osd.11.smithi167.stdout:Aug 19 01:45:22 smithi167 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-11[46643]: 2024-08-19T01:45:22.717+0000 7f447f624640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-08-19T01:44:39.282638+0000 front 2024-08-19T01:44:39.282723+0000 (oldest deadline 2024-08-19T01:45:04.582709+0000) 2024-08-19T01:45:23.199 INFO:journalctl@ceph.osd.11.smithi167.stdout:Aug 19 01:45:22 smithi167 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-11[46643]: 2024-08-19T01:45:22.717+0000 7f447f624640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.83:6814 osd.3 since back 2024-08-19T01:44:45.083762+0000 front 2024-08-19T01:44:45.083777+0000 (oldest deadline 2024-08-19T01:45:10.383738+0000) 2024-08-19T01:45:23.199 INFO:journalctl@ceph.osd.11.smithi167.stdout:Aug 19 01:45:22 smithi167 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-11[46643]: 2024-08-19T01:45:22.717+0000 7f447f624640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.117:6812 osd.4 since back 2024-08-19T01:44:55.585644+0000 front 2024-08-19T01:44:55.585746+0000 (oldest deadline 2024-08-19T01:45:20.885631+0000) 2024-08-19T01:45:23.466 INFO:journalctl@ceph.osd.8.smithi167.stdout:Aug 19 01:45:23 smithi167 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-8[39928]: 2024-08-19T01:45:23.207+0000 7f17fe820640 -1 osd.8 89 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-08-19T01:44:22.747507+0000 front 2024-08-19T01:44:22.747516+0000 (oldest deadline 2024-08-19T01:44:48.047324+0000) 2024-08-19T01:45:23.466 INFO:journalctl@ceph.osd.8.smithi167.stdout:Aug 19 01:45:23 smithi167 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-8[39928]: 2024-08-19T01:45:23.207+0000 7f17fe820640 -1 osd.8 89 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-08-19T01:44:32.049249+0000 front 2024-08-19T01:44:32.049153+0000 (oldest deadline 2024-08-19T01:44:53.749126+0000) 2024-08-19T01:45:23.466 INFO:journalctl@ceph.osd.8.smithi167.stdout:Aug 19 01:45:23 smithi167 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-8[39928]: 2024-08-19T01:45:23.207+0000 7f17fe820640 -1 osd.8 89 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-08-19T01:44:36.050034+0000 front 2024-08-19T01:44:36.049973+0000 (oldest deadline 2024-08-19T01:45:06.050849+0000) 2024-08-19T01:45:23.466 INFO:journalctl@ceph.osd.8.smithi167.stdout:Aug 19 01:45:23 smithi167 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-8[39928]: 2024-08-19T01:45:23.207+0000 7f17fe820640 -1 osd.8 89 heartbeat_check: no reply from 172.21.15.117:6812 osd.4 since back 2024-08-19T01:44:56.552909+0000 front 2024-08-19T01:44:56.552897+0000 (oldest deadline 2024-08-19T01:45:20.652757+0000) 2024-08-19T01:45:23.466 INFO:journalctl@ceph.osd.8.smithi167.stdout:Aug 19 01:45:23 smithi167 systemd[1]: Stopping Ceph osd.8 for bad8780e-5dc9-11ef-bccf-c7b262605968... 2024-08-19T01:45:23.654 INFO:journalctl@ceph.osd.9.smithi083.stdout:Aug 19 01:45:23 smithi083 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-9[51472]: 2024-08-19T01:45:23.361+0000 7f54e8c00640 -1 osd.9 89 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-08-19T01:44:20.873282+0000 front 2024-08-19T01:44:20.873259+0000 (oldest deadline 2024-08-19T01:44:46.773413+0000) 2024-08-19T01:45:23.654 INFO:journalctl@ceph.osd.9.smithi083.stdout:Aug 19 01:45:23 smithi083 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-9[51472]: 2024-08-19T01:45:23.361+0000 7f54e8c00640 -1 osd.9 89 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-08-19T01:44:26.774104+0000 front 2024-08-19T01:44:26.774036+0000 (oldest deadline 2024-08-19T01:44:53.774823+0000) 2024-08-19T01:45:23.654 INFO:journalctl@ceph.osd.9.smithi083.stdout:Aug 19 01:45:23 smithi083 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-9[51472]: 2024-08-19T01:45:23.361+0000 7f54e8c00640 -1 osd.9 89 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-08-19T01:44:39.676617+0000 front 2024-08-19T01:44:39.676511+0000 (oldest deadline 2024-08-19T01:45:03.776684+0000) 2024-08-19T01:45:23.654 INFO:journalctl@ceph.osd.9.smithi083.stdout:Aug 19 01:45:23 smithi083 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-9[51472]: 2024-08-19T01:45:23.361+0000 7f54e8c00640 -1 osd.9 89 heartbeat_check: no reply from 172.21.15.83:6814 osd.3 since back 2024-08-19T01:44:46.678051+0000 front 2024-08-19T01:44:46.678168+0000 (oldest deadline 2024-08-19T01:45:11.378113+0000) 2024-08-19T01:45:23.655 INFO:journalctl@ceph.osd.9.smithi083.stdout:Aug 19 01:45:23 smithi083 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-9[51472]: 2024-08-19T01:45:23.361+0000 7f54e8c00640 -1 osd.9 89 heartbeat_check: no reply from 172.21.15.117:6812 osd.4 since back 2024-08-19T01:44:54.279278+0000 front 2024-08-19T01:44:54.279303+0000 (oldest deadline 2024-08-19T01:45:18.379436+0000) 2024-08-19T01:45:23.702 INFO:journalctl@ceph.osd.10.smithi117.stdout:Aug 19 01:45:23 smithi117 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-10[46616]: 2024-08-19T01:45:23.485+0000 7f34b3bf7640 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-08-19T01:44:24.060751+0000 front 2024-08-19T01:44:24.060716+0000 (oldest deadline 2024-08-19T01:44:47.560677+0000) 2024-08-19T01:45:23.702 INFO:journalctl@ceph.osd.10.smithi117.stdout:Aug 19 01:45:23 smithi117 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-10[46616]: 2024-08-19T01:45:23.485+0000 7f34b3bf7640 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-08-19T01:44:28.661503+0000 front 2024-08-19T01:44:28.661522+0000 (oldest deadline 2024-08-19T01:44:56.262298+0000) 2024-08-19T01:45:23.703 INFO:journalctl@ceph.osd.10.smithi117.stdout:Aug 19 01:45:23 smithi117 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-10[46616]: 2024-08-19T01:45:23.485+0000 7f34b3bf7640 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-08-19T01:44:38.563674+0000 front 2024-08-19T01:44:38.563520+0000 (oldest deadline 2024-08-19T01:45:02.063517+0000) 2024-08-19T01:45:23.703 INFO:journalctl@ceph.osd.10.smithi117.stdout:Aug 19 01:45:23 smithi117 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-10[46616]: 2024-08-19T01:45:23.485+0000 7f34b3bf7640 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.83:6814 osd.3 since back 2024-08-19T01:44:47.265176+0000 front 2024-08-19T01:44:47.265102+0000 (oldest deadline 2024-08-19T01:45:09.564999+0000) 2024-08-19T01:45:23.703 INFO:journalctl@ceph.osd.10.smithi117.stdout:Aug 19 01:45:23 smithi117 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-10[46616]: 2024-08-19T01:45:23.485+0000 7f34b3bf7640 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.117:6812 osd.4 since back 2024-08-19T01:44:54.865666+0000 front 2024-08-19T01:44:54.865695+0000 (oldest deadline 2024-08-19T01:45:19.565735+0000) 2024-08-19T01:45:23.748 INFO:journalctl@ceph.osd.8.smithi167.stdout:Aug 19 01:45:23 smithi167 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-8[39928]: 2024-08-19T01:45:23.482+0000 7f1805a22640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.8 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-08-19T01:45:23.749 INFO:journalctl@ceph.osd.8.smithi167.stdout:Aug 19 01:45:23 smithi167 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-8[39928]: 2024-08-19T01:45:23.482+0000 7f1805a22640 -1 osd.8 89 *** Got signal Terminated *** 2024-08-19T01:45:23.749 INFO:journalctl@ceph.osd.8.smithi167.stdout:Aug 19 01:45:23 smithi167 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-8[39928]: 2024-08-19T01:45:23.482+0000 7f1805a22640 -1 osd.8 89 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-08-19T01:45:24.198 INFO:journalctl@ceph.osd.11.smithi167.stdout:Aug 19 01:45:23 smithi167 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-11[46643]: 2024-08-19T01:45:23.748+0000 7f447f624640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-08-19T01:44:22.880210+0000 front 2024-08-19T01:44:22.880404+0000 (oldest deadline 2024-08-19T01:44:45.780352+0000) 2024-08-19T01:45:24.198 INFO:journalctl@ceph.osd.11.smithi167.stdout:Aug 19 01:45:23 smithi167 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-11[46643]: 2024-08-19T01:45:23.748+0000 7f447f624640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-08-19T01:44:31.081546+0000 front 2024-08-19T01:44:31.081654+0000 (oldest deadline 2024-08-19T01:44:53.981595+0000) 2024-08-19T01:45:24.198 INFO:journalctl@ceph.osd.11.smithi167.stdout:Aug 19 01:45:23 smithi167 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-11[46643]: 2024-08-19T01:45:23.748+0000 7f447f624640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-08-19T01:44:39.282638+0000 front 2024-08-19T01:44:39.282723+0000 (oldest deadline 2024-08-19T01:45:04.582709+0000) 2024-08-19T01:45:24.198 INFO:journalctl@ceph.osd.11.smithi167.stdout:Aug 19 01:45:23 smithi167 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-11[46643]: 2024-08-19T01:45:23.748+0000 7f447f624640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.83:6814 osd.3 since back 2024-08-19T01:44:45.083762+0000 front 2024-08-19T01:44:45.083777+0000 (oldest deadline 2024-08-19T01:45:10.383738+0000) 2024-08-19T01:45:24.198 INFO:journalctl@ceph.osd.11.smithi167.stdout:Aug 19 01:45:23 smithi167 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-11[46643]: 2024-08-19T01:45:23.748+0000 7f447f624640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.117:6812 osd.4 since back 2024-08-19T01:44:55.585644+0000 front 2024-08-19T01:44:55.585746+0000 (oldest deadline 2024-08-19T01:45:20.885631+0000) 2024-08-19T01:45:24.466 INFO:journalctl@ceph.osd.8.smithi167.stdout:Aug 19 01:45:24 smithi167 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-8[39928]: 2024-08-19T01:45:24.197+0000 7f17fe820640 -1 osd.8 89 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-08-19T01:44:22.747507+0000 front 2024-08-19T01:44:22.747516+0000 (oldest deadline 2024-08-19T01:44:48.047324+0000) 2024-08-19T01:45:24.466 INFO:journalctl@ceph.osd.8.smithi167.stdout:Aug 19 01:45:24 smithi167 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-8[39928]: 2024-08-19T01:45:24.197+0000 7f17fe820640 -1 osd.8 89 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-08-19T01:44:32.049249+0000 front 2024-08-19T01:44:32.049153+0000 (oldest deadline 2024-08-19T01:44:53.749126+0000) 2024-08-19T01:45:24.466 INFO:journalctl@ceph.osd.8.smithi167.stdout:Aug 19 01:45:24 smithi167 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-8[39928]: 2024-08-19T01:45:24.197+0000 7f17fe820640 -1 osd.8 89 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-08-19T01:44:36.050034+0000 front 2024-08-19T01:44:36.049973+0000 (oldest deadline 2024-08-19T01:45:06.050849+0000) 2024-08-19T01:45:24.466 INFO:journalctl@ceph.osd.8.smithi167.stdout:Aug 19 01:45:24 smithi167 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-8[39928]: 2024-08-19T01:45:24.197+0000 7f17fe820640 -1 osd.8 89 heartbeat_check: no reply from 172.21.15.117:6812 osd.4 since back 2024-08-19T01:44:56.552909+0000 front 2024-08-19T01:44:56.552897+0000 (oldest deadline 2024-08-19T01:45:20.652757+0000) 2024-08-19T01:45:24.654 INFO:journalctl@ceph.osd.9.smithi083.stdout:Aug 19 01:45:24 smithi083 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-9[51472]: 2024-08-19T01:45:24.393+0000 7f54e8c00640 -1 osd.9 89 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-08-19T01:44:20.873282+0000 front 2024-08-19T01:44:20.873259+0000 (oldest deadline 2024-08-19T01:44:46.773413+0000) 2024-08-19T01:45:24.654 INFO:journalctl@ceph.osd.9.smithi083.stdout:Aug 19 01:45:24 smithi083 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-9[51472]: 2024-08-19T01:45:24.393+0000 7f54e8c00640 -1 osd.9 89 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-08-19T01:44:26.774104+0000 front 2024-08-19T01:44:26.774036+0000 (oldest deadline 2024-08-19T01:44:53.774823+0000) 2024-08-19T01:45:24.654 INFO:journalctl@ceph.osd.9.smithi083.stdout:Aug 19 01:45:24 smithi083 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-9[51472]: 2024-08-19T01:45:24.393+0000 7f54e8c00640 -1 osd.9 89 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-08-19T01:44:39.676617+0000 front 2024-08-19T01:44:39.676511+0000 (oldest deadline 2024-08-19T01:45:03.776684+0000) 2024-08-19T01:45:24.655 INFO:journalctl@ceph.osd.9.smithi083.stdout:Aug 19 01:45:24 smithi083 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-9[51472]: 2024-08-19T01:45:24.393+0000 7f54e8c00640 -1 osd.9 89 heartbeat_check: no reply from 172.21.15.83:6814 osd.3 since back 2024-08-19T01:44:46.678051+0000 front 2024-08-19T01:44:46.678168+0000 (oldest deadline 2024-08-19T01:45:11.378113+0000) 2024-08-19T01:45:24.655 INFO:journalctl@ceph.osd.9.smithi083.stdout:Aug 19 01:45:24 smithi083 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-9[51472]: 2024-08-19T01:45:24.393+0000 7f54e8c00640 -1 osd.9 89 heartbeat_check: no reply from 172.21.15.117:6812 osd.4 since back 2024-08-19T01:44:54.279278+0000 front 2024-08-19T01:44:54.279303+0000 (oldest deadline 2024-08-19T01:45:18.379436+0000) 2024-08-19T01:45:24.702 INFO:journalctl@ceph.osd.10.smithi117.stdout:Aug 19 01:45:24 smithi117 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-10[46616]: 2024-08-19T01:45:24.476+0000 7f34b3bf7640 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-08-19T01:44:24.060751+0000 front 2024-08-19T01:44:24.060716+0000 (oldest deadline 2024-08-19T01:44:47.560677+0000) 2024-08-19T01:45:24.702 INFO:journalctl@ceph.osd.10.smithi117.stdout:Aug 19 01:45:24 smithi117 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-10[46616]: 2024-08-19T01:45:24.476+0000 7f34b3bf7640 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-08-19T01:44:28.661503+0000 front 2024-08-19T01:44:28.661522+0000 (oldest deadline 2024-08-19T01:44:56.262298+0000) 2024-08-19T01:45:24.702 INFO:journalctl@ceph.osd.10.smithi117.stdout:Aug 19 01:45:24 smithi117 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-10[46616]: 2024-08-19T01:45:24.476+0000 7f34b3bf7640 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-08-19T01:44:38.563674+0000 front 2024-08-19T01:44:38.563520+0000 (oldest deadline 2024-08-19T01:45:02.063517+0000) 2024-08-19T01:45:24.703 INFO:journalctl@ceph.osd.10.smithi117.stdout:Aug 19 01:45:24 smithi117 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-10[46616]: 2024-08-19T01:45:24.476+0000 7f34b3bf7640 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.83:6814 osd.3 since back 2024-08-19T01:44:47.265176+0000 front 2024-08-19T01:44:47.265102+0000 (oldest deadline 2024-08-19T01:45:09.564999+0000) 2024-08-19T01:45:24.703 INFO:journalctl@ceph.osd.10.smithi117.stdout:Aug 19 01:45:24 smithi117 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-10[46616]: 2024-08-19T01:45:24.476+0000 7f34b3bf7640 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.117:6812 osd.4 since back 2024-08-19T01:44:54.865666+0000 front 2024-08-19T01:44:54.865695+0000 (oldest deadline 2024-08-19T01:45:19.565735+0000) 2024-08-19T01:45:24.966 INFO:journalctl@ceph.osd.11.smithi167.stdout:Aug 19 01:45:24 smithi167 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-11[46643]: 2024-08-19T01:45:24.711+0000 7f447f624640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-08-19T01:44:22.880210+0000 front 2024-08-19T01:44:22.880404+0000 (oldest deadline 2024-08-19T01:44:45.780352+0000) 2024-08-19T01:45:24.966 INFO:journalctl@ceph.osd.11.smithi167.stdout:Aug 19 01:45:24 smithi167 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-11[46643]: 2024-08-19T01:45:24.711+0000 7f447f624640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-08-19T01:44:31.081546+0000 front 2024-08-19T01:44:31.081654+0000 (oldest deadline 2024-08-19T01:44:53.981595+0000) 2024-08-19T01:45:24.966 INFO:journalctl@ceph.osd.11.smithi167.stdout:Aug 19 01:45:24 smithi167 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-11[46643]: 2024-08-19T01:45:24.711+0000 7f447f624640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-08-19T01:44:39.282638+0000 front 2024-08-19T01:44:39.282723+0000 (oldest deadline 2024-08-19T01:45:04.582709+0000) 2024-08-19T01:45:24.966 INFO:journalctl@ceph.osd.11.smithi167.stdout:Aug 19 01:45:24 smithi167 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-11[46643]: 2024-08-19T01:45:24.711+0000 7f447f624640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.83:6814 osd.3 since back 2024-08-19T01:44:45.083762+0000 front 2024-08-19T01:44:45.083777+0000 (oldest deadline 2024-08-19T01:45:10.383738+0000) 2024-08-19T01:45:24.966 INFO:journalctl@ceph.osd.11.smithi167.stdout:Aug 19 01:45:24 smithi167 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-11[46643]: 2024-08-19T01:45:24.711+0000 7f447f624640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.117:6812 osd.4 since back 2024-08-19T01:44:55.585644+0000 front 2024-08-19T01:44:55.585746+0000 (oldest deadline 2024-08-19T01:45:20.885631+0000) 2024-08-19T01:45:25.654 INFO:journalctl@ceph.osd.9.smithi083.stdout:Aug 19 01:45:25 smithi083 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-9[51472]: 2024-08-19T01:45:25.379+0000 7f54e8c00640 -1 osd.9 89 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-08-19T01:44:20.873282+0000 front 2024-08-19T01:44:20.873259+0000 (oldest deadline 2024-08-19T01:44:46.773413+0000) 2024-08-19T01:45:25.654 INFO:journalctl@ceph.osd.9.smithi083.stdout:Aug 19 01:45:25 smithi083 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-9[51472]: 2024-08-19T01:45:25.379+0000 7f54e8c00640 -1 osd.9 89 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-08-19T01:44:26.774104+0000 front 2024-08-19T01:44:26.774036+0000 (oldest deadline 2024-08-19T01:44:53.774823+0000) 2024-08-19T01:45:25.655 INFO:journalctl@ceph.osd.9.smithi083.stdout:Aug 19 01:45:25 smithi083 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-9[51472]: 2024-08-19T01:45:25.379+0000 7f54e8c00640 -1 osd.9 89 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-08-19T01:44:39.676617+0000 front 2024-08-19T01:44:39.676511+0000 (oldest deadline 2024-08-19T01:45:03.776684+0000) 2024-08-19T01:45:25.655 INFO:journalctl@ceph.osd.9.smithi083.stdout:Aug 19 01:45:25 smithi083 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-9[51472]: 2024-08-19T01:45:25.379+0000 7f54e8c00640 -1 osd.9 89 heartbeat_check: no reply from 172.21.15.83:6814 osd.3 since back 2024-08-19T01:44:46.678051+0000 front 2024-08-19T01:44:46.678168+0000 (oldest deadline 2024-08-19T01:45:11.378113+0000) 2024-08-19T01:45:25.655 INFO:journalctl@ceph.osd.9.smithi083.stdout:Aug 19 01:45:25 smithi083 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-9[51472]: 2024-08-19T01:45:25.379+0000 7f54e8c00640 -1 osd.9 89 heartbeat_check: no reply from 172.21.15.117:6812 osd.4 since back 2024-08-19T01:44:54.279278+0000 front 2024-08-19T01:44:54.279303+0000 (oldest deadline 2024-08-19T01:45:18.379436+0000) 2024-08-19T01:45:25.666 INFO:journalctl@ceph.osd.8.smithi167.stdout:Aug 19 01:45:25 smithi167 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-8[39928]: 2024-08-19T01:45:25.232+0000 7f17fe820640 -1 osd.8 89 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-08-19T01:44:22.747507+0000 front 2024-08-19T01:44:22.747516+0000 (oldest deadline 2024-08-19T01:44:48.047324+0000) 2024-08-19T01:45:25.666 INFO:journalctl@ceph.osd.8.smithi167.stdout:Aug 19 01:45:25 smithi167 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-8[39928]: 2024-08-19T01:45:25.232+0000 7f17fe820640 -1 osd.8 89 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-08-19T01:44:32.049249+0000 front 2024-08-19T01:44:32.049153+0000 (oldest deadline 2024-08-19T01:44:53.749126+0000) 2024-08-19T01:45:25.667 INFO:journalctl@ceph.osd.8.smithi167.stdout:Aug 19 01:45:25 smithi167 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-8[39928]: 2024-08-19T01:45:25.232+0000 7f17fe820640 -1 osd.8 89 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-08-19T01:44:36.050034+0000 front 2024-08-19T01:44:36.049973+0000 (oldest deadline 2024-08-19T01:45:06.050849+0000) 2024-08-19T01:45:25.667 INFO:journalctl@ceph.osd.8.smithi167.stdout:Aug 19 01:45:25 smithi167 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-8[39928]: 2024-08-19T01:45:25.232+0000 7f17fe820640 -1 osd.8 89 heartbeat_check: no reply from 172.21.15.117:6812 osd.4 since back 2024-08-19T01:44:56.552909+0000 front 2024-08-19T01:44:56.552897+0000 (oldest deadline 2024-08-19T01:45:20.652757+0000) 2024-08-19T01:45:25.952 INFO:journalctl@ceph.osd.10.smithi117.stdout:Aug 19 01:45:25 smithi117 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-10[46616]: 2024-08-19T01:45:25.524+0000 7f34b3bf7640 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-08-19T01:44:24.060751+0000 front 2024-08-19T01:44:24.060716+0000 (oldest deadline 2024-08-19T01:44:47.560677+0000) 2024-08-19T01:45:25.952 INFO:journalctl@ceph.osd.10.smithi117.stdout:Aug 19 01:45:25 smithi117 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-10[46616]: 2024-08-19T01:45:25.524+0000 7f34b3bf7640 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-08-19T01:44:28.661503+0000 front 2024-08-19T01:44:28.661522+0000 (oldest deadline 2024-08-19T01:44:56.262298+0000) 2024-08-19T01:45:25.953 INFO:journalctl@ceph.osd.10.smithi117.stdout:Aug 19 01:45:25 smithi117 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-10[46616]: 2024-08-19T01:45:25.524+0000 7f34b3bf7640 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-08-19T01:44:38.563674+0000 front 2024-08-19T01:44:38.563520+0000 (oldest deadline 2024-08-19T01:45:02.063517+0000) 2024-08-19T01:45:25.953 INFO:journalctl@ceph.osd.10.smithi117.stdout:Aug 19 01:45:25 smithi117 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-10[46616]: 2024-08-19T01:45:25.524+0000 7f34b3bf7640 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.83:6814 osd.3 since back 2024-08-19T01:44:47.265176+0000 front 2024-08-19T01:44:47.265102+0000 (oldest deadline 2024-08-19T01:45:09.564999+0000) 2024-08-19T01:45:25.953 INFO:journalctl@ceph.osd.10.smithi117.stdout:Aug 19 01:45:25 smithi117 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-10[46616]: 2024-08-19T01:45:25.524+0000 7f34b3bf7640 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.117:6812 osd.4 since back 2024-08-19T01:44:54.865666+0000 front 2024-08-19T01:44:54.865695+0000 (oldest deadline 2024-08-19T01:45:19.565735+0000) 2024-08-19T01:45:25.953 INFO:journalctl@ceph.osd.10.smithi117.stdout:Aug 19 01:45:25 smithi117 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-10[46616]: 2024-08-19T01:45:25.524+0000 7f34b3bf7640 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.167:6812 osd.5 since back 2024-08-19T01:45:03.066733+0000 front 2024-08-19T01:45:03.066769+0000 (oldest deadline 2024-08-19T01:45:25.366703+0000) 2024-08-19T01:45:25.965 INFO:journalctl@ceph.osd.11.smithi167.stdout:Aug 19 01:45:25 smithi167 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-11[46643]: 2024-08-19T01:45:25.666+0000 7f447f624640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-08-19T01:44:22.880210+0000 front 2024-08-19T01:44:22.880404+0000 (oldest deadline 2024-08-19T01:44:45.780352+0000) 2024-08-19T01:45:25.966 INFO:journalctl@ceph.osd.11.smithi167.stdout:Aug 19 01:45:25 smithi167 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-11[46643]: 2024-08-19T01:45:25.666+0000 7f447f624640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-08-19T01:44:31.081546+0000 front 2024-08-19T01:44:31.081654+0000 (oldest deadline 2024-08-19T01:44:53.981595+0000) 2024-08-19T01:45:25.966 INFO:journalctl@ceph.osd.11.smithi167.stdout:Aug 19 01:45:25 smithi167 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-11[46643]: 2024-08-19T01:45:25.666+0000 7f447f624640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-08-19T01:44:39.282638+0000 front 2024-08-19T01:44:39.282723+0000 (oldest deadline 2024-08-19T01:45:04.582709+0000) 2024-08-19T01:45:25.966 INFO:journalctl@ceph.osd.11.smithi167.stdout:Aug 19 01:45:25 smithi167 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-11[46643]: 2024-08-19T01:45:25.666+0000 7f447f624640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.83:6814 osd.3 since back 2024-08-19T01:44:45.083762+0000 front 2024-08-19T01:44:45.083777+0000 (oldest deadline 2024-08-19T01:45:10.383738+0000) 2024-08-19T01:45:25.966 INFO:journalctl@ceph.osd.11.smithi167.stdout:Aug 19 01:45:25 smithi167 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-11[46643]: 2024-08-19T01:45:25.666+0000 7f447f624640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.117:6812 osd.4 since back 2024-08-19T01:44:55.585644+0000 front 2024-08-19T01:44:55.585746+0000 (oldest deadline 2024-08-19T01:45:20.885631+0000) 2024-08-19T01:45:26.404 INFO:journalctl@ceph.osd.9.smithi083.stdout:Aug 19 01:45:26 smithi083 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-9[51472]: 2024-08-19T01:45:26.343+0000 7f54e8c00640 -1 osd.9 89 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-08-19T01:44:20.873282+0000 front 2024-08-19T01:44:20.873259+0000 (oldest deadline 2024-08-19T01:44:46.773413+0000) 2024-08-19T01:45:26.405 INFO:journalctl@ceph.osd.9.smithi083.stdout:Aug 19 01:45:26 smithi083 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-9[51472]: 2024-08-19T01:45:26.343+0000 7f54e8c00640 -1 osd.9 89 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-08-19T01:44:26.774104+0000 front 2024-08-19T01:44:26.774036+0000 (oldest deadline 2024-08-19T01:44:53.774823+0000) 2024-08-19T01:45:26.405 INFO:journalctl@ceph.osd.9.smithi083.stdout:Aug 19 01:45:26 smithi083 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-9[51472]: 2024-08-19T01:45:26.343+0000 7f54e8c00640 -1 osd.9 89 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-08-19T01:44:39.676617+0000 front 2024-08-19T01:44:39.676511+0000 (oldest deadline 2024-08-19T01:45:03.776684+0000) 2024-08-19T01:45:26.405 INFO:journalctl@ceph.osd.9.smithi083.stdout:Aug 19 01:45:26 smithi083 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-9[51472]: 2024-08-19T01:45:26.343+0000 7f54e8c00640 -1 osd.9 89 heartbeat_check: no reply from 172.21.15.83:6814 osd.3 since back 2024-08-19T01:44:46.678051+0000 front 2024-08-19T01:44:46.678168+0000 (oldest deadline 2024-08-19T01:45:11.378113+0000) 2024-08-19T01:45:26.405 INFO:journalctl@ceph.osd.9.smithi083.stdout:Aug 19 01:45:26 smithi083 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-9[51472]: 2024-08-19T01:45:26.343+0000 7f54e8c00640 -1 osd.9 89 heartbeat_check: no reply from 172.21.15.117:6812 osd.4 since back 2024-08-19T01:44:54.279278+0000 front 2024-08-19T01:44:54.279303+0000 (oldest deadline 2024-08-19T01:45:18.379436+0000) 2024-08-19T01:45:26.405 INFO:journalctl@ceph.osd.9.smithi083.stdout:Aug 19 01:45:26 smithi083 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-9[51472]: 2024-08-19T01:45:26.343+0000 7f54e8c00640 -1 osd.9 89 heartbeat_check: no reply from 172.21.15.167:6812 osd.5 since back 2024-08-19T01:45:04.181394+0000 front 2024-08-19T01:45:04.181366+0000 (oldest deadline 2024-08-19T01:45:25.881307+0000) 2024-08-19T01:45:26.466 INFO:journalctl@ceph.osd.8.smithi167.stdout:Aug 19 01:45:26 smithi167 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-8[39928]: 2024-08-19T01:45:26.204+0000 7f17fe820640 -1 osd.8 89 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-08-19T01:44:22.747507+0000 front 2024-08-19T01:44:22.747516+0000 (oldest deadline 2024-08-19T01:44:48.047324+0000) 2024-08-19T01:45:26.466 INFO:journalctl@ceph.osd.8.smithi167.stdout:Aug 19 01:45:26 smithi167 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-8[39928]: 2024-08-19T01:45:26.204+0000 7f17fe820640 -1 osd.8 89 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-08-19T01:44:32.049249+0000 front 2024-08-19T01:44:32.049153+0000 (oldest deadline 2024-08-19T01:44:53.749126+0000) 2024-08-19T01:45:26.467 INFO:journalctl@ceph.osd.8.smithi167.stdout:Aug 19 01:45:26 smithi167 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-8[39928]: 2024-08-19T01:45:26.204+0000 7f17fe820640 -1 osd.8 89 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-08-19T01:44:36.050034+0000 front 2024-08-19T01:44:36.049973+0000 (oldest deadline 2024-08-19T01:45:06.050849+0000) 2024-08-19T01:45:26.467 INFO:journalctl@ceph.osd.8.smithi167.stdout:Aug 19 01:45:26 smithi167 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-8[39928]: 2024-08-19T01:45:26.204+0000 7f17fe820640 -1 osd.8 89 heartbeat_check: no reply from 172.21.15.117:6812 osd.4 since back 2024-08-19T01:44:56.552909+0000 front 2024-08-19T01:44:56.552897+0000 (oldest deadline 2024-08-19T01:45:20.652757+0000) 2024-08-19T01:45:26.854 INFO:journalctl@ceph.osd.10.smithi117.stdout:Aug 19 01:45:26 smithi117 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-10[46616]: 2024-08-19T01:45:26.523+0000 7f34b3bf7640 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-08-19T01:44:24.060751+0000 front 2024-08-19T01:44:24.060716+0000 (oldest deadline 2024-08-19T01:44:47.560677+0000) 2024-08-19T01:45:26.854 INFO:journalctl@ceph.osd.10.smithi117.stdout:Aug 19 01:45:26 smithi117 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-10[46616]: 2024-08-19T01:45:26.523+0000 7f34b3bf7640 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-08-19T01:44:28.661503+0000 front 2024-08-19T01:44:28.661522+0000 (oldest deadline 2024-08-19T01:44:56.262298+0000) 2024-08-19T01:45:26.854 INFO:journalctl@ceph.osd.10.smithi117.stdout:Aug 19 01:45:26 smithi117 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-10[46616]: 2024-08-19T01:45:26.523+0000 7f34b3bf7640 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-08-19T01:44:38.563674+0000 front 2024-08-19T01:44:38.563520+0000 (oldest deadline 2024-08-19T01:45:02.063517+0000) 2024-08-19T01:45:26.854 INFO:journalctl@ceph.osd.10.smithi117.stdout:Aug 19 01:45:26 smithi117 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-10[46616]: 2024-08-19T01:45:26.523+0000 7f34b3bf7640 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.83:6814 osd.3 since back 2024-08-19T01:44:47.265176+0000 front 2024-08-19T01:44:47.265102+0000 (oldest deadline 2024-08-19T01:45:09.564999+0000) 2024-08-19T01:45:26.854 INFO:journalctl@ceph.osd.10.smithi117.stdout:Aug 19 01:45:26 smithi117 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-10[46616]: 2024-08-19T01:45:26.523+0000 7f34b3bf7640 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.117:6812 osd.4 since back 2024-08-19T01:44:54.865666+0000 front 2024-08-19T01:44:54.865695+0000 (oldest deadline 2024-08-19T01:45:19.565735+0000) 2024-08-19T01:45:26.854 INFO:journalctl@ceph.osd.10.smithi117.stdout:Aug 19 01:45:26 smithi117 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-10[46616]: 2024-08-19T01:45:26.523+0000 7f34b3bf7640 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.167:6812 osd.5 since back 2024-08-19T01:45:03.066733+0000 front 2024-08-19T01:45:03.066769+0000 (oldest deadline 2024-08-19T01:45:25.366703+0000) 2024-08-19T01:45:26.966 INFO:journalctl@ceph.osd.11.smithi167.stdout:Aug 19 01:45:26 smithi167 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-11[46643]: 2024-08-19T01:45:26.682+0000 7f447f624640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-08-19T01:44:22.880210+0000 front 2024-08-19T01:44:22.880404+0000 (oldest deadline 2024-08-19T01:44:45.780352+0000) 2024-08-19T01:45:26.966 INFO:journalctl@ceph.osd.11.smithi167.stdout:Aug 19 01:45:26 smithi167 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-11[46643]: 2024-08-19T01:45:26.682+0000 7f447f624640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-08-19T01:44:31.081546+0000 front 2024-08-19T01:44:31.081654+0000 (oldest deadline 2024-08-19T01:44:53.981595+0000) 2024-08-19T01:45:26.966 INFO:journalctl@ceph.osd.11.smithi167.stdout:Aug 19 01:45:26 smithi167 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-11[46643]: 2024-08-19T01:45:26.682+0000 7f447f624640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-08-19T01:44:39.282638+0000 front 2024-08-19T01:44:39.282723+0000 (oldest deadline 2024-08-19T01:45:04.582709+0000) 2024-08-19T01:45:26.966 INFO:journalctl@ceph.osd.11.smithi167.stdout:Aug 19 01:45:26 smithi167 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-11[46643]: 2024-08-19T01:45:26.682+0000 7f447f624640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.83:6814 osd.3 since back 2024-08-19T01:44:45.083762+0000 front 2024-08-19T01:44:45.083777+0000 (oldest deadline 2024-08-19T01:45:10.383738+0000) 2024-08-19T01:45:26.966 INFO:journalctl@ceph.osd.11.smithi167.stdout:Aug 19 01:45:26 smithi167 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-11[46643]: 2024-08-19T01:45:26.682+0000 7f447f624640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.117:6812 osd.4 since back 2024-08-19T01:44:55.585644+0000 front 2024-08-19T01:44:55.585746+0000 (oldest deadline 2024-08-19T01:45:20.885631+0000) 2024-08-19T01:45:27.466 INFO:journalctl@ceph.osd.8.smithi167.stdout:Aug 19 01:45:27 smithi167 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-8[39928]: 2024-08-19T01:45:27.173+0000 7f17fe820640 -1 osd.8 89 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-08-19T01:44:22.747507+0000 front 2024-08-19T01:44:22.747516+0000 (oldest deadline 2024-08-19T01:44:48.047324+0000) 2024-08-19T01:45:27.466 INFO:journalctl@ceph.osd.8.smithi167.stdout:Aug 19 01:45:27 smithi167 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-8[39928]: 2024-08-19T01:45:27.173+0000 7f17fe820640 -1 osd.8 89 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-08-19T01:44:32.049249+0000 front 2024-08-19T01:44:32.049153+0000 (oldest deadline 2024-08-19T01:44:53.749126+0000) 2024-08-19T01:45:27.466 INFO:journalctl@ceph.osd.8.smithi167.stdout:Aug 19 01:45:27 smithi167 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-8[39928]: 2024-08-19T01:45:27.173+0000 7f17fe820640 -1 osd.8 89 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-08-19T01:44:36.050034+0000 front 2024-08-19T01:44:36.049973+0000 (oldest deadline 2024-08-19T01:45:06.050849+0000) 2024-08-19T01:45:27.466 INFO:journalctl@ceph.osd.8.smithi167.stdout:Aug 19 01:45:27 smithi167 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-8[39928]: 2024-08-19T01:45:27.173+0000 7f17fe820640 -1 osd.8 89 heartbeat_check: no reply from 172.21.15.117:6812 osd.4 since back 2024-08-19T01:44:56.552909+0000 front 2024-08-19T01:44:56.552897+0000 (oldest deadline 2024-08-19T01:45:20.652757+0000) 2024-08-19T01:45:27.654 INFO:journalctl@ceph.osd.9.smithi083.stdout:Aug 19 01:45:27 smithi083 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-9[51472]: 2024-08-19T01:45:27.293+0000 7f54e8c00640 -1 osd.9 89 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-08-19T01:44:20.873282+0000 front 2024-08-19T01:44:20.873259+0000 (oldest deadline 2024-08-19T01:44:46.773413+0000) 2024-08-19T01:45:27.655 INFO:journalctl@ceph.osd.9.smithi083.stdout:Aug 19 01:45:27 smithi083 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-9[51472]: 2024-08-19T01:45:27.293+0000 7f54e8c00640 -1 osd.9 89 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-08-19T01:44:26.774104+0000 front 2024-08-19T01:44:26.774036+0000 (oldest deadline 2024-08-19T01:44:53.774823+0000) 2024-08-19T01:45:27.655 INFO:journalctl@ceph.osd.9.smithi083.stdout:Aug 19 01:45:27 smithi083 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-9[51472]: 2024-08-19T01:45:27.293+0000 7f54e8c00640 -1 osd.9 89 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-08-19T01:44:39.676617+0000 front 2024-08-19T01:44:39.676511+0000 (oldest deadline 2024-08-19T01:45:03.776684+0000) 2024-08-19T01:45:27.655 INFO:journalctl@ceph.osd.9.smithi083.stdout:Aug 19 01:45:27 smithi083 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-9[51472]: 2024-08-19T01:45:27.293+0000 7f54e8c00640 -1 osd.9 89 heartbeat_check: no reply from 172.21.15.83:6814 osd.3 since back 2024-08-19T01:44:46.678051+0000 front 2024-08-19T01:44:46.678168+0000 (oldest deadline 2024-08-19T01:45:11.378113+0000) 2024-08-19T01:45:27.655 INFO:journalctl@ceph.osd.9.smithi083.stdout:Aug 19 01:45:27 smithi083 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-9[51472]: 2024-08-19T01:45:27.293+0000 7f54e8c00640 -1 osd.9 89 heartbeat_check: no reply from 172.21.15.117:6812 osd.4 since back 2024-08-19T01:44:54.279278+0000 front 2024-08-19T01:44:54.279303+0000 (oldest deadline 2024-08-19T01:45:18.379436+0000) 2024-08-19T01:45:27.655 INFO:journalctl@ceph.osd.9.smithi083.stdout:Aug 19 01:45:27 smithi083 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-9[51472]: 2024-08-19T01:45:27.293+0000 7f54e8c00640 -1 osd.9 89 heartbeat_check: no reply from 172.21.15.167:6812 osd.5 since back 2024-08-19T01:45:04.181394+0000 front 2024-08-19T01:45:04.181366+0000 (oldest deadline 2024-08-19T01:45:25.881307+0000) 2024-08-19T01:45:27.952 INFO:journalctl@ceph.osd.10.smithi117.stdout:Aug 19 01:45:27 smithi117 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-10[46616]: 2024-08-19T01:45:27.573+0000 7f34b3bf7640 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-08-19T01:44:24.060751+0000 front 2024-08-19T01:44:24.060716+0000 (oldest deadline 2024-08-19T01:44:47.560677+0000) 2024-08-19T01:45:27.953 INFO:journalctl@ceph.osd.10.smithi117.stdout:Aug 19 01:45:27 smithi117 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-10[46616]: 2024-08-19T01:45:27.573+0000 7f34b3bf7640 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-08-19T01:44:28.661503+0000 front 2024-08-19T01:44:28.661522+0000 (oldest deadline 2024-08-19T01:44:56.262298+0000) 2024-08-19T01:45:27.953 INFO:journalctl@ceph.osd.10.smithi117.stdout:Aug 19 01:45:27 smithi117 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-10[46616]: 2024-08-19T01:45:27.573+0000 7f34b3bf7640 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-08-19T01:44:38.563674+0000 front 2024-08-19T01:44:38.563520+0000 (oldest deadline 2024-08-19T01:45:02.063517+0000) 2024-08-19T01:45:27.953 INFO:journalctl@ceph.osd.10.smithi117.stdout:Aug 19 01:45:27 smithi117 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-10[46616]: 2024-08-19T01:45:27.573+0000 7f34b3bf7640 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.83:6814 osd.3 since back 2024-08-19T01:44:47.265176+0000 front 2024-08-19T01:44:47.265102+0000 (oldest deadline 2024-08-19T01:45:09.564999+0000) 2024-08-19T01:45:27.953 INFO:journalctl@ceph.osd.10.smithi117.stdout:Aug 19 01:45:27 smithi117 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-10[46616]: 2024-08-19T01:45:27.573+0000 7f34b3bf7640 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.117:6812 osd.4 since back 2024-08-19T01:44:54.865666+0000 front 2024-08-19T01:44:54.865695+0000 (oldest deadline 2024-08-19T01:45:19.565735+0000) 2024-08-19T01:45:27.953 INFO:journalctl@ceph.osd.10.smithi117.stdout:Aug 19 01:45:27 smithi117 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-10[46616]: 2024-08-19T01:45:27.573+0000 7f34b3bf7640 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.167:6812 osd.5 since back 2024-08-19T01:45:03.066733+0000 front 2024-08-19T01:45:03.066769+0000 (oldest deadline 2024-08-19T01:45:25.366703+0000) 2024-08-19T01:45:27.966 INFO:journalctl@ceph.osd.11.smithi167.stdout:Aug 19 01:45:27 smithi167 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-11[46643]: 2024-08-19T01:45:27.633+0000 7f447f624640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-08-19T01:44:22.880210+0000 front 2024-08-19T01:44:22.880404+0000 (oldest deadline 2024-08-19T01:44:45.780352+0000) 2024-08-19T01:45:27.966 INFO:journalctl@ceph.osd.11.smithi167.stdout:Aug 19 01:45:27 smithi167 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-11[46643]: 2024-08-19T01:45:27.633+0000 7f447f624640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-08-19T01:44:31.081546+0000 front 2024-08-19T01:44:31.081654+0000 (oldest deadline 2024-08-19T01:44:53.981595+0000) 2024-08-19T01:45:27.966 INFO:journalctl@ceph.osd.11.smithi167.stdout:Aug 19 01:45:27 smithi167 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-11[46643]: 2024-08-19T01:45:27.633+0000 7f447f624640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-08-19T01:44:39.282638+0000 front 2024-08-19T01:44:39.282723+0000 (oldest deadline 2024-08-19T01:45:04.582709+0000) 2024-08-19T01:45:27.966 INFO:journalctl@ceph.osd.11.smithi167.stdout:Aug 19 01:45:27 smithi167 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-11[46643]: 2024-08-19T01:45:27.633+0000 7f447f624640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.83:6814 osd.3 since back 2024-08-19T01:44:45.083762+0000 front 2024-08-19T01:44:45.083777+0000 (oldest deadline 2024-08-19T01:45:10.383738+0000) 2024-08-19T01:45:27.966 INFO:journalctl@ceph.osd.11.smithi167.stdout:Aug 19 01:45:27 smithi167 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-11[46643]: 2024-08-19T01:45:27.633+0000 7f447f624640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.117:6812 osd.4 since back 2024-08-19T01:44:55.585644+0000 front 2024-08-19T01:44:55.585746+0000 (oldest deadline 2024-08-19T01:45:20.885631+0000) 2024-08-19T01:45:28.466 INFO:journalctl@ceph.osd.8.smithi167.stdout:Aug 19 01:45:28 smithi167 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-8[39928]: 2024-08-19T01:45:28.210+0000 7f17fe820640 -1 osd.8 89 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-08-19T01:44:22.747507+0000 front 2024-08-19T01:44:22.747516+0000 (oldest deadline 2024-08-19T01:44:48.047324+0000) 2024-08-19T01:45:28.466 INFO:journalctl@ceph.osd.8.smithi167.stdout:Aug 19 01:45:28 smithi167 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-8[39928]: 2024-08-19T01:45:28.210+0000 7f17fe820640 -1 osd.8 89 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-08-19T01:44:32.049249+0000 front 2024-08-19T01:44:32.049153+0000 (oldest deadline 2024-08-19T01:44:53.749126+0000) 2024-08-19T01:45:28.466 INFO:journalctl@ceph.osd.8.smithi167.stdout:Aug 19 01:45:28 smithi167 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-8[39928]: 2024-08-19T01:45:28.210+0000 7f17fe820640 -1 osd.8 89 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-08-19T01:44:36.050034+0000 front 2024-08-19T01:44:36.049973+0000 (oldest deadline 2024-08-19T01:45:06.050849+0000) 2024-08-19T01:45:28.466 INFO:journalctl@ceph.osd.8.smithi167.stdout:Aug 19 01:45:28 smithi167 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-8[39928]: 2024-08-19T01:45:28.210+0000 7f17fe820640 -1 osd.8 89 heartbeat_check: no reply from 172.21.15.117:6812 osd.4 since back 2024-08-19T01:44:56.552909+0000 front 2024-08-19T01:44:56.552897+0000 (oldest deadline 2024-08-19T01:45:20.652757+0000) 2024-08-19T01:45:28.654 INFO:journalctl@ceph.osd.9.smithi083.stdout:Aug 19 01:45:28 smithi083 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-9[51472]: 2024-08-19T01:45:28.248+0000 7f54e8c00640 -1 osd.9 89 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-08-19T01:44:20.873282+0000 front 2024-08-19T01:44:20.873259+0000 (oldest deadline 2024-08-19T01:44:46.773413+0000) 2024-08-19T01:45:28.654 INFO:journalctl@ceph.osd.9.smithi083.stdout:Aug 19 01:45:28 smithi083 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-9[51472]: 2024-08-19T01:45:28.248+0000 7f54e8c00640 -1 osd.9 89 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-08-19T01:44:26.774104+0000 front 2024-08-19T01:44:26.774036+0000 (oldest deadline 2024-08-19T01:44:53.774823+0000) 2024-08-19T01:45:28.655 INFO:journalctl@ceph.osd.9.smithi083.stdout:Aug 19 01:45:28 smithi083 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-9[51472]: 2024-08-19T01:45:28.248+0000 7f54e8c00640 -1 osd.9 89 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-08-19T01:44:39.676617+0000 front 2024-08-19T01:44:39.676511+0000 (oldest deadline 2024-08-19T01:45:03.776684+0000) 2024-08-19T01:45:28.655 INFO:journalctl@ceph.osd.9.smithi083.stdout:Aug 19 01:45:28 smithi083 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-9[51472]: 2024-08-19T01:45:28.248+0000 7f54e8c00640 -1 osd.9 89 heartbeat_check: no reply from 172.21.15.83:6814 osd.3 since back 2024-08-19T01:44:46.678051+0000 front 2024-08-19T01:44:46.678168+0000 (oldest deadline 2024-08-19T01:45:11.378113+0000) 2024-08-19T01:45:28.655 INFO:journalctl@ceph.osd.9.smithi083.stdout:Aug 19 01:45:28 smithi083 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-9[51472]: 2024-08-19T01:45:28.248+0000 7f54e8c00640 -1 osd.9 89 heartbeat_check: no reply from 172.21.15.117:6812 osd.4 since back 2024-08-19T01:44:54.279278+0000 front 2024-08-19T01:44:54.279303+0000 (oldest deadline 2024-08-19T01:45:18.379436+0000) 2024-08-19T01:45:28.655 INFO:journalctl@ceph.osd.9.smithi083.stdout:Aug 19 01:45:28 smithi083 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-9[51472]: 2024-08-19T01:45:28.248+0000 7f54e8c00640 -1 osd.9 89 heartbeat_check: no reply from 172.21.15.167:6812 osd.5 since back 2024-08-19T01:45:04.181394+0000 front 2024-08-19T01:45:04.181366+0000 (oldest deadline 2024-08-19T01:45:25.881307+0000) 2024-08-19T01:45:28.907 INFO:journalctl@ceph.osd.11.smithi167.stdout:Aug 19 01:45:28 smithi167 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-11[46643]: 2024-08-19T01:45:28.609+0000 7f447f624640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-08-19T01:44:22.880210+0000 front 2024-08-19T01:44:22.880404+0000 (oldest deadline 2024-08-19T01:44:45.780352+0000) 2024-08-19T01:45:28.907 INFO:journalctl@ceph.osd.11.smithi167.stdout:Aug 19 01:45:28 smithi167 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-11[46643]: 2024-08-19T01:45:28.609+0000 7f447f624640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-08-19T01:44:31.081546+0000 front 2024-08-19T01:44:31.081654+0000 (oldest deadline 2024-08-19T01:44:53.981595+0000) 2024-08-19T01:45:28.907 INFO:journalctl@ceph.osd.11.smithi167.stdout:Aug 19 01:45:28 smithi167 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-11[46643]: 2024-08-19T01:45:28.609+0000 7f447f624640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-08-19T01:44:39.282638+0000 front 2024-08-19T01:44:39.282723+0000 (oldest deadline 2024-08-19T01:45:04.582709+0000) 2024-08-19T01:45:28.907 INFO:journalctl@ceph.osd.11.smithi167.stdout:Aug 19 01:45:28 smithi167 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-11[46643]: 2024-08-19T01:45:28.609+0000 7f447f624640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.83:6814 osd.3 since back 2024-08-19T01:44:45.083762+0000 front 2024-08-19T01:44:45.083777+0000 (oldest deadline 2024-08-19T01:45:10.383738+0000) 2024-08-19T01:45:28.907 INFO:journalctl@ceph.osd.11.smithi167.stdout:Aug 19 01:45:28 smithi167 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-11[46643]: 2024-08-19T01:45:28.609+0000 7f447f624640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.117:6812 osd.4 since back 2024-08-19T01:44:55.585644+0000 front 2024-08-19T01:44:55.585746+0000 (oldest deadline 2024-08-19T01:45:20.885631+0000) 2024-08-19T01:45:28.907 INFO:journalctl@ceph.osd.11.smithi167.stdout:Aug 19 01:45:28 smithi167 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-11[46643]: 2024-08-19T01:45:28.609+0000 7f447f624640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.167:6812 osd.5 since back 2024-08-19T01:45:02.586329+0000 front 2024-08-19T01:45:02.586392+0000 (oldest deadline 2024-08-19T01:45:28.486376+0000) 2024-08-19T01:45:28.952 INFO:journalctl@ceph.osd.10.smithi117.stdout:Aug 19 01:45:28 smithi117 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-10[46616]: 2024-08-19T01:45:28.538+0000 7f34b3bf7640 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-08-19T01:44:24.060751+0000 front 2024-08-19T01:44:24.060716+0000 (oldest deadline 2024-08-19T01:44:47.560677+0000) 2024-08-19T01:45:28.952 INFO:journalctl@ceph.osd.10.smithi117.stdout:Aug 19 01:45:28 smithi117 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-10[46616]: 2024-08-19T01:45:28.538+0000 7f34b3bf7640 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-08-19T01:44:28.661503+0000 front 2024-08-19T01:44:28.661522+0000 (oldest deadline 2024-08-19T01:44:56.262298+0000) 2024-08-19T01:45:28.952 INFO:journalctl@ceph.osd.10.smithi117.stdout:Aug 19 01:45:28 smithi117 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-10[46616]: 2024-08-19T01:45:28.538+0000 7f34b3bf7640 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-08-19T01:44:38.563674+0000 front 2024-08-19T01:44:38.563520+0000 (oldest deadline 2024-08-19T01:45:02.063517+0000) 2024-08-19T01:45:28.952 INFO:journalctl@ceph.osd.10.smithi117.stdout:Aug 19 01:45:28 smithi117 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-10[46616]: 2024-08-19T01:45:28.538+0000 7f34b3bf7640 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.83:6814 osd.3 since back 2024-08-19T01:44:47.265176+0000 front 2024-08-19T01:44:47.265102+0000 (oldest deadline 2024-08-19T01:45:09.564999+0000) 2024-08-19T01:45:28.953 INFO:journalctl@ceph.osd.10.smithi117.stdout:Aug 19 01:45:28 smithi117 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-10[46616]: 2024-08-19T01:45:28.538+0000 7f34b3bf7640 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.117:6812 osd.4 since back 2024-08-19T01:44:54.865666+0000 front 2024-08-19T01:44:54.865695+0000 (oldest deadline 2024-08-19T01:45:19.565735+0000) 2024-08-19T01:45:28.953 INFO:journalctl@ceph.osd.10.smithi117.stdout:Aug 19 01:45:28 smithi117 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-10[46616]: 2024-08-19T01:45:28.538+0000 7f34b3bf7640 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.167:6812 osd.5 since back 2024-08-19T01:45:03.066733+0000 front 2024-08-19T01:45:03.066769+0000 (oldest deadline 2024-08-19T01:45:25.366703+0000) 2024-08-19T01:45:29.192 INFO:journalctl@ceph.osd.8.smithi167.stdout:Aug 19 01:45:28 smithi167 podman[57009]: 2024-08-19 01:45:28.905999059 +0000 UTC m=+5.528581943 container died f2b1c182317cee0949d7562bc55b181aaf0a31e446338c1fcb283eea9ee93f22 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c9dd28080a631be25c77c36c63b17a3ef340ffb2e21c87d4d7b56b6398436d9, name=ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-8, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, io.buildah.version=1.36.0, RELEASE=squid-de09dd0, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.license=GPLv2) 2024-08-19T01:45:29.465 INFO:journalctl@ceph.osd.8.smithi167.stdout:Aug 19 01:45:29 smithi167 podman[57009]: 2024-08-19 01:45:29.26565767 +0000 UTC m=+5.888240550 container cleanup f2b1c182317cee0949d7562bc55b181aaf0a31e446338c1fcb283eea9ee93f22 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c9dd28080a631be25c77c36c63b17a3ef340ffb2e21c87d4d7b56b6398436d9, name=ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-8, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, RELEASE=squid-de09dd0, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, io.buildah.version=1.36.0) 2024-08-19T01:45:29.466 INFO:journalctl@ceph.osd.8.smithi167.stdout:Aug 19 01:45:29 smithi167 bash[57009]: ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-8 2024-08-19T01:45:29.654 INFO:journalctl@ceph.osd.9.smithi083.stdout:Aug 19 01:45:29 smithi083 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-9[51472]: 2024-08-19T01:45:29.268+0000 7f54e8c00640 -1 osd.9 89 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-08-19T01:44:20.873282+0000 front 2024-08-19T01:44:20.873259+0000 (oldest deadline 2024-08-19T01:44:46.773413+0000) 2024-08-19T01:45:29.654 INFO:journalctl@ceph.osd.9.smithi083.stdout:Aug 19 01:45:29 smithi083 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-9[51472]: 2024-08-19T01:45:29.268+0000 7f54e8c00640 -1 osd.9 89 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-08-19T01:44:26.774104+0000 front 2024-08-19T01:44:26.774036+0000 (oldest deadline 2024-08-19T01:44:53.774823+0000) 2024-08-19T01:45:29.654 INFO:journalctl@ceph.osd.9.smithi083.stdout:Aug 19 01:45:29 smithi083 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-9[51472]: 2024-08-19T01:45:29.268+0000 7f54e8c00640 -1 osd.9 89 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-08-19T01:44:39.676617+0000 front 2024-08-19T01:44:39.676511+0000 (oldest deadline 2024-08-19T01:45:03.776684+0000) 2024-08-19T01:45:29.655 INFO:journalctl@ceph.osd.9.smithi083.stdout:Aug 19 01:45:29 smithi083 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-9[51472]: 2024-08-19T01:45:29.268+0000 7f54e8c00640 -1 osd.9 89 heartbeat_check: no reply from 172.21.15.83:6814 osd.3 since back 2024-08-19T01:44:46.678051+0000 front 2024-08-19T01:44:46.678168+0000 (oldest deadline 2024-08-19T01:45:11.378113+0000) 2024-08-19T01:45:29.655 INFO:journalctl@ceph.osd.9.smithi083.stdout:Aug 19 01:45:29 smithi083 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-9[51472]: 2024-08-19T01:45:29.268+0000 7f54e8c00640 -1 osd.9 89 heartbeat_check: no reply from 172.21.15.117:6812 osd.4 since back 2024-08-19T01:44:54.279278+0000 front 2024-08-19T01:44:54.279303+0000 (oldest deadline 2024-08-19T01:45:18.379436+0000) 2024-08-19T01:45:29.655 INFO:journalctl@ceph.osd.9.smithi083.stdout:Aug 19 01:45:29 smithi083 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-9[51472]: 2024-08-19T01:45:29.268+0000 7f54e8c00640 -1 osd.9 89 heartbeat_check: no reply from 172.21.15.167:6812 osd.5 since back 2024-08-19T01:45:04.181394+0000 front 2024-08-19T01:45:04.181366+0000 (oldest deadline 2024-08-19T01:45:25.881307+0000) 2024-08-19T01:45:29.893 INFO:journalctl@ceph.osd.8.smithi167.stdout:Aug 19 01:45:29 smithi167 podman[57024]: 2024-08-19 01:45:29.567085106 +0000 UTC m=+0.655871082 container remove f2b1c182317cee0949d7562bc55b181aaf0a31e446338c1fcb283eea9ee93f22 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c9dd28080a631be25c77c36c63b17a3ef340ffb2e21c87d4d7b56b6398436d9, name=ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-8, ceph=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , io.buildah.version=1.36.0, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-de09dd0, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image) 2024-08-19T01:45:29.893 INFO:journalctl@ceph.osd.11.smithi167.stdout:Aug 19 01:45:29 smithi167 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-11[46643]: 2024-08-19T01:45:29.652+0000 7f447f624640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-08-19T01:44:22.880210+0000 front 2024-08-19T01:44:22.880404+0000 (oldest deadline 2024-08-19T01:44:45.780352+0000) 2024-08-19T01:45:29.893 INFO:journalctl@ceph.osd.11.smithi167.stdout:Aug 19 01:45:29 smithi167 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-11[46643]: 2024-08-19T01:45:29.652+0000 7f447f624640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-08-19T01:44:31.081546+0000 front 2024-08-19T01:44:31.081654+0000 (oldest deadline 2024-08-19T01:44:53.981595+0000) 2024-08-19T01:45:29.893 INFO:journalctl@ceph.osd.11.smithi167.stdout:Aug 19 01:45:29 smithi167 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-11[46643]: 2024-08-19T01:45:29.652+0000 7f447f624640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-08-19T01:44:39.282638+0000 front 2024-08-19T01:44:39.282723+0000 (oldest deadline 2024-08-19T01:45:04.582709+0000) 2024-08-19T01:45:29.893 INFO:journalctl@ceph.osd.11.smithi167.stdout:Aug 19 01:45:29 smithi167 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-11[46643]: 2024-08-19T01:45:29.653+0000 7f447f624640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.83:6814 osd.3 since back 2024-08-19T01:44:45.083762+0000 front 2024-08-19T01:44:45.083777+0000 (oldest deadline 2024-08-19T01:45:10.383738+0000) 2024-08-19T01:45:29.893 INFO:journalctl@ceph.osd.11.smithi167.stdout:Aug 19 01:45:29 smithi167 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-11[46643]: 2024-08-19T01:45:29.653+0000 7f447f624640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.117:6812 osd.4 since back 2024-08-19T01:44:55.585644+0000 front 2024-08-19T01:44:55.585746+0000 (oldest deadline 2024-08-19T01:45:20.885631+0000) 2024-08-19T01:45:29.894 INFO:journalctl@ceph.osd.11.smithi167.stdout:Aug 19 01:45:29 smithi167 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-11[46643]: 2024-08-19T01:45:29.653+0000 7f447f624640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.167:6812 osd.5 since back 2024-08-19T01:45:02.586329+0000 front 2024-08-19T01:45:02.586392+0000 (oldest deadline 2024-08-19T01:45:28.486376+0000) 2024-08-19T01:45:29.952 INFO:journalctl@ceph.osd.10.smithi117.stdout:Aug 19 01:45:29 smithi117 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-10[46616]: 2024-08-19T01:45:29.522+0000 7f34b3bf7640 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-08-19T01:44:24.060751+0000 front 2024-08-19T01:44:24.060716+0000 (oldest deadline 2024-08-19T01:44:47.560677+0000) 2024-08-19T01:45:29.952 INFO:journalctl@ceph.osd.10.smithi117.stdout:Aug 19 01:45:29 smithi117 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-10[46616]: 2024-08-19T01:45:29.522+0000 7f34b3bf7640 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-08-19T01:44:28.661503+0000 front 2024-08-19T01:44:28.661522+0000 (oldest deadline 2024-08-19T01:44:56.262298+0000) 2024-08-19T01:45:29.952 INFO:journalctl@ceph.osd.10.smithi117.stdout:Aug 19 01:45:29 smithi117 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-10[46616]: 2024-08-19T01:45:29.522+0000 7f34b3bf7640 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-08-19T01:44:38.563674+0000 front 2024-08-19T01:44:38.563520+0000 (oldest deadline 2024-08-19T01:45:02.063517+0000) 2024-08-19T01:45:29.952 INFO:journalctl@ceph.osd.10.smithi117.stdout:Aug 19 01:45:29 smithi117 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-10[46616]: 2024-08-19T01:45:29.522+0000 7f34b3bf7640 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.83:6814 osd.3 since back 2024-08-19T01:44:47.265176+0000 front 2024-08-19T01:44:47.265102+0000 (oldest deadline 2024-08-19T01:45:09.564999+0000) 2024-08-19T01:45:29.952 INFO:journalctl@ceph.osd.10.smithi117.stdout:Aug 19 01:45:29 smithi117 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-10[46616]: 2024-08-19T01:45:29.522+0000 7f34b3bf7640 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.117:6812 osd.4 since back 2024-08-19T01:44:54.865666+0000 front 2024-08-19T01:44:54.865695+0000 (oldest deadline 2024-08-19T01:45:19.565735+0000) 2024-08-19T01:45:29.953 INFO:journalctl@ceph.osd.10.smithi117.stdout:Aug 19 01:45:29 smithi117 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-10[46616]: 2024-08-19T01:45:29.522+0000 7f34b3bf7640 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.167:6812 osd.5 since back 2024-08-19T01:45:03.066733+0000 front 2024-08-19T01:45:03.066769+0000 (oldest deadline 2024-08-19T01:45:25.366703+0000) 2024-08-19T01:45:30.216 INFO:journalctl@ceph.osd.8.smithi167.stdout:Aug 19 01:45:29 smithi167 podman[57111]: 2024-08-19 01:45:29.892875574 +0000 UTC m=+0.111221316 container create a3ded800d258e837b270cad2c332c677ac78f5154f8ae60017b2d378f9254f6c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c9dd28080a631be25c77c36c63b17a3ef340ffb2e21c87d4d7b56b6398436d9, name=ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-8-deactivate, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.36.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, RELEASE=squid-de09dd0, org.label-schema.license=GPLv2, ceph=True, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0) 2024-08-19T01:45:30.216 INFO:journalctl@ceph.osd.8.smithi167.stdout:Aug 19 01:45:29 smithi167 podman[57111]: 2024-08-19 01:45:29.798936225 +0000 UTC m=+0.017281984 image pull ea7a5189f30bad3aca9524788e2965d2368d7ce786df09d4edcd8c3edf9cefba quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c9dd28080a631be25c77c36c63b17a3ef340ffb2e21c87d4d7b56b6398436d9 2024-08-19T01:45:30.216 INFO:journalctl@ceph.osd.8.smithi167.stdout:Aug 19 01:45:30 smithi167 podman[57111]: 2024-08-19 01:45:30.136950672 +0000 UTC m=+0.355296430 container init a3ded800d258e837b270cad2c332c677ac78f5154f8ae60017b2d378f9254f6c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c9dd28080a631be25c77c36c63b17a3ef340ffb2e21c87d4d7b56b6398436d9, name=ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-8-deactivate, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, io.buildah.version=1.36.0, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=squid-de09dd0) 2024-08-19T01:45:30.216 INFO:journalctl@ceph.osd.8.smithi167.stdout:Aug 19 01:45:30 smithi167 podman[57111]: 2024-08-19 01:45:30.140393959 +0000 UTC m=+0.358739705 container start a3ded800d258e837b270cad2c332c677ac78f5154f8ae60017b2d378f9254f6c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c9dd28080a631be25c77c36c63b17a3ef340ffb2e21c87d4d7b56b6398436d9, name=ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-8-deactivate, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=squid-de09dd0, GIT_CLEAN=True, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_BRANCH=HEAD, io.buildah.version=1.36.0, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0) 2024-08-19T01:45:30.605 INFO:journalctl@ceph.osd.8.smithi167.stdout:Aug 19 01:45:30 smithi167 podman[57111]: 2024-08-19 01:45:30.220175685 +0000 UTC m=+0.438521447 container attach a3ded800d258e837b270cad2c332c677ac78f5154f8ae60017b2d378f9254f6c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c9dd28080a631be25c77c36c63b17a3ef340ffb2e21c87d4d7b56b6398436d9, name=ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-8-deactivate, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, io.buildah.version=1.36.0, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , RELEASE=squid-de09dd0, ceph=True, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-08-19T01:45:30.605 INFO:journalctl@ceph.osd.8.smithi167.stdout:Aug 19 01:45:30 smithi167 podman[57111]: 2024-08-19 01:45:30.326057966 +0000 UTC m=+0.544403717 container died a3ded800d258e837b270cad2c332c677ac78f5154f8ae60017b2d378f9254f6c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c9dd28080a631be25c77c36c63b17a3ef340ffb2e21c87d4d7b56b6398436d9, name=ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-8-deactivate, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_CLEAN=True, RELEASE=squid-de09dd0, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, io.buildah.version=1.36.0) 2024-08-19T01:45:30.655 INFO:journalctl@ceph.osd.9.smithi083.stdout:Aug 19 01:45:30 smithi083 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-9[51472]: 2024-08-19T01:45:30.251+0000 7f54e8c00640 -1 osd.9 89 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-08-19T01:44:20.873282+0000 front 2024-08-19T01:44:20.873259+0000 (oldest deadline 2024-08-19T01:44:46.773413+0000) 2024-08-19T01:45:30.655 INFO:journalctl@ceph.osd.9.smithi083.stdout:Aug 19 01:45:30 smithi083 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-9[51472]: 2024-08-19T01:45:30.251+0000 7f54e8c00640 -1 osd.9 89 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-08-19T01:44:26.774104+0000 front 2024-08-19T01:44:26.774036+0000 (oldest deadline 2024-08-19T01:44:53.774823+0000) 2024-08-19T01:45:30.655 INFO:journalctl@ceph.osd.9.smithi083.stdout:Aug 19 01:45:30 smithi083 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-9[51472]: 2024-08-19T01:45:30.251+0000 7f54e8c00640 -1 osd.9 89 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-08-19T01:44:39.676617+0000 front 2024-08-19T01:44:39.676511+0000 (oldest deadline 2024-08-19T01:45:03.776684+0000) 2024-08-19T01:45:30.655 INFO:journalctl@ceph.osd.9.smithi083.stdout:Aug 19 01:45:30 smithi083 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-9[51472]: 2024-08-19T01:45:30.251+0000 7f54e8c00640 -1 osd.9 89 heartbeat_check: no reply from 172.21.15.83:6814 osd.3 since back 2024-08-19T01:44:46.678051+0000 front 2024-08-19T01:44:46.678168+0000 (oldest deadline 2024-08-19T01:45:11.378113+0000) 2024-08-19T01:45:30.655 INFO:journalctl@ceph.osd.9.smithi083.stdout:Aug 19 01:45:30 smithi083 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-9[51472]: 2024-08-19T01:45:30.251+0000 7f54e8c00640 -1 osd.9 89 heartbeat_check: no reply from 172.21.15.117:6812 osd.4 since back 2024-08-19T01:44:54.279278+0000 front 2024-08-19T01:44:54.279303+0000 (oldest deadline 2024-08-19T01:45:18.379436+0000) 2024-08-19T01:45:30.655 INFO:journalctl@ceph.osd.9.smithi083.stdout:Aug 19 01:45:30 smithi083 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-9[51472]: 2024-08-19T01:45:30.251+0000 7f54e8c00640 -1 osd.9 89 heartbeat_check: no reply from 172.21.15.167:6812 osd.5 since back 2024-08-19T01:45:04.181394+0000 front 2024-08-19T01:45:04.181366+0000 (oldest deadline 2024-08-19T01:45:25.881307+0000) 2024-08-19T01:45:30.855 INFO:journalctl@ceph.osd.8.smithi167.stdout:Aug 19 01:45:30 smithi167 podman[57134]: 2024-08-19 01:45:30.840241846 +0000 UTC m=+0.508070649 container remove a3ded800d258e837b270cad2c332c677ac78f5154f8ae60017b2d378f9254f6c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c9dd28080a631be25c77c36c63b17a3ef340ffb2e21c87d4d7b56b6398436d9, name=ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-8-deactivate, ceph=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, RELEASE=squid-de09dd0, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, io.buildah.version=1.36.0) 2024-08-19T01:45:30.855 INFO:journalctl@ceph.osd.8.smithi167.stdout:Aug 19 01:45:30 smithi167 systemd[1]: ceph-bad8780e-5dc9-11ef-bccf-c7b262605968@osd.8.service: Deactivated successfully. 2024-08-19T01:45:30.856 INFO:journalctl@ceph.osd.8.smithi167.stdout:Aug 19 01:45:30 smithi167 systemd[1]: Stopped Ceph osd.8 for bad8780e-5dc9-11ef-bccf-c7b262605968. 2024-08-19T01:45:30.856 INFO:journalctl@ceph.osd.8.smithi167.stdout:Aug 19 01:45:30 smithi167 systemd[1]: ceph-bad8780e-5dc9-11ef-bccf-c7b262605968@osd.8.service: Consumed 1min 21.161s CPU time. 2024-08-19T01:45:30.856 INFO:journalctl@ceph.osd.11.smithi167.stdout:Aug 19 01:45:30 smithi167 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-11[46643]: 2024-08-19T01:45:30.604+0000 7f447f624640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-08-19T01:44:22.880210+0000 front 2024-08-19T01:44:22.880404+0000 (oldest deadline 2024-08-19T01:44:45.780352+0000) 2024-08-19T01:45:30.856 INFO:journalctl@ceph.osd.11.smithi167.stdout:Aug 19 01:45:30 smithi167 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-11[46643]: 2024-08-19T01:45:30.604+0000 7f447f624640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-08-19T01:44:31.081546+0000 front 2024-08-19T01:44:31.081654+0000 (oldest deadline 2024-08-19T01:44:53.981595+0000) 2024-08-19T01:45:30.856 INFO:journalctl@ceph.osd.11.smithi167.stdout:Aug 19 01:45:30 smithi167 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-11[46643]: 2024-08-19T01:45:30.604+0000 7f447f624640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-08-19T01:44:39.282638+0000 front 2024-08-19T01:44:39.282723+0000 (oldest deadline 2024-08-19T01:45:04.582709+0000) 2024-08-19T01:45:30.856 INFO:journalctl@ceph.osd.11.smithi167.stdout:Aug 19 01:45:30 smithi167 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-11[46643]: 2024-08-19T01:45:30.604+0000 7f447f624640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.83:6814 osd.3 since back 2024-08-19T01:44:45.083762+0000 front 2024-08-19T01:44:45.083777+0000 (oldest deadline 2024-08-19T01:45:10.383738+0000) 2024-08-19T01:45:30.856 INFO:journalctl@ceph.osd.11.smithi167.stdout:Aug 19 01:45:30 smithi167 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-11[46643]: 2024-08-19T01:45:30.604+0000 7f447f624640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.117:6812 osd.4 since back 2024-08-19T01:44:55.585644+0000 front 2024-08-19T01:44:55.585746+0000 (oldest deadline 2024-08-19T01:45:20.885631+0000) 2024-08-19T01:45:30.856 INFO:journalctl@ceph.osd.11.smithi167.stdout:Aug 19 01:45:30 smithi167 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-11[46643]: 2024-08-19T01:45:30.604+0000 7f447f624640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.167:6812 osd.5 since back 2024-08-19T01:45:02.586329+0000 front 2024-08-19T01:45:02.586392+0000 (oldest deadline 2024-08-19T01:45:28.486376+0000) 2024-08-19T01:45:30.858 DEBUG:teuthology.orchestra.run.smithi167:> sudo pkill -f 'journalctl -f -n 0 -u ceph-bad8780e-5dc9-11ef-bccf-c7b262605968@osd.8.service' 2024-08-19T01:45:30.937 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-08-19T01:45:30.938 INFO:tasks.cephadm.osd.8:Stopped osd.8 2024-08-19T01:45:30.938 INFO:tasks.cephadm.osd.9:Stopping osd.9... 2024-08-19T01:45:30.938 DEBUG:teuthology.orchestra.run.smithi083:> sudo systemctl stop ceph-bad8780e-5dc9-11ef-bccf-c7b262605968@osd.9 2024-08-19T01:45:30.952 INFO:journalctl@ceph.osd.10.smithi117.stdout:Aug 19 01:45:30 smithi117 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-10[46616]: 2024-08-19T01:45:30.492+0000 7f34b3bf7640 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-08-19T01:44:24.060751+0000 front 2024-08-19T01:44:24.060716+0000 (oldest deadline 2024-08-19T01:44:47.560677+0000) 2024-08-19T01:45:30.952 INFO:journalctl@ceph.osd.10.smithi117.stdout:Aug 19 01:45:30 smithi117 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-10[46616]: 2024-08-19T01:45:30.492+0000 7f34b3bf7640 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-08-19T01:44:28.661503+0000 front 2024-08-19T01:44:28.661522+0000 (oldest deadline 2024-08-19T01:44:56.262298+0000) 2024-08-19T01:45:30.952 INFO:journalctl@ceph.osd.10.smithi117.stdout:Aug 19 01:45:30 smithi117 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-10[46616]: 2024-08-19T01:45:30.492+0000 7f34b3bf7640 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-08-19T01:44:38.563674+0000 front 2024-08-19T01:44:38.563520+0000 (oldest deadline 2024-08-19T01:45:02.063517+0000) 2024-08-19T01:45:30.952 INFO:journalctl@ceph.osd.10.smithi117.stdout:Aug 19 01:45:30 smithi117 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-10[46616]: 2024-08-19T01:45:30.492+0000 7f34b3bf7640 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.83:6814 osd.3 since back 2024-08-19T01:44:47.265176+0000 front 2024-08-19T01:44:47.265102+0000 (oldest deadline 2024-08-19T01:45:09.564999+0000) 2024-08-19T01:45:30.952 INFO:journalctl@ceph.osd.10.smithi117.stdout:Aug 19 01:45:30 smithi117 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-10[46616]: 2024-08-19T01:45:30.492+0000 7f34b3bf7640 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.117:6812 osd.4 since back 2024-08-19T01:44:54.865666+0000 front 2024-08-19T01:44:54.865695+0000 (oldest deadline 2024-08-19T01:45:19.565735+0000) 2024-08-19T01:45:30.952 INFO:journalctl@ceph.osd.10.smithi117.stdout:Aug 19 01:45:30 smithi117 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-10[46616]: 2024-08-19T01:45:30.492+0000 7f34b3bf7640 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.167:6812 osd.5 since back 2024-08-19T01:45:03.066733+0000 front 2024-08-19T01:45:03.066769+0000 (oldest deadline 2024-08-19T01:45:25.366703+0000) 2024-08-19T01:45:31.347 INFO:journalctl@ceph.osd.9.smithi083.stdout:Aug 19 01:45:31 smithi083 systemd[1]: Stopping Ceph osd.9 for bad8780e-5dc9-11ef-bccf-c7b262605968... 2024-08-19T01:45:31.347 INFO:journalctl@ceph.osd.9.smithi083.stdout:Aug 19 01:45:31 smithi083 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-9[51472]: 2024-08-19T01:45:31.227+0000 7f54e8c00640 -1 osd.9 89 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-08-19T01:44:20.873282+0000 front 2024-08-19T01:44:20.873259+0000 (oldest deadline 2024-08-19T01:44:46.773413+0000) 2024-08-19T01:45:31.347 INFO:journalctl@ceph.osd.9.smithi083.stdout:Aug 19 01:45:31 smithi083 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-9[51472]: 2024-08-19T01:45:31.227+0000 7f54e8c00640 -1 osd.9 89 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-08-19T01:44:26.774104+0000 front 2024-08-19T01:44:26.774036+0000 (oldest deadline 2024-08-19T01:44:53.774823+0000) 2024-08-19T01:45:31.347 INFO:journalctl@ceph.osd.9.smithi083.stdout:Aug 19 01:45:31 smithi083 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-9[51472]: 2024-08-19T01:45:31.227+0000 7f54e8c00640 -1 osd.9 89 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-08-19T01:44:39.676617+0000 front 2024-08-19T01:44:39.676511+0000 (oldest deadline 2024-08-19T01:45:03.776684+0000) 2024-08-19T01:45:31.348 INFO:journalctl@ceph.osd.9.smithi083.stdout:Aug 19 01:45:31 smithi083 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-9[51472]: 2024-08-19T01:45:31.227+0000 7f54e8c00640 -1 osd.9 89 heartbeat_check: no reply from 172.21.15.83:6814 osd.3 since back 2024-08-19T01:44:46.678051+0000 front 2024-08-19T01:44:46.678168+0000 (oldest deadline 2024-08-19T01:45:11.378113+0000) 2024-08-19T01:45:31.348 INFO:journalctl@ceph.osd.9.smithi083.stdout:Aug 19 01:45:31 smithi083 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-9[51472]: 2024-08-19T01:45:31.227+0000 7f54e8c00640 -1 osd.9 89 heartbeat_check: no reply from 172.21.15.117:6812 osd.4 since back 2024-08-19T01:44:54.279278+0000 front 2024-08-19T01:44:54.279303+0000 (oldest deadline 2024-08-19T01:45:18.379436+0000) 2024-08-19T01:45:31.348 INFO:journalctl@ceph.osd.9.smithi083.stdout:Aug 19 01:45:31 smithi083 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-9[51472]: 2024-08-19T01:45:31.227+0000 7f54e8c00640 -1 osd.9 89 heartbeat_check: no reply from 172.21.15.167:6812 osd.5 since back 2024-08-19T01:45:04.181394+0000 front 2024-08-19T01:45:04.181366+0000 (oldest deadline 2024-08-19T01:45:25.881307+0000) 2024-08-19T01:45:31.348 INFO:journalctl@ceph.osd.9.smithi083.stdout:Aug 19 01:45:31 smithi083 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-9[51472]: 2024-08-19T01:45:31.345+0000 7f54f1800640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.9 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-08-19T01:45:31.348 INFO:journalctl@ceph.osd.9.smithi083.stdout:Aug 19 01:45:31 smithi083 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-9[51472]: 2024-08-19T01:45:31.345+0000 7f54f1800640 -1 osd.9 89 *** Got signal Terminated *** 2024-08-19T01:45:31.348 INFO:journalctl@ceph.osd.9.smithi083.stdout:Aug 19 01:45:31 smithi083 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-9[51472]: 2024-08-19T01:45:31.345+0000 7f54f1800640 -1 osd.9 89 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-08-19T01:45:31.707 INFO:journalctl@ceph.osd.11.smithi167.stdout:Aug 19 01:45:31 smithi167 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-11[46643]: 2024-08-19T01:45:31.593+0000 7f447f624640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-08-19T01:44:22.880210+0000 front 2024-08-19T01:44:22.880404+0000 (oldest deadline 2024-08-19T01:44:45.780352+0000) 2024-08-19T01:45:31.707 INFO:journalctl@ceph.osd.11.smithi167.stdout:Aug 19 01:45:31 smithi167 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-11[46643]: 2024-08-19T01:45:31.593+0000 7f447f624640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-08-19T01:44:31.081546+0000 front 2024-08-19T01:44:31.081654+0000 (oldest deadline 2024-08-19T01:44:53.981595+0000) 2024-08-19T01:45:31.707 INFO:journalctl@ceph.osd.11.smithi167.stdout:Aug 19 01:45:31 smithi167 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-11[46643]: 2024-08-19T01:45:31.593+0000 7f447f624640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-08-19T01:44:39.282638+0000 front 2024-08-19T01:44:39.282723+0000 (oldest deadline 2024-08-19T01:45:04.582709+0000) 2024-08-19T01:45:31.708 INFO:journalctl@ceph.osd.11.smithi167.stdout:Aug 19 01:45:31 smithi167 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-11[46643]: 2024-08-19T01:45:31.593+0000 7f447f624640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.83:6814 osd.3 since back 2024-08-19T01:44:45.083762+0000 front 2024-08-19T01:44:45.083777+0000 (oldest deadline 2024-08-19T01:45:10.383738+0000) 2024-08-19T01:45:31.708 INFO:journalctl@ceph.osd.11.smithi167.stdout:Aug 19 01:45:31 smithi167 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-11[46643]: 2024-08-19T01:45:31.593+0000 7f447f624640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.117:6812 osd.4 since back 2024-08-19T01:44:55.585644+0000 front 2024-08-19T01:44:55.585746+0000 (oldest deadline 2024-08-19T01:45:20.885631+0000) 2024-08-19T01:45:31.708 INFO:journalctl@ceph.osd.11.smithi167.stdout:Aug 19 01:45:31 smithi167 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-11[46643]: 2024-08-19T01:45:31.593+0000 7f447f624640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.167:6812 osd.5 since back 2024-08-19T01:45:02.586329+0000 front 2024-08-19T01:45:02.586392+0000 (oldest deadline 2024-08-19T01:45:28.486376+0000) 2024-08-19T01:45:31.952 INFO:journalctl@ceph.osd.10.smithi117.stdout:Aug 19 01:45:31 smithi117 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-10[46616]: 2024-08-19T01:45:31.506+0000 7f34b3bf7640 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-08-19T01:44:24.060751+0000 front 2024-08-19T01:44:24.060716+0000 (oldest deadline 2024-08-19T01:44:47.560677+0000) 2024-08-19T01:45:31.952 INFO:journalctl@ceph.osd.10.smithi117.stdout:Aug 19 01:45:31 smithi117 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-10[46616]: 2024-08-19T01:45:31.506+0000 7f34b3bf7640 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-08-19T01:44:28.661503+0000 front 2024-08-19T01:44:28.661522+0000 (oldest deadline 2024-08-19T01:44:56.262298+0000) 2024-08-19T01:45:31.952 INFO:journalctl@ceph.osd.10.smithi117.stdout:Aug 19 01:45:31 smithi117 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-10[46616]: 2024-08-19T01:45:31.506+0000 7f34b3bf7640 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-08-19T01:44:38.563674+0000 front 2024-08-19T01:44:38.563520+0000 (oldest deadline 2024-08-19T01:45:02.063517+0000) 2024-08-19T01:45:31.952 INFO:journalctl@ceph.osd.10.smithi117.stdout:Aug 19 01:45:31 smithi117 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-10[46616]: 2024-08-19T01:45:31.506+0000 7f34b3bf7640 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.83:6814 osd.3 since back 2024-08-19T01:44:47.265176+0000 front 2024-08-19T01:44:47.265102+0000 (oldest deadline 2024-08-19T01:45:09.564999+0000) 2024-08-19T01:45:31.952 INFO:journalctl@ceph.osd.10.smithi117.stdout:Aug 19 01:45:31 smithi117 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-10[46616]: 2024-08-19T01:45:31.506+0000 7f34b3bf7640 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.117:6812 osd.4 since back 2024-08-19T01:44:54.865666+0000 front 2024-08-19T01:44:54.865695+0000 (oldest deadline 2024-08-19T01:45:19.565735+0000) 2024-08-19T01:45:31.953 INFO:journalctl@ceph.osd.10.smithi117.stdout:Aug 19 01:45:31 smithi117 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-10[46616]: 2024-08-19T01:45:31.506+0000 7f34b3bf7640 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.167:6812 osd.5 since back 2024-08-19T01:45:03.066733+0000 front 2024-08-19T01:45:03.066769+0000 (oldest deadline 2024-08-19T01:45:25.366703+0000) 2024-08-19T01:45:32.654 INFO:journalctl@ceph.osd.9.smithi083.stdout:Aug 19 01:45:32 smithi083 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-9[51472]: 2024-08-19T01:45:32.230+0000 7f54e8c00640 -1 osd.9 89 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-08-19T01:44:20.873282+0000 front 2024-08-19T01:44:20.873259+0000 (oldest deadline 2024-08-19T01:44:46.773413+0000) 2024-08-19T01:45:32.654 INFO:journalctl@ceph.osd.9.smithi083.stdout:Aug 19 01:45:32 smithi083 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-9[51472]: 2024-08-19T01:45:32.230+0000 7f54e8c00640 -1 osd.9 89 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-08-19T01:44:26.774104+0000 front 2024-08-19T01:44:26.774036+0000 (oldest deadline 2024-08-19T01:44:53.774823+0000) 2024-08-19T01:45:32.655 INFO:journalctl@ceph.osd.9.smithi083.stdout:Aug 19 01:45:32 smithi083 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-9[51472]: 2024-08-19T01:45:32.230+0000 7f54e8c00640 -1 osd.9 89 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-08-19T01:44:39.676617+0000 front 2024-08-19T01:44:39.676511+0000 (oldest deadline 2024-08-19T01:45:03.776684+0000) 2024-08-19T01:45:32.655 INFO:journalctl@ceph.osd.9.smithi083.stdout:Aug 19 01:45:32 smithi083 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-9[51472]: 2024-08-19T01:45:32.230+0000 7f54e8c00640 -1 osd.9 89 heartbeat_check: no reply from 172.21.15.83:6814 osd.3 since back 2024-08-19T01:44:46.678051+0000 front 2024-08-19T01:44:46.678168+0000 (oldest deadline 2024-08-19T01:45:11.378113+0000) 2024-08-19T01:45:32.655 INFO:journalctl@ceph.osd.9.smithi083.stdout:Aug 19 01:45:32 smithi083 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-9[51472]: 2024-08-19T01:45:32.230+0000 7f54e8c00640 -1 osd.9 89 heartbeat_check: no reply from 172.21.15.117:6812 osd.4 since back 2024-08-19T01:44:54.279278+0000 front 2024-08-19T01:44:54.279303+0000 (oldest deadline 2024-08-19T01:45:18.379436+0000) 2024-08-19T01:45:32.655 INFO:journalctl@ceph.osd.9.smithi083.stdout:Aug 19 01:45:32 smithi083 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-9[51472]: 2024-08-19T01:45:32.230+0000 7f54e8c00640 -1 osd.9 89 heartbeat_check: no reply from 172.21.15.167:6812 osd.5 since back 2024-08-19T01:45:04.181394+0000 front 2024-08-19T01:45:04.181366+0000 (oldest deadline 2024-08-19T01:45:25.881307+0000) 2024-08-19T01:45:32.952 INFO:journalctl@ceph.osd.10.smithi117.stdout:Aug 19 01:45:32 smithi117 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-10[46616]: 2024-08-19T01:45:32.475+0000 7f34b3bf7640 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-08-19T01:44:24.060751+0000 front 2024-08-19T01:44:24.060716+0000 (oldest deadline 2024-08-19T01:44:47.560677+0000) 2024-08-19T01:45:32.953 INFO:journalctl@ceph.osd.10.smithi117.stdout:Aug 19 01:45:32 smithi117 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-10[46616]: 2024-08-19T01:45:32.475+0000 7f34b3bf7640 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-08-19T01:44:28.661503+0000 front 2024-08-19T01:44:28.661522+0000 (oldest deadline 2024-08-19T01:44:56.262298+0000) 2024-08-19T01:45:32.953 INFO:journalctl@ceph.osd.10.smithi117.stdout:Aug 19 01:45:32 smithi117 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-10[46616]: 2024-08-19T01:45:32.475+0000 7f34b3bf7640 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-08-19T01:44:38.563674+0000 front 2024-08-19T01:44:38.563520+0000 (oldest deadline 2024-08-19T01:45:02.063517+0000) 2024-08-19T01:45:32.953 INFO:journalctl@ceph.osd.10.smithi117.stdout:Aug 19 01:45:32 smithi117 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-10[46616]: 2024-08-19T01:45:32.475+0000 7f34b3bf7640 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.83:6814 osd.3 since back 2024-08-19T01:44:47.265176+0000 front 2024-08-19T01:44:47.265102+0000 (oldest deadline 2024-08-19T01:45:09.564999+0000) 2024-08-19T01:45:32.954 INFO:journalctl@ceph.osd.10.smithi117.stdout:Aug 19 01:45:32 smithi117 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-10[46616]: 2024-08-19T01:45:32.475+0000 7f34b3bf7640 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.117:6812 osd.4 since back 2024-08-19T01:44:54.865666+0000 front 2024-08-19T01:44:54.865695+0000 (oldest deadline 2024-08-19T01:45:19.565735+0000) 2024-08-19T01:45:32.954 INFO:journalctl@ceph.osd.10.smithi117.stdout:Aug 19 01:45:32 smithi117 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-10[46616]: 2024-08-19T01:45:32.475+0000 7f34b3bf7640 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.167:6812 osd.5 since back 2024-08-19T01:45:03.066733+0000 front 2024-08-19T01:45:03.066769+0000 (oldest deadline 2024-08-19T01:45:25.366703+0000) 2024-08-19T01:45:32.966 INFO:journalctl@ceph.osd.11.smithi167.stdout:Aug 19 01:45:32 smithi167 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-11[46643]: 2024-08-19T01:45:32.582+0000 7f447f624640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-08-19T01:44:22.880210+0000 front 2024-08-19T01:44:22.880404+0000 (oldest deadline 2024-08-19T01:44:45.780352+0000) 2024-08-19T01:45:32.966 INFO:journalctl@ceph.osd.11.smithi167.stdout:Aug 19 01:45:32 smithi167 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-11[46643]: 2024-08-19T01:45:32.582+0000 7f447f624640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-08-19T01:44:31.081546+0000 front 2024-08-19T01:44:31.081654+0000 (oldest deadline 2024-08-19T01:44:53.981595+0000) 2024-08-19T01:45:32.966 INFO:journalctl@ceph.osd.11.smithi167.stdout:Aug 19 01:45:32 smithi167 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-11[46643]: 2024-08-19T01:45:32.582+0000 7f447f624640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-08-19T01:44:39.282638+0000 front 2024-08-19T01:44:39.282723+0000 (oldest deadline 2024-08-19T01:45:04.582709+0000) 2024-08-19T01:45:32.966 INFO:journalctl@ceph.osd.11.smithi167.stdout:Aug 19 01:45:32 smithi167 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-11[46643]: 2024-08-19T01:45:32.582+0000 7f447f624640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.83:6814 osd.3 since back 2024-08-19T01:44:45.083762+0000 front 2024-08-19T01:44:45.083777+0000 (oldest deadline 2024-08-19T01:45:10.383738+0000) 2024-08-19T01:45:32.966 INFO:journalctl@ceph.osd.11.smithi167.stdout:Aug 19 01:45:32 smithi167 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-11[46643]: 2024-08-19T01:45:32.582+0000 7f447f624640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.117:6812 osd.4 since back 2024-08-19T01:44:55.585644+0000 front 2024-08-19T01:44:55.585746+0000 (oldest deadline 2024-08-19T01:45:20.885631+0000) 2024-08-19T01:45:32.966 INFO:journalctl@ceph.osd.11.smithi167.stdout:Aug 19 01:45:32 smithi167 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-11[46643]: 2024-08-19T01:45:32.582+0000 7f447f624640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.167:6812 osd.5 since back 2024-08-19T01:45:02.586329+0000 front 2024-08-19T01:45:02.586392+0000 (oldest deadline 2024-08-19T01:45:28.486376+0000) 2024-08-19T01:45:33.654 INFO:journalctl@ceph.osd.9.smithi083.stdout:Aug 19 01:45:33 smithi083 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-9[51472]: 2024-08-19T01:45:33.190+0000 7f54e8c00640 -1 osd.9 89 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-08-19T01:44:20.873282+0000 front 2024-08-19T01:44:20.873259+0000 (oldest deadline 2024-08-19T01:44:46.773413+0000) 2024-08-19T01:45:33.654 INFO:journalctl@ceph.osd.9.smithi083.stdout:Aug 19 01:45:33 smithi083 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-9[51472]: 2024-08-19T01:45:33.190+0000 7f54e8c00640 -1 osd.9 89 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-08-19T01:44:26.774104+0000 front 2024-08-19T01:44:26.774036+0000 (oldest deadline 2024-08-19T01:44:53.774823+0000) 2024-08-19T01:45:33.655 INFO:journalctl@ceph.osd.9.smithi083.stdout:Aug 19 01:45:33 smithi083 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-9[51472]: 2024-08-19T01:45:33.190+0000 7f54e8c00640 -1 osd.9 89 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-08-19T01:44:39.676617+0000 front 2024-08-19T01:44:39.676511+0000 (oldest deadline 2024-08-19T01:45:03.776684+0000) 2024-08-19T01:45:33.655 INFO:journalctl@ceph.osd.9.smithi083.stdout:Aug 19 01:45:33 smithi083 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-9[51472]: 2024-08-19T01:45:33.190+0000 7f54e8c00640 -1 osd.9 89 heartbeat_check: no reply from 172.21.15.83:6814 osd.3 since back 2024-08-19T01:44:46.678051+0000 front 2024-08-19T01:44:46.678168+0000 (oldest deadline 2024-08-19T01:45:11.378113+0000) 2024-08-19T01:45:33.655 INFO:journalctl@ceph.osd.9.smithi083.stdout:Aug 19 01:45:33 smithi083 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-9[51472]: 2024-08-19T01:45:33.190+0000 7f54e8c00640 -1 osd.9 89 heartbeat_check: no reply from 172.21.15.117:6812 osd.4 since back 2024-08-19T01:44:54.279278+0000 front 2024-08-19T01:44:54.279303+0000 (oldest deadline 2024-08-19T01:45:18.379436+0000) 2024-08-19T01:45:33.655 INFO:journalctl@ceph.osd.9.smithi083.stdout:Aug 19 01:45:33 smithi083 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-9[51472]: 2024-08-19T01:45:33.190+0000 7f54e8c00640 -1 osd.9 89 heartbeat_check: no reply from 172.21.15.167:6812 osd.5 since back 2024-08-19T01:45:04.181394+0000 front 2024-08-19T01:45:04.181366+0000 (oldest deadline 2024-08-19T01:45:25.881307+0000) 2024-08-19T01:45:33.655 INFO:journalctl@ceph.osd.9.smithi083.stdout:Aug 19 01:45:33 smithi083 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-9[51472]: 2024-08-19T01:45:33.190+0000 7f54e8c00640 -1 osd.9 89 heartbeat_check: no reply from 172.21.15.83:6822 osd.6 since back 2024-08-19T01:45:09.982478+0000 front 2024-08-19T01:45:09.982507+0000 (oldest deadline 2024-08-19T01:45:32.882664+0000) 2024-08-19T01:45:33.702 INFO:journalctl@ceph.osd.10.smithi117.stdout:Aug 19 01:45:33 smithi117 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-10[46616]: 2024-08-19T01:45:33.428+0000 7f34b3bf7640 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-08-19T01:44:24.060751+0000 front 2024-08-19T01:44:24.060716+0000 (oldest deadline 2024-08-19T01:44:47.560677+0000) 2024-08-19T01:45:33.702 INFO:journalctl@ceph.osd.10.smithi117.stdout:Aug 19 01:45:33 smithi117 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-10[46616]: 2024-08-19T01:45:33.428+0000 7f34b3bf7640 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-08-19T01:44:28.661503+0000 front 2024-08-19T01:44:28.661522+0000 (oldest deadline 2024-08-19T01:44:56.262298+0000) 2024-08-19T01:45:33.702 INFO:journalctl@ceph.osd.10.smithi117.stdout:Aug 19 01:45:33 smithi117 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-10[46616]: 2024-08-19T01:45:33.428+0000 7f34b3bf7640 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-08-19T01:44:38.563674+0000 front 2024-08-19T01:44:38.563520+0000 (oldest deadline 2024-08-19T01:45:02.063517+0000) 2024-08-19T01:45:33.703 INFO:journalctl@ceph.osd.10.smithi117.stdout:Aug 19 01:45:33 smithi117 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-10[46616]: 2024-08-19T01:45:33.428+0000 7f34b3bf7640 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.83:6814 osd.3 since back 2024-08-19T01:44:47.265176+0000 front 2024-08-19T01:44:47.265102+0000 (oldest deadline 2024-08-19T01:45:09.564999+0000) 2024-08-19T01:45:33.703 INFO:journalctl@ceph.osd.10.smithi117.stdout:Aug 19 01:45:33 smithi117 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-10[46616]: 2024-08-19T01:45:33.428+0000 7f34b3bf7640 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.117:6812 osd.4 since back 2024-08-19T01:44:54.865666+0000 front 2024-08-19T01:44:54.865695+0000 (oldest deadline 2024-08-19T01:45:19.565735+0000) 2024-08-19T01:45:33.703 INFO:journalctl@ceph.osd.10.smithi117.stdout:Aug 19 01:45:33 smithi117 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-10[46616]: 2024-08-19T01:45:33.428+0000 7f34b3bf7640 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.167:6812 osd.5 since back 2024-08-19T01:45:03.066733+0000 front 2024-08-19T01:45:03.066769+0000 (oldest deadline 2024-08-19T01:45:25.366703+0000) 2024-08-19T01:45:33.703 INFO:journalctl@ceph.osd.10.smithi117.stdout:Aug 19 01:45:33 smithi117 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-10[46616]: 2024-08-19T01:45:33.428+0000 7f34b3bf7640 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.83:6822 osd.6 since back 2024-08-19T01:45:11.168172+0000 front 2024-08-19T01:45:11.168153+0000 (oldest deadline 2024-08-19T01:45:32.867945+0000) 2024-08-19T01:45:33.966 INFO:journalctl@ceph.osd.11.smithi167.stdout:Aug 19 01:45:33 smithi167 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-11[46643]: 2024-08-19T01:45:33.562+0000 7f447f624640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-08-19T01:44:22.880210+0000 front 2024-08-19T01:44:22.880404+0000 (oldest deadline 2024-08-19T01:44:45.780352+0000) 2024-08-19T01:45:33.966 INFO:journalctl@ceph.osd.11.smithi167.stdout:Aug 19 01:45:33 smithi167 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-11[46643]: 2024-08-19T01:45:33.562+0000 7f447f624640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-08-19T01:44:31.081546+0000 front 2024-08-19T01:44:31.081654+0000 (oldest deadline 2024-08-19T01:44:53.981595+0000) 2024-08-19T01:45:33.966 INFO:journalctl@ceph.osd.11.smithi167.stdout:Aug 19 01:45:33 smithi167 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-11[46643]: 2024-08-19T01:45:33.562+0000 7f447f624640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-08-19T01:44:39.282638+0000 front 2024-08-19T01:44:39.282723+0000 (oldest deadline 2024-08-19T01:45:04.582709+0000) 2024-08-19T01:45:33.966 INFO:journalctl@ceph.osd.11.smithi167.stdout:Aug 19 01:45:33 smithi167 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-11[46643]: 2024-08-19T01:45:33.562+0000 7f447f624640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.83:6814 osd.3 since back 2024-08-19T01:44:45.083762+0000 front 2024-08-19T01:44:45.083777+0000 (oldest deadline 2024-08-19T01:45:10.383738+0000) 2024-08-19T01:45:33.966 INFO:journalctl@ceph.osd.11.smithi167.stdout:Aug 19 01:45:33 smithi167 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-11[46643]: 2024-08-19T01:45:33.562+0000 7f447f624640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.117:6812 osd.4 since back 2024-08-19T01:44:55.585644+0000 front 2024-08-19T01:44:55.585746+0000 (oldest deadline 2024-08-19T01:45:20.885631+0000) 2024-08-19T01:45:33.966 INFO:journalctl@ceph.osd.11.smithi167.stdout:Aug 19 01:45:33 smithi167 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-11[46643]: 2024-08-19T01:45:33.562+0000 7f447f624640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.167:6812 osd.5 since back 2024-08-19T01:45:02.586329+0000 front 2024-08-19T01:45:02.586392+0000 (oldest deadline 2024-08-19T01:45:28.486376+0000) 2024-08-19T01:45:34.654 INFO:journalctl@ceph.osd.9.smithi083.stdout:Aug 19 01:45:34 smithi083 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-9[51472]: 2024-08-19T01:45:34.227+0000 7f54e8c00640 -1 osd.9 89 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-08-19T01:44:20.873282+0000 front 2024-08-19T01:44:20.873259+0000 (oldest deadline 2024-08-19T01:44:46.773413+0000) 2024-08-19T01:45:34.654 INFO:journalctl@ceph.osd.9.smithi083.stdout:Aug 19 01:45:34 smithi083 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-9[51472]: 2024-08-19T01:45:34.227+0000 7f54e8c00640 -1 osd.9 89 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-08-19T01:44:26.774104+0000 front 2024-08-19T01:44:26.774036+0000 (oldest deadline 2024-08-19T01:44:53.774823+0000) 2024-08-19T01:45:34.654 INFO:journalctl@ceph.osd.9.smithi083.stdout:Aug 19 01:45:34 smithi083 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-9[51472]: 2024-08-19T01:45:34.227+0000 7f54e8c00640 -1 osd.9 89 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-08-19T01:44:39.676617+0000 front 2024-08-19T01:44:39.676511+0000 (oldest deadline 2024-08-19T01:45:03.776684+0000) 2024-08-19T01:45:34.655 INFO:journalctl@ceph.osd.9.smithi083.stdout:Aug 19 01:45:34 smithi083 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-9[51472]: 2024-08-19T01:45:34.227+0000 7f54e8c00640 -1 osd.9 89 heartbeat_check: no reply from 172.21.15.83:6814 osd.3 since back 2024-08-19T01:44:46.678051+0000 front 2024-08-19T01:44:46.678168+0000 (oldest deadline 2024-08-19T01:45:11.378113+0000) 2024-08-19T01:45:34.655 INFO:journalctl@ceph.osd.9.smithi083.stdout:Aug 19 01:45:34 smithi083 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-9[51472]: 2024-08-19T01:45:34.227+0000 7f54e8c00640 -1 osd.9 89 heartbeat_check: no reply from 172.21.15.117:6812 osd.4 since back 2024-08-19T01:44:54.279278+0000 front 2024-08-19T01:44:54.279303+0000 (oldest deadline 2024-08-19T01:45:18.379436+0000) 2024-08-19T01:45:34.655 INFO:journalctl@ceph.osd.9.smithi083.stdout:Aug 19 01:45:34 smithi083 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-9[51472]: 2024-08-19T01:45:34.227+0000 7f54e8c00640 -1 osd.9 89 heartbeat_check: no reply from 172.21.15.167:6812 osd.5 since back 2024-08-19T01:45:04.181394+0000 front 2024-08-19T01:45:04.181366+0000 (oldest deadline 2024-08-19T01:45:25.881307+0000) 2024-08-19T01:45:34.655 INFO:journalctl@ceph.osd.9.smithi083.stdout:Aug 19 01:45:34 smithi083 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-9[51472]: 2024-08-19T01:45:34.227+0000 7f54e8c00640 -1 osd.9 89 heartbeat_check: no reply from 172.21.15.83:6822 osd.6 since back 2024-08-19T01:45:09.982478+0000 front 2024-08-19T01:45:09.982507+0000 (oldest deadline 2024-08-19T01:45:32.882664+0000) 2024-08-19T01:45:34.702 INFO:journalctl@ceph.osd.10.smithi117.stdout:Aug 19 01:45:34 smithi117 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-10[46616]: 2024-08-19T01:45:34.467+0000 7f34b3bf7640 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-08-19T01:44:24.060751+0000 front 2024-08-19T01:44:24.060716+0000 (oldest deadline 2024-08-19T01:44:47.560677+0000) 2024-08-19T01:45:34.703 INFO:journalctl@ceph.osd.10.smithi117.stdout:Aug 19 01:45:34 smithi117 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-10[46616]: 2024-08-19T01:45:34.467+0000 7f34b3bf7640 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-08-19T01:44:28.661503+0000 front 2024-08-19T01:44:28.661522+0000 (oldest deadline 2024-08-19T01:44:56.262298+0000) 2024-08-19T01:45:34.703 INFO:journalctl@ceph.osd.10.smithi117.stdout:Aug 19 01:45:34 smithi117 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-10[46616]: 2024-08-19T01:45:34.467+0000 7f34b3bf7640 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-08-19T01:44:38.563674+0000 front 2024-08-19T01:44:38.563520+0000 (oldest deadline 2024-08-19T01:45:02.063517+0000) 2024-08-19T01:45:34.703 INFO:journalctl@ceph.osd.10.smithi117.stdout:Aug 19 01:45:34 smithi117 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-10[46616]: 2024-08-19T01:45:34.467+0000 7f34b3bf7640 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.83:6814 osd.3 since back 2024-08-19T01:44:47.265176+0000 front 2024-08-19T01:44:47.265102+0000 (oldest deadline 2024-08-19T01:45:09.564999+0000) 2024-08-19T01:45:34.703 INFO:journalctl@ceph.osd.10.smithi117.stdout:Aug 19 01:45:34 smithi117 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-10[46616]: 2024-08-19T01:45:34.467+0000 7f34b3bf7640 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.117:6812 osd.4 since back 2024-08-19T01:44:54.865666+0000 front 2024-08-19T01:44:54.865695+0000 (oldest deadline 2024-08-19T01:45:19.565735+0000) 2024-08-19T01:45:34.703 INFO:journalctl@ceph.osd.10.smithi117.stdout:Aug 19 01:45:34 smithi117 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-10[46616]: 2024-08-19T01:45:34.467+0000 7f34b3bf7640 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.167:6812 osd.5 since back 2024-08-19T01:45:03.066733+0000 front 2024-08-19T01:45:03.066769+0000 (oldest deadline 2024-08-19T01:45:25.366703+0000) 2024-08-19T01:45:34.703 INFO:journalctl@ceph.osd.10.smithi117.stdout:Aug 19 01:45:34 smithi117 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-10[46616]: 2024-08-19T01:45:34.467+0000 7f34b3bf7640 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.83:6822 osd.6 since back 2024-08-19T01:45:11.168172+0000 front 2024-08-19T01:45:11.168153+0000 (oldest deadline 2024-08-19T01:45:32.867945+0000) 2024-08-19T01:45:34.966 INFO:journalctl@ceph.osd.11.smithi167.stdout:Aug 19 01:45:34 smithi167 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-11[46643]: 2024-08-19T01:45:34.562+0000 7f447f624640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-08-19T01:44:22.880210+0000 front 2024-08-19T01:44:22.880404+0000 (oldest deadline 2024-08-19T01:44:45.780352+0000) 2024-08-19T01:45:34.966 INFO:journalctl@ceph.osd.11.smithi167.stdout:Aug 19 01:45:34 smithi167 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-11[46643]: 2024-08-19T01:45:34.562+0000 7f447f624640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-08-19T01:44:31.081546+0000 front 2024-08-19T01:44:31.081654+0000 (oldest deadline 2024-08-19T01:44:53.981595+0000) 2024-08-19T01:45:34.966 INFO:journalctl@ceph.osd.11.smithi167.stdout:Aug 19 01:45:34 smithi167 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-11[46643]: 2024-08-19T01:45:34.562+0000 7f447f624640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-08-19T01:44:39.282638+0000 front 2024-08-19T01:44:39.282723+0000 (oldest deadline 2024-08-19T01:45:04.582709+0000) 2024-08-19T01:45:34.966 INFO:journalctl@ceph.osd.11.smithi167.stdout:Aug 19 01:45:34 smithi167 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-11[46643]: 2024-08-19T01:45:34.562+0000 7f447f624640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.83:6814 osd.3 since back 2024-08-19T01:44:45.083762+0000 front 2024-08-19T01:44:45.083777+0000 (oldest deadline 2024-08-19T01:45:10.383738+0000) 2024-08-19T01:45:34.966 INFO:journalctl@ceph.osd.11.smithi167.stdout:Aug 19 01:45:34 smithi167 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-11[46643]: 2024-08-19T01:45:34.562+0000 7f447f624640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.117:6812 osd.4 since back 2024-08-19T01:44:55.585644+0000 front 2024-08-19T01:44:55.585746+0000 (oldest deadline 2024-08-19T01:45:20.885631+0000) 2024-08-19T01:45:34.966 INFO:journalctl@ceph.osd.11.smithi167.stdout:Aug 19 01:45:34 smithi167 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-11[46643]: 2024-08-19T01:45:34.562+0000 7f447f624640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.167:6812 osd.5 since back 2024-08-19T01:45:02.586329+0000 front 2024-08-19T01:45:02.586392+0000 (oldest deadline 2024-08-19T01:45:28.486376+0000) 2024-08-19T01:45:35.654 INFO:journalctl@ceph.osd.9.smithi083.stdout:Aug 19 01:45:35 smithi083 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-9[51472]: 2024-08-19T01:45:35.218+0000 7f54e8c00640 -1 osd.9 89 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-08-19T01:44:20.873282+0000 front 2024-08-19T01:44:20.873259+0000 (oldest deadline 2024-08-19T01:44:46.773413+0000) 2024-08-19T01:45:35.654 INFO:journalctl@ceph.osd.9.smithi083.stdout:Aug 19 01:45:35 smithi083 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-9[51472]: 2024-08-19T01:45:35.218+0000 7f54e8c00640 -1 osd.9 89 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-08-19T01:44:26.774104+0000 front 2024-08-19T01:44:26.774036+0000 (oldest deadline 2024-08-19T01:44:53.774823+0000) 2024-08-19T01:45:35.655 INFO:journalctl@ceph.osd.9.smithi083.stdout:Aug 19 01:45:35 smithi083 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-9[51472]: 2024-08-19T01:45:35.218+0000 7f54e8c00640 -1 osd.9 89 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-08-19T01:44:39.676617+0000 front 2024-08-19T01:44:39.676511+0000 (oldest deadline 2024-08-19T01:45:03.776684+0000) 2024-08-19T01:45:35.655 INFO:journalctl@ceph.osd.9.smithi083.stdout:Aug 19 01:45:35 smithi083 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-9[51472]: 2024-08-19T01:45:35.218+0000 7f54e8c00640 -1 osd.9 89 heartbeat_check: no reply from 172.21.15.83:6814 osd.3 since back 2024-08-19T01:44:46.678051+0000 front 2024-08-19T01:44:46.678168+0000 (oldest deadline 2024-08-19T01:45:11.378113+0000) 2024-08-19T01:45:35.655 INFO:journalctl@ceph.osd.9.smithi083.stdout:Aug 19 01:45:35 smithi083 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-9[51472]: 2024-08-19T01:45:35.218+0000 7f54e8c00640 -1 osd.9 89 heartbeat_check: no reply from 172.21.15.117:6812 osd.4 since back 2024-08-19T01:44:54.279278+0000 front 2024-08-19T01:44:54.279303+0000 (oldest deadline 2024-08-19T01:45:18.379436+0000) 2024-08-19T01:45:35.655 INFO:journalctl@ceph.osd.9.smithi083.stdout:Aug 19 01:45:35 smithi083 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-9[51472]: 2024-08-19T01:45:35.218+0000 7f54e8c00640 -1 osd.9 89 heartbeat_check: no reply from 172.21.15.167:6812 osd.5 since back 2024-08-19T01:45:04.181394+0000 front 2024-08-19T01:45:04.181366+0000 (oldest deadline 2024-08-19T01:45:25.881307+0000) 2024-08-19T01:45:35.655 INFO:journalctl@ceph.osd.9.smithi083.stdout:Aug 19 01:45:35 smithi083 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-9[51472]: 2024-08-19T01:45:35.218+0000 7f54e8c00640 -1 osd.9 89 heartbeat_check: no reply from 172.21.15.83:6822 osd.6 since back 2024-08-19T01:45:09.982478+0000 front 2024-08-19T01:45:09.982507+0000 (oldest deadline 2024-08-19T01:45:32.882664+0000) 2024-08-19T01:45:35.702 INFO:journalctl@ceph.osd.10.smithi117.stdout:Aug 19 01:45:35 smithi117 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-10[46616]: 2024-08-19T01:45:35.445+0000 7f34b3bf7640 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-08-19T01:44:24.060751+0000 front 2024-08-19T01:44:24.060716+0000 (oldest deadline 2024-08-19T01:44:47.560677+0000) 2024-08-19T01:45:35.702 INFO:journalctl@ceph.osd.10.smithi117.stdout:Aug 19 01:45:35 smithi117 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-10[46616]: 2024-08-19T01:45:35.445+0000 7f34b3bf7640 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-08-19T01:44:28.661503+0000 front 2024-08-19T01:44:28.661522+0000 (oldest deadline 2024-08-19T01:44:56.262298+0000) 2024-08-19T01:45:35.702 INFO:journalctl@ceph.osd.10.smithi117.stdout:Aug 19 01:45:35 smithi117 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-10[46616]: 2024-08-19T01:45:35.445+0000 7f34b3bf7640 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-08-19T01:44:38.563674+0000 front 2024-08-19T01:44:38.563520+0000 (oldest deadline 2024-08-19T01:45:02.063517+0000) 2024-08-19T01:45:35.703 INFO:journalctl@ceph.osd.10.smithi117.stdout:Aug 19 01:45:35 smithi117 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-10[46616]: 2024-08-19T01:45:35.445+0000 7f34b3bf7640 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.83:6814 osd.3 since back 2024-08-19T01:44:47.265176+0000 front 2024-08-19T01:44:47.265102+0000 (oldest deadline 2024-08-19T01:45:09.564999+0000) 2024-08-19T01:45:35.703 INFO:journalctl@ceph.osd.10.smithi117.stdout:Aug 19 01:45:35 smithi117 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-10[46616]: 2024-08-19T01:45:35.445+0000 7f34b3bf7640 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.117:6812 osd.4 since back 2024-08-19T01:44:54.865666+0000 front 2024-08-19T01:44:54.865695+0000 (oldest deadline 2024-08-19T01:45:19.565735+0000) 2024-08-19T01:45:35.703 INFO:journalctl@ceph.osd.10.smithi117.stdout:Aug 19 01:45:35 smithi117 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-10[46616]: 2024-08-19T01:45:35.445+0000 7f34b3bf7640 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.167:6812 osd.5 since back 2024-08-19T01:45:03.066733+0000 front 2024-08-19T01:45:03.066769+0000 (oldest deadline 2024-08-19T01:45:25.366703+0000) 2024-08-19T01:45:35.703 INFO:journalctl@ceph.osd.10.smithi117.stdout:Aug 19 01:45:35 smithi117 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-10[46616]: 2024-08-19T01:45:35.445+0000 7f34b3bf7640 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.83:6822 osd.6 since back 2024-08-19T01:45:11.168172+0000 front 2024-08-19T01:45:11.168153+0000 (oldest deadline 2024-08-19T01:45:32.867945+0000) 2024-08-19T01:45:35.966 INFO:journalctl@ceph.osd.11.smithi167.stdout:Aug 19 01:45:35 smithi167 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-11[46643]: 2024-08-19T01:45:35.608+0000 7f447f624640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-08-19T01:44:22.880210+0000 front 2024-08-19T01:44:22.880404+0000 (oldest deadline 2024-08-19T01:44:45.780352+0000) 2024-08-19T01:45:35.966 INFO:journalctl@ceph.osd.11.smithi167.stdout:Aug 19 01:45:35 smithi167 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-11[46643]: 2024-08-19T01:45:35.608+0000 7f447f624640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-08-19T01:44:31.081546+0000 front 2024-08-19T01:44:31.081654+0000 (oldest deadline 2024-08-19T01:44:53.981595+0000) 2024-08-19T01:45:35.966 INFO:journalctl@ceph.osd.11.smithi167.stdout:Aug 19 01:45:35 smithi167 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-11[46643]: 2024-08-19T01:45:35.608+0000 7f447f624640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-08-19T01:44:39.282638+0000 front 2024-08-19T01:44:39.282723+0000 (oldest deadline 2024-08-19T01:45:04.582709+0000) 2024-08-19T01:45:35.966 INFO:journalctl@ceph.osd.11.smithi167.stdout:Aug 19 01:45:35 smithi167 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-11[46643]: 2024-08-19T01:45:35.608+0000 7f447f624640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.83:6814 osd.3 since back 2024-08-19T01:44:45.083762+0000 front 2024-08-19T01:44:45.083777+0000 (oldest deadline 2024-08-19T01:45:10.383738+0000) 2024-08-19T01:45:35.966 INFO:journalctl@ceph.osd.11.smithi167.stdout:Aug 19 01:45:35 smithi167 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-11[46643]: 2024-08-19T01:45:35.608+0000 7f447f624640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.117:6812 osd.4 since back 2024-08-19T01:44:55.585644+0000 front 2024-08-19T01:44:55.585746+0000 (oldest deadline 2024-08-19T01:45:20.885631+0000) 2024-08-19T01:45:35.966 INFO:journalctl@ceph.osd.11.smithi167.stdout:Aug 19 01:45:35 smithi167 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-11[46643]: 2024-08-19T01:45:35.608+0000 7f447f624640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.167:6812 osd.5 since back 2024-08-19T01:45:02.586329+0000 front 2024-08-19T01:45:02.586392+0000 (oldest deadline 2024-08-19T01:45:28.486376+0000) 2024-08-19T01:45:36.655 INFO:journalctl@ceph.osd.9.smithi083.stdout:Aug 19 01:45:36 smithi083 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-9[51472]: 2024-08-19T01:45:36.217+0000 7f54e8c00640 -1 osd.9 89 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-08-19T01:44:20.873282+0000 front 2024-08-19T01:44:20.873259+0000 (oldest deadline 2024-08-19T01:44:46.773413+0000) 2024-08-19T01:45:36.655 INFO:journalctl@ceph.osd.9.smithi083.stdout:Aug 19 01:45:36 smithi083 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-9[51472]: 2024-08-19T01:45:36.217+0000 7f54e8c00640 -1 osd.9 89 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-08-19T01:44:26.774104+0000 front 2024-08-19T01:44:26.774036+0000 (oldest deadline 2024-08-19T01:44:53.774823+0000) 2024-08-19T01:45:36.655 INFO:journalctl@ceph.osd.9.smithi083.stdout:Aug 19 01:45:36 smithi083 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-9[51472]: 2024-08-19T01:45:36.217+0000 7f54e8c00640 -1 osd.9 89 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-08-19T01:44:39.676617+0000 front 2024-08-19T01:44:39.676511+0000 (oldest deadline 2024-08-19T01:45:03.776684+0000) 2024-08-19T01:45:36.655 INFO:journalctl@ceph.osd.9.smithi083.stdout:Aug 19 01:45:36 smithi083 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-9[51472]: 2024-08-19T01:45:36.217+0000 7f54e8c00640 -1 osd.9 89 heartbeat_check: no reply from 172.21.15.83:6814 osd.3 since back 2024-08-19T01:44:46.678051+0000 front 2024-08-19T01:44:46.678168+0000 (oldest deadline 2024-08-19T01:45:11.378113+0000) 2024-08-19T01:45:36.655 INFO:journalctl@ceph.osd.9.smithi083.stdout:Aug 19 01:45:36 smithi083 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-9[51472]: 2024-08-19T01:45:36.217+0000 7f54e8c00640 -1 osd.9 89 heartbeat_check: no reply from 172.21.15.117:6812 osd.4 since back 2024-08-19T01:44:54.279278+0000 front 2024-08-19T01:44:54.279303+0000 (oldest deadline 2024-08-19T01:45:18.379436+0000) 2024-08-19T01:45:36.655 INFO:journalctl@ceph.osd.9.smithi083.stdout:Aug 19 01:45:36 smithi083 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-9[51472]: 2024-08-19T01:45:36.217+0000 7f54e8c00640 -1 osd.9 89 heartbeat_check: no reply from 172.21.15.167:6812 osd.5 since back 2024-08-19T01:45:04.181394+0000 front 2024-08-19T01:45:04.181366+0000 (oldest deadline 2024-08-19T01:45:25.881307+0000) 2024-08-19T01:45:36.655 INFO:journalctl@ceph.osd.9.smithi083.stdout:Aug 19 01:45:36 smithi083 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-9[51472]: 2024-08-19T01:45:36.217+0000 7f54e8c00640 -1 osd.9 89 heartbeat_check: no reply from 172.21.15.83:6822 osd.6 since back 2024-08-19T01:45:09.982478+0000 front 2024-08-19T01:45:09.982507+0000 (oldest deadline 2024-08-19T01:45:32.882664+0000) 2024-08-19T01:45:36.702 INFO:journalctl@ceph.osd.10.smithi117.stdout:Aug 19 01:45:36 smithi117 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-10[46616]: 2024-08-19T01:45:36.410+0000 7f34b3bf7640 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-08-19T01:44:24.060751+0000 front 2024-08-19T01:44:24.060716+0000 (oldest deadline 2024-08-19T01:44:47.560677+0000) 2024-08-19T01:45:36.703 INFO:journalctl@ceph.osd.10.smithi117.stdout:Aug 19 01:45:36 smithi117 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-10[46616]: 2024-08-19T01:45:36.410+0000 7f34b3bf7640 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-08-19T01:44:28.661503+0000 front 2024-08-19T01:44:28.661522+0000 (oldest deadline 2024-08-19T01:44:56.262298+0000) 2024-08-19T01:45:36.703 INFO:journalctl@ceph.osd.10.smithi117.stdout:Aug 19 01:45:36 smithi117 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-10[46616]: 2024-08-19T01:45:36.410+0000 7f34b3bf7640 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-08-19T01:44:38.563674+0000 front 2024-08-19T01:44:38.563520+0000 (oldest deadline 2024-08-19T01:45:02.063517+0000) 2024-08-19T01:45:36.703 INFO:journalctl@ceph.osd.10.smithi117.stdout:Aug 19 01:45:36 smithi117 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-10[46616]: 2024-08-19T01:45:36.410+0000 7f34b3bf7640 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.83:6814 osd.3 since back 2024-08-19T01:44:47.265176+0000 front 2024-08-19T01:44:47.265102+0000 (oldest deadline 2024-08-19T01:45:09.564999+0000) 2024-08-19T01:45:36.703 INFO:journalctl@ceph.osd.10.smithi117.stdout:Aug 19 01:45:36 smithi117 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-10[46616]: 2024-08-19T01:45:36.410+0000 7f34b3bf7640 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.117:6812 osd.4 since back 2024-08-19T01:44:54.865666+0000 front 2024-08-19T01:44:54.865695+0000 (oldest deadline 2024-08-19T01:45:19.565735+0000) 2024-08-19T01:45:36.703 INFO:journalctl@ceph.osd.10.smithi117.stdout:Aug 19 01:45:36 smithi117 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-10[46616]: 2024-08-19T01:45:36.410+0000 7f34b3bf7640 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.167:6812 osd.5 since back 2024-08-19T01:45:03.066733+0000 front 2024-08-19T01:45:03.066769+0000 (oldest deadline 2024-08-19T01:45:25.366703+0000) 2024-08-19T01:45:36.703 INFO:journalctl@ceph.osd.10.smithi117.stdout:Aug 19 01:45:36 smithi117 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-10[46616]: 2024-08-19T01:45:36.410+0000 7f34b3bf7640 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.83:6822 osd.6 since back 2024-08-19T01:45:11.168172+0000 front 2024-08-19T01:45:11.168153+0000 (oldest deadline 2024-08-19T01:45:32.867945+0000) 2024-08-19T01:45:36.966 INFO:journalctl@ceph.osd.11.smithi167.stdout:Aug 19 01:45:36 smithi167 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-11[46643]: 2024-08-19T01:45:36.620+0000 7f447f624640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-08-19T01:44:22.880210+0000 front 2024-08-19T01:44:22.880404+0000 (oldest deadline 2024-08-19T01:44:45.780352+0000) 2024-08-19T01:45:36.966 INFO:journalctl@ceph.osd.11.smithi167.stdout:Aug 19 01:45:36 smithi167 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-11[46643]: 2024-08-19T01:45:36.620+0000 7f447f624640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-08-19T01:44:31.081546+0000 front 2024-08-19T01:44:31.081654+0000 (oldest deadline 2024-08-19T01:44:53.981595+0000) 2024-08-19T01:45:36.966 INFO:journalctl@ceph.osd.11.smithi167.stdout:Aug 19 01:45:36 smithi167 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-11[46643]: 2024-08-19T01:45:36.620+0000 7f447f624640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-08-19T01:44:39.282638+0000 front 2024-08-19T01:44:39.282723+0000 (oldest deadline 2024-08-19T01:45:04.582709+0000) 2024-08-19T01:45:36.966 INFO:journalctl@ceph.osd.11.smithi167.stdout:Aug 19 01:45:36 smithi167 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-11[46643]: 2024-08-19T01:45:36.620+0000 7f447f624640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.83:6814 osd.3 since back 2024-08-19T01:44:45.083762+0000 front 2024-08-19T01:44:45.083777+0000 (oldest deadline 2024-08-19T01:45:10.383738+0000) 2024-08-19T01:45:36.966 INFO:journalctl@ceph.osd.11.smithi167.stdout:Aug 19 01:45:36 smithi167 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-11[46643]: 2024-08-19T01:45:36.620+0000 7f447f624640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.117:6812 osd.4 since back 2024-08-19T01:44:55.585644+0000 front 2024-08-19T01:44:55.585746+0000 (oldest deadline 2024-08-19T01:45:20.885631+0000) 2024-08-19T01:45:36.966 INFO:journalctl@ceph.osd.11.smithi167.stdout:Aug 19 01:45:36 smithi167 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-11[46643]: 2024-08-19T01:45:36.620+0000 7f447f624640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.167:6812 osd.5 since back 2024-08-19T01:45:02.586329+0000 front 2024-08-19T01:45:02.586392+0000 (oldest deadline 2024-08-19T01:45:28.486376+0000) 2024-08-19T01:45:37.158 INFO:journalctl@ceph.osd.9.smithi083.stdout:Aug 19 01:45:36 smithi083 podman[85014]: 2024-08-19 01:45:36.905700469 +0000 UTC m=+5.673739587 container died 09811e0a09a531eaff3fc6a0c2183a828546c902562821dd31ba1397c029dd04 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c9dd28080a631be25c77c36c63b17a3ef340ffb2e21c87d4d7b56b6398436d9, name=ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-9, ceph=True, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, RELEASE=squid-de09dd0, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.36.0, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, GIT_CLEAN=True) 2024-08-19T01:45:37.553 INFO:journalctl@ceph.osd.9.smithi083.stdout:Aug 19 01:45:37 smithi083 podman[85014]: 2024-08-19 01:45:37.368147104 +0000 UTC m=+6.136186221 container cleanup 09811e0a09a531eaff3fc6a0c2183a828546c902562821dd31ba1397c029dd04 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c9dd28080a631be25c77c36c63b17a3ef340ffb2e21c87d4d7b56b6398436d9, name=ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-9, maintainer=Guillaume Abrioux , io.buildah.version=1.36.0, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, ceph=True, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-de09dd0, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image) 2024-08-19T01:45:37.553 INFO:journalctl@ceph.osd.9.smithi083.stdout:Aug 19 01:45:37 smithi083 bash[85014]: ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-9 2024-08-19T01:45:37.702 INFO:journalctl@ceph.osd.10.smithi117.stdout:Aug 19 01:45:37 smithi117 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-10[46616]: 2024-08-19T01:45:37.400+0000 7f34b3bf7640 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-08-19T01:44:24.060751+0000 front 2024-08-19T01:44:24.060716+0000 (oldest deadline 2024-08-19T01:44:47.560677+0000) 2024-08-19T01:45:37.702 INFO:journalctl@ceph.osd.10.smithi117.stdout:Aug 19 01:45:37 smithi117 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-10[46616]: 2024-08-19T01:45:37.400+0000 7f34b3bf7640 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-08-19T01:44:28.661503+0000 front 2024-08-19T01:44:28.661522+0000 (oldest deadline 2024-08-19T01:44:56.262298+0000) 2024-08-19T01:45:37.703 INFO:journalctl@ceph.osd.10.smithi117.stdout:Aug 19 01:45:37 smithi117 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-10[46616]: 2024-08-19T01:45:37.400+0000 7f34b3bf7640 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-08-19T01:44:38.563674+0000 front 2024-08-19T01:44:38.563520+0000 (oldest deadline 2024-08-19T01:45:02.063517+0000) 2024-08-19T01:45:37.703 INFO:journalctl@ceph.osd.10.smithi117.stdout:Aug 19 01:45:37 smithi117 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-10[46616]: 2024-08-19T01:45:37.400+0000 7f34b3bf7640 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.83:6814 osd.3 since back 2024-08-19T01:44:47.265176+0000 front 2024-08-19T01:44:47.265102+0000 (oldest deadline 2024-08-19T01:45:09.564999+0000) 2024-08-19T01:45:37.703 INFO:journalctl@ceph.osd.10.smithi117.stdout:Aug 19 01:45:37 smithi117 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-10[46616]: 2024-08-19T01:45:37.400+0000 7f34b3bf7640 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.117:6812 osd.4 since back 2024-08-19T01:44:54.865666+0000 front 2024-08-19T01:44:54.865695+0000 (oldest deadline 2024-08-19T01:45:19.565735+0000) 2024-08-19T01:45:37.703 INFO:journalctl@ceph.osd.10.smithi117.stdout:Aug 19 01:45:37 smithi117 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-10[46616]: 2024-08-19T01:45:37.400+0000 7f34b3bf7640 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.167:6812 osd.5 since back 2024-08-19T01:45:03.066733+0000 front 2024-08-19T01:45:03.066769+0000 (oldest deadline 2024-08-19T01:45:25.366703+0000) 2024-08-19T01:45:37.703 INFO:journalctl@ceph.osd.10.smithi117.stdout:Aug 19 01:45:37 smithi117 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-10[46616]: 2024-08-19T01:45:37.400+0000 7f34b3bf7640 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.83:6822 osd.6 since back 2024-08-19T01:45:11.168172+0000 front 2024-08-19T01:45:11.168153+0000 (oldest deadline 2024-08-19T01:45:32.867945+0000) 2024-08-19T01:45:37.904 INFO:journalctl@ceph.osd.9.smithi083.stdout:Aug 19 01:45:37 smithi083 podman[85029]: 2024-08-19 01:45:37.552739064 +0000 UTC m=+0.638812355 container remove 09811e0a09a531eaff3fc6a0c2183a828546c902562821dd31ba1397c029dd04 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c9dd28080a631be25c77c36c63b17a3ef340ffb2e21c87d4d7b56b6398436d9, name=ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-9, CEPH_POINT_RELEASE=, io.buildah.version=1.36.0, RELEASE=squid-de09dd0, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS) 2024-08-19T01:45:37.966 INFO:journalctl@ceph.osd.11.smithi167.stdout:Aug 19 01:45:37 smithi167 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-11[46643]: 2024-08-19T01:45:37.580+0000 7f447f624640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-08-19T01:44:22.880210+0000 front 2024-08-19T01:44:22.880404+0000 (oldest deadline 2024-08-19T01:44:45.780352+0000) 2024-08-19T01:45:37.966 INFO:journalctl@ceph.osd.11.smithi167.stdout:Aug 19 01:45:37 smithi167 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-11[46643]: 2024-08-19T01:45:37.580+0000 7f447f624640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-08-19T01:44:31.081546+0000 front 2024-08-19T01:44:31.081654+0000 (oldest deadline 2024-08-19T01:44:53.981595+0000) 2024-08-19T01:45:37.966 INFO:journalctl@ceph.osd.11.smithi167.stdout:Aug 19 01:45:37 smithi167 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-11[46643]: 2024-08-19T01:45:37.580+0000 7f447f624640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-08-19T01:44:39.282638+0000 front 2024-08-19T01:44:39.282723+0000 (oldest deadline 2024-08-19T01:45:04.582709+0000) 2024-08-19T01:45:37.966 INFO:journalctl@ceph.osd.11.smithi167.stdout:Aug 19 01:45:37 smithi167 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-11[46643]: 2024-08-19T01:45:37.580+0000 7f447f624640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.83:6814 osd.3 since back 2024-08-19T01:44:45.083762+0000 front 2024-08-19T01:44:45.083777+0000 (oldest deadline 2024-08-19T01:45:10.383738+0000) 2024-08-19T01:45:37.966 INFO:journalctl@ceph.osd.11.smithi167.stdout:Aug 19 01:45:37 smithi167 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-11[46643]: 2024-08-19T01:45:37.580+0000 7f447f624640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.117:6812 osd.4 since back 2024-08-19T01:44:55.585644+0000 front 2024-08-19T01:44:55.585746+0000 (oldest deadline 2024-08-19T01:45:20.885631+0000) 2024-08-19T01:45:37.966 INFO:journalctl@ceph.osd.11.smithi167.stdout:Aug 19 01:45:37 smithi167 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-11[46643]: 2024-08-19T01:45:37.580+0000 7f447f624640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.167:6812 osd.5 since back 2024-08-19T01:45:02.586329+0000 front 2024-08-19T01:45:02.586392+0000 (oldest deadline 2024-08-19T01:45:28.486376+0000) 2024-08-19T01:45:38.404 INFO:journalctl@ceph.osd.9.smithi083.stdout:Aug 19 01:45:38 smithi083 podman[85117]: 2024-08-19 01:45:38.054093772 +0000 UTC m=+0.029088967 image pull ea7a5189f30bad3aca9524788e2965d2368d7ce786df09d4edcd8c3edf9cefba quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c9dd28080a631be25c77c36c63b17a3ef340ffb2e21c87d4d7b56b6398436d9 2024-08-19T01:45:38.405 INFO:journalctl@ceph.osd.9.smithi083.stdout:Aug 19 01:45:38 smithi083 podman[85117]: 2024-08-19 01:45:38.161992959 +0000 UTC m=+0.136988139 container create a4465325245dc0dba11a044e0336ac747e1a00690bbf3c0264293a28e043da0e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c9dd28080a631be25c77c36c63b17a3ef340ffb2e21c87d4d7b56b6398436d9, name=ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-9-deactivate, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, RELEASE=squid-de09dd0, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, ceph=True, io.buildah.version=1.36.0, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-08-19T01:45:38.405 INFO:journalctl@ceph.osd.9.smithi083.stdout:Aug 19 01:45:38 smithi083 podman[85117]: 2024-08-19 01:45:38.389642957 +0000 UTC m=+0.364638144 container init a4465325245dc0dba11a044e0336ac747e1a00690bbf3c0264293a28e043da0e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c9dd28080a631be25c77c36c63b17a3ef340ffb2e21c87d4d7b56b6398436d9, name=ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-9-deactivate, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-de09dd0, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True, io.buildah.version=1.36.0, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=) 2024-08-19T01:45:38.405 INFO:journalctl@ceph.osd.9.smithi083.stdout:Aug 19 01:45:38 smithi083 podman[85117]: 2024-08-19 01:45:38.396356252 +0000 UTC m=+0.371351439 container start a4465325245dc0dba11a044e0336ac747e1a00690bbf3c0264293a28e043da0e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c9dd28080a631be25c77c36c63b17a3ef340ffb2e21c87d4d7b56b6398436d9, name=ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-9-deactivate, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-de09dd0, CEPH_POINT_RELEASE=, io.buildah.version=1.36.0, ceph=True, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image) 2024-08-19T01:45:38.702 INFO:journalctl@ceph.osd.10.smithi117.stdout:Aug 19 01:45:38 smithi117 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-10[46616]: 2024-08-19T01:45:38.423+0000 7f34b3bf7640 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-08-19T01:44:24.060751+0000 front 2024-08-19T01:44:24.060716+0000 (oldest deadline 2024-08-19T01:44:47.560677+0000) 2024-08-19T01:45:38.702 INFO:journalctl@ceph.osd.10.smithi117.stdout:Aug 19 01:45:38 smithi117 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-10[46616]: 2024-08-19T01:45:38.423+0000 7f34b3bf7640 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-08-19T01:44:28.661503+0000 front 2024-08-19T01:44:28.661522+0000 (oldest deadline 2024-08-19T01:44:56.262298+0000) 2024-08-19T01:45:38.702 INFO:journalctl@ceph.osd.10.smithi117.stdout:Aug 19 01:45:38 smithi117 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-10[46616]: 2024-08-19T01:45:38.423+0000 7f34b3bf7640 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-08-19T01:44:38.563674+0000 front 2024-08-19T01:44:38.563520+0000 (oldest deadline 2024-08-19T01:45:02.063517+0000) 2024-08-19T01:45:38.702 INFO:journalctl@ceph.osd.10.smithi117.stdout:Aug 19 01:45:38 smithi117 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-10[46616]: 2024-08-19T01:45:38.423+0000 7f34b3bf7640 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.83:6814 osd.3 since back 2024-08-19T01:44:47.265176+0000 front 2024-08-19T01:44:47.265102+0000 (oldest deadline 2024-08-19T01:45:09.564999+0000) 2024-08-19T01:45:38.703 INFO:journalctl@ceph.osd.10.smithi117.stdout:Aug 19 01:45:38 smithi117 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-10[46616]: 2024-08-19T01:45:38.423+0000 7f34b3bf7640 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.117:6812 osd.4 since back 2024-08-19T01:44:54.865666+0000 front 2024-08-19T01:44:54.865695+0000 (oldest deadline 2024-08-19T01:45:19.565735+0000) 2024-08-19T01:45:38.703 INFO:journalctl@ceph.osd.10.smithi117.stdout:Aug 19 01:45:38 smithi117 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-10[46616]: 2024-08-19T01:45:38.423+0000 7f34b3bf7640 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.167:6812 osd.5 since back 2024-08-19T01:45:03.066733+0000 front 2024-08-19T01:45:03.066769+0000 (oldest deadline 2024-08-19T01:45:25.366703+0000) 2024-08-19T01:45:38.703 INFO:journalctl@ceph.osd.10.smithi117.stdout:Aug 19 01:45:38 smithi117 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-10[46616]: 2024-08-19T01:45:38.423+0000 7f34b3bf7640 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.83:6822 osd.6 since back 2024-08-19T01:45:11.168172+0000 front 2024-08-19T01:45:11.168153+0000 (oldest deadline 2024-08-19T01:45:32.867945+0000) 2024-08-19T01:45:38.904 INFO:journalctl@ceph.osd.9.smithi083.stdout:Aug 19 01:45:38 smithi083 podman[85117]: 2024-08-19 01:45:38.522785137 +0000 UTC m=+0.497780332 container attach a4465325245dc0dba11a044e0336ac747e1a00690bbf3c0264293a28e043da0e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c9dd28080a631be25c77c36c63b17a3ef340ffb2e21c87d4d7b56b6398436d9, name=ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-9-deactivate, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, ceph=True, io.buildah.version=1.36.0, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-de09dd0, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS) 2024-08-19T01:45:38.904 INFO:journalctl@ceph.osd.9.smithi083.stdout:Aug 19 01:45:38 smithi083 podman[85117]: 2024-08-19 01:45:38.648822584 +0000 UTC m=+0.623817775 container died a4465325245dc0dba11a044e0336ac747e1a00690bbf3c0264293a28e043da0e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c9dd28080a631be25c77c36c63b17a3ef340ffb2e21c87d4d7b56b6398436d9, name=ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-9-deactivate, ceph=True, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, RELEASE=squid-de09dd0, GIT_CLEAN=True, io.buildah.version=1.36.0, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-08-19T01:45:38.966 INFO:journalctl@ceph.osd.11.smithi167.stdout:Aug 19 01:45:38 smithi167 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-11[46643]: 2024-08-19T01:45:38.595+0000 7f447f624640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-08-19T01:44:22.880210+0000 front 2024-08-19T01:44:22.880404+0000 (oldest deadline 2024-08-19T01:44:45.780352+0000) 2024-08-19T01:45:38.966 INFO:journalctl@ceph.osd.11.smithi167.stdout:Aug 19 01:45:38 smithi167 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-11[46643]: 2024-08-19T01:45:38.595+0000 7f447f624640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-08-19T01:44:31.081546+0000 front 2024-08-19T01:44:31.081654+0000 (oldest deadline 2024-08-19T01:44:53.981595+0000) 2024-08-19T01:45:38.966 INFO:journalctl@ceph.osd.11.smithi167.stdout:Aug 19 01:45:38 smithi167 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-11[46643]: 2024-08-19T01:45:38.595+0000 7f447f624640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-08-19T01:44:39.282638+0000 front 2024-08-19T01:44:39.282723+0000 (oldest deadline 2024-08-19T01:45:04.582709+0000) 2024-08-19T01:45:38.966 INFO:journalctl@ceph.osd.11.smithi167.stdout:Aug 19 01:45:38 smithi167 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-11[46643]: 2024-08-19T01:45:38.595+0000 7f447f624640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.83:6814 osd.3 since back 2024-08-19T01:44:45.083762+0000 front 2024-08-19T01:44:45.083777+0000 (oldest deadline 2024-08-19T01:45:10.383738+0000) 2024-08-19T01:45:38.966 INFO:journalctl@ceph.osd.11.smithi167.stdout:Aug 19 01:45:38 smithi167 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-11[46643]: 2024-08-19T01:45:38.595+0000 7f447f624640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.117:6812 osd.4 since back 2024-08-19T01:44:55.585644+0000 front 2024-08-19T01:44:55.585746+0000 (oldest deadline 2024-08-19T01:45:20.885631+0000) 2024-08-19T01:45:38.966 INFO:journalctl@ceph.osd.11.smithi167.stdout:Aug 19 01:45:38 smithi167 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-11[46643]: 2024-08-19T01:45:38.595+0000 7f447f624640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.167:6812 osd.5 since back 2024-08-19T01:45:02.586329+0000 front 2024-08-19T01:45:02.586392+0000 (oldest deadline 2024-08-19T01:45:28.486376+0000) 2024-08-19T01:45:39.242 INFO:journalctl@ceph.osd.9.smithi083.stdout:Aug 19 01:45:39 smithi083 podman[85139]: 2024-08-19 01:45:39.234414721 +0000 UTC m=+0.572596992 container remove a4465325245dc0dba11a044e0336ac747e1a00690bbf3c0264293a28e043da0e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c9dd28080a631be25c77c36c63b17a3ef340ffb2e21c87d4d7b56b6398436d9, name=ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-9-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, ceph=True, org.label-schema.schema-version=1.0, io.buildah.version=1.36.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, RELEASE=squid-de09dd0, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=) 2024-08-19T01:45:39.267 DEBUG:teuthology.orchestra.run.smithi083:> sudo pkill -f 'journalctl -f -n 0 -u ceph-bad8780e-5dc9-11ef-bccf-c7b262605968@osd.9.service' 2024-08-19T01:45:39.342 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-08-19T01:45:39.342 INFO:tasks.cephadm.osd.9:Stopped osd.9 2024-08-19T01:45:39.342 INFO:tasks.cephadm.osd.10:Stopping osd.10... 2024-08-19T01:45:39.342 DEBUG:teuthology.orchestra.run.smithi117:> sudo systemctl stop ceph-bad8780e-5dc9-11ef-bccf-c7b262605968@osd.10 2024-08-19T01:45:39.680 INFO:journalctl@ceph.osd.10.smithi117.stdout:Aug 19 01:45:39 smithi117 systemd[1]: Stopping Ceph osd.10 for bad8780e-5dc9-11ef-bccf-c7b262605968... 2024-08-19T01:45:39.680 INFO:journalctl@ceph.osd.10.smithi117.stdout:Aug 19 01:45:39 smithi117 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-10[46616]: 2024-08-19T01:45:39.446+0000 7f34b3bf7640 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-08-19T01:44:24.060751+0000 front 2024-08-19T01:44:24.060716+0000 (oldest deadline 2024-08-19T01:44:47.560677+0000) 2024-08-19T01:45:39.680 INFO:journalctl@ceph.osd.10.smithi117.stdout:Aug 19 01:45:39 smithi117 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-10[46616]: 2024-08-19T01:45:39.446+0000 7f34b3bf7640 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-08-19T01:44:28.661503+0000 front 2024-08-19T01:44:28.661522+0000 (oldest deadline 2024-08-19T01:44:56.262298+0000) 2024-08-19T01:45:39.680 INFO:journalctl@ceph.osd.10.smithi117.stdout:Aug 19 01:45:39 smithi117 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-10[46616]: 2024-08-19T01:45:39.446+0000 7f34b3bf7640 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-08-19T01:44:38.563674+0000 front 2024-08-19T01:44:38.563520+0000 (oldest deadline 2024-08-19T01:45:02.063517+0000) 2024-08-19T01:45:39.681 INFO:journalctl@ceph.osd.10.smithi117.stdout:Aug 19 01:45:39 smithi117 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-10[46616]: 2024-08-19T01:45:39.446+0000 7f34b3bf7640 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.83:6814 osd.3 since back 2024-08-19T01:44:47.265176+0000 front 2024-08-19T01:44:47.265102+0000 (oldest deadline 2024-08-19T01:45:09.564999+0000) 2024-08-19T01:45:39.681 INFO:journalctl@ceph.osd.10.smithi117.stdout:Aug 19 01:45:39 smithi117 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-10[46616]: 2024-08-19T01:45:39.446+0000 7f34b3bf7640 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.117:6812 osd.4 since back 2024-08-19T01:44:54.865666+0000 front 2024-08-19T01:44:54.865695+0000 (oldest deadline 2024-08-19T01:45:19.565735+0000) 2024-08-19T01:45:39.681 INFO:journalctl@ceph.osd.10.smithi117.stdout:Aug 19 01:45:39 smithi117 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-10[46616]: 2024-08-19T01:45:39.446+0000 7f34b3bf7640 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.167:6812 osd.5 since back 2024-08-19T01:45:03.066733+0000 front 2024-08-19T01:45:03.066769+0000 (oldest deadline 2024-08-19T01:45:25.366703+0000) 2024-08-19T01:45:39.681 INFO:journalctl@ceph.osd.10.smithi117.stdout:Aug 19 01:45:39 smithi117 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-10[46616]: 2024-08-19T01:45:39.446+0000 7f34b3bf7640 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.83:6822 osd.6 since back 2024-08-19T01:45:11.168172+0000 front 2024-08-19T01:45:11.168153+0000 (oldest deadline 2024-08-19T01:45:32.867945+0000) 2024-08-19T01:45:39.681 INFO:journalctl@ceph.osd.10.smithi117.stdout:Aug 19 01:45:39 smithi117 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-10[46616]: 2024-08-19T01:45:39.679+0000 7f34b6df1640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.10 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-08-19T01:45:39.681 INFO:journalctl@ceph.osd.10.smithi117.stdout:Aug 19 01:45:39 smithi117 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-10[46616]: 2024-08-19T01:45:39.679+0000 7f34b6df1640 -1 osd.10 89 *** Got signal Terminated *** 2024-08-19T01:45:39.681 INFO:journalctl@ceph.osd.10.smithi117.stdout:Aug 19 01:45:39 smithi117 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-10[46616]: 2024-08-19T01:45:39.679+0000 7f34b6df1640 -1 osd.10 89 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-08-19T01:45:39.965 INFO:journalctl@ceph.osd.11.smithi167.stdout:Aug 19 01:45:39 smithi167 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-11[46643]: 2024-08-19T01:45:39.598+0000 7f447f624640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-08-19T01:44:22.880210+0000 front 2024-08-19T01:44:22.880404+0000 (oldest deadline 2024-08-19T01:44:45.780352+0000) 2024-08-19T01:45:39.966 INFO:journalctl@ceph.osd.11.smithi167.stdout:Aug 19 01:45:39 smithi167 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-11[46643]: 2024-08-19T01:45:39.598+0000 7f447f624640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-08-19T01:44:31.081546+0000 front 2024-08-19T01:44:31.081654+0000 (oldest deadline 2024-08-19T01:44:53.981595+0000) 2024-08-19T01:45:39.966 INFO:journalctl@ceph.osd.11.smithi167.stdout:Aug 19 01:45:39 smithi167 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-11[46643]: 2024-08-19T01:45:39.598+0000 7f447f624640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-08-19T01:44:39.282638+0000 front 2024-08-19T01:44:39.282723+0000 (oldest deadline 2024-08-19T01:45:04.582709+0000) 2024-08-19T01:45:39.966 INFO:journalctl@ceph.osd.11.smithi167.stdout:Aug 19 01:45:39 smithi167 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-11[46643]: 2024-08-19T01:45:39.598+0000 7f447f624640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.83:6814 osd.3 since back 2024-08-19T01:44:45.083762+0000 front 2024-08-19T01:44:45.083777+0000 (oldest deadline 2024-08-19T01:45:10.383738+0000) 2024-08-19T01:45:39.966 INFO:journalctl@ceph.osd.11.smithi167.stdout:Aug 19 01:45:39 smithi167 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-11[46643]: 2024-08-19T01:45:39.598+0000 7f447f624640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.117:6812 osd.4 since back 2024-08-19T01:44:55.585644+0000 front 2024-08-19T01:44:55.585746+0000 (oldest deadline 2024-08-19T01:45:20.885631+0000) 2024-08-19T01:45:39.966 INFO:journalctl@ceph.osd.11.smithi167.stdout:Aug 19 01:45:39 smithi167 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-11[46643]: 2024-08-19T01:45:39.598+0000 7f447f624640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.167:6812 osd.5 since back 2024-08-19T01:45:02.586329+0000 front 2024-08-19T01:45:02.586392+0000 (oldest deadline 2024-08-19T01:45:28.486376+0000) 2024-08-19T01:45:40.702 INFO:journalctl@ceph.osd.10.smithi117.stdout:Aug 19 01:45:40 smithi117 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-10[46616]: 2024-08-19T01:45:40.406+0000 7f34b3bf7640 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-08-19T01:44:24.060751+0000 front 2024-08-19T01:44:24.060716+0000 (oldest deadline 2024-08-19T01:44:47.560677+0000) 2024-08-19T01:45:40.702 INFO:journalctl@ceph.osd.10.smithi117.stdout:Aug 19 01:45:40 smithi117 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-10[46616]: 2024-08-19T01:45:40.406+0000 7f34b3bf7640 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-08-19T01:44:28.661503+0000 front 2024-08-19T01:44:28.661522+0000 (oldest deadline 2024-08-19T01:44:56.262298+0000) 2024-08-19T01:45:40.703 INFO:journalctl@ceph.osd.10.smithi117.stdout:Aug 19 01:45:40 smithi117 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-10[46616]: 2024-08-19T01:45:40.406+0000 7f34b3bf7640 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-08-19T01:44:38.563674+0000 front 2024-08-19T01:44:38.563520+0000 (oldest deadline 2024-08-19T01:45:02.063517+0000) 2024-08-19T01:45:40.703 INFO:journalctl@ceph.osd.10.smithi117.stdout:Aug 19 01:45:40 smithi117 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-10[46616]: 2024-08-19T01:45:40.406+0000 7f34b3bf7640 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.83:6814 osd.3 since back 2024-08-19T01:44:47.265176+0000 front 2024-08-19T01:44:47.265102+0000 (oldest deadline 2024-08-19T01:45:09.564999+0000) 2024-08-19T01:45:40.703 INFO:journalctl@ceph.osd.10.smithi117.stdout:Aug 19 01:45:40 smithi117 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-10[46616]: 2024-08-19T01:45:40.406+0000 7f34b3bf7640 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.117:6812 osd.4 since back 2024-08-19T01:44:54.865666+0000 front 2024-08-19T01:44:54.865695+0000 (oldest deadline 2024-08-19T01:45:19.565735+0000) 2024-08-19T01:45:40.703 INFO:journalctl@ceph.osd.10.smithi117.stdout:Aug 19 01:45:40 smithi117 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-10[46616]: 2024-08-19T01:45:40.406+0000 7f34b3bf7640 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.167:6812 osd.5 since back 2024-08-19T01:45:03.066733+0000 front 2024-08-19T01:45:03.066769+0000 (oldest deadline 2024-08-19T01:45:25.366703+0000) 2024-08-19T01:45:40.703 INFO:journalctl@ceph.osd.10.smithi117.stdout:Aug 19 01:45:40 smithi117 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-10[46616]: 2024-08-19T01:45:40.406+0000 7f34b3bf7640 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.83:6822 osd.6 since back 2024-08-19T01:45:11.168172+0000 front 2024-08-19T01:45:11.168153+0000 (oldest deadline 2024-08-19T01:45:32.867945+0000) 2024-08-19T01:45:40.966 INFO:journalctl@ceph.osd.11.smithi167.stdout:Aug 19 01:45:40 smithi167 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-11[46643]: 2024-08-19T01:45:40.591+0000 7f447f624640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-08-19T01:44:22.880210+0000 front 2024-08-19T01:44:22.880404+0000 (oldest deadline 2024-08-19T01:44:45.780352+0000) 2024-08-19T01:45:40.966 INFO:journalctl@ceph.osd.11.smithi167.stdout:Aug 19 01:45:40 smithi167 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-11[46643]: 2024-08-19T01:45:40.591+0000 7f447f624640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-08-19T01:44:31.081546+0000 front 2024-08-19T01:44:31.081654+0000 (oldest deadline 2024-08-19T01:44:53.981595+0000) 2024-08-19T01:45:40.966 INFO:journalctl@ceph.osd.11.smithi167.stdout:Aug 19 01:45:40 smithi167 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-11[46643]: 2024-08-19T01:45:40.591+0000 7f447f624640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-08-19T01:44:39.282638+0000 front 2024-08-19T01:44:39.282723+0000 (oldest deadline 2024-08-19T01:45:04.582709+0000) 2024-08-19T01:45:40.966 INFO:journalctl@ceph.osd.11.smithi167.stdout:Aug 19 01:45:40 smithi167 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-11[46643]: 2024-08-19T01:45:40.591+0000 7f447f624640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.83:6814 osd.3 since back 2024-08-19T01:44:45.083762+0000 front 2024-08-19T01:44:45.083777+0000 (oldest deadline 2024-08-19T01:45:10.383738+0000) 2024-08-19T01:45:40.967 INFO:journalctl@ceph.osd.11.smithi167.stdout:Aug 19 01:45:40 smithi167 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-11[46643]: 2024-08-19T01:45:40.591+0000 7f447f624640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.117:6812 osd.4 since back 2024-08-19T01:44:55.585644+0000 front 2024-08-19T01:44:55.585746+0000 (oldest deadline 2024-08-19T01:45:20.885631+0000) 2024-08-19T01:45:40.967 INFO:journalctl@ceph.osd.11.smithi167.stdout:Aug 19 01:45:40 smithi167 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-11[46643]: 2024-08-19T01:45:40.591+0000 7f447f624640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.167:6812 osd.5 since back 2024-08-19T01:45:02.586329+0000 front 2024-08-19T01:45:02.586392+0000 (oldest deadline 2024-08-19T01:45:28.486376+0000) 2024-08-19T01:45:41.703 INFO:journalctl@ceph.osd.10.smithi117.stdout:Aug 19 01:45:41 smithi117 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-10[46616]: 2024-08-19T01:45:41.388+0000 7f34b3bf7640 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-08-19T01:44:24.060751+0000 front 2024-08-19T01:44:24.060716+0000 (oldest deadline 2024-08-19T01:44:47.560677+0000) 2024-08-19T01:45:41.703 INFO:journalctl@ceph.osd.10.smithi117.stdout:Aug 19 01:45:41 smithi117 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-10[46616]: 2024-08-19T01:45:41.388+0000 7f34b3bf7640 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-08-19T01:44:28.661503+0000 front 2024-08-19T01:44:28.661522+0000 (oldest deadline 2024-08-19T01:44:56.262298+0000) 2024-08-19T01:45:41.703 INFO:journalctl@ceph.osd.10.smithi117.stdout:Aug 19 01:45:41 smithi117 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-10[46616]: 2024-08-19T01:45:41.388+0000 7f34b3bf7640 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-08-19T01:44:38.563674+0000 front 2024-08-19T01:44:38.563520+0000 (oldest deadline 2024-08-19T01:45:02.063517+0000) 2024-08-19T01:45:41.703 INFO:journalctl@ceph.osd.10.smithi117.stdout:Aug 19 01:45:41 smithi117 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-10[46616]: 2024-08-19T01:45:41.388+0000 7f34b3bf7640 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.83:6814 osd.3 since back 2024-08-19T01:44:47.265176+0000 front 2024-08-19T01:44:47.265102+0000 (oldest deadline 2024-08-19T01:45:09.564999+0000) 2024-08-19T01:45:41.703 INFO:journalctl@ceph.osd.10.smithi117.stdout:Aug 19 01:45:41 smithi117 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-10[46616]: 2024-08-19T01:45:41.388+0000 7f34b3bf7640 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.117:6812 osd.4 since back 2024-08-19T01:44:54.865666+0000 front 2024-08-19T01:44:54.865695+0000 (oldest deadline 2024-08-19T01:45:19.565735+0000) 2024-08-19T01:45:41.703 INFO:journalctl@ceph.osd.10.smithi117.stdout:Aug 19 01:45:41 smithi117 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-10[46616]: 2024-08-19T01:45:41.388+0000 7f34b3bf7640 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.167:6812 osd.5 since back 2024-08-19T01:45:03.066733+0000 front 2024-08-19T01:45:03.066769+0000 (oldest deadline 2024-08-19T01:45:25.366703+0000) 2024-08-19T01:45:41.704 INFO:journalctl@ceph.osd.10.smithi117.stdout:Aug 19 01:45:41 smithi117 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-10[46616]: 2024-08-19T01:45:41.388+0000 7f34b3bf7640 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.83:6822 osd.6 since back 2024-08-19T01:45:11.168172+0000 front 2024-08-19T01:45:11.168153+0000 (oldest deadline 2024-08-19T01:45:32.867945+0000) 2024-08-19T01:45:41.966 INFO:journalctl@ceph.osd.11.smithi167.stdout:Aug 19 01:45:41 smithi167 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-11[46643]: 2024-08-19T01:45:41.622+0000 7f447f624640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-08-19T01:44:22.880210+0000 front 2024-08-19T01:44:22.880404+0000 (oldest deadline 2024-08-19T01:44:45.780352+0000) 2024-08-19T01:45:41.966 INFO:journalctl@ceph.osd.11.smithi167.stdout:Aug 19 01:45:41 smithi167 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-11[46643]: 2024-08-19T01:45:41.622+0000 7f447f624640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-08-19T01:44:31.081546+0000 front 2024-08-19T01:44:31.081654+0000 (oldest deadline 2024-08-19T01:44:53.981595+0000) 2024-08-19T01:45:41.966 INFO:journalctl@ceph.osd.11.smithi167.stdout:Aug 19 01:45:41 smithi167 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-11[46643]: 2024-08-19T01:45:41.622+0000 7f447f624640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-08-19T01:44:39.282638+0000 front 2024-08-19T01:44:39.282723+0000 (oldest deadline 2024-08-19T01:45:04.582709+0000) 2024-08-19T01:45:41.966 INFO:journalctl@ceph.osd.11.smithi167.stdout:Aug 19 01:45:41 smithi167 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-11[46643]: 2024-08-19T01:45:41.622+0000 7f447f624640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.83:6814 osd.3 since back 2024-08-19T01:44:45.083762+0000 front 2024-08-19T01:44:45.083777+0000 (oldest deadline 2024-08-19T01:45:10.383738+0000) 2024-08-19T01:45:41.966 INFO:journalctl@ceph.osd.11.smithi167.stdout:Aug 19 01:45:41 smithi167 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-11[46643]: 2024-08-19T01:45:41.622+0000 7f447f624640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.117:6812 osd.4 since back 2024-08-19T01:44:55.585644+0000 front 2024-08-19T01:44:55.585746+0000 (oldest deadline 2024-08-19T01:45:20.885631+0000) 2024-08-19T01:45:41.966 INFO:journalctl@ceph.osd.11.smithi167.stdout:Aug 19 01:45:41 smithi167 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-11[46643]: 2024-08-19T01:45:41.622+0000 7f447f624640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.167:6812 osd.5 since back 2024-08-19T01:45:02.586329+0000 front 2024-08-19T01:45:02.586392+0000 (oldest deadline 2024-08-19T01:45:28.486376+0000) 2024-08-19T01:45:42.702 INFO:journalctl@ceph.osd.10.smithi117.stdout:Aug 19 01:45:42 smithi117 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-10[46616]: 2024-08-19T01:45:42.377+0000 7f34b3bf7640 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-08-19T01:44:24.060751+0000 front 2024-08-19T01:44:24.060716+0000 (oldest deadline 2024-08-19T01:44:47.560677+0000) 2024-08-19T01:45:42.703 INFO:journalctl@ceph.osd.10.smithi117.stdout:Aug 19 01:45:42 smithi117 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-10[46616]: 2024-08-19T01:45:42.377+0000 7f34b3bf7640 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-08-19T01:44:28.661503+0000 front 2024-08-19T01:44:28.661522+0000 (oldest deadline 2024-08-19T01:44:56.262298+0000) 2024-08-19T01:45:42.703 INFO:journalctl@ceph.osd.10.smithi117.stdout:Aug 19 01:45:42 smithi117 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-10[46616]: 2024-08-19T01:45:42.377+0000 7f34b3bf7640 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-08-19T01:44:38.563674+0000 front 2024-08-19T01:44:38.563520+0000 (oldest deadline 2024-08-19T01:45:02.063517+0000) 2024-08-19T01:45:42.703 INFO:journalctl@ceph.osd.10.smithi117.stdout:Aug 19 01:45:42 smithi117 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-10[46616]: 2024-08-19T01:45:42.377+0000 7f34b3bf7640 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.83:6814 osd.3 since back 2024-08-19T01:44:47.265176+0000 front 2024-08-19T01:44:47.265102+0000 (oldest deadline 2024-08-19T01:45:09.564999+0000) 2024-08-19T01:45:42.703 INFO:journalctl@ceph.osd.10.smithi117.stdout:Aug 19 01:45:42 smithi117 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-10[46616]: 2024-08-19T01:45:42.377+0000 7f34b3bf7640 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.117:6812 osd.4 since back 2024-08-19T01:44:54.865666+0000 front 2024-08-19T01:44:54.865695+0000 (oldest deadline 2024-08-19T01:45:19.565735+0000) 2024-08-19T01:45:42.703 INFO:journalctl@ceph.osd.10.smithi117.stdout:Aug 19 01:45:42 smithi117 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-10[46616]: 2024-08-19T01:45:42.377+0000 7f34b3bf7640 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.167:6812 osd.5 since back 2024-08-19T01:45:03.066733+0000 front 2024-08-19T01:45:03.066769+0000 (oldest deadline 2024-08-19T01:45:25.366703+0000) 2024-08-19T01:45:42.703 INFO:journalctl@ceph.osd.10.smithi117.stdout:Aug 19 01:45:42 smithi117 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-10[46616]: 2024-08-19T01:45:42.377+0000 7f34b3bf7640 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.83:6822 osd.6 since back 2024-08-19T01:45:11.168172+0000 front 2024-08-19T01:45:11.168153+0000 (oldest deadline 2024-08-19T01:45:32.867945+0000) 2024-08-19T01:45:42.966 INFO:journalctl@ceph.osd.11.smithi167.stdout:Aug 19 01:45:42 smithi167 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-11[46643]: 2024-08-19T01:45:42.579+0000 7f447f624640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-08-19T01:44:22.880210+0000 front 2024-08-19T01:44:22.880404+0000 (oldest deadline 2024-08-19T01:44:45.780352+0000) 2024-08-19T01:45:42.966 INFO:journalctl@ceph.osd.11.smithi167.stdout:Aug 19 01:45:42 smithi167 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-11[46643]: 2024-08-19T01:45:42.579+0000 7f447f624640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-08-19T01:44:31.081546+0000 front 2024-08-19T01:44:31.081654+0000 (oldest deadline 2024-08-19T01:44:53.981595+0000) 2024-08-19T01:45:42.966 INFO:journalctl@ceph.osd.11.smithi167.stdout:Aug 19 01:45:42 smithi167 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-11[46643]: 2024-08-19T01:45:42.579+0000 7f447f624640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-08-19T01:44:39.282638+0000 front 2024-08-19T01:44:39.282723+0000 (oldest deadline 2024-08-19T01:45:04.582709+0000) 2024-08-19T01:45:42.966 INFO:journalctl@ceph.osd.11.smithi167.stdout:Aug 19 01:45:42 smithi167 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-11[46643]: 2024-08-19T01:45:42.579+0000 7f447f624640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.83:6814 osd.3 since back 2024-08-19T01:44:45.083762+0000 front 2024-08-19T01:44:45.083777+0000 (oldest deadline 2024-08-19T01:45:10.383738+0000) 2024-08-19T01:45:42.966 INFO:journalctl@ceph.osd.11.smithi167.stdout:Aug 19 01:45:42 smithi167 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-11[46643]: 2024-08-19T01:45:42.579+0000 7f447f624640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.117:6812 osd.4 since back 2024-08-19T01:44:55.585644+0000 front 2024-08-19T01:44:55.585746+0000 (oldest deadline 2024-08-19T01:45:20.885631+0000) 2024-08-19T01:45:42.966 INFO:journalctl@ceph.osd.11.smithi167.stdout:Aug 19 01:45:42 smithi167 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-11[46643]: 2024-08-19T01:45:42.579+0000 7f447f624640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.167:6812 osd.5 since back 2024-08-19T01:45:02.586329+0000 front 2024-08-19T01:45:02.586392+0000 (oldest deadline 2024-08-19T01:45:28.486376+0000) 2024-08-19T01:45:42.966 INFO:journalctl@ceph.osd.11.smithi167.stdout:Aug 19 01:45:42 smithi167 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-11[46643]: 2024-08-19T01:45:42.579+0000 7f447f624640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.117:6820 osd.7 since back 2024-08-19T01:45:20.188393+0000 front 2024-08-19T01:45:20.188432+0000 (oldest deadline 2024-08-19T01:45:42.488373+0000) 2024-08-19T01:45:43.702 INFO:journalctl@ceph.osd.10.smithi117.stdout:Aug 19 01:45:43 smithi117 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-10[46616]: 2024-08-19T01:45:43.369+0000 7f34b3bf7640 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-08-19T01:44:24.060751+0000 front 2024-08-19T01:44:24.060716+0000 (oldest deadline 2024-08-19T01:44:47.560677+0000) 2024-08-19T01:45:43.703 INFO:journalctl@ceph.osd.10.smithi117.stdout:Aug 19 01:45:43 smithi117 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-10[46616]: 2024-08-19T01:45:43.369+0000 7f34b3bf7640 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-08-19T01:44:28.661503+0000 front 2024-08-19T01:44:28.661522+0000 (oldest deadline 2024-08-19T01:44:56.262298+0000) 2024-08-19T01:45:43.703 INFO:journalctl@ceph.osd.10.smithi117.stdout:Aug 19 01:45:43 smithi117 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-10[46616]: 2024-08-19T01:45:43.369+0000 7f34b3bf7640 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-08-19T01:44:38.563674+0000 front 2024-08-19T01:44:38.563520+0000 (oldest deadline 2024-08-19T01:45:02.063517+0000) 2024-08-19T01:45:43.703 INFO:journalctl@ceph.osd.10.smithi117.stdout:Aug 19 01:45:43 smithi117 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-10[46616]: 2024-08-19T01:45:43.369+0000 7f34b3bf7640 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.83:6814 osd.3 since back 2024-08-19T01:44:47.265176+0000 front 2024-08-19T01:44:47.265102+0000 (oldest deadline 2024-08-19T01:45:09.564999+0000) 2024-08-19T01:45:43.703 INFO:journalctl@ceph.osd.10.smithi117.stdout:Aug 19 01:45:43 smithi117 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-10[46616]: 2024-08-19T01:45:43.369+0000 7f34b3bf7640 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.117:6812 osd.4 since back 2024-08-19T01:44:54.865666+0000 front 2024-08-19T01:44:54.865695+0000 (oldest deadline 2024-08-19T01:45:19.565735+0000) 2024-08-19T01:45:43.703 INFO:journalctl@ceph.osd.10.smithi117.stdout:Aug 19 01:45:43 smithi117 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-10[46616]: 2024-08-19T01:45:43.369+0000 7f34b3bf7640 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.167:6812 osd.5 since back 2024-08-19T01:45:03.066733+0000 front 2024-08-19T01:45:03.066769+0000 (oldest deadline 2024-08-19T01:45:25.366703+0000) 2024-08-19T01:45:43.703 INFO:journalctl@ceph.osd.10.smithi117.stdout:Aug 19 01:45:43 smithi117 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-10[46616]: 2024-08-19T01:45:43.369+0000 7f34b3bf7640 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.83:6822 osd.6 since back 2024-08-19T01:45:11.168172+0000 front 2024-08-19T01:45:11.168153+0000 (oldest deadline 2024-08-19T01:45:32.867945+0000) 2024-08-19T01:45:43.966 INFO:journalctl@ceph.osd.11.smithi167.stdout:Aug 19 01:45:43 smithi167 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-11[46643]: 2024-08-19T01:45:43.614+0000 7f447f624640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-08-19T01:44:22.880210+0000 front 2024-08-19T01:44:22.880404+0000 (oldest deadline 2024-08-19T01:44:45.780352+0000) 2024-08-19T01:45:43.966 INFO:journalctl@ceph.osd.11.smithi167.stdout:Aug 19 01:45:43 smithi167 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-11[46643]: 2024-08-19T01:45:43.614+0000 7f447f624640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-08-19T01:44:31.081546+0000 front 2024-08-19T01:44:31.081654+0000 (oldest deadline 2024-08-19T01:44:53.981595+0000) 2024-08-19T01:45:43.966 INFO:journalctl@ceph.osd.11.smithi167.stdout:Aug 19 01:45:43 smithi167 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-11[46643]: 2024-08-19T01:45:43.614+0000 7f447f624640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-08-19T01:44:39.282638+0000 front 2024-08-19T01:44:39.282723+0000 (oldest deadline 2024-08-19T01:45:04.582709+0000) 2024-08-19T01:45:43.966 INFO:journalctl@ceph.osd.11.smithi167.stdout:Aug 19 01:45:43 smithi167 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-11[46643]: 2024-08-19T01:45:43.614+0000 7f447f624640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.83:6814 osd.3 since back 2024-08-19T01:44:45.083762+0000 front 2024-08-19T01:44:45.083777+0000 (oldest deadline 2024-08-19T01:45:10.383738+0000) 2024-08-19T01:45:43.966 INFO:journalctl@ceph.osd.11.smithi167.stdout:Aug 19 01:45:43 smithi167 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-11[46643]: 2024-08-19T01:45:43.614+0000 7f447f624640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.117:6812 osd.4 since back 2024-08-19T01:44:55.585644+0000 front 2024-08-19T01:44:55.585746+0000 (oldest deadline 2024-08-19T01:45:20.885631+0000) 2024-08-19T01:45:43.966 INFO:journalctl@ceph.osd.11.smithi167.stdout:Aug 19 01:45:43 smithi167 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-11[46643]: 2024-08-19T01:45:43.614+0000 7f447f624640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.167:6812 osd.5 since back 2024-08-19T01:45:02.586329+0000 front 2024-08-19T01:45:02.586392+0000 (oldest deadline 2024-08-19T01:45:28.486376+0000) 2024-08-19T01:45:43.967 INFO:journalctl@ceph.osd.11.smithi167.stdout:Aug 19 01:45:43 smithi167 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-11[46643]: 2024-08-19T01:45:43.614+0000 7f447f624640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.117:6820 osd.7 since back 2024-08-19T01:45:20.188393+0000 front 2024-08-19T01:45:20.188432+0000 (oldest deadline 2024-08-19T01:45:42.488373+0000) 2024-08-19T01:45:44.702 INFO:journalctl@ceph.osd.10.smithi117.stdout:Aug 19 01:45:44 smithi117 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-10[46616]: 2024-08-19T01:45:44.346+0000 7f34b3bf7640 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-08-19T01:44:24.060751+0000 front 2024-08-19T01:44:24.060716+0000 (oldest deadline 2024-08-19T01:44:47.560677+0000) 2024-08-19T01:45:44.703 INFO:journalctl@ceph.osd.10.smithi117.stdout:Aug 19 01:45:44 smithi117 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-10[46616]: 2024-08-19T01:45:44.346+0000 7f34b3bf7640 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-08-19T01:44:28.661503+0000 front 2024-08-19T01:44:28.661522+0000 (oldest deadline 2024-08-19T01:44:56.262298+0000) 2024-08-19T01:45:44.703 INFO:journalctl@ceph.osd.10.smithi117.stdout:Aug 19 01:45:44 smithi117 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-10[46616]: 2024-08-19T01:45:44.346+0000 7f34b3bf7640 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-08-19T01:44:38.563674+0000 front 2024-08-19T01:44:38.563520+0000 (oldest deadline 2024-08-19T01:45:02.063517+0000) 2024-08-19T01:45:44.703 INFO:journalctl@ceph.osd.10.smithi117.stdout:Aug 19 01:45:44 smithi117 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-10[46616]: 2024-08-19T01:45:44.346+0000 7f34b3bf7640 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.83:6814 osd.3 since back 2024-08-19T01:44:47.265176+0000 front 2024-08-19T01:44:47.265102+0000 (oldest deadline 2024-08-19T01:45:09.564999+0000) 2024-08-19T01:45:44.703 INFO:journalctl@ceph.osd.10.smithi117.stdout:Aug 19 01:45:44 smithi117 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-10[46616]: 2024-08-19T01:45:44.346+0000 7f34b3bf7640 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.117:6812 osd.4 since back 2024-08-19T01:44:54.865666+0000 front 2024-08-19T01:44:54.865695+0000 (oldest deadline 2024-08-19T01:45:19.565735+0000) 2024-08-19T01:45:44.703 INFO:journalctl@ceph.osd.10.smithi117.stdout:Aug 19 01:45:44 smithi117 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-10[46616]: 2024-08-19T01:45:44.346+0000 7f34b3bf7640 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.167:6812 osd.5 since back 2024-08-19T01:45:03.066733+0000 front 2024-08-19T01:45:03.066769+0000 (oldest deadline 2024-08-19T01:45:25.366703+0000) 2024-08-19T01:45:44.703 INFO:journalctl@ceph.osd.10.smithi117.stdout:Aug 19 01:45:44 smithi117 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-10[46616]: 2024-08-19T01:45:44.346+0000 7f34b3bf7640 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.83:6822 osd.6 since back 2024-08-19T01:45:11.168172+0000 front 2024-08-19T01:45:11.168153+0000 (oldest deadline 2024-08-19T01:45:32.867945+0000) 2024-08-19T01:45:44.966 INFO:journalctl@ceph.osd.11.smithi167.stdout:Aug 19 01:45:44 smithi167 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-11[46643]: 2024-08-19T01:45:44.622+0000 7f447f624640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-08-19T01:44:22.880210+0000 front 2024-08-19T01:44:22.880404+0000 (oldest deadline 2024-08-19T01:44:45.780352+0000) 2024-08-19T01:45:44.966 INFO:journalctl@ceph.osd.11.smithi167.stdout:Aug 19 01:45:44 smithi167 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-11[46643]: 2024-08-19T01:45:44.622+0000 7f447f624640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-08-19T01:44:31.081546+0000 front 2024-08-19T01:44:31.081654+0000 (oldest deadline 2024-08-19T01:44:53.981595+0000) 2024-08-19T01:45:44.966 INFO:journalctl@ceph.osd.11.smithi167.stdout:Aug 19 01:45:44 smithi167 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-11[46643]: 2024-08-19T01:45:44.622+0000 7f447f624640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-08-19T01:44:39.282638+0000 front 2024-08-19T01:44:39.282723+0000 (oldest deadline 2024-08-19T01:45:04.582709+0000) 2024-08-19T01:45:44.966 INFO:journalctl@ceph.osd.11.smithi167.stdout:Aug 19 01:45:44 smithi167 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-11[46643]: 2024-08-19T01:45:44.622+0000 7f447f624640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.83:6814 osd.3 since back 2024-08-19T01:44:45.083762+0000 front 2024-08-19T01:44:45.083777+0000 (oldest deadline 2024-08-19T01:45:10.383738+0000) 2024-08-19T01:45:44.966 INFO:journalctl@ceph.osd.11.smithi167.stdout:Aug 19 01:45:44 smithi167 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-11[46643]: 2024-08-19T01:45:44.622+0000 7f447f624640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.117:6812 osd.4 since back 2024-08-19T01:44:55.585644+0000 front 2024-08-19T01:44:55.585746+0000 (oldest deadline 2024-08-19T01:45:20.885631+0000) 2024-08-19T01:45:44.966 INFO:journalctl@ceph.osd.11.smithi167.stdout:Aug 19 01:45:44 smithi167 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-11[46643]: 2024-08-19T01:45:44.622+0000 7f447f624640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.167:6812 osd.5 since back 2024-08-19T01:45:02.586329+0000 front 2024-08-19T01:45:02.586392+0000 (oldest deadline 2024-08-19T01:45:28.486376+0000) 2024-08-19T01:45:44.966 INFO:journalctl@ceph.osd.11.smithi167.stdout:Aug 19 01:45:44 smithi167 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-11[46643]: 2024-08-19T01:45:44.622+0000 7f447f624640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.117:6820 osd.7 since back 2024-08-19T01:45:20.188393+0000 front 2024-08-19T01:45:20.188432+0000 (oldest deadline 2024-08-19T01:45:42.488373+0000) 2024-08-19T01:45:45.420 INFO:journalctl@ceph.osd.10.smithi117.stdout:Aug 19 01:45:45 smithi117 podman[56427]: 2024-08-19 01:45:45.095039745 +0000 UTC m=+5.546993284 container died 64fc4fdcde4889038a00b1b67290382c1c9c26c83748790241d103167d05c4d6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c9dd28080a631be25c77c36c63b17a3ef340ffb2e21c87d4d7b56b6398436d9, name=ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-10, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.license=GPLv2, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, RELEASE=squid-de09dd0, org.label-schema.vendor=CentOS, io.buildah.version=1.36.0, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-08-19T01:45:45.702 INFO:journalctl@ceph.osd.10.smithi117.stdout:Aug 19 01:45:45 smithi117 podman[56427]: 2024-08-19 01:45:45.419890828 +0000 UTC m=+5.871844366 container cleanup 64fc4fdcde4889038a00b1b67290382c1c9c26c83748790241d103167d05c4d6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c9dd28080a631be25c77c36c63b17a3ef340ffb2e21c87d4d7b56b6398436d9, name=ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-10, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, RELEASE=squid-de09dd0, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, io.buildah.version=1.36.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux ) 2024-08-19T01:45:45.702 INFO:journalctl@ceph.osd.10.smithi117.stdout:Aug 19 01:45:45 smithi117 bash[56427]: ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-10 2024-08-19T01:45:45.702 INFO:journalctl@ceph.osd.10.smithi117.stdout:Aug 19 01:45:45 smithi117 podman[56444]: 2024-08-19 01:45:45.629573682 +0000 UTC m=+0.528629921 container remove 64fc4fdcde4889038a00b1b67290382c1c9c26c83748790241d103167d05c4d6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c9dd28080a631be25c77c36c63b17a3ef340ffb2e21c87d4d7b56b6398436d9, name=ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-10, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.build-date=20240731, io.buildah.version=1.36.0, ceph=True, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=squid-de09dd0, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=) 2024-08-19T01:45:45.966 INFO:journalctl@ceph.osd.11.smithi167.stdout:Aug 19 01:45:45 smithi167 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-11[46643]: 2024-08-19T01:45:45.620+0000 7f447f624640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-08-19T01:44:22.880210+0000 front 2024-08-19T01:44:22.880404+0000 (oldest deadline 2024-08-19T01:44:45.780352+0000) 2024-08-19T01:45:45.966 INFO:journalctl@ceph.osd.11.smithi167.stdout:Aug 19 01:45:45 smithi167 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-11[46643]: 2024-08-19T01:45:45.620+0000 7f447f624640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-08-19T01:44:31.081546+0000 front 2024-08-19T01:44:31.081654+0000 (oldest deadline 2024-08-19T01:44:53.981595+0000) 2024-08-19T01:45:45.966 INFO:journalctl@ceph.osd.11.smithi167.stdout:Aug 19 01:45:45 smithi167 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-11[46643]: 2024-08-19T01:45:45.620+0000 7f447f624640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-08-19T01:44:39.282638+0000 front 2024-08-19T01:44:39.282723+0000 (oldest deadline 2024-08-19T01:45:04.582709+0000) 2024-08-19T01:45:45.966 INFO:journalctl@ceph.osd.11.smithi167.stdout:Aug 19 01:45:45 smithi167 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-11[46643]: 2024-08-19T01:45:45.620+0000 7f447f624640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.83:6814 osd.3 since back 2024-08-19T01:44:45.083762+0000 front 2024-08-19T01:44:45.083777+0000 (oldest deadline 2024-08-19T01:45:10.383738+0000) 2024-08-19T01:45:45.966 INFO:journalctl@ceph.osd.11.smithi167.stdout:Aug 19 01:45:45 smithi167 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-11[46643]: 2024-08-19T01:45:45.620+0000 7f447f624640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.117:6812 osd.4 since back 2024-08-19T01:44:55.585644+0000 front 2024-08-19T01:44:55.585746+0000 (oldest deadline 2024-08-19T01:45:20.885631+0000) 2024-08-19T01:45:45.966 INFO:journalctl@ceph.osd.11.smithi167.stdout:Aug 19 01:45:45 smithi167 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-11[46643]: 2024-08-19T01:45:45.620+0000 7f447f624640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.167:6812 osd.5 since back 2024-08-19T01:45:02.586329+0000 front 2024-08-19T01:45:02.586392+0000 (oldest deadline 2024-08-19T01:45:28.486376+0000) 2024-08-19T01:45:45.966 INFO:journalctl@ceph.osd.11.smithi167.stdout:Aug 19 01:45:45 smithi167 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-11[46643]: 2024-08-19T01:45:45.620+0000 7f447f624640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.117:6820 osd.7 since back 2024-08-19T01:45:20.188393+0000 front 2024-08-19T01:45:20.188432+0000 (oldest deadline 2024-08-19T01:45:42.488373+0000) 2024-08-19T01:45:46.333 INFO:journalctl@ceph.osd.10.smithi117.stdout:Aug 19 01:45:46 smithi117 podman[56537]: 2024-08-19 01:45:45.959877855 +0000 UTC m=+0.017016437 image pull ea7a5189f30bad3aca9524788e2965d2368d7ce786df09d4edcd8c3edf9cefba quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c9dd28080a631be25c77c36c63b17a3ef340ffb2e21c87d4d7b56b6398436d9 2024-08-19T01:45:46.333 INFO:journalctl@ceph.osd.10.smithi117.stdout:Aug 19 01:45:46 smithi117 podman[56537]: 2024-08-19 01:45:46.064032071 +0000 UTC m=+0.121170685 container create bc819740f171964cce7f3a58c3fb5b86dffb0805e6370831e8731640f386d170 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c9dd28080a631be25c77c36c63b17a3ef340ffb2e21c87d4d7b56b6398436d9, name=ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-10-deactivate, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, io.buildah.version=1.36.0, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , RELEASE=squid-de09dd0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True) 2024-08-19T01:45:46.333 INFO:journalctl@ceph.osd.10.smithi117.stdout:Aug 19 01:45:46 smithi117 podman[56537]: 2024-08-19 01:45:46.25775568 +0000 UTC m=+0.314894250 container init bc819740f171964cce7f3a58c3fb5b86dffb0805e6370831e8731640f386d170 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c9dd28080a631be25c77c36c63b17a3ef340ffb2e21c87d4d7b56b6398436d9, name=ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-10-deactivate, GIT_BRANCH=HEAD, RELEASE=squid-de09dd0, io.buildah.version=1.36.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, ceph=True) 2024-08-19T01:45:46.334 INFO:journalctl@ceph.osd.10.smithi117.stdout:Aug 19 01:45:46 smithi117 podman[56537]: 2024-08-19 01:45:46.260937695 +0000 UTC m=+0.318076265 container start bc819740f171964cce7f3a58c3fb5b86dffb0805e6370831e8731640f386d170 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c9dd28080a631be25c77c36c63b17a3ef340ffb2e21c87d4d7b56b6398436d9, name=ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-10-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, io.buildah.version=1.36.0, org.label-schema.vendor=CentOS, RELEASE=squid-de09dd0, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.build-date=20240731, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image) 2024-08-19T01:45:46.334 INFO:journalctl@ceph.osd.10.smithi117.stdout:Aug 19 01:45:46 smithi117 podman[56537]: 2024-08-19 01:45:46.332732279 +0000 UTC m=+0.389870859 container attach bc819740f171964cce7f3a58c3fb5b86dffb0805e6370831e8731640f386d170 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c9dd28080a631be25c77c36c63b17a3ef340ffb2e21c87d4d7b56b6398436d9, name=ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-10-deactivate, org.label-schema.build-date=20240731, io.buildah.version=1.36.0, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, ceph=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , RELEASE=squid-de09dd0) 2024-08-19T01:45:46.702 INFO:journalctl@ceph.osd.10.smithi117.stdout:Aug 19 01:45:46 smithi117 podman[56537]: 2024-08-19 01:45:46.454232723 +0000 UTC m=+0.511371303 container died bc819740f171964cce7f3a58c3fb5b86dffb0805e6370831e8731640f386d170 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c9dd28080a631be25c77c36c63b17a3ef340ffb2e21c87d4d7b56b6398436d9, name=ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-10-deactivate, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, ceph=True, org.label-schema.license=GPLv2, RELEASE=squid-de09dd0, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, io.buildah.version=1.36.0, org.label-schema.name=CentOS Stream 9 Base Image) 2024-08-19T01:45:46.966 INFO:journalctl@ceph.osd.11.smithi167.stdout:Aug 19 01:45:46 smithi167 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-11[46643]: 2024-08-19T01:45:46.594+0000 7f447f624640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-08-19T01:44:22.880210+0000 front 2024-08-19T01:44:22.880404+0000 (oldest deadline 2024-08-19T01:44:45.780352+0000) 2024-08-19T01:45:46.966 INFO:journalctl@ceph.osd.11.smithi167.stdout:Aug 19 01:45:46 smithi167 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-11[46643]: 2024-08-19T01:45:46.594+0000 7f447f624640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-08-19T01:44:31.081546+0000 front 2024-08-19T01:44:31.081654+0000 (oldest deadline 2024-08-19T01:44:53.981595+0000) 2024-08-19T01:45:46.966 INFO:journalctl@ceph.osd.11.smithi167.stdout:Aug 19 01:45:46 smithi167 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-11[46643]: 2024-08-19T01:45:46.594+0000 7f447f624640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-08-19T01:44:39.282638+0000 front 2024-08-19T01:44:39.282723+0000 (oldest deadline 2024-08-19T01:45:04.582709+0000) 2024-08-19T01:45:46.966 INFO:journalctl@ceph.osd.11.smithi167.stdout:Aug 19 01:45:46 smithi167 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-11[46643]: 2024-08-19T01:45:46.594+0000 7f447f624640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.83:6814 osd.3 since back 2024-08-19T01:44:45.083762+0000 front 2024-08-19T01:44:45.083777+0000 (oldest deadline 2024-08-19T01:45:10.383738+0000) 2024-08-19T01:45:46.966 INFO:journalctl@ceph.osd.11.smithi167.stdout:Aug 19 01:45:46 smithi167 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-11[46643]: 2024-08-19T01:45:46.594+0000 7f447f624640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.117:6812 osd.4 since back 2024-08-19T01:44:55.585644+0000 front 2024-08-19T01:44:55.585746+0000 (oldest deadline 2024-08-19T01:45:20.885631+0000) 2024-08-19T01:45:46.966 INFO:journalctl@ceph.osd.11.smithi167.stdout:Aug 19 01:45:46 smithi167 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-11[46643]: 2024-08-19T01:45:46.594+0000 7f447f624640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.167:6812 osd.5 since back 2024-08-19T01:45:02.586329+0000 front 2024-08-19T01:45:02.586392+0000 (oldest deadline 2024-08-19T01:45:28.486376+0000) 2024-08-19T01:45:46.967 INFO:journalctl@ceph.osd.11.smithi167.stdout:Aug 19 01:45:46 smithi167 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-11[46643]: 2024-08-19T01:45:46.594+0000 7f447f624640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.117:6820 osd.7 since back 2024-08-19T01:45:20.188393+0000 front 2024-08-19T01:45:20.188432+0000 (oldest deadline 2024-08-19T01:45:42.488373+0000) 2024-08-19T01:45:46.974 DEBUG:teuthology.orchestra.run.smithi117:> sudo pkill -f 'journalctl -f -n 0 -u ceph-bad8780e-5dc9-11ef-bccf-c7b262605968@osd.10.service' 2024-08-19T01:45:46.997 INFO:journalctl@ceph.osd.10.smithi117.stdout:Aug 19 01:45:46 smithi117 podman[56560]: 2024-08-19 01:45:46.944209417 +0000 UTC m=+0.484263681 container remove bc819740f171964cce7f3a58c3fb5b86dffb0805e6370831e8731640f386d170 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c9dd28080a631be25c77c36c63b17a3ef340ffb2e21c87d4d7b56b6398436d9, name=ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-10-deactivate, ceph=True, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , RELEASE=squid-de09dd0, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.36.0, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS) 2024-08-19T01:45:46.997 INFO:journalctl@ceph.osd.10.smithi117.stdout:Aug 19 01:45:46 smithi117 systemd[1]: ceph-bad8780e-5dc9-11ef-bccf-c7b262605968@osd.10.service: Deactivated successfully. 2024-08-19T01:45:46.997 INFO:journalctl@ceph.osd.10.smithi117.stdout:Aug 19 01:45:46 smithi117 systemd[1]: Stopped Ceph osd.10 for bad8780e-5dc9-11ef-bccf-c7b262605968. 2024-08-19T01:45:46.997 INFO:journalctl@ceph.osd.10.smithi117.stdout:Aug 19 01:45:46 smithi117 systemd[1]: ceph-bad8780e-5dc9-11ef-bccf-c7b262605968@osd.10.service: Consumed 1min 26.682s CPU time. 2024-08-19T01:45:47.012 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-08-19T01:45:47.012 INFO:tasks.cephadm.osd.10:Stopped osd.10 2024-08-19T01:45:47.012 INFO:tasks.cephadm.osd.11:Stopping osd.11... 2024-08-19T01:45:47.012 DEBUG:teuthology.orchestra.run.smithi167:> sudo systemctl stop ceph-bad8780e-5dc9-11ef-bccf-c7b262605968@osd.11 2024-08-19T01:45:47.310 INFO:journalctl@ceph.osd.11.smithi167.stdout:Aug 19 01:45:47 smithi167 systemd[1]: Stopping Ceph osd.11 for bad8780e-5dc9-11ef-bccf-c7b262605968... 2024-08-19T01:45:47.600 INFO:journalctl@ceph.osd.11.smithi167.stdout:Aug 19 01:45:47 smithi167 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-11[46643]: 2024-08-19T01:45:47.308+0000 7f4486826640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.11 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-08-19T01:45:47.600 INFO:journalctl@ceph.osd.11.smithi167.stdout:Aug 19 01:45:47 smithi167 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-11[46643]: 2024-08-19T01:45:47.308+0000 7f4486826640 -1 osd.11 89 *** Got signal Terminated *** 2024-08-19T01:45:47.601 INFO:journalctl@ceph.osd.11.smithi167.stdout:Aug 19 01:45:47 smithi167 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-11[46643]: 2024-08-19T01:45:47.308+0000 7f4486826640 -1 osd.11 89 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-08-19T01:45:47.966 INFO:journalctl@ceph.osd.11.smithi167.stdout:Aug 19 01:45:47 smithi167 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-11[46643]: 2024-08-19T01:45:47.599+0000 7f447f624640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-08-19T01:44:22.880210+0000 front 2024-08-19T01:44:22.880404+0000 (oldest deadline 2024-08-19T01:44:45.780352+0000) 2024-08-19T01:45:47.966 INFO:journalctl@ceph.osd.11.smithi167.stdout:Aug 19 01:45:47 smithi167 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-11[46643]: 2024-08-19T01:45:47.599+0000 7f447f624640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-08-19T01:44:31.081546+0000 front 2024-08-19T01:44:31.081654+0000 (oldest deadline 2024-08-19T01:44:53.981595+0000) 2024-08-19T01:45:47.966 INFO:journalctl@ceph.osd.11.smithi167.stdout:Aug 19 01:45:47 smithi167 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-11[46643]: 2024-08-19T01:45:47.599+0000 7f447f624640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-08-19T01:44:39.282638+0000 front 2024-08-19T01:44:39.282723+0000 (oldest deadline 2024-08-19T01:45:04.582709+0000) 2024-08-19T01:45:47.966 INFO:journalctl@ceph.osd.11.smithi167.stdout:Aug 19 01:45:47 smithi167 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-11[46643]: 2024-08-19T01:45:47.599+0000 7f447f624640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.83:6814 osd.3 since back 2024-08-19T01:44:45.083762+0000 front 2024-08-19T01:44:45.083777+0000 (oldest deadline 2024-08-19T01:45:10.383738+0000) 2024-08-19T01:45:47.966 INFO:journalctl@ceph.osd.11.smithi167.stdout:Aug 19 01:45:47 smithi167 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-11[46643]: 2024-08-19T01:45:47.599+0000 7f447f624640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.117:6812 osd.4 since back 2024-08-19T01:44:55.585644+0000 front 2024-08-19T01:44:55.585746+0000 (oldest deadline 2024-08-19T01:45:20.885631+0000) 2024-08-19T01:45:47.966 INFO:journalctl@ceph.osd.11.smithi167.stdout:Aug 19 01:45:47 smithi167 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-11[46643]: 2024-08-19T01:45:47.599+0000 7f447f624640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.167:6812 osd.5 since back 2024-08-19T01:45:02.586329+0000 front 2024-08-19T01:45:02.586392+0000 (oldest deadline 2024-08-19T01:45:28.486376+0000) 2024-08-19T01:45:47.966 INFO:journalctl@ceph.osd.11.smithi167.stdout:Aug 19 01:45:47 smithi167 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-11[46643]: 2024-08-19T01:45:47.599+0000 7f447f624640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.117:6820 osd.7 since back 2024-08-19T01:45:20.188393+0000 front 2024-08-19T01:45:20.188432+0000 (oldest deadline 2024-08-19T01:45:42.488373+0000) 2024-08-19T01:45:48.966 INFO:journalctl@ceph.osd.11.smithi167.stdout:Aug 19 01:45:48 smithi167 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-11[46643]: 2024-08-19T01:45:48.579+0000 7f447f624640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-08-19T01:44:22.880210+0000 front 2024-08-19T01:44:22.880404+0000 (oldest deadline 2024-08-19T01:44:45.780352+0000) 2024-08-19T01:45:48.966 INFO:journalctl@ceph.osd.11.smithi167.stdout:Aug 19 01:45:48 smithi167 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-11[46643]: 2024-08-19T01:45:48.579+0000 7f447f624640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-08-19T01:44:31.081546+0000 front 2024-08-19T01:44:31.081654+0000 (oldest deadline 2024-08-19T01:44:53.981595+0000) 2024-08-19T01:45:48.966 INFO:journalctl@ceph.osd.11.smithi167.stdout:Aug 19 01:45:48 smithi167 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-11[46643]: 2024-08-19T01:45:48.579+0000 7f447f624640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-08-19T01:44:39.282638+0000 front 2024-08-19T01:44:39.282723+0000 (oldest deadline 2024-08-19T01:45:04.582709+0000) 2024-08-19T01:45:48.967 INFO:journalctl@ceph.osd.11.smithi167.stdout:Aug 19 01:45:48 smithi167 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-11[46643]: 2024-08-19T01:45:48.579+0000 7f447f624640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.83:6814 osd.3 since back 2024-08-19T01:44:45.083762+0000 front 2024-08-19T01:44:45.083777+0000 (oldest deadline 2024-08-19T01:45:10.383738+0000) 2024-08-19T01:45:48.967 INFO:journalctl@ceph.osd.11.smithi167.stdout:Aug 19 01:45:48 smithi167 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-11[46643]: 2024-08-19T01:45:48.579+0000 7f447f624640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.117:6812 osd.4 since back 2024-08-19T01:44:55.585644+0000 front 2024-08-19T01:44:55.585746+0000 (oldest deadline 2024-08-19T01:45:20.885631+0000) 2024-08-19T01:45:48.967 INFO:journalctl@ceph.osd.11.smithi167.stdout:Aug 19 01:45:48 smithi167 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-11[46643]: 2024-08-19T01:45:48.579+0000 7f447f624640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.167:6812 osd.5 since back 2024-08-19T01:45:02.586329+0000 front 2024-08-19T01:45:02.586392+0000 (oldest deadline 2024-08-19T01:45:28.486376+0000) 2024-08-19T01:45:48.967 INFO:journalctl@ceph.osd.11.smithi167.stdout:Aug 19 01:45:48 smithi167 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-11[46643]: 2024-08-19T01:45:48.579+0000 7f447f624640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.117:6820 osd.7 since back 2024-08-19T01:45:20.188393+0000 front 2024-08-19T01:45:20.188432+0000 (oldest deadline 2024-08-19T01:45:42.488373+0000) 2024-08-19T01:45:49.886 INFO:journalctl@ceph.osd.11.smithi167.stdout:Aug 19 01:45:49 smithi167 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-11[46643]: 2024-08-19T01:45:49.619+0000 7f447f624640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-08-19T01:44:22.880210+0000 front 2024-08-19T01:44:22.880404+0000 (oldest deadline 2024-08-19T01:44:45.780352+0000) 2024-08-19T01:45:49.886 INFO:journalctl@ceph.osd.11.smithi167.stdout:Aug 19 01:45:49 smithi167 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-11[46643]: 2024-08-19T01:45:49.619+0000 7f447f624640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-08-19T01:44:31.081546+0000 front 2024-08-19T01:44:31.081654+0000 (oldest deadline 2024-08-19T01:44:53.981595+0000) 2024-08-19T01:45:49.886 INFO:journalctl@ceph.osd.11.smithi167.stdout:Aug 19 01:45:49 smithi167 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-11[46643]: 2024-08-19T01:45:49.619+0000 7f447f624640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-08-19T01:44:39.282638+0000 front 2024-08-19T01:44:39.282723+0000 (oldest deadline 2024-08-19T01:45:04.582709+0000) 2024-08-19T01:45:49.886 INFO:journalctl@ceph.osd.11.smithi167.stdout:Aug 19 01:45:49 smithi167 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-11[46643]: 2024-08-19T01:45:49.619+0000 7f447f624640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.83:6814 osd.3 since back 2024-08-19T01:44:45.083762+0000 front 2024-08-19T01:44:45.083777+0000 (oldest deadline 2024-08-19T01:45:10.383738+0000) 2024-08-19T01:45:49.887 INFO:journalctl@ceph.osd.11.smithi167.stdout:Aug 19 01:45:49 smithi167 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-11[46643]: 2024-08-19T01:45:49.619+0000 7f447f624640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.117:6812 osd.4 since back 2024-08-19T01:44:55.585644+0000 front 2024-08-19T01:44:55.585746+0000 (oldest deadline 2024-08-19T01:45:20.885631+0000) 2024-08-19T01:45:49.887 INFO:journalctl@ceph.osd.11.smithi167.stdout:Aug 19 01:45:49 smithi167 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-11[46643]: 2024-08-19T01:45:49.619+0000 7f447f624640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.167:6812 osd.5 since back 2024-08-19T01:45:02.586329+0000 front 2024-08-19T01:45:02.586392+0000 (oldest deadline 2024-08-19T01:45:28.486376+0000) 2024-08-19T01:45:49.887 INFO:journalctl@ceph.osd.11.smithi167.stdout:Aug 19 01:45:49 smithi167 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-11[46643]: 2024-08-19T01:45:49.619+0000 7f447f624640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.117:6820 osd.7 since back 2024-08-19T01:45:20.188393+0000 front 2024-08-19T01:45:20.188432+0000 (oldest deadline 2024-08-19T01:45:42.488373+0000) 2024-08-19T01:45:49.887 INFO:journalctl@ceph.osd.11.smithi167.stdout:Aug 19 01:45:49 smithi167 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-11[46643]: 2024-08-19T01:45:49.619+0000 7f447f624640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.167:6820 osd.8 since back 2024-08-19T01:45:28.389156+0000 front 2024-08-19T01:45:28.389138+0000 (oldest deadline 2024-08-19T01:45:49.489168+0000) 2024-08-19T01:45:50.966 INFO:journalctl@ceph.osd.11.smithi167.stdout:Aug 19 01:45:50 smithi167 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-11[46643]: 2024-08-19T01:45:50.663+0000 7f447f624640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-08-19T01:44:22.880210+0000 front 2024-08-19T01:44:22.880404+0000 (oldest deadline 2024-08-19T01:44:45.780352+0000) 2024-08-19T01:45:50.966 INFO:journalctl@ceph.osd.11.smithi167.stdout:Aug 19 01:45:50 smithi167 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-11[46643]: 2024-08-19T01:45:50.663+0000 7f447f624640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-08-19T01:44:31.081546+0000 front 2024-08-19T01:44:31.081654+0000 (oldest deadline 2024-08-19T01:44:53.981595+0000) 2024-08-19T01:45:50.966 INFO:journalctl@ceph.osd.11.smithi167.stdout:Aug 19 01:45:50 smithi167 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-11[46643]: 2024-08-19T01:45:50.663+0000 7f447f624640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-08-19T01:44:39.282638+0000 front 2024-08-19T01:44:39.282723+0000 (oldest deadline 2024-08-19T01:45:04.582709+0000) 2024-08-19T01:45:50.966 INFO:journalctl@ceph.osd.11.smithi167.stdout:Aug 19 01:45:50 smithi167 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-11[46643]: 2024-08-19T01:45:50.663+0000 7f447f624640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.83:6814 osd.3 since back 2024-08-19T01:44:45.083762+0000 front 2024-08-19T01:44:45.083777+0000 (oldest deadline 2024-08-19T01:45:10.383738+0000) 2024-08-19T01:45:50.966 INFO:journalctl@ceph.osd.11.smithi167.stdout:Aug 19 01:45:50 smithi167 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-11[46643]: 2024-08-19T01:45:50.663+0000 7f447f624640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.117:6812 osd.4 since back 2024-08-19T01:44:55.585644+0000 front 2024-08-19T01:44:55.585746+0000 (oldest deadline 2024-08-19T01:45:20.885631+0000) 2024-08-19T01:45:50.966 INFO:journalctl@ceph.osd.11.smithi167.stdout:Aug 19 01:45:50 smithi167 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-11[46643]: 2024-08-19T01:45:50.663+0000 7f447f624640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.167:6812 osd.5 since back 2024-08-19T01:45:02.586329+0000 front 2024-08-19T01:45:02.586392+0000 (oldest deadline 2024-08-19T01:45:28.486376+0000) 2024-08-19T01:45:50.966 INFO:journalctl@ceph.osd.11.smithi167.stdout:Aug 19 01:45:50 smithi167 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-11[46643]: 2024-08-19T01:45:50.663+0000 7f447f624640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.117:6820 osd.7 since back 2024-08-19T01:45:20.188393+0000 front 2024-08-19T01:45:20.188432+0000 (oldest deadline 2024-08-19T01:45:42.488373+0000) 2024-08-19T01:45:50.966 INFO:journalctl@ceph.osd.11.smithi167.stdout:Aug 19 01:45:50 smithi167 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-11[46643]: 2024-08-19T01:45:50.663+0000 7f447f624640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.167:6820 osd.8 since back 2024-08-19T01:45:28.389156+0000 front 2024-08-19T01:45:28.389138+0000 (oldest deadline 2024-08-19T01:45:49.489168+0000) 2024-08-19T01:45:51.966 INFO:journalctl@ceph.osd.11.smithi167.stdout:Aug 19 01:45:51 smithi167 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-11[46643]: 2024-08-19T01:45:51.680+0000 7f447f624640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-08-19T01:44:22.880210+0000 front 2024-08-19T01:44:22.880404+0000 (oldest deadline 2024-08-19T01:44:45.780352+0000) 2024-08-19T01:45:51.966 INFO:journalctl@ceph.osd.11.smithi167.stdout:Aug 19 01:45:51 smithi167 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-11[46643]: 2024-08-19T01:45:51.680+0000 7f447f624640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-08-19T01:44:31.081546+0000 front 2024-08-19T01:44:31.081654+0000 (oldest deadline 2024-08-19T01:44:53.981595+0000) 2024-08-19T01:45:51.966 INFO:journalctl@ceph.osd.11.smithi167.stdout:Aug 19 01:45:51 smithi167 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-11[46643]: 2024-08-19T01:45:51.680+0000 7f447f624640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-08-19T01:44:39.282638+0000 front 2024-08-19T01:44:39.282723+0000 (oldest deadline 2024-08-19T01:45:04.582709+0000) 2024-08-19T01:45:51.966 INFO:journalctl@ceph.osd.11.smithi167.stdout:Aug 19 01:45:51 smithi167 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-11[46643]: 2024-08-19T01:45:51.680+0000 7f447f624640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.83:6814 osd.3 since back 2024-08-19T01:44:45.083762+0000 front 2024-08-19T01:44:45.083777+0000 (oldest deadline 2024-08-19T01:45:10.383738+0000) 2024-08-19T01:45:51.967 INFO:journalctl@ceph.osd.11.smithi167.stdout:Aug 19 01:45:51 smithi167 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-11[46643]: 2024-08-19T01:45:51.680+0000 7f447f624640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.117:6812 osd.4 since back 2024-08-19T01:44:55.585644+0000 front 2024-08-19T01:44:55.585746+0000 (oldest deadline 2024-08-19T01:45:20.885631+0000) 2024-08-19T01:45:51.967 INFO:journalctl@ceph.osd.11.smithi167.stdout:Aug 19 01:45:51 smithi167 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-11[46643]: 2024-08-19T01:45:51.680+0000 7f447f624640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.167:6812 osd.5 since back 2024-08-19T01:45:02.586329+0000 front 2024-08-19T01:45:02.586392+0000 (oldest deadline 2024-08-19T01:45:28.486376+0000) 2024-08-19T01:45:51.967 INFO:journalctl@ceph.osd.11.smithi167.stdout:Aug 19 01:45:51 smithi167 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-11[46643]: 2024-08-19T01:45:51.680+0000 7f447f624640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.117:6820 osd.7 since back 2024-08-19T01:45:20.188393+0000 front 2024-08-19T01:45:20.188432+0000 (oldest deadline 2024-08-19T01:45:42.488373+0000) 2024-08-19T01:45:51.967 INFO:journalctl@ceph.osd.11.smithi167.stdout:Aug 19 01:45:51 smithi167 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-11[46643]: 2024-08-19T01:45:51.680+0000 7f447f624640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.167:6820 osd.8 since back 2024-08-19T01:45:28.389156+0000 front 2024-08-19T01:45:28.389138+0000 (oldest deadline 2024-08-19T01:45:49.489168+0000) 2024-08-19T01:45:52.927 INFO:journalctl@ceph.osd.11.smithi167.stdout:Aug 19 01:45:52 smithi167 podman[57228]: 2024-08-19 01:45:52.642745608 +0000 UTC m=+5.432292653 container died a6866e0c738db7be8c141a7527f56d95d78eb237793ce0bef1101c70e9b58892 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c9dd28080a631be25c77c36c63b17a3ef340ffb2e21c87d4d7b56b6398436d9, name=ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-11, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, io.buildah.version=1.36.0, org.label-schema.build-date=20240731, ceph=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=squid-de09dd0, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_CLEAN=True) 2024-08-19T01:45:53.216 INFO:journalctl@ceph.osd.11.smithi167.stdout:Aug 19 01:45:53 smithi167 podman[57228]: 2024-08-19 01:45:53.000807488 +0000 UTC m=+5.790354539 container cleanup a6866e0c738db7be8c141a7527f56d95d78eb237793ce0bef1101c70e9b58892 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c9dd28080a631be25c77c36c63b17a3ef340ffb2e21c87d4d7b56b6398436d9, name=ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-11, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , io.buildah.version=1.36.0, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, RELEASE=squid-de09dd0, org.label-schema.license=GPLv2, GIT_CLEAN=True) 2024-08-19T01:45:53.216 INFO:journalctl@ceph.osd.11.smithi167.stdout:Aug 19 01:45:53 smithi167 bash[57228]: ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-11 2024-08-19T01:45:53.628 INFO:journalctl@ceph.osd.11.smithi167.stdout:Aug 19 01:45:53 smithi167 podman[57258]: 2024-08-19 01:45:53.29374185 +0000 UTC m=+0.641311642 container remove a6866e0c738db7be8c141a7527f56d95d78eb237793ce0bef1101c70e9b58892 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c9dd28080a631be25c77c36c63b17a3ef340ffb2e21c87d4d7b56b6398436d9, name=ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-11, org.label-schema.build-date=20240731, io.buildah.version=1.36.0, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, RELEASE=squid-de09dd0, ceph=True, CEPH_POINT_RELEASE=) 2024-08-19T01:45:53.914 INFO:journalctl@ceph.osd.11.smithi167.stdout:Aug 19 01:45:53 smithi167 podman[57347]: 2024-08-19 01:45:53.627781772 +0000 UTC m=+0.108223522 container create 04eb8d6d43bca79cfdcb68d2a3bd61763e5e732b180e63ea6f21dd87537c0af2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c9dd28080a631be25c77c36c63b17a3ef340ffb2e21c87d4d7b56b6398436d9, name=ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-11-deactivate, RELEASE=squid-de09dd0, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.vendor=CentOS, io.buildah.version=1.36.0, ceph=True, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image) 2024-08-19T01:45:53.914 INFO:journalctl@ceph.osd.11.smithi167.stdout:Aug 19 01:45:53 smithi167 podman[57347]: 2024-08-19 01:45:53.53710375 +0000 UTC m=+0.017545499 image pull ea7a5189f30bad3aca9524788e2965d2368d7ce786df09d4edcd8c3edf9cefba quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c9dd28080a631be25c77c36c63b17a3ef340ffb2e21c87d4d7b56b6398436d9 2024-08-19T01:45:53.914 INFO:journalctl@ceph.osd.11.smithi167.stdout:Aug 19 01:45:53 smithi167 podman[57347]: 2024-08-19 01:45:53.830589269 +0000 UTC m=+0.311031030 container init 04eb8d6d43bca79cfdcb68d2a3bd61763e5e732b180e63ea6f21dd87537c0af2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c9dd28080a631be25c77c36c63b17a3ef340ffb2e21c87d4d7b56b6398436d9, name=ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-11-deactivate, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, io.buildah.version=1.36.0, GIT_BRANCH=HEAD, GIT_CLEAN=True, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, RELEASE=squid-de09dd0) 2024-08-19T01:45:53.914 INFO:journalctl@ceph.osd.11.smithi167.stdout:Aug 19 01:45:53 smithi167 podman[57347]: 2024-08-19 01:45:53.833828109 +0000 UTC m=+0.314269867 container start 04eb8d6d43bca79cfdcb68d2a3bd61763e5e732b180e63ea6f21dd87537c0af2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c9dd28080a631be25c77c36c63b17a3ef340ffb2e21c87d4d7b56b6398436d9, name=ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-11-deactivate, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , io.buildah.version=1.36.0, RELEASE=squid-de09dd0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.build-date=20240731) 2024-08-19T01:45:53.914 INFO:journalctl@ceph.osd.11.smithi167.stdout:Aug 19 01:45:53 smithi167 podman[57347]: 2024-08-19 01:45:53.913720352 +0000 UTC m=+0.394162101 container attach 04eb8d6d43bca79cfdcb68d2a3bd61763e5e732b180e63ea6f21dd87537c0af2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c9dd28080a631be25c77c36c63b17a3ef340ffb2e21c87d4d7b56b6398436d9, name=ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-11-deactivate, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, io.buildah.version=1.36.0, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, RELEASE=squid-de09dd0, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2) 2024-08-19T01:45:54.216 INFO:journalctl@ceph.osd.11.smithi167.stdout:Aug 19 01:45:54 smithi167 podman[57347]: 2024-08-19 01:45:54.020201146 +0000 UTC m=+0.500642903 container died 04eb8d6d43bca79cfdcb68d2a3bd61763e5e732b180e63ea6f21dd87537c0af2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c9dd28080a631be25c77c36c63b17a3ef340ffb2e21c87d4d7b56b6398436d9, name=ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-osd-11-deactivate, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-de09dd0, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, io.buildah.version=1.36.0, CEPH_POINT_RELEASE=, GIT_CLEAN=True) 2024-08-19T01:45:54.586 DEBUG:teuthology.orchestra.run.smithi167:> sudo pkill -f 'journalctl -f -n 0 -u ceph-bad8780e-5dc9-11ef-bccf-c7b262605968@osd.11.service' 2024-08-19T01:45:54.625 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-08-19T01:45:54.625 INFO:tasks.cephadm.osd.11:Stopped osd.11 2024-08-19T01:45:54.625 INFO:tasks.cephadm.mds.a:Stopping mds.a... 2024-08-19T01:45:54.625 DEBUG:teuthology.orchestra.run.smithi083:> sudo systemctl stop ceph-bad8780e-5dc9-11ef-bccf-c7b262605968@mds.a 2024-08-19T01:45:54.972 INFO:journalctl@ceph.mds.a.smithi083.stdout:Aug 19 01:45:54 smithi083 systemd[1]: Stopping Ceph mds.a for bad8780e-5dc9-11ef-bccf-c7b262605968... 2024-08-19T01:45:55.406 INFO:journalctl@ceph.mds.a.smithi083.stdout:Aug 19 01:45:55 smithi083 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-mds-a[58063]: 2024-08-19T01:45:55.046+0000 7fdb13000640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-08-19T01:45:55.406 INFO:journalctl@ceph.mds.a.smithi083.stdout:Aug 19 01:45:55 smithi083 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-mds-a[58063]: 2024-08-19T01:45:55.046+0000 7fdb13000640 -1 mds.a *** got signal Terminated *** 2024-08-19T01:45:59.380 INFO:journalctl@ceph.mds.a.smithi083.stdout:Aug 19 01:45:59 smithi083 podman[85217]: 2024-08-19 01:45:59.058215133 +0000 UTC m=+4.139939268 container died 16051d2af2ea3f9f2d320e2248da4cf2fb41acbbdcef5355fc8a263984ad9120 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c9dd28080a631be25c77c36c63b17a3ef340ffb2e21c87d4d7b56b6398436d9, name=ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-mds-a, GIT_BRANCH=HEAD, ceph=True, org.label-schema.schema-version=1.0, GIT_CLEAN=True, RELEASE=squid-de09dd0, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.36.0, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=) 2024-08-19T01:45:59.654 INFO:journalctl@ceph.mds.a.smithi083.stdout:Aug 19 01:45:59 smithi083 podman[85217]: 2024-08-19 01:45:59.56962051 +0000 UTC m=+4.651344641 container cleanup 16051d2af2ea3f9f2d320e2248da4cf2fb41acbbdcef5355fc8a263984ad9120 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c9dd28080a631be25c77c36c63b17a3ef340ffb2e21c87d4d7b56b6398436d9, name=ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-mds-a, org.label-schema.vendor=CentOS, RELEASE=squid-de09dd0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, io.buildah.version=1.36.0, GIT_CLEAN=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-08-19T01:45:59.654 INFO:journalctl@ceph.mds.a.smithi083.stdout:Aug 19 01:45:59 smithi083 bash[85217]: ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-mds-a 2024-08-19T01:46:00.051 INFO:journalctl@ceph.mds.a.smithi083.stdout:Aug 19 01:45:59 smithi083 podman[85240]: 2024-08-19 01:45:59.863454271 +0000 UTC m=+0.790416187 container remove 16051d2af2ea3f9f2d320e2248da4cf2fb41acbbdcef5355fc8a263984ad9120 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c9dd28080a631be25c77c36c63b17a3ef340ffb2e21c87d4d7b56b6398436d9, name=ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-mds-a, GIT_CLEAN=True, maintainer=Guillaume Abrioux , RELEASE=squid-de09dd0, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.36.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0) 2024-08-19T01:46:00.082 DEBUG:teuthology.orchestra.run.smithi083:> sudo pkill -f 'journalctl -f -n 0 -u ceph-bad8780e-5dc9-11ef-bccf-c7b262605968@mds.a.service' 2024-08-19T01:46:00.154 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-08-19T01:46:00.155 INFO:tasks.cephadm.mds.a:Stopped mds.a 2024-08-19T01:46:00.155 INFO:tasks.cephadm.mds.d:Stopping mds.d... 2024-08-19T01:46:00.155 DEBUG:teuthology.orchestra.run.smithi083:> sudo systemctl stop ceph-bad8780e-5dc9-11ef-bccf-c7b262605968@mds.d 2024-08-19T01:46:00.404 INFO:journalctl@ceph.mds.d.smithi083.stdout:Aug 19 01:46:00 smithi083 systemd[1]: Stopping Ceph mds.d for bad8780e-5dc9-11ef-bccf-c7b262605968... 2024-08-19T01:46:00.904 INFO:journalctl@ceph.mds.d.smithi083.stdout:Aug 19 01:46:00 smithi083 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-mds-d[56883]: 2024-08-19T01:46:00.593+0000 7fef86600640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.d -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-08-19T01:46:00.905 INFO:journalctl@ceph.mds.d.smithi083.stdout:Aug 19 01:46:00 smithi083 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-mds-d[56883]: 2024-08-19T01:46:00.593+0000 7fef86600640 -1 mds.d *** got signal Terminated *** 2024-08-19T01:46:03.026 INFO:journalctl@ceph.mds.d.smithi083.stdout:Aug 19 01:46:02 smithi083 podman[85350]: 2024-08-19 01:46:02.734338391 +0000 UTC m=+2.293349557 container died c3e18ff80221dee8b0363077499a81eda105420ec5817270279d2f57e89f0f5c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c9dd28080a631be25c77c36c63b17a3ef340ffb2e21c87d4d7b56b6398436d9, name=ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-mds-d, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.license=GPLv2, RELEASE=squid-de09dd0, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, io.buildah.version=1.36.0, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-08-19T01:46:03.404 INFO:journalctl@ceph.mds.d.smithi083.stdout:Aug 19 01:46:03 smithi083 podman[85350]: 2024-08-19 01:46:03.207724811 +0000 UTC m=+2.766735975 container cleanup c3e18ff80221dee8b0363077499a81eda105420ec5817270279d2f57e89f0f5c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c9dd28080a631be25c77c36c63b17a3ef340ffb2e21c87d4d7b56b6398436d9, name=ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-mds-d, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, ceph=True, io.buildah.version=1.36.0, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, RELEASE=squid-de09dd0) 2024-08-19T01:46:03.404 INFO:journalctl@ceph.mds.d.smithi083.stdout:Aug 19 01:46:03 smithi083 bash[85350]: ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-mds-d 2024-08-19T01:46:03.769 DEBUG:teuthology.orchestra.run.smithi083:> sudo pkill -f 'journalctl -f -n 0 -u ceph-bad8780e-5dc9-11ef-bccf-c7b262605968@mds.d.service' 2024-08-19T01:46:03.817 INFO:journalctl@ceph.mds.d.smithi083.stdout:Aug 19 01:46:03 smithi083 podman[85364]: 2024-08-19 01:46:03.550764288 +0000 UTC m=+0.800254641 container remove c3e18ff80221dee8b0363077499a81eda105420ec5817270279d2f57e89f0f5c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c9dd28080a631be25c77c36c63b17a3ef340ffb2e21c87d4d7b56b6398436d9, name=ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-mds-d, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, RELEASE=squid-de09dd0, io.buildah.version=1.36.0, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-08-19T01:46:03.817 INFO:journalctl@ceph.mds.d.smithi083.stdout:Aug 19 01:46:03 smithi083 systemd[1]: ceph-bad8780e-5dc9-11ef-bccf-c7b262605968@mds.d.service: Deactivated successfully. 2024-08-19T01:46:03.817 INFO:journalctl@ceph.mds.d.smithi083.stdout:Aug 19 01:46:03 smithi083 systemd[1]: Stopped Ceph mds.d for bad8780e-5dc9-11ef-bccf-c7b262605968. 2024-08-19T01:46:03.817 INFO:journalctl@ceph.mds.d.smithi083.stdout:Aug 19 01:46:03 smithi083 systemd[1]: ceph-bad8780e-5dc9-11ef-bccf-c7b262605968@mds.d.service: Consumed 1.524s CPU time. 2024-08-19T01:46:03.839 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-08-19T01:46:03.839 INFO:tasks.cephadm.mds.d:Stopped mds.d 2024-08-19T01:46:03.839 INFO:tasks.cephadm.mds.g:Stopping mds.g... 2024-08-19T01:46:03.839 DEBUG:teuthology.orchestra.run.smithi083:> sudo systemctl stop ceph-bad8780e-5dc9-11ef-bccf-c7b262605968@mds.g 2024-08-19T01:46:04.154 INFO:journalctl@ceph.mds.g.smithi083.stdout:Aug 19 01:46:03 smithi083 systemd[1]: Stopping Ceph mds.g for bad8780e-5dc9-11ef-bccf-c7b262605968... 2024-08-19T01:46:04.654 INFO:journalctl@ceph.mds.g.smithi083.stdout:Aug 19 01:46:04 smithi083 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-mds-g[57656]: 2024-08-19T01:46:04.256+0000 7fc8a0400640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.g -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-08-19T01:46:04.654 INFO:journalctl@ceph.mds.g.smithi083.stdout:Aug 19 01:46:04 smithi083 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-mds-g[57656]: 2024-08-19T01:46:04.256+0000 7fc8a0400640 -1 mds.g *** got signal Terminated *** 2024-08-19T01:46:08.130 INFO:journalctl@ceph.mds.g.smithi083.stdout:Aug 19 01:46:07 smithi083 podman[85471]: 2024-08-19 01:46:07.860958317 +0000 UTC m=+3.733430092 container died dbdad0fb100703a56cd236f9dfcc6438dbdae6baf9d550bb9b4416a556ab714c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c9dd28080a631be25c77c36c63b17a3ef340ffb2e21c87d4d7b56b6398436d9, name=ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-mds-g, io.buildah.version=1.36.0, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.build-date=20240731, RELEASE=squid-de09dd0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2) 2024-08-19T01:46:08.404 INFO:journalctl@ceph.mds.g.smithi083.stdout:Aug 19 01:46:08 smithi083 podman[85471]: 2024-08-19 01:46:08.344782809 +0000 UTC m=+4.217254583 container cleanup dbdad0fb100703a56cd236f9dfcc6438dbdae6baf9d550bb9b4416a556ab714c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c9dd28080a631be25c77c36c63b17a3ef340ffb2e21c87d4d7b56b6398436d9, name=ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-mds-g, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, io.buildah.version=1.36.0, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, RELEASE=squid-de09dd0, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2) 2024-08-19T01:46:08.404 INFO:journalctl@ceph.mds.g.smithi083.stdout:Aug 19 01:46:08 smithi083 bash[85471]: ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-mds-g 2024-08-19T01:46:08.803 DEBUG:teuthology.orchestra.run.smithi083:> sudo pkill -f 'journalctl -f -n 0 -u ceph-bad8780e-5dc9-11ef-bccf-c7b262605968@mds.g.service' 2024-08-19T01:46:08.892 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-08-19T01:46:08.892 INFO:tasks.cephadm.mds.g:Stopped mds.g 2024-08-19T01:46:08.892 INFO:tasks.cephadm.mds.j:Stopping mds.j... 2024-08-19T01:46:08.892 DEBUG:teuthology.orchestra.run.smithi083:> sudo systemctl stop ceph-bad8780e-5dc9-11ef-bccf-c7b262605968@mds.j 2024-08-19T01:46:09.154 INFO:journalctl@ceph.mds.j.smithi083.stdout:Aug 19 01:46:08 smithi083 systemd[1]: Stopping Ceph mds.j for bad8780e-5dc9-11ef-bccf-c7b262605968... 2024-08-19T01:46:09.654 INFO:journalctl@ceph.mds.j.smithi083.stdout:Aug 19 01:46:09 smithi083 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-mds-j[56472]: 2024-08-19T01:46:09.293+0000 7f9f58e00640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.j -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-08-19T01:46:09.654 INFO:journalctl@ceph.mds.j.smithi083.stdout:Aug 19 01:46:09 smithi083 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-mds-j[56472]: 2024-08-19T01:46:09.293+0000 7f9f58e00640 -1 mds.j *** got signal Terminated *** 2024-08-19T01:46:11.052 INFO:journalctl@ceph.mds.j.smithi083.stdout:Aug 19 01:46:10 smithi083 podman[85592]: 2024-08-19 01:46:10.760327354 +0000 UTC m=+1.594317367 container died 86e4712742a36404d69bb63aebdf070527cdb6d6442eda3ddb27ae5ee651a42e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c9dd28080a631be25c77c36c63b17a3ef340ffb2e21c87d4d7b56b6398436d9, name=ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-mds-j, org.label-schema.build-date=20240731, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, io.buildah.version=1.36.0, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-de09dd0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0) 2024-08-19T01:46:11.313 INFO:journalctl@ceph.mds.j.smithi083.stdout:Aug 19 01:46:11 smithi083 podman[85592]: 2024-08-19 01:46:11.22443409 +0000 UTC m=+2.058424100 container cleanup 86e4712742a36404d69bb63aebdf070527cdb6d6442eda3ddb27ae5ee651a42e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c9dd28080a631be25c77c36c63b17a3ef340ffb2e21c87d4d7b56b6398436d9, name=ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-mds-j, io.buildah.version=1.36.0, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, GIT_CLEAN=True, RELEASE=squid-de09dd0, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0) 2024-08-19T01:46:11.313 INFO:journalctl@ceph.mds.j.smithi083.stdout:Aug 19 01:46:11 smithi083 bash[85592]: ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-mds-j 2024-08-19T01:46:11.654 INFO:journalctl@ceph.mds.j.smithi083.stdout:Aug 19 01:46:11 smithi083 podman[85604]: 2024-08-19 01:46:11.45084808 +0000 UTC m=+0.679342536 container remove 86e4712742a36404d69bb63aebdf070527cdb6d6442eda3ddb27ae5ee651a42e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c9dd28080a631be25c77c36c63b17a3ef340ffb2e21c87d4d7b56b6398436d9, name=ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-mds-j, ceph=True, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, RELEASE=squid-de09dd0, io.buildah.version=1.36.0) 2024-08-19T01:46:11.720 DEBUG:teuthology.orchestra.run.smithi083:> sudo pkill -f 'journalctl -f -n 0 -u ceph-bad8780e-5dc9-11ef-bccf-c7b262605968@mds.j.service' 2024-08-19T01:46:11.788 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-08-19T01:46:11.789 INFO:tasks.cephadm.mds.j:Stopped mds.j 2024-08-19T01:46:11.789 INFO:tasks.cephadm.mds.b:Stopping mds.b... 2024-08-19T01:46:11.789 DEBUG:teuthology.orchestra.run.smithi117:> sudo systemctl stop ceph-bad8780e-5dc9-11ef-bccf-c7b262605968@mds.b 2024-08-19T01:46:12.102 INFO:journalctl@ceph.mds.b.smithi117.stdout:Aug 19 01:46:11 smithi117 systemd[1]: Stopping Ceph mds.b for bad8780e-5dc9-11ef-bccf-c7b262605968... 2024-08-19T01:46:12.359 INFO:journalctl@ceph.mds.b.smithi117.stdout:Aug 19 01:46:12 smithi117 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-mds-b[51905]: 2024-08-19T01:46:12.099+0000 7f3e43ffa640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-08-19T01:46:12.359 INFO:journalctl@ceph.mds.b.smithi117.stdout:Aug 19 01:46:12 smithi117 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-mds-b[51905]: 2024-08-19T01:46:12.099+0000 7f3e43ffa640 -1 mds.b *** got signal Terminated *** 2024-08-19T01:46:16.127 INFO:journalctl@ceph.mds.b.smithi117.stdout:Aug 19 01:46:15 smithi117 podman[56639]: 2024-08-19 01:46:15.83072786 +0000 UTC m=+3.843021590 container died 6c1e5cbab019b0b8c63b841b6f19872a440fdb4a45b9f93dbc7371afe2a6e47c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c9dd28080a631be25c77c36c63b17a3ef340ffb2e21c87d4d7b56b6398436d9, name=ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-mds-b, RELEASE=squid-de09dd0, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.schema-version=1.0, GIT_CLEAN=True, io.buildah.version=1.36.0) 2024-08-19T01:46:16.452 INFO:journalctl@ceph.mds.b.smithi117.stdout:Aug 19 01:46:16 smithi117 podman[56639]: 2024-08-19 01:46:16.217230531 +0000 UTC m=+4.229524263 container cleanup 6c1e5cbab019b0b8c63b841b6f19872a440fdb4a45b9f93dbc7371afe2a6e47c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c9dd28080a631be25c77c36c63b17a3ef340ffb2e21c87d4d7b56b6398436d9, name=ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-mds-b, RELEASE=squid-de09dd0, ceph=True, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, io.buildah.version=1.36.0, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-08-19T01:46:16.452 INFO:journalctl@ceph.mds.b.smithi117.stdout:Aug 19 01:46:16 smithi117 bash[56639]: ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-mds-b 2024-08-19T01:46:16.605 DEBUG:teuthology.orchestra.run.smithi117:> sudo pkill -f 'journalctl -f -n 0 -u ceph-bad8780e-5dc9-11ef-bccf-c7b262605968@mds.b.service' 2024-08-19T01:46:16.641 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-08-19T01:46:16.641 INFO:tasks.cephadm.mds.b:Stopped mds.b 2024-08-19T01:46:16.641 INFO:tasks.cephadm.mds.e:Stopping mds.e... 2024-08-19T01:46:16.641 DEBUG:teuthology.orchestra.run.smithi117:> sudo systemctl stop ceph-bad8780e-5dc9-11ef-bccf-c7b262605968@mds.e 2024-08-19T01:46:16.952 INFO:journalctl@ceph.mds.e.smithi117.stdout:Aug 19 01:46:16 smithi117 systemd[1]: Stopping Ceph mds.e for bad8780e-5dc9-11ef-bccf-c7b262605968... 2024-08-19T01:46:17.452 INFO:journalctl@ceph.mds.e.smithi117.stdout:Aug 19 01:46:16 smithi117 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-mds-e[51290]: 2024-08-19T01:46:16.962+0000 7ff4f0f16640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.e -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-08-19T01:46:17.452 INFO:journalctl@ceph.mds.e.smithi117.stdout:Aug 19 01:46:16 smithi117 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-mds-e[51290]: 2024-08-19T01:46:16.962+0000 7ff4f0f16640 -1 mds.e *** got signal Terminated *** 2024-08-19T01:46:20.112 INFO:journalctl@ceph.mds.e.smithi117.stdout:Aug 19 01:46:19 smithi117 podman[56771]: 2024-08-19 01:46:19.829454825 +0000 UTC m=+2.967173551 container died 8f8b5c7000c43c452fb20951b381f665f4ee4c9c6eead8dd38648981cbc1bac0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c9dd28080a631be25c77c36c63b17a3ef340ffb2e21c87d4d7b56b6398436d9, name=ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-mds-e, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-de09dd0, io.buildah.version=1.36.0, org.label-schema.vendor=CentOS) 2024-08-19T01:46:20.113 INFO:journalctl@ceph.mds.e.smithi117.stdout:Aug 19 01:46:20 smithi117 podman[56771]: 2024-08-19 01:46:20.112213488 +0000 UTC m=+3.249932204 container cleanup 8f8b5c7000c43c452fb20951b381f665f4ee4c9c6eead8dd38648981cbc1bac0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c9dd28080a631be25c77c36c63b17a3ef340ffb2e21c87d4d7b56b6398436d9, name=ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-mds-e, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, RELEASE=squid-de09dd0, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.36.0, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-08-19T01:46:20.363 INFO:journalctl@ceph.mds.e.smithi117.stdout:Aug 19 01:46:20 smithi117 bash[56771]: ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-mds-e 2024-08-19T01:46:20.497 DEBUG:teuthology.orchestra.run.smithi117:> sudo pkill -f 'journalctl -f -n 0 -u ceph-bad8780e-5dc9-11ef-bccf-c7b262605968@mds.e.service' 2024-08-19T01:46:20.531 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-08-19T01:46:20.532 INFO:tasks.cephadm.mds.e:Stopped mds.e 2024-08-19T01:46:20.532 INFO:tasks.cephadm.mds.h:Stopping mds.h... 2024-08-19T01:46:20.532 DEBUG:teuthology.orchestra.run.smithi117:> sudo systemctl stop ceph-bad8780e-5dc9-11ef-bccf-c7b262605968@mds.h 2024-08-19T01:46:20.702 INFO:journalctl@ceph.mds.h.smithi117.stdout:Aug 19 01:46:20 smithi117 systemd[1]: Stopping Ceph mds.h for bad8780e-5dc9-11ef-bccf-c7b262605968... 2024-08-19T01:46:21.202 INFO:journalctl@ceph.mds.h.smithi117.stdout:Aug 19 01:46:20 smithi117 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-mds-h[51591]: 2024-08-19T01:46:20.856+0000 7f53b404a640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.h -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-08-19T01:46:21.202 INFO:journalctl@ceph.mds.h.smithi117.stdout:Aug 19 01:46:20 smithi117 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-mds-h[51591]: 2024-08-19T01:46:20.856+0000 7f53b404a640 -1 mds.h *** got signal Terminated *** 2024-08-19T01:46:24.110 INFO:journalctl@ceph.mds.h.smithi117.stdout:Aug 19 01:46:23 smithi117 podman[56891]: 2024-08-19 01:46:23.841215377 +0000 UTC m=+3.087648013 container died 682ad742e1a1298db016c9feaa2629176284daf32257eed1eef5c95a55c3b4ef (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c9dd28080a631be25c77c36c63b17a3ef340ffb2e21c87d4d7b56b6398436d9, name=ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-mds-h, RELEASE=squid-de09dd0, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_CLEAN=True, io.buildah.version=1.36.0, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS) 2024-08-19T01:46:24.450 INFO:journalctl@ceph.mds.h.smithi117.stdout:Aug 19 01:46:24 smithi117 podman[56891]: 2024-08-19 01:46:24.198738953 +0000 UTC m=+3.445171590 container cleanup 682ad742e1a1298db016c9feaa2629176284daf32257eed1eef5c95a55c3b4ef (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c9dd28080a631be25c77c36c63b17a3ef340ffb2e21c87d4d7b56b6398436d9, name=ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-mds-h, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, RELEASE=squid-de09dd0, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, io.buildah.version=1.36.0, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS) 2024-08-19T01:46:24.450 INFO:journalctl@ceph.mds.h.smithi117.stdout:Aug 19 01:46:24 smithi117 bash[56891]: ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-mds-h 2024-08-19T01:46:24.599 DEBUG:teuthology.orchestra.run.smithi117:> sudo pkill -f 'journalctl -f -n 0 -u ceph-bad8780e-5dc9-11ef-bccf-c7b262605968@mds.h.service' 2024-08-19T01:46:24.631 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-08-19T01:46:24.631 INFO:tasks.cephadm.mds.h:Stopped mds.h 2024-08-19T01:46:24.631 INFO:tasks.cephadm.mds.k:Stopping mds.k... 2024-08-19T01:46:24.631 DEBUG:teuthology.orchestra.run.smithi117:> sudo systemctl stop ceph-bad8780e-5dc9-11ef-bccf-c7b262605968@mds.k 2024-08-19T01:46:25.202 INFO:journalctl@ceph.mds.k.smithi117.stdout:Aug 19 01:46:24 smithi117 systemd[1]: Stopping Ceph mds.k for bad8780e-5dc9-11ef-bccf-c7b262605968... 2024-08-19T01:46:25.202 INFO:journalctl@ceph.mds.k.smithi117.stdout:Aug 19 01:46:24 smithi117 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-mds-k[50975]: 2024-08-19T01:46:24.952+0000 7f92c84e2640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.k -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-08-19T01:46:25.202 INFO:journalctl@ceph.mds.k.smithi117.stdout:Aug 19 01:46:24 smithi117 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-mds-k[50975]: 2024-08-19T01:46:24.952+0000 7f92c84e2640 -1 mds.k *** got signal Terminated *** 2024-08-19T01:46:26.778 INFO:journalctl@ceph.mds.k.smithi117.stdout:Aug 19 01:46:26 smithi117 podman[57013]: 2024-08-19 01:46:26.505882937 +0000 UTC m=+1.648891773 container died 13892578aec1ff716a77dc2b13bf84642671ed95c4a3995b43855be57447b37f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c9dd28080a631be25c77c36c63b17a3ef340ffb2e21c87d4d7b56b6398436d9, name=ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-mds-k, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, GIT_BRANCH=HEAD, io.buildah.version=1.36.0, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.build-date=20240731, RELEASE=squid-de09dd0) 2024-08-19T01:46:27.128 INFO:journalctl@ceph.mds.k.smithi117.stdout:Aug 19 01:46:26 smithi117 podman[57013]: 2024-08-19 01:46:26.869229615 +0000 UTC m=+2.012238451 container cleanup 13892578aec1ff716a77dc2b13bf84642671ed95c4a3995b43855be57447b37f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c9dd28080a631be25c77c36c63b17a3ef340ffb2e21c87d4d7b56b6398436d9, name=ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-mds-k, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, GIT_CLEAN=True, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , io.buildah.version=1.36.0, ceph=True, RELEASE=squid-de09dd0, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS) 2024-08-19T01:46:27.129 INFO:journalctl@ceph.mds.k.smithi117.stdout:Aug 19 01:46:26 smithi117 bash[57013]: ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-mds-k 2024-08-19T01:46:27.300 DEBUG:teuthology.orchestra.run.smithi117:> sudo pkill -f 'journalctl -f -n 0 -u ceph-bad8780e-5dc9-11ef-bccf-c7b262605968@mds.k.service' 2024-08-19T01:46:27.331 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-08-19T01:46:27.331 INFO:tasks.cephadm.mds.k:Stopped mds.k 2024-08-19T01:46:27.331 INFO:tasks.cephadm.mds.c:Stopping mds.c... 2024-08-19T01:46:27.332 DEBUG:teuthology.orchestra.run.smithi167:> sudo systemctl stop ceph-bad8780e-5dc9-11ef-bccf-c7b262605968@mds.c 2024-08-19T01:46:27.642 INFO:journalctl@ceph.mds.c.smithi167.stdout:Aug 19 01:46:27 smithi167 systemd[1]: Stopping Ceph mds.c for bad8780e-5dc9-11ef-bccf-c7b262605968... 2024-08-19T01:46:27.965 INFO:journalctl@ceph.mds.c.smithi167.stdout:Aug 19 01:46:27 smithi167 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-mds-c[51904]: 2024-08-19T01:46:27.641+0000 7f43210d7640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-08-19T01:46:27.966 INFO:journalctl@ceph.mds.c.smithi167.stdout:Aug 19 01:46:27 smithi167 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-mds-c[51904]: 2024-08-19T01:46:27.641+0000 7f43210d7640 -1 mds.c *** got signal Terminated *** 2024-08-19T01:46:31.918 INFO:journalctl@ceph.mds.c.smithi167.stdout:Aug 19 01:46:31 smithi167 podman[57450]: 2024-08-19 01:46:31.650012574 +0000 UTC m=+4.122607194 container died 2ce4263dad56a40f241a6eee6cdf3e9d02f92e141a847bc3ad074e3b5da74cd7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c9dd28080a631be25c77c36c63b17a3ef340ffb2e21c87d4d7b56b6398436d9, name=ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-mds-c, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-de09dd0, ceph=True, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.36.0, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0) 2024-08-19T01:46:32.216 INFO:journalctl@ceph.mds.c.smithi167.stdout:Aug 19 01:46:32 smithi167 podman[57450]: 2024-08-19 01:46:32.016790188 +0000 UTC m=+4.489384816 container cleanup 2ce4263dad56a40f241a6eee6cdf3e9d02f92e141a847bc3ad074e3b5da74cd7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c9dd28080a631be25c77c36c63b17a3ef340ffb2e21c87d4d7b56b6398436d9, name=ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-mds-c, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , io.buildah.version=1.36.0, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, RELEASE=squid-de09dd0) 2024-08-19T01:46:32.216 INFO:journalctl@ceph.mds.c.smithi167.stdout:Aug 19 01:46:32 smithi167 bash[57450]: ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-mds-c 2024-08-19T01:46:32.449 DEBUG:teuthology.orchestra.run.smithi167:> sudo pkill -f 'journalctl -f -n 0 -u ceph-bad8780e-5dc9-11ef-bccf-c7b262605968@mds.c.service' 2024-08-19T01:46:32.485 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-08-19T01:46:32.485 INFO:tasks.cephadm.mds.c:Stopped mds.c 2024-08-19T01:46:32.485 INFO:tasks.cephadm.mds.f:Stopping mds.f... 2024-08-19T01:46:32.485 DEBUG:teuthology.orchestra.run.smithi167:> sudo systemctl stop ceph-bad8780e-5dc9-11ef-bccf-c7b262605968@mds.f 2024-08-19T01:46:32.830 INFO:journalctl@ceph.mds.f.smithi167.stdout:Aug 19 01:46:32 smithi167 systemd[1]: Stopping Ceph mds.f for bad8780e-5dc9-11ef-bccf-c7b262605968... 2024-08-19T01:46:33.218 INFO:journalctl@ceph.mds.f.smithi167.stdout:Aug 19 01:46:32 smithi167 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-mds-f[51605]: 2024-08-19T01:46:32.829+0000 7f20c9366640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.f -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-08-19T01:46:33.218 INFO:journalctl@ceph.mds.f.smithi167.stdout:Aug 19 01:46:32 smithi167 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-mds-f[51605]: 2024-08-19T01:46:32.829+0000 7f20c9366640 -1 mds.f *** got signal Terminated *** 2024-08-19T01:46:36.180 INFO:journalctl@ceph.mds.f.smithi167.stdout:Aug 19 01:46:35 smithi167 podman[57582]: 2024-08-19 01:46:35.900407003 +0000 UTC m=+3.179937988 container died 15e840bcee59960895a4c4909923b8621aac75d8677afa2a96ef1ce9e2da0653 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c9dd28080a631be25c77c36c63b17a3ef340ffb2e21c87d4d7b56b6398436d9, name=ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-mds-f, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, io.buildah.version=1.36.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, RELEASE=squid-de09dd0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-08-19T01:46:36.466 INFO:journalctl@ceph.mds.f.smithi167.stdout:Aug 19 01:46:36 smithi167 podman[57582]: 2024-08-19 01:46:36.262125685 +0000 UTC m=+3.541656711 container cleanup 15e840bcee59960895a4c4909923b8621aac75d8677afa2a96ef1ce9e2da0653 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c9dd28080a631be25c77c36c63b17a3ef340ffb2e21c87d4d7b56b6398436d9, name=ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-mds-f, ceph=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, RELEASE=squid-de09dd0, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True, io.buildah.version=1.36.0, org.label-schema.license=GPLv2) 2024-08-19T01:46:36.466 INFO:journalctl@ceph.mds.f.smithi167.stdout:Aug 19 01:46:36 smithi167 bash[57582]: ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-mds-f 2024-08-19T01:46:36.709 DEBUG:teuthology.orchestra.run.smithi167:> sudo pkill -f 'journalctl -f -n 0 -u ceph-bad8780e-5dc9-11ef-bccf-c7b262605968@mds.f.service' 2024-08-19T01:46:36.743 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-08-19T01:46:36.744 INFO:tasks.cephadm.mds.f:Stopped mds.f 2024-08-19T01:46:36.744 INFO:tasks.cephadm.mds.i:Stopping mds.i... 2024-08-19T01:46:36.744 DEBUG:teuthology.orchestra.run.smithi167:> sudo systemctl stop ceph-bad8780e-5dc9-11ef-bccf-c7b262605968@mds.i 2024-08-19T01:46:37.216 INFO:journalctl@ceph.mds.i.smithi167.stdout:Aug 19 01:46:36 smithi167 systemd[1]: Stopping Ceph mds.i for bad8780e-5dc9-11ef-bccf-c7b262605968... 2024-08-19T01:46:37.216 INFO:journalctl@ceph.mds.i.smithi167.stdout:Aug 19 01:46:37 smithi167 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-mds-i[52219]: 2024-08-19T01:46:37.066+0000 7fa3ef1d2640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.i -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-08-19T01:46:37.216 INFO:journalctl@ceph.mds.i.smithi167.stdout:Aug 19 01:46:37 smithi167 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-mds-i[52219]: 2024-08-19T01:46:37.066+0000 7fa3ef1d2640 -1 mds.i *** got signal Terminated *** 2024-08-19T01:46:40.100 INFO:journalctl@ceph.mds.i.smithi167.stdout:Aug 19 01:46:39 smithi167 podman[57705]: 2024-08-19 01:46:39.830577778 +0000 UTC m=+2.861318784 container died 7b30511a109093df3b1d6a32480cc08f09b5186d048c958e19c4c35f9cab2947 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c9dd28080a631be25c77c36c63b17a3ef340ffb2e21c87d4d7b56b6398436d9, name=ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-mds-i, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.36.0, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, RELEASE=squid-de09dd0, ceph=True) 2024-08-19T01:46:40.450 INFO:journalctl@ceph.mds.i.smithi167.stdout:Aug 19 01:46:40 smithi167 podman[57705]: 2024-08-19 01:46:40.199414988 +0000 UTC m=+3.230155983 container cleanup 7b30511a109093df3b1d6a32480cc08f09b5186d048c958e19c4c35f9cab2947 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c9dd28080a631be25c77c36c63b17a3ef340ffb2e21c87d4d7b56b6398436d9, name=ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-mds-i, GIT_CLEAN=True, org.label-schema.vendor=CentOS, io.buildah.version=1.36.0, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, RELEASE=squid-de09dd0, ceph=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731) 2024-08-19T01:46:40.451 INFO:journalctl@ceph.mds.i.smithi167.stdout:Aug 19 01:46:40 smithi167 bash[57705]: ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-mds-i 2024-08-19T01:46:40.594 DEBUG:teuthology.orchestra.run.smithi167:> sudo pkill -f 'journalctl -f -n 0 -u ceph-bad8780e-5dc9-11ef-bccf-c7b262605968@mds.i.service' 2024-08-19T01:46:40.626 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-08-19T01:46:40.626 INFO:tasks.cephadm.mds.i:Stopped mds.i 2024-08-19T01:46:40.626 INFO:tasks.cephadm.mds.l:Stopping mds.l... 2024-08-19T01:46:40.626 DEBUG:teuthology.orchestra.run.smithi167:> sudo systemctl stop ceph-bad8780e-5dc9-11ef-bccf-c7b262605968@mds.l 2024-08-19T01:46:40.965 INFO:journalctl@ceph.mds.l.smithi167.stdout:Aug 19 01:46:40 smithi167 systemd[1]: Stopping Ceph mds.l for bad8780e-5dc9-11ef-bccf-c7b262605968... 2024-08-19T01:46:40.966 INFO:journalctl@ceph.mds.l.smithi167.stdout:Aug 19 01:46:40 smithi167 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-mds-l[51299]: 2024-08-19T01:46:40.953+0000 7f320dccd640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.l -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-08-19T01:46:40.966 INFO:journalctl@ceph.mds.l.smithi167.stdout:Aug 19 01:46:40 smithi167 ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-mds-l[51299]: 2024-08-19T01:46:40.953+0000 7f320dccd640 -1 mds.l *** got signal Terminated *** 2024-08-19T01:46:44.379 INFO:journalctl@ceph.mds.l.smithi167.stdout:Aug 19 01:46:44 smithi167 podman[57828]: 2024-08-19 01:46:44.107133983 +0000 UTC m=+3.258698492 container died 3d38eebe4a75426e5a17ca0f3c599c4447342abd046d26b3f0ee8c9410644187 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c9dd28080a631be25c77c36c63b17a3ef340ffb2e21c87d4d7b56b6398436d9, name=ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-mds-l, RELEASE=squid-de09dd0, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, io.buildah.version=1.36.0, GIT_CLEAN=True, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0) 2024-08-19T01:46:44.716 INFO:journalctl@ceph.mds.l.smithi167.stdout:Aug 19 01:46:44 smithi167 podman[57828]: 2024-08-19 01:46:44.47810437 +0000 UTC m=+3.629668885 container cleanup 3d38eebe4a75426e5a17ca0f3c599c4447342abd046d26b3f0ee8c9410644187 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c9dd28080a631be25c77c36c63b17a3ef340ffb2e21c87d4d7b56b6398436d9, name=ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-mds-l, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=squid-de09dd0, io.buildah.version=1.36.0, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=) 2024-08-19T01:46:44.716 INFO:journalctl@ceph.mds.l.smithi167.stdout:Aug 19 01:46:44 smithi167 bash[57828]: ceph-bad8780e-5dc9-11ef-bccf-c7b262605968-mds-l 2024-08-19T01:46:44.902 DEBUG:teuthology.orchestra.run.smithi167:> sudo pkill -f 'journalctl -f -n 0 -u ceph-bad8780e-5dc9-11ef-bccf-c7b262605968@mds.l.service' 2024-08-19T01:46:44.941 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-08-19T01:46:44.941 INFO:tasks.cephadm.mds.l:Stopped mds.l 2024-08-19T01:46:44.941 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid bad8780e-5dc9-11ef-bccf-c7b262605968 --force --keep-logs 2024-08-19T01:46:45.219 INFO:teuthology.orchestra.run.smithi083.stdout:Deleting cluster with fsid: bad8780e-5dc9-11ef-bccf-c7b262605968 2024-08-19T01:46:49.061 DEBUG:teuthology.orchestra.run.smithi117:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid bad8780e-5dc9-11ef-bccf-c7b262605968 --force --keep-logs 2024-08-19T01:46:49.264 INFO:teuthology.orchestra.run.smithi117.stdout:Deleting cluster with fsid: bad8780e-5dc9-11ef-bccf-c7b262605968 2024-08-19T01:46:51.812 DEBUG:teuthology.orchestra.run.smithi167:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid bad8780e-5dc9-11ef-bccf-c7b262605968 --force --keep-logs 2024-08-19T01:46:52.014 INFO:teuthology.orchestra.run.smithi167.stdout:Deleting cluster with fsid: bad8780e-5dc9-11ef-bccf-c7b262605968 2024-08-19T01:46:54.420 DEBUG:teuthology.orchestra.run.smithi083:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-08-19T01:46:54.484 DEBUG:teuthology.orchestra.run.smithi117:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-08-19T01:46:54.518 DEBUG:teuthology.orchestra.run.smithi167:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-08-19T01:46:54.545 INFO:tasks.cephadm:Archiving crash dumps... 2024-08-19T01:46:54.547 DEBUG:teuthology.misc:Transferring archived files from smithi083:/var/lib/ceph/bad8780e-5dc9-11ef-bccf-c7b262605968/crash to /home/teuthworker/archive/teuthology-2024-08-18_21:24:02-fs-squid-distro-default-smithi/7860117/remote/smithi083/crash 2024-08-19T01:46:54.548 DEBUG:teuthology.orchestra.run.smithi083:> sudo tar c -f - -C /var/lib/ceph/bad8780e-5dc9-11ef-bccf-c7b262605968/crash -- . 2024-08-19T01:46:54.604 INFO:teuthology.orchestra.run.smithi083.stderr:tar: /var/lib/ceph/bad8780e-5dc9-11ef-bccf-c7b262605968/crash: Cannot open: No such file or directory 2024-08-19T01:46:54.604 INFO:teuthology.orchestra.run.smithi083.stderr:tar: Error is not recoverable: exiting now 2024-08-19T01:46:54.607 DEBUG:teuthology.misc:Transferring archived files from smithi117:/var/lib/ceph/bad8780e-5dc9-11ef-bccf-c7b262605968/crash to /home/teuthworker/archive/teuthology-2024-08-18_21:24:02-fs-squid-distro-default-smithi/7860117/remote/smithi117/crash 2024-08-19T01:46:54.608 DEBUG:teuthology.orchestra.run.smithi117:> sudo tar c -f - -C /var/lib/ceph/bad8780e-5dc9-11ef-bccf-c7b262605968/crash -- . 2024-08-19T01:46:54.692 INFO:teuthology.orchestra.run.smithi117.stderr:tar: /var/lib/ceph/bad8780e-5dc9-11ef-bccf-c7b262605968/crash: Cannot open: No such file or directory 2024-08-19T01:46:54.692 INFO:teuthology.orchestra.run.smithi117.stderr:tar: Error is not recoverable: exiting now 2024-08-19T01:46:54.694 DEBUG:teuthology.misc:Transferring archived files from smithi167:/var/lib/ceph/bad8780e-5dc9-11ef-bccf-c7b262605968/crash to /home/teuthworker/archive/teuthology-2024-08-18_21:24:02-fs-squid-distro-default-smithi/7860117/remote/smithi167/crash 2024-08-19T01:46:54.695 DEBUG:teuthology.orchestra.run.smithi167:> sudo tar c -f - -C /var/lib/ceph/bad8780e-5dc9-11ef-bccf-c7b262605968/crash -- . 2024-08-19T01:46:54.728 INFO:teuthology.orchestra.run.smithi167.stderr:tar: /var/lib/ceph/bad8780e-5dc9-11ef-bccf-c7b262605968/crash: Cannot open: No such file or directory 2024-08-19T01:46:54.728 INFO:teuthology.orchestra.run.smithi167.stderr:tar: Error is not recoverable: exiting now 2024-08-19T01:46:54.730 INFO:tasks.cephadm:Checking cluster log for badness... 2024-08-19T01:46:54.730 DEBUG:teuthology.orchestra.run.smithi083:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/bad8780e-5dc9-11ef-bccf-c7b262605968/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v FS_DEGRADED | egrep -v FS_INLINE_DATA_DEPRECATED | egrep -v FS_WITH_FAILED_MDS | egrep -v MDS_ALL_DOWN | egrep -v MDS_DAMAGE | egrep -v MDS_DEGRADED | egrep -v MDS_FAILED | egrep -v MDS_INSUFFICIENT_STANDBY | egrep -v MDS_UP_LESS_THAN_MAX | egrep -v 'filesystem is online with fewer MDS than max_mds' | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'do not have an application enabled' | egrep -v 'overall HEALTH_' | egrep -v 'Replacing daemon' | egrep -v 'deprecated feature inline_data' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v MON_DOWN | egrep -v PG_AVAILABILITY | egrep -v PG_DEGRADED | egrep -v 'Reduced data availability' | egrep -v 'Degraded data redundancy' | egrep -v 'slow metadata IO' | egrep -v SLOW_OPS | egrep -v 'slow request' | head -n 1 2024-08-19T01:46:54.806 INFO:teuthology.orchestra.run.smithi083.stdout:2024-08-19T01:37:36.960897+0000 mds.b (mds.0) 19 : cluster [WRN] Scrub error on inode 0x10000001d0f (/volumes/qa/sv_1/05b773d9-8346-4397-99ec-d49959a55703/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34) see mds.b log and `damage ls` output for details 2024-08-19T01:46:54.807 WARNING:tasks.cephadm:Found errors (ERR|WRN|SEC) in cluster log 2024-08-19T01:46:54.808 DEBUG:teuthology.orchestra.run.smithi083:> sudo egrep '\[SEC\]' /var/log/ceph/bad8780e-5dc9-11ef-bccf-c7b262605968/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v FS_DEGRADED | egrep -v FS_INLINE_DATA_DEPRECATED | egrep -v FS_WITH_FAILED_MDS | egrep -v MDS_ALL_DOWN | egrep -v MDS_DAMAGE | egrep -v MDS_DEGRADED | egrep -v MDS_FAILED | egrep -v MDS_INSUFFICIENT_STANDBY | egrep -v MDS_UP_LESS_THAN_MAX | egrep -v 'filesystem is online with fewer MDS than max_mds' | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'do not have an application enabled' | egrep -v 'overall HEALTH_' | egrep -v 'Replacing daemon' | egrep -v 'deprecated feature inline_data' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v MON_DOWN | egrep -v PG_AVAILABILITY | egrep -v PG_DEGRADED | egrep -v 'Reduced data availability' | egrep -v 'Degraded data redundancy' | egrep -v 'slow metadata IO' | egrep -v SLOW_OPS | egrep -v 'slow request' | head -n 1 2024-08-19T01:46:54.883 DEBUG:teuthology.orchestra.run.smithi083:> sudo egrep '\[ERR\]' /var/log/ceph/bad8780e-5dc9-11ef-bccf-c7b262605968/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v FS_DEGRADED | egrep -v FS_INLINE_DATA_DEPRECATED | egrep -v FS_WITH_FAILED_MDS | egrep -v MDS_ALL_DOWN | egrep -v MDS_DAMAGE | egrep -v MDS_DEGRADED | egrep -v MDS_FAILED | egrep -v MDS_INSUFFICIENT_STANDBY | egrep -v MDS_UP_LESS_THAN_MAX | egrep -v 'filesystem is online with fewer MDS than max_mds' | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'do not have an application enabled' | egrep -v 'overall HEALTH_' | egrep -v 'Replacing daemon' | egrep -v 'deprecated feature inline_data' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v MON_DOWN | egrep -v PG_AVAILABILITY | egrep -v PG_DEGRADED | egrep -v 'Reduced data availability' | egrep -v 'Degraded data redundancy' | egrep -v 'slow metadata IO' | egrep -v SLOW_OPS | egrep -v 'slow request' | head -n 1 2024-08-19T01:46:54.960 DEBUG:teuthology.orchestra.run.smithi083:> sudo egrep '\[WRN\]' /var/log/ceph/bad8780e-5dc9-11ef-bccf-c7b262605968/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v FS_DEGRADED | egrep -v FS_INLINE_DATA_DEPRECATED | egrep -v FS_WITH_FAILED_MDS | egrep -v MDS_ALL_DOWN | egrep -v MDS_DAMAGE | egrep -v MDS_DEGRADED | egrep -v MDS_FAILED | egrep -v MDS_INSUFFICIENT_STANDBY | egrep -v MDS_UP_LESS_THAN_MAX | egrep -v 'filesystem is online with fewer MDS than max_mds' | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'do not have an application enabled' | egrep -v 'overall HEALTH_' | egrep -v 'Replacing daemon' | egrep -v 'deprecated feature inline_data' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v MON_DOWN | egrep -v PG_AVAILABILITY | egrep -v PG_DEGRADED | egrep -v 'Reduced data availability' | egrep -v 'Degraded data redundancy' | egrep -v 'slow metadata IO' | egrep -v SLOW_OPS | egrep -v 'slow request' | head -n 1 2024-08-19T01:46:55.034 INFO:teuthology.orchestra.run.smithi083.stdout:2024-08-19T01:37:36.960897+0000 mds.b (mds.0) 19 : cluster [WRN] Scrub error on inode 0x10000001d0f (/volumes/qa/sv_1/05b773d9-8346-4397-99ec-d49959a55703/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34) see mds.b log and `damage ls` output for details 2024-08-19T01:46:55.035 INFO:tasks.cephadm:Compressing logs... 2024-08-19T01:46:55.036 DEBUG:teuthology.orchestra.run.smithi083:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-08-19T01:46:55.040 DEBUG:teuthology.orchestra.run.smithi117:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-08-19T01:46:55.042 DEBUG:teuthology.orchestra.run.smithi167:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-08-19T01:46:55.067 INFO:teuthology.orchestra.run.smithi167.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-08-19T01:46:55.067 INFO:teuthology.orchestra.run.smithi167.stderr:: No such file or directory 2024-08-19T01:46:55.067 INFO:teuthology.orchestra.run.smithi167.stderr:gzip -5 --verbose -- /var/log/ceph/bad8780e-5dc9-11ef-bccf-c7b262605968/ceph-volume.log 2024-08-19T01:46:55.067 INFO:teuthology.orchestra.run.smithi167.stderr:/var/log/ceph/cephadm.log: gzip -5 --verbose -- /var/log/ceph/bad8780e-5dc9-11ef-bccf-c7b262605968/ceph-mgr.z.log 2024-08-19T01:46:55.068 INFO:teuthology.orchestra.run.smithi167.stderr:/var/log/ceph/bad8780e-5dc9-11ef-bccf-c7b262605968/ceph-volume.log: gzip -5 --verbose -- /var/log/ceph/bad8780e-5dc9-11ef-bccf-c7b262605968/ceph-osd.2.log 2024-08-19T01:46:55.069 INFO:teuthology.orchestra.run.smithi167.stderr:/var/log/ceph/bad8780e-5dc9-11ef-bccf-c7b262605968/ceph-mgr.z.log: gzip -5 --verbose -- /var/log/ceph/bad8780e-5dc9-11ef-bccf-c7b262605968/ceph-osd.11.log 2024-08-19T01:46:55.069 INFO:teuthology.orchestra.run.smithi167.stderr:gzip -5 --verbose -- /var/log/ceph/bad8780e-5dc9-11ef-bccf-c7b262605968/ceph-mds.f.log 2024-08-19T01:46:55.069 INFO:teuthology.orchestra.run.smithi167.stderr:/var/log/ceph/bad8780e-5dc9-11ef-bccf-c7b262605968/ceph-osd.2.log: /var/log/ceph/bad8780e-5dc9-11ef-bccf-c7b262605968/ceph-osd.11.log: gzip -5 --verbose -- /var/log/ceph/bad8780e-5dc9-11ef-bccf-c7b262605968/ceph.cephadm.log 2024-08-19T01:46:55.069 INFO:teuthology.orchestra.run.smithi167.stderr: 92.2% -- replaced with /var/log/ceph/cephadm.log.gz 2024-08-19T01:46:55.072 INFO:teuthology.orchestra.run.smithi167.stderr:/var/log/ceph/bad8780e-5dc9-11ef-bccf-c7b262605968/ceph-mds.f.log: gzip -5 --verbose -- /var/log/ceph/bad8780e-5dc9-11ef-bccf-c7b262605968/ceph.audit.log 2024-08-19T01:46:55.072 INFO:teuthology.orchestra.run.smithi167.stderr:/var/log/ceph/bad8780e-5dc9-11ef-bccf-c7b262605968/ceph.cephadm.log: gzip -5 --verbose -- /var/log/ceph/bad8780e-5dc9-11ef-bccf-c7b262605968/ceph-osd.8.log 2024-08-19T01:46:55.073 INFO:teuthology.orchestra.run.smithi167.stderr: 82.7% -- replaced with /var/log/ceph/bad8780e-5dc9-11ef-bccf-c7b262605968/ceph.cephadm.log.gz 2024-08-19T01:46:55.073 INFO:teuthology.orchestra.run.smithi167.stderr:/var/log/ceph/bad8780e-5dc9-11ef-bccf-c7b262605968/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/bad8780e-5dc9-11ef-bccf-c7b262605968/ceph-mds.i.log 2024-08-19T01:46:55.073 INFO:teuthology.orchestra.run.smithi167.stderr:/var/log/ceph/bad8780e-5dc9-11ef-bccf-c7b262605968/ceph-osd.8.log: gzip -5 --verbose -- /var/log/ceph/bad8780e-5dc9-11ef-bccf-c7b262605968/ceph-mds.c.log 2024-08-19T01:46:55.073 INFO:teuthology.orchestra.run.smithi167.stderr:/var/log/ceph/bad8780e-5dc9-11ef-bccf-c7b262605968/ceph-mds.i.log: 90.5% -- replaced with /var/log/ceph/bad8780e-5dc9-11ef-bccf-c7b262605968/ceph.audit.log.gz 2024-08-19T01:46:55.075 INFO:teuthology.orchestra.run.smithi167.stderr: 94.7% -- replaced with /var/log/ceph/bad8780e-5dc9-11ef-bccf-c7b262605968/ceph-mds.f.log.gz 2024-08-19T01:46:55.076 INFO:teuthology.orchestra.run.smithi167.stderr:gzip -5 --verbose -- /var/log/ceph/bad8780e-5dc9-11ef-bccf-c7b262605968/ceph-osd.5.log 2024-08-19T01:46:55.076 INFO:teuthology.orchestra.run.smithi167.stderr: 92.8% -- replaced with /var/log/ceph/bad8780e-5dc9-11ef-bccf-c7b262605968/ceph-mgr.z.log.gz 2024-08-19T01:46:55.076 INFO:teuthology.orchestra.run.smithi167.stderr:/var/log/ceph/bad8780e-5dc9-11ef-bccf-c7b262605968/ceph-mds.c.log: gzip -5 --verbose -- /var/log/ceph/bad8780e-5dc9-11ef-bccf-c7b262605968/ceph.log 2024-08-19T01:46:55.076 INFO:teuthology.orchestra.run.smithi167.stderr:/var/log/ceph/bad8780e-5dc9-11ef-bccf-c7b262605968/ceph-osd.5.log: gzip -5 --verbose -- /var/log/ceph/bad8780e-5dc9-11ef-bccf-c7b262605968/ceph-mon.c.log 2024-08-19T01:46:55.076 INFO:teuthology.orchestra.run.smithi117.stderr:find: ‘/var/log/rbd-target-api’: No such file or directory 2024-08-19T01:46:55.077 INFO:teuthology.orchestra.run.smithi117.stderr:gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-08-19T01:46:55.077 INFO:teuthology.orchestra.run.smithi117.stderr:gzip -5 --verbose -- /var/log/ceph/bad8780e-5dc9-11ef-bccf-c7b262605968/ceph-volume.log 2024-08-19T01:46:55.078 INFO:teuthology.orchestra.run.smithi117.stderr:/var/log/ceph/cephadm.log: gzip -5 --verbose -- /var/log/ceph/bad8780e-5dc9-11ef-bccf-c7b262605968/ceph-mds.e.log 2024-08-19T01:46:55.078 INFO:teuthology.orchestra.run.smithi117.stderr:/var/log/ceph/bad8780e-5dc9-11ef-bccf-c7b262605968/ceph-volume.log: gzip -5 --verbose -- /var/log/ceph/bad8780e-5dc9-11ef-bccf-c7b262605968/ceph-mds.h.log 2024-08-19T01:46:55.078 INFO:teuthology.orchestra.run.smithi117.stderr:/var/log/ceph/bad8780e-5dc9-11ef-bccf-c7b262605968/ceph-mds.e.log: gzip -5 --verbose -- /var/log/ceph/bad8780e-5dc9-11ef-bccf-c7b262605968/ceph.cephadm.log 2024-08-19T01:46:55.081 INFO:teuthology.orchestra.run.smithi167.stderr:/var/log/ceph/bad8780e-5dc9-11ef-bccf-c7b262605968/ceph.log: 86.2% -- replaced with /var/log/ceph/bad8780e-5dc9-11ef-bccf-c7b262605968/ceph.log.gz 2024-08-19T01:46:55.081 INFO:teuthology.orchestra.run.smithi167.stderr:gzip -5 --verbose -- /var/log/ceph/bad8780e-5dc9-11ef-bccf-c7b262605968/ceph-mds.l.log 2024-08-19T01:46:55.081 INFO:teuthology.orchestra.run.smithi167.stderr:/var/log/ceph/bad8780e-5dc9-11ef-bccf-c7b262605968/ceph-mon.c.log: 90.9% -- replaced with /var/log/ceph/bad8780e-5dc9-11ef-bccf-c7b262605968/ceph-volume.log.gz 2024-08-19T01:46:55.083 INFO:teuthology.orchestra.run.smithi117.stderr:/var/log/ceph/bad8780e-5dc9-11ef-bccf-c7b262605968/ceph-mds.h.log: gzip -5 --verbose -- /var/log/ceph/bad8780e-5dc9-11ef-bccf-c7b262605968/ceph.audit.log 2024-08-19T01:46:55.083 INFO:teuthology.orchestra.run.smithi117.stderr:/var/log/ceph/bad8780e-5dc9-11ef-bccf-c7b262605968/ceph.cephadm.log: gzip -5 --verbose -- /var/log/ceph/bad8780e-5dc9-11ef-bccf-c7b262605968/ceph-mds.b.log 2024-08-19T01:46:55.083 INFO:teuthology.orchestra.run.smithi117.stderr:/var/log/ceph/bad8780e-5dc9-11ef-bccf-c7b262605968/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/bad8780e-5dc9-11ef-bccf-c7b262605968/ceph-osd.10.log 2024-08-19T01:46:55.083 INFO:teuthology.orchestra.run.smithi117.stderr:/var/log/ceph/bad8780e-5dc9-11ef-bccf-c7b262605968/ceph-mds.b.log: gzip -5 --verbose -- /var/log/ceph/bad8780e-5dc9-11ef-bccf-c7b262605968/ceph-osd.4.log 2024-08-19T01:46:55.083 INFO:teuthology.orchestra.run.smithi117.stderr:/var/log/ceph/bad8780e-5dc9-11ef-bccf-c7b262605968/ceph-osd.10.log: gzip -5 --verbose -- /var/log/ceph/bad8780e-5dc9-11ef-bccf-c7b262605968/ceph-mds.k.log 2024-08-19T01:46:55.083 INFO:teuthology.orchestra.run.smithi117.stderr:/var/log/ceph/bad8780e-5dc9-11ef-bccf-c7b262605968/ceph-osd.4.log: gzip -5 --verbose -- /var/log/ceph/bad8780e-5dc9-11ef-bccf-c7b262605968/ceph-mon.b.log 2024-08-19T01:46:55.083 INFO:teuthology.orchestra.run.smithi117.stderr:/var/log/ceph/bad8780e-5dc9-11ef-bccf-c7b262605968/ceph-mds.k.log: gzip -5 --verbose -- /var/log/ceph/bad8780e-5dc9-11ef-bccf-c7b262605968/ceph-mgr.y.log 2024-08-19T01:46:55.083 INFO:teuthology.orchestra.run.smithi117.stderr:/var/log/ceph/bad8780e-5dc9-11ef-bccf-c7b262605968/ceph-mon.b.log: gzip -5 --verbose -- /var/log/ceph/bad8780e-5dc9-11ef-bccf-c7b262605968/ceph-osd.7.log 2024-08-19T01:46:55.084 INFO:teuthology.orchestra.run.smithi117.stderr:/var/log/ceph/bad8780e-5dc9-11ef-bccf-c7b262605968/ceph-mgr.y.log: gzip -5 --verbose -- /var/log/ceph/bad8780e-5dc9-11ef-bccf-c7b262605968/ceph-osd.1.log 2024-08-19T01:46:55.084 INFO:teuthology.orchestra.run.smithi117.stderr:/var/log/ceph/bad8780e-5dc9-11ef-bccf-c7b262605968/ceph-osd.7.log: gzip -5 --verbose -- /var/log/ceph/bad8780e-5dc9-11ef-bccf-c7b262605968/ceph.log 2024-08-19T01:46:55.084 INFO:teuthology.orchestra.run.smithi117.stderr:/var/log/ceph/bad8780e-5dc9-11ef-bccf-c7b262605968/ceph-osd.1.log: /var/log/ceph/bad8780e-5dc9-11ef-bccf-c7b262605968/ceph.log: 94.6% -- replaced with /var/log/ceph/bad8780e-5dc9-11ef-bccf-c7b262605968/ceph-mds.k.log.gz 2024-08-19T01:46:55.089 INFO:teuthology.orchestra.run.smithi083.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-08-19T01:46:55.089 INFO:teuthology.orchestra.run.smithi083.stderr:: No such file or directory 2024-08-19T01:46:55.090 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/bad8780e-5dc9-11ef-bccf-c7b262605968/ceph-volume.log 2024-08-19T01:46:55.091 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/cephadm.log: gzip -5 --verbose -- /var/log/ceph/bad8780e-5dc9-11ef-bccf-c7b262605968/ceph-osd.3.log 2024-08-19T01:46:55.091 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/bad8780e-5dc9-11ef-bccf-c7b262605968/ceph-volume.log: gzip -5 --verbose -- /var/log/ceph/bad8780e-5dc9-11ef-bccf-c7b262605968/ceph-mds.d.log 2024-08-19T01:46:55.092 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/bad8780e-5dc9-11ef-bccf-c7b262605968/ceph-osd.3.log: gzip -5 --verbose -- /var/log/ceph/bad8780e-5dc9-11ef-bccf-c7b262605968/ceph-mgr.x.log 2024-08-19T01:46:55.094 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/bad8780e-5dc9-11ef-bccf-c7b262605968/ceph-mds.d.log: gzip -5 --verbose -- /var/log/ceph/bad8780e-5dc9-11ef-bccf-c7b262605968/ceph.cephadm.log 2024-08-19T01:46:55.094 INFO:teuthology.orchestra.run.smithi167.stderr:/var/log/ceph/bad8780e-5dc9-11ef-bccf-c7b262605968/ceph-mds.l.log: 94.6% -- replaced with /var/log/ceph/bad8780e-5dc9-11ef-bccf-c7b262605968/ceph-mds.l.log.gz 2024-08-19T01:46:55.095 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/bad8780e-5dc9-11ef-bccf-c7b262605968/ceph-mgr.x.log: gzip -5 --verbose -- /var/log/ceph/bad8780e-5dc9-11ef-bccf-c7b262605968/ceph.audit.log 2024-08-19T01:46:55.097 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/bad8780e-5dc9-11ef-bccf-c7b262605968/ceph.cephadm.log: gzip -5 --verbose -- /var/log/ceph/bad8780e-5dc9-11ef-bccf-c7b262605968/ceph-mon.a.log 2024-08-19T01:46:55.097 INFO:teuthology.orchestra.run.smithi083.stderr: 82.7% -- replaced with /var/log/ceph/bad8780e-5dc9-11ef-bccf-c7b262605968/ceph.cephadm.log.gz 2024-08-19T01:46:55.098 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/bad8780e-5dc9-11ef-bccf-c7b262605968/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/bad8780e-5dc9-11ef-bccf-c7b262605968/ceph-mds.a.log 2024-08-19T01:46:55.098 INFO:teuthology.orchestra.run.smithi083.stderr: 94.4% -- replaced with /var/log/ceph/bad8780e-5dc9-11ef-bccf-c7b262605968/ceph-mds.d.log.gz 2024-08-19T01:46:55.099 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/bad8780e-5dc9-11ef-bccf-c7b262605968/ceph-mon.a.log: 93.9%gzip -- replaced with /var/log/ceph/cephadm.log.gz -5 2024-08-19T01:46:55.099 INFO:teuthology.orchestra.run.smithi083.stderr: --verbose -- /var/log/ceph/bad8780e-5dc9-11ef-bccf-c7b262605968/ceph-osd.9.log 2024-08-19T01:46:55.100 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/bad8780e-5dc9-11ef-bccf-c7b262605968/ceph-mds.a.log: gzip -5 --verbose -- /var/log/ceph/bad8780e-5dc9-11ef-bccf-c7b262605968/ceph-osd.6.log 2024-08-19T01:46:55.101 INFO:teuthology.orchestra.run.smithi083.stderr: 90.3% -- replaced with /var/log/ceph/bad8780e-5dc9-11ef-bccf-c7b262605968/ceph.audit.log.gz 2024-08-19T01:46:55.102 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/bad8780e-5dc9-11ef-bccf-c7b262605968/ceph-osd.9.log: gzip -5 --verbose -- /var/log/ceph/bad8780e-5dc9-11ef-bccf-c7b262605968/ceph-mds.j.log 2024-08-19T01:46:55.104 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/bad8780e-5dc9-11ef-bccf-c7b262605968/ceph-osd.6.log: /var/log/ceph/bad8780e-5dc9-11ef-bccf-c7b262605968/ceph-mds.j.log: 94.6% -- replaced with /var/log/ceph/bad8780e-5dc9-11ef-bccf-c7b262605968/ceph-mds.a.log.gz 2024-08-19T01:46:55.105 INFO:teuthology.orchestra.run.smithi083.stderr: 90.9% -- replaced with /var/log/ceph/bad8780e-5dc9-11ef-bccf-c7b262605968/ceph-volume.log.gz 2024-08-19T01:46:55.105 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/bad8780e-5dc9-11ef-bccf-c7b262605968/ceph-osd.0.log 2024-08-19T01:46:55.107 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/bad8780e-5dc9-11ef-bccf-c7b262605968/ceph.log 2024-08-19T01:46:55.109 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/bad8780e-5dc9-11ef-bccf-c7b262605968/ceph-osd.0.log: /var/log/ceph/bad8780e-5dc9-11ef-bccf-c7b262605968/ceph.log: gzip -5 --verbose 94.6% -- replaced with /var/log/ceph/bad8780e-5dc9-11ef-bccf-c7b262605968/ceph-mds.j.log.gz 2024-08-19T01:46:55.109 INFO:teuthology.orchestra.run.smithi083.stderr: -- /var/log/ceph/bad8780e-5dc9-11ef-bccf-c7b262605968/ceph-mds.g.log 2024-08-19T01:46:55.112 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/bad8780e-5dc9-11ef-bccf-c7b262605968/ceph-mds.g.log: 86.2% -- replaced with /var/log/ceph/bad8780e-5dc9-11ef-bccf-c7b262605968/ceph.log.gz 2024-08-19T01:46:55.171 INFO:teuthology.orchestra.run.smithi117.stderr: 90.9% -- replaced with /var/log/ceph/bad8780e-5dc9-11ef-bccf-c7b262605968/ceph-volume.log.gz 2024-08-19T01:46:55.224 INFO:teuthology.orchestra.run.smithi117.stderr: 92.6% -- replaced with /var/log/ceph/bad8780e-5dc9-11ef-bccf-c7b262605968/ceph-mgr.y.log.gz 2024-08-19T01:46:55.254 INFO:teuthology.orchestra.run.smithi117.stderr: 82.8% -- replaced with /var/log/ceph/bad8780e-5dc9-11ef-bccf-c7b262605968/ceph.cephadm.log.gz 2024-08-19T01:46:55.255 INFO:teuthology.orchestra.run.smithi117.stderr: 91.8% -- replaced with /var/log/ceph/cephadm.log.gz 2024-08-19T01:46:55.338 INFO:teuthology.orchestra.run.smithi117.stderr: 90.4% -- replaced with /var/log/ceph/bad8780e-5dc9-11ef-bccf-c7b262605968/ceph.audit.log.gz 2024-08-19T01:46:55.347 INFO:teuthology.orchestra.run.smithi117.stderr: 86.2% -- replaced with /var/log/ceph/bad8780e-5dc9-11ef-bccf-c7b262605968/ceph.log.gz 2024-08-19T01:46:55.369 INFO:teuthology.orchestra.run.smithi083.stderr: 90.3% -- replaced with /var/log/ceph/bad8780e-5dc9-11ef-bccf-c7b262605968/ceph-mgr.x.log.gz 2024-08-19T01:46:55.494 INFO:teuthology.orchestra.run.smithi167.stderr: 91.7% -- replaced with /var/log/ceph/bad8780e-5dc9-11ef-bccf-c7b262605968/ceph-mon.c.log.gz 2024-08-19T01:46:55.917 INFO:teuthology.orchestra.run.smithi167.stderr: 95.2% -- replaced with /var/log/ceph/bad8780e-5dc9-11ef-bccf-c7b262605968/ceph-mds.c.log.gz 2024-08-19T01:46:56.233 INFO:teuthology.orchestra.run.smithi083.stderr: 90.4% -- replaced with /var/log/ceph/bad8780e-5dc9-11ef-bccf-c7b262605968/ceph-mon.a.log.gz 2024-08-19T01:46:57.569 INFO:teuthology.orchestra.run.smithi117.stderr: 91.6% -- replaced with /var/log/ceph/bad8780e-5dc9-11ef-bccf-c7b262605968/ceph-mon.b.log.gz 2024-08-19T01:46:58.278 INFO:teuthology.orchestra.run.smithi117.stderr: 94.9% -- replaced with /var/log/ceph/bad8780e-5dc9-11ef-bccf-c7b262605968/ceph-mds.e.log.gz 2024-08-19T01:47:01.087 INFO:teuthology.orchestra.run.smithi083.stderr: 94.0% -- replaced with /var/log/ceph/bad8780e-5dc9-11ef-bccf-c7b262605968/ceph-osd.9.log.gz 2024-08-19T01:47:01.742 INFO:teuthology.orchestra.run.smithi083.stderr: 94.9% -- replaced with /var/log/ceph/bad8780e-5dc9-11ef-bccf-c7b262605968/ceph-mds.g.log.gz 2024-08-19T01:47:01.919 INFO:teuthology.orchestra.run.smithi167.stderr: 93.8% -- replaced with /var/log/ceph/bad8780e-5dc9-11ef-bccf-c7b262605968/ceph-osd.8.log.gz 2024-08-19T01:47:03.804 INFO:teuthology.orchestra.run.smithi167.stderr: 93.9% -- replaced with /var/log/ceph/bad8780e-5dc9-11ef-bccf-c7b262605968/ceph-osd.11.log.gz 2024-08-19T01:47:04.199 INFO:teuthology.orchestra.run.smithi167.stderr: 93.9% -- replaced with /var/log/ceph/bad8780e-5dc9-11ef-bccf-c7b262605968/ceph-osd.2.log.gz 2024-08-19T01:47:05.076 INFO:teuthology.orchestra.run.smithi167.stderr: 93.8% -- replaced with /var/log/ceph/bad8780e-5dc9-11ef-bccf-c7b262605968/ceph-osd.5.log.gz 2024-08-19T01:47:05.155 INFO:teuthology.orchestra.run.smithi083.stderr: 93.9% -- replaced with /var/log/ceph/bad8780e-5dc9-11ef-bccf-c7b262605968/ceph-osd.0.log.gz 2024-08-19T01:47:05.892 INFO:teuthology.orchestra.run.smithi083.stderr: 93.7% -- replaced with /var/log/ceph/bad8780e-5dc9-11ef-bccf-c7b262605968/ceph-osd.6.log.gz 2024-08-19T01:47:06.051 INFO:teuthology.orchestra.run.smithi083.stderr: 93.7% -- replaced with /var/log/ceph/bad8780e-5dc9-11ef-bccf-c7b262605968/ceph-osd.3.log.gz 2024-08-19T01:47:06.054 INFO:teuthology.orchestra.run.smithi083.stderr: 2024-08-19T01:47:06.054 INFO:teuthology.orchestra.run.smithi083.stderr:real 0m10.986s 2024-08-19T01:47:06.054 INFO:teuthology.orchestra.run.smithi083.stderr:user 0m41.944s 2024-08-19T01:47:06.054 INFO:teuthology.orchestra.run.smithi083.stderr:sys 0m3.524s 2024-08-19T01:47:11.207 INFO:teuthology.orchestra.run.smithi117.stderr: 93.8% -- replaced with /var/log/ceph/bad8780e-5dc9-11ef-bccf-c7b262605968/ceph-osd.1.log.gz 2024-08-19T01:47:13.761 INFO:teuthology.orchestra.run.smithi167.stderr: 94.7% -- replaced with /var/log/ceph/bad8780e-5dc9-11ef-bccf-c7b262605968/ceph-mds.i.log.gz 2024-08-19T01:47:13.763 INFO:teuthology.orchestra.run.smithi167.stderr: 2024-08-19T01:47:13.763 INFO:teuthology.orchestra.run.smithi167.stderr:real 0m18.706s 2024-08-19T01:47:13.763 INFO:teuthology.orchestra.run.smithi167.stderr:user 0m52.780s 2024-08-19T01:47:13.763 INFO:teuthology.orchestra.run.smithi167.stderr:sys 0m1.840s 2024-08-19T01:47:35.636 INFO:teuthology.orchestra.run.smithi117.stderr: 93.9% -- replaced with /var/log/ceph/bad8780e-5dc9-11ef-bccf-c7b262605968/ceph-osd.10.log.gz 2024-08-19T01:47:40.620 INFO:teuthology.orchestra.run.smithi117.stderr: 93.9% -- replaced with /var/log/ceph/bad8780e-5dc9-11ef-bccf-c7b262605968/ceph-osd.4.log.gz 2024-08-19T01:47:54.865 INFO:teuthology.orchestra.run.smithi117.stderr: 94.2% -- replaced with /var/log/ceph/bad8780e-5dc9-11ef-bccf-c7b262605968/ceph-mds.h.log.gz 2024-08-19T01:48:01.142 INFO:teuthology.orchestra.run.smithi117.stderr: 94.0% -- replaced with /var/log/ceph/bad8780e-5dc9-11ef-bccf-c7b262605968/ceph-osd.7.log.gz 2024-08-19T01:52:43.386 INFO:teuthology.orchestra.run.smithi117.stderr: 93.1% -- replaced with /var/log/ceph/bad8780e-5dc9-11ef-bccf-c7b262605968/ceph-mds.b.log.gz 2024-08-19T01:52:43.388 INFO:teuthology.orchestra.run.smithi117.stderr: 2024-08-19T01:52:43.388 INFO:teuthology.orchestra.run.smithi117.stderr:real 5m48.330s 2024-08-19T01:52:43.388 INFO:teuthology.orchestra.run.smithi117.stderr:user 4m55.121s 2024-08-19T01:52:43.388 INFO:teuthology.orchestra.run.smithi117.stderr:sys 0m12.291s 2024-08-19T01:52:43.388 INFO:tasks.cephadm:Archiving logs... 2024-08-19T01:52:43.389 DEBUG:teuthology.misc:Transferring archived files from smithi083:/var/log/ceph to /home/teuthworker/archive/teuthology-2024-08-18_21:24:02-fs-squid-distro-default-smithi/7860117/remote/smithi083/log 2024-08-19T01:52:43.390 DEBUG:teuthology.orchestra.run.smithi083:> sudo tar c -f - -C /var/log/ceph -- . 2024-08-19T01:52:48.580 DEBUG:teuthology.misc:Transferring archived files from smithi117:/var/log/ceph to /home/teuthworker/archive/teuthology-2024-08-18_21:24:02-fs-squid-distro-default-smithi/7860117/remote/smithi117/log 2024-08-19T01:52:48.581 DEBUG:teuthology.orchestra.run.smithi117:> sudo tar c -f - -C /var/log/ceph -- . 2024-08-19T01:53:35.809 DEBUG:teuthology.misc:Transferring archived files from smithi167:/var/log/ceph to /home/teuthworker/archive/teuthology-2024-08-18_21:24:02-fs-squid-distro-default-smithi/7860117/remote/smithi167/log 2024-08-19T01:53:35.810 DEBUG:teuthology.orchestra.run.smithi167:> sudo tar c -f - -C /var/log/ceph -- . 2024-08-19T01:53:42.529 INFO:tasks.cephadm:Removing cluster... 2024-08-19T01:53:42.529 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid bad8780e-5dc9-11ef-bccf-c7b262605968 --force 2024-08-19T01:53:42.807 INFO:teuthology.orchestra.run.smithi083.stdout:Deleting cluster with fsid: bad8780e-5dc9-11ef-bccf-c7b262605968 2024-08-19T01:53:43.366 DEBUG:teuthology.orchestra.run.smithi117:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid bad8780e-5dc9-11ef-bccf-c7b262605968 --force 2024-08-19T01:53:43.918 INFO:teuthology.orchestra.run.smithi117.stdout:Deleting cluster with fsid: bad8780e-5dc9-11ef-bccf-c7b262605968 2024-08-19T01:53:44.443 DEBUG:teuthology.orchestra.run.smithi167:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid bad8780e-5dc9-11ef-bccf-c7b262605968 --force 2024-08-19T01:53:44.647 INFO:teuthology.orchestra.run.smithi167.stdout:Deleting cluster with fsid: bad8780e-5dc9-11ef-bccf-c7b262605968 2024-08-19T01:53:44.961 INFO:tasks.cephadm:Removing cephadm ... 2024-08-19T01:53:44.961 DEBUG:teuthology.orchestra.run.smithi083:> rm -rf /home/ubuntu/cephtest/cephadm 2024-08-19T01:53:44.998 DEBUG:teuthology.orchestra.run.smithi117:> rm -rf /home/ubuntu/cephtest/cephadm 2024-08-19T01:53:45.015 DEBUG:teuthology.orchestra.run.smithi167:> rm -rf /home/ubuntu/cephtest/cephadm 2024-08-19T01:53:45.031 INFO:tasks.cephadm:Teardown complete 2024-08-19T01:53:45.031 DEBUG:teuthology.run_tasks:Unwinding manager install 2024-08-19T01:53:45.045 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2024-08-19T01:53:45.045 DEBUG:teuthology.orchestra.run.smithi083:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-08-19T01:53:45.050 DEBUG:teuthology.orchestra.run.smithi117:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-08-19T01:53:45.058 DEBUG:teuthology.orchestra.run.smithi167:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-08-19T01:53:45.172 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-08-19T01:53:45.183 INFO:teuthology.task.clock:Checking final clock skew... 2024-08-19T01:53:45.183 DEBUG:teuthology.orchestra.run.smithi083:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-08-19T01:53:45.187 DEBUG:teuthology.orchestra.run.smithi117:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-08-19T01:53:45.190 DEBUG:teuthology.orchestra.run.smithi167:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-08-19T01:53:45.202 INFO:teuthology.orchestra.run.smithi117.stderr:bash: line 1: ntpq: command not found 2024-08-19T01:53:45.204 INFO:teuthology.orchestra.run.smithi167.stderr:bash: line 1: ntpq: command not found 2024-08-19T01:53:45.208 INFO:teuthology.orchestra.run.smithi167.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-08-19T01:53:45.208 INFO:teuthology.orchestra.run.smithi167.stdout:=============================================================================== 2024-08-19T01:53:45.208 INFO:teuthology.orchestra.run.smithi167.stdout:^* hv01.front.sepia.ceph.com 3 6 377 64 +254us[ +266us] +/- 48ms 2024-08-19T01:53:45.208 INFO:teuthology.orchestra.run.smithi167.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 1 -298us[ -298us] +/- 84ms 2024-08-19T01:53:45.208 INFO:teuthology.orchestra.run.smithi167.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 2 -129us[ -129us] +/- 87ms 2024-08-19T01:53:45.209 INFO:teuthology.orchestra.run.smithi167.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-08-19T01:53:45.217 INFO:teuthology.orchestra.run.smithi083.stderr:bash: line 1: ntpq: command not found 2024-08-19T01:53:45.223 INFO:teuthology.orchestra.run.smithi083.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-08-19T01:53:45.224 INFO:teuthology.orchestra.run.smithi083.stdout:=============================================================================== 2024-08-19T01:53:45.224 INFO:teuthology.orchestra.run.smithi083.stdout:^* hv01.front.sepia.ceph.com 3 6 377 2 +266us[ +278us] +/- 49ms 2024-08-19T01:53:45.224 INFO:teuthology.orchestra.run.smithi083.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 64 -316us[ -309us] +/- 83ms 2024-08-19T01:53:45.224 INFO:teuthology.orchestra.run.smithi083.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 4 -146us[ -134us] +/- 87ms 2024-08-19T01:53:45.224 INFO:teuthology.orchestra.run.smithi083.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-08-19T01:53:45.231 INFO:teuthology.orchestra.run.smithi117.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-08-19T01:53:45.232 INFO:teuthology.orchestra.run.smithi117.stdout:=============================================================================== 2024-08-19T01:53:45.232 INFO:teuthology.orchestra.run.smithi117.stdout:^* hv01.front.sepia.ceph.com 3 6 377 2 +225us[ +237us] +/- 49ms 2024-08-19T01:53:45.232 INFO:teuthology.orchestra.run.smithi117.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 65 -317us[ -307us] +/- 83ms 2024-08-19T01:53:45.232 INFO:teuthology.orchestra.run.smithi117.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 64 -153us[ -143us] +/- 86ms 2024-08-19T01:53:45.232 INFO:teuthology.orchestra.run.smithi117.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-08-19T01:53:45.232 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-08-19T01:53:45.271 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-08-19T01:53:45.272 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-08-19T01:53:45.281 DEBUG:teuthology.orchestra.run.smithi083:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-08-19T01:53:45.391 DEBUG:teuthology.orchestra.run.smithi117:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-08-19T01:53:45.482 DEBUG:teuthology.orchestra.run.smithi167:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-08-19T01:53:45.536 DEBUG:teuthology.orchestra.run.smithi083:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-08-19T01:53:45.595 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-08-19T01:53:45.596 DEBUG:teuthology.orchestra.run.smithi117:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-08-19T01:53:45.622 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-08-19T01:53:45.623 DEBUG:teuthology.orchestra.run.smithi167:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-08-19T01:53:45.649 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-08-19T01:53:45.650 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-08-19T01:53:45.660 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-08-19T01:53:45.685 INFO:teuthology.task.internal:Duration was 2009.938051 seconds 2024-08-19T01:53:45.685 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-08-19T01:53:45.694 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-08-19T01:53:45.694 DEBUG:teuthology.orchestra.run.smithi083:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-08-19T01:53:45.699 DEBUG:teuthology.orchestra.run.smithi117:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-08-19T01:53:45.701 DEBUG:teuthology.orchestra.run.smithi167:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-08-19T01:53:45.740 INFO:teuthology.orchestra.run.smithi167.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-08-19T01:53:45.761 INFO:teuthology.orchestra.run.smithi117.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-08-19T01:53:45.781 INFO:teuthology.orchestra.run.smithi083.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-08-19T01:53:46.244 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-08-19T01:53:46.244 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi083.front.sepia.ceph.com 2024-08-19T01:53:46.245 DEBUG:teuthology.orchestra.run.smithi083:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-08-19T01:53:46.294 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi117.front.sepia.ceph.com 2024-08-19T01:53:46.294 DEBUG:teuthology.orchestra.run.smithi117:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-08-19T01:53:46.324 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi167.front.sepia.ceph.com 2024-08-19T01:53:46.324 DEBUG:teuthology.orchestra.run.smithi167:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-08-19T01:53:46.357 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-08-19T01:53:46.357 DEBUG:teuthology.orchestra.run.smithi083:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-08-19T01:53:46.361 DEBUG:teuthology.orchestra.run.smithi117:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-08-19T01:53:46.367 DEBUG:teuthology.orchestra.run.smithi167:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-08-19T01:53:46.424 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-08-19T01:53:46.424 DEBUG:teuthology.orchestra.run.smithi083:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-08-19T01:53:46.428 DEBUG:teuthology.orchestra.run.smithi117:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-08-19T01:53:46.431 DEBUG:teuthology.orchestra.run.smithi167:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-08-19T01:53:47.190 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-08-19T01:53:47.202 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-08-19T01:53:47.202 DEBUG:teuthology.orchestra.run.smithi083:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-08-19T01:53:47.258 DEBUG:teuthology.orchestra.run.smithi117:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-08-19T01:53:47.316 DEBUG:teuthology.orchestra.run.smithi167:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-08-19T01:53:47.343 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-08-19T01:53:47.355 DEBUG:teuthology.orchestra.run.smithi083:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-08-19T01:53:47.359 DEBUG:teuthology.orchestra.run.smithi117:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-08-19T01:53:47.362 DEBUG:teuthology.orchestra.run.smithi167:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-08-19T01:53:47.389 INFO:teuthology.orchestra.run.smithi117.stdout:kernel.core_pattern = core 2024-08-19T01:53:47.407 INFO:teuthology.orchestra.run.smithi167.stdout:kernel.core_pattern = core 2024-08-19T01:53:47.425 INFO:teuthology.orchestra.run.smithi083.stdout:kernel.core_pattern = core 2024-08-19T01:53:47.455 DEBUG:teuthology.orchestra.run.smithi083:> test -e /home/ubuntu/cephtest/archive/coredump 2024-08-19T01:53:47.488 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-08-19T01:53:47.489 DEBUG:teuthology.orchestra.run.smithi117:> test -e /home/ubuntu/cephtest/archive/coredump 2024-08-19T01:53:47.504 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-08-19T01:53:47.504 DEBUG:teuthology.orchestra.run.smithi167:> test -e /home/ubuntu/cephtest/archive/coredump 2024-08-19T01:53:47.519 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-08-19T01:53:47.520 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-08-19T01:53:47.531 INFO:teuthology.task.internal:Transferring archived files... 2024-08-19T01:53:47.533 DEBUG:teuthology.misc:Transferring archived files from smithi083:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/teuthology-2024-08-18_21:24:02-fs-squid-distro-default-smithi/7860117/remote/smithi083 2024-08-19T01:53:47.533 DEBUG:teuthology.orchestra.run.smithi083:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-08-19T01:53:47.609 DEBUG:teuthology.misc:Transferring archived files from smithi117:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/teuthology-2024-08-18_21:24:02-fs-squid-distro-default-smithi/7860117/remote/smithi117 2024-08-19T01:53:47.609 DEBUG:teuthology.orchestra.run.smithi117:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-08-19T01:53:47.643 DEBUG:teuthology.misc:Transferring archived files from smithi167:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/teuthology-2024-08-18_21:24:02-fs-squid-distro-default-smithi/7860117/remote/smithi167 2024-08-19T01:53:47.644 DEBUG:teuthology.orchestra.run.smithi167:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-08-19T01:53:47.679 INFO:teuthology.task.internal:Removing archive directory... 2024-08-19T01:53:47.679 DEBUG:teuthology.orchestra.run.smithi083:> rm -rf -- /home/ubuntu/cephtest/archive 2024-08-19T01:53:47.683 DEBUG:teuthology.orchestra.run.smithi117:> rm -rf -- /home/ubuntu/cephtest/archive 2024-08-19T01:53:47.685 DEBUG:teuthology.orchestra.run.smithi167:> rm -rf -- /home/ubuntu/cephtest/archive 2024-08-19T01:53:47.728 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-08-19T01:53:47.742 INFO:teuthology.task.internal:Not uploading archives. 2024-08-19T01:53:47.742 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-08-19T01:53:47.753 INFO:teuthology.task.internal:Tidying up after the test... 2024-08-19T01:53:47.753 DEBUG:teuthology.orchestra.run.smithi083:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-08-19T01:53:47.759 DEBUG:teuthology.orchestra.run.smithi117:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-08-19T01:53:47.761 DEBUG:teuthology.orchestra.run.smithi167:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-08-19T01:53:47.774 INFO:teuthology.orchestra.run.smithi117.stdout: 262166 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Aug 19 01:53 /home/ubuntu/cephtest 2024-08-19T01:53:47.784 INFO:teuthology.orchestra.run.smithi167.stdout: 262166 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Aug 19 01:53 /home/ubuntu/cephtest 2024-08-19T01:53:47.789 INFO:teuthology.orchestra.run.smithi083.stdout: 393586 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Aug 19 01:53 /home/ubuntu/cephtest 2024-08-19T01:53:47.792 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-08-19T01:53:47.804 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-08-19T01:53:47.832 INFO:teuthology.run:Summary data: client.0-kernel-sha1: 7503b862ff5bc4e556b3daee1c7881aa3e90ed62 description: fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate 3-modules} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/kclient/{base/{mount-syntax/{v2} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} ms_mode/legacy wsync/no} objectstore-ec/bluestore-bitmap omap_limit/10 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/random export-check n/3 replication/default} standby-replay tasks/{0-subvolume/{with-quota} 1-check-counter 2-scrub/yes 3-snaps/no 4-flush/yes 5-quiesce/no 6-workunit/suites/blogbench}} duration: 2009.9380505084991 failure_reason: '"2024-08-19T01:37:36.960897+0000 mds.b (mds.0) 19 : cluster [WRN] Scrub error on inode 0x10000001d0f (/volumes/qa/sv_1/05b773d9-8346-4397-99ec-d49959a55703/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34) see mds.b log and `damage ls` output for details" in cluster log' flavor: default owner: scheduled_teuthology@teuthology success: false 2024-08-19T01:53:47.832 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-08-19T01:53:47.924 INFO:teuthology.run:FAIL